# Investigating the Role of Centering Theory in Coreference Resolution

###### Abstract

Centering theory [grosz-etal-1995-centering, CT] provides a linguistic
analysis of the structure of discourse. According to the theory, local
coherence of discourse arises from the manner and extent to which successive
utterances make reference to the same entities. In this paper, we investigate
the connection between centering theory and modern coreference resolution
systems. We provide an operationalization of centering and systematically
investigate if the state-of-the-art coreference resolution models adhere to
the principles of centering theory by defining various centering metrics and
developing a search-based methodology. Our information-theoretic analysis
further reveals a positive dependence between coreference and centering; but
also shows that SOTA coreference models may not benefit much from explicitly
modelling centering ideas. Our analysis further shows that contextualized
embeddings contain much of the coherence information, which helps explain why
CT can only provide little gains to modern neural coreference resolution
systems based on pretrained representations. 111We will release code after the
paper is accepted.

##  1 Introduction

  * •

What is centering theory?

  * •

Centering theory was a popular backbone for ruled-based and feature-based
conreference resolution; people swithced to a pure neural network–based
approach

  * •

A natural question is whether centering theory is still useful for neural
coreference resolution? Do neural coreference resolvers work in a similar way
as centering theory?

  * •

Summary of the empirical findings

Centering theory (CT) [grosz-etal-1995-centering] has served as the
theoretical foundation for a lot of work in discourse analysis. Centering
describes the notion of local coherence in natural language text through
modeling the relationship between discourse entities and their references.

Since coreference resolution is the task of linking various referring
expressions of a given entity in the text, algorithms based on CT have been
often cited and used in coreference resolution literature [brennan-
etal-1987-centering, sidner1979towards, iida2003incorporating,
beaver2004optimization, kong-etal-2009-employing]. Some early works claimed
that the constraints proposed by the theory can be useful to guide coreference
resolution [elango2005coreference, deemter2000coreferring]. However, as neural
networks have regained popularity, people switched to pure unconstrained end-
to-end neural approaches [lee-etal-2017-end, joshi-etal-2019-bert] of building
modern coreference resolution systems.

A natural question is whether CT is still useful for neural coreference
resolution and whether neural coreference resolvers work in a similar way as
CT. We answer these questions by carefully analyzing neural coreference models
based on discourse information and conducting appropriate statistical tests.
Since CT is a linguistic theory rather than a computational one, this first
requires operationalizing centering theory, that is, building a computational
CT framework. This requires defining the various undefined parameters in the
original centering theory [grosz-etal-1995-centering]. Then, we adopt a
search-based procedure [karamanis2004evaluating] on the outputs of these
coreference resolution models and randomly shuffle the orderings of utterances
in a specific discourse to compute various center-based metrics described in
previous works [poesio-etal-2004-centering]. The assumption behind these
metrics is that the best-scoring ordering among the candidate orderings should
be the original ordering. Then, we conduct experiments establishing a positive
correlation between the centering metrics and coreference model performance.
We further explore the space of parameter configurations of CT using modern
natural language processing techniques, measuring the impact of different ways
of setting the parameters, choice of different pretrained embeddings and
verifying the robustness of our analytical results.

By a series of systematic analyses, we show that 1) state-of-the-art
coreference resolution models achieve relatively high scores assigned by
centering metrics, indicating they do contain some information about discourse
coherence, even though they are not trained by any CT signals; 2) there is a
positive dependence between CoNLL F1 and centering metrics, verifying the
psycholinguistic claim about coreference and CT [chambers1998structural,
gordon1998representation]; 3) the scores of centering metrics converge as
CoNLL F1 increase: the dependence between them decreases when CoNLL F1 score
reaches above 55%, however, this interval is where all modern coreference
resolution models lie. This, to some extent, indicates that the current
improvements in coreference resolution performance no longer improve entity-
coherence. We also show that contextualized embeddings contain much coherence
information by information-theoretic analysis, which helps explain why little
gains can CT provide to SOTA coreference models.

The article is organized as follows. Section 3 describes a joint computational
framework of CT and coreference. We then operationalize CT in Section 4.
Section 5 details the analysis methodology. In Section 6, 7 and 8, we present
our experimental results and analysis, which are discussed in Section 9.

##  2 Old Introduction

author=ryan,color=violet!40,size=,fancyline,caption=,author=ryan,color=violet!40,size=,fancyline,caption=,todo:
author=ryan,color=violet!40,size=,fancyline,caption=,We need a better
inroductory papergraph. The purpose of this first paragraph is two take two
disparate lines of research and convince the reader that they should be
unified or, at least, discussed in the same context. Instead, it provides a
super short overview of center theory. A topical sentence about the entire
paper would help.

Centering theory [grosz-etal-1995-centering] has served as the theoretical
foundation for a lot of work in discourse analysis. Centering describes the
notion of local coherence in natural language text through modeling the
relationship between discourse entities and their references.

Coreference resolution, on the other hand, is the process of linking various
referring expressions of a given entity in the
text.author=ryan,color=violet!40,size=,fancyline,caption=,author=ryan,color=violet!40,size=,fancyline,caption=,todo:
author=ryan,color=violet!40,size=,fancyline,caption=,This feels like an abrupt
change. I expected the text to discuss coreference NLP, but we jump to
psycholinguistics. It has been claimed in previous psycholinguistic literature
[chambers1998structural, gordon1998representation] that there is a connection
between coreference and centering theory.
author=ryan,color=violet!40,size=,fancyline,caption=,author=ryan,color=violet!40,size=,fancyline,caption=,todo:
author=ryan,color=violet!40,size=,fancyline,caption=,This is the first
connection the two, but this would have been confusing to me if I didn’t know
what the paper was about. This connection has also been explored in subsequent
NLP literature. Algorithms based on centering theory have been often cited and
used in coreference resolution literature [brennan-etal-1987-centering,
sidner1979towards, iida2003incorporating, beaver2004optimization, kong-
etal-2009-employing]. Some early works claimed that the constraints proposed
by the theory can be useful to guide coreference resolution
[elango2005coreference, deemter2000coreferring]. However, some more recent
works [yang2004improving] have also called this into question and have shown
that the use of these constraints fails to improve or even harms the
performance of coreference resolution systems.

author=ryan,color=violet!40,size=,fancyline,caption=,author=ryan,color=violet!40,size=,fancyline,caption=,todo:
author=ryan,color=violet!40,size=,fancyline,caption=,I think this paragraph
wants to say that we should use modern neural coreference systems to test
parts of centering theory!

As neural networks have regained popularity, end-to-end neural approaches
[lee-etal-2017-end, joshi-etal-2019-bert] have become the main building-block
of modern coreference resolution systems. However, the questions: (a) to what
extent do these approaches model discourse structure in text, (b) to what
extent do these models follow the principles of centering theory, and (c) can
these models further benefit from an improved modelling of discourse structure
remain unexplored.

The goal of this paper is to answer the above questions in the context of
modern end-to-end coreference resolution systems. We answer these questions by
carefully analyzing these coreference models based on discourse information
and conducting appropriate statistical tests. Since centering theory is a
linguistic theory and not a computational one, this first requires
operationalizing centering theory, that is, building a computational CT
framework. This requires defining the various undefined parameters in the
original centering theory [grosz-etal-1995-centering]. Then, we adopt a
search-based procedure [karamanis2004evaluating] on the outputs of these
coreference resolution models and randomly shuffle the orderings of utterances
in a specific discourse to compute various center-based metrics described in
previous works [poesio-etal-2004-centering]. The assumption behind these
mentrics is that the best-scoring ordering among the candidate orderings
should be the original ordering. Then, we conduct experiments establishing a
positive correlation between the centering metrics and coreference model
performance. We further explore the space of parameter configurations of CT
using modern natural language processing techniques, measuring the impact of
different ways of setting the parameters, choice of different pretrained
embeddings and verifying the robustness of our analytical results.

By a series of systematic analyses, we show that 1) state-of-the-art
coreference resolution models achieve relatively high scores assigned by
center-based metrics, indicating they do contain some information about
discourse-level entity coherence, even though they are not trained by any CT
signals; 2) there is a positive dependence between coref F1 and center-based
metrics, verifying the psycholinguistic claim about coreference and CT; 3) the
scores of center-based metrics converge as coref F1 increase: the dependence
between them decreases when CoNLL F1 score reaches above 55%, however, this
interval is where all modern coreference resolution models lie. This, to some
extent, indicates that the current improvements in coreference resolution
performance no longer improve entity-coherence. We also show that
contextualized embeddings contain much coherence information by information-
theoretic analysis, which helps explain why little gains can CT provide to
SOTA coreference models.

The article is organized as follows. Section 3 briefly describes some related
work. We then summarize the basic concepts of CT in Section 3. Section 4
proposes a general centering framework of coreference and coherence. Section 6
details the analysis methodology. In Section 7 and 8 we present our
experimental results and analysis, which are discussed in Section 9.

##  3 A Joint Computational Framework of Coreference and Coherence

In this section, we overview the necessary background on CT and describe a
joint computational framework of coreference and coherence.

At a high level, CT offers an explanation of local discourse structure that
models the interaction of referential continuity and the salience of discourse
entities in the internal organization of a text. It was one of the first
formal treatments of discourse, and remains one of the most influential. As
the name of the theory would suggest, CT revolves around the notion of
centering, which is the shifting of focus from one discourse entity to
another.

Formally, given a corpus of discourses $\mathcal{D}$ with size
$N=|\mathcal{D}|$, a set of candidate mention spans $\mathcal{M}_{i}$ is to be
considered for each discourse $i$. The $i$-th discourse
$\mathcal{D}_{i}=[U_{1},U_{2},\ldots,U_{|\mathcal{D}_{i}|}$], where $U_{j}$
denotes an utterance (the basic unit in centering theory, e.g. a sentence or a
finite clause). $M_{j}=\\{m_{1},\ldots,m_{p}\\}$ denotes a set of mention
spans which appears in $U_{j}$ ($M_{j}\in\mathcal{M}_{i}$), where
$m_{j}=\langle s_{j},e_{j}\rangle$ is a span with a start position $s_{j}$ and
an end position $e_{j}$. Each discourse $\mathcal{D}_{i}$ comes with a set of
entities $\mathcal{E}_{i}=\\{e_{1},e_{2},\ldots,e_{|\mathcal{E}_{i}|}\\}$.

##### The mention-entity mapping function
$f:\mathbb{N}\times\mathbb{N}\mapsto\mathbb{N}$

maps mention $m$ to entity $e$. The range of $f$ includes an extra value
$\epsilon$ representing “not a entity”.

##### The list of forward-looking centers

$\mathbf{C_{f}}(U_{j})=[e_{j_{1}},e_{j_{2}},\ldots,e_{j_{|\mathbf{C}_{f}(U_{j})|}}]$
is defined as an ordered set of entities corresponding to utterance $U_{j}$,
where the set of possible assignments for each $e\in\mathbf{C_{f}}(U_{j})$ is
$f(M_{j})$. This corresponds to Sidner’s notion of “potential discourse foci”
[sidner1979towards]. Intuitively, $C_{f}$ is a list of important entities in
the current utterance and it gets updated after every utterance.
$C_{f}(U_{j})$s depend only on the expressions that constitute that utterance;
they are not constrained by features of any previous utterance in the segment.

##### The preferred center

$C_{p}(U_{j})$ is the highest-ranked element of the CF list, i.g.
$\left[\mathbf{C_{f}}(U_{j})\right]_{1}$. Intuitively, this denotes the most
prominent discourse foci in $U_{j}$.

##### The backward-looking center

$C_{b}(U_{j})$ is the most highly ranked element of $\mathbf{C}_{f}(U_{j-1})$
that is realized in $U_{j}$. The definition of realize is that $U$ is an
utterance of some phrase for which center $c$ is the semantic interpretation,
e.g. “the current US president” directly realizes “Donald Trump”.222For the
formal definition of realization, please refer to Appendix A. The connection
between the backward-looking center and the preferred center forms the
theory’s key claims about coherence.

CT gives a formal account of how discourse involves continuous updates to a
local attentional state, which we describe below.

Let $\mathcal{C}_{h}$ denotes the random variable of coherence. We describe an
approach to relate the coherence $\mathcal{C}_{h}$ of a discourse with its
coreference information $\mathcal{C}_{r}$:

$\mathbb{P}(\mathcal{C}_{h},\mathcal{C}_{r}\mid\mathcal{D})=\mathbb{P}(\mathcal{E}\mid\mathcal{D})\,\mathbb{P}(\mathcal{C}_{h}\mid\mathcal{E},\mathcal{D})\,\mathbb{P}(\mathcal{C}_{r}\mid\mathcal{E}),$

where $\mathbb{P}(\mathcal{E}\mid\mathcal{D})$ denotes the process of mention
detection, mention pruning and entity linking. The possibility modeling of
coherence can be further decomposed as

$\displaystyle\mathbb{P}(\mathcal{M}\mid\mathcal{D})\,\mathbb{P}(\mathcal{E}\mid\mathcal{M},\mathcal{D})\,\mathbb{P}(\mathcal{C}_{r}\mid\mathcal{E},\mathcal{M})\,\mathbb{P}(\mathcal{C}_{h}\mid\mathcal{C}_{f})$

where $\mathcal{M}$, $\mathcal{E}$ and $\mathcal{C}_{r}$ are fields of random
variables representing mentions, the clustering of coreferring mentions and
the forward-looking centers.

###  3.1 Centering Theory: A Review

In this section, we overview the necessary background on centering theory. At
a high level, centering theory offers an explanation of local discourse
structure that models the interaction of referential continuity and the
salience of discourse entities in the internal organization of a text. It was
one of the first formal treatments of discourse, and remains one of the most
influential. As the name of the theory would suggest, centering theory
revolves around the notion of centering, which is the shifting of focus from
one discourse entity to another. The centering theory gives a formal account
of how discourse involves continuous updates to a local attentional state,
which we describe below.

###  3.2 Formal Framework

Let $E$ be a set of entities. This is a discrete (but perhaps unbounded) set
author=ryan,color=violet!40,size=,fancyline,caption=,author=ryan,color=violet!40,size=,fancyline,caption=,todo:
author=ryan,color=violet!40,size=,fancyline,caption=,want a better definition
of an entity

A discourse segment (DS) is a sequence of utterances $U_{1},U_{2}\ldots$. A
center of an utterance $U_{n}$ Each utterance $U_{n}$ in the discourse segment
is assigned:

  * •

A list of forward-looking centers $C_{f}(U_{n})$: This corresponds to
correspond to Sidner’s notion of “potential discourse foci”. Intuitively,
$C_{f}$ is a list of important entities in the current utterance and it gets
updated after every utterance. $C_{f}(U_{n})$s depend only on the expressions
that constitute that utterance; they are not constrained by features of any
previous utterance in the segment.

  * •

A single preferred center $C_{p}(U_{n})$: The highest-ranked element of the
$C_{f}(U_{n})$ list. Intuitively, this denotes the most prominent discourse
foci in $U_{n}$.

  * •

A single backward-looking center $C_{b}(U_{n})$: One of the $C_{f}$’s is also
characterized as a backward looking center. The most highly ranked element of
$C_{f}(U_{n-1})$ that is realized in $U_{n}$ is the $C_{b}(U_{n})$. The
connection between the backward-looking center and the preferred center forms
the theory’s key claims about coherence.

###  3.3 Four Types of Transition Relations

Table 1 presents the four types of Transition Relations in Centering Theory.

|  | $C_{b}(U_{n})=C_{b}(U_{n-1})$
---
or $C_{b}(U_{n-1})$ undefined
$C_{b}(U_{n})\neq C_{b}(U_{n-1})$
$C_{b}(U_{n})=C_{p}(U_{n})$ | Continue | Smooth Shift
$C_{b}(U_{n})\neq C_{p}(U_{n})$ | Retain | Rough Shift

Table 1: Four Types of Transitions in Centering Theory

The transition relations are used to hypothesize that discourses are easier to
process when successive utterances are perceived as being “about” a unique
discourse entity. These are described in terms of the claims below:

###  3.4 Claims of Centering Theory

####  3.4.1 Constrains

  * •

C1. There is precisely one $C_{b}$.

  * •

C2. Every element of $C_{f}(U_{n})$ must be realized333For example, “the
current US president” directly realizes “Donald Trump”. For the fromal
definition of realization, please refer to Appendix A. in $U_{n}$.

  * •

C3. $C_{b}(U_{n})$ is the most highly ranked element of $C_{f}(U_{n-1})$ that
is realized in $U_{n}$.

####  3.4.2 Rules

  * •

R1. Some element of $C_{f}(U_{n-1})$ is realized as a pronoun in $U_{n}$, then
so is $C_{b}(U_{n})$.

  * •

R2. Transitions have the descending preference order of “Continue $>$ Retain
$>$ Smooth Shift $>$ Rough Shift” (This rule captures the essence of the
theory aiming to minimize the number of focus shifts). The definition of these
transitions is provided in Table 1.

###  3.5 Factors governing the ranking of $C_{f}$ list

One of the most important parameters of centering is the ranking function.
Previous works assume that several factors play a role in determining the
ranking. These include:

  * •

$C_{b}$ is preferentially realized by a pronoun.

  * •

Grammatical role [grosz-etal-1995-centering]: Subjects are ranked more highly
than objects and other roles – Subject $>$ Object $>$ Others.

  * •

Semantic roles [kong-etal-2009-employing]: ARG0/agent $>$ ARG1/patient $>$
Others.

###  3.6 The connection between Centering and Coreference

CT, which is an approach to modeling certain aspects of local coherence in
discourse, includes within it the component that models coreference
resolution. CT utilizes this preference for the “center of attention” to
continue over successive utterances. The relative preference of the “actor
focus” as the next center of attention is also captured with the “preferred
center” in CT. The entities in the focus space are possible antecedents. Many
linguistics-based approaches for coreference resolution leverage differences
in coherence result from changes in the center of attention, captured by the
Centering transitions and transition ordering, and from the different
expressions in which centers are realized. However, CT itself is not a theory
of coreference resolution.

author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=, See Section 3 in
https://drive.google.com/file/d/1XV80k-uOWuSHqkS9_CH9h8xtTPcZyzkZ/view?usp=sharing.

As discussed above, what coreference resolution models do is, essentially, the
implementation of the mention detection & pruning and the mention-entity
mapping function $f$, which can be modeled as
$\mathbb{P}(\mathcal{M}\mid\mathcal{D})$ and
$\mathbb{P}(\mathcal{E}\mid\mathcal{M},\mathcal{D})$. CONLL F1 i.e. the
average F1 score of MUC [vilain-etal-1995-model], $B^{3}$
[bagga1998algorithms] and $\mathrm{CEAF}_{\phi_{4}}$ [luo-2005-coreference] is
often used as the main evaluation metric of coreference resolution. We treat
CONLL F1 $\in[0,1]$ as the random variable representing coreference quality,
i.e. $\mathbb{P}(\mathcal{C}_{r}\mid\mathcal{E})$. We use various state-of-
the-art neural coreference resolution models: e2e [lee-etal-2017-end], c2f
[lee-etal-2018-higher], the incremental model [xia-etal-2020-incremental] to
model $\mathbb{P}(\mathcal{C}_{r}\mid\mathcal{E})$. We describe these models
in detail in Appendix B. We talk about the modeling of
$\mathbb{P}(\mathcal{C}_{h}\mid\mathcal{E})$ in the next section.

##  4 Operationalizing Centering Theory

Centering theory is a high-level theory in that it discusses the focality
using high-level semantic notions, e.g. the salience of an entity in an
utterance. However, to make very specific claims about centering theory using
data, we have to make these high-level notions precise. For instance, we have
to precisely define what we mean by utterance and there are multiple good
choices, e.g. a clause, sentence or a paragraph? We call choices the
parameters of the operationalization and we describe and justify our choices
below.

##### Utterance.

textttutterance: can be defined as “sentences”, “coordinated clauses” or other
linguistic units.

##### Realization.

can be “direct realization” or “indirect realization”.

##### Previous Utterance.

can be “true” or “false”. This dictates if we should ignore so-called null
utterances i.e. utterances that contains no mention spans.

##### CF Ranking.

The ranking function that ranks $C_{f}$ based on “grammatical role labels” or
“semantic role labels” (see Section 3.5). In our experiments, we set the
specific rules as “Pronoun(Subject) $>$ Pronoun(Object) $>$ Subject $>$ Object
$>$ others” for “grammatical role labels”. For “semantic role labels”, we set
it as “Pronoun(Agent) $>$ Pronoun(Patient) $>$ Agent $>$ Patient $>$ Others”.

##### CF Candidate.

Can be “cluster only” or “include singleton”. This parameter has never been
considered in previous centering literature since an entity list was always
annotated in the dataset considered by previous CT work. But in our context,
this parameter can be set appropriately to allow us to consider singletons or
not.

##### F Function.

Another parameter is how we implement the mention-entity mapping function $f$.
This can be set to either a coreference resolution model or simply manual
annotation.

An instantiation is a particular way to set these parameters. By specifying a
particular instantiation of CT, we compute the $C_{f}$ list, $C_{b}$ and
$C_{p}$ in a deterministic manner.

##  5 Metrics of Centering Theory

Give a sequence of $C_{f}$-lists, the transition information can be used to
determine the extent to which a text conforms with, or violates, the
principles of centering theory. Let $T=U-1$ be the number of transitions a
discourse contains. Previous work [poesio-etal-2004-centering,
karamanis2004evaluating] advocates the following CT-related statistics:

1) M.NOCBS $=U_{\mathrm{nocb}}/T$, where $U_{\mathrm{nocb}}$ is the number of
utterances with NOCB. NOCBs occur when $C_{b}(U_{n})$ is undefined, which
implies a very sharp shift of focus since no element in $C_{f}(U_{n-1})$ is
realized in $U_{n}$.

2) M.CHEAP $=U_{\mathrm{cheap}}/T$, where $U_{\mathrm{cheap}}=|\\{U_{n}\mid
C_{b}(U_{n})\neq C_{p}(U_{n-1})\\}|$. It indicates the ratios of “expensive”
transitions [strube-hahn-1999-functional].

3) M.COHERENCE $=U_{\mathrm{coh}}/T$, where $U_{\mathrm{coh}}=|\\{U_{n}\mid
C_{b}(U_{n})\neq C_{b}(U_{n-1})\\}|$. Intuitively, a coherent discourse should
keep its focus CB as constant as possible.

4) M.SALIENCE $=U_{\mathrm{sal}}/T$, where $U_{\mathrm{sal}}=|\\{U_{n}\mid
C_{b}(U_{n})\neq C_{p}(U_{n})\\}|$. It indicates whether the maintained focus
CB is the most salient entity ($C_{p}$) in the current utterance.

5) M.KP is the average of M.NOCB, M.CHEAPNESS, M.COHERENCE, and M.SALIENCE.

6) M.TRAN is a ranking metric, which employs the preferences of transitions as
expressed by Rule 2, i.g. given a set of discourses of the same length, we
sort them by the number of Continue transitions first, then use #Retain,
#SmoothShift, #RoughShift as tie breakers sequentially.

author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,I dont understand the
last one. Can we explain better or just cite? I think it would be good to add
a 2 line intuitive difference between these metrics here to give the readers a
better understand of these beyond the formulas.

###  5.1 Assessing CT-based Coherence

The aforementioned naive statistics cannot be directly used as a measurement
of coherence quality, as properties of discourse that are not related to
coherence also affect these statistics. For example, discourse that contains
more entities and mentions tends to have a higher rate of violations. In order
to alleviate this issue, we adopt a search-based methodology to quantify CT-
based coherence $\mathcal{C}_{h}$, which is first brought up in
[karamanis2004evaluating] for text structuring.

The key assumption behind $\mathcal{C}_{h}$ is that among all the possible
permutations of a given discourse, the original permutation should be the most
coherent one. We randomly sample a set of permutations from the space of
possible permutations of the discourse $\mathcal{D}_{i}$ for discourses longer
than 5 utterances and take the entire permutations space for short discourses.
We then rank all candidate orderings of utterances by a given centering-based
metric $M$ mentioned above. The explored search space is divided into sets of
orderings that score better, equal, or worse than the original $D_{i}$
according to Metric $M$. The third step is to calculate the coherence score:

$C_{h}(M,\mathcal{E}_{i},\mathcal{D}_{i})=\text{Worse}(M,\mathcal{E})+\text{Equal}(M,\mathcal{E})/2.$

A higher $C_{h}(M,\mathcal{E}_{i},\mathcal{D}_{i})$ is indicative of a set of
entity clusters $\mathcal{E}_{i}$ with better CT-based coherence quality.
Finally, the CT-based coherence quality $\mathbf{C_{h}}$ on the entire corpus
$\mathcal{D}$ containing $m$ discourses is summarized as the average coherence
score:

$\mathbf{C_{h}}(M,\mathcal{E},\mathcal{D})=\frac{1}{m}\sum_{\mathcal{D}_{i}\in\mathcal{D}}C_{h}(M,\mathcal{D}_{i}).$

##  6 Experimental Setup

author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,Since we have multiple
genres of text, can we add a small analysis in our experiments on how the
correlation of CT and CF varies across genres of text or other parameter
choices made in this
paper?author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=,we put this part in the
Appendix
:)author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,Lets describe in at the
end close to our qualitative analysis. This would be an interesting thing to
also discuss in the main paper even if the detailed results are in the
supplementary.author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,Also, I think the
qualitative analysis can be further expanded.

###  6.1 Datasets

We use the OntoNotes 5.0 (CoNLL-2012 shared task) dataset
[weischedel2013ontonotes, pradhan2013towards] for our experiments. This is the
most widely used benchmark dataset in coreference resolution. The dataset
contains documents spanning several genres, including those with multiple
speakers (broadcast and telephone conversations) and those without (broadcast
news, newswire, magazines, weblogs, and the Bible). It also contains rich
syntactic and semantic annotation of part-of-speech, syntax and semantic role
information which is also useful in our analysis. The statistics of the
OntoNotes 5.0 dataset are shown in Table 2. There are 2802, 343, and 348
documents in the training, development, and test set respectively. It worth
noting that all previous work on CT has been based on much smaller datasets
with significantly lesser number of documents and shorter documents (e.g. 75
times of GNOME [poesio-etal-2004-centering]).
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,What is 75 times? length?
num docs?

| #Sent | #Token
---|---|---
| mean | min | max | mean. | min | max
Train | 26.83 | 1 | 188 | 463.71 | 24 | 4009
Dev | 27.99 | 2 | 127 | 475.52 | 33 | 2314
Test | 27.24 | 2 | 140 | 487 | 34 | 1798
Table 2: The statistics of the number of sentences and the number of tokens
per document of OntoNotes.

###  6.2 Parameters Settings

For CT parameters, We define utterances as “sentences”. realization is set to
“direct realization” and previous_utterance is set to “true”, following the
very mainstream views. For CF_ranking and CF_candidate, we explore all the
combinations of parameter settings (two possible values each, as described in
Section 5.1). We explore several coreference resolution models with various
embedding approaches: the e2e-SpanBERT, c2f-SpanBERT (base and large) [joshi-
etal-2020-spanbert], c2f-GloVe, c2f-one-hot [lee-etal-2017-end], corefQA [wu-
etal-2020-corefqa] and Incremental coreference [xia-etal-2020-incremental]. We
list the coreference model parameters and training details in Appendix C.
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,I think the readers will
find it a bit odd that all these models have been considered together and
separate abalysis of various models is not provided. We should address that.

##  7 Experimental Results

###  7.1 Evaluating Operationalizations of CT.

author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=,Shows: Our
operationalizations are valid (also some statistics which is complementary to
previous work).
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,IMP: What do we conclude
from table 3&4?
author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=,Table 3 is just some
intuitive numbers showing how frequent these different types of CT violations
exists in the test dataset (under two specific CT instantiations. This tabel
gives us an intuition of how large and how dispersed the violation frequencies
can be. Table 4 shows how many artificial orderings are worse than the
original ordering based on 6 CT violation
metrics.author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,This needs to be better
explained then in the text

The goal of this corpus analysis is to establish an upper bound for how
successful different CT operationalizations and instantiations are in
evaluating the coherence of a certain discourse. Here we evaluate all the 6
operationalizations of 4 instantiations.

|  $\%$ valid_u | $\%\overline{nocb}$ |  $\%$ Cheap |  $\%$ Coh. |  $\%$ Sal.
---|---|---|---|---|---
Gram. | 80.02 (19.26) | 60.14 (17.48) | 42.78 (17.57) | 46.51 (15.25) | 42.85 (16.05)
Seman. | 79.48 (21.20) | 59.31 (19.09) | 43.15 (18.41) | 45.04 (16.06) | 41.90 (16.62)
| %Cont. | %Ret. | %SShift | %RShift | %KP
Gram. | 39.17 (23..68) | 14.15 (12.46) | 9.99 (10.22) | 36.12 (20.02) | 48.07 (14.80)
Seman. | 39.79 (25.86) | 13.78 (13.24) | 10.22 (11.86) | 35.27 (22.09) | 47.35 (15.32)
Table 3: The statistics of CT-based metrics on OntoNotes 5.0 test set. Gram.
and Seman. represent grammatical-role-based and semantic-role-based CF ranking
respectively. %valid_u and %$\overline{nocb}$ represent the percentage of
valid utterances (those with non-empty CF-candidates) and utterances with
valid CB. %Cheap, %Coh, %Sal, %Cont., %Ret., %SShift, and %RShift represent
the percentage of utterances marked with cheapness, salience, coherence,
continue, retain, soft shift, rough shift respectively. %KP represent the sum
of cheapness, salience and coherence.555The complete statistics is listed in
Appendix D.

The statistics of center-based metrics on OntoNotes 5.0 test set is presented
in Table 5. For grammatical-role-based CF ranking, the entire OntoNotes test
set is used, while the experiment with semantic-role-based CF ranking is
conducted on a subset of OntoNotes test set, where semantic role annotations
are available. The subset contains 61.2% documents. The CF_candidate is set to
the ground truth “coref spans” with entity id annotation available. Table 5
shows that about 80% of utterances are valid, and the violations of different
CT principles are about 50% under this two installations.

Table 4 compares different center-based metrics by the permutation-based
methodology stated above, adopting the ground truth of coreferred clusters
provided by annotation. Most of the center-based metrics are above 70% with
both grammatical-role-based and semantic-role-based instantiations. This
indicates that these center-based metrics do provide a certain degree of
coherence information to distinguish the correct ordering from its random
sampled counterparts. This also lends support to the centering theory’s claims
about local coherence, in particular, the constrain C1 (CB existence). We also
report that more than 80% transitions do not violate the CT principles.

However, we find no significant difference between different CT
operationalizations. One interesting finding is that, though previous
researchers claim that the measuring of CB existence is the essential part of
centering theory, since the majority of the recorded transitions in the
configuration of CT are NOCBs [karamanis2004evaluating] (about 40% in our
study), our experimental results show that the other CT-based metrics based on
salience and transitions are as informative as M.NOCB.

###  7.2 Centering Performance of Coreference Systems

We compare different center-based metrics based on the CF-candidates provided
by two coreference models, a strong one (well-trained c2f-SpanBERT-large, with
a coref F1 of 78.87%) and a weak one (underfitted c2f-SpanBERT-base, with a
coref F1 of 17.51%), under four instantiations of CT. The results are shown in
Table 4.

| gold | c2f-SpanBERT-large | c2f-SpanBERT-base
---|---|---|---
| Gram. | Seman. | Gram. | Seman. | Gram. | Seman.
Metric |  |  | CO | IS | CO | IS | CO | IS | CO | IS
M.NOCB | 82.74 | 78.94 | 82.72 | 80.58 | 77.88 | 76.15 | 46.54 | 44.65 | 42.15 | 40.99
M.CHEAP | 79.05 | 76.86 | 77.71 | 74.51 | 75.20 | 72.17 | 45.55 | 42.10 | 43.23 | 39.46
M.COHERENCE | 76.68 | 71.52 | 75.98 | 75.73 | 70.16 | 70.09 | 45.29 | 45.12 | 40.38 | 40.11
M.SALIENCE | 83.76 | 79.66 | 82.61 | 83.36 | 77.44 | 78.63 | 45.95 | 46.89 | 41.02 | 42.30
M.KP | 83.61 | 79.59 | 82.61 | 81.68 | 78.63 | 78.01 | 46.36 | 45.43 | 42.24 | 41.47
M.TRAN | 79.26 | 74.90 | 78.65 | 79.35 | 73.72 | 74.19 | 61.11 | 62.34 | 52.74 | 54.16
Table 4: The comparison of different center-based metrics by a permutation-
based methodology based on the CF-candidates provided by the gold annotations
and coreference models. Gram. and Seman. represent the grammatical-role-based
and semantic-role-based CF-ranking respectively. CO and IS represent Cf-
candidates being “cluster only” and “include singletons”.

We observe that no CT scores based on the coreference system’s output exceed
the upper bound provided by the ground truth annotation under any combinations
of parameter settings we experimented on, indicating the ground truth
annotation of reference links adhere to the constraints of centering theory
better than those produced by the coreference system.

On the other hand, the c2f-SpanBERT-large model achieves relatively higher
centering scores under all settings, indicating it does contain a significant
information about entity coherence defined by CT, even though it is not
trained with any CT-related supervision. Moreover, the difference in CT-based
metrics between the stronger models and the weaker one is significant, which
provides a basis for taking CT constraints as an optimization goal in order to
improve coreference performance.

We also found that, including singletons (from mention spans) lowers the
M.NOCB and M.CHEAP by 1-2%, and improves M.TRAN by 1-2% in all parameter
settings. 666The same trend can be observed in our experiments on other
coreference systems. We believe it is a universal phenomenon.

((a)) All Models

((b)) SOTAs

Figure 1: The CT-CONLL F1 curves.

###  7.3 Exploring the Relationship between Coreference and Coherence

To further systematically investigate the shape of the relationship between CT
and coreference, we scale up our experiment. Let $X,Y,Z$ be three random
variables taking values in $\mathcal{X}\times\mathcal{Y}\times\mathcal{Z}$,
where $\mathcal{X}$ represents the space of entity clusters, and
$\mathcal{Y}=(0,1)$ is the normalized rates of CT violations, and
$\mathcal{Z}=(0,1)$ is the coref F1. Our task is to explore the relationship
between the coherence quality $Y$ (measured by the violations of CT rules) and
the coreference quality $\mathcal{Z}Z$ (measured by coref F1).

We compute the center-based metrics for all 8,600 checkpoints of the
coreference models we implemented 777See the detailed list of checkpoints in
Appendix C.4., with the CT instantiation of “grammatical-role-based” CF-
ranking and “cluster only” CF-candidates. We plot the center-based metrics (as
$Y$) vs. coref F1 (as $Z$) of those models in Figure 1, and also compute the
correlations and mutual information in Table 5. There is a monotonic
relationship between the center-based metrics and coref F1. All center-based
metrics have high positive correlations and high mutual information with
coreference F1, with significance tested by t test. These results confirm the
psycholinguistic claim that there is a strong relationship between coreference
and CT.

Figure 1(a) also indicates the shape of this relationship: the curve becomes
more and more flat as the F1 score increases, and CT scores stop growing when
F1 reaches above a certain level (about 60%). This interval is where all
modern coreference resolution models (c2f-large-SpanBERT,
pretrained-c2f-large-SpanBERT, corefQA and Incremental) lie (shown in Figure
1(b)). As shown in Table 5, the correlation and mutual information888We adopt
the default bin estimation in infotheo package for mutual information
estimation. of these two r.v. are significantly lower for high-quality SOTA
coreference models.

This, to some extent, indicates that the usefulness of CT for improving
coreference systems is not a universal thing: When the model prediction $X$
has a low coreference F1 (low-quality coref models), CT is more useful; while
the model prediction $X$ has a high coreference F1, CT provides little
information gain. The most direct conclusion from Figure 1 is that minimizing
the number of violations of CT constraints should not be the only objective
when designing coreference resolution systems. Coreference models optimize
with respect to center-based metrics only to a certain extent.999We will
explore the mismatch between the two later in Section 10.

| All | SOTAs
---|---|---
Metric | Cor. | MI | Cor. | MI
M.NOCB |  0.882${\dagger}$ | 0.456 |  0.521${\dagger}$ | 1.06e-3
M.CHEAP |  0.858${\dagger}$ | 0.384 |  -0.191${\dagger}$ | 3.52e-2
M.COHERENCE |  0.864${\dagger}$ | 0.346 | -0.026 | 3.52e-2
M.SALIENCE |  0.897${\dagger}$ | 0.490 |  0.802${\dagger}$ | 6.73e-2
M.KP |  0.883${\dagger}$ | 0.451 |  0.459${\dagger}$ | 1.49e-3
M.TRAN |  0.912${\dagger}$ | 0.296 |  0.477${\dagger}$ | 1.20e-2
Table 5: The Pearson correlation coefficients and mutual information of coref
F1 with center-based metrics. p-value $<0.01$ is denoted as ${\dagger}$.

##  8 Analysis: Overlapping Information

Centering Theory is useful for feature-based or rule-based systems by nature,
because those systems cannot leverage any information of discourse-level
coherence unless we explicitly combine coherence information (e.g. CT rules)
into it. However, our experiments show that it is not always the case for
modern end-to-end neural coreference systems: Even though those models are
trained without any Centering Theory signals, they can still get relative high
CT scores. We suspect it is because some parts of the neural model may already
contain information of discourse-level coherence, which overlaps with the
information CT could provide.

Thus, we examine (1) the discourse-level coherence information (i.g. CT
information) is incorporated into which component of the end-to-end
coreference system, and (2) which component of the systems contain the most
amount of discourse-level coherence information.
author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=,(1) Qualitative analysis,
(2) Quantitative analysis Two possible sources of CT information are
pretrained embeddings and coreference annotations in the training dataset.

###  8.1 Pretrained Embedding.

Here we explore how the shape of this relationship varies as a function of
embedding types. We compare the c2f model with contextualized embeddings
(c2f-SpanBERT), with uncontextualized Glove embeddings (c2f-GloVe), and
without any pretrained embeddings where tokens are represented as their one-
hot vectors (c2f-one-hot). We report the mean score of 5 independent runs of
the model with different seeds.

We show both coref F1s and CT scores as a function of training epoch in Figure
11. We observe that the CT score increases as coref F1 increases for all the
models. The CT score increases more gradually for c2f-one-hot and c2f-Glove.
In contrast, c2f-SpanBERT has reached a fairly high level of CT score at the
very early training epoch this no longer improves. The CT score for c2f-one-
hot is initially higher than c2f-Glove. This may due to the fact that c2f-one-
hot may have learned all CT information from Ontonotes. The result suggests
that 1) both the GloVe and SpanBERT embedding contain some entity-coherence
information that overlaps with CT, 2) the contextualized SpanBERT embedding
may contains more CT information as compared to non-contextualized embeddings.

Figure 2: The CONLL F1 score (square, solid lines) and the CT score M.KP
(triangle, dotten lines) as a function of training epoch for c2f-SpanBERT,
c2f-GloVe and c2f-one-hot. Models are trained on 100% OntoNotes training
dataset. 111111We only show the relationship between %kp and coref F1 here,
the plots of other CT operationlinzations can be found in the Appendix.

We further compare the shape of the CT score as a function of coref F1 for
these three types of embeddings in Figure 3. The shape of c2f-SpanBERT has the
slowest slope, while the shape of c2f-GloVe is even steeper than c2f-one-hot,
indicating the CT information contained in SpanBERT embeddings does not
overlap a lot with coref annotations; SpanBERT thus can get further gain on CT
scores by training on the coref objective (unlike GloVe). This further
confirms that the conditional entropy $H(Y\mid X_{\mathrm{SpanBERT}})$ is
smaller than $H(Y\mid X_{\mathrm{GloVe}})$.
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,Better to make the
argument without bringing in conditional entropy since it has not been
introduced before

Figure 3: The CT score M.KP a function of CONLL F1 for c2f-SpanBERT, c2f-GloVe
and c2f-one-hot models. The x-ticks and y-ticks represent the distribution of
the samples used to measure CONLL F1 and M.KP respectively in our experiments.
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,Better to make the
argument without bringing in conditional entropy since Figure 4 is not
plotting conditional entropy. The argument can be directly made by looking at
the shape of the curves?

###  8.2 Amount of Coreference Training Data.

author=eleanor,color=yellow,size=,fancyline,caption=,author=eleanor,color=yellow,size=,fancyline,caption=,todo:
author=eleanor,color=yellow,size=,fancyline,caption=,Do different sizes of
coref training data influence the shape of this relationship a lot? If it’s
true, then it means the information contained in annotated coreference data
contains discourse-level entity-coherence information, i.g. MI(Centering Data
$\mid$ Coref Data) is large.

As shown in Figure 4, for GloVe and one-hot, there exists some critical point
in terms of coreference traning data where the conditional entropy $H(Y|X)$
becomes zero. In contrast, for SpanBERT, such a critical point does not exist;
CT scores are high even when trained on a very small amount of training data.
This indicates that annotated coreference data does contain CT information.
However, the CT information it covers overlaps with the information contained
in SpanBERT embedding to a large extent.

Figure 4: The CONLL F1 score (square) and the CT score M.KP (triangle) as a function of the amount of coreference training data for c2f-SpanBERT, c2f-GloVe and c2f-one-hot. X-axis represents the ratio of the Ontonotes dataset used in training. | SpanBERT | GloVe | One-hot
---|---|---|---
Metric | Cor. | MI | Cor. | MI | Cor. | MI
M.NOCB |  0.903${\dagger}$ | 0.341 |  0.939${\dagger}$ | 0.788 |  0.846${\dagger}$ | 0.504
M.CHEAP |  0.872${\dagger}$ | 0.120 |  0.929${\dagger}$ | 0.684 |  0.837${\dagger}$ | 0.481
M.COHERENCE |  0.871${\dagger}$ | 0.134 |  0.934${\dagger}$ | 0.650 |  0.837${\dagger}$ | 0.452
M.SALIENCE |  0.913${\dagger}$ | 0.382 |  0.942${\dagger}$ | 0.777 |  0.850${\dagger}$ | 0.492
M.KP |  0.903${\dagger}$ | 0.324 |  0.941${\dagger}$ | 0.796 |  0.858${\dagger}$ | 0.542
M.TRAN |  0.899${\dagger}$ | 0.211 |  0.964${\dagger}$ | 0.648 |  0.844${\dagger}$ | 0.469
Table 6: The Pearson correlation coefficients and mutual information of coref
F1 with center-based metrics. p-value $<0.01$ is denoted as ${\dagger}$.

((a)) SpanBERT

((b)) GloVe

((c)) One-hot

Figure 5: The mutual information between coref F1 and CT scores as a function
of the amount of coreference training data.

###  8.3 Which part contains more CT information?

To quantify the overlapping discourse-level entity-coherence information, we
report the mutual information and correlations under different conditions.

We report the correlations and MI for c2f-SpanBERT, c2f-GloVe and c2f-One-hot
models respectively in Table 6 and also plot the MIs of the checkpoints
trained on different sizes of dataset separately in Figure 5.121212Same as in
Section 7.3, we adopt the default bin estimation. We also tried other MI
estimation techniques (including KDE, KSG, Partitioning) and observed similar
trends. Notably, both the correlation and mutual information of c2f-SpanBERT
are smaller than those in c2f-GloVe (Table 6). On the other hand, as observed
Figure 5, no matter what embedding is adopted, there exist a consistent trend
that the more coreference training data, the smaller the mutual information.
We also notice that the variation of c2f-SpanBERT models (Figure 5(a)) is
significant higher since not much MI comes from the coreference training
signals but contextualized embeddings in the c2f-SpanBERT models. Another
observation is that the difference $\bigtriangleup$MI between different
training data sizes is more significant than the $\bigtriangleup$MI between
different embeddings.

This part of analysis confirms that (1) coreference and centering theory do
share some similar objectives, (2) contextualized embeddings contain more
discourse-level coherence information compared to uncontexturalized
embeddings, (3) while coreference training signal is the main source of CT
information, contextualized embeddings provide some CT information that
coreference training signal does not cover.

a) |  To express its determination , the Chinese securities
---|---
|  regulatory department compares this stock reform
|  to a die that has been cast.
b) |  It takes time to prove whether the stock reform can
| really meet expectations, and whether any deviati…
c) |  Dear viewers , the China News will end here.
d) |  Thank you everyone for watching .
e) |  Coming up is the Focus Today hosted by Shilin.
f) |  Good-bye , dear viewers.
Table 7: Qualitative Analysis. CBs are underlined. Singletons are marked in
gray.

##  9 Discussion

Previous researchers [poesio-etal-2004-centering] suggested that minimizing CT
violations should not be the overriding goal, even for coherence measuring.
Our research further confirms its limitations to optimizing coreference
resolution. These findings suggest, even though incorporating CT constraints
in coreference resolution models could be beneficial to some extent, CT-based
metrics (at least the instantiations we investigated in this paper) saturate
rapidly as the coreference resolution performance approaches the current
state-of-the-art level. In other words, the experimental results in this paper
suggests that we may not get significant gains over and above the strong fully
supervised systems for coref by using CT ideas.

We provide a qualitative analysis of a subtle case to highlight the mismatch
of CT and coreference objectives in Table 7. While linking “the Focus Today”
in utterance e to the entity cluster marked in blue (“dear viewer”) minimizes
the violations of CT, it is not the correct choice a coreference system should
make.
author=mrinmaya,color=green,size=,fancyline,caption=,author=mrinmaya,color=green,size=,fancyline,caption=,todo:
author=mrinmaya,color=green,size=,fancyline,caption=,I like the qualitative
analysis here. Can we do more of such analysis in the main paper? When do CT
and CF agree and when they do not.

##  10 Related Work

##### Centering Theory:

Centering Theory [grosz-etal-1983-providing, grosz-etal-1995-centering] is one
of the most well-known linguistic theories in discourse. CT presents the idea
that certain entities mentioned in an utterance are more central than others
and that this property imposes constraints on a speaker’s use of different
types of referring expressions. While many components of the theory were
underspecified in the original work [grosz-etal-1983-providing], subsequent
work [poesio-etal-2004-centering] investigated in a systematic fashion the
effect on the theory’s claims. [karamanis2004evaluating] compare metrics of
coherence based on CT with respect to their potential usefulness for text
structuring in natural language generation. From the centering theory view,
our work can be viewed as a continuation and extension of [poesio-
etal-2004-centering].

##### Connections between Centering Theory and Coreference:

Previous works have also explored the connection between CT and coreference
resolution. [sidner1979towards] investigated the process of focussing as a
description and explanation of the comprehension of certain anaphoric
expressions in English discourse. [brennan-etal-1987-centering] presented a
formalization of the centering approach and used it as the basis for an
algorithm to track discourse context and bind pronouns, which is also called
the BFP algorithm. Then a critical evaluation was proposed by
[kehler1997current]. Roughly the same time, [walker1998centering] argued that
the restriction of centering to operating within a discourse segment should be
abandoned in order to integrate centering with a model of global discourse
structure. [tetreault-2001-corpus] then compared pronoun resolution algorithms
and introduced a centering algorithm (LeftRight Centering). After that,
[beaver2004optimization] reformulated the Centering model of anaphora
resolution and discourse coherence. Later, [kong-etal-2009-employing] employed
centering theory in pronoun resolution from a semantic perspective. An
interesting review on the connection between centering and anaphora can be
found in [JOSHI2006223]. While many researchers have attempted to use
centering theory or some modified versions of it for coreference resolution,
the centering theory itself is not a theory of coreference resolution. The
systematic analysis in this paper addresses the issue of the under-explored
correlation between these two and provides a further theoretical basis for the
use of centering theory in coreference. Moreover, we explore the relationship
between CT and coreference in the context of modern end-to-end coreference
systems.

##### End-to-end coreference resolution:

Another related line of work is on coreference resolution models, which are
our main experimental subjects. [lee-etal-2017-end] proposed an end-to-end
neural framework for coreference resolution, and further extended it to higher
order for global consistency [lee-etal-2018-higher]. Later work substituted
GloVe embeddings with pretrained transformers-based embeddings [joshi-
etal-2019-bert, joshi-etal-2020-spanbert]. The main focus of this work is to
investigate how much these modern coreference resolution models know about
entity coherence and adhere to the rules and constraints laid out by centering
theory.

##  11 Conclusion

This paper attempts to answer the following two research questions: 1) whether
neural coreference resolution models capture coherence information in terms of
centering theory, and 2) how much entity coherence modeling a model needs in
order to solve coreference resolution. The main conclusions are as follows: 1)
CT itself is not a theory of coreference: Solely minimizing the violations of
Centering Theory does not bring perfect coreference clustering performance;
there is a mismatch between the CT objective and coreference objective. 2) But
the distribution of CT and coreference do share quite large mutual
information, indicating there is a strong relationship (both linear and
nonlinear) between Coref and CT. 3) The relationship is not linear and it has
a very instructive shape: steep $\rightarrow$ flat. The mutual information
between coref and CT is large on low quality coref models, but when the coref
F1 of the model is at a certain level, the mutual information between these
two becomes quite small. 4) Contextualized embeddings contain much discourse-
level coherence information. That helps explain why little gains can CT
provide to the SOTA coref models.

##  Appendix A Realization in Centering Theory.

The definition of realize depends on the semantic theory one adopts.

##### Direct realization

$U$ directly realizes $c$ where $U$ is an utterance of some phrase for which
$c$ is the semantic interpretation. NP directly realizes $c$ can hold in cases
where NP is a definite description and $c$ is its denotation, its value-free
interpretation, e.g. “the current US president” directly realizes “Donald
Trump”.

##### Indirect realization

$U$ indirectly realizes $c$ when one of the noun phrases in $U$ is an
associative reference to that discourse entity [hawkins2015definiteness]. The
generalization from direct realization to indirect realization allows
capturing certain regularities in the use of definite descriptions and
pronouns.

##  Appendix B Reformulating Coreference Models in CT Framework as
$\mathbb{P}(\mathcal{E}\mid\mathcal{D})$.

In this appendix, we reformulate the state-of-the-art neural coreference
resolution models in the proposed CT framework.

###  B.1 e2e Model

The most popular end-to-end coreference framework, e2e [lee-etal-2017-end],
can be seen as a latent variable model. They obtain a set of candidate mention
spans $\mathcal{M}_{i}$ for each discourse by considering spans until a
certain length, those that do not cross sentence boundaries, etc.

They then implement the mention-entity mapping function $f$ by a chain-linking
algorithm. For each discourse $\mathcal{D}_{i}(i=1\dots N)$, we define a
sequence of latent variables
$\bm{A}_{i}=\langle\bm{a}_{i,1},\ldots,\bm{a}_{i,p}\rangle$, where
$\bm{a}_{i,j}$ takes values in $\mathcal{A}(i,j)=\\{0,1,\dots,j-1\\}$. Here,
$0$ is reserved for $\epsilon$ i.e. “new cluster” or “not a entity” or “not a
mention”.

They aim to learn a conditional probability distribution
$\mathbb{P}(\bm{A}_{1},\ldots,\bm{A}_{N}|\mathcal{D})$ whose most likely
configuration produces the correct clustering for each discourse. They use a
product of multinomials for each span:

$\mathbb{P}(\bm{A}_{1},\ldots,\bm{A}_{N}|\mathcal{D})=\prod_{i=1}^{N}\prod_{j=1}^{|\mathcal{M}_{i}|}\mathbb{P}(\bm{a}_{i,j}|\mathcal{D}_{i}).$

They further model this probability in the form of a softmax:

$\displaystyle\mathbb{P}(\bm{a}_{i,j}|\mathcal{D}_{i})=$
$\displaystyle\prod_{j=1}^{|\mathcal{M}_{i}|}\frac{\exp(s(m_{i,j},a_{i,j}))}{\sum_{a^{\prime}\in\mathcal{A}(i,j)}\exp(s(m_{i,j},a^{\prime}))},$

where $s(m_{i,j},a_{i,j})$ is a pairwise score for a coreference link between
mention span $m_{i,j}$ and $a_{i,j}$ in discourse $\mathcal{D}$ as

$\left\\{\begin{array}[]{ll}s_{m}(m_{i,j})+s_{m}(a_{i,j})+s_{a}(m_{i,j},a_{i,j})&a_{i,j}\neq\epsilon\\\
0&a_{i,j}=\epsilon\end{array}\right.$

Here $s_{m}(m_{i,j})$ is a unary score for span $m_{i,j}$ being a mention, and
$s_{a}(m_{i,j},a_{i,j})$ is pairwise score for span $a_{i,j}$ being an
antecedent of span $m_{i,j}$. The scoring functions are computed via feed-
forward neural networks as:

$\displaystyle s_{m}(m_{i,j})=$ $\displaystyle\bm{w}_{m}\cdot
f_{m}(\bm{g}(m_{i,j})),$ $\displaystyle s_{a}(m_{i,j},a_{i,j})=$
$\displaystyle\bm{w}_{a}\cdot f_{a}(\bm{g}(m_{i,j}),\bm{g}(a_{i,j}),$
$\displaystyle\bm{g}(m_{i,j})\circ\bm{g}(a_{i,j}),\phi(i,j))$

The antecedent scoring function $s_{a}(m_{i,j},a_{i,j})$ includes explicit
element-wise similarity of each span $\bm{g}(m_{i,j})\circ\bm{g}(a_{i,j})$ and
a feature vector $\phi(i,j)$ encoding speaker and genre information from the
metadata and the distance between the two spans.

##### Learning

: They learn their model by maximizing the marginal log-likelihood of all
correct antecedents implied by the gold clustering as

$\log\prod_{i=1}^{N}\prod_{j=1}^{|M_{i}|}\sum_{\hat{a}\in\mathcal{A}(i,j)\cap
GOLD(i,j)}\mathbb{P}(\hat{a}|\mathcal{D}).$

###  B.2 Higher Order Model

In the e2e model, only pairs of entity mentions are scored by the model. It
makes independent decisions about coreference links instead of adding mentions
to clusters globally. c2f [lee-etal-2018-higher] improves the first order
model to higher orders. They use the antecedent distribution from a span-
ranking architecture as an attention mechanism to iteratively refine span
representations. With the definitions of
$\mathbb{P}(\bm{a}_{i,j}|\mathcal{D}_{i})$, $s(m_{i,j},a_{i,j})$,
$s_{m}(m_{i,j})$ and $s_{a}(m_{i,j},a_{i,j})$ being the same, the span
representations $\bm{g}(m_{i,j})$ and $\bm{g}(a_{i,j})$ are computed via
iterations:

$\displaystyle\bm{h}_{i,j}=$
$\displaystyle\sum\limits_{a^{\prime}\in\mathcal{A}(i,j)}\mathbb{P}(a^{\prime}|\mathcal{D}_{i})\cdot\bm{g}(a^{\prime})^{t}$
$\displaystyle\bm{g}(m_{i,j})^{t+1}=$ $\displaystyle
f_{h}(\bm{g}(m_{i,j}))^{t}\circ\bm{g}(m_{i,j})^{t}$
$\displaystyle+(1-f_{h}(\bm{g}(m_{i,j})^{t}))\circ\bm{h}_{i,j}$
$\displaystyle\bm{g}(a_{i,j})^{t+1}=$ $\displaystyle
f_{h}(\bm{g}(a_{i,j}))^{t}\circ\bm{g}(a_{i,j})^{t}$
$\displaystyle+(1-f_{h}(\bm{g}(a_{i,j})^{t}))\circ\bm{h}_{i,j}$

where $\bm{g}(m_{i,j})^{t}$ and $\bm{g}(a_{i,j})^{t}$ are the span
representations of iteration $t$. The initialization of $\bm{g}(m_{i,j})^{1}$
and $\bm{g}(a_{i,j})^{1}$ is done by the first order model.

They also introduce a coarse-to-fine approach to alleviate the computational
cost.

###  B.3 Incremental Coreference Model

[xia-etal-2020-incremental] implements the mention-entity mapping function $f$
by a incremental algorithm as introduced in Algorithm 1.

Create an empty Entity Set $\mathcal{E}_{i}$;

for  _$U\in\mathcal{D}_{i}$_ do

$M\leftarrow$ SPANS$(U)$;

for  _$m\in M$_ do

scores $\leftarrow$ PAIRSCORE$(m,\mathcal{E}_{i})$;

top_score $\leftarrow$ max(scores);

top_e $\leftarrow$ argmax(scores);

if  _top_score $>0$_ then

Update(top_e, $m$);

end if

AddNewEntity($\mathcal{E}_{i},m$);

end for

EVICT($\mathcal{E}_{i}$)

end for

Algorithm 1 FindCluster($\mathcal{D}_{i}$)

They attach each $e\in\mathcal{E}_{i}$ with its own representation. For each
span, a scorer scores the span representation against all the cluster
representations. This is used to determine to which (if any) of the pre-
existing $e\in\mathcal{E}_{i}$ the current span should be mapped to. Upon
inclusion of the span in the cluster, the $e$’s representation is subsequently
updated via a (learned) function. Periodically, the model evicts less salient
entities, writing them to disk (based on cluster size and distance from the
end of the segment). Under this algorithm, each clustering decision is
permanent.

##  Appendix C Coreference Model Parameters Settings.

###  C.1 Parameters Settings

For c2f-SpanBERT models, we follow the implementation in [joshi-
etal-2019-bert]. We use the cased English variants in all our experiments. We
set the span to word ratio (spans_per_word) to 0.5, 0.4 or 0.2 for different
models. Documents are split into segments of max_segment_len of 512 for
SpanBERT-large and 384 for SpanBERT-base. The dimension of all feed-forward
networks is 1500 for SpanBERT-large and 1000 for SpanBERT-base. The most of
models adopt inference_order of 1, while for the best performing ones we
implemented we set it to 2 with coarse_to_fine being true.

For c2f-one-hot models and c2f-GloVe models, we follow the implementation in
[lee-etal-2018-higher]. c2f-GloVe models take GloVe embeddings as input while
c2f-one-hot models use one-hot word vectors as input and does not levarage any
pretrained embeddings. We set the LSTM context layer to be 1 layer. We set the
span to word ratio (spans_per_word) to 0.4. The dimension of all feed-forward
networks is 1500. They all adopt inference_order of 1.

For all the models mentioned above, we set the upper bound number of spans to
max_antecedents of 50. We also filtered out spans that are longer than 30
tokens. The dimension of the feature fed into feed-forward networks is 20.

For corefQA [wu-etal-2020-corefqa] and Incremental [xia-
etal-2020-incremental], we follow the parameter settings specified in their
papers.

###  C.2 Training Details

We trained models with Allennlp [Gardner2017AllenNLP] on the OntoNotes dataset
for 40 epochs, using a dropout rate of 0.3. We adopted early stopping with
patience being 10. We used a batch size of 1 document for SpanBERT-large and 4
for SpanBERT-base. SpanBERT-large models are trained on 4 Nvidia Tesla V100
GPUs while other models are trained on 1 Nvidia Tesla V100 GPU.

###  C.3 Two models in Section 7.2.

In Section 7.2, we compare different center-based metrics based on the CF-
candidates provided by two coreference models, a strong one (well-trained
c2f-SpanBERT-large, with a coref F1 of 78.87%) and a weak one (underfitted
e2e-SpanBERT-base, with a coref F1 of 17.51%), under four instantiations of
CT. These two models follow the parameters settings for SpanBERT-large and
SpanBERT-base in C.1. The inference_order of c2f-SpanBERT-large is set to 2;
the inference_order of e2e-SpanBERT-base is set to 1. c2f-SpanBERT-large is
trained for 40 epochs till convergence, while e2e-SpanBERT-base is trained
only for 2 epochs.

###  C.4 Checkpoints in Section 7.3.

We plot all epoches of all types of embeddings (SpanBERT, GloVe, One-hot) on
all different sizes of datasets ($10\%,20\%,\cdots,100\%$ of the original
training set):

  * •

c2f-base: 40 epoches * 3 types of embedding * 10 datasets * 5 runs = 6000

  * •

c2f-large-SpanBERT: 40 epoches * 10 datasets * 5 runs = 2000

  * •

pretrained-c2f-large-SpanBERT: 40 epoches * 5 runs = 200

  * •

corefQA: 40 epoches * 5 runs = 200

  * •

Incremental: 40 epoches * 5 runs = 200

In total, there are 8,600 checkpoints. The best epoches of c2f-large-SpanBERT,
pretrained-c2f-large-SpanBERT, corefQA and Incremental trained on the entire
training set (100%) are shown in Figure 1, referred as the state-of-the-art
models in recent years.

##  Appendix D Corpus-based Analysis of Centering Theory.

Table 8 and Table 9 present detailed statistics of the CT metrics on OntoNotes
5.0 test set. #valid_u is the number of valid utterances (utterences with non-
empty CF-list candidates) per document. #M and %M are the number of utterances
which do not violate the metric M and the percentage of those utterances out
of all valid utterances. Coh is short for Coherence, Sal is short for
Salience, Cont. is short for Continue, Ret. is short for Retain, and
$\overline{nocb}$ means the utterances with CB.

| #valid_u | #nocb | #Cheap | #Coh. | #Sal. | #Cont. | #Ret. | #SShift | #RShift | #KP
---|---|---|---|---|---|---|---|---|---|---
mean | 21.47 | 7.63 | 9.87 | 10.45 | 9.64 | 7.39 | 3.07 | 2.26 | 7.77 | 10.95
std | 18.22 | 6.40 | 10.16 | 10.12 | 9.58 | 7.51 | 3.36 | 2.75 | 7.13 | 10.74
min | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00
25% | 7.00 | 2.00 | 2.00 | 3.00 | 3.00 | 2.00 | 1.00 | 0.00 | 2.00 | 3.00
50% | 15.00 | 6.00 | 6.00 | 7.00 | 6.00 | 5.00 | 2.00 | 1.00 | 6.00 | 7.00
75% | 31.25 | 11.00 | 14.00 | 15.00 | 14.00 | 11.00 | 5.00 | 3.00 | 12.00 | 15.25
max | 92.00 | 31.00 | 63.00 | 57.00 | 54.00 | 39.00 | 19.00 | 16.00 | 32.00 | 63.50
| %valid_u |  %¯nocb | %Cheap | %Coh. | %Sal. | %Cont. | %Ret. | %SShift | %RShift | %KP
mean | 80.02 | 60.14 | 42.78 | 46.51 | 42.85 | 39.17 | 14.15 | 9.99 | 36.12 | 48.07
std | 19.26 | 17.48 | 17.57 | 15.25 | 16.05 | 23.68 | 12.46 | 10.22 | 20.02 | 14.80
min | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00
25% | 70.00 | 50.00 | 32.71 | 37.06 | 33.33 | 24.76 | 4.17 | 0.00 | 25.00 | 39.48
50% | 84.62 | 61.45 | 44.44 | 47.11 | 44.44 | 35.56 | 12.50 | 9.09 | 36.36 | 50.00
75% | 95.06 | 73.08 | 53.67 | 54.96 | 52.17 | 50.00 | 20.00 | 14.85 | 50.00 | 57.54
max | 100.00 | 93.48 | 87.50 | 87.50 | 87.50 | 100.00 | 100.00 | 50.00 | 100.00 | 87.50
Table 8: The detailed statistics of CT-based metrics on OntoNotes, with CF candidates being “cluster only” and CF ranking being grammatical-role-based. | #valid_u | #nocb | #Cheap | #Coh. | #Sal. | #Cont. | #Ret. | #SShift | #RShift | #KP
---|---|---|---|---|---|---|---|---|---|---
mean | 21.47 | 7.63 | 9.60 | 10.22 | 8.79 | 6.68 | 3.54 | 2.10 | 8.15 | 10.61
std | 18.22 | 6.40 | 9.86 | 9.88 | 8.67 | 6.83 | 3.82 | 2.52 | 7.46 | 10.36
min | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00
25% | 7.00 | 2.00 | 2.75 | 3.00 | 2.00 | 2.00 | 1.00 | 0.00 | 2.00 | 3.00
50% | 15.00 | 6.00 | 6.00 | 7.00 | 6.00 | 4.00 | 2.00 | 1.00 | 6.00 | 6.75
75% | 31.25 | 11.00 | 14.00 | 15.00 | 13.00 | 10.00 | 5.00 | 3.00 | 13.00 | 15.56
max | 92.00 | 31.00 | 60.00 | 60.00 | 52.00 | 42.00 | 18.00 | 12.00 | 32.00 | 63.00
| %valid_u |  %¯nocb | %Cheap | %Coh. | %Sal. | %Cont. | %Ret. | %SShift | %RShift | %KP
mean | 80.02 | 60.14 | 41.62 | 45.79 | 39.67 | 36.50 | 16.10 | 9.23 | 37.59 | 46.81
std | 19.26 | 17.48 | 17.03 | 14.75 | 15.64 | 23.93 | 13.34 | 9.44 | 19.86 | 14.10
min | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | -0.00 | 0.00
25% | 70.00 | 50.00 | 30.19 | 36.09 | 29.45 | 22.22 | 7.55 | 0.00 | 25.82 | 38.65
50% | 84.62 | 61.45 | 42.86 | 46.15 | 40.00 | 33.33 | 15.38 | 8.33 | 38.46 | 47.79
75% | 95.06 | 73.08 | 52.50 | 54.17 | 50.00 | 46.15 | 23.08 | 14.29 | 50.00 | 56.51
max | 100.00 | 93.48 | 80.00 | 85.71 | 80.00 | 100.00 | 100.00 | 50.00 | 100.00 | 80.00
Table 9: The detailed statistics of CT-based metrics on OntoNotes, with CF
candidates being “cluster only” and CF ranking being semantic-role-based.

##  Appendix E Coreference Performance of models

Table 10 presents the complete results of coreference evaluation on the
OntoNotes 5.0 test set. The metrics being used are MUC [vilain-
etal-1995-model], $B^{3}$ [bagga1998algorithms] and $\mathrm{CEAF}_{\phi_{4}}$
[luo-2005-coreference]. We include performance of systems proposed in the past
5 years for reference. We reevaluated the top performing models (e2e, c2e,
Incremental and CorefQA), and report both the results directly from their
papers and those of our implementations. Here we observed small differences in
performance between our implementations and those reported in original papers,
due to slightly different implementation details and parameter settings.

| MUC | $B^{3}$ | $CEAF_{\phi_{4}}$ |
---|---|---|---|---
Model | P | R | F1 | P | R | F1 | P | R | F1 | Avg. F1
Martschat and Strube (2015) [martschat-strube-2015-latent] | 76.7 | 68.1 | 72.2 | 66.1 | 54.2 | 59.6 | 59.5 | 52.3 | 55.7 | 62.5
Clark and Manning (2015) [clark-manning-2015-entity] | 76.1 | 69.4 | 72.6 | 65.6 | 56.0 | 60.4 | 59.4 | 53.0 | 56.0 | 63.0
Wiseman et al. (2015) [wiseman-etal-2015-learning] | 76.2 | 69.3 | 72.6 | 66.2 | 55.8 | 60.5 | 59.4 | 54.9 | 57.1 | 63.4
Wiseman et al. (2016) [wiseman-etal-2016-learning] | 77.5 | 69.8 | 73.4 | 66.8 | 57.0 | 61.5 | 62.1 | 53.9 | 57.7 | 64.2
Clark and Manning (2016a) [clark-manning-2016-deep] | 79.9 | 69.3 | 74.2 | 71.0 | 56.5 | 63.0 | 63.8 | 54.3 | 58.7 | 65.3
Clark and Manning (2016b) [clark-manning-2016-improving] | 79.2 | 70.4 | 74.6 | 69.9 | 58.0 | 63.4 | 63.5 | 55.5 | 59.2 | 65.7
e2e [lee-etal-2017-end] | 78.4 | 73.4 | 75.8 | 68.6 | 61.8 | 65.0 | 62.7 | 59.0 | 60.8 | 67.2
c2f [lee-etal-2018-higher] | 81.4 | 79.5 | 80.4 | 72.2 | 69.5 | 70.8 | 68.2 | 67.1 | 67.6 | 73.0
Fei et al. (2019) [fei-etal-2019-end] | 85.4 | 77.9 | 81.4 | 77.9 | 66.4 | 71.7 | 70.6 | 66.3 | 68.4 | 73.8
EE [kantor-globerson-2019-coreference] | 82.6 | 84.1 | 83.4 | 73.3 | 76.2 | 74.7 | 72.4 | 71.1 | 71.8 | 76.6
e2e-BERT-base [joshi-etal-2019-bert] | 80.2 | 82.4 | 81.3 | 69.6 | 73.8 | 71.6 | 69.0 | 68.6 | 68.8 | 73.9
c2f-BERT-large [joshi-etal-2019-bert] | 84.7 | 82.4 | 83.5 | 76.5 | 74.0 | 75.3 | 74.1 | 69.8 | 71.9 | 76.9
Incremental-large [xia-etal-2020-incremental] | 85.7 | 84.8 | 85.3 | 78.1 | 77.5 | 77.8 | 76.3 | 74.1 | 75.2 | 79.4
c2f-SpanBERT-large[joshi-etal-2020-spanbert] | 85.8 | 84.8 | 85.3 | 78.3 | 77.9 | 78.1 | 76.4 | 74.2 | 75.3 | 79.6
CorefQA-base [wu-etal-2020-corefqa] | 85.2 | 87.4 | 86.3 | 78.7 | 76.5 | 77.6 | 76.0 | 75.6 | 75.8 | 79.9
CorefQA-large [wu-etal-2020-corefqa] | 88.6 | 87.4 | 88.0 | 82.4 | 82.0 | 82.2 | 79.9 | 78.3 | 79.1 | 83.1
e2e-BERT-base (our) | 79.9 | 82.0 | 80.9 | 70.3 | 72.1 | 71.2 | 69.2 | 67.5 | 68.3 | 73.5
c2f-BERT-base (our) | 81.0 | 81.6 | 81.3 | 70.2 | 73.3 | 71.7 | 69.4 | 69.0 | 69.2 | 74.1
e2e-SpanBERT-base (our) | 85.6 | 82.5 | 84.0 | 76.8 | 74.9 | 75.8 | 75.9 | 70.2 | 72.9 | 77.6
c2f-SpanBERT-base (our) | 86.8 | 83.2 | 85.0 | 77.2 | 75.3 | 76.2 | 76.5 | 71.1 | 73.7 | 78.3
Incremental-base (our) | 85.2 | 82.9 | 84.0 | 77.0 | 75.1 | 76.0 | 75.4 | 70.5 | 72.9 | 77.6
CorefQA-base (our) | 85.0 | 86.2 | 85.6 | 78.9 | 75.6 | 77.2 | 76.2 | 76.8 | 76.5 | 79.8
e2e-BERT-large (our) | 83.9 | 84.3 | 84.1 | 75.5 | 74.2 | 74.8 | 73.0 | 69.0 | 70.9 | 76.6
c2f-BERT-large (our) | 84.7 | 82.4 | 83.5 | 76.5 | 74.0 | 75.3 | 74.1 | 69.8 | 71.9 | 76.9
e2e-SpanBERT-large (our) | 88.0 | 82.8 | 85.3 | 78.2 | 75.4 | 76.8 | 77.0 | 70.8 | 73.8 | 78.6
c2f-SpanBERT-large (our) | 84.9 | 85.2 | 85.0 | 77.3 | 77.0 | 77.1 | 78.0 | 76.2 | 77.1 | 79.8
Incremental-large (our) | 85.2 | 84.2 | 84.7 | 77.9 | 78.1 | 78.0 | 78.0 | 72.4 | 75.1 | 79.3
CorefQA-large (our) | 89.7 | 88.5 | 89.1 | 80.2 | 81.9 | 81.0 | 77.5 | 80.0 | 78.7 | 83.0
Table 10: Complete results of different coreference models on the OntoNotes
5.0 test set with three coreference resolution metrics: MUC, $B^{3}$,
$CEAF_{\phi_{4}}$ and the average F1 of them. $P$, $R$ and $F1$ in the first
row represent precision, recall and F1 score respectively. The bottom part is
the results of our implementations. base and large represent the embedding
encoder used.

##  Appendix F Additional Analysis Results

Figure 6: The CT scores as a function of coreference F1 score for
c2f-SpanBERT, c2f-GloVe and c2f-one-hot. Figure 7: The coref F1 score and the
other 5 CT scores as a function of training epoch for c2f-SpanBERT, c2f-GloVe
and c2f-one-hot. Models are trained on 100% OntoNotes training dataset. Figure
8: The CT scores as a function of coreference F1 score for c2f-SpanBERT,
c2f-GloVe and c2f-one-hot.

##  Appendix G Analysis: Properties of Discourse.

In this section, we exam whether different properties of discourse influence
this relationship between coreference and Centering Theory.

###  G.1 Document Length and #Token per sentence

As we can see from Figure 9, we observe that 1) the CT scores (80%) is an
average of all the documents, where longer documents tend to have higher CT
scores and shorter documents have lower scores; 2) this is not the case for
Coreference F1 score, where the predicted clusterings of shorter documents are
actually better in the eye of Coreference resolution F1. These figures show
that for both the distribution of Y condictioned on the predicted $X$ and the
real distribution of $Y$, longer documents tend to have higher CT scores
(while it is not the case for coreference F1 scores). These obersvations
indicates that the conditional distribution $P(Y\mid X_{SpanBERT})$ is very
close to the real distribution $P(Y)$ compared to $P(Y\mid X_{GloVe})$ and
$P(Y\mid X_{One-hot})$; the mutual information between $X_{SpanBERT}$ and $Y$
is quite large.

Figure 9: The CT scores and the coreference F1 score as a function of
discourse properties (i.g. #sentences, #tokens, average #tokens per sentence)
for c2f-SpanBERT, c2f-GloVe and c2f-one-hot and Ontonotes ground truth
annotations.

###  G.2 Genres

We report the normalized mutual information and correlations for different
genres in Figure 10.

Figure 10: The mutual information between coref F1 and CT scores for different
genres.