# Direct Approach of Indefinite Linear-Quadratic Mean Field Games ††thanks:
This work is supported by National Key R&D Program of China (2022YFA1006104),
National Natural Science Foundations of China (12271304), and Shandong
Provincial Natural Science Foundations (ZR2022JQ01, ZR2020ZD24, ZR2019ZD42).

Wenyu Cong , Jingtao Shi School of Mathematics, Shandong University, Jinan
250100, P.R. China, E-mail<EMAIL_ADDRESS>author.
School of Mathematics, Shandong University, Jinan 250100, P.R. China, E-mail:
<EMAIL_ADDRESS>
Abstract: This paper is concerned with an indefinite linear-quadratic mean
field games of stochastic large-population system, where the individual
diffusion coefficients can depend on both the state and the control of the
agents. Moreover, the control weights in the cost functionals could be
indefinite. A direct approach is used to derive the $\epsilon$-Nash
equilibrium strategy. First, we formally solving an $N$-player game problem
within a vast and finite population setting. Subsequently, decoupling or
reducing high-dimensional systems by introducing two Riccati equations
explicitly yields centralized strategies, contingent on the state of a
specific player and the average state of the population. As the population
size $N$ goes infinity, the construction of decentralized strategies becomes
feasible. Then, we demonstrated they are an $\epsilon$-Nash equilibrium.
Numerical examples are provided to demonstrate the effectiveness of the
proposed strategies.

Keywords: mean field games, direct approach, linear-quadratic stochastic
differential game, forward-backward stochastic differential equation,
$\epsilon$-Nash equilibrium, indefinite control weight

Mathematics Subject Classification: 93E20, 60H10, 49K45, 49N70, 91A23

##  1 Introduction

Mean field games (MFGs) have been attracting growing scholarly attention,
finding applications in diverse fields such as system control, applied
mathematics, and economics ([2], [6], [4], [5]). MFG theory serves as a
framework for describing the behavior of models characterized by large
populations, where the influence of the overall population is significant,
despite the negligible impact on individual entities. It investigates the
existence of Nash equilibria. This is accomplished by leveraging the
relationship between the finite and corresponding infinite-limit population
problems. The methodological foundations of MFG, initially proposed by Lasry
and Lions [14] and independently by Huang et al. [12], have proven effective
and tractable for analyzing weakly coupled stochastic controlled systems with
mean field interactions, establishing approximate Nash equilibria. In
particular, within the linear-quadratic (LQ) framework, MFGs provide a
versatile modeling tool applicable to a wide range of practical problems. The
solutions derived from LQ-MFGs exhibit notable and elegant properties. Current
scholarly discourse has extensively explored MFGs, particularly within the LQ
framework ([15], [3], [17], [13]). Huang et al. [11] conducted a study on
$\epsilon$-Nash equilibrium strategies in the context of LQ-MFGs with
discounted costs. This investigation was rooted in the fixed-point approach.
Subsequently, it was applied to scenarios involving long run average costs, in
Li and Zhang [15]. In the domain of MFGs featuring major players, Huang [10]
delved into continuous-time LQ games, providing insights into $\epsilon$-Nash
equilibrium strategies. Huang et al. [8] introduced a backward-major and
forward-minor setup for an LQ-MFG, and decentralized $\epsilon$-Nash
equilibrium strategies for major and minor agents were obtained. Huang et al.
[9] examined the backward LQ-MFG of weakly coupled stochastic large population
systems under both full and partial information scenarios. Huang and Li [7]
delved into an LQ-MFG concerning a class of stochastic delayed systems. Xu and
Zhang [20] explored a general LQ-MFG for stochastic large population systems,
where the individual diffusion coefficient is contingent on the state and
control of the agent. Bensoussan et al. [1] considered an LQ-MFG with partial
observation and common noise.

Conventionally, two approaches are employed in the resolution of mean field
games. One is termed the fixed-point approach (or top-down approach, see [11],
[12], [15], [2], [5]), which initiates the process by employing mean field
approximation and formulating a fixed-point equation. By tackling the fixed-
point equation and scrutinizing the optimal response of a representative
player, decentralized strategies can be formulated. The alternative approach
is known as the direct approach (or bottom-up approach, refer to [14], [19],
[13], [18]). This method commences by formally solving an $N$-player game
problem within a vast and finite population setting. Subsequently, by
decoupling or reducing high-dimensional systems, centralized control can be
explicitly derived, contingent on the state of a specific player and the
average state of the population. As the population size $N$ approaches
infinity, the construction of decentralized strategies becomes feasible. In
the direct approach, there are no fixed-point conditions. In [13], the authors
discussed the connection between these two routes within an LQ setting.

In this paper, we explore an indefinite LQ-MFG, where the individual diffusion
coefficients can depend on both the state and the control of the agents, the
control weights in the cost functionals could be indefinite. We solve the
problem by directly decoupling high-dimensional forward-backward stochastic
differential equations (FBSDEs) via introducing two Riccati equations, which
in turn gives the centralized strategy. Then, inspired by the form of above
Riccati equations, the limiting form of the Riccati equations is devised and
its solvability is analyzed. The solvability of the original Riccati equations
follows from the continuous dependence of the solution on the parameter. By
the law of large numbers, we get the decentralized strategy, and verify that
it is an $\epsilon$-Nash equilibrium strategy.

The main contributions of the paper are outlined as follows.

$\bullet$ In the drift and diffusion terms of the individual’s dynamical
system, both incorporate the individual’s own state and control. There is also
a coupling of the individual state to the population state average in the cost
function. These new type bring mathematical difficulties, and, to some degree,
they have some potential applications in reality.

$\bullet$ Instead of employing the traditional fixed-point methodology such as
[20], we adopt a direct approach to address the complexities inherent in our
game problem, which was used in [13], [19]. This strategic shift involves the
meticulous decoupling of the high-dimensional Hamiltonian system using mean
field approximations. By disentangling the high-dimensional Hamiltonian system
through mean field approximations, we formulate a set of decentralized
strategies for all players. This set is subsequently demonstrated to
constitute an $\epsilon$-Nash equilibrium.

The paper is organized as follows. In Section 2, we formulate the problem. In
Section 3, we construct decentralized strategies. In Section 4, the proof of
the asymptotic optimality is rigorously presented. Numerical examples are
provided in Section 5 to demonstrate the effectiveness of the proposed
strategies. Finally, some conclusions are given in Section 6.

The following notations will be used throughout this paper. We use $||\cdot||$
to denote the norm of a Euclidean space, or the Frobenius norm for matrices.
For a symmetric matrix $Q$ and a vector $z$, $||z||^{2}_{Q}\equiv z^{\top}Qz$.
For any real-valued scalar functions $f(\cdot)$ and $g(\cdot)$ defined on
$\mathbb{R}$, $f(x)=O(g(x))$ means that there exists a constant $C>0$ such
that $\lim_{x\to\infty}|\frac{f(x)}{g(x)}|=C$, where $|\cdot|$ is an absolute
value, which is also equivalent to saying that there exist $C>0$ and $x$ such
that $|f(x)|\leq C|g(x)|$ for any $x\geq x^{\prime}$.

Let $T>0$ be a finite time duration and
$(\Omega,\mathcal{F},\\{\mathcal{H}_{t}\\}_{0\leq t\leq T},\mathbb{P})$ be a
complete filtered probability space with the filtration
$\\{\mathcal{H}_{t}\\}_{0\leq t\leq T}$ augmented by all the $\mathbb{P}$-null
sets in $\mathcal{F}$. $\mathbb{E}$ denoted the expectation with respect to
$\mathbb{P}$. Let $L_{\mathcal{H}}^{2}(0,T;\cdot)$ be the set of all vector-
valued (or matrix-valued) $\mathcal{H}_{t}$-adapted processes $f(\cdot)$ such
that $\mathbb{E}\int^{T}_{0}||f(t)||^{2}dt<\infty$ and
$L^{2}_{\mathcal{H}_{t}}(\Omega;\cdot)$ be the set of
$\mathcal{H}_{t}$-measurable random variables, for $t\in[0,T]$.

##  2 Problem formulation

We consider a large-population system with $N$ agents, where $N$ can be
arbitrarily large. The states equation of the $i$th agent $\mathcal{A}_{i}$,
is given by the following SDE:

$\left\\{\begin{aligned}
dx_{i}(t)=&\big{[}A(t)x_{i}(t)+B(t)u_{i}(t)+f(t)\big{]}dt\\\
&+\big{[}C(t)x_{i}(t)+D(t)u_{i}(t)+g(t)\big{]}dW_{i}(t),\quad t\in[0,T],\\\
x_{i}(0)=&\ \xi_{i},\end{aligned}\right.$ (2.1)

where $1\leq i\leq N$, $x_{i}(\cdot)$, $u_{i}(\cdot)$ and $\xi_{i}\in
L^{2}_{\mathcal{F}_{0}^{i}}(\Omega;\mathbb{R})$ are the state process, control
process and initial value (random variable) of $\mathcal{A}_{i}$. In this
paper, for simplicity, we assume the dimensions of state process and control
process are both one-dimensional. Here, $A$, $B$, $C$, $D$, $f$ and $g$ are
deterministic $\mathbb{R}$-valued functions. $W_{i}(\cdot),i=1,\cdots,N$ are a
sequence of one-dimensional Brownian motions defined on
$(\Omega,\mathcal{F},\\{\mathcal{F}_{t}\\}_{0\leq t\leq T},\mathbb{P})$. Let
$\mathcal{F}_{t}$ be the $\sigma$-algebra generated by
$\\{\xi_{i},W_{i}(s),s\leq t,1\leq i\leq N\\}$. Denote $\mathcal{F}_{t}^{i}$
be the $\sigma$-algebra generated by $\\{\xi_{i},W_{i}(s),s\leq t\\},1\leq
i\leq N$. We define the decentralized control set for all agents and the $i$th
agent $\mathcal{A}_{i}$ as follows:

$\displaystyle\mathscr{U}_{d}[0,T]$
$\displaystyle:=\big{\\{}(u_{1}(\cdot),\cdots,u_{N}(\cdot))|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}^{i}}(0,T;\mathbb{R})\big{\\}},$
$\displaystyle\mathscr{U}_{d}^{i}[0,T]$
$\displaystyle:=\big{\\{}u_{i}(\cdot)|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}^{i}}(0,T;\mathbb{R})\big{\\}},$

and the centralized control set for all agents and the $i$th agent
$\mathcal{A}_{i}$:

$\displaystyle\mathscr{U}_{c}[0,T]$
$\displaystyle:=\big{\\{}(u_{1}(\cdot),\cdots,u_{N}(\cdot))|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R}),1\leq i\leq N\big{\\}},$
$\displaystyle\mathscr{U}_{c}^{i}[0,T]$
$\displaystyle:=\big{\\{}u_{i}(\cdot)|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R})\big{\\}}.$

The cost functional of $\mathcal{A}_{i}$ is given by

$\displaystyle J^{N}_{i}(u_{i}(\cdot),u_{-i}(\cdot))$
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[\Big{|}\Big{|}x_{i}(t)-\Gamma(t)x^{(N)}(t)-\eta(t)\Big{|}\Big{|}^{2}_{Q(t)}+||u_{i}(t)||^{2}_{R(t)}\right]dt\right.$
(2.2)
$\displaystyle\qquad\qquad\left.+\Big{|}\Big{|}x_{i}(T)-\Gamma_{0}x^{(N)}(T)-\eta_{0}\Big{|}\Big{|}^{2}_{H}\right\\},$

where $Q$, $R$, $\Gamma$ and $\eta$ are deterministic $\mathbb{R}$-valued
functions, $H$, $\Gamma_{0}$ and $\eta_{0}$ are scalar constants.
$x^{(N)}(t):=\frac{1}{N}\sum_{j=1}^{N}x_{j}(t)$ is called the state average or
mean field term of all agents,
$u_{-i}(\cdot):=(u_{1}(\cdot),\cdots,u_{i-1}(\cdot),u_{i+1}(\cdot),\cdots,u_{N}(\cdot))$.

We propose the following assumptions:

(A1) $\\{\xi_{i},1\leq i\leq N\\}$ is a sequence of independent and
identically distributed random variables with
$\mathbb{E}[\xi_{i}]=\bar{\xi},i=1,2,\cdots,N$, and there exists a constant
$C$ such that $\sup_{1\leq i\leq N}\mathbb{E}[||\xi_{i}||^{2}]\leq C.$

(A2) $\\{W_{i}(t),1\leq i\leq N\\}$ are mutually independent, which are
independent of $\\{\xi_{i},1\leq i\leq N\\}$.

(A3) $Q(t)\geq 0,\forall\ 0\leq t\leq T$, $H\geq 0$.

In this paper, we explore the following two problems.

(P): Find a Nash equilibrium strategy
$u^{N*}(\cdot):=(u_{1}^{*}(\cdot),\cdots,u_{N}^{*}(\cdot))$ to (2.2), subject
to (2.1), i.e., for all $i$, $1\leq i\leq N$,

$J^{N}_{i}(u_{i}^{*}(\cdot),u_{-i}^{*}(\cdot))\leq\inf_{u_{i}(\cdot)\in\mathscr{U}_{c}^{i}[0,T]}J^{N}_{i}(u_{i}(\cdot),u_{-i}^{*}(\cdot)).$

(P’): Find an $\epsilon$-Nash equilibrium strategy $u^{N*}(\cdot)$ to (2.2),
subject to (2.1), i.e., there exists a constant $\epsilon\geq 0$ such that for
all $i$, $1\leq i\leq N$,

$J^{N}_{i}(u_{i}^{*}(\cdot),u_{-i}^{*}(\cdot))\leq\inf_{u_{i}(\cdot)\in\mathscr{U}_{d}^{i}[0,T]}J^{N}_{i}(u_{i}(\cdot),u_{-i}^{*}(\cdot))+\epsilon.$

##  3 Design of decentralized strategies

Subsequently, the time arguments of functions may be omitted if their
exclusion does not lead to confusion. We initially derive the following
result.

######  Theorem 3.1.

Under Assumption (A1)-(A3), for the initial value $\xi_{i},i=1,\cdots,N$, (P)
admits an optimal control
$\check{u}_{i}(\cdot)\in\mathscr{U}_{c}^{i}[0,T],i=1,\cdots,N$, if and only if
the following two conditions hold:

(i) For $i=1,\cdots,N$, the adapted solution
$(\check{x}_{i}(\cdot),\check{p}_{i}(\cdot),\check{q}^{j}_{i}(\cdot),j=1,\cdots,N)$
to the FBSDE

$\left\\{\begin{aligned}
d\check{x}_{i}&=\big{[}A\check{x}_{i}+B\check{u}_{i}+f\big{]}dt+\big{[}C\check{x}_{i}+D\check{u}_{i}+g\big{]}dW_{i},\\\
d\check{p}_{i}&=-\left[A\check{p}_{i}+C\check{q}^{i}_{i}+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\eta\right)\right]dt\\\
&\quad+\sum_{j=1}^{N}\check{q}_{i}^{j}dW_{j},\quad t\in[0,T],\\\
\check{x}_{i}(0)&=\xi_{i},\quad\check{p}_{i}(T)=H\left(1-\frac{\Gamma_{0}}{N}\right)\left(\check{x}_{i}(T)-\Gamma_{0}\check{x}^{(N)}(T)-\eta_{0}\right),\end{aligned}\right.$
(3.1)

satisfies the following stationarity condition:

$B\check{p}_{i}+D\check{q}^{i}_{i}+R\check{u}_{i}=0,\quad a.e.,\ a.s..$ (3.2)

(ii) For $i=1,\cdots,N$, the following convexity condition holds:

$\mathbb{E}\left\\{\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\right]dt+H\left(1-\frac{\Gamma_{0}}{N}\right)^{2}\tilde{x}_{i}^{2}(T)\right\\}\geq
0,\quad\forall u_{i}(\cdot)\in\mathscr{U}_{c}^{i}[0,T],$ (3.3)

where $\tilde{x}_{i}(\cdot)$ represents the solution to the following SDE:

$\left\\{\begin{aligned}
d\tilde{x}_{i}&=\left[A\tilde{x}_{i}+Bu_{i}\right]dt+\left[C\tilde{x}_{i}+Du_{i}\right]dW_{i},\quad
t\in[0,T],\\\ \tilde{x}_{i}(0)&=0.\end{aligned}\right.$ (3.4)

###### Proof.

We consider the agent $\mathcal{A}_{i}$. For given $\xi_{i}\in
L^{2}_{\mathcal{F}_{0}^{i}}(\Omega;\mathbb{R})$, suppose that
$\check{u}_{i}(\cdot)$ is a candidate of the optimal strategy for
$\mathcal{A}_{i}$, suppose that
$(\check{x}_{i}(\cdot),\check{p}_{i}(\cdot),\check{q}^{j}_{i}(\cdot),j=1,\cdots,N)$
is an adapted solution to FBSDE (3.1). For any
$u_{i}(\cdot)\in\mathscr{U}_{c}^{i}[0,T]$ and $\theta\in\mathbb{R}$, let
$x_{i}^{\theta}(\cdot)$ be the solution to the following perturbed state
equation:

$\left\\{\begin{aligned}
dx_{i}^{\theta}&=\big{[}Ax_{i}^{\theta}+B(\check{u}_{i}+\theta
u_{i})+f\big{]}dt+\big{[}Cx_{i}^{\theta}+D(\check{u}_{i}+\theta
u_{i})+g\big{]}dW_{i},\\\ x_{i}^{\theta}(0)&=\xi_{i}.\end{aligned}\right.$
(3.5)

Then,
$\tilde{x}_{i}^{\theta}(\cdot):=\frac{x_{i}^{\theta}(\cdot)-\check{x}_{i}(\cdot)}{\theta}$
is independent of $\theta$ and satisfies (3.4). Applying Itô’s formula to
$\check{p}_{i}(\cdot)\tilde{x}_{i}(\cdot)$, we obtain

$\displaystyle\mathbb{E}$
$\displaystyle\bigg{[}H\left(1-\frac{\Gamma_{0}}{N}\right)\left(\check{x}_{i}(T)-\Gamma_{0}\check{x}^{(N)}(T)-\eta_{0}\right)\tilde{x}_{i}(T)\bigg{]}$
$\displaystyle=\mathbb{E}$
$\displaystyle\big{[}\check{p}_{i}(T)\tilde{x}_{i}(T)-\check{p}_{i}(0)\tilde{x}_{i}(0)\big{]}$
$\displaystyle=\mathbb{E}$
$\displaystyle\int_{0}^{T}\Biggl{\\{}-\left[A\check{p}_{i}+C\check{q}^{i}_{i}+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\eta\right)\right]\tilde{x}_{i}$
$\displaystyle\qquad+\left[A\tilde{x}_{i}+Bu_{i}\right]\check{p}_{i}+\left[C\tilde{x}_{i}+Du_{i}\right]\check{q}^{i}_{i}\Biggr{\\}}dt$
$\displaystyle=\mathbb{E}$
$\displaystyle\int_{0}^{T}\left\\{-\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\eta\right)\tilde{x}_{i}+Bu_{i}\check{p}_{i}+Du_{i}\check{q}^{i}_{i}\right\\}dt.$

Therefore,

$\displaystyle J^{N}_{i}(\check{u}_{i}(\cdot)+\theta
u_{i}(\cdot),u_{-i}(\cdot))-J^{N}_{i}(\check{u}_{i}(\cdot),u_{-i}(\cdot))$
$\displaystyle=$ $\displaystyle\
\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[\Big{|}\Big{|}x_{i}^{\theta}-\Gamma
x^{(N)\theta}-\eta\Big{|}\Big{|}^{2}_{Q}+||\check{u}_{i}+\theta
u_{i}||^{2}_{R}\right]dt+\Big{|}\Big{|}x_{i}^{\theta}(T)-\Gamma_{0}x^{(N)\theta}(T)-\eta_{0}\Big{|}\Big{|}^{2}_{H}\right\\}$
$\displaystyle-\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[\Big{|}\Big{|}\check{x}_{i}-\Gamma\check{x}^{(N)}-\eta\Big{|}\Big{|}^{2}_{Q}+||\check{u}_{i}||^{2}_{R}\right]dt+\Big{|}\Big{|}\check{x}_{i}(T)-\Gamma_{0}\check{x}^{(N)}(T)-\eta_{0}\Big{|}\Big{|}^{2}_{H}\right\\}$
$\displaystyle=$ $\displaystyle\
\frac{1}{2}\mathbb{E}\Biggl{\\{}\int_{0}^{T}\left[\bigg{|}\bigg{|}\left(1-\frac{\Gamma}{N}\right)(\check{x}_{i}+\theta\tilde{x}_{i})-\Gamma
x^{(N-1)}_{-i}-\eta\bigg{|}\bigg{|}^{2}_{Q}+||\check{u}_{i}+\theta
u_{i}||^{2}_{R}\right]dt$
$\displaystyle\qquad+\bigg{|}\bigg{|}\left(1-\frac{\Gamma_{0}}{N}\right)(\check{x}_{i}(T)+\theta\tilde{x}_{i}(T))-\Gamma_{0}x^{(N-1)}_{-i}(T)-\eta_{0}\bigg{|}\bigg{|}^{2}_{H}\Biggr{\\}}$
$\displaystyle-\frac{1}{2}\mathbb{E}\Biggl{\\{}\int_{0}^{T}\left[\bigg{|}\bigg{|}\left(1-\frac{\Gamma}{N}\right)\check{x}_{i}-\Gamma
x^{(N-1)}_{-i}-\eta\bigg{|}\bigg{|}^{2}_{Q}+||\check{u}_{i}||^{2}_{R}\right]dt$
$\displaystyle\qquad\quad+\bigg{|}\bigg{|}\left(1-\frac{\Gamma_{0}}{N}\right)\check{x}_{i}(T)-\Gamma_{0}x^{(N-1)}_{-i}(T)-\eta_{0}\bigg{|}\bigg{|}^{2}_{H}\Biggr{\\}}$
$\displaystyle=$ $\displaystyle\
\frac{1}{2}\theta^{2}\mathbb{E}\left\\{\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\bigg{]}dt+H\left(1-\frac{\Gamma_{0}}{N}\right)^{2}\tilde{x}_{i}^{2}(T)\right\\}$
$\displaystyle+\theta\mathbb{E}\Biggl{\\{}\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\eta\right)+u_{i}R\check{u}_{i}\right]dt$
$\displaystyle\qquad+H\left(1-\frac{\Gamma_{0}}{N}\right)\left(\check{x}_{i}(T)-\Gamma_{0}\check{x}^{(N)}(T)-\eta_{0}\right)\tilde{x}_{i}(T)\Biggr{\\}}$
$\displaystyle=$ $\displaystyle\
\frac{1}{2}\theta^{2}\mathbb{E}\left\\{\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\bigg{]}dt+H\left(1-\frac{\Gamma_{0}}{N}\right)^{2}\tilde{x}_{i}^{2}(T)\right\\}$
$\displaystyle+\theta\mathbb{E}\left\\{\int_{0}^{T}u_{i}\left[B\check{p}_{i}+D\check{q}^{i}_{i}+R\check{u}_{i}\right]dt\right\\},$

where only here $x^{(N)\theta}(\cdot):=\frac{1}{N}\sum_{j\neq
i}x_{j}(\cdot)+\frac{1}{N}x_{i}^{\theta}(\cdot)$,
$\check{x}^{(N)}(\cdot):=\frac{1}{N}\sum_{j\neq
i}x_{j}(\cdot)+\frac{1}{N}\check{x}_{i}(\cdot)$ and
$x^{(N-1)}_{-i}(\cdot):=\frac{1}{N}\sum_{j\neq i}x_{j}(\cdot)$. Thus, we have,

$J^{N}_{i}(\check{u}_{i}(\cdot),u_{-i}(\cdot))\leq
J^{N}_{i}(\check{u}_{i}(\cdot)+\theta u_{i}(\cdot),u_{-i}(\cdot))$

if and only if (3.2) and (3.3) hold. The proof is complete. ∎

The next step is to obtain a proper form for deriving the decentralized
feedback representation of optimal strategy.

We denote $\xi^{(N)}:=\check{x}^{(N)}(0)=\frac{1}{N}\sum_{j=1}^{N}\xi_{j}$,
$\check{u}^{(N)}(\cdot):=\frac{1}{N}\sum_{j=1}^{N}\check{u}_{j}$. It follows
from (3.1) that

$\left\\{\begin{aligned}
d\check{x}^{(N)}=&\big{[}A\check{x}^{(N)}+B\check{u}^{(N)}+f\big{]}dt+\frac{1}{N}\sum_{j=1}^{N}\big{[}Cx_{j}+Du_{j}+g\big{]}dW_{j},\quad
t\in[0,T],\\\ \check{x}^{(N)}(0)=&\ \xi^{(N)}.\end{aligned}\right.$ (3.6)

We consider the transformation

$\check{p}_{i}(\cdot)=P_{N}(\cdot)\check{x}_{i}(\cdot)+K_{N}(\cdot)\check{x}^{(N)}(\cdot)+\phi_{N}(\cdot),\quad
i=1,\cdots,N,$

where $P_{N}(\cdot)$, $K_{N}(\cdot)$ and $\phi_{N}(\cdot)$ are differential
functions with $P_{N}(T)=H\left(1-\frac{\Gamma_{0}}{N}\right)$,
$K_{N}(T)=-H\left(1-\frac{\Gamma_{0}}{N}\right)\Gamma_{0}$,
$\phi_{N}(T)=-H\left(1-\frac{\Gamma_{0}}{N}\right)\eta_{0}$.

Then by the first equation of (3.1), (3.6) and Itô’s formula, we get

$\displaystyle d\check{p}_{i}$
$\displaystyle=\dot{P}_{N}\check{x}_{i}dt+P_{N}\Big{\\{}\big{[}A\check{x}_{i}+B\check{u}_{i}+f\big{]}dt+\big{[}C\check{x}_{i}+D\check{u}_{i}+g\big{]}dW_{i}\Big{\\}}+\dot{K}_{N}\check{x}^{(N)}dt$
(3.7)
$\displaystyle\quad+K_{N}\Big{\\{}\big{[}A\check{x}^{(N)}+B\check{u}^{(N)}+f\big{]}dt+\frac{1}{N}\sum_{j=1}^{N}\big{[}Cx_{j}+Du_{j}+g\big{]}dW_{j}\Big{\\}}+\dot{\phi}_{N}.$

Comparing the coefficients of the corresponding diffusion terms with the
second equation of (3.1), we have

$\check{q}^{i}_{i}=\left(P_{N}+\frac{1}{N}K_{N}\right)\big{[}C\check{x}_{i}+D\check{u}_{i}+g\big{]},$
(3.8)
$\check{q}^{j}_{i}=\frac{1}{N}K_{N}\big{[}C\check{x}_{j}+D\check{u}_{j}+g\big{]},\quad
j\neq i.$ (3.9)

From (3.2) and (3.8), we have for any $i=1,\cdots,N$,

$\alpha_{N}\check{u}_{i}+\beta_{N}\check{x}_{i}+\gamma_{N}\check{x}^{(N)}+\delta_{N}=0,$
(3.10)

where

$\left\\{\begin{aligned}
&\alpha_{N}:=R+(P_{N}+\frac{1}{N}K_{N})D^{2},\quad\beta_{N}:=BP_{N}+(P_{N}+\frac{1}{N}K_{N})CD,\\\
&\gamma_{N}:=BK_{N},\quad\delta_{N}:=B\phi_{N}+(P_{N}+\frac{1}{N}K_{N})gD.\end{aligned}\right.$
(3.11)

We introduce the following assumption:

(A4) $\alpha_{N}\neq 0$.

If (A4) holds, we obtain that the optimal control is given by

$\check{u}_{i}=-\alpha_{N}^{-1}\left[\beta_{N}\check{x}_{i}+\gamma_{N}\check{x}^{(N)}+\delta_{N}\right].$
(3.12)

Furthermore, we have

$\check{u}^{(N)}=-\alpha_{N}^{-1}\left[(\beta_{N}+\gamma_{N})\check{x}^{(N)}+\delta_{N}\right].$
(3.13)

This together with (3.7) gives

$\left\\{\begin{aligned}
&\dot{P}_{N}+2AP_{N}-P_{N}B\alpha_{N}^{-1}\beta_{N}+C\left(P_{N}+\frac{1}{N}K_{N}\right)\left(C-D\alpha_{N}^{-1}\beta_{N}\right)+\left(1-\frac{\Gamma}{N}\right)Q=0,\\\
&P_{N}(T)=H\left(1-\frac{\Gamma_{0}}{N}\right),\end{aligned}\right.$ (3.14)
$\left\\{\begin{aligned}
&\dot{K}_{N}+2AK_{N}-P_{N}B\alpha_{N}^{-1}\gamma_{N}-K_{N}B\alpha_{N}^{-1}\left(\beta_{N}+\gamma_{N}\right)\\\
&\
-CD\left(P_{N}+\frac{1}{N}K_{N}\right)\alpha_{N}^{-1}\gamma_{N}-\left(1-\frac{\Gamma}{N}\right)Q\Gamma=0,\\\
&K_{N}(T)=-H\left(1-\frac{\Gamma_{0}}{N}\right)\Gamma_{0},\end{aligned}\right.$
(3.15) $\left\\{\begin{aligned}
&\dot{\phi}_{N}+fP_{N}+fK_{N}+A\phi_{N}-P_{N}B\alpha_{N}^{-1}\delta_{N}-K_{N}B\alpha_{N}^{-1}\delta_{N}\\\
&\
+C\left(P_{N}+\frac{1}{N}K_{N}\right)\left(g-D\alpha_{N}^{-1}\delta_{N}\right)-\left(1-\frac{\Gamma}{N}\right)Q\eta=0,\\\
&\phi_{N}(T)=-H\left(1-\frac{\Gamma_{0}}{N}\right)\eta_{0}.\end{aligned}\right.$
(3.16)

Note that (3.16) is a linear backward ordinary differential equation (backward
ODE). If (3.14)-(3.15) admit solutions, then (3.16) has a solution. From the
above discussion, Theorem 3.1 and Theorem 4.1 on page 47 of Ma and Yong [16],
we have the following result.

######  Proposition 3.1.

Under Assumptions (A1)-(A4), if (3.14)-(3.15) respectively admits a solution,
then Problem (P) admits a unique solution.

$\check{u}_{i}=-\alpha_{N}^{-1}\left[\beta_{N}\check{x}_{i}+\gamma_{N}\check{x}^{(N)}+\delta_{N}\right],\quad
i=1,\cdots,N.$ (3.17)

Inspired by the discussion above, we denote

$\left\\{\begin{aligned} &\alpha:=R+PD^{2},\quad\beta:=BP+PCD,\\\
&\gamma:=BK,\quad\delta:=B\phi+PgD,\end{aligned}\right.$ (3.18)

where $P$, $K$ and $\phi$ satisfy

$\left\\{\begin{aligned} &\dot{P}+2AP+CPC-\alpha^{-1}\beta^{2}+Q=0,\\\
&P(T)=H,\end{aligned}\right.$ (3.19) $\left\\{\begin{aligned} &\dot{K}+2AK-
KB\alpha^{-1}(\beta+\gamma)-\beta\alpha^{-1}\gamma-Q\Gamma=0,\\\
&K(T)=-H\Gamma_{0},\end{aligned}\right.$ (3.20) $\left\\{\begin{aligned}
&\dot{\phi}+fP+fK+A\phi+CPg-
KB\alpha^{-1}\delta-\beta\alpha^{-1}\delta-Q\eta=0,\\\
&\phi(T)=-H\eta_{0}.\end{aligned}\right.$ (3.21)

In the following we will discuss the solvability of the Riccati equations
(3.19) and (3.20). By (3.18), we can rewrite (3.19) as

$\dot{P}+(2A+C^{2})P+Q-(R+D^{2}P)^{-1}(B+CD)^{2}P^{2}=0,\quad P(T)=H.$

Two results for the solvability of this Riccati equation are given in [20].
When the solvability condition of the Riccati equation (3.19) holds, we
reorganize the Riccati equation (3.20) as

$\left\\{\begin{aligned} 0=&\
\dot{K}-Q\Gamma+2\big{[}A-B(R+PD^{2})^{-1}(BP+PCD)\big{]}K-(R+PD^{2})^{-1}B^{2}K^{2},\\\
:=&\ \dot{K}+a+bK+cK^{2},\\\ K(T)=&-H\Gamma_{0}.\end{aligned}\right.$ (3.22)

According to the basic conclusion of ODE, if there exists constant $M\geq 0$,
such that

$|a(t)|+|b(t)|+|c(t)|\leq M,\quad c(t)\neq 0,\quad\forall t\in[0,T],$

then (3.22) admits a unique solution, i.e., (3.20) admits a unique solution.

By leveraging the continuous dependence of the solution on the parameter, as
discussed in [13], and under the aforementioned conditions, we ascertain that
for a sufficiently large $N$, (3.14)-(3.16) admit solutions, respectively.

Let $N\to\infty$. By the law of large numbers, we may approximate $x^{(N)}$ in
(3.6) with $\bar{x}$, which satisfies

$d\bar{x}=\big{[}(A-B\alpha^{-1}(\beta+\gamma))\bar{x}-B\alpha^{-1}\delta+f\big{]}dt,\quad\bar{x}(0)=\bar{\xi}.$
(3.23)

By Proposition 3.1, the decentralized strategy for agent
$\mathcal{A}_{i},i=1,\cdots,N$ may be taken as

$\hat{u}_{i}=-\alpha^{-1}[\beta\hat{x}_{i}+\gamma\bar{x}+\delta],$ (3.24)

where $\hat{x}_{i}$ satisfies

$\left\\{\begin{aligned} d\hat{x}_{i}=&\
\big{[}(A-B\alpha^{-1}\beta)\hat{x}_{i}-B\alpha^{-1}\gamma\bar{x}-B\alpha^{-1}\delta+f\big{]}dt\\\
&+\big{[}(C-D\alpha^{-1}\beta)\hat{x}_{i}-D\alpha^{-1}\gamma\bar{x}-D\alpha^{-1}\delta+g\big{]}dW_{i},\quad
t\in[0,T],\\\ \hat{x}_{i}(0)=&\ \xi_{i}.\end{aligned}\right.$ (3.25)

##  4 $\epsilon$-Nash equilibria

In this section, we aim to demonstrate that the decentralized strategies
(3.24) of agent $\mathcal{A}_{i},i=1,\cdots,N$, constitute an approximated
$\epsilon$-Nash equilibrium.

######  Theorem 4.1.

Assume that (A1)-(A4) hold. For problem (P’),
$(\hat{u}_{1}(\cdot),\cdots,\hat{u}_{N}(\cdot))$ given in (3.24) constitutes
an $\epsilon$-Nash equilibrium, where
$\epsilon=O\left(\frac{1}{\sqrt{N}}\right)$.

###### Proof.

Through (3.25), we have

$\left\\{\begin{aligned} d\hat{x}^{(N)}=&\
\big{[}(A-B\alpha^{-1}\beta)\hat{x}^{(N)}-B\alpha^{-1}\gamma\bar{x}-B\alpha^{-1}\delta+f\big{]}dt\\\
&+\frac{1}{N}\sum_{j=1}^{N}\big{[}(C-D\alpha^{-1}\beta)\hat{x}_{i}-D\alpha^{-1}\gamma\bar{x}-D\alpha^{-1}\delta+g\big{]}dW_{j},\\\
\hat{x}^{(N)}(0)=&\ \xi^{(N)}.\end{aligned}\right.$ (4.1)

By (3.23) and (4.1), it can be verified that

$\mathbb{E}\int_{0}^{T}||\hat{x}^{(N)}-\bar{x}||^{2}dt=O\left(\frac{1}{N}\right).$
(4.2)

For $i=1,\cdots,N$, denote
$\tilde{u}_{i}(\cdot):=u_{i}(\cdot)-\hat{u}_{i}(\cdot)$ and
$\tilde{x}_{i}(\cdot):=x_{i}(\cdot)-\hat{x}_{i}(\cdot)$. Then for fixed
$u_{-i}$, $\tilde{x}_{i}(\cdot)$ satisfies

$d\tilde{x}_{i}=\left[A\tilde{x}_{i}+B\tilde{u}_{i}\right]dt+\left[C\tilde{x}_{i}+D\tilde{u}_{i}\right]dW_{i},\quad\tilde{x}_{i}(0)=0,\quad
i=1,\cdots,N.$

Thus,

$\sum_{i=1}^{N}\mathbb{E}\int_{0}^{T}\left(||\tilde{x}_{i}(t)||^{2}+||\tilde{u}_{i}(t)||^{2}\right)dt<\infty.$

From (2.2), we have

$J_{i}^{N}(u_{i}(\cdot),\hat{u}_{-i}(\cdot))=J_{i}^{N}(\hat{u}_{i}(\cdot),\hat{u}_{-i}(\cdot))+\tilde{J}_{i}^{N}(\tilde{u}_{i}(\cdot),\hat{u}_{-i}(\cdot))+\mathcal{I}_{i}^{N},$
(4.3)

where

$\tilde{J}_{i}^{N}(\tilde{u}_{i}(\cdot),\hat{u}_{-i}(\cdot)):=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[\bigg{|}\bigg{|}\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\bigg{|}\bigg{|}^{2}_{Q}+||\tilde{u}_{i}||^{2}_{R}\right]dt+\bigg{|}\bigg{|}\left(1-\frac{\Gamma_{0}}{N}\right)\tilde{x}_{i}(T)\bigg{|}\bigg{|}^{2}_{H}\right\\},$

and

$\displaystyle\mathcal{I}_{i}^{N}:=\mathbb{E}\Biggl{\\{}$
$\displaystyle\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\left(\hat{x}_{i}-\Gamma\hat{x}^{(N)}-\eta\right)+R\tilde{u}_{i}\hat{u}_{i}\bigg{]}dt$
$\displaystyle+\left(1-\frac{\Gamma_{0}}{N}\right)H\tilde{x}_{i}(T)\left(\hat{x}_{i}(T)-\Gamma_{0}\hat{x}^{(N)}(T)-\eta_{0}\right)\Biggr{\\}}.$

Let $\hat{p}_{i}=P\hat{x}_{i}+K\bar{x}+\phi$. By (3.25), (3.23) and Itô’s
formula, we obtain

$\displaystyle d\hat{p}_{i}=$ $\displaystyle\
\big{[}\dot{P}+P(A-B\alpha^{-1}\beta)\big{]}\hat{x}_{i}dt+\big{[}\dot{K}-PB\alpha^{-1}\gamma+K(A-B\alpha^{-1}(\beta+\gamma))\big{]}\bar{x}dt$
$\displaystyle+\big{[}\dot{\phi}-PB\alpha^{-1}\delta+Pf-
KB\alpha^{-1}\delta+Kf\big{]}dt$
$\displaystyle+P\big{[}(C-D\alpha^{-1}\beta)\hat{x}_{i}-D\alpha^{-1}\gamma\bar{x}-D\alpha^{-1}\delta+g\big{]}dW_{i}.$

Applying Itô’s formula to $\tilde{x}_{i}(\cdot)\hat{p}_{i}(\cdot)$, we have

$\displaystyle\mathbb{E}\big{[}H\tilde{x}_{i}(T)\left(\hat{x}_{i}(T)-\Gamma_{0}\bar{x}(T)-\eta_{0}\right)\big{]}$
$\displaystyle=$ $\displaystyle\
\mathbb{E}\big{[}\tilde{x}_{i}(T)\big{(}P(T)\hat{x}_{i}(T)+K(T)\bar{x}(T)+\phi(T)\big{)}-\tilde{x}_{i}(0)\big{(}P(0)\hat{x}_{i}(0)+K(0)\bar{x}(0)+\phi(0)\big{)}\big{]}$
$\displaystyle=$ $\displaystyle\
\mathbb{E}\int_{0}^{T}\bigg{\\{}\left[A\tilde{x}_{i}+B\tilde{u}_{i}\right](P\hat{x}_{i}+K\bar{x}+\phi)+\big{[}\dot{P}+P(A-B\alpha^{-1}\beta)\big{]}\tilde{x}_{i}\hat{x}_{i}$
$\displaystyle\qquad+\big{[}\dot{K}-PB\alpha^{-1}\gamma+K(A-B\alpha^{-1}(\beta+\gamma))\big{]}\tilde{x}_{i}\bar{x}$
$\displaystyle\qquad+\big{[}\dot{\phi}-PB\alpha^{-1}\delta+Pf-
KB\alpha^{-1}\delta+Kf\big{]}\tilde{x}_{i}$
$\displaystyle\qquad+\left[C\tilde{x}_{i}+D\tilde{u}_{i}\right]P\big{[}(C-D\alpha^{-1}\beta)\hat{x}_{i}-D\alpha^{-1}\gamma\bar{x}-D\alpha^{-1}\delta+g\big{]}\bigg{\\}}$
$\displaystyle=$ $\displaystyle\
\mathbb{E}\int_{0}^{T}\bigg{\\{}\left[\dot{P}+2AP-
PB\alpha^{-1}\beta+CP(C-D\alpha^{-1}\beta)\right]\tilde{x}_{i}\hat{x}_{i}+\big{[}B\hat{p}_{i}+D\hat{q}^{i}_{i}\big{]}\tilde{u}_{i}$
$\displaystyle\qquad+\left[\dot{K}+2KA-PB\alpha^{-1}\gamma-
KB\alpha^{-1}(\beta+\gamma)-CPD\alpha^{-1}\gamma\right]\tilde{x}_{i}\bar{x}$
$\displaystyle\qquad+\biggl{[}\dot{\phi}+A\phi+Pf+Kf-PB\alpha^{-1}\delta-
KB\alpha^{-1}\delta-
CPD\alpha^{-1}\delta+CPg\biggr{]}\tilde{x}_{i}\bigg{\\}}dt$ $\displaystyle=$
$\displaystyle\
\mathbb{E}\int_{0}^{T}\bigg{\\{}-Q\tilde{x}_{i}\hat{x}_{i}+Q\Gamma\tilde{x}_{i}\bar{x}+Q\eta\tilde{x}_{i}-R\hat{u}_{i}\tilde{u}_{i}\bigg{\\}}dt$
$\displaystyle=$ $\displaystyle\
\mathbb{E}\int_{0}^{T}\bigg{\\{}-Q\tilde{x}_{i}(\hat{x}_{i}-\Gamma\bar{x}-\eta)-R\hat{u}_{i}\tilde{u}_{i}\bigg{\\}}dt,$

where $\hat{q}^{i}_{i}:=P\big{(}C\hat{x}_{i}+D\hat{u}_{i}+g\big{)}$. The
penultimate equation is due to (3.2) and (3.19)-(3.21). Then,

$\displaystyle\mathcal{I}_{i}^{N}=$ $\displaystyle\
\mathbb{E}\bigg{\\{}\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\big{(}\hat{x}_{i}-\Gamma\bar{x}-\eta\big{)}\
+Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\Gamma\left(\bar{x}-\hat{x}^{(N)}\right)+R\tilde{u}_{i}\hat{u}_{i}\bigg{]}dt$
$\displaystyle\quad+H\left(1-\frac{\Gamma_{0}}{N}\right)\tilde{x}_{i}(T)\big{(}\hat{x}_{i}(T)-\Gamma_{0}\bar{x}(T)-\eta_{0}\big{)}$
$\displaystyle\quad+H\left(1-\frac{\Gamma_{0}}{N}\right)\tilde{x}_{i}(T)\Gamma_{0}\left(\bar{x}(T)-\hat{x}^{(N)}(T)\right)\bigg{\\}}$
$\displaystyle=$ $\displaystyle\
\mathbb{E}\biggl{\\{}\int_{0}^{T}\biggl{[}\frac{\Gamma}{N}R\hat{u}_{i}\tilde{u}_{i}+Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\Gamma\left(\bar{x}-\hat{x}^{(N)}\right)\biggr{]}dt$
$\displaystyle\quad+\frac{\Gamma-\Gamma_{0}}{N}H\tilde{x}_{i}(T)\big{(}\hat{x}_{i}(T)-\Gamma_{0}\bar{x}(T)-\eta_{0}\big{)}$
$\displaystyle\quad+H\left(1-\frac{\Gamma_{0}}{N}\right)\tilde{x}_{i}(T)\Gamma_{0}\left(\bar{x}(T)-\hat{x}^{(N)}(T)\right)\biggr{\\}}=O\left(\frac{1}{\sqrt{N}}\right).$

Thereby,

$J_{i}^{N}(\hat{u}_{i}(\cdot),\hat{u}_{-i}(\cdot))\leq
J_{i}^{N}(u_{i}(\cdot),\hat{u}_{-i}(\cdot))+\epsilon.$

Thus, $(\hat{u}_{1}(\cdot),\cdots,\hat{u}_{N}(\cdot))$ is an $\epsilon$-Nash
equilibrium, where $\epsilon=O\left(\frac{1}{\sqrt{N}}\right)$. ∎

##  5 Numerical examples

In this section, an example of numerical simulation of Problem (P’) is given
to verify the conclusions we obtained in the previous section. Let
$A=B=f=C=D=g=Q=R=H=\Gamma=\eta=\Gamma_{0}=\eta_{0}=1$. The time interval is
set to $[0,10]$. Each player’s initial state is drawn independently from a
uniform distribution on $[0,20]$. The curve of $P(t)$ and $K(t)$, described by
(3.19)-(3.20), are shown in Figure 1.

Figure 1: The curves of $P(t)$ and $K(t)$

We denote the performance of the decentralized strategy by
$\epsilon(N)=\bigl{(}\mathbb{E}\int_{0}^{T}||\hat{x}^{(N)}-\bar{x}||^{2}dt\bigr{)}^{\frac{1}{2}}$.
A plot of $\epsilon(N)$ with respect to $N$ is shown in Figure 2, which
confirms the consistency of the mean field approximation.

Figure 2: $\epsilon(N)$ with respect to $N$

##  6 Conclusions

In this paper, we have investigated an indefinite LQ mean field stochastic
differential game. We use the direct approach to tackle this problem, and the
resultant decentralized strategy is demonstrated to be an $\epsilon$-Nash
equilibrium. Direct approach ([13], [19], [18]) applying to more kinds of mean
field games is our future research interest.

## References

  * [1] A. Bensoussan, X.W. Feng, and J.H. Huang, Linear-quadratic-Gaussian mean-field-game with partial observation and common noise. Math. Control Relat. Fields, 2021, 11(1): 23-46.
  * [2] A. Bensoussan, J. Frehse, and P. Yam, Mean Field Games and Mean Field Type Control Theory. Springer, New York, 2013.
  * [3] A. Bensoussan, K.C.J. Sung, S.C.P. Yam, and S.P. Yung, Linear-quadratic mean field games. J. Optim. Theory Appl., 2016, 169(2): 496-529.
  * [4] P.E. Caines, M.Y. Huang, and R.P. Malhamé, Mean field games. In T. Başar, G. Zaccour, Handbook of Dynamic Game Theory, Springer, Berlin, 2017.
  * [5] R. Carmona, F. Delarue, Probabilistic Theory of Mean Field Games with Applications I, II. Springer, Cham, Switzerland, 2018.
  * [6] D.A. Gomes, J. Saúde, Mean field games models-a brief survey. Dyn. Games Appl., 2014, 4(2): 110-154.
  * [7] J.H. Huang, N. Li, Linear-quadratic mean-field game for stochastic delayed systems. IEEE Trans. Automat. Control, 2018, 63(8): 2711-2729.
  * [8] J.H. Huang, S.J. Wang, and Z. Wu, Backward-forward linear-quadratic mean-field games with major and minor agents. Probab. Uncer. Quant. Risk, 2016, 1: 8.
  * [9] J.H. Huang, S.J. Wang, and Z. Wu, Backward mean-field linear-quadratic-Gaussian (LQG) games: full and partial information. IEEE Trans. Automat. Control, 2016, 61(12): 3784-3796.
  * [10] M.Y. Huang, Large-population LQG games involving a major player: the Nash certainty equivalence principle, SIAM J. Control Optim., 2010, 48(5): 3318-3353.
  * [11] M.Y. Huang, P.E. Caines, and R.P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\epsilon$-Nash equilibria. IEEE Trans. Automat. Control, 2007, 52(9): 1560-1571.
  * [12] M.Y. Huang, R.P. Malhamé, and P.E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst., 2006, 6(3): 221-251.
  * [13] M.Y. Huang, M.J. Zhou, Linear quadratic mean field games: asymptotic solvability and relation to the fixed point approach. IEEE Trans. Automat. Control, 2020, 65(4): 1397-1412.
  * [14] J.M. Lasry, P.L. Lions, Mean field games. Jpn. J. Math., 2007, 2(1): 229-260.
  * [15] T. Li, J.F. Zhang, Asymptotically optimal decentralized control for large population stochastic multiagent systems. IEEE Trans. Automat. Control, 2008, 53(7): 1643-1660.
  * [16] J. Ma, J.M. Yong, Forward-backward Stochastic Differential Equations and Their Applications. Lecture Notes Math., Springer, New York, 1999.
  * [17] J. Moon, T. Başar, Linear quadratic risk-sensitive and robust mean field games. IEEE Trans. Automat. Control, 2017, 62(3): 1062-1077.
  * [18] B.C. Wang, Leader-follower mean field LQ games: a direct method. Asian J. Control, 2024, 26(2), 617-625.
  * [19] B.C. Wang, H.S. Zhang, and J.F. Zhang, Mean field linear-quadratic control: uniform stabilization and social optimality. Automatica, 2020, 121: 109088.
  * [20] R.M. Xu, F. Zhang, $\epsilon$-Nash mean-field games for general linear-quadratic systems with applications. Automatica, 2020, 114: 108835.


# The SABRE South Experiment at the Stawell Underground Physics Laboratory

I. Bolognino<EMAIL_ADDRESS>Department of Physics, The
University of Adelaide, Adelaide, SA 5005, Australia  ARC Centre of Excellence
for Dark Matter Particle Physics, Australia  for the SABRE South
collaboration: E. Barberio  T. Baroncelli  ARC Centre of Excellence for Dark
Matter Particle Physics, Australia  School of Physics, The University of
Melbourne, Melbourne, VIC 3010, Australia  J. L. Bignell  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  Department of Nuclear
Physics and Accelerator Applications, The Australian National University,
Canberra, ACT 2601, Australia  I. Bolognino  Department of Physics, The
University of Adelaide, Adelaide, SA 5005, Australia  ARC Centre of Excellence
for Dark Matter Particle Physics, Australia  G. Brooks  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  School of Engineering,
Swinburne University of Technology, Hawthorn, VIC 3122, Australia  F. Dastgiri
ARC Centre of Excellence for Dark Matter Particle Physics, Australia
Department of Nuclear Physics and Accelerator Applications, The Australian
National University, Canberra, ACT 2601, Australia  A. R. Duffy  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  Centre for
Astrophysics and Supercomputing, Swinburne University of Technology, Hawthorn,
VIC 3122, Australia  M. Froehlich  ARC Centre of Excellence for Dark Matter
Particle Physics, Australia  Department of Nuclear Physics and Accelerator
Applications, The Australian National University, Canberra, ACT 2601,
Australia  G. Fu  M. S. M. Gerathy  ARC Centre of Excellence for Dark Matter
Particle Physics, Australia  School of Physics, The University of Melbourne,
Melbourne, VIC 3010, Australia  G. C. Hill  Department of Physics, The
University of Adelaide, Adelaide, SA 5005, Australia  ARC Centre of Excellence
for Dark Matter Particle Physics, Australia  S. Krishnan  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  School of Engineering,
Swinburne University of Technology, Hawthorn, VIC 3122, Australia  G. J. Lane
ARC Centre of Excellence for Dark Matter Particle Physics, Australia
Department of Nuclear Physics and Accelerator Applications, The Australian
National University, Canberra, ACT 2601, Australia  G. Lawrence  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  Centre for
Astrophysics and Supercomputing, Swinburne University of Technology, Hawthorn,
VIC 3122, Australia  K. T. Leaver  Department of Physics, The University of
Adelaide, Adelaide, SA 5005, Australia  ARC Centre of Excellence for Dark
Matter Particle Physics, Australia  I. Mahmood  ARC Centre of Excellence for
Dark Matter Particle Physics, Australia  School of Physics, The University of
Melbourne, Melbourne, VIC 3010, Australia  P. McGee  Department of Physics,
The University of Adelaide, Adelaide, SA 5005, Australia  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  L. J. McKie  P.C.
McNamara  ARC Centre of Excellence for Dark Matter Particle Physics, Australia
Department of Nuclear Physics and Accelerator Applications, The Australian
National University, Canberra, ACT 2601, Australia  M. Mews  W. J. D.
Melbourne  ARC Centre of Excellence for Dark Matter Particle Physics,
Australia  School of Physics, The University of Melbourne, Melbourne, VIC
3010, Australia  G. Milana  ARC Centre of Excellence for Dark Matter Particle
Physics, Australia  School of Engineering, Swinburne University of Technology,
Hawthorn, VIC 3122, Australia  L. J. Milligan  ARC Centre of Excellence for
Dark Matter Particle Physics, Australia  School of Physics, The University of
Melbourne, Melbourne, VIC 3010, Australia  J. Mould  ARC Centre of Excellence
for Dark Matter Particle Physics, Australia  Centre for Astrophysics and
Supercomputing, Swinburne University of Technology, Hawthorn, VIC 3122,
Australia  F. Nuti  ARC Centre of Excellence for Dark Matter Particle Physics,
Australia  School of Physics, The University of Melbourne, Melbourne, VIC
3010, Australia  F. Scutti  ARC Centre of Excellence for Dark Matter Particle
Physics, Australia  Centre for Astrophysics and Supercomputing, Swinburne
University of Technology, Hawthorn, VIC 3122, Australia  Z. Slavkovská  N. J.
Spinks  ARC Centre of Excellence for Dark Matter Particle Physics, Australia
Department of Nuclear Physics and Accelerator Applications, The Australian
National University, Canberra, ACT 2601, Australia  O. Stanley  ARC Centre of
Excellence for Dark Matter Particle Physics, Australia  School of Physics, The
University of Melbourne, Melbourne, VIC 3010, Australia  A. E. Stuchbery  ARC
Centre of Excellence for Dark Matter Particle Physics, Australia  Department
of Nuclear Physics and Accelerator Applications, The Australian National
University, Canberra, ACT 2601, Australia  G. N. Taylor  P. Urquijo  ARC
Centre of Excellence for Dark Matter Particle Physics, Australia  School of
Physics, The University of Melbourne, Melbourne, VIC 3010, Australia  A. G.
Williams  Department of Physics, The University of Adelaide, Adelaide, SA
5005, Australia  ARC Centre of Excellence for Dark Matter Particle Physics,
Australia  Y. Y. Zhong  ARC Centre of Excellence for Dark Matter Particle
Physics, Australia  Department of Nuclear Physics and Accelerator
Applications, The Australian National University, Canberra, ACT 2601,
Australia  M. J. Zurowski  ARC Centre of Excellence for Dark Matter Particle
Physics, Australia  School of Physics, The University of Melbourne, Melbourne,
VIC 3010, Australia

###### Abstract

The SABRE (Sodium iodide with Active Background REjection) experiment aims to
detect an annual rate modulation from dark matter interactions in ultra-high
purity NaI(Tl) crystals in order to provide a model independent test of the
signal observed by DAMA/LIBRA. It is made up of two separate detectors; SABRE
South located at the Stawell Underground Physics Laboratory (SUPL), in
regional Victoria, Australia, and SABRE North at the Laboratori Nazionali del
Gran Sasso (LNGS).

SABRE South is designed to disentangle seasonal or site-related effects from
the dark matter-like modulated signal by using an active veto and muon
detection system. Ultra-high purity NaI(Tl) crystals are immersed in a linear
alkyl benzene (LAB) based liquid scintillator veto, further surrounded by
passive steel and polyethylene shielding and a plastic scintillator muon veto.
Significant work has been undertaken to understand and mitigate the background
processes, that take into account radiation from the detector materials, from
both intrinsic and cosmogenic activated processes, and to understand the
performance of both the crystal and veto systems.

SUPL is a newly built facility located 1024 m underground ($\sim$2900 m water
equivalent) within the Stawell Gold Mine and its construction has been
completed in mid-2022. The laboratory will house rare event physics searches,
including the upcoming SABRE dark matter experiment, as well as measurement
facilities to support low background physics experiments and applications such
as radiobiology and quantum computing.

The SABRE South commissioning is expected to occur in 2023. This paper
describes the setup and projections for the experiment, and the description of
the underground laboratory.

dark matter, model-independent annual modulation signature, low-background
NaI(Tl) scintillators, SUPL

##  I INTRODUCTION

Dark Matter (DM) represents one of the most important open problems in modern
physics [1]. One search technique is direct detection which attempts to
observe the recoil of a target after scattering with a DM particle,
hypothesised as a WIMP (Weakly Interacting Massive Particles) with masses on
the Gev/c2-TeV/c2 and cross sections on the weak scale. This interaction rate
should modulate annually, with a sinusoidal trend, due to the rotation of the
Earth around the Sun as it moves through the galactic dark matter halo. To
date, only one direct detection experiment, DAMA/LIBRA, has observed a
modulation compatible with that expected from DM [2]. However, this signal is
in tension with every other direct detection experiment; all of which report
null results for a standard WIMP [3]. These are possible explanations: the
majority of experiments do not use the same target material as DAMA/LIBRA and
the current experiments with sodium iodide crystals do not achieve the same
sensitivity [4, 5]. Further, it can be argued that the modulation observed by
DAMA/LIBRA is due to seasonal effects, for example muons which also present a
modulation with a maximum and minimum in the same period, in the Northern
Hemisphere, as DM [6]. The SABRE Collaboration was formed to provide a model
independent test of this DAMA/LIBRA signal. It foresees two detectors located
in two different hemispheres: SABRE North at Gran Sasso National Laboratory
(LNGS), in Italy, and SABRE South at the Stawell Underground Physics
Laboratory (SUPL), in Australia. The two detectors have a number of common
features as they are centred around the same detector module concept, and use
common simulation, DAQ, and software frameworks. The two detectors differ in
their shielding designs - where SABRE South will utilise a liquid scintillator
system for in-situ evaluation and validation of the background to provide
background rejection and particle identification. SABRE North has opted for a
fully passive shielding since the organic scintillator has been phased out by
LNGS. The ultra-low background components will achieve the same sensitivity as
DAMA/LIBRA; moreover, the site in the Southern Hemisphere will allow for the
exclusion of systematic seasonal effects.

##  II The SABRE South setup

SABRE South is made up of three different subdetector systems: the NaI(Tl)
crystal detector system, the liquid scintillator veto system, and the muon
paddle detectors. The crystals and the liquid scintillator veto system are
further shielded by steel and polyethelyne walls. The full experimental setup
is shown in Figure 1. The experiment can host seven NaI(Tl) cylindrical
crystals 25 cm long and 5 cm in radius resulting in a mass of 7.2 kg per
crystal (50.4 kg in total). Each crystal is wrapped in PTFE foil, mounted on
by PTFE crystal holders, and coupled to a Hamamatsu R11065 PMT (7.6 cm
diameter). Crystals are then encapsulated in cylindical oxygen-free high-
thermal-conductivity copper enclosures flushed with nitrogen.

Figure 1: Rendering of the SABRE South detector (top). Detail of the crystal
enclosure (bottom).

Enclosures are submersed into a liquid scintillator which is primarily used to
tag and remove high energy decay products such as those from 40K. The system
has a 4$\pi$ coverage, made up of 12 kL of linear alkyl benzene (sourced from
JUNO) doped with PPO and Bis-MSB. Around the sides of the steel vessel this is
encased in eighteen 20.4 cm Hamamatsu R5912 PMTs that are sampled at a rate of
500 MS/s. Based on preliminary optical simulations in Geant4, this system is
expected to have a light yield of approximately 0.12 photoelectrons/keV,
though this has a strong position dependence. With a threshold of 50 keV the
inclusion of this system is able to reduce the total background by 27%,
providing a background of less than 1 cpd/kg/keV [7].
The muon detection system is made up of eight 3 m long EJ200 detector paddles
and has a total coverage of 9.6 m2 above the main vessel. Each paddle is
coupled to a Hamamatsu R13089 PMT on each end and sampled at a rate of 3.2
GS/s. Calibrations are still ongoing, but the energy threshold of these
detectors is expected to be around 1 MeV. The detectors have approximately 400
ps timing resolution resulting in 5 cm position resolution, allowing for a
long term measurement of the muon flux, and particle identification when used
with the liquid veto system.

##  III High-purity crystals

Crystal radiopurity is the most important aspect of the experiment. Indeed, as
already demonstrated by other existing NaI(Tl)-based experiments,[2, 4, 5], a
large fraction of the background for DM search comes from residual radioactive
contaminants in the crystal themselves, especially 40K and 210Pb; the latter
is a daughter of the uranium decay. Crystals are grown from Merck’s Astrograde
powder, the highest purity NaI powder commercially available, which has a
potassium contamination below 10 ppb, and uranium and thorium contamination
below 1 ppt.

The SABRE collaboration, through an R&D effort based out of Princeton in
collaboration with Radiation Monitoring Devices company, has developed some of
the lowest background crystals in the world with only 4.6 ppb of potassium
(Figure 2). The analysed crystal mass is equal to 4.3 kg.

Figure 2: NaI-33 crystal measurements [8] in red compared the ones from
DAMA/LIBRA [9], ANAIS [10] and COSINE [11]. Potassium, Thorium and Uranium
concentrations are very low respect to the other NaI(Tl) experiments.

RMD recently grew a NaI(Tl) crystal (NaI-35) for the SABRE South collaboration
of a final mass of 3.3 kg which is currently being characterised at LNGS. Its
light yield is $\sim$ 11.6 Photoelectrons/keV.

##  IV Background model and sensitivity

A full detector simulation was conducted to provide a background model for the
radioactive contamination [7]. The total radioactive background of less than
0.72 cpd/kg/keV (Table 1), dominated by NaI background (radiogenic and
cosmogenic) with 40K effectively suppressed with $\sim$80% efficiency [7].
This meets SABRE South’s goal of having less than 10% of background from non-
crystal sources. External backgrounds, shielding, LS bulk, and veto PMTs
contribute less than 10-3 cpd/kg/keV.

| Rate | Veto Efficiency
---|---|---
| [cpd/kg/kev] | [%]
Crystal radiogenic | $5.2\cdot 10^{-1}$ | 13
Crystal cosmogenic | $1.6\cdot 10^{-1}$ | 40
Crystal PMTs | $3.8\cdot 10^{-2}$ | 60
PTFE wrap | $4.5\cdot 10^{-3}$ | 13
Enclosures | $3.2\cdot 10^{-3}$ | 85
Conduits | $1.9\cdot 10^{-5}$ | 96
Liquid scintillator | $4.9\cdot 10^{-8}$ | $>99$
Steel vessel | $1.4\cdot 10^{-5}$ | $>99$
Veto PMTs | $1.9\cdot 10^{-5}$ | $>99$
Shielding | $3.9\cdot 10^{-6}$ | $>99$
External |  O($10^{-4}$) | $>99$
Total | $7.2\cdot 10^{-1}$ | 27
Table 1: Background rate in the dark matter region for the SABRE South
components, and the corresponding veto efficiency.

These simulations also indicated that as well as providing shielding and
vetoing, the liquid scintillator system also allows for in-situ measurements
of background contamination to help inform fits due to well defined
correlations in the liquid scintillator and crystals, an example of which is
shown in Figure 3.

Figure 3: Top: energy depositions in both the crystal and liquid scintillator.
Indicated in blue and pink are events due to 40K and 121Te. Bottom left: rate
recorded in crystals from 40K and 121Te. Bottom right: rate recorded in liquid
scintillator from 40K and 121Te.

The projected sensitivity of SABRE South assuming this background model from
simulated radioactivity and a total crystal mass of 50 kg is shown in Figures
4 and 5 assuming a standard spin independent WIMP and showing the evolution of
discovery and exclusion power for the DAMA/LIBRA modulation. Based on these
results SABRE South will be able to refute the interpretation of the
DAMA/LIBRA modulation as a dark matter signal with 3$\sigma$ CL. In the event
of observation of the annual modulation, this signal would reach a
significance of 5$\sigma$ CL with two full years of data.

Figure 4: 90% C.L. assuming a standard spin independent WIMP, a SABRE South
total crystal mass of 50 kg and background of 0.72 cpd/kg/keV. Figure 5:
Exclusion and discovery power of SABRE South for the DAMA/LIBRA modulation.

##  V The Stawell Underground Physics Laboratory

SABRE South is to be placed at the recently completed Stawell Underground
Physics Laboratory (SUPL). This is located in Western Victoria, Australia, 240
km from Melbourne. The laboratory is 1025 m below ground with a flat over
burden of basalt, providing almost 3 km of water equivalent shielding, and a
muon flux similar to that of Boulby, Figure 6.
Internal walls are pinned with steel, sprayed with low radioactivity shotcrete
and coated with Tekflex to keep the radioactivity level low.
Construction was completed in mid-2022; background muon, gamma, and neutron
measurements are planned to start in late 2022.

Figure 6: Muon flux at a number of underground laboratories.

##  VI CONCLUSIONS

SABRE South is part of the SABRE Collaboration designed to test DAMA/LIBRA
modulation. SABRE South is the first dark matter direct-detection experiment
in the Southern Hemisphere and will be located inside the new SUPL underground
laboratory.
High purity crystals and large active veto give ultra-low background of 0.72
cpd/kg/keV.
SABRE South will have 5$\sigma$ discovery (3$\sigma$ exclusion) power to a
DAMA-like signal with 2 years of data taking.
Construction will begin later this year with commissioning expected mid-2023,
after which data taking will commence.

###### Acknowledgements.

The SABRE South program is supported by the Australian Government through the
Australian Research Council (Grants: CE200100008, LE190100196, LE170100162,
LE160100080, DP190103123, DP170101675, LP150100705). This research was
partially supported by Australian Government Research Training Program
Scholarships and Melbourne Research Scholarships. This research was supported
by The University of Melbourne’s Research Computing Services and the Petascale
Campus Initiative. We thank the SABRE North collaboration for their
contribution to the SABRE South experiment design and to the simulation
framework. We also thank the Australian Nuclear Science and Technology
Organisation for the assistance with the material screening and the
measurement of background radiation at SUPL.

## References

  * [1] G. Bertone and D. Hooper, History of dark matter, Rev. Mod. Phys, 90, (2018), 045002,
https://doi.org/10.1103/RevModPhys.90.045002.

  * [2] R. Bernabei et al., The dark matter: DAMA/LIBRA and its perspectives (2021),
https://arxiv.org/abs/2110.04734.

  * [3] R. L. Workman and Others,Review of Particle Physics, PTEP 2022, 083C01 (2022),87
https://doi.org/10.1093/ptep/ptac097.

  * [4] G. Adhikari, et al., Three-year annual modulation search with COSINE-100 (2021) https://doi.org/10.48550/arXiv.2111.08863.
  * [5] J. Amaré, et al., Annual modulation results from three-year exposure of ANAIS-112, Phys. Rev. D 103(10), 102005 (2021) https://doi.org/10.1103/PhysRevD.103.102005.
  * [6] M. Agostini et al., Modulations of the cosmic muon signal in ten years of Borexino data,JCAP02 (2019), 046 https://doi.org/10.1088/1475-7516/2019/02/046.
  * [7] E. Barberio et al., Simulation and background characterisation of the SABRE South experiment, (2022) https://arxiv.org/abs/2205.13849.
  * [8] Characterization of SABRE crystal NaI-33 with direct underground counting, Eur. Phys. J. C 81, 299 (2021) https://doi.org/10.1140/epjc/s10052-021-09098-5.
  * [9] R. Bernabei et al., The DAMA/LIBRA apparatus, NIMA 592(3) (2008) https://doi.org/10.1016/j.nima.2008.04.082.
  * [10] J. Amaré et al., Analysis of backgrounds for the ANAIS-112 dark matter experiment, Eur. Phys. J. C 79, 412 (2019) https://doi.org/10.1140/epjc/s10052-019-6911-4.
  * [11] G. Adhikari et al., Search for a Dark Matter-Induced Annual Modulation Signal in NaI(Tl) with the COSINE-100 Experiment, Phys. Rev. Lett. 123, 031302 (2019) https://doi.org/10.1103/PhysRevLett.123.031302


# On Generalization in Coreference Resolution

Shubham Toshniwal1 , Patrick Xia211footnotemark: 1 , Sam Wiseman3, Karen
Livescu1, Kevin Gimpel1
1Toyota Technological Institute at Chicago
2Johns Hopkins University
3Duke University
{shtoshni, klivescu<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Equal Contribution

###### Abstract

While coreference resolution is defined independently of dataset domain, most
models for performing coreference resolution do not transfer well to unseen
domains. We consolidate a set of 8 coreference resolution datasets targeting
different domains to evaluate the off-the-shelf performance of models. We then
mix three datasets for training; even though their domain, annotation
guidelines, and metadata differ, we propose a method for jointly training a
single model on this heterogeneous data mixture by using data augmentation to
account for annotation differences and sampling to balance the data
quantities. We find that in a zero-shot setting, models trained on a single
dataset transfer poorly while joint training yields improved overall
performance, leading to better generalization in coreference resolution
models. This work contributes a new benchmark for robust coreference
resolution and multiple new state-of-the-art results.111Code available at
https://github.com/shtoshni92/fast-coref

##  1 Introduction

Coreference resolution is a core component of the NLP pipeline, as determining
which mentions in text refer to the same entity is used for a wide variety of
downstream tasks like knowledge extraction Li et al. (2020), question
answering Dhingra et al. (2018), and dialog systems Gao et al. (2019). As
these tasks span many domains, we need coreference models to generalize well.

Meanwhile, models for coreference resolution have improved due to neural
architectures with millions of parameters and the emergence of pretrained
encoders. However, model generalization across domains has always been a
challenge Yang et al. (2012); Zhao and Ng (2014); Poot and van Cranenburgh
(2020); Aktaş et al. (2020). Since these models are usually engineered for a
single dataset, they capture idiosyncrasies inherent in that dataset. As an
example, OntoNotes Weischedel et al. (2013), a widely-used general-purpose
dataset, provides metadata, like the document genre and speaker information.
However, this assumption cannot be made more broadly, especially if the input
is raw text Wiseman et al. (2016).

| Num. Docs | Words/ | Mentions/ | Mention | Cluster | % of singleton
---|---|---|---|---|---|---
Dataset | Train | Dev. | Test | Doc | Doc | length | size | mentions
OntoNotes |  12802 | 343 | 348 |  1467 |  156 | 2.3 | 4.4 |  10.0
LitBankk |  1180 |  110 | 10 | 2105 | 291 | 2.0 | 3.7 | 19.8
PreCo | 36120 | 500 | 500 |  1337 | 105 | 2.7 | 1.6 | 52.0
Character Identification |  11987 | 122 | 192 |  1262 |  136 | 1.0 | 5.1 |  16.4
WikiCoref |  11110 |  110 | 30 | 1996 | 230 | 2.6 | 5.0 |  10.0
Quiz Bowl Coreferencek |  11110 |  110 | 400 |  1126 |  124 | 2.7 | 2.0 | 26.0
Gendered Ambiguous Pronouns p |  12000 | 400 | 2000 |  1195 |  113 | 2.0 | - | -
Winograd Schema Challengep |  11110 |  110 | 271 |  1116 |  113 | 1.5 | - | -
Table 1: Statistics of datasets. Datasets with k indicate that prior work uses
k-fold cross-validation; we record the splits used in this work. Datasets with
p are partially annotated, so we do not include cluster details.

Furthermore, while there are datasets aimed at capturing a broad set of genres
Weischedel et al. (2013); Poesio et al. (2018); Zhu et al. (2021), they are
not mutually compatible due to differences in annotation guidelines. For
example, some datasets do not annotate singleton clusters (clusters with a
single mention). Ideally, we would like a coreference model to be robust to
all the standard datasets. In this work, we consolidate 8 datasets spanning
multiple domains, document lengths, and annotation guidelines. We use them to
evaluate the off-the-shelf performance of models trained on a single dataset.
While they perform well within-domain (e.g., a new state-of-the-art of 79.3 F1
on LitBank), they still perform poorly out-of-domain.

To address poor out-of-domain performance, we propose joint training for
coreference resolution, which is challenging due to the incompatible training
procedues for different datasets. Among other things, we need to address
(unannotated) singleton clusters, as OntoNotes does not include singleton
annotations. We propose a data augmentation process to add predicted
singletons, or pseudo-singletons, into the training data to match the other
datasets which have gold singleton annotations.

Concretely, we contribute a benchmark for coreference to highlight the
disparity in model performance and track generalization. We find joint
training highly effective and show that including more datasets is almost
“free”, as performance on any single dataset is only minimally affected by
joint training. We find that our data augmentation method of adding pseudo-
singletons is also effective. With all of these extensions, we increase the
macro average F1 across all datasets by 9.5 points and achieve a new state-of-
the-art on LitBank and WikiCoref.

##  2 Datasets

We organize our datasets into three types. Training datasets (Sec. 2.1) are
large in terms of number of tokens and clusters and more suitable for
training. Evaluation datasets (Sec. 2.2) are out-of-domain compared to our
training sets and are entirely held out. Analysis datasets (Sec. 2.3) contain
annotations aimed at probing specific phenomena. Table 1 lists the full
statistics.

###  2.1 Training Datasets

#### OntoNotes 5.0 (ON)

Weischedel et al. (2013) is a collection of news-like, web, and religious
texts spanning seven distinct genres. Some genres are transcripts (phone
conversations and news). As the primary training and evaluation set for
developing coreference resolution models, many features specific to this
corpus are tightly integrated into publicly released models. For example, the
metadata includes information on the document genre and the speaker of every
token (for spoken transcripts). Notably, it does not contain singleton
annotations.

#### LitBank (LB)

Bamman et al. (2020) is a set of public domain works of literature drawn from
Project Gutenberg. On average, coreference in the first 2,000 tokens of each
work is fully annotated for six entity types.222They are people, facilities,
locations, geopolitical entities, organizations, and vehicles. We only use the
first cross-validation fold of LitBank, which we call LB0.

#### PreCo (PC)

Chen et al. (2018) contains documents from reading comprehension examinations,
each fully annotated for coreference resolution. Notably, the corpus is the
largest such dataset released.

###  2.2 Evaluation Datasets

#### Character Identification (CI)

Zhou and Choi (2018) has multiparty conversations derived from TV show
transcripts. Each scene in an episode is considered a separate document. This
character-centric dataset only annotates mentions of people.

#### WikiCoref (WC)

Ghaddar and Langlais (2016) contains documents from English Wikipedia. This
corpus contains sampled and annotated documents of different lengths, from 209
to 9,869 tokens.

#### Quiz Bowl Coreference (QBC)

Guha et al. (2015) contains questions from Quiz Bowl, a trivia competition.
These paragraph-long questions are dense with entities. Only certain entity
types (titles, authors, characters, and answers) are annotated.

Model | Training | ON | LB0 | PC | CI | WC | QBC | GAP | WSC | Macro Avg.
---|---|---|---|---|---|---|---|---|---|---
longdoc | ON | 79.0 | 54.8 | 44.3 | 49.8 | 59.6 | 36.8 | 88.9 | 59.8 | 59.1
longdoc S | ON | 79.6 | 54.6 | 44.0 | 58.7 | 60.1 | 36.4 | 89.8 | 59.4 | 60.3
longdoc S, G | ON | 79.5 | 54.7 | 44.5 | 59.5 | 59.9 | 37.0 | 89.0 | 58.7 | 60.3
longdoc S | ON + PS 60K | 80.6 | 56.6 | 49.1 | 55.6 | 62.1 | 40.1 | 89.3 | 61.3 | 61.8
longdoc | LB0 | 56.6 | 77.2 | 46.8 | 53.3 | 47.5 | 50.5 | 85.3 | 32.8 | 56.3
longdoc | PC | 58.8 | 50.3 | 87.8 | 39.5 | 50.7 | 46.5 | 87.3 | 62.7 | 60.5
longdoc S | Joint | 79.2 | 78.2 | 87.6 | 59.4 | 60.3 | 42.9 | 88.6 | 60.1 | 69.5
longdoc S | Joint + PS 30K | 79.6 | 78.2 | 87.5 | 58.4 | 62.5 | 45.5 | 88.7 | 59.4 | 70.0
Table 2: Performance of each model on 8 datasets measured by CoNLL F1 Pradhan
et al. (2012), except for GAP (F1) and WSC (accuracy). Some models use speaker
(S) features, genre (G) features, or pseudo-singletons (PS).

###  2.3 Analysis Datasets

#### Gendered Ambiguous Pronouns (GAP)

Webster et al. (2018) is a corpus of ambiguous pronoun-name pairs derived from
Wikipedia. While only pronoun-name pairs are annotated, they are provided
alongside their full-document context. This corpus has been previously used to
study gender bias in coreference resolution systems.

#### Winograd Schema Challenge (WSC)

Levesque et al. (2012) is a challenge dataset for measuring common sense in AI
systems.333https://cs.nyu.edu/~davise/papers/WinogradSchemas/WSCollection.html
Unlike the other datasets, each document contains one or two sentences with a
multiple-choice question. We manually align the multiple choices to the text
and remove 2 of the 273 examples due to plurals.

##  3 Models

###  3.1 Baselines

We first evaluate a recent system Xu and Choi (2020) which extends a mention-
ranking model Lee et al. (2018) by making modifications in the decoding step.
We find disappointing out-of-domain performance and difficulties with longer
documents present in LB0 and WC (Appendix B.1). To overcome this issue, we
study the _longdoc_ model by Toshniwal et al. (2020), which is an entity-
ranking model designed for long documents that reported strong results on both
OntoNotes and LitBank.

The original longdoc model uses a pretrained SpanBERT Joshi et al. (2020)
encoder which we replace with Longformer-large Beltagy et al. (2020) as it can
incorporate longer context. We retrain the longdoc model and finetune the
Longformer encoder for each dataset, which proves to be competitive for
coreference.444The model scores 79.5 on OntoNotes and achieves state-of-the-
art on LitBank with 79.3. Details are in Appendix B.2. For OntoNotes we train
with and without the metadata of: (a) genre embedding, and (b) speaker
identity which is introduced as part of the text as in Wu et al. (2020).

###  3.2 Joint Training

With copious amounts of text in OntoNotes, PreCo, and LitBank, we can train a
joint model on the combined dataset. However, this is impractical as the
annotation guidelines between the datasets are misaligned (OntoNotes does not
annotate singletons and uses metadata) and because there are substantially
more documents in PreCo.

#### Augmenting Singletons

Since OntoNotes does not annotate for singletons, our training objective for
OntoNotes is different from that of PreCo and LitBank. To address this, we
introduce pseudo-singletons that are silver mentions derived from first
training a mention detector on OntoNotes and selecting the top-scoring
mentions outside the gold mentions.555This mention detector is architecturally
the first half of the longdoc model. We experiment with adding 30K, 60K, and
90K pseudo-singletons (in total, there are 156K gold mentions). We find adding
60K to be the best fit for OntoNotes-only training, and 30K is the best for
joint training (Appendix B.3).

#### Data Imbalance

PreCo has 36K training documents, compared to 2.8K and 80 training documents
for OntoNotes and LitBank respectively. A naive dataset-agnostic sampling
strategy would mostly sample PreCo documents. To address this issue, we
downsample OntoNotes and PreCo to 1K documents per epoch. Downsampling to 0.5K
documents per epoch led to slightly worse performance (Appendix B.4).

#### Metadata Embeddings

For the joint model to be applicable to unknown domains, we avoid using any
domain or dataset-identity embeddings, including the OntoNotes genre
embedding. We do make use of speaker identity in the joint model because: (a)
this is possible to obtain in conversational and dialog data, and (b) it does
not affect other datasets that are known to be single-speaker at test time.

Dataset |  Instance
---|---
(1) QBC |  (This poem) is often considered the counterpart of another poem …name this poem about a creature “burning bright, in the forests of the night," …
(2) QBC |  This author’s non fiction works …another work, a plague strikes secluded valley where teenage boys have been evacuated …name this author of Nip the Buds, Shoot the Kids …
(3) QBC |  This poet of “(I) felt a Funeral in (my) Brain" and “I’m Nobody, Who are you?" wrote about a speaker who hears a Blue, uncertain, stumbling buzz before expiring in “(I) heard a fly buzz when (I) died". For 10 points, name this female American poet of Because (I) could not stop for Death.
(4) CI |  Chandler Bing: Okay, I don’t sound like that. (That) is so not true. (That) is so not … (That) is so not … That … Oh , shut up !
Table 3: Joint + PS 30K error analysis for zero-shot evaluation sets. Each row
highlights one cluster where spans in parenthesis are predicted by the model
while the blue-colored spans represent ground truth annotations. Thus, in (2)
the model misses out on the ground truth cluster entirely while in (3) and (4)
the model predicts an additional cluster.

##  4 Results

Table 2 shows the results for all our models on all 8 datasets. We report each
dataset’s associated metric (e.g., CoNLL F1) and a macro average across all
eight datasets to compare generalizability.

Among the longdoc baseline models trained on one of OntoNotes, PreCo, or
LitBank, we observe a sharp drop in out-of-domain evaluations. The LitBank
model is generally substantially worse than the models trained on OntoNotes
and PreCo, likely due to both a smaller training set and a larger domain
shift. Interestingly, the LitBank model performs the best among all models on
QBC, which can be attributed to both LB and QBC being restricted to a similar
set of markable entity types. Meanwhile, all OntoNotes-only models perform
well on WC and GAP, possibly due to the more diverse set of genres within ON
and because WC also does not contain singletons.

For models trained on OntoNotes, we find that the addition of speaker tokens
leads to an almost 9 point increase on CI, which is a conversational dataset,
but has little impact for non-conversational evaluations. Surprisingly, the
addition of genre embeddings has almost no impact on the overall
evaluation.666In fact, we find that for the model trained with genre
embeddings, modifying the genre value during inference has almost no impact on
the final performance. Finally, the addition of pseudo-singletons leads to
consistent significant gains across almost all the evaluations, including
OntoNotes.

The joint models, which are trained on a combination of OntoNotes, LitBank,
and PreCo, suffer only a small drop in performance on OntoNotes and PreCo, and
achieve the best performance for LitBank. Like the results observed when
training with only OntoNotes, we see a significant performance gain with
pseudo-singletons in joint training as well, which justifies our intuition
that they can bridge the annotation gap. The “Joint + PS 30K” model also
achieves the state of the art for WC.

##  5 Analysis

#### Impact of Singletons

Singletons are known to artificially boost the coreference metrics Kübler and
Zhekova (2011), and their utility for downstream applications is arguable. To
determine the impact of singletons on final scores, we present separate
results for singleton and non-singleton clusters in QBC in Table 4. For non-
singleton clusters we use the standard CoNLL F1 but for singleton clusters the
CoNLL score is undefined, and hence, we use the vanilla F1-score.

The poor performance of ON models for singletons is expected, as singletons
are not seen during training. Adding pseudo-singletons improves the
performance of both the ON and the Joint model for singletons. Interestingly,
adding pseudo-singletons also leads to a small improvement for non-singleton
clusters.

The PC model has the best performance for non-singleton clusters while the LB0
model, which performs the best in the overall evaluation, has the worst
performance for non-singleton clusters. This means that the gains for the LB0
model can be all but attributed to the superior mention detection performance
which can be explained by the fact that both LB and QBC are restricted to a
similar set of markable entity types.

#### Impact of Domain Shift

Table 3 presents instances where the Joint + PS 30K model makes mistakes. In
examples (1) and (2), the model misses out on mentions referring to literary
works which is because references to literary texts are rare in the joint
training data. Example (2) also requires world knowledge to make a connection
between the description of the work and its title. In example (3) the model
introduces an extraneous cluster consisting of first person pronouns mentioned
in titles of different works. The model lacks the domain knowledge that
narrators across different works are not necessarily related. Apart from the
language shift, there are annotation differences across datasets as well. For
example (4) drawn from CI, the model predicts a valid cluster (for Chandler
Bing’s speaking style) according to the ON annotation guidelines but the CI
dataset doesn’t annotate such clusters.

Data | Singleton | Non-singleton | Overall
---|---|---|---
ON |  10.4 | 43.9 | 36.4
ON + PS 60K | 14.4 | 44.4 | 40.1
LB0 | 44.9 | 41.2 | 50.5
PC | 28.8 | 50.3 | 46.5
Joint | 21.7 | 47.3 | 42.9
Joint + PS 30K | 26.7 | 48.6 | 45.5
Table 4: Performance on singleton and non-singleton clusters for QBC.
ON=longdocS and PS=pseudo-singletons.

##  6 Related work

Joint training is commonly used in NLP for training robust models, usually
aided by learning dataset, language, or domain embeddings (e.g., Stymne et al.
(2018) for parsing; Kobus et al. (2017); Tan et al. (2019) for machine
translation). This is essentially what models for OntoNotes already do with
genre embeddings Lee et al. (2017). Unlike prior work, our test domains are
unseen, so we cannot learn test-domain embeddings.

For coreference resolution, Aralikatte et al. (2019) augment annotations using
relation extraction systems to better incorporate world knowledge, a step
towards generalization. Subramanian and Roth (2019) use adversarial training
to target names, with improvements on GAP. Moosavi and Strube (2018)
incorporate linguistic features to improve generalization to WC. Recently, Zhu
et al. (2021) proposed the OntoGUM dataset which consists of multiple genres.
However, compared to the datasets used in our work, OntoGUM is much smaller,
and is also restricted to a single annotation scheme. To the best of our
knowledge, our work is the first to evaluate generalization at scale.

Missing singletons in OntoNotes has been previously addressed through new data
annotations, leading to the creation of the ARRAU Poesio et al. (2018) and
PreCo Chen et al. (2018) corpora. While we include PreCo in this work, ARRAU
contains additional challenges, like split-antecedents, that further increase
the heterogeneity, and its domain overlaps with OntoNotes. Pipeline models for
coreference resolution that first detect mentions naturally leave behind
unclustered mentions as singletons, although understanding singletons can also
improve performance Recasens et al. (2013).

Recent end-to-end neural models have been evaluated on OntoNotes, and
therefore conflate “not a mention” with “is a singleton” Lee et al. (2017,
2018); Kantor and Globerson (2019); Wu et al. (2020). For datasets with
singletons, this has been addressed explicitly through a cluster-based model
Toshniwal et al. (2020); Yu et al. (2020). For those without, they can be
implicitly accounted for with auxiliary objectives Zhang et al. (2018);
Swayamdipta et al. (2018). We go one step further by augmenting with pseudo-
singletons, so that the training objective is identical regardless of whether
the training set contains annotated singletons.

##  7 Conclusion

Our eight-dataset benchmark highlights disparities in coreference resolution
model performance and tracks cross-domain generalization. Our work begins to
address cross-domain gaps, first by handling differences in singleton
annotation via data augmentation with pseudo-singletons, and second by
training a single model jointly on multiple datasets. This approach produces
promising improvements in generalization, as well as new state-of-the-art
results on multiple datasets. We hope that future work will continue to use
this benchmark to measure progress towards truly general-purpose coreference
resolution.

## Acknowledgements

This material is based upon work supported by the National Science Foundation
under Award No. 1941178.

## References

  * Aktaş et al. (2020) Berfin Aktaş, Veronika Solopova, Annalena Kohnert, and Manfred Stede. 2020\.  Adapting Coreference Resolution to Twitter Conversations.  In _Findings of EMNLP_.
  * Aralikatte et al. (2019) Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Herschcovich, Chen Qiu, Anders Sandholm, Michael Ringaard, and Anders Søgaard. 2019.  Rewarding Coreference Resolvers for Being Consistent with World Knowledge.  In _EMNLP-IJCNLP_.
  * Bamman et al. (2020) David Bamman, Olivia Lewke, and Anya Mansoor. 2020.  An Annotated Dataset of Coreference in English Literature.  In _LREC_.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020.  Longformer: The Long-Document Transformer.  _arXiv:2004.05150_.
  * Chen et al. (2018) Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018.  PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution.  In _EMNLP_.
  * Dhingra et al. (2018) Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2018\.  Neural Models for Reasoning over Multiple Mentions Using Coreference.  In _NAACL-HLT_.
  * Gao et al. (2019) Yifan Gao, Piji Li, Irwin King, and Michael R. Lyu. 2019.  Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling.  In _ACL_.
  * Ghaddar and Langlais (2016) Abbas Ghaddar and Phillippe Langlais. 2016.  WikiCoref: An English Coreference-annotated Corpus of Wikipedia Articles.  In _LREC_.
  * Guha et al. (2015) Anupam Guha, Mohit Iyyer, Danny Bouman, and Jordan Boyd-Graber. 2015.  Removing the Training Wheels: A Coreference Dataset that Entertains Humans and Challenges Computers.  In _NAACL-HLT_.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020.  SpanBERT: Improving Pre-training by Representing and Predicting Spans.  _TACL_ , 8.
  * Kantor and Globerson (2019) Ben Kantor and Amir Globerson. 2019.  Coreference Resolution with Entity Equalization.  In _ACL_.
  * Kobus et al. (2017) Catherine Kobus, Josep Crego, and Jean Senellart. 2017.  Domain Control for Neural Machine Translation.  In _RANLP_.
  * Kübler and Zhekova (2011) Sandra Kübler and Desislava Zhekova. 2011.  Singletons and Coreference Resolution Evaluation.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing 2011_.
  * Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017.  End-to-end Neural Coreference Resolution.  In _EMNLP_.
  * Lee et al. (2018) Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018.  Higher-Order Coreference Resolution with Coarse-to-Fine Inference.  In _NAACL-HLT_.
  * Levesque et al. (2012) Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012.  The Winograd Schema Challenge.  In _Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning_.
  * Li et al. (2020) Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, and Marjorie Freedman. 2020.  GAIA: A Fine-grained Multimedia Knowledge Extraction System.  In _ACL: System Demonstrations_.
  * Moosavi and Strube (2018) Nafise Sadat Moosavi and Michael Strube. 2018.  Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers.  In _EMNLP_.
  * Poesio et al. (2018) Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu, and Heike Zinsmeister. 2018.  Anaphora Resolution with the ARRAU Corpus.  In _Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference_.
  * Poot and van Cranenburgh (2020) Corbèn Poot and Andreas van Cranenburgh. 2020.  A Benchmark of Rule-Based and Neural Coreference Resolution in Dutch Novels and News.  In _CRAC_.
  * Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012.  CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes.  In _Joint Conference on EMNLP and CoNLL - Shared Task_.
  * Recasens et al. (2013) Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013.  The Life and Death of Discourse Entities: Identifying Singleton Mentions.  In _NAACL-HLT_.
  * Stymne et al. (2018) Sara Stymne, Miryam de Lhoneux, Aaron Smith, and Joakim Nivre. 2018.  Parser Training with Heterogeneous Treebanks.  In _ACL_.
  * Subramanian and Roth (2019) Sanjay Subramanian and Dan Roth. 2019.  Improving Generalization in Coreference Resolution via Adversarial Training.  In _Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics_.
  * Swayamdipta et al. (2018) Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018.  Syntactic Scaffolds for Semantic Structures.  In _EMNLP_.
  * Tan et al. (2019) Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, and Tie-Yan Liu. 2019.  Multilingual Neural Machine Translation with Language Clustering.  In _EMNLP-IJCNLP_.
  * Toshniwal et al. (2020) Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020.  Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks.  In _EMNLP_.
  * Webster et al. (2018) Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018.  Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns.  _TACL_ , 6.
  * Weischedel et al. (2013) Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013.  OntoNotes Release 5.0.  _Linguistic Data Consortium_.
  * Wiseman et al. (2016) Sam Wiseman, Alexander M. Rush, and Stuart Shieber. 2016.  Antecedent Prediction Without a Pipeline.  In _Proceedings of the Workshop on Coreference Resolution Beyond OntoNotes (CORBON)_.
  * Wu et al. (2020) Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020.  CorefQA: Coreference Resolution as Query-based Span Prediction.  In _ACL_.
  * Xia et al. (2020) Patrick Xia, João Sedoc, and Benjamin Van Durme. 2020.  Incremental Neural Coreference Resolution in Constant Memory.  In _EMNLP_.
  * Xu and Choi (2020) Liyan Xu and Jinho D. Choi. 2020.  Revealing the Myth of Higher-Order Inference in Coreference Resolution.  In _EMNLP_.
  * Yang et al. (2012) Jian Bo Yang, Qi Mao, Qiao Liang Xiang, Ivor Wai-Hung Tsang, Kian Ming Adam Chai, and Hai Leong Chieu. 2012.  Domain Adaptation for Coreference Resolution: An Adaptive Ensemble Approach.  In _EMNLP_.
  * Yu et al. (2020) Juntao Yu, Alexandra Uma, and Massimo Poesio. 2020.  A Cluster Ranking Model for Full Anaphora Resolution.  In _LREC_.
  * Zhang et al. (2018) Rui Zhang, Cícero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir Radev. 2018.  Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering.  In _ACL_.
  * Zhao and Ng (2014) Shanheng Zhao and Hwee Tou Ng. 2014.  Domain Adaptation with Active Learning for Coreference Resolution.  In _Proceedings of the 5th International Workshop on Health Text Mining and Information Analysis (Louhi)_.
  * Zhou and Choi (2018) Ethan Zhou and Jinho D. Choi. 2018.  They Exist! Introducing Plural Mentions to Coreference Resolution and Entity Linking.  In _Proceedings of the 27th International Conference on Computational Linguistics_.
  * Zhu et al. (2021) Yilun Zhu, Sameer Pradhan, and Amir Zeldes. 2021.  OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres.  In _ACL_.

##  Appendix A Model and Training Details

###  A.1 Model

Our model follows the typical coreference pipeline of encoding the document,
followed by mention proposal, and finally mention clustering. The model is
architecturally the same as Toshniwal et al. (2020), and so we re-present
their formulation throughout this section. However, we use the Longformer
encoder as it accommodates longer documents. Otherwise, the model is identical
to Toshniwal et al. (2020) in terms of model size and weight dimensions. We
next explain the mention proposal and mention clustering stages briefly.

#### Mention Proposal

Given a document $\mathcal{D}$, we score all mentions of length $\leq 20$
subword tokens and choose the $K=0.4\times|\mathcal{D}|$ top spans among them.
This is an initial pruning step that speeds up the model and reduces memory
usage. Let $X(K)=\\{(x_{i})_{i=1}^{K}\\}$ represent the top-$K$ candidate
mention spans and $s_{m}(x_{i})$ be a learned scoring function for span
$x_{i}$, which represents how likely a span is an entity mention. $s_{m}$ is
trained to assign positive score to gold mentions (any mention in a gold
cluster), and negative score otherwise. The training objective only uses spans
in $X(K)$, i.e. loss is computed after pruning. During inference, we can
therefore further prune down to $\\{x_{i}:x_{i}\in X(K),s_{m}(x_{i})\geq
0\\}$, which we then pass into the clustering step. During training, we use
teacher forcing and only pass gold mentions among the top-$K$ mentions to the
clustering step.

#### Mention Clustering

The entity-based model tracks $M$ entities (initially $M=0$). Let
$E=(e_{m})_{m=1}^{M}$ represent the $M$ entities. For ease of notation, we
will overload the terms $x_{i}$ and $e_{j}$ to also correspond to their
respective representations.

The model decides whether the span $x_{i}$ refers to any of the entities in
$E$ as follows:

$\displaystyle s_{c}(x_{i},e_{j})$
$\displaystyle\\!=\\!f_{c}([x_{i};e_{j};x_{i}\odot e_{j};g(x_{i},e_{j})])$
$\displaystyle s_{c}^{\mathit{top}}$ $\displaystyle\\!=\\!\max_{j=1\dotsc
M}s_{c}(x_{i},e_{j})$ $\displaystyle e^{\mathit{top}}$
$\displaystyle\\!=\\!\underset{{j=1\dotsc M}}{\arg\max}\;s_{c}(x_{i},e_{j})$

where $\odot$ represents the element-wise product, and $f_{c}(\cdot)$
corresponds to a learned feedforward neural network. The term $g(x_{i},e_{j})$
corresponds to a concatenation of feature embeddings that includes embeddings
for (a) number of mentions in $e_{j}$, and (b) number of tokens between
$x_{i}$ and last mention of $e_{j}$. If $s_{c}^{\mathit{top}}>0$ then $x_{i}$
is considered to refer to $e^{\mathit{top}}$, and $e^{\mathit{top}}$ is
updated accordingly.777We use weighted averaging where the weight for
$e^{\mathit{top}}$ corresponds to the number of previous mentions seen for
$e^{\mathit{top}}$. Otherwise, we initiate a new cluster: $E=E\cup x_{i}$.
During training, we use teacher-forcing i.e. the clustering decisions are
based on ground truth.

###  A.2 Training

We train all the models for 100K gradient steps with a batch size of 1
document. Only the LB-only models are trained for 8K gradient steps which
corresponds to 100 epochs for LB. The models are evaluated a total of 20 times
(every 5K training steps) for all models except the LB-only models which are
evaluated every 400 steps. We use early stopping and a patience of 5 i.e.
training stops if the validation performance doesn’t improve for 5 consecutive
evaluations.

We use the full context size of 4096 tokens for Longformer-large. All training
documents used in this work except 1 ON document fit in a single context
window. For optimizer, we use AdamW with a weight decay of 0.01 and initial
learning rate of 1e-5 for the Longformer encoder, and Adam with an initial
rate of 3e-4 for the rest of the model parameters. The learning rate is
linearly decayed throughout the training.

Training | Evaluation
---|---
Num ON | Num PC | Num PS | ON | LB | PC | WC | QB
1500 |  1500 |  110 | 79.4 | 78.8 | 85.0 | 60.8 | 45.3
1500 |  1500 | 30K | 79.4 | 79.5 | 84.8 | 61.2 | 47.7
1000 | 1000 |  10 | 79.7 | 79.4 | 85.1 | 60.3 | 42.9
1000 | 1000 | 30K | 79.5 | 78.7 | 85.1 | 62.5 | 45.5
1000 | 1000 | 60K | 78.9 | 77.4 | 85.1 | 61.3 | 46.6
1000 | 1000 | 90K | 78.5 | 77.7 | 85.1 | 60.7 | 47.4
Table 5: Validation set scores of datasets when downsampling OntoNotes (ON) and PreCo (PC) in joint training. | Training | ON | LB | PC | CI | WC | QBC | Macro Avg.
---|---|---|---|---|---|---|---|---
longdoc | ON | 88.8 | 62.8 | 42.6 | 65.7 | 72.1 | 53.1 | 64.2
longdoc S | ON | 89.2 | 61.7 | 41.8 | 75.5 | 69.9 | 52.5 | 65.1
longdoc S, G | ON | 89.4 | 62.8 | 42.7 | 77.0 | 72.5 | 52.9 | 66.2
longdoc S | ON + PS 60K | 87.7 | 81.5 | 81.0 | 76.1 | 71.6 | 70.4 | 78.0
longdoc | LB0 | 77.6 | 85.8 | 81.4 | 67.6 | 69.8 | 70.2 | 75.4
longdoc | PC | 76.8 | 81.1 | 93.6 | 66.5 | 67.0 | 77.3 | 77.0
longdoc S | Joint | 90.9 | 86.9 | 93.4 | 77.2 | 76.6 | 68.7 | 82.3
longdoc S | Joint + PS 30K | 89.4 | 87.1 | 93.3 | 77.3 | 74.6 | 71.7 | 82.2
Table 6: Result of all the models with gold mentions. Some models use speaker
(s) features, genre (g) features, or pseudo-singletons (PS). The metric for
the training and evaluation datasets is CoNLL F-score. We skip the analysis
datasets because they lack the set of true gold mentions.

##  Appendix B Other Results

###  B.1 Xu and Choi (2020) Baselines

We run the off-the-shelf model on the test sets of ON, LB0, PC, and QBC. LB0
requires a 24GB GPU, while WC runs out of memory even on that hardware. The
model shows strong in-domain performance with 80.2 on ON. However, out-of-
domain performance is weak: 57.2 on LB0, 49.3 on PC, and 37.6 on QBC. These
are roughly on par with the ON longdoc models.

###  B.2 LitBank Cross-Validation Results

Table 7 presents the results for all the cross-validation splits of LitBank.
The overall performance of 79.3 CoNLL F1 is state of the art for LitBank,
outperforming the previous state of the art of 76.5 by Toshniwal et al.
(2020). Note that in this work, the joint model outperformed (78.2 vs. 77.2)
this baseline model on split 0 (LB0). However, training 10 joint models
contradicts the purpose of this work, which is to create a single,
generalizable model. Realistically, we recommend jointly training with the
entirely of LitBank.

Cross-val split | Dev | Test
---|---|---
0 | 78.8 | 77.2
1 | 78.6 | 80.3
2 | 81.1 | 78.7
3 | 79.0 | 79.1
4 | 80.0 | 78.7
5 | 79.7 | 78.7
6 | 77.7 | 80.7
7 | 81.9 | 79.1
8 | 78.0 | 80.8
9 | 81.8 | 78.7
Total | 79.7 | 79.3
Table 7: LitBank cross-validation results.

###  B.3 Singleton Results for OntoNotes

Num PS | Val. | Test
---|---|---
110 | 79.9 | 79.6
30K | 79.9 | 80.5
60K | 80.0 | 80.6
90K | 79.9 | 80.0
Table 8: Validation and Test results for ON-only model trained on different
amount of pseduo-singletons (PS).

For ON-only models, we tune over the number of pseudo-singletons sampled among
{30K, 60K, 90K}. Table 8 shows that 60K pseudo-singletons is the best choice
based on validation set results on ON.

###  B.4 Downsampling and Singleton Results for Joint

In preliminary experiments, we sample 500 docs from ON and PC. Table 5 shows
the results, confirming that 1K is slightly better than 500. Using more
examples (e.g. 5K PC) begins to hurt performance on LB, likely due to data
imbalance.

For the 1K downsampling setting, we tune over the number of pseudo-singletons
sampled among {30K, 60K, 90K}. We find 30K to be the best choice based on
validation set results.

###  B.5 Results with Gold Mentions

In Table 6, we report the results with gold mentions for the training and
evaluation sets. The analysis sets are skipped as they are partially
annotated. We find that joint training is also helpful in this setting, as
results mirror findings with predicted mentions. In particular, this shows
that it is not just a failure to predict mentions that is preventing ON from
performing well on LB, PC, and QBC.

##  Appendix C Compute Resources

Given that we are finetuning the Longformer model and using a maximum context
size of 4096 tokens, the memory requirements of the model are quite large even
though the cluster-ranking paradigm is considered memory efficient Xia et al.
(2020). We were able to train the PreCo-only model on a 12 GB GPU in 20 hrs
(even the longest PreCo documents are shorter than 2048 tokens with the
Longformer tokenization). All other models were trained over GPUs with memory
24GB or higher (Titan RTX and A6000). On an A6000, the LB-only models can be
trained within 4 hrs, the ON-only models within 16 hrs, and the joint models
within 20 hrs.