# A new characterization of Cohen-Macaulay rings

Kamal Bahmanpour and Reza Naghipour∗ Department of Mathematics, Ardabil
branch, Islamic Azad University, P.O. Box 5614633167, Ardabil, Iran; and
School of Mathematic, Institute for studies in Theoretical Physics and
Mathematics (IPM), P.O. Box 19395-5746, Tehran, Iran<EMAIL_ADDRESS>Department of Mathematics, University of Tabriz, Tabriz, Iran; and School of
Mathematic, Institute for studies in Theoretical Physics and Mathematics
(IPM), P.O. Box 19395-5746, Tehran, Iran<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

The purpose of this article is to provide a new characterization of Cohen-
Macaulay local rings. As a consequence we deduce that a local (Noetherian)
ring $R$ is Gorenstein if and only if every parameter ideal of $R$ is
irreducible.

###### Key words and phrases:

Cohen-Macaulay rings, Gorenstein rings, Parameter ideals.

2010 Mathematics Subject Classification: 13H10.
This research was been in part supported by a grant from IPM (No. 91130022)
∗Corresponding author: e-mail<EMAIL_ADDRESS>(Reza Naghipour)

##  1\. Introduction

Let $(R,\operatorname{\mathfrak{m}})$ denote a commutative Noetherian local
ring. It is well-known that $R$ is Gorenstein if and only if $R$ is Cohen-
Macaulay and some ideal generated by a system of parameters (s.o.p) of $R$ is
irreducible. Perhaps less widely known is a nice result of D.G. Northcott and
D. Rees which states that if every ideal generated by a s.o.p of $R$
(henceforth a parameter ideal) is irreducible, then $R$ is Cohen-Macaulay (see
[4, Theorem 1]). Thus, $R$ is Gorenstein if and only if every parameter ideal
is irreducible. The purpose of this note is to establish a new
characterization of Cohen-Macaulayness of $R$. Our main theorem is the
following:

######  Theorem 1.1.

Let $(R,\mathfrak{m})$ be a local (Noetherian) ring. Then the following
conditions are equivalent:

(i) $R$ is a Cohen-Macaulay ring.

(ii) For any parameter ideals $\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$, we have

$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong R/\mathfrak{a}.$

As a consequence we obtain that a local (Noetherian) ring $R$ is Gorenstein if
and only if every parameter ideal of $R$ is irreducible.

One of our tools for proving Theorem 1.1 is the following:

######  Lemma 1.2.

Let $(R,\mathfrak{m})$ be a local (Noetherian) ring.

(i) If $x\in\operatorname{\mathfrak{m}}$, then there exists a non-negative
integer $n$ such that $(x^{n})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)=0$.

(ii) If $\dim\,R=d\geq 1$ and $\Gamma_{\operatorname{\mathfrak{m}}}(R)\neq 0$,
then there exists a system of parameters $x_{1},\dots,x_{d}$ of $R$ such that
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}\,R/(x_{1},\dots,x_{i})$
for all $1\leq i\leq d$.

Throughout this article, $R$ will always be a commutative Noetherian local
ring with maximal ideal $\operatorname{\mathfrak{m}}$ and $\dim R=d$. For any
$R$-module $L$, the submodule $\bigcup_{n\geq
1}(0:_{L}\operatorname{\mathfrak{m}}^{n})$ of $L$ is denoted by
$\Gamma_{\operatorname{\mathfrak{m}}}(L)$. Also, the socle of $L$ is defined
to be $(0:_{L}\operatorname{\mathfrak{m}})=\\{x\in
L|\,\operatorname{\mathfrak{m}}x=0\\}$, and is denoted by ${\rm Soc}_{R}\,L$.
For any unexplained notation and terminology we refer the reader to [1] and
[2].

##  2\. Proof of the main theorem

In order our main result, we prepare the following lemma.

######  Lemma 2.1.

(i) If $x\in\operatorname{\mathfrak{m}}$, then there exists an integer $n\geq
1$ such that $(x^{n})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)=0$.

(ii) If $\dim R=d\geq 1$ and
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}R$, then there exists a
system of parameters $a_{1},\dots,a_{d}$ of $R$ such that
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}R/(a_{1},\dots,a_{i})$
for all $1\leq i\leq d$.

###### Proof.

(i) Since $\Gamma_{\operatorname{\mathfrak{m}}}(R)$ is an Artinian $R$-module,
the descending sequence of the submodules of
$\Gamma_{\operatorname{\mathfrak{m}}}(R)$,

$(x)\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)\supseteq(x^{2})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)\supseteq\cdots$

is stationary, i.e., there exists a positive integer $n$ such that

$(x^{n+i})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)=(x^{n})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R),$

for all integers $i\geq 1$. Now, we have

$(x^{n})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)\subseteq\bigcap_{i\geq
n}(x^{i})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)\subseteq\bigcap_{i\geq
1}(x^{i}),$

and so by Krull’s Intersection Theorem
$(x^{n})\cap\Gamma_{\operatorname{\mathfrak{m}}}(R)=0$, as required.

(ii) Let $I=\Gamma_{\operatorname{\mathfrak{m}}}(R)$. Since
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}\,R$, it follows that
$I\neq 0$. Now let $b_{1},\dots,b_{d}$ be a system of parameters of $R$. Then
in view of (i) there exists an integer $n_{1}\geq 1$ such that
$(b_{1}^{n_{1}})\cap I=0$. Set $a_{1}:=b_{1}^{n_{1}}$. Then
$I+(a_{1})/(a_{1})\neq 0$ and
$I+(a_{1})/(a_{1})\subseteq\Gamma_{\operatorname{\mathfrak{m}}}(R/(a_{1}))$.
It is clear that $a_{1}$ is a part of a system of parameters of $R$. Let
$R_{1}:=R/(a_{1})$ and
$\operatorname{\mathfrak{m}}_{1}:=\operatorname{\mathfrak{m}}/(a_{1})$. Then
$\Gamma_{\operatorname{\mathfrak{m}}}(R_{1})=\Gamma_{\operatorname{\mathfrak{m}}_{1}}(R_{1})\neq
0$ and so $\operatorname{\mathfrak{m}}_{1}\in\operatorname{Ass}_{R_{1}}R_{1}$.
Moreover, the elements $b_{2}+(a_{1}),\dots,b_{d}+(a_{1})$ is a system of
parameters of $R_{1}$. (Note that $\dim\,R/(a_{1})=d-1$.) Therefore, there is
an integer $n_{2}\geq 1$ such that
$b_{2}^{n_{2}}R_{1}\cap\Gamma_{\operatorname{\mathfrak{m}}_{1}}(R_{1})=0$.
Now, we put $a_{2}:=b_{2}^{n_{2}}$. Then
$\Gamma_{\operatorname{\mathfrak{m}}}(R/(a_{1},a_{2}))\neq 0$ and
$a_{1},a_{2}$ is a part of a system of parameters of $R$. Continue in this
way: there exist positive integers $n_{1},\dots,n_{d}$ such that
$a_{1}=b_{1}^{n_{1}},\dots,a_{d}=b_{d}^{n_{d}}$ is a system of parameters of
$R$ and
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}\,R/(a_{1},\dots,a_{i})$
for all $1\leq i\leq d$. ∎

######  Corollary 2.2.

Suppose that for every system of parameters $a_{1},\dots,a_{d}$ of $R$, the
element $a_{d}$ is an $R/(a_{1},\dots,a_{d-1})$-regular. Then
$\operatorname{\mathfrak{m}}\not\in\operatorname{Ass}_{R}R$.

###### Proof.

Suppose that $\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}R$ and look
for a contradiction. To do this, in view of Lemma 2.1, there exists a system
of parameters $x_{1},\dots,x_{d}$ of $R$ such that
$\operatorname{\mathfrak{m}}\in\operatorname{Ass}_{R}\,R/(x_{1},\dots,x_{d-1})$,
and so $x_{d}$ is not an $R/(x_{1},\dots,x_{d-1})$-regular element. With this
contradiction the proof is complete.∎

Now, we are ready to state and prove the main theorem.

######  Theorem 2.3.

The following conditions are equivalent:

(i) $R$ is a Cohen-Macaulay ring.

(ii) For any parameter ideals $\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$, we have

$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong R/\mathfrak{a}.$

###### Proof.

First we show (i)$\Rightarrow$(ii). To this end, let
$\mathfrak{a}=(a_{1},\dots,a_{d})$ and $\mathfrak{b}=(b_{1},\dots,b_{d})$ be
two parameter ideals of $R$ with $\mathfrak{b}\subseteq\mathfrak{a}$. Then, as
$R$ is a Cohen-Macaulay ring of dimension $d$, it follows that
$\\{a_{i}\\}_{i=1}^{d}$ and $\\{b_{i}\\}_{i=1}^{d}$ are two maximal
$R$-sequences in $\mathfrak{a}$. Consequently, in view of Rees’ theorem (see
[1, Lemma 1.2.4]), we have

$\operatorname{Ext}^{d}_{R}(R/\mathfrak{a},R)\cong\operatorname{Hom}_{R/\mathfrak{a}}(R/\mathfrak{a},R/\mathfrak{a})\cong
R/\mathfrak{a},\,\,\,\,\,{\rm and}$
$\operatorname{Ext}^{d}_{R}(R/\mathfrak{a},R)\cong\operatorname{Hom}_{R/\mathfrak{b}}(R/\mathfrak{a},R/\mathfrak{b})=\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b}).$

Therefore, $\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong
R/\mathfrak{a},$ as required.

In order to prove (ii)$\Rightarrow$(i), we use induction on $d$. When $d=1$,
there exists $a\in R$ such that $\sqrt{(a)}=\operatorname{\mathfrak{m}}$.
Then, in view of Lemma 2.1 there is a positive integer $n$ such that
$(a^{n})\cap I=0$, where $I=\Gamma_{\operatorname{\mathfrak{m}}}(R)$.
Moreover, there exists an integer $k$ such that
$\operatorname{\mathfrak{m}}^{k}I=0$, and so $a^{k}I=0$. Let
$r_{1}=\max\\{n,k\\}$ and $b=a^{r_{1}}$. Then
$\sqrt{(b)}=\operatorname{\mathfrak{m}}$, $\Gamma_{(b)}(R)=I$, $bI=0$ and
$(b)\cap I=0$. On the other hand, in view of [3, Proposition 4.7.13], there
exists an integer $r_{2}\geq 1$ such that for all integers $t\geq r_{2}$,

$(b^{t}):_{R}b=b^{t-r_{2}}((b^{r_{2}}):_{R}b)+(0:_{R}b).$

As $(0:_{R}b)=I$, it follows that

$(b^{t}):_{R}b=b^{t-r_{2}}((b^{r_{2}}):_{R}b)+I.$

Now, let $s\geq r_{2}+1$ be an integer. Then in view of assumption (ii) we
have

$R/(b)\cong\operatorname{Hom}_{R}(R/(b),R/(b^{s}))\cong\frac{(b^{s}):_{R}b}{(b^{s})}=\frac{b^{s-r_{2}}((b^{r_{2}}):_{R}b)+I}{(b^{s})}.$

Next, we show that

$b^{s-r_{2}}((b^{r_{2}}):_{R}b)\cap(I+(b^{s}))=(b^{s}).$

To do this, suppose $c\in I$ and $\delta b^{s-r_{2}}\in
b^{s-r_{2}}((b^{r_{2}}):_{R}b)$ such that $\delta b^{s-r_{2}}=c+ub^{s}$, where
$\delta\in(b^{r_{2}}):_{R}b$ and $u\in R$. Then

$c=\delta b^{s-r_{2}}-ub^{s}\in(b)\cap I,$

and so $c=0$. Thus $\delta b^{s-r_{2}}=ub^{s}\in(b^{s})$. Consequently,

$\frac{R}{(b)}\cong\frac{b^{s-r_{2}}((b^{r_{2}}):_{R}b)}{(b^{s})}\oplus\frac{I+(b^{s})}{(b^{s})}.$

Since $R/(b)$ is a local ring, it follows that

$b^{s-r_{2}}((b^{r_{2}}):_{R}b)=(b^{s})\,\,\,\,\,{\rm
or}\,\,\,\,\,I+(b^{s})=(b^{s}).$

If $b^{s-r_{2}}((b^{r_{2}}):_{R}b)=(b^{s})$, then as $b^{s-1}\in
b^{s-r_{2}}((b^{r_{2}}):_{R}b)$, it follows that $(b^{s-1})=(b^{s})$. Hence in
view of NAK’s Lemma, $b^{s-1}=0$, and so
$\operatorname{\mathfrak{m}}=\sqrt{(0)}$, which is a contradiction. Therefore,
$I\subseteq(b^{s})$, and so $I=I\cap(b^{s})\subseteq I\cap(b)=0$. Thus $I=0$,
and so $\operatorname{\mathfrak{m}}\not\in\operatorname{Ass}_{R}R$. Since
$\dim R=1$, it follows that $R$ is a Cohen-Macaulay ring.

Assume, inductively, that $d\geq 2$ and that the result has been proved for
$d-1$. Let $x_{1},\dots,x_{d}$ be an arbitrary system of parameters for $R$.
For each integer $n\geq 1$, set

$\mathfrak{a}:=(x_{1},\dots,x_{d})\,\,\,\,\,{\rm
and}\,\,\,\,\,\mathfrak{b}:=(x_{1},\dots,x_{d-1},x_{d}^{n}).$

Then

$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong R/\mathfrak{a}.$

Let $S=R/(x_{1},\dots,x_{d-1})$. Then

$\operatorname{Hom}_{S}(S/x_{d}S,S/x_{d}^{n}S)\cong\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong
R/\mathfrak{a}\cong S/x_{d}S.$

Now, if $\overline{y}:=y+(x_{1},\dots,x_{d-1})$ is a system of parameters of
$S$, then $x_{1},\dots,x_{d-1},y$ is a system of parameters of $R$, (note that
$\dim S=1$). Hence

$\operatorname{Hom}_{S}(S/(\overline{y}),S/(\overline{y})^{n})\cong
S/(\overline{y}),$

and so applying the method used above it follows that $S$ is a Cohen-Macaulay
ring. Thus $x_{d}$ is an $R/(x_{1},\dots,x_{d-1})$-regular element, and hence
by Corollary 2.2,
$\operatorname{\mathfrak{m}}\not\in\operatorname{Ass}_{R}\,R$. Accordingly,
there exists $\eta_{1}\in\operatorname{\mathfrak{m}}$ such that
$\eta_{1}\not\in Z_{R}(R)$. Therefore, $\eta_{1}$ is a part of a system of
parameters of $R$. Now, let $T=R/(\eta_{1})$ and

$\mathfrak{c}=(\bar{\eta}_{2},\dots,\bar{\eta}_{d}),\,\,\,\,\,{\rm
and}\,\,\,\,\,\mathfrak{d}=(\bar{\xi}_{2},\dots,\bar{\xi}_{d})$

be two parameter ideals of $T$ such that $\mathfrak{d}\subseteq\mathfrak{c}$,
where $\bar{\eta}_{i}=\eta_{i}+(\eta_{1})$ and
$\bar{\xi}_{i}=\xi_{i}+(\eta_{1})$ for every $i=2,\dots,d$. Then

$\mathfrak{a}^{\prime}=(\eta_{1},\dots,\eta_{d})\,\,\,\,\,{\rm
and}\,\,\,\,\,\mathfrak{b}^{\prime}=(\eta_{1},\xi_{2},\dots,\xi_{d})$

are two parameter ideals of $R$. Moreover, we have

$\mathfrak{b}^{\prime}\subseteq\mathfrak{a}^{\prime}\,\,\,\,\,{\rm
and}\,\,\,\,\,\operatorname{Hom}_{R}(R/\mathfrak{a}^{\prime},R/\mathfrak{b}^{\prime})\cong
R/\mathfrak{a}^{\prime}.$

Therefore

$\operatorname{Hom}_{T}(T/\mathfrak{c},T/\mathfrak{d})\cong\operatorname{Hom}_{R}(R/\mathfrak{a}^{\prime},R/\mathfrak{b}^{\prime})\cong
R/\mathfrak{a}^{\prime}\cong T/\mathfrak{c}.$

As, $\dim\,T=d-1$, it follows from the inductive hypothesis that $T$ is a
Cohen-Macaulay ring. Hence $R$ is a Cohen-Macaulay ring, as required. ∎

######  Corollary 2.4.

The following conditions are equivalent:

(i) $R$ is a Cohen-Macaulay ring.

(ii) For any parameter ideals $\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$ the $R$-module
$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})$ is cyclic.

###### Proof.

The result follows from the proof of Theorem 2.3.∎

######  Corollary 2.5.

The following conditions are equivalent:

(i) $R$ is a Gorenstein ring.

(ii) For any parameter ideals $\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$,
$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})\cong R/\mathfrak{a}$
and there exists an irreducible parameter ideal.

(iii) For any parameter ideals $\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$ the $R$-module
$\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})$ is cyclic and there
exists an irreducible parameter ideal.

###### Proof.

The result follows from the proof of Theorem 2.3 and [2, Theorem 18.1].∎

The final result shows that a local (Noetherian) ring $R$ is Gorenstein if and
only if every parameter ideal of $R$ is irreducible.

######  Corollary 2.6.

$R$ is Gorenstein if and only if every parameter ideal of $R$ is irreducible.

###### Proof.

In view of Corollary 2.5, it is enough to show that for any parameter ideals
$\mathfrak{a}$ and $\mathfrak{b}$ of $R$ with
$\mathfrak{b}\subseteq\mathfrak{a}$ we have
$\operatorname{Hom}(R/\mathfrak{a},R/\mathfrak{b})\cong R/\mathfrak{a}$. To do
this end, since $\mathfrak{a}$ and $\mathfrak{b}$ are irreducible it follows
that

$\dim_{R/\operatorname{\mathfrak{m}}}\,{\rm
Soc}_{R}\,R/\mathfrak{a}=1=\dim_{R/\operatorname{\mathfrak{m}}}\,{\rm
Soc}_{R}\,R/\mathfrak{b}.$

In particular, $R/\mathfrak{a}$ and $R/\mathfrak{b}$ are two Gorenstein local
rings of dimension zero. Hence
$E_{R/\mathfrak{a}}(R/\operatorname{\mathfrak{m}})\cong R/\mathfrak{a}$ and
$E_{R/\mathfrak{b}}(R/\operatorname{\mathfrak{m}})\cong R/\mathfrak{b}$.
Therefore,

$\displaystyle\operatorname{Hom}_{R}(R/\mathfrak{a},R/\mathfrak{b})$
$\displaystyle\cong$
$\displaystyle\operatorname{Hom}_{R/\mathfrak{b}}(R/\mathfrak{a},R/\mathfrak{b})$
$\displaystyle\cong$
$\displaystyle\operatorname{Hom}_{R/\mathfrak{b}}(R/\mathfrak{a},E_{R/\mathfrak{b}}(R/\operatorname{\mathfrak{m}}))$
$\displaystyle\cong$ $\displaystyle
E_{R/\mathfrak{a}}(R/\operatorname{\mathfrak{m}})$ $\displaystyle\cong$
$\displaystyle R/\mathfrak{a}.$

This completes the proof.

Acknowledgments

We would like to thank from School of Mathematics, Institute for Research in
Fundamental Sciences (IPM), for its financial support.

## References

  * [1] W. Bruns and J. Herzog, Cohen-Macaulay rings, Cambridge Studies in Advanced Mathematics, Vol. 39, Cambridge Univ. Press, Cambridge, UK, 1998.
  * [2] H. Matsumura, Commutative ring theory, Cambridge Univ. Press, Cambridge, UK, 1986.
  * [3] D.G. Northcott, Lessons on rings, modules and multiplicites, Cambridge Univ. Press, Cambridge, UK, 1968.
  * [4] D.G. Northcott and D. Rees, Principal systems, Quart. J. Math. 8 (1957), 119-127.


11institutetext: University of Information Technology - VNUHCM
22institutetext: Vietnam National University, Ho Chi Minh City, Vietnam
22email<EMAIL_ADDRESS>22email<EMAIL_ADDRESS>
22email<EMAIL_ADDRESS>
# Software Mention Recognition with a Three-Stage Framework Based on BERTology
Models at SOMD 2024

Nguyen Thi Thuy 1122 Nguyen Viet Anh 1122 Dang Van Thin Corresponding author:
<EMAIL_ADDRESS>Ngan Luu-Thuy Nguyen 1122

###### Abstract

This paper describes our systems for the sub-task I in the Software Mention
Detection in Scholarly Publications shared-task. We propose three approaches
leveraging different pre-trained language models (BERT, SciBERT, and XLM-R) to
tackle this challenge. Our best-performing system addresses the named entity
recognition (NER) problem through a three-stage framework. (1) Entity Sentence
Classification - classifies sentences containing potential software mentions;
(2) Entity Extraction - detects mentions within classified sentences; (3)
Entity Type Classification - categorizes detected mentions into specific
software types. Experiments on the official dataset demonstrate that our
three-stage framework achieves competitive performance, surpassing both other
participating teams and our alternative approaches. As a result, our framework
based on the XLM-R-based model achieves a weighted F1-score of 67.80%,
delivering our team the 3rd rank in Sub-task I for the Software Mention
Recognition task. We release our source code at this
repository111https://github.com/thuynguyen2003/NER-Three-Stage-Framework-for-
Software-Mention-Recognition.

###### Keywords:

Software mention recognition Named entity recognition Transformer Three-stage
framework.

##  1 Introduction

Named Entity Recognition (NER) is an important task in NLP that involves
identifying and classifying named entities in text. That will transform them
into structured data, making it easier to categorize and perform search
processing or carry out other NLP tasks [5] on that data such as text
classification, sentiment analysis, and contextual analysis, … particularly in
the domain of Biomedical Named Entity Recognition (Bio-NER), which is
challenged by a range of entities like genes, proteins, medications, and
diseases [9].

The SOMD 2024 shared-task, hosted within Natural Scientific Language
Processing and Research Knowledge Graphs (NSLP 2024) workshop [8], is designed
to extract mentioned software and metadata from documents. In this context,
both the software and the metadata are identified as specific intervals in the
original documents. Understand and identify the software mentioned in
documents, which is especially important to support information extraction in
scientific documents.

In this paper, we present three different approaches to address the challenge
of sub-task I, including:

  * •

Approach 1: Fine-tuning pre-trained language models as a token classification
problem.

  * •

Approach 2: Two-stage framework for entity extraction and classification.

  * •

Apporach 3: Three-stage framework for entity sentence classification, entity
extraction, and entity type classification.

##  2 Related Work

In recent years, pre-training language models (PLMs) have made significant
advancements in Named Entity Recognition (NER) tasks [16]. Among these, the
most popular model is BERT [7] and its variations like SciBERT [2], RoBERT
[3], and BiLSTM [11]. These models are often paired with machine learning
techniques, particularly Conditional Random Fields (CRF) [10]. Additionally,
some approaches involve breaking down the NER task into two simpler tasks
using question-answering methods [1], achieving notable results on various
datasets like BioNLP13CG, CTIReports, OntoNotes5.0 [12], and WNUT17 [6] based
on the F1 measure.

With the emergence of ChatGPT, researchers have been exploring the use of
Large Language Models (LLMs) for NER tasks [15, 17], with some studies
demonstrating that ChatGPT can be distilled into smaller UniversalNER models
for open NER [18]. These UniversalNER models have shown exceptional accuracy
across 43 datasets spanning diverse fields such as biomedicine, programming,
social media, law, and finance, without requiring direct supervision.
UniversalNER surpasses traditional guideline-tuned models like Alpaca and
Vicuna by an average of over 30 F1 points and achieves a high F1 score of 0.8
on SoMeSci. In this paper, BERT, SciBERT, and XML-R models are still utilized
to address the first task of the shared SOMD 2024 challenge.

##  3 Approach

To address the Software Mention Recognition task, we utilize the power of
different pre-trained transformer-based language model in different
approaches. Figure 1 illustrates three approaches to participate in the
competition. Because shared-task is related to each token in the sentence and
whether words are in capital letters or not also greatly affects the
recognition of entities. Therefore, we do not apply any preprocessing
techniques but use data directly from the organizers. Also the tokenize method
will depend on the default tokenier of the models. In our work, we employ
various pre-trained language models, including the XLM-Roberta (XLM-R) [4],
BERT [7], and SciBERT [2] as our main backbones. The detail of our three
approaches are present as follow.

Figure 1: Overview system of three approaches: Sample input is ”Celeste was
written in C #” with two entities are E_1 and E_2. E_1 and E_2 play the role
of two entity types in this example

###  3.1 Approach 1: Token classification with BERTs

For the first approach, we address the task by fine-tuning different
transformer BERT-base models for the token classification task. We adapted
different pre-trained language models to the training dataset. After
tokenizing the input, we feed the token sequence to backbones models to
extract the fixed vector in the last layer as the final representation of the
input sentence. Then, we apply a fully connected layer to process the vectors
and predict labels for each input token using a softmax function. There are a
total of 27 labels (in Table 1), where 26 correspond to 13 different entity
types, and one label represents non-entities. Figure 1 illustrates the
overview of our first approach.

Table 1: List of labels for token classification task in Approach 1

Index | Label | Index | Label
---|---|---|---
1 | B-Application_ Creation | 15 | B-PlugIn_Deposition
2 | I-Application_ Creation | 16 | I-PlugIn_Deposition
3 | B-Application_Deposition | 17 | B-PlugIn_Mention
4 | I-Application_Deposition | 18 | I-PlugIn_Mention
5 | B-Application_Mention | 19 | B-PlugIn_Usage
6 | I-Application_Mention | 20 | I-PlugIn_Usage
7 | B-Application_Usage | 21 | B-ProgrammingEnvironment_Mention
8 | I-Application_Usage | 22 | I-ProgrammingEnvironment_Mention
9 | B-OperatingSystem_Mention | 23 | B-ProgrammingEnvironment_Usage
10 | I-OperatingSystem_Mention | 24 | I-ProgrammingEnvironment_Usage
11 | B-OperatingSystem_Usage | 25 | B-SoftwareCoreference_Deposition
12 | I-OperatingSystem_Usage | 26 | I-SoftwareCoreference_Deposition
13 | B-PlugIn_Creation | 27 | O
14 | I-PlugIn_Creation |  |

###  3.2 Approach 2: Two-stage framework for Entity Extraction and
Classification

Motivated by recent work by [1], we address Task 1 - Software Mention
Recognition with a two-stage framework composed of entity extraction and
entity classification components. However, our components are re-designed to
improve the overall performance than original framework proposed by [1].
Figure 1 illustrates the overview of this approach, the detail of each
component is presented below:

  * •

Stage 1 - Entity extraction: This stage aims to identify whether each token in
a given input sentence belongs to an entity or not. We achieve this through
token classification, similar to Approach 1. However, instead of using 27
labels for different token types, we only use 3 labels as:

    * –

O: Non-entity token

    * –

B-X: Beginning token of an entity of type X (where X represents one of the 13
entity types)

    * –

I-X: Token within an entity of type X

Using separate labels for the beginning (B) and inside (I) positions of tokens
within an entity allows us to efficiently extract all words belonging to the
same entity in stage 2.

  * •

Stage 2 - Entity classification: In this stage, we classify the detected
entities from stage 1. We use a classifier with 13 labels corresponding to the
13 entity types, discarding the B-I prefix distinction used for token
position. This classifier is built by fine-tuning a transformer-based model
like BERT.

[14] During fine-tuning for classification tasks, it’s common practice to use
the hidden state associated with the [CLS] token as input for a classifier.
However, in this approach, we fine-tune the entire transformer model end-to-
end. This means the hidden states are not treated as fixed features, but are
trained alongside the classification head (a component added on top of the
pre-trained model) for optimal performance. Additionally, to leverage the
knowledge of transformer models, we format this classifier as a question-and-
answering model by constructing the input as the following prompt:

    * –

Input: What is $<$entity$>$ in the sentence: $<$input sentence$>$

    * –

Output: Type of entity

###  3.3 Approach 3: Three-stage framework

Our analysis in Table 2 revealed a limited number of sentences containing
entities within the training set. This disparity raised concerns about
potential biases in the label information during the training process for the
previously mentioned approaches. To address this, we introduce a new three-
stage framework, which integrate a binary classification with Approach 2. We
simply built a binary classification model to detect the sentences which
contain the entity. As shown in Figure 1, if a sentence is classified as class
0, assign all tokens in the sentence as O, otherwise, this sentence will be
passed to Approach 2 to extract the entity and its type.

##  4 Experimental Setup

###  4.1 Data and Evaluation Metrics

This shared-task uses the SoMeSci dataset [13] which included 39768 sentences
and 3756 software mentions divided into a training set and a private test set.
We train our systems only on the training set and evaluate the performance of
our model on the private test set using weighted precision, recall, and
F1-score. In Table 2, we summarize some general information about the two data
sets. Where #Sentence denotes the number of sentences, #Sentence with entity
denotes the number of sentences containing the entity, and Total entity is the
total of entities in all sentences. Max length and Avg length are the maximum
length and average length of the sentences in each set, respectively. This
dataset contains six groups of entity Application, OperatingSystem, PlugIn,
ProgrammingEnvironment, and SoftwareConference. Each group can have the entity
belong to four types [Creation, Deposition, Mention, Usage]. In Table 3 we
indicate the distribution of each entity in the dataset

Table 2: General statistics in the training set and private test set Information | Training set | Private test set
---|---|---
#Sentence | 39768 | 8180
#Sentence with entity | 2353 | 374
Total entity | 3241 | 515
Max length | 568 | 347
Avg length | 28.32 | 28.82
Table 3: Statistics the number of entities in each entity type entity in each
entity group in the training set and private test set

Entity group | Entity | Training set | Testing set
---|---|---|---
Quantity | Total | Quantity | Total
Application | Application_Creation | 150 | 2353 | 47 | 348
Application_Deposition | 80 | 22
Application_Mention | 162 | 31
Application_Usage | 1958 | 248
OperatingSystem | OperatingSystem_Mention | 13 | 140 | 17 | 33
OperatingSystem_Usage | 127 | 16
PlugIn | PlugIn_Creation | 53 | 344 | 17 | 81
PlugIn_Deposition | 21 | 8
PlugIn_Mention | 40 | 11
PlugIn_Usage | 230 | 45
ProgrammingEnvironment | ProgrammingEnvironment_Mention | 41 | 372 | 6 | 49
ProgrammingEnvironment_Usage | 331 | 43
SoftwareCoreference | SoftwareCoreference_Deposition | 35 | 35 | 4 | 4

###  4.2 System Settings

We conduct all experiments on three approaches, using three base-version
backbones: XLM-R222https://huggingface.co/FacebookAI/xlm-roberta-base,
BERT333https://huggingface.co/google-bert/bert-base-uncased, and
SciBERT444https://huggingface.co/allenai/scibert-scivocab-uncased. We loaded
the weights of the backbones from the HuggingFace library and carried out
training on an NVIDIA T4(x2) GPU provided by Kaggle. The corresponding hyper-
parameters for each approach are presented below:

  * •

Approach 1:  batch size = 32, learning rate = 5e-05, and the number of epoch =
25 with XLM-R model and the number of epoch = 20 both remain backbones.

  * •

Approach 2:

    * –

Stage 1: batch size = 32, learning rate = 5e-05 and the number of epoch = 20
for all three backbones.

    * –

Stage 2: batch size = 16, learning rate = 2e-05 and the number of epoch = 25
with XLM-R model and epoch = 20 two remainder models.

  * •

Approach 3:

    * –

Stage 1: batch size = 32, learning rate = 2e-5 and the number of epoch = 10
for all three backbones.

    * –

Stage 2 and Stage 3: Using the configuration and architecture as the Approach
2.

##  5 Main results

According to the organizing committee, this sub-task will be evaluated by
F1-Score based on exact matches. As shown in Table 4, we provide a tabulated
summary of 9 experiments, each representing one of the 9 final systems
generated from three different approaches and using three distinct backbones.

The experimental results in Table 4 indicate that Approach 3, a three-stage
system, demonstrates the best performance across all backbones, with the XLM-
RoBERTa backbone exhibiting the highest efficacy among all approaches.
However, this result is for reference only and is only true in all of my
experiments. It’s important to acknowledge that different contexts, set up or
datasets might yield different outcomes, and we are not sure this is the best
result that each backbone could give in other cases. Finally, the best system
was built according to approach 3 with XLM-R backbone and our best submission
was ranked 3rd. Table 6 show the final score of the top 5 participants.

Table 4: Comparative performance of our three Approaches with different pre-
trained language models on the test set.

Models | Approach 1 | Approach 2 | Approach 3
---|---|---|---
Precision | Recall | F1-score | Precision | Recall | F1-score | Precision | Recall | F1-score
BERT | 0.675 | 0.594 | 0.625 | 0.682 | 0.643 | 0.653 | 0.690 | 0.629 | 0.650
SciBERT | 0.658 | 0.621 | 0.623 | 0.719 | 0.645 | 0.670 | 0.736 | 0.631 | 0.670
XLM-R | 0.716 | 0.614 | 0.649 | 0.707 | 0.654 | 0.671 | 0.729 | 0.649 | 0.678

Table 5: Official scoreboard666https://codalab.lisn.upsaclay.fr/competitions/16935#resultsfor the sub-task I: Software mention recognition. Participant | Ranking | Evaluation metrics
---|---|---
Precision | Recall | F1-score
phinx | Top 1 | 0.761 | 0.750 | 0.740
david-s477 | Top 2 | 0.739 | 0.711 | 0.692
ottowg | Top 4 | 0.679 | 0.664 | 0.652
vampire | Top 5 | 0.682 | 0.637 | 0.648
Our best system | Top 3 | 0.729 | 0.649 | 0.678

With the test dataset labels provided by the organizing committee, we
evaluated the performance of our best system for each entity class in Table 6.
We observed that the SoftwareCoreference_Deposition entity achieved the
highest Precision score, while the ProgrammingEnvironment_Usage entity
attained the highest Recall and F1 score, top 5 F1-score classes are
ProgrammingEnvironment_Usage, SoftwareCoreference_Deposition, and
OperatingSystem_Mention. It is evident that entities belonging to the PlugIn
group typically scored lower than those in other groups shows that it has
difficulty in the regconization process. Although, the number of PlugIn_Usage
entities in the training set is pretty large the result on the test set is not
positive. Besides that, PlugIn_Creation and PlugIn_Deposition entities have
the sample in the training set are pretty low and their score moves forward to
zero. The number of OperatingSystem_Mention entities in the training set is
low and the score on the test set is high so we predict the mention entity
type in this group is featured and easier to recognize than other groups.

Additionally, in Table 7, we evaluated each individual stage in our final
three-stage system by assuming that the accuracy of the stages before it is
100%. The first stage works well with an F1-score of 0.992 in classifying
whether a sentence contains an entity or not. Moving to stage 2, tasked with
detecting entities in sentences, achieved an F1 score at a relatively good
level, but a significant difference between Precision and Recall (12.6%
difference) is evident, which also affects the overall system performance. In
the final stage, the scores between the three metrics are relatively balanced,
but it appears that the task of classifying 13 entity classes had some impact
on this stage with relatively lower overall performance. The propagation of
errors between the three stages has a significant impact on the entire system,
with the final F1-score of the entire system being 0.678.

Table 6: Performance of the final system on the test dataset across entity classes evaluated by Precision, Recall, and F1-score. Entity class | Precision | Recall | F1-score
---|---|---|---
Application_Creation | 0.692 | 0.766 | 0.727
Application_Deposition | 0.615 | 0.727 | 0.667
Application_Mention | 0.560 | 0.452 | 0.500
Application_Usage | 0.812 | 0.730 | 0.769
OperatingSystem_Mention | 0.867 | 0.765 | 0.812
OperatingSystem_Usage | 0.579 | 0.688 | 0.629
PlugIn_Creation | 0.200 | 0.059 | 0.091
PlugIn_Deposition | 0.000 | 0.000 | 0.000
PlugIn_Mention | 0.667 | 0.364 | 0.471
PlugIn_Usage | 0.682 | 0.333 | 0.448
ProgrammingEnvironment_Mention | 0.500 | 0.167 | 0.250
ProgrammingEnvironment_Usage | 0.886 | 0.907 | 0.897
SoftwareCoreference_Deposition | 1.000 | 0.750 | 0.857
Table 7: Performance of components in our final three-stage framework. Stage | Precision | Recall | F1-score
---|---|---|---
Stage 1 | 0.992 | 0.992 | 0.992
Stage 2 | 0.912 | 0.786 | 0.845
Stage 3 | 0.786 | 0.806 | 0.784

##  6 Conclusion and Future Work

In this paper, we present and evaluate three approaches for tackling sub-task
I in the Software Mention Detection in Scholarly Publications shared task.
While we explored the use of suitable transformer models like BERT, our three-
stage system leveraging the XLM-R model achieved the highest performance in
the competition. As a result, our best system achieved the Top 3 in the
private test. In future work, our intention is to analyze the error
propagation between the three stages to enhance the performance of the entire
three-stage system. Additionally, with access to more substantial
computational resources, we aim to experiment with fine-tuning sub-tasks using
larger batch sizes and epochs for each backbone in order to investigate the
effects of these hyper-parameters on the model’s performance.

## Acknowledgements

This research was supported by The VNUHCM-University of Information
Technology’s Scientific Research Support Fund. We also thank the anonymous
reviewers for their valuable comments on our manuscript.

## References

  * [1] Arora, J., Park, Y.: Split-NER: Named entity recognition via two question-answering-based classifications. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 416–426. Association for Computational Linguistics, Toronto, Canada (Jul 2023). https://doi.org/10.18653/v1/2023.acl-short.36, https://aclanthology.org/2023.acl-short.36
  * [2] Beltagy, I., Lo, K., Cohan, A.: SciBERT: A pretrained language model for scientific text. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 3615–3620. Association for Computational Linguistics, Hong Kong, China (Nov 2019). https://doi.org/10.18653/v1/D19-1371, https://aclanthology.org/D19-1371
  * [3] Chen, T., Qian, Y., Wang, Y., Chen, X., Ouyang, D., Dong, S., Li, X., Zhao, J., Huang, L.: Robert-agr: An entity relationship extraction model of massive agricultural text based on the roberta and crf algorithm. In: 2023 IEEE 8th International Conference on Big Data Analytics (ICBDA). pp. 113–120. IEEE (2023)
  * [4] Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., Stoyanov, V.: Unsupervised cross-lingual representation learning at scale. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 8440–8451. Association for Computational Linguistics, Online (Jul 2020). https://doi.org/10.18653/v1/2020.acl-main.747, https://aclanthology.org/2020.acl-main.747
  * [5] Dash, A., Darshana, S., Yadav, D.K., Gupta, V.: A clinical named entity recognition model using pretrained word embedding and deep neural networks. Decision Analytics Journal p. 100426 (2024)
  * [6] Derczynski, L., Nichols, E., van Erp, M., Limsopatham, N.: Results of the WNUT2017 shared task on novel and emerging entity recognition. In: Proceedings of the 3rd Workshop on Noisy User-generated Text. pp. 140–147. Association for Computational Linguistics, Copenhagen, Denmark (Sep 2017). https://doi.org/10.18653/v1/W17-4418, https://www.aclweb.org/anthology/W17-4418
  * [7] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019). https://doi.org/10.18653/v1/N19-1423, https://aclanthology.org/N19-1423
  * [8] Elena Simperl, Peter Clark, K.K.: Natural scientific language processing and research knowledge graphs. In: Lecture Notes in Artificial Intelligence (2024)
  * [9] Li, L., Zhou, R., Huang, D.: Two-phase biomedical named entity recognition using crfs. Computational biology and chemistry 33(4), 334–338 (2009)
  * [10] Lopez, P., Du, C., Cohoon, J., Ram, K., Howison, J.: Mining software entities in scientific literature: document-level ner for an extremely imbalance and large-scale task. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. pp. 3986–3995 (2021)
  * [11] Luo, L., Yang, Z., Yang, P., Zhang, Y., Wang, L., Lin, H., Wang, J.: An attention-based bilstm-crf approach to document-level chemical named entity recognition. Bioinformatics 34(8), 1381–1388 (2018)
  * [12] Pradhan, S., Moschitti, A., Xue, N., Ng, H.T., Björkelund, A., Uryupina, O., Zhang, Y., Zhong, Z.: Towards robust linguistic analysis using OntoNotes. In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning. pp. 143–152. Association for Computational Linguistics, Sofia, Bulgaria (Aug 2013), https://aclanthology.org/W13-3516
  * [13] Schindler, D., Bensmann, F., Dietze, S., Krüger, F.: Somesci-a 5 star open data gold standard knowledge graph of software mentions in scientific articles. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. pp. 4574–4583 (2021)
  * [14] Tunstall, L., Werra, L.v., Wolf, T.: Natural language processing with transformers: Building language applications with hugging face. O’Reilly (2022)
  * [15] Wang, S., Sun, X., Li, X., Ouyang, R., Wu, F., Zhang, T., Li, J., Wang, G.: Gpt-ner: Named entity recognition via large language models. arXiv preprint arXiv:2304.10428 (2023)
  * [16] Zhang, H., Li, X., Gu, R., Qu, X., Meng, X., Hu, S., Liu, S.: Samsung research china-beijing at semeval-2023 task 2: An al-r model for multilingual complex named entity recognition. In: Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). pp. 114–120 (2023)
  * [17] Zhang, Z., Zhao, Y., Gao, H., Hu, M.: Linkner: Linking local named entity recognition models to large language models using uncertainty. arXiv preprint arXiv:2402.10573 (2024)
  * [18] Zhou, W., Zhang, S., Gu, Y., Chen, M., Poon, H.: Universalner: Targeted distillation from large language models for open named entity recognition. In: The Twelfth International Conference on Learning Representations (2023)


# BERT-$\beta$: A Proactive Probabilistic Approach to Text Moderation

Fei Tan, Yifan Hu, Kevin Yen, Changwei Hu
Yahoo Research, New York, NY, USA
{fei.tan, yifanhu, kevinyen<EMAIL_ADDRESS>
###### Abstract

Text moderation for user generated content, which helps to promote healthy
interaction among users, has been widely studied and many machine learning
models have been proposed. In this work, we explore an alternative perspective
by augmenting reactive reviews with proactive forecasting. Specifically, we
propose a new concept text toxicity propensity to characterize the extent to
which a text tends to attract toxic comments. Beta regression is then
introduced to do the probabilistic modeling, which is demonstrated to function
well in comprehensive experiments. We also propose an explanation method to
communicate the model decision clearly. Both propensity scoring and
interpretation benefit text moderation in a novel manner. Finally, the
proposed scaling mechanism for the linear model offers useful insights beyond
this work.

##  1 Introduction

Text moderation is essential for maintaining a non-toxic online community for
media platforms Nobata et al. (2016). Many efforts from both academia and
industry have been made to address this critical problem. Recently, the most
prototypical thread is to do sophisticated feature engineering or develop
powerful learning algorithms Nobata et al. (2016); Badjatiya et al. (2017);
Bodapati et al. (2019); Tan et al. (2020); Tran et al. (2020). Automatic
comment moderation schemes plus human review are certainly the cornerstone of
the fight against toxicity.

These existing works, however, are reactive approaches to handling user
generated text in response to the publication of new articles. In this paper,
we revisit this challenge from a proactive perspective. Specifically, we
introduce a novel concept text toxicity propensity to quantify how likely an
article is prone to incur toxic comments. This is a proactive outlook index
for news articles prior to the publication, which differs radically from the
existing reactive approaches to comments.

In this context, reactive describes comment-level moderation algorithms after
the publication of news articles (e.g., Perspective Perspectiveapi ), which
quantifies whether comments are toxic and should be taken down or sent for
human review. Proactive emphasizes article-level moderation effort before the
publication (without access to comments), which forecasts how likely articles
are to attract toxic comments in the future and gives suggestions (e.g.,
rephrase news articles properly) in advance. Our work can be viewed as the
first machine learning effort for a proactive stance against toxicity.

Formally, we propose a probabilistic approach based on Beta distribution Beta
to regress article toxicity propensity on article text. For previously
published news articles with comments, we take the average of comments’
toxicity scores as the ground-truth label for model learning. The
effectiveness of this approach is shown in both test set and human labeling.
We also develop a scheme that can provide convincing explanation to the
decision of the deep learning model.

##  2 Related Works

To our best knowledge, there’s no prior research thread on proactive text
moderation. Nonetheless, many reactive approaches have been explored including
hand-crafted feature engineering Chen et al. (2012); Warner and Hirschberg
(2012); Nobata et al. (2016), neural networks Badjatiya et al. (2017);
Pavlopoulos et al. (2017); Agrawal and Awekar (2018); Zhang et al. (2018) and
transformer variants Bodapati et al. (2019); Tan et al. (2020).

Recently, context, in the form of parent posts, has been studied but it is
only viewed as regular text snippets for lifting the performance of toxicity
classifiers Pavlopoulos et al. (2020) while screening posts. Our work instead
focuses on predicting the proactive toxicity propensity of articles before
they receive user comments.

Beta distribution is usually utilized as a priori in Bayesian statistics. The
most popular example in natural language processing is Topic Model Blei et al.
(2003), where the multivariate version of beta distribution (a.k.a. Dirichlet
distribution) generates parameters of mixture models. Beta regression is
originally proposed for modeling rate and proportion data Ferrari and Cribari-
Neto (2004) by parameterizing mean and dispersion and regressing parameters of
interest. It has been applied to evaluate grid search parameters in
optimization McKinney-Bock and Bedrick (2019), model emotional dimensions
Aggarwal et al. (2020) and statistical processes of child-adult linguistic
coordination and alignment Misiek et al. (2020).

##  3 Beta Regression

In this work, both comment toxicity score and the derived article toxicity
propensity score (to be detailed in the subsequent section 4.1) range from $0$
to $1$. Empirically, their distributions exhibit an asymmetry and may not be
modelled well with the Gaussian distribution (Figs. 2 and 3 of Appendix A).
Furthermore, comment toxicity score distributions of individual articles vary
with article content as shown in Fig. 3 of Appendix A. Modelling the entire
distribution of an article comment toxicity scores is thus a reasonable
approach. Beta distribution is very flexible and it can model quite a wide
range of well-known distribution families from symmetric uniform
($\alpha=\beta=1$) and bell-shaped distributions ($\alpha=\beta=2$) to
asymmetric shapes ($\alpha\neq\beta$).

In this context, the toxicity propensity score $y$ is assumed to follow the
Beta distribution with probability density function (pdf):

$\begin{split}p(y|\alpha,\beta)&=\mathrm{Beta}(\alpha,\beta)=\frac{y^{\alpha-1}(1-y)^{\beta-1}}{B(\alpha,\beta)}\\\
\end{split}$ (1)

where $\alpha$ and $\beta$ are two positive shape parameters to control the
distribution. $B(\alpha,\beta)$ is the normalization constant and support $y$
meets $y\in[0,1]$. Eq. 1 holds the probabilistic randomness given $\alpha$ and
$\beta$, we thus impose a regression structure of them on text content.

Formally, given a training set
$\mathcal{D}=\\{(\mathbf{x_{n}},y_{n})\\}_{n=1}^{N}$ with raw text feature
vector $\mathbf{x_{n}}$ and label $y_{n}$ for sample $n$, we apply feature
engineering or text embedding $g(\cdot)$ and then regress $\alpha_{n}(>0)$ and
$\beta_{n}(>0)$ on $g(\mathbf{x_{n}})$ respectively as

$\begin{split}\mathrm{log}(\alpha_{n})=f_{\alpha}(g(\boldsymbol{x}_{n}))\\\
\mathrm{log}(\beta_{n})=f_{\beta}(g(\boldsymbol{x}_{n}))\end{split}$ (2)

where $f_{\alpha}(\cdot)$ and $f_{\beta}(\cdot)$ are learned jointly.
$g(\cdot)$ can be either pre-fixed or learned together with
$f_{\alpha}(\cdot)$ and $f_{\beta}(\cdot)$, which is detailed in the
subsequent section. Specifically, the learning procedure of
$f_{\alpha}(\cdot)$, $f_{\beta}(\cdot)$ and $g(\cdot)$ (if applicable) is to
minimize loss

$\begin{split}\mathcal{L}&=-\frac{1}{N}\sum_{n=1}^{N}\mathrm{log}\big{(}p(y_{n}|\alpha_{n},\beta_{n})\big{)}\\\
\end{split}$ (3)

Substituting Eqs. 1 and 2 into it gives the final objective function.

In the inference phase, with learned $f_{\alpha}(\cdot)$, $f_{\beta}(\cdot)$
and $g(\cdot)$, $\alpha_{m}$ and $\beta_{m}$ for a new sample $\mathbf{x_{m}}$
can be readily derived from Eq. 2. We take the mean of Eq. 1 as a point
estimator: $\widehat{y_{m}}=\frac{\alpha_{m}}{\alpha_{m}+\beta_{m}}$ because
we are predicting the average toxicity.

##  4 Experiments

###  4.1 Dataset

We collect a dataset of articles published on Yahoo media outlets, which are
all written in English. We also exclude articles with low comment volume to
make the distribution learning reliable. The number of comments for 99% of the
analyzed articles lie in [10, 8K], with 25% quantile of 20, median of 50 and
mean of 448. The employed dataset is then split into training, validation and
test parts based on the publishing date with ratio of 8:1:1 as described in
Table 1. It’s worthwhile to note that input text $\mathbf{x_{n}}$ is the
concatenation of article title and text body. The toxicity propensity score
$y_{n}$ of article $n$ is defined as the average toxicity score of all
associated comments. Comments are scored by Google’s Perspective
Perspectiveapi , which lies in [0, 1]. Perspective intakes user generated text
and outputs toxicity probability. It’s a convolutional neural net Noever
(2018) trained on a comments dataset 111https://www.kaggle.com/c/jigsaw-toxic-
comment-classification-challenge of wikipedia labeled by multiple people per
majority rule.

Table 1: Basic statistics of dataset breakdown

Training Validation Test Sample Size 536,711 70,946 70,946 Publishing Date
2004 - 05/2020 05/2020-06/2020 06/2020-09/2020

###  4.2 Experiment Setup

In Eq. 2, we set both $f_{\alpha}(\cdot)$ and $f_{\beta}(\cdot)$ to single-
layer neural networks. For $g(\cdot)$, we experiment with either Bag of Words
(BOW) or BERT embedding (BERT) Devlin et al. (2019). Specifically, we take
uni-gram and bi-gram words sequence and compute the corresponding Term
Frequency-Inverse Document Frequency (TF-IDF) vectors, which leads to around
$5.8$ million tokens for BOW. For BERT, we take the base version and then
fine-tune $f_{\alpha}(\cdot)$ and $f_{\beta}(\cdot)$ on top of the [CLS]
embedding, which ends up with $110$ million parameters. If input text exceeds
the maximum length ($510$ as [CLS] and [SEP] are reserved), we adopt a simple
yet effective truncation scheme Sun et al. (2019). Specifically, we
empirically select the first 128 and the last 382 tokens for long text. The
rationale is that the informative snippets are more likely to reside in the
beginning and end. Batch size is 16 and learning rate is $1e-5$ for Adam
optimizer Kingma and Ba (2015). They are called BOW-$\beta$ and BERT-$\beta$
for short.

###  4.3 Baseline Methods and Metrics

We compare with the linear regression method using BOW features, as well as
the BERT base model. Both are combined with one of two loss functions, Mean
Absolute Error (MAE) or Mean Squared Error (MSE). We call them BOW-MAE, BOW-
MSE, BERT-MAE, BERT-MSE, respectively. The experiment settings are same as the
Beta regression.

Since we are interested in identifying articles of high toxicity propensity,
we want to make sure that an article with high average toxicity is ranked
higher than one with low propensity. Thus in addition to mean absolute error,
root mean squared error (RMSE) and AUC@Precision-Recall curves (AUC@PR), we
measure performance using two ranking metrics, Kendall coefficient (Kendall)
and Spearman’s coefficient (Spearman).

###  4.4 Results

We perform evaluation on the whole test set and on human labels.

####  4.4.1 Test Set

Table 2 details the performance comparisons. Overall, Beta regression stands
out across different metrics regardless of feature engineering due to its
modeling flexibility. BERT-based methods also outperform BOW ones in terms of
feature engineering and representation. This is reasonable as the former has
20 times as large parameters as the latter and offers the contextual
embedding. Interestingly, MAE and MSE schemes don’t achieve the minimum MAE
and RMSE although they are dedicated to this goal, which might result from the
limitation of point estimator.

Table 2: Performance comparisons on test set

Kendall Spearman MAE RMSE val test val test val test val test BOW-MAE 0.332
0.314 0.488 0.464 0.076 0.081 0.095 0.100 BOW-MSE 0.428 0.402 0.606 0.574
0.057 0.063 0.076 0.084 BOW-$\beta$ 0.437 0.413 0.617 0.589 0.056 0.061 0.075
0.081 BERT-MAE 0.360 0.333 0.525 0.489 0.072 0.076 0.092 0.095 BERT-MSE 0.442
0.423 0.621 0.598 0.070 0.073 0.089 0.093 BERT-$\beta$ 0.462 0.440 0.642 0.617
0.056 0.065 0.075 0.085

####  4.4.2 Human Labels

As labels are derived from machine, we want a sanity check to ensure that the
model decision conforms to human intuition. Namely, when the model classifies
an article as having high toxicity propensity, we want to make sure that it
correlates well with human judgement. To this end, we divide test set into 10
equal buckets with an interval of $0.1$ and merge the last 4 buckets into
[0.6, 1] due to much fewer articles with score being above $0.6$ (as shown in
Fig. 2). There are thus a total of $7$ buckets [0, 0.1), [0.1, 0.2), [0.2,
0.3), [0.3, 0.4), [0.4, 0.5), [0.5, 0.6) and [0.6, 1.0]. We then randomly take
$100$ samples per bucket and set aside $10\%$ for human training and the
remaining are labelled by the human judges as the benchmark set. We recruit
two groups of people for independent annotation, which are required to pick
one from five levels (a reasonable balance between smoothness and accuracy for
manually labeling toxicity propensity per judges’ suggestion) to describe the
propensity extent to which an article is likely to attract toxic comments:
Very Unlikely (VU), Unlikely (U), Neutral (N), Likely (L) and Very Likely
(VL). Table 3 is the confusion matrix showing how much two groups of human
judges agree with each other.

Table 3: Confusion matrix of two groups of human annotation G1 and G2

G1 G2 VU U N L VL Total VU 89 28 0 23 1 141 U 30 26 0 37 3 96 2-3 N 0 1 0 3 1
5 5-6 L 31 25 0 34 56 146 VL 18 21 0 87 124 250 Total 168 101 0 184 185 638

Figure 1: PR curves on human labelled data.

Interestingly, even humans don’t agree with each other on all examples.
Roughly, the perception of the two groups is consist on $74\%$ samples (top
left and bottom right boxes in Table 3). Moreover, Cohen’s Kappa is about
$0.23$ by taking expected chance agreement into account
222https://en.wikipedia.org/wiki/Cohen%27s_kappa. In light of this, we jointly
score the set by assigning $-2$, $-1$, $0$, $1$ and $2$ to VU, U, N, L and VL,
respectively. Since each article has two labels, the addition gives an integer
score interval [-4,4]. Table 4 reports the performance with human labels as
the ground truth, which confirms the previous findings that BERT-$\beta$
performs the best. Additionally, we pick scores $2$, $3$ and $4$ as thresholds
to monitor precision and recall curves (Fig. 1). Likewise, the proposed
schemes achieve compelling performance widely.

Taken together, our probabilistic methods agree more with both machine and
human judgements.

Table 4: Performance on human labelled set

BOW-MAE BOW-MSE BOW-$\beta$ Kendall 0.402 0.481 0.491 Spearman 0.562 0.635
0.649 BERT-MAE BERT-MSE BERT-$\beta$ Kendall 0.441 0.508 0.522 Spearman 0.599
0.665 0.679

###  4.5 Explanation

As we focus on the pre-publication text moderation, a reasonable explanation
is an essential step to convince stake-holders of subsequent operations. For
BERT-$\beta$ explanation, we adopt gradient-based saliency map variants from
computer vision Simonyan et al. (2013); Shrikumar et al. (2017). We compute
the gradient $\nabla f(\mathbf{x})$ with respect to input tokens embedding
$\mathbf{e(x)}$, where
$f(\mathbf{x})=\alpha(\mathbf{x})/(\alpha(\mathbf{x})+\beta(\mathbf{x}))$ is
the mean prediction for sample $\mathbf{x}$ (Section 3), and
$\mathbf{x}=(t_{1},t_{2},\cdots,t_{L})$ where $t_{l}(l=1,2,\cdots,L)$ is a
single token. The element of $\nabla f(\mathbf{x})$ is partial derivative
$\frac{\partial{f}}{\partial{\mathbf{e}(t_{l})}}(\mathbf{x})$ to measure the
token-level contribution to the scoring. The explanation is conducted by
assuming the article is controversial, and we want to figure out which words
cause some comments to be toxic. So it also makes sense to maximize the
maximum toxicity of the comments. We thus experiment with
$f(\mathbf{x})=(\alpha(\mathbf{x})-1)/(\alpha(\mathbf{x})+\beta(\mathbf{x})-2)$,
which is the mode (corresponding to the peak in the PDF of Beta distribution)
under reasonable assumption ($\alpha,\beta>1$). We denote the resulting scheme
by subscript "mode".

For saliency map (SM) Simonyan et al. (2013), the metric is
$\|\frac{\partial{f}}{\partial{\mathbf{e}(t_{l})}}(\mathbf{x})\|_{2}$ without
direction. A variant is dot product (DP) between token embedding and gradient
element
$\mathbf{e}(t_{l})^{T}\cdotp\frac{\partial{f}}{\partial{\mathbf{e}(t_{l})}}(\mathbf{x})$
with direction Shrikumar et al. (2017). We also propose a hybrid (HB) scheme
to take magnitude of SM and direction of DP to form a new metric. We perform
an ablation study (AS) to delete single token $t_{l}$ alternately and then
compute the score discrepancy between original $\mathbf{x}$ and
$\mathbf{x}_{\neg{l}}$ as well. As a reference, we examine the regression
coefficients (RC) of linear BOW-MSE, which are easy to check for explaining
the contribution of corresponding words.

A few well-trained human judges are recruited to tag $k$ (example-specific,
determined by annotators) most important words. We then prioritize tokens with
different metrics and pick top $k$ ones as candidates. Hit rate (proportion of
human annotated tokens covered by schemes) is used to compare different tools.
We take $1,000$ examples for human review and compute the average hit rate, as
compared in Table 5.

All schemes for BERT-$\beta$ are much better than linear scheme RC, which is
consistent with the predictive performance discrepancy. SM and HB are close
and outperform black-box ablation study, which implies the valuable role of
model-aware gradients in the explanation. DP is inferior to AS and seems not
consistent with human annotation as well as other gradient based methods. In
practice, we take SM for the explanation (Appendix B) due to its out-
performance and simplicity. As expected, mode (SM${}_{\text{mode}}$) covers
more annotated words than mean (SM) on average (more discussions in Appendix
C).

Table 5: Performance (average hit rate) comparison SM | DP | HB | AS | RC | SM${}_{\text{mode}}$
---|---|---|---|---|---
0.549 | 0.430 | 0.543 | 0.467 | 0.382 | 0.553

##  5 Additional Study

Linear regression (BOW-MSE) is inferior to BERT-$\beta$. Nonetheless, it is
much faster in training, inference and explanation as it is about 20 times as
small as BERT-$\beta$. Thus, we investigate if the performance of the linear
model could be improved for industrial deployment.

Inspired by NBSVM Wang and Manning (2012), we scale TF-IDF vectors of BOW-MSE
by a weight vector defined as
$\mathbf{w}=\frac{\big{(}\mathbf{\tau(X)}-\overline{\mathbf{\tau(X)}}\big{)}^{T}\big{(}\mathbf{y}-\overline{\mathbf{y}}\big{)}}{\big{\|}\mathbf{\tau(X)}-\overline{\mathbf{\tau(X)}}\big{\|}^{2}}$
where $\mathbf{X}$ is the training corpus. $\mathbf{y}\in[0,1]^{N\times 1}$
and $\mathbf{\tau(X)}\in\mathbb{Z}^{N\times M}$ ($M=5.8$ million) are the
training labels and TF-IDF matrix. $\overline{\mathbf{y}}$ and
$\overline{\mathbf{\tau(X)}}$ are their column-wise means. The pre-computed
$\mathbf{w}$ can be viewed as a surrogate of the regression coefficient for
the linear regression problem, which is used to scale TF-IDF of BOW-MSE in
both training and inference phases. We call it Naive Bayes Linear Regression
(NBLR) for short.

The scaling benefits the performance, as compared in Table 6. As can be seen,
NBLR improves upon BOW-MSE significantly, although it is not as good as
BERT-$\beta$.

Table 6: Performance on test set and human labels

Test Set Human Label Kendall Spearman Kendall Spearman BOW-MSE 0.402 0.574
0.481 0.635 NBLR 0.413 (+.011) 0.588 (+.014) 0.501 (+.020) 0.656 (+.021)
BERT-$\beta$ 0.440 0.617 0.522 0.679 Human Label AUC@PR at 2 AUC@PR at 3
AUC@PR at 4 BOW-MSE 0.77 0.74 0.48 NBLR 0.78 (+.01) 0.76 (+.02) 0.53 (+.05)
BERT-$\beta$ 0.80 0.77 0.59

##  6 Discussion and Future Work

Our work can benefit text moderation. The proactive propensity offers a
toxicity outlook for comments, which could be utilized in multiple ways. For
example, stricter moderation rules are enforced for articles that are
predicted to have a high toxicity propensity. Furthermore, the propensity
could be used as an additional feature for the downstream reactive toxicity
recognition models, as well as for allocation of appropriate human resources.

The explanation tool can also be used to remind editors to rephrase some
controversial words to mitigate the odds of attracting toxic comments. Text
moderation is an important yet challenging task, our proactive work is
attempting to open up a new perspective to augment the traditional reactive
procedure. Our current model, however, is not perfect as shown by article b in
Fig. 3 of Appendix A where the learned distribution doesn’t fit well the
observed histogram. Technically, NBLR is an encouraging lightweight extension
to Linear Regression. Likewise, we will continue to work towards the
improvement of the non-linear Beta regression.

##  7 Conclusion

We approach text moderation by developing a well-motivated probabilistic model
to learn a proactive toxicity propensity. An explanation scheme is also
proposed to visually explain the connection between this new prospective
score, and text content. Our experiment shows the superior performance of the
proposed BERT-$\beta$ algorithm, compared with a number of baselines, in
predicting both the average toxicity score, and the human judgement.

## References

  * Aggarwal et al. (2020) Jai Aggarwal, Ella Rabinovich, and Suzanne Stevenson. 2020.  Exploration of gender differences in covid-19 discourse on reddit.  In _Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020_.
  * Agrawal and Awekar (2018) Sweta Agrawal and Amit Awekar. 2018.  Deep learning for detecting cyberbullying across multiple social media platforms.  In _European Conference on Information Retrieval_ , pages 141–153. Springer.
  * Badjatiya et al. (2017) Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017.  Deep learning for hate speech detection in tweets.  In _Proceedings of the 26th International Conference on World Wide Web Companion_ , pages 759–760.
  * (4) Wiki Distribution Beta.  https://en.wikipedia.org/wiki/Beta_distribution.
  * Blei et al. (2003) David M Blei, Andrew Y Ng, and Michael I Jordan. 2003.  Latent dirichlet allocation.  _the Journal of machine Learning research_ , 3:993–1022.
  * Bodapati et al. (2019) Sravan Babu Bodapati, Spandana Gella, Kasturi Bhattacharjee, and Yaser Al-Onaizan. 2019.  Neural word decomposition models for abusive language detection.  _arXiv preprint arXiv:1910.01043_.
  * Chen et al. (2012) Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012.  Detecting offensive language in social media to protect adolescent online safety.  In _Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confernece on Social Computing (SocialCom)_ , pages 71–80. IEEE.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Ferrari and Cribari-Neto (2004) Silvia Ferrari and Francisco Cribari-Neto. 2004.  Beta regression for modelling rates and proportions.  _Journal of applied statistics_ , 31(7):799–815.
  * Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _ICLR (Poster)_.
  * McKinney-Bock and Bedrick (2019) Katy McKinney-Bock and Steven Bedrick. 2019.  Classification of semantic paraphasias: Optimization of a word embedding model.  In _Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP_ , pages 52–62.
  * Misiek et al. (2020) Thomas Misiek, Benoit Favre, and Abdellah Fourtassi. 2020.  Development of multi-level linguistic alignment in child-adult conversations.  In _Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics_ , pages 54–58.
  * Nobata et al. (2016) Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016\.  Abusive language detection in online user content.  In _Proceedings of the 25th international conference on world wide web_ , pages 145–153.
  * Noever (2018) David Noever. 2018.  Machine learning suites for online toxicity detection.  _arXiv preprint arXiv:1810.01869_.
  * Pavlopoulos et al. (2017) John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017.  Deeper attention to abusive user content moderation.  In _Proceedings of the 2017 conference on empirical methods in natural language processing_ , pages 1125–1135.
  * Pavlopoulos et al. (2020) John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020.  Toxicity detection: Does context really matter?  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4296–4305.
  * (17) Google Perspectiveapi.  https://www.perspectiveapi.com.
  * Shrikumar et al. (2017) Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017.  Learning important features through propagating activation differences.  In _International Conference on Machine Learning_ , pages 3145–3153. PMLR.
  * Simonyan et al. (2013) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  _arXiv preprint arXiv:1312.6034_.
  * Sun et al. (2019) Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019.  How to fine-tune bert for text classification?  In _China National Conference on Chinese Computational Linguistics_ , pages 194–206. Springer.
  * Tan et al. (2020) Fei Tan, Yifan Hu, Changwei Hu, Keqian Li, and Kevin Yen. 2020.  Tnt: Text normalization based pre-training of transformers for content moderation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4735–4741.
  * Tran et al. (2020) Thanh Tran, Yifan Hu, Changwei Hu, Kevin Yen, Fei Tan, Kyumin Lee, and Se Rim Park. 2020.  Habertor: An efficient and effective deep hatespeech detector.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7486–7502.
  * Wang and Manning (2012) Sida I Wang and Christopher D Manning. 2012.  Baselines and bigrams: Simple, good sentiment and topic classification.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 90–94.
  * Warner and Hirschberg (2012) William Warner and Julia Hirschberg. 2012.  Detecting hate speech on the world wide web.  In _Proceedings of the Second Workshop on Language in Social Media_ , pages 19–26. Association for Computational Linguistics.
  * Zhang et al. (2018) Ziqi Zhang, David Robinson, and Jonathan Tepper. 2018.  Detecting hate speech on twitter using a convolution-gru based deep neural network.  In _European Semantic Web Conference_ , pages 745–760. Springer.

##  Appendix A Toxicity Score and Beta Distribution

The distribution of news articles’ toxicity propensity score is reported in
Fig. 2. Comment score distributions of two articles with predictive
distribution are given in Fig. 3.

Figure 2: Toxicity propensity score (mean comment toxicity scores)
distribution of news articles.

Figure 3: Toxicity score histogram density of comments for articles a (top)
and b (bottom). Solid red lines represent predictive beta distribution for
individual articles.

##  Appendix B SM Explanation Examples

We pick two samples from the test set and then leverage SM in section 4.5 to
highlight key words for the illustration purpose, as shown in Fig. 4. The
color intensity is proportional to the normalized saliency map value. The
darker the color of a token is, the more important it’s to the scoring.
There’s also a positional bias towards the first sentence as it’s the article
title.

Figure 4: Model explanation examples.

##  Appendix C BERT-$\beta$ mode

We also explore the mode of BERT-$\beta$ as a point estimator and compare it
with the mean. Table 7 details the performance discrepancy between the test
set and human labels. For the toxicity propensity prediction in the test set,
it does make sense for mean to slightly outperform mode as ground-truth labels
are the score mean of comments. When it comes to human labels and explanation,
people annotate news articles based on the perceived controversial words most
likely to incur toxic comments. Mode is thus able to capture the worst case
better and agrees more with human annotations. This finding is in line with
the better explanation performance, as compared in Table 5.

Table 7: Performance of BERT-$\beta$ point estimators on test set and human
labels

Test Set Human Label Kendall Spearman RMSE MAE Kendall Spearman mean 0.440
0.617 0.065 0.085 0.522 0.679 mode 0.439 0.614 0.077 0.099 0.543 0.704 Human
Label AUC@PR at 2 AUC@PR at 3 AUC@PR at 4 mean 0.8 0.77 0.59 mode 0.82 0.79
0.63


Let $\pi_{f,p}\boxtimes\chi_{f,p}$ be a local component of
$\Pi_{f}\boxtimes\chi_{f}$. Then,

$\mathrm{rank}_{\mathcal{O}_{\mathbf{L}_{f}}}\
\mathcal{L}_{f,p}^{H_{p},\mathrm{unv}}=\mathrm{rank}_{\mathcal{O}_{\mathbf{L}_{f}}}\
\operatorname{Hom}_{\mathcal{O}_{\mathbf{L}_{f}}[\mathfrak{H}_{p}]}\left(\mathcal{H}_{f,p}^{H_{p},\mathrm{unv}},\mathcal{O}_{\mathbf{L}_{f}}\right)=1$

###### Proof.

As remarked in [LW12], each such local factor $\pi_{f,p}$ is given by the
irreducible unramified principal series
$\mathcal{B}(\sigma_{p,1},\sigma_{p,2})$ with
$\sigma_{p,1}(p)=\alpha_{p}p^{\frac{1-k}{2}}$ and
$\sigma_{p,2}(p)=\beta_{p}p^{\frac{1-k}{2}}$ where $\alpha_{p}$ and
$\beta_{p}$ are the roots of the Hecke polynomial at $p$:

$X^{2}-a_{p}(f)X+\varepsilon(p)p^{k-1}.$

Hence by construction, and the fact that
$p\in\mathcal{O}_{\mathbf{L}_{f}}^{\times}$ (and $k$ is even), we see that the
$\overline{\mathbf{Q}}_{\ell}$-valued spherical Hecke eigensystem of
$\pi_{f,p}\boxtimes\chi_{f,p}$ restricts to a morphism
$\mathcal{R}_{\mathcal{O}_{\mathbf{L}_{f}}}^{H_{p}}\rightarrow\mathcal{O}_{\mathbf{L}_{f}}.$
The result now follows from the local statements of 5.0.1 and 5.0.2. ∎

We can also define a linear form
$\mathcal{Z}:\Pi_{f}\boxtimes\chi_{f}\rightarrow\overline{\mathbf{Q}}_{\ell}$
given on pure tensors by the base change to $\overline{\mathbf{Q}}_{\ell}$ via
$\iota$, of the complex linear form $\prod_{p\notin S}\mathcal{Z}^{H_{p}}$.
Recall that $\mathcal{Z}^{H_{p}}$ is as in 4.0.10 and varies depending on
whether $H_{p}$ is split or non-split in $G_{p}$. Similarly to $2.1$ in
[HS01], the space
$\operatorname{Hom}_{\mathfrak{H}(\mathbf{A}^{S})}(\Pi_{f}\boxtimes\chi_{f},\mathbf{1})$
is one dimensional. By the local theory of Section 4 the linear form
$\mathcal{Z}$ is a non-zero (basis) element of this space that takes the value
one on the spherical vector $W_{f}^{\mathrm{sph}}:=\otimes_{p\notin
S}(W_{\pi_{f,p}}^{\mathrm{sph}}\otimes 1)\in\Pi_{f}\boxtimes\chi_{f}$.

######  Corollary 8.2.2.

The linear form $\mathcal{Z}$ takes values in $\mathcal{O}_{\mathbf{L}_{f}}$
when evaluated on elements of the form

$\frac{1}{\operatorname{vol}_{H(\mathbf{A}^{S})}\left(H(\hat{\mathbf{Z}}^{S})\cap
gG(\hat{\mathbf{Z}}^{S})g^{-1}\right)}(g,h)\cdot
W_{f}^{\mathrm{sph}}\in\Pi_{f}\boxtimes\chi_{f}\ \ \ ,\
(g,h)\in\mathfrak{G}(\mathbf{A}^{S}).$

###### Proof.

The proof follows at once by combining 8.2.1 and 5.0.3. ∎

##  Appendix A Technical lemmas

This appendix includes proofs of 2.1.3, 2.1.4, 6.3.1 and 7.0.1. We start by
proving the first two decomposition results. The following standard result
will be used in the proof of 2.1.3. We state it here for completeness.

######  Proposition A.0.1.

Let $p$ be an odd prime. Write $x=up^{n}\in\mathbf{Q}_{p}^{\times}$. Then $x$
is a square if and only if $n$ is even and $u$ is a square in
$\mathbf{Z}_{p}^{\times}/1+p\mathbf{Z}_{p}$

###### Proof.

[Ser12] Section $3.3$. ∎

######  Lemma A.0.2.

For $H$ non-split, we have the following double coset decomposition

$G_{p}=\bigcup_{\lambda\in\mathbf{Z}_{\geq 0}}H_{p}s(\lambda)G_{p}^{\circ}$

where $s(\lambda):=\left[\begin{smallmatrix}p^{\lambda}&\\\
&1\end{smallmatrix}\right]$. Furthermore, the decomposition is disjoint.

###### Proof.

By Iwasawa decomposition for $\operatorname{GL}_{2}(\mathbf{Q}_{p})$ we have
that

(18) $\displaystyle
G_{p}=\bigcup_{\begin{subarray}{c}r_{1},r_{2},r_{3}\in\mathbf{Z}_{\geq 0}\\\
u\in\mathbf{Z}_{p}^{\times}\cup\\{0\\}\end{subarray}}H_{p}\left[\begin{smallmatrix}p^{r_{1}}&up^{r_{2}}\\\
&p^{r_{3}}\end{smallmatrix}\right]G_{p}^{\circ}$

For $u=0$ note that

$\left[\begin{smallmatrix}&1\\\
-D\end{smallmatrix}\right]\left[\begin{smallmatrix}p^{r_{1}}&\\\
&p^{r_{3}}\end{smallmatrix}\right]\left[\begin{smallmatrix}&-D^{-1}\\\
1&\end{smallmatrix}\right]=\left[\begin{smallmatrix}p^{r_{3}}&\\\
&p^{r_{1}}\end{smallmatrix}\right]$

and $\left[\begin{smallmatrix}&-D^{-1}\\\ 1&\end{smallmatrix}\right]$ is in
$G_{p}^{\circ}$. Hence, we may assume that $r_{3}\leq r_{1}$ and in this case
we are done. We can now assume that $u$ is a unit. It suffices to show that
each double coset in (18) is generated by a diagonal matrix.

If $r_{2}\geq r_{1}$ we have $\left[\begin{smallmatrix}p^{r_{1}}&up^{r_{2}}\\\
&p^{r_{3}}\end{smallmatrix}\right]=\left[\begin{smallmatrix}p^{r_{1}}&\\\
&p^{r_{2}}\end{smallmatrix}\right]\left[\begin{smallmatrix}1&up^{r_{2}-r_{1}}\\\
&1\end{smallmatrix}\right]$ and the right most matrix is in $G_{p}^{\circ}$
since $r_{2}-r_{1}\geq 0$, so we are done. As a result, from now, we also
assume that $r_{1}>r_{2}$.

Case 1:$(r_{2}>r_{3})$. In this case we use the following matrix relation

$\left[\begin{smallmatrix}Dp^{r_{2}-r_{3}}&u^{-1}\\\
-u^{-1}D&Dp^{r_{2}-r_{2}}\end{smallmatrix}\right]\left[\begin{smallmatrix}p^{r_{1}}&up^{r_{2}}\\\
&p^{r_{3}}\end{smallmatrix}\right]=\left[\begin{smallmatrix}p^{r_{3}}&\\\
&p^{r_{1}}\end{smallmatrix}\right]\left[\begin{smallmatrix}Dp^{r_{1}+r_{2}-2r_{3}}&u^{-1}+Dup^{2(r_{2}-r_{3})}\\\
-Du^{-1}&\end{smallmatrix}\right]$

where the left most matrix is an element of $H_{p}$ and the right most matrix
is an element of $G_{p}^{\circ}$ since we are assuming that
$r_{1}>r_{2}>r_{3}$. So in this case we are once again done.

Case 2:$(r_{2}\leq r_{3}$). In this case we use the following matrix relation
instead

$\left[\begin{smallmatrix}p^{r_{3}-r_{2}}&-u\\\
uD&p^{r_{3}-r_{2}}\end{smallmatrix}\right]\left[\begin{smallmatrix}p^{r_{1}}&up^{r_{2}}\\\
&p^{r_{3}}\end{smallmatrix}\right]=\left[\begin{smallmatrix}p^{r_{1}+r_{3}-r_{2}}&\\\
&p^{r_{2}}\end{smallmatrix}\right]\left[\begin{smallmatrix}1&\\\
uDp^{r_{1}-r_{2}}&u^{2}D+p^{2(r_{3}-r_{2})}\end{smallmatrix}\right]$

where again the left most matrix is in $H_{p}$. Now, the right most matrix is
clearly in $G_{p}^{\circ}$ when $r_{2}<r_{3}$. We need tocheck it’s again an
element of $G_{p}^{\circ}$ even when $r_{2}=r_{3}$. That is, we need to check
that $u^{2}D+1$ is a unit in $\mathbf{Z}_{p}$, but this follows from A.0.1.
Thus, our claim holds and we are done in all cases.

Now we check disjointedness. Suppose we have an equality of double cosets

$H_{p}t(\lambda_{1})^{-1}G_{p}^{\circ}=H_{p}t(\lambda_{2})^{-1}G_{p}^{\circ}$

where without loss of generality we assume $\lambda_{1}\geq\lambda_{2}\geq 0$.
After carrying all the necessary matrix manipulations, the double coset
equality implies that the matrix

$\left[\begin{matrix}a&bp^{-\lambda_{2}}\\\
-bDp^{\lambda_{1}}&ap^{\lambda_{1}-\lambda_{2}}\end{matrix}\right]$

belongs in $G_{p}^{\circ}$. Here $\left[\begin{smallmatrix}a&b\\\
-bD&a\end{smallmatrix}\right]$ is an element of $H_{p}$. This implies that
both $a$ and $b$ are elements of $\mathbf{Z}_{p}$. The determinant of the
above matrix is equal to $p^{\lambda_{1}-\lambda_{2}}(a^{2}+b^{2}D)$ and this
is an element of $\mathbf{Z}_{p}^{\times}$. Since $a,b\in\mathbf{Z}_{p}$, we
conclude that $\lambda_{1}=\lambda_{2}$. ∎

######  Corollary A.0.3 (2.1.3).

If $H$ is non-split then we have the following double coset decomposition

$\mathfrak{G}_{p}=\bigcup_{\begin{subarray}{c}\lambda\in\mathbf{Z}_{\geq 0}\\\
a\in\mathbf{Z}\end{subarray}}\mathfrak{H}_{p}\left(s(\lambda),p^{a}\right)\mathfrak{G}_{p}^{\circ}$

where $s(\lambda):=\left[\begin{smallmatrix}p^{\lambda}&\\\
&1\end{smallmatrix}\right]$. Furthermore, the decomposition is disjoint.

###### Proof.

Existence follows at once from A.0.2. Disjointness follows again from A.0.2
and the fact that the pullback of $\mathbf{Z}_{p}^{\times}$ under
$(v_{p}\circ\det):H_{p}\rightarrow\mathbf{Q}_{p}^{\times}$, is precisely
$H_{p}^{\circ}$. ∎

######  Lemma A.0.4.

Let $H$ be split. We have the following double coset decomposition

$G_{p}=\bigcup_{\lambda\in\mathbf{Z}_{\geq
0}}H_{p}n_{0}s(\lambda)G_{p}^{\circ}$

where $n_{0}:=\left[\begin{smallmatrix}1&1\\\ &1\end{smallmatrix}\right]$, and
$s(\lambda):=\left[\begin{smallmatrix}p^{\lambda}&\\\
&1\end{smallmatrix}\right]$. Furthermore, this decomposition is disjoint.

###### Proof.

This is a special case of Proposition $2.3$ in [MS96] upon writing
$\left[\begin{smallmatrix}1&p^{-\lambda}\\\
&1\end{smallmatrix}\right]=s(\lambda)^{-1}n_{0}s(\lambda).$ ∎

######  Corollary A.0.5 (2.1.4).

If $H$ is split then we have the following double coset decomposition

$\mathfrak{G}_{p}=\bigcup_{\begin{subarray}{c}\lambda\in\mathbf{Z}_{\geq 0}\\\
a,b\in\mathbf{Z}\end{subarray}}\mathfrak{H}_{p}\left(n_{0}s(\lambda),\left[\begin{smallmatrix}p^{a}&\\\
&1\end{smallmatrix}\right]p^{b}\right)\mathfrak{G}_{p}^{\circ}.$

Furthermore, this decomposition is disjoint.

###### Proof.

The existence of such a decomposition follows at once from A.0.4. To see
disjointness, suppose that we have the following equality of elements of
$\mathfrak{G}_{p}$

$\left(n_{0}s(\lambda),\left[\begin{smallmatrix}p^{a}&\\\
&1\end{smallmatrix}\right]p^{b}\right)=\left(hn_{0}s(\lambda_{0})k_{1},h^{-1}\left[\begin{smallmatrix}p^{a_{0}}&\\\
&1\end{smallmatrix}\right]p^{b_{0}}k_{2}\right)$

with $h\in H_{p}$ and $(k_{1},k_{2})\in\mathfrak{G}_{p}^{\circ}$. It follows
that $h$ is equal to $\left[\begin{smallmatrix}p^{a_{0}-a}&\\\
&1\end{smallmatrix}\right]p^{b_{0}-b}k_{2}$ and thus

$n_{0}s(\lambda)=\left[\begin{smallmatrix}p^{a_{0}-a}&\\\
&1\end{smallmatrix}\right]p^{b_{0}-b}k_{2}n_{0}s(\lambda_{0})k_{1}.$

By A.0.4, this instantly implies that $\lambda=\lambda_{0}$. The $(2,1)$-entry
of the left hand side is zero, hence the $(2,1)$-entry of $k_{1}$ must be zero
and hence it’s $(2,2)$-entry must be a unit. Taking $p$-adic valuation of the
$(2,2)$-entry on both sides, we see that $b=b_{0}$. Finally taking $p$-adic
valuation of determinants, yields $a=a_{0}$. This concludes the proof. ∎

We now give a proof for 7.0.1. We first need a bit of preparation.

######  Definition A.0.6.

For $k\geq 1$, we define the element $\mathcal{T}(p^{k})$ of
$\mathcal{H}_{G_{p}}^{\circ}$, to be the characteristic function of the set of
all elements in $\mathrm{Mat}_{2}(\mathbf{Z}_{p})$, whose determinant
generates the ideal $p^{k}\mathbf{Z}_{p}$ in $\mathbf{Z}_{p}$.

######  Remark A.0.7.

It will be useful for our later computations to have the following explicit
formula for the characteristic function $\mathcal{T}(p^{k})$.

$\mathcal{T}(p^{k})=\sum_{\begin{subarray}{c}0\leq j\leq i\leq k\\\ \\\
i+j=k\end{subarray}}\operatorname{ch}\left(G_{p}^{\circ}\left[\begin{smallmatrix}p^{i}&\\\
&p^{j}\end{smallmatrix}\right]G_{p}^{\circ}\right).$

In our notation so far, the operator $\mathcal{T}(p)$ is nothing more than
$\mathcal{T}_{p}$.

######  Lemma A.0.8.

For any $k\in\mathbf{Z}_{\geq 2}$, the characteristic function
$\mathcal{T}(p^{k})$ satisfies the following recurrence relation:

$\mathcal{T}(p^{k})=\mathcal{T}_{p}\mathcal{T}(p^{k-1})-p\mathcal{S}_{p}\mathcal{T}(p^{k-2}).$

###### Proof.

[Bum97] Proposition $4.6.4$. ∎

######  Lemma A.0.9 (7.0.1).

For any $\lambda\in\mathbf{Z}_{\geq 1}$, the characteristic function
$\operatorname{ch}(G_{p}^{\circ}s(\lambda)G_{p}^{\circ})$ is an element of the
ideal
$\left(p+1,\mathcal{T}_{p}\right)\subseteq\mathcal{H}_{G_{p}}^{\circ}(\mathbf{Z}[1/p])$.

###### Proof.

For $\lambda=1$, there is nothing to prove. For $\lambda\geq 2$, we prove the
result by induction on $\lambda\geq 2$. We again write $\phi_{s(\lambda)}$ for
the function
$\operatorname{ch}\left(G_{p}^{\circ}\left[\begin{smallmatrix}p^{\lambda}&\\\
&1\end{smallmatrix}\right]G_{p}^{\circ}\right).$ For $\lambda=2$, we have

$\displaystyle\phi_{s(2)}$ $\displaystyle=\mathcal{T}(p^{2})-\mathcal{S}_{p}$
$\displaystyle=\mathcal{T}_{p}^{2}-(p+1)\mathcal{S}_{p}$

Now let $\lambda>2$. We have

$\mathcal{T}(p^{\lambda})=\phi_{s(\lambda)}+\sum_{\begin{subarray}{c}0<j\leq
i<\lambda\\\ i+j=\lambda\end{subarray}}\mathcal{S}_{p}^{j}\phi_{s(i-j)}$

and similar identities by replacing $\lambda$ with $\lambda-1$, $\lambda-2$.
Using A.0.8, we have

$\displaystyle\phi_{s(\lambda)}$
$\displaystyle=\mathcal{T}_{p}\mathcal{T}(p^{\lambda-1})-p\mathcal{S}_{p}\mathcal{T}(p^{\lambda-2})-\sum_{\begin{subarray}{c}0<j\leq
i<\lambda\\\ i+j=\lambda\end{subarray}}\mathcal{S}_{p}^{j}\phi_{s(i-j)}$ (19)
$\displaystyle=\left(\mathcal{T}_{p}\sum_{\begin{subarray}{c}0\leq j\leq
i\leq\lambda-1\\\
i+j=\lambda-1\end{subarray}}\mathcal{S}_{p}^{j}\phi_{s(i-j)}\right)-\left(p\mathcal{S}_{p}\sum_{\begin{subarray}{c}0\leq
j\leq i\leq\lambda-2\\\
i+j=\lambda-2\end{subarray}}\mathcal{S}_{p}^{j}\phi_{s(i-j)}\right)-\left(\sum_{\begin{subarray}{c}0<j\leq
i<\lambda\\\
i+j=\lambda\end{subarray}}\mathcal{S}_{p}^{j}\phi_{s(i-j)}\right).$

The first of the three brackets always lies in the ideal in question. If the
parity of the integers $\lambda,\lambda-1,\lambda-2$ is odd, even, odd, then
every summand in the second and third brackets is of the form
$\mathcal{S}_{p}^{j}\phi_{s(i-j)}$, with $i-j$ strictly bigger than zero and
strictly less than $\lambda$. Hence in this case, we are done by induction and
the case $\lambda=1$. If on the other hand the parity of the integers
$\lambda,\lambda-1,\lambda-2$ is even, odd, even, then the second bracket
contains a summand of the form $p\mathcal{S}_{p}^{\frac{\lambda}{2}}$ and the
third bracket contains a summand of the form
$\mathcal{S}_{p}^{\frac{\lambda}{2}}.$ Adding the two together, we end up with
$(p+1)\mathcal{S}_{p}^{\frac{\lambda}{2}}$ which is an element of the ideal in
question. Finally, every other summand in the second and third brackets of
(A.0.9) lies in this ideal by induction, and the case $\lambda=1$. This
concludes the proof of the lemma. ∎

The last thing left to do is to give a proof of 6.3.1.

######  Lemma A.0.10 (6.3.1).

For $\gamma\in H_{p}^{\circ}/H_{p}^{\circ}(\lambda)$, we write $\gamma
s(\lambda)=p^{z_{\gamma}}\left[\begin{smallmatrix}p^{r_{\gamma}}&\\\
&1\end{smallmatrix}\right]\left[\begin{smallmatrix}1&n_{\gamma}\\\
&1\end{smallmatrix}\right]k_{\gamma}\in Z_{p}A_{p}N_{p}G_{p}^{\circ}$. There
exists a complete set of distinct coset representatives of
$H_{p}^{\circ}/H_{p}^{\circ}(\lambda)$ that can be partitioned as follows:

  * •

There are $\varphi(p^{\lambda})$ elements for which
$v_{p}(n_{\gamma})=-\lambda$ and $z_{\gamma}=0$

  * •

For each $1\leq i\leq\lambda-1$, there are $\varphi(p^{\lambda-i})$ elements
for which $v_{p}(n_{\gamma})=i-\lambda$ and $z_{\gamma}=i$. Additionally,
there are $\varphi(p^{\lambda-i})$ elements for which
$v_{p}(n_{\gamma})=i-\lambda$ and $z_{\gamma}=0$.

  * •

Finally, there’s a unique element for which $n_{\gamma}=0$ and
$z_{\gamma}=\lambda$, and there is a unique element for which $n_{\gamma}=0$
and $z_{\gamma}=0$.

###### Proof.

Let $\lambda\in\mathbf{Z}_{\geq 0}.$ We have the following complete set of
distinct coset representatives for $H_{p}^{\circ}/H_{p}^{\circ}(\lambda)$

$\left\\{\left[\begin{smallmatrix}a&-D\\\
1&a\end{smallmatrix}\right],\left[\begin{smallmatrix}1&-bD\\\
b&1\end{smallmatrix}\right]\ |\ a\in\mathbf{Z}/p^{\lambda}\mathbf{Z}\ ,\ b\in
p\mathbf{Z}/p^{\lambda}\mathbf{Z}\right\\}.$

For $\gamma=\left[\begin{smallmatrix}a&-D\\\ 1&a\end{smallmatrix}\right]$, we
have $\gamma s(\lambda)=\left[\begin{smallmatrix}ap^{\lambda}&-D\\\
p^{\lambda}&a\end{smallmatrix}\right].$ It is not hard to check that Iwasawa
decomposition gives

$v_{p}(n_{\gamma})=\begin{cases}v_{p}(a)-\lambda\ ,\ &\text{if
}v_{p}(a)\leq\lambda-1\\\ \infty\ ,\ &\text{if }a=0\end{cases}\ \ \ ,\
z_{\gamma}=\begin{cases}v_{p}(a)\ ,\ &\text{if }v_{p}(a)\leq\lambda-1\\\
\lambda\ ,\ &\text{if }a=0.\end{cases}$

If on the other hand $\gamma=\left[\begin{smallmatrix}1&-bD\\\
b&1\end{smallmatrix}\right]$, then $\gamma
s(\lambda)=\left[\begin{smallmatrix}p^{\lambda}&-bD\\\
bp^{\lambda}&1\end{smallmatrix}\right]$, and this time Iwasawa decomposition
gives

$v_{p}(n_{\gamma})=\begin{cases}v_{p}(a)-\lambda\ ,\ &\text{if
}v_{p}(a)\leq\lambda-1\\\ \infty\ ,\ &\text{if }a=0\end{cases}\ \ ,\ \
z_{\gamma}=0.$

The result now follows from a simple counting argument. ∎

## References

  * [BL95] Laure Barthel and Ron Livné.  Modular representations of gl2 of a local-field: The ordinary, unramified case.  Journal of Number Theory, 55(1):1–27, 1995.
  * [Bum97] Daniel Bump.  Automorphic Forms and Representations.  Cambridge Studies in Advanced Mathematics. Cambridge University Press, 1997.
  * [DS05] Fred Diamond and Jerry Michael Shurman.  A first course in modular forms, volume 228.  Springer, 2005.
  * [Gro20] Giada Grossi.  On norm relations for asai-flach classes, 2020.
  * [HS01] Michael Harris and Anthony J Scholl.  A note on trilinear forms for reducible representations and beilinson’s conjectures.  Journal of the European Mathematical Society, 3(1):93–104, 2001.
  * [JL06] Hervé Jacquet and Robert P Langlands.  Automorphic Forms on GL (2): Part 1, volume 114.  Springer, 2006.
  * [JPSS83] Hervé Jacquet, Ilya I Piatetskii-Shapiro, and Joseph A Shalika.  Rankin-selberg convolutions.  American journal of mathematics, 105(2):367–464, 1983.
  * [KMS03] SHIN-ICHI KATO, ATSUSHI MURASE, and TAKASHI SUGANO.  Whittaker-shintani functions for orthogonal groups.  Tohoku Mathematical Journal, Second Series, 55(1):1–64, 2003.
  * [Loe21] David Loeffler.  Gross–prasad periods for reducible representations.  In Forum Mathematicum, volume 33, pages 1169–1177. De Gruyter, 2021.
  * [LSZ21a] David Loeffler, Chris Skinner, and Sarah Livia Zerbes.  Euler systems for gsp(4), 2021.
  * [LSZ21b] David Loeffler, Christopher Skinner, and Sarah Livia Zerbes.  An euler system for GU(2, 1).  Mathematische Annalen, 382(3-4):1091–1141, jul 2021.
  * [LW12] David Loeffler and Jared Weinstein.  On the computation of local components of a newform.  Mathematics of Computation, 81(278):1179–1200, 2012.
  * [MS91] Atsushi Murase and Takashi Sugano.  Whittaker-shintani functions on the symplectic group of fourier-jacobi type.  Compositio Mathematica, 79(3):321–349, 1991.
  * [MS96] Atsushi Murase and Takashi Sugano.  Shintani functions and automorphic l-functions for gl (n) dedicated to professor hideo shimizu on his sixtieth birthday.  Tohoku Mathematical Journal, Second Series, 48(2):165–202, 1996.
  * [MS16] A. Minguez and V. Secherre.  Unramified -modular representations of GLn(F) and its inner forms.  International Mathematics Research Notices, 2016.
  * [MW10] Colette Mœglin and Jean-Loup Waldspurger.  La conjecture locale de gross-prasad pour les groupes sp$\backslash$’eciaux orthogonaux: le cas g$\backslash$’en$\backslash$’eral.  arXiv preprint arXiv:1001.0826, 2010.
  * [Sai93] Hiroshi Saito.  On tunnell’s formula for characters of $gl(2)$.  Compositio Mathematica, 85(1):99–108, 1993.
  * [Ser12] Jean-Pierre Serre.  A course in arithmetic, volume 7.  Springer Science & Business Media, 2012.
  * [Shi76] Takuro Shintani.  On an explicit formula for class-1 “whittaker functions” on gln over _?? _-adic fields.  Proceedings of the Japan Academy, 52(4):180–182, 1976.
  * [Vig89] Marie-France Vignéras.  Représentations modulaires de $gl(2,f)$ en caractéristique $l$, $f$ corps $p$-adique, $p\neq l$.  Compositio Mathematica, 72(1):33–66, 1989.

Mathematics Institute, Zeeman Building, University of Warwick, Coventry CV4
7AL, England.
Email address<EMAIL_ADDRESS>