# Neopronouns: to model or not to model?

Anne Lauscher
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>
Archie Crowley
Linguistics
University of South Carolina
Columbia, SC, USA
<EMAIL_ADDRESS>
Dirk Hovy
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>

# Pronouns – to Model or Not to Model?

Anne Lauscher
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>
Archie Crowley
Linguistics
University of South Carolina
Columbia, SC, USA
<EMAIL_ADDRESS>
Dirk Hovy
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>

# Welcome to the Modern World of Pronouns – Is it Fair to Model them?

Anne Lauscher
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>
Archie Crowley
Linguistics
University of South Carolina
Columbia, SC, USA
<EMAIL_ADDRESS>
Dirk Hovy
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>

# Identity-Inclusive NLP beyond Gender
– Welcome to the Modern World of Pronouns

Anne Lauscher
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>
Archie Crowley
Linguistics
University of South Carolina
Columbia, SC, USA
<EMAIL_ADDRESS>
Dirk Hovy
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>

# Welcome to the Modern World of Pronouns:
Identity-Inclusive Natural Language Processing beyond Gender

Anne Lauscher
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>
Archie Crowley
Linguistics
University of South Carolina
Columbia, SC, USA
<EMAIL_ADDRESS>
Dirk Hovy
MilaNLP
Università Luigi Bocconi
Milan, Italy
<EMAIL_ADDRESS>

###### Abstract

_Trigger warning: This paper contains some examples which might be offensive
to some users._

The worls of pronouns is changing. From a closed class of words with few
members to a much more open set of terms to reflect identities. However,
Natural Language Processing (NLP) is barely reflecting this linguistic shift,
even though recent work outlined the harms of gender-exclusive language
technology. Particularly problematic is the current modeling 3rd person
pronouns, as it largely ignores various phenomena like neopronouns, i.e.,
pronoun sets that are novel and not (yet) widely established. This omission
contributes to the discrimination of marginalized and underrepresented groups,
e.g., non-binary individuals. However, other identity-expression phenomena
beyond gender are also ignored by current NLP technology. In this paper, we
provide an overview of 3rd person pronoun issues for NLP. Based on our
observations and ethical considerations, we define a series of desiderata for
modeling pronouns in language technology. We evaluate existing and novel
modeling approaches w.r.t. these desiderata qualitatively, and quantify the
impact of a more discrimination-free approach on established benchmark data.

##  1 Introduction

Pronouns are an essential component of many languages and often one of the
most frequent word classes. Accordingly, NLP has long studied tasks related to
them, e.g., pronoun resolution (e.g., Hobbs, 1978). Simplistically, they can
be defined as _“a word (such as I, he, she, you, it, we, or they) that is used
instead of a noun or noun phrase”_.111Essential definition provided by the
Mirriam Webster Online Dictionaly at https://www.merriam-
webster.com/dictionary/pronoun Linguistic studies have pointed out the
complexity of pronouns, though (e.g., Postal et al., 1969; McKay, 1993).
Pronouns can carry demographic information – in English, for example,
information about the _number_ of referees and a single referee’s
(grammatical) _gender_.222Grammatical, biological, and self-identified gender
should not be confounded, but are often treated interchangeably by lay
audiences. Even more information can be conveyed by pronouns in other, non-
pro-drop languages. Consider Arabana-Wangkangurru, a language spoken in
Australia, in which a speaker uses different pronouns depending on whether the
referee is part of the same social or ritual group (moiety) (Hercus, 1994). As
such, pronouns shape how we perceive individuals and can even reflect cultural
aspects (e.g., Kashima and Kashima, 1998) and ideologies (e.g., Muqit, 2012).
Consequently, pronoun usage should be considered a sensitive aspect of natural
language use.

Accordingly, in many western societies, these phenomena have been drawing more
and more attention. For instance, in 2020, the American Dialect Society voted
_“(My) Pronouns”_ as the 2019 Word of the Year and _Singular “They”_ as the
Word of the Decade (Roberts, 2020). Recently, there has been a shift in
pronoun usage (Krauthamer, 2021), partially due to shifts in the perception of
_gender_ , driven by the queer-feminist discourse (e.g., Butler, 1990, 2004).
Related to this is the open discussion of identity beyond binary gender. For
instance, a person who does not identify their gender within the gender binary
(e.g., a _nonbinary_ or _genderqueer_ person) might use _singular “they”_ as
their pronoun. Recently, the French dictionary “Le Robert” added the non-
binary pronoun _“iel”_ to its list of
words.333https://dictionnaire.lerobert.com/dis-moi-robert/raconte-moi-
robert/mot-jour/pourquoi-le-robert-a-t-il-integre-le-mot-iel-dans-son-
dictionnaire-en-ligne.html

This “social push” to respect diverse gender identities also affects aspects
of NLP. Recent studies have pointed out the potential harms from the current
lack of non-binary representation in NLP data sets, embeddings, and tasks (Cao
and Daumé III, 2021; Dev et al., 2021), and the related issue of unfair
stereotyping of queer individuals (Barikeri et al., 2021). However, the
research landscape on modern pronoun usage is still surprisingly scarce,
hindering progress for a fair and inclusive NLP.

Further linguistic research has identified identity aspects of pronouns beyond
gender (Miltersen, 2016). Specifically, _nounself_ pronouns, functionally
turning pronouns from a _closed_ to an _open_ word class. To the best of our
knowledge, these aspects have been completely ignored by NLP so far. We did
not find a single work systematically describing all of the currently existing
phenomena even just in English third-person pronoun usage (let alone other
languages).444For instance, while we found hits for the Google Scholar query
_“neopronoun”_ , we did not get any results for variants of _“nameself
pronoun”_ , or _”emojiself pronoun”_. In contrast, a fair number of
discussions are taking place in queer Wikis and forums. While it is still
unclear which of these phenomena will persist over the next decades, people
are using and discussing them, and accordingly, we as a research community
should adapt.

#### Contributions.

In this “living draft”, 1) we are the first to provide a systematic overview
of existing phenomena in English 3rd person pronoun usage. Our results will
inform future NLP research on ethical NLP and non-binary representation. We
provide the first NLP work acknowledging _otherkin_ identities. We support our
observations with a corpus analysis on Reddit. 2) Based on our overview, we
derive five desiderata for modeling third-person pronouns. Based on these, 3)
we discuss various existing and novel paradigms for _when_ and _how_ to model
pronouns. 4) Finally, we quantify the impact of discrimination-free non-
modeling of pronouns on a widely established benchmark.

##  2 Related Work

While there are some works in NLP on gender-inclusion (e.g., Dev et al., 2021)
and gender bias in static (e.g., Bolukbasi et al., 2016; Gonen and Goldberg,
2019; Lauscher et al., 2020, _inter alia_) and contextualized (e.g., Kurita et
al., 2019; Bordia and Bowman, 2019; Lauscher et al., 2021, _inter alia_)
language representations as well as works focusing on specific gender bias in
downstream tasks, e.g., natural language inference (Dev et al., 2020) and co-
reference resolution (e.g., Rudinger et al., 2018; Webster et al., 2018), we
are not aware of any work that deals with the broader field of identity-
inclusion. Thus, there is no other NLP work that deals with a larger variety
of pronouns and acknowledges pronouns as an open word class. For surveys on
the general topic of unfair bias in NLP we refer to Blodgett et al. (2020) and
Shah et al. (2020). Recently, Dev et al. (2021) pointed broadly at the harms
(Barocas et al., 2017) arising from gender-exclusivity in NLP. They surveyed
queer individuals and assessed non-binary representations in existing data set
and language representations. In contrast to them, we specifically look at
third-person pronoun usage and how to model such phenomena. Webster et al.
(2018) provide a balanced co-reference resolution corpus with a focus on the
fair distribution of pronouns but only focus on the gendered binary case.
Closest to us, Cao and Daumé III (2021) discuss gender inclusion throughout
the machine learning pipeline beyond the binary gender conception. While they
are also the first to consider non-binary pronouns, including some
neopronouns, in co-reference resolution, they do not acknowledge the broader
spectrum of identity-related pronoun phenomena.

##  3 A Note on Identity and Pronouns

This work focuses on the relationship between identity and pronouns.
_Identity_ refers to an individual’s self-concept, relating to the question of
what makes each of us unique (Maalouf, 2000). It can be seen as a two-way
process between an individual and others (Grandstrand, 1998), and relates to
different dimensions, e.g., one’s gender.

#### Gender Identity.

Gender identity, as opposed to gender expression or sex, is one’s subjective
sense of gender (Stryker, 2017). In this work, we conceptualize gender
identities beyond the binary notion (_man_ , _woman_), e.g., non-binary
gender, transgender, agender, polygender, etc.

#### Otherkin Identity.

Individuals with otherkin identity do not entirely identify as human (Laycock,
2012), e.g., vamp. Miltersen (2016) note that otherkin individuals often
identify with _nounself_ pronouns matching their kin.

Stryker (2017) highlights the strong relationship between gender identity and
pronouns. As Raymond (2016) notes, pronoun choices construct the individual’s
identity in conversations and the relationship between interlocutors.
According to Cao and Daumé III (2021), pronouns are a way of expressing
referential gender. Referring to an individual with sets of pronouns they do
not identify with, e.g., resulting in misgendering, is considered harmful (Dev
et al., 2021).

##  4 Phenomena in Third-person Pronoun-Usage

Nom. | Acc. | Poss. | Poss. | Reflexive
---|---|---|---|---
(dep.) | (indep.)
_Gendered Pronouns_
he | him | his | his | himself
she | her | her | hers | herself
_Gender-Neutral Pronouns_
they | them | their | theirs | themself
_Neopronouns_
thon | thon | thons | thons | thonself
e | em | es | ems | emself
ae | aer | aer | aers | aerself
co | co | cos | cos | coself
ve/ vi | ver/ vir | vis | vers/ virs | verself/ virself
xe | xem | xyr | xyrs | xemself
ey | em | eir | eirs | emself
e | em | eir | eirs | emself
ze | zir | zir | zirs | zirself
… |  |  |  |
_Nounself Pronouns_
star | star | stars | stars | starself
vam | vamp | vamps | vamps | vampself
… |  |  |  |
_Emojiself Pronouns_
* |  * |  *s |  *s |  *self
* |  * |  *s |  *s |  *self
… |  |  |  |
_Numberself Pronouns_
$0$ | $0$ |  $0$s |  $0$s |  $0$self
$\frac{1}{3}$ | $\frac{1}{3}$ |  $\frac{1}{3}$s |  $\frac{1}{3}$s |  $\frac{1}{3}$self
… |  |  |  |
_Nameself Pronouns_
John | John | Johns | Johns | Johnself
… |  |  |  |
Table 1: Non-exhaustive overview of phenomena related to third-person pronoun
usage in English.

We describe existing phenomena and analyze their presence in a collection of
threads from Reddit.555https://www.reddit.com

###  4.1 Existing Phenomena

Overall, individuals can choose $n$ sets of pronouns with $n\geq 0$. If $n=0$,
the individual does not identify with any singular 3rd person pronoun. If
$n>1$, the individual identifies with more than one set of pronouns, possibly
each set reflecting overlapping or non-overlapping aspects of their identity.
We provide examples of these sets in Table 1. Note that this list is non-
exhaustive and that the described phenomena are non-exclusive.

#### Gendered Pronouns.

In English, two sets of standard gendered pronouns are available,
_he/him/himself_ and _she/her/herself_.

#### Gender-Neutral Pronouns.

Given the history of generic singular _they_ in English (e.g., Who was at the
door? _They_ left a note.), there has been an uptake of singular _they_ by
non-binary individuals as a gender-netural pronoun
option666https://gendercensus.com/results/2021-worldwide-summary/ (Conrod,
2019; Konnelly and Cowper, 2020). Further, there has been increasing
institutional recognition with dictionaries and style guides supporting its
use.

#### Neopronouns.

As an alternative to the singular _they_ , individuals started creating and
sharing novel sets of 3rd person pronouns (McGaughey, 2020). More traditional
and rather well-known sets of neopronouns include, e.g., the so-called Spivak
pronouns _e/emself_ (used in (Spivak, 1990)) and related variations. During
our research, we were able to observe various subcategories of neopronouns,
partially described in the academic literature.

_Nounself Pronouns._ According to Miltersen (2016), nounself pronouns are
pronouns that are “[…] prototypically transparently derived from a specific
word, usually a noun”. Individuals may identify with certain nouns, possibly
corresponding to distinct aspects of their identity, e.g., _kitten/kittenself_
, _vamp/vampself_. The author notes the difficulty of clearly defining
nounself pronouns, neopronouns, and other phenomena. The phenomenon is assumed
to have first appeared in 2013.

_Emojiself Pronouns._ Similar to nounself pronouns, individuals may identify
with sets of emojis, possibly reflecting different aspects of their identity,
e.g., *_/_ *_self_. Emojiself pronouns are intended for written communication.
Note, that, at the time of writing this manuscript, there seem to exist no
academic description of emojiself pronouns. However, we were able to find
evidence of their existence on several social media platforms and wikis, e.g.,
Tumblr,777E.g., https://pronoun-archive.tumblr.com/post/188520170831 MOGAI
Wiki,888https://mogai.miraheze.org/wiki/Emojiself; according to the article,
the origin of emojiself pronouns is unclear but might date back to 2017
Twitter,999Example of a user complaining about LinkedIn not allowing for
emojiself pronouns in the pronoun field:
https://twitter.com/frozenpandaman/status/1412314202119700480/photo/1 and
Reddit.101010E.g.,
https://www.reddit.com/r/QueerVexillology/comments/p09nek/i_made_a_flag_for_the_emojiself_pronoun_set/

_Numberself Pronouns._ Another form of neopronouns/ nounself pronouns are
numberself pronouns. Analogous to before, we assume that here, the individual
identifies or partially identified with a number, e.g.,
_0/0/0s/0s/0self_.111111https://pronoun-
provider.tumblr.com/post/148452374817/i-think-numbers-as-pronouns-would-be-
pretty-cool

_Nameself Pronouns._ Individuals may identify with pronouns build from their
name, e.g., _John_ /_Johnself_ , overlapping with
nullpronomials.121212https://pronoun.fandom.com/wiki/Nullpronominal

#### Alternating Pronouns.

Given that people can identify with $n>1$ sets of pronouns, the pronouns they
identify with can be either equally identified-with sets, or change
potentially depending on the context (_mutopronominal_). For instance,
individuals who are also performer may use _stage pronouns_. Similarly,
genderfluid individual may identify with a certain pronoun at a certain point
in time (_pronoun fluidity_ , (Cherry-Reid, 2020)). Some individuals identify
with the pronouns of the person who is referring to them (_mirrored
pronouns_). Other individuals use set(s) of _auxiliary pronouns_ , e.g., for
situations, in which individuals referring to them have problems with using
the most identified-with sets of pronouns (e.g., in the case of emojiself
pronouns and oral communication). Note that alternating pronoun sets may be
even used in the same sentence for referring to the same
individual.131313https://www.reddit.com/r/NonBinary/comments/jasv5r/alternating_pronouns_in_same_sentence/

#### No Pronouns.

Some individuals do not identify with any pronouns. In this case, some
individuals identify most with their name being used to refer to them,
nameself pronouns, or avoid pronouns.

Figure 1: Token ranks (log-scale) and rank counts of the tokens returned
against our reflexive regular expression pattern from Reddit with example
annotations.

###  4.2 Corpus Analysis: Neopronouns in Reddit

Match |  Subreddit |  Thread Title |  Thread Excerpt
---|---|---|---
meowself |  monsterhunterrage |  _Fureedom Mewnite can die in my litterbox._ |  _I don’t like this game. But I still want meowself to play it, meow. Cause it’s fun, even though I hate it._
|  offmychest |  _Neopronouns are going too far._ |  _I get some pronouns like ze/zir, xe/xem, etc. I agree with those. But why are people using ghost/ghostself and meow/meowself? That’s really utter bullshit._
bunself |  TiADiscussion |  _I am genderfluid, pansexual, and mentally ill. I have a lot of SJW friends. AMA!_ |  _They/them pronouns are coolest with me, but I won’t be angry if you use he or she. You can use bun/buns/bunself, if you are feeling special. (That was a joke.)_
|  rpdrcirclejerk |  _Xi am so proud to announce that the new word of the year is….._ |  _–Cinnagender– which means you identify with our beloved and innocent cinnamon buns. The pronoun set is cinne/cinns/cinnself or alternatively bun/buns/bunself i am so happy to be a member of a community that ignores the oppressive gender binary, which is a social construct, i.e., it is not real_
zirself |  mypartneristrans |  _Ran into our first roadblock_ |  _I asked what I could do to help zir lowering the feeling of disphoria, and ze said zed́ maybe feel better about zirself if zed́ drink a tea._
|  Negareddit |  _No, Redditors. If you’re a horrible person online, you’re probably a horrible person offline too._ |  _Hello folks. Omg. I think this person is about to kill zirself! (emphasis on "zirself". COMEDIC GENIUS)_
Table 2: Example neopronouns and corresponding excerpts from Reddit retrieved
via our heuristic method. We slightly modified the excerpts to lower
searchability and increase the privacy of the users.

#### Setup.

We conduct an additional quantitative analysis for the presence of neopronouns
in Reddit. To this end, we use Reddit threads created between 2010 and 2021,
cleaned by previous work and provided through Huggingface Datasets
(127,445,911 lines).141414https://huggingface.co/datasets/sentence-
transformers/reddit-title-body As we are interested in capturing novel
pronouns and as the list of possible pronouns is indefinite, we proxy
neopronouns via the suffixes _self_ and _selves_ indicating the reflexive case
and match them through a regular expression. Additionally, we filter out
non-3rd person pronouns (e.g., _yourself_ , _ourselves_ , plural _themselves_)
as well as common variations of these (e.g., _urself_) and other common non-
pronoun expressions we found in the data (e.g., _do-it-yourself_). This
process leaves us with a total of 9,075 unique tokens with in total 74,768
textual mentions.

#### Results.

An initial manual analysis reveals that, unsurprisingly, many of the matches
are false positives, i.e., not real neopronouns (e.g., _non-self_ , a common
concept in Buddhist philosophy, and _tomyself_ , a common spelling mistake of
_to myself_). However, our method still finds relevant cases. Examples are
depicted in Table 2. Many discussions in which we detect nounself pronouns
center on the phenomena themselves, including, e.g., individuals stating that
they are interested in using a specific pronoun or individuals stating that
they refuse to acknowledge the phenomenon. Some discussions involve people
reporting on personal experiences and problems and seeking advice. To obtain a
quantitative view of the results, we plot the ranks (i.e., number of
occurrences of a token) against their number of tokens (Figure 1). The result
is a highly skewed Zipf’s distribution: while the highest ranks appear only
once (e.g., _themself_ with 24,697 mentions), some tokens appear only a couple
of times (e.g., the neopronoun _xemself_ with 24 mentions), and the vast
majority appears only once (e.g., many nounself pronouns such as _peachself_).

##  5 How Can and Should We Model Pronouns?

We devise five desiderata based on our previous observations, personal
experiences, and expert knowledge from interactions with LGBTQIA+ associates.
Additionally, we collect informal feedback from individuals who use gender-
neutral pronouns. We then assess how well classic and novel pronoun modeling
paradigms fulfil the five criteria.

###  5.1 Desiderata

#### D1. Refrain from assuming an individual’s identity and pronouns.

A model should not assume an individual’s identity, e.g., gender, or pronouns
based on, e.g., statistical cues about an individual’s name, also not in a
binary gender setup. Only because the name _John_ typically appears together
with the pronoun _he_ , the model should never assume that a person with the
name _John_ identifies as a man and that every _John_ uses the pronoun _he_.

#### D2. Allow for the existing sets of pronouns as well as for neopronouns.

A model should be able to handle not only the existing set of "standard"
pronouns in a language but also other existing pronouns, e.g., neopronouns.

#### D3. Allow for novel pronouns at any point in time.

On top of D2, a model should allow for novel, i.e., unseen, pronouns to appear
at any point in time. This condition is necessary to handle the fact that
neopronouns are not a fixed set, but evolving, and because related phenomena
(emojiself and nameself pronouns) turn pronouns from a _closed_ to an _open
class_ part of speech.

#### D4. Allow for multiple, alternating, and changing pronouns.

A model should not assume that the pronoun set for an individuum at time $t$
will be the same as at time $t-1$. Even within the same sequence, pronoun sets
might change.

#### D5. Provide an option to set up individuals’ sets of pronouns.

While most NLP models are trained offline and do not interact with the user,
some are designed to interact with individuals, e.g., dialog systems. In this
context, setting up individuals’ sets of pronouns can help avoid harmful
interactions (depending on the concrete sociotechnical deployment scenario).

###  5.2 Modeling Paradigms

Paradigm | D1 | D2 | D3 | D4 | D5
---|---|---|---|---|---
Classic | ✗ | ✗ | ✗ | ✗ | ✗
Bucketing | ✗ | ✓ | ✓ | ? | ✗
Delexicalization | ✓ | ✓ | ✓ | ✓ | ✗
Post-hoc | ✓ | ✓ | ✓ | ✓ | ✓
Table 3: Modeling paradigms and how they allow for fullfilling the desiderata
D1–D5.

We compare four general modeling paradigms with D1–D5 in Table 3.

#### Classic Statistical Modeling.

Traditionally, pronouns have been treated as a _closed word class_. Generally,
statistical models do not make assumptions about this (except if the
vocabulary is manually curated). However, in models exploiting cooccurences,
e.g., via word embeddings (GloVe (Pennington et al., 2014)) or deep language
models (BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019)), the models
will likely misrepresent underrepresented pronoun-related phenomena. Dev et
al. (2021) provided an initial insight by showing that singular _they_ and the
neopronouns _xe_ and _ze_ do not have meaningful vectors in GloVe and BERT.

#### Bucketing.

One option, previously discussed by Dev et al. (2021), is to apply bucketing,
i.e., to decide on a fixed number of majority classes, e.g., _male_ pronouns,
_female_ pronouns, and one or multiple classes for the “rest of the pronouns”,
e.g., _other_. The advantage of this approach is that it can map existing and
novel pronouns to the _other_ class. However, it still makes identity
assumptions – and due to unequal representations of _main_ and _other_
classes, it will inevitably lead to discrimination.

#### No Modeling – Delexicalization.

Given that the classic approach and bucketing both lead to unfair treatment of
underrepresented groups, the alternative is to explicitly not model pronouns
in their surface forms. This process, commonly named delexicalization, has
proved helpful for other tasks where models capture misleading lexical
information, e.g., fact verification (e.g., Suntwal et al., 2019), or
resource-lean scenarios, e.g., cross-lingual parsing (e.g., McDonald et al.,
2011).151515In fact, accounting for novel pronouns and novel ways of using
pronouns is a resource-lean scenario. In this case, the model is forced to not
rely on spurious lexical cues related to gender, e.g., that _John_ occurs most
often with the pronoun _he_. Instead, the model learns a single representation
for all pronouns and relies on other task-related conceptual and commonsense
information for disambiguation.

#### Post-hoc Injection of Modeling Information/ Modeling at Test Time.

For human-to-human interactions, several LGBTQIA+ guides recommend to (1)
first try generic pronouns (e.g., singular _they_), and (2) switch to other
sets of pronouns once the conversation partner communicates them. For uncommon
or novel pronouns, several web pages have explicitly been set up for
practising how to use them.161616E.g.,
https://www.practicewithpronouns.com/#/?_k=66emp7 In this work, _we propose
that NLP systems should work similarly_ – if technically possible and
depending on the concrete sociotechnical deployment scenario. To this end, we
can use intermediate training procedures (e.g., Hung et al., 2021) for
pronoun-related model refinement. E.g., we can use synthetic data created
through similar procedures as the ones employed on these websites. Another
option is only model pronouns at test time, e.g., through simple replacement
procedures.

##  6 How Much Would We Loose?

In §5.2, we discussed delexicalization, i.e., not modeling lexical surface
forms of pronouns, as one way to counter exclusion in statistical modeling and
bucketing. However, a possible counter-argument against this approach is that
omitting the surface forms will lead to poor model performance on pronoun-
related tasks. We experimentally quantify the loss from (fairer)
delexicalization compared to statistical modeling in co-reference resolution.

###  6.1 Experimental Setup

| Train | Dev | Test | Total
---|---|---|---|---
PRP | 64,476 | 7,881 | 8,067 | 80,424
PRP$ | 14,535 | 1,783 | 1,935 | 18,253
Total | 79,011 | 9,664 | 10,002 | 98,677
Table 4: Number of pronoun replacements in the training, development, and test portion of OntoNotes 5.0 for PRP and PRP$, respectively. | MUC | CEAFϕ4 | B3 | AVG
---|---|---|---|---
| P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1
(Dobrovolskii, 2021) | 84.9 | 87.9 | 86.3 | 76.1 | 77.1 | 76.6 | 77.4 | 82.6 | 79.9 | – | – | 81.0
\- reproduction | 84.7 | 87.5 | 86.1 | 75.6 | 76.7 | 76.1 | 77.2 | 82.0 | 79.5 | 79.2 | 82.1 | 80.6
\- replace test set | 69.7 | 70.7 | 70.2 | 63.2 | 49.1 | 55.2 | 50.1 | 56.1 | 52.9 | 61.0 | 58.6 | 59.4
$\Delta_{\mathit{repl.test-repr.}}$ | -15.0 | -16.8 | -15.9 | -12.4 | -27.6 | -20.9 | -27.1 | -25.9 | -26.6 | -18.2 | -23.5 | -21.2
\- replace all | 81.6 | 83.1 | 82.4 | 73.08 | 72.9 | 73.0 | 72.3 | 75.3 | 73.7 | 75.7 | 77.1 | 76.4
$\Delta_{\mathit{repl.all-repr.}}$ | -3.1 | -4.4 | -3.7 | -2.5 | -3.8 | -3.1 | -4.9 | -6.7 | -5.8 | -3.5 | -5.0 | -4.2
Table 5: Results of the delexicalization experiment. We report the results of
the RoBERTa large-based word-level co-reference resolution model as reported
by Dobrovolskii (2021), our reproduction, as well as variants trained and/ or
tested on versions of the data set in which we replace the pronouns. All
scores were produced using the offical CoNLL-2012 scorer. We report precision
(P), recall (R), and F1-score (F1) for MUC, CEAFϕ4, and B3, respectively, as
well as the averages (AVG). The rows highlighted in gray indicate the obtained
losses.

#### Task, Dataset, and Measures.

We resort to co-reference resolution, a task where knowledge about pronouns
and related gender identity assumptions play an important role. We use the
English portion of the _OntoNotes 5.0_ dataset (Weischedel et al., 2012),
which consists of texts annotated with co-reference information across five
domains (news, conversational telephone speech, weblogs, usenet newsgroups,
broadcast, and talk shows). We prepare three variants: (i) the first version
consists of the plain original data; (ii) in the second variant, we _replace
all pronouns_ in the test set with the respective part-of-speech token,
according to the Penn Treebank Project (Santorini, 1990), i.e., PRP for
personal pronouns, and PRP$ for possessive pronouns. Finally, we provide a
version (iii) in which we replace pronouns in the train, dev, and test splits.
Note that our strategy is pessimistic, as we also replace non-3rd person
pronouns, i.e., _I_ , _you_ , _ourselves_ , etc. We show the number of
replacements in Table 4. For scoring, we use the official CoNLL-2012 scorer
(Pradhan et al., 2012). We report the results in terms of MUC (Vilain et al.,
1995), B3 (Bagga and Baldwin, 1998), and CEAFϕ4 (Luo, 2005) precision, recall,
and F1-measure, as well as the averages across these scores.

#### Models and Baselines.

We want to obtain an intuition about the tradeoffs in the delexicalization
setup, not to outperform previous results. For this reason, we resort to the
recently proposed word-level co-reference model (Dobrovolskii, 2021), a highly
efficient model competitive with the state-of-the-art. The model consists of a
separate co-reference resolution module and a separate span extraction module.
In an initial step, we compute token representations from a Transformer
(Vaswani et al., 2017)-based encoder through aggregation of initial
representations via learnable weights. In a next step, we compute co-reference
relationships. To this end, the token representations are passed into an
antecedent pruning procedure based on a bilinear scoring function for
obtaining $k$ antecedent candidates for each token through coarse-grained
scoring. Then an additional feed-forward neural network computes finer-grained
scores. The final antecedent score is the sum of these two scores. We select
the candidate with the highest score as the antecedent. Negative scores
indicate no antecent for a token. Tokens assumed to be part of a co-reference
relationship are passed into the span extraction module. The module consists
of an additional feed-forward network, which is followed by convolutions with
two output channels (for start and end scores). For further details see the
original work. Our baseline is the model trained and evaluated on the original
OntoNotes portions (_reproduction_). We compare with the evaluation of this
model on the pronoun-replaced test set (_replace test set_) and a version of
this model trained on the replaced training set and evaluated on the replaced
test set, respectively (_replace all_).

#### Model Configuration, Training, and Optimization.

We choose RoBERTa large (Liu et al.,
2019)171717https://huggingface.co/roberta-large as the base encoder and fix
all other hyperparameters to the ones provided in the original implementation
of Dobrovolskii (2021): the window size is set to $512$ tokens, dropout rate
to $0.3$, the learning rate of the encoder is set to $1\cdot 10^{-5}$ and of
the task-specific layers to $3\cdot 10^{-4}$, respectively. We train the co-
reference module with a combination of the negative log marginal likelihood
and binary cross-entropy as an additional regularization factor (weight set to
$0.5$). The span extraction module is trained using cross-entropy loss. We
optimize the sum of the two losses jointly with Adam (Kingma and Ba, 2015) for
20 epochs and apply early stopping based on validation set performance (word-
level F1) with a patience of 3 epochs.

###  6.2 Results and Discussion

We show the results in Table 5. We are roughly able to reproduce the results
reported by (Dobrovolskii, 2021), confirming the effectiveness of their
approach and the validity of our experimental setup. When we replace pronouns
in the test set, the results drop massively, with up to $-27.6$ percentage
points CEAFϕ4 recall. On average, the results drop by $21.2$ percentage points
in F1-measure. This decrease demonstrates the heavy reliance of this model on
the lexical surface forms of the pronoun sets seen in the training. However,
when we replace the pronouns in the training portion of OntoNotes with the
special tokens, we can mitigate these losses by a large margin (losses up to
$-5.8$ B3 F1, and on average $-4.2$ F1). These results are highly encouraging,
given that a) we replaced _all_ pronouns, including non-third person pronouns,
and b) the model has not been trained on these placeholders in the pretraining
phase. The model can not rely on possibly discriminating correlations between
names or occupations and pronoun sets and will represent neopronouns the same
way as it will represent established pronoun sets. So a delexicalization
approach can increase fairness in co-reference resolution and retain high
system performance, as we can expect even smaller drops from a more careful
selection of replacements.

##  7 Conclusion

This work provides an initial overview of the plethora of current phenomena in
3rd person pronoun usage in the English language. For practical and ethical
reasons, the NLP community should acknowledge the broad spectrum of possible
identities and the respective manifestations in written and oral
communication. Especially since many emerging phenomena are still under-
researched, and even while it remains to be seen if and how these become more
established ways of referring to individuals. Language is consistently
evolving, and NLP researchers and practitioners should account for this to
provide genuinely inclusive systems. Notably, pronouns, traditionally handled
as a close class of words, currently seem to function closer to an open class.
Based on our observations, which originate from literature research, research
in non-academic publicly available writing, as well as a corpus study, we have
defined a series of five desiderata and applied those to the discussion of
existing and novel modeling paradigms. In this context, we raised the
questions _when_ and _how_ to model pronouns and whether and how to _include
users_ in these decisions. We consider this document an initial and living
draft and hope to start a broader discussion on the topic. Our study can
inform future NLP research and beyond and serve as a starting point for
creating novel modeling procedures. In the future, we will look at pronoun-
related issues within concrete tasks and in multilingual scenarios.

## Acknowledgments

The work of Anne Lauscher and Dirk Hovy is funded by the European Research
Council (ERC) under the European Union’s Horizon 2020 research and innovation
program (grant agreement No. 949944, INTEGRATOR). We thank Emily Bender and
Chandler May for sharing their ideas related to our project.

## Further Ethical Discussion

We have described and experimented with phenomena related to third-person
pronouns focusing on the English language only. Naturally, this work comes
with several limitations. For instance, while we pointed the reader to the
variety of pronoun-related phenomena in other languages, a thorough
_multilingual and cross-lingual discussion_ would have exceeded the scope of
this manuscript. This lacuna includes the discussion of neopronouns in other
languages. Similarly, while we acknowledged identities beyond the binary
gender as well as otherkin identities, due to our focus on pronouns, we did
not investigate _other identity-related terms_. This aspect includes their
handling in NLP and the range of issues related to identity-exclusivity.
Finally, at the current state of the manuscript, the desiderata discussed are,
as reported, based on our expert knowledge, our activities within the LGBTQIA+
community, and informal exchanges with individuals using gender-neutral
pronouns. In the future, we will validate these assumptions through a
structured survey to present a more inclusive perspective on the discussed
issues.

## References

  * Bagga and Baldwin (1998) Amit Bagga and Breck Baldwin. 1998.  Algorithms for scoring coreference chains.  In _Proc. Linguistic Coreference Workshop at the first Conf. on Language Resources and Evaluation (LREC)_ , pages 563–566, Granada, Spain.
  * Barikeri et al. (2021) Soumya Barikeri, Anne Lauscher, Ivan Vulić, and Goran Glavaš. 2021.  RedditBias: A real-world resource for bias evaluation and debiasing of conversational language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1941–1955, Online. Association for Computational Linguistics.
  * Barocas et al. (2017) Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017.  The problem with bias: Allocative versus representational harms in machine learning.  In _9th Annual Conference of the Special Interest Group for Computing, Information and Society_.
  * Blodgett et al. (2020) Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020.  Language (technology) is power: A critical survey of “bias” in NLP.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5454–5476, Online. Association for Computational Linguistics.
  * Bolukbasi et al. (2016) Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. 2016.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  In _Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain_ , pages 4349–4357.
  * Bordia and Bowman (2019) Shikha Bordia and Samuel R. Bowman. 2019.  Identifying and reducing gender bias in word-level language models.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop_ , pages 7–15, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Butler (1990) Judith Butler. 1990.  _Gender trouble_ , 1st edition.  Routledge Classics, New York, NY, USA.
  * Butler (2004) Judith Butler. 2004.  _Undoing Gender_ , 1st edition.  Routledge, New York, NY, USA.
  * Cao and Daumé III (2021) Yang Trista Cao and Hal Daumé III. 2021.  Toward gender-inclusive coreference resolution: An analysis of gender and bias throughout the machine learning lifecycle*.  _Computational Linguistics_ , 47(3):615–661.
  * Cherry-Reid (2020) Katharine A Cherry-Reid. 2020.  _Music to Our Ears: Using a Queer Folk Song Pedagogy to do Gender and Sexuality Education_.  Ph.D. thesis, University of Toronto (Canada).
  * Conrod (2019) Kirby Conrod. 2019.  _Pronouns Raising and Emerging_.  Ph.D. thesis, University of Washington.
  * Dev et al. (2020) Sunipa Dev, Tao Li, Jeff M. Phillips, and Vivek Srikumar. 2020.  On measuring and mitigating biased inferences of word embeddings.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 7659–7666. AAAI Press.
  * Dev et al. (2021) Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff Phillips, and Kai-Wei Chang. 2021.  Harms of gender exclusivity and challenges in non-binary representation in language technologies.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 1968–1994, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dobrovolskii (2021) Vladimir Dobrovolskii. 2021.  Word-level coreference resolution.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7670–7675, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Gonen and Goldberg (2019) Hila Gonen and Yoav Goldberg. 2019.  Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 609–614, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Grandstrand (1998) Ove Grandstrand. 1998.  Identity and deception in the virtual community.  In Peter Kollock and Marc Smith, editors, _Communities in Cyberspace_ , 1st edition, chapter 2. Routledge, London, UK.
  * Hercus (1994) Luise A Hercus. 1994.  _A grammar of the Arabana-Wangkangurru language, Lake Eyre Basin, South Australia (Pacific linguistics. Series C)_ , 1st edition.  Dept. of Linguistics, Research School of Pacific and Asian Studies, Australian National University.
  * Hobbs (1978) Jerry R Hobbs. 1978.  Resolving pronoun references.  _Lingua_ , 44(4):311–338.
  * Hung et al. (2021) Chia-Chien Hung, Anne Lauscher, Simone Paolo Ponzetto, and Goran Glavaš. 2021\.  DS-TOD: Efficient domain specialization for task oriented dialog.  _arXiv preprint arXiv:2110.08395_.
  * Kashima and Kashima (1998) Emiko S. Kashima and Yoshihisa Kashima. 1998.  Culture and language: The case of cultural dimensionsand personal pronoun use.  _Journal of Cross-Cultural Psychology_ , 29(3):461–486.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Konnelly and Cowper (2020) Lex Konnelly and Elizabeth Cowper. 2020.  Gender diversity and morphosyntax: An account of singular they.  _Glossa: a journal of general linguistics_ , 5(1).
  * Krauthamer (2021) Helene Seltzer Krauthamer. 2021.  _The Great Pronoun Shift: The Big Impact of Little Parts of Speech_ , 1st edition.  Routledge.
  * Kurita et al. (2019) Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019.  Measuring bias in contextualized word representations.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 166–172, Florence, Italy. Association for Computational Linguistics.
  * Lauscher et al. (2020) Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vulić. 2020\.  A general framework for implicit and explicit debiasing of distributional word vector spaces.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 8131–8138.
  * Lauscher et al. (2021) Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021.  Sustainable modular debiasing of language models.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 4782–4797, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Laycock (2012) Joseph P Laycock. 2012.  “We are spirits of another sort”: Ontological rebellion and religious dimensions of the otherkin community.  _Nova Religio: The Journal of Alternative and Emergent Religions_ , 15(3):65–90.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Luo (2005) Xiaoqiang Luo. 2005.  On coreference resolution performance metrics.  In _Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing_ , pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics.
  * Maalouf (2000) Amin Maalouf. 2000.  _On identity_ , 1st, translated by barbara bray edition.  Vintage.
  * McDonald et al. (2011) Ryan McDonald, Slav Petrov, and Keith Hall. 2011.  Multi-source transfer of delexicalized dependency parsers.  In _Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing_ , pages 62–72, Edinburgh, Scotland, UK. Association for Computational Linguistics.
  * McGaughey (2020) Sebastian McGaughey. 2020.  Understanding neopronouns.  _The Gay & Lesbian Review Worldwide_, 27(2):27–29.
  * McKay (1993) John C McKay. 1993.  On the term “pronoun” in italian grammars.  _Italica_ , 70(2):168–181.
  * Miltersen (2016) Ehm Hjorth Miltersen. 2016.  Nounself pronouns: 3rd person personal pronouns as identity expression.  _Journal of Language Works-Sprogvidenskabeligt Studentertidsskrift_ , 1(1):37–62.
  * Muqit (2012) Abd Muqit. 2012.  Ideology and power relation reflected in the use of pronoun in osama bin laden’s speech text.  _International Journal of Social Science and Humanity_ , 2(6):557.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Postal et al. (1969) Paul Postal, David A Reibel, and Sanford A Schane. 1969.  On so-called pronouns in english.  _Readings in English transformational grammar_ , pages 12–25.
  * Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012.  Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes.  In _Joint Conference on EMNLP and CoNLL-Shared Task_ , pages 1–40.
  * Raymond (2016) Chase Wesley Raymond. 2016.  Linguistic reference in the negotiation of identity and action: Revisiting the t/v distinction.  _Language_ , 92:636–670.
  * Roberts (2020) Julie Roberts. 2020.  2019 word of the year is “(my) pronouns,” word of the decade is singular “they” as voted by american dialect society.  Press Release, American Dialect Society.
  * Rudinger et al. (2018) Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018.  Gender bias in coreference resolution.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 8–14, New Orleans, Louisiana. Association for Computational Linguistics.
  * Santorini (1990) Beatrice Santorini. 1990.  Part-of-speech tagging guidelines for the penn treebank project.
  * Shah et al. (2020) Deven Santosh Shah, H. Andrew Schwartz, and Dirk Hovy. 2020.  Predictive biases in natural language processing models: A conceptual framework and overview.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5248–5264, Online. Association for Computational Linguistics.
  * Spivak (1990) Michael Spivak. 1990.  _The Joy of TeX: A Gourmet Guide to Typesetting with the AMSTeX Macro Package_ , 2nd edition.  American Mathematical Society.
  * Stryker (2017) Susan Stryker. 2017.  _Transgender history: The roots of today’s revolution_ , 2nd edition.  Seal Press.
  * Suntwal et al. (2019) Sandeep Suntwal, Mithun Paul, Rebecca Sharp, and Mihai Surdeanu. 2019.  On the importance of delexicalization for fact verification.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3413–3418, Hong Kong, China. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Vilain et al. (1995) Marc Vilain, John D Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995.  A model-theoretic coreference scoring scheme.  In _Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995_.
  * Webster et al. (2018) Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018.  Mind the GAP: A balanced corpus of gendered ambiguous pronouns.  _Transactions of the Association for Computational Linguistics_ , 6:605–617.
  * Weischedel et al. (2012) Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Nianwen Xue, Martha Palmer, Jena D Hwang, Claire Bonial, et al. 2012.  Ontonotes release 5.0.


April, 2022

“fourteenitGeneralized Rogers Ramanujan Expressions for Some Non–Singlet
Twisted Affine Algebras.

“fourteenitDoron Gepner

“twelveit Department of Particle Physics and Astrophysics, Weizmann Institute,
“twelveit Rehovot 76100, Israel

ABSTRACT

Hatayama et al. described generalized Rogers–Ramanujan (GRR) expressions for
the string functions of the singlet representation of twisted affine algebras.
We give here such GRR expressions for some non-singlet string functions. In
the case of the algebra $A_{2}^{(2)}$ this gives all the string functions. We
verify these expressions using Freudenthal–Kac formula.

Generalized Rogers–Ramanujan (GRR) identities are of great mathematical
interest. In physics they appeared as expressions for the characters of
two–dimensional conformal field theories $[\twelveit1]$. Of special interest
are the conformal field theories associated to Lie groups called generalized
parafermions $[\twelveit2]$. The characters of these theories were shown there
to be given by the string functions of affine Lie algebras. These are
generating functions of a string of weights in some representation
$[\twelveit3]$. Thus, we seek GRR identities for the string functions of
affine algebras. In a seminal work, Lepowsky and Primc found such expressions
for $SU(2)$ $[\twelveit4]$. Later, GRR expressions for the untwisted affine
algebras were given for the singlet representation $[\twelveit5]$, and
generalized to some other representations in $[\twelveit6]$. These GRR
expressions, for the untwisted algebras, were recently proved in
$[\twelveit7]$. The GRR for the twisted affine algebras and the singlet
representation were given by Hatayama et al. $[\twelveit8]$. These GRR
expressions were just proved for all the twisted algebras except
$A_{2l}^{(2)}$ $[\twelveit9]$, and for the singlet representation. Our purpose
here is to give expressions for some other representations of the twisted
affine algebras. We find such GRR expressions for some non–singlet
representations. In the case of $A_{2}^{(2)}$ algebra we find all the string
functions. We verify our conjecture for low grades using the computer program
ALGEBRA, which is based on Freudenthal–Kac formula $[\twelveit3]$. In the case
of singlet representations this verification was done in $[\twelveit10]$.

The twisted affine algebras were classified by Kac $[\twelveit3]$. The degree
of the twist is given by $r=2$ or $3$. The algebras are:

$A_{2}^{(2)}:\,A_{1},\quad A_{2l}^{(2)}:\,B_{l},\quad
A^{(2)}_{2l-1}:\,C_{l},\quad D^{(2)}_{l+1}:\,B_{l},\quad
E_{6}^{(2)}:\,F_{4},\quad D_{4}^{(3)}:\,G_{2},$ $None$

where the superscript of the algebra is the degree of the twist $r$ and the
algebra on the right is the algebra invariant under the twist, $G_{\bar{0}}$.
We denote by $\alpha_{a}$ the simple roots of $G_{\bar{0}}$. The scalar
product matrix is normalized so that the long root square is $2r$. The basis
that we choose for the simple roots is as follows:

$A_{2}^{(2)}:\,\alpha_{1}=2\epsilon_{1},$ $None$ $A_{2l}^{(2)}(l\geq
2):\,\alpha_{i}=\sqrt{2}(\epsilon_{i}-\epsilon_{i+1}),\quad
i=1,2,\ldots,l-1,\quad\epsilon_{l}=\sqrt{2}\epsilon_{l},$ $None$

$A^{(2)}_{2l-1}(l\geq 3):\,\alpha_{i}=\epsilon_{i}-\epsilon_{i+1},\quad
i=1,2,\ldots,l-1,\quad\alpha_{l}=2\epsilon_{l},$ $None$ $D^{(2)}_{l+1}(l\geq
2):\,\alpha_{i}=\sqrt{2}(\epsilon_{i}-\epsilon_{i+1}),\quad
i=1,2,\ldots,l-1,\quad\alpha_{l}=\sqrt{2}\epsilon_{l},$ $None$
$E_{6}^{(2)}:\,\alpha_{i}=\sqrt{2}\\{{1\over
2}(\epsilon_{1}-\epsilon_{2}-\epsilon_{3}-\epsilon_{4}),\epsilon_{4},\epsilon_{3}-\epsilon_{4},\epsilon_{2}-\epsilon_{3}\\},$
$None$
$D_{4}^{(3)}:\alpha_{1}=\epsilon_{1}-\epsilon_{2},\quad\alpha_{2}=-2\epsilon_{1}+\epsilon_{2}+\epsilon_{3},$
$None$

where $\epsilon_{s}$, $s=1,2,\ldots$ are orthogonal unit vectors.

We define the array,

$K_{i,j}^{a,b}={1\over 2}\alpha_{a}\cdot\alpha_{b}[\min(i,j)-ij/m],$ $None$

where $m$ is the level. We also use $a_{0}$ which is equal to one except for
the algebra $A_{2l}^{(2)}$, $l=1,2,\ldots$, which is set equal to two. We
define

$t_{a}=\max(\alpha_{a}^{2},a_{0}).$ $None$

Denote by $M$ the root lattice spanned over the integers by
$\alpha_{a},\,a=1,2,\ldots,l$. We define the linear map

${\twelvebfI}(\alpha_{a})=\delta_{a}\alpha_{a},$ $None$

where $\delta_{a}$ is equal to one except for the algebra $A_{2l}^{(2)}$ and
$a=l$ (the long root) where it is equal to two. We use the Pochhammer symbol,

$(q)_{n}=\prod_{j=1}^{n}(1-q^{j}),$ $None$

where $n$ is an integer. Let $q_{a}=q^{t_{a}}$. We use the set of integers,
$n_{j}^{(a)}$, where $a=1,2,\ldots,l$ and $j=1,2,\ldots,m-1$. We set

$(q)_{\twelvebfn}=\prod_{a=1}^{l}\prod_{j=1}^{m-1}(q_{a})_{n_{j}^{(a)}}.$
$None$

The string functions of the twisted affine algebras are defined by the sum
$[\twelveit3]$,

$c_{\lambda}^{\Lambda}(\tau)=q^{d}\sum_{n=0}^{\infty}p_{n}e^{2\pi i\tau n},$
$None$

where $d$ is some dimension, $q=\exp(2\pi i\tau)$ and $p_{n}$ is the
multiplicity of a weight at grade $n$ and at the highest weight representation
$\Lambda$ and the finite weight $\lambda$. Here $i=\sqrt{-1}$ and the level is
denoted by $m$.

We can state now our main result which is a Rogers–Ramanujan type expression
for the string functions. We define the sum,

$N_{m,\lambda}^{\Lambda}(q)=\prod_{a=1}^{l}(q_{a})_{\infty}^{-1}\sum_{n_{j}^{(a)}\geq
0}{q^{{\twelvebfL}({\twelvebfn})+{\twelvebfV}({\twelvebfn})}\over(q)_{\twelvebfn}},$
$None$

where

${\twelvebfL}({\twelvebfn})=\sum_{a,b=1}^{l}\sum_{i,j=1}^{m-1}n_{i}^{(a)}n_{j}^{(b)}K_{i,j}^{a,b}.$
$None$

The sum is limited to the integers $n_{j}^{(a)}$ which obey,

$\sum_{a=1}^{l}\sum_{j=1}^{m-1}j\alpha_{a}n_{j}^{(a)}={\twelvebfI}(\lambda-\Lambda)\mathop{\twelveitmod}\nolimits
mM,$ $None$

where the linear map $\twelvebfI$ was defined in eq. (10).

The string functions are given by,

$c_{\lambda}^{\Lambda}(q)=N_{m,\lambda}^{\Lambda}(q^{1/a_{0}}),$ $None$

For the $\Lambda$ and ${\twelvebfV}({\twelvebfn})$ which will be specified
below in three cases.

Case 1: Here we take $\Lambda=0$, the singlet representation and
${\twelvebfV}({\twelvebfn})=0$, for all the twisted algebras. This is the
expression of Hatayama et al. $[\twelveit8]$.

Case 2: We need the matrix which is the inverse Cartan matrix of $SU(m)$,
where $m$ as before is the level,

$B_{i,j}=\min(i,j)-ij/m,$ $None$

Here, $i,j=0,1,\ldots,m-1$. We have an expression for the string functions for
all the short weights which have a mark one. For the algebra $A_{2l-1}^{(2)}$
this is the weight $\Lambda=r\Lambda_{1}$, where $\Lambda_{1}$ is the first
fundamental weight. We set in this case $a=1$. The other case is the algebra
$D_{l+1}^{(2)}$. Here we take $\Lambda=r\Lambda_{l}$. Here we set $a=l$. These
two algebras are the only ones which have such roots. For these weights we
have the GRR expression, eq. (14), where we choose,

${\twelvebfV}({\twelvebfn})=-\sum_{i=1}^{m-1}B_{m-r,i}n_{i}^{(a)}.$ $None$

Case 3: We consider the algebra $A_{2l}^{(2)}$, where $l=1,2,\ldots$. We take
for the weight, $\Lambda=r\Lambda_{l}$, which corresponds to the long root of
the algebra, and $r=1,2,\ldots,m/2$. We define the vector,

$F_{i}=\min(i,r)-i/2+s_{i},$ $None$

where $s_{i}$ is ${1\over 2}$ for odd $i$ and is zero otherwise. Here we take
$i=1,2,\ldots 2r-1$ and we take $F_{i}=0$ otherwise. We define the vector,

$M_{i}=F_{i}-B_{r,i},$ $None$

and we take

${\twelvebfV}({\twelvebfn})=2\sum_{i=1}^{m-1}M_{m-i}n_{i}^{(l)}.$ $None$

This completes our description of the GRR for the twisted affine algebras. The
expression holds for all the $\lambda$, in eq. (14) and the $\Lambda$
specified above. For the algebra $A_{2}^{(2)}$ (case 3), eq. (14) gives all
the string functions.

To verify our conjectures (cases 2 and 3), we use Freudenthal– Kac formula
$[\twelveit3]$, which we computerized in the fortran program ALGEBRA (written
in a collaboration with A. Abouelsaud).

Let us give an example of $A_{2}^{(2)}$ with $\Lambda=\Lambda_{1}$ at level
$m=3$ (case 3). Here the matrix $K$ is

$K={1\over 3}\pmatrix{2&1\cr 1&2\cr}.$ $None$

Then, our conjecture for the string functions becomes, for $\lambda=\Lambda$,

$(q)_{\infty}\chi^{\Lambda}_{\Lambda}(q)=\sum_{m_{1},m_{2}\geq 0\atop
m_{1}+2m_{2}=0\mathop{\twelveitmod}\nolimits
3}{q^{\\{m_{1},m_{2}\\}.K.\\{m_{1},m_{2}\\}/2+(m_{1}-m_{2})/3}\over(q)_{m_{1}}(q)_{m_{2}}},$
$None$

Using a Mathematica program we can compute the expression, eq. (24), up to
order 8 and we find

$\chi_{\Lambda}^{\Lambda}(q)=1+2q+6q^{2}+12q^{3}+26q^{4}+48q^{5}+90q^{6}+156q^{7}+270q^{8}+\ldots.$
$None$

From the program ALGEBRA we find the exact same expression up to grade 8.

Let us give an example of the algebra $D_{3}^{(2)}$ at level two with
$\Lambda=\Lambda_{2}$ (case 2), and $\lambda=\Lambda$. The matrix of scalar
products is in this example,

$K=\pmatrix{4&-2\cr-2&2\cr}.$ $None$

From the general conjecture, eq. (14), we find for the character

$(q)_{\infty}(q^{2})_{\infty}\chi^{\Lambda}_{\Lambda}(q)=\sum_{m_{1},m_{2}\geq
0\atop m_{1},m_{2}=0\mathop{\twelveitmod}\nolimits
2}{q^{\\{m_{1},m_{2}\\}.K.\\{m_{1},m_{2}\\}/4-m_{2}/2}\over(q^{2})_{m_{1}}(q)_{m_{2}}},$
$None$

Using Mathematica, this expression is evaluated to

$\chi_{\Lambda}^{\Lambda}(q)=1+3q+8q^{2}+19q^{3}+41q^{4}+83q^{5}+161q^{6}+299q^{7}+\ldots.$
$None$

Using the program ALGEBRA we find the exact same expression for the string
function, up to grade $7$.

Acknowledgements:

I thank A. Abouelsaud for his collaboration in the program ALGEBRA and J.
Ramos for remarks on the manuscript.

REFERENCES @plus 1em minus @relax1 A.A. Belavin, A.M. Polyakov and A.B.
Zamolodchikov, Nucl. Phys. B 241 (1984) 333. relax2 D. Gepner, Nucl. Phys. ‘B
290 (1987) 10. relax3 V.G. Kac, “Infinite dimensional Lie algebras”, Cambridge
University Press (1990) relax4 J. Lepowsky and M. Primc, Contemporary
Mathematics, Vol. 46 AMS (1985). relax5 A. Kuniba, T. Nakanishi and J. Suzuki,
Mod.Phys.Lett. A 8 (1993) 1649. relax6 D. Gepner, Lett. Math. Phys. 105 (2015)
6, 769. relax7 M. Butorac, S. Kozic and M. Primc, Math. Zeit. 298 (2021) 1003.
relax8 G. Hatayama, A. Kuniba, M. Okado, T. Takagi and Z. Tsuboi, Prog. Math.
Phys. 23 (2002) 205. relax9 M. Okado and R. Takenaka, arXiv: 2109.08892
(2021). relax10 A. Genish and D. Gepner, Mod. Phys. Lett. A, Vol. 32, No. 21
(2017) 1750110.