# Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in
Human-Bot Interactions

Paras Bhatt  Anthony Rios
Department of Information Systems and Cyber Security
University of Texas at San Antonio
{Paras.Bhatt<EMAIL_ADDRESS>

###### Abstract

Language generation models’ democratization benefits many domains, from
answering health-related questions to enhancing education by providing AI-
driven tutoring services. However, language generation models’ democratization
also makes it easier to generate human-like text at-scale for nefarious
activities, from spreading misinformation to targeting specific groups with
hate speech. Thus, it is essential to understand how people interact with bots
and develop methods to detect bot-generated text. This paper shows that bot-
generated text detection methods are more robust across datasets and models if
we use information about how people respond to it rather than using the bot’s
text directly. We also analyze linguistic alignment, providing insight into
differences between human-human and human-bot conversations.

##  1 Introduction

Bots are useful in a wide variety of applications areas including business
Kaczorowska-Spychalska (2019), education Kerlyl et al. (2006), and health
Yadav et al. (2019); Liednikova et al. (2020). For instance, Yadav et al.
(2019) studied the use of chatbots as a drop-in first-point-of-contact for
women in India seeking breastfeeding information. Similarly, researchers have
studied bots to answer COVID-19-related questions Oniani and Wang (2020) and
screen individuals for risks of contracting the virus Martin et al. (2020).
Overall, the wide availability of software packages, tools, and pre-trained
models has democratized the creation of bots.

Even with the increasing interest in bots for social good (e.g., COVID-related
chatbots), there is still a concern regarding their abuse to spread
misinformation, be used for targeted discrimination, deceive users, and
perform fraud Daniel et al. (2019). Given the potential good and harm bots can
create, it is essential to study how the bots should act and how people do
interact with such bots for specific applications.

Automatically identifying bots online is well studied Garcia-Silva et al.
(2019); Herzig et al. (2019); Kosmajac and Keselj (2019); Ippolito et al.
(2020); Jawahar et al. (2020). Bot detection methods rely on two forms of
information Orabi et al. (2020): behavior and content. Behavior relates to
measuring how often bots post, the time posts are created, and conversational
network structures Beskow and Carley (2018). Content involves using the bot’s
text directly. Focusing on text suffers from generalization issues, making it
challenging to detect bots that discuss different topics or operate in
different domains. Likewise, behavioral approaches assume that bots will
behave differently than people at a superficial level, such as posting more
often than humans. Even in the network analysis of conversation structures
Beskow and Carley (2018), many of the human-bot interactions on social media
are not human-like interactions. For instance, it is understandable that
humans will interact with a bot that converts pounds to kilograms differently
than other humans. Given bots’ current use-cases ranging from counselors to
healthcare information providers, it is vital to understand how humans and
bots interact beyond trivial applications.

Before looking at how humans interact with bots, it is essential to understand
how humans interact. This paper focuses on Communication Accommodation Theory
(CAT) and general language use to analyze interactions. CAT is used to study
language use in various domains to understand human behavior Giles et al.
(1973); Tausczik and Pennebaker (2010). More specifically, we study linguistic
accommodation—where speakers come to talk more (or less) similarly as they
interact—which has been analyzed in mental health support on Reddit, showing a
positive link with informational and emotional support Sharma and De Choudhury
(2018). The alignment of two people in terms of linguistic style has
positively predicted successful outcomes of negotiations Taylor and Thomas
(2008), and doctors are recommended to accommodate perspective to improve
patient care Kline and Ceropski (1984); Wood (2019). The way people use and
accommodate pronouns can indicate power, where high-status individuals use “I”
less and “you/we” words more than low-status individuals Kacewicz et al.
(2014).

Linguistic accommodation in human-bot interactions has been studied in system
design, showcasing that there is a strong link between user experience and
language style of the system Chaves et al. (2019); Chaves (2020); Thomas et
al. (2020). Recent research has also shown a link between language style in
chatbots and user engagement for e-commerce Elsholz et al. (2019). Our focus
is to understand real long open-domain human-bot interactions better. Our
study can also influence how bots are detected on social media and provide a
better understanding of how human-human interactions differ from human-bot
interactions is essential for bot development. For instance, when should bots
accommodate towards the user? Can we tell how well a system performs for a
specific task based on whether the user accommodates the bot? If human-bot
conversations are linguistically different than human-human interactions,
research from human-human studies may not generalize to human-bot
interactions.

Toward addressing the potential societal impacts of open-domain bots, this
paper addresses the following research questions (RQs) in the context of bot
Detection:

  1. RQ1.

How do humans and bots align in human-bot interactions?

  2. RQ2.

How does the alignment in human-bot interactions compare to the alignment in
human-human interactions?

  3. RQ3.

Are differences in alignment between human-human and human-bot interactions
similar across domains and language generation learning methods?

Overall, we analyze whether human users’ language changes stylistically with
bots as compared to human-human conversations. Furthermore, we show that small
differences in language style provides robust information, compared to the
bots’ language patterns, to accurately detect bots.

##  2 Related Work

Bot Detection. Detecting bots in the wild is a widely studied problem Garcia-
Silva et al. (2019); Herzig et al. (2019); Kosmajac and Keselj (2019);
Ippolito et al. (2020); Jawahar et al. (2020). bot detection methods rely on
two forms of information Orabi et al. (2020): Content and Behavior. Garcia-
Silva et al. (2019) studied how pre-trained language models perform for the
task of bot detection. Kosmajac and Keselj (2019) developed “language-
independent” stylistic features that measure language diversity to detect
bots. Knauth (2019) empirically explored content and behavioral features for
bot detection. Beskow and Carley (2018) analyzed human-bot interactions by
developing methods to detect bots using network analysis of conversation
structures. Much of the prior work has focused on detecting bots “in the
wild.“ Thus, many of the interactions between bots and humans are superficial
(e.g., receiving movie quotes from a bot). This paper differs from prior work
in two ways. First, rather than using content from the bot or general
behavioral information (e.g., post frequency), we try to detect bots by
analyzing how humans respond to them (compared to human-human interactions).
Second, rather than exploring bots in the wild, we explore two types of
datasets: one where researchers instruct participants to interact with bots
and humans in the same way, and another dataset where researchers instruct
participants to converse with a bot.

Analyzing Human-Human Conversations. Linguistic alignment is known to interact
with a wide array of social factors. For instance, the level of alignment of
people’s linguistic style in a conversation has been claimed to be affected by
their relative social power Gnisci (2005); Xu et al. (2018); Danescu-
Niculescu-Mizil et al. (2011). Cooperative decision-making tasks are
positively related to the participants’ linguistic convergence Fusaroli et al.
(2012); Kacewicz et al. (2014). Recently, Sharma and De Choudhury (2018)
analyzed mental health support forums on Reddit, showing that linguistic
accommodation is positively linked with informational and emotional support.
Similarly, Taylor and Thomas (2008) analyzed negotiation outcomes and
linguistic alignment, showing that alignment positively predicted successful
negotiation outcomes. There is also the support of linguistic alignment by
doctors to enhance patient care by improving trust and adherence to a
treatment plan Kline and Ceropski (1984); Wood (2019). Alignment is a powerful
tool that can be used to improve understanding, trust, and potentially patient
outcomes. Therefore, understanding when and how people should align with each
other is an important area of research. Furthermore, it is essential to draw a
line between engaging with a culture and language style and appropriating it,
thereby potentially causing harm rather than building rapport with the
partners in a conversation.

With the goal of understanding human-human interactions, there has also been
progress in linguistic alignment measurement techniques Niederhoffer and
Pennebaker (2002); Danescu-Niculescu-Mizil et al. (2011); Jones et al. (2014);
Wang et al. (2014); Doyle and Frank (2016); Shin and Doyle (2018). Danescu-
Niculescu-Mizil et al. (2011) presented an easy-to-compute expression that
measures the increase in the conditional probability given that a
conversational partner has used it. An issue with the method proposed by
Danescu-Niculescu-Mizil et al. (2011) is that it assumes messages between two
people have similar length. To overcome this limitation, Doyle and Frank
(2016) introduced the Word-Based Hierarchical Alignment Model (WHAM), a
hierarchical graphical model where the parameters are learned using Bayesian
inference. Another method called the Simplified Word-Based Alignment Model
(SWAM) was recently proposed by Shin and Doyle (2018). SWAM attempts to
compare alignment between different groups when the alignment scores are
assumed to differ substantially. Unfortunately, SWAM only estimates group-
level alignment, not conversation-level between two specific users/bots.
Linguistic alignment has also been shown to be predictive of specific tasks.
In this paper, we use the method proposed by Danescu-Niculescu-Mizil et al.
(2011) to estimate linguistic alignment. For instance, Niven and Kao (2019)
use alignment features to predict discourse acts.

Analyzing Human-Bot Conversations. Research studying the interaction between
bots and humans has been explored from a wide array of perspectives. For
example, systems that use emotionally expressive interjections (“wow”, “ahem”)
in their text to speech responses can significantly improve the user
experience Cohn et al. (2019). Given the popularity of bots in application
areas from business Kaczorowska-Spychalska (2019) to healthcare Pieraccini et
al. (2009), it is also important to understand how language generation style
and alignment impacts their intended use. There has been a recent interest in
analyzing accommodation and similar concepts in human-bot interactions. For
instance, Ahn et al. (2020) show that humans will match code-switching
patterns introduced by a chat system. Moreover, users tend to have positive
reactions towards systems that code-switch. Ma and Lalor (2020) measure
lexical entrainment between a specific Reddit bot and users. They find that
sentiment of bot has a positive effect on the sentiment of the humans
response. Furthermore, Ma and Lalor (2020) show that human responses tend to
overlap with the bots original post. Compared to prior work studying
accommodation-related aspects of human-bot interactions, this paper differs in
two ways. First, just analyzing linguistic alignment, we ground our study of
human-bot interactions in the real-world task of bot detection. Second, we
analyze cross-domain generalization of accommodation patterns in human-bot
interactions, where cross-domain includes types of bots (e.g., retrieval and
transformer-based models), data collection procedures (e.g., sampling bias),
and conversation topics.

##  3 Datasets

Figure 1: This figure depicts an potential conversation between a known human
user and an unknown user. The unknown user may be a human or a bot.

An overview of the task we introduce in this paper is shown in Figure 1. We
formulate a bot detection task between two entities, either a human and bot or
a human and human. We assume that one entity is always human, and the other
entity is unknown—either a human or a bot. Formally, let
$D=[u^{h}_{1},u^{o}_{2},\ldots,u^{h}_{N-1},u^{o}_{N}]$ represents a sequence
of utterances, where $u^{h}_{i}$ represents the i-th utterance in a
conversation and that it was made by a human $h$. Likewise, $u^{o}_{i}$
represents an unknown entity’s utterance (human or bot). $N$ is the total
number of utterances in the conversation. Our goal is to develop a classifier
$f(D)$ that maps to a class in the set $T=\\{\text{human-human},\text{human-
bot}\\}$, where the human-bot means a bot is a part of the conversation. We
formulate two bot detection datasets for this task consisting of three known
bot datasets: ConvAI2, WOCHAT, and DailyDialog. The basic statistics of each
dataset are shown in Table 1. WOCHAT and DailyDialog are used to form a
Control dataset. We describe each dataset below:

Dataset Name | # $D$ | # $u$ |  Avg. $u$ |  Avg. W
---|---|---|---|---
ConvAI2 Data
INTERMEDIATE | 291 | 4317 | 14.83 | 6.66
TOLOKERS | 3127 | 39155 | 12.52 | 7.05
VOLUNTEER | 1111 | 14623 | 13.16 | 6.60
PERSONA-CHAT | 18878 | 278478 | 14.75 | 1.85
Control Data
IRIS | 163 | 5687 | 34.89 | 5.46
TICKTOCK | 206 | 5462 | 26.51 | 7.07
DailyDialog | 13118 | 102980 | 7.85 | 13.08

Table 1: Summary of each dataset, including the number of dialogues (# $D$),
average number of utterances per dialogue (Avg. $u$), average number of words
per utterance (Avg. W) and total number of utterances (U)

ConvAI2. We use four datasets from the Second Conversational Intelligence
Challenge (ConvAI2). The motivation behind the ConvAI2 challenge was to
develop new approaches towards open-domain chatbots Zhang et al. (2018); Dinan
et al. (2019). Overall, we use two types of ConvAI2 datasets: training
datasets containing human-human interactions and evaluation phase datasets
containing human-bot interactions. First, we use the PERSONA-CHAT dataset, a
collection of human-human interactions where researchers instructed Mechanical
Turk users to converse with one another, assuming specific personas (i.e.,
profile descriptions). The ConvAI2 competition used automatic (e.g.,
Perplexity) and human evaluation procedures. The human evaluation procedures
involve either paid workers or volunteers that interact with models built
using PERSONA-CHAT, after which the volunteers/workers are asked to rate their
interactions. Moreover, the human evaluator is instructed to interact with the
bots in the same way as the Mechanical Turk users who participated in creating
the PERSONA-CHAT dataset.

We use three “evaluation-phase” datasets 111http://convai.io/data/: TOLOKERS,
VOLUNTEERS, and INTERMEDIATE. The TOLOKERS used solicited workers to chat with
the models, similar to the PERSONA-CHAT’s Mechanical Turk setup. Specifically,
TOLOKERS consists of data collected during DeepHack.Chat
222http://deephack.me/chat hackathon via paid workers using the Yandex.Toloka
service. The INTERMEDIATE dataset consists of more dialogues by the bots from
DeepHack.Chat, but the interactions come from volunteers. Finally, the
VOLUNTEER dataset was collected during the final “wild evaluation” round of
the ConvAI2 competition. Human-bot interactions were collected from volunteers
through the Facebook Messenger and Telegram APIs.

Control Dataset. One of the research questions we explore in this paper is
related to out-of-domain performance. Specifically, can we detect bots based
on human response when data were collected under different settings and where
the conversation topics differ? To address the out-of-domain research
question, we use two additional datasets collected in the The Workshop on
Chatbots and Conversational Agent Technologies (WOCHAT) Kong-Vega et al.
(2019). As part of a shared task, the workshop makes several bots available,
has participants contribute new bots, and participants interact with the bots
providing utterance-level feedback regarding their performance. We use two
human-bot interaction datasets released by the organizers: IRIS and the
TickTock dataset. 333http://workshop.colips.org/wochat/data/index.html both
IRIS Banchs and Li (2012) and TickTock Yu et al. (2015) are retrieval-based
bot variants that were “trained” on different datasets. Unlike the ConvAI2
task, humans are not instructed to converse with the bots as if they are
another human. Thus, sometimes humans will ask things such as, “Are you a
Robot?”. This point provides a unique aspect that increases cross-data
differences.

Unfortunately, human-human conversations were not released from the WOCHAT
shared task. The focus was annotating human-bot interactions. Thus, we need to
augment WOCHAT with real human-human conversations. Our study uses the
DailyDialog dataset as our source of human-human interactions Li et al.
(2017). Unlike the ConvAI2-related data, DailyDialog consists of conversations
between English learners practicing everyday English dialog in daily life.

Data Processing. We split the data into three groups: Unpaid (U), Paid (P),
and Control (C). The U dataset consists of PERSONA-CHAT, INTERMEDIATE, and
VOLUNTEER ConvAI2 datasets. The P dataset consists of PERSONA-CHAT and
TOLOKERS—the name Paid comes from the fact that the TOLOKERS were paid as part
of a crowdsourcing task. We split this group from the others in case this
affects conversation behavior. The C dataset consists of IRIS, TICKTOCK, and
DailyDialog. Each dataset is divided into 70%, 10%, and 20% training,
validation, and test splits, respectively. It is important to note that the
same PERSONA-CHAT training, validation, and test examples are used in both the
U and P datasets.

##  4 Method

To detect human-bot conversations, we explore two types of features below:
Content and Stylistic features.

###  4.1 Content Features

We define content information as features describing “what” humans and bots
say in their interactions. Specifically, we describe two sets of content
features: bag-of-words and embeddings.

Bag-of-words. As a simple baseline, we use TF-IDF-weighted unigrams from a
dialog to detect whether a bot is part of the conversation. We explore three
settings for the TF-IDF features: human-only, unknown-only, and human-unknown.
The unknown user in the conversation can be either a bot or a human. Thus, in
the human-Only setting, only the human’s unigrams from each dialogue
$D_{h}=[u^{h}_{1},u^{h}_{3},\ldots,u^{h}_{N-1}]$ are used to detect human-bot
interactions. The unknown-only setting uses only the unknown user’s unigrams
from each dialogue $D_{o}=[u^{o}_{2},u^{o}_{4},\ldots,u^{o}_{N}]$, and the
human-unknown setting uses both the human’s and unknown user’s unigrams from
each dialogue $D=[u^{h}_{1},u^{o}_{2},\ldots,u^{h}_{N-1},u^{o}_{N}]$.

Embedding Features. We encoded each dialogue $D$ using BERT Devlin et al.
(2019)444We use the bert-base-uncased pre-trained model available in the
HuggingFace package Wolf et al. (2019). by feeding the first 510 WordPieces
and then averaging the word representations extracted from the second-to-last
layer. We explore three variants, human-Only, Unknown-Only, and human-Unknown.
Depending on the variant, the exact WordPieces passed to BERT change. For
instance, in the human-Only setting, the first 510 WordPieces made by the
human from a concatenation of all of the humans utterances
$D_{h}=[u^{h}_{1},u^{h}_{3},\ldots,u^{h}_{N-1}]$ are passed to BERT.
Similarly, all of the unknown user’s utterances are used for the unknown-only
variant to obtain $D_{o}=[u^{o}_{2},u^{o}_{4},\ldots,u^{o}_{N}]$. In the
human-unknown setting, all human and unknown utterances are concatenated in
$D=[u^{h}_{1},u^{o}_{2},\ldots,u^{h}_{N-1},u^{o}_{N}]$ before generating
features with BERT.

###  4.2 Stylistic Features

The stylistic features encode “how” humans and bots speak in their
interactions. We use two sets of stylistic features: Linguistic Inquiry and
Word Count (LIWC) Pennebaker et al. (2015) and linguistic accommodation
Danescu-Niculescu-Mizil et al. (2011).

LIWC. We experiment with the psychologically validated word categories (e.g.,
positive emotion, cognitive, and social processes) in LIWC as features. These
lexicons might reveal more about a writer’s thought processes, emotional
states, and intentions. For LIWC features, we use the same word categories
described in the Linguistic Accommodation Section below to train our LIWC-
based classifier. Specifically, we use LIWC to process each utterance. Next,
each utterance’s LIWC scores are averaged together to form a dialogue-specific
feature vector. We experiment with LIWC features variants: human-only,
unknown-only, and human-unknown. Each setting averages the LIWC scores across
a different set of utterances, similar to the bag-of-words and embedding
features.

Linguistic Accommodation. We use the method proposed by Danescu-Niculescu-
Mizil et al. (2011) to analyze linguistic alignment. It includes two primary
group-level and conversation-level measures: baseline word usage and
alignment. The group-level baseline word use corresponds to the rate at which
a person uses a given word category when it has not been used previously in a
conversation. The group-level alignment score reflects the proportionate
increase—compared to the baseline score—in the likelihood of the word being
used when it has been used previously in the conversation. Similarly, the
conversation-level metrics measure accommodation and usage between a human and
Unknown (bot or human) user in the each dialogue $D_{i}$. We measure the
probability of seeing a word category (e.g., LIWC categories) $w_{c}$ given
$w_{c}$ appeared in the previous utterance $P(w_{c}\in u^{o}_{i}|w_{c}\in
u^{h}_{i-1})$555The probabilities are from the perspective of the Unknown user
in the conversation. They are also calculated from the human’s perspective.
and the baseline probability of seeing $w_{c}$ in the conversation $P(w_{c}\in
u^{o}_{i})$. Next, the accommodation score is calculated by taking the
difference between both probabilities, $acc(c)=P(w_{c}\in u^{o}_{i}|w_{c}\in
u^{h}_{i-1})-P(w_{c}\in u^{o}_{i})$. These empirical probabilities are at the
conversation level (i.e., measuring how two specific users align). To obtain
the group-level estimates, we simply average the accommodation scores $acc(c)$
over all conversations $D_{i}$ for each word category. We use the following 17
LIWC categories Pennebaker et al. (2015): i, you, we, they, social, cogproc,
posemo, negemo, article, prep, certain, conj, discrep, negate, pronoun, quant,
and tentat. We experiment with two main settings as described for the previous
methods: Human-Only, Unknown-Only. The Human-Only setting consists of just
using the known human’s alignment scores in each conversation and vice-versa
for the Unknown-Only setting.

Model Training Details. For the content feature sets, we train a Logistic
Regression classifier from the Scikit-Learn package Pedregosa et al. (2011).
Using the validation split for each dataset, we grid-search over the C-values
{.0001, .001, .01, .1, 1., 10.}, the logistic regression class weight
parameters {None, balanced}, and normalization procedures {standardize, unit
normalize, None}. For the Stylistic features, we train a Random Forest
classifier from the Scikit-Learn package. Again, using the validation split,
we grid-search over the class weight parameters {None, balanced,
balanced_subsample}, criterion measures {gini, entropy}, max features {sqrt,
log2, None}, and bootstrap parameters {True, False}. For all experiments using
the Random Forest classifier, we set n_estimators to 1000.

| U $\to$ U | P $\to$ P | C $\to$ C
---|---|---|---
Baselines
Most Frequent | .482 | .462 | .493
Most Infrequent | .065 | .125 | .027
Stratified (random) | .491 | .498 | .487
Content Features
Human Bag-of-Words | .980 | .987 | .939
Human BERT | .989 | .996 | .987
Unknown Bag-of-Words | .971 | .970 | .997
Unknown BERT | .996 | .996 | .983
Human + Unknown Bag-of-Words | .958 | .963 | .986
Human + Unknown BERT | .990 | .992 | .990
Stylistic Features
Human LIWC | .878 | .899 | .705
Human Accommodation | .989 | .987 | .627
Unknown LIWC | .838 | .862 | .707
Unknown Accommodation | .887 | .897 | .674
Human & Unknown LIWC | .885 | .903 | .601
Human & Unknown Accommodation | .988 | .990 | .729

Table 2: Source $\to$ Source Macro F1 Results for Bot Detection

|  | U $\to$ C | P $\to$ C | C $\to$ U | C $\to$ P | AVG
---|---|---|---|---|---|---
Baselines | 1\. Most Frequent | .493 | .493 | .482 | .462 | .483
2\. Most Infrequent | .027 | .027 | .065 | .125 | .061
3\. Stratified (Random) | .502 | .479 | .499 | .476 | .489
Content: What the bots and humans write? | 4\. Human bag-of-words | .518 | .504 | .536 | .608 | .541
5\. Human BERT | .493 | .493 | .482 | .462 | .483
6\. Unknown bag-of-words | .493 | .493 | .535 | .475 | .499
7\. Unknown BERT | .493 | .493 | .482 | .462 | .483
8\. Unknown and Human bag-of-words | .509 | .510 | .639 | .608 | .567
9\. Unknown and Human BERT | .493 | .493 | .522 | .478 | .497
Stylistic: How the bots and humans write? | 10\. Human LIWC | .480 | .491 | .524 | .512 | .502
11\. Human Accommodation | .631 | .591 | .604 | .510 | .584
12\. Human LIWC + Accommodation | .605 | .608 | .703 | .724 | .660
13\. Unknown LIWC | .474 | .478 | .483 | .476 | .478
14\. Unknown Accommodation | .428 | .424 | .503 | .494 | .462
15\. Unknown LIWC + Accommodation | .462 | .436 | .502 | .497 | .474
13\. Human & Unknown LIWC | .521 | .504 | .532 | .556 | .528
14\. Human & Unknown Accommodation | .611 | .620 | .642 | .677 | .637
15\. Human & Unknown LIWC + Accommodation | .622 | .633 | .642 | .677 | .643

Table 3: This table reports the cross-dataset Macro F1 score for detecting
human-bot conversations for three datasets: Unpaid (U), Paid (P), and Control
(C). The largest Macro F1 score in each column is in bold.

##  5 Results

In this section, we report two sets of results. First, in Subsection 5.1 we
present the performance of the bot detection models we explain in Section 4.
Beyond the methods described in Section 4, we also compare three baselines:
Most Frequent, Most Infrequent, and Stratified. The Most Frequent baseline
predicts the most frequent class for every example (i.e., human-human). The
Most Infrequent baseline predicts the most infrequent class (i.e., human-bot)
and the Stratified baseline makes random prediction proportional to each
class’s frequency. Second, in Subsection 5.2, we present a fine-grained
analysis of the linguistic accommodation results on the ConvAI2 datasets. For
all results, we report the Macro F1 (average F1 for the human-human and human-
bot conversation classes).

###  5.1 Bot Detection Experiments

The source dataset results are reported in Table 2. Overall, we find that
content features are the most predictive for bot detection, when training and
testing on the same train-test splits from the same dataset. Furthermore, we
find that the BERT-based models are able to outperform the Bag-of-Words models
on average. For instance, the Human Bag-of-Words model on dataset C obtains an
F1 of .939. Yet, the Human BERT model obtains an F1 of .987. Likewise, for the
stylistic features, we find that simply using LIWC works better than using the
accommodation features alone for dataset C. However, combining both
Accommodation and LIWC features from both the Human and the Bot is better than
using either feature set individually with an F1 of .729.

The cross-dataset bot detection results are presented in Table 3.
Specifically, the scores are from experiments where we train on a source
dataset and evaluate each model on a target dataset’s test split (i.e., source
$\to$ target). Overall, we make three major findings. First, for both Content
and Stylistic features, we find that analyzing the known human’s language in
each conversation is more informative than analyzing the bot’s content. For
instance, the average (AVG) score for human bag-of-words is .541, while the
bot bag-of-words model AVG result is nearly 4% lower (.499). We have similar
findings between human LIWC (.502) and bot LIWC (.478). We find that combining
both bot and human LIWC improves the AVG performance of the human-only LIWC
model with an F1 of 0.528. Second, while the BERT-based model performs better
when applied to data from the same datasets in Table 2 (e.g., U $\to$ U) in
many settings, when the test dataset changes substantially (e.g., U $\to$ C),
the generalization performance of BERT drops compared to using bag-of-words.
This result is potentially caused by overfitting to random source-specific
characteristics. Third, we find that Accommodation features outperform all
other individual feature sets with an AVG bot-detection Macro F1 of .584, with
the exception of using both the human’s and bot’s accommodation features which
has a Macro F1 of .637. Interestingly, human accommodation information (.584)
is more predictive than the bot’s text (.462). Intuitively, the type of
responses generated by a bot can differ substantially depending on the bot’s
training data (e.g., ConvAI2 vs. DailyDialog) and model (e.g., transformer vs.
retrieval-based models). Hence, the human’s responses are more consistent with
regard to alignment. The best combination overall is the combination of the
human’s LIWC features with the human’s accommodation features achieving a
Macro F1 of .660. Yet, with the bot (Unknown) feature combination, the
performance drops from .768 with Unknown LIWC to .674 after combining
accommodation features.

Feature Group | Feature | FI
---|---|---
Accomodation | pronoun | .598
LIWC | social | .063
LIWC | article | .043
LIWC | prep | .028
LIWC | pronoun | .020
Accomodation | quant | .018
LIWC | tentat | .017
Accomodation | i | .017
LIWC | you | .016
LIWC | i | .015

Table 4: The top ten most informative features for the Random Forest model
trained on dataset P’s human responses to detect human-bot conversations with
LIWC and Accommodation features.

(a)  Human TickTock Alignment

(b)  Bot TockTock Alignment

(c)  Human INTERMEDIATE Alignment

(d)  Bot INTERMEDIATE Alignment

Figure 2:  Alignment scores for the TickTock and INTERMEDIATE datasets.

In Table 4, we analyze the Random Forest’s importance scores for the best
model (trained on dataset P’s human responses) features using the model that
combines human LIWC and accommodation features. The most informative feature
is pronoun accommodation. See Section 6 for more details. Other informative
features include the human’s use of social, quant (quantitative), and
tentative words.

###  5.2 Accommodation Analysis

In this section, we analyze the group-level (averaged) accommodation scores
for the INTERMEDIATE and TickTock datasets. Again, note that the group-level
scores are dataset averages for each group (human and bot), the conversation-
level estimates can vary from the group scores. The accommodation analysis is
displayed in Figure 2. The bot results (Figures 2(d) and 2(b)) indicate how
the bot responds to a human, and the human scores (Figures 2(c) and 2(a))
represent how a human responds to a bot. We report all LIWC categories,
including Pronoun usage (e.g., pronoun, I, you, we, and they) and Rhetorical
words (e.g., social, cogproc, posemo, and negemo) results. Find the analysis
for all of the datasets in the Supplementary Material.

Overall, some indicators are robust across datasets, whereas others can vary.
For instance, for the general pronoun category, we find that human generally
accommodate less towards bots than bots accommodate towards them. The pronoun
accommodation pattern for pronouns is robust across both datasets. Similarly,
for both datasets, humans accomodate more often than bots for social words.
Yet, for negative emotion words (negemo), we find that Bot’s accommodate less
than humans in the TickTock dataset, yet they accommodate more in the
INTERMEDIATE dataset. In contrast, in the ConvAI2 and DailyDialog datasets,
human accommodation is the same on average when two humans participate in a
conversation together (not shown in the Figures).

##  6 Discussion

Why is humans’ accommodation such a powerful predictor that they are in a
conversation with a bot—even when they are supposedly treating the
conversation the same as a human-human interaction? Overall, we found two
major reasons to explain the model’s high performance. First, modern bots from
the transformer to retrieval-based models generate bad responses. For
instance, in the conversation below from the VOLUNTEER dataset, the human and
bot have already had a conversation about hair color:

Bot: What is your hair color?Human: I said its black....Bot: I know the
feeling.Human: What?From the example, we can see that the bot is asking about
hair color, which was already discussed, then the bot responds with a
nonsensical output, “I know the feeling.” Thus, we see that the final human
response does not accommodate with with“I” nor any first singular pronoun. The
lower alignment score for humans (compared to bots) for the word category I in
Figure 2 supports this finding. Furthermore, from Table 4, we know that
accommodation in terms of pronoun usage is highly predictive of a human-bot
conversation. Similar posts can be found in all datasets where bad outputs can
explain why humans align differently concerning “I” in bot-human interactions
than human-human interactions. This finding is interesting because it is
similar to the findings in human-human interactions where alignment is
positively correlated with understanding in cooperative decision-making tasks
Fusaroli et al. (2012); Kacewicz et al. (2014). If humans don’t understand the
conversation, they align less, at least with regards to pronoun usage in our
case. The next example from the VOLUNTEER dataset is a better, more coherent
conversation:

Human: What is your Hobby?Bot: I love music! I love Taylor SwiftHuman: Me too.
And what about Iggy Pop?We can see that the final human response aligns with
the first person singular word “me” to the bot’s use of “I”. A potential use
that warrants future work in the area of bots evaluation involves using
accommodation information as an additional evaluation metric in human
evaluation studies if certain types of accommodation correlate with bad system
responses. If human ratings of bot performance are noisy, the use of
accomodation information can provide insight into how well a bot performs, if
a human interacts with it differently than they do in human-human
conversations.

The second reason for the significant predictive performance can be seen in
adversarial conversations between the human and bot. Specifically, in
instances where the human is trying to trick the bot or simply trying to ask
questions, bot-human alignment can differ from human-human conversations. The
example below from the TickTock dataset

Human: Are you a robot?Bot: No.Human: Where are you from?shows repeated “You”
usage by the user, repeatedly asking multiple questions. Again, differences in
pronoun usage can reflect adversarial users. Again, in terms of bot
evaluation, if a human is acting adversarially, and it is possible to
differentiate adversarial interactions from the human not understanding, this
can provide insight into . More examples can be found in the Supplementary
Material.

##  7 Conclusion

This study has is an exploration of the levels of linguistic alignment in
human-bot conversations and presents evidence about how bot-human interactions
are different than human-human conversations. This work’s insights have
implications for future bot detection methodologies. How humans and bots come
together and converse manifested by their level of agreement and disagreement
is a first step towards understanding human interactions with general
conversational agents. Furthermore, as future work, we will explore the use of
linguistic alignment to improve bot evaluation frameworks involving human-bot
interactions. Moreover, we will also explore methods of controlling the
accommodation level of bots themselves. Given the implicit social implications
of certain types of accommodation, controlling it can help important tasks
(e.g., counseling or customer service-related bots).

## References

  * Ahn et al. (2020) Emily Ahn, Cecilia Jimenez, Yulia Tsvetkov, and Alan Black. 2020.  What code-switching strategies are effective in dialogue systems?  _Proceedings of the Society for Computation in Linguistics_ , 3(1):308–318.
  * Banchs and Li (2012) Rafael E Banchs and Haizhou Li. 2012.  Iris: a chat-oriented dialogue system based on the vector space model.  In _Proceedings of the ACL 2012 System Demonstrations_ , pages 37–42.
  * Beskow and Carley (2018) David M Beskow and Kathleen M Carley. 2018.  Bot conversations are different: leveraging network metrics for bot detection in twitter.  In _2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)_ , pages 825–832. IEEE.
  * Chaves (2020) Ana Paula Chaves. 2020.  Should my chatbot be register-specific? designing appropriate utterances for tourism.  In _Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems_ , pages 1–11.
  * Chaves et al. (2019) Ana Paula Chaves, Eck Doerry, Jesse Egbert, and Marco Gerosa. 2019.  It’s how you say it: Identifying appropriate register for chatbot language design.  In _Proceedings of the 7th International Conference on Human-Agent Interaction_ , pages 102–109.
  * Cohn et al. (2019) Michelle Cohn, Chun-Yen Chen, and Zhou Yu. 2019.  A large-scale user study of an alexa prize chatbot: Effect of tts dynamism on perceived quality of social dialog.  In _Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue_ , pages 293–306.
  * Danescu-Niculescu-Mizil et al. (2011) Cristian Danescu-Niculescu-Mizil, Michael Gamon, and Susan Dumais. 2011.  Mark my words! linguistic style accommodation in social media.  In _Proceedings of the 20th international conference on World wide web_ , pages 745–754.
  * Daniel et al. (2019) Florian Daniel, Cinzia Cappiello, and Boualem Benatallah. 2019.  Bots acting like humans: Understanding and preventing harm.  _IEEE Internet Computing_ , 23(2):40–49.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Dinan et al. (2019) Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2019.  The second conversational intelligence challenge (convai2).  _arXiv preprint arXiv:1902.00098_.
  * Doyle and Frank (2016) Gabriel Doyle and Michael C Frank. 2016.  Investigating the sources of linguistic alignment in conversation.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 526–536.
  * Elsholz et al. (2019) Ela Elsholz, Jon Chamberlain, and Udo Kruschwitz. 2019.  Exploring language style in chatbots to increase perceived product value and user engagement.  In _Proceedings of the 2019 Conference on Human Information Interaction and Retrieval_ , pages 301–305.
  * Fusaroli et al. (2012) Riccardo Fusaroli, Bahador Bahrami, Karsten Olsen, Andreas Roepstorff, Geraint Rees, Chris Frith, and Kristian Tylén. 2012.  Coming to terms: Quantifying the benefits of linguistic coordination.  _Psychological science_ , 23(8):931–939.
  * Garcia-Silva et al. (2019) Andres Garcia-Silva, Cristian Berrio, and José Manuel Gómez-Pérez. 2019\.  An empirical study on pre-trained embeddings and language models for bot detection.  In _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_ , pages 148–155.
  * Giles et al. (1973) Howard Giles, Donald M Taylor, and Richard Bourhis. 1973.  Towards a theory of interpersonal accommodation through language: Some canadian data.  _Language in society_ , pages 177–192.
  * Gnisci (2005) Augusto Gnisci. 2005.  Sequential strategies of accommodation: A new method in courtroom.  _British Journal of Social Psychology_ , 44(4):621–643.
  * Herzig et al. (2019) Jonathan Herzig, Tommy Sandbank, Michal Shmueli-Scheuer, and David Konopnicki. 2019\.  Bot2vec: Learning representations of chatbots.  In _Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (* SEM 2019)_ , pages 75–84.
  * Ippolito et al. (2020) Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020.  Automatic detection of generated text is easiest when humans are fooled.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1808–1822.
  * Jawahar et al. (2020) Ganesh Jawahar, Muhammad Abdul-Mageed, and VS Laks Lakshmanan. 2020.  Automatic detection of machine generated text: A critical survey.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 2296–2309.
  * Jones et al. (2014) Simon Jones, Rachel Cotterill, Nigel Dewdney, Kate Muir, and Adam Joinson. 2014\.  Finding zelig in text: A measure for normalising linguistic accommodation.  In _Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers_ , pages 455–465.
  * Kacewicz et al. (2014) Ewa Kacewicz, James W Pennebaker, Matthew Davis, Moongee Jeon, and Arthur C Graesser. 2014.  Pronoun use reflects standings in social hierarchies.  _Journal of Language and Social Psychology_ , 33(2):125–143.
  * Kaczorowska-Spychalska (2019) Dominika Kaczorowska-Spychalska. 2019.  How chatbots influence marketing.  _Management_ , 23(1):251–270.
  * Kerlyl et al. (2006) Alice Kerlyl, Phil Hall, and Susan Bull. 2006.  Bringing chatbots into education: Towards natural language negotiation of open learner models.  In _International Conference on Innovative Techniques and Applications of Artificial Intelligence_ , pages 179–192. Springer.
  * Kline and Ceropski (1984) Susan L Kline and Janet M Ceropski. 1984.  Person-centered communication in medical practice.  _Human decision-making_ , pages 120–141.
  * Knauth (2019) Jürgen Knauth. 2019.  Language-agnostic twitter-bot detection.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)_ , pages 550–558.
  * Kong-Vega et al. (2019) Naomi Kong-Vega, Mingxin Shen, Mo Wang, and Luis Fernando D’Haro. 2019.  Subjective annotation and evaluation of three different chatbots wochat: Shared task report.  In _9th International Workshop on Spoken Dialogue System Technology_ , pages 371–378. Springer.
  * Kosmajac and Keselj (2019) Dijana Kosmajac and Vlado Keselj. 2019.  Twitter bot detection using diversity measures.  In _Proceedings of the 3rd International Conference on Natural Language and Speech Processing_ , pages 1–8.
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017.  Dailydialog: A manually labelled multi-turn dialogue dataset.  In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 986–995.
  * Liednikova et al. (2020) Anna Liednikova, Philippe Jolivet, Alexandre Durand-Salmon, and Claire Gardent. 2020\.  Learning health-bots from training data that was automatically created using paraphrase detection and expert knowledge.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 638–648.
  * Ma and Lalor (2020) Ming-Cheng Ma and John P Lalor. 2020.  An empirical analysis of human-bot interaction on reddit.  In _Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)_ , pages 101–106.
  * Martin et al. (2020) Alistair Martin, Jama Nateqi, Stefanie Gruarin, Nicolas Munsch, Isselmou Abdarahmane, Marc Zobel, and Bernhard Knapp. 2020.  An artificial intelligence-based first-line defence against covid-19: digitally screening citizens for risks via a chatbot.  _Scientific reports_ , 10(1):1–7.
  * Niederhoffer and Pennebaker (2002) Kate G Niederhoffer and James W Pennebaker. 2002.  Linguistic style matching in social interaction.  _Journal of Language and Social Psychology_ , 21(4):337–360.
  * Niven and Kao (2019) Timothy Niven and Hung-Yu Kao. 2019.  Detecting argumentative discourse acts with linguistic alignment.  In _Proceedings of the 6th Workshop on Argument Mining_ , pages 104–112.
  * Oniani and Wang (2020) David Oniani and Yanshan Wang. 2020.  A qualitative evaluation of language models on automatic question-answering for covid-19.  In _Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics_ , pages 1–9.
  * Orabi et al. (2020) Mariam Orabi, Djedjiga Mouheb, Zaher Al Aghbari, and Ibrahim Kamel. 2020.  Detection of bots in social media: a systematic review.  _Information Processing & Management_, 57(4):102250.
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011.  Scikit-learn: Machine learning in python.  _the Journal of machine Learning research_ , 12:2825–2830.
  * Pennebaker et al. (2015) James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015.  The development and psychometric properties of liwc2015.  Technical report.
  * Pieraccini et al. (2009) Roberto Pieraccini, David Suendermann, Krishna Dayanidhi, and Jackson Liscombe. 2009\.  Are we there yet? research in commercial spoken dialog systems.  In _International Conference on Text, Speech and Dialogue_ , pages 3–13. Springer.
  * Sharma and De Choudhury (2018) Eva Sharma and Munmun De Choudhury. 2018.  Mental health support and its relationship to linguistic accommodation in online communities.  In _Proceedings of the 2018 CHI conference on human factors in computing systems_ , pages 1–13.
  * Shin and Doyle (2018) Hagyeong Shin and Gabriel Doyle. 2018.  Alignment, acceptance, and rejection of group identities in online political discourse.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop_ , pages 1–8.
  * Tausczik and Pennebaker (2010) Yla R Tausczik and James W Pennebaker. 2010.  The psychological meaning of words: Liwc and computerized text analysis methods.  _Journal of language and social psychology_ , 29(1):24–54.
  * Taylor and Thomas (2008) Paul J Taylor and Sally Thomas. 2008.  Linguistic style matching and negotiation outcome.  _Negotiation and Conflict Management Research_ , 1(3):263–281.
  * Thomas et al. (2020) Paul Thomas, Daniel McDuff, Mary Czerwinski, and Nick Craswell. 2020.  Expressions of style in information seeking conversation with an agent.  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1171–1180.
  * Wang et al. (2014) Yafei Wang, David Reitter, and John Yen. 2014.  Linguistic adaptation in conversation threads: Analyzing alignment in online health communities.  _ACL 2014_ , page 55.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _arXiv preprint arXiv:1910.03771_.
  * Wood (2019) Nathan I Wood. 2019.  Departing from doctor-speak: A perspective on code-switching in the medical setting.  _Journal of general internal medicine_ , 34(3):464–466.
  * Xu et al. (2018) Yang Xu, Jeremy Cole, and David Reitter. 2018.  Not that much power: Linguistic alignment is influenced more by low-level linguistic features rather than social power.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 601–610.
  * Yadav et al. (2019) Deepika Yadav, Prerna Malik, Kirti Dabas, and Pushpendra Singh. 2019.  Feedpal: Understanding opportunities for chatbots in breastfeeding education of women in india.  _Proceedings of the ACM on Human-Computer Interaction_ , 3(CSCW):1–30.
  * Yu et al. (2015) Zhou Yu, Alexandros Papangelis, and Alexander Rudnicky. 2015.  Ticktock: A non-goal-oriented multimodal dialog system with engagement awareness.  In _2015 AAAI Spring symposium series_.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.  Personalizing dialogue agents: I have a dog, do you have pets too?  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2204–2213.

##  Appendix A Expanded Alignment Analysis

The complete group-level alignment scores for the IRIS and Tolokers datasets
can be found in Figures 3(a), 3(b), 3(c), and 3(d).

##  Appendix B Example Conversations

Here we present examples of conversations between humans and bots that are
representative of good conversational structure in terms of alignment between
the two parties such as the usage of pronoun “I” in the first example.

Bot: I love iphone! i just bought new iphone!Human: Thats good for you i m not
very into new tech

Human: I am fine. I like hip hop. What about you?Bot: I like r b and pop are
you 100 years old?Human: What? No. You are rude

Human: Do you have dogs?Bot: Yes i have two dogs and a cat.Human: Cool I love
dogs. But i have siblings instead...Bot: I have two brothers and a sister and
a sister.

Here conversations between humans and bots involve irregular pronoun usage
that are representative of a break in the conversational pattern and little or
no alignment between the two parties such as the usage of pronouns “You” and
resultant use of “I” reply in the first example.

Human: See you then!Bot: I m not going to see you.

Bot: I am 24 and I have a dog named her name is named nameHuman: Really her
name is name?Bot: She is a good name.

Human: who is obamaBot: i partially have interest in thatHuman: can you tell
me more about obamaBot: i hate pigs i live a farm

These conversations signify a complete breakdown of conversational structure
with respect to alignment, as there is none. The chats seem haphazard and
vague with either the bot or the human replying in an ambiguous manner.

User: Good afternoonSystem: Well you know I’m not sure. What do you think
about moviesUser: That didn’t make any sense

User: hi againSystem: Of course. Any thoughts?User: no

User: HelloSystem: Do not argue with an idiot. He will drag you down to his
level and beat you with experience.User: Um Okay that was random

(a)  Human IRIS Alignment

(b)  Bot IRIS Alignment

(c)  Human Tolokers Alignment

(d)  Bot Tolokers Alignment

Figure 3: Alignment scores for the IRIS and Tolokers dataset


# Fermi-LAT GeV excess and muon $g-2$ in a modular $A_{4}$ symmetry

Jongkuk Kim<EMAIL_ADDRESS>School of Physics, KIAS, Seoul 02455, Korea
Hiroshi Okada<EMAIL_ADDRESS>Asia Pacific Center for Theoretical
Physics (APCTP) - Headquarters San 31, Hyoja-dong, Nam-gu, Pohang 790-784,
Korea  Department of Physics, Pohang University of Science and Technology,
Pohang 37673, Republic of Korea

###### Abstract

The recent measurement of muon anomalous magnetic dipole moment (muon $g-2$)
suggests that there might exist new physics that dominantly interacts with
muons. The observed gamma-ray excess from Fermi-LAT indicates that dark matter
annihilates into a specific charged fermions. We propose a successful model
simultaneously to explain the Fermi-LAT GeV gamma-ray excess and sizable muon
$g-2$ with a modular $A_{4}$ symmetry. Due to nature of this symmetry, our DM
only interacts with pairs of muon and we explain sizable muon $g-2$ without
suffering from constraints of any lepton flavor violations. We numerically
show our allowed spaces on each measurements of Fermi-LAT, relic density of DM
and muon $g-2$, randomly scanning our input parameters.

††preprint: KIAS-P230xx, APCTP Pre2023-00x

##  I Introduction

Modular non-Abelian discrete flavor symmetries are widely applied to various
flavor (new) physics such as quark and lepton Yukawa sectors to predict
experimental values such as their masses, mixings, and phases as well as
reproduce them, baryon asymmetry of Universe via leptogenesis, muon/electron
anomalous magnetic dipole moment (muon/electron $g-2$), electron dipole moment
(EDM), flavor changing processes like $\mu\to e\gamma$, $b\to
s\ell\bar{\ell}$, and dark matter (DM). All of them (except quark and charged-
lepton sectors) are expected to be existence of ”beyond the standard
model(BSM)”. Historically, this group was initially proposed by F. Feruglio in
the paper Feruglio (2019) and he investigated the lepton sector via a modular
$A_{4}$ group. Subsequently, a large number of possibilities are arisen. For
example, the following refs. are applied to lepton and quark sectors through
the modular $A_{4}$ symmetry Criado and Feruglio (2018); Kobayashi et al.
(2018a); Okada and Tanimoto (2019a, b); de Anda et al. (2020); Novichkov et
al. (2019); Nomura and Okada (2019a); Okada and Orikasa (2019); Ding et al.
(2019a); Nomura et al. (2020); Kobayashi et al. (2019a); Zhang (2020); Ding et
al. (2019b); Kobayashi et al. (2020); Nomura et al. (2019); Wang (2020); Okada
and Shoji (2020); Okada and Tanimoto (2020a); Behera et al. (2020a); Nomura
and Okada (2020a, b); Asaka et al. (2020a); Okada and Tanimoto (2020b); Nagao
and Okada (2020); Okada and Tanimoto (2020c); Kobayashi et al. (2018b); Yao et
al. (2021); Chen et al. (2021); de Medeiros Varzielas and Lourenço (2022);
Nomura et al. (2021a); Ding et al. (2021); Nagao and Okada (2022); Okada and
Qi (2021); Nomura et al. (2021b); Liu and Ding (2022); Nomura and Okada
(2022); Otsuka and Okada (2022); Kang et al. (2022); Ishiguro et al. (2022);
Nomura et al. (2022); Du and Wang (2023); Kikuchi et al. (2022); Gogoi et al.
(2022); Abbas and Khalil (2022); Petcov and Tanimoto (2022); Kikuchi et al.
(2023). 111We restrict ourselves the refs. of $A_{4}$, even though various
groups are adapted for flavor models. In addition to the lepton sector,
leptogenesis is discussed in refs. Kobayashi et al. (2019b); Asaka et al.
(2020b); Kashav and Verma (2021); Okada et al. (2021); Dasgupta et al. (2021);
Behera et al. (2020b); Ding et al. (2022). Semi-leptonic flavor changing
neutral processes such as $b\to s\ell\bar{\ell}$, lepton flavor violations
(LFVs), muon/electron $g-2$, and EDM are mainly discussed in refs. Hutauruk et
al. (2020); Kobayashi et al. (2022a, b). Inflation model is also discussed in
ref. Gunji et al. (2022). Interestingly, a successful DM candidate; i.e.,
stable DM, is realized by applying a modular symmetry. Up to now, two methods
are found. The first way is to apply a nonzero modular weight to DM that has a
remnant symmetry such as $Z_{2}$ even after the modular symmetry breaking and
the remnant one assures the stability of DM Nomura and Okada (2019b). Another
way is to make use of a recovering $Z_{N}$ symmetry that occurs at three fixed
points of modulus $\tau$. Here, we have $Z_{2}$ on $\tau=i$ and $Z_{3}$ on
$\tau=\omega,\ i\infty$. Ref. Kobayashi et al. (2022c) realizes the stable DM
on $\tau=i$ including the lepton sector where recovering symmetry is $Z_{2}$.
222The DM stability has been achieved by a way of a representation of
traditional non-Abelian discrete flavor symmetries e.g. Lavoura et al. (2013);
Hirsch et al. (2010); Boucenna et al. (2011). Decaying DM with flavor specific
case has also been studied by these traditional symmetries e.g.Kajiyama and
Okada (2011); Haba et al. (2011); Kajiyama et al. (2014).

Recently, the Fermilab Muon $g-2$ Collaboration reported the anomalous muon
magnetic moment Abi et al. (2021). The observation is consistent with the
measurement by Brookhaven E821 Collaboration Bennett et al. (2006). The
$4.2\sigma$ deviation from the SM expectation may require new physics
contributions to the muon $g-2$. There have been comprehensive study on the
context of the muon $g-2$ and other phenomena He et al. (1991); Baek et al.
(2001); Chun et al. (2019); Asai et al. (2021); Drees and Zhao (2022); Holst
et al. (2022); Baek et al. (2022); Kim (2022).

From a decade ago, a gamma-ray excess at $E_{\gamma}\sim\mathcal{O}(1)$GeV
coming from the Galactic center was found. In a lot of analyses from
independent groups, it is claimed that the gamma-ray excess spectrum is in
good agreement with the emission expected from DM annihilation into a specific
charged particles. The $b\bar{b}$ final state was understood by Higgs portal
DM models and was studied Okada and Seto (2014); Ko et al. (2014); Mondal and
Basak (2015). Also, the leptonic final state was understood by
$U(1)_{L_{\mu}-L_{\tau}}$ model and investigated Park et al. (2016).

In this paper, we explain the Fermi-LAT GeV excess that DM must annihilate
into a pair of muons that produce gamma-ray to be observed. The unique
interaction of DM can be achieved by a modular $A_{4}$ symmetry, and we apply
the first way to stabilize our DM assigning nonzero modular weight. As a bonus
of our model, we also explain the sizable muon $g-2$ without worry about the
LFVs such as $\mu\to e\gamma$ due to non-interactions except muon pair.

This paper is organized as follows. In Sec. II, we show our model setup
starting from the relevant superpotential and soft breaking terms, and
formulate the heavy charged lepton mass matrix. In Sec. III, we firstly
discuss our DM candidate on how the DM interact with the muon pair, and
evaluate cross section requested by Fermi-LAT experiment as well as relic
density of DM. Then, we demonstrate the formula of muon $g-2$. In Sec. IV, we
present our allowed region to satisfy the Fermi-LAT measurement, relic density
of DM, and muon $g-2$, randomly scanning our input parameters. In Sec. V, we
devote the conclusions and discussions.

| SM leptonic superfields | New superfields
---|---|---
|  $\widehat{L}_{e}$ |  $\widehat{L}_{\mu}$ |  $\widehat{L}_{\tau}$ |  $\widehat{\overline{e}}$ |  $\hat{\overline{\mu}}$ |  $\widehat{\overline{\tau}}$ |  $\widehat{L}^{\prime}$ |  $\widehat{\overline{L^{\prime}}}$ |  $\widehat{E}$ |  $\widehat{\overline{E}}$ |  $\widehat{\chi}$
$SU(2)_{L}$ | $\bm{2}$ | $\bm{2}$ | $\bm{2}$ | $\bm{1}$ | $\bm{1}$ | $\bm{1}$ | $\bm{2}$ | $\bm{2}$ | $\bm{1}$ | $\bm{1}$ | $\bm{1}$
$U(1)_{Y}$ | $-\frac{1}{2}$ | $-\frac{1}{2}$ | $-\frac{1}{2}$ | $1$ | $1$ | $1$ | $-\frac{1}{2}$ | $\frac{1}{2}$ | ${-1}$ | ${1}$ | ${0}$
$A_{4}$ | $1$ | $1$ | $1^{\prime\prime}$ | $1$ | $1^{\prime\prime}$ | $1^{\prime}$ | $1$ | $1$ | $1^{\prime}$ | $1^{\prime\prime}$ | $1$
$-k$ | $0$ | $-2$ | $0$ | $0$ | $-2$ | $0$ | $-3$ | $-1$ | $-3$ | $-1$ | $-3$
Table 1: Field contents of the matter superfields and their charge assignments
under $SU(2)_{L}\otimes U(1)_{Y}\otimes A_{4}$, where $-k$ is the number of
modular weight.

##  II Model setup

In this Section, we explain our model construction, introducing the SM
leptonic superfields and new ones and assigning the charges under the
symmetries of $SU(2)_{L}\otimes U(1)_{Y}\otimes A_{4}$, where $-k$ is the
number of modular weight and ”hat” over the fields represents superfields.
Here, we add one vector-like matter superfields
($\widehat{E},\widehat{\overline{E}},\widehat{L}^{\prime},\widehat{\overline{L^{\prime}}}$)
where we apply only fermionic parts of
$\widehat{E},\widehat{\overline{E}},\widehat{L^{\prime}},\widehat{\overline{L^{\prime}}}$
for our model. And $E,\overline{E}$ are singly-charged fermions, while
$L^{\prime}\equiv[N^{\prime},E^{\prime}]^{T},\overline{L^{\prime}}\equiv[\overline{N^{\prime}},\overline{E^{\prime}}]^{T}$
are isospin doublet ones. We use only a bosonic part of $\widehat{\chi}$ that
is identified as a DM candidate, therefore $\chi$ is vanishing VEV. $\chi$ is
denoted by $\chi=(\chi_{R}+i\chi_{I})/\sqrt{2}$ that is $A_{4}$ trivial
singlet with $-3$ modular weight. The DM $\chi$ also plays a role in inducing
the muon $g-2$ together with $E$ and $L^{\prime}$. The superfield contents and
their charged assignments are shown in Table 1. Superfields $\widehat{H}_{u}$
and $\widehat{H}_{d}$ are introduced as MSSM. We denote their boson parts are
written by $H_{u}=[h_{u}^{+},(v_{u}+h_{u}+iz_{u})/\sqrt{2}]^{T}$ and
$H_{d}=[(v_{d}+h_{d}+iz_{d})/\sqrt{2},h_{d}^{-}]^{T}$ with totally neutral
charges under $A_{4}$ and $(-k)$, where the structure is exactly the same as
the minimum supersymmetric theory. The SM VEV is defined by
$v_{H}\equiv\sqrt{v_{u}^{2}+v_{d}^{2}}\equiv 246$ GeV. Under these symmetries,
the valid superpotential is found as follows:

$\displaystyle\mathcal{W}_{Y}$
$\displaystyle=y_{e}\widehat{\overline{e}}\widehat{H}_{d}\widehat{L}_{e}+y_{\mu}\widehat{\overline{\mu}}\widehat{H}_{d}\widehat{L}_{\mu}+y_{\tau}\widehat{\overline{\tau}}\widehat{H}_{d}\widehat{L}_{\tau}+g_{E}\widehat{\overline{L^{\prime}}}\widehat{L}_{\mu}\widehat{\chi}+y_{E}\widehat{\overline{\mu}}\widehat{E}\widehat{\chi}$
$\displaystyle
h_{E}\widehat{\overline{E}}\widehat{H}_{d}\widehat{L}^{\prime}+M_{E}\widehat{\overline{E}}\widehat{E}+M_{L^{\prime}}\widehat{\overline{L^{\prime}}}\widehat{L}^{\prime}+\mu_{H}\widehat{H}_{u}\widehat{H}_{d}+\mu_{\chi}\widehat{\chi}\widehat{\chi},$
(II.1)

where all couplings except $y_{e},\ y_{\tau},\ \mu_{H}$ implicitly includes
modular Yukawa couplings that are determined only by modulus value. Charged-
lepton sector does not mix each other, therefore the charged-lepton fields are
mass eigenstates. Due to even (odd) number of assignments for SM (new) fields
in modular weight, it is forbidden for any mixing terms between SM and new
fields such as $\widehat{\overline{\ell}}\widehat{H_{d}}\widehat{L^{\prime}}$,
$\widehat{\overline{\ell}}\widehat{E}$, and
$\widehat{\overline{L^{\prime}}}\widehat{L_{i}}$ where $\ell=e,\mu,\tau$. This
is because any odd numbers of modular couplings are mathematically forbidden.
However, we implicitly impose R-parity in order to prohibit superpotential
such as $\widehat{\overline{e}}\widehat{L_{\mu}}\widehat{L_{\mu}}$,
$\widehat{L_{e}}\widehat{H_{u}}$,
$\widehat{\overline{e}}\widehat{H_{d}}\widehat{H_{d}}$. The bosonic mass of
$\chi$ is obtained by the following soft SUSY breaking terms;

$\displaystyle-{\cal L}_{\rm soft}\sim$
$\displaystyle\mu_{B\chi}^{2}\chi^{2}+m^{2}_{\chi}|\chi|^{2}+{\rm h.c.},$
(II.2)

where $m_{\chi}^{2}$ includes F-term. Thus, mass of $\chi$ is given by
$m_{\chi_{R}}^{2}=m_{\chi}^{2}+\mu_{B\chi}^{2}$ and
$m_{\chi_{I}}^{2}=m_{\chi}^{2}-\mu_{B\chi}^{2}$, and we select DM as
$\chi\equiv\chi_{I}$, taking $0<\mu_{B\chi}^{2}$.

###  II.1 Heavy charged lepton mass matrix

After the electroweak spontaneous symmetry breaking, a heavily charged lepton
mass matrix is found on the basis of $[E,E^{\prime}]_{L}^{T}$ as

$\displaystyle\mathcal{M}_{E}=\begin{pmatrix}M_{E}&m_{E}\\\
0&M_{L^{\prime}}\\\ \end{pmatrix},$ (II.3)

where $m_{E}\equiv h_{E}v_{d}/\sqrt{2}$, and all the mass components are
supposed to be real without loss of generality after rephasing of fields.
Then, $\mathcal{M}_{E}$ is diagonalized by bi-unitary matrix as
diag$[m_{1},m_{2}]=V^{\dagger}_{E_{R}}\mathcal{M}_{E}V_{E_{L}}$, where we
define

$\displaystyle\begin{pmatrix}E^{\pm}\\\ E^{\prime\pm}\\\
\end{pmatrix}\equiv\begin{pmatrix}c_{E}&-s_{E}\\\ s_{E}&c_{E}\\\
\end{pmatrix}\begin{pmatrix}\psi_{E_{1}}^{\pm}\\\ \psi_{E_{2}}^{\pm}\\\
\end{pmatrix},$ (II.4)

where $s_{E}(c_{E})$ is short-hand notation for
$\sin\theta_{E}(\cos\theta_{E})$, and we abbreviate the electric charge
hereafter. Since these fermions have nonzero electric charges with specific
decays, there exist lower bounds on masses. If it decays into muon and
missing, the bound is about 90 GeV Workman and Others (2022). In our case,
$\psi_{E_{1}}^{\pm}$ decays (in limit of $s_{E}=0$) into muon and missing
energy ($=$DM) via $y_{E}$ and $\psi_{E_{2}}^{\pm}$ decays (in limit of
$s_{E}=0$) into muon, muon-neutrino, and the missing energy via kinetic term
of $L^{\prime}$ and $g_{E}$. We adopt the relaxing lower mass bound of 90 GeV
in our numerical analysis.

##  III Dark matter and muon anomalous magnetic dipole moment

###  III.1 Dark Matter and Fermi-LAT GeV excess

We consider a DM candidate $\chi_{I}$. Then, the dominant contribution to the
relic density of DM comes from the following relevant Lagrangian:

$\displaystyle-{\cal L}$
$\displaystyle=\frac{i}{\sqrt{2}}g_{E}\bar{E^{\prime}}\ell_{\mu}\chi_{I}+\frac{i}{\sqrt{2}}y_{E}{\bar{\mu}}E\chi_{I}+{\rm
c.c.}$
$\displaystyle=\frac{i}{\sqrt{2}}g_{E}(s_{E}\bar{\psi}_{E_{1}}+c_{E}\bar{\psi}_{E_{2}})\ell_{\mu}\chi_{I}+\frac{i}{\sqrt{2}}y_{E}{\bar{\mu}}(c_{E}\psi_{E_{1}}-s_{E}\psi_{E_{2}})\chi_{I}+{\rm
c.c.},$ (III.1)

where we suppose $g_{E},\ y_{E}$ to be real without loss of generality. Then,
the cross-section is expanded by relative velocity $v_{\rm rel}$ that is given
by $\sigma v_{\rm rel}\approx a_{\rm eff}+b_{\rm eff}v^{2}_{\rm rel}+{\cal
O}(v^{4}_{\rm rel})$, where $a_{\rm eff}$ is s-wave and $b_{\rm eff}$ is
p-wave. 333Even though we show this expansion up to p-wave, we have computed
the relic density of DM up to d-wave that is proportional to $v^{4}_{\rm
rel}$. Each of waves is written by

$\displaystyle a_{\rm eff}$
$\displaystyle\approx\frac{(y_{E}g_{E}s_{E}c_{E})^{2}}{2\pi}\left[\frac{m_{1}^{2}}{(m_{\chi}^{2}+m_{1}^{2})^{2}}+\frac{m_{2}^{2}}{(m_{\chi}^{2}+m_{2}^{2})^{2}}\right],$
(III.2) $\displaystyle b_{\rm eff}$
$\displaystyle\approx-\frac{(y_{E}g_{E}s_{E}c_{E}m_{\chi})^{2}}{6\pi(m_{\chi}^{2}+m_{1}^{2})^{4}(m_{\chi}^{2}+m_{2}^{2})^{4}}\left[m_{2}^{2}(m_{1}^{8}+4m_{1}^{6}m_{\chi}^{2}+m_{\chi}^{8})(3m_{2}^{2}+m_{\chi}^{2})\right.$
$\displaystyle\left.\hskip
14.22636pt+3m_{1}^{4}(m_{2}^{8}+4m_{2}^{6}m_{\chi}^{2}+12m_{2}^{4}m_{\chi}^{4}+6m_{2}^{2}m_{\chi}^{6}+m_{\chi}^{8})\right.$
$\displaystyle\left.\hskip
14.22636pt+m_{1}^{2}m_{\chi}^{2}(m_{2}^{8}+4m_{2}^{6}m_{\chi}^{2}+18m_{2}^{4}m_{\chi}^{4}+8m_{2}^{2}m_{\chi}^{6}+m_{\chi}^{8})\right],$
(III.3)

where we define the DM mass to be $m_{\chi}$. Since the cross section of
Fermi-LAT $(\sigma v)_{\rm FL}$ is measured at the present Universe, $v_{\rm
rel}$ is almost zero. It leads us to be $a_{\rm eff}\approx(\sigma v)_{\rm
FL}$. Then, one simply solves this equation in terms of one of our input
parameters $s_{E},\ y_{E},\ g_{E}$. Here, we solve it in term of $g_{E}$ and
the result is straightforwardly given by

$\displaystyle g_{E}=\pm\sqrt{\frac{2(\sigma v)_{\rm
FL}}{(y_{E}s_{E}c_{E})^{2}\left[\frac{m_{1}^{2}}{(m_{\chi}^{2}+m_{1}^{2})^{2}}+\frac{m_{2}^{2}}{(m_{\chi}^{2}+m_{2}^{2})^{2}}\right]}}.$
(III.4)

Thus, $g_{E}$ is not independent parameter any more hereafter, and we have to
impose perturbative limit which we take $|g_{E}|\leq 1$ for more conservative
manner.

To explain the gamma-ray GeV excess from galactic center, DM annihilation to
the different SM particles is suggested. One of the solutions for the Fermi-
LAT Gamma-ray excess is that the DM annihilation cross section to muons is
$(3.9_{-0.6}^{+0.5})\times 10^{-26}$cm3/sec with DM mass $58_{-9}^{+11}$ GeV
at 1$\sigma$ interval Di Mauro and Winkler (2021). The required DM
annihilation cross section is very close to the canonical value for thermal
freeze-out DM relic.

Relic density of DM is found in terms of expansion coefficients of $v_{\rm
rel}$ and approximately given by Bertone et al. (2005)

$\displaystyle\Omega_{\rm DM}h^{2}\approx\frac{1.07\times
10^{9}x_{F}^{2}}{\sqrt{g_{*}}M_{\rm PL}(a_{\rm eff}x_{F}+3b_{\rm eff})},$
(III.5)

where $M_{\rm PL}=1.22\times 10^{19}$ GeV, and $x_{F}\approx 24$. In our
numerical analysis, we apply the relaxed observed relic density
$0.11\leq\Omega_{\rm DM}h^{2}\leq 0.13$ that is about 5$\sigma$ interval
instead of exact value $\Omega_{\rm DM}h^{2}=0.1197\pm 0.0022$ at 1$\sigma$
interval Ade et al. (2016).

###  III.2 Muon anomalous magnetic dipole moment: $\Delta a_{\mu}$

A muon anomalous magnetic dipole moment ($\Delta a_{\mu}$ or muon $g-2$) has
been first reported by Brookhaven National Laboratory (BNL) Bennett et al.
(2006). They found that the muon $g-2$ data has a discrepancy at the
3.3$\sigma$ level from the SM prediction. Recent experimental result of muon
$g-2$ suggests the following value at $4.2\sigma$ Abi et al. (2021):

$\displaystyle\Delta a_{\mu}=a_{\mu}^{\rm{EXP}}-a_{\mu}^{\rm{SM}}=(25.1\pm
5.9)\times 10^{-10}.$ (III.6)

To get sizable muon $g-2$ at one-loop level, we would need a chiral flip
diagram and this contribution is obtained by the same term as Eq.(III.1). Our
muon $g-2$ at one-loop level is found as follows Hutauruk et al. (2020):

$\displaystyle\Delta
a_{\mu}=\frac{m_{\mu}}{(4\pi)^{2}}y_{E}g_{E}s_{E}c_{E}\left[F_{I}(m_{\chi},m_{1})-F_{I}(m_{\chi},m_{2})\right],$
(III.7) $\displaystyle
F_{I}(m_{a},m_{b})\simeq-\frac{m_{b}\left(3m_{a}^{4}-4m_{a}^{2}m_{b}^{2}+m_{b}^{4}+4m_{a}^{4}\ln\left[\frac{m_{b}}{m_{a}}\right]\right)}{2(m_{a}^{2}-m_{b}^{2})^{3}}.$
(III.8)

##  IV Numerical analysis

In our numerical analysis, we randomly scan free parameters such that

$\displaystyle\\{y_{E},s_{E}\\}\in[0.01,1],\quad\\{m_{1},m_{2}\\}\in[90,1000]\
{\rm GeV},$ (IV.1)

where the other parameter $g_{E}$ is determined by Eq. (III.4). Importing the
measured observables $m_{\chi},\ (\sigma v)_{\rm FL},\ \Omega_{\rm DM}h^{2},\
\Delta a_{\mu}$, we plot two figures below where we take them up to 2$\sigma$
interval for $(\sigma v)_{\rm FL},\ \Delta a_{\mu}$, 1$\sigma$ interval for
$m_{\chi}$, and 5$\sigma$ for $\Omega_{\rm DM}h^{2}$ as discussed in the
previous section.

Figure 1: (Left) Allowed region in terms of $(\sigma v)_{\rm FL}\times
10^{26}{\rm cm^{3}/sec}$ and $\Delta a_{\mu}\times 10^{10}$. (Right) Preferred
region for $(\Omega_{\rm DM}h^{2},\Delta a_{\mu})$. All of the points are
satisfied with Fermi-LAT GeV excess within $1\sigma$. The red horizontal line
indicates the central value of the DM relic abundance observed by Planck Ade
et al. (2016). In both left and right figures, the blue plot represents
1$\sigma$ interval of $\Delta a_{\mu}$ while the green one 2$\sigma$. the
black vertical line is the best fit value of $\Delta a_{\mu}$.

In the left panel of Fig. 1, we show the allowed region in terms of $(\sigma
v)_{\rm FL}\times 10^{26}{\rm cm^{3}/sec}$ and $\Delta a_{\mu}\times 10^{10}$.
The blue plot represents 1$\sigma$ interval of $\Delta a_{\mu}$ while the
green one 2$\sigma$. The black vertical line is the best fit value of $\Delta
a_{\mu}$. The figure suggests that we do not have allowed region within
1$\sigma$ of $(\sigma v)_{\rm FL}$ but have within 2$\sigma$. In the right
panel of Fig. 1, we show the allowed region in terms of $\Omega_{\rm DM}h^{2}$
and $\Delta a_{\mu}\times 10^{10}$. This figure implies that most of the
region is below the BF of $\Omega_{\rm DM}h^{2}$ but a few points reach at
this value.

##  V Conclusions and discussions

We have shown the successful explanation of Fermi-LAT GeV excess where DM is
supposed to be muon specific, applying a modular $A_{4}$ symmetry. Thanks to
nature of the symmetry, our DM interacts with pairs of muon only. Moreover, we
have explained sizable muon anomalous magnetic dipole moment without suffering
from lepton flavor violations. We have numerically demonstrated our allowed
spaces on each the measurements above, randomly scanning our input parameters.

###### Acknowledgements.

The work of J.K. is supported in part by Korea Institute for Advanced Study
(KIAS) Individual Grant. No. PG074202. The work of H.O. was supported by the
Junior Research Group (JRG) Program at the Asia-Pacific Center for Theoretical
Physics (APCTP) through the Science and Technology Promotion Fund and Lottery
Fund of the Korean Government and was supported by the Korean Local
Governments-Gyeongsangbuk-do Province and Pohang City.

## References

  * Feruglio (2019) F. Feruglio, _Are neutrino masses modular forms?_ (2019), pp. 227–266, eprint 1706.08749.
  * Criado and Feruglio (2018) J. C. Criado and F. Feruglio, SciPost Phys. 5, 042 (2018), eprint 1807.01125.
  * Kobayashi et al. (2018a) T. Kobayashi, N. Omoto, Y. Shimizu, K. Takagi, M. Tanimoto, and T. H. Tatsuishi, JHEP 11, 196 (2018a), eprint 1808.03012.
  * Okada and Tanimoto (2019a) H. Okada and M. Tanimoto, Phys. Lett. B 791, 54 (2019a), eprint 1812.09677.
  * Okada and Tanimoto (2019b) H. Okada and M. Tanimoto (2019b), eprint 1905.13421.
  * de Anda et al. (2020) F. J. de Anda, S. F. King, and E. Perdomo, Phys. Rev. D 101, 015028 (2020), eprint 1812.05620.
  * Novichkov et al. (2019) P. Novichkov, S. Petcov, and M. Tanimoto, Phys. Lett. B 793, 247 (2019), eprint 1812.11289.
  * Nomura and Okada (2019a) T. Nomura and H. Okada (2019a), eprint 1906.03927.
  * Okada and Orikasa (2019) H. Okada and Y. Orikasa (2019), eprint 1907.13520.
  * Ding et al. (2019a) G.-J. Ding, S. F. King, and X.-G. Liu, JHEP 09, 074 (2019a), eprint 1907.11714.
  * Nomura et al. (2020) T. Nomura, H. Okada, and O. Popov, Phys. Lett. B 803, 135294 (2020), eprint 1908.07457.
  * Kobayashi et al. (2019a) T. Kobayashi, Y. Shimizu, K. Takagi, M. Tanimoto, and T. H. Tatsuishi, Phys. Rev. D 100, 115045 (2019a), [Erratum: Phys.Rev.D 101, 039904 (2020)], eprint 1909.05139.
  * Zhang (2020) D. Zhang, Nucl. Phys. B 952, 114935 (2020), eprint 1910.07869.
  * Ding et al. (2019b) G.-J. Ding, S. F. King, X.-G. Liu, and J.-N. Lu, JHEP 12, 030 (2019b), eprint 1910.03460.
  * Kobayashi et al. (2020) T. Kobayashi, T. Nomura, and T. Shimomura, Phys. Rev. D 102, 035019 (2020), eprint 1912.00637.
  * Nomura et al. (2019) T. Nomura, H. Okada, and S. Patra (2019), eprint 1912.00379.
  * Wang (2020) X. Wang, Nucl. Phys. B 957, 115105 (2020), eprint 1912.13284.
  * Okada and Shoji (2020) H. Okada and Y. Shoji, Nucl. Phys. B 961, 115216 (2020), eprint 2003.13219.
  * Okada and Tanimoto (2020a) H. Okada and M. Tanimoto (2020a), eprint 2005.00775.
  * Behera et al. (2020a) M. K. Behera, S. Singirala, S. Mishra, and R. Mohanta (2020a), eprint 2009.01806.
  * Nomura and Okada (2020a) T. Nomura and H. Okada (2020a), eprint 2007.04801.
  * Nomura and Okada (2020b) T. Nomura and H. Okada (2020b), eprint 2007.15459.
  * Asaka et al. (2020a) T. Asaka, Y. Heo, and T. Yoshida, Phys. Lett. B 811, 135956 (2020a), eprint 2009.12120.
  * Okada and Tanimoto (2020b) H. Okada and M. Tanimoto (2020b), eprint 2009.14242.
  * Nagao and Okada (2020) K. I. Nagao and H. Okada (2020), eprint 2010.03348.
  * Okada and Tanimoto (2020c) H. Okada and M. Tanimoto (2020c), eprint 2012.01688.
  * Kobayashi et al. (2018b) T. Kobayashi, K. Tanaka, and T. H. Tatsuishi, Phys. Rev. D 98, 016004 (2018b), eprint 1803.10391.
  * Yao et al. (2021) C.-Y. Yao, J.-N. Lu, and G.-J. Ding, JHEP 05, 102 (2021), eprint 2012.13390.
  * Chen et al. (2021) P. Chen, G.-J. Ding, and S. F. King, JHEP 04, 239 (2021), eprint 2101.12724.
  * de Medeiros Varzielas and Lourenço (2022) I. de Medeiros Varzielas and J. a. Lourenço, Nucl. Phys. B 979, 115793 (2022), eprint 2107.04042.
  * Nomura et al. (2021a) T. Nomura, H. Okada, and Y. Orikasa, Eur. Phys. J. C 81, 947 (2021a), eprint 2106.12375.
  * Ding et al. (2021) G.-J. Ding, S. F. King, and J.-N. Lu, JHEP 11, 007 (2021), eprint 2108.09655.
  * Nagao and Okada (2022) K. I. Nagao and H. Okada, Phys. Dark Univ. 36, 101039 (2022), eprint 2108.09984.
  * Okada and Qi (2021) H. Okada and Y.-h. Qi (2021), eprint 2109.13779.
  * Nomura et al. (2021b) T. Nomura, H. Okada, and Y.-h. Qi (2021b), eprint 2111.10944.
  * Liu and Ding (2022) X.-G. Liu and G.-J. Ding, JHEP 03, 123 (2022), eprint 2112.14761.
  * Nomura and Okada (2022) T. Nomura and H. Okada (2022), eprint 2201.10244.
  * Otsuka and Okada (2022) H. Otsuka and H. Okada (2022), eprint 2202.10089.
  * Kang et al. (2022) D. W. Kang, J. Kim, T. Nomura, and H. Okada, JHEP 07, 050 (2022), eprint 2205.08269.
  * Ishiguro et al. (2022) K. Ishiguro, H. Okada, and H. Otsuka, JHEP 09, 072 (2022), eprint 2206.04313.
  * Nomura et al. (2022) T. Nomura, H. Okada, and Y. Shoji (2022), eprint 2206.04466.
  * Du and Wang (2023) X. K. Du and F. Wang, JHEP 01, 036 (2023), eprint 2209.08796.
  * Kikuchi et al. (2022) S. Kikuchi, T. Kobayashi, M. Tanimoto, and H. Uchida (2022), eprint 2207.04609.
  * Gogoi et al. (2022) J. Gogoi, N. Gautam, and M. K. Das (2022), eprint 2207.10546.
  * Abbas and Khalil (2022) M. Abbas and S. Khalil (2022), eprint 2212.10666.
  * Petcov and Tanimoto (2022) S. T. Petcov and M. Tanimoto (2022), eprint 2212.13336.
  * Kikuchi et al. (2023) S. Kikuchi, T. Kobayashi, K. Nasu, S. Takada, and H. Uchida (2023), eprint 2302.03326.
  * Kobayashi et al. (2019b) T. Kobayashi, Y. Shimizu, K. Takagi, M. Tanimoto, T. H. Tatsuishi, and H. Uchida, Phys. Lett. B 794, 114 (2019b), eprint 1812.11072.
  * Asaka et al. (2020b) T. Asaka, Y. Heo, T. H. Tatsuishi, and T. Yoshida, JHEP 01, 144 (2020b), eprint 1909.06520.
  * Kashav and Verma (2021) M. Kashav and S. Verma, JHEP 09, 100 (2021), eprint 2103.07207.
  * Okada et al. (2021) H. Okada, Y. Shimizu, M. Tanimoto, and T. Yoshida, JHEP 07, 184 (2021), eprint 2105.14292.
  * Dasgupta et al. (2021) A. Dasgupta, T. Nomura, H. Okada, O. Popov, and M. Tanimoto (2021), eprint 2111.06898.
  * Behera et al. (2020b) M. K. Behera, S. Mishra, S. Singirala, and R. Mohanta (2020b), eprint 2007.00545.
  * Ding et al. (2022) G.-J. Ding, S. F. King, J.-N. Lu, and B.-Y. Qu, JHEP 10, 071 (2022), eprint 2206.14675.
  * Hutauruk et al. (2020) P. T. P. Hutauruk, D. W. Kang, J. Kim, and H. Okada (2020), eprint 2012.11156.
  * Kobayashi et al. (2022a) T. Kobayashi, H. Otsuka, M. Tanimoto, and K. Yamamoto, Phys. Rev. D 105, 055022 (2022a), eprint 2112.00493.
  * Kobayashi et al. (2022b) T. Kobayashi, H. Otsuka, M. Tanimoto, and K. Yamamoto, JHEP 08, 013 (2022b), eprint 2204.12325.
  * Gunji et al. (2022) Y. Gunji, K. Ishiwata, and T. Yoshida, JHEP 11, 002 (2022), eprint 2208.10086.
  * Nomura and Okada (2019b) T. Nomura and H. Okada, Phys. Lett. B 797, 134799 (2019b), eprint 1904.03937.
  * Kobayashi et al. (2022c) T. Kobayashi, H. Okada, and Y. Orikasa, Phys. Dark Univ. 37, 101080 (2022c), eprint 2111.05674.
  * Lavoura et al. (2013) L. Lavoura, S. Morisi, and J. W. F. Valle, JHEP 02, 118 (2013), eprint 1205.3442.
  * Hirsch et al. (2010) M. Hirsch, S. Morisi, E. Peinado, and J. W. F. Valle, Phys. Rev. D 82, 116003 (2010), eprint 1007.0871.
  * Boucenna et al. (2011) M. S. Boucenna, M. Hirsch, S. Morisi, E. Peinado, M. Taoso, and J. W. F. Valle, JHEP 05, 037 (2011), eprint 1101.2874.
  * Kajiyama and Okada (2011) Y. Kajiyama and H. Okada, Nucl. Phys. B 848, 303 (2011), eprint 1011.5753.
  * Haba et al. (2011) N. Haba, Y. Kajiyama, S. Matsumoto, H. Okada, and K. Yoshioka, Phys. Lett. B 695, 476 (2011), eprint 1008.4777.
  * Kajiyama et al. (2014) Y. Kajiyama, H. Okada, and T. Toma, Eur. Phys. J. C 74, 2722 (2014), eprint 1304.2680.
  * Abi et al. (2021) B. Abi et al. (Muon g-2), Phys. Rev. Lett. 126, 141801 (2021), eprint 2104.03281.
  * Bennett et al. (2006) G. W. Bennett et al. (Muon g-2), Phys. Rev. D 73, 072003 (2006), eprint hep-ex/0602035.
  * He et al. (1991) X. G. He, G. C. Joshi, H. Lew, and R. R. Volkas, Phys. Rev. D 43, 22 (1991).
  * Baek et al. (2001) S. Baek, N. G. Deshpande, X. G. He, and P. Ko, Phys. Rev. D 64, 055006 (2001), eprint hep-ph/0104141.
  * Chun et al. (2019) E. J. Chun, J. Kim, and T. Mondal, JHEP 12, 068 (2019), eprint 1906.00612.
  * Asai et al. (2021) K. Asai, S. Okawa, and K. Tsumura, JHEP 03, 047 (2021), eprint 2011.03165.
  * Drees and Zhao (2022) M. Drees and W. Zhao, Phys. Lett. B 827, 136948 (2022), eprint 2107.14528.
  * Holst et al. (2022) I. Holst, D. Hooper, and G. Krnjaic, Phys. Rev. Lett. 128, 141802 (2022), eprint 2107.09067.
  * Baek et al. (2022) S. Baek, J. Kim, and P. Ko (2022), eprint 2204.04889.
  * Kim (2022) J. Kim, Phys. Lett. B 832, 137220 (2022), eprint 2205.01437.
  * Okada and Seto (2014) N. Okada and O. Seto, Phys. Rev. D 89, 043525 (2014), eprint 1310.5991.
  * Ko et al. (2014) P. Ko, W.-I. Park, and Y. Tang, JCAP 09, 013 (2014), eprint 1404.5257.
  * Mondal and Basak (2015) T. Mondal and T. Basak, Phys. Lett. B 744, 208 (2015), eprint 1405.4877.
  * Park et al. (2016) J.-C. Park, J. Kim, and S. C. Park, Phys. Lett. B 752, 59 (2016), eprint 1505.04620.
  * Workman and Others (2022) R. L. Workman and Others (Particle Data Group), PTEP 2022, 083C01 (2022).
  * Di Mauro and Winkler (2021) M. Di Mauro and M. W. Winkler, Phys. Rev. D 103, 123005 (2021), eprint 2101.11027.
  * Bertone et al. (2005) G. Bertone, D. Hooper, and J. Silk, Phys. Rept. 405, 279 (2005), eprint hep-ph/0404175.
  * Ade et al. (2016) P. A. R. Ade et al. (Planck), Astron. Astrophys. 594, A13 (2016), eprint 1502.01589.