# Towards Automating Codenames Spymasters with Deep Reinforcement Learning

Sherman Siu<EMAIL_ADDRESS>
Cheriton School of Computer Science
University of Waterloo
Waterloo, ON N2L 3G1, Canada

Keywords: reinforcement learning, deep learning, board games, natural language
understanding, Codenames

##  1 Introduction

In artificial intelligence, games are often used as a benchmark for measuring
how well artificial intelligence agents can interact with the world. In the
recent years, there have been many breakthroughs in AI for single-player games
like Atari games (Mnih et al., 2013) and Go (Silver et al., 2016), or
competitive multi-player games like Starcraft II (Vinyals et al., 2019).
However, much less work has been done investigating multi-player co-operative
games. Nevertheless, some work has been made in this area, such as for Hanabi
(Lerer et al., 2020), a co-operative game with imperfect information and
online multiplayer Quake III Arena Capture the Flag (Jaderberg et al., 2019).
Although traditional multi-agent reinforcement learning (RL) techniques tend
to work well when RL agents work with each other, they fail to work well when
co-operating with humans (Siu et al., 2021; Bakhtin et al., 2021).

For reinforcement learning, text-based games have received comparatively less
attention than other types of games and they are an open problem for AI-based
research. One of the main challenges with text-based games is the large action
space available to agents. Another challenge is the fact that text-based games
often require some common sense about how the world works. Thus, text-based
games are a good benchmark for natural language understanding (Côté et al.,
2018). Thus, Codenames is a good benchmark for both human-AI co-operation and
text-based reinforcement learning, which are both several important areas of
AI research.

###  1.1 Overview of Codenames

(a) An in-progress Codenames game. Both teams have the same number of cards,
but it is blue’s turn.

(b) The same game, except the red team accidentally chose the black assassin
card. The colours of the remaining cards are revealed.

Figure 1: A game of Codenames has an associated 5×5 board of words, each of
which has a hidden colour known only to the spymasters of each team until the
cards are picked.

Codenames is a text-based board game that is both text-based and involves
asymmetric cooperation. In Codenames, there are two teams of approximately
equal size, a “red” team and a “blue” team (Chvátil, 2015). Each team has one
spymaster, who has complete information of the board and one or more
operatives, who have incomplete information. At the beginning of the game, 25
word cards are drawn from the deck and placed on the board, as shown in Figure
1. Each word is also assigned one of four labels: red, blue, bystander, and
assassin. The goal of each spymaster is to guide their team’s operatives to
choosing their own team’s cards, while avoiding the others. These labels are
known to the spymasters from the start of the game, while the operatives know
none of the associated labels at the beginning of the game. As the game
progresses, the operatives discover the labels of the cards that have been
chosen by the operatives of either team.

Although the spymaster has perfect information, the spymaster is only allowed
to communicate with their team’s operatives by providing a single hint each
term, consisting of a single word and number, as shown in Figure 2 (Chvátil,
2015). The word cannot be a superstring or substring of any of the words on
the board. The word cannot give a directional clue, such as “left” to suggest
that the correct words are on the left side of the board. The number indicates
how many words on the board relate to the hint word. Thus, the spymaster must
only provide a hint that relies on the meaning of the words.

[.wok 3 iron ham Beijing ] [.fist 3 fighter round palm ] [.Christmas 2 tree
winter ] [.weightlifting 2 iron plate ]

Figure 2: Examples of spymaster hints, with the intended guesser words below.
The guesser words are supposed to be similar to the spymaster hints in some
way.

The hint number can also be 0 to indicate that none of the team’s remaining
words relate to that hint word or infinity to indicate that the team can make
as many guesses as they want before choosing to end their turn (Chvátil,
2015).

Codenames is a zero-sum, turn-based game. Regardless of the team’s colour, the
first team to start playing has 9 cards belonging to their team and the second
has 8 (Chvátil, 2015). There are always 7 bystander cards and 1 assassin card.
After hearing the spymaster’s hint, the operatives of each team have the
choice of either flipping over a word card or ending their turn. If the card’s
colour isn’t their team’s colour, their team’s turn immediately ends.
Otherwise, the team has the choice to either continue flipping cards or ending
their turn. If a team flips the assassin card, that team immediately loses. If
none of the teams flip over the assassin card, the first team to flip over all
of their cards wins. Hence, spymasters must be careful to not suggest hints
that might help the other team or cause the assassin card to be flipped.

###  1.2 My Contributions

To the best of my knowledge, there are no reinforcement learning-based
spymaster agents that have been used for Codenames, despite the success of
reinforcement learning (Mnih et al., 2013; Christiano et al., 2017; Silver et
al., 2018), even when applied to environments in which natural language
processing plays a huge part (Ouyang et al., 2022; Bakhtin et al., 2022a).

  1. 1.

As a result, I formulate a game of Codenames from a Spymaster’s perspective as
a Markov Decision Process (MDP) and implement several OpenAI Gym (Brockman et
al., 2016) environments for a spymaster agent and

  2. 2.

demonstrate that several agents (SAC, SAC with HER), PPO, and A2C) do not
cause converge when trained on all of the above environments. Moreover, both a
random policy and a simply greedy policy beat all of the above models on the
Codenames environments.

  3. 3.

I formulate two simplified versions of the environment called “ClickPixel” and
“Whack-a-Mole” and demonstrate that while A2C and sometimes PPO converges when
the environment is small, they fail to converge as the state space and action
space are scaled up.

##  2 Related works

Prior works have explored both how to automate the spymaster and how to
approximate how humans come up with the word associations used in the game
Codenames.

###  2.1 Pairwise Word Similarity

A lot of the research in the existing literature focuses on improving the word
similarity function used by both the automated spymasters and guessers. This
is because an improvement to the word similarity function directly translates
to an improvement in the performance of both spymasters and guessers.

All existing spymaster algorithms leverage a pairwise word similarity function
$\sigma(w_{i},w_{j})$ or non-negative distance function $d(w_{i},w_{j})$ to
suggest hints $h=(c,n)$ to the guessers, where $c$ denotes the clue word and
$n$ denotes the number of words to guess. Prior work has explored $\sigma$ or
$d$ functions derived from knowledge graphs (Kim et al., 2019; Koyyalagunta et
al., 2021), the cosine similarity of text embeddings (Shen et al., 2018; Kim
et al., 2019; Jaramillo et al., 2020; Koyyalagunta et al., 2021; Mills, 2022;
Cserháti et al., 2022; Mills, 2022), TF-IDF (Jaramillo et al., 2020), and
Naive Bayes (Jaramillo et al., 2020). Similarity functions that directly
leverage the frequency of the collocation of bigrams have also been used (Shen
et al., 2018; Mills, 2022), as well as ones based off the related normalized
pointwise mutual information (NMPI) metric Cserháti et al. (2022), which is
closely related. Word embedding and co-occurrence metric approaches have been
the most effective methods for approximating human word associations in
Codenames. While word embedding approaches are more time- and space-efficient
for calculating the similarity between different words, they occasionally fail
to capture word associations from collocations (Smrz, 2019).

Some of the word embeddings that have been used for autonomous Codenames
agents include Word2Vec (Mikolov et al., 2013), GloVe (Pennington et al.,
2014), fastText (Bojanowski et al., 2016), Conceptnet Numberbatch (Speer et
al., 2017), and Sentence-BERT (Reimers and Gurevych, 2019), none of which are
derived from human-labeled word associations. Kumar et al. (2021) constructed
embeddings from several word association datasets, such as the Small World of
Words (SWOW) dataset (De Deyne et al., 2018) and the University of Southern
Florida (USF) dataset (Nelson et al., 2004), which each record the frequency
of the first words that come to mind when people are presented with word
prompts. Kumar et al. (2021) found that these embeddings produced word
associations that were more similar to human chosen ones than word2vec and
GloVe in a simplified version of Codenames.

###  2.2 Automating the Spymaster

Let $M$ denote the set of words on the current player’s team and $U=O\cup
B\cup A$ denote the set of “bad” words, which includes words from the opposing
team, bystander, and assassin words. Let $I_{n}$ denote the target or intended
set of $n$ words on the board that is supposed to be targeted with the clue
word $c$. When cosine similarity is used, the range of $\sigma$ is $[-1,1]$.
Cosine distance is calculated from the cosine similarity by
$d(w_{i},w_{j})\coloneqq 1-\sigma(w_{i},w_{j})$.

In the literature, the spymasters choose words within the vocabulary $V$ which
either maximize a scoring function $g(c,n)$ or minimize an energy function
$f(c,n)$. Kim et al. (2019) uses the energy function

$f_{Kim}(c,n)=\begin{cases}\max_{w\in I_{n}}d(c,w)&\text{if}\max_{w\in
I_{n}}d(c,w)<\min_{u\in U}d(c,u)\\\ &\text{ and }\max_{w\in
I_{n}}d(c,w)<\lambda_{T}\\\ \infty&\text{otherwise}\end{cases}$ (1)

where $\lambda_{T}$ is a parameter that governs how aggressive the spymaster
is (low=0.3, high=0.7).

In all of the three final approaches mentioned in Jaramillo et al. (2020),
they reweigh unintended words by -1 for bystanders, -2 for the opposing team’s
words, and -3 for the assassin, which I’ll denote by $k(u)$ for $u\in U$. Let
$\vec{v}_{w}$ denote the embedding for word $w$ and let
$csm(\vec{v},\vec{x})=\frac{\vec{v}\cdot\vec{x}}{\lVert\vec{v}\rVert\lVert\vec{x}\rVert}$.
Their best-performing algorithm leveraged word embeddings, with the scoring
function

$g_{Jara}(c,n)=csm\left(\vec{c},\frac{1}{\lvert I_{n}\rvert+\lvert
U\rvert}(\sum_{w\in I_{n}}\vec{w}+\sum_{u\in U}k(u)\vec{u})\right)$ (2)

Jaramillo et al. (2020) derived equation (2) by adding bad word reweighing to

$g_{mean}=csm(\vec{c},\frac{1}{\lvert I_{n}\rvert}\sum_{w\in I_{n}}\vec{w})$
(3)

Koyyalagunta et al. (2021) and Cserháti et al. (2022) also define their own
scoring functions, but only test them in a simplified Codenames environment
with 20 words and just red and blue words.

Unlike prior works, I leave the bad word penalization to the RL agent and thus
I do not directly penalize unintended words in the scoring function. Moreover,
my RL environment is designed to work for a full game of Codenames.

###  2.3 Automating the Guessers

All prior work automates the guessers by greedily choosing the word on the
board that maximizes $\sigma(c,w)$ or that minimizes $d(c,w)$ for a given clue
word $c$. Some works also looked at the correlation between the similarity of
the scoring function and the human accuracy of guessing words (Kumar et al.,
2021) and (Cserháti et al., 2022).

##  3 Methodology

To adapt a version Codenames for reinforcement learning, I first describe it
as a Markov Decision Process (MDP), that is solely acted upon by the
spymaster. I then describe how this MDP has been implemented as several OpenAI
Gym (Brockman et al., 2016) environments. I then describe the reinforcement
learning (RL) agents that I will evaluate the environments on. Because
convergence on a Codenames environment fails empirically, I will also describe
a simplified Codenames environment, called “ClickPixel”.

###  3.1 Formulation of Codenames as a MDP

Let $\mathcal{B}=M\cup U$, the set of 25 words on the board. Let $L$ denote
the set of labels for the words on the board. Let $\ell\colon B\mapsto L$ be
the oracle label function and let $m_{t}\colon B\mapsto\\{0,1\\}$ be the
masking function that indicates whether a word is chosen (1) or not (0) at
step $t$. Let $\ell^{\prime}_{t}$ be the normalized label function that
relabels the “blue” and “red” labels to “mine” and “opposing” depending on
whose turn it is at time step $t$. The state
$s_{t}=(\mathcal{B},m_{t},\ell^{\prime}_{t})$ at time $t$.

In general, the action space for the spymasters is $A=V\times[0,9]$, where $V$
is the vocabulary. Infinite word hints can be reduced to hints where $n$ is
the number of words remaining, which is at most $9$. For the sake of
simplicity, zero-word hints will not be handled for now, so $A=V\times[1,9]$.

Between each step, all of the guessers for the team will make guesses based on
the given action $a_{t}=(c_{t},n_{t})$ and until their turn ends. The guessers
will guess the most similar $n_{t}$ words to the clue $c_{t}$ as long as
$s(c,w)>\lambda$ for some hyperparameter $\lambda$, for each candidate word
$w$, in descending order of similarity.

To match the scoring function used in Kim et al. (2019), the agent will
receive a reward of $-1$ for each turn when the agent has not won yet, $-25$
once the spymaster chooses the assassin word and $0$ once the spymaster wins.
Thus, if the assassin word is never chosen, the final score is the negation of
the number of turns it took for both teams to complete the board.

###  3.2 Codenames Gym Environment

I implement the above Codenames MDP as a a Gym (Brockman et al., 2016)
environment in my experiments as described below.

####  3.2.1 State Encoding Method

##### Pairwise Word Cosine Similarity Matrix (PWCSM)

The state space is encoded as a $[-1,1]^{j\lvert\mathcal{B}\rvert+\lvert
L\rvert+2\times\lvert\mathcal{B}\rvert}$ matrix, where $j$ is the number of
word embeddings I want the agent to refer to. The first
$j\lvert\mathcal{B}\rvert$ rows encode $j$ similarity matrices encoding the
pairwise cosine similarity of each word on the board. $j$ is set to 1 in
practice, as no improvements were found when using different similarity
matrices computed with different word embeddings.

##### One-Hot Word Encoding (OHWE)

Assuming that $\lvert V\rvert\equiv 0\;(\text{mod }\lvert\mathcal{B}\rvert)$,
which is true for a default game of Codenames ($\lvert V\rvert=400$,
$\lvert\mathcal{B}\rvert=25$), the state space is encoded as a $[0,1]^{\lvert
V\rvert/\lvert\mathcal{B}\rvert+\lvert
L\rvert+2\times\lvert\mathcal{B}\rvert}$ matrix. The first $\lvert
V\rvert/\lvert\mathcal{B}\rvert$ rows are a wrapped one-hot membership vector
for whether the words in the vocabulary are on the board.

####  3.2.2 Action Encoding Method

Let $G$ be the set of scoring functions used by the spymaster. The action is
encoded as a $[0,1]^{2\lvert\mathcal{B}\rvert+\lvert G\rvert\kappa\times 1}$
matrix, where $\kappa$ represents the number of word candidates generated. In
other words, this is a one-hot vector of each word indicating which words were
included in the target set $I_{n}$, which scoring strategy was used, and which
of the $\kappa$ generated words were chosen by the spymaster.

####  3.2.3 Spymaster Strategy

The first scoring function used is $g_{mean}$, which was introduced by
Jaramillo et al. (2020). The second is

$g_{minimax}(c,n)=\begin{cases}\min_{w\in I_{n}}s(c,w)&\text{if}\min_{w\in
I_{n}}s(c,w)>\lambda_{T}\\\ 0&\text{otherwise}\end{cases}$ (4)

which is essentially $g_{Kim}$ except I remove the constraint on ensuring that
bad words are not recommended, as I want the reinforcement learning algorithm
to learn which actions are bad.

####  3.2.4 Guesser Strategy

Although the environment can be configured to use (1) the greedy word
selection strategy as in prior works, I also introduce (2) a novel stochastic
guesser strategy. $n_{t}$ words are drawn without replacement from the
distribution $softmax(\frac{s(c,\cdot)}{\tau})$, where the temperature $\tau$
is set to $0.05$ in practice. Empirically, the words are roughly chosen in the
same order as the greedy approach, though words $w$ that have a close $s(c,w)$
value may have a slightly similar order.

####  3.2.5 Word Embeddings

Although the environment can support both GloVe and Conceptnet Numberbatch
embeddings, the uncased GloVe-300d embeddings trained on 6 billion tokens from
the Wikipedia 2014 and Gigaword 5 corpora are used for the experiments.

####  3.2.6 Approximate Nearest Neighbour Acceleration

Because searching for the $n_{t}$ nearest neighbours to $I_{n}$ in $V$
according to the cosine similarity function is slow for large $V$ using exact
distances, the Scalable Nearest Neighbour (ScaNN) algorithm (Guo et al., 2020)
is used to calculate the approximate $n_{t}$ nearest neighbours according to
the cosine similarity. Using ScaNN to accelerate the environment speeds up the
training from about 1-2 seconds per iteration to about 65-80 iterations per
second.

###  3.3 RL Agents

I use the soft actor critic (SAC) (Haarnoja et al., 2018), SAC with hindsight
experience replay (HER) (Andrychowicz et al., 2017), proximal policy
optimization (PPO) (Schulman et al., 2017), and advantage actor critic (A2C)
(Mnih et al., 2016) agents implemented in Stable Baselines 3 (Raffin et al.,
2021) for my experiments.

###  3.4 ClickPixel Gym Environment

To figure out why the agents failed to converge on the Codenames environment,
a simplified environment called “ClickPixel” was created. In ClickPixel, a
single pixel is highlighted in a board of $q\times q$ pixels. The rest of the
pixels are unhighlighted by default and are highlighted when chosen by the
agent. Because only PPO and A2C support discrete action spaces in Stable
Baselines 3, only these two agents are evaluated on ClickPixel.

###  3.5 Whack-a-Mole Gym Environment

Since in ClickPixel, the agent is required to select the a single grid on the
board, whereas in Codenames, the agent is required to select a subset of the
grids on the board, we also introduce the Whack-a-Mole environment, where the
agent has to select the highlighted subset of words at each time step. The
agent is rewarded with the mean classification accuracy of choosing or not
choosing each word shown on the board.

##  4 Results

I find that none of the Codenames environments converged after being trained
for 100k steps. In fact, a random policy sampled from the action space had
produced a return ($-32.00\pm 3.97$) that was similar to the return produced
by all four agents and a greedy strategy choosing one word at a time, using
the first word recommended from the $g_{mean}$ scoring function had a mean
return of $-10.4\pm 5.9$. However, when using a $2\times 2$ board with a word
of each colour, all four agents converge when using a PWCSM environment and
both PPO and A2C converge on a OHWE environment, though neither of the SAC
agents converge.

###  4.1 ClickPixel

Interestingly enough, while both PPO and A2C converge when $q=2$, only A2C
converges when $q=5$ (the width of a board in Codenames) and neither agent
converges when $q=36$ (the minimum dimensions required for CNNs to be used as
the encoding head in Stable Baselines 3), which highlights the difficulty of
RL algorithms converging to an optimal policy from a randomly initialized one
when the action space is large.

Surprisingly, frame stacking (4 consecutive frames) hurts convergence and
giving the RL agents only one move to make the correct action hurts how
quickly the agents learn the optimal policy, which is surprising given how
ClickPixel can be reduced to learning the identity function with a neural
network.

###  4.2 Whack-a-Mole

While A2C converges in Whack-a-Mole when $q=2$, it fails to converge for $q=5$
and above. Moreover, PPO fails to converge in both cases, with a mean
classification return of $48.45\pm 0.87$, over 99 steps, which is close to a
50% accuracy from guessing randomly in binary classification.

##  5 Limitations

In this paper, I formulate Codenames as a single-agent environment, but a true
game of Codenames is a zero-sum game with two players. Thus, an agent that
learns an optimal policy in my environment might give a hint that makes it
easy for the opponent to suggest a hint that takes a lot of their words off
the board, whereas in a true game of Codenames, the spymasters would not be
incentivized to do so.

##  6 Future Work

As for future research, I would recommend trying to collect a large dataset of
human Codenames games and learning a scoring function using this database of
Codenames games. Then, by combining self-play while regularizing the learned
policy to be similar to a learned human policy using KL-Divergence, a strong
human-like policy can be obtained (Jacob et al., 2022; Bakhtin et al., 2022a,
b).

Another approach would be to fine-tune a language model such as FLAN-T5 (Chung
et al., 2022) on predicting properties about synthetic Codenames games that
are encoded using a text template. Reinforcement learning from human feedback
(RHLF) (Christiano et al., 2017) can then be used to further fine-tune the LLM
to play as an effective Codenames spymaster.

##  7 Conclusion

So far, the best technique is A2C, as evaluated on the ClickPixel environment.
None of the techniques evaluated are good enough to declare the problem
solved, as none of SAC, PPO, and A2C were able to converge to a policy that
beat a random policy or a naive greedy policy evaluated on the environment.

Acknowledgments

I would like to thank Pascal Poupart for providing feedback for my research
project.

##  A Speculative Discussion

Because what each action does is completely dependent on what words are on the
board, I hypothesize that this causes the agents to have a harder time finding
an optimal policy.

##  B Goal Encoding Method

To ensure that the Codenames environment supports hindsight experience replay
(HER) (Andrychowicz et al., 2017), details about the intended goal are encoded
into the environment. The goal is encoded as a
$[0,9]^{\lvert\mathcal{B}\rvert\times 1}$ matrix where the first entry encodes
the number of remaining words, the second entry encodes whether the other team
has won yet and the third entry encodes whether the assassin word has been
chosen. The rest of the entries in the vector are zeros. The reason why the
vector must be encoded as a $\lvert\mathcal{B}\rvert\times 1$ vector when only
three dimensions are used is because it has to be possible to vstack the
observation and the goal together.

##  C Experiment Plots

Some of the experiment plots for the Codenames, ClickPixel, and Whack-a-Mole
environments are included below.

Because the plots for each Codenames experiment are almost completely
identical, only the plot for a PWCSM experiment is included below.

###  C.1 Codenames

None of the four agents converge when trained on a Codenames environment for a
$5\times 5$ board, as shown in Figure 3.

Figure 3: The episode rewards from training all four agents with 100k steps.

###  C.2 ClickPixel

Unfortunately, the models had a difficult time learning an optimal policy for
ClickPixel, even when frame stacking and reward shaping were added.

Figure 4: Training PPO on the ClickPixel environment for 10M steps doesn’t
cause it to converge to a good policy. In fact, it learns the worst policy.
Neither PPO nor A2C learn a better policy when using a $5\times 5$ board.
Figure 5: However, both agents can learn an optimal policy for a smaller
$2\times 2$ board. Figure 6: Frame stacking actually hurts the performance of
the algorithms, surprisingly. Figure 7: Giving the agent one move in an
episode to make the right decision causes A2C to take longer to converge and
doesn’t help PPO at all.

Figure 8: As expected, increasing the board size to $36\times 36$ doesn’t help
the agents at all. Less expected is the fact that switching to a CNN policy
head doesn’t help the agents learn a good policy.

Figure 9: Although making the negative per-step reward proportional to the
number of steps taken helps the PPO agent learn a better policy on a $5\times
5$ board, it doesn’t scale up to get the agents to converge to a good policy
on a $36\times 36$ board.

###  C.3 Whack-a-Mole

A2C was able to learn a good policy on a $2\times 2$ board as shown in Figure
10, but not on the larger $5\times 5$ board, as shown in Figure 11.

Figure 10: A2C is able to learn a better policy when using a $2\times 2$
board. Figure 11: Neither PPO nor A2C learn a better policy when using a
$5\times 5$ board.

## References

  * Andrychowicz et al. (2017) Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba.  Hindsight experience replay.  _Advances in neural information processing systems_ , 30, 2017.
  * Bakhtin et al. (2021) Anton Bakhtin, David Wu, Adam Lerer, and Noam Brown.  No-press diplomacy from scratch, 2021.  URL https://arxiv.org/abs/2110.02924.
  * Bakhtin et al. (2022a) Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, and Markus Zijlstra.  Human-level play in the game of ¡i¿diplomacy¡/i¿ by combining language models with strategic reasoning.  _Science_ , 378(6624):1067–1074, 2022a.  doi: 10.1126/science.ade9097.  URL https://www.science.org/doi/abs/10.1126/science.ade9097.
  * Bakhtin et al. (2022b) Anton Bakhtin, David J Wu, Adam Lerer, Jonathan Gray, Athul Paul Jacob, Gabriele Farina, Alexander H Miller, and Noam Brown.  Mastering the game of no-press diplomacy via human-regularized reinforcement learning and planning, 2022b.  URL https://arxiv.org/abs/2210.05492.
  * Bojanowski et al. (2016) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov.  Enriching word vectors with subword information, 2016.  URL https://arxiv.org/abs/1607.04606.
  * Brockman et al. (2016) Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba.  Openai gym.  _arXiv preprint arXiv:1606.01540_ , 2016.
  * Christiano et al. (2017) Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei.  Deep reinforcement learning from human preferences.  _Advances in neural information processing systems_ , 30, 2017.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al.  Scaling instruction-finetuned language models.  _arXiv preprint arXiv:2210.11416_ , 2022.
  * Chvátil (2015) Vlaada Chvátil.  Codenames rules - czech games edition, 2015.  URL https://czechgames.com/files/rules/codenames-rules-en.pdf.
  * Côté et al. (2018) Marc-Alexandre Côté, Akos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, et al.  Textworld: A learning environment for text-based games.  In _Workshop on Computer Games_ , pages 41–75. Springer, 2018.
  * Cserháti et al. (2022) Réka Cserháti, Istvan Kollath, András Kicsi, and Gábor Berend.  Codenames as a game of co-occurrence counting.  In _Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics_ , pages 43–53, Dublin, Ireland, May 2022. Association for Computational Linguistics.  doi: 10.18653/v1/2022.cmcl-1.5.  URL https://aclanthology.org/2022.cmcl-1.5.
  * De Deyne et al. (2018) Simon De Deyne, Danielle Navarro, Amy Perfors, Marc Brysbaert, and Gert Storms.  The “small world of words” english word association norms for over 12,000 cue words.  _Behavior Research Methods_ , 51, 10 2018.  doi: 10.3758/s13428-018-1115-7.
  * Guo et al. (2020) Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar.  Accelerating large-scale inference with anisotropic vector quantization.  In Hal Daumé III and Aarti Singh, editors, _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 3887–3896. PMLR, 13–18 Jul 2020.  URL https://proceedings.mlr.press/v119/guo20h.html.
  * Haarnoja et al. (2018) Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine.  Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018.  URL https://arxiv.org/abs/1801.01290.
  * Jacob et al. (2022) Athul Paul Jacob, David J Wu, Gabriele Farina, Adam Lerer, Hengyuan Hu, Anton Bakhtin, Jacob Andreas, and Noam Brown.  Modeling strong and human-like gameplay with KL-regularized search.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pages 9695–9728. PMLR, 17–23 Jul 2022.  URL https://proceedings.mlr.press/v162/jacob22a.html.
  * Jaderberg et al. (2019) Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castañeda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel.  Human-level performance in 3d multiplayer games with population-based reinforcement learning.  _Science_ , 364(6443):859–865, 2019.  doi: 10.1126/science.aau6249.  URL https://www.science.org/doi/abs/10.1126/science.aau6249.
  * Jaramillo et al. (2020) Catalina Jaramillo, Megan Charity, Rodrigo Canaan, and Julian Togelius.  Word autobots: Using transformers for word association in the game codenames.  _Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment_ , 16(1):231–237, Oct. 2020.  doi: 10.1609/aiide.v16i1.7435.  URL https://ojs.aaai.org/index.php/AIIDE/article/view/7435.
  * Kim et al. (2019) Andrew Kim, Maxim Ruzmaykin, Aaron Truong, and Adam Summerville.  Cooperation and codenames: Understanding natural language processing via codenames.  _Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment_ , 15(1):160–166, Oct. 2019.  doi: 10.1609/aiide.v15i1.5239.  URL https://ojs.aaai.org/index.php/AIIDE/article/view/5239.
  * Koyyalagunta et al. (2021) Divya Koyyalagunta, Anna Sun, Rachel Lea Draelos, and Cynthia Rudin.  Playing codenames with language graphs and word embeddings.  _J. Artif. Intell. Res._ , 71:319–346, 2021.
  * Kumar et al. (2021) Abhilasha A. Kumar, Mark Steyvers, and David A. Balota.  Semantic memory search and retrieval in a novel cooperative word game: A comparison of associative and distributional semantic models.  _Cognitive Science_ , 45(10):e13053, 2021.  doi: https://doi.org/10.1111/cogs.13053.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/cogs.13053.
  * Lerer et al. (2020) Adam Lerer, Hengyuan Hu, Jakob Foerster, and Noam Brown.  Improving policies via search in cooperative partially observable games.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 7187–7194, 2020.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ , 2013.
  * Mills (2022) Tracey Mills.  Probing nlp conceptual relatedness judgments through the word-based board game codenames.  2022\.
  * Mnih et al. (2013) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning.  _arXiv preprint arXiv:1312.5602_ , 2013.
  * Mnih et al. (2016) Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu.  Asynchronous methods for deep reinforcement learning.  2016\.  doi: 10.48550/ARXIV.1602.01783.  URL https://arxiv.org/abs/1602.01783.
  * Nelson et al. (2004) Douglas L Nelson, Cathy L McEvoy, and Thomas A Schreiber.  The university of south florida free association, rhyme, and word fragment norms.  _Behavior Research Methods, Instruments, & Computers_, 36(3):402–407, 2004.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  Training language models to follow instructions with human feedback.  _arXiv preprint arXiv:2203.02155_ , 2022.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar, October 2014. Association for Computational Linguistics.  doi: 10.3115/v1/D14-1162.  URL https://aclanthology.org/D14-1162.
  * Raffin et al. (2021) Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann.  Stable-baselines3: Reliable reinforcement learning implementations.  _Journal of Machine Learning Research_ , 22(268):1–8, 2021.  URL http://jmlr.org/papers/v22/20-1364.html.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, 11 2019\.  URL https://arxiv.org/abs/1908.10084.
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms, 2017.  URL https://arxiv.org/abs/1707.06347.
  * Shen et al. (2018) Judy Hanwen Shen, Matthias Hofer, Bjarke Felbo, and Roger Levy.  Comparing models of associative meaning: An empirical investigation of reference in simple language games.  In _Proceedings of the 22nd Conference on Computational Natural Language Learning_ , pages 292–301, Brussels, Belgium, October 2018. Association for Computational Linguistics.  doi: 10.18653/v1/K18-1029.  URL https://aclanthology.org/K18-1029.
  * Silver et al. (2016) David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis.  Mastering the game of go with deep neural networks and tree search.  _Nature_ , 529:484–503, 2016.  URL http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html.
  * Silver et al. (2018) David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al.  A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.  _Science_ , 362(6419):1140–1144, 2018.
  * Siu et al. (2021) Ho Chit Siu, Jaime D. Pena, Edenna Chen, Yutai Zhou, Victor J. Lopez, Kyle Palko, Kimberlee C. Chang, and Ross E. Allen.  Evaluation of human-ai teams for learned and rule-based agents in hanabi, 2021.  URL https://arxiv.org/abs/2107.07630.
  * Smrz (2019) Pavel Smrz.  Crowdsourcing complex associations among words by means of a game.  In _CS & IT Conference Proceedings_, volume 9. CS & IT Conference Proceedings, 2019.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi.  ConceptNet 5.5: An open multilingual graph of general knowledge.  pages 4444–4451, 2017.  URL http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972.
  * Vinyals et al. (2019) Oriol Vinyals, Igor Babuschkin, Wojciech Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John Agapiou, Max Jaderberg, and David Silver.  Grandmaster level in starcraft ii using multi-agent reinforcement learning.  _Nature_ , 575, 11 2019.  doi: 10.1038/s41586-019-1724-z.


11institutetext: Yahui Fu, Tatsuya Kawahara 22institutetext: Graduate School
of Informatics, Kyoto University, Japan
22email<EMAIL_ADDRESS>33institutetext: Haiyue
Song44institutetext: Graduate School of Informatics, Kyoto University, Japan
and NICT, Japan
44email<EMAIL_ADDRESS>55institutetext: Tianyu
Zhao66institutetext: rinna Co., Ltd., Japan
66email<EMAIL_ADDRESS>
# Enhancing Personality Recognition in Dialogue by Data Augmentation and
Heterogeneous Conversational Graph Networks

Yahui Fu  Haiyue Song  Tianyu Zhao  Tatsuya Kawahara

###### Abstract

Personality recognition is useful for enhancing robots’ ability to tailor
user-adaptive responses, thus fostering rich human-robot interactions. One of
the challenges in this task is a limited number of speakers in existing
dialogue corpora, which hampers the development of robust, speaker-independent
personality recognition models. Additionally, accurately modeling both the
interdependencies among interlocutors and the intra-dependencies within the
speaker in dialogues remains a significant issue. To address the first
challenge, we introduce personality trait interpolation for speaker data
augmentation. For the second, we propose heterogeneous conversational graph
networks to independently capture both contextual influences and inherent
personality traits. Evaluations on the RealPersonaChat corpus demonstrate our
method’s significant improvements over existing baselines.111Our codes are
available at https://github.com/fuyahuii/Personality-Recognition-on-
RealPersonaChat.

Figure 1: Homogeneous and different heterogeneous models.
$u_{a1},u_{a2},u_{b1}$ represents alternant utterance of speaker $a$ and $b$.
$\sigma(\cdot)$ represents activation function.

##  1 Introduction

Personality recognition aims to identify an individual’s characteristic
patterns of feeling, thinking, and behaving, which make each different from
one another Mairesse et al. (2007). Such capability is essential in the realm
of human-robot interaction, where correctly detecting the user’s personality
can significantly enhance the robot’s ability to tailor user-adaptive
responses, thus fostering richer and more effective human-robot dialogues.
Big-five traits McCrae and John (1992), which encompass the dimensions of
Openness, Neuroticism, Extraversion, Agreeableness, and Conscientiousness, and
their respective counter-traits are commonly used in the community for
personality assessment. In contrast to prior approaches that infer personality
traits from self-reported essays Pennebaker and King (1999); Ramezani et al.
(2022); Wen et al. (2023), first impressions Ponce-López et al. (2016); Amin
et al. (2023), or social media activities Gjurković et al. (2021), this study
focuses on the extraction of personality traits from dialogue Rissola et al.
(2019); Jiang et al. (2020); Chen et al. (2022); Mehl et al. (2006); Yamashita
et al. (2023).

However, the lack of data is a major obstacle because annotating dialogue-
level data with personality information is expensive and time-consuming. Each
dialogue involves two participants and personality traits are obtained through
psychology questionnaires. Thus, we investigate a data augmentation approach.
While previous data augmentation studies focus on generating sentence-level
data invariants Dhole et al. (2021); Wei and Zou (2019); Gao et al. (2020);
Sennrich et al. (2016) without corresponding labels, in this study, we
generate both the synthetic dialogue data and corresponding synthetic
personality traits through the proposed data interpolation method, which fuses
two existing data points controlled by a continuous ratio variable.

Additionally, accurately modeling both the inter-dependencies between context
and interlocutors, as well as the intra-dependencies within speakers in
dialogues, remains a significant challenge. Previous homogeneous models, such
as the graph attention network Veličković et al. (2017) Brody et al. (2021),
did not consider the variations in link types. Heterogeneous models like
relational graph networks (RGCN) employ distinct relation types to model
various dependencies. Yet, they utilize shared coefficients across all
relation types, which may fail to capture the unique attributes of each
relation type, as shown in Fig 1. To address this issue, we propose a method
to independently model heterogeneous conversational interactions, capturing
both contextual influences and inherent personality traits. Our main
contributions are as follows:

  * •

We propose a data augmentation method for personality recognition by
interpolation from any two existing data points.

  * •

We propose a heterogeneous conversational graph network (HC-GNN) to
independently model both the interdependencies among interlocutors, as well as
the intra-dependencies within the speaker in dialogues.

  * •

Experimental results using the RealPersonaChat corpus demonstrate that
increasing speaker diversity significantly improves personality recognition in
both monologue and dialogue settings. The proposed HC-GNN method outperforms
baseline models, showcasing its effectiveness.

##  2 Related Work

###  2.1 Personality Recognition in Dialogue

Mehl et al. Mehl et al. (2006) pioneered the automatic personality assessment
of all Big Five personality traits using various psycholinguistic attributes.
To this end, they analyzed a collection of daily-life conversations by 96
participants over 2 days. However, it only contains the subjects’
conversation; we also want to analyze how interlocutors impact the subject’s
personality expression in the dialogue. Jiang et al. Jiang et al. (2020) and
Chen et al. Chen et al. (2022) collected conversation corpus with rich data
based on the “Friends” and 40 Chinese television series, respectively.
Nonetheless, the Big Five personality labels were assigned by external
observers rather than derived from self-assessments by the speakers. Most
recently, Yamashita et al. Yamashita et al. (2023) presented the
RealPersonaChat (RPC) corpus by documenting the authentic personality traits
of the participants and allowing them to freely engage in dialogues. This
corpus aligns closely with our research objectives, as it provides a
foundation for evaluating the personality traits of subjects who may engage in
chit-chatting dialogue with a conversational agent. However, this corpus has a
relatively limited number of speakers (233). This sparsity poses a challenge
in effectively detecting the personality traits of unseen speakers. We propose
a data augmentation method to enrich the speaker diversity.

###  2.2 Data Augmentation

Data augmentation (DA) tries to fill the gap between the data distribution of
the training set and the real data with no annotation cost. Previous DA
studies focus on generating data invariants. In the computer vision field,
simple geometric transformations like cropping, rotation, and noise injection
can be easily applied to continuous image data Shorten and Khoshgoftaar
(2019). Due to the discrete nature of language data, previous DA studies in
NLP usually involve discrete noises including 1) character-level modification
like changing character case Dhole et al. (2021), 2) subword-level
regularization such as BPE-dropout Provilkov et al. (2020), 3) word-level
replacement, insertion, or deletion Wei and Zou (2019), and 4) sentence-level
modification such as paraphrasing Wang et al. (2019); Gao et al. (2020) and
back-translation Sennrich et al. (2016); Xie et al. (2018); Graça et al.
(2019); Mojapelo and Buys (2023). Our method differs from them in two aspects.
First, we generate data variants including both dialogue data and
corresponding personality labels. Second, we generate synthetic personality
traits following continuous distribution from existing discrete trait data by
introducing a random fusion variable. This bridges the gap between the
discrete distribution in the corpus and the continuous distribution in
reality. A similar work is the example extrapolation method Wei (2021) which
generates augmented embeddings of the target domain by leveraging the
similarity of embedding spaces from another assisting domain. Different from
it, our interpolation method requires only one dataset.

###  2.3 Graph Neural Networks

Graph Neural Networks (GNNs) and their variants have been widely applied in
dialogue-related tasks, like conversational emotion recognition Fu et al.
(2021, 2022), and dialog act classification Changzeng et al. (2023). This is
primarily due to the adjacency matrix in GNNs effectively simulating
interactions within conversations. Yang et al. Tao et al. (2023) proposed a
dynamic deep graph convolutional network for personality detection on social
media posts. Existing methodologies, whether employing static or dynamic
approaches to construct interactions within graphs, mainly focus on
homogeneous or heterogeneous conversation modeling. Nevertheless, various
types of nodes and links have different traits and their features may fall in
different spaces. For instance, as illustrated in Fig 1, traditional
heterogeneous models like RGCN Schlichtkrull et al. (2018) utilize shared
coefficients across all relation types, potentially failing to capture the
unique attributes of each relation type. To solve this, this paper proposes a
modification to the existing heterogeneous model framework. We introduce
separate GNNs to distinctly capture the diverse relation types, thereby
respecting the unique properties of each node and link type.

##  3 Preliminary Dataset Analysis

Table 1: Pearson correlation between pairs of Big-Five personality traits in the dataset. N, E, O, A, and C represent Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness, respectively. Big-Five Pair | (N, E) | (N, O) | (N, A) | (N, C) | (E, O) | (E, A) | (E, C) | (O, A) | (O, C) | (A, C)
---|---|---|---|---|---|---|---|---|---|---
Pearson Correlation | -0.49 | -0.23 | -0.27 | -0.15 | 0.47 | 0.46 | 0.26 | 0.40 | 0.15 | 0.36
|  |  |  |  |  |  (for all pairs $p<.05$)

We conduct experiments using the RealPersonaChat corpus Yamashita et al.
(2023), comprising 14,000 Japanese dialogues and a total of 421,203
utterances. In this corpus, 233 participants (141 females, 90 males) completed
a questionnaire regarding their Big Five personality score (in a range from
1-7) and then engaged in unstructured conversation. We normalize the score to
0-1. We analyze the correlations between pairs of personality traits in the
dataset, as shown in Table 1. The $p$-value of the Pearson correlation for
each pair is less than 0.05, indicating statistically significant
relationships between each pair of Big-Five personality traits.

##  4 Proposed Method

###  4.1 Data Interpolation

This section describes math notations, how to fuse two existing data points to
generate synthetic dialogue and Big-five traits, and variants of the proposed
method.

Notations. Each dialogue $D$ contains utterances $u_{a}$ from the speaker a or
$u_{b}$ from the speaker b in alternant turns, that is
$D=\\{u_{a1},u_{b1},u_{a2},u_{b2},...,u_{an},u_{bn}\\}$, where $n$ is the
number of turns. Each dialogue is accompanied by a label y that is a vector
containing the Big-five personality traits of the target speaker. We aim to
generate synthetic dialogue $D_{syn}$ and its label $\textbf{y}_{syn}$ from
two existing dialogues $D_{1}$ and $D_{2}$ and their labels $\textbf{y}_{1}$
and $\textbf{y}_{2}$ .222Since we focus on the personality of the initiating
speaker a in the experiments, $\textbf{y}_{1}$ or $\textbf{y}_{2}$ refers to
the personality of speaker a.

Dialogue Interpolation. First, we randomly select two dialogues $D_{1}$ and
$D_{2}$ in the training set. Second, we split each dialogue into chunks ($c$)
each containing $t$ turns, which is a hyper-parameter controlling the context
length we desire (we set $t$=3). This results in
$D_{1}=\\{c_{1},c_{2},...,c_{l}\\}$ and
$D_{2}=\\{c^{\prime}_{1},c^{\prime}_{2},...,c^{\prime}_{l}\\}$, where
$l=\frac{n}{t}$ is the number of chunk one dialogue contains. Finally, we
combine chunks from $D_{1}$ and $D_{2}$ to generate $D_{syn}$ using a fusion
ratio $\beta$ that is a random variable independently sampled from
$\text{Uniform}(0,1)$ for each synthetic data point. $D_{syn}$ can be
represented as:

$\displaystyle D_{\text{syn}}$ $\displaystyle=\beta
D_{1}\oplus(1-\beta)D_{2},\text{resulting in}$ (1) $\displaystyle
D_{\text{syn}}$ $\displaystyle=\\{c^{\text{syn}}_{i}\mid 1\leq i\leq
l\\},\text{where}$ $\displaystyle c^{\text{syn}}_{i}$
$\displaystyle=\begin{cases}c_{i}&\text{with probability }\beta,\\\
c^{\prime}_{i}&\text{with probability }1-\beta.\end{cases}$

Specially, when generating synthetic monologue data, we split each monologue
$D$ into utterances instead of chunks.

Label Interpolation. Because each label is a vector of real numbers
represented as $\textbf{y}\in\mathbb{R}^{5}$, we can simply obtain the
synthetic label through:

$\displaystyle\textbf{y}_{syn}=\beta\textbf{y}_{1}+(1-\beta)\textbf{y}_{2}.$
(2)

Method Variants. There are three types of variants of the proposed method
(former setting used). First, we can sample $\beta\sim\text{Uniform}(0,1)$ or
fixing $\beta$ to $0.5$. The former setting produces a richer variety of data.
Second, we can either select two dialogues possibly from different speakers,
where $\textbf{y}_{1}$ and $\textbf{y}_{2}$ are independently and identically
distributed ($\textbf{y}_{1}\overset{\text{iid}}{\sim}\textbf{y}_{2}$), or
select two dialogues from the same speaker $\textbf{y}_{1}=\textbf{y}_{2}$
which results in $\textbf{y}_{1}=\textbf{y}_{2}=\textbf{y}_{syn}$. The former
setting can produce new speakers with synthetic personality traits. Third, the
synthetic dialogue can be truncated to length
$l^{\prime}\sim\text{Uniform}(t_{min},|D_{syn}|)$ (we set $t_{min}=2$). This
enables personality recognition in early turns which is preferred in real
application.

###  4.2 Heterogeneous Conversational Graph Neural Network

Figure 2: Heterogeneous conversational graph neural network (HC-GNN), which
captures the interdependencies among interlocutors (acquired) and the intra-
dependencies within speaker a or b (innate).

We map each utterance in the dialogue into embeddings, as presented in Section
4.2. Subsequently, we describe heterogeneous conversation modeling in Section
4.2, followed by an explanation of heterogeneous conversational graph network
feature encoding and fusion in Section 4.2.

#### Dialogue Encoding

For each dialogue $D=\\{u_{a1},u_{b1},u_{a2},u_{b2},u_{a3}...\\}$, we employ a
BERT-like model Japanese Language Understanding with Knowledge-based
Embeddings (LUKE) Ikuya et al. (2020) 333https://huggingface.co/studio-
ousia/luke-japanese-base to encode each utterance in the dialogue:

$\textbf{h}_{u_{i}}=\texttt{LUKE}(u_{i})\in\mathbb{R}^{1\times d}$ (3)

where $h_{u_{i}}$ denotes the final hidden state of the “[CLS]” token to
represent the meaning of the whole utterance, and $d$ is the dimension of the
output.

#### Heterogeneous Conversation Modeling

To explicitly model the interaction between speakers, we independently model
the intra-dependency (innate personality) and inter-dependency (acquired
personality which is influenced by the interlocutor), as shown in Fig m 2. We
introduce the following notation: we denote directed and labeled multi-graphs
as $\mathcal{G}_{r}=(\mathcal{V}_{r},\mathcal{E}_{r},r)$ with nodes
$\vartheta_{r,i}\in\mathcal{V}_{r}$ and labeled edges (relations)
$(\vartheta_{r,i},r,\vartheta_{r,j})\in\mathcal{E}_{r}$, where
$r\in\mathcal{R}$ represents one of the conversation relation types {$spk$ $a$
$\rightarrow$ $spk$ $a$, $spk$ $b$ $\rightarrow$ $spk$ $b$, $spk$ $b$
$\rightarrow$ $spk$ $a$, $spk$ $a$ $\rightarrow$ $spk$ $b$} .

#### HC-GNNs Feature Encoding and Fusion

For each relation type in each graph $\mathcal{G}$, we then encode the
features of node $i$ with dynamic attention and graph attention networks
(GATv2) Brody et al. (2021) to aggregate the interactions between each group
of speakers (self and interlocutor):

$\displaystyle\textbf{h}_{i}^{n(l)}=\sigma(\sum_{k\in K}\sum_{j\in
N_{i}^{r}}\frac{a_{i,j}^{(k)}}{N_{i}^{r}}\textbf{W}_{r}^{(l)}\textbf{h}_{u_{j}}+a_{i,i}^{(k)}\textbf{W}_{0}^{(l)}\textbf{h}_{u_{i}})$
(4)

where $N_{i}^{r}$ denotes the neighboring indices of node $i$ under relation
$r\in\mathcal{R}$. This notation encompasses both forward and backward
relation directions. $K$ represents the number of attention head.
$\textbf{W}_{r}^{(l)}$, $\textbf{W}_{0}^{(l)}$ are the learnable weight
metrics, $l$ is the number of layer of HC-GNN, and $\sigma(.)$ is an ReLU
activation function. The attention scores are normalized across all neighbors
$j\in N_{i}^{r}$ using softmax, and the attention function is defined as:

$\displaystyle
a_{i,j}=softmax_{j}(e(\textbf{h}_{u_{i}},\textbf{h}_{u_{j}}))=\frac{exp(e(\textbf{h}_{u_{i}},\textbf{h}_{u_{j}}))}{\sum_{j^{{}^{\prime}}\in
N_{i}^{r}}exp(e(\textbf{h}_{u_{i}},\textbf{h}_{u_{j^{\prime}}}))}$ (5)
$\displaystyle
e(\textbf{h}_{u_{i}},\textbf{h}_{u_{j}})=\mathbf{a}^{\top}LeakyReLU(\textbf{W}\cdot[\textbf{h}_{u_{i}}||\textbf{h}_{u_{j}}])$

where $\mathbf{a}\in{\mathbb{R}^{d}}$ and
$\textbf{W}\in{\mathbb{R}^{d^{{}^{\prime}}\times d}}$ are learned, $||$
denotes vector concatenation. We use graph neural network (GCN) Veličković et
al. (2017) to capture the deeper interaction representations:

$\displaystyle\textbf{h}_{i}^{n(l+1)}=\sigma(\sum_{j\in
N_{i}^{r}}\textbf{W}^{(l+1)}\textbf{h}_{j}^{n(l)}+\textbf{W}_{0}^{(l+1)}\textbf{h}_{i}^{n(l)})$
(6)

where $\textbf{W}^{(l+1)}$,$\textbf{W}_{0}^{(l+1)}$ are learnable metrics.
Given the latent representation $\textbf{g}_{r}$ Veličković et al. (2017) of
each graph which corresponds to a distinct relation, we then use the self-
attention mechanism to fuse the graph outputs of innate and acquired
relations:

$\displaystyle\textbf{z}=\textbf{g}_{0}||\textbf{g}_{1}||...||\textbf{g}_{r}$
(7) $\displaystyle\textbf{z}^{{}^{\prime}}=Attn(\textbf{z,z,z})$

###  4.3 Personality Recognition with Multi-task Learning

Since each pair of Big-Five is statistically significantly related as shown in
Table 1, we recognize Openness, Neuroticism, Extraversion, Agreeableness, and
Conscientiousness in the manner of multi-task learning using five linear
layers. The $i_{th}$ layer can be represented as:

$\mathcal{P}_{i}=\sigma(\textbf{W}_{p}^{i}\textbf{z}^{{}^{\prime}}+b_{p}^{i})$
(8)

where $\sigma$ denotes the activation function ReLU, $\textbf{W}_{p}^{i}$ and
$b_{p}^{i}$ are the learnable weight matrix and bias. We treat them as
regression tasks and use the mean absolute error (MAE) as the loss function
for model optimization. The loss item of each data sample $j$ is denoted as
$l(\mathcal{\textbf{P}}^{j},\textbf{y}^{j})$ that is calculated by averaging
the loss of five tasks, where $\mathcal{\textbf{P}}^{j}$ is the vector of $5$
predictions and $\textbf{y}^{j}$ is the vector of $5$ ground truth personality
traits. The loss item of one batch containing $N$ data samples is denoted as
$L$. They are calculated as follows:

$\displaystyle
l(\mathcal{\textbf{P}}^{j},\textbf{y}^{j})=\frac{1}{5}\sum_{i=1}^{5}\left|\mathcal{P}^{j}_{i}-y^{j}_{i}\right|$
(9) $\displaystyle
L=\frac{1}{N}\sum_{j=1}^{N}l(\mathcal{\textbf{P}}^{j},\textbf{y}^{j}).$

##  5 Experimental Settings

###  5.1 Dataset Partitioning

We conduct two experimental settings, one based on monologues and the other on
dialogues. The monologue setting focuses on a speaker’s own utterances, while
the dialogue setting integrates utterances from both the speaker and the
interlocutor for personality recognition. In the monologue experiments, we
implemented strict speaker splitting to ensure no overlap among speakers
across the training, validation, and test sets. This approach meant the model
was evaluated on unseen speakers.

In the dialogue experiments, ensuring non-overlap of both speakers in all
datasets proved challenging. Therefore, we ensured that only the initiating
speaker was non-repeating across datasets, and the model was tasked with
predicting only the initiating speaker’s personality. Therefore, we only use
relation spk a$\rightarrow$spk a and spk b$\rightarrow$spk a in the HC-GNN
model. We created training, validation, and test sets by randomly dividing the
speakers in an 8:1:1 ratio for $100$ times and selecting the split that most
closely matched 8:1:1 distribution for monologues and dialogues. We then fixed
the split across all experiments.

###  5.2 Models

All the models are based on a BERT-like model which converts utterances into
embeddings. In all experiments, we used pooled output from LUKE base model
Ikuya et al. (2020) because it showed the best performance in terms of average
balanced accuracy among LUKE-base/large, RoBERTa-base/large, xlm-roberta
base/large, mdeberta-v3-base models. We experimented with different models
after the base model:

MLP: Five linear module joints with regression heads are used to predict each
personality. Each regression head contains $2$ linear layers where the first
layer maps embedding from LUKE to embedding with a size $16$ and the second
layer maps embedding with a size $16$ to $1$.

Homogeneous Methods:

GCNs Kipf and Welling (2016): represents one of the most prevalent methods for
handling graph-structured data, particularly in node classification and link
prediction tasks.

GATv2 Brody et al. (2021): introduces a significant enhancement by
transitioning from a static to a dynamic attention mechanism.

In our experiments, we utilize either two GraphConv layers or a single
GATv2Conv layer to process the union of relation types within the set
$\mathcal{R}$.

Heterogenous Methods:

RGCN Schlichtkrull et al. (2018): relational graph convolutional network,
which is developed to handle multi-relational data with a heterogeneous
architecture, as shown in Fig. 1. Specifically, we use one RGCNConv layer
connected by one GraphConv layer to handle the union of relation types in
$\mathcal{R}$, setting the number of relations to 2.

HCGNN: our proposed method, the heterogeneous conversation graph neural
network, independently models the interdependencies and intra-dependencies
within speakers. Specifically, a GATv2Conv layer, interconnected with two
GraphConv layers, is employed to extract distinct relational information
corresponding to each relation type in the set $\mathcal{R}$. GraphConv,
GATv2Conv and RGCNConv layers are imported from torch_geometric.nn 444
https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html. The number
of the attention head in GATv2Conv is set to 2.

###  5.3 Training

We used the Adam optimizer with $\beta_{1}=0.9,\beta_{2}=0.999$ and a learning
rate of $1\times 10^{-5}$. We used a linear scheduler with warmup step $=150$.
We used Mean Absolute Error (MAE) criterion because it outperformed the Mean
Squared Error (MSE) greatly in terms of balanced accuracy. We set the batch
size to $128$ for linear models and $32$ for graph neural network models,
which reach the memory limitation of eight 32G GPUs. We calculated the loss on
the validation set after each epoch and applied early stopping when no
improvement was observed for $3$ epochs. We conducted single-round experiments
for each configuration. We employed four distinct metrics to assess the
model’s performance across five different personality traits, which will offer
insight into the model’s overall capabilities.

###  5.4 Evaluation Metrics

We report binary classification accuracy and balanced accuracy together with
Pearson correlation and Spearman correlation for regression tasks. The
threshold for the binary classification task of each personality trait is set
to the median score in the training set. Here are the details of each metric:

Accuracy: a metric that summarizes the performance of a classification task,
which is the number of correctly predicted data points out of all the data
points.

Balanced Accuracy: arithmetic mean of sensitivity and specificity to deal with
imbalanced data.

Pearson Correlation: a correlation coefficient that measures the linear
correlation between the predicted personality values and the ground truth.

Spearman Correlation: a nonparametric measure of rank correlation (statistical
dependence between the rankings of two variables).

##  6 Results and Analysis

###  6.1 Monologue

Table 2: Accuracy results in monologue setting with original data and augmented dataset. The best result in each column is in bold. Data | Accuracy | Balanced Accuracy
---|---|---
N | E | O | A | C | Avg. | N | E | O | A | C | Avg.
Original | 66.2 | 52.6 | 57.6 | 52.7 | 57.9 | 57.4 | 59.7 | 54.3 | 58.8 | 50.3 | 55.0 | 55.6
+10k | 74.3 | 55.5 | 56.6 | 49.9 | 51.2 | 57.5 | 60.4 | 52.5 | 55.4 | 47.8 | 52.8 | 53.8
+20k | 74.5 | 54.3 | 55.5 | 59.0 | 60.1 | 60.7 | 65.9 | 53.8 | 56.0 | 58.5 | 48.3 | 56.5
+50k | 60.5 | 53.0 | 60.9 | 58.7 | 59.8 | 58.6 | 60.2 | 55.0 | 61.2 | 62.0 | 52.4 | 58.2
+500k | 62.7 | 58.2 | 62.0 | 57.9 | 65.4 | 61.2 | 64.6 | 56.0 | 61.3 | 60.3 | 59.7 | 60.4
Table 3: Correlation results in monologue setting with various data sizes. Data | Pearson Correlation | Spearman Correlation
---|---|---
N | E | O | A | C | N | E | O | A | C
Original | .279 | -.040 | .473 | .105 | .166 | .292 | -.014 | .282 | .104 | .200
+10k | .389 | -.092 | .510 | .100 | .200 | .283 | -.064 | .277 | .091 | .180
+20k | .492 | .054 | .510 | .268 | .113 | .495 | .063 | .276 | .261 | .126
+50k | .224 | .067 | .486 | .387 | .099 | .264 | .085 | .378 | .375 | .119
+500k | .267 | .025 | .459 | .232 | .164 | .333 | .045 | .388 | .230 | .203
|  |  |  |  |  |  (Italic means $p<.05$)

Main Results. Table 2 presents a comparative analysis of the accuracy and
balanced accuracy of the data augmentation method at various data sizes. With
the addition of augmented data, both accuracy and balanced accuracy show
significant improvements, increasing from 57.4% to 61.2% and from 55.6% to
60.4%, respectively. Table 3 shows the Pearson and Spearman correlation
results. We observe that data augmentation generally improves correlation in
most traits, and the impact of augmentation is more pronounced in N and A than
in others. We fail to predict personality trait E in any setting. We believe
this may be due to our dataset being based on first-meeting spontaneous
situations, where people tend not to exhibit extrovert traits explicitly. We
also find that the optimal results for different personality traits, in terms
of accuracy, balanced accuracy, Pearson correlation, and Spearman correlation,
do not consistently align with the same data augmentation size. We attribute
this variability primarily to the inherent randomness associated with data
synthesis.

Results of Data Augmentation Variants.

1\. Fusing Ratio. We compare results using $\beta\sim\text{Uniform}(0,1)$ and
fixed $\beta=0.5$. With 500k additional data, using random $\beta$ achieved
$61.2\%$ averaged accuracy and $60.4\%$ averaged balanced accuracy whereas
using fixed $\beta$ showed $59.3\%$ accuracy and $58.4\%$ balanced accuracy.
This is because using random $\beta$ results in more various data. Since the
original label score distribution is discrete (there are only a limited number
of score values), using $\beta$ following a continuous distribution yields a
higher variety of labels and corresponding textual data compared to using
fixed $\beta$.

2\. Speaker Choice. We compare generating synthetic dialogue from the same
speaker or two different speakers. We observe using dialogues from different
speakers not only enables continuous data distribution as shown in Fig 3 but
also showed much higher averaged accuracy ($61.2\%$ vs $57.3\%$) and balanced
accuracy ($60.4\%$ vs $58.1\%$), which demonstrates that speaker variety is
more crucial than the number of conversations for personality recognition
within this dataset.

3\. Various Context Lengths. Real-time personality recognition in dialogue is
essential for human-robot interaction. To enable this, we propose to use
various context lengths of dialogues for training. Table 4 presents
comparative results on a range of test context turns when training with
various context lengths and full context lengths, using an augmented data set
of 500k. When testing the first $2$ turns using various context lengths during
training results in $58.2\%$ averaged balanced accuracy, which is comparable
to the result of $60.4\%$ using the full contexts during inference. On the
other hand, we got only $55.5\%$ when training with full contexts
(approximately $15$ turns). In addition to context turns being 2 during
inference, similar trends can also be observed in other numbers.

(a) Generate synthetic data by combining two dialogues from two different
speakers.

(b) Generate synthetic data by combining two dialogues from the same speaker.

Figure 3: Data distribution of augmented data and original data. Table 4: Comparative results of accuracy and balanced accuracy for a range of test context turns: training with various context turns ($\geq 2$) vs. full context turns, using an augmented data set of 500k. test train | Accuracy | Balanced Accuracy
---|---|---
various context | full context | various context | full context
2 | 64.1 | 57.8 | 58.2 | 55.5
3 | 64.9 | 57.6 | 57.8 | 55.8
4 | 64.8 | 58.1 | 58.4 | 56.6
5 | 64.5 | 58.0 | 58.6 | 56.3
10 | 63.0 | 57.6 | 59.6 | 56.6
full | 62.7 | 57.1 | 60.4 | 57.0
Table 5: Accuracy results in the comparisons among monologue and dialogue.

Model | Accuracy | Balanced Accuracy
---|---|---
N | E | O | A | C | Avg. | N | E | O | A | C | Avg.
Monologue | MLP | 66.2 | 52.6 | 57.6 | 52.7 | 57.9 | 57.4 | 59.7 | 54.3 | 58.8 | 50.3 | 55.0 | 55.6
Dialogue | MLP | 66.8 | 53.2 | 57.9 | 49.0 | 58.4 | 57.1 | 59.5 | 56.9 | 57.9 | 49.8 | 47.5 | 54.3
GCN Kipf and Welling (2016) | 69.5 | 40.3 | 52.9 | 33.3 | 72.1 | 53.6 | 50.0 | 50.4 | 53.7 | 34.4 | 50.1 | 47.7
GAT Brody et al. (2021) | 66.4 | 54.5 | 52.7 | 59.9 | 54.6 | 57.6 | 60.4 | 51.3 | 50.1 | 52.2 | 52.1 | 53.2
RGCN Schlichtkrull et al. (2018) | 68.5 | 57.2 | 55.4 | 41.9 | 56.7 | 55.9 | 63.0 | 54.5 | 55.9 | 47.2 | 51.9 | 54.5
HC-GNN (ours) | 69.0 | 53.5 | 55.6 | 65.9 | 52.2 | 59.2 | 60.9 | 54.3 | 52.6 | 59.6 | 54.6 | 56.4

Table 6: Correlation results in the comparisons among monologue and dialogue. Model | Pearson Correlation | Spearman Correlation
---|---|---
N | E | O | A | C | N | E | O | A | C
Monologue | MLP | .279 | -.040 | .473 | .105 | .166 | .292 | -.014 | .282 | .104 | .200
Dialogue | MLP | .288 | .048 | .492 | .174 | .077 | .214 | .088 | .262 | .123 | .072
GCN Kipf and Welling (2016) | .170 | -.015 | .298 | -.203 | .079 | .173 | .017 | .164 | -.227 | .105
GAT Brody et al. (2021) | .307 | -.067 | .420 | .079 | .134 | .284 | -.030 | .226 | .082 | .135
RGCN Schlichtkrull et al. (2018) | .377 | .048 | .490 | .152 | .148 | .375 | .078 | .311 | .146 | .160
HC-GNN (ours) | .285 | .040 | .347 | .216 | .169 | .304 | .066 | .243 | .191 | .193
|  |  |  |  |  | (Italic means $p<.05$)

###  6.2 Dialogue

Comparisons between Monologue and Dialogue. To explore the impact of context
on personality recognition, we first prepend the [SPK1] or [SPK2] token to the
respective utterances and then concatenate all utterances using the [SEP]
token. We employ the same model as in the monologue experiment. As indicated
in Table 5 and 6, the results with conventional methods using the context
(dialogue) show a decrease in performance compared to the monologue setting
across most evaluation metrics. We hypothesize that merely concatenating
utterances between two speakers is not an effective method for modeling the
interactions between interlocutors. Therefore, we propose independently
modeling both the interdependency among speakers and the intra-dependency
within the speaker. The results, as shown in Tables 5 and 6, indicate that our
proposed method surpasses all baseline methods in the dialogue setting and
marginally improves upon the results in the monologue setting.

Data Augmentation in Dialogue. We test the effectiveness of data augmentation
in the dialogue setting. The results, as shown in Table 7 and 8, indicate that
increasing speaker variety can enhance personality recognition in dialogue.
Although the highest balanced accuracy achieved in the dialogue setting is
58.6, falling short of the monologue setting’s best result of 60.4. Due to our
focus on predicting only speaker a’s personality in the dialogue setting, the
original dataset lost half of its conversational data for augmentation
purposes. This loss is an inevitable trade-off in the pursuit of speaker-
independent personality recognition within dialogue settings.

Table 7: Accuracy results of HC-GNN in dialogue setting with various data sizes. Data | Accuracy | Balanced Accuracy
---|---|---
N | E | O | A | C | Avg. | N | E | O | A | C | Avg.
Monologue | +500k | 62.7 | 58.2 | 62.0 | 57.9 | 65.4 | 61.2 | 64.6 | 56.0 | 61.3 | 60.3 | 59.7 | 60.4
Dialogue | Original | 69.0 | 53.5 | 55.6 | 65.9 | 52.2 | 59.2 | 60.9 | 54.3 | 52.6 | 59.6 | 54.6 | 56.4
(HC-GNN) | +10k | 73.9 | 59.7 | 54.0 | 63.9 | 54.6 | 61.2 | 59.5 | 52.8 | 50.7 | 55.6 | 59.0 | 55.5
+20k | 74.3 | 60.1 | 58.4 | 67.5 | 40.8 | 60.2 | 60.8 | 54.4 | 55.8 | 61.6 | 55.5 | 57.6
+50k | 69.6 | 54.7 | 58.7 | 62.2 | 63.8 | 61.8 | 61.9 | 55.5 | 58.4 | 62.3 | 55.1 | 58.6
+500k | 66.3 | 56.6 | 58.8 | 60.1 | 59.0 | 60.2 | 63.7 | 53.8 | 57.0 | 57.7 | 59.2 | 58.3
Table 8: Correlation results of HC-GNN in dialogue setting with various data sizes. Data | Pearson Correlation | Spearman Correlation
---|---|---
N | E | O | A | C | N | E | O | A | C
Monologue | +500k | .267 | .025 | .459 | .232 | .164 | .333 | .045 | .388 | .230 | .203
Dialogue | Original | .285 | .040 | .347 | .216 | .169 | .304 | .066 | .243 | .191 | .193
(HC-GNN) | +10k | .369 | .019 | .329 | .235 | .234 | .343 | .058 | .226 | .234 | .263
+20k | .327 | -.052 | .441 | .314 | .110 | .301 | .003 | .245 | .278 | .142
+50k | .345 | .030 | .312 | .300 | .162 | .374 | .050 | .250 | .281 | .183
+500k | .426 | .046 | .411 | .209 | .223 | .439 | .062 | .315 | .194 | .242
(Italic means $p<.05$)

##  7 Conclusion

We have proposed a data augmentation method for personality recognition, which
involves interpolating between two existing data points to enhance speaker
diversity. Additionally, we have introduced the HC-GNN method to independently
model the interdependencies among interlocutors, as well as the intra-
dependencies within the speaker in dialogues. Experimental results from the
RealPersonaChat corpus demonstrate that increasing speaker diversity
significantly improves personality recognition in both monologue and dialogue
settings. Our HC-GNN method outperforms baseline models, showcasing its
effectiveness. However, our experiments suggest that context did not make a
large improvement in personality recognition. Further exploration of the
dialogue setting will be the focus of our future work.

###### Acknowledgements.

This work was supported by JST Moonshot R&D Goal 1 Avatar Symbiotic Society
Project (JPMJMS2011). This work was also supported by JST, the establishment
of university fellowships towards the creation of science and technology
innovation, Grant Number JPMJFS2123, and JSPS KAKENHI Grant Number 21J23124.

## References

  * Mairesse et al. [2007] François Mairesse, Marilyn A. Walker, Matthias R. Mehl, and Roger K. Moore.  Using linguistic cues for the automatic recognition of personality in conversation and text.  _J. Artif. Int. Res._ , 30(1):457–500, nov 2007.  ISSN 1076-9757.
  * McCrae and John [1992] Robert R McCrae and Oliver P John.  An introduction to the five-factor model and its applications.  _Journal of personality_ , 60(2):175–215, 1992.
  * Pennebaker and King [1999] James W Pennebaker and Laura A King.  Linguistic styles: language use as an individual difference.  _Journal of personality and social psychology_ , 77(6):1296, 1999.
  * Ramezani et al. [2022] Majid Ramezani, Mohammad-Reza Feizi-Derakhshi, and Mohammad-Ali Balafar.  Text-based automatic personality prediction using kgrat-net: a knowledge graph attention network classifier.  _Scientific Reports_ , 12(1):21453, 2022.
  * Wen et al. [2023] Zhiyuan Wen, Jiannong Cao, Yu Yang, Haoli Wang, Ruosong Yang, and Shuaiqi Liu.  Desprompt: Personality-descriptive prompt tuning for few-shot personality recognition.  _Information Processing & Management_, 60(5):103422, 2023.
  * Ponce-López et al. [2016] Víctor Ponce-López, Baiyu Chen, Marc Oliu, Ciprian Corneanu, Albert Clapés, Isabelle Guyon, Xavier Baró, Hugo Jair Escalante, and Sergio Escalera.  Chalearn lap 2016: First round challenge on first impressions-dataset and results.  In _Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14_ , pages 400–418. Springer, 2016.
  * Amin et al. [2023] M. M. Amin, E. Cambria, and B. W. Schuller.  Can chatgpt’s responses boost traditional natural language processing?  _IEEE Intelligent Systems_ , 38(05):5–11, sep 2023.  ISSN 1941-1294.  doi: 10.1109/MIS.2023.3305861.
  * Gjurković et al. [2021] Matej Gjurković, Mladen Karan, Iva Vukojević, Mihaela Bošnjak, and Jan Snajder.  PANDORA talks: Personality and demographics on Reddit.  In _Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media_ , pages 138–152, June 2021.
  * Rissola et al. [2019] Esteban Andres Rissola, Seyed Ali Bahrainian, and Fabio Crestani.  Personality recognition in conversations using capsule neural networks.  In _IEEE/WIC/ACM International Conference on Web Intelligence_ , pages 180–187, 2019.
  * Jiang et al. [2020] Hang Jiang, Xianzhe Zhang, and Jinho D Choi.  Automatic text-based personality recognition on monologues and multiparty dialogues using attentive networks and contextual embeddings (student abstract).  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 34, pages 13821–13822, 2020.
  * Chen et al. [2022] Yirong Chen, Weiquan Fan, Xiaofen Xing, Jianxin Pang, Minlie Huang, Wenjing Han, Qianfeng Tie, and Xiangmin Xu.  CPED: A large-scale chinese personalized and emotional dialogue dataset for conversational ai.  _arXiv preprint arXiv:2205.14727_ , 2022.  URL https://arxiv.org/abs/2205.14727.
  * Mehl et al. [2006] Matthias R Mehl, Samuel D Gosling, and James W Pennebaker.  Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life.  _Journal of personality and social psychology_ , 90(5):862, 2006.
  * Yamashita et al. [2023] Sanae Yamashita, Koji Inoue, Ao Guo, Shota Mochizuki, Tatsuya Kawahara, and Ryuichiro Higashinaka.  RealPersonaChat: A realistic persona chat corpus with interlocutors’ own personalities.  In _Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation_ , 2023.
  * Dhole et al. [2021] Kaustubh D Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Shrivastava, Samson Tan, et al.  Nl-augmenter: A framework for task-sensitive natural language augmentation.  _arXiv preprint arXiv:2112.02721_ , 2021.
  * Wei and Zou [2019] Jason Wei and Kai Zou.  EDA: Easy data augmentation techniques for boosting performance on text classification tasks.  In _EMNLP-IJCNLP_ , pages 6382–6388, November 2019.
  * Gao et al. [2020] Silin Gao, Yichi Zhang, Zhijian Ou, and Zhou Yu.  Paraphrase augmented task-oriented dialog generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 639–649, July 2020.
  * Sennrich et al. [2016] Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 86–96, August 2016.
  * Veličković et al. [2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.  Graph attention networks.  _arXiv preprint arXiv:1710.10903_ , 2017.
  * Brody et al. [2021] Shaked Brody, Uri Alon, and Eran Yahav.  How attentive are graph attention networks?  _arXiv preprint arXiv:2105.14491_ , 2021.
  * Shorten and Khoshgoftaar [2019] Connor Shorten and Taghi M Khoshgoftaar.  A survey on image data augmentation for deep learning.  _Journal of big data_ , 6(1):1–48, 2019.
  * Provilkov et al. [2020] Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita.  BPE-dropout: Simple and effective subword regularization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1882–1892, July 2020.
  * Wang et al. [2019] Su Wang, Rahul Gupta, Nancy Chang, and Jason Baldridge.  A task in a suit and a tie: paraphrase generation with semantic augmentation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 7176–7183, 2019.
  * Xie et al. [2018] Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Ng, and Dan Jurafsky.  Noising and denoising natural language: Diverse backtranslation for grammar correction.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 619–628, June 2018.
  * Graça et al. [2019] Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, and Hermann Ney.  Generalizing back-translation in neural machine translation.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)_ , pages 45–52, August 2019.
  * Mojapelo and Buys [2023] Maxwell Mojapelo and Jan Buys.  Data augmentation for low resource neural machine translation for sotho-tswana languages.  In _Proceedings of Southern African Conference for AI Research (SACAIR 2023)_ , Johannesburg, South Africa, 2023.
  * Wei [2021] Jason Wei.  Good-enough example extrapolation.  In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 5923–5929, November 2021.
  * Fu et al. [2021] Yahui Fu, Shogo Okada, Longbiao Wang, Lili Guo, Yaodong Song, Jiaxing Liu, and Jianwu Dang.  CONSK-GCN: conversational semantic-and knowledge-oriented graph convolutional network for multimodal emotion recognition.  In _2021 IEEE International Conference on Multimedia and Expo (ICME)_ , pages 1–6. IEEE, 2021.
  * Fu et al. [2022] Yahui Fu, Okada Shogo, Longbiao Wang, Guo Lili, Song Yaodong, Liu Jiaxing, and Dang Jianwu.  Context-and knowledge-aware graph convolutional network for multimodal emotion recognition.  _IEEE MultiMedia_ , 29(3):91–100, 2022.
  * Changzeng et al. [2023] Fu Changzeng, Chen Zhenghan, Shi Jiaqi, Wu Bowen, Liu Chaoran, Ishi Carlos Toshinori, and Ishiguro Hiroshi.  Hag: Hierarchical attention with graph network for dialogue act classification in conversation.  In _ICASSP_ , pages 1–5. IEEE, 2023.
  * Tao et al. [2023] Yang Tao, Deng Jinghao, Quan Xiaojun, and Wang Qifan.  Orders are unwanted: dynamic deep graph convolutional network for personality detection.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, pages 13896–13904, 2023.
  * Schlichtkrull et al. [2018] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling.  Modeling relational data with graph convolutional networks.  In _The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15_ , pages 593–607. Springer, 2018.
  * Ikuya et al. [2020] Yamada Ikuya, Asai Akari, Shindo Hiroyuki, Takeda Hideaki, and Matsumoto Yuji.  Luke: deep contextualized entity representations with entity-aware self-attention.  _arXiv preprint arXiv:2010.01057_ , 2020.
  * Kipf and Welling [2016] Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ , 2016.