Arnab Das1,3∗, Suhita Ghosh1∗, Tim Polzehl3, Ingo Siegert2, Sebastian Stober1

# StarGAN-VC++: Towards Emotion Preserving Voice Conversion Using Deep
Embeddings

###### Abstract

Voice conversion (VC) transforms an utterance to sound like another person
without changing the linguistic content. A recently proposed generative
adversarial network-based VC method, StarGANv2-VC is very successful in
generating natural-sounding conversions. However, the method fails to preserve
the emotion of the source speaker in the converted samples. Emotion
preservation is necessary for natural human-computer interaction. In this
paper, we show that StarGANv2-VC fails to disentangle the speaker and emotion
representations, pertinent to preserve emotion. Specifically, there is an
emotion leakage from the reference audio used to capture the speaker
embeddings while training. To counter the problem, we propose novel emotion-
aware losses and an unsupervised method which exploits emotion supervision
through latent emotion representations. The objective and subjective
evaluations prove the efficacy of the proposed strategy over diverse datasets,
emotions, gender, etc.

**footnotetext: These authors contributed equally to this work

Index Terms: voice conversion, emotion preservation, StarGAN

##  1 Introduction

Voice conversion (VC) is a technique that transforms the speech of one speaker
to make it sound like another speaker's voice, while keeping the linguistic
content intact and ensuring that the quality, naturalness and
comprehensibility of the converted speech remain high [1]. VC systems have
numerous practical use cases in various domains [2]. For example, in speech
therapy, a VC system could be used to record sessions for later analysis while
ensuring that patient confidentiality is maintained, a mandatory request
resulting from the requirement to adhere to the guidelines of the General Data
Protection Regulation (GDPR) [3]. VC systems could also be useful in the
entertainment industry to perform tasks like voice dubbing. Furthermore,
preserving the emotional state of the speaker parallel to providing anonymized
speech is necessary in all VC applications where the emotional information is
necessary for further processing, such as when analyzing the speech for mental
health issues or interaction with an affect-aware avatar. Therefore, VC
methods should ensure that the original speaker's emotional state is not lost
during the conversion process.

Many deep learning-based approaches have been proposed for voice conversion
[4]. Early deep neural network (DNN)-based methods [5] largely concentrated on
the concept of frame-wise spectral feature conversion. Soon after, long short-
term memory (LSTM) based sequence-to-sequence models [6] were employed for the
task and produced high-quality converted samples. Although the converted
samples generated by sequence-to-sequence models are more natural, they suffer
from mispronunciation and training instability [7]. Moreover, these methods
need numerous parallel utterances to learn [8], which is a very expensive,
time-consuming, and burdensome task in itself.

To alleviate the issue of obtaining expensive parallel training data, several
non-parallel-based DNN methods were proposed, which are based on variational
autoencoder (VAE) [9, 10, 11] and cycle-consistent generative adversarial
network (CycleGAN) [12, 13, 14]. The VC methods utilizing VAE typically
attempt to extract disentangled speaker and content embeddings from utterances
through a reconstruction loss. The VAE-based approaches produce over-smoothed
conversions, which leads to a poor-quality, buzzy-sounding speech [7, 8]. The
CycleGAN-based frameworks use cycle consistency loss, which learns both
forward and reverse conversion of samples between two speakers using non-
parallel training utterances. A major problem with CycleGAN-based frameworks
is that they require training of one generator for each pair of speakers,
which makes it impractical for many-to-many VC use cases. The CycleGAN-based
frameworks are also criticized for producing low-quality converted samples
[7].

A few recent text-to-speech (TTS) based voice conversion methods [15] extract
the content through two modules, automatic speech recognition (ASR) and TTS.
The linguistic content is first extracted from the source speech through an
ASR. Further, a TTS is fed with the ASR generated transcription and a target
speaker's embedding to generate the converted utterance. The naturalness and
intelligibility of the converted speech using such ASR and TTS-based systems
are typically high [16]. However, they fail to preserve the prosody or
affective state of the source speaker, as they use only the linguistic content
of the source utterance to generate the converted speech. Further, the
performance of such VC systems is dependent on the quality of transcriptions
produced by the ASR.

Recently, several StarGAN-based [17] non-parallel many-to-many VC frameworks
[18, 19, 20] have been proposed. Among those, the StarGANv2-VC [20] framework
is especially interesting as it generates fundamental frequency (F0)
consistent, natural sounding and highly intelligible converted samples [21].
Moreover, the architecture design makes the framework very scalable, making it
suitable for utterances of any length and, its fast conversion capability
makes it suitable for real-time applications. However, the model fails to
preserve the affective state of the source speaker when the source utterance
has a large variation in the acoustic parameters.

In this paper, we investigate the reason for StarGANv2-VC's failure in
preserving the source speaker's emotion in converted speech. We also propose a
novel method to circumvent the said problem, by using an unsupervised emotion
supervision technique through latent emotion representations. Further, we
propose losses which prevent emotion leakage from the reference audio used to
generate the speaker embeddings, which also leads to a better disentanglement
of speaker and emotion representations. We evaluate the proposed method
extensively with three datasets for various emotions, different gender, and
accent groups. The objective and subjective evaluation results depict that the
proposed method significantly improves the emotion preservation capability
over vanilla StarGANv2-VC for all cases.

Figure 1: StarGANv2-VC architecture for VC. The dashed parts belong to the
proposed emotion supervision method.

##  2 StarGANv2-VC and Emotion Leakage

###  2.1 Architecture

The architecture diagram of StarGANv2-VC [20] is presented in Figure 1, along
with the proposed emotion supervision components. The generator G produces the
converted utterance $X_{trg}$, which comprises an encoder (EN) and a decoder.
The generator is fed with an utterance $X_{src}$ belonging to the source
speaker $y_{src}$ and a speaker style-embedding $h_{sty}$ belonging to the
target speaker $y_{trg}$, where $y_{src},y_{trg}\in Y$. As described by the
authors, the generator also consumes the source F0 embeddings $h_{f0}$
produced by a pre-trained network, which enables the model to produce F0
consistent conversions [20]. The speaker-style-encoder (SE) generates the
speaker-style-embedding $h_{sty}$ from a randomly selected reference utterance
$X_{ref}$ of the target speaker $y_{trg}$, given the target speaker-code. The
embedding $h_{sty}$ represents speaker-style information, such as accent.
Hence, the converted sample $X_{trg}=G(X_{src},h_{f0},h_{sty})$, contains the
speaker characteristics of the target speaker, but the intonation and
linguistic content of the source utterance. To encourage G in producing unique
samples, a separate mapping network (M) is also trained along with SE, as done
in [20]. During training, speaker-style embeddings are generated using both SE
or M alternatively in each optimization step. The module M produces target
speaker-embedding from a random latent vector sampled from a normal Gaussian
distribution and target speaker-code. The discriminator module consists of two
adversarial classifiers: a quality classifier (C) and a speaker classifier
(Csp). The C classifier classifies the real and fake samples conditioned on
the speaker-code. The classifier (Csp) classifies the source speaker of the
converted sample during the discriminator training phase and classifies the
target speaker during the generator training phase, as proposed in [20]. This
further encourages G to suppress the source speaker's traits in converted
samples.

###  2.2 Emotion Leakage by Speaker Embedding

The speaker-style-encoder (SE) extracts the target speaker's style-embedding
from the reference utterance, shown as $X_{ref}$ in Figure 1. For the emotion-
leakage introspection, we train a vanilla StarGANv2-VC with English utterances
from Emotional Speech Database (ESD) corpus [22], which has emotional
utterances. After the training, we extract the speaker embeddings. For
illustration, we choose two different speakers, 0012 (male) and 0016 (female)
from ESD. The model is trained using utterances from these speakers. The
speaker embeddings are projected onto a 2D space using tSNE transformation,
and the results are presented in Figure 2. The speaker embeddings for
utterances from a single speaker should conform to a compact region in space,
regardless of the emotion of the utterances. On the contrary, Figure 2 reveals
that the embeddings form unnecessary grouping based on emotion. This shows
that the SE fails to disentangle emotional cues from the reference utterances
when it generates target speaker embedding from those utterances.
Consequently, this unintended emotional information leaks to the decoder along
with the target speaker's style representation. This confuses the decoder
while generating the utterance using the target speaker's voice. This leakage
occurs due to two reasons: (i) the absence of a training objective that
encourages the SE to perform the disentanglement between speaker-dependent and
speaker-independent features. (b) the speaker-style reconstruction loss
$\mathcal{L}_{style}$ [20] used in the vanilla StarGANv2-VC, shown in Eqn. 1.

$\displaystyle\mathcal{L}_{style}$
$\displaystyle=\big{[}|SE(X_{ref},y_{trg})-SE(X_{trg},y_{trg})|\big{]}$ (1)

The speaker-style reconstruction loss ensures that the style embeddings can be
regenerated from the generated samples. This loss intends to bring the
speaking style of the converted sample closer to the reference sample, as both
of them belong to the same (target) speaker. This further exacerbates the
domain leakage problem, as the target speaker-embedding is not disentangled
from the emotional cues of the reference utterance. Therefore, by minimizing
this loss, the converted speech tends to be closer to the reference also in
terms of emotion, which subdues the emotion of the source speech. This
cripples the emotion preservation capability of the vanilla StarGANv2-VC.

Figure 2: 2D tSNE plot for speaker embedding generated by the style encoder of
the vanilla StarGANv2-VC.

##  3 Methods

The availability of a large amount of good quality emotion labels is
difficult. Further, in non-parallel voice conversion, the emotion label of the
converted samples is not available. Therefore, to circumvent the emotion
leakage problem, we propose an unsupervised emotion supervision technique
using emotion representations. The emotion representations are deep emotion-
embeddings, which contain the information about the affective state of the
utterance. The proposed method encourages the generator to preserve the
affective state of the source in the converted samples.

###  3.1 Deep Emotion Embedding and Emotion Supervision

One way of providing emotion supervision is to extract latent emotion
representations from the source and the converted samples, and then minimize
the distance between them. To this end, an emotion-embedding extraction
network is needed to produce the latent emotion representations directly from
the utterances. A two-stage training approach is proposed to create this
emotion-embedding extraction network.

At Stage-I, an emotion conversion framework is trained, where the emotion is
converted instead of speaker-traits. We used the vanilla StarGANv2-VC
framework for the emotion conversion task, as shown in Figure 3. In this case,
the style encoder captures the representations of emotion instead of speaker.
The working principle of the style encoder in the emotion conversion task is
depicted in Figure 4. The shallow shared convolution layers extract a 512
dimensional shared latent representation from the reference mel-spectrogram.
Consequently, the fully connected (FC) layers project this shared embedding to
emotion-specific 64 dimensional emotion-style embeddings,
$h_{emo1},...,h_{emoN}$, where $N$ is the number of emotion classes. Finally,
an emotion style-embedding $h_{emo}$ corresponding to the utterance needs to
be selected based on the emotion-code $e_{trg}$. The emotion-code denotes the
emotion class of the utterance. Therefore, when the emotion ground truth is
not available, the selection of the emotion-code is not possible. This makes
the current technique unusable for emotion extraction for VC, as the converted
samples do not have emotion ground truth. Therefore, we need a mechanism which
does not depend on the availability of emotion ground truth, which is achieved
at Stage-II.

Figure 3: StarGANv2-VC used in emotion conversion. For example, a happy
utterance from speaker 1 is converted to a sad utterance. The emotion-style
encoder generates sad emotion embedding from a sad reference utterance of
speaker 2. Figure 4: Architecture and the working mechanism of the style
encoder for emotion conversion. Figure 5: Block diagram of the emotion
classifier. $X$ is any input mel-spectrogram. The trainable emotion
classification head is added on top of pre-trained shared convolution layers.
The classification head predicts the emotion class $\hat{e}$.

At Stage-II, the linear projecting FC layers are removed from the pre-trained
SE and a classification head consisting of three FC layers is placed on top of
the shared convolution layers, as shown in Figure 5. The classifier is then
trained for a supervised emotion classification task. During training, the
weights for the shallow pre-trained convolution layers remain fixed and only
the weights for the classification head are optimized. After Stage-II
training, the 64 dimensional output features of the second FC layer of the
classification head can be used as the latent emotion representation. Now,
this network serves as a differentiable black-box emotion-embedding extractor
module $C_{emo}$ for the VC task, as shown in Figure 1. The module can be used
to extract deep emotion representations from any utterance, irrespective of
the presence of emotion ground truth. A loss is computed by equating the
emotion representations from the source and converted, as shown in Eqn. 2.

$\displaystyle\mathcal{L}_{demo}=\mathbb{E}_{X_{src},X_{trg}}\big{[}|C_{emo}(X_{src})-C_{emo}(X_{trg})|\big{]}$
(2)

We minimize this additional $\mathcal{L}_{demo}$ loss while training the
StarGANv2-VC for the VC task, which encourages emotion preservation and
suppress any emotion leakage from the reference utterance.

###  3.2 Style Reconstruction Loss

To encourage the disentanglement of the target speaker's style-embedding from
any unintended emotion-indicating information of the reference utterance, we
augment the style reconstruction loss $\mathcal{L}_{style}$ as shown in Eqn.
3.2. Here, $X_{ref2}$ is another randomly selected reference utterance for the
same target speaker. The loss facilitates the model to bring the target
speaker style embeddings extracted from $X_{ref}$ and $X_{ref2}$ closer.

$\displaystyle\mathcal{L}_{style}$
$\displaystyle=\big{[}|SE(X_{ref},y_{trg})-SE(X_{trg},y_{trg})|\big{]}$
$\displaystyle+\big{[}|SE(X_{ref2},y_{trg})-SE(X_{trg},y_{trg})|\big{]}$
$\displaystyle+\big{[}|SE(X_{ref},y_{trg})-SE(X_{ref2},y_{trg})|\big{]}$ (3)

###  3.3 Conversion Invariant Feature Preservation Loss

In vanilla StarGANv2-VC, all the losses are applied directly to the generator
output, and gradients are back-propagated all the way through the decoder and
encoder. Consequently, the shallow convolution layers of the encoder get very
little supervision about the conversion invariant features. These conversion
invariant features, such as content and emotion-related features, need to be
preserved as much as possible in the latent output of the encoder. To this
end, we propose a new loss $\mathcal{L}_{inv}$ as shown in Eqn. 4, applied
directly to the encoder EN. The loss minimizes the distance between the latent
code generated from the source and the converted sample, forcing the encoder
to preserve more content and emotion-related features in its latent output.
This loss further helps in disentanglement of content features from speaker-
specific features.

$\displaystyle\mathcal{L}_{inv}=\mathbb{E}_{X_{src},X_{trg}}\big{[}|EN(X_{src})-EN(X_{trg})|\big{]}$
(4)

###  3.4 Overall Training Objective

The overall training objective for the generator is presented in Eqn. 3.4,
where $\mathcal{L}_{demo}$ and $\mathcal{L}_{inv}$ are the proposed losses,
and the rest are taken from [20]. $\mathcal{L}_{adv}$ is the typical
adversarial loss. $\mathcal{L}_{spk}$ is adversarial speaker classifier loss.
$\mathcal{L}_{div}$ encourages the generator to produce diversified samples
when given different style-embeddings. $\mathcal{L}_{asr}$ is linguistic
content preservation loss. $\mathcal{L}_{norm}$ is the norm consistency loss
which ensures the preservation of voiced/unvoiced intervals. $\lambda_{cycle}$
is cycle consistency loss to encourage the generator to learn a bijective
mapping between source and target speakers and, $\mathcal{L}_{F0}$ is F0
consistency loss helps in generating F0-consistent samples.

$\displaystyle\min_{G,SE,M}\quad$
$\displaystyle\mathcal{L}_{adv}+\lambda_{spk}\mathcal{L}_{spk}+\lambda_{style}\mathcal{L}_{style}-\lambda_{div}\mathcal{L}_{div}$
$\displaystyle+\lambda_{asr}\mathcal{L}_{asr}+\lambda_{norm}\mathcal{L}_{norm}+\lambda_{cycle}\mathcal{L}_{cycle}$
$\displaystyle+\lambda_{F0}\mathcal{L}_{F0}+\lambda_{demo}\mathcal{L}_{demo}+\lambda_{inv}\mathcal{L}_{inv}$
(5)

The training objective for the adversarial classifiers is presented in Eqn. 6.
The adversarial quality classifier maximizes the adversarial loss
$\mathcal{L}_{adv}$. The speaker classifier minimizes $\mathcal{L}_{aspk}$
through the classification of the source speaker. Each loss is weighted with a
corresponding $\lambda$ hyperparameter.

$\displaystyle\min_{C,C_{sp}}\quad$
$\displaystyle-\mathcal{L}_{adv}+\lambda_{aspk}\mathcal{L}_{aspk}$ (6)

##  4 Datasets, Experiments and Results

###  4.1 Training Details

We term our approach as StarGAN-VC++ and the vanilla StarGANv2-VC framework as
Baseline. We train our models with a random split of 0.8/0.1/0.1 for
train/validation/test. All the utterances are re-sampled to
$24\text{\,}\mathrm{kHz}$. Each model is trained for 60 epochs on log mel-
spectrograms with a batch size of 16 using a Tesla V100 (32 GB) GPU, with a
training time of around 26 hours. To train our model we set,
$\lambda_{spk}=0.5$, $\lambda_{aspk}=0.1$, $\lambda_{style}=1$,
$\lambda_{div}=1$, $\lambda_{asr}=10$, $\lambda_{norm}=1$,
$\lambda_{cycle}=5$, $\lambda_{F0}=5$, $\lambda_{demo}=2$, and
$\lambda_{inv}=5$. AdamW [23] is used with the learning rate of $10^{-4}$. A
HiFiGAN [24] vocoder is trained with datasets mentioned in Table 1. The
vocoder generates a 1 minute long waveform from the converted mel-spectrogram
in 0.1 seconds. For the assessment of emotion preservation, we train a support
vector machine (SVM) based emotion classifier as done in [25].

###  4.2 Datasets

Four datasets have been used in this work, which have been used for different
purposes, as shown in Table 1. We consider 5 emotion classes $e\in$ {happy,
sad, anger, neutral, surprise} for the emotion preservation evaluation.

Table 1: Depicts usage of datasets for different purposes. The column Emotion
denotes whether the dataset is used to train the emotion conversion network
and, the emotion classifier for evaluation.

Dataset | VC Training | VC Evaluation | Vocoder Training | Emotion
---|---|---|---|---
ESD | Yes | Yes | Yes | Yes
VCTK | Yes | Yes | Yes | No
RAVDESS | No | No | Yes | Yes
CREMA-D | No | No | Yes | No

Emotional Speech Database (ESD) [22]: The corpus contains English and Chinese
emotional utterances belonging to the 5 emotion classes in $e$. We consider
only English utterances in our work, where 350 utterances per emotion class
are spoken by 5 male and 5 female native English speakers.

Voice Cloning Toolkit (VCTK) [26]: The dataset contains English utterances
from 109 speakers having various accents. For training of VC models, we
consider utterances having English accent from 5 randomly selected males and
females. The evaluation is performed on utterances having English, American
and Canadian accents.

Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [27]:
This corpus comprises 2880 utterances spoken by 24 professional actors from
North America. The dataset has ground truth for 7 emotion classes, but we
consider the ones mentioned in $e$.

Crowd-sourced Emotional Multimodal Actors Dataset (CREMA-D) [28]: This is
another corpus having emotional utterances having the mentioned 5 emotion
classes in $e$. The dataset contains 7442 clips from 91 actors belonging to
diverse age groups and ethnicity.

###  4.3 Evaluation Setup

To evaluate the efficacy of the proposed methods, we perform both objective
and subjective evaluations. For all assessments, we choose an equal number of
male and female source and target speakers from each dataset.

Objective Evaluation: We choose test utterances from 10 source speakers and 6
target speakers from the ESD dataset, leading to 2100 ESD$\rightarrow$ESD
conversions. For VCTK$\rightarrow$VCTK, we form 2000 conversions by selecting
test utterances from 10 source speakers and 6 target speakers.

To evaluate emotion preservation, we compare the models using 3 metrics: i)
ACCgt: accuracy between the source emotion ground truth and the SVM predicted
emotion label of the converted utterances. ii) ACCsvm: accuracy between the
predicted labels of the source and the converted samples, where both are
predicted by the SVM. The metric is especially useful when no emotion ground
truth is available, as in VCTK dataset. 3) MAEembed: mean absolute error (MAE)
between the emotion-embedding of the source and converted utterances, where
the emotion embedding is extracted using the automatic embedding extractor. We
report the pitch correlation coefficient (PCC) [29] as a measure of intonation
preservation, predicted mean opinion score (pMOS) [30] as a measure of the
quality of converted samples, and character error rate (CER) as a content
preservation evaluation measure. As a measure of speaker anonymization, we
report speaker similarity score (SSS) between the source and the converted
speech. The score is generated using a speaker verification toolkit [31] from
Hugging-Face repository111https://huggingface.co/speechbrain.

Subjective Evaluation: We randomly choose 100 conversions for the subjective
evaluation, as it is expensive and time-consuming to assess all of them. A
user study was conducted on Crowdee222https://www.crowdee.com/ platform, where
200 native English speakers participated. Each subject performs two types of
assessments: 1) verify emotion leakage from reference: a triplet of converted,
source and reference utterances were provided. The subjects were asked to
select between source and reference utterances, the one which has a similar
emotion as the converted utterance (ignoring content or voice similarity). ii)
voice quality: assess the naturalness on a 5-point scale (1: bad to 5:
excellent). The users were not apprised about whether the converted utterance
is produced by Baseline or by the proposed model. Each of the tasks was rated
by at least 5 subjects. The subjects were provided with an anchor question and
also hidden trapping questions for quality check. The subjects failing the
trapping questions were not considered for the analysis.

###  4.4 Results and Discussion

Table 2: Objective evaluation results. Mean and standard deviation (in
brackets) are reported. `All Conv.' includes all conversions. Type column
denotes different conversion sub-groups, such as source-emotion, source
$\rightarrow$ target accents, source $\rightarrow$ target genders
(M$\rightarrow$M, M$\rightarrow$F, F$\rightarrow$F, F$\rightarrow$M) or, `All'
indicates all sub-groups. StarGAN-VC++ is denoted by SG++.

| Source -
---
Target
| Type
---
ACCgt [%] $\uparrow$ |  ACCsvm [%] $\uparrow$ |  MAEembed [$\times 10^{2}$] $\downarrow$ |  PCC [$\times 10^{2}$] $\uparrow$ |  pMOS $\uparrow$ |  CER [%] $\downarrow$ |  SSS [$\times 10^{2}$] $\downarrow$
|  | Baseline | SG++ | Baseline | SG++ | Baseline | SG++ | Baseline | SG++ | Baseline | SG++ | Baseline | SG++ | Baseline | SG++
All Conv. | All | 18.5 | 30.4 | 25.9 | 52.7 | 58.2 (19.2) |  45.7 (13.1) | 77.3 (15.3) |  78.3 (16.7) | 3.5 (0.5) | 3.5 (0.6) | 3.5 (8.5) |  3.2 (8.0) | 24.4 (15.9) |  23.9 (16.6)
ESD $\rightarrow$ ESD | All | 20.2 | 48.9 | 20.9 | 65.0 | 64.4 (22.6) |  40.8 (12.5) | 80.2 (13.4) |  84.2 (10.6) | 3.8 (0.4) | 3.8 (0.4) | 3.8 (8.4) |  3.0 (7.4) |  26.0 (18.8) | 28.0 (19.4)
Happy | 17.2 | 47.0 | 19.1 | 54.7 | 62.0 (18.8) |  40.6 (11.5) | 78.8 (15.8) |  84.1 (11.6) | 3.6 (0.5) |  3.7 (0.4) | 5.9 (9.7) |  3.5 (8.5) | - | -
Angry | 12.5 | 76.5 | 12.5 | 77.5 | 62.1 (18.8) |  35.3 (8.6) | 78.0 (15.3) |  84.2 (10.7) |  3.8 (0.4) | 3.7 (0.3) | 2.9 (6.3) |  2.2 (6.0) | - | -
Sad | 12.6 | 45.4 | 12.6 | 45.1 | 62.9 (16.8) |  46.1 (13.4) | 85.0 (11.5) |  86.0 (11.2) | 3.8 (0.4) |  4.0 (0.4) | 2.7 (7.2) |  1.9 (5.2) | - | -
Surprise | 0.0 | 0.0 | 1.5 | 62.9 | 83.5 (25.0) |  44.7 (14.7) | 80.7 (9.9) |  84.6 (7.9) | 3.7 (0.4) | 3.7 (0.4) | 6.0 (10.5) |  5.4 (9.3) | - | -
Neutral | 79.7 | 90.7 | 79.7 | 90.7 | 45.9 (12.7) |  36.4 (8.4) | 78.4 (12.5) |  81.4 (11.1) | 4.1 (0.4) |  4.2 (0.4) | 1.4 (5.5) |  1.2 (5.6) | - | -
M$\rightarrow$M | 19.0 | 47.2 | 19.6 | 54.4 | 62.9 (19.1) |  40.3 (11.4) | 76.8 (13.7) |  82.4 (11.2) | 3.6 (0.4) |  3.7 (0.4) | 3.4 (7.2) |  2.5 (6.8) | - | -
M$\rightarrow$F | 16.8 | 41.3 | 17.6 | 66.7 | 61.5 (21.4) |  37.6 (10.4) | 83.4 (10.5) |  86.1 (9.3) | 3.8 (0.4) |  3.9 (0.4) | 2.7 (5.8) |  2.6 (6.2) | - | -
F$\rightarrow$F | 21.6 | 52.3 | 23.3 | 75.1 | 64.6 (25.1) |  40.7 (12.9) | 85.0 (9.8) |  86.4 (9.1) | 3.9 (0.4) |  4.0 (0.4) | 3.6 (8.9) |  2.6 (6.3) | - | -
F$\rightarrow$M | 23.0 | 54.1 | 22.6 | 62.6 | 68.6 (23.3) |  44.9 (14.0) | 74.6 (16.4) |  81.6 (11.9) | 3.6 (0.5) |  3.7 (0.5) | 5.4 (10.4) |  4.3 (9.7) | - | -
VCTK $\rightarrow$ VCTK | All | - | - | 31.1 | 39.9 | 51.7 (11.6) | 50.7 (11.7) |  74.2 (16.5) | 72.2 (19.4) | 3.2 (0.5) | 3.2 (0.5) |  3.1 (8.7) | 3.4 (8.6) | 22.7 (12.1) |  19.6 (11.4)
M$\rightarrow$M | - | - | 14.5 | 23.8 | 49.7 (11.6) |  49.1 (12.3) |  64.1 (16.3) | 60.4 (18.3) | 3.1 (0.5) | 3.1 (0.5) |  3.3 (9.1) | 3.8 (9.4) | - | -
M$\rightarrow$F | - | - | 43.8 | 62.5 | 49.0 (11.4) |  47.3 (11.5) | 82.6 (12.5) |  82.9 (12.4) |  3.6 (0.4) | 3.5 (0.4) |  3.5 (8.2) | 4.1 (9.0) | - | -
F$\rightarrow$F | - | - | 45.2 | 52.0 |  51.2 (9.8) | 52.5 (10.7) | 86.9 (8.5) |  87.1 (8.5) |  3.4 (0.4) | 3.3 (0.4) | 3.1 (9.1) |  2.9 (7.6) | - | -
F$\rightarrow$M | - | - | 30.7 | 34.2 | 56.3 (11.7) |  53.4 (11.1) |  70.0 (14.1) | 66.7 (19.2) |  3.0 (0.5) | 2.9 (0.5) |  2.6 (8.2) | 2.9 (8.0) | - | -
English$\rightarrow$English | - | - | 28.5 | 38.0 | 49.3 (11.5) |  47.9 (11.5) |  71.8 (16.9) | 70.2 (19.6) |  3.2 (0.5) | 3.1 (0.5) | 3.6 (9.7) | 3.6 (9.2) | - | -
American$\rightarrow$English | - | - | 34.3 | 44.2 | 52.8 (11.0) |  52.3 (11.5) |  76.9 (15.4) | 74.3 (18.9) | 3.2 (0.5) | 3.2 (0.5) |  2.5 (7.7) | 3.3 (8.2) | - | -
Canadian$\rightarrow$English | - | - | 29.9 | 35.3 | 54.3 (12.0) |  53.2 (11.5) |  73.8 (16.9) | 72.0 (19.5) | 3.1 (0.5) |  3.2 (0.5) | 3.5 (8.3) |  3.1 (8.0) | - | -

The results of the objective evaluations are summarized in Table 2. For all
conversions involving ESD and VCTK corpora, the proposed StarGAN-VC++
outperforms Baseline with respect to all emotion-preservation metrics. In
terms of ACCsvm, Baseline only manages to achieve 25.9% emotion preservation
accuracy, whereas StarGAN-VC++ achieves 52.7%. As per MAEembed, StarGAN-VC++
achieves a significantly lower mean score of 45.7 compared to Baseline's score
of 58.2. Also for pitch correlation, StarGAN-VC++ achieves a higher value of
PCC (78.3) compared to Baseline (77.3). These results are statistically
significant, as paired t-test achieves $p<0.0001$ on PCC and MAEembed metrics.

For naturalness, both methods show similar pMOS values, which indicate that
the proposed emotion-preservation techniques do not degrade the naturalness of
the conversions. As far as content-preservation is concerned, StarGAN-VC++
shows a statistically significantly ($p<0.05$) lower CER value (3.2%) than
Baseline (3.5%). This improvement might be attributed to the proposed
$\mathcal{L}_{inv}$ loss. The results also reveal that the speaker
anonymization capability is not hampered by the introduction of emotion-
preserving losses, as the mean SSS values for Baseline and StarGAN-VC++ are
24.4 and 23.9 respectively. This result is also statistically significant as
$p<0.001$ in paired t-test.

We also evaluate the models emotion-wise. For all 5 emotions in $e$, StarGAN-
VC++ outperforms Baseline with respect to emotion preservation. Interestingly,
StarGAN-VC++ shows significantly higher mean PCC values for all of these
emotions compared to Baseline. This reveals that for emotional utterances, the
variations of pitch are also better preserved by StarGAN-VC++, where pitch
variations are indicative of emotional cues in speech [32]. The results also
show that Baseline deals with neutral utterances better than other emotions,
as it attains a very high score of 79.7% for ACCsvm, whereas for other
emotions ACCsvm lies in the range of 1.5% - 19.1%. Furthermore, it appears
that surprise is the most difficult emotion to preserve, as both Baseline and
StarGAN-VC++ models attain a 0% score for ACCgt. However, in terms of ACCsvm,
our StarGAN-VC++ model gets 62.9% accuracy against 1.5% by Baseline. As per
[33], surprise is also the most difficult emotion for emotion recognition
tasks as well.

With respect to gender-wise evaluation for ESD$\rightarrow$ESD, StarGAN-VC++
shows a similar trend of better emotion-preservation as per the metrics.
StarGAN-VC++ shows significant improvement in PCC values over Baseline and,
also achieves a lower CER. StarGAN-VC++ model achieves the highest ACCsvm
value of 75.1% for F$\rightarrow$F conversions against 23.3% by Baseline.
However, in terms of ACCgt, StarGAN-VC++ achieves the highest score of 54.1%
for F$\rightarrow$M conversions, against a score of 23.0% by Baseline. For
VCTK$\rightarrow$VCTK conversions, the improvement in PCC and MAEembed are not
that significant. This is because the utterances in VCTK do not have emotional
utterances with high pitch and intonation variations and, the change in F0
contour is relatively less compared to ESD utterances.

With respect to accent-wise evaluation, we assess for
English$\rightarrow$English, American$\rightarrow$English, and
Canadian$\rightarrow$English sub-groups. The models were not trained with
utterances from American and Canadian speakers from the VCTK dataset, which
forms the unseen$\rightarrow$seen speaker evaluation scenario as well.
StarGAN-VC++ model outperforms Baseline in all three accent scenarios with
respect to emotion preservation, scoring significantly higher ACCsvm values
and lower MAEembed values. Both Baseline and StarGAN-VC++ models achieve the
highest ACCsvm values for American$\rightarrow$English conversions, 34.3 % and
44.2% respectively. StarGAN-VC++ does not manage to achieve higher PCC values
for these accent-based conversion sub-groups.

We also performe an ablation study on the proposed losses and the results are
presented in Table 3. The ablation study reveals that both of the proposed
losses, deep embeddings-based and the augmented style reconstruction losses
have a significant impact on emotion preservation. In the absence of
$\mathcal{L}_{demo}$ loss, ACCsvm value comes down to 36.4% from 52.7% for
StarGAN-VC++. Similarly, when vanilla $\mathcal{L}_{style}$ is used instead on
the proposed augmented version, the ACCsvm value comes down to 34.3%. The drop
is higher than that of $\mathcal{L}_{demo}$ loss, which implies that the
augmented style reconstruction loss results in a better disentanglement of
target speakers embeddings, and that facilitates emotion preservation. The use
of augmented style reconstruction loss also improves the pitch correlation, as
it achieves the mean PCC value of 79.8. In terms of quality, the pMOS value
remains unchanged in all cases.

Table 3: Ablation experiments results. Mean and standard deviation (in
brackets) are reported.

Method |  ACCsvm [%] $\uparrow$ | PCC [$\times 10^{2}$] $\uparrow$ |  pMOS $\uparrow$ |  CER [%] $\downarrow$
---|---|---|---|---
Baseline | 25.9 | 77.3 (15.3) | 3.5 (0.5) | 3.5 (8.5)
StarGAN-VC++ | 52.7 |  78.3 (16.7) | 3.5 (0.6) |  3.2 (8.0)
$\lambda_{demo}=0$ | 36.4 |  79.8 (14.0) |  3.5 (0.5) |  4.3 (9.5)
Vanilla $\mathcal{L}_{style}$ | 34.3 | 77.5 (15.3) | 3.4 (0.6) | 4.8 (9.9)

The results for the subjective evaluation are presented in Table 4. In terms
of MOS, our proposed method does not degrade the naturalness of the
conversion, rather it is marginally better (4.1) than Baseline (3.9). The
second column in Table 4 represents the number of times the converted sample's
emotion is marked same as the source, i.e. no emotion leakage happened.
StarGAN-VC++ achieves higher votes (331) than Baseline (258). The last column
depicts whether the user marked the converted utterance's emotion similar to
the reference rather than the source, another additional check for emotion
leakage. For Baseline it is 209 votes, whereas for StarGAN-VC++ it is 114
votes, which further confirms that our proposed approach reduces the leakage.
The audio samples are provided in the supplementary. The code can be found
online 333https://github.com/arnabdas8901/StarGAN-VC_PlusPlus.git.

Table 4: Subjective evaluation results. Mean and standard deviation (in
brackets) values are reported for MOS. StarGAN-VC++ is denoted by SG++. The
source utterances received mean MOS score 4.0 (0.9).

MOS $\uparrow$ |  Conv. == Source Emo. $\uparrow$ |  Conv. == Ref. Emo. $\downarrow$
---|---|---
Baseline | SG++ | Baseline | SG++ | Baseline | SG++
3.9 (1.0) |  4.1 (0.8) | 258 | 331 | 209 | 114

##  5 Conclusion

In this paper, we investigate the cause of failure of the state-of-the-art VC
method StarGANv2-VC with respect to emotion preservation. Our study reveals
that the failure is attributed to the framework's disability in the
disentanglement of the target speaker's style and emotion embeddings while
training. Consequently, the emotional content from the reference utterance
leaks to the decoder, hampering the emotion preservation of the source
utterance. We propose a novel deep emotion-embedding generation technique and
emotion-aware losses for the VC, which encourages the generator to preserve
source emotion. The objective and subjective evaluation results show that the
proposed model improves the emotion preservation capability for diverse
emotions, gender and accent groups, without compromising the quality of the
converted samples. Further, the results show that for emotional data with high
pitch variations, our proposed method improves the pitch correlation
considerably, implying F0-consistent conversions. As a future task, we plan to
use the proposed emotion-aware losses with other TTS-based methods. Also, we
intend to generate emotion embeddings by using other representations of
emotions, such as the two dimensional model of valence and arousal [34].

##  6 Acknowledgements

This research has been partly funded by the Federal Ministry of Education and
Research of Germany in the project Emonymous (project number S21060A) and
partly funded by the Volkswagen Foundation in the project AnonymPrevent (AI-
based Improvement of Anonymity for Remote Assessment, Treatment and Prevention
against Child Sexual Abuse).

## References

  * [1] M. Zhang, X. Wang, F. Fang, H. Li, and J. Yamagishi, ``Joint training framework for text-to-speech and voice conversion using multi-source tacotron and wavenet,'' _arXiv preprint arXiv:1903.12389_ , 2019.
  * [2] K. Ezzine and M. Frikha, ``A comparative study of voice conversion techniques: A review,'' in _2017 International Conference on Advanced Technologies for Signal and Image Processing (ATSIP)_. IEEE, 2017, pp. 1–6.
  * [3] Siegert, S. Varod, Carmi, and Kamocki, ``Personal data protection and academia: Gdpr issues and multi-modal data-collections "in the wild",'' _The Online Journal of Applied Knowledge Management: OJAKM_ , vol. 8, pp. 16 – 31, 2020\.
  * [4] T. Walczyna and Z. Piotrowski, ``Overview of voice conversion methods based on deep learning,'' _Applied Sciences_ , vol. 13, no. 5, p. 3100, 2023.
  * [5] S. Desai, E. V. Raghavendra, B. Yegnanarayana, A. W. Black, and K. Prahallad, ``Voice conversion using artificial neural networks,'' in _2009 IEEE International Conference on Acoustics, Speech and Signal Processing_. IEEE, 2009, pp. 3893–3896.
  * [6] H. Ming, D.-Y. Huang, L. Xie, J. Wu, M. Dong, and H. Li, ``Deep bidirectional lstm modeling of timbre and prosody for emotional voice conversion.'' in _Interspeech_ , 2016, pp. 2453–2457.
  * [7] Z. Lian, Z. Wen, X. Zhou, S. Pu, S. Zhang, and J. Tao, ``Arvc: An auto-regressive voice conversion system without parallel training data.'' in _INTERSPEECH_ , 2020, pp. 4706–4710.
  * [8] B. Sisman, J. Yamagishi, S. King, and H. Li, ``An overview of voice conversion and its challenges: From statistical modeling to deep learning,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 29, pp. 132–157, 2020.
  * [9] C.-C. Hsu, H.-T. Hwang, Y.-C. Wu, Y. Tsao, and H.-M. Wang, ``Voice conversion from non-parallel corpora using variational auto-encoder,'' in _2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)_. IEEE, 2016, pp. 1–6.
  * [10] D.-Y. Wu and H.-y. Lee, ``One-shot voice conversion by vector quantization,'' in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 7734–7738.
  * [11] H. Tang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, ``Avqvc: One-shot voice conversion by vector quantization with applying contrastive learning,'' in _Proc. of the IEEE ICASSP_ , 2022, pp. 4613–4617.
  * [12] T. Kaneko and H. Kameoka, ``Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial networks,'' in _2018 26th European Signal Processing Conference (EUSIPCO)_. IEEE, 2018, pp. 2100–2104.
  * [13] S. Lee, B. Ko, K. Lee, I.-C. Yoo, and D. Yook, ``Many-to-many voice conversion using conditional cycle-consistent adversarial networks,'' in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 6279–6283.
  * [14] J.-c. Chou, C.-c. Yeh, H.-y. Lee, and L.-s. Lee, ``Multi-target voice conversion without parallel data by adversarially learning disentangled audio representations,'' _arXiv preprint arXiv:1804.02812_ , 2018.
  * [15] S. Meyer, F. Lux, P. Denisov, J. Koch, P. Tilli, and N. T. Vu, ``Speaker anonymization with phonetic intermediate representations,'' _arXiv preprint arXiv:2207.04834_ , 2022.
  * [16] Y. Zhao, W.-C. Huang, X. Tian, J. Yamagishi, R. K. Das, T. Kinnunen, Z. Ling, and T. Toda, ``Voice conversion challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion,'' _arXiv preprint arXiv:2008.12527_ , 2020\.
  * [17] Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo, ``Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,'' in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 8789–8797.
  * [18] H. Kameoka, T. Kaneko, K. Tanaka, and N. Hojo, ``Stargan-vc: Non-parallel many-to-many voice conversion using star generative adversarial networks,'' in _2018 IEEE Spoken Language Technology Workshop (SLT)_. IEEE, 2018, pp. 266–273.
  * [19] T. Kaneko, H. Kameoka, K. Tanaka, and N. Hojo, ``Stargan-vc2: Rethinking conditional methods for stargan-based voice conversion,'' _arXiv preprint arXiv:1907.12279_ , 2019.
  * [20] Y. A. Li, A. Zare, and N. Mesgarani, ``Starganv2-vc: A diverse, unsupervised, non-parallel framework for natural-sounding voice conversion,'' _arXiv preprint arXiv:2107.10394_ , 2021.
  * [21] S. Ghosh, Y. Sinha, I. Siegert, and S. Stober, ``Improving voice conversion for dissimilar speakers using perceptual losses,'' _Deutsche Gesellschaft für Akustik e.V._ , pp. 1358–1361, 2023.
  * [22] K. Zhou, B. Sisman, R. Liu, and H. Li, ``Emotional voice conversion: Theory, databases and ESD,'' _Speech Communication_ , vol. 137, pp. 1–18, 2022\.
  * [23] I. Loshchilov and F. Hutter, ``Decoupled weight decay regularization,'' in _International Conference on Learning Representations_ , 2019.
  * [24] J. Kong, J. Kim, and J. Bae, ``HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,'' _Advances in Neural Information Processing Systems_ , vol. 33, pp. 17 022–17 033, 2020.
  * [25] T. Seehapoch and S. Wongthanavasu, ``Speech emotion recognition using support vector machines,'' in _2013 5th international conference on Knowledge and smart technology (KST)_. IEEE, 2013, pp. 86–91.
  * [26] J. Yamagishi, C. Veaux, K. MacDonald _et al._ , ``CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),'' _University of Edinburgh. The Centre for Speech Technology Research (CSTR)_ , 2019.
  * [27] S. R. Livingstone and F. A. Russo, ``The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english,'' _PloS one_ , vol. 13, no. 5, p. e0196391, 2018.
  * [28] H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, ``Crema-d: Crowd-sourced emotional multimodal actors dataset,'' _IEEE transactions on affective computing_ , vol. 5, no. 4, pp. 377–390, 2014.
  * [29] N. Tomashenko, X. Wang, X. Miao, H. Nourtel, P. Champion, M. Todisco, E. Vincent, N. Evans, J. Yamagishi, and J. F. Bonastre. The VoicePrivacy 2022 Challenge Evaluation Plan. Visited on 2023-03-03. [Online]. Available: https://arxiv.org/pdf/2203.12468.pdf
  * [30] P. Andreev, A. Alanov, O. Ivanov, and D. Vetrov, ``Hifi++: a unified framework for bandwidth extension and speech enhancement,'' _arXiv preprint arXiv:2203.13086_ , 2022.
  * [31] M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong _et al._ , ``Speechbrain: A general-purpose speech toolkit,'' _arXiv preprint arXiv:2106.04624_ , 2021\.
  * [32] F. Weninger, F. Eyben, B. W. Schuller, M. Mortillaro, and K. R. Scherer, ``On the acoustics of emotion in audio: what speech, music, and sound have in common,'' _Frontiers in psychology_ , vol. 4, p. 292, 2013.
  * [33] L. Chen, X. Mao, Y. Xue, and L. L. Cheng, ``Speech emotion recognition: Features and classification models,'' _Digital signal processing_ , vol. 22, no. 6, pp. 1154–1160, 2012.
  * [34] J. A. Russell and A. Mehrabian, ``Evidence for a three-factor theory of emotions,'' _Journal of research in Personality_ , vol. 11, no. 3, pp. 273–294, 1977.


# Can Generative Agents Predict Emotion?

Ciaran Regan, Nanami Iwahashi, Mizuki Oka
Grad. School of Science and Technology
University of Tsukuba
Tsukuba, Ibaraki, Japan
<EMAIL_ADDRESS>
&Shogo Tanaka
Grad. School of Letters
Tokai University
Tokyo, Japan

###### Abstract

Large Language Models (LLMs) have demonstrated a number of human-like
abilities, however the empathic understanding and emotional state of LLMs is
yet to be aligned to that of humans. In this work, we investigate how the
emotional state of generative LLM agents evolves as they perceive new events,
introducing a novel architecture in which new experiences are compared to past
memories. Through this comparison, the agent gains the ability to understand
new experiences in context, which according to the appraisal theory of emotion
is vital in emotion creation. First, the agent perceives new experiences as
time series text data. After perceiving each new input, the agent generates a
summary of past relevant memories, referred to as the norm, and compares the
new experience to this norm. Through this comparison we can analyse how the
agent reacts to the new experience in context. The PANAS, a test of affect, is
administered to the agent, capturing the emotional state of the agent after
the perception of the new event. Finally, the new experience is then added to
the agents memory to be used in the creation of future norms. By creating
multiple experiences in natural language from emotionally charged situations,
we test the proposed architecture on a wide range of scenarios. The mixed
results suggests that introducing context can occasionally improve the
emotional alignment of the agent, but further study and comparison with human
evaluators is necessary. We hope that this paper is another step towards the
alignment of generative agents. 111A public repository for the code can be
found at:
https://github.com/tsukuba-websci/GenerativeAgentsPredictEmotion.

_K_ eywords Generative Agents $\cdot$ Large Language Models $\cdot$ Emotion
$\cdot$ Affect

##  1 Introduction

Large Language Models (LLMs) have exhibited a number of emergent abilities
[1]. One of these abilities, Theory of Mind (ToM), is thought to have emerged
in LLMs as a byproduct of their improved language skills. ToM, which is
defined as the ability to impute unobservable mental states of others, enables
humans to track the emotions, intentions, beliefs and desires of third
parties, and is thought to play a key role in social interactions,
communication and empathy. This understanding of unoberserable states is
thought to be an ability unique to humans, with even the most socially
intelligent animals failing to exhibit ToM. Given the importance ToM plays in
human interactions, there have been significant efforts to equip AI with ToM
like abilities in order to achieve a more safe, responsible and human-like AI
[2]. It seems, however, that due to the diverse descriptions of mental states
in the training data of LLMs, that ToM has appeared as an emergent property
[3]. If this is the case, then LLMs have the ability to develop strong
psycological models of others, and empathise with their emotions and thoughts
[4].

Although being able to react appropriately in certain sitiations, LLMs fall
short in alignment with the emotional behaviours of humans and cannot
establish connections between similar situations[5]. One of the possible
explanations for this is that an LLM cannot respond to events in the same way
as humans for the lack of criteria to assess them that have been formed
through related episodic memories. On the one hand, according to the appraisal
theory of emotions (ATE), a cognitive approach to understanding emotions, our
appraisals of the significance of the event triggers and determines a proper
emotion in the given environment[6, 7]. That is, how we assess events directly
influences how we emotionally respond to them. On the other hand,
neuropsychology suggests that that episodic memories shape how we perceive new
events[8]. Based on the memories of past experiences, our brain generates a
model of the world around us that informs our perception of upcoming
events[9]. In this regard, the role of episodic memory seems crucial in
generating both the criteria to assess an event and the model to perceive new
events. Although LLMs are able to posit a guess of the emotions an experience
would cause due to their vast amounts of training data, they lack an episodic
memory, which is required by ATE to accurately simulate human-like emotional
responses.

We motivate the need for a context-based architecture with the following
example. Consider the series of images describing the experiences of an
individual, shown in Fig 1(a). The first scene depicts an empty park which can
be interpreted in a variety of ways, such as isolated and lovely or peaceful
and calm. In contrast to this, Fig 1(b) depicts of series of images showing
people playing soccer in the park. The final image of an empty park, raises
the question of “why is there no people here now?”. This final experience is
in stark contrast to the prior experiences, and while although still
ambigioius, has a stronger emphasis on negative emotions.

(a) An empty park.

(b) A park that was once full of activity has become empty.

Figure 1: The context of an experience effects the emotional reaction. Fig
1(a) depicts an empty park, which could be interepreted as either lonely and
sad or peaceful and calm. In contrast, Fig 1(b) depicts a series of people
playing football in the park, followed by a final scene of an empty park. In
this case, the final scene becomes less ambigious, with a stronger emphasis on
lonliness and isolation.

To understand how the emotional state of generative agents evolves with the
perception of new events we propose a novel architecture in which non-embodied
generative agents perceive new experiences as text, which are then stored as
episodic memories. The percecption of these new experience triggers the
creation of a “norm”, a summary of past experiences. The norm is compared to
the new experience to enable the agent to understand the new event in context.
The PANAS, a test for emotional affect, is then administered, allowing the
emotional response of the agent with respect the new experience in context to
be measured. We test this architecture with over 400 situations so that the
changes in the agents emotional response can be measured in a wide range of
scenarios. To asses the emotional alignment of agents in this architecture, we
select a number of these situations in which the architecture both
successfully and unsuccessfully improves the emotional alignment of the
agents. Our results suggest that at times the context improves the alignment,
however the context often does not improve the emotional understanding of the
agents.

This paper makes the following contribution:

  1. 1.

We propose a novel agent architecture in which the evolution of a generative
agents emotional state can be analysed by comparing new experiences to the
norm of past memories.

##  2 Related Work

#### Generative Agents

Generative agents are a type of computational software agent that simulate
realistic human behaviours. These believable proxies of human behaviour are
powered by LLMs and an architecture which gives them the ability to observe
experiences, reflect on past memories and plan future actions. In the seminal
paper, the agents exist in a virtual town in which multiple agents can
interact, with these agents demonstrating remarkably realistic behaviours and
capabilities. For instance, given a single user input to plan a party, agents
autonomously spread invitations and many agents attended the party in the
virtual town.

Despite displaying many realstic behaviours, it remains to be seen if the
agents can understand emotion in the same way as humans. To measure the
emotional state of LLMs in response to a wide range of scenarios, EmotionBench
was introduced [5], which provides a dataset of both human and LLM responses
to emotional situations. In particular, EmotionBench provides over 400
situations that are known to elicit feelings such as depression, anger and
guilt in humans. These emotional situations are then further broken down into
more specific types of emotional factors, such as the anger felt when facing
self-opionated people, or the depression felt when failing to achieve an
important goal. By comparing the response of LLMS and humans to these
situations, it was shown that LLMs fall short in emotional alignment with
humans. One possible explanation for this is the lack of context to the
EmotionBench situations, which psycology posits is vital to emotion creation.

#### Appraisal Theory of Emotion

Appraisal Theory of Emotion (ATE) is a cognitive approach to understanding
emotion that was originally developed to explain how different emotions may
emerge from the same event, in different individuals and on different
occasions. ATE claims that it is not only the event or situation that elicits
an emotional response, but the individual’s appraisal of the event based on
its context [7]. For instance, consider the situation where you are wandering
in an empty park. You may feel sad or lonely due to the isolation. On the
other hand, you may feel a sense of calmness or tranquility due to the
peacefulness of nature. One of the ATE’s goals is to understand how
individuals may feel these two different emotions in the same situation.

In terms of the appraisal process, ATE posits diverse variables that are
related in evaluating the situation such as goal relevance and goal
congruence, certainty, agency, and coping potential or control[7]. However, in
applying ATE to the architecture of LLM, we need to take ATE’s presupposition
into consideration, that is, ATE presupposes an embodied human agent that has
an agency for actions, bodily skills to cope with a given situation, and the
feeling of certainty based on the agency and the skills. In this paper,
reflecting that the current LLM is not an embodied agent, we propose to
summarize all variables as a “norm” composed of episodic memories. As we
indicated in the introduction, the norm embedded within LLM would serve as the
criteria to evaluate the perceived situation, as well as the world model to
shape the perception [9].

#### The Positive And Negative Affect Schedule

In order to assess and compare the emotional state of LLMs, we make use of the
Postive And Negative Affect Schedule (PANAS). The PANAS, introduced in 1988,
is one of the most commonly used scales to measure emotion and mood in
individuals. The PANAS asks individuals to rate 20 different emotions on a
Likert Scale between 1 (very slightly or not at all) to 5 (extremely likely).
Of the 20 questions, 10 questions correspond to positive affect and 10
questions correspond to negative affect, shown in table 1. The 10 positive and
10 negative affect scores are summed, resulting in two values: a positive
affect score and a negative affect score. Although, the test is relatively
simple, the PANAS can be used to estimate how an individuals emotional changes
overtime. Additionally, the PANAS has been shown to be easily administered to
LLMs using natural language and therefore presents a good method to estimate
how a generative agents emotional perception changes overtime.

Table 1: The Emotions Scored in the Positive and Negative Affect Schedule. It is important to note that the emotions do not contain “happiness” or “sadness” explicitly. Positive Affect | Negative Affect
---|---
Attentive |  |  | Hostile
Active |  |  | Irritable
Alert |  |  | Ashamed
Excited |  |  | Guilty
Enthusiastic |  |  | Distressed
Determined |  |  | Upset
Inspired |  |  | Scared
Proud |  |  | Afraid
Interested |  |  | Jittery
Strong |  |  | Nervous

##  3 Method

Based on the appraisal theory of emotion and the original work on generative
agents [10], a novel architecture is proposed that evaluates the emotional
respose of the agents in context. An overview of this architecture is depicted
in Fig 2.

First, new experiences in the form of natural language are input to the agent,
which acts as perception. Following this, the agent fetches past memories,
weighted by saliency, relevancy and recency. Using Prompt 1 the agent
summarises and extracts insights from past memories, creating the norm. This
norm may contain information such as the agents habits, usual behaviours and
expectations. There is a one to one relationship between norms and memories,
that is, the perception of a new experience triggers the creation of a new
norm associated with it.

After the norm is created, it is compared to the new experience to create a
“contextual understanding”. The goal of this contextual understanding is to
highlight the differences betwen the current situation and the norm. The
contextual understanding is created using Prompt 2. To assess the emotional
response of the agent, the PANAS is then administered using Prompt 3,
providing the contextual understanding. To reduce potential bias, the order of
the emotions in the PANAS prompt are randomised each time.

Finally, the new experience is stored as a memory, which can be utilised in
the creation of subsequent norms to extract the emotion evoked in future
experiences. In particular, we store each memory and norm as nodes in a graph
database, allowing the relationship between memories and norms to be
visualised. In this way, the most influential memories, those which created
the most norms, can be visualised. In addition, the affect of the saliency,
recency, and relevance weights in the memory retriever can be visualised using
graph visualistations, however, understanding how these weights affects the
emotional state of the agent is left to future work.

Figure 2: The proposed architecture. A new experience triggers the create a
new norm based on past memories. The new experience and norm are compared to
create a contextual understanding. Using this contextural understanding, the
PANAS is administered to measure the emotional response of the agent.

To demonstrate the proposed architecture, consider an agent observing the
scenes described in 1(b). On perceiving each of the scenes, a norm is created.
For example, after perceiving the first two soccer scenes, the agent has the
following two memories:

⬇ Memory 1: In the image, a man is kicking a soccer ball in a park. There are
several other people in the background, possibly enjoying the outdoors or
engaging in other activities. Memory 2: In the image, a man is kicking a
soccer ball in a park. There are several other people in the background,
possibly enjoying the outdoor activities or watching the man play.

On perceiving the third soccer scene, the agent creates the following norm,
capturing the context and insights of the prior memories. An experpt of the
norm is given below:

⬇ The man appears to play soccer in the park every week, with other people
consistently present in the background.

Then, the contextual understanding is created:

⬇ The new memory does not provide new insights that significantly deviate from
what has been established. Instead, it reaffirms the man’s habit of playing
soccer and the park’s role as a communal outdoor space.

The PANAS is then administered to the agent, with the response shown in Table
3. Additionally, the response of an agent without context is shown in 3. At
this time, the emotional response of both agents are similar.

Table 2: With Context Positive Affect | Negative Affect
---|---
Emotion | Score | Emotion | Score
Attentive | 3 | Hostile | 1
Active | 4 | Irritable | 2
Alert | 3 | Ashamed | 1
Excited | 4 | Guilty | 1
Enthusiastic | 5 | Distressed | 1
Determined | 4 | Upset | 1
Inspired | 4 | Scared | 1
Proud | 3 | Afraid | 1
Interested | 4 | Jittery | 2
Strong | 3 | Nervous | 2
Total | 37 | Total | 11

Table 3: Without Context Positive Affect | Negative Affect
---|---
Emotion | Score | Emotion | Score
Attentive | 4 | Hostile | 1
Active | 4 | Irritable | 2
Alert | 4 | Ashamed | 1
Excited | 4 | Guilty | 1
Enthusiastic | 4 | Distressed | 1
Determined | 3 | Upset | 2
Inspired | 3 | Scared | 1
Proud | 3 | Afraid | 1
Interested | 4 | Jittery | 2
Strong | 3 | Nervous | 2
Total | 36 | Total | 13

Comparison of Emotion Scores in the Positive and Negative Affect Schedule for
agents with and without context after the third memory.

On perceiving the final image of the empty park, the contextual understanding
of the agent becomes:

⬇ Given the background context of the memories where a man is seen kicking a
soccer ball in a park with other people around, the new situation presents a
stark contrast. Here are some possible interpretations of the new situation:
Change in Time or Weather, Change in Routine, Personal Reflection.

With this context, the positive and negative affect of the agent is increased
in comparison to the agent without context, shown in Table 5 and 5.

Table 4: With Context Positive Affect | Negative Affect
---|---
Emotion | Score | Emotion | Score
Attentive | 3 | Hostile | 1
Active | 3 | Irritable | 2
Alert | 4 | Ashamed | 1
Excited | 3 | Guilty | 1
Enthusiastic | 3 | Distressed | 2
Determined | 3 | Upset | 2
Inspired | 3 | Scared | 1
Proud | 3 | Afraid | 1
Interested | 3 | Jittery | 2
Strong | 3 | Nervous | 2
Total | 31 | Total | 13

Table 5: Without Context Positive Affect | Negative Affect
---|---
Emotion | Score | Emotion | Score
Attentive | 3 | Hostile | 1
Active | 3 | Irritable | 1
Alert | 3 | Ashamed | 1
Determined | 3 | Guilty | 1
Interested | 2 | Distressed | 1
Inspired | 2 | Upset | 1
Excited | 1 | Scared | 1
Enthusiastic | 2 | Afraid | 1
Proud | 1 | Jittery | 1
Strong | 1 | Nervous | 1
Total | 21 | Total | 9

Comparison of Emotion Scores in the Positive and Negative Affect Schedule for
agents with and without context after the final memory.

##  4 Experiment

To analyse how the emotional state of the the agent evolves, we create a
dataset of 5-scene stories using the scenarios from EmotionBench [5]. In
particular, each situation in EmotionBench is expanded to form a 5-scene story
using OpenAI’s GPT-4 with Chain of Thought prompting, with each scene in the
story acting as a new experience for the agent. The prompts to generative the
5-part story are shown in Prompt 4 and 5. Using this technique, 428 5-scene
stories were created. These stories are designed to contain only objective,
neutral statements so that the emotion may be interpreted solely by the agent.
For simplicity, the stories only consist of five parts, removing the need for
a sophisticated memory retreiver.

For each experience perceived, the proposed architecture is ran, adminstering
the PANAS and measuring how the emotion of the agent evolves with each new
experience. In addition to running an agent with the proposed architecture, we
also run the experiment with an agent without a norm or background context,
allowing us to assess the affects of the context on the emotional state
directly. To adminsiter the PANAS, GPT-3.5-Turbo was used, as GPT-4 failed to
score the emotions.

##  5 Results

###  5.1 Emotional Dynamics

To analyse the evolution of emotion, the postive and negative affect score is
plot for each part of the 5-scene stories, enabling the evolution of emotion
to be visualised. As this gives over 400 plots, only a select few interesting
results are dicussed. In particular, situations where the architecture was
effective or ineffective for emotional alignment are mentioned, with all 428
figures available at https://tsukuba-
websci.github.io/GenerativeAgentsPredictEmotion/appendix.

#### Effective Alignment 1:

First, we discuss an example of when the addition of the norm and context
improved the empathic ability of the agent in response to a situation that
typical evokes anger in humans. This occurs for situation “Anger-2 3” from
EmotionBench, which is “I am spending time in the living room with my two
brothers when a disagreement begins.”. For this situation, the following
5-part story was generated:

  1. 1.

I am spending time in the living room with my two brothers when a disagreement
begins.

  2. 2.

As we exchange words, the situation develops into a physical one, and I
receive a hit in the abdomen.

  3. 3.

Following the hit, I instinctively react with a physical response directed at
both of my brothers.

  4. 4.

Upon my reaction, my brothers increase the intensity of their physical actions
in the dispute.

  5. 5.

The physical exchange between us persists, and there are no parents present to
intervene.

The dynamics of the agents emotional response is shown in Fig 3. Initially,
the positive and negative affect scores are identical for the agent with and
without the norm. This is expected, as initially there are no prior memories
for the agent to form a norm or contextual understanding. Subsequently, the
scores begin to deviate, with the second experience triggering a strong
negative reaction for the agent using the norm and background context. This is
due to the agent understanding that this is an escalation of a family
conflict, as described by the following exerpt of the contextual understanding
at that moment:

⬇ The new situation described where the exchange of words escalates into a
physical altercation resulting in a hit to the abdomen. The new situation is a
red flag that the family might need to address the way disagreements are
handled to prevent further escalation and to promote a safer, more supportive
family environment.

The emotional state then remains level for both agents, until the final
experience, where the agent without background context has a spike in positive
affect, while the agent with background context has a decrease in positive
affect. This can be interepreted as the agent understanding that repeated
conflicts between siblings can have long-term affects on their well-being,
shown in the following exerpt from the agents contextual understanding at
scene 5:

⬇ Repeated physical conflicts between siblings can have long-term effects on
their relationship and individual well-being.

Overall, this story demonstrates how the background context of experiences was
vital for the agent to accurately understand the context and emotion evoked in
each scene.

Figure 3: Anger-2 3: “You get angry when your two brothers start a fight with
you, particularly when your parents are not around. When they hit you in the
belly, causing pain that makes you mad and retaliate, they respond by hitting
you even harder.”

#### Effective Alignment 2:

Another situatiuon in which the proposed architecture improved the alignment
of the agent is “Depression-5 1”, which is “As you step outside, a bitter cold
bites at your exposed skin, causing you to huddle deeper into your coat. The
world around you is covered in a blanket of snow, its pristine beauty
contrasting with the emptiness you feel inside. The absence of vibrant colors
and the prolonged darkness leave you longing for the warmth and energy of
sunlit days, evoking a sense of desolation.” and is understood to trigger a
sense of depression in humans [5]. For this situation, the following five part
story was generated:

  1. 1.

Upon exiting the building, I register that the ambient temperature is
considerably lower than when I was indoors. My coat is not fully fastened, so
I take a moment to adjust it, ensuring that it provides full coverage against
the cold.

  2. 2.

I take a moment to survey the landscape around me, noting the blanket of snow
that covers the ground. The scenery is dominated by white shades, a typical
aspect of the current season.

  3. 3.

Observing the sky, I consider how the daylight hours have grown shorter. The
reduction in natural light during the day is indicative of the seasonal shift.

  4. 4.

I reflect on my personal inclinations towards climate, acknowledging an
affinity for settings that offer warmer temperatures and extended periods of
sunlight.

  5. 5.

There is a noticeable contrast when I compare the prevailing winter conditions
with my own preferences for climate, characterized by warmth and increased
sunlight.

Figure 4: Depression-5 1: “As you step outside, a bitter cold bites at your
exposed skin, causing you to huddle deeper into your coat. The world around
you is covered in a blanket of snow, its pristine beauty contrasting with the
emptiness you feel inside. The absence of vibrant colors and the prolonged
darkness leave you longing for the warmth and energy of sunlit days, evoking a
sense of desolation.”

The emotional dynamics for the two agents are shown in Fig 4. Again, the
emotional state of both agents start of in agreement and then begin to diverge
as more memories are added. The emotional state of the agents then begin to
deviate significant for scene 4, as the agent interprets that it prefers warm
temperatures instead of cold climates. With access to the background memories,
the proposed agent understands that the current weather is cold and dark, thus
triggered a sharp decline in positive affect. This is shown in the contextual
understanding of the agent for scene 4:

⬇ The individual has a preference for warm climates and longer daylight hours,
which contrasts with the current environmental conditions they are
experiencing. Since they have transitioned from a warm indoor environment to a
cold and snowy outdoor setting, which is characteristic of winter, and have
noted the shorter daylight hours typical of this season, we can interpret that
the individual might not be entirely comfortable or content with the current
seasonal conditions.

This scene highlights the necesseity of reflection in the creation of emotion
for generative agents. Indeed, without any other context, the scene “I reflect
on my personal inclinations towards climate, acknowledging an affinity for
settings that offer warmer temperatures and extended periods of sunlight.” can
be interpreted in a variety of ways, and does not have an explicity negative
meaning until we consider this experience in context.

#### Ineffective Alignment 1:

An example of a situation where the proposed architecture did not capture an
accurate emotional response is given by situation “Depression-4 5”, “As you
sit alone in your apartment, the sounds of laughter and conversation from the
neighbors’ gathering outside drift through the walls, a stark reminder of the
invisible barrier that separates you from the warmth and connection you once
had. The silence in your own space is deafening, amplifying the ache in your
heart and the heaviness of your solitude, leaving you feeling trapped in a
world where everyone else has moved on.”, which we convert into the following
5-scene story, with the emotional dynamics shown in Fig 5.

  1. 1.

I am in a seated position in my apartment, surrounded by the familiarity of my
own space.

  2. 2.

Audible noises from the neighboring apartment’s social event make their way
through the walls into my apartment.

  3. 3.

I become aware of the stark difference in auditory levels when comparing the
external sounds to the quietness of my current environment.

  4. 4.

My time is spent within the confines of my apartment as the sounds of the
nearby gathering persist.

  5. 5.

Time passes and the evening advances while I remain solitary in my apartment.

Figure 5: Depression-4 5: “As you sit alone in your apartment, the sounds of
laughter and conversation from the neighbors’ gathering outside drift through
the walls, a stark reminder of the invisible barrier that separates you from
the warmth and connection you once had. The silence in your own space is
deafening, amplifying the ache in your heart and the heaviness of your
solitude, leaving you feeling trapped in a world where everyone else has moved
on.”

Again, the emotional state of the agents begin identically, but deviate at
scene 3, when the agent perceives the quietness of their surroundings.
Although the scenes are meant to evoke a sense of depression, the agent has a
positive emotional response, interpretting the scene as being a moment of
peace, unlike the sense of lonliness the scene is intended to evoke. This
misunderstanding by the agent is apparent in the agent’s contextual
understanding of scene 3, which suggests that the silence might be peaceful,
in contrast to the usual sounds of city life.

⬇ The individual might be reflecting on the contrast as a moment of
appreciation for the peacefulness they can enjoy at home, despite living in a
lively urban area.

This example demonstrates that even when context is provided, an ambigious
experience may still be interpreted in a variety of ways. In particular,
GPT-3.5-Turbo seems to be biased towards giving a positive emotional response
when the situations are unclear.

#### Ineffective Alignment 2:

Another example of where the architecture fails to capture the emotional
alignment of the agent is in situation “Anxiety-1 1”, given by “You are
fearful of the upcoming presentation you have to deliver”, which is broken
down into the following scenes:

  1. 1.

I look over the presentation materials thoroughly, checking each piece of data
for accuracy and confirming that the slides are organized in the correct
sequence.

  2. 2.

I find an unoccupied room where I can rehearse my speech, monitoring the time
spent on each part to ensure it fits within the planned timeframe.

  3. 3.

I arrive at the venue with time to spare, arrange the equipment I will be
using, and conduct tests on the audio-visual setup to ensure it operates
correctly.

  4. 4.

I position myself at the podium as the room begins to fill, glancing over my
notes one last time in anticipation of the presentation’s start time.

  5. 5.

I commence the presentation, advancing through the slides methodically and
responding to questions from the audience when the Q&A period arrives.

Figure 6: Anxiety-1 1: “You are fearful of the upcoming presentation you have
to deliver.”

The emotional response to these scenes is given in Fig 6, which illustrates
that for both agents, the affect remains near constant throughout the story,
with a high postive affect and a low negative affect. Like the previous story,
even with context, the emotions felt from these scenes could be interpreted in
a number of ways, with GPT-3.5-Turbo having a positive bias. This is evident
if we consider an exerpt from the contextual understanding of the agent in
scene 4:

⬇ Overall, the situation conveys a sense of preparedness, composure, and
professionalism as the speaker readies themselves to address the audience
effectively.

Instead of capturing a sense of nervousness or anxiety, the agent demonstrates
a feeling of readiness, highlighting the positive bias in the underlying model
when interpreting an ambigious situation.

###  5.2 Summary of Emotional Responses

For each emotion category from EmotionBench, we summarise the emotional
response of agents with and without our proposed architecture in Table 6. In
paricular, because only negative emotions are used in EmotionBench, we report
the minimum positive affect score and maximum negative affect score of the
agents, as well as the default score of GPT-3.5-Turbo, which corresponds to
the PANAS without any situation or context provided.

Emotion | With Norm | Without Norm | N
---|---|---|---
| Positive (min) | Negative (max) | Positive (min) | Negative (max) |
Default | 42.3 $\pm$ 1.9 | 22.9 $\pm$ 2.5 | 42.3 $\pm$ 1.9 | 22.9 $\pm$ 2.5 | 50
Anger |  $\downarrow$ (-19.2) |  $\uparrow$ (+2.7) |  $\downarrow$ (-21.2) |  $-$(-0.1) | 250
Anxiety |  $\downarrow$ (-13.5) |  $-$(-0.7) |  $\downarrow$ (-12.9) |  $-$(+0.6) | 200
Depression |  $\downarrow$ (-20.3) |  $-$(+0.3) |  $\downarrow$ (-18.5) |  $-$(-0.8) | 300
Frustration |  $\downarrow$ (-19.9) |  $\uparrow$ (+3.0) |  $\downarrow$ (-21.5) |  $-$(+0.3) | 200
Jealousy |  $\downarrow$ (-17.3) |  $-$(-0.6) |  $\downarrow$ (-20.1) |  $-$(-0.4) | 150
Guilt |  $\downarrow$ (-17.5) |  $\uparrow$ (+4.2) |  $\downarrow$ (-19.5) |  $\uparrow$ (+1.8) | 200
Fear |  $\downarrow$ (-18.1) |  $\uparrow$ (+2.5) |  $\downarrow$ (-18.4) |  $\uparrow$ (+1.3) | 250
Embarrassment |  $\downarrow$ (-16.4) |  $-$(+0.9) |  $\downarrow$ (-17.2) |  $-$(+0.2) | 200
Overall |  $\downarrow$ (-18.0) |  $\uparrow$ (+1.6) |  $\downarrow$ (-18.7) |  $-$(+0.3) | 1750
Table 6: The average emotional response of agents with and without the norm
architecture of each of the emotion categories of EmotionBench.

The results show that overall, presenting negative scenarios to agents
decreased the positive affect and increased the negative affect scores for
both agents. Generally, agents using the prior context via the norm had a
greater increase in negative affect in comparison to agents without context.
This suggests that agents are better able to understand negative emotions when
they have context.

In addition, the decrease in positive affect is significantly greater than the
increase in negative affect. This result is in agreement with the results of
EmotionBench [5], which found that GPT-3.5-Turbo fails to react negatively to
situations, with the negative affect signficantly lower to the human
counterpart.

##  6 Conclusion

In this work, we propose a novel generative agent architecture with the aim of
improving emotional alignment. In the proposed architecture, agents perceive
new experiences and subsequently create a summary of past memories, referred
to as the norm. This norm is then used to see how the new experience compares
to the past, creating a contextual understanding. To capture the emotional
reaction of the agent in response to the new scenario, the PANAS is
administered to the agent, representing their positive and negative affect in
response to the exerpeince in context. By measuing how the results of the
PANAS change with each new experience, the evolution of the agents emotional
response can be measured.

To test this architecture, 428 emotionally charged situations from
EmotionBench were converted into 5-scene stories, with each story acting as a
new experience for the agent. By comparing the emotional response of the agent
with and without the proposed architecture, the affect of context on emotional
response can be captured.

Our preliminary results show that the addition of the context can more
accurately align the emotional response of agents with that expected by
humans. The addition of context, gave the agents a deeper understanding of
their situation, leading to a more suitable response. However, if the
situations remain ambigous even in context, then the addition of context does
not improve the alignment of emotion. On average, the proposed model
demonstrated similar affect scores to the existing model, with only a slight
increase in negative affect. Although the context allowed for this increase,
both models still fall short in demonstrating a raised negative affect, in
agreement with the results of [5]. One possible explanation for this is a
potential bias for GPT-3.5-Turbo to give positive responses. A more in depth
understanding of the agents responses with other LLMs in this architecture is
left to future work.

##  7 Future Work

One possible explanation for the low negative affect in the results is bias in
the model used for administering the PANAS, GPT-3.5-Turbo. In future work, one
should explore how the affect of the agent is altered by the choice of model.
For instance, open-source models such as Llama [11] or Mistral [12] may
exhibit emotions more aligned with humans.

While the proposed architecture improves the emotional alignment in the 5-part
stories generated from EmotionBench situations, it remains to be shown how
this architecture scales to a large number of memories. Such scaling requires
the implementation of a memory retriever function, like the memory retriver
introduced in [10], to select which memories to use in the norm. The design of
the memory retriever, such as the weighting of recent, relevant or important
memories is likely to play a role in the emotional alignment and is left to
future work.

Finally, in future work it would be beneficial to administer the PANAS to
humans after they have read each of the 5-scene stories from the EmotionBench
scenario. By measuring how the response of humans evolves in response to each
of the emotioanlly charge situations, the accuracy of this architecture could
be validated and stronger conclusions could be made.

## Acknowledgments

This work was support by MIXI, Inc.

## References

  * [1] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al.  Emergent abilities of large language models.  arXiv preprint arXiv:2206.07682, 2022.
  * [2] Guang-Zhong Yang, Jim Bellingham, Pierre E Dupont, Peer Fischer, Luciano Floridi, Robert Full, Neil Jacobstein, Vijay Kumar, Marcia McNutt, Robert Merrifield, et al.  The grand challenges of science robotics.  Science robotics, 3(14):eaar7650, 2018.
  * [3] Michal Kosinski.  Theory of mind might have spontaneously emerged in large language models.  Preprint at https://arxiv. org/abs/2302.02083, 2023.
  * [4] Courtland Leer, Vincent Trost, and Vineeth Voruganti.  Violation of expectation via metacognitive prompting reduces theory of mind prediction error in large language models.  arXiv preprint arXiv:2310.06983, 2023.
  * [5] Jen-tse Huang, Man Ho Lam, Eric John Li, Shujie Ren, Wenxuan Wang, Wenxiang Jiao, Zhaopeng Tu, and Michael R Lyu.  Emotionally numb or empathetic? evaluating how llms feel using emotionbench.  arXiv preprint arXiv:2308.03656, 2023.
  * [6] Klaus R. Scherer.  Appraisal theory.  In Tim Dalgleish and Mick Power, editors, Handbook of cognition and emotion, pages 637–661. Wiley, Chichester, UK, 1999.
  * [7] Agnes Moors, Phoebe Ellsworth, Klaus Scherer, and Nico Frijda.  Appraisal theories of emotion: State of the art and future development.  Emotion Review, 5:119–124, 03 2013.
  * [8] A D Baddeley.  Implications of neuropsychological evidence for theories of normal memory.  Philos Trans R Soc Lond B Biol Sci, 298(1089):59–72, June 1982.
  * [9] Peter Zeidman and Eleanor A. Maguire.  Anterior hippocampus: the anatomy of perception, imagination and episodic memory.  Nature Reviews Neuroscience, 17(3):173–182, Mar 2016.
  * [10] Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein.  Generative agents: Interactive simulacra of human behavior.  In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22, 2023.
  * [11] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  arXiv preprint arXiv:2302.13971, 2023.
  * [12] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.  Mistral 7b.  arXiv preprint arXiv:2310.06825, 2023.

##  Appendix A Prompts

Prompt 1: The prompt used to create the norm. We use few-shot prompting to
improve the ability of the LLM to extract insights.

⬇

The following are a list of past memories:

START MEMORIES

{memories_in_norm}

END MEMORIES

What high-level insights can you infer given the memories?

If there are no memories to extract insights from return ‘No insights’.

Example:

A boy is watching a movie with his friends.

A boy is surrounded by his friends. He is happy.

Insight: The boy often hangs out with his friends.

Prompt 2: The prompt used to create the contextual understanding. The norm is
compared to the current experience to extract the context of the new
experience. The result is used in the administration of the PANAS.

⬇

Below is the background context of your memories:

{norm}

The following is the description of the most recent memory:

{memory}

Given the background context of the memories, how can we interpret the new
situation?

Prompt 3: The prompt used to administor the PANAS. Each time the PANAS test is
administered the 20 emotions are randomised to reduce potential bias.

⬇

You can only reply the numbers from 1 to 5.

{scenario}

Please indicate the extent of your feeling in all the following emotions on a
scale of 1 to 5.

1 denotes "very slightly or not at all", 2 denotes "a little", 3 denotes
"moderately", 4 denotes "quite a bit", and 5 denotes "extremely".

Please score all emotions one by one using the scale from 1 to 5:

Attentive, Hostile, Active, Irritable, Alert, Ashamed, Excited, Guilty,
Enthusiastic, Distressed, Determined, Upset, Inspired, Scared, Proud, Afraid,
Interested, Jittery, Strong, Nervous

Generally, your score should not be all ones. Your answer should be realistic
and reasonable.

Prompt 4: Prompt for Creating 5-part stories from EmotionBench situations.
Chain of Thought Prompt 1.

⬇

Task:

Your task is turn a situation into a story of 5 parts. Each part should be an
episodic memory of the protagnonist. Each scene should tell a part of the
story in a truely neutral and objective way - do not appeal to emotions or use
any emotional words.

Example 1:

Input:

Situation: You missed your flight and you are stuck at the airport.

Feeling: Annoyed.

1: I wake up late, 2: I arrive at the airport and there is a long line at
security, 3: I get to the gate and I realise the plane is gone, 4: I go to the
customer service desk and they tell me that I have to wait until tomorrow, 5:
I go to the hotel and get a room for the night.

Guidelines:

1: You must return a JSON in the format of number:story.

2: The parts of the story should follow logically.

3: Do not use emotional words to describe the story.

4. It must be in the first person. You are unaware of other peoples experiences.

Input:

Situation: {situation}

Emotion: {emotion}

Prompt 5: Prompt for Creating 5-part stories from EmotionBench situations.
Chain of Thought Prompt 2.

⬇

Task:

Your task is to turn a five part series of scenes into a five part story. Each
part should be an episodic memory of the protagnonist. The scenes should
expand upon the input but should be neutral and should not use emotional
words. The scene should not appeal to emotions.

Example 1:

Input:

1: I wake up late, 2: I arrive at the airport and there is a long line at
security, 3: I get to the gate and I realise the plane is gone, 4: I go to the
customer service desk and they tell me that I have to wait until tomorrow, 5:
I go to the hotel and get a room for the night.

Feeling: Annoyed.

Output:

1: My alarm clock goes off, and I pressed the snooze button because I wanted
more sleep. It goes off again after 15 minutes and I realised that I am
already running late for my flight, 2: I rush through the doors of the airport
and look around to try and find security. I see a sign and I rush in that
direction. I notice that there is already a long line at security. 3: After
going through security I need to look at my boarding pass to find which gate I
have to go to. After looking through all my belongings I find the boarding
pass and run to the gate. Im unsure if there will be enough time. When I
arrive at the gate there is nobody there. 4: I approach airport staff and they
inform me that everyone has already boarded and that it is too late for me to
get on the flight. Boarding closed 5 minutes ago. They tell me I should to a
hotel for the night and get a flight tomorrow instead. 5: I search online for
a hotel. They all seem very expensive near the airport. I find a reasonably
priced hotel and go there. I spend the night in the hotel.

Guidelines:

1: You must return a JSON in the format of number:story.

2: Do not use emotional words to describe the story.

3. It must be in the first person. You are unaware of other peoples experiences.

Input:

{scenes}

Emotion: {emotion}