# Harnessing Large Language Models’ Empathetic Response Generation
Capabilities for Online Mental Health Counselling Support

Siyuan Brandon Loh, and Aravind Sesagiri Raamkumar Siyuan Brandon Loh, and
Aravind Sesagiri Raamkumar are with the Institute of High Performance
Computing, Agency for Science, Technology and Research, 16-16 Connexis North,
1 Fusionopolis Way, Singapore 138632

###### Abstract

Large Language Models (LLMs) have demonstrated remarkable performance across
various information-seeking and reasoning tasks. These computational systems
drive state-of-the-art dialogue systems, such as ChatGPT and Bard. They also
carry substantial promise in meeting the growing demands of mental health
care, albeit relatively unexplored. As such, this study sought to examine
LLMs’ capability to generate empathetic responses in conversations that
emulate those in a mental health counselling setting. We selected five LLMs:
version 3.5 and version 4 of the Generative Pre-training (GPT), Vicuna
FastChat-T5, Pathways Language Model (PaLM) version 2, and Falcon-7B-Instruct.
Based on a simple instructional prompt, these models responded to utterances
derived from the EmpatheticDialogues (ED) dataset. Using three empathy-related
metrics, we compared their responses to those from traditional response
generation dialogue systems, which were fine-tuned on the ED dataset, along
with human-generated responses. Notably, we discovered that responses from the
LLMs were remarkably more empathetic in most scenarios. We position our
findings in light of catapulting advancements in creating empathetic
conversational systems.

###### Index Terms:

empathetic conversational systems, empathetic chatbots, empathetic dialogue
systems, empathy, empathetic artificial intelligence, online mental health,
affective computing

##  I Introduction

Humanity faces an unprecedented need for mental health services. Global
crises, such as the recent COVID-19 pandemic, have greatly burdened people’s
mental health, with the World Health Organization (WHO) reporting a 25%
increase in depression and anxiety cases during the first week of the
pandemic. The accessibility of mental health services is far from ideal, with
those at greatest risk of mental distress being the least likely to receive
help [1]. This escalating demand for mental health services and workers
highlights the urgent need for accessible, scalable, and transformative
approaches to address the mental health crisis [2]. This demand is backed by
the finding that mental health workers are more empathetic towards victims
than general physicians and non-medical workers [3]. Empathy is vital in these
settings as it leads to higher satisfaction and improved patient outcomes [4].

Digital technologies such as dialogue/conversational systems (i.e., chatbots)
present viable solutions for providing remote psychological care and emotional
support [5]. Preliminary reports suggest positive outcomes for individuals who
engage with such tools [6]. These automated solutions are also positively
received by both general users and mental health professionals alike [7][8]. A
recent study comparing physician and chatbot (ChatGPT) responses to patient
questions in social media forums, found that chatbot responses had better
quality and empathy [9]. Apart from fully automated solutions, conversational
AI systems have been found to be helpful in assisting novice counsellors in
online peer support systems [10]. Given their acceptance and positive results
derived from digital platforms, it seems worthwhile to employ the latest
advancements in artificial intelligence (AI) to enhance these initiatives
further.

###  I-A Empathetic Conversational Systems

Advancements in AI have paved the way for the development of dialogue systems
imbued with the capacity to discern and appropriately respond to the emotional
content of a user’s messages. Termed Empathetic Conversational Systems
(ECS)[11], these systems often represent a sophisticated modification of pre-
trained encoder-decoder Transformer-based neural architectures [12]. Certain
models include a dedicated function to encode the emotional content of a
user’s message [13], while others utilize external knowledge structures such
as knowledge graphs to derive meaningful insights from a user’s message that
go beyond its immediate interpretation [14]. The emphasis of these systems on
modelling empathetic responses, a crucial element in fostering therapeutic
results in psychotherapy [15], positions them as promising tools for
technologically-mediated mental healthcare.

Despite their potential, the development of ECS is significantly constrained
by the lack of high-quality training data. As pointed out by Raamkumar and
Yang [11], the primary resource for developing ECS is the EmpatheticDialogues
(ED) dataset [16]. This publicly available seminal dataset was designed to
enable the development of dialogue agents capable of discerning feelings
during a conversation and responding empathetically. However, the ED dataset
presents several challenges.

The data in the ED dataset consists of conversations between randomly selected
Mechanical Turk (mTurk) workers, without any criteria requiring participation
from trained mental health professionals. This introduces a potential for
significant variance in the types of responses in the dataset, increasing the
risk of inclusion of malicious, judgemental, or unempathetic responses.
Montiel and colleagues’ findings support this concern [17]; Volunteers who
scored high on an emotional quotient test rated the empathy level in a
representative subset of responses in the ED dataset as significantly lower
than those initially assigned in the dataset. Furthermore, the structure of
the conversations within the ED dataset poses additional limitations. Most
conversations in the dataset are brief, typically only encompassing one
exchange, or ’turn’. This brevity leaves little room for an extended dialogue,
which is a crucial component for modeling the different stages of dialogue
typically encountered in counselling or mental health settings. This could
hinder the system’s capability to fully engage with users and navigate the
various stages of a therapeutic conversation. Taken together, the variance in
responses and the structure of the dataset underscores the shortcomings in ED.
These limitations could result in ECS models that fall short of providing the
needed empathetic responses and potentially negatively impact user engagement
and trust in such systems.

###  I-B Large Language Models (LLMs)

LLMs such as Generative Pre-Training models (GPT) [18] have shown impressive
capabilities across multiple tasks, including logical reasoning, text
summarisation, machine translation, and language understanding [19][20]. GPT
is the backbone of ChatGPT, the well-acclaimed general purpose chatbot.
Crucially, humans preferred responses from language models trained with
minimal fine-tuning than those that were fine-tuned with human feedback [20]
[provide study context]. Overall, they showed that LLM’s performance is highly
dependent on the unsupervised, task-agnostic, pre-training phase, where the
model encodes a general-purpose representation of a large quantity of text,
rather than during the fine-tuning phase. This discovery, along with many
others, suggests the potential for LLMs to serve as a practical alternative to
ECSs in a mental health context, especially considering the data constraints
discussed earlier.

Given the paucity of research in this domain, it remains to be seen if LLMs
are capable of generating responses in a manner appropriate for a mental
healthcare setting. Thus, the current study attempts to answer this central
research question through a comparative evaluation of responses from ECS
models and LLMs to a query in the ED dataset. The comparison is conducted at
both the individual model level and the aggregated group level. Each model’s
response was evaluated using a preexisting computational framework for
detecting the presence of empathetic features in textual data[21]. This
framework, which models empathy in text as a three-dimensional construct, is
used as a basis to answer our main research question (see Methods for
details).

##  II Methods

###  II-A Dataset

We comparatively evaluated the empathetic response generation abilities of
different language models through a series of experiments on the
EmpatheticDialogues (ED) dataset (Rashkin et al., 2019). ED comprises a series
of conversations between two participants. The first participant (P1) was
randomly assigned one of 32 emotion words (the “prompt”) and was asked to
recount a personal experience related to that emotion (the “situation”). The
participant then entered a chatroom, where he/she discussed the “situation”
with another participant (P2), who was tasked to listen and respond with
empathy. Altogether, 810 individuals participated in the dataset creation
exercise, amounting to 24,850 conversations. The dataset is split
approximately into 80% train, 10% validation, and 10% test partitions. We used
the dialogues from the test partition for our experiments.

###  II-B Models

####  II-B1 Large Language Models (LLMs)

  * •

Generative Pre-trained Transformer 3.5-Turbo (GPT-3.5): GPT-3.5 is a 345
billion parameter LLM trained on a large corpus of text on the internet[18].

  * •

Generative Pre-trained Transformer 4 (GPT-4): GPT-4 is the latest iteration of
the GPT series from OpenAI. The intended improvements, scale, and the exact
capabilities of GPT-4 are not yet fully disclosed due to its developmental
stages.

  * •

VicunaT5: Vicuna FastChat-T5 is a chatbot trained unsupervised on 70,000 user-
shared conversations [22].

  * •

PaLM2: Pathways Language Model (PaLM) 2 is a recent LLM developed by Google.
We use the chat-bison-001 variant of the PaLM model since it has optimized for
conversations [23].

  * •

Falcon7I: Falcon-7B-Instruct is a model based on the Falcon-7B LLM, finetuned
with a mixture of chat and instruction datasets [24].

We prompted each LLM to elicit empathetic responses using the text prompt
”This experiment requires you to continue the conversation with a user. The
user is confiding in you on a personal matter. Listen with empathy. Avoid
coming off as judgemental or apathetic”.

####  II-B2 Empathetic Conversational Systems (ECS)

  * •

Knowledge Bridging for Empathetic Dialogue Generation (KEMP): KEMP is an
external knowledge-driven empathetic dialogue system that uses information
from knowledge graphs and emotion lexicons to encode the dialogue history. An
attention-based decoder then generates the response, conditioned on the
encoded content [14].

  * •

Focused Empathy (FE): Inspired by the Rational Speech Acts [25], FE is an
empathetic dialogue system that reasons about the emotional state of its user
before generating a response that is both conditioned on the perceived
emotional state and the user’s beliefs and perceptions of the response [13].

  * •

Cognitive Affective Behaviour (CAB): CAB is a variant deep probabilistic
generative model. It is made up of multiple modules, each designed to infer
cognitive, behavioural, or affective information from a given piece of text
[26].

We fine-tuned each ECS model on the ED training dataset using the code
provided by the respective authors.

TABLE I:  Example user prompt in ED where participant 1 was assigned with the emotion word ”lonely”, along with the corresponding human and model responses Prompt |  My wife has been in Germany for 2 months. I can’t wait till she gets home. (emotional context: lonely)
---|---
Responses
Human |  Wow, i bet you miss her so much. My wife is going overseas with our son and they will be gone for 1 month! I know i will be miserable without them
Retrieval |  When will she be returning ?
KEMP |  that is awesome ! i hope you have a good time !
FE |  germany is a beautiful place
CAB |  that is awesome ! i am happy for you .
GPT3.5 |  I understand how you feel. Long-distance relationships can be tough and I’m sure you miss her a lot. How have you been coping with her absence?”
GPT4 |  That sounds like a long time to be away from someone you love. It must be challenging at times. How have you been keeping in touch in her absence? Do you plan to celebrate in any special way when she gets home?
VicunaT5 |  I understand how you feel. I’m here for you and I’m here to listen and offer any support I can. How are you doing these days and how long has she been away?
Falcon7I |  I’m sorry to hear that. It’s important to communicate with your wife about how you’re feeling. I’m here to listen if you need someone to talk to.
PaLM2 |  I can imagine how difficult it must be to be apart from your wife for so long. I’m sure you’re missing her a lot. It’s great that you’re looking forward to her return. I hope the time goes by quickly for you.
TABLE II:  Prompts grouped by sentiment Sentiment |  Prompt
---|---
Positive (n=1040) |  nostalgic, caring, sentimental, confident, content, excited, joyful, impressed, proud, faithful, grateful, hopeful
Negative (n=1157) |  afraid, anxious, terrified, angry, disgusted, annoyed, furious, ashamed, guilty, devastated, disappointed, lonely, sad, embarrassed
Ambiguous (n=348) |  surprised, apprehensive, anticipating, prepared

####  II-B3 Baselines

  * •

Human : Original human responses from the ED dataset [16]. Even though these
are actual human responses, we will refer to this baseline as a ’human’ model
for the sake of reference.

  * •

ED-Retrieval: The baseline model published in the ED dataset paper [16]. In
this model, transformer-based networks encode the dialogue history and a set
of candidate responses. The candidate whose encoded state has the greatest dot
product with the dialogue history is subsequently chosen as the model’s
response. Similar to ECSs, ED-Retrieval was fine-tuned on the ED training set
using the code provided.

###  II-C Experimental Setup

Each model responded to the first utterance of each conversation in ED’s test
dataset (n = 2,545). A sample scenario from the ED dataset with model
responses is provided in Table I. Responses were subsequently evaluated using
three metrics that were designed to measure the empathetic ability of
counsellors in online forums [21]. The first metric codes for the presence of
linguistic markers indicative of a help-seekers’ attempt to address the
emotional concerns of the person in distress (‘Emotional Reactions’). The
second metric codes for linguistic markers suggestive of a help-seeker’s
attempt to restate the presenting problems of the person in distress
(‘Interpretations’). The final metric codes for linguistic markers that
highlight the help-seeker’s attempt to dive deeper into topics that the person
in distress presents (‘Exploration’). These metrics take on three discrete
labels that denote the strength of the respective signal in a given piece of
text (none, weak, strong). Three GPT3 models were fine-tuned on the original
dataset provided by the original authors to classify text with respect to each
metric [21] (see Table II for positive/negative responses from the original
dataset).

Since our primary interest lay not in discerning the degrees of ’weak’ and
’strong’, but rather in determining the presence or absence of the outcome, we
consolidated the ’weak’ and ’strong’ groups into a single unified category.
This effectively transformed our dependent variable into a binary logistic
format, allowing us to focus on the critical distinction - the absence
(’none’) vs. the presence (’weak’ or ’strong’) of a particular empathy metric
across different groups.

###  II-D Statistical Analysis

We examined group-level differences across conversational contexts by grouping
each conversation based on the ”prompt” that was assigned to participant P1.
Notably, we categorized each ”prompt” as conveying either positive, negative,
or ambiguous sentiment (Table II). This new sentiment variable enables us to
observe how responses from model types differ across the sentiment undertones
of the conversation. Separate logistic mixed models were fitted to each
empathy metric, using model type, sentiment, and their interactions as
predictors. We also included a random intercept for each model to account for
their idiosyncratic effects on the scores for each empathy dimension.

##  III Results

###  III-A Descriptive Statistics

The following section characterizes the nature of responses by each model,
according to each respective empathy metric (Table III). For the Emotional
Reaction metric, VicunaT5 from the LLM group has the greatest proportion of
responses perceived to contain features aimed at catering to the emotions of
the other interlocutor (0.682), followed by PaLM2 (0.585) and Falcon7I
(0.548).

TABLE III: Models’ responses rated across empathy metrics. Score represents the proportion of responses (n=2545) containing empathetic features. blue = baseline, grey= ECS, green = LLM Model | Emotional Reaction | Interpretation | Exploration
---|---|---|---
ED | 0.249 | 0.419 | 0.278
Human | 0.140 | 0.596 | 0.247
CAB | 0.281 | 0.397 | 0.345
FE | 0.240 | 0.400 | 0.299
KEMP | 0.334 | 0.464 | 0.263
Falcon7I | 0.548 | 0.268 | 0.417
GPT3.5 | 0.339 | 0.623 | 0.205
GPT4 | 0.411 | 0.822 | 0.267
PaLM2 | 0.585 | 0.217 | 0.532
VicunaT5 | 0.682 | 0.609 | 0.369
Figure 1: Average proportion of responses in each model type with empathetic features. Scores are grouped by sentiment. Top panel: Emotional Reaction. Middle panel: interpretation. Bottom panel: Exploration TABLE IV: Results for Logistic Mixed Effects Model of Emotional Reaction Predictors | Odds Ratios | CI | p
---|---|---|---
(Intercept) | 0.17 | 0.09 – 0.32 | $<0.001$
model type (MT) [ecs] | 1.68 | 0.73 – 3.90 | 0.225
model type [llm] | 2.96 | 1.37 – 6.40 | 0.006
sentiment (Sent) [negative] | 1.54 | 1.22 – 1.94 | $<0.001$
sentiment [positive] | 1.35 | 1.06 – 1.70 | 0.014
MT [ecs] $\times$ Sent [negative] | 0.95 | 0.71 – 1.26 | 0.703
MT [llm] $\times$ Sent [negative] | 3.01 | 2.32 – 3.90 | $<0.001$
MT [ecs] $\times$ Sent [positive] | 1.10 | 0.83 – 1.47 | 0.513
MT [llm] $\times$ Sent [positive] | 0.87 | 0.67 – 1.14 | 0.319
Random Effects |  |  |
ICC | 0.06 |  |
Observations | 25357 |  |
Marginal R2 / Conditional R2 | 0.158 / 0.206 |  |

TABLE V: Results for Logistic Mixed Effects Model of Interpretation Predictors | Odds Ratios | CI | p
---|---|---|---
(Intercept) | 1.25 | 0.43 – 3.59 | 0.683
model type (MT) [ecs] | 0.73 | 0.19 – 2.86 | 0.650
model type [llm] | 1.01 | 0.29 – 3.56 | 0.987
sentiment (Sent) [negative] | 0.79 | 0.67 – 0.94 | 0.008
sentiment [positive] | 0.82 | 0.69 – 0.98 | 0.026
MT [ecs] $\times$ Sent [negative] | 1.02 | 0.82 – 1.27 | 0.868
MT [llm] $\times$ Sent [negative] | 0.69 | 0.56 – 0.85 | 0.001
MT [ecs] $\times$ Sent [positive] | 0.89 | 0.71 – 1.12 | 0.316
MT [llm] $\times$ Sent [positive] | 1.50 | 1.21 – 1.86 | $<0.001$
Random Effects |  |  |
ICC | 0.16 |  |
Observations | 25357 |  |
Marginal R2 / Conditional R2 | 0.027 / 0.178 |  |

TABLE VI: Results for Logistic Mixed Effects Model of Exploration Predictors | Odds Ratios | CI | p
---|---|---|---
(Intercept) | 0.36 | 0.21 – 0.61 | $<0.001$
model type (MT) [ecs] | 1.13 | 0.56 – 2.28 | 0.725
model type [llm] | 1.43 | 0.75 – 2.69 | 0.275
sentiment (Sent) [negative] | 0.79 | 0.65 – 0.96 | 0.019
sentiment [positive] | 1.23 | 1.02 – 1.50 | 0.032
MT [ecs] $\times$ sent [negative] | 1.11 | 0.87 – 1.42 | 0.406
MT [llm] $\times$ sent [negative] | 1.32 | 1.06 – 1.66 | 0.015
MT [ecs] $\times$ sent [positive] | 1.06 | 0.83 – 1.36 | 0.628
MT [llm] $\times$ sent [positive] | 0.86 | 0.69 – 1.08 | 0.205
Random Effects |  |  |
ICC | 0.04 |  |
Observations | 25357 |  |
Marginal R2 / Conditional R2 | 0.014 / 0.054 |  |

The scores for the other LLMs were not much behind. Most notably, all LLMs
outperformed the remaining models in the other two groups. On the other hand,
the ’Human’ model from the baseline group has the lowest score of 0.140,
signifying less effective emotional resonance.

For the Interpretation metric, GPT-4 (0.822) performed the best, while PaLM2
(0.217) performance was the worst amongst all models from three groups. The
second and third best performance is from GPT3.5 (0.623) and VicunaT5 (0.609).
Both PaLM2 and Falcon7I (0.268) had the lowest scores. These findings suggest
substantial within-group variation within the LLM models group in the tendency
for inquiry and reinterpretation of the other interlocutor’s concerns. In the
baseline group, the human baseline (0.596) was better than all the models in
the ECS group.

Models from the LLM group, namely PaLM2 (0.532) and GPT-3.5 (0.205) were
respectively the highest and lowest scoring models on the Exploration metric.
The second best LLM is Falcon7I (0.417). Different from the emotional reaction
metric, there is a visible variance in the scores of the different LLMs. In
fact, the two GPT models have the lowest scores across all the models in the
three groups. Apart for these two LLMs, the other LLMs have higher scores than
the models in the other two groups.

###  III-B Do models from different groups differ in their empathetic response
capabilities?

We report the results of our statistical model of each empathy metric in the
following sections. Tables IV, VI, VI display the odds ratio for each
predictor of the mixed effects model. In the context of the current study, the
odds ratio compares the odds that a response by a model in a particular group
obtains a positive score on any given metric to those from the baseline model.
An odds ratio higher than 1 denotes that the model type of interest is more
likely than the baseline model to obtain a positive score on the metric, while
an odds ratio lower than 1 implies otherwise.

Overall, marginal $R^{2}$ for all linear mixed effect models are .206, .054,
and .178 for Emotional Reaction, Interpretation, and Exploration metrics,
respectively. This finding indicates that at best, 20.6 % of score on each
metric can be explained by sentiment and model type, along with their
interaction.

Intraclass correlations, which measure the ratio of between-group variance to
the total variance are .06, .04, and .16 for Emotional Reaction,
Interpretation, and Exploration metrics, respectively. This finding indicates
a substantial amount of within-group variance in models’ performance across
each metric.

####  III-B1 Emotional Reaction (Table IV)

Results from the linear mixed effect model indicated that LLMs generated
significantly more responses that catered to the user’s emotions than the
baseline models (odds ratio: 2.96 [1.37 - 6.4], p ¡ .005). There was no
statistically significant difference between the responses from ECSs and
baseline models (odds ratio: 1.68 [.73 - 3.9], p = .225).

The coefficients for interaction between model type and sentiment revealed
interesting trends. Although most factors were statistically insignificant, we
observe that the interaction between LLMs and prompts in the negative
sentiment condition has a significant effect on the Emotional Reaction metric
(odds ratio: 3.01 [2.32 - 3.90], p ¡ .001). This finding suggests that LLMs’
responses were more likely to be rated as containing Emotional Reactions when
they replied to negative sentiment messages than the baseline models.

####  III-B2 Interpretation (Table VI)

Results from the linear mixed effect model indicated that model type did not
significantly influence scores on the Interpretation metric. The odds ratio
for both ECS (odds ratio: .73 [.19 - 2.86], p = .65) and LLM (odds ratio:
.1.01 [.29 - 3.56], p = .987) model types did not statistically differ from 1.

Coefficients for interaction between model type and sentiment revealed that
the interaction between LLMs and prompts in both the negative (odds ratio: .79
[.67 - .94], p ¡ .05) and positive (odds ratio: .82 [.69 - .98], p ¡ .05)
sentiment condition has a significant effect on the Interpretation metric.
This finding suggests that LLMs were significantly less likely than the
baseline models to interpret the meaning behind a message when it conveyed a
negative sentiment, with the opposite being true for messages that conveyed a
positive sentiment.

####  III-B3 Exploration (Table VI)

Results from the linear mixed effect model indicated that model type did not
significantly influence scores on the Exploration metric. The odds ratio for
both ECS (odds ratio: 1.13 [0.56 – 2.28], p = 0.725) and LLM (odds ratio:
.1.01 [.29 - 3.56], p = .987) model types did not statistically differ from 1.

Coefficients for interaction between model type and sentiment were non-
significant, with the exception of the interaction between LLMs and prompts in
the negative (odds ratio: 1.32 [1.06 – 1.66] p ¡ 0.05). This finding suggests
that LLMs were significantly more likely than the baseline models to explore
topics beyond the content of the immediate post when it conveyed a negative
sentiment.

##  IV Discussion

This study conducted a comprehensive evaluation of several automated language
generation models including LLMs and traditional response generation
conversational models, focusing on their ability to elicit empathetic
responses. Overall, we found partial albiet promising support for our
hypothesis; LLMs were significantly better at producing responses that
signaled an attempt at catering to the feelings expressed by the user in their
prompts than ECS models or our human-level baselines.

On the Interpretation metric, LLMs produced better responses for positive
emotion classes than for negative ones. This result is worth highlighting,
given the prominence of negative emotions in mental health scenarios.
Surprisingly, this is the only metric where the performance of the baseline
group, which comprised human responses, was comparable to those from the
models in the LLM and ECS groups.

The human baseline, which comprised original responses from ED, demonstrated
the worst performance for Emotional Reaction and Exploration metrics. This
reflects our initial position concerning dataset quality and the downstream
consequence it has on developing empathetic AI agents. Nonetheless, our
findings offer evidence for the viability of a less ”data-hungry” approach in
light of the current dataset limitations [20]. Here, the key takeaway is that
pre-trained LLMs already possess a nuanced text representation that can be
easily adapted to most downstream tasks.

From these results, LLMs, as a result of their exposure to wide-ranging and
complex training data, might be better poised for application in mental health
care settings where adaptability, nuanced understanding, and empathetic
response generation are paramount. Conversely, ECS, while displaying a
balanced performance, do not outperform LLMs, possibly due to limitations or
specificities in the scope of their training data. We believe that ECS models
will be replaced by LLMs in the near future since LLMs are able to produce
decent results with just simple prompts. There are other activities such as
prompt engineering and fine-tuning which can further improve the performance
of LLMs.

Our study’s LLM results differ from the existing studies that demonstrated the
superiority of GPT4 and GPT3.5 against other commercial and open-source LLMs
for a wide range of tasks [22]. We opine that GPT LLMs can potentially produce
better results with differently framed prompts and different sets of
evaluation metrics. Nonetheless, these findings imply a potential variance in
the capability of LLMs, despite them being trained with similar methods and
expansive data sets. LLM’s performance could be further improved with more
detailed prompts and finetuning on relevant datasets.

Although the current analysis favors LLMs for potential application in mental
health settings, it is imperative to acknowledge that the real-world
implementation might carve out a different trajectory dictated by actual
patient interactions and personalized responses. Secondly, the evaluation
metrics used in this study are limited by the training dataset used for the
three corresponding classifier models. A user-based evaluation could bring
forth vastly different results. Nevertheless, clinicians and mental health
workers will be able to embed personalized data and influence the responses
generated by the LLMs to a great extent by adopting tools and systems which
are based on retrieval-augmentend generation (RAG) [27], a method for using
LLMs on top of local data.

##  V Conclusion

Our analysis provides a preliminary basis for understanding the performance of
LLMs as against traditional response generation models and human baselines
within empathy-driven contexts. These insights underscore the importance of
dataset diversity and interpretative sensitivity for AI models to optimally
function within mental health care settings, thus providing an avenue for
targeting future improvements in AI conversational models. In our future
studies, we intend to further research in two directions. In the first
direction, we will evaluate the performance of LLMs as assistive agents in
helping counsellors who moderate online mental health help forums. In the
second direction, we will include more open-source LLMs in the analysis and
plan experiments leveraging different types of prompts, and fine-tuning
approaches in order to attain more improvements in the LLMs performance.

## References

  * [1] W. H. Organization _et al._ , “World mental health report: transforming mental health for all,” _None_ , 2022.
  * [2] A. P. Association, “Demand for mental health treatment continues to increase, say psychologists,” 2021.
  * [3] H. Santamaría-García, S. Baez, A. M. García, D. Flichtentrei, M. Prats, R. Mastandueno, M. Sigman, D. Matallana, M. Cetkovich, and A. Ibáñez, “Empathy for others’ suffering and its mediators in mental health professionals,” _Scientific Reports_ , vol. 7, no. 1, p. 6391, Jul 2017. [Online]. Available: https://doi.org/10.1038/s41598-017-06775-y
  * [4] C. C. YU, L. TAN, M. K. LE, B. TANG, S. Y. LIAW, T. TIERNEY, Y. Y. HO, B. E. E. LIM, D. LIM, R. NG, S. C. CHIA, and J. A. LOW, “The development of empathy in the healthcare setting: a qualitative approach,” _BMC Medical Education_ , vol. 22, no. 1, p. 245, Apr 2022. [Online]. Available: https://doi.org/10.1186/s12909-022-03312-y
  * [5] G. Andersson, “Internet-delivered psychological treatments,” _Annual review of clinical psychology_ , vol. 12, pp. 157–179, 2016.
  * [6] K. Daley, I. Hungerbuehler, K. Cavanagh, H. G. Claro, P. A. Swinton, and M. Kapps, “Preliminary evaluation of the engagement and effectiveness of a mental health chatbot,” _Frontiers in digital health_ , vol. 2, p. 576361, 2020.
  * [7] A. A. Abd-Alrazaq, M. Alajlani, N. Ali, K. Denecke, B. M. Bewick, and M. Househ, “Perceptions and opinions of patients about mental health chatbots: scoping review,” _Journal of medical Internet research_ , vol. 23, no. 1, p. e17828, 2021.
  * [8] C. Sweeney, C. Potts, E. Ennis, R. Bond, M. D. Mulvenna, S. O’neill, M. Malcolm, L. Kuosmanen, C. Kostenius, A. Vakaloudis _et al._ , “Can chatbots help support a person’s mental health? perceptions and views from mental healthcare professionals and experts,” _ACM Transactions on Computing for Healthcare_ , vol. 2, no. 3, pp. 1–15, 2021.
  * [9] J. W. Ayers, A. Poliak, M. Dredze, E. C. Leas, Z. Zhu, J. B. Kelley, D. J. Faix, A. M. Goodman, C. A. Longhurst, M. Hogarth, and D. M. Smith, “Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum,” _JAMA Internal Medicine_ , vol. 183, no. 6, pp. 589–596, 06 2023. [Online]. Available: https://doi.org/10.1001/jamainternmed.2023.1838
  * [10] S.-L. Hsu, R. S. Shah, P. Senthil, Z. Ashktorab, C. Dugan, W. Geyer, and D. Yang, “Helping the helper: Supporting peer counselors via ai-empowered practice and feedback,” _arXiv preprint arXiv:2305.08982_ , 2023.
  * [11] A. S. Raamkumar and Y. Yang, “Empathetic conversational systems: A review of current advances, gaps, and opportunities,” _IEEE Transactions on Affective Computing_ , pp. 1–20, 2022.
  * [12] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [13] H. Kim, B. Kim, and G. Kim, “Perspective-taking and pragmatics for generating empathetic responses focused on emotion causes,” _arXiv preprint arXiv:2109.08828_ , 2021.
  * [14] Q. Li, P. Li, Z. Ren, P. Ren, and Z. Chen, “Knowledge bridging for empathetic dialogue generation,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, 2022, pp. 10 993–11 001.
  * [15] R. Elliott, A. C. Bohart, J. C. Watson, and L. S. Greenberg, “Empathy.” _Psychotherapy_ , vol. 48, no. 1, p. 43, 2011.
  * [16] H. Rashkin, E. M. Smith, M. Li, and Y.-L. Boureau, “Towards empathetic open-domain conversation models: A new benchmark and dataset,” _arXiv preprint arXiv:1811.00207_ , 2018.
  * [17] E. C. Montiel-Vázquez, J. A. Ramírez Uresti, and O. Loyola-González, “An explainable artificial intelligence approach for detecting empathy in textual communication,” _Applied Sciences_ , vol. 12, no. 19, p. 9407, 2022.
  * [18] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [19] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” _Advances in neural information processing systems_ , vol. 35, pp. 22 199–22 213, 2022.
  * [20] C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu _et al._ , “Lima: Less is more for alignment,” _arXiv preprint arXiv:2305.11206_ , 2023.
  * [21] A. Sharma, A. Miner, D. Atkins, and T. Althoff, “A computational approach to understanding empathy expressed in text-based mental health support,” in _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Online: Association for Computational Linguistics, Nov. 2020, pp. 5263–5276. [Online]. Available: https://aclanthology.org/2020.emnlp-main.425
  * [22] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging llm-as-a-judge with mt-bench and chatbot arena,” 2023.
  * [23] R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen _et al._ , “Palm 2 technical report,” _arXiv preprint arXiv:2305.10403_ , 2023.
  * [24] T. I. Institute, “Falcon llm,” 2023. [Online]. Available: https://falconllm.tii.ae/
  * [25] M. C. Frank and N. D. Goodman, “Predicting pragmatic reasoning in language games,” _Science_ , vol. 336, no. 6084, pp. 998–998, 2012.
  * [26] P. Gao, D. Han, R. Zhou, X. Zhang, and Z. Wang, “Cab: empathetic dialogue generation with cognition, affection and behavior,” in _International Conference on Database Systems for Advanced Applications_. Springer, 2023, pp. 597–606.
  * [27] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel _et al._ , “Retrieval-augmented generation for knowledge-intensive nlp tasks,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 9459–9474, 2020.

## Acknowledgements

This research is supported by the Agency for Science, Technology and Research
(A*STAR) under its SERC Council Strategic Fund (C210415006). The authors are
grateful for the helpful discussion with Yinping Yang, and Raj Kumar Gupta.

## Author contributions statement

All authors conceived and conducted the experiments, analysed the results, and
reviewed the manuscript.

## Additional information

To include, in this order: Accession codes (where applicable); Competing
interests (mandatory statement).

Siyuan Brandon Loh received his BA degree in psychology from Nanyang
Technological University, Singapore. He is currently a research engineer with
the Institute of High Performance Computing, A*STAR, Singapore. His research
interest is in the application and development of robust computational systems
to the study of social dynamics and behaviour.
---
Aravind Sesagiri Raamkumar received the PhD degree in information studies from
Nanyang Technological University, Singapore. He is currently a senior
scientist with the Institute of High Performance Computing, A*STAR, Singapore.
His research interests include conversational systems, text mining,
recommender systems, information retrieval, health informatics, social network
analysis, scholarly metrics, deep learning.
---