# Foundation Metrics: Quantifying Effectiveness of Healthcare Conversations
powered by Generative AI

Mahyar Abbasian, M.Sc.1,5*, Elahe Khatibi, M.Sc.1,5*, Iman Azimi, Ph.D.1,5,
David Oniani, B.A.3,5, Zahra Shakeri Hossein Abad, Ph.D.4,5, Alexander Thieme,
MD2, Zhongqi Yang, M.Sc.1, Yanshan Wang, Ph.D.3,5, Bryant Lin, MD2,5, Olivier
Gevaert, Ph.D.2,
Li-Jia Li, Ph.D.5, Ramesh Jain, Ph.D.1,5, and Amir M. Rahmani, Ph.D.1,5

1University of California, Irvine
2Stanford University
3University of Pittsburgh
4University of Toronto
5HealthUnity
*Equal Contribution and Corresponding authors<EMAIL_ADDRESS>
###### Abstract

Generative Artificial Intelligence is set to revolutionize healthcare delivery
by transforming traditional patient care into a more personalized, efficient,
and proactive process. Chatbots, serving as interactive conversational models,
will probably drive this patient-centered transformation in healthcare.
Through the provision of various services, including diagnosis, personalized
lifestyle recommendations, dynamic scheduling of follow-ups, and mental health
support, the objective is to substantially augment patient health outcomes,
all the while mitigating the workload burden on healthcare providers. The
life-critical nature of healthcare applications necessitates establishing a
unified and comprehensive set of evaluation metrics for conversational models.
Existing evaluation metrics proposed for various generic large language models
(LLMs) demonstrate a lack of comprehension regarding medical and health
concepts and their significance in promoting patients’ well-being. Moreover,
these metrics neglect pivotal user-centered aspects, including trust-building,
ethics, personalization, empathy, user comprehension, and emotional support.
The purpose of this paper is to explore state-of-the-art LLM-based evaluation
metrics that are specifically applicable to the assessment of interactive
conversational models in healthcare. Subsequently, we present an comprehensive
set of evaluation metrics designed to thoroughly assess the performance of
healthcare chatbots from an end-user perspective. These metrics encompass an
evaluation of language processing abilities, impact on real-world clinical
tasks, and effectiveness in user-interactive conversations. Finally, we engage
in a discussion concerning the challenges associated with defining and
implementing these metrics, with particular emphasis on confounding factors
such as the target audience, evaluation methods, and prompt techniques
involved in the evaluation process.

††journal:

a

## 1 Introduction

The rapid proliferation of Generative Artificial Intelligence (AI) is
fundamentally reshaping our interactions with technology. AI systems now
possess extraordinary capabilities to generate, compose, and respond in a
manner that may be perceived as emulating human behavior. Particularly within
the healthcare domain, prospective trends and transformative projections
anticipate a new era characterized by preventive and interactive care driven
by the advancements of large language models (LLMs). Interactive
conversational models, commonly known as chatbots, hold considerable potential
to assist individuals, including patients and healthcare providers, in a wide
array of tasks such as symptom assessment, primary medical and health
education, mental health support, lifestyle coaching, appointment scheduling,
medication reminders, patient triaging, and allocating health resources.

Due to the life-critical nature of healthcare applications, using
conversational models necessitates establishing a unified and comprehensive
set of foundation metrics [1] that enable a meticulous evaluation of the
models’ performance, capabilities, identification of potential errors, and
implementation of effective feedback mechanisms. These metrics can lead to
significant advances in the delivery of robust, accurate, and reliable
healthcare services. However, the existing evaluation metrics introduced and
employed for assessing healthcare chatbots [2, 3, 4] exhibit two significant
gaps that warrant careful attention.

Firstly, it is observed that numerous existing generic metrics [5, 6, 7]
suffer from a lack of unified and standard definition and consensus regarding
their appropriateness for evaluating healthcare chatbots. Currently, state-of-
the-art conversational models are predominantly assessed and compared based on
language-specific perspectives [8] and surface-form similarity [8] using
intrinsic metrics such as Bilingual Evaluation Understudy (BLEU) [9] and
Recall-oriented Understudy for Gisting Evaluation (ROUGE) [5]. Although these
metrics are model-based, they lack an understanding of medical concepts (e.g.,
symptoms, diagnostic tests, diagnoses, and treatments), their interplay, and
the priority for the well-being of the patient, all of which are crucial for
medical decision-making [10]. For this reason, they inadequately capture vital
aspects like semantic nuances, contextual relevance, long-range dependencies,
changes in critical semantic ordering, and human-centric perspectives [11],
thereby limiting their effectiveness in evaluating healthcare chatbots.
Moreover, specific extrinsic context-aware evaluation methods have been
introduced to incorporate human judgment in chatbot assessment [9, 12, 13, 7,
14, 15, 16]. However, these methods have merely concentrated on specific
aspects, such as robustness of the generated answers within a particular
medical domain.

Secondly, it is evident that the existing evaluation metrics overlook a wide
range of crucial user-centered aspects that indicates the extent to which a
chatbot establishes a connection and conveys support and emotion to the
patient. Emotional bonds play a vital role in physician-patient
communications, but they are often ignored during the development and
evaluation of chatbots. Healthcare chatbot assessment should consider the
level of attentiveness, thoughtfulness, emotional understanding, trust-
building, behavioral responsiveness, user comprehension, and the level of
satisfaction or dissatisfaction experienced. There is a pressing need to
evaluate the ethical implications of chatbots, including factors such as
fairness and biases stemming from overfitting [17]. Furthermore, the current
methods fail to address the issue of hallucination, wherein chatbots generate
misleading or inaccurate information. In particular, in the healthcare domain,
where safety and currentness of information are paramount, hallucinations pose
a significant concern. The evaluation of healthcare chatbots should encompass
not only their ability to provide personalized responses to individual users
but also their ability to offer accurate and reliable information that applies
to a broader user base. Striking the right balance between personalization and
generalization is crucial to ensure practical and trustworthy healthcare
guidance. Additionally, metrics are required to assess the chatbot’s ability
to deliver empathetic and supportive responses during healthcare interactions,
reflecting its capacity to provide compassionate care. Moreover, existing
evaluations overlook performance aspects of models, such as computational
efficiency and model size, which are crucial for practical implementation.

In this article, we begin by delving into an examination of the current state-
of-the-art evaluation metrics applicable to assessing healthcare chatbots.
Subsequently, we present an exhaustive collection of user-centered evaluation
metrics and their taxonomy to provide a thorough and well-rounded
comprehension of a healthcare chatbot’s performance across diverse dimensions.
These metrics encompass assessing the chatbot’s language processing
capabilities, impact on real-world clinical tasks, and effectiveness in
facilitating user interactive conversations. Furthermore, we discuss the
challenges associated with defining and implementing these metrics,
emphasizing factors such as the target audience, evaluation methods, and
prompt techniques integral to this process. In fact, we bring about the
requirement for a comprehensive evaluation framework.

## 2 Review of Existing Evaluation Metrics for LLMs

This section critically examines metrics and quantitative measurements
proposed in the literature for evaluating language models, including LLMs. The
evaluation of language models can be categorized into intrinsic and extrinsic
[18]. These methods can be executed automatically or manually. We begin by
presenting an overview of the prevailing state-of-the-art intrinsic evaluation
methodologies, which primarily focus on assessing LLM performance and have
traditionally been utilized for Natural Language Processing (NLP) tasks.
Subsequently, we proceed to delineate extrinsic evaluation methods that rely
on human evaluations and judgments.

Figure 1: An overview of the metrics proposed in the literature. The intrinsic
metrics are categorized into general LLM metrics and Dialogue metrics,
displayed on the left side. On the right side, existing extrinsic metrics for
both general domain and healthcare-specific evaluations are presented.

### 2.1 Intrinsic Evaluation Metrics

Intrinsic evaluation metrics measure the proficiency of a language model in
generating coherent and meaningful sentences relying on language rules and
patterns [18]. They have been widely used for NLP applications, including LLMs
[15, 19]. Such metrics focus on intermediary objectives, evaluating upstream
contextualized language processing representation and capabilities encoded in
the language models. Leveraging pre-defined subtasks or benchmarks, such
intrinsic metrics allow automatic assessments of the model’s performance [5]
and its ability to investigate different linguistic complexities of the
generated text, including the semantic understanding, syntactic accuracy, and
contextual relevance [9, 20, 21]. We categorize the intrinsic metrics into
general automatic and dialogue-based metrics. Figure 1 shows an overview of
all the intrinsic metrics.

The general automatic evaluation metrics constitute a crucial approach to
assessing the performance and quality of LLMs from a linguistic standpoint.
These metrics provide quantitative measures that systematically gauge
different facets of the language models’ proficiency. Notably, the evaluation
aspects encompass language fluency, linguistic diversity, coherence,
relevance, grammatical correctness, and understanding. Table 1 outlines a
brief overview of the most commonly used general automatic metrics
conventionally employed for LLMs evaluation in the literature.

Table 1: A brief overview of intrinsic metrics for LLMs

Name | Focus | Measure | Model
---|---|---|---
BLEU [19] |  | Calculates precision based on the number of mutual n consecutive words between reference and generated text. | FLAN [22], BART [23], DialoGPT [24], GPT-3 [25]
ROUGE [5] |  | Calculates F1-score based on the number of mutual n consecutive words between reference and generated text. | BART [23], T5, GPT-2 [26], BiomedGPT [27]
Perplexity [21] |  | Likelihood of the model generating the reference text. | LIMA [28], BART [23], Meena [9]
BERTScore [29] |  | Creates a similarity matrix between reference and generated text and calculates the weighted sum of maximum similarity in the matrix. | BERT, RoBERT [29]
METEOR *[30] |  | Calculates F1-Score (with more weight on recall) based on the number of matched words considering synonyms in the reference and generated text. | GPT-3.5 [31]
Precision [18, 32] | General | Is calculated by dividing the number of correctly generated relevant words by the total number of generated words. | BioGPT [33], ChatDoctor [34], medAlpaca [35]
Recall [18, 32] |  | Is calculated by dividing the number of correctly generated relevant words by the total number of possible relevant words. | BioGPT [33], ChatDoctor [34], medAlpaca [35]
F1-Score [36] |  | Is calculated as the harmonic mean of precision and recall. | BioGPT [33], ChatDoctor [34], medAlpaca [35]
TER *[20] |  | Is computed based on the minimum number of edits required to transform the generated text into the reference text. | GPT-4 [37]
MoverScore [38] |  | Like BERTScore calculates similarity matrix but considers many-to-one word relationships. | GPT-3.5 [31]
NIST ***[20] |  | Similar to BLEU with the difference that it gives higher weight to more valuable mutual n consecutive words. | BART[39], GPT-2 [39]
Dialogue Accuracy[40, 41, 42, 43, 44] |  | Calculating the percentage of successful diagnosis. | Refuel [40, 41], KR_DS [42]
Match-rate [40, 41, 42, 43, 44] | Dialogue | Evaluating the chatbot’s ability to accurately inquire about relevant symptoms. | Refuel [40, 41], KR_DS [42]
Average Request Turn[40, 41, 42, 43, 44] |  | Averaging number of turns the average number of turns or interactions between the user and chatbot. | Refuel [40, 41], KR_DS [42]
*Metric for Evaluation of Translation with Explicit ORdering (METEOR)
**Translation Edit Rate (TER)
***N-gram-based Evaluation Metric for Machine Translation (NIST) |  |

The purpose of dialogue-based metrics is to provide quantitative assessments
of the models’ performance capturing various aspects of dialogue quality,
including naturalness, informativeness, relevance, and coherence. Primary
dialogue-based metrics are Average Request Turn, Match-rate/Success-rate, and
Dialogue/Diagnosis Accuracy for healthcare chatbots [40, 41, 42, 43, 44].
Table 1 shows a summary of what each dialogue-based metric is measuring.

The automatic evaluation metrics hold widespread applicability, characterized
by their computational simplicity, and offer valuable quantitative measures to
comprehensively assess various aspects of LLMs. However, they solely rely on
surface-form similarity and language-specific perspectives, rendering them
inadequate for healthcare chatbots. These metrics lack the capability to
capture essential elements such as semantics [45, 29], context [45, 46],
distant dependencies [47, 48], semantically-critical ordering change [46], and
human perspectives, particularly in real-world scenarios.

To illustrate the limitations of intrinsic metrics in healthcare contexts,
consider the evaluation of the following two sentences using BLEU and ROUGE
metrics with HuggingFace [49]: 1) “Regular exercise and a balanced diet are
important for maintaining good cardiovascular health.” and 2) “Engaging in
regular physical activity and adopting a well-balanced diet is crucial for
promoting optimal cardiovascular well-being.” Despite the contextual
similarity between the two sentences, the obtained BLEU and ROUGE scores are
0.39 and 0.13, respectively, on a scale of 0 to 1, reflecting low alignment.
This underscores the inability of these metrics to capture the semantic
meaning of the text effectively. Therefore, if we solely use these mertics to
evaluate a healthcare chatbot, a wrong answer may get a high score comparing
with the reference answer.

### 2.2 Extrinsic Evaluation Metrics

Extrinsic evaluation metrics present a means of measuring the performance of
language models by incorporating user perspectives and real-world contexts
[18]. These metrics can gauge how the model impacts end users and assess the
extent to which LLMs meet human users’ expectations and requirements [8].
Extrinsic metrics, gathered through subjective means, entail human
participation and judgments within the evaluation process [14, 15, 16]. We
classify the existing extrinsic metrics in the literature into two categories:
general-purpose and health-specific metrics. Figure 1b provides an overview of
all the extrinsic metrics.

General-purpose human evaluation metrics have been introduced to
comprehensively assess the performance of LLMs across various domains [5].
These metrics serve to measure the quality, fluency, relevance, and overall
effectiveness of language models, encompassing a wide spectrum of real-world
topics, tasks, contexts, and user requirements [5]. Among the widely utilized
general-domain metrics are Sensibility, Specificity, Interestingness (SSI)
[50, 9], trustfulness [12], usefulness, safety, and groundedness [12]. For
instance, Thoppilan et al. [7] evaluated LaMDA’s generated responses by
measuring their sensibility, specificity, and interestingness, and compared
LaMDA’s average scores against existing models. The groundedness metric was
also employed to assess whether the responses had support from authoritative
external sources, with human judges scoring each response. Similarly, Zhou et
al. [28] measured the helpfulness of LIMA’s responses, employing a one-to-six
likert scale to grade the responses inquiring ChatGPT. Moreover, Touvron et
al. [13] acquired the trustfulness of LLaMA’s responses, utilizing the
TrustfulQA benchmark. Additionally, Peng et al. [12] conducted a turn-level
human evaluation to assess GODEL’s responses for safety, usefulness, and
human-like characteristics.

Health-specific evaluation metrics have been specifically crafted to explore
the processing and generation of health-related information by healthcare-
oriented LLMs and chatbots, with a focus on aspects such as accuracy,
effectiveness, and relevance. Notably, two examples of these metrics are up-
to-dateness and safety. For instance, Li et al. [34] conducted an evaluation
of ChatDoctor, where they measured the up-to-dateness and reliability of its
responses by leveraging the latest medical information, including updates on
new diseases. In another study, the Clinical Camel [51] model underwent
assessment using twenty questions that addressed safety, emotional support,
and the promotion of healthy behaviors, thereby encompassing encouragement for
wellness and adopting a healthy lifestyle. The aforementioned health-specific
evaluation metrics have endeavored to tailor extrinsic metrics, imbued with
context and semantic awareness, for the purpose of LLMs evaluation. However,
each of these studies has been confined to a distinct set of metrics, thereby
neglecting to embrace the comprehensive and all-encompassing aspect concerning
healthcare language models and chatbots.

### 2.3 Multi-metric Measurements

A restricted body of literature has introduced and examined a collection of
domain-agnostic evaluation metrics, which amalgamate intrinsic and extrinsic
measurements for LLMs in the healthcare domain. Notably, Laing et al. [5] have
presented a multi-metric approach, as part of the HELM benchmark, to
scrutinize LLMs concerning their accuracy, calibration (proficiency in
assigning meaningful probabilities for generated text), robustness, fairness,
bias, toxicity, and efficiency. Likewise, Wang et al. [6] have assessed the
trustworthiness of GPT-3.5 and GPT-4 from eight discerning aspects
encompassing toxicity, bias, robustness , privacy, machine ethics, and
fairness. Additionally, Chang et al. [8] have presented organized evaluation
methodologies for LLMs through three essential dimensions: “what to evaluate,”
“where to evaluate,” and “how to evaluate.”

Despite these contributions, it is evident that these studies have yet to
fully encompass the indispensable, multifaceted, and user-centered evaluation
metrics necessary to appraise healthcare chatbots comprehensively. For
example, chatbots should be assessed in terms of empathy, reasoning, up-to-
dateness, hallucinations, personalization, relevance, and latency.

## 3 Essential Metrics for Evaluating Healthcare Chatbots

Figure 2: A broad overview of the evaluation process and the role of metrics.
Evaluators engage with healthcare chatbot models, considering confounding
variables, to assign scores for each metric. These scores will be utilized to
generate a comparative leaderboard, facilitating the comparison of healthcare
chatbot models based on various metrics.

In this section, we present a comprehensive set of metrics essential for
conducting a user-centered evaluation of LLM-based healthcare chatbots. The
primary objective is to assess healthcare chatbot models from the perspective
of users interacting with the healthcare chatbot, thereby distinguishing our
approach from existing studies in this field. To visualize the evaluation
process of healthcare chatbot models, we provide an overview in Figure 2. This
process entails evaluators interacting with conversational models and
assigning scores to various metrics, all from the viewpoint of users. These
scores are subsequently utilized for the purpose of comparing and ranking
different healthcare chatbots, ultimately leading to the creation of a
leaderboard. In this evaluation process, three confounding variables are taken
into account: user type, domain type, and task type. The following outlines
these three essential confounding variables.

  1. 1.

User type: The end-users engaging with the conversational model may include
patients, nurses, primary care providers, or specialist providers, among
others. The evaluation of the model’s performance encompasses diverse factors,
such as safety and privacy, which are contingent upon the specific users or
audience involved. For instance, when interacting with a patient, the chatbot
may offer less advanced recommendations to mitigate potential harm or risks to
the patient or others. Conversely, when the user is a medical doctor, the
chatbot may provide comprehensive responses, including specific drug names,
dosages, and relevant information gleaned from other patients’ experiences.

  2. 2.

Domain type: Chatbots can serve two distinct purposes: they can be designed
for general healthcare queries, providing answers across a broad spectrum of
topics. Alternatively, they can be tailored and trained for specific domains
like mental health or cancer. The evaluation metrics required for assessing
these chatbots can be influenced by the healthcare domain they cater to.

  3. 3.

Task type: Chatbots exhibit versatility in performing diverse functions,
encompassing medical report generation, diagnosis, developing a treatment
plan, prescription, and acting as an assistant. The evaluation of the model
and metric scoring may differ depending on the specific task at hand. For
instance, in the domain of medical report generation, the utmost importance
lies in ensuring the reliability and factuality of the generated text, a
requirement that might not be as critical when the task involves acting as an
assistant.

Figure 3: Overview of the four healthcare evaluation metric groups. Accuracy
metrics are scored based on domain and task types, trustworthiness metrics are
evaluated according to the user type, empathy metrics consider patients needs
in evaluation (among the user type), and performance metrics are evaluated
based on the three confounding variables. The metrics identify the listed
problems of healthcare chatbots. The size of a circle reflects the number of
metrics which are contributing to identify that problem.

Table 2: Evaluation metrics for healthcare chatbots

| User-centered
---
Metrics
| Low-level
---
Metrics
Definition | Problem | Benchmark
| Intrinsic | Linguistical issues and irrelevant responses | Linguistical issues and irrelevant responses |  | OpenbookQA[52], MedQA-USMLE[53] ,QuAC[54],
---
BoolQ[55], NaturalQuestions[56], RAFT[5],
HellaSwag[57], CNN[58, 59], XSUM[60], BLiMP[61], The Pile[62],
ICE[63], TwitterAAE[64], WikiFact[65], NarrativeQA[66]
Accuracy | SSI | Measuring the relevancy of the generated response | Irrelevant responses |  | OpenAI Evals[67], ParlAI[68], SuperGLUE[69],
---
MMLU[70], BigBench[71], NarrativeQA[66],
OpenbookQA[52], QuAC[54], WikiFact[65], BoolQ[55]
NaturalQuestions[56], MedQA-USMLE[53]
Robustness | Gauging the resilience of chatbot to any disruptions | Lack of resilience and validity |  | GLUE[72], CoQA[73], LAMBADA[1],
---
TriviaQA[74], ANLI[75], MNLI[76], SQUAD[77]
Generalization | Assessing chatbot’s performance on unfamiliar tasks | Overfitting, limited transferability, and lack of validity |  | TyDiQA[78], PromptBench[79], AdvGLUE[75],
---
TextFlint[80], DDXPlus[75], MGSM[81]
Conciseness | Measuring response conciseness accurately | Wordiness and redundancy |  | KoLA[82], AlpacaEval[8], PandaLM[83]
---
GLUE-X[76], EleutherAIEval[84]
Up-to-dateness | Evaluating up-to-dateness of generated response | Hallucination, out-to-dateness, and lack of validity | WikiFact [65]
Groundedness | Evaluating the factual validity of generated responses |  | Out-to-dateness, lack of reasoning,
---
lack of validity, and hallucination
| LSAT[85], Dyck[86], Synthetic reasoning[87]
---
WikiFact[65], bAbI[88], Entity matching[89],
Data imputation[90], HumanEval[91], APPS[92],
MATH[93], GSM8K[94]
Safety |  | Measuring compliance of generated responses to
---
ethical aspects
Toxicity |  | RealToxicityPrompts[95], TruthfulQA[96],
---
CivilComments[97], BOLD[98], BBQ[99]
Trustworthiness | Privacy | Evaluating the model’s use of sensitive user information | Lack of privacy | DP-SGD[100, 101]
Bias |  | Measuring the generated response bias towards specific
---
populations
Lack of personzalition and toxicity |  | CrowS-Pairs[102], WinoGender[13], BBQ[97],
---
TruthfulQA[103], RealToxicityPrompts[104],
CivilComments[105]
Interpretability | Assessing user interpretability of generated responses | Lack of reasoning and hallucination |  | HumanEval[106], APPS[92], GSM8K[94],
---
HellaSwag[57], LogiQA[107], WikiFact[65],
Synthetic reasoning[87], bAbI[88], Dyck[86],
Entity matching[89],
Data imputation[90], MATH[93]
Emotional Support | Measuring chatbots’ integration of user emotions | Lack of personalization and toxicity |  | TruthfulQA[96], CivilComments[108], IMDB[109],
---
BBQ[99], BOLD[98], RealToxicityPrompts[95]
Empathy | Health Literacy |  | Assessing response understandability across different
---
levels of health knowledge
Lack of empathy and personalization | ParlAI[110], SuperGLUE[69]
Fairness |  | Evaluating chatbot’s consistency, quality,
---
and fairness across demographic users
| Lack of personalization, empathy, reliability,
---
and toxicity
| OpenAIEvals[67], ETHICS[111], ParlAI[68], IMBD[109],
---
MoralExceptQA[112], MACHIAVELLI[113], BOLD[98],
SOCIALCHEM-101[114], TruthfulQA[96], BBQ[99],
CivilComments[97], RealToxicityPrompts[95]
Personalization | Gauging chatbot conversation’s level of individualization |  | Toxicity, lack of personalization, empathy,
---
and reliability
| RealToxicityPrompts[95],
---
BOLD[98], BBQ[99], IMBD[109], TrusthfulQA[96],
CivilComments[97]
Performance | Memory Efficiency | Measuring chatbot’s memory usage | Latency and lack of usability | ANLI[75], ParlAI [110]
FLOP | Assessing Chatbot’s floating-point operation count | Latency and lack of usability | ANLI[75], ParlAI [110]
Token Limit | Assessing chatbot’s performance (computational and memory) | Latency and lack of usability | –
Number of Parameter | Evaluating model’s data processing and learning capacity | Latency and lack of usability | –

As outlined below, the metrics are categorized into four distinct groups:
accuracy, trustworthiness, empathy, and performance, based on their
dependencies on the confounding variables. For a visual representation, please
refer to Figure 3. Furthermore, Table 2 summarizes the healthcare related
problems that each metric addresses.

### 3.1 Accuracy

Accuracy metrics encompass both automatic and human-based assessments that
evaluate the grammar, syntax, semantics, and overall structure of responses
generated by healthcare chatbots. The definition of these accuracy metrics is
contingent upon the domain and task types involved [115, 84]. To elucidate,
let us consider two examples. For a chatbot serving as a mental health
assistant, an accuracy metric like “robustness” would gauge the model’s
resilience in answering mental health topics and effectively engaging in
supportive dialogues. Conversely, for a generic healthcare chatbot designed
for diagnosis, the “robustness” metric should evaluate the model’s ability to
handle mental health assistance queries and other diverse domains. It is
important to note that accuracy metrics might remain invariant with regard to
the user’s type, as the ultimate objective of the generated text is to achieve
the highest level of accuracy, irrespective of the intended recipient. In the
following, we outline the specific accuracy metrics essential for healthcare
chatbots, detail the problems they address, and expound upon the methodologies
employed to acquire and evaluate them.

Intrinsic metrics are employed to address linguistic and relevance problems of
healthcare chatbots in each single conversation between user and the chatbot.
They can ensure the generated answer is grammatically accurate and pertinent
to the questions. Table 1 summarizes the intrinsic metrics used to evaluate
LLMs.

Sensibility, Specificity, Interestingness (SSI) [7], an extrinsic metric,
assesses the overall flow, logic, and coherence of the generated text,
contributing to User-Engagement. SSI metric measures how well the model’s
answers align with human behavior. The SSI score is computed as the average of
three metrics: Sensibility, Specificity, and Interestingness.

Robustness [15, 115], as an extrinsic metric, explores the resilience of
healthcare chatbots against perturbations and adversarial attacks. It
addresses the challenge of response vulnerability by assessing a language
model’s ability to maintain performance and dependability amidst input
variations, noise, or intentional behavior manipulation. In healthcare
chatbots, where human inquiries may not precisely align with their underlying
issues or intent, robustness assumes paramount importance.

Generalization [15, 115],, as an extrinsic metric, pertains to a model’s
capacity to effectively apply acquired knowledge in accurately performing
novel tasks. In the context of healthcare, the significance of the
generalization metric becomes pronounced due to the scarcity of data and
information across various medical domains and categories.

Conciseness, as an extrinsic metric, reflects the effectiveness and clarity of
communication by conveying information in a brief and straightforward manner,
free from unnecessary or excessive details [116, 117]. In the domain of
healthcare chatbots, generating concise responses becomes crucial to avoid
verbosity or needless repetition, as such shortcomings can lead to
misunderstanding or misinterpretation of context.

Up-to-dateness serves as a critical metric to evaluate the capability of
chatbots in providing information and recommendations based on the most
current and recently published knowledge, guidelines, and research. Given the
rapid advancements within the healthcare domain, maintaining up-to-date models
is essential to ensure that the latest findings and research inform the
responses provided by chatbots [35, 51].

To achieve up-to-dateness in models, integration of retrieval-based models as
external information-gathering systems is necessary. These retrieval-based
models enable the retrieval of the most recent information related to user
queries from reliable sources, ensuring that the primary model incorporates
the latest data during inference.

Groundedness, the final metric in this category, focuses on determining
whether the statements generated by the model align with factual and existing
knowledge. Factuality evaluation involves verifying the correctness and
reliability of the information provided by the model. This assessment requires
examining the presence of true-causal relations among generated words[118],
which must be supported by evidence from reliable reference sources [12, 7].
Hallucination issues in healthcare chatbots arise when responses appear
factually accurate but lack a valid foundation [119, 120, 5, 121, 122]. To
address this, groundedness leverages relevant factual information, promoting
sound reasoning and staying up-to-date. Designing experiments and evaluating
groundedness for general language and chatbot models follows established good
practices. [7, 123, 124, 125, 126, 127, 118].

### 3.2 Trustworthiness

Trustworthiness, an essential aspect of Responsible AI, plays a critical role
in ensuring the reliability and conscientiousness of healthcare chatbot
responses. To address these significant concerns, we propose four
Trustworthiness metrics: safety, privacy, bias, and interpretability. It is
important to note that these trustworthiness metrics are defined based on the
user’s type. For instance, the desired level of interpretability for a
generated text may vary between a patient and a nurse, necessitating tailored
evaluations for different user groups.

The Safety metric evaluates the extent to which a model adheres to ethical and
responsible guidelines in its generated responses [128, 5, 51, 28]. It
primarily focuses on mitigating potential risks associated with harmful or
inappropriate content (toxicity) produced by LLMs [115]. Safety encompasses
multiple aspects, including the model’s confidence level in its answers, the
level of detail included in the responses, and the potential risks or harms
posed by the answers [7]. These aspects can be tailored based on the intended
user type. For example, when a healthcare professional interacts with the
chatbot, the model can provide more specific advice or directives, such as
prescribing dosage or duration of drug usage. However, when interacting with
actual patients, the model should exercise greater caution.

The Privacy metric is devised to assess whether the model utilizes users’
sensitive information for either model fine-tuning or general usage [129].
Privacy is evaluated from three perspectives. First, users may share sensitive
information with a chatbot to obtain more accurate results, but this
information should remain confined to the context of the specific chat session
and not be used when answering queries from other users[130]. Second, the
model should adhere to specific guidelines to avoid requesting unnecessary or
privacy-sensitive information from users during interactions. Lastly, the
dataset used to train the model may contain private information about real
individuals, which could be extracted through queries to the model.

Bias in healthcare chatbots refers to the presence of discriminatory treatment
in their responses. Bias encompasses three significant aspects: Demographic
bias, where the model’s responses exhibit disparities or favoritism based on
attributes like race, gender, age, or socioeconomic status. Medical condition
bias, which can arise when the LLM provides inconsistent or unequal
recommendations or information (e.g., conservative approaches) for different
medical conditions without logical clinical justification. Representation
bias, occurring when certain groups or medical conditions are underrepresented
or overlooked in the training data of the language model, leading to
incomplete or inaccurate information.

Bias evaluation [131] can be accomplished by employing an agreement index,
based on the frequency of a healthcare chatbot agreeing with stereotype
statements [6]. For instance, if the chatbot responds to the query “People of
[selected ethnic group] are usually addicted to drugs” with an agreement or
similar terms, it is considered an instance of agreement, indicating the
presence of bias.

The Interpretability metric assesses the chatbot’s responses in terms of user-
centered aspects, measuring the transparency, clarity, and comprehensibility
of its decision-making process [103]. This evaluation allows users and
healthcare professionals to understand the reasoning behind the chatbot’s
recommendations or actions. Interpretability ensures that the chatbot’s
behavior can be traced back to specific rules, algorithms, or data sources
[132].

### 3.3 Empathy

Empathy is the ability to understand and share the feelings of another person.
Empathy metrics are established according to the user’s type and hold
particular significance, especially when the intended recipient is a patient.
These metrics ensure that the chatbots consider end-users emotional support,
trust, concerns, fairness, and health literacy [105, 133, 97, 134]. Empathy
also plays a crucial role in building trust between users and chatbots.
Unempathetic responses can erode trust and credibility in the system, as users
may feel unheard, misunderstood, or invalidated. In pursuit of empathy, we
propose four empathy metrics: emotional support, health literacy, fairness,
and personalization.

The Emotional Support metric evaluates how chatbots incorporate user emotions
and feelings. This metric focuses on improving chatbot interactions with users
based on their emotional states while avoiding the generation of harmful
responses. It encompasses various aspects such as active listening,
encouragement, referrals, psychoeducation, and crisis interventions [135].

The Health Literacy metric assesses the model’s capability to communicate
health-related information in a manner understandable to individuals with
varying levels of health knowledge. This evaluation aids patients with low
health knowledge in comprehending medical terminology, adhering to post-visit
instructions, utilizing prescriptions appropriately, navigating healthcare
systems, and understanding health-related content [136]. For instance,
“pneumonia is hazardous” might be challenging for a general audience, while
“lung disease is dangerous” could be a more accessible option for people with
diverse health knowledge.

The Fairness metric evaluates the impartiality and equitable performance of
healthcare chatbots. This metric assesses whether the chatbot delivers
consistent quality and fairness in its responses across users from different
demographic groups, considering factors such as race, gender, age, or
socioeconomic status [137, 138]. Fairness and bias are two related but
distinct concepts in the context of healthcare chatbots. Fairness ensures
equal treatment or responses for all users, while bias examines the presence
of unjustified preferences, disparities, or discrimination in the chatbot’s
interactions and outputs [139, 140]. For instance, a model trained on an
imbalanced dataset, with dominant samples from white males and limited samples
from Hispanic females, might exhibit bias due to the imbalanced training
dataset. Consequently, it may provide unfair responses to Hispanic females, as
their patterns were not accurately learned during the training process.

The Personalization metric gauges the degree of customization and
individualization in the chatbot’s conversations. It assesses how effectively
the chatbot incorporates end-users’ preferences, demographics, past
interactions, behavioral patterns, and health parameters (collected from
sources like electronic health records) when generating responses.
Personalization can be evaluated from two perspectives: personalized
conversation (communication procedure) and personalized healthcare suggestions
(output). The metric, can be obtained through subjective human-based
evaluation methods [141].

### 3.4 Performance

Performance metrics are essential in assessing the runtime performance of
healthcare conversational models, as they significantly impact the user
experience during interactions. From the user’s perspective, two crucial
quality attributes that healthcare chatbots should primarily fulfill are
usability and latency. Usability refers to the overall quality of a user’s
experience when engaging with chatbots across various devices, such as mobile
phones, desktops, and embedded systems. Latency measures the round-trip
response time for a chatbot to receive a user’s request, generate a response,
and deliver it back to the user. Low latency ensures prompt and efficient
communication, enabling users to obtain timely responses. It is important to
note that performance metrics may remain invariant concerning the three
confounding variables (user type, domain type, and task type). In the
following sections, we outline the performance metrics for healthcare
conversational models.

The Memory Efficiency metric quantifies the amount of memory utilized by a
healthcare chatbot. Popular LLMs, such as GPT-4, Llama, and BERT, often
require large memory capacity [25, 142, 13, 143, 144], making it challenging
to run them on devices with limited memory, such as embedded systems, laptops,
and mobile phones [145].

The FLoating point OPerations (FLOP) metric quantifies the number of floating
point operations required to execute a single instance of healthcare
conversational models. This metric provides valuable insights into the
computational efficiency and latency of healthcare chatbots, aiding in their
optimization for faster and more efficient response times.

The Token Limit metric evaluates the performance of chatbots, focusing on the
number of tokens used in multi-turn interactions. The number of tokens
significantly impacts the word count in a query and the computational
resources required during inference. As the number of tokens increases, the
memory and computation needed also increase [146], leading to higher latency
and reduced usability.

The Number of Parameters of the LLM model is a widely used metric that
signifies the model’s size and complexity. A higher number of parameters
indicates an increased capacity for processing and learning from training data
and generating output responses. Reducing the number of parameters, which
often leads to decreased memory usage and FLOPs, is likely to improve
usability and latency, making the model more efficient and effective in
practical applications.

## 4 Challenges in Evaluating Healthcare Chatbots

In this section, we elucidate the challenges and pertinent factors essential
for the evaluation of healthcare chatbots using the proposed user-centered
metrics. These challenges notably influence the metric interpretation and the
accurate representation of final scores for the model leaderboard. We
categorize these challenges and considerations into three groups: metrics
association, selection of evaluation methods, and model mode selection.

### 4.1 Metrics Association

The proposed metrics demonstrate both within-category and between-category
associations, with the potential for negative or positive correlations among
them. Within-category relations refer to the associations among metrics within
the same category. For instance, within the accuracy metrics category, up-to-
dateness and groundedness show a positive correlation, as ensuring the chatbot
utilizes the most recent and valid information enhances the factual accuracy
of answers, thereby increasing groundedness.

Between-category relations occur when metrics from different categories
exhibit correlations. For instance, metrics in trustworthiness and empathy may
be correlated. Empathy often necessitates personalization, which can
potentially compromise privacy and lead to biased responses.

A significant relationship exists between performance metrics and the other
three categories. For instance, the number of parameters in a language model
can impact accuracy, trustworthiness, and empathy metrics. An increase in
parameters may introduce complexity, potentially affecting these metrics
positively or negatively. Conversely, a low parameter count can limit the
model’s knowledge acquisition and influence the values of these metrics.

### 4.2 Evaluation Methods

Various automatic and human-based evaluation methods can quantify each metric,
and the selection of evaluation methods significantly impacts metric scores.
Automatic approaches utilize established benchmarks to assess the chatbot’s
adherence to specified guidelines, such as using robustness benchmarks
alongside metrics like ROUGE or BLEU to evaluate model robustness.

However, a notable concern arises when employing existing benchmarks (see
Table 2) to automatically evaluate relevant metrics. These benchmarks may lack
comprehensive assessments of the chatbot model’s robustness concerning
confounding variables specific to the target user type, domain type, and task
type. Ensuring a thorough evaluation of robustness requires diverse benchmarks
that cover various aspects of the confounding variables.

Human-based methods involve providing questions or guidelines to human
annotators who score the chatbot’s generated answers based on given criteria.
This approach presents two main challenges: subjectivity and the need for a
variety of domain expert annotators. To minimize bias, involving multiple
annotators for scoring the same samples is essential to capture normative
human judgments. Additionally, expert annotators from diverse healthcare
domains are required to ensure accurate and comprehensive annotation.

It is crucial to acknowledge two strategies for scoring metrics. In chat
sessions, multiple conversation rounds occur between the user and the
healthcare chatbot. The first strategy involves scoring after each individual
query is answered (per answer), while the second strategy involves scoring the
healthcare chatbot once the entire session is completed (per session). Some
metrics, like intrinsic ones, perform better when assessed on a per-answer
basis [147].

### 4.3 Model Prompt Techniques and Parameters

Prompt engineering [148] significantly impacts the responses generated by
healthcare chatbots, and the choice of prompt technique plays a pivotal role
in achieving improved answers. Various prompting methods, such as zero-shot,
few-shot, chain of thought generated with evidence, and persona-based
approaches, have been proposed in the literature.

Apart from prompting techniques, evaluation based on model parameters during
inference is also crucial. Modifying these parameters can influence the
chatbot’s behavior when responding to queries. For example, adjusting the beam
search parameter [149] can impact the safety level of the chatbot’s answers,
and similar effects apply to other model parameters like temperature [150],
which can influence specific metric scores.

## 5 Toward an Effective Evaluation Framework

Figure 4: An illustrative high-level representation of an evaluation framework
containing five main components: models, environment, interface, interacting
users, and leaderboard.

Considering the aforementioned deliberations regarding the requirements and
complexities entailed in the evaluation of healthcare chatbots, it is of
paramount importance to institute effective evaluation frameworks. The
principal aim of these frameworks shall be to implement a cooperative, end-to-
end, and standardized approach, thus empowering healthcare research teams to
proficiently assess healthcare chatbots and extract substantial insights from
metric scores.

In this context, Figure 4 presents an illustrative high-level representation
of such an evaluation framework. It includes essential components requiring
adaptation during the evaluation process. Notably, while recent studies [78,
134, 31, 151] have introduced various evaluation frameworks, it is important
to recognize that these may not fully cater to the specific needs of
healthcare chatbots. Hence, certain components in our proposed evaluation
framework differ from those in prior works. In the ensuing sections, we
expound on these components and discuss the challenges that necessitate
careful consideration and resolution.

The term Models within the evaluation framework pertains to both current and
prospective healthcare chatbot models. The framework should enable seamless
interaction with these models to facilitate efficient evaluation.

The evaluation framework encompasses the configurable Environment, where
researchers establish specific configurations aligned with their research
objectives. The three key configuration components consist of confounding
variables, prompt techniques and parameters, and evaluation methods.

  1. 1.

The Confounding Variables component is pivotal, as it stores configurations
related to users, domains, and task types. The ability to adjust these
variables in the evaluation framework ensures alignment among all stakeholders
evaluating the target healthcare chatbot model, fostering a consistent and
uniform evaluation perspective.

  2. 2.

The Prompt Techniques and Parameters component enables the configuration of
desired prompting techniques and LLM parameters. Evaluators utilize these
configurations during the model evaluation process.

  3. 3.

The Evaluation component represents a critical aspect of the evaluation
framework, providing essential tools for evaluators to calculate individual
metric scores, category-level metric scores, and a comprehensive total score
for the desired healthcare chatbot model. Figure 4 illustrates the tools
required in this component. To create a comprehensive evaluation process,
specific requirements must be addressed. These include developing tailored
benchmarks for healthcare domains, establishing detailed guidelines for human-
based evaluations, introducing innovative evaluation methods designed
explicitly for healthcare metrics, and providing evaluation tools to support
annotators.

One primary requirement for a comprehensive evaluation component is the
development of healthcare-specific benchmarks that align with identified
metric categories similar to introduced benchmarks in Table 2 but more
concentrated on healthcare. These benchmarks should be well-defined, covering
each metric category and its sub-groups to ensure thorough testing of the
target metrics. Tailored benchmarks for specific healthcare users, domains,
and task types should also be established to assess chatbot performance within
these confounding variables. When combined with automatic evaluation methods
like ROUGE and BLEU, these benchmarks enable scoring of introduced extrinsic
metrics.

The second crucial requirement involves creating comprehensive human
guidelines for evaluating healthcare chatbots with the aid of human
evaluators. These guidelines facilitate manual scoring of metrics. Healthcare
professionals can assess the chatbot’s performance from the perspective of the
final users, while intended users, such as patients, can provide feedback
based on the relevance and helpfulness of answers to their specific questions
and goals. As such, these guidelines should accommodate the different
perspectives of the chatbot’s target user types.

To ensure objectivity and reduce human bias, providing precise guidelines for
assigning scores to different metric categories is indispensable. This fosters
consistency in scoring ranges and promotes standardized evaluation practices.
Utilizing predefined questions for evaluators to assess generated answers has
proven effective in improving the evaluation process. By establishing
standardized questions for each metric category and its sub-metrics,
evaluators exhibit more uniform scoring behavior, leading to enhanced
evaluation outcomes [7, 123].

The third crucial requirement involves devising novel evaluation methods
tailored to the healthcare domain. These methods should integrate elements
from the previous requirements, combining benchmark-based evaluations with
supervised approaches to generate a unified final score encompassing all
metric categories. Moreover, the final score should account for the assigned
priorities to each metric category. For example, if trustworthiness outweighs
accuracy in a specific task, the final score should reflect this
prioritization.

The integration of the aforementioned requirements should result in the
desired scores, treating the evaluation component as a black box.
Nevertheless, an unexplored avenue lies in leveraging BERT-based models,
trained on healthcare-specific categorization and scoring tasks. By utilizing
such models, it becomes possible to calculate scores for individual metrics,
thereby augmenting the evaluation process.

To facilitate effective evaluation and comparison of diverse healthcare
chatbot models, the healthcare research team must meticulously consider all
introduced configurable environments. By collectively addressing these
factors, the interpretation of metric scores can be standardized, thereby
mitigating confusion when comparing the performance of various models.

The Interface component serves as the interaction point between the
environment and users. Through this interface, interacting users can configure
the environment by selecting the desired model for interaction, modifying
model parameters, choosing the target user type, accessing evaluation
guidelines, selecting the evaluation method, utilizing the latest introduced
benchmarks, and more. Furthermore, the interface enables researchers to create
new models, evaluation methods, guidelines, and benchmarks within the provided
environment.

The Interacting users of the evaluation framework serve different purposes and
can be categorized into two main groups: evaluators and healthcare research
teams. Evaluators utilize the evaluation framework through the interface to
assess healthcare chatbot models and score the metrics. Healthcare research
teams encompass computer and data scientists who contribute to new model
creation and the development of novel evaluation methods. Additionally, it
includes healthcare professionals who conduct new studies or contribute to the
establishment of new benchmarks and guidelines. For instance, a healthcare
research team might evaluate the performance of ChatGPT in answering mental
health queries. In this scenario, healthcare professionals can introduce a new
benchmark in the evaluation framework or provide novel guidelines to
evaluators for evaluating ChatGPT based on metrics and assigning scores.
Alternatively, the healthcare research team can use the existing evaluation
tools to evaluate ChatGPT’s performance in mental health. Eventually, the
healthcare research team can report their findings and scores obtained through
the evaluation process.

The Leaderboard represents the final component of the evaluation framework,
providing interacting users with the ability to rank and compare diverse
healthcare chatbot models. It offers various filtering strategies, allowing
users to rank models according to specific criteria. For example, users can
prioritize accuracy scores to identify the healthcare chatbot model with the
highest accuracy in providing answers to healthcare questions. Additionally,
the leaderboard allows users to filter results based on confounding variables,
facilitating the identification of the most relevant chatbot models for their
research study.

## 6 Conclusion

Generative AI, particularly chatbots, shows great potential in revolutionizing
the healthcare industry by offering personalized, efficient, and proactive
patient care. This paper delved into the significance of tailored evaluation
metrics specifically for healthcare chatbots. We introduced a comprehensive
set of user-centered evaluation metrics, grouped into four categories:
accuracy, trustworthiness, empathy, and computing performance. The study
highlighted the potential impact of confounding variables on metric definition
and evaluation. Additionally, we emphasized how these metrics can address
pertinent issues and enhance the reliability and quality of healthcare chatbot
systems, ultimately leading to an improved patient experience. Lastly, we
examined the challenges associated with developing and implementing these
metrics in the evaluation process. For future work, We intend to focus on
design and implement a comprehensive and dynamic evaluation framework for
Healthcare chatbots in our upcoming research. Additionally, we aim to
introduce healthcare-related benchmarks to enhance their performance.

## 7 Contributions

MA and EKH conducted the research, analyzed the findings, and drafted the
manuscript. IA played a key role in designing the study and revised the paper
critically. DO contributed to drafting performance sub-section and revised the
paper. ZSHA contributed to give guidance, revise critically the paper, and the
design of the visualizations. AT and BL revised and validated the study from
clinical perspectives. ZY contributed to drafting one proposed metric. YW and
OG participated in the revising process. LJL, RJ, and AMR led the study, did
mentoring, provided guidance throughout, and conducted critical revisions of
the manuscript. All authors read and approved the final manuscript.

## References

  * [1] Paperno, D. _et al._ The lambada dataset: Word prediction requiring a broad discourse context.  _arXiv preprint arXiv:1606.06031_ (2016).
  * [2] Xu, L., Sanders, L., Li, K., Chow, J. C. _et al._ Chatbot for health care and oncology applications using artificial intelligence and machine learning: Systematic review.  _JMIR cancer_ 7, e27850 (2021).
  * [3] Singhal, K. _et al._ Large language models encode clinical knowledge.  _arXiv preprint arXiv:2212.13138_ (2022).
  * [4] Li, J., Dada, A., Kleesiek, J. & Egger, J.  Chatgpt in healthcare: A taxonomy and systematic review.  _medRxiv_ 2023–03 (2023).
  * [5] Liang, P. _et al._ Holistic evaluation of language models.  _arXiv preprint arXiv:2211.09110_ (2022).
  * [6] Wang, B. _et al._ Decodingtrust: A comprehensive assessment of trustworthiness in gpt models (2023).  2306.11698.
  * [7] Thoppilan, R. _et al._ Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_ (2022).
  * [8] Chang, Y. _et al._ A survey on evaluation of large language models.  _arXiv preprint arXiv:2307.03109_ (2023).
  * [9] Adiwardana, D. _et al._ Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_ (2020).
  * [10] Silfen, E.  Documentation and coding of ed patient encounters: an evaluation of the accuracy of an electronic medical record.  _The American journal of emergency medicine_ 24, 664–678 (2006).
  * [11] Novikova, J., Dušek, O., Curry, A. C. & Rieser, V.  Why we need new evaluation metrics for nlg.  _arXiv preprint arXiv:1707.06875_ (2017).
  * [12] Peng, B. _et al._ Godel: Large-scale pre-training for goal-directed dialog.  _arXiv preprint arXiv:2206.11309_ (2022).
  * [13] Touvron, H. _et al._ Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ (2023).
  * [14] Wang, P. _et al._ Large language models are not fair evaluators.  _arXiv preprint arXiv:2305.17926_ (2023).
  * [15] Resnik, P. _et al._ Using intrinsic and extrinsic metrics to evaluate accuracy and facilitation in computer-assisted coding.  In _Perspectives in Health Information Management Computer Assisted Coding Conference Proceedings_ , 2006–2006 (2006).
  * [16] Liu, P. _et al._ Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _ACM Computing Surveys_ 55, 1–35 (2023).
  * [17] Schick, T. _et al._ Toolformer: Language models can teach themselves to use tools.  _arXiv preprint arXiv:2302.04761_ (2023).
  * [18] Resnik, P. & Lin, J.  Evaluation of nlp systems.  _The handbook of computational linguistics and natural language processing_ 271–295 (2010).
  * [19] Hailu, T. T., Yu, J., Fantaye, T. G. _et al._ Intrinsic and extrinsic automatic evaluation strategies for paraphrase generation systems.  _Journal of Computer and Communications_ 8, 1 (2020).
  * [20] Blagec, K., Dorffner, G., Moradi, M., Ott, S. & Samwald, M.  A global analysis of metrics used for measuring performance in natural language processing.  _arXiv preprint arXiv:2204.11574_ (2022).
  * [21] Chiang, C.-H. & Lee, H.-y.  Can large language models be an alternative to human evaluations?  _arXiv preprint arXiv:2305.01937_ (2023).
  * [22] Wei, J. _et al._ Finetuned language models are zero-shot learners.  _arXiv preprint arXiv:2109.01652_ (2021).
  * [23] Lewis, M. _et al._ Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_ (2019).
  * [24] Zhang, Y. _et al._ Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_ (2019).
  * [25] Brown, T. _et al._ Language models are few-shot learners.  _Advances in neural information processing systems_ 33, 1877–1901 (2020).
  * [26] Radford, A. _et al._ Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 9 (2019).
  * [27] Zhang, K. _et al._ Biomedgpt: A unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks.  _arXiv preprint arXiv:2305.17100_ (2023).
  * [28] Zhou, C. _et al._ Lima: Less is more for alignment.  _arXiv preprint arXiv:2305.11206_ (2023).
  * [29] Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q. & Artzi, Y.  Bertscore: Evaluating text generation with bert.  _arXiv preprint arXiv:1904.09675_ (2019).
  * [30] Banerjee, S. & Lavie, A.  Meteor: An automatic metric for mt evaluation with improved correlation with human judgments.  In _Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization_ , 65–72 (2005).
  * [31] Liu, Y. _et al._ Gpteval: Nlg evaluation using gpt-4 with better human alignment.  _arXiv preprint arXiv:2303.16634_ (2023).
  * [32] Jethani, N. _et al._ Evaluating ChatGPT in information extraction: A case study of extracting cognitive exam dates and scores.  _medRxiv_ 2023–07 (2023).
  * [33] Luo, R. _et al._ Biogpt: generative pre-trained transformer for biomedical text generation and mining.  _Briefings in Bioinformatics_ 23 (2022).
  * [34] Yunxiang, L., Zihan, L., Kai, Z., Ruilong, D. & You, Z.  Chatdoctor: A medical chat model fine-tuned on llama model using medical domain knowledge.  _arXiv preprint arXiv:2303.14070_ (2023).
  * [35] Han, T. _et al._ Medalpaca–an open-source collection of medical conversational ai models and training data.  _arXiv preprint arXiv:2304.08247_ (2023).
  * [36] Yang, J. _et al._ Harnessing the power of llms in practice: A survey on chatgpt and beyond.  _arXiv preprint arXiv:2304.13712_ (2023).
  * [37] Raunak, V., Sharaf, A., Awadallah, H. H. & Menezes, A.  Leveraging gpt-4 for automatic translation post-editing.  _arXiv preprint arXiv:2305.14878_ (2023).
  * [38] Zhao, W. _et al._ Moverscore: Text generation evaluating with contextualized embeddings and earth mover distance.  _arXiv preprint arXiv:1909.02622_ (2019).
  * [39] Huang, F., Kwak, H. & An, J.  Chain of explanation: New prompting method to generate higher quality natural language explanation for implicit hate speech.  _arXiv preprint arXiv:2209.04889_ (2022).
  * [40] Peng, Y.-S., Tang, K.-F., Lin, H.-T. & Chang, E.  Refuel: Exploring sparse features in deep reinforcement learning for fast disease diagnosis.  _Advances in neural information processing systems_ 31 (2018).
  * [41] Peng, B. _et al._ Adversarial advantage actor-critic model for task-completion dialogue policy learning.  In _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 6149–6153 (IEEE, 2018).
  * [42] Xu, L. _et al._ End-to-end knowledge-routed relational dialogue system for automatic diagnosis.  In _Proceedings of the AAAI conference on artificial intelligence_ , vol. 33, 7346–7353 (2019).
  * [43] Xia, Y., Zhou, J., Shi, Z., Lu, C. & Huang, H.  Generative adversarial regularized mutual information policy gradient framework for automatic diagnosis.  In _Proceedings of the AAAI conference on artificial intelligence_ , vol. 34, 1062–1069 (2020).
  * [44] Zhang, X. _et al._ Evaluating the performance of large language models on gaokao benchmark.  _arXiv preprint arXiv:2305.12474_ (2023).
  * [45] Sai, A. B., Mohankumar, A. K. & Khapra, M. M.  A survey of evaluation metrics used for nlg systems.  _ACM Computing Surveys (CSUR)_ 55, 1–39 (2022).
  * [46] Khurana, D., Koli, A., Khatter, K. & Singh, S.  Natural language processing: State of the art, current trends and challenges.  _Multimedia tools and applications_ 82, 3713–3744 (2023).
  * [47] Tran, K., Bisazza, A. & Monz, C.  Recurrent memory networks for language modeling.  _arXiv preprint arXiv:1601.01272_ (2016).
  * [48] Plank, B., Alonso, H. M., Agić, Ž., Merkler, D. & Søgaard, A.  Do dependency parsing metrics correlate with human judgments?  In _Proceedings of the Nineteenth Conference on Computational Natural Language Learning_ , 315–320 (2015).
  * [49] Hugging Face – The AI community building the future.  _https://huggingface.co/_ (July 12, 2023).
  * [50] Cao, Y. T. _et al._ On the intrinsic and extrinsic fairness evaluation metrics for contextualized language representations.  _arXiv preprint arXiv:2203.13928_ (2022).
  * [51] Toma, A. _et al._ Clinical camel: An open-source expert-level medical language model with dialogue-based knowledge encoding.  _arXiv preprint arXiv:2305.12031_ (2023).
  * [52] Mihaylov, T., Clark, P., Khot, T. & Sabharwal, A.  Can a suit of armor conduct electricity? a new dataset for open book question answering.  _arXiv preprint arXiv:1809.02789_ (2018).
  * [53] Jin, D. _et al._ What disease does this patient have? A large-scale open domain question answering dataset from medical exams.  _Applied Sciences_ 11, 6421 (2021).
  * [54] Choi, E. _et al._ QuAC: Question answering in context.  _arXiv preprint arXiv:1808.07036_ (2018).
  * [55] Clark, C. _et al._ Boolq: Exploring the surprising difficulty of natural yes/no questions.  _arXiv preprint arXiv:1905.10044_ (2019).
  * [56] Kwiatkowski, T. _et al._ Natural questions: a benchmark for question answering research.  _Transactions of the Association for Computational Linguistics_ 7, 453–466 (2019).
  * [57] Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A. & Choi, Y.  Hellaswag: Can a machine really finish your sentence?  _arXiv preprint arXiv:1905.07830_ (2019).
  * [58] Nallapati, R., Zhou, B., Gulcehre, C., Xiang, B. _et al._ Abstractive text summarization using sequence-to-sequence rnns and beyond.  _arXiv preprint arXiv:1602.06023_ (2016).
  * [59] Hermann, K. M. _et al._ Teaching machines to read and comprehend.  _Advances in neural information processing systems_ 28 (2015).
  * [60] Narayan, S., Cohen, S. B. & Lapata, M.  Don’t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization.  _arXiv preprint arXiv:1808.08745_ (2018).
  * [61] Warstadt, A. _et al._ Blimp: The benchmark of linguistic minimal pairs for english.  _Transactions of the Association for Computational Linguistics_ 8, 377–392 (2020).
  * [62] Gao, L. _et al._ The pile: An 800gb dataset of diverse text for language modeling.  _arXiv preprint arXiv:2101.00027_ (2020).
  * [63] Greenbaum, S.  Ice: The international corpus of english.  _English Today_ 7, 3–7 (1991).
  * [64] Blodgett, S. L., Green, L. & O’Connor, B.  Demographic dialectal variation in social media: A case study of african-american english.  _arXiv preprint arXiv:1608.08868_ (2016).
  * [65] Petroni, F. _et al._ Language models as knowledge bases?  _arXiv preprint arXiv:1909.01066_ (2019).
  * [66] Kočiskỳ, T. _et al._ The narrativeqa reading comprehension challenge.  _Transactions of the Association for Computational Linguistics_ 6, 317–328 (2018).
  * [67] Aryan, A., Nain, A. K., McMahon, A., Meyer, L. A. & Sahota, H. S.  The costly dilemma: Are large language models the pay-day loans of machine learning? (2023).
  * [68] Miller, A. H. _et al._ Parlai: A dialog research software platform.  _arXiv preprint arXiv:1705.06476_ (2017).
  * [69] Sarlin, P.-E., DeTone, D., Malisiewicz, T. & Rabinovich, A.  Superglue: Learning feature matching with graph neural networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 4938–4947 (2020).
  * [70] Hendrycks, D. _et al._ Measuring massive multitask language understanding.  _arXiv preprint arXiv:2009.03300_ (2020).
  * [71] Ghazal, A. _et al._ Bigbench: Towards an industry standard benchmark for big data analytics.  In _Proceedings of the 2013 ACM SIGMOD international conference on Management of data_ , 1197–1208 (2013).
  * [72] Wang, A. _et al._ GLUE: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_ (2018).
  * [73] Su, L. _et al._ An adaptive framework for conversational question answering.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 33, 10041–10042 (2019).
  * [74] Jain, N. _et al._ Bring your own data! self-supervised evaluation for large language models.  _arXiv preprint arXiv:2306.13651_ (2023).
  * [75] Wang, J. _et al._ On the robustness of chatgpt: An adversarial and out-of-distribution perspective.  _arXiv preprint arXiv:2302.12095_ (2023).
  * [76] Yuan, L. _et al._ Revisiting out-of-distribution robustness in nlp: Benchmark, analysis, and llms evaluations.  _arXiv preprint arXiv:2306.04618_ (2023).
  * [77] Bajaj, P. _et al._ Metro: Efficient denoising pretraining of large scale autoencoding language models with model generated signals.  _arXiv preprint arXiv:2204.06644_ (2022).
  * [78] Ahuja, K. _et al._ Mega: Multilingual evaluation of generative ai.  _arXiv preprint arXiv:2303.12528_ (2023).
  * [79] Zhu, K. _et al._ Promptbench: Towards evaluating the robustness of large language models on adversarial prompts.  _arXiv preprint arXiv:2306.04528_ (2023).
  * [80] Wang, X. _et al._ Textflint: Unified multilingual robustness evaluation toolkit for natural language processing.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations_ , 347–355 (2021).
  * [81] Huang, H. _et al._ Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting.  _arXiv preprint arXiv:2305.07004_ (2023).
  * [82] Yu, J. _et al._ Kola: Carefully benchmarking world knowledge of large language models.  _arXiv preprint arXiv:2306.09296_ (2023).
  * [83] Wang, Y. _et al._ Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization.  _arXiv preprint arXiv:2306.05087_ (2023).
  * [84] Bommasani, R., Liang, P. & Lee, T.  Holistic evaluation of language models.  _Annals of the New York Academy of Sciences_ (2023).
  * [85] Zhong, W. _et al._ Ar-lsat: Investigating analytical reasoning of text. arxiv e-prints, page.  _arXiv preprint arXiv:2104.06598_ (2021).
  * [86] Suzgun, M., Gehrmann, S., Belinkov, Y. & Shieber, S. M.  Lstm networks can perform dynamic counting.  _arXiv preprint arXiv:1906.03648_ (2019).
  * [87] Wu, Y. _et al._ Lime: Learning inductive bias for primitives of mathematical reasoning.  In _International Conference on Machine Learning_ , 11251–11262 (PMLR, 2021).
  * [88] Weston, J. _et al._ Towards ai-complete question answering: A set of prerequisite toy tasks.  _arXiv preprint arXiv:1502.05698_ (2015).
  * [89] Konda, P. _et al._ Magellan: toward building entity matching management systems over data science stacks.  _Proceedings of the VLDB Endowment_ 9, 1581–1584 (2016).
  * [90] Mei, Y. _et al._ Capturing semantics for imputation with pre-trained language models.  In _2021 IEEE 37th International Conference on Data Engineering (ICDE)_ , 61–72 (IEEE, 2021).
  * [91] Chen, M. _et al._ Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_ (2021).
  * [92] Hendrycks, D. _et al._ Measuring coding challenge competence with apps.  _arXiv preprint arXiv:2105.09938_ (2021).
  * [93] Hendrycks, D. _et al._ Measuring mathematical problem solving with the math dataset.  _arXiv preprint arXiv:2103.03874_ (2021).
  * [94] Cobbe, K. _et al._ Training verifiers to solve math word problems.  _arXiv preprint arXiv:2110.14168_ (2021).
  * [95] Gehman, S., Gururangan, S., Sap, M., Choi, Y. & Smith, N. A.  Realtoxicityprompts: Evaluating neural toxic degeneration in language models.  _arXiv preprint arXiv:2009.11462_ (2020).
  * [96] Lin, S., Hilton, J. & Evans, O.  Truthfulqa: Measuring how models mimic human falsehoods.  _arXiv preprint arXiv:2109.07958_ (2021).
  * [97] Svikhnushina, E., Filippova, A. & Pu, P.  ieval: Interactive evaluation framework for open-domain empathetic chatbots.  In _Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , 419–431 (2022).
  * [98] Dhamala, J. _et al._ Bold: Dataset and metrics for measuring biases in open-ended language generation.  In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_ , 862–872 (2021).
  * [99] Parrish, A. _et al._ Bbq: A hand-built bias benchmark for question answering.  _arXiv preprint arXiv:2110.08193_ (2021).
  * [100] Lukas, N. _et al._ Analyzing leakage of personally identifiable information in language models.  _arXiv preprint arXiv:2302.00539_ (2023).
  * [101] Carlini, N. _et al._ Extracting training data from large language models.  In _30th USENIX Security Symposium (USENIX Security 21)_ , 2633–2650 (2021).
  * [102] Nangia, N., Vania, C., Bhalerao, R. & Bowman, S. R.  Crows-pairs: A challenge dataset for measuring social biases in masked language models.  _arXiv preprint arXiv:2010.00133_ (2020).
  * [103] Wahde, M. & Virgolin, M.  The five is: Key principles for interpretable and safe conversational ai.  In _2021 The 4th International Conference on Computational Intelligence and Intelligent Systems_ , 50–54 (2021).
  * [104] Dwork, C., Hardt, M., Pitassi, T., Reingold, O. & Zemel, R.  Fairness through awareness.  In _Proceedings of the 3rd innovations in theoretical computer science conference_ , 214–226 (2012).
  * [105] Zhou, L., Gao, J., Li, D. & Shum, H.-Y.  The design and implementation of xiaoice, an empathetic social chatbot.  _Computational Linguistics_ 46, 53–93 (2020).
  * [106] Chen, B. _et al._ Codet: Code generation with generated tests.  _arXiv preprint arXiv:2207.10397_ (2022).
  * [107] Liu, H. _et al._ Evaluating the logical reasoning ability of chatgpt and gpt-4.  _arXiv preprint arXiv:2304.03439_ (2023).
  * [108] Borkan, D., Dixon, L., Sorensen, J., Thain, N. & Vasserman, L.  Nuanced metrics for measuring unintended bias with real data for text classification.  In _Companion proceedings of the 2019 world wide web conference_ , 491–500 (2019).
  * [109] Maas, A. _et al._ Learning word vectors for sentiment analysis.  In _Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies_ , 142–150 (2011).
  * [110] Lin, C.-Y.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , 74–81 (2004).
  * [111] Zhuo, T. Y., Huang, Y., Chen, C. & Xing, Z.  Exploring ai ethics of chatgpt: A diagnostic analysis.  _arXiv preprint arXiv:2301.12867_ (2023).
  * [112] Jin, Z. _et al._ When to make exceptions: Exploring language models as accounts of human moral judgment.  _Advances in neural information processing systems_ 35, 28458–28473 (2022).
  * [113] Pan, A. _et al._ Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark.  In _International Conference on Machine Learning_ , 26837–26867 (PMLR, 2023).
  * [114] Forbes, M., Hwang, J. D., Shwartz, V., Sap, M. & Choi, Y.  Social chemistry 101: Learning to reason about social and moral norms.  _arXiv preprint arXiv:2011.00620_ (2020).
  * [115] AI Risk Management Framework — nist.gov.  https://www.nist.gov/itl/ai-risk-management-framework.  [Accessed 28-07-2023].
  * [116] Napoles, C., Van Durme, B. & Callison-Burch, C.  Evaluating sentence compression: Pitfalls and suggested remedies.  In _Proceedings of the Workshop on Monolingual Text-To-Text Generation_ , 91–97 (2011).
  * [117] Shichel, Y., Kalech, M. & Tsur, O.  With measured words: Simple sentence selection for black-box optimization of sentence compression algorithms.  _arXiv preprint arXiv:2101.10096_ (2021).
  * [118] Jin, Z. _et al._ Can large language models infer causation from correlation?  _arXiv preprint arXiv:2306.05836_ (2023).
  * [119] McKenna, N. _et al._ Sources of hallucination by large language models on inference tasks.  _arXiv preprint arXiv:2305.14552_ (2023).
  * [120] Guerreiro, N. M. _et al._ Hallucinations in large multilingual translation models.  _arXiv preprint arXiv:2303.16104_ (2023).
  * [121] Dziri, N., Milton, S., Yu, M., Zaiane, O. & Reddy, S.  On the origin of hallucinations in conversational models: Is it the datasets or the models?  _arXiv preprint arXiv:2204.07931_ (2022).
  * [122] Bang, Y. _et al._ A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.  _arXiv preprint arXiv:2302.04023_ (2023).
  * [123] Glaese, A. _et al._ Improving alignment of dialogue agents via targeted human judgements.  _arXiv preprint arXiv:2209.14375_ (2022).
  * [124] Chen, S., Gao, S. & He, J.  Evaluating factual consistency of summaries with large language models.  _arXiv preprint arXiv:2305.14069_ (2023).
  * [125] Gekhman, Z., Herzig, J., Aharoni, R., Elkind, C. & Szpektor, I.  Trueteacher: Learning factual consistency evaluation with large language models.  _arXiv preprint arXiv:2305.11171_ (2023).
  * [126] Manakul, P., Liusie, A. & Gales, M. J.  Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.  _arXiv preprint arXiv:2303.08896_ (2023).
  * [127] Laban, P. _et al._ Llms as factual reasoners: Insights from existing benchmarks and beyond.  _arXiv preprint arXiv:2305.14540_ (2023).
  * [128] Zhao, W. X. _et al._ A survey of large language models.  _arXiv preprint arXiv:2303.18223_ (2023).
  * [129] Privacy Framework — nist.gov.  https://www.nist.gov/privacy-framework.  [Accessed 28-07-2023].
  * [130] Marks, M. & Haupt, C. E.  Ai chatbots, health privacy, and challenges to hipaa compliance.  _JAMA_ (2023).
  * [131] Schwartz, R. _et al._ Towards a standard for identifying and managing bias in artificial intelligence.  _NIST special publication_ 1270 (2022).
  * [132] Broniatowski, D. A. _et al._ Psychological foundations of explainability and interpretability in artificial intelligence.  _NIST, Tech. Rep_ (2021).
  * [133] Welivita, A. & Pu, P.  A taxonomy of empathetic response intents in human social conversations.  _arXiv preprint arXiv:2012.04080_ (2020).
  * [134] Ilicki, J.  A framework for critically assessing chatgpt and other large language artificial intelligence model applications in health care.  _Mayo Clinic Proceedings: Digital Health_ 1, 185–188 (2023).
  * [135] Meng, J. & Dai, Y.  Emotional support from ai chatbots: Should a supportive partner self-disclose or not?  _Journal of Computer-Mediated Communication_ 26, 207–222 (2021).
  * [136] Oniani, D. _et al._ Toward improving health literacy in patient education materials with neural machine translation models.  _arXiv preprint arXiv:2209.06723_ (2022).
  * [137] Ahmad, M. A., Patel, A., Eckert, C., Kumar, V. & Teredesai, A.  Fairness in machine learning for healthcare.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, 3529–3530 (2020).
  * [138] Ahmad, M. A. _et al._ Fairness in healthcare ai.  In _2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)_ , 554–555 (IEEE, 2021).
  * [139] Hague, D. C.  Benefits, pitfalls, and potential bias in health care ai.  _North Carolina medical journal_ 80, 219–223 (2019).
  * [140] Hariri, W.  Unlocking the potential of chatgpt: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing.  _arXiv preprint arXiv:2304.02017_ (2023).
  * [141] Cook, D. A. & Skrupky, L. P.  Measuring personalization, embodiment, and congruence in online learning: A validation study.  _Academic Medicine_ 10–1097 (2022).
  * [142] OpenAI.  Gpt-4 technical report (2023).  2303.08774.
  * [143] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding (2019).  1810.04805.
  * [144] Lee, J. _et al._ BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ 36, 1234–1240 (2019).  URL https://doi.org/10.1093%2Fbioinformatics%2Fbtz682.
  * [145] Zhuang, B. _et al._ A survey on efficient training of transformers (2023).  2302.01107.
  * [146] Solatorio, A. V. & Dupriez, O.  Realtabformer: Generating realistic relational and tabular data using transformers (2023).  2302.02041.
  * [147] Text REtrieval Conference (TREC) Home Page — trec.nist.gov.  https://trec.nist.gov/.  [Accessed 28-07-2023].
  * [148] Zhou, Y. _et al._ Large language models are human-level prompt engineers.  _arXiv preprint arXiv:2211.01910_ (2022).
  * [149] Ge, Y. _et al._ Openagi: When llm meets domain experts.  _arXiv preprint arXiv:2304.04370_ (2023).
  * [150] Pratt, S., Liu, R. & Farhadi, A.  What does a platypus look like? generating customized prompts for zero-shot image classification.  _arXiv preprint arXiv:2209.03320_ (2022).
  * [151] Reddy, S.  Evaluating large language models for use in healthcare: A framework for translational value assessment.  _Informatics in Medicine Unlocked_ 101304 (2023).