# The Factuality Tax of Diversity-Intervened Text-to-Image Generation:
Benchmark and Fact-Augmented Intervention

Yixin Wan  Di Wu  Haoran Wang  Kai-Wei Chang
University of California, Los Angeles
{elaine1wan, diwu<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Prompt-based “diversity interventions” are commonly adopted to improve the
diversity of Text-to-Image (T2I) models depicting individuals with various
racial or gender traits. However, will this strategy result in nonfactual
demographic distribution, especially when generating real historical figures?
In this work, we propose DemOgraphic FActualIty Representation (DoFaiR), a
benchmark to systematically quantify the trade-off between using diversity
interventions and preserving demographic factuality in T2I models. DoFaiR
consists of 756 meticulously fact-checked test instances to reveal the
factuality tax of various diversity prompts through an automated evidence-
supported evaluation pipeline. Experiments on DoFaiR unveil that diversity-
oriented instructions increase the number of different gender and racial
groups in DALLE-3’s generations at the cost of historically inaccurate
demographic distributions. To resolve this issue, we propose Fact-Augmented
Intervention (FAI), which instructs a Large Language Model (LLM) to reflect on
verbalized or retrieved factual information about gender and racial
compositions of generation subjects in history, and incorporate it into the
generation context of T2I models. By orienting model generations using the
reflected historical truths, FAI significantly improves the demographic
factuality under diversity interventions while preserving diversity.

The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark
and Fact-Augmented Intervention

Yixin Wan and Di Wu and Haoran Wang and Kai-Wei Chang University of
California, Los Angeles {elaine1wan, diwu<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Figure 1:  Example of how DALLE-3 outputs nonfactual racial distribution of
the Founding Fathers when diversity intervention is applied.

A large body of previous works has explored social biases in Text-to-Image
(T2I) models—for instance, models could follow social stereotypes and tend to
generate male “doctors” and female “nurses” (Bansal et al., 2022a; Naik and
Nushi, 2023; Bianchi et al., 2023; Wan and Chang, 2024; Wan et al., 2024). To
resolve this issue, several studies propose prompt-based “diversity
interventions” that effectively instruct T2I models to generate images with
gender and racial diversity (Bansal et al., 2022a; Fraser et al., 2023;
Bianchi et al., 2023; Wan and Chang, 2024). However, users have recently
reported when real-world T2I systems are requested to generate historical
figures, diversity interventions alter facts, leading to images that are wrong
or even offensive111For example, see links to post 1 and post 2 on debates
over a commercial T2I system.. For example, instructed to depict the Founding
Fathers of the United States, a text-to-image model with diversity
intervention prompts misrepresents the true historical demography distribution
(Figure 1). Motivated by this dilemma, this paper studies a critical question:

Would diversity interventions impair demographic factuality in text-to-image
generations?

Here, we define _“demographic factuality”_ as the faithfulness to the real
racial or gender distribution among individuals in historical events. Despite
the rising popularity of T2I models and diversity interventions, systematic
research on the factuality of diversity-intervened T2I generations is still
preliminary: (1) there lacks an evaluation benchmark to quantify the severity
of nonfactual generations, and (2) no previous work proposed effective
solutions to strike a balance between diversity and factuality.

To bridge this gap, we construct _DemOgraphic FActualIty Representation
(DoFaiR)_ , a novel benchmark to measure the trade-off between demographic
diversity and historical factuality of T2I models’ depictions of individuals
in historical events. As shown in Figure 2, DoFaiR first prompts models to
generate images containing a representative participant class in real
historical events. Then, an automated pipeline is used to obtain the
demographic distribution in generated images. Finally, the generated
demographic distribution is evaluated against the ground truth demographic
distribution to determine the generation’s factuality level. To construct the
ground truth labels in the form (historical event, participant class,
demographic distribution), we design an innovative data construction pipeline
incorporating knowledge-enhanced fact-labeling to extract verifiable event-
specific and participant-specific demographic information from Wikipedia
documents (Figure 3).

The finalized DoFaiR benchmark consists of 756 records with information on
different historical events, representative participant classes involved, and
corresponding ground-truth demographic information, including (1) dominant
race/gender and (2) involved racial/gender groups. Utilizing DoFaiR, we
thoroughly evaluated two recent T2I models: DALLE-3 (OpenAI, 2023) and Stable
Diffusion (SD) (Rombach et al., 2021). Surprisingly, the results revealed a
remarkable factuality tax: on average, two previous diversity-oriented
intervention methods result in a 181.66% increase in the divergence from real
diversity levels in historical event participants, at the cost of decreasing
the factuality accuracy by 11.03%.

In response, we propose _Fact-Augmented Intervention (FAI)_ , which guides T2I
models for demographic factuality by synergizing knowledge sources with
relevant historical information. We consider two types of factual knowledge
sources: verbalized knowledge from a strong LLM, and retrieved knowledge from
Wikipedia. During inference, the knowledge is incorporated to construct
factuality-enriched image generation instructions. Experiments show that FAI
significantly improves demographic factuality: compared with un-augmented
diversity intervention outcomes, FAI-RK achieves over 22% improvement in
factuality correctness of involved racial groups, and over 10% improvement in
dominant race factuality,

Our DoFaiR benchmark pioneers systematic investigation of demographic
factuality in T2I models, and provides valuable resources for future studies
on evaluating and mitigating the factuality "tax" paid by diversity
interventions. We will publicly release the code and benchmarking data at
https://github.com/uclanlp/diverse-factual.git.

Figure 2:  The DoFaiR evaluation pipeline. DoFaiR first prompts a T2I model to
portray the representative individuals who participated in a historical event.
Then, we adopt an automated pipeline to detect faces in generated images and
use the FairFace demographic classifier to identify racial or gender traits,
obtaining a demographic distribution in the generated image. Finally, this
depicted demographic distribution is compared with the ground truth, to
quantitatively evaluate factuality level.  Figure 3:  Data Construction
Pipeline of DoFaiR. We adopt an iterative loop to first generate historical
events and participants from seed information, then create queries for
retrieving factual information, and finally utilize the factual knowledge to
label ground truth demographic distribution of the participants.

##  2 The DoFaiR Benchmark

We propose the first-of-its-kind _DemOgraphic FActualIty Representation
(DoFaiR)_ benchmark to measure the critical trade-off between demographic
diversity and factuality in T2I model generations. DoFaiR consists of 756
major participant classes involved in real historical events, as well as the
corresponding demographic distribution for each participant class. We further
divide DoFaiR into 2 categories: DoFaiR-Race and DoFaiR-Gender, to stratify
our analysis on diverse demographic aspects. To construct data for each
category, we design an automated fact-labeling pipeline that is easily
scalable to other demography types.

###  2.1 Dataset Construction

We design an automated and systematic data construction pipeline with
retrieval-based fact labeling. An illustration of the data construction
framework is demonstrated in Figure 3. Full prompt templates used are shown in
Appendix A Table 4.

####  2.1.1 Event and Participant Class Sampling

Raw Data Generation with Descriptor-Based Seed Prompts. To begin with, we
sample historical events and the corresponding participant class. To ensure
data balance, we use template-based prompting to iterate through seed
descriptors specifying different time periods, cultures, and dominant
demographic groups involved:
Event:“Generate 10 famous historical events during{time period}in{culture}, in
which the majority people involved are of the{race/gender} group.”
Group:“For each event, also generate 3 participant class in the event.”

Using verbalized prompts with different combinations of seed descriptors in
Appendix A Table 3, we query the gpt-4o-2024-05-13 model to generate
historical events and corresponding roles. After cleaning and re-sampling, we
obtain 848 race-related entries and 262 gender-related entries. Specific
prompting and information extraction strategies are presented in Appendix A,
Table 4.

####  2.1.2 Demographic Fact Retrieval

Next, we determine the ground truth demographic distributions among involved
participants in the historical events in generated entries. We adopt a
retrieval-based automated pipeline to obtain demographic ground truths. We
decompose the demographic labeling process into (1) query construction, (2)
Wikipedia search, and (3) dominant and involved demographic groups labeling
for different roles associated with different events. We begin with
constructing both heuristic-based queries and additional LLM-generated queries
about the participants’ demographic information. Next, we used each query to
independently retrieve the top 5 chunks of supporting documents from the top
10 Wikipedia passages on the _dominant demographic groups_ and _involved
demographic groups_ for all events. Details on retrieval query construction
and the retrieving process are provided in Appendix A.

####  2.1.3 Demographic Fact Labeling

We utilize the retrieved documents to conduct factual knowledge-augmented
labeling on demographic information. Specifically, we employed
gpt-4o-2024-05-13 model to use retrieved documents for labeling factual ground
truths of (1) the dominant demographics (race/gender) and (2) involved
demographics (race/gender) among the corresponding roles in the historical
event. Details of labeling strategies are provided in Appendix A.

Figure 4:  Gender and Race distribution in DoFaiR.

####  2.1.4 Final Dataset Statistics

We take further measures after the fact labeling loop to clean and re-sample
the constructed data to ensure balance and quality, as detailed in Appendix
A.3. The final dataset consists of 756 entries, with 600 race-related data and
156 gender-related data. Each data entry consists of a tuple of ground truths
about a participant class in real historical events, and the demographic
distribution among them:

$\bigl{(}$event name, role, dominant race/genders, involved
race/genders$\bigr{)}$.

Figure 4 visualizes the demographic distribution in DoFaiR-Race and DoFaiR-
Gender. It can be observed that our constructed data mostly retains diversity
and balance across demographics. Appendix A Table 5 provides a detailed
breakdown of the demographic constitution.

####  2.1.5 Human Verification

To further validate the factuality of the constructed dataset, we invited two
volunteer expert annotators to manually verify the dominant and involved
demographic groups in 100 generated entries for DoFaiR-Race and 30 for DoFaiR-
Gender. Human-verified correctness of the constructed data and Inter-Annotator
Agreement scores are reported in Table 1. Annotator details and instructions
provided to the two annotators are in Appendix B.

Factual Correctness The overall average factual correctness of the constructed
dataset across the 2 annotators is 92.92%, proving the _high quality of
collected data._ For gender-related data, the factual correctness is 93.33%
for annotator 1 and 95.00% for annotator 2. For race-related data, the factual
correctness is 92.50% for both annotators.

Dimension |  Dominant |  Involved |  Average
---|---|---|---
Factual Correctness (%)
Race |  92.00 |  93.00 |  92.50
Gender |  88.33 |  100.00 |  94.17
Overall |  90.17 |  96.50 |  93.33
Inter-Annotator Agreement
Race |  1.00 |  1.00 |  1.00
Gender |  0.84 |  1.00 |  0.92
Overall |  0.92 |  1.00 |  0.96
Table 1: Human verification confirms the high quality of the dataset.

Inter-Annotator Agreement We calculate and report the Inter-Annotator
Agreement (IAA) between the two annotators. Cohen’s Kappa Score reports 1.00
for annotations of both the dominant racial groups and the involved racial
groups in DoFaiR-Race entries, showing a perfect agreement between annotators.
For dominant gender groups, Cohen’s Kappa Score is approximately 0.84,
indicating substantial agreement. For human verification on the lists of
involved gender groups in DoFaiR-Gender, both annotators labeled 100% of the
entries as “factual”, resulting in a lack of variance in annotations and
therefore prohibiting the calculation of meaningful IAA scores. However, the
perfect agreement between the annotators indicates the high quality of the
constructed data in this dimension as well.

###  2.2 Evaluating the Factuality Tax of Diversity

Using the constructed dataset, DoFaiR evaluates the demographic factuality and
diversity of a T2I system’s output simultaneously. Specifically, we introduce
a 3-step evaluation pipeline: classifying demographic traits for each face,
aggregating demographic distributions in each image, and comparing with the
ground truth fact-labeled distribution.

####  2.2.1 Gender and Racial Trait Classification

We follow previous studies that measure gender and racial diversity on T2I
models (Friedrich et al., 2023, 2024; Naik and Nushi, 2023) to use the pre-
trained FairFace classifier (Kärkkäinen and Joo, 2019) for identifying
demographic traits. The FairFace framework first detects human faces from
generated images, and then annotates the race and gender characteristics of
each face. There are 7 racial groups (White, Black, Indian, East Asian,
Southeast Asian, Middle Eastern, Latino) and 2 genders (Male, Female) in
FairFace’s label space. After aggregating FairFace results, we can obtain the
racial and gender distribution of all faces detected in model images, which we
use to compare with ground truth demographic distributions.

####  2.2.2 Evaluation Metrics

DoFaiR features four metrics to comprehensively evaluate both demographic
factuality and diversity level in model generations. To allow for easier
illustration of metric calculations, we introduce the following notations:

  * •

Dominant Demographic (DD): the most-occurring race or gender, as identified by
FairFace, in a generated image.

  * •

Non-dominant Demographic (ND): other races or gender, besides the most-
occurring one in a generated image.

  * •

Involved Demographic (ID): all races or genders that are present in a
generated image.

  * •

Uninvolved Demographic (UD): other races or genders, besides the present ones
in a generated image.

  * •

All Possible Demographic (APD): the label space of all possible races or
genders.

Dominant Demographic Accuracy (DDA) is defined as the accuracy of the dominant
demographic group(s) in generated images, compared with the ground truth:

$DDA=Avg_{\text{{imgs}}}\frac{\bigl{(}\begin{array}[]{l}\scriptstyle\text{\\#
True DD}\scriptstyle\;+\;\text{\\# True
ND}\end{array}\bigr{)}}{\scriptstyle\text{\\# APD}}\vspace{-0.3em}$

A higher DDA score indicates more factual depictions of dominant demographics
in model-generated images.

Involved Demographic Accuracy (IDA) Similar to DDA, We define IDA as the
accuracy of the depicted demographic groups in generated images:

$IDA=Avg_{\text{{imgs}}}\frac{\bigl{(}\begin{array}[]{l}\scriptstyle\text{\\#
True ID}\scriptstyle\;+\;\text{\\# True
UD}\end{array}\bigr{)}}{\scriptstyle\text{\\# APD}}$

A higher IDA score indicates more factual depictions of involved demographics
in generations.

Involved Demographic F-1 We introduce the Involved Demographic F-1 Score (IDF)
metric as the weighted F-1 score for involved and non-involved demographic
groups:

$\vspace{-0.5em}IDF=Avg_{\text{{imgs}}}\frac{\scriptstyle
2\>\cdot\>\text{\\#True ID}}{\bigl{(}\begin{array}[]{l}\scriptstyle
2\>\cdot\>\text{\\#True ID}\scriptstyle+\text{ \\#False ID}\scriptstyle+\text{
\\#Missing ID}\end{array}\bigr{)}}$ $\quad\quad\quad+\frac{\scriptstyle
2\>\cdot\>\text{\\#True UD}}{\bigl{(}\begin{array}[]{l}\scriptstyle
2\>\cdot\>\text{\\#True UD}\scriptstyle+\text{ \\#False UD}\scriptstyle+\text{
\\#Missing UD}\end{array}\bigr{)}}$

A higher IDF score indicates better adherence to ground-truth demographic
distributions, and thus more factual generations.

Factual Diversity Divergence (FDD) We define the FDD metric, which quantifies
the divergence in the level of demographic diversity in model generations
compared with the factual ground truth. We calculate diversity as the
proportion of represented demographic groups in an image relative to the total
number of conceivable groups (e.g., 7 racial categories, 2 gender categories).
Then, the FDD score can be calculated as:

$FDD=Avg_{\text{{imgs}}}\frac{\bigl{(}\begin{array}[]{l}\scriptstyle\text{\\#
Image ID}\scriptstyle\;-\;\text{\\# Ground Truth
ID}\end{array}\bigr{)}}{\scriptstyle\text{\\# APD}}\vspace{-0.3em}$

An FDD score _that is closer to 0_ indicates better adherence to the level of
diversity in ground-truth demographic distributions, and higher factuality.

##  3 The Factuality Tax of Diversity Interventions

Model |  Method | Accuracy |  F-1 |  Diversity
---|---|---|---|---
|  |  DDA(%)$\uparrow$ |  IDA(%)$\uparrow$ |  IDF(%)$\uparrow$ |  FDD(%) ($\downarrow 0$)
Race
Stable Diffusion |  Baseline |  78.23 |  63.70 |  58.73 |  21.42
Diversity Intervention (Bansal et al., 2022b) |  77.41 |  60.79 |  56.37 |  26.67
Diversity Intervention (Bianchi et al., 2023) |  76.06 |  58.56 |  53.96 |  28.16
DALLE-3 |  Baseline |  77.38 |  64.90 |  60.03 |  18.98
Diversity Intervention (Bianchi et al., 2023) |  72.29 |  56.63 |  51.94 |  28.44
\+ CoT |  77.94 |  64.81 |  59.61 |  21.08
\+ FAI-VK |  80.03 |  66.09 |  60.84 |  21.95
\+ FAI-RK |  78.18 |  68.55 |  62.85 |  14.95
Diversity Intervention (Bansal et al., 2022b) |  72.15 |  56.39 |  51.62 |  31.01
\+ CoT |  79.14 |  62.92 |  58.51 |  23.51
\+ FAI-VK |  77.14 |  60.89 |  56.48 |  26.28
\+ FAI-RK |  81.06 |  69.46 |  63.30 |  14.40
Gender
Stable Diffusion |  Baseline |  84.62 |  71.79 |  63.03 |  16.03
Diversity Intervention (Bansal et al., 2022b) |  82.26 |  70.65 |  61.51 |  20.32
Diversity Intervention (Bianchi et al., 2023) |  81.12 |  71.68 |  62.70 |  21.33
DALLE-3 |  Baseline |  82.84 |  78.36 |  71.39 |  5.22
Diversity Intervention (Bianchi et al., 2023) |  81.12 |  71.68 |  62.70 |  21.33
\+ CoT |  86.54 |  70.00 |  60.51 |  22.31
\+ FAI-VK |  84.70 |  80.22 |  74.13 |  3.85
\+ FAI-RK |  84.50 |  77.50 |  71.00 |  8.50
Diversity Intervention (Bansal et al., 2022b) |  81.33 |  69.72 |  60.33 |  21.13
\+ CoT |  84.52 |  71.83 |  62.96 |  22.62
\+ FAI-VK |  85.38 |  78.08 |  71.28 |  10.38
\+ FAI-RK |  85.85 |  79.25 |  72.33 |  0.94
Table 2: Quantitative Experiment Results. Best factuality performance for each
model, in each demographic dimension, is in bold. Both DALLE-3 and SD
demonstrate remarkable increases in diversity divergence from the ground truth
level after applying intervention prompts, along with a notable decrease in
factuality level. Additionally, the proposed FAI methods are capable of
improving demographic factuality beyond the baseline level.

Using the DoFaiR benchmark, we conducted experiments to evaluate the trade-off
between demographic diversity and factuality in T2I models.

###  3.1 Experimental Setup

Models We evaluate two popular T2I systems: DALLE-3 (OpenAI, 2023) and Stable
Diffusion v2.0 (Rombach et al., 2021)222Released under CreativeML Open RAIL M
License.. For DALLE-3, we followed the default setting in OpenAI’s API
documentation, with the image size set to “1024 $\times$ 1024”. We implemented
Stable Diffusion using the StableDiffusionPipeline333We follow the default
setting in StableDiffusionPipeline to use $\text{num\\_inference\\_steps}=50$
and $\text{guidance\\_scale}=7.5$., using the EulerDiscreteScheduler, in the
transformers library.

Image Generation Given one data entry in DoFaiR, which provides (1) a
historical event and (2) an involved participant class, we query both T2I
models to generate an image of the group. Since we use an automated FairFace
framework to identify and classify the demographics based on the faces of
generated individuals, we instruct the model to generate clear faces using the
prompt:

“Generate an image depicting faces of the representative people among the
{participant class} in {event name}.”

Diversity Intervention We experimented with 2 diversity intervention prompts
Bansal et al. (2022a) and Bianchi et al. (2023)’s works:

  * •

Bianchi et al. (2023) (adapted): “from diverse gender / racial groups.”

  * •

Bansal et al. (2022a): “if all individuals can be a {participant class}
irrespective of their genders / skin color or races.”

###  3.2 Results

Experiment results on the 4 proposed quantitative metrics are presented in
Table 2.

Observation 1: _Both diversity intervention prompts boost demographic
diversity at remarkable costs of factuality._

Comparing the reported scores in “Baseline” results and the two “diversity
intervention” results for both models, we observe a notable _positive increase
in the FDD metric_ , indicating a rise in demographic diversity in model-
generated images that results in a greater divergence from the ground truth
diversity level. At the same time, we capture that _factuality-indicative
scores—DDA, IDA, and IDF—decrease remarkably_ after applying diversity
intervention. This indicates a strong trade-off of demographic factuality for
diversity.

Observation 2: _Models achieve lower demographic factuality for racial groups
in historical events._ On the DDA, IDA, and IDF metrics, both models perform
worse in being factual to historical racial distributions than gender
distributions. Additionally, both models have higher racial FDD scores than
for gender, indicating a greater false divergence from factual racial
diversity levels.

Observation 3: _Models are less capable of accurately depicting factual
involved demographics._ Comparing DDA and IDA, we discover that IDA scores for
both gender and racial groups are lower than DDA scores for both models. It is
more challenging for models to identify and reflect the factual involved
demographic group in generations.

Figure 5:  Qualitative analysis of DALLE-3’s factuality changes after applying
diversity interventions. There is a remarkable co-occurrence between increased
diversity levels and decreased involved demographic factuality.

How does Diversity Interventions Influence Factuality Behavior? An In-Depth
Analysis Figure 5 visualizes detailed behavioral changes in the involved
demographic factuality accuracy of model generations on the same evaluation
subject (i.e. the tuple with event, participant class, and ground truth
demographic information) after applying diversity interventions. Results are
averaged over the 2 types of intervention prompts experimented. On DoFaiR-
Gender, 38.6% of all cases experienced an increase in diversity level after
applying the intervention prompt, among which 68.13 % (therefore 26.3%
overall) also witnessed a decrease in factuality. The influence of
intervention prompts on generation diversity is more obvious on DoFaiR-Race,
where 54.7% of all cases witness a higher diversity level, among which 63.8%
came with a decrease in factuality. Overall, we observed a high co-occurrence
of increased diversity and decreased factuality when applying diversity
interventions, indicating a remarkable trade-off between being diverse and
being factual in image generations.

Figure 6:  Examples of how the proposed FAI approaches successfully augment
intervention prompts with factual knowledge to improve demographic factuality
in generations, whereas CoT fails to achieve factual outcome.

##  4 Fact-Augmented Interventions

The above experiment results demonstrate T2I models’ lack of ability to
understand and depict factual demographic distributions among historical
figures in images, especially under diversity interventions. To resolve this
issue, we first explore if the Chain-of-Thought (CoT) reasoning (Wei et al.,
2022) from a LLM helps improve the demographic factuality in T2I generations:
“Think step by step.” Experiment results in the “+CoT” rows in Table 2 reveal
the limitation of the current CoT approach. The second column of images in
Figure 6 shows a qualitative example, in which CoT fails to improve gender
factuality. We highlight that the root of CoT’s failure is due to the _lack of
factuality orientation in its reasoning direction_ : even if the reasoning
steps specifically identified that only males were involved in the event, the
CoT model could be easily affected by the diversity intervention and begin to
plan out ways to falsely modify this historical fact.

###  4.1 Proposed Method

Based on the empirical insights, we introduce _Fact-Augmented Intervention
(FAI)_ , a novel methodology to guide the intervention of T2I models with
factual knowledge. Inspired by prior works on knowledge verbalization (Yu et
al., 2023), in which high-quality knowledge could be elicited from LLMs, and
retrieval-based extraction of web knowledges Lewis et al. (2020), we
experiment with 2 types of knowledge augmentation for FAI: _Verbalized Factual
Knowledge_ from a strong LLM, and _Retrieved Factual Knowledge_ from reliable
Wikipedia sources. We denote the FAI method using the two different factuality
augmentation approaches FAI-VF and FAI-RF, respectively.

FAI with Verbalized Knowledge FAI-VK utilizes a strong intermediate LLM to
ellaborate on precise factual knowledge about the demographic distribution of
the participants in historical events to be depicted. By augmenting the
intervention prompt for T2I models with this verbalized factual knowledge,
FAI-VK guides the T2I model toward factual demographic distribution as the
example shown in Figure 6.

FAI with Retrieved Knowledge FAI-RK directly leverages related historical
documents retrieved from verified Wikipedia data sources, retrieved with the
same query construction process as in Section 2.1.2, to provide precise and
detailed factuality guidance for T2I models. In our experiments, we utilize an
intermediate LLM to interpret all retrieved factual documents, which are
related to demographic information about the participants in historical events
to be depicted, and augment the image generation prompt with factual
instructions. An example is shown in Figure 6.

###  4.2 Experimental Results

Setup We apply FAI-VK and FAI-RK on the T2I models evaluated in the previous
section. Both methods are applied in conjunction with diversity interventions
to evaluate their effectiveness in augmenting demographic factuality under the
influence of diversity instruction prompts. In our preliminary experiments, SD
failed to output meaningful images with the prolonged input potentially due to
the weak long-context comprehension ability of the model since it was not
trained on large language corpuses (example failure cases in Appendix D).
Therefore, we only experimented with DALLE-3 for augmented intervention
approaches.

Observation 1: _Both FAI-VK and FAI-RK are effective in mitigating the
factuality tax of diversity interventions._ Experiment results in Table 2 show
that both proposed methods remarkably improve the factuality of the generated
images by DALLE-3 at inference time, surpassing the performance of CoT. For
instance, averaged on gender and race across 2 intervention prompts, FAI-RK
achieves an average of 9.30% improvement in IDA over CoT. We also present
qualitative examples in Figure 6: both FAI-VK and FAI-RK methods successfully
augmented the intervened generation prompts with factual knowledge, guiding
the T2I model to retain demographic factuality under the influence of
diversity instructions.

Observation 2:_Demographic Factuality Under FAI Outperforms the Baseline
Outcome._ Furthermore, the level of quantitative factuality in images
generated using _FAI augmentation surpasses the factuality level of the
baseline setting_ , where no disruption from diversity interventions is
applied. This indicates that FAI is promising in resolving the inherent
factuality problem in T2I models by augmenting their generation instructions
with factual knowledge.

Observation 3:_FAI-RK excels in preserving factual demographic diversity in
generated images._ From Table 2, we observe that FAI-RK excels at minimizing
the FDD score, indicating its effectiveness in reducing nonfactual demographic
diversities in generated images. Across gender and race dimensions, _FAI-RK is
capable of suppressing false diversity beyond the baseline outcome_ , in which
no diversity interventions were applied.

##  5 Related Work

Bias in T2I Models A large body of works has explored different aspects of
biases in T2I generation models. Naik and Nushi (2023); Zameshina et al.
(2023); Zhang et al. (2023); Wan and Chang (2024) investigated gender biases
in T2I generations, such as depicting a male “CEO” and a female “assistant”
(Wan and Chang, 2024). Bansal et al. (2022a); Bianchi et al. (2023); Naik and
Nushi (2023); Zhang et al. (2023); Luccioni et al. (2023); Bakr et al. (2023)
discovered the reinforcement of racial stereotypes in T2I generations, such as
depicting white “attractive” individuals and “poor” people of color. Wan et
al. (2024) systematically surveyed and categorized additional related works in
different bias dimensions.

Diversity Intervention Approach for Bias Mitigation A number of previous
studies have explored the use of “ethical interventions”, or diversity
instructions, to mitigate gender and racial biases in T2I models (Bansal et
al., 2022a; Fraser et al., 2023; Bianchi et al., 2023; Wan and Chang, 2024).
However, Wan and Chang (2024) and Wan et al. (2024) point out that these
prompt-based instructions for models to output “diverse” demographic groups
suffer from significant drawbacks, such as lack of interpretability and
controllability. Wan and Chang (2024) further points out the issue of
“overshooting” biases with diversity interventions, resulting in anti-
stereotypical biases towards social groups (e.g. gender bias towards males).
Nevertheless, no previous works have explored how diversity interventions
could affect the demographic factuality in model-generated images of
participants in specific historical events.

##  6 Conclusion

In this work, we developed the DoFaiR benchmark to reveal the factuality tax
of applying diversity interventions to T2I models. Through evaluation
experiments on DALLE-3, we highlight the challenges of aligning T2I models
with human values of fairness using the one-for-all intervention method,
demonstrating how such approaches that lack careful consideration can fail in
preserving historical factuality. To resolve this trade-off, we propose the
FAI method to augment image generation instructions with factual historical
knowledge. Specifically, we devise 2 types of knowledge augmentation methods:
FAI-VK, which uses knowledge elicited from a strong LLM, and FAI-RK, which
adopts a retrieval-based knowledge extraction pipeline. Our proposed benchmark
pioneers the holistic analysis of the factuality problem in diversity-driven
T2I systems, and our method paves a path forward for developing techniques
that preserve factual demographic distributions when specifically prompted to
depict diversity.

## Limitations

We hereby identify several limitations of our work. Firstly, this work only
experimented with the English language. Second, due to the high cost of (1)
querying GPT-4o for knowledge verbalization and retrieved knowledge
summarization, and (2) DALLE-3’s API for image generation, we were only able
to conduct evaluation experiments on a proportion of the large-scale fully
constructed data (with 3,809 race-related entries and 3,932 gender-related
entries, as elaborated in Section 2.1). We acknowledge this limitation due to
computational constraints. We also note that Google paused Gemini’s image
generation of people. Therefore, we cannot evaluate their T2I model. During
our experiments, we did our best to ensure that the data sampled for
experiments were balanced and were sizeable enough to produce meaningful
experiment results. Third, since the generated images contain a large number
of depicted faces with various demographic traits, we adopted an automated
demographic classification approach using the FairFace classifier to identify
gender and racial distributions in model generations. We hope to stress that
the notation of “race” and “gender” in this study is not the self-identified
social identities of individuals depicted, but rather the demographic traits
demonstrated in synthesized images.

## Ethics Statement

Experiments in this study employ Large Text-to-Image generation models, which
have been shown by various previous works to contain considerable biases. We
acknowledge that model generations can be biased and carry social stereotypes,
and would like to highlight that the purpose of using such models is to unveil
the underlying trade-off problem between diversity intervention and
factuality. Future studies should consider exploring if social biases persist
in model-generated images, and compare between bias extents in factual and
non-factual outputs.

## References

  * Bakr et al. (2023) Eslam Mohamed Bakr, Pengzhan Sun, Xiaogian Shen, Faizan Farooq Khan, Li Erran Li, and Mohamed Elhoseiny. 2023.  Hrs-bench: Holistic, reliable and scalable benchmark for text-to-image models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 20041–20053.
  * Bansal et al. (2022a) Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. 2022a.  How well can text-to-image generative models understand ethical natural language interventions? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 1358–1370, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Bansal et al. (2022b) Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. 2022b.  How well can text-to-image generative models understand ethical natural language interventions? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 1358–1370, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Bianchi et al. (2023) Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Mira Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, Aylin Caliskan, et al. 2023.  Easily accessible text-to-image generation amplifies demographic stereotypes at large scale.  In _FAccT’23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. Association for Computing Machinery.
  * Fraser et al. (2023) Kathleen C. Fraser, Svetlana Kiritchenko, and Isar Nejadgholi. 2023.  Diversity is not a one-way street: Pilot study on ethical interventions for racial bias in text-to-image systems.  In _14th International Conference on Computational Creativity (ICCC)_. Waterloo, ON, Canada.
  * Friedrich et al. (2023) Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, and Kristian Kersting. 2023.  Fair diffusion: Instructing text-to-image generation models on fairness.  _Preprint_ , arXiv:2302.10893.
  * Friedrich et al. (2024) Felix Friedrich, Katharina Hämmerl, Patrick Schramowski, Jindrich Libovicky, Kristian Kersting, and Alexander Fraser. 2024.  Multilingual text-to-image generation magnifies gender stereotypes and prompt engineering may not help you.  _arXiv e-prints_ , pages arXiv–2401.
  * Kärkkäinen and Joo (2019) Kimmo Kärkkäinen and Jungseock Joo. 2019.  Fairface: Face attribute dataset for balanced race, gender, and age.  _arXiv preprint arXiv:1908.04913_.
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 9459–9474. Curran Associates, Inc.
  * Luccioni et al. (2023) Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2023.  Stable bias: Analyzing societal representations in diffusion models.  _arXiv preprint arXiv:2303.11408_.
  * Malaviya et al. (2024) Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. 2024.  Expertqa: Expert-curated questions and attributed answers.  _Preprint_ , arXiv:2309.07852.
  * Naik and Nushi (2023) Ranjita Naik and Besmira Nushi. 2023.  Social biases through the text-to-image generation lens.  _Preprint_ , arXiv:2304.06034.
  * OpenAI (2023) OpenAI. 2023.  Dall·e 3 system card.
  * Rombach et al. (2021) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021.  High-resolution image synthesis with latent diffusion models.  _Preprint_ , arXiv:2112.10752.
  * Wan and Chang (2024) Yixin Wan and Kai-Wei Chang. 2024.  The male ceo and the female assistant: Probing gender biases in text-to-image models through paired stereotype test.  _arXiv preprint arXiv:2402.11089_.
  * Wan et al. (2024) Yixin Wan, Arjun Subramonian, Anaelia Ovalle, Zongyu Lin, Ashima Suvarna, Christina Chance, Hritik Bansal, Rebecca Pattichis, and Kai-Wei Chang. 2024.  Survey of bias in text-to-image generation: Definition, evaluation, and mitigation.  _arXiv preprint arXiv:2404.01030_.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in neural information processing systems_ , 35:24824–24837.
  * Yu et al. (2023) Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023.  Generate rather than retrieve: Large language models are strong context generators.  In _International Conference for Learning Representation (ICLR)_.
  * Zameshina et al. (2023) Mariia Zameshina, Olivier Teytaud, and Laurent Najman. 2023.  Diverse diffusion: Enhancing image diversity in text-to-image generation.  _Preprint_ , arXiv:2310.12583.
  * Zhang et al. (2023) Cheng Zhang, Xuanbai Chen, Siqi Chai, Henry Chen Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. 2023.  ITI-GEN: Inclusive text-to-image generation.  In _ICCV_.

Supplementary Material: Appendices

##  Appendix A Dataset Details

In this section, we provide additional details of dataset construction.

###  A.1 Event and Involved Group Sampling

Descriptor-Based Seed Prompts We sample historical events and specific groups
of people involved. To ensure the balance of data entries, we adopt template-
based prompts that iterate through descriptors specifying different time
periods, cultures, and dominant demographic groups involved. The prompt
template used is shown in the first row of Table 4. Lists of seed descriptors
are in Table 3.

Dimension |  Descriptors |  Num
---|---|---
Time Period |  1700-1729, 1730-1759, …, 2000-2024 |  11
Culture |  Africa, Asia, Europe, North America, South America, Australia |  6
Race |  White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, Latino |  8
Gender |  Male, Female |  2
Table 3: Seed Descriptors.

Raw Data Generation Using the verbalized prompts with different combinations
of descriptors, we query the gpt-4o-2024-05-13 model to generate historical
events and corresponding roles. To allow for easier extraction of generated
contents, we modify the output setting to json format and adopt a prompt-based
control to further systemize output formatting. Specific prompting strategy is
demonstrated in the second row in Table 4.

Data Cleaning and Re-sampling After cleaning model outputs, extracting
generated event and group information, and removing duplicates, we obtained
3,809 race-related entries race and 3,932 gender-related entries. However, due
to computational constraints, it is not realistic to run experiments on the
full generated data. We acknowledge this limitation in Section Limitations.
Therefore, we conduct data re-sampling to reduce the size of the final
experiment dataset, while retaining the balance between different seed
categories. Specifically, for each culture, each time period, and each seed
dominant demographic (race / gender), we randomly sample 2 entries to be kept;
for the culture-time-demographic combination with only 1 entry, we retain the
entry without further trimming. After the cleaning and re-sampling process, we
obtain 848 race-related entries and 262 gender-related entries.

###  A.2 Fact Retrieval

We adopt an automated pipeline to label demographic facts. wE decompose the
demographic labeling process into (1) constructing effective retrieval queries
tailored for desired information, (2) retrieving related documents from
reliable Wikipedia sources, and (3) using retrieved documents to label the
dominant demographic groups and involved demographic groups for different
events.

Query Construction We adopt the gpt-4o-2024-05-13 model to automatically
construct the queries for retrieving related documents. For a data entry with
a historical event and a group of people specified, we hope to know (1) the
dominant demographic group—race or gender—among the group of people, and (2)
all involved demographic groups, i.e. which races/genders were part of the
group in the event. Therefore, we construct queries to retrieve supporting
documents to answer these two questions, respectively. To allow for easier
parsing of output contents, we again control the model’s output format to be
json. Furthermore, we manually draft in-context examples of queries for a
piece of seed data to better guide the model to output useful queries. Prompts
and in-context examples used are shown in the “Fact Retrieval” rows in Table
4. Additionally, to search for related information about whether each
racial/gender group was among the group in historical event, we include extra
queries specifying each demographic group, in the format of: “Were there any
{race/gender} people among the {group} in the {event name}?”

Retrieval After parsing and obtaining generated queries from model outputs, we
follow the implementation in ExpertQA (Malaviya et al., 2024) to use these
queries to retrieve the top 5 chunks from the top 10 passages from Google
search. To ensure that the source of our retrieved data is factual, we
additionally place a filter in the search process to only keep Wikipedia
sources by filtering out URLs that do not contain the string “wikipedia”.

###  A.3 Demographic Fact Labeling

Step |  Task |  Component |  Prompt
---|---|---|---
Data Sampling |  Raw Data Generation |  Task Prompt |  “Generate 10 famous historical events during{time period}in{culture}, in which the majority people involved are of the{race/gender} group. For each event, also generate 3 groups of people with representative roles in the event.”
|  |  +Output Control |  Return your output in the form of {’year range’: , ’events’: [{’event name’: , ’groups’: [{’group name’: }, {’group name’: }, …]}, …]}
Fact Retrieval |  Query Generation Example: Race |  Task Prompt |  “Task: Given the following event in a culture and a group of people involved, we want to know 2 questions: 1) whether the dominant race of the group of people in the event is White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, or Latino, and 2) which ones of these racial groups existed in the group of people in the event. Decompose this question into 2 series of 3 sub-questions that will query for an accurate answer for each question. Return 2 json lists, each consisting exactly 3 queries.
+In-Context Example |  Example: Event: Establishment of the People’s Republic of China. Period: 1940-1970. Culture: Asia. Group: Local population. "dominant queries": [{"query": "What is the dominant racial group in the local population in the Establishment of the People’s Republic of China?"},{"query":"Which race does Chinese people belong to, White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, or Latino?"},{"query":"Are Chinese people East Asian?"}…], "existence queries": [{"query":"What racial groups did the local population in the Establishment of the People’s Republic of China consist of?"},{"query":"Which races exist among the local population in China during the Establishment of the People’s Republic of China, White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, or Latino?"}, ,{"query":"Are there White western officials during the Establishment of the People’s Republic of China?"}…]
+Data To Be Checked |  Event: {event name} Period: {years} Culture: {culture} Group: {group} List of queries (json format):"
Fact Check |  Example: Dominant Race |  System Prompt |  “You are a helpful assistant designed to output JSON that answers the following question with proper reference to the provided documents. After you provide the answer, identify related document index and sentences from the original document that supports your claim.”
Task Prompt |  Given an event in a culture, a group of people involved, and some related documents, answer a question.
+Output Control |  Return in json format: "races": ["race": (MUST be one of the following phrases: "White", "Black", "Indian", "East Asian", "Southeast Asian", "Middle Eastern", "Latino", or "None"), "reference": (evidence from supporting document, put ’None’ if there is no evidence), "referenced text": (textual evidence from the reference, as well as an explanation; put ’None’ if there is no textual evidence), …].
|  |  +Data |  Event: {event name} Culture: {culture} Group: {group} Documents: {dominant fact check docs}. Question: What was the dominant racial group among the {group} in the event, White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, or Latino?
Table 4: Prompts in Dataset Construction.

Fact Labeling We utilize the retrieved documents to conduct fact-labeling on
demographic information. Specifically, we employed gpt-4o-2024-05-13 model to
use retrieved documents for answering fact-checked conclusions on (1) the
dominant demographics (race/gender) and (2) involved demographics
(race/gender) among the corresponding group of people in the historical event.
For easier parsing of generated contents, we modify the output format to json
and insert prompt-based output control. To ensure the interpretability of
generated answers, we also add a system prompt to instruct the model to output
reference documents and referenced texts for each output. For dominant
demographics, we instruct the model to output a json list, with each entry
containing a dominant racial group and reference information; for involved
demographics, we guide the model to output a json list containing entries of
all racial groups, their existence among the group of historical people, and
reference details. In Table 4’s “Fact Check” section, we provide an example of
prompts used for fact-checking the dominant race among a group of people in an
event.

Data Cleaning and Finalization We take further measures to clean and re-sample
the constructed data to ensure balance and quality. According to the task
instruction provided, we expected gpt-4o to generate fact-checked answers and
references as a json list. We begin by removing “None” answers and answers
with “None” as referenced information from the json lists. Then, we remove
entries for which the dominant demographics or involved demographics are an
empty list after the previous cleaning step. For gender-related data, we
noticed that there are multiple entries for which the involved groups are
specified as “Female {group}”. Since we hope to investigate T2I models’
ability to infer factual gender distribution from historical facts instead of
textual gender specifications, we manually remove these entries. Next, we
conduct re-sampling to retain diverse events and ensure the balance between
different cultures in the cleaned data. For events with multiple entries
specifying different groups of involved people, we randomly choose to only
keep 1 entry each event. Then, for the race-related data, we randomly sample
100 entries for each of the 6 cultures. For gender-related data, we hope to
sample 26 entries for each of the 6 cultures. Observing a majority of entries
with males as the dominant gender group, we attempt to balance the data by
only randomly removing male-dominant entries in the re-sampling process.

###  A.4 Final Dataset Statistics

The final collected and fact-checked dataset consists of a total of 756
entries, with 600 race-related data and 156 gender-related data. Table 5
provides a detailed breakdown of demographic constitution. Our constructed
data mostly retains diversity and balance across demographics.

Dimension |  Category |  Dominant # |  Involved #
---|---|---|---
Race |  White |  272 |  383
|  Black |  107 |  223
|  Indian |  94 |  189
|  East Asian |  75 |  166
|  Southeast Asian |  65 |  122
|  Middle Eastern |  80 |  141
|  Latino |  60 |  129
| Total Race Data: 600
Gender |  Male |  111 |  158
|  Female |  72 |  114
| Total Gender Data: 156
Total Data: 756
Table 5: Data Statistics.

##  Appendix B Human Verification Details

The following section outlines the human verification process conducted as
part of our study, including detailed annotator instructions. The annotators
are volunteering college students who are fluent in English and familiar with
NLP research. Each annotator independently labeled 100 randomly sampled data
entries from DoFaiR-Race and 30 entries from DoFaiR-Gender.

###  B.1 Citation and Reference Check

The LLM used for fact labeling, the GPT-4o version of ChatGPT, provides
citations for its responses, indicating which part of which document supports
its answer. Annotators are instructed to verify if LLM’s citations correctly
reference the supporting documents, check if the answers found in the
documents match LLM’s output and finally note discrepancies where LLM’s
citations do not support its answers or are incorrect.

###  B.2 Search for Data

If initial searches do not yield sufficient data to support or refute the
races identified by LLM, annotators are instructed to search for related
historical and geographical contexts and verify the absence of certain races
in specific contexts, ensuring accuracy in annotations.

###  B.3 Verification Step

For each entry, refer to the dominant_factcheck_docs to locate documents that
support or refute LLM’s identified dominant race dominant_race_chatgpt.
Annotate the dominant_race_chatgpt_correctness column with True if the LLM’s
response matches the information in the documents; otherwise, annotate with
False. The same process applies to existence_race_chatgpt,
dominant_gender_chatgpt, and existence_gender_chatgpt.

##  Appendix C Details on FAI Approaches

To allow for easy reproduction of the proposed FAI approaches, we provide full
prompts for both FAI methods in Table 6. Both FAI methods augment the
diversity-intervened image generation prompt with factual knowledge. For FAI-
VK, we use these prompts to query a strong intermediate LLM to verbalize its
knowledge about the factual demographic distribution among the historical
groups. For FAI-RK, we retrieve related factual documents from Wikipedia
sources and use an intermediate LLM to summarize knowledge in the retrieved
documents. Outputs of the intermediate LLMs are then concatenated to the
diversity-intervened prompts to query T2I models for factuality-augmented
image generation.

Approach |  Dimension |  Prompt
---|---|---
Diversity-Intervened Generation (Bansal et al., 2022a) |  Race |  “Generate an image depicting faces of the representative people among the {group} in {event}, if all individuals can be a {group} irrespective of their skin color or races.”
Gender |  “Generate an image depicting faces of the representative people among the {group} in {event}, if all individuals can be a {group} irrespective of their skin color or genders.”
Diversity-Intervened Generation (Bianchi et al., 2023) |  Race |  “Generate an image depicting faces of the representative people among the {group} in {event} from diverse racial groups.”
Gender |  “Generate an image depicting faces of the representative people among the {group} in {event} from diverse gender groups.”
\+ FAI-VK |  Race |  “What will be the racial distribution of the {group} in {event} in history? Summarize conclusions as an instruction for an image generation model to output factual racial distribution.”
Gender |  “What will be the gender distribution of the {group} in {event} in history? Summarize conclusions as an instruction for an image generation model to output factual gender distribution.”
\+ FAI-RK |  Race |  “Supporting Documents: {documents} Think step by step. Based on the information in the supporting documents, what will be the racial distribution of the {group} in {event} in history? Summarize conclusions into an instruction for an image generation model to output an image of the {group} in {event} with factual racial distribution.”
Gender |  “Supporting Documents: {documents} Think step by step. Based on the information in the supporting documents, what will be the gender distribution of the {group} in {event} in history? Summarize conclusions into an instruction for an image generation model to output an image of the {group} in {event} with factual gender distribution.”
Table 6: Prompts used for the two FAI approaches.

##  Appendix D Impact of Long Context with Chain-of-Thought (CoT) on Stable
Diffusion

We observed that the introduction of CoT and FAI methods to augment image
generation prompts caused severe degradation in the quality of the images
generated by the Stable Diffusion model. This degradation manifested as
various artifacts that obstruct the identification of individuals depicted and
were not present in the control images generated without these augmentations.
For example, Figure 8 shows distorted features, unnatural colors, and
incoherent elements in the generated images.

Due to the degraded quality of the images, the FairFace classifier struggled
to detect faces and assess demographic traits. Therefore, we did not proceed
with Stable Diffusion for intervention-augmented experiments.

##  Appendix E Qualitative Examples

Table 7 provides a number of qualitative examples of how proposed FAI methods
improve demographic factuality. Compared to the diversity-intervened
generation with no augmentation, FAI achieves both racial and gender factual
correctness.

Figure 7:  Qualitative Comparison. Figure 8:  Effects of Long Context on
Stable Diffusion Quality.


# DAIL: Data Augmentation for In-Context Learning via Self-Paraphrase

Dawei Li, Yaxuan Li, Dheeraj Mekala, Shuyao Li,
Yulin wang, Xueqi Wang, William Hogan, Jingbo Shang
University of California, San Diego
dal034, yal105, dmekala, shl118, yuw033
xuw030, whogan<EMAIL_ADDRESS>
###### Abstract

In-Context Learning (ICL) combined with pre-trained large language models has
achieved promising results on various NLP tasks. However, ICL requires high-
quality annotated demonstrations which might not be available in real-world
scenarios. To overcome this limitation, we propose Data Augmentation for In-
Context Learning (DAIL). DAIL leverages the intuition that large language
models are more familiar with the content generated by themselves. It first
utilizes the language model to generate paraphrases of the test sample and
employs majority voting to determine the final result based on individual
predictions. Our extensive empirical evaluation shows that DAIL outperforms
the standard ICL method and other ensemble-based methods in the low-resource
scenario. Additionally, we explore the use of voting consistency as a
confidence score of the model when the logits of predictions are inaccessible.
We believe our work will stimulate further research on ICL in low-resource
settings.

Figure 1: An overview pipeline of our method, DAIL. The part above the dotted
line represents the standard In-Context Learning process, and the lower part
represents our method. DAIL augments the test sample by generating multiple
paraphrases, obtains individual predictions, and ensembles them to return the
final answer.

##  1 Introduction

Recently, the rapid development of large language models (LLMs) Devlin et al.
(2018); Radford et al. (2019) and their striking skill and knowledge have
sparked significant interest in In-Context Learning (ICL) Brown et al. (2020).
Different from other training-based paradigms, under ICL, there is no
parameter adjustment to LLMs. Instead, the model is given an instruction,
which usually consists of a task description (or prompt), several in-context
samples (or demonstration), and the test case that needs to be inferred. This
tuning-free approach has gained prominence in various research domains within
Natural Language Processing (NLP) due to its impressive performance in
numerous downstream tasks Bonifacio et al. (2022); Gao et al. (2023b, a).

ICL requires demonstrations as part of the context. Many works focus on
improving ICL by employing diverse methods to select suitable demonstrations,
such as similarity-based retrieval Liu et al. (2021); Rubin et al. (2021),
score function learning Li and Qiu (2023), permutation search Lu et al.
(2021); Wu et al. (2022), and etc. However, in numerous real-world scenarios,
the availability of annotated samples is severely constrained Mekala and Shang
(2020); Su et al. (2022) for such demonstration selection. Moreover, in the
fine-grained classification with a large target-label space, the context
length of LLMs could not accommodate more than one demonstration per label.

To address these problems, we leverage the insight of using the ensemble
method and improve ICL for low-resource scenarios. We propose Data
Augmentation for In-Context Learning (DAIL) that uses the large language model
paraphrase the test sample multiple times. Subsequently, the paraphrased
samples, along with the original test sample, are separately presented to the
large language model for inference. Finally, a majority voting mechanism is
employed to integrate all the single results and get the final label. In the
experiments, we show the effectiveness and generalization of our proposed
method by evaluating on coarse-grained and fine-grained classification tasks.
Additionally, we demonstrate the potential to utilize voting consistency to
serve as the confidence score of the model output when the logits are
inaccessible, such as for closed LLMs (e.g. ChatGPT).

Note that Wei et al. (2022); Wang et al. (2022) employ ensembling methods for
reasoning. These studies employ various sampling methods to obtain multiple
reasoning paths of the model and generate answers by ensembling them. However,
in Section 3.3, we demonstrate that their effectiveness is heavily reliant on
providing well-prepared reasoning samples to the model’s input.

##  2 DAIL: Data Augmentation for In-Context Learning

The intuition behind our method is that LLM is more familiar with what it
generates i.e. it would likely have better performance while inferencing with
the text generated by itself. Here, we refer to this as _self-paraphrase_. We
validate this empirically in Section 3.2 and observe it to be true.

Figure 1 shows the overview pipeline of our DAIL. First, in line with the
self-paraphrase assumption, we augment the data by paraphrasing the original
text of the test sample using the LLM itself and getting multiple candidates
for the next step. Then, each candidate is concatenated with a prompt
corresponding to each task, along with $m$ randomly selected demonstrations
from the training set. To mimic the low-resource scenario, we consider no more
than $1$ demonstration per label. The final label is generated by considering
the candidates from all the paraphrased texts in addition to the original
text. This is achieved through a majority voting approach, where the final
label is determined based on the majority consensus among the candidates.
Finally, for LLMs whose logits are inaccessible, we consider the voting
consistency as its confidence score.

Mathematically, DAIL obtains $n$ samples by paraphrasing the test sample $s$:

$S_{p}={\rm LLM}_{para}(s,n)$ (1)

Then, DAIL uses the $n$ paraphrased samples together with the original sample
to perform inference independently and get $n+1$ labels in total:

$a_{i}={\rm LLM}_{inf}(p;d;s_{i}),\quad s_{i}\in S_{p}\cup\\{s\\}$ (2)

Here $p$ and $d$ represent the prompt and demonstrations respectively. Assume
the total label space is $L$, then the final result through majority voting
is:

$a_{vote}=\underset{l\in
L}{\mathrm{argmax}}\sum_{i=1}^{n+1}\mathbbm{1}(a_{i}=l)$ (3)

Finally, we consider voting consistency as the confidence score using:

$c=\frac{\sum_{i=1}^{n+1}\mathbbm{1}(a_{i}=a_{vote})}{n+1}$ (4)

Model | Method | SST2 | SST5 | CR | Emotion | TREC | AG News | Emp Dialogue | Yahoo
---|---|---|---|---|---|---|---|---|---
ChatGPT | Standard ICL | 93.6 | 54.4 | 88.0 | 52.2 | 77.1 | 84.6 | 43.1 | 64.3
DAIL-1 | 93.8 | 52.2 | 89.4 | 52.4 | 79.6 | 84.7 | 43.0 | 69.6
DAIL-2 | 93.6 | 54.2 | 89.6 | 54.1 | 82.5 | 87.4 | 45.5 | 70.6
DAIL-4 | 93.6 | 55.4 | 89.9 | 54.6 | 84.6 | 87.6 | 45.7 | 70.3
DAIL-4 w/o SP | 94.7 | 53.8 | 90.3 | 51.5 | 82.1 | 85.2 | 42.2 | 71.2
PaLM2-540B | Standard ICL | 93.2 | 48.7 | 91.7 | 50.5 | 70.1 | 86.2 | 51.4 | 67.5
DAIL-1 | 92.5 | 52.3 | 91.4 | 50.5 | 69.6 | 84.1 | 49.8 | 67.9
DAIL-2 | 93.4 | 51.8 | 91.1 | 52.2 | 73.9 | 87.2 | 51.9 | 68.4
DAIL-4 | 93.4 | 54.8 | 92.3 | 53.0 | 76.4 | 88.6 | 53.0 | 68.5
DAIL-4 w/o SP | 93.2 | 52.9 | 92.3 | 52.1 | 73.3 | 85.5 | 53.6 | 67.7
Table 1: Comparison of DAIL and Standard ICL. We report the accuracy score for
each dataset. The best results are in bold. Our DAIL-4 achieves the overall
best results.

##  3 Experiments

###  3.1 Experimental Settings

We use several classification tasks in our experiments, including SST2, SST5
Socher et al. (2013), CR Hu and Liu (2004), Emotion Saravia et al. (2018),
TREC Voorhees and Tice (2000) and AGNews Zhang et al. (2015). We also use two
fine-grained classification datasets, Empathetic Dialogues Rashkin et al.
(2018) and Yahoo Answer Topics Sileo (2023) to evaluate DAIL’s performance
with large label spaces. Table 2 shows the detailed statistics of the
aforementioned datasets. For baseline models, we choose
ChatGPT111https://platform.openai.com/docs/models/gpt-3-5 and PaLM2-540B Anil
et al. (2023). We use the official APIs of each model to conduct evaluations.
For ChatGPT, we use a stable version GPT-3.5-turbo-0301 for our experiments.
Due to the lack of a stable version of the PaLM2-540B model, we repeat the
experiments three times and report the average results. To simulate a low-
resource scenario, we establish the number of demonstrations for each label
category in each dataset to be 1. We report the accuracy score for each
dataset.

We compare our method with standard ICL. For our method, we compare several
numbers of paraphrases per sample ranging from 1 to 4 and DAIL-$i$ indicates
$i$ paraphrases per sample. For DAIL-1, we simply replace the original test
sample with a paraphrase generated by the LLM and perform inference with it.
For DAIL-2 and DAIL-4, we follow the method we illustrate in Section DAIL:
Data Augmentation for In-Context Learning via Self-Paraphrase to generate and
ensemble paraphrases. More details about our paraphrase and prompt format can
be found in Appendix A

Task | Dataset | Testset Size
---|---|---
Sentiment Classification | SST2 | 872
SST5 | 2210
CR | 376
Topic Classification | AGNews | 7600
TREC | 500
Yahoo | 2000
Emotion Classification | Emotion | 2000
Emp Dialogue | 10900
Table 2: Detailed statistics of the datasets we use in the experiment.

###  3.2 Main Results

Table 1 shows the comparison between DAIL and standard ICL method. To further
validate our assumption underlying the self-paraphrase mechanism, we also
conduct an ablation study on DAIL-4 by using paraphrases interchangeably,
denoted by DAIL-4 w/o SP. We use the paraphrase generated by PaLM for ChatGPT
and vice versa.

Here DAIL-1 doesn’t exhibit a significant improvement in all datasets,
suggesting the necessity of adopting an ensemble-based approach in DAIL. When
the number of paraphrase samples used in DAIL increases, the performance
consistently improves. DAIL-4 attains the best overall results across various
datasets for both models. For datasets such as Yahoo that have $10$ target
classes, the performance improvement is up to $6$ points in the case of
ChatGPT. It is also notable that the performance in both models drops when we
replace the self-paraphrase mechanism in DAIL-4. This confirms the assumption
we put forth that large language models are more accustomed to the content
they generate.

| SST5 | TREC | Emotion
---|---|---|---
Standard ICL | 54.4 | 77.1 | 52.2
| Self-Consistency
---
Wang et al. (2022)
54.8 | 78.2 | 55.6
| Prompt-Ensemble
---
Gao et al. (2020)
54.2 | 81.9 | 55.5
DAIL | 55.4 | 84.6 | 55.6
Table 3: Comparison of DAIL and other ensemble-based methods. The best results
are in bold. We observe DAIL out-performing other ensemble-based methods for
classification.

###  3.3 Comparison to Other Ensemble-based Methods

| Text | Prediction | Ground Truth
---|---|---|---
Org | A well acted and well intentioned snoozer. | Neutral | Negative
Para | Although well-performed and well-meaning, it can be quite dull. | Negative
It’s a yawn-inducing film, but the acting and themes are commendable. | Neutral
The movie is a bore, despite the admirable acting and good intentions. | Negative
While well-acted and with good intentions, the movie is a bit of a snooze fest. | Negative
Table 4: An example in SST5 dataset. The text in green represent positive
content and the text in red represent negative content

We conduct a comparison between our method and two other ensemble-based
approaches, namely Self-Consistency Wang et al. (2022) and Prompt-Ensemble Gao
et al. (2020).

Self-Consistency is a method that incorporates sampling different reasoning
paths during decoding. The various sampled results are subsequently used for
voting to determine the final result. Prompt-Ensemble Gao et al. (2020), on
the other hand, employs different prompts for multiple inferences and then
uses the outputs of the model to determine the final prediction through
voting. In our experiments, we manually write prompts for each dataset. More
details about it can be found in Appendix C.

Table 3 displays the comparison results. While the three methods demonstrate
similar performance on the Emotion dataset, DAIL exhibits a significant
advantage over the other two methods in terms of SST5 and TREC. Additionally,
we observe the performance of Self-Consistency in the classification datasets
we use is not as strong as reported in the original paper for several
reasoning datasets. One possible explanation for this discrepancy is that the
classification datasets we employ lack reasoning samples that are provided to
the model. In contrast, DAIL does not rely on any external sources, making it
a more comprehensive approach.

(a) Binary classification

(b) Multi-class classification

Figure 2: Model Accuracy vs Voting consistency score using ChatGPT. We observe
a positive correlation between voting consistency score and accuracy, thus
making it a reliable confidence metric.

###  3.4 Voting Consistency as Confidence Score

In this section, we assess the reliability of employing voting consistency as
the confidence score for the model. We plot the accuracy of ChatGPT vs several
voting consistency thresholds illustrating their relationship in Figure 2. For
the binary classification datasets (SST2, CR), we utilize confidence values of
0.6, 0.8, and 1.0. For the multiple classification datasets (SST5, Emotion,
TREC, AGNews), we use confidence values of 0.4, 0.6, 0.8, and 1.0. We put the
results of PaLM in Appendix B.

Although the confidence score doesn’t completely correspond to the accuracy,
we observe a significant positive correlation with each other. By considering
the confidence level, we can roughly assess the reliability of the model’s
output. We believe this could serve as a confidence score for addressing the
problems in LLMs such as hallucination Ji et al. (2023) and confidence
calibration Wang et al. (2020); Mekala et al. (2022, 2023).

###  3.5 Case Study

To explain our method, we provide a case study from ChatGPT’s results on SST5.
Consider a negative sentiment sentence: “A well acted and well intentioned
snoozer.”. Here both positive expression (“well acted” and “well intentioned”)
and negative expressions (“snoozer”) are present, making it a challenging one
for predicting sentiment.

For standard ICL, the model fails to capture this subtle sentiment and gives
the wrong label ’Neutral’. The paraphrases generated by DAIL are mentioned in
Table 4. For DAIL, one of the paraphrases generated by ChatGPT is “Although
well-performed and well-meaning, it can be quite dull”. It emphasizes the
switch in relationship with the word “although” and gets the correct label
’Negative’. By paraphrasing several times and using majority voting, DAIL
predicts it correctly.

##  4 Conclusion

In this work, we introduce a data augmentation technique to ICL and propose
DAIL. It involves augmenting the test sample by generating multiple
paraphrases and ensembling the individual results to obtain the final
prediction. In our experiments, we compare DAIL with standard ICL and other
ensemble-based methods. The evaluation results demonstrate the effectiveness
of DAIL in the low-resource ICL scenario. Furthermore, we investigate the
feasibility of utilizing voting consistency as a confidence estimation method
and discover a positive correlation between voting consistency and model
accuracy.

##  5 Limitations

Similar to other ensemble-based methods, DAIL requires multiple inferences to
obtain individual labels for each paraphrased sample. This introduces
additional costs compared to standard ICL. Additionally, our self-paraphrase
mechanism relies on the same language model to generate paraphrases for the
test sample. Consequently, DAIL may not be suitable for smaller language
models that lack the capability to produce high-quality paraphrases.

## References

  * Anil et al. (2023) Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023.  Palm 2 technical report.  _arXiv preprint arXiv:2305.10403_.
  * Bonifacio et al. (2022) Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022.  Inpars: Data augmentation for information retrieval using large language models.  _arXiv preprint arXiv:2202.05144_.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Gao et al. (2023a) Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, and Xiaojun Wan. 2023a.  Human-like summarization evaluation with chatgpt.  _ArXiv_ , abs/2304.02554.
  * Gao et al. (2020) Tianyu Gao, Adam Fisch, and Danqi Chen. 2020.  Making pre-trained language models better few-shot learners.  _arXiv preprint arXiv:2012.15723_.
  * Gao et al. (2023b) Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023b.  Chat-rec: Towards interactive and explainable llms-augmented recommender system.  _arXiv preprint arXiv:2303.14524_.
  * Hu and Liu (2004) Minqing Hu and Bing Liu. 2004.  Mining and summarizing customer reviews.  In _Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 168–177.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38.
  * Li and Qiu (2023) Xiaonan Li and Xipeng Qiu. 2023.  Finding supporting examples for in-context learning.  _arXiv preprint arXiv:2302.13539_.
  * Liu et al. (2021) Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021.  What makes good in-context examples for gpt-$3$?  _arXiv preprint arXiv:2101.06804_.
  * Lu et al. (2021) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021\.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  _arXiv preprint arXiv:2104.08786_.
  * Mekala et al. (2022) Dheeraj Mekala, Chengyu Dong, and Jingbo Shang. 2022.  LOPS: Learning order inspired pseudo-label selection for weakly supervised text classification.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 4894–4908, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Mekala et al. (2023) Dheeraj Mekala, Adithya Samavedhi, Chengyu Dong, and Jingbo Shang. 2023.  Selfood: Self-supervised out-of-distribution detection via learning to rank.  _arXiv preprint arXiv:2305.14696_.
  * Mekala and Shang (2020) Dheeraj Mekala and Jingbo Shang. 2020.  Contextualized weak supervision for text classification.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 323–333.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Rashkin et al. (2018) Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018.  Towards empathetic open-domain conversation models: A new benchmark and dataset.  _arXiv preprint arXiv:1811.00207_.
  * Rubin et al. (2021) Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2021.  Learning to retrieve prompts for in-context learning.  _arXiv preprint arXiv:2112.08633_.
  * Saravia et al. (2018) Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018\.  Carer: Contextualized affect representations for emotion recognition.  In _Proceedings of the 2018 conference on empirical methods in natural language processing_ , pages 3687–3697.
  * Sileo (2023) Damien Sileo. 2023.  tasksource: A dataset harmonization framework for streamlined nlp multi-task learning and evaluation.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 conference on empirical methods in natural language processing_ , pages 1631–1642.
  * Su et al. (2022) Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, et al. 2022.  Selective annotation makes language models better few-shot learners.  _arXiv preprint arXiv:2209.01975_.
  * Voorhees and Tice (2000) Ellen M Voorhees and Dawn M Tice. 2000.  Building a question answering test collection.  In _Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval_ , pages 200–207.
  * Wang et al. (2020) Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. 2020.  On the inference calibration of neural machine translation.  _arXiv preprint arXiv:2005.00963_.
  * Wang et al. (2022) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022.  Self-consistency improves chain of thought reasoning in language models.  _arXiv preprint arXiv:2203.11171_.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_.
  * Wu et al. (2022) Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2022.  Self-adaptive in-context learning.  _arXiv preprint arXiv:2212.10375_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _Advances in neural information processing systems_ , 28.

##  Appendix A Prompt for DAIL

We present the prompt we use to generate paraphrases and do inference with
LLMs in Table 5 and Table 6. We utilize <Label Space> to represent all labels
in certain datasets, <Demonstrations> to represent the demonstrations used,
and <Test Sample> to represent the sample being inferred.

Task | Dataset | Paraphrase Prompt
---|---|---
Sentiment Classification | SST2 | Please paraphrase this sentence <Para-Num>times without changing the original sentiment:
SST5
CR
Emotion Classification | Emotion | Please paraphrase this sentence <Para-Num>times without changing the original emotion:
Empathetic Dialogue
Question Classification | TREC | Please paraphrase this question <Para-Num>times without changing the original semantic:
Yahoo
News Classification | AG News | Please paraphrase this news <Para-Num>times without changing the original semantic:
Table 5: Prompt we use to generate paraphrases of the test samples for each dataset. Task | Dataset | Inference Prompt
---|---|---
Sentiment Classification | SST2 |  Label the sentiment class of the sentence, please choose from <Label Space><Demonstrations><Test Sample>
SST5
CR
Emotion Classification | Emotion |  Label the emotion class of the sentence, please choose from <Label Space><Demonstrations><Test Sample>
Empathetic Dialogue
Question Classification | TREC |  Label the category of the question, please choose from <Label Space><Demonstrations><Test Sample>
Yahoo
News Classification | AG News |  | Label the category of the news,
---
please choose from <Label Space><Demonstrations><Test Sample>
Table 6: Prompt we use to do inference for each dataset.

##  Appendix B Voting Consistency as Confidence Score: PaLM

Figure 3 shows the analysis of adopting voting consistency as confidence score
on PaLM.

(a) Binary classification

(b) Multi-class classification

Figure 3: Model Accuracy vs Voting consistency score using PaLM. We observe a
similar positive correlation between voting consistency score and accuracy,
thus making it a reliable confidence metric.

##  Appendix C Prompt for Prompt-Ensemble

Table 7 shows the manually written prompt we prepare for each dataset in the
Prompt-Ensemble method.

Dataset | Prompt
---|---
SST5 | Label the sentiment class of the sentence.
What is the sentiment expressed in this message?
What sentiment does this message express?
How will you feel about the message in terms of its sentiment?
What sentiment does the writer express for the message?
TREC | Label the categories of the given question.
What is the categories of the given question?
What type of information does this question express?
How will you feel about the question about its category?
What type of information does the writer express for the question?
Emotion | Label the emotion class of the sentence.
What is the emotion expressed in this message?
What emotion does this message express?
How will you feel about the message in terms of its emotion?
What emotion does the writer express for the message?
Table 7: Prompt we manually write for Prompt-Ensemble approach in each
dataset.


* [6] G. Compere and A. Fiorucci, _Advanced Lectures on General Relativity_ , _Lect. Notes Phys._ 952 (2019) 150, [1801.07064].
  * [7] S. De Haro, _Noether’s Theorems and Energy in General Relativity_ , [2103.17160].
  * [8] H. Bondi, _Gravitational Waves in General Relativity_ , _Nature_ 186 (1960) 535–535.
  * [9] H. Bondi, M. G. J. van der Burg and A. W. K. Metzner, _Gravitational waves in general relativity. 7. Waves from axisymmetric isolated systems_ , _Proc. Roy. Soc. Lond._ A269 (1962) 21–52.
  * [10] R. Sachs, _Asymptotic symmetries in gravitational theory_ , _Phys. Rev._ 128 (1962) 2851–2864.
  * [11] R. K. Sachs, _Gravitational waves in general relativity. 8. Waves in asymptotically flat space-times_ , _Proc. Roy. Soc. Lond._ A270 (1962) 103–126.
  * [12] G. Barnich and C. Troessaert, _Aspects of the BMS/CFT correspondence_ , _JHEP_ 05 (2010) 062, [1001.1541].
  * [13] E. E. Flanagan and D. A. Nichols, _Conserved charges of the extended Bondi-Metzner-Sachs algebra_ , _Phys. Rev. D_ 95 (2017) 044002, [1510.03386].
  * [14] T. Mädler and J. Winicour, _Bondi-Sachs Formalism_ , _Scholarpedia_ 11 (2016) 33528, [1609.01731].
  * [15] G. Compère, A. Fiorucci and R. Ruzziconi, _The $\Lambda$-BMS4 charge algebra_, _JHEP_ 10 (2020) 205, [2004.10769].
  * [16] R. Ruzziconi, _On the Various Extensions of the BMS Group_.  PhD thesis, 9, 2020.  2009.01926.
  * [17] L. Freidel, R. Oliveri, D. Pranzetti and S. Speziale, _The Weyl BMS group and Einstein’s equations_ , [2104.05793].
  * [18] A. Fiorucci, _Leaky covariant phase spaces: Theory and application to $\Lambda$-BMS symmetry_.  PhD thesis, Brussels U., Intl. Solvay Inst., Brussels, 2021.  2112.07666.
  * [19] M. Cadoni and S. Mignemi, _Asymptotic symmetries of AdS(2) and conformal group in d = 1_ , _Nucl. Phys. B_ 557 (1999) 165–180, [hep-th/9902040].
  * [20] J. Navarro-Salas and P. Navarro, _AdS(2) / CFT(1) correspondence and near extremal black hole entropy_ , _Nucl. Phys. B_ 579 (2000) 250–266, [hep-th/9910076].
  * [21] D. Grumiller, W. Kummer and D. Vassilevich, _Dilaton gravity in two-dimensions_ , _Phys. Rept._ 369 (2002) 327–430, [hep-th/0204253].
  * [22] J. Maldacena, D. Stanford and Z. Yang, _Conformal symmetry and its breaking in two dimensional Nearly Anti-de-Sitter space_ , _PTEP_ 2016 (2016) 12C104, [1606.01857].
  * [23] A. Blommaert, T. G. Mertens and H. Verschelde, _Fine Structure of Jackiw-Teitelboim Quantum Gravity_ , [1812.00918].
  * [24] H. Afshar, H. A. Gonzlez, D. Grumiller and D. Vassilevich, _Flat space holography and complex SYK_ , [1911.05739].
  * [25] V. Godet and C. Marteau, _New boundary conditions for AdS 2_, [2005.08999].
  * [26] R. Ruzziconi and C. Zwikel, _Conservation and integrability in lower-dimensional gravity_ , [2012.03961].
  * [27] H. Adami, M. Sheikh-Jabbari, V. Taghiloo, H. Yavartanoo and C. Zwikel, _Symmetries at null boundaries: two and three dimensional gravity cases_ , _JHEP_ 10 (2020) 107, [2007.12759].
  * [28] T. G. Mertens and G. J. Turiaci, _Liouville quantum gravity – holography, JT and matrices_ , _JHEP_ 01 (2021) 073, [2006.07072].
  * [29] J. D. Brown and M. Henneaux, _Central Charges in the Canonical Realization of Asymptotic Symmetries: An Example from Three-Dimensional Gravity_ , _Commun. Math. Phys._ 104 (1986) 207–226.
  * [30] A. Ashtekar, J. Bicak and B. G. Schmidt, _Asymptotic structure of symmetry reduced general relativity_ , _Phys. Rev. D_ 55 (1997) 669–686, [gr-qc/9608042].
  * [31] G. Barnich and G. Compere, _Classical central extension for asymptotic symmetries at null infinity in three spacetime dimensions_ , _Class. Quant. Grav._ 24 (2007) F15–F23, [gr-qc/0610130].
  * [32] G. Barnich, A. Gomberoff and H. A. González, _The Flat limit of three dimensional asymptotically anti-de Sitter spacetimes_ , _Phys. Rev. D_ 86 (2012) 024020, [1204.3288].
  * [33] D. Grumiller and M. Riegler, _Most general AdS 3 boundary conditions_, _JHEP_ 10 (2016) 023, [1608.01308].
  * [34] D. Grumiller, W. Merbis and M. Riegler, _Most general flat space boundary conditions in three-dimensional Einstein gravity_ , _Class. Quant. Grav._ 34 (2017) 184001, [1704.07419].
  * [35] H. Afshar, D. Grumiller and M. Sheikh-Jabbari, _Near horizon soft hair as microstates of three dimensional black holes_ , _Phys. Rev. D_ 96 (2017) 084032, [1607.00009].
  * [36] F. Alessio, G. Barnich, L. Ciambelli, P. Mao and R. Ruzziconi, _Weyl charges in asymptotically locally AdS 3 spacetimes_, _Phys. Rev. D_ 103 (2021) 046003, [2010.15452].
  * [37] M. Geiller and C. Goeller, _Dual diffeomorphisms and finite distance asymptotic symmetries in 3d gravity_ , [2012.05263].
  * [38] M. Geiller, C. Goeller and C. Zwikel, _3d gravity in Bondi-Weyl gauge: charges, corners, and integrability_ , _JHEP_ 09 (2021) 029, [2107.01073].
  * [39] M. P. Ryan and L. C. Shepley, _Homogeneous Relativistic Cosmologies_.  Princeton Series in Physics. Princeton University Press, Princeton, 1975\.
  * [40] R. Kantowski and R. K. Sachs, _Some spatially homogeneous anisotropic relativistic cosmological models_ , _J. Math. Phys._ 7 (1966) 443.
  * [41] F. Sartini, _Hidden Symmetries in Gravity: Black holes and other minisuperspaces_.  PhD thesis, ENS de Lyon, 2022.
  * [42] J. Ben Achour and E. R. Livine, _Thiemann complexifier in classical and quantum FLRW cosmology_ , _Phys. Rev. D_ 96 (2017) 066025, [1705.03772].
  * [43] J. Ben Achour and E. R. Livine, _Protected $SL(2,\mathbb{R})$ Symmetry in Quantum Cosmology_, _JCAP_ 09 (2019) 012, [1904.06149].
  * [44] J. Ben Achour and E. R. Livine, _Polymer Quantum Cosmology: Lifting quantization ambiguities using a $SL(2,\mathbb{R})$ conformal symmetry_, _Phys. Rev. D_ 99 (2019) 126013, [1806.09290].
  * [45] J. Ben Achour and E. R. Livine, _Cosmology as a CFT 1_, _JHEP_ 12 (2019) 031, [1909.13390].
  * [46] J. Ben Achour and E. R. Livine, _Cosmological spinor_ , _Phys. Rev. D_ 101 (2020) 103523, [2004.06387].
  * [47] J. Ben Achour and E. R. Livine, _Conformal structure of FLRW cosmology: spinorial representation and the $\mathfrak{so}$ (2, 3) algebra of observables_, _JHEP_ 03 (2020) 067, [2001.11807].
  * [48] B. Pioline and A. Waldron, _Quantum cosmology and conformal invariance_ , _Phys. Rev. Lett._ 90 (2003) 031302, [hep-th/0209044].
  * [49] J. B. Achour, _Proper time reparametrization in cosmology: Möbius symmetry and Kodama charges_ , _JCAP_ 12 (2021) 005, [2103.10700].
  * [50] M. Geiller, E. R. Livine and F. Sartini, _Symmetries of the Black Hole Interior and Singularity Regularization_ , _SciPost Phys._ 10 (2021) 022, [2010.07059].
  * [51] J. B. Achour and E. R. Livine, _Symmetries and conformal bridge in Schwarschild-(A)dS black hole mechanics_ , _JHEP_ 12 (2021) 152, [2110.01455].
  * [52] F. Sartini, _Group quantization of the black hole minisuperspace_ , _Phys. Rev. D_ 105 (2022) 126003, [2110.13756].
  * [53] J. Ben Achour and E. R. Livine, _The Cosmological Constant from Conformal Transformations: Möbius Invariance and Schwarzian Action_ , [2004.05841].
  * [54] M. Geiller, E. R. Livine and F. Sartini, _BMS 3 mechanics and the black hole interior_, _Class. Quant. Grav._ 39 (2022) 025001, [2107.03878].
  * [55] T. Pailas, N. Dimakis, A. Paliathanasis, P. A. Terzis and T. Christodoulakis, _The infinite dimensional symmetry groups of the Friedmann equations_ , _Phys. Rev. D_ 102 (2020) 063524, [2005.11726].
  * [56] T. Christodoulakis, N. Dimakis, P. A. Terzis, B. Vakili, E. Melas and T. Grammenos, _Minisuperspace canonical quantization of the Reissner-Nordström black hole via conditional symmetries_ , _Phys. Rev. D_ 89 (2014) 044031, [1309.6106].
  * [57] T. Christodoulakis, N. Dimakis and P. A. Terzis, _Lie point and variational symmetries in minisuperspace Einstein gravity_ , _J. Phys. A_ 47 (2014) 095202, [1304.4359].
  * [58] T. Christodoulakis, N. Dimakis, P. A. Terzis, G. Doulis, T. Grammenos, E. Melas et al., _Conditional Symmetries and the Canonical Quantization of Constrained Minisuperspace Actions: the Schwarzschild case_ , _J. Geom. Phys._ 71 (2013) 127–138, [1208.0462].
  * [59] N. Dimakis, P. A. Terzis and T. Christodoulakis, _Contact symmetries of constrained systems and the associated integrals of motion_ , _J. Phys. Conf. Ser._ 633 (2015) 012040.
  * [60] P. A. Terzis, N. Dimakis, T. Christodoulakis, A. Paliathanasis and M. Tsamparlis, _Variational contact symmetries of constrained Lagrangians_ , _J. Geom. Phys._ 101 (2016) 52–64, [1503.00932].
  * [61] N. Dimakis, P. A. Terzis, A. Zampeli and T. Christodoulakis, _Decoupling of the reparametrization degree of freedom and a generalized probability in quantum cosmology_ , _Phys. Rev. D_ 94 (2016) 064013, [1606.05116].
  * [62] T. Christodoulakis, A. Karagiorgos and A. Zampeli, _Symmetries in Classical and Quantum Treatment of Einstein’s Cosmological Equations and Mini-Superspace Actions_ , _Symmetry_ 10 (2018) 70.
  * [63] M. Cariglia and F. K. Alves, _The Eisenhart lift: a didactical introduction of modern geometrical concepts from Hamiltonian dynamics_ , _Eur. J. Phys._ 36 (2015) 025018, [1503.07802].
  * [64] M. Cariglia, C. Duval, G. W. Gibbons and P. A. Horvathy, _Eisenhart lifts and symmetries of time-dependent systems_ , _Annals Phys._ 373 (2016) 631–654, [1605.01932].
  * [65] N. Kan, T. Aoyama, T. Hasegawa and K. Shiraishi, _Eisenhart-Duval lift for minisuperspace quantum cosmology_ , _Phys. Rev. D_ 104 (2021) 086001, [2105.09514].
  * [66] J. Ben Achour, E. R. Livine, D. Oriti and G. Piani, _Schrödinger symmetry in cosmology and black hole mechanics_ , [2207.07312].
  * [67] J. Ben Achour, E. R. Livine, S. Mukohyama and J.-P. Uzan, _Hidden symmetry of the static response of black holes: applications to Love numbers_ , _JHEP_ 07 (2022) 112, [2202.12828].
  * [68] V. Aldaya and J. L. Jaramillo, _Space-time structures from critical values in 2-D quantum gravity_ , _Class. Quant. Grav._ 17 (2000) 1649–1666, [gr-qc/9907071].
  * [69] G. Caviglia, C. Zordan and F. Salmistraro, _Equation of geodesic deviation and killing tensors_ , _International Journal of Theoretical Physics_ 21 (1982) 391–396.
  * [70] A. Ashtekar and J. Samuel, _Bianchi cosmologies: The Role of spatial topology_ , _Class. Quant. Grav._ 8 (2011) 2191.