# How Far Can We Extract Diverse Perspectives from Large Language Models?
Criteria-Based Diversity Prompting!

Shirley Anugrah Hayati Minhwa Lee Dheeraj Rajagopal Dongyeop Kang
University of Minnesota Google Research
{hayat023, lee03533<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Collecting diverse human data on subjective NLP topics is costly and
challenging. As Large Language Models (LLMs) have developed human-like
capabilities, there is a recent trend in collaborative efforts between humans
and LLMs for generating diverse data, offering potential scalable and
efficient solutions. However, the extent of LLMs’ capability to generate
diverse perspectives on subjective topics remains an unexplored question. In
this study, we investigate LLMs’ capacity for generating diverse perspectives
and rationales on subjective topics, such as social norms and argumentative
texts. We formulate this problem as diversity extraction in LLMs and propose a
criteria-based prompting technique to ground diverse opinions and measure
perspective diversity from the generated criteria words. Our results show that
measuring semantic diversity through sentence embeddings and distance metrics
is not enough to measure perspective diversity. To see how far we can extract
diverse perspectives from LLMs, or called diversity coverage, we employ a
step-by-step recall prompting for generating more outputs from the model in an
iterative manner. As we apply our prompting method to other tasks (hate speech
labeling and story continuation), indeed we find that LLMs are able to
generate diverse opinions according to the degree of task subjectivity.111Our
code and data are available at https://github.com/minnesotanlp/diversity-
extraction-from-llms

##  1 Introduction

Using NLP models for tasks that require social reasoning and involve human
subjectivity, such as argumentation over an issue Hidey et al. (2017) or
toxicity detection Sap et al. (2019), often calls for diverse perspectives.
Rather than providing a single response for subjective tasks, an ideal NLP
model should be able to take into account as many different perspectives as
possible so that it will not be biased toward only one dominant viewpoint.
Previous works have highlighted this need to incorporate and model multiple
perspectives Plank (2022); Abercrombie et al. (2022). To address this
challenge, some past works have collected responses from multiple human
annotators preferably with diverse backgrounds and values Rottger et al.
(2022); Santy et al. (2023). However, collecting data from people with
different backgrounds is challenging and expensive in time and resources.

Figure 1: LLMs are trained on multiple texts written by different people who
may have distinct perspectives. Our study examines whether LLMs can do
“reverse modeling” of humans’ perspectives from the training data and how much
diversity coverage LLMs are able to generate. A check mark denotes a person’s
stance is "Agree" and a cross mark represents "Disagree."

Recent advancements of large language models (LLMs), such as ChatGPT and GPT-4
OpenAI (2023), have gained much interest from researchers to exploit their
capability of creative generation Bubeck et al. (2023) for data augmentation,
with less cost and higher diversity Cegin et al. (2023); Chung et al. (2023);
Bubeck et al. (2023). As a result, human-LLM collaboration becomes a common
approach to generate human-like diverse texts for problems that need
diversity, creativity, and social reasoning Kim et al. (2022); Chen et al.
(2023); Wang et al. (2023); Taori et al. (2023).

In this study, we are interested in a specific scope of diversity from LLM -
“perspective diversity”. Besides generating lexical and syntactic diversity,
we argue that LLMs should be able to model humans’ diverse opinions well to
promote diversity and inclusivity of humans with different values.

Figure 2: People can have different opinions given a subjective task. Given a
statement, humans can agree or disagree with the statement with their own
criteria in deciding their stances.

Figure 1 illustrates the significance of understanding the extent of
perspective diversity achievable by LLMs. During training, LLMs have access to
various writings by humans who may have distinct values and perspectives.
However, can LLMs reflect each diverse opinion during its generation phase? Or
are they only expressing majority opinions? Then, compared to human-written
opinions, how much do the human opinions and LLM-generated opinions overlap?
If the purple circle in Figure 1 hypothetically illustrates the maximum
diversity, we are curious to probe LLMs’ capacity to cover this diversity.

In real-world settings, humans may have different stances toward a subjective
statement as shown in Figure 2. One who values teamwork and goal achievements
may agree with the statement shown in the figure. On the contrary, others who
value creativity and innovation may argue over the statement. Then, how many
and what diverse perspectives do LLMs model?

We introduce a new problem called diversity extraction. Formally, our problem
is defined as follows. Given a subjective statement, we ask LLMs to generate
as many different stances as possible, explain their reason for generating
their stances, and produce a set of criteria words that are important for
their reasoning in choosing their stances. We apply our diversity prompting to
four different subjective tasks: (1) social norms; (2) argumentation; (3) hate
speech labeling; and (4) story continuation. We discover that semantic
diversity does not always mean perspective diversity.

#### Contributions

First, we propose the idea of perspective diversity for generative LLMs,
unlike lexical diversity, syntactical diversity, and semantic diversity that
have been main interests in previous works. We conduct various experiments to
measure LLMs’ ability to generate maximum perspective diversity. Second, we
thus introduce a new prompting technique called criteria-based diversity
prompting, as a way for extracting and grounding diverse perspectives from
LLMs. Finally, as it is unclear how much diversity LLMs can cover, we suggest
a step-by-step approach for measuring the coverage of LLMs’ diversity
generation (i.e., measuring the recall for diversity prompting). We then
compare this coverage between LLM’s generated opinions and human-written
opinions.

Model Input (Step-by-step recall prompting, N=2)
---
Statement: It’s okay to have privacy
Tell me opinions about the statement as many as possible from 2 different
people with, “Agree” or “Disagree,” one-word or
one-phrase criteria that is important for their opinions, and explain how they
have different opinions
Output:
{1:{‘‘Stance’’ :‘‘Agree’’,
‘‘Criteria’’: [‘‘personal boundaries’’, ‘‘autonomy’’],
‘‘Reason’’: ‘‘Having privacy allows individuals to establish personal
boundaries and maintain.’’
their autonomy."},
2: {‘‘Stance’’:
Model Output
‘‘Disagree",
‘‘Criteria’’: [‘‘transparency’’, ‘‘trust’’],
‘‘Reason’’: ‘‘Lack of privacy can promote transparency and build trust in
relationships.’’
Table 1: Example criteria-based step-by-step recall prompting for
investigating LLMs’ diversity coverage where N = the number of opinions we ask
LLMs to generate. The number of opinions in the model input is incremented
step-by-step.

##  2 Criteria-based Diversity Prompting

#### Step 1: Think of Your Criteria First before Making Opinions

To examine an LLM’s capability to generate diverse perspectives and maximize
their diversity, we first formulate the following prompting method. Given a
(subjective) statement, we prompt the model to generate its Stance and explain
its Reasons with a list of Criteria that affect its perspective (see Figure
2). This criteria-based prompting method allows the model to generate
important criteria that are used for explaining the model’s decision and
generating the reason. We can consider the criteria list as the model’s values
as well. The usage of criterion generation follows human underlying reasoning
process where people’s values often serve as guidelines in their lives and
affect their opinions and behavior Rokeach (1973); Kesberg and Keller (2018).

While we can allow the model to generate only a group of free-form texts as
opinions, we formulate the task to choose a binary type of stance so that it
would be easier for measuring perspective diversity.222We later show how to
extend this for non-labeling task Criterion generation guides the model to
only generate a list of words or short phrases. Finally, the model’s output
reason is a free-form text that explains its stance (See Table 1 and Table 9
in the Appendix). Except for the recall prompting experiment, we use in-
context prompting to explore the model’s ability to generate diverse opinions
motivated by recent work showing how few-shot learning has been able to guide
the model for even difficult tasks Perez et al. (2021); Min et al. (2022b, a);
Lu et al. (2022).

The output should follow a structured format so that it would be easier to
parse for the diversity evaluation. Table 9 shows an example of a model
producing opinions in the structured format. We provide ten different
opinions, with five agreeing with the statement and five disagreeing, for each
of the few-shot examples.

#### Step 2: Incrementally Maximize Diversity via Step-by-Step Recall
Prompting

After we find the best setups, we extend our diversity prompting into
diversity step-by-step recall prompting to see the LLMs’ diversity coverage.
In this experiment, we do not provide an example in the prompt. Instead, we
only extract one opinion for the given statement and ask the model to continue
generating more opinions until N is reached, where N equals the number of
generated opinions we specify. Figure 3 illustrates the process of step-by-
step recall prompting. In this experiment, we specify
$N\in\\{2,5,8,11,14,17,20\\}$ with a step size of 3.333For the Social-Chem-101
dataset, we additionally have $N=3,4$ to see how the model behaves for such
smaller $N$. The first opinion is generated by the LLM as well and is used to
guide the structured format for the output since we do not do few-shot
prompting for this experiment set-up.

Figure 3: Step-by-step recall prompting. The statement and first generated
opinion become the demonstration for prompting the LLM to generate $N$
opinions. The blue-colored parts (Steps 1 and 2) are done incrementally with
step size = 3.

##  3 Experimental Setups

###  3.1 Models

For the large language models, we experimented with three GPT variants: GPT-4,
ChatGPT, and GPT-3 (text-davinci-002) Brown et al. (2020). As our baselines,
we also provide a free-form prompting where the model is not asked to generate
criteria words when it generates its stance and reasoning given a statement.

For our experiment, we compare one-shot prompting against five-shot. We tried
zero-shot prompting during our initial experiments. However, without examples,
the generated output was not structured, hence unable to parse for proper
evaluation. Thus, we decided to experiment between one and five exemplars for
the model to follow In our few-shot examples, each statement has 10
corresponding opinions. We also experimented with a different number of
opinions in our step-by-step recall prompting experiment.

###  3.2 Datasets

Our main dataset for examining variations of the few-shot learning experiments
is Social-Chem-101 Forbes et al. (2020) and Change My View Hidey et al.
(2017). The task for the two datasets is quite similar as the model needs to
generate a stance and reasoning for their stance. Then for the step-by-step
recall prompting, we add two more datasets, Hate Speech Vidgen et al. (2021)
and Moral Stories Emelin et al. (2021), to show how our prompting method can
be used for other tasks such as text labeling and language generation.

#### Social-Chem-101

contains 292k free-text rules-of-thumbs (RoTs) of social norms and moral
judgments for a given situation written by crowd-workers Forbes et al. (2020).
Social norms depend on many factors such as the group’s beliefs and cultures
Ajzen (1991); Shweder (1990). Therefore, this dataset is suitable for our task
of maximizing LLM’s diversity capability. For our study, we randomly sampled
500 RoTs and 200 RoTs for the step-by-step recall prompting.

#### Change My View (CMV)

consists of 78 instances from online forum threads of the subreddit Change My
View collected by Hidey et al. (2017). Each instance contains a title, which
is usually a claim, and a thread of discussion where other posters will
attempt to change the original poster’s perspective. We only take the title of
each discussion since it is usually the claim of the argument. In Hidey et al.
(2017)’s dataset, positive or negative labels indicate whether the original
poster’s view is successfully changed or maintained. For our study, we combine
the positive and negative data and only take the claim as the statement for
the model input prompt, thus resulting in a total of 67 unique claims. We use
this dataset to examine if LLMs can produce diverse opinions on a highly
subjective task because an argumentation task could be highly controversial
Van Eemeren et al. (2015). Therefore, we hypothesize that when the more
controversial and subjective a statement is, the model should be able to
generate more diverse opinions.

#### Hate Speech

dataset contains texts labeled as hate speech and not, from Vidgen et al.
(2021). We randomly select 200 instances only from Round 3 and 4 because they
are implicit hate which is harder to detect. Hate speech detection is a
subjective task because annotators’ background may affect how they rate the
hate speech label of a text Sap et al. (2019); Ghosh et al. (2021)

#### Moral Stories

is a crowd-sourced narrative story dataset Emelin et al. (2021). Each story
has a situation, intention, moral action, immoral action, and their
consequences. However, for this study, we only consider the situation part as
a story and ask the LLM to continue the story. We also random sample 200
instances from this dataset.

###  3.3 Evaluation Metric

#### Semantic diversity

We first examine the diversity of the model’s reasons for both criteria-based
prompting and free-form prompting. To do so, for each statement, we first
model the generated “reasons” from LLMs as sentence embeddings using
SentenceBERT Reimers and Gurevych (2019). We then measure the cosine distance
among every pair of reasons and compute the average cosine distance across all
the pairs. This becomes a statement’s semantic diversity score. Finally, we
average all statements’ semantic diversity scores to be the final semantic
diversity result, as shown in Table 2.

All ($\uparrow$) Agree ($\uparrow$) Disagree ($\uparrow$) GPT-4 ChatGPT GPT-3
GPT-4 ChatGPT GPT-3 GPT-4 ChatGPT GPT-3 Social-Chem-101 1-Criteria 0.4055
0.3237 0.2756 0.3751 0.3102 0.2551* 0.4360* 0.3372* 0.2960* 1-Freeform 0.4021
0.2508 0.2035 0.378 0.2982 0.2982 0.4261 0.2991 0.2074 5-Criteria 0.3612
0.2508 0.2186 0.3247 0.2375* 0.2041* 0.3978 0.2640* 0.2331* 5-Freeform 0.3739
0.2269 0.1734 0.3353 0.2235 0.172 0.4125 0.2303 0.1749 ChangeMyView 1-Criteria
0.3992 0.3071 0.2148 0.4003 0.3228* 0.2031* 0.3982 0.2913* 0.2266* 1-Freeform
0.3872 0.2530 0.1884 0.3999 0.2694 0.1475 0.3745 0.2366 0.1658 5-Criteria
0.3523 0.2591 0.1566 0.3407 0.2648 0.1803* 0.3639 0.2533 0.1965* 5-Freeform
0.3602 0.2448 0.1325 0.3593 0.2559 0.1365 0.3610 0.2337 0.1284

Table 2: Semantic diversity (cosine distance) results for criteria-based
prompting vs. free-form prompting and LLM variants. 1-criteria refers to one-
shot criteria-based prompting and so on. Text for the lowest similarity score
within the same LLM type is made bold. * p < 0.05 when comparing criteria-
based prompting with free-form prompting

#### Perspective diversity

In order to examine the perspective diversity during the step-by-step recall
prompting, we utilize the criteria words generated by LLM.444When we manually
examine the quality of criteria word clusters by randomly selecting 100
clusters, we find that all clusters make sense. Some words with similar
meanings can be conveyed in different ways. For example, given a statement “It
is expected that friends will enjoy being around each other,” the model could
generate two opinions where one opinion has the word “joy” in its criteria
list and the other has the word “happiness.” Both words have similar meaning,
so we utilize GPT-4 to cluster similar words into one group. Then for
opinionn, if all words in its criteria list have appeared in opinion<n, we
call that there is a duplicate in the generated opinion for that statement. A
perspective diversity score for a statement $S$ which contains a set of
opinions $O_{S}$ = {$o_{1},...,o_{n}$} is the percentage of how many opinions
in $O_{S}$ are not duplicated. The higher the score is, the more diverse the
set of generated opinions is.

(a) Social-Chem-101

(b) CMV

(c) Hate Speech

(d) Moral Stories

(e) Social-Chem-101

(f) CMV

(g) Hate Speech

(h) Moral Stories

Figure 4: (Top) X-axis is the number of generated opinions for our diversity
coverage experiment and Y-axis is the average number of unique criteria
clusters for all statements. (Bottom) X-axis is the number of generated
opinions for our diversity coverage experiment and Y-axis is the average
number of unique criteria clusters for all statements. Moral Stories do not
have stances, so the line is only for all generated continued stories.

##  4 Results with Automatic Evaluation

We summarize the main findings from our experiments in the following
paragraphs:

###  4.1 Semantically Diverse Opinions about Social Norms and Argumentation.

Table 2 presents the results of semantic diversity for our experiments. In
general, one-shot prompting on GPT-4 generates more semantically diverse
reasons compared to ChatGPT and GPT-3 for all agree and disagree cases. We
assume that for five-shot prompting, the model tends to follow the given
examples more strongly so that its generated output is not as diverse as in
the one-shot setting.

For Social-Chem-101, the generated reasons for opinions that disagree with a
statement tend to be more diverse than reasons that agree with the statement.
Due to the nature of the RoTs which are social norms considered mostly good by
perhaps a group of people whose data are also used for training a model, the
model tends to be more agreeable toward the topic of the RoTs. For example,
given the statement “You should go on vacation with your partner,” the model
generates criteria related to “quality time”, “adventure”, and “memories” for
the “Agree” opinions, whereas it can find more diverse topics for the
“Disagree” opinions such as “independence”, “interest”, “financial”,
“logistics,” and “conflict.”

For the argumentation task in CMV, we also see a similar trend that GPT-4 one-
shot criteria-based prompting is the best setting. However, it is not always
the case that disagree reasons are more diverse the agree reasons.

###  4.2 Perspective Diversity in Step-by-Step Recall Prompting

Figure 4 (Top) shows the result for the step-by-step recall prompting. As we
increase the number of generated opinions ($N$) in an input prompt from 2 to
20, the model generates more duplicated criteria clusters. Note that the
saturation point is when at least 80% of the generated opinions do not indeed
reveal a new perspective. When $N=8$, the model reaches its saturation point
for Social-Chem-101 and Hate Speech, but we observe a different trend for CMV
and Moral Stories.

For CMV, the number of duplicated perspectives does not change substantially
when $N=11$. The saturation point is reached when $N>11$. Meanwhile, the line
for Moral Stories is smoother than the line for CMV, the N to reach the
saturation point is also higher (N = 11). This means that CMV and Moral
Stories have a more diverse perspective as it has higher $N$ until the model
reaches the saturation point.

Our hypothesis that a controversial topic means a more diverse set of opinions
is not supported by semantic diversity metrics, as the cosine distance for CMV
is higher than for Social-Chem-101. However, the analysis of perspective
diversity aligns with our hypothesis, affirming that argumentation inherently
exhibits greater subjectivity than social norms. Argumentation tasks tend to
yield a more diverse set of opinions. This is also shown in the task of story
continuation of Moral Stories, due to the open-ended nature of the task
allowing for a multitude of potential storylines.

Another interesting observation is that starting $N>2$, observed among all
datasets, the model will generate duplicate criteria for at least some
statements. Thus, duplication is unavoidable, but we can set a threshold of
how much duplication we allow to model outputs.

Figure 4 (Bottom) displays how many unique criteria clusters are generated as
the $N$ increases. For Social-Chem-101 and CMV tasks, the model on average can
only produce 8 unique criteria clusters for agree and and 7 for disagree
opinions (details on Table 3.

(a) Social-Chem-101

(b) CMV

(c) Hate Speech

(d) Moral Stories

Figure 5: Scatter plot for X= semantic diversity (cosine distance) of opinions
in each statement, Y = perspective diversity (% of statements without
duplicate opinions). A green circle refers to one statement with agree/hate
speech reasons while a red triangle refers to statements with disagree/not
hate opinions. Story continuation in Moral Stories does not have stances and
each story is represented by a purple circle. There is no correlation between
semantic diversity score and perspective diversity for all datasets, except
for Hate Speech where there is a negative correlation (Pearson’s r = =-0.6459
for hate speech labels and -0.4683 for not hate speech labels, p < 0.001).

For Hate Speech, the average number of unique criteria clusters is even lower
(4 for hate and 5 for not hate). This confirms that the hate speech labeling
task is less subjective compared to social norms (Social-Chem-101) or
argumentation (CMV). Again we see a different trend for Moral Stories as the
model can produce a greater number of unique criteria clusters (20 clusters on
the average). Table 3 summarizes how task subjectivity influences the
diversity coverage that LLMs can generate. Examples of three generated
opinions per statement for the recall prompting experiment are shown in Table
4 and Table 5.

###  4.3 Correlation between Semantic Diversity and Perspective Diversity?

We further examine whether there is a correlation between semantic diversity
and perspective diversity. We plot a scatter plot for each statement with an
X-axis for the semantic diversity score and a Y-axis for the perspective
diversity score, as displayed in Figure 5. We also compute Pearson’s
correlation $r$ between the same aspects.

By looking at Figure 5 and the computed correlation score, we find that there
is no linear correlation between semantic diversity score and perspective
diversity score, except for the Hate Speech data. Pearson’s $r$ correlation is
$-0.6459$ when the label is “Hate Speech” and $-0.4683$ for “Not Hate Speech”;
$p<0.001$). As we manually examine the generated criteria words and opinions
for Hate Speech data, we find that the reasons are quite homogeneous and the
criteria words mostly talk about “stereotyping,” “insulting,” “profanity,"
etc., so there are not many new values to be introduced along with the
reasons.

Task Type | Dataset | Max | Median
---|---|---|---
Stance | Social-Chem-101
| Agree | 17 | 8
| Disagree | 16 | 7
Stance | CMV
| Agree | 17 | 7
| Disgree | 14 | 7
Labeling | Hate Speech
| Hate | 14 | 4
| Not Hate | 16 | 5
Generation | Moral Stories
| All | 47 | 20
Table 3: Different numbers of LLMs’ generated unique criteria clusters for
different task types. Max and median refer to the maximum and the median of
the number of unique criteria clusters. More details on Figure 7 are in the
Appendix.

Dataset | Change My View |
---|---|---
Statement | If a child struggles to achieve in grade school it is virtually always the parents’ fault. |
GPT-4 | Agree |  | 1\. Parents have a huge impact on the early education of a child, and struggles in grade school may reflect
---
inadequate parenting.
Criteria: early education, parenting |
| 2\. The family environment and support system play a vital role in a child’s
achievement in school
---
and parents are responsible for creating it.
Criteria: family environment, support system |
| 3\. Parents’ responsibility to instill healthy habits and provide a
conducive study environment can have
---
a major impact on a child’s school performance.
Criteria: healthy habits, study environment |
Disagree |  | 1\. Sometimes, the influence of peers and socio-economic conditions can override parental influence
---
leading to struggles in school.
Criteria: the influence of peers, socio-economic conditions |
| 2\. Systemic issues and lack of resources can be major obstacles to a
child’s academic progress, which
---
parents may not have any control over.
Criteria: Systemic issues, lack of resources |
| 3\. Some children may have undiagnosed psychological issues or a unique
learning style, impacting
---
their academic success.
Criteria: psychological issues,  learning style |
Human | Agree |  | 1\. The child’s parents also need to check their kids study activities while they are at home.
---
| 2\. Sometimes parents allow their child to play for lot of time it leads to
get poor result.
---
| 3\. Virtual schooling puts stress on parents because they have to supervise
what their kids do at home.
---
Disagree |  | 1\. The main reason behind the child’s struggle in the study was teachers they need to monitor the kids.
---
| 2\. Not only does parental involvement lead to improved academic outcomes,
but it also has a positive impact on
---
students’ social and emotional skills and decreases instances of delinquency.
| 3\. Some kids have trouble with online classes because they lack the
structure that in-person school
---
traditionally provides.

Dataset | Social-Chem-101 |
---|---|---
Statement | It’s good to be a hard worker. |
GPT-4 | Agree |  | 1\. Being a hard worker increases productivity and pushes one towards success and achieving their goals.
---
Criteria: productivity, success |
| 2\. Being a hard worker can bring a sense of fulfillment, achievement, and
boost self-esteem.
---
Criteria: fulfillment, achievement,, self-esteem. |
| 3\. Being a hard worker develops resilience, stokes determination and builds
hardiness
---
Criteria: resilience, determination,  hardiness |
Disagree |  | 1\. Being a hard worker can damage work-life balance, cause burnout and high levels of stress.
---
Criteria: work-life balance, burnout, stress |
| 2\. Being a hard worker can decrease leisure time, family time and
negatively affect health.
---
Criteria: leisure, family, health |
| 3\. Being a hard worker can prevent peace of mind, disrupt contentment, and
break restfulness.
---
Criteria: peace of mind, contentment, restfulness |
Human | Agree |  | 1\. Successful people are not gifted; they just work hard, then succeed on purpose.
---
| 2\. Through hard word anyone could earn good social status.
---
| 3\. Ethically hard working is considered as a good quality.
---
Disagree |  | 1\. Hard work isn’t enough to succeed in today’s world.
---
| 2\. Hard wording drains more energy.
---
| 3\. Working hard can take away from other commitments in your life like
family.
---

Table 4: Examples of opinions generated by humans and GPT-4 by different
stances about a statement from CMV (Top) and Social-Chem-101 (Bottom).

Dataset | Hate Speech |
---|---|---
Statement | Your burqini ain’t welcome at the pool, it’s BIKINI or swimsuit. |
GPT-4 | Hate Speech |  | 1\. They discern this as an attempt to exclude and belittle individuals who chooses to wear burqini,
---
therefore promoting inequality.
Criteria: exclusion, belittlement |
| 2\. They see this as a lack of acceptance towards other cultures and
promoting a specific dress code
---
by discriminating certain types of wear.
Criteria: lack of acceptance, discrimination |
| 3.They categorize the statement as hateful because it manifests intolerance
and bigotry against
---
individuals who choose to wear burqini.
Criteria: intolerance, bigotry |
Not Hate Speech |  | 1.They believe the statement expresses personal views about swimwear which is part of
---
the freedom of expression.
Criteria: personal views, freedom of expression |
| 2\. They perceive the statement as hyperbolic or exaggerated, not intended
to cause harm or incite hate.
---
Criteria: hyperbole, exaggeration |
| 3\. They may interpret this statement as a joke or a misunderstanding,
rather than a deliberate attack
---
on a specific group.
Criteria: joke,  misunderstanding |

Dataset | Moral Stories |
---|---|---
Statement | Amanda was upset that a stranger cut in line in front of her. |
GPT-4 | Continued Story 1 |  | However, instead of getting angry, Amanda decided to calmly explain to the stranger
---
why queueing was important.
Reason: I chose to continue the story this way to show Amanda’s reaction to
the situation,
using communication instead of confrontation to express her displeasure about
the situation.
Criteria: non-confrontation, communication |
Continued Story 2 |  | Amanda scoffed audibly, rolled her eyes and muttered an ironic ‘nice manners’ under her breath.
---
Reason: I chose this continuation as it displays a subtle way of confrontation
where Amanda indirectly addresses the situation using sarcasm.
Criteria: sarcasm, indirect confrontation |
Continued Story 3 |  | “Excuse me, but I believe I was next.” Amanda said in a firm yet polite voice.
---
Reason: I chose this continuation to depict Amanda as an assertive and polite
person
who doesn’t hesitate to stand up for what is right.
Criteria: assertiveness, politeness |

Table 5: Examples of GPT-4 generated opinions for a statement from Hate Speech
(Top) and another from Moral Stories (Bottom).

##  5 Results with Human Evaluation

To evaluate the effectiveness of criteria-based prompting as a method for
extracting maximum diversity from LLMs, we involved the crowd workers to
perform these two tasks: (1) human preference for LLMs’ generation with
criteria-based prompting and (2) human capability of writing diverse
opinions.555More details on AMT experiments setup, remuneration, and
interfaces are described in Appendix A.6 and A.7.

###  5.1 Human Preference on Criteria-based Prompting

With three recruited workers via Amazon Mechanical Turk (AMT), we examined
whether humans prefer opinions generated by criteria-based prompting vs. free-
form prompting, in terms of “perspective diversity.” Fleiss’s Kappa among
three annotators was on average 0.46 for both Social-Chem-101 and CMV
datasets, signifying a moderate agreement among the workers for both datasets.

We computed the proportion of statements where criteria-based prompting wins,
out of the 50 randomly selected statements from Social-Chem-101 and 30 from
CMV datasets (denoted as “win rate of criteria-based prompting”). Table 6
shows the win rate of criteria-based prompting on each stance session of both
datasets. In general, the opinions generated by criteria-based prompting were
preferred by humans in more than half of the total statements, except for the
“agree” session of CMV. This indicates that in more than half of the total
samples, a majority of crowd-workers consider the set of opinions about a
statement generated by criteria-based prompting as one presenting more diverse
reasons that support the stance of the opinions.

###  5.2 Human Capabilities on Diverse Perspective Generation

Lastly, we measured the human capacity to generate diverse opinions for any
given statement. First, we used the same 100 and 67 statements from Social-
Chem-101 and CMV that we used for the step-by-step recall prompting experiment
(Section 4.2). Then for each statement, we asked workers to complete the
following three tasks: (1) choose their stance with the statement (e.g., agree
or disagree); (2) write opinions of agreement as many as they can; and (3)
write opinions of disagreement as many as they can, regardless of their
position on the statement. Therefore, we gathered five workers’ stances and
opinions for each statement, resulting in a collection of 500 and 335 human-
written opinions for 100 statements of Social-Chem-101 and 67 statements of
CMV, respectively.

| Social-chem-101 | CMV
---|---|---
Agree | 0.58 | 0.37
Disagree | 0.6 | 0.53
Table 6: Win rate of criteria prompting for opinions of each stance. For
example, humans preferred the “agree” opinions generated by criteria-based
prompting with 58% out of the total given statements.

|  | Social-Chem-101 | CMV
---|---|---|---
Agree | Human | 2.39 $\pm 0.76$ | 1.70$\pm 0.66$
GPT-4 | 6.25$\pm 1.93$ | 6.24 $\pm 1.72$
Disagree | Human | 2.03 $\pm 0.83$ | 1.38$\pm 0.48$
GPT-4 | 6.01 $\pm 1.84$ | 6.21 $\pm 2.05$

Table 7: Average number of human-written opinions per statement with standard
deviations, versus GPT-4. For example, the workers generated an average of
2.39 opinions for a statement when asked to provide opinions that agree with
the statement. We observe that GPT-4 generated more opinions than humans.

As shown in Table 7, humans tend to present an average of 2 opinions for
statements from Social-Chem-101 and 1 opinion for CMV. Considering the task
characteristics between the two datasets, this indicates that humans are
likely to make fewer opinions to the statements that take on more
controversial and subjective sentiments. Also, we can also see that GPT-4 can
generate more opinions compared to humans 666This is done after filtering out
opinions with duplicated criteria clusters. This confirms GPT-4’s capability
of modeling more diverse perspectives that it gains from its training phase as
compared to a much smaller number of humans (5 people) in this study.

(a) Human Stance: Agree

(b) Human Stance: Disagree

(c) Human Stance: Agree

(d) Human Stance: Disagree

Figure 6: The distribution of human-written opinions, separated by the human
stance on given statements, in Social-chem-101 (top) and CMV (bottom). Paler-
colored bars represent instances where participants were asked to write
opinions that opposed their personal stances on a statement. Each bar
indicates the count of participants who provided the number of opinions
corresponding to the bar’s position on the x-axis.

In Figure 6, we observe the common trend that there is a higher number of
people who provided zero opinions to the stance that they do not support.
According to Figure 6(a) and 6(c), the number of people who personally agree
with a given statement but give zero opinions of disagreement is higher than
that of people who also agree to the statement but are asked to provide
reasons of agreement.

Likewise, the number of people providing zero opinions is more concentrated
when workers disagree with a given statement but are asked to explain the
rationales behind the agreement, compared to that of people who were asked to
provide disagreement (Figures 6(b) and 6(d)). This finding informs us that
humans tend to present their opinions only to the stance they stand with. Even
if they provided multiple opinions on the stance against theirs, as observed
in all four subplots of Figure 6, the number of people who provided multiple
opinions ($>1$) on a stance against their personal stance is comparatively
lower than that on the stance that they stand with. This also indicates that
humans are likely to be more conservative toward the stances against their
preference.

##  6 Related Work

Diversity What is measured? Common metrics Lexical Unique words Ngram overlap,
BLEU Syntactic Unique POS Distance between POS Semantic Embedding distance
BERTScore Perspective Stances/Viewpoints Ours

Table 8: Different types of measuring diversity for text generation task. Our
work introduces a new metric - perspective diversity.

#### Diversity in NLP

Diversity in NLP has been extensively studied in various aspects, such as
lexical variability Dušek and Kasner (2020); Tevet and Berant (2021); Li et
al. (2016), syntactical diversity Giulianelli et al. (2023); Huang et al.
(2023), semantic diversity Stasaski and Hearst (2022); Reimers and Gurevych
(2019); Zhang et al. (2020), and perspective diversity Plank (2022); Hayati et
al. (2021); Santy et al. (2023) (Table 8). Some studies focus on the diversity
among annotators Rottger et al. (2022); Wich et al. (2021); others focus on
the diversity of the generated language Hashimoto et al. (2019); Liu et al.
(2023). Our work is closely related to prior studies on perspective diversity,
as we focus on examining the generated stances and rationales obtained from
LLMs. Previous work on perspective diversity has predominantly revolved around
categorical labels. However, our investigation extends beyond mere categorical
labels to encompass sentence-level reasoning diversity, thereby introducing a
more nuanced perspective. Joshi et al. (2020) argue that NLP field is biased
toward Western perspectives. As a result, the viewpoints of minority
populations are relatively overlooked, as their perspectives may significantly
diverge from the majority. Thus, we argue that our work is important to
uncover the extent to which perspective diversity can be extracted from LLMs.

#### Diversity Generation by LLMs

Diversity generation by LLM has been tightly correlated with using LLM for
producing diverse synthetic datasets. For example, Cegin et al. (2023)
proposes ChatGPT to generate paraphrasing data as they show that the
paraphrases generated by ChatGPT have higher lexical and syntactical diversity
compared to crowd-sourced datasets. Yuan et al. (2021) generates a synthetic
dataset of structured wiki-like bios for notable people using LLMs. Wang et
al. (2023); Taori et al. (2023); Honovich et al. (2023) utilize ChatGPT to
generate an instruction dataset and claim that their data is more diverse than
human-written instructions. Unlike other previous works that focus on
generating diverse large-scale data, Lahoti et al. (2023); Giulianelli et al.
(2023) closely examine variability within the model responses. Our work is
closely related to Lahoti et al. (2023) in that we both promote fairness in
LLM generation through diversity. Lahoti et al. (2023) proposes a new
prompting technique - collective-critique and self-voting (CCSV) - to improve
the diversity of representations in an LLM’s response for human prompts
related to gender and culture. Our work is similar since perspective diversity
promotes fairness in LLMs rather than merely the semantic content variability.
In addition, our work is also quite similar to Giulianelli et al. (2023) as we
generate more than one response for a single prompt. In order to promote
opinion diversity of LLMs, Aroyo et al. (2023) introduces a new dataset which
is labeled by human raters from different demographic populations (gender,
age, ethnic/racial groups) and focuses specifically on safety (bias,
misinformation, harmful content, etc.,).

Despite the previous claim that LLMs can produce diverse content, some studies
have found that co-writing with some LLMs can affect human writers’ opinions
Jakesch et al. (2023) and may reduce the writing diversity Padmakumar and He
(2023). Our work addresses the gap by introducing a method to generate more
diverse perspectives instead of only one dominant opinion.

#### Reasoning by LLMs

When we prompt an LLM to generate as many diverse responses as possible, we
also ask the model to explain itself by providing their reasoning. Our work
was inspired by the Chain-of-Thought (CoT) prompting which allows the model to
show reasoning abilities through few-shot demonstrations Wei et al. (2023);
Kojima et al. (2022). While CoT prompting deals with non-subjective tasks,
such as arithmetic and commonsense reasoning, our work involves subjective
task reasoning, particularly argumentative and social topics.

##  7 Conclusion and Future Work

To the best of our knowledge, this is the first work that tackles the maximum
perspective diversity extraction from LLMs. In this study, we propose a new
problem of probing LLMs’ capacity to generate perspective diversity and to
explain their stances through subjective reasoning. Our analysis shows that
semantic diversity is not always positively correlated with perspective
diversity. Thus, we suggest that when humans collaborate with LLMs for
diversity generation for social tasks, we should take perspective diversity
extraction into consideration, instead of only semantic diversity. While the
number of criteria clusters does not precisely mean the ideal maximum
diversity, it indicates that we could use LLMs to push further perspective
diversity to include more marginalized people’s opinions. This study suggests
that there is a bigger room to explore for studying more advanced diversity
"quantification" or "maximization" methods in the future.

There are also many application possibilities for extending this work in the
future, such as inspecting the cultural aspects, persona, or human values.
Looking at the criteria words across LLMs and examining the usefulness of
having diverse data for downstream tasks, such as diverse outputs for
instruction-tuning training data, could also be possible future work.

In addition, we observe in the human-written opinions that people were not
likely to embrace more perspectives against their stance on a statement. We
suggest that humans’ interaction with LLMs powered by our criteria-based
prompting can also maximize diversity within the human perspectives, creating
more inclusive environments within a society. Feasible future work could also
be the evaluation of criteria-based prompting on human perspectives in an
interactive setting.

## Limitations

While our prompting approach does not generate an exhaustive, complete list of
diverse opinions, our study serves as a comparative study that examines the
capability of various language models for generating diverse opinions given
various numbers of examples and input datasets. Moreover, for now, we only
experimented with our proposed criteria-based prompting technique for
subjective tasks. It would be interesting future work to try the same
technique on non-subjective tasks.

## Acknowledgment

We would like to thank members of the Minnesota NLP lab and Google reviewers
for their valuable feedback and suggestions on the paper draft.

## References

  * Abercrombie et al. (2022) Gavin Abercrombie, Valerio Basile, Sara Tonelli, Verena Rieser, and Alexandra Uma, editors. 2022.  _Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022_. European Language Resources Association, Marseille, France.
  * Ajzen (1991) Icek Ajzen. 1991.  The theory of planned behavior.  _Organizational behavior and human decision processes_ , 50(2):179–211.
  * Aroyo et al. (2023) Lora Aroyo, Alex S Taylor, Mark Diaz, Christopher M Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, and Ding Wang. 2023.  Dices dataset: Diversity in conversational ai evaluation for safety.  _arXiv preprint arXiv:2306.11247_.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Bubeck et al. (2023) Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023.  Sparks of artificial general intelligence: Early experiments with gpt-4.  _arXiv e-prints_ , pages arXiv–2303.
  * Cegin et al. (2023) Jan Cegin, Jakub Simko, and Peter Brusilovsky. 2023.  Chatgpt to replace crowdsourcing of paraphrases for intent classification: Higher diversity and comparable model robustness.  _arXiv preprint arXiv:2305.12947_.
  * Chen et al. (2023) Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, and Dilek Hakkani-Tur. 2023.  PLACES: Prompting language models for social conversation synthesis.  In _Findings of the Association for Computational Linguistics: EACL 2023_ , pages 844–868, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Chung et al. (2023) John Chung, Ece Kamar, and Saleema Amershi. 2023.  Increasing diversity while maintaining accuracy: Text data generation with large language models and human interventions.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 575–593, Toronto, Canada. Association for Computational Linguistics.
  * Dušek and Kasner (2020) Ondřej Dušek and Zdeněk Kasner. 2020.  Evaluating semantic accuracy of data-to-text generation with natural language inference.  In _Proceedings of the 13th International Conference on Natural Language Generation_ , pages 131–137, Dublin, Ireland. Association for Computational Linguistics.
  * Emelin et al. (2021) Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, and Yejin Choi. 2021.  Moral stories: Situated reasoning about norms, intents, actions, and their consequences.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 698–718, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Forbes et al. (2020) Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020.  Social chemistry 101: Learning to reason about social and moral norms.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 653–670, Online. Association for Computational Linguistics.
  * Ghosh et al. (2021) Sayan Ghosh, Dylan Baker, David Jurgens, and Vinodkumar Prabhakaran. 2021.  Detecting cross-geographic biases in toxicity modeling on social media.  In _Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)_ , pages 313–328, Online. Association for Computational Linguistics.
  * Giulianelli et al. (2023) Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, and Barbara Plank. 2023.  What comes next? evaluating uncertainty in neural text generators against human production variability.  _arXiv preprint arXiv:2305.11707_.
  * Hashimoto et al. (2019) Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. 2019.  Unifying human and statistical evaluation for natural language generation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1689–1701, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Hayati et al. (2021) Shirley Anugrah Hayati, Dongyeop Kang, and Lyle Ungar. 2021.  Does BERT learn as humans perceive? understanding linguistic styles through lexica.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6323–6331, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Hidey et al. (2017) Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan, and Kathy McKeown. 2017.  Analyzing the semantic types of claims and premises in an online persuasive forum.  In _Proceedings of the 4th Workshop on Argument Mining_ , pages 11–21, Copenhagen, Denmark. Association for Computational Linguistics.
  * Honovich et al. (2023) Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023.  Unnatural instructions: Tuning language models with (almost) no human labor.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 14409–14428, Toronto, Canada. Association for Computational Linguistics.
  * Huang et al. (2023) Kuan-Hao Huang, Varun Iyer, I-Hung Hsu, Anoop Kumar, Kai-Wei Chang, and Aram Galstyan. 2023.  ParaAMR: A large-scale syntactically diverse paraphrase dataset by AMR back-translation.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8047–8061, Toronto, Canada. Association for Computational Linguistics.
  * Jakesch et al. (2023) Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023.  Co-writing with opinionated language models affects users’ views.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , pages 1–15.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6282–6293, Online. Association for Computational Linguistics.
  * Kesberg and Keller (2018) Rebekka Kesberg and Johannes Keller. 2018.  The relation between human values and perceived situation characteristics in everyday life.  _Frontiers in psychology_ , 9:1676.
  * Kim et al. (2022) Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. 2022.  ProsocialDialog: A prosocial backbone for conversational agents.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 4005–4029, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.  Large language models are zero-shot reasoners.  _Advances in neural information processing systems_ , 35:22199–22213.
  * Lahoti et al. (2023) Preethi Lahoti, Nicholas Blumm, Xiao Ma, Raghavendra Kotikalapudi, Sahitya Potluri, Qijun Tan, Hansa Srinivasan, Ben Packer, Ahmad Beirami, Alex Beutel, et al. 2023.  Improving diversity of demographic representation in large language models via collective-critiques and self-voting.  _arXiv preprint arXiv:2310.16523_.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 110–119, San Diego, California. Association for Computational Linguistics.
  * Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023.  Gpteval: Nlg evaluation using gpt-4 with better human alignment.  _arXiv preprint arXiv:2303.16634_.
  * Lu et al. (2022) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8086–8098, Dublin, Ireland. Association for Computational Linguistics.
  * Min et al. (2022a) Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022a.  Metaicl: Learning to learn in context.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2791–2809.
  * Min et al. (2022b) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022b.  Rethinking the role of demonstrations: What makes in-context learning work? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 11048–11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report. arxiv 2303.08774.  _View in Article_.
  * Padmakumar and He (2023) Vishakh Padmakumar and He He. 2023.  Does writing with language models reduce content diversity?  _arXiv preprint arXiv:2309.05196_.
  * Perez et al. (2021) Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021.  True few-shot learning with language models.  _Advances in neural information processing systems_ , 34:11054–11070.
  * Plank (2022) Barbara Plank. 2022.  The “problem” of human label variation: On ground truth in data, modeling and evaluation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Rokeach (1973) Milton Rokeach. 1973.  _The nature of human values._ Free press.
  * Rottger et al. (2022) Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. 2022.  Two contrasting data annotation paradigms for subjective NLP tasks.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 175–190, Seattle, United States. Association for Computational Linguistics.
  * Santy et al. (2023) Sebastin Santy, Jenny Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. 2023.  NLPositionality: Characterizing design biases of datasets and models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 9080–9102, Toronto, Canada. Association for Computational Linguistics.
  * Sap et al. (2019) Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019.  The risk of racial bias in hate speech detection.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1668–1678, Florence, Italy. Association for Computational Linguistics.
  * Shweder (1990) Richard A Shweder. 1990.  In defense of moral realism: Reply to gabennesch.  _Child Development_ , 61(6):2060–2067.
  * Stasaski and Hearst (2022) Katherine Stasaski and Marti A Hearst. 2022.  Semantic diversity in dialogue with natural language inference.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 85–98.
  * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca.
  * Tevet and Berant (2021) Guy Tevet and Jonathan Berant. 2021.  Evaluating the evaluation of diversity in natural language generation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 326–346, Online. Association for Computational Linguistics.
  * Van Eemeren et al. (2015) Frans H Van Eemeren, Frans H van Eemeren, Sally Jackson, and Scott Jacobs. 2015.  Argumentation.  _Reasonableness and effectiveness in argumentative discourse: Fifty contributions to the development of Pragma-dialectics_ , pages 3–25.
  * Vidgen et al. (2021) Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021.  Learning from the worst: Dynamically generated datasets to improve online hate detection.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1667–1682, Online. Association for Computational Linguistics.
  * Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  Self-instruct: Aligning language models with self-generated instructions.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 13484–13508, Toronto, Canada. Association for Computational Linguistics.
  * Wei et al. (2023) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, Denny Zhou, et al. 2023.  Chain-of-thought prompting elicits reasoning in large language models.  In _Advances in Neural Information Processing Systems_.
  * Wich et al. (2021) Maximilian Wich, Christian Widmer, Gerhard Hagerer, and Georg Groh. 2021.  Investigating annotator bias in abusive language datasets.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)_ , pages 1515–1525, Held Online. INCOMA Ltd.
  * Yuan et al. (2021) Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann. 2021.  Synthbio: A case study in human-ai collaborative curation of text datasets.  _arXiv preprint arXiv:2111.06467_.
  * Zhang et al. (2020) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.

##  Appendix A Appendix

###  A.1 Prompting

  * •

Free-form prompting

Statement: It’s rude to use profanity.

Tell me opinions about the statement as many as possible from different people
with "Agree" or "Disagree" and explain how they have different opinions.

Output:

{1: {"Stance" : ‘‘Agree’’, ‘‘Reason": ‘‘I was brought up in a family and
culture where using profanity was considered impolite and disrespectful, and I
have continued to hold that belief in my adult life...

  * •

Prompt for criteria-based prompting:

Statement: It’s rude to use profanity.

Tell me opinions about the statement as many as possible from different people
with "Agree" or "Disagree", one-word or one-phrase criteria that is important
for their opinions, and explain how they have different opinions.

Output:

{1: {"Stance" : ‘‘Agree’’, ‘‘Criteria": [‘‘impolite’’, ‘‘disrespectful’’,
‘‘culture’’, ‘‘family’’], "Reason": ‘‘I was brought up in a family and culture
where using profanity was considered impolite and disrespectful, and I have
continued to hold that belief in my adult life...

  * •

Prompt for criteria-based step-by-step recall prompting (n = 2):

Statement: It’s rude to use profanity.

Tell me opinions about the statement as many as possible from 2 different
people with "Agree" or "Disagree", one-word or one-phrase criteria that is
important for their opinions, and explain how they have different opinions.

Output:

{1: {"Stance" : ‘‘Agree’’, ‘‘Criteria": [‘‘impolite’’, ‘‘disrespectful’’,
‘‘culture’’, ‘‘family’’], "Reason": ‘‘I was brought up in a family and culture
where using profanity was considered impolite and disrespectful, and I have
continued to hold that belief in my adult life...

Model Input (one-shot example below)
---
Statement: It’s okay to have privacy
Tell me opinions about the statement as many as possible from different people
with, “Agree” or “Disagree,” one-word or
one-phrase criteria that is important for their opinions, and explain how they
have different opinions
Output:
{1:{‘‘Stance’’ :‘‘Agree’’,
‘‘Criteria’’: [‘‘personal boundaries’’, ‘‘autonomy’’],
‘‘Reason’’: ‘‘Having privacy allows individuals to establish personal
boundaries and maintain.’’
their autonomy."},
2: {‘‘Stance’’: ‘‘Disagree",
‘‘Criteria’’: [‘‘transparency’’, ‘‘trust’’],
‘‘Reason’’: ‘‘Lack of privacy can promote transparency and build trust in
relationships.’’
…
10: {‘‘Stance’’: ‘‘...", ‘‘Criteria’’: [...], ‘‘Reason’’: ‘‘ ..."}}
…
Statement: You’re expected to do what you are told
Tell me opinions about the statement as many as possible from different people
with, “Agree” or “Disagree,” one-word or
one-phrase criteria that is important for their opinions, and explain how they
have different opinions
Output:
Model Output
{1:{‘‘Stance’’ : ‘‘Agree",
‘‘Criteria’’: [‘‘teamwork’’, ‘‘goals’’],
‘‘Reason’’: ‘‘In a team setting, following instructions or orders can be
necessary for achieving
shared goals.’’},
…
10: {‘‘Stance’’: ‘‘...", ‘‘Criteria’’: [...], ‘‘Reason’’: ‘‘ ..."} }
Table 9: Example prompt for criteria-based diversity prompting. For the Hate
Speech prediction task, we ask the model to provide the label of the statement
with “Hate Speech” or “Not Hate Speech”. For Moral Stories, we ask the model
the continue the story.

###  A.2 Example Generated Opinions

Example generated opinions by various LLMs for a statement from Social-
Chem-101 are shown in Figure 8 while those written by human crowd-workers are
shown in Figure 9. For LLMs’ opinions, we only show the first 3 opinions due
to space.

###  A.3 Recall Results

Figure 7 shows how many statements generate that many unique criteria
clusters. The minimum number could be 0 because GPT-4 clustering is not 100%
covering all the words. During our robustness check on the CMV dataset, 0.4%
of the criteria words are not grouped (5 out of 1276 in criteria words).

(a) Social-Chem-101

(b) CMV

(c) Hate Speech

(d) Moral Stories

Figure 7: X-axis is the number of unique clusters while Y-axis is the number
of statements which have that number of unique clusters.  Figure 8: Social-
Chem-101 generated opinions from various LLMs. Figure 9: Human-written
opinions for Social-Chem-101 statements. Each crowd-worker are asked to share
their personal stance (Agree or Disagree), and then give as many opinions as
possible for both stances regardless their personal stance.

###  A.4 ChatGPT’s Capability on Instruction Alignment

We analyzed if each opinion generated by ChatGPT with a one-shot setting not
only entails the corresponding statement but also aligns with the appropriate
stance. In our approach, we used stratified sampling to validate the quality
of opinions by criteria-based and free-form prompting methods, thus obtaining
a total of 100 opinions from each dataset: 50 were generated by criteria-based
prompting, and 50 by free-form prompting. From these, we randomly selected 25
opinions that agreed with a given statement and 25 that were in disagreement.
Then, we examined the following two criteria for each pair of statement and
opinion: (1) if an opinion entails the statement and (2) if the stance of an
opinion aligns with the reason in the opinion. 777The computed percent
agreement between authors was 98% and 99% for (1) and (2), respectively.

We observed that all 100 and 99 opinions from Social-Chem-101 and CMV
respectively were found to entail their corresponding statements, indicating
that the ChatGPT-generated opinions tend to correctly understand the context
of each given statement and follow the instructions of the prompts.

In addition, more than 98% of the samples in each of the two datasets have
their opinions regarding a given statement aligned with the generated stance.
We found a misalignment mostly from the opinions generated by ChatGPT that
disagree with their statement. For example, in the Social-Chem-101 dataset,
for a statement “It’s okay to want your partner to do more chores.” ChatGPT
generated with criteria-based prompting the following opinion that disagrees:
“It’s important to recognize that everyone has different strengths and
weaknesses, and it’s okay if one partner is better at certain chores than the
other. The reason for the opinion itself is contrary to its stance, as it in
fact supports the attitude of people making their partner do more chores.

We also found another type of logical contradiction in a sample of cmv that
shows misalignment between the stance and the reason for its generated
opinions. In the statement The EGG came first before the chicken, ChatGPT
generates with free-from prompting the following disagreeing opinion: “The
genetic mutation that led to the first chicken would have occurred in the
chicken’s parents, so the chicken came first.” The opinion sounds seemingly
reasonable that supports chicken before egg; however, the paradox inside it
(i.e., the world’s first chicken’s parents are chickens.) does not support its
stance. As a result, we determined this example is the only sample that shows
a contradiction between opinion and statement as well.

###  A.5 Evaluation of Generated Criteria Words

(Total # opinions = 100) Social-Chem-101 CMV Entailment of Criteria (%)
Explicit Criteria (%) Implicit Criteria (%) Entailment of Criteria (%)
Explicit Criteria (%) Implicit Criteria (%) ChatGPT 1-shot 96 72 24 99 84 15
5-shot 100 75 25 100 80 20 GPT-3 1-shot 100 53 47 99 79 20 5-shot 100 68 32 98
82 16

Table 10: Number of generated opinions with criteria-based prompting in four
different LLM variants. We have randomly sampled 100 opinions for each LLM
variant and counted the number of opinions under that category. For example,
96 (%) out of 100 opinions generated by criteria-prompting with ChatGPT 1-shot
setting were entailed by its respective criteria words.

We examined whether the criteria words generated by ChatGPT deliver the main
message of the corresponding opinion 888The percent agreement between authors
was greater than 98% for all LLM variants for both datasets.. We found that at
least 96% of 100 opinions generated by all LLM variants entailed its
corresponding criteria words, for both Social-Chem-101 and CMV datasets (Table
10).

Although most generated criteria words directly match their corresponding
opinions (denoted as ’explicit’), we observed that LLMs can also generate
“implicit” criteria words, subtly conveying the reasoning behind opinions. For
instance, the opinion “Some people are just more irritable by nature, and
there’s nothing wrong with that.” implicitly contains criteria personality and
temperament, which resonate with the core idea of being irritable by nature.

While most of the generated criteria words seemed to entail their
corresponding opinions, we also found some samples whose criteria words could
be added by different words in the opinions for better representation. For
example, for a statement “If you don’t want to be somewhere anymore, you
should leave” ChatGPT in one-shot setting generated the following opinion with
bolded criteria words: “Leaving a situation that goes against one’s personal
values and integrity is an act of self-respect and authenticity.” However, we
agreed to consider self-respect and authenticity as the additional criteria
words, since these words also emphasize an individual’s ability to determine
their stance in such a situation.

Also, there are some samples of criteria words that entail their corresponding
opinion but do not cover the main value of the opinion. For a statement “If
you’re a nudist, you should go to a nudist beach”, the criteria words
generated by GPT-3 in a one-shot setting are nudist and beach in the following
opinion: If you’re a nudist, you should go to a nudist beach so you can be
around like-minded people and feel comfortable. However, these words are just
the repetition of the exact words in the statement and do not deliver the main
reason behind the opinion; that is, forming a community with the same
perspective and value. We agreed to decide like-minded, comfortable, or
community as better criteria words of this opinion alternatively.

###  A.6 Details about AMT Experiment Setups

#### For Section 5.1

We recruited three workers from the Amazon Mechanical Turk (AMT) platform,
each of whom (1) lives in one of the five English-speaking countries (U.S.,
Canada, Australia, New Zealand, and the United Kingdom), (2) achieved a Human
Intelligence Task (HIT) approval rate of 98% or higher, as well as (3) the
number of HITs approved greater than 10000 on the platform. Each HIT consists
of five statements and a pairwise comparison set of A and B, where A and B are
either criteria-prompting or free-form prompting outputs under anonymity.

For each statement, workers were then asked to provide their own stance on
that statement and choose between A and B in terms of which set possesses a
more diverse perspective toward the statement. To acquire a pool of workers
with better-quality responses, we ran a training session before the real task,
where we also filtered out the workers whose responses did not make sense.
Only the workers who showed above a threshold of our own were able to proceed
to the real tasks. For their efforts, each participant received at least $0.5
USD payment for completing the HIT regardless of the quality of their
responses, considering the simplicity of the task and an anticipated time of
completion to be less than 15 minutes. For those who showed a well-done
performance that passed our threshold, we remunerated each of them with a
bonus of at least $2 (USD).

First, we randomly sampled 50 statements from a dataset. For each statement,
we created two separate evaluation sessions, one for the ’agree’ stance and
the other for ’disagree.’ In each session, we present the two sets of opinions
that have that stance and are generated by ChatGPT in an one-shot setting: (1)
Set A, generated with criteria-based prompting, and (2) Set B with free-form
prompting. The workers were first asked to choose their own stance on the
statement. Then, for each session, they select either set A or B, which they
think includes a greater diversity of perspectives that show the same stance.

After the response collection, we aggregated the majority of workers’
preferences per statement under each stance session, and if the majority
preference is the set generated by criteria-based prompting, we considered
that criteria-prompting wins over the free-form prompting for the statement.
Lastly, we computed the proportion of statements where criteria-based
prompting wins, out of the 50 statements (denoted as “win rate of criteria-
based prompting”).

#### For Section 5.2

Similarly, we engaged five workers from the Amazon Mechanical Turk (AMT)
platform, each of whom (1) lives in one of the five English-speaking countries
(U.S., Canada, Australia, New Zealand, and the United Kingdom), (2) achieved a
Human Intelligence Task (HIT) approval rate of 99% or higher, as well as (3)
the number of HITs approved greater than 10000 on the platform. Each HIT
consists of five statements, and per statement each worker was supposed to
provide opinions that both agree and disagree with the statement as much as
they could, regardless of their personal stance on the statement.

To acquire a pool of workers with better-quality responses, we manually
reviewed every response from HITs once provided, filtering out the responses
that fell short into the following types: (1) if a worker did not provide any
reason; (2) if a worker repeated writing the statement without providing their
own reasons; and (3) if a worker’s response consists of irrelevant texts
(e.g., good, bad, I love it.). We compensated those participants with $3 USD
payment for their participation. For those whose responses were accepted, we
gave them a bonus payment of $2 (USD).

Tables 11, 12, and 13 presents the demographic details of the participants in
Section 5.2, for both Social-Chem-101 and CMV datasets.

Age group |  | # Participants
---
(Social-Chem-101)
| # Participants
---
(CMV)
17 - 24 | 6 | 5
25 - 34 | 42 | 31
35 - 44 | 35 | 10
45 - 54 | 8 | 0
55 - 64 | 6 | 5
65 - 74 | 2 | 0

Table 11: Demographic statistics of the participants of Section 5.2 \- (1) Age
Group. Most of the participants belong to the age group between 25 to 44.

Race group |  | # Participants
---
(Social-Chem-101)
| # Participants
---
(CMV)
White | 89 | 63
Asian | 7 | 0
Black | 1 | 0
Native | 1 | 0
Pacific | 1 | 1

Table 12: Demographic statistics of the participants of Section 5.2 \- (2)
Race Group. Almost all of the participants identified themselves as White.

| Highest
---
Education
Level
| # Participants
---
(Social-Chem-101)
| # Participants
---
(CMV)
High School | 3 | 1
Associate | 6 | 1
Bachelor’s | 67 | 38
Master’s | 23 | 24

Table 13: Demographic statistics of the participants of Section 5.2 \- (3)
Highest Education Attainment. We observed most of the participants obtained
bachelor’s degrees.

###  A.7 Interface Design

We present the interface design temple for each of the AMT experiment setups
(Section 5.1 and 5.2). The original prototype of the interfaces presented in
Figures 10 and 11 is referenced from Hayati et al. (2021).

Figure 10: The AMT interface design for the human evaluation experiment in
Section 5.1. Figure 11: The AMT interface design for the human evaluation
experiment in Section 5.2.