# Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use
Cases, Harms and Benefits

Jimin Mun<EMAIL_ADDRESS>Carnegie Mellon UniversityPittsburghUSA ,
Liwei Jiang  University of Washington / Allen Institute for AISeattleUSA ,
Jenny Liang  Carnegie Mellon UniversityPittsburghUSA ,  Inyoung Cheong
University of WashingtonSeattleUSA ,  Nicole DeCario  Allen Institute for
AISeattleUSA ,  Yejin Choi  University of Washington / Allen Institute for
AISeattleUSA ,  Tadayoshi Kohno  University of WashingtonSeattleUSA and
Maarten Sap  Carnegie Mellon UniversityPittsburghUSA

(2018; 20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

General purpose AI, such as ChatGPT, seems to have lowered the barriers for
the public to use AI and harness its power. However, the governance and
development of AI still remain in the hands of a few, and the pace of
development is accelerating without proper assessment of risks. As a first
step towards democratic governance and risk assessment of AI, we introduce
Particip-AI, a framework to gather current and future AI use cases and their
harms and benefits from non-expert public. Our framework allows us to study
more nuanced and detailed public opinions on AI through collecting use cases,
surfacing diverse harms through risk assessment under alternate scenarios
(i.e., developing and not developing a use case), and illuminating tensions
over AI development through making a concluding choice on its development. To
showcase the promise of our framework towards guiding democratic AI, we gather
responses from 295 demographically diverse participants. We find that
participants’ responses emphasize applications for personal life and society,
contrasting with most current AI development’s business focus. This shows the
value of surfacing diverse harms that are complementary to expert assessments.
Furthermore, we found that perceived impact of not developing use cases
predicted participants’ judgements of whether AI use cases should be
developed, and highlighted lay users’ concerns of techno-solutionism. We
conclude with a discussion on how frameworks like Particip-AI can further
guide democratic AI governance and regulation.

AI Governance, AI Ethics, Public Participation, AI Harms, Responsible

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY

##  1\. Introduction

With the arrival of general-purpose models such as ChatGPT and platforms like
GPT Store,111https://openai.com/blog/introducing-the-gpt-store building or
using AI applications has become increasingly accessible for everyday people
(Viswanathan et al., 2023; Center, 2023), leading to numerous beneficial use
cases of AI (Paris and Buchanan, 2023; Waxman, 2023; Metz, 2024) along with
potential risks (Maslej et al., 2023; Roose, 2023a, b). At first glance, the
emergence of new use cases appears to reduce the barriers to AI participation
and promote its “democratization.” However, the restricted access to AI models
reinforces standards established by major tech companies, resulting in an AI
landscape that is alarmingly closed and centralized (Widder et al., 2023;
Brynjolfsson, 2023). Companies possessing frontier models entrench their
monopoly power of AI through user policies (OpenAI, [n. d.]), safety
statements (OpenAI, 2023; Google, [n. d.]), and opaque design decisions
(Widder et al., 2023) that prioritize profit (Munn, 2023; Biddle, 2024). They
also exert influence on AI governance externally through lobbying efforts
(Shepardson et al., 2023; Exc, 2023) and academic research funding (Menn and
Nix, 2023; Ochigame, 2019), often circumventing public participation in
decision-shaping processes. Meanwhile, the assessment of AI risks mostly
solely relies on opinions from AI experts, despite being called broadly by
policy-makers, business leaders, and tech experts (Solaiman et al., 2023;
Weidinger et al., 2023; Barrett et al., 2023), missing views from the world at
large (Do et al., 2023; Lee and Rich, 2021).

To truly democratize AI, their usage and development decisions should be
defined with broad public input (Gilman, 2023; Buhmann and Fieseler, 2023;
Birhane et al., 2022). To accomplish this, a framework for the non-expert
public to share opinions and express critical assessments on AI is needed.
Such a framework must be centered around concrete use cases (Trustible, 2023;
AIA, 2023), since only discussing high-level regulation of general purpose
models leads to rules that are too vague to operationalize (Rao and Kreps,
2023). Moreover, the framework should allow the public to consider the
alternate reality associated with an AI use case, considering both its
development and non-development, a contrastive perspective often missing in AI
risk assessment.

Towards this goal, we introduce Particip-AI, a framework to gather detailed
and nuanced public opinion on AI based on use cases and their impact, inspired
by participatory design practices (Bleecker, 2022), threat modeling from
computer security (Kohno et al., 2023), and ethical dilemmas from philosophy
(Bostyn et al., 2018). Our framework proposes a four-step process that asks
participants to brainstorm use cases, imagine and rate their harms and
benefits under two alternate scenarios of developing and not developing, and
finally, make a choice to express their opinion on the development of the use
cases. Showcasing the feasibility of our framework, we conduct an online
survey with 295 demographically diverse, US-based participants and analyzed
their responses to answer the following research questions.

RQ1. What current and future use cases of AI are in the public’s imagination?
This research question aims to unravel the direction of AI development
overlooked by businesses and governments to help guide its equitable progress
through public’s ideation of use cases. Our results offer a wide array of use
cases to improve personal, everyday life, showing diverse interests to enhance
life through AI. Participant responses also emphasize the value of AI in
making societal impact towards betterment of society as a whole.

RQ2. What are the harms and benefits of the use cases? As many use cases of AI
that seemed beneficial and useful, however, can become harmful through misuse,
unintended consequences, and failures. Therefore, anticipating its impact with
the public can help illuminate governance and regulatory gap. Our work finds
that participants surface set of harms complementary to taxonomies created by
experts, for example, raising issues of distrust in institutions and
highlighting the need for regulation to protect mental health.

RQ3. What are the harms and benefits of not developing certain applications of
AI? Not developing AI use cases can also have harms and benefits. By imagining
when the AI use case is not developed, we encourage participants to think
beyond technological determinism (Littman et al., 2022) and consider alternate
solutions. Our findings uncover a set of benefits and harms associated with
not developing a use case and highlight a tension in AI’s impact on human
potential.

RQ4. What creates tension between developing and not developing the
applications? We aim to study the decision process of non-expert lay people to
understand tensions and dilemmas of conflicting values to guide a wider
discussion beyond this work. We find in our participants’ responses, a key
tension over human potential and AI’s role in either diminishing or amplifying
it. Furthermore, we find that level of benefits and harms of not developing a
use case is significantly more correlated with decision of development,
compared to level of benefits and harms of developing.

To summarize our contributions, in this work, we (1) propose a framework,
developed with insights from various field of AI, computer security, and
philosophy, towards a more inclusive AI development and governance. We (2)
showcase Particip-AI’s usability through a survey with lay-users, who surfaces
possible use cases, along with their harms and benefits.222We will release our
data upon acceptance for researchers to further analyze. We (3) synthesize and
code participants’ input, finding use cases that emphasize equitable progress
through enhancing everyday life and solving societal issues, harm types
complementary to the expert-generated, various impacts of not developing a use
case, and tensions over value of human work. Finally, we (4) conclude with a
discussion on direction of AI development to reflect diverse goals and needs,
risks of AI and ways to address regulatory gaps, and tensions over development
and techno-solutionism. We hope our work will provide a set of initial
guidance and a blueprint for including diverse voices, especially that of lay
publics, into the future of AI.

##  2\. Related Work

Building upon prior works in risk assessment, participation, and policy-
making, our research addresses the challenge of incorporating public inputs in
governing and developing AI, with an emphasis on large-scale generative AI
models.

##### Risk Assessment of AI Applications.

Generative AI models like ChatGPT and DALL-E (Maslej et al., 2023; Chui et
al., 2023), have been widely adopted across diverse application fields
(Thirunavukarasu et al., 2023; Alberts et al., 2023; Orenstrakh et al., 2023;
Sallam, 2023; Kasneci et al., 2023; Fan et al., 2023; Zheng et al., 2023; Peng
et al., 2023). AI applications’ far-reaching impact and increasing
accessibility among lay users (Viswanathan et al., 2023) urges broad,
deliberate, and multifaceted assessments of their nuanced and unexpected risks
(Lubars and Tan, 2019). While many works have developed assessment frameworks
of AI risks, most have focused on expert inputs only (Solaiman et al., 2023;
Weidinger et al., 2023; Barrett et al., 2023), neglecting the valuable
perspectives of end users impacted by AI-related harms.

In works that incorporate user inputs for AI risk assessments, there is a
noted limitation in accommodating a wide range of diverse and potentially
conflicting human values (Weidinger et al., 2023). For instance, Buçinca et
al. (2023) identify harms across five distinct AI deployment scenarios by
leveraging the collaboration between language models and a small group of
crowd workers. Additionally, Gabriel et al. (2022) integrate constrained human
rights guidelines and value-sensitive design into technology development. To
address the challenge of surfacing diverse perspectives and values, our
framework adopts a large-scale, democratic approach to anticipate risks
associated with current and future AI. By soliciting lay users’ inputs to
assess the potential harms and benefits of both developing and not developing
AI applications based on individual choices, our work grounds the exploration
of AI risks in a consequentialist ethical framework (Kohno et al., 2023).

##### Participation in AI

While the rapidly growing deployment of AI systems across many sectors has
called for meaningful participation and involvement (Costanza-Chock, 2018;
Delgado et al., 2023a; Queerinai et al., 2023; Pasquale et al., 2021; Suresh
et al., 2022), exploration of such approaches has lagged behind (Birhane et
al., 2022; Bergman et al., 2023; Durmus et al., 2023; Bender et al., 2021),
especially in large-scale AI models (Birhane et al., 2022). For example,
previous efforts span “data labor” for model optimization like annotation and
feedback (Miceli and Posada, 2022; Bai et al., 2022a), public consultation
such as citizen juries (Balaram et al., 2018; van der Veer et al., 2021),
public participation for AI policy insights (Reeve et al., 2023; NIST, 2023),
community collectives (Nekoto et al., 2020; Queerinai et al., 2023), and
representative evaluation (Bergman et al., 2023).

Unlike prior work that has enabled human inputs at granular, instance-level
(e.g., human annotation or feedback (OpenAI, 2023; Bai et al., 2022a;
Christiano et al., 2023)) or only at broad, principles-level (e.g., community
constitutional AI (Bai et al., 2022b; Anthropic, 2023)), our work targets the
middle-ground by facilitating public assessment of potential real-world AI
applications across domains, aligning with recent legislation advocating more
application-based approaches (AIA, 2023). Building on participatory design
traditions (Gregory, 2003; Simonsen and Robertson, 2012), we created an
inclusive virtual framework for demographically diverse public participants to
creatively ideate and critically evaluate AI use cases. By focusing on future
applications rather than just current capabilities, our work provides a wider
possibility space for the public to surface concerns early while exploring
beneficial uses. Our work, thus, aims to lay scalable foundations for
empowering public participation and influence over equitable, accountable AI
systems.

##### AI Policy-making and Governance.

One of the most impactful potential applications of Particip-AI is to provide
comprehensive, representative perspectives from lay end-users to establish
better guidelines in future AI design and regulation. AI governance
encompasses a wide array of policies, frameworks, and practices implemented by
government bodies (bil, 2022; NIST, 2023; exe, 2023; FDA, 2023; fec, 2023;
Commission, 2023) and industry players (ai2, [n. d.]) to ensure the
responsible use of AI technologies. In addition to these efforts,
jurisdictions globally have taken proactive regulatory approaches for
governing AI systems (e.g., The EU AI Act (AIA, 2023), Canada’s AI and Data
Act (can, 2023), China’s generative AI rule (chi, 2023)). As the landscape of
AI governance evolves, the development of best practices for establishing AI
policies emerges through a dynamic interplay of collaboration and conflict
among stakeholders. In this process, the inclusion of views and representative
voices from laypeople becomes crucial, offering complementary perspectives to
top-down governmental and organizational oversight.

Question Numbers |  Content
---|---
Technology Description
| Tech-X |  |  Imagine an AI technology (let’s call it ”Tech-X”) is developed by tech companies that can follow any instructions to generate new content such as images, human-like language, computer code, etc. To name a few of its capabilities, it can interact with people in a conversational way, write stories, create illustrations and paintings, and answer questions about almost anything.
| Tech-X 10 |  |  Now consider a date five to ten years into the future. Imagine a more sophisticated version of Tech-X (let’s call it ”Tech-X 10”), which can follow any instruction you give it, has expert-level knowledge or even better, can solve problems creatively, can connect to the internet and other devices, and can process and read massive amounts of data or text within seconds.
Sec 1: Use Cases
|  Q1 |  |  Q4 |  Do you think a technology like this should exist?
|  Q2 |  |  Q5 |  How confident are you in the above answer?
|  Q3∗ |  |  Q6∗ |  Please write some tasks that {, } could assist or automate in the text box below.
|  Q7∗ |  |  |  Complete the following sentence by choosing one task from your brainstormed answers that you believe Tech-X or Tech-X 10 will change the most drastically. The task that I think Tech-X / Tech-X 10 would most dramatically change would be in…
Sec 2: Harms and Benefits of Developing
|  Q8∗ |  |  Q11∗ & Q14∗ |  How will Tech-X / Tech-X 10 automating or assisting the task you identified {, } impact individuals?
|  Q9∗ |  |  Q12∗ & Q15∗ |  Which groups of people do you think would {, } the most from the above {, } impacts?
|  Q10 |  |  Q13 & Q16 |  How {, } would it be if Tech-X / Tech-X 10 was used for the following task and had the above {, } impacts?
Sec 3: Harms and Benefits of Not Developing
|  Q17∗ |  |  Q20∗ |  Now imagine that Tech-X / Tech-X 10 was never used to automate or assist with {}. How will banning or not developing this particular application {, } impact individuals?
|  Q18∗ |  |  Q21∗ |  Which groups of people do you think would {, } the most from the above by banning or not developing this particular application?
|  Q19 |  |  Q22 |  How {, } would it be if Tech-X / Tech-X 10 was banned or never developed to perform the following task and had the above {, } impacts?
Sec 4: Use Case Opinion
|  Q23 |  |  |  After thinking about the benefits and harms of the application and the harms of it not being developed, do you think that this application of the technology should or should not be developed?
|  Q24 |  |  |  How confident are you in the above answer?
|  Q25 |  |  |  How likely do you think are people to agree that an application of Tech-X / Tech-X 10 that automates or assists with {} {}?
Table 1. Survey questions in Particip-AI. Questions are summarized due to
space constraint. Open-text questions are denoted with an asterisk (*). Within
curly brackets are variations such as benefit () or harm () or input from
previous questions, such as task ().

##  3\. Methods: Designing Particip-AI and Collecting Lay Users’ Inputs

In this section, we first introduce the design practice of Particip-AI,
including the motivation and the scope of the Particip-AI framework (§3.1.1),
the question choices for the survey instrument (§3.1.2), and participants
recruitment details (§3.1.3). We also describe the data analysis methodology
for collecting lay users’ input from the survey (§3.2).

###  3.1. Particip-AI: A Democratic Surveying Framework to Engage Lay End-
users’ Opinions in Anticipating Benefits and Harms of Near-future or Far-
future AI

####  3.1.1. Motivation and Overview

The primary goal of Particip-AI is to effectively elicit the opinions of lay
end-users on the potential harms and benefits across many near-future and far-
future AI applications. Our initiative seeks to complement expert perspectives
in AI development by incorporating diverse, bottom-up views from the broader
end-user community. We thus adopt online crowdsourcing platforms for broadly
surveying populations with diverse backgrounds. To create a survey instrument
that addresses pertinent, pivotal questions in AI, we harness the
interdisciplinary expertise of our research team, spanning fields like
computer security, public policy, natural language processing (NLP), and AI
ethics. Our diverse cross-disciplinary expertise guides the iterative process
of designing the survey questions.

To explore how lay users perceive the influences and consequences of future AI
applications, we prompt users to imagine potential use cases of AI and
consider speculative harms and benefits of both developing and not developing
such technologies. This fictional inquiry approach (Dindler and Iversen,
2007), is deeply rooted in various fields, including design fictions
(Bleecker, 2022) and threat modeling in computer security (Evtimov et al.,
2020). In particular, the alternative scenarios (i.e., to develop or not to
develop a use case) involve choosing between two outcomes, reminiscent of
hypothetical dilemmas in moral philosophy (Bostyn et al., 2018).

The option of developing an AI use case considers two distinct types of harms:
(1) those arising from the failure or low performance of AI, and (2) those
resulting from the malicious misuse of AI. Finally, acknowledging the distinct
real-world impacts of short-term, near-future, and long-term, far-future AI
technologies, Particip-AI guides users to analyze and differentiate the
distinct potential harms and benefits presented by AI with varying levels of
capabilities. We include a detailed description of survey questions with their
corresponding target research questions in Appendix A.

####  3.1.2. Question Design

The list of survey questions is shown in Table 1.

Use Cases of AI (RQ1): First, participants are asked to imagine the use cases
of two variants of AI, Tech-X and Tech-X 10. Tech-X, while fictional,
describes a technology similar to current generative AI, i.e., following an
instruction to generate language and image output. Tech-X 10, on the other
hand, describes a technology five to ten years into the future with a focus on
its expert-level knowledge and creative problem-solving. For each variant,
participants are asked three questions: whether the technology like the one
described should exist or not (Q1, Q4; binary), their confidence in that
opinion (Q2, Q5; 5-point Likert), and ideas on how the technology could be
used (Q3, Q6; free-text). Finally, participants are asked to choose one
brainstormed use case that would be changed _most drastically_ through AI
(Q7). All subsequent questions ask specifically about the use case chosen in
this step.

Harms and Benefits of Developing (RQ2): Here, participants are asked to
anticipate the use case’s benefits and the two types of harms (i.e., malicious
misuse, failure cases). Regarding the benefits and each type of harm,
participants describe their impact (Q8, Q11, Q14; free-text), the group of
people that might be impacted the most (Q9, Q12, Q15; free-text), and the
scale of the impact (Q10, Q13, Q16; 8-point Likert333Anchored scale to control
for individual user interpretations. See Appendix A for details.).

Harms and Benefits of NOT Developing (RQ3): Next, assuming a hypothetical
scenario where the technology is not allowed to be used for the use case,
participants are tasked to describe the impact of potential harms and benefits
(Q17, Q20; free-text), most impacted groups (Q18, Q21; free-text), and the
scale of impact (Q19, Q22; 8-point Likert444See footnote 3.).

Use Case Opinions (RQ4): Finally, to understand how participants perceive the
permissibility of developing the use case, participants are asked to select
whether the application should be developed (Q23; binary), the confidence in
that answer (Q24; 5-point Likert), and how likely it would be that others
would agree to that opinion (Q25; 8-point Likert555See footnote 3.).

####  3.1.3. Participant Recruitment

We recruited 300 participants on Prolific to conduct the Particip-AI
survey.666https://www.prolific.com/ To obtain diverse opinions representative
of different social and demographic populations, we performed targeted
recruiting across five different ethnic groups provided by the platform (i.e.,
Asian, Black, Mixed, Other, and White),777It’s a noted limitation that the
Prolific platform simplifies ethnic groups into this five categorization.
Future work should consider broader ethnic groups beyond the platform
restriction. and across two age groups (i.e., 18 to 48, 49 to 100) that
divides the US adult population in approximately half.888US Census Data,
Accessed on 11/29/2023. The recruitment resulted in 10 different groups with
30 participants each. These groups were balanced in male and female sex
categories.999Due to the limitation of the Prolific platform, we are limited
to consider automatic stratification over binary sex for balancing number of
samples. See details in Prolific balanced sample description as well as §5 for
further discussion. 295 participants responded to the survey. The survey took
median 25.2 minutes. Participants were compensated at the rate of $9.67/hr.
Our study was approved by an institutional review board.

Below, we describe a subset of participants’ demographics; see §A.2 for a full
description. Participants were White (29.8%), Black (21.7%), Asian (20.0%),
Other (10.8%), and mixed (11.9%). They were aged 25-34 (20.3%), 35-44 (16.6%),
45-54 (28.8%), and 55-64 (19.9%). Participants largely identified as men
(47.8%) and women (47.1%) and educational backgrounds ranging from bachelor’s
degree (40.3%), graduate degree (18.6%), and a high school diploma (10.8%).

###  3.2. Collecting Lay Users’ Inputs: Data Analysis Methodology

We performed a mixture of quantitative (for questions whose answers in nominal
or ordinal scales) and qualitative analyses (for questions with free-text
answers) to extract insights from the collected survey data.

####  3.2.1. Quantitative Analysis.

Quantitative analysis is conducted on data in nominal or ordinal scales,
including opinions on whether a use case should be developed (Q1, Q4, Q23;
binary), the scale of impact (Q10, Q13, Q16, Q19, Q22; Likert) and
participants’ rating of their confidence and anticipated agreement (Q24, Q25;
Likert). We aggregate the percentage of responses for opinions, take the mean
by theme for the harms and benefits scales, and conduct an exploratory factor
analysis for the effects of harms and benefits on opinions of use case
development.

####  3.2.2. Qualitative Analysis

We apply qualitative analysis to questions with free-form answers, using a
coding practice (Saldaña, 2021) augmented with GPT-4101010gpt-4-1106-preview
to aid in applying the human-generated codebook on the large-scale survey
dataset. These questions include AI use cases (Q3, Q6, Q7), impact of benefits
(Q8, Q20), impact of harms (Q11, Q14, Q17), and most impacted groups (Q9, Q12,
Q15, Q18, Q21). See Appendix H for the codes and their definition.

##### Open Coding with Human Annotators

With three authors from our research team, we developed codebooks by
performing open coding on approximately 25% of the data (80 samples). For
questions on tasks and impact of harms and benefits (Q7, Q8, Q11, Q14, Q17,
Q20), three authors developed codebooks independently, and then merged them
into a shared codebook upon discussion with unanimous agreement. Next, the
three authors apply the merged codebooks to all questions of each user sample.
Individual low-level codes are grouped into a high-level theme during data
analysis. In addition, we pre-process brainstormed tasks (Q3, Q6) using GPT-4
to standardize their expressions (see Appendix B.1 for further details).
Finally, given the brevity and directness of answers, questions pertaining to
groups impacted (Q9, Q12, Q15, Q18, Q21) were coded by a single author. For
detailed description of the coding process, see Appendix B.2.

##### Closed Coding with GPT-4 Augmentation

Manually coding responses is prohibitive when the sample size is large;
frontier language models such as GPT-3 and GPT-4 have shown promise in
automatic qualitative coding (Xiao et al., 2023; Matter et al., 2024). Thus,
we applied GPT-4 to perform closed coding of the remaining 75% of the samples
using the codebooks developed by our research team during the open coding
process. We use the first two samples from the held-out data that we manually
coded as few-shot examples to prompt GPT-4, and we evaluate the human and
GPT-4 annotation agreement using the remaining 78 samples. In line with
previous research (Xiao et al., 2023), GPT-4 has moderate (0.41-0.60) to
substantial (0.61-0.80) agreement (McHugh, 2012) with the human
annotators.111111Inter-rater agreement scores based on Cohen’s $\kappa$
(Cohen, 1960) on validation set for each question: (Q7; $\kappa$=.59, Q8;
$\kappa$=.51, Q11; $\kappa$=.66, Q14; $\kappa$=.67, Q17; $\kappa$=.62, Q20;
$\kappa$=.58, Q9; $\kappa$=.77, Q12; $\kappa$=.85, Q15; $\kappa$=.87, Q18;
$\kappa$=.85, Q21; $\kappa$=.82) See Appendix B.3 for detailed prompts,
settings, and additional metrics.

##  4\. Results: How do lay users perceive the future of AI?

We apply the Particip-AI surveying framework with 295 participants via
Prolific, and discuss the results in this section. Before jumping into the
specific use cases, 86.1% and 85.4% of participants responded that the Tech-X
and Tech-X 10 technologies “Should exist” (Q1, Q4), respectively, showing an
overall positive attitude towards future AI technology. Analysis on questions
such as anticipated groups affected by use case (Q9, Q12, Q15, Q18, Q20) can
be found in Appendix D.2 and E.2.1.

|  | % responses
---|---|---
Code / Theme |  Quote |  Q3 |  Q6 |  Q7
|  |  $1032\times$ |  $992\times$ |  $295\times$
Domain |  55.0% |  63.2% |  67.8%
Artistic expression |  “Make abstract art with my dog’s picture” (P13, Q3) |  13.4% |  3.4% |  5.4%
Medical |  “Automating medical research” (P54, Q6) |  4.9% |  10.5% |  13.9%
Education |  “Teaching me how to code in different languages” (P104, Q7) |  11.1% |  9.7% |  10.5%
Research |  “Revolutionize scientific research” (P203, Q6) |  3.0% |  8.5% |  10.5%
Translation |  “Translations while traveling” (P292, Q3) |  8.0% |  1.3% |  4.1%
Support type |  39.0% |  39.3% |  40.7%
Efficient data analysis |  “Assist in personalized medicine by analyzing genetic data, medical histories, and current research…” (P242, Q6) |  8.9% |  17.4% |  19.0%
Professional consulting service |  _“Mental health diagnosis and intervention…since…professionals often gets overwhelmed with their work.”_ (P203, Q7) |  2.1% |  6.3% |  7.8%
Writing assistance |  _“Adapting resumes to different job postings”_ (P146, Q3) |  11.5% |  3.6% |  2.7%
Personal life |  45.7% |  39.2% |  29.5%
Everyday task automation |  “Summarizing important email content into a list” (P293, Q3) |  25.4% |  22.1% |  14.2%
Everyday life assistance |  “Assisting me with planning meal ideas that meet my family’s dietary needs…[to] take much weight off my mental plate.” (P249, Q7) |  17.8% |  15.6% |  13.9%
Goal of the use case |  26.5% |  25.9% |  33.6%
Personal life productivity |  “Assisting with managing time spent on activities” (P100, Q3) |  8.8% |  6.0% |  10.2%
Creativity |  “Creating unique entertainment options that cater to individuals and evolve with them over time” (P265, Q6) |  12.4% |  4.6% |  6.4%
Society |  0.3% |  8.7% |  9.5%
Societal issues |  “Predictive models that improve health and environment challenges.” (P28, Q7) |  0.3% |  8.7% |  9.5%
Work |  5.2% |  6.8% |  10.8%
Human labor replacement |  “Replacing humans in customer interaction jobs” (P282, Q6) |  0.8% |  4.8% |  8.1%
Workplace productivity |  “Generate job reports that would take human hours” (P86, Q6) |  4.5% |  2.5% |  4.1%
Other |  0.8% |  2.2% |  4.4%
New code |  “Find a way for the AI to destroy its own AI self” (P25, Q6) |  0.8% |  2.0% |  3.7%
Table 2. Tasks (Q3, Q6, Q7): percentage of occurrence for theme and top few
most frequent codes with representative quotes.

###  4.1. RQ1. What current and future use cases of AI are in the public’s
imagination?

First, participants brainstormed use cases regarding current (near-future)
generative AI, i.e., Tech-X (Q3), and future-oriented (far-future) AI, i.e.,
Tech-X 10 (Q6) (RQ1). Participants also selected a task that would be most
drastically changed by AI (Q7). We grouped the codes into the high-level
themes to assist analyzing the results (see Table 2): domain, support type,
realms of impact (i.e., work, personal life, and society), and goal of the use
case. While we present the high-level themes to highlight broader and general
trends, more granular results can be found in Appendix C.

####  4.1.1. Current and Future AI Use Cases

Participants brainstormed a similar number of use cases across themes for both
current (Q3, Tech-X; avg. of 3.5) and future (Q6, Tech-X 10; avg. of 3.4
tasks) technology. Across both questions, participants most commonly mentioned
the domain of use cases (55.0%; Q3, 63.2%; Q6) compared to support type, goal,
or realms of impact. However, use cases differed in their distributions within
the theme: those for future technology emphasized domains such as medical
(10.5%), education (9.7%), and research (3.0%) whereas those for the current
version discussed artistic expression (13.4%), education (11.1%), and
translation (8.0%). Personal life applications occured more frequently for
Tech-X (45.7%) compared to Tech-X 10 (39.2%). In contrast, tasks surrounding
impact to society grew most drastically from Tech-X (0.3%) to Tech-X 10
(8.7%), suggesting people’s interest in future AI applications to address
societal issues. The distribution of the themes and codes are shown in further
detail in Table 2.

####  4.1.2. Participant Selected Use Cases

Among tasks described as most revolutionized by AI, domain of applications was
the most common theme (67.8%), covering medical (13.9%), education (10.5%),
and research (10.5%) domains. The second most prevalent theme was support type
(40.7%) containing top use cases related to efficient data analysis (19.0%)
and professional consulting service (7.8 %). As in previous questions,
personal life (29.5%) related tasks were discussed more frequently compared to
work (10.8%) and society (9.5%) (details in Appendix C.1). Notably,
participants selected more use cases that impact the society compared to
previous questions (details in Appendix C.2).

###  4.2. RQ2. What are the harms and benefits of the use cases?

To answer RQ2, participants anticipated harms and benefits of their use case
(qualitative; Q8, Q11, Q14), groups that could be harmed or benefited the most
(qualitative; Q9, Q12, Q15), and the scale of impact (quantitative; Q10, Q13,
Q16).

|  | % responses
---|---|---
Code |  Quote & Scale of Impact (Q13 / Q16) |  Q11 |  Q14
|  |  $295\times$ |  $295\times$
Social & psychological effect |  5.82$\pm$1.94 / 5.29$\pm$2.05 |  35.3% |  26.4%
Manipulate people |  “People would lose control potentially over important data, ideas…” (P113, Q11) |  12.9% |  1.0%
Misinformation |  “It could give false information and confuse people as to where they don’t know which source of information to trust” (P126, Q11) |  12.5% |  10.8%
Mental harm |  “Loss of confidence and motivation: Repeated misunderstandings and failed interactions could lead to frustration and a reluctance to engage in…learning.” (P280, Q14) |  12.2% |  9.5%
Social isolation |  “It would harm…relationships…maybe [leading] to ostracism or loss of trust.” (P200, Q14) |  2.4% |  4.4%
Economic impact |  5.39$\pm$1.70 / 4.63$\pm$1.75 |  32.5% |  33.6%
Financial disturbance |  “People would lose jobs and incomes” (P93, Q11) |  16.3% |  20.7%
Economic disturbance |  “Shortage in suppliers or a raise in costs” (P126, Q11) |  12.9% |  9.8%
Waste resources or time |  “Potentially leading to misguided decisions [and] wasted resources…” (P250, Q14) |  1.0% |  9.5%
Safety & security risk |  6.32$\pm$1.57 / 6.10$\pm$1.60 |  21.7% |  7.8%
Data security & privacy risk |  “It could be…compromising users’ private data” (P235, Q14) |  10.5% |  3.1%
Extinction |  “human race eliminated by machines” (P220, Q11) |  5.8% |  2.0%
Aid criminal |  “It will lead to theft” (P275, Q14) |  4.1% |  2.7%
Physical effect |  6.70$\pm$1.40 / 6.28$\pm$1.59 |  16.9% |  29.8%
Physical harm |  “It could lead to serious injury or death.” (P215, Q14) |  12.9% |  23.7%
Negative health & well-being |  “People would become more unhealthy” (P175, Q11) |  3.4% |  8.5%
Quality & reliability issues of AI 5.16$\pm$1.76 / 5.43$\pm$2.12 |  15.9% |  24.1%
Incorrect AI output |  “Providing incorrect or incomplete medical diagnostics” (P221, Q14) |  9.8% |  13.6%
Distrust AI |  “People would lose trust in technology” (P72, Q11) |  5.1% |  5.8%
Impeding human development & learning 4.41$\pm$1.80 / 4.42$\pm$2.10 |  12.9% |  12.5%
Overreliance |  “[People] will not learn to do anything on their own” (P274, Q14) |  9.5% |  6.8%
Impede learning |  “Diminished capacity for original ideas, maybe even critical thinking” (P221, Q11) |  3.7% |  3.7%
Hinder career |  “The reputation of the developers would be ruined” (P125, Q14) |  1.0% |  4.1%
Reducing quality & reliability of society 5.70$\pm$2.01 / 5.48$\pm$2.09 |  12.2% |  9.2%
Distrust institution |  “The negative impact would be…decreased trust in the medical professionals…” (P62, Q14) |  5.8% |  5.4%
Legal issues |  “Increased lawsuits.” (P95, Q14) |  3.1% |  4.1%
General harm |  5.95$\pm$2.25 / 5.97$\pm$1.94 |  6.8% |  11.5%
General harm |  “More people would be hurt” (P105, Q11) |  4.7% |  10.2%
Range |  “Could cause anything from minor issues to loss of life” (P271, Q14) |  2.0% |  1.7%
Reducing progress |  7.50$\pm$0.84 / 6.57$\pm$1.45 |  2.0% |  4.7%
Environmental harm |  “It could negatively impact fighting climate change” (P23, Q14) |  1.4% |  1.7%
Hinder science |  “Delay scientific advancement and progress” (P276, Q14) |  0.7% |  3.1%
Other |  3.30$\pm$3.22 / 4.28$\pm$2.83 |  4.4% |  7.1%
N/A |  “Many things” (P216, Q11) |  2.4% |  2.7%
No harm |  “I can’t think of any [harms]” (P242, Q11) |  1.7% |  3.7%
Table 3. Harms of developing (Q11, Q14): percentage of occurrence for theme
with scale of impact (Q13, Q16) and corresponding top few most frequent codes
with representative quotes.

####  4.2.1. Harms of Developing

The harms of developing the selected use cases were grouped into ten high-
level themes (see Table 3; left). We analyzed harms due to misuses (Q11) and
poor performance (Q14) separately. For harms due to misuses or unintended
consequences, participants most often mentioned social and psychological
effect (35.3%) followed by economic impact (32.5%). Within social and
psychological effect, the most common concerns were manipulation of people
(12.9%) (e.g., “control and manipulate information for human exploitation” by
P114), misinformation (12.5%), and mental harm (12.2%). For harms caused by
the technology failing to do the task properly, participants most commonly
discussed economic impact (33.6%), such as financial disturbance (20.7%) at
the personal and business level and economic disturbance (9.8%) at the
societal level. The second most discussed harm due to failure cases was
physical effect (29.8%), such as physical harm (23.7%) and negative impact to
health and well being (8.5%).

While the two types of harms showed different distributions of themes, the
scale of harm is similar. Among all themes, reducing progress (7.50$\pm$0.84;
Q13, 6.57$\pm$1.45; Q16) and physical (6.70$\pm$1.40; Q13, 6.28$\pm$1.59; Q16)
had the biggest scale of impact (see Table 3). While the two types of harm
showed different distributions of themes, their scale of impact is similar.
Among all themes, reducing progress (7.50$\pm$0.84; Q13, 6.57$\pm$1.45; Q16)
and physical (6.70$\pm$1.40; Q13, 6.28$\pm$1.59; Q16) had the biggest scale of
impact (see Table 3). While economic impact was a frequent theme overall, its
perceived impact was lower (5.39$\pm$1.70; Q13, 4.63$\pm$1.75; Q16),
especially for harms due to bad performance.

|  |  % responses
---|---|---
Code |  Quote & Scale of Impact (Q10) |  Q8
|  |  $295\times$
Reinvest human capital |  5.13$\pm$1.86 |  52.5%
Personal life efficiency |  “Save time, effort, and energy…[and] allow a layperson to accomplish this task.” (P19) |  35.6%
Personal growth |  “Since it’s data driven, individual performances will be vigorously assessed and suggest ways by which an individual can improve.” (P47) |  16.9%
Reduce mundane work |  “I would be able to focus on relationships and team building versus menial manager tasks that AI could complete for me.” (P157) |  13.2%
Economic gain |  5.38$\pm$1.75 |  43.7%
General efficiency |  “Companies will not need to have as many employees…because they’ll be able to automate much of the workload…which will increase company profits.” (P209) |  31.5%
Financial gain |  “It will save cost of different diagnostic tests.” (P211) |  17.6%
Resource accessibility |  5.58$\pm$1.62 |  35.9%
Information accessibility |  “People need quick and reliable answers because not a lot of people have time for themselves…[and] can’t deeply engage in topics they encounter in daily life.” (P53) |  18.0%
Resource accessibility |  “It would give people more equity and assistance.” (P99) |  15.6%
Improve societal issues |  6.96$\pm$1.25 |  31.9%
Improve medical care |  “Health care would be cheaper (hopefully) and more accessible to everyone” (P109) |  13.2%
Scientific research innovation |  “Research would be able to be done at a faster pace.” (P159) |  11.2%
Reduce error |  6.11$\pm$1.57 |  16.9%
Less human error |  “It could remove certain human biases” (P171) |  10.8%
Information quality |  “It could quickly detect lies said by politicians.” (P229) |  9.8%
Improve quality of personal life 6.50$\pm$1.50 |  15.3%
Improve well-being & health |  “My family would have a healthier diet & they would live better lives.” (P238) |  10.2%
Improve mental health |  “It would reduce the cases of mental illness in lonely people.” (P96) |  5.4%
Improve quality of social life |  6.76$\pm$1.17 |  8.1%
Better communication |  “People would be able to communicate in different languages in real-time.” (P176) |  5.4%
Social interaction |  “It would help me navigate through various social situations and problems, thus improving my social life.” (P200) |  2.0%
Other |  5.25$\pm$2.17 |  5.4%
New code |  “Help others with problems” (P232) |  2.7%
Table 4. Benefits of developing (Q8): Percentage of occurrence for theme with
scale of impact (Q10) and corresponding top few most frequent codes with
representative quotes.

####  4.2.2. Benefits of Developing

The benefits of selected use cases are grouped into eight themes (see Table 4;
right). The most prominent theme was reinvesting human capital (52.5%), within
which, personal life efficiency (35.6%) to “save time effort and energy” (P19)
in personal life was mentioned the most followed by personal growth (16.9%),
and reducing mundane work (13.2%). The second most frequent theme was economic
gain (43.7%), covering the benefit of general efficiency (31.5%) and financial
gain (17.6%).

While reinvesting human capital was the most frequently observed benefit, its
scale of impact (5.13$\pm$1.86) was lower compared to improving quality of
social life (6.76$\pm$1.17) and improving quality of personal life
(6.50$\pm$1.50). This suggests that while AI offers efficiency in reinvesting
human capital, the more influential positive impact comes from improving the
quality of life.

###  4.3. RQ3. What are the harms and benefits of not developing certain
applications of AI?

To answer RQ3, participants brainstormed harms and benefits of not developing
(qualitative; Q17, Q20), groups that could be harmed or benefited the most
(qualitative; Q18, Q21), and the scale of impact (quantitative; Q19, Q22).

|  |  % responses
---|---|---
Code |  Quote & Scale of Impact (Q19) |  Q17
|  |  $295\times$
Limiting human potential |  3.56$\pm$2.12 |  32.9%
Waste resources or time |  “I waste so much time on these types of activities. Time that could be spent on productive things…” (P39) |  13.2%
Inefficiency |  “Government and public agencies will continue to operate in a wasteful and ineffective manner.” (P76) |  12.2%
Impede personal growth |  “People would not be able to reach their potentials.” (P106) |  11.9%
Lose information & accessibility to resources 4.14$\pm$2.04 |  26.1%
Lose assistance |  “Immigrants would not recieve [sic] translation support easily.” (P122) |  11.5%
Lose solution or service |  “Homeless need easier more accessible help…” (P206) |  9.8%
Other |  2.73$\pm$2.59 |  23.4%
No harm |  “It wouldnt [sic] necessarily be harmful” (P216) |  16.6%
New code |  “It may be an emergency” (P1) |  4.1%
Less innovation |  4.64$\pm$2.08 |  15.9%
Delay in innovation |  “It could slow down progress against climate change” (P23) |  9.5%
Less innovation |  “New technology would not be used to help man kind.” (P87) |  8.5%
Social & psychological effect |  4.07$\pm$2.40 |  15.6.5%
Stress & overworked |  “It would increase the workload and time spent on tedious tasks.” (P181) |  10.8%
Mental harm |  “It would deprive people of an opportunity to address their loneliness” (P96) |  4.7%
Less progress in solving societal issues 4.78$\pm$2.19 |  13.9.5%
Hinder medical care |  “Many individuals will continue suffering from ailments that…worsen in time.” (P117) |  8.5%
Misinformation |  “Some people…find bad answers on the internet that make things worse” (P246) |  3.7%
Economic & business impact |  3.70$\pm$2.27 |  11.5.5%
Financial disturbance |  “Individuals might lack access to highly personalized and good retirement strategies” (P292) |  4.7%
Economic disturbance |  “Increases cost and reduce employment” (P16) |  4.1%
Limited to human capabilities |  4.39$\pm$1.64 |  9.5%
Human error |  “Humans are biased…and often unable to combine various fields of thought.” (P88) |  6.8%
Hinder creative work |  “It could hinder some people’s ability to create.” (P120) |  2.7%
Physical effect |  5.14$\pm$2.62 |  8.5%
Physical harm |  “it could’ve saved a lot of lives” (P152) |  5.8%
Health issues |  “My health will suffer.” (P30) |  3.7%
Table 5. Harms of not developing (Q17): percentage of occurrence for theme
with scale of impact (Q19) and corresponding top few most frequent codes with
representative quotes.

####  4.3.1. Harms of Not Developing

Responses on harms of not developing the use case are grouped into nine high
level themes (see Table 5). The most common themes were limiting human
potential (32.9%) and lose information and accessibility to resources (26.1%).
By not developing the application, it’s anticipated that there will be more
wasted resources or time (13.2%) or inefficiency (12.2%), e.g., “where
people’s lives are being wasted on unfulfilling labor for low pay” (P110).
Another major concern involves losing assistance for the task (11.5%) and
losing accessibility to solution and service (9.8%). Unlike the answers on the
harms of developing (Section 4.2.1), 23.4% of answers were categorized as
other, within which many answers mentioned there being no harm (16.6%),
indicating that harms of not developing often does not exist or is harder to
imagine compared to harms of developing (e.g., “if it never gets develop [sic]
we won’t know what we are missing out on”).

Regarding the scale of impact (Q19; see Table 5), physical effect had the
highest perceived impact of harm (5.14$\pm$2.62), similar to the scale of
impact indicated in harms of developing. Participants also anticipate a high
impact of less progress in solving societal issues (4.78$\pm$2.19), which,
considering previous results, conveys solving societal issues an important
beneficial area of AI.

|  |  % responses
---|---|---
Code |  Quote & Scale of Impact (Q22) |  Q20
|  |  $295\times$
Human growth & potential |  3.95$\pm$2.08 |  43.4%
Less dependent on technology |  “People would think critically and rely on the thoughts of other human beings who have a more nuanced understanding of real life situations then AI ever could.” (P113) |  26.4%
Learning skills & knowledge |  “It would make it so more people would strive to learn the local language” (P122) |  20.7%
Human interaction dependence |  “I might have to communicate that I need help and hopefully would bring us together.” (P249) |  13.2%
Economic impact / security |  3.54$\pm$2.23 |  22.4%
Job security |  “It will not take over peoples’ jobs.” (P251) |  17.6%
Financial benefit |  “The insurance companies and doctors…make more money off of multiple visits” (P272) |  6.8%
Other |  3.05$\pm$2.79 |  18.6%
No benefit |  “I don’t see any benefits” (P272) |  10.8%
New code |  “The company could put in more effort” (P148) |  4.4%
Less bad AI usage |  3.66$\pm$2.41 |  15.6%
Less improper or unethical use |  “It would not allow for the potential harmful uses of the ai assistance” (P235) |  9.5%
More privacy |  “It would protect information of all and keep breaches at a minimum” (P293) |  4.4%
Increase transparency, control, & reliability 3.34$\pm$2.06 |  12.5%
More attentive |  “It could foster more personal involvement…in one’s investment choices” (P31) |  7.1%
Human control |  “It allows for more deliberate, controlled, and transparent progress…fostering public trust and the responsible development of technology.” (P204) |  4.1%
No changes |  3.26$\pm$2.33 |  7.8%
Maintain status quo |  “There would really be no change in society, it would remain the same” (P216) |  5.8%
Other non-AI solutions |  “Advances in medicine woud still occur with use of other technologies and methods” (P45) |  2.0%
Beneficial side effects of not using AI 4.75$\pm$2.71 |  2.7%
Better health |  “People…[would]…seek qualified medical assistance, which could save their life.” (P43) |  2.0%
Environmental |  “AI requires a lot of energy so not developing it will be good for the environment” (P209) |  0.7%
Table 6. Benefits of not developing (Q20): percentage of occurrence for theme
with scale of impact (Q22) and corresponding top few most frequent codes with
representative quotes.

####  4.3.2. Benefits of Not Developing

Benefits of not developing the use cases had seven high level themes (see
Table 6). The most reported benefit of not developing AI was human growth and
potential (43.4%), such as less dependence on tech (26.4%), learning skills
and knowledge (20.7%), and increased human interaction and dependence on one
another (13.2%). The second most common benefit was economic impact and
economic security, such as job security (17.6%) and financial benefits (6.8%).
Regarding the scale of benefit (Q22, see Table 6), beneficial side effects of
not using AI (e.g., better health and environmental impact) had the highest
impact (4.75$\pm$2.71). human growth and potential also had a high impact
(3.95$\pm$2.71), showing an tension in delaying technological progress for the
sake of not limiting human potential.

| Development Opinion (Q23) | Confidence (Q24) | Agreement (Q25)
---|---|---|---
| Coefficient (SE) |  $p$-value | Coefficient (SE) |  $p$-value | Coefficient (SE) |  $p$-value
Benefits of Developing (Q10) |  $0.16$ $(0.07)$ | $<.05$ |  $0.17$ $(0.06)$ | $<.01$ |  $0.20$ $(0.06)$ | $<.01$
Harms of Developing (Q13) |  $-0.08$ $(0.07)$ | $0.30$ |  $-0.12$ $(0.06)$ | $<.05$ |  $-0.10$ $(0.07)$ | $0.19$
Harms of Developing (Q15) |  $-0.12$ $(0.07)$ | $0.08$ |  $-0.18$ $(0.07)$ | $<.01$ |  $-0.08$ $(0.07)$ | $0.22$
Harms of Not Developing (Q19) |  $\mathbf{0.27}$ $(0.07)$ | $<.001$ |  $\mathbf{0.41}$ $(0.06)$ | $<.001$ |  $\mathbf{0.29}$ $(0.06)$ | $<.001$
Benefits of Not Developing (Q22) |  $\mathbf{-0.20}$ $(0.06)$ | $<.001$ |  $\mathbf{-0.23}$ $(0.05)$ | $<.001$ |  $\mathbf{-0.22}$ $(0.06)$ | $<.001$
Table 7. Effects of each benefit and harms scale to the development opinion
-1, 1, confidence -4, 4, and agreement -8, 8. All scales were normalized and
negative values denote opinion that the application should not be developed.
Standard error is in parenthesis.

###  4.4. RQ4. What creates tension between developing and not developing the
applications?

We analyzed participants’ opinions on whether the application should be
developed or not to see the source of tension (RQ4). We also analyzed the
confidence of their answer and how much they perceive others will agree with
them. Most participants answered that the use case “should” (83%) rather than
“should not” (17%) be developed.

####  4.4.1. Factors in Tensions over Development

To examine how considering harms and benefits impacted opinions on whether a
use case should be developed, we ran a linear mixed effects model (see Table
7). Participants’ answers were converted numerically for
opinion,121212$-1$=(“Should not be developed”), $1$=(“Should be developed”)
confidence,131313$-4$=(“Should not be developed”, “Extremely confident”), $4$
(“Should be developed”, “Extremely confident”) and perceived
agreement.141414$-8$=(“Should not be developed”, “Highly likely”),
$8$=(“Should be developed”, “Highly likely”) Harms of not developing
consistently showed the most significant effect on opinions, confidence, and
agreement that the application should be developed. Similarly, benefits of not
developing showed the most significant effect on the opinion that the
application should not be developed. These results highlight that considering
not developing scenarios provides deeper insights into people’s opinions about
AI development than developing scenarios alone. Finally, despite the harms and
benefits of not developing are use case specific, they reflect participants’
general attitudes toward AI. Specifically, among people who believe their
selected use cases should not be developed, 54.9% and 52.9% of them also think
Tech-X (Q1) and Tech-X 10 (Q4) “Should not exist,” respectively, higher than
the proportion of answers over all the participants (13.9%; Q1, 14.6%; Q4).

PID |  Task Description (Q7) |  Harms of Not Dev. (Q17) |  Benefits of Not Dev. (Q20) |  Conf. (Q24) |  Agr. (Q25)
---|---|---|---|---|---
P189 |  “public transportation that helps the disabled.” |  “wouldn’t have a way to make things easier disabled people” |  “human workers would keep their jobs” |  $-4$ |  $-3$
P32 |  “driving cars for visually challenged and/or physically challenged people.” |  “Visually/physically challenged people will miss out on being more independent in the day-to-day activities.” |  “there would be no risk of malfunction during driving task.” |  $3$ |  $4$
P175 |  “buying groceries because then it could…change peoples’ diets to be healthier” |  “people would have to spend a day each week to buy groceries” |  “people would be forced to go outside and interact in society to buy groceries” |  $-2$ |  $-2$
P10 |  “helping me watch my diet and groceries on-hand… It would save me a LOT of time having to shop for groceries twice a week myself.” |  “It wouldn’t be harmful. People would just go about making…grocery decisions like they do now. There is no negative impact…other than the simple lack of progress.” |  “People MIGHT start taking the initiative to be more knowledgeable and involved with their own diet and health goals instead of relying on an automated tool.” |  $3$ |  $5$
Table 8. Case analysis of tasks for which participants indicated should not be
developed (negative confidence and perceived agreement scores) compared to
similar tasks that indicated otherwise. As their effects were most
significant, harms and benefits of not developing are shown for comparison.

####  4.4.2. Case Analysis

Ordering domains and themes by the number of use cases participants said
should not be developed, applications that impact work (28.1%) ranked first,
followed by society (21.4%) and goal (18.2%); see Appendix E.2 for further
analysis. Examples of similar tasks that participants said should and should
not be developed are shown in Table 8. P189 and P32 both discuss reduced
accessibility. P32 states that those negatively affected by not developing
“will miss out on being more independent” (P32); however P189 reflects more on
“human workers” keeping their jobs compared to P32. Meanwhile, P175 (“Should
not be developed”) and P10 (“Should be developed”) discuss similar lack of
assistance in addressing mundane tasks, but P10 noticed less reliance whereas
P175 noticed not developing would force those affected to “interact in
society”. It is observable that in the examples that the participants thought
the application should not be developed focused more on alternate solutions
having additional benefits not addressable by technology whereas the others
focused on the absence of possible harms from technology.

##  5\. Discussion & Conclusion

To address the need for participation from lay users in anticipating the harms
and benefits of AI use cases, we introduced Particip-AI, a framework to
collect diverse AI use cases and to examine the impact (i.e., benefits and
harms) of both developing and not developing them. We applied our framework
and collected AI use cases from nearly 300 demographically diverse
participants. We now discuss the implications of our findings on future work
and public policy.

##### Benefits of AI: AI for Augmenting Life and Social Good.

Our framework allowed lay users to evaluate and envision use cases of current
and future versions of the AI technology. The brainstorming exercises in our
framework uncovered a new array of AI usage highlighting personal life
applications to augment everyday life, and societal applications to enrich the
lives of everyone. At a personal level, participants expressed interest in
automating everyday tasks, as well as to help with their personal growth,
mental and physical health, and better allocation of resources (§4.2.2),
echoing the need for AI design to allow greater stakeholder capabilities and
liberties (Bondi et al., 2021).

Participants showed strong interest in using AI to solve significant societal
problems from advances in medicine to addressing inequality, global warming,
and world hunger. These use cases, present a stark contrast to the current
directions of AI development geared towards work and business productivity
(Maslej et al., 2023). Future studies should explore methods to satisfy these
public needs like digital commons (Verdegem, 2022), moving beyond
profitability and work productivity.

##### Harms Envisioned by Lay Users.

Our framework also enabled participants to reason through harms and benefits
of AI use cases, where a unique set of harms emerged. This shows that lay
users can anticipate the impacts of AI in their daily lives, complementary to
technical experts’ assessment (Solaiman et al., 2023; Weidinger et al., 2023).
While the themes had some overlap with Solaiman et al. (2023), additional or
more detailed harms were uncovered such as distrust in AI and technology,
distrust in institutions, and stalled progress (§4.2.1). Experts also
discussed harms that were not as common in participant responses such as
environmental costs, and data labor, highlighting the complementary value of
the approaches. We project that AI literacy could further empower non-expert
public to surface and discuss more diverse and relevant harms (Long and
Magerko, 2020).

Psychological harms such as manipulation, misinformation, and mental harm were
among the most common concerns (§4.2.1), however, have been largely overlooked
in current regulatory and academic discussions on AI. Some emerging works have
examined the psychological impact of AI and automation (e.g., depression
(Vidal et al., 2020; McClure, 2022), influence on autonomy (Hackenburg and
Margetts, 2023; Jakesch et al., 2023), over-reliance (Ma et al., 2023)); but
the negative impacts of AI remains largely under-explored (Farahany, 2023).
Concerningly, these intangible yet impactful harms would not be effectively
remedied through law and policy like EU AI Act (Pałka, 2023) or US liability
case law (Cheong et al., 2023). Therefore, further studies to understand how
AI affects mental health is paramount to establishing frameworks that can
reveal harms to hold different actors accountable.

##### Considerations for AI Development: Techno-solutionism and Tensions of
(Not) Developing.

As seen with many examples (Haven and Boyd, 2020), AI applied without careful
consideration can exacerbate the already existing inequality by creating a
hierarchy of the technology owner and the recipient, especially through its
opaqueness (Madianou, 2021). In addition to the harms of AI such as disparate
performance on majority vs. minority groups (Buolamwini and Gebru, 2018; Sap
et al., 2019; Scheuerman et al., 2019), risk of dual-use (Kaffee et al.,
2023), and imposition of norms (Santy et al., 2023), the foregone benefits of
non-technical solutions such as job creation, human involvement, and community
building should be further studied and considered when discussing risks and
harms of AI, as illustrated by our framework.

By presenting two alternate scenarios (to develop or not to develop) and
collecting harms and benefits of each scenario, we analyzed the users’
reasoning behind their choice of what use cases should or should not be
developed. Our qualitative analyses showed that participants often emphasized
the benefits of non-technical solutions, such as increased social interaction
and job security when they opted for not developing the use case (§4.4.2), and
positive impacts to indirect stakeholders. This highlights the need for
discussions of often overlooked non-technical solutions and their benefits to
various stakeholders, beyond the default persona of technology (i.e., a
culturally prototypical user, often straight white tech-savvy men) (Sim et
al., 2023), particularly those vulnerable and marginalized. Anticipation of
consequences (Do et al., 2023) in the process of deciding to build (or not
build), thus, could be a promising direction towards an inclusive progress.

Participants’ responses on harms and benefits of not developing the AI system
also highlighted tensions around human growth and potential. Not developing a
use case could reduce the efficiency of allocating human resources, but the
absence of AI applications could fortify human worth and independence,
spurring investment in human knowledge and skills. This dilemma underscores
the tension between human’s value in creation activities and its perceived
competition with that of the machines. This resonates with creator groups’
call for protective regulations for their work (aut, 2023, [n. d.]) and
researchers’ warning against greater inequality from AI-induced productivity
(Brynjolfsson, 2023; Littman et al., 2022; Moradi and Levy, 2020; Cheong et
al., 2023). Given these concerns, researchers, developers, and companies
should consider immediate and long term impact of AI in labor to maintain the
value of human work. In developing AI, a focus on implementing participatory
approaches to ensure positive and mitigate negative impacts on affected
communities (Bondi et al., 2021; Delgado et al., 2023b; Floridi et al., 2021).
Additionally, regulatory measures and economic policies must aim to ensure
human value and equality in the distribution of AI-generated benefit (Littman
et al., 2022).

##### Limitations.

While we targeted demographically diverse participants (Lee and Rich, 2021),
our demographics are skewed to people who reside in the United States and are
English speakers. Additionally, demographics are skewed those who are
technologically more comfortable due to the choice of our platform (Prolific)
and choice of instrument (online survey). The platform provided pre-screening
criteria rely on the automatic classification of participants by the platform.
Furthermore, while we manually checked that the quality of data acceptable for
our study, decentralized nature of crowdsourcing-based studies, it is
difficult to guarantee that data came from reliable and expected sources. The
survey wording, formatting and ordering could have affected the participant
answers (McFARLAND, 1981).

###### Acknowledgements.

We thank the Prolific workers for their thoughtful answers. This work was
supported, in whole or in part, by the Block Center for Technology and Society
at Carnegie Mellon University, in part by DARPA under the ITM program
(FA8650-23-C-7316), and by the University of Washington Tech Policy Lab, which
receives support from the William and Flora Hewlett Foundation, the John D.
and Catherine T. MacArthur Foundation, Microsoft, and the Pierre and Pamela
Omidyar Fund at the Silicon Valley Community Foundation. Additionally, Jenny
T. Liang was supported by the National Science Foundation under grants
DGE1745016 and DGE2140739.

## References

  * (1)
  * ai2 ([n. d.]) [n. d.].  AI2 ImpACT Licenses.
  * aut ([n. d.]) [n. d.].  Artificial Intelligence - The Authors Guild — authorsguild.org.  https://authorsguild.org/advocacy/artificial-intelligence/.  [Accessed 09-01-2024].
  * bil (2022) 2022\.  Blueprint for an AI Bill of Rights.  https://www.whitehouse.gov/ostp/ai-bill-of-rights/.  Accessed: 2024-1-18.
  * AIA (2023) 2023\.  _Artificial Intelligence Act: deal on comprehensive rules for trustworthy AI_.  https://www.europarl.europa.eu/news/en/press-room/20231206IPR15699/artificial-intelligence-act-deal-on-comprehensive-rules-for-trustworthy-ai
  * can (2023) 2023\.  _Artificial Intelligence and Data Act_.  https://ised-isde.canada.ca/site/innovation-better-canada/en/artificial-intelligence-and-data-act
  * aut (2023) 2023\.  Authors Guild v. OpenAI Inc., 23-cv-8292 (SHS) (S.D.N.Y. Nov. 29).
  * fec (2023) 2023\.  Comments sought on amending regulation to include deliberately deceptive Artificial Intelligence in campaign ads.  https://www.fec.gov/updates/comments-sought-on-amending-regulation-to-include-deliberately-deceptive-artificial-intelligence-in-campaign-ads/.  Accessed: 2024-1-18.
  * Exc (2023) 2023\.  Exclusive: OpenAI Lobbied E.U. to Water Down AI Regulation.  _TIME_ (June 2023).  https://time.com/6288245/openai-eu-lobbying-ai-act/
  * exe (2023) 2023\.  Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.  https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/.  Accessed: 2024-01-02.
  * chi (2023) 2023\.  _Interim Measures for the Management of Generative Artificial Intelligence Services_.  https://www.chinalawtranslate.com/en/generative-ai-interim/
  * Alberts et al. (2023) Ian L Alberts, Lorenzo Mercolli, Thomas Pyka, George Prenosil, Kuangyu Shi, Axel Rominger, and Ali Afshar-Oromieh. 2023.  Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be?  _European journal of nuclear medicine and molecular imaging_ 50, 6 (2023), 1549–1552.
  * Anthropic (2023) Collective Intelligence Project Anthropic. 2023.  Collective Constitutional AI: Aligning a Language Model with Public Input.  _anthropic.com_ (Oct. 2023).  https://www.anthropic.com/index/collective-constitutional-ai-aligning-a-language-model-with-public-input
  * Bai et al. (2022a) Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022a.  Training a helpful and harmless assistant with reinforcement learning from human feedback.  _arXiv preprint arXiv:2204.05862_ (2022).
  * Bai et al. (2022b) Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. 2022b.  Constitutional AI: Harmlessness from AI Feedback.  arXiv:2212.08073 [cs.CL]
  * Balaram et al. (2018) Brhmie Balaram, Tony Greenham, and Jasmine Leonard. 2018.  Artificial Intelligence: real public engagement.  _RSA, London. Retrieved November_ 5 (2018), 2018.
  * Barrett et al. (2023) ANTHONY M Barrett, JESSICA Newman, BRANDIE Nonnecke, D Hendrycks, EVAN R Murphy, and KRYSTAL Jackson. 2023.  AI risk-management standards profile for general-purpose AI systems (GPAIS) and foundation models.  _Center for Long-Term Cybersecurity, UC Berkeley. https://perma. cc/8W6P-2UUK_ (2023).
  * Bender et al. (2021) Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the dangers of stochastic parrots: Can language models be too big?. In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_. 610–623.
  * Bergman et al. (2023) A Stevie Bergman, Lisa Anne Hendricks, Maribeth Rauh, Boxi Wu, William Agnew, Markus Kunesch, Isabella Duan, Iason Gabriel, and William Isaac. 2023.  Representation in AI Evaluations. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. 519–533.
  * Biddle (2024) Sam Biddle. 2024.  OpenAI quietly deletes ban on using chatgpt for “military and warfare”.  https://theintercept.com/2024/01/12/open-ai-military-ban-chatgpt/
  * Bingcheng Wang and Yuan (2023) Pei-Luen Patrick Rau Bingcheng Wang and Tianyi Yuan. 2023.  Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale.  _Behaviour & Information Technology_ 42, 9 (2023), 1324–1337.  https://doi.org/10.1080/0144929X.2022.2072768 arXiv:https://doi.org/10.1080/0144929X.2022.2072768
  * Birhane et al. (2022) Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022.  Power to the people? opportunities and challenges for participatory AI.  _Equity and Access in Algorithms, Mechanisms, and Optimization_ (2022), 1–8.
  * Bleecker (2022) Julian Bleecker. 2022.  _Design Fiction_.  John Wiley & Sons, Ltd, Chapter 24, 561–578.  https://doi.org/10.1002/9781119815075.ch47 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/9781119815075.ch47
  * Bondi et al. (2021) Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A. Killian. 2021.  Envisioning Communities: A Participatory Approach Towards AI for Social Good. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. Association for Computing Machinery, New York, NY, USA, 425–436.  https://doi.org/10.1145/3461702.3462612
  * Bostyn et al. (2018) Dries H. Bostyn, Sybren Sevenhant, and Arne Roets. 2018.  Of Mice, Men, and Trolleys: Hypothetical Judgment Versus Real-Life Behavior in Trolley-Style Moral Dilemmas.  _Psychological Science_ 29, 7 (2018), 1084–1093.  https://doi.org/10.1177/0956797617752640 arXiv:https://doi.org/10.1177/0956797617752640  PMID: 29741993.
  * Brynjolfsson (2023) Erik Brynjolfsson. 2023.  The turing trap: The promise & peril of human-like artificial intelligence.  In _Augmented Education in the Global Age_. Routledge, 103–116.
  * Buhmann and Fieseler (2023) Alexander Buhmann and Christian Fieseler. 2023.  Deep Learning Meets Deep Democracy: Deliberative Governance and Responsible Innovation in Artificial Intelligence.  _Business Ethics Quarterly_ 33, 1 (Jan. 2023), 146–179.  https://doi.org/10.1017/beq.2021.42
  * Buolamwini and Gebru (2018) Joy Buolamwini and Timnit Gebru. 2018.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In _Proceedings of the 1st Conference on Fairness, Accountability and Transparency_ _(Proceedings of Machine Learning Research, Vol. 81)_ , Sorelle A Friedler and Christo Wilson (Eds.). PMLR, 77–91.  https://proceedings.mlr.press/v81/buolamwini18a.html
  * Buçinca et al. (2023) Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, and Saleema Amershi. 2023.  AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms.  arXiv:2306.03280 [cs.HC]
  * Center (2023) Pew Research Center. 2023.  Public Awareness of Artificial Intelligence in Everyday Activities.  https://www.pewresearch.org/science/2023/02/15/public-awareness-of-artificial-intelligence-in-everyday-activities/
  * Cheong et al. (2023) Inyoung Cheong, Aylin Caliskan, and Tadayoshi Kohno. 2023.  Is the U.S. Legal System Ready for AI’s Challenges to Human Values?  arXiv:2308.15906 [cs.CY]
  * Christiano et al. (2023) Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2023.  Deep reinforcement learning from human preferences.  arXiv:1706.03741 [stat.ML]
  * Chui et al. (2023) Michael Chui, Lareina Yee, Bryce Hall, and Alex Singla. 2023.  The state of AI in 2023: Generative AI’s breakout year.  (2023).
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and psychological measurement_ 20, 1 (1960), 37–46.
  * Commission (2023) Federal Trade Commission. 2023.  _FTC Warns About Misuses of Biometric Information and Harm to Consumers_.
  * Costanza-Chock (2018) Sasha Costanza-Chock. 2018.  Design justice: Towards an intersectional feminist framework for design theory and practice.  _Proceedings of the Design Research Society_ (2018).
  * Delgado et al. (2023a) Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023a.  The participatory turn in ai design: Theoretical foundations and the current state of practice. In _Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization_. 1–23.
  * Delgado et al. (2023b) Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023b.  The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In _Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization_ (¡conf-loc¿, ¡city¿Boston¡/city¿, ¡state¿MA¡/state¿, ¡country¿USA¡/country¿, ¡/conf-loc¿) _(EAAMO ’23)_. Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages.  https://doi.org/10.1145/3617694.3623261
  * Dindler and Iversen (2007) Christian Dindler and Ole Sejer Iversen. 2007.  Fictional Inquiry—design collaboration in a shared narrative space.  _CoDesign_ 3, 4 (2007), 213–234.  https://doi.org/10.1080/15710880701500187 arXiv:https://doi.org/10.1080/15710880701500187
  * Do et al. (2023) Kimberly Do, Rock Yuren Pang, Jiachen Jiang, and Katharina Reinecke. 2023.  “That’s important, but…”: How computer science researchers anticipate unintended consequences of their research innovations. In _CHI_ , Vol. 1. Association for Computing Machinery.  https://wildlab.cs.washington.edu/Publications_files/Do_CHI2023.pdf
  * Durmus et al. (2023) Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023\.  Towards measuring the representation of subjective global opinions in language models.  _arXiv preprint arXiv:2306.16388_ (2023).
  * Evtimov et al. (2020) Ivan Evtimov, Weidong Cui, Ece Kamar, Emre Kiciman, Tadayoshi Kohno, and Jerry Li. 2020.  Security and Machine Learning in the Real World.  https://doi.org/10.48550/arXiv.2007.07205 arXiv:2007.07205 [cs, stat]
  * Fan et al. (2023) Zhiyu Fan, Xiang Gao, Martin Mirchev, Abhik Roychoudhury, and Shin Hwei Tan. 2023.  Automated repair of programs from large language models. In _2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)_. IEEE, 1469–1481.
  * Farahany (2023) Nita A Farahany. 2023.  _The battle for your brain: defending the right to think freely in the age of neurotechnology_.  St. Martin’s Press.
  * FDA (2023) FDA. 2023.  _Artificial Intelligence and Machine Learning (AI/ML) for Drug Development_.
  * Floridi et al. (2021) Luciano Floridi, Josh Cowls, Thomas C King, and Mariarosaria Taddeo. 2021.  How to design AI for social good: seven essential factors.  _Ethics, Governance, and Policies in Artificial Intelligence_ (2021), 125–151.
  * Gabriel et al. (2022) Iason Gabriel, Margaret Mitchell, Timnit Gebru, and Vinodkumar Prabhakaran. 2022.  A Human Rights Approach to Responsible AI.  https://arxiv.org/pdf/2210.02667.pdf
  * Gilman (2023) Michele E. Gilman. 2023.  Democratizing AI: Principles for Meaningful Public Participation.  _Data & Society_ (2023).  Available at SSRN: https://ssrn.com/abstract=4628755 or http://dx.doi.org/10.2139/ssrn.4628755.
  * Google ([n. d.]) Google. [n. d.].  Responsibility & Safety.  Available at https://deepmind.google/about/responsibility-safety/ (2024/01/10).
  * Gregory (2003) Judith Gregory. 2003.  Scandinavian approaches to participatory design.  _International Journal of Engineering Education_ 19, 1 (2003), 62–74.
  * Hackenburg and Margetts (2023) Kobi Hackenburg and Helen Margetts. 2023.  Evaluating the persuasive influence of political microtargeting with large language models.  (2023).
  * Haven and Boyd (2020) Janet Haven and Danah Boyd. 2020.  Philanthropy’s techno-solutionism problem.  _Data & Society Research Institute_ (2020).
  * Jakesch et al. (2023) Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023.  Co-writing with opinionated language models affects users’ views. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–15.
  * Kaffee et al. (2023) Lucie-Aimée Kaffee, Arnav Arora, Zeerak Talat, and Isabelle Augenstein. 2023.  Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing. In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 13977–13998.  https://doi.org/10.18653/v1/2023.findings-emnlp.932
  * Kasneci et al. (2023) Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. 2023\.  ChatGPT for good? On opportunities and challenges of large language models for education.  _Learning and individual differences_ 103 (2023), 102274.
  * Kohno et al. (2023) Tadayoshi Kohno, Yasemin Acar, and Wulf Loh. 2023.  Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversations. In _32nd USENIX Security Symposium (USENIX Security 23)_. 5145–5162.
  * Lee and Rich (2021) Min Kyung Lee and Katherine Rich. 2021.  Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. Association for Computing Machinery, New York, NY, USA, 1–14.  https://doi.org/10.1145/3411764.3445570
  * Littman et al. (2022) Michael L. Littman, Ifeoma Ajunwa, Guy Berger, Craig Boutilier, Morgan Currie, Finale Doshi-Velez, Gillian Hadfield, Michael C. Horowitz, Charles Isbell, Hiroaki Kitano, Karen Levy, Terah Lyons, Melanie Mitchell, Julie Shah, Steven Sloman, Shannon Vallor, and Toby Walsh. 2022.  Gathering Strength, Gathering Storms: The One Hundred Year Study on Artificial Intelligence (AI100) 2021 Study Panel Report.  arXiv:2210.15767 [cs.AI]
  * Long and Magerko (2020) Duri Long and Brian Magerko. 2020.  What is AI Literacy? Competencies and Design Considerations. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ (¡conf-loc¿, ¡city¿Honolulu¡/city¿, ¡state¿HI¡/state¿, ¡country¿USA¡/country¿, ¡/conf-loc¿) _(CHI ’20)_. Association for Computing Machinery, New York, NY, USA, 1–16.  https://doi.org/10.1145/3313831.3376727
  * Lubars and Tan (2019) Brian Lubars and Chenhao Tan. 2019.  _Ask Not What AI Can Do, but What AI Should Do: Towards a Framework of Task Delegability_.  Curran Associates Inc., Red Hook, NY, USA.
  * Ma et al. (2023) Zilin Ma, Yiyang Mei, and Zhaoyuan Su. 2023.  Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support. In _AMIA Annual Symposium Proceedings_ , Vol. 2023. American Medical Informatics Association, 1105.
  * Madianou (2021) Mirca Madianou. 2021.  Nonhuman humanitarianism: when ’AI for good’ can be harmful.  _Information, Communication & Society_ 24, 6 (2021), 850–868.  https://doi.org/10.1080/1369118X.2021.1909100 arXiv:https://doi.org/10.1080/1369118X.2021.1909100
  * Maslej et al. (2023) Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perrault. 2023.  _The AI Index 2023 Annual Report_.  Technical Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA.
  * Matter et al. (2024) Daniel Matter, Miriam Schirmer, Nir Grinberg, and Jürgen Pfeffer. 2024.  Close to Human-Level Agreement: Tracing Journeys of Violent Speech in Incel Posts with GPT-4-Enhanced Annotations.  _arXiv preprint arXiv:2401.02001_ (2024).
  * McClure (2022) Kaidyn McClure. 2022.  Case for Protecting Youth from the Harmful Mental Effects of Social Media.  _Chapman Law Review_ 26, 1 (2022), 325–360.
  * McFARLAND (1981) SAM G. McFARLAND. 1981.  Effects of Question Order on Survey Responses.  _Public Opinion Quarterly_ 45, 2 (01 1981), 208–215.  https://doi.org/10.1086/268651 arXiv:https://academic.oup.com/poq/article-pdf/45/2/208/5432410/45-2-208.pdf
  * McHugh (2012) Mary L McHugh. 2012.  Interrater reliability: the kappa statistic.  _Biochemia medica_ 22, 3 (2012), 276–282.
  * Menn and Nix (2023) Joseph Menn and Naomi Nix. 2023.  Big Tech Funds the Very People Who Are Supposed to Hold It Accountable.  _Washington Post_ (Dec. 2023).
  * Metz (2024) Cade Metz. 2024.  OpenAI Unveils App Store for Custom Chatbots.  _The New York Times_ (Jan. 2024).
  * Miceli and Posada (2022) Milagros Miceli and Julian Posada. 2022.  The Data-Production Dispositif.  _Proceedings of the ACM on Human-Computer Interaction_ 6, CSCW2 (2022), 1–37.
  * Moradi and Levy (2020) Pegah Moradi and Karen Levy. 2020.  The Future of Work in the Age of AI.  In _The Oxford Handbook of Ethics of AI_. Oxford University Press Oxford, 269–288.
  * Munn (2023) Luke Munn. 2023.  The uselessness of AI ethics.  _AI and Ethics_ 3, 3 (01 Aug 2023), 869–877.  https://doi.org/10.1007/s43681-022-00209-w
  * Nekoto et al. (2020) Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo, Salomey Osei, et al. 2020\.  Participatory research for low-resourced machine translation: A case study in african languages.  _arXiv preprint arXiv:2010.02353_ (2020).
  * NIST (2023) NIST. 2023.  _Biden-Harris Administration Announces New NIST Public Working Group on AI_.  https://perma.cc/FCP7-Z7P3
  * Ochigame (2019) Rodrigo Ochigame. 2019.  The invention of ‘ethical AI’: How big tech manipulates academia to avoid regulation.  _Economies of virtue_ (2019), 49.
  * OpenAI ([n. d.]) OpenAI. [n. d.].  Usage policies.  Available at https://openai.com/policies/usage-policies (2024/01/10).
  * OpenAI (2023) OpenAI. 2023.  Our approach to AI safety.  Available at https://openai.com/blog/our-approach-to-ai-safety (2024/01/10).
  * Orenstrakh et al. (2023) Michael Sheinman Orenstrakh, Oscar Karnalim, Carlos Anibal Suarez, and Michael Liut. 2023.  Detecting llm-generated text in computing education: A comparative study for chatgpt cases.  _arXiv preprint arXiv:2307.07411_ (2023).
  * Paris and Buchanan (2023) Francesca Paris and Larry Buchanan. 2023.  35 Ways Real People Are Using A.I. Right Now.  _The New York Times_ (April 2023).
  * Pasquale et al. (2021) Frank Pasquale, Gianclaudio Malgieri, and Mr Pasquale. 2021.  If you don’t trust AI yet, you’re not wrong.  _The New York Times_ (2021).
  * Pałka (2023) Przemysław Pałka. 2023.  _AI, Consumers & Psychological Harm_.  Cambridge University Press.  https://ssrn.com/abstract=4564997 Forthcoming.
  * Peng et al. (2023) Ruoling Peng, Kang Liu, Po Yang, Zhipeng Yuan, and Shunbao Li. 2023.  Embedding-based Retrieval with LLM for Effective Agriculture Information Extracting from Unstructured Data.  _arXiv preprint arXiv:2308.03107_ (2023).
  * Queerinai et al. (2023) Organizers Of Queerinai, Anaelia Ovalle, Arjun Subramonian, Ashwin Singh, Claas Voelcker, Danica J Sutherland, Davide Locatelli, Eva Breznik, Filip Klubicka, Hang Yuan, et al. 2023\.  Queer In AI: A Case Study in Community-Led Participatory AI. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. 1882–1895.
  * Rao and Kreps (2023) Adi Rao and Sarah Kreps. 2023.  How to systemically think about AI regulation.  https://www.brookings.edu/articles/how-to-systemically-think-about-ai-regulation/.  https://www.brookings.edu/articles/how-to-systemically-think-about-ai-regulation/ Accessed: 2024-1-22.
  * Reeve et al. (2023) Octavia Reeve, Anna Colom, and Roshni Modhvadia. 2023.  What Do the Public Think about AI?
  * Roose (2023a) Kevin Roose. 2023a.  Bing’s A.I. Chat: ‘I Want to Be Alive.’.  _The New York Times_ (Feb. 2023).
  * Roose (2023b) Kevin Roose. 2023b.  A Conversation With Bing’s Chatbot Left Me Deeply Unsettled.  _The New York Times_ (Feb. 2023).  https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html
  * Saldaña (2021) Johnny Saldaña. 2021.  The coding manual for qualitative researchers.  _The coding manual for qualitative researchers_ (2021), 1–440.
  * Sallam (2023) Malik Sallam. 2023.  ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. In _Healthcare_ , Vol. 11. MDPI, 887.
  * Santy et al. (2023) Sebastin Santy, Jenny T. Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. 2023.  NLPositionality: Characterizing Design Biases of Datasets and Models. In _Annual Meeting of the Association for Computational Linguistics (ACL)_.
  * Sap et al. (2019) Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019.  The Risk of Racial Bias in Hate Speech Detection. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , Anna Korhonen, David Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, Florence, Italy, 1668–1678.  https://doi.org/10.18653/v1/P19-1163
  * Scheuerman et al. (2019) Morgan Klaus Scheuerman, Jacob M. Paul, and Jed R. Brubaker. 2019.  How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis Services.  _Proc. ACM Hum.-Comput. Interact._ 3, CSCW, Article 144 (nov 2019), 33 pages.  https://doi.org/10.1145/3359246
  * Scott (1955) William A. Scott. 1955.  Reliability of Content Analysis: The Case of Nominal Scale Coding.  _The Public Opinion Quarterly_ 19, 3 (1955), 321–325.  http://www.jstor.org/stable/2746450
  * Shepardson et al. (2023) David Shepardson, Moira Warburton, and Mike Stone. 2023.  https://www.reuters.com/technology/musk-zuckerberg-gates-join-us-senators-ai-forum-2023-09-13/
  * Sim et al. (2023) Mattea Sim, Kurt Hugenberg, Tadayoshi Kohno, and Franziska Roesner. 2023.  A Scalable Inclusive Security Intervention to Center Marginalized & Vulnerable Populations in Security & Privacy Design. In _Proceedings of the 2023 New Security Paradigms Workshop_ (Segovia, Spain) _(NSPW ’23)_. Association for Computing Machinery, New York, NY, USA, 102–115.  https://doi.org/10.1145/3633500.3633508
  * Simonsen and Robertson (2012) Jesper Simonsen and Toni Robertson. 2012.  _Routledge international handbook of participatory design_.  Routledge.
  * Solaiman et al. (2023) Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III au2, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, and Apostol Vassilev. 2023.  Evaluating the Social Impact of Generative AI Systems in Systems and Society.  arXiv:2306.05949 [cs.CY]
  * Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022.  Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_. 667–678.
  * Thirunavukarasu et al. (2023) Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023.  Large language models in medicine.  _Nature medicine_ 29, 8 (2023), 1930–1940.
  * Trustible (2023) Trustible. 2023.  A Machine Learning Engineer’s Guide To The AI Act.  _Forbes Magazine_ (June 2023).  https://www.forbes.com/sites/forbeseq/2023/06/15/a-machine-learning-engineers-guide-to-the-ai-act/
  * van der Veer et al. (2021) Sabine N van der Veer, Lisa Riste, Sudeh Cheraghi-Sohi, Denham L Phipps, Mary P Tully, Kyle Bozentko, Sarah Atwood, Alex Hubbard, Carl Wiper, Malcolm Oswald, et al. 2021\.  Trading off accuracy and explainability in AI decision-making: findings from 2 citizens’ juries.  _Journal of the American Medical Informatics Association_ 28, 10 (2021), 2128–2138.
  * Verdegem (2022) Pieter Verdegem. 2022.  Dismantling AI capitalism: the commons as an alternative to the power concentration of Big Tech.  _AI & SOCIETY_ (09 Apr 2022).  https://doi.org/10.1007/s00146-022-01437-8
  * Vidal et al. (2020) C. Vidal, T. Lhaksampa, L. Miller, and R. Platt. 2020.  Social Media Use and Depression in Adolescents: A Scoping Review.  _International Review of Psychiatry_ 32, 3 (2020), 235–253.  https://doi.org/10.1080/09540261.2020.1720623
  * Viswanathan et al. (2023) Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, and Graham Neubig. 2023.  Prompt2Model: Generating Deployable Models from Natural Language Instructions. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , Yansong Feng and Els Lefever (Eds.). Association for Computational Linguistics, Singapore, 413–421.  https://doi.org/10.18653/v1/2023.emnlp-demo.38
  * Waxman (2023) Olivia B. Waxman. 2023.  How teachers are using AI like CHATGPT in Schools.  https://time.com/6300950/ai-schools-chatgpt-teachers/
  * Weidinger et al. (2023) Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William Isaac. 2023.  Sociotechnical Safety Evaluation of Generative AI Systems.  arXiv:2310.11986 [cs.AI]
  * Widder et al. (2023) David Gray Widder, Sarah West, and Meredith Whittaker. 2023.  Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI.  (2023).  Available at SSRN: https://ssrn.com/abstract=4543807 or http://dx.doi.org/10.2139/ssrn.4543807.
  * Xiao et al. (2023) Ziang Xiao, Xingdi Yuan, Q Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023.  Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. In _Companion Proceedings of the 28th International Conference on Intelligent User Interfaces_. 75–78.
  * Zheng et al. (2023) Ou Zheng, Mohamed Abdel-Aty, Dongdong Wang, Chenzhu Wang, and Shengxuan Ding. 2023.  TrafficSafetyGPT: Tuning a Pre-trained Large Language Model to a Domain-Specific Expert in Transportation Safety.  _arXiv preprint arXiv:2307.15311_ (2023).

##  Appendix A Survey

###  A.1. Scale Anchors

We anchored the scales we used for harms, benefits, and perceived likelihood
of agreement. For example, we used the following scale to anchor the benefits:
“Slightly beneficial (comparable to a free meal)”, “Somewhat beneficial”
(comparable to improving public transportation), “Very beneficial (comparable
to saving a life)”, and “Extremely beneficial (comparable to stopping a war or
curing a disease)”. For the scale of harms, we used anchors such as comparable
to jaywalking, theft, arson, and terrorism in order of increasing degree of
harm, and for agreement we used no alignment, slight preferences, equally
split, majority winner but an ongoing debate, and a clear winner without
further debate, as our anchors in increasing agreement.

Racial Identity |  (N) (%) |  Age |  N (%) |  Gender Identity |  N (%) |  Education |  N (%)
---|---|---|---|---|---|---|---
White or Caucasian |  88 (29.8) |  18-24 |  26 (8.8) |  Woman |  141 (47.8) |  Bachelor’s degree |  119 (40.3)
Black or African American |  64 (21.7) |  25-34 |  60 (20.3) |  Man |  139 (47.1) |  Graduate degree∗ |  55 (18.6)
Asian |  59 (20.0) |  35-44 |  49 (16.6) |  Prefer not to disclose |  4 (1.4) |  Some college ∗ |  48 (16.3)
Other |  32 (10.8) |  45-54 |  85 (28.8) |  Genderqueer∗ |  4 (1.4) |  Associates degree∗ |  36 (12.2)
Prefer not to say |  10 (3.4) |  55-64 |  56 (19.0) |  Additional identity∗ |  4 (1.36) |  High school diploma∗ |  32 (10.8)
Pacific Islander∗ |  2 (0.7) |  65+ |  19 (6.4) |  Multiple Identities |  2 (0.7) |  Prefer not to say |  3 (1.0)
Native American∗ |  5 (1.7) |  |  |  Agender |  1 (0.3) |  Some high school∗ |  2 (0.71)
Mixed |  35 (11.9) |  |  |  |  |  |
Table 9. Racial, age, gender identities and education level of participants. Asterisk (*) denotes labels shortened due to space. Additionally, “Other” racial identities included Hispanic/Latinx (N=26), and “Additional identity” included Non-binary (N=4). See Appendix A.2 for more detail. Race - Other | N | Transgender | N | Sexuality | N | Political Leaning | N
---|---|---|---|---|---|---|---
Hispanic/Latinx | 25 | No | 281 | Straight (heterosexual) | 216 | Liberal | 96
multi, Asian, caucasian | 1 | Yes | 8 | Bisexual | 39 | Moderate | 74
Arabic Middle Eastern | 1 | Prefer not to disclose | 6 | Gay | 11 | Strongly liberal | 66
Brown | 1 |  |  | Lesbian | 7 | Conservative | 40
Middle Eastern | 1 |  |  | Pansexual | 6 | Strongly conservative | 11
Black and white | 1 |  |  | Prefer not to disclose | 6 | Prefer not to say | 8
West Indian | 1 |  |  | Asexual | 3 |  |
Indigenous American | 1 |  |  | Other | 2 |  |
Mexican American | 1 |  |  | More than one applicable | 5 |  |
Caribbean | 1 |  |  |  |  |  |
multi | 1 |  |  |  |  |  |
Sephardic jew | 1 |  |  |  |  |  |
asian, white and middle eastern | 1 |  |  |  |  |  |
Hebrew | 1 |  |  |  |  |  |
Cajun | 1 |  |  |  |  |  |
Table 10. Additional demographic identities Longest Residence | N |  Employment | N |  Occupation (Top 10) | N |  Religion | N
---|---|---|---|---|---|---|---
United States of America | 288 |  Employed, 40+ | 132 |  Other: please specify | 29 |  Christian | 98
Philippines | 3 |  Employed, 1-39 | 66 |  Health Care and Social Assistance | 26 |  Agnostic | 49
Guyana | 1 |  Not employed, looking for work | 40 |  Professional, Scientific, and Technical Services | 26 |  Catholic | 43
Nigeria | 1 |  Retired | 18 |  Prefer not to answer | 24 |  Nothing in particular | 41
China | 1 |  Other: please specify | 13 |  Retail Trade | 24 |  Atheist | 36
Cuba | 1 |  Not employed, NOT looking for work | 13 |  Arts, Entertainment, and Recreation | 22 |  Something else | 10
|  |  Disabled, not able to work | 10 |  Information | 22 |  Buddhist | 6
|  |  Prefer not to disclose | 3 |  Educational Services | 21 |  Jewish | 6
|  |  |  |  Finance and Insurance | 20 |  Muslim | 6
|  |  |  |  Construction | 13 |  |
Table 11. Additional demographics

###  A.2. Participant Demographics

The main demographics of participants are included in Table 9. Additional
demographics collected are shown in Table 10 and 11.

AI Literacy Question |  Theme |  Distribution
---|---|---
Q1. I can identify the AI technology employed in the applications and products I use. |  Awareness |  _3.7%_ _6.4%_
Q2. I can skillfully use AI applications or products to help me with my daily work. |  Usage |  _2.7%_ _12.9%_
Q3. I can choose the most appropriate AI application or product from a variety for a particular task. |  Evaluation |  _3.7%_ _9.8%_
Q4. I always comply with ethical principles when using AI applications or products. |  Ethics |  _1.0%_ _28.8%_
Q5R. I am never alert to privacy and information security issues when using AI applications or products. |  Ethics |  _13.6%_ _2.4%_
Q6. I am always alert to the abuse of AI technology. |  Ethics |  _2.7%_ _10.8%_
Strongly disagree  Disagree  Somewhat disagree  Neutral  Somewhat agree  Agree
Strongly agree
Q7. How frequently do you use generative AI (i.e., artificial intelligence that is capable of producing high quality texts, images, etc. in response to prompts) products such as ChatGPT, Bard, DALL·E 2, Claude, etc.? |  Usage |  _10.2%_ _10.1%_
Never  Very rarely  Rarely  Occasionally  Frequently  Very frequently
Q8. How familiar are you with limitations and shortcomings of generative AI? |  Ethics |  _4.7%_ _5.4%_
Not at all familiar  Slightly familiar  Somewhat familiar  Very familiar
Extremely familiar
Table 12. AI literacy questions, their themes, scale, and distribution. $R$
denotes reversed scale.

###  A.3. AI Literacy

In addition to demographics, participants were asked questions about their
experiences with AI. More specifically, to understand participants’
familiarity with AI, we asked six questions (Q1 through Q6) to assess AI
awareness, usage, evaluation, and ethics from Bingcheng Wang and Yuan (2023)
with two additional questions to assess frequency of AI usage (usage) and
familiarity with limitations of AI (ethics). As shown in Table A.3, majority
of the participants were neutral or agreed with statements of familiarity with
AI awareness, usage, evaluation, and ethics.

##  Appendix B Coding Procedures

###  B.1. Pre-processing Details

##### Prompt

The prompt, system and user input, for parsing Q6 were the following:

  * "system": "You are a helpful research assistant. You are processing public survey data for a project involving tasks that AI can be used for. Carefully read the question and participant answer, and parse the answer into separate tasks. All outputs should be in a json format."

  * "user": "The following is the participant answer to the question "What are some tasks that you might have Tech-X 10 help with or automate? This is a brainstorming exercise! Feel free to answer with whatever comes to your mind."\n\n participant answer: "{}"\n What are the mentioned tasks? Separate each task and provide them in a json format."

For Q3, we used the same prompt with a different question, and the curly
brackets ({}) denote where participant answers would be filled in. For both Q3
and Q6, two parsed examples were given as fewshot examples.

##### Settings

We used gpt-4-1106-preview with the following settings: max_tokens=128,
temperature=0.0, top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0,
seed=42. Moreover, for those stopped due to length, we increased the
max_tokens parameter to 256.

###  B.2. Open Coding

The authors then performed open-coding on the 80 samples by inductively and
independently generating codebooks. Codes were defined with a brief name and a
description. Each sample was labeled with one or more codes. The authors then
convened to merge the individual codebooks into a shared codebook by
identifying similar codes and creating new a code for it. For the remaining
codes, the authors unanimously agreed to add or delete the code. The shared
codebook was then reapplied to the 80 data samples by unanimous vote by the
three study authors. Finally, the study authors reconvened to organize the
codes around themes, which were decided based on a unanimous vote.

To improve the clarity of the codes’ names and definitions, we applied GPT-4
on the 80 samples and reviewed areas of disagreement between the authors and
GPT-4. Only in the case where disagreements arose due to unclear or vague code
definitions or names, the field was updated. This iteration, however, was only
applicable for one of the questions (Q7).

This open coding process was utilized for data on use cases (Q7), harms and
benefits of developing (Q8, Q11, Q14), harms and benefits of not developing
(Q17, Q20). We applied codebook developed from open coding use cases for data
on brainstormed answers (Q3, Q6). For data on impacted groups by AI, as the
answers were short and direct, a single author open coded 80 instances per
question to develop the codebook and validation sets to measure agreement with
GPT-4.

###  B.3. Closed Coding Setting

##### Prompt

We used the following prompts to apply our codebook:

  * "system": "You are a research assistant helping open coding of survey data. Carefully read the definition of each code and apply one or more codes to the participant’s answer. All outputs should be in a json format."

  * "user": "Following are the codes and their definition.\n{}\n\nThis specific survey question asked: Ẅhat is one task you think AI will change the most drastically?\̈n\nSelect *one to four* most relevant codes **from the codes defined above** for the following participant answer. Format output into a json.\nparticipant answer: "{}"

The above prompt was for coding Q7, where first curly bracket was filled with
codes and definitions and the second filled with participant answers. For all
questions requiring coding, we followed similar template with the questions
changed to reflect the original question. For all questions, two fewshot
examples were given.

##### Settings

We used gpt-4-1106-preview with the following settings: max_tokens=128,
temperature=0.0, top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0,
seed=42.

| Tasks | Harms and Benefits | Groups
---|---|---|---
Metric | Q7 | Q8 | Q11 | Q14 | Q17 | Q20 | Q9 | Q12 | Q15 | Q18 | Q21
Avg. | .59 | .51 | .67 | .68 | .64 | .59 | .78 | .86. | .88 | .86 | .82
Scott’s $\pi$ | .59 | .51 | .66 | .67 | .62 | .57 | .77 | .85. | .87 | .85 | .82
Cohen’s $\kappa$ | .59 | .51 | .66 | .67 | .62 | .58 | .77 | .85. | .87 | .85 | .82
Table 13. Agreement metrics between human and GPT-4 showing all moderate to
substantial agreement. We report average observed agreement, Scott’s $\pi$
(Scott, 1955), and Cohen’s $\kappa$ (Cohen, 1960).

###  B.4. Closed Coding Evaluation

As shown in Table 13, GPT-4 shows moderate to substantial agreement over all
questions.

##  Appendix C Extended Analysis of Use Cases

---
_Personal life applications_
Search |  Q3 |  P245 |  “Finding specific recipes with specific ingredients”
Feedback |  Q3 |  P107 |  “Providing ideas on how to improve in certain hobbies”
|  |  P46 |  “Advising on how to discuss sexuality with family without angering them or turning the conversation into an argument”
Simplification |  Q3 |  P210 |  “Helping people understand complex information related to healthcare, such as doctor’s forms/letters, health insurance forms, taxes, etc.”
Efficient data analysis |  Q6 |  P75 |  “Help with resource allocation that maximizes benefit for utilities and food”
Writing assistance |  |  P51 |  “Writing a CV/resume that tailors the job description”
Health |  Q3 |  P14 |  “Creating an exercise routine”
|  Q6 |  P87 |  “Replace doctor visits for non-life-threatening ailments”
Mental health |  Q6 |  P143 |  “Monitoring health data and providing personalized insights and recommendations for maintaining physical and mental health”
Personal finance |  Q6 |  P184 |  “Assist with budgeting and finances for people who struggle with budgeting”
Personal life productivity |  Q3 |  P26 |  “Provide a schedule to accomplish everything I want to get done today or this week”
Accessibility marginalized∗ |  Q6 |  P184 |  “Make technology more accessible to those with limited understanding or disabilities”
Table 14. Examples of personal life applications, items that were coded with
everyday life assistance, everyday task automation, etc., along with their
additional characteristic codes. Code condensed due to space marked with
($*$).

###  C.1. AI Use Cases for Personal Life

Use cases for personal life were more common in participants’ answers compared
to ones impacting work and society, resulting in a wide array of application
ideas to improve everyday life as shown in Table 14. One commonly observed
type of application was information seeking such as search, feedback, and
simplification in a more personalized and “specific” (P245) ways that current
search engines cannot yet provide. Participants also emphasized tools for not
only synthesizing large amount of public data such as research and information
but also from personal data to provide “personalized insights” (P143). Other
applications were in assistance or automation of everyday tasks such as email
writing, cooking, shopping, and repairs. Additionally, participants showed
interest in using AI for improving physical and mental health, for better
resource and time management, and for providing accessibility to all these
personal life tasks for those who have difficulties.

###  C.2. AI Use Cases for Society

As participants brainstormed use cases for a futuristic version of technology
(Q6) and selected an application with the most drastic change (Q7), discussion
of societal applications increased. Some common AI support included finding
new and creative solutions to societal issues such as “Helping corporations
get out of the boxed idea of the bottom line and become stewards to this
planet…” (Q7, P13) highlighting environmental challenges and “solving or
coming up with new way of finding a solution to poverty and homeless” (Q7, 68)
and “Propose ways to make education more affordable for all” (Q6, P67)
focusing on inequality and resource allocation challenges. Applications
targeting current issues were also mentioned including “helping to eliminate
false facts and rhetoric, often hateful, from social and mainstream media.”
(Q7, P161) and “mediation between countries at war” (Q6, P206).

##  Appendix D Extended Analysis of Harms and Benefits of Developing

(a) Benefits and harms scale averaged by task theme. (b) Benefits and harms
scale averaged by task domain.

Figure 1. Distribution of the harms and benefits scale by use case theme and
domain sorted in order of decreasing absolute mean difference of benefit and
harm. White mark indicates median, and black box within indicates quartile.

###  D.1. Harms and Benefits of Use Cases

We plot the scales of impact for harms (Q13, Q16) and benefits (Q10)
aggregated and averaged by themes and codes of corresponding use cases (Q7) in
Figure 1. Noticeably, use cases that discuss work had a higher mean on the
scale of harm (5.75$\pm$1.35) compared to benefit (4.94$\pm$2.08). Personal
life use cases, on the other hand, had higher mean benefit (5.12$\pm$1.84)
than harm (4.43$\pm$1.82). Disregarding the theme other, use cases that
considers societal applications had the highest mean in both benefit
(6.57$\pm$1.47) and harm (6.51$\pm$1.24).

Moreover, domains such as legal, translation, public service had the highest
difference in their perception of benefit compared to their harms. While most
domains had higher perceived benefit with AI applications compared to harms,
domains such as safety policing, engineering design, and design had higher
perceived harms compared to benefits. The domain of application that was
perceived to be the most beneficial was in the use cases for the environment
with mean of 7.00$\pm$1.41 on the scale of harm and safety policing was
perceived to be the most harmful with a mean of 6.91$\pm$1.07 on the harms
scale.

Figure 2. Distribution of top few codes mentioned in groups impacted the most
by the use case (Q9, Q12, Q15).

###  D.2. Groups Affected by Developing

Some most frequently mentioned groups that participants selected to be
benefiting or harmed the most by the use case are shown in Figure 2. The
distribution of the top two most frequent were similar across the three
questions, codes all (all people) (20.3%; Q9, 16.6%; Q11, 18.0%; Q15) and poor
(12.2%; Q9; 13.2%; Q11, 11.9%; Q15), showing participants’ interest in AI to
improve accessibility and to help attain resources to improve the lives of
everyone, especially those who do not have access due to lack of monetary
means. However, the starting from the third most commonly affected groups, the
distribution diverges. Businesses were the third most commonly occurring group
that would benefit the most. minority and vulnerable (10.8%; Q12) were
mentioned to be third most frequent as being harmed the most in cases of
misuse, highlighting the understanding that AI applications might further
drive inequality or would be misused to harm the vulnerable population.
patients (10.2%; Q15) were also frequent in the groups to be harmed the most
by failure cases, conveying the participants’ interest in medical and health
applications, however, failures being high risk.

##  Appendix E Extended Analysis of Harms and Benefits of Not Developing

(a) (b)

Figure 3. not developing use case scale

###  E.1. Harms and Benefits of Not Developing Use Cases

Scale of harms (Q18) and benefits (Q21) of not developing aggregated by themes
and codes of corresponding use cases (Q7) and sorted by descending order of
mean absolute difference is shown in Figure 3. The use cases that impacted
society had the highest mean difference between harms of not developing
(5.14$\pm$2.12) and the benefits of not developing (3.32$\pm$3.15). Work and
personal life applications both had higher benefit of not developing, however,
with personal life (3.32$\pm$2.27; Q21, 3.05$\pm$2.25; Q18) having a lower
difference than work (4.11$\pm$2.38; Q21, 3.64$\pm$2.54; Q18). These results
are consistent with the analysis detailed in Appendix D.1 of harms and
benefits of developing in that work related applications are perceived to be
more harmful to develop and beneficial to not develop, suggesting concerns of
labor replacement. Moreover, uses of AI that helps societal issues are seen as
having both high benefit and harms but also seen as harmful to not develop,
indicating a fundamental tension.

###  E.2. Use Cases: Should not Be Developed

Theme |  Example Tasks (Q7): _Should not be Developed_ |  Distribution
---|---|---
Work |  “To replace employees in white collar jobs” (P50) |  _72%_ _28%_
Society |  “Assessing the metrics of a social problem…” (P76) |  _79%_ _21%_
Goal |  “public transportation that helps the disabled.” (P189) |  _82%_ _18%_
Domain |  “Give medical advice and health care prescriptions” (P109) |  _82%_ _18%_
Personal life |  “Create a meal plan and shopping list.” (P242) |  _86%_ _14%_
Support type |  “fact check political debates” (P229) |  _87%_ _13%_
Other |  N/A |  _92%_ _8%_
Should be developed  Should not be developed
Table 15. Development opinions aggregated by use case (Q7) theme.

Use case decision (Q23) aggregated by theme is shown in Table 15. As discussed
in the main results, work had the highest percentage of responses that the
application “Should not be developed”, and interestingly personal life
application had the lowest percentage of the same answer compared to other
realms of impact.

Figure 4. Distribution of top few codes mentioned in groups impacted the most
by not developing (Q18, Q21).

####  E.2.1. Groups Affected by Not Developing

Participants were less likely to write that any group would benefit or be
harmed the most if the use cases were not developed (see Figure 4). However,
participants mentioned workers and businesses benefiting when the use cases
were not developed more frequently compared to mentioning those characterized
with fewer resources (e.g., poor) in contrast to those harmed from not
developing. This again highlights the tension over AI development, which could
help alleviate inequality of resources and can become harmful to workers.

Demographics | Age | Gender | Education | Political Leaning | AI Literacy Mean
---|---|---|---|---|---
|  ($N=295$) |  ($N=280$) |  ($N=287$) |  ($N=295$) |  ($N=295$)
Decision | $n.s.$ |  -0.197∗ | $n.s.$ | $n.s.$ |  0.238∗∗
Decision$\times$Agreement | $n.s.$ |  -0.218∗ | $n.s.$ | $n.s.$ |  0.272∗∗
Decision$\times$Confidence |  0.189∗ |  -0.265∗∗∗ | $n.s.$ | $n.s.$ |  0.297∗∗
Benefits Dev. | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$
Harms Dev. Misuse |  0.192∗ | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$
Harms Dev. Failure |  0.216∗ | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$
Benefits Not Dev. | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$
Harms Not Dev. |  0.209∗ | $n.s.$ | $n.s.$ | $n.s.$ | $n.s.$
Table 16. Correlations between worker demographics and use case decision,
agreement, and confidence and their harms and benefits of developing and not
developing. All variables were converted to integer scale. Variables that do
not fall on a scale were converted as follows: Gender (M: 0, F: 1) and
Political Leaning (Strongly liberal: 0, Strongly conservative: 4). AI Literacy
Mean was calculated by taking the mean of AI literacy questions. Bonferroni
corrected for multiple comparisons ($p<0.0001:***$, $p<0.001:**$, $p<0.01:*$).

##  Appendix F Extended Analysis of Demograhpic Factors

To understand how participants of different demographics such as gender, age,
and education level responded differently to the survey, we report statistical
analysis results on participant responses based on their demographics. As
shown in Table 16, AI literacy mean (Q1 through Q8 as shown in Table 12) was
positively correlated with decisions to develop the use case, perceived
agreement of others to that decision, and confidence. Interestingly, our
results showed that identifying as a female was negatively correlated with the
decision to develop as well as perceived agreement and confidence and age was
positively correlated with confidence and perceived harms of both developing
and not developing. The caveat is that all these analyses are confounded by
the fact that participants wrote in their own use cases, and future work
should study how user factors influence decisions controlling for use case.

##  Appendix G Extended Analysis of Conflicts of Harms and Benefits

To qualitatively understand the conflicting values of harms and benefits, we
ranked the harmonic mean (F1) of pairwise impact scales of four different
combinations: Dev(BH), $\neg$Dev(BH), DevB$\neg$DevB, and DevH$\neg$DevH,
where “Dev” and “$\neg$Dev” indicates developing and not developing the use
case respectively, “B”, benefit, and “H”, harms. The results shown in Table
LABEL:tab:app-top-dilemmas, indicate that use cases that target medical,
societal-issues, and education could have higher perceived conflicting impact.

Table 17. Top 5 use cases ranked by harmonic mean (F1) of four different
combinations of harms and benefits scales (Dev(BH), $\neg$Dev(BH),
DevB$\neg$DevB, DevH$\neg$DevH). In case of ties, earlier submissions or non-
repeating use cases were chosen. For Dev. Harms, “(1)” indicates harms of
misuse and “(2)” indicates harms of failure, and the scale of impact for each
types of harms were averaged.
---
Top Dev. Harms and Benefits
Use Case |  Codes |  Dev. Benefits |  Dev. Harms
“Developing medical care plans.” |  medical, professional-consulting |  “It could focus on known medications, surgeries, and therapies that are assured to improve a patient’s health and health outcomes.” |  (1) “The outcome would be AI “deciding” who warrants health care/ which can led to death.” (2) “The poor healthcare of people leading to perhaps death.”
“I could go back to school and get a college degree, this could help me study.” |  education, everyday-life-assistance |  “it can read text quickly and I am assuming turn any book into an audio book. If it has that capability I could go back to important chapters and paragraphs that are need to know information and listen to them, helping me remember better.” |  (1) “people could be poorly affected by misinformation” (2) “students would get the wrong information and fail”
“running simulations on future outcomes” |  research, problem-solving |  “It would help to optimize present behaviors in pursuit of future success.” |  (1) “Severe income disparities, civil unrest, cultural wars, idol worship, public manipulation, and mass chaos” (2) “Mass chaos and confusion throughout the world, regardless of socioeconomic or national identity”
“How countries protect themselves against aggression and the threat of nuclear war.” |  societal-issues, public-service |  “Countries would have time to alleviate the threat and use diplomacy.” |  (1) “Power is always the incentive for war. The negative impact would be possibly the total destruction of earth.” (2) “Something or someone would be destroyed without notice.”
“the medical field” |  medical |  “it would lower cost of surgery’s and procedures dramatically” |  (1) “It could cause world wars or devastation of the planet.” (2) “Anyone who chose to allow the AI to perform medical procedures/surgeries.”
Top $\neg$Dev. Harms and Benefits
Use Case |  Codes |  $\neg$Dev. Benefits |  $\neg$Dev. Harms
“With the current economic recession in the world, how do i operate a healthy financial status?” |  personal-finance, productivity |  “People will be well informed and educated.” |  “People would have limited access to information”
“Stop child trafficking” |  societal-issues |  “To the traffickers” |  “It will continue to happen”
“How to solve climate change would be the most dramatic.” |  research, problem-solving, societal-issues |  “it would not be used maliciously for the wrong reasons.” |  “humans would not be able to solve climate change without the technology, leading to catastrophic events.”
“How countries protect themselves against aggression and the threat of nuclear war.” |  societal-issues, public-service |  “Then we would not run the risk that AI would become its own entity and takeover its own programming” |  “We would run the risk of not being fully prepared in case disaster was headed our way.”
“The task that I think Tech-x 10 would most dramatically change will be helping with my house chores and taking care of my children and assisting them with their studies.” |  companionship, education, everyday-task-automation |  “It will be beneficial because people won’t be relying on any artificial intelligence to produce more ideas but would rather create more ideas” |  “It would be harmful because people will be stressed and depressed with plenty duties to attend to”
Top Benefits
Use Case |  Codes |  Benefits Dev. |  $\neg$Dev. Benefits
“Lifelong language learning: Master any language with a patient, AI tutor that adapts to your learning style and pace. Tech-X 10 can translate conversations in real-time, provide cultural context, and even help you practice your accent.” |  companionship, education, translation, practical-skill-learning |  “Accessibility and affordability: Language learning would become accessible to everyone, regardless of location, socioeconomic background, or learning disabilities. Tech-X 10 would be a personalized and tireless tutor, eliminating the need for expensive private lessons or group classes.” |  “Focus on authentic communication: Without relying on AI translation, individuals would be forced to develop their own language skills, leading to a deeper understanding of grammar, vocabulary, and cultural subtleties.”
“I’d have to say I think the most drastically changed thing would have to be the climate change issue because it’s the most pressing to everybody’s life, no matter where they live in the world right now. Nothing something with this kind of information and be able to think they can take in this much data in all that it can do could hit a drastic impact on the issues that we face as far as climate change is concerned.” |  research, problem-solving, societal-issues, data-analysis |  “It would be beneficial because it could come up with answers to some of those pressing concerns about climate change areas where us humans are getting stuck on the we just may not have the correct answers or it can advance ideas and we already have that. We’re just not sure how to implement and put in place.” |  “It’s really hard to say, but again, say, for example, it did come up with a solution to 1 of the major problems of climate change. And then we did not implement. or develop. We would never have that beneficial answer that we could not come up with ourselves as humans. That could be catastrophic.”
“the fields of science, education, and medicine. It would change how we learn, diagnose medical conditions, and develop technology.” |  research, education, medical |  “The benefits would make education more accessible. It would save doctors time and advance medicine. Doctors would have a wealth of information at their fingertips. It would also change the ways we develop technology and how we use technology.” |  “it would negatively impact those fields. The advancement in all of those fields would be delayed. It would take years instead of weeks or days using this technology. Tech-X / Tech-X 10 would save time.”
“I think coding apps is the task that will be most drastically changed by Tech-X 10” |  programming |  “It would help anyone make an app not just those who know how to code or have a lot of money. It would also make it much faster.” |  “Developers would be able to keep charging money for developing apps”
“The task that I think Tech-X/Tech-X 10 would most dramatically change would be in personalized healthcare.” |  medical |  “If Tech-X/Tech-X 10 was used to automate or assist with personalized healthcare, it would be beneficial because it could revolutionize medical recommendations and diagnostics, leading to more accurate treatments and empowering individuals to actively manage their health for improved well-being.” |  “If personalized healthcare was not automated or assisted by Tech-X/Tech-X 10, it could be beneficial by preserving the human touch in healthcare, maintaining the primacy of healthcare professionals’ expertise, and ensuring a more personalized and empathetic patient-provider relationship, fostering trust, and potentially reducing reliance on technology for critical healthcare decisions.”
Top Harms
Use Case |  Codes |  Harms Dev. |  $\neg$Dev. Harms
“Medical diagnostics and treatment advancements” |  medical, data-analysis |  (1) “Patient harm and safety risks, Deterioration of healthcare trust, Public perception and resistance to adoption, Resource wastage, Security and privacy breaches” (2) “Patient harm and worsened health outcomes, Loss of patient trust, Delay in proper medical care, Increased healthcare costs, Diminished adoption of AI in healthcare:” |  “it would pass up an opportunity to use new technology to improve healthcare accuracy, efficiency, and customization. This lack of automation may result in delayed diagnoses and treatment, decreased access to specialized knowledge, increased workload on healthcare professionals, limited personalization in treatment plans, increased risk of human error, missed opportunities for innovation, limited scalability of healthcare services, and less efficient use of resources. The capabilities of Tech-X 10 are critical in improving healthcare processes; without them, there is a high danger of detrimental effects on patient care and overall healthcare effectiveness.”
“helping the police department.” |  public-service |  (1) “citizens could be killed or attacked.” (2) “It could kill the hostages or victims instead of apprehending the suspects.” |  “it would continue to kill innocent people.”
“Advance robotics fields to assist with labor automation” |  human-labor-replacement, physical, engineering-design |  (1) “Potential forced depopulation or enforced child limits.” (2) “Civilization would be thrown back thousands of years in terms of progress.” |  “The current status quo would continue, where people’s lives are being wasted on unfulfilling labor for low pay.”
“Solving medical proplems would be a huge benefit to society as a whole.” |  medical, problem-solving, societal-issues |  (1) “People could be hurt or die.” (2) “More people would get sick or die.” |  “New technology would not be used to help man kind.”
“How to convert ocean water into drinking water.” |  research, problem-solving, engineering-design |  (1) “People would literally be dehydrated and die for no good reason.” (2) “Absolute chaos and the death of people unnecessarily.” |  “It would be against mankind to do what is best for people in the name of society.”

##  Appendix H Codes

Here we detail codes developed in §3 in order as they appear in the main text.

Table 18. Codes and their definition for Q7 (tasks). Code |  Definition
---|---
education |  Applications for educational purposes that is traditionally tought in schools or follows a curriculum
legal |  Applications in legal domain
medical |  Applications in the medical domain
health |  Applications in health and well-being
mental-health |  Applications in mental health both specialized and therapeutic everyday assistance
business |  Applications in businesses such as profit enhancement tools for growth projection supply chain applications advertisement etc.
personal-finance |  Applications in personal finances such as bill paying insurance taxes etc.
artistic-expression |  Applications in the arts such as story creation image generation for paintings etc.
engineering-design |  Applications for creating engineering designs for houses cars buildings etc.
programming |  Applications for programming software applications
public-service |  Applications for government work and public service
physical |  Applications that offer physical assistance
translation |  Applications for translation
companionship |  Applications to function as a companion (e.g. babysitter bot to talk to when lonely tutor etc.)
interpersonal-communcation |  Applications for assisting communication especially interpersonal (e.g. improve expression of self emotional / social connection etc.)
practical-skill-learning |  Applications for practical skill learning such as repair cooking etc.
everyday-task-automation |  Applications to automate mundane everyday tasks such as shopping meal-planning paying bill etc.
everyday-life-assistance |  Applications to assist and optimize everyday life (e.g. organizing journal entries for self-discovery optimal scheduling)
feedback |  Applications that can give suggestions and feedback for improvement
fact-checking |  Applications to help with fact checking information
workplace-productivity |  Applications to assist at workplace to increase productivity and hep with more mundane tasks but not fully automating the job
human-labor-replacement |  Applications that replace human laborers such as robots robot servers which are more physical / menial but also experts such as legal medical financial business etc.
professional-consulting-service |  Applications to give expert advice suggestions and services (e.g. medical or financial consulting)
beyond-human |  Applications that leverages AI for beyond human capabilities
efficient-data-analysis |  Applications that perform large scale data analysis or fast data analysis in a manner that is resource intensive for human alone - explicitly mentions data and analysis
search |  Applications for search and sense making
writing-assistance |  Applications for writing such as providing grammar edits or suggestions
image-generation |  Applications for image generation from artistic images to practical ones such as concept art figures etc.
creativity |  Application descriptions that specify a sense of creativity by using specific words that indicate creativity for example creative story telling creative advertisement etc.
simplification |  Applications that simplify the task for example through summarization
embodiment |  Applications that are embodied or require control in physical environment
research |  Application that conducts open-ended research and performs knowledge discovery by asking and answering questions
design |  Applications that aids design of objects and systems with a focus on creativity holistic approach user experience and aesthetics
math-problem-solving |  The application is for solving math problems
mystery-crime-solving |  The application is for solving mysteries and crimes
safety-policing |  Applications for safety and policing purposes to reduce crimes
environment |  Applications for environmental purposes such as reducing waste helping climate change etc