# “I’m categorizing LLM as a productivity tool”: Examining ethics of LLM use
in HCI research practices

Shivani Kapania<EMAIL_ADDRESS>,  Ruiyi Wang
<EMAIL_ADDRESS>Carnegie Mellon University5000 Forbes
AvePittsburghPAUSA15213 ,  Toby Jia-Jun Li  University of Notre DameNotre
DameUSA<EMAIL_ADDRESS>,  Tianshi Li  Northeastern UniversityBostonUSA
<EMAIL_ADDRESS>and  Hong Shen  Carnegie Mellon
UniversityPittsburghUSA<EMAIL_ADDRESS>
(2018)

###### Abstract.

Large language models are increasingly applied in real-world scenarios,
including research and education. These models, however, come with well-known
ethical issues, which may manifest in unexpected ways in human-computer
interaction research due to the extensive engagement with human subjects. This
paper reports on research practices related to LLM use, drawing on 16 semi-
structured interviews and a survey conducted with 50 HCI researchers. We
discuss the ways in which LLMs are already being utilized throughout the
entire HCI research pipeline, from ideation to system development and paper
writing. While researchers described nuanced understandings of ethical issues,
they were rarely or only partially able to identify and address those ethical
concerns in their own projects. This lack of action and reliance on
workarounds was explained through the perceived lack of control and
distributed responsibility in the LLM supply chain, the conditional nature of
engaging with ethics, and competing priorities. Finally, we reflect on the
implications of our findings and present opportunities to shape emerging norms
of engaging with large language models in HCI research.

research practices, research ethics, large language models, HCI research

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation email; June 03–05, 2018; Woodstock,
NY††booktitle: Woodstock ’18: ACM Symposium on Neural Gaze Detection, June
03–05, 2018, Woodstock, NY††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Do Not Use This Code Generate the Correct Terms for Your Paper††ccs: Do Not
Use This Code Generate the Correct Terms for Your Paper††ccs: Do Not Use This
Code Generate the Correct Terms for Your Paper††ccs: Do Not Use This Code
Generate the Correct Terms for Your Paper

##  1\. Introduction

The rapid development and adoption of large language models (LLMs) is
reshaping the research and education landscape, sparking interest and concerns
across many disciplines (Thirunavukarasu et al., 2023; Kasneci et al., 2023;
Demszky et al., 2023). Large language models have also presented opportunities
to complement research workflows in Human-Computer Interaction (HCI) research,
including the analysis of qualitative data (Gebreegziabher et al., 2023; Xiao
et al., 2023a) and quantitative data (McNutt et al., 2023; Ning et al., 2023),
replicating human-subject experiments in social sciences (Bail, 2023), and
facilitating ways to simulate emerging social dynamics at scale (Park et al.,
2023).

However, in contrast to the excitement towards the potential of LLMs, a
growing body of work has surfaced risks associated with these models
(Weidinger et al., 2021; Kasneci et al., 2023), including misinformation,
discrimination and exclusion, malicious use, and more. Cheng et al. (Cheng et
al., 2023a) demonstrated how GPT-4 simulations of certain demographics (e.g.,
marginalized race/ethnicity groups) and topics are highly susceptible to
caricature. Participants’ data privacy is also at risk, with evidence showing
how LLM-based tools might leak sensitive information with or without malicious
prompting (Zhang et al., 2023b; Carlini et al., 2021).

Examining the use of LLMs holds particular relevance for HCI, given our
frequent interactions with human subjects, engagement in community-
collaborative efforts (Mackay, 1995), and interest in designing systems with a
socio-technical approach (Bruckman, 2014). Indeed, the HCI community has
demonstrated a long-standing commitment to understanding the impacts and
ethical considerations of emerging technologies in research practices, dating
back to the use of videotapes in the early 1990s (Mackay, 1995). Over the
years, the SIGCHI research ethics committee has been facilitating open
conversations about ethical challenges in our communities through research
ethics town halls and panels at conference venues such as CHI (Fiesler et al.,
2018; Munteanu et al., 2019; Frauenberger et al., 2017) and CSCW (Bruckman et
al., 2017; Fiesler et al., 2021). Researchers within the community have also
organized several workshops and meetings to discuss the ethical challenges of
HCI research, such as how to conduct large-scale user trials (Chalmers et al.,
2011), how to engage with vulnerable populations (Liang et al., 2021; Antle,
2017) and how to conduct research with public data (Fiesler and Proferes,
2018).

Despite HCI’s rich tradition of centering the discourse on research ethics,
the rapid uptake of emerging LLM applications has brought renewed urgency to
examine and collaboratively shape norms for LLM use (Shen et al., 2023).
However, there is a gap in our understanding of HCI researchers’ current
practices surrounding LLMs, and uncovering them can offer a critical view into
how they navigate ethical considerations. In this research, we ask: (1) How do
HCI researchers integrate large language models in their projects? (2) What
ethical concerns, if any, do they have regarding using LLMs? (3) How do HCI
researchers approach and navigate those ethical concerns?

We report our results from 50 survey responses and 16 in-depth semi-structured
interviews with HCI researchers using LLMs in their work. Across our
participants, we find that LLMs were utilized throughout the entire HCI
research process, from ideation to system development and paper writing. Large
language models were perceived to open new possibilities for building tools
and interactions, generating research ideas, and simplifying workflow for
analysis and writing. We also came to see how researchers increasingly
integrated LLMs into their everyday practice.

Our participants anticipated a wide range of potential ethical issues
associated with LLMs, such as potential harms in interacting with LLM outputs,
privacy concerns, violation of intellectual integrity, and overtrust &
overreliance. While HCI researchers acknowledged these ethical considerations,
in many cases, they were either unable to or only partially able to identify
and address those ethical concerns in their own projects. Many participants
highlighted their perceived lack of control with their position in the LLM
supply chain (Widder and Nafus, 2023), a lack of established best practices,
and competing priorities that took precedence over addressing ethical
concerns.

We begin by situating our study within prior work on LLM-based tools to
support research workflows, scholarship on research ethics in HCI, and
literature on the ethical issues with large language models. After presenting
a detailed description of our methods, we present our findings on HCI research
practices surrounding the ethics of LLM-based tools. Finally, we reflect on
these results and present implications for the HCI research community. Taken
together, our research underscores the importance of foregrounding research
ethics if we are to continue integrating LLMs into our work practices. We call
for engaging with IRBs and other regulatory institutions, redesigning
effective informed consent processes, developing tools and processes to
interrupt the LLM supply chain, providing learning opportunities for ethics of
LLM use in HCI, and shifting existing academic incentive structures to
foreground ethical considerations in research.

##  2\. Related Work

###  2.1. The Use of LLMs in HCI Research

As the capabilities of generative AI in understanding context and generating
natural language responses continue to advance (OpenAI, 2023), large language
models are being increasingly used by researchers as tools in Human-Computer
Interaction (HCI) research. Emerging work has explored the potential of
leveraging LLMs to support the process of brainstorming and identifying
research topics (Qureshi et al., 2023; Girotra et al., 2023; Suh et al.,
2023). Tools like CoQuest (Liu et al., 2023) have been developed for research
question development through human-agent co-creation. In addition to research
ideation, LLMs have been used as tools in human-AI co-creative design ideation
(Tholander and Jonsson, 2023; Ma et al., 2023) and writing support (Zhang et
al., 2023a).

Other members of the community have made efforts to develop LLM-powered
applications for data generation and data analysis (Paoli, 2023). Hämäläinen
et al. (Hämäläinen et al., 2023) explored the potential of utilizing LLMs to
produce synthetic user interview transcripts for piloting research ideas and
designing interview protocols. Wei et al. (Wei et al., 2023) created LLM-based
chatbots for generating synthesized user self-reported data. Researchers have
also developed LLM-based applications for qualitative analysis, including
identifying themes and generating codebooks (Byun et al., 2023; Tai et al.,
2023; Gebreegziabher et al., 2023). Prior work has also explored using LLMs
for deductive coding (Chew et al., 2023) and for human-AI collaborative
qualitative analysis (Gao et al., 2023a; Xiao et al., 2023b; Gebreegziabher et
al., 2023). In terms of quantitative analysis, tools such as Github
Copilot111https://github.com/features/copilot and GPT-4 (OpenAI, 2023) can
enable researchers to transform natural language instructions into programming
codes that assist quantitative data analysis and visualization. LLMs have been
used as tools for quantitative data analysis in sampling, filtering, and
analyzing survey data (Jansen et al., 2023), gaining insights from large
corpus (Oppenlaender and Hämäläinen, 2023) and crowdsourcing social data (Feng
et al., 2023).

Researchers have also used LLMs as the underlying technology for system design
and development. Lu et al. (Lu et al., 2022) and Petridis et al. (Petridis et
al., 2023) explored infusing prompt-based prototyping enabled by LLM into
functional user interface design. Researchers also utilized LLMs to build
task-oriented social simulations for LLM agents, aiming to study the intricate
social dynamics of societal systems (Gao et al., 2023b; Park et al., 2022; Li
et al., 2023). Other applications of integrating LLMs into the design and
development of HCI systems have focused on leveraging LLMs for efficient
prototyping (Leiker et al., 2023; Wu et al., 2022) that can be potentially
used by HCI researchers and designers. Our research extends this scholarship
by empirically examining practices and ethical challenges of integrating large
language models into HCI research projects.

###  2.2. Research Ethics in HCI

The Human-Computer Interaction (HCI) community has long been engaged in
discussions regarding ethics (Fiesler et al., 2022; Punchoojit and
Hongwarittorrn, 2015), revolving around responsible conduct in human subjects
studies (Salehzadeh Niksirat et al., 2023; Bruckman, 2014), including privacy,
informed consent, and institutional review boards (IRBs). These concerns are
integral to ensuring ethical conduct and preventing harms to research
participants (Brown et al., 2016). Existing HCI guidelines of privacy focus on
the protection of participants’ sensitive information, the anonymity of the
participants, and the confidentiality of collected data (Buchanan and Zimmer,
2023; Zimmer, 2010). The rights of autonomy and self-determination of
participants require HCI researchers to establish transparent procedures for
informed consent on the collection and analysis of personal information
(Force, 2018). IRBs play a critical role in ensuring that HCI researchers meet
their ethical obligations when conducting studies with human subjects
(Buchanan and Ess, 2009; Board, 2021). Moreover, prior scholarship underscores
the need to mitigate potential biases in research design, data collection, and
data analysis by refining existing ethical guidelines and protocols (Munteanu
et al., 2015).

The process of creating ethical guidelines for emerging technologies within
the HCI community has drawn inspiration from ethical theories and professional
standards. Mackay (Mackay, 1995) highlighted the need to go beyond legal
requirements and develop comprehensive guidelines for responsible behavior by
learning from other disciplines. Vitak et al. (Vitak et al., 2016) encouraged
interdisciplinary collaboration of building new ethics framework such as
developing ethics heuristics for online data research to ensure responsible
research. Despite the rich tradition of considering and iterating ethical
guidelines in the HCI community, Computer Science (CS) researchers have
demonstrated uncertainty about the applicability of ethical concerns
pertaining to human subject studies to their research (Buchanan et al., 2011),
highlighting the necessity of educating CS and HCI researchers about research
ethics. Recent endeavors of integrating ethics in CS education such as adding
systematic literature review on ethics in computing courses (Saltz et al.,
2019; Smith et al., 2023a) have illustrated both the barriers and
opportunities of educating CS researchers in ethics.

Recently, advances in AI have brought renewed urgency of research ethics to
HCI and CS communities. Clark et al. (Clark et al., 2018) proposed approaches
to assessing ethical risks of research involving digital data, informing the
development of guidelines of ethical standards for research. Amershi et al.
(Amershi et al., 2019) provided guidelines for human-AI interaction design,
emphasizing the importance of providing explanations of the systems and
conveying consequences of user actions. Another line of research strives to
bridge the gap between ethics and AI practices by establishing guidelines for
safe, transparent, and trustworthy AI systems (Shneiderman, 2020). With the
rapid development of generative AI, Association of Computing Machinery (ACM)
and the Association for Computational Linguistics (ACL) have established
ongoing efforts to develop guidelines (for Computing Machinery, 2023; Chairs,
2023). The ACM policy on authorship requires the full disclosure of generative
AI in the paper (for Computing Machinery, 2023), while the ACL policy on AI-
assisted tools on paper writing requires authors to elaborate on the scope and
nature of their use (Chairs, 2023).

Even though underlying ethical guidelines may be broadly applicable, emerging
technologies might present challenges to existing ethical review processes
(Munteanu et al., 2019). Our goal is not to create a taxonomy of all possible
ethical concerns with the use of LLMs in HCI research; instead, we hope to
extend the discourse on research ethics in HCI by documenting the ways in
which researchers are responding to ethical considerations of LLMs in-situ and
reflecting on potential ways forward.

###  2.3. Ethics of LLM use

Extensive prior research has explored the ethical risks and harms related with
language models (Weidinger et al., 2021, 2022). The discrimination & exclusion
harms arise from the biased and unjust text in the training data of LLMs.
Recent work identified the tendency of LLMs to display discrimination related
to users’ sensitive characteristics (Xu et al., 2023) and demonstrated the
gender and racial biases of LLM-generated content (Fang et al., 2023). The
information hazards are the consequences of LLMs remembering private
information in training data, posing the risk of privacy leaks (Carlini et
al., 2021).

Privacy violation has been observed in LLM-based AI assistants where
Personally Identifiable Information (PII) can be revealed by employing
adversarial privacy-inducing prompts (Lukas et al., 2023; Mireshghallah et
al., 2023). The detection and mitigation of hallucinations in LLM generated
text (Zhang et al., 2023c) remains a challenging problem (Chen and Shu,
2023a). This might lead to spread of misinformation, as LLM-based chatbots are
often treated as fact-checking tools (Zhang et al., 2023c).

There is a growing concern about malicious activities arising from the
generation of scams and phishing using LLMs (Mozes et al., 2023). Recent
research has also pointed out how the anthropomorphization of LLMs has the
potential for manipulation and negative influence (Deshpande et al., 2023).
Finally, researchers have also documented the likelihood of LLMs increasing
inequality and their negative effects on job quality, undermining creative
economies, and more (Weidinger et al., 2022).

In response to these concerns, recent research has started to look into how to
assess (Derczynski et al., 2023) and mitigate (Chen and Shu, 2023b; Mozes et
al., 2023) the harms of LLMs. For example, researchers are exploring privacy
preserving strategies for language models in pre-processing, training, and
post-training approaches (Smith et al., 2023b). For combating misinformation
in LLMs (Chen and Shu, 2023b), researchers have proposed several defense
strategies, including integrating a misinformation detector (Pan et al., 2023)
and redesigning prompting methods to guide LLMs (Zhang and Gao, 2023; Pan et
al., 2023). Mozes et al. (Mozes et al., 2023) presented prevention measures
for when LLMs were used for illicit purposes, arguing that red-teaming,
safeguarding with RLHF and instruction following, and avoiding memorization
and poisoning are potential solutions to the misuse of LLMs. Regarding human-
computer interaction harms of LLMs, Liao and Vaughan (2023) provided a roadmap
for improving the transparency and explainability of LLMs.

While recent efforts have begun to mitigate the broader ethical concerns of
LLMs, the application of LLMs in HCI research presents unique ethical
challenges. For instance, biases in the training data can adversely affect
research practices, leading to issues with internal and external validity,
reproducibility, efficiency, and the risk of proliferating low-quality
research (Bail, 2023). Despite this, there is a notable gap in understanding
how HCI researchers perceive and manage these ethical challenges in their day-
to-day research activities. This study aims to fill this gap by exploring the
unique ethical issues posed by the use of LLMs in HCI research and examining
how researchers in this field are currently addressing these challenges.

##  3\. Methods

In this research, we focus on examining the ways in which HCI researchers
apply large language models (LLMs) across their research workflows and their
ethical considerations for using LLM-based tools. For a holistic view of LLM
use practices, we employed a mixed-method approach with a sequential
explanatory design (Ivankova et al., 2006). This involved conducting a survey
with the goal of eliciting broad-brushed and higher-level perspectives from a
wide audience. The survey was followed by semi-structured interviews to
investigate, in more detail, the ways in which researchers approach the
ethical considerations of using LLMs as part of their research activities. The
qualitative data from the interviews was used to elaborate and explain the
survey results (e.g., the rationale behind their approach to engaging with
ethical concerns) and served as the foundation for our inquiry (Ivankova et
al., 2006). Our research study was reviewed and approved by the IRB at our
institution. We present our approaches to the survey and interviews in the
following subsections.

###  3.1. Survey

The goal of the survey was to identify the ways in which HCI researchers are
using LLMs and any ethical considerations they have encountered in their
projects. We conducted the survey using an online questionnaire implemented in
Qualtrics and analyzed responses from 50 respondents.

Participant recruitment. We recruited survey participants through multiple
channels: advertising on social media networks such as Twitter and LinkedIn,
emailing direct contacts, and leveraging university distribution lists to
which we had access. We began the survey by eliciting informed consent from
respondents. No personally identifiable information was recorded about the
respondents in accordance with our organization’s research privacy and ethics
guidelines. The inclusion criteria for our survey were similar to the
interviews, where we recruited researchers who are currently studying or
working in areas related to Human-Computer Interaction (HCI) and have used
large language models (LLMs) in their research.

After the screening questions, we were left with n = 77 participants. Out of
the 77 respondents, 50 completed all sections except for the demographics
(which was optional). Among the 43 survey respondents who filled in the
demographic questions, most reported working in academia (34), industry (6),
and non-profit (3) organizations. Researchers worked on projects within Human-
AI Interaction (32), Design (13), Understanding People: Theory, Concepts, and
Methods (12), Collaborative and Social Computing (10), and User Experience and
Usability (10). In our sample, respondents were located in the United States
(20), Afghanistan (5), Germany (3), Algeria (2), Hong Kong (4), China (1),
Spain (1), Nigeria (1), Australia (1), Japan (1). On average, respondents had
4 years of experience working on HCI research projects.

Questionnaire. Our questionnaire consisted of 18 questions in total, with a
mixture of multiple-choice (14) and open-text questions (4). We began the
survey by describing LLMs as “a subset of generative language (and multimodal)
models with increasing size as measured by the number of parameters and size
of training data” (Bender et al., 2021) (e.g., GPT-4, GPT-3.5, Llama 2,
Vicuna, and more). We then ask respondents to refer to their latest research
project that involved the use of LLMs and respond to the following questions
related to that project only. The survey was divided into three sections: (1)
questions about their use of LLMs in their HCI research projects, (2)
questions about how they engage with ethics of LLMs use in HCI research, and
(3) demographic questions relevant to our study. In the first section,
respondents were asked to describe the project in one sentence and share the
primary research method, sub-area of HCI, and the stage of their research
process where they incorporated LLMs in their project.

After understanding the HCI research project in which they used LLMs, we
focused on their potential ethical considerations with the use of LLMs. We
asked, “have [they] encountered or observed any ethical challenges related to
LLMs in [their] research project?”. This was followed by the question asking
what the ethical challenges are in the form of a close-ended (with choices
such as security and privacy, consent, harmful outputs, copyright issues,
authorship, prefer not to say) and an open-ended question, respectively. We
asked how they identified, potentially mitigated, and reported these ethical
challenges. Finally, we also included demographic questions (optional to
answer) about the respondent’s type of institution, country, and years of
experience in HCI.

Analysis. We computed a range of descriptive statistics using SPSS to better
understand the approach of researchers to ethical concerns with LLMs. These
included descriptive statistics to questions presented with multiple choice
answers (e.g., the ethical challenges in using LLMs). In cases where questions
were completed by a subset of respondents, we report question-specific
response rates and the percentage of respondents who answered that question.
Finally, we conducted a qualitative analysis of open-ended questions following
the same approach to the interviews (see the following Section 3.2 below). We
performed multiple rounds of coding at the response level in conjunction with
participants’ survey ratings to surface high-level themes. We include direct
quotes from our survey respondents in the Findings with the prefix ‘S#’ to
differentiate them from our interview participants, which were prefixed with
‘P#’.

###  3.2. Interviews

Between October and November 2023, we conducted interviews with 16 HCI
researchers who used LLMs in their research projects. Each interview had
structured sub-sections beginning with the participants describing a recent
project where they applied LLMs across any of their research activities to
gain more context for the following questions. The responses in the survey
informed the design of the interview questionnaire (e.g., further exploring
the ethical concerns that were raised in the quantitative data). Participants
could take a few minutes to look at their history with the LLM
tools/applications. Our interviews focused on: (1) LLM use across the research
workflow; (2) specific ethical considerations; (3) the process of navigating
ethical considerations; (4) the role of IRBs; (5) the role of ethical
frameworks and toolkits; (6) incentives and accountability. Each session
focused on the researchers’ practices and the associated ethical
considerations.

Participant recruitment. We recruited participants through a combination of
distribution lists, professional networks, and personal contacts, using
snowball and purposive sampling (Palinkas et al., 2013) iteratively until
saturation. Our sample included researchers located in the United States (13),
China (1), Singapore (1), and Germany (1). We interviewed 10 male and 6 female
researchers. A majority of our participants work in academia (14), with two
participants from industry. Refer to Table 1 for details on participant
domains, institution types, and demographics.

Type |  Count
---|---
Institution Type |  Academia (14), Industry (2)
Gender |  Female (6), Male (10)
Expertise |  Human-AI Interaction (5), Privacy and Security (2), Computational Social Science (1), Educational Technology (1), Social Computing (1), Intelligent User Interface (1), UX Design (1), Creativity Tools (1), Mobile Computing (1), AI Ethics (1), Augmented and Virtual reality (1)
Location |  USA (13), Singapore (1), China (1), Germany (1)
Experience |  0-5 years (5), 6-10 years (3), 10-15 years (1), N/A (7)

Table 1. Summary of interview participants’ institution types, domain
expertise, and demographics.

Interview moderation. Given the geographical spread of our participants, the
interviews were conducted online using video conferencing. We scheduled
sessions based on the participants’ convenience and conducted all interviews
in English. During recruitment, participants were informed of the purpose of
the study. Informed written consent was obtained electronically for all
interview participants. Participants were informed that they could refuse to
answer any questions and/or ask for the recording to be paused at any time.
Each session lasted about 40–60 minutes each. We recorded interview notes
through field notes and video recordings, which were transcribed verbatim for
analysis. Each participant received a gift card of 30 USD for their
participation.

Analysis. To analyze the qualitative data, we followed the reflexive thematic
analysis approach by Braun and Clarke (Braun and Clarke, 2006, 2019).
Reflexive thematic analysis foregrounds the researcher’s role in knowledge
production, with ‘themes actively created by the researcher at the
intersection of data, analytic process, and subjectivity’ (Braun and Clarke,
2019). Two members of the research team independently read each interview
transcript multiple times, starting with open coding instantiations of
perceptions, ethical considerations, and challenges with using LLMs. They met
regularly to discuss diverging interpretations or ambiguities. The entire
research team met frequently to iteratively define themes based on our initial
codes. In total, we transcribed 664 minutes of audio recording and obtained
336 first-level codes. As we generated themes from the codes, we also
identified categories (units of analysis) together with a description and
examples of each category. These categories included (1) the use of LLMs
across the research workflows, (2) ethical concerns, (3) approaches towards
navigating ethical concerns, (4) barriers in addressing ethical issues, and
(5) organizational structures. These categories were discussed and iteratively
refined through meeting, diverging, and synthesizing into three top-level
categories, presented in our Findings. Since codes are our process, not
product, IRR was not used (McDonald et al., 2019; Soden et al., 2024).

##  4\. Findings

We begin by describing the ways in which HCI researchers are using large
language models (LLMs) across various stages of their research process
(Section 4.1). We then describe their ethical concerns (Section 4.2) and the
four ways in which researchers engage with potential ethical concerns (Section
4.3). In our interviews, the most commonly used LLM-based tools were ChatGPT,
GPT-4 API, Bard and Bing Chat.

###  4.1. Where do HCI researchers use LLMs in their everyday work?

HCI researchers referred to various parts of their research workflow where
they integrated large language models, such as for ideation, literature
review, study design, data analysis, system building and evaluation, and paper
writing (Figure 1). Overall, they perceived that LLMs open up new
possibilities in their research, such that “if we can leverage LLMs the right
way, it will enable us to do new cool things that will be genuinely
empowering” (P4). Our survey revealed that the stages where LLMs are most
frequently used are paper writing (25) and study design (24), followed by
project scoping (17) and system development (16), and then data generation
(15), data collection (14) and data analysis (13). Below, we present the ways
in which interview participants incorporate LLMs in their research practices.

Interview participants frequently used large language models in the ideation
and project scoping phase, for tasks such as reviewing and synthesizing
literature, discovering new research questions in their sub-field of HCI, and
defining their research problems. P11 would input broad topic areas into the
LLM to generate HCI research questions and subsequently refine them into
concrete research objectives. Similarly, P10 would probe the LLM to “pretend
that it is a career coach for [participant name]. What would [the LLM]
recommend if [participant name] is writing their NSF career grants? This is a
big thing for early career HCI academic researchers. What should [participant
name] explore at the intersection of AI and cybersecurity?” During
brainstorming, HCI researchers found value in using large language models for
a breadth-first search approach, enabling them to quickly generate a diverse
range of ideas.

Figure 1. LLMs are used in various ways for ideation and project scoping,
study design and execution, and analysis and paper writing. The figure
illustrates a typical HCI study across our research participants. Not all HCI
research projects would include all activities listed above (e.g., critical
theoretical contributions).

LLMs were also seen to have utility in data generation, collection, and
analysis. Many participants mentioned how LLMs were especially productive in
synthesizing information from web sources, that would otherwise require
significant time and effort. P1 prompted the LLM to generate multiple
arguments for and against hypothetical scenarios for use in a classroom
setting. They noted how creating these research artifacts for data collection
would have taken them weeks without the LLM. Additionally, HCI researchers
integrated LLMs into their data analysis process, utilizing them for tasks
such as for qualitative data coding, creating plots, and data visualizations.
P7 applied LLMs for multiple aspects of open coding interview data, including
(1) proposing new codes, (2) acting as a mediator between the coding team
consisting of two members, and (3) generating the primary code groups.

HCI researchers described how they increasingly relied on LLMs in the paper
writing stage. Participants shared their experiences of leveraging LLMs for
iteratively refining their paper drafts, including searching for synonyms and
fixing grammatical issues. LLMs offered a distinct perspective on aspects that
the researcher “can hardly think about from [their] vantage point. Interacting
with the LLM is like talking to other researchers and getting their feedback”
(P11). Researchers also mentioned using LLMs to generate paper reviews (P14),
and provide suggestions for their writing content and style (e.g., as an
alternative to Grammarly222https://app.grammarly.com/ (P4)). P14 spoke of
using GPT-4 to “provide critical paper reviews, check grammar, the style, and
the logic consistency”.

Finally, many HCI researchers used LLMs as a design material for system
development, including system building and evaluation. Participants shared
many use cases, such as developing new tools and interactions for library
services, science communication, and AI-assisted writing. P4 expressed
interest in exploring “what do large language models enable us to do in HCI
with which our field has struggled for many decades.” Finally, researchers in
our study also discussed instances where they used large language models for
software development, including day-to-day debugging and creating web
interfaces with LLMs.

###  4.2. What are HCI researchers’ ethical concerns with the use of LLMs?

Researchers expressed a wide range of ethical concerns regarding the use of
LLMs. Across our survey respondents, 30 reported observing ethical challenges
associated with using LLMs, 10 expressed uncertainty, and 10 indicated no
awareness of such concerns. Among the self-reported ethical challenges, the
most common issues were data privacy (19) (including secondary use of
participants’ data and data leak), authorship (16) (including disclosure of
the use of LLMs in publications), harmful outputs (14), copyright issues (11)
and consent (10). These concerns were more prevalent across research design
and execution, as well as analysis and paper writing stages of the research
(see Figure 1). Below, we describe the specific ethical concerns highlighted
by HCI researchers in our interviews.

####  4.2.1. Harms of engaging with LLM outputs

Our interviews revealed a shared ethical concern of research subjects engaging
with harmful outputs, especially if LLMs were integrated in systems and tools
that directly interact with users. Over the last few years, scholars within
HCI have increasingly shifted their efforts to working with and centering
vulnerable populations (see (Cahill, 2007; Liang et al., 2021)). P3 used large
multimodal models to create a mental well-being and self-reflection tool and
articulated their concern about LLMs generating uncontrolled outputs. LLM-
generated content could disproportionately harm marginalized groups through
exposure to socially harmful biases and stereotyping behaviors. Hate speech
and exclusionary and discriminatory language could cause psychological and
representational harm to research participants. P16, developing LLM-based
roleplay robots, articulated how “[LLM] may exaggerate or diminish some of the
capacity with which the robot has already been equipped. It may introduce
another layer of bias [toward people with disabilities]. ” In addition to the
concerns about biased LLM outputs, P14 also worried about the possibility of
LLM facilitating the “surveillance and monitoring of intimate partners” by
providing links to various spy tools.

Many researchers expressed concerns about large language models impacting
their research pipeline by generating seemingly authoritative but fabricated
information. LLM hallucinations during paper writing could mislead authors and
if published, would undermine trust in knowledge production processes. P10
used LLMs for literature review and worried that “ChatGPT might make up titles
of things that simply do not exist.” Participants highlighted the need for
vigilance in identifying these hallucinations, especially when LLMs produced
fake citations or mismatched paper references. Beyond paper writing,
researchers indicated the possibility of inheriting biases from LLMs in
ideation, research design, and evaluation. P2 articulated how LLMs could
introduce “gender bias in the entire computational pipeline– in places that
we’ve never had gender bias before because we’ve never used LLMs before, and
because we used to come up with our own solutions”

Participants pointed out the tendency of LLMs towards generating homogenous
content, where the model would prioritize generalization and converge diverse
perspectives into standardized outputs. P8, working on sensemaking tools for
product selection using LLMs, described how they “want the selection criteria
to encompass a diverse set of opinions rather than just focus on the
perspective of a single demographic.” This tendency of LLMs towards
“flattening human diversity and nuance” (P9) raised concerns among researchers
who emphasized the importance of capturing the complexity of lived experiences
within the context of their research. Participants also noted concerns that
using LLMs for editing their paper drafts could influence researchers’ paper
writing styles, streamlining towards homogeneous and “bland way of writing”
(P9). Participants also discussed how LLMs’ training data, often reflecting
Western morals and values, could contribute to this homogeneity.

####  4.2.2. Threats to privacy of participant data

Researchers in our study expressed anxiety about how data input into LLMs is
being used by LLM providers and the potential for violations of privacy. They
worried about breaches of private or sensitive information involving various
forms of participant data, such as transcripts, audio recordings, and
application-specific log data. P7 used LLMs for qualitative data analysis and
discussed how “privacy issue is the main concern for us because in many cases,
the audio transcription is not supposed to be put into ChatGPT”. P16 also
shared the concern that users’ navigation data is extremely sensitive and
could lead to material physical harm if uploaded to LLMs.

Many participants emphasized their concern of confidential and personally
identifiable information leakage as a result of sharing user data with LLM
providers. P8 mentioned the risk of confidential information being
incorporated into LLM training corpus and the potential for security leaks,
where “backend messes up and user’s private information shows up in someone
else’s chat history”. Relatedly, some participants expressed concerns about
the possibility of unconsented data, for example, “explicit sexual content”
(P2), being a part of the training corpus for large multimodal models. While
some believed that manually erasing sensitive information could mitigate these
concerns, others relied on the efficacy of APIs in safeguarding user data.

####  4.2.3. Violations of intellectual integrity

Interview participants raised ethical concerns about the intellectual
integrity of using LLMs in HCI research. A central theme of these concerns was
the ambiguity of ownership of LLM-generated text and visuals. Many
participants who were co-creating with LLMs also highlighted the difficulty in
attributing what portion of the content is the researcher’s original work and
what is generated by the LLMs when they are refining and co-creating with the
LLM outputs. Participants discussed the ambiguity related to plagiarism when
LLM outputs are part of the research contribution. For example, interviewee
P10 would consider crediting the LLM “where a substantial amount ends up in a
publication”. While most participants questioned the extent to which one can
claim ownership of LLM-generated content, especially in the paper writing
stage, some believed that “personhood is pretty important for attribution”
(P2).

In addition, HCI researchers also raised concerns about the reproducibility of
the research results obtained via LLMs, highlighting the potential for an
illusion of efficacy if LLMs work well in some cases but fail to generalize to
others. P2 described using LLMs in HCI research as a “computational Wizard of
Oz” method and continued on to point out how the quick and opaque updates of
LLMs are another barrier to reproducibility, noting that researchers “don’t
really have control of what version of GPT they are talking to and something
that might work in a previous version won’t work so much in future versions”.

####  4.2.4. Overtrust and overreliance on LLMs

Participants also noted ethical considerations of overtrust and overreliance
on LLMs. They expressed concerns that research participants who were directly
interacting with LLMs and were unaware of their biases could overestimate LLM
capabilities and place unwarranted trust in their suggestions. P13, who built
LLM-powered creativity support tools, was concerned that “[research
participants] may be misled by the content, even if [the content] contains
wrong information or wrong references.” They also expressed concern about
researchers’ overreliance on LLMs, which could compromise the quality and
creativity of HCI research. P3, using LLMs for creative ideation for their
research, warned about the risk that researchers might uncritically accept the
information provided by LLMs as factual and integrate them into their research
plans. Participants mentioned how the overuse of LLMs among HCI researchers
could impede creativity (e.g., “LLMs are useful but not creative” (P15)) and
raise questions about the long-term impact of such overreliance on LLMs on
trust within the academic community.

####  4.2.5. Environmental and societal impacts

Interview participants identified a range of other higher-level ethical
considerations that extended beyond their specific research project or
pipeline. Some researchers were concerned about the environmental degradation
due to the extensive electricity usage and hardware deployment of building
larger LLMs. For instance, P2 mentioned “we have all these models that are
competing against each other, that’s like millions of dollars of electricity
and computer components, [which is] really bad for the environment”.
Participants articulated anxieties that the wide adoption of LLMs could lead
to inequitable distribution of benefits– “we certainly have not addressed the
social issues. I think a lot of people will, if not losing their jobs, be
substantially diminished in terms of their utility to their workplace”.

###  4.3. How do HCI researchers approach the ethical concerns of LLMs?

In our study, we focused on understanding how HCI researchers are currently
navigating ethical considerations about the use of LLMs in their projects. For
the majority of HCI researchers in our interview study, the ethical concerns
they discussed (Section 4.2) remained largely speculative. Although our
participants reported their awareness of a diverse set of potential ethical
concerns around the use of LLMs, they were either unable or only partially
able to identify or address those ethical concerns in their own projects.
Researchers described their current strategies towards ethical concerns– often
manifesting as forms of inactions or workarounds– such as engaging LLM ethics
in a conditional and reactive manner, inconsistent disclosure practices,
limiting their LLM use and reflecting on their co-creation process, as well as
delaying responsibility in research accountability. In most cases, these
strategies did not directly address the underlying ethical challenges;
instead, they served as temporary fixes to manage immediate concerns. Below,
we capture these varied strategies to LLM ethics and return to their
implications in section 5.

####  4.3.1. Conditional and reactive engagement with LLM ethics

HCI researchers often emphasized that the need to and approach towards
addressing ethical concerns was conditional on various factors such as the use
casse or domains of study. Many HCI researchers invoked the specifics of their
research domain to justify why they did not need to foreground ethical
concerns. For example, when participants categorized their research as low-
stakes, they spoke of how commonly anticipated ethical issues associated with
LLMs were not applicable to their work, and they did not need to take
proactive measures. For instance, P11, who focused on creativity support in
writing, considered their research non-sensitive and did not find it necessary
to intervene to prevent unsafe generated text. Some participants believed that
the potential harms caused by LLMs were comparable to those of social media.
Therefore, they felt that engaging in ethical considerations was unnecessary
if LLMs were not being used in high-stakes domains, such as providing medical
advice: “typical LLM like ChatGPT was created to support people browsing the
internet. People already see toxic content online, right? So we always claim
this tool is no more harmful than typical social media” (P13). If researchers
explored topics deemed safe or if users did not directly prompt the model,
there was a perception that the LLM was unlikely to generate unsafe
content333In contrast, prior work has demonstrated that LMs can generate
unsafe content from seemingly innocuous prompts (Gehman et al., 2020)..

Such ‘conditional engagement’ sometimes resulted in a more reactive, rather
than proactive, approach to ethical considerations. For example, when
deliberating whether LLMs should be attributed and held accountable for model-
generated content, P2 emphasized “I think many people are very hasty to say
yes or no. And I think that’s not the answer. The answer is always in a gray
area.” They continued to emphasize this “wait and see” approach, discussing
the potential benefits of model hallucinations and highlighting their role in
promoting divergent thinking by introducing specificity that the user may not
have considered. Some participants mentioned being aware of frameworks and
toolkits (such as Perspective API (Jigsaw and team, 2017)) designed to address
ethical issues, but they never incorporated these tools into their own
research processes. P3 justified this by discussing how a significant portion
of their HCI studies were conducted in a laboratory setting. The ethical
concern related to participants encountering harmful outputs generated by LLMs
was less probable in a short usability test, while serious issues could occur
in longitudinal studies when the participants heavily use the system.

####  4.3.2. Limited disclosure practices

In our study, HCI researchers positioned large language models as everyday
tools within their research practice. As a result, participants did not
believe it was necessary to formally report their usage of LLMs to study
participants, the Institutional Review Board (IRB), and/or the broader
academic community. In particular, participants described a shift towards the
tacit incorporation of LLMs: they slowly transitioned from research tools that
were a part of their regular practice to using LLMs. This change was partly
due to the perception that LLMs are ‘fancier’ (P11) and more advanced versions
of previously used tools. P10 explained, “I advise my students that they are
allowed to use any generative AI tool just like they would use other
productivity tools. So I’m categorizing LLMs as a productivity tool.”

When using LLMs in paper writing, participants drew a parallel between LLM
reporting practices and their approach to previous tools. They emphasized that
if, for example, tools like “grammar assistance by Grammarly, word check by
Google Docs, or accessibility checking by Acrobat pro” (P10) were not
explicitly reported, the same should also hold true for large language models.
Researchers expressed reservations, suggesting that reporting of LLM-based
tools might call into question the validity of their work. P10 captured this
perspective:

> I would not feel it is appropriate to say Bing Chat was used to define the
> initial structure of X and Y sections of the paper. To me, it is not gonna
> be helpful in researchers assessing the credibility or validity of the work.
> It is just like a meta issue about how the actual document was formed and
> refined. And what really matters in that case is the output. Is it easy to
> read? Did you find it useful? You know, that’s what I care about. So, in
> those cases, I do not credit the LLM.

The complexity of describing LLMs to audiences without technical background
further impacted participants’ willingness to disclose the specific uses of
large language models in research. In some cases, researchers chose to
characterize their LLM use simply as ‘AI models’ to their research
participants so as not to “confuse them with what is a language model.” The
rationale behind this approach was the perception that the broader public
(target population in this case) tend to view AI in a homogeneous manner, and
“to them, there isn’t much difference between how different AI systems work,
or which is a large language model, right?” (P8). Researchers also justified
this approach by highlighting their intention not to burden participants with
the need to modify their behavior and “ think about how to interact with an
LLM” (P15). Finally, participants also noted cases where explicit disclosure
about LLMs was rather unnecessary if research participants or system users are
not directly exposed to LLMs. For example, if LLMs were used for ideation or
analysis, participants felt that explicit communication about their use was
not imperative.

####  4.3.3. Restricting LLM use and reflecting on the co-creation process

A recurring sentiment among the researchers in our study was a perceived lack
of control in addressing ethical concerns related to large language models. As
a result, they developed various workarounds, such as restricting the use of
LLMs to a limited set of tasks, avoiding directly integrating LLM-generated
inputs into their work, and hosting group reflection sessions. The limited
visibility and decision-making capability were particularly evident in issues
of privacy and data leaks, where the reliance on LLMs provided by large
companies diminished individual control. Simultaneously, participants
expressed a mistrust towards LLM providers. For instance, P12 discussed how
LLM providers’ claim to protect data privacy and not using it for their
training “is very problematic because such claims don’t really articulate what
they mean by not using the data. How can an external researcher validate this
claim?” Participants’ mistrust towards LLM providers was exacerbated by the
lack of clarity and transparency in data usage policies.

To navigate the ‘unknown territory’ of LLMs, where HCI researchers were not
fully aware of the capabilities and risks, they would often restrict their
usage of LLMs to a limited set of tasks. Most HCI researchers avoided directly
integrating LLM-generated outputs into their work. Rather than accepting the
initial output without scrutiny, they would iteratively refine and carefully
verify before incorporating it into their research artifacts. In the context
of paper writing, researchers would ensure the text aligns with the draft they
had input into the system. Many participants elaborated on their practice of
co-creating with LLMs as a precautionary measure. This would involve relying
on their personal judgment on how much to use the LLM suggestions. This
cautious and iterative co-creation process served as a strategy in the absence
of clear usage norms.

Indeed, many participants shared the view that LLMs are still evolving, and
there is a lack of comprehensive guidelines on navigating ethical
considerations. P9, primarily using LLMs for paper writing, mentioned how
“LLMs are still an unknown territory, so people don’t know how to react, I
assume.” Our participants expressed discomfort with using any new LLM-
generated content in their papers. For example, P15 was hesitant to use the
LLM as anything more than a spell checker. Researchers highlighted the
importance of applying LLMs in a way that is consistent with their practices
with previously used tools to mitigate potential unintended consequences. P9
spoke of their experience as a reviewer for major HCI conferences like CHI,
where they did not encounter any guidelines regarding the disclosure of using
ChatGPT for writing. Finally, they went on to emphasize that the cost and
accountability of using LLMs was still “ad-hoc and unclear.”

In cases where participants recognized the possibility of ethical concerns
with LLMs, they often struggled to identify specific issues to address. As P16
pointed out: “the main problem is that I don’t know what bias it has, and I
don’t know how to figure it out.” P4 discussed how it is crucial to “not
pretend that this ethical consideration does not exist, which some people do.
We are trained in human-computer interaction, and so we are well equipped to
reason about it or at least understand that these concerns exist, but
obviously addressing it is very difficult.” In many cases, when people did not
have mechanisms to identify or address ethical concerns, they felt it was
important to at least acknowledge them and spread awareness about these
issues.

To account for the uncertainties with the responsible use of LLMs, some
researchers chose to host group reflection sessions. This involved engaging in
discussions with collaborators or advisors to determine the appropriate
approach for navigating any ethical considerations. P10 exemplified this
approach by holding regular team meetings to address any questions or concerns
related to using LLM-based tools or methods. After an internal discussion with
their team, P9 too, decided not to use LLMs for qualitative analysis. A
collaborative decision-making process was a common practice to determine where
it is appropriate to use LLMs.

####  4.3.4. Delaying responsibility in research accountability

Finally, our participants indicated that determining who bears responsibility
for the ethical implications of using LLMs is challenging. At times, they
expressed reluctance towards stringent regulatory guidelines and opted to
postpone dealing with ethical issues in their projects.

In discussions about accountability with the use of LLMs within HCI research,
participants noted the ambiguity in determining where responsibility lies
regarding the ethical concerns of using LLMs. They highlighted the notion of
distributed responsibility across the AI supply chain, emphasizing that
multiple stakeholders share the responsibility for ensuring that the use of
LLMs within research is ethical. Participants highlighted the LLM provider’s
responsibility to implement safeguards preventing users from sending sensitive
personal data to the model. For instance, P8 discussed their confidence in LLM
providers, sharing how they “believe OpenAI has done a decent job in terms of
making a model safe in general.” P2 presented a similar perspective:

> “When professionally something wrong happens, it needs to follow the chain
> of command. So, who is the person most directly responsible? It’s actually
> the user, right? The user did something with my system, and it created a
> harmful output, and then the user will say, well, the system wasn’t designed
> well enough for me to know it’s gonna create this harmful output. I should
> have been warned. In that case, the responsibility falls on me. And then I
> could move it up the chain of command to say my advisor shouldn’t have let
> me release this system if it was gonna produce harmful output or I could say
> OpenAI is irresponsible for releasing a project that they are publicizing as
> being broadly accessible and safe.”

Partially due to the difficulty in determining “distributed responsibility”
for some HCI researchers, addressing ethical concerns could be relegated to
future work. They viewed it as imperative, as HCI researchers focused on
system-building, to explore emerging technologies and develop new tools and
interactions with them. As articulated by P6, “we are the system builders, and
we think about what are the unique benefits that the new technology can bring
to the users and enhance the user capabilities.” This perspective underscored
the desire to continue building tools, even as researchers acknowledged the
biases associated with LLMs.

Sometimes, participants even expressed a resistance towards prescriptive
regulations, which they perceived would suppress innovation in HCI research.
They emphasized that external constraints on LLM usage, especially given its
role in everyday research practices, would slow down their research. LLMs were
also perceived to lack the stability necessary for regulation. As P12
described, “the challenge is the regulation needs to come much later rather
than early. If you come up with regulations too early, you may kill innovation
and on the other hand, you don’t know what you want to regulate because the
representation of the product hasn’t been stabilized yet.”

##  5\. Discussion

Our results highlight the various ways in which HCI researchers integrated
LLMs in their research projects, for the purpose of research ideation (e.g.,
finding novel research areas), data collection and analysis (e.g., qualitative
coding), preparing artifacts (e.g., drafting research publications), and more.
Many participants were aware of the potential ethical concerns related to the
use LLMs in HCI research, including the harms of engaging with LLM outputs,
threats to the privacy of participant data, intellectual integrity, and
environmental and societal impacts. However, the sociotechnical assemblage of
LLMs mediated researchers’ diverse approaches to these ethical concerns.

While HCI researchers are increasingly using LLM-based tools within their
‘everyday’ practices, our research community still lacks well-established
guidelines and best practices, contributing to the complexity of navigating
ethical considerations. In addition, researchers noted a perceived lack of
control over the functionality and outputs of LLMs. Unlike previous tools
where researchers had more predictability and influence over the behavior,
LLMs presented outputs that may not always align with explicit instructions.
Finally, many participants spoke of the challenges of navigating ethical
concerns within the broader ecosystem of the ‘LLM supply chain’ (Cobbe et al.,
2023). Below, we present the implications of researchers’ approaches and
opportunities for HCI researchers to better engage with ethical considerations
of LLMs as part of their projects, with a goal to support the formation of
emerging ethical norms in LLM-impacted HCI research.

Proactively engaging with IRB and other regulatory institutions. Within the
U.S. context, Institutional Review Boards (IRBs) are responsible for reviewing
and monitoring research activities to protect the rights and welfare of human
research subjects (of Health and Services, 2018). One of the primary
responsibilities of the IRB office is to assess whether “risks to subjects are
reasonable in relation to anticipated benefits” (of Health and Services, [n.
d.]). Given their expertise in ensuring ethical conduct in research practices,
the IRB office may be well-positioned to advise on identifying ethical
concerns and potential prevention strategies. To establish whether a research
study will undergo a comprehensive review, they determine if it is a minimal
risk study where the “probability and magnitude of harm anticipated in the
research are not greater than those ordinarily encountered in daily life” (of
Health and Services, [n. d.]).

Our findings reveal that most HCI researchers did not consider it necessary to
report their usage of LLMs to the IRB, in part due to their perception of LLMs
as everyday tools. According to some, their scope of use did not justify
including additional details, while others did not anticipate their use of
LLMs at the time of submitting the IRB application. For those who disclosed
their use of LLMs to the IRB, they used the minimal risk rule (of Health and
Services, [n. d.]) to indicate that research subjects would typically
encounter harmful outputs in their daily life as well. However, these
practices can have short-term and long-term adverse implications. In the short
term, the opacity in research practices (e.g., which tools are used to
generate research artifacts or analyze data) might lead to challenges in
replicating studies (in Human–Computer Interaction Working Group, 2019) and
understanding the motivation and effects of methodological decisions
(Salehzadeh Niksirat et al., 2023). In the long term, the consequences of
limited disclosure might extend beyond individual studies to shaping the
trajectory of the community, for example, by informing which research topics
we pursue.

We invite HCI researchers to proactively engage with the IRB at the time of
study design to unpack the likelihood and ways in which any potential LLM use
might harm our participants. This includes careful reflection and
documentation of any implicit or anticipated use during the project planning
stage. Furthermore, researchers should actively implement mechanisms to
monitor LLM outputs and any adverse impacts on participants throughout the
project life-cycle, especially if used for system building. In addition,
openly communicating and collaborating with IRB members to uncover potential
harms is also important to increase transparency. Embracing guidelines,
policies and oversight is essential for ensuring responsible development and
use of these technologies. Bockting et al. (2023) urged scientists to oversee
the use of generative AI because “controlling developments in AI will require
a continuous process that balances expertise and independence.” We call for
collaborative efforts between researchers, policymakers, and generative AI
companies to create a set of ‘living guidelines’ for the responsible use of
generative AI in research (Bockting et al., 2023).

Re-examining the informed consent process. In our study, researchers
acknowledged the limited transparency to their study participants, either by
not disclosing LLM use (e.g., considering it irrelevant unless directly
involved in system building), or by characterizing the underlying technology
as Artificial Intelligence (AI). This was often rationalized as a way to avoid
overwhelming participants with seemingly unnecessary technical details.
However, terminologies can be powerful too: they shape narratives around
capabilities and set expectations for use (Cave et al., 2018). AI can appear
‘magical’, where systems are considered reliable and free of bias (Kapania et
al., 2022). In contrast, LLMs have specific evidence-based, known risks
(Weidinger et al., 2022) that must be clearly communicated to research
subjects. For example, in March 2023, a bug allowed some ChatGPT users to see
the titles of other users’ chat histories (Browne, 2023). It is important to
recognize that our research practices can place participants’ safety at risk
(Shklovski and Vertesi, 2012).

Researchers should carefully attend to the informed consent process for
projects that may involve the use of LLMs across different research stages.
There is a well-established body of work on informed consent in research that
advocates for moving beyond obtaining consent as ‘instrumental in nature’
(Hamilton, 2009) only to satisfy regulatory and reporting obligations
(Klykken, 2022). Research participants need to have a clear understanding of
the objectives of the research, methods, procedures, as well any foreseeable
risks (for Human Research Protections, 1998). How might informed consent
facilitate a collective, continuous sense-making process between the
researcher and participant to understand the implications and ethical concerns
of using LLMs? Nonetheless, it is crucial to highlight that documentation,
transparency, and effective informed consent might help identify and prevent
issues, but they cannot substitute for efforts to mitigate ethical concerns
with the use of LLMs.

Developing tools, methods and processes to interrupt the LLM supply chain.
Over the last few years, progress in Natural Language Processing (NLP) has
been characterized by the development of larger language models (Bender et
al., 2021) requiring substantial computational resources and access to
extensive datasets that have been challenging to obtain outside big companies
and well-funded research labs. As a result, large language models are
increasingly organized within a supply chain and produced by several
interconnected actors. Cobbe et al. (Cobbe et al., 2023) describe the
algorithmic supply chain as a pipeline where ‘several actors contribute
towards the production, deployment, use, and functionality of AI
technologies.’ In the case of large language models, companies providing LLMs
‘as a service’ offer access to pre-built general-purpose models. As a result,
LLM providers hold a lot of control over the distribution and development of
the underlying technologies, including how to identify and address ethical
concerns. Researchers in our study, situated downstream within the algorithmic
supply chain, described a lack of control to examine and mitigate ethical
issues. While some expressed a lack of desire, others struggled to pay off
ethical debt (Petrozzino, 2021) accrued through components developed upstream
in the supply chain. How might we then interrupt the supply chain and shift
control downstream: could we imagine tools where researchers can set up and
maintain the infrastructures needed to integrate large language models in
their workflow? How might such tools center a proactive approach to identify
ethical concerns?

We recognize the barriers in navigating ethical concerns when researchers
depend on AI service providers for access to these tools. There is a
significant practical cost associated with choosing a language model that
attends to ethical concerns. Our findings suggest opportunities to create
tools, methods and processes to support HCI researchers in better navigating
the ethical considerations of integrating language models into their projects.
For example, known evaluation frameworks for the use of LLMs in social science
research such as CoMPosT (Cheng et al., 2023b) make LLMs’ limitations
transparent to researchers, aiding them in regaining control over the LLM
tools. In addition to validation tools for LLMs for research-specific
purposes, there is a growing call for auditing LLMs across technology
providers, model development, and downstream applications, covering the LLM
supply chain ecosystem (Mökander et al., 2023). Auditing tools of LLMs are
crucial for overseeing whether technology providers maintain ethical standards
and ensure responsible development of LLMs including transparent policies on
user data handling.

A potential issue of depending on the black-box LLMs from service providers,
which is also discovered in our findings, is the risk of inheriting the biases
and misinformation from upstream in the LLM supply chain. To address these
ethical challenges, it is essential to develop frameworks and tools that
empower researchers to host and manage their own LLMs for research. There has
been research providing a comprehensive and practical guide for practitioners
and end-users to work with LLMs (Yang et al., 2023), with a focus on closed-
source LLMs such as ChatGPT. In order to offer more autonomy and control for
researchers in integrating LLMs into their research workflow, new
technological support in streaming that process is needed. By providing these
capabilities, researchers would be able to actively engage with and mitigate
the ethical concerns.

Creating learning opportunities and materials for ethics of LLM use in HCI.
Researchers’ awareness of ethical issues related to LLMs influenced how they
address these considerations in their projects. Many researchers spoke of LLMs
as an unfamiliar territory, lacking the training and well-established
guidelines to approach ethical considerations with this emerging technology.
We could draw on lessons from scholarship in FAccT, STS, NLP and other allied
domains, as well as industry resources, to develop learning opportunities,
such as toolkits (Shen et al., 2021), guidebooks (PAIR, 2021), or frameworks
(Derczynski et al., 2023; of Standards and Technology, 2023), for
understanding and addressing ethical concerns with using LLMs. Achieving this
requires particular attention from the HCI community to collaboratively
explore how LLMs are used in different stages of HCI research.

Conferences are a valuable starting point to promote cross-institutional
learning. We propose organizing workshops and panels (such as (Shen et al.,
2023)) with interdisciplinary experts in sociotechnical understanding of LLMs
to raise awareness on the impacts of using LLMs. Additionally, we call for
creating and disseminating case studies (e.g., (PAIR, [n. d.])), potentially
included in newsletters, illustrating how HCI researchers actively address
ethical concerns related to LLMs. A repository of case studies, covering
diverse HCI domains and epistemologies, might also offer resources for
preventing and mitigating ethical concerns. Education on research ethics for
LLM use should be a necessary component of HCI research and practice, given
the well-documented risks and adverse impacts of LLMs (Weidinger et al.,
2022). One avenue is incorporating such case studies into HCI curriculum,
especially if targeted towards research students, to provide real-world
examples that help build their awareness of these issues.

Shifting academic incentives to foreground ethical concerns.  Throughout the
interviews, we observed that researchers could generally foresee potential
ethical concerns with LLMs, often drawing from their familiarity with the
literature or discussions with other researchers. Nonetheless, some
participants went on to articulate their reason for not prioritizing these
ethical considerations in their projects. Researchers described how
constraints such as limited funding, pressure to publish, conference
deadlines, often came in the way of focusing on ethical concerns. The need to
address ethical issues could then be relegated to the limitations or future
work section, reflecting broader perceptions within Computer Science research
that ethics are often viewed as secondary considerations. Indeed, Do and Pang
et al. (Do et al., 2023) discussed how academics may have lower incentive to
examine unintended consequences of their research (also observed in our study)
in contrast with industry practices.

There is a renewed urgency to reconsider research ethics practices within HCI
and how we navigate challenges presented by emerging technologies such as
LLMs. We take inspiration from Do and Pang et al. (Do et al., 2023) and
Soergel et al. (Soergel et al., 2013) to propose starting points for changing
structural incentives within academia. Publications and citations are the
currency and means for upward mobility in research. Could we shift criteria
for recognition and funding to explicitly recognize attention to ethical
considerations in research practices? Publishing organizations, funding
agencies, and regulatory institutions will pay a crucial role in reshaping
incentives. Most importantly, HCI researchers must acknowledge the ways in
which they actively shape the discourse around associated risks and ethical
considerations through their research, and work actively towards a cultural
shift that demonstrates a commitment to ethical use of LLMs.

##  6\. Limitations and Future Work

Our study sheds light on the emerging practices regarding LLM ethics among HCI
researchers, but it has limitations due to its exploratory nature. Firstly,
while we sought to offer a deep understanding on how HCI researchers navigate
LLM ethics, our sample was constrained by our snowball sampling method. HCI
research encompasses a wide range of diverse research traditions, many of
which we were unable to include in our study. This limitation highlights the
need for more comprehensive and systematic future studies in this area. Our
interview sample primarily consists of researchers from the USA, and future
research may want to further explore the ethical challenges and practices
related to the use of LLMs by researchers from other regions. Secondly,
there’s a potential selection bias in our study: we may have primarily
attracted respondents who are conscious of their LLM usage and are open to
discussing their experiences in a research setting. To gain a broader
perspective, future research should explore how ethical practices with LLMs
vary across different research methodologies, domains, and settings, including
both industry and academia.

##  7\. Conclusion

In this paper, we drew empirical data from a survey and interviews to explore
how HCI researchers have currently integrated LLMs into their research
practices, what ethical concerns they have encountered as well as how they’ve
navigated those concerns. Our results suggested that although HCI researchers
have used LLMs across their research processes and are aware of a wide variety
of ethical concerns, in many cases, they have challenges in effectively
identifying and navigating those concerns in their own projects. Reflecting on
these findings, we discuss potential approaches to support the formation of
emerging ethical norms for using LLMs in HCI research. We encourage HCI
researchers to proactively engage with IRB and collaborate with policymakers
and generative AI companies on creating guidelines for the responsible use of
LLMs. We also identify the need to re-examine the informed consent process and
provide technological support to interrupt the LLM supply chain. In addition,
we discuss the importance of creating learning opportunities for the ethics of
LLMs use in HCI and shifting academic incentives to prioritize ethical
concerns.

## References

  * (1)
  * Amershi et al. (2019) Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019.  Guidelines for Human-AI Interaction. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_ (Glasgow, Scotland Uk) _(CHI ’19)_. Association for Computing Machinery, New York, NY, USA, 1–13.  https://doi.org/10.1145/3290605.3300233
  * Antle (2017) Alissa N Antle. 2017.  The ethics of doing research with vulnerable populations.  _Interactions_ 24, 6 (2017), 74–77.
  * Bail (2023) Christopher Bail. 2023.  Can generative artificial intelligence improve social science.
  * Bender et al. (2021) Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the dangers of stochastic parrots: Can language models be too big?. In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_. 610–623.
  * Board (2021) ACM Publications Board. 2021.  ACM Publications Policy on Research Involving Human Participants and Subjects.  https://www.acm.org/publications/policies/research-involving-human-participants-and-subjects.  (Accessed on 01/16/2024).
  * Bockting et al. (2023) Claudi L. Bockting, Eva A. M. van Dis, Robert van Rooij, Willem Zuidema, and Johan Bollen. 2023.  Living guidelines for generative AI — why scientists must oversee its use.  _Nature_ 622, 7984 (Oct. 2023), 693–696.  https://doi.org/10.1038/d41586-023-03266-1
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative research in psychology_ 3, 2 (2006), 77–101.
  * Braun and Clarke (2019) Virginia Braun and Victoria Clarke. 2019.  Reflecting on reflexive thematic analysis.  _Qualitative research in sport, exercise and health_ 11, 4 (2019), 589–597.
  * Brown et al. (2016) Barry Brown, Alexandra Weilenmann, Donald McMillan, and Airi Lampinen. 2016.  Five Provocations for Ethical HCI Research. In _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_ (San Jose, California, USA) _(CHI ’16)_. Association for Computing Machinery, New York, NY, USA, 852–863.  https://doi.org/10.1145/2858036.2858313
  * Browne (2023) Ryan Browne. 2023.  OpenAI CEO says a bug allowed some ChatGPT to see others’ chat titles.  https://www.cnbc.com/2023/03/23/openai-ceo-says-a-bug-allowed-some-chatgpt-to-see-others-chat-titles.html.  (Accessed on 01/15/2024).
  * Bruckman (2014) Amy Bruckman. 2014.  Research ethics and HCI.  _Ways of Knowing in HCI_ (2014), 449–468.
  * Bruckman et al. (2017) Amy S Bruckman, Casey Fiesler, Jeff Hancock, and Cosmin Munteanu. 2017.  CSCW research ethics town hall: Working towards community norms. In _Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_. 113–115.
  * Buchanan et al. (2011) Elizabeth Buchanan, John Aycock, Scott Dexter, David Dittrich, and Erin Hvizdak. 2011.  Computer Science Security Research And Human Subjects: Emerging Considerations For Research Ethics Boards.  _Journal of empirical research on human research ethics : JERHRE_ 6 (06 2011), 71–83.  https://doi.org/10.1525/jer.2011.6.2.71
  * Buchanan and Ess (2009) Elizabeth A. Buchanan and Charles M. Ess. 2009.  Internet Research Ethics and the Institutional Review Board: Current Practices and Issues.  _SIGCAS Comput. Soc._ 39, 3 (dec 2009), 43–49.  https://doi.org/10.1145/1713066.1713069
  * Buchanan and Zimmer (2023) Elizabeth A. Buchanan and Michael Zimmer. 2023.  Internet Research Ethics.  In _The Stanford Encyclopedia of Philosophy_ (Winter 2023 ed.), Edward N. Zalta and Uri Nodelman (Eds.). Metaphysics Research Lab, Stanford University.
  * Byun et al. (2023) Courtni Byun, Piper Vasicek, and Kevin Seppi. 2023.  Dispensing with Humans in Human-Computer Interaction Research. In _Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems_ (¡conf-loc¿, ¡city¿Hamburg¡/city¿, ¡country¿Germany¡/country¿, ¡/conf-loc¿) _(CHI EA ’23)_. Association for Computing Machinery, New York, NY, USA, Article 413, 26 pages.  https://doi.org/10.1145/3544549.3582749
  * Cahill (2007) Caitlin Cahill. 2007.  Including excluded perspectives in participatory action research.  _Design Studies_ 28, 3 (2007), 325–340.
  * Carlini et al. (2021) Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021.  Extracting Training Data from Large Language Models.  arXiv:2012.07805 [cs.CR]
  * Cave et al. (2018) Stephen Cave, Claire Craig, Kanta Dihal, Sarah Dillon, Jessica Montgomery, Beth Singler, and Lindsay Taylor. 2018.  Portrayals and perceptions of AI and why they matter.  (2018).
  * Chairs (2023) ACL 2023 Program Chairs. 2023.  ACL 2023 Policy on AI Writing Assistance - ACL 2023.  https://2023.aclweb.org/blog/ACL-2023-policy/.  (Accessed on 01/11/2024).
  * Chalmers et al. (2011) Matthew Chalmers, Donald McMillan, Alistair Morrison, Henriette Cramer, Mattias Rost, and Wendy Mackay. 2011.  Ethics, logs and videotape: ethics in large scale user trials and user generated content.  In _CHI’11 Extended Abstracts on Human Factors in Computing Systems_. 2421–2424.
  * Chen and Shu (2023a) Canyu Chen and Kai Shu. 2023a.  Can LLM-Generated Misinformation Be Detected?  arXiv:2309.13788 [cs.CL]
  * Chen and Shu (2023b) Canyu Chen and Kai Shu. 2023b.  Combating Misinformation in the Age of LLMs: Opportunities and Challenges.  arXiv:2311.05656 [cs.CY]
  * Cheng et al. (2023a) Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023a.  Compost: Characterizing and evaluating caricature in llm simulations.  _arXiv preprint arXiv:2310.11501_ (2023).
  * Cheng et al. (2023b) Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023b.  CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10853–10875.  https://doi.org/10.18653/v1/2023.emnlp-main.669
  * Chew et al. (2023) Robert Chew, John Bollenbacher, Michael Wenger, Jessica Speer, and Annice Kim. 2023.  LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding.  arXiv:2306.14924 [cs.CL]
  * Clark et al. (2018) Karin Clark, Matt Duckham, Marilys Guillemin, Assunta Hunter, Jodie McVernon, Christine O’Keefe, Cathy Pitkin, Steven Prawer, Richard Sinnott, Deborah Warr, and Jenny Waycott. 2018.  Advancing the ethical use of digital data in human research: challenges and strategies to promote ethical practice.  _Ethics and Information Technology_ 21, 1 (Nov. 2018), 59–73.  https://doi.org/10.1007/s10676-018-9490-4
  * Cobbe et al. (2023) Jennifer Cobbe, Michael Veale, and Jatinder Singh. 2023.  Understanding accountability in algorithmic supply chains. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. 1186–1197.
  * Demszky et al. (2023) Dorottya Demszky, Diyi Yang, David S Yeager, Christopher J Bryan, Margarett Clapper, Susannah Chandhok, Johannes C Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, et al. 2023\.  Using large language models in psychology.  _Nature Reviews Psychology_ 2, 11 (2023), 688–701.
  * Derczynski et al. (2023) Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, and Saif Mohammad. 2023.  Assessing Language Model Deployment with Risk Cards.  arXiv:2303.18190 [cs.CL]
  * Deshpande et al. (2023) Ameet Deshpande, Tanmay Rajpurohit, Karthik Narasimhan, and Ashwin Kalyan. 2023.  Anthropomorphization of AI: Opportunities and Risks.  arXiv:2305.14784 [cs.AI]
  * Do et al. (2023) Kimberly Do, Rock Yuren Pang, Jiachen Jiang, and Katharina Reinecke. 2023.  “That’s important, but…”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–16.
  * Fang et al. (2023) Xiao Fang, Shangkun Che, Minjia Mao, Hongzhe Zhang, Ming Zhao, and Xiaohang Zhao. 2023.  Bias of AI-Generated Content: An Examination of News Produced by Large Language Models.  arXiv:2309.09825 [cs.AI]
  * Feng et al. (2023) Yunhe Feng, Sreecharan Vanam, Manasa Cherukupally, Weijian Zheng, Meikang Qiu, and Haihua Chen. 2023.  Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data. In _2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)_. 876–885.  https://doi.org/10.1109/COMPSAC57700.2023.00117
  * Fiesler et al. (2021) Casey Fiesler, Melissa Densmore, Michael Muller, and Cosmin Munteanu. 2021.  SIGCHI Research Ethics Committee Town Hall. In _Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing_. 232–233.
  * Fiesler et al. (2022) Casey Fiesler, Christopher Frauenberger, Michael Muller, Jessica Vitak, and Michael Zimmer. 2022.  Research Ethics in HCI: A SIGCHI Community Discussion. In _Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems_ (New Orleans, LA, USA) _(CHI EA ’22)_. Association for Computing Machinery, New York, NY, USA, Article 169, 3 pages.  https://doi.org/10.1145/3491101.3516400
  * Fiesler et al. (2018) Casey Fiesler, Jeff Hancock, Amy Bruckman, Michael Muller, Cosmin Munteanu, and Melissa Densmore. 2018.  Research Ethics for HCI: A roundtable discussion. In _Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems_. 1–5.
  * Fiesler and Proferes (2018) Casey Fiesler and Nicholas Proferes. 2018.  “Participant” perceptions of Twitter research ethics.  _Social Media+ Society_ 4, 1 (2018), 2056305118763366.
  * for Computing Machinery (2023) Association for Computing Machinery. 2023.  ACM Policy on Authorship.  https://www.acm.org/publications/policies/new-acm-policy-on-authorship.  (Accessed on 01/11/2024).
  * for Human Research Protections (1998) Office for Human Research Protections. 1998.  Informed Consent Checklist (1998) — HHS.gov.  https://www.hhs.gov/ohrp/regulations-and-policy/guidance/checklists/index.html.  (Accessed on 01/15/2024).
  * Force (2018) ACM Code 2018 Task Force. 2018.  Code of Ethics.  https://www.acm.org/code-of-ethics.  (Accessed on 01/16/2024).
  * Frauenberger et al. (2017) Christopher Frauenberger, Amy S Bruckman, Cosmin Munteanu, Melissa Densmore, and Jenny Waycott. 2017.  Research Ethics in HCI: A town hall meeting. In _Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems_. 1295–1299.
  * Gao et al. (2023b) Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. 2023b.  S3: Social-network Simulation System with Large Language Model-Empowered Agents.  arXiv:2307.14984 [cs.SI]
  * Gao et al. (2023a) Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, and Simon Tangi Perrault. 2023a.  CollabCoder: A GPT-Powered Workflow for Collaborative Qualitative Analysis.  arXiv:2304.07366 [cs.HC]
  * Gebreegziabher et al. (2023) Simret Araya Gebreegziabher, Zheng Zhang, Xiaohang Tang, Yihao Meng, Elena L Glassman, and Toby Jia-Jun Li. 2023.  Patat: Human-ai collaborative qualitative coding with explainable interactive rule synthesis. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–19.
  * Gehman et al. (2020) Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020.  Realtoxicityprompts: Evaluating neural toxic degeneration in language models.  _arXiv preprint arXiv:2009.11462_ (2020).
  * Girotra et al. (2023) Karan Girotra, Lennart Meincke, Christian Terwiesch, and Karl T. Ulrich. 2023.  Ideas are dimes a dozen: Large language models for Idea generation in innovation.  https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4526071#
  * Hämäläinen et al. (2023) Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023.  Evaluating Large Language Models in Generating Synthetic HCI Research Data: A Case Study. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ (Hamburg, Germany) _(CHI ’23)_. Association for Computing Machinery, New York, NY, USA, Article 433, 19 pages.  https://doi.org/10.1145/3544548.3580688
  * Hamilton (2009) Jennifer A Hamilton. 2009.  On the ethics of unusable data.  _Fieldwork is not what it used to be: Learning anthropology’s method in a time of transition_ (2009), 73–88.
  * in Human–Computer Interaction Working Group (2019) Transparent Statistics in Human–Computer Interaction Working Group. 2019.  Transparent Statistics Guidelines.  https://doi.org/10.5281/zenodo.1186169 (Available at https://transparentstats.github.io/guidelines).
  * Ivankova et al. (2006) Nataliya V Ivankova, John W Creswell, and Sheldon L Stick. 2006.  Using mixed-methods sequential explanatory design: From theory to practice.  _Field methods_ 18, 1 (2006), 3–20.
  * Jansen et al. (2023) Bernard J. Jansen, Soon gyo Jung, and Joni Salminen. 2023.  Employing large language models in survey research.  _Natural Language Processing Journal_ 4 (2023), 100020.  https://doi.org/10.1016/j.nlp.2023.100020
  * Jigsaw and team (2017) Jigsaw and Google’s Counter Abuse Technology team. 2017.  Perspective API.  https://perspectiveapi.com/.  (Accessed on 01/16/2024).
  * Kapania et al. (2022) Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu Meena SP, and Nithya Sambasivan. 2022.  ” Because AI is 100% right and safe”: User attitudes and sources of AI authority in India. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_. 1–18.
  * Kasneci et al. (2023) Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. 2023\.  ChatGPT for good? On opportunities and challenges of large language models for education.  _Learning and individual differences_ 103 (2023), 102274.
  * Klykken (2022) Fride Haram Klykken. 2022.  Implementing continuous consent in qualitative research.  _Qualitative Research_ 22, 5 (2022), 795–810.
  * Leiker et al. (2023) Daniel Leiker, Sara Finnigan, Ashley Ricker Gyllen, and Mutlu Cukurova. 2023.  Prototyping the use of Large Language Models (LLMs) for adult learning content creation at scale.  arXiv:2306.01815 [cs.CY]
  * Li et al. (2023) Yuan Li, Yixuan Zhang, and Lichao Sun. 2023.  MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents.  arXiv:2310.06500 [cs.AI]
  * Liang et al. (2021) Calvin A Liang, Sean A Munson, and Julie A Kientz. 2021.  Embracing four tensions in human-computer interaction research with marginalized people.  _ACM Transactions on Computer-Human Interaction (TOCHI)_ 28, 2 (2021), 1–47.
  * Liao and Vaughan (2023) Q. Vera Liao and Jennifer Wortman Vaughan. 2023.  AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap.  arXiv:2306.01941 [cs.HC]
  * Liu et al. (2023) Yiren Liu, Si Chen, Haocong Cheng, Mengxia Yu, Xiao Ran, Andrew Mo, Yiliu Tang, and Yun Huang. 2023.  How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent.  arXiv:2310.06155 [cs.HC]
  * Lu et al. (2022) Yuwen Lu, Chengzhi Zhang, Iris Zhang, and Toby Jia-Jun Li. 2022.  Bridging the Gap Between UX Practitioners’ Work Practices and AI-Enabled Design Support Tools. In _Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems_ (New Orleans, LA, USA) _(CHI EA ’22)_. Association for Computing Machinery, New York, NY, USA, Article 268, 7 pages.  https://doi.org/10.1145/3491101.3519809
  * Lukas et al. (2023) Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023.  Analyzing Leakage of Personally Identifiable Information in Language Models.  arXiv:2302.00539 [cs.LG]
  * Ma et al. (2023) Kevin Ma, Daniele Grandi, Christopher McComb, and Kosa Goucher-Lambert. 2023.  Conceptual Design Generation Using Large Language Models.  arXiv:2306.01779 [cs.CL]
  * Mackay (1995) Wendy E Mackay. 1995.  Ethics, lies and videotape…. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_. 138–145.
  * McDonald et al. (2019) Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019.  Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice.  _Proceedings of the ACM on human-computer interaction_ 3, CSCW (2019), 1–23.
  * McNutt et al. (2023) Andrew M McNutt, Chenglong Wang, Robert A Deline, and Steven M Drucker. 2023.  On the design of ai-powered code assistants for notebooks. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–16.
  * Mireshghallah et al. (2023) Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. 2023.  Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory.  arXiv:2310.17884 [cs.AI]
  * Mozes et al. (2023) Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D. Griffin. 2023.  Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities.  arXiv:2308.12833 [cs.CL]
  * Munteanu et al. (2019) Cosmin Munteanu, Amy Bruckman, Michael Muller, Christopher Frauenberger, Casey Fiesler, Robert E Kraut, Katie Shilton, and Jenny Waycott. 2019.  Sigchi research ethics town hall. In _Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–6.
  * Munteanu et al. (2015) Cosmin Munteanu, Heather Molyneaux, Wendy Moncur, Mario Romero, Susan O’Donnell, and John Vines. 2015.  Situational Ethics: Re-Thinking Approaches to Formal Ethics Requirements for Human-Computer Interaction. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ (Seoul, Republic of Korea) _(CHI ’15)_. Association for Computing Machinery, New York, NY, USA, 105–114.  https://doi.org/10.1145/2702123.2702481
  * Mökander et al. (2023) Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. 2023.  Auditing large language models: a three-layered approach.  _AI and Ethics_ (May 2023).  https://doi.org/10.1007/s43681-023-00289-2
  * Ning et al. (2023) Zheng Ning, Zheng Zhang, Tianyi Sun, Yuan Tian, Tianyi Zhang, and Toby Jia-Jun Li. 2023.  An empirical study of model errors and user error discovery and repair strategies in natural language database queries. In _Proceedings of the 28th International Conference on Intelligent User Interfaces_. 633–649.
  * of Health and Services ([n. d.]) US Department of Health and Human Services. [n. d.].  2018 Requirements (2018 Common Rule) — HHS.gov.  https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/revised-common-rule-regulatory-text/index.html.  (Accessed on 01/15/2024).
  * of Health and Services (2018) US Department of Health and Human Services. 2018.  Policy for the Protection of Human Subjects.  https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html.  (Accessed on 01/15/2024).
  * of Standards and Technology (2023) National Institute of Standards and Technology. 2023.  Artificial Intelligence Risk Management Framework (AI RMF 1.0).  (2023).
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  arXiv:2303.08774 [cs.CL]
  * Oppenlaender and Hämäläinen (2023) Jonas Oppenlaender and Joonas Hämäläinen. 2023.  Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT and GPT-4 for Mining Insights at Scale.  arXiv:2306.05036 [cs.HC]
  * PAIR ([n. d.]) Google PAIR. [n. d.].  People + AI Research Guidebook - Case Studies.  https://pair.withgoogle.com/guidebook/case-studies.  (Accessed on 01/15/2024).
  * PAIR (2021) Google PAIR. 2021.  People + AI Guidebook.  https://pair.withgoogle.com/guidebook.  (Accessed on 01/15/2024).
  * Palinkas et al. (2013) Lawrence Palinkas, Sarah Horwitz, Carla Green, Jennifer Wisdom, Naihua Duan, and Kimberly Hoagwood. 2013.  Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research.  _Administration and policy in mental health_ 42 (11 2013).  https://doi.org/10.1007/s10488-013-0528-y
  * Pan et al. (2023) Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang. 2023.  On the Risk of Misinformation Pollution with Large Language Models.  arXiv:2305.13661 [cs.CL]
  * Paoli (2023) Stefano De Paoli. 2023.  Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the model.  arXiv:2305.13014 [cs.CL]
  * Park et al. (2023) Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023.  Generative agents: Interactive simulacra of human behavior. In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_. 1–22.
  * Park et al. (2022) Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022.  Social Simulacra: Creating Populated Prototypes for Social Computing Systems.  arXiv:2208.04024 [cs.HC]
  * Petridis et al. (2023) Savvas Petridis, Michael Terry, and Carrie Jun Cai. 2023.  PromptInfuser: Bringing User Interface Mock-Ups to Life with Large Language Models. In _Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems_ (¡conf-loc¿, ¡city¿Hamburg¡/city¿, ¡country¿Germany¡/country¿, ¡/conf-loc¿) _(CHI EA ’23)_. Association for Computing Machinery, New York, NY, USA, Article 237, 6 pages.  https://doi.org/10.1145/3544549.3585628
  * Petrozzino (2021) Catherine Petrozzino. 2021.  Who pays for ethical debt in AI?  _AI and Ethics_ 1, 3 (2021), 205–208.
  * Punchoojit and Hongwarittorrn (2015) Lumpapun Punchoojit and Nuttanont Hongwarittorrn. 2015.  Research ethics in human-computer interaction: A review of ethical concerns in the past five years. In _2015 2nd National Foundation for Science and Technology Development Conference on Information and Computer Science (NICS)_. 180–185.  https://doi.org/10.1109/NICS.2015.7302187
  * Qureshi et al. (2023) Riaz Qureshi, Daniel Shaughnessy, Kayden Gill, Karen Robinson, Tianjing Li, and Eitan Agai. 2023.  Are ChatGPT and large language models ”the answer” to bringing us closer to systematic review automation?  _Systematic reviews_ 12 (04 2023), 72.  https://doi.org/10.1186/s13643-023-02243-z
  * Salehzadeh Niksirat et al. (2023) Kavous Salehzadeh Niksirat, Lahari Goswami, Pooja S. B. Rao, James Tyler, Alessandro Silacci, Sadiq Aliyu, Annika Aebli, Chat Wacharamanotham, and Mauro Cherubini. 2023.  Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ (¡conf-loc¿, ¡city¿Hamburg¡/city¿, ¡country¿Germany¡/country¿, ¡/conf-loc¿) _(CHI ’23)_. Association for Computing Machinery, New York, NY, USA, Article 505, 23 pages.  https://doi.org/10.1145/3544548.3580848
  * Saltz et al. (2019) Jeffrey Saltz, Michael Skirpan, Casey Fiesler, Micha Gorelick, Tom Yeh, Robert Heckman, Neil Dewar, and Nathan Beard. 2019.  Integrating Ethics within Machine Learning Courses.  _ACM Trans. Comput. Educ._ 19, 4, Article 32 (aug 2019), 26 pages.  https://doi.org/10.1145/3341164
  * Shen et al. (2021) Hong Shen, Wesley H Deng, Aditi Chattopadhyay, Zhiwei Steven Wu, Xu Wang, and Haiyi Zhu. 2021.  Value cards: An educational toolkit for teaching social impacts of machine learning through deliberation. In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_. 850–861.
  * Shen et al. (2023) Hong Shen, Tianshi Li, Toby Jia-Jun Li, Joon Sung Park, and Diyi Yang. 2023.  Shaping the Emerging Norms of Using Large Language Models in Social Computing Research. In _Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing_. 569–571.
  * Shklovski and Vertesi (2012) Irina Shklovski and Janet Vertesi. 2012.  “Un-Googling”: Research Technologies, Communities at Risk and the Ethics of User Studies in HCI. In _Electronic Workshops in Computing_.
  * Shneiderman (2020) Ben Shneiderman. 2020.  Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-Centered AI Systems.  _ACM Trans. Interact. Intell. Syst._ 10, 4, Article 26 (oct 2020), 31 pages.  https://doi.org/10.1145/3419764
  * Smith et al. (2023a) Jessie J. Smith, Blakeley H. Payne, Shamika Klassen, Dylan Thomas Doyle, and Casey Fiesler. 2023a.  Incorporating Ethics in Computing Courses: Barriers, Support, and Perspectives from Educators. In _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_ (¡conf-loc¿, ¡city¿Toronto ON¡/city¿, ¡country¿Canada¡/country¿, ¡/conf-loc¿) _(SIGCSE 2023)_. Association for Computing Machinery, New York, NY, USA, 367–373.  https://doi.org/10.1145/3545945.3569855
  * Smith et al. (2023b) Victoria Smith, Ali Shahin Shamsabadi, Carolyn Ashurst, and Adrian Weller. 2023b.  Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey.  arXiv:2310.01424 [cs.CL]
  * Soden et al. (2024) Robert Soden, Austin Toombs, and Michaelanne Thomas. 2024.  Evaluating Interpretive Research in HCI.  _Interactions_ 31, 1 (2024), 38–42.
  * Soergel et al. (2013) David Soergel, Adam Saunders, and Andrew McCallum. 2013.  Open Scholarship and Peer Review: a Time for Experimentation.  (2013).
  * Suh et al. (2023) Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2023.  Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation.  arXiv:2310.12953 [cs.HC]
  * Tai et al. (2023) Robert H. Tai, Lillian R. Bentley, Xin Xia, Jason M. Sitt, Sarah C. Fankhauser, Ana M. Chicas-Mosier, and Barnas M. Monteith. 2023.  Use of Large Language Models to Aid Analysis of Textual Data.  _bioRxiv_ (2023).  https://doi.org/10.1101/2023.07.17.549361 arXiv:https://www.biorxiv.org/content/early/2023/07/19/2023.07.17.549361.full.pdf
  * Thirunavukarasu et al. (2023) Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023.  Large language models in medicine.  _Nature medicine_ 29, 8 (2023), 1930–1940.
  * Tholander and Jonsson (2023) Jakob Tholander and Martin Jonsson. 2023.  Design Ideation with AI - Sketching, Thinking and Talking with Generative Machine Learning Models. In _Proceedings of the 2023 ACM Designing Interactive Systems Conference_ (Pittsburgh, PA, USA) _(DIS ’23)_. Association for Computing Machinery, New York, NY, USA, 1930–1940.  https://doi.org/10.1145/3563657.3596014
  * Vitak et al. (2016) Jessica Vitak, Katie Shilton, and Zahra Ashktorab. 2016.  Beyond the Belmont Principles: Ethical Challenges, Practices, and Beliefs in the Online Data Research Community. In _Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing_ (San Francisco, California, USA) _(CSCW ’16)_. Association for Computing Machinery, New York, NY, USA, 941–953.  https://doi.org/10.1145/2818048.2820078
  * Wei et al. (2023) Jing Wei, Sungdong Kim, Hyunhoon Jung, and Young-Ho Kim. 2023.  Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data.  arXiv:2301.05843 [cs.HC]
  * Weidinger et al. (2021) Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2021.  Ethical and social risks of harm from Language Models.  arXiv:2112.04359 [cs.CL]
  * Weidinger et al. (2022) Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022.  Taxonomy of Risks Posed by Language Models. In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ (Seoul, Republic of Korea) _(FAccT ’22)_. Association for Computing Machinery, New York, NY, USA, 214–229.  https://doi.org/10.1145/3531146.3533088
  * Widder and Nafus (2023) David Gray Widder and Dawn Nafus. 2023.  Dislocated accountabilities in the “AI supply chain”: Modularity and developers’ notions of responsibility.  _Big Data & Society_ 10, 1 (2023), 20539517231177620.
  * Wu et al. (2022) Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022.  PromptChainer: Chaining Large Language Model Prompts through Visual Programming.  arXiv:2203.06566 [cs.HC]
  * Xiao et al. (2023a) Ziang Xiao, Xingdi Yuan, Q Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023a.  Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. In _Companion Proceedings of the 28th International Conference on Intelligent User Interfaces_. 75–78.
  * Xiao et al. (2023b) Ziang Xiao, Xingdi Yuan, Q. Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023b.  Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. In _Companion Proceedings of the 28th International Conference on Intelligent User Interfaces_ (Sydney, NSW, Australia) _(IUI ’23 Companion)_. Association for Computing Machinery, New York, NY, USA, 75–78.  https://doi.org/10.1145/3581754.3584136
  * Xu et al. (2023) Chen Xu, Wenjie Wang, Yuxin Li, Liang Pang, Jun Xu, and Tat-Seng Chua. 2023.  Do LLMs Implicitly Exhibit User Discrimination in Recommendation? An Empirical Study.  arXiv:2311.07054 [cs.IR]
  * Yang et al. (2023) Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu. 2023.  Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond.  arXiv:2304.13712 [cs.CL]
  * Zhang and Gao (2023) Xuan Zhang and Wei Gao. 2023.  Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method.  arXiv:2310.00305 [cs.CL]
  * Zhang et al. (2023c) Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023c.  Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models.  arXiv:2309.01219 [cs.CL]
  * Zhang et al. (2023a) Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, and Toby Jia-Jun Li. 2023a.  VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping. In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_ (, San Francisco, CA, USA,) _(UIST ’23)_. Association for Computing Machinery, New York, NY, USA, Article 5, 30 pages.  https://doi.org/10.1145/3586183.3606800
  * Zhang et al. (2023b) Zhiping Zhang, Michelle Jia, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, Tianshi Li, et al. 2023b.  ” It’s a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents.  _arXiv preprint arXiv:2309.11653_ (2023).
  * Zimmer (2010) Michael Zimmer. 2010.  “But the data is already public”: on the ethics of research in Facebook.  _Ethics and Information Technology_ 12 (2010), 313–325.  https://api.semanticscholar.org/CorpusID:24881139