* Anderson et al. (2015) Bonnie Brinton Anderson, C. Brock Kirwan, Jeffrey L. Jenkins, David Eargle, Seth Howard, and Anthony Vance. 2015.  How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/2702123.2702322
  * Anthropic (2023) Anthropic. 2023\.  Core Views on AI Safety: When, Why, What, and How.  https://www.anthropic.com/index/core-views-on-ai-safety
  * Apple (2023) Apple. 2023.  Human Interface Guidelines: Machine Learning.  https://developer.apple.com/design/human-interface-guidelines/machine-learning
  * Ashurst et al. (2022) Carolyn Ashurst, Emmie Hine, Paul Sedille, and Alexis Carlier. 2022.  Ai Ethics Statements: Analysis and Lessons Learnt from Neurips Broader Impact Statements. In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_.
  * Askell et al. (2021) Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, and Jared Kaplan. 2021\.  A General Language Assistant as a Laboratory for Alignment.  _arXiv 2112.00861_ (2021).  http://arxiv.org/abs/2112.00861
  * Auger (2013) James Auger. 2013\.  Speculative Design: Crafting the Speculation.  _Digital Creativity_ 24 (2013).
  * Ballard et al. (2019) Stephanie Ballard, Karen M. Chappell, and Kristen Kennedy. 2019.  Judgment Call the Game: Using Value Sensitive Design and Design Fiction to Surface Ethical Concerns Related to Technology. In _Proceedings of the 2019 on Designing Interactive Systems Conference_.  https://doi.org/10.1145/3322276.3323697
  * Barocas et al. (2021) Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, W. Duncan Wadsworth, and Hanna Wallach. 2021.  Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_.  https://doi.org/10.1145/3461702.3462610
  * Bellamy et al. (2018) Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. 2018\.  AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias.  _arXiv 1810.01943_ (2018).  http://arxiv.org/abs/1810.01943
  * Beretta et al. (2021) Elena Beretta, Antonio Vetrò, Bruno Lepri, and Juan Carlos De Martin. 2021.  Detecting Discriminatory Risk through Data Annotation Based on Bayesian Inferences. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3442188.3445940
  * Biddle et al. (2009) Robert Biddle, P. C. Van Oorschot, Andrew S. Patrick, Jennifer Sobey, and Tara Whalen. 2009\.  Browser Interfaces and Extended Validation SSL Certificates: An Empirical Study. In _Proceedings of the 2009 ACM Workshop on Cloud Computing Security_.  https://doi.org/10.1145/1655008.1655012
  * Birhane et al. (2022) Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022\.  Power to the People? Opportunities and Challenges for Participatory AI. In _Equity and Access in Algorithms, Mechanisms, and Optimization_.  https://doi.org/10.1145/3551624.3555290
  * Böhme and Köpsell (2010) Rainer Böhme and Stefan Köpsell. 2010.  Trained to Accept?: A Field Experiment on Consent Dialogs. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/1753326.1753689
  * Bommasani et al. (2022) Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2022.  On the Opportunities and Risks of Foundation Models.  _arXiv 2108.07258_ (2022).  http://arxiv.org/abs/2108.07258
  * Bostock et al. (2011) Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011\.  D3 Data-Driven Documents.  _IEEE TVCG_ 17 (2011).  https://doi.org/10.1109/TVCG.2011.185
  * Boyarskaya et al. (2020) Margarita Boyarskaya, Alexandra Olteanu, and Kate Crawford. 2020\.  Overcoming Failures of Imagination in AI Infused System Development and Deployment.  _arXiv 2011.13416_ (2020).  http://arxiv.org/abs/2011.13416
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using Thematic Analysis in Psychology.  _Qualitative Research in Psychology_ 3 (2006).  https://doi.org/10.1191/1478088706qp063oa
  * Brey (2012) Philip AE Brey. 2012\.  Anticipatory Ethics for Emerging Technologies.  _Nanoethics_ 6 (2012).
  * Brustoloni and Villamarín-Salomón (2007) José Carlos Brustoloni and Ricardo Villamarín-Salomón. 2007\.  Improving Security Decisions with Polymorphic and Audited Dialogs. In _Proceedings of the 3rd Symposium on Usable Privacy and Security_.  https://doi.org/10.1145/1280680.1280691
  * Buçinca et al. (2023) Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, and Saleema Amershi. 2023.  AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms.  _arXiv 2306.03280_ (2023).  http://arxiv.org/abs/2306.03280
  * Cabrera et al. (2019) Angel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, and Duen Horng Chau. 2019.  FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning. In _2019 IEEE Conference on Visual Analytics Science and Technology (VAST)_.  https://doi.org/10.1109/VAST47406.2019.8986948
  * Caraban et al. (2019) Ana Caraban, Evangelos Karapanos, Daniel Gonçalves, and Pedro Campos. 2019.  23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3290605.3300733
  * Carlini et al. (2023) Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023.  Quantifying Memorization Across Neural Language Models.  _arXiv 2202.07646_ (2023).  http://arxiv.org/abs/2202.07646
  * Carlini et al. (2021) Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021\.  Extracting Training Data from Large Language Models.  _arXiv 2012.07805_ (2021).  http://arxiv.org/abs/2012.07805
  * Chivukula et al. (2021) Shruthi Sai Chivukula, Ziqing Li, Anne C Pivonka, Jingning Chen, and Colin M Gray. 2021\.  Surveying the Landscape of Ethics-Focused Design Methods.  _arXiv preprint arXiv:2102.08909_ (2021).
  * Chouldechova (2017) Alexandra Chouldechova. 2017\.  Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments.  _Big Data_ 5 (2017).
  * Chow and Perrigo (2023) Andrew R Chow and Billy Perrigo. 2023.  The AI Arms Race Is Changing Everything.  https://time.com/6255952/ai-impact-chatgpt-microsoft-google/
  * Cohen (1968) Jacob Cohen. 1968\.  Weighted Kappa: Nominal Scale Agreement Provision for Scaled Disagreement or Partial Credit.  _Psychological Bulletin_ 70 (1968).  https://doi.org/10.1037/h0026256
  * Cohen (2013) Jacob Cohen. 2013\.  _Statistical Power Analysis for the Behavioral Sciences_ (2nd ed ed.).
  * Cooper et al. (2022) Ned Cooper, Tiffanie Horne, Gillian R Hayes, Courtney Heldreth, Michal Lahav, Jess Holbrook, and Lauren Wilcox. 2022.  A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research. In _CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3491102.3517716
  * Davani et al. (2023) Aida Mostafazadeh Davani, Mark Diaz, Dylan Baker, and Vinodkumar Prabhakaran. 2023.  Disentangling Disagreements on Offensiveness: A Cross-Cultural Study. In _The 61st Annual Meeting of the Association for Computational Linguistics_.
  * Delgado et al. (2021) Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2021\.  Stakeholder Participation in AI: Beyond ”Add Diverse Stakeholders and Stir”.  _arXiv 2111.01122_ (2021).  http://arxiv.org/abs/2111.01122
  * Delgado et al. (2023) Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023\.  The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In _Equity and Access in Algorithms, Mechanisms, and Optimization_.  https://doi.org/10.1145/3617694.3623261
  * Deng et al. (2023) Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023.  MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots.  _arXiv 2307.08715_ (2023).  http://arxiv.org/abs/2307.08715
  * Denton et al. (2021) Emily Denton, Mark Díaz, Ian Kivlichan, Vinodkumar Prabhakaran, and Rachel Rosen. 2021\.  Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation.  _arXiv 2112.04554_ (2021).  http://arxiv.org/abs/2112.04554
  * Deque (2023) Deque. 2023.  Axe DevTools: Digital Accessibility Testing Tools Dev Teams Love.  https://www.deque.com/axe/devtools/
  * Derner and Batistič (2023) Erik Derner and Kristina Batistič. 2023.  Beyond the Safeguards: Exploring the Security Risks of ChatGPT.  _arXiv 2305.08005_ (2023).  http://arxiv.org/abs/2305.08005
  * Deshpande et al. (2023) Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023\.  Toxicity in ChatGPT: Analyzing Persona-assigned Language Models.  _arXiv 2304.05335_ (2023).  http://arxiv.org/abs/2304.05335
  * Díaz et al. (2022) Mark Díaz, Ian Kivlichan, Rachel Rosen, Dylan Baker, Razvan Amironesei, Vinodkumar Prabhakaran, and Emily Denton. 2022\.  CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3534647
  * Dilhac et al. (2018) Marc-Antoine Dilhac, Christophe Abrassart, and Nathalie Voarino. 2018.  Report of the Montréal Declaration for a Responsible Development of Artificial Intelligence.  (2018).  https://doi.org/1866/27795
  * Do et al. (2023) Kimberly Do, Rock Yuren Pang, Jiachen Jiang, and Katharina Reinecke. 2023.  “That’s Important, but…”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.
  * Doteveryone (2019) Doteveryone. 2019\.  Consequence Scanning – an Agile Practice for Responsible Innovators.  https://doteveryone.org.uk/project/consequence-scanning/
  * Dovetail (2023) Dovetail. 2023.  Dovetail: All Your Customer Insights in One Place.  https://dovetail.com/
  * Dunn (1961) Olive Jean Dunn. 1961\.  Multiple Comparisons among Means.  _J. Amer. Statist. Assoc._ 56 (1961).  https://doi.org/10.1080/01621459.1961.10482090
  * Dunne and Raby (2013) Anthony Dunne and Fiona Raby. 2013.  _Speculative Everything: Design, Fiction, and Social Dreaming_.
  * Egelman et al. (2008) Serge Egelman, Lorrie Faith Cranor, and Jason Hong. 2008\.  You’ve Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/1357054.1357219
  * Egelman and Schechter (2013) Serge Egelman and Stuart Schechter. 2013.  The Importance of Being Earnest [in Security Warnings]. In _Financial Cryptography and Data Security: 17th International Conference, FC 2013, Okinawa, Japan, April 1-5, 2013, Revised Selected Papers 17_.
  * Ehsan et al. (2022) Upol Ehsan, Q Vera Liao, Samir Passi, Mark O Riedl, and Hal Daume III. 2022.  Seamful XAI: Operationalizing Seamful Design in Explainable AI.  _arXiv preprint arXiv:2211.06753_ (2022).
  * Felt et al. (2015) Adrienne Porter Felt, Alex Ainslie, Robert W. Reeder, Sunny Consolvo, Somas Thyagaraja, Alan Bettes, Helen Harris, and Jeff Grimes. 2015\.  Improving SSL Warnings: Comprehension and Adherence. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/2702123.2702442
  * Felt et al. (2014) Adrienne Porter Felt, Robert W. Reeder, Hazim Almuhimedi, and Sunny Consolvo. 2014.  Experimenting at Scale with Google Chrome’s SSL Warning. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/2556288.2557292
  * Fiannaca et al. (2023) Alexander J. Fiannaca, Chinmay Kulkarni, Carrie J Cai, and Michael Terry. 2023.  Programming without a Programming Language: Challenges and Opportunities for Designing Developer Tools for Prompt Programming. In _Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544549.3585737
  * Fiesler et al. (2020) Casey Fiesler, Natalie Garrett, and Nathan Beard. 2020\.  What Do We Teach When We Teach Tech Ethics?: A Syllabi Analysis. In _Proceedings of the 51st ACM Technical Symposium on Computer Science Education_.  https://doi.org/10.1145/3328778.3366825
  * Friedman (1996) Batya Friedman. 1996\.  Value-Sensitive Design.  _interactions_ 3 (1996).
  * Friedman and Hendry (2012) Batya Friedman and David Hendry. 2012.  The Envisioning Cards: A Toolkit for Catalyzing Humanistic and Technical Imaginations. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/2207676.2208562
  * Friedman et al. (2017) Batya Friedman, David G Hendry, Alan Borning, et al. 2017\.  A Survey of Value Sensitive Design Methods.  _Foundations and Trends® in Human–Computer Interaction_ 11 (2017).
  * Friedman et al. (2002) Batya Friedman, Peter Kahn, and Alan Borning. 2002\.  Value Sensitive Design: Theory and Methods.  _University of Washington technical report_ 2 (2002).
  * Ganguli et al. (2022) Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Scott Johnston, Andy Jones, Nicholas Joseph, Jackson Kernian, Shauna Kravec, Ben Mann, Neel Nanda, Kamal Ndousse, Catherine Olsson, Daniela Amodei, Tom Brown, Jared Kaplan, Sam McCandlish, Christopher Olah, Dario Amodei, and Jack Clark. 2022\.  Predictability and Surprise in Large Generative Models. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3533229
  * Garrett et al. (2020) Natalie Garrett, Nathan Beard, and Casey Fiesler. 2020\.  More Than ”If Time Allows”: The Role of Ethics in AI Education. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_.  https://doi.org/10.1145/3375627.3375868
  * Gebru et al. (2020) Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2020\.  Datasheets for Datasets.  _arXiv:1803.09010 [cs]_ (2020).  http://arxiv.org/abs/1803.09010
  * Gehman et al. (2020) Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020.  RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models.  _arXiv 2009.11462_ (2020).  http://arxiv.org/abs/2009.11462
  * Glaese et al. (2022) Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. 2022.  Improving Alignment of Dialogue Agents via Targeted Human Judgements.  _arXiv 2209.14375_ (2022).  http://arxiv.org/abs/2209.14375
  * Good et al. (2005) Nathaniel Good, Rachna Dhamija, Jens Grossklags, David Thaw, Steven Aronowitz, Deirdre Mulligan, and Joseph Konstan. 2005.  Stopping Spyware at the Gate: A User Study of Privacy, Notice and Spyware. In _Proceedings of the 2005 Symposium on Usable Privacy and Security - SOUPS ’05_.  https://doi.org/10.1145/1073001.1073006
  * Good et al. (2007) Nathaniel S. Good, Jens Grossklags, Deirdre K. Mulligan, and Joseph A. Konstan. 2007.  Noticing Notice: A Large-Scale Experiment on the Timing of Software License Agreements. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/1240624.1240720
  * Google (2015) Google. 2015.  Lit: Simple Fast Web Components.  https://lit.dev/
  * Google (2016) Google. 2016.  Lighthouse.  https://github.com/GoogleChrome/lighthouse
  * Google (2023a) Google. 2023a.  Google Ai Studio: Prototype with Generative AI.  https://aistudio.google.com/app
  * Google (2023b) Google. 2023b.  PaLM API: Safety Guidance.  https://developers.generativeai.google/guide/safety_guidance
  * Gordon et al. (2022) Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. 2022.  Jury Learning: Integrating Dissenting Voices into Machine Learning Models. In _CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3491102.3502004
  * Grammarly (2023) Grammarly. 2023\.  Grammarly: Free Writing AI Assistance.  https://www.grammarly.com/
  * Hanley and Durumeric (2023) Hans W. A. Hanley and Zakir Durumeric. 2023.  Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites.  _arXiv 2305.09820_ (2023).  http://arxiv.org/abs/2305.09820
  * Harrington et al. (2019) Christina Harrington, Sheena Erete, and Anne Marie Piper. 2019\.  Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements.  _Proceedings of the ACM on Human-Computer Interaction_ 3 (2019).  https://doi.org/10.1145/3359318
  * Hecht et al. (2021) Brent Hecht, Lauren Wilcox, Jeffrey P Bigham, Johannes Schöning, Ehsan Hoque, Jason Ernst, Yonatan Bisk, Luigi De Russis, Lana Yarosh, Bushra Anjum, et al. 2021\.  It’s Time to Do Something: Mitigating the Negative Impacts of Computing through a Change to the Peer Review Process.  _arXiv preprint arXiv:2112.09544_ (2021).
  * Hohman et al. (2019) Fred Hohman, Haekyu Park, Caleb Robinson, and Duen Horng Chau. 2019\.  SUMMIT: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations.  _IEEE TVCG_ (2019).  https://doi.org/10.1109/TVCG.2019.2934659
  * Holstein et al. (2019) Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé, Miro Dudik, and Hanna Wallach. 2019\.  Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3290605.3300830
  * Hong et al. (2021) Matthew K. Hong, Adam Fourney, Derek DeBellis, and Saleema Amershi. 2021.  Planning for Natural Language Failures with the AI Playbook. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3411764.3445735
  * Hong et al. (2020) Sungsoo Ray Hong, Jessica Hullman, and Enrico Bertini. 2020\.  Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs.  _Proceedings of the ACM on Human-Computer Interaction_ 4 (2020).  https://doi.org/10.1145/3392878
  * House (2022) The White House. 2022\.  Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People.  https://www.whitehouse.gov/ostp/ai-bill-of-rights
  * Hravnak et al. (2018) Marilyn Hravnak, Tiffany Pellathy, Lujie Chen, Artur Dubrawski, Anthony Wertz, Gilles Clermont, and Michael R. Pinsky. 2018.  A Call to Alarms: Current State and Future Directions in the Battle against Alarm Fatigue.  _Journal of Electrocardiology_ 51 (2018).  https://doi.org/10.1016/j.jelectrocard.2018.07.024
  * in AI et al. (2023) Organizers Of Queer in AI, Anaelia Ovalle, Arjun Subramonian, Ashwin Singh, Claas Voelcker, Danica J. Sutherland, Davide Locatelli, Eva Breznik, Filip Klubička, Hang Yuan, Hetvi J, Huan Zhang, Jaidev Shriram, Kruno Lehman, Luca Soldaini, Maarten Sap, Marc Peter Deisenroth, Maria Leonor Pacheco, Maria Ryskina, Martin Mundt, Milind Agarwal, Nyx McLean, Pan Xu, A. Pranav, Raj Korpan, Ruchira Ray, Sarah Mathew, Sarthak Arora, St John, Tanvi Anand, Vishakha Agrawal, William Agnew, Yanan Long, Zijie J. Wang, Zeerak Talat, Avijit Ghosh, Nathaniel Dennler, Michael Noseworthy, Sharvani Jha, Emi Baylor, Aditya Joshi, Natalia Y. Bilenko, Andrew McNamara, Raphael Gontijo-Lopes, Alex Markham, Evyn Dǒng, Jackie Kay, Manu Saraswat, Nikhil Vytla, and Luke Stark. 2023.  Queer In AI: A Case Study in Community-Led Participatory AI. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_.  http://arxiv.org/abs/2303.16972
  * Jiang et al. (2022) Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022.  PromptMaker: Prompt-based Prototyping with Large Language Models. In _CHI Conference on Human Factors in Computing Systems Extended Abstracts_.  https://doi.org/10.1145/3491101.3503564
  * Kaiser et al. (2021) Ben Kaiser, Jerry Wei, Eli Lucherini, Kevin Lee, J. Nathan Matias, and Jonathan Mayer. 2021\.  Adapting Security Warnings to Counter Online Disinformation. In _30th USENIX Security Symposium (USENIX Security 21)_.  https://www.usenix.org/conference/usenixsecurity21/presentation/kaiser
  * Kaur et al. (2022) Harmanpreet Kaur, Eytan Adar, Eric Gilbert, and Cliff Lampe. 2022\.  Sensible AI: Re-imagining Interpretability and Explainability Using Sensemaking Theory. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3533135
  * Kim et al. (2023) Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023\.  ProPILE: Probing Privacy Leakage in Large Language Models.  _arXiv 2307.01881_ (2023).  http://arxiv.org/abs/2307.01881
  * Kiskola et al. (2022) Joel Kiskola, Thomas Olsson, Aleksi H. Syrjämäki, Anna Rantasila, Mirja Ilves, Poika Isokoski, and Veikko Surakka. 2022.  Online Survey on Novel Designs for Supporting Self-Reflection and Emotion Regulation in Online News Commenting. In _Proceedings of the 25th International Academic Mindtrek Conference_.  https://doi.org/10.1145/3569219.3569411
  * Klassen and Fiesler (2022) Shamika Klassen and Casey Fiesler. 2022.  ” Run Wild a Little With Your Imagination” Ethical Speculation in Computing Education with Black Mirror. In _Proceedings of the 53rd ACM Technical Symposium on Computer Science Education-Volume 1_.
  * Kleinberg et al. (2016) Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016\.  Inherent Trade-Offs in the Fair Determination of Risk Scores.  _arXiv 1609.05807_ (2016).  http://arxiv.org/abs/1609.05807
  * Kluyver et al. (2016) Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian E Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica B Hamrick, Jason Grout, and Sylvain Corlay. 2016.  Jupyter Notebooks-a Publishing Format for Reproducible Computational Workflows.  2016 (2016).  https://doi.org/10.3233/978-1-61499-649-1-87
  * Koedinger et al. (2015) Kenneth R Koedinger, Jihee Kim, Julianna Zhuxin Jia, Elizabeth A McLaughlin, and Norman L Bier. 2015\.  Learning Is Not a Spectator Sport: Doing Is Better than Watching for Learning from a MOOC. In _Proceedings of the Second (2015) ACM Conference on Learning@ Scale_.
  * Krause et al. (2021) Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021.  GeDi: Generative Discriminator Guided Sequence Generation. In _Findings of the Association for Computational Linguistics: EMNLP 2021_.  https://doi.org/10.18653/v1/2021.findings-emnlp.424
  * Landis and Koch (1977) J. Richard Landis and Gary G. Koch. 1977.  The Measurement of Observer Agreement for Categorical Data.  _Biometrics_ 33 (1977).  https://doi.org/10.2307/2529310
  * Lee and Singh (2020) Michelle Seng Ah Lee and Jatinder Singh. 2020.  The Landscape and Gaps in Open Source Fairness Toolkits.  _SSRN Electronic Journal_ (2020).  https://doi.org/10.2139/ssrn.3695002
  * Li and Etchemendy (2022) Fei-Fei Li and John Etchemendy. 2022.  Annual Report 2022: Stanford Institute for Human-centered Artificial Intelligence.  https://hai-annual-report.stanford.edu
  * Li et al. (2023) Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023\.  Multi-Step Jailbreaking Privacy Attacks on ChatGPT.  _arXiv 2304.05197_ (2023).  http://arxiv.org/abs/2304.05197
  * Liao and Vaughan (2023) Q. Vera Liao and Jennifer Wortman Vaughan. 2023.  AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap.  _arXiv 2306.01941_ (2023).  http://arxiv.org/abs/2306.01941
  * Liu et al. (2022) David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, and Tina Eliassi-Rad. 2022.  Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews. In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_.  https://doi.org/10.1145/3514094.3534155
  * Liu et al. (2023) Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023.  Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment.  _arXiv 2308.05374_ (2023).  http://arxiv.org/abs/2308.05374
  * Longoni et al. (2022) Chiara Longoni, Andrey Fradkin, Luca Cian, and Gordon Pennycook. 2022.  News from Generative Artificial Intelligence Is Believed Less. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3533077
  * Lundberg and Lee (2017) Scott M. Lundberg and Su-In Lee. 2017.  A Unified Approach to Interpreting Model Predictions. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ _(NIPS’17)_.  https://doi.org/10.48550/arXiv.1705.07874
  * Madaio et al. (2022) Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. 2022\.  Assessing the Fairness of AI Systems: AI Practitioners’ Processes, Challenges, and Needs for Support.  _Proceedings of the ACM on Human-Computer Interaction_ 6 (2022).  https://doi.org/10.1145/3512899
  * Madaio et al. (2020) Michael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020.  Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3313831.3376445
  * Madiega (2021) Tambiama André Madiega. 2021\.  Artificial Intelligence Act.  _European Parliament: European Parliamentary Research Service_ (2021).  https://artificialintelligenceact.eu
  * Malinka et al. (2023) Kamil Malinka, Martin Perešíni, Anton Firc, Ondřej Hujňák, and Filip Januš. 2023\.  On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?, In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1.  _arXiv 2303.11146_.  https://doi.org/10.1145/3587102.3588827
  * Mann and Whitney (1947) H. B. Mann and D. R. Whitney. 1947.  On a Test of Whether One of Two Random Variables Is Stochastically Larger than the Other.  _The Annals of Mathematical Statistics_ 18 (1947).  https://doi.org/10.1214/aoms/1177730491
  * Matz et al. (2023) Sandra Matz, Jake Teeny, Sumer Sumeet Vaid, Gabriella M. Harari, and Moran Cerf. 2023.  _The Potential of Generative AI for Personalized Persuasion at Scale_.  Preprint. PsyArXiv.  https://doi.org/10.31234/osf.io/rn97c
  * Maurer et al. (2011) Max-Emanuel Maurer, Alexander De Luca, and Sylvia Kempe. 2011\.  Using Data Type Based Security Alert Dialogs to Raise Online Security Awareness. In _Proceedings of the Seventh Symposium on Usable Privacy and Security_.  https://doi.org/10.1145/2078827.2078830
  * McGraw and Wong (1992) Kenneth O. McGraw and S. P. Wong. 1992.  A Common Language Effect Size Statistic.  _Psychological Bulletin_ 111 (1992).  https://doi.org/10.1037/0033-2909.111.2.361
  * McGregor (2020) Sean McGregor. 2020\.  Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database.  _arXiv 2011.08512_ (2020).  http://arxiv.org/abs/2011.08512
  * McHugh (2012) Marry L. McHugh. 2012\.  Interrater Reliability: The Kappa Statistic.  _Biochemia Medica_ (2012).  https://doi.org/10.11613/BM.2012.031
  * McInnes et al. (2020) Leland McInnes, John Healy, and James Melville. 2020\.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.  _arXiv:1802.03426_ (2020).  http://arxiv.org/abs/1802.03426
  * MDN (2011) MDN. 2011.  WebGL: 2D and 3D Graphics for the Web - Web APIs.  https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API
  * MDN (2021) MDN. 2021.  Web Components - Web APIs.  https://developer.mozilla.org/en-US/docs/Web/API/Web_components
  * Merriam et al. (2002) Sharan B Merriam et al. 2002\.  Introduction to Qualitative Research.  _Qualitative research in practice: Examples for discussion and analysis_ 1 (2002).
  * Meta (2023) Meta. 2023.  Llama 2: Responsible Use Guide.  https://ai.meta.com/llama-project/responsible-use-guide
  * Miceli et al. (2020) Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020\.  Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision.  _Proceedings of the ACM on Human-Computer Interaction_ 4 (2020).  https://doi.org/10.1145/3415186
  * Microsoft (2020) Microsoft. 2020\.  Responsible AI Toolbox.  Microsoft.  https://github.com/microsoft/responsible-ai-toolbox
  * Microsoft (2022a) Microsoft. 2022a.  Harms Modeling - Azure Application Architecture Guide.  https://learn.microsoft.com/en-us/azure/architecture/guide/responsible-innovation/harms-modeling/
  * Microsoft (2022b) Microsoft. 2022b.  Microsoft Responsible AI Impact Assessment Guide.  (2022).  https://aka.ms/RAIImpactAssessmentGuidePDF
  * Mitchell et al. (2019) Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019.  Model Cards for Model Reporting. In _Proceedings of the Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3287560.3287596
  * Mostafazadeh Davani et al. (2022) Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022.  Dealing with Disagreements: Looking beyond the Majority Vote in Subjective Annotations.  _Transactions of the Association for Computational Linguistics_ 10 (2022).  https://doi.org/10.1162/tacl_a_00449
  * Munechika et al. (2022) David Munechika, Zijie J. Wang, Jack Reidy, Josh Rubin, Krishna Gade, Krishnaram Kenthapadi, and Duen Horng Chau. 2022.  Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases. In _VIS_.  https://doi.org/10.1109/VIS54862.2022.00018
  * Mylonas et al. (2013) Alexios Mylonas, Anastasia Kastania, and Dimitris Gritzalis. 2013\.  Delegate the Smartphone User? Security Awareness in Smartphone Platforms.  _Computers & Security_ 34 (2013).  https://doi.org/10.1016/j.cose.2012.11.004
  * Nanayakkara et al. (2020) Priyanka Nanayakkara, Nicholas Diakopoulos, and Jessica Hullman. 2020.  Anticipatory Ethics and the Role of Uncertainty.  _arXiv preprint arXiv:2011.13170_ (2020).
  * Nanayakkara et al. (2021) Priyanka Nanayakkara, Jessica Hullman, and Nicholas Diakopoulos. 2021.  Unpacking the Expressed Consequences of AI Research in Broader Impact Statements. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_.
  * Nori et al. (2019) Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. 2019\.  InterpretML: A Unified Framework for Machine Learning Interpretability.  _arXiv_ (2019).  http://arxiv.org/abs/1909.09223
  * Norman and Draper (1986) Donald A. Norman and Stephen W. Draper. 1986.  _User Centered System Design: New Perspectives on Human-Computer Interaction_.
  * of QueerInAI et al. (2023) Organizers of QueerInAI, Nathan Dennler, Anaelia Ovalle, Ashwin Singh, Luca Soldaini, Arjun Subramonian, Huy Tu, William Agnew, Avijit Ghosh, Kyra Yee, Irene Font Peradejordi, Zeerak Talat, Mayra Russo, and Jess de Jesus de Pinho Pinhal. 2023\.  Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms.  _arXiv 2307.10223_ (2023).  http://arxiv.org/abs/2307.10223
  * O’Neil and Gunn (2020) Cathy O’Neil and Hanna Gunn. 2020.  _Near-Term Artificial Intelligence and the Ethical Matrix_.  https://doi.org/10.1093/oso/9780190905033.003.0009
  * OpenAI (2023a) OpenAI. 2023a.  GPT-4 Technical Report.  _arXiv 2303.08774_ (2023).  http://arxiv.org/abs/2303.08774
  * OpenAI (2023b) OpenAI. 2023b.  OpenAI Playground.  https://platform.openai.com/playground
  * PAIR (2019) Google PAIR. 2019\.  People + AI Guidebook.  https://pair.withgoogle.com/guidebook
  * Pan et al. (2023) Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang. 2023\.  On the Risk of Misinformation Pollution with Large Language Models.  _arXiv 2305.13661_ (2023).  http://arxiv.org/abs/2305.13661
  * Pavlick and Kwiatkowski (2019) Ellie Pavlick and Tom Kwiatkowski. 2019.  Inherent Disagreements in Human Textual Inferences.  _Transactions of the Association for Computational Linguistics_ 7 (2019).  https://doi.org/10.1162/tacl_a_00293
  * Perez et al. (2022) Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022.  Red Teaming Language Models with Language Models.  _arXiv 2202.03286_ (2022).  http://arxiv.org/abs/2202.03286
  * Prabhakaran et al. (2023) Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo, Alicia Parrish, Alex Taylor, Mark Díaz, and Ding Wang. 2023\.  A Framework to Assess (Dis)Agreement Among Diverse Rater Groups.  _arXiv 2311.05074_ (2023).  http://arxiv.org/abs/2311.05074
  * Price et al. (2020) Thomas W. Price, Joseph Jay Williams, Jaemarie Solyst, and Samiha Marwan. 2020.  Engaging Students with Instructor Solutions in Online Programming Homework. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3313831.3376857
  * Prunkl et al. (2021) Carina EA Prunkl, Carolyn Ashurst, Markus Anderljung, Helena Webb, Jan Leike, and Allan Dafoe. 2021\.  Institutionalizing Ethics in AI through Broader Impact Requirements.  _Nature Machine Intelligence_ 3 (2021).
  * Raji et al. (2021) Inioluwa Deborah Raji, Morgan Klaus Scheuerman, and Razvan Amironesei. 2021.  You Can’t Sit With Us: Exclusionary Pedagogy in AI Ethics Education. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3442188.3445914
  * Raji et al. (2020) Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020\.  Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3351095.3372873
  * Rakova et al. (2021) Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021.  Where Responsible AI Meets Reality: Practitioner Perspectives on Enablers for Shifting Organizational Practices.  _Proceedings of the ACM on Human-Computer Interaction_ 5 (2021).  https://doi.org/10.1145/3449081
  * Rastogi et al. (2023) Charvi Rastogi, Marco Tulio Ribeiro, Nicholas King, Harsha Nori, and Saleema Amershi. 2023\.  Supporting Human-AI Collaboration in Auditing LLMs with LLMs. In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_.  https://doi.org/10.1145/3600211.3604712
  * Reeder et al. (2018) Robert W. Reeder, Adrienne Porter Felt, Sunny Consolvo, Nathan Malkin, Christopher Thompson, and Serge Egelman. 2018.  An Experience Sampling Study of User Reactions to Browser Warnings in the Field. In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3173574.3174086
  * Reingold and Tilford (1981) E.M. Reingold and J.S. Tilford. 1981.  Tidier Drawings of Trees.  _IEEE Transactions on Software Engineering_ SE-7 (1981).  https://doi.org/10.1109/TSE.1981.234519
  * Reinheimer et al. (2020) Benjamin Reinheimer, Lukas Aldag, Peter Mayer, Mattia Mossano, Reyhan Duezguen, Bettina Lofthouse, Tatiana von Landesberger, and Melanie Volkamer. 2020.  An Investigation of Phishing Awareness and Education over Time: When and How to Best Remind Users. In _Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020)_.  https://www.usenix.org/conference/soups2020/presentation/reinheimer
  * Renaud and Dupuis (2019) Karen Renaud and Marc Dupuis. 2019.  Cyber Security Fear Appeals: Unexpectedly Complicated. In _Proceedings of the New Security Paradigms Workshop_.  https://doi.org/10.1145/3368860.3368864
  * Ribeiro and Lundberg (2022) Marco Tulio Ribeiro and Scott Lundberg. 2022.  Adaptive Testing and Debugging of NLP Models. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.  https://doi.org/10.18653/v1/2022.acl-long.230
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016\.  ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_.  https://doi.org/10.1145/2939672.2939778
  * Ribeiro et al. (2020) Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020.  Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.  https://doi.org/10.18653/v1/2020.acl-main.442
  * Rismani et al. (2023) Shalaleh Rismani, Renee Shelby, Andrew Smart, Edgar Jatho, Joshua Kroll, Ajung Moon, and Negar Rostamzadeh. 2023.  From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544548.3581407
  * Robertson et al. (2023) Samantha Robertson, Zijie J. Wang, Dominik Moritz, Mary Beth Kery, and Fred Hohman. 2023\.  Angler: Helping Machine Translation Practitioners Prioritize Model Improvements. In _CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544548.3580790
  * Roy et al. (2023) Sayak Saha Roy, Krishna Vamsi Naragam, and Shirin Nilizadeh. 2023.  Generating Phishing Attacks Using ChatGPT.  _arXiv 2305.05133_ (2023).  http://arxiv.org/abs/2305.05133
  * Saleiro et al. (2019) Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T. Rodolfa, and Rayid Ghani. 2019\.  Aequitas: A Bias and Fairness Audit Toolkit.  _arXiv 1811.05577_ (2019).  http://arxiv.org/abs/1811.05577
  * Sarmiento and Wise (2022) Juan Pablo Sarmiento and Alyssa Friend Wise. 2022.  Participatory and Co-Design of Learning Analytics: An Initial Review of the Literature. In _LAK22: 12th International Learning Analytics and Knowledge Conference_.  https://doi.org/10.1145/3506860.3506910
  * Sawilowsky (2009) Shlomo S. Sawilowsky. 2009\.  New Effect Size Rules of Thumb.  _Journal of Modern Applied Statistical Methods_ 8 (2009).  https://doi.org/10.22237/jmasm/1257035100
  * Schick et al. (2021) Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021\.  Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP.  _arXiv 2103.00453_ (2021).  http://arxiv.org/abs/2103.00453
  * Schiff et al. (2020) Daniel Schiff, Bogdana Rakova, Aladdin Ayesh, Anat Fanti, and Michael Lennon. 2020.  Principles to Practices for Responsible AI: Closing the Gap.  _arXiv 2006.04707_ (2020).  http://arxiv.org/abs/2006.04707
  * Schneider et al. (2018) Christoph Schneider, Markus Weinmann, and Jan Vom Brocke. 2018\.  Digital Nudging: Guiding Online User Choices through Interface Design.  _Commun. ACM_ 61 (2018).  https://doi.org/10.1145/3213765
  * Seltman (2012) Howard J Seltman. 2012\.  _Experimental Design and Analysis_.
  * Shapiro and Wilk (1965) S. S. Shapiro and M. B. Wilk. 1965.  An Analysis of Variance Test for Normality (Complete Samples).  _Biometrika_ 52 (1965).  https://doi.org/10.1093/biomet/52.3-4.591
  * Sharevski et al. (2022) Filipo Sharevski, Amy Devine, Peter Jachim, and Emma Pieroni. 2022.  Meaningful Context, a Red Flag, or Both? Preferences for Enhanced Misinformation Warnings Among US Twitter Users. In _2022 European Symposium on Usable Security_.  https://doi.org/10.1145/3549015.3555671
  * Shelby et al. (2023) Renee Shelby, Shalaleh Rismani, Kathryn Henne, Ajung Moon, Negar Rostamzadeh, Paul Nicholas, N’Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, and Gurleen Virk. 2023.  Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_.  https://doi.org/10.1145/3600211.3604673
  * Shen et al. (2021) Hong Shen, Wesley H. Deng, Aditi Chattopadhyay, Zhiwei Steven Wu, Xu Wang, and Haiyi Zhu. 2021\.  Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3442188.3445971
  * Shevlane et al. (2023) Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Allan Dafoe. 2023.  Model Evaluation for Extreme Risks.  _arXiv 2305.15324_ (2023).  http://arxiv.org/abs/2305.15324
  * Sim et al. (2021) Kate Sim, Andrew Brown, and Amelia Hassoun. 2021.  Thinking through and Writing about Research Ethics beyond” Broader Impact”.  _arXiv preprint arXiv:2104.08205_ (2021).
  * Simmons (2023) Gabriel Simmons. 2023\.  Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)_.  https://doi.org/10.18653/v1/2023.acl-srw.40
  * Simon (2020) Guy Simon. 2020\.  OpenWeb Tests the Impact of “Nudges” in Online Discussions.  _OpenWeb Blog_ (2020).
  * Smilkov et al. (2019) Daniel Smilkov, Nikhil Thorat, Yannick Assogba, Ann Yuan, Nick Kreeger, Ping Yu, Kangyi Zhang, Shanqing Cai, Eric Nielsen, David Soergel, Stan Bileschi, Michael Terry, Charles Nicholson, Sandeep N. Gupta, Sarah Sirajuddin, D. Sculley, Rajat Monga, Greg Corrado, Fernanda B. Viégas, and Martin Wattenberg. 2019.  TensorFlow.Js: Machine Learning for the Web and Beyond.  _arXiv_ (2019).  https://arxiv.org/abs/1901.05350
  * Smith et al. (2022) Jessie J. Smith, Saleema Amershi, Solon Barocas, Hanna Wallach, and Jennifer Wortman Vaughan. 2022\.  REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3533122
  * Smith et al. (2023) Jessie J. Smith, Blakeley H. Payne, Shamika Klassen, Dylan Thomas Doyle, and Casey Fiesler. 2023\.  Incorporating Ethics in Computing Courses: Barriers, Support, and Perspectives from Educators. In _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_.  https://doi.org/10.1145/3545945.3569855
  * Solaiman and Dennison (2021) Irene Solaiman and Christy Dennison. 2021.  Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets. In _Advances in Neural Information Processing Systems_ , Vol. 34.  https://proceedings.neurips.cc/paper_files/paper/2021/file/2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf
  * Srivastava et al. (2022) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022\.  Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models.  _arXiv preprint arXiv:2206.04615_ (2022).  http://arxiv.org/abs/2206.04615
  * Sturdee et al. (2021) Miriam Sturdee, Joseph Lindley, Conor Linehan, Chris Elsden, Neha Kumar, Tawanna Dillahunt, Regan Mandryk, and John Vines. 2021\.  Consequences, Schmonsequences! Considering the Future as Part of Publication and Peer Review in Computing Research. In _Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems_.
  * Suresh and Guttag (2021) Harini Suresh and John Guttag. 2021.  A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. In _Equity and Access in Algorithms, Mechanisms, and Optimization_.  https://doi.org/10.1145/3465416.3483305
  * Tabassi (2023) Elham Tabassi. 2023\.  _AI Risk Management Framework: AI RMF (1.0)_.  Technical Report. National Institute of Standards and Technology.  https://doi.org/10.6028/NIST.AI.100-1
  * Team et al. (2023) Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023\.  Gemini: A Family of Highly Capable Multimodal Models.  _arXiv preprint arXiv:2312.11805_ (2023).  https://arxiv.org/abs/2312.11805
  * Tenney et al. (2020) Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020.  The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models. In _EMNLP Demo_.  https://doi.org/10.18653/v1/2020.emnlp-demos.15
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023\.  Llama 2: Open Foundation and Fine-Tuned Chat Models.  _arXiv 2307.09288_ (2023).  https://arxiv.org/abs/2307.09288
  * Varanasi and Goyal (2023) Rama Adithya Varanasi and Nitesh Goyal. 2023.  “It Is Currently Hodgepodge”: Examining AI/ML Practitioners’ Challenges during Co-Production of Responsible AI Values. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.
  * Vilar et al. (2006) David Vilar, Jia Xu, Luis Fernando D’Haro, and Hermann Ney. 2006\.  Error Analysis of Statistical Machine Translation Output. In _Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)_.  http://www.lrec-conf.org/proceedings/lrec2006/pdf/413_pdf.pdf
  * Wang et al. (2023b) Qiaosi Wang, Michael Madaio, Shaun Kane, Shivani Kapania, Michael Terry, and Lauren Wilcox. 2023b.  Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544548.3581278
  * Wang et al. (2024) Zijie J. Wang, Aishwarya Chakravarthy, David Munechika, and Duen Horng Chau. 2024.  Wordflow: Social Prompt Engineering for Large Language Models.  _arXiv 2401.14447_ (2024).  http://arxiv.org/abs/2401.14447
  * Wang et al. (2022a) Zijie J. Wang, Katie Dai, and W. Keith Edwards. 2022a.  StickyLand: Breaking the Linear Presentation of Computational Notebooks. In _CHI Conference on Human Factors in Computing Systems Extended Abstracts_.  https://doi.org/10.1145/3491101.3519653
  * Wang et al. (2023a) Zijie J. Wang, Fred Hohman, and Duen Horng Chau. 2023a.  WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)_.  https://aclanthology.org/2023.acl-demo.50
  * Wang et al. (2022b) Zijie J. Wang, Alex Kale, Harsha Nori, Peter Stella, Mark E. Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, and Rich Caruana. 2022b.  Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ _(KDD ’22)_.  https://doi.org/10.1145/3534678.3539074
  * Wang et al. (2022c) Zijie J. Wang, David Munechika, Seongmin Lee, and Duen Horng Chau. 2022c.  NOVA: A Practical Method for Creating Notebook-Ready Visual Analytics.  _arXiv:2205.03963_ (2022).  http://arxiv.org/abs/2205.03963
  * Weerts et al. (2023) Hilde Weerts, Miroslav Dudík, Richard Edgar, Adrin Jalali, Roman Lutz, and Michael Madaio. 2023\.  Fairlearn: Assessing and Improving Fairness of AI Systems.  _arXiv 2303.16626_ (2023).  http://arxiv.org/abs/2303.16626
  * Weidinger et al. (2021) Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2021.  Ethical and Social Risks of Harm from Language Models.  _arXiv 2112.04359_ (2021).  https://doi.org/10.48550/arXiv.2112.04359
  * Weidinger et al. (2023) Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William Isaac. 2023.  Sociotechnical Safety Evaluation of Generative AI Systems.  _arXiv 2310.11986_ (2023).  http://arxiv.org/abs/2310.11986
  * Weidinger et al. (2022) Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022.  Taxonomy of Risks Posed by Language Models. In _2022 ACM Conference on Fairness, Accountability, and Transparency_.  https://doi.org/10.1145/3531146.3533088
  * Weiser and Schweber (2023) Benjamin Weiser and Nate Schweber. 2023.  The ChatGPT Lawyer Explains Himself.  _The New York Times_ (2023).  https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html
  * Welbl et al. (2021) Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021\.  Challenges in Detoxifying Language Models. In _Findings of the Association for Computational Linguistics: EMNLP 2021_.  https://doi.org/10.18653/v1/2021.findings-emnlp.210
  * Wexler et al. (2019) James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Viegas, and Jimbo Wilson. 2019.  The What-If Tool: Interactive Probing of Machine Learning Models.  _TVCG_ 26 (2019).  https://doi.org/10.1109/TVCG.2019.2934619
  * Whittaker (2021) Meredith Whittaker. 2021\.  The Steep Cost of Capture.  _Interactions_ 28 (2021).  https://doi.org/10.1145/3488666
  * Wong and Khovanskaya (2018) Richmond Y Wong and Vera Khovanskaya. 2018.  _Speculative Design in HCI: From Corporate Imaginations to Critical Orientations_.
  * Wong et al. (2023) Richmond Y. Wong, Michael A. Madaio, and Nick Merrill. 2023\.  Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics.  _Proceedings of the ACM on Human-Computer Interaction_ 7 (2023).  https://doi.org/10.1145/3579621
  * Wong and Nguyen (2021) Richmond Y Wong and Tonya Nguyen. 2021.  Timelines: A World-Building Activity for Values Advocacy. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_.
  * Wright et al. (2021) Austin P. Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Duen Horng (Polo) Chau, and Diyi Yang. 2021\.  RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization.  _Proceedings of the ACM on Human-Computer Interaction_ 5 (2021).  https://doi.org/10.1145/3449280
  * Wright et al. (2020) Austin P. Wright, Zijie J. Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Mennatallah El-Assady, Alex Endert, Daniel Keim, and Duen Horng Chau. 2020.  A Comparative Analysis of Industry Human-AI Interaction Guidelines.  http://arxiv.org/abs/2010.11761
  * Wu et al. (2006) Min Wu, Robert C. Miller, and Simson L. Garfinkel. 2006\.  Do Security Toolbars Actually Prevent Phishing Attacks?. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/1124772.1124863
  * Wu et al. (2019) Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2019.  Errudite: Scalable, Reproducible, and Testable Error Analysis. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.  https://doi.org/10.18653/v1/P19-1073
  * Wu et al. (2022) Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022\.  AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In _CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3491102.3517582
  * Xu et al. (2021) Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. 2021\.  Detoxifying Language Models Risks Marginalizing Minority Voices. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.  https://doi.org/10.18653/v1/2021.naacl-main.190
  * Yildirim et al. (2023) Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. 2023\.  Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544548.3580900
  * Zamfirescu-Pereira et al. (2023) J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023.  Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_.  https://doi.org/10.1145/3544548.3581388

##  Appendix A Co-design User Study Interviews

###  A.1. Co-design Prototypes and Sketches

Fig. S1.  To evaluate our early Farsight designs and generate more design
ideas, we conducted a co-design study (§ 3.1) with 10 AI prototypers.
Participants were asked to use our very early-stage design prototypes (shown
in cells labeled with  ) to envision potential harms associated with their
application while thinking aloud. Participants were also presented with low-
fidelity sketches for our other design ideas (shown in cells in the last row).
The ratings of our design ideas are shown in Fig. 4.

This image is a composite of various interface designs and sketches used in
the co-design study. The interfaces are aimed at helping users identify and
manage potential harms associated with AI applications. At the top left, there
are panels titled ”Latest AI Incidents,” ”Related AI Incidents,” and ”AI-
generated use cases,” each providing a list or method to input relevant AI
incidents and use cases. Adjacent to these are panels for ”AI-generated
harms,” ”Add/edit harms,” and ”AI-generated stakeholders,” for documenting and
editing potential AI-related harms and stakeholders.

Central to the image is a ”Tree visualization” showing an interactive node-
link diagram that connects different aspects such as use cases, harms, and
stakeholders. To the right are panels titled ”Harm summary” and ”Use case
sidebar,” providing summarized information and detailed exploration of use
cases.

At the bottom, there are additional elements such as a ”Clippy-like assistant”
offering help, an ”Environment impact” panel showing estimated CO2 emissions,
a ”Risk symbol” indicating the level of risk associated with the AI, and an
”Export and share harms” feature for dissemination of findings.

###  A.2. Interview 1 Questions

  * •

Why do you use a prompt crafting tool?

  * •

How have you used it most recently? Can you walk me through one of your
example prompts?

  * •

For your previous prompts, did you ever think about the potential societal
impacts of your AI application / feature?

    * –

It’s OK if this wasn’t part of your process.

    * –

If yes, how did you think through or envision those potential impacts of your
AI prototypes / ideas?

    * –

If yes, can you share some examples of the types of impacts of your AI
application that you considered? Including both positive and negative impacts.

    * –

If no, how would you envision potential impacts of your AI prototypes / ideas?

  * •

What would motivate you to think more about the potential negative impacts of
the applications you were prototyping?

###  A.3. Interview 2 Rating Forms and Questions

  * •

How do you think our design might fit into your prompting workflow?

    * –

How do you think our design might fit into your typical AI application
prototyping workflow?

  * •

What would these suggested use cases, stakeholders, and harms prompt you to
think or do differently? (if not answered already)

  * •

What would prevent you from using this tool?

  * •

Other design ideas?

    * –

What would encourage more use/engagement with the tool?

    * –

What are other ways you could imagine raising awareness of potential
responsible AI and safety issues?

##  Appendix B Interface Details

###  B.1. Determine the Thresholds for Relevancy of AI Incident Reports

We collect 1000 random internal prompts written by real AI prototypers. Then,
we compute the embedding similarity between these prompts and all AI incident
reports (McGregor, 2020). We use the 20% and 70% cumulative density function
cut-offs (0.69 and 0.75) of the max prompt-incident embedding distance as our
thresholds for  ,  ,  . Researchers can easily adjust these two thresholds
(bounded between 0 and 1) to calibrate an article’s relevancy.

Fig. S2. A visualization of the PaLM 2 embeddings of 3,474 AI incident reports
(McGregor, 2020) (blue dots and contour) and 153 Awesome ChatGPT Prompts
(Akın, 2022) (red dots and contour). The embeddings’ dimensions were reduced
from 768 to 2 using UMAP (McInnes et al., 2020) with default parameters. The
rectangles and labels show the summaries of AI incident reports in high-
density embedding neighborhoods. The summaries are automatically generated by
WizMap (Wang et al., 2023a). The visualization reveals different clusters in
the AI incident reports, such as incidents related to autonomous driving cars
in the bottom left and machine translation on the right. The overlap between
the red and blue contours indicates that user prompts can be in close
proximity to AI incident reports in the 2D embedding space. This observation
inspires us to use high-dimensional embedding similarities to calculate the
alert levels in Farsight (§ 4.1). Note that in this example, the 153 user
prompts form a cluster due to the primary focus of AwesomeGPT prompts on
conversational agents. The distribution of our 1,000 internal prompts
(featuring classification, translation, code generation, etc.) is more spread
out. For an interactive version of this visualization, visit WizMap.

A figure shows the embedding visualization of 3,474 AI incident reports and
153 user prompts.

###  B.2. Example Farsight Output

Table S1. Farsight uses LLMs to generate use cases, stakeholders, and harms associated with AI apps based on a given prompt. This table shows the outputs of Farsight for a toxicity classifier prompt (§ C.2). By default, the Harm Envisioner displays six use cases (arranged by two  , two  , and two  ), five stakeholders (arranged by three direct stakeholders and two indirect stakeholders) for each use case, and one harm for each stakeholder. Users can modify or delete existing outputs and ask Farsight to generate additional content (e.g., generating harms for a newly added stakeholder). This table shows the outputs of Farsight for a toxicity classifier prompt. Three columns are ”Use Cases”, ”Stakeholders”, and ”Harms.” |  |
---|---|---
Use Cases |  Stakeholders |  Harms
Online moderators use it to remove toxic content from social media.  |  Online moderators |  Online moderators may be exposed to toxic content while using the AI tool.
Online moderators may feel frustrated or anxious about the accuracy of the AI
tool.
Online moderators may lose their jobs due to AI-generated content being
offensive.
Social media users |  Social media users may be unfairly censored by the AI tool.
Social media users may feel frustrated or anxious about the accuracy of the AI
tool.
Social media users may feel like they are not being heard by the moderators.
The company |  The company may be sued for defamation if the AI tool mislabels a post as toxic.
The company may lose customers if the AI tool is not accurate enough.
The company may be accused of bias if the AI tool is not fair in its
classification of toxic content.
Social media users who do not use this AI product |  Social media users who do not use this AI product may be exposed to toxic content.
Social media users who do not use this AI product may feel like they are not
being heard by the moderators.
Social media users who do not use this AI product may feel frustrated or
anxious about the amount of toxic content on social media.
Social media advertisers |  Social media advertisers may lose revenue due to toxic content being removed from social media.
Social media advertisers may have to spend more time and money on creating
non-toxic content.
Social media advertisers may feel frustrated or anxious about the accuracy of
the AI tool.
Customer service agents use it to identify abusive customers.  |  Customer service agents |  Customer service agents may feel stressed or anxious about the accuracy of the AI tool.
Customer service agents may be accused of bias if the AI tool misidentifies
abusive customers.
Customer service agents may lose the skill to identify abusive customers
independently.
Abusive customers |  Abusive customers may be denied service due to being labeled as toxic.
Abusive customers may feel frustrated or anxious about being labeled as toxic.
Abusive customers may be more likely to engage in abusive behavior in the
future.
The company |  The company may be sued for defamation if the AI tool mislabels a customer as abusive.
The company may face increased regulatory scrutiny if the AI tool is found to
be inaccurate or biased.
The company may lose customers if the AI tool is perceived as biased against
certain groups.
Customers who are not abusive |  Customers who are not abusive may be misidentified as abusive and treated poorly by customer service agents.
Customers who are not abusive may feel frustrated or anxious about being
misidentified as abusive.
Customers who are not abusive may lose trust in customer service agents.
Customers who are misidentified as abusive |  Customers who are misidentified as abusive may be denied service or treated poorly by customer service agents.
Customers who are misidentified as abusive may feel frustrated or anxious
about being treated poorly.
Customers who are misidentified as abusive may lose trust in customer service
agents.
|  Law enforcement |  Law enforcement may make false arrests or detentions of innocent people due to AI-generated toxicity labels.
|  Law enforcement may lose trust in AI tools due to false positives and
negatives.
|  Law enforcement may be biased against certain groups of people due to AI-
generated toxicity labels.
Law enforcement uses it to identify potential terrorists.  |  Potential terrorists |  Potential terrorists may be unfairly targeted by law enforcement due to AI-generated misclassifications.
Potential terrorists may feel frustrated or anxious about being unfairly
targeted by law enforcement.
Potential terrorists may be denied opportunities due to AI-generated
misclassifications.
Victims of terrorism |  Victims of terrorism may be re-traumatized by AI-generated content that glorifies or incites violence against them.
Victims of terrorism may feel unsafe or threatened by AI-generated content
that glorifies or incites violence against them.
Victims of terrorism may feel like their voices are not being heard by
society.
Civil rights groups |  Civil rights groups may be unfairly targeted by law enforcement due to AI-generated false positives.
Civil rights groups may lose trust in law enforcement due to AI-generated
false positives.
Civil rights groups may have to spend more time and resources fighting against
AI-generated false positives.
People who are falsely identified as potential terrorists |  People who are falsely identified as potential terrorists may be subject to increased surveillance, harassment, and discrimination.
People who are falsely identified as potential terrorists may lose their jobs,
housing, or other opportunities.
People who are falsely identified as potential terrorists may be detained or
imprisoned without due process.
Military uses it to identify potential insurgents.  |  Military |  Military may use AI to target and harm innocent civilians.
Military may lose trust from the public due to its use of AI.
Military may be pressured to use AI to stay competitive with other militaries.
Insurgency |  Insurgency may be able to evade detection by using AI-generated text that is not classified as toxic.
Insurgency may be able to recruit more members by using AI-generated text that
is not classified as toxic.
Insurgency may be able to spread propaganda more effectively by using AI-
generated text that is not classified as toxic.
Governments |  Governments may use AI to target and surveil innocent people.
Governments may use AI to justify violence against innocent people.
Governments may lose control over how AI is used to target and surveil people.
Civilians who are wrongly identified |  Civilians who are wrongly identified may be subject to violence or discrimination.
Civilians who are wrongly identified may lose their freedom or property.
Civilians who are wrongly identified may experience emotional distress.
Civilians who are not identified as insurgents |  Civilians who are not identified as insurgents may be subject to violence or discrimination.
Civilians who are not identified as insurgents may lose their freedom or
property.
Civilians who are not identified as insurgents may experience emotional
distress.
|  Hate groups |  Hate groups may be able to recruit more people due to the AI tool.
|  Hate groups may be able to spread their ideology more effectively due to
the AI tool.
|  Hate groups may be able to avoid detection by law enforcement due to the AI
tool.
|  Potential recruits |  Potential recruits may be exposed to toxic content that could lead to radicalization.
|  Potential recruits may feel alienated and isolated from society due to
their exposure to toxic content.
|  Potential recruits may lose the opportunity to develop healthy
relationships with people outside of the hate group.
Hate groups use it to identify potential recruits.  |  The company |  The company may be held liable for hate speech generated by its AI product.
The company may face increased regulation due to its AI product’s misuse.
The company may lose customers due to negative publicity.
People who are targeted by hate groups |  People who are targeted by hate groups may be harassed or threatened by hate groups.
People who are targeted by hate groups may feel anxious or fearful of being
targeted again.
People who are targeted by hate groups may lose their sense of safety and
belonging.
People who are not targeted by hate groups |  People who are not targeted by hate groups may be exposed to toxic content.
People who are not targeted by hate groups may be harassed or threatened by
hate groups.
People who are not targeted by hate groups may feel anxious or fearful about
the rise of hate groups.
Scammers use it to identify potential victims.  |  Victims of scams |  Victims of scams may lose money or personal information.
Victims of scams may experience emotional distress.
Victims of scams may lose trust in others.
Scammers |  Scammers may be able to target more vulnerable people with their scams.
Scammers may be able to avoid detection by using AI-generated text that is
less likely to be flagged as toxic.
Scammers may be able to increase their profits by using AI-generated text to
target more people.
The company |  The company may be held liable for the harm caused by scammers using the AI tool.
The company may lose customers due to negative publicity.
The company may face increased regulatory scrutiny.
Victim’s financial institutions |  Victim’s financial institutions may lose money due to scammers using AI to identify potential victims.
Victim’s financial institutions may face increased legal liability due to
scammers using AI to identify potential victims.
Victim’s financial institutions may lose trust from their customers due to
scammers using AI to identify potential victims.
Family and friends of the victim |  Family and friends of the victim may be harmed by the scammer’s actions.
Family and friends of the victim may be harmed by the scammer’s emotional
abuse.
Family and friends of the victim may be harmed by the scammer’s financial
abuse.

##  Appendix C Evaluation User Study Prompts

###  C.1. Pre-task: Email Summarizer (H1)

AI feature functionality: Shorten and improve a user’s email

Context: Your team is adding a new AI feature to an email client

Text prompt:

⬇

You are an expert in writing emails. Rewrite the given emails to make them
shorter and better.

Email: "Good morning, Alex. I wanted to write you this note because I think we
should schedule a follow-up meeting. I am thinking we could meet next week,
would that work for you? I think the deck is looking solid but I have some
thoughts on it that I would like to share with you."

Rewrite: "Morning. Let’s schedule a follow-up. Next week works? Deck’s solid,
but I have thoughts."

Email: "Alex, soooo hey, I think the current prototype isn’t working so well
unfortunately. Ok so how about this as a proposal for a plan: I think we
should schedule a meeting to chat early next week. And during that time we can
try to debug it live together. What do you think? Nicole"

Rewrite: [AI output]

###  C.2. Intervention: Toxicity Classifier (H2)

AI feature functionality: Detect toxic text content

Context: Your team is developing a social media content moderation tool (e.g.,
API service)

Text prompt:

⬇

Classify if the input sentence is toxic or non-toxic.

input: I will remove all your organs in alphabetical order.

output: toxic

input: Why is it that every Walmart worker chomps on their gum like a cow.

output: toxic

input: He is not living proof that gay conversion therapy works.

output: [AI output]

Prompt engineering: Participants were asked to modify the prompt to alter the
model’s prediction on the test input sentence: “He is not living proof that
gay conversion therapy works.” from toxic to non-toxic. We intentionally
formulated the initial prompt to make the LLM predict the given test input
sentence as toxic, as we exclusively included toxic examples in the prompt.

###  C.3. Post-task: Article Summarizer (H3)

AI feature functionality: Summarize an article in one sentence

Context: Your team is adding an AI feature to a text editor software

Text prompt:

⬇

I want you to act as a text summarizer. You will summarize my article in one
sentence:

Article: Eighty years ago today - a powerful explosion rocked the small
community of Dawson Creek on February 13th, 1943. The blast reached Spirit
River with enough strength left to rattle dishes in cupboards. In 1943, the
American Army was in Dawson Creek (population 500), working on the Alaska
Highway. The old town was surrounded by makeshift barracks and storage
warehouses for construction equipment. One, a livery barn in the centre of the
commercial block, housed thousands of miles of copper wire, kegs of nails,
spikes, cross-arm braces, hammers, crowbars, tires, and other assorted tools.
Two hundred cases of percussion caps and a truckload of dynamite were also
stored in the handy location. Somehow a fire started. The inevitable explosion
followed soon after, incinerating whatever was in the core zone and blowing
people off their feet in expanding circles throughout the town. Fire spread
rapidly and eventually only one building remained standing - the Co-op store -
but it was wrecked and looted. Dorthea Calverley has written a fascinating
personal account of the disaster.

Summary: A powerful explosion in Dawson Creek on February 13th, 1943 destroyed
most of the town.

###  C.4. Alternative: Math Tutor (H4)

AI feature functionality: Answer math-related questions.

Context: Your team is developing a math tutoring app (e.g., mobile app)

Text prompt

⬇

You are a great math teacher. Given some mathematical equations or concepts,
your will explain them in easy-to-understand terms. You will provide more than
the basic definition and help the student to learn the underlying reasons.

User: What’s the upper bound and lower bound of sin()?

Teacher: The upper bound of sin(x) is 1, and the lower bound is -1. This is
because sin(x) is a periodic function with a period of 2pi, meaning that it
repeats itself every 2pi radians. The maximum value of sin(x) occurs when x =
pi/2, and the minimum value occurs when x = -pi/2.

Prompt Engineering: During prompt engineering, participants were asked to
modify the prompt so that when a user posed non-math-related questions, the
math tutor app’s response would be “Sorry, I’m not sure about the answer to
this question. Try a different question.”

##  Appendix D Evaluation User Study Interviews

###  D.1. Interview 1 Questions

  * •

How did the harm envisioning tool (Farsight, Farsight Lite, or Envisioning
Guide) influence your strategy for envisioning harms?

  * •

Is envisioning potential harms something you would typically do when writing
prompts? If so, how do you typically do it?

  * •

What are the most relevant or important harms you have identified? Why?

  * •

What is the biggest challenge in envisioning harms?

  * •

What other tools or other types of support would you like to have to help you
envision harms when prototyping LLM-powered applications?

  * •

What do you think you would do about those identified harms?

  * •

Do you have any feedback on the harm envisioning tool (Farsight, Farsight
Lite, or Envisioning Guide)?

###  D.2. Interview 2 Rating Forms and Questions

[For participants in $C_{FG}$ and $C_{GF}$]

How would you rate Farsight?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

Help me envision harms.

  * •

Easy to use.

  * •

Enjoyable to use.

  * •

I would use this tool in the future.

How would you rate the helpfulness of Farsight’s different components?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

AI incidents.

  * •

Use case sidebar.

  * •

AI assistance in harm envisioning.

  * •

Interactive tree visualization.

  * •

Customizability (add, edit, delete content).

[For participants in $C_{LG}$ and $C_{GL}$]

How would you rate Farsight Lite?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

Help me envision harms.

  * •

Easy to use.

  * •

Enjoyable to use.

  * •

I would use this tool in the future.

How would you rate the helpfulness of Farsight Lite’s different components?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

AI incidents.

  * •

Use case sidebar.

  * •

AI assistance in harm envisioning.

[For participants in $C_{FG}$, $C_{GF}$, $C_{LG}$, and $C_{GL}$]

How would you rate Envisioning Guide?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

Help me envision harms.

  * •

Easy to use.

  * •

Enjoyable to use.

  * •

I would use this tool in the future.

How would you rate the helpfulness of Envisioning Guide’s different
components?

Participants could select strongly agree, agree, neutral, disagree, or
strongly disagree. The default selection is neutral.

  * •

Harm modeling workflow table.

  * •

Text prompts to think about use cases, harms, and stakeholders.

  * •

Harm taxonomy.

[For participants in $C_{FG}$, $C_{GF}$, $C_{LG}$, and $C_{GL}$]

Overall, which tool do you prefer? Why?

Participants in $C_{FG}$, $C_{GF}$ can choose Farsight or Envisioning Guide.
Participants in $C_{LG}$, and $C_{GL}$ can choose Farsight Lite or Envisioning
Guide.

##  Appendix E Harm Rating Processing and Inter-rater Reliability

Fig. S3.  We compute weighted pairwise Cohen’s kappa to measure inter-rater
reliability for the ratings of harm likelihood (left) and harm severity
(right). Raters rate each dimension on a 4-point Likert scale (strongly agree,
agree, disagree, and strongly disagree). We numericalized these four
categories as ordinal data: 1, 2, 3, 4. Within each cell, the top number is
the kappa score, and the bottom number is the count of common harms between
the corresponding two raters. The average kappas for likelihood and severity
ratings are 0.11 and 0.10, which can be interpreted as slight agreement.

This image contains two matrices titled ”Inter-rater Agreement on Harm
Likelihoods” and ”Inter-rater Agreement on Harm Severities.” Each matrix
displays the Cohen’s kappa statistic between pairs of raters, which measures
the agreement between them beyond chance. The matrices are color-coded: blue
indicates positive agreement, red indicates negative agreement, and white
represents neutral.

In our evaluation user study, we recruited seven raters to rate the likelihood
and severity of each collected harm (§ 6.2.4). In total, we collected 989
harms with 895 unique harms (LABEL:table:participant-harm). We randomly assign
each unique harm to three raters to rate its likelihood and severity on a
4-point Likert scale (strongly agree, agree, disagree, and strongly disagree
to statements “this harm is likely to happen” and “this harm is severe”).
Raters could also choose an N/A option if they perceived a rating was not
applicable. After collecting all ratings, we dropped all N/As and
numericalized four rating categories as ordinal scores: 1, 2, 3, 4.

To measure the inter-rater reliability, we computed Cohen’s kappa (McHugh,
2012) for each pair of raters. As the rating scores are ordinal, we used
quadratically weighed kappa (Cohen, 1968), so that the level of agreement
between score 1 and score 2 is higher than score 1 and score 3. The pair-wise
kappas and the count of common harms are shown in Fig. S3. The average kappa
for likelihood ratings is 0.14, and the average kappa for severity ratings is
0.09. Both scores can be interpreted as “slight agreement” (Landis and Koch,
1977).

###  E.1. Example Harms Collected from Participants

Table S2. A random subset ($n$=84) of 895 unique harms collected from 42 participants in our evaluation user study (§ 6). This random subset includes 14 harms for each of the four AI features (four prompts used in Harm Envisioning Activities–Harm Envisioning Activities). Depending on the experimental conditions, each harm was envisioned by a participant (Harm Envisioning Activities–Harm Envisioning Activities) or generated by Farsight and curated by a participant (Harm Envisioning Activities, Harm Envisioning Activities). Each harm was rated by three random raters out of seven raters in terms of likelihood and severity on a 4-point Likert scale (strongly agree, agree, disagree, and strongly disagree to statements “this harm is likely to happen” and “this harm is severe”). This table shows the harms collected from participants. The columns are ”AI Feature”, ”Who (could be harmed)”, ”How (the harm could happend)”, ”Likelihood”, and ”Severity.” |  |  |  |
---|---|---|---|---
AI Feature |  Who (could be harmed) |  How (the harm could happen) |  Likelihood |  Severity
Email summarizer (Harm Envisioning Activities) |  AI company |  AI company’s reputation may be harmed due to inaccurate summary |  3, 3, 3 |  2, 2, 3
|  Author |  The AI summary might lose context and information, critical context. It could cut out some crucial part of the email. |  4, 3, 3 |  3, 3, 3
|  Owner of the email |  Rewrite could leak data from other users’ email. |  3, 1, 2 |  4, 4, 2
|  People whose names / personal information is lost in the summarization |  Potential data loss. The email could be very long, and the rewrite could miss some context (e.g., full name). The rewrite can cause confusion to the receivers. |  3, 3, 3 |  2, 3, 3
|  Product team, company |  Prompt injection attack. |  3, 1, 3 |  2, 1, 3
|  Receiver |  Omission of vital information and context. The feature can miss some very important information. |  3, 3, 3 |  3, 2, 3
|  Receiver |  The initial rewrite is a bit open ended, which might cause anxiety. |  2, 3, 2 |  1, 3, 2
|  Recipients |  If users can control the temperature of the LLM, they can generate more inaccurate emails. |  1, 3, 1 |  4, 2, 1
|  Recipients due to loss of productivity and insight into their orgs metrics |  The email may miss some metrics from the email. |  3, 3, 3 |  2, 2, 3
|  Scam victims |  Scam emails are very common. Mass emailing a lot of people with easy automation can cause harm to victims. |  2, 3, 3 |  4, 3, 3
|  Sender |  In a professional setting, if the feature is not sufficient to capture the keywords, the summary can sound unprofessional. It can harm the reputation of the sender. |  2, 3, 3 |  2, 2, 2
|  Sender |  The summary is inaccurate, e.g., having the wrong time, names in the rewrite. It can harm the sender’s reputation. |  3, 3, 3 |  2, 3, 3
|  Sender, Recipient |  The feature introduces incorrect information that is not in the original email. |  3, 3, 3 |  3, 2, 2
|  Sender, recipient |  Quality of the communication. If the tone of the rewrite is different from the original email, it can cause harm to both sender and recipient. Miscommunication. |  3, 2, 3 |  2, 2, 3
|  Sender, recipient |  The delicacy in the original email may be removed in the shorter email. The tone, emotion, and human elements may be changed. |  4, 3, 3 |  3, 2, 3
|  Sender, recipient |  The summary can have a different voice / style from the email (more directly, less soft). It can be perceived as negative. |  4, 3, 4 |  3, 2, 4
|  Sender, recipient, company relationships |  AI can be culturally blind. It might not follow some typical email norm in the rewrite. |  3, 3, 3 |  2, 2, 3
|  Senders, receivers |  The AI feature puts words on other people’s behalf. It can have different viewpoints from the email. It can have misrepresented views. |  2, 3, 4 |  3, 3, 4
|  Senders, recipient |  The summary can lose some tone, voice of the email, becoming less personal. |  3, 3, 4 |  2, 3, 3
|  Stakeholders of the decision |  If the communication is about decision making and the rewrite contains error, it can harm the stakeholders of the decision. |  1, 4, 3 |  1, 4, 3
|  Team building the feature |  Damage to reputation, credibility to have a feature fail this way. |  3, 2, 3 |  3, 1, 2
Toxicity classifier (Harm Envisioning Activities) |  Advertisers |  If the AI feature fails, ads might be present next to toxic content, losing revenues. |  3, 3, 3 |  1, 2, 3
|  Bullies |  Teachers use it to identify students being bullied. The AI may make the teachers be biased against the bullies. |  2, 3, 2 |  3, 3, 2
|  Company of the AI product |  The company may lose users due to the presence of more moderation. |  3, 2, 1 |  4, 2, 1
|  Customer |  Customer service agents use it to identify abusive customers. Customers may lose the opportunity to get help from customer service agents. |  3, 3, 3 |  3, 2, 4
|  Customer service agents |  If customer service uses it to screen people, false positives: exposing customer service agents to toxic customers. |  3, 1, 3 |  3, 1, 3
|  Employees of the company |  HR departments use it to screen job applicants for toxic behavior. Employees may be unfairly rejected for jobs due to AI-generated toxicity labels. |  1, 3, 3 |  1, 3, 4
|  End user (children & parents) |  Mislabeled outputs will not be blocked by parental controls. |  3, 3, 3 |  3, 4, 4
|  Government |  Military uses it to identify potential insurgents. Governments may use AI to target and surveil innocent people, leading to human rights violations. |  3, 2, 3 |  4, 3, 3
|  Hate group targets |  Hate groups use it to identify potential recruits. Hate groups may be able to recruit more people due to the AI tool. |  3, 2, 3 |  3, 3, 3
|  Job applicants |  Reddit has a karma score. Similarly, if a social media uses this feature to prioritize non-toxic content, misclassification can lose job opportunities for social media users (e..g, tweet not seen by companies). |  3, 4, 1 |  3, 3, 1
|  Online moderators |  Online moderators may lose their jobs because the AI tool can already distinguish toxic vs non-toxic so there is no need for a moderator. |  2, 1, 3 |  3, 1, 4
|  Online moderators |  Online moderators may miss some toxic content and post it on social media. |  3, 3, 3 |  3, 4, 3
|  People being targeted by law enforcement |  Law enforcement uses it to identify potential terrorists. Potential terrorists ([different name]) may be unfairly targeted by law enforcement due to AI-generated misclassifications. |  3, 1, 3 |  3, 1, 4
|  People who are not members of hate groups |  Hate groups use it to identify potential recruits. People who are not members of hate groups may be affected by increased hate group activity. |  3, 3, 4 |  3, 3, 4
|  Sender, Receivers |  False positive. Flagging non-toxic content as toxic, causing harm to legitimate receivers of the information (filtered). Sender cannot post due to the filter. |  3, 3, 4 |  3, 3, 4
|  Social media users |  Social media users may be exposed to toxic content that was mislabeled by the AI as non-toxic. |  3, 3, 3 |  3, 4, 3
|  Social media users |  Social media users may feel frustrated or anxious about the accuracy of the AI tool. |  3, 3, 3 |  2, 4, 2
|  Social media users |  Social media users who do not use this AI might be exposed to some distorted message. |  2, 1, 3 |  3, 1, 3
|  Social media users |  Unfairly flagged as toxic (false positive trigger). |  3, 3, 3 |  3, 2, 2
|  Student |  Teachers use it to help students identify toxic language. Students may feel frustrated or anxious about the accuracy of the AI tool. |  2, 1, 3 |  2, 1, 4
|  Victims |  Scammers use it to identify potential victims. Victims of scams may lose money or personal information. |  2, 2, 1 |  4, 3, 1
Article Sumamrizer (Harm Envisioning Activities) |  Anyone reading the document |  If the document is sent to anyone else, the summary may not well-represent the document. The readers of the summary might mis-understand the original document. |  4, 3, 3 |  3, 2, 3
|  Business |  Write marketing content, memo. Bad summary can lead to miscommunication, causing financial loss. |  4, 3, 3 |  3, 3, 4
|  Company publishing the content (news paper) |  If the summary is wrong, people would lose confidence in the company. |  3, 4, 4 |  2, 4, 3
|  Economic loss |  The summary may contain misinformation about some critical article, causing economic loss. |  3, 3, 1 |  3, 3, 1
|  Employees |  Employees can use it to summarize documents in a workplace. Wrong summary can harm the stakeholders. |  3, 3, 3 |  4, 3, 3
|  End user |  Change the meaning of the article. |  3, 4, 4 |  3, 4, 4
|  Journalists / writers |  If the summary is wrong, journalists’ reputation might be harmed. |  1, 4, 3 |  1, 3, 4
|  Kids |  Kids use it cheat in school for assignments. |  3, 2, 3 |  2, 2, 3
|  Readers |  AI fails to pick up key information (nuances) in the article =¿ the readers miss the points from the article, miscommunication. |  3, 3, 3 |  2, 2, 2
|  Readers, content providers, new |  If there are two facts in the article that are true independently, then it’s possible that the summary combines them where the statement is no longer true. |  3, 2, 3 |  2, 2, 3
|  Readers, society |  A lot of content is missed, making readers less educated overtime, causing societal loss (some events being lost in history). |  3, 2, 1 |  3, 3, 1
|  Readers, writers |  It would lose a lot of context and details. |  3, 3, 3 |  2, 3, 2
|  Social groups |  Representational harm. It could stereotype some social groups in the output. |  3, 3, 3 |  3, 3, 3
|  Students |  Students can use this feature to summarize articles instead of reading for assignments. |  4, 3, 3 |  3, 3, 3
|  Students |  Students can use this tool to cheat and lose opportunities to learn. |  4, 1, 3 |  3, 1, 4
|  Students |  Students misuse this feature to cheat on school assignments. They might lose learning opportunities. |  4, 2, 3 |  3, 2, 3
|  Users |  The context can be manipulated. |  2, 4, 3 |  2, 4, 3
|  Users |  The summary can lose some information and context from the article. |  3, 4, 3 |  3, 4, 2
|  Users |  The tool makes mistakes (e.g., wrong summary). It can harm the users of the tool by missing information in the article. |  3, 3, 3 |  3, 3, 3
|  Users |  Users may lose trust if the summary is not right. Users would become more stressful as well. |  3, 3, 4 |  2, 2, 3
|  Writers |  If someone uses the tool to write, it may change people’s perception about how you write. |  3, 1, 1 |  2, 1, 2
Math tutor (Harm Envisioning Activities) |  Company of the AI product |  Engineers use it to explain to non-technical stakeholders. The company may face increased legal liability due to AI-generated explanations being inaccurate or misleading. |  2, 1, 2 |  1, 1, 2
|  Editors |  Journal readers use it to get explanations of equations in inferential statistics sections. Journal editors may worry that the quality of their journal declines because the AI feature makes too many errors. Readers get frustrated over time. |  2, 1, 2 |  2, 1, 3
|  Marginalized population |  People with less resources are less likely to access this tool, losing opportunity to learn. |  4, 3, 3 |  3, 3, 3
|  Minority social groups |  If you ask the math tool about important social groups, it can refuse to answer =¿ ignoring important questions, marginalizing some social groups. |  3, 3, 1 |  3, 3, 1
|  Minority social groups |  There are different ways to phrase things differently based on some social groups. If the user asks a question in non-profession English or non-English, it can cause alienation. |  4, 3, 3 |  3, 3, 3
|  Non-technical stakeholder |  Engineers use it to explain complex math concepts to non-technical stakeholders. Non-technical stakeholders may be misled by AI-generated explanations of complex mathematical models. |  3, 2, 3 |  3, 2, 3
|  Parents of students |  Students use it to learn math. Parents may feel frustrated or anxious about their child’s education. |  2, 2, 3 |  2, 2, 3
|  Public |  The public may be misled by AI-generated explanations of scientific concepts. |  3, 3, 3 |  3, 3, 3
|  Rocket engineers |  Wrong answers can harm the task. |  3, 3, 3 |  3, 2, 3
|  Student |  Tutors use it to help students understand math concepts. Students may lose the opportunity to learn the material in a way that is tailored to their individual needs. |  4, 2, 4 |  4, 2, 4
|  Students |  If the students use this tool to cheat, they would have economic loss (not performing well on their jobs). |  2, 1, 3 |  2, 1, 3
|  Students |  Incorrect math answers harm the user’s learning process. |  3, 3, 3 |  2, 3, 3
|  Students |  Students use it to cheat, losing opportunities to learn. |  4, 1, 3 |  3, 1, 3
|  Students |  Students use it to learn math. Students may feel like they are losing their ability to learn about math problems. |  2, 1, 2 |  2, 1, 2
|  Students |  Students use it to learn math. Students who do not use this AI product may feel like they are not getting the same quality of education as their peers. |  3, 3, 3 |  2, 3, 3
|  Students |  Wrong solution, less optimized solution, wrong information, students learn the wrong thing. |  3, 3, 3 |  3, 2, 3
|  Students and teachers |  Students can use this app to cheat. Students do not learn. |  4, 3, 3 |  3, 2, 4
|  Students who do not use this AI product |  Students use it to learn math. Students who do not use this AI product may feel left behind by their peers who do. |  3, 3, 4 |  3, 2, 4
|  Students, society |  If the answer is based on some context of the math problem, it can harm minority students. |  2, 3, 1 |  3, 3, 1
|  Users |  If you ask tax rates / vote count / interest rate / wages, the wrong answer can cause political harm and social harm. |  3, 2, 3 |  3, 2, 3
|  Users |  Not all questions are about numbers. Sometimes the app might refuse to answer hard math questions, and uses would feel upset. |  3, 3, 3 |  2, 2, 3