* Muller and Weisz (2022) Michael Muller and Justin Weisz. 2022.  Extending a Human-AI Collaboration Framework with Dynamism and Sociality. In _2022 Symposium on Human-Computer Interaction for Work_ _(CHIWORK 2022)_. ACM, Article 10.  https://doi.org/10.1145/3533406.3533407
  * Muller et al. (2021) Michael Muller, Christine T. Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan. 2021\.  Designing Ground Truth and the Social Life of Labels. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM, Article 94.  https://doi.org/10.1145/3411764.3445402
  * Nakao et al. (2022) Yuri Nakao, Simone Stumpf, Subeida Ahmed, Aisha Naseer, and Lorenzo Strappelli. 2022.  Toward Involving End-Users in Interactive Human-in-the-Loop AI Fairness.  _ACM Trans. Interact. Intell. Syst._ , Article 18 (July 2022).  https://doi.org/10.1145/3514258
  * Nashed et al. (2021) Samer Nashed, Justin Svegliato, and Shlomo Zilberstein. 2021\.  Ethically Compliant Planning within Moral Communities. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. ACM.  https://doi.org/10.1145/3461702.3462522
  * National Institute of Standards and Technology (2023) National Institute of Standards and Technology. 2023.  _AI Risk Management Framework_.  Retrieved February 2023 from https://www.nist.gov/itl/ai-risk-management-framework
  * Neuhauser and Kreps (2011) Linda Neuhauser and Gary L Kreps. 2011.  Participatory Design and Artificial Intelligence: Strategies to Improve Health Communication for Diverse Audiences. In _AAAI Spring Symposium: AI and Health Communication_.  https://researchers.mq.edu.au/en/publications/participatory-design-and-artificial-intelligence-strategies-to-im
  * Nielsen (2021) Aileen Nielsen. 2021\.  Measuring Lay Reactions to Personal Data Markets. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. ACM.  https://doi.org/10.1145/3461702.3462582
  * Niforatos et al. (2020) Evangelos Niforatos, Adam Palma, Roman Gluszny, Athanasios Vourvopoulos, and Fotis Liarokapis. 2020\.  Would You Do It?: Enacting Moral Dilemmas in Virtual Reality for Understanding Ethical Decision-Making. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ _(CHI ’20)_. ACM.  https://doi.org/10.1145/3313831.3376788
  * Nokia Bell Labs (2022) Nokia Bell Labs. 2022\.  _Responsible AI_.  Retrieved January 2023 from https://www.bell-labs.com/research-innovation/responsible-ai/
  * Olteanu et al. (2019) Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019.  Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries.  _Frontiers in Big Data_ (2019).  https://doi.org/10.3389/fdata.2019.00013
  * Orphanou et al. (2022) Kalia Orphanou, Jahna Otterbacher, Styliani Kleanthous, Khuyagbaatar Batsuren, Fausto Giunchiglia, Veronika Bogina, Avital Shulner Tal, Alan Hartman, and Tsvi Kuflik. 2022.  Mitigating Bias in Algorithmic Systems–A Fish-Eye View.  _ACM Comput. Surv._ (Dec. 2022).  https://doi.org/10.1145/3527152
  * Page et al. (2021) Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, Roger Chou, Julie Glanville, Jeremy M Grimshaw, Asbjørn Hróbjartsson, Manoj M Lalu, Tianjing Li, Elizabeth W Loder, Evan Mayo-Wilson, Steve McDonald, Luke A McGuinness, Lesley A Stewart, James Thomas, Andrea C Tricco, Vivian A Welch, Penny Whiting, and David Moher. 2021\.  The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.  _BMJ_ (2021).  https://doi.org/10.1136/bmj.n71
  * Park et al. (2021) Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2021.  Human-AI Interaction in Human Resource Management: Understanding Why Employees Resist Algorithmic Evaluation at Workplaces and How to Mitigate Burdens. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM.  https://doi.org/10.1145/3411764.3445304
  * Park et al. (2022) Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2022.  Designing Fair AI in Human Resource Management: Understanding Tensions Surrounding Algorithmic Evaluation and Envisioning Stakeholder-Centered Solutions. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517672
  * Passi and Jackson (2017) Samir Passi and Steven Jackson. 2017.  Data Vision: Learning to See Through Algorithmic Abstraction. In _Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_ _(CSCW ’17)_. ACM.  https://doi.org/10.1145/2998181.2998331
  * Pessach and Shmueli (2022) Dana Pessach and Erez Shmueli. 2022.  A Review on Fairness in Machine Learning.  _ACM Comput. Surv._ (Feb. 2022).  https://doi.org/10.1145/3494672
  * Peters et al. (2013) Anicia Peters, Heike Winschiers-Theophilus, and Brian Mennecke. 2013\.  Bridging the Digital Divide through Facebook Friendships: A Cross-Cultural Study. In _Proceedings of the 2013 Conference on Computer Supported Cooperative Work Companion_ _(CSCW ’13)_. ACM.  https://doi.org/10.1145/2441955.2442014
  * Pine and Liboiron (2015) Kathleen H. Pine and Max Liboiron. 2015.  The Politics of Measurement and Action. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ _(CHI ’15)_. ACM.  https://doi.org/10.1145/2702123.2702298
  * Pushkarna et al. (2022) Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. 2022\.  Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533231
  * PwC (2022) PwC. 2022.  _PwC’s Responsible AI_.  Retrieved February 2023 from https://www.pwc.com/gx/en/issues/data-and-analytics/artificial-intelligence/what-is-responsible-ai/pwc-responsible-ai.pdf
  * Rakova et al. (2021) Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021.  Where Responsible AI Meets Reality: Practitioner Perspectives on Enablers for Shifting Organizational Practices.  _Proc. ACM Hum.-Comput. Interact._ (April 2021).  https://doi.org/10.1145/3449081
  * Ramesh et al. (2022) Divya Ramesh, Vaishnav Kameswaran, Ding Wang, and Nithya Sambasivan. 2022.  How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533237
  * Raz et al. (2021) Daniella Raz, Corinne Bintz, Vivian Guetler, Aaron Tam, Michael Katell, Dharma Dailey, Bernease Herman, P. M. Krafft, and Meg Young. 2021.  Face Mis-ID: An Interactive Pedagogical Tool Demonstrating Disparate Accuracy Rates in Facial Recognition. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. ACM.  https://doi.org/10.1145/3461702.3462627
  * Reinecke et al. (2013) Katharina Reinecke, Minh Khoa Nguyen, Abraham Bernstein, Michael Näf, and Krzysztof Z. Gajos. 2013.  Doodle around the World: Online Scheduling Behavior Reflects Cultural Differences in Time Perception and Group Decision-Making. In _Proceedings of the 2013 Conference on Computer Supported Cooperative Work_ _(CSCW ’13)_. ACM.  https://doi.org/10.1145/2441776.2441784
  * Reitmaier et al. (2022) Thomas Reitmaier, Electra Wallington, Dani Kalarikalayil Raju, Ondrej Klejch, Jennifer Pearson, Matt Jones, Peter Bell, and Simon Robinson. 2022.  Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517639
  * Richards et al. (2019) John Richards, David Piorkowski, Michael Hind, Stephanie Houde, and Aleksandra Mojsilović. 2019\.  FactSheets: Increasing trust in AI services through supplier’s declarations of conformity.  _IBM Journal of Research and Development_ (2019).  https://doi.org/10.1147/JRD.2019.2942288
  * Richardson et al. (2021) Brianna Richardson, Jean Garcia-Gathright, Samuel F. Way, Jennifer Thom, and Henriette Cramer. 2021\.  Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM.  https://doi.org/10.1145/3411764.3445604
  * Rismani et al. (2023) Shalaleh Rismani, Renee Shelby, Andrew Smart, Edgar Jatho, Joshua Kroll, AJung Moon, and Negar Rostamzadeh. 2023.  From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581407
  * Roemmich and Andalibi (2021) Kat Roemmich and Nazanin Andalibi. 2021.  Data Subjects’ Conceptualizations of and Attitudes Toward Automatic Emotion Recognition-Enabled Wellbeing Interventions on Social Media.  _Proc. ACM Hum.-Comput. Interact._ (Oct. 2021).  https://doi.org/10.1145/3476049
  * Sambasivan et al. (2021a) Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021a.  Re-Imagining Algorithmic Fairness in India and Beyond. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’21)_. ACM.  https://doi.org/10.1145/3442188.3445896
  * Sambasivan et al. (2021b) Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021b.  “Everyone Wants to Do the Model Work, Not the Data Work”: Data Cascades in High-Stakes AI. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM.  https://doi.org/10.1145/3411764.3445518
  * Saxena et al. (2020) Devansh Saxena, Karla Badillo-Urquiola, Pamela J. Wisniewski, and Shion Guha. 2020.  A Human-Centered Review of Algorithms Used within the U.S. Child Welfare System. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ _(CHI ’20)_. ACM.  https://doi.org/10.1145/3313831.3376229
  * Schiff et al. (2020) Daniel Schiff, Justin Biddle, Jason Borenstein, and Kelly Laas. 2020.  What’s Next for AI Ethics, Policy, and Governance? A Global Overview. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’20)_. ACM.  https://doi.org/10.1145/3375627.3375804
  * Septiandri et al. (2023) Ali Akbar Septiandri, Marios Constantinides, Mohammad Tahaei, and Daniele Quercia. 2023.  WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and Democratic is FAccT?. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’23)_. ACM.  https://doi.org/10.1145/3593013.3593985
  * Shahid and Vashistha (2023) Farhana Shahid and Aditya Vashistha. 2023.  Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony?. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581538
  * Sharma et al. (2021) Shubham Sharma, Alan H. Gee, David Paydarfar, and Joydeep Ghosh. 2021.  FaiR-N: Fair and Robust Neural Networks for Structured Data. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. ACM.  https://doi.org/10.1145/3461702.3462559
  * Shaw et al. (2018) Nolan P. Shaw, Andreas Stöckel, Ryan W. Orr, Thomas F. Lidbetter, and Robin Cohen. 2018\.  Towards Provably Moral AI Agents in Bottom-up Learning Frameworks. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’18)_. ACM.  https://doi.org/10.1145/3278721.3278728
  * Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021.  Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors.  _Proc. ACM Hum.-Comput. Interact._ (Oct. 2021).  https://doi.org/10.1145/3479577
  * Shen et al. (2020) Hong Shen, Haojian Jin, Ángel Alexander Cabrera, Adam Perer, Haiyi Zhu, and Jason I. Hong. 2020\.  Designing Alternative Representations of Confusion Matrices to Support Non-Expert Public Understanding of Algorithm Performance.  _Proc. ACM Hum.-Comput. Interact._ (Oct. 2020).  https://doi.org/10.1145/3415224
  * Shneiderman (2020) Ben Shneiderman. 2020\.  Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-Centered AI Systems.  _ACM Trans. Interact. Intell. Syst._ (Oct. 2020).  https://doi.org/10.1145/3419764
  * Shneiderman (2022) Ben Shneiderman. 2022\.  _Human-centered AI_.  Oxford University Press.
  * Siala and Wang (2022) Haytham Siala and Yichuan Wang. 2022.  SHIFTing artificial intelligence to be responsible in healthcare: A systematic review.  _Social Science & Medicine_ (2022).  https://doi.org/10.1016/j.socscimed.2022.114782
  * Siapka (2022) Anastasia Siapka. 2022\.  Towards a Feminist Metaethics of AI. In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’22)_. ACM.  https://doi.org/10.1145/3514094.3534197
  * Silva and Kenney (2018) Selena Silva and Martin Kenney. 2018.  Algorithms, platforms, and ethnic bias: An integrative essay.  _Phylon (1960-)_ (2018).  https://www.jstor.org/stable/26545017
  * Simons et al. (2021) Joshua Simons, Sophia Adams Bhatti, and Adrian Weller. 2021\.  Machine Learning and the Meaning of Equal Treatment. In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’21)_. ACM.  https://doi.org/10.1145/3461702.3462556
  * Sloane and Zakrzewski (2022) Mona Sloane and Janina Zakrzewski. 2022.  German AI Start-Ups and “AI Ethics”: Using A Social Practice Lens for Assessing and Implementing Socio-Technical Innovation. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533156
  * Smith et al. (2022) Jessie J. Smith, Saleema Amershi, Solon Barocas, Hanna Wallach, and Jennifer Wortman Vaughan. 2022\.  REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533122
  * Stapleton et al. (2022) Logan Stapleton, Min Hun Lee, Diana Qing, Marya Wright, Alexandra Chouldechova, Ken Holstein, Zhiwei Steven Wu, and Haiyi Zhu. 2022\.  Imagining New Futures beyond Predictive Systems in Child Welfare: A Qualitative Study with Impacted Stakeholders. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533177
  * Stark and Hoey (2021) Luke Stark and Jesse Hoey. 2021.  The Ethics of Emotion in Artificial Intelligence Systems. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’21)_. ACM.  https://doi.org/10.1145/3442188.3445939
  * Steiger et al. (2021) Miriah Steiger, Timir J Bharucha, Sukrit Venkatagiri, Martin J. Riedl, and Matthew Lease. 2021\.  The Psychological Well-Being of Content Moderators: The Emotional Labor of Commercial Moderation and Avenues for Improving Support. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM.  https://doi.org/10.1145/3411764.3445092
  * Subramonyam et al. (2022) Hariharan Subramonyam, Jane Im, Colleen Seifert, and Eytan Adar. 2022.  Solving Separation-of-Concerns Problems in Collaborative Design of Human-AI Systems through Leaky Abstractions. In _CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517537
  * Sun et al. (2019) Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019\.  Mitigating Gender Bias in Natural Language Processing: Literature Review. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.  https://doi.org/10.18653/v1/P19-1159
  * Tahaei et al. (2023) Mohammad Tahaei, Marios Constantinides, Daniele Quercia, Sean Kennedy, Michael Muller, Simone Stumpf, Q. Vera Liao, Ricardo Baeza-Yates, Lora Aroyo, Jess Holbrook, Ewa Luger, Michael Madaio, Ilana Golbin Blumenfeld, Maria De-Arteaga, Jessica Vitak, and Alexandra Olteanu. 2023.  Human-Centered Responsible Artificial Intelligence: Current & Future Trends. In _Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI EA ’23)_. ACM.  https://doi.org/10.1145/3544549.3583178
  * Tahaei and Vaniea (2019) Mohammad Tahaei and Kami Vaniea. 2019.  A Survey on Developer-Centred Security. In _2019 IEEE European Symposium on Security and Privacy Workshops (EuroS &PW)_. IEEE.  https://doi.org/10.1109/EuroSPW.2019.00021
  * Tahaei et al. (2020) Mohammad Tahaei, Kami Vaniea, and Naomi Saphra. 2020\.  Understanding Privacy-Related Questions on Stack Overflow. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ _(CHI ’20)_. ACM, 14 pages.  https://doi.org/10.1145/3313831.3376768
  * Terzis (2020) Petros Terzis. 2020\.  Onward for the Freedom of Others: Marching beyond the AI Ethics. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ _(FAT* ’20)_. ACM.  https://doi.org/10.1145/3351095.3373152
  * The European Parliament and the Council of the European Union (2018) The European Parliament and the Council of the European Union. 2018.  _General Data Protection Regulation (GDPR)_.  The European Parliament and the Council of the European Union.  Retrieved January 2023 from https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679
  * The Organisation for Economic Co-operation and Development (2019) (OECD) The Organisation for Economic Co-operation and Development (OECD). 2019.  _Recommendation of the Council on Artificial Intelligence_.  Retrieved February 2023 from https://legalinstruments.oecd.org/en/instruments/oecd-legal-0449
  * To et al. (2021) Alexandra To, Hillary Carey, Geoff Kaufman, and Jessica Hammer. 2021.  Reducing Uncertainty and Offering Comfort: Designing Technology for Coping with Interpersonal Racism. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ _(CHI ’21)_. ACM.  https://doi.org/10.1145/3411764.3445590
  * Tolmeijer et al. (2022) Suzanne Tolmeijer, Markus Christen, Serhiy Kandul, Markus Kneer, and Abraham Bernstein. 2022.  Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517732
  * Toreini et al. (2020) Ehsan Toreini, Mhairi Aitken, Kovila Coopamootoo, Karen Elliott, Carlos Gonzalez Zelaya, and Aad van Moorsel. 2020.  The Relationship between Trust in AI and Trustworthy Machine Learning Technologies. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ _(FAT* ’20)_. ACM.  https://doi.org/10.1145/3351095.3372834
  * Tsosie (2019) Rebecca Tsosie. 2019\.  Tribal Data Governance and informational privacy: constructing indigenous data sovereignty.  _Mont. L. Rev._ (2019).
  * United States Patent and Trademark Office (2023) United States Patent and Trademark Office. 2023.  _United States Patent and Trademark Office_.  Retrieved June 2023 from https://www.uspto.gov
  * Vaccaro et al. (2021) Kristen Vaccaro, Ziang Xiao, Kevin Hamilton, and Karrie Karahalios. 2021.  Contestability For Content Moderation.  _Proc. ACM Hum.-Comput. Interact._ (Oct. 2021).  https://doi.org/10.1145/3476059
  * Vaisman (2021) Carmel Vaisman. 2021\.  Your Next Robotic Boy/Girlfriend.  _Afeka Journal of Engineering and Science_ (Oct. 2021).  Issue 3.
  * Valencia et al. (2023) Stephanie Valencia, Richard Cave, Krystal Kallarackal, Katie Seaver, Michael Terry, and Shaun K. Kane. 2023.  “The Less I Type, the Better”: How AI Language Models Can Enhance or Impede Communication for AAC Users. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581560
  * van Berkel et al. (2023) Niels van Berkel, Zhanna Sarsenbayeva, and Jorge Goncalves. 2023\.  The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT.  _International Journal of Human-Computer Studies_ (2023).
  * Varanasi and Goyal (2023) Rama Adithya Varanasi and Nitesh Goyal. 2023.  “It is Currently Hodgepodge”: Examining AI/ML Practitioners’ Challenges during Co-Production of Responsible AI Values. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3580903
  * Verma et al. (2023) Himanshu Verma, Jakub Mlynar, Roger Schaer, Julien Reichenbach, Mario Jreige, John Prior, Florian Evéquoz, and Adrien Depeursinge. 2023.  Rethinking the Role of AI with Physicians in Oncology: Revealing Perspectives from Clinical and Research Workflows. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581506
  * Vigil-Hayes et al. (2017) Morgan Vigil-Hayes, Marisa Duarte, Nicholet Deschine Parkhurst, and Elizabeth Belding. 2017.  #Indigenous: Tracking the Connective Actions of Native American Advocates on Twitter. In _Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_ _(CSCW ’17)_. ACM.  https://doi.org/10.1145/2998181.2998194
  * Viswanathan et al. (2022) Sruthi Viswanathan, Cecile Boulard, Adrien Bruyat, and Antonietta Maria Grasso. 2022.  Situational Recommender: Are You On the Spot, Refining Plans, or Just Bored?. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3501909
  * Vitos et al. (2017) Michalis Vitos, Julia Altenbuchner, Matthias Stevens, Gillian Conquest, Jerome Lewis, and Muki Haklay. 2017.  Supporting Collaboration with Non-Literate Forest Communities in the Congo-Basin. In _Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_ _(CSCW ’17)_. ACM.  https://doi.org/10.1145/2998181.2998242
  * Waldman (2021) Ari Ezra Waldman. 2021\.  _Industry Unbound: The Inside Story of Privacy, Data, and Corporate Power_.  Cambridge University Press.  https://doi.org/10.1017/9781108591386
  * Walter and Suina (2019) Maggie Walter and Michele Suina. 2019.  Indigenous data, indigenous methodologies and indigenous data sovereignty.  _International Journal of Social Research Methodology_ (2019).
  * Wang et al. (2022b) Ding Wang, Shantanu Prabhat, and Nithya Sambasivan. 2022b.  Whose AI Dream? In Search of the Aspiration in Data Annotation.. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3502121
  * Wang et al. (2019a) Dakuo Wang, Justin D. Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019a.  Human-AI Collaboration in Data Science: Exploring Data Scientists’ Perceptions of Automated AI.  _Proc. ACM Hum.-Comput. Interact._ (Nov. 2019).  https://doi.org/10.1145/3359313
  * Wang et al. (2019b) Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019b.  Designing Theory-Driven User-Centric Explainable AI. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_ _(CHI ’19)_. ACM.  https://doi.org/10.1145/3290605.3300831
  * Wang et al. (2022c) Ge Wang, Jun Zhao, Max Van Kleek, and Nigel Shadbolt. 2022c.  Informing Age-Appropriate AI: Examining Principles and Practices of AI for Children. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3502057
  * Wang et al. (2021) Liuping Wang, Dakuo Wang, Feng Tian, Zhenhui Peng, Xiangmin Fan, Zhan Zhang, Mo Yu, Xiaojuan Ma, and Hongan Wang. 2021.  CASS: Towards Building a Social-Support Chatbot for Online Health Community.  _Proc. ACM Hum.-Comput. Interact._ (April 2021).  https://doi.org/10.1145/3449083
  * Wang et al. (2022a) Qiaosi Wang, Ida Camacho, Shan Jing, and Ashok K. Goel. 2022a.  Understanding the Design Space of AI-Mediated Social Interaction in Online Learning: Challenges and Opportunities.  _Proc. ACM Hum.-Comput. Interact._ (April 2022).  https://doi.org/10.1145/3512977
  * Wang et al. (2023) Qiaosi Wang, Michael Madaio, Shaun Kane, Shivani Kapania, Michael Terry, and Lauren Wilcox. 2023\.  Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581278
  * Washington and Kuo (2020) Anne L. Washington and Rachel Kuo. 2020.  Whose Side Are Ethics Codes on? Power, Responsibility and the Social Good. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ _(FAT* ’20)_. ACM.  https://doi.org/10.1145/3351095.3372844
  * Watkins (2023) Elizabeth Anne Watkins. 2023\.  Face Work: A Human-Centered Investigation into Facial Verification in Gig Work.  _Proc. ACM Hum.-Comput. Interact._ (April 2023).  https://doi.org/10.1145/3579485
  * Weick (1995) Karl E Weick. 1995\.  _Sensemaking in organizations_.  Sage.  https://us.sagepub.com/en-us/nam/sensemaking-in-organizations/book4988
  * Weidinger et al. (2022) Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022.  Taxonomy of Risks Posed by Language Models. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533088
  * Whittlestone et al. (2019) Jess Whittlestone, Rune Nyrup, Anna Alexandrova, and Stephen Cave. 2019.  The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’19)_. ACM.  https://doi.org/10.1145/3306618.3314289
  * Widder et al. (2022) David Gray Widder, Dawn Nafus, Laura Dabbish, and James Herbsleb. 2022.  Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533779
  * Wilkinson et al. (2020) Dominic Wilkinson, Hazem Zohny, Andreas Kappes, Walter Sinnott-Armstrong, and Julian Savulescu. 2020\.  Which factors should be included in triage? An online survey of the attitudes of the UK general public to pandemic triage dilemmas.  _BMJ open_ (2020).  https://doi.org/10.1136/bmjopen-2020-045593
  * Willen et al. (2022) Sarah S. Willen, Abigail Fisher Williamson, Colleen C. Walsh, Mikayla Hyman, and William Tootle. 2022.  Rethinking flourishing: Critical insights and qualitative perspectives from the U.S. Midwest.  _SSM - Mental Health_ (2022).  https://doi.org/10.1016/j.ssmmh.2021.100057
  * Winecoff and Watkins (2022) Amy A. Winecoff and Elizabeth Anne Watkins. 2022.  Artificial Concepts of Artificial Intelligence: Institutional Compliance and Resistance in AI Startups. In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’22)_. ACM.  https://doi.org/10.1145/3514094.3534138
  * Wong et al. (2023a) Richmond Y. Wong, Andrew Chong, and R. Cooper Aspegren. 2023a.  Privacy Legislation as Business Risks: How GDPR and CCPA Are Represented in Technology Companies’ Investment Risk Disclosures.  _Proc. ACM Hum.-Comput. Interact._ (April 2023).  https://doi.org/10.1145/3579515
  * Wong et al. (2023b) Richmond Y. Wong, Michael A. Madaio, and Nick Merrill. 2023b.  Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics.  _Proc. ACM Hum.-Comput. Interact._ (April 2023).  https://doi.org/10.1145/3579621
  * Xu et al. (2022) Jie Xu, Yunyu Xiao, Wendy Hui Wang, Yue Ning, Elizabeth A Shenkman, Jiang Bian, and Fei Wang. 2022.  Algorithmic fairness in computational medicine.  _EBioMedicine_ (2022).  https://doi.org/10.1016/j.ebiom.2022.104250
  * Yang et al. (2023) Qian Yang, Yuexing Hao, Kexin Quan, Stephen Yang, Yiran Zhao, Volodymyr Kuleshov, and Fei Wang. 2023.  Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581393
  * Yang et al. (2022) Yu Yang, Aayush Gupta, Jianwei Feng, Prateek Singhal, Vivek Yadav, Yue Wu, Pradeep Natarajan, Varsha Hedau, and Jungseock Joo. 2022.  Enhancing Fairness in Face Detection in Computer Vision Systems by Demographic Bias Mitigation. In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’22)_. ACM.  https://doi.org/10.1145/3514094.3534153
  * Yfantidou et al. (2023) Sofia Yfantidou, Marios Constantinides, Dimitris Spathis, Athena Vakali, Daniele Quercia, and Fahim Kawsar. 2023.  Beyond Accuracy: A Critical Review of Fairness in Machine Learning for Mobile and Wearable Computing.  _arXiv preprint arXiv:2303.15585_ (2023).
  * Yildirim et al. (2022) Nur Yildirim, Alex Kass, Teresa Tung, Connor Upton, Donnacha Costello, Robert Giusti, Sinem Lacin, Sara Lovic, James M O’Neill, Rudi O’Reilly Meehan, Eoin Ó Loideáin, Azzurra Pini, Medb Corcoran, Jeremiah Hayes, Diarmuid J Cahalane, Gaurav Shivhare, Luigi Castoro, Giovanni Caruso, Changhoon Oh, James McCann, Jodi Forlizzi, and John Zimmerman. 2022\.  How Experienced Designers of Enterprise Applications Engage AI as a Design Material. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517491
  * Yildirim et al. (2023) Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. 2023\.  Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3580900
  * Young (2021) James Young. 2021\.  Danger! This robot may be trying to manipulate you.  _Science Robotics_ (2021).  https://doi.org/10.1126/scirobotics.abk3479
  * Young et al. (2022) Meg Young, Michael Katell, and P.M. Krafft. 2022\.  Confronting Power and Corporate Capture at the FAccT Conference. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533194
  * Yuan et al. (2023) Chien Wen (Tina) Yuan, Nanyi Bi, Ya-Fang Lin, and Yuen-Hsien Tseng. 2023.  Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3580945
  * Yurrita et al. (2022) Mireia Yurrita, Dave Murray-Rust, Agathe Balayn, and Alessandro Bozzon. 2022.  Towards a Multi-Stakeholder Value-Based Assessment Framework for Algorithmic Systems. In _2022 ACM Conference on Fairness, Accountability, and Transparency_ _(FAccT ’22)_. ACM.  https://doi.org/10.1145/3531146.3533118
  * Yuval Noah Harari (2021) Yuval Noah Harari. 2021\.  _Lessons from a year of Covid_.  Financial Times.  Retrieved January 2023 from https://www.ft.com/content/f1b30f2c-84aa-4595-84f2-7816796d6841
  * Zhang et al. (2023a) Angie Zhang, Alexander Boltz, Jonathan Lynn, Chun-Wei Wang, and Min Kyung Lee. 2023a.  Stakeholder-Centered AI Design: Co-Designing Worker Tools with Gig Workers through Data Probes. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581354
  * Zhang et al. (2023b) Angie Zhang, Olympia Walker, Kaci Nguyen, Jiajun Dai, Anqing Chen, and Min Kyung Lee. 2023b.  Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation.  _Proc. ACM Hum.-Comput. Interact._ (April 2023).  https://doi.org/10.1145/3579601
  * Zhang et al. (2022) Wencan Zhang, Mariella Dimiccoli, and Brian Y Lim. 2022\.  Debiased-CAM to Mitigate Image Perturbations with Faithful Visual Explanations of Machine Learning. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI ’22)_. ACM.  https://doi.org/10.1145/3491102.3517522
  * Zheng et al. (2023) Chengbo Zheng, Yuheng Wu, Chuhan Shi, Shuai Ma, Jiehui Luo, and Xiaojuan Ma. 2023\.  Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581131
  * Zhou et al. (2023) Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. 2023\.  Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI ’23)_. ACM.  https://doi.org/10.1145/3544548.3581318
  * Zytko et al. (2022) Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric P. S. Baumer, and Min Kyung Lee. 2022\.  Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains. In _Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems_ _(CHI EA ’22)_. ACM, Article 154\.  https://doi.org/10.1145/3491101.3516506

##  Appendix A Appendix: Summary of All Reviewed HCER-AI Research Papers

Table 4. A list of all HCER-AI reviewed papers with their research methods and themes (n=164). Citation | Research method | Year | Venue | Explainability | Human flourishing | Privacy | Security | Fairness | Governance
---|---|---|---|---|---|---|---|---|---
Maas (2018) | Theory | 2018 | AIES |  |  |  |  |  |
Whittlestone et al. (2019) | Review & Theory | 2019 | AIES |  |  |  |  |  |
Schiff et al. (2020) | Review & Theory | 2020 | AIES |  |  |  |  |  |
Erdélyi and Erdélyi (2020) | Review | 2020 | AIES |  |  |  |  |  |
Hopkins and Booth (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Henriksen et al. (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Nielsen (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Klinova and Korinek (2021) | Theory | 2021 | AIES |  |  |  |  |  |
Raz et al. (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Abdalla and Abdalla (2021) | Quantitative | 2021 | AIES |  |  |  |  |  |
Simons et al. (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Lee et al. (2021) | Theory | 2021 | AIES |  |  |  |  |  |
Deshpande and Sharp (2022) | Review | 2022 | AIES |  |  |  |  |  |
Bertrand et al. (2022) | Review | 2022 | AIES |  |  |  |  |  |
Bessen et al. (2022) | Qualitative | 2022 | AIES |  |  |  |  |  |
Aka et al. (2021) | Quantitative | 2022 | AIES |  |  |  |  |  |
Sharma et al. (2021) | Quantitative | 2021 | AIES |  |  |  |  |  |
Winecoff and Watkins (2022) | Quantitative | 2021 | AIES |  |  |  |  |  |
Nashed et al. (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Leavy et al. (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Cruz Cortés and Ghosh (2020) | Quantitative | 2020 | AIES |  |  |  |  |  |
Kasirzadeh and Klein (2021) | Theory | 2021 | AIES |  |  |  |  |  |
Yang et al. (2022) | Theory | 2021 | AIES |  |  |  |  |  |
Flathmann et al. (2021) | Quantitative | 2022 | AIES |  |  |  |  |  |
Shaw et al. (2018) | Quantitative | 2018 | AIES |  |  |  |  |  |
Benthall and Goldenfein (2021) | Qualitative | 2021 | AIES |  |  |  |  |  |
Kim et al. (2018) | Theory | 2021 | AIES |  |  |  |  |  |
Eicher et al. (2018) | Quantitative | 2018 | AIES |  |  |  |  |  |
Chi et al. (2021) | Theory | 2018 | AIES |  |  |  |  |  |
Siapka (2022) | Theory | 2022 | AIES |  |  |  |  |  |
Beede et al. (2020) | Qualitative | 2020 | CHI |  |  |  |  |  |
Liao et al. (2020) | Qualitative | 2020 | CHI |  |  |  |  |  |
Cheng et al. (2021) | Qualitative | 2021 | CHI |  |  |  |  |  |
Morrison et al. (2021) | Qualitative | 2021 | CHI |  |  |  |  |  |
Park et al. (2021) | Quantitative | 2021 | CHI |  |  |  |  |  |
Bansal et al. (2021) | Qualitative | 2021 | CHI |  |  |  |  |  |
Ehsan et al. (2021a) | Qualitative | 2021 | CHI |  |  |  |  |  |
Sambasivan et al. (2021b) | Mixed | 2021 | CHI |  |  |  |  |  |
Langer et al. (2022) | Review | 2021 | CHI |  |  |  |  |  |
Lee and Rich (2021) | Qualitative | 2021 | CHI |  |  |  |  |  |
Steiger et al. (2021) | Theory | 2022 | CHI |  |  |  |  |  |
Muller and Strohmayer (2022) | Quantitative | 2022 | CHI |  |  |  |  |  |
Cheng et al. (2022) | Mixed | 2022 | CHI |  |  |  |  |  |
Park et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Mlynar et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Wang et al. (2022c) | Mixed | 2022 | CHI |  |  |  |  |  |
Kapania et al. (2022) | Review | 2022 | CHI |  |  |  |  |  |
Wang et al. (2022b) | Qualitative | 2022 | CHI |  |  |  |  |  |
Reitmaier et al. (2022) | Mixed | 2022 | CHI |  |  |  |  |  |
Yildirim et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Hall et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Long et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Docherty and Biega (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Kang et al. (2022) | Theory | 2022 | CHI |  |  |  |  |  |
Zhang et al. (2022) | Qualitative | 2022 | CHI |  |  |  |  |  |
Viswanathan et al. (2022) | Mixed | 2022 | CHI |  |  |  |  |  |
Tolmeijer et al. (2022) | Review | 2022 | CHI |  |  |  |  |  |
Subramonyam et al. (2022) | Quantitative | 2022 | CHI |  |  |  |  |  |
Lin and Van Brummelen (2021) | Quantitative | 2020 | CHI |  |  |  |  |  |
Niforatos et al. (2020) | Qualitative | 2021 | CHI |  |  |  |  |  |
Elsayed-Ali et al. (2023) | Mixed | 2021 | CHI |  |  |  |  |  |
Varanasi and Goyal (2023) | Quantitative | 2021 | CHI |  |  |  |  |  |
To et al. (2021) | Qualitative | 2021 | CHI |  |  |  |  |  |
Wang et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Lima et al. (2021) | Qualitative | 2023 | CHI |  |  |  |  |  |
Richardson et al. (2021) | Qualitative | 2023 | CHI |  |  |  |  |  |
Kamikubo et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Moore et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Verma et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Deng et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Chu et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Zhou et al. (2023) | Quantitative | 2023 | CHI |  |  |  |  |  |
Kapania et al. (2023) | Quantitative | 2023 | CHI |  |  |  |  |  |
Chen and Sundar (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Yildirim et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Ayobi et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Alfrink et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Liao et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Zhang et al. (2023a) | Qualitative | 2023 | CHI |  |  |  |  |  |
Lee et al. (2023a) | Mixed | 2023 | CHI |  |  |  |  |  |
Cabitza et al. (2023) | Quantitative | 2023 | CHI |  |  |  |  |  |
Valencia et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Hsieh et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Ashktorab et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Lam et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Baughan et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Zheng et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Rismani et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Shahid and Vashistha (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Li et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Lima et al. (2023) | Quantitative | 2023 | CHI |  |  |  |  |  |
Yang et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Kim et al. (2023) | Mixed | 2023 | CHI |  |  |  |  |  |
Yuan et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Burgess et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Lewicki et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Choi et al. (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Capel and Brereton (2023) | Qualitative | 2023 | CHI |  |  |  |  |  |
Rakova et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Gu et al. (2021) | Qualitative | 2022 | CSCW |  |  |  |  |  |
Heger et al. (2022) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Lam et al. (2022) | Qualitative | 2022 | CSCW |  |  |  |  |  |
Lee et al. (2019) | Qualitative | 2019 | CSCW |  |  |  |  |  |
Bawa et al. (2020) | Mixed | 2020 | CSCW |  |  |  |  |  |
Shen et al. (2020) | Mixed | 2020 | CSCW |  |  |  |  |  |
Lee et al. (2023b) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Wang et al. (2019a) | Qualitative | 2019 | CSCW |  |  |  |  |  |
Ehsan et al. (2023) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Roemmich and Andalibi (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Banovic et al. (2023) | Quantitative | 2023 | CSCW |  |  |  |  |  |
Watkins (2023) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Feuston and Brubaker (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Wang et al. (2021) | Mixed | 2021 | CSCW |  |  |  |  |  |
Chancellor et al. (2019) | Qualitative | 2019 | CSCW |  |  |  |  |  |
Wang et al. (2022a) | Qualitative | 2022 | CSCW |  |  |  |  |  |
Wong et al. (2023b) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Long et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
McDonald and Pan (2020) | Qualitative | 2020 | CSCW |  |  |  |  |  |
Holstein et al. (2023) | Mixed | 2023 | CSCW |  |  |  |  |  |
Cabrera et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Moitra et al. (2022) | Qualitative | 2022 | CSCW |  |  |  |  |  |
Jia et al. (2022) | Quantitative | 2022 | CSCW |  |  |  |  |  |
Shen et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Zhang et al. (2023b) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Burrell et al. (2019) | Mixed | 2019 | CSCW |  |  |  |  |  |
Lyons et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Buçinca et al. (2021) | Quantitative | 2021 | CSCW |  |  |  |  |  |
Fox et al. (2023) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Wong et al. (2023a) | Qualitative | 2023 | CSCW |  |  |  |  |  |
Greiffenhagen et al. (2023) | Quantitative | 2023 | CSCW |  |  |  |  |  |
Vaccaro et al. (2021) | Qualitative | 2021 | CSCW |  |  |  |  |  |
Miceli et al. (2022) | Qualitative | 2022 | CSCW |  |  |  |  |  |
Kroll (2021) | Qualitative | 2021 | FAccT |  |  |  |  |  |
Krafft et al. (2021) | Theory | 2021 | FAccT |  |  |  |  |  |
Sambasivan et al. (2021a) | Qualitative | 2021 | FAccT |  |  |  |  |  |
Knowles and Richards (2021) | Theory | 2021 | FAccT |  |  |  |  |  |
Jakesch et al. (2022) | Quantitative | 2022 | FAccT |  |  |  |  |  |
Pushkarna et al. (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Liao and Sundar (2022) | Theory | 2022 | FAccT |  |  |  |  |  |
Bell et al. (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Stapleton et al. (2022) | Quantitative | 2022 | FAccT |  |  |  |  |  |
Weidinger et al. (2022) | Review | 2022 | FAccT |  |  |  |  |  |
Ramesh et al. (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Ha (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Smith et al. (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Donahue et al. (2022) | Mixed | 2022 | FAccT |  |  |  |  |  |
Crisan et al. (2022) | Theory | 2022 | FAccT |  |  |  |  |  |
Huang and Liem (2022) | Quantitative | 2022 | FAccT |  |  |  |  |  |
Black et al. (2022) | Review & Theory | 2022 | FAccT |  |  |  |  |  |
Sloane and Zakrzewski (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Ashurst et al. (2022) | Theory | 2022 | FAccT |  |  |  |  |  |
Kaur et al. (2022) | Theory | 2022 | FAccT |  |  |  |  |  |
Costanza-Chock et al. (2022) | Mixed | 2022 | FAccT |  |  |  |  |  |
Widder et al. (2022) | Qualitative | 2022 | FAccT |  |  |  |  |  |
Kasirzadeh and Smart (2021) | Review | 2021 | FAccT |  |  |  |  |  |
Engelmann et al. (2022) | Quantitative | 2022 | FAccT |  |  |  |  |  |
Terzis (2020) | Theory | 2020 | FAccT |  |  |  |  |  |
Toreini et al. (2020) | Review & Theory | 2020 | FAccT |  |  |  |  |  |
Boag et al. (2022) | Review & Theory | 2022 | FAccT |  |  |  |  |  |
Bietti (2020) | Theory | 2020 | FAccT |  |  |  |  |  |
Washington and Kuo (2020) | Qualitative | 2020 | FAccT |  |  |  |  |  |
Lima et al. (2022) | Theory | 2022 | FAccT |  |  |  |  |  |
Stark and Hoey (2021) | Theory | 2021 | FAccT |  |  |  |  |  |
Young et al. (2022) | Theory | 2022 | FAccT |  |  |  |  |  |

##  Appendix B Appendix: Interactions With ChatGPT 4.0

We first let ChatGPT 4.0 know that our content is larger than its text limit
with the following prompt:

“The total length of the content that I want to send you is too large to send
in only one piece. For sending you that content, I will follow this rule:
[START PART 1/15] this is the content of the part 1 out of 15 in total [END
PART 1/15] Then you just answer: ‘Received part 1/15’

And when I tell you ‘ALL PARTS SENT,’ then you can continue processing the
data and answering my requests.”

###  B.1. Thematic Analysis for Research Papers

ChatGPT generated the following text with minor stylist changes using all the
abstracts of the 164 papers in our review. We first asked for “apply thematic
analysis to all of the previous 15 prompts based on the method proposed by
Braun and Clarke (2008) and generate themes,” which resulted in step-by-step
instructions for how to do the thematic analysis. In a follow-up prompt, we
asked to “produce a report based on the thematic analysis proposed by Braun
and Clarke (2008).” ChatGPT produced the following paragraphs with this
prompt:

This approach led to the identification of four main themes, each underscoring
key areas of interest and investigation among researchers in the domain of AI
ethics. The themes are as follows: Ethical Implications of AI, Practical
Applications of AI, Understanding and Documenting Data, and User Engagement.

##### Theme 1: Ethical Implications of AI

This theme emerges prominently, signaling that ethical considerations are of
primary concern in AI research and practice. The studies under this theme
scrutinize ethics codes, considering how they may conflate consumers with
society and ignore the political aspects of shared resources. The analyses
highlight a gulf between the ideals presented in ethics codes and lived
experiences, particularly for marginalized communities.

Moreover, this theme reflects the power of collective action among tech
workers as a countervailing force against corporate impulses to prioritize
growth or profit over ethical considerations. Successful campaigns are
discussed, along with frameworks for future action. This suggests a
recognition of the potential for internal dynamics within tech companies to
act as a transformative lever in the pursuit of ethical AI.

##### Theme 2: Practical Applications of AI

This theme examines the deployment of AI in real-world contexts, such as
healthcare and disaster risk management. AI’s practical applications can offer
significant benefits, from improving clinician workflows and patient outcomes
in the detection of diabetic eye disease to supporting disaster risk
management by making data collection or analysis practices faster, less
costly, or more accurate.

However, these studies also acknowledge the potential for negative
consequences. In healthcare, socio-environmental factors can impact model
performance and patient experience, emphasizing the need for human-centered
evaluative research. In disaster risk management, discussions with technical
experts reveal the need for careful consideration of potential negative
impacts and the identification of mitigation strategies.

##### Theme 3: Understanding and Documenting Data

This theme addresses the challenges and importance of understanding and
documenting the data that fuel AI models. Recognizing that the origins,
development, intent, and evolution of datasets influence the resultant models,
researchers propose a shift from merely documenting datasets towards
documenting data production.

One notable contribution is the proposal of “Data Cards,” structured summaries
of essential facts about ML datasets, designed to support transparent,
purposeful, and human-centered documentation of datasets. The goal is to make
data documentation a user-centric product, providing stakeholders with the
critical information necessary for responsible AI development.

##### Theme 4: User Engagement

The final theme highlights the importance of user engagement in AI systems. A
growing interest among researchers and practitioners lies in user-engaged
approaches to algorithm auditing. The challenges faced by industry
practitioners, such as recruiting and incentivizing user auditors, scaffolding
user audits, and deriving actionable insights from user-engaged audit reports,
are thoroughly discussed.

Further, the studies suggest a complex relationship between practitioners and
user auditors, which represents organizational obstacles to user-engaged
auditing. Addressing these obstacles could lead to more robust and user-
responsive AI systems, amplifying the potential for effective AI auditing and
user-focused AI implementation.

In conclusion, the themes identified through the thematic analysis reveal the
multifaceted nature of AI ethics research, which is as concerned with
technical aspects as it is with the ethical implications, practical
applications, understanding and documenting data, and engaging users in the AI
systems lifecycle. The interplay of these themes will undoubtedly continue to
shape the discourse in the field.

###  B.2. Research Methods of Research Papers

Here we asked ChatGPT to about the research methods used in the abstract of
the 164 papers using this prompt: “what are the research methods used the all
the previous prompts?” Below is the answer:

  * •

Interviews: Semi-structured interviews were a common method across several
summaries. They were used to gather information about experiences and
perspectives from various stakeholders such as data workers, industry
practitioners, community organizers, and clinic staff.

  * •

Participatory Design and Co-Design Workshops: These methods involve users or
stakeholders in the design process to ensure that the resulting system or
product meets their needs and is usable in the intended context.

  * •

Prototyping: This approach involves creating early samples, models, or
releases of a product to test a concept or process.

  * •

Observation: Some of the summaries mentioned direct observation as a way to
understand current workflows and contexts, such as in the case of observing
eye-screening workflows in Thai clinics.

  * •

Critical Discourse Analysis: This is a qualitative research method used to
analyze written or spoken language within its social context. One study used
this method to scrutinize data ethics codes.

  * •

Iterative Co-Design Activities: This approach combines co-design, which
involves stakeholders in the design process, with iteration, repeating a
process to refine and improve the outcome.

  * •

Characterization of Collective Actions: This method was used to analyze a
collection of documented actions to identify patterns, themes, or trends.

  * •

Case Studies: Several summaries reported the use of case studies, which
involve in-depth investigations of a single individual, group, or event. This
approach can provide a detailed context to understand complex issues.

  * •

Prospective Evaluations of Model Accuracy: These evaluations involve testing a
model’s performance against a pre-established benchmark or goal.

  * •

Convening Technical Experts for Discussion: In one summary, experts were
brought together to discuss potential negative impacts of AI in disaster risk
management.

###  B.3. Future Research Recommendations Based on the Research Papers

Here we asked ChatGPT to about the research recommendations and future
research directions based on the abstract of the 164 papers using this prompt:
“what are some the research recommendation and future research directions
based on the thematic analysis ?” Below is the answer:

  * •

User-Centric Design: It’s evident that user-focused design needs to be central
in the development of AI systems. Future research can focus on understanding
the needs of different user groups, developing better tools to engage users in
the design process, and exploring ways to make AI systems more transparent and
understandable.

  * •

Greater Transparency in AI: The need for more transparency in AI systems was
highlighted in the data. Future research can explore methods and techniques to
improve transparency, including the development of comprehensive documentation
practices and standards.

  * •

AI in Marginalized Communities: The impact of AI and data technologies on
marginalized communities was a key concern. Future work can focus on
understanding these impacts in greater detail, developing technologies that
support these communities, and exploring methods to ensure that their needs
and concerns are considered in the design of AI systems.

  * •

Collective Action in Tech Industry: The potential power of collective action
within the tech industry to address ethical issues was highlighted. Future
research can explore this further, looking at how to support and facilitate
such actions and understanding their impacts and effectiveness.

  * •

Real-World AI Evaluations: The need for more real-world evaluation of AI
systems was evident. Future research can focus on developing methodologies for
these evaluations, exploring the impacts of real-world contexts on AI
performance, and understanding how AI systems can be effectively integrated
into existing workflows.

  * •

AI in Disaster Risk Management: AI has potential to assist in disaster risk
management, but more research is needed to understand its possible negative
consequences. Future work could focus on investigating these consequences,
developing strategies to mitigate potential harms, and building frameworks for
responsible use of AI in this context.

  * •

Effective AI Communication: Developing effective communication strategies for
AI, particularly for non-experts, is critical. Future research should
investigate how to best communicate about AI, its capabilities and
limitations, and the ethical issues it raises.

  * •

AI Ethics Integration in Education: The integration of AI ethics into
educational programs is a key area for future exploration. Research could
focus on developing and evaluating curricula, teaching methods, and
assessments that effectively engage students in these topics.

  * •

Work-Integrated Learning for AI: Exploring work-integrated learning in the
context of AI is a promising area for future research. This could involve
studying the effectiveness of different models of work-integrated learning,
and how they can be used to prepare students for ethical challenges in the AI
industry.

  * •

AI Governance Practices: Future work should delve into establishing effective
AI governance practices. This includes studying how AI can be regulated, what
forms of oversight are most effective, and how organizations can implement
ethical AI practices.

###  B.4. Thematic Analysis for Patents

ChatGPT generated the following text with minor stylist changes using all the
abstracts of the 28 patents in our review. We first asked for “produce a
report based on the thematic analysis proposed by Braun and Clarke (2008),”
ChatGPT produced the following paragraphs with this prompt:

  * •

AI-driven decision making and predictions: This theme appeared as a prevalent
aspect in many of the patent abstracts. In these cases, AI technologies are
being utilized to predict and inform decisions in various areas, such as media
content, environmental data, brain state, and security provisioning. For
example, one of the inventions predicts moral insight scores from media
content using an AI model. This theme demonstrates the value and potential of
AI in shaping future decision-making processes in diverse fields.

  * •

Transparency, Interpretability, and Explainability in AI: This theme
encapsulates the patents that aim to make AI models more understandable and
transparent. Given the often ’black-box’ nature of AI systems, efforts are
being made to develop systems that can provide explanations for the output
they produce, thus increasing their reliability, credibility, and usability.
For instance, one patent describes a system that couples interpretable and
non-interpretable systems to provide explanations for AI-driven decisions.
This theme underlines the importance of accountability and interpretability in
the development and application of AI systems.

  * •

Machine Learning Model Performance & Enhancement: A subset of patents deal
with improving the performance and capabilities of machine learning models,
ranging from accuracy and efficiency to robustness and fairness. For instance,
one patent discusses using a stochastic process and a novel hypothesis test to
compare the performance of black-box models, providing a method to derive
global and local explanations. This theme underscores the ongoing innovation
in the development and refinement of machine learning models, which is
critical to maximizing their utility and application in different fields.


11institutetext:  Shiraz University of Technology, Shiraz, Iran
22institutetext: Tehran University, Tehran, Iran
33institutetext: Khajeh Nasir Toosi University of Technology, Tehran, Iran
44institutetext: Atomic Energy High School
44email<EMAIL_ADDRESS>44email<EMAIL_ADDRESS>
44email<EMAIL_ADDRESS>44email<EMAIL_ADDRESS>
44email<EMAIL_ADDRESS>44email<EMAIL_ADDRESS>
44email<EMAIL_ADDRESS>
# Cyrus 2D Simulation Team Description Paper 2016

Nader Zare 33 Ashkan Keshavarzi 22 Seyed Ehsan Beheshtian 11 Hadi Mowla 11
Aryan Akbarpour 44 Hossein Jafari 11 Keyvan Arab Baraghi 11 Mohammad Amin
Zarifi 11 Reza Javidan 11

###### Abstract

This description includes some explanation about algorithms and also
algorithms that are being implemented by Cyrus team members. The objectives of
this description is to express a brief explanation about shoot, block, mark
and defensive decision will be given. It also explained about the parts that
has been implemented. The base code that Cyrus used is agent 3.11.

###### Keywords:

RoboCup Soccer Simulation 2D Shoot.

##  1 Introduction

Cyrus robotic team members founded this team five years ago with the goal of
student scientific improvement in Artificial Intelligence and multi agent
field. At first members of this team were bachelors of Information Technology
school of Shiraz University of Technology and presently two of them are
studding master degree in Tehran University and Khajeh Nasir University. This
team has got permission to participate in world competition years 2013 to 2015
and could get 8th, 5th and 9th rank sequentially Also in 2014 achieved First
place of Iran Open competition, First place in Kordestan 2013 and First place
in Fazasazan 2012. This team has also participated in IranOpen 2013, IranOpen
2012, Sharif Cup 2012, IranOpen 2011, Sama RoboCup 2011 and etc.

Presently Cyrus team members are trying to improve team source code with
Artificial Intelligence and multi agent algorithms. In this description shoot
behavior which is implemented by rough neural networks algorithm is explained
and also by using a simulation software developed with centralized processing
of different defensive decision algorithm are being tested. We selected an
optimized method of defensive decision and explained it which isn’t
implemented yet.

##  2 Shoot

In paper [1] implement the shoot behavior with use neural network have one
hidden layer. Briefly, at first in this method goal area will divide to 28
points with equidistant. Then for creating dataset of agent’s opponent
positions, use ball position as input and best target of 28 points as output
of neural network will be used, also use the best target in each run to learn
weights of neural network. Best target due to the location of the ball and the
opponent will calculate in this way that first, the path of the ball towards
the specific goal is simulated and determines whether it is possible that the
ball takeover by the opponent in this path or not. To clarify this point,
location of the ball in each cycle calculated according to the initial speed
of the ball toward the goal and ground friction. If before the ball crossing
the goal line, in none of the point opponent can’t reach that point then shoot
reach the target. Otherwise shoot can’t reach the target. After determining
the status of all targets, the longest period of consecutive targets that are
caused the ball reach the goal area is chosen as the best period for shooting
and the center of this period is chosen as the best target for shooting. The
number of target to be selected as the main output of intended state. This
issue is shown in Figure1.

Figure 1: Target of shoots in paper [3] method.

As mentioned previously, for learning neural network the status which consist
of opponent’s location and location of the ball, to be considered as the input
and number of the best target to be considered as the output of intended
neural network. This approach has three problems:

  1. 1.

This approach finds the best target if only one player would have existed.

  2. 2.

The goal is obtained may not be the best goal in the dataset.

  3. 3.

The opponent’s action isn’t affected the algorithm, when this opponent has not
seen yet.

In the next section, the algorithm to find the best target is described.

###  2.1 Cyrus shoot algorithm

In the shoot algorithm implementation in Cyrus team, in the first step the
opponents’ position and also ball position evaluation will be calculated for
each shoot to specific target. Then the shoot with highest evaluation number
is chosen as the best possible shoot to achieve the goal. In this algorithm,
the first ball movement to the specified target is simulated, then opponent
player’s movement based on the ball path will be simulate and time that need
to opponent reach this path will be calculate to find best shoot base on
evaluations. World model of soccer 2D simulation has uncertain and in
observable environment. So we cannot tell for sure that shot is going to be
scored or not. At first we will evaluate the targets with some parameters
like: position of the opponent agents, opponent agent’s type, ball position
and the maximum initial velocity of the ball. For evaluating the targets of
the shoot we will use the above parameters and simulate the ball path to the
target position as way ball position in each cycle to be clear. Then in this
algorithm, the time is opponent need to reach the ball will be calculated.

Figure 2: Shoots can be reach the targets and opponent reach time.

In first step as described in previous sections, this algorithm is scoring all
point of the path from ball to the target and output of this algorithm is
shown in figure 2.

###  2.2 Using neural networks

The goal of using neural networks in the Cyrus shoot algorithm is to obtain
evaluation of each shoot to goal based on one player. In the Cyrus team
because of calculation time in the soccer 2D simulation environment to
simulate opponent reach time to the shoot path is much expensive and we use
neural network. Presently we used opponent player’s positions, target position
and ball position as input of our neural networks, and also first ball speed
is considered 3.0 but in future before world competition we want to add body
angle, opponent player’s first speed and shoot speed to learn neural networks.
Also in soccer 2D environment, poscount has direct impact on shoot behavior
and other behaviors in the soccer 2D simulation environments. To optimize
neural networks output with use of poscount, multiplied with a certain factor
and add to output of neural networks. Also this factor will be calculated by
reinforcement learning in huge of games dataset.

Figure 3: Add poscount to neural network output.

###  2.3 Comparison

In next sections and figure 7 a compare between shoot implemented in agent2D
[2] version 3.1.1, paper [1] and Cyrus implemented shoot is shown:

Table 1: compare shoot implementation. Algorithms: | Agent2D | Paper[1] | CYRUS NN
---|---|---|---
Win Rate | 48.6% | 55.3% | 59.2%

###  2.4 Optimization using pattern recognition

For optimizing data from previous method first we turn our data into patterns
and then we use our new data set as our neural networks inputs. In this method
our previous data consisting of ball position, opponent players and target
position are turned into ball to target distance, ball to opponent distance
and the angle between ball-target and ball-opponent vectors. In this method
symmetry property of neural networks is kept and better results are obtained.
In figure 8 we will see errors has been obtained by learning our two sets of
data.

Figure 4: Errors obtained by learning our two sets of data.

##  3 Future Works

Cyrus team wants to use rough deep neural networks for offensive decision
making. In this method we want to use Helios [4], WrightEagle [5], Gliders [6]
and Oxsy [7] binaries to use player positions as input and player’s behavior
as output of neural networks. After learning the neural networks, we want to
use reinforcement algorithm to optimize this neural networks. For optimizing
deep neural networks by reinforcement algorithm we want to divide field into
different parts and according to ball reaching to those places based on the
dangerous and importance of each region we will reward each player and then
with help of these rewards we will update weights of our neural networks. This
method can be used in three different ways:

  * •

Learning other teams’ offensive behavior and using it for defensive decisions.

  * •

Each of our players can predict its teammate’s behavior.

  * •

We can improve our team’s offensive decisions by Re-Enforcement Learning.

We also want to use generalized kalman filter’s algorithm online to predict
team- mate and opponent positions. In this algorithm we use player position in
each cycle as input of our neural networks and get its next position and by
observing its next real position. We will compare our prediction to reality
and learn our neural networks while running. In the live game we want to
reduce poscount impact by using this method and also in offensive time by
using chain-action we can improve the ball holder’s future vision.

## References

  * [1] Training a Simulated Soccer Agent how to Shoot Using Artificial Neural Network. M. Dezfoulian, N. Kaviani, A. Nikanjam, M. Rafaee, 13th Multi-disciplinary Iranian Researchers Conference in Europe (IRCE), Leeds, UK, 2005.
  * [2] “Agent 2D-3.1.0 RoboCup tools - OSDN.” [Online]. Available: http://en.osdn.jp/projects/rctools/downloads/51943/agent2d-3.1.0.tar.gz/. [Accessed: 22-Jan-2016].
  * [3] Teshneh Lab, Mohammad, and Pouria Jafari. Neural Networks and Advanced Neuro-Controlers, A Rough Neural Network Approach. Khajeh Nasir University of Technology, 2015.
  * [4] H. Akiyama, T. Nakashima, and S. Mifune, “HELIOS2015 Team Description Paper,” pp. 1–6, 2015.
  * [5] H. Zhang, M. Jiang, H. Dai, A. Bai, and X. Chen, “WrightEagle 2D Soccer Simulation Team Description 2015,” Wrighteagle.Org, pp. 3–8, 2015.
  * [6] M. Prokopenko, P. Wang, and O. Obst, “Gliders2015 : Opponent avoidance with bio- inspired flocking behaviour,” pp. 1–5, 2015.
  * [7] S. Marian, D. Luca, B. Sarac, and O. Cotarlea, “OXSY 2015 Team Description,” 2015.
  * [8] Akiyama, H., Noda, I.: Multi-agent positioning mechanism in the dynamic environment . In: Visser, U., Ribeiro,F., Ohashi, T., Dellaert, F., eds.: RoboCup 2007: Robot Soccer World Cup XI, Lecture Notes in Artificial Intelligence. Volume 5001., Springer (2008) 377-C384.
  * [9] Slotine, Jean-Jacques E., and Weiping Li. Applied nonlinear control. Vol. 60. Englewood Cliffs, NJ: Prentice-Hall, 1991.
  * [10] Haykin S., ”Neural Networks: A Comprehensive Foundation (2 ed.)”, Prentice Hall, 1998.
  * [11] N.Zare, M.Karimi, A.Keshavarzi, E.Asali, H.Alipour, A.Aminian, E.Beheshtian, H.Mowla, H.Jafari, M.J.Khademian, “CYRUS 2D Simulation Team Description Paper 2015”, The 19th annual RoboCup International Symposium, China, Hefei, 2015\.
  * [12] R.Khayami, N.Zare, M.karimi, P.Mahor, F.Tekara, E.Asali, A.Keshavarzi, A.Afshar, M.Asadi, M.Najafi, “CYRUS 2D Simulation Team Description Paper 2014”, The 18th annual RoboCup International Symposium, Brazil, Joao Pessoa, 2014\.