# Understanding Diversity in Human-AI Data: What Cognitive Style
Disaggregation Reveals

Andrew Anderson<EMAIL_ADDRESS>0000-0003-4964-6059 ,  Rupika
Dikkala<EMAIL_ADDRESS>Oregon State University ,  Tianyi Li
<EMAIL_ADDRESS>Virginia Tech ,  Mihaela Vorvoreanu
<EMAIL_ADDRESS>Microsoft and  Margaret Burnett
<EMAIL_ADDRESS>Oregon State University

(9999)

###### Abstract.

Artificial Intelligence (AI) is becoming more pervasive through all levels of
society, trying to help us be more productive. Research like Amershi et al.’s
18 guidelines for human-AI interaction aim to provide high-level design
advice, yet little remains known about how people react to Applications or
Violations of the guidelines. This leaves a gap for designers of human-AI
systems applying such guidelines, where AI-powered systems might be working
better for certain sets of users than for others, inadvertently introducing
inclusiveness issues. To address this, we performed a secondary analysis of
1,016 participants across 16 experiments, disaggregating their data by their 5
cognitive problem-solving styles from the Gender-inclusiveness Magnifier
(GenderMag) method and illustrate different situations that participants found
themselves in. We found that across all 5 cogniive style spectra, although
there were instances where applying the guidelines closed inclusiveness
issues, there were also stubborn inclusiveness issues and inadvertent
introductions of inclusiveness issues. Lastly, we found that participants’
cognitive styles not only clustered by their gender, but they also clustered
across different age groups.

Intelligent User Interfaces, Human-Computer Interaction

††copyright: acmcopyright††journalyear: 9999††doi:
99.9999/9999999.9999999††journal: TIIS††journalvolume: 99††journalnumber:
99††article: 99††publicationmonth: 99††ccs: Human-centered computing User
studies††ccs: Computing methodologies Intelligent agents

##  1\. Introduction

Consider the following scenario:

Timara sits at their desk, ready to start another day of work. They’re working
on a presentation, and the software asks “Would you like to use a new AI
feature that automatically generates content for your presentation based on
what you’ve done?” They excitedly clicks yes, thinking about how much time
this feature might save them. The next day, Abishek, Timara’s partner, is
working on a different presentation. The software asks him the same question,
but he declines, thinking of all the things that could go wrong and how much
time he would have to spend fixing its errors.

The above situation illustrates how two people might respond very differently
to interacting with an AI-powered feature. Since no two people are exactly
alike, the ways users and stakeholders of AI systems interact with and respond
to AI are different, complex, and nuanced. AI has already made waves for
sometimes having unintended disparate impacts on some sub-populations, which
are further aggravated by those sub-populations being already marginalized in
society.

Despite a lot of existing research seeking to improve human-AI interaction,
such as in medical assistance technology (Gandhi et al., 2019; Lin et al.,
2019; Jacobs et al., 2021), autonomous vehicles (Haboucha et al., 2017;
Schneider et al., 2021), productivity software (Sundaresan, 2019), and more,
little remains known about how people’s cognitive style influences their
approach to these technologies and how they might feel while interacting with
the AI. The Gender-inclusiveness Magnifier (GenderMag) method has helped make
interfaces for non-AI powered interfaces more inclusive for people with
diverse cognitive styles GenderMag is an inclusive design and evaluation
method, and it has revealed through its 5 cognitive style (facet) spectra that
people on each of the 5 dimensions can behave and react to technology very
differently from each other (Padala et al., 2020; Steine-Hanson et al., 2019).
It has also been effective at identifying and affording design remedies to
user workflows that disproportionately raise obstacles to one gender more than
another (Burnett et al., 2016). In this paper, we illustrate that
disaggregating human-AI data by GenderMag’s cognitive styles can reveal
previously undiscovered insights into potential disparities to inform human-AI
designers.

Our investigation sought to answer the following research questions:

  1. (1)

RQ-Cognitive-Diversity: _What insights can be uncovered for the guidelines for
human-AI interaction through cognitive style disaggregation?_

  2. (2)

RQ-Demographics: _How do the cognitive styles align with participants’ other
demographics?_

To accomplish this, we performed a secondary analysis of data from 16
experiments, performed by a group of researchers at Microsoft Research (Li et
al.) to evaluate the effects of applying (or violating) Amershi et al.’s
guidelines for human-AI interaction (Amershi et al., 2019). A secondary
analysis is analysis of data “which were originally collected for other
purposes” (Glaser, 1963; Johnston, 2014). Doing so enables the analyst to
pursue a research interest that is distinct from that of the original work. We
disaggregate these data through by five cognitive styles from the GenderMag
method (Burnett et al., 2016) to investigate how participants with diverse
cognitive styles may have reacted differently to various fictional AI
products.

This paper contributes the following:

  * •

an investigation into how disaggregating by cognitive styles can yield new
insights about how applying or violating human-AI interaction guidelines
impact the user experience of diverse users;

  * •

illustrations of the approach’s utility on data from 16 human-AI experiments;
and

  * •

an investigation into how cognitive styles of the 1000+ participants from
these 16 experiments relate to gender- and age-diversity demographics.

##  2\. Background & Related Work

Table 1. The personas of the Gender Inclusiveness Magnifier (GenderMag) method
with their facets and values (Burnett et al., 2016). People’s cognitive
problem-solving styles fall along 5 different spectra in the GenderMag method.
the orange values are assigned to the Abi persona and the blue values are
assigned to the Tim persona, as per statistical clusterings reported in
underlying research (e.g., (Burnett et al., 2016; Stumpf et al., 2020;
Vorvoreanu et al., 2019)).

###  2.1. Gender-Inclusiveness Magnifier (GenderMag) method: MMB: D1. MMB
needs to work on it

One of the underpinnings of this paper is the GenderMag cognitive styles
(facets). GenderMag (Burnett et al., 2016) is an inclusive design and
evaluation method that has been shown to be effective at identifying and
affording design remedies to user workflows that disproportionately raise
obstacles to one gender more than another (Burnett et al., 2016). (Usually,
but not always, the gender affected most negatively is women.) At the core of
the method is five cognitive styles (facets) for each of 3 personas (Abi, Pat,
and Tim):

  * •

Attitude towards Risk when using technology

  * •

Computer Self-Efficacy (confidence) about using unfamiliar technology

  * •

Motivations for using technology

  * •

Information Processing Style for gathering information to solve problems

  * •

Learning Styles for learning new technology

Each facet captures a spectrum of cognitive style values (Table 1). The table
shows three examples of the many possible configurations of facet values an
individual could have. For example, Abi wants what tech can accomplish, is a
comprehensive information processor, and so on, whereas Tim sees tech as a
source of fun, is a selective information processor, and so on.

The GenderMag facets have not only been very effective at identifying such
issues, but also have been reasonably accurate in connecting these issues with
the cognitive styles of those who experience them. For example, in one study,
GenderMag-identified cognitive facets tied with issues found agreed 75% of the
time with empirical users’ self-reported facet values (Vorvoreanu et al.,
2019).

The self-reported facet values in the above study were collected using a facet
survey, which was included in the paper’s Supplemental Documents (Vorvoreanu
et al., 2019). That survey was a streamlined version of a more complete
survey. The study by Li et al. that produced our data (described in Section 3)
used the complete survey to collect their participants’ self-reported facets.
The 36 question survey asks questions that together map participants to a
location along each of the five facets’ spectra. Our analysis uses the
responses to this survey to disaggregate by these facets as per the
calculation process described in Section 3.2.2. We include the full survey in
Appendix B.

Research related to our own broadly fit into two categories: 1) work on
equitable and fair AI, 2) work on human-AI interaction.

###  2.2. Equitable and Fair AI

The first significant body of work that seeks to help AI systems perform more
inclusively and equitably deals with fairness in artificial intelligence. This
area has seen a surge in publications in the last decade, with entire
conferences now devoted to reporting on such issues (e.g., the ACM Conference
on Fairness, Accountability, and Transparency (FAccT)). Some common approaches
to helping with how equitable and inclusive artificial intelligence include
diversifying training data, the model, or the inference that the model makes
(Gong et al., 2019). For example, take Larson et al.’s reporting of unfair
behavior in the COMPAS recidivism dataset (Larson et al., 2016). Crawford
discussed this in her keynote (Crawford, 2017) and noted that fairness in AI
is a socio-technical problem with many types of harms to try and resolve, such
as: quality-of-service harms, harms of allocation, harms of representation,
and more. These harms continue to be a problem, such as Robertson, Nguyen, and
Salehi’s recent report on how matching students to schools might not fulfil
their purpose (Robertson et al., 2021).

Although not a comprehensive list, some of these harms have arisen from AI
struggling to recognize voices across multiple demographic dimensions (e.g.,
sex or dialects); Tatman found disparities in YouTube’s automatic captioning
feature between male and female voices, dialects from different areas
(Scottish dialects’ error rates were the highest), and the interaction between
them (Tatman, 2017). They discussed how disparities like these demonstrate
damage inclusiveness, since errors in such systems are of immediate concern to
the likes of the Deaf community.

Harms also arise in Natural Language Processing (NLP) systems of textual
information. For example, Leavy reported on gender bias in NLP through 5
different dimensions, such as naming (e.g. “female lawyer” vs. “lawyer”),
ordering (e.g., having man before woman in “man/woman”), and biased adjectives
(e.g., use of the term “gossiping” for women vs. “affable” for men). These
findings highlight potential negative stereotypes, which disparately impact
certain sub-populations (in Leavy’s case, females). Sun et al. provide a more
comprehensive review of problems and mitigation strategies for NLP than is
represented here (Sun et al., 2019), but they touched on what Gong et al.
identified — the need for diversifying training data (Zhao et al., 2019; Zhao
et al., 2018), the model (Zhang et al., 2018), and the inference (Lu et al.,
2020; Kiritchenko and Mohammad, 2018) .

Research has also uncovered biases in image data for gender, race, and their
intersection. Buolamwini & Gebru found that the image classifiers they
investigated performed better on male faces vs. female faces, and they
performed worst on darker female faces (Buolamwini and Gebru, 2018). Also,
Rose reported on a phenomenon with Nikon cameras, where the camera mistakenly
believed that people of Asian descent were blinking if they were smiling for
the photo (Rose, 2010). Some of the potential disparate impacts of such
systems for speech, text, and image data for people with disabilities are
reported on in Guo et al.’s position paper (Guo et al., 2019). These works
investigate how people are potentially impacted by the decisions that AI make,
and they are only a handful of the rich research dealing with the biases
discovered in AI (Nasr and Tschantz, 2020; Yang et al., 2020) However, for
more comprehensive surveys of fairness and algorithmic accountability in AI
literature, see either Mehrabi et al. or Wieringa (Mehrabi et al., 2019;
Wieringa, 2020). Our work differs by investigating how people react to an AI’s
decisions through user studies, the body of literature we turn to next.

###  2.3. Users and Artificial Intelligence

The problem of unfairness in some AI systems points to a need for transparency
in AI through explanations, which might empower users to at least notice and
potentially understand the instances of unfairness. A resurgent subarea,
eXplainable AI (XAI), aims to bring transparency to AI by helping AI’s users
form reasonably correct mental models. (In the context of user experiences
with AI, a mental model is someone’s internal representation of how an AI
works (Norman and Gentner, 1983).) For example, Lim & Dey researched
intelligibility types to leverage the information that people asked a system
to inform them of what information explanations might contain (Lim et al.,
2009; Lim and Dey, 2009; Lim, 2012), and Kulesza et al.’s worked on a set of
principles of how developers might explain intelligent systems (Kulesza et
al., 2015). One area of research in XAI has looked at the granularity of
explanations, such as global explanations (how the system reasons about
categories of choices (Wu et al., 2020; Mikołajczyk et al., 2020; Popordanoska
et al., 2020)), local explanations (explaining specific choices that the
system makes (Plumb et al., 2018; Guidotti, 2021; Nguyen, 2018)), or both
(Setzu et al., 2021; Hohman et al., 2019; Lundberg et al., 2020). However,
there are also myriad ways to explain either of these explanation types, such
as counterfactual reasoning (showing what needed to have changed to create a
different outcome (Goyal et al., 2019; Mothilal et al., 2020; Neal et al.,
2018)), saliency explanations (showing what the AI was “looking at” when
making its decision (Anderson et al., 2020; Alqaraawi et al., 2020; Kenny et
al., 2021)), and Q-value reward explanations (the AI’s “crystal ball” looking
to some future desired outcome (Anderson et al., 2020; van der Waa et al.,
2018)) 111A full survey of the XAI literature is beyond the scope of this
paper, but such surveys exist, like Mueller et al. (Mueller et al., 2019)..
Such works focused on how people’s mental models changed based on exposure to
such explanations. Our work does not measure mental models, instead measuring
diverse people’s reactions toward AI systems applying or violating a human-AI
guideline.

The works most closely related to this paper involves research on how well AI
serves different populations and research on users’ attitudes or reactions to
an AI system. For example, Pal et al.’s work investigating satisfaction levels
between native English speakers versus non-native English speakers using voice
assistants (Pal et al., 2019) found that the participants who were native
English speakers were significantly more satisfied with their experience than
the non-native speakers, which relates to the work on algorithmic performance
with accents done by Tatman (Section 2.2). van Berkel et al.’s study on
attitudes toward fairness (recidivism and loans) found gender differences: the
men in their study were significantly more likely to say that systems were
fair in both loan-provision and recidivism domains than women (van, [n.d.]).
Another work similar to our own is Gillath et al., who found a significant
difference among difference age groups, where older people were less likely to
trust AI (Gillath et al., 2021).

Other researchers have measured users’ trust of AI systems. For example,
Madumal et al. found no differences in trust among 4 possible explanations
(Madumal et al., 2019), and Juravle et al. found that medical practitioners
trusted AI-assisted diagnoses significantly less than human-assisted diagnoses
(Juravle et al., 2020). In investigating the paradigm of social transparency’s
usefulness for XAI, Ehsan et al. found that this paradigm assisted their
participants in calibrating trust in AI systems, improved AI-mediated
decision-making, and helped explanations. All of their participants had direct
knowledge of using, developing, or designing AI systems, implying a
fundamental knowledge of the probabilistic nature of such systems; in
contrast, few of our paper’s users had this kind of background.

###  2.4. 18 Guidelines for Human-AI interaction

Our work performs a secondary analysis of empirical data that had been
collected to gain insights into user preferences and perceptions around
Amershi et al.’s guidelines for human-AI interaction (Amershi et al., 2019)
(included in Figure 1). To develop their guidelines, Amershi et al. started
with 168 HCI design guidelines from the literature, and distilled from that
set 18 guidelines for human-AI interaction. The guidelines provide high-level
advice for designers through providing a name (e.g., “Make clear what the
system can do”), a brief description of what it means (e.g., “Help the user
understand what the AI system is capable of doing”), and an example of the
guideline in a real interface (such as PowerPoint’s QuickStarter feature). A
group of researchers at Microsoft Research (Li et al.) then performed a set of
follow-up experiments evaluating the effects of following these guidelines on
AI users’ experiences. Our work contributes additional information from Li et
al.’s data by considering them from the perspective of people’s cognitive
diversity.

Figure 1. Amershi et al.’s 18 guidelines for human-AI interaction (Amershi et
al., 2019). For the 4 phases (left column), each guideline has a number,
title, and brief description. Our analysis excludes the two guidelines (greyed
out) whose experiments did not pass the manipulation check.

##  3\. Methodology **AAA - D2

Table 2. Guideline 1’s vignettes for application (left) and violation (right).
Each vignette had 3 components. (1) a product and feature introduction. (2)
the behavior description of the AI-powered component. (3) the AI performance
description. Li et al.’s original work dropped (3) from their analyses, since
AI-optimality was not a significant factor in the experiments, so we also drop
it from our analyses. (Appendix C shows all 16 experiments’ vignettes.)

###  3.1. Summary of Li et al.’s Methodology

Li et al. ran 18 independent 2 x 2 factorial experiments, one for each of the
guidelines for Human-AI interaction (Amershi et al., 2019). In total, 1,043
Amazon Mechanical Turk (MTurk) participants were randomly assigned to one of
these 18 experiments; the average sample size per experiment was 58.

Each experiment used adherence to the guidelines as a within-subjects factor
(levels: Guideline Violation & Guideline Application) and AI-quality as a
between-subjects factor (levels: optimal & sub-optimal). They analyzed each
experiment’s overall performance with manipulation checks and discarded two
experiments (Guideline 2’s and Guideline’s 16) which did not pass these
checks. Our secondary analysis also uses the 16 experiments that passed these
checks.

As with some other AI research (e.g., (Lima et al., 2021)), Li et al.
represented their AI system through vignettes; Table 2 shows the vignettes for
Guideline 1: make clear what the system can do. Each vignette had 3 parts: (1)
Product and feature introduction — introducing the fictional product and its
AI-powered feature that the participant would “interact” with, (2) product
behavior description — describing the systems behavior. Notice how these
descriptions differed between the guideline Application or Violation to
manipulate the independent variable, and (3) AI performance description —
providing a statement about how well the product performed. Li et al.’s
original analysis found insufficient interaction effect for the AI’s
optimality, so we do not consider AI optimality in this paper.

Participants in each experiment randomly saw one of the vignettes (either the
Application or Violation) and answered a set of questions about that vignette.
Then, they saw the other vignette and answered the same set of questions about
it. After seeing both, they were asked which of the vignettes they preferred
and to explain why they preferred it. In each experiment, participants saw
both the Application and Violation vignettes in a random order and responded
to the 10 questions in Table 3 for each vignette. These questions asked how
participants would feel while using the product (Benedek and Miner, 2002), how
useful they thought the product would be and how much they would trust the
product (Sheridan, 1989; Jian et al., 2000). Lastly, participants filled out a
demographics survey and responded to the GenderMag facet survey (See Appendix
B).

Dependent Variable | Wording
---|---
Feel In Control | “I would feel in control while using the product.”
Feel Secure | “I would feel secure while using the product.”
Feel Adequate | “I would feel adequate while using the product.”
Feel Certain | “I would feel certain while using the product.”
Feel Productive | “I would feel productive while using the product.”
Perceived Useful | “I would find the product useful.”
Not Suspicious |  “I would not be suspicious of the intent, action, or outputs of the product.”
Not Harmful |  “I would not expect the product to have a harmful or injurious outcome.”
Product Reliable | “I would expect the product to be reliable.”
Trust Product | “I would trust the product.”
Table 3. The 10 analyzed dependent variables in the scope of our paper.
Participants answered these 7-point agreement scale questions twice – once for
both the Violation product and Application product – in a random order. The
following were reverse-coded: Feel Adequate, Feel Certain, Not Suspicious, Not
Harmful.

###  3.2. Our Secondary Analysis Methodology

####  3.2.1. Our subset of Li et al.’s participants

Of Li et al.’s 1,043 participants who passed the experiment attention checks,
27 did not pass at least one of the attention checks in the GenderMag facet
survey, leaving a total of 1,016 participants. They came from all walks of
life, from different ages (18 — 74), education (less than high school —
Masters or higher), occupation, and gender-identity. When we refer to the term
“gender,” we mean the gender identity/identities that participants reported to
us. This meant that they could select as many of the gender identities that
described them best.

####  3.2.2. Deriving Cognitive Style

Participants responded to the GenderMag cognitive style survey through 9-point
agreement scales for 36 statements, with an option to reply with “I don’t
know” if needed.

Cognitive Style | Comparison | If True: | If False:
---|---|---|---
Attitude Towards Risk |  $>$ median | more risk-averse | more risk-tolerant
Computer Self-Efficacy |  $<$ median | lower | higher
Information Processing Style |  $>$ median | more comprehensive | more selective
Motivations |  $<$ median | more for task | more for fun
Learning Style |  $<$ median | more by process | more by tinkering
Table 4. The rules for each cognitive style, used to categorize which
cognitive style (Table 1) participants exhibited from their survey responses
compared to the other participants in the same experiment.

For the set of questions measuring each cognitive style, we summed up the
responses to the questions that were not “I don’t know” to create 5 scores,
one for each cognitive style, multiplying each one by the percent of the
questions they had actually answered for that style. (For example, if their
self-efficacy “score” was 10, but they answered half of the self-efficacy
questions with “I don’t know”, their score became 10/0.5 = 20.) Finally, for
each facet in each experiment, we separated participant facet values at the
median score for that facet. For example, for attitudes toward risk, those who
were above the median were categorized as more  risk-averse than their  risk-
tolerant peers whose scores were below the median. See Table 4.

####  3.2.3. Testing Methodology

With 16 independent experiments, it is important to establish a testing
paradigm that maintains rigor and avoid p-hacking. To this end, for each
guideline’s experiment, we selected potential candidate dependent variables by
visually assessing the box plots, excluding plots where there was clearly no
difference in the means. If we ran more than 5 tests for any of the
Application products or Violation products in Section 4 (i.e., more than half
of the dependent variables), we applied Bonferroni correction to our outcomes.

##  4\. Results: Cognitive style impacted guideline responses

In order to find the kinds of insights cognitive style disaggregation can
uncover (our RQ-Cognitive-Diversity), we disaggregated Li et al.’s 16
independent experiments’ data by their GenderMag cognitive styles222Here,
because the guideline experiment was a random assignment to assess within-
subjects the effect of each guideline Application vs. Violation, we did not
combine the experiments’ data..

Since AI has shown itself to be a “risky” technology across myriad domains,
such as self-driving cars (Ajenaghughrure et al., 2020), healthcare (Sunarti
et al., 2021), and replicating biases in data (Osoba and Welser IV, 2017), we
begin our discussion by looking at how participants’ attitudes toward risk
factored into their ratings of different interfaces for various AI products
and found three categories of insights:

  1. (1)

Application AI products sometimes closed inclusiveness gaps from the Violation
AI products.

  2. (2)

However, they could also introduce inclusiveness gaps.

  3. (3)

Some inclusiveness gaps persisted, despite AI product changes.

###  4.1. Attitudes Toward Risk: Three categories of outcomes

####  4.1.1. Category 1: Some AI product changes closed inclusiveness gaps

Figure 2. Legend to understand box plots for attitudes toward risk.

FIXME  Guideline 6 Guideline 7 Guideline 8 Guideline 14 Figure 3. All
dependent variables from experiments where the Application products closed
inclusiveness gaps from the Violation products. The  risk-averse were
significantly disadvantaged by the Violation products.

The Application AI products attempted to improve upon Violation AI products,
and some of the Application products closed inclusiveness gaps that had been
present in the corresponding Violations products (Figure 3). For example, in
Guideline 6’s (mitigate social biases) Violation AI product (the hatched ///
boxes), the  risk-averse participants felt significantly less adequate than
the  risk-tolerant (t(62) = -2.943, p = .0046, d333Cohen’s d, measuring
practical significance, where $d<.2=$ unsubstantiated, $d\in[0.2,0.5)=$ small,
$d\in[0.5,0.8)=$ medium, $d>0.8=$ large = .736). This gap closed with the
Application AI product, in which the same risk-averse participants felt
significantly more adequate (paired t-test, t(31) = 4.981, p $<$ .0001,
d444Within-subjects Cohen’s d -
${|\bar{X}_{1}-\bar{X}_{2}|}/{\sqrt{s_{1}^{2}+s_{2}^{2}-(2rs_{1}s_{2})}}$ =
.874). Further, the gap between the risk-averse and the risk-tolerant
participants closed, with no significant difference remaining between the two
groups’ ratings of their feelings of adequacy (t(62) = -1.059, p = .294, d =
.265). Figure 3 shows other examples of gaps closing across different
experiments; Appendix A.1 shows all data.

Guideline 9 | Guideline 10 | Guideline 11 | Guideline 13 | Guideline 18
---|---|---|---|---
|  |  |  |
|  |  |  |
Figure 4. All dependent variables where both ends of the attitudes toward risk spectrum were helped by the Application product, but one end was helped more than the other. Notably, a majority of the time, the  risk-averse ended up with a significantly lower average rating for the Application product than the  risk-tolerant . Guideline 1 | Guideline 4 | Guideline 14 | Guideline 18
---|---|---|---
|  |  |  |
Figure 5. All dependent variables where only one end of the attitudes toward
risk spectrum were helped by the Application product. Most of the time, the
risk-averse were unsupported by the Application products, but the  risk-
tolerant received significant support from them.

####  4.1.2. Category 2: Some AI product changes introduced inclusiveness gaps

Sometimes, changing the AI product inadvertently opened inclusiveness gaps.
Figure 4 shows the first flavor of this insight, where both risk-averse and
risk-tolerant participants rated the Violation AI product similarly across
several experiments and dependent variables. In all of these cases, both sets
of participants’ average ratings improved with the corresponding Application
products—but not equally. Thus, the Application AI products for these
guidelines introduced an inclusiveness gap, where either the risk-averse (G9,
G11, and G18) or risk-tolerant (G10 and G13) were “left behind”.

For example, consider the “Perceived Useful” dependent variable for Guideline
9—support efficient dismissal in the left column of Figure 4. The risk-averse
and risk-tolerant participants’ rated the Violation AI product’s usefulness
similarly, with no significant difference between the averages (t-test, t(56)
= .5436, p = .589, d = .144). Further, both the  risk-averse (paired t-test,
t(25) = 3.4913, p = .0018, d = .685) and the  risk-tolerant (paired t-test,
t(31) = 6.9778, p $<.0001$, d = 1.224) average ratings underwent significant
changes between the Violation AI product and the Application AI product.
However, the risk-averse participants’ smaller change meant that they rated
the Application AI product as being significantly less useful than the risk-
tolerant (t(56) = -2.206, p = .0315, d = .582) participants did.

The second flavor of this insight category was when only one of the attitudes
toward risk was supported by the Application products (Figure 5). Similarly to
before, in Guideline 1’s (make clear what the system can do) experiment, the
risk-averse and risk-tolerant participants rated their productivity similarly
for the Violation product (t(54) = .6406, p = .524, d = .236). However, only
the risk-averse participants’ ratings significantly changed with the
Application AI product (paired t-test, t(22) = 2.3349, p = .029, d = .512);
the risk-tolerant did not (paired t-test, t(32) = 1.000, p = .325, d = .122),
which left the risk-averse feeling significantly more productive with the
Application product than the risk-tolerant (t(54) = 2.499, p = .016, d =
.670).

Results such as these suggest that although the Application products helped
participants (i.e., their average ratings got a boost between the Violation
and Application products), they did not always do so equally for participants
with diverse attitudes toward risk across different situations.

####  4.1.3. Category 3: Some inclusiveness gaps persisted, despite AI product
changes.

The last category of inclusiveness issues were when significant differences
persisted throughout both the Violation products and the Application products.
Guideline 4’s experiment (show contextually relevant information) in Figure 6
illustrates the phenomenon especially well, with Table 5 showing the
statistics for the stubborn inclusiveness issues. The top table shows that the
risk-averse participants rated the Violation product significantly lower than
the risk-tolerant for 7 of the dependent variables5554 of these remained
significant, even under Bonferroni correction., and the inclusiveness
disparity remained statistically significant for the same dependent variables
with the Application AI product6665 differences remained significant under
Bonferroni correction., suggesting that something about the Application
product was not satisfying the needs of the risk-averse.

Further, since 90% of the dependent variables in Guideline 4 exhibited
significant differences in the Application product, this might help explain
why, when participants were asked in a separate question which of the two
systems they preferred, only 51.5% ($\hat{\pi}_{averse}=0.515$) of the risk-
averse participants preferred Guideline 4’s Application product over the
Violation product (i.e., equal odds777Odds:
$\hat{\omega}_{averse}=\hat{\pi}_{averse}/(1-\hat{\pi}_{averse})$,
$\hat{\omega}_{averse}=1.06$). On the other end of the risk-spectrum, 71.9% of
the risk-tolerant (i.e., $\approx$2:1 odds, $\hat{\omega}_{risk-
tolerant}=2.556$) preferred the Application product, so the odds ratio888Odds
ratio: $\hat{\phi}=\hat{\omega}_{risk-tolerant}/\hat{\omega}_{risk-averse}$
uncovered that the  risk-tolerant participants were more than twice as likely
to prefer the Guideline 4 Application product as the  risk-averse participants
($\hat{\phi}=2.12$).

Figure 6.  Guideline 4 participants’ ratings for the 10 dependent variables,
both in the aggregate (black and white) and disaggregated by the  risk-averse
and  risk-tolerant . For the disaggregated data, notice how large the
distances between the average rating are when the guideline is applied. Table
5 below provides the statistics.  Violation Product
---
Dependent | Distance b/w |  |  |  | Bonferroni |
variable | Means | df | t-statistic | p-value | correction | Cohen’s d
Feel In Control | 0.4444 | - | \- - - | \- - - | \- - - | \- - -
Feel Secure | 0.6667 | 62 | 2.3006 | .0249* | .174 | 0.580
Feel Adequate | 0.8704 | 61 | 2.9179 | .0049* | .034* | 0.743
Feel Certain | 1.2798 | 62 | 3.6258 | .0005* | .0035* | 0.910
Feel Productive | 0.5397 | - | \- - - | \- - - | \- - - | \- - -
Perceived Useful | 0.5119 | - | \- - - | \- - - | \- - - | \- - -
Not Suspicious | 1.6468 | 34 | 4.8630 |  $<.0001$* |  $<.001$* | 1.337
Not Harmful | 0.7893 | 39 | 2.8694 | .006* | .042* | 0.763
Product Reliable | 0.6312 | 49 | 2.0046 | .051 | .357 | 0.518
Trust Product | 0.7519 | 63 | 2.6615 | .010* | .070 | 0.664
Application Product
Dependent | Distance b/w |  |  |  | Bonferroni |
variable | Means | df | t-statistic | p-value | correction | Cohen’s d
Feel In Control | 0.9643 | - | \- - - | \- - - | \- - - | \- - -
Feel Secure | 1.0093 | 61 | 3.1984 | .0022* | .022* | 0.814
Feel Adequate | 0.8492 | 62 | 2.4955 | .0153* | .153 | 0.629
Feel Certain | 1.6916 | 63 | 4.7513 |  $<.0001$* | .0001* | 1.186
Feel Productive | 0.4598 | 63 | \- - - | \- - - | \- - - | \- - -
Perceived Useful | 0.7854 | - | \- - - | \- - - | \- - - | \- - -
Not Suspicious | 1.8008 | 42 | 5.1692 |  $<.0001$* | .0001* | 1.364
Not Harmful | 0.9607 | 44 | 3.1328 | .0031* | .031* | 0.821
Product Reliable | 0.8170 | 50 | 2.6298 | .0113* | .113 | 0.680
Trust Product | 1.0635 | 62 | 3.7170 | .0004* | .004* | 0.937
Table 5. Guideline 4’s t-test results for risk-averse vs. risk-tolerant
participants’ average ratings of the independent Violation AI product (top)
and Application AI product (bottom), for all 10 dependent variables. If dashes
(- - -) are present, no statistical test was performed. Notice that the 7
significant differences in the Violation products stubbornly remained in the
Application products, and a majority of the differences remained even under
Bonferroni correction.

###  4.2. What might designers do? 5 explicit risks from participants’ quotes

From a human-AI design perspective, there are explicit ties between these
findings and particular attitudes toward risk, which are arguably more
prescriptive than overall preferences or findings by demographic. Further,
when participants explained why they preferred one of the products (whether
the Application or Violation), they sometimes brought their attitudes toward
risk into their reasoning. Below, we highlight 5 types of risks that
participants raised across multiple experiments.

Risk 1– the risk of privacy/security in AI: The first risk that participants
revealed was about privacy/security risks. Some participants were concerned
about the privacy/security risks of the system outputs. For example, one risk-
averse participant from Guideline 3’s experiment (“Time services based on
context”) explicitly mentioned privacy/security to justify why they preferred
the Application product:

> G3-3414-risk-averse : “[Application product] does a better job [than
> Violation] at keeping notifications secure and not visible…”

Some participants at both ends of the risk spectrum also worried about the
privacy of the input data the system gathered:

> G4-3905-risk-averse : “I would prefer to use [Violation product] because
> [Application product] feels … intrusive. I would be nervous that it is
> pulling data from things like my other software and my browsing history.”

> G3-1959-risk-averse : “[Application product] is tracking me somehow because
> it holds the notifications, so I prefer the less invasive [Violation
> product].”

> G4-3947-risk-tolerant : “I don’t like the idea of [Application product]
> taking definitions from my workplace. It makes me worry I’m being listened
> to or that my network is being monitored, etc. ”

For human-AI designers, these statements are not about reducing users’
perceptions of security/privacy risks, but rather to proactively consider how
to either enable users to re-align their perceptions of risk with the actual
risks the AI product is imposing upon them or reassure them that such measures
are already in place.

For example, Lim & Dey worked on intelligibility types, explicitly considering
“inputs” and “outputs” in their taxonomy (Lim et al., 2019). Adding such
intelligibility explanations to an AI product may improve the users’ ability
to better align their perceptions with the actual risks the product imposes.

Risk 2–the risks of tolerating mistakes: Different participants’ attitudes
towards risk showed up in different ways. For example, some risk-tolerant
participants noticed the risk of the AI producing inaccurate information but
were willing to tolerate that risk:

> G3-1765-risk-tolerant : “It may mess up but most of the time [but it] works
> as it should.”

Remarks like this echo Saisubramanian et al.’s observation that some
participants tolerated negative side effects, as long as the AI achieved its
goal (Saisubramanian et al., 2021)999Gathering risk data on Saisubramanian et
al.’s participants might have uncovered whether the tolerance for negative
side effects correlated with their attitudes toward risk., but such attitudes
might also have unintended consequences. Designers might consider ways to
engage the user in critically evaluating the AI, perhaps by highlighting what
inaccuracies might cost the user to re-align their tolerances.

Risk 3–the risk of wasted time: Participants also expressed the risk of
pointlessly wasting time, such as this participant in Guideline 7’s (support
efficient invocation) and Guideline 11’s (make clear why the system did what
it did) experiments:

> G7-2138-risk-averse : “I like that [the feature] is easy to locate because
> there is nothing more frustrating than trying to find something and not
> being able to find it.”

> G11-2921-risk-averse : “[Violation product] would be a call to the help desk
> and a big time suck if there was inaccurate data.”

Sentiments such as these are reminiscent of Blackwell’s theory of Attention
Investment (Blackwell, 2002), which explains that in deciding whether to use a
new system or feature set, users often perform a cost/benefit/risk
calculation. Specifically, in this theory, users weigh their expectation of
the time cost of learning/using the new feature set against their expectation
of benefit (time saved) they will receive—tempered by their expectation of
risk (probability) that spending that time will not result in the benefits
after all. Thus, risk-averse users may place more weight on the risk aspect of
that calculation, so designers might work on ways to help users correctly
guess-timate how much time is actually involved in their interactions.

Risk 4–the risks of unfair AI: Participants were also concerned about the risk
of harm to others. This risk arose in the Guideline 6 experiment, which was
explicitly about social biases that can harm other people. Bird et al.’s term
the “harm of denigration” captures the harm that arises when one group of
people are being stereotyped in a derogatory way (Bird et al., 2020).
Participants across the risk spectrum called out this risk in Guideline 6’s
experiment:

> G6-1162-risk-averse : “[Violation product] obviously doesn’t think that
> women and people of color can hold highly-regarded, high-paying, and/or
> highly-educated positions. The search results I’ve landed upon on
> [Application product] are more inclusive and represent the values that are
> important to me…”

> G6-1410-risk-averse : “[Application product] is not trying to be harmful by
> indicating that women and people of color are not as smart or successful.
> [Violation product] sounds like something a white nationalist group would
> come up with to indicate their superiority over everyone, including women.”

However, this form of risk may be tied less to participants’ attitudes toward
risk than to their attitudes toward social justice, some of which seemed to
differ markedly from the above:

> G6-2796-risk-tolerant : “Most CEOs and doctors are white males. I feel like
> the first one is forcing its agenda of being inclusive”

Entire conferences, such as the ACM conference on Fairness, Accountability,
and Transparency (FAccT), are devoted to trying to solve such issues in AI.
Further, technical solutions exist to enable designers to critically evaluate
data sets for fairness disparities in AI systems, such as the open source
FairLearn package. We refer designers to these resources to learn more about
how they might address fairness-related concerns.

Risk 5–the risks with controlling AI: Lastly, participants raised the risks of
undesirable outcomes ensuing when they have little control over the system.
Risk-averse participants verbalized this risk across numerous experiments:

> G4-3897-risk-averse : “[Application product] would give me a limited amount
> of acronyms… [Violation product] seems like it would give me more control.”

> G3-953-risk-averse : “I prefer manual control rather than having an app
> decide when something should or should not happen automatically.”

> G10-1833-risk-averse : “I would rather have to pick the word I want if the
> program is not sure. I don’t want the program to insert a word I don’t want
> or that does not really fit the context.”

Some participants also brought up a related but slightly different consequence
of loss of control, namely potential over-reliance on the system:

> G4-3936-risk-averse : “ With [Application product product], if I relied on
> it too much and it make a mistake, of course I would be the one held
> responsible, not the document program. I would feel safer relying on my own
> expertise and using [Violation product].”

These observations by the participants were consistent with other researchers’
reports of users’ attitudes, sometimes with them overly relying on the
system’s outputs (Kaur et al., 2020) or becoming inattentive because of
lightened workload or boredom (Cunningham and Regan, 2015). Similarly, others
have reported on users’ attitudes reflecting an unwillingness to give up
control without the ability to regain it, hesitant about an unproven
technology (Howard and Dai, 2014; Haboucha et al., 2017).

These 5 form of risk highlight that designers of human-AI products can take
steps to better communicate the amount of risk involved and/or can mitigate
the amount of risk the system is actually asking them to take, provided that
the specific form of risk-aversion/tolerance the system triggers in some
participants is identified.

###  4.3. Beyond risk - What the other 4 cognitive styles show

Table 6. Summary of all 16 experiments (Figure 1), showing the guidelines (G#)
with significant differences within each cognitive style (rows) by the
guideline’s interaction phase (columns) from Figure 1. The colors in the cells
reflect which cognitive style participants gave significantly lower ratings
for the AI products for that G#: the  “Abi”-like or the  “Tim”-like ) style
(Table 1).

In addition to disaggregating participants’ ratings by their attitudes toward
risk, we also disaggregated by the other four GenderMag cognitive facets to
illustrate the importance of focusing on more than one of the styles. Table 6
shows not only the statistical disparities that existed for all 4 of the other
cognitive styles, analogous to those of risk, but it also highlight that such
trade-offs that the Application AI products sometimes incurred. To continue
our discussion, we first turn to the three cognitive styles that did not
exhibit a pattern in the data and then to the last one which did.

####  4.3.1. Motivations, Learning Style, & Information Processing Style

Although disaggregating participants’ data by their motivations101010The
motivations cognitive style is about the spectrum of reasons why people
interact with technology: from being  mainly to accomplish a task to
interacting with technology  for the enjoyment of technology .  , learning
style111111The learning styles spectrum referred to is specific to how people
go about figuring out new technologies: from a  “top-down”, process-oriented
style (i.e., learning the features in the context of a process that involves
those features) to a more  “bottom-up”, tinkering style of exploring new
features without the context of a process.  , and information processing style
did reveal significant differences in 13 unique experiments for the three
categories of insights in Section 4.1 (Appendices A.5, A.2, & A.4),
participants’ free-text responses did not help to clear up why these
differences existed, like in Section 4.2. For example, there was no references
to concepts commonly associated with their motivations (such as “accomplishing
a task/having fun”), learning style (like “following a process/clicking
around”), or information processing style (like needing “more/less”
information).

Figure 7. Guideline 5–disaggregated by participants’ computer self-efficacy.
(left): Insight Category 1 - For some of the dependent variables, the
Application product closed disparities between the ratings for the  lower
self-efficacy participants. (middle): Insight Category 2 - However, it also
introduced others, something that designers need to be aware of when designing
for human-AI interaction. (right): Insight Category 3 - Further, there were
also stubborn inclusiveness issues, where disparities existed in both the
Violation AI product and the Application AI product.

####  4.3.2. Computer Self-Efficacy

Prior research with computer self-efficacy has uncovered information about how
people solve problems, apply cognitive strategies, and how persistent they
will be in getting through a problem (Stumpf et al., 2020; Bandura, 1986).
Thus, the notable aspect of the Computer Self-Efficacy row (Table 6, bottom)
was that it was the only row with multiple instances of all three insight
categories from Section 4.1, but it was only the  Abi-like lower self-efficacy
participants who were disparately impacted across 10 distinct experiments.

Guideline 5’s (match relevant social norms) experiment highlighted this
particularly well (Figure 7), where disaggregating by participants’ computer
self-efficacy uncovered instances of all three insight categories. For
example, Figure 7 (left) illustrates the first insight category, where the
lower self-efficacy participants rated that they would be significantly more
suspicious of the Violation AI product’s intent or output (t(67) = -2.0091, p
= .0468, d = .484) than the  Tim-like higher self-efficacy participants, yet
the Application AI product closed this disparity (t(66) = -1.2674, p = .210).

However, there was a trade-off to closing this gap, where the Application AI
product introduced a different gap (i.e., Insight category 2) Figure 7, middle
shows that the lower self-efficacy participants thought that the Application
AI product would be more harmful than their higher self-efficacy peers (t(67)
= -2.3692, p = .021, d = .571), even though there was no evidence of a
difference for the Violation AI product (t(66) = -0.7737, p = .442).

Lastly, the third insight category also showed up (Figure 7, right), where the
lower self-efficacy participants felt significantly less in control than the
higher self-efficacy participants with both the Violation AI product (t(66) =
-2.893, p = .005, d = .702) and the Application AI product (t(66) = -2.097, p
= .040, d = .509).

What differentiated the self-efficacy disaggregation from the motivations,
learning style, and information processing style was that there were explicit
ties between participants’ quotes and their computer self-efficacy. For
example, some risk-averse participants thought that the Violation AI product’s
word choice was abrasive, which helped explain why they were less suspicious
of the system’s intent or output:

> G5-1196-lower : “I do think that the wording used by [Violation AI product
> product] is a little too abrasive and straight to the point. It could
> potentially upset someone…”

> G5-2493-lower : “It would be disconcerting to be constantly told I was
> making mistakes. Much prefer a suggestion like [Application AI product
> product].”

The prevalence of the Category 2 and 3 insights for self-efficacy in Table 6
suggested that neither the Violation nor the Application products seemed to
meet the needs of the lower self-efficacy participants for these experiments;
this is an important note for designers, since Stumpf et al. reported that a
failure to meet the needs of people like these participants could lead them to
not be as willing to use it in the future (Stumpf et al., 2020).

Table 7 shows through data triangulation the prevalence of inclusiveness
issues across the 5 cognitive style disaggregations, where each checkmark (✓)
represents a dependent variable exhibiting one of the three insight categories
from Table 6. On average, across all 5 styles and all 16 experiments, there
were about 19 inclusiveness issues found per cognitive style (SD = 9.503),
suggesting the need for further research to find (and fix) inclusiveness
issues with AI-powered systems through cognitive style disaggregation.

Table 7. Triangulation table: Existence of inclusivity disparities were
confirmed by 55 sources of evidence across 14 of the 16 experiments. Further,
each facet’s inclusivity disparities were confirmed by evidence from 4–11
separate experiments.

##  5\. Results: Cognitive styles meet gender demographics

The results by cognitive style are not only informative about AI users’
diverse approaches to interacting with AI, they also can shed light on
demographic differences that human-AI designers may observe. To demonstrate,
we consider RQ-Demographics, which asks how the five cognitive styles relate
to demographic diversity. Here, we consider gender and age.

For this analysis, our measurement unit transitions from the ratings
participants generated to the participants themselves. In the previous
section, we were comparing participants’ ratings of 16 different pairs of AI
products, one unique pair per experiment, which allowed comparing with only
participants in the same experiment. In contrast, RQ-Demographics analyzes the
relationship between the participants’ responses to the cognitive style
questionnaire (Appendix B) and their gender/age, without considering the AI
products.

This meant that the participants’ “peer group” became all 1,016 participants,
so we counted up how many of the participant’s 5 self-reported cognitive
styles were in the Abi-direction (as vs. the Tim-direction), separated by the
grand medians across all participants121212See Appendix D for a detailed
discussion of this method.. For example, suppose participant X had an Abi-
side-of-the-grand-median self-efficacy and Tim-side-of-the-grand-median for
the remaining four facets. Then participant X’s Abi-direction count would be
1.

5.1 Women, men, and age:

Over 98% of our participants identified as either a woman (only) or a man
(only), so we first compare these two genders. For such participants, Figure 9
show that the number of “Abi” and “Tim” cognitive styles clustered by gender.
That is, the women skewed toward the Abi direction, and the men skewed toward
the Tim direction (echoing Vorvoreanu et al.’s (Vorvoreanu et al., 2019)
earlier findings in a smaller sample, shown in Figure 9). To compute the
significance of these proportion differences, we used Fisher’s exact test to
compare the number of women vs. men whose Abi-direction counts were 3 or more,
where 3 was the threshold with the least chance of showing significance (the
maximum sum of p-values (Ramsey and Schafer, 2012)). Under this test, the
proportions of gender differences were highly significant (Fisher’s exact
test, k = 222, $p<.0001$).

Figure 8. The count of participants (Y-axis) by the number of Abi-direction or
Tim-direction cognitive styles each participant reported (X-axis). The  women
(orange) skewed toward Abi styles, and the  men (blue) toward Tim styles.

FIXME  Figure 9. Counts of Vorvoreanu et al.’s 20 participants who identified
as either women or men by their facet values (Vorvoreanu et al., 2019). Note
the similar skews to Figure 9.

The women’s and men’s cognitive styles also differed across age ranges. Figure
10 shows the proportions131313Proportions used for visualization purposes
only. All statistical tests in this section still use counts. of participants
in each age group by the number of Abi-direction or Tim-direction styles,
similar to Figure 9. In every age group, women visually skewed toward the Abi
direction, and men skewed toward the Tim direction. In three of these age
groups, the gender proportion differences were significant: 25–34: p = .001;
35–44: p = .0105; 45–54: p = .01 (Fisher’s exact test)141414Table 11 in
Appendix E.. Literature from the social sciences and the HCI community that
align with the 5 cognitive styles used in this paper may help to suggest why
such a skew existed.

Figure 10. The cognitive styles alignment proportions for the number of Abi
cognitive styles, factoring in age groups. As in the aggregate (Figure 9), all
age groups demonstrated a women’s left-skew and men’s right-skew. Colors are
the same as in Figure 9.

FIXME

For risk, research related to attitudes toward risk found that 18–24 year old
people were more likely than older age groups to engage in potentially
dangerous activities (Frühauf et al., 2017; Zicat et al., 2018), reflecting a
more Tim-like risk attitude amongst younger people than older people.
Similarly, work tying risk to technology usage found that older people’s
perceptions of autonomous vehicles were more negative than younger people’s,
due to perceived risks (Hulse et al., 2018). Further, McIntosh et al. reported
a usability bias against elderly people by leveraging people’s risk attitudes
in their e-commerce analysis (McIntosh et al., 2021).

For computer self-efficacy, Lopez et al. (López-Vargas et al., 2017) found a
strong negative correlation between people’s ages and their self-efficacy with
information communication technology151515Their correlation coefficient was
significant even under $\alpha=.01$.. Similarly, van der Groot & Pilgrim,
reported that their participants’ age influenced their Motivations to interact
with technology (van der Goot and Pilgrim, 2019).

Related to information processing style, Gell et al. reported about how people
interact with information-rich technology such as email might change based on
barriers such as vision and memory impairments as they grow older, which might
help explain the more uniform distribution in the 55 – 64 year old age group
(Gell et al., 2015), and these factors were also reported on by McIntosh et
al. (McIntosh et al., 2021).

Literature investigating people’s learning styles and age have measured
learning style differently, using surveys such as the Learning Style Inventory
(ADMIRAAL and SWART, 2011) or the Myer-Briggs Type Indicator (Li et al.,
2011). However, both of these examples found insufficient evidence to suggest
an interaction between age and learning style.

The tie between this literature and technology usage specifically needs
further investigation, but this much is clear: only through collecting
people’s cognitive styles throughout all stages of life can we hope to
understand how to better cater to diverse people across all phases of their
lives.

5.2 LGBTQIA* participants:

There is a paucity of data on the cognitive styles of members of the
LGBTQIA*161616LGBTQIA* used based on Scheuerman et al.’s living document
(Scheuerman et al., 2020). community. In this paper’s dataset, 13 of the 1,016
participants identified with LGBTQIA* genders. To help start filling this data
gap, we enumerate their cognitive style data in Table 8 . We believe this to
be the first report of GenderMag cognitive styles for this community.

Table 8. Each row shows one LGBTQIA* participant’s cognitive style
information. W = woman, M = man, T = transgender, NB = Non-Binary,NC = Gender
Non-Conforming, I = intersex. These 13 participants tended toward the “Tim”
side of four of their cognitive styles, but for Information Processing Style,
these participants tended toward a comprehensive style.

##  6\. Threats to Validity

Every empirical study has limitations and threats to validity (Wohlin et al.,
2000; Ko et al., 2015). One challenge for our paper is the limitation of being
a secondary analysis of existing data, which meant that we had to operate and
analyze under Li et al.’s experimental methodology.

One possible limitation of our analysis is that the 10 UX metrics we analyze
from Li et al.’s experiments might not have captured all information about
people’s reactions, although we cannot ask participants every possible
question, having to balance research goals with participants’ fatigue. For
example, some participants in Section 4.1 mentioned how they would feel secure
while interacting with certain products, but there was no direct metric for
how secure someone would feel.

As we mentioned in Section 3, participants had the option to say “I don’t
know” for any of the questions. Although methods like imputation exist to try
and handle such data, we deferred to dropping the degrees of freedom in our
statistical tests because there are many potential methods to leverage (e.g.,
hot-deck, regression, or cold-deck imputation), but inferences are then
limited to the imputed data, rather than the original data.

Another challenge for our analysis was balancing the number of statistical
tests we ran and subsequently reported in the paper. In Section 3.2.3, we
attempted to address concerns of p-hacking through a conservative testing
paradigm. Concepts like “small” and “large” were intuitive assessments of
scale. However, since we discarded dependent variables because the distance
between means looked too small, there may be missing inferences in the data.
In order to try and address rigor, we decided to exclude anything marginally
significant ($p\in(0.5,0.1]$) from our analyses, and we also applied
Bonferroni correction where necessary. We also provide all box plots in
Appendices A – E in the interest of transparency.

Threats like these can only be addressed through additional studies across a
spectrum of empirical methods, to isolate different independent variables of
study and establish generality of findings across different AI applications,
different measurements, and different populations in order to create more
harmonious human-AI interaction.

##  7\. Conclusion

In this paper, we presented one of the first applications of the GenderMag
cognitive styles to AI-powered applications through a secondary analysis of 16
experiments, based on Li et al.’s investigation of the guidelines for human-AI
interaction (Amershi et al., 2019). Our results indicated that:

  * •

There were 3 ways that the AI Application product changes influenced
inclusiveness gaps:

    * –

They sometimes closed them.

    * –

Other times, they opened them.

    * –

The inclusiveness gaps persisted, despite AI product changes.

  * •

There were 5 forms of actionable risks for designers to consider for these
guidelines, derived from participants’ responses about why they preferred one
of the products.

  * •

Not only did these participants’ cognitive styles cluster by gender, similar
to Vorvoreanu et al. (Vorvoreanu et al., 2019), but there were similar
clustering behaviors across age groups.

These results suggest that in order to further understand human-AI data,
cognitive style disaggregation is a potentially powerful tool to help uncover
and understand inclusively designing for diverse people across the life-cycle
of human-AI interaction.

## Acknowledgments

We thank Catherine Hu, Christopher Perdriau, Caleb Matthews, and Sai Raja for
their help with this paper. This work was supported in part by DARPA
#N66001-17-2-4030 and by NSF #1901031 and #2042324. Any opinions, findings,
conclusions, or recommendations expressed are the authors’ and do not
necessarily reflect the views of the sponsors.

## References

  * (1)
  * van ([n.d.]) [n.d.].  Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors.  ([n. d.]).
  * ADMIRAAL and SWART (2011) Wilfried ADMIRAAL and José SWART. 2011.  Learning styles, motivation and performance: differences between two school levels in Dutch secondary education.  _education_ 15 (2011), 14.
  * Ajenaghughrure et al. (2020) Ighoyota Ben Ajenaghughrure, Sonia Claudia da Costa Sousa, and David Lamas. 2020.  Risk and Trust in artificial intelligence technologies: A case study of Autonomous Vehicles. In _2020 13th International Conference on Human System Interaction (HSI)_. IEEE, 118–123.
  * Alqaraawi et al. (2020) Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020\.  Evaluating saliency map explanations for convolutional neural networks: a user study. In _Proceedings of the 25th International Conference on Intelligent User Interfaces_. 275–285.
  * Amershi et al. (2019) Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019\.  Guidelines for human-AI interaction. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. ACM, 3.
  * Anderson et al. (2020) Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Matthew Olson, Alan Fern, and Margaret Burnett. 2020\.  Mental Models of Mere Mortals with Explanations of Reinforcement Learning.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ 10, 2 (2020), 1–37.
  * Bandura (1986) Albert Bandura. 1986\.  The explanatory and predictive scope of self-efficacy theory.  _Journal of social and clinical psychology_ 4, 3 (1986), 359–373.
  * Benedek and Miner (2002) Joey Benedek and Trish Miner. 2002.  Measuring Desirability: New methods for evaluating desirability in a usability lab setting.  _Proceedings of Usability Professionals Association_ 2003, 8-12 (2002), 57.
  * Bird et al. (2020) Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020.  Fairlearn: A toolkit for assessing and improving fairness in AI.  _Microsoft, Tech. Rep. MSR-TR-2020-32_ (2020).
  * Blackwell (2002) Alan F Blackwell. 2002\.  First steps in programming: A rationale for attention investment models. In _Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments_. IEEE, 2–10.
  * Buolamwini and Gebru (2018) Joy Buolamwini and Timnit Gebru. 2018.  Gender shades: Intersectional accuracy disparities in commercial gender classification. In _Conference on fairness, accountability and transparency_. PMLR, 77–91.
  * Burnett et al. (2016) Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016\.  GenderMag: A method for evaluating software’s gender inclusiveness.  _Interacting with Computers_ 28, 6 (2016), 760–787.
  * Crawford (2017) Kate Crawford. 2017\.  The trouble with bias. In _Conference on Neural Information Processing Systems (NIPS)_.
  * Cunningham and Regan (2015) Mitchell Cunningham and Michael A Regan. 2015.  Autonomous vehicles: human factors issues and future research. In _Proceedings of the 2015 Australasian Road safety conference_ , Vol. 14.
  * Frühauf et al. (2017) Anika Frühauf, Will AS Hardy, Daniel Pfoestl, Franz-Georg Hoellen, and Martin Kopp. 2017\.  A qualitative approach on motives and aspects of risks in freeriding.  _Frontiers in psychology_ 8 (2017), 1998.
  * Gandhi et al. (2019) Meera Gandhi, Vishal Kumar Singh, and Vivek Kumar. 2019\.  IntelliDoctor-AI based Medical Assistant. In _2019 Fifth International Conference on Science Technology Engineering and Mathematics (ICONSTEM)_ , Vol. 1. IEEE, 162–168.
  * Gell et al. (2015) Nancy M Gell, Dori E Rosenberg, George Demiris, Andrea Z LaCroix, and Kushang V Patel. 2015\.  Patterns of technology use among older adults with and without disabilities.  _The Gerontologist_ 55, 3 (2015), 412–421.
  * Gillath et al. (2021) Omri Gillath, Ting Ai, Michael S Branicky, Shawn Keshmiri, Robert B Davison, and Ryan Spaulding. 2021\.  Attachment and trust in artificial intelligence.  _Computers in Human Behavior_ 115 (2021), 106607.
  * Glaser (1963) B. G. Glaser. 1963\.  Retreading research materials: The use of secondary analysis by the independent researcher.  _The American Behavioural Scientist_ 6, 10 (1963), 11–14.
  * Gong et al. (2019) Z. Gong, P. Zhong, and W. Hu. 2019.  Diversity in Machine Learning.  _IEEE Access_ 7 (2019), 64323–64350.  https://doi.org/10.1109/ACCESS.2019.2917620
  * Goyal et al. (2019) Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019\.  Counterfactual visual explanations. In _International Conference on Machine Learning_. PMLR, 2376–2384.
  * Guidotti (2021) Riccardo Guidotti. 2021\.  Evaluating local explanation methods on ground truth.  _Artificial Intelligence_ 291 (2021), 103428.
  * Guo et al. (2019) Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, and Meredith Ringel Morris. 2019\.  Toward fairness in AI for people with disabilities: A research roadmap.  _arXiv preprint arXiv:1907.02227_ (2019).
  * Haboucha et al. (2017) Chana J Haboucha, Robert Ishaq, and Yoram Shiftan. 2017\.  User preferences regarding autonomous vehicles.  _Transportation Research Part C: Emerging Technologies_ 78 (2017), 37–49.
  * Hohman et al. (2019) Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, and Steven M Drucker. 2019.  Gamut: A design probe to understand how data scientists understand machine learning models. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. ACM, 579.
  * Howard and Dai (2014) Daniel Howard and Danielle Dai. 2014.  Public perceptions of self-driving cars: The case of Berkeley, California. In _Transportation research board 93rd annual meeting_ , Vol. 14. 1–16.
  * Hulse et al. (2018) Lynn M Hulse, Hui Xie, and Edwin R Galea. 2018.  Perceptions of autonomous vehicles: Relationships with road users, risk, gender and age.  _Safety science_ 102 (2018), 1–13.
  * Jacobs et al. (2021) Maia Jacobs, Jeffrey He, Melanie F. Pradier, Barbara Lam, Andrew C. Ahn, Thomas H. McCoy, Roy H. Perlis, Finale Doshi-Velez, and Krzysztof Z. Gajos. 2021.  Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens.  (2021).
  * Jian et al. (2000) Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000\.  Foundations for an empirically determined scale of trust in automated systems.  _International journal of cognitive ergonomics_ 4, 1 (2000), 53–71.
  * Johnston (2014) Melissa P. Johnston. 2014\.  Secondary Data Analysis: A Method of which the Time Has Come.  _Qualitative and Quantitative Methods in Libraries (QQML)_ 3 (2014), 619––626.
  * Juravle et al. (2020) Georgiana Juravle, Andriana Boudouraki, Miglena Terziyska, and Constantin Rezlescu. 2020\.  Trust in artificial intelligence for medical diagnoses.  _Real-World Applications in Cognitive Neuroscience_ 253 (2020), 263\.
  * Kaur et al. (2020) Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020.  Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Kenny et al. (2021) Eoin M Kenny, Courtney Ford, Molly Quinn, and Mark T Keane. 2021\.  Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies.  _Artificial Intelligence_ 294 (2021), 103459.
  * Kiritchenko and Mohammad (2018) Svetlana Kiritchenko and Saif M Mohammad. 2018.  Examining gender and race bias in two hundred sentiment analysis systems.  _arXiv preprint arXiv:1805.04508_ (2018).
  * Ko et al. (2015) A J Ko, T D Latoza, and M M Burnett. 2015.  A practical guide to controlled experiments of software engineering tools with human participants.  _Empirical Software Engineering_ 20, 1 (2015), 110–141.
  * Kulesza et al. (2015) Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015.  Principles of explanatory debugging to personalize interactive machine learning. In _ACM Intl. Conf. on Intelligent User Interfaces_ _(IUI ’15)_. ACM, 126–137.
  * Larson et al. (2016) Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016\.  How We Analyzed the COMPAS Recidivism Algorithm.  https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
  * Li et al. (2011) Yuh-Shiow Li, Hsiu-Mei Chen, Bao-Huan Yang, and Chin-Fang Liu. 2011.  An exploratory study of the relationship between age and learning styles among students in different nursing programs in Taiwan.  _Nurse education today_ 31, 1 (2011), 18–23.
  * Lim (2012) Brian Y Lim. 2012\.  _Improving understanding and trust with intelligibility in context-aware applications_.  Ph.D. Dissertation. figshare.
  * Lim and Dey (2009) Brian Y Lim and Anind K Dey. 2009.  Assessing demand for intelligibility in context-aware applications. In _Proceedings of the 11th international conference on Ubiquitous computing_. ACM, 195–204.
  * Lim et al. (2009) Brian Y Lim, Anind K Dey, and Daniel Avrahami. 2009.  Why and why not explanations improve the intelligibility of context-aware intelligent systems. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_. 2119–2128.
  * Lim et al. (2019) Brian Y Lim, Qian Yang, Ashraf M Abdul, and Danding Wang. 2019\.  Why these Explanations? Selecting Intelligibility Types for Explanation Goals.. In _IUI Workshops_.
  * Lima et al. (2021) Gabriel Lima, Nina Grgić-Hlača, and Meeyoung Cha. 2021\.  Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making.  (2021).
  * Lin et al. (2019) Steven Y Lin, Megan R Mahoney, and Christine A Sinsky. 2019\.  Ten ways artificial intelligence will transform primary care.  _Journal of general internal medicine_ 34, 8 (2019), 1626–1630.
  * López-Vargas et al. (2017) Omar López-Vargas, Leydy Duarte-Suárez, and Jaime Ibáñez-Ibáñez. 2017.  Teacher’s computer self-efficacy and its relationship with cognitive style and TPACK.  _Improving Schools_ 20, 3 (2017), 264–277.
  * Lu et al. (2020) Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2020.  Gender bias in neural natural language processing.  In _Logic, Language, and Security_. Springer, 189–202.
  * Lundberg et al. (2020) Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020\.  From local explanations to global understanding with explainable AI for trees.  _Nature machine intelligence_ 2, 1 (2020), 56–67.
  * Madumal et al. (2019) Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2019.  Explainable Reinforcement Learning Through a Causal Lens.  _CoRR_ abs/1905.10958 (2019).  arXiv:1905.10958 http://arxiv.org/abs/1905.10958
  * McIntosh et al. (2021) Jennifer McIntosh, Xiaojiao Du, Zexian Wu, Giahuy Truong, Quang Ly, Richard How, Sriram Viswanathan, and Tanjila Kanij. 2021.  Evaluating Age Bias In E-commerce. In _2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)_. IEEE, 31–40.
  * Mehrabi et al. (2019) Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019\.  A survey on bias and fairness in machine learning.  _arXiv preprint arXiv:1908.09635_ (2019).
  * Mikołajczyk et al. (2020) Agnieszka Mikołajczyk, Michał Grochowski, and Arkadiusz Kwasigroch. 2020.  Global explanations for discovering bias in data.  _arXiv preprint arXiv:2005.02269_ (2020).
  * Mothilal et al. (2020) Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. 2020\.  Explaining machine learning classifiers through diverse counterfactual explanations. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. 607–617.
  * Mueller et al. (2019) Shane T Mueller, Robert R Hoffman, William Clancey, Abigail Emrey, and Gary Klein. 2019.  Explanation in human-AI systems: A literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI.  _arXiv preprint arXiv:1902.01876_ (2019).
  * Nasr and Tschantz (2020) Milad Nasr and Michael Carl Tschantz. 2020.  Bidding strategies with gender nondiscrimination constraints for online ad auctions. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. 337–347.
  * Neal et al. (2018) Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. 2018.  Open Set Learning with Counterfactual Images. In _Proceedings of the European Conference on Computer Vision (ECCV)_.
  * Nguyen (2018) Dong Nguyen. 2018\.  Comparing automatic and human evaluation of local explanations for text classification. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_. 1069–1078.
  * Norman and Gentner (1983) Donald Norman and Dedra Gentner. 1983.  Mental models.  _Lawrence Erlbaum Associates, Hillsdale, NJ_ (1983).
  * Osoba and Welser IV (2017) Osonde A Osoba and William Welser IV. 2017.  _An intelligence in our image: The risks of bias and errors in artificial intelligence_.  Rand Corporation.
  * Padala et al. (2020) Susmita Hema Padala, Christopher John Mendez, Luiz Felipe Dias, Igor Steinmacher, Zoe Steine Hanson, Claudia Hilderbrand, Amber Horvath, Charles Hill, Logan Dale Simpson, Margaret Burnett, et al. 2020\.  How gender-biased tools shape newcomer experiences in OSS projects.  _IEEE Transactions on Software Engineering_ (2020).
  * Pal et al. (2019) D. Pal, C. Arpnikanondt, S. Funilkul, and V. Varadarajan. 2019.  User Experience with Smart Voice Assistants: The Accent Perspective. In _2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT)_. 1–6.  https://doi.org/10.1109/ICCCNT45670.2019.8944754
  * Plumb et al. (2018) Gregory Plumb, Denali Molitor, and Ameet Talwalkar. 2018\.  Model agnostic supervised local explanations.  _arXiv preprint arXiv:1807.02910_ (2018).
  * Popordanoska et al. (2020) Teodora Popordanoska, Mohit Kumar, and Stefano Teso. 2020\.  Machine guides, human supervises: Interactive learning with global explanations.  _arXiv preprint arXiv:2009.09723_ (2020).
  * Ramsey and Schafer (2012) Fred Ramsey and Daniel Schafer. 2012.  _The statistical sleuth: a course in methods of data analysis_.  Cengage Learning.
  * Robertson et al. (2021) Samantha Robertson, Tonya Nguyen, and Niloufar Salehi. 2021\.  Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms.  (2021).
  * Rose (2010) Adam Rose. 2010\.  Are Face-Detection Cameras Racist?  http://content.time.com/time/business/article/0,8599,1954643,00.html
  * Saisubramanian et al. (2021) Sandhya Saisubramanian, Shannon Roberts, and Shlomo Zilberstein. 2021.  Understanding User Attitudes Towards Negative Side Effects of AI Systems.  _Proceedings of the 2021 CHI conference on Human Factors in Computing systems_ (2021).
  * Scheuerman et al. (2020) Morgan Klaus Scheuerman, Katta Spiel, Oliver L Haimson, Foad Hamidi, and Stacy M Branham. 2020\.  HCI guidelines for gender equity and inclusivity.  _UMBC Faculty Collection_ (2020).
  * Schneider et al. (2021) Tobias Schneider, Sabiha Ghellal, Steve Love, and Ansgar R.S. Gerlicher. 2021.  Increasing the User Experience in Autonomous Driving through Different Feedback Modalities. In _26th International Conference on Intelligent User Interfaces_ _(IUI ’21)_. Association for Computing Machinery, New York, NY, USA, 7–10.  https://doi.org/10.1145/3397481.3450687
  * Setzu et al. (2021) Mattia Setzu, Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and Fosca Giannotti. 2021\.  GLocalX-From Local to Global Explanations of Black Box AI Models.  _Artificial Intelligence_ 294 (2021), 103457.
  * Sheridan (1989) TB Sheridan. 1989\.  Trustworthiness of command and control systems.  In _Analysis, Design and Evaluation of Man–Machine Systems 1988_. Elsevier, 427–431.
  * Steine-Hanson et al. (2019) Zoe Steine-Hanson, Claudia Hilderbrand, Lara Letaw, Jillian Emard, Christopher Perdriau, Christopher Mendez, Margaret Burnett, and Anita Sarma. 2019\.  Fixing Inclusivity Bugs for Information Processing Styles and Learning Styles.  _arXiv preprint arXiv:1905.02813_ (2019).
  * Stumpf et al. (2020) Simone Stumpf, Anicia Peters, Shaowen Bardzell, Margaret Burnett, Daniela Busse, Jessica Cauchard, and Elizabeth Churchill. 2020\.  Gender-Inclusive HCI Research and Design: A Conceptual Review.  _Foundations and Trends in Human-Computer Interaction_ 13, 1 (2020), 1–69.
  * Sun et al. (2019) Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019\.  Mitigating gender bias in natural language processing: Literature review.  _arXiv preprint arXiv:1906.08976_ (2019).
  * Sunarti et al. (2021) Sri Sunarti, Ferry Fadzlul Rahman, Muhammad Naufal, Muhammad Risky, Kresna Febriyanto, and Rusni Masnina. 2021\.  Artificial intelligence in healthcare: opportunities and risk for future.  _Gaceta Sanitaria_ 35 (2021), S67–S70.
  * Sundaresan (2019) Neel Sundaresan. 2019\.  From Code to Data: AI at Scale for Developer Productivity. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 3175–3175.
  * Tatman (2017) Rachael Tatman. 2017\.  Gender and dialect bias in YouTube’s automatic captions. In _Proceedings of the First ACL Workshop on Ethics in Natural Language Processing_. 53–59.
  * van der Goot and Pilgrim (2019) Margot J van der Goot and Tyler Pilgrim. 2019.  Exploring age differences in motivations for and acceptance of chatbot communication in a customer service context. In _International Workshop on Chatbot Research and Design_. Springer, 173–186.
  * van der Waa et al. (2018) Jasper van der Waa, Jurriaan van Diggelen, Karel van den Bosch, and Mark Neerincx. 2018\.  Contrastive explanations for reinforcement learning in terms of expected consequences.  (2018).
  * Vorvoreanu et al. (2019) Mihaela Vorvoreanu, Lingyi Zhang, Yun-Han Huang, Claudia Hilderbrand, Zoe Steine-Hanson, and Margaret Burnett. 2019.  From gender biases to gender-inclusive design: An empirical investigation. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Wieringa (2020) Maranke Wieringa. 2020\.  What to account for when accounting for algorithms: A systematic literature review on algorithmic accountability. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. 1–18.
  * Wohlin et al. (2000) Claes Wohlin, Per Runeson, Martin Höst, Magnus Ohlsson, Björn Regnell, and Anders Wesslén. 2000.  _Experimentation in Software Engineering: An Introduction_.  Kluwer Academic Publishers, Norwell, MA, USA.
  * Wu et al. (2020) Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R Lyu, and Yu-Wing Tai. 2020.  Towards Global Explanations of Convolutional Neural Networks With Concept Attribution. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 8652–8661.
  * Yang et al. (2020) Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. 2020.  Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. 547–558.
  * Zhang et al. (2018) Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018\.  Mitigating unwanted biases with adversarial learning. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_. 335–340.
  * Zhao et al. (2019) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019\.  Gender bias in contextualized word embeddings.  _arXiv preprint arXiv:1904.03310_ (2019).
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018.  Gender bias in coreference resolution: Evaluation and debiasing methods.  _arXiv preprint arXiv:1804.06876_ (2018).
  * Zicat et al. (2018) Emma Zicat, Joanne M. Bennett, Eugene Chekaluk, and Jennifer Batchelor. 2018.  Cognitive function and young drivers: The relationship between driving, attitudes, personality and cognition.  _Transportation Research Part F: Traffic Psychology and Behaviour_ 55 (2018), 341–352.  https://doi.org/10.1016/j.trf.2018.03.013

##  Appendix A Cognitive Styles Boxplots

###  A.1. Attitudes Toward Risk

Link to Our Data Folder. Focus on the by-facet Sheets (eg, SE-Data, etc.)

Figure 11. Guideline 1 — Make clear what the system can do — disaggregated by
participants’ attitudes toward risk. Figure 12. Guideline 3 — Time services
based on context — disaggregated by participants’ attitudes toward risk.

Figure 13. Guideline 4 — Show contextually relevant information —
disaggregated by participants’ attitudes toward risk. Violation Product
---
Dependent |  |  |  |  | Bonferroni |
variable | $|\bar{X}_{1}-\bar{X}_{2}|$ | df | t-statistic | p-value | correction | Cohen’s d
Feel In Control | 0.4444 | - | \- - - | \- - - | \- - - | \- - -
Feel Secure | 0.6667 | 62 | 2.3006 | .0249* | .174 | 0.580
Feel Adequate | 0.8704 | 61 | 2.9179 | .0049* | .034* | 0.743
Feel Certain | 1.2798 | 62 | 3.6258 | .0005* | .0035* | 0.910
Feel Productive | 0.5397 | - | \- - - | \- - - | \- - - | \- - -
Perceived Useful | 0.5119 | - | \- - - | \- - - | \- - - | \- - -
Not Suspicious | 1.6468 | 34 | 4.8630 |  $<.0001$* |  $<.001$* | 1.337
Not Harmful | 0.7893 | 39 | 2.8694 | .006* | .042* | 0.763
Product Reliable | 0.6312 | 49 | 2.0046 | .051 | .357 | 0.518
Trust Product | 0.7519 | 63 | 2.6615 | .010* | .070 | 0.664
Application Product
Dependent |  |  |  |  | Bonferroni |
variable | $|\bar{X}_{1}-\bar{X}_{2}|$ | df | t-statistic | p-value | correction | Cohen’s d
Feel In Control | 0.9643 | 45 | 3.0663 | .0037* | .037* | 0.810
Feel Secure | 1.0093 | 61 | 3.1984 | .0022* | .022* | 0.814
Feel Adequate | 0.8492 | 62 | 2.4955 | .0153* | .153 | 0.629
Feel Certain | 1.6916 | 63 | 4.7513 |  $<.0001$* | .0001* | 1.186
Feel Productive | 0.4598 | 63 | 1.8462 | .070 | .700 | 0.461
Perceived Useful | 0.7854 | 63 | 2.9810 | .0041* | .041* | 0.744
Not Suspicious | 1.8008 | 42 | 5.1692 |  $<.0001$* | .0001* | 1.364
Not Harmful | 0.9607 | 44 | 3.1328 | .0031* | .031* | 0.821
Product Reliable | 0.8170 | 50 | 2.6298 | .0113* | .113 | 0.680
Trust Product | 1.0635 | 62 | 3.7170 | .0004* | .004* | 0.937
Table 9. Guideline 4’s t-test results for risk-averse vs. risk-tolerant
participants’ average ratings of the independent Violation AI product (top)
and Application AI product (bottom), for all 10 dependent variables. If dashes
(- - -) are present, no statistical test was performed. If the number of tests
exceeded 5 for each product, Bonferroni correction applied by multiplying
p-values by the number of tests performed.

Figure 14. Guideline 5 — Match relevant social norms — disaggregated by
participants’ attitudes toward risk. Figure 15. Guideline 6 — Mitigate social
biases — disaggregated by participants’ attitudes toward risk. Figure 16.
Guideline 7 — Support efficient invocation — disaggregated by participants’
attitudes toward risk. Figure 17. Guideline 8 — Support efficient dismissal —
disaggregated by participants’ attitudes toward risk. Figure 18. Guideline 9 —
Support efficient correction — disaggregated by participants’ attitudes toward
risk. Figure 19. Guideline 10 — Scope services when in doubt — disaggregated
by participants’ attitudes toward risk. Figure 20. Guideline 11 — Make clear
why the system did what it did — disaggregated by participants’ attitudes
toward risk. Figure 21. Guideline 12 — Remember recent interactions —
disaggregated by participants’ attitudes toward risk. Figure 22. Guideline 13
— Learn from user behavior — disaggregated by participants’ attitudes toward
risk. Figure 23. Guideline 14 — Update and adapt cautiously — disaggregated by
participants’ attitudes toward risk. Figure 24. Guideline 15 — Encourage
granular feedback — disaggregated by participants’ attitudes toward risk.
Figure 25. Guideline 17 — Provide global controls — disaggregated by
participants’ attitudes toward risk. Figure 26. Guideline 18 — Notify users
about changes — disaggregated by participants’ attitudes toward risk.

###  A.2. Learning Style

Figure 27. Guideline 1 — Make clear what the system can do — disaggregated by
participants’ learning style: by Process vs. by Tinkering. Figure 28.
Guideline 3 — Time services based on context — disaggregated by participants’
learning style: by Process vs. by Tinkering. Figure 29. Guideline 4 — Show
contextually relevant information — disaggregated by participants’ learning
style: by Process vs. by Tinkering. Figure 30. Guideline 5 — Match relevant
social norms — disaggregated by participants’ learning style: by Process vs.
by Tinkering. Figure 31. Guideline 6 — Mitigate social biases — disaggregated
by participants’ learning style: by Process vs. by Tinkering. Figure 32.
Guideline 7 — Support efficient invocation — disaggregated by participants’
learning style: by Process vs. by Tinkering. Figure 33. Guideline 8 — Support
efficient dismissal — disaggregated by participants’ learning style: by
Process vs. by Tinkering. Figure 34. Guideline 9 — Support efficient
correction — disaggregated by participants’ learning style: by Process vs. by
Tinkering. Figure 35. Guideline 10 — Scope services when in doubt —
disaggregated by participants’ learning style: by Process vs. by Tinkering.
Figure 36. Guideline 11 — Make clear why the system did what it did —
disaggregated by participants’ learning style: by Process vs. by Tinkering.
Figure 37. Guideline 12 — Remember recent interactions — disaggregated by
participants’ learning style: by Process vs. by Tinkering. Figure 38.
Guideline 13 — Learn from user behavior — disaggregated by participants’
learning style: by Process vs. by Tinkering. Figure 39. Guideline 13 — Update
and adapt cautiously — disaggregated by participants’ learning style: by
Process vs. by Tinkering. Figure 40. Guideline 15 — Encourage granular
feedback — disaggregated by participants’ learning style: by Process vs. by
Tinkering. Figure 41. Guideline 17 — Provide global controls — disaggregated
by participants’ learning style: by Process vs. by Tinkering. Figure 42.
Guideline 18 — Notify users about changes — disaggregated by participants’
learning style: by Process vs. by Tinkering.

###  A.3. Self-Efficacy

Figure 43. Guideline 1 — Make clear what the system can do — disaggregated by
participants’ computer self-efficacy. Figure 44. Guideline 3 — Time services
based on context — disaggregated by participants’ computer self-efficacy.
Figure 45. Guideline 4 — Show contextually relevant information —
disaggregated by participants’ computer self-efficacy. Figure 46. Guideline 5
— Match relevant social norms — disaggregated by participants’ computer self-
efficacy. Figure 47. Guideline 6 — Mitigate social biases — disaggregated by
participants’ computer self-efficacy. Figure 48. Guideline 7 — Support
efficient invocation — disaggregated by participants’ computer self-efficacy.
Figure 49. Guideline 8 — Support efficient dismissal — disaggregated by
participants’ computer self-efficacy. Figure 50. Guideline 9 — Support
efficient correction — disaggregated by participants’ computer self-efficacy.
Figure 51. Guideline 10 — Scope services when in doubt — disaggregated by
participants’ computer self-efficacy. Figure 52. Guideline 11 — Make clear why
the system did what it did — disaggregated by participants’ computer self-
efficacy. Figure 53. Guideline 12 — Remember recent interactions —
disaggregated by participants’ computer self-efficacy. Figure 54. Guideline 13
— Learn from user behavior — disaggregated by participants’ computer self-
efficacy. Figure 55. Guideline 14 — Update and adapt cautiously —
disaggregated by participants’ computer self-efficacy. Figure 56. Guideline 15
— Encourage granular feedback — disaggregated by participants’ computer self-
efficacy. Figure 57. Guideline 17 — Provide global controls — disaggregated by
participants’ computer self-efficacy. Figure 58. Guideline 18 — Notify users
about changes — disaggregated by participants’ computer self-efficacy.

###  A.4. Information Processing Style

Figure 59. Guideline 1 — Make clear what the system can do — disaggregated by
participants’ information processing style. Figure 60. Guideline 3 — Time
services based on context — disaggregated by participants’ information
processing style. Figure 61. Guideline 4 — Show contextually relevant
information — disaggregated by participants’ information processing style.
Figure 62. Guideline 5 — Match relevant social norms — disaggregated by
participants’ information processing style. Figure 63. Guideline 6 — Mitigate
social biases — disaggregated by participants’ information processing style.
Figure 64. Guideline 7 — Support efficient invocation — disaggregated by
participants’ information processing style. Figure 65. Guideline 8 — Support
efficient dismissal — disaggregated by participants’ information processing
style. Figure 66. Guideline 9 — Support efficient correction — disaggregated
by participants’ information processing style. Figure 67. Guideline 10 — Scope
services when in doubt — disaggregated by participants’ information processing
style. Figure 68. Guideline 11 — Make clear why the system did what it did —
disaggregated by participants’ information processing style. Figure 69.
Guideline 12 — Remember recent interactions — disaggregated by participants’
information processing style. Figure 70. Guideline 13 — Learn from user
behavior — disaggregated by participants’ information processing style. Figure
71. Guideline 14 — Update and adapt cautiously — disaggregated by
participants’ information processing style. Figure 72. Guideline 15 —
Encourage granular feedback — disaggregated by participants’ information
processing style. Figure 73. Guideline 17 — Provide global controls —
disaggregated by participants’ information processing style. Figure 74.
Guideline 18 — Notify users about changes — disaggregated by participants’
information processing style.

###  A.5. Motivations

Figure 75. Guideline 1 — Make clear what the system can do — disaggregated by
participants’ motivations. Figure 76. Guideline 3 — Time services based on
context — disaggregated by participants’ motivations. Figure 77. Guideline 4 —
Show contextually relevant information — disaggregated by participants’
motivations. Figure 78. Guideline 5 — Match relevant social norms —
disaggregated by participants’ motivations. Figure 79. Guideline 6 — Mitigate
social biases — disaggregated by participants’ motivations. Figure 80.
Guideline 7 — Support efficient invocation — disaggregated by participants’
motivations. Figure 81. Guideline 8 — Support efficient dismissal —
disaggregated by participants’ motivations. Figure 82. Guideline 9 — Support
efficient correction — disaggregated by participants’ motivations. Figure 83.
Guideline 10 — Scope services when in doubt — disaggregated by participants’
motivations. Figure 84. Guideline 11 — Make clear why the system did what it
did — disaggregated by participants’ motivations. Figure 85. Guideline 12 —
Remember recent interactions — disaggregated by participants’ motivations.
Figure 86. Guideline 13 — Learn from user behavior — disaggregated by
participants’ motivations. Figure 87. Guideline 14 — Update and adapt
cautiously — disaggregated by participants’ motivations. Figure 88. Guideline
15 — Encourage granular feedback — disaggregated by participants’ motivations.
Figure 89. Guideline 17 — Provide global controls — disaggregated by
participants’ motivations. Figure 90. Guideline 18 — Notify users about
changes — disaggregated by participants’ motivations.

##  Appendix B Facet Survey

Figure 91. The GenderMag facet survey. Q1, parts 1 — 14 & Q2, part 1 deal with
computer self-efficacy; Q1, parts 15 — 18 & Q2, parts 2, 3 deal with attitudes
toward risk; Q1, parts 19 — 21 & Q2, part 4 deal with information processing
style; Q1, parts 23 — 27 & Q2, part 5 deal with motivations; and Q1, parts 29
— 34 deal & Q2, part 6 deal with learning style.

##  Appendix C Vignettes

##  Appendix D Analysis change in Section 5

###  D.1. Why we combine participants from all 16 experiments in this section.

In Section 4, participants (the experimental units) generated 2 data points
(measurement units) for each of the 10 dependent variables discussed. We
interpreted those measurements by looking through one of their 5 cognitive
styles, like their attitudes toward risk. However, these were dependent on
their experimental assignment. For example, take participant 205 in Guideline
1’s experiment (G1-205) had a risk score of 26. The median score for all 64
participants in this experiment was 25, so G1-205 was risk-averse relative to
their 63 peers (see Table 4 for the rules).

What would have happened if participant 205 had been placed in any other
experiment, since experimental assignment was equally likely? Since they would
have answered the exact same questions in the survey (Appendix B) the exact
same way, they would have still scored a 26 for risk. However, in Guideline
6’s experiment, the median was 27, so participant 205 would have been risk-
tolerant relative to those 68 participants. It turns out that participant 205
would be risk-averse in 8 of the experiments, but they would be risk-tolerant
in the other 8. These variations happened for all 5 of the cognitive styles
(Table 1).

In Section 5, we count up the number of times that participants were
classified with an “Abi”-like cognitive style (left column of Table 1), but we
had to re-compute their cognitive styles because there was a dependency
between their styles and their experimental assignment. For example,
participant 205 above could have between 2 “Abi” styles (in Guidelines 5 & 6)
and 4 “Abi” styles (in Guidelines 1, 3, 9, 11, 13, & 15), so they could have
been in either the “$\geq 3$” or “$<3$” categories in Tables 10 & 11. We
needed to ensure that our outcome variable (number of “Abi” styles) could not
change. Thus, it could not depend on the experimental assignment, so we re-
computed their cognitive styles (Section 2.1) relative to grand medians of all
1,016 participants to analyze the data in Section 5.

##  Appendix E Section 5 \- Fisher’s Exact Test Contingency Tables

|  $\geq 3$ “Abi” styles |  $<3$ “Abi” styles
---|---|---
Woman | 262 | 256
Man | 161 | 308
Estimate (k) | 222
p-value | $<.0001$
Table 10. The 2x2 contingency table used for Fisher’s exact test to investigate the counts of men and women who exhibit a majority (i.e., 3 or more) of the “Abi” cognitive styles from Table 1. Fisher’s exact test statistic ($k$) is the estimated value of the top-left cell, calculated by $R_{1}(C_{1})/T$, were $R_{1}$ & $C_{1}$ are the first row and column totals respectively, and $T$ is the table total. The difference is highly significant. | 18 – 24 | 25 – 34 | 35 – 44 | 45 – 54 | 55 – 64
---|---|---|---|---|---
| $\geq 3$ | $<3$ | $\geq 3$ | $<3$ | $\geq 3$ | $<3$ | $\geq 3$ | $<3$ | $\geq 3$ | $<3$
Woman | 24 | 37 | 97 | 101 | 73 | 74 | 45 | 31 | 23 | 13
Man | 19 | 40 | 72 | 147 | 36 | 72 | 22 | 39 | 12 | 10
k | 21.858 | 80.245 | 62.835 | 37.168 | 13.655
p-value | .4507 | .0010* | .0105* | .0100* | .583
Table 11. The 2x2 contingency tables used for each age group, as with Table
11. The middle three agegroup categories had significant gender differences
(Fisher’s exact test).