d\left(\dot{q}^{i}a_{i}+\dot{p}_{i}b^{i}+\dot{z}\right).$

######  Definition 10.2 (Gradient vector field).

Given a Hamiltonian $H\in\mathcal{C}^{\infty}(M),$ define the gradient vector
field of $H$ as

$\operatorname{grad}H:=\sharp_{\lambda,\omega}(dH).$

In Darboux coordinates, the gradient vector field is written

$\operatorname{grad}H=\left(\frac{\partial H}{\partial
p_{i}}-b^{i}\frac{\partial H}{\partial z}\right)\frac{\partial}{\partial
q^{i}}+\left(-\frac{\partial H}{\partial q^{i}}+a_{i}\frac{\partial
H}{\partial z}\right)\frac{\partial}{\partial p_{i}}+\left(b^{i}\frac{\partial
H}{\partial q^{i}}-a_{i}\frac{\partial H}{\partial p_{i}}+\frac{\partial
H}{\partial z}\right)\frac{\partial}{\partial z}.$

It is easily checked that $X:M\rightarrow TM$ is locally a gradient vector
field if and only if $X(M)$ is a Lagrangian submanifold of $(TM,\Omega_{0})$.
Indeed, we have the equality

$X^{*}\Omega_{0}=-d\flat_{\lambda,\omega}(X).$

When $\Lambda$ comes from a Jacobi bracket on $M$, that is, when

$d\lambda=f\omega,\,\,\,[f\mathcal{R},\Lambda]=0,$

for some function $f$ on $M$, we have the Hamiltonian vector field of the
Jacobi structure $(\Lambda,f\mathcal{R})$:

$X_{H}=\sharp_{\Lambda}(dH)+fH\mathcal{R}=-\operatorname{grad}H+(\mathcal{R}(H)+fH)\mathcal{R}.$

In Darboux coordinates it has the expression:

$X_{H}=\left(-\frac{\partial H}{\partial p_{i}}+b^{i}\frac{\partial
H}{\partial q^{i}}\right)\frac{\partial}{\partial q^{i}}+\left(\frac{\partial
H}{\partial q^{i}}-a_{i}\frac{\partial H}{\partial
z}\right)\frac{\partial}{\partial p_{i}}+\left(a_{i}\frac{\partial H}{\partial
p_{i}}-b^{i}\frac{\partial H}{\partial
q^{i}}+fH\right)\frac{\partial}{\partial z}.$

Let us interpret the Hamiltonian vector field as a Lagrangian submanifold of
$TM$, with some symplectic form. First, observe that

$\displaystyle
X_{H}^{*}\Omega_{0}=-d(\flat_{\lambda,\omega}(dH))=-d\left(\mathcal{R}(H)\lambda+fH\lambda\right).$

Therefore, defining the symplectic form

$\Omega_{H}:=\Omega_{0}+d(\mathcal{R}(H)\lambda+fH\lambda)^{v},$

we have that $X_{H}$ defines a Lagrangian submanifold of $(TM,\Omega_{H}).$

###  10.2. Vertical coisotropic reduction

######  Theorem 10.1 (Vertical coisotropic reduction in stable Hamiltonian
structures).

Let $i:N\hookrightarrow M$ be a vertical coisotropic submanifold such that
$(TN)^{\perp_{\Lambda}}$ defines an integrable distribution. Let $\mathcal{F}$
be the set of leaves and suppose that $N/\mathcal{F}$ admits a manifold
structure such that the canonical projection $\pi:N\rightarrow N/\mathcal{F}$
defines a submersion. If $i^{*}d\lambda=0$ in $TN\cap\mathcal{H}$, then
$N/\mathcal{F}$ admits an unique stable Hamiltonian system structure
$(\omega_{N},\lambda_{N})$ such that $\pi^{*}\omega_{N}=i^{*}\omega$ and
$\pi^{*}\lambda_{N}=i^{*}\lambda$. The following diagram summarizes the
situation:

${N}$${M}$${N/\mathcal{F}}$$\scriptstyle{i}$$\scriptstyle{\pi}$

###### Proof.

The proof is similar to the the proof of  Theorem 5.1. Asking
$i^{*}d\lambda=0$ is necessary to guarantee the well-definedness of
$\lambda_{N}$ in the quotient using

$\mathcal{L}_{X}\lambda_{0}=i_{X}d\lambda-0+di_{X}\lambda_{0}=0,$

where $\lambda_{0}=i^{*}\lambda$. It would only remain to check that

$\operatorname{Ker}\omega_{N}\subseteq\operatorname{Ker}d\lambda_{N}.$

Indeed, since
$\operatorname{Ker}\omega_{N}=\langle\mathcal{\mathcal{R}_{N}}\rangle$, and
$\mathcal{R}_{N}=\pi_{*}\mathcal{R}$, it follows from

$\pi^{*}(i_{\mathcal{R}_{N}}d\lambda_{N})=i_{\mathcal{R}}d\lambda=0.$

∎

####  10.2.1. Projection of Lagrangian submanifolds

We have the result:

######  Proposition 10.4 (Projection of Lagrangian submanifolds).

Let $i:L\hookrightarrow M$ be a Lagrangian submanifold. If $L$ and $N$ have
clean intersection and $\pi(L\cap N)$ is a submanifold in $N/\mathcal{F}$,
then it is Lagrangian.

###### Proof.

The proof is similar to the the proof of Proposition 5.5 and Proposition 5.6,
since the proof reduces to the study of each tangent space. ∎

###  10.3. Horizontal coisotropic reduction

######  Theorem 10.2 (Horizontal coisotropic reduction in stable Hamiltonian
structures).

Let $i:N\rightarrow M$ be a coisotropic horizontal submanifold such that
$(TN)^{\perp_{\Lambda}}$ defines an integrable distribution. Let $\mathcal{F}$
be the set of leaves of the foliation and suppose that $N/\mathcal{F}$ admits
a manifold structure such that the canonical projection $\pi:N\rightarrow
N/\mathcal{F}$ defines a submersion. Then $N/\mathcal{F}$ admits and unique
symplectic structure $\omega_{N}$ such that $\pi^{*}\omega_{N}=i^{*}\omega$.
The following diagram summarizes the situation:

${N}$${M}$${N/\mathcal{F}}$$\scriptstyle{i}$$\scriptstyle{\pi}$

###### Proof.

The proof is similar to the the proof of Theorem 3.2. ∎

####  10.3.1. Projection of Lagrangian submanifolds

######  Proposition 10.5 (Projection of Lagrangian submanifolds).

Let $L\hookrightarrow M$ be a Lagrangian submanifold. If $L$ and $N$ have
clean intersection and $\pi(L\cap N)$ is a submanifold in $N/\mathcal{L}$,
then it is Lagrangian.

###### Proof.

The proof is similar to the the proof of Proposition 3.4 since we only need to
check it in every tangent space. ∎

##  11\. Conclusions

In this paper we have reviewed the concept of coisotropic and Lagrangian (and
Legendrian) submanifolds in different geometric settings. We have shown the
connection of these geometric constructions with the different dynamics that
usually appear in mechanics. All these different geometries (symplectic,
cosymplectic, contact, cocontact) can be classified within Jacobi geometry (in
some case, Poisson, a particular case of Jacobi structure). This approach
allows in a simple way to see these particular situations from a more general
point of view. Sometimes it is important to stand at a certain altitude in
order to realise that these particular situations respond to the same
geometrical pattern.

In each case we have introduced the notions of coisotropic and Lagrangian
submanifold and studied in detail the corresponding coisotropic reduction
theorems. The interpretations of the different types of vector fields in the
different types of geometry as Lagrangian or Legendrian submanifolds are
summarized in Table 1. Also, the results on coisotropic reduction are
summarized in Table 2

Table 1: Interpretation of vector fields as Lagrangian or Legendrian
submanifolds

Geometry Vector field Type of submanifold Ambient manifold Symplectic
Hamiltonian Lagrangian $(TM,\omega_{0})$, symplectic $(M,\omega)$
$X_{H}=\sharp_{\omega}(dH)$ $\omega_{0}=\flat_{\omega}^{*}\omega_{M}$
Cosymplectic Gradient Lagrangian $(TM,\Omega_{0})$, symplectic
$(M,\Omega,\theta)$ $\operatorname{grad}H=\sharp_{\theta,\Omega}(dH)$
$\Omega_{0}=\flat_{\theta,\Omega}^{*}\Omega_{M}$ Hamiltonian Lagrangian
$(TM,\Omega_{H})$, symplectic
$X_{H}=\operatorname{grad}H-\mathcal{R}(H)\mathcal{R}$
$\Omega_{H}=\Omega_{0}+(\operatorname{d}(\mathcal{R}(H))\wedge\theta)^{v}$
Lagrangian $(TM\times\mathbb{R},\Omega_{H},ds)$, cosymplectic Evolution
Lagrangian $(TM,\Omega_{H})$, symplectic
$\mathcal{E}_{H}=\operatorname{grad}H+\mathcal{R}$ Contact Gradient Lagrangian
$(TM,\Omega_{0})$, symplectic $(M,\eta)$
$\operatorname{grad}H=\sharp_{\eta}(dH)$
$\Omega_{0}=\flat_{\eta}^{*}\Omega_{M}$ Hamiltonian Lagrangian
$(TM,\Omega_{H})$, symplectic
$X_{H}=\operatorname{grad}H-(\mathcal{R}(H)+H)\mathcal{R}$
$\Omega_{H}=\Omega_{0}-\operatorname{d}(\mathcal{R}(H)\eta+H\eta)^{v}$
Legendrian $(TM\times\mathbb{R},\hat{\eta})$, contact
$\hat{\eta}=\eta^{c}+t\eta^{v}$ Evolution Lagrangian
$(TM\times\mathbb{R},\widetilde{\Omega}_{H})$, symplectic
$\mathcal{E}_{H}=X_{H}+H\mathcal{R}$
$\widetilde{\Omega}_{H}=\Omega_{0}-\operatorname{d}(\mathcal{R}(H)\eta)^{v}$
Cocontact Gradient Lagrangian $(TM,\Omega_{0})$, symplectic $(M,\theta,\eta)$
$\operatorname{grad}H=\sharp_{\theta,\eta}(dH)$
$\Omega_{0}=\flat_{\eta,\theta}^{*}\Omega_{M}$ Hamiltonian Lagrangian
$(TM,\Omega_{H})$, symplectic
$X_{H}=\operatorname{grad}(H)-(\mathcal{R}_{z}(H)+H)\mathcal{R}_{z}$
$\Omega_{H}=\Omega_{0}-\operatorname{d}(\mathcal{R}_{z}(H)\eta+H\eta)^{v}$
$\quad\quad\quad+(1-\mathcal{R}_{t}(H))\mathcal{R}_{t}$
$\quad\quad\quad+d(\theta-\mathcal{R}_{t}(H)\theta)^{v}$ Legendrian
$(TM\times\mathbb{R}\times\mathbb{R},\widetilde{\theta},\widetilde{\eta})$,
cocontact $\widetilde{\eta}=\eta^{c}+s\eta^{v}+\theta^{c}+e\theta^{v}$
$\widetilde{\theta}=\theta^{c}$ SHS Gradient Lagrangian $(TM,\Omega_{0})$,
symplectic $(M,\omega,\lambda)$
$\operatorname{grad}H=\sharp_{\lambda,\omega}({dH})$
$\Omega_{0}=\flat_{\lambda,\omega}^{*}\Omega_{M}$ If $d\lambda=f\omega,$
Hamiltonian Lagrangian $(TM,\Omega_{H})$, symplectic
$\sharp_{\lambda,\Omega}(df)\in\mathcal{V}$
$X_{H}=-\operatorname{grad}H+(\mathcal{R}(H)+fH)\mathcal{R}$
$\Omega_{H}=\Omega_{0}+\operatorname{d}(\mathcal{R}(H)\lambda+fH\lambda)^{v}$

Table 2: Summary of results on coisotropic reduction

Geometry Coisotropic submanifold Resulting manifold Projection of Lagrangian
$N\hookrightarrow M$ and Legendrian submanifolds Symplectic Arbitrary
$(N/\mathcal{F},\omega_{N})$, symplectic $L$ Lagrangian $\implies$ $L_{N}$
Lagrangian Cosymplectic Vertical $(N/\mathcal{F},\theta_{N},\Omega_{N})$,
cosymplectic $L$ Lagrangian $\implies$ $L_{N}$ Lagrangian Horizontal
$(N,\Omega_{N})$, symplectic $L$ Lagrangian $\implies$ $L_{N}$ Lagrangian
Arbitrary Foliation consisting of symplectic manifolds of $N/\mathcal{F}$
Contact Vertical $(N/\mathcal{F},\eta_{N})$, contact $L$ Legendrian $\implies$
$L_{N}$ Legendrian Horizontal $\operatorname{dim}N/\mathcal{F}=0$ Cocontact
$tz$-vertical $(N/\mathcal{F},\theta_{N},\eta_{N})$, cocontact $L$ Legendrian
$\implies$ $L_{N}$ Legendrian $t$-vertical, $z$-horizontal
$\operatorname{dim}N/\mathcal{F}=1$, $\theta_{N}\neq 0$ $z$-vertical,
$t$-horizontal $(N/\mathcal{F},\eta_{N}),$ contact $L$ Legendrian $\implies$
$L_{N}$ Legendrian $tz$-horizontal $\operatorname{dim}N/\mathcal{F}=0$ SHS
Vertical $(N/\mathcal{F},\omega_{N},\lambda_{N})$, stable Hamiltonian $L$
Lagrangian $\implies$ $L_{N}$ Lagrangian Horizontal
$(N/\mathcal{F},\omega_{N})$, symplectic $L$ Lagrangian $\implies$ $L_{N}$
Lagrangian

##  12\. Acknowledgements

We acknowledge the financial support of Grant PID2019-106715GBC21, the Severo
Ochoa Programme for Centres of Excellence in R&D (CEX2019-000904-S), and JAE
Intro Programme 2022 (Becas de Introducción a la Investigación para
estudiantes universitarios). Finally, we also thank the referees for their
corrections and suggestions.

## References


# Toward Supporting Perceptual Complementarity in Human-AI Collaboration via
Reflection on Unobservables

Kenneth Holstein<EMAIL_ADDRESS>Carnegie Mellon University5000
Forbes AvePittsburghPennsylvaniaUSA ,  Maria De-Arteaga
<EMAIL_ADDRESS>University of Texas at Austin110 Inner Campus
DriveAustinTexasUSA ,  Lakshmi Tumati<EMAIL_ADDRESS>Carnegie Mellon
University5000 Forbes AvePittsburghPennsylvaniaUSA and  Yanghuidi Cheng
<EMAIL_ADDRESS>Carnegie Mellon University5000 Forbes
AvePittsburghPennsylvaniaUSA

###### Abstract.

In many real world contexts, successful human-AI collaboration requires humans
to productively integrate complementary sources of information into AI-
informed decisions. However, in practice human decision-makers often lack
understanding of what information an AI model has access to, in relation to
themselves. There are few available guidelines regarding how to effectively
communicate about unobservables: features that may influence the outcome, but
which are unavailable to the model. In this work, we conducted an online
experiment to understand whether and how explicitly communicating potentially
relevant unobservables influences how people integrate model outputs and
unobservables when making predictions. Our findings indicate that presenting
prompts about unobservables can change how humans integrate model outputs and
unobservables, but do not necessarily lead to improved performance.
Furthermore, the impacts of these prompts can vary depending on decision-
makers’ prior domain expertise. We conclude by discussing implications for
future research and design of AI-based decision support tools.

human-AI complementarity, unobservables, algorithm-assisted decision-making,
behavioral experiment

††copyright: rightsretained††journal: PACMHCI††journalyear:
2023††journalvolume: 7††journalnumber: CSCW1††article: 152††publicationmonth:
4††doi: 10.1145/3579628

##  1\. Introduction

AI-based decision support tools (ADS) are being used to augment human
decision-making across a growing range of domains—from predictive risk models
used in public services (De-Arteaga et al., 2020; Holten Møller et al., 2020;
Kawakami et al., 2022a; Levy et al., 2021; Saxena et al., 2020), to AI-based
teacher support tools used in K-12 education (An et al., 2020; Chine et al.,
2022; Dickler et al., 2021; Holstein et al., 2020; Ritter et al., 2016), to
clinical decision support tools used in healthcare (Lee et al., 2021; Patel et
al., 2019; Wang et al., 2021; Yang et al., 2016; Yang et al., 2019). The
sociotechnical design of these systems often relies, whether implicitly or
explicitly, on the potential for human-AI complementarity. To date, however,
scientific and design knowledge remains scarce regarding how we might bring
out the best of both human and algorithmic judgment in practice. Recent work
offers evidence that AI and human decision-makers can complement each other’s
capabilities and help to overcome each other’s limitations (Cheng et al.,
2022; De-Arteaga et al., 2020; Hemmer et al., 2022; Holstein and Aleven,
2022). Yet achieving such synergy in human-AI decision-making is far from
guaranteed. For instance, a long line of literature demonstrates that human
decision-makers are often either too skeptical of useful AI outputs or too
reliant upon erroneous or harmfully biased AI outputs (Buçinca et al., 2021;
Dietvorst et al., 2015; Green and Chen, 2019b; Lee and See, 2004; Poursabzi-
Sangdeh et al., 2021).

In many real-world settings where ADS are employed, humans use AI outputs as
just one of several sources of information at their disposal to inform a
decision (De-Arteaga et al., 2020; Holstein et al., 2020; Kawakami et al.,
2022a). Thus, humans’ ability to successfully integrate information
communicated by the ADS with other sources of information is critical to
effective decision-making. A central dimension of human-AI complementarity is
the often complementary nature of the information sources that the human and
the AI model each have access to and are able to parse (De-Arteaga et al.,
2020; Holstein et al., 2020; Kawakami et al., 2022a). For example, in the
context of AI-augmented healthcare, an AI system may have greater ability to
parse through large quantities of time-series data collected via continuous
bedside monitoring, while a physician can better perceive changes in patients’
emotional state via their physical presentation and by listening to their
lived experience (Lee et al., 2021; Yang et al., 2016). Similarly, in the
context of AI-augmented child welfare decision-making, AI models may have
access to large quantities of administrative data, while human workers have
access to the rich information communicated during a phone conversation with a
caller to a child maltreatment hotline (De-Arteaga et al., 2020, 2021;
Kawakami et al., 2022a, b).

More generally, consider ADS tools that rely on machine learned prediction
models, which estimate an outcome or a probability of an event $Y$ (e.g., a
predicted house selling price or the risk of an adverse health outcome), given
a set of covariates or features $X$ available to the model. Often, there are
additional features $Z$ (often called “unobservables” in the machine learning
literature (Lakkaraju et al., 2017)) that influence the outcome but are
unavailable to the model. In many real-world settings, such as the examples
described above, some of these features are available to the human, and it is
the hope of those deploying ADS tools that the humans using these tools will
be able to complement the AI model’s capabilities by integrating this
additional information into their decisions. However, prior field research
indicates that humans are often unaware of exactly which sources of
information are uniquely available to themselves versus an AI model (Holstein
et al., 2019; Kawakami et al., 2022a, b). Little is known regarding how we
might design ADS tools to amplify this form of human-AI complementarity.

In this paper, we investigate the impacts of interventions that prompt people
to reflect on complementary abilities between themselves and AI models. We
focus on perceptual complementarity, a form of human-AI complementarity that
has been observed in prior work studying real-world human-AI collaborations.
In a series of field studies observing how classroom teachers integrate real-
time AI recommendations into their decision-making, Holstein et al. (Holstein
et al., 2019; Holstein and Aleven, 2022) observed that teachers typically
cross-checked what the AI system told them about their students with what they
were able to see with their own eyes and ears (e.g., student body language and
students’ own descriptions of the challenges they were facing). In many cases,
considering multiple sources of information led teachers to re-interpret the
AI recommendations or deem them irrelevant to the actual situation at hand.
Similarly, in retrospective data analyses and field observations of AI-
assisted decision-making by child maltreatment hotline call screeners, De-
Arteaga et al. and Kawakami et al. (De-Arteaga et al., 2020; Kawakami et al.,
2022a) found evidence that call workers did not indiscriminately adhere to AI
recommendations. Instead, call workers were more likely to avoid errors of
omission, which correspond to false negative errors, by integrating other
sources of information instead of relying on AI outputs alone (De-Arteaga et
al., 2020). However, a series of interview studies with call screeners
revealed that they felt uncertain about which information the AI model did or
did not have access to, and that they wanted more support from the ADS
interface in understanding how exactly this information complemented other
information sources at their disposal (Kawakami et al., 2022a, b).

In real-world decision-making contexts like the education and child welfare
examples above, information that is available to humans but unobservable by AI
models is often difficult to precisely characterize and externalize without
losing important, decision-relevant context and nuance. Indeed, the difficulty
of fully capturing such human judgments and perceptions in structured data is
one reason why such information may be unavailable to an AI model in the first
place. In these settings, human experts often exhibit an impressive ability to
effectively integrate rich implicit inferences into their decision-making
(Hemmer et al., 2022; Kawakami et al., 2022a; Koedinger et al., 2012; Lake et
al., 2017; Rastogi et al., 2022). In several decision-making contexts,
however, unobservables may correspond to structured data. For instance,
consider the case of healthcare algorithms, which frequently rely on only a
subset of the data available. An algorithm meant to assist physicians in post-
cardiac arrest care of comatose patients may predict the likelihood of a
positive neurological recovery by relying on EEG signals (De-Arteaga et al.,
2019). Meanwhile, physicians may also have access to complementary information
regarding the patient’s health (e.g., a series of recorded test results). In
either of these scenarios—whether AI unobservables are externalized in
structured data or exist primarily in human decision-makers’ minds—we expect
that promoting reflection on human-AI perceptual complementarity has the
potential to improve AI-assisted decision-making.

To investigate, we conduct a 3-condition online experiment to explore how AI-
assisted decision-making is impacted by interface prompts that encourage
people to reflect on asymmetries in information between themselves and an ADS.
In particular, we ask whether and how such prompts might impact (1) how people
integrate model outputs with information about features that are unobserved by
the model, and (2) whether such prompts improve humans’ predictive accuracy.
As a context for this study, we consider an AI-assisted house price prediction
task, in which participants are presented with structured data on features of
houses (Chiang and Yin, 2021; Poursabzi-Sangdeh et al., 2021), some of which
are unobserved by the AI model and some of which are more informative than
others, and are asked to predict each house’s sale price.

Overall, we found that presenting prompts about unobservables can change how
people integrate different sources of information. Participants who were shown
in-the-moment prompts were more likely to make predictions based on a
combination of model predictions and their own judgments. Despite this shift,
prompting participants about unobservables did not lead to improved predictive
performance in the context of our study: human+AI predictions were more
accurate than AI predictions across all conditions. Furthermore, we observed
that the impacts of these prompts can vary depending on decision-makers’ prior
domain expertise. For example, among participants who had less prior
experience with house price prediction, increasing their overall awareness of
the presence of unobservables during a training phase had the effect of
harming their predictive accuracy. However, this effect was mitigated when
these less-experienced participants were also presented with in-the-moment
prompts, beyond the training phase, that served as continuous reminders
regarding which specific features are unobservable to the model.

This work contributes to an emerging body of research in human-computer
interaction and CSCW that investigates new ways to help human decision-makers
more effectively integrate AI capabilities with their own expertise. Our
research builds upon recent work that looks beyond the design of “AI
explanations,” exploring a broader design space for interactions to support
better human-AI decision-making (Buçinca et al., 2021; Chiang and Yin, 2021;
Poursabzi-Sangdeh et al., 2021). Extending this body of work, we contribute an
initial empirical investigation into the impacts of a theoretically promising,
yet under-explored interface-level intervention for ADS: in-the-moment prompts
that promote reflection on unobservables. We close by discussing key
implications of our findings for future work, paying close attention to the
complementary roles of different types of unobservables, the volume of
information that humans must parse, and the role of the feedback signals
available to humans in a given context.

##  2\. Background and Related Work

The design and evaluation of ADS tools to complement human strengths and
improve decision quality has received considerable attention in recent years.
In this section we briefly review two bodies of literature that are of
particular relevance to our work.

###  2.1. Design for human-AI complementarity

ADS are increasingly used to augment human decision-making across a range of
real-world contexts, including education (An et al., 2020; Dickler et al.,
2021; Holstein et al., 2020), healthcare (Lee et al., 2021; Wang et al., 2021;
Yang et al., 2016), social work (Chouldechova et al., 2018; Levy et al., 2021;
Saxena et al., 2020), and criminal justice (Albright, 2019; Green and Chen,
2019a; Kleinberg et al., 2018). Such systems seek to improve decision quality
by leveraging the complementary skills of humans and AI systems. To achieve
this, different configurations for dividing labor and integrating human and
algorithmic assessments have been proposed. A stream of work in machine
learning has focused on developing automated mechanisms to divide tasks among
humans and AI systems (e.g., to ensure that AI systems handle the instances
that are most difficult for humans and vice versa), (Madras et al., 2018;
Wilder et al., 2020; Gao et al., 2021; Rastogi et al., 2022). However, the
standard practice in real-world, high-stakes domains maintains decision-making
power in the hands of humans: AI outputs are used by a human decision-maker as
one of multiple available sources of information.

Central to the design of these systems is the role of humans’ discretionary
power. Typically, the hope is that integrating an AI model as a source of
information will improve humans’ decision quality, while retaining humans’
autonomy and responsibility in a high-stakes decision process. Accordingly,
recent field research, conducted in real-world AI-assisted decision-making
contexts, provides early evidence that, when human workers are empowered to
evaluate and (as appropriate) second-guess AI predictions, this may support
more effective and equitable decision-making (Cheng et al., 2022; De-Arteaga
et al., 2020; Holstein and Aleven, 2022). However, empirical results in this
area have been mixed, and it remains an open research question how ADS tools
can best be designed to foster such human–AI synergy. In some prior studies,
integrations of human and machine intelligence have been shown to be more
effective than either humans or AI systems working alone (e.g., (De-Arteaga et
al., 2020; Holstein and Aleven, 2022)). Yet in other studies, human–AI
collaboration has failed to improve or have even harmed decision quality
(e.g., (Green and Chen, 2019b; Poursabzi-Sangdeh et al., 2021; Tan et al.,
2018)). For example, empirical work in the criminal justice domain has shown
that humans’ discretionary adherence may serve to exacerbate disparities
across demographic groups (Albright, 2019; Stevenson and Doleac, 2021), while
findings in the child welfare domain have shown that human discretion may
mitigate disparities when compared to AI assessments in isolation (Cheng et
al., 2022; Fogliato et al., 2022).

To date, scientific and design knowledge remains scarce regarding what factors
facilitate or hinder complementary human-AI performance. To investigate, a
line of research in human-computer interaction and CSCW has begun to explore
the design space of interactions that can support humans in more effectively
integrating AI capabilities with their own strengths as human experts. For
example, several recent experimental studies have investigated the impacts of
interfaces that present human decision-makers with in-the-moment explanations
for specific predictions or recommendations from an AI model. Recent results
have shown that, contrary to researchers’ intuitions, presenting such
explanations can often backfire, encouraging humans to over-rely on AI outputs
even in the presence of large errors that they may have otherwise been able to
notice (e.g., (Bansal et al., 2021; Poursabzi-Sangdeh et al., 2021)).

Moving beyond the design of “AI explanations”, recent work has begun to
explore a broader design space for interactions to support better human-AI
decision-making. For example, recent experimental results signal potential for
relatively simple cognitive cues (e.g., real-time interface prompts warning
that a given instance may be out-of-distribution for an AI model) to help
foster more effective use of ADS (Buçinca et al., 2021; Chiang and Yin, 2021;
Poursabzi-Sangdeh et al., 2021). Our research builds upon this prior work,
providing an initial empirical investigation into the impacts of a
theoretically promising interface-level intervention for ADS: in-the-moment
prompts that encourage human decision-makers to reflect on complementary
abilities between themselves and AI models.

###  2.2. Algorithmic prediction under unobservables

In many AI-assisted decision-making settings, humans have access to sources of
information that are not available to the AI model, offering a potential
source of complementarity. Formally speaking, when ADS are machine learned
prediction models that estimate the probability that an event $Y$ will occur
given a set of covariates or features $X$, unobservables refer to features $Z$
that influence the outcome $Y$ but are not available to the AI model (Wansbeek
and Meijer, 2001). In some cases, a subset of the features $Z$ may be
observable to the human. For instance, in the child welfare context an ADS may
use administrative data associated with the child and family members to
predict the probability that an investigation following a call to a child
abuse hotline will result in out of home placement, but the information
communicated in the call is only accessible to the call worker (Chouldechova
et al., 2018).

When important information is not observed by the AI model, there is a risk of
unreliable and misestimated predictions (Wansbeek and Meijer, 2001). This risk
may be exacerbated in the context of ADS, where labels suffer from the
“selective labels problem” (Lakkaraju et al., 2017). For example, in the
historical data used to train AI models in the child welfare domain, the
result of a child welfare investigation is only available if a human decided
to the screen in the call (De-Arteaga et al., 2021). The selective nature of
labels available for training requires the use of sampling bias correction
methods that rely on the assumption that there are no features that are
unobservable to the AI model but observed by humans (Lakkaraju et al., 2017).

Empirical work has shown that even in settings where AI models display overall
better performance, humans can outperform the algorithm for instances with
unique or complex characteristics (Karlinsky-Shichor and Netzer, 2019). Recent
research has also demonstrated the potential of information asymmetry as a
source of complementary human-AI performance, showing that humans can
successfully integrate contextual information when making use of AI
recommendations (Cheng et al., 2022; De-Arteaga et al., 2020; Hemmer et al.,
2022; Holstein and Aleven, 2022). However, in many settings human decision-
makers do not have a clear understanding of what information is and is not
available to the algorithm (Holstein et al., 2019; Kawakami et al., 2022a, b),
which may hinder their ability to integrate contextual information.
Furthermore, little is known regarding how to best design ADS to support human
decision-makers in effectively integrating across AI outputs and
unobservables. In this study, we investigate whether prompting people to
reflect on asymmetries in information changes the way in which humans
integrate the information available to them, and whether this significantly
improves human-AI performance by allowing human decision-makers to make better
use of unobservables.

##  3\. Methods

In this study, we conducted an online behavioral experiment to understand
whether and how explicitly communicating potentially relevant unobservables
influences the way people make use of algorithmic assistance when making
predictions. Our primary research questions are:

  * •

RQ1: Does presenting reflection prompts about unobservables change how people
integrate model outputs and unobservables when making predictions? If so, how?

  * •

RQ2: Does presenting reflection prompts about unobservables help people
integrate model outputs and unobservables more effectively?

###  3.1. Task and Dataset Selection

To explore these questions, we sought to select a task for which our study
population (crowdworkers and participants recruited via social media) might be
expected to have relevant prior knowledge and experience. As such, we avoided
specialized tasks used in prior studies exploring AI-assisted decision-making,
such as judicial decision-making tasks (Fogliato et al., 2021; Green and Chen,
2019a; Lurie and Mulligan, 2020). Instead, building upon other recent
crowdsourcing study designs (e.g., (Chiang and Yin, 2021; Hemmer et al., 2022;
Poursabzi-Sangdeh et al., 2021)), we chose AI-assisted house price prediction
as a setting for this study, for several reasons. First, many people may have
relevant knowledge or experience in predicting house prices. In addition,
house price prediction models like Zillow’s Zestimate are widely deployed,
meaning that participants may already be familiar with AI-assisted decision-
making in this setting. Finally, people may find this task interesting, even
if they have not themselves purchased a home (Chiang and Yin, 2021; Fan et
al., 2018; Poursabzi-Sangdeh et al., 2021). The housing data used in this
study came from the Ames, Iowa Housing Dataset (De Cock, 2011).

###  3.2. Participant Recruitment and Compensation

We recruited a total of 664 participants via the crowdsourcing platform
Prolific and through social media channels (Facebook groups and Nextdoor). All
participants were based in the US, and were over the age of 18. On Facebook,
we targeted relevant special interest groups such as “First Time Home Buyers”
and “Real Estate Investing,” with the aim of recruiting participants who have
significant experience with house price prediction. Similarly, on Prolific, we
advertised for participants who have prior experience browsing homes online
(e.g., using online platforms such as Zillow or Redfin). Our online survey
included a background question to track participants’ self-reported level of
experience browsing for homes: participants were asked to indicate how much
time they have previously spent browsing homes for sale on a 5-point scale
ranging from “None at all” to “A great deal”. In our analysis, we categorize
participants who report having previously spent “A lot” or “A great deal” of
time browsing homes online as having higher prior experience, and the
remaining participants as having lower prior experience. All participants
received a base payment of $6. To encourage participants to carefully use the
available information to make accurate predictions during the main phase of
the study, each participant was informed that they would receive an additional
$6 payment if their prediction accuracy ranked in the top 10% of participants.

###  3.3. Task Design

Participants performed a sequence of 24 house price prediction tasks, split
evenly across a training phase and a testing phase, with the assistance of a
pre-trained model. All participants saw the same set of 12 houses in both the
training and testing phases, although the order was randomized across
participants. The training phase was intended to familiarize participants with
the house price prediction task and the model’s capabilities, and to help them
learn to calibrate their predictions to the specific, unnamed US town used for
this study (Chiang and Yin, 2021; Poursabzi-Sangdeh et al., 2021). After
reviewing a house’s information (described in more detail below), along with a
model’s predicted sale price, participants were asked to predict the house’s
actual sale price. During the training phase, participants received immediate
feedback after making a prediction: they were shown the actual sale price of
the house, alongside their own prediction, the model’s prediction, and the
house’s information. During the testing phase, participants were tasked with
predicting prices for 12 houses without receiving any feedback.

For each house, participants were shown “Facts and Features”, representing the
values for the eight features in the dataset that were most predictive of
house sale prices. These included the year built, the type of heating, the
numbers of full baths and half baths, whether the house had a paved driveway,
the zoning classification, and ratings of the house’s material and finish and
overall condition. We chose to present only eight features to ensure that the
full set of features would not be unmanageable for participants to scan
through during the study (Poursabzi-Sangdeh et al., 2021). In addition, to
minimize the chances that participants would draw upon prior knowledge about
house prices in a specific time period or region of the US, participants were
not told the actual location and time of sale for each house. Instead,
participants were simply told that all houses were located in “the same US
town” and that all houses “sold around the same time.”

Following prior online experiments studying AI-assisted house price prediction
(e.g., (Chiang and Yin, 2021; Poursabzi-Sangdeh et al., 2021)), participants
had access only to this tabular information, and were not able to view images
of the homes.111See Hemmer et al. (2022) for a recent exception. In their
study, participants also have access to images of houses. In a real-world
deployment setting, we envision that the kinds of interface prompts explored
in our study would prompt human decision-makers to reflect on particular
unobservables, without necessarily requiring that these unobservables are
externalized in structured data. For example, an interface might prompt
reflection on their own, internal perceptions of variables that a ADS
interface designer expects to be relevant for decision-making, but which lie
outside of the set of features available to to the AI model (such as a
patient’s presentation in a healthcare context, or a student’s current
motivations and life circumstances in an education context). However, to
simulate such settings in our online experimental study, we not only draw
participants’ attention to particular unobservables, but also present study
participants with explicit values for each unobservable.

The model prediction for each house was generated by a linear regression model
trained on the Ames, Iowa Housing Dataset (De Cock, 2011). To simulate a real-
world scenario in which we might expect to see complementary predictive
performance between humans and a trained model (cf. (Bansal et al., 2021)), we
artificially induced unobservables in the model prediction by removing three
of the eight features listed above from the information that the model had
access to: rating of material and finish, number of full baths, and type of
heating. As discussed below, these features were chosen as the unobservables
for this study given that they were the most predictive of a house’s sales
price in isolation. Thus, participants in our study had access to three
features that were unobservable to the model. The resulting “partial” model
used in this study had an accuracy of 69.77%, compared with an accuracy of
80.63% for the “full” model that includes all three unobservables. Note that
participants never interacted with the “full” model, which was only used to
guide the study design. We wished to minimize the risk that participants could
succeed at our prediction task by learning an overly simple rule such as
“always predict a higher price than the model.” Thus, we randomly sampled the
houses shown to participants subject to the constraint that half of the houses
within each of the training and testing phases were ones for which including
the unobservable features in a trained model would result in a higher house
price prediction compared with omitting them (i.e., houses for which the full
model would make higher predictions than the partial model), and the other
half were ones for which including the unobservables would result in a lower
house price prediction compared with omitting them.

After completing all of the prediction tasks in the testing phase,
participants were asked to briefly describe how they believed they had been
making their predictions during the study. In particular, participants were
invited to share reflections regarding how the model’s predictions informed
their own predictions, and whether they paid particular attention to certain
features of each house.

###  3.4. Unobservable Features

The three features selected as unobservables, which hold the greatest
predictive power in isolation, exemplify the heterogeneity in the types of
unobservables that may be present in an AI-assisted decision-making scenario.

  * •

Unobservables with complementary predictive power: The rating of material and
finish and the number of bathrooms complement the information that is captured
by the features observed by the model. Taking these features into account
improves predictive power, over and above the model’s capabilities. Including
the rating of material and finish during model training yields a 6.23%
increase in accuracy, and including the number of bathrooms yields an
additional 4.64% increase.

  * •

Unobservables with minimal complementary predictive power: The type of heating
holds less complementary predictive power beyond the information already
captured by the model. Including the type of heating during model training
yields a 0.80% increase in accuracy, suggesting that much of the signal this
feature carries is already accounted for by the model via correlations with
features that the model can observe.

  * •

Unobservables with low variation within the data observed by humans: In
addition, although the type of heating was highly predictive on the larger
dataset upon which the model was trained, this categorical feature exhibited
little variation within the smaller sample of houses presented to study
participants. During the training phase, 10 out of 12 houses shown to
participants had the same type of heating. This property limits what humans
can learn about an unobservable’s predictive power based on the immediate
feedback provided during the training phase. However, participants who have
higher prior experience on the prediction task may still be able to leverage
their prior knowledge about this feature’s influence.

###  3.5. Experimental Design

Each participant was randomly assigned to one of three experimental
conditions. The No prompts condition served as a control condition:
participants were not presented with any prompts about unobservables during
the study. In the Initial prompt condition, participants received a prompt at
the very beginning of the training phase, informing them that the model did
not have access to three of the features that they themselves were able to
observe, and showing them which features these were (see Figure 1). However,
participants in this condition were not prompted about unobservables again at
any point during the study. In the Initial + Real-time prompts condition, in
addition to receiving this initial prompt, participants were prompted to
reflect on unobservables, as in Figure 1, every time they were shown house
information during the training and testing phases. We included both of these
experimental treatments in an effort to tease apart the potential impacts of
making participants aware of the presence of unobservables, versus promoting
active reflection on unobservables at the points of prediction and learning
from feedback.

Figure 1. Example of the information presented to participants about a given
house. For each house, participants in all conditions were shown a set of
eight house features, together with a prediction from a model. In the Initial
+ Real-time prompts condition, shown here, unobservable features were visually
highlighted for each house shown, alongside a message indicating that the
model does not have access to this information.

##  4\. Results

To analyze participants’ behavior across conditions, we relied on two central
regression models that allowed us to understand how participants integrate
model outputs and unobservables across conditions, and whether prediction
quality varied across conditions. Let $Y$, $\hat{Y}$, and $\hat{Y}_{h}$ denote
the true selling price of a home, the model’s prediction, and the human’s
prediction, respectively. Let $Z_{bath}$, $Z_{heat}$, and $Z_{rating}$, denote
the three unobservables, corresponding to the number of full baths, type of
heating, and rating of material and finish, respectively. Finally, let
$C\in\\{C_{N},C_{I},C_{I+R}\\}$ denote the experimental condition a
participant is assigned to, corresponding to the No prompts, Initial prompt,
and Initial + Real-time prompts conditions respectively.

The first model, which we use to answer our first research question below,
captures the relationship between participants’ prediction and the model’s
prediction and unobservables, as shown in Equation (1), where $\psi_{j}$
denotes random effects for the specific house, $j$.

(1) $\displaystyle\hat{Y}_{h}\sim
1+\hat{Y}+C+Z_{bath}+Z_{heat}+Z_{rating}+[\hat{Y}+Z_{bath}+Z_{heat}+Z_{rating}]C+\psi_{j}$

The second model, used to answer our second research question, captures
whether and how the magnitude of human’s error with respect to the true
selling price varies across conditions (see Equation (2)).

(2) $\displaystyle|\hat{Y}_{h}-Y|\sim 1+C+\psi_{j}$

To support interpretation of our statistical results, we also examined the
open-text responses that participants provided at the end of the study,
capturing participants’ own reflections regarding how the model predictions
and other information about a house informed their predictions. Relevant
statistics on participants’ open-text responses are used to supplement
reporting of our main findings throughout this section, along with relevant
quotes from participants. Unless otherwise noted, all statistical results
presented correspond to data from the testing phase of the experiment.
Regression coefficients presented throughout this section are standardized.

###  4.1. RQ1: Effects of prompting on information integration

We find that presenting prompts about unobservables can change how humans
integrate model outputs and unobservables. In particular, when people have
constant reminders of unobservables (in the Initial + Real-time prompts
condition), their predictions are indeed better explained by a combination of
the unobservables and the model prediction. This can be observed when
analyzing how the Mean Squared Error (MSE) varies across conditions for
regression (1), shown in Figure 2. This measure indicates how well the
participants’ predictions $\hat{Y}_{h}$ can be explained as a regression over
the algorithmic prediction and the unobservables. Overall, the MSE is lower in
the Initial + Real-time prompts condition, reducing to almost half.

Figure 2. Mean Squared Error (MSE) of regression (1). In the Initial + Real-
time Prompts condition, participants’ predictions $\hat{Y}_{h}$ are better
explained as a regression over the model’s prediction and the unobservables.
This is driven by a change in how participants with higher prior experience
use the information presented to them.

As shown in Figure 2, this overall reduction in MSE across conditions is
entirely driven by a change in participants who reported having higher prior
experience browsing for homes (i.e., participants who reported having spent “A
lot” of time browsing homes online, or greater). When interpreting this
result, it is key to consider the difference in magnitudes of MSE between
participants with lower versus higher prior experience. Compared with other
groups, in the No prompts and Initial prompt conditions, the MSE is much
larger for participants who reported having more prior experience with house
price prediction. This indicates that in general, participants with higher
prior experience may rely less upon the model’s prediction and more upon their
own integration of features, including features that are observable to the
model, which are not included as independent variables in regression (1). In
line with this interpretation, participants with higher prior experience often
expressed low confidence in the model in their open-text responses. For
instance, a participant with higher prior experience in the Initial prompt
condition acknowledged that the model was missing information to which they
themselves had access, but used this as a justification for ignoring the
model’s predictions:

> “The model predictions did not help my own predictions at all, especially
> since they were missing information. The main things I looked at were number
> of bathrooms and the material quality.”

Similarly, a participant in the No prompts condition wrote:

> “I felt the model predictions were completely off and started to ignore them
> after the training phase - I felt as though they were making me guess
> incorrectly. I tried to look closest at the number of full and half baths
> and the year built instead.”

By contrast, in the Initial + Real-time prompts condition, the MSE of
regression 1 for participants with higher prior experience drops to be on par
with that of participants with lower prior experience, meaning that in this
condition much more of the variation in participants’ predictions can be
explained by the model predictions and unobservables. This suggests that, in
line with the goal of this intervention, continuous prompting about
unobservables leads participants with higher prior experience to rely more
heavily on a combination of the model prediction and the unobservables than
they otherwise would. Open text responses from participants with higher prior
experience in the Initial + Real-time prompts condition support this
interpretation. For instance, one participant in this condition described how
they integrated their own prior knowledge with the model predictions and
information about unobservables:

> “I started with the model predictions and assumed the model is taking an
> average stance (1 or 2 full baths, average material quality) on the unknown
> variables. I then increased or decreased the price based on those variables
> and my knowledge of how they affect home prices.”

Another participant similarly described a prediction strategy that used the
model prediction as a starting point, and then adjusted based on their
knowledge of the unobservables:

> “[I used the] model prediction as a starting base, looked at number of full
> bathrooms and the overall material rating. If the material rating was 7+ and
> more than one full bathroom I predicted price to be over the model
> prediction.”

Table 1. Results for regression (1): Relationship between participants’
predictions (dependent variable) and model predictions and unobservables
(covariates). Standard errors are shown below each coefficient estimate.
Significance levels are indicated as: ${}^{*}($p$<$0.05), ${}^{**}($p$<$0.01),
${}^{***}($p$<$0.001).

| Dependent variable: $\hat{Y}_{h}$ (human prediction)
---|---
|  Overall |  Lower prior experience |  Higher prior experience
$C_{I}$ |  0.064∗∗∗ |  0.093∗∗∗ |  -0.022
|  (0.023) |  (0.216) |  (0.065)
$C_{I+R}$ |  -0.009 |  -0.010 |  -0.059
|  (0.022) |  (0.021) |  (0.061)
$\hat{Y}$ |  0.381∗∗∗ |  0.374∗∗∗ |  0.400∗∗∗
|  (0.073) |  (0.077) |  (0.094)
$Z_{bath}$ |  0.245∗∗∗ |  0.251∗∗ |  0.230∗∗
|  (0.053) |  (0.056) |  (0.068)
$Z_{heat}$ |  -0.067 |  -0.081 |  -0.027
|  (0.056) |  (0.060) |  (0.073)
$Z_{rating}$ |  0.223∗∗ |  0.217∗ |  0.241∗∗
|  (0.065) |  (0.069) |  (0.084)
$C_{I}*\hat{Y}$ |  -0.058 |  -0.024 |  -0.170
|  (0.048) |  (0.045) |  (0.135)
$C_{I+R}*\hat{Y}$ |  -0.047 |  -0.050 |  -0.043
|  (0.046) |  (-0.044) |  (0.125)
$C_{I}*Z_{bath}$ |  0.038 |  0.053 |  -0.022
|  (0.035) |  (0.032) |  (0.098)
$C_{I+R}*Z_{bath}$ |  0.007 |  -0.012 |  0.055
|  (0.034) |  (0.032) |  (0.091)
$C_{I}*Z_{heat}$ |  0.021 |  0.022 |  0.025
|  (0.037) |  (0.035) |  (0.105)
$C_{I+R}*Z_{heat}$ |  0.035 |  0.055 |  -0.018
|  (0.036) |  (0.035) |  (0.098)
$C_{I}*Z_{rating}$ |  0.033 |  0.005 |  0.133
|  (0.043) |  (0.035) |  (0.012)
$C_{I+R}*Z_{rating}$ |  0.073 |  0.093∗ |  0.019
|  (0.042) |  (0.035) |  (0.112)
Observations |  7,968 |  5,904 |  2,064

In Table 1, we also observe a significant interaction between the Initial +
Real-time Prompts condition and the most informative unobservable for
participants with lower prior experience. This suggests that continuous
prompting about unobservables leads participants with lower prior experience
to make greater use of unobservables when making their predictions. We did not
observe comparable effects of presenting a single, initial prompt about
unobservables at the beginning of the training phase, in the Initial Prompt
condition.

Despite these differences across conditions in the way participants integrated
the information presented to them, Table 1 shows that participants across all
conditions and levels of experience learned to make use of two of the
unobservable features: the rating of material and finish and the number of
bathrooms. In line with these results, we observed that across all conditions,
open-text responses from 80% of participants explicitly referenced taking into
account one or more of the unobservable features. Participants’ open-text
responses indicated that some participants in our control condition learned to
leverage unobservables in order to adjust for limitations of the model’s
prediction. For example, a participant in the No prompts condition wrote:

> “I noticed the model was occasionally spot-on but also very far off
> sometimes. So I used it as sort of a middle ground. The number of full
> bathrooms and year built were the main factors I used to determine the
> home’s value.”

Another participant in the No prompts condition similarly stated that: “If the
[ratings] were above 6, I would predict a higher price than the model
prediction and vice versa. I also looked at the year the house was built and
used that with the other information to determine a price.” Across conditions,
62% of participants explicitly stated that they took the number of bathrooms
into account when making their predictions, 51% said they accounted for the
rating of material and finish, and 4% said they took the type of heating into
account.

In interpreting this set of results, it is important to note that participants
in our study were presented with a relatively small set of eight features. Had
participants been presented with a significantly larger number of total
features, making it more difficult and time-consuming to manually inspect all
of them, it is possible that participants would have been less likely to pick
up on the importance of these unobservables without the type of explicit
prompting provided in our experimental conditions. This, combined with the
presence of feedback during the training phase, is likely to have played an
important role (Lejarraga and Hertwig, 2021). The interaction of feedback and
prompts may have helped participants calibrate their use of unobservables. For
participants with lower prior experience, we observe a significant interaction
between the unobservable with low variation and minimal complementary
predictive power (type of heating), and the Initial + Real-time Prompts and
Initial conditions during training (see Appendix). This suggests that
presenting prompts stating that the model does not observe this information
nudges participants to try to make use of it, if they lack prior experience
with house price prediction that may guide them to act otherwise. However,
this effect disappears in the test phase, presumably after they have learned
from feedback that this feature is not useful. Notably, across conditions,
participants with higher prior experience quickly honed in on using the model
value and the two most informative unobservables, relying primarily on these
features to inform their predictions (see Appendix).

###  4.2. RQ2: Effects of prompting on human predictive accuracy

Overall, we found that participants were able to successfully integrate across
the information presented in order to make more accurate predictions compared
with the model in isolation. Across all conditions, participants’ predictions
were $17,840.84 USD closer to a house’s true sales price than the model’s
predictions on average (p¡0.001). Broken down by condition, this gap in mean
absolute error was $17,465.84 in the No Prompts condition, $15,559.71 in the
Initial prompt condition, and $20,225.97 in the Initial + Real-time prompts
condition.

However, as shown in Table 2, we did not observe significant improvements in
human predictive accuracy over the No prompts condition in either the Initial
prompt condition or the Initial + Real-time Prompts condition. Although
continuous prompting about unobservables did affect how participants
integrated the information presented to them, this did not translate into
better overall predictive performance. It may be that we do not observe
significant improvements in predictive performance across conditions because,
as discussed in Section 4.1, participants across all conditions and levels of
experience learned to make use of the unobservables in our study. As we will
discuss in the next section, participants may be less able to do so without
the aid of prompts when presented with a larger total number of features.
Future work is needed to explore whether the impacts we observe on
participants’ information integration may translate to improved predictive
performance in other contexts.

Interestingly, we observe that the Initial prompt condition led to an increase
in prediction error among participants who had lower prior experience on the
prediction task. We interpret this finding in light of our prior observations
that (1) this group of participants made greater use of unobservables when
making predictions in the Initial + Real-time prompts condition but not in the
Initial prompt condition (Section 4.1), and (2) the Initial prompt condition
had a significant overall impact on the magnitude of participants’ predictions
among those with lower prior experience, but not those with higher prior
experience (see Table 1, Row 1). Taken together, these findings suggest that
simply increasing participants’ awareness of the presence of unobservables
during training may lead those with less experience astray, influencing them
to adjust their predictions in ways that harm their predictive accuracy
compared with those in other conditions. By contrast, continuous reminders
regarding which specific features are unobservable to the model appear to
mitigate this effect, in the Initial + Real-time prompts condition.

Table 2. Results for regression (2): relationship between participants’
absolute prediction error (dependent variable) and experimental condition
(covariates). Standard errors are shown below each coefficient estimate.
Significance levels are indicated as: ${}^{*}($p$<$0.05), ${}^{**}($p$<$0.01),
${}^{***}($p$<$0.001).

| Dependent variable: $|\hat{Y}_{h}-Y|$
---|---
|  Overall |  Lower prior experience |  Higher prior experience
$C_{I}$ |  0.017 |  0.035∗ |  -0.028
|  (0.020) |  (0.018) |  (0.058)
$C_{I+R}$ |  -0.030 |  -0.009 |  -0.087
|  (0.019) |  (0.018) |  (0.054)
Observations |  7,968 |  5,904 |  2,064

##  5\. Discussion and Future Work

In this work, we have conducted an online experiment to study the impact of
interventions that prompt people to reflect on complementary sources of
information available to themselves versus an AI model. We found that
presenting prompts about information that is unobservable to the model can
change how humans integrate integrate different sources of information when
making predictions. Even in the absence of improvements in predictive
performance, it is noteworthy that such prompts can have an impact on human
information integration. Given the prevalence of unobservables in real-world
AI-assisted decision-making settings, more work is needed to investigate what
other forms of prompting may be helpful and in which settings.

Moreover, a change in information integration may be of practical relevance
even if it does not translate into better overall predictive performance. In
the machine learning literature, researchers have studied the phenomenon of
predictive multiplicity: multiple different models (or human-model teams) may
yield the same overall performance (Marx et al., 2020), while having very
different fairness properties and underperforming for different subsets of
cases (Chouldechova and G’Sell, 2017). Thus, given our finding that prompts
about unobservables can shift how humans integrate information, future work
should investigate the impacts of such prompts on relevant metrics beyond
predictive accuracy. For instance, future work may consider the fairness
implications of prompts that invite reflection on complementary information.
From a fairness perspective, it is worth noting that a shift in the grounds of
decision-making may also have relevance from a procedural justice perspective,
insofar as it may ensure the consideration of important sources of
information.

In our experiment, these changes in the ways participants integrate
information when receiving prompts about unobservables did not translate into
an overall improvement in human predictive performance. When interpreting
these results, it is important to consider several characteristics of our
study. First, the number of features available to humans in this initial study
was relatively small (eight features), which means that it was possible for
them to scan through all of the available information in a short period of
time. Yet in many real-world settings where AI-based decision support tools
are used, the total number of model-observable features and relevant
unobservables is significantly larger. For example, call workers tasked with
screening child maltreatment calls have hundreds of features available to them
(Chouldechova et al., 2018; Kawakami et al., 2022b). In settings where a
larger number of features are present, it is possible that highlighting the
pieces of information that are not available to an AI model would have a
greater effect in improving their predictive performance. Furthermore, our
experiment included a training phase in which participants were provided with
immediate feedback on the accuracy of their own predictions and the model’s
predictions (cf. (Poursabzi-Sangdeh et al., 2021)). Combined with the small
number of features present, this may have facilitated participants’ learning
of what information held complementary power (Lejarraga and Hertwig, 2021). In
domains where decision-makers do not receive immediate feedback (Holstein et
al., 2019; Kawakami et al., 2022a), prompts emphasizing unobservables may have
a greater effect in enhancing complementarity. Further research is needed to
investigate how the number of features present and the availability of such
granular feedback may mediate the impacts of interface prompts about
unobservables.

The tension between facilitating better information integration and inducing
algorithm aversion (Dietvorst et al., 2015) is one that merits further
attention. As discussed in Section 4.2, the Initial Prompt condition led to an
increase in prediction error among participants who had lower prior experience
with this task. This may indicate that the prompt served to reduce
participants’ trust in the model, rather than guiding them on how to use the
model more effectively. Thus, designers of interventions that communicate
models’ limitations, in an effort to help decision-makers calibrate their
reliance on these models, must pay close attention to the risk of
“overshooting” with these interventions and thus inducing algorithm aversion.
It is worth noting that in our experiment, the apparent induction of algorithm
aversion observed in the Initial Prompt condition was mitigated when
participants were continuously reminded about which specific features were
unobservable to the model in the Initial + Real-time Prompts condition. It is
possible that continuous prompting helped participants to scope their
skepticism about model predictions appropriately, so that they were still able
to benefit from the model’s predictive strengths.

Finally, we note that a nuanced view of different types of unobservables is
fundamental to the design of sociotechnical systems that aim to facilitate
human-AI complementarity. In Section 3.4 we differentiate between different
types of unobservables, highlighting both (1) their complementary predictive
power and (2) the possibilities they present to support humans in learning how
to leverage them alongside model predictions, based on feedback. The risk of
“redundancy” of features that are individually predictive but do not
complement the information already captured by the model is particularly
interesting to consider. Although a feature may be highly predictive of an
outcome of interest in isolation, accounting for this feature may not
meaningfully improve predictive power, due to correlations with the observable
features that the model already takes into account. This points to a risk that
has received little attention in the literature on AI-assisted decision-making
to date: human decision-makers may be at risk of “double counting” when
presented with features that are evidently informative in isolation but whose
signal is already accounted for by the model via correlations. Interestingly,
we do not observe instances of such double counting in our experiments.
However, this may be because the feature that has this property in our study
is also one that exhibits relatively little variation within the set of
instances observed by participants—thereby making it easier for participants
to learn from feedback that this feature is of little relevance. Thus, we
believe the phenomenon of double counting in AI-assisted decision-making
merits further research. We note that the challenge of redundant unobservables
can be particularly difficult to tackle from a design perspective: when a
feature is an unobservable, this is often because it is not encoded in
existing datasets. This means that it is not possible to conduct a data-driven
diagnosis to determine whether its predictive power is complementary with that
of the observed features. Thus, in the absence of strong theoretical or expert
domain knowledge, it is difficult to know whether prompting about a given
unobservable could risk misleading participants into “double counting” if the
feature is already _indirectly_ accounted for.

Our work has direct implications for cases in which it is easy to characterize
unobservables and communicate about them. For example, in the AI-assisted
child welfare context, call workers have structured ways of identifying
certain key pieces of information communicated in calls made to child
maltreatment hotlines. Research suggests that in this setting there is
significant heterogeneity in call workers’ awareness that the information
communicated in the call is not visible to the algorithm (Kawakami et al.,
2022a). The findings we present suggest that such heterogeneity likely impacts
how they integrate ADS recommendations into their decisions, and prompts that
bring awareness and reflection around unobservables could help to reduce this
variation. However, as discussed above, in various real-world setting,
unobservables may be challenging to precisely characterize and externalize
without losing important, decision-relevant context and nuance. This is often
the case, for example, in healthcare settings where physicians consider
patients’ presentation and emotional state, or in education settings where
teachers consider how a student’s current life circumstances may impact their
academic performance. As a simplifying assumption in this study, we presented
participants with explicit values for each unobservable in a tabular format,
mirroring the design of prior studies that have adopted this AI-assisted
housing prediction task (e.g., (Chiang and Yin, 2021; Poursabzi-Sangdeh et
al., 2021)). However, further research is needed to better understand the
impacts of different forms of prompting in settings where unobservables are
not externalized in structured data, and exist only as perceptions in human
decision-makers’ minds.

Importantly, it is not necessary for humans to be able to explicitly
characterize and externalize unobservables in order to incorporate them into
their decision-making. A vast body of research in psychology and human-
computer interaction shows that human experts exhibit an impressive ability to
integrate rich implicit inferences about the world into their decision-making
(Hemmer et al., 2022; Kawakami et al., 2022a; Lake et al., 2017; Rastogi et
al., 2022). In fact, encouraging human decision-makers to externalize such
complex inferences (e.g., by attempting to explicitly write down the criteria
they are using to inform their decisions) can sometimes backfire by driving
them towards the use of easier-to-communicate, yet less reliable criteria
(Koedinger et al., 2012; Lake et al., 2017; Nosofsky et al., 2005). However,
it is worth noting that prompting humans to attend to AI unobservables could
be a double-edged sword, and further research is needed to understand risks
and benefits across a broader range of decision-making tasks and contexts. In
practice, human decision-makers’ ability to reason about certain unobservables
may vary across cases. For instance, a physician’s ability to accurately
interpret a patient’s emotional state may depend on whether they have a shared
sociocultural background. In such settings, there is a risk that bringing
awareness to such unobservables could exacerbate existing undesirable biases
in human decision-making. Thus, future work should investigate the effects of
prompting about unobservables in sensitive context, and special attention
should be devoted to heterogeneity of these effects across different decision-
makers.

In sum, we have found that presenting interface prompts to support human
decision-makers in reflecting upon information asymmetries between themselves
and an AI model can measurably change how they integrate model outputs with
model-unobservable features. However, the impacts of these prompts on human
information integration and predictive performance are significantly more
complex than we had anticipated at the outset of this project. We now
hypothesize that—in addition to being influenced by decision-makers’ prior
task experience and the frequency of prompting—the impacts of such prompts may
also be influenced by (1) the total number of features presented to human
decision-makers, (2) the types of unobservables that are included within
prompts (e.g., the extent to which these unobservables truly complement the
information presented by the model), and (3) the affordances available in a
given context for humans to learn how to integrate unobservables with model
outputs (e.g., whether humans have the opportunity to learn via practice with
immediate feedback (Lejarraga and Hertwig, 2021)). Future research exploring
each of these hypotheses is needed in order to understand how we can help
human decision-makers better leverage complementary perceptual abilities in
the context of human-AI collaboration. It is our hope that these findings will
inform further research and design toward the design of tools that can bring
out the best of both human and AI abilities.

###### Acknowledgements.

This work was supported by an award from the UL Research Institutes through
the Center for Advancing Safety of Machine Intelligence (CASMI) at
Northwestern University, by the Carnegie Mellon University Block Center for
Technology and Society (Award No. 53680.1.5007718), and by Good Systems, a UT
Austin Grand Challenge to develop responsible AI technologies.

## References

  * (1)
  * Albright (2019) Alex Albright. 2019\.  If you give a judge a risk score: Evidence from Kentucky bail decisions.  _Harvard John M. Olin Fellow’s Discussion Paper_ 85 (2019), 16.
  * An et al. (2020) Pengcheng An, Kenneth Holstein, Bernice d’Anjou, Berry Eggen, and Saskia Bakker. 2020.  The TA Framework: Designing real-time teaching augmentation for K-12 classrooms. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–17.
  * Bansal et al. (2021) Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021\.  Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–16.
  * Buçinca et al. (2021) Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021.  To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making.  _Proceedings of the ACM on Human-Computer Interaction_ 5, CSCW1 (2021), 1–21.
  * Cheng et al. (2022) Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, and Haiyi Zhu. 2022\.  How child welfare workers reduce racial disparities in algorithmic decisions. In _CHI Conference on Human Factors in Computing Systems_. 1–22.
  * Chiang and Yin (2021) Chun-Wei Chiang and Ming Yin. 2021.  You’d better stop! Understanding human reliance on machine learning models under covariate shift. In _13th ACM Web Science Conference 2021_. 120–129.
  * Chine et al. (2022) Danielle R Chine, Cassandra Brentley, Carmen Thomas-Browne, J Elizabeth Richey, Abdulmenaf Gul, Paulo F Carvalho, L Branstetter, and KR Koedinger. 2022.  Educational equity through combined human-AI personalization: A propensity matching evaluation. In _International Conference on Artificial Intelligence in Education. Springer, Cham_.
  * Chouldechova et al. (2018) Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018\.  A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In _Conference on Fairness, Accountability and Transparency_. PMLR, 134–148.
  * Chouldechova and G’Sell (2017) Alexandra Chouldechova and Max G’Sell. 2017.  Fairer and more accurate, but for whom?  _arXiv preprint arXiv:1707.00046_ (2017).
  * De-Arteaga et al. (2019) Maria De-Arteaga, Jieshi Chen, Peter Huggins, Jonathan Elmer, Gilles Clermont, and Artur Dubrawski. 2019\.  Predicting neurological recovery with canonical autocorrelation embeddings.  _PloS one_ 14, 1 (2019), e0210966.
  * De-Arteaga et al. (2021) Maria De-Arteaga, Artur Dubrawski, and Alexandra Chouldechova. 2021\.  Leveraging expert consistency to improve algorithmic decision support.  _arXiv preprint arXiv:2101.09648_ (2021).
  * De-Arteaga et al. (2020) Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020\.  A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–12.
  * De Cock (2011) Dean De Cock. 2011\.  Ames, Iowa: Alternative to the Boston housing data as an end of semester regression project.  _Journal of Statistics Education_ 19, 3 (2011).
  * Dickler et al. (2021) Rachel Dickler, Janice Gobert, and Michael Sao Pedro. 2021\.  Using innovative methods to explore the potential of an alerting dashboard for science inquiry.  _Journal of Learning Analytics_ 8, 2 (2021), 105–122.
  * Dietvorst et al. (2015) Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015\.  Algorithm aversion: people erroneously avoid algorithms after seeing them err.  _Journal of Experimental Psychology: General_ 144, 1 (2015), 114\.
  * Fan et al. (2018) Chenchen Fan, Zechen Cui, and Xiaofeng Zhong. 2018.  House prices prediction with machine learning algorithms. In _Proceedings of the 2018 10th International Conference on Machine Learning and Computing_. 6–10.
  * Fogliato et al. (2021) Riccardo Fogliato, Alexandra Chouldechova, and Zachary Lipton. 2021.  The impact of algorithmic risk assessments on human predictions and its analysis via crowdsourcing studies.  _Proceedings of the ACM on Human-Computer Interaction_ 5, CSCW2 (2021), 1–24.
  * Fogliato et al. (2022) Riccardo Fogliato, Maria De-Arteaga, and Alexandra Chouldechova. 2022\.  A case for humans-in-the-loop: Decisions in the presence of misestimated algorithmic scores.  _Available at SSRN 4050125_ (2022).
  * Gao et al. (2021) Ruijiang Gao, Maytal Saar-Tsechansky, Maria De-Arteaga, Ligong Han, Min Kyung Lee, and Matthew Lease. 2021.  Human-AI Collaboration with Bandit Feedback.  _arXiv preprint arXiv:2105.10614_ (2021).
  * Green and Chen (2019a) Ben Green and Yiling Chen. 2019a.  Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. In _Proceedings of the conference on fairness, accountability, and transparency_. 90–99.
  * Green and Chen (2019b) Ben Green and Yiling Chen. 2019b.  The principles and limits of algorithm-in-the-loop decision making.  _Proceedings of the ACM on Human-Computer Interaction_ 3, CSCW (2019), 1–24.
  * Hemmer et al. (2022) Patrick Hemmer, Max Schemmer, Niklas Kühl, Michael Vössing, and Gerhard Satzger. 2022\.  On the Effect of Information Asymmetry in Human-AI Teams.  _arXiv preprint arXiv:2205.01467_ (2022).
  * Holstein and Aleven (2022) Kenneth Holstein and Vincent Aleven. 2022.  Designing for human–AI complementarity in K-12 education.  _AI Magazine_ 43, 2 (2022), 239–248.
  * Holstein et al. (2020) Kenneth Holstein, Vincent Aleven, and Nikol Rummel. 2020\.  A conceptual framework for human–AI hybrid adaptivity in education. In _International conference on Artificial Intelligence in Education_. Springer, 240–254.
  * Holstein et al. (2019) Kenneth Holstein, Bruce M McLaren, and Vincent Aleven. 2019\.  Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity.  _Journal of Learning Analytics_ 6, 2 (2019).
  * Holten Møller et al. (2020) Naja Holten Møller, Irina Shklovski, and Thomas T Hildebrandt. 2020.  Shifting concepts of value: Designing algorithmic decision-support systems for public services. In _Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society_. 1–12.
  * Karlinsky-Shichor and Netzer (2019) Yael Karlinsky-Shichor and Oded Netzer. 2019.  Automating the B2B salesperson pricing decisions: Can machines replace humans and when.  _Available at SSRN_ 3368402 (2019).
  * Kawakami et al. (2022a) Anna Kawakami, Venkatesh Sivaraman, Hao-Fei Cheng, Logan Stapleton, Yanghuidi Cheng, Diana Qing, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, and Kenneth Holstein. 2022a.  Improving human-AI partnerships in child welfare: Understanding worker practices, challenges, and desires for algorithmic decision support. In _CHI Conference on Human Factors in Computing Systems_. 1–18.
  * Kawakami et al. (2022b) Anna Kawakami, Venkatesh Sivaraman, Logan Stapleton, Hao-Fei Cheng, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, and Kenneth Holstein. 2022b.  “Why do I care what’s similar?” Probing challenges in AI-assisted child welfare decision-making through worker-AI interface design concepts. In _Designing Interactive Systems Conference_. 454–470.
  * Kleinberg et al. (2018) Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018\.  Human decisions and machine predictions.  _The quarterly journal of economics_ 133, 1 (2018), 237–293.
  * Koedinger et al. (2012) Kenneth R Koedinger, Albert T Corbett, and Charles Perfetti. 2012.  The Knowledge-Learning-Instruction framework: Bridging the science-practice chasm to enhance robust student learning.  _Cognitive science_ 36, 5 (2012), 757–798.
  * Lake et al. (2017) Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. 2017.  Building machines that learn and think like people.  _Behavioral and brain sciences_ 40 (2017).
  * Lakkaraju et al. (2017) Himabindu Lakkaraju, Jon Kleinberg, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2017\.  The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables. In _Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 275–284.
  * Lee and See (2004) John D Lee and Katrina A See. 2004.  Trust in automation: Designing for appropriate reliance.  _Human Factors_ 46, 1 (2004), 50–80.
  * Lee et al. (2021) Min Hun Lee, Daniel P Siewiorek, Asim Smailagic, Alexandre Bernardino, and Sergi Bermúdez i Badia. 2021\.  A human-AI collaborative approach for clinical decision making on rehabilitation assessment. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Lejarraga and Hertwig (2021) Tomás Lejarraga and Ralph Hertwig. 2021.  How experimental methods shaped views on human competence and rationality.  _Psychological Bulletin_ 147, 6 (2021), 535\.
  * Levy et al. (2021) Karen Levy, Kyla Chasalow, and Sarah Riley. 2021\.  Algorithms and decision-making in the public sector.  _arXiv preprint arXiv:2106.03673_ (2021).
  * Lurie and Mulligan (2020) Emma Lurie and Deirdre K Mulligan. 2020.  Crowdworkers are not judges: Rethinking crowdsourced vignette studies as a risk assessment evaluation technique. In _Proceedings of the Workshop on Fair and Responsible AI at CHI_.
  * Madras et al. (2018) David Madras, Toni Pitassi, and Richard Zemel. 2018\.  Predict responsibly: improving fairness and accuracy by learning to defer.  _NeurIPS_ 31 (2018), 6147–6157.
  * Marx et al. (2020) Charles Marx, Flavio Calmon, and Berk Ustun. 2020\.  Predictive multiplicity in classification. In _International Conference on Machine Learning_. PMLR, 6765–6774.
  * Nosofsky et al. (2005) Robert M Nosofsky, Roger D Stanton, and Safa R Zaki. 2005\.  Procedural interference in perceptual classification: Implicit learning or cognitive complexity?  _Memory & Cognition_ 33, 7 (2005), 1256–1271.
  * Patel et al. (2019) Bhavik N Patel, Louis Rosenberg, Gregg Willcox, David Baltaxe, Mimi Lyons, Jeremy Irvin, Pranav Rajpurkar, Timothy Amrhein, Rajan Gupta, Safwan Halabi, et al. 2019\.  Human–machine partnership with artificial intelligence for chest radiograph diagnosis.  _NPJ digital medicine_ 2, 1 (2019), 1–10.
  * Poursabzi-Sangdeh et al. (2021) Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021.  Manipulating and measuring model interpretability. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–52.
  * Rastogi et al. (2022) Charvi Rastogi, Liu Leqi, Kenneth Holstein, and Hoda Heidari. 2022\.  A Unifying Framework for Combining Complementary Strengths of Humans and ML toward Better Predictive Decision-Making.  _arXiv preprint arXiv:2204.10806_ (2022).
  * Ritter et al. (2016) Steven Ritter, Michael Yudelson, Stephen Fancsali, and Susan R Berman. 2016.  Towards integrating human and automated tutoring systems.. In _EDM_. Citeseer, 626–627.
  * Saxena et al. (2020) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020.  A human-centered review of algorithms used within the US child welfare system. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. 1–15.
  * Stevenson and Doleac (2021) Megan T Stevenson and Jennifer L Doleac. 2021.  Algorithmic risk assessment in the hands of humans.  _Available at SSRN 3489440_ (2021).
  * Tan et al. (2018) Sarah Tan, Julius Adebayo, Kori Inkpen, and Ece Kamar. 2018\.  Investigating human+ machine complementarity for recidivism predictions.  _arXiv preprint arXiv:1808.09123_ (2018).
  * Wang et al. (2021) Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, and Feng Tian. 2021\.  “Brilliant AI doctor” in rural clinics: Challenges in AI-powered clinical decision support system deployment. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–18.
  * Wansbeek and Meijer (2001) Tom Wansbeek and Erik Meijer. 2001.  Measurement error and latent variables.  _A companion to theoretical econometrics_ (2001), 162–179.
  * Wilder et al. (2020) Bryan Wilder, Eric Horvitz, and Ece Kamar. 2020\.  Learning to Complement Humans.  _arXiv_ (2020).
  * Yang et al. (2019) Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019\.  Unremarkable AI: Fitting intelligent decision support into critical, clinical decision-making processes. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–11.
  * Yang et al. (2016) Qian Yang, John Zimmerman, Aaron Steinfeld, Lisa Carey, and James F Antaki. 2016.  Investigating the heart pump implant decision process: opportunities for decision support tools to help. In _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_. 4477–4488.

##  Appendix A Appendix

Table 3. Results for regression (1) on participant data from the training
phase. Standard errors are shown below each coefficient estimate. Significance
levels are indicated as: ${}^{*}($p$<$0.05), ${}^{**}($p$<$0.01),
${}^{***}($p$<$0.001).

| Dependent variable: $\hat{Y}_{h}$ (human prediction)
---|---
|  Overall |  Lower prior experience |  Higher prior experience
$C_{I}$ |  0.013 |  0.001 |  0.045
|  (0.025) |  (0.029) |  (0.048)
$C_{I+R}$ |  0.001 |  0.000 |  0.003
|  (0.024) |  (0.029) |  (0.044)
$\hat{Y}$ |  0.287∗∗∗ |  0.264∗∗∗ |  0.351∗∗∗
|  (0.033) |  (0.039) |  (0.040)
$Z_{bath}$ |  0.284∗∗∗ |  0.32∗∗∗ |  0.182∗∗∗
|  (0.041) |  (0.049) |  (0.049)
$Z_{heat}$ |  0.120∗∗ |  0.146∗∗ |  0.049
|  (0.036) |  (0.043) |  (0.044)
$Z_{rating}$ |  0.159∗∗∗ |  0.145∗∗ |  0.195∗∗∗
|  (0.032) |  (0.038) |  (0.040)
$C_{I}*\hat{Y}$ |  0.038 |  0.064 |  0.045
|  (0.029) |  (0.034) |  (0.048)
$C_{I+R}*\hat{Y}$ |  0.042 |  0.078∗ |  0.003
|  (0.029) |  (0.034) |  (0.044)
$C_{I}*Z_{bath}$ |  0.005 |  -0.034 |  0.113
|  (0.036) |  (0.043) |  (0.070)
$C_{I+R}*Z_{bath}$ |  -0.009 |  -0.044 |  0.086
|  (0.035) |  (0.042) |  (0.065)
$C_{I}*Z_{heat}$ |  -0.065∗ |  -0.093∗ |  0.011
|  (0.032) |  (0.038) |  (0.062)
$C_{I+R}*Z_{heat}$ |  -0.068∗ |  -0.102∗∗ |  0.024
|  (0.031) |  (0.037) |  (0.057)
$C_{I}*Z_{rating}$ |  0.010 |  0.027 |  -0.04
|  (0.029) |  (0.034) |  (0.055)
$C_{I+R}*Z_{rating}$ |  -0.048 |  0.054 |  0.032
|  (0.028) |  (0.033) |  (0.051)
Observations |  7,968 |  5,904 |  2,064