# Developing Situational Awareness for Joint Action with Autonomous Vehicles

Robert Kaufman<EMAIL_ADDRESS>University of California, San Diego9500
Gilman DrLa JollaCaliforniaUSA92093 ,  David Kirsh<EMAIL_ADDRESS>University
of California, San DiegoLa JollaCaliforniaUSA and  Nadir Weibel
<EMAIL_ADDRESS>University of California, San DiegoLa JollaCaliforniaUSA

(2024)

###### Abstract.

Unanswered questions about how human-AV interaction designers can support
rider’s informational needs hinders Autonomous Vehicles (AV) adoption. To
achieve joint human-AV action goals - such as safe transportation, trust, or
learning from an AV - sufficient situational awareness must be held by the
human, AV, and human-AV system collectively. We present a systems-level
framework that integrates cognitive theories of joint action and situational
awareness as a means to tailor communications that meet the criteria necessary
for goal success. This framework is based on four components of the shared
situation: AV traits, action goals, subject-specific traits and states, and
the situated driving context. AV communications should be tailored to these
factors and be sensitive when they change. This framework can be useful for
understanding individual, shared, and distributed human-AV situational
awareness and designing for future AV communications that meet the
informational needs and goals of diverse groups and in diverse driving
contexts.

Autonomous vehicles, Situational awareness, Joint action

††copyright: acmlicensed††journalyear: 2024††ccs: Human-centered computing HCI
theory, concepts and models

##  1\. Introduction

Advances in Autonomous Vehicle (AV) technology promise huge individual and
societal benefits, from increased driving safety to reduced environmental
impact (Fagnant and Kockelman, 2015). However, unanswered questions about how
AV interaction design can support the critical informational needs of people
riding with AVs are hindering real-world adoption (Hegner et al., 2019; Detjen
et al., 2021) A well documented example is that people often report a lack of
trust in AV decision-making, particularly when the AV’s decision procedures
are opaque and not human-understandable (Kohn et al., 2021; Meyer-Waarden and
Cloarec, 2022; Morra et al., 2019; Ekman et al., 2017; Frison et al., 2019;
Wiegand et al., 2019).

One reason we think that the success of AV technologies will likely depend on
accurate designing for the social informational needs of people, is because we
know that theory of mind is a core part of how we think and move about the
world, particularly when we are trying to understand decisions made by other
actors (Cuzzolin et al., 2020; Premack and Woodruff, 1978; Wang et al., 2021).
A person seeking to make decisions within a human-AV system necessarily
confronts many questions about AV decision-making. Most pressingly for the
decisions of the person, questions involve both present and future actions on
the road and their predictability: Why is the AV moving to the left? Does the
AV see the pedestrian ahead? What will it do next? Will this always be the
case?

Concern over the effectiveness of human-AI communication may be exacerbated in
safety-critical situations (Stanton et al., 2001; Liu and Qi, 2023) or when a
person may have additional interaction goals like learning from the AI’s
decisions (Ghai et al., 2021; Soltani et al., 2022; Kaufman et al., 2024). AVs
of the (not so distant) future may play an expanded role in facilitating more
than just safe transportation: managing social, emotional, educational, and
cultural experiences for their passengers. For example, an AV could take the
scenic route through a city, comment on notable historical centers, and adjust
its communication style to avoid waking a sleepy travel companion. AVs could
teach driving skills, soothe nervous passengers, and entertain children on a
road trip. There are near-endless opportunities. In this paper we provide an
illustrative example of what future human-AV interaction may look like, and
then outline how it is likely that there will be multiple AI systems working
together: some managing the task of driving and analyzing the outside
environment, while others will focus on detecting and moderating passenger
experiences. Without addressing the salient information needs of passengers,
however, very few AV functions can be fully realized.

Autonomous Vehicles and Situational Awareness – A theory which has helped
researchers understand many of the informational problems that stem from
interacting with an AV is through the lens of situational awareness (SA),
where people want to be able to perceive, comprehend, and project into the
future what is happening in a given driving scenario (Endsley, 1995). A lack
of knowledge in how an AV will behave or why it does so, prevents a person
interacting with the AV from achieving situational awareness. This is because
they do not have a means to access the information required to develop
understanding or predict future behaviors that the AV might take (Capallera et
al., 2022).

Lacking situational awareness:

  * •

Impedes a person from being able to develop trust, a reliance strategy, or
expertise from learning, limiting the usefulness of the technology (Vorm and
Combs, 2022; Marangunić and Granić, 2015).

  * •

Removes a person’s agency, forcing them to rely on blind faith and abandon
their desire for understanding (Tulli and Aha, 2024).

However, achieving necessary situational awareness in human-AV partnership
requires bi-directional communication (Chen et al., 2018). Some situational
awareness needs to be held by the human and AV independently, while other SA
must be shared or distributed between them (Salas et al., 1995; Stanton et
al., 2006). Numerous questions remain on how communication needs should be
satisfied and how to account for the complex socio-technical system within
which the human and AV interact (Stanton et al., 2006).

There have been many models for building situational awareness to support
human-AV interaction. These tend to focus on either the needs of the human(s)
in the system or the AV in the system independently of one another. Models for
the AV’s situational awareness are often fairly technical in nature, aiming to
computationally capture what is happening in an external driving situation so
that an autonomous vehicle can maneuver safely to a destination (Reich and
Trapp, 2020). Models aimed at building human situational awareness tend to
focus on providing justification for an AV’s decisions. These range from
human-machine interfaces (HMIs) that directly apply Endley’s SA theory for
design of in-car communication design (Capallera et al., 2022) to those
encompassing aspects of working memory and comprehension (Baumann and Krems,
2007; Krems and Baumann, 2009). There is also a large body of work on how best
to measure SA in vehicles (Endsley, 1988; Rangesh et al., 2018; Sirkin et al.,
2017; Ma and Kaber, 2007). Though useful and informative in their own right,
prior work fails to fully account for the joint, system-situated nature of
achieving human-AV interaction goals (Kridalukmana et al., 2020). They also
fail to show how SA requirements – including what needs to be held
individually, shared, or distributed by the humans and AV in the system – may
change when aspects of the communicative context changes.

In this paper we introduce an integrated framework that illustrates how a
theory of joint action (Sebanz et al., 2006) can be used by human-AV
interaction designers to understand four foundational factors that affect
situational awareness during the moment of human-AV interaction: (1) joint
action goals, (2) AV traits, (3) subject-specific traits and states, and (4)
driving context characteristics. We posit that by understanding how these
factors form an interdependent system, communications can be designed that
meet the informational needs demanded by a particular communicative context.
To our knowledge, no prior models or frameworks have examined human-AV
communication from the lens of these cognitive and behavioral phenomena. Our
framework demonstrates how the perspective of situated human-AV joint action
can be used to inform the methods by which we seek to understand SA
requirements.

The systems view of human-AV joint action and situational awareness presented
here can be used to inform a future research agenda for human-AV interaction
research and design. Applying this framework reveals several pressing
knowledge gaps which, if filled, can be used to design situationally-aware
human-AV systems. We provide specific calls to action for future research.
Though framed within the human-autonomous driving domain, many of the
knowledge gaps identified likely apply to other human-AI interaction domains,
such as those in healthcare or military applications.

The remainder of this paper is structured as follows. First, we introduce the
context of our investigation through an illustrative example (Section 2).
Next, we present an overview of our novel framework, which can be used to
understand the development of situational awareness supporting situated joint
action within Human-AV systems (Section 3). In the discussion, we
contextualize this framework within existing work and cognitive theory,
grounding the framework in examples of driving interaction and providing open
areas for future work (Section 4); the discussion is broken into three parts:
human-AI teaming to achieve action goals (Section 4.1), the development
situational awareness (Section 4.2), and wider system aspects and
communication needs (Section 4.3). We conclude the paper with an overall
description of how this framework can add to the scientific community’s
understanding of how to design communications meeting situational awareness
needs (Section 5) and a system summary (Section 6).

##  2\. Illustrative Example

This section contains a fictional story about a small family (mother and son)
riding in an AV. Through this example, we illustrate the factors that need to
be accounted for when designing communications supporting human-AV interaction
goals. This example will be referenced throughout this paper.

A young mother is riding with her 15 year old son in the back seat. They are
on vacation and have summoned an AV taxi to help them get from the airport to
their hotel. The action goals of the mother are fairly simple: get to the
hotel safely. Her son has an additional goal– he is learning to drive, and is
interested in understanding how the vehicle maneuvers busy city streets. He
loves new technologies and is far more willing to take risks than his mother.
Both family members need to rely on the AV to achieve their goals. Trust is a
precondition that needs to be addressed to achieve appropriate reliance, which
is in itself necessary for safe transportation and learning. The mother and
son, their social dynamics and their activities inside the vehicle compose the
internal driving context.

The external driving context is a traffic-heavy commute on a city street. It’s
raining and visibility is low. The vehicle is moving through a busy urban
environment, with many intersections and pedestrians. The external non-driving
context includes beautiful architecture on either side of the road. AV traits
are attributes of the AV which impact the way it acts and can be acted with.
The AV’s driving abilities, communication affordances, justification for
decisions, and even the algorithms it is trained with are examples of the AV’s
traits.

In this example, the AV is fully aware of the external driving context via
sensors and capable of driving in these conditions safely with the family, so
we will assume that the AV is fully capable of performing adequately. For the
goal of safe transportation and learning to be satisfied, certain additional
actions need to be taken. Specifically, the preconditional criteria to achieve
these goals need to be addressed for the human-AV team. These include
establishing trust and determining information that can augment the son’s
learning experience.

Each person in the vehicle has subject-specific traits and states which, for
each of their interaction goals, culminates in a set of action needs. The
mother is predisposed to distrusting the AV’s decisions: she’s never ridden in
an AV before, she is not generally open to new experiences, and she is
stressed and fatigued from a long day of travel. On top of this, the presence
of her son lowers her tolerance for risk, and the lack of visibility due to
rain makes her question the AV’s performance in this higher complexity
external driving context. Her son is less concerned about the rain than his
mom, and is instead intent on understanding why the vehicle is making the
decisions that it is: why is it shifting lanes, how does it merge and at what
pace does it slow down?

Some action needs can be filled without explicit communication: the AV can
turn left, slow down, and monitor the external and internal driving contexts.
The mother and son can converse, observe the AV’s driving, and exit the
vehicle.

Many action needs are informational in nature and best filled via
communication between the people and the AV. The goals, driving context, AV
traits, and subject-specific traits and states, and actions combine to form
what we call the communicative context. This is a snapshot of the system as a
whole, where there are informational needs for all agents that need to be
satisfied. If the needs are not satisfied, goals like trust or learning cannot
be achieved.

Given the case that the AV can actually perform quite well, the AV will need
to communicate to the mother sufficient information such that she feels she
can trust – and therefore rely on – the AV’s driving decisions. In this way,
the AV is building the situational awareness of the mother to include an
understanding of its own abilities and actions. To know how to do this, the AV
itself needs to either be told by the mother, sense her discomfort and assume
it is caused by a lack of trust, or be pre-programmed to communicate
information that can increase trust.

The AV says via a robotic voice “don’t worry, I am trained to handle this
situation and have never crashed.” Though this may be sufficient for the risk-
loving 15 year old son, the mother was not fully satisfied by the AV’s first
communication– there is still an informational need to fill. She asks, “can
you tell me more?”

The AV now tries a different communicative strategy. It presents different
information via a different modality: on a visual dashboard display, it
validates that it can sense all of the vehicles around it. It also shows a
projection of where it and other vehicles will drive next so the mother can
predict their behavior. This satisfies the mother’s need for trust. Now she
can form a reliance strategy, thus allowing the goal of safe transportation to
be achieved.

The son loves the visualization, but seeks further justification for decisions
in order to learn. Why is it moving to the left lane? The AV provides further
explanations to the son, saying audibly “I am moving to the left lane to
prepare for a left turn at the next stoplight.” In this simplistic example,
let’s say this explanation satisfies the boy’s learning goal.

The audible explanations from the vehicle are making the mother’s head hurt.
She hits a button and the AV switches its instructional explanations to be
visual, using text bubbles on the display to continue the son’s instruction.
Are these as effective as auditory explanations?

A pedestrian crosses in front of the AV and the AV brakes suddenly to avoid
collision. The mother’s anxiety spikes: she has new informational needs to
fill now.

This example shows how successfully achieving goals and subgoals like trust or
learning require coordinated actions between the people and the AV. This, in
turn, requires taking into account the information needs of all parties and
communicating information in the right way to develop sufficient situational
awareness. Only through this situational awareness can informational goals be
achieved. When the communicative context changes, so do the SA requirements.

##  3\. Framework for situated joint action within a human-AV system

The framework presented in this paper shows how situational awareness can be
developed through joint actions with autonomous vehicles. To summarize: the
driving context, human subject traits and states, and AV traits determine
actions that need to be taken to achieve specific joint-action goals, such as
safe transportation, trust, or learning. Each goal has a success function that
determines the conditions that need to be met for that goal to be achieved.
The success function is sensitive to the wider context of interaction, meaning
when the communicative context changes, so do the conditions for success, so
actions must change too. Actions taken to achieve goals and goal criteria may
be communicative or non-communicative; some goals have subgoal conditions
which need to be met before a higher-level goal can be achieved. Actions are
coordinated and bidirectional, meaning that the human and AV need to account
for the action needs of each other. To achieve many joint action goals,
situational awareness (SA) is necessary. The human and AV both also have
individual SA, and when this individual SA matches it is considered to be
shared SA. The SA that emerges from the system as a whole is called
distributed SA, as the SA possessed by the human and AV does not always need
to match, but simply be complimentary with respect to action goals. To achieve
a joint action goal, the informational needs required for each type of SA must
reach a necessary threshold and be paired with appropriate non-communicative
actions.

Figure 1 details our framework, describing the external and internal factors
of the human-AV interaction. We show how action goals, the driving context,
and different traits (human and AV traits) are fundamental parameters
determining what actions - communicative and not - need to be jointly taken.
Actions build, and are iteratively impacted by, the different types of
situational awareness held by the system. In the next section, we expand on
and contextualizes this framework, discussing the specific elements that are
at its core.

Figure 1. Framework overview for situated joint action based on situational
awareness within a human-AV system. Joint Action Goals and AV Traits are
discussed in Section 4.1, AV traits and the development of situational
awareness through strategic communicative actions are discussed in Section
4.2, and the driving context and human traits and states are discussed in
Section 4.3.

##  4\. Discussion

We now discuss the framework in three detailed subsections, contextualizing it
in existing work and cognitive theory. The illustrative example presented in
Section 2 will help ground the framework in a realistic driving situation. We
first discuss how human-AI teams can work together to achieve joint action
goals and subgoals. Then, we show how these goals and subgoals can be achieved
through the development of individual, shared, and distributed situational
awareness; this development has classically used different communication
strategies to communicate various information about the AV (which we term AV
traits). In the last part of this section, we discuss how two additional
aspects of the human-AV system impact the success criteria (and thus necessary
communicative actions) to achieve goals and subgoals. These aspects of the
system include the internal and external driving and non-driving contexts as
well as the subject-specific traits and states of a human in the loop. Along
the way, we provide an overview of related work and knowledge gaps for the
human-AV community to address in future work.

###  4.1. Human-AI teaming for Action Goal Success

As AI-based systems advance, their increasing scope of decision-making actions
shifts them from being tools for human use to acting as agents in a human-AI
team (Dafoe et al., 2021). In the case of autonomous vehicles, current theory
proposes that successful designs should conceptualize a human and an AV as
team members that must work together to achieve common goals, such as safe
transportation.

Action Goals – In human-AI teaming, a person and an AV would be bound by
principles of joint action similar to those of coordinated human-to-human
activities (Sebanz et al., 2006). Joint actions are those which involve two or
more people or agents working together in coordination to achieve a common
goal, and include activities ranging from dressing a child to team rally car
driving. For example, the mother and the AV in our example needed to act
jointly, in coordination, to achieve the goal of safe transportation. We adopt
Sebanz et. al’s definition of action goals as those which direct and orient
behavior (Sebanz et al., 2006). These include higher-level goals like safe
transportation or learning as well as lower-level subgoals like building
trust, slowing for a pedestrian, or asking for more information. These
subgoals are preconditions that need to be met to achieve higher-level goals;
to achieve a higher level goal, the subgoals underneath it need to be
addressed. It is clear in our example of the family in the AV: without the
coordinated action of both the people and the AV, it would not have been
possible to achieve action goals and subgoal criteria.

Some subgoals like building trust in an AV’s decisions have received a lot of
focus by the research community. The issue of trust spans nearly all human-AV
goals, and is perhaps the most pressing issue hindering AV adoption and the
adoption of AI systems more generally (Choi and Ji, 2015).

Goals like learning from an AV’s decisions are severely underexplored by the
research community (Kaufman et al., 2024). Knowing how to design for the goal
of learning has implications both for the use of AVs as driving instructors as
well as the design of communications that can help a person build familiarity,
expertise, trust, and comfort with an AV by learning how the AV makes
decisions. Learning, like trust, is at the root of many human-AV interaction
problems.

Success Functions Determine Actions Necessary to Achieve Goals – Action goals
determine the success function for the system. The success function is
composed of a set of criteria which need to be met to achieve that particular
goal. We can think of these criteria like a multi-level checklist, where goals
have criteria to meet, some of which may be subgoals which, themselves, have
their own criteria to meet. There are similar, but not identical to, the goal
hierarchies described by Kridalukmana et al. for collaboration with automated
vehicles (Kridalukmana et al., 2020). Figure 2 shows an example. To
successfully achieve the action goal of safe transportation, for example, the
subgoal of calibrated reliance must be met amongst others. To achieve
calibrated reliance, knowing when and when not to trust an AV is necessary. To
know when and when not to trust an AV, a person needs to be aware of specific
information that can be used to develop a trust and reliance strategy. This
information has its own criteria, based on principles of communication
including the intentional design of the type of information needed to convey
as well as how it is conveyed. In many cases, criteria for a particular goal
or subgoal cannot be satisfied by a single agent alone; team members must work
together to satisfy their function.

Figure 2. Example Success function for the goal of Safe Transportation. The
success function for the goal of safe transportation is composed of sub-goals
which need to be met for the overall goal to be achieved. Each of these
subgoals have their own conditions, which themselves may have conditions, etc.
Joint communicative action may be required between a human and an AV to
achieve some goals or subgoals. This example is very simplified and for
illustrative purposes only.

For example, the mother in our story wasn’t confident that the AV could
perform well in the current rainy driving conditions. She did not trust the
AV, and did not know if she could rely on it. To know if she could rely on the
AV, she wanted to verify the AV’s capabilities. Specifically, she had criteria
of driving competence to fill: was the AV aware of the other cars around it,
and what would it do next? These are gaps in the mother’s situational
awareness. She could not satisfy these criteria alone. To help her address
these criteria, the AV produced information on a display which addressed each
criteria that could help her build calibrated trust and a reliance strategy.
The information produced had their own sub-criteria for success, based on
choosing the appropriate content, modality, etc. For the mother and the AV,
there was an adjustment period where the first communications produced by the
AV did not satisfy the mother’s criteria. This dissatisfaction was
communicated back to the AV, letting it know the communications needed to be
changed. In the end, because the right information was communicated in the
right way, the criteria for success were satisfied, and the joint action goal
was achieved. In sum, from a joint-action perspective, the success of a human-
AV team working towards a goal depends on members’ ability to (1) share
representations, (2) predict the other’s needs and actions, and (3) integrate
these predicted actions into their own behavior (Sebanz et al., 2006).

###  4.2. Developing Situational Awareness

Individual Situational Awareness and AV Traits – The root of satisfying the
success criteria for many goals is by satisfying information needs; this is
done by developing the individual situational awareness of each team member.
Chen et al.’s Situation Awareness-Based Agent Transparency (SAT) model of
situational awareness for human-agent interaction (Chen et al., 2014) suggests
three categories of agent information are necessary to build a person’s SA:
the agent’s current actions and plans, reasoning processes, and outcome
predictions. Chen et al.’s updated version of this model (specifically for
human-AI teams) (Chen et al., 2018) adds system transparency and bidirectional
communication. Per this updated model, the system’s communications should help
a person understand the current state (including execution of actions),
reasoning process, and future projections of the system, including how certain
the system’s predictions are. We will collectively refer to attributes and
affordances of the AV that determine how an AV acts and can be acted with –
including these listed by Chen et al. -– as ‘AV traits’. In satisfactorily
communicating information related to AV traits, Chen et al. suggest that a
person can achieve situational awareness when interacting with AI-based
systems. In our family example, the crucial AV traits to be communicated were
the driving abilities and decision rationale of the system; communicating
these was crucial for helping both the mother and son satisfy their
informational goals.

As human cognition and decision-making capacity are necessarily limited, an
emphasis on explainability in AI has recently emerged as a key factor
supporting successful human-agent interactions. This approach emphasizes that
by giving the right information, in the right way, at the right time – an AV
can help a person achieve many of their interaction goals, thereby increasing
the overall situational awareness that an individual has when driving with
their vehicle. This concept is a central fixture for the field of explainable
AI (XAI), which emerged in recent years as a means to generate trust with
autonomous decision-makers like AVs (Dzindolet et al., 2003; Benbasat and
Wang, 2005). DARPA’s XAI program posed explainable AI as a means for humans to
gain insight into the “black box” nature of machine learned systems so that
they know why a decision has been made, when to trust the decision, why errors
have occurred, and in what contexts the system will succeed and fail (Gunning
and Aha, 2019). Explainability may be especially important because automation
does not have an understanding of cause and effect (Pearl and Mackenzie,
2018). Building off this agenda, XAI systems have been used to implement
explanations for decisions in many domains (Das and Rad, 2020; Pazzani et al.,
2022; Kaufman and Kirsh, 2022, 2023), including autonomous driving (Koo et
al., 2015; Atakishiyev et al., 2021; Capallera et al., 2022).

Communication Strategies – Huge knowledge gaps remain in the human-AV
literature on how to provide the right information, in the right way, at the
right time to satisfy the criteria for a particular driving goal. For
simplicity, we’ll term any communicative action method by which a person and
an AV transfer information to eachother during a real-time moment of
interaction as ‘communication strategies’. These may include explicit,
implicit, or communication via sensing (Capallera et al., 2022). We are not
including data used for prior model training as a communication strategy,
though it is relevant to the design of interactive human-AV systems that use
this data to decide what communications to produce. To design effective
communications, it is important to know both what type of information is most
effective for building satisfactory situational awareness as well as what
modalities of presentation will most effectively allow access to that
information. Choosing the right communication strategy is based on satisfying
the information criteria necessary to achieve a joint action goal. To make XAI
communication bi-directional as Chen et al. suggests, knowledge gaps remain on
what information should be provided back to the AV, in what way, and at what
time.

In our example, we saw that the first communication strategy (saying “I never
crash”) was ineffective at eliciting trust from the mother, while the second
communication strategy (display of surroundings and future behavior) was
effective for the mother’s trust but ineffective at promoting learning for the
son. Different goals and subgoals, people, and contexts will require different
communication strategies. This is because different informational success
criteria require different informational strategies to meet those criteria.
Some studies show visual techniques using heads-up-displays (HUDs) are
preferred by people with certain goals in certain driving situations
(Schartmüller et al., 2019; Chang et al., 2016; Kaufman et al., 2024), while
others suggest auditory feedback as a better strategy (Jeon et al., 2009;
Löcken et al., 2017). There have even been a few studies involving haptics
specifically for situational awareness during driving (Capallera et al.,
2019a; Fink et al., 2023). Studies on the impact of the type and content of
information presented also show mixed results, including differences between
‘what is happening’ or ‘why it is happening’ information (Koo et al., 2015;
Omeiza et al., 2021; Kaufman et al., 2024).

Knowing how to convey information and how information requirements may change
for different people or contexts is crucial to designing systems that can meet
informational goal criteria. Without careful consideration of these factors,
ill-fitting communications may fail to meet rider’s needs or even derail
progress towards a goal. We can easily imagine a situation where presenting
the wrong information at the wrong time may overwhelm, confuse, or cause a
person to lose trust in an AV. These experience would their ability to learn
and understand what is happening in a situation.

Until this point, we have discussed situational awareness held by an
individual. To understand how to best build human-AV communications that
support coordinated joint action, it is helpful to examine situational
awareness from two additional perspectives: shared situational awareness and
distributed situational awareness.

Shared Situational Awareness – Shared representation and predictability for
joint action (criteria 1 and 2) are quite similar to what would be classically
required to achieve shared situational awareness. As such, it is possible to
examine shared situational awareness within the context of a joint action
between a human and an AV. Individual situational awareness becomes shared
when there is a common representation of a situation where each team member
can understand and predict a particular aspect of their environment in
synchrony (Salas et al., 1995; Nofi, 2000; Saner et al., 2009). Individuals
with shared situational awareness will have representations of a situation
that are similar to each other. For instance, in our story, when the AV
displayed the other cars around it, it allowed the family to share a common
representation of what the AV perceives with the AV and also with each other.
If there was a car in front of the AV that was not displayed, the mother would
know that there was a gap in the AV’s situational awareness of the external
driving context. This may impact her trust and reliance strategy. Shared
situational awareness is often required to satisfy criteria for joint action
goals. Shared situational awareness can be at the goal and subgoal
establishing level, too. The AV and mother both knowing that there is an
informational need to fill to achieve trust is another example of shared
situational awareness.

Distributed Situational Awareness – Distributed situational awareness, by
contrast, proposes that SA is emergent from the system as a whole, rather than
held by a given individual (Stanton et al., 2006; Salmon and Plant, 2022;
Salmon et al., 2020). In terms of a joint action, teams with distributed
situational awareness may comprise individuals with different but compatible
representations of a situation. By this view, an autonomous vehicle will have
its own situational awareness and will also contribute to the total SA of the
distributed system as a whole. The distributed view of situational awareness
provides a practical way to see how agents with different roles in a joint
action team can collaborate on achieving a goal. Using our example, the mother
is aware that driving in rain can be risky, that she lacks trust, and when her
trust criteria are met. The AV is aware of the distance to other cars on the
road, its predictions about their behavior, and how those predictions impact
its own decisions. It must also be aware of the informational needs of the
mother. In this case, we see that even when internal representations differ –
such as within human-AV teams – successful joint action and coordination can
still be possible.

Whether the team is said to have distributed or shared situational awareness
may be a matter of shifting perspective between the role of the team and the
role of the individual. In most cases, it is likely that both distributed and
shared situational awareness may be necessary to achieve a particular action
goal. The shift from individual to shared to distributed can best be viewed
from the perspective of what is needed to satisfy action goal criteria. For
example, both a human and an AV may need to share an accurate representation
of the “big picture” for a particular driving context – including an
understanding of the joint subgoal criteria of trust– while the information
required to achieve this subgoal can be distributed between independent
members. To achieve trust, specific decisions regarding whether to trust the
AV may be delegated to the mother, while accurately calculating the distance
to the nearest car is left to the AV (Salmon et al., 2020). Of course, these
are interrelated, as communications given to the mother by the AV about the
nearest car will affect whether her criteria for trust are satisfied. The
implication is that designing for human-AV situational awareness is possible
despite fundamental differences in the modalities of representation employed
by humans and AI-based systems like AVs, provided appropriate information is
communicated in accessible ways to each. The awareness held by each, and the
communicative actions taken by each, are relative to achieving the action
goal.

Crucial to applying this work for human-AV interaction designers, it is
important to once again emphasize that joint actions are coordinated and
bidirectional: each partner must integrate their partner’s needs and behaviors
in space and time to their own action planning (joint action criteria 3).
Action coordination affects the affordances of what can be done, motivates
what should be done, and permits joint anticipatory adjustment of future
behavior to achieve a joint goal. This means that to achieve trust, the
mother, AV and system as a whole must have sufficient situational awareness –
and ability to act on this awareness – such that they can account for the
needs and actions of the other.

###  4.3. Wider System Factors Impacting Success Criteria and Communication
Needs

Situated Driving and Non-Driving Contexts – The human-AV system does not exist
in a vacuum; it is sensitive to the situated, contextual demands of the
physical and social environment (Clancey, 1997; Kirsh, 2009). We use ‘driving
context’ as an umbrella term encompassing the scene, situation, and scenario
within which the human and AV are interacting (Ulbrich et al., 2015).
Succinctly, a scene is a momentary snapshot of elements like traffic and an
agent’s self-representation of these elements; a situation includes all
relevant conditions, options and determinants for action derived from a scene
via information selection and goal-driven processes; and a scenario is a
sequence of scenes. The driving context includes both factors outside of the
vehicle and inside the vehicle.

The external driving context includes features like road conditions, traffic,
and visibility (Capallera et al., 2019b). These may impact the perceived risk
of harm or cognitive complexity of knowing what to do in a particular
situation. In our example, the external driving context was a high traffic
commute, in an urban city, during rain. This can be further divided by the
specific commute features, like moving through an intersection or stopping for
an unexpected pedestrian.

The external non-driving context includes factors outside of the vehicle which
do not necessarily have direct implications on driving, but may impact the
driving experience itself. In our example, this includes the sight of
beautiful building architecture outside of the vehicle, which does not impact
the AV’s driving but may impact passenger preferences and how attention is
directed throughout the ride. Predicting how the external non-driving context
will impact passenger states and goals requires general intelligence.

The internal driving context may include social dynamics between riders or
distractions that may arise inside the vehicle cabin, such as Non-Driving
Related Tasks (NDRTs) (Naujoks et al., 2018). We adopt a situated action view,
where the driving context includes viewing driving as a social, culturally-
dependent activity where cognitive processes are distributed across the system
(Hollan et al., 2000). The driving context from our example changed when the
unexpected pedestrian appeared in the road, requiring the AV to stop. It would
also change if another person entered the vehicle (internal) or if the rain
stopped (external).

As the driving context changes, so do the criteria required to satisfy the
success function for a particular goal. It is clear that joint actions between
a human and an AV must take many aspects of the distributed system into
account in order to allow the team to reach their goals. Several studies have
incorporated external driving conditions into their situational awareness
studies (Capallera et al., 2022; De Salis et al., 2020), however nearly all
focus on identification of obstacles via interfaces (Sirkin et al., 2017) with
very little attention given to internal driving contexts or adapting
communication between different driving contexts. Underlying aspects of the
driving contexts -– like risk of harm and situation complexity -– which may
help meaningfully categorize contexts to give a deeper insight into the
situated context, are likewise unexplored. Open research questions remain
regarding how contextual changes impact the informational criteria of
particular goals or subgoals like learning or developing trust in an AV. What
contextual factors matter, to what degree, and how they should be communicated
about? Understanding the potentially differential impact of context on
communication needs is paramount to building context-aware systems that can
work even when the driving situation changes (Schilit et al., 1994).

Subject-specific Traits and States – To coordinate joint actions, an AV will
need a representation of the needs, behaviors, and attributes of their human
team member(s), just as the person needs a representation of the AV’s traits.
We call the attributes that determine needs and behaviors for a particular
person their ‘subject-specific traits and states’, which will vary from person
to person. In building trust with automated systems, Hoff and Bashir’s model
divides subject-specific factors into three categories: situational,
dispositional, and learned (Hoff and Bashir, 2015). Though these are not
specifically for the development of situational awareness, they are useful as
a scaffold for understanding the wide range of subject-specific factors which
may need to be accounted for in designing situationally-aware and adaptable
human-AV systems. According to Hoff and Bashir, situational states vary by a
specific situation and include a person’s perceptions of a system’s risk and
complexity, cognitive load, confidence, and subject-matter expertise. The
latter three – cognitive load, confidence, and expertise – are where a
majority of human-AV situational awareness studies have focused on (Capallera
et al., 2022; Saner et al., 2009). Subject-specific traits that are
dispositional (generally fixed) in nature, including measures of culture,
personality, gender, cognitive style, and information processing abilities are
fairly underexplored (Ferronato and Bashir, 2020). The same can be said for
learned factors stemming from prior experiences and preexisting knowledge.
Learned factors include technology attitudes, expectations, system
understanding, and changes based on a momentary evaluation of a system’s
performance (Ayoub et al., 2021).

It is not difficult to see how variance in subject-specific traits and states
may impact the informational needs people may develop. For example, the mother
in our story had never ridden in an AV before, was not generally open to new
experiences, and was stressed and fatigued from a long day of travel. Her
risk-willingness was also lower due to the presence of her child. We can
imagine that she may have different informational needs than a single, open-
minded tech lover who enjoys risk-taking. We would also expect that there may
be differences in her needs between her first time interacting with an AV and
her hundredth.

Open questions remain regarding which subject-specific traits and states may
be relevant, and to what degree they need to be represented. A sufficient
understanding of the impact of subject-specific traits can be used to design
flexible, tailored communications for diverse audiences. Integrating these
together during model training can be used to customize communications to a
specific person.

Communications Should Be Sensitive To The Communicative Context – The AV’s
communications should be sensitive to the human they are interacting with as
well as the relevant goals and contextual factors, causing a complex
interaction between actions (including communication strategies), AV traits,
subject-specific traits and states, action goals, and the driving context. For
simplicity, we collectively refer to these factors in combination as the
‘communicative context’. We can think of a single communicative context as a
momentary snapshot of the full driving system, which determines the criteria
which need to be satisfied for a particular goal as well as the ways in which
that goal can be satisfied. When an aspect of the system changes – such as
when a new passenger gets into the AV or the vehicle enters into a new driving
context -– the communicative context changes. In some cases, the criteria for
success will change as well as the possible ways to satisfy that criteria.

With the right AV communications, an AV may be able to deliver information
such that sufficient situational awareness is achieved, and joint actions
taken by the team can reach a team’s action goals. What counts as sufficient
is based on the success function for the specific action goal in question –
informational criteria for a goal may be met even with partial situational
awareness. For different communicative contexts, the proportion of necessary
individual, shared, and distributed situational awareness may change. Success
criteria for particular action goals are not often clearly articulated by the
research community. However, many preliminary XAI studies on human-AV
interaction show promise that certain AV representations can help increase
human trust, understanding, and situational awareness. Open questions remain
regarding how much situational awareness is necessary for each team member to
possess to permit joint action and goal success. In addition, the degree to
which different factors (like subject-specific traits or communication
strategies) may impact the amount of situational awareness obtained remains
unknown. For example, it is possible that people who are predisposed to
trusting automation will require less situational awareness when interacting
with a vehicle. It is also possible that XAI communications will be more
effective for open-minded novices or for those with low cognitive load. There
is likely a nonlinear relationship between the amount of information and the
usefulness of that information, dependent on information content. In essence,
there are dozens of potential avenues for future work.

##  5\. Applying the Framework

This framework can be used to show how accounting for AV traits, action goals,
subject-specific traits and states, and the driving context can be used to
design communications that can satisfy human-AV situational awareness success
criteria for joint action goals like learning from an AV or conditional
subgoals like building trust. In an applied sense, breaking the human-AV
system into its constituent parts (e.g. subject-specific traits, contextual
characteristics) and emphasizing interactions between these parts (e.g. trust
in an AV may be lower in rain) provides a scaffold that can be used by future
researchers and designers to understand many complex human-AI interaction
problems in a diverse range of domains. For real-world deployment of
autonomous vehicles and most other AI systems, ensuring that interactions meet
the needs of their (often diverse) user bases and can remain contextually-
relevant when goals or contexts change can be a distinct design challenge.
This framework can be used as a tool to find combinations of factors that need
to be accounted for when designing these interactions. Past and future
research can determine what factors are important to consider, and when. If we
design a system without taking each aspect into account, AI communications run
the risk of failing to meet the needs users. This may alienate a user or - in
the case of AVs and other safety-critical systems - cause real-world physical
or financial harm.

One procedure that interaction designers can take is to work through the
framework systematically. Starting with the goals of the human-AV team, an
initial success function can be written (similar to the example in Figure 2,
but more comprehensive) showing what actions are needed to achieve that goal
and its subgoals. What is known about the AV traits, subject traits and
states, and driving context can be included from the beginning as a means to
determine the human-AV team’s needs, with knowledge gaps highlighted as areas
needed to be filled. For example, a designer might know the information
content desired by a particular user for a particular goal and in a particular
driving context, but may not know the amount, modality, or timing of the
information to be presented, or how information should be adjusted should a
new driving context or user enter the system. As information is collected
about what is known and not, the subgoals and subgoal criteria can be
iteratively adjusted in the success function in order to show what actions
should be taken. It is unrealistic to assume that all aspects of the system
can be accounted for at all times, so an emphasis should be on what is
necessary and sufficient to include. Determining the importance of different
factors and what should be included will be part of the design challenge.
Different functions may need to be drawn for different user types, contexts,
or goals. Through this exercise, designers can gain a comprehensive mapping
for how to design communications that meet a success function’s criteria
and/or an understanding of what additional research is needed.

In line with the process used for designing interactions outlined above, this
framework can also serve as a foundation to deepen our study of situational
awareness and joint action within complex human-AI systems. A similar process
of walking through success criteria and determining what is known, unknown,
and important from each of the system factors can be used by the research
community to articulate gaps in knowledge that must be filled before AI
systems can reach their full potential (e.g. need to know how to design
communications for low-trust people in the rain). From the standpoint of
cognitive theory, experiments targeting specific areas of the system can help
differentiate what type and amount of SA - individual, shared, or distributed
- is necessary for different conditions and what the most efficient methods
are for developing this SA within different communicative contexts. Showing
how SA requirements may differ for people with high or low cognitive load
capacity, for example, can give additional insight into the relationship
between cognitive load and working memory.

##  6\. Conclusion: Framework and System Summary

In this work, we present a framework for developing situational awareness
within human-autonomous vehicle interactive systems based on a theory of joint
action. To summarize the framework: individual, shared and distributed
situational awareness – up to a particular threshold – are necessary for a
human-AV team to satisfy the success criteria to jointly achieve action goals.
Joint action goals include high-level goals such as safe transportation and
learning from an AV’s behavior, which themselves often have lower-level
subgoals like trust which need to be met. In many cases, both the human and
the AV must have a sufficiently accurate shared representation of each other
and the situated driving context. They also must have the individual
situational awareness required to coordinate and complete each of their own
distinct actions. The situational awareness of the distributed system as a
whole must be sufficient that the team together can accomplish action goals.
Information can be transferred via communication strategies like XAI
explanations for a person, and real-time human input and sensing for an AV.
Training data can be used by an AV as a means to predict and tailor
communications for a specific communicative context ahead of time.
Understanding four intersecting components of the shared situation – AV
traits, action goals, subject-specific traits and states, and the driving
context – is necessary to promote situational awareness and permit joint
action. Successful joint action requires the extra step of predicting and
adjusting behavior based on individual and shared situational awareness to
reach the team’s goal. For human-AV XAI design, the implication is that AV
communications should be tailored based on these four factors and be sensitive
when the communicative context changes. Huge knowledge gaps remain in each of
these areas.

###### Acknowledgements.

We would like to thank Emi Lee, Janzen Molina, Saumitra Sapre, and Rohan Bhide
for their support on projects related to human-AV interaction which inspired
the work in this manuscript. We would like to thank Cat Hicks for feedback
related to theoretical components of this framework.

## References

  * (1)
  * Atakishiyev et al. (2021) Shahin Atakishiyev, Mohammad Salameh, Hengshuai Yao, and Randy Goebel. 2021.  Explainable artificial intelligence for autonomous driving: A comprehensive overview and field guide for future research directions.  _arXiv preprint arXiv:2112.11561_ (2021).
  * Ayoub et al. (2021) Jackie Ayoub, X Jessie Yang, and Feng Zhou. 2021.  Modeling dispositional and initial learned trust in automated vehicles with predictability and explainability.  _Transportation research part F: traffic psychology and behaviour_ 77 (2021), 102–116.
  * Baumann and Krems (2007) Martin Baumann and Josef F Krems. 2007.  Situation awareness and driving: A cognitive model.  In _Modelling driver behaviour in automotive environments: Critical issues in driver interactions with intelligent transport systems_. Springer, 253–265.
  * Benbasat and Wang (2005) Izak Benbasat and Weiquan Wang. 2005.  Trust in and adoption of online recommendation agents.  _Journal of the association for information systems_ 6, 3 (2005), 4.
  * Capallera et al. (2022) Marine Capallera, Leonardo Angelini, Quentin Meteier, Omar Abou Khaled, and Elena Mugellini. 2022.  Human-vehicle interaction to support driver’s situation awareness in automated vehicles: a systematic review.  _IEEE Transactions on intelligent vehicles_ 8, 3 (2022), 2551–2567.
  * Capallera et al. (2019a) Marine Capallera, Peïo Barbé-Labarthe, Leonardo Angelini, Omar Abou Khaled, and Elena Mugellini. 2019a.  Convey situation awareness in conditionally automated driving with a haptic seat. In _Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings_. 161–165.
  * Capallera et al. (2019b) Marine Capallera, Quentin Meteier, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, and Elena Mugellini. 2019b.  Owner manuals review and taxonomy of ADAS limitations in partially automated vehicles. In _Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_. 156–164.
  * Chang et al. (2016) Chun-Cheng Chang, Jaka Sodnik, and Linda Ng Boyle. 2016.  Don’t speak and drive: cognitive workload of in-vehicle speech interactions. In _Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_. 99–104.
  * Chen et al. (2018) Jessie YC Chen, Shan G Lakhmani, Kimberly Stowers, Anthony R Selkowitz, Julia L Wright, and Michael Barnes. 2018.  Situation awareness-based agent transparency and human-autonomy teaming effectiveness.  _Theoretical issues in ergonomics science_ 19, 3 (2018), 259–282.
  * Chen et al. (2014) Jessie Y Chen, Katelyn Procci, Michael Boyce, Julia Wright, Andre Garcia, and Michael Barnes. 2014.  Situation awareness-based agent transparency.  _US Army Research Laboratory_ April (2014), 1–29.
  * Choi and Ji (2015) Jong Kyu Choi and Yong Gu Ji. 2015.  Investigating the importance of trust on adopting an autonomous vehicle.  _International Journal of Human-Computer Interaction_ 31, 10 (2015), 692–702.
  * Clancey (1997) William J Clancey. 1997.  _Situated cognition: On human knowledge and computer representations_.  Cambridge university press.
  * Cuzzolin et al. (2020) Fabio Cuzzolin, Alice Morelli, Bogdan Cirstea, and Barbara J Sahakian. 2020.  Knowing me, knowing you: theory of mind in AI.  _Psychological medicine_ 50, 7 (2020), 1057–1061.
  * Dafoe et al. (2021) Allan Dafoe, Yoram Bachrach, Gillian Hadfield, Eric Horvitz, Kate Larson, and Thore Graepel. 2021.  Cooperative AI: machines must learn to find common ground.
  * Das and Rad (2020) Arun Das and Paul Rad. 2020.  Opportunities and challenges in explainable artificial intelligence (xai): A survey.  _arXiv preprint arXiv:2006.11371_ (2020).
  * De Salis et al. (2020) Emmanuel De Salis, Marine Capallera, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, and Stefano Carrino. 2020.  Designing an AI-companion to support the driver in highly autonomous cars. In _Human-Computer Interaction. Multimodal and Natural Interaction: Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22_. Springer, 335–349.
  * Detjen et al. (2021) Henrik Detjen, Sarah Faltaous, Bastian Pfleging, Stefan Geisler, and Stefan Schneegass. 2021.  How to increase automated vehicles’ acceptance through in-vehicle interaction design: A review.  _International Journal of Human–Computer Interaction_ 37, 4 (2021), 308–330.
  * Dzindolet et al. (2003) Mary T Dzindolet, Scott A Peterson, Regina A Pomranky, Linda G Pierce, and Hall P Beck. 2003.  The role of trust in automation reliance.  _International journal of human-computer studies_ 58, 6 (2003), 697–718.
  * Ekman et al. (2017) Fredrick Ekman, Mikael Johansson, and Jana Sochor. 2017.  Creating appropriate trust in automated vehicle systems: A framework for HMI design.  _IEEE Transactions on Human-Machine Systems_ 48, 1 (2017), 95–101.
  * Endsley (1988) Mica R Endsley. 1988.  Situation awareness global assessment technique (SAGAT). In _Proceedings of the IEEE 1988 national aerospace and electronics conference_. IEEE, 789–795.
  * Endsley (1995) Mica R Endsley. 1995.  Toward a theory of situation awareness in dynamic systems.  _Human factors_ 37, 1 (1995), 32–64.
  * Fagnant and Kockelman (2015) Daniel J Fagnant and Kara Kockelman. 2015.  Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations.  _Transportation Research Part A: Policy and Practice_ 77 (2015), 167–181.
  * Ferronato and Bashir (2020) Priscilla Ferronato and Masooda Bashir. 2020.  An examination of dispositional trust in human and autonomous system interactions. In _Human-Computer Interaction. Human Values and Quality of Life: Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part III 22_. Springer, 420–435.
  * Fink et al. (2023) Paul DS Fink, Anas Abou Allaban, Omoruyi E Atekha, Raymond J Perry, Emily S Sumner, Richard R Corey, Velin Dimitrov, and Nicholas A Giudice. 2023.  Expanded situational awareness without vision: a novel haptic interface for use in fully autonomous vehicles. In _Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction_. 54–62.
  * Frison et al. (2019) Anna-Katharina Frison, Philipp Wintersberger, Andreas Riener, Clemens Schartmüller, Linda Ng Boyle, Erika Miller, and Klemens Weigl. 2019.  In UX we trust: Investigation of aesthetics and usability of driver-vehicle interfaces and their impact on the perception of automated driving. In _Proceedings of the 2019 CHI conference on human factors in computing systems_. 1–13.
  * Ghai et al. (2021) Bhavya Ghai, Q Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2021.  Explainable active learning (xal) toward ai explanations as interfaces for machine teachers.  _Proceedings of the ACM on Human-Computer Interaction_ 4, CSCW3 (2021), 1–28.
  * Gunning and Aha (2019) David Gunning and David Aha. 2019.  DARPA’s explainable artificial intelligence (XAI) program.  _AI magazine_ 40, 2 (2019), 44–58.
  * Hegner et al. (2019) Sabrina M Hegner, Ardion D Beldad, and Gary J Brunswick. 2019.  In automatic we trust: investigating the impact of trust, control, personality characteristics, and extrinsic and intrinsic motivations on the acceptance of autonomous vehicles.  _International Journal of Human–Computer Interaction_ 35, 19 (2019), 1769–1780.
  * Hoff and Bashir (2015) Kevin Anthony Hoff and Masooda Bashir. 2015.  Trust in automation: Integrating empirical evidence on factors that influence trust.  _Human factors_ 57, 3 (2015), 407–434.
  * Hollan et al. (2000) James Hollan, Edwin Hutchins, and David Kirsh. 2000.  Distributed cognition: toward a new foundation for human-computer interaction research.  _ACM Transactions on Computer-Human Interaction (TOCHI)_ 7, 2 (2000), 174–196.
  * Jeon et al. (2009) Myounghoon Jeon, Benjamin K Davison, Michael A Nees, Jeff Wilson, and Bruce N Walker. 2009.  Enhanced auditory menu cues improve dual task performance and are preferred with in-vehicle technologies. In _Proceedings of the 1st international conference on automotive user interfaces and interactive vehicular applications_. 91–98.
  * Kaufman et al. (2024) Robert Kaufman, Jean Costa, and Everlyne Kimani. 2024.  Learning Racing From an AI Coach: Effects of Multimodal Autonomous Driving Explanations on Driving Performance, Cognitive Load, Expertise, and Trust.  _arXiv preprint arXiv:2401.04206_ (2024).
  * Kaufman and Kirsh (2023) Robert Kaufman and David Kirsh. 2023.  Explainable AI And Visual Reasoning: Insights From Radiology.  _arXiv preprint arXiv:2304.03318_ (2023).
  * Kaufman and Kirsh (2022) Robert A Kaufman and David Kirsh. 2022.  Cognitive differences in human and AI explanation. In _Proceedings of the Annual Meeting of the Cognitive Science Society_ , Vol. 44.
  * Kirsh (2009) David Kirsh. 2009.  Problem solving and situated cognition.  (2009).
  * Kohn et al. (2021) Spencer C Kohn, Ewart J De Visser, Eva Wiese, Yi-Ching Lee, and Tyler H Shaw. 2021.  Measurement of trust in automation: A narrative review and reference guide.  _Frontiers in psychology_ 12 (2021), 604977.
  * Koo et al. (2015) Jeamin Koo, Jungsuk Kwac, Wendy Ju, Martin Steinert, Larry Leifer, and Clifford Nass. 2015.  Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance.  _International Journal on Interactive Design and Manufacturing (IJIDeM)_ 9 (2015), 269–275.
  * Krems and Baumann (2009) Josef F Krems and Martin RK Baumann. 2009.  Driving and situation awareness: A cognitive model of memory-update processes. In _Human Centered Design: First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009 Proceedings 1_. Springer, 986–994.
  * Kridalukmana et al. (2020) Rinta Kridalukmana, Hai Yan Lu, and Mohsen Naderpour. 2020.  A supportive situation awareness model for human-autonomy teaming in collaborative driving.  _Theoretical Issues in Ergonomics Science_ 21, 6 (2020), 658–683.
  * Liu and Qi (2023) Miao Liu and Bo Qi. 2023.  Design study on the effect of intelligent vehicles interaction mode on drivers’ cognitive load. In _International Conference on Human-Computer Interaction_. Springer, 42–57.
  * Löcken et al. (2017) Andreas Löcken, Shadan Sadeghian Borojeni, Heiko Müller, Thomas M Gable, Stefano Triberti, Cyriel Diels, Christiane Glatz, Ignacio Alvarez, Lewis Chuang, and Susanne Boll. 2017.  Towards adaptive ambient in-vehicle displays and interactions: Insights and design guidelines from the 2015 AutomotiveUI dedicated workshop.  _Automotive User Interfaces: Creating Interactive Experiences in the Car_ (2017), 325–348.
  * Ma and Kaber (2007) Ruiqi Ma and David B Kaber. 2007.  Situation awareness and driving performance in a simulated navigation task.  _Ergonomics_ 50, 8 (2007), 1351–1364.
  * Marangunić and Granić (2015) Nikola Marangunić and Andrina Granić. 2015.  Technology acceptance model: a literature review from 1986 to 2013.  _Universal access in the information society_ 14 (2015), 81–95.
  * Meyer-Waarden and Cloarec (2022) Lars Meyer-Waarden and Julien Cloarec. 2022.  “Baby, you can drive my car”: Psychological antecedents that drive consumers’ adoption of AI-powered autonomous vehicles.  _Technovation_ 109 (2022), 102348.
  * Morra et al. (2019) Lia Morra, Fabrizio Lamberti, F Gabriele Pratticó, Salvatore La Rosa, and Paolo Montuschi. 2019.  Building trust in autonomous vehicles: Role of virtual reality driving simulators in HMI design.  _IEEE Transactions on Vehicular Technology_ 68, 10 (2019), 9438–9450.
  * Naujoks et al. (2018) Frederik Naujoks, Dennis Befelein, Katharina Wiedemann, and Alexandra Neukum. 2018.  A review of non-driving-related tasks used in studies on automated driving. In _Advances in Human Aspects of Transportation: Proceedings of the AHFE 2017 International Conference on Human Factors in Transportation, July 17- 21, 2017, The Westin Bonaventure Hotel, Los Angeles, California, USA 8_. Springer, 525–537.
  * Nofi (2000) Albert A Nofi. 2000.  Defining and measuring shared situational awareness.  (2000).
  * Omeiza et al. (2021) Daniel Omeiza, Konrad Kollnig, Helena Web, Marina Jirotka, and Lars Kunze. 2021.  Why not explain? effects of explanations on human perceptions of autonomous driving. In _2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO)_. IEEE, 194–199.
  * Pazzani et al. (2022) Michael Pazzani, Severine Soltani, Robert Kaufman, Samson Qian, and Albert Hsiao. 2022.  Expert-informed, user-centric explanations for machine learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 36. 12280–12286.
  * Pearl and Mackenzie (2018) Judea Pearl and Dana Mackenzie. 2018.  _The book of why: the new science of cause and effect_.  Basic books.
  * Premack and Woodruff (1978) David Premack and Guy Woodruff. 1978.  Does the chimpanzee have a theory of mind?  _Behavioral and brain sciences_ 1, 4 (1978), 515–526.
  * Rangesh et al. (2018) Akshay Rangesh, Nachiket Deo, Kevan Yuen, Kirill Pirozhenko, Pujitha Gunaratne, Heishiro Toyoda, and Mohan M Trivedi. 2018.  Exploring the situational awareness of humans inside autonomous vehicles. In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_. IEEE, 190–197.
  * Reich and Trapp (2020) Jan Reich and Mario Trapp. 2020.  SINADRA: towards a framework for assurable situation-aware dynamic risk assessment of autonomous vehicles. In _2020 16th European dependable computing conference (EDCC)_. IEEE, 47–50.
  * Salas et al. (1995) Eduardo Salas, Carolyn Prince, David P Baker, and Lisa Shrestha. 1995.  Situation awareness in team performance: Implications for measurement and training.  _Human factors_ 37, 1 (1995), 123–136.
  * Salmon and Plant (2022) Paul M Salmon and Katherine L Plant. 2022.  Distributed situation awareness: From awareness in individuals and teams to the awareness of technologies, sociotechnical systems, and societies.  _Applied Ergonomics_ 98 (2022), 103599.
  * Salmon et al. (2020) Paul M Salmon, Neville A Stanton, and Guy H Walker. 2020.  13 Distributed Situation Awareness and Vehicle Automation.  _Handbook of human factors for automated, connected, and intelligent vehicles_ (2020).
  * Saner et al. (2009) Lelyn D Saner, Cheryl A Bolstad, Cleotilde Gonzalez, and Haydee M Cuevas. 2009.  Measuring and predicting shared situation awareness in teams.  _Journal of cognitive engineering and decision making_ 3, 3 (2009), 280–308.
  * Schartmüller et al. (2019) Clemens Schartmüller, Klemens Weigl, Philipp Wintersberger, Andreas Riener, and Marco Steinhauser. 2019.  Text comprehension: Heads-up vs. auditory displays: Implications for a productive work environment in sae level 3 automated vehicles. In _Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_. 342–354.
  * Schilit et al. (1994) Bill Schilit, Norman Adams, and Roy Want. 1994.  Context-aware computing applications. In _1994 first workshop on mobile computing systems and applications_. IEEE, 85–90.
  * Sebanz et al. (2006) Natalie Sebanz, Harold Bekkering, and Günther Knoblich. 2006.  Joint action: bodies and minds moving together.  _Trends in cognitive sciences_ 10, 2 (2006), 70–76.
  * Sirkin et al. (2017) David Sirkin, Nikolas Martelaro, Mishel Johns, and Wendy Ju. 2017.  Toward measurement of situation awareness in autonomous vehicles. In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_. 405–415.
  * Soltani et al. (2022) Severine Soltani, Robert A Kaufman, and Michael J Pazzani. 2022.  User-centric enhancements to explainable ai algorithms for image classification. In _Proceedings of the Annual Meeting of the Cognitive Science Society_ , Vol. 44.
  * Stanton et al. (2001) Neville A Stanton, Peter RG Chambers, and John Piggott. 2001.  Situational awareness and safety.  _Safety science_ 39, 3 (2001), 189–204.
  * Stanton et al. (2006) Neville A Stanton, Rebecca Stewart, Don Harris, Robert J Houghton, Chris Baber, Richard McMaster, Paul Salmon, Geoff Hoyle, Guy Walker, Mark S Young, et al. 2006\.  Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology.  _Ergonomics_ 49, 12-13 (2006), 1288–1311.
  * Tulli and Aha (2024) Silvia Tulli and David W Aha. 2024.  _Explainable Agency in Artificial Intelligence: Research and Practice_.  CRC Press.
  * Ulbrich et al. (2015) Simon Ulbrich, Till Menzel, Andreas Reschka, Fabian Schuldt, and Markus Maurer. 2015.  Defining and substantiating the terms scene, situation, and scenario for automated driving. In _2015 IEEE 18th international conference on intelligent transportation systems_. IEEE, 982–988.
  * Vorm and Combs (2022) Eric S Vorm and David JY Combs. 2022.  Integrating transparency, trust, and acceptance: The intelligent systems technology acceptance model (ISTAM).  _International Journal of Human–Computer Interaction_ 38, 18-20 (2022), 1828–1845.
  * Wang et al. (2021) Qiaosi Wang, Koustuv Saha, Eric Gregori, David Joyner, and Ashok Goel. 2021.  Towards mutual theory of mind in human-ai interaction: How language reflects what students perceive about a virtual teaching assistant. In _Proceedings of the 2021 CHI conference on human factors in computing systems_. 1–14.
  * Wiegand et al. (2019) Gesa Wiegand, Matthias Schmidmaier, Thomas Weber, Yuanting Liu, and Heinrich Hussmann. 2019.  I drive-you trust: Explaining driving behavior of autonomous cars. In _Extended abstracts of the 2019 chi conference on human factors in computing systems_. 1–6.