# A Review of Virtual Reality Studies on Autonomous Vehicle–Pedestrian
Interaction

Tram Thi Minh Tran<EMAIL_ADDRESS>0000-0002-4958-2465 Design Lab,
Sydney School of Architecture, Design and Planning, The University of
SydneySydneyNSWAustralia ,  Callum Parker<EMAIL_ADDRESS>0000-0002-2173-9213 Design Lab, Sydney School of Architecture, Design and
Planning, The University of SydneySydneyNSWAustralia and  Martin Tomitsch
<EMAIL_ADDRESS>0000-0003-1998-2975 Design Lab, Sydney School of
Architecture, Design and Planning, The University of SydneySydneyNSWAustralia

(2024)

###### Abstract.

An increasing number of studies employ virtual reality (VR) to evaluate
interactions between autonomous vehicles (AVs) and pedestrians. VR simulators
are valued for their cost-effectiveness, flexibility in developing various
traffic scenarios, safe conduct of user studies, and acceptable ecological
validity. Reviewing the literature between 2010 and 2020, we found 31
empirical studies using VR as a testing apparatus for both implicit and
explicit communication. By performing a systematic analysis, we identified
current coverage of critical use cases, obtained a comprehensive account of
factors influencing pedestrian behavior in simulated traffic scenarios, and
assessed evaluation measures. Based on the findings, we present a set of
recommendations for implementing VR pedestrian simulators and propose
directions for future research.

autonomous vehicles, pedestrians, external human-machine interfaces, virtual
reality

††journalyear: 2024††copyright: rightsretained††conference: Proceedings of the
CHI Conference on Human Factors in Computing Systems; May 11–16, 2024;
Honolulu, HI, USA††booktitle: Proceedings of the CHI Conference on Human
Factors in Computing Systems (CHI ’24), May 11–16, 2024, Honolulu, HI,
USA††doi: 10.1145/3613904.3642118††isbn: 979-8-4007-0330-0/24/05

##  1\. Introduction

The driverless technology industry demonstrates enormous promise to
revolutionize passenger transportation by increasing riders’ independence,
comfort, and safety (Anderson et al., 2014). Mass deployment of AVs, however,
depends not only on technological progress and regulatory approval but also on
public acceptance (Nordhoff et al., 2018). Therefore, pedestrian interaction
with AVs has been a research area of critical importance. Lacking any physical
protections, pedestrians are the road users most vulnerable to fatalities and
injuries in traffic collisions (WHO, 2018). Additionally, pedestrians are
likely to be less familiar with self-driving technology compared to vehicle
occupants (Robert Jr, 2019). Practical applications aimed at solving
AV–pedestrian interaction problems include pedestrian intention estimation and
reasoning algorithms and human-centered design methods involving the overt
communication of AVs (Rasouli and Tsotsos, 2020). The latter are the focus of
this paper. Although pedestrians tend to rely on implicit vehicle movements to
make crossing decisions (Moore et al., 2019), external human-machine
interfaces (eHMIs) that utilize the vehicle’s exterior surface, its immediate
surroundings, and wearable devices have been found to support safe and
intuitive interactions (Dey et al., 2020).

While several methods exist to investigate AV–pedestrian interaction, the
development of VR in the past decade has paved the way for the increasing use
of immersive simulations. VR-based experiments obviate the following
limitations of real-world testing: 1) Building fully functional prototypes and
testing AVs in real traffic environments is expensive and time-consuming; 2)
Regulations require the presence of a test driver to monitor and directly
intervene if needed; and 3) Real-world study settings may entail considerable
physical risks to the participants. Beyond addressing these limitations, VR
simulators also offer great experimental control, enable researchers to
reproduce other scholars’ work easily, and produce a multitude of behavioral
data via the tracking system (Feng et al., 2020).

Popular simulation platforms, including screen-based setup, CAVE, and VR head-
mounted displays (HMD), differ in their levels of complexity and immersion.
Currently, researchers’ use of VR HMD is growing due to the technology’s high
level of immersion, improved ergonomics, increased performance, and lower
prices. While immersion is defined as a property of a VR system, presence is
the subjective psychological response of a user experiencing that system
(Slater and Wilbur, 1997). Presence thus constitutes a major consideration in
the development and validation of VR simulators. To a great extent, behavioral
and emotional reactions elicited in the virtual environment (VE) were
consistent with those in real-life situations (Deb et al., 2017), and the
average walking speed in VEs was found to match real-world norms (Deb et al.,
2017). Nevertheless, discrepancies between the real and virtual worlds remain
an issue. Without a real danger of physical injury, participants were more
inclined to exhibit risky behavior (Holländer et al., 2019a). In addition, it
was observed that participants overestimated vehicle speeds (Bhagavathula et
al., 2018) and accepted a lower time-to-contact when crossing in VR (Feldstein
and Dyszak, 2020). Technical limitations of VR such as a narrow field of view
and relatively low display resolution might also impact the crossings, making
it difficult for participants to spot oncoming vehicles and estimate
distances(Mahadevan et al., 2019). Additionally, various side effects are
associated with the use of VR, including discomfort, sickness, and other
adverse after-effects (Stanney et al., 2003). While these shortcomings limit
the generalization of obtained results, they neither invalidate the findings
nor dismiss the usefulness of VR as an investigation method.

Several studies have analyzed the current use of VR simulators in pedestrian
behavior research (Schneider and Bengler, 2020), the development of AV safety
(Nascimento et al., 2019) and AV external communication research (Colley et
al., 2019). In assessing 87 studies that employed pedestrian simulators,
Schneider and Bengler (Schneider and Bengler, 2020) focused on four main
characteristics: the research question, experimental task, technical setup,
and participant sample. The review covered pedestrian behavior studies with
and without AVs. Nascimento et al. (Nascimento et al., 2019) examined the
training of driving algorithms and the evaluation of user behavior in VEs.
Since the research aimed to holistically examine the role of VR in enhancing
the safety of automated driving technology, its analysis included only a small
number of AV–pedestrian interaction studies. In a recent literature review by
Colley et al. (Colley et al., 2019), the authors identified seven publications
and preprints reporting the use of VR for AV external communication research.
The simulators were analyzed based on their inherent advantages, such as
customizability and objective data collection. Each of these works contributes
to understanding VR potential as an evaluation method and provides
considerations for future studies.

Despite its contributions, the existing literature exhibits limitations. In
particular, the increasing popularity of VR simulators in AV–pedestrian
interaction research generates a vast number of traffic situations with
diverse settings, which impedes effort to compare findings across studies and
establish good implementation practices. It is, therefore, imperative to
review existing efforts and conduct a thorough analysis of investigated use
cases and scenarios, as well as evaluation methods. The contribution of our
work is threefold. Firstly, it summarizes the status quo. Secondly, it
provides a set of considerations for researchers and industry practitioners.
Thirdly, it highlights research gaps that are most pressing to further
understanding AV–pedestrian interaction and any associated safety issues.

##  2\. Approach

To perform an analysis of VR-based research on AV–pedestrian interaction, we
adopted a structured approach based on the methodological framework by Arksey
& O’Malley (Arksey and O’Malley, 2005).

###  2.1. Research questions

This review is guided by the following questions:

  1. (1)

What are the main use cases of AV–pedestrian interaction being investigated in
VR?

  2. (2)

How are the scenarios configured with respect to factors influencing
pedestrian behavior?

  3. (3)

What types of measures are used to evaluate AV–pedestrian interaction in VR?

###  2.2. Search strategies

####  2.2.1. Data sources

To identify relevant studies, we queried four research databases: ACM Digital
Library, IEEE Xplore Digital Library, ScienceDirect, and Google Scholar. ACM
Digital Library offers the most comprehensive collection of literature in
computing and information technology. IEEE Xplore Digital Library provides
access to high-quality technical publications in engineering and technology.
ScienceDirect has a wide range of interdisciplinary research, while Google
Scholar allows for a broad search across many disciplines and publishing
formats. We restricted the search to include publications dated between 2010
and 2020 considering the increasing adoption of VR pedestrian simulators
during this period (Schneider and Bengler, 2020). The last search date was
November 7, 2020.

####  2.2.2. Keywords

We selected four keywords that could represent the main concepts of our
research topic, namely “autonomous vehicles”, “pedestrians”, “interaction” and
“virtual reality”. We also included synonyms and related words in making the
search queries; however, abbreviations such as AV, VRU, HMI, VR have many
different associated definitions and were not used to avoid irrelevant
results.

####  2.2.3. Search procedure and results

We combined the keywords using AND/OR operators and performed the search
within the title, abstract, keywords, and full-text of each article. After
testing different keyword combinations, we selected the search query that
yielded the highest number of results. The exact keywords and queries for each
database can be found in Appendix A. The search yielded a total of 431 entries
(ACM = 29, IEEE = 41, ScienceDirect = 19, Google Scholar = 342). We imported
all research results to a spreadsheet to identify and remove 90 duplicate
entries. One entry was not accessible to our academic institution. As a
result, 341 publications remained.

###  2.3. Study selection

We chose conference proceedings and journal papers based on the following
criteria:

  * •

Published as original full papers.

  * •

Written in the English language.

  * •

Contained empirical studies with evaluation results.

  * •

Investigated AV–pedestrian interaction using VR as an evaluation method, with
the AV–pedestrian communication being either explicit or implicit.

We excluded preprints, studies whose focus was to report the development of a
VR simulator, and studies that offered little information to address the
research questions.

Since the number of studies found in our search was manageable, and the
selection criteria were straightforward, one reviewer took charge of screening
the publications. The process involved reading articles’ titles, abstracts,
and full-texts to eliminate those that did not meet the selection criteria. A
total of 31 publications was selected for the final analysis. A summary of
these publications is available in Appendix B.

Figure 1. A summary of different factors influencing pedestrian experience and
behavior.

###  2.4. Charting the data

Based on the research questions, we charted relevant information from each
included publication and kept the records in a spreadsheet. The captured
information was: author, year, title, VR system (platform), VR content type
(computer-generated or 360-degree real-world video, contextualized or
decontextualized), game engine, traffic scenario settings (see Fig. 1), and
evaluation measures.

##  3\. Results

This section presents the results from our analysis in a structure
corresponding to the research questions.

###  3.1. Overview

The analysis shows that immersive VR HMD was the system choice of all 31
reviewed studies. High-end headsets such as HTC Vive (22, 71%) and Oculus Rift
(3, 10%) were favored over the low-budget phone-based VR (1, 3%) since they
offer powerful PC-tethered performance, fully-developed motion controls, and
room-scale experience. Of note, five studies (16%) did not specify the HMD
model. In terms of content development, a large number of studies (30, 97%)
implemented contextualized scenarios. Nevertheless, one study (3%) (Lee et
al., 2019b) examined the comprehensibility of eHMIs in a decontextualized
environment. Thirty studies (97%) implemented computer-generated VR and one
study (3%) (Velasco et al., 2019) used a 360-degree real-world video.
Concerning 30 studies whose simulations were made of 3D models, the VEs were
consistently developed with Unity111https://unity.com/ (27, 87%). Only one
study (3%) (Chen et al., 2020) opted for the open-sourced
jMonkeyEngine222https://jmonkeyengine.org/ which was made specifically for
Java developers, and two studies (6%) (Colley et al., 2020a; Jayaraman et al.,
2018) did not mention the game engines used.

###  3.2. Critical use cases and scenarios

Various attempts have been made to define the most relevant use cases and
scenarios of AV–pedestrian interaction. For example, Wilbrink et al. (Wilbrink
et al., 2017) derived main use cases from workshops involving all project
partners. In determining the relevance of each use case, they considered
factors such as accident data, frequency of occurrence, the necessity of
interaction, and cross-cultural differences. The outcome included two
essential use cases concerning interactions in ambiguous situations,
specifically at crossings without traffic lights and in car parks. Utilizing a
different approach, Wang et al. (Wang et al., 2019) identified safety-critical
scenarios from a video data set capturing pedestrian behavior at unmarked mid-
block pedestrian crossings. The researchers subsequently classified scenarios
based on the number of AVs and pedestrians in the interaction.

To highlight the extent to which existing research has investigated critical
use cases and scenarios, we categorized 31 selected studies based on the right
of way (Wilbrink et al., 2017) and the number of AVs and pedestrians in the
interaction (Wang et al., 2019).

####  3.2.1. Right of way

All reviewed studies investigated unsignalized pedestrian crossings, except
for one study (3%) (Jayaraman et al., 2018) that examined the impact of both
signalized and unsignalized crosswalks on pedestrians’ trust and certainty.
Two studies (6%) (Fuest et al., 2019; Weber et al., 2019), included ambiguous
situations with undefined right of way, such as those in shared spaces and car
parks. The absence of crossing facilities, such as crosswalks, indicated that
AVs had the right of way in 17 studies (55%), while their presence suggested
that pedestrians had the right of way in 15 studies (48%). Importantly, in
countries such as Germany and the Netherlands, vehicles are obliged to yield
to pedestrians at crosswalks; therefore, unmarked roads were chosen to ensure
participants exercised caution (Ackermans et al., 2020; De Clercq et al.,
2019). However, in places with different social norms and road rules, a
crosswalk still represents an unsafe situation and requires participants to
watch out for oncoming vehicles (Deb et al., 2020).

####  3.2.2. The number of AVs and pedestrians in the interaction

Most studies (29, 94%) involved one-to-one interactions. In making the
crossing decision, participants only needed to observe one vehicle at a time.
Fifteen studies (48%) included multiple vehicles in traffic; however, the
AV–pedestrian interaction in these experiments remained one-to-one. For
example, in the study by Ackermans et al. (Ackermans et al., 2020), 20
vehicles appeared in one experimental block, a new vehicle appeared only after
the previous vehicle had been out of sight. Two studies (6%) (Colley et al.,
2020c; Mahadevan et al., 2019) required the participant to interact with
multiple vehicles while crossing. Specifically, the participant crossed a two-
lane street where vehicles approached from both lanes. Only one study (3%)
(Mahadevan et al., 2019) investigated the scenario of multiple vehicles -
multiple pedestrians, and there was no instance where multiple pedestrians
interacted with one vehicle.

###  3.3. Scenario configuration

Even though there are a few instances of using 360-degree videos, the most
common approach in VR simulation is to develop synthetic traffic scenarios.
The process of creating a scenario involves the configuration of many
different parameters, such as the number of vehicles and vehicle speed. Many
of these parameters are also factors that influence the decision-making
process of pedestrians when they cross the street. Rasouli and Tsotsos
(Rasouli and Tsotsos, 2020), from their review of studies on pedestrian
behavior, identified two groups of factors, namely pedestrian factors and
environmental factors. Fuest et al. (Fuest et al., 2017) derived from real
traffic situations a set of attributes that influence communication between
AVs and other road users (e.g., pedestrians, cyclists, and human drivers).
Meanwhile, Mahadevan et al. (Mahadevan et al., 2019), through a preliminary
design exercise and literature review, decided on 19 factors under four
categories: pedestrians, vehicles, traffic and street characteristics, and
interface prototypes in the development of a pedestrian–mixed-traffic
simulator. To analyze the scenario configuration of selected VR studies, we
synthesized factors relevant to AV–pedestrian interaction from the three
studies mentioned above (Rasouli and Tsotsos, 2020; Mahadevan et al., 2019;
Fuest et al., 2017), and grouped them into five categories using affinity
diagramming. In the process, we added “traffic type” to account for mixed
traffic (sae, 2021) and “sound,” an important factor in navigation and
decision making of the visually impaired (Koutsoklenis and Papadopoulos, 2011)
(see Fig. 1).

This section presents the result of our analysis concerning: (1) which factors
have been explored in the context of AV–pedestrian interaction and their
influence on pedestrian experience and behavior; and (2) how the values of
these factors are commonly specified in virtual traffic scenarios. We included
the authors’ explanations, where given, for selecting a particular value (see
Table 1).

Vehicle factors

_Vehicle appearance - Type and size:_ Depending on its type, a vehicle can be
categorized as small, medium, or large. In this paper, small refers to single-
seat or 2-seater vehicles; medium refers to 5-seater vehicles; and large
refers to public buses, shuttles, and trucks. We created this coarse-grained
classification of vehicle size based on the information from the reviewed
studies rather than the actual dimensions of each vehicle model. The impact of
vehicle sizes on AV–pedestrian interaction was investigated in three studies
(10%)(De Clercq et al., 2019; Deb et al., 2019; Camara et al., 2020). Clercq
et al. (De Clercq et al., 2019) found that larger vehicles were perceived as
less safe. However, the effect sizes were small for both yielding and non-
yielding vehicles. Vehicle sizes were found to significantly affect the
participant’s ability to comprehend eHMI messages (Deb et al., 2019).
Specifically, the large vehicle size may have taken more time for the
pedestrians to grasp the vehicle’s intentions. Only in one study (Camara et
al., 2020), did vehicle size not elicit any difference in pedestrian crossing
behavior. However, it is worth noting that the number of participants in this
study was low.

_Vehicle appearance - Color:_ White was the most common vehicle color,
possibly because it makes the vehicle more noticeable in VR (Mahadevan et al.,
2019). Nevertheless, whether a vehicle’s color may affect its visibility and
hence influence pedestrian behavior was not explored in any reviewed studies.
This factor might be relevant to consider in VR simulations, where light and
color rendering is usually less realistic.

_Vehicle appearance - Design:_ A few studies tested friendly and futuristic
vehicle models, such as Citroën C-Zero (Holländer et al., 2019a, b), Waymo
Firefly (Hudson et al., 2018; Deb et al., 2018, 2019, 2020), Mercedes-Benz F
015 (Löcken et al., 2019), Easymile EZ10 and similar AV shuttles (Böckle et
al., 2017; Stadler et al., 2019; Velasco et al., 2019). Their futuristic
designs might indicate the vehicles’ autonomous capability and help
distinguish them from human-driven vehicles. Furthermore, conspicuous sensor
systems mounted on top of an AV and eHMIs attached to its front made the
vehicle’s autonomous nature more pronounced. Ackermans et al. (Ackermans et
al., 2020) found that such a conspicuous appearance led to an increased
willingness to cross by pedestrians who held a negative perception of AVs.

_Automation level:_ By comparing pedestrian behavior when interacting with two
types of vehicles – manually driven and fully autonomous – Chen et al. (Chen
et al., 2020) found a tendency of pedestrians to make crossing decisions based
on their legacy strategy of gap acceptance, even if the vehicles they
encountered were autonomous and equipped with eHMI displaying assistance
information. Velasco et al. (Velasco et al., 2019) also reported no difference
in pedestrians’ crossing intention; however, participants who recognized the
automated-driving capability of the vehicle showed a higher level of trust in
automation. It is worth noting that the higher trust scores somewhat
contradicted the lower intention to cross; therefore, it was probable that the
participants responded to the trust questionnaire while thinking about a
future version of AVs instead of the currently available version (Velasco et
al., 2019). Only in one study (3%) (Mahadevan et al., 2019), a semi-AV was
included in mixed traffic, but without an eHMI, the vehicle’s automation level
was ambiguous to the pedestrian participants. The study highlighted the
potential problem of pedestrians’ inability to assess who is in charge of the
vehicle’s operation and suggested that semi-AVs may need to communicate their
driving statuses to other road users.

_Driver behavior:_ Two studies (6%) (Deb et al., 2020; Hudson et al., 2018)
set out to identify the effect of AV operator statuses on pedestrian behavior.
Studies by Deb et al. (Deb et al., 2020) and Hudson et al. (Hudson et al.,
2018) reported no significant differences in external interface ratings
between no-operator status and attentive-operator status. However, the
presence of a distracted operator on a fully automated vehicle (SAE level 5,
(sae, 2021)) negatively affected the participants’ perception of safety and
lowered their ratings of the interface.

_Driving direction:_ There was no instance of backward-driving behavior even
though a vehicle reversing into a car park was considered one of the must-have
use cases because of the high number of accidents related to this situation
(Wilbrink et al., 2017). However, since the reversing maneuvers could be
signaled by reversing lights and audible reversing alarms, many researchers
considered this use case not as critical (Kaß et al., 2020).

_Vehicle speed:_ The effect of vehicle speed on pedestrian crossing behavior
was investigated in four studies (13%) (Velasco et al., 2019; Lee et al.,
2019a; Deb et al., 2019; Othersen et al., 2018). When participants interacted
with vehicles driving at a slower speed, they had a lower intention to cross
(Velasco et al., 2019) and made fewer crossings (Lee et al., 2019a). These
findings could probably be explained by the fact that for the same time gap,
vehicles traveling faster provided a larger distance gap (Lee et al., 2019a).
However, it is essential to note that a car traveling at higher speeds induced
a lower safety margin (Lee et al., 2019a) and when it started to decelerate,
pedestrians took longer to initiate the crossing (Othersen et al., 2018) and
crossed less often (Lee et al., 2019a). In a study by Deb et al. (Deb et al.,
2019) the impact of speed was examined in combination with distance. The
results showed that while the adults exhibited normal behavior, the children
made risky and hasty crossings when the AV was driving at high speed and the
gap was narrow.

_Intention to yield:_ In two studies (6%) (Chen et al., 2020; Stadler et al.,
2019), the vehicle would never yield its right of way. Participants were told
to identify a safe gap to cross (Chen et al., 2020) or were given sufficient
time to cross a single lane of traffic (Stadler et al., 2019). In most
scenarios (22, 71%), the AV could either yield to oncoming pedestrians or
continue driving. The unpredictable behavior prompted participants to pay
attention to vehicle kinematics or external car displays. In a smaller number
of studies (5, 16%), the AV decelerated in every trial. This setup was chosen
because the vehicle had to comply with road traffic regulations (Fuest et al.,
2019), the proposed eHMIs indicated safe crossing conditions only (Deb et al.,
2018; Colley et al., 2020c), or different yielding behavior (Pillai, 2017) and
the impact of eHMIs in communicating yielding intent (Böckle et al., 2017)
were the key parameters under inquiry in the research. However, a potential
problem of this conservative behavior of AVs, as mentioned by Deb et al. (Deb
et al., 2018), is that participants might expect the car to always stop for
them, which might encourage them to step onto the road immediately instead of
acting cautiously.

_Driving behavior:_ Typically, AVs were designed to maintain a certain speed
or yield to pedestrians following a predetermined deceleration curve. However,
in seven studies (23%), different driving behaviors were included to explore
how pedestrians inferred intentions from vehicle kinematics and their
expectations of AV driving behavior. Results showed that an early deceleration
was able to communicate an AV’s intention of giving way (Fuest et al., 2019),
reduced the Crossing Initiation Time to Vehicle Stop (CITVS) (Dietrich et al.,
2020, 2019) and provided high levels of comfort to pedestrians (Pillai, 2017).
Yielding early and slowly to pedestrians may also be a good approach to
increase traffic efficiency in crossing situations since the AV might be able
to accelerate again without making a complete stop (Dietrich et al., 2019). In
contrast, late and abrupt braking was perceived as aggressive driving and
decreased trust in AV technology (Jayaraman et al., 2018). Atypical vehicle
trajectories could even lead to mistrust (Schmidt et al., 2019). When
interacting with the AV, participants related its driving behaviors to their
own or other people’s actions (Pillai, 2017). It was found that people could
derive social cues from different vehicle trajectories [50] and expected the
AV to drive like a reasonable human driver [47]. Notably, in the study by
Camara et al. (Camara et al., 2020), participants interacted with a game-
theoretic AV whose movement was adapted in real-time based on their positions.
The authors reported that different participants displayed different
preferences for AV algorithm parameters.

_Communication:_ The impact of eHMIs compared to no eHMI was explored in many
studies (18, 58%). In general, even though people still rely on vehicle
kinematics to make crossing decisions (Chen et al., 2020), they expressed
preferences for AVs to be equipped with external interfaces (Deb et al.,
2020). Of note, older adults found the external interfaces more useful
compared to their younger counterparts (Deb et al., 2020). Regarding
subjective experience, eHMIs were found to provide pedestrians with a higher
level of comfort (Mahadevan et al., 2019; Böckle et al., 2017), perceived
safety (De Clercq et al., 2019; Böckle et al., 2017; Velasco et al., 2019;
Chang et al., 2017), and trust (Colley et al., 2020a). When a vehicle
explicitly communicated its intent to yield, this increased pedestrians’
crossing intention (Kooijman et al., 2019) and willingness to cross (Ackermans
et al., 2020). The users were able to make quicker crossing decisions (Chang
et al., 2017; Stadler et al., 2019; Holländer et al., 2019a) as well as
initiating crossings earlier (Mahadevan et al., 2019; Dietrich et al., 2020;
Othersen et al., 2018). Furthermore, a study by Deb et al. (Deb et al., 2018)
reported that the inclusion of an external interface led to a significant
improvement in pedestrian’s receptiveness to AVs. Nevertheless, Weber et al.
(Weber et al., 2019) recommended the use of eHMIs in indicating safe crossing
situations only, since a non-yielding intent might lead to more incorrect
interpretations of the vehicle’s intention.

_Longitudinal distance:_ This factor may influence objective safety and affect
the timing of explicit communication, especially in the case of short distance
(Fuest et al., 2017). In the study by Deb et al. (Deb et al., 2019), two
distances were selected for every speed limit to present one risky situation
(one that required hard braking) and one safe situation. The results showed
that children took a longer time to cross the street during trials with wider
gaps, whereas they rushed across the road during trials with higher speeds and
narrower gaps. Koojiman et al. (Kooijman et al., 2019) reported that the eHMI
had the strongest effect on pedestrians’ forward velocity when they crossed
through a 20-meter gap (as opposed to a 30-meter gap).

_Lateral distance:_ The setup for most studies (22, 71%) was to have
pedestrians standing at the curb of the pavement and the AV approaching from
the nearest lane. In only three studies (10%) (Hudson et al., 2018; Deb et
al., 2018, 2020), the AV traveled from the farthest lane.

Traffic factors

_Number of vehicles:_ In about half of the reviewed studies (15, 48%),
participants encountered one vehicle in the scenario. However, many
publications (15, 48%) reported a setup in which multiple vehicles were
driving past the participants’ location. In ten studies (32%), participants
could only cross between vehicles after one or several vehicles had passed
(Chen et al., 2020; Dietrich et al., 2019, 2020; Othersen et al., 2018; De
Clercq et al., 2019; Lee et al., 2019a; Deb et al., 2019, 2020; Hudson et al.,
2018; Kooijman et al., 2019). According to Chen et al. (Chen et al., 2020), by
giving participants sufficient opportunity to prepare and adapt to the
context, the setup could enhance the ecological perception of the following
trial vehicle(s) and ensure that participants provide natural responses.

_Gap between vehicles:_ One critical aspect concerning the inclusion of
multiple vehicles in the scenario is the (time or distance) gap between
vehicles since this factor was found to influence pedestrians’ crossing
decision. It is important to create a setup in which the participants cannot
determine which vehicle will yield or provide a passable gap (Dietrich et al.,
2020) so that they act as they would in a real-world traffic situation (Deb et
al., 2020). Therefore, many studies varied the gap between vehicles in a fleet
(Chen et al., 2020; De Clercq et al., 2019; Lee et al., 2019a; Othersen et
al., 2018) or designed multiple experimental conditions with different gaps
(Deb et al., 2019; Dietrich et al., 2020, 2019; Kooijman et al., 2019). In our
analysis, three studies (10%) (Deb et al., 2020; Hudson et al., 2018;
Mahadevan et al., 2019) reported impassable gaps between vehicles in the same
convoy. These gaps were insufficient for the participants to make a safe
crossing, which prompted them to look for deceleration cues or an explicit
intention to yield in the approaching vehicles.

_Traffic direction:_ Most interaction scenarios involved traffic moving on
one-way streets (27, 87%). Two-way traffic was implemented in only three
studies (10%) (Hudson et al., 2018; Colley et al., 2020c; Deb et al., 2020).
Koojiman et al. (Kooijman et al., 2019) suggested the testing of eHMIs in
bidirectional traffic situations where participants have to divide their
attention.

_Traffic type:_ The analysis found two studies (6%) (Mahadevan et al., 2019;
Chen et al., 2020) examined AV–pedestrian interaction in mixed traffic. These
efforts simulated two automation levels at a time. Chen et al. (Chen et al.,
2020) reported that a mixed-traffic environment of human-driven vehicles and
fully automated vehicles did not affect pedestrian behavior. Mahadevan et al.
(Mahadevan et al., 2019) also did not find the influence of traffic
composition on pedestrian crossing strategy to be statistically significant.

Environment factors

_Street scene:_ While most studies (29, 94%) opted for a (Western) urban road
environment, Mahadevan et al. (Mahadevan et al., 2019) enabled two types of
street scenes in their simulator: urban and rural. However, it remains unclear
how these different environments might influence pedestrian behavior. Camara
et al. (Camara et al., 2020) examined AV–pedestrian interaction using a narrow
pathway in the park and a wide tarmac road, but the reported findings were
still preliminary due to the low number of participants. Regarding the use of
decontextualized VEs, while the absence of context might cause the eHMIs to be
interpreted differently in the real world, it offers a fast and cost-effective
method to investigate the intuitiveness of eHMIs’ colors and patterns in
isolation from other factors (Lee et al., 2019b).

_Street delineation:_ Traffic lights and zebra crossings are essential for the
safe coexistence of people and vehicles. In four studies (13%) (Velasco et
al., 2019; Jayaraman et al., 2018; Fuest et al., 2019; Weber et al., 2019),
the effect of these facilities was investigated in detail. Jayaraman et al.
(Jayaraman et al., 2018) found that signalized crossings increased the trust
in AVs and moderated the negative impact of aggressive driving behavior. At
crossings, pedestrians also showed a higher intention to cross (Velasco et
al., 2019). Nevertheless, different rights of way did not elicit any
differences in intention recognition time (IRTs) when pedestrians inferred an
AV’s intent from its movement (Fuest et al., 2019). When the AVs are equipped
with eHMIs, the effect of priority given to pedestrians’ correct
interpretation, the certainty of choice, and IRTs varied depending on the
cultural setting (Weber et al., 2019).

_Street width:_ A great majority of studies (18, 58%) did not report street
width, even though street width has been reported to impact the level of
crossing risk (Rasouli and Tsotsos, 2020). Multi-lane streets are common in
cities, and several study participants raised their concerns about whether the
eHMIs displaying explicit instruction to act (e.g., a projected zebra
crossing) would take into consideration other vehicles in other lanes (Löcken
et al., 2019).

_Road structure:_ In two studies (6%) (Fuest et al., 2019; Weber et al.,
2019), shared space and a car park were selected to represent traffic
situations with undefined right of way. The remaining studies investigated
AV–pedestrian interaction at mid-block crossings (19, 61%) or intersections
(9, 29%). Since intersections have a higher number of conflict points, we
argue that this type of road structure will play an important role in studies
that examine traffic scenarios with a scaled-up number of vehicles and
pedestrians.

_Weather, road conditions and lighting conditions:_ One study (3%) (Pillai,
2017) investigated pedestrians’ understanding of AV driving behavior under two
contrasting visibility conditions: a sunny day and night-time rain and fog.
The findings revealed that participants felt less comfortable in adverse
weather conditions, tended to act more cautiously, and relied more on audio
cues to assess the situation.

Social factors

_Group size:_ This aspect refers to the number of pedestrians interacting with
the AV in the scenario and does not include background people, such as those
standing idly or chatting on the pavement. One study (3%) (Mahadevan et al.,
2019) included in the traffic scenarios several AI-based virtual pedestrians
of varying demographics (gender, ethnicity, and age) who would cross the
street together with the study participants.

_Group behavior:_ Mahadevan et al. [14] configured two crossing behaviors for
the virtual pedestrians, including early crossers (crossing when the vehicles
start to slow down) and timely crossers (crossing when the vehicles are almost
at a complete stop). The findings indicated that the impact of group influence
on behavior was not statistically significant.

Sound factors

_Engine sound:_ Audio cues from approaching vehicles are essential not only
for people with full or partial losses of eye-sight but also for those who use
smartphones while walking. Yet, only three studies (10%) (De Clercq et al.,
2019; Colley et al., 2020c; Pillai, 2017) mentioned the implementation of
driving sounds. In one study (3%) (Colley et al., 2020c), an Acoustic Vehicle
Alerting System designed for electric vehicles was utilized.

_Ambient sound:_ The urban soundscape (i.e., natural sounds and human-produced
sounds) was described in nine studies (29%). Notably, the sound effect of
traffic lights was simulated in one study (3%) (Colley et al., 2020c), which
included visually impaired individuals as participants.

Table 1. The impact of different factors on pedestrian experience and behavior

[t]  _Factor_ _Studies 1_ _Impact on pedestrians_ _Values 2_ _# Cases 3_
_Notes_ Vehicle factors Appearance: type-size (De Clercq et al., 2019; Deb et
al., 2019) Perceived safety (De Clercq et al., 2019) Car (small, medium) 28
(90%) Cars and light trucks accounted for most of pedestrians’ fatalities (Deb
et al., 2019) (Camara et al., 2020) Understanding of messages (Deb et al.,
2019) Bus/Shuttle (large) 5 (16%) Light truck (large) 2 (6%) Heavy truck
(large) 1 (3%) 1-person pod (small) 1 (3%) Appearance: color White/Silver 14
(45%) White could be used to make the vehicle easier to spot in VR (Mahadevan
et al., 2019) Other 7 (23%) Black/Dark 6 (19%) Red 5 (16%) Unspecified 4 (13%)
Appearance: design (Ackermans et al., 2020) Willingness to cross (Ackermans et
al., 2020) External interface 24 (77%) Sensor mounted on top of the vehicle
helps to distinguish AVs from conventional vehicles (Chen et al., 2020)
Futuristic design 9 (29%) Normal appearance 7 (23%) Conspicuous sensor 3 (10%)
Automation level (Velasco et al., 2019; Chen et al., 2020) Trust (Velasco et
al., 2019) Fully autonomous 31 (100%) Manually-driven 3 (10%) Semi-autonomous
1 (3%) Driver behavior (Deb et al., 2020; Hudson et al., 2018) Perceived
safety (Deb et al., 2020; Hudson et al., 2018) No driver 23 (74%) Unspecified
6 (19%) Attentive 5 (16%) Distracted 4 (13%) Driving direction Forward 29
(94%) Not applicable 2 (6%) Vehicle speed (Velasco et al., 2019; Deb et al.,
2019) Crossing intention (Velasco et al., 2019) 30–50 km/h 22 (71%) 50 km/h -
Singapore’s speed limit regulations for one lane roads(Stadler et al., 2019)
(Lee et al., 2019a; Othersen et al., 2018) Crossing time (Deb et al., 2019)
Less than 30 km/h 4 (13%) Crossing decision (Lee et al., 2019a) More than 50
km/h 4 (13%) Safety margin (Lee et al., 2019a) Unspecified 4 (13%) CITVS
(Othersen et al., 2018) 0 km/h 2 (6%) Intention to yield Vary 22 (71%) A 5-sec
gap acceptance was implemented so that participants could cross without the AV
slowing down (Stadler et al., 2019) Always yield 5 (16%) Never yield 2 (6%)
Not applicable 2 (6%) Driving behavior (Jayaraman et al., 2018; Fuest et al.,
2019) Intention recognition (Fuest et al., 2019) Unspecified 18 (58%) Selected
deceleration rates reflected normal braking in previous research (Ackermans et
al., 2020; Pillai, 2017) or were suggested by the UK Department for Transport
(Lee et al., 2019a) (Camara et al., 2020; Pillai, 2017) CITVS (Dietrich et
al., 2020, 2019) Specified 9 (29%) (Dietrich et al., 2020, 2019; Schmidt et
al., 2019) Trust (Jayaraman et al., 2018; Schmidt et al., 2019) Not applicable
4 (13%) Comfort (Pillai, 2017) Communication (Holländer et al., 2019a;
Mahadevan et al., 2019) Comfort (Mahadevan et al., 2019; Böckle et al., 2017)
Visual eHMI 23 (74%) (Velasco et al., 2019; Chen et al., 2020; Colley et al.,
2020a) Perceived safety (De Clercq et al., 2019; Chang et al., 2017; Böckle et
al., 2017; Velasco et al., 2019) Auditory eHMI 9 (29%) (Weber et al., 2019;
Ackermans et al., 2020; De Clercq et al., 2019; Deb et al., 2020) Trust
(Colley et al., 2020a) Haptic eHMI 1 (3%) (Hudson et al., 2018; Deb et al.,
2018) Intention recognition (Weber et al., 2019) Implicit 8 (26%) (Böckle et
al., 2017; Stadler et al., 2019) CITVS (Dietrich et al., 2020; Mahadevan et
al., 2019; Othersen et al., 2018) (Dietrich et al., 2020; Othersen et al.,
2018) Crossing intention (Kooijman et al., 2019) (Chang et al., 2017; Kooijman
et al., 2019; Gruenefeld et al., 2019) Willingness to cross (Ackermans et al.,
2020) Decision time (Chang et al., 2017; Stadler et al., 2019; Holländer et
al., 2019a) AV receptivity (Deb et al., 2018) Longitudinal distance (Deb et
al., 2019; Kooijman et al., 2019) Crossing time (Deb et al., 2019) More than
10 m 20 (65%) Forward velocity (Kooijman et al., 2019) Unspecified 9 (29%) Not
applicable 2 (6%) 3–10 m 1 (3%) Lateral distance Nearest lane 22 (71%) The AV
approached from the farthest lane to avoid possible AV–pedestrian collision
(Deb et al., 2018) Unspecified 4 (13%) Farthest lane 3 (10%) Not applicable 2
(6%)

  * 1

Studies included the factor as an independent variable.

  * 2

_Not applicable_ refers to studies in which the factor does not exist by
design (e.g., decontextualized scenario, single vehicle).

  * 3

The total number of cases might be greater than the number of selected papers
because a study might include multiple vehicles or multiple scenarios.

Table I (continued) |  |  |  |  |
---|---|---|---|---|---
_Factor_ |  _Studies_ |  _Impact on pedestrians_ |  _Values_ |  _# Cases_ |  _Notes_
Traffic factors |  |  |  |  |
Number of vehicles |  |  |  One vehicle |  15 (48%) |  The absence of other vehicles was to ensure pedestrians’ decision free from environmental distractions (Deb et al., 2018)
|  |  |  Multiple vehicles |  15 (48%)
|  |  |  Not applicable |  1 (3%)
Gap between vehicles |  |  |  Not applicable |  16 (52%)
|  |  |  Specified |  12 (39%) |
|  |  |  Unspecified |  3 (10%) |
Traffic direction |  |  |  One-way |  27 (87%) |
|  |  |  Two-way |  3 (10%) |
|  |  |  Not applicable |  1 (3%) |
Traffic type |  (Mahadevan et al., 2019; Chen et al., 2020) |  |  Homogeneous |  29 (94%) |
|  |  |  Mixed traffic |  2 (6%) |
|  |  |  Not applicable |  1 (3%) |
Environment factors |  |  |  |  |
Street scene |  (Camara et al., 2020) |  |  Urban |  29 (94%) |  An urban setting was selected because of the high rate of accidents between vehicles and pedestrians (Pillai, 2017)
|  |  |  Tarmac road |  1 (3%)
|  |  |  Park (garden) |  1 (3%)
|  |  |  Not applicable |  1 (3%) |
Street delineation |  (Velasco et al., 2019; Jayaraman et al., 2018) |  Crossing intention (Velasco et al., 2019) |  Unsignalized |  30 (97%) |  In countries where vehicles are obliged to yield to pedestrians at crosswalks; unmarked roads were chosen to ensure participants exercise caution (Ackermans et al., 2020; De Clercq et al., 2019)
|  (Fuest et al., 2019; Weber et al., 2019) |  Trust(Jayaraman et al., 2018) |  Signalized |  1 (3%)
|  |  |  Unmarked road |  17 (55%)
|  |  |  Marked road |  15 (48%)
|  |  |  Not applicable |  1 (3%) |
Street width |  |  |  Unspecified |  18 (58%) |  The configuration was based on the country’s standards, e.g., 3.5 m single lane road in the UK (Lee et al., 2019a) and Germany (Löcken et al., 2019)
|  |  |  Two-lane |  8 (26%)
|  |  |  One-lane |  3 (10%)
|  |  |  Four-lane |  1 (3%)
|  |  |  Not applicable |  1 (3%) |
Road structure |  |  |  Mid-block |  19 (61%) |
|  |  |  Intersection |  9 (29%) |
|  |  |  Parking/Shared space |  2 (6%) |
|  |  |  Unspecified |  2 (6%) |
|  |  |  Not applicable |  1 (3%) |
Road condition |  |  |  Normal |  30 (97%) |
|  |  |  Wet |  1 (3%) |
|  |  |  Not applicable |  1 (3%) |
Weather |  (Pillai, 2017) |  Comfort(Pillai, 2017) |  Normal |  29 (94%) |  The extreme situation served to explore how pedestrians interact with approaching vehicles in a poor visibility condition(Pillai, 2017)
|  |  Crossing decision (Pillai, 2017) |  Cloudy |  1 (3%)
|  |  |  Rainy/Foggy |  1 (3%)
|  |  |  Not applicable |  1 (3%)
Lighting condition |  (Pillai, 2017) |  Comfort(Pillai, 2017) |  Photopic (daylight) |  29 (94%) |  A dusk scenario was selected to ensure visibility of the projected display concepts (Nguyen et al., 2019)
|  |  Crossing decision (Pillai, 2017) |  Mesopic (twilight) |  1 (3%)
|  |  |  Scotopic (night) |  1 (3%)
|  |  |  Not applicable |  1 (3%)
Social factors |  |  |  |
Group size |  (Mahadevan et al., 2019) |  |  One person |  30 (97%) |
|  |  |  Multiple pedestrians |  1 (3%) |
|  |  |  Not applicable |  1 (3%) |
Group behavior |  (Mahadevan et al., 2019) |  |  Not applicable |  30 (97%) |
|  |  |  Early crosser |  1 (3%) |
|  |  |  Timely crosser |  1 (3%) |
Sound factors |  |  |  |  |
Engine sound |  |  |  Unspecified |  26 (84%) |
|  |  |  Included |  3 (10%) |
|  |  |  Not included |  1 (3%) |
|  |  |  Not applicable |  1 (3%) |
Ambient sound |  |  |  Unspecified |  19 (61%) |  Background noise was included to increase the participants’ sense of presence (Holländer et al., 2019b)
|  |  |  Included |  9 (29%)
|  |  |  Not included |  2 (6%)
|  |  |  Not applicable |  1 (3%)

###  3.4. Evaluation measures

Responding to the third research question, this section classifies the
measures used to evaluate AV–pedestrian interaction (see Table 2) and
discusses prevalent evaluation methods in VR.

Table 2. Evaluating AV–pedestrian interaction in VR

[t]  _Main categories_ _Subcategories_ _# Cases 1_ Measurement Both 26 (84%)
Subjective only 4 (13%) Objective only 1 (3%) Measure Questionnaire/Rating 28
(90%) HMD-logged data 21 (68%) Interview 17 (55%) Controller-input data 7
(23%) Video/VR recording 5 (16%) Direct observation 2 (6%) Motion suit 1 (3%)
Experimental task Cross the street 23 (74%) Indicate response 8 (26%)
Evaluated aspect2 Crossing behavior 24 (77%) Comprehension 12 (39%) Perceived
safety 12 (39%) Preference 8 (26%) Trust 8 (26%) Comfort/Affect 5 (16%) Mental
workload 4 (13%)

  * 1

The total number of cases might be greater than the number of selected papers
because a study might employ various measures.

  * 2

The list is not exhaustive. We do not include rarely used metrics (with 1–2
occurrences).

####  3.4.1. Experimental tasks

In eight studies (26%), participants indicated their responses verbally
(Velasco et al., 2019) or using a controller (Chang et al., 2017; Fuest et
al., 2019; Lee et al., 2019b; Mahadevan et al., 2019; Ackermans et al., 2020;
De Clercq et al., 2019; Weber et al., 2019). The remaining studies implemented
naturalistic walking, which is essential for pedestrians to display more
realistic responses and for researchers to extract behavioral data (De Clercq
et al., 2019). However, in addition to ample physical space, this experimental
task requires a careful VE design to ensure participants’ safety and enhance
spatial presence. For example, to minimize the risk of collision between
participants and objects in the physical world, researchers using HMD could
implement a virtual blue grid showing the limits of the walking area (Dietrich
et al., 2020) or conceal physical barriers of the real world with
inconspicuous virtual object (Holländer et al., 2019a, b). In terms of
embodiment, allowing the users to see their feet and hands might enable a
full-body ownership illusion (Petkova and Ehrsson, 2008; Kooijman et al.,
2019), whereas providing tactile feedback as participants step down from the
sidewalk is expected to yield a more compelling sense of presence than does
the current flat-floor setup (Kooijman et al., 2019).

Naturalistic walking is particularly valuable when researchers seek insights
into pedestrian crossing behavior. If the objective is to evaluate the clarity
of eHMIs only, an immersive high-fidelity setup may not be required (Kooijman
et al., 2019). Furthermore, HMDs can display either computer-generated content
or 360-degree videos. While the former evokes more realistic physiological
responses, the latter elicit better psychological responses despite its
limited interactivity (Higuera-Trujillo et al., 2017). Thorough research into
the application of 360-degree videos in AV–pedestrian interaction could be
helpful for researchers who seek to improve their prototyping process.

####  3.4.2. Evaluated aspects

Various behavioral data were collected through video recordings, direct
observations, and tracking of pedestrian positions and head movements. General
reactions (e.g., hesitating before crossing) and objective metrics (e.g.,
waiting time) allowed the experimenters to assess the efficacy of the designed
interaction and gain insights into pedestrians’ decision-making process. Due
to studies employing a wide variety of metrics, the definition and calculation
of seemingly similar metrics might nevertheless vary from one study to
another. For example, while Locken et al. (Löcken et al., 2019) calculated the
crossing time from the time the AV started braking to the time the pedestrian
reached the opposite sidewalk, Deb et al. (Deb et al., 2019) chose to start
the timer when the participant initiated the crossing.

Pedestrians’ attitudes and experiences were generally evaluated with
consistent instruments. For instance, Likert scales were primarily utilized to
measure perceived safety and comfort, while the NASA-TLX (Hart and Staveland,
1988) and Self-Assessment Manikin (Bradley and Lang, 1994) were reliably
validated to assess workload and emotions, respectively. However, no consensus
exists regarding the appropriate instruments to measure trust. While the
simplicity of Likert scales [37], [48], [58] makes them easy to administer,
they examine only a particular aspect of trust and, therefore, have limited
validity. Standardized questionnaires can evaluate multi-dimensional
constructs; however, different studies (Colley et al., 2020c, a; Jayaraman et
al., 2018; Löcken et al., 2019; Velasco et al., 2019) employed different trust
questionnaires, resulting in limited comparability across results.

##  4\. Discussion

Overall, the reviewed studies focused on facilitating one-to-one AV–pedestrian
interaction in ambiguous traffic situations where pedestrians could not
predict AV intent, i.e., in unsignalized crosswalks. After considering myriad
factors, researchers have identified vehicle-related aspects as a major
influence on pedestrian experience and behavior. External communication
research accounted for many studies, but the efficacy of the design concepts
has been restricted primarily to traffic scenarios with typical environmental
conditions and uncomplicated communication relationships. With regard to
evaluation measures, the high experimental control of VR allows rich
behavioral data to be collected, but these objective measures need to be
obtained more consistently.

In this section, we reflect on the results and discuss research gaps and
avenues for future research. Additionally, we propose relevant considerations
for examining AV–pedestrian interaction in VR, the need for standardization
and benchmarking across VR-based studies, and the limitations of this review.

###  4.1. Research gaps

The great potential of VR pedestrian simulators lies in their ability to
develop complex and high-risk traffic scenarios that are otherwise difficult
to realize via real-world testing. In addition, VR enables the prototyping of
design solutions that involve drastic changes in urban infrastructure (e.g.,
smart roads). Therefore, in the following parts, we highlight research gaps in
AV–pedestrian interaction, which are crucial to address and can be explored
with a virtual test-bed.

####  4.1.1. Scalability

Many researchers have expressed their concerns about: (1) the possibility of
external displays to impose high cognitive load onto pedestrians (Mahadevan et
al., 2019; Robert Jr, 2019; Moore et al., 2019); and (2) the need to select an
optimal communication strategy for AVs to avoid misinterpretation when
pedestrians with different rights of way are within the vehicle’s proximity
(Dey et al., 2020; Robert Jr, 2019). These scalability-associated problems
are, by no means, exhaustive and they are far from being resolved. At the
early stage of AV–pedestrian interaction research, a majority of the studies
have focused on single pedestrian and single AV scenarios, while very few have
attempted scalability testing (Colley et al., 2020b). Nevertheless, as the
number of eHMI design concepts introduced in academia and industry continues
to increase (Dey et al., 2020), the capability of eHMIs to cope with complex
traffic situations in which multiple AVs and pedestrians cross paths has now
become one of the most critical aspects determining its prevalence.

####  4.1.2. Mixed traffic

Mixed traffic, consisting of vehicles with different levels of automation
(sae, 2021), requires further attention for several reasons. First,
pedestrians might need to gauge the intention of various vehicle types (i.e.,
conventional vehicles, semi-AVs, AVs) and operator statuses (i.e., no driver,
attentive driver, distracted driver), which can be mentally demanding. Second,
some AV–pedestrian communication solutions, such as the “omniscient narrator”
concept in which one vehicle communicates on behalf of other vehicles (Colley
et al., 2020c), seem improbable in mixed traffic. However, mixed traffic
research remains relatively underexplored in the literature. In our review,
only two studies (Mahadevan et al., 2019; Chen et al., 2020) examined the
impact of mixed traffic on pedestrian experience and behavior, and these
efforts were limited to simulating two automation levels at a time.

####  4.1.3. Environmental conditions

To maximize the potential impacts of AVs on pedestrian safety, car
manufacturers continue to develop AVs’ capabilities to drive in inclement
weather and at night. By contrast, VR studies of AV–pedestrian interaction
have mostly been conducted in temperate weather conditions during daytime
hours. By including varying environmental conditions, future studies could
generate additional design requirements and provide insights for developing
more holistic and contextual solutions. Furthermore, pedestrians have been
found to rely mainly on vehicle motion patterns in making crossing decisions
(Moore et al., 2019). Hence, scenarios where vehicle movement becomes
difficult to observe will yield more insights into the relevance of eHMI
solutions.

####  4.1.4. Vehicle behavior in VR

Vehicles can be controlled by human participants in coupled or distributed
simulators (Bazilinskyy et al., 2020; Sadraei et al., 2020). In our analysis,
however, all selected studies utilized the approach of programming vehicle
behavior in VR. Through ray casting, a vehicle can detect any objects ahead of
it and respond accordingly (Dalipi et al., 2020). Nguyen et al. (Nguyen et
al., 2019), for instance, implemented a method to determine a vehicle’s
driving behavior based on the presence of a human object in its visual field.
To a considerable extent, these conditional instructions enable effective
simulation of vehicle behavior; however, their simplicity does not allow real-
time, back-and-forth interactions between AVs and pedestrians. Out of 31
reviewed studies, only one (Camara et al., 2020) utilized a game-theoretic
model to continuously adapt the vehicle’s driving behavior to participants’
movement. Findings from this study suggested that mathematical models
determining AV driving behavior could learn from real-world human inputs while
adding value to VR pedestrian simulators. By allowing ongoing dynamic
interaction between AVs and pedestrians, future research could investigate the
negotiating process common among road users today while also enhancing
simulation realism and participant’s sense of presence.

###  4.2. Considerations

Based on our analysis of factors, we present five considerations to synthesize
the observations from the reviewed studies in the form of actionable key
takeaways. These considerations should be taken into account when developing
future AV–pedestrian simulation studies.

####  4.2.1. Simulating human-like driving behavior

Vehicle kinematics remains the key contributing factor to pedestrian
experience and behavior. In many studies, people stated their preferences for
human-like vehicle driving behaviors (Pillai, 2017) that conform to social
expectations (Schmidt et al., 2019); therefore, it is essential for external
communication research to replicate these behaviors accurately. Several
studies have referred to local regulations and guidelines on speed limits
(Stadler et al., 2019; Velasco et al., 2019) and stopping distances (Lee et
al., 2019a). We suggest the use of naturalistic driving data (Ackermans et
al., 2020; Pillai, 2017) or a tested driving profile that takes into
consideration both the traffic context and pedestrians’ perception of safety.

####  4.2.2. Developing appropriate traffic complexity

To enable participants to sustain their attention on the trial AV and make
crossing decisions based solely on the installed communication features or
vehicle kinematics, studies should consider eliminating visual distractions
that might be caused by other vehicles (Deb et al., 2020). This could be
achieved by excluding ambient traffic (Deb et al., 2018; Holländer et al.,
2019b) or ensuring that the study participants interact with the trial car
only when other vehicles are no longer present in the scene (Deb et al., 2020;
Ackermans et al., 2020). If, however, the study objective is to explore the
efficacy of the design solutions in a realistic environment, including other
moving traffic will introduce additional visual and auditory stimuli to the
scenario.

####  4.2.3. Considering familiar elements and traffic cultures

The development of a familiar VE (e.g., an urban downtown scene (Deb et al.,
2018), an inner-city road in Munich (Dietrich et al., 2020), or a well-known
suburb (Nguyen et al., 2019)) could increase study participants’ level of
comfort (Nguyen et al., 2019). Similar to a VR familiarization phase (Deb et
al., 2020), these VEs might help reduce potential distractions caused by the
novelty effect. Aside from the physical aspects, it is important to attend to
the local traffic norms in designing traffic scenarios. For instance, an
unsignalized crosswalk can represent either a safe (Ackermans et al., 2020; De
Clercq et al., 2019) or unsafe situation (Deb et al., 2020) depending on
cultures.

####  4.2.4. Creating a social atmosphere

Simply rendering a few people and their background chatter within a scene
(Colley et al., 2020c) can bring liveliness to the simulated world and
contribute to a more realistic experience (Schuemie et al., 2001). These
additions should be placed at a distance from the participant in a study’s
virtual environment to avoid diverting their attention, especially when users
might react strongly to humanoid agents in VR (Hoggenmüller et al., 2021)
because of the potential uncanny valley effect (Mori et al., 2012).

####  4.2.5. Utilizing background noise

Including an ambient soundscape in the simulation was found to heighten the
simulation’s realism and users’ perceived presence (Kern and Ellermeier,
2020). Therefore, studies should reproduce sound effects spatially in VR and
consider suppressing the sounds of the actual environment with noise-canceling
headphones (Stadler et al., 2019; Othersen et al., 2018). To validate the
effect of auditory cues in a realistic implementation, studies must simulate
an average urban noise level (e.g., 52 dB (Deb et al., 2020)). However, if the
simulated scenario features dense urban areas, the noise generated by road
traffic and other street activity could become excessive and obscure any
auditory signals intended for pedestrians (Mahadevan et al., 2019).

###  4.3. Standardization and benchmarking

Published studies have not always reported the configuration of VR scenarios
in detail. Indeed, several factors, such as street width or vehicle slowdown
characteristics, remained unspecified in many studies. The limited
availability of vehicle driving data could pose a challenge for researchers
trying to replicate studies. This is of particular importance as vehicle
kinematics have been identified as essential cues on which pedestrians rely
when making crossing decisions.

Except for a handful of VR pedestrian simulators that have been developed with
a high level of customization (Deb et al., 2017; Mahadevan et al., 2019), the
majority of simulators have been implemented for specific study objectives.
Consequently, these simulators have featured a limited number of traffic
scenarios, eHMI design concepts, and vehicle trajectories. Furthermore, only a
few of the published studies have made their simulator source codes publicly
accessible. Only recently, Dalipi et al. (Dalipi et al., 2020) created a VR
platform intended as an open-source benchmark that allows users to experience
different interaction design concepts. We argue that making simulators open-
source could also facilitate researchers’ efforts to compare different
concepts.

Another significant aspect that needs standardization is the measures employed
to assess AV–pedestrian interaction. While definitions and guidelines exist to
determine driver performance in the field and driving simulators (sae, 2015),
there are no established measures to evaluate pedestrian behavior and
subjective experience when interacting with AVs.

###  4.4. Limitations

We cannot claim to have included all relevant studies. Indeed, studies could
have been omitted due to our choice of keywords. However, it is intended that
the identified factors and considerations are continually refined and built
upon as the research field of AV–pedestrian interaction expands. Furthermore,
not all reviewed papers included a colored figure or a video depicting the
simulated scenario, making it challenging to retrieve all needed information
for the analysis.

##  5\. Conclusion

In this paper, we presented a comprehensive review of 31 VR-based studies on
AV–pedestrian interaction. To uncover the focus of existing research efforts,
traffic scenarios were classified based on the right of way and the number of
pedestrians and AVs in the interaction. The analysis provided a detailed
account of different factors influencing pedestrian experience and behavior
and a set of considerations to accompany the development of VR pedestrian
simulators. Additionally, we reported prevalent evaluation measures and
highlighted research gaps that provide avenues for future research in this
domain.

###### Acknowledgements.

This research is supported by an Australian Government Research Training
Program (RTP) Scholarship and through the ARC Discovery Project DP200102604,
Trust and Safety in Autonomous Mobility Systems: A Human-centred Approach.

## References

  * (1)
  * sae (2015) 2015\.  Operational definitions of driving performance measures and statistics.  _J2944, SAE International_ (2015).  https://doi.org/10.4271/J2944{_}201506
  * sae (2021) 2021\.  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles.  _J3016, SAE International_ (2021).  https://doi.org/10.4271/J3016{_}202104
  * Ackermans et al. (2020) Sander Ackermans, Debargha Dey, Peter Ruijten, Raymond H Cuijpers, and Bastian Pfleging. 2020.  The effects of explicit intention communication, conspicuous sensors, and pedestrian attitude in interactions with automated vehicles. In _Proc. 2020 CHI Conf. Human Factors Comput. Syst._ 1–14.
  * Anderson et al. (2014) James M Anderson, Kalra Nidhi, Karlyn D Stanley, Paul Sorensen, Constantine Samaras, and Oluwatobi A Oluwatola. 2014.  _Autonomous vehicle technology: A guide for policymakers_.  Rand Corporation.
  * Arksey and O’Malley (2005) Hilary Arksey and Lisa O’Malley. 2005.  Scoping studies: Towards a methodological framework.  _Int. J. Social Res. Methodology_ 8, 1 (2005), 19–32.
  * Bazilinskyy et al. (2020) Pavlo Bazilinskyy, Lars Kooijman, Dimitra Dodou, and Joost de Winter. 2020.  Coupled simulator for research on the interaction between pedestrians and (automated) vehicles. In _Proc. 19th Driving Simul. Conf._ Antibes, France.
  * Bhagavathula et al. (2018) Rajaram Bhagavathula, Brian Williams, Justin Owens, and Ronald Gibbons. 2018.  The reality of virtual reality: A comparison of pedestrian behavior in real and virtual environments.  _Proc. Human Factors Ergonom. Soc. Annu. Meeting_ 62, 1 (2018), 2056–2060.  https://doi.org/10.1177/1541931218621464
  * Böckle et al. (2017) Marc-Philipp Böckle, Anna Pernestål Brenden, Maria Klingegård, Azra Habibovic, and Martijn Bout. 2017.  SAV2P: Exploring the impact of an interface for shared automated vehicles on pedestrians’ experience. In _Adjunct Proc. 9th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 136–140.
  * Bradley and Lang (1994) Margaret M Bradley and Peter J Lang. 1994.  Measuring emotion: The self-assessment manikin and the semantic differential.  _J. Behav. Therapy Exp. Psychiatry_ 25, 1 (1994), 49–59.
  * Camara et al. (2020) Fanta Camara, Patrick Dickenson, Natasha Merat, and Charles Fox. 2020.  Examining pedestrian-autonomous vehicle interactions in virtual reality. In _Proc. 8th Trans. Res. Arena_. Transport Research Arena.
  * Chang et al. (2017) Chia-Ming Chang, Koki Toda, Daisuke Sakamoto, and Takeo Igarashi. 2017.  Eyes on a car: An interface design for communication between an autonomous car and a pedestrian. In _Proc. 9th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 65–73.
  * Chen et al. (2020) Wenxiang Chen, Qianni Jiang, Xiangling Zhuang, and Guojie Ma. 2020.  Comparison of pedestrians’ gap acceptance behavior towards automated and human-driven vehicles. In _Int. Conf. Human-Comput. Interact._ Springer, 253–261.
  * Colley et al. (2020a) Mark Colley, Stefanos Can Mytilineos, Marcel Walch, Jan Gugenheimer, and Enrico Rukzio. 2020a.  Evaluating highly automated trucks as signaling lights. In _Proc. 12th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 111–121.
  * Colley et al. (2020c) Mark Colley, Marcel Walch, Jan Gugenheimer, Ali Askari, and Enrico Rukzio. 2020c.  Towards inclusive external communication of autonomous vehicles for pedestrians with vision impairments. In _Proc. 2020 CHI Conf. Human Factors Comput. Syst._ 1–14.
  * Colley et al. (2019) Mark Colley, Marcel Walch, and Enrico Rukzio. 2019.  For a better (simulated) world: Considerations for VR in external communication research. In _Adjunct Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 442–449.
  * Colley et al. (2020b) Mark Colley, Marcel Walch, and Enrico Rukzio. 2020b.  Unveiling the lack of scalability in research on external communication of autonomous vehicles. In _Extended Abstr. 2020 CHI Conf. Human Factors Comput. Syst._ 1–9.
  * Dalipi et al. (2020) Ana Fiona Dalipi, Dongfang Liu, Xiaolei Guo, Yingjie Chen, and Christos Mousas. 2020.  VR-PAVIB: The virtual reality pedestrian-autonomous vehicle interaction benchmark. In _Proc. 12th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 38–41.
  * De Clercq et al. (2019) Koen De Clercq, Andre Dietrich, Juan Pablo Núñez Velasco, Joost de Winter, and Riender Happee. 2019.  External human-machine interfaces on automated vehicles: Effects on pedestrian crossing decisions.  _Human Factors_ 61, 8 (2019), 1353–1370.
  * Deb et al. (2019) Shuchisnigdha Deb, Daniel W Carruth, Muztaba Fuad, Laura M Stanley, and Darren Frey. 2019.  Comparison of child and adult pedestrian perspectives of external features on autonomous vehicles using virtual reality experiment. In _Int. Conf. Appl. Human Factors Ergonom._ Springer, 145–156.
  * Deb et al. (2020) Shuchisnigdha Deb, Daniel W Carruth, and Christopher R Hudson. 2020.  How communicating features can help pedestrian safety in the presence of self-driving vehicles: Virtual reality experiment.  _IEEE Trans. Human-Mach. Syst._ 50, 2 (2020), 176–186.
  * Deb et al. (2017) Shuchisnigdha Deb, Daniel W Carruth, Richard Sween, Lesley Strawderman, and Teena M Garrison. 2017.  Efficacy of virtual reality in pedestrian safety research.  _Appl. Ergonom._ 65 (2017), 449–460.
  * Deb et al. (2018) Shuchisnigdha Deb, Lesley J Strawderman, and Daniel W Carruth. 2018.  Investigating pedestrian suggestions for external features on fully autonomous vehicles: A virtual reality experiment.  _Transp. Res. Part F: Traffic Psychol. Behav._ 59 (2018), 135–149.
  * Dey et al. (2020) Debargha Dey et al. 2020\.  Taming the eHMI jungle: A classification taxonomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces.  _Transp. Res. Interdisciplinary Perspectives_ 7 (2020), 100174.  https://doi.org/10.1016/j.trip.2020.100174
  * Dietrich et al. (2019) André Dietrich, Philipp Maruhn, Lasse Schwarze, and Klaus Bengler. 2019.  Implicit communication of automated vehicles in urban scenarios: Effects of pitch and deceleration on pedestrian crossing behavior. In _Int. Conf. Human Syst. Eng. Design: Future Trends Appl._ Springer, 176–181.
  * Dietrich et al. (2020) André Dietrich, M Tondera, and K Bengler. 2020.  Automated vehicles in urban traffic: the effect of kinematics and eHMIs on pedestrian crossing behavior.  _Adv. Transp. Stud._ SI 2 (2020), 73–84.
  * Feldstein and Dyszak (2020) Ilja T Feldstein and Georg N Dyszak. 2020.  Road crossing decisions in real and virtual environments: A comparative study on simulator validity.  _Accident Anal. Prevention_ 137 (2020), 105356.
  * Feng et al. (2020) Yan Feng, Dorine Duives, Winnie Daamen, and Serge Hoogendoorn. 2020.  Data collection methods for studying pedestrian behaviour: A systematic review.  _J. Building Environ._ (2020), 107329.
  * Fuest et al. (2019) Tanja Fuest, Anna Sophia Maier, Hanna Bellem, and Klaus Bengler. 2019.  How should an automated vehicle communicate its intention to a pedestrian?–A virtual reality study. In _Int. Conf. Human Syst. Eng. Design: Future Trends Appl._ Springer, 195–201.
  * Fuest et al. (2017) Tanja Fuest, Lenja Sorokin, Hanna Bellem, and Klaus Bengler. 2017.  Taxonomy of traffic situations for the interaction between automated vehicles and human road users. In _Int. Conf. Appl. Human Factors Ergonom._ Springer, 708–719.
  * Gruenefeld et al. (2019) Uwe Gruenefeld, Sebastian Weiß, Andreas Löcken, Isabella Virgilio, Andrew L Kun, and Susanne Boll. 2019.  VRoad: Gesture-based interaction between pedestrians and automated vehicles in virtual reality. In _Adjunct Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 399–404.
  * Hart and Staveland (1988) Sandra G Hart and Lowell E Staveland. 1988.  Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.  _Adv. Psychol._ 52, C (jan 1988), 139–183.  https://doi.org/10.1016/S0166-4115(08)62386-9
  * Higuera-Trujillo et al. (2017) Juan Luis Higuera-Trujillo, Juan López-Tarruella Maldonado, and Carmen Llinares Millán. 2017.  Psychological and physiological human responses to simulated and real environments: A comparison between Photographs, 360 Panoramas, and Virtual Reality.  _Appl. Ergonom._ 65 (2017), 398–409.
  * Hoggenmüller et al. (2021) Marius Hoggenmüller, Martin Tomitsch, Luke Hespanhol, Tram Thi Minh Tran, Stewart Worrall, and Eduardo Nebot. 2021.  Context-based interface prototyping: Understanding the effect of prototype representation on user feedback. In _Proc. 2021 CHI Conf. Human Factors Comput. Syst._ 1–14.
  * Holländer et al. (2019a) Kai Holländer, Ashley Colley, Christian Mai, Jonna Häkkilä, Florian Alt, and Bastian Pfleging. 2019a.  Investigating the influence of external car displays on pedestrians’ crossing behavior in virtual reality. In _Proc. 21st Int. Conf. Human-Comput. Interact. Mobile Devices Services_. 1–11.
  * Holländer et al. (2019b) Kai Holländer, Philipp Wintersberger, and Andreas Butz. 2019b.  Overtrust in external cues of automated vehicles: An experimental investigation. In _Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 211–221.
  * Hudson et al. (2018) Christopher R Hudson, Shuchisnigdha Deb, Daniel W Carruth, John McGinley, and Darren Frey. 2018.  Pedestrian perception of autonomous vehicles with external interacting features. In _Int. Conf. Appl. Human Factors Ergonom._ Springer, 33–39.
  * Jayaraman et al. (2018) Suresh Kumaar Jayaraman et al. 2018\.  Trust in AV: An uncertainty reduction model of AV-pedestrian interactions. In _Companion 2018 ACM/IEEE Int. Conf. Human-Robot Interact._ 133–134.
  * Kaß et al. (2020) Christina Kaß, Stefanie Schoch, Frederik Naujoks, Sebastian Hergeth, Andreas Keinath, and Alexandra Neukum. 2020.  Standardized test procedure for external human–machine Interfaces of automated vehicles.  _Information_ 11, 3 (2020), 173.
  * Kern and Ellermeier (2020) Angelika C Kern and Wolfgang Ellermeier. 2020.  Audio in VR: Effects of a soundscape and movement-triggered step sounds on presence.  _Frontiers Robot. AI_ 7 (2020), 20.
  * Kooijman et al. (2019) Lars Kooijman, Riender Happee, and Joost C F de Winter. 2019.  How do eHMIs affect pedestrians’ crossing behavior? A study using a head-mounted display combined with a motion suit.  _Information_ 10, 12 (2019), 386.
  * Koutsoklenis and Papadopoulos (2011) Athanasios Koutsoklenis and Konstantinos Papadopoulos. 2011.  Auditory cues used for wayfinding in urban environments by individuals with visual impairments.  _J. Visual Impairment Blindness_ 105, 10 (2011), 703–714.
  * Lee et al. (2019a) Yee Mun Lee et al. 2019a.  Investigating pedestrians’ crossing behaviour during car deceleration using wireless head mounted display: an application towards the evaluation of eHMI of automated vehicles. In _Proc. 10th Int. Driving Symp. Human Factors in Driving Assessment, Training and Vehicle Design_. 252–258.
  * Lee et al. (2019b) Yee Mun Lee et al. 2019b.  Understanding the messages conveyed by automated vehicles. In _Proc. 11th Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 134–143.
  * Löcken et al. (2019) Andreas Löcken, Carmen Golling, and Andreas Riener. 2019.  How should automated vehicles interact with pedestrians? A comparative analysis of interaction concepts in virtual reality. In _Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 262–274.
  * Mahadevan et al. (2019) Karthik Mahadevan, Elaheh Sanoubari, Sowmya Somanath, James E Young, and Ehud Sharlin. 2019.  AV-Pedestrian interaction design using a pedestrian mixed traffic simulator. In _Proc. 2019 Designing Interactive Syst. Conf._ 475–486.
  * Moore et al. (2019) Dylan Moore, Rebecca Currano, G Ella Strack, and David Sirkin. 2019.  The case for implicit external human-machine interfaces for autonomous vehicles. In _Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 295–307.  https://doi.org/10.1145/3342197.3345320
  * Mori et al. (2012) Masahiro Mori, Karl F MacDorman, and Norri Kageki. 2012.  The uncanny valley [from the field].  _IEEE Robot. Automat. Mag._ 19, 2 (2012), 98–100.
  * Nascimento et al. (2019) Alexandre M Nascimento et al. 2019\.  The role of virtual reality in autonomous vehicles’ safety. In _2019 IEEE Int. Conf. Artif. Intell. Virtual Reality_. 50–507.
  * Nguyen et al. (2019) Trung Thanh Nguyen, Kai Holländer, Marius Hoggenmueller, Callum Parker, and Martin Tomitsch. 2019.  Designing for projection-based communication between autonomous vehicles and pedestrians. In _Proc. 11th ACM Int. Conf. Automot. User Interfaces Interact. Veh. Appl._ 284–294.
  * Nordhoff et al. (2018) Sina Nordhoff, Joost De Winter, Miltos Kyriakidis, Bart Van Arem, and Riender Happee. 2018.  Acceptance of driverless vehicles: Results from a large cross-national questionnaire study.  _J. Adv. Transp._ 2018 (2018), 1–22.  https://doi.org/10.1155/2018/5382192
  * Othersen et al. (2018) I Othersen, Antonia S Conti-Kufner, André Dietrich, Philipp Maruhn, and Klaus Bengler. 2018.  Designing for automated vehicle and pedestrian communication: Perspectives on eHMIs from older and younger persons.  _Proc. Human Factors Ergonom. Soc. Europe_ (2018), 135–148.
  * Petkova and Ehrsson (2008) Valeria I Petkova and H Henrik Ehrsson. 2008.  If I were you: perceptual illusion of body swapping.  _PloS One_ 3, 12 (2008), e3832.
  * Pillai (2017) Anantha Pillai. 2017.  _Virtual reality based study to analyse pedestrian attitude towards autonomous vehicles_.  Master’s thesis. Aalto Univ.  http://urn.fi/URN:NBN:fi:aalto-201710307409
  * Rasouli and Tsotsos (2020) Amir Rasouli and John K Tsotsos. 2020.  Autonomous vehicles that interact with pedestrians: A survey of theory and practice.  _IEEE Trans. Intell. Transp. Syst._ 21, 3 (2020), 900–918.  https://doi.org/10.1109/TITS.2019.2901817
  * Robert Jr (2019) Lionel P Robert Jr. 2019.  The future of pedestrian-automated vehicle interactions.  _XRDS: Crossroads, ACM Mag. Students_ 25, 3 (apr 2019), 30–33.  https://doi.org/10.1145/3313115
  * Sadraei et al. (2020) Ehsan Sadraei et al. 2020\.  Vehicle-pedestrian interaction: A distributed simulation study. In _Proc. 19th Driving Simul. Conf._ Antibes, France, 145–154.
  * Schmidt et al. (2019) Henri Schmidt, Jack Terwilliger, Dina AlAdway, and Lex Fridman. 2019.  Hacking nonverbal communication between pedestrians and vehicles in virtual reality. In _Proc. 10th Int. Driving Symp. Human Factors in Driver Assessment, Training and Vehicle Design_. 78–84.
  * Schneider and Bengler (2020) Sonja Schneider and Klaus Bengler. 2020.  Virtually the same? Analysing pedestrian behaviour by means of virtual reality.  _Transp. Res. Part F: Traffic Psychol. Behav._ 68 (2020), 231–256.
  * Schuemie et al. (2001) Martijn J Schuemie, Peter Van Der Straaten, Merel Krijn, and Charles APG Van Der Mast. 2001.  Research on presence in virtual reality: A survey.  _Cyberpsychol. Behav._ 4, 2 (2001), 183–201.
  * Slater and Wilbur (1997) Mel Slater and Sylvia Wilbur. 1997.  A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments.  _Presence Teleoperators Virtual Environ._ 6, 6 (1997), 603–616.
  * Stadler et al. (2019) Sebastian Stadler, Henriette Cornet, Tatiana Novaes Theoto, and Fritz Frenkler. 2019.  A tool, not a toy: Using virtual reality to evaluate the communication between autonomous vehicles and pedestrians.  In _Augmented Reality and Virtual Reality_. Springer, 203–216.
  * Stanney et al. (2003) Kay M Stanney, Mansooreh Mollaghasemi, Leah Reeves, Robert Breaux, and David A Graeber. 2003.  Usability engineering of virtual environments (VEs): Identifying multiple criteria that drive effective VE system design.  _Int. J. Human-Comput. Stud._ 58, 4 (2003), 447–481.
  * Velasco et al. (2019) J Pablo Nuñez Velasco, Haneen Farah, Bart van Arem, and Marjan P Hagenzieker. 2019.  Studying pedestrians’ crossing behavior when interacting with automated vehicles using virtual reality.  _Transp. Res. Part F: Traffic Psychol. Behav._ 66 (2019), 1–14.
  * Wang et al. (2019) Peggy Wang et al. 2019\.  _Safety implications of automated vehicles providing external communication to pedestrians_.  Technical Report. Univ. California Inst. Transp. Stud.  https://escholarship.org/uc/item/82n7043g
  * Weber et al. (2019) Florian Weber, Ronee Chadowitz, Kathrin Schmidt, Julia Messerschmidt, and Tanja Fuest. 2019.  Crossing the street across the globe: A study on the effects of eHMI on pedestrians in the US, Germany and China. In _Int. Conf. Human-Comput. Interact._ Springer, 515–530.
  * WHO (2018) WHO. 2018.  Global status report on road safety 2018: Summary.  https://apps.who.int/iris/handle/10665/277370
  * Wilbrink et al. (2017) Marc Wilbrink et al. 2017\.  D1.1 Definition of interACT use cases and scenarios.  https://elib.dlr.de/116445