# Designing for Projection-based Communication between Autonomous Vehicles and
Pedestrians

Trung Thanh Nguyen  School of Architecture, Design and PlanningThe University
of SydneySydneyAustralia<EMAIL_ADDRESS>,  Kai Holländer
LMU MunichMunichGermany<EMAIL_ADDRESS>,  Marius Hoggenmueller
School of Architecture, Design and PlanningThe University of
SydneySydneyAustralia<EMAIL_ADDRESS>,  Callum Parker
School of Architecture, Design and PlanningThe University of
SydneySydneyAustralia<EMAIL_ADDRESS>and  Martin Tomitsch  School
of Architecture, Design and Planning, The University of SydneyCAFA Beijing
Visual Art Innovation Institute, China<EMAIL_ADDRESS>
(2019)

###### Abstract.

Recent studies have investigated new approaches for communicating an
autonomous vehicle’s (AV) intent and awareness to pedestrians. This paper adds
to this body of work by presenting the design and evaluation of in-situ
projections on the road. Our design combines common traffic light patterns
with aesthetic visual elements. We describe the iterative design process and
the prototyping methods used in each stage. The final design concept was
represented as a virtual reality simulation and evaluated with 18 participants
in four different street crossing scenarios, which included three scenarios
that simulated various degrees of system errors. We found that different
design elements were able to support participants’ confidence in their
decision even when the AV failed to correctly detect their presence. We also
identified elements in our design that needed to be more clearly communicated.
Based on these findings, the paper presents a series of design recommendations
for projection-based communication between AVs and pedestrians.

autonomous vehicles, vehicle-to-pedestrian communication, design process,
projection, virtual reality

††copyright: none††copyright: acmlicensed††journalyear: 2019††conference: 11th
International Conference on Automotive User Interfaces and Interactive
Vehicular Applications; September 21–25, 2019; Utrecht,
Netherlands††booktitle: 11th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications (AutomotiveUI ’19),
September 21–25, 2019, Utrecht, Netherlands††price: 15.00††doi:
10.1145/3342197.3344543††isbn: 978-1-4503-6884-1/19/09††ccs: Human-centered
computing Interaction design

##  1\. Introduction

Urban robots are currently making the transition from laboratory and test-bed
environments to being evaluated in real-world urban contexts. This transition
is driven by public authorities and industry, mainly with the aim to automate
public services, logistics and transportation. A main focus is the development
of the driverless car, leading to speculation and debate about how this
technology may shape our cities in the near future (Gandia et al., 2019). One
consideration which has seen much attention by the community recently, is how
driverless cars and other autonomous vehicles111Some scholars have argued for
using the term “automated vehicles” as such vehicles are not (yet) strictly
autonomous. We use “autonomous” in this paper as it is, at the time of
writing, still widely used in previous research studies and in the industry to
refer to these types of vehicles, and since our research focuses on future
scenarios where vehicles might indeed be fully autonomous. (AVs) communicate
their internal state to nearby pedestrians (Mahadevan et al., 2018; Rasouli
and Tsotsos, 2018). This is an important challenge as there is no human driver
that is able to interact with pedestrians (Guéguen et al., 2015) and for AVs
operating safely in an urban environment, effective communication channels are
critical to help pedestrians in making decisions, e.g. about when and where to
cross a road. To address this challenge, various design concepts have been
developed by academic research, design studios and the automotive industry.
For example, the London-based design studio Umbrellium developed an
interactive LED-based pedestrian
crossing222http://umbrellium.co.uk/initiatives/starling-crossing/; last
accessed: April 2019 that responds dynamically to cars and pedestrians.
However, as scaling up such design concepts would require the upgrade of
existing city infrastructure, resulting in high costs, others focused on using
the external surface of cars as displays (Colley et al., 2017), ranging from
embedded low-resolution LED displays (Gandia et al., 2019), on-car projections
(Colley et al., 2018) to attached screens333https://www.drive.ai/; last
accessed: April 2019. Using the car itself as a carrier to display relevant
information for surrounding pedestrians, Colley et al. (2018) made the case
for designing cars as a shared resource. However, as cars have a major
influence on the cityscape, it is important that such displays on cars are not
only designed for maximised efficiency and attention, but are also
aesthetically pleasing and not contributing towards the increasing
oversaturation of public spaces with pervasive displays (Davies, 2018).

To address this challenge, we present the iterative design of in-situ
projections to help pedestrians in making crossing decisions. Projections were
chosen as a suitable display technology as: (1) they are visible to a large
number of pedestrians; (2) they can be pointed towards the street, which is
already used as a surface for traffic signage, such as pedestrian crossings,
thus providing a display location that pedestrians are familiar with; and (3)
they can be easily switched off when not needed. While previous work on AV-to-
pedestrian displays mainly focused on the effective encoding of information,
in this paper we present methods for designing aesthetic aspects, treading the
fine line between efficiency, safety and aesthetics. Throughout the design
process, we developed prototypes of various fidelity levels, including
sketches, animated mock-ups and interactive virtual reality (VR) simulations.
We tested the final design concept in a VR user study, thereby focusing on a
street-crossing scenario with driverless cars, to evaluate how AVs can
communicate their intent and awareness to pedestrians via projections,
especially in situations that are not facilitated by traffic lights. Testing
the final design in VR enabled us to also evaluate the influence of sensor
failures within the prototype on participants’ confidence regarding the
visualisations, in a safe, risk-free environment.

The contribution of this paper is three-fold: (1) We report on our approach of
designing projection-based information cues using prototyping methods of
various fidelity levels; (2) we present insights and findings from a VR
simulation study evaluating the efficiency of those cues in terms of their
comprehensibility with potential users; and (3) based on those findings, we
derive design recommendations for projection-based communication between AVs
and pedestrians.

##  2\. Related Work

The following areas of research form the foundation for the study presented in
this paper: (1) research on pedestrian behaviour when crossing a road, (2)
publications addressing the communication between AVs and pedestrians, and (3)
work that used projections as a communication channel.

###  2.1. Pedestrian Crossing Behaviour

Previous research stressed that the main source of motivation for pedestrians
to cross a road in front of a vehicle is the size of the gap between
approaching cars and the car’s behaviour (e.g. accelerating or breaking)
(Yannis et al., 2013; Rasouli et al., 2018b; Dey and Terken, 2017). Amongst
others, Beggiato et al. (Beggiato et al., 2017) investigated what size a gap
should have and at what moment an autonomous vehicle should break in order to
create a natural perception of driving behaviour from a pedestrian’s
perspective. A longer observation of oncoming traffic indicates pedestrian’s
willingness to cross a road (Rasouli et al., 2017, 2017). After a decision is
made, communication between a driver and pedestrians is an additional crucial
aspect to foster safety and comfort (Rasouli et al., 2017, 2018a). For
example, pedestrians can provoke human drivers to come to a halt through
staring (Guéguen et al., 2015) or gesturing (Zhuang and Wu, 2014).

###  2.2. AVs and Pedestrian Communication

As a consequence of increasing automation in vehicle control systems, drivers
are able to engage in non-driving-related activities (Pfleging et al., 2016;
Milakis et al., 2017). Hence, advanced driving-assistance systems with an SAE
level higher than three do not require “drivers” to interact with their
surroundings (International, 2018). However, pedestrians benefit from
information about the awareness and intent of highly automated vehicles,
especially during crossing decisions (Mahadevan et al., 2018; Rothenbücher et
al., 2016; Habibovic et al., 2018). To overcome the lack of driver-to-
pedestrian communication, research and industry developed various concepts to
help with this (Rasouli and Tsotsos, 2019). As of now, most prototypes feature
one or more external displays, with the display technology being embedded or
attached to the outer shell of the vehicle (Clamann et al., 2016; Fridman et
al., 2017; Li et al., 2018). However, using conventional screens comes with
the limitation that the content is only visible from a specific angle, which
limits the number of pedestrians that can receive the information. The issue
of multi-user support also accounts for externally attached contraptions, for
example, additional eyes which follow pedestrians (Chang et al., 2018, 2017;
Mahadevan et al., 2018). To enable communication with multiple users,
approaches featuring personal devices such as smartphones have been tested as
an alternative to displays on the car (Mahadevan et al., 2018; NISSAN MOTOR
Co., [n. d.]). Finally, researchers stressed that AV-to-pedestrian
communication channels need to be designed following a user-centred design
process (Holländer, 2018), which is challenging as prospective users cannot
yet draw on experiences interacting with AVs (Frison et al., 2017).

###  2.3. Projection-Based Concepts

A promising approach is the use of projections on the road as this provides a
display surface that is only restricted by the distance of the projector to
the ground and the utilised hardware, rather than the design of the car
itself, which limits the display’s size and the deployment location. Further,
projections are capable of presenting information in a wide range of visual
encodings, from more ambient representations, such as colour and patterns, to
symbols and high-resolution images. Mercedes Benz presented a
prototype444https://www.mercedes-benz.com/en/mercedes-
benz/innovation/research-vehicle-f-015-luxury-in-motion/; last accessed: March
2019 which is able to project a crosswalk on the ground to yield the way for
non-motorised road users.
Mitsubishi555http://www.mitsubishielectric.com/news/2015/pdf/1023.pdf/; last
accessed: March 2019 uses projection to indicate a vehicle’s intended
direction (straight, back, left or right). Colley et al. (Colley et al., 2018)
explored projections on the car and ground to remind drivers about forgotten
items (e.g. phone and keys) or to display safety information and intentions
outside of the vehicle. In their study, displaying safety-critical information
to pedestrians was perceived as most beneficial and received social
acceptance. Dancu et al. (Dancu et al., 2015) applied a projection of
directional intentions to a bicycle and showed that their system simplified
predicting movements for other road users. A common shortcoming of projection-
based systems is their limited visibility during bright daylight. This
limitation is increasingly addressed through the availability of laser
projection systems. To account for visibility issues, we designed our system
and study in a dusk scenario. In future implementations, projection-based
solutions could also take on a complementary function to support other
communication systems in poor lighting conditions. Uneven road surfaces (e.g.
gravel) or interference of multiple vehicle projections might hinder the
perception of signals. However, these challenges go beyond the scope of the
aims of our study and offer avenues for future research.

##  3\. Design Process

In this section, we report on the iterative process of designing in-situ
projections on the road for safety critical situations. As the context -
designing for communication channels between autonomous vehicles and
pedestrians - is still rather underrepresented in the field of human-computer
interaction (HCI), we could not fall back on existing purpose-built methods
and tools. We therefore applied common interaction design approaches and
tailored them to our specific context.

###  3.1. Situation

We designed the projection-based visualisations for the situation in which a
pedestrian intends to cross the street at a pedestrian crossing and sees an AV
approaching. In this situation, supporting information cues are of importance
as the traffic flow is not facilitated by a traffic light and interactions
between the driver and pedestrian are not possible. This means that the
pedestrian has to solely rely on the AV’s sensors, which can lead to
uncertainty in the pedestrian’s decision to cross the road. Furthermore, it
has been reported that 25% of all accidents in Europe that resulted in the
death of pedestrians happened at a pedestrian crossing (Guéguen et al., 2015).

Therefore, the crossing scenario has been well researched (e.g. Rothenbücher
et al. (2016)), however, not with a projection-based design. Applying our
projection concepts to the crossing scenario allows us to discuss our results
in relation to the insights from previous research that used other display
solutions. Previous research suggested that malfunction and errors reduce
people’s reliance on automation systems (Yang et al., 2017). In this study, we
aimed to understand to what extent pedestrians still rely on information cues
from AVs - to make decisions and ensure their safety - even when the vehicle’s
sensors are malfunctioning (e.g. not detecting the participant).

###  3.2. Projection Concepts

Designing the projection concepts for the crossing situation, we followed an
iterative design process, including low-fidelity sketches, digital mock-ups,
animated 2D prototypes in the java-based programming environment Processing
666https://www.processing.org; last accessed: April 2019 and the final 3D
prototype developed in Unity777https://www.unity.com; last accessed: April
2019. We used low-fidelity prototyping methods to quickly explore a wide range
of ideas and concepts. 2D prototypes were then developed to refine the design
concepts and to derive specifications for the visual appearance of the
projections (e.g. visual elements, animation sequences).

We adapted the template for mapping interaction between driverless cars and
pedestrians from the framework created by Owensby et al. (Owensby et al.,
2018). In our chosen situation, a vehicle’s ideal response to prioritise
pedestrian safety is to slow down, stop in front of the crossing lines and
wait until all pedestrians finish crossing before starting again. We broke
down this sequence to the vehicle’s status, its awareness of pedestrians,
pedestrian’s intent to cross and the distance between the vehicle and
pedestrians. Using these parameters, we define four stages of the AV in the
crossing situation as moving, slowing down, stopping, and about to go (See
table on the left in Figure 2). Owensby et al.’s framework highlights the need
for pedestrians to know the current status of the car, its intent and whether
it is aware of them. We added an additional consideration (“Message to be
conveyed”) to interpret the meaning of our visualisation from an observer’s
point of view. We then designed four types of visualisations corresponding to
the four statuses of the AV. A storyboard was developed with preliminary
visualisations to understand how they can support the interaction flow in the
contextual situation. Figure 2 shows the revised, final high-fidelity
visualisations for each status of the AV.

####  3.2.1. Ideation

Figure 1. (Left) Japanese Zen garden with gravel that mimics wave pattern,
creating a calm feeling. Image credit: PlusMinus via Wikimedia. (Right) A
traffic colour scheme was used in our design solution to take advantage of
existing traffic communication practices. Image credit: Harshal Desai on
Unsplash. Figure 2. The four key visualisations signifying the intent and
awareness of the AV in an ideal scenario (left), and the four scenarios used
in the VR study, including the ideal scenario (A) and three scenarios with
varied degrees of sensor failures (B, C, D) (right).

We sketched various ideas for projection visualisations on paper. This
included text as Chang et al. (Chang et al., 2018) found that presenting text
information was preferred by participants when communicating a vehicle’s
intention compared to other modalities (eyes on a car, smile icon, lights and
projections). However, using text has limitations over symbols in particular
in multi-cultural environments where multiple languages are spoken.
Furthermore, text excludes illiterate people and children that are not yet
able to read, reading text requires the foreground of user’s attention, and
readability is hampered when the car is in motion. Another study also found
that people do not interpret text quickly enough in this context (Clamann et
al., 2016). These considerations led us to using aesthetic ambient lighting
displays as a reference, as these types of displays are typically designed to
communicate information to people in a low cognitive load approach. In
aesthetic ambient lighting displays, the use of natural phenomena, such as
ocean waves to visualise data of a system, has been proven to attract
attention subtly and help observers understand how the systems works (Jones,
2018; Shelton and Nesbitt, 2017). Sonar, a method to communicate or detect
objects in the water (Ainslie, 2010), was the main inspiration for the wave
pattern in our projection-based design concept. As part of human verbal
communication, sound when visualised can highlight the expressive content of
the communication channel (Pietrowicz and Karahalios, 2013). The wave pattern
also appears in Japanese gardens, which exhibit a calming atmosphere (Van
Tonder and Lyons, 2005). The gravel with wave shapes (see Figure 1) around the
objects highlights the presence of objects (Van Tonder and Lyons, 2005).

In the following step, we converted the sketches to digital mock-ups to
investigate variations of colours, shapes and the visualisation’s relative
size to the vehicle through many rounds of iteration. One of the challenges we
had to overcome with our design proposal was the limited number of visual
elements to make four types of visualisations distinguishable in reflection of
the four states of the AV. We used three visual elements in our visualisation:
colours, patterns and movement created by shifting patterns and colour. The
traffic light colour scheme was mapped to the car’s status. We used a wave
pattern for when the car was still moving and crossing lines for when the car
had stopped and it was safe for the pedestrian to cross in front of the car.
The combination of colours, patterns and movements allowed us to create
distinctive visualisations to communicate the car’s intent and awareness of
pedestrians in all four states.

####  3.2.2. Dynamic Prototype

Processing was used as a medium to prototype the patterns and to fine-tune the
visual aspects such as size, shape and movement. The speed of radiating waves
and chasing green crossing lines in this prototoype was controlled manually
through keyboard input and mapped to the vehicle’s speed.

####  3.2.3. Virtual Reality Prototype

We created the final prototype representation in VR, allowing us to adopt VR
as a testing method, similar to previous studies of AV-to-pedestrian
communication systems (e.g. (Pillai, 2017; Mahadevan et al., 2018)). While
previous studies asked people to indicate whether they would start crossing
the street by pressing a button (e.g. (Mahadevan et al., 2018)), we aimed to
implement a situation that was closer to a real-world situation. To that end,
we focused on achieving visual realism, by using a high frame-rate, field of
view, head tracking system and position tracking in VR, which has been
demonstrated to enhance realistic behavioural responses (Slater et al.,
2009b). Testing our solution in the VR environment also allowed us to put
participants in risky situations which would not be safe to evaluate in real-
world settings.

The prototype was created using Unity and 3D models. The vehicle is a 5-seater
midsize sedan model. The wheels had animation that closely matched the car’s
speed and turning movement. We did not model any person inside the vehicle,
and improved the lighting condition so that participants could observe the
absence of a human driver. The environment was created using pre-made models
of an urban environment with a resemblance to a suburb familiar to our
participants. This was chosen to create a comfortable feeling by providing a
sense of familiarity in the scene. The daylight was adjusted to create long
shadows in the initial view of participants, supporting visual realism. The
overall brightness was dimmed to match the dusk time and to be more suitable
for visibility of the projection on the ground. A HTC Vive VR headset was used
to experience this environment.

We developed a logic to control the car speed, braking and turning of the
wheels according to the driving path and the visibility of a human’s 3D object
in the environment. The status of the car was communicated via the projection
in such a way that a change of the cars’ status triggered a change of the
displayed information. We developed a test control of the car so that we could
disable the car’s sensor anytime, which allowed us to create deliberate
malfunctions in some of our study scenarios.

##  4\. Evaluation Study

###  4.1. Experiment Design

As our research objective was to study how people perceived the vehicle’s
intent and awareness via projection-based communication in different
situations (including malfunctioning scenarios where the sensor would fail to
“see” the pedestrian, e.g. due to poor light conditions), we prepared four
scenarios (See Figure 2, diagram on the right). This allowed us to evaluate
how pedestrians would react and behave across different scenarios and whether
the communication channel enables people to make safe decisions. The scenarios
are:

  * •

Scenario A: Car sensor works correctly throughout the scenario.

  * •

Scenario B: Car sensor works correctly at first and detects the pedestrian on
the side of the road, but it fails to correctly detect the pedestrian while
they are crossing.

  * •

Scenario C: Car sensor works correctly initially, detecting the pedestrian and
making the car start to slow down, but then fails to detect the pedestrian
correctly before the car completely stops.

  * •

Scenario D: Car does not detect the presence of the pedestrian at all.

Factors that influence pedestrian behaviour in a crossing situation are
driving speed and gap distance between the pedestrian and the car (Clamann et
al., 2016). We kept these control variables constant by having the same street
and the same AV coming to the crosswalk from the same side in all scenarios.

###  4.2. Procedure

As a sense of presence in VR is essential for participants to display
realistic responses (Slater et al., 2009a), we followed the suggestions made
in previous research (Slater et al., 2002) to have different activities before
the test, such as asking people to walk around while wearing the VR headset.
Body movement, such as walking, can be tracked in a virtual environment and
can help produce a sense of presence more than pointing and clicking (Slater
et al., 2002).

In our experiment, the participant was placed on the footpath near a marked
pedestrian crossing. Before starting the test scenarios, the participant was
asked to practice crossing the street when there were no cars approaching.
This was to help participants become familiar with the environment. We also
asked several questions about the scene to help participants adapt their
visual perception to a new environment. Next, we started a familiarisation
scene, in which the participant stood on the pavement at some distance from
the road to observe cars passing by without having to make a crossing
decision. The cars throughout this scene functioned properly and only
displayed the _Moving_ visualisation as the pedestrian was too far from the
road to be detected as intending to cross the road. As part of this scene, we
also asked participants if they could identify the cars as driverless,
prompting them to look closely in order to realise that there was no driver
sitting behind the steering wheel. This was important to ensure participants
were aware that their decision to safely cross the road could not rely on
interpersonal communication.

Before starting the first scenario, the participant was asked to adjust their
standing position on the pavement until they felt comfortable and safe. We
then started the first test scenario. The sequence of the four scenarios was
randomised to counterbalance any potential learning effects. After each
scenario, the participant took off the headset and sat down in a nearby booth
to complete a short interview and questionnaire. After all four scenarios were
completed, the participant was asked to complete a post-study interview about
the overall experience in VR and to compare all four scenarios.

###  4.3. Measurement

Figure 3. The scenario set-up in VR: The car starts its trajectory at the
corner of the street and approaches the pedestrian from the right side.

We used the following methods to collect both qualitative and quantitative
data about participants’ behaviour, perception and feedback:

(1) Video recordings: Participants’ behaviour (e.g. stepping forward or
stepping backward) while wearing the VR headset were recorded via a video
camera. The video recordings were later used to verify participants’
statements during the data analysis. Further, people have their own crossing
strategies (Clamann et al., 2016), making it difficult to compare
participants’ response times across scenarios.

(2) Post-scenario semi-structured interview questions: After each scenario,
participants were asked to answer a semi-structured interview to capture their
understanding of the visualisation. The interview questions were adapted from
previous studies (Pillai, 2017; Owensby et al., 2018) and included questions
about participants’ understanding of the vehicle’s behaviour and which factors
they used to make the crossing decision. Each interview was between 5 and 10
minutes and audio-recorded for later transcription.

(3) Post-study semi-structured interview questions: After completing all four
scenarios, participants were asked about their overall experience and to
compare the four scenarios. The aim of this interview was to understand how
participants understood and learnt the vehicle’s behaviour. Each interview
lasted between 10 and 15 minutes and was audio-recorded for later
transcription.

We did not record the time it took pedestrians to respond as we did not
compare different visualisation approaches, which has been the focus in
previous AV-to-pedestrian interface studies (e.g. (Clamann et al., 2016)).
Instead, our aim was to understand the participants’ reactions in each of the
four different scenarios (e.g. car slowing down versus not slowing down).

###  4.4. Participants and Location

Our study involved 18 participants (between 18 and 44 years; 11 female, 7
male; students and working professionals). Out of our participants, 15 had
previous experience with VR (from social to gaming applications), possibly
because our call for participants stated that the study involved a VR
experience. Three participants were working in the research fields of smart
cities and autonomous vehicles; one of them had interacted with real AVs
through their research.

In terms of visual acuity, 13 participants had normal vision, the other 5 had
to wear glasses to correct their eyesight. Participants with short-sighted
vision were asked to wear their usual glasses throughout the study. All
participants were able to read, listen and speak fluently in English. None of
the participants had any mobility impairment. All participants read the
Participant Information Statement (approved by our university’s ethics board)
and signed the consent form to permit us to record video, photos and audio.

The study was carried out in our lab, which offers a free space of about 6 by
6 metres. The tracked space through the VR sensors was not large enough for
participants to completely cross the entire street in VR, but it was
sufficient for participants to step from the footpath onto the road to signal
their intention to cross.

Figure 4. The study environment and setup, including the (a) interview booth;
(b) control computer; (c) VR space; and (d) virtual space.

###  4.5. Pilot Study

Prior to our main investigation, we conducted two pilot studies (3
participants each, who did not participate in the main study) to refine the
study setup and materials. The first pilot study involved participants wearing
the headset throughout all four scenarios including each of the post-scenario
interviews. Our intention was to allow participants to remain immersed in the
virtual environment while completing the interviews, however, we found that
this increased the levels of discomfort and risk of nausea because of the
study duration. In the second pilot study, to increase the comfort of
participants, we asked them to take off the VR headset and complete the post-
scenario interview while sitting in the interview booth. The second pilot
study also allowed us to test and revise the way we recorded the video and how
the instructions were delivered to participants. Afterwards, we conducted the
main study with 18 participants by following the aforementioned procedure.

##  5\. Results and Discussion

For the analysis, we transcribed the interviews that we conducted with the
participants after each scenario. We then analysed the transcriptions using
affinity diagramming in order to sort the data into similar concepts
(clustering) and identify themes (Lazar et al., 2010). Upon transcribing
statements made in the interviews that involved an action by the participant
(e.g. stepping back), we used the video recordings to verify these statements.
If the interview statement did not match the observation in the video, we
excluded the respective data. In the following, we discuss the findings of our
qualitative data analysis, including a set of design recommendations (DR) (see
Table 1).

excel-table

Table 1. List of design recommendations.

###  5.1. Interpretation of Colours

In our design, colours were mapped to the vehicle’s status and intent. The
interviews confirmed our observations in the sense that most participants
acknowledged this association after performing all four scenarios: in total,
17 participants stated that in their view a change of colour signalled a
change of the vehicle’s behaviour. However, as previously reported (Zhang et
al., 2018) pedestrians might not necessarily understand whether the displayed
information is an instruction or expresses the vehicle’s intentions.
Commenting on the hesitation to cross when seeing the green crossing
visualisation in the first scenario, participant P8 said “(At first) I think
it is indicating that I shouldn’t go when it is red. But when the light is in
front of the car, it felt that the light is [an] indicator for the car, not
for me”. In the post-study interview, P6 said “I don’t know who is the object
of the light. The traffic light is usually for the driver.” Participant P8
commented on the straight crossing bars changing from green to red colour:
“The light changed. The radar changed from green to red so [the vehicle]
indicated that it was about to start the engine.” One participant (P2)
misunderstood the projections in the sense that she referred to them as the
“normal lighting of the car” and concluded that therefore “there is no
significance”. She mentioned this in the post-study interview, even after
finishing all scenarios, including the one in which the car’s sensors worked
correctly. As a consequence, she used the car’s speed as the main factor to
make a decision at the crosswalk. For participants that interpreted the colour
encoding correctly, the speed of the vehicle was an important factor to
reinforce the interpretation of meaning: “[The vehicle] slowed down and […]
the process of changing colours makes me feel that it changed [colours]
because of me being there […]” (P9). Another participant stated: “Yellow means
[the vehicle] is slowing down, noticing something and checking for caution”
(P8). The real-time synchronisation between the car’s speed and the change of
colour was perceived by the participants in the sense that the vehicle was
trying to detect or already detected their presence. This was articulated by
all participants after they finished the study, though the level of certainty
varied. People were most certain about the car’s awareness of them when the
car stopped.“The car stopped so it must have noticed I am here” (P8).

####  5.1.1. Design Recommendation (DR) 1

Implement matching sequence and colour patterns: In line with previous
research, we found that the car’s kinematics are an important factor for
pedestrians to make a crossing decision (Dey and Terken, 2017). Therefore,
additional visual design elements should match their sequence with the car’s
motion in order to reinforce people’s existing understanding and foster
intuitiveness. Furthermore, adopting the well-known colour indications from
traffic lights has shown to help people adapt quickly to the sequential stages
of the car’s behaviour. However, a limitation seems to be that pedestrians
need to experience the full colour spectrum at least once. Two of the
participants who experienced the successful scenario at the end of the
experiment mentioned that the red colour would be misleading and concluded
that only after seeing the full sequence of red-yellow-green they did
understand the meaning of the red colour. Referring to scenario D, where the
car did not stop at all, one participant stated that the red colour “doesn’t
look safe”. However, this participant experienced the situation rather in the
sense that “the car is just passing by and it’s not that dangerous” (P5).
Another participant (P6) mentioned instead: “I will be cautious with the red
colour, especially when I don’t get to know about the yellow and green
[colour]”.

###  5.2. Interpretation of Crossing Patterns

While the majority of the colour encoding was the primary source to make sense
of the visualisations, two participants also referred to the crossing patterns
in more detail. When being asked which factors informed them about the
vehicle’s intention, they described the shape and movement of patterns closely
to the design: “The pattern changed as well, from concentric circles to
parallel bars” (P12). Another added that the crossing patterns were a helpful
way to provide additional directional information at the crossing: “The
pattern changed from circles to crossing lines. It moved and told me to go in
a certain direction. It felt like [the visualisation] was leading me to go
across” (P9). In general, those subtle movements of light were rather
perceived as an aesthetic “add-on”, with one participant (P15) mentioning “how
the wave is transforming, from lighter to darker colour […], is visually very
attractive”.

####  5.2.1. DR 2

Using multiple visual elements to encode information: Though encoding safety-
critical information using colours seems to be most appropriate for the
majority of users, for others the pattern design also played an important role
in their decision-making. Under different conditions, people might not see the
change of colours correctly so the differences in movement, shapes and size
should be used as a redundancy measure. This consideration will be also
helpful for people with red-green visual impairment, guaranteeing
inclusiveness in the design.

####  5.2.2. DR 3

Increasing the number of visual cues in relation to vehicle-to-pedestrian
distances and the vehicle’s speed: From far distances, people are most
sensitive to changes in colour. At closer distances and when the vehicle slows
down, more detailed visual elements, such as patterns and animations, or even
explicit information, such as text, can be perceived.

####  5.2.3. DR 4

Providing additional information during the crossing procedure: Pedestrians
feel more confident if they receive information about the time they have to
finish the crossing. Additional information, in form of a timer, could be
visualised using an explicit layer of text (re DR 3). Though under optimal
conditions, the autonomous vehicle would wait until pedestrians have finished
the crossing procedure, additional real-time visualisations indicating that
the vehicle is still aware of the pedestrian’s presence, could increase the
perceived trust.

###  5.3. Catering for Pedestrian Non-Detection

Participants expressed the need to receive more detailed information about the
vehicle’s decision to start the engine again. Even when they followed the
instructions correctly and made safe decisions to cross the road, they were
uncertain what triggered the car to change the visualisation. After
experiencing scenario B when the car sensor failed in the middle of the
crossing procedure, two participants (P5, P7) expressed their concern about
the uncertainty of how long they had time to finish the crossing, stating:
“What makes me feel unsafe is how long the car would be waiting for me” (P5);
and “I’m not sure if I have enough time to go back to the safe area” (P7).

####  5.3.1. DR 5

Complying with full colour sequences and adding additional caution signals
when a sequence is not followed: In malfunction scenarios, when the full
visualisation sequence (i.e. red-yellow-green) is not followed correctly,
pedestrians can be confused due to not having seen the full sequence of
signals (re: DR 1 - traffic light scheme). Therefore, we suggest that
additional visualisations need to be designed for scenarios in which the full
sequence is not followed (e.g. due to sensor malfunctioning). For example, if
the visualisation sequence transitions from slowing down (yellow) to
accelerating (red), an additional visual cue should be added to inform
pedestrians about potential system failures.

####  5.3.2. DR 6

Signalling clearly if a sudden non-detection occurs during the crossing
procedure: Building on DR 5, in case of a sensor failing to detect the
pedestrian after successfully having stopped for a pedestrian, the car should
not immediately start accelerating. To ensure the safety of pedestrians, it
should display a countdown timer - possibly supported through audio cues - to
signal its intention to any pedestrian that might potentially still be
attempting to cross.

###  5.4. Pedestrian’s Negotiation with Vehicles

Previous research has demonstrated the current methods of communication with
autonomous vehicles such as using hand gestures or stepping forward to
communicate the intention to the cars (Sucha et al., 2017). Such behaviour was
observed in this study as well. “I stepped down to signify that I wanted to
cross” (P10). Participants also felt curious about how the car identified
whether they were going to cross. “Why didn’t it change back to green when I
made clear that I want to cross this time.” (P5 tried to step forward to
signal her intent of crossing to the car)

Participants did not appreciate the AV giving them commands. For instance, “I
feel like the car was commanding me to do things. I don’t like that.” (P14).
Pedestrians also mentioned that there was “no negotiation” (P2, P6) between
them and the AV. This was due to having no real-time feedback to respond to a
participant’s gesture when they intended to cross.

####  5.4.1. DR 7

Supporting bidirectional communication between AVs and pedestrians: The AVs
need to be able to sense and interpret common pedestrian behaviour - e.g.
stepping onto the road, making hand gestures or gazing at the vehicle - and
react accordingly. Their interpretation and corresponding response need to be
communicated in the visualisation.

###  5.5. Limitations

Among 18 participants, we had 14 international participants who recently moved
to or had stayed for less than 5 years in Sydney, where the study was
conducted. Their crossing behaviour may have been influenced by their cultural
background and norms from their home cities. This represents a factor that we
did not control in our study.

VR has been shown to be a useful testing medium for interaction between
pedestrian participants and AVs. However, using a simulated environment meant
that we did not consider real-world environmental factors such as sounds,
changing lighting conditions, weather conditions, other pedestrians, and
vehicles - all conditions that may influence pedestrians’ behaviours (Pillai,
2017). Further, there was a spatial difference between VR and the real world:
5 participants (P2, P5, P6, P9, P11) displayed a loss of balance when stepping
down to the road for the first time in the VR environment as they expected a
step when they were actually moving on a flat surface. This spatial difference
may have affected our participants’ behaviour to some extent. Similarly, the
requirement of wearing a VR headset may have had an effect on how people moved
and behaved in the VR environment. In our study, only one participant
expressed that the VR headset felt uncomfortable. We were able to address this
potential issue to some extent by getting participants to take the headset off
between scenarios and sitting down in a comfortable booth for the post-
scenario interviews.

##  6\. Conclusions and Future Work

In this research, we designed a novel projection-based design solution to
enable AV-to-pedestrian communication and evaluated it with participants in VR
with four street-crossing scenarios, three of which involved varied degrees of
sensor malfunction. The evaluation gave us insights into people’s
understanding of communication messages and their behaviour around autonomous
vehicles. Based on the findings from the evaluation, we proposed seven design
recommendations for projection-based AV-to-pedestrian communication systems
(see Table 1).

In summary, our research offers three contributions for future work on AV-to-
pedestrian communication:

  1. (1)

We explored a novel projection-based design solution for communication between
AVs and pedestrians.

  2. (2)

We contributed insights on how people understand and interpret projection-
based visual communication signals, including malfunction (non-detection)
scenarios.

  3. (3)

We presented a series of design recommendations to guide future design
exploration.

Future studies can build on our work to investigate scenarios that involve
multiple vehicles and pedestrians at complex crossings. Such studies might
help to refine and expand our initial set of design recommendations for this
emerging field of research. In particular, our findings highlight three areas
for future exploration. First, in the context of projections, solutions for
diverse road surfaces, weather conditions and lighting conditions could be
considered. Second, a targeted investigation of malfunctions (leading to non-
detection) and their long-term influence on (over-)trust and behaviour of
pedestrians could reveal additional design considerations and valuable
insights for the design of future AVs. Third, as visualising a car’s intent
can add important information for pedestrians, future research could
investigate how long in advance intentions should be visualised by an AV.

## References

  * (1)
  * Ainslie (2010) Michael Ainslie. 2010.  _Principles of Sonar Performance Modelling_.  https://doi.org/10.1007/978-3-540-87662-5
  * Beggiato et al. (2017) Matthias Beggiato, Claudia Witzlack, and Josef F. Krems. 2017.  Gap Acceptance and Time-To-Arrival Estimates As Basis for Informal Communication Between Pedestrians and Vehicles. In _Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_ _(AutomotiveUI ’17)_. ACM, New York, NY, USA, 50–57.  https://doi.org/10.1145/3122986.3122995
  * Chang et al. (2018) Chia-Ming Chang, Koki Toda, Takeo Igarashi, Masahiro Miyata, and Yasuhiro Kobayashi. 2018.  A Video-based Study Comparing Communication Modalities Between an Autonomous Car and a Pedestrian. In _Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_ _(AutomotiveUI ’18)_. ACM, New York, NY, USA, 104–109.  https://doi.org/10.1145/3239092.3265950
  * Chang et al. (2017) Chia-Ming Chang, Koki Toda, Daisuke Sakamoto, and Takeo Igarashi. 2017.  Eyes on a Car: An Interface Design for Communication Between an Autonomous Car and a Pedestrian. In _Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_ _(AutomotiveUI ’17)_. ACM, New York, NY, USA, 65–73.  https://doi.org/10.1145/3122986.3122989
  * Clamann et al. (2016) Clamann, Michael, Aubert, Miles, Cummings, and Mary L. 2016.  Evaluation of Vehicle-to-Pedestrian Communication Displays for Autonomous Vehicles.  https://trid.trb.org/View/1437891
  * Colley et al. (2018) Ashley Colley, Jonna Häkkilä, Meri-Tuulia Forsman, Bastian Pfleging, and Florian Alt. 2018.  Car Exterior Surface Displays: Exploration in a Real-World Context. In _Proceedings of the 7th ACM International Symposium on Pervasive Displays_ _(PerDis ’18)_. ACM, New York, NY, USA, Article 7, 8 pages.  https://doi.org/10.1145/3205873.3205880
  * Colley et al. (2017) Ashley Colley, Jonna Häkkilä, Bastian Pfleging, and Florian Alt. 2017.  A Design Space for External Displays on Cars. In _Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct_ _(AutomotiveUI ’17)_. ACM, New York, NY, USA, 146–151.  https://doi.org/10.1145/3131726.3131760
  * Dancu et al. (2015) Alexandru Dancu, Velko Vechev, Adviye Ayca Unluer, Simon Nilson, Oscar Nygren, Simon Eliasson, Jean-Elie Barjonet, Joe Marshall, and Morten Fjeld. 2015.  Gesture Bike: Examining Projection Surfaces and Turn Signal Systems for Urban Cycling.  https://doi.org/10.1145/2817721.2817748
  * Davies (2018) Nigel Davies. 2018.  Saturated Display Environments.  _Keynote at International Symposium on Pervasive Displays (PerDis ’18)_ (2018).
  * Dey and Terken (2017) Debargha Dey and Jacques Terken. 2017.  Pedestrian Interaction with Vehicles: Roles of Explicit and Implicit Communication. In _Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_ _(AutomotiveUI ’17)_. ACM, New York, NY, USA, 109–113.  https://doi.org/10.1145/3122986.3123009
  * Fridman et al. (2017) Lex Fridman, Bruce Mehler, Lei Xia, Yangyang Yang, Laura Yvonne Facusse, and Bryan Reimer. 2017.  To Walk or Not to Walk: Crowdsourced Assessment of External Vehicle-to-Pedestrian Displays.  _CoRR_ abs/1707.02698 (2017).  arXiv:1707.02698 http://arxiv.org/abs/1707.02698
  * Frison et al. (2017) Anna-Katharina Frison, Bastian Pfleging, Andreas Riener, Myounghoon Philart Jeon, Ignacio Alvarez, and Wendy Ju. 2017.  Workshop on User-Centered Design for Automated Driving Systems. In _Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct_ _(AutomotiveUI ’17)_. ACM, New York, NY, USA, 22–27.  https://doi.org/10.1145/3131726.3131734
  * Gandia et al. (2019) Rodrigo Marçal Gandia, Fabio Antonialli, Bruna Habib Cavazza, Arthur Miranda Neto, Danilo Alves de Lima, Joel Yutaka Sugano, Isabelle Nicolai, and Andre Luiz Zambalde. 2019.  Autonomous vehicles: scientometric and bibliometric review.  _Transport Reviews_ (2019).  https://doi.org/10.1080/01441647.2018.1518937
  * Guéguen et al. (2015) Nicolas Guéguen, Sebastien Meineri, and Chloé Eyssartier. 2015.  A pedestrian’s stare and drivers’ stopping behavior: A field experiment at the pedestrian crossing.  _Safety Science_ 75 (06 2015).  https://doi.org/10.1016/j.ssci.2015.01.018
  * Habibovic et al. (2018) Azra Habibovic, Victor Malmsten Lundgren, Jonas Andersson, Maria Klingegård, Tobias Lagström, Anna Sirkka, Johan Fagerlönn, Claes Edgren, Rikard Fredriksson, Stas Krupenia, Dennis Saluäär, and Pontus Larsson. 2018.  Communicating Intent of Automated Vehicles to Pedestrians.  _Frontiers in Psychology_ 9 (2018), 1336.  https://doi.org/10.3389/fpsyg.2018.01336
  * Holländer (2018) Kai Holländer. 2018.  Applying the User-Centered Design Process to External Car Displays. In _Mensch und Computer 2018 - Workshopband_ , Raimund Dachselt and Gerhard Weber (Eds.). Gesellschaft für Informatik e.V., Bonn.
  * International (2018) SAE International. 2018.  _Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles_.  https://doi.org/10.4271/J3016_201806
  * Jones (2018) Lee Jones. 2018.  Your Body of Water: A Display that Visualizes Aesthetic Heart Rate Data from a 3D Camera.  Figure 1 (2018), 286–291.  https://doi.org/10.1145/3173225.3173284
  * Lazar et al. (2010) Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2010.  _Research Methods in Human-Computer Interaction_.  Wiley Publishing.
  * Li et al. (2018) Yeti Li, Murat Dikmen, Thana Hussein, Yahui Wang, and Catherine Burns. 2018.  To Cross or Not to Cross: Urgency-Based External Warning Displays on Autonomous Vehicles to Improve Pedestrian Crossing Safety.  https://doi.org/10.1145/3239060.3239082
  * Mahadevan et al. (2018) Karthik Mahadevan, Sowmya Somanath, and Ehud Sharlin. 2018.  Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction. In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_ _(CHI ’18)_. ACM, New York, NY, USA, Article 429, 12 pages.  https://doi.org/10.1145/3173574.3174003
  * Milakis et al. (2017) Dimitris Milakis, Bart van Arem, and Bert van Wee. 2017.  Policy and society related implications of automated driving: A review of literature and directions for future research.  _Journal of Intelligent Transportation Systems_ 21, 4 (2017), 324–348.  https://doi.org/10.1080/15472450.2017.1291351 arXiv:https://doi.org/10.1080/15472450.2017.1291351
  * NISSAN MOTOR Co. ([n. d.]) Ltd NISSAN MOTOR Co. [n. d.].  GPS Mobile ITS for Pedestrian Safety.  Retrieved April 2, 2019 from https://www.nissan-global.com/EN/TECHNOLOGY/OVERVIEW/gps_mobile_its.html.
  * Owensby et al. (2018) Chelsea Owensby, Martin Tomitsch, and Callum Parker. 2018.  A framework for designing interactions between pedestrians and driverless cars.  _Proceedings of the 30th Australian Conference on Computer-Human Interaction_ (2018), 359–363.  https://doi.org/10.1145/3292147.3292218
  * Pfleging et al. (2016) Bastian Pfleging, Maurice Rang, and Nora Broy. 2016.  Investigating User Needs for Non-driving-related Activities During Automated Driving. In _Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia_ _(MUM ’16)_. ACM, New York, NY, USA, 91–99.  https://doi.org/10.1145/3012709.3012735
  * Pietrowicz and Karahalios (2013) Mary Pietrowicz and Karrie Karahalios. 2013.  Sonic Shapes : Vocal Sonic Shapes :.  _International Conference on Auditory Display (ICAD) 2013_ (2013), 157–164.
  * Pillai (2017) Anantha Pillai. 2017.  Virtual Reality based Study to Analyse Pedestrian Attitude towards Autonomous Vehicles.  (2017).
  * Rasouli et al. (2017) A. Rasouli, I. Kotseruba, and J. K. Tsotsos. 2017.  Agreeing to cross: How drivers and pedestrians communicate. In _2017 IEEE Intelligent Vehicles Symposium (IV)_. 264–269.  https://doi.org/10.1109/IVS.2017.7995730
  * Rasouli et al. (2017) Amir Rasouli, Iuliia Kotseruba, and John K. Tsotsos. 2017.  Are They Going to Cross? A Benchmark Dataset and Baseline for Pedestrian Crosswalk Behavior. In _The IEEE International Conference on Computer Vision (ICCV) Workshops_.
  * Rasouli et al. (2018a) A. Rasouli, I. Kotseruba, and J. K. Tsotsos. 2018a.  Towards Social Autonomous Vehicles: Understanding Pedestrian-Driver Interactions. In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_. 729–734.  https://doi.org/10.1109/ITSC.2018.8569324
  * Rasouli et al. (2018b) A. Rasouli, I. Kotseruba, and J. K. Tsotsos. 2018b.  Understanding Pedestrian Behavior in Complex Traffic Scenes.  _IEEE Transactions on Intelligent Vehicles_ 3, 1 (March 2018), 61–70.  https://doi.org/10.1109/TIV.2017.2788193
  * Rasouli and Tsotsos (2018) Amir Rasouli and John K. Tsotsos. 2018.  Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice.  _IEEE Transactions on Intelligent Transportation Systems_ PP (2018), 1–19.  https://doi.org/10.1109/TITS.2019.2901817
  * Rasouli and Tsotsos (2019) A. Rasouli and J. K. Tsotsos. 2019.  Autonomous Vehicles That Interact With Pedestrians: A Survey of Theory and Practice.  _IEEE Transactions on Intelligent Transportation Systems_ (2019), 1–19.  https://doi.org/10.1109/TITS.2019.2901817
  * Rothenbücher et al. (2016) D. Rothenbücher, J. Li, D. Sirkin, B. Mok, and W. Ju. 2016.  Ghost driver: A field study investigating the interaction between pedestrians and driverless vehicles. In _2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)_. 795–802.  https://doi.org/10.1109/ROMAN.2016.7745210
  * Shelton and Nesbitt (2017) Ben Shelton and Keith Nesbitt. 2017.  Evaluating WaveWatch: an ambient display of web traffic.  _Proceedings of the Australasian Computer Science Week Multiconference, ACSW 2017, Geelong, Australia, January 31 - February 3, 2017_ (2017), 9:1–9:9.  https://doi.org/10.1145/3014812.3014821
  * Slater et al. (2009a) Mel Slater, Pankaj Khanna, Jesper Mortensen, and Insu Yu. 2009a.  Response in an Immersive Virtual.  _IEEE Computer Graphics and Applications_ (2009), 76–84.  https://doi.org/10.1109/MCG.2009.55
  * Slater et al. (2009b) Mel Slater, Beau Lotto, Maria Marta Arnold, and Maria V. Sanchez-Vives. 2009b.  How we experience immersive virtual environments: The concept of presence and its measurement.  _Anuario de Psicologia_ 40, 2 (2009), 193–210.
  * Slater et al. (2002) Mel Slater, Martin Usoh, and Anthony Steed. 2002.  Taking steps: the influence of a walking technique on presence in virtual reality.  _ACM Transactions on Computer-Human Interaction_ 2, 3 (2002), 201–219.  https://doi.org/10.1145/210079.210084
  * Sucha et al. (2017) Matus Sucha, Daniel Dostal, and Ralf Risser. 2017.  Pedestrian-driver communication and decision strategies at marked crossings.  _Accident Analysis and Prevention_ 102 (2017), 41–50.  https://doi.org/10.1016/j.aap.2017.02.018
  * Van Tonder and Lyons (2005) Gert J. Van Tonder and Michael J. Lyons. 2005.  Visual perception in Japanese rock garden design.  _Axiomathes_ 15, 3 (2005), 353–371.  https://doi.org/10.1007/s10516-004-5448-8
  * Yang et al. (2017) X. Jessie Yang, Vaibhav V. Unhelkar, Kevin Li, and Julie A. Shah. 2017.  Evaluating Effects of User Experience and System Transparency on Trust in Automation.  _2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI_ (2017), 408–416.  https://doi.org/10.1145/2909824.3020230
  * Yannis et al. (2013) George Yannis, Eleonora Papadimitriou, and A Theofilatos. 2013.  Pedestrian gap acceptance for mid-block street crossing.  _Transportation planning and technology_ 36, 5 (2013), 450–462.
  * Zhang et al. (2018) Jingyi Zhang, Erik Vinkhuyzen, and Melissa Cefkin. 2018.  Evaluation of an Autonomous Vehicle External Communication System Concept: A Survey Study. In _Advances in Human Aspects of Transportation_ , Neville A Stanton (Ed.). Springer International Publishing, Cham, 650–661.
  * Zhuang and Wu (2014) Xiangling Zhuang and Changxu Wu. 2014.  Pedestrian gestures increase driver yielding at uncontrolled mid-block road crossings.  _Accident Analysis & Prevention_ 70 (2014), 235 – 244.  https://doi.org/10.1016/j.aap.2013.12.015


# Optical conductivity as a probe of the interaction-driven metal in
rhombohedral trilayer graphene

Vladimir Juričić<EMAIL_ADDRESS><EMAIL_ADDRESS>Departamento de
Física, Universidad Técnica Federico Santa María, Casilla 110, Valparaíso,
Chile.  Nordita, KTH Royal Institute of Technology and Stockholm University,
Hannes Alfvéns väg 12, 106 91 Stockholm, Sweden  Enrique Muñoz
<EMAIL_ADDRESS>Facultad de Física, Pontificia Universidad Católica de
Chile, Vicuña Mackenna 4860, Santiago, Chile  Rodrigo Soto-Garrido
<EMAIL_ADDRESS>Facultad de Física, Pontificia Universidad Católica de Chile,
Vicuña Mackenna 4860, Santiago, Chile

###### Abstract

Study of the strongly correlated states in van der Waals heterostructures is
one of the central topics in modern condensed matter physics. Among these, the
rhombohedral trilayer graphene (RTG) occupies a prominent place since it hosts
a variety of interaction-driven phases, with the metallic ones yielding exotic
superconducting orders upon doping [H. Zhou et al., Nature 598, 429 (2021);
ibid. 598, 434 (2021)]. Motivated by these experimental findings, we show
within the framework of the low-energy Dirac theory that the optical
conductivity can distinguish different candidates for a paramagnetic metallic
ground state in this system. In particular, this observable shows a single
peak in the fully gapped valence-bond state. On the other hand, the bond-
current state features two pronounced peaks in the optical conductivity as the
probing frequency increases. Finally, the rotational symmetry breaking charge-
density wave exhibits a minimal conductivity with the value independent of the
amplitude of the order parameter, which corresponds precisely to the splitting
of the two cubic nodal points at the two valleys into two triplets of the band
touching points featuring linearly dispersing quasiparticles. These features
represent the smoking gun signatures of different candidate order-parameters
for the paramagnetic metallic ground state, which should motivate further
experimental studies of the RTG.

_Introduction._ Quasi-two-dimensional graphene-based van der Waals (vdW)
heterostructures, such as bilayer and trilayer graphene, have recently emerged
as a groundbreaking territory for the discovery of new electronic states of
quantum matter driven by the electron interactions [1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12]. It is rather remarkable that by externally tuning the twist
angle, doping, and/or the magnetic field a new landscape of exotic insulating,
metallic and superconducting states has been unearthed in these systems.
Particularly prominent in this respect is the interplay between the metallic
and the superconducting phases that gives rise to very rich phase diagrams
(for a recent review, see Ref. [13]). However, their theoretical understanding
is often hampered by the difficulty in distinguishing possible candidate
ground states in these systems, as, for instance, when considering the
emergence of superconductivity from a parent metallic state.

Rhombohedral trilayer graphene (RTG) has recently emerged as a rather
prominent example in this respect, where superconducting instabilities are in
proximity to metallic ground states in different doping regimes [11, 12], with
a few theoretical scenarios proposed to explain the rich phenomenology [14,
15, 16, 17, 18, 19, 20, 21]. Most interestingly, very little is known about an
exotic metal in the proximity of the experimentally identified SC1 order [12],
except that it exhibits paramagnetic nature (see Extended Fig. 7 in Ref. [12]
and the discussion therein). In fact, a few candidates for such a state that
may be driven by electron interactions have been identified, each of them
breaking different microscopic symmetries: the valence bond order (VBO), bond-
current order (BCO) and smectic charge-density wave (sCDW) orders [16],
respectively.

The optical conductivity is a well established tool in studying correlated
electron materials, which is directly related to the excitation spectrum [22].
In particular, it has been studied in various vdW materials both theoretically
[23, 24, 25, 26, 27, 28] and experimentally [29, 30]. In this work, we focus
on the collisionless or high-frequency regime of the optical conductivity,
pertaining to the frequencies $\hbar\omega\gg k_{B}T$, since in this regime
this observable shows a universal scaling that depends only on the form of the
dispersion of the low-energy quasiparticles, the dimensionality of the system,
and the scaling dimension of the electron-electron interactions [31].
Importantly, the scaling dimension of the optical conductivity in a
$d$-dimensional system is equal to $d-2$ in units of momentum. Therefore,
exactly in $d=2$, at finite frequency and temperature, with other energy
scales set to zero, $\sigma(\omega,T)=(e^{2}/h)f(\hbar\omega/k_{B}T)$, with
$f(x)$ as a universal dimensionless scaling function. In the limit
$x\to\infty$, this function tends to a constant, yielding a universal
amplitude for the collisionless optical conductivity.

Figure 1: Real part of the optical conductivity (in units of $e^{2}/\hbar$) in
the collisionless regime for: (a) the valence-bond order (VBO); (b) the bond
current order (BCO); (c) smectic charge-density wave (sCDW), respectively,
given by Eqs. (6), (8), (13), at the neutrality point $\mu=0$ and at $T=0$.
$u$ and $\Delta$ are given in units of the bandwidth scale $t$, see also the
discussion after Eq. 1. In panels (a) and (b), the green dashed line
corresponds to the universal optical conductivity for the non-interacting
spinless RTG in the collisionless regime, $\sigma_{0}=3/8$.

Motivated by these developments, we show that the collisionless optical
conductivity can distinguish different candidate paramagnetic metallic ground
states in RTG. In particular, this observable shows a single peak in the fully
gapped valence-bond state, as displayed in Fig. 1(a). On the other hand, the
bond-current state features two pronounced peaks in the optical conductivity
as the probing frequency increases, see Fig. 1(b). This behavior can be
directly related to the Dirac nature of the valence-bond and the bond-current
order-parameters obeying, respectively, anticommutation and commutation
relations with the single-particle noninteracting Hamiltonian given by Eq.
(1), and with the behavior of the density of states (DOS); see Figs 2(a) and
(b). Finally, the sCDW state is characterized by a minimal conductivity, which
is independent of the amplitude of the sCDW order parameter [Fig. 1(c)], and
corresponds precisely to the splitting of the two cubic nodal points at the
two valleys into two triplets of linearly dispersing band touching points
(Fig. 3). This behavior is also consistent with the form of the DOS,
particularly at low energies, displaying the linear scaling with the energy
[Fig. 2(c)]. As such, these characteristic features in the optical
conductivity represent the smoking gun signatures of different candidate
order-parameters for the paramagnetic metallic ground state of the RTG. These,
in turn, can serve as a starting point for the study of the superconductivity
in this system and should motivate further its experimental studies.

_Model._ We consider the effective low energy model for the rhombohedral
($ABC$ stacked) trilayer graphene obtained after integrating out high-energy
degrees of freedom corresponding to the four gapped bands consisting of the
states at the dimerized sites [one in the bottom ($A$), two in the middle
($B$), and one in the top ($C$) layer] [32, 33, 34]. Taking into account the
layer (or equivalently sublattice) and valley degrees of freedom, the single-
particle Hamiltonian for noninteracting electrons reads

$\displaystyle
H_{0}=\alpha\left[f_{1}(\bm{k})\Gamma_{31}+f_{2}(\bm{k})\Gamma_{02}\right]+u\Gamma_{03}-\mu\Gamma_{00}$
(1)

where $\alpha=t^{3}_{0}a^{3}/t_{\perp}$, $a\simeq 0.25$nm is the lattice
spacing within the single graphene layer, while $t_{0}\simeq 2.5$eV and
$t_{\perp}\simeq 0.5$eV are the intralayer and the interlayer nearest-neighbor
hoppings, respectively [33]. The corresponding bandwidth is given by the
cutoff scale for the low-energy model in Eq. 1, $t\sim t_{\perp}\sim 0.5$eV.
The form factors $f_{1}(\bm{k})=k_{x}(k^{2}_{x}-3k^{2}_{y})$,
$f_{2}(\bm{k})=-k_{y}(k^{2}_{y}-3k^{2}_{x})$, respectively, transform under
$A_{1u}$ and $A_{2u}$ representations of the $D_{3d}$ point group of the RTG.
Momentum $\bm{k}$ is measured from the respective band-touching points
(valleys). Electron (hole) doping corresponds to $\mu>0\;(\mu<0)$, and we set
$\hbar=k_{B}=e=1$ hereafter. The four-dimensional matrices are
$\Gamma_{\mu\nu}=\tau_{\mu}\sigma_{\nu}$, where $\\{\tau_{\mu}\\}$ and
$\\{\sigma_{\nu}\\}$ are the sets of Pauli matrices that act on the valley and
sublattice (layer) indices, respectively. Since we here consider only
paramagnetic metallic states, we suppress the spin indices, see also Sec. S1
of the Supplemental Information (SI) [35]. Taking the possible patterns of
symmetry breaking and the low-energy degrees of freedom, three candidates
emerge in this respect [16]. First of all, the VBO, which fully gaps the
system out and breaks the sublattice symmetry. Second, the BCO breaks, besides
the sublattice symmetry, also the time-reversal. Finally, the sCDW breaks both
the U(1) rotational symmetry about the $z-$axis, generated by the matrix
$\Gamma_{33}$, down to the discrete $C_{3}$ subgroup, and the lattice-
translational symmetry, generated by the matrix $\Gamma_{30}$. These three
orders are represented by the following matrices ($\rho=1,2$):

$\Gamma_{\rho 1}\,(\text{VBO}),\>\Gamma_{\rho 2}\,(\text{BCO}),\>(\Gamma_{\rho
0},\Gamma_{\rho 3})\,(\text{sCDW}).$ (2)

The corresponding irreducible representations of the $D_{3}$ group are $A_{1}$
and $A_{2}$, for the VBO and BCO, respectively, while the sCDW transforms
under the two-dimensional $E$ representation. Notice that we now reduce the
symmetry down to $D_{3}$ subgroup of the full $D_{3d}$ point group of the
noninteracting low-energy Hamiltonian in Eq. (1) because we allow for the
backscattering processes that mix the valleys, and in turn may yield the
translational-symmetry breaking orders, such as the sCDW.

_Optical conductivity._ To distinguish different candidate paramagnetic
metallic ground states in RTG, we now compute the optical conductivity. To
this end, we use the Kubo formula for the linear optical conductivity [36]

$\displaystyle\sigma_{ij}(\Omega)$
$\displaystyle=\lim_{i\Omega_{n}\to\Omega+i0^{+}}\frac{i\Pi_{ij}(i\Omega_{n})}{\Omega},$
(3)

where the polarization tensor reads

$\displaystyle\Pi_{ij}(i\Omega_{n})=$
$\displaystyle-T\sum_{n}\int\frac{d^{2}k}{(2\pi)^{2}}\text{Tr}\left[\hat{v}_{i}G(i\omega_{n}+i\Omega_{n},\bm{k})\right.$
$\displaystyle\left.\times\hat{v}_{j}G(i\omega_{n},\bm{k})\right].$ (4)

Here, the velocity operator $\hat{v}_{i}=\partial H/\partial k_{i}$,
$G(i\omega_{n},\bm{k})=[i\omega_{n}-H]^{-1}$ is the Matsubara Green’s
function, $\omega_{n}=(2n+1)\pi T$ are the fermionic Matsubara frequencies,
and the analytical continuation onto real frequencies is carried out,
$i\Omega_{n}\to\Omega+i0^{+}$, see also Sec. S2 of the SI for details [35].

Figure 2: The density of states (DOS) in the candidate paramagnetic states.
(a) valence-bond order (VBO); (b) bond current order(BCO); (c) smectic charge-
density wave(sCDW). The DOS in the three phases is given in Eqs. 7, Optical
conductivity as a probe of the interaction-driven metal in rhombohedral
trilayer graphene, and (S69) in the SI. The energy ($E$) is in units of the
bandwidth ($t$), see also Fig. 1. $u$ and $\Delta$ are given in units of $t$.
The inset in (c) corresponds to the DOS for the noninteracting case, where the
low-energy DOS scales as $|E|^{-1/3}$.

_Valence bond order._ We start by computing the optical conductivity for the
simplest case, the VBO. The Hamiltonian is given by
$H=H_{0}+\Delta_{1}\Gamma_{11}+\Delta_{2}\Gamma_{21}$. Notice that since time-
reversal symmetry (TRS) is preserved, the $xy$ component of the polarization
tensor vanishes. In addition, since rotational symmetry is conserved,
$\Pi_{xx}=\Pi_{yy}$, so we need to compute only one of the polarization tensor
components. We then find, with the details shown in the SI, Sec. S3.A [35],

$\displaystyle\Pi_{xx}(\Omega)$ $\displaystyle=$ $\displaystyle
72\alpha^{2}\int\frac{d^{2}k}{(2\pi)^{2}}\frac{k^{4}\left(u^{2}+\Delta^{2}+\alpha^{2}k^{4}k_{x}^{2}\right)}{E(k)\left[4E^{2}(k)+\Omega_{n}^{2}\right]}$
(5) $\displaystyle\times\Theta(\Omega-2\sqrt{u^{2}+\Delta^{2}})\delta
n_{F}(E),$

where $E(k)=\sqrt{\alpha^{2}k^{6}+u^{2}+\Delta^{2}}$ is the quasiparticle
dispersion, $n_{F}(z)=\left(e^{z/T}+1\right)^{-1}$ is the Fermi-Dirac
distribution, and we defined $\delta n_{F}(E)=n_{F}(-E-\mu)-n_{F}(E-\mu)$.
After performing the analytic continuation to real frequency,
$i\Omega_{n}\rightarrow\Omega+i0^{+}$, we obtain the real part for the optical
conductivity,

$\displaystyle\text{Re}{\sigma}_{xx}(\Omega)$ $\displaystyle=$
$\displaystyle-\frac{\text{Im}{\,}\Pi_{xx}(\Omega)}{\Omega}$ (6)
$\displaystyle=$
$\displaystyle\sigma_{0}\frac{\left(\Omega^{2}+4(u^{2}+\Delta^{2})\right)}{\Omega^{2}}\delta
n_{F}\left(|\Omega|/2\right)$
$\displaystyle\times\Theta(\Omega-2\sqrt{u^{2}+\Delta^{2}}),$

where $\sigma_{0}=3/8$ is the universal optical conductivity for the
noninteracting spinless RTG in the collisionless regime (in units
$e^{2}/\hbar$), and we defined $\delta n_{F}(E)=n_{F}(-E-\mu)-n_{F}(E-\mu)$.
The technical details are given in Sec. S3.A of the SI [35].

In Fig. 1(a) we can see a plot for the optical conductivity at zero
temperature and at the neutrality point $\mu=0$. Notice that the optical
conductivity is zero until $\Omega\geq 2\sqrt{u^{2}+\Delta^{2}}$, where there
is a maximum. This is expected, since $2\sqrt{u^{2}+\Delta^{2}}$ is the gap
between the conducting and valence bands. The asymptotic value is given by
$\text{Re}{e}\sigma_{xx}(\Omega)\to\sigma_{0}$ when
$\Omega\gg\sqrt{u^{2}+\Delta^{2}}$.

To further elucidate the origin of the peaks in the optical conductivity in
the VBO, we analyze the corresponding DOS

$\displaystyle
N_{VBO}(E)=\frac{|E|({E^{2}-u^{2}-\Delta^{2}})^{-\frac{2}{3}}}{6\pi\alpha^{2/3}}\Theta(|E|-\sqrt{u^{2}+\Delta^{2}}).$
(7)

A remarkable feature of this expression is that the DOS develops a pole at the
minimum of the conduction band $E=\sqrt{u^{2}+\Delta^{2}}$, and hence it grows
as compared to the gapless case $u=\Delta=0$. This effect manifests in an
enhancement of the optical conductivity with the gap as compared to the
universal value $\sigma_{0}$ for the gapless case, as directly follows from
Eq. (6).

_Bond current order._ We now follow the same procedure for the BCO case, where
the Hamiltonian is given by
$H=H_{0}+\Delta_{1}\Gamma_{12}+\Delta_{2}\Gamma_{22}$. As for the VBO case,
the rotation symmetry is preserved in the BCO, so $\Pi_{xx}=\Pi_{yy}$. The
real part of the optical conductivity then reads

$\displaystyle\text{Re}{e}\sigma_{xx}(\Omega)$ $\displaystyle=$
$\displaystyle\frac{3}{2\Omega^{2}}\left[\Theta\left(|\Omega|-2u\right)\Theta\left(2\sqrt{u^{2}+\Delta^{2}}-|\Omega|\right)\sum_{j=1,2}\mathcal{F}_{-}[\epsilon_{j}]\delta
n_{F}(E_{-}(\epsilon_{j}))+\Theta\left(|\Omega|-2\sqrt{u^{2}+\Delta^{2}}\right)\right.$
(8)
$\displaystyle\times\left.\left\\{\sum_{s=\pm}\mathcal{G}_{s}[\epsilon_{3}]\delta
n_{F}(E_{s}(\epsilon_{3}))+\mathcal{F}_{+}[\epsilon_{2}]\delta
n_{F}(E_{+}(\epsilon_{2}))\right\\}\right].$

Here, the functions $\mathcal{F}_{\pm}(\epsilon)$ and
$\mathcal{G}_{\pm}(\epsilon)$ are defined by Eq. (S.45) in the SI, together
with the dispersions

$\displaystyle E_{\pm}(\epsilon)$ $\displaystyle=$
$\displaystyle\sqrt{\left(\sqrt{\epsilon^{2}-u^{2}}\pm\Delta\right)^{2}},$ (9)

and the frequency-dependent coefficients

$\displaystyle\epsilon_{1,2}$ $\displaystyle=$
$\displaystyle\sqrt{u^{2}+\left(\Delta\pm\sqrt{\frac{\Omega^{2}}{4}-u^{2}}\right)^{2}},$
$\displaystyle\epsilon_{3}$ $\displaystyle=$
$\displaystyle\sqrt{\frac{\Omega^{2}}{4}+\frac{u^{2}\Delta^{2}}{\Delta^{2}-\Omega^{2}/4}}.$
(10)

The optical conductivity is given in Fig. 1(b), and features two peaks, which
represent hallmarks of this state. It is worthwhile noticing that the weight
of the conductivity peak at the lower frequency scales inversely with the
amplitude of the order-parameter $\Delta$. For the technical details, consult
Sec. S3.B of the SI [35].

To further shed light on the presence of the two peaks in the optical
conductivity, we analyze the corresponding DOS, with the form given by (Sec.
S3.B [35])

$\displaystyle
N_{BC}(E)=\frac{|E|}{6\pi\alpha^{2/3}}\frac{1}{\sqrt{E^{2}-u^{2}}}\left[\left(\sqrt{E^{2}-u^{2}}-\Delta\right)^{-\frac{1}{3}}\right.$
$\displaystyle\times\left.\Theta(E^{2}-u^{2}-\Delta^{2})+\left(\sqrt{E^{2}-u^{2}}+\Delta\right)^{-\frac{1}{3}}\Theta(E^{2}-u^{2})\right]$
(11)

where we used that in this case, the dispersion in each of the four bands is
given by $E(k)=\pm\sqrt{u^{2}+(\alpha k^{3}\pm\Delta)^{2}}$. This form of the
band structure with two valence and two conduction bands, yielding the peaks
in the DOS [Eq. (Optical conductivity as a probe of the interaction-driven
metal in rhombohedral trilayer graphene)], is therefore consistent with the
form of optical conductivity in this phase, as shown in Fig. 1(b). We now
analyze this observable in the remaining candidate phase for the paramagnetic
metal, namely, the sCDW phase.

_Charge density wave._ Smectic charge-density wave yields a peculiar structure
of the dispersion as it transforms under the two-dimensional $E$
representation of the $D_{3}$ point group, with the vector realized in the
valley space. As such, the sCDW mixes the valleys, with the bands given by

$E(\bm{k})=\pm\sqrt{u^{2}+\alpha^{2}k^{6}+\Delta^{2}\pm
2\Delta\sqrt{u^{2}+\alpha^{2}k^{6}\sin^{2}3\phi}},$ (12)

and the corresponding band structure shown in Fig. 3. In fact, the sCDW splits
two cubic band touching points, with the voriticity $\pm 3\pi$, living in the
two valleys, into two triplets of the points with the vorticity $\pm\pi$, as
explicitly shown in the SI, Sec. 3C.1 [35]. Furthermore, the form of the low-
energy DOS, featuring the linear-$E$ behavior, shown in Fig. 2(c), is
consistent with the splitting of the nodal points in the sCDW phase. See also
Sec. S3.C2 for the technical details [35].

We now compute the optical conductivity in the case when one of the components
of this order parameter is nonzero, e.g. $\Gamma_{\rho 0}$, and for the sake
of the clarity, since the term $\sim u$ acts as a gap, we also set $u=0$.
Furthermore, we take $\mu=0$ and $T=0$, in the collisionless limit
$\Omega\gg\Delta$, with $\Delta$ as the order-parameter amplitude, to obtain
$\sigma_{ij}=\sigma\delta_{ij}$, with

$\text{Re}{\sigma}(\Omega)=\sigma_{0}=\frac{3}{8},$ (13)

with the technical details shown in Sec. S3.C3 of the SI [35]. The identical
results are obtained if instead we take $\Gamma_{\rho 3}$ to represent the
sCDW order parameter, as expected from the corresponding commutation relations
with the noninteracting Hamiltonian in Eq. (1). In fact, the sCDW state can be
thought of as a two-dimensional analogue of a correlation-driven nematic phase
in a three-dimensional multi-Weyl semimetal [37].

This behavior of the optical conductivity is consistent with the splitting of
the noninteracting band touching points with the vorticity of $\pm 3\pi$, into
two triplets of the non-degenerate ones with the vorticity $\pm\pi$. We
emphasize that the obtained form of the optical conductivity singles out the
sCDW phase as compared with the previously discussed valence-bond and bond-
current states. Finally, the obtained mean-field value of the conductivity is
expected to decrease when fluctuations are taken into account due to the
scattering between quantum-critical fermionic and bosonic excitations [38,
39].

Figure 3: The band structure for the noninteracting Hamiltonian given by Eq.
(1)(left panel) , where the the pairs of degenerate bands corresponding to the
same valley are superimposed. In the smectic charge-density wave with the
order parameter $\Gamma_{\rho 0}$, $\rho=1,2$, the two cubic band touching
points split into six points, with the triplets featuring opposite vorticities
$\pm\pi$ (right panel). Notice that the smectic charge-density wave order-
parameter mixes the two valleys and therefore also breaks the original lattice
translation symmetry, besides the rotational one. The rotational symmetry is,
however, restored close to each of the new band touching points.

_Conclusions and outlook._ In this paper, we showed that the optical
conductivity can distinguish possible paramagnetic metallic ground states
yielding a superconducting order in RTG, as shown in Fig. 1. In particular,
the valence-bond and bond-current states are distinguishable by the number of
peaks in the optical conductivity, while this observable in the sCDW order
features a minimum with a value matching the noniniteracting one at the mean-
field level, due to the splitting of the cubically dispersing nodal points at
each valley. We emphasize that we here analyzed only the real part of the
high-frequency optical conductivity due to its universal features. On the
other hand, its imaginary part can be trivially obtained by integration over
frequency via the Kramers-Kronig relation
$\text{Im}{\,}\sigma_{ij}(\Omega)=-2\Omega\pi^{-1}\mathcal{P}\int_{0}^{t/\hbar}d\omega\,\text{Re}{\sigma}_{ij}(\omega)/(\omega^{2}-\Omega^{2})$.
However, its specific features are sensitive to (nonuniversal) microscopic
details, such as the bandwidth, and hence does not provide a direct probe of
the possible interaction-driven ground states in RTG.

The approach employed for the identification of the possible interaction-
driven metallic ground states is quite general and can be applied to other vdW
systems to distinguish possible correlated insulating and metallic states, as
in the case of twisted bilayer graphene [27]. The same applies to the case of
Bernal bilayer graphene (without twist), and other vdW materials, such as MoS2
and WSe2, where an analogous analysis can also be used to distinguish possible
interaction-driven metallic ground states.

We here emphasize that our conclusions are based on the mean-field picture
where the role of the fluctuations on the conductivity has been neglected. To
account for the further corrections in the quantum-critical regime, we need to
address the full quantum-critical theory describing the quantum-critical
transition, as for instance in case of monolayer graphene [38], and we plan to
pursue this problem in the future. Another open avenue emerging from this work
concerns the features of the interaction-driven phases in vdW materials in the
nonlinear transport [39]. Finally, our findings should stimulate future
theoretical and experimental work on the out-of-plane optical response of the
vdW materials [40], and particularly the role of the electron-electron
interactions in this respect.

_Acknowledgments:_ The authors are grateful to Bitan Roy for useful
discussions. This research was funded by Fondecyt grants number 1190361 and
1200399, by ANID PIA Anillo ACT/192023 and the Swedish Research Council Grant
No. VR 2019-04735 (V.J.). Nordita is partially supported by Nordforsk.

## References

  * Cao  _et al._ [2018a] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, _et al._ , Correlated insulator behaviour at half-filling in magic-angle graphene superlattices, Nature 556, 80 (2018a), 1802.00553 .
  * Cao  _et al._ [2018b] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Unconventional superconductivity in magic-angle graphene superlattices, Nature 556, 43 (2018b).
  * Lu  _et al._ [2019] X. Lu, P. Stepanov, W. Yang, M. Xie, M. A. Aamir, I. Das, C. Urgell, K. Watanabe, T. Taniguchi, G. Zhang, A. Bachtold, A. H. MacDonald, and D. K. Efetov, Superconductors, orbital magnets and correlated states in magic-angle bilayer graphene, Nature 574, 653 (2019).
  * Jiang  _et al._ [2019] Y. Jiang, X. Lai, K. Watanabe, T. Taniguchi, K. Haule, J. Mao, and E. Y. Andrei, Charge order and broken rotational symmetry in magic-angle twisted bilayer graphene, Nature 573, 91 (2019), 1904.10153 .
  * Yankowitz  _et al._ [2019] M. Yankowitz, S. Chen, H. Polshyn, Y. Zhang, K. Watanabe, T. Taniguchi, D. Graf, A. F. Young, and C. R. Dean, Tuning superconductivity in twisted bilayer graphene, Science 363, 1059 (2019), 1808.07865 .
  * Chen  _et al._ [2019] G. Chen, A. L. Sharpe, P. Gallagher, I. T. Rosen, E. J. Fox, L. Jiang, B. Lyu, H. Li, K. Watanabe, T. Taniguchi, J. Jung, Z. Shi, D. Goldhaber-Gordon, Y. Zhang, and F. Wang, Signatures of tunable superconductivity in a trilayer graphene moiré superlattice, Nature 572, 215 (2019).
  * Liu  _et al._ [2020] X. Liu, Z. Hao, E. Khalaf, J. Y. Lee, Y. Ronen, H. Yoo, D. Haei Najafabadi, K. Watanabe, T. Taniguchi, A. Vishwanath, and P. Kim, Tunable spin-polarized correlated states in twisted double bilayer graphene, Nature 583, 221 (2020).
  * Serlin  _et al._ [2020] M. Serlin, C. Tschirhart, H. Polshyn, Y. Zhang, J. Zhu, K. Watanabe, T. Taniguchi, L. Balents, and A. Young, Intrinsic quantized anomalous hall effect in a moiré heterostructure, Science 367, 900 (2020), 1907.00261 .
  * Park  _et al._ [2021] J. M. Park, Y. Cao, K. Watanabe, T. Taniguchi, and P. Jarillo-Herrero, Tunable strongly coupled superconductivity in magic-angle twisted trilayer graphene, Nature 590, 249 (2021).
  * Rubio-Verdu  _et al._ [2022] C. Rubio-Verdu, S. Turkel, Y. Song, and et al.., Moire nematic phase in twisted double bilayer graphene, Nature Physics 18, 196 (2022).
  * Zhou and et al. [2021a] H. Zhou and et al., Half- and quarter-metals in rhombohedral trilayer graphene, Nature 598, 429 (2021a).
  * Zhou and et al. [2021b] H. Zhou and et al., Superconductivity in rhombohedral trilayer graphene, Nature 598, 434 (2021b).
  * Törmä  _et al._ [2022] P. Törmä, S. Peotta, and B. A. Bernevig, Superconductivity, superfluidity and quantum geometry in twisted multilayer systems, Nature Reviews Physics 4, 528 (2022).
  * Chou  _et al._ [2021] Y.-Z. Chou, F. Wu, J. D. Sau, and S. Das Sarma, Acoustic-phonon-mediated superconductivity in rhombohedral trilayer graphene, Phys. Rev. Lett. 127, 187001 (2021).
  * Ghazaryan  _et al._ [2021] A. Ghazaryan, T. Holder, M. Serbyn, and E. Berg, Unconventional superconductivity in systems with annular fermi surfaces: Application to rhombohedral trilayer graphene, Phys. Rev. Lett. 127, 247001 (2021).
  * Szabó and Roy [2022] A. L. Szabó and B. Roy, Metals, fractional metals, and superconductivity in rhombohedral trilayer graphene, Phys. Rev. B 105, L081407 (2022).
  * Cea  _et al._ [2022] T. Cea, P. A. Pantaleón, V. o. T. Phong, and F. Guinea, Superconductivity from repulsive interactions in rhombohedral trilayer graphene: A kohn-luttinger-like mechanism, Phys. Rev. B 105, 075432 (2022).
  * Dai  _et al._ [2021] H. Dai, J. Hou, X. Zhang, Y. Liang, and T. Ma, Mott insulating state and $d+id$ superconductivity in an abc graphene trilayer, Phys. Rev. B 104, 035104 (2021).
  * You and Vishwanath [2022] Y.-Z. You and A. Vishwanath, Kohn-luttinger superconductivity and intervalley coherence in rhombohedral trilayer graphene, Phys. Rev. B 105, 134524 (2022).
  * Chatterjee  _et al._ [2021] S. Chatterjee, T. Wang, E. Berg, and M. P. Zaletel, Inter-valley coherent order and isospin fluctuation mediated superconductivity in rhombohedral trilayer graphene, arxiv:2109.00002 (2021).
  * Dong and Levitov [2021] Z. Dong and L. Levitov, Superconductivity in the vicinity of an isospin-polarized state in a cubic dirac band, arxiv:2109.01133 (2021).
  * Basov  _et al._ [2011] D. N. Basov, R. D. Averitt, D. van der Marel, M. Dressel, and K. Haule, Electrodynamics of correlated electron materials, Rev. Mod. Phys. 83, 471 (2011).
  * Tabert and Nicol [2013] C. J. Tabert and E. J. Nicol, Optical conductivity of twisted bilayer graphene, Phys. Rev. B 87, 121402 (2013).
  * Moon and Koshino [2013] P. Moon and M. Koshino, Optical absorption in twisted bilayer graphene, Phys. Rev. B 87, 205404 (2013).
  * Stauber and  _et al._ [2013] T. Stauber and  _et al._ , Optical conductivity, drude weight and plasmons in twisted graphene bilayers, New J. Phys. 15, 113050 (2013).
  * Jang  _et al._ [2019] J. Jang, S. Ahn, and H. Min, Optical conductivity of black phosphorus with a tunable electronic structure, 2D Materials 6, 025029 (2019).
  * Calderón and Bascones [2020] M. J. Calderón and E. Bascones, Correlated states in magic angle twisted bilayer graphene under the optical conductivity scrutiny, npj Quantum Mater. 5, 57 (2020).
  * Tan  _et al._ [2021] C.-Y. Tan, C.-X. Yan, Y.-H. Zhao, H. Guo, and H.-R. Chang, Anisotropic longitudinal optical conductivities of tilted dirac bands in $1{T}^{{}^{\prime}}\text{$-$}\mathrm{Mo}{\mathrm{s}}_{2}$, Phys. Rev. B 103, 125425 (2021).
  * Song and et al. [2019] B. Song and et al., Complex optical conductivity of two-dimensional mos2: A striking layer dependency, J. Phys. Chem. Lett. 10, 6246 (2019).
  * Zhang  _et al._ [2020] G. Zhang, S. Huang, F. Wang, Q. Xing, C. Song, C. Wang, Y. Lei, M. Huang, and H. Yan, The optical conductivity of few-layer black phosphorus by infrared spectroscopy, Nature Communications 11, 1847 (2020).
  * Sachdev [2011] S. Sachdev,  _Quantum Phase Transitions_, 2nd ed. (Cambridge University Press, 2011).
  * Koshino and McCann [2009] M. Koshino and E. McCann, Trigonal warping and berry’s phase $n\pi$ in abc-stacked multilayer graphene, Phys. Rev. B 80, 165409 (2009).
  * Zhang  _et al._ [2010] F. Zhang, B. Sahu, H. Min, and A. H. MacDonald, Band structure of $abc$-stacked graphene trilayers, Phys. Rev. B 82, 035409 (2010).
  * Cvetkovic and Vafek [2012] V. Cvetkovic and O. Vafek, Topology and symmetry breaking in abc trilayer graphene, arXiv:1210.4923 (2012).
  * [35] Supplementary information, .
  * Mahan [2013] G. D. Mahan,  _Many-particle physics_ (Springer Science & Business Media, 2013).
  * Roy  _et al._ [2017] B. Roy, P. Goswami, and V. Juričić, Interacting weyl fermions: Phases, phase transitions, and global phase diagram, Phys. Rev. B 95, 201102 (2017).
  * Roy and Juričić [2018] B. Roy and V. Juričić, Collisionless transport close to a fermionic quantum critical point in dirac materials, Phys. Rev. Lett. 121, 137601 (2018).
  * Rostami and Juričić [2020] H. Rostami and V. Juričić, Probing quantum criticality using nonlinear hall effect in a metallic dirac system, Phys. Rev. Research 2, 013069 (2020).
  * Xu  _et al._ [2021] Z. Xu, D. Ferraro, A. Zaltron, N. Galvanetto, A. Martucci, L. Sun, P. Yang, Y. Zhang, Y. Wang, Z. Liu, J. D. Elliott, M. Marsili, L. Dell’Anna, P. Umari, and M. Merano, Optical detection of the susceptibility tensor in two-dimensional crystals, Communications Physics 4, 215 (2021).