# Inferring Belief States in Partially-Observable Human-Robot Teams

Jack Kolb and Karen M. Feigh This work was supported by Amazon Consumer
Robotics.The authors thank Hae Won Park and Animesh Garg for their
insights.Jack Kolb and Karen M. Feigh are with the College of Engineering at
the Georgia Institute of Technology, North Avenue, Atlanta, GA 30332, USA
{kolb<EMAIL_ADDRESS>
###### Abstract

We investigate the real-time estimation of human situation awareness using
observations from a robot teammate with limited visibility. In human factors
and human-autonomy teaming, it is recognized that individuals navigate their
environments using an internal mental simulation, or mental model. The mental
model informs cognitive processes including situation awareness, contextual
reasoning, and task planning. In teaming domains, the mental model includes a
team model of each teammate’s beliefs and capabilities, enabling fluent
teamwork without the need for explicit communication. However, little work has
applied team models to human-robot teaming. We compare the performance of two
current methods at estimating user situation awareness over varying visibility
conditions. Our results indicate that the methods are largely resilient to
low-visibility conditions in our domain, however opportunities exist to
improve their overall performance.

##  I Introduction

The human factors community recognizes that when individuals conduct taskwork,
they construct an internal simulation termed a mental model. The mental model
broadly informs the individual’s abstract understanding of the environment,
such as contextual information, the status of the task at hand, contingencies,
and actionable plans [3, 17].

In collaborative tasks, the mental model expands to include a team model
incorporating a prediction teammate beliefs, characteristics, capabilities,
and roles [23, 17]. By constructing a team model in addition to their own
mental model, individuals apply the information they believe their teammates
know – and do not know – in their own plans. This allows individuals to work
proactively by anticipating teammate needs without explicit communication [22,
14].

Recent research interest has sought to apply team models for human-robot and
human-autonomy teaming [11] [15]. Researchers in the human-robot interaction
(HRI) community anticipate that an inferred understanding of human teammate
beliefs can support a variety of robot planning, comprehension, and decision-
making goals [21]. Related work has proposed frameworks for constructing team
models [19] [2], however few work have evaluated these frameworks in human-
robot teaming domains.

Human factors researchers often use situation awareness as a proxy for
pertinent components of the mental model [8, 21]. Situation awareness is an
individual’s understanding of the current environment along three levels: the
raw world state (e.g., object locations), contextual information (e.g.,
inferred properties and relationships between objects), and future projection
(e.g., plans and dynamic models). In practice, situation awareness is
typically measured in-situ by periodically halting the environment and asking
the user questions about the task’s current state [9].

Our objective is to evaluate two models that enable robots to predict human
teammates’ belief states. We define the belief state as measurable information
of the teammate’s situation awareness. We split the problem into two
components – estimating the user’s awareness of world state elements, and
estimating how that information is used to answer Level 1 (world state) and
Level 2 (contextual) situation awareness questions. Each model addresses the
second component by reasoning over a scene graph of the user’s estimated
knowledge of the current task.

Concretely, this paper contributes the following:

  1. 1.

An online human-robot teaming domain and situation awareness dataset featuring
partial observability and two belief state prediction models – a logical
predicates model and a large language model (LLM).

  2. 2.

Findings that the belief state prediction models were resilient to low-
observability conditions and moderately agreed with human user responses.

Our results indicate that both models were successful at inferring user
situation awareness. Moreover, both models maintained their performance under
varying visibility conditions. The LLM performed on par with the hand-crafted
logical predicates model, indicating that the LLM can be leveraged to create
rules for logical predicates models, or encapsulate the process entirely.

All data and source code for this paper is available at
https://jackkolb.com/tmm-hri.

##  II Related Works

The HRI community has primarily utilized mental models with task-dependent
factors aligned to the three levels of situation awareness. For example, in a
process-driven task (e.g., manufacturing, housework, spaceflight), the mental
model may contain a representation of the overall procedure (Level 1), where
in the procedure the environment is currently (Level 2), and what that implies
for the immediate future (Level 3). Contextual information is derived from the
world state, which supports higher-level reasoning about the future of the
environment and task. In addition, mental models can draw upon prior
experiences to form a stronger model of the risks, failure modes, and
practical effects of different actions and task strategies [22].

Mental models are also believed to play an important role in teaming [22]. In
human-human teams, people predict the mental model of their human teammates,
the team model. If an individual believes their teammate’s mental model is
incorrect, or suspects their own mental model is outdated, they adapt their
plans or communicate with their teammate accordingly. Related work has found
that a strong team model is often important for high-performing teams [14].

A team model also facilitates theory of mind [5]. Theory of mind is an agent’s
understanding of another agent’s beliefs, such as the other’s knowledge,
intentions, emotions, and opinions. Theory of mind is generally studied in the
context of education, where instructors aim to shape the belief states of
their students, and lends itself to orders of intentionality (“I believe that
you know X”). Theory of mind is inherently team-centric, and has been studied
in several close-proximity human-robot collaboration domains [6, 20].

Most related work on mental models has focused on humans and human-human
teams, however in recent years there has been interest in expanding mental
model theory to human-robot teams [17, 12, 10, 13]. Work in this space has
been challenged by the extraordinary complexity of representing human
cognition, which remains an existential cognitive science problem. This has
led researchers to use simplified domains and team models. This work uses a
domain that more closely represents real-world human-robot teaming
applications.

###  II-A Logical Predicates Representation

The most prevalent representation of mental models in the HRI literature uses
logical predicates, or sets of known logical truths [7, 18, 10, 19]. Each
predicate is an observed feature of the environment state taken as an absolute
truth, e.g., the location of an object and its properties. Complex functions
are hand-crafted to derive contextual information or model how the environment
will change over time. When predicting the mental model of a teammate, a robot
agent can represent the teammate’s mental model using the same logical
predicates framework, and filter the observational inputs by the workspace
region visible to the teammate.

The logical predicates model is attractive for environments with low
observational uncertainty. When there is little room for the human teammate to
misunderstand their observations, and the robot has full observability, the
teammate’s mental model is a subset of the robot’s mental model. With these
assumptions, a rigorously-designed logical predicates model can capture many
if not all the important aspects of the environment and answer a wide range of
inference questions.

However, the robot’s mental model can be misled when it does not contain all
information available to the teammate’s mental model, for example, in
partially observable conditions. As the logical predicates model is fact-based
and cannot model uncertainty, hard-coding reasoning rules to resolve
conflicting information can negatively impact downstream planning and
reasoning.

Research opportunities currently exist to consider uncertainty within mental
models. Specifically, the literature identifies three areas for error:
misperception of events, miscomprehension of perceived events, and imperfect
mapping of perceived events to the updated mental model [2].

In this work we assess whether an LLM can match a logical predicates model at
inference over a scene graph, challenging the need for hand-crafted inference
rules.

###  II-B Markov Decision Process Representation

Nikolaidis et al. proposed representing the team model as a partially-
observable Markov decision process (POMDP), where the environment is
represented as a set of states, environment events, transition probabilities,
and state rewards [15]. In this representation, the mental model is the
predicted probability that the human believes the system is in each state.
Higher levels of situation awareness can be estimated by rolling out the
Markov process and assuming players and opponents will perform actions that
maximize their reward. Additionally, the state rewards and transition
probabilities can be tuned to individual teammates to personalize the mental
model to the teammate’s experience and characteristics.

The POMDP representation meets the criteria of a mental model in that it
models the belief state of an agent. However, it does so by inverting the
representation from an object-based belief state to a state-based belief
state. In a low-dimensional environment, it is feasible to represent all
possible states and their transition probabilities. However, the complex
environments sought by HRI domains can overwhelm the capabilities of a POMDP.
Few works have applied the POMDP representation to real-world scenarios.

Figure 1: Overview of the predicted user belief state system. The ground truth
world state information is filtered by the robot’s visibility ($R^{robot})$
and used to update the robot’s belief state, $\beta^{robot}$. The robot’s
belief state is then filtered by the user’s visibility to update the predicted
user belief state, $\beta^{pred}$. The resulting belief state is what the
robot thinks the user is aware of, i.e., a theory of mind that can inform
downstream reasoning tasks.

###  II-C Fuzzy Mental Model Finite State Machine (FMMFSM)

Bolton et al. recently proposed FMMFSM, a model inspired by fuzzy logic
concepts that uses membership functions to estimate a teammate’s belief state
[2]. FMMFSM addresses the uncertainty problem of the logical predicates model,
and uses an object-based scene graph. While the logical predicates model
translates directly from observations to an updated mental model, FMMFSM
introduces three intermediate functions:

  * $\alpha$

maps possible environment events to the probability that each event is
perceived by the teammate (addressing the observability limitation of logical
predicates).

  * $\phi$

maps perceived environment events to the probability that the event was
correctly perceived.

  * $\delta$

maps perceived environment events to the probability that the event changes
each aspect of the system state.

The function implementations are arbitrary and not limited to traditional
hand-crafted fuzzy membership functions.

While FMMFSM addresses key shortcomings of the logical predicates and POMDP
models, it solely targets Level 1 situation awareness, or the construction of
the scene graph. Predictions of contextual or future information would need to
be derived from its output. As this work focuses on inference over a scene
graph, instead of the scene graph itself, we chose not to utilize FMMFSM in
this work. However, FMMFSM presents a compelling model that warrants
evaluation in real-world human-robot teaming domains.

##  III Problem Formulation

Our objective is to map a set of observable environment states $O$ to a set of
belief states $\\{\beta^{true},\beta^{robot},\beta^{pred}\\}$ over a time
series. The belief states represent task-relevant world state and contextual
information about the environment for an oracle ($\beta^{true}$), a robot
($\beta^{robot}$), and a human user ($\beta^{pred}$). $\beta^{true}$ is the
ground truth representation of the environment, and is updated from the full
environment state $O_{t}$ at each timestep $t$. $\beta^{robot}$ uses the
subset of $O_{t}$ that is observable by the robot within an observable region,
$R^{robot}$. Lastly, $\beta^{pred}$ is a predicted belief state of the human
user from the robot’s perspective, updated using $\beta^{robot}$ and the
user’s observable region $R^{user}$.

We approach the problem by defining two functions, $\Phi$ and $\mathcal{B}$,
to update the belief state from observations.

$\Phi:O_{t},R\mapsto O^{\prime}_{t}$ (1)

$\Phi$ filters environment observations $O_{t}$ by a human or robot agent’s
observable region $R$ to obtain the subset of observations $O^{\prime}_{t}$
that are directly observable by the agent. In a 2D domain, if $R$ is a circle
of radius $r$, then $O^{\prime}_{t}$ is the set of observations from $O_{t}$
that are within $r$ from the agent. In a 3D domain, $R$ can use the agent’s
pose and perception capabilities to model the agent’s visible region. When
inferring belief states, such as $\beta^{pred}$, $O_{t}$ is extracted from
$\beta^{robot}$ and can differ from the true world state.

$\mathcal{B}:O^{\prime}_{t},\beta_{t-1}\mapsto\beta_{t}$ (2)

$\mathcal{B}$ updates a belief state from observations. $\beta$ is the current
belief state, and $O^{\prime}_{t}$ is the subset of environment observations
that are used to update the belief state at a time $t$.

The core research objective of inferring a human teammate’s belief state is to
identify the function $\mathcal{B}$ that maximizes the agreement between
$\beta_{t}$ and the teammate’s internal belief state. In this work, we model a
human user’s belief state ($\beta^{pred}_{t}$) and score the agreement between
$\beta^{pred}_{t}$ and queries asked of study participants ($\beta^{human}$).
In Fig. 1, $\beta^{true}$ is the ground truth of the environment, equivalent
to a robot with full observability.

We evaluate the following hypotheses:

  1. H1

A fully observable logical predicates baseline ($\mathcal{B}^{LP}$) will
result in responses to situation awareness queries that moderately agree with
user responses.

  2. H2

The performance of the logical predicates baseline ($\mathcal{B}^{LP}$) at
inferring user situation awareness will sharply degrade in partially
observable conditions.

  3. H3

A large language model ($\mathcal{B}^{LP+LLM}$) will match the logical
predicates baseline ($\mathcal{B}^{LP}$) across fully observable and partially
observable environments.

##  IV Methods

Our study extends the Overcooked-AI environment to predict user belief states
in partial observability conditions [4]. The environment simulates a
restaurant kitchen where several agents collaborate to cook and deliver soups
using ingredients scattered across the kitchen’s counters. While the domain is
a discretized representation of a cooking task, it presents a team-oriented
structure and a fast-paced, engaging task. Additionally, the discrete nature
is ideal for evaluating the logical predicates model, which the HRI community
has only explored in domains that are even more simplistic [11].

In Overcooked-AI, agents in a human-robot team perform a sequence of subtasks
to cook soups. Agents first move three ingredients (onions or tomatoes) from
countertops to a pot. The pot then cooks for $10$ seconds, and once complete,
an agent ‘plates’ the soup using a dish. Lastly, the plated soup is brought to
a designated serving station, where it leaves the gameboard. The process
repeats until all ingredients have been used or $90$ seconds have elapsed.
Users are told to focus on completing all possible soups, but are not
evaluated on their performance.

The agents share a workspace, but cannot communicate, meaning the team
benefits from agents being proactive in their work and considerate of their
teammate’s belief state. While related work has used reinforcement learning
for the robot agent [4], our preliminary work found that a task-oriented state
machine with A* planning was effective across our kitchen layouts. In this
work, we are focused on predicting the user belief state ($\beta^{pred}$), not
applying that prediction to planning and decision-making.

###  IV-A Partial Observability

We adapted the Overcooked-AI environment to simulate real-world conditions by
introducing partial observability. Our visibility function, $\Phi$, limits an
agent’s field of view to a region defined by an agent-specific parameter $R$.
The parameter includes a type (V, O, or D) and a radius:

  * •

V-type is a $90^{\circ}$ conic view facing the agent’s direction, which
represents a narrow field of view.

  * •

O-type is a $360^{\circ}$ circular view of the surrounding area, and can model
full observability when the radius exceeds the environment dimensions.

  * •

D-type is a $180^{\circ}$ semi-circular view of the area in front of the
agents, and can model half observability when the radius exceeds the
environment dimensions.

Fig. 2 shows an example of each visibility type. Agents are able to see the
environment objects within their visible region $R$. As we would expect real-
world agents to track teammates via sound and sight, as well as have a strong
knowledge of the static kitchen components, agents can always see the
locations of their teammate, the kitchen floorplan, and the kitchen appliances
(pot, serving station).

Figure 2: The task domain as shown to the user (green agent), with three
visibility types and radii (e.g., V3 indicates V-type with radius $3$). The
green shaded region is visible to the user. Each type is a unique shape
representing a common field of view, and the radius indicates the extent of
the visible region. For our experiment, all users had D4 visibility, and we
varied the visibility of the robot (blue agent) in our post-hoc analysis.
Figure 3: Four views of the same scene. The Raw Observations view shows the
direct world state $O_{t}$, the Full Observability Belief State
($\beta^{true}$) shows the ground truth beliefs, the Robot’s Partial
Observability Belief State ($\beta^{robot}$) is updated from the world state
within the blue region around the robot ($R^{robot}$), and the Estimated
Human’s Belief State ($\beta^{user}$) is updated from the robot’s belief state
filtered by the green region around the user ($R^{user}$). Grey edges indicate
that two objects can be used with each other at this time. A false belief is
highlighted in red: the user set a plate upon a counter, however since it
occurred outside the view of the robot, the robot’s belief state and the
estimated human belief state still show the plate as being held.

###  IV-B Object Permanence in the Scene Graph

We considered belief states as two components – a scene graph constructed from
the filtered environment observations $O^{\prime}_{t}$, and an inference over
the scene graph to infer contextual information. However, observations do not
inherently track objects, and without object permanence the agent’s belief
state cannot support objects moving in and out of the visible region. Thus, we
require a function to track objects across updates to the scene graph and
resolve conflicts. Object permanence is still a challenging open research
problem [24].

In addition to object permanence, the cooking environment is challenging
because of object transformation. When ingredients are used to make a soup,
the ingredients simply no longer exist. Similarly, soups leave the gameboard
when they are placed on the serving station. Our system must not only track
objects through time, but also identify when objects have transformed and
predict which objects were transformed. To help simplify this problem, we
assume that all ingredient objects are known at the task’s initialization.

Object permanence was maintained using a structured three-pass logical
approach. In the first pass, static environment objects are identified and
matched to their corresponding known object. While this naive case can be
tricked by swapping an item with an identical item, it is not necessary for
our belief states to identify that edge condition.

The second pass matches objects to their closest known object. Similar to the
naive case, the pass matches each remaining observed object to the closest
remembered object of the same class. This is most effective with frequent low-
ambiguity observations, which is feasible for a two-agent team. To resolve
objects that were picked up by agents, the pass defaults to the closest
unmatched known object.

The final pass addresses transformed objects. These are observed objects that
have no fit to a known object (i.e., a newly-made soup), or known objects that
have no observed match (i.e., a plated soup that was delivered). Due to the
partially observable environment, it can be challenging to determine which
objects transformed into other objects, however many cases can be inferred by
tracking a soup’s ingredients. For example, a new “tomato, onion, and tomato
soup” object implies that two known tomatoes and one known onion no longer
exist on the gameboard. As each agent’s belief state is initialized with the
true gameboard state, and we can assume no ingredients are added during
gameplay, this pass can reasonably identify which ingredients were used to
create a soup and when a soup has likely been served.

Due to the freedom users have to select ingredients and complete the task, no
object permanence system can be perfect in all partially observable
environments.

Fig. 3 shows a reconstruction of the gameboard using three belief states.
Notably, the figure shows that the robot’s belief state $\beta^{robot}$
mistracks a plate that the user set down outside of the robot’s visible region
$R$, which cascades to a misbelief in the predicted user belief state
$\beta^{pred}$.

###  IV-C Inferring Context and Situation Awareness

The value of modeling a teammate’s belief state lies in applying the belief
state to correctly infer additional contextual information and answer
questions. We use the SAGAT method [8] to obtain the user’s real belief state,
$\beta^{human}$, by periodically pausing the game and asking the user
questions about the environment. Up to two questions are asked every $30$
seconds. We include questions about the first two levels of situation
awareness – world state understanding and contextual information. All
situation awareness questions were multiple choice and randomly chosen from a
set question bank.

As shown in Fig. 1, the user responses are compared to the predicted belief
state $\beta^{pred}$ to evaluate the performance of the belief state function
$\mathcal{B}$. We compare two functions, $\mathcal{B}^{LP}$ and
$\mathcal{B}^{LP+LLM}$.

For the logical predicates model, $\mathcal{B}^{LP}$, hand-crafted rules use
the current scene graph to derive responses to each situation awareness
question. For example, to answer the contextual information question “How many
more soups can be made/delivered?”, $\mathcal{B}^{LP}$ calculates the maximum
number of remaining soups using the ingredients and plates known to
$\beta^{pred}$. Consequently, the logical predicates model is unable to answer
questions that are not hand-crafted beforehand.

$\mathcal{B}^{LP+LLM}$ uses $\mathcal{B}^{LP}$ to construct the predicted
scene graph component of $\beta^{pred}$, and an LLM (GPT4 [1]) to reason upon
$\beta^{pred}$ to answer situation awareness questions. The model is prompted
with a description of the game and the current scene graph representation, as
well as the query “You are A1 […] Please answer the question using only one of
responses below […] What is your answer?” To fairly compare $\mathcal{B}^{LP}$
and $\mathcal{B}^{LP+LLM}$, the models do not consider prior world states nor
data from other participants, although exploring alternative methods of
constructing the inferred scene graph is an opportunity for future work. The
primary benefit of using an LLM for this task is the LLM’s ability to answer
open vocabulary questions about the scene, which could supercede a logical
predicates model. Additionally, an LLM can distill its reasoning into a
logical format compatible with a $\mathcal{B}^{LP}$ model for efficient repeat
queries.

###  IV-D Scoring and Evaluation

Belief states are scored by comparing the responses of the belief states to
situation awareness questions. Correct responses are awarded one point, and
incorrect responses zero. For questions based on spatial reasoning, we awarded
partial credit for near-correct answers. For example, for a question asking
“Where is the closest tomato?”, responding “Center” when the correct answer
was one tile off-center scored $0.5$. The scoring function is defined as:

$Score_{u,l}=\frac{\sum\limits_{q,t\in
N}(score_{q}(\beta^{A}_{t},\beta^{B}_{t})}{|N|}$ (3)

Where $q$ is a situation awareness question asked to the user $u$ on layout
$l$ at time $t$, $N$ is the total number of situation awareness questions
asked, and $score_{q}(\beta^{A}_{t},\beta^{B}_{t})\mapsto\mathbb{R}$ is the
scoring function for $q$ using two belief states $\beta^{A}_{t}$ and
$\beta^{B}_{t}$. Our scoring function was symmetric, such that
$score_{q}(\beta^{A}_{t},\beta^{B}_{t})=score_{q}(\beta^{B}_{t},\beta^{A}_{t})$.

##  V User Study Design

We evaluated our methods with data collected through an online user study
approved by Georgia Tech’s IRB. Participants were recruited from the research
platform Prolific [16]. The study’s screening criteria included accessing the
study using a personal computer, being within the ages of 18-89, self-
identification as not part of any vulnerable group, being physically located
in the United States, and having English as a first language. The study took
approximately 20 minutes to complete, and participants were compensated $\$5$
USD.

Participants completed two practice runs and then four trials with our
experiment domain. Each trial used a different kitchen layout. The game’s
state was recorded at each frame ($10$hz), as well as responses to situation
awareness questions.

Post-hoc, the game state histories were replayed and the logical predicates
model was used to construct $\beta^{true}$, $\beta^{robot}$, and
$\beta^{pred}$ across a range of visibilities for $R^{robot}$. The
$\mathcal{B}^{LP}$ and $\mathcal{B}^{LP+LLM}$ models were then used to compare
responses derived from the models to the actual user responses.

Figure 4: Violin plot of the score distributions of user responses to
situation awareness questions ($\beta^{human}$) with respect to the ground
truth oracle ($\beta^{true}$). The broad variances show the task is
appropriately difficult.

##  VI Results & Discussion

A total of $42$ participants completed the study ($50\%$ female, ages
$22-74$). We excluded $11$ participants who either failed to complete the
study or failed to outperform random selection of the situation awareness
responses, resulting in $31$ participants included in our evaluation.

###  VI-A User Performance

We first judge whether the task was appropriately difficult. Fig. 4 shows the
distribution of user scores with respect to the ground truth across the four
evaluated kitchen layouts. While Layout 3 appears to be easier than the other
layouts, the other score distributions have similar medians, indicating
consistent difficulty. Despite several perfect scores at each layout, the
broad score distributions indicate that the task was neither trivial nor
overwhelming for users.

Notably, few users scored perfectly (scores of $100\%$), and no users scored
consistently well across all layouts. This demonstrates that users were
generally unable to fully track the environment, motivating the potential
benefits of predicting user belief states in this partially observable domain.

###  VI-B Fully Observable Robot Belief State

Hypothesis H1 evaluates the logical predicates baseline in its ideal
conditions. We are interested in how accurately a robot with full
observability can predict the situation awareness responses of a human user
with partial observability. Unlike our evaluation of the User Performance,
wherein the human user was scored against the ground truth oracle, in this
case the robot does not assume that the user has any knowledge of the
environment outside of what the robot knows the user has explicitly seen.

We find that the scores from the predicted user belief state ($\beta^{pred}$)
with respect to the actual user responses ($\beta^{human})$ are almost
identical to Fig. 4. The finding indicates that there were virtually no
instances in our study where a model assuming perfect user perception could
have had a false belief, i.e., environment changes that would have impacted
the situation awareness responses were quickly observed by users, although
seemingly not always recalled.

Additionally, Fig. 2 shows that the full observability model’s performance
averaged between $55\%$ and $75\%$. This suggests that users had nuanced
perception, comprehension, or memory that was not captured by
$\mathcal{B}^{LP}$. Therefore, opportunities exist for alternative models to
utilize human factors that can contribute to a user’s belief state.

Thus, we affirm our hypothesis H1.

Figure 5: Line plot of the mean performance of $\beta^{pred}$ using
$\mathcal{B}^{LP}$ with respect to the user responses, across various
visibility parameters $R^{robot}$. While performance declines between the
three visibility types, there is not a meaningful decline between visibility
conditions.

###  VI-C Partial Robot Observability

Hypothesis H2 addresses how $\mathcal{B}^{LP}$ affects the performance of
$\beta^{robot}$ across a range of visibilities. For a belief state function
$\mathcal{B}$ to be feasible in human-robot teaming domains, it is valuable to
see how the function’s performance declines with decreasing visibility.

Fig. 5 shows the performance of $\mathcal{B}^{LP}$ across $14$ visibility
conditions. Counter to our hypothesis H2, the performance of
$\mathcal{B}^{LP}$ only slightly decreased between high visibility and low
visibility conditions. We believe this due to our logical predicates
implementation accurately resolving conflicting beliefs during the task. The
findings support using $\mathcal{B}^{LP}$ as a baseline for more complex
partially observable domains.

Thus, we refute our hypothesis H2.

Figure 6: Chart of the mean performances of $\beta^{pred}$ constructed using
the $\mathcal{B}^{LP}$ and $\mathcal{B}^{LP+LLM}$ models over two
observability conditions. Error bars show the variance. $\mathcal{B}^{LP+LLM}$
has comparable means and variances to $\mathcal{B}^{LP}$ when scored against
user responses, indicating that $\mathcal{B}^{LP+LLM}$ is able to infer an
inference ruleset similar to the hand-crafted developed for
$\mathcal{B}^{LP}$. Higher scores indicate superior performance.

###  VI-D Large Language Model

Hypothesis H3 explores whether an LLM can predict and reason about the user’s
belief state as a zero-shot system. The open vocabulary reasoning capability
of LLMs has no equivalent for logical predicates, and would be valuable for
broadly applying predicted belief states. As a first step, we are interested
in whether an LLM can match the performance of the logical predicates model
($\mathcal{B}^{LP}$).

Fig. 6 shows the performance of $\mathcal{B}^{LP}$ and $\mathcal{B}^{LP+LLM}$
at predicting user responses to situation awareness questions. The two models
were presented with the same scene graph and prompted with which answer the
user would choose, and we found no meaningful difference in mean nor variance
between the two models. The findings support the use of LLMs for predicting
user belief states, and as a baseline upon which to compare future models.

Thus, we affirm our hypothesis H3.

##  VII Conclusion

For decades, the logical predicates model has been a de-facto representation
of world states, and has recently been extended to represent belief state
prediction for human-robot teaming. The logical structure makes the model
compatible with state machines, linear temporal logic planners, and case-based
reasoning, which is attractive for real-time systems seeking transparent
knowledge representations.

However, the logical predicates model does not consider the probabilistic
nature of human cognition – people do not see the world in absolute binary
truths, but as probability and managed uncertainty. As such, the logical
predicates model is challenged to represent individual human differences in
their interpretations of the environment. Moreover, very few works have
operationalized the model to human-robot teaming domains, and the literature
typically assumes that robot agents have full observability over their
environments.

Unable to consider uncertainty, the logical predicates model is limited to
low-dimension state spaces and high-certainty predicates. As human-robot
teaming domains typically operate in complex environments, misinterpretations
can have consequences on downstream planning.

In this work, we explored the strengths and limitations of using a logical
predicates belief state model in human-robot teaming. We defined the problem
of predicting a co-located human teammate’s belief state, measured by the
model’s responses to situation awareness questions. Through a user study we
collected a dataset of responses at various points throughout a collaborative
cooking game, which we utilized to test the accuracy of a logical predicates
model and an LLM at predicting the user’s responses under several degrees of
full and partial observability.

Key takeaways from this work include:

  1. 1.

The logical predicates model resulted in a scene graph where neither inference
model scored above $50-75\%$, motivating the integration of human factors
techniques to construct predicted scene graphs for teammates.

  2. 2.

The performance of the logical predicates model only slightly degraded between
fully observable and partially observable conditions, supporting the model’s
use in more realistic partially observable domains if a careful implementation
is feasible.

  3. 3.

The performance of the logical predicates model was matched by a large
language model in a zero-shot manner, suggesting that LLMs can be leveraged to
help construct logical predicate models, or even supercede them in more
complex environments.

## References

  * [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [2] Matthew L Bolton, Elliot Biltekoff, and Kevin Byrne.  Fuzzy mental model finite state machines: A mental modeling formalism for assessing mode confusion and human-machine “trust”.  In 2022 IEEE 3rd International Conference on Human-Machine Systems (ICHMS), pages 1–4. IEEE, 2022.
  * [3] Janis A Cannon-Bowers, Eduardo Salas, and Sharolyn Converse.  Shared mental models in expert team decision making.  Current issues in individual and group decision making. Lawrence Erlbaum, Hillsdale, NJ, pages 221–246, 1993.
  * [4] Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan.  On the utility of learning about humans for human-ai coordination.  Advances in neural information processing systems, 32, 2019.
  * [5] Antonio Chella, Francesco Lanza, Arianna Pipitone, Valeria Seidita, et al.  Human-robot teaming: Perspective on analysis and implementation issues.  In AIRO@ AI* IA, pages 12–17, 2018.
  * [6] Devleena Das, Sonia Chernova, and Been Kim.  State2explanation: Concept-based explanations to benefit agent learning and user understanding.  Advances in Neural Information Processing Systems, 36:67156–67182, 2023.
  * [7] Gwendolyn Edgar, Matthew McWilliams, and Matthias Scheutz.  Improving human-robot team performance with proactivity and shared mental models.  In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 2322–2324, 2023.
  * [8] Mica R Endsley.  Design and evaluation for situation awareness enhancement.  In Proceedings of the Human Factors Society annual meeting, volume 32, pages 97–101. Sage Publications Sage CA: Los Angeles, CA, 1988.
  * [9] Mica R Endsley.  Situation awareness global assessment technique (sagat).  In Proceedings of the IEEE 1988 national aerospace and electronics conference, pages 789–795. IEEE, 1988.
  * [10] Felix Gervits, Terry W Fong, and Matthias Scheutz.  Shared mental models to support distributed human-robot teaming in space.  In 2018 aiaa space and astronautics forum and exposition, page 5340, 2018.
  * [11] Felix Gervits, Dean Thurston, Ravenna Thielstrom, Terry Fong, Quinn Pham, and Matthias Scheutz.  Toward genuine robot teammates: Improving human-robot team performance using robot shared mental models.  In Aamas, pages 429–437, 2020.
  * [12] Nader Hanna and Deborah Richards.  The impact of multimodal communication on a shared mental model, trust, and commitment in human–intelligent virtual agent teams.  Multimodal Technologies and Interaction, 2(3):48, 2018.
  * [13] William G Kennedy and J Gregory Trafton.  Using simulations to model shared mental models.  In Proceedings of the eighth international conference on cognitive modeling, pages 253–254. Taylor & Francis Psychology Press Ann Arbor, 2007.
  * [14] Janice Langan-Fox, Sharon Code, and Kim Langfield-Smith.  Team mental models: Techniques, methods, and analytic approaches.  Human factors, 42(2):242–271, 2000.
  * [15] Stefanos Nikolaidis and Julie Shah.  Human-robot teaming using shared mental models.  ACM/IEEE HRI, 2012.
  * [16] Stefan Palan and Christian Schitter.  Prolific. ac—a subject pool for online experiments.  Journal of Behavioral and Experimental Finance, 17:22–27, 2018.
  * [17] William B Rouse, Janis A Cannon-Bowers, and Eduardo Salas.  The role of mental models in team performance in complex systems.  IEEE transactions on systems, man, and cybernetics, 22(6):1296–1308, 1992.
  * [18] Matthias Scheutz.  Computational mechanisms for mental models in human-robot interaction.  In Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments: 5th International Conference, VAMR 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part I 5, pages 304–312. Springer, 2013.
  * [19] Matthias Scheutz, Scott A DeLoach, and Julie A Adams.  A framework for developing and using shared mental models in human-agent teams.  Journal of Cognitive Engineering and Decision Making, 11(3):203–224, 2017.
  * [20] Mariah L Schrum, Erin Hedlund-Botti, Nina Moorman, and Matthew C Gombolay.  Mind meld: Personalized meta-learning for robot-centric imitation learning.  In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 157–165. IEEE, 2022.
  * [21] David Schuster, Scott Ososky, Florian Jentsch, Elizabeth Phillips, Christian Lebiere, and William A Evans.  A research approach to shared mental models and situation assessment in future robot teams.  In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 55, pages 456–460. SAGE Publications Sage CA: Los Angeles, CA, 2011.
  * [22] Aaquib Tabrez, Matthew B Luebbers, and Bradley Hayes.  A survey of mental modeling techniques in human–robot teaming.  Current Robotics Reports, 1:259–267, 2020.
  * [23] Andrew B Talone, Elizabeth Phillips, Scott Ososky, and Florian Jentsch.  An evaluation of human mental models of tactical robot movement.  In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 59, pages 1558–1562. SAGE Publications Sage CA: Los Angeles, CA, 2015.
  * [24] Basile Van Hoorick, Pavel Tokmakov, Simon Stent, Jie Li, and Carl Vondrick.  Tracking through containers and occluders in the wild.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13802–13812, 2023.


# A first principles investigated optical spectra of oxizided graphene

N. Singh1,2, T. P. Kaloni1, and Udo Schwingenschlögl1
<EMAIL_ADDRESS><EMAIL_ADDRESS>1 Physical
Science & Engineering Division, KAUST, Thuwal 23955-6900, Kingdom of Saudi
Arabia  2 Solar and Photovoltaic Energy Research Center (SPERC), KAUST, Thuwal
23955-6900, Kingdom of Saudi Arabia

###### Abstract

The electronic and optical properties of mono, di, tri, and tetravacancies in
graphene are studied in comparison to each other, using density functional
theory. In addition, oxidized monovacancies are considered for different
oxygen concentrations. Pristine graphene is found to be more absorptive than
any defect configuration at low energy. We demonstrate characteristic
differences in the optical spectra of the various defects for energies up to 3
eV. This makes it possible to quantify by optical spectroscopy the ratios of
the defect species present in a sample.

two-dimensional materials, graphene oxide, desity functional theory, and
absorption spectrum

While graphene is a zero band gap material, but a finite band gap is needed
for various applications aiming at graphene based electronic devices schwierz
; kaloni3 . Functionalization is one of the possible methods to open a band
gap for example by simple oxidation wu ; priya ; kaloni . Graphene oxide (GO)
with epoxy, carbonyl, or hydroxyl groups could allow to tune the a band gap
and therefor tailor the electronic, mechanical, and optical properties dai ;
andre . The atomic structure of GO has been studied experimentally weiwei ;
ajayan  and theoretically sumit . Recently, GO nanostructures have created a
lot of attention due to the fact that it paved the way for solution based
synthesis of graphene sheets, low cost, easy processibility, and compatibility
with various substrates Joung . The band gap of GO can be tunable by just
varying the oxidation level. Fully oxidized GO can act as an electrical
insulator and partially oxidized GO can act as a semiconductor Loh . Moreover,
experiments demonstrat that GO nanostructures have promising applications in
photocatalysis Krishnamoorthy . Reduction of GO may pave the way to mass
production of graphene shenoy .

While GO is usually insulating, a controlled deoxidation can lead to an
electrically and optically active material that is transparent and conducting.
Furthermore, in contrast to pristine graphene, GO is fluorescent over a broad
range of wavelengths, owing to its heterogeneous electronic structure Loh . It
can contain different chemical compositions of carbon, oxygen, and hydrogen
weiwei ; lu . While commonly hydroxyl and epoxy groups are found, there can be
small contributions of carbonyl and carboxyl groups. Experimentally, a
coverage of between 25% and 75% has been observed, reflecting that typically a
quarter of the C$-$C bonds are double bonds whereas the rest are single bonds
Katsnelson . Adsorption behavior of oxygen atoms on the graphene sheets has
been studied by using first-principles calculations ito  and found that the
lattice constant increases with the increase of the ratio of $O/C$ because of
the formation of the epoxy group. At 50% $O/C$ ratio, a finite band gap of
3.39 eV is reported ito .

The attachment of a carbonyl group leads to an almost planar $sp^{2}$
electronic configuration because the formation of C=O bonds induces little
strain in the graphene sheet. On the contrary, the attachment of an epoxy
group leads to a non-planar distorted $sp^{3}$ electronic configuration for
those C atoms which are connected to O, which creates a significant strain on
neighboring C$-$C bonds shenoy . The combination of $sp^{2}$ and $sp^{3}$
configurations as well as defects breaks the hexagonal symmetry of pristine
graphene and a band gap is opened. The coexistence of $sp^{2}$ and $sp^{3}$
configurations is confirmed experimentally yun  and theoretically shenoy . The
defects associated with dangling bonds enhance the reactivity substantially.

Figure 1: Calculated band structures of (a) a mono-vacancy (b) a di-vacancy,
(c) tri-vacancy, and (d) tetra-vacancy in graphene along the path K’1 (0.6667
0.0000 0.0000), K2 (0.3333 -0.5774 0.0000), $\Gamma$ (0.0000 0.0000 0.0000),
K’1 (0.6667 0.0000 0.0000), K1 (0.3333 0.5773 0.0000), M1 (0.500 0.2887
0.0000), and $\Gamma$ (0.0000 0.0000 0.0000) in unit of $\frac{2\pi}{a}$.

Recently, a theoretical investigation of the electronic and optical properties
of GO (without vacancies) for different functional groups and various
compositions has been reported in Ref priya . The authors found that carbonyl
groups are favourable for photoluminescense and that the optical gap of
reduced GO is samaller than the optical gap of pristine and fully oxidize
graphene. Theoretical and experimental studies Katsnelson1 ; Wang  indicate
that hydroxyl, carboxyl, and other functional groups can easily be attached to
vacancies in graphene than to pristine graphene. Therefore, a study of the
electronic and optical properties of defective GO becomes critical. Optical
properties of oxidized mono-, di-, tri-, and tetra-vacancies in graphene have
not been reported so far. In this work, we use first principles calculations
of to provide insight into this topic.

Our calculations are based on density functional theory and carried out using
the generalized gradient approximation of Quantum Espresso pacakage, paolo ;
pbe . All calculations are performed with a plane wave cutoff energy of 544
eV. We use a Monkhorst-Pack Monk  of $8\times 8\times 1$ k-mesh for the
Brillouin zone integration in order to relaxing the structures and achieving
highly accurate electronic structure. We observe that a $5\times 5$ supercell
of graphene is sufficiently large for monovacancies and oxidized monovacancies
kaloni . Our supercell has a lattice constant of $a=12.2$ Å and extends $c=20$
Å in the perpendicular direction. It has been reported that nearby vacancies
behave independently when they are separated by $\sim$7 Å andrey . Hence, we
use a $6\times 6$ supercell for di-, tri-, and tetra-vacancies to avoid
artificial interaction of the periodic images. This supercell has a lattice
constant of $a=14.69$ Å and again $c=20$ Å. The cell parameters and atomic
positions are fully relaxed until a force convergence of 0.05 eV/Å and an
energy convergence of 10-7 eV are reached. The relaxed structures are used to
calculate the optical properties by WIEN2k Wien2k  code. For the optical
calculations, a dense mesh of uniformely distributed k-points is required.
Hence, the Brillouin zone integration is performed using tetrahedron method
with 180 k-points in the irreducible part of the Brillouin zone. Well
converged solutions are obtained for $R_{mt}\times K_{max}$ =7, where
$K_{max}$ is the plane-wave cut-off and $R_{mt}$ is the smallest of all atomic
sphere radii.

The dielectric function
($\varepsilon(\omega)=\varepsilon_{1}(\omega)+i\varepsilon_{2}(\omega)$) is
known to describe the optical response of the medium. The interband
contribution to the imaginary part of the dielectric function
$\varepsilon(\omega)$ is calculated by summing all transitions from occupied
to unoccupied states (with fixed k) over the Brillouin zone, weighted with the
appropriate matrix elements giving the probability for the transitions. The
imaginary part of dielectric function $\varepsilon_{2}(\omega)$ is given as in
wooten  by

$\epsilon_{2}(\omega)=\frac{4\pi^{2}e^{2}}{m^{2}\omega^{2}}\sum_{i,j}\int<i|M|j>^{2}\times
f_{i}(1-f_{i})\delta(E_{f}-E_{i}-\omega)d^{3}k$ (1)

Where M is dipole matrix, _i_ and j are the initial and final states,
respectively, $\emph{f}_{i}$ is the Fermi distribution function for the
$i_{t}h$ states, and $E_{i}$ is the energy of electron in the $i_{t}h$ state.
The real part of the dielectric function can be extrated from the imaginary
part using the Kramers-Kronig relation wooten ; Yu  in the from

$\epsilon_{1}(\omega)=1+\frac{2}{\pi}P\int_{0}^{\infty}\frac{\omega^{{}^{\prime}}\epsilon_{2}(\omega^{{}^{\prime}})}{\omega^{{}^{\prime}2}-\omega^{2}}d\omega^{{}^{\prime}}$
(2)

Where P implies the principle value of the integral.

The reflectivity spectra are derived from Fresnel’s formula for normal
incidence assuming an orientation of the crystal surface parallel to the
optical axis using the relation fox

$R(\omega)=|\frac{\sqrt{\varepsilon(\omega)}-1}{\sqrt{\varepsilon(\omega)}+1}|^{2}$
(3)

The knowledge of both real and imaginary parts of the dielectric tensor allows
the calculations of the important optical functions. We calculate the
absorption, the real part of optical conductivity, and the electron energy-
loss spectrum using the following expressions fox ; delin

$\alpha(\omega)=\sqrt{2}\omega(\sqrt{\varepsilon_{1}(\omega)^{2}+\varepsilon_{2}(\omega)^{2}}-\varepsilon_{1}(\omega))^{1/2}$
(4)

$Re\sigma(\omega)=\frac{\omega\varepsilon_{2}}{4\pi}$ (5)

$L(\omega)=\frac{\varepsilon_{2}(\omega)}{\varepsilon(\omega)^{2}+\varepsilon_{2}(\omega)^{2}}$
(6)

This approach has been successfully applied to narrow band gap materials
including rare earth Zintl compounds Zintl ; Zintl1 , and oxides singh . For a
reliable integration, a set of 180 k-points in the irreducible wedge of the
Brillouin zone is used. A Lorentzian broadening is used to simulate the
effects of finite life-time and finite resolution of the optical measurement.

Figure 2: Calculated optical absorption $\alpha(\omega)$ in $10^{4}$ cm-1,
optical conductivity $\sigma(\omega)$, reflectivity $R(\omega)$ in %, and
energy loss function $L(\omega)$ of pristine graphene, as compared to graphene
with mono, di-, tri, and tetra-vacancies and oxidized monovacancy.

The calculated values of formation energy of mono-, di-, tri-, and tetra-
vacancies in graphene are 7.50, 6.94 eV, 11.45 eV, and 12.58 eV, respectively.
This means the formation of a divacancy in graphene is more favourable than
the formation of a single vacancy. Furthermore, a divacancy is known to be
more stable than two isolated monovacancies (whose migration energy barrier is
rather low), because the dangling C-C bonds of atoms next to the vacancy can
be passivated by eachother Lee . The vacancies in graphene induce
ferromagnetism with total magnetic moments of 1.35 $\mu_{B}$, 1.00 $\mu_{B}$,
2.00 $\mu_{B}$ for mono-, tri-, tetra-vacancies, respectively. Di-vacancies
shows no spin-polarization. The results of our band structure (BS)
calculations for mono-, di-, tri-, and tetra-vacancies are shown in Fig. 1
together with the corresponding DOS. For the sake of comparison we have
included the BS (dotted lines) of pristine graphene. For an oxidized
monovacancy, the magnetic and electronic properties have been reported in
previous literature kaloni ; dai ; Yazyev . In $6\times 6$ supercell, the
Dirac cone is shifted to the $\Gamma$-point due to Brillouin zone folding in
$6\times 6$ supercell. In case of the tetra-vacancy, the BS shows that a
single minority spin band crosses the Fermi energy ($E_{F}$) at the
$\Gamma$-point and leaves the system metallic, whereas for the di-, tri-
vacancies both majority and minority bands cross $E_{F}$. Moreover, an upward
shift of the Dirac point is indicative of a hole-doped system. In case of the
di-vacancy, the DOS is identical for the majority and minority spins,
reflecting spin-degeneracy. It means pristine graphene becomes ferromagnetic
by a single vacancy defect and can be non-magnetic metal by divacancy.

The optical spectroscopy is a valuable tool in material science. Here, In the
optical calculations, selfenergy and excitonic effects are not taken into
account. It has been shown for graphene that in the energy range upto 3 eV,
where the approximation of Dirac particles is valid, the influence of the
many-particle effects is negligible. The calculated optical spectra of
pristine graphene and its functionalized derivatives are addressed in Fig 1.
The optical spectra show that for the two adsorbed O atoms, a band gap of 0.5
eV is opened due to the symmtery breaking and increased $sp^{3}$ characters. A
semiconducting behaviour for this configuration is conformed by our previous
calculations of BS and DOS kaloni . As comparised to pristine graphene, all
defects are found to create metallic states as shown in Fig 1.

In general, pristine graphene is more absorptive as compared to the other
systems but for di-vacancy, absoption between 7.5 eV to 10 eV is higher than
pristine graphene. The $\alpha(\omega)$ peak at 4.5 eV for pristine graphene
splits into two peaks at 2.7 eV and 5 eV by the splitting of the Dirac cone
for two attached O atoms. The splitting increasing with the O coverage. For
di-vacancy and three adsorbed O atoms an additional sharp peak at 1.25 eV
(visible region) is found which is absent for other systems. The pristine
graphene have high reflectivity in the low energy as compared to other cases.
The reflectivity is higher for the tetra-vacancy (most pronounced metallicity)
as compare to mono-, di-, and tri-vacancies, but lower than for pristine
graphene. The reflectivity in low energy range is the lowest for the case of
monovacancy with two attached O atoms, due to its semiconducting nature and
increases again for three and four attached O atoms. The most prominent peaks
in $\sigma(\omega)$ become broaden and the magnitude also decreases as one
moves from pristine graphene to tetra-vacancies. The $\sigma(\omega)$ is low
for tri-, tetra-vacancies and monovacancy with four attached O atoms as
compared to remaining systems. A large peak is observed at around 4.6 eV in
all optical spectra for all the systems which is attributed to the
$\pi-\pi^{*}$ transitions of the aromatic C$-$C atoms. The maximum peak in
energy loss spectra is at 4.9 eV, which is assigned to the energy of the
volume plasmon $\hbar$ $\omega_{p}$. This maximum peak positions remain same
for all systems. The peaks in optical spectra originates from the transition
from valence band to conduction band. The dominating peak in the energy loss
spectrum broadens from graphene to the oxidize vacancies with monovacancy.

In conclusion, we have studied the optical properties of graphene derivatives
(clean and oxidized vacancies) by means of density functional theory. We find
that the formation of divacancies in graphene is energetically favourable.
Divacancies are also exceptional in the sense that they do not lead to a local
magnetic moment. Mono, di, tri, and tetravacancies are found to be metallic,
while an oxidized monovacancy with two adsorbed oxygen atoms leads to a band
gap of 0.5 eV (due to a splitting of the Dirac cone). Our optical spectra show
that pristine graphene has the highest absorption in the energy range below
2.5 eV. In two cases (tetravacancy and monovacancy with three adsorbed oxygen
atoms) a prominent absorption peak appears in the visible range. Our
calculations suggest that the types of (oxidized) defects present in a
graphene sample can be quantified by optical spectroscopy. With this
knowledge, the electronic and optical properties of graphene derivatives can
be tuned by controlled oxidation and reduction.

## References

  * (1) F. Schwierz, Nat. Nanotechnol. 5, 487 (2010).
  * (2) T. P. Kaloni, Y. C. Cheng and U. Schwingenschlögl, J. Mater. Chem. 22, 919 (2012)
  * (3) X. Wu, M. Sprinkle, X. Li, F. Ming, C. Berger, and W. A. de Heer, Phys. Rev. Lett. 101, 026801 (2008)
  * (4) P. Johari and V. B. Shenoy, ACS Nano 9, 7640 (2011).
  * (5) T. P. Kaloni, Y. C. Cheng, R. Faccio, and U. Schwingenschlögl, J. Mater. Chem. 21, 18284 (2011).
  * (6) J. Dai and J. Yuan, Phys. Rev. B 81, 165414 (2010).
  * (7) K. A. Mkhoyan, A. W. Contryman, J. Silcox, D. A. Stewart, G. Eda, C. Mattevi, S. Miller, and M. Chhowalla, Nano Lett. 9 1058 (2009).
  * (8) W. Cai, R. D. Piner, F. J. Stadermann, S. Park, M. A. Shaibat, Y. Ishii, D. Yang, A. Velamakanni, S. J. An, M. Stoller, J. An, D. Chen, and R. S. Ruoff, Science 321, 1815 (2008).
  * (9) W. Gao, L. B. Alemany, L. Ci, and P. M. Ajayan, Nat. Chem. 1, 403 (2009).
  * (10) S. Saxena, T. A. Tyson, S. Shukla, E. Negusse, H. Chen, and J. Bai, Appl. Phys. Lett. 99, 013104 (2011).
  * (11) D. Joung, A. Chunder, L. Zhai, and S. I. Khondaker, Appl. Phys. Lett. 97, 093105 (2010)
  * (12) K. P. Loh, Q. Bao, G. Eda, and M. Chhowalla, Nat. Chem. 2, 907 (2010).
  * (13) K. Krishnamoorthy, R. Mohan, and S. J. Kim, Appl. Phys. Lett. 98, 244101 (2011).
  * (14) A. Bagri, C. Mattevi, M. Acik, Y. J. Chabal, M. Chhowalla, and V. B. Shenoy, Nat. Chem. 2, 581 (2010).
  * (15) N. Lu, Y. Huang, H.-B. Li, Z. Li, and J. Yang, J. Chem. Phys. 133, 034502 (2010).
  * (16) D. W. Boukhvalov and M. I. Katsnelson, J. Am. Chem. Soc. 130, 10697 (2008).
  * (17) J. Ito, J. Nakamuraa, and A. Natori, J. Appl. Phys. 103, 113712 (2008).
  * (18) H. K. Jeong, Y. P. Lee, R. J. W. E. Lahaye, M.-H. Park, K. H. An, I. J. Kim, C.-W. Yang, C. Y. Park, R. S. Ruoff, and Y. H. Lee, J. Am. Chem. Soc. 130, 1362 (2008).
  * (19) D. W. Boukhvalov and M. I. Katsnelson, Nano Lett. 8, 4373 (2008).
  * (20) X. Wang, S. M. Tabakman, and H. Dai, J. Am. Chem. Soc. 130, 8152 (2008).
  * (21) P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, A. Dal Corso, S. de Gironcoli, S. Fabris, G. Fratesi, R. Gebauer, U. Gerstmann, C. Gougoussis, A. Kokalj, M. Lazzeri, L. Martin-Samos, N. Marzari, F. Mauri, R. Mazzarello, S. Paolini, A. Pasquarello, L. Paulatto, C. Sbraccia, S. Scandolo, G. Sclauzero, A. P. Seitsonen, A. Smogunov, P. Umari, and R. M. Wentzcovitch, J. Phys. Condens. Matt. 21, 395502 (2009).
  * (22) J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * (23) H. J. Monkhorst and J. D. Pack, Phys. Rev. B 13, 5188 (1976).
  * (24) V. V. Nelayev and A. I. Mironchik, Materials Phys. and Mechanics 9, 26 (2010).
  * (25) P. Blaha, K. Schwarz, G. Madsen, D. Kvasicka, and J. Luitz, WIEN2k, An Augmented Plane Wave + Local Orbitals Program for Calculating Crystal Properties (Technical University of Vienna, Vienna, 2001).
  * (26) F. Wooten: Optical Properties of Solids (Academic Press, New York, 1972).
  * (27) Y. P. Yu and M. Cardona 1999 Fundamentals of Semiconductors: Physics and Materials Properties 2nd edn, Berlin: Springer (1999)
  * (28) M. Fox, Optical Properties of Solids, New York: Oxford University Press (2001).
  * (29) A. Delin, O. Eriksson, R. Ahuja, B. Johansson, M. S. S. Brooks, T. Gasche, S. Auluck and J. M. Wills, Phys. Rev. B 54 1673 (1996).
  * (30) N. Singh and U. Schwingenschlögl, Chem. Phys. Lett. 508, 29 (2011).
  * (31) N. Singh and U. Schwingenschlögl, Appl. Phys. Lett. 100, 151906 (2012).
  * (32) D. J. Singh, R.C. Rai, J.L. Musfeldt, S. Auluck, N. Singh, P. Khalifah, S. McClure, and D. G. Mandrus, Chem. Mat. 18, 2696 (2006).
  * (33) G.-D. Lee, C. Z. Wang, E. Yoon, N.-M. Hwang, D.-Y. Kim, and K. M. Ho, Phys. Rev. Lett. 95, 205501 (2005).
  * (34) O. V. Yazyev and L. Helm, Phys. Rev. B 75, 125408 (2007).