# Robot Learning Theory of Mind through Self-Observation: Exploiting the
Intentions-Beliefs Synergy

Francesca Bianco1 and Dimitri Ognibene2,1,∗ *This work was not supported by
any organization1University of Essex, Colchester, UK2Università degli Studi di
Milano Bicocca, Milano<EMAIL_ADDRESS>
###### Abstract

In complex environments, where the human sensory system reaches its limits,
our behaviour is strongly driven by our beliefs about the state of the world
around us. Accessing others’ beliefs, intentions, or mental states in general,
could thus allow for more effective social interactions in natural contexts.
Yet these variables are not directly observable. Theory of Mind (TOM), the
ability to attribute to other agents’ beliefs, intentions, or mental states in
general, is a crucial feature of human social interaction and has become of
interest to the robotics community. Recently, new models that are able to
learn TOM have been introduced. In this paper, we show the synergy between
learning to predict low-level mental states, such as intentions and goals, and
attributing high-level ones, such as beliefs. Assuming that learning of
beliefs can take place by observing own decision and beliefs estimation
processes in partially observable environments and using a simple feed-forward
deep learning model, we show that when learning to predict others’ intentions
and actions, faster and more accurate predictions can be acquired if beliefs
attribution is learnt simultaneously with action and intentions prediction. We
show that the learning performance improves even when observing agents with a
different decision process and is higher when observing beliefs-driven chunks
of behaviour. We propose that our architectural approach can be relevant for
the design of future adaptive social robots that should be able to
autonomously understand and assist human partners in novel natural
environments and tasks.

##  I INTRODUCTION

Due to recent technological developments, the interactions between AI and
humans have become pervasive and heterogeneous, extending from voice
assistants or recommender systems supporting the online experience of millions
of users to autonomous driving cars. Principled models to represent the human
collaborators’ needs are being adopted [1] while robotic perception in complex
environments is becoming more flexible and adaptive [2, 3, 4, 5] even in
social contexts, robot sensory limits are starting to be actively managed [6,
7]. However, robots and intelligent systems still have a limited understanding
of how sensory limits affect human partners’ behaviour and lead them to rely
on internal beliefs about the state of the world. This strongly impacts human-
robot mutual understanding [8] and calls for an effort to transfer the advance
in robot perception management to methods to better cope with human
collaborators’ perceptual limits[9, 10, 11].

The possibility of introducing in robots and AI systems a Theory of Mind (TOM)
[12], the ability to attribute to other agents’ beliefs, intentions, or mental
states in general, has recently raised hopes to further improve robots’ social
skills [13, 14, 15, 16]. While some studies have explored human partners’
tendency to attribute mental states to robots [17, 18, 19, 20], the expected
practical impact of TOM led to a diverse set of TOM implementations on robots.
Several implementations relied on hardwired agents and task models that could
be applied to infer mental states in settings known at design time [21, 22,
23, 24, 25]. A step forward is presented in [26] with an algorithm to
understand unknown agents relying upon Belief-Desire-Intention models of
previously met agents.

Recently, following [27] seminal work, several models have introduced deep
learning based TOM implementations [28, 29, 30, 31, 32, 33]. This novel
approach, learning both beliefs and intention attribution, should allow
improved collaboration and adaptive human-robot collaboration in complex
environments through a better understanding of humans’ mental states. In this
paper, we explore if the data-driven approach proposed in [27] and related
works leads to improved predictions of the partner’s intentions, which is
often the mental state with the highest impact on the interaction performance.
The prediction of partners’ intentions, even within a system producing
prediction on several others’ unobservable mental states, such as beliefs,
will still rely only on the processing of observable behavioural inputs, aka
state-action trajectories. In a purely supervised learning setting, such as
that proposed in [27], it is not immediate why performing an additional set of
predictions, increasing the demands on the social perception system, should
result in higher accuracy for the prediction of others’ intentions. This
approach introduces additional complexity and noise that may hinder
performance (see [34]). Moreover, deep learning models as those proposed in
[27] are usually data hungry, which may further affect the value of the
approach. These factors may be some of the reasons for the long time required
for the full development of TOM in infants [12, 35].

While all these considerations sound technically valid, our results with
simplified versions of the architecture proposed in [27] show that the
original hypothesis may be true: learning is faster and more accurate if it
takes place simultaneously for the prediction of both intentions and beliefs
together. Our results also show that the impact of learning beliefs
attribution on intention prediction is stronger in conditions of strong
partial observability, e.g. when the observed agent does not still know where
his target is. We found that when the system learns to predict intentions and
beliefs at the same time it can better disambiguate and discard unrelated
objects that are or have been in the sensory field of the observed agent. This
can be particularly relevant for assistive applications, especially those
based on egovision that can monitor the sensory state of the partners [36].
Our hypothesis is that this is due to the regularization effect of multitask
learning when tasks don’t present conflict but synergy [37, 38, 39, 40].
Indeed the observed accuracy gain decreases when the dataset size exceeds a
certain threshold and makes regularization less helpful. On the other side,
when very limited experience is available, the performance is slightly worse
for joint prediction, maybe explaining the necessity for the complex and
multi-system developmental process that seems to characterize humans [12].

Another issue of the approach proposed in [27] is the lack of training
samples, as others’ mental states are usually not available through direct
behaviour observation or in datasets for training or prediction. We propose
that self-observation, the observation of the agent’s own behaviours and the
internal decision and beliefs estimation processes can be used as the training
signal. We tested that mental states prediction can then be generalized to
different agents, even if within certain limits.

From a cognitive modelling perspective, the proposed architecture detaches
from the motor simulation tradition common in robotics [7, 41, 6, 42, 43],
where the motor control system is used for both action execution and
perception. In our model, the motor control system is only the source of the
reference signal for the training of the social perception system. Among the
others, this strategy has the advantage of reducing the strong interference
between action and observation that a motor simulation account of social
perception could present [44, 45, 46]. It could also reduce the computational
demands and delays that could be involved in simulating others’ actions while
it is still able to exploit the observer’s (motor or action) knowledge for
social perception [47]. The presented architecture can be related to the
associative hypothesis of social perception [48] and in particular, to Hommel
et al.’s[49] interpretation, as described in [50], which has extended the
associative view to intention interpretation and proposes that associations
between behaviours (actions) and underlying intentions (effect) develop from
an early age and can be used in later in life as a means to infer and predict
others’ intentions and behaviours. The associative account has also been
previously related to others’ understanding by proposals of its involvement in
mirror neurons development (e.g., [51]), as well as in sensorimotor matching
for imitation [52]. The architecture proposed in the present paper differs
from these accounts as it focuses on beliefs [53]. Specifically, It relies on
associations between explicit belief representations learnt through self-
experience and consequent behaviours to improve the prediction performance of
others’ beliefs-driven behaviours.

##  II METHODS

###  II-A Architecture

The agent is composed of two components (see Fig.1), one is the Social
Perception System (SPS), that interpret and allows for predictions of observed
agents behaviour, in terms of next actions, goals and even beliefs. The other
component is the control system that define the agent behaviour to perform
tasks similar to that it interprets in the SPS. The quantities, e.g.
intentions, beliefs, actions, produced during the performance of the task are
also used to fully train the SPS, while others’ behaviours are used only for
partial training, as others’ beliefs cannot be observed.

Figure 1: The architecture utilised in the here reported studies, formed of a
shared prediction net torso and subsequently of separate prediction heads. For
the NoBeliefs architecture, the following prediction heads are considered: 1.
Target position, 2. Actor’s next action, and 3. Actor’s next state. For the
Beliefs architecture, the 4. belief prediction head (in red) is also
considered

####  II-A1 Social Perception System (SPS)

The SPS’s goal is to make predictions about the observed actors’ future
behaviour, with a specific interest on the actor’s target position. Two types
of SPS were trained: NoBelief SPS, the baseline, which predicts an actor’s
target position, target class, next action, and next resulting state but not
the beliefs; Beliefs SPS, was asked to predict, in addition to the previous,
also the actor’s beliefs.

##### Input sensing and routing

The SPS can observe themselves or others; the input vector for the system can
then be provided by a common reference frame to represent either the self-
localisation state of the observer, during self-observation learning, or the
physical state of the other actor, both for learning and prediction of other
actor’s behaviour. Several architectures have studied the problem [7, 41] of
how to switch between the processing of own and others’ data and how to
acquire others’ physical states. Note that in this case this function, while
important, poses less constraints to behaviour performance as it feeds not the
execution process but the social learning one [54].

##### Input encoding and pre-processing

The inputs is formed by a number (max 5 in the reported experiment) of past
steps of a trajectory on a single grid map. Observed actions-states pairs are
combined through a spatialisation-concatenation operation, whereby actions are
tiled over space into a tensor and concatenated to form a single tensor of
shape (11 x 11 x 20). While 11 x 11 represents the size of the grid world
environments, 20 vectors are provided as inputs consisting of information
regarding (a) actions (9 possible actions in experiments, thus 9 vectors); (b)
objects coordinates, including the target position (4 objects, thus 4 vectors,
one for each object class); (c) actor’s position in past steps (5 past steps,
thus 5 vectors); (d) 1 feature plane for the walls in environments; and (e) 1
vector for the actor’s current position.

##### Prediction Net

Following spatialisation-concatenation operation, tensors are passed through a
deep learning architecture. The architectures utilised to conduct the reported
experiments are formed of a shared prediction net torso and subsequently of
separate prediction heads.

  * •

The torso is implemented as a 2-layer ResNet with 32 channels, leaky ReLU
nonlinearities, and batch-norm. The heads have all a similar architecture but
differ in terms of output size and format.

  * •

Target Location Prediction Head. The output from the torso is inputted into a
1-layer Convnet with 32 channels and leaky RELU, another 1-layer Convnet with
16 channels and leaky RELU, followed separately by (a) a fully connected layer
to 121-dim logits (11 x 11 grid world) and (b) another 1-layer Convnet with 4
channels to 1. These are then summed, thus following a residual network
approach.

  * •

Next Action Prediction Head. The output from the torso is inputted into a
1-layer Convnet with 32 channels and leaky RELU, followed by average pooling,
and 2 fully connected layers to 9-dimensions (9 possible actions).

  * •

Next State Prediction Head. The output from the torso is inputted into a
1-layer Convnet with 32 channels and leaky RELU, another 1-layer Convnet with
16 channels and leaky RELU, followed separately by (a) a fully connected layer
to 121-dim logits (11 x 11 grid world) and (b) another 1-layer Convnet with 4
channels to 1. These are then summed.

  * •

Beliefs Prediction Head. The output from the torso is inputted into a 1-layer
Convnet with 32 channels and leaky RELU, another 1-layer Convnet with 16
channels and leaky RELU, followed separately by (a) a fully connected layer to
121-dim logits (11 x 11 grid world) and (b) another 1-layer Convnet with 4
channels to 1. These are then summed.

####  II-A2 Agent Control System

The agent has partial observability over the target position in the
environment, i.e., they could see it only when it was in their 5x5 field of
view, but they were fully informed of their position and that of the walls. To
account for the resulting uncertainty and related information-gathering
behaviours [55], the actors’ trajectories were generated using the POMDP
planner based on Montecarlo tree search proposed by [56] which extends [57].
It integrates a Bayesian filter that explicitly represents the actor’s beliefs
about the state of the task, i.e. the probability distribution on the target
position. Note that while the beliefs design of the observed actor are
necessarily determined by its task, this does not affect the generality of the
observer’s performance because it is blind to the task and belief design
assumptions.

###  II-B Environments

For all experiments in this study, the environments consisted of $11\times 11$
grid world maps, which varied in the location of walls, columns, and free
cells to move around the map (see Fig. 2 for a visualisation of example random
maps and connected behaviours). To assess the impact of dataset size on the
proposed approach, we created multiple training datasets comprising 5, 10, 15,
20, 25, 30, 60, 120, and 300 maps, together with a 10 maps testing set. The
environments enabled a common action space (north, east, south, west,
northeast, northwest, southeast, southwest, stay) with deterministic results.
During each trial, both training and testing, the target, the distractors, and
the agent are randomly positioned on free cells.

###  II-C Trajectory Dataset Generation

A total of 30 trajectories were generated per grid world map by randomly
selecting for each trajectory the initial locations of both the actor and
target. For example, when considering the dataset with 60 grid world maps, a
total of 1800 actors’ behaviours were created, while 9000 total behaviours
were created for that with 300 maps. At training and test time, 3 distractor
objects, identical to the target, were positioned at different empty locations
on the map. The positioning of the distractor objects was random in
experiments where it is not specified differently. This enabled a wider
environment and behavioural diversity with no additional computational cost.

Figure 2: Visualisation of example 11x11 grid world maps and observed
trajectories, which varied in the location of walls, columns and free cells to
move around the map. Colour code - Black: walls; White: empty cells; Yellow:
target; Green: distractor objects; Blue: current actor’s position; Pink: past
actor’s positions.

###  II-D Training

All architectures (both Beliefs and NoBeliefs) were trained with the Adam
optimiser, with varying learning rates (we tested 6 levels from 0.00015 to
0.001), using batches of size 32. A learning rate scheduler with the following
parameters was used in all experiments: milestones = [30, 60, 80, 160], gamma
= 0.5. The Cross Entropy loss function was utilised for all heads training,
except for the Belief head, for which the Kullback–Leibler divergence loss
function was used instead. This choice was driven by the fact that the Action,
State and Target prediction heads all required one-hot encoding to identify
one single position in the map, whereas a distribution of probabilities over
each map location was needed for the Belief prediction head. A preliminary L1
and L2 regularisation tuning was performed using the L1 and L2 factor search;
the final values of the L1 and L2 parameters used in all experiments were
0.005 and 0.001, respectively. In addition, early stopping was also integrated
during training as a means to prevent overfitting.

##  III RESULTS

###  III-A Global comparison

Table I shows the performance of the two architectures evaluated over the
whole test set. The worst observed performance is $59.26$ (BEL architecture, 5
training maps) while the best is $72.43$ (BEL, 300 maps). The Beliefs
architecture shows better performance than the NoBeliefs one in all conditions
but the first one with 5 maps. The gain is statistically significant
($p-value<0.01$) after the agent could observe the task performed over more
than 20 training maps (600 task executions). Under 5 training maps (150
observed task executions) the performance variance across trained nets is
high. The additional learning constraint posed by the beliefs head seems to
strengthen the over-fit of the network. However, over the learning trajectory,
with a more reasonable dataset size, the beliefs prediction contributes to
achieving higher performance. The maximum performance difference observed is
of $1.89\%$ over a performance of the NoBeliefs architecture of $67.57\%$ when
the agent was trained with 25 maps (750 task executions). Similar results were
also obtained when changing the number of layers in the SPS components or the
number of objects in the scene. An interesting trend to note is that the
contribution of the beliefs attribution learning is stronger in the middle of
the trajectory, when the architecture stops memorizing the beliefs samples and
obtains regularization effects, while becomes less useful with a high number
of samples when regularization is not needed. On the other side, the lower
performance with 5 maps may point to the necessity of a complementary
intention prediction dedicated system for such early stages, relating to the
complexity of TOM development observed in infants [12, 35].

TABLE I: Target prediction accuracy

| BEL | NoBEL |  |
---|---|---|---|---
Train Maps (N) | Best LR |  | Avg Acc
---
(%)
Var | Best LR |  | Avg Acc
---
(%)
Var |  Bel-NoBel (%) | p-value
5 | 0.001 | 59.26 | 18.84 | 0.001 | 61.51 | 20.30 | -2.25 | .043
10 | 0.001 | 65.75 | 8.99 | 0.001 | 65.46 | 11.82 | 0.29 | .488
15 | 0.001 | 67.54 | 2.79 | 0.001 | 66.85 | 1.84 | 0.69 | .077
20 | 0.001 | 68.74 | 2.08 | 0.001 | 67.47 | 1.51 | 1.27 | .001
25 | 0.001 | 69.45 | 1.63 | 0.001 | 67.57 | 1.44 | 1.87 | <.001
30 | 0.001 | 69.49 | 1.60 | 0.001 | 67.79 | 1.76 | 1.69 | <.001
60 | 0.00075 | 70.49 | 1.23 | 0.001 | 69.32 | 1.33 | 1.17 | <.001
120 | 0.00075 | 71.47 | 0.87 | 0.0005 | 70.46 | 0.89 | 1.00 | <.001
300 | 0.00015 | 72.43 | 0.28 | 0.00015 | 71.59 | 0.43 | 0.84 | <.001

Best target prediction accuracies by best learning rate obtained for the
Beliefs vs NoBeliefs architectures based on the number of maps made available
during training. Accuracies were calculated as averages across 18 initial
network weights; the associated variances were reported.
BEL= Beliefs architecture; NoBEL = NoBeliefs architecture

###  III-B Comparison with hidden target

Beliefs representations determine behavioural chunks when the target object is
not still visible and its position is not deterministically known. In such
conditions, shown in table II, the performance difference between the two
architectures were stronger, with a maximum difference of about $7\%$ over a
performance of the NoBeliefs architecture $51\%$ (thus a relative gain of
$14\%$) when the agent was trained with 15 maps (450 task executions). We must
note also that this setting is also more challenging with the worst
performance at $37.44\%$ and the best at $63.11\%$. Indeed, being the target
position is not known to the agent, it should produce wandering behaviours
that are not informative about the actual target position. However, the
performance was above chance (25%), we thus explored how the SPS could extract
information from the presence of non-target objects, i.e. distractors.

TABLE II: Target prediction accuracy with hidden target and 3 distractors in
view

| Target not visible – 3 Objects visible |
---|---|---
| BEL | NoBEL |  |
| Train Maps
---
(N)
| Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var |  Bel-NoBel (%) | p-value
5 | 37.44 | 96.97 | 38.56 | 31.44 | -1.11 | .681
10 | 52.61 | 29.31 | 48.83 | 17.44 | 3.78 | .025
15 | 58.17 | 27.09 | 51.11 | 16.69 | 7.06 | <.001
20 | 56.94 | 8.29 | 54.17 | 12.85 | 2.78 | .015
25 | 57.11 | 8.58 | 51.56 | 13.91 | 5.56 | <.001
30 | 56.22 | 8.18 | 53.22 | 6.42 | 3.00 | .002
60 | 59.39 | 19.08 | 54.44 | 12.73 | 4.94 | .001
120 | 59.39 | 13.08 | 55.11 | 10.46 | 4.28 | .001
300 | 61.61 | 5.19 | 63.11 | 4.34 | -1.50 | .047

Best target prediction accuracies by best learning rate obtained for the
Beliefs vs NoBeliefs architectures based on the number of maps made available
during training. Accuracies were calculated as averages across 18 initial
network weights; the associated variances were reported. BEL= Beliefs
architecture; NoBEL = NoBeliefs architecture

###  III-C Extracting information from avoided distractors

We reasoned that the most informative distractors would be those that were
avoided (passed through) by the agent during his previous steps. Table III
shows the corresponding performance with a varying number of avoided, or
better neglected, objects, with target visible or out of the field of view.
The performance were much higher with $99\%$ for 3 avoided objects and visible
target independently of the architecture and number of training maps. The
worst performance were still at $79.67\%$ for 1 only avoided object and not
visible target. Learning to predict beliefs is useful even when 3 objects have
been avoided, especially after few training maps. This implies that the
gradient for the belief prediction helps learning to discard the avoided
objects. The maximum gain of the Beliefs architecture is $6.56$ is observed
with 1 avoided objects and a high number of training maps. We assume that this
reflects a strong contribution of the belief learning in discarding not yet
avoided objects but that were not efficiently approached even when the agent
does not still have goal approaching behaviour as it is still searching for
the target. We moved next to study this specific condition.

TABLE III: Target prediction accuracies with varying number of avoided objects
and target visibility

| Target visible – 25 maps |  |  | Target not visible – 25 maps |  |
---|---|---|---|---|---|---
| BEL | NoBEL |  |  | BEL | NoBEL |  |
| Neglected
---
(N)
| Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val |  | Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val
3 | 99 | 0 | 99 | 0 | 0 | 1 | 98.56 | 0.51 | 97.33 | 0.84 | 1.22 | <.001
2 | 98.89 | 0.1 | 98.44 | 0.26 | 0.44 | 0.01 | 94.17 | 0.38 | 91.78 | 2.07 | 2.39 | <.001
1 | 97.72 | 0.21 | 96.67 | 0.59 | 1.06 | <.001 | 85.28 | 0.68 | 81.89 | 6.46 | 3.39 | <.001
| Target visible – 120 maps |  |  | Target not visible – 120 maps |  |
3 | 99 | 0 | 99 | 0 | 0 | 1 | 98.94 | 0.24 | 98.72 | 0.46 | 0.22 | 0.77
2 | 99 | 0 | 98.61 | 0.37 | 0.39 | 0.01 | 94.67 | 0.24 | 90.78 | 13.48 | 3.89 | <.001
1 | 98.44 | 0.26 | 96.83 | 1.44 | 1.61 | <.001 | 86.22 | 0.77 | 79.67 | 33.65 | 6.56 | <.001

Target prediction accuracies for the Beliefs vs NoBeliefs architectures in the
conditions with varying number of neglected objects and target visible by the
actor (25 maps (A), 120 maps (C)) or target not visible by the actor (25 maps
(B), 120 maps (C)). Accuracies were calculated as averages across 18 initial
network weights; the associated variances were reported.

###  III-D Extracting information from target-aligned distractors

We assumed that the least informative distractors would be those that are
aligned with future trajecotry of the agent toward the target. As table IV
shows, the presence of such distractors strongly impacted the system
performance with a worse performance of $11.56\%$ for 25 training maps,
NoBeliefs architecture and 3 aligned objects. The gain of the Beliefs
architecture where even stronger in this hard condition with a top gain of
$12.83\%$ over a NoBeliefs performance of $43.83\%$, thus a relative gain of
about $30\%$ during target visible codnitions. We interpreted this as a clue
that the beliefs learning provides a more efficient way of learning to discard
distractors that have become visible before the targets even if they are
aligned with it and not efficiently approached before. When the target is not
visible the performance gain of Belief network is even stronger, moving from a
minimum go $27\%$ to a maximum of $37.28\%$. This shows that learning to
predict beliefs helps learning to interpret beliefs driven searching behaviour
that would be suboptimal in terms of object approaching.

TABLE IV: Target prediction accuracies with varying number of aligned objects
and target visibility

| Target visible - 25 maps |  |  | Target not visible - 25 maps |  |
---|---|---|---|---|---|---
| BEL | NoBEL |  |  | BEL | NoBEL |  |
| N Objects
---
Aligned
| Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val |  | Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val
3 | 43.5 | 12.85 | 36.22 | 12.65 | 7.28 | <.001 | 16.61 | 14.02 | 11.56 | 8.38 | 5.06 | <.001
2 | 40.94 | 19.7 | 31.61 | 15.78 | 9.33 | <.001 | 17.56 | 23.91 | 10.83 | 10.5 | 6.72 | <.001
1 | 51.44 | 18.97 | 45.78 | 28.07 | 5.67 | 0.001 | 42.56 | 33.08 | 34.83 | 37.44 | 7.72 | <.001
| Target visible - 120 maps |  |  | Target not visible - 120 maps |  |
| N Objects
---
Aligned
| Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val |  | Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var | Bel-NoBel | p-val
3 | 49.33 | 6.82 | 43.22 | 14.18 | 6.11 | <.001 | 15.39 | 16.02 | 12.11 | 16.34 | 3.28 | 0.02
2 | 46.61 | 3.01 | 36.83 | 4.05 | 9.78 | <.001 | 14.94 | 20.17 | 10.89 | 16.22 | 4.06 | 0.007
1 | 56.67 | 15.88 | 43.83 | 12.97 | 12.83 | <.001 | 46.28 | 33.27 | 35.06 | 31.35 | 11.22 | <.001

Target prediction accuracies for the Beliefs vs NoBeliefs architectures in the
conditions with varying number of aligned objects and target visibility. (A)
Target visible, 25 maps; (B) Target not visible, 25 maps; (C) Target visible,
120 maps; and (D) Target not visible, 120 maps. Accuracies were calculated as
averages across 18 initial network weights; the associated variances were
reported

###  III-E Generalization to different agents

Next we explored if learning to attribute beliefs would help interpret agents
with a different decision system than the observer, and thus interpret
behaviours with a different style and effectiveness compared to those that
were used for training under the self-observation hypothesis. We tested
different ways, e.g. changing the action speed or beliefs representation
precision, to differentiate the agents with consistent results. The results
shown in table V were obtained decreasing the number of samples used by the
POMCP algorithm [57], which resulted in target search times up to 4 times
longer, and show a similar gain from learning to attribute beliefs as the data
on observing similar agents, reported in table I.

TABLE V: Target prediction accuracies with varying observed agent efficiency

250 max samples – Original
---
| Bel | NoBel |  |
| Train Maps
---
(N)
| Avg Acc
---
(%)
Var |  | Avg Acc
---
(%)
Var |  Bel-NoBel (%) | p-val
25 | 69.45 | 1.63 | 67.57 | 1.44 | 1.87 | <.001
120 | 71.47 | 0.87 | 70.46 | 0.89 | 1 | <.001
150 max samples
25 | 68.61 | 2.25 | 67.44 | 2.73 | 1.17 | 0.033
120 | 70.94 | 1.47 | 70.39 | 0.96 | 0.56 | 0.139
50 max samples
25 | 68.22 | 1.59 | 66.78 | 1.59 | 1.44 | 0.002
120 | 70.11 | 1.28 | 69.61 | 0.96 | 0.5 | 0.165
25 max samples
25 | 65.83 | 1.21 | 65 | 1.18 | 0.83 | 0.028
120 | 68.5 | 0.74 | 68.61 | 1.31 | -0.11 | 0.744

Target prediction accuracies for the Beliefs vs NoBeliefs architectures in the
conditions with varying complexity of the actor’s cognitive capabilities.
Number of max samples underlying actor’s behaviour: (A) 250 (Original); (B)
150; (C) 50; (D) 25. Accuracies were calculated as averages across 18 initial
network weights; the associated variances were reported.

##  IV CONCLUSIONS

Beliefs play an important role in driving efficient behaviours when the sensor
system cannot capture instantaneously the whole state of the environment. Such
partial observable conditions are common in unstructured and natural
environments in which artificial agents’ ability to collaborate and support
humans is still limited. Dealing with the impact of partial observability and
belief-driven behaviours would especially improve out-of-the-lab human robot
interactions and ego-vision-based systems [36]. Our results show the
importance of the synergy of learning beliefs and intention attribution
simultaneously, especially when limited samples are available and acquiring
new ones is expensive or risky. We showed that this architectural solution is
particularly helpful in the most complex situations resulting from
interpreting behaviours performed with limited sensory input, such as
searching when multiple elements are in the field of view of the observed
actor. We have also shown that the acquired TOM model, trained by the
observing agent’s own behaviours and mental processes (self-observation), is
able to generalize to different agents while preserving the gain brought by
the beliefs-intentions simultaneous learning. Future work will focus on
extending these results to more complex and realistic settings to further test
the applicability of this approach.

Finally, we believe that our findings on the importance of the learning
synergy between beliefs attribution and intention prediction as well as the
effectiveness of using own decision and belief estimation processes as
learning signals to develop TOM are relevant beyond the development of social
robot architectures [1, 14, 23, 53] and can also inform the developmental
psychology and neuroscience fields [12, 35, 58, 59, 60, 61].

## References

  * [1] S. Rossi, F. Ferland, and A. Tapus, “User profiling and behavioral adaptation for hri: A survey,” _Pattern Recognition Letters_ , vol. 99, pp. 3–12, 2017\.
  * [2] T. Van de Maele, T. Verbelen, O. Çatal, C. De Boom, and B. Dhoedt, “Active vision for robot manipulators using the free energy principle,” _Frontiers in neurorobotics_ , vol. 15, p. 642780, 2021.
  * [3] D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 4247–4258, 2020.
  * [4] P. Ammirato, P. Poirson, E. Park, J. Košecká, and A. C. Berg, “A dataset for developing and benchmarking active vision,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_ , 2017, pp. 1378–1385.
  * [5] S. K. Ramakrishnan, D. Jayaraman, and K. Grauman, “Emergence of exploratory look-around behaviors through active observation completion,” _Science Robotics_ , vol. 4, no. 30, p. eaaw6326, 2019.
  * [6] D. Ognibene and Y. Demiris, “Towards active event recognition.” in _IJCAI_ , 2013, pp. 2495–2501.
  * [7] Y. Demiris* and M. Johnson, “Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning,” _Connection science_ , vol. 15, no. 4, pp. 231–243, 2003.
  * [8] T. Taniguchi, “Symbol emergence in robotics for long-term human-robot collaboration**this research was partially supported by a grant-in-aid for young scientists (b) 2012-2014 (24700233) and a grant-in-aid for young scientists (a) 2015-2019 (15h05319) funded by the ministry of education, culture, sports, science, and technology, japan, and by crest, jst.” _IFAC-PapersOnLine_ , vol. 49, no. 19, pp. 144–149, 2016, 13th IFAC Symposium on Analysis, Design, and Evaluation ofHuman-Machine Systems HMS 2016\. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405896316320663
  * [9] X. Wang, A. Haji Fathaliyan, and V. J. Santos, “Toward shared autonomy control schemes for human-robot systems: action primitive recognition using eye gaze features,” _Frontiers in Neurorobotics_ , vol. 14, p. 567571, 2020.
  * [10] S. Fuchs and A. Belardinelli, “Gaze-based intention estimation for shared autonomy in pick-and-place tasks,” _Frontiers in Neurorobotics_ , vol. 15, p. 647930, 2021.
  * [11] Y. Hu, N. Abe, M. Benallegue, N. Yamanobe, G. Venture, and E. Yoshida, “Toward active physical human–robot interaction: Quantifying the human state during interactions,” _IEEE Transactions on Human-Machine Systems_ , vol. 52, no. 3, pp. 367–378, 2022.
  * [12] B. Sodian and S. Kristen, “Theory of mind,” in _Handbook of epistemic cognition_. Routledge, 2016, pp. 80–97.
  * [13] J. Jara-Ettinger, “Theory of mind as inverse reinforcement learning,” _Current Opinion in Behavioral Sciences_ , vol. 29, pp. 105–110, 2019, artificial Intelligence. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2352154618302055
  * [14] F. Bianco and D. Ognibene, “Functional advantages of an adaptive theory of mind for robotics: a review of current architectures,” in _2019 11th Computer Science and Electronic Engineering (CEEC)_ , 2019, pp. 139–143.
  * [15] M. M. Çelikok, T. Peltola, P. Daee, and S. Kaski, “Interactive ai with a theory of mind,” in _Computational Modeling in Human-Computer Interaction_ , 2019.
  * [16] F. Cuzzolin, A. Morelli, B. Cîrstea, and B. J. Sahakian, “Knowing me, knowing you: theory of mind in ai,” _Psychological Medicine_ , vol. 50, no. 7, p. 1057–1061, 2020.
  * [17] J. Perez-Osorio, E. Wiese, and A. Wykowska, _Theory of Mind and Joint Attention_ , 1st ed. New York, NY, USA: Association for Computing Machinery, 2021, p. 311–348. [Online]. Available: https://doi.org/10.1145/3477322.3477332
  * [18] Y. Zhang, W. Song, Z. Tan, Y. Wang, C. M. Lam, S. P. Hoi, Q. Xiong, J. Chen, and L. Yi, “Theory of robot mind: false belief attribution to social robots in children with and without autism,” _Frontiers in psychology_ , vol. 10, p. 1732, 2019.
  * [19] Q. Wang, K. Saha, E. Gregori, D. Joyner, and A. Goel, “Towards mutual theory of mind in human-ai interaction: How language reflects what students perceive about a virtual teaching assistant,” in _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , ser. CHI ’21. New York, NY, USA: Association for Computing Machinery, 2021. [Online]. Available: https://doi.org/10.1145/3411764.3445645
  * [20] J. Banks, “Theory of mind in social robots: replication of five established human tests,” _International Journal of Social Robotics_ , vol. 12, no. 2, pp. 403–414, 2020.
  * [21] C. Baker, R. Saxe, and J. Tenenbaum, “Bayesian theory of mind: Modeling joint belief-desire attribution,” in _Proceedings of the annual meeting of the cognitive science society_ , vol. 33, no. 33, 2011.
  * [22] S. Devin and R. Alami, “An implemented theory of mind to improve human-robot shared plans execution,” in _2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI)_. IEEE, 2016, pp. 319–326.
  * [23] S. Lemaignan, M. Warnier, E. A. Sisbot, A. Clodic, and R. Alami, “Artificial cognition for social human–robot interaction: An implementation,” _Artificial Intelligence_ , vol. 247, pp. 45–69, 2017, special Issue on AI and Robotics. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0004370216300790
  * [24] S. Vinanzi, M. Patacchiola, A. Chella, and A. Cangelosi, “Would a robot trust you? developmental robotics model of trust and theory of mind,” _Philosophical Transactions of the Royal Society B_ , vol. 374, no. 1771, p. 20180032, 2019.
  * [25] A. F. Winfield, “Experiments in artificial theory of mind: From safety to story-telling,” _Frontiers in Robotics and AI_ , vol. 5, p. 75, 2018.
  * [26] M. Persiani and T. Hellström, “Inference of the intentions of unknown agents in a theory of mind setting,” in _Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. The PAAMS Collection_ , F. Dignum, J. M. Corchado, and F. De La Prieta, Eds. Cham: Springer International Publishing, 2021, pp. 188–200.
  * [27] N. Rabinowitz, F. Perbet, F. Song, C. Zhang, S. A. Eslami, and M. Botvinick, “Machine theory of mind,” in _International conference on machine learning_. PMLR, 2018, pp. 4218–4227.
  * [28] Y. Wen, Y. Yang, R. Luo, J. Wang, and W. Pan, “Probabilistic recursive reasoning for multi-agent reinforcement learning,” in _7th International Conference on Learning Representations, ICLR 2019_ , 2019.
  * [29] R. Chandra, A. Bera, and D. Manocha, “Stylepredict: Machine theory of mind for human driver behavior from trajectories,” _arXiv e-prints_ , pp. arXiv–2011, 2020.
  * [30] I. Oguntola, D. Hughes, and K. Sycara, “Deep interpretable models of theory of mind,” in _2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)_, 2021, pp. 657–664.
  * [31] J. Aru, A. Labash, O. Corcoll, and R. Vicente, “Mind the gap: Challenges of deep learning approaches to theory of mind,” _arXiv preprint arXiv:2203.16540_ , 2022.
  * [32] D. Nguyen, P. Nguyen, H. Le, K. Do, S. Venkatesh, and T. Tran, “Learning theory of mind via dynamic traits attribution,” in _Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems_ , 2022, pp. 954–962.
  * [33] S. Vadalà, C. Esposito, L. Zampini, E. Farina, and E. Datteri, “Robots for the study of false belief attribution in autistic children: an exploratory study,” in _Zecca, L., Cotza, V., Datteri, E., Inclusive Science Education and Robotics. Studies and Experiences_. Milano: Franco Angeli, 2023.
  * [34] R. Shah, N. Gundotra, P. Abbeel, and A. Dragan, “On the feasibility of learning, rather than assuming, human biases for reward inference,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 5670–5679.
  * [35] F. Bianco and D. Ognibene, “Transferring adaptive theory of mind to social robots: Insights from developmental psychology to robotics,” in _International Conference on Social Robotics_. Springer, 2019, pp. 77–87.
  * [36] D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, _et al._ , “The epic-kitchens dataset: Collection, challenges and baselines,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, no. 11, pp. 4125–4141, 2020.
  * [37] A. Langdon, M. Botvinick, H. Nakahara, K. Tanaka, M. Matsumoto, and R. Kanai, “Meta-learning, social cognition and consciousness in brains and machines,” _Neural Networks_ , vol. 145, pp. 80–89, 2022.
  * [38] S. Thrun and L. Pratt, _Learning to learn_. Springer Science & Business Media, 2012.
  * [39] M. Crawshaw, “Multi-task learning with deep neural networks: A survey,” _arXiv preprint arXiv:2009.09796_ , 2020.
  * [40] S. Ruder, “An overview of multi-task learning in deep neural networks,” _arXiv e-prints_ , pp. arXiv–1706, 2017.
  * [41] D. M. Wolpert, K. Doya, and M. Kawato, “A unifying computational framework for motor control and social interaction,” _Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences_ , vol. 358, no. 1431, pp. 593–602, 2003.
  * [42] M. A. Giese and G. Rizzolatti, “Neural and computational mechanisms of action processing: Interaction between visual and motor representations,” _Neuron_ , vol. 88, no. 1, pp. 167–180, 2015.
  * [43] H. Dindo, D. Zambuto, and G. Pezzulo, “Motor simulation via coupled internal models using sequential monte carlo,” in _Twenty-Second International Joint Conference on Artificial Intelligence_ , 2011.
  * [44] J. Zwickel, M. Grosjean, and W. Prinz, “On interference effects in concurrent perception and action,” _Psychological Research PRPF_ , vol. 74, no. 2, pp. 152–171, 2010.
  * [45] T. Chaminade, D. W. Franklin, E. Oztop, and G. Cheng, “Motor interference between humans and humanoid robots: Effect of biological and artificial motion,” in _Proceedings. The 4th International Conference on Development and Learning, 2005_. IEEE, 2005, pp. 96–101.
  * [46] K. Friston, J. Mattout, and J. Kilner, “Action understanding and active inference,” _Biological cybernetics_ , vol. 104, no. 1, pp. 137–160, 2011\.
  * [47] A. N. Meltzoff, “‘like me’: a foundation for social cognition,” _Developmental science_ , vol. 10, no. 1, pp. 126–134, 2007.
  * [48] W. James, “The principles of psychology (vol. 1) new york: Henry holt & co,” _Inc. http://dx. doi. org/10.1037/11059-000_ , 1890.
  * [49] B. Hommel, J. Müsseler, G. Aschersleben, and W. Prinz, “The theory of event coding (tec): A framework for perception and action planning,” _Behavioral and brain sciences_ , vol. 24, no. 5, pp. 849–878, 2001.
  * [50] G. Csibra and G. Gergely, “‘obsessed with goals’: Functions and mechanisms of teleological interpretation of actions in humans,” _Acta psychologica_ , vol. 124, no. 1, pp. 60–78, 2007.
  * [51] C. Heyes, “Where do mirror neurons come from?” _Neuroscience & Biobehavioral Reviews_, vol. 34, no. 4, pp. 575–583, 2010.
  * [52] J. Decety and T. Chaminade, “When the self represents the other: A new cognitive neuroscience view on psychological identification,” _Consciousness and cognition_ , vol. 12, no. 4, pp. 577–596, 2003.
  * [53] Y. Demiris, “Prediction of intent in robotics and multi-agent systems,” _Cognitive processing_ , vol. 8, no. 3, pp. 151–158, 2007.
  * [54] E. Chinellato, D. Ognibene, L. Sartori, and Y. Demiris, “Time to change: deciding when to switch action plans during a social interaction,” in _Conference on Biomimetic and Biohybrid Systems_. Springer, 2013, pp. 47–58.
  * [55] K. Friston, F. Rigoli, D. Ognibene, C. Mathys, T. Fitzgerald, and G. Pezzulo, “Active inference and epistemic value,” _Cognitive neuroscience_ , vol. 6, no. 4, pp. 187–214, 2015.
  * [56] D. Ognibene, L. Mirante, and L. Marchegiani, “Proactive intention recognition for joint human-robot search and rescue missions through monte-carlo planning in pomdp environments,” in _International Conference on Social Robotics_. Springer, 2019, pp. 332–343.
  * [57] D. Silver and J. Veness, “Monte-carlo planning in large pomdps,” _Advances in neural information processing systems_ , vol. 23, 2010.
  * [58] R. Brooks and A. N. Meltzoff, “Connecting the dots from infancy to childhood: A longitudinal study connecting gaze following, language, and explicit theory of mind,” _Journal of experimental child psychology_ , vol. 130, pp. 67–78, 2015.
  * [59] A. Carr, L. Slade, N. Yuill, S. Sullivan, and T. Ruffman, “Minding the children: A longitudinal study of mental state talk, theory of mind, and behavioural adjustment from the age of 3 to 10,” _Social Development_ , vol. 27, no. 4, pp. 826–840, 2018.
  * [60] C. C. Peterson and H. M. Wellman, “Longitudinal theory of mind (tom) development from preschool to adolescence with and without tom delay,” _Child Development_ , vol. 90, no. 6, pp. 1917–1934, 2019.
  * [61] D. Poulin-Dubois, N. Azar, B. Elkaim, and K. Burnside, “Testing the stability of theory of mind: A longitudinal approach,” _Plos one_ , vol. 15, no. 11, p. e0241721, 2020.


Revisiting semiconductor bulk Hamiltonians using quantum computers]Revisiting semiconductor bulk hamiltonians using quantum computers

Raphael César de Souza Pimenta^1 & Anibal Thiago Bezerra^2

$^1$ Departamento de Física, Universidade Federal de Santa Catarina,  Roberto Sampaio Gonzaga s/n, Florianópolis-SC, Brasil

$^2$ Departamento de Física, Universidade Federal de Alfenas, Jovino Fernandes Sales 2600, Alfenas-MG, Brazil

August 2022

With the advent of near-term quantum computers, the simulation of properties of solids using quantum algorithms becomes possible. By an adequate description of the system's Hamiltonian, variational methods enable to fetch the band structure and other fundamental properties as transition probabilities. Here, we use k$\cdot$p Hamiltonians to describe semiconductor structures of the III-V family and obtain their band structures using a state vector solver, a probabilistic simulator, and a real noisy-device simulator. The resulting band structures are in good agreement with the ones obtained by direct diagonalization of the Hamiltonian. Simulation times depend on the optimizer, circuit depth, and simulator used. Finally, with the optimized eigenstates, we convey the inter-band absorption probability, demonstrating the possibility of analyzing the fundamental properties of crystalline systems using quantum computers.

Keywords: Quantum Computing, Semiconductor Band Structure, k$\cdot$p method

§ INTRODUCTION

Materials science has been developed over the last decades with the great help of digital simulations on supercomputers. Such development enabled for several new applications by manipulating the structures, compositions, and atomic arrangement, for a broad range of materials <cit.>. We observe a change of paradigm when the scientific community starts primarily simulating the material's response, predicting the desired behaviors, and subsequently moving to the lab to concretely produce new devices with outstanding properties <cit.>. The screening of materials' and structures' properties is highly bound to the balance between the systems we can simulate and the computational time invested in the modeling <cit.>. However, there are known limits to how far digital simulations can reach. The complexity of the simulation increases exponentially with the number of constituents of the system, and with the quantum correlations between electrons and nuclei <cit.>. Hence, approximations should be employed in the models to ease the complexity. The density functional theory (DFT) is an example of success in the materials science area still at the cost of time-consuming simulations <cit.>. The rhythm to which computational power is increased seems not enough to follow the necessities of simulating new systems with desired properties <cit.>. Therefore, the question is if there are viable alternatives to burden the barriers imposed by the limited computational power we have access to perform the simulations.

Quantum computers appeared as a powerful tool to tackle complicated computational problems <cit.>. By performing computation using quantum states and their superposition, quantum computers are being employed to scale down the complexity of non-polynomial-hard problems, such as the decomposition of big numbers <cit.>, obtaining optimized routes <cit.>, and coloring graphs <cit.>. Quantum computers available today are the so-called Noise Intermediate Scale Quantum computers (NISQ) <cit.>, where the development of algorithms generally involves programming at the gate level, with gates subject to depolarization <cit.>. Although a plethora of algorithms have been proposed to scale down the complexity of computational problems using NISQ devices, for those problems related to solid state physics, the quantum advantage of quantum computers is still a promise <cit.>, with a few exceptions, such as for two-dimensional out-of-equilibrium dynamics <cit.>. To promote the quantum advantage, fault-tolerant gate-based quantum computers are required, emphasizing the importance of understanding and using NISQ devices.

Since quantum computer states could mimic the physical behavior of microscopic systems, their use to obtain the electronic properties of solids seems to be the most natural application of quantum computing <cit.>. The major difficulty is determining how to map the system on a quantum device. Some attempts to a fully ab-initio simulation have been proposed by employing wave-function-based Hamiltonian descriptions, such as coupled clusters with single and double excitations (CCSD) theories. Some advance has been shown with periodic hydrogen chains <cit.> using variational algorithms and unitary coupled clusters (UCSSD) to evaluate both ground states <cit.> and excited states <cit.>. Symmetries have been shown to reduce the number of terms of the Hamiltonian, allowing the use of NISQ computers <cit.>. Still, for hydrogenoid chains, Adaptive Derivative-Assembled Pseudo-Trotter (ADAPT) <cit.>, and other bases as Slater-type orbitals (STO-3G) <cit.> ansatzes have been shown as alternatives to UCSSD description, yet adequate to be used for systems with periodic boundary conditions.

Tight-binding models also play a crucial role in the endeavor of simulating the electronic properties of crystalline solids using NISQ computers. Sherbert and co-workers <cit.> propose an algorithm to evaluate the band structure of a simple cubic lattice structure by implementing the variational quantum eigensolver (VQE), the variational quantum deflation (VQD), and quantum phase estimation (QPE), obtaining both the ground and excited states sampled across a high-symmetry path in the Brillouin zone. The authors demonstrate results from s and p orbitals contribution, using both quantum simulators and real quantum hardware, applying error mitigation. For the quantum device simulator, error mitigation seems enough to help reproduce the expected band structure. However, for the real hardware, the mitigation routine returned worse results. Likewise, Cerasoli and co-workers <cit.> evaluate the silicon band structure using a tight-binding Hamiltonian with VQE and VQD. The authors elucidate the influence of the classical optimization method and emphasize the importance of error analysis and mitigation when dealing with noisy devices. For a more general picture, Bassman and co-workers  <cit.> reviewed the use of quantum computers to evaluate the electronic properties of crystalline solids.

In the present work, we follow a different route from purely ab-initio and tight-binding Hamiltonians. We employ a k$\cdot$p Hamiltonian to represent the crystalline structure of bulk semiconductors from the III-V family. The Hamiltonian is decomposed in terms of Pauli strings, whose expectation values can be directly evaluated using a quantum computer. We then bring a benchmark on the efficiency of the SSVQE algorithm to determine the band structure for a class of crystalline structures using quantum devices. We use the PeenyLane library to implement the hybrid algorithm using a state vector solver, a probabilistic simulator, and a quantum machine simulator. The resulting band structures are compared with the outcome of numerical diagonalizations. Additionally, we investigate the influence of different configurations in the simulation, such as the number of layers of the ansatz, and the optimizers for the classical optimization routine. The results are evaluated by comparing the time and cycles needed for the algorithm to converge. Finally, we use the optimized ansatz to determine the interband transition rate of several semiconductor compounds.

§ METHODS

§.§ The $k\cdot p$ method

Since its development, the $k\cdot p$ method has been widely used to analyze the electronic structure of semiconductors in the vicinity of high-symmetry points of the Brillouin zone <cit.>. The method is grounded on the influence of the crystalline periodic potential profile over the electron in the structure, taking into account the effective mass approximation <cit.>. For diamond-like structures, the electronic dispersion around the high-symmetry $\Gamma$-point ($\vec k = 2\pi(0,0,0)/a$) conveys the relevant knowledge pertaining to the optical and transport properties. Using the translational symmetry of the periodic lattice, together with the Block theorem, one can expand the Hamiltonian of the structure in series keeping only relevant contributions. The Schröedinger equation on the basis of the Bloch states $\ket{q\vec k}$, for wave vectors in the vicinity of the center of the Brillouin zone, reads <cit.>

\begin{equation}\label{eq:KP_Hamiltonian}
\sum_l \left[\left(\varepsilon_{q\Gamma}-\varepsilon_{q\vec k}+\frac{\hbar^2 k^2}{2m_0}\delta_{ql}+ \frac{\hbar \vec k}{2m_0}\cdot \mel{q\Gamma}{\vec p + \frac{\hbar}{4m_0 c^2}(\vec \sigma \times \grad V)}{l\Gamma}\right) \right]D_l(\vec k) = 0,
\end{equation}

where $\varepsilon_{q\Gamma (\vec k)}$ is the energy of the q$^{th}$ band at $\Gamma$ point ($\vec k$ point). The sum runs over the $|l\Gamma\rangle$ Bloch states at $\Gamma$, where $D_l(\vec k)$ is the wave vector-dependent expansion coefficient. The term with $(\vec \sigma \times \grad V)$ accounts for the spin-orbit coupling, where $V$ determines the crystalline periodic potential. For a basis formed by the linear combination of four band edge Bloch functions <cit.>, we can write eq. <ref> as a hermitian matrix <cit.>

\begin{equation}\label{eq:KP_Hamiltonian_matrix}
H\left(\vec k\right) = \begin{bmatrix}
T & -S& i\frac{T-Q}{\sqrt{2}} & -i\sqrt{\frac{2}{3}}P_z\\
& Q & -i\frac{S^\dagger}{\sqrt{2}}& -P_+ \\
&  & \frac{Q+T}{2} + \Delta & -\frac{P_z}{3} \\
&  &  & E_\Gamma
\end{bmatrix},
\end{equation}

\begin{eqnarray}
T = -\frac{\hbar^2}{2m^*}(\gamma_1-\gamma_2)(k_x^2+k_y^2) + (\gamma_1+2\gamma_2)k_z^2\nonumber,\\
S = i\frac{\hbar^2}{2m^*}(2\sqrt{3}\gamma_3  k_z  k_-),\nonumber\\
Q = \frac{\hbar^2}{2m^*}(\gamma_1+\gamma_2)(k_x^2+k_y^2) + (\gamma_1-2\gamma_2)k_z^2,\\
P_z = Pk_z,\nonumber\\
E_\Gamma = \epsilon_\Gamma + \frac{\hbar^2}{2}(k_x^2+k_y^2+k_z^2)\left\{ \frac{1}{m^*} - \frac{E_p}{3}\left[\frac{2}{\epsilon_\Gamma}  + \frac{1}{\epsilon_\Gamma + \Delta}\right] \right\},\nonumber
\end{eqnarray}

where $\gamma_1$, $\gamma_2$, and $\gamma_3$ are empirical Luttinger parameters,  $\varepsilon_\Gamma$ is the band gap at the Brillouin zone center, $P$ is related to the Kane energy $E_p$, $k_\pm=(k_x \pm i k_y)/\sqrt{2}$, $\Delta$ is the spin-orbit coupling of the valence band, and $m^*$ is the effective mass. The diagonalization of eq. <ref> for wave vectors within the Brillouin zone returns the electronic band dispersion. For compounds of III-V family, the $k\cdot p$ method has been shown to return accurate band structures around high symmetry points <cit.>.

§.§ Subspace-Search Variational Quantum Eigensolver

The Variational Quantum Eigensolver (VQE) method has been shown to be a powerful tool to obtain the ground state of Hamiltonians using noisy intermediate-scale quantum (NISQ) computers <cit.>. It is an hybrid quantum-classical algorithm where the expectation value of the Hamiltonian under analysis, $\expval{H(\vec k)},$ is minimized using together a quantum and a classical computer.
For a parameterized ansatz circuit $U(\vec\theta)$ (fig.<ref>), the quantum computer generates the ansatz state $\ket{\phi(\vec\theta)}=U(\vec\theta)\ket{\psi}$ such that the expectation value $\expval{H(\vec k, \vec\theta)}=  \expval{H(\vec k)}{\phi(\vec\theta)}$ is minimized iteratively, feeding $U(\vec\theta)$ back with optimization parameters $\vec\theta$ gathered from the classical machine. The minimized expectation value of the Hamiltonian tends to the ground-state energy of the system.

However, to determine the band structures, not only the ground state is desirable, but also the excited states. To assess both the ground and the excited states of the Hamiltonian of Eq. <ref> utilizing a quantum computer, we employ the Subspace-Search Variational Quantum Eigensolver (SSVQE) <cit.>. The SSVQE method refines the VQE method to obtain excited states without substantially raising computational cost, which is advantageous with NISQ machines.

The motivation behind the SSVQE method is to perform a single optimization procedure, and use the optimized parameters $\vec\theta$ to evaluate the expectation value of the Hamiltonian for a group of initially orthonormal states. First, for $n$ qubits we employ as the ansatz $U(\vec\theta)$ a strongly entangled circuit <cit.>, consisting of N layers of single qubit rotations and entanglers (fig. <ref>). Then, p-copies of the circuit are initialized with p-orthogonal states $\ket{\psi_p}=\ket{\psi}^{\otimes n}$, with p representing the number of excited states to be evaluated. A weighted cost function $C_w=\sum_l^p w_l \expval{U^{\dagger}(\vec\theta) H U(\vec\theta)}{\psi_l}$ is minimized with the weights $w \in [l, l-1,\cdots,1]$. Once the minimization attains to a global minimum, the ansatz becomes unitary mapping the pre-orthonormal states $\ket{\psi_p}$ to the orthonormal eigenstates of the Hamiltonian.

[wires=2]$\ket{\psi}^{\otimes n}$     U(θ_1, θ_2, θ_3) [2,steps=3,style=dashed,rounded corners,fill=green!20, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]
N Layers          1       [2,steps=1,style=dashed,rounded corners,fill=blue!15, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]qwc

U(θ_4, θ_5, θ_6)     -1

(color online) Schematics of the circuit employed to obtain the eigenenergies of the Hamiltonian. The left green box highlights the strongly entangled states used as the ansatz composed of N layers and n qubits. The right blue box highlights the measurement scheme using qubit-wise commutativity (QWC).

To obtain the expectation values used with the cost function, the Hamiltonian of Eq. <ref> is decomposed using the Jordan-Wigner decomposition <cit.> in terms of a linear combination of Pauli matrices, $H=\sum_j h_j \sigma^{\otimes n}$. We apply qubit-wise commutativity (QWC) to reduce the number of measurements <cit.>.

§ RESULTS

§.§ Band Structures

Here we utilized the PennyLane framework <cit.> to create and execute the quantum circuits. Figure <ref>(a) shows the band structure obtained for the Gallium Arsenide (GaAs) compound using the state vector solver (open symbols), the probabilistic simulator (full symbols), and the direct diagonalization (solid lines). The structural parameters used with the Hamiltonian of Eq. <ref> are found in the supplementary table S1. Employing an ansatz with 5 layers (see Fig. <ref>), the band structure is evaluated for a uniformly distributed path across the first Brillouin zone through the high-symmetry points $X-\Gamma-L$, for the split-off band (black curve), the heavy hole band (blue curve), light hole band (orange curve), and conduction band (red curve). For the classical optimization process, we use the Adam optimizer with an step rate 0.01. We stop the optimization process when a difference of 10$^{-7}$ eV among two consecutive steps is reached. The agreement between the results from the SSVQE and the direct diagonalization proved to be excellent.  Figs. <ref>(b-e) show the absolute energy differences between the quantum simulation, using both the state vector solver (open symbols) and the probabilistic simulator (full symbols), with respect to the expected eigenvalues for each band (dashed lines). The state vector solver represents the best scenario for the simulation since it returns the exact solution expected for the quantum system. For it, the maximum energy difference ranges for up to tens of mili-electron volts for the worst case of the light hole band next to the $X$ point (Fig. <ref>(d)).

To acquire a better picture of the expected results of a quantum device, we calculated the GaAs band structure using the probabilistic simulator <cit.> (full symbols in fig. <ref>(a)), that samples the evaluation of the expectation values for a finite number of shots, retrieving the probabilistic nature of the quantum measurement. We also adopt the Adam optimizer for an ansatz with 5 layers. The use of the probabilistic simulator bring difficulties for the optimization procedure. The simulation is unable to converge with the same precision of the state vector solver, since the probabilistic fluctuation inherent to quantum processes is added to the measurement results. Therefore, we lowered the convergence threshold to $1e^{-4}$ eV, and use 10000 shots. Furthermore, the simulation time increases considerably. The mean time of each cycle raised to 3480 s, while the mean number of cycles is 97. As we can observe in fig. <ref>(a), the results from the probabilistic simulator (full symbols) do not perfectly match the ones form the exact diagonalization (dashed lines). The error of the probabilistic simulator in comparison to the state vector solver results  is increased, with a maximum of 400 meV for the light hole band around the X high symmetry point (fig. <ref>(d)).

Once we demonstrate the possibility of obtaining the band structure of bulk crystalline semiconductor structures using a quantum circuit, in what follows, we present a time benchmark for the use of the SSVQE method with k$\cdot$p crystalline bulk Hamiltonians.

(a) GaAs band structure for the lower conduction band state (red squares), and top heavy hole (blue pluses), light hole (orange diamonds) and split-off (black circles) valence band states, evaluated using both the state vector solver (open symbols), the probabilistic simulator (full symbols), and numerical diagonalization (dashed lines). Absolute energy difference between the results from the numerical diagonalization and the state vector solver (open symbols), and from the numerical diagonalization and from the probabilistic simulator (full symbols) for (b) the split-off band, (c) the heavy hole band, (d) the light hole, and (e) the conduction band.

§.§ Execution Times

As previously advocated, the iterative hybrid quantum-classical SSVQE method relies on minimizing the expectation value for a parameterized quantum circuit, carrying a search in the parameter space using classical optimizers. The parameter space size, defined by the ansatz depth, bounds the performance of the minimization process. For each point in the k-path, the execution time is a trade between the time invested running the quantum circuit to recover the expectation value, and the number of optimization cycles necessary to minimize it. Shallow ansatzes require a smaller parameter space, facilitating the optimization procedure. However, owing to the small amount of adjustable parameters, the optimizer might need additional optimization cycles, which raises the number of measurements and also the computation time within the quantum machine. In turn, deeper ansatzes may be more generic, allowing for fewer optimization steps yet demanding costly expectation value evaluations. Choosing the ideal scenario depends upon the available resources. For NISQ computers, the hardware restricts the circuit depth since decoherence times should be greater than the time spent performing the circuit transpiling and execution  <cit.>. Additionally, up to now the access to quantum computers is mainly restricted to cloud services, with job-based queues. After each expectation values for each cycle of the optimization, the job is sent back to the queue. Therefore, minimizing both the number of measurements in the expansion of the Hamiltonian, and the number of cycles, while keeping the ansatz shallow, is mandatory.

(a) In orange, mean time in seconds for convergence of each point in the k-path for 2-7 layers in the quantum circuit, and in blue the mean time to complete each cycle of convergence attempt. (b) In the bars we have the mean number of cycles to complete the convergence, for all the k-path for 2-7 layers in the quantum circuit, the black bars represent the maximum and minimum number of cycle for a point.

Figure <ref> shows the mean times necessary to obtain the band structure of the GaAs bulk Hamiltonian, in terms of the number of layers of the ansatz and using the state vector solver. The mean is done taking into account all points in the k-path (see Fig. <ref>).  Overall, the mean execution time remains around 150 s for ansatz with up to 5 layers, escalating significantly to 262 s for 7 layers (Fig. <ref>(a)). As expected, the mean time per cycle rises monotonically with the number of layers, since the number of gates employed in the quantum circuit also increases. Figure <ref>(b) shows the mean number of cycles with respect to the number of layers. The horizontal lines in each bar represent the minimum and maximum number of cycles. For 2 layers, a shallow ansatz demands the greater mean number of cycles (420), with a maximum of 818 cycles for one specific k-point in the path. For 5 layers, we see the lower mean number of cycles (207) with a maximum of 285 cycles and a minimum of 145 cycles. The number of cycles, maximums and minimums rise back for 6 and 7 layers. As we can observe, the strongly entangled ansatz with 5 layers reveals to be the more appropriate for simulating 4$\times$4 k$\cdot$p Hamiltonians.

The choice of the optimizer is also fundamental, since its performance determines the number of circuit evaluations. Supplementary figure S1 shows the mean execution times for the optimizers: Adam, Adagrad, Conjugated Gradient, and Nesterov. The optimization is done using 5 layers and 0.01 for the step rate, with a tolerance of 10$^{-7}$ eV. The mean time per cycle remains constant since it depends on expectation value evaluation using the quantum circuit. However, the Adam optimizer shows the lowest mean time (137 s), while the Conjugated Gradient shows the bigger one (1134 s) (Fig S1(a)). The short mean time obtained by the Adam optimizer is related to its efficiency in finding the total minimum with the small number of steps (248 on average). Since the band structure is evaluated from the eigenvalues of Hamiltonians that differ only from each other by the neighbor k-points, it is worth mentioning that we used the optimized parameters from the previous k-point as a starting point for optimizing the next k-point.

Band structure for different materials evaluated using both the quantum circuit (dots) and numerical diagonalization (solid lines). On (a) we have the result for InP structure, (b) for InAs, (c) InSb, (d) AIP, (e) GaP and (f) for GaSb. All the structures were evaluated using the Penny Lane State Vector.

After validating the application of the SSVQE method to obtain the band structure of the GaAs structure, we applied the Adam optimizer and the strongly entangled ansatz with 5 layers to evaluate the electronic structure of other compounds of the III-V family. Figure <ref> shows the band structure for the Indium Phosphide (InP, fig. <ref>(a)), the Indium Arsenide (InAs, fig. <ref>(b)), the Indium Antimonide (InSb, fig. <ref>(c)), the Aluminium Phosphide (AlP, fig. <ref>(d)), the Galium Phosphide (GaP, fig. <ref>(e)), and the Galium Antimonide (GaSb, fig. <ref>(f)). For all graphs in fig. <ref> the dashed lines are the results from the direct diagonalization of the Hamiltonian employing the specific parameters for each compound, and the symbols are the results for the quantum circuit simulation. Once, the agreement between the expected and the simulated values is excellent. Supplementary figure S2, shows the energy differences. As for the GaAs structure, the error is small, with a maximum value of 4 meV for the heavy hole band of the InP compound (see supplementary figure S2(a).

§.§ Quantum Device

Besides the ability of obtaining the band structures using both the state vector and the probabilistic simulators, we also used a real-device-based noise model to simulate the expected response of a real quantum computer. Namely, we exploit the Qiskit Aer noise module to generate a noise model as an approximation of the real errors in the execution of the ibmq-lima quantum device <cit.>. The ibmq-lima is an IBM's quantum processor of Falcon type, with 5 qubits, and  94.03$\mu$s (101.39$\mu$s) of average T1 (T2) error.
Supplementary figure S3 shows the results simulated using the ibmq-lima noise model. As in the case of the probabilistic simulator, the convergence of SSVQE method is hampered by the fluctuations of the quantum measurement. The convergence threshold is lowered to $10^{-3}$ eV with 10000 shots, but the model marginally attaches to a global minima. The energy differences between the device and the exact diagonalization are mainly in the range of tens of electron volts. Moreover, the worst scenario is for the electron band (the more energetic), with an error of 1.8 eV for the high-symmetry point $L$ (see supplementary figure S2(e)). Each cycle of the simulation using the noise model is time consuming. The execution time increased considerably in comparison to the probabilistic simulator, taking around 12h to simulate each point within the k-path.

§.§ Absorption

Simulated interband absorption of semiconductor compounds (a) GaAs, (b) InP, (c) InSb, (d) InAs, (e) GaSb, (f) AlP, and (g) GaP.

NISQ computers, aided by variational methods as the SSVQE, are presumed to generate ground and excited state wave functions not classically feasible <cit.>. With them, transition amplitudes, related to experimental properties as emission and absorption, can be readily evaluated. Noting that $\expval{\hat T_{vc}} = \mel{\psi_v}{\hat T}{\psi_c} = \mel{\phi_v}{U^{\dagger}(\vec\theta)\hat T U(\vec\theta)}{\phi_c}$ determines the expected value for the transition operator $\hat T$ between valence $\ket{\psi_v}$ and conduction $\ket{\psi_c}$ states, where $U(\vec\theta)$ represents the optimized ansatz, the evaluation of the transition amplitude can be written as <cit.>

\begin{equation}
\begin{split}
\Re{\mel{\phi_v}{\Tilde{T}(\vec\theta) }{\phi_c}}&=\\
&\mel{\phi_{+,vc}^{x}}{\Tilde{T}(\vec\theta)}{\phi_{+,vc}^{x}}-\frac{1}{4}\mel{\phi_v}{\Tilde{T}(\vec\theta) }{\phi_v}\mel{\phi_c}{\Tilde{T}(\vec\theta) }{\phi_c},
\end{split}
\end{equation}

\begin{equation}
\begin{split}
\Im{\mel{\phi_v}{\Tilde{T}(\vec\theta) }{\phi_c} }&=\\
&\mel{\phi_{+,vc}^{y}}{\Tilde{T}(\vec\theta)}{\phi_{+,vc}^{y}}-\frac{1}{2}\mel{\phi_v}{\Tilde{T}(\vec\theta) }{\phi_v}-\frac{1}{2}\mel{\phi_c}{\Tilde{T}(\vec\theta) }{\phi_c},
\end{split}
\end{equation}

where $\Tilde{T}(\vec\theta)=U^{\dagger}(\vec\theta)\hat T U(\vec\theta)$, $\sqrt 2\ket{\phi_{+,vc}^{x}} = \ket{\phi_v}+\ket{\phi_c}$, and $\sqrt 2\ket{\phi_{+,vc}^{y}} = \ket{\phi_v}+i\ket{\phi_c}$. Note that obtaining the transition rate involves few additional evaluations of the quantum circuit for the optimized set of parameters $\vec\theta$. To evaluate the absorption, in the case of interband transitions, the transition amplitude (or the oscillator strength) is given by the overlap between the initial and final states. Therefore, we can assume the transition operator $\hat T$ as the identity operator.

The absorption reads <cit.>

\begin{equation}
\alpha(\hbar\omega) \propto \frac{1}{\hbar\omega}\sum_{v,\vec k} \abs{\mel{\phi_v(\vec k)}{\Tilde{T}(\theta) }{\phi_c(\vec k)}}^2\Theta(\varepsilon_{vc}(\vec k)-\hbar\omega),
\end{equation}

where the sum runs over the valence band states, and over the $\vec k$ point within the path. $\Theta(\varepsilon_{vc}(\vec k)-\hbar\omega)$ is a step function with $\varepsilon_{vc}(\vec k) = \varepsilon_{c} - \varepsilon_{v}$ being the energy separation between the valence and the conduction band states at $\vec k$ point. Figures <ref>(a)-(g) show the interband absorption between the valence and conduction bands for several semiconductor compounds, simulated using the wave functions from the quantum simulation of the band structures. Even considering the contribution of a small amount of bands, the results are in good agreement with the expected absorption spectrum for semiconductor compounds of the III-V family. Simulating the absorption using a quantum computer, for an optimized ansatz, is a considerably cheap operation in terms of system requirements. For each point in k-path, to evaluate the absorption is necessary only 10 calls of the quantum computer, barely the same number of calls used in one cycle of the optimization process.

§ DISCUSSION

We show SSVQE method as a viable, yet costly, tool for obtaining the band structure of semiconductor bulk compounds using a quantum computer. Despite it being an algorithm specially designed to operate on NISQ computers, we need to be careful when dealing with band structures of crystalline materials. As we can observe, using the k$\cdot$p method, we are able to deal with small Hamiltonians, which is advantageous with the limited computational power of NISQ processors <cit.>, still capturing relevant information about electronic movement within the material around high symmetry points. Notwithstanding, even for such small Hamiltonians, the simulation times are not competitive with traditional diagonalization methods.

The increased simulation time is related to both the number of cycles in the optimization process, and the number of terms of the Hamiltonian decomposed as a combination of Pauli strings <cit.>. For each step of the optimization process and for each term in the decomposition, an evaluation of the quantum circuit is required. To reduce the number of cycles we should play with the optimization process itself. We have to choose the appropriated optimizer, the appropriated precision for the convergence, and, primarily, the appropriated ansatz circuit. The number of qubits raises with the complexity of the Hamiltonian, increasing considerably the number of decomposition terms. To reduce the number of terms, quantum-wise commuting (QWC) scheme can be used. It employs family commutativity to group the terms required to obtain the expectation values  <cit.>. The QWC, on average, reduces a large number of terms by a factor of three in comparison to the total number of Pauli strings. Since we are dealing with only two qubits, here we reduced the number of terms from 16 to 9. More efficient methods can be applied, such as the general commutativity (GC), which tends to reduce the number of terms by a factor of ten for large Hamiltonians <cit.>. However, such methods use graphs and coloring theory which are themselves NP-hard problems,  potentially increasing the time in the classical step of the simulation <cit.>. Fortunately, quantum computers can also aid with a significantly speedup on NP-hard problems using, for example, Grover's algorithm <cit.>.

When moving from the state vector solver to a probabilistic simulator or to a device simulator, the difficulties increase considerably. The parameters' optimization landscapes for VQE-based methods are non-convex. Therefore, reaching a minimum requires at least hundreds of steps. For the calculation of excited state with the SSVQE method, the weighted cost function further increases the optimization challenge <cit.>, in special with noisy devices. Since the cost relies on the summation of noisy expectation values <cit.>, the variance in the total energy gains a stochastic component hampering the optimization routine. An additional problem is related to the evaluation of gradients of the cost function that tends to vanish, prejudicing the action of gradient based optimizers. Such phenomenon is referred to as barren plateaus <cit.>. A possible detour would be to employ gradient-free methods for the optimization. Nevertheless, even gradient-free methods have been shown to present barren plateaus <cit.>.

As we can observe from our results, we are able to accurately obtain the band structure of semiconductor bulk compounds using the SSVQE method. However, this is not an easy task to be implemented on a real NISQ device, at least in a competitive fashion. There are several aspects to consider, emphasizing the importance of the design of structure- and method-dependent approaches <cit.>. The evaluation of excited states imposes additional difficulties with more complex cost functions. Fortunately, incorporating symmetries of the Hamiltonian in the cost function proved beneficial <cit.>. Nevertheless, barren plateaus represent a challenging to be surpassed. They have been show to be a problem also with hardware efficient ansatz <cit.>. Beyond the plateaus originated by the non-convex parameters' landscape, noise-induced plateaus have been shown to appear with NISQ devices <cit.>.
The optimization process itself deserves a special care. There are some proposals of more robust techniques to be employed with NISQ devices. The Quantum Natural Gradient (QNG) for example, tends to reduce the number of optimization steps in comparison to gradient-based methods by computing the Fubiny-Study metric tensor <cit.>. However, each step of the optimization requires more calls from the quantum computer, which can be a problem for NISQ computers. Alternatively, in the Quantum Analytic Descent approach <cit.> the optimization landscape can be approximated in the vicinity of reference points, allowing for the cheaper evaluation of the gradients using the classical computer. The quantum device is exclusively employed to evaluate the cost function at the reference point and for some neighbor points, being the Quantum Analytic Descent method closer to a classical than to a pure quantum algorithm. Just as importantly, the stochastic nature of the measurement is further explored in the Doubly Stochastic Gradient Decent method <cit.>. As a consequence, fewer measurements and circuits executions are required yet ensuring convergence for restricted settings. However, the effect of noise is not included in the Doubly Stochastic Gradient Decent method, which can be an impediment with NISQ computers.

Besides the difficulties, the NISQ computers' architecture can be exploited in favor of the band structures of bulk crystalline systems, specially when utilizing k$\cdot$p-based methods. NISQ computers are designed with a reduced quantity of qubits forming a not entirely connected grid. As mentioned earlier, k$\cdot$p Hamiltonians are in general small, still preserving important aspects of the electronic structure of crystalline materials. Therefore, a reduced number of qubits is sufficient to simulate the Hamiltonians. For instance, a 4$\times$4 Hamiltonian demands only 2 qubits. To acquire quantitatively correctly description of valence and conduction bands, slightly bigger Hamiltonians are needed. For example, a precise description of valence bands and two lowest conduction bands of direct and indirect gap bulk semiconductors can be obtained with 24$\times$24 basis <cit.>. Nonetheless, a  40$\times$40 band k$\cdot$p is enough to accurately reproduce the overall band structure of strained and unstrained semiconductor alloys <cit.>. Even a big 40$\times$40 k$\cdot$p Hamiltonian, from the quantum computer perspective, requires solely 6 qubits.

Moreover, since one challenge with hybrid algorithms is to reduce the number of calls of the quantum device, a crude parallelization approach can be used. Considering the decomposition of the Hamiltonian as a combination of non-commuting Pauli strings, groups of measurements can be done simultaneously using untangled groups of qubits of the same chip, taking advantage of single calls of the quantum device. Here, the absence of connections among the qubits is not a complication, provided that the measurements should be done individually for non-commuting observables.

§ CONCLUSION

In summary, we presented the use of NISQ computers to obtain the band structures of crystalline semiconductors of the III-V family. The structures are described in terms of k$\cdot$p Hamiltonians, whose expectation values are minimized using the SSVQE method. The execution times are closely dependent on the optimization parameters, the depth of the ansatz, the number of terms of the Hamiltonian decomposition, and the quantum simulator employed. Using the state-vector solver, we observe an excellent agreement between the simulated band structures and the expected ones. For the probabilistic simulator and the simulator of a real NISQ computer, the minimization is intricate, considerably raising the overall simulation time and lowering the precision, still returning good results. The use of k$\cdot$p Hamiltonians is advantageous with NISQ devices. It allows for a precise description of the electronic properties of semiconductor structures using a small number of qubits. Furthermore, with the optimized ansatzes, we obtain the interband absorption spectra of the structures. It requires only a few additional runs of the quantum device, demonstrating the possibility of using quantum computers to readily obtain the properties of crystalline systems. Using k$\cdot$p Hamiltonians opens up the possibility of using NISQ computers to study the influence of strain, stress, and confinement in the semiconductor structures <cit.>.

§ CODE AVAILABILITY

The code is available under request to the corresponding author.

§ ACKNOWLEDGMENTS

We acknowledge the use of IBM Quantum services for this work. The views expressed are those of the authors, and do not reflect the official policy or position of IBM or the IBM Quantum team. Our special thanks to the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and to the Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina (FAPESC) for providing resources to this research.