# CASPER: Cognitive Architecture for Social Perception and Engagement in
Robots

Samuele Vinanzi* and Angelo Cangelosi  Cognitive Robotics Lab, The University
of Manchester, Manchester, United Kingdom

*Correspondence<EMAIL_ADDRESS>
###### Abstract

Our world is being increasingly pervaded by intelligent robots with varying
degrees of autonomy. To seamlessly integrate themselves in our society, these
machines should possess the ability to navigate the complexities of our daily
routines even in the absence of a human’s direct input. In other words, we
want these robots to understand the intentions of their partners with the
purpose of predicting the best way to help them. In this paper, we present
CASPER (Cognitive Architecture for Social Perception and Engagement in
Robots): a symbolic cognitive architecture that uses qualitative spatial
reasoning to anticipate the pursued goal of another agent and to calculate the
best collaborative behavior. This is performed through an ensemble of parallel
processes that model a low-level action recognition and a high-level goal
understanding, both of which are formally verified. We have tested this
architecture in a simulated kitchen environment and the results we have
collected show that the robot is able to both recognize an ongoing goal and to
properly collaborate towards its achievement. This demonstrates a new use of
Qualitative Spatial Relations applied to the problem of intention reading in
the domain of human-robot interaction.

###### Index Terms:

cognitive human-robot interaction, cognitive architectures, cooperating
robots, social human-robot interaction, intention reading, artificial
intelligence

##  I Introduction

Autonomous robots are increasingly present in our everyday life. Once limited
to research laboratories and industrial settings, they now frequently inhabit
our living spaces and interact with us during our day. This new generation of
intelligent machines, categorized under the umbrella term of “social
robotics”, is expected to navigate a complex and uncertain landscape made of
human beliefs, desires, intentions and social norms. Additionally, it is
desirable for these agents to be able to act autonomously, that means without
direct input from their human partners.

Figure 1: The simulated robot equipped with CASPER observes the actions of
another agent (in this case, a simulated human) in order to predict their goal
and the best collaborative plan.

In order to integrate these robots into our society, it is paramount for them
to be endowed with the same set of cognitive and mental skills that regulate
the way in which we, as people, interact with other agents. One of the most
fundamental of such skills is known as “intention reading” and represents the
capacity to understand the implicit goal that is driving the actions of
another agent [1]. By making appropriate use of this ability, we can allow a
social robot to use its observations of another agent to extrapolate their
underlying goal, reconstruct their expected future actions and, finally,
determine how and when to enact collaborative behavior.

This paper presents CASPER (Cognitive Architecture for Social Perception and
Engagement in Robots), a platform-independent cognitive architecture that uses
a mixture of symbolic and data-driven artificial intelligence methodologies to
perform intention reading and collaboration in a human-robot interaction (HRI)
scenario. “Social perception” refers to the act of identifying and using
social cues to make judgments about others, while with the term “engagement”
we highlight the system’s ability of translating this knowledge into practical
involvement and interaction.

This system lays its foundations on the use of Qualitative Spatial Relations
(QSRs) that describe how the observed partner moves inside the environment
with respect to the Objects of Interest (OOIs). CASPER analyzes the temporal
evolution of these descriptors to predict future actions, which are
subsequently validated by a knowledge-base system and processed by a
probabilistic plan recognizer. This data-driven approach to recognition is
complemented by a top-down decision-making strategy to generate assistive
activity.

Our contributions to the field include the following:

  * •

A new cognitive architecture that implements intention reading and
collaborative behavior capabilities for human-robot interaction, the need of
which arises from the scarcity of such models in the current literature [2].

  * •

To the best of our knowledge, CASPER represents one of the first attempts to
introduce the use of QSR descriptors to perform efficient and easily
generalizable intention reading for embodied robots. This paper presents a
proof of concept on the untapped potential of these mathematical tools and
tries to promote their use in future cognitive architectures for action and
goal recognition.

  * •

On a technical level, we present a collection of novel algorithms for social
perception and reasoning that take inspiration from well-established
psychology and cognitive science principles.

  * •

We provide the first case study for this cognitive architecture: a
demonstration of its possible implementation to solve a collaborative task
based in a kitchen environment.

Our long-term goal is to implement CASPER into an heterogeneous multi-agent
teaming scenario, where a team of distributed agents, both humans and robots
of different kind, are engaged in joint action to achieve a common goal. In
order to do so, this paper builds the foundations of this cognitive
architecture and presents a first case study involving a dyadic interaction
that takes place inside a kitchen. Within this specific application of CASPER,
the robot is expected to observe its partner, identify the goal that is
driving their actions and calculate the best way to assist with their task.
Our experiments carried out in simulation (Figure 1) show that the robot using
this cognitive architecture is able not only to accurately predict the
partner’s goals before they are achieved, but also to formulate appropriate
collaborative decision-making. This allowed the human to silently and
implicitly delegate part of the task to their artificial companion.

The organization of the paper is as follows: Section II offers a general
background on artificial cognitive architectures and intention reading, other
than introducing the notion of QSRs. Section III discusses CASPER’s design
specifications and algorithmic details. Section IV covers the implementation
of the general CASPER architecture to the specific example of the simulated
kitchen environment. Section V discusses the performance of the system on the
selected case study. Finally, Section VI concludes and highlights possible
future directions.

##  II Previous Work

###  II-A Cognitive Architectures for Robots

Cognitive Robotics is a discipline that lies at the intersection of robotics
and cognitive science, which is the scientific study of the mind and its
processes such as perception, attention, anticipation, planning, memory,
learning, and reasoning. It has been defined as “the field that combines
insights and methods from artificial intelligence, as well as cognitive and
biological sciences, to robotics” [3]. This definition highlights the
interdisciplinary nature of this approach, which takes inputs from
linguistics, psychology, neuroscience, philosophy, computer science and
anthropology. Its aim is to create intelligent robots which are endowed with
the same set of mental skills as a human being.

Cognitive science views the mind as an information processor and studies the
operations through which perceptual stimulus are combined to obtain higher-
level mental functions [4]. These principles are easily transferable to an
embodied robotic platform that can implement the same functions despite the
difference in the underlying structure (a brain versus a computer). This is
done by designing what is known as an “artificial cognitive architecture”: a
computational system which instantiates one or more cognitive theories using
artificial intelligence methodologies in an attempt to model the human mind.

A recent review has estimated the existence of around 300 cognitive
architectures in the current literature [2]. The vast majority of them
specialize on modeling particular aspects of cognition such as attention [5],
emotion [6] or problem solving [7], while only a fraction aims to achieve
Artificial General Intelligence. The latter case includes some of the most
famous architectures in the current literature, such as ACT-R [8], Soar [9],
LIDA [10] and NARS [11]. These are all implemented as general frameworks that
can be deployed to specific use cases, including applications in robotics
[12]. For example, ACT-R is written as a Common Lisp interpreter and its
applications come in the form of scripts in the ACT-R language. Despite their
purpose, every cognitive architectures usually models one or more aspects of
cognition such as perception, attention, action selection, memory, learning
and reasoning [2].

CASPER belongs to the category of more specialized architectures and focuses
on modeling specifically human-robot collaboration (HRC) mental capabilities
and, in particular, intention reading. Other cognitive systems that belong to
the same class make use of different techniques such as unsupervised
clustering of human postures [1] and artificial mirror neuron networks [13].

###  II-B Human-Robot Collaboration

The human species’ success is ascribable to its ability to collaborate with
others to obtain otherwise inaccessible goals. This instinct towards
cooperation is shared by our close relatives in the animal kingdom, which
however use it as a means to achieve a purpose, rather than being
intrinsically motivated in pursuing it [14]. Given the importance that
collaboration has for humans, it seems natural to try and transpose this skill
to the autonomous intelligent machines that we are designing as everyday
companions.

HRC is a branch of HRI which studies the best ways to ensure a safe and
effective interaction between humans and robots engaged in joint tasks with
common goals: we call this a “team”. The vast majority of works in this
field’s literature focus on industrial settings, where robotic arms such as
Sawyer or Kuka robots offer assistance in some kind of assembly task [15, 16,
17]. Many of these studies deal with the scheduling and subdivision of tasks
between the two agents. Hoffman et al. [18] argue that collaborative robots
(or “cobots”) should possess communication mechanisms in order to both
understand humans and to inform them about their own goals, and in so doing
maintain a set of shared beliefs which support the execution of a joint plan.
In fact, many HRC models focus on direct verbal cooperation and implement
dialogue managers [19, 20, 21]. More recently, a number of studies have
adopted Machine Learning methodologies (sometimes encased in artificial
cognitive architectures) to learn and modulate the robot’s response to human
tasks [22].

###  II-C Intention Reading

On a psychological point of view, one of the main cognitive skills that enable
social and collaborative attitudes in humans is known as “intention reading”
[23]: this is the ability to understand the driving goal of another agent by
the observation of their physical clues. This is possible because we don’t
understand the behavior of others as a series of unrelated motions through
space, but rather as sets of goal-directed actions [24]. Intention reading is
a fundamental skill to implement in a collaborative intelligence because an
agent has to first understand what their partner’s goal is before knowing how
to offer its assistance.

Balwdin et al. [25] state that humans process continuous actions as streams of
hierarchical relations that link low-level intentions (such as grasping a
plate, bringing it to the sink and opening the tap) to high-level intentions
(to wash the dishes or clean the kitchen). Additionally, they state that
adults more reliably identify the higer-level goals based on some actions that
are understood to be more crucial than others (for example, they point to the
fact that scrubbing a plate is a stronger signal for the intention to wash the
dishes than the equally necessary but less central action of turning on the
water). Some of the most important social cues that are used for this purpose
are biological motion and gaze direction [26].

On a computational perspective, there have been many attempts to develop
intention reading models for HRI. A notorious experiment by Dominey and
Warneken [27] investigates the shared intentionality in a turn-taking game
between a human and a robotic arm, where the artificial agent would build a
representation of the shared plan and subdivide the actions between itself and
its partner. Duarte et al. [28] perform action anticipation exploiting several
social cues such as saccadic eye movement, gaze directing and arm movements
processed through a Gaussian Mixture Model. Bien et al. [29] developed a
system that analyzes posture and movements in elderly people and tries to
decode the inner intentions they are driven by. Other relevant researches
involve the use of dynamic Bayesian networks [30], self-organizing maps [31],
first order logic [32] or the imitation of the biological mirror neuron system
[33]. An interesting approach has been adopted by Granada et al. [34], who
divide the intention reading task on a low-level action recognition paired
with a high-level goal understanding and execute the task combining a
convolutional neural network with a symbolic plan recognizer.

Previous works by the authors [35, 36, 1] have explored the use of social cues
and psychological theories to develop intention reading capabilities in
humanoid robots. In these works, we analyzed kinematic signals (body posture,
head gaze) to infer the human’s current goal using clustering algorithms
paired with probabilistic modeling. Our interest is shifting from dyadic
interactions to heterogeneous multi-agent environments in which the Team Goal
is shared among sub-teams distributed along a structured world. For this
reason, we felt the need to adopt a higher-level approach and shift our focus
from subtle kinematic signals to qualitative relations between the entities in
the environment. This work lays the foundations of CASPER, which will be in
the future applied to a complex scenario like the one we have just described.

###  II-D Qualitative Spatial Relations

Most mathematical and engineering modeling of phenomena rely on _quantitative_
representations: measurements made on standard units that define physical
properties. This is not how human commonsense generally works: in order to
process information in a timely and efficient manner, we tend to discard
unnecessary details, reducing the concept of space into coarse categories. For
example, in our everyday life we would say that something is “far away”
without specifying the exact number of meters. This is an example of
_qualitative_ spatial reasoning.

QSRs are tools that allow commonsense reasoning about space and time using
qualitative relations for different spatial aspects such as topology,
direction and position [37]. In other words, QSRs enable an agent to reason
about actions such as “the human is approaching the fridge” without having to
maintain precise metric information on the positions of both the actor and the
reference point (in this example, the human and the fridge). Such a
representation does not depend on factors such as the starting positions of
the actor and reference, the speed of movement or its exact trajectory and so
it is easily generalizable.

QSRs have been used for a broad range of artificial intelligence applications,
such as human activity learning [38] and monitoring [39], language learning
[40], imitation learning [41] and even to encode spatial structure features
for artificial neural networks [42].

The use of QSRs for intention reading purposes is, to the best of our
knowledge, novel. Many papers that make use of them focus on action
recognition, which is an important but non exhaustive step in the process of
understanding the underlying goal that is driving an agent’s behavior. One of
the main scientific contributions of this paper is to demonstrate that QSRs
can be effectively used in this domain, since they are more prone to
generalization than the features usually employed by intention reading models
(such as motion trajectories or posture keypoints).

##  III Proposed Method

Figure 2: Overview of the proposed system. CASPER is composed of several
parallel processes that interact with each other in a joint effort to decode
an agent’s intention and to formulate an appropriate response. The Perception
module transforms visual observations into QSRs. The latter are used by the
Low-Level process to predict the actions that are being performed in the
environment and passes this information to the High-Level component, which
tries to match them against the plan library to infer the pursued goal. A
knowledge base, enveloped in the Verification module, ensures the step-by-step
soundness of these predictions. Finally, the Supervisor coordinates all the
other processes, collects the results and composes a collaborative plan that
will be executed by the robot.

The purpose of our work is the development of an artificial cognitive
architecture that will allow an autonomous social robot to observe the actions
of a partner (be it a human or a humanoid), understand their underlying goal
and collaborate on the ongoing task. Our design choices are inspired by the
low- and high-level subdivision of actions and goals theorized by
psychologists and use biologically plausible inputs [26]. We allow this model
to generate an appropriate assistive response by leveraging on the prediction
of the current goal, the observed actions and the incomplete part of the plan.
The overall system is depicted in Figure 2. The architecture is composed of
several parallel processes which gather, elaborate and share data between
them.

###  III-A Goal and Intention Representation

Figure 3: A plan in CASPER. Each goal is formed by a temporally ordered set of
sub-elements with varying levels of abstraction. This structure is used for
both intention reading and collaborative behavior generation.

Before giving a detailed description of each component showed in Figure 2, it
is worth explaining how this cognitive architecture represents intentions.
Within CASPER, every goal is described by a Plan: a sequence of events that
have to be executed in order for the task to be accomplished. Figure 3
describes this structure as a non-binary tree. The main _goal_ is formed by a
collection of _sub-goals_ , each of which is achievable by a sequence of
_actions_. The latter, in turn, are composed by a set of _movements_. The
nodes of the Plan are temporally ordered from left to right.

The nodes represented as ellipses in Figure 3 are the abstract, conceptual
representations of the plan and fall under the domain of the High-Level module
(Section III-E), while the ones drawn as rectangles are the ones that can be
directly observed by the Low-Level module (Section III-D). The hierarchical
organization of goals is inspired by cognitive science [4].

This data structure is used both when reading the partner’s intention (bottom-
up, data-driven inference) and when generating an appropriate collaborative
behavior for the robot (top-down, conceptually-driven inference).

###  III-B Environment and Robot Control

This architecture assumes the existence of a robot $r$ which is observing
another agent $a$ performing actions within an environment $E$. The latter
will contain a set $O=\\{o_{1},o_{2},...,o_{n}\\}$ of OOIs which can be
interacted with. For example, possible OOIs in a kitchen would be pieces of
cutlery and food items, but not structural elements such as walls and floors.

We have designed this cognitive architecture to be platform-independent. This
means that any robot with basic vision capabilities can be programmed to
interact with CASPER. Of course, the physical limitations of the chosen robot
define how it will be able to interact with the world and assist its partner,
but this information can be easily encapsulated in the knowledge base (see
Section III-F).

###  III-C Perception

The Perception module interacts directly with the Robot Control system and is
used to produce a qualitative description of the relations between the
observed agent and the OOIs in the environment. The agent will periodically
update this component by transmitting a World State: a dictionary that, at
every timestep, records the absolute coordinates of each OOIs calculated from
the robot’s visual sensors (such as RGB or RGBD cameras). This World State is
then processed by the QSR Engine to obtain the qualitative spatial descriptors
of the scene at the current timestep. For our purposes, we chose to use QSRlib
[43], an open-source software library designed to calculate QSRs from a scene
description.

The QSRs we calculate from our World State are the following:

  * •

Qualitative Distance Calculus (QDC) [44].

  * •

Qualitative Trajectory Calculus (QTC) [45].

  * •

Moving or Stationary (MOS).

  * •

Holding Object (HOLD).

QDC: defines the qualitative Euclidean distance between two entities in the
scene, which in our case are $a$ and $o_{i}\in O$. On an intuitive level, this
QSR describes how close the agent is to the OOIs under consideration. The
thresholds are parameterized within QSRlib and for our purposes we have
defined the following: ‘touch’ [0-0.6m], ‘near’ (0.6-2m], ‘medium’ (2-3m],
‘far’ (3-5m] and ‘ignore’ for distances greater that 5m.

QTC: represents the relative motion between a set of moving point objects
having a free trajectory in an $n$-dimensional space. The current literature
contains several variations of this descriptor [46], out of which we chose
QTCB11. The latter involves two points (which for our purposes will be $a$ and
$o_{i}$) and makes use of the Euclidean distance calculated on the reference
line that connects them. Because of the nature of this QSR, it can only be
calculated over two distinct timesteps. The results of this calculation can be
either: $a$ is stationary with respect to $o_{i}$ (represented by the symbol
$0$), $a$ is moving towards $o_{i}$ ($-$) or $a$ is moving away from $o_{i}$
($+$).

MOS: a unary QSR that describes whether the entity is in motion or stationary
between two different timesteps.

HOLD: this unary descriptor indicates whether the agent $a$ is holding an
object in one of their hands or not.

After having produced a QSR description of the current timestep, the
Perception module stores it in a local library, which acts as the system’s
sensory memory. The latter contains all the time-ordered QSRs calculated since
the beginning of the activity and it can be accessed on request by other
processes.

###  III-D Low-Level Action Recognition

This component of the cognitive architecture is in charge of predicting the
first elements of the data-driven inference as shown in Figure 3: movements
and actions. The main idea behind this module is to incrementally aggregate
and refine data: a set of QSRs will be classified as a movement and a set of
movements will define an action.

####  III-D1 Focus Estimation

TABLE I: Encodings for the QSRs used in the Focus Estimator.

QDC | Encoding
---|---
touch | 0.5
near | 0.25
medium | 0.125
far | 0

QTC | Encoding
---|---
0 | 0.5
- | 0.25
+ | 0

If there are $n$ OOIs in the environment, the Low-Level would be able to
perform $n$ distinct classifications, one for each OOI. This means that, at
every timestep, CASPER would be able to identify $n$ different movements. In
reality, we know that the partner is an intentional agent which is performing
one movement at a time, directed towards a specific entity in the scene which
we call the “target”.

The challenge, then, is to identify the target using perceptual information
from the observed scene. Given the inherent uncertainty of this task, it seems
natural to solve it through the use of probabilistic models. Our algorithm
assigns a score $S$ to each OOI based on the following equation, which
envelops the use of motion and gaze as perceptual inputs to the intention
reading cognitive functions theorized by Tomasello et al. [26]:

$S(o_{i})=\frac{w_{QDC}\cdot QDC(o_{i})+w_{QTC}\cdot QTC(o_{i})}{1+\theta}$
(1)

for $o_{i}\in O$. In the above equation, $QDC(o_{i})$ and $QTC(o_{i})$
represent the QDC and QTC QSRs calculated on OOI $o_{i}$. These are
categorical variables, so they need to be encoded into numerical values using
the conversion shown in Table I. $w_{QDC}$ and $w_{QTC}$ are the positive
weights that we assign to these components, with $w_{QDC}+w_{QTC}=1$.

Given an uniform weight distribution, the numerator in Equation 1 is maximized
when the agent is maintaining touching distance with the object. On the
contrary, it is minimized when the agent is at a far distance, walking away
from it. This value is scaled by $\theta$, which represents the angle between
the agent’s heading and the reference line connecting them with the OOI.
Intuitively, the denominator penalizes OOIs which are not in the field of view
of the agent.

After calculating the attention score for every OOI, these are normalized into
probabilities. To win the competition and be elected as the target, an element
must both possess the maximum score and surpass the threshold $\tau=0.5$. To
eliminate any possible noise in the prediction that would affect the
processing chain, this item is not forwarded as it is, instead it is inserted
into a sliding window of size $w=4$ which allows the system to select the
target as a measure of central tendency. Any OOI that at any time occupies the
majority of the sliding window slots is declared to be the current target of
the observed agent’s focus.

The Focus Estimator keeps also track of the second-highest scoring OOI,
processing it independently using the same procedure described above. This
element, when it exists, is understood to be the “destination”, which will be
later used to contextualize the agent’s action. For example, if the agent is
transporting an empty glass towards a bottle, then they are probably going to
fill it and have a drink, conversely if the destination is the sink, then they
will likely going to wash it.

In case a tie occurs between the OOIs, if one of them was previously declared
as a target it will maintain its status, while the other one will be regarded
as the destination.

####  III-D2 Movement Classification

Figure 4: The Decision Tree maps a set of QSRs into a movement.

Once the Focus Estimator has identified the partner’s target, it is possible
to proceed with the movement prediction with respect to the inferred OOI. For
this purpose, we make use of a symbolic data-driven model: specifically, a
Decision Tree. We have chosen this model because it fits well for our purpose,
that is to form a mapping from a set of QSRs to a domain of discrete
movements, each representing a motion that the observed agent is performing in
a single timestep.

A graphical representation of this process can be viewed in Figure 4.

####  III-D3 Action Prediction

Through the procedures described in the previous sections, CASPER is able to
determine the movement performed by the agent at each timestep, providing the
robot with an instantaneous information of what is happening in a single unit
of time. To fully understand the behavior of the partner, however, we need to
analyze the temporal evolution of these movements: we call this an “action”.

To represent the composition of each action, we use a Markov-chain Finite
State Machine (FSM). This model describes a process in which the transition to
a state at time $t+1$ is probabilistic rather than deterministic and depends
only on the state at time $t$. Then, we combine these FSMs in an ensemble
which is used to classify a sequence of observations. Every time the Movement
Classifier generates new data, this is initially filtered such that only
transitions between different states are considered. This technique, already
implemented in other activity recognition and prediction models [1, 47],
allows the system to be time-invariant: this means that the speed at which the
action is performed does not influence its representation.

Every time a new filtered observation is detected, it is queued up with the
previous ones. The ensemble samples each of its FSMs using the initial
observation as the first state of the chain. Thereafter, it calculates the
similarity between the ordered sequence of observations and the generated
samples using the Ratcliff-Obershelp Pattern Recognition algorithm [48]. Each
FSM is assigned a score in $[0,1]$ based on this metric. An action is
predicted when there is a clear winner between the models, but only if the
score surpasses a certain threshold: this allows the system to be more robust
to transient effects that will create noise in the inference.

Actions might be ambiguous and require contextualization based on the location
within the environment in which they are performed. When this is the case, we
have opted to use a simple lookup table in which we use information from the
destination (see Section III-D1) to disambiguate actions.

###  III-E High-Level Goal Prediction

Figure 5: A visual demonstration of the Plan Library’s scoring system. These
trees represent two plans for two distinct goals with a single observation
$\hat{\sigma_{1}}=A$. The non-root nodes are drawn differently based on their
status: filled if observed, dashed if unobserved and textured if missed. In
this example, $P({G1}|\hat{\sigma})=0.59$ and $P({G2}|\hat{\sigma})=0.39$, so
$G1$ is considered the best explanation.

The purpose of this component is to form a computational representation of the
plan structure described in Figure 3 on which it is possible to execute
inference and reasoning. We model this data structure as a non-binary tree
where the root represents the goal and each terminal node is a possible action
(derived from sequences of movements as described in Section III-D).

The Plan Library $L$ is then defined as:

$L=\\{\Sigma,NT,G,T\\}$ (2)

Where $\Sigma$ is a set of terminal symbols that represent the observable
actions, $NT$ are the non-terminal symbols which stand as sub-goals, $G$ is
the set of goals and $T$ are the trees that describe the ordered production
rules that compose the plans.

During the intention reading process, the Low-Level module will produce a
serialized set of observed actions
$\hat{\sigma}=\\{\hat{\sigma}_{1},...,\hat{\sigma}_{n}\\}\in\Sigma$ that have
to be matched against the available plans $T$ in the Plan Library in order to
infer which goal $g\in G$ is driving the observable actions of the agent. We
work on the assumption that $\hat{\sigma}$ is temporally ordered, which means
that $\hat{\sigma}_{1}$ happens and is observed before $\hat{\sigma}_{2}$
occurs.

The intuition behind our design is the following: using Occam’s razor
principle, the plan that better describes the data is the one that more simply
fits the observations and that leaves less gaps in the explanation (intended
as nodes that should have been observed but are not present in
$\hat{\sigma}$).

The probability that goal $g$ is generating the observations $\hat{\sigma}$
is:

$P(g|\hat{\sigma})=\eta\cdot s(g)$ (3)

Where the score $s$ is defined as a function on the number of observed nodes
and the missed ones:

$s(g)=observed\cdot(1-missed)$ (4)

And $\eta$ is a normalization factor calculated as:

$\eta=\frac{1}{\sum_{g\in G}s(g)}$ (5)

Equation 4 penalizes the explanations that contain missed nodes, which are
nodes that should be present in the description but don’t appear in
$\hat{\sigma}$. This process is explained graphically in Figure 5.

Input: A plan library $L$, a set of observations $\hat{\sigma}$

Output: A set of ranked explanations $P$

Initialize $P$ with the unmarked plans $T$ in $L$

foreach  _ $\hat{\sigma_{i}}\in\hat{\sigma}$_ do

Initialize $P^{\prime}$ to empty

while  _$P$ is not empty_ do

Pop $p$ from $P$

foreach  _unobserved node $n$ in $p$ named $\hat{\sigma_{i}}$_ do

Generate a copy $p\prime$ of $p$

Mark $n$ as observed in $p\prime$

if  _there are any unobserved nodes on the left of $n$ in $p\prime$_ then

Mark them as missed

end if

Insert $p\prime$ in $P^{\prime}$

end foreach

end while

Insert $P^{\prime}$ in $P$

end foreach

Calculate the score for each plan;

return _The generated explanations $P$, ordered by score_

Algorithm 1 Explanations from partial observations

The procedure to derive the best explanation from a set of observations is
described in Algorithm 1. The process dynamically generates a set of
explanations, each of which accounts for every possible interpretation of the
observed symbols. This means that Algorithm 1 is able to deal with missed
observations, where the robot might for any reason not record some of the
actions performed by the partner. The explanation with the highest score at
the end of the computation is chosen as the inferred intention.

###  III-F Real-time Verification

The Verification module represents the robot’s cognitive common sense and is
responsible for the correctness of the predictions formulated step-by-step by
both the Low-Level and the High-Level. In its essence, it serves the purpose
of filtering out predictions that might arise at any level of abstraction
which do not constitute valid statements on the state of the world. For
example, an action prediction such as “the human picks up the table and places
it in the oven” might be a possible statement generated by the Low-Level, but
it makes no logical sense and must be a product of noise or a transient state.

In order to address the problem, we make use of a knowledge base in the form
of an ontology in which we represent the entities of our world and the
relations between them. Every time the Low-Level infers an action or the High-
Level generates an explanation, these are verified through a Semantic Reasoner
and if they are proven invalid they are discarded from the processing
pipeline. Additionally, this component is also used by CASPER to decide which
actions are assignable to the robot during the final collaborative decision-
making. As an example, it would not be possible for the robot to “eat the
meal”, while on the contrary it would be capable of performing the action
“wash the dishes”.

On a technical level, CASPER makes use of an OWL2 ontology and a Pellet
reasoner [49].

###  III-G Collaborative Intelligence

The final element of CASPER is the Supervisor, which is in charge of the
coordination of the sub-processes that constitute the cognitive architecture.
In particular, it manages the communication between these processes and
collects the data produced by the Low-Level and High-Level in order to achieve
the final purpose of this architecture: generate a collaborative behavior to
help the partner with whatever tasks they are involved with.

To do so, it uses the goal explanation produced by the High-Level to generate
an appropriate assistive plan. This involves the robot understanding which
actions are yet to be executed and reason about which ones it is able to
assist with. From the goal explanation, the Supervisor obtains the “frontier”,
that is the ordered set of all the unobserved nodes: these are the actions
that have yet to be performed in order to achieve the goal. Using the
validation procedure explained in Section III-F, it then identifies the
longest sequence of actions which the robot itself is able to perform. It then
continues observing, waiting for the partner to execute the rest of the plan
up to the point where the collaboration will start and in that moment it will
send instructions to the Robot Control.

##  IV Experiments

Figure 6: The experimental kitchen environment rendered in Webots. A TIAGo++
robot is tasked with observing a human performing actions in the scenery. The
7 OOIs are marked and annotated.

###  IV-A Experimental setup

In order to test the effectiveness of CASPER, we have decided to deploy it on
a selected case study. In particular, we have developed an experiment
involving a human and a TIAGo++ robot interacting inside a kitchen containing
7 OOIs: a water bottle, a canned meal, a box of biscuits, an empty glass, a
plate, a set of hobs and a sink. This environment is shown in Figure 6. This
was developed as a virtual environment created using the Webots open-source
physics simulator [50]

The robot is instructed to visually find and track the human, collect
observations on their actions within the room and process them through CASPER
to infer their goal and generate an appropriate collaborative behavior. Both
the human and the robot are able to navigate the environment, grasp items and
release them.

###  IV-B Low-Level

The _movements_ we have defined for our experiment are the following:

  * •

STILL: the agent is fixed in space, neither moving or interacting.

  * •

WALK: the agent moves in the environment, holding nothing.

  * •

TRANSPORT: same as WALK, but performed while carrying an object.

  * •

PICK: the agent collects and item.

  * •

PLACE: the agent positions a previously collected object somewhere in the
environment.

In order to train the Decision Tree to map a set of QSRs to these movements,
we need to generate a dataset. We do this by positioning the human in a random
position inside the room and tasking them to stay still for a while, then walk
to a random OOI, pick it up, transport it to a random destination, place it
and then stand still again. This demonstrates the full range of movements that
we wish to learn through our model. The robot, in turn, will observe the
scene, calculate the QSRs and associate them to a label which is manually
provided by the experimenter. A Decision Tree is a small-data model so we
don’t require a large amount of training samples: we repeat the previous
procedure 10 times, then fit the model with the collected data.

(a)

(b)

(c)

Figure 7: The FSMs which describe how each action is composed from the
primitive movements: (a) Pick and Place, (b) Use, (c) Relocate.

The chosen _actions_ for the kitchen setup are:

  * •

Pick and place: a PICK movement followed by a TRANSPORT and terminated by a
PLACE.

  * •

Use: a loop of PICK and PLACE movements.

  * •

Relocate: STILL, followed by WALK and another STILL movement.

The FSMs that describe these actions are reported in Figure 7.

There is one action which requires contextualization: Use. This is because the
latter can have a different meaning based on the location in which it is
performed, or in other words the destination of the action. The lookup table
that we use to contextualize it is reported in Table II.

TABLE II: Contextualization for the action ‘Use’. Destination | Contextualized Action
---|---
Sink | Wash
Hobs | Cook
Plate | Eat
Glass | Sip

###  IV-C High-Level

(a)

(b)

(c)

Figure 8: Detailed plans for each of the goals of the kitchen experiment: (a)
Breakfast, (b) Drink, (c) Lunch.

We define 3 distinct goals for our experiment:

  * •

Breakfast: the human will collect the biscuits, bring them to the plate and
eat them, then move the plate to the sink and rinse it.

  * •

Drink: the human will fetch the bottle of water, bring it to the glass and
have a sip, then wash the glass.

  * •

Lunch: the human will walk to the fridge and collect the canned meal, place it
on the hobs and cook it. Afterwards, they will bring it to the plate, eat and
wash the dishes.

The detailed plans for each of these goals are depicted in Figure 8.

In our plan library, as defined in Section III-E:

$\displaystyle\Sigma=[PickAndPlace,Wash,Cook,Eat,Sip]$ (6) $\displaystyle
NT=[PrepareMeal,Warm,Clean]$ (7)

Note that the action Relocate is missing from Equation 6: this is because our
current experiment involves a single room. We have nevertheless implemented
this action because of our future development plans for CASPER (see Section
VI).

###  IV-D Verification

Figure 9: The ontology used as a knowledge base to perform verification during
the kitchen experiment.

The ontology which we use to describe the kitchen experiment is reported in
Figure 9. This knowledge base defines each entity in the environment as
belonging to one of three macro-groups: Goals, Agents or Objects. Each Agent
can be a Human or a Robot, the latter only containing the TIAGo++ robot we are
using but potentially expandable to include several kinds of robots grouped by
their capabilities (for example, humanoid and non-humanoid). The Objects can
be Items or Furniture: the former includes both vessels and food.

TABLE III: Data properties for the ontological representation of the kitchen experimental environment. | Move | Eat | Drink | Cook | Wash
---|---|---|---|---|---
Sink |  |  |  |  |
Hobs |  |  |  |  |
Plate | ✓ |  |  |  | ✓
Glass | ✓ |  | ✓ |  | ✓
Biscuits | ✓ | ✓ |  |  |
Meal | ✓ | ✓ |  | ✓ |
WaterBottle | ✓ |  | ✓ |  |

Each element is characterized by some properties which define the kind of
interactions that are possible with each of them. These are reported in Table
III. For example, the biscuits can be eaten and moved, but not cooked, drinked
or washed. At the same time, the ontology defines some object properties, i.e.
relations between entities of the knowledge base which impose limitations
useful to verify the validity of the statements produced by the Low-Level and
the High-Level. For example, the Eat action might only be performed by Humans
with a target which is eatable and a destination which is a Vessel. A pair of
object properties are defined for each of the actions (including the
contextualized ones).

Finally, we use Semantic Web Rule Language (SWRL) [51] definitions to allow
the reasoner to perform inferences on incomplete statements. For example, we
know that if an Agent is washing an item, then the destination must be a Sink.

##  V Results and Discussion

###  V-A Focus Evolution in Time

Figure 10: Each OOI is annotated with a graph showing the evolution of the
focus estimation probability in time during the execution of the goal ‘Lunch’.
The arrow on the floor shows the trajectory of the human through the
environment.

Figure 10 explores the output of the Focus Estimator while observing a human
performing the goal Lunch (which, we recall, is executed by collecting the
meal from the fridge, cooking it on the hobs, eating it at the table and
finally washing the dishes). In particular, each OOI $o_{i}$ is annotated with
a graph describing the temporal evolution of the assigned normalized
probability values $P(o_{i})$. Of course, at each timestep: $\sum_{o_{i}\in
O}P(o_{i})=1$.

At the start of the simulation, the farthest OOIs have the lowest chance to be
considered as the target of the observed agent’s attention. When the human
starts moving and turns to their right, the probability for the Bottle
increases up to the designed threshold of 0.5, but soon after they face away
and the score drops. Thanks to the sliding window, this OOI is not
realistically considered as the human’s target.

The agent then continues its path to the north: the Plate’s probability
decreases as it exits their field of view, while the probabilities for the
Meal increases steadily. Around timestep 20, the human turns momentarily
towards the Biscuits and once again creates a probability spike which does not
last long enough to influence the system. Once the Meal is grasped,
$QDC(Meal)=Touch$ and $QTC(Meal)=0$, so the focus estimation for this OOI is
high.

Further ahead in time, around timestep 50, the focus is evenly divided between
the Meal and the Hobs during the Cook action. The human then turns towards the
table: the probability for the Hobs decreases rapidly while the ones for the
Plate increase. At this point, around timestep 90, the human finishes
approaching the table and starts eating. The Focus Estimation divides more or
less equally the probability for both the Meal and the Plate, with a 60/40
split.

Finally, the Plate is brought to the Sink. The focus score for the former is
high and eventually evens out with the latter, while the probability value of
the Meal drops to 0 very quickly.

Overall, Figure 10 shows that the Focus Estimator module is correctly able to
predict the human’s attention while they move around the environment.

###  V-B Decision Tree Training

Figure 11: The Decision Tree trained from the experimental data in the kitchen
collaboration environment. The QSRs are classified as one of the following
movements: Still, Walk, Transport, Pick or Place.

The Decision Tree that acts as the Movement Predictor is trained on a dataset
generated in the simulated environment following the procedure described in
Section IV-B. This dataset contains 362 training examples obtained from 10
random trials. The total number of samples per class is the following: STILL
(46), WALK (97), PICK (30), TRANSPORT (161) and PLACE (28). The model fitted
on this data is shown in Figure 11.

To evaluate the fitness of this model, we have performed a 10-fold cross-
validation: we split our dataset into the 10 groups from which it was
generated, leave one aside for testing and train on the remaining, then repeat
for each of the unique groups. Our average 10-fold cross-validation accuracy
is 0.94.

###  V-C Markov Chain Finite-State Machines

(a)

(b)

(c)

Figure 12: Temporal response of the three FSMs which define the actions ‘Pick
and Place’, ‘Use’ and ‘Relocate’ on different sequences of movements (reported
below each graph). The vertical dashed line indicates the moment in which the
ensemble has inferred the action.

To evaluate the fitness of the Action Predictor, we generate 3 sequences of 9
movements and we input them incrementally into the ensemble to analyze its
temporal response to the observations. The results are reported in Figure 12,
where we have plotted the similarity score calculated through the Ratcliff-
Obershelp algorithm at each timestep.

Since the sampling of each FSM uses the initial observed symbol as the
starting state, the similarity score is maximum for the first iterations.
Despite that, none of the models prevails on the others and no winner is
declared yet. As soon as more symbols are fed into the system, the scores
start to oscillate and differ, leaving one clear winner: this is the model
that best describes the observed sequence of observations. The first two
sequences produce a prediction on timestep 6, while the third one receives an
inference on timestep 5. These predictions are in line with what we would
expect given the symbols in the input sequences.

###  V-D Explanation Generation

TABLE IV: Experimental results on the Goal Reasoner. See Section V-D for the full explanation. TRIAL | ACTIONS | EXPLANATIONS | TIME ($\mu s$) | CONFIDENCE | OUTCOME
---|---|---|---|---|---
#1 | Pick&Place | 7 | 3.58 | 0.23 |
Eat | 3 | 1.91 | 0.53 | Breakfast
#2 | Pick&Place | 7 | 5.72 | 0.23 |
Sip | 1 | 2.15 | 1 | Drink
#3 | Pick&Place | 7 | 3.34 | 0.23 |
Cook | 1 | 1.91 | 1 | Lunch
#4 | Pick&Place | 7 | 3.58 | 0.23 |
Pick&Place | 5 | 3.34 | 0.28 |
Eat | 1 | 1.43 | 1 | Lunch
#5 | Pick&Place | 7 | 4.29 | 0.23 |
Pick&Place | 5 | 3.34 | 0.28 |
Pick&Place | 1 | 1.19 | 1 | Lunch
#6 | Pick&Place | 7 | 3.34 | 0.23 |
Pick&Place | 5 | 2.62 | 0.28 |
Wash | 5 | 2.62 | 0.3 |
#7 | Sip | 1 | 3.1 | 1 | Drink
#8 | Eat | 2 | 2.38 | 0.69 |
#9 | Wash | 3 | 4.29 | 0.41 |
Cook | 0 | 4.53 | 0 |

To test the Goal Reasoner embedded in the High-Level module, we have run 9
trials in which we have provided it with several sequences of observations.
For the rest of this discussion, please refer to Figure 8 for the structure of
our Goal Library.

Table IV summarizes the data we have collected. The latter shows, from left to
right: the id of the trial, the observation (action) that was incrementally
input in the system, the number of explanations generated by the reasoner, the
time in microseconds required to produce the result, the confidence of the
top-scoring explanation and finally the output of the component. If the latter
is blank, then the reasoner could not formulate a prediction, otherwise it
will report the name of the goal whose plan explains the observations. The
system was reset between each trial.

Trial 1 presents to the High-Level the actions ‘Pick and place’ and ‘Eat’.
There are three possible explanations for these observations: one that
describes the goal Breakfast with only observed and unobserved nodes and two
that represent the plan for Lunch that accounts for several missing nodes
(recall that an unobserved node is marked as missed if another node is
observed on its right-hand side). Since our model assigns a higher score to
the simplest model to describe the data, Breakfast is chosen as the
prediction.

Trials 2 and 3 are straightforward: the goals Drink and Lunch are the only
ones that contain respectively the actions ‘Pick and Place’ followed by either
‘Sip’ and ‘Cook’, so the Goal Reasoner can formulate a very confident
prediction.

Trial 4 and 5 are more ambiguous: both of them begin with two ‘Pick and place’
actions which could describe each of the models with similar probability. Only
when the system receives the third observation it can commit to a clear
inference. Trial 6 follows a similar narrative, but the third input is still
not able to disambuigate the goal: each of the possible plans share an uniform
probability distribution and the reasoner fails to produce a prediction. No
additional inputs would change the situation, since the rightmost node of the
tree has been observed (remember that we assume that each observation happens
after the preceding one). Of course, one could object that the goal could
indeed by identified by the OOIs with which the human is interacting. This
will be done in the full-scale experiment by the Verification component, which
uses its ontology to filter out invalid explanations such as the goal Lunch if
the first action ‘Pick and Place’ has been performed on the Biscuits.

In trial 7, we provide the system with a single observation that, alone, is
able to discern the goal. We try doing the same with the observation ‘Eat’ in
trial 8, but that action on its own could describe both Breakfast or Lunch. No
further observation would be able to disambiguate this scenario, since the
only next possible actions are ‘Pick and Place’ and ‘Wash’, which are common
to both the candidates.

Finally, trial 9 shows an invalid sequence of actions: there is no goal plan
in which the action ‘Wash’ is followed by any other action. For this reason,
the system produced no valid explanations.

###  V-E Intention Reading and Collaboration

TABLE V: CASPER’s performance on the kitchen experiment. See Section V-E for
the full explanation.

Goal | Observed | Missed | Waiting | Planned | Accuracy | Time (s)
---|---|---|---|---|---|---
Breakfast | 1.0 | 0.2 | 0.0 | 1.8 | 100% | 42.21
Drink | 1.0 | 0.4 | 0.0 | 1.6 | 100% | 54.14
Lunch | 1.8 | 0.2 | 1.8 | 2.0 | 100% | 82.46

Having verified the single components that constitute the Low-Level and High-
Level modules of CASPER, we are now ready to analyze the overall performance
of the cognitive architecture working together to read the human’s intention
and producing collaborative decision-making. The procedure we have followed is
the following: we have run 5 trials for each of the 3 goals, randomizing the
human’s starting position and collecting a total of 15 data samples. For each
of these, we have recorded: the number of observed and missed nodes in the
winning explanation, the number of actions that the robot waits for the
partner to complete before collaborating, the number of actions that the
robots plans to execute, the accuracy of the prediction and the time, in
seconds, needed to make an inference. The mean values of these variables are
collected in Table V.

The first thing to notice is that the robot was always able to correctly read
the human’s intention, despite some noise in the perceptual data collected by
the system: the synergistic interaction of CASPER’s components results in a
robust intention reading performance. Table V also indicates that the time and
observations required to infer the goal Lunch were higher than for the other
two goals: this is in line with the higher complexity of its plan compared to
the ones for Breakfast and Lunch.

The collaborative plan calculated from the cognitive architecture in each case
was to wait for the human to finish eating their meal and then clean up the
kitchen, which involves transporting the plate to the sink and washing it.

###  V-F Verification

Figure 13: Average prediction time of CASPER with and without the Verification
module. Despite the computational overhead of the Pellet reasoner, the
verification module ensures a faster inference from less observations.

An additional experiment was carried out to investigate the performance of the
Verification module. In particular, we run the same experiment used to
generate the data we have discussed in Section V-E, this time disabling the
formal verification of both the Low-Level and the High-Level: we call this the
Non-Verified condition, as opposed as the Verified condition which represents
the full cognitive architecture.

Figure 13 shows the comparison between the two configurations. The results
that we collected prove that by disabling the Verification module we don’t
hurt the accuracy of the system, but instead we cause a slower prediction
time. Despite the computational overhead introduced by the semantic reasoner,
the Verified condition outperformed the Non-Verified one. This happens because
the latter requires more observations to make sense of the environment, whilst
the Verification module is able to discard noisy observations and illogical
explanations before either of them are further processed, cutting down the
overall inference time.

##  VI Conclusion and Future Work

In this paper, we have introduced CASPER: a symbolic cognitive architecture
designed to perform intention reading and to calculate collaborative behaviors
for human-robot teaming scenarios. Our system is able to accomplish the task
through a set of parallel processes that communicate with each other and that
can translate QSR descriptors into movements, then into actions and finally
into goals and sub-goals using a bottom-up approach. Through the
implementation of a simulated experimental case study based on a kitchen
environment, we have proven the soundness of our methodologies.

The design of this system is driven by the requirement to embed in social
robots the ability to autonomously integrate themselves in the structure of
our daily routines, without the need for a human operator to explicitly
provide instructions for the machine. Instead, by being able to understand the
actions of other agents within the environment, a robot endowed with this
cognitive architecture is able to seamlessly cooperate with them. In fact,
despite our focus on human-robot interaction, this architecture would be
equally applicable to robot-robot interactions.

Our main scientific contribution is the demonstration that QSRs can be used as
an efficient means to achieve intention reading capabilities in artificial
intelligence systems, a proof of concept that is lacking in the current state-
of-the-art. Our technological contribution comes in the form of a cognitive
architecture that incorporates novel algorithms for perception, reasoning and
action selection which take inspiration from psychology and cognitive science.

According to the taxonomy defined by Kotseruba and Tsotsos [2], CASPER is a
symbolic architecture which implements the most common cognitive mechanisms,
which we shall now summarize. Perception, the process that transforms raw
input into the system’s internal representation for carrying out cognitive
tasks, is performed by vision. Attention is modeled as a viewpoint/gaze
selection mechanism: this means that the robot endowed with CASPER is able to
select a target and track it through space. Action selection, which drives
decision-making, is the result of a planning strategy which aims to maximize
the relevance of the selected behavior. This architecture possesses all three
types of memory: sensory (the QSR Library), working (handled by the Low- and
High-Level) and long-term (in the form of the learned models and ontology).
Learning comes in the form of declarative knowledge, which is a collection of
facts about the world and various relationships defined between them.
Reasoning is a cognitive ability which is present and central to each and
every cognitive architecture, including this one. Finally, CASPER also
implements metacognition, that is the ability to reason about one’s own
thoughts: this is done by the Verification module, which constantly monitors
the other internal processes, identifying and correcting any erroneous
decisions.

A limitation of our work is represented by the intrinsic nature of symbolic
artificial intelligence: our methodologies suffer from the Knowledge
Acquisition Bottleneck [52], which refers to the human intervention required
to translate real-world conditions in symbolic inputs for the intelligent
system. In our case, this comes in the form of the selection of OOIs, the
plans for each goal and the ontology that envelops the properties of the
environment. Despite this, we argue that this disadvantage is compensated by
the interpretability of each of the components that build up CASPER (the
decisions of which can be explained at each step of computation) and the lack
of computationally expensive and data-hungry processes.

Another limitation is given by the fact that we have tested CASPER in
simulation and with fairly simple goals. The reason for this is that the work
presented here is foundational to the true purpose of this cognitive
architecture: that is, to offer support for intention reading and trust
considerations in heterogeneous multi-agent teaming scenarios. Our planned
future work involves the creation of a more complex environment composed of
multiple rooms where distributed groups of humans will be interacting with
robots of different make and capabilities in order to achieve a Team Goal. In
this kind of setting, the individual robots will not be able to gather all the
necessary information needed to predict the shared objective, rather they will
have to rely on partial observations and communication with their peers.
Moreover, performing empirical trials in simulation gives us the freedom to
experiment with arbitrarily different environments, including varying numbers
and type of agents.

One additional future expansion on CASPER is the inclusion of artificial trust
considerations [35, 53, 54] through which the endowed robot will be able to
assess the capabilities of other agents (humans or robots) to pursue the
desired goal. Our hypothesis is that this cognitive skill will be valuable in
allowing the group of heterogeneous robots to sort each other between the
several available collaborative assignments required to assist the humans of
their team.

## References

  * [1] Samuele Vinanzi, Christian Goerick, and Angelo Cangelosi.  Mindreading for robots: Predicting intentions via dynamical clustering of human postures.  In 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pages 272–277, 2019.
  * [2] Iuliia Kotseruba and John K Tsotsos.  40 years of cognitive architectures: core cognitive abilities and practical applications.  Artificial Intelligence Review, 53(1):17–94, 2020.
  * [3] Angelo Cangelosi and Minoru Asada.  Cognitive robotics.  MIT Press, 2022.
  * [4] Jay Friedenberg, Gordon Silverman, and Michael J Spivey.  Cognitive science: an introduction to the study of mind.  Sage Publications, 2021.
  * [5] John K Tsotsos.  Attention and cognition: principles to guide modeling.  In Computational and cognitive neuroscience of vision, pages 277–295. Springer, 2017.
  * [6] Usef Faghihi, Pierre Poirier, and Othalia Larue.  Emotional cognitive architectures.  In International Conference on Affective Computing and Intelligent Interaction, pages 487–496. Springer, 2011.
  * [7] Susan L Epstein.  Metaknowledge for autonomous systems.  In Proceedings of AAAI Spring Symposium on Knowledge Representation and Ontology for Autonomous Systems. AAAI, pages 61–68, 2004\.
  * [8] John R Anderson.  The architecture of cognition.  Psychology Press, 2013.
  * [9] John E Laird, Allen Newell, and Paul S Rosenbloom.  Soar: An architecture for general intelligence.  Artificial intelligence, 33(1):1–64, 1987.
  * [10] Javier Snaider, Ryan McCall, and Stan Franklin.  The lida framework as a general tool for agi.  In International Conference on Artificial General Intelligence, pages 133–142. Springer, 2011.
  * [11] Pei Wang.  Non-axiomatic reasoning system: Exploring the essence of intelligence.  Indiana University, 1995.
  * [12] A Chella, F Lanza, and V Seidita.  A cognitive architecture for human-robot teaming interaction.  In 6th International Workshop on Artificial Intelligence and Cognition, AIC 2018, volume 2418, pages 82–89. CEUR-WS, 2019.
  * [13] Ji-Hyeong Han and Jong-Hwan Kim.  Human-robot interaction by reading human intention based on mirror-neuron system.  In 2010 IEEE International Conference on Robotics and Biomimetics, pages 561–566. IEEE, 2010.
  * [14] Alicia P Melis.  The evolutionary roots of human collaboration: coordination and sharing of resources.  Annals of the New York Academy of Sciences, 1299(1):68–76, 2013\.
  * [15] Ilias El Makrini, Kelly Merckaert, Dirk Lefeber, and Bram Vanderborght.  Design of a collaborative architecture for human-robot assembly tasks.  In IROS, pages 1624–1629, 2017.
  * [16] Lihui Wang, R Gao, József Váncza, J Krüger, Xi Vincent Wang, S Makris, and G Chryssolouris.  Symbiotic human-robot collaborative assembly.  CIRP annals, 68(2):701–726, 2019.
  * [17] Panagiota Tsarouchi, Alexandros-Stereos Matthaiakis, Sotiris Makris, and George Chryssolouris.  On a human-robot collaboration in an assembly cell.  International Journal of Computer Integrated Manufacturing, 30(6):580–589, 2017.
  * [18] Guy Hoffman and Cynthia Breazeal.  Collaboration in human-robot teams.  In AIAA 1st Intelligent Systems Technical Conference, page 6434, 2004.
  * [19] William Bluethmann, Robert Ambrose, Myron Diftler, Scott Askew, Eric Huber, Michael Goza, Fredrik Rehnmark, Chris Lovchik, and Darby Magruder.  Robonaut: A robot designed to work with humans in space.  Autonomous robots, 14(2-3):179–197, 2003.
  * [20] Stéphane Lallée, Eiichi Yoshida, Anthony Mallet, Francesco Nori, Lorenzo Natale, Giorgio Metta, Felix Warneken, and Peter Ford Dominey.  Human-robot cooperation based on interaction learning.  In From motor learning to interaction learning in robots, pages 491–536. Springer, 2010.
  * [21] Joelle Pineau, Michael Montemerlo, Martha Pollack, Nicholas Roy, and Sebastian Thrun.  Towards robotic assistants in nursing homes: Challenges and results.  Robotics and autonomous systems, 42(3-4):271–281, 2003.
  * [22] Francesco Semeraro, Alexander Griffiths, and Angelo Cangelosi.  Human-robot collaboration and machine learning: a systematic review of recent research.  Robotics and Computer-Integrated Manufacturing, July 2022.
  * [23] Amanda L Woodward, Jessica A Sommerville, Sarah Gerson, Annette ME Henderson, and Jennifer Buresh.  The emergence of intention attribution in infancy.  Psychology of learning and motivation, 51:187–222, 2009.
  * [24] Bertram F Malle, Louis J Moses, and Dare A Baldwin.  Intentions and intentionality: Foundations of social cognition.  MIT press, 2001.
  * [25] Dare A Baldwin and Jodie A Baird.  Discerning intentions in dynamic human action.  Trends in cognitive sciences, 5(4):171–178, 2001.
  * [26] Michael Tomasello, Malinda Carpenter, Josep Call, Tanya Behne, and Henrike Moll.  Understanding and sharing intentions: The origins of cultural cognition.  Behavioral and Brain Sciences, 28(5):675–691, 2005.
  * [27] Peter Ford Dominey and Felix Warneken.  The basis of shared intentions in human and robot cognition.  New Ideas in Psychology, 29(3):260–274, 2011.
  * [28] Nuno Ferreira Duarte, Mirko Raković, Jovica Tasevski, Moreno Ignazio Coco, Aude Billard, and José Santos-Victor.  Action anticipation: Reading the intentions of humans and robots.  IEEE Robotics and Automation Letters, 3(4):4132–4139, 2018.
  * [29] Z Zenn Bien, Kwang-Hyun Park, Jin-Woo Jung, and Jun-Hyeong Do.  Intention reading is essential in human-friendly interfaces for the elderly and the handicapped.  IEEE Transactions on Industrial Electronics, 52(6):1500–1505, 2005\.
  * [30] Karim A Tahboub.  Intelligent human-machine interaction based on dynamic bayesian networks probabilistic intention recognition.  Journal of Intelligent & Robotic Systems, 45(1):31–52, 2006.
  * [31] Miriam Buonamente, Haris Dindo, and Magnus Johnsson.  Recognizing actions with the associative self-organizing map.  In Information, Communication and Automation Technologies (ICAT), 2013 XXIV International Symposium on, pages 1–5. IEEE, 2013.
  * [32] Bart Jansen and Tony Belpaeme.  A computational model of intention reading in imitation.  Robotics and Autonomous Systems, 54(5):394–402, 2006.
  * [33] Erhan Oztop, Mitsuo Kawato, and Michael Arbib.  Mirror neurons and imitation: A computationally guided review.  Neural Networks, 19(3):254–271, 2006.
  * [34] Roger Granada, Ramon Fraga Pereira, Juarez Monteiro, Rodrigo Barros, Duncan Ruiz, and Felipe Meneguzzi.  Hybrid activity and plan recognition for video streams.  In The 2017 AAAI Workshop on Plan, Activity, and Intent Recognition. Greer, JE and Koehn, GM, pages 54–59, 1995.
  * [35] Samuele Vinanzi, Angelo Cangelosi, and Christian Goerick.  The collaborative mind: Intention reading and trust in human-robot interaction.  iScience, page 102130, 2021.
  * [36] S. Vinanzi, A. Cangelosi, and C. Goerick.  The role of social cues for goal disambiguation in human-robot cooperation.  In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 971–977, 2020.
  * [37] Reinhard Moratz.  Qualitative Spatial Reasoning, pages 1700–1707.  Springer International Publishing, Cham, 2017.
  * [38] Paul Duckworth, David C Hogg, and Anthony G Cohn.  Unsupervised human activity analysis for intelligent mobile robots.  Artificial Intelligence, 270:67–92, 2019.
  * [39] Ardhendu Behera, Anthony G Cohn, and David C Hogg.  Workflow activity monitoring using dynamics of pair-wise qualitative spatial relations.  In International Conference on Multimedia Modeling, pages 196–209. Springer, 2012.
  * [40] Krishna Dubba, Miguel de Oliveira, Gi Hyun Lim, Hamidreza Kasaei, Luis Seabra Lopes, Ana Tomé, and Anthony Cohn.  Grounding language in perception for scene conceptualization in autonomous robots.  In 2014 AAAI Spring Symposium Series, 2014.
  * [41] Jay Young and Nick Hawes.  Learning by observation using qualitative spatial relations.  In International Conference on Autonomous Agents & Multiagent Systems, 2015, pages 745–751. Association for Computing Machinery, 2015.
  * [42] Nikhil Krishnaswamy, Scott Friedman, and James Pustejovsky.  Combining deep learning and qualitative spatial reasoning to learn complex structures from sparse examples with noise.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2911–2918, 2019.
  * [43] Yiannis Gatsoulis, Muhannad Alomari, Chris Burbridge, Christian Dondrup, Paul Duckworth, Peter Lightbody, Marc Hanheide, Nick Hawes, DC Hogg, AG Cohn, et al.  Qsrlib: a software library for online acquisition of qualitative spatial relations from video.  In Workshop on Qualitative Reasoning (QR16), at IJCAI-16, 2016.
  * [44] Eliseo Clementini, Paolino Di Felice, and Daniel Hernández.  Qualitative representation of positional information.  Artificial intelligence, 95(2):317–356, 1997.
  * [45] Matthias Delafontaine, Anthony G Cohn, and Nico Van de Weghe.  Implementing a qualitative calculus to analyse moving point objects.  Expert Systems with Applications, 38(5):5187–5196, 2011.
  * [46] Nico Van de Weghe.  Representing and reasoning about moving objects: A qualitative approach.  PhD thesis, Ghent University, 2004.
  * [47] Alessandro Manzi, Paolo Dario, and Filippo Cavallo.  A human activity recognition system based on dynamic clustering of skeleton data.  Sensors, 17(5):1100, 2017.
  * [48] John W Ratcliff and David E Metzener.  Pattern-matching-the gestalt approach.  Dr Dobbs Journal, 13(7):46, 1988.
  * [49] Bijan Parsia and Evren Sirin.  Pellet: An owl dl reasoner.  In Third international semantic web conference-poster, volume 18, page 13. Citeseer, 2004.
  * [50] Olivier Michel.  Cyberbotics ltd. webots™: professional mobile robot simulation.  International Journal of Advanced Robotic Systems, 1(1):5, 2004\.
  * [51] Ian Horrocks, Peter F Patel-Schneider, Harold Boley, Said Tabet, Benjamin Grosof, Mike Dean, et al.  Swrl: A semantic web rule language combining owl and ruleml.  W3C Member submission, 21(79):1–31, 2004.
  * [52] JetABRYMAN Cullen and Alan Bryman.  The knowledge acquisition bottleneck: time for reassessment?  Expert Systems, 5(3):216–225, 1988.
  * [53] Samuele Vinanzi, Massimiliano Patacchiola, Antonio Chella, and Angelo Cangelosi.  Would a robot trust you? Developmental robotics model of trust and theory of mind.  Philosophical Transactions of the Royal Society of London B, 2019\.
  * [54] Carolina Centeio Jorge, Myrthe L Tielman, and Catholijn M Jonker.  Artificial trust as a tool in human-ai teams.  In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, pages 1155–1157, 2022.