# SMEFT truncation effects in Higgs boson pair production at NLO QCD

Gudrun Heinrich  Jannis Lang  Institute for Theoretical Physics, Karlsruhe
Institute of Technology, 76131 Karlsruhe, Germany<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We present results for Higgs boson pair production in gluon fusion at next-to-
leading order in QCD, including effects of anomalous couplings within Standard
Model Effective Field Theory (SMEFT). In particular, we investigate truncation
effects of the SMEFT series, comparing different ways to treat powers of
dimension-six operators and double operator insertions.

##  1 Introduction

Higgs boson pair production in gluon fusion offers the possibility to measure
the trilinear Higgs boson self-coupling and therefore to verify whether the
form of the Higgs potential assumed in the Standard Model (SM) is correct.
Deviations from this form, manifesting themselves in anomalous Higgs boson
self-couplings, would be a clear sign of new physics and most likely would
come along with other non-SM Higgs couplings. Therefore it is important to
control the uncertainties of the theory predictions in simulations that
include anomalous couplings. The theoretical uncertainties have various
sources, the dominant ones in the SM being uncertainties related to the top
quark mass renormalisation scheme. Theory predictions with full top quark mass
dependence are available at NLO QCD [1, 2, 3, 4] and have been included in
calculations where higher orders have been performed in the heavy top limit
[5, 6, 7], thus reducing the scale uncertainties and the uncertainties due to
missing top quark mass effects, such that the top mass scheme uncertainties
currently constitute the main uncertainties [8] of the SM predictions.

Going beyond the SM description of the process $gg\to HH$, considering in
particular effective field theory (EFT) parametrisations of new physics
effects, new uncertainties arise, coming mainly from the truncation of the EFT
expansion.

In the following we will present results at NLO SMEFT for this process,
including also double operator insertions. Our implementation allows us to
investigate various scenarios of truncation and to assess the related
uncertainties. For more details we refer to Ref. [9].

##  2 Effective field theory descriptions of Higgs boson pair production

###  2.1 HEFT and SMEFT

In Standard Model Effective Field Theory (SMEFT) [10, 11], an effective
description of unknown interactions at a new physics scale $\Lambda$ is
constructed as an expansion in inverse powers of $\Lambda$, with operators
$\mathcal{O}_{i}$ of canonical dimension larger than four and corresponding
Wilson coefficients $C_{i}$,

$\mathcal{L}_{\text{SMEFT}}=\mathcal{L}_{\text{SM}}+\sum_{i}\frac{C_{i}^{(6)}}{\Lambda^{2}}\mathcal{O}_{i}^{\rm{dim6}}+{\cal
O}(\frac{1}{\Lambda^{3}})\;.$ (1)

In SMEFT it is assumed that the physical Higgs boson is part of a doublet
transforming linearly under $SU(2)_{L}\times U(1)$.

Higgs Effective Field Theory (HEFT) instead is based on an expansion in terms
of loop orders, which also can be formulated in terms of chiral dimension
counting [12, 13]. The expansion parameter is given by
$f^{2}/\Lambda^{2}\simeq\frac{1}{16\pi^{2}}$, where $f$ is a typical energy
scale at which the EFT expansion is valid (for example the pion decay constant
in chiral perturbation theory),

$\displaystyle{\cal L}_{d_{\chi}}={\cal
L}_{(d_{\chi}=2)}+\sum_{L=1}^{\infty}\sum_{i}\left(\frac{1}{16\pi^{2}}\right)^{L}c_{i}^{(L)}O^{(L)}_{i}\;.$
(2)

The SMEFT Lagrangian is typically given in the so-called Warsaw basis [10],
the terms relevant to the process $gg\to HH$ read

$\begin{split}\Delta\mathcal{L}_{\text{Warsaw}}&=\frac{C_{H,\Box}}{\Lambda^{2}}(\phi^{\dagger}\phi)\Box(\phi^{\dagger}\phi)+\frac{C_{HD}}{\Lambda^{2}}(\phi^{\dagger}D_{\mu}\phi)^{*}(\phi^{\dagger}D^{\mu}\phi)+\frac{C_{H}}{\Lambda^{2}}(\phi^{\dagger}\phi)^{3}\\\
&+\left(\frac{C_{uH}}{\Lambda^{2}}\phi^{\dagger}{\phi}\bar{q}_{L}\phi^{c}t_{R}+h.c.\right)+\frac{C_{HG}}{\Lambda^{2}}\phi^{\dagger}\phi
G_{\mu\nu}^{a}G^{\mu\nu,a}\;.\end{split}$ (3)

The dipole operator $\bar{{\cal O}}_{uG}$ is not included here because it can
be shown that it carries an extra loop suppression factor $1/16\pi^{2}$
relative to the other contributions if weak coupling to the heavy sector is
assumed [14, 15]. In the strong coupling case an expansion in the canonical
dimension only would not be the appropriate description.

The HEFT Lagrangian relevant to Higgs boson pair production in gluon fusion
can be parametrised by five a priori independent anomalous couplings as
follows [14]

$\displaystyle\Delta{\cal
L}_{\text{HEFT}}=-m_{t}\left(c_{t}\frac{h}{v}+c_{tt}\frac{h^{2}}{v^{2}}\right)\,\bar{t}\,t-c_{hhh}\frac{m_{h}^{2}}{2v}h^{3}+\frac{\alpha_{s}}{8\pi}\left(c_{ggh}\frac{h}{v}+c_{gghh}\frac{h^{2}}{v^{2}}\right)\,G^{a}_{\mu\nu}G^{a,\mu\nu}\;.$
(4)

Expanding the Higgs doublet in eq. (3) around its vacuum expectation value and
applying a field redefinition for the physical Higgs boson

$\displaystyle h\to
h+v^{2}\frac{C_{H,kin}}{\Lambda^{2}}\left(h+h^{2}+\frac{h^{3}}{3}\right)\;,$
(5)

with
$\frac{C_{H,kin}}{\Lambda^{2}}:=\frac{C_{H,\Box}}{\Lambda^{2}}-\frac{1}{4}\frac{C_{HD}}{\Lambda^{2}}$,
the Higgs kinetic term acquires its canonical form (up to ${\cal
O}\left(\Lambda^{-4}\right)$ terms). After that, relating the couplings
through a comparison of the coefficients of the corresponding terms in the
Lagrangian leads to the expressions given in Table 1.

HEFT | Warsaw
---|---
$c_{hhh}$ | $1-2\frac{v^{2}}{\Lambda^{2}}\frac{v^{2}}{m_{h}^{2}}\,C_{H}+3\frac{v^{2}}{\Lambda^{2}}\,C_{H,kin}$
$c_{t}$ | $1+\frac{v^{2}}{\Lambda^{2}}\,C_{H,kin}-\frac{v^{2}}{\Lambda^{2}}\frac{v}{\sqrt{2}m_{t}}\,C_{uH}$
$c_{tt}$ | $-\frac{v^{2}}{\Lambda^{2}}\frac{3v}{2\sqrt{2}m_{t}}\,C_{uH}+\frac{v^{2}}{\Lambda^{2}}\,C_{H,kin}$
$c_{ggh}$ | $\frac{v^{2}}{\Lambda^{2}}\frac{8\pi}{\alpha_{s}}\,C_{HG}$
$c_{gghh}$ | $\frac{v^{2}}{\Lambda^{2}}\frac{4\pi}{\alpha_{s}}\,C_{HG}$
Table 1: Leading order translation between different operator basis choices.

However, it should be emphasized that a translation between the coefficients
at Lagrangian level must be applied with care. The EFT parametrisations have a
validity range limited by unitarity constraints and the assumption that
$C_{i}/\Lambda^{2}$ in SMEFT is a small quantity. Furthermore there are
relations between the coefficients in SMEFT which are not present in HEFT.
Therefore a naive translation from HEFT (which is more general) to SMEFT can
lead out of the validity range for a given point in the coupling parameter
space, even if it is a perfectly valid point in HEFT.

###  2.2 SMEFT truncation

Another delicate point in the EFT expansion is the question how to treat terms
with inverse powers of $\Lambda$ higher than two at cross section level, i.e.
when squaring the amplitude. These are terms related to squared dim-6
operators, double operator insertions in a single diagram and combinations
thereof. Related issues have been discussed recently in Ref. [16].

We now present a Monte Carlo program which allows us to study the truncation
effects systematically. In order to construct the different truncation options
we first deconstruct the amplitude into three parts: the pure SM contribution
(SM), single dim-6 operator insertions ($\rm{dim6}$) and double dim-6 operator
insertions ($\rm{dim6}^{2}$):

$\displaystyle{\cal M}=$ $\displaystyle\ \begin{aligned}
\vbox{\hbox{\includegraphics[page=8,scale={0.7}]{figures/HH_presentation}}}\\\\[-6.0pt]
\scalebox{0.1}{{\color[rgb]{1,1,1}dummy}}\end{aligned}+\vbox{\hbox{\includegraphics[page=9,scale={0.7}]{figures/HH_presentation}}}+\vbox{\hbox{\includegraphics[page=10,scale={0.7}]{figures/HH_presentation}}}$
$\displaystyle\
+\vbox{\hbox{\includegraphics[page=11,scale={0.7}]{figures/HH_presentation}}}+\vbox{\hbox{\includegraphics[page=12,scale={0.7}]{figures/HH_presentation}}}+\dots$
$\displaystyle=$ $\displaystyle\ {\cal M}_{\text{SM}}+{\cal
M}_{\rm{dim6}}+{\cal M}_{\rm{dim6}^{2}}\;,$ (6)

where $c^{\prime}$ denotes the corresponding coupling combination listed in
Table 1. For the squared amplitude forming the cross section, we consider four
possibilities to choose which parts of $|{\cal M}|^{2}$ from eq. (6) may
enter:

$\displaystyle\sigma\simeq\left\\{\begin{aligned} &\
\sigma_{\text{SM}}+\sigma_{\text{SM}\times\rm{dim6}}\\\ &\
\sigma_{\left(\text{SM}+\rm{dim6}\right)\times\left(\text{SM}+\rm{dim6}\right)}\\\
&\
\sigma_{\left(\text{SM}+\rm{dim6}\right)\times\left(\text{SM}+\rm{dim6}\right)}+\sigma_{\text{SM}\times\rm{dim6}^{2}}\\\
&\
\sigma_{\left(\text{SM}+\rm{dim6}+\rm{dim6}^{2}\right)\times\left(\text{SM}+\rm{dim6}+\rm{dim6}^{2}\right)}\;.\end{aligned}\right.$
(7)

The first line is the first order of an expansion of $\sigma\sim|{\cal
M}|^{2}$ in $\Lambda^{-2}$, the second term is the first order of an expansion
of ${\cal M}$ in $\Lambda^{-2}$. The third line includes all terms of ${\cal
O}\left(\Lambda^{-4}\right)$ coming from single and double dim-6 operator
insertions, however it lacks the contribution at the same order from dim-8
operators and ${\cal O}\left(\Lambda^{-4}\right)$ terms following the field
redefinition of eq. (5). The fourth line is the naive translation from HEFT to
SMEFT using Table 1. Typically, only the first two options are used for
predictions and measurements using SMEFT, since both are unambiguous wrt.
basis change and gauge invariance, however there is still a debate about the
recommendations for their application to experimental analyses [16]. Thus, we
include all of the presented options in our calculation, which can serve to
contrast different outcomes of the predictions.

##  3 NLO Implementation into the event generator program POWHEG and results

###  3.1 Parametrisation of the $gg\to HH$ total cross section

Our implementation is based on the publicly available NLO HEFT code presented
in Refs. [17, 18], converted to the SMEFT framework and extended such that the
different options described in the previous section can be calculated,
including NLO QCD corrections.

For the real emission, a modified GoSam [19] version is built that splits the
amplitude evaluation according to eq. (6) and is able to generate the squared
amplitude with the truncation option which can be set by the user via an input
variable. For the generation of the GoSam files in POWHEG [20], a model in UFO
format [21] has been produced which specifies the anomalous couplings such
that GoSam is able to calculate the different contributions according to the
chosen truncation option. The existing interface to POWHEG has been modified
to hand over event parameters to GoSam such that the factor $\alpha_{s}$
between Higgs-gluon couplings in HEFT and SMEFT is evaluated at the correct
energy scale.

The virtual part is based on grids encoding the virtual 2-loop amplitudes.
These grids can be used to reconstruct the amplitude for any given combination
of anomalous couplings. The $a_{i}$ listed below are defined as the
coefficients in the representation of the sqared amplitude as a linear
combination of all coupling combinations possible in HEFT at NLO QCD.

$\displaystyle|{\cal M}_{BSM}|^{2}=$ $\displaystyle a_{1}\cdot
c_{t}^{4}+a_{2}\cdot c_{tt}^{2}+a_{3}\cdot c_{t}^{2}c_{hhh}^{2}+a_{4}\cdot
c_{ggh}^{2}c_{hhh}^{2}+a_{5}\cdot c_{gghh}^{2}+a_{6}\cdot
c_{tt}c_{t}^{2}+a_{7}\cdot c_{t}^{3}c_{hhh}$ $\displaystyle+$ $\displaystyle
a_{8}\cdot c_{tt}c_{t}c_{hhh}+a_{9}\cdot c_{tt}c_{ggh}c_{hhh}+a_{10}\cdot
c_{tt}c_{gghh}+a_{11}\cdot c_{t}^{2}c_{ggh}c_{hhh}+a_{12}\cdot
c_{t}^{2}c_{gghh}$ $\displaystyle+$ $\displaystyle a_{13}\cdot
c_{t}c_{hhh}^{2}c_{ggh}+a_{14}\cdot c_{t}c_{hhh}c_{gghh}+a_{15}\cdot
c_{ggh}c_{hhh}c_{gghh}+a_{16}\cdot c_{t}^{3}c_{ggh}$ $\displaystyle+$
$\displaystyle a_{17}\cdot c_{t}c_{tt}c_{ggh}+a_{18}\cdot
c_{t}c_{ggh}^{2}c_{hhh}+a_{19}\cdot c_{t}c_{ggh}c_{gghh}+a_{20}\cdot
c_{t}^{2}c_{ggh}^{2}$ $\displaystyle+$ $\displaystyle a_{21}\cdot
c_{tt}c_{ggh}^{2}+a_{22}\cdot c_{ggh}^{3}c_{hhh}+a_{23}\cdot
c_{ggh}^{2}c_{gghh}\;.$

Since the first and the third truncation options of eq. (7) are expansions at
cross section level, and the fourth option is the direct translation from HEFT
to SMEFT, for those cases the application of the translation of Table 1,
including all terms at the desired order in inverse $\Lambda$, is sufficient.
In the case of the second truncation option, some of the grids can be reused
as well, but the determination of the coefficients needs more care, as there
are additional combinations. Note that we do not include RGE running of the
couplings as we only consider NLO QCD corrections to the amplitude, which
factorise.

Figure 1:  Heat maps showing the dependence of the cross section on the
couplings $C_{H}$, $C_{uH}$ (left) and $C_{H}$, $C_{H,kin}$ (right) with
$\Lambda=1$ TeV for different truncation options. Top: option 1 (linear
dim-6), bottom: option 2 (quadratic dim-6). The white areas denote regions in
parameter space where the corresponding cross section would be negative.

In Fig. 1 we show that the results for the total cross sections (normalised to
the SM case) are substantially different between option 1 (linear dim-6, top)
and option 2 (quadratic dim-6, bottom). The white areas come from the fact
that taking into account only linear dim6-contributions leads to negative
cross sections over large parts of the parameter space. Furthermore, in the
linear dim-6 case, there appears to be a completely flat direction in the
observed parameter range for a combined variation of the respective Wilson
coefficients in the diagrams. Flat directions are apparent in option 2 as
well, however they correspond to an elliptic shape of equipotential lines due
to the quadratic terms in the cross section.

###  3.2 Investigation of truncation effects for the Higgs boson pair
invariant mass distribution

Now we turn to differential results, showing the effects of the different
truncation options on the Higgs boson pair invariant mass distribution
$m_{hh}$. We present results at two benchmark points, given in Table 2, which
were derived analogously to [22] based on an analysis of characteristic shapes
of the $m_{hh}$ distribution, but with the inclusion of current experimental
constraints. The upper panels show results for $\Lambda=1$ TeV, the lower
panels show results for the same point for $\Lambda=2$ TeV, for the different
truncation options. One can clearly see that (a) the negative differential
cross section values in the linear dim-6 case (blue) indicate that parameter
points in anomalous coupling parameter space which are valid in HEFT can lead,
upon naive translation, to parameter points for which the SMEFT expansion is
not valid, (b) destructive interference between different parts of the
amplitude (e.g. box- and triangle-type diagams) can be enhanced or diminished
depending on the truncation option, (c) increasing $\Lambda$ reduces the
differences between the results as they are smaller deformations of the SM
parameter space. In addition, we observe that the contribution from the
interference of double dim-6 operator insertions with the SM appears to be
subdominant in the example of benchmark point $1^{\ast}$, as can be seen by
comparing truncation option 2 (orange) with option 3 (red), the latter
including the double operator insertions. We also should point out that the
difference between HEFT (cyan) and SMEFT with truncation option 4 (green) is
due to the scale dependence of $\alpha_{s}$, coming from the definition of
$C_{HG}$ in the Warsaw basis, see Table 1.

| benchmark
---
(${}^{\ast}=$ modified)
$c_{hhh}$ | $c_{t}$ | $c_{tt}$ | $c_{ggh}$ | $c_{gghh}$ | $C_{H,\textrm{kin}}$ | $C_{H}$ | $C_{uH}$ | $C_{HG}$ | $\Lambda$
SM | $1$ | $1$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ |  $1\;$TeV
$1^{\ast}$ | $5.105$ | $1.1$ | $0$ | $0$ | $0$ | $4.95$ | $-6.81$ | $3.28$ | $0$ |  $1\;$TeV
$6^{\ast}$ | $-0.684$ | $0.9$ | $-\frac{1}{6}$ | $0.5$ | $0.25$ | $0.561$ | $3.80$ | $2.20$ | $0.0387$ |  $1\;$TeV
Table 2: Benchmark points used for the invariant mass distributions. The
benchmark points were derived analogously to [22], but are slightly modified
compared to the ones given in [22], to take into account current experimental
constraints. The value of $C_{HG}$ is determined using
$\alpha_{s}(m_{Z})=0.118$.

Figure 2:  Differential cross sections for the Higgs boson pair invariant
mass. Top row: $\Lambda=1$ TeV, bottom row: $\Lambda=2$ TeV. Left: benchmark
point $1^{\ast}$, right: benchmark point $6^{\ast}$ of Table 2. The gray and
orange bands show the uncertainty from 3-point scale variations
$\mu_{R}=\mu_{F}=c\cdot m_{hh}/2$, with $c\in\\{\frac{1}{2},1,2\\}$, for the
SM and SMEFT $\sigma_{(SM+dim6)\times(SM+dim6)}$, respectively.

##  4 Summary

We have presented NLO QCD corrections to Higgs boson pair production in
combination with a Standard Model Effective Field Theory (SMEFT)
parametrisation of effects of physics beyond the Standard Model. The
calculation has been implemented into the GoSam+POWHEG Monte Carlo program
framework in a way which allows to choose different options for the truncation
of the EFT series and to compare to results in (non-linear) Higgs Effective
Field Theory (HEFT). The results show that a naive translation between HEFT
and SMEFT has pitfalls and that the various truncation options can lead to
large differences in the theory predictions.

### Acknowledgements

We would like to thank Stephen Jones, Matthias Kerner and Ludovic Scyboz for
collaboration in the $ggHH$@NLO project and Gerhard Buchalla for useful
discussions. Special thanks to Ludovic for providing up-to-date benchmark
points. This research was supported by the Deutsche Forschungsgemeinschaft
(DFG, German Research Foundation) under grant 396021762 - TRR 257.

## References

## References

  * [1] Borowka S, Greiner N, Heinrich G, Jones S P, Kerner M, Schlenk J, Schubert U and Zirke T 2016 Phys. Rev. Lett. 117 012001 [Erratum: Phys.Rev.Lett. 117, 079901 (2016)] (Preprint 1604.06447)
  * [2] Borowka S, Greiner N, Heinrich G, Jones S P, Kerner M, Schlenk J and Zirke T 2016 JHEP 10 107 (Preprint 1608.04798)
  * [3] Baglio J, Campanario F, Glaus S, Mühlleitner M, Spira M and Streicher J 2019 Eur. Phys. J. C 79 459 (Preprint 1811.05692)
  * [4] Baglio J, Campanario F, Glaus S, Mühlleitner M, Ronca J, Spira M and Streicher J 2020 JHEP 04 181 (Preprint 2003.03227)
  * [5] Grazzini M, Heinrich G, Jones S, Kallweit S, Kerner M, Lindert J M and Mazzitelli J 2018 JHEP 05 059 (Preprint 1803.02463)
  * [6] Chen L B, Li H T, Shao H S and Wang J 2020 Phys. Lett. B 803 135292 (Preprint 1909.06808)
  * [7] Chen L B, Li H T, Shao H S and Wang J 2020 JHEP 03 072 (Preprint 1912.13001)
  * [8] Baglio J, Campanario F, Glaus S, Mühlleitner M, Ronca J and Spira M 2021 Phys. Rev. D 103 056002 (Preprint 2008.11626)
  * [9] Heinrich G, Lang J and Scyboz L 2022 JHEP 08 079 (Preprint 2204.13045)
  * [10] Grzadkowski B, Iskrzynski M, Misiak M and Rosiek J 2010 JHEP 10 085 (Preprint 1008.4884)
  * [11] Brivio I and Trott M 2019 Phys. Rept. 793 1–98 (Preprint 1706.08945)
  * [12] Buchalla G, Catá O and Krause C 2014 Phys. Lett. B 731 80–86 (Preprint 1312.5624)
  * [13] Krause C G 2016 Higgs Effective Field Theories - Systematics and Applications Ph.D. thesis Munich U. (Preprint 1610.08537)
  * [14] Buchalla G, Capozi M, Celis A, Heinrich G and Scyboz L 2018 JHEP 09 057 (Preprint 1806.05162)
  * [15] Arzt C, Einhorn M B and Wudka J 1995 Nucl. Phys. B 433 41–66 (Preprint hep-ph/9405214)
  * [16] Brivio I et al. 2022 (Preprint 2201.04974)
  * [17] Heinrich G, Jones S P, Kerner M, Luisoni G and Scyboz L 2019 JHEP 06 066 (Preprint 1903.08137)
  * [18] Heinrich G, Jones S P, Kerner M and Scyboz L 2020 JHEP 10 021 (Preprint 2006.16877)
  * [19] Cullen G et al. 2014 Eur. Phys. J. C 74 3001 (Preprint 1404.7096)
  * [20] Alioli S, Nason P, Oleari C and Re E 2010 JHEP 06 043 (Preprint 1002.2581)
  * [21] Degrande C, Duhr C, Fuks B, Grellscheid D, Mattelaer O and Reiter T 2012 Comput. Phys. Commun. 183 1201–1214 (Preprint 1108.2040)
  * [22] Capozi M and Heinrich G 2020 JHEP 03 091 (Preprint 1908.08923)


# Interactive and Incremental Learning of Spatial Object Relations from Human
Demonstrations

Rainer Kartmann ∗ and Tamim Asfour

###### Abstract

Humans use semantic concepts such as spatial relations between objects to
describe scenes and communicate tasks such as “Put the tea to the right of the
cup” or “Move the plate between the fork and the spoon.” Just as children,
assistive robots must be able to learn the sub-symbolic meaning of such
concepts from human demonstrations and instructions. We address the problem of
incrementally learning geometric models of spatial relations from few
demonstrations collected online during interaction with a human. Such models
enable a robot to manipulate objects in order to fulfill desired spatial
relations specified by verbal instructions. At the start, we assume the robot
has no geometric model of spatial relations. Given a task as above, the robot
requests the user to demonstrate the task once in order to create a model from
a single demonstration, leveraging cylindrical probability distribution as
generative representation of spatial relations. We show how this model can be
updated incrementally with each new demonstration without access to past
examples in a sample-efficient way using incremental maximum likelihood
estimation, and demonstrate the approach on a real humanoid robot.

##  1 Keywords:

Cognitive Robotics, Learning Spatial Object Relations, Semantic Scene
Manipulation, Incremental Learning, Interactive Learning

##  2 Introduction

While growing up, humans show impressive capabilities to continually learn
intuitive models of the physical world as well as concepts which are essential
to communicate and interact with others. While an understanding of the
physical world can be created through exploration, concepts such as the
meaning of words and gestures are learned by observing and imitating others.
If necessary, humans give each other explicit explanations and demonstrations
to purposefully help the learner improve their understanding of a specific
concept. These can be requested by the learner after acknowledging their
incomplete understanding, or by the teacher when observing a behavior that
does not match their internal model (Grusec, 1994). Assistive robots that
naturally interact with humans and support them in their daily lives should be
equipped with such continual and interactive learning abilities, allowing them
to improve their current models and learn new concepts from their users
interactively and incrementally.

One important class of concepts children need to learn are the meanings of
spatial prepositions such as _right of_ , _above_ or _close to_. Such
prepositions define geometrical relationships between spatial entities
(O’Keefe, 2003), such as objects, living beings or conceptual areas, which are
referred to as _spatial relations_ (Stopp et al., 1994; Aksoy et al., 2011;
Rosman and Ramamoorthy, 2011). Spatial relations play an important role in
communicating manipulation tasks in natural language, e. g., in “Set the table
by placing a plate _on_ the table, the fork to _the left_ of the plate, and
the knife to the _right of_ the plate.” By abstracting from precise metric
coordinates and the involved entities’ shapes, spatial relations allow the
expression of tasks on a semantic, symbolic level. However, a robot performing
such a task must be able to derive subsymbolic placing positions that are
needed to parameterize actions. Such _signal-to-symbol gap_ remains a grand
challenge in cognitive robotics (Krüger et al., 2011). Just like a child, a
robot should be able to learn such mapping of spatial object relations from
demonstrations provided by humans.

In this work, we consider a robot that has no prior knowledge about the
geometric meaning of any spatial relations yet. When given the task to
manipulate a scene to fulfill a desired spatial relation between two or more
objects, such as a cup _in front of_ a bottle, the robot should request a
demonstration from the user if it has no model of the spatial relation or if
its current model is insufficient (Fig. 1). Similarly, the robot should be
able to receive corrections from the human after executing the task. Finally,
having received a new demonstration, the robot should be able to derive a
model of the spatial relation from the very first sample and subsequently
update its model incrementally with each new demonstration, i. e., without the
need to retrain the model with all previously observed demonstrations (Losing
et al., 2018).

These goals pose hard requirements for the underlying representation of
spatial relations and the cognitive system as a whole. The robot needs to
inform the user in case it cannot perform the task by asking for help while
maintaining an internal state of the interaction. In addition, the robot will
only receive very sparse demonstrations – every single demonstration should be
used to update the robot’s model of the spatial relation at hand. As a
consequence, we require a very sample-efficient representation that can be
constructed from few demonstrations and incrementally updated with new ones.

Figure 1:  Incremental learning of spatial relations from human
demonstrations.

Obtaining a sample-efficient representation can be achieved by introducing
bias about the hypothesis space (Mitchell, 1982). Bias reduces the model’s
capacity, and thereby its potential variance, but more importantly, also
reduces the amount of data required to train the model. This effect is also
known as the bias-variance tradeoff. Compared to a partially model-driven
approach with a stronger bias, a purely data-driven black-box model can offer
superfluous capacity, slowing down training. Bias can be introduced by
choosing a model whose structure matches that of the problem at hand. For the
problem of placing objects according to desired spatial relations, we proposed
to represent spatial relations as parametric probability distributions defined
in cylindrical coordinates in our previous work (Kartmann et al., 2020, 2021),
observing that capturing relative positions in terms of horizontal distance,
horizontal direction, and vertical distance closely matches the notions of
common spatial prepositions.

An interesting consideration arises when learning a model from a single
demonstration. A single example does not hold any variance; consequently, the
learner’s only option is to reproduce this demonstration as closely as
possible. When receiving more examples, the learner can add variance to its
model, thus increasing its ability to adapt to more difficult scenarios. This
principle is leveraged by version space algorithms (Mitchell, 1982), which
have been used in robotics to incrementally learn task precedence (Pardowitz
et al., 2005). While our approach is not a version space algorithm per se, it
behaves similarly with respect to incremental learning: The model starts with
no variance and acquires more variance with new demonstrations, which
increases its ability to handle more difficult scenarios such as cluttered
scenes.

We summarize our contributions as follows:

  1. 1.

We present an approach for a robot interacting with a human that allows the
robot to (a) request demonstrations from a human for how to manipulate the
scene according to desired spatial relations specified by a language
instruction if the robot has not sufficient knowledge about how to perform
such a task,¸ as well as (b) to use corrections after each execution to
continually improve its internal model about spatial relations.

  2. 2.

We show how our representation of spatial relations based on cylindrical
distributions proposed in (Kartmann et al., 2021) can be incrementally learned
from few demonstrations based only on the current model and a new
demonstration using incremental maximum likelihood estimation.

We evaluate our approach in simulation and real world experiments on the
humanoid robot ARMAR-6 (Asfour et al., 2019)111https://youtu.be/x6XKUZd_SeE.

##  3 Related Work

In this section, we discuss related work in the areas of using spatial
relations in the context of human-robot interaction and the incremental
learning of spatial relation models.

###  3.1 Spatial Relations in Human-Robot Interaction and Learning from
Demonstrations

Spatial relations have been used to enrich language-based human-robot
interaction. Many works focus on using spatial relations to resolve referring
expressions identifying objects (Bao et al., 2016; Tan et al., 2014) as well
as locations (Fasola and Matarić, 2013; Tellex et al., 2011) for manipulation
and navigation. In these works, the robot passively tries to parse the given
command or description without querying the user for more information if
necessary. As a consequence, if the sentence is not understood correctly, the
robot is unable to perform the task. Other works use spatial relations in
dialog to resolve such ambiguities. Hatori et al. (2018) query for additional
expressions if the resolution did not score a single object higher by a margin
than all other objects. Shridhar and Hsu (2018); Shridhar et al. (2020) and
Doğan et al. (2022) formulate clarification questions describing candidate
objects using, among others, spatial relations between them. These works use
spatial relations in dialog to identify objects which the robot should
interact with. However, our goal is to perform a manipulation task defined by
desired spatial relations between objects.

A special form of human-robot interaction arises in the context of robot
learning from human demonstrations (Ravichandar et al., 2020). There, the goal
is to teach the robot a new skill or task instead of performing a command.
Spatial relations have been used in such settings to specify parameters of
taught actions. Similar to the works above, given the language command and
current context, Forbes et al. (2015) resolve referring expressions to
identify objects using language generation to find the most suitable
parameters for a set of primitive actions. Prepositions from natural language
commands are incorporated as action parameters in a task representation based
on part-of-speech tagging by Nicolescu et al. (2019). These works focus on
learning the structure of a task including multiple actions and their
parameters. However, action parameters are limited to a finite set of values,
and spatial relations are implemented as fixed position offsets. In contrast,
our goal is learning continuous, geometric models of the spatial relations
themselves.

###  3.2 Learning Spatial Relation Models

Many works have introduced models to classify existing spatial relations
between objects to improve scene understanding (Rosman and Ramamoorthy, 2011;
Sjöö and Jensfelt, 2011; Fichtl et al., 2014; Yan et al., 2020) and human
activity recognition (Lee et al., 2020; Zampogiannis et al., 2015; Dreher et
al., 2020). These models are either hand-crafted or are not learned
incrementally. In contrast, our models are learned incrementally from
demonstrations collected during interaction. Few works consider models for
classifying spatial relations which could be incrementally updated with new
data. Mees et al. (2020) train a neural network model to predict a pixel-wise
probability map of placement positions given a camera image of the scene and
an object to be placed according to a spatial relation. However, their models
are not trained incrementally, and training neural networks incrementally is,
in general, not trivial (Losing et al., 2018). In an earlier work, Mees et al.
(2017) propose a metric learning approach to model spatial relations between
two objects represented by point clouds. The authors learn a distance metric
measuring how different the realized spatial relations in two scenes are.
Recognizing the spatial relation in a given scene is then reduced to a search
of known examples that are similar to the given scene according to the learned
metric. Once the metric is learned and kept fixed, this approach inherently
allows adding new samples to the knowledge base, which potentially changes the
classification of new, similar scenes. However, their method requires storing
all encountered samples to keep a notion of known spatial relations, while our
models can be updated incrementally with a limited budget of stored examples
(Losing et al., 2018). Mota and Sridharan (2018) follow a related idea to
learn classification models of spatial relations incrementally. They encode
spatial relations as 1D and 2D histograms over the relative distances or
directions (encoded as azimuth and elevation angles), of points in two point
clouds representing two objects. These histograms can be incrementally updated
by merely adding a new observation to the current frequency bins.

However, all of these models are _discriminative_ , i. e., they determine the
existing relations between two objects in the current scene. In contrast, our
goal is to _generate_ a new target scene given desired spatial relations.
While discriminative models can still be applied by exhaustively sampling the
solution space (e. g., possible locations of manipulated objects), classifying
the relations in these candidates and choosing one that contains the desired
relation, we believe that it is more effective to directly learn and apply
generative geometric models of spatial relations. In our previous works, we
introduced generative representations of 3D spatial relations in the form of
parametric probability distributions over placing positions (Kartmann et al.,
2021). These probabilistic models can be sampled to obtain suitable placing
positions for an object to fulfill a desired spatial relation to one or
multiple reference objects. We have shown how these models can be learned from
human demonstrations which were collected offline. In this work, we show how
demonstrations can be given interactively and how the models can be updated in
a fully incremental manner, i. e., relying solely on the current model and a
new demonstration.

##  4 Problem Formulation and Concept

In the following, we formulate the problem of interactive and incremental
learning of spatial relation models from human demonstrations and introduce
the general concept of the work. In Section 4.1, we summarize the actual task
of semantic scene manipulation which the robot has to solve. In Section 4.2,
we describe the semantic memory system as part of the entire cognitive control
architecture used on the robot. In Section 4.3, we formulate the problem of
incremental learning of spatial relation models. In Section 4.4, we describe
the envisioned human-robot interaction task and explain how we approached each
subproblem in Section 5.

###  4.1 Semantic Scene Manipulation

We consider the following problem: Given a scene with a set of objects and a
language command specifying spatial relations between these objects, the robot
must transfer the initial scene to a new scene fulfilling the specified
relations by executing an action of a set of actions. We denote points in time
as ${t_{0}},\dots,{t_{k}},\ldots\ \left({t_{k}}\in\mathbb{R}\right)$ that can
be viewed as events where the robot is given a command or makes an
observation. The scene model is part of the robot’s working memory and
contains the current configuration

$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}=\left\\{\boldsymbol{\mathbf{P}}_{1}^{{\left({t_{k}}\right)}},\dots,\boldsymbol{\mathbf{P}}_{n}^{{\left({t_{k}}\right)}}\right\\}$
(1)

of $n^{{\left({t_{k}}\right)}}$ objects at time ${{t_{k}}}$, where
$\boldsymbol{\mathbf{P}}_{i}^{{\left({t_{k}}\right)}}\in\mathrm{SE(3)}$ is the
pose of object $i$ with position
$\boldsymbol{\mathbf{p}}_{i}^{{\left({t_{k}}\right)}}\in\mathbb{R}^{3}$ and
orientation
$\boldsymbol{\mathbf{Q}}_{i}^{{\left({t_{k}}\right)}}\in\mathrm{SO(3)}$.
$\mathrm{SE(3)}$ and $\mathrm{SO(3)}$ denote the special Euclidean group and
special orthogonal group, respectively. The desired relation

$R^{*}=\left({s^{*}},{u^{*}},{\mathcal{V}^{*}}\right)\leftarrow\mathrm{ground}\\!\left(C\right)$
(2)

is obtained by parsing and grounding the natural language command $C$, i. e.,
extracting the phrases referring to objects and relations and mapping them to
the respective entities in the robot’s working and long-term memories222Please
refer to Kartmann et al. (2021) for more details on the natural language
understanding part.. This desired relation consists of a symbol
${s^{*}}\in\mathcal{S}$ describing the identity of the relation, a _target_
object ${u^{*}}\in\left\\{1,\dots,n^{{\left({t_{k}}\right)}}\right\\}$ and a
set of _reference_ objects
${\mathcal{V}^{*}}\subseteq\left\\{1,\dots,n^{{\left({t_{k}}\right)}}\right\\}\setminus\left\\{{u^{*}}\right\\}$.
$\mathcal{S}$ is the set of known spatial relation symbols. The robot’s task
is to place the target object ${u^{*}}$ at an appropriate pose
$\boldsymbol{\mathbf{P}}^{{\left({t_{k+1}}\right)}}_{u^{*}}$ which fulfills
the desired spatial relation $R^{*}$. We aim at keeping the object’s original
orientation, therefore this task is reduced to finding a suitable position
$\boldsymbol{\mathbf{p}}^{{\left({t_{k+1}}\right)}}_{u^{*}}$.

Our approach to finding suitable placing positions is based on a generative
model $G$ of spatial relations. This model is able to generate suitable target
object positions fulfilling a relation $R=\left(s,u,\mathcal{V}\right)$ based
on the current scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}$ and semantic
object information $\mathcal{O}$ (object names and geometry, see 5 below),
that is formally

$\boldsymbol{\mathbf{p}}_{u}^{{\left({t_{k+1}}\right)}}\sim
G_{s}\\!\left(u,\mathcal{V},\mathcal{O},\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}\right).$
(3)

The generative models $G_{s}$ of spatial relations $s$ can be learned from
human demonstrations. In our previous work, we recorded human demonstrations
for each spatial relation using real objects and learned the generative models
$G_{s}$ offline. Each demonstration consisted of the initial scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}$, a desired
relation $R^{*}=\left({s^{*}},{u^{*}},{\mathcal{V}^{*}}\right)$ verbalized as
a language command for the human demonstrator, and the resulting scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}$ created by the
demonstrator by manipulating the initial scene. Therefore, each demonstration
has the form

$D=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}},R,\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}\right),\quad
R=\left(s,u,\mathcal{V}\right)$ (4)

and can be used to learn the generative model $G_{s}$ of the relation $s$. In
contrast to the previous work, in this work we consider the problem of
interactively collecting samples by querying demonstrations from the user and
incrementally updating the generative models of the spatial relations in the
robot’s memory with each newly collected sample.

###  4.2 Robot Semantic Memory

The robot’s semantic memory consists of two parts: the _prior knowledge_ , i.
e., information defined a-priori by a developer, and the _long-term memory_ ,
i. e., experience gathered by the robot itself (Asfour et al., 2017). In our
scenario, the prior knowledge contains semantic information about $N$ known
objects,

$\mathcal{O}=\left\\{O_{i}\right\\}_{i=1}^{N}=\left\\{\left(g_{i},\eta_{i}\right)\right\\}_{i=1}^{N},$
(5)

including object names $\eta_{i}$ and 3D models $g_{i}$, as well as names of
spatial relations, so that language phrases referring to both objects and
relations can be grounded in entities stored in the robot’s memory by natural
language understanding as indicated in 2. We assume no prior knowledge about
the geometric meaning of the relations. Instead, this geometric meaning is
learned by the robot during interaction in a continual manner, and is thus
part of the long-term memory. In other words, the long-term memory contains
generative models $G_{s}^{{{\left({t_{k}}\right)}}}$ of spatial relations
$s\in\mathcal{S}^{{\left({t_{k}}\right)}}$ representing the robot’s current
understanding of spatial relations at time ${{t_{k}}}$,

${\mathcal{G}}^{{\left({t_{k}}\right)}}=\left\\{\left.G_{s}^{{\left({t_{k}}\right)}}\right|s\in\mathcal{S}^{{{\left({t_{k}}\right)}}}\right\\},$
(6)

where $\mathcal{S}^{{\left({t_{k}}\right)}}\subseteq\mathcal{S}$ is the set of
spatial relations for which the robot has learned a generative model at
${{t_{k}}}$. These models are based on samples collected from human
demonstrations ${\mathcal{D}}$, which are contained in the long-term memory as
well:

$\displaystyle{\mathcal{D}}^{{\left({t_{k}}\right)}}$
$\displaystyle=\left\\{\mathcal{D}_{s}^{{\left({t_{k}}\right)}}\left|\
s\in\mathcal{S}^{{\left({t_{k}}\right)}}\right.\right\\},$ (7)
$\displaystyle\mathcal{D}_{s}^{{\left({t_{k}}\right)}}$
$\displaystyle=\left\\{D_{sj}\right\\}_{j=1}^{m_{s}^{{\left({t_{k}}\right)}}}=\left\\{\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k_{sj}}}\right)}}_{sj},R_{sj},\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k_{sj}+1}}\right)}}_{sj}\right\\}_{j=1}^{m_{s}^{{\left({t_{k}}\right)}}},$

where $\mathcal{D}_{s}^{{\left({t_{k}}\right)}}$ is the set of
$m_{s}^{{\left({t_{k}}\right)}}$ collected samples of spatial relation $s$ at
time ${{t_{k}}}$, and ${t_{k_{sj}}},{t_{k_{sj}+1}}$ refer to the time a sample
was collected. At the beginning, the robot’s long-term memory is empty,
therefore

${\mathcal{G}}^{{\left({t_{0}}\right)}}={\mathcal{D}}^{{\left({t_{0}}\right)}}=\mathcal{S}^{{\left({t_{0}}\right)}}=\emptyset.$
(8)

###  4.3 Learning Spatial Relations: Batch vs Incremental

During interactions, the robot will collect new samples from human
demonstrations. When the robot receives a new demonstration for relation $s$
at time ${t_{k}}$ $(k>0)$, it first stores the new sample $D_{s}$ in its long-
term memory:

$\mathcal{D}_{s}^{{{\left({t_{k}}\right)}}}\leftarrow\left\\{D_{s}\right\\}\cup\begin{cases}\mathcal{D}_{s}^{{\left({t_{k-1}}\right)}},&s\in\mathcal{S}^{{\left({t_{k-1}}\right)}}\\\
\emptyset,&\text{else}\end{cases}$ (9)

Afterwards, the robot may query its long-term memory for the relevant samples
$\mathcal{D}_{s}^{{{\left({t_{k}}\right)}}}$ collected to date and use them to
update its model $G_{s}$ of $s$:

$G_{s}^{{\left({t_{k}}\right)}}\leftarrow\mathrm{update}\\!\left(G_{s}^{{\left({t_{k-1}}\right)}},\mathcal{D}_{s}^{{{\left({t_{k}}\right)}}}\right)$
(10)

Note that such an incremental model update with each new sample requires a
very sample-efficient representation. We refer to 10 as the _batch update
problem_. The model is expected to adapt meaningfully to each new sample, but
all previous samples are needed for the model update. In the machine learning
community, incremental learning can be defined as proposed by Losing et al.
(2018):

> “We define an incremental learning algorithm as one that generates on a
> given stream of training data $s_{1},s_{2},\dots,s_{t}$ a sequence of models
> $h_{1},h_{2},\dots,h_{t}$. In our case […]
> $h_{i}:\mathbb{R}^{n}\rightarrow\\{1,\dots,C\\}$ is a model function solely
> depending on $h_{i-1}$ and the recent $p$ examples $s_{i},\dots,s_{i-p}$,
> with $p$ being strictly limited.”

Comparing this definition to 10, it becomes clear that it is not an instance
of incremental learning in this sense, as the number of samples
$|\mathcal{D}_{s}|$ in the memory is not bounded by a constant. This raises
the question of whether representations of spatial relations in the form of
cylindrical distributions as proposed in our previous work can be learned in a
truly incremental way using a limited budget of stored samples. In this work,
we further investigate the question of incremental learning of spatial
relations without even storing _any_ samples, i. e., solely based on the
current model $G_{s}^{{\left({t_{k-1}}\right)}}$ and the latest sample
$D_{s}$, forming the _incremental update problem_ :

$G_{s}^{{\left({t_{k}}\right)}}\leftarrow\mathrm{update}\\!\left(G_{s}^{{\left({t_{k-1}}\right)}},D_{s}\right)$
(11)

###  4.4 Interactive Learning of Spatial Relations

We now describe a scenario of a robot interacting with a human where the human
gives a semantic manipulation command to the robot while the robot can gather
new samples from human demonstrations. The scheme is illustrated in Fig. 2.
The procedure can be described as follows:

Figure 2:  Scheme for a robot interacting with a human to learn geometric
models of spatial relations.

  1. 1.

The human gives a command to the robot at ${{t_{k}}}$, specifying a spatial
relation $R^{*}=\left({s^{*}},{u^{*}},{\mathcal{V}^{*}}\right)$ as in 2.

  2. 2.

The robot observes the current scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}$ and plans the
execution of the given task using its current model
$G_{{s^{*}}}^{{\left({t_{k}}\right)}}$. Planning may be successful or fail due
to different reasons. Assuming the task given by the user is solvable, the
failure can be attributed to an insufficient model.

  3. 3.

Depending on the outcome of 2.:

    1. 3a.

_Planning is successful:_ The robot found a suitable placing position and
executes the task by manipulating the scene. If the execution was successful
and the user is satisfied, the interaction is finished.

    2. 3b.

_Planning fails:_ The robot’s current model is insufficient, thus, it queries
the human for a demonstration of the task.

  4. 4.

The human was queried for a demonstration (3b.) or wants to correct the
robot’s execution (3a.). In both cases, the human performs the task by
manipulating the scene.

  5. 5.

The human signals that the demonstration is complete by giving a speech cue
(e. g., “Put it here”) to the robot.

  6. 6.

When receiving the speech cue, the robot observes the changed scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}$, creates a new
sample
$D=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}},R^{*},\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}\right)$,
stores the sample in the long-term memory and updates its model to obtain the
new model $G_{{s^{*}}}^{{\left({t_{k+1}}\right)}}$ as described in Section
4.3.

##  5 Methods and Implementation

To solve the underlying task of semantic scene manipulation, we rely on our
previous work which is briefly described in Section 5.1. We outline the
implementation of the robot’s semantic memory in Section 5.2. We describe how
each new sample is used to update a spatial relation’s model in Section 5.3.
Finally, we explain how we implement the defined interaction scenario in
Section 5.4.

###  5.1 3D Spatial Relations as Cylindrical Distributions

In (Kartmann et al., 2021), we proposed a model of spatial relations based on
a cylindrical distribution $\mathcal{C}$

$\left(r,\phi,h\right)\sim\mathcal{C}\\!\left(\theta\right),\quad\theta=\left(\theta_{rh},\theta_{\phi}\right)$
(12)

over the cylindrical coordinates radius $r\in\mathbb{R}_{\geq 0}$, azimuth
$\phi\in\left[-\pi,\pi\right]$ and height $h\in\mathbb{R}$. The radius and
height follow a joint Gaussian distribution $\mathcal{N}$, while the azimuth,
as an angle, follows a von Mises distribution $\mathcal{M}$ (which behaves
similar to a Gaussian distribution but is defined on the unit circle),

$\left(r,h\right)\sim\mathcal{N}\\!\left(\theta_{rh}\right),\quad\theta_{rh}=\left(\boldsymbol{\mathbf{\mu}}_{rh},\boldsymbol{\mathbf{\Sigma}}_{rh}\right),\qquad\phi\sim\mathcal{M}\\!\left(\theta_{\phi}\right),\quad\theta_{\phi}=\left(\mu_{\phi},\kappa_{\phi}\right)$
(13)

so the joint probability density function of $\mathcal{C}$ is given by

$p_{\theta}\\!\left(r,\phi,h\right)=p_{\theta_{rh}}\\!\left(r,h\right)\cdot
p_{\theta_{\phi}}\\!\left(\phi\right)$ (14)

with $p_{\theta_{\phi}}\\!\left(\phi\right)$ and
$p_{\theta_{rh}}\\!\left(r,h\right)$ being the respective probability density
functions of the distributions in 13. We claim that cylindrical coordinates
are a suitable space for representing spatial relations as they inherently
encode horizontal distance (_close to_ , _far from_), direction (_left of_ ,
_behind_) and vertical distance (_above_ , _below_), which are the qualities
many spatial relations used by humans are defined by. Therefore, we leverage a
cylindrical distribution as a distribution over suitable placing positions of
a target object relative to one or more reference objects. The corresponding
cylindrical coordinate system is centered at the bottom-projected centroid of
the axis-aligned bounding box enclosing all reference objects and scales with
the size of that bounding box. By defining the cylindrical coordinate system
based on the reference objects’ joint bounding box, we can apply the same
spatial relation models to single and multiple reference objects. Especially,
this allows considering spatial relations that inherently involve multiple
reference objects (e. g., _between_ , _among_).

###  5.2 Initialization of Robot’s Semantic Memory

We build our implementation of the cognitive architecture of our robot in
ArmarX (Vahrenkamp et al., 2015; Asfour et al., 2017). The architecture
consists of three-layer for 1) low-level sensorimotor control, 2) high-level
for semantic reasoning and task planning and 3) mid-level as a memory system
and mediator between the symbolic high-level and subsymbolic low-level. The
memory system contains segments for different modalities, such as object
instances or text recognized from speech. A segment contains any number of
entities that can receive new observations and thus evolve over time. Here, we
define three new segments:

  * •

The _sample segment_ implements the set of human demonstrations
${\mathcal{D}}$ in 7. It contains one entity for each
$s\in\mathcal{S}^{{\left({t_{k}}\right)}}$, each one representing “all
collected samples of relation $s$.” New samples are added as new observations
to the respective entity. Thus, $\mathcal{D}_{s}^{{\left({t_{k}}\right)}}$ in
7 is obtained by querying the sample segment for all observations of entity
$s$.

  * •

The _spatial relation segment_ contains the robot’s knowledge about spatial
relations. There is one entity per $s\in\mathcal{S}$, which holds the
information from prior knowledge such as the relations’ names for language
grounding and verbalization. In addition, each entity contains the current
geometric model $G_{s}^{{\left({t_{k}}\right)}}=\theta$ with $\theta$ as in
12.

  * •

The _relational command_ segment contains the semantic manipulation tasks
$R^{*}=\left({s^{*}},{u^{*}},{\mathcal{V}^{*}}\right)$ extracted from language
commands 2. The latest observation is the current or previous task.

In the beginning, the sample segment and the relational command segment are
initialized to be empty, i. e., there are no collected samples yet and no
command has been given yet. The spatial relation segment is partially
initialized from prior knowledge. However, in accordance with the sample
segment, the entities contain no geometric model $G_{s}$ yet, i. e.
$G_{s}^{{\left({t_{0}}\right)}}=\emptyset$.

###  5.3 Incremental Learning of Spatial Relations

Now, we describe how the batch update problem can be solved using cylindrical
distributions. Then, we show how the same mathematical operations can be
performed incrementally, i. e., without accessing past samples.

Batch Updates Due to the simplicity of our representation of spatial
relations, updating the geometric model of a spatial relation, i. e., its
cylindrical distribution, is relatively straightforward. To implement the
batch update 10, we query all samples of the relation of interest collected so
far, perform Maximum Likelihood Estimation (MLE) to obtain the cylindrical
distribution’s parameters,

$G_{s}^{{\left({t_{k}}\right)}}=\theta_{s}^{{\left({t_{k}}\right)}}\leftarrow\mathrm{MLE}\\!\left(\mathcal{D}_{s}^{{{\left({t_{k}}\right)}}}\right),$
(15)

and update the spatial relation segment.

A cylindrical distribution is a combination of a bivariate Gaussian
distribution over $(r,h)$ and a von Mises distribution over $\phi$, see 13.
Hence, to perform the MLE of a cylindrical distribution in 15, two independent
MLEs are performed for the Gaussian distribution
$\mathcal{N}\\!\left(\theta_{rh}\right)$ and the von Mises distribution
$\mathcal{M}\\!\left(\theta_{\phi}\right)$, respectively,

$\theta_{rh}^{*}\leftarrow\mathrm{MLE}_{\mathcal{N}}\\!\left(\left\\{r_{j},h_{j}\right\\}_{j=1}^{m_{s}^{{\left({t_{k}}\right)}}}\right),\qquad\theta_{\phi}^{*}\leftarrow\mathrm{MLE}_{\mathcal{M}}\\!\left(\left\\{\phi_{j}\right\\}_{j=1}^{m_{s}^{{\left({t_{k}}\right)}}}\right),$
(16)

where $\left(r_{j},\phi_{j},h_{j}\right)$ are the cylindrical of sample
$D_{j}\in\mathcal{D}_{s}^{{{\left({t_{k}}\right)}}}$ (see (Kartmann et al.,
2021) for more details).

Updating the model with each new sample requires a representation that can be
generated from few examples, including just one, which is the case for
cylindrical distributions. However, special attention has to be paid to the
case of encountering the first sample ($|\mathcal{D}_{s}|=1$). As a single
sample holds no variance, an estimated distribution would collapse in a single
point. Note that the model’s expected behavior is well-defined: It should
reproduce the single sample when generating placing positions, which often is
a valid candidate. Because cylindrical distributions generalize to objects of
different sizes, the model can be directly applied to new scenes. The caveat
is that the model is not able to provide alternatives if this placing
candidate is, e. g., in collision with other objects. If more samples are
added ($|\mathcal{D}_{s}|\geq 2$), the model is generated using standard MLE.
The resulting distribution will have a variance according to the samples,
which allows the robot to generate different candidates for placing positions.

Technically, in order to allow deriving a generative model from a single
demonstration while avoiding special cases in the mathematical formulation, we
perform a small data augmentation step: After transforming the sample to its
corresponding cylindrical coordinates
$\boldsymbol{\mathbf{c}}=(r,\phi,h)^{\top}$, we create $n$ copies
$\boldsymbol{\mathbf{c}}_{1}^{\prime},\dots,\boldsymbol{\mathbf{c}}_{n}^{\prime}$
of the sample to which we add small Gaussian noise

$\boldsymbol{\mathbf{c}}_{i}^{\prime}\leftarrow\boldsymbol{\mathbf{c}}+\left(\varepsilon_{r},\varepsilon_{\phi},\varepsilon_{h}\right)^{\top},\quad\varepsilon_{r},\varepsilon_{\phi},\varepsilon_{h}\sim\mathcal{N}\\!\left(0,\sigma_{\varepsilon}\right)$
(17)

for $i\in\left\\{1,\dots,n\right\\}$, and perform MLE on
$\left\\{\boldsymbol{\mathbf{c}},\boldsymbol{\mathbf{c}}_{1}^{\prime},\dots,\boldsymbol{\mathbf{c}}_{n}^{\prime}\right\\}$.
This keeps the distribution concentrated on the original sample while allowing
small variance parameters to keep the mathematical formulation simple and
computations numerically stable. In our experiments, we used $n=2$ and
$\sigma_{\varepsilon}=10^{-3}$. Note that the variance created by this
augmentation step is usually negligible compared to the variance created by
additional demonstrations.

Incremental Updates Now, we present how the model update 15 can be performed
in an incremental manner, i. e., with access only to the current model and the
new sample. We follow the method by Welford (1962) for the incremental
calculation of the parameters of a univariate Gaussian distribution. By
applying this method to the multivarate Gaussian and using a similar method
for von Mises distributions to implement the MLEs 16, we can incrementally
estimate cylindrical distributions.

Welford (1962) proved that the mean $\mu$ and standard deviation $\sigma$ of a
one-dimensional series $(x_{i}),$ $x_{i}\in\mathbb{R},$ can be incrementally
calculated as follows. Let $n$ be the current time step, $\mu_{1}=x_{1}$ and
$\sigma=\frac{1}{n}\;\tilde{\sigma}$,

$\displaystyle\mu_{n}$
$\displaystyle=\left(\frac{n-1}{n}\right)\cdot\mu_{n-1}+\frac{1}{n}\cdot
x_{n}$ $\displaystyle\quad(n>1)\;,$ (18) $\displaystyle\tilde{\sigma}_{n}$
$\displaystyle=\tilde{\sigma}_{n-1}+\left(\frac{n-1}{n}\right)\cdot\left(x_{n}-\mu_{n-1}\right)^{2}$
$\displaystyle\quad(n\geq 1)\;.$ (19)

This method can be extended to a multivariate Gaussian
$\mathcal{N}\\!\left(\boldsymbol{\mathbf{\mu}},\boldsymbol{\mathbf{\Sigma}}\right)$
over a vector-valued series $\left(\boldsymbol{\mathbf{x}}_{i}\right)$ with
$\boldsymbol{\mathbf{x}}\in\mathbb{R}^{d}$. With analogous conditions as
above,

$\displaystyle\boldsymbol{\mathbf{\mu}}_{n}$
$\displaystyle=\left(\frac{n-1}{n}\right)\cdot\boldsymbol{\mathbf{\mu}}_{n-1}+\frac{1}{n}\cdot\boldsymbol{\mathbf{x}}_{n}$
$\displaystyle\quad(n>1)\;,$ (20)
$\displaystyle\tilde{\boldsymbol{\mathbf{\Sigma}}}_{n}$
$\displaystyle=\tilde{\boldsymbol{\mathbf{\Sigma}}}_{n-1}+\left(\frac{n-1}{n}\right)\cdot\left(\boldsymbol{\mathbf{x}}_{n}-\boldsymbol{\mathbf{\mu}}_{n-1}\right)\left(\boldsymbol{\mathbf{x}}_{n}-\boldsymbol{\mathbf{\mu}}_{n-1}\right)^{\top}$
$\displaystyle\quad(n\geq 1)\;.$ (21)

A von Mises distribution $\mathcal{M}\\!\left(\mu,\kappa\right)$ over angles
$\phi_{i}$ is estimated as follows (Kasarapu and Allison, 2015). Let
$\boldsymbol{\mathbf{x}}_{i}$ be the directional vectors in the 2D plane
corresponding to the directions $\phi_{i}$, and $\bar{r}$ the normalized
length of their sum,

$\displaystyle\boldsymbol{\mathbf{x}}_{i}\coloneqq$
$\displaystyle\begin{pmatrix}\cos(\phi_{i})\\\
\sin(\phi_{i})\end{pmatrix}\in\mathbb{R}^{2}\quad(i=1,\dots,n),\qquad\bar{r}\coloneqq\frac{\left\|\sum_{i=1}^{n}\boldsymbol{\mathbf{x}}_{i}\right\|}{n}\;.$
(22)

The mean angle $\mu$ is the angle corresponding to the mean direction
$\tilde{\boldsymbol{\mathbf{\mu}}}\in\mathbb{R}^{2}$,

$\displaystyle\tilde{\boldsymbol{\mathbf{\mu}}}=$
$\displaystyle\begin{pmatrix}\tilde{\mu}_{x}\\\
\tilde{\mu}_{y}\end{pmatrix}=\frac{1}{n\cdot\bar{r}}\sum_{i=1}^{n}\boldsymbol{\mathbf{x}}_{i},\qquad\mu=\tan^{-1}\\!\left(\frac{\tilde{\mu}_{y}}{\tilde{\mu}_{x}}\right).$
(23)

The concentration $\kappa$ is computed as the solution of

$\displaystyle A_{d}\\!\left(\kappa\right)=\bar{r}\;,\qquad\text{where}\
A_{d}\\!\left(\kappa\right)\coloneqq\frac{I_{d/2}(\kappa)}{I_{d/2-1}(\kappa)}\;,$
(24)

$I_{s}\\!\left(\kappa\right)$ denotes the modified Bessel function of the
first kind and order $s$, and $d$ denotes the dimension of vectors
$\boldsymbol{\mathbf{x}}\in\mathbb{R}^{d}$ on the $(d-1)$-dimensional sphere
$\mathbb{S}^{d-1}$ (since we consider the circle embedded in the 2D plane,
$d=2$ in our case). Equation 24 is usually solved using closed-form
approximations (Sra, 2012; Kasarapu and Allison, 2015). However, the important
insight here is that 24 does not depend on the values
$\boldsymbol{\mathbf{x}}_{i}$, but only on the normalized length $\bar{r}$ of
their _sum_. This means that the MLE for both $\mu$ and $\kappa$ of a von
Mises distribution only depends on the sum
$\boldsymbol{\mathbf{r}}\coloneqq\sum_{i=1}^{n}\boldsymbol{\mathbf{x}}_{i}$ of
the directional vectors $\boldsymbol{\mathbf{x}}_{i}$, which can easily be
computed incrementally. With
$\boldsymbol{\mathbf{r}}_{1}=\boldsymbol{\mathbf{x}}_{1}$,

$\displaystyle\boldsymbol{\mathbf{r}}_{n}$
$\displaystyle=\boldsymbol{\mathbf{r}}_{n-1}+\boldsymbol{\mathbf{x}}_{n}$
$\displaystyle\quad(n>1),$ (25) $\displaystyle\qquad\bar{r}$
$\displaystyle=\frac{\left\|\boldsymbol{\mathbf{r}}_{n}\right\|}{n}\;,\quad\tilde{\boldsymbol{\mathbf{\mu}}}=\frac{1}{n}\;\boldsymbol{\mathbf{r}}_{n}$
$\displaystyle\quad(n\geq 1),$ (26)

and the remaining terms as in 23 and 24. Overall, this allows to estimate
cylindrical distributions fully incrementally. Note that the batch and
incremental updates are mathematically equivalent and thus yield the same
results.

###  5.4 Interactive Teaching of Spatial Relations

Finally, we explain how we implement the different steps of the interaction
scenario sketched in Sections 4.4 and 2. An example with a real robot is shown
in Fig. 3.

_1\. Command:_ Following (Kartmann et al., 2021), we use a Named Entity
Recognition (NER) model to parse object and relation phrases from a language
command and ground them to objects and relations in the robot’s memory using
substring matching. In addition, the resulting task
$R^{*}=\left({s^{*}},{u^{*}},{\mathcal{V}^{*}}\right)$ is stored in the
relational command segment of the memory system. This is required to later
construct the sample from a demonstration.

_2\. Plan:_ We query the current model $G_{{s^{*}}}^{{\left({t_{k}}\right)}}$
from the spatial relation memory segment. If
$G_{{s^{*}}}^{{\left({t_{k}}\right)}}=\emptyset$, the geometric meaning of the
spatial relation is unknown, and the robot cannot solve the task. In this
case, the robot verbally expresses its lack of knowledge and requests a
demonstration of what to do from the human. Otherwise,
$G_{{s^{*}}}^{{\left({t_{k}}\right)}}=\theta$ defines a cylindrical
distribution, which is used to sample a given number of candidates ($50$ in
our experiments). As in (Kartmann et al., 2021), the candidates are filtered
for feasibility, ranked according to the distribution’s probability density
function $p_{\theta}\\!\left(r,\phi,h\right)$, and the best candidate is
selected for execution. We consider a candidate to be feasible if it is free
of collisions, reachable and stable. To decide whether a candidate is free of
collisions, the target object is placed virtually at the candidate position
and tested for collisions with other objects within a margin of
$25\text{\,}\frac{\mathrm{mm}}{}$ at $8$ different rotations around the
gravity vector in order to cope with imprecision of action execution, e. g.,
the object rotating inside the robot’s hand while grasping or placing (note
that our method aims to keep the object’s orientation; the different rotations
are only used for collision checking). If a feasible candidate is found,
planning is successful. However, after filtering, it is possible that none of
the sampled candidates is feasible. This is especially likely when only few
samples have been collected so far and, thus, the model’s variance is low.
Before, this event marked a failure, but in this work, the robot is able to
recover by expressing its inability to solve the task and ask the human for a
demonstration. Note that the query mechanism of both failure cases, i. e. a
missing model and an insufficient one, are structurally identical. In both
cases, the human is asked to perform the task it originally gave to the robot,
i. e., manipulate the scene to fulfill the relation $R^{*}$.

_3a. Execution by Robot and 3b. Query:_ If planning was successful (3a.), the
robot executes the task by grasping the target object and executing a placing
action parameterized based on the selected placing position. If planning
failed (3b.), the robot verbally requests a demonstration of the task from the
human using sentences generated from templates and its speech-to-text system.
Examples of generated verbal queries are “I am sorry, I don’t know what
‘right’ means yet, can you show me what to do?” (no model) and “Sorry, I
cannot do it with my current knowledge. Can you show me what I should do?”
(insufficient model). Then, the robot waits for the speech cue signaling the
finished demonstration.

_4\. Execution by Human and 5. Speech Cue:_ After being queried for a
demonstration (3b.), the human relocates the target object to fulfill the
requested spatial relation. To signal the completed demonstration, the human
gives a speech cue such as “Place it here” to the robot, that is detected
using simple keyword spotting and triggers the recording of a new sample. This
represents a third case where demonstrations can be triggered: The robot may
have successfully executed the task in a qualitative sense (3a.), but the
human may not be satisfied with the chosen placing position. In this case, the
human can simply change the scene in order to demonstrate what the robot
_should have done_ , and give a similar speech cue “No, put it here.” Again,
note that this case is inherently handled by the framework without a special
case: When the speech cue is received, the robot has all the knowledge it
requires to create a new sample, independently of whether it executed the task
before or explicitly asked for a demonstration.

_6\. Observation and Model Update:_ When the robot receives the speech cue, it
assembles a new sample by querying its memory for the relevant information. It
first queries its relational command segment for the latest command, which
specifies the requested relation $R^{*}$ that was just fulfilled by the
demonstration. It then queries its object instance segment for the state of
the scene $\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}$ when
the command was given and the current state
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}$. Combined, this
information forms a new sample
$D=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}},R^{*},\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}\right)$.
Afterwards, the robot stores the new sample in its long-term memory and
updates $G_{{s^{*}}}$ as described in Section 4.4. Finally, the robot thanks
the human demonstrator for the help, e. g., “Thanks, I think I now know the
meaning of ‘right’ a bit better.”

##  6 Results and Discussion

We evaluate our method quantitatively by simulating the interaction scheme
with a virtual robot and (Asfour et al. (2019)).

Figure 3:  Interactively teaching the humanoid robot ARMAR-6 to place objects
_to the right of_ other objects. After the first command by the human (A), the
robot queries a demonstration (B). The human demonstrates the task (C) which
is used by the robot to create or update its model (D). When given the next
command (E), the robot can successfully perfom the task (F).

###  6.1 Quantitative Evaluation

With our experiments, we aim at investigating two questions: (1) How many
demonstrations are necessary to obtain a useful model for a given spatial
relation? And (2) how does our model perform compared to baseline models using
fixed offsets instead of learned cylindrical distributions? To this end, we
implement the proposed human-robot interaction scheme based on human
demonstrations collected in simulation.

####  6.1.1 Experimental Setup

For the design of our experiments, a human instructs the robot to manipulate
an object according to a spatial relation to other objects. The robot tries to
perform the task, and if it is successful, the interaction is finished. If the
robot is unable to solve the task, it requests the user to demonstrate the
task and updates its model of the spatial relation from the given
demonstration. We call this procedure one _interaction_. We are interested in
how the robot’s model of a spatial relation develops over the course of
multiple interactions, where it only receives a new demonstration if it fails
to perform the task at hand.

In our experiments, we consider the $12$ spatial relations $\mathcal{S}$
listed in Table 1. Consider one relation $s\in\mathcal{S}$, and assume we have
$10$ demonstrations of this relation. As defined in 4, each demonstration
$D_{i}$ $(i=1,\dots,10)$ has the form
$D_{i}=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}_{i},R_{i},\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}_{i}\right)$,
with initial object poses
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}_{i}$, desired
spatial relation $R_{i}$, and object poses after the demonstration
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}_{i}$. Each
demonstration $D_{i}$ also implicitly defines a _task_
$T_{i}=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}_{i},R_{i}\right)$
consisting of only the initial scene and the desired relation. Given this
setup, we define a _learning scenario_ as follows: First, we initialize a
virtual robot that has no geometric model of $s$ as it has not received any
demonstrations yet. Then, we consecutively perform one interaction for each
task $T_{1},\dots,T_{10}$. After each interaction, we evaluate the robot’s
current model of $s$ on all tasks.

More precisely, in the beginning we ask the robot to perform the first task
$T_{1}$. As it has no model of $s$ at this point, it will always be unable to
solve the task. Following our interaction scheme, the robot requests the
corresponding demonstration and is given the target scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}_{i}$. The robot
learns its first model of $s$ from this demonstration, which concludes the
first interaction. We then evaluate the learned model on all tasks
$T_{1},\dots,T_{10}$, i. e., for each task we test whether the model can
generate a feasible placing position for the target object. Here, we consider
$T_{1}$ as _seen_ , and the remaining tasks $T_{2},\dots,T_{10}$ as _unseen_.
Accordingly, we report the success ratios, i. e., the proportion of solved
tasks, among the seen task, the unseen tasks and all tasks. We then proceed
with the second interaction, where the robot is asked to perform $T_{2}$. This
time, the model learned from the previous interaction may already be
sufficient to solve the task. If this is the case, the robot does _not_
receive the corresponding demonstration and thus keeps its current model.
Otherwise, it is given the new demonstration $D_{2}$ and can incrementally
update its model of $s$. Again, we test the new model on all tasks, where
$T_{1}$ and $T_{2}$ are now considered as seen and $T_{3},\dots,T_{10}$ as
unseen. We continue in this manner for all tasks left. After the last
interaction, all tasks $T_{1},\dots,T_{10}$ are seen and there are no unseen
tasks. For completeness, we also perform an evaluation on all tasks before the
first interaction; here, the robot trivially fails on all tasks since it
initially has no model of $s$.

Overall, for one learning scenario of relation $s$, we report the proportions
of solved tasks among all tasks, the seen tasks and the unseen tasks after
each interaction (and before the first interaction). Note that, as explained
above, the number of seen and unseen tasks change over the course of a
learning scenario. In addition, we report the number of demonstrations the
robot has been given after each interaction. Note that this number may be
smaller than the number of performed interactions, as a demonstration is only
given during an interaction if the robot fails to perform the task at hand. As
the results depend on the order of the tasks and demonstrations, we run $10$
repetitions of each learning scenario with the tasks randomly shuffled for
each repetition, and report all metrics’ means and standard deviations over
the repetitions. Finally, to obtain an overall result over all spatial
relations, we aggregate the results of all repetitions of learning scenarios
of all relations $s\in\mathcal{S}$, and report the resulting means and
standard deviations of success ratios and number of demonstrations with
respect to the number of performed interactions.

Spatial Relations | Baseline Model
---|---
$(s\in\mathcal{S})$ |  Description |  Placing Position $\boldsymbol{\mathbf{p}}_{u}^{{\left({t_{k+1}}\right)}}$
_right of_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ in $+x$ direction | $\boldsymbol{\mathbf{p}}_{v}+$20\text{\,}\frac{\mathrm{cm}}{}$\cdot\boldsymbol{\mathbf{x}}$
_left of_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ in $-x$ direction | $\boldsymbol{\mathbf{p}}_{v}-$20\text{\,}\frac{\mathrm{cm}}{}$\cdot\boldsymbol{\mathbf{x}}$
_behind_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ in $+y$ direction | $\boldsymbol{\mathbf{p}}_{v}+$20\text{\,}\frac{\mathrm{cm}}{}$\cdot\boldsymbol{\mathbf{y}}$
_in front of_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ in $-y$ direction | $\boldsymbol{\mathbf{p}}_{v}-$20\text{\,}\frac{\mathrm{cm}}{}$\cdot\boldsymbol{\mathbf{y}}$
_on top of_ |  $10\text{\,}\frac{\mathrm{cm}}{}$ in $+z$ direction | $\boldsymbol{\mathbf{p}}_{v}+$10\text{\,}\frac{\mathrm{cm}}{}$\cdot\boldsymbol{\mathbf{z}}$
_close to_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ towards initial position of $u$ | $\boldsymbol{\mathbf{p}}_{v}+$10\text{\,}\frac{\mathrm{cm}}{}$\cdot\mathrm{dir}\\!\left(\boldsymbol{\mathbf{p}}_{u}-\boldsymbol{\mathbf{p}}_{v}\right)$
_far from_ |  $60\text{\,}\frac{\mathrm{cm}}{}$ towards initial position of $u$ | $\boldsymbol{\mathbf{p}}_{v}+$50\text{\,}\frac{\mathrm{cm}}{}$\cdot\mathrm{dir}\\!\left(\boldsymbol{\mathbf{p}}_{u}-\boldsymbol{\mathbf{p}}_{v}\right)$
_between_ |  Mid point of reference objects | $\nicefrac{{1}}{{n}}\cdot\sum_{i=1}^{n}\boldsymbol{\mathbf{p}}_{v_{i}}$
_among_ |  $20\text{\,}\frac{\mathrm{cm}}{}$ from reference objects’ mid point towards initial position of $u$ | $\boldsymbol{\mathbf{p}}_{v}+$10\text{\,}\frac{\mathrm{cm}}{}$\cdot\mathrm{dir}\\!\left(\boldsymbol{\mathbf{p}}_{u}-\nicefrac{{1}}{{n}}\cdot\sum_{i=1}^{n}\boldsymbol{\mathbf{p}}_{v_{i}}\right)$
_closer_ |  Half the distance from reference to $u$ | $\boldsymbol{\mathbf{p}}_{v}+\nicefrac{{1}}{{2}}\cdot\left(\boldsymbol{\mathbf{p}}_{u}-\boldsymbol{\mathbf{p}}_{v}\right)$
_farther from_ |  Twice the distance from reference to $u$ | $\boldsymbol{\mathbf{p}}_{v}+2\cdot\left(\boldsymbol{\mathbf{p}}_{u}-\boldsymbol{\mathbf{p}}_{v}\right)$
_on the other side of_ |  Distance between reference and target in opposite direction of target | $\boldsymbol{\mathbf{p}}_{v}-\left(\boldsymbol{\mathbf{p}}_{u}-\boldsymbol{\mathbf{p}}_{v}\right)$
Table 1:  Left column: Spatial relation symbols $s\in\mathcal{S}$ used in the
experiments. Right columns: Their corresponding baseline model, where
$\boldsymbol{\mathbf{p}}_{v},\boldsymbol{\mathbf{p}}_{v_{i}}\in\mathbb{R}^{3}$
refer to the positions of reference objects, $\boldsymbol{\mathbf{p}}_{u}$
refers to the initial position of target object $u$,
$\boldsymbol{\mathbf{x}},\boldsymbol{\mathbf{y}},\boldsymbol{\mathbf{z}}$ are
the unit vectors in the robot’s coordinate system, and
$\mathrm{dir}\\!\left(\boldsymbol{\mathbf{v}}\right)=\nicefrac{{\boldsymbol{\mathbf{v}}}}{{\left\|\boldsymbol{\mathbf{v}}\right\|}}$
is the direction of a vector $\boldsymbol{\mathbf{v}}\in\mathbb{R}^{3}$.

We compare our method with baseline models for all relations which place the
target object at a fixed offset from the reference objects. Table 1 gives
their descriptions and the definitions of the placing position
$\boldsymbol{\mathbf{p}}_{u}^{{\left({t_{k+1}}\right)}}$ they generate. The
baseline models of direction-based static relations (_right of_ , _left of_ ,
_behind_ , _in front of_ , _on top of_) yield a candidate at a fixed distance
towards the respective direction. Distance-based static relations (_close to_
, _far from_ , _among_) place the target object at a fixed distance towards
its initial direction. The baseline models of dynamic spatial relations
(_closer_ , _farther from_ , _on the other side_) are similar, but the
distance of the placing position to the reference object is defined relative
to the current distance between the objects. We conduct the experiments on the
baseline models in the same way as described above, except that (1) the robot
has access to the models from the beginning, and (2) we only report the
success ratio among all tasks over one repetition, as the baseline models are
constant and do not change during a learning scenario or depending on the
order of tasks.

For collecting the required demonstrations for each relation, we use a setup
with a human, several objects on a table in a simulated environment and a
command generation based on sentence templates. Given the objects present in
the initial scene and the set of spatial relation symbols $\mathcal{S}$, we
generate a verbal command specifying a desired spatial relation
$R=\left(s,u,\mathcal{V}\right)$ such as “Place the cup between the milk and
the plate.” The human is given the generated command and performs the task by
moving the objects in the scene. We record the initial scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}}$, the desired
spatial relation $R$, and the final scene
$\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}$, describing a
full demonstration
$D=\left(\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k}}\right)}},R,\mathcal{\boldsymbol{\mathbf{P}}}^{{\left({t_{k+1}}\right)}}\right)$
as required for the experiment. Aside from the objects involved in the spatial
relation, the scenes also contain inactive objects which can lead to
collisions when placing the object at a generated position, thus rendering
some placements infeasible.

####  6.1.2 Results

Fig. 4 shows the means and standard deviations of the percentage of solved
tasks among all tasks, the seen tasks and the unseen tasks after each
interaction aggregated over all relations and repetitions as described above.
Furthermore, it shows the averages of total number of demonstrations the robot
has received after each interaction. As explained above, note that not all
interactions result in a demonstration. Also, note that the number of seen and
unseen tasks change with the number of interactions; especially, there are no
seen tasks before the first interaction and there are no unseen tasks after
the last one. For comparison, the success ratio of the baseline models
averaged over all relations are shown as well.

In addition, Fig. 5 gives examples of success ratios and number of received
demonstrations aggregated only over the repetitions of learning scenarios of
single spatial relations. Note that here, the baseline’s performance is
constant over all repetitions as the number of tasks it can solve is
independent of the order of tasks in the learning scenario. Finally, Fig. 6
presents concrete examples of solved and failed tasks involving the spatial
relations in Fig. 5 demonstrating the behavior of our method and the baseline
during the experiment. Note that the cases represented by the rows in Fig. 6
are not equally frequent; our method more often outperforms the baseline
models than vice-versa as indicated by the success ratios in Figs. 4 and 5.

Figure 4:  Means and standard deviations of percentage of tasks solved by our
method among the seen (green), unseen (orange) and all (blue) tasks, by the
baseline models using fixed offsets (gray) as well as number of demonstrations
(purple) the robot has been given after a given number of interactions. All
metrics are aggregated over multiple repetitions and all spatial relations.
Figure 5:  Examples of success ratios and number of demonstrations aggregated
over the repetitions of single relations. Colors are as in Fig. 4. Note that
for single relations, the success ratios of the baseline models are constant
and, thus, have no variance.  Figure 6:  Examples of solved and failed tasks
from our experiment. In all images, the reference objects are highlighted in
yellow, the target object is highlighted in light blue, and the generated
placing position is visualized in green if successful and red if no feasible
candidate was found. The orange arrow indicates the mean of a cylindrical
distribution (Ours) and the placement according to the baseline model,
respectively. The cylindrical distribution’s p.d.f. is visualized around the
reference objects (low values in purple, high values in yellow). Sampled
positions are shown as grey marks. The left column of each relations shows our
model, while the right column shows the result of the baseline model in the
same task. For our model, the current cylindrical distribution is visualized
as well. For good qualitative coverage, each row show another case with
respect to the models’ success: (1) both succeed, (2) ours succeeds, baseline
fails, (3) ours fails, baseline succeeds, (4) both fail.

####  6.1.3 Discussion

The baseline models achieved only $52.5\pm 24.9\text{\,}\frac{\mathrm{\char
37\relax}}{}$ success rate over all relations, showing that many tasks in our
demonstrations were not trivially solvable using fixed offsets. Also, the
standard deviation of 52.5 $\pm$ 24.9 %. The models were successful if their
candidate placing position was free and on the tables. However, if the single
candidate was, e. g., obstructed by other objects, the models could not fall
back to other options. This was especially frequent among relations such as
_close to_ , _between_ and _among_ which tend to bring the target object close
to other objects. Other collisions were caused because the sizes of the
involved objects were not taken into account, especially among relations with
fixed distances.

As expected, using our method the robot could never solve a task before the
first interaction. However, after the first interaction (which always results
in a demonstration), the robot could already solve about $83.3\pm
13.1\text{\,}\frac{\mathrm{\char 37\relax}}{}$ of seen and about $51.9\pm
18.3\text{\,}\frac{\mathrm{\char 37\relax}}{}$ of unseen tasks on average,
almost equaling the baseline models on the unseen tasks. After just two
interactions, the mean success ratios on seen tasks reaches a plateau of at
$89\text{\,}\frac{\mathrm{\char 37\relax}}{}92\text{\,}\frac{\mathrm{\char
37\relax}}{}$, with $1.53\pm 0.19$ demonstrations on average. Importantly, the
success ratios among the seen tasks stays high and does not decrease after
more interactions, which indicates that our method generally does not “forget”
what it has learned from past demonstrations. The success ratios among unseen
tasks rise consistently with the number of interactions, although more slowly
than among seen tasks. Nonetheless, after five interactions, the robot could
solve $85.3\pm 9.9\text{\,}\frac{\mathrm{\char 37\relax}}{}$ of unseen tasks
after having received $2.13\pm 0.41$ demonstrations on average, which shows
that our method can generalize from few demonstrations. After completing each
learning scenario, i. e., after all interactions have been performed, the
robot could solve $93.0\pm 09.1\text{\,}\frac{\mathrm{\char 37\relax}}{}$ of
all tasks while having received $2.81\pm 0.87$ demonstrations on average.

One might wonder why the robot is not always able to successfully reproduce
the first demonstration on the single seen task after the first interaction.
This can be explained in two ways: First, the human is free to change the
target object’s orientation during the demonstration, which can allow the
human to place it closer to other objects than without rotating it. However,
as our method only generates new positions while trying to keep the original
object orientation, the robot cannot reproduce this demonstration as it would
lead to a collision. Second, we use a rather conservative strategy for
collision detection in order to anticipate inaccuracies in action execution
(e. g., the object rotating in the robot’s hand during grasping or placing).
More precisely, we use a collision margin of $25\text{\,}\frac{\mathrm{mm}}{}$
to check for collisions at different hypothetical rotations of the target
object (see Section 5.4). Therefore, if the human placed the object very
closely to another in the demonstration, the robot might discard this solution
to avoid collisions. These are situations that can only be solved after
increasing the model’s variance through multiple demonstrations.

We can observe the behavior of our method in more detail by focusing on the
results of single relations shown in Fig. 5 and the examples in Fig. 6. First,
the standard deviation over the success ratios among the unseen tasks tends to
increase towards the end of the learning scenarios. This is likely due to the
decreasing number of unseen tasks towards the end: Before the final
interaction, there is only one unseen task left, so the success ratio is
either $0\text{\,}\frac{\mathrm{\char 37\relax}}{}$ or
$100\text{\,}\frac{\mathrm{\char 37\relax}}{}$ in each repetition, leading to
a higher variance than when the success ratio is averaged over more tasks. As
for the relation _right of_ , common failure cases were caused by the
conservative collision checking in combination with a finite number of sampled
candidates (Fig. 6, third row), or the mean distance of the learned
distribution being too small for larger objects such as the plate (Fig. 6,
fourth row). With the relation _further_ , failures were often caused by the
candidate positions being partly off the table in combination with the
distance variance being too small to generate alternatives (Fig. 6, third
row), or scenes where the sampled area was either blocked by other objects or
off the table (Fig. 6, fourth row).

The relation _between_ was one of the relations that were more difficult to
learn, with a success ratio of only $67.0\pm 04.6\text{\,}\frac{\mathrm{\char
37\relax}}{}$ among all tasks after receiving an average of $4.90\pm 0.83$
demonstrations at the end of the learning scenario (the baseline model
achieved only $10\text{\,}\frac{\mathrm{\char 37\relax}}{}$). In the
demonstrations, the area between the two reference objects was often
cluttered, which prevented our method from finding a collision free placing
location. The success ratio among the seen tasks starts at $70.0\pm
45.8\text{\,}\frac{\mathrm{\char 37\relax}}{}$ after one interaction. The
large variance indicates that, compared to other relations, there were many
demonstrations that could not be reproduced after the first interaction, with
the success ratio among seen tasks being either $0\text{\,}\frac{\mathrm{\char
37\relax}}{}$ or $100\text{\,}\frac{\mathrm{\char 37\relax}}{}$ leading to a
high variance, similar to the success ratios among unseen tasks towards the
end of the learning scenarios. Moreover, the success ratio among the seen
tasks decreases to $56.7\pm 26.0\text{\,}\frac{\mathrm{\char 37\relax}}{}$
after the third interaction, although it slightly increases again afterwards.
In this case, the $1.50\pm 0.50$ additional demonstrations caused the model to
“unlearn” to solve the first tasks in some cases. However, after the third
interaction, the success ratios among seen and unseen tasks stabilize and do
not change significantly with more demonstrations. Apparently, the models
reached their maximum variance after a few interactions, with new
demonstrations not changing the model significantly; however our conservative
collision detection often caused all candidates to be considered infeasible.
One especially difficult task is shown in Fig. 6 (fourth row), where the two
reference objects were standing very close to each other, leaving little space
for the target object. Finally, note that the tasks for the _between_ relation
shown in the first and the third row of Fig. 6 are the same. This is because
the baseline could only solve this single task. The corresponding failure
example of our model (third row) was shows a model learned from only one
demonstration. Indeed, with two or more demonstrations, our method was always
able to solve the this task (example in first row).

To summarize, while some aspects can still be improved, the overall results
demonstrate that our generative models of spatial relations can be effectively
learned in an incremental manner from few demonstrations, while our
interaction scheme allows the robot to obtain new demonstrations to improve
its models if they prove insufficient for a task at hand.

###  6.2 Validation Experiments on Real Robot

We performed validation experiments on the real humanoid robot (Asfour et al.
(2019)) which are shown in the video333https://youtu.be/x6XKUZd_SeE. An
example is shown in Fig. 3 and described in more detail here: In the
beginning, the robot has no geometric model of the _right of_ relation. First,
the user commands the robot to place an object _on the right side of_ a second
object (step 1. in Section 4.4). The robot grounds the relation phrase “on the
right side of” to the respective entry in its memory (2.), and responds that
it has “not learned what _right_ means yet,” and requests the user to show it
what to do (3a.). Consequently, the user gives a demonstration by performing
the requested task (4.) and gives the speech cue “Put it here” (5.). The robot
observes the change in the scene and creates a model of the spatial relation
_right of_ (6.). Afterwards, the user starts a second interaction by
instructing the robot to put the object _on the right side of_ a third one
(1.). This time, the robot has geometric model of _right of_ in its memory
(2.) and is able to perform the task (3b.). Beyond that, we show examples of
demonstrating and manipulation the scene according to the relations _in front
of_ , _on top of_ , _on the opposite side of_ , and _between_.

##  7 Conclusion and Future Work

In this work, we described how a learning humanoid robot which has the task to
manipulate the scene based on desired spatial object relations can query and
use demonstrations from a human during interaction to incrementally learn
generative models of spatial relations. We demonstrated how the robot can
communicate its inability to solve a task in order to collect more
demonstrations in a continual manner. In addition, we showed how a parametric
representation of object spatial relations can be learned incrementally from
few demonstrations. In future work, we would like to make the human-robot
interaction even more natural by detecting when a demonstration is finished,
thus releasing the requirement of a speech cue indicating this event.
Furthermore, we want to explore how knowledge about different spatial
relations can be transferred between them and leveraged for learning new ones.

## Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any
commercial or financial relationships that could be construed as a potential
conflict of interest.

## Author Contributions

RK developed the methods and their implementation and performed the evaluation
experiments. The entire work was conceptualized and supervised by TA. The
initial draft of the manuscript was written by RK and revised jointly by RK
and TA.

## Funding

This work has been supported by the Carl Zeiss Foundation through the JuBot
project and by the German Federal Ministry of Education and Research (BMBF)
through the OML project (01IS18040A).

## Supplemental Material

This paper is supplemented by a video giving an overview of our approach and
showing the validation experiments described in Section 6.2.

## Data Availability Statement

The raw data supporting the conclusions of this article will be made available
by the authors, without undue reservation.

## References

  * Aksoy et al. (2011) Aksoy, E. E., Abramov, A., Dörr, J., Ning, K., Dellen, B., and Wörgötter, F. (2011).  Learning the semantics of object–action relations by observation.  _International Journal of Robotics Research_ 30, 1229–1249
  * Asfour et al. (2017) Asfour, T., Dillmann, R., Vahrenkamp, N., Do, M., Wächter, M., Mandery, C., et al. (2017).  The Karlsruhe ARMAR Humanoid Robot Family.  In _Humanoid Robotics: A Reference_ , eds. A. Goswami and P. Vadakkepat (Springer Netherlands). 1–32
  * Asfour et al. (2019) Asfour, T., Wächter, M., Kaul, L., Rader, S., Weiner, P., Ottenhaus, S., et al. (2019).  ARMAR-6: A High-Performance Humanoid for Human-Robot Collaboration in Real World Scenarios.  _Robotics & Automation Magazine_ 26, 108–121
  * Bao et al. (2016) Bao, J., Hong, Z., Tang, H., Cheng, Y., Jia, Y., and Xi, N. (2016).  Teach robots understanding new object types and attributes through natural language instructions.  In _International Conference on Sensing Technology (ICST)_ (Nanjing, China), vol. 10, 1–6
  * Doğan et al. (2022) Doğan, F. I., Torre, I., and Leite, I. (2022).  Asking Follow-Up Clarifications to Resolve Ambiguities in Human-Robot Conversation.  In _ACM/IEEE International Conference on Human-Robot Interaction_ (Sapporo, Hokkaido, Japan), 461–469
  * Dreher et al. (2020) Dreher, C. R. G., Wächter, M., and Asfour, T. (2020).  Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networks.  _IEEE Robotics and Automation Letters (RA-L)_ 5, 187–194
  * Fasola and Matarić (2013) Fasola, J. and Matarić, M. J. (2013).  Using semantic fields to model dynamic spatial relations in a robot architecture for natural language instruction of service robots.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 143–150
  * Fichtl et al. (2014) Fichtl, S., McManus, A., Mustafa, W., Kraft, D., Krüger, N., and Guerin, F. (2014).  Learning spatial relationships from 3D vision using histograms.  In _IEEE International Conference on Robotics and Automation (ICRA)_. 501–508
  * Forbes et al. (2015) Forbes, M., Rao, R. P. N., Zettlemoyer, L., and Cakmak, M. (2015).  Robot Programming by Demonstration with Situated Spatial Language Understanding.  In _IEEE International Conference on Robotics and Automation (ICRA)_. 2014–2020
  * Grusec (1994) Grusec, J. E. (1994).  _Social Learning Theory and Developmental Psychology: The Legacies of Robert R. Sears and Albert Bandura_.  A Century of Developmental Psychology (Washington, DC, US: American Psychological Association)
  * Hatori et al. (2018) Hatori, J., Kikuchi, Y., Kobayashi, S., Takahashi, K., Tsuboi, Y., Unno, Y., et al. (2018).  Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_. 3774–3781
  * Kartmann et al. (2021) Kartmann, R., Liu, D., and Asfour, T. (2021).  Semantic Scene Manipulation Based on 3D Spatial Object Relations and Language Instructions.  In _IEEE-RAS International Conference on Humanoid Robots (Humanoids)_ (Munich, Germany), 306–313
  * Kartmann et al. (2020) Kartmann, R., Zhou, Y., Liu, D., Paus, F., and Asfour, T. (2020).  Representing Spatial Object Relations as Parametric Polar Distribution for Scene Manipulation Based on Verbal Commands.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 8373–8380
  * Kasarapu and Allison (2015) Kasarapu, P. and Allison, L. (2015).  Minimum message length estimation of mixtures of multivariate Gaussian and von Mises-Fisher distributions.  _Machine Learning_ 100, 333–378
  * Krüger et al. (2011) Krüger, N., Geib, C., Piater, J., Petrick, R., Steedman, M., Wörgötter, F., et al. (2011).  Object–Action Complexes: Grounded abstractions of Sensory–Motor processes.  _Robotics and Autonomous Systems_ 59, 740–757
  * Lee et al. (2020) Lee, S. U., Hong, S., Hofmann, A., and Williams, B. (2020).  QSRNet: Estimating Qualitative Spatial Representations from RGB-D Images.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 8057–8064
  * Losing et al. (2018) Losing, V., Hammer, B., and Wersing, H. (2018).  Incremental On-Line Learning: A Review and Comparison of State of the Art Algorithms.  _Neurocomputing_ 275, 1261–1274
  * Mees et al. (2017) Mees, O., Abdo, N., Mazuran, M., and Burgard, W. (2017).  Metric Learning for Generalizing Spatial Relations to New Objects.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 3175–3182
  * Mees et al. (2020) Mees, O., Emek, A., Vertens, J., and Burgard, W. (2020).  Learning Object Placements For Relational Instructions by Hallucinating Scene Representations.  In _IEEE International Conference on Robotics and Automation (ICRA)_. 94–100
  * Mitchell (1982) Mitchell, T. M. (1982).  Generalization as Search.  _Artificial Intelligence_ 18, 203–226
  * Mota and Sridharan (2018) Mota, T. and Sridharan, M. (2018).  Incrementally Grounding Expressions for Spatial Relations Between Objects.  In _International Joint Conferences on Artificial Intelligence (IJCAI)_. 1928–1934
  * Nicolescu et al. (2019) Nicolescu, M., Arnold, N., Blankenburg, J., Feil-Seifer, D., Banisetty, S. B., Nicolescu, M., et al. (2019).  Learning of Complex-Structured Tasks from Verbal Instruction.  In _IEEE/RAS International Conference on Humanoid Robots (Humanoids)_. 770–777
  * O’Keefe (2003) O’Keefe, J. (2003).  Vector Grammar, Places, and the Functional Role of the Spatial Prepositions in English.  In _Representing Direction in Language and Space_ (Oxford University Press). 69–85
  * Pardowitz et al. (2005) Pardowitz, M., Zollner, R., and Dillmann, R. (2005).  Learning sequential constraints of tasks from user demonstrations.  In _IEEE-RAS International Conference on Humanoid Robots_. 424–429
  * Ravichandar et al. (2020) Ravichandar, H., Polydoros, A. S., Chernova, S., and Billard, A. (2020).  Recent Advances in Robot Learning from Demonstration.  _Annual Review of Control, Robotics, and Autonomous Systems_ 3, 297–330
  * Rosman and Ramamoorthy (2011) Rosman, B. and Ramamoorthy, S. (2011).  Learning spatial relationships between objects.  _International Journal of Robotics Research_ 30, 1328–1342
  * Shridhar and Hsu (2018) Shridhar, M. and Hsu, D. (2018).  Interactive Visual Grounding of Referring Expressions for Human-Robot Interaction.  In _Robotics: Science & Systems (RSS)_
  * Shridhar et al. (2020) Shridhar, M., Mittal, D., and Hsu, D. (2020).  INGRESS: Interactive visual grounding of referring expressions.  _International Journal of Robotics Research_ 39, 217–232
  * Sjöö and Jensfelt (2011) Sjöö, K. and Jensfelt, P. (2011).  Learning spatial relations from functional simulation.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 1513–1519
  * Sra (2012) Sra, S. (2012).  A short note on parameter approximation for von Mises-Fisher distributions: And a fast implementation of Is(x).  _Computational Statistics_ 27, 177–190
  * Stopp et al. (1994) Stopp, E., Gapp, K.-P., Herzog, G., Laengle, T., and Lueth, T. C. (1994).  Utilizing Spatial Relations for Natural Language Access to an Autonomous Mobile Robot.  In _KI-94: Advances in Artificial Intelligence_ , eds. B. Nebel and L. Dreschler-Fischer (Berlin, Heidelberg: Springer), Lecture Notes in Computer Science, 39–50
  * Tan et al. (2014) Tan, J., Ju, Z., and Liu, H. (2014).  Grounding Spatial Relations in Natural Language by Fuzzy Representation for Human-Robot Interaction.  In _IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)_. 1743–1750
  * Tellex et al. (2011) Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S., et al. (2011).  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_. vol. 25 of _1_ , 1507–1514
  * Vahrenkamp et al. (2015) Vahrenkamp, N., Wächter, M., Kröhnert, M., Welke, K., and Asfour, T. (2015).  The robot software framework ArmarX.  _it - Information Technology_ 57, 99–111
  * Welford (1962) Welford, B. P. (1962).  Note on a method for calculating corrected sums of squares and products.  _Technometrics : a journal of statistics for the physical, chemical, and engineering sciences_ 4, 419–420
  * Yan et al. (2020) Yan, F., Wang, D., and He, H. (2020).  Robotic Understanding of Spatial Relationships Using Neural-Logic Learning.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. 8358–8365
  * Zampogiannis et al. (2015) Zampogiannis, K., Yang, Y., Fermuller, C., and Aloimonos, Y. (2015).  Learning the Spatial Semantics of Manipulation Actions Through Preposition Grounding.  In _IEEE International Conference on Robotics and Automation (ICRA)_. 1389–1396