# Automatic self-contained calibration of an industrial dual-arm robot with
cameras using self-contact, planar constraints, and self-observation

Karla Stepanova  Jakub Rozlivek  Frantisek Puciow  Pavel Krsek  Tomas Pajdla
Matej Hoffmann<EMAIL_ADDRESS>Department of Cybernetics, Faculty
of Electrical Engineering, Czech Technical University in Prague, Czech
Republic  Czech Institute of Informatics, Robotics, and Cybernetics, Czech
Technical University in Prague, Czech Republic  Both authors contributed
equally.

###### Abstract

We present a robot kinematic calibration method that combines complementary
calibration approaches: self-contact, planar constraints, and self-
observation. We analyze the estimation of the end effector parameters, joint
offsets of the manipulators, and calibration of the complete kinematic chain
(DH parameters). The results are compared with ground truth measurements
provided by a laser tracker. Our main findings are: (1) When applying the
complementary calibration approaches in isolation, the self-contact approach
yields the best and most stable results. (2) All combinations of more than one
approach were always superior to using any single approach in terms of
calibration errors and the observability of the estimated parameters.
Combining more approaches delivers robot parameters that better generalize to
the workspace parts not used for the calibration. (3) Sequential calibration,
i.e. calibrating cameras first and then robot kinematics, is more effective
than simultaneous calibration of all parameters. In real experiments, we
employ two industrial manipulators mounted on a common base. The manipulators
are equipped with force/torque sensors at their wrists, with two cameras
attached to the robot base, and with special end effectors with fiducial
markers. We collect a new comprehensive dataset for robot kinematic
calibration and make it publicly available. The dataset and its analysis
provide quantitative and qualitative insights that go beyond the specific
manipulators used in this work and apply to self-contained robot kinematic
calibration in general.

###### keywords:

automatic robot calibration , self-contained calibration , self-contact ,
self-observation , self-calibration , kinematic calibration

##  1 Introduction

Accurate calibration is essential for the performance of every robot.
Traditional calibration procedures involve some form of external measuring
apparatus and become impractical if the robot itself or the site where it is
deployed change frequently—the current trend in automation with the shift from
mass to small batch production. Furthermore, collaborative and social robots
often employ cheaper and more elastic materials, making them less accurate and
prone to frequent re-calibration. At the same time, advances in sensor
technology make affordable but increasingly accurate devices such as standard
or RGB-D cameras, tactile, force, or inertial sensors available; often, robots
come already equipped with a selection of them. These factors together
constitute the need and the opportunity to perform an automated self-contained
calibration relying on redundant information in these sensory streams
originating from the robot platform itself.

Figure 1: Setups for automatic calibration of the dual-arm robot with graphs
of the kinematic chains. Setup for self-touch calibration (left) and
calibration when a horizontal plane is touched (right). All chains originate
in a common base frame (bottom yellow circle). The left and right arm chains
are drawn in purple and green, respectively. The eye chains are drawn in
white. Red lines denote reprojection into the cameras. Cyan indicates the
distance between end effector centers (one diameter) or between the end
effector and a plane (end effector radius).

We present a robot kinematic calibration method, which combines complementary
calibration approaches: self-contact, planar constraints, and self-
observation. There are two main approaches to robot calibration [1]: (i) open-
loop approaches, where the manipulator is not in mechanical (physical) contact
with the environment and an external metrology system is used to measure robot
pose components, and (ii) closed-loop approaches, where physical constraints
on robot pose components substitute for external measurements. However, in
their standard formulations, both approaches require a setup that is external
to the robot itself. This work aims to extend these well-established tools to
automatic self-contained robot calibration [2] to include camera reprojection
errors, constraints arising from robot self-contact, or simultaneous
calibration of multiple kinematic chains.

In this work, we employ two industrial manipulators mounted on a common base
with force/torque sensors at their wrists, two cameras overlooking the
workspace, and special end effectors with fiducial markers. Using this setup,
we study different kinematic calibration methods: self-contact, planar
constraints, and self-observation. These can be employed separately or in
combination (simultaneously or sequentially). Fig. 1 shows an example: a self-
contact configuration provides constraints that can be exploited to calibrate
the kinematic chain of one or both arms. At the same time, the end effectors
are observed by the cameras, providing additional data (equations) for
calibrating the manipulators’ kinematics and the camera orientation
parameters.

We evaluate the performance of individual approaches on the calibration of:
(i) parameters of the custom end effector, (ii) joint offsets of the complete
kinematic chain of one arm, and (iii) the full Denavit-Hartenberg (DH)
representation of the platform. Calibration using an external measurement
device (laser tracker) is performed and serves together with the nominal
manipulators’ kinematic parameters as a reference. The calibration performance
of individual methods is compared on an independent testing dataset which
covers a significant portion of the robot workspace.

We present four main contributions. First, we provide a thorough experimental
comparison of geometric kinematic calibration using self-contained
approaches—self-contact, planar constraints, and self-observation—with
kinematic calibration based on an external laser tracker. We demonstrate the
viability of the self-contained approaches. Second, we design ways how the
individual methods can be combined into a single cost function. The merits of
such a synergistic approach leading to better identifiability of the
calibrated parameters and generalization to the part of the robot workspace,
which was not used for parameter calibration, are empirically evaluated.
Third, we compare simultaneous (all parameters at once) with sequential
calibration. Fourth, we collect a new comprehensive dataset for robot
kinematic calibration and make it publicly available. This dataset and its
analysis provide quantitative and qualitative insights that go beyond the
specific manipulators used in this work and apply to self-contained robot
kinematic calibration in general.

To our knowledge, this work is the first that bridges the realm of very
accurate industrial calibration using external metrology with compact
approaches relying on sensors on the robot—typical for humanoid or social
robotics. The innovation this work brings is that it expands the portfolio of
traditional calibration methods by self-contained approaches such as self-
contact or self-observation. While traditionally industrial robots did not
have means to detect contacts, this is changing now with the advent of
collaborative robots where sensitive contact detection is necessary for safe
human-robot collaboration. Furthermore, torque sensing within the robot
structure or joint/torque sensors at the flange facilitate dexterous
manipulation and interaction with the environment. At the same time,
integrated vision sensors (cameras, RGB-D sensors) are becoming popular. Here
we provide a framework that makes it possible to exploit all available
calibration methods and sensory streams simultaneously and combine them in an
optimal way, including weighting by their respective accuracy.

This article is structured as follows. Related work is reviewed in Sec. 2.
Section 3 describes Experimental setup and data acquisition: Robot (Sec. 3.1)
and Laser tracker (Sec. 3.2) setup, robot control using force feedback (Sec.
3.3), robot and end effector dimensions (Sec. 3.4), camera calibration (Sec.
3.5), and description of individual acquired datasets (Sec. 3.6). The
optimization problem formulation for different combinations of kinematic
chains is described in Sec. 4. Experimental results for end effector,
manipulator offsets, and all DH parameters calibration (including laser
tracker reference) are presented in Sec. 5. Finally, we present the conclusion
(Sec. 6) and discussion, including future work (Sec. 7). The accompanying
video illustrating the experiments is available at
https://youtu.be/LMwINqA1t9w; the dataset is available from [3].

##  2 Related work

Standard robot kinematic calibration employs different mechanical systems—for
example, measurement arms [4]—or contactless measurement systems like laser
trackers [5, 6, 7, 8]. Cameras are used less often: for example, stereo
cameras combined with a calibration sphere [9] or a camera together with a
laser pointer at the end effector [10]. All of these systems require
specialized external equipment. Our platform was previously calibrated using
two different methods: (1) Redundant parallel Calibration and measuring
Machine (RedCaM) by Beneš et al. [11], Volech et al. [12], and (2) Leica laser
tracker. Petrík and Smutný [13] reviewed the precision of these methods using
a linear probe sensor.

Since we aim at automatic self-contained multisensorial calibration—that is,
calibration using sensors on the robot, involving multiple sensory modalities
like vision and touch—we next focus on reviewing the state of the art in
calibration methods that do not rely on external metrology systems. We will
also pay special attention to humanoid-like setups that offer the richest
possibilities for self-contained calibration.

Calibration by self-observation. Cameras mounted on a robot can be used to
calibrate the robot by closing the calibration loop through self-observation
of its end effectors. The theory for this approach is laid out in [14] for a
stereo camera system observing a robot arm. The manipulator’s kinematics and
extrinsic and intrinsic camera parameters are calibrated. Self-observation has
been applied to humanoid robots viewing their hands with fiducial markers
using online methods to calibrate kinematics relying on gradient descent by
Hersch et al. [15] and recursive least squares estimation by Martinez-Cantin
et al. [16]. Fiducial markers can sometimes be avoided when the robot wrist,
hand, or fingertip are identified directly in the image [17, 18]. The work of
Birbach et al. [2] on the humanoid Agile Justin will be discussed in more
detail below.

Calibration using physical constraints. The next family of approaches exploits
physical contacts of the end effector with the environment, such as fixing the
end effector to the ground [19] or using more complex setups [20, 21, 22].
Some form of force sensing on the part of the manipulator is required.
Kinematic calibration using plane constraints (with known or unknown
parameters of the plane) was explored by Ikits and Hollerbach [23]; they
proposed a new approach focusing on the proper definition of the base and end
link frames and evaluated primarily in simulation. Zhuang et al. [24] explored
multiple variants of plane constraints and the option with/without known plane
parameters and demonstrated their results on a PUMA 560 robot. In particular,
they showed that a single-plane constraint does not necessarily guarantee that
all kinematic parameters of the robot will be observable. On the other hand, a
multiple-plane constraint should be a remedy to this problem. They show that
the data collected from 3 planar constraints is equivalent to the data
collected from a point measurement device provided that: 1) all three planes
are mutually non-parallel; 2) the identification Jacobian of the unconstrained
system is nonsingular; and 3) the measured points from each individual plane
do not lie on a line on that plane. Joubair and Bonev [25] showed how multi-
planar constraints can significantly improve the accuracy of calibration of a
six-axis serial robot. Zenha et al. [26] had the simulated iCub humanoid touch
three known planes, employing adaptive parameter estimation (Extended Kalman
Filter) for kinematic calibration. Khusainov et al. [27] exploit a specific
type of mechanical coupling as they fix the end effector of a manipulator to
the legs of a humanoid robot that is being calibrated. A point and distance
constraint can also be obtained from visual sensing [28].

Calibration by self-contact (self-touch). Self-contact constitutes a specific,
less common way of kinematic loop closure available only to humanoid-like or
dual arm setups. Additionally, corresponding sensory and motor equipment such
that this self-contact can be performed in a controlled manner is needed. One
possibility is to utilize artificial electronic skins covering specific areas
or complete robot bodies (see [29, 30] for recent overviews). A tactile array
may be used for contact detection. If accurate spatial calibration of the skin
is available, then additional components of the self-contact
configuration—where contact occurs on each of the two intersecting chains—can
be measured. Roncone et al. [31] performed kinematic calibration on the iCub
using autonomous self-touch—index finger on the contralateral forearm; Li et
al. [32] employed a dual KUKA arm setup with a sensorized “finger” and a
tactile array on the other manipulator. Mittendorfer and Cheng [33] also
exploit artificial skin to learn models of robots, but their methods primarily
utilize the signals from accelerometers embedded in their multimodal skin.
Alternatively, if controlled self-contact can be established but the exact
position is not measured—typically when using force/torque sensing—such
constraints can also be employed for calibration, as will be demonstrated in
this work.

Self-contained multisensorial calibration. There are only a few approaches
that exploit “multisensorial” (or “multimodal”) input for self-contained
calibration. Birbach et al. [2] calibrated the humanoid robot Justin observing
its wrist. Sensors were fused by minimizing a single cost function that
aggregates the errors obtained by comparing the discrepancies between
simulated projections (left and right camera images, Kinect image, Kinect
disparity) and the wrist position from forward kinematics. An inertial term
from an IMU in the head was also considered. It is claimed that while pair-
wise calibration can lead to inconsistencies, calibrating everything together
in a “mutually supportive way” is the most efficient. Limoyo et al. [34] used
contact constraints from sliding on a surface together with RGB-D camera
information to formulate a self-calibration problem for a mobile manipulator
to estimate camera extrinsic parameters and manipulator joint angle biases.
The former part is also experimentally verified. Stepanova et al. [35]
systematically studied on the simulated iCub humanoid robot how self-
observation, self-contact, and their combination can be used for self-
calibration and evaluated the relative performance of these approaches by
varying the number of poses, initial parameter perturbation and measurement
noise. They found that employing multiple kinematic chains (“self-observation”
and “self-touch”) is superior in terms of optimization results as well as
observability.

Largely orthogonal to the calibration types mentioned above, observability and
identifiability of the system and speed of optimization convergence can be
improved by i) combination of geometric and parametric approaches to kinematic
identification [36], (ii) improving the error model by incorporating impact of
strain wave gearing errors [37] and unifying various error sources [38], or
iii) improving the optimization method [39].

In summary, existing works on self-contained automatic calibration have
typically focused on a single approach—relying on self-observation, physical
constraints, or self-contact. Birbach et al. [2] combined multiple sensors.
However, first, essentially only “self-observation” chains closed through
different vision-like sensors in the robot head were used. Second, only the
case where all chains were combined using a single cost function was
considered. In [34], self-observation and contact information are combined,
but the results have a proof-of-concept character. Our work is inspired by the
simulation study of Stepanova et al. [35] but presents a completely new
setting and results on a different platform. Using a dual-arm industrial robot
with force/torque sensing and cameras, we present several formulations of the
optimization problem for geometric kinematic calibration and empirical
verification using (i) self-contact, (ii) planar constraints, and (iii) self-
observation. The pros and cons of the individual methods and their synergies
are assessed and compared to an independent calibration using an industrial
quality laser tracker.

##  3 Experimental setup and data acquisition

In this section, we introduce the experimental setup: the robot platform and
its control, dimensions, and camera calibration. Then we present the process
of data acquisition and the structure of the collected datasets.

###  3.1 Robot setup description

For our experiments, we used a robotic platform developed in the CloPeMa
project [40] – Fig. 1 and Fig. 2. It consisted of two industrial manipulators
Yaskawa-Motoman MA1400 installed on top of a Yaskawa R750 robotic turntable,
which allows rotation of the two manipulators around the vertical axis, a
control unit, and two computers connected to a local network. Two Nikon D3100
cameras with Nikkor 18-55 AF-S DX VR lens were mounted side-by-side on the
turntable over the robot base, moving along with the turntable.

Figure 2: Setup of the whole dual arm robot: visualisation of individual links
and their location on the robot. DH parameters of these links are listed in
Table 1.

Manipulators were equipped with ATI Industrial Automation Mini45 6-axis
force/torque (F/T) sensors, placed between the last link of the manipulator
and the end effector. All the different parts of the robot system were
integrated into and operated with Robot Operating System (ROS) [41]. MoveIt!
planning framework was used to control the robot.

Figure 3: Custom end effector with icosahedron and laser tracker
retroreflector (left), its collision model in self-contact configuration ($d$
and $o$ are DH parameters of end effector and $r$ is the radius of
icosahedrons) (center), and spherical tiles used for contacts with their
reference frames (right).

We further equipped the robot with custom-made end effectors, one on each
robot arm, that can be used to achieve self-contact and, at the same time,
visual self-observation by the two cameras. To this end, we designed and 3D
printed icosahedron shapes (Figure 3 (left)) that have 20 flat faces, where
fiducial markers can be placed, interleaved with 10 spherical surface regions
that can be used for self-contact. The end effectors (further also referred to
as icosahedron) were then attached to a steel rod. The collision model (Fig.
3, center) for the end effector was added to the robot model. For calibration
employing self-contact or contact with a plane, the end effectors are treated
as spheres (see Fig. 3, center). The spherical surfaces with corresponding
reference frames are shown in Fig. 3, right.

For visual self-observation, having 20 evenly spaced ArUco markers should
ensure that at least 3 of them are always seen by each camera unless another
part of the robot occludes the view. We used the OpenCV ArUco module to detect
the markers in images.

###  3.2 Laser tracker setup

For acquiring reference parameter values and a baseline for self-contained
calibration, we used an external “laser tracker”, i.e. Leica Geosystems
Absolute Tracker AT402, which is a portable 3D absolute tracker for fully
guided measurement processes [42]. The tracker collects the 3D coordinates of
points in the coordinate system of the tracker. The measurement has a
resolution of 0.1 microns. The tracker was placed approximately 4 m from the
manipulator. Thus, the typical $U_{xyz}$ uncertainty of the measurement was
$\pm 7.5+3\times 4$ microns [43, 42].

The retroreflector was attached to the robot approximately 25 cm from the last
joint. During data collection, the distance from the laser tracker was between
2.5 and 4.5 m. The retroreflector and its collision model are shown in Figure
3 (left and center).

###  3.3 Robot control for contact configurations using force feedback

Driving the two manipulator arms into physical contact is necessary for
exploiting self-contact in robot calibration. A point in the workspace was
chosen where the end effectors should touch. Then, the configuration of each
arm and the movement trajectory were obtained using MoveIt!. Each contact
consists of three or four phases depending on the experiment. In the first
phase, the robot right arm moves at a speed of
$0.7$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$ to a point close to the desired
pose. In the self-contact experiment, this is followed by an analogous
movement of the left arm. Then the right arm starts moving to an anticipated
contact point (in fact, a small negative distance was used) with the left arm
or a plane at a speed of $0.1$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$ until the
collision is detected by F/T sensors. The contact thresholds were determined
empirically. Once the end effector gets into contact, the arm stops moving and
cameras are triggered to take photographs. Joint angles are recorded.
Afterwards, the right arm is slowly
($0.1$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$) moving to the former position
(from the end of the first phase). Then the new target is selected and the
whole process is repeated. For laser tracker experiments, the deceleration
phase is skipped and the robot arm chosen moves directly at a speed of
$0.7$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$$ in free space to predefined
configurations which were sampled in a way that a significant part of the
robot joint space is covered.

###  3.4 Robot and end effector dimensions and representation

A parametric model of the whole robot including the custom-made end effectors
was created. The robot’s complete kinematic model was described by Denavit-
Hartenberg (DH) notation, including cameras, end effectors, and laser tracker
retroreflector.

Initialization of parameters. To obtain the initial model, we used the nominal
parameters of the robot provided by the manufacturer, previously acquired
transformation matrices describing the mounting of the robot on the base, CAD
model and manual measurements of the custom end effector and laser tracker
retroreflector placement (the link EEL1/2 (7b) represents the transformation
to the retroreflector from the last robot joint – see Fig. 2, Fig. 3, and
Table 1), and manual measurement of the cameras’ attachment. Using the
Denavit-Hartenberg (DH) notation [44], Table 1 shows parameters of the
manipulators, including the custom end effector or the retroreflector
placement. Table 2 shows the two links to every camera.

| Manipulator 1 (right arm) | Manipulator 2 (left arm)
---|---|---
Link | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad] | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad]
0 | TT1 | 0 | -0.263 | $0.262$ | $-1.571$ | TT2 | 0 | -0.263 | $-0.262$ | $-1.571$
1 | S1 | 0.150 | 1.416 | $-1.571$ | 0 | S2 | 0.150 | 1.416 | $-1.571$ | $0$
2 | L1 | 0.614 | 0 | $3.142$ | $-1.571$ | L2 | 0.614 | 0 | $3.142$ | $-1.571$
3 | U1 | 0.200 | 0 | $-1.571$ | 0 | U2 | 0.200 | 0 | $-1.571$ | 0
4 | R1 | 0 | -0.640 | $1.571$ | 0 | R2 | 0 | -0.640 | $1.571$ | 0
5 | B1 | 0.030 | 0 | $1.571$ | $-1.571$ | B2 | 0.030 | 0 | $1.571$ | $-1.571$
6 | T1 | 0 | 0.200 | 0 | 0 | T2 | 0 | 0.200 | 0 | 0
7a | EE1 | 0 | 0.350 | 0 | 0 | EE2 | 0 | 0.350 | 0 | 0

| Laser tracker: Manipulator 1 (right arm) | Laser tracker: Manipulator 2 (left arm)
---|---|---
Link | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad] | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad]
6 | T1 | 0 | 0.200 | 0 | 0 | T2 | 0 | 0.200 | 0 | 0
7b | EEL1 | 0.02 | 0.250 | 0 | $1.571$ | EEL2 | 0.020 | 0.250 | 0 | $1.571$

Table 1: Complete initial DH parameter description of both arm kinematic chains. 7a is the last link to the icosahedron end effector; 7b is the link to the laser tracker retroreflector. The parameters with gray shading were not calibrated unless otherwise stated (when calibrating using the laser tracker, we do not calibrate the $6$th link). The individual links are visualized in Fig. 2. | Camera 1 | Camera 2
---|---|---
Link | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad] | Link name | a [m] | d [m] |  $\boldsymbol{\alpha}$ [rad] |  $\boldsymbol{o}$ [rad]
0 | TT3 | 0.2315 | 1.8034 | -2.5086 | -2.7753 | TT4 | 0.2315 | 1.8602 | 2.5486 | -0.0860
1 | C1 | 0 | -0.5670 | 0 | 0.2863 | C2 | 0 | -0.4982 | 0 | 3.0618
Table 2: Initial DH parameters of camera chains. Parameters with gray shading
were not subject to calibration unless otherwise stated.

The manipulator has 6 actuated rotational joints, denoted ${S}$, ${L}$, ${U}$,
${R}$, ${B}$ and ${T}$ (in italics), connected by links which are denoted S,
L, U, R, B, and T in the order from the turntable to the end effector. Joint
${S}$ connects the turntable with link S of the robot; joint ${L}$ connects
link S with link L, and so on – see Fig. 2. We added one last link to every
manipulator’s kinematic chain. These links represent the transformation to the
end effector (icosahedron) (EE1, EE2) or retroreflector (EEL1, EEL2). Links
between the origin and the turntables (TT1 and TT2) were added as well. The
camera chain was also expressed using DH representation and is composed of 2
links: the link between the origin and the camera turntable (TT3, TT4) and the
link to the camera entrance pupil (C1, C2). These last links (EE1, EE2, EEL1,
EEL2, C1, C2) are not connected by an actual joint, and thus their joint angle
was set to zero. These joints may still have a non-zero offset $o$ though.

We consider the complete kinematic system of the robot platform to be composed
of four individual kinematic chains described by the DH parameters: (1) Right
arm (TT1, S1, L1, U1, R1, B1, T1, EE1), (2) Left arm (TT2, S2, L2, U2, R2, B2,
T2, EE2), (3) Right camera (TT3, C1) and (4) Left camera (TT4, C2).
Additionally, we consider kinematic chains for laser tracker measurement (5)
Right arm (TT1, S1, L1, U1, R1, B1, T1, EEL1), and (6) Left arm (TT2, S2, L2,
U2, R2, B2, T2, EEL2). All the kinematic chains start in the base frame. The
transformation from the base frame to the first joint, i.e. to the turntable
joint, is identity. For calibration using the laser tracker, we replaced the
icosahedron end effector links with links ending in the retroreflectors (EEL1,
EEL2) (see Fig. 2, Table 1 for robot DH parameters, and Table 2 for camera DH
parameters). The left and right arm chains finish with the end effectors; the
last frame is in the center of each icosahedron or in the laser tracker
retroreflector. Note that since the mounting of the two manipulators on the
turntable is not identical, the first 4-tuple of DH parameters is also not the
same; we recognize two distinct turntable links in the arm chains and two more
turntable links in the camera chains. All the chains have the same rotation of
the first joint, but all other joints and parameters are independent.

###  3.5 Camera calibration

The calibration of the intrinsic parameters of the cameras was carried out
with a dot pattern. The dataset used for calibration was composed of 22
pattern images. Each of the captured images had a different position and
orientation w.r.t. cameras. Calibration of the camera matrix $K$ and
distortion coefficients vector $\mathbf{d}$ was performed using OpenCV camera
calibration function calibrateCamera [45], and the following pinhole camera
model extended with radial and tangential distortion coefficients:

$\begin{split}x_{c}^{\prime}=x_{c}/z_{c},&\qquad
y_{c}^{\prime}=y_{c}/z_{c},\qquad r=\sqrt{x_{c}^{\prime 2}+y_{c}^{\prime
2}}\\\
x_{c}^{\prime\prime}=&x_{c}^{\prime}(1+d_{1}r^{2}+d_{2}r^{4}+d_{3}r^{6})+\\\
&2d_{4}x_{c}^{\prime}y_{c}^{\prime}+d_{5}(r^{2}+2x_{c}^{\prime 2})\\\
y_{c}^{\prime\prime}=&y_{c}^{\prime}(1+d_{1}r^{2}+d_{2}r^{4}+d_{3}r^{6})+\\\
&d_{4}(r^{2}+2y_{c}^{\prime 2})+2d_{5}x_{c}^{\prime}y_{c}^{\prime}\\\
\begin{bmatrix}u\\\ v\\\
1\end{bmatrix}=&K\begin{bmatrix}x_{c}^{\prime\prime}\\\
y_{c}^{\prime\prime}\\\ 1\end{bmatrix}\end{split}$ (1)

where $[x_{c},y_{c},z_{c}]$ is a 3D point in camera frame, $[u,v]$ are the
image coordinates.

New camera matrices $K$ and the distortion coefficients $\mathbf{d}_{R}$ and
$\mathbf{d}_{L}$ for right and left camera, respectively, were found as:

$\displaystyle K_{R}=\begin{bmatrix}8185.397&0.000&2009.318\\\
0.000&8170.401&2963.960\\\ 0.000&0.000&1.000\end{bmatrix},$ $\displaystyle
K_{L}=\begin{bmatrix}8110.478&0.000&1949.921\\\ 0.000&8098.218&2991.727\\\
0.000&0.000&1.000\end{bmatrix},$ (2)
$\begin{split}\mathbf{d}_{R}&=[\begin{matrix}-0.020602&-0.205606&-0.001819\end{matrix}\\\
&\qquad\qquad\begin{matrix}-0.000820&0.718890\end{matrix}]\end{split}$
$\begin{split}\mathbf{d}_{L}&=[\begin{matrix}-0.022546&-0.213094&-0.000684\end{matrix}\\\
&\qquad\qquad\begin{matrix}-0.000512&0.662333\end{matrix}]\end{split}$ (3)

The distortion coefficients are presented here in the order as they are
returned from OpenCV camera calibration function calibrateCamera [45].

###  3.6 Data acquisition and description

An accompanying video illustrating the experiments is available at
https://youtu.be/LMwINqA1t9w. There were 5 distinct datasets collected: (1)
self-contact/self-touch experiment (both robot end effectors get into
contact), (2) contact with a lower and upper horizontal plane, (3) contact
with a vertical plane (“wall”), (4) repeatability measurement, and (5) laser
tracker dataset. Datasets were acquired using a simple GUI applet [46]. Laser
tracker data were also recorded for parts of self-contact and planar contact
datasets (not used in this paper but included in the published dataset) [3].

Individual datasets. The whole dataset $\boldsymbol{D}^{whole}$ is a set of
individual datasets:

$\boldsymbol{D}^{whole}=\\{\boldsymbol{D}^{st},\boldsymbol{D}^{hp},\boldsymbol{D}^{vp},\boldsymbol{D}^{lt}\\},$

where $\boldsymbol{D}^{st}$, $\boldsymbol{D}^{hp}$, $\boldsymbol{D}^{vp}$, and
$\boldsymbol{D}^{lt}$ are datasets collected during self-contact/self-touch
experiment (st), contact with horizontal planes (hp), contact with a vertical
plane (vp), and laser tracker experiments (lt), respectively.

The world coordinate system is a right-handed Cartesian coordinate system with
axes denoted by $x$, $y$, $z$. Units are meters; the origin is in the center
of the robot base on the floor, the $y$-axis points behind the robot, and
$z$-axis points up. Euler angles refer to the orientation of the end effector
with respect to a coordinate system $x^{\prime}$, $y^{\prime}$, $z^{\prime}$
attached to a moving body (end effector). In our notation, $\alpha$, $\beta$,
and $\gamma$ represent the first, second, and third rotation, respectively.
The rotation axis is indicated by the subscript ($x$, $y$, $z$ and
$x^{\prime}$, $y^{\prime}$, $z^{\prime}$ denote the fixed and current axes,
respectively).

Dataset |  Contact positions |  Orientations |  No. datapoints
---|---|---|---
Self-touch ($\boldsymbol{D}^{st}$) |  4$\times$4$\times$2 grid in the $xyz$-box with $x\in[-0.3,0.2]$\mathrm{m}$$, $y\in[-1.1,-0.6]$\mathrm{m}$$ and $z=\\{0.8,\,1\\}$\mathrm{m}$$ |  9 combinations of right and left arm orientations – see Fig. 4 |  566 datapoints (10 not logged)
Horizontal planes ($\boldsymbol{D}^{hp}$) |  5$\times$5 grid in the $xy$-plane with $x\in[-0.4,0.3]$\mathrm{m}$$, $y\in[-1.35,-0.65]$\mathrm{m}$$, and $z=\\{0.67,\,85\\}$\mathrm{m}$$ |  5 orientations in Euler angles $(\alpha_{x},\beta_{y},\gamma_{z})$: {$(0,185,52)$, $(0,180,-20)$, $(0,180,124)$, $(0,180,-164)$, $(0,180,-92)\\}$ |  454 datapoints (48 not logged)
Vertical plane ($\boldsymbol{D}^{vp}$) |  5$\times$5 grid in the $yz$-plane with $y\in[-1.2,-0.7]\text{ m}$, $z=[0.7,1.1]\text{ m}$, and $x=0.05\text{ m}$ |  5 orientations in Euler angles $(\alpha_{z},\beta_{y},\gamma_{z})$: {$(180,90,0)$, $(108,90,0)$, $(36,90,0)$, $(-36,90,0)$, $(-108,90,0)\\}$ |  248 datapoints (2 not logged)
Repeatability measurement |  st $[x,y,z]^{T}=[0.225,-0.85,0.8]^{T}$\mathrm{m}$$ $[x,y,z]^{T}=[-0.025,-0.85,1]^{T}$\mathrm{m}$$ |  right arm $(\alpha_{z},\beta_{y},\gamma_{z}):(-108,90,0)$ left arm $(\alpha_{z},\beta_{y},\gamma_{z}):(0,-90,0)$ |  20 datapoints
hp $[x,y,z]^{T}=[0.3,-0.825,0.67]^{T}$\mathrm{m}$$ $[x,y,z]^{T}=[-0.4,-0.825,0.67]^{T}$\mathrm{m}$$ |  right arm $(\alpha_{x},\beta_{y},\gamma_{z}):(0,185,52)$ |  20 datapoints
vp $[x,y,z]^{T}=[0.17,-0.825,1.1]^{T}$\mathrm{m}$$ $[x,y,z]^{T}=[0.17,-0.95,0.7]^{T}$\mathrm{m}$$ |  right arm $(\alpha_{z},\beta_{y},\gamma_{z}):(-108,90,0)$ |  20 datapoints
Laser tracker ($\boldsymbol{D}^{lt}$) |  whole configuration space of the manipulator was sampled (see text for details) |  - |  586 datapoints (99 not logged)
Table 3: Dataset overview.

Contact points of end effectors in datasets $\boldsymbol{D}^{st}$,
$\boldsymbol{D}^{hp}$, and $\boldsymbol{D}^{vp}$ were planned on grids in the
manipulators’ workspace, as shown in Fig. 4 and Fig. 5 for the self-contact
and contact with planes, respectively. Several end effector orientations were
tested for every position. A detailed description of individual datasets with
contact points and orientations is provided in Table 3. Every experiment was
repeated twice; a few configurations could not be reached due to robot motion
planner failure. In all cases, photographs of one or both icosahedrons in
every pose were taken and added to the dataset. In addition, for every setup
(self-touch, horizontal and vertical plane), 20 repetitions in 2 different
positions were performed to evaluate the measurements’ repeatability
(Repeatability measurement dataset). For kinematic calibration, the
distribution of robot joint angles is important. It is visualized in Fig. 6
for the different experiments.

Figure 4: Positions of end effector centers during the self-contact
experiment. The end of each arrow denotes a position of end effector center in
individual poses (red – left arm, blue – right arm). The distance between left
and right arm position is given by two times the radius of the end effectors
in contact.

Figure 5: Positions of end effector centers in the horizontal plane contact
datasets (left, higher plane in blue, lower plane in green) and vertical plane
contact dataset (right) with the information whether the pose was logged by
laser tracker (blue) or not (red).

Figure 6: Distributions of robot joint angles (S1,L1,U1,R1,B1,T1) for measured
poses for the right arm across different experiments (only the right arm was
evaluated in all experiments). “Planar datasets” subfigure compares the
vertical plane contact in blue with the horizontal plane contact in red.
“Whole dataset” subfigure compares the self-touch experiment in red with the
combination of all planar setups (2 horizontal + 1 vertical plane) in blue.

Laser tracker experiment ($\boldsymbol{D}^{lt}$). We sampled the whole range
of the first 4 joints of the manipulator and added uniform noise to the values
to cover the whole range of joint angles. The joint angles of the last two
joints were set so that the retroreflector faced the laser tracker.
Configurations which would be in collision with the robot or the surrounding
environment were excluded. This resulted in 685 configurations for the right
arm, out of which 586 poses were actually recorded by the laser tracker ((see
Fig. 7; Fig. 8 for joint space distribution)).

Figure 7: Positions of the retroreflector in the Laser tracker dataset (right
arm) with information whether the pose was additionally logged by the tracker
or not (blue – captured by the tracker (586 poses), red – not captured by the
tracker (99 poses) due to the poor visibility of the retroreflector). Figure
8: Distribution of joint angles for Laser tracker dataset (right arm).

Dataset structure. The whole dataset $\boldsymbol{D}^{whole}$ contains $M$
dataset points (in fact, they are row vectors):
$\boldsymbol{D}^{whole}=\\{\boldsymbol{D}_{1},\dots,\boldsymbol{D}_{M}\\}$.
Each dataset point $\boldsymbol{D}_{j}$ of the dataset
$\boldsymbol{D}^{whole}$ consists of the assumed pose of right and left
icosahedron centers $\boldsymbol{x}^{I,r}$ and $\boldsymbol{x}^{I,l}$,
respectively (computed from forward kinematics); the joint configuration of
the robot
$\boldsymbol{\theta}=\\{\theta^{ra}_{1},...,\theta^{ra}_{N},\theta^{la}_{1},...,\theta^{la}_{N}\\}$
($ra$ and $la$ denoting right and left arm, respectively); coordinates of
every marker ($K$ being the number of markers) in each of the cameras
$\boldsymbol{u}=\\{\boldsymbol{u}^{rc}_{1},...,\boldsymbol{u}^{rc}_{K},\boldsymbol{u}^{lc}_{1},...,\boldsymbol{u}^{lc}_{K}\\}$
($rc$ and $lc$ denoting right camera and left camera, respectively); position
of laser tracker ball retroreflector in the tracker coordinate system
including uncertainties U95 [42] of position detection and the tracker
measurement timestamp $\boldsymbol{L}=\\{\mathbf{x}^{L},u^{L},t^{L}\\}$. We
also saved for reference the magnitude of force measured by both force sensors
before ($F_{b}$) and during ($F_{a}$) the contact for each arm
$\boldsymbol{F}=\\{F^{ra}_{b}$, $F^{ra}_{a}$, $F^{la}_{b}$, $F^{la}_{a}\\}$;
and the names of saved camera images $n^{r},n^{l}$. Individual dataset points
are organized in a matrix, where each line $i$ of the matrix corresponds to
the individual dataset point:

$\begin{split}&\boldsymbol{D}^{whole}=[\boldsymbol{D}_{1},\dots,\boldsymbol{D}_{M}]^{T},\mathrm{where:}\\\
&\boldsymbol{D}_{i}=[\boldsymbol{x}^{I,r}_{i},\boldsymbol{x}^{I,l}_{i},\boldsymbol{\theta}_{i},\boldsymbol{F}_{i},\boldsymbol{L}_{i},n^{r},n^{l},\boldsymbol{u}_{i}]^{T}.\end{split}$
(4)

The whole dataset $\boldsymbol{D}^{whole}$ contains $M=1268$ logged poses with
23022 marker reprojections in total (12371 from the right camera and 10651
from the left camera), which makes approx. 25 marker reprojection per pose for
self-contact experiment and 11 marker reprojections per pose for planar
constraints. There are also 586 poses acquired by the laser tracker—this
dataset does not contain marker reprojections. The reprojections are sorted
from the lowest to the highest marker ID for every camera image. If a marker
was not found in the image, its coordinates are denoted as $(NaN,NaN)$.
Similarly, if data were not measured or captured by Leica tracker, all
corresponding values are filled with $NaN$.

For optimization, we transformed the dataset so that one line would relate to
one data point. That is, for a single robot configuration with multiple marker
reprojections, the corresponding datapoint is unfolded into multiple rows—1
per marker detected. A number defining the robot configuration ($i$ from the
original dataset $\boldsymbol{D}^{whole}$) is repeated on every line. Thus,
one line consists of a number defining the robot pose, a face number of the
detected marker, index of the arm (1 for right or 2 for left), index of the
camera (1 for right or 2 for left), position of the marker center in the
camera $\boldsymbol{u}=(u,v)$ in [px], and the current robot joint
configuration (turntable, S1, L1, U1, R1, B1, T1, S2, L2, U2, R2, B2, T2) in
[rad], position of the tracker ball retroreflector in the tracker coordinate
system, and uncertainty U95 of the measurement.

The dataset and its description can be downloaded from [3]. For the positions
including LEICA measurements, the appropriate csv file with ($x$,$y$,$z$)
positions detected by the laser tracker scanner are available.

##  4 Multi-chain robot calibration

In multi-chain robot calibration we estimate parameter vector
${\boldsymbol{\phi}}=\\{[a_{1},...,a_{n}],[d_{1},...,d_{n}],[\alpha_{1},...,\alpha_{n}],[o_{1},...,o_{n}]\\}$
with $k\in N$, where $N=\\{1,..,n\\}$ is a set of indices identifying
individual links; $a_{k}$, $d_{k}$ and $\alpha_{k}$ are the first three
parameters of the DH formulation of link $k$; $o_{k}$ is the offset that
specifies the positioning of the encoders on the joints with respect to the DH
representation. This is sometimes referred to as static geometric calibration.
We often estimate a subset of these parameters only, assuming that the others
are known. This subset can for example consist of a subset of links
$N^{\prime}\subset N$ (e.g., only parameters of one arm are to be calibrated)
or a subset of the link parameters. Here we focus on offsets in the revolute
joints $\mathbf{o}$ (sometimes dubbed “daily calibration” [47]).

Let $\boldsymbol{D}\subset\boldsymbol{D}^{whole}$ denote the set of robot
configurations (dataset points) used for optimization:

$\boldsymbol{D}_{i}=[m_{i},c_{i},\boldsymbol{u}_{i},\boldsymbol{\theta}_{i}]$

where $i\in\\{1,..,M^{\prime}\\}$ is an index identifying one particular
dataset point, $M^{\prime}$ is the number of dataset points used for
optimization, $m_{i}$ is face number of the detected marker, $c_{i}$ is the
index of the used camera, $\boldsymbol{u}_{i}=(u_{i},v_{i})$ is the position
of the marker center in the camera, and $\boldsymbol{\theta}_{i}$ is the
current robot joint configuration (joint angles from joint encoders for the
given robot configuration).

Estimation of the parameter vector $\boldsymbol{\phi}$ is done by optimizing a
given objective function
$\boldsymbol{f}(\boldsymbol{\phi},\boldsymbol{D},\boldsymbol{\zeta})$:

$\displaystyle\boldsymbol{\phi}^{*}=$
$\displaystyle\operatorname*{arg\,min}_{\boldsymbol{\phi}}\boldsymbol{f}(\boldsymbol{\phi},\boldsymbol{D},\boldsymbol{\zeta}),$
(5)
$\displaystyle\boldsymbol{f}(\boldsymbol{\phi},\boldsymbol{D},\boldsymbol{\zeta})=$
$\displaystyle\|\boldsymbol{g}(\boldsymbol{\phi},\boldsymbol{D},\boldsymbol{\zeta})\|^{2}=\sum_{i=1}^{M^{\prime}}g(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})^{2},$
(6)

where $M^{\prime}$ is the number of robot configurations and corresponding end
effector positions used for calibration (hereafter often referred to as
“poses” for short), ${\boldsymbol{\phi}}$ is a given parameter estimate,
dataset point $\boldsymbol{D}_{i}$ includes joint angles
${\boldsymbol{\theta}}_{i}$ from joint encoders for the given robot
configuration, and constant vector $\boldsymbol{\zeta}$ defines all other
necessary parameters such as camera calibration, fixed transformations, fixed
DH parameters, or other properties of the robot. For chains involving cameras,
the function ${g}(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})$
refers to reprojection error as described in the next section while for the
chains including contact, it corresponds to the distance between a real
(observed) end effector position ${\boldsymbol{p}}_{i}^{r}$ and an estimated
end effector position ${\boldsymbol{p}}_{i}^{e}$ computed using forward
kinematic function for a given parameter estimate ${\boldsymbol{\phi}}$ and
joint angles from joint encoders ${\boldsymbol{\theta}}_{i}$. For the case of
planar constraints, the shortest distance of end effector position
(icosahedron center) to the parameterized plane is minimized.

We study different combinations of intersecting chains and their performance
in calibrating one another. Specific form of the function
$g(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})$ for individual
considered chains and their combinations is discussed in the following
subsections. At the end of every subsection, we also state how many components
of the pose can be measured using the different methods—similarly to the
analysis for standard open-loop and closed-loop calibration approaches in [1].

For problem representation, optimization, and some visualization, we employed
the multisensorial calibration toolbox [48].

###  4.1 Self-contact – two arms chain (LA-RA)

This corresponds to the self-contact scenario in which contact occurs directly
between the end effectors of the Left Arm (LA) and Right Arm (RA). As
described in Section 3.1, for contact, the end effectors can be treated as
spheres and the contact can occur between any of the 10 spherical tiles. The
newly established kinematic chain for the upper body includes both arms; the
head and eyes are excluded (see Fig. 9). To optimize parameters describing
this chain, we minimize the distance between estimated positions in the 3D
space of left and right arm end effectors (see Fig. 10).

Figure 9: Self-contact experiment. The left and right arm chains are drawn in
purple and green, respectively. Cyan indicates the distance between end
effector centers.

Figure 10: Visualisation of self-contact scenario. When end effectors are in
contact, they are separated by distance $d$. This can happen in multiple
configurations. For simplicity, we visualize the case when only end effector
link length is estimated for one arm (left) or both arms (right), with the
additional assumption that the link length is the same for both arms.

In this case, the parameter vector ${\boldsymbol{\phi}}$ consists of the
following parameters:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{ra},{\boldsymbol{\phi}}^{la}\\}$,
where ${\boldsymbol{\phi}}^{ra}$ and $\boldsymbol{\phi}^{la}$ are DH
parameters being calibrated corresponding to the robot right and left arm,
respectively. The objective function to be optimized is:

$\begin{split}\boldsymbol{g^{st}}(\boldsymbol{\phi},\boldsymbol{D}^{st,u},\zeta)=&[c(\boldsymbol{\phi},\boldsymbol{D}_{1},\boldsymbol{\zeta})-q(\boldsymbol{\zeta}),...,\\\
&c(\boldsymbol{\phi},\boldsymbol{D}_{M},\boldsymbol{\zeta})-q(\boldsymbol{\zeta})]\end{split}$
(7)

where the function
$c(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})$ computes the
distance of the end effector centers in the configuration given by the dataset
point $\boldsymbol{D}_{i}$, where
$\boldsymbol{D}_{i}\in\boldsymbol{D}^{st,u}$,
$\boldsymbol{D}^{st,u}\subset\boldsymbol{D}^{st}$ is a subset of the dataset
points with a particular robot configuration (see Dataset structure in Sec.
3.6 for details). The distance of the end effector centers, marked as
$q(\boldsymbol{\zeta})$, is equal to one icosahedron diameter $2r$, because
both end effectors have identical shape. For the icosahedron diameter, we took
the value from CAD model of $2r=116$ mm and kept it fixed [46].

As can be seen, the objective function contains a set of constraints on the
distances between the end effector positions. These constraints can be written
for the case of self-contact as follows: Let $(x^{r}_{i},y^{r}_{i},z^{r}_{i})$
and $(x^{l}_{i},y^{l}_{i},z^{l}_{i})$, be the centre of right and left arm end
effector computed from forward kinematics for data point $i$ with the given
joint configuration $\boldsymbol{\theta}_{i}$, respectively. Then for each
data point $\boldsymbol{D}_{i}$ in the dataset, we have the following
constraint (left side of the equation corresponds to
$c(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})$ and right side to
$q(\boldsymbol{\zeta})$ in Eq. 7):

$\sqrt{(x^{l}_{i}-x^{r}_{i})^{2}+(y^{l}_{i}-y^{r}_{i})^{2}+(z^{l}_{i}-z^{r}_{i})^{2}}=2r$
(8)

According to [1], this corresponds to restricting 1 position parameter. The
parameters of the touched surface ($2r$) are in this case known—same as for
planar constraints with known plane parameters.

###  4.2 Planar constraints – one arm chain in contact with a plane (LA/RA)

Figure 11:  Contact with planar constraints: horizontal plane (left), vertical
plane (right).

This corresponds to the scenario where the Left (LA) or Right (RA) robotic arm
is getting into contact with a plane (see Fig. 11). In this type of
optimization problem, we can distinguish formulations including single-plane
or multiple-plane constraints [24, 25]. The classical formulations of the
problem use either a general equation of the constraint plane or plane normals
[23]. The general equation of a plane is:

$ax+by+cz+d=0,$ (9)

where $\mathbf{n}=(a,b,c)$ is a plane normal vector. The parameters of the
plane can be known in advance (as in [23], [26] or [20] where calibration cube
is used), or unknown (as in our case or [24]). The planes and their
corresponding parameters are visualized in Fig. 12.

Figure 12: Visualisation of planes with their corresponding parameters
($\boldsymbol{n}^{hp}$, $d^{hp}$ and $\boldsymbol{n}^{vp}$, $d^{vp}$) and
coordinate frames. End effector with radius $r$ in contact with horizontal
plane in multiple places is shown.

When the parameters of the plane are unknown, the parameter vector
$\boldsymbol{\phi}$ consists of the following parameters:
$\boldsymbol{\phi}=\\{\boldsymbol{\phi}^{ra/la},\mathbf{n},d\\}$, where
$\boldsymbol{\phi}^{ra/la}$ are the DH parameters of the robot arm in contact,
$\mathbf{n}$ is a plane normal, and $d$ is the distance of the plane from the
origin. We formulated the objective function as the distances between contacts
and a single or multiple fitted planes:

$\boldsymbol{g}^{p}(\boldsymbol{\phi}^{p},\boldsymbol{D}^{p},\boldsymbol{\zeta})=\left[\begin{matrix}\mathbf{c}(\boldsymbol{\phi}^{hp1},\boldsymbol{D}^{hp1},\boldsymbol{\zeta})-r,\end{matrix}\right.\\\
\left.\begin{matrix}\;\mathbf{c}(\boldsymbol{\phi}^{hp2},\boldsymbol{D}^{hp2},\boldsymbol{\zeta})-r,\mathbf{c}(\boldsymbol{\phi}^{vp},\boldsymbol{D}^{vp},\boldsymbol{\zeta})-r\end{matrix}\right],\
\ \ \ \ \ \ \ \ \ \ \ $ (10)

where $\boldsymbol{D}^{p}\subset\boldsymbol{D}^{whole}$,
$\boldsymbol{D}^{p}=\\{\boldsymbol{D}^{hp1},\boldsymbol{D}^{hp2},\boldsymbol{D}^{vp}\\}$
is a set of datasets where contacts between the end effector and lower
horizontal plane ($\boldsymbol{D}^{hp1}$), higher horizontal plane
($\boldsymbol{D}^{hp2}$) or vertical plane ($\boldsymbol{D}^{vp}$) were
performed. The set
$\boldsymbol{\phi}^{p}=\\{\boldsymbol{\phi}^{hp1},\boldsymbol{\phi}^{hp2},\boldsymbol{\phi}^{vp}\\}$
is a set of parameters for lower horizontal plane ($\boldsymbol{\phi}^{hp1}$),
higher horizontal plane ($\boldsymbol{\phi}^{hp2}$), and vertical plane
($\boldsymbol{\phi}^{vp}$), respectively. The vector
$\mathbf{c}(\mathbf{\boldsymbol{\phi}}^{j},\boldsymbol{D}^{j},\boldsymbol{\zeta})$
is a vector of distances between individual end effector positions and the
given plane j for each datapoint $\boldsymbol{D}^{j}_{i}$ from the given
dataset $\boldsymbol{D}^{j}$. The distance is computed using plane normals and
corresponding plane coordinates as follows:
$c(\boldsymbol{\phi}^{j},\boldsymbol{D}^{j}_{i},\boldsymbol{\zeta})=||\boldsymbol{n}^{j}\boldsymbol{p}^{j}_{i}(\boldsymbol{\phi}^{j,ra/la})+d||$.
Point $\mathbf{p}^{j}_{i}$ is the centre of the end effector computed by
forward kinematics from dataset point $\boldsymbol{D}^{j}_{i}$;
$\boldsymbol{\phi}^{j,ra/la}$ is the estimated parameter vector corresponding
to the DH parameters of the touching arm; $\mathbf{n}^{j}$ $=[a\ b\ c]$ is the
plane normal; $d^{j}$ is the distance of the plane from the origin. These
plane parameters ($\mathbf{n}^{j}$ and $d^{j}$) are estimated at each
iteration of the optimization process based on current point coordinates
estimates by SVD method as described below. The $\boldsymbol{\zeta}$ wraps up
all other necessary parameters.

To acquire parameters of the fitted plane in each iteration, the measured
points are converted to homogeneous coordinates and their centre of gravity is
computed and subtracted from all points. Afterwards, Matlab function _SVD_ is
called. The singular vector corresponding to the smallest singular value is
set as a normal of the plane. Parameter $d$ in Equation 9 is calculated from:

$d=-ax_{0}-by_{0}-cz_{0},$ (11)

where $(x_{0},y_{0},z_{0})$ are coordinates of the points center.

Let $\boldsymbol{n}=[a,b,c]$ be a plane normal, $d$ the distance of the plane
from the origin, $(x^{r}_{i},y^{r}_{i},z^{r}_{i})$ the centre of the right arm
end effector computed from forward kinematics for data point
$\boldsymbol{D}_{i}$ with the given joint configuration $\theta_{i}$, and $r$
the radius of the icosahedron. Then for each data point $\boldsymbol{D}_{i}$
in the dataset $\boldsymbol{D}^{hp/vp}$ we get the following constraint (left
side of the equation corresponds to
$\mathbf{c}(\boldsymbol{\phi}^{vp/hp},\boldsymbol{D}^{vp/hp},\boldsymbol{\zeta})$
in Eq. 10):

$\sqrt{(ax^{r}_{i}+by^{r}_{i}+cz^{r}_{i}+d)^{2}}=r$ (12)

As per [1], this type of calibration corresponds to the restriction of 1
position parameter. Compared to the self-contact setup, in the case of planes
with unknown parameters (our case), new calibration parameters (parameters of
the plane $\boldsymbol{n}$ and $d$) have to be added.

###  4.3 Self-observation by cameras (LA-LEye, LA-REye, RA-LEye, RA-REye)

This corresponds to the scenario where we observe the Left Arm (LA) or Right
Arm (RA) end effector with AruCo markers via Left Camera (LEye) or Right
Camera (REye) (see Fig. 13). We calibrate: (i) extrinsic parameters of the
cameras (in our case as DH links) while assuming the robot DH parameters to be
known, or (ii) the whole kinematic chain of the robot arm simultaneously with
camera extrinsic parameters. In this case, the optimization is done by
minimizing the reprojection error between the observed AruCo markers’
positions in the camera and the estimated position using the current estimated
kinematic model.

Figure 13:  Self-observation chains: Red denotes reprojection from individual
AruCo markers to left and right camera.

To obtain projected marker positions (which determine end effector position)
in each of the robot cameras, we apply the calibrated camera model. The camera
was precalibrated using a standard camera intrinsic calibration (see Section
3.5). The extrinsic parameters (in our case expressed in DH parameters) of the
camera were precalibrated based on reprojections of AruCo markers using the
fixed nominal parameters of the arm. First, we have to transform marker
positions to the camera frame:

$\begin{bmatrix}x_{c}&y_{c}&z_{c}&1\end{bmatrix}^{T}=\boldsymbol{T}_{m}^{camera}\cdot\begin{bmatrix}0&0&0&1\end{bmatrix}^{T},$

where $[x_{c},y_{c},z_{c},1]^{T}$ are homogeneous coordinates of marker
position in the frame of the given camera and $\boldsymbol{T}_{m}^{camera}$ is
a transformation from the marker $M_{m}$ to the given camera achieved through
standard approach for forward kinematics using DH parameters. Afterwards, we
apply a standard pinhole camera model extended with radial and tangential
distortion coefficients and transform the 3D point in camera frame
($[x_{c},y_{c},z_{c}]$) into image coordinates $[u,v]$ (2D plane of the
camera) (see camera model in Sec. 3.5). The actual marker position means the
center of the ArUco marker. The OpenCV function calibrateCamera provides
calibration with resolution of whole pixels. ArUco marker detection is done by
the OpenCV function cv2.aruco.detectMarkers, which provides the coordinates of
all four marker corners. From these, the center of the marker is calculated as
an intersection of the diagonals connecting the marker corners. The error
resulting from this assumption is smaller than the calibration error [46].

The parameter vector ${\boldsymbol{\phi}}$ consists of the following
parameters:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{ra/la},\boldsymbol{\phi}^{rc/lc}\\}$,
where ${\boldsymbol{\phi}}^{ra}$, $\boldsymbol{\phi}^{la}$,
$\boldsymbol{\phi}^{rc}$, and $\boldsymbol{\phi}^{lc}$ are calibrated DH
parameters corresponding to the right arm, left arm, right camera, and left
camera, respectively. The objective function is formulated as the distance
between projected markers and their pixel coordinates in the images:

$\boldsymbol{g}^{so}(\boldsymbol{\phi},\boldsymbol{D}^{so},\boldsymbol{\zeta})=\left[\begin{matrix}p(\boldsymbol{\phi},\boldsymbol{D}_{1},\boldsymbol{\zeta})-{z}(\boldsymbol{D}_{1}),...,\end{matrix}\right.\\\
\left.\begin{matrix}p(\boldsymbol{\phi},\boldsymbol{D}_{M}^{\prime},{\zeta})-{z}(\boldsymbol{D}_{M}^{\prime})\end{matrix}\right]$
(13)

where $p(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})$ is the
reprojection of marker $m_{i}$ from dataset point $\boldsymbol{D}_{i}$ where
$i\in\\{1,...,M^{\prime}\\}$, $M^{\prime}$ is the length of the dataset
$\boldsymbol{D}$. The ${z}(\boldsymbol{D}_{i})$ is the actual marker position
in the camera image.

This approach does not require information from both eyes and enables us to
estimate only one side of the robot body (e.g., parameters of the left arm and
left camera). For example, the estimated parameter vector $\boldsymbol{\phi}$
in the case of the kinematic chain connecting left arm and left camera
consists of the following parameters:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{l},{\boldsymbol{\phi}}^{lc}\\}$,
where ${\boldsymbol{\phi}}^{l}$ and ${\boldsymbol{\phi}}^{lc}$ are parameters
corresponding to the robot left arm and to the left camera, respectively.

The conditions in the objective function can be expressed as follows: Let
$\boldsymbol{x}^{m}$ be a centre of origin of marker $m_{i}$ (obtained by
forward kinematics for the given DH parameters estimate), $\boldsymbol{x}^{c}$
be a marker position in the given camera frame, $\boldsymbol{T}^{EEF}_{m_{i}}$
be a transformation from marker $m_{i}$ to the arm end effector (transforms to
individual ArUco markers can be found at the dataset page at [3]) and
$\boldsymbol{T}^{camera}_{EEF}$ transformation from the end effector to the
given camera frame, $p(\boldsymbol{x^{c}})$ a reprojection of 3D point in
camera frame to image coordinates $[u,v]$ (see Eq. 1):

$\begin{bmatrix}u\\\ v\\\
1\end{bmatrix}=p(\boldsymbol{x}^{c})=p(\boldsymbol{T}_{EEF}^{camera}\boldsymbol{T}_{m}^{EEF}\cdot\boldsymbol{x}^{m}).$

Then, for each data point $i$ (corresponding to position of marker $m_{i}$ in
the configuration $\boldsymbol{\theta}_{i}$), reprojection of marker $m_{i}$
to camera frame $[u_{i},v_{i}]$, and each measured marker position in the
camera image $(u^{m}_{i},v^{m}_{i})$ we get two equations:

$\begin{split}&u_{i}=u^{m}_{i},\\\ &v_{i}=v^{m}_{i}.\end{split}$ (14)

According to [1], this type of calibration corresponds to the restriction of 2
position parameters. Still, we have to add camera DH parameters (see Table 2)
to parameters being calibrated to enable the reprojection of markers to the
camera frame.

###  4.4 Combining multiple chains (LA-RA-LEye, LA-RA-LEye-REye)

Figure 14: Illustration of calibration combining multiple chains: self-contact
and self-observation. All chains originate in a common base frame (bottom
yellow circle). The left and right arm chains are drawn in purple and green,
respectively. The eye chains are drawn in white. Red lines denote reprojection
into the cameras. The cyan mark indicates the distance between end effector
centers (one diameter).

In order to estimate all kinematic parameters of the robot, we can take
advantage of combining some or all of the above-mentioned kinematic chains.
For example, in the case that we combine LA-RA, LA-LEye and LA-REye chains
together into LA-RA-LReye (see Fig. 14), the estimated parameter vector
$\boldsymbol{\phi}$ consists of the following parameters:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{r},{\boldsymbol{\phi}}^{l},{\boldsymbol{\phi}}^{rc},{\boldsymbol{\phi}}^{lc}\\}$,
where ${\boldsymbol{\phi}}^{l}$, ${\boldsymbol{\phi}}^{r}$,
${\boldsymbol{\phi}}^{rc}$, and ${\boldsymbol{\phi}}^{lc}$ are parameters
corresponding to the left arm, right arm, right camera, and left camera,
respectively. Similarly, contact of right arm with a horizontal and vertical
plane can be combined with self-observation by right camera, resulting in the
parameter vector $\boldsymbol{\phi}$:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{r},{\boldsymbol{\phi}}^{rc},\boldsymbol{n}^{hp},d^{hp},\boldsymbol{n}^{vp},d^{vp}\\}$,
where $\boldsymbol{n}^{hp}$ and $d^{hp}$ are parameters defining the
horizontal plane, and $\boldsymbol{n}^{vp}$ and $d^{vp}$ are parameters
defining the vertical plane.

The overall objective function can be generally defined as (depending on which
datasets and criteria we want to use for calibration):

$\boldsymbol{g}(\boldsymbol{\phi},D,\boldsymbol{\zeta})=\left[\begin{matrix}\boldsymbol{k}^{st}\odot\boldsymbol{g}^{st}(\boldsymbol{\phi},\boldsymbol{D}^{st},\boldsymbol{\zeta}),\boldsymbol{k}^{p}\odot\boldsymbol{g}^{p}(\boldsymbol{\phi},\boldsymbol{D}^{p},\boldsymbol{\zeta}),\end{matrix}\right.\\\
\left.\begin{matrix}\boldsymbol{k}^{so}\odot\boldsymbol{g}^{so}(\boldsymbol{\phi},\boldsymbol{D}^{so},\boldsymbol{\zeta})\end{matrix}\right],$
(15)

where $\boldsymbol{D}^{st}$,
$\boldsymbol{D}^{p}=\\{\boldsymbol{D}^{hp1},\boldsymbol{D}^{hp2},\boldsymbol{D}^{vp}\\}$
and $\boldsymbol{D}^{so}=\\{\boldsymbol{D}^{st},\boldsymbol{D}^{p}\\}$ are
datasets for self-touch, planar constraints optimization, and self-
observation, respectively. Parameters
$\boldsymbol{k}^{st},\boldsymbol{k}^{p}$, $\boldsymbol{k}^{so}$ are scale
factors to reflect the different uncertainty/reliability of the components,
the number of measurements per configuration, and transformations from
distance errors given in meters with the reprojection errors in pixels. Symbol
$\odot$ marks a Hadamard product: i.e.
($\boldsymbol{k}^{st}\odot\boldsymbol{g}^{st})_{i}=k^{st}_{i}\cdot
g^{st}_{i}$. The value of these parameters is set independently for each pose:
$k^{st}_{i}=\eta^{st}_{i}\cdot p^{st}\cdot\mu_{i}$,
$k^{p}_{i}=\eta^{p}_{i}\cdotp^{p}\cdot\mu_{i}$, and
$k^{so}_{i}=\eta^{so}_{i}\cdot p^{so}$, where $\eta^{st},\eta^{p}$,
$\eta^{so}$ reflect the reliability of the measurement (e.g.,
$\eta_{i}=\sigma_{i}^{-\frac{1}{2}}$, where $\sigma_{i}$ is the uncertainty of
the measurement in the given pose). In this work, $\eta=1$ was used for all
approaches. The parameter $p_{i}$ reflects the fact that there are multiple
markers detected by cameras for the given contact configuration. Therefore, in
the case of (planar constraints / self-touch) contact and self-observation
combination $p^{p}=10$, $p^{st}=20$ (there are two icosahedrons in contact and
on average 20 marker detections per contact event), and $p^{so}=1$. The
coefficient $\mu_{i}$ is determined from intrinsic parameters of cameras
($60\deg$ horizontal view angle, image size $4000\times
6000$\mathrm{p}\mathrm{x}$$) and distance $d_{i}$ of the end effector from the
camera: $\mu_{i,x}=4000$\mathrm{p}\mathrm{x}$/(d_{i}(\pi/3))$,
$\mu_{i,y}=6000$\mathrm{p}\mathrm{x}$/(d_{i}(\pi/3))$. For simplicity,
$\mu_{i,x}=\mu_{i,y}$ was used. Figure 14 shows connections of different
calibration chains and constraints (e.g., distance between end effectors
during a self-contact or distance between end effector and a plane for a plane
constraint).

###  4.5 Calibration using laser tracker

In this scenario, the robot arm with attached retroreflector is moving in free
space to the configurations selected by sampling a joint space in the way that
the retroreflector is facing the laser tracker (more details under Laser
tracker experiment in Section 3.6). The distance between the position of the
retroreflector acquired by the laser tracker and the position computed from
current robot arm DH parameters (plus current joint angle values and using
forward kinematics) is minimized.

The parameter vector ${\boldsymbol{\phi}}$ consists of the following
parameters:
${\boldsymbol{\phi}}=\\{{\boldsymbol{\phi}}^{ra/la},\boldsymbol{R},\boldsymbol{T}\\}$,
where ${\boldsymbol{\phi}}^{ra/la}$ are DH parameters corresponding to the
right/left robot arm (with the link EEL1/EEL2 to the retroreflector – see
Table 1), $\boldsymbol{R}$ and $\boldsymbol{T}$ are rotation and translation
matrices defining laser tracker position w.r.t. the robot base frame.

The objective function is formulated as the error of distances:

$\boldsymbol{g}^{L}(\boldsymbol{\phi},D,\boldsymbol{\zeta})=\begin{bmatrix}p(\boldsymbol{\phi},\boldsymbol{D}_{1},\boldsymbol{\zeta}),...,\;p(\boldsymbol{\phi},\boldsymbol{D}_{M},\boldsymbol{\zeta})\end{bmatrix},$
(16)

where the function
$p(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta})=||\boldsymbol{x}_{i}^{L}-\boldsymbol{x}_{i}^{r}||$
computes the distance of the transformed point $\mathbf{x}_{i}^{L}$ from laser
tracker and the point $\mathbf{x}_{i}^{r}$ gained from forward kinematics and
current estimate of robot DH parameters in the configuration given by the
dataset point $\boldsymbol{D}_{i}$.

To calibrate the robot DH parameters, we minimize the distance between these 2
sets of 3D points (set $\mathbf{X}^{L}$ contains points from laser tracker in
its coordinate system and set $\mathbf{X}^{r}$ includes points in the base
coordinate system computed from a joint configuration using forward kinematics
and the current estimate of robot DH parameters) using an iterative approach.

In each iteration of the optimization process we:

  1. 1.

recompute the estimate of robot DH parameters

  2. 2.

recompute rotation and translation matrix defining laser tracker position
w.r.t. base frame. The relation between corresponding points in sets is
generally:

$\mathbf{x}^{r}_{i}=\boldsymbol{R}\mathbf{x}^{L}_{i}+\boldsymbol{T}+N_{i},$
(17)

where $\boldsymbol{R}$ and $\boldsymbol{T}$ are rotation and translation
matrices defining laser tracker position w.r.t. the robot base frame, $N_{i}$
is noise for the $i$-th datapoint. We used an algorithm introduced by Arun et
al. [49] for finding least-squares solution of $\boldsymbol{R}$ and
$\boldsymbol{T}$. It is a non-iterative algorithm using the singular value
decomposition of a 3$\times$3 matrix.

This is a standard open-loop calibration method in which 3 components of the
pose are measured: the 3D position, not orientation, is acquired from the
laser tracking system [1]. The transform ($\boldsymbol{R}$, $\boldsymbol{T}$)
relating the robot base frame to the laser tracker frame of reference has to
be added to calibration. Since we express rotation by rotation vector, this
corresponds to adding $6$ parameters to calibration.

###  4.6 Non-linear least squares optimization

For solving the optimization problem, the Levenberg-Marquardt iterative
algorithm was employed. This is a standard choice for kinematic calibration
(e.g., [14, 50, 2]) and combines the advantages of gradient descent and Gauss-
Newton algorithms, which can be also applied to this problem. For
implementation, we used the Matlab Optimization Toolbox and the nonlinear
least-squares solver _lsqnonlin_ with the Levenberg-Marquardt option and
parameters _typicalx_ and _scaleproblem_.

###  4.7 Observability and identifiability of parameters

According to [51], the observability index measures the quality of the dataset
based on the identification Jacobian matrix J, which represents the
sensitivity of minimized values to the change of individual parameters. Borm
and Menq [52] proposed a measure $O_{1}$; Driels and Pathre [53] proposed
$O_{2}$; Nahvi and Hollerbach proposed measures $O_{3}$ [54] and $O_{4}$ [55].
All these measures can be computed from the singular value decomposition of J.
They are defined as:

$\displaystyle
O_{1}=\frac{(\sigma_{1}\sigma_{2}\ldots\sigma_{m})^{1/m}}{\sqrt{n}},\quad
O_{2}=\frac{\sigma_{min}}{\sigma_{max}},$ $\displaystyle\quad
O_{3}=\sigma_{min},\quad O_{4}=\frac{\sigma_{min}^{2}}{\sigma_{max}},$ (18)

where $\sigma_{j}$ is $j$-th singular number, $m$ is the number of independent
parameters to be identified and $n$ is the number of calibration
configurations.

The identification Jacobian matrix itself shows us the identifiability of
individual optimized parameters: $\textbf{J}(i,j)=\frac{\partial
X_{i}}{\partial\phi_{j}}$, where $X_{i}$ is a distance (Eq. 7 and Eq. 10) or a
reprojection error (Eq. 13) and $\phi_{j}$ is the parameter to be estimated.
If a matrix column related to a parameter consists only of zeros, the
parameter is unidentifiable which leads to an unobservable problem (the
minimal singular number is zero). According to [1], an unidentifiable
parameter means that the experimental setup does not allow it to be
identified, not that it is intrinsically unidentifiable. The identifiability
can be improved by adding additional sensors to the setup as well as by
extending the amount of poses in the dataset. In our analysis, we compare
observability indices $O_{1}$ (representing the volume of a hyperellipsoid
specified by singular numbers) and $O_{4}$ (noise amplification index which
measures both eccentricity of the hyperellipsoid through $O_{2}$ and size of
the hyperellipsoid through $O_{3}$) (see [1] for an overview) for individual
chains and estimated parameter vectors.

###  4.8 Perturbation of the initial parameters estimate

To evaluate the dependence of the optimization performance on the quality of
the initial estimates of the parameters, we perturbed all estimated parameters
by a perturbation factor $p=\\{1,3,10\\}$ (in experimental section, we show
results for $p=3$). We perturbed all initial offset values $o_{i}$ as follows:

$o^{new}_{i}=0.1p\cdot U(-1,1)+o_{i}\>[rad],$ (19)

where $U(a,b)$ is uniform distribution between $a$ and $b$. It is reasonable
to expect that the remaining DH parameters ($\alpha$, $a$, and $d$) will be in
general more accurate as they can be extracted from CAD models and there is no
moving part and no encoder involved. Therefore, their perturbation was chosen
as follows:

$\begin{split}&\alpha:\alpha^{new}_{i}=0.01p\cdot
U(-1,1)+\alpha_{i}\>[rad],\\\ &a,d:\Phi^{new}_{i}=0.01p\cdot
U(-1,1)+\Phi_{i}\>[m].\\\ \end{split}$ (20)

###  4.9 Evaluation

Before optimization, we randomly divided the dataset into training and testing
data with a ratio of approximately 70 to 30. Training and testing datasets
contained distinct sets of poses—multiple observations of markers in a given
pose were all added to the same batch. Optimization was based on the training
data; testing data were used to evaluate the result. We used root-mean-square
(RMS) error to compare the results from multiple optimization runs. It is
computed as:

$\displaystyle
RMS_{c}=\sqrt{\frac{1}{M}\sum_{i=1}^{M}(k_{i}^{c}g^{c}(\boldsymbol{\phi},\boldsymbol{D}_{i},\boldsymbol{\zeta}))^{2}}=$
$\displaystyle\sqrt{\frac{1}{M}\|\boldsymbol{k^{c}}\odot\boldsymbol{g^{c}}(\boldsymbol{\phi},\boldsymbol{D}^{c},\boldsymbol{\zeta})\|^{2}},$
(21)

where $M$ is the number of observations/measurements, $k^{c}$ is a scale
factor for the given calibration approach $c$ (see Sec. 4.4) and
$g^{c}(\boldsymbol{\phi},\boldsymbol{D}^{c}_{i},\boldsymbol{\zeta})$ is the
corresponding objective function (see Sec. 4.1, 4.2, 4.3 or 4.5 for specific
form of individual objective functions). Symbol $\odot$ marks a Hadamard
product: i.e.
($\boldsymbol{k}^{st}\odot\boldsymbol{g}^{st})_{i}=k^{st}_{i}\cdot
g^{st}_{i}$.

###  4.10 Accuracy of measuring individual components

The accuracy of several components constitutes a lower bound on the overall
accuracy. In particular, we estimated or experimentally evaluated the accuracy
of the following (we note in brackets for which chains is the accuracy of the
given component relevant):

  * 1.

3D printed parts and their dimensions: $0.1$\mathrm{m}\mathrm{m}$$ error based
on the printer specification; we assume the end effector rod to be straight.
(_self-contact (Sec. 4.1) and contact with a plane chains (Sec. 4.2)_)

  * 2.

Intrinsic camera calibration: we evaluated error of intrinsic camera
calibration using multiple calibration patterns split to training and testing
dataset. The resulting error is $0.73$ mean reprojection error for calibration
points in pixels on the testing set. This error is composed of the accuracy of
detecting the dot pattern and the calibration itself. (_self-observation
chains (Sec. 4.3)_)

  * 3.

Pose extraction from ArUco markers: The calibration object with the Aruco
markers was mounted on a linear positioning table. The object was moved along
straight lines with a 0.01 mm precision. We captured images of the object in
different positions and detected centers of the observed Aruco markers as
shown in Fig. 15. Due to camera reprojection, the markers move along lines in
the undistorted image (radial undistortion). We estimate the error of marker
detection as distance of the detected markers from interpolated straight line
(Fig. 15). The mean error is around $0.33$ pixels for well visible markers (ID
107, 112, 113) and $0.63$ pixels for other markers with maximal error of $1.5$
pixels. We investigate the error along the line with similar results.
Intrinsic camera calibration puts a lower bound on these results. (_self-
observation chains (Sec. 4.3)_)

Figure 15: Pose extraction from ArUco markers. (top) Icosahedron moving along
a straight line with markers positions reprojected to camera frame ($u$ and
$v$ are coordinates of the camera image). (bottom) Errors measured as
perpendicular distance from the line for individual markers at specific
positions.

  * 4.

Repeatability of measurements: The details of robot movements and how they
were stopped once contact was detected are described in Sec. 3.3. We used the
laser tracker to measure the repeatability of these movements, using 20
repetitions of the same movement at two different positions sampled from the
grid for planar constraints (Fig. 5). These involved only small robot
movements (10 cm) and repeatability was high – Fig. 16. For self-touch, larger
movements from the robot home position were executed. However, self-contact
experiments involved both large movements and small movements similar to those
for contact with a plane. Thus, the statistics for Fig. 16 was combined for
the self-touch distribution, resulting in overall lower repeatability. Results
in $x$-coordinate are shown. Similar distribution and range of errors was
observed for $y$ and $z$ coordinate. _(all chains)_

  * 5.

Camera projection error propagation: Cameras measure projections to the image
plane, producing higher uncertainty in the z-axis of the image coordinate
system (image depth). The uncertainties in all three axes can be obtained by
error propagation from object position uncertainty. The projection uncertainty
can be written as a quadric equation $u^{T}Qu=1$, where $Q$ is a matrix (4x4
identity matrix assumed here) and $u$ is a 4x1 vector of both cameras’
projection pixel coordinates. The projections vector $u$ can be expressed as
$u=J_{x}X$, where $J_{x}$ is the Jacobian matrix computed from projection
equations (Eq. 1) and $X$ are the projected point coordinates. Combining these
two equations, we obtain the equation for position uncertainty
$X^{T}Q_{x}X=1$, where $Q_{x}=J_{x}^{T}QJ_{x}$, which can be interpreted as an
ellipsoid. Then, the eigenvalues of $Q_{x}$ are $a^{-2}$, $b^{-2}$, $c^{-2}$,
where $a$, $b$, $c$ are half the principal axes’ length in meters. In our
case, the mean values are $a=0.146~{}$\mathrm{m}\mathrm{m}$$,
$b=0.150~{}$\mathrm{m}\mathrm{m}$$, $c=1.330~{}$\mathrm{m}\mathrm{m}$$, i.e.
an error of 1 pixel corresponds to more than 1 mm error in object position in
one axis. The comparison of uncertainties can be seen in Fig. 16. The length
of the last axis of the ellipsoid (image depth) is one order of magnitude
higher than the other lengths and the touch uncertainty. (_self-observation
chains (Sec. 4.3)_)

Figure 16: Error distribution of different components. Data for self-touch and
planes obtained from repeatability experiments. Camera errors from camera
resolution and 3D space projection from robot workspace.

Based on the above-mentioned analysis, we can see that the lower bound of
error for self-observation will consist of the combination of intrinsic camera
calibration error ($e^{i}$) and ArUco marker detection error ($e^{ArUco}$):
$e^{so}$ = $e^{i}\bigoplus e^{ArUco}$. The measured displacement of the end
effector between contacts is influenced by $e^{ArUco}$ and the observed error
is systematic. In the worst case, we can estimate the error to be independent
on the direction of the impact and for all directions consider the maximum
observed error. As can be seen, repeatability measurements show that the
accuracy is very high and the errors are below 0.03 mm. Finally, an analysis
of camera errors in 3D space reveals their limited accuracy, in particular in
one axis corresponding to the depth.

##  5 Experimental results

To evaluate and compare the results of kinematic parameters calibration across
individual approaches and their combinations, we show results for the right
arm of the robot only—this kinematic chain can be calibrated using all the
datasets collected. First, we show the results for end effector length and
angle offset (Section 5.1) without and with camera calibration. Afterward, we
show results of “daily calibration” (calibrating only joint angle offsets
$\boldsymbol{o}$) of the whole right robot arm (including length of the end
effector), consisting of: reference values acquired by the laser tracker (Sec.
5.2), calibration by individual approaches without and with camera calibration
(Sec. 5.3). Finally, the results for all DH calibration are shown in Sec. 5.6
and evaluated on the independent laser tracker dataset.

The following notation/labeling of experiments is used for the individual
evaluated approaches: self-contact/self-touch (Sec. 4.1); 1 horizontal plane,
2 horizontal planes, all planes (2 horizontal and vertical plane) (Sec. 4.2);
all (combination of all planes and self-contact calibration, Sec. 4.4); self-
observation (Sec. 4.3).

Each dataset (see Sec. 3.6 for details) was split into training and testing
part (70:30). The resulting RMS errors (according to Eq. 21) for both training
and testing datasets are shown and compared to the case where nominal
parameters are used (_before_). We show resulting RMSE separately for self-
contact distances between icosahedrons (_’dist’_), distances to plane (_’plane
dist’_), and for camera reprojections in pixels (_’mark.’_) (e.g., Fig. 17).
In addition, we investigate corrections to the parameter values and their mean
and variance.

Calibration methods relying on physical contact—self-contact and planar
constraints—can be employed independently or in combination with calibration
using cameras (self-observation). We distinguish three possibilities that will
be used throughout the rest of the Experimental results section:

  * 1.

No cameras. Only contact-based calibration was employed and cameras were not
used at all.

  * 2.

Fixed cameras. Cameras’ DH parameters were precalibrated (see Table 2) using
the contact-based datasets (self-contact and contact with plane) from marker
reprojection errors, using nominal values of the robot kinematic parameters.
Then, to calibrate the robot right arm, reprojection of markers to the camera
frame is used together with contact information, while the camera extrinsics
stay fixed.

  * 3.

Camera position calibration (Cam. calib.). Cameras were precalibrated in the
same way as in Fixed cameras. During robot kinematic calibration, reprojection
errors are combined with contact-based constraints. Additionally, camera
extrinsics (expressed in DH parameters) are subject to calibration as well.

Self-observation (S-O), i.e., information from camera reprojection errors, can
also be employed independently. The datasets used are those featuring contact
($D^{st}$, $D^{hp}$, $D^{whole}$, etc.), but the constraints arising from
physical contact are not employed.

Table 4 provides an overview which parameters can be subject to calibration
under the different approaches. The parameters that can be calibrated using
the contact-based approaches are marked in black. Fixed cameras enable
calibration of the end effector orientation ($o_{EE1}$, in green; contact
information alone does not provide any information about orientation – see
Figures 10 and 12). Camera extrinsic parameters, which may be also subject to
calibration, are shown in blue.

Calibration |  Calibrated parameters
---|---
End effector |  $d_{EE1},$ $o_{EE1}$
Offsets |  $o_{L1},$ $o_{U1},$ $o_{R1},$ $o_{B1},$ $d_{EE1},$ $o_{EE1}$
Offsets by ext. device |  $o_{L1},$ $o_{U1},$ $o_{R1},$ $o_{B1},$ $a_{EEL},$ $d_{EEL},$ $o_{EEL}$
All DH parameters |  $a_{L_{1}},$ $d_{L_{1}},$ $o_{L1}$, $a_{U_{1}},$ $d_{U_{1}},$ $\alpha_{U_{1}},$ $o_{U1},$ $a_{R_{1}},$ $d_{R_{1}},$ $\alpha_{R_{1}},$ $o_{R1},$ $a_{B_{1}},$ $d_{B_{1}},$ $\alpha_{B_{1}},$ $o_{B1},$ $a_{T1},$ $\alpha_{T1},$ $d_{EE1},$ $o_{EE1}$
Camera calibration |  $a_{TT3},$ $d_{TT3},$ $\alpha_{TT3},$ $o_{TT3},$ $d_{C1},$ $o_{C1},$ $a_{TT4},$ $d_{TT4},$ $\alpha_{TT4},$ $o_{TT4},$ $d_{C2},$ $o_{C2}$
Table 4: Overview of calibrated parameters. Parameters in green are added to
the corresponding approach under fixedCameras. When using calibCameras, all
the blue parameters are added. See Tables 1 and 2 for details about DH
parameters.

###  5.1 Calibration of end effector length and joint angle offset

First, we evaluated the ability to calibrate the length and joint angle offset
of the last link (EE1/EE2) by individual approaches. The diameter of the final
part of the custom end effector—assuming it is a sphere in this case as the
contact is at the spherical tiles—is known. The parameter being calibrated in
all cases is the end effector length $d_{EE1}$; with fixed cameras, the
orientation, $o_{EE1}$ can be also calibrated. In the case of planar
constraints, the plane parameters $\\{\boldsymbol{n}^{hp/vp},d^{hp/vp}\\}$ are
estimated in addition.

In the self-contact scenario, we compared the case where we assume that the
length and offset for the left arm end effector are known and the case where
we calibrate both EE1 and EE2 with the assumption that both end effectors have
the same length—this is necessary to avoid compensation of the length of one
end effector by the other end effector. In Table 5, we show results for the
case when both end effectors lengths are calibrated simultaneously.

Table 5, the top part, summarizes the corrections to the nominal values of the
length of the end effector achieved by individual calibration approaches. The
most consistent estimates (mean corrections over 20 repetitions) across
individual approaches are achieved with fixed precalibrated cameras, where the
range of estimates is from $3.89~{}$\mathrm{m}\mathrm{m}$$ (all) to
$4.84~{}$\mathrm{m}\mathrm{m}$$ (2 horizontal planes) and from
$3.11~{}$\mathrm{m}\mathrm{m}$$ to $5.95~{}$\mathrm{m}\mathrm{m}$$ for the
self-observation approach trained on three different datasets.

Calibrating end effector length – corrections
---
$[$mm$]$ | No cameras | Cam. calib. | Fixed cameras
1 h. plane | 10 $\pm$ 4 | 12 $\pm$ 2 | 3.32 $\pm$ 0.26
2 h. planes | 14 $\pm$ 2 | 15 $\pm$ 2 | 4.84 $\pm$ 0.11
all planes | 13 $\pm$ 3 | 10.2 $\pm$ 0.2 | 4.34 $\pm$ 0.07
self-contact | 4.31 $\pm$ 0.05 | 4.21 $\pm$ 0.04 | 4.12 $\pm$ 0.04
all | 4.31 $\pm$ 0.04 | 4.25 $\pm$ 0.04 | 3.89 $\pm$ 0.04
S-O ($D^{hp}$) | - | 8 $\pm$ 18 | 3.11 $\pm$ 0.24
S-O ($D^{planes}$) | - | 10.17 $\pm$ 0.25 | 4.38 $\pm$ 0.11
S-O ($D^{whole}$) | - | 6.25 $\pm$ 0.07 | 5.95 $\pm$ 0.10

Calibrating end effector orientation – corrections
---
$[$rad$]10^{-2}$ | No cameras | Cam.calib. | Fixed cameras
1 h. plane | - | 2.96 $\pm$ 0.11 | 4.25 $\pm$ 0.23
2 h. planes | - | 3.29 $\pm$ 0.09 | 5.76 $\pm$ 0.21
all planes | - | 3.28 $\pm$ 0.09 | 3.69 $\pm$ 0.17
self-contact | - | 7.04 $\pm$ 0.09 | 1.06 $\pm$ 0.13
all | - | 6.34 $\pm$0.08 | 2.20 $\pm$ 0.08
S-O ($D^{hp}$) | - | 3.06 $\pm$ 0.15 | 4.17 $\pm$ 0.38
S-O ($D^{planes}$) | - | 3.23 $\pm$ 0.13 | 3.60 $\pm$ 0.12
S-O ($D^{whole}$) | - | 6.40 $\pm$ 0.09 | 2.39 $\pm$ 0.09

Table 5: End effector length calibration (corrections to the manually acquired
values). Comparing calibration using plane constraints (h. plane – 1
horizontal plane, 2 h. planes – 2 horizontal planes, all planes), self-contact
calibration, combination of planar constraints and self-contact (all) with
self-observation only (on different datasets – e.g., S-O ($D^{hp}$) – self-
observation calibration on a dataset from touching a horizontal plane). No
cameras / Cam. calib / Fixed cameras – see text. Mean and standard deviation
over 20 repetitions is displayed.

The quality of the individual estimates can also be evaluated with respect to
the standard deviation (s.d.) across 20 repetitions. For planar constraints
without cameras, we get a very high standard deviation (around
$4~{}$\mathrm{m}\mathrm{m}$$) indicating poor estimates. On the contrary,
using the self-contact approach even without cameras results in standard
deviation around $0.05~{}$\mathrm{m}\mathrm{m}$$, which is comparable to the
standard deviation achieved when camera information is included (either
precalibrated cameras or cameras calibrated together with end effector
length). The addition of cameras also improves consistency of end effector
length estimates by planar constraints (standard deviation drops from $4$ to
$0.2~{}$\mathrm{m}\mathrm{m}$$ for calibrated cameras and to
$0.07~{}$\mathrm{m}\mathrm{m}$$ for fixed cameras). We also evaluated the
calibration using only self-observation information. When we used
precalibrated cameras on the whole dataset, we achieve comparable results to
other estimates. When we calibrate the cameras together with end effector
length, the length of the end effector is overestimated and the standard
deviation (based on the used training data) is also significantly higher.

We compare orientation corrections using only self-observation calibration
(with cameras precalibrated on different subsets of datasets) or self-
observation combined with other constraints (self-contact, planar constraints)
– see Table 5, bottom part. The corrections to manually measured parameters of
the orientation vary between $0-0.06~{}$\mathrm{r}\mathrm{a}\mathrm{d}$$
$(0-3.4^{\circ})$.

The resulting RMS errors (see Section 4.9) for end effector calibration in
$\mathrm{m}\mathrm{m}$ and $\mathrm{p}\mathrm{x}$ are shown in Fig. 17. For
all variants of the optimization problem, both mean _dist_ RMSE and _markers_
RMSE decreased compared to the nominal end effector value (Fig. 17). For the
variant all and Fixed cameras, the distance RMSE decreased from
$3.8~{}$\mathrm{m}\mathrm{m}$$ before calibration to
$1.40~{}$\mathrm{m}\mathrm{m}$$ after calibration and markers RMSE decreased
from $20.8~{}$\mathrm{p}\mathrm{x}$$ before to
$18.0~{}$\mathrm{p}\mathrm{x}$$. Without cameras, we achieved slightly better
results for self-contact distance RMSE ($1.34~{}$\mathrm{m}\mathrm{m}$$), but
in this case the end effector orientation is not calibrated—resulting in
higher error in camera reprojection. For the case with calibrated cameras, the
resulting RMSE are slightly smaller (mean self-contact distance
$1.34~{}$\mathrm{m}\mathrm{m}$$ and camera reprojection RMSE
$16.0~{}$\mathrm{p}\mathrm{x}$$) than for the case with fixed cameras. The
calibration resulted in the estimation of end effector length and
orientation—the corrections of these parameters are listed in the Table 5.

Figure 17: RMS errors – end effector calibration. Distances in 3D (left
y-axis, $mm$); camera reprojection (right y-axis, $px$). plane_dist –
Euclidean distance from a plane, dist – Eucl. distance between end effectors
in self-contact, mark. – error of individual markers reprojected to cameras.
No cameras – only planar constraints or self-touch; Fixed cameras –
precalibrated but fixed cameras; Calib cameras – precalibrated cameras with
additional optimization. Error on both training (Train) and testing (Test)
dataset shown. before – nominal parameters, hp – 1 horizontal plane, ap – all
planes (2 horizontal, 1 vertical), all – self-contact + planar constraints.

For the end effector parameters, we do not have any reference value apart from
manual measurements (laser tracker calibration cannot be applied due to the
retroreflector placement – see Fig. 3). However, the corrections found using
the whole dataset and the combination of all chains (self-touch + planar
constraints + cameras) were reliable enough. Consistent adaptation was
observed for the length. Hence, for subsequent calibration of the robot arm
kinematics, we used the end effector length estimate of
$35.4~{}$\mathrm{c}\mathrm{m}$$. For the orientation, no significant
adaptation was arrived at and the nominal value was kept for the remaining
experiments. (see Table 1).

To validate the selection of the end effector length—which will be used in
what follows as an initial value of this parameter—we performed two additional
experiments. First, we systematically varied the initial end effector length
parameter before calibration in the range from 10 to 70 mm and evaluated the
RMS error after calibration on the self-contact dataset – see Fig. 18. Two
solutions were found with minimal RMS error ($35.38~{}$\mathrm{c}\mathrm{m}$$
and $61.4~{}$\mathrm{c}\mathrm{m}$$), but one of them is not to be considered,
as we do not expect errors bigger than 1 cm from manual measurement (35
$\mathrm{c}\mathrm{m}$). These two solutions can be explained by the nature of
the self-contact as there are multiple possibilities arising from the
geometrical consideration of self-contact (see Fig. 10, right).

Figure 18: RMS error on self-contact dataset for different values of end
effector length.

Second, the sensitivity to perturbation of the initial end effector length for
individual approaches was evaluated in the case with/without cameras and
compared to the case without perturbation. The resulting corrections can be
seen in Fig. 19. For the case with perturbation and without cameras (centre),
two solutions were found by _self-touch_ calibration, corresponding to Fig.
17. When fixed precalibrated cameras are added, we achieve results with a low
standard deviation for all calibration setups; for _self-touch_ calibration,
only one solution is selected. For _self-touch_ and _all_ conditions with
fixed cameras, we achieve the same results both for perturbed initial value
and non-perturbed.

Figure 19: Corrections of end effector length compared to nominal values:
(left) no perturbation + no cameras, (centre) perturbation + no cameras,
(right) perturbation + fixed cameras. all planes – 2 horizontal, 1 vertical;
hpvp – upper horizontal + vertical plane; 1hp (lower) – lower horizontal
plane; lpvp – lower horizontal + vertical plane; etc.; all – self-contact +
all planes

###  5.2 Calibration of robot joint offsets by an external measurement device
(reference)

The whole dataset $D^{tracker}$ collected by the laser tracker (Leica) was
used for right arm joint angle offsets calibration to create a reference value
to our other calibration approaches (see Sec. 4.5). Parameters subject to
calibration are listed in Tab. 4. First, we estimated the tracker position
($R$, $T$) and retroreflector DH parameters ($a$, $d$, and joint angle offset
$o$) (link 7b in Table 1). Afterwards, we calibrated all offsets of the robot
arm including the retroreflector DH parameters.

As can be seen in Fig. 20, left, RMSE improved with further calibration: from
$6.08~{}$\mathrm{m}\mathrm{m}$$ before calibration to
$3.03~{}$\mathrm{m}\mathrm{m}$$ after retroreflector calibration and to
$2.64~{}$\mathrm{m}\mathrm{m}$$ after all offsets calibration. To achieve the
best possible solution quality, we have not split the original dataset into
training and testing part in this case. Instead, we evaluated the RMSE on a
separate dataset (self-contact dataset $D^{st}$) using the parameters
estimated by the laser tracker calibration and compared to RMSE for nominal
parameters (Fig. 20, right). In this case, we used same parameters of the end
effector for both methods and compared the error at the end of the $5$th link
(see Fig. 20). A slight improvement of RMSE can be observed also on this
dataset: from $3.755~{}$\mathrm{m}\mathrm{m}$$ to
$3.708~{}$\mathrm{m}\mathrm{m}$$ with the same uncalibrated end effector. The
distribution of errors on the laser tracker dataset before and after
calibration can be seen in Fig. 21.

Figure 20: (left) RMS errors comparison on the laser tracker dataset (Nominal
DH – nominal parameters are used), after only retroreflector calibration, and
after all right arm offsets including all DH parameters of the tracker
retroreflector. (right) RMS errors comparison on self-contact dataset;
parameters obtained by laser tracker calibration compared to RMS error
achieved by Nominal DH parameters.

Figure 21: Residual errors for individual datapoints before calibration using
nominal parameters (left) and after calibration of all offsets of robotic arm
including the tracker retroreflector full DH parameters (right).

The corrections for retroreflector parameters (compared to nominal parameters)
$\\{d_{EEL1}$, $a_{EEL1}\\}$ are $0.72~{}$\mathrm{m}\mathrm{m}$$, and
$0.92~{}$\mathrm{m}\mathrm{m}$$, respectively, and for offset parameters
$\\{o_{L1}$, $o_{U1}$, $o_{R1}$, $o_{B1}$, and $o_{ELL1}\\}$ are $\\{-1.43$,
$-0.18$, $-0.59$, $-1.03$, and $4.123\\}$
$\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{d}$, respectively.

###  5.3 Calibration of robot joint offsets – self-contained approaches

We compared the quality of the right arm joint angle offsets calibration,
including the end effector length, by individual approaches. Self-contact
approach (Sec. 4.1), planar constraints (using 1-3 planes; Sec. 4.2) and pure
self-observation calibration (Sec. 4.3) were compared to the results for
nominal parameters and the reference parameters acquired by the laser tracker
measurement device (Sec. 5.2). For the contact-based methods (self-contact,
planar constraints), we compared the results with no camera information, fixed
precalibrated cameras, and cameras being part of the calibration process.
Parameters subject to calibration are listed in Tab. 4. Depending on the
particular method, additional parameters may be subject to calibration (e.g.,
parameters of the plane) (see corresponding subsections in Sec. 4).

RMSE after calibration of offsets and end effector length without perturbation
can be seen in Fig. 22. The RMS error of distance in self-contact (in
$\mathrm{m}\mathrm{m}$) drops from $3.80~{}$\mathrm{m}\mathrm{m}$$ before
calibration to $1.40~{}$\mathrm{m}\mathrm{m}$$ after end effector calibration,
and to $1.31~{}$\mathrm{m}\mathrm{m}$$ after all offsets calibration. Distance
to plane (in $\mathrm{m}\mathrm{m}$) drops from
$1.00~{}$\mathrm{m}\mathrm{m}$$ before calibration to
$0.93~{}$\mathrm{m}\mathrm{m}$$after end effector calibration, and to
$0.89~{}$\mathrm{m}\mathrm{m}$$ after all offsets calibration without cameras
($0.91~{}$\mathrm{m}\mathrm{m}$$ with camera calibration and
$0.92~{}$\mathrm{m}\mathrm{m}$$ with fixed cameras). Distance of reprojected
markers in camera (in pixels) drops from $20.8~{}$\mathrm{p}\mathrm{x}$$
before calibration to $18.0~{}$\mathrm{p}\mathrm{x}$$ after end effector
calibration to $15.3~{}$\mathrm{p}\mathrm{x}$$ after all offsets calibration
including camera calibration and to $15.6~{}$\mathrm{p}\mathrm{x}$$ for fixed
cameras calibration.

Corrections of the parameters to the nominal parameters are shown in Fig. 24.
Corrections of the parameters are the smallest (also having the smallest
standard deviation) compared to the values achieved by laser tracker
calibration for the variant _all_ with fixed precalibrated cameras, which
means that we combined all of the chains (planar constraints, self-touch,
self-observation) to optimize the offset parameters. The resulting correction
for parameter $d_{EE1}$ is $5.46\pm 0.14~{}$\mathrm{m}\mathrm{m}$$. The
corrections for parameters $\\{o_{L1}$, $o_{U1}$, $o_{R1}$, $o_{B1}$, and
$o_{EE1}\\}$ are: $\\{-2.10\pm 0.38$, $-0.89\pm 0.24$, $1.48\pm 0.13$,
$-0.76\pm 0.06$, and $32.2\pm 1.6\\}$
$\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{d}$, respectively. The corresponding
corrections from laser tracker calibration for parameters $\\{o_{L1}$,
$o_{U1}$, $o_{R1}$, and $o_{B1}\\}$ are $\\{-1.43$, $-0.18$, $-0.59$, and
$-1.03\\}$ $\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{d}$, respectively.

Figure 22: RMS errors – right arm offsets and end effector calibration.
Distances in 3D (left y-axis, $mm$); camera reprojection (right y-axis, $px$).
before – nominal parameters, EEF cal. – after end effector calibration. For
detailed legend see Fig. 17.

###  5.4 Comparison of nominal DH parameters and former
calibration/calibration performed by laser tracker

Figure 23: Comparison of error on the testing dataset ($D^{tracker}$)– dataset
used for laser tracker reference calibration. Error computed towards laser
tracker retroreflector data. $x$ axis – individual datasets/methods used for
finding robot kinematics parameters. We compare self-observation (Self-obs.)
with one/both precalibrated cameras (on $D^{whole}$) calibrating robot offsets
based on data from individual datasets, calibration using plane constraints
(horizontal plane (lower/upper), 2 horizontal planes (tables), vertical plane
(vp) and all planes (ap)), self-touch calibration (self-touch), and self-touch
+ planar constraints (all). Comparison when no cameras are used (Without
cams), precalibrated (using self-observation on $D^{whole}$) but fixed cameras
(Fixed cams.), and precalibrated cameras (on $D^{whole}$) being part of the
calibration (With cams calib).

Figure 24: Corrections of offsets and end effector parameters ($d_{EE1}$,
$o_{EE1}$) after calibrating end effector and offsets of right arm without
cameras (using only planar constraints or self-touch), with precalibrated but
fixed cameras and with precalibrated cameras which are calibrated during the
experiment. Offset of the end effector cannot be calibrated without cameras
and nominal values are used. Laser tracker calibration (ground truth) values
in turquoise.

Figure 25: Corrections of offsets and end effector parameters ($d_{EE1}$,
$o_{EE1}$) after calibrating end effector and offsets of right arm without
cameras (using only planar constraints or self-touch), with precalibrated but
fixed cameras and with precalibrated cameras which are calibrated during the
experiment. Offset of the end effector cannot be calibrated without cameras
and nominal values are used. Laser tracker (Leica) calibration (ground truth)
values in turquoise. Initial perturbation p = 3.

To evaluate the resulting offset parameters for the robot acquired through
calibration using individual approaches, we conducted a comparison study on
the laser tracker testing dataset $D^{tracker}$ (which covers the whole robot
right arm workspace and not only the area where calibration using self-
contained approaches was performed). To be able to perform such an evaluation,
the transform to laser retroreflector, acquired from laser tracker
calibration, was used for all compared calibration results. These results are
shown in Fig. 23. Using the calibrated parameters by variant _all_ , we can
achieve better RMSE than nominal parameters—both in the case without cameras
or with fixed precalibrated cameras. The resulting RMSE is
$3.06~{}$\mathrm{m}\mathrm{m}$$, $2.63~{}$\mathrm{m}\mathrm{m}$$,
$3.04~{}$\mathrm{m}\mathrm{m}$$, $2.73~{}$\mathrm{m}\mathrm{m}$$ for nominal
parameters, laser tracker, variant _all_ without cameras, and variant _all_
with precalibrated fixed cameras, respectively. Adding the option of camera
calibration increases the RMSE for the variant _all_ to
$3.18~{}$\mathrm{m}\mathrm{m}$$ on this laser tracker testing dataset
$D^{tracker}$ compared to the nominal parameters
($3.06~{}$\mathrm{m}\mathrm{m}$$).

The lowest RMS errors (for some setups lower than RMSE with nominal
parameters, which is $3.06~{}$\mathrm{m}\mathrm{m}$$) are achieved for
different calibration approaches with fixed precalibrated cameras—the
resulting RMSE are $9.3~{}$\mathrm{m}\mathrm{m}$$,
$4.5~{}$\mathrm{m}\mathrm{m}$$, $3.8~{}$\mathrm{m}\mathrm{m}$$,
$3.31~{}$\mathrm{m}\mathrm{m}$$, $2.84~{}$\mathrm{m}\mathrm{m}$$, and
$2.73~{}$\mathrm{m}\mathrm{m}$$ for vertical plane (_vp_), lower horizontal
plane (_hp_), both horizontal planes (_tables_), self-contact (_selftouch_),
all planes (_ap_), and variant _all_ , respectively.

Pure self-observation (SO) highly depends on the quality of the training
dataset. For the datasets from planar contact with one plane, we have only
$250$ datapoints and the resulting error is $29.4~{}$\mathrm{m}\mathrm{m}$$
for 2 cameras and $23.7~{}$\mathrm{m}\mathrm{m}$$ for 1 camera. When more data
points are added—all planes ($750$ datapoints) or self-contact ($566$
datapoints)—the RMSE drops to $10.2~{}$\mathrm{m}\mathrm{m}$$ and
$12.15~{}$\mathrm{m}\mathrm{m}$$ for all planes and self-contact,
respectively. When $D^{whole}$ is used ($1316$ datapoints), the RMSE drops
below RMSE achieved for nominal parameters ($3.06~{}$\mathrm{m}\mathrm{m}$$):
to $2.90~{}$\mathrm{m}\mathrm{m}$$ for 2 cameras setup and
$2.82~{}$\mathrm{m}\mathrm{m}$$ for 1 camera setup (still higher than variant
_all_ with fixed cameras – $2.73~{}$\mathrm{m}\mathrm{m}$$).

###  5.5 Calibration of robot joint offsets – sensitivity to perturbation

To evaluate the sensitivity of calibration approaches to perturbation, we run
the experiment with initial perturbed parameters ($p=3$) (see Section 4.8 for
details). The results can be seen in Fig. 25. For fixed precalibrated cameras
we achieve very similar results as without perturbation with comparable low
standard deviation. For example, for the variant _all_ with fixed
precalibrated cameras, the resulting correction for parameter $d_{EE1}$ is
$5.46\pm 0.14~{}$\mathrm{m}\mathrm{m}$$ without perturbation and $5.52\pm
0.14~{}$\mathrm{m}\mathrm{m}$$with perturbation. The corrections for offset
parameters $\\{o_{L1}$, $o_{U1}$, $o_{R1}$, $o_{B1}$, and $o_{EE1}\\}$ are:
$\\{-2.10\pm 0.38$, $-0.89\pm 0.24$, $1.48\pm 0.13$, $-0.76\pm 0.06$, $32.2\pm
1.6\\}$$10^{-3}~{}$\mathrm{r}\mathrm{a}\mathrm{d}$$ without perturbation, and
$\\{-2.24\pm 0.28$, $-0.96\pm 0.23$, $1.55\pm 0.20$, $-0.75\pm 0.07$, $32.1\pm
1.3\\}$$10^{-3}~{}$\mathrm{r}\mathrm{a}\mathrm{d}$$ with perturbation,
respectively. Without cameras, the corrections have higher standard deviation,
especially for the ’self-touch’ calibration. For all perturbation levels, the
performance is stable (low error s.d.) in the case of fixed cameras.

###  5.6 Calibration of all DH parameters

Figure 26: RMS errors – right arm all DH parameters calibration. Distances in
3D (left y-axis, $mm$); camera reprojection (right y-axis, $px$). Results only
for the variant _all_ using dataset $D^{whole}$. noCams – only planar
constraints + self-touch; FixedCam – precalibrated but fixed cameras; CamCalib
– precalibrated cameras with additional optimization, S-O – self-observation
only. before – nominal parameters, EEF cal. – after end effector calibration,
offset cal. – after calibration of robot offsets + end effector. For detailed
legend see Fig. 17.

Figure 27:  Corrections of all DH parameters for the right arm by the variant
_all_ with initial perturbation ($p=3$). Results without cameras (planar
constraints + self-touch, noCams), with precalibrated cameras which are
calibrated during the experiment (camCalib), and with precalibrated but fixed
cameras (FixedCams). Offset of the end effector cannot be calibrated without
cameras – nominal values are used. Laser tracker calibration (reference)
values in tyrquis. Results shown over 50 repetitions.

In Figure 26, we show a comparison of RMSE after calibration of all DH
parameters (see Table 4) by the method _all_ for different setups (fixed
cameras, calibrated cameras and no cameras). Results on both training and
testing datasets are shown and compared to the case where nominal parameters
are used (_before_), EEF length and orientation calibration by the variant
_all_ (EEF cal.) and right arm offset calibration by the variant _all_ (offset
cal.). The self-contact distance, plane distance, and camera reprojection RMSE
drops from $1.33~{}$\mathrm{m}\mathrm{m}$$, $0.92~{}$\mathrm{m}\mathrm{m}$$,
and $15.6~{}$\mathrm{p}\mathrm{x}$$, respectively, after offset calibration,
to $1.00~{}$\mathrm{m}\mathrm{m}$$, $0.58~{}$\mathrm{m}\mathrm{m}$$, and
$14.8~{}$\mathrm{p}\mathrm{x}$$ after calibration of all DH parameters with
calibrated cameras, and further to $1.00~{}$\mathrm{m}\mathrm{m}$$,
$0.55~{}$\mathrm{m}\mathrm{m}$$, and $15.2~{}$\mathrm{p}\mathrm{x}$$ after
calibration with precalibrated fixed cameras. When cameras are not used
(noCams), self-contact distance further drops to
$0.59~{}$\mathrm{m}\mathrm{m}$$ and planar distance to
$0.64~{}$\mathrm{m}\mathrm{m}$$. In this case, the end effector offset is not
calibrated and the camera reprojection error would be higher.

In Figure 27, we show the resulting corrections of DH parameters (corrections
are calculated as the difference from the nominal parameters) for the variant
_all_ when perturbed DH parameters were used as initial value (perturbation
factor $p=3$). We show initial values of perturbed DH parameters, the results
of calibration by method _all_ in the case without cameras (using only contact
information from planar constraints and self-touch), with precalibrated
cameras which are calibrated during the experiment, and with precalibrated but
fixed cameras. In all compared cases, we reach a lower dispersion of the final
values and these values are very close to the industrial nominal DH
parameters. The best results with the lowest standard deviation and closest to
the industrial nominal values are consistently reached by the method when
precalibrated fixed cameras are used. When no cameras are used, we can see
that the corrections of DH parameters for variant _all_ are unrealistically
far from the nominal parameters (e.g., for parameter $a_{B1}$ we get
corrections around $0.2~{}$\mathrm{m}$$). When also camera calibration is
employed, the corresponding DH parameters corrections are reasonably close to
nominal parameters and the calibration method is able to even for initially
perturbed parameters (up to 10 $\mathrm{c}\mathrm{m}$ difference from the
nominal parameters) reach reasonable DH parameters values (all corrections are
under $5~{}$\mathrm{m}\mathrm{m}$$). We also show the results of the
parameters from the laser tracker for comparison. End effector parameters
cannot be calibrated by laser tracker thanks to its placement (see Fig. 3).
Therefore, the value is not changing from the initial parameter. For the end
effector length, we visualise as the nominal parameter the correction to the
initial end effector length value achieved by the former end effector
calibration (Sec. 5.1).

Figure 28: Comparison of RMSE for all DH calibration on the whole laser
tracker dataset ($D^{tracker}$). Results are shown for the whole laser tracker
dataset (red) and for the subset of the robot workspace which corresponds to
the area where other self-contact methods were trained (blue). Results for the
calibration by the variant _all_ without cameras (noCams), with fixed
precalibrated cameras (FixedCams) and cameras being calibrated (CamCalib) is
compared to the self-observation approach (S-O) trained on the dataset
$D^{whole}$, Nominal parameters and the calibration by laser tracker.

In Figure 28, we show the resulting RMSE of our calibration contact-based
approach on the testing laser tracker dataset $D^{tracker}$. The results for
self-contact approach with fixed precalibrated cameras and with cameras
calibrated as a part of the calibration approach are compared to self-
observation approach where only camera information is used. All approaches are
trained on the same $D^{whole}$ dataset. Comparison to the RMSE when nominal
parameters and when DH parameters from laser tracker calibration are used, is
provided. We show results for the whole laser tracker dataset and for the
subset of the dataset which corresponds to the area where self-contact and
self-observation approaches were calibrated. On the whole dataset, RMSE is
$\\{3.06\pm 0$, $2.63\pm 0$, $224\pm 77$, $3.68\pm 0.20$, $3.58\pm 0.21$, and
$12.36\pm 0.55\\}~{}$\mathrm{m}\mathrm{m}$$ for nominal parameters calibration
(Nominal), laser tracker, self-contact calibration without cameras (noCams
(all)), self-contact with cameras being part of calibration (CamCalib (all)),
self-contact calibration with fixed precalibrated cameras (FixedCams (all)),
and self-observation approach (S-O (all)), respectively. On the subset of the
dataset, RMSE is $\\{1.73\pm 0$, $1.71\pm 0$, $205\pm 71$, $2.75\pm 0.17$,
$2.28\pm 0.17$, and $11.88\pm 0.38\\}~{}$\mathrm{m}\mathrm{m}$$ for nominal
parameters calibration (Nominal), laser tracker, self-contact calibration
without cameras (noCams (all)), self-contact with cameras being part of
calibration (CamCalib (all)), self-contact calibration with fixed
precalibrated cameras (FixedCams (all)), and self-observation approach (S-O
(all)), respectively. We can see that including self-contact information as a
part of calibration provides significantly better results than using pure
self-observation approach.

###  5.7 Observability analysis of individual approaches

We evaluated the observability indices $O_{1}$ and $O_{4}$ for the different
combinations of calibration approaches and parameters subject to calibration.
An overview is shown in Fig. 29 with $O_{1}$ on top and $O_{4}$ at the bottom.
Please refer to Section 4.7 for details about how the indices are calculated.
The bar groups on the x-axis correspond to the parameters subject to
calibration and the number of parameters (columns of the identification
Jacobian matrix J). Calibration using the laser tracker adds two additional
parameters pertaining to the retroreflector placement (see Table 4). Color
coding of individual bars marks the calibration approach with the no. data
points in a given dataset without cameras / with cameras (rows of J). Due to
the variable size of J, comparisons are to be made with caution. As expected,
for the cases without cameras (noCams), there is a trend that with the number
of calibrated parameters increasing (from ee-noCams, over offsets-noCams, to
allDH-noCams), the observability indices are lower. When self-observation as a
calibration method and corresponding camera extrinsic parameters are added
(calibCams), observability indices $O_{1}$ in general significantly increase.
Observability index $O_{4}$ increases after adding camera chains to
calibration for the allDH calibration case, but drops for end effector length
calibration—adding new parameters to be calibrated (columns of J) outweighs
the effect of additional data from self-observation (rows of J). Comparing
situations where the dataset and hence J has a similar size, we see that
horizontal plane was more effective than vertical plane in our setup and that
selftouch slightly outperformed combinations of all planar constraints (all
planes). This is largely consistent with the RMSE errors after calibration
(Fig. 23). Finally, for calibrating all DH parameters, the inclusion of
cameras seems necessary.

Note that the observability analysis is also affected by measurement noise
that may increase the effective rank of the Jacobian matrix. We assume that
the measurement noise in this case is sufficiently small (experimental
verification is presented at [3]).

Figure 29: Observability indices $O_{1}$ (top) and $O_{4}$ (bottom). Groups on
the x-axis correspond to parameters subject to calibration (ee – end effector
length; offsets – $o_{i}$ of individual links; allDH – all DH parameters;
noCams / calibCams – without / with the “self-observation chains”; Table 4 for
details). Color coding of individual bars marks the calibration approach and
dataset (no. data points without cameras / with cameras).

##  6 Conclusion

Using a dual-arm industrial robot with force/torque sensing and cameras, we
presented a thorough experimental comparison of geometric kinematic
calibration using “self-contained approaches” using sensors on the robot—
self-contact, planar constraints, and self-observation—with calibration using
an external laser tracker. The main findings are summarized below.

First, we studied estimation of the kinematic parameters of a new tool—a
custom end effector (Section 5.1). To calibrate the tool length, self-contact
alone proved effective; planar constraints or self-observation alone did not
perform as well in isolation and improved only in synergy with one of the
other approaches/datasets. Testing RMS errors of approximately 1 to 1.5 mm on
the part of the workspace where calibration was performed were achieved. For
orientation of the tool, the addition of cameras (self-observation) was
needed. Combining different methods/kinematic chains proved effective,
supported also by the observability analysis (Section 5.7 and Fig. 29).

Second, we analyzed the performance of “daily calibration”: the joint angle
offsets of a complete robot arm were added to the tool calibration (Section
5.3). While the end effector calibration is responsible for most of the error
in the workspace, further improvement from the offset calibration is achieved
with all methods. To assess whether the calibration is effective only
locally—on the particular dataset—or whether better kinematic parameters of
the robot were learned, a comparison on an independent dataset covering the
whole workspace (laser tracker calibration) was performed (Section 5.4 and
Fig. 23). Corrections to robot parameter values were also analyzed.
Importantly, all self-contained approaches except for the planar constraints
in isolation were able to achieve good results, sometimes outperforming the
robot nominal parameters. Self-contact alone proved the most effective; the
best results were achieved using a combination of datasets and methods. Self-
contact was further improved by the addition of planar constraints. Self-
observation alone was sensitive to the size of the dataset but was effective
in combination with contact-based methods; the best results were achieved
using sequential calibration: camera extrinsics were calibrated first,
followed by robot arm kinematics and tool calibration. Additionally, similar
results were achieved when perturbing the initial parameter estimates (Section
5.5).

Third, we performed a complete geometric kinematic calibration of one robot
arm plus the tool (all DH parameters) using different methods (Section 5.6).
Good results are achieved on the individual datasets (Fig. 26), but further
analysis reveals that the calibration suffered from overfitting to the
particular dataset / part of the robot workspace. On an independent dataset,
the performance of nominal parameters was not matched (Fig. 28). We also found
that when all DH parameters are subject to calibration, the need for the
synergy of different approaches increases, as testified by the unrealistic
parameter corrections with the contact-only (noCams) approaches in Fig. 27 and
the observability analysis (Fig. 29, _allDHnoCams_) or the self-observation
only approach in Fig. 28. Like previously, the sequential calibration—first
cameras, then robot kinematics—gives the best results. Although the
performance of nominal or laser tracker calibration parameters on the whole
workspace could not be matched (Fig. 28), the performance of the combination
of self-contained approaches—also in the case of initial parameter
perturbation—is reasonable (less than 4 mm error) and for a less accurate
platform like a service robot may suffice.

Fourth, the comprehensive dataset collected is made publicly available [3] and
can be used for additional analyses. This constitutes an additional
contribution of this work.

##  7 Discussion and future work

Geometric kinematic calibration of industrial robots is usually performed
using external metrology—measurement arms (e.g., [4]) or contactless
measurement systems like laser trackers [5, 6, 7, 8]. Newman et al. [7]
calibrated a Motoman P-8 robot using an SMX laser tracker, improving the RMS
error from 3.595 mm to 2.524 mm. Specifically related to the setup used in
this work, our platform was previously calibrated using two different methods:
(1) Redundant parallel Calibration and measuring Machine (RedCaM) by Beneš et
al. [11], Volech et al. [12], and (2) Leica laser tracker. Petrík and Smutný
[13] reviewed the precision of these methods using a linear probe sensor.
Based on a dataset of 43 different poses with touching end effectors, they
calculated the mean error as 0.67 (range 2.92) mm on CAD model, 0.54 (range
2.55) mm on Leica based calibration and 2.45 (range 9.92) mm on RedCaM based
calibration. Other approaches—see [8] and additional references cited
therein—usually achieve sub-millimeter accuracy. It was not our goal to
directly compete with these works; instead, our aim was to assess the
potential of automatic self-contained kinematic calibration: using sensors on
the robot and avoiding the need for external metrology. We could demonstrate
that such self-contained approaches—even if the initial robot parameters are
perturbed—can yield less than 4 mm position errors over the robot workspace.
The accuracy increases when a combination or synergy of these approaches
(e.g., self-contact and self-observation) is exploited. We chose our platform,
an industrial robot, out of convenience and to have a stable enough plant that
can provide stationary results. However, we see the main application area in
collaborative and service robotics. These platforms are typically more
lightweight, flexible, and less precise and they may be often redeployed,
their kinematic configuration changed etc. At the same time, they often come
with a rich set of inexpensive but increasingly powerful sensors. Cameras and
means to detect physical contact are becoming common. All these factors pave
the way for automatic multisensorial self-contained calibration as
demonstrated here.

We provide not only a thorough experimental investigation, but also a
conceptual contribution in that we develop methods how to combine several of
the methods into a single cost function. Further, unlike Birbach et al. [2]
who claim that simultaneous calibration using all the available sensors is
advantageous, we consistently found sequential calibration to perform best:
camera calibration followed by robot kinematics. Conversely, calibrating robot
kinematics together with camera extrinsics simultaneously was not as
successful. Furthermore, if camera extrinsics were not precalibrated and
inaccurate nominal parameters were used, the optimization often converged to
physically impossible local minima. One known limitation of closed-loop
calibration approaches—those relying on physical constraints on the end
effector position or orientation—is that the set of poses is limited, which
may affect the identifiability of parameters [1]. This holds pretty much for
all self-contained calibration—self-contact or self-observation—as these also
naturally constrain the set of robot configurations available. This has also
been the case in this study. Better coverage of individual joint ranges within
the approaches used here—in particular for the planar constraints—would
further improve the results and yield better generalization to other parts of
the workspace. Combining different methods / kinematic chains significantly
mitigates this problem.

It should also be noted that although we used specially designed end effectors
that were developed to combine self-contact (acting as a sphere) and self-
observation (with flat tiles to host fiducial markers), the methods developed
here have wider applicability. Contact can occur at any part of the robot
provided that the link can be represented in the kinematic model; fiducial
markers can be placed on any robot part or avoided altogether by tracking
parts of the robot directly (e.g., [2, 17, 18]).

There are several directions for future work. First, when combining several
calibration approaches into a single cost function (Sec. 4.4), the errors
obtained from the different components could be scaled by coefficients that
are inversely proportional to their uncertainty. We attempted to acquire such
coefficients using repeatability measurements, but failed to obtain estimates
that would reflect the true uncertainty associated with different approaches
and possibly individual data points. Additional measurements of the
uncertainty of individual components (beyond Sec. 4.10) and their propagation
would be required. Thus, all components were weighted equally in this work,
but this can be changed in the future.

Second, the methods presented here can be extended to exploit the existing
sensors differently or to incorporate additional sensory modalities. For
example, the two cameras in our setup were not used explicitly as a stereo
head. Instead of reprojecting the end effector into the 2 image frames, one
could also project the observed position of the end effector in image
coordinates of both eyes (pixel $(u,v)$) to 3D space ($X^{eye}$) (similar to
[17, 56]). The self-contact approach would yield more than a 1-dimensional
constraint in case the contact position (and hence 3 components of the pose)
could be measured such as when using an artificial electronic skin [31, 32,
35]. Inertial sensors—in the robot head ([2]) or distributed on the robot body
[57, 58]—could be also added. Finally, one could also calibrate both
manipulators simultaneously.

Third, for online recalibration to be performed repeatedly, the number of
poses / data points needed can be reduced by employing intelligent pose
selection (e.g., [59, 60, 61, 62]).

Fourth, the standard calibration method using non-linear least squares
optimization (Levenberg-Marquardt algorithm) can be compared with filtering
approaches [18, 26] or with methods that pose fewer assumptions on the initial
model available (e.g., [63]).

## Acknowledgements

We would like to thank Martin Hoffmann for designing and fabricating several
versions of the custom end effectors, with assistance from Tomáš Báča for 3D
printing. We are also indebted to CIIRC CTU for support in using the robot and
the Leica absolute tracker (Libor Wágner, Vladimír Smutný, and Václav Hlaváč).
Tomáš Svoboda provided general support, consultation, and pointer to the
article [49].

## Funding

This work was supported by the Czech Science Foundation (GA ČR), project EXPRO
(no. 20-24186X); T.P. was supported by the European Regional Development Fund
under project Robotics for Industry 4.0 (no.
CZ.02.1.01/0.0/0.0/15$\\_$003/0000470) and EU H2020 SPRING Project (no.
871245).

## References

  * [1] J. Hollerbach, W. Khalil, M. Gautier, Model identification, in: B. Siciliano, O. Khatib (Eds.), Springer Handbook of Robotics, 2nd Edition, Springer, 2016, pp. 113–138.
  * [2] O. Birbach, U. Frese, B. Bäuml, Rapid calibration of a multi-sensorial humanoid’s upper body: An automatic and self-contained approach, The International Journal of Robotics Research 34 (4-5) (2015) 420–436.
  * [3] K. Stepanova, J. Rozlivek, F. Puciow, P. Krsek, T. Pajdla, M. Hoffmann, Self-calibration dataset, http://karlastepanova.cz/publications/self-calibration-industry/ (2020).
  * [4] L. S. Ginani, J. M. S. Motta, Theoretical and practical aspects of robot calibration with experimental verification, Journal of the Brazilian Society of Mechanical Sciences and Engineering 33 (1) (2011) 15–21.
  * [5] I.-C. Ha, Kinematic parameter calibration method for industrial robot manipulator using the relative position, Journal of Mechanical Science and Technology 22 (6) (2008) 1084.
  * [6] H.-N. Nguyen, J. Zhou, H.-J. Kang, A new full pose measurement method for robot calibration, Sensors 13 (7) (2013) 9132–9147.
  * [7] W. S. Newman, C. E. Birkhimer, R. J. Horning, A. T. Wilkey, Calibration of a motoman p8 robot based on laser tracking, in: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), Vol. 4, IEEE, 2000, pp. 3597–3602.
  * [8] A. Nubiola, I. A. Bonev, Absolute calibration of an abb irb 1600 robot using a laser tracker, Robotics and Computer-Integrated Manufacturing 29 (1) (2013) 236–245.
  * [9] M. Švaco, B. Šekoranja, F. Šuligoj, B. Jerbić, Calibration of an industrial robot using a stereo vision system, Procedia Engineering 69 (2014) 459–463.
  * [10] J.-S. Hu, J.-J. Wang, Y.-J. Chang, Kinematic calibration of manipulator using single laser pointer, in: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2012, pp. 426–430.
  * [11] P. Beneš, M. Valášek, Z. Šika, V. Bauma, V. Hamrle, Experiments with redundant parallel calibration and measuring machine RedCaM, Tech. rep., University of West Bohemia (2007).
  * [12] J. Volech, L. Mráz, Z. Šika, M. Valášek, Concepts of robot accuracy enhancement by integrated redundant measurements., Bulletin of Applied Mechanics 9 (33) (2013).
  * [13] V. Petrík, V. Smutný, Comparison of calibrations for the CloPeMa robot, Tech. rep., Czech Technical University in Prague (2014).
  * [14] D. J. Bennett, J. M. Hollerbach, Autonomous calibration of single-loop closed kinematic chains formed by manipulators with passive endpoint constraints, IEEE Transactions on Robotics and Automation 7 (5) (1991) 597–606.
  * [15] M. Hersch, E. Sauser, A. Billard, Online learning of the body schema, International Journal of Humanoid Robotics 5 (2008) 161–181.
  * [16] R. Martinez-Cantin, M. Lopes, L. Montesano, Body schema acquisition through active learning, in: Proc. Int. Conf. on Robotics and Automation (ICRA), 2010\.
  * [17] S. R. Fanello, U. Pattacini, I. Gori, V. Tikhanoff, M. Randazzo, A. Roncone, F. Odone, G. Metta, 3D stereo estimation and fully automated learning of eye-hand coordination in humanoid robots, in: 2014 IEEE-RAS Int. Conf. on Humanoid Robots - HUMANOIDS ’14, 2014.
  * [18] P. Vicente, L. Jamone, A. Bernardino, Online body schema adaptation based on internal mental simulation and multisensory feedback, Frontiers in Robotics and AI 3 (2016) 7.
  * [19] D. J. Bennett, J. M. Hollerbach, Self-calibration of single-loop, closed kinematic chains formed by dual or redundant manipulators, in: Decision and Control, 1988., Proceedings of the 27th IEEE Conference on, IEEE, 1988, pp. 627–629.
  * [20] A. Joubair, I. A. Bonev, Kinematic calibration of a six-axis serial robot using distance and sphere constraints, The International Journal of Advanced Manufacturing Technology 77 (1-4) (2015) 515–523.
  * [21] M. C. Koval, M. Klingensmith, S. S. Srinivasa, N. S. Pollard, M. Kaess, The manifold particle filter for state estimation on high-dimensional implicit manifolds, in: Robotics and Automation (ICRA), 2017 IEEE International Conference on, IEEE, 2017, pp. 4673–4680.
  * [22] X. Xu, D. Zhu, J. Wang, S. Yan, H. Ding, Calibration and accuracy analysis of robotic belt grinding system using the ruby probe and criteria sphere, Robotics and Computer-Integrated Manufacturing 51 (2018) 189–201.
  * [23] M. Ikits, J. M. Hollerbach, Kinematic calibration using a plane constraint, in: Proceedings of International Conference on Robotics and Automation, Vol. 4, IEEE, 1997, pp. 3191–3196.