# Kinematic Optimization of a Robotic Arm for Automation Tasks with Human
Demonstration

Inbar Meir, Avital Bechar and Avishai Sintov I. Meir and A. Sintov are with
the School of Mechanical Engineering, Tel-Aviv University, Israel. e-mail:
{inbarb3<EMAIL_ADDRESS>Bechar is with the The Institute of
Agricultural Engineering, Volcani Institute, Israel.This research was partly
supported by the Ministry of Science and Technology of Israel.

###### Abstract

Robotic arms are highly common in various automation processes such as
manufacturing lines. However, these highly capable robots are usually degraded
to simple repetitive tasks such as pick-and-place. On the other hand,
designing an optimal robot for one specific task consumes large resources of
engineering time and costs. In this paper, we propose a novel concept for
optimizing the fitness of a robotic arm to perform a specific task based on
human demonstration. Fitness of a robot arm is a measure of its ability to
follow recorded human arm and hand paths. The optimization is conducted using
a modified variant of the Particle Swarm Optimization for the robot design
problem. In the proposed approach, we generate an optimal robot design along
with the required path to complete the task. The approach could reduce the
time-to-market of robotic arms and enable the standardization of modular
robotic parts. Novice users could easily apply a minimal robot arm to various
tasks. Two test cases of common manufacturing tasks are presented yielding
optimal designs and reduced computational effort by up to 92%.

##  I Introduction

Robotic arms are the foundation of modern automation for manufacturing. They
accommodate production lines and perform the majority of tasks such as
assembly, machining, painting, welding and packaging [1, 2, 3]. While robotic
arms are highly capable with multiple degrees-of-freedom (DOF), a robot is
usually degraded to solely perform one simple repetitive task along the
production line [4]. These robotic arms are commonly standard off-the-shelve
hardware which are redundant for the task at hand and, therefore, highly
expensive having direct impact on the final product cost. Hence, many highly
capable and expensive robots are purchased and integrated just to perform
simple tasks. While simpler and cheaper robots can usually perform the same
tasks, they are rarely available as an off-the-shelve product [5]. On the
other hand, designing designated robots for each specific task requires
complex and careful work by professional engineers. The process consumes a
considerable amount of engineering time and may increase final product cost.

Figure 1: Illustration of a robot kinematics evaluated on a recorded task. In
this example, the robot end-effector must follow a recorded path of the human
hand (circular red markers) while its arm should follow the human arm markers
(circular blue and green markers) to avoid obstacles. The dashed curves are
the paths of the optimal robotic arm that best track the recorded human ones.

Some operational tasks are planned to comply with the natural motion of the
human arm. Transitioning from manual work to robotic automation usually
involves off-the-shelve robotic arms with kinematics that may not perform well
in such environment. Hence, an optimally designed kinematic structure is
expected to improve performance and reduce costs. Furthermore, while
integrating the new robot, a path to be repeated over and over is usually pre-
recorded by a technician while considering robot coordinates [6]. The
recording is acquired through the teach pendant which provides non-intuitive
and non-natural control abilities [7]. Consequently, the resulted path may be
non-optimal to the task nor to the environment. Alternatively, explicit
mapping of the environment is required in order to validate the ability of a
candidate robot to execute a collision-free path [8].

In this work, we propose a novel concept of task-oriented robot design based
on expert demonstration. We observe a human expert performing a task. The
whole arm motion of the expert is recorded by tracking paths of markers from
shoulder to hand (or grasped tool). Then, we formulate an optimization problem
that searches for an optimal robotic arm that can accurately track the
recorded task. In order to be able to avoid obstacles as the expert did,
tracking includes the End-Effector (EE) of the robot as well as its entire
kinematic chain as in Figure 1. The optimization searches for the Denavit-
Hartenberg (DH) parameters [8] that define a robot kinematics.

In this study, we propose the Robot Arm Particle Swarm Optimization (RA-PSO)
algorithm to efficiently solve the design problem. RA-PSO is a modified
version of the known PSO method [9] and is particularly aimed to optimize
robotic arms based on recorded paths. RA-PSO has modified rules to fit the
design space of robots and for efficient exploration of the valid search
space. The search space in this problem is highly non-linear and non-
continuous. We also observe the Computational Effort (CE) and compare it to
the traditional PSO. Our approach distributes the search over different design
variables to enable efficient exploration and low CE.

The method does not only provide an optimal robot design but also defines a
collision free joint path to complete the task. Hence, no additional motion
planning nor obstacle mapping are required. Indeed, an experienced
practitioner can plan a more suitable path without considering demonstrations.
However, a novice user may not have such capabilities and, therefore, the
proposed approach reduces required engineering work and effort. A novice user
can simply record an expert path and acquire both a robot design and a
suitable motion. To reduce costs, this approach motivates the development of
standard and modular robotic hardware, as proposed in [10], such that the
novice user can rapidly assemble and operate custom arms for different tasks
based on the optimization outputs. Consequently, the resultant arm could be
low-cost, minimal and suitable for the environment and task with low
engineering and computational efforts.

##  II Related Work

Many research studies have addressed the kinematic optimization of robotic
arms while focusing mostly on increasing dexterity and workspace volume [11,
12]. Recent work in [13] used the Denavit-Hartenberg (DH) representation [14]
to define the design variables of an eight degrees-of-freedom (DOF) upper-limb
rehabilitation exoskeleton. A constrained multi-objective optimization problem
was formulated utilizing the Genetic Algorithm (GA) to search for the design
variables that provide minimum size and maximum dexterity. The kinematic
design optimization of an anthropomorphic robot using GA was proposed to
maximize the multi-fingered precision grasping capability of the robot hand
[15]. While these addressed enhancing capabilities for general tasks, others
have addressed a task-specific optimization problem [16, 17]. A kinematic
optimization framework was also presented to find the DH parameters of a robot
such that it can reach any task point in a defined task subset [8].

Kinematic optimization solutions, such as the ones previously mentioned, use
different performance measures to evaluate the kinematics of a robotic arm for
general tasks [18]. A common measure is the Manipulability ellipsoid that
approximates the distance to singular configurations [19]. Similarly, the
Global Isotropy Index (GII) measures the ratio between the minimum and maximum
singular values of the Jacobian [20]. These, however, do not consider tracking
accuracy for task-specific paths.

Since cost functions that reflect the kinematics of the robot arm are non-
linear, gradient-based techniques will most likely fail to find global
solutions and, therefore, are not common [13]. On the other hand, nature-
inspired meta-heuristic approaches (e.g., PSO, Artificial Bee Colony, Ant
Colony Optimization) are more capable in searching for the global minimum in
complex problems [21]. For instance, Simulated Annealing (SA) was used in the
design of a surgical robot to optimize anatomical visibility [17]. GA was used
to optimize the design of a manipulator with the GII measure [22]. Similarly,
PSO was used to identify the optimal cable routing for a cable-driven
manipulator [23]. We observe the performance of these methods for robot
design.

Human demonstrations have been used in robotics for many applications such as
augmenting reinforcement learning [24], motion re-targeting [25], imitation
learning [26], motion planning [27] and computational design [28, 29]. In
addition, task-centric optimization was introduced to select pose
configurations of an existing robot for efficient assistive tasks based on
sampled human poses [30]. However and to the best of the author’s knowledge,
no attempt has been made to optimize the kinematics of a robotic arm for a
whole arm tracking of a human demonstrated task. A closely related work
proposed the synthesis of an arm by fitting EE poses on a set of points [31].
Similarly, kinematic synthesis was proposed such that the EE of an under-
actuated robotic arm would track some path [32]. However, the derivative of
the path must be available at any point for generalized differential Inverse
Kinematics (IK). Such derivative cannot be acquired qualitatively from a
recorded demonstration. Furthermore, optimization has been attempted to solely
decide joint displacements and, did not observe human demonstrations. In
addition, having an EE track a path does not ensure that it will be able to
prevent collisions with the arm links. Hence, we consider the path of the
entire arm in order to avoid obstacles in the environment and provide a
complete solution.

##  III Methodology

###  III-A Optimization objective

A task is given in which an $n$ degrees-of-freedom (DOF) robotic arm must move
in an industrial environment with obstacles. The task could be moving an
object from one pose to another (e.g., packaging) or tracking a path (e.g.,
welding). The assumption is that the environment is static. The human path is
recorded once and, therefore, dynamic obstacles cannot be considered. We aim
to find an optimal robotic arm that can perform the task. Without loss of
generality, we consider only rotary joints. An optimal robot is the one that
provides best accuracy during task execution. We consider position errors for
the entire arm from base to end-effector so that arm distance from obstacles
can be maximized. In this work, we address the optimization problem of the
kinematic properties while non-optimal dynamic parameters are commonly
compensated using slow motions and control techniques. When the system and
task environment are much larger than human capabilities, a scaled mock-up can
be used with the proposed approach to compute an optimal robot and later be
scaled-up.

###  III-B Approach

As discussed previously, we consider a task in a static environment. Hence and
in light of the above objective, we seek for a robotic arm with minimal DOF
that can accurately track a single path. The single path, in this work, is a
path recorded by a human demonstrator. We assume that a path recorded by a
human expert is the shortest while maximizing the distance from obstacles. The
robot is modeled with thin segments and, therefore, we assume that the
recorded paths are far enough from obstacles. Hence, the better a robot is
able to accurately track the path, the less it is sensitive to positioning
errors and risk of collisions.

Given a task, the motion of the arm of a human expert is recorded. Let $m$ be
the number of tracked markers on the arm, a recorded task is given by $m$
paths

$\mathcal{P}=\\{P_{1},P_{2},\ldots,P_{m}\\}$ (1)

where $P_{i}=\\{\mathbf{p}_{i,0},\ldots,\mathbf{p}_{i,T}\\}$ is a path of the
$i^{th}$ marker relative to the shoulder. $\mathbf{p}_{i,t}\in\mathbb{R}^{3}$
is the spatial coordinates of the $i^{th}$ marker at time $t$ (Figure 1). We
note that $P_{m}$ is the path of the human hand, which implies about the
desired path of the end-effector. We also record a set
$\mathcal{V}=\\{\mathbf{u}_{0},\ldots,\mathbf{u}_{T}\\}$ where elements in
$\mathbf{u}_{t}\in\mathbb{R}^{3}$ are the Euler angles of the hand or tool at
time $t$. In a case where the recorded paths are not smooth due to hand
shaking or vibration, some filtering can be applied. The recorded task can now
be used to evaluate the ability of some robot arm to accurately track
$\mathcal{P}$.

|
---|---
(a) | (b)
Figure 2: (a) Illustration of a robot parameterized by $\Phi$ where any point
along it can be represented by $\mathbf{s}_{\Phi}(\sigma,\mathbf{q})$. Points
$\sigma=0$ and $\sigma=1$ are the base and end-effector points, respectively.
(b) Area to minimize (yellow) for a better fit between the robot arm (solid
lines) and lines (dashed) formed by the recorded markers (red).

###  III-C Robot design variables

To optimize the design of an $n$-DOF robotic arm, we first define the
variables that describe its kinematics. The kinematics of a robot can be
defined by a set of coordinates described by the Denavit-Hartenberg (DH)
method [14]. In DH, the forward kinematics of an arm is represented by the
product of a set of canonical homogeneous transformation matrices $A_{i}\in
SE(3)$ where $i\in\\{1,\ldots,n\\}$. Transformation matrix
$A_{i}(a_{i},\alpha_{i},\theta_{i},d_{i})$ defines the position and
orientation of the coordinate frame of joint $i$ relative to frame $i-1$. The
four parameters $a_{i}$, $\alpha_{i}$, $\theta_{i}$ and $d_{i}$ are standard
DH geometric quantities. Matrix $A_{i}$ incorporates only one DOF and, thus,
depends only on one variable $q_{i}$. This variable would be the joint angle
$q_{i}=\theta_{i}$ for a revolute one. The remaining three are constants and
part of the design of the robot. Let $\phi_{i}\in\mathbb{R}^{3}$ be a
parameterization vector that encodes the parameters of transformation from
frame $i$ to frame $i-1$. Hence, $\phi_{i}$ is given by

$\phi_{i}=(\alpha_{i},a_{i},d_{i}).$ (2)

We define $\mathbf{q}=(q_{1},\ldots,q_{n})\in\mathcal{C}$ as the vector of
joint values of the arm where $\mathcal{C}\subset\mathbb{R}^{n}$ is the
configuration space of the robot. Vector $\Phi\in\Omega$, where
$\Omega\subseteq\mathbb{R}^{3n}$, is the concatenation of all $\phi_{i}$ given
by $\Phi=(\phi_{1},..,\phi_{n})$ and is the encoding of the constant
parameters. Consequently, the product

$A_{ee}(\mathbf{q},\Phi)=\prod_{i=1}^{n}A_{i}(q_{i},\phi_{i})$ (3)

is the robots forward kinematics that express the position and orientation of
its EE with regards to the base frame $\mathcal{O}_{b}$. Vector $\Phi$ in (3)
fully-defines the design of a robot and, therefore, $\Omega$ is the search
space for the optimization problem.

###  III-D Allocate robot tracking point

Given a robot defined by $\Phi$, let
$\mathbf{s}_{\Phi}:[0,1]\times\mathcal{C}\to\mathbb{R}^{3}$ be a map such that
$\mathbf{s}_{\Phi}(0,\mathbf{q})=\mathbf{0}$ is the position of the robot base
and $\mathbf{s}_{\Phi}(1,\mathbf{q})$ is the position of its EE. Map
$\mathbf{s}_{\Phi}$ is computed using the forward-kinematics described in
Section III-C. Hence, $\mathbf{s}_{\Phi}(\sigma,\mathbf{q})$ with
$\sigma\in[0,1]$ is a parametric function providing the position of any given
point $\sigma$ along a one-dimensional representation of the arm at
configuration $\mathbf{q}$ as seen in Figure 2a. Function $\mathbf{s}_{\Phi}$
is non-smooth due to transitions between links at the joints. We also define
function $\mathbf{b}_{\Phi}(\mathbf{q}):\mathcal{C}\to\mathbb{R}^{3}$ which
outputs the Euler angles of the EE for robot $\Phi$ at configuration
$\mathbf{q}$.

###  III-E Robot temporal fitness

A recorded task (1) is composed of a sequence of time frames. For each
individual frame, we seek to evaluate the fitness of a given robot to the
corresponding recorded points. A temporal fitness at time $t$ is defined to be
the weighted root-mean-square error (wRMSE) between points
$\mathbf{p}_{1,t},\ldots,\mathbf{p}_{m,t}$ and the robot. Therefore, the wRMSE
at time $t$ for robot configuration $\mathbf{q}$ is given by

$\displaystyle
g_{\Phi}(t,\mathbf{q})=\frac{1}{m}\sqrt{\sum_{i=1}^{m}w_{i}\|\mathbf{s}_{\Phi}(\sigma_{i},\mathbf{q})-\mathbf{p}_{i,t}\|^{2}}+$
(4) $\displaystyle+w_{0}\|\mathbf{b}_{\Phi}(\mathbf{q})-\mathbf{u}_{t}\|$

where parameter $\sigma_{i}$ associates $\mathbf{p}_{i,t}$ to the closest
point on the robot. Scalars $\\{w_{0},\ldots,w_{m}\\}\in\mathbb{R}^{+}$ with
$\sum_{0}^{m}w_{i}=1$ are user-defined weight values that prioritize the
position importance of the robot segments relative to the recorded path. In a
non-cluttered environment, for instance, the position of the end-effector may
have higher significance compared to other links. On the other hand, when
working in cluttered environments, the body link positions have higher
importance.

The temporal robot fitness $G_{t}$ at time $t$ for a robot represented by
$\Phi$ is given by

$\displaystyle G_{\Phi,t}=\min_{\mathbf{\sigma},\mathbf{q}_{t}}$
$\displaystyle g_{\Phi}(t,\mathbf{q}_{t})$ (5) s.t.
$\displaystyle\mathbf{q}_{t}\in\mathcal{C}_{f}$
$\displaystyle\|\mathbf{q}_{t}-\mathbf{q}_{t-1}\|\leq\epsilon$
$\displaystyle\sigma_{j}<\sigma_{k},~{}~{}\forall j<k$

where $\mathbf{\sigma}=(\sigma_{1},\ldots,\sigma_{m})^{T}$,
$\mathbf{q}_{t}\in\mathcal{C}$ is the joint configuration at time $t$ and
$\mathcal{C}_{f}\subset\mathcal{C}$ is the feasible configuration subspace to
be discussed below. The second constraint enforces joint path continuity by
requiring that two consecutive configurations have a distance smaller than
$\epsilon>0$. Hence, the initial guess for optimization problem (5) would be
$\mathbf{q}_{t-1}$. Optimization problem (5) minimizes the wRMSE of the
euclidean distances for all recorded markers from the robot for a single time
frame. The solution of problem (5) is, in fact, the solution of a multi-point
Inverse Kinematics. Next, we use this formulation to calculate the fitness
along an entire path.

###  III-F Robot path fitness

Given path set $\mathcal{P}$, the path fitness $f(\Phi)$ of a query robot
design $\Phi$ is the mean of the temporal fitnesses given by (5) for all time
frames $t\in[0,T]$. Formally, a path fitness for robot $\Phi$ is defined as

$f(\Phi)=\frac{1}{T}\sum_{t=0}^{T}G_{\Phi,t}.$ (6)

Since we require continuous joint motion and we fix the initial pose of the
end-effector based on the task, (6) is computed sequentially from $t=0$.

Minimizing $f(\Phi)$ reduces the distance between the robot and recorded
markers. However, such minimization is not sufficient since the output may
yield a long arm that over-fits the markers as seen on Figure 2b. That is, the
path fitness can be zero while the arm is very long. Hence, we also minimize
the area $E(\Phi)$ (yellow region in the figure) between robot links and
represented markers. $E(\Phi)$ is computed as follows. Recall that, at each
time frame, the arm is separated into $m$ parts by $\sigma$ from the solution
of (5). We divide each part into segments of length $\delta_{m}$ along the
robot axis. For each segment, we calculate the distance from the line formed
by the two closest markers. $E(\Phi)$ is the mean distance along the robot
length and at all time frames. Note that minimizing $E(\Phi)$ indirectly also
minimizes robot length.

Finally, the optimal robotic arm $\Phi^{*}$ that can track path $\mathcal{P}$
most accurately is the one with the minimal robot fitness value and is the
solution of

$\begin{array}[]{lrrclcl}\Phi^{*}=&\displaystyle
arg\min_{\Phi}&\lx@intercol\lambda_{f}f(\Phi)+\lambda_{E}E(\Phi)\hfil\lx@intercol\\\
&\textrm{s.t.}&\Phi\in\Omega_{f}\\\ \end{array}$ (7)

where $\lambda_{f}>0$ and $\lambda_{E}>0$ are weighting values. Hence, we wish
to minimize the tracking accuracy, link lengths and spatial link movements.
Minimization of link lengths will result in a smaller and cheaper arm, along
with lower joint loads, i.e., not requiring high-torque actuators. It is
important to note that the higher the dimensionality of $\Phi$, i.e., more
DOF, it is more likely for the robot to better track the path. However, we
seek for a design comprising of minimal DOF. Subset $\Omega_{f}\subset\Omega$
is the allowed robot design search space defined in the next section.

###  III-G Boundaries and Constraints

We now define the joint and design search spaces, $\mathcal{C}_{f}$ and
$\Omega_{f}$, respectively. The allowed joint search space $\mathcal{C}_{f}$
in (5) ensures proper physical movement of a query robot configuration and is
defined by the joint limits

$\mathcal{C}_{f}=\\{\mathbf{q}\in\mathcal{C}|~{}\mathbf{q}_{min}<\mathbf{q}<\mathbf{q}_{max}\\}$
(8)

where $\mathbf{q}_{min}$ and $\mathbf{q}_{max}$ are the minimum and maximum
actuators angle limits, respectively. For any query robot $\Phi$, the world
coordinate frame is positioned at the base. The EE coordinate frame is fixed
to the last robot link. The static values for the DH parameters in (2) are
limited according to

$\alpha_{i}\in[\alpha_{min},\alpha_{max}],~{}a_{i}\in[0,a_{max}]\text{ and
}d_{i}\in[0,d_{max}]$ (9)

where $\alpha_{min},~{}\alpha_{max},~{}a_{max}$ and $d_{max}$ are pre-defined
user variables. Furthermore, we constrain any two consecutive joints to be
non-concentric in order to avoid redundancy. Such constraint assists in
tightening the search sub-space. A robot solution is considered invalid and
omitted if its total length $L$ does not satisfy

$L_{min}<L<L_{max}$ (10)

where $L_{min}$ and $L_{max}$ are user-defined values. This constraint also
aims to filter-out lengthy designs early on. Moreover, if the tested robot
configuration cannot reach EE position at $t=0$ with accuracy of 1 mm, the
rest of the path is not calculated. As stated above, the valid search space
for $\mathbf{q}_{t+1}$ depends on the previous solution $\mathbf{q}_{t}$.
However, the valid search space for $\mathbf{q}_{0}$ is defined by the joint
angular limits. Hence, the robot can be initialized from different joint
values and the rest of the path will be defined accordingly. Nevertheless,
preliminary tests show high repetition for $\mathbf{q}_{0}$.

Let $\Omega_{l}\subset\Omega$ be the set of design configurations that satisfy
(9)-(10) and let $\mathcal{Q}_{\Phi}$ be the workspace of robot $\Phi$ such
that $\mathbf{s}_{\Phi}(\sigma,\mathbf{q})\in\mathcal{Q}_{\Phi}$ for any
$\mathbf{q}\in\mathcal{C}_{f}$ and $\sigma\in[0,1]$. Consequently, a valid
robot configuration must lie within sub-set $\Omega_{f}\subset\Omega$ defined
as

$\Omega_{f}=\\{\Phi\in\Omega|~{}\Phi\in\Omega_{l},~{}\mathcal{P}\subset\mathcal{Q}_{\Phi}\\}.$
(11)

Subset $\Omega_{f}$ contains all valid robot configurations.

|
---|---
Figure 3: Recording of a human arm path in an industrial (Scenario I)
packaging and (Scenario II) welding. A robotic arm is to be optimized to
accurately track the recorded path. The end-effector of the robot must follow
the path of the hand markers while its arm should follow the human arm markers
to avoid obstacles. For Scenario II, the EE must also track the orientation of
the human hand.

###  III-H Optimization algorithm

The definition of the robot fitness and constraints can now be used to search
for the optimal design $\Phi^{*}$ of a robot. As discussed in Section I,
kinematic optimization problems are commonly highly non-linear and cannot be
efficiently solved using gradient-based techniques. Hence, we employ a meta-
heuristic search approach.

Our proposed algorithm is based on the PSO, a meta-heuristic optimization
algorithm. In PSO, we randomly sample $N$ particles
$\mathcal{K}=\\{\Phi_{1},\ldots,\Phi_{N}\\}$ from a uniform distribution in
$\Omega_{l}$. Each particle $\Phi_{j}$ is evaluated yielding robot fitness
$r_{j}$ and the corresponding path
$Q_{j}=\\{\mathbf{q}_{0},\ldots,\mathbf{q}_{T}\\}$ by solving the cost
function in (7). We store and update a personal best position vector $bp_{j}$
for particle $j$ based on robot fitness. The global best position for all the
particles in $\mathcal{K}$ is stored in $\Phi^{*}$ with fitness $r^{*}$. Each
particle is then updated according to
$\Phi_{j}\leftarrow\Phi_{j}+\mathbf{v}_{j}$ where $\mathbf{v}_{j}$ is updated
by

$\mathbf{v}_{j}\leftarrow
w~{}\mathbf{v}_{j}+\beta_{1}c_{1}(bp_{j}-\Phi_{j})+\beta_{2}c_{2}(\Phi^{*}-\Phi_{j})),$
(12)

constants $w$, $c_{1}$ and $c_{2}$ are tunable parameters, and $\beta_{1}$ and
$\beta_{2}$ are random numbers in $[0,1]$. The particles are iteratively
updated for $M$ iterations. Preliminary results have shown that the standard
update rules of PSO face difficulties converging to a near optimal solution
due to the large hyper-volume and non-linear behavior of $\Omega$.

In order to improve the search, we propose the RA-PSO algorithm, a designated
variant of PSO, designed to address the properties of search space $\Phi$. RA-
PSO maximizes exploitation of particles within $\Omega_{f}$ while avoiding
missing-out improved solutions nearby. In order to improve local search, we
include two modifications in the update rules of PSO. First, we distinguish
between particles currently in $\Omega_{f}$ and ones that are not. Particles
not in $\Omega_{f}$ are updated according to the standard PSO which cause
aggressive change in the candidate robot kinematics to explore new regions. On
the other hand, we update a particle $\Phi_{j}$ that is within $\Omega_{f}$ in
a significantly lower rate such that its new position is nearby for local
refinement and effective exploitation. The velocity $\mathbf{v}_{j}$ of an
agent in $\Omega_{f}$ is updated according to

$\mathbf{v}_{j}\leftarrow\mathbf{v}_{j}+\texttt{random}(c_{min},c_{max})\cdot\phi_{range}$
(13)

where $c_{min},c_{max}$ are random number bounds and $\phi_{range}$ is the
range of the corresponding DH feature in $\Omega_{f}$.

The second modification improves local search in $\Omega_{f}$. Since $\Omega$
is large, high-dimensional and non-linear, we explore different dimensions of
$\Omega$ by their physical representation. That is, we separately update DH
angular (i.e., $\alpha_{i}$) and length (i.e., $a_{i}$ and $d_{i}$) variables.
In such way, the algorithm is able to better explore subspaces of
$\Omega_{f}$. Let $D\in\mathbb{N}$ be the update frequency of the angular
variables. Then, angular variables in $\Phi_{j}$ are updated every $D$
iterations. This method assists in testing various link lengths with the same
angular variables and better improve the exploration of $\Omega_{f}$.

TABLE I: Optimization parameters

User-defined parameters $l$ 0.7 m $L_{min}$ 0.6 m $L_{max}$ 1.2 m $\epsilon$
$10^{\circ}$ $a_{max}$ $0.5$ m $d_{max}$ $0.5$ m $\alpha_{min}$ $-90^{\circ}$
$\alpha_{max}$ $90^{\circ}$ $q_{max}$ $180^{\circ}$ $q_{min}$ $-180^{\circ}$
$\delta_{m}$ 10 mm Optimization hyper-parameters $w$ $0.8$ $c_{1}$ $0.4$
$c_{2}$ $0.6$ $N~{}400$ $c_{max}$ $0.5$ $c_{min}$ $-0.5$ $M~{}200$ Scenario I
\- Optimization weights $w_{0}$ 0 $w_{1}$ $1/6$ $w_{2}$ $1/3$ $w_{3}$ $1/2$
$\lambda_{f}$ 15 $\lambda_{E}$ 5 Scenario II \- Optimization weights $w_{0}$
0.2 $w_{1}$ $0$ $w_{2}$ $0.1$ $w_{3}$ $0.7$ $\lambda_{f}$ 15 $\lambda_{E}$ 5

##  IV Experiments

###  IV-A Setup and apparatus

Using a motion capture system, we have recorded a human arm preforming desired
tasks. During recording, four bands are places on the shoulder, upper arm,
forearm and hand. Each band has several reflective markers considered together
as a rigid body. Hence, the motion capture system provides real-time data
stream of spatial positions of the bands. Data stream may also include
orientation of the bands and the hand band in particular. The shoulder is
considered as the base position. The desired robot motion $\mathcal{P}$ is,
therefore, the paths of the $m=3$ distal bands relative to the shoulder band.
All computations were implemented in Python on an Intel-Core Intel Xeon Gold
6130 Ubuntu machine with 12$\times$32GB of RAM. Optimization algorithms are
based on the Opytimizer package [33].

TABLE II: Optimization performance for solving robot temporal fitness (5)
using various meta-heuristic algorithms

Mean $G_{\Phi,t}$ (mm) Mean comp. $n$ 3 4 5 6 time (s) GA 49.0 $\pm$25.1
72.9$\pm$50.9 48.5$\pm$41.7 36.1$\pm$17.6 10.1 PSO 49.3 $\pm$25.0 71.9
$\pm$52.1 48.7$\pm$42.9 32.7 $\pm$17.9 5.7 WOA 53.1$\pm$42.8 74.7 $\pm$50.0
71.8$\pm$9.9 40.9 $\pm$19.5 5.8 ES 48.9 $\pm$25.1 71.9$\pm$51.8 49.4$\pm$42.8
35.9 $\pm$18.4 8.7 ABC 48.7 $\pm$25.2 69.5 $\pm$53.2 44.4$\pm$43.6 29.5
$\pm$18.4 17.5 SLSQP 179.2$\pm$33.5 207.4 $\pm$34.3 203.9$\pm$40.7
214.8$\pm$30.9 0.021 SA 51.0 $\pm$24.6 82.4$\pm$51.7 63.0$\pm$45.3
54.9$\pm$19.6 11.5

###  IV-B Test cases

Two test scenarios, seen in Figure 3, are considered. In packaging Scenario I,
an item is picked-up from a conveyor belt, manipulated around an obstacle and
placed within a cardboard box; and a welding Scenario II where a tool must
trace a path across an object. The figures also show the recorded paths
$\mathcal{P}$ of the three distal bands relative to the shoulder band. In
Scenario II, we constrain the robot EE to also trace the orientation of the
human hand. In both scenarios, one repetition of the human hand performing the
task was recorded. The user-defined parameters and optimization hyper-
parameters used for both tests cases are shown in Table I. The hyper-
parameters were chosen based on preliminary analysis.

###  IV-C Results & Discussion

A comparison analysis was conducted on seven optimization algorithms to solve
robot temporal fitness $G_{\Phi,t}$ in (5). The optimization algorithms
include PSO, Artificial Bee Colony (ABC), Evolution Strategies (ES), Whale
Optimization Algorithm (WOA), Genetic Algorithm (GA), Simulated Annealing (SA)
and Sequential Least Squares Programming (SLSQP) [34]. We note that RA-PSO is
proposed for minimizing path fitness while these are benchmarked to solve only
for the temporal fitness. The results of each algorithm are averaged over ten
different robot configurations in three different time instants along
$\mathcal{P}$. Results are summarized in Table II. Presented time results were
computed on the same computer. First, SLSQP does not converge well and is,
therefore, not suitable for this problem. All other algorithms converge
relatively similar with small variations about the mean temporal fitness. In
particular, PSO provides somewhat good convergence with the lowest computation
time. Therefore, the temporal fitness is further solved using PSO.

Furthermore, we provide mean (over 30 repetitions) robot fitness results over
all $n$ for solving problem (7) with four different algorithms: ABC, GA, PSO
and SA. The results, presented in Table III, show that PSO provides best
fitness and computation time. This provides motivation for pushing forward
with PSO to improve convergence.

We next observe the impact of update frequency $D$ in RA-PSO on fitness and
computational effort (CE) for both scenarios. While both PSO and RA-PSO share
the same complexity, their CE is significantly affected by two factors. First,
the more valid agents in $\Omega_{f}$ results in more cost evaluations. If the
two algorithms reach the same fitness while one requires less agents, its
effort is lower. Second, more iterations to convergence requires more
computations. The CE is, therefore, $\bar{N}_{v}\cdot W$ where $\bar{N}_{v}$
is the mean number of valid agents and $W$ is the number of iterations to
convergence [35]. An algorithm with a lower CE is more computation efficient.
Mean fitness and CE improvement results (over 30 repetitions) with regards to
$D$ are shown in Table IV. Negative results indicate an improvement compared
to PSO. While fitness improvement marginally occurs in some cases, RA-PSO
significantly improves CE over all $D$ for both scenarios. Results show the
ability of RA-PSO to better explore $\Omega_{f}$ with less agents and
iterations, while also maintaining low fitness.

TABLE III: Optimization performance over all $n$ for solving robot fitness (7)
using various meta-heuristic algorithms

Alg. ABC GA PSO SA Mean $\Phi^{*}$ 0.64 0.70 0.51 0.71 Comp. time (h) 3.59
4.00 0.91 1.28

TABLE IV: Optimization performance of RA-PSO for solving robot fitness (7)

$\Phi^{*}$ improvement (%) CE improvement (%) D D n Sc. 2 3 4 5 2 3 4 5 I
-10.5 -15.6 -5.9 -7.4 -90.6 -92.0 -90.4 -88.9 3 II 3.0 9.8 0.8 5.0 -89.9 -91.8
-91.0 -88.9 I 1.4 0.9 -0.4 -7.0 -66.7 -75.0 -64.2 -66.7 4 II 2.9 8.7 9.1 7.7
-85.4 -78.5 -86.8 -79.9 I 0.7 -1.1 5.5 12.0 -61.7 -54.6 -67.9 -86.9 5 II -0.2
4.2 9.2 8.1 -56.6 -69.0 -89.0 -80.6 I -2.3 -3.1 3.8 -2.3 -8.4 -23.5 -44.3
-32.9 6 II 3.6 7.2 7.1 6.8 -72.7 -66.3 -74.7 -78.6 Mean -0.17 1.37 3.65 2.86
-66.5 -68.8 -76.0 -75.4

We seek for $D$ that provides the best performance both in $\Omega_{f}$ and
CEI. One can choose the value for $D$ individually for a scenario and a
specific $n$. However and for generality, we choose the trade-off between CEI
and fitness improvements. Hence, we choose to use $D=2$ in which fitness is
less compromised while providing significant CEI improvement of 66.5%. Figure
4 presents the fitness results for both scenarios with $D=2$ and with regards
to $n$. For Scenario I, we prefer $n=3$ since the difference in path fitness
from larger $n$ is lower than just 4%. In Scenario II, however, the difference
is larger as the EE must track the hand orientation. Hence, we choose the
minimal fitness design in which $n=5$. Table V presents the parameters of the
optimal robots. Figures 5 and 6 show a ROS-Gazebo simulation of the optimal
robots tracking recorded paths. Videos of optimal robots can be seen in the
supplementary material.

|
---|---
Figure 4: Fitness results for Scenarios (left) I and (right) II. TABLE V:
Final parameters of the optimal robotic arms

Sc. $n$ $\Phi^{*}$ $f(\Phi)$ (mm) $E(\Phi)$ $i$ $\alpha_{i}$ $a_{i}$ $d_{i}$ I
3 0.43 17.59 33.23 $0$ 0.515 0 0 $1$ -1.57 0.11 0 $2$ -0.27 0.5 0 $EE$ -1.57 0
0.1 II 5 0.83 35.83 58.32 $0$ $-1.66$ $0$ $0$ $1$ $-0.10$ $0.09$ $0.31$ $2$
$0.86$ $0$ $0$ $3$ $-0.20$ $0.28$ $0$ $4$ $0.66$ $0$ $0$ $EE$ $-0.2$ $0$ $0.1$

Figure 5: A near-optimal 3-DOF robotic arm tracking the human recorded paths
and performing the pick-and-place task. Figure 6: A near-optimal 5-DOF robotic
arm tracking the human recorded paths and performing a welding task.

##  V Conclusions

In this work, we have proposed a novel approach for designing the kinematics
of a robotic arm to track human demonstrated paths. The design considers the
capability of the future robot to accurately track the path of the entire
human arm. We have also proposed RA-PSO, a variant of PSO, to efficiently
search within the space of design parameters. RA-PSO is shown to be suitable
for robot optimization while reducing computational effort. The approach
provides a minimal design of a robot arm to do some specific task. The output
of the optimization provides also a path in the configuration space to perform
the task. The result can save valuable engineering resources in the design
phase. In addition, standard and modular hardware can be used to rapidly
assemble the outputted optimal kinematics. These may significantly shorten the
time-to-operation of the robot. Future work may use the proposed method to
optimize both a robotic arm and a computed path to track.

## References

  * [1] H. M. Do, C. Park, and J. H. Kyung, “Dual arm robot for packaging and assembling of it products,” in _IEEE International Conference on Automation Science and Engineering_ , 2012, pp. 1067–1070.
  * [2] M. Beschi, S. Mutti, G. Nicola, M. Faroni, P. Magnoni, E. Villagrossi, and N. Pedrocchi, “Optimal robot motion planning of redundant robots in machining and additive manufacturing applications,” _Electronics_ , vol. 8, p. 1437, Dec 2019.
  * [3] P. Chutima, “A comprehensive review of robotic assembly line balancing problem,” _J. of Intelligent Manuf._ , vol. 8, pp. 1572–8145, 2020.
  * [4] M. Cefalo, G. Oriolo, and M. Vendittelli, “Planning safe cyclic motions under repetitive task constraints,” in _IEEE International Conference on Robotics and Automation_ , 2013, pp. 3807–3812.
  * [5] V. Bloch, A. Degani, and A. Bechar, “A methodology of orchard architecture design for an optimal harvesting robot,” _Biosystems Engineering_ , vol. 166, pp. 126–137, 2018.
  * [6] M. Perrollaz, S. Khorbotly, A. Cool, J. Yoder, and E. Baumgartner, “Teachless teach-repeat: Toward vision-based programming of industrial robots,” in _IEEE Inter. Conf. on Rob. & Auto._, 2012, pp. 409–414.
  * [7] S. Schaal, “Learning from demonstration,” in _Advances in Neural Information Processing Systems_ , M. Mozer, M. Jordan, and T. Petsche, Eds., vol. 9. MIT Press, 1996.
  * [8] S. Patel and T. Sobh, “Task based synthesis of serial manipulators,” _Journal of Advanced Research_ , vol. 6, no. 3, pp. 479 – 492, 2015.
  * [9] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in _International Conference on Neural Networks_ , vol. 4, 1995, pp. 1942–1948.
  * [10] Shicai Shi, Xiaohui Gao, Zongwu Xie, Fenglei Ni, Hong Liu, E. Kraemer, G. Hirzinger, and S. C. Shi, “Development of reconfigurable space robot arm,” in _International Symposium on Systems and Control in Aerospace and Astronautics_ , 2006, pp. 6 pp.–143.
  * [11] R. Vijaykumar, K. Waldron, and M. Tsai, “Geometric optimization of serial chain manipulator structures for working volume and dexterity,” _Int. Journal of Robotics Research_ , vol. 5, no. 2, pp. 91–103, 1986.
  * [12] M. Stock and K. Miller, “Optimal kinematic design of spatial parallel manipulators: Application to linear delta robot,” _Journal of Mechanical Design_ , vol. 125, 06 2003.
  * [13] A. Zeiaee, R. Soltani-Zarrin, R. Langari, and R. Tafreshi, “Kinematic design optimization of an eight degree-of-freedom upper-limb exoskeleton,” _Robotica_ , vol. 37, no. 12, p. 2073–2086, 2019.
  * [14] R. M. Murray, Z. Li, and S. S. Sastry, _A Mathematical Introduction to Robotic Manipulation_ , 1st ed. CRC Press, Mar. 1994.
  * [15] W. S. You, Y. Lee, G. Kang, H. Oh, J. Seo, and H. Choi, “Kinematic design optimization for anthropomorphic robot hand based on interactivity of fingers,” _Intelligent Service Robotics_ , pp. 1–12, 04 2019.
  * [16] M. J. H. Lum, J. Rosen, M. N. Sinanan, and Hannaford B, “Kinematic optimization of a spherical mechanism for a minimally invasive surgical robot,” in _IEEE Int. Conf. on Rob. & Aut._, 2004, pp. 829–834.
  * [17] A. Kuntz, C. Bowen, C. Baykal, A. W. Mahoney, P. L. Anderson, F. Maldonado, R. J. Webster, and R. Alterovitz, “Kinematic design optimization of a parallel surgical robot to maximize anatomical visibility via motion planning,” in _IEEE International Conference on Robotics and Automation (ICRA)_ , 2018, pp. 926–933.
  * [18] J. Rastegar and B. Fardanesh, “Manipulation workspace analysis using the monte carlo method,” _Mechanism and Machine Theory_ , vol. 25, no. 2, pp. 233 – 239, 1990.
  * [19] N. Vahrenkamp, T. Asfour, G. Metta, G. Sandini, and R. Dillmann, “Manipulability analysis,” in _IEEE-RAS International Conference on Humanoid Robots_ , 2012, pp. 568–573.
  * [20] L. Stocco, S. E. Salcudean, and F. Sassani, “Fast constrained global minimax optimization of robot parameters,” _Robotica_ , vol. 16, no. 6, p. 595–605, 1998.
  * [21] C. Giladi and A. Sintov, “Manifold learning for efficient gravitational search algorithm,” _Information Sciences_ , vol. 517, pp. 18 – 36, 2020.
  * [22] S. Khatami and F. Sassani, “Isotropic design optimization of robotic manipulators using a genetic algorithm method,” in _IEEE Internatinal Symposium on Intelligent Control_ , 2002, pp. 562–567.
  * [23] J. Bryson, X. Jin, and S. Agrawal, “Optimal design of cable-driven manipulators using particle swarm optimization,” _ASME. J. Mechanisms Robotics_ , vol. 8, no. 4, 2016.
  * [24] Y. Zhu, Z. Wang, J. Merel, A. Rusu, T. Erez, S. Cabi, S. Tunyasuvunakool, J. Kramár, R. Hadsell, N. de Freitas, and N. Heess, “Reinforcement and imitation learning for diverse visuomotor skills,” in _Robotics: Science and Systems_ , 2018.
  * [25] K. Ayusawa and E. Yoshida, “Motion retargeting for humanoid robots based on simultaneous morphing parameter identification and motion optimization,” _IEEE Transactions on Robotics_ , vol. 33, no. 6, pp. 1343–1357, 2017.
  * [26] L. Smith, N. Dhawan, M. Zhang, P. Abbeel, and S. Levine, “Avid: Learning multi-stage tasks via pixel-level translation of human videos,” in _Robotics: Science and Systems (RSS)_ , 2020.
  * [27] N. García, J. Rosell, and R. Suárez, “Motion planning by demonstration with human-likeness evaluation for dual-arm robots,” _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ , vol. 49, no. 11, pp. 2298–2307, 2019\.
  * [28] D. Ceylan, W. Li, N. J. Mitra, M. Agrawala, and M. Pauly, “Designing and fabricating mechanical automata from mocap sequences,” _ACM Trans. Graph._ , vol. 32, no. 6, 2013.
  * [29] S. Coros, B. Thomaszewski, G. Noris, S. Sueda, M. Forberg, R. W. Sumner, W. Matusik, and B. Bickel, “Computational design of mechanical characters,” _ACM Trans. Graph._ , vol. 32, no. 4, 2013.
  * [30] A. Kapusta and C. C. Kemp, “Optimization of robot configurations for assistive tasks,” _Georgia Tech Library_ , 2016.
  * [31] A. Perez-Gracia and J. M. McCarthy, “Kinematic synthesis of spatial serial chains using clifford algebra exponentials,” _Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science_ , vol. 220, no. 7, pp. 953–968, 2006.
  * [32] S. Shirafuji and J. Ota, “Kinematic synthesis of a serial robotic manipulator by using generalized differential inverse kinematics,” _IEEE Transactions on Robotics_ , vol. 35, no. 4, pp. 1047–1054, 2019.
  * [33] D. R. Gustavo H. de Rosa and J. P. Papa, “Opytimizer: A nature-inspired python optimizer,” 2019.
  * [34] T. Dokeroglu, E. Sevinc, T. Kucukyilmaz, and A. Cosar, “A survey on new generation metaheuristic algorithms,” _Computers & Industrial Engineering_, vol. 137, p. 106040, 2019.
  * [35] S. M. H. Mousakazemi, “Computational effort comparison of genetic algorithm and particle swarm optimization algorithms for the proportional–integral–derivative controller tuning of a pressurized water nuclear reactor,” _Annals of Nuclear Energy_ , vol. 136, p. 107019, 2020\.