# Unified Attitude Determination Problem from Vector Observations and Hand-eye
Measurements

Jin Wu ${}^{\href https://orcid.org/0000-0001-5930-4170\,}$, The author is
with Department of Electronic and Computer Engineering, Hong Kong University
of Science and Technology, Hong Kong, China and is also with Tencent Robotics
X. (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The hand-eye measurements have recently been proven to be very efficient for
spacecraft attitude determination relative to an ellipsoidal asteroid.
However, recent method does not guarantee full attitude observability for all
conditions. This paper refines this problem by taking the vector observations
into account so that the accuracy and robustness of the spacecraft attitude
estimation can be improved. The vector observations come from many sources
including visual perspective geometry, optical navigation and point clouds
that frequently occur in aerospace electronic systems. Completely closed-form
solutions along with their uncertainty descriptions are presented for the
proposed problem. Experiments using our simulated dataset and real-world
spacecraft measurements from NASA dawn spacecraft verify the effectiveness and
superiority of the derived solution.

###### Index Terms:

Attitude Determination, Vector Observations, Hand-eye Measurements,
Uncertainty Description, Optical Navigation

##  I Introduction

###  I-A Background and Motivations

The research around planetary systems especially our solar system, has
revealed many amazing secrets of the outer space. There are eight major
planets and some other dwarf planets as well as asteroids in the solar system.
In 1977, two Voyager spacecrafts were launched by National Aeronautics and
Space Administration (NASA) to discover these celestial bodies [1]. Recently,
some robots are sent to collect geology data from some planets and their
satellites e.g. the Spirit launched in 2004 [2] and the Curiosity launched in
2011 by NASA for discovery of the Mars [3], the YuTu (Jade Rabbit) launched in
2013 by China National Space Administration (CNSA) for inspection of the Moon
[4]. A kernel problem behind these space projects is the precise navigation of
various spacecrafts. The stability of nowdays space navigation systems, mainly
constituted by strapdown installed measurement units, has been highly
dependent on the accuracy of the attitude determination sub-system.
Apart from the advances in inertial navigation technologies [5, 6, 7],
attitude determination from heterogeneous sensor sources is a crucial problem
in aerospace engineering since any spacecraft requires high-precision attitude
information for motion control. The optimal attitude determination using
vector observations from star trackers, sun sensors and magnetometers, namely
the Wahba’s problem, has been studied for over fifty years [8]. Many efficient
algorithms including the QUaternion ESTimator (QUEST, [9]), The Estimator of
Optimal Quaternion (ESOQ, [10]), The Optimal Linear Attitude Estimator (OLAE,
[11]) and etc. [12] are developed to achieve accurate and reliable estimation.
When the gyroscope measurements are taken into account, the vector-observation
attitude estimation can be significantly enhanced via nonlinear observers on
the special orthogonal group $SO(3)$ [13, 14]. The hand-eye measurements
formulate the attitude determination problem of the hand-eye type
$\bm{AX}=\bm{XB}$ where $\bm{A},\bm{B}$ are known and $\bm{X}$ is the unknown
transformation to be figured out. The terminology ’hand-eye’ was first
introduced into robotics in late 1980s for calibration i.e. extrinsic
transformation estimation between the robotic gripper and attached camera so
that visual motion reconstruction can be precisely mapped to the gripper frame
for accurate grasping [15, 16]. The idea of using the hand-eye measurements
has recently been discovered by Modenini, who proposed a new method using
images of ellipsoids. Modenini’s method has been verified to be efficient with
attitude accuracy of up to several arc seconds [17, 18]. Recently, an improved
version with dimensionless matrices has also been proposed [19]. Fig. 1 shows
such principle of imaging a space ellipsoidal object.

Figure 1: The projection of an ellipsoid onto the imaging plane where the
colors on the ellipsoid indicate some non-opportunistic features.

Figure 2: The shading conics of the simulated Ceres with multiple light
projection positions.

Modenini’s new algorithm has given us a brand-new tool for spacecraft attitude
determination relative to an ellipsoid. The problem studied in this paper is
an extension of Modenini’s contribution in which only one pair of symmetric
hand-eye measurements are employed for attitude determination. According to
the limitation of perspective geometry, any single shot of the celestial body
can not completely reflect the exact details of 3D reconstruction [20].
Therefore, the observability of attitude angles in all directions can not be
fully guaranteed which requires further verification with historical estimates
from other sources [17, 18]. To solve the problem, in this paper, instead, we
incorporate the vector and hand-eye measurements together aiming for precise
relative attitude determination between spacecraft and astroid. In Modenini’s
recent study, the hand-eye equation is established by means of the perspective
geometry when the camera captures the image of an ellipsoid. A common
knowledge exists in our mind is that there may be some distinctive (non-
opportunistic) features on the surface of the ellipsoid, indicating that
vector observations can form a unified attitude determination system together
with hand-eye measurements. Another shortcoming of the existing results in
[17, 18] is that the solving process depends on the spectrum decomposition
which is nonlinear and an analytical covariance analysis can hardly be
performed. The covariance information, however, guarantees the later fusion
with other sensors using a Kalman filter, which is a common practice for
reliability in aerospace engineering.

###  I-B Sources of Vector Observations

There are many approaches that can provide vector observations. According to
the principles of computer vision and robotics, the vector observations can be
obtained either using point clouds from 3D laser [21], terrian reconstruction
from the synthetic aperture radar (SAR) or rotation matrix from perspective-n-
point (PnP, [22]). Here we introduce three major ones.

####  I-B1 Perspective Geometry

In aerospace engineering, practitioners usually first estimate the camera
poses with respect to some celestial references. Then the camera poses can be
transformed into the spacecraft attitude through pre-calibrated base-camera
parameters. To achieve this, the perspective geometry is often invoked which
consists of two representative methods i.e. the perspective-n-point (PnP)
algorithm and epipolar geometry [23].
The PnP solves the pose estimation problem when we have some interesting
features in the 2D image and their corresponding perspective coordinates in
the 3D frame. To get the accurate coordinates in the 3D frame, the precise 3D
model or measurement of the imaged object is needed. The 2D-3D correspondences
can be determined according to current 2D-3D registration techniques. The
mathematical framework of the PnP problem allows for multiple solvers with
various precisions including the direct linear transform (DLT, [24]),
efficient PnP (EPnP, [25]), bundle adjustment (BA, [26]) and some recent
analytical solutions [27].

####  I-B2 Optical Navigation

The orbits of some celestial bodies can be acquired using historical celestial
observations. With such data, the planets in the solar system and their
relations with the Sun will be beneficial for spacecraft attitude
determination. For instance, using conic extraction from binarized images of
Moon, the Moon-Sun attitude sensor mechanism is established [28]. For
aerospace engineering, when the orbit of the astroid is known and the
luminance of the astroid is satisfactory for binarization, the vector
observations can also come from such Asteriod-Sun relationship. Fig. 2 depicts
the conic variations of the modeled Ceres under different Sun projections.
From another aspect, if the relative position between the Sun and the planet
is known according to existing orbitary observations, then the Sun sensor, as
a special kind of star trackers, can also provide observed Sun vector
observation [29].

####  I-B3 Point Clouds

There are some airborne instruments that can give direct or indirect point-
cloud measurements e.g. the laser scanner and SAR. By comparing the point
clouds with existing 3D terrain, the spacecraft attitude is computed. When
there is completely no knowledge of 3D terrain, the relative attitude can also
be propagated sequentially using results from the iterative closest points
(ICP, [30, 31]).

###  I-C Main Contributions and Arrangement of Contents

The main contributions of this paper is to study the feasibility of such
attitude determination scheme using vector observations and hand-eye
measurements simultaneously. We are aimed to solve the following problems:

  1. 1.

Derive the completely analytical solution to the proposed unified attitude
determination problem.

  2. 2.

Give an intuitive closed-form covariance analysis on the derived solution.

  3. 3.

Study the characteristics of the proposed solution subject to different types
of input values.

  4. 4.

Study the real-world performances of the proposed scheme according to
authentic spacecraft data.

The remainder of this paper is organized as follows: Section II contains
notation descriptions, problem formulation and proposed fully analytical
solution. Covariance analysis is also given in the Section II showing some
probabilistic characteristics of the proposed scheme. In Section III we
conduct several experiments to deduce the in-flight attitude determination
accuracy and robustness. Section IV consists of concluding remarks and future
works.

##  II Problem Formulation and Solution

###  II-A Mathematical Preliminaries

We inherit the main usages of notations presented in [32]. The $n$-dimensional
Euclidean vector space is described with ${\mathbb{R}^{n}}$. We use
$\mathbb{R}^{n\times m}$ to denote the real space containing all matrices with
row dimension of $n$ and column dimension of $m$. The identity matrix has the
notation of $\bm{I}$ and owns its proper size.
${{\bm{X}}^{\top}},{{\bm{X}}^{-1}}$ mean the transpose and inverse of a given
matrix $\bm{X}$ respectively in which the inverse exists when $\bm{X}$ is
squared and nonsingular. And the Moore-Penrose generalized inverse of a given
$\bm{X}$ with arbitrary structure is denoted as $\bm{X}^{+}$ which has the
property that $\left(\bm{X}^{\top}\right)^{+}=\left(\bm{X}^{+}\right)^{\top}$.
The notations ${\rm{tr}}(\bm{X})$ and ${\rm{diag}}(\cdots)$ denote the trace
of a squared matrix $\bm{X}$ and a diagonal matrix formed by diagonal elements
of $\cdots$, respectively. The $n$-dimensional special orthogonal group
$SO(n)$ contains all $n$-dimensional rotation matrices and is expressed with
${\bm{X}}\in SO(n)\Rightarrow\bm{X}\in\mathbb{R}^{n\times
n},{\bm{X}}{{\bm{X}}^{\top}}={{\bm{X}}^{\top}}{\bm{X}}={\bm{I}},\det({\bm{X}})=+1$.
The covariance of a given vector $\bm{x}$ perturbed by noises is denoted as
$\bm{\Sigma}_{\bm{x}}$. The vectorization of a matrix
$\bm{X}=\left(\bm{x}_{1},\bm{x}_{2},\cdots,\bm{x}_{n}\right)$ of column size
$n$ is defined by
${\rm{vec}}(\bm{X})=\left(\bm{x}_{1}^{\top},\bm{x}_{2}^{\top},\cdots,\bm{x}_{n}^{\top}\right)^{\top}$
while the inverse operator ${\rm{mat}}[{\rm{vec}}(\bm{X})]$ restores the
vectorization into its original matrix form of $\bm{X}$. The Kronecker product
of two arbitrary matrices $\bm{X}$ and $\bm{Y}$ is denoted as
$\bm{X}\otimes\bm{Y}$ and commonly
$\left(\bm{X}\otimes\bm{Y}\right)^{\top}=\bm{X}^{\top}\otimes\bm{Y}^{\top}$.
The convenience of the Kronecker product is that, given matrices
$\bm{A},\bm{B},\bm{C},\bm{X}$ with proper sizes, one can write the equality
$\bm{AXB}=\bm{C}$ into
$\left(\bm{B}^{\top}\otimes\bm{A}\right){\rm{vec}}(\bm{X})={\rm{vec}}(\bm{C})$.
Also, one may derive
$\left(\bm{A}\otimes\bm{B}\right)\left(\bm{C}\otimes\bm{X}\right)=\left(\bm{AC}\otimes\bm{BX}\right)$.
For probabilistic descriptions, we use $\left\langle\cdots\right\rangle$ to
represent the operation of expectation and $\bm{\Sigma}_{\bm{X}}$ or
$\bm{\Sigma}_{\bm{XX}}$ is the auto-covariance of a random variable $\bm{X}$.
The covariance matrix between two random variables $\bm{X}$ and $\bm{Y}$ are
denoted as $\bm{\Sigma}_{\bm{X},\bm{Y}}$. The notation
$\mathcal{N}(\bm{\alpha},\bm{\Sigma})$ denotes the normal (Gaussian)
distribution with mean of $\bm{\alpha}$ and covariance of $\bm{\Sigma}$ while
$\mathcal{MN}$ represents the normal distribution for matrix. The
probabilistic density function of a given squared matrix
$\bm{X}\sim{\mathcal{MN}}\left(\bm{M},\bm{Y},\bm{W}\right)$ with dimension $n$
is

$\begin{gathered}p\left(\bm{X}|\bm{M},\bm{Y},\bm{W}\right)\hfill\\\
=\frac{\exp\left\\{-\frac{1}{2}{\rm{tr}}\left[\bm{W}^{-1}(\bm{X}-\bm{M})^{\top}\bm{Y}^{-1}(\bm{X}-\bm{M})\right]\right\\}}{(2\pi)^{\frac{n^{2}}{2}}\left[\det(\bm{Y})\det(\bm{W})\right]^{\frac{n}{2}}}\hfill\\\
\end{gathered}$ (1)

where $\bm{M}$ is the mean of $\bm{X}$ and $\bm{Y},\bm{W}$ represent the
second moments of $\bm{X}$ satisfying

$\begin{gathered}\left<(\bm{X}-\bm{M})(\bm{X}-\bm{M})^{\top}\right>=\bm{Y}\
{\rm{tr}}\ \bm{W}\hfill\\\
\left<(\bm{X}-\bm{M})^{\top}(\bm{X}-\bm{M})\right>=\bm{W}\ {\rm{tr}}\
\bm{Y}\hfill\\\ \end{gathered}$ (2)

The vectorization of $\bm{X}$ will then be subjected to the normal
distribution such that

${\rm{vec}}(\bm{X})\sim{\mathcal{N}}\left[{\rm{vec}}(\bm{M}),\bm{Y}\otimes\bm{W}\right]$
(3)

Figure 3: Formation-aided relative attitude determination between satellites.

###  II-B The Unified Attitude Determination Problem

We consider that there are $N$ pairs of $n$-dimensional vector observations
and $M$ pairs of $n$-dimensional hand-eye measurements available for attitude
determination, all related by the $n$-dimensional rotation matrix $\bm{R}\in
SO(n)$:

${\rm{Vector:}}\
\left\\{{\begin{array}[]{*{20}{c}}{{{\bm{b}}_{1}}={\bm{R}}{{\bm{r}}_{1}}}\\\
{{{\bm{b}}_{2}}={\bm{R}}{{\bm{r}}_{2}}}\\\ \vdots\\\
{{{\bm{b}}_{{N}}}={\bm{R}}{{\bm{r}}_{{N}}}}\end{array}}\right.,{\rm{Hand-
Eye:}}\
\left\\{{\begin{array}[]{*{20}{c}}{{{\bm{A}}_{1}}{\bm{R}}={\bm{R}}{{\bm{B}}_{1}}}\\\
{{{\bm{A}}_{2}}{\bm{R}}={\bm{R}}{{\bm{B}}_{2}}}\\\ \vdots\\\
{{{\bm{A}}_{{M}}}{\bm{R}}={\bm{R}}{{\bm{B}}_{{M}}}}\end{array}}\right.$ (4)

where there is no constraint for
$\bm{b}_{i}=\left(b_{i,1},b_{i,2},\cdots,b_{i,n}\right)^{T},\bm{r}_{i}=\left(r_{i,1},r_{i,2},\cdots,r_{i,n}\right)^{T},i=1,2,\cdots,N$
and $\bm{A}_{i},\bm{B}_{i},i=1,2,\cdots,M$. That is to say the vector
observations do not have to maintain unit norms as presented in the Wahba’s
problem and hand-eye measurements are also arbitrary and do not have to be
symmetric or orthonormal matrices as presented in [17] and [15]. Fig. 3 shows
a potential example that incorporates the rigid case for $\bm{A}_{i}$ and
$\bm{B}_{i}$. There are four satellites establishing a formation where the
satellite $\alpha$ is conducting attitude synchronization with satellite
$\beta$. The aim is to compute the relative attitude $\bm{R}_{\alpha}^{\beta}$
which have already been given by the relative projective geometry using camera
with pre-built 3-D models of $\alpha$ and $\beta$. Meanwhile, two passing-
through satellites $\gamma$ and $\xi$ also observe the $\alpha$ and $\beta$
respectively. By using the projective geometry likewise, the relative attitude
matrices at time instants $k-1$ and $k$ are estimated. Then using the
differential rotation and hand-eye relationship, with the control efforts that
$\alpha$ and $\beta$ are attitude-synchronized, we are able to construct

$\begin{gathered}\bm{A}_{i}=\bm{R}_{\gamma,k}^{\alpha}\left(\bm{R}_{\gamma,k-1}^{\alpha}\right)^{\top}\hfill\\\
\bm{B}_{i}=\left(\bm{R}_{\xi,k}^{\beta}\right)^{\top}\bm{R}_{\xi,k-1}^{\beta}\hfill\\\
\end{gathered}$ (5)

which denotes the rigid case and will give hand-eye measurements to the
original results.
Here, the vector observations and hand-eye measurements are assumed to be
synchronized that can be satisfied for common data sampling and wireless
transmission sub-blocks of the aerospace electronic central system. To derive
a general $n$-dimensional closed-form solution to this problem, unlike
existing methods used in [17], [32] and [33], we use vectorization of rotation
matrix for mathematical representation, which has been proven to be effective
for hand-eye calibration [34, 35]. For equations of vector observations, we
can write them into the compact form

$\bm{P}=\bm{RQ}$ (6)

where

$\begin{gathered}\bm{P}=\left(\bm{b}_{1},\bm{b}_{2},\cdots,\bm{b}_{N}\right)\hfill\\\
\bm{Q}=\left(\bm{r}_{1},\bm{r}_{2},\cdots,\bm{r}_{N}\right)\hfill\\\
\end{gathered}$ (7)

which can be further vectorized as

${\rm{vec}}(\bm{P})=\left(\bm{Q}^{\top}\otimes\bm{I}\right){\rm{vec}}(\bm{R})$
(8)

An analytical solution to the hand-eye calibration problem [34] solves the
$\bm{R}$ in the hand-eye part of (4) by the following homogeneous system of
${\rm{vec}}(\bm{R})$

$\left(\begin{gathered}\bm{I}-\bm{A}_{1}\otimes\bm{B}_{1}\\\
\bm{I}-\bm{A}_{2}\otimes\bm{B}_{2}\\\ \cdots\\\
\bm{I}-\bm{A}_{M}\otimes\bm{B}_{M}\\\
\end{gathered}\right){\rm{vec}}(\bm{R})=\bm{0}$ (9)

This result has been proven to be effective but in fact there is a strong
coupling of $\bm{A}_{i},\bm{B}_{i}$ for $i=1,2,\cdots,M$ which sets obstacle
for a further covariance analysis. Therefore, we use another intuitive
solution such that

$\left(\begin{gathered}\bm{A}_{1}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{1}^{\top}\\\
\bm{A}_{2}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{2}^{\top}\\\ \cdots\\\
\bm{A}_{M}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{M}^{\top}\\\
\end{gathered}\right){\rm{vec}}(\bm{R})=\bm{0}$ (10)

The unified attitude determination result from vector observations and hand-
eye measurements then can be given by the following problem

$\begin{gathered}\mathop{\arg\min}\limits_{\bm{x}}\
{\mathcal{L}}=\bm{x}^{\top}\bm{H}\bm{x}+\hfill\\\ \ \ \ \ \ \ \ \ \ \ \ \
\left[\left(\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}-{\rm{vec}}(\bm{P})\right]^{\top}\left[\left(\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}-{\rm{vec}}(\bm{P})\right]\hfill\\\
\end{gathered}$ (11)

in which $\bm{x}={\rm{vec}}(\bm{R})$ and

$\bm{H}=\sum\limits_{i=1}^{M}\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)^{\top}\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)$
(12)

It should be noted that if the weights of the measurements are taken into
account, we should use the following matrices instead:

$\begin{gathered}\bm{P}=\left(\sqrt{w_{1}}\bm{b}_{1},\sqrt{w_{2}}\bm{b}_{2},\cdots,\sqrt{w_{N}}\bm{b}_{N}\right)\hfill\\\
\bm{Q}=\left(\sqrt{w_{1}}\bm{r}_{1},\sqrt{w_{2}}\bm{r}_{2},\cdots,\sqrt{w_{N}}\bm{r}_{N}\right)\hfill\\\
\bm{H}=\sum\limits_{i=1}^{M}v_{i}\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)^{\top}\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)\hfill\\\
\end{gathered}$ (13)

with $w_{1},w_{2},\cdots,w_{N}$ the positive weights of $N$ vector
measurements and $v_{1},v_{2},\cdots,v_{M}$ the positive weights of $M$ hand-
eye measurements. The ratio
$\varrho=\left(\sum\limits_{i=1}^{N}w_{i}\right)/\left(\sum\limits_{i=1}^{M}v_{i}\right)$
describes the relative accuracy between the vector and hand-eye measurements.
The objective function $\mathcal{L}$ can be evaluated as

$\begin{gathered}\
{\mathcal{L}}=\bm{x}^{\top}\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]\bm{x}-\hfil\\\
\ \ \ \ \ \ \ \ \ \ \
\bm{x}^{\top}\left(\bm{Q}\otimes\bm{I}\right)^{\top}{\rm{vec}}(\bm{P})-\hfill\\\
\ \ \ \ \ \ \ \ \ \ \
{\rm{vec}}(\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}+{\rm{vec}}(\bm{P})^{\top}{\rm{vec}}(\bm{P})\hfill\\\
\end{gathered}$ (14)

The optimal $\bm{x}$ occurs at

$\begin{gathered}\frac{\partial{\mathcal{L}}}{\partial\bm{x}}=2\bm{x}^{\top}\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]-\hfill\\\
\ \ \ \ \ \ \ \ \
2{\rm{vec}}(\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\ \ \
\ \ \ =\bm{0}\hfill\\\ \end{gathered}$ (15)

which generates

$\bm{x}=\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})$
(16)

The obtained result indicates that the general observability of attitude
angles are governed by the rank of
$\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]$.
When there is no vector observation, (16) can not hold since
$\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})$ becomes null. However,
when there is no hand-eye measurement, (16) could also make sense which
depends on the number of vector observations and the collinearity between
vector pairs [36, 37, 38]. That is so say, in this way the following equation
is also a Wahba’s solution

$\bm{x}=\left[\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})$
(17)

For instance, with only two vector observation pairs, we may have already
compute the attitude [39]. When there are one vector observation pair and one
set of hand-eye measurements, there could be sufficient information for a
full-attitude determination since the single system $\bm{AR}=\bm{RB}$ has been
proven to own two ambiguous solutions [17] and the optimal can be then
selected by integrating an external vector pair.
When there are only hand-eye measurements, the optimization (11) will be

$\mathop{\arg\min}\limits_{\bm{x}}\ {\mathcal{L}}=\bm{x}^{\top}\bm{H}\bm{x}$
(18)

indicating that $\bm{x}$ is the eigenvector of $\bm{H}$ associated with its
minimum eigenvalue. As $\bm{x}$ and $-\bm{x}$ are all such eigenvectors, it is
able to verify the reconstructed $\bm{R}={\rm{mat}}(\bm{x})$ via loss function
$\mathcal{L}$.
The obtained attitude reconstruction, since may be put into a further Kalman
filter for more accurate state estimation and gyro bias cancellation, should
provide its uncertainty descriptions [40, 41]. The detailed derivations are
presented in the next sub-section.

###  II-C Covariance Analysis

In this sub-section, the covariance analysis of the derived solution (16) will
be performed. Note that when there are only hand-eye measurements, the problem
degenerates to (18). In such case, the uncertainty descriptions of $\bm{x}$
are available via [42]. From (15), one easily observes that the solution of
$\bm{x}$ is linear. Thus a perturbed model of $\bm{x}$ can be obtained by

$\small\begin{gathered}\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]\delta\bm{x}+\delta\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]\bm{x}\hfill\\\
\ \ \ \ \
=\delta\left[\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})\right]\hfill\\\
\Rightarrow\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]\delta\bm{x}+\hfill\\\
\left[\begin{gathered}\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\\\
\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\end{gathered}\right]\bm{x}\\\
\ \ \ \ \
=\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\hfill\\\
\end{gathered}$ (19)

where the second-order differential terms are ommited. The covariance of
$\bm{x}$ can be computed via

$\small\begin{gathered}\bm{\Sigma}_{\bm{x}\bm{x}}=\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\times\hfill\\\
\ \ \ \ \
\left\langle\begin{gathered}\left\\{\begin{gathered}\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]-\hfill\\\
\left[\begin{gathered}\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\end{gathered}\right]\bm{x}\hfill\\\ \end{gathered}\right\\}\\\
\left\\{\begin{gathered}\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]-\hfill\\\
\left[\begin{gathered}\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\end{gathered}\right]\bm{x}\hfill\\\
\end{gathered}\right\\}^{\top}\end{gathered}\right\rangle\hfill\\\ \ \ \ \ \
\times\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\hfill\\\
\end{gathered}$ (20)

As the operation of the type
$\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})$ is linear both in
elements of $\bm{Q}$ and $\bm{P}$, one can write the matrix manipulations into

$\begin{gathered}\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})={\mathcal{Z}}(\bm{P}){\rm{vec}}(\delta\bm{Q}^{\top})\end{gathered}$
(21)

where ${\mathcal{Z}}(\bm{P})$ a linearly mapping. Note that

$\begin{gathered}{\rm{mat}}\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})\right]=\bm{P}\delta\bm{Q}^{\top}\hfill\\\
\Rightarrow{\rm{vec}}\left(\bm{P}\delta\bm{Q}^{\top}\right)=\left(\bm{I}\otimes\bm{P}\right){\rm{vec}}\left(\delta\bm{Q}^{\top}\right)\end{gathered}$
(22)

then we have

${\mathcal{Z}}\left(\bm{P}\right)=\left(\bm{I}\otimes\bm{P}\right)$ (23)

In this way, one has

Figure 4: Case 1: The estimated results using symmetric
$\bm{A}_{i},\bm{B}_{i}$ and their $3\sigma$ bounds when
$\epsilon_{\rm{vector}}=1\times 10^{-1}$, $\epsilon_{\rm{hand-eye}}=1\times
10^{-5}$, $N=30$ and $M=1$.

Figure 5: Case 1: The estimated results using symmetric
$\bm{A}_{i},\bm{B}_{i}$ and their $3\sigma$ bounds when
$\epsilon_{\rm{vector}}=1\times 10^{-1}$, $\epsilon_{\rm{hand-eye}}=1\times
10^{-5}$, $N=150$ and $M=1$.

$\begin{gathered}\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}\hfill\\\
={\mathcal{Z}}\left\\{{\rm{mat}}\left[\left(\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}\right]\right\\}]{\rm{vec}}(\delta\bm{Q}^{\top})\hfill\\\
={\mathcal{Z}}\left(\bm{\tilde{R}Q}\right){\rm{vec}}(\delta\bm{Q}^{\top})\hfill\\\
\end{gathered}$ (24)

where $\bm{\tilde{R}}={\rm{mat}}\left[{\rm{vec}}(\bm{x})\right]$ is the
reconstructed rotation matrix and is not strictly on the $SO(n)$. In the same
manner, we can also obtain

$\begin{gathered}\delta\bm{Hx}\hfill\\\
=\sum\limits_{i=1}^{M}v_{i}\left[\begin{gathered}\left(\delta\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\delta\bm{B}_{i}^{\top}\right)^{\top}\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)+\\\
\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)^{\top}\left(\delta\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\delta\bm{B}_{i}^{\top}\right)\end{gathered}\right]\bm{x}\hfill\\\
=\sum\limits_{i=1}^{M}v_{i}\left\\{\begin{gathered}{\mathcal{F}}\left[\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)\bm{x}\right]\left[\begin{gathered}{\rm{vec}}(\delta\bm{A}_{i})\\\
{\rm{vec}}(\delta\bm{B}_{i})\\\ \end{gathered}\right]+\\\
\left(\bm{A}_{i}^{\top}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}\right){\mathcal{F}}(\bm{x})\left[\begin{gathered}{\rm{vec}}(\delta\bm{A}_{i}^{\top})\\\
{\rm{vec}}(\delta\bm{B}_{i}^{\top})\\\
\end{gathered}\right]\end{gathered}\right\\}\hfill\\\ \end{gathered}$ (25)

where the function $\mathcal{F}(\bm{x})$ is a linear mapping which can be
derived as follows

$\begin{gathered}\left(\delta\bm{A}\otimes\bm{I}-\bm{I}\otimes\delta\bm{B}^{\top}\right)^{\top}{\rm{vec}}({\bm{C}})\hfill\\\
=\left(\delta\bm{A}^{\top}\otimes\bm{I}-\bm{I}\otimes\delta\bm{B}\right){\rm{vec}}({\bm{C}})\hfill\\\
=\left(\delta\bm{A}^{\top}\otimes\bm{I}\right){\rm{vec}}\left(\bm{C}\right)-\left(\bm{I}\otimes\delta\bm{B}\right){\rm{vec}}({\bm{C}})\hfill\\\
={\rm{vec}}\left(\bm{C}\delta\bm{A}\right)-{\rm{vec}}\left(\delta\bm{B}\bm{C}\right)\hfill\\\
=\left(\bm{I}\otimes\bm{C}\right){\rm{vec}}(\delta\bm{A})-\left(\bm{C}^{\top}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{B})\hfill\\\
=\left(\begin{gathered}\bm{I}\otimes\bm{C},-\bm{C}^{\top}\otimes\bm{I}\end{gathered}\right)\left[\begin{gathered}{\rm{vec}}(\delta\bm{A})\\\
{\rm{vec}}(\delta\bm{B})\\\ \end{gathered}\right]\hfill\\\
\Rightarrow{\mathcal{F}}\left[{\rm{vec}}(\bm{C})\right]=\left(\begin{gathered}\bm{I}\otimes\bm{C},-\bm{C}^{\top}\otimes\bm{I}\end{gathered}\right)\hfill\\\
\end{gathered}$ (26)

Figure 6: Case 2: The estimated results using rigid $\bm{A}_{i},\bm{B}_{i}$
and their $3\sigma$ bounds when $\epsilon_{\rm{vector}}=1\times 10^{-1}$,
$\epsilon_{\rm{hand-eye}}=1\times 10^{-5}$, $N=30$ and $M=1$.

Figure 7: Case 2: The estimated results using rigid $\bm{A}_{i},\bm{B}_{i}$
and their $3\sigma$ bounds when $\epsilon_{\rm{vector}}=1\times 10^{-1}$,
$\epsilon_{\rm{hand-eye}}=1\times 10^{-5}$, $N=150$ and $M=1$.

Thus, the simplified form of $\bm{\Sigma}_{\bm{x}\bm{x}}$ is presented by

$\begin{gathered}\bm{\Sigma}_{\bm{xx}}=\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\times\hfill\\\
\ \ \ \ \ \ \ \ \ \
\left(\bm{S}_{1}+\bm{S}_{2}+\bm{S}_{3}+\bm{S}_{3}^{\top}\right)\left[\bm{H}+\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{+}\hfill\\\
\end{gathered}$ (27)

in which the detailed derivations along with
$\bm{S}_{1},\bm{S}_{2},\bm{S}_{3}$ are in the appendix. Here, we need to note
that $\bm{x}={\rm{vec}}(\bm{R})$ is the vectorization of $\bm{R}$ and to get a
proper $\bm{R}$ in $SO(n)$, one should orthonormalize
${\rm{mat}}\left(\bm{x}\right)$ from (16). Typically, such normalization is
achieved by

$\bm{R}=\bm{U}{\rm{diag}}\left[1,1,\cdots,\det(\bm{UV})\right]\bm{V}^{\top}$
(28)

where $\bm{USV}^{\top}={\rm{mat}}\left(\bm{x}\right)$ is the singular value
decomposition (SVD) of ${\rm{mat}}\left(\bm{x}\right)$. When $\bm{R}\in
SO(3)$, the orthonormalization can also be conducted by solutions to Wahba’s
problem [43] that involves an eigenvalue problem of $4\times 4$ matrices. As
SVD is highly nonlinear, we introduce a recent linear method for generalized
$n$-dimensional registration [44] with uncertainty descriptions so that the
following system can be established

$\left\\{{\begin{array}[]{*{20}{c}}{{{\bm{d}}_{1}}={\bm{R}}{{\bm{e}}_{1}}}\\\
{{{\bm{d}}_{2}}={\bm{R}}{{\bm{e}}_{2}}}\\\ \vdots\\\
{{{\bm{d}}_{{n}}}={\bm{R}}{{\bm{e}}_{{n}}}}\end{array}}\right.$ (29)

where
$\bm{x}={\rm{vec}}(\bm{R})=\left(\bm{d}_{1}^{\top},\bm{d}_{2}^{\top},\cdots,\bm{d}_{n}^{\top}\right)^{\top}$
and $\bm{e}_{1}=\left(1,0,0,\cdots,0\right)^{\top}$,
$\bm{e}_{2}=\left(0,1,0,\cdots,0\right)^{\top}$, $\cdots$,
$\bm{e}_{n}=\left(0,\cdots,0,1\right)^{\top}$ are standard orthogonal unit
bases of Euclidean space $\mathbb{R}^{n}$. In such model, the vectors
$\bm{e}_{1},\bm{e}_{2},\cdots,\bm{e}_{n}$ are noise-free and the covariance of
$\bm{d}_{1},\bm{d}_{2},\cdots,\bm{d}_{n}$ are invoked directly from
$\bm{\Sigma}_{\bm{xx}}$, so that the covariance $\bm{\Sigma}_{\bm{R}}$ is
obtained. (29) frames a standard form of point-cloud registration, and
$\bm{R}$ is its optimal solution on $SO(n)$. First, we need to reconstruct the
matrix

$\bm{G}=\sum\limits_{i=1}^{n}\frac{1}{n}{\mathcal{P}}^{\top}(\bm{\tau}_{i}){\mathcal{P}}(\bm{\tau}_{i})$
(30)

where $\bm{\tau}_{i}=\bm{d}_{i}+\bm{e}_{i}$. Then reconstruct the vector

$\bm{v}=\sum\limits_{i=1}^{n}\frac{1}{n}{\mathcal{P}}^{\top}(\bm{\tau}_{i})\bm{\varrho}_{i}$
(31)

where $\bm{\varrho}=\bm{e}_{i}-\bm{d}_{i}$. $\bm{R}$ satisfies the following
Caylay transform

$\bm{R}=\left(\bm{I}+\bm{g}_{\odot}\right)^{-1}\left(\bm{I}-\bm{g}_{\odot}\right)$
(32)

where $\bm{g}_{\odot}$ gives a skew-symmetric matrix by a linear mapping from
${\bm{g}}={\left[{{g_{1}},{g_{2}},\cdots,{g_{\frac{{n(n-1)}}{2}}}}\right]^{\top}}$,
such that

$\begin{gathered}\bm{g}_{\odot}=\hfill\\\
\left({\begin{array}[]{*{20}{c}}0&{{g_{1}}}&{{g_{2}}}&\cdots&{{g_{n-1}}}\\\
{-{g_{1}}}&0&{{g_{n}}}&\cdots&{{g_{2n-3}}}\\\
{-{g_{2}}}&{{g_{n}}}&\ddots&\cdots&\vdots\\\
\vdots&\vdots&\vdots&0&{{g_{\frac{{n\left({n-1}\right)}}{2}}}}\\\
{-{g_{n-1}}}&{-{g_{2n-3}}}&\cdots&{-{g_{\frac{{n\left({n-1}\right)}}{2}}}}&0\end{array}}\right)\hfill\\\
\end{gathered}$ (33)

The matrix $\mathcal{P}$ meets the following transform

$\bm{g}_{\odot}\bm{\tau}_{i}={\mathcal{P}}(\bm{\tau}_{i})\bm{g}$ (34)

and can be evaluated via various symbolic engines like MATLAB and Mathematica.
Then the covariance of $\bm{g}$ i.e. $\bm{\Sigma}_{\bm{g}}$ can be given by
(43) in [44]. After that, the rotation uncertainty can be characterized by
means of

$\bm{\Sigma}_{\bm{R}}=(\bm{I}+\bm{g}_{\odot})^{-1}\left[{\sum\limits_{i=1}^{n}{{\mathcal{P}}\left({{{\bm{\zeta}}_{i}}}\right){{\bm{\Sigma}}_{\bm{g}}}{{\mathcal{P}}^{\top}}\left({{{\bm{\zeta}}_{i}}}\right)}}\right]{\left[{{{\left({{\bm{I}}+\bm{g}_{\odot}}\right)}^{-1}}}\right]^{\top}}$
(35)

in which

${\bm{R}}+{\bm{I}}=\left({{{\bm{\zeta}}_{1}},{{\bm{\zeta}}_{2}},\cdots,{{\bm{\zeta}}_{n}}}\right)$
(36)

##  III Experimental Results

###  III-A Simulation Results

In this sub-section, we simulate several cases to demonstrate the performances
of the proposed algorithm. The rotation error is defined as

$\eta={\rm{arc}}\cos\frac{{\rm{tr}}(\bm{R}\bm{R}_{{\rm{true}}}^{\top})-1}{2}$
(37)

with $\bm{R}_{{\rm{true}}}$ the reference (true) rotation matrix and
$\epsilon_{\rm{vector}},\epsilon_{\rm{hand-eye}}$ denoting the noise densities
of the vector observations and hand-eye measurements respectively. The vector
observations and hand-eye measurements are simulated using additive noises
such that

$\begin{gathered}\bm{b}_{i}=\bm{R}_{{\rm{true}}}\bm{r}_{i}+\bm{\varepsilon}_{i}\hfill\\\
\bm{A}_{i}\bm{R}_{{\rm{true}}}-\bm{R}_{{\rm{true}}}\bm{B}_{i}=\bm{\Xi}_{i}\hfill\\\
\end{gathered}$ (38)

with $\bm{\varepsilon}_{i}$ and $\bm{\Xi}_{i}$ the noises subject to Gaussian
distribution such that

$\begin{gathered}\bm{\varepsilon}_{i}\sim{\mathcal{N}}\left(\bm{0},\epsilon_{\rm{vector}}\bm{I}\right)\hfill\\\
\bm{\Xi}_{i}\sim{\mathcal{MN}}\left(\bm{0},\epsilon_{\rm{hand-
eye}}\bm{I},\epsilon_{\rm{hand-eye}}\bm{I}\right)\hfill\\\ \end{gathered}$
(39)

The sequence of the true rotation matices is generated using a modified
quaternion model in [45]:

$\begin{gathered}{\bm{q}}=\left[{\begin{array}[]{*{20}{c}}{\sin\left({-0.8334\times
2\times{10^{-3}}k+{\rm{1.3679}}}\right)}\\\ {\sin\left({-1.5833\times
2\times{10^{-3}}k-0.1479}\right)}\\\ {\sin\left({3.0038\times
2\times{10^{-3}}k+2.0061}\right)}\\\ {\sin\left({-1.1200\times
2\times{10^{-3}}k-0.0179}\right)}\end{array}}\right],{\bm{q}}=\frac{\bm{q}}{\sqrt{\bm{q}^{\top}\bm{q}}}\end{gathered}$
(40)

where $k=1,2,\cdots,10000$ denote the indices. We validate our algorithm by
converting the quaternions into rotation matrices. The vector observations are
assumed to have been obtained by rotation matching from feature extraction
algorithms. Since the feature extraction may be opportunistic, the single-
point covariance parameter of the vector observations is set to
$\epsilon_{\rm{vector}}=1\times 10^{-1}$. While in [18], it is reported that
the hand-eye measurements can reach the attitude determination accuracy up to
several arc seconds, $\epsilon_{\rm{hand-eye}}$ is set to $1\times 10^{-5}$
i.e. this hand-eye source is regarded much more accurate than the feature-
matching based approach. It is assumed that here only one asteroid is in the
field-of-view (FOV) of the camera so $M=1$. For the first set of cases, we
simulate the symmetric $\bm{A}_{i},\bm{B}_{i}$ as appeared in [17, 18]. Two
cases with different vector observation numbers $N=30$ and $N=150$ are
simulated. The results along with the $3\sigma$ bounds calculated from the
covariance matrices are shown in Fig. 5 and 5. For another set of cases,
$\bm{A}_{i},\bm{B}_{i}$ are rigid ones like those in the hand-eye calibration
while the results are presented in Fig. 7 and 7.
From the two sets of cases, one can observe that with fewer vector
observations, the attitude estimator has worse accuracy and the uncertainty of
the $3\sigma$ bounds will be larger. These $3\sigma$ bounds are taken from the
square roots of the diagonal elements of the covariance matrix. Comparing the
first and second sets of cases, we can also see that the system with symmetric
$\bm{A}_{i},\bm{B}_{i}$ owns worse accuracy than those with rigid hand-eye
measurements. To study the relationship between the rotation error and numbers
of vector observations and hand-eye measurements, we choose $N$ and $M$ from 1
to 5 for simulation. In each step, the vector observations and hand-eye
measurements are randomly sampled for 5000 times and the results are averaged
so the statistical trends can be well reflected. The noises are chosen to be
subjected to

$\begin{gathered}\bm{\varepsilon}_{i}\sim{\mathcal{N}}\left(\bm{0},5\times
10^{-1}\bm{I}\right)\hfill\\\
\bm{\Xi}_{i}\sim{\mathcal{MN}}\left(\bm{0},5\times 10^{-1}\bm{I},5\times
10^{-1}\bm{I}\right)\hfill\\\ \end{gathered}$ (41)

i.e. both types of measurements are very noisy so the error scales can be
magnified. We also study two cases where $\bm{A}_{i},\bm{B}_{i}$ are rigid or
symmetric and the relationships are then shown in Fig. 8 and 9.

Figure 8: The rotation errors subject to different values of $N$ and $M$ with
$\bm{A}_{i},\bm{B}_{i}\in SO(n),n=3$ (rigid case). Figure 9: The rotation
errors subject to different values of $N$ and $M$ with $\bm{A}_{i},\bm{B}_{i}$
being $3\times 3$ symmetric matrices.

Figure 10: The matched features in successive images of Ceres from the Dawn
spacecraft using BRISK.

Figure 11: The matched features in successive images of Ceres from the Dawn
spacecraft using SURF.

Figure 12: The matched features in successive images of Ceres from the Dawn
spacecraft using ORB.

With growing $N$ and $M$, the rotation errors rapidly decrease while the
decreasing ratio for the hand-eye measurements is lower than that of vector
observations. This shows that the vector observations are effective in aiding
the attitude determination with only hand-eye measurements. Also, one can
obviously see that using symmetric $\bm{A}_{i}$ and $\bm{B}_{i}$, the accuracy
is much lower than that in rigid ones. As in [17, 18] $\bm{A}_{i},\bm{B}_{i}$
are symmetric, that is to say the method in Modenini’s work can only achieve
limited accuracy, regardless of its inevitable drawback in the angle
observability. When combined with vector observations, the accuracy,
robustness and observability can be dramatically improved. In the next sub-
section, we are going to conduct real-world experiments using data from the
Dawn spacecraft.

Figure 13: The reconstructed terrain and altitude data using historical
observations from the Dawn spacecraft. The darker color indicates lower
altitude and vice versa.

###  III-B Dawn Spacecraft Validation

The Dawn spacecraft was launch by NASA on 27 Sept 2007 and has been aimed to
discover the two dwarf planets i.e. Vesta and Ceres in the Kuiper belt. The
Dawn mission ends on 1st Nov 2018 with the spacecraft consumed all the fuel
for attitude control of its solar panel. The memorable Dawn mission had
fullfiled the dreams of the scientists and enthusiasts for inspection of
distant dwarf planets in the solar system. On the Dawn spacecraft, there were
two framing cameras imaging the targets. Ceres has better geometric shape than
Vesta since it is highly ellipsoidal and owns the semi-major axis of 482 km
and semi-minor axis of 446 km. Therefore, in the study of Modenini [17, 18],
it has been successfully proven that such ellipsoid imaged in the camera frame
can be used for accurate spacecraft attitude determination.
As described before, the visual measurements can also provide attitude
information from another aspect. Therefore, we verify the two sources of them
i.e. the PnP and epipolar geometry in this sub-section. There are many
representative point feature extraction methods proposed previously and each
algorithm has its pros and cons. For the motion estimation using Ceres images,
we choose the speed up robust features (SURF, [46]), binary robust invariant
scalable keypoints (BRISK, [47]), oriented FAST and rotated BRIEF (ORB, [48])
for comparisons. The conic detection and the elliptic fittings are conducted
according to [49, 50, 51].
Using the epipolar geometry, the camera poses can be restored by integrating
all sequential relative rotation matrices between successive images. Fig. 12,
12, 12 present the feature matching using BRISK, SURF and ORB respectively,
for the images FC21B0088753_17042144605F2C and FC21B0088785_17042150857F2C.
Here, the random sample consensus (RANSAC, [52, 53]) is invoked for figuring
out the correspondences between 2D-2D image feature points. All the algorithms
in this sub-section are implemented using the MATLAB computer vision toolbox.
The feature numbers, the RANSAC successfull ratio and the execution time of
various algorithms are summarized in Table I.

Table I: Stats of Various Feature Extraction Algorithms

Algorithms | Feature Numbers | RANSAC Successful Ratio | Time (s)
---|---|---|---
BRISK | $84$ | $97.62\%$ | 0.885
SURF | $237$ | $84.38\%$ | 1.277
ORB | $24$ | $100\%$ | 1.092

From the statistics, we can see that ORB extracts too few features but the
RANSAC matching is the most successful. While the SURF consumes the most
computation time and least RANSAC successful ratio (see the red markers in
Fig. 12), the BRISK maintains a balance among the three ones. Therefore the
BRISK feature descriptor is ultilized for the attitude determination
validation. With the PnP method, a nonlinear semidefinite programming is
employed to solve the 2D-3D registration problem between 2D image and the 3D
terrain map [54]. Here the 3D terrain map is generated using the images of the
Dawn spacecraft during the Survey mission of the Ceres, which can be found out
at https://sbib.psi.edu. Fig. 13 shows the exact details of such 3D terrain
map.
The reference attitude information from the Dawn spacecraft attitude estimator
can be acquired from LBL files in the data folder. The real-world images from
the Dawn spacecraft during the GRaND mission are grabbed for the verification
of the proposed algorithm since in that image sequence the illuminance and the
shape of the imaged Ceres are more appropriate than that in other datasets.
The relative weighting of the vector observations and hand-eye measurements is
tuned to $\varrho=0.5$ indicating that the hand-eye measurements own twice
accuracy than a single pair of feature correspondence. The PnP and epipolar
geometry do not give 3D vector observation pairs directly. Instead, they are
nonlinear and only the rotation matrices can be estimated. Suppose we have
estimated the attitude matrices from PnP and epipolar geometry i.e.
$\bm{R}_{\rm{PnP}},\bm{R}_{\rm{epipolar}}$, with the similar approach in (29),
we can reconstruct the vector observations by

$\left\\{{\begin{array}[]{*{20}{c}}{{{\bm{d}}_{{\rm{PnP}},1}}={\bm{R}}_{\rm{PnP}}{{\bm{e}}_{1}}}\\\
{{{\bm{d}}_{{\rm{PnP}},2}}={\bm{R}}_{\rm{PnP}}{{\bm{e}}_{2}}}\\\
{{{\bm{d}}_{{\rm{PnP}},3}}={\bm{R}}_{\rm{PnP}}{{\bm{e}}_{3}}}\\\
\end{array}}\right.,\left\\{{\begin{array}[]{*{20}{c}}{{{\bm{d}}_{{\rm{epipolar}},1}}={\bm{R}}_{\rm{epipolar}}{{\bm{e}}_{1}}}\\\
{{{\bm{d}}_{{\rm{epipolar}},2}}={\bm{R}}_{\rm{epipolar}}{{\bm{e}}_{2}}}\\\
{{{\bm{d}}_{{\rm{epipolar}},3}}={\bm{R}}_{\rm{epipolar}}{{\bm{e}}_{3}}}\\\
\end{array}}\right.$ (42)

The reason for such reconstruction is two-fold: 1) If we directly solve hand-
eye, PnP and epipolar geometry problems together, there will be too many local
minima involved in the global searching for the highly non-convex combined new
problem; 2) Due the local minima, estimating a controllable and accurate
covariance description may be very tough and trivial. Such reconstruction of
vectors will not loose accuracy of the obtained rotation since the rotation is
exactly the optimal one on $SO(n)$corresponding to the above vector-matching
problem [9, 43]. For the epipolar geometry, the rotation matrices are
propagated sequentially based on the estimates in the latest time instant.
Then the attitude determination results are processed with the proposed method
and previous representatives. Table II consists of the root mean squared (RMS)
statistics of the attitude errors in Euler angles.

Table II: RMS Accuracy Performances of Various Algorithms

Algorithms | Roll (deg) | Pitch (deg) | Yaw (deg)
---|---|---|---
Proposed-PnP | $5.33\times 10^{-03}$ | $3.21\times 10^{-03}$ | $5.81\times 10^{-02}$
Proposed-Epipolar | $8.54\times 10^{-03}$ | $7.46\times 10^{-03}$ | $3.11\times 10^{-02}$
PnP-Only | $1.22\times 10^{-03}$ | $3.01\times 10^{-03}$ | $3.78\times 10^{-02}$
Modenini [17] | $8.21\times 10^{-03}$ | $6.04\times 10^{-03}$ | $1.99\times 10^{+00}$

It is clearly presented that the original algorithm by Modenini can give very
accurate estimates of roll and pitch but does not have good performance in
estimating the yaw angles. The reason has been shown in [17, 18] that for some
perspective views the imaged ellipsoids are very close to spheroids which
limits the observability on the yaw direction. The reason is not algorithmical
but very physical since in many cases the flattenings of the imaged ellipsoid
are small. When introducing the PnP and the epipolar geometry, the accuracy
for yaw angles is significantly improved. In fact, the proposed method solves
the attitude determination problem between various weighted measurements. The
reason that PnP achieves better yaw estimation performance is that the imaging
direction well corresponds to the yaw (see Fig. 12). So the point feature
correspondences significantly improve the the yaw accuracy. It also deserves a
notification that the epipolar geometry does not achieve better roll and pitch
precisions since the attitude from such source is sequentially integrated and
thus it may suffer from low sampling frequency. However, epipolar geometry
does not need to have preliminaries on the 3D model of the imaged ellipsoid.
Therefore, from this aspect, epipolar geometry is more flexible than PnP. If
the 3D terrain of the imaged ellipsoid is already known e.g. the Moon, then
PnP should definitely replace the epipolar geometry for better accuracy and
robustness.

Figure 14: The captured Lunar images during the simulated flight.

###  III-C Integrating with Angular Rate

In this sub-section, we simulate a case in the presence of angular rate
measurements. Angular rate measurements mainly come from the onboard gyroscope
or the estimation from star trackers, which are accurate and reliable devices.
These measurements will provide two advances for the attitude determination
scheme proposed in this paper:

  1. 1.

They will largely improve the attitude update frequency.

  2. 2.

They will significantly improve the attitude estimation accuracy.

A spacecraft carrying high definition camera and gyroscope is modeled, whose
inertia matrix is ${\mathcal{I}}=4500\ {\rm{kg\cdot m^{2}}}\bm{I}$. The
gyroscope contains unknown constant bias term to be estimated in further
filter loops. We use the Moon as the central body and the orbit propagation is
conducted via the high-precision orbit propagator (HPOP) from Jet Propulsion
Laboratory (JPL). The captured Moon in different perspectives will be shown in
Fig. 14. The Lunar flattening is quite small, making it almost a perfect
sphere with semi-major axis of 3,476.2 km and semi-minor axis of 3,472.0 km.
Therefore, the observability of yaw is completely lost according to Modenini’s
theory [17]. Still, the vector observations are reconstructed from the
rotation estimation using the epipolar geometry as described in the last sub-
section. We use the Kalman filters for the state estimation. The preset
attitude profile has been generated using the following unit quaternion model

$\begin{gathered}\bm{q}_{i}=\sin(iT\bm{\phi}+\bm{\psi}),\ \ \ \ \
i=1,2,\cdots,20000\hfill\\\
\bm{q}_{i}=\bm{q}_{i}/\left\|\bm{q}_{i}\right\|\hfill\\\ \end{gathered}$ (43)

with $T=1/(1000\ \rm{Hz})$ being the sampling period and

$\begin{gathered}\bm{\phi}=(-0.12352,-0.31294,0.62993,-0.27127)^{\top}\\\
\bm{\psi}=(-0.74532,-0.24811,0.66610,-0.54501)^{\top}\\\ \end{gathered}$ (44)

The state vector is
$\tilde{\bm{x}}=\left(\bm{x}^{\top},\bm{\omega}^{\top}\right)^{\top}$ where
$\bm{\omega}$ is the angular rate vector that owns the model of

$\bm{\omega}=\hat{\bm{\omega}}+\bm{\omega}_{\rm{bias}}+\bm{\omega}_{\rm{noise}}$
(45)

in which
$\bm{\omega}_{\rm{bias}}=\left(\omega_{x,{\rm{bias}}},\omega_{y,{\rm{bias}}},\omega_{z,{\rm{bias}}}\right)^{\top}$
is the bias term and $\bm{\omega}_{\rm{noise}}$ denotes the noise term such
that
$\bm{\omega}_{\rm{noise}}\sim{\mathcal{N}}(\bm{0},\sigma_{\bm{\omega}}^{2}\bm{I})$
with isotropic standard deviation of $\sigma_{\bm{\omega}}$. The process model
of the Kalman filter is simply

$\left\\{\begin{gathered}{\rm{mat}}(\dot{\bm{x}})=-\bm{\omega}_{\times}{\rm{mat}}(\bm{x})\hfill\\\
\dot{\bm{\omega}}_{\rm{bias}}=\bm{0}\hfill\\\ \end{gathered}\right.$ (46)

where $\bm{\omega}_{\times}$ is the skew-symmetric matrix of $\bm{\omega}$.
The measurement model is

$\bm{y}=\bm{K}\tilde{\bm{x}}$ (47)

where
$\bm{y}=\left(\bm{x}_{\rm{meas}}^{\top},\bm{0}\right)^{\top}\in\mathbb{R}^{12}$
is the measurement vector and the measurement matrix
$\bm{K}\in\mathbb{R}^{12\times 12}$ is

$\bm{K}=\left(\begin{gathered}\bm{I}\ \ \ \bm{0}\\\ \bm{0}\ \ \ \bm{0}\\\
\end{gathered}\right)$ (48)

Here $\bm{x}_{\rm{meas}}$ denotes the computed measurement rotation matrix
using proposed method from (16). Since here the state $\bm{x}$ is constrained
on $SO(n)$, a recent filter with such constraint has been invoked [55]. The
extended Kalman filter (EKF) and unscented Kalman filter (UKF) mechanisms are
employed to [55] for state estimation. The related parameters are

  1. 1.

Initial state:
$\bm{\alpha}_{0}=\left[{\rm{vec}}(\bm{I}),\bm{0}\right]^{\top}$.

  2. 2.

Initial state covariance: $\bm{\Sigma}_{\tilde{\bm{x}}_{0}}=1\times
10^{-1}\bm{I}$.

  3. 3.

Gyroscope standard deviation: $\sigma_{\bm{\omega}}=1\times 10^{-2}\
{\rm{rad/s}}$.

  4. 4.

Bias standard deviation: $\sigma_{\bm{\omega}_{\rm{bias}}}=1\times 10^{-3}\
{\rm{rad/s}}$.

  5. 5.

Measurement covariance:
$\bm{\Sigma}_{\bm{y}}=\left(\begin{gathered}\bm{\Sigma}_{\bm{x}\bm{x}}\ \ \
\bm{0}\\\ \bm{0}\ \ \ \ \ \bm{0}\\\ \end{gathered}\right)$.

The state estimation and proposed algorithms are implemented on an embedded
STM32F407VG chip with clock speed of 168MHz. The feature extraction has been
conducted using the BRISK descriptor on a field programmable gate array (FPGA)
guaranteeing a stable processing image processing speed of 120fps. The
predicting loop is performed at 1000Hz and the measurement correction is
deployed at 120Hz in accordance with the feature extraction. The codes are
programmed using the C++11 standard. The attitude estimation results in the
form of quaternion from various algorithms are shown in Fig. 16. The proposed
analytical results can follow the reference but the noise scale is quite
large. When the Kalman filter is introduced for fusion with gyroscope, the
estimation errors significantly decrease, leading to accurate estimation of
the gyroscope biases presented in Fig. 16. Both EKF and UKF can converge to
the correct reference values and EKF converges a little bit slower than UKF.
This verifies that when combined with inertial measurements, the proposed
method along with its covariance information are accurate and reliable. The
load of the central processing unit (CPU) has been summarized via the internal
scheduling unit of the embedded operating system. Table III. From the
presented stats, we are able to see that the proposed method together with its
Kalman filter version with inertial measurements are computationally efficient
for execution on the embedded platform.

Table III: Computational CPU Load (Averaged)

Algorithms | CPU Load)
---|---
Proposed - No Covariance | $3.986\%$
Proposed - With Covariance | $6.710\%$
Proposed - EKF | $27.043\%$
Proposed - UKF | $34.925\%$

Figure 15: Attitude estimation results using the angular rate, epipolar
geometry and hand-eye measurements.

Figure 16: Estimated gyroscope biases.

##  IV Conclusion

The original attitude determination using hand-eye measurements derived from
images of ellipsoid has limited accuracy, robustness and observability. Vector
observations can well aid the spacecraft attitude determination with hand-eye
measurements and thus enhance the overall performances. It is noted that the
vector observations can come from various sources so the proposed scheme is
quite universal. These aspects contribute to a widened approach for visual
attitude determination for spacecraft in deep spaces. The experimental studies
have verified the effectiveness of the proposed scheme for in-flight attitude
determination. The shown results indicates that the proposed combined approach
is accurate for Dawn spacecraft. A further synthetic study on the fusion with
inertial measurements has also been conducted showing the flexibility and
superiority of multiple-source integration.
The presented approach also has its drawback i.e. the estimated $\bm{x}$ is
not strictly under the constraint of $SO(n)$ and requires a further
orthonormalization operation. The next task for us is to find a fully
$SO(n)$-constrained solution on Lie groups. Besides, the weighting strategy in
this paper is currently empirical. And how to optimally determine a best set
of weights in real-time will also be a challenging task.
For low-cost purposes, the ellipsoidal markers can be installed onto
spacecrafts for relative attitude determination using visual correspondences
and hand-eye measurements. Also, in the total design of the Dawn spacecraft,
there are two framing cameras mounted targeting to the same direction. This
will motivate us to invoke the stereo imaging principles to give more accurate
attitude determination results in the future.

$\small\begin{gathered}\bm{S}_{2}=\hfill\\\ \ \ \ \
\left<\begin{gathered}\left[\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\right]\bm{x}\bm{x}^{\top}\left[\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{\top}\end{gathered}\right>\hfill\\\
\approx\left<\delta\bm{H}\bm{x}\bm{x}^{\top}\delta\bm{H}+\left[\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\right]\bm{x}\bm{x}^{\top}\left[\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{\top}\right>\hfill\\\
=\underbrace{\left(\sum\limits_{i=1}^{M}v_{i}{\mathcal{F}}\left[\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)\bm{x}\right]\right)}_{n^{2}\times
2n^{2}}\bm{\Sigma}_{\bm{m}_{i}}\underbrace{\left(\sum\limits_{i=1}^{M}v_{i}{\mathcal{F}}\left[\left(\bm{A}_{i}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}^{\top}\right)\bm{x}\right]\right)^{\top}}_{2n^{2}\times
n^{2}}\hfill\\\ \ \ \ \ \
+\left[\sum\limits_{i=1}^{M}v_{i}\underbrace{\left(\bm{A}_{i}^{\top}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}\right)}_{n^{2}\times
n^{2}}\underbrace{{\mathcal{F}}(\bm{x})}_{n^{2}\times
2n^{2}}\right]\bm{\Sigma}_{\bm{n}_{i}}\left[\sum\limits_{i=1}^{M}v_{i}\left(\bm{A}_{i}^{\top}\otimes\bm{I}-\bm{I}\otimes\bm{B}_{i}\right){\mathcal{F}}(\bm{x})\right]^{\top}\hfill\\\
\ \ \ \ \
+{\mathcal{Z}}\left(\bm{RQ}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right)}{\mathcal{Z}}^{\top}\left(\bm{RQ}\right)+\left(\bm{Q}\otimes\bm{R}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}\right)}\left(\bm{Q}^{\top}\otimes\bm{R}^{\top}\right)\hfill\\\
\ \ \ \ \
+{\mathcal{Z}}\left(\bm{RQ}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right),{\rm{vec}}(\bm{Q})}\left(\bm{Q}^{\top}\otimes\bm{R}^{\top}\right)+\left(\bm{Q}\otimes\bm{R}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}\right),{\rm{vec}}\left(\bm{Q}^{\top}\right)}{\mathcal{Z}}^{\top}\left(\bm{RQ}\right)\hfill\\\
\end{gathered}$ (49)

$\begin{gathered}\bm{S}_{3}=\hfill\\\ \ \ \
-\left<\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]\bm{x}^{\top}\left[\delta\bm{H}+\left(\delta\bm{Q}\otimes\bm{I}\right)\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\right]^{\top}\right>\hfill\\\
\approx-\left<\left[{\mathcal{Z}}(\bm{P}){\rm{vec}}(\delta\bm{Q}^{\top})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]\left[{\mathcal{Z}}\left(\bm{RQ}\right){\rm{vec}}\left(\delta\bm{Q}^{\top}\right)+\left(\bm{Q}\otimes\bm{R}\right){\rm{vec}}(\delta\bm{Q})\right]^{\top}\right>\hfill\\\
=-{\mathcal{Z}}\left(\bm{P}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right)}{\mathcal{Z}}^{\top}\left(\bm{RQ}\right)-\left(\bm{Q}\otimes\bm{I}\right)\bm{\Sigma}_{{\rm{vec}}(\bm{P}),{\rm{vec}}(\bm{Q})}\left(\bm{Q}^{\top}\otimes\bm{R}^{\top}\right)\hfill\\\
\ \ \
-{\mathcal{Z}}\left(\bm{P}\right)\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right),{\rm{vec}}(\bm{Q})}\left(\bm{Q}^{\top}\otimes\bm{R}^{\top}\right)-\left(\bm{Q}\otimes\bm{I}\right)\bm{\Sigma}_{{\rm{vec}}(\bm{P}),{\rm{vec}}\left(\bm{Q}^{\top}\right)}{\mathcal{Z}}^{\top}\left(\bm{RQ}\right)\hfill\\\
\end{gathered}$ (50)

##  Appendix A Matrix Derivations

The items inside the internal expectation of $\bm{\Sigma}_{\bm{xx}}$ can be
derived as follows

$\begin{gathered}\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]\hfill\\\
\ \ \ \ \
\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]^{\top}\hfill\\\
=\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]\hfill\\\
\ \ \ \ \
\left[{\rm{vec}}(\bm{P})^{\top}\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)+{\rm{vec}}(\delta\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\right]\hfill\\\
=\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P}){\rm{vec}}(\bm{P})^{\top}\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\ \ \ \ \
\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P}){\rm{vec}}(\delta\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\ \ \ \ \
\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P}){\rm{vec}}(\bm{P})^{\top}\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\ \ \ \ \
\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P}){\rm{vec}}(\delta\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\Rightarrow\bm{S}_{1}=\left<\begin{gathered}\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]\\\
\left[\left(\delta\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\bm{P})+\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P})\right]^{\top}\end{gathered}\right>\hfill\\\
=\left<\begin{gathered}{\mathcal{Z}}(\bm{P}){\rm{vec}}(\delta\bm{Q}^{\top}){\rm{vec}}(\delta\bm{Q}^{\top})^{\top}{\mathcal{Z}}^{\top}(\bm{P})+\hfill\\\
{\mathcal{Z}}(\bm{P}){\rm{vec}}(\delta\bm{Q}^{\top}){\rm{vec}}(\delta\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)+\hfill\\\
\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P}){\rm{vec}}(\delta\bm{Q}^{\top})^{\top}{\mathcal{Z}}^{\top}(\bm{P})+\hfill\\\
\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}(\delta\bm{P}){\rm{vec}}(\delta\bm{P})^{\top}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\end{gathered}\right>\hfill\\\
={\mathcal{Z}}(\bm{P}){\bm{\Sigma}}_{{\rm{vec}}(\bm{Q}^{\top})}{\mathcal{Z}}^{\top}(\bm{P})\hfill\\\
\ \ \ \ \
+{\mathcal{Z}}(\bm{P})\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right),{\rm{vec}}(\bm{P})}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\ \ \ \ \
+\left(\bm{Q}\otimes\bm{I}\right)\bm{\Sigma}_{{\rm{vec}}(\bm{P}),{\rm{vec}}\left(\bm{Q}^{\top}\right)}{\mathcal{Z}}^{\top}(\bm{P})\hfill\\\
\ \ \ \ \
+\left(\bm{Q}\otimes\bm{I}\right){\bm{\Sigma}}_{{\rm{vec}}(\bm{P})}\left(\bm{Q}^{\top}\otimes\bm{I}\right)\hfill\\\
\end{gathered}$ (51)

Denoting

$\begin{gathered}\bm{\Sigma}_{\bm{m}_{i}}=\underbrace{\begin{array}[]{l}\left[{\begin{array}[]{*{20}{c}}{\bm{\Sigma}_{{\rm{vec}}(\bm{A}_{i})}}&{\bm{\Sigma}_{{\rm{vec}}(\bm{A}_{i}),{\rm{vec}}(\bm{B}_{i})}}\\\
{\bm{\Sigma}_{{\rm{vec}}(\bm{B}_{i}),{\rm{vec}}(\bm{A}_{i})}}&{\bm{\Sigma}_{{\rm{vec}}(\bm{B}_{i})}}\end{array}}\right]\end{array}}_{2n^{2}\times
2n^{2}}\hfill\\\
\bm{\Sigma}_{\bm{n}_{i}}=\underbrace{\begin{array}[]{l}\left[{\begin{array}[]{*{20}{c}}{\bm{\Sigma}_{{\rm{vec}}(\bm{A}_{i}^{\top})}}&{\bm{\Sigma}_{{\rm{vec}}(\bm{A}_{i}^{\top}),{\rm{vec}}(\bm{B}_{i}^{\top})}}\\\
{\bm{\Sigma}_{{\rm{vec}}(\bm{B}_{i}^{\top}),{\rm{vec}}(\bm{A}_{i}^{\top})}}&{\bm{\Sigma}_{{\rm{vec}}(\bm{B}_{i}^{\top})}}\end{array}}\right]\end{array}}_{2n^{2}\times
2n^{2}}\hfill\\\ \end{gathered}$ (52)

and invoking

$\begin{gathered}\left(\bm{Q}\otimes\bm{I}\right)\left(\delta\bm{Q}^{\top}\otimes\bm{I}\right)\bm{x}=\left(\bm{Q}\otimes\bm{I}\right){\rm{vec}}\left(\bm{R}\delta\bm{Q}\right)\hfill\\\
=\left(\bm{Q}\otimes\bm{I}\right)\left(\bm{I}\otimes\bm{R}\right){\rm{vec}}(\delta\bm{Q})\hfill\\\
=\left(\bm{Q}\otimes\bm{R}\right){\rm{vec}}(\delta\bm{Q})\hfill\\\
\end{gathered}$ (53)

we obtain $\bm{S}_{2},\bm{S}_{3}$ in (49) and (50). Note that in these
derivations, the cross correlation between vector observations and hand-eye
measurements is ignored as they come from completely different sources so a
cross-correlation evaluation may be trivial. Since we have

$\begin{gathered}\bm{P}^{\top}=\left(\begin{gathered}\sqrt{w_{1}}\bm{b}_{1}^{\top}\\\
\sqrt{w_{2}}\bm{b}_{2}^{\top}\\\ \vdots\\\ \sqrt{w_{N}}\bm{b}_{N}^{\top}\\\
\end{gathered}\right),\bm{Q}^{\top}=\left(\begin{gathered}\sqrt{w_{1}}\bm{r}_{1}^{\top}\\\
\sqrt{w_{2}}\bm{r}_{2}^{\top}\\\ \vdots\\\ \sqrt{w_{N}}\bm{r}_{N}^{\top}\\\
\end{gathered}\right)\end{gathered}$ (54)

let

$\bm{p}_{i}=\left(\begin{gathered}\sqrt{w_{1}}b_{1,i}\\\
\sqrt{w_{2}}b_{2,i}\\\ \vdots\\\ \sqrt{w_{N}}b_{N,i}\\\
\end{gathered}\right),\bm{q}_{i}=\left(\begin{gathered}\sqrt{w_{1}}r_{1,i}\\\
\sqrt{w_{2}}r_{2,i}\\\ \vdots\\\ \sqrt{w_{N}}r_{N,i}\\\
\end{gathered}\right),i=1,2,\cdots,n$ (55)

we have

$\begin{gathered}\bm{P}^{\top}=\left(\bm{p}_{1},\bm{p}_{2},\cdots,\bm{p}_{n}\right)\hfill\\\
\bm{Q}^{\top}=\left(\bm{q}_{1},\bm{q}_{2},\cdots,\bm{q}_{n}\right)\hfill\\\
\end{gathered}$ (56)

$\begin{gathered}\bm{\Sigma}_{{\rm{vec}}\left(\bm{P}^{\top}\right),{\rm{vec}}(\bm{Q})}=\left<{\rm{vec}}\left(\delta\bm{P}^{\top}\right){\rm{vec}}(\delta\bm{Q})^{\top}\right>\hfill\\\
=\left<\left(\begin{gathered}\delta\bm{p}_{1}\\\ \delta\bm{p}_{2}\\\ \vdots\\\
\delta\bm{p}_{n}\\\
\end{gathered}\right)\left(\sqrt{w_{1}}\delta\bm{r}_{1}^{\top},\sqrt{w_{2}}\delta\bm{r}_{2}^{\top},\cdots,\sqrt{w_{N}}\delta\bm{r}_{N}^{\top}\right)\right>\hfill\\\
=\left<\left(\begin{gathered}\sqrt{w_{1}}\delta\bm{p}_{1}\delta\bm{r}_{1}^{\top},\sqrt{w_{2}}\delta\bm{p}_{1}\delta\bm{r}_{2}^{\top},\cdots,\sqrt{w_{N}}\delta\bm{p}_{1}\delta\bm{r}_{N}^{\top}\hfill\\\
\sqrt{w_{1}}\delta\bm{p}_{2}\delta\bm{r}_{1}^{\top},\sqrt{w_{2}}\delta\bm{p}_{2}\delta\bm{r}_{2}^{\top},\cdots,\sqrt{w_{N}}\delta\bm{p}_{2}\delta\bm{r}_{N}^{\top}\hfill\\\
\ \ \ \ \ \ \ \vdots\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \vdots\ \ \ \ \ \
\ddots\ \ \ \ \ \ \ \ \ \vdots\hfill\\\
\sqrt{w_{1}}\delta\bm{p}_{n}\delta\bm{r}_{1}^{\top},\sqrt{w_{2}}\delta\bm{p}_{n}\delta\bm{r}_{2}^{\top},\cdots,\sqrt{w_{N}}\delta\bm{p}_{n}\delta\bm{r}_{N}^{\top}\hfill\\\
\end{gathered}\right)\right>\hfill\\\ \end{gathered}$
$\begin{gathered}=\underbrace{\left(\begin{gathered}\sqrt{w_{1}}\bm{\Sigma}_{\bm{p}_{1},\bm{r}_{1}},\sqrt{w_{2}}\bm{\Sigma}_{\bm{p}_{1},\bm{r}_{2}},\cdots,\sqrt{w_{N}}\bm{\Sigma}_{\bm{p}_{1},\bm{r}_{N}}\hfill\\\
\sqrt{w_{1}}\bm{\Sigma}_{\bm{p}_{2},\bm{r}_{1}},\sqrt{w_{2}}\bm{\Sigma}_{\bm{p}_{2},\bm{r}_{2}},\cdots,\sqrt{w_{N}}\bm{\Sigma}_{\bm{p}_{2},\bm{r}_{N}}\hfill\\\
\ \ \ \ \ \ \ \vdots\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \vdots\ \ \ \ \ \ \ddots\ \
\ \ \ \ \ \ \ \vdots\hfill\\\
\sqrt{w_{1}}\bm{\Sigma}_{\bm{p}_{n},\bm{r}_{1}},\sqrt{w_{2}}\bm{\Sigma}_{\bm{p}_{n},\bm{r}_{2}},\cdots,\sqrt{w_{N}}\bm{\Sigma}_{\bm{p}_{n},\bm{r}_{N}}\hfill\\\
\end{gathered}\right)}_{nN\times nN}\end{gathered}$ (57)

Then
$\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right),{\rm{vec}}(\bm{Q})},\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right),{\rm{vec}}(\bm{P})},\bm{\Sigma}_{{\rm{vec}}\left(\bm{P}\right),{\rm{vec}}(\bm{Q}^{\top})},$
can be computed in the same manner. Note that if there is no auto-correlation
between $\bm{b}_{1},\bm{b}_{2},\cdots,\bm{b}_{N}$, no auto-correlation between
$\bm{r}_{1},\bm{r}_{2},\cdots,\bm{r}_{N}$ and no cross-correlation between
$\bm{b}_{i},i=1,2,\cdots,N$ and $\bm{r}_{j},j=1,2,\cdots,N$, one arrives at

$\begin{gathered}\bm{\Sigma}_{\bm{p}_{i},\bm{r}_{j}}=\bm{0}\\\
\bm{\Sigma}_{\bm{q}_{i},\bm{b}_{j}}=\bm{0}\\\
\underbrace{\bm{\Sigma}_{\bm{p}_{i},\bm{b}_{j}}=\left(\begin{gathered}\ddots\
\ \ \ \ \ \ \ \bm{0}\\\ \sigma_{b_{j,i}}^{2}\\\ \bm{0}\ \ \ \ \ \ \ \ \
\ddots\\\ \end{gathered}\right)}_{N\times
n},\underbrace{\bm{\Sigma}_{\bm{q}_{i},\bm{r}_{j}}=\left(\begin{gathered}\ddots\
\ \ \ \ \ \ \ \bm{0}\\\ \sigma_{r_{j,i}}^{2}\\\ \bm{0}\ \ \ \ \ \ \ \ \
\ddots\\\ \end{gathered}\right)}_{N\times n}\hfill\\\ \end{gathered}$ (58)

where $i=1,2,\cdots,n$, $j=1,2,\cdots,N$. The auto-covariance of
${\rm{vec}}\left(\bm{P}^{\top}\right)$ is given by

$\begin{gathered}\bm{\Sigma}_{{\rm{vec}}\left(\bm{P}^{\top}\right)}=\left<{\rm{vec}}\left(\delta\bm{P}^{\top}\right){\rm{vec}}\left(\delta\bm{P}^{\top}\right)^{\top}\right>\hfill\\\
\end{gathered}$
$\begin{gathered}=\left<\left(\begin{gathered}\delta\bm{p}_{1}\\\
\delta\bm{p}_{2}\\\ \vdots\\\ \delta\bm{p}_{n}\\\
\end{gathered}\right)\left(\delta\bm{p}_{1}^{\top},\delta\bm{p}_{2}^{\top},\cdots,\delta\bm{p}_{n}^{\top}\right)\right>\hfill\\\
=\left(\begin{gathered}\bm{\Sigma}_{\bm{p}_{1},\bm{p}_{1}},\bm{\Sigma}_{\bm{p}_{1},\bm{p}_{2}},\cdots,\bm{\Sigma}_{\bm{p}_{1},\bm{p}_{n}}\hfill\\\
\bm{\Sigma}_{\bm{p}_{2},\bm{p}_{1}},\bm{\Sigma}_{\bm{p}_{2},\bm{p}_{2}},\cdots,\bm{\Sigma}_{\bm{p}_{2},\bm{p}_{n}}\hfill\\\
\ \ \ \ \ \vdots\ \ \ \ \ \ \ \ \ \vdots\ \ \ \ddots\ \ \ \ \ \
\vdots\hfill\\\
\bm{\Sigma}_{\bm{p}_{n},\bm{p}_{1}},\bm{\Sigma}_{\bm{p}_{n},\bm{p}_{2}},\cdots,\bm{\Sigma}_{\bm{p}_{n},\bm{p}_{n}}\hfill\\\
\end{gathered}\right)\end{gathered}$ (59)

and likewise $\bm{\Sigma}_{{\rm{vec}}\left(\bm{Q}^{\top}\right)}$ can be
obtained. If there is no auto-correlation between
$\bm{b}_{1},\bm{b}_{2},\cdots,\bm{b}_{N}$ and no auto-correlation between
$\bm{r}_{1},\bm{r}_{2},\cdots,\bm{r}_{N}$, we have

$\begin{gathered}\bm{\Sigma}_{\bm{p}_{i},\bm{p}_{j}}=\underbrace{{\rm{diag}}\left(\sigma_{b_{1,i},b_{1,j}}^{2},\sigma_{b_{2,i},b_{2,j}}^{2},\cdots,\sigma_{b_{n,i},b_{n,j}}^{2}\right)}_{N\times
N}\\\
\bm{\Sigma}_{\bm{q}_{i},\bm{q}_{j}}=\underbrace{{\rm{diag}}\left(\sigma_{r_{1,i},r_{1,j}}^{2},\sigma_{r_{2,i},r_{2,j}}^{2},\cdots,\sigma_{r_{n,i},r_{n,j}}^{2}\right)}_{N\times
N}\\\ \end{gathered}$ (60)

## Acknowledgment

The author would like to thank Dr. D. Modenini from University of Bologna,
Italy for his constructive discussion on the hand-eye calibration and attitude
determination problems.
This research has been supported by National Natural Science Foundation of
China under the grant of No. 41604025 and in part by the Open Project of
Shanghai Key Laboratory of Navigation and Location-based Services, Shanghai
Jiao Tong University.
Some illustrative examples and codes of this paper are public and can be
accessed at https://github.com/zarathustr/vec_hand_eye_att.

## References

  * [1] C. Kohlhase and P. Penzo

> Voyager Mission Description
> _Space Sci. Rev._ , vol. 21, no. 2, pp. 77–101, 1977.

  * [2] R. E. Arvidson _et al._

> Spirit Mars Rover Mission to The Columbia Hills, Gusev Crater: Mission
> Overview and Selected Results from The Cumberland Ridge to Home Plate
> _J. Geophysical Research: Planets_ , vol. 113, no. E12, 2008.

  * [3] D. M. Hassler _et al._

> Mars’ Surface Radiation Environment Measured with the Mars Science
> Laboratory’s Curiosity Rover
> _Science_ , vol. 343, no. 6169, p. 1244797, 2014.

  * [4] L. Xiao _et al._

> A Young Multilayered Rerrane of The Northern Mare Imbrium Revealed by
> Chang’E-3 Mission
> _Science_ , vol. 347, no. 6227, pp. 1226–1229, 2015.

  * [5] Yuanxin Wu _et al._

> Strapdown Inertial Navigation System Algorithms Based on Dual Quaternions
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 41, no. 1, pp. 110–132, 2005.

  * [6] M. Wang _et al._

> High-order Attitude Compensation in Coning and Rotation Coexisting
> Environment
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 51, no. 2, pp. 1178–1190, 2015.

  * [7] Y. Wu and G. Yan

> Attitude Reconstruction from Inertial Measurements : QuatFIter and Its
> Comparison with RodFIter
> _IEEE Trans. Aerosp. Elect. Syst._ , 2019.

  * [8] J. L. Crassidis, F. L. Markley, and Y. Cheng

> Survey of Nonlinear Attitude Estimation Methods
> _AIAA J. Guid. Control Dyn._ , vol. 30, no. 1, pp. 12–28, 2007.

  * [9] M. D. Shuster and S. D. Oh

> Three-axis Attitude Determination from Vector Observations
> _AIAA J. Guid. Control Dyn._ , vol. 4, no. 1, pp. 70–77, 1981.

  * [10] D. Mortari

> Second Estimator of the Optimal Quaternion
> _AIAA J. Guid. Control Dyn._ , vol. 23, no. 5, pp. 885–888, 2000.

  * [11] D. Mortari, F. L. Markley, and P. Singla

> Optimal Linear Attitude Estimator
> _AIAA J. Guid. Control Dyn._ , vol. 30, no. 6, pp. 1619–1627, 2007.

  * [12] D. Choukroun, I. Y. Bar-Itzhack, and Y. Oshman

> Novel Quaternion Kalman Filter
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 42, no. 1, pp. 174–190, 2006.

  * [13] R. Mahony, T. Hamel, and J. Pflimlin

> Nonlinear Complementary Filters on the Special Orthogonal Group
> _IEEE Trans. Autom. Control_ , vol. 53, no. 5, pp. 1203–1218, 2008.

  * [14] S. Bahrami and M. Namvar

> Global Attitude Estimation Using Single Delayed Vector Measurement and
> Biased Gyro
> _Automatica_ , vol. 75, pp. 88–95, 2017.

  * [15] R. Y. Tsai and R. K. Lenz

> A New Technique for Fully Autonomous and Efficient 3D Robotics Hand/Eye
> Calibration
> _IEEE Trans. Robot. Autom._ , vol. 5, no. 3, pp. 345–358, 1989.

  * [16] K. Daniilidis

> Hand-Eye Calibration Using Dual Quaternions
> _Int. J. Robot. Research_ , vol. 18, no. 3, pp. 286–298, 1999.

  * [17] D. Modenini

> Attitude Determination from Ellipsoid Observations: A Modified Orthogonal
> Procrustes Problem
> _AIAA J. Guid. Control Dyn._ , vol. 41, no. 10, pp. 2320–2325, 2018.

  * [18] D. Modenini

> Five-Degree-of-Freedom Pose Estimation from an Imaged Ellipsoid of
> Revolution
> _AIAA J. Spacecraft Rockets_ , vol. 56, no. 3, pp. 952–958, 2019.

  * [19] D. Modenini and M. Zannoni

> A High Accuracy Horizon Sensor for Small Satellites
> IEEE 5th International Workshop on Metrology for AeroSpace (MetroAeroSpace),
> Torino, Italy, pp. 451–456, 2019.

  * [20] D. Man and A. Vision

> A Computational Investigation into the Human Representation and Processing
> of Visual Information
> 1982.

  * [21] B. Bercovici and J. W. McMahon

> Point-Cloud Processing Using Modified Rodrigues Parameters for Relative
> Navigation
> _AIAA J. Guid. Control Dyn._ , vol. 40, no. 12, pp. 3167–3179, 2017.

  * [22] X.-S. Gao _et al._

> Complete Solution Classification for the Perspective-Three-Point Problem
> _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 25, no. 8, pp. 930–943,
> 2003.

  * [23] Z. Zhang

> Determining the Epipolar Geometry and its Uncertainty: A Review
> _Int. J. Comput. Vis._ , vol. 27, no. 2, pp. 161–195, 1998.

  * [24] Y. Abdel-Aziz, H. Karara, and M. Hauck

> Direct Linear Transformation from Comparator Coordinates into Object Space
> Coordinates in Close-range Photogrammetry
> _Photogram. Eng. & Remote Sensing_, vol. 81, no. 2, pp. 103–107, 2015.

  * [25] V. Lepetit, F. Moreno-Noguer, and P. Fua

> EPnP: An accurate O(n) solution to the PnP problem
> _Int. J. Comput. Vis._ , vol. 81, no. 2, pp. 155–166, 2009.

  * [26] G. Briskin _et al._

> Estimating Pose and Motion Using Bundle Adjustment and Digital Elevation
> Model Constraints
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 53, no. 4, pp. 1614–1624, 2017.

  * [27] Q. Cai _et al._

> Equivalent Constraints for Two-View Geometry: Pose Solution/Pure Rotation
> Identification and 3D Reconstruction
> _Int. J. Comput. Vis._ , vol. 127, no. 2, pp. 163-180, 2019.

  * [28] D. Mortari

> Moon-Sun Attitude Sensor
> _AIAA J. Spacecraft Rockets_ , vol. 34, no. 4, pp. 360–364, 1997.

  * [29] D. Choukroun _et al._

> Direction Cosine Matrix Estimation from Vector Observations Using a Matrix
> Kalman Filter
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 46, no. 1, pp. 61–79, 2010.

  * [30] P. J. Besl and N. D. McKay

> A Method for Registration of 3-D Shapes
> _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 14, no. 2, pp. 239–256,
> 1992.

  * [31] J. Wu _et al._

> Fast Symbolic 3D Registration Solution
> _IEEE Trans. Autom. Sci. Eng._ , 2019.

  * [32] J. Wu _et al._

> Hand-eye Calibration: 4D Procrustes Analysis Approach
> _IEEE Trans. Instrum. Meas._ , 2019.

  * [33] J. Wu, M. Liu, and Y. Qi

> Computationally Efficient Robust Algorithm for Generalized Sensor
> Calibration Problem AR = RB
> _IEEE Sensors J._ , 2019.

  * [34] N. Andreff, R. Horaud, and B. Espiau

> Robot Hand-Eye Calibration Using Structure-from-Motion
> _Int. J. Robot. Research_ , vol. 20, no. 3, pp. 228–248, 2001.

  * [35] A. Tabb and K. M. Ahmad Yousef

> Solving the Robot-World Hand-eye(s) Calibration Problem with Iterative
> Methods
> _Machine Vis. Appl._ , vol. 28, no. 5-6, pp. 569–590, 2017.

  * [36] J. Wu _et al._

> Attitude Determination Using a Single Sensor Observation : Analytic
> Quaternion Solutions and Property Discussion
> _IET Sci. Meas. Tech._ , vol. 11, no. 3, pp. 731–739, 2017.

  * [37] J. Wu et al.

> Recursive Linear Continuous Quaternion Attitude Estimator from Vector
> Observations
> _IET Radar Sonar Nav._ , vol. 12, no. 11, pp. 1196–1207, 2018.

  * [38] F. L. Markley and Y. Cheng

> Wahba’s Problem with One Dominant Observation
> _AIAA J. Guid. Control Dyn._ , no. 9, pp. 1–6, 2018.

  * [39] F. L. Markley

> Fast Quaternion Attitude Estimation from Two Vector Measurements
> _AIAA J. Guid. Control Dyn._ , vol. 25, no. 2, pp. 411–414, 2002.

  * [40] Z. Zhou _et al._

> Equality Constrained Robust Measurement Fusion for Adaptive Kalman-Filter-
> Based Heterogeneous Multi-sensor Navigation
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 49, no. 4, pp. 2146–2157, 2013.

  * [41] Z. Zhou

> Optimal Batch Distributed Asynchronous Multi-sensor Fusion with Feedback
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 55, no. 1, pp. 46–56, 2019.

  * [42] A. Liounis and J. Christian

> Techniques for Generating Analytic Covariance Expressions for Eigenvalues
> and Eigenvectors
> _IEEE Trans. Signal Process._ , vol. 64, no. 7, pp. 1808–1821, 2016.

  * [43] I. Y. Bar-Itzhack

> New Method for Extracting the Quaternion from a Rotation Matrix
> _AIAA J. Guid. Control Dyn._ , vol. 23, no. 6, pp. 1085–1087, 2019.

  * [44] J. Wu _et al._

> Generalized $n$-Dimensional Rigid Registration: Theory and Application
> 2019\. DOI: 10.13140/RG.2.2.14007.68000

  * [45] J. Wu

> Optimal Continuous Unit Quaternions from Rotation Matrices
> _AIAA J. Guid. Control Dyn._ , vol. 42, no. 4, pp. 919–922, 2019.

  * [46] H. Bay _et al._

> Speeded-Up Robust Features (SURF)
> _Comput. Vis. Image Undestand._ , vol. 110, no. 3, pp. 346–359, 2008.

  * [47] S. Leutenegger, M. Chli, and R. Y. Siegwart

> BRISK: Binary Robust Invariant Scalable Keypoints
> _IEEE ICCV_ , 2011.

  * [48] E. Rublee _et al._

> ORB: An Efficient Alternative to SIFT or SURF
> _IEEE ICCV_ , 2011.

  * [49] A. Fitzgibbon, M. Pilu, and R. B. Fisher

> Direct Least Square Fitting of Ellipses
> _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 21, no. 5, pp. 476–480,
> 1999.

  * [50] L. Quan

> Conic Reconstruction and Correspondence from Two Views
> _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 18, no. 2, pp. 151–160,
> 1996.

  * [51] Z. Zhang

> Parameter Estimation Techniques: A Tutorial with Application to Conic
> Fitting
> _Image Vis. Comput._ , vol. 15, no. 1, pp. 59–76, 1997.

  * [52] O. Chum and J. Matas

> Optimal Randomized RANSAC
> _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 30, no. 8, pp. 1472–1482,
> 2008.

  * [53] S. A. Rawashdeh and J. E. Lumpp

> Image-based Attitude Propagation for Small Satellites Using RANSAC
> _IEEE Trans. Aerosp. Elect. Syst._ , vol. 50, no. 3, pp. 1864–1875, 2014.

  * [54] Y. Khoo and A. Kapoor

> Non-Iterative Rigid 2D / 3D Point-Set Registration Using Semidefinite
> Programming
> _IEEE Trans. Image Process._ , vol. 25, no. 7, pp. 2956–2970, 2016.

  * [55] A. H. J. de Ruiter and J. R. Forbes

> Discrete-Time SO(n)-Constrained Kalman Filtering
> _AIAA J. Guid. Control Dyn._ , vol. 40, no. 1, pp. 28–37, 2017.

|  Jin Wu was born in May, 1994 in Zhenjiang, China. He received the B.S.
Degree from University of Electronic Science and Technology of China, Chengdu,
China. He has been a research assistant in Department of Electronic and
Computer Engineering, Hong Kong University of Science and Technology since
2018. His research interests include robot navigation, multi-sensor fusion,
automatic control and mechatronics. He is a co-author of over 40 technical
papers in representative journals and conference proceedings of IEEE, AIAA,
IET and etc. Mr. Jin Wu received the outstanding reviewer award for Asian
Journal of Control. One of his papers published in IEEE Transactions on
Automation Science and Engineering was selected as the ESI Highly Cited Paper
by ISI Web of Science during 2017 to 2018. He is a member of IEEE.
Mr. J. Wu started software programming using C++ since 2004. He has been in
the UAV industry from 2012 and has launched two companies ever since. He has
been a part-time Robotics Engineer in Tencent Robotics X, Shenzhen, China and
Shenzhen Unity Drive, Shenzhen, China, since 2019. He is the inventor of the
world’s first ultra low-cost circuit for 3-D and higher dimensional
registration. He is currently in charge of some open projects of robotics and
aerospace engineering from several State Key Laboratories in China.
---|---