* Kober et al. (2013) Jens Kober, J Andrew Bagnell, and Jan Peters. 2013\.  Reinforcement learning in robotics: A survey.  _The International Journal of Robotics Research_ 32, 11 (2013), 1238–1274.
  * Kundu (2021) Shinjini Kundu. 2021\.  AI in medicine must be explainable.  _Nature medicine_ 27, 8 (2021), 1328–1328.
  * Lakkaraju et al. (2017) Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2017.  Interpretable & explorable approximations of black box models.  _arXiv preprint arXiv:1707.01154_ (2017).
  * Landajuela et al. (2021) Mikel Landajuela, Brenden K Petersen, Sookyung Kim, Claudio P Santiago, Ruben Glatt, Nathan Mundhenk, Jacob F Pettit, and Daniel Faissol. 2021.  Discovering symbolic policies with deep reinforcement learning. In _International Conference on Machine Learning_. PMLR, 5979–5989.
  * Laugel et al. (2017) Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017\.  Inverse classification for comparison-based interpretability in machine learning.  _arXiv preprint arXiv:1712.08443_ (2017).
  * Lillicrap et al. (2015) Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015.  Continuous control with deep reinforcement learning.  _arXiv preprint arXiv:1509.02971_ (2015).
  * Lipton (1990) Peter Lipton. 1990\.  Contrastive explanation.  _Royal Institute of Philosophy Supplements_ 27 (1990), 247–266.
  * Liu et al. (2014) Chunming Liu, Xin Xu, and Dewen Hu. 2014.  Multiobjective reinforcement learning: A comprehensive overview.  _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ 45, 3 (2014), 385–398.
  * Liu et al. (2018) Guiliang Liu, Oliver Schulte, Wang Zhu, and Qingcan Li. 2018\.  Toward interpretable deep reinforcement learning with linear model u-trees. In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 414–429.
  * Looveren and Klaise (2021) Arnaud Van Looveren and Janis Klaise. 2021.  Interpretable counterfactual explanations guided by prototypes. In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 650–665.
  * Lundberg and Lee (2017) Scott M Lundberg and Su-In Lee. 2017.  A unified approach to interpreting model predictions. In _Proceedings of the 31st international conference on neural information processing systems_. 4768–4777.
  * Madumal et al. (2020) Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2020.  Explainable reinforcement learning through a causal lens. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 2493–2500.
  * Mahajan et al. (2019) Divyat Mahajan, Chenhao Tan, and Amit Sharma. 2019.  Preserving causal constraints in counterfactual explanations for machine learning classifiers.  _arXiv preprint arXiv:1912.03277_ (2019).
  * Miller (2019) Tim Miller. 2019\.  Explanation in artificial intelligence: Insights from the social sciences.  _Artificial intelligence_ 267 (2019), 1–38.
  * Mnih et al. (2013) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013.  Playing atari with deep reinforcement learning.  _arXiv preprint arXiv:1312.5602_ (2013).
  * Molnar (2019) Christoph Molnar. 2019\.  _Interpretable Machine Learning_.
  * Mothilal et al. (2020) Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. 2020\.  Explaining machine learning classifiers through diverse counterfactual explanations. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. 607–617.
  * Olson et al. (2019) Matthew L Olson, Lawrence Neal, Fuxin Li, and Weng-Keen Wong. 2019\.  Counterfactual states for atari agents via generative deep learning.  _arXiv preprint arXiv:1909.12969_ (2019).
  * Pearl and Mackenzie (2018) Judea Pearl and Dana Mackenzie. 2018.  _The book of why: the new science of cause and effect_.  Basic books.
  * Peng et al. (2018) Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018.  Deepmimic: Example-guided deep reinforcement learning of physics-based character skills.  _ACM Transactions on Graphics (TOG)_ 37, 4 (2018), 1–14.
  * Pouyanfar et al. (2018) Samira Pouyanfar, Saad Sadiq, Yilin Yan, Haiman Tian, Yudong Tao, Maria Presa Reyes, Mei-Ling Shyu, Shu-Ching Chen, and Sundaraja S Iyengar. 2018.  A survey on deep learning: Algorithms, techniques, and applications.  _ACM Computing Surveys (CSUR)_ 51, 5 (2018), 1–36.
  * Poyiadzi et al. (2020) Rafael Poyiadzi, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020\.  FACE: feasible and actionable counterfactual explanations. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_. 344–350.
  * Puiutta and Veith (2020) Erika Puiutta and Eric Veith. 2020.  Explainable reinforcement learning: A survey. In _International cross-domain conference for machine learning and knowledge extraction_. Springer, 77–95.
  * Puri et al. (2019) Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, and Sameer Singh. 2019.  Explain your move: Understanding agent actions using specific and relevant feature attribution.  _arXiv preprint arXiv:1912.12191_ (2019).
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016\.  ” Why should i trust you?” Explaining the predictions of any classifier. In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_. 1135–1144.
  * Samoilescu et al. (2021) Robert-Florian Samoilescu, Arnaud Van Looveren, and Janis Klaise. 2021\.  Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning.  _arXiv preprint arXiv:2106.02597_ (2021).
  * Sequeira and Gervasio (2020) Pedro Sequeira and Melinda Gervasio. 2020.  Interestingness elements for explainable reinforcement learning: Understanding agents’ capabilities and limitations.  _Artificial Intelligence_ 288 (2020), 103367.
  * Sharma et al. (2019) Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2019\.  Certifai: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models.  _arXiv preprint arXiv:1905.07857_ (2019).
  * Simonyan et al. (2013) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013\.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  _arXiv preprint arXiv:1312.6034_ (2013).
  * Sokol and Flach (2019) Kacper Sokol and Peter A Flach. 2019.  Counterfactual explanations of machine learning predictions: opportunities and challenges for AI safety.  _SafeAI@ AAAI_ (2019).
  * Stepin et al. (2021) Ilia Stepin, Jose M Alonso, Alejandro Catala, and Martín Pereira-Fariña. 2021\.  A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence.  _IEEE Access_ 9 (2021), 11974–12001.
  * Štrumbelj and Kononenko (2014) Erik Štrumbelj and Igor Kononenko. 2014.  Explaining prediction models and individual predictions with feature contributions.  _Knowledge and information systems_ 41, 3 (2014), 647–665.
  * Sukkerd et al. (2020) Roykrong Sukkerd, Reid Simmons, and David Garlan. 2020\.  Tradeoff-focused contrastive explanation for MDP planning. In _2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)_. IEEE, 1041–1048.
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013\.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ (2013).
  * Topin and Veloso (2019) Nicholay Topin and Manuela Veloso. 2019.  Generation of policy-level explanations for reinforcement learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 2514–2521.
  * Ustun et al. (2019) Berk Ustun, Alexander Spangher, and Yang Liu. 2019\.  Actionable recourse in linear classification. In _Proceedings of the conference on fairness, accountability, and transparency_. 10–19.
  * van der Waa et al. (2018) Jasper van der Waa, Jurriaan van Diggelen, Karel van den Bosch, and Mark Neerincx. 2018\.  Contrastive explanations for reinforcement learning in terms of expected consequences.  _arXiv preprint arXiv:1807.08706_ (2018).
  * Verma (2019) Abhinav Verma. 2019\.  Verifiable and interpretable reinforcement learning through program synthesis. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 9902–9903.
  * Verma et al. (2018) Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. 2018\.  Programmatically interpretable reinforcement learning. In _International Conference on Machine Learning_. PMLR, 5045–5054.
  * Verma et al. (2020) Sahil Verma, John Dickerson, and Keegan Hines. 2020\.  Counterfactual explanations for machine learning: A review.  _arXiv preprint arXiv:2010.10596_ (2020).
  * Verma et al. (2021) Sahil Verma, John Dickerson, and Keegan Hines. 2021\.  Counterfactual Explanations for Machine Learning: Challenges Revisited.  _arXiv preprint arXiv:2106.07756_ (2021).
  * Vinyals et al. (2017) Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, et al. 2017\.  Starcraft ii: A new challenge for reinforcement learning.  _arXiv preprint arXiv:1708.04782_ (2017).
  * Wachter et al. (2017) Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017\.  Counterfactual explanations without opening the black box: Automated decisions and the GDPR.  _Harv. JL & Tech._ 31 (2017), 841.
  * Wang et al. (2016) Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016\.  Dueling network architectures for deep reinforcement learning. In _International conference on machine learning_. PMLR, 1995–2003.
  * Wells and Bednarz (2021) Lindsay Wells and Tomasz Bednarz. 2021.  Explainable ai and reinforcement learning—a systematic review of current approaches and trends.  _Frontiers in artificial intelligence_ 4 (2021), 550030.
  * Xiong et al. (2018) Jiechao Xiong, Qing Wang, Zhuoran Yang, Peng Sun, Lei Han, Yang Zheng, Haobo Fu, Tong Zhang, Ji Liu, and Han Liu. 2018.  Parametrized deep q-networks learning: Reinforcement learning with discrete-continuous hybrid action space.  _arXiv preprint arXiv:1810.06394_ (2018).
  * Zhang et al. (2020) Ke Zhang, Peidong Xu, and Jun Zhang. 2020.  Explainable AI in deep reinforcement learning models: A shap method applied in power system emergency control. In _2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2)_. IEEE, 711–716.
  * Zrihem et al. (2016) Nir Ben Zrihem, Tom Zahavy, and Shie Mannor. 2016\.  Visualizing dynamics: from t-sne to semi-mdps.  _arXiv preprint arXiv:1606.07112_ (2016).


# Generalisations of Singular Value Decomposition to dual-numbered matrices

Ran Gutin
Department of Computer Science
Imperial College London

###### Abstract

We present two generalisations of Singular Value Decomposition from real-
numbered matrices to dual-numbered matrices. We prove that every dual-numbered
matrix has both types of SVD. Both of our generalisations are motivated by
applications, either to geometry or to mechanics.

Keywords: singular value decomposition; linear algebra; dual numbers

##  1 Introduction

In this paper, we consider two possible generalisations of Singular Value
Decomposition (_$T$ -SVD_ and _$*$ -SVD_) to matrices over the ring of dual
numbers. We prove that both generalisations always exist. Both types of SVD
are motivated by applications.

A _dual number_ is a number of the form $a+b\epsilon$ where $a,b\in\mathbb{R}$
and $\epsilon^{2}=0$. The dual numbers form a commutative, assocative and
unital algebra over the real numbers. Let $\mathbb{D}$ denote the dual
numbers, and $M_{n}(\mathbb{D})$ denote the ring of $n\times n$ dual-numbered
matrices.

Dual numbers have applications in automatic differentiation [4], mechanics
(via screw theory, see [1]), computer graphics (via the _dual quaternion_
algebra [5]), and geometry [10]. Our paper is motivated by the applications in
geometry described in Section 1.1 and in mechanics given in Section 1.2.

Our main results are the existence of the _$T$ -SVD_ and the existence of the
_$*$ -SVD_. $T$-SVD is a generalisation of Singular Value Decomposition that
resembles that over real numbers, while $*$-SVD is a generalisation of SVD
that resembles that over complex numbers. (Note that in earlier editions of
this paper, we called the $T$-SVD the R-SVD, and the $*$-SVD the C-SVD). We
prove the following:

######  Theorem (Dual $T$-SVD).

Given a square dual number matrix $M\in M_{n}(\mathbb{D})$, we can decompose
the matrix as:

$M=U\Sigma V^{T}$

where $U^{T}U=V^{T}V=I$, and $\Sigma$ is a diagonal matrix.

######  Theorem (Dual $*$-SVD).

Every square dual-numbered matrix $M\in M_{n}(\mathbb{D})$ can be decomposed
as

$M=U\Sigma\overline{V}^{T}$

where $U\overline{U}^{T}=V\overline{V}^{T}=I$, and $\Sigma$ is a block-
diagonal matrix where each block is of one of the forms
$\begin{pmatrix}\sigma_{i}\end{pmatrix}$,
$\begin{pmatrix}\sigma_{i}&-\epsilon\sigma^{\prime}_{i}\\\
\epsilon\sigma_{i}^{\prime}&\sigma_{i}\end{pmatrix}$ or
$\begin{pmatrix}\epsilon\sigma_{i}^{\prime}\end{pmatrix}$, where each
$\sigma_{i}$ and $\sigma_{i}^{\prime}$ is real, and $\sigma_{i}^{\prime}\neq
0$.

###  1.1 Dual $*$-SVD

Our study of $*$-SVD is motivated by Yaglom’s 1968 book _Complex numbers in
geometry_ ([10]). Yaglom considers the group of Laguerre transformations,
which is analogous to the group of Moebius transformations over the complex
numbers. The Laguerre transformations are the group of functions of the form
$z\mapsto\frac{az+b}{cz+d}$ where $a,b,c,d$ are elements of $\mathbb{D}$, $z$
is a variable over $\mathbb{D}$, and $ad-bc$ is not a zero divisor. It can
easily be seen that every Laguerre transformation $z\mapsto\frac{az+b}{cz+d}$
can be represented as the $2\times 2$ matrix $\begin{pmatrix}a&b\\\
c&d\end{pmatrix}$.

Yaglom classifies the elements of this group in a geometric way. We restate
the classification in the language of matrices. Yaglom argues that every
invertible $2\times 2$ matrix over the dual numbers can be expressed in
exactly one of the following two ways:

  * •

$U\Sigma\overline{V}^{T}$ where $U\overline{U}^{T}=V\overline{V}^{T}=I$ and
$\Sigma$ is a diagonal matrix with real-valued entries.

  * •

$U\Sigma$ where $U\overline{U}^{T}=I$,
$\Sigma=\begin{pmatrix}\sigma&-\epsilon\sigma^{\prime}\\\
\epsilon\sigma^{\prime}&\sigma\end{pmatrix}$, and both $\sigma$ and
$\sigma^{\prime}$ are real (and $\sigma^{\prime}$ is non-zero).

The first of these forms resembles Singular Value Decomposition. What we
propose in this paper is a generalisation of Singular Value Decomposition to
square dual-numbered matrices which includes both forms as special cases.

###  1.2 Dual $T$-SVD

Our study of $T$-SVD is motivated by recent research [8, 2, 3, 7] in
mechanics, where the authors consider either a form of SVD that is essentially
$T$-SVD, or a form of Polar Decomposition that is essentially $T$-Polar
Decomposition. For completeness, we describe $T$-Polar Decomposition below.

######  Corollary (Dual $T$-Polar Decomposition).

Every square dual-numbered matrix $M$ can be expressed in the form $M=UP$
where $U^{T}U=I$ and $P$ is symmetric.

One of the main applications of the $T$-SVD is in finding the Moore-Penrose
generalised inverse of a dual matrix (whenever it exists). This has
applications in kinematic synthesis (see [7]).

Very recently, Udwadia et al. [9] studied the existence of the Moore-Penrose
generalised inverse for dual-numbered matrices and showed that unlike in the
case of real and complex matrices, not all dual-numbered matrices have such
inverses. In contrast to their result, we prove that the $T$-SVD does exist
for all dual-numbered matrices. Our result is new as none of the authors above
have proved that the $T$-SVD exists in general.

###  1.3 Structure of the paper

Section 2 contains preliminaries. In section 3, we will prove the
$*$/$T$-spectral theorems. In section 4, we will show that every square dual-
number matrix has a $*$/$T$-SVD.

##  2 Preliminaries

###  2.1 Dual numbers

The dual numbers are the ring $\mathbb{R}[\epsilon]/(\epsilon^{2})$. In other
words the dual numbers are pairs of real numbers, usually written as
$a+b\epsilon$, with the following operations defined on them:

  * •

$(a+b\epsilon)+(c+d\epsilon)=(a+c)+(b+d)\epsilon$.

  * •

$(a+b\epsilon)(c+d\epsilon)=ac+(ad+bc)\epsilon$.

  * •

$(a+b\epsilon)^{-1}=a^{-1}-\epsilon ba^{-2}$.

Given a polynomial with real coefficients $F(X)\in\mathbb{R}[X]$, evaluating
$F$ on a dual number gives $F(a+b\epsilon)=F(a)+\epsilon bF^{\prime}(a)$ where
$F^{\prime}$ denotes the derivative of $F$. Multiples of the dual number
$\epsilon$ are sometimes referred to as “infinitesimal” with the intuition
being that $\epsilon$ is “so small” that it squares to $0$.

The dual numbers contrast with the complex numbers. The complex numbers are
defined as the ring $\mathbb{R}[i]/(i^{2}+1)$. Like the dual numbers, the
complex numbers are pairs of real numbers with certain operations defined on
them. The difference is that while the complex numbers are defined by
adjoining an element $i$ such that $i^{2}=-1$, the dual numbers are defined by
adjoining an element $\epsilon$ such that $\epsilon^{2}=0$.

Unlike the complex numbers, the dual numbers do not form a field. As such,
instead of talking about vector spaces over dual numbers, one must talk about
_modules_ over dual numbers. Modules over dual numbers don’t necessarily have
bases: For instance, the module $\epsilon\mathbb{D}$ (that is, multiples of
the dual number $\epsilon$) doesn’t have a basis. For this reason, one cannot
in general talk about the “rank” of a matrix over the dual numbers. A module
over the dual numbers (or generally any ring) that does have a basis is
referred to as a _free_ module.

###  2.2 Spectral theorem over symmetric real matrices

The spectral theorem over symmetric matrices states that a symmetric matrix
$A$ over the real numbers can be orthogonally diagonalised. In other words, if
$A$ is a real matrix such that $A=A^{T}$, then there exists a matrix $P$ such
that $P^{T}P=I$ and $A=PDP^{T}$ for some diagonal matrix $D$.

###  2.3 Spectral theorem over skew-symmetric real matrices

The spectral theorem over skew-symmetric matrices states that a skew-symmetric
matrix $A$ over the real number can be orthogonally block-diagonalized, where
every block is a skew-symmetric $2\times 2$ block except possibly for one
block, which has dimensions $1\times 1$ and whose only entry equals $0$.

###  2.4 Notation and terms

By $\overline{a+b\epsilon}$, we mean $a-b\epsilon$. Given a dual number
$a+b\epsilon$, we call $a$ the _standard part_ and $b$ the _infinitesimal_
part. We sometimes denote the standard and infinitesimal parts of a dual
number $z$ by $\operatorname{st}(z)$ and $\Im(z)$ respectively. We also use
the term _infinitesimal_ to describe a dual number whose standard part is
zero, and _appreciable_ to describe a dual number whose standard part is non-
zero. We sometimes call a dual number whose infinitesimal part is zero a real
number. All the above terms generalise to matrices and vectors over dual
numbers in the obvious way. Likewise, the operations _infinitesimal part_ and
_standard part_ generalise to sets of dual-numbered vectors by applying these
operations to each element of the set.

We sometimes write $M^{*}$ for $\overline{M}^{T}$.

We use $(u,v)$ to denote $u^{T}v$, and $\langle u,v\rangle$ to denote
$u^{T}\overline{v}$. Two vectors are considered _$T$ -orthogonal_ if
$(u,v)=0$, and _$*$ -orthogonal_ if $\langle u,v\rangle=0$.

A dual matrix $U$ which satisfies $U\bar{U}^{T}=\bar{U}^{T}U=I$ is called
_unitary_. Similarly, one which satisfies $U^{T}U=UU^{T}=I$ is called _$T$
-orthogonal_.

A dual matrix $A$ such that $A=\bar{A}^{T}$ is called _Hermitian_. Similarly,
one which satisfies $A=A^{T}$ is called _symmetric_.

A vector $v$ is called an _eigenvector_ of a dual-numbered matrix $A$ if $v$
is appreciable and $Av=\lambda v$ for some $\lambda\in\mathbb{D}$.

##  3 Dual-number $*$/$T$-spectral theorem

######  Theorem (Dual $*$-spectral).

Given a Hermitian dual number matrix $M\in M_{n}(\mathbb{D})$, we can
decompose the matrix as:

$M=V\Sigma V^{*}$

where $V$ is unitary, and $\Sigma$ is a block-diagonal matrix where each block
is either of the form:

  * •

$\begin{pmatrix}\sigma_{i}\end{pmatrix}$,

  * •

or $\begin{pmatrix}\sigma_{i}&-\epsilon\sigma^{\prime}_{i}\\\
\epsilon\sigma_{i}^{\prime}&\sigma_{i}\end{pmatrix}$

where each $\sigma_{i}$ and $\sigma_{i}^{\prime}$ is real, and
$\sigma^{\prime}_{i}\neq 0$.

###### Proof.

Let $M$ be a Hermitian matrix. We find a real matrix $S$ such that
$S\operatorname{st}(A)S^{T}$ is diagonal. We let $M^{\prime}=SMS^{T}$, which
we write as a block matrix

$\begin{pmatrix}\lambda_{1}I+\epsilon B_{11}&\epsilon B_{12}&\dotsb&\epsilon
B_{1n}\\\ -\epsilon B^{T}_{12}&\lambda_{2}I+\epsilon B_{22}&\ddots&\vdots\\\
\vdots&\ddots&\ddots&\epsilon B_{n-1,n}\\\ -\epsilon
B^{T}_{1n}&\dotsb&-\epsilon B_{n-1,n}^{T}&\lambda_{n}I+\epsilon
B_{nn}\end{pmatrix}.$

where each $B_{ii}$ is skew-symmetric. We let

$P=\begin{pmatrix}I&\frac{\epsilon
B_{12}}{\lambda_{1}-\lambda_{2}}&\dotsb&\frac{\epsilon
B_{1n}}{\lambda_{1}-\lambda_{n}}\\\ \frac{\epsilon
B_{12}^{T}}{\lambda_{1}-\lambda_{2}}&I&\ddots&\vdots\\\
\vdots&\ddots&\ddots&\frac{\epsilon B_{n-1,n}}{\lambda_{n-1}-\lambda_{n}}\\\
\frac{\epsilon B_{1n}^{T}}{\lambda_{1}-\lambda_{n}}&\dotsb&\frac{\epsilon
B_{n-1,n}^{T}}{\lambda_{n-1}-\lambda_{n}}&I\end{pmatrix},$

and let $M^{\prime\prime}=PM^{\prime}P^{*}$. We end up with $M^{\prime\prime}$
being equal to a direct sum of matrices:
$M^{\prime\prime}=(\lambda_{1}I+\epsilon B_{11})\oplus(\lambda_{2}I+\epsilon
B_{22})\oplus\dotsb\oplus(\lambda_{n}I+\epsilon B_{nn})$. We finally use the
spectral theorem for skew-symmetric matrices to find matrices $Q_{i}$ such
that $Q_{i}B_{ii}Q_{i}^{T}$ is equal to a direct sum of skew-symmetric
$2\times 2$ blocks except for potentially one zero block. We thus get that
$(Q_{1}\oplus Q_{2}\oplus\dotsb\oplus Q_{n})M^{\prime\prime}(Q_{1}\oplus
Q_{2}\oplus\dotsb\oplus Q_{n})^{T}$ is a block-diagonal matrix in the desired
form. ∎

######  Theorem (Dual $T$-spectral).

Every symmetric matrix can be orthogonally diagonalised.

###### Proof.

The proof is the same as for the $*$-spectral case except that

$M^{\prime}=\begin{pmatrix}\lambda_{1}I+\epsilon B_{11}&\epsilon
B_{12}&\dotsb&\epsilon B_{1n}\\\ \epsilon B^{T}_{12}&\lambda_{2}I+\epsilon
B_{22}&\ddots&\vdots\\\ \vdots&\ddots&\ddots&\epsilon B_{n-1,n}\\\ \epsilon
B^{T}_{1n}&\dotsb&\epsilon B_{n-1,n}^{T}&\lambda_{n}I+\epsilon
B_{nn}\end{pmatrix},$

and

$P=\begin{pmatrix}I&\frac{\epsilon
B_{12}}{\lambda_{1}-\lambda_{2}}&\dotsb&\frac{\epsilon
B_{1n}}{\lambda_{1}-\lambda_{n}}\\\ -\frac{\epsilon
B_{12}^{T}}{\lambda_{1}-\lambda_{2}}&I&\ddots&\vdots\\\
\vdots&\ddots&\ddots&\frac{\epsilon B_{n-1,n}}{\lambda_{n-1}-\lambda_{n}}\\\
-\frac{\epsilon B_{1n}^{T}}{\lambda_{1}-\lambda_{n}}&\dotsb&-\frac{\epsilon
B_{n-1,n}^{T}}{\lambda_{n-1}-\lambda_{n}}&I\end{pmatrix},$

and $M^{\prime\prime}=PMP^{T}$. And instead of using the spectral theorem for
skew-symmetric matrices, we use the one for symmetric matrices. ∎

The following theorem can be used to show that the $*$/$T$-spectral
decompositions are unique. Note that in order to show that the $*$-spectral
decomposition is unique, one must first fully diagonalise the matrix
$\begin{pmatrix}\sigma&-\epsilon\sigma^{\prime}\\\
\epsilon\sigma^{\prime}&\sigma\end{pmatrix}$ to
$\begin{pmatrix}\sigma+i\epsilon\sigma^{\prime}&0\\\
0&\sigma-i\epsilon\sigma^{\prime}\end{pmatrix}$ by working over
$\mathbb{C}\otimes\mathbb{D}$.

######  Theorem (Uniqueness of eigenvalues).

An eigenbasis of a linear endomorphism $T:\mathbb{D}^{n}\to\mathbb{D}^{n}$
corresponds to a unique multi-set of eigenvalues.

###### Proof.

Let $e_{1},e_{2},\dotsc,e_{n}$ be a basis made up of eigenvectors of $T$. Let
the corresponding eigenvalues be $\lambda_{1},\lambda_{2},\dotsc,\lambda_{n}$
respectively.

We intend to show that for any eigenvalue $\lambda$ of $T$, the number of
“$e_{i}$”s in our basis with eigenvalue equal to $\lambda$ is equal to
$\dim(\operatorname{st}(V_{\lambda}))$. Since the quantity
$\dim(\operatorname{st}(V_{\lambda}))$ depends only on $T$, it implies that
every eigenbasis has the same amount of eigenvectors with eigenvalue equal to
$\lambda$.

Assume that $e_{1},e_{2},\dotsc,e_{k}$ all have some eigenvalue $\lambda$, and
no other $e_{i}$ has eigenvalue $\lambda$. We will show that
$k=\dim(\operatorname{st}(V_{\lambda}))$ by showing that
$\operatorname{st}(e_{1}),\operatorname{st}(e_{2}),\dotsc,\operatorname{st}(e_{k})$
form a basis of $\operatorname{st}(V_{\lambda})$.

#### Claim 1.

$\operatorname{st}(e_{1}),\operatorname{st}(e_{2}),\dotsc,\operatorname{st}(e_{k})$
form a spanning set of $\operatorname{st}(V_{\lambda})$ (the standard part of
the eigenspace corresponding to $\lambda$).

_Proof of claim 1._ Given a vector in $\operatorname{st}(V_{\lambda})$, we
wish to express it as a linear combination of the above vectors. Our vector is
equal to $\operatorname{st}(e)$ for some $e\in V_{\lambda}$. We can express
$e$ as $e=\sum_{i=1}^{n}\alpha_{i}e_{i}$. From this we can conclude that
$\operatorname{st}(e)=\sum_{i=1}^{n}\operatorname{st}(\alpha_{i})\operatorname{st}(e_{i})$.
But notice that this sum includes vectors $\operatorname{st}(e_{k+1})$ to
$\operatorname{st}(e_{n})$, which we don’t want in our linear combination. We
thus need to show that $\operatorname{st}(\alpha_{i})=0$ for all $i>k$. To
this end, notice that $T(e)=\lambda
e=\lambda\left(\sum_{i=1}^{n}\alpha_{i}e_{i}\right)$ and
$T(e)=\sum_{i=1}^{n}\alpha_{i}T(e_{i})=\sum_{i=1}^{n}\alpha_{i}\lambda_{i}e_{i}$.
Since we know that the $e_{i}$ are linearly independent, we have that
$\alpha_{i}(\lambda_{i}-\lambda)=0$ for all $i$. From
$\alpha_{i}(\lambda_{i}-\lambda)=0$, one can check that if
$\lambda\neq\lambda_{i}$ then $\alpha_{i}$ is infinitesimal. Since for all
$i>k$, we have that $\lambda_{i}\neq\lambda$, we must have that $\alpha_{i}$
is infinitesimal for $i>k$. Recalling that
$e=\sum_{i=1}^{k}\alpha_{i}e_{i}+\sum_{i=k+1}^{n}\alpha_{i}e_{i}$, taking
standard parts of both sides gives
$\operatorname{st}(e)=\sum_{i=1}^{k}\operatorname{st}(\alpha_{i})\operatorname{st}(e_{i})+0$,
which is what we sought to show.

#### Claim 2.

$\operatorname{st}(e_{1}),\operatorname{st}(e_{2}),\dotsc,\operatorname{st}(e_{k})$
forms a linearly independent set.

_Proof of claim 2._ Assume that there exist $(\alpha_{i})$ (where each
$\alpha_{i}$ is real) such that
$\sum_{i=1}^{k}\alpha_{i}\operatorname{st}(e_{i})=0$. We then have that
$\operatorname{st}\left(\sum_{i=1}^{k}\alpha_{i}e_{i}\right)=0$. If
$\alpha_{j}\neq 0$ for some $j$ then we get that
$\sum_{i=1}^{k}\epsilon\alpha_{i}e_{i}=0$, which contradicts the linear
independence of $e_{1},e_{2},\dotsc,e_{n}$. Therefore all $\alpha_{i}$ equal
$0$. ∎

##  4 Dual-number $*$/$T$-Singular Value Decomposition

These are the theorems we will prove in this section.

######  Theorem (Dual $*$-SVD).

Given a square dual number matrix $M\in M_{n}(\mathbb{D})$, we can decompose
the matrix as:

$M=U\Sigma V^{*}$

where $U$ and $V$ are unitary, and $\Sigma$ is a block-diagonal matrix where
each block is either of the form $\begin{pmatrix}\sigma_{i}\end{pmatrix}$,
$\begin{pmatrix}\sigma_{i}&-\epsilon\sigma^{\prime}_{i}\\\
\epsilon\sigma_{i}^{\prime}&\sigma_{i}\end{pmatrix}$ or
$\begin{pmatrix}\epsilon\sigma_{i}^{\prime}\end{pmatrix}$, where each
$\sigma_{i}$ and $\sigma_{i}^{\prime}$ is real, and $\sigma^{\prime}_{i}\neq
0$.

######  Theorem (Dual $T$-SVD).

Given a square dual number matrix $M\in M_{n}(\mathbb{D})$, we can decompose
the matrix as:

$M=U\Sigma V^{T}$

where $U$ and $V$ are $T$-orthogonal, and $\Sigma$ is a diagonal matrix.

We will start by proving these theorems for invertible matrices.

######  Theorem (SVD for invertible matrices).

Every invertible matrix has a $T$/$*$-SVD.

###### Proof.

We shall prove this for the $T$-SVD, but the argument is the same for the
$*$-SVD.

Let $M$ be an arbitrary dual matrix. Observe that $M^{T}M$ is a symmetric
matrix. As such, by the $T$-spectral theorem, we have that $M^{T}M=V\Sigma
V^{T}$ for some orthogonal matrix $V$ and diagonal $\Sigma$. Also observe that
the standard part of $M^{T}M$ is positive-definite. From this, it follows that
the standard part of $\Sigma$ is positive. It follows that $\sqrt{M^{T}M}$
exists and is equal to $V\sqrt{\Sigma}V^{T}$. Observe also that
$M(\sqrt{M^{T}M})^{-1}$ is an orthogonal matrix which we shall call $U$.
Finally, we have that $(UV)\sqrt{\Sigma}V^{T}$ is the $T$-SVD of $M$. ∎

######  Theorem (Dual $*$-SVD).

Given a square dual number matrix $M\in M_{n}(\mathbb{D})$, we can decompose
the matrix as:

$M=U\Sigma W^{*}$

where $U$ and $W$ are unitary, and $\Sigma$ is a block-diagonal matrix where
each block is either of the form $\begin{pmatrix}\sigma_{i}\end{pmatrix}$,
$\begin{pmatrix}\sigma_{i}&-\epsilon\sigma^{\prime}_{i}\\\
\epsilon\sigma_{i}^{\prime}&\sigma_{i}\end{pmatrix}$ or
$\begin{pmatrix}\epsilon\sigma_{i}^{\prime}\end{pmatrix}$, and each
$\sigma_{i}$ and $\sigma_{i}^{\prime}$ is real.

###### Proof.

Let $T:V\to V$ be a linear endomorphism over a finite-dimensional free
$\mathbb{D}$-module.

The operator $T^{*}T$ is Hermitian. Thus, apply the $*$-spectral theorem to it
to get an orthonormal basis $B=\\{b_{1},b_{2},\dotsc,b_{n}\\}$. Let $V^{L}$ be
the span of the set of vectors in $B$ that get mapped to appreciable vectors
by $T$. Let $V^{R}$ be the span of the set of vectors in $B$ that get mapped
to infinitesimal vectors by $T$.111It so happens that $V^{L}$ is equal to the
image of $T^{*}T$, and that $V^{R}$ is equal to both $\ker(T^{*}T)$ and
$\ker(T^{t}T)$ (where $T^{t}$ denotes the tranpose of $T$). These are easy to
prove, and we won’t do so here.

Observe that $T$ is injective over $V^{L}$. Let $I$ be the image of $V^{L}$
under $T$. Observe that $I$ has the same dimension as $V^{L}$. Thus, there
exists a unitary operator $U$ such that $U(I)=V^{L}$. Clearly,
$UT(V^{L})=V^{L}$. Observe that $UT(V^{R})\subseteq V^{R}$.

Since $UT|_{V^{L}}$ is a linear endomorphism, we may take its $*$-SVD by
theorem _Dual $*$-SVD for invertible matrices_. We also observe that
$UT|_{V^{R}}$ is an infinitesimal map (meaning that it maps every argument to
infinitesimals), and can therefore be expressed as $\epsilon T^{\prime}$. We
can thus take the SVD of $T^{\prime}$. Taking the direct sum of the $*$-SVDs
of $UT|_{V^{L}}$ and $UT|_{V^{R}}$ yields the $*$-SVD of $UT$.

The block types $\begin{pmatrix}\sigma_{i}\end{pmatrix}$ and
$\begin{pmatrix}\sigma_{i}&-\epsilon\sigma^{\prime}_{i}\\\
\epsilon\sigma_{i}^{\prime}&\sigma_{i}\end{pmatrix}$ come from $UT|_{V^{L}}$,
and the block type $\begin{pmatrix}\epsilon\sigma_{i}^{\prime}\end{pmatrix}$
comes from $UT|_{V^{R}}$.

Finally, we multiply the SVD of $UT$ on the left by $U^{-1}$, and we are done.
∎

######  Lemma 1.

The following claims made in the above proof are true:

  1. 1.

The Gram-Schmidt process is a valid algorithm for producing an orthonormal
basis for some free $\mathbb{D}$-module $W$ given a basis for $W$.

  2. 2.

$T$ is injective over $V^{L}$.

  3. 3.

There exists a unitary map $U$ such that $UT(V^{L})=V^{L}$.

  4. 4.

$UT(V^{L})\perp UT(V^{R})$.

  5. 5.

$UT(V^{R})\subseteq V^{R}$.

###### Proof.

We proceed to prove each claim.

_Proof of claim 1._ The proof is by induction. Assume that $\dim(W)=0$ and the
basis is $\emptyset$. Then Gram-Schmidt finishes immediately and produces the
orthonormal basis $\emptyset$. Now assume that given a set of vectors
$\\{v_{1},v_{2},\dotsc,v_{n},v_{n+1}\\}$, the first $n$ iterations of Gram-
Schmidt succeed in producing an orthonormal set of vectors
$\\{e_{1},e_{2},\dotsc,e_{n}\\}$ whose span is equal to the span of
$\\{v_{1},v_{2},\dotsc,v_{n}\\}$. We perform another iteration and get
$e_{n+1}=\frac{v_{n+1}-\sum_{i=1}^{n}\langle e_{i},v_{n+1}\rangle
e_{i}}{\left|v_{n+1}-\sum_{i=1}^{n}\langle e_{i},v_{n+1}\rangle e_{i}\right|}$
where $|u|$ means $|\operatorname{st}(u)|$. We need to check that the division
is sound. Assuming it isn’t, we would have that
$\left|v_{n+1}-\sum_{i=1}^{n}\langle e_{i},v_{n+1}\rangle e_{i}\right|=0$,
which implies that $\epsilon\left(v_{n+1}-\sum_{i=1}^{n}\langle
e_{i},v_{n+1}\rangle e_{i}\right)=0$, which contradicts the linear
independence of $\\{v_{1},v_{2},\dotsc,v_{n},v_{n+1}\\}$. The linear
independence, spanning property and orthonormality of
$\\{e_{1},e_{2},\dotsc,e_{n},e_{n+1}\\}$ are easily shown.

_Proof of claim 2._ It’s easily seen that $T^{*}T$ is injective over $V^{L}$
(based on the definition of $V^{L}$). So then assume that $T(u)=T(v)$ for some
$u$ and $v$ in $V^{L}$. We then have that $T^{*}T(u)=T^{*}T(v)$. By
injectivity of $T^{*}T$, we have that $u=v$.

_Proof of claim 3._ We begin by constructing an orthonormal basis for
$T(V^{L})$. Clearly, given an orthonormal basis
$\\{b_{1},b_{2},\dotsc,b_{k}\\}$ of $V^{L}$, we get a basis
$\\{T(b_{1}),T(b_{2}),\dotsc,T(b_{k})\\}$ of $T(V^{L})$. But this latter basis
may not be orthonormal. We perform Gram-Schmidt on this latter basis to get an
orthonormal basis $\\{c_{1},c_{2},\dotsc,c_{k}\\}$ of $T(V^{L})$. We extend
the orthonormal basis $\\{b_{1},b_{2},\dotsc,b_{k}\\}$ of $V^{L}$ to an
orthonormal basis $\\{b_{1},b_{2},\dotsc,b_{k},b_{k+1},\dotsc,b_{n}\\}$ of the
whole space, and the orthonormal basis $\\{c_{1},c_{2},\dotsc,c_{k}\\}$ of
$T(V^{L})$ to an orthonormal basis
$\\{c_{1},c_{2},\dotsc,c_{k},c_{k+1},\dotsc,c_{n}\\}$ of the whole space. We
leave it to the reader to verify that this extension is possible. We now
define $U(c_{i})=b_{i}$ for all $i$, and observe that this $U$ is unitary, and
maps $T(V^{L})$ to $V^{L}$.

_Proof of claim 4._ We take the inner product of the spaces $UT(V^{L})$ and
$UT(V^{R})$. We get $\langle UT(V^{L}),UT(V^{R})\rangle=\langle
T(V^{L}),T(V^{R})\rangle=\langle T^{*}T(V^{L}),V^{R}\rangle=\langle
V^{L},V^{R}\rangle=0$. We’ve used the fact that $T^{*}T(V^{L})=V^{L}$, which
should be easy to verify.

_Proof of claim 5._ Let $\\{b_{1},b_{2},\dotsc,b_{k}\\}$ be basis of $V^{L}$,
and $\\{b_{k+1},b_{k+2},\dotsc,b_{n}\\}$ be a basis of $V^{R}$. Their union is
a basis for the whole space. Let $v\in UT(V^{R})$. Express $v$ as
$\sum_{i=1}^{n}\lambda_{i}b_{i}$. Since $UT(V^{R})\perp V^{L}$, we have that
for $i\leq k$, $\lambda_{i}=0$. It follows that $v$ is in the span of
$\\{b_{1},b_{2},\dotsc,b_{k}\\}$, and therefore that $v$ is in $V^{R}$. ∎

######  Theorem (Dual $T$-SVD).

Given a square dual number matrix $M\in M_{n}(\mathbb{D})$, we can decompose
the matrix as:

$M=U\Sigma V^{T}$

where $U$ and $V$ are $T$-orthogonal, and $\Sigma$ is a diagonal matrix.

###### Proof.

Essentially the same as for the $*$-SVD, except that the role of the
$*$-spectral theorem is changed to the $T$-spectral theorem. ∎

##  5 Acknowledgements

I am thankful to Gregory Gutin and Ilya Spitkovsky for helpful comments and
suggestions. The software Sympy v1.6 ([6]) was helpful for finding some of the
proofs.

## References

  * [1] Ian S. Fischer.  Dual-Number Methods in Kinematics, Statics and Dynamics.  CRC Press, 1998.
  * [2] Shilei Han and Olivier A. Bauchau.  On the global interpolation of motion.  Computer Methods in Applied Mechanics and Engineering, 337:352 – 386, 2018.
  * [3] Shilei Han and Olivier A. Bauchau.  Spectral collocation methods for the periodic solution of flexible multibody dynamics.  Nonlinear Dynamics, 92:1599–1618, 2018.
  * [4] Philipp H. Hoffmann.  A hitchhiker’s guide to automatic differentiation.  Numer. Algorithms, 72(3):775–811, July 2016.
  * [5] Ben Kenwright.  A beginners guide to dual-quaternions: What they are, how they work, and how to use them for 3d.  In Character Hierarchies”, The 20th International Conference on Computer Graphics, Visualization and Computer Vision, WSCG 2012 Communication Proceedings,pp.1-13, 2012.
  * [6] Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rathnayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Štěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz.  Sympy: symbolic computing in python.  PeerJ Computer Science, 3:e103, January 2017.
  * [7] E. Pennestri, P. P. Valentini, and D. de Falco.  The Moore-Penrose Dual Generalized Inverse Matrix With Application to Kinematic Synthesis of Spatial Linkages.  Journal of Mechanical Design, 140(10), 07 2018.  102303\.
  * [8] E. Pennestrì and R. Stefanelli.  Linear algebra and numerical algorithms using dual numbers.  Multibody System Dynamics, 18:323–344, Oct 2007.
  * [9] Firdaus Udwadia, Ettore Pennestri, and Domenico de Falco.  Do all dual matrices have dual moore-penrose generalized inverses?  Mechanism and Machine Theory, 151, Sept 2020.
  * [10] I.M. Yaglom.  Complex Numbers in Geometry.  Academic Press, 1968.