# Geodesic orbit metrics in a class of homogeneous bundles over quaternionic
Stiefel manifolds

Andreas Arvanitoyeorgos, Nikolaos Panagiotis Souris and Marina Statha
University of Patras, Department of Mathematics, GR-26500 Rion, Greece
<EMAIL_ADDRESS>University of Patras, Department of Mathematics,
GR-26500 Rion, Greece<EMAIL_ADDRESS>University of Patras, Department of
Mathematics, GR-26500 Rion, Greece<EMAIL_ADDRESS>
###### Abstract.

Geodesic orbit spaces (or g.o. spaces) are defined as those homogeneous
Riemannian spaces $(M=G/H,g)$ whose geodesics are orbits of one-parameter
subgroups of $G$. The corresponding metric $g$ is called a geodesic orbit
metric. We study the geodesic orbit spaces of the form
$(\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s}),g)$,
with $0<n_{1}+\cdots+n_{s}\leq n$. Such spaces include spheres, quaternionic
Stiefel manifolds, Grassmann manifolds and quaternionic flag manifolds. The
present work is a contribution to the study of g.o. spaces $(G/H,g)$ with $H$
semisimple.

2020 Mathematics Subject Classification. Primary 53C25; Secondary 53C30.

Keywords: Homogeneous geodesic; geodesic vector; geodesic orbit space;
isotropy representation; generalized quaternionic flag manifold; quaternionic
Stiefel manifold

##  1\. Introduction

Geodesic orbit spaces $(M=G/H,g)$ are defined by the simple property that any
geodesic $\gamma$ has the form

$\gamma(t)=\exp(tX)\cdot o,$

where $\exp$ is the exponential map on $G$, $o=\gamma(0)$ is a point in $M$
and $\cdot$ denotes the action of $G$ on $M$. These spaces were initially
considered in [14] and up to today they have been extensively studied within
various geometric contexts, including the Riemannian ([13]), pseudo-Riemannian
([10]), Finsler ([24]) and affine ([12]) context. The classification of g.o.
spaces remains an open problem.

There are diverse examples of g.o. spaces, including the classes of symmetric
spaces, weakly symmetric spaces ([9], [23]), isotropy irreducible spaces
([22]), $\delta$-homogeneous spaces ([6]), Clifford-Wolf homogeneous spaces
([7]) and _naturally reductive spaces_. Reviews about g.o. spaces up to 2017
can be found in [5] and in the introduction of [15]. Various results up to
2020 are included in the recently published book [8].

Determining the g.o. metrics among the $G$-invariant metrics on a space $G/H$
presents some challenges. The main challenge lies in the fact that the space
of $G$-invariant metrics may have complicated structure, depending on whether
the _isotropy representation_ of $H$ on the tangent space $T_{o}(G/H)$
contains pairwise equivalent submodules. To remedy this obstruction, various
simplification results for g.o. metrics have been established (e.g. [15],
[19]). A general observation is that the existence and the form of the g.o.
metrics on $G/H$ depends to a large extent on the structure of the tangent
space $T_{o}(G/H)$ induced from the isotropy representation and on the Lie
algebraic relations between the corresponding submodules (e.g. [11]).

When $G$ is compact semisimple, the classification of the g.o. spaces
$(G/H,g)$ with $H$ abelian and $H$ simple has been obtained in the works [20]
and [11] respectively. On the other hand, the classification of compact g.o.
spaces $(G/H,g)$ with $H$ semisimple remains open, while no general results
are known for this case. The present paper is a continuation of our work [4]
towards a study of g.o. metrics on a general family of spaces $G/H$, such that
the isotropy representation of all of its members has a similar description.
In that work we studied the geodesic orbit metrics on the spaces
$\operatorname{SO}(n)/\operatorname{SO}(n_{1})\times\cdots\times\operatorname{SO}(n_{s})$
and
$\operatorname{U}(n)/\operatorname{U}(n_{1})\times\cdots\times\operatorname{U}(n_{s})$
with $0<n_{1}+\cdots+n_{s}\leq n$. These are spaces $G/H$ where $G$ is a
compact classical Lie group and $H$ is a diagonally embedded product of Lie
groups of the same type as $G$.

In the present paper we study the geodesic orbit metrics on the spaces
$\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$
with $0<n_{1}+\cdots+n_{s}\leq n$. These spaces properly include the spheres
$\mathbb{S}^{4n-1}=\operatorname{Sp}(n)/\operatorname{Sp}(n-1)$, the
quaternionic Stiefel manifolds $\operatorname{Sp}(n)/\operatorname{Sp}(n-k)$,
the Grassmann manifolds
$\operatorname{Sp}(n)/\operatorname{Sp}(k)\times\operatorname{Sp}(n-k)$, and
the quaternionic flag manifolds
$\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$,
$n_{1}+\cdots+n_{s}=n$ ([18]). If $n_{1}+\cdots+n_{s}<n$, each of these spaces
can be viewed as a total space over a quaternionic Stiefel manifold, with
fiber a quaternionic flag manifold, i.e.

$\operatorname{Sp}(m)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})\rightarrow\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})\rightarrow\operatorname{Sp}(n)/\operatorname{Sp}(m),$

with $m=n_{1}+\cdots+n_{s}$.

Our main result is the following.

######  Theorem 1.1.

Let $G/H$ be the space $Sp(n)/Sp(n_{1})\times\cdots\times Sp(n_{s})$, where
$0<n_{1}+\cdots+n_{s}\leq n$. If $G/H\neq Sp(n)/Sp(n-1)$ (i.e. if
$n-(n_{1}+\cdots+n_{s})\neq 1$ or $s>1$) then a $G$-invariant Riemannian
metric on $G/H$ is geodesic orbit if and only if it is the standard metric
induced from the Killing form on the Lie algebra $\mathfrak{sp}(n)$ of
$Sp(n)$.

If $G/H=Sp(n)/Sp(n-1)$ (i.e. $s=1$ and $n-n_{1}=1$) then a $G$-invariant
metric $g$ on $G/H$ is geodesic orbit if and only if $g=g_{\mu}$, $\mu>0$,
where $g_{\mu}$ denotes a one-parameter family of deformations of the standard
metric $g_{1}$, along the fiber $Sp(1)$ of the fibration
$Sp(n)/Sp(n-1)\rightarrow Sp(n)/Sp(1)\times Sp(n-1)$.

We remark that the non standard geodesic orbit metric $g_{\mu}$ appears in
[16] and [21].

The paper is structured as follows: In Section 2 we present preliminary facts
about homogeneous spaces and some special properties of the isotropy
representation, to be used in our study. In Section 3 we state and prove some
useful properties of g.o. spaces. In Section 4, we describe the isotropy
representation of the spaces $G/H=Sp(n)/Sp(n_{1})\times\cdots\times Sp(n_{s})$
and the structure of the tangent space $T_{o}(G/H)$ induced from the isotropy
representation. In Section 5 we state and prove some preliminary propositions
required for the proof of Theorem 1.1. Finally, in Section 6 we prove Theorem
1.1.

Acknowledgement. This research is co-financed by Greece and the European Union
(European Social Fund- ESF) through the Operational Programme “Human Resources
Development, Education and Lifelong Learning 2014-2020” in the context of the
project “Geodesic orbit metrics on homogeneous spaces of classical Lie groups”
(MIS 5047124).

##  2\. Preliminaries

###  2.1. Invariant metrics on homogeneous spaces

Let $G/H$ be a homogeneous space with origin $o=eH$ and assume that $G$ is
compact. Let $\mathfrak{g},\mathfrak{h}$ be the Lie algebras of $G,H$
respectively. Moreover, let
$\operatorname{Ad}:G\rightarrow\operatorname{Aut}(\mathfrak{g})$ and
$\operatorname{ad}:\mathfrak{g}\rightarrow\operatorname{End}(\mathfrak{g})$ be
the adjoint representations of $G$ and $\mathfrak{g}$ respectively, where
$\operatorname{ad}(X)Y=[X,Y]$. Since $G$ is compact, there exists an
$\operatorname{Ad}$-invariant (and hence $\operatorname{ad}$ skew-symmetric)
inner product $B$ on $\mathfrak{g}$, which we henceforth fix. In turn, we have
a $B$-orthogonal _reductive decomposition_

$\mathfrak{g}=\mathfrak{h}\oplus\mathfrak{m},$ (1)

where the subspace $\mathfrak{m}$ is $\operatorname{Ad}(H)$-invariant (and
$\operatorname{ad}(\mathfrak{h})$-invariant) and is naturally identified with
the tangent space of $G/H$ at the origin.

A Riemannian metric $g$ on $G/H$ is called $G$-invariant if for any $x\in G$
the left translations $\tau_{x}:G/H\rightarrow G/H$, $pH\mapsto(xp)H$, are
isometries of $(G/H,g)$. The $G$-invariant metrics are in one to one
correspondence with $\operatorname{Ad}(H)$-invariant inner products $\langle\
,\ \rangle$ on $\mathfrak{m}$. Moreover, any such product corresponds to a
unique endomorphism $A:\mathfrak{m}\rightarrow\mathfrak{m}$, called the
_corresponding metric endomorphism_ , that satisfies

$\langle X,Y\rangle=B(AX,Y)\ \ \makebox{for all}\ \ X,Y\in\mathfrak{m}.$ (2)

It follows from Equation (2) that the metric endomorphism $A$ is symmetric
with respect $B$, positive definite and $\operatorname{Ad}(H)$-equivariant,
that is $(\operatorname{Ad}(h)\circ A)(X)=(A\circ\operatorname{Ad}(h))(X)$ for
all $h\in H$ and $X\in\mathfrak{m}$. Conversely, any endomorphism on
$\mathfrak{m}$ with the above properties determines a unique $G$-invariant
metric on $G/H$.

Since $A$ is diagonalizable, there exists a decomposition
$\mathfrak{m}=\bigoplus_{j=1}^{l}\mathfrak{m}_{\lambda_{j}}$ into eigenspaces
$\mathfrak{m}_{\lambda_{j}}$ of $A$, corresponding to distinct eigenvalues
$\lambda_{j}$. Each eigenspace $\mathfrak{m}_{\lambda_{j}}$ is
$\operatorname{Ad}(H)$-invariant. When an $\operatorname{Ad}$-invariant inner
product $B$ and a $B$-orthogonal reductive decomposition (1) have been fixed,
we will make no distinction between a $G$-invariant metric $g$ and its
corresponding metric endomorphism $A$.

The form of the $G$-invariant metrics on $G/H$ depends on the _isotropy
representation_
$\operatorname{Ad}^{G/H}:H\rightarrow\operatorname{Gl}(\mathfrak{m})$, defined
by $\operatorname{Ad}^{G/H}(h)X:=(d\tau_{h})_{o}(X)$, $h\in H$,
$X\in\mathfrak{m}$. We consider a $B$-orthogonal decomposition

$\mathfrak{m}=\mathfrak{m}_{1}\oplus\cdots\oplus\mathfrak{m}_{s},$ (3)

into $\operatorname{Ad}^{G/H}$-invariant and irreducible submodules. We recall
that two submodules $\mathfrak{m}_{i}$ and $\mathfrak{m}_{j}$ are equivalent
if there exists an $\operatorname{Ad}^{G/H}$-equivariant isomorphism
$\phi:\mathfrak{m}_{i}\rightarrow\mathfrak{m}_{j}$. The simplest case occurs
when all the submodules $\mathfrak{m}_{i}$ are pairwise inequivalent. Then any
$G$-invariant metric $A$ on $G/H$ has a diagonal expression with respect to
decomposition (3). In particular,
$\left.A\right|_{\mathfrak{m}_{j}}=\lambda_{j}\operatorname{Id}$,
$j=1,\dots,s$.

The next proposition is useful to compute the isotropy representation of a
reductive homogeneous space.

######  Proposition 2.1.

([3]) Let $G/H$ be a homogeneous space and let
$\mathfrak{g}=\mathfrak{h}\oplus\mathfrak{m}$ be a reductive decomposition of
$\mathfrak{g}$. Let $h\in H$, $X\in\mathfrak{h}$ and $Y\in\mathfrak{m}$. Then

$\operatorname{Ad}^{G}(h)(X+Y)=\operatorname{Ad}^{H}(h)X+\operatorname{Ad}^{G/H}(h)Y$

that is, the restriction $\operatorname{Ad}^{G}\big{|}_{H}$ splits into the
sum $\operatorname{Ad}^{H}\oplus\operatorname{Ad}^{G/H}$. We denote by $\chi$
the $\operatorname{Ad}^{G/H}$.

The following lemma provides a simple condition for proving that two
$\operatorname{Ad}^{G/H}$-submodules are inequivalent.

######  Lemma 2.2.

Let $G/H$ be a homogeneous space with reductive decomposition
$\mathfrak{g}=\mathfrak{h}\oplus\mathfrak{m}$ and let
$\mathfrak{m}_{i},\mathfrak{m}_{j}\subset\mathfrak{m}$ be submodules of the
isotropy representation $\operatorname{Ad}^{G/H}$. Assume that for any pair of
non-zero vectors $X\in\mathfrak{m}_{i}$, $Y\in\mathfrak{m}_{j}$ there exists a
vector $a\in\mathfrak{h}$ such that $[a,X]=0$ and $[a,Y]\neq 0$. Then the
submodules $\mathfrak{m}_{i},\mathfrak{m}_{j}$ are
$\operatorname{Ad}^{G/H}$-inequivalent.

###### Proof.

If $\mathfrak{m}_{i},\mathfrak{m}_{j}$ are equivalent, then there exists an
$\operatorname{Ad}^{G/H}$-equivariant isomorphism
$\phi:\mathfrak{m}_{i}\rightarrow\mathfrak{m}_{j}$. The
$\operatorname{Ad}^{G/H}$-equivariance of $\phi$ implies that $\phi$ is
$\operatorname{ad}_{\mathfrak{h}}$-equivariant. In other words,

$\phi([a,X])=[a,Y],$

for any $a\in\mathfrak{h}$. However, $\phi$ is an isomorphism, therefore,
$[a,X]$ is non-zero if and only if $[a,Y]$ is non-zero, which contradicts the
hypothesis of the lemma. Hence, the submodules
$\mathfrak{m}_{i},\mathfrak{m}_{j}$ are inequivalent. ∎

######  Remark 2.3.

For any two $\operatorname{Ad}^{G/H}$-submodules
$\mathfrak{m}_{1},\mathfrak{m}_{2}$, we denote by
$[\mathfrak{m}_{1},\mathfrak{m}_{2}]$ the space generated by the vectors
$[X_{1},X_{2}]$ where $X_{1}\in\mathfrak{m}_{1}$ and
$X_{2}\in\mathfrak{m}_{2}$. Similarly, denote by
$[\mathfrak{h},\mathfrak{m}_{1}]$ the space generated by the vectors
$[a,X_{1}]$ where $a\in\mathfrak{h}$ and $X_{1}\in\mathfrak{m}_{1}$. If
$\mathfrak{m}_{1},\mathfrak{m}_{2}$ are $B$-orthogonal then
$[\mathfrak{m}_{1},\mathfrak{m}_{2}]\subseteq\mathfrak{m}$. Indeed,
$[\mathfrak{m}_{1},\mathfrak{m}_{2}]$ is $B$-orthogonal to $\mathfrak{h}$
because $B([\mathfrak{m}_{1},\mathfrak{m}_{2}],\mathfrak{h})\subseteq
B(\mathfrak{m}_{1},[\mathfrak{m}_{2},\mathfrak{h}])\subseteq
B(\mathfrak{m}_{1},\mathfrak{m}_{2})=\\{0\\}$. Moreover, by the Jacobi
identity, $[\mathfrak{m}_{1},\mathfrak{m}_{2}]$ is also an
$\operatorname{Ad}^{G/H}$-submodule and $[\mathfrak{h},\mathfrak{m}_{1}]$ is
an $\operatorname{Ad}^{G/H}$-submodule of $\mathfrak{m}_{1}$.

##  3\. Properties of geodesic orbit spaces

######  Definition 3.1.

A $G$-invariant metric $g$ on $G/H$ is called a geodesic orbit metric (g.o.
metric) if any geodesic of $(G/H,g)$ through $o$ is an orbit of a one
parameter subgroup of $G$. Equivalently, $g$ is a geodesic orbit metric if for
any geodesic $\gamma$ of $(G/H,g)$ though $o$ there exists a non-zero vector
$X\in\mathfrak{g}$ such that $\gamma(t)=\exp(tX)\cdot o$, $t\in\mathbb{R}$.
The space $(G/H,g)$ is called a geodesic orbit space (g.o. space).

Let $G/H$ be a homogeneous space with $G$ compact, fix an
$\operatorname{Ad}$-invariant inner product $B$ on $\mathfrak{g}$ and consider
the $B$-orthogonal reductive decomposition (1). Moreover, identify each
$G$-invariant metric on $G/H$ with the corresponding metric endomorphism
$A:\mathfrak{m}\rightarrow\mathfrak{m}$.

We have the following condition.

######  Proposition 3.2.

_([1], [19])_ The metric $A$ on $G/H$ is geodesic orbit if and only if for any
vector $X\in\mathfrak{m}\setminus\left\\{{0}\right\\}$ there exists a vector
$a\in\mathfrak{h}$ such that

$[a+X,AX]=0.$ (4)

The following result, which we will call the _normalizer lemma_ , can be used
to simplify the necessary form of the g.o. metrics on $G/H$ by using the
normalizer $N_{G}(H^{0})$.

######  Lemma 3.3.

_([17])_ The inner product $\langle\ ,\ \rangle$, generating the metric of a
geodesic orbit Riemannian space $(G/H,g)$, is not only
$\operatorname{Ad}(H)$-invariant but also
$\operatorname{Ad}(N_{G}(H^{0}))$-invariant, where $N_{G}(H^{0})$ is the
normalizer of the unit component $H^{0}$ of the group $H$ in $G$.

As a result of the normalizer lemma, the metric endomorphism $A$ of a g.o.
metric on $G/H$ is $\operatorname{Ad}(N_{G}(H^{0}))$-equivariant. We will now
state a complementary result to the normalizer lemma for compact spaces, that
characterizes the restriction of a g.o. metric to the compact Lie group
$N_{G}(H^{0})/H^{0}$.

######  Lemma 3.4.

Let $(G/H,g)$ be a compact geodesic orbit space with the $B$-orthogonal
reductive decomposition $\mathfrak{g}=\mathfrak{h}\oplus\mathfrak{m}$, where
$B$ is an $\operatorname{Ad}$-invariant inner product on $\mathfrak{g}$. Let
$A:\mathfrak{m}\rightarrow\mathfrak{m}$ be the corresponding metric
endomorphism of $g$, and let $\mathfrak{n}\subseteq\mathfrak{m}$ be the Lie
algebra of the compact Lie group $N_{G}(H^{0})/H^{0}$. Then the restriction of
$A$ to $\mathfrak{n}$ defines a bi-invariant metric on $N_{G}(H^{0})/H^{0}$.

###### Proof.

We denote by $\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h})\subset\mathfrak{g}$
the Lie algebra of $N_{G}(H^{0})$. We have a $B$-orthogonal decomposition
$\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h})=\mathfrak{h}\oplus\mathfrak{n}$,
where $\mathfrak{n}$ coincides with the Lie algebra of $N_{G}(H^{0})/H^{0}$.
Moreover, we have a $B$-orthogonal decomposition
$\mathfrak{m}=\mathfrak{n}\oplus\mathfrak{p}$, where $\mathfrak{p}$ coincides
with the tangent space of $G/N_{G}(H^{0})$ at the origin. By the normalizer
lemma, the restriction of $A$ on $\mathfrak{p}$ defines an invariant metric on
$G/N_{G}(H^{0})$, and thus $A\mathfrak{p}\subseteq\mathfrak{p}$. By taking
into account the symmetry of $A$ with respect to the product $B$, we deduce
that $B(A\mathfrak{n},\mathfrak{p})=B(\mathfrak{n},A\mathfrak{p})\subseteq
B(\mathfrak{n},\mathfrak{p})=\\{0\\}$. Hence, the image $A\mathfrak{n}$ is
$B$-orthogonal to $\mathfrak{p}$ which, along with decomposition
$\mathfrak{m}=\mathfrak{n}\oplus\mathfrak{p}$, yields
$A\mathfrak{n}\subseteq\mathfrak{n}$. Therefore, the restriction
$\left.A\right|_{\mathfrak{n}}:\mathfrak{n}\rightarrow\mathfrak{n}$ defines a
left-invariant metric on $N_{G}(H^{0})/H^{0}$. Since $A$ is a g.o. metric on
$G/H$, Proposition 3.2 implies that for any $X\in\mathfrak{n}$ there exists a
vector $a\in\mathfrak{h}$ such that
$0=[a+X,AX]=[a+X,\left.A\right|_{\mathfrak{n}}X]$. Therefore, by the same
proposition, $\left.A\right|_{\mathfrak{n}}$ defines a g.o. metric on
$N_{G}(H^{0})/H^{0}$. On the other hand, any left-invariant g.o. metric on a
Lie group is necessarily bi-invariant ([2]), and hence
$\left.A\right|_{\mathfrak{n}}$ is a bi-invariant metric on
$N_{G}(H^{0})/H^{0}$. ∎

######  Remark 3.5.

Let $\mathfrak{p}\subset\mathfrak{m}$ be the tangent space of
$G/N_{G}(H^{0})$, let $\mathfrak{n}\subset\mathfrak{m}$ be the Lie algebra of
$N_{G}(H^{0})/H^{0}$, and consider the decomposition
$\mathfrak{m}=\mathfrak{n}\oplus\mathfrak{p}$. By combining lemmas 3.3 and
3.4, we conclude that the metric endomorphism $A$ corresponding to a g.o.
metric has the block-diagonal form

$A=\begin{pmatrix}\left.A\right|_{\mathfrak{n}}&0\\\
0&\left.A\right|_{\mathfrak{p}}\end{pmatrix},$

where $\left.A\right|_{\mathfrak{n}}$ defines a bi-invariant metric on
$N_{G}(H^{0})/H^{0}$ and $\left.A\right|_{\mathfrak{p}}$ defines a g.o. metric
on $G/N_{G}(H^{0})$.

The following lemma describes the invariant g.o. metrics on compact Lie groups
in terms of their metric endomorphism with respect to an
$\operatorname{Ad}$-invariant inner product $B$. Recall that the Lie algebra
$\mathfrak{g}$ of a compact Lie group $G$ has a direct sum decomposition
$\mathfrak{g}=\mathfrak{g}_{1}\oplus\cdots\oplus\mathfrak{g}_{k}\oplus\mathfrak{z}$,
where $\mathfrak{g}_{j}$ are simple ideals of $\mathfrak{g}$ and
$\mathfrak{z}$ is its center.

######  Lemma 3.6.

_([2], [19])_ Let $G$ be a compact Lie group with Lie algebra
$\mathfrak{g}=\mathfrak{g}_{1}\oplus\cdots\oplus\mathfrak{g}_{k}\oplus\mathfrak{z}$.
A left invariant metric $A$ on $G$ is a g.o. metric if and only if it is bi-
invariant. In particular, $A$ is a g.o. metric if and only if

$A=\begin{pmatrix}\lambda_{1}\left.\operatorname{Id}\right|_{\mathfrak{g}_{1}}&0&\cdots&0\\\
\vdots&\ddots&\cdots&\vdots\\\
0&\cdots&\lambda_{k}\left.\operatorname{Id}\right|_{\mathfrak{g}_{k}}&0\\\
0&\cdots&0&\left.A\right|_{\mathfrak{z}}\end{pmatrix},\ \lambda_{j}>0.$

The following terminology will be useful. Let $W$ be a subspace of a vector
space $V$ and we write $V=W\oplus W^{\bot}$ with respect to some inner product
$B$ on $V$. Then, for $v\in V$ it is $v=w+w^{\prime}$, where $w\in W$ and
$w^{\prime}\in W^{\bot}$.

######  Definition 3.7.

The vector $v$ has _non zero projection on $W$_ if $w^{\prime}\neq 0$. A
subset $S$ of $V$ has non zero projection on $W$ if there exists a vector
$v\in S$ that has non zero projection on $W$.

Finally, the folowing lemma is useful, since it will enable us to equate some
of the eigenvalues of a g.o. metric.

######  Lemma 3.8.

_([19])_ Let $(G/H,g)$ be a g.o. space with $G$ compact and with corresponding
metric endomorphism $A$ with respect to an $\operatorname{Ad}$-invariant inner
product $B$. Let $\mathfrak{m}$ be the $B$-orthogonal complement of
$\mathfrak{h}$ in $\mathfrak{g}$.
1. Assume that $\mathfrak{m}_{1},\mathfrak{m}_{2}$ are $\operatorname{ad}(\mathfrak{h})$-invariant, pairwise $B$-orthogonal subspaces of $\mathfrak{m}$ such that $[\mathfrak{m}_{1},\mathfrak{m}_{2}]$ has non zero projection on ${(\mathfrak{m}_{1}\oplus\mathfrak{m}_{2})^{\bot}}$. Let $\lambda_{1},\lambda_{2}$ be eigenvalues of $A$ such that $\left.A\right|_{\mathfrak{m}_{i}}=\lambda_{i}\operatorname{Id}$, $i=1,2$. Then $\lambda_{1}=\lambda_{2}$.
2. Assume that $\mathfrak{m}_{1},\mathfrak{m}_{2},\mathfrak{m}_{3}$ are $\operatorname{ad}(\mathfrak{h})$-invariant, pairwise $B$-orthogonal subspaces of $\mathfrak{m}$ such that $[\mathfrak{m}_{1},\mathfrak{m}_{2}]$ has non-zero projection on $\mathfrak{m}_{3}$. Let $\lambda_{1},\lambda_{2},\lambda_{3}$ be eigenvalues of $A$ such that $\left.A\right|_{\mathfrak{m}_{i}}=\lambda_{i}\operatorname{Id}$, $i=1,2,3$. Then $\lambda_{1}=\lambda_{2}=\lambda_{3}$.

##  4\. The space
$\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$,
$\sum n_{i}\leq n$

We compute the isotropy representation of
$M=G/H=\operatorname{Sp}(n)/\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$,
$n_{1}+\cdots+n_{s}\leq n$. Denote by $\nu_{2n}$ the standard representation
of $\operatorname{Sp}(n)$, that is
$\nu_{2n}:\operatorname{Sp}(n)\to\operatorname{Aut}(\mathbb{C}^{2n})$. Then
the complexified adjoint representation of $\operatorname{Sp}(n)$ is given by
$\operatorname{Ad}^{\operatorname{Sp}(n)}\otimes\mathbb{C}=S^{2}\nu_{2n}$,
where $S^{2}$ is the second symmetric power of $\nu_{2n}$. Let
$\sigma_{n_{i}}:\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})\to\operatorname{Sp}(n_{i})$
be the projection onto the $i$-factor and
$\varphi_{i}=\nu_{2n_{i}}\circ\sigma_{n_{i}}$ be the projection of the
standard representation of $H$, i.e.

$\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})\stackrel{{\scriptstyle\sigma_{n_{i}}}}{{\longrightarrow}}\operatorname{Sp}(n_{i})\stackrel{{\scriptstyle\nu_{2n_{i}}}}{{\longrightarrow}}\operatorname{Aut}(\mathbb{C}^{2n_{i}}).$

We set $n_{0}:=n-(n_{1}+n_{2}+\cdots+n_{s})$.

Then we have:

$\displaystyle\operatorname{Ad}^{G}\otimes\mathbb{C}\big{|}_{H}$
$\displaystyle=$ $\displaystyle
S^{2}\nu_{2n}\big{|}_{H}=S^{2}(\varphi_{1}\oplus\cdots\oplus\varphi_{s}\oplus\mathbbm{1}_{2n_{0}})=S^{2}\varphi_{{1}}\oplus
S^{2}\varphi_{{2}}\oplus\cdots\oplus S^{2}\varphi_{s}\oplus
S^{2}\mathbbm{1}_{2n_{0}}$ (5)
$\displaystyle\oplus\\{(\varphi_{{1}}\otimes\varphi_{{2}})\oplus\cdots\oplus(\varphi_{{1}}\otimes\varphi_{{s}})\\}\oplus\\{(\varphi_{{2}}\otimes\varphi_{{3}})\oplus\cdots\oplus(\varphi_{2}\otimes\varphi_{s})\\}\oplus\cdots\oplus(\varphi_{s-1}\otimes\varphi_{s})$
$\displaystyle\oplus(\varphi_{{1}}\otimes\mathbbm{1}_{2n_{0}})\oplus(\varphi_{{2}}\otimes\mathbbm{1}_{2n_{0}})\oplus\cdots\oplus(\varphi_{{s}}\otimes\mathbbm{1}_{2n_{0}}),$

where $S^{2}\mathbbm{1}_{2n_{0}}=\mathbbm{1}\oplus\cdots\oplus\mathbbm{1}$ is
the sum of $\binom{2n_{0}+1}{2}=n_{0}(2n_{0}+1)$ trivial representations.

The representation $S^{2}\varphi_{{1}}\oplus
S^{2}\varphi_{{2}}\oplus\cdots\oplus S^{2}\varphi_{s}$ is equal to
$\operatorname{Ad}^{H}\otimes\mathbb{C}$, the complexified adjoint
representation of
$H=\operatorname{Sp}(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$.

Then, in view of Equation (5), Proposition 2.1 implies that the complexified
isotropy representation of $G/H$ is given by

$\displaystyle\chi\otimes\mathbb{C}$ $\displaystyle=$ $\displaystyle
S^{2}\mathbbm{1}_{2n_{0}}\oplus\\{(\varphi_{{1}}\otimes\varphi_{{2}})\oplus\cdots\oplus(\varphi_{{1}}\otimes\varphi_{{s}})\\}\oplus\\{(\varphi_{{2}}\otimes\varphi_{{3}})\oplus\cdots\oplus(\varphi_{2}\otimes\varphi_{s})\\}\oplus\cdots\oplus(\varphi_{s-1}\otimes\varphi_{s})$
(6)
$\displaystyle\oplus(\varphi_{{1}}\otimes\mathbbm{1}_{2n_{0}})\oplus(\varphi_{{2}}\otimes\mathbbm{1}_{2n_{0}})\oplus\cdots\oplus(\varphi_{{s}}\otimes\mathbbm{1}_{2n_{0}})$
$\displaystyle=$ $\displaystyle S^{2}\mathbbm{1}_{2n_{0}}\bigoplus_{1\leq
i<j\leq
s}{(\varphi_{{i}}\otimes\varphi_{{j}})}\bigoplus_{j=1}^{s}{(\varphi_{{j}}\otimes\mathbbm{1}_{2n_{0}})}.$

Also, $\dim_{\mathbb{C}}({\varphi_{i}\otimes\varphi_{j}})=4n_{i}n_{j}$ and
each of
$\varphi_{j}\otimes\mathbbm{1}_{2n_{0}}\cong(\varphi_{j}\otimes\mathbbm{1})\oplus\cdots\oplus(\varphi_{j}\otimes\mathbbm{1})$
is a sum of $2n_{0}$ equivalent representations, each of dimension $2n_{j}$,
$j=1,2,\ldots,s$ (and hence
$\dim_{\mathbb{C}}({\varphi_{j}\otimes\mathbbm{1}_{2n_{0}}})=4kn_{j}$). One
can confirm that

$\dim(\chi\otimes\mathbb{C})=n_{0}(2n_{0}+1)+\sum_{1\leq i<j\leq
s}4n_{i}n_{j}+\sum_{j=1}^{s}4n_{0}n_{j}=2n^{2}-2\sum_{j=1}^{s}n_{j}^{2}+n_{0}=\dim
M.$

Expression (6) induces a decomposition of the complexified tangent space
$\mathfrak{m}$ of $G/H$ as

$\mathfrak{m}\otimes\mathbb{C}=\mathfrak{n}_{1}^{\prime}\oplus\cdots\oplus\mathfrak{n}_{n_{0}(2n_{0}+1)}^{\prime}\bigoplus_{1\leq
i<j\leq s}\mathfrak{n}_{ij}\bigoplus_{j=1}^{s}\mathfrak{n}_{0j},$

where $\dim_{\mathbb{C}}(\mathfrak{n}_{i}^{\prime})=1$,
$\dim_{\mathbb{C}}\mathfrak{n}_{ij}=4n_{i}n_{j}$ and
$\mathfrak{n}_{0j}=\mathfrak{n}_{1}^{j}\oplus\mathfrak{n}_{2}^{j}\oplus\cdots\oplus\mathfrak{n}_{2n_{0}}^{j}$,
with $\mathfrak{n}_{\alpha}^{j}\cong\mathfrak{n}_{\beta}^{j}$,
$\alpha\neq\beta$ and $\dim_{\mathbb{C}}(\mathfrak{n}^{j}_{\ell})=2n_{j}$,
$\ell=1,2,\ldots,2n_{0}$.

The (real) decomposition of the tangent space of $G/H$ is given by

$\mathfrak{m}=\mathfrak{n}_{1}\oplus\cdots\oplus\mathfrak{n}_{n_{0}(2n_{0}+1)}\bigoplus_{1\leq
i<j\leq s}\mathfrak{m}_{ij}\bigoplus_{j=1}^{s}\mathfrak{m}_{0j},$ (7)

where $\mathfrak{m}_{ij}\otimes\mathbb{C}=\mathfrak{n}_{ij}$,
$\mathfrak{m}_{0j}\otimes\mathbb{C}=\mathfrak{n}_{0j}$,
$\dim_{\mathbb{R}}(\mathfrak{n}_{i})=1$,
$\dim_{\mathbb{R}}(\mathfrak{m}_{ij})=4n_{i}n_{j}$,
$\dim_{\mathbb{R}}(\mathfrak{m}_{0j})=4n_{0}n_{j}$. Also,
$\mathfrak{m}_{0j}=\mathfrak{m}_{1}^{j}\oplus\cdots\oplus\mathfrak{m}_{2n_{0}}^{j}$
with $\mathfrak{m}_{\ell}^{j}\otimes\mathbb{C}=\mathfrak{n}_{\ell}^{j}$,
$\mathfrak{m}_{\alpha}^{j}\cong\mathfrak{m}_{\beta}^{j}$, $\alpha\neq\beta$
and $\dim_{\mathbb{R}}(\mathfrak{m}_{\ell}^{j})=2n_{j}$,
$\ell=1,\dots,2n_{0}$. Note that
$\mathfrak{n}_{1}\oplus\cdots\oplus\mathfrak{n}_{n_{0}(2n_{0}+1)}\cong\mathfrak{sp}(n_{0})$.

######  Remark 4.1.

If $n_{0}=0$ the isotropy representation of $G/H$ has no equivalent
representations and all above expressions simplify.

We now give explicit matrix representations of
$\mathfrak{m}_{ij},\mathfrak{m}_{0j}$. Recall the Lie algebra of
$\operatorname{Sp}(n)$,

$\mathfrak{sp}(n)=\left\\{\begin{pmatrix}X&-{}\bar{Y}\\\
Y&\bar{X}\end{pmatrix}\ \Big{|}\begin{array}[]{l}X\in\mathfrak{u}(n),\ Y\
\mbox{ is a}\ n\times n\ \mbox{complex symmetric
matrix}\end{array}\right\\}\subset\mathfrak{u}(2n).\ $

For $i=1,\dots,s$, we embed
$\mathfrak{sp}(n_{i})=\left\\{\begin{pmatrix}X_{i}&-{}\bar{Y}_{i}\\\
Y_{i}&\bar{X}_{i}\end{pmatrix}\right\\}$ in $\mathfrak{sp}(n)$ as

$\left\\{\left(\begin{array}[]{ccccccccccc}0&\cdots&0&\cdots&0&\vline&0&\cdots&0&\cdots&0\\\
\vdots&&\vdots&&\vdots&\vline&\vdots&&\vdots&&0\\\
0&\cdots&X_{i}&\cdots&0&\vline&0&\cdots&-{}\bar{Y}_{i}&\cdots&0\\\
\vdots&&\vdots&&\vdots&\vline&\vdots&&\vdots&&0\\\
0&\cdots&0&\cdots&0&\vline&0&\cdots&0&\cdots&0\\\ \hline\cr
0&\cdots&0&\cdots&0&\vline&0&\cdots&0&\cdots&0\\\
\vdots&&\vdots&&\vdots&\vline&\vdots&&\vdots&&0\\\
0&\cdots&Y_{i}&\cdots&0&\vline&0&\cdots&\bar{X}_{i}&\cdots&0\\\
\vdots&&\vdots&&\vdots&\vline&\vdots&&\vdots&&0\\\
0&\cdots&0&\cdots&0&\vline&0&\cdots&0&\cdots&0\end{array}\right)\right\\}.$

We consider the $\operatorname{Ad}(\operatorname{Sp}(n))$-invariant inner
product $B:\mathfrak{sp}(n)\times\mathfrak{sp}(n)\to\mathbb{R}$, given by

$B(X,Y)=-{\rm Trace}(XY),\quad X,Y\in\mathfrak{sp}(n),$ (8)

and we obtain a $B$-orthogonal decomposition
$\mathfrak{g}=\mathfrak{h}\oplus\mathfrak{m}$, where
$\mathfrak{h}=\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{s})$ and
$\mathfrak{m}\cong T_{o}(G/H)$. We note that $B$ is a multiple of the Killing
form of $\mathfrak{sp}(n)$.

Next, we consider a basis for $\mathfrak{g}=\mathfrak{sp}(n)$ as follows. Let
$M_{2n}\mathbb{C}$ be the set of $2n\times 2n$ complex matrices and we
consider the following matrices in $M_{2n}\mathbb{C}$ with zeros in all
entries except the ones indicated:

$E_{ab}$ with $1$ in $(a,b)$-entry and $1$ in $(n+a,n+b)$-entry.

$F_{ab}$ with $i$ in $(a,b)$-entry and $-i$ in $(n+a,n+b)$-entry.

$G_{ab}$ with $-1$ in $(a,n+b)$-entry and $1$ in $(n+b,a)$-entry.

$H_{ab}$ with $i$ in $(a,n+b)$-entry and $i$ in $(n+b,a)$-entry.

For $1\leq a<b\leq 2n$ we set

$e_{ab}=E_{ab}-E_{ba},\quad f_{ab}=F_{ab}+F_{ba},\quad
g_{ab}=G_{ab}+G_{ba},\quad h_{ab}=H_{ab}+H_{ba}.$ (9)

Then the set $\mathcal{B}=\\{e_{ab},f_{ab},g_{ab},h_{ab}:1\leq a<b\leq n;\
f_{aa},g_{aa},h_{aa}:1\leq a\leq n\\}$ constitutes a basis of
$\mathfrak{sp}(n)$, which is orthogonal with respect to $B$.

The above matrices have the form

$e_{ab}=\left(\begin{array}[]{ccccccccccc}\square&&&&&\vline&\square&&&&\\\
&\ddots&&&&\vline&&\ddots&&&\\\ &&\square&1&&\vline&&&\square&&\\\
&&-1&\ddots&&\vline&&&&\ddots&\\\ &&&&\square&\vline&&&&&\square\\\
\hline\cr\square&&&&&\vline&\square&&&&\\\ &\ddots&&&&\vline&&\ddots&&&\\\
&&\square&&&\vline&&&\square&1&\\\ &&&\ddots&&\vline&&&-1&\ddots&\\\
&&&&\square&\vline&&&&&\square\end{array}\right),f_{ab}=\left(\begin{array}[]{ccccccccccc}\square&&&&&\vline&\square&&&&\\\
&\ddots&&&&\vline&&\ddots&&&\\\ &&\square&i&&\vline&&&\square&&\\\
&&i&\ddots&&\vline&&&&\ddots&\\\ &&&&\square&\vline&&&&&\square\\\
\hline\cr\square&&&&&\vline&\square&&&&\\\ &\ddots&&&&\vline&&\ddots&&&\\\
&&\square&&&\vline&&&\square&-i&\\\ &&&\ddots&&\vline&&&-i&\ddots&\\\
&&&&\square&\vline&&&&&\square\end{array}\right),$

$g_{ab}=\left(\begin{array}[]{ccccccccccc}\square&&&&&\vline&\square&&&&\\\
&\ddots&&&&\vline&&\ddots&&&\\\ &&\square&&&\vline&&&\square&-1&\\\
&&&\ddots&&\vline&&&-1&\ddots&\\\ &&&&\square&\vline&&&&&\square\\\
\hline\cr\square&&&&&\vline&\square&&&&\\\ &\ddots&&&&\vline&&\ddots&&&\\\
&&\square&1&&\vline&&&\square&&\\\ &&1&\ddots&&\vline&&&&\ddots&\\\
&&&&\square&\vline&&&&&\square\end{array}\right),h_{ab}=\left(\begin{array}[]{ccccccccccc}\square&&&&&\vline&\square&&&&\\\
&\ddots&&&&\vline&&\ddots&&&\\\ &&\square&&&\vline&&&\square&i&\\\
&&&\ddots&&\vline&&&i&\ddots&\\\ &&&&\square&\vline&&&&&\square\\\
\hline\cr\square&&&&&\vline&\square&&&&\\\ &\ddots&&&&\vline&&\ddots&&&\\\
&&\square&i&&\vline&&&\square&&\\\ &&i&\ddots&&\vline&&&&\ddots&\\\
&&&&\square&\vline&&&&&\square\end{array}\right).$

Recall also the relations $e_{ba}=-e_{ab}$, $f_{ba}=f_{ab}$, $g_{ba}=g_{ab}$
and $h_{ba}=h_{ab}$. The next lemma follows from straightforward calculations.

######  Lemma 4.2.

The Lie-bracket relations among the vectors (9) are given as follows:

$\displaystyle[e_{ij},e_{lm}]$
$\displaystyle=\delta_{jl}e_{im}+\delta_{im}e_{jl}-\delta_{il}e_{jm}-\delta_{jm}e_{il}$
$\displaystyle[e_{ij},f_{lm}]$
$\displaystyle=\delta_{jl}f_{im}-\delta_{im}f_{jl}-\delta_{il}f_{jm}+\delta_{jm}f_{il}$
$\displaystyle[e_{ij},g_{lm}]$
$\displaystyle=-\delta_{jl}g_{im}+\delta_{im}g_{jl}-\delta_{il}g_{jm}+\delta_{jm}g_{il}$
$\displaystyle[e_{ij},h_{lm}]$
$\displaystyle=-\delta_{jl}h_{im}+\delta_{im}h_{jl}-\delta_{il}h_{jm}+\delta_{jm}h_{il}$
$\displaystyle[f_{ij},f_{lm}]$
$\displaystyle=-\delta_{jl}e_{im}-\delta_{im}e_{jl}-\delta_{il}e_{jm}-\delta_{jm}e_{il}$
$\displaystyle[f_{ij},g_{lm}]$
$\displaystyle=-\delta_{jl}h_{im}-\delta_{im}h_{jl}-\delta_{il}h_{jm}-\delta_{jm}h_{il}$
$\displaystyle[f_{ij},h_{lm}]$
$\displaystyle=\delta_{jl}g_{im}+\delta_{im}g_{jl}+\delta_{il}g_{jm}+\delta_{jm}g_{il}$
$\displaystyle[g_{ij},g_{lm}]$
$\displaystyle=-\delta_{jl}e_{im}-\delta_{im}e_{jl}-\delta_{il}e_{jm}-\delta_{jm}e_{il}$
$\displaystyle[g_{ij},h_{lm}]$
$\displaystyle=-\delta_{jl}f_{im}-\delta_{im}f_{jl}-\delta_{il}f_{jm}-\delta_{jm}f_{il}$
$\displaystyle[h_{ij},h_{lm}]$
$\displaystyle=-\delta_{jl}e_{im}-\delta_{im}e_{jl}-\delta_{il}e_{jm}-\delta_{jm}e_{il}.$

A choice for the modules in the decomposition (7) is the following:

$\displaystyle\mathfrak{m}_{ij}$ $\displaystyle=$
$\displaystyle\operatorname{span}_{\mathbb{R}}\\{e_{ab},f_{ab},g_{ab},h_{ab}:n_{0}+n_{1}+\cdots+n_{i-1}+1\leq
a\leq n_{0}+n_{1}+\cdots+n_{i},$ $\displaystyle\ \ \ \ \ \ \ \ \ \quad\
\quad\quad n_{0}+n_{1}+\cdots+n_{j-1}+1\leq b\leq
n_{0}+n_{1}+\cdots+n_{j}\\},\ 1\leq i<j\leq s$
$\displaystyle\mathfrak{m}_{0j}$ $\displaystyle=$
$\displaystyle\operatorname{span}_{\mathbb{R}}\\{e_{ab},f_{ab},g_{ab},h_{ab}:1\leq
a\leq n_{0},\ n_{0}+n_{1}+\cdots+n_{j-1}+1\leq b\leq
n_{0}+n_{1}+\cdots+n_{j}\\},\ j=1,\dots,s.$

The $n_{0}(2n_{0}+1)$ trivial representations in (7) generate the Lie algebra

$\mathfrak{n}=\operatorname{span}_{\mathbb{R}}\\{e_{ab},f_{ab},g_{ab},h_{ab}:1\leq
a<b\leq n_{0};\ f_{aa},g_{aa},h_{aa}:1\leq a\leq n_{0}\\},$

isomorphic to $\mathfrak{sp}(n_{0})$.

The equivalent modules in the decomposition of $\mathfrak{m}_{0j}$ are given
by

$\displaystyle\mathfrak{m}^{j}_{\ell}=\operatorname{span}_{\mathbb{R}}\\{e_{\ell
b},f_{\ell b},g_{\ell b},h_{\ell b}:n_{0}+n_{1}+\cdots+n_{j-1}+1\leq b\leq
n_{0}+n_{1}+\cdots+n_{j}\\},\ \ell=1,\dots,2k.$

Also, for $j=1,\dots,s$, we have

$\displaystyle\mathfrak{sp}(n_{j})$ $\displaystyle=$
$\displaystyle\operatorname{span}_{\mathbb{R}}\\{e_{ab},f_{ab},g_{ab},h_{ab}:n_{0}+n_{1}+\cdots+n_{j-1}+1\leq
a<b\leq n_{0}+n_{1}+\cdots+n_{j};$ $\displaystyle\ \ \ \ \ \ \ \
f_{aa},g_{aa},h_{aa}:n_{0}+n_{1}+\cdots+n_{j-1}+1\leq a\leq
n_{0}+n_{1}+\cdots+n_{j}\\}.$

In summary, we obtain the $B$-orthogonal decomposition

$\mathfrak{m}=\mathfrak{n}\oplus\mathfrak{p},$ (10)

where

$\mathfrak{n}\cong\mathfrak{sp}(n_{0}),\quad\mathfrak{p}=\bigoplus_{1\leq
i<j\leq
s}\mathfrak{m}_{ij}\bigoplus_{j=1}^{s}\mathfrak{m}_{0j}=\bigoplus_{0\leq
i<j\leq s}\mathfrak{m}_{ij}.$

Decomposition (10) as a subspace of $\mathfrak{sp}(n)$ can be depicted in the
following $2n\times 2n$ complex matrix, where the positions represented by
$\square$ are zero matrices (corresponding to positions of
$\mathfrak{sp}(n_{i})$, $i=1,\dots s$):

$\left(\begin{array}[]{cccccccccccc}\mathfrak{n}&\mathfrak{m}_{01}&\cdots&&\mathfrak{m}_{0s}&\vline&\mathfrak{n}&\mathfrak{m}_{01}&\cdots&&\mathfrak{m}_{0s}\\\
\mathfrak{m}_{01}&\square&\mathfrak{m}_{12}&\cdots&\mathfrak{m}_{1s}&\vline&\mathfrak{m}_{01}&\square&\mathfrak{m}_{12}&\cdots&\mathfrak{m}_{1s}\\\
\mathfrak{m}_{02}&\mathfrak{m}_{12}&\square&\ddots&\vdots&\vline&\mathfrak{m}_{02}&\mathfrak{m}_{12}&\square&\ddots&\vdots\\\
\vdots&\vdots&&\ddots&\mathfrak{m}_{s-1,s}&\vline&\vdots&\vdots&&\ddots&\mathfrak{m}_{s-1,s}\\\
\mathfrak{m}_{0s}&\mathfrak{m}_{1s}&\cdots&&\square&\vline&\mathfrak{m}_{02}&\mathfrak{m}_{1s}&\cdots&&\square&\\\
\hline\cr\mathfrak{n}&\mathfrak{m}_{01}&\cdots&&\mathfrak{m}_{0s}&\vline&\mathfrak{n}&\mathfrak{m}_{01}&\cdots&&\mathfrak{m}_{0s}\\\
\mathfrak{m}_{01}&\square&\mathfrak{m}_{12}&\cdots&\mathfrak{m}_{1s}&\vline&\mathfrak{m}_{01}&\square&\mathfrak{m}_{12}&\cdots&\mathfrak{m}_{1s}\\\
\mathfrak{m}_{02}&\mathfrak{m}_{12}&\square&\ddots&\vdots&\vline&\mathfrak{m}_{02}&\mathfrak{m}_{12}&\square&\ddots&\vdots\\\
\vdots&\vdots&&\ddots&\mathfrak{m}_{s-1,s}&\vline&\vdots&\vdots&&\ddots&\mathfrak{m}_{s-1,s}\\\
\mathfrak{m}_{0s}&\mathfrak{m}_{1s}&\cdots&&\square&\vline&\mathfrak{m}_{02}&\mathfrak{m}_{1s}&\cdots&&\square&\end{array}\right).$

######  Proposition 4.3.

The following relations are satisfied by the modules in the decomposition (7):

$[\mathfrak{sp}(n_{0}),\mathfrak{sp}(n_{0})]\subseteq\mathfrak{sp}(n_{0}),$
(11) $[\mathfrak{m}_{ij},\mathfrak{m}_{jk}]\subseteq\mathfrak{m}_{ik},\ \
0\leq i<j<k\leq s.$ (12)
$[\mathfrak{sp}(n_{i}),\mathfrak{m}_{lm}]=\left\\{\begin{array}[]{lll}\mathfrak{m}_{lm},\quad\makebox{if
$i=l$ or $i=m$}\\\ \\{0\\},\quad\makebox{otherwise},\end{array}\right.\ \
l<m,\ \ i,l,m=0,\dots,s.$ (13)

###### Proof.

The first relation is true since $\mathfrak{sp}(n_{0})$ is a subalgebra of
$\mathfrak{g}=\mathfrak{sp}(n)$. The second and the third relations follow
from Lemma 4.2 and the expression of the submodules $\mathfrak{m}_{ij}$ in
terms of the basis $\mathcal{B}$. ∎

######  Remark 4.4.

The above relations are not the only relations among the modules in
decomposition (7) which are valid, however these are the ones which we use in
our study.

##  5\. Some preliminary results

In order to prove our main theorem 1.1 we will need some propositions and
lemmas, which we collect in the present section.

The following can be easily proved by induction on $s$.

######  Lemma 5.1.

Let $R_{s}=\\{\lambda_{ij}:\ 0\leq i<j\leq s\\}$ be a set of real numbers such
that $\lambda_{ij}=\lambda_{jk}=\lambda_{ik}$ for all $0\leq i<j<k\leq s$.
Then $R_{s}$ is a singleton.

Consider the space $G/H=Sp(n)/Sp(n_{1})\times\cdots\times Sp(n_{s})$ and
recall the description of its tangent space given in Section 4. Let $g$ be a
$G$-invariant g.o. metric on $G/H$ with corresponding metric endomorphism
$A:\mathfrak{m}\rightarrow\mathfrak{m}$ (cf. Equation (2)), where we take $B$
to be the multiple of the Killing form (8). Recall the spaces
$\mathfrak{n}=\mathfrak{sp}(n_{0})$ ($n_{0}=n-(n_{1}+\cdots+n_{s})$) and
$\mathfrak{p}$, which were defined in decomposition (10).

######  Proposition 5.2.

$\left.A\right|_{\mathfrak{p}}=\lambda\operatorname{Id}$, where $\lambda>0$.

###### Proof.

By the normalizer Lemma 3.3 the metric endomorphism $A$ is
$\operatorname{Ad}(N_{G}(H^{0}))$-equivariant. As a result, $A$ is
$\operatorname{ad}(\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h}))$-equivariant,
where
$\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h})=\\{Y\in\mathfrak{g}:[Y,\mathfrak{h}]\subseteq\mathfrak{h}\\}$
is the Lie algebra of $N_{G}(H^{0})$. In our case
$\mathfrak{h}=\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{s})$ and

$\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h})=\mathfrak{sp}(n_{0})\oplus\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{s}).$

By taking into account Lemma 4.2 and the expressions of the subspaces
$\mathfrak{m}_{0j},\mathfrak{m}_{ij},\mathfrak{sp}(n_{j})$, $j=0,1,\dots,s$ in
terms of the basis $\mathcal{B}$, we deduce that the submodules
$\mathfrak{m}_{0j},\mathfrak{m}_{ij}$ are
$\operatorname{ad}(\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h}))$-invariant and
$\operatorname{ad}(\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h}))$-irreducible.
Apart from being
$\operatorname{ad}(\mathfrak{n}_{\mathfrak{g}}(\mathfrak{h}))$-irreducible,
the $\operatorname{Ad}(N_{G}(H^{0}))$-submodules $\mathfrak{m}_{ij}$, $0\leq
i<j\leq s$, are pairwise inequivalent. To see this, firstly observe that if
$\mathfrak{m}_{ij}$ and $\mathfrak{m}_{lm}$, with $0\leq i<j\leq s$ and $0\leq
l<m\leq s$, are two distinct $\operatorname{Ad}(N_{G}(H^{0}))$-submodules then
there exists at least one index $i_{0}$ such that either

1) $i_{0}=i$ or $i_{0}=j$ and $i_{0}\neq l,m$, or
2) $i_{0}=l$ or $i_{0}=m$ and $i_{0}\neq i,j$.

For case 1), taking into account Equation (13) we obtain
$[\mathfrak{sp}(n_{i_{0}}),\mathfrak{m}_{ij}]=\mathfrak{m}_{ij}$ and
$[\mathfrak{sp}(n_{i_{0}}),\mathfrak{m}_{lm}]=\\{0\\}$. For case 2), Equation
(13) yields $[\mathfrak{sp}(n_{i_{0}}),\mathfrak{m}_{ij}]=\\{0\\}$ and
$[\mathfrak{sp}(n_{i_{0}}),\mathfrak{m}_{lm}]=\mathfrak{m}_{lm}$. By virtue of
Lemma 2.2 we deduce that the submodules $\mathfrak{m}_{ij}$ and
$\mathfrak{m}_{lm}$ are inequivalent in both cases.

On the other hand, and in view of Remark 3.5, the restriction
$\left.A\right|_{\mathfrak{p}}=\left.A\right|_{\bigoplus_{0\leq i<j\leq
s}\mathfrak{m}_{ij}}$ defines a g.o. metric on $G/N_{G}(H^{0})$. Since the
submodules $\mathfrak{m}_{ij}$ are
$\operatorname{Ad}(N_{G}(H^{0}))$-irreducible and pairwise inequivalent, we
have

$\left.A\right|_{\mathfrak{m}_{ij}}=\lambda_{ij}\operatorname{Id},\ \ 0\leq
i<j<l\leq s.$ (14)

Also, Equation (12) yields

$[\mathfrak{m}_{ij},\mathfrak{m}_{jl}]\subseteq\mathfrak{m}_{il}\ \
\makebox{for all}\ \ 0\leq i<j<l\leq s.$

In particular, since $[\mathfrak{m}_{ij},\mathfrak{m}_{jl}]$ is an
$\operatorname{Ad}(N_{G}(H^{0}))$-submodule and $\mathfrak{m}_{il}$ is
irreducible, we have
$[\mathfrak{m}_{ij},\mathfrak{m}_{jl}]=\mathfrak{m}_{il}$. Due to the fact
that $\left.A\right|_{\mathfrak{p}}$ defines a g.o. metric on
$G/N_{G}(H^{0})$, along with Equation (14), the
$\operatorname{ad}({\mathfrak{h}})$-invariance of
$\mathfrak{m}_{ij},\mathfrak{m}_{jl}$ and their $B$-orthogonality, part 2. of
Lemma 3.8 yields

$\lambda_{ij}=\lambda_{jl}=\lambda_{il}\ \ \makebox{for all}\ \ 0\leq
i<j<l\leq s.$

By Lemma 5.1, we deduce that the set $R_{s}=\\{\lambda_{ij}:\ 0\leq i<j\leq
s\\}$ of the eigenvalues of $\left.A\right|_{\mathfrak{p}}$ contains only one
element, and thus $\left.A\right|_{\mathfrak{p}}=\lambda\operatorname{Id}$. ∎

Now assume that $0<n_{0}\neq 1$. The algebra
$\mathfrak{n}=\mathfrak{sp}(n_{0})$ coincides with the Lie algebra of
$N_{G}(H^{0})/H^{0}$. By Lemma 3.4, $\left.A\right|_{\mathfrak{n}}$ defines a
bi-invariant metric on $N_{G}(H^{0})/H^{0}$, which in turn corresponds to an
$\operatorname{Ad}$-invariant inner product on $\mathfrak{sp}(n_{0})$. Note
that $\mathfrak{sp}(n_{0})$ is simple and the only
$\operatorname{Ad}$-invariant inner product is a scalar multiple of the
Killing form. Therefore,

$\left.A\right|_{\mathfrak{n}}=\mu\operatorname{Id},\ \ \mu>0.$ (15)

Next, we have the following:

######  Proposition 5.3.

We assume that $n_{0}>1$. Then $\lambda=\mu$, where $\lambda$ is given by
Proposition 5.2 and $\mu$ is given by Equation (15).

###### Proof.

Recall the $\operatorname{Ad}(G/H)$-submodules
$\mathfrak{m}_{l}^{j}\subseteq\mathfrak{m}_{0j}\subset\mathfrak{p}$ defined in
Section 4. By Proposition 5.2, we have

$\left.A\right|_{\mathfrak{m}_{j}^{i}}=\lambda\operatorname{Id}.$

We choose the vectors $e_{12}\in\mathfrak{n}$ and
$e_{1k+1}\in\mathfrak{m}^{1}_{1}$. Since $n_{0}>1$, the above vectors do not
coincide, hence by Lemma 4.2 it follows that

$[e_{12},e_{1k+1}]=-e_{2k+1}\in\mathfrak{m}^{1}_{2}.$

Therefore, $[\mathfrak{n},\mathfrak{m}_{1}^{1}]$ has non zero projection on
$(\mathfrak{n}\oplus\mathfrak{m}_{1}^{1})^{\bot}$. Along with the
$\operatorname{ad}(\mathfrak{h})$-invariance of $\mathfrak{n}$ and
$\mathfrak{m}_{1}^{1}$ and the facts that
$\left.A\right|_{\mathfrak{m}_{j}^{i}}=\lambda\operatorname{Id}$ and
$\left.A\right|_{\mathfrak{n}}=\mu\operatorname{Id}$, part 1. of Lemma 3.8
yields $\lambda=\mu$. ∎

Finally, we consider the case $n_{0}=1$.

######  Proposition 5.4.

Let $n_{0}=1$ and $s>1$. Then $\lambda=\mu$, where $\lambda$ is given by
Proposition 5.2 and $\mu$ is given by Equation (15).

The proof requires the following.

######  Lemma 5.5.

Let $\mathfrak{m}_{ij}$, $i>0$, be one of the submodules defined in Section 4.
Let
$a\in\mathfrak{h}=\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{s})$
such that $[a,\mathfrak{m}_{ij}]=\\{0\\}$. Then the projection of $a$ on
$\mathfrak{sp}(n_{i})\oplus\mathfrak{sp}(n_{j})$ is zero, i.e.
$a\in\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{i-1})\oplus\mathfrak{sp}(n_{i+1})\oplus\cdots\oplus\mathfrak{sp}(n_{j-1})\oplus\mathfrak{sp}(n_{j+1})\oplus\cdots\oplus\mathfrak{sp}(n_{s})$.

###### Proof.

Let $\pi_{j}(a)$ be the projection of $a$ on $\mathfrak{sp}(n_{j})$ and set
$\mathfrak{h}_{ij}:=\mathfrak{sp}(n_{i})\oplus\mathfrak{sp}(n_{j})$. By
relation (13), we deduce that
$[\mathfrak{h},\mathfrak{m}_{ij}]=[\mathfrak{h}_{ij},\mathfrak{m}_{ij}]=\mathfrak{m}_{ij}$.
Then we have that
$\\{0\\}=[a,\mathfrak{m}_{ij}]=[(\pi_{i}+\pi_{j})(a),\mathfrak{m}_{ij}]$,
therefore $(\pi_{i}+\pi_{j})(a)$ lies in the space

$c_{ij}:=\\{X\in\mathfrak{h}_{ij}:[X,Y]=0\ \ \makebox{for all}\ \
Y\in\mathfrak{m}_{ij}\\}.$

Using the Jacobi identity and the
$\operatorname{ad}(\mathfrak{h}_{ij})$-invariance of $\mathfrak{m}_{ij}$, it
is not hard to verify that the space $c_{ij}$ is an ideal of the Lie algebra
$\mathfrak{h}_{ij}$. Since $\mathfrak{h}_{ij}$ is semisimple, $c_{ij}$ is
necessarily one of the ideals $\\{0\\}$, $\mathfrak{sp}(n_{i})$,
$\mathfrak{sp}(n_{j})$ or $\mathfrak{h}_{ij}$. On the other hand, relation
(13) implies that $[\mathfrak{sp}(n_{i}),\mathfrak{m}_{ij}]\neq\\{0\\}$,
$[\mathfrak{sp}(n_{j}),\mathfrak{m}_{ij}]\neq\\{0\\}$ and
$[\mathfrak{sp}(n_{i})\oplus\mathfrak{sp}(n_{j}),\mathfrak{m}_{ij}]\neq\\{0\\}$.
We conclude that $c_{ij}=\\{0\\}$ and thus $(\pi_{i}+\pi_{j})(a)=0$. ∎

_Proof of Proposition 5.4_. In view of Remark 3.5, Proposition 5.2 and
Equation (15), the g.o. metric on $G/H$ has the form

$A=\begin{pmatrix}\left.\mu\operatorname{Id}\right|_{\mathfrak{n}}&0&\\\
0&\left.\lambda\operatorname{Id}\right|_{\mathfrak{p}}\end{pmatrix}.$ (16)

Let $X\in\mathfrak{m}=\mathfrak{n}\oplus\mathfrak{p}$ and write
$X=X_{\mathfrak{n}}+X_{\mathfrak{p}}$, where $X_{\mathfrak{n}}$ is the
projection of $X$ on $\mathfrak{n}$ and $X_{\mathfrak{p}}$ is the projection
of $X$ on $\mathfrak{p}$. Further, we write

$X_{\mathfrak{p}}=\sum_{0\leq i<j\leq s}X_{ij},$ (17)

where $X_{ij}$ denotes the projection of $X$ on the space $\mathfrak{m}_{ij}$.
Taking into account the definition of the spaces $\mathfrak{n}$,
$\mathfrak{m}_{ij}$ in terms of the basis $\mathcal{B}$, along with Lemma 4.2,
we deduce that

$[\mathfrak{n},\mathfrak{m}_{ij}]\subseteq\left\\{\begin{array}[]{ll}\mathfrak{m}_{ij},\
\ \mbox{if}\ \ i=0\\\
\left\\{{0}\right\\},\quad\mbox{otherwise}.\end{array}\right.$ (18)

Since $A$ is a g.o. metric, Proposition 3.2 along with expression (16) of $A$,
implies that there exists a vector
$a\in\mathfrak{h}=\mathfrak{sp}(n_{1})\oplus\cdots\oplus\mathfrak{sp}(n_{s})$
such that

$0=[a+X_{\mathfrak{n}}+X_{\mathfrak{p}},\mu X_{\mathfrak{n}}+\lambda
X_{\mathfrak{p}}]=\mu[a,X_{\mathfrak{n}}]+\lambda[a,X_{\mathfrak{p}}]+(\lambda-\mu)[X_{\mathfrak{n}},X_{\mathfrak{p}}].$

Taking into account Equation (17) along with relation (18) and the fact that
$[\mathfrak{n},\mathfrak{h}]=\\{0\\}$ (recall that
$\mathfrak{n}=\mathfrak{sp}(n_{0})$), the above equation is equivalent to

$0=\lambda\sum_{0\leq i<j\leq
s}[a,X_{ij}]+(\lambda-\mu)\sum_{j=1}^{s}[X_{\mathfrak{n}},X_{0j}].$ (19)

We choose $X_{\mathfrak{n}}:=f_{11}$, $X_{01}:=e_{12}$ and
$X_{\mathfrak{p}}:=X_{01}+X_{12}=e_{12}+X_{12}\in\mathfrak{m}_{01}+\mathfrak{m}_{12}$
(since $s>1$, $\mathfrak{m}_{12}\neq\\{0\\}$), where we consider $X_{12}$ as
arbitrary. Then Equation (19) along with Lemma 4.2 yield

$0=\lambda[a,e_{12}]+\lambda[a,X_{12}]+2(\lambda-\mu)f_{21}.$ (20)

By the $\operatorname{ad}(\mathfrak{h})$-invariance of $\mathfrak{m}_{ij}$ and
the fact that $a\in\mathfrak{h}$, the first two terms of Equation (20) lie in
$\mathfrak{m}_{01}$ and $\mathfrak{m}_{12}$ respectively, while the last term
lies in $\mathfrak{m}_{01}$. Therefore, Equation (20) yields the system

$\displaystyle\lambda[a,e_{12}]+2(\lambda-\mu)f_{21}$ $\displaystyle=$
$\displaystyle 0$ (21) $\displaystyle\lambda[a,X_{12}]$ $\displaystyle=$
$\displaystyle 0.$ (22)

Since $X_{12}\in\mathfrak{m}_{12}$ is arbitrary, Equation (22) implies that
$[a,\mathfrak{m}_{12}]=\\{0\\}$. Lemma 5.5 then yields
$(\pi_{1}+\pi_{2})(a)=0$, where $\pi_{j}$ denotes the projection of $a$ on
$\mathfrak{sp}(n_{j})$. Hence $\pi_{1}(a)=\pi_{2}(a)=0$. On the other hand,
the fact that $e_{12}\in\mathfrak{m}_{01}$ along with relation (13), imply
that $[a,e_{12}]=[\pi_{1}(a),e_{12}]=0$. Substituting into Equation (21), we
obtain $2(\lambda-\mu)f_{21}=0$ and thus $\lambda=\mu$. ∎

##  6\. Proof of Theorem 1.1

We can now combine the results of the previous section to give a proof of
Theorem 1.1.

The standard metric on
$G/H=Sp(n)/Sp(n_{1})\times\cdots\times\operatorname{Sp}(n_{s})$ is a g.o.
metric, hence the sufficiency part of the theorem holds trivially. For the
necessity part, let $g$ be a $G$-invariant g.o. metric on $G/H$. If $n_{0}=0$,
i.e. $\mathfrak{n}=\\{0\\}$, then Proposition 5.2 implies that $g$ is the
standard metric.

Now assume that $n_{0}>0$. If $n_{0}\neq 1$, then the theorem follows from
Remark 3.5 and Propositions 5.2 and 5.3. If $n_{0}=1$ then
$\mathfrak{n}=\mathfrak{sp}(1)$ and we consider two cases, $s=1$ and $s>1$.

If $s=1$, then $G/H=Sp(n)/Sp(n-1)$ and in this case any metric endomorphism
$A$ such that $\left.A\right|_{\mathfrak{p}}=\lambda\operatorname{Id}$ and
$\left.A\right|_{\mathfrak{n}}=\mu\operatorname{Id}$ is a g.o. metric and vice
versa ([16]). Therefore, the $G$-invariant g.o metrics on $G/H$ are precisely
the metrics of the form
$A=\begin{pmatrix}\left.\mu\operatorname{Id}\right|_{\mathfrak{n}}&0&\\\
0&\left.\lambda\operatorname{Id}\right|_{\mathfrak{p}}\end{pmatrix}$. It
follows that the $G$-invariant g.o. metrics on $G/H$ are homothetic to the
metrics
$g_{\mu}=\begin{pmatrix}\left.\mu\operatorname{Id}\right|_{\mathfrak{n}}&0&\\\
0&\left.\operatorname{Id}\right|_{\mathfrak{p}}\end{pmatrix}$. This settles
Theorem 1.1 for the case $n_{0}=1$ and $s=1$. Finally, if $n_{0}=1$ and $s>1$,
Theorem 1.1 follows from Proposition 5.4.∎

## References

  * [1] D. V. Alekseevsky and A. Arvanitoyeorgos: Riemannian flag manifolds with homogeneous geodesics, Trans. Amer. Math. Soc. 359(8) (2007) 3769–3789.
  * [2] D. V. Alekseevsky and Yu. G. Nikonorov: Compact Riemannian manifolds with homogeneous geodesics. SIGMA Symmetry Integrability Geom. Methods Appl. 5(093) (2009), 16 pages.
  * [3] A. Arvanitoyeorgos: An Introduction to Lie Groups and the Geometry of Homogeneous Spaces, Amer. Math. Soc, Student Math. Library, Vol. 22, 2003.
  * [4] A. Arvanitoyeorgos, N. P. Souris and M. Statha: Geodesic orbit metrics in a class of homogeneous bundles over complex and real Stiefel manifolds, preprint.
  * [5] Arvanitoyeorgos A.: Homogeneous manifolds whose geodesics are orbits. Recent results and some open problems, Irish Math. Soc. Bulletin 79 (2017) 5–29.
  * [6] Berstovskii V. N. and Nikonorov Y. G.: On $\delta$-homogeneous Riemannian manifolds, Differential Geom. Appl. 26 (2008), 514–535.
  * [7] Berstovskii V. N. and Nikonorov Y. G.: Clifford-Wolf homogeneous Riemannian manifolds, J. Differential Geom. 82(3) (2009), 467–500.
  * [8] Berstovskii V. N. and Nikonorov Y. G.: Riemannian Manifolds and Homogeneous Geodesics, Springer 2020.
  * [9] Berndt J., Kowalski O. and Vanhecke L.: Geodesics in weakly symmetric spaces, Ann. Global Anal. Geom. 15 (1997), 153–156.
  * [10] Calvaruso G. and Zaeim, A.: Four-dimensional pseudo-Riemannian g.o. spaces and manifolds, J. Geom. Phys. 130, 63–80 (2018). vGeodesic orbit metrics on compact simple Lie groups arising from flag manifolds,
  * [11] Chen Z., Nikolayevski Y. and Nikonorov Y. G.: Compact geodesic orbit spaces with a simple isotropy group, arXiv:2009.06329 (2020).
  * [12] Dušek Z.: Homogeneous geodesics and g.o. manifolds, Note Mat. 38 (1) 1–15 (2018).
  * [13] Gordon C. S. and Nikonorov Y. G.: Geodesic orbit Riemannian structures on $R^{n}$, J. Geom. Phys. 134, 235–243 (2018).
  * [14] Kowalski O. and Vanhecke L.: Riemannian manifolds with homogeneous geodesics, Un. Mat. Ital. B 7 (5) (1991), 189–246.
  * [15] Nikonorov Y. G.: On the structure of geodesic orbit Riemannian spaces, Ann. Global Anal. Geom. 52, 289–311 (2017).
  * [16] Yu G. Nikonorov: Geodesic orbit Riemannian metrics on spheres, Vladik. Math. J. (13) (3) (2013) 67–76.
  * [17] Yu G. Nikonorov: On the structure of geodesic orbit Riemannian spaces, Ann. Global Anal. Geom. 52 (2017) 289–311.
  * [18] S. Ramanujam: Application of Morse theory to some homogeneous spaces, Tohoku Math. J. 21(3) (1969) 343–353.
  * [19] Souris N. P.: Geodesic orbit metrics in compact homogeneous manifolds with equivalent isotropy submodules. Transform. Groups 23 (2018) 1149–1165.
  * [20] Souris N. P.: On a class of geodesic orbit spaces with abelian isotropy subgroup, Manuscripta Math. (2020). https://doi.org/10.1007/s00229-020-01236-9 arXiv:2008.13588v2 (2020).
  * [21] Tamaru H.: Riemannian g.o. spaces fibered over irreducible symmetric spaces, Osaka J. Math. 36, 835–851 (1999).
  * [22] Wolf J. A.: The geometry and structure of isotropy irreducible homogeneous spaces, Acta Math. 120, 59-148 (1968).
  * [23] Wolf J.A.: Harmonic analysis on commutative spaces, Mathematical Surveys and Monographs, Vol. 142, American Mathematical Society, Providence, RI, 2007.
  * [24] Yan Z. and Deng S.: Finsler spaces whose geodesics are orbits, Differential Geom. Appl. 36, 1–23 (2014).


# Hierarchical Reinforcement Learning for Temporal Pattern Prediction

Faith Johnson  Faith Johnson
Rutgers University
<EMAIL_ADDRESS>Kristin Dana
Rutgers University
<EMAIL_ADDRESS>
(Sometime in the Future)

###### Abstract

In this work, we explore the use of hierarchical reinforcement learning (HRL)
for the task of temporal sequence prediction. Using a combination of deep
learning and HRL, we develop a stock agent to predict temporal price sequences
from historical stock price data and a vehicle agent to predict steering
angles from first person, dash cam images. Our results in both domains
indicate that a type of HRL, called feudal reinforcement learning, provides
significant improvements to training speed and stability and prediction
accuracy over standard RL. A key component to this success is the multi-
resolution structure that introduces both temporal and spatial abstraction
into the network hierarchy.

##  1 Introduction

Reinforcement learning (RL) has made major strides over the past decade, from
learning to play Atari games [8] to mastering chess and Go [10]. However, RL
algorithms tend to work in a specific, controlled environment and are often
difficult to train. In response to this brittleness, hierarchical
reinforcement learning (HRL) is growing in popularity.

We combine deep learning and HRL for temporal sequence prediction in two
application domains where publicly available data is abundant. First, we
develop stock agents to execute trades in a market environment. The training
data consists of historical stock prices from 1995 to 2018. Second, we develop
a vehicle agent to predict steering angles given visual input. We use the
Udacity dataset[11] as training data, which consists of five videos with a
total duration of 1694 seconds.

In HRL, a manager network operates at a lower temporal resolution and produces
goals that it passes to the worker network. The worker network uses this goal
to produce a policy over micro-actions at a higher temporal resolution than
the manager[12]. Within the stock market, there are two natural hierarchies.
The first hierarchy involves temporal scales. A trader can consider how a
stock’s price fluctuates over the course of an hour, but also over the course
of a week, month, or year. The second hierarchy is the separation of the
different market sectors, each containing a multitude of stocks. In order to
trade in the market effectively, stock brokers must consider both the
relationships between sectors and the relationships between the stocks in each
sector.

In the same vein, the task of autonomous navigation is complicated because, at
all times, human drivers have two levels of things they are paying attention
to. The first level is on a fine grain: don’t immediately crash the vehicle by
hitting obstacles. The second level is on a coarser grain: plan actions a few
steps ahead to keep the vehicle going in the correct direction as efficiently
as possible. In both domains, financial and vehicluar, we implement agents
with both RL and HRL. We show that HRL provides improved training stability
and prediction performance.

##  2 Methods

###  2.1 LSTM Stock Predictions

First, we set up a baseline for future result comparison using a simple LSTM
network. Using stock market data gathered from Kaggle[7], we predict the
closed price for a single day given the open price of that day. We build a
simple LSTM model in Keras[2] with ten neurons and a ReLU activation function
followed by a fully connected layer. This network is trained for ten epochs
using the mean squared error loss function and an adam optimizer.

Next, we predict a sequence of open prices for a particular stock given a
sequence of previous open prices. For this, we use a slightly larger LSTM
network with three layers of LSTMs, each with ten neurons and ReLU activation
functions followed by a fully connected layer. The loss function and optimizer
for this experiment are also mean squared error and Adam respectively.
However, this network is trained for twenty epochs. We conduct this experiment
with several sequence pairings: 1 previous price to predict the next price, 3
previous prices to predict the next 3 prices, and 5 previous prices to predict
the next 5 prices.

Finally, we implement a reinforcement learning stock agent to predict open
stock prices by learning a multiplier to transform the previous open price to
the next open price. It takes a sequence of historical open prices for a
single stock as input and passes them through several layers of LSTMs with
ReLU activation functions and a fully connected layer. The new price is
computed by multiplying the current open price by the output to produce the
predicted open price for the next day.

###  2.2 Stock Environment

After the multiplier agent, we rethink our approach to stock price prediction
by moving away from predicting the price itself, and thus away from computing
the answer to a regression problem. The next set of experiments involves
executing trades in a stock market environment with the goal of doubling the
value in a given portfolio. The rationale behind this change is that
reinforcement learning excels at learning a policy over a given set of
actions. In the regression problem of learning a multiplier, the action space
is essentially infinite. In addition, the state space is infinite because it
consists of all possible stock prices. Learning a policy over an infinite set
of possibilities is almost impossible.

To create our stock environment for these new reinforcement learning agents,
we use Quadnl[9] to collect market information for a small subset of stocks
from six sectors. The six stock sectors are technology, energy, finance,
healthcare, utilities, and transportation. The goal is to have a diverse
selection of stocks and sectors from the market in order for the agent to be
able to glean the relationships between the sectors along with the
relationship between stocks within each sector.

We define the action space to consist of three actions: buy, sell, and hold.
All agents buy or sell only one stock per action, unless otherwise stated. If
an agent does not have enough money to execute the buy action, it is forced to
hold. The same is true if it does not have enough shares to execute the sell
action. The environment keeps track of an agent’s current balance of cash and
portfolio value, as well as giving the agents a reward for their actions,
which is generally the change in the agent’s total portfolio value.

###  2.3 Hard Coded Stock Agent

The baseline for performance in the stock environment comes from a hard coded
stock agent whose aim is to double the value of a given portfolio. First, it
defines two thresholds, a selling threshold and a buying threshold. When two
consecutive open prices differ by less than the selling threshold, the agent
decides to sell shares in the stock. When the two prices differ by more than
the buying threshold, the agent decides to buy a share in the stock. The idea
is that the buying threshold is positive and the selling threshold is negative
so that the agent will sell when the price goes down and buy when the price
goes up. When the price difference lies between the two thresholds, the agent
takes the hold action.

###  2.4 Reinforcement Learning Stock Agents

####  2.4.1 Q Learning Agent

For the Q learning agent, the state space is limited to the combinations of
whether or not the price of a stock has gone up or down and whether or not the
agent currently possesses shares in said stock. In q learning, the agent keeps
track of the policy over actions and states using a q table. The q table is
indexed by the actions and the states, as in Table 1, and is initially filled
with zeros. As an agent takes actions in the environment, it receives a reward
that it uses to update the table to reflect the utility of each action given a
certain state. An agent decides which action to take by referencing the
portion of this q table corresponding to its current state and choosing the
action associated with the highest q value.

| Buy | Hold | Sell
---|---|---|---
Price Increases & Have Shares |  |  |
Price Increases & Have No Shares |  |  |
Price Decreases & Have Shares |  |  |
Price Decreases & Have No Shares |  |  |
Table 1: Example q table for the q learning stock agent. It is indexed by the
three actions (buy, hold, sell) and the combination of price fluctuation and
share possession.

####  2.4.2 Deep Q Network (DQN) Agent

This stock agent builds upon the same ideas as the previous q learning agent,
but it chooses it’s actions differently. Instead of using the reward from the
environment to update the q table, it uses a deep q network, or DQN, to
approximate the q value of a certain action given a state. This network is
made up of three LSTMs with ReLU activation functions in sequence, followed by
a fully connected layer. The first LSTM has 32 hidden layers, and the last two
have 64 layers.

In the case of this network, the state is the previous three open prices for
the stocks in each of the sectors. However, the action space remains the same.
With the q learning agent, having an infinite state space would not be ideal
for optimal policy convergence, but the DQN is still able to converge on a
solution.

The reward from the environment plays a role in the loss back-propagation of
the network. For each action taken by the agent, a tuple containing the
initial state, $s_{0}$, the final state, $s$, the action taken, $a$, and the
corresponding reward, $r$, is saved in a replay buffer. During training, a
random tuple is sampled from this buffer, and the loss, which in this case is
the reward, is back-propagated through the network as in [8].

###  2.5 Feudal Reinforcement Learning

In feudal reinforcement learning, the manager network operates at a lower
temporal resolution than the worker network. It receives state input from the
environment and communicates with the worker network through a goal vector.
This goal vector encapsulates a temporally extended action that the manager
thinks will receive the highest reward from the environment. The worker
executes atomic actions in the environment based on this goal vector and its
own state information. This process of manager/worker communication through
temporal abstraction helps to break down a problem into more easily digestible
pieces.

To explain the concept of temporal abstraction further, take the case of an
agent attempting to leave a room through a door. When a person thinks of
completing this action, they don’t do it at the low level of straight,
straight, left, straight, right, etc. In other words, they do not consciously
think of each atomic action required to exit the room. Instead, they think in
terms of temporal abstraction. Find the door. Approach it. Pass through it.
Each of those actions encapsulates multiple atomic actions that need to be
executed in a specific order for the agent to complete the task.

For a feudal network to solve the room example, the manager would create goal
vectors for the “find the door”, “approach it”, and “pass through it”
operations. Then, the worker would only have to focus on executing atomic
actions to complete one of these smaller tasks at a time, which is much
simpler than the original task of exiting the room as a whole. This makes it
easier to generate an ideal policy. Additionally, the idea of temporal
abstraction can be applied to space. Incorporating different spatial
resolutions into a feudal network can break down problems into spatial
abstractions which make them easier to solve in the same way.

###  2.6 Maze Environment

To test the performance of feudal reinforcement learning, we use a maze
environment as proposed in Dayan et al.[3]. In this environment, there are
multiple levels of the same maze, each at a lower spatial resolution than the
previous level. The agent at the highest spatial resolution is the worker, who
receives goal vectors from the agent at the next lowest resolution, who is its
manager. This manager becomes the worker for the agent at the next lowest
resolution, and so on, until you reach the level with the lowest spatial
resolution, where the ultimate manager resides. For example, the worker on the
level with the highest spatial resolution will operate in a 16x16 grid and
have a manager who operates in an 8x8 grid. This manager will be the worker
for the agent in the 4x4 grid, etc., until you get to the final manager in the
1x1 grid.

(a) Manager View

(b) Worker View

Figure 1: (a) Manager’s 2x2 view of the maze. (b) Worker’s corresponding 4x4
view of the same maze.

We create this maze by editing the gym-maze[1] github repository code. In our
maze, there are only two levels. The worker operates in a 4x4 version of the
maze, and the manager operates in a 2x2 version, as in Figure 1. Each square
in the manager’s 2x2 rendition of the maze corresponds to a 2x2 section of the
worker’s maze. We omit the 1x1 manager from the Dayan et al. experiment
because it is computationally irrelevant for this task. The state space of
each agent is comprised of each square in the grid of their respective maze
resolutions. The objective of the agents is to reach some goal square in the
maze. This goal is mapped to the same equivalent location in all maze levels
and can be specified at run-time. The action space is comprised of moving
north/south/east/west or declaring that the goal is the current space. There
is a base reward of

$\frac{-0.1}{X_{dim}*Y_{dim}}$

applied to every movement for all agents, where$X_{dim}$ and $Y_{dim}$ are the
x and y dimensions of the maze. However, if an agent finds the goal, it
receives a reward of 1.

###  2.7 Maze Agents

We test several agents in our maze environment, starting with a reinforcement
learning model as a baseline, before moving to a feudal reinforcement learning
implementation.

####  2.7.1 Q Learning Agent

The first agent we built for the maze environment uses q learning to navigate
a single, 4x4 level of the maze in search of a goal square. When it reaches
this goal, the experiment ends. The q table of the agent is the same size as
the maze with each square in the maze corresponding to one entry in the table.
The values in this table are updated based on the reward received from the
environment.

####  2.7.2 Feudal Q Learning Agent

The feudal network solves the maze using q learning as well. The manager
network receives it’s location in the maze as it’s current state. It uses this
to choose and execute the best action from it’s q table. The manager is able
to move in any of the four directions, and this direction is the basis for the
goal vector that tells the worker what quadrant to move to. If the manager
declares that the goal is in its current space, it is telling the worker that
it should look for its own goal in a specific quadrant of the maze. When the
worker receives a goal vector from the manager, it moves to the specified
quadrant and waits for more instructions. If it is indicated that the worker
should look for the goal in a quadrant, it continues to move until the goal is
found.

Both the manager and the worker receive a base negative reward for every
action that doesn’t result in finding the goal. In this way, the spaces
furthest away from the goal will have a more negative q value than those
closest to the goal. In addition, the manager receives a negative reward if
the worker finds the goal space without following the instructions from the
goal vector. While the individual reward values resulting from exploring the
maze may be the same, the manager and worker do not receive the same reward
signals. The worker takes many more steps in the environment due to the
difference in spatial resolution, so it will receive a reward more often than
the manager. In this way, the spatial abstraction of the maze results in a
temporal abstraction of the reward signals.

###  2.8 Feudal Reinforcement Learning Stock Agents

Once we discovered the performance improvements of feudal reinforcement
learning, we decide to return to the stock market portfolio experiments with
feudal reinforcement learning agents.

####  2.8.1 Feudal Q Learning Agent

Our feudal q learning stock agent operates in the same environment as the
previous reinforcement learning agents and has the same goal of doubling the
value of some portfolio. Its input is a sequence of open prices for each stock
in the six predetermined sectors. The q table structure and state space are
also the same. The main difference is the division of labor between the
manager and worker networks.

The manager receives the price input from the environment and determines
whether or not each of the six sectors should be traded or not. This decision
is passed to the worker in the goal vector. The worker then decides whether to
buy or sell the stocks in the sectors specified by the manager. For each goal
vector from the manager, the worker acts a fixed number of times to introduce
temporal abstraction to the problem in addition to the spatial abstraction
already present. The reward of the manager is the overall portfolio value
change, while the worker receives a reward for the portfolio value change of
each sector after each action it executes.

####  2.8.2 Feudal Networks with Multiple Workers

The feudal reinforcement learning problem can be extended to a vertical
hierarchy with multiple managers and workers in sequence, as we’ve already
explored, but this concept can also be extended horizontally to one manager
with multiple workers. To this end, we have implemented two different
experiments, both using q learning. The first involved a manager network with
a set of three different worker networks, where each worker makes a different
number of transactions in the environment. The manager’s action is choosing
which of the workers will act in the environment at a given time.

The second involved a manager network with a set of three workers, where each
of these workers have a different hard coded behavior. The first buys when a
stock’s price increases and sells when it decreases, the second sells when a
stock’s price increases and buys when it decreases, and the third executes a
random action. The manager’s action set consists of choosing which worker will
interact with the environment.

###  2.9 Driving Environment

We also test feudal reinforcement learning in the domain of autonomous
vehicles. For that, we use the Udacity driving dataset[11]. They provide
steering angles, first-person dash cam images, braking, and throttle pressure
data. We augment this dataset to increase its size and influence model
training by performing several transformations on the image and angle data.
First, we implement a horizontal flip to effectively double the size of the
dataset. For this change, we negate the angles associated with the flipped
images. As an additional option, we use the horizontal and vertical optical
flow images. The horizontal optical flow image, $i_{x}$, is obtained by
convolving the image with the row vector $[1,0,-1]$, while the vertical
optical flow image is obtained by convolving the image with the column vector

$\begin{bmatrix}1\\\ 0\\\ -1\\\ \end{bmatrix}$

Finally, all images are scaled and normalized so that their pixel values lie
in the range $[-1,1]$.

###  2.10 Steering Angle Experiments

####  2.10.1 Steering Angle Prediction

We started simple, so our first task was to predict steering angles based on
visual input. After some initial difficulty with our model, we found a
network[4] from the Udacity challenge that accurately predicts steering
angles. It has a convolutional layer with a ReLU activation function followed
by a dropout layer. The output of this is saved to use for a skip connection
later on in the network. This is repeated four times before the output is fed
through some fully connected layers, also with ReLU activation functions. At
this point, the output and the intermediary representations are added
together, passed through an ELU layer, and normalized. Then, the previous
steering angle and the output of the ELU layer are passed through an LSTM.
Finally, the output of the LSTM is passed through a fully connected layer to
produce the steering angle. Note that this network takes in a sequence of
images as well as the previous angle in order to make its predictions.

####  2.10.2 Subroutine ID Prediction

Being able to predict steering angles is useful, but for feudal reinforcement
learning we also need to classify the steering angles into their temporally
abstracted categories (such as go right, go left, go straight). This can be
done by hand, but it would be a lengthy process. Instead, we take inspiration
from Kumar et al.[5] to learn these subroutines, otherwise called options or
macro-actions, using a neural network.

To do this, we jointly train two networks. The first takes in a sequence of
angles and predicts the subroutine ID. The second takes in the subroutine ID,
a sequence of images, and the previously predicted angle and predicts the next
steering angle in the sequence. A problem we encountered with the steering
angle prediction from the previous section is that it appears as if the
network is simply predicting that the previous angle will be the next steering
angle. To circumvent this, we give the second network the previously predicted
angle instead of the ground truth angle. Additionally during training, the
sequence of angles fed into the first network contains the angle it is trying
to predict. However, during testing, we only use a sequence of angles
preceding the angle we aim to predict in order to avoid this conflict.

####  2.10.3 t-SNE Prediction

Figure 2: Steering, braking, and throttle data are concatenated every m time
steps to make a vector of length 3m. Each 3m vector corresponds to one set of
coordinates in the 2D t-SNE space. The t-SNE coordinates act like a manager
for the steering angle prediction and operate at a lower temporal scale. In
our experiments, m=10. $t$ and $\tau$ correspond to the final time step for
the driving data and t-SNE coordinates respectively.

Figure 3: Total plot of the t-SNE coordinates for the Udacity data. The colors
correspond to the average sign of the angles in each length 3m vector used to
generate the points.

Figure 4: K-Means clustering (k=20) of the TSNE coordinates of the Udacity
data with the centroids pictured in red. Not only to distinct clusters form in
the data, but each cluster corresponds to a unique action of the vehicle.
Figure 5: Example training images are shown with their corresponding t-SNE
centroids. Notice that the bottom right of the figure contains sharp right
turns. As you move upwards, the right turn gets less sharp until the vehicle
begins to go straight. By the top left of the figure, the vehicle is making
sharp left turns.

Ideally, we want an angle prediction network that does not take in the
previous steering angle at all. To accomplish this, we explored using t-SNE[6]
as an embedding space for our driving data and as the subroutine IDs
themselves. To do this, we arranged the steering angle, braking, and throttle
pressure data into vectors of length ten. Then, the vectors from each category
that correspond to the same time steps are concatenated together to make
vectors of length thirty. The collection of these vectors is passed through
the unsupervised t-SNE algorithm to create a coordinate space for the driving
data.

Each vector of length thirty is given one x and y coordinate pair as
illustrated in Figure 4. The greater collection of all of the generated points
is in Figure 4. The coloring of the points in this figure is hard coded. The
points corresponding to vectors with primarily negative steering angles are in
blue. The points corresponding to vectors with positive steering angles are in
green. The orange points correspond to vectors with steering angles that are
relatively close to zero.

Once we have the t-SNE embedding of the data, we use K-Means clustering on the
coordinates and take the centroids of the clusters as our new subroutine IDs,
as shown in Figure 4. We vary k from ten to twenty to determine if different
numbers of clusters improve prediction performance. Then, we train a network
to take in the centroids as the subroutine ID, as well as a sequence of
images, in order to predict the next steering angle.

In order to ensure that no data pertaining to the predicted steering angle is
used as input to this network, we use the t-SNE centroid corresponding to the
data of the previous 3m steering, braking, and throttle data as input to the
network. To illustrate, refer back to Figure 4. If we are predicting an angle
from the range $t\in[2m,3m]$, then the t-SNE centroid used for the subroutine
ID input to the angle prediction network will be the centroid at $\tau=2$,
which was made with the steering, braking, and throttle data from
$t\in[m,2m]$. In this way, the angle we are attempting to predict will not be
used to compute the t-SNE centroid used as the subroutine ID. This shift also
incorporates an extra level of temporal abstraction into our network.

Additionally, we create a tool that displays the visual data corresponding to
the different t-SNE coordinates, allowing the user to visually inspect that
neighboring points in the embedding space correspond to similar driving
behaviors. Figure 5 attempts to replicate this by showing example training
images that correspond to some of the t-SNE centroids. Notice that the bottom
right of the figure contains sharp right turns. Moving diagonally upwards, the
right turns get less sharp until the vehicle begins to go straight. Then, this
straight motion gradually begins to become a left turn until, by the top left
of the figure, the vehicle is making sharp left turns.

##  3 Results

###  3.1 LSTM Experiments

Figure 6: LSTM loss comparison for predictions on regular data and smoothed
data. The more smoothed the data, the more quickly the loss decays.

Our first experiment used LSTMs to predict open prices of stocks. We varied
the input and output window sizes from one to fifteen and compared the
results. A subset of the prediction graphs are available in Figure 8 for
predicting two, four, ten, and twelve prices out. Also included in each graph
is a line representing the average of the last two, four, ten, or twelve
prices as a comparison to the prediction. The prediction with a window of two
works extremely well, as evidenced by the fact that the three lines (the real,
prediction, and average price) are almost directly on top of each other.
However, as the window length increases, there is a clear divergence of the
prediction from the real price. There is a trade off between accuracy and the
length of the prediction, which is expected because data farther out in time
will have less of a dependence on the input to the LSTM. Also, the predictions
become much noisier, which is to be expected for the same reason.

We also tested LSTM prediction on smoothed data. After training the LSTM on
this smoothed data, we tested the model on new smoothed and non-smoothed data
and compared the loss values, computed through mean squared error, in Figure
6. The blue line is the loss associated with the smoothed data and the orange
line corresponds to the regular data. The loss value is on the y-axis, and the
x-axis shows how much smoothing was applied to the training data. We smoothed
using a moving average filter, so the x-axis points represent how many data
points were used in the average. The graph shows that feeding smoothed data
into an LSTM increases the accuracy of its predictions and leads to a quicker
decay of the loss.

The results for our first stock agent that learns a multiplier to predict the
next open price based on the previous stock price can be found in Figure 7
(a). The blue line represents the real stock open price, and the green line is
the prediction. The y-axis is the price in dollars, and the x-axis is the time
step. To get a better idea of exactly how accurate the predictions are, Figure
7 (b) also shows the difference between the real open prices and the predicted
prices for each time step. Most of the predictions differ by less than a
dollar, which is an order of magnitude less than the prices themselves. The
largest differences between the real and predicted prices occur during drastic
changes to the stock price.

(a) Real (blue) versus predicted (green) stock price predicted using the
multiplier stock agent

(b) Difference between the real stock price and the predicted price for the
multiplier stock agent

Figure 7: Results for the multiplier stock agent. (a) shows that the
predictions match very closely, and (b) shows that the areas where the
predicted and real price differ the most occur during drastic price changes.

(a) Window of 2

(b) Window of 4

(c) Window of 10

(d) Window of 12

Figure 8: Prediction results for an LSTM when the size of the input and output
prediction windows are from two, four, ten, and twelve respectively. The LSTM
predictions match more closely with smaller windows than larger windows and
provide less noisy results. However, the larger windows allow for longer term
predictions.

We then compare these predictions with those from the LSTM. In general, the
reinforcement agent predictions, as pictured in Figure 9 (a), are much more
accurate than the LSTM prediction, as pictured in Figure 9 (b). It seems that
LSTMs do a decent job at predicting smaller, local changes, but their
performance falls short when a large change in price occurs. The reinforcement
learning agent is more robust to being able to handle these changes.
Additionally, the LSTM predictions are much more noisy.

(a) Multiplication RL Stock Agent

(b) LSTM

Figure 9: Comparison of the multiplication stock agent predictions and the
LSTM predictions. The multiplication agent matches the real prices closer than
the LSTM and provides less noisy predictions overall.

We also compare the LSTM and multiplication stock agent’s abilities to predict
open prices for multiple stocks at once in Figure 10. Once again, we see that
the predictions of the reinforcement learning stock agent are much better than
the LSTM. For this experiment, we used stocks from the same sector in order to
increase the likelihood that there would be correlations in the stocks’
behavior. Reinforcement learning is better able to find and exploit these
relationships to help make predictions than the LSTM. With this in mind, we
shift our focus towards reinforcement learning and other techniques that can
be derived from it.

(a) Multiplier Agent Predictions

(b) LSTM Predictions

Figure 10: Comparison between the multiplier stock agent and an LSTM for
predicting prices for multiple stocks at once. The stock symbols are provided
in the legend (top left). The LSTM struggles to capture the behavior of
multiple stocks at once.

###  3.2 Maze Experiments

Moving to the maze experiments, we compare the relative performance of
reinforcement learning and feudal reinforcement learning. We had three
different agents navigate a maze until they reached some goal space. For this
experiment, we used a fixed maze with a fixed goal space, but it is possible
to randomize both the maze structure and the goal location at run time. The
standard q learning agent’s performance is documented in Figure 11 and Figure
12 in blue.

The next two agents are different variations of feudal q learning networks. In
the first, the goal vector the worker receives from the manager tells it which
direction to take in the maze. The results for this agent are in red in
Figures 11 and 12. In the second feudal q learning agent, the goal vector
received by the worker tells it which quadrant to go to in the maze. This is
different than the previous agent because the worker is not explicitly told
which direction to take to reach this goal quadrant. This agent’s results are
pictured in Figures 11 and 12 in green.

Figure 11: Comparison of time steps required per episode for three agents in
the maze environment. Feudal reinforcement learning (green) takes less time to
solve the maze overall and explores the maze more efficiently. Figure 12:
Comparison of the reward per episode for three agents in the maze environment.
The reward for feudal reinforcement learning (green) converges faster than the
other two methods.

In Figure 11, we compare the amount of time steps per episode for each of the
three agents. The q learning agent takes the most amount of time overall,
closely followed by the feudal network with a direction as the goal vector.
Additionally, these two methods solve the maze in approximately the same
amount of time once they have found the optimal path through the maze.
However, the feudal network with a quadrant as the goal vector is both
significantly faster during training and finds a better solution to the maze,
as evidenced by the fact that its solution takes less time to navigate the
maze than the other two agents.

Figure 13: Comparison of hard coded, q learning, DQN, and feudal q learning
agent performance, respectively, in the task of doubling the value of a
portfolio in the stock market. The hard coded agent is the slowest, followed
by the q learning agent. The DQN agent and the feudal reinforcement learning
agent performed comparably. Note the difference in the x-axis scales.

Figure 14: We run the experiment from Figure 13 multiple times and record the
optimal solution durations from each trial in histograms. Notice the
difference in time scales between the first two and the last two histograms.
The ranking from the previous figure still holds. However, feudal
reinforcement learning has cemented itself as the fastest method, as evidenced
by the fact that it’s histogram is skewed to the left more than the DQN’s.

Figure 12 shows the reward per episode for each of the three maze agents. The
reward of the three agents converges to the same number, by design, but it is
clear from the graph that the feudal agent with a quadrant as the goal vector
performs the best. Its reward reaches the convergence value much faster than
either of the other two agents. However, both of the feudal agents have a
large dip in the reward early on in training that is not present in the q
learning agent, indicating that the feudal networks do a lot more of their
exploration in the earlier stages of their training than the q learning
network.

###  3.3 Portfolio Stock Experiments

Now that we’ve discovered the power of feudal reinforcement learning, we
revisit the problem of predicting stock prices. However, instead of attempting
to solve a regression problem, we shift our focus to learning a policy over
actions. We compare the performance of a hard coded agent, a q learning agent,
a DQN agent, and a feudal Q learning agent at the task of doubling the value
of a given portfolio in the stock market in Figure 13. Pay attention to the
x-axes in this figure to note the difference between the presented methods.
The hard coded agent takes the longest amount of time to accomplish this task,
as expected, with a duration of 1453 time steps. Also unsurprisingly, the q
learning agent has the next longest duration with 802 time steps. The DQN
agent doubles its portfolio value in 679 time steps, and the feudal q learning
agent achieves this goal in 680 time steps.

The main takeaway from this result is that we achieved comparable results with
feudal q learning, which is a relatively simple method, as we did with the
DQN, which is a relatively complicated, deep learning method. We wanted to
verify that this was always the case, so we repeated this portfolio doubling
experiment multiple times for each agent and recorded the duration results in
the histograms in Figure 14. The y-axis is the number of trials in each bin,
and the x-axis is the duration of each trial. Extra attention should be paid
to the x-axes of the graphs in Figure 14. The hard coded and q learning agents
have an x-axis from 0 to 2500, while the other two agent’s x-axes are capped
at 1000.

We can see that the q learning agent has values skewed more towards zero than
the hard coded agent, so we expect an overall faster average duration from
that agent. The average duration for the hard coded agent was 1521 time steps,
and the q learning agent had an average duration of 1326 time steps, so this
we prove this claim to be correct. In the same way, the feudal q learning
agent has a histogram that is skewed more towards zero than the DQN agent, so
we expect this to be the faster method. The DQN agent took an average of 651
time steps, and the feudal q learning agent took an average of 573 time steps.
Therefore, we show that our original result was an understatement, and feudal
q learning is, on average, much faster than a DQN at doubling a portfolio’s
value in the stock market.

###  3.4 Steering Angle Experiments

In the stock portfolio experiments, we prove the effectiveness of feudal
reinforcement learning. In this section, we aim to explore the boundary of its
abilities in the driving domain. Our first experiment involves predicting
steering angles based on image input. We create an image cube with ten
sequential frames that we feed into our modified Udacity challenge network[4],
along with the previous steering angle, to predict the next steering angle.
Figure 15 shows a subset of real steering angles from the Udacity[11] dataset,
in blue, and the corresponding predicted angles, in orange. The predictions
follow the real angles very closely except when there are drastic changes in
the steering angles where it tends to over/under estimate the steering angle,
which is the same issue we encountered with the LSTM stock experiments.

Figure 15: Steering angle predictions on the Udacity dataset from the modified
Udacity steering challenge winner network. Note that this network takes the
previous angle as input when predicting the next angle which gives it an
unfair advantage.

Our ultimate goal, however, is to use feudal networks to predict steering
angles. To do this, we first need to label subroutines within the data in
order to have data with which to train the manager network. Instead of doing
this by hand, we jointly train two networks: one that takes in a sequence of
angles and predicts their subroutine ID, and another that takes in this
subroutine ID, an image cube, and the previously predicted angle and predicts
the next angle in the sequence. Figure 16 shows these prediction results. The
left graph contains the steering angle predictions. The real angles are in
blue, and the predicted angles are in orange. The right graph shows the
predicted subroutine IDs. The blue line is the raw prediction values, and the
orange line shows the binned values. For this, we map the predicted subroutine
IDs to their closest value in the set $\\{-1,0,1\\}$. In this way, we have
three discrete subroutine IDs corresponding to left turns, right turns, and
going straight.

However, there are two problems with these solutions. The first is that it
stands to reason that there could be more than just three subroutines
represented in the driving data. Driving is a complex task that involves a lot
of minutia. For instance, we could expand left turns to turning a little left,
turning a moderate amount of left, and turning a lot left. The same could be
done for right turns and even going straight. Therefore, constraining the
subroutine IDs to fit into three discrete categories, as inspired by [5], may
not allow us to represent an agent’s actions thoroughly enough. The second
problem is that, ideally, we want a network that predicts steering angles
without explicitly taking in information about the previous angle because this
gives the network an unfair advantage.

Figure 16: Angle and subroutine ID prediction results on the Udacity dataset.
Notice that the subroutine ID’s behavior mimics the real angle behavior.

To this end, we shift our focus from handcrafting our subroutine ID
definitions to using t-SNE to do it automatically. We embed the data into 2D
space and use those coordinate pairs as the subroutine IDs. However, before we
attempt to predict the t-SNE coordinates from image data, we run an experiment
to determine if the t-SNE coordinates will work as subroutine IDs. We use the
ground truth value of the t-SNE centroids as the subroutine ID in our angle
prediction network, along with an image cube of size ten, to determine whether
or not it would be worthwhile to attempt to predict the centroids. If using
t-SNE as the subroutine ID produces inaccurate results, then we would need to
explore other avenues. The results of this are in Figure 17. The blue lines
are the real steering angle, and the orange lines are the predicted angle.
While the results in this figure are less accurate than our other prediction
results, the predictions are more relevant to real world applications because
they are computed using only visual input.

Figure 17: Results of steering angle prediction when the t-SNE coordinates of
the input data are used as the subroutine IDs. Notice that, for these results,
we use a network that does not take the previous angle as input.

##  4 Discussion

In this work, we show that feudal reinforcement learning is more effective
than reinforcement learning at the tasks of stock price prediction and
steering angle prediction. We originally considered using generative
adversarial networks (GANs) for sequence prediction, but our early experiments
pointed to the effectiveness of feudal reinforcement learning instead. With
our maze experiments, we find that feudal reinforcement learning is faster
during training than reinforcement learning. We also find that feudal
reinforcement learning achieves the maximum reward more quickly than
reinforcement learning. Both of these effects are due to feudal reinforcement
learning’s temporal abstraction. Breaking down the problem into more easily
digestible pieces narrows the focus of the worker agent and allows the optimal
policy to be found more quickly.

Additionally, temporal abstraction also helps alleviate the problems of long
term credit assignment and sparse reward signals. The lower temporal
resolution of the manager shortens the period of time between rewards overall.
In addition to the original sparse reward, the worker network also receives a
reward for obeying the goals from the manager. This feedback can be much more
frequent than the sparse reward, thus allowing for more consistent network
updates. In this way, we were able to achieve better results with feudal q
learning in our stock portfolio experiments than with a DQN.

Finally, we find that a t-SNE embedding space can be useful as the goal space
for the manager in feudal reinforcement learning in our steering angle
prediction experiment. We use the centroid corresponding to steering angle,
braking, and throttle data from the previous ten time steps as the subroutine
ID in our angle prediction network and were able to predict future steering
angles without the direct use of the steering angle from the previous time
step. The temporal abstraction inherent in the t-SNE centroid creation mimics
the role of the manager network and allows the worker to be able to more
accurately predict steering angles than if it attempted this task on its own.

## References

  * [1] Matthew Chan.  gym-maze.  https://github.com/MattChanTK/gym-maze, 2017.
  * [2] François Chollet et al.  Keras.  https://keras.io, 2015.
  * [3] Peter Dayan and Geoffrey E Hinton.  Feudal reinforcement learning.  In Advances in neural information processing systems, pages 271–278, 1993.
  * [4] Komanda.  https://github.com/udacity/self-driving-car/blob/master/steering-models/community-models/komanda/solution-komanda.ipynb, 2016\.
  * [5] Ashish Kumar, Saurabh Gupta, and Jitendra Malik.  Learning navigation subroutines by watching videos.  arXiv preprint arXiv:1905.12612, 2019.
  * [6] Laurens van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  Journal of machine learning research, 9(Nov):2579–2605, 2008.
  * [7] Boris Marjanovic.  Huge stock market dataset.  https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-stocks-etfs, Nov 2017.
  * [8] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning.  arXiv preprint arXiv:1312.5602, 2013.
  * [9] Quandl.  WIKI various end-of-day data.  https://www.quandl.com/data/WIKI, 2016.
  * [10] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al.  Mastering chess and shogi by self-play with a general reinforcement learning algorithm.  arXiv preprint arXiv:1712.01815, 2017.
  * [11] Udacity.  Udacity self-driving car driving data 10/3/2016 (dataset-2-2.bag.tar.gz).
  * [12] Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, and Koray Kavukcuoglu.  Feudal networks for hierarchical reinforcement learning.  In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3540–3549. JMLR. org, 2017.