# A Comparison of Learning Algorithms on the Arcade Learning Environment

Aaron Defazio<EMAIL_ADDRESS>
NICTA/Australian National University  Canberra  ACT  Australia  Thore Graepel
<EMAIL_ADDRESS>
Microsoft Research Cambridge  UK

###### Abstract

Reinforcement learning agents have traditionally been evaluated on small toy
problems. With advances in computing power and the advent of the Arcade
Learning Environment, it is now possible to evaluate algorithms on diverse and
difficult problems within a consistent framework. We discuss some challenges
posed by the arcade learning environment which do not manifest in simpler
environments. We then provide a comparison of model-free, linear learning
algorithms on this challenging problem set.

## Introduction

Reinforcement learning in general environments is one of the core problems in
AI. The goal of reinforcement learning is to develop agents that learn,
interact and adapt in complex environments, based off of feedback in the form
of rewards.

What is a ‘general’ environment? Agents that use raw bit streams exist at one
end of the spectrum (McCallum, 1996), whereas methods highly tuned for
specific problem classes exist at the other. In this work we will be
considering the Arcade Learning Environment (ALE, Bellemare et al. 2013),
which consists of a large and diverse set of arcade games. These games possess
substantial structure; their state is presented to the player as low-
resolution images, consisting of basic bitmap graphics. However they are
varied enough that it is difficult to program an agent that work well across
games.

The diversity in the arcade learning environment poses challenges that are not
well addressed in the reinforcement learning literature. Methods that work for
well-worn test problems (Mountain Car, Grid-World, etc. ) can not necessarily
handle the additional complexity in the Arcade Learning Environment.

In this work we single out a number of classic and modern reinforcement
learning algorithms. These methods were chosen as representative methods in
their respective classes. Each of the methods we consider learns a linear
policy or value function, and so they are on equal footing with regards to the
representational power they possess. The restriction to linear methods is
party computational and partly pragmatic. Each of the methods we consider can
in principle use non-linear function approximation instead.

We also evaluated a number of high-level variations on a 5-game subset using
SARSA. These include varying discounting, decay, exploration frequency and
length. The results we present are a useful guidance to other researchers
working on the ALE environment.

Figure 1: Screen shots from the 55 games considered in this work

##  1 Environment Overview

The arcade learning environment is a wrapper around the Stella
emulatorhttp://stella.sourceforge.net/ for the Atari 2600. It wraps the
emulator in a traditional reinforcement learning interface, where a agent acts
in an environment in response to observed states and rewards. Atari 2600 games
are remarkably well suited to being used as reinforcement learning evaluation
problems; the majority of games consist of short, episodic game-play and
present a score to the player which can be interpreted as a reward without
issue. More modern video game consoles have longer game-play episodes, and
have moved away from score based game-play.

Agents interact with ALE through a set of 18 actions, for which the actions
0-5 can be considered _basic_ actions, with the remaining actions being
combinations of the basic actions. The basic action set consists of movement
in 4 directions, firing or a NO-OP action. The state presented to the agent
consists of a 210 pixel high by 160 wide screen, with each pixel a 0-127 (7
bit) color value, with the exception of a small number of games which have a
slightly different screen height. The Atari 2600 console supported a reduced
color space known as SECAM, which was used on early European TV sets. Mapping
the 128 colors to the 8 SECAM colors gives a reduced state space size without
much loss of information.

The ALE environment supports the agent acting at a rate of up to 60 steps per
second. This rate is unnecessarily rapid, and results in inhumanly jittery
play for most games. We recommend following Bellemare et al. (2013)’s
suggestion and acting at 12 steps per second. This is specified by setting
frame_skip=5 within the stellarc configuration file.

ALE is straight forward to build and run on Linux and Mac OS X environments.
Some work is required to build on Windows, so we have published a precompiled
executable on the first author’s website. The ALE software supports 61 Atari
2600 games as of version 0.4. Each game has to be specifically targeted, as
the game’s score needs to be extracted from the game’s RAM, and episode
termination conditions must be identified. Of the supported games, 5 have been
singled out by Bellemare et al. (2013) as a training set for setting hyper-
parameters, and 50 others as the test set. Of these games, three have non-
standard screen height (Carnival, Journey Escape and Pooyan).

##  2 Challenges

The arcade learning environment is substantially more complex than text-book
reinforcement learning problems. It exemplifies many of the attributes found
in other hard reinforcement learning problems. In this section we discuss a
number of practical issues which need to be addressed by agents.

###  2.1 Exploration

The need for exploration is a well known issue that must be addressed by any
reinforcement learning agent. We found that this problem is particularly
exacerbated in the arcade learning environment. When using model-free methods,
the following approaches are typically used:

#### Epsilon-greedy policy

The epsilon-greedy approach, where with a small probability ($\epsilon\approx
0.05$) a random action is taken. This method has substantial trouble with the
rapid acting rate in the arcade learning environment. Each single random
action only slightly perturbs the state of the system, resulting in very
little exploration. For example, in the game Space Invaders, the player’s ship
is restricted to 1 dimension of movement. Even then, the exploratory actions
exhibit slow movement due to random-walk like behavior. Very rarely does the
ship move from one end of the position range to the other.

More complex exploratory action sequences necessary for high-level game-play
also do not occur. For example, in the game Seaquest, the player’s submarine
must surface for air occasionally in order to obtain human-level performance.
The sequence of actions required to surface, then re-submerge, is not
performed during exploration with an epsilon-greedy approach.

In order to achieve the best results in practice with on-policy RL methods,
the $\epsilon$ parameter needs to be reduced over time. However, this is
rarely done in published research, as determining a reduction schedule
requires a tedious parameter search, and is extremely problem dependent.

Figure 2 shows the effect of varying epsilon on Seaquest. Interesting, the
standard exploration amount used in the literature ($\epsilon=0.05$) also
gives good results here. Degenerate levels ($<0.02$) of course give poor
results, but a reasonable policy is still learned for very random policies
($\epsilon>0.2$). We believe this is caused by the short time scales the agent
is acting on. The noise is averaged out over the longer time-scales that game-
play occurs at.

#### Softmax policy

Another simple approach to exploration, where Q-values are used to form a
Gibbs distribution over actions at each step. The chosen action is then a
sample from this distribution. This approach avoids the need to choose an
exploration reduction schedule, instead just requiring a scalar temperature
parameter to be set. In the arcade learning environment, we found this
approach to be unworkable in practice. The main issue being an extreme
sensitivity to the temperature. In order for it to work at all, the
temperature needs to be fine tuned for each game and each agent.

Figure 3 shows the effect of varying the temperature on Seaquest. Clearly the
agent is only successful for a small range of temperatures, roughly 0.8-2.
None of the values we tried gave results comparable to the $\epsilon$-greedy
policy at the best $\epsilon$.

#### Optimistic initialization

Perhaps the most effective approach for simplistic grid-world like-
environments is to initialize the agent so that it believes unvisited states
are unreasonably good. This encourages exploration of all states at least
once. In environments with a small number of states, such as grid-world mazes
and the like, this is extremely effective. For the arcade learning
environment, it is not clear how similar results can be achieved. The position
of the agent alone can be captured fairly directly with some state encodings,
but position is not the only property that needs to be explored. The matter is
complicated further when non-linear value function approximation is used, as
the value of states will be pushed down from their original optimistic values
during learning, even before they have been visited.

Figure 2: Effect of varying epsilon on Seaquest with a SARSA agent

Figure 3: Effect of varying temperature on Seaquest

#### Imitation learning

Another approach to exploration is to guide the agent in someway using an
outside ‘teacher’. This has been explored in depth in robotics, where it is
known as imitation learning (He et al., 2012; Price and Boutilier, 2003) or
demonstration learning (Argall et al., 2009). The simplest variant is where
the reinforcement learning agent can query the teacher, to find out what
action the teacher suggests it should take in any state. A number of practical
issues arise in the ALE setting however. A human teacher can not be used due
to the large number of games and rate of play of the agents. It would require
impractical amounts of time. Using a computerized agent observed by the
learner is the other possibility. This would require domain specific AI to be
programmed for each game.

###  2.2 Computational Resources

Stella is capable of simulating the Atari 2600 at around 7,200 frames per
second on our 3.6 GHz Xeon machine; much faster than real time (60 fps). When
running with the additional overhead of ALE, and communicating via pipes with
a separate Python process, we found that a basic agent could act at roughly
1000 actions per second. When using frame_skip=5 as discussed Section 1, which
is used in all the experiments detailed in this work, this is reduced to 200
actions per second. In practice, when including the overhead of the agents
calculations, we were able to run our agents at approximately 130 actions per
second.

Given this simulation rate, training for 5000 episodes for a single
agent/environment pair usually took 1-3 days. We believe a carefully coded C++
implementation would be several times faster than this, but even then,
simulations are quite computationally prohibitive. We would not recommend
experimenting with ALE without access to a computing cluster to run the
experiments on.

The 5,000 episode limit used by Bellemare et al. (2013) is large but
necessary. While some methods we experimented with learned reasonable
parameters in a few as 500 episodes, some were still improving at the 5,000
episode limit. This is of course very much larger than what a human player
requires, but without providing prior knowledge about general game mechanics
and spatio-temporal relationships, a reinforcement learning agent will always
be slower to learn.

###  2.3 State space size

Given the large size of the state space, running non-linear value function
learning algorithms is also quite computationally expensive. The 210x160
screen images are similar in size to datasets used in current deep learning
research (256x256), and the several million frames encountered during training
is also comparable. Even when using reduced state spaces, such as the BASIC
representation detailed in Bellemare et al. (2013), simple neural network
models are quite slow, only simulating at 1-3x real-time.

Tile coding is the most practical feature extraction technique. We also
experimented with convolutional features, where a set of predefined filters
were run over the image each step. The large number of convolutions required
was too slow, at least using OpenCV http://opencv.org/ or
Theanohttp://deeplearning.net/software/theano/ convolutional codes.

We performed our experiments using a variant of the BASIC representation,
limited to the SECAM color set. This representation is simply an encoding of
the screen with a courser grid, with a resolution of 14x16. Colors that occur
in each 15x10 block are encoded using indicator features, 1 for each of the 8
SECAM colors. Background subtraction is used before encoding, as detailed in
Bellemare et al. (2013).

##  3 Learning Algorithms

Here we briefly outline the algorithms that we tested. We are considering the
standard reinforcement learning framework; at each time step $t$ the agent
observes a state $s^{(t)}$ and reward $r^{(t)}$. The action selected at each
time step is denoted $a^{(t)}$. The state-action pair is processed using a
state encoder function into a vector $\phi(s^{(t)},a^{(t)})=\phi^{(t)}$. This
binary vector contains the BASIC SECAM representation discussed above, of
length $n$, followed by the (flattened) Cartesian product of the
representation and a 1-of-18 action indicator vector. An additional bias
feature which is always active was used.

We will consider algorithms that learn a linear value function associating a
real number with each state-action pair
($Q_{t}(s,a)=\langle\theta^{(t)},\phi(s,a)\rangle$), whose interpretation
depends on the agent. The parameter vector $\theta$ and the corresponding
function $Q$ change over time, so they are indexed by the time-step also. The
parameter vector has an associated step size parameter $\alpha$. Reward
discounting was used, denoted $\gamma$, with $\gamma=0.993$ used unless noted
otherwise.

All agents we consider will use eligibility traces $e^{(t)}$, so that they are
able to learn from delayed rewards (Singh and Sutton, 1996). This introduces a
parameter $\lambda$, which we will discuss further in Section 4.2. We
implemented them in the replacing fashion, with the following update being
used:

$e^{(t)}_{i}=\begin{cases}1&\textrm{if }\phi^{(t)}_{i}=1\\\ \gamma\lambda
e^{(t-1)}_{i}&\textrm{if }\phi^{(t)}_{i}=0\\\ \end{cases}.$ (1)

#### SARSA($\lambda$)

The first algorithm we tested was SARSA (Rummery and Niranjan, 1994). This is
perhaps the most widely used reinforcement learning algorithm. Although since
its discovery other algorithms have been developed that have better
theoretical guarantees, or better performance on specific problems, SARSA
still gives state-of-the-art performance. SARSA is defined by the update
equations:

$\theta^{(t+1)}=\theta^{(t)}+\alpha\delta^{(t)}e^{(t)}$ (2)

where:

$\delta^{(t)}=r^{(t+1)}+\gamma
Q_{t}(s^{(t+1)},a^{(t+1)})-Q_{t}(s^{(t)},a^{(t)}).$ (3)

#### Q($\lambda$)

We also tested against Q-learning (Watkins, 1989), the off-policy alternative
to SARSA. It differs by the update for $\delta$:

$\delta^{(t)}=r^{(t+1)}+\gamma\max_{a^{\prime}}Q_{t}(s^{(t+1)},a^{\prime})-Q_{t}(s^{(t)},a^{(t)})$
(4)

Q-learning can potentially learn a better policy than SARSA in games where
death can easily be caused by the random moves included in the
$\epsilon$-greedy policies learned by on-policy methods. The downside is a
stochastic policy can give better results in some games. Off-policy methods
are also known to diverge in some cases when function approximation is used
(Baird et al., 1995).

#### ETTR($\lambda$)

We also implemented a _shorted-sighted_ agent, that aims to minimize the
expected time to next positive reward (ETTR), instead of the discounted
expected future reward optimized by the other agents. This has the advantage
of potentially being easier to learn, as it gets a non-noisy signal whenever
it actually reaches a positive reward. The disadvantage is a lack of long term
planning and poorer risk-aversion. Within the temporal difference framework,
with decaying eligibility traces, ETTR uses the following update for $\delta$:

$\delta^{(t)}=\begin{cases}-Q_{t}(s^{(t)},a^{(t)})+1\\\
\quad+Q_{t}(s^{(t+1)},a^{(t+1)})&\textrm{if }r^{(t+1)}=0\\\
-Q_{t}(s^{(t)},a^{(t)})&\textrm{if }r^{(t+1)}>0\end{cases}.$ (5)

No discounting is used. We only applied ETTR to games where the reward
structure contained positive rewards as the primary motivator of the agent.
This excluded 10 of the 55 games, mainly games where negative reward was given
until a goal was achieved (Boxing, Double Dunk, Enduro, Fishing Derby, Ice
Hockey, Journey Escape, Pong, Private Eye, Skiing, Tennis). These games could
be approached by a similar scheme, where expected length of episode is
minimized instead.

#### R($\lambda$)

Another class of reinforcement learning agents seek to optimize the expected
reward per time step instead. R-learning is the primary example of such a
method in the off-policy case (Schwartz, 1993). Such a formulation avoids the
need for discounting, but introduces an additional step size parameter
$\beta$, which controls the rate at which an estimate of the expected reward
per time-step $\rho$ changes. The update equations are:

$\delta^{(t)}=r^{(t+1)}-\rho^{(t)}+\max_{a^{\prime}}Q_{t}(s^{(t+1)},a^{\prime})-Q_{t}(s^{(t)},a^{(t)}),$
(6)

where $\rho$ is updated whenever we take on policy actions, with:

$\begin{split}\rho^{(t+1)}={}&\rho^{(t)}+\beta\left(r^{(t+1)}-\rho^{(t)}\right.\\\
+&\max_{a}Q_{t}(s^{(t+1)},a)-\left.\max_{a^{\prime}}Q_{t}(s^{(t)},a^{\prime})\right).\end{split}$
(7)

Expected reward methods have seen more use in the Actor-Critic setting (see
below).

#### GQ($\lambda$)

The _gradient temporal difference_ (GTD, Sutton et al. (2008)) class of
algorithms are an attempt to improve the convergence properties and robustness
of classical temporal difference algorithms. They phrase learning as a
stochastic gradient descent problem, for which stronger convergence properties
are known. We applied the GQ($\lambda$) algorithm (Maei and Sutton, 2010) to
the ALE environment. GTD-style algorithms maintain a weight vector $w$ as well
as a parameter vector $\theta$. Both vectors are updated at each step, with
step sizes $\alpha$ and $\beta$ respectively. The update equations are
modified as:

$\begin{split}\delta^{(t)}&=r^{(t+1)}+\gamma
E_{a}\left[Q_{t}(s^{(t+1)},a)\right]-Q_{t}(s^{(t)},a^{(t)}),\\\
\theta^{(t+1)}&=\theta^{(t)}+\alpha\left(\delta^{(t)}e^{(t)}-\gamma(1-\lambda)\langle
w^{(t)},e^{(t)}\rangle\bar{\phi}^{(t+1)}\right),\\\
w^{(t+1)}&=w^{(t)}+\beta\left(\delta^{(t)}e^{(t)}-\langle
w^{(t)},\phi^{(t)}\rangle\phi^{(t)}\right).\end{split}$ (8)

The expectation over actions is taken with respect to the epsilon-greedy
policy.

#### Actor-Critic

Actor-Critic methods (Barto et al., 1983) decouple the policy and value
function predictors. The _critic_ is used for value function estimation, and
the _actor_ maintains the policy. We will consider the simplest case, where
both are linear. The key advantage of (linear) actor critic methods is that we
can use different learning rate parameters for the actor and the critic.
Typically the actor (step length $\beta$) is set to evolve slower than the
critic (step length $\alpha$), so that the value function estimates have more
time to stabilize as the policy changes. We tested a non-standard variant that
uses epsilon-greedy policies, rather than the Gibbs/Boltzmann methods more
common in the literature. This choice was made based on the results in Section
2.1. Let $\nu$ denote the critic’s weight vector. Then $\delta$ remains the
same as for SARSA, but with the value function approximation changed to
($Q_{t}(s,a)=\langle\nu^{(t)},\phi(s,a)\rangle$). The parameter updates are:

$\begin{split}\nu^{(t+1)}&=\nu^{(t)}+\alpha\delta^{(t)}e^{(t)},\\\
\theta^{(t+1)}&=\theta^{(t)}+\beta\delta^{(t)}e^{(t)}.\\\ \end{split}$ (9)

Action selection by the actor is done as with SARSA, by taking the action that
maximizes $\langle\theta^{(t)},\phi(s^{(t)},a)\rangle$.

#### Other methods

Several families of methods could not be tested due to computational
considerations. We briefly discuss some of them here. Least-squares methods
such as LSPI (Lagoudakis and Parr, 2003) are perhaps the most prominent
alternative within the class of linear value function approximation methods.
Unfortunately, they require quadratic time operations in the state vector
size. Even though we are using the smallest screen based state representation
considered in the literature for ALE, the quadratic time operations were too
slow for us to use.

There is also a large literature on methods that store past visited
state/action information, the main example being Fitted Q-iteration (Gordon,
1995). Given the millions of states visited during training, only a small
subset of the history can tractably be stored for ALE. Investigating tractable
variants of history-based methods for ALE is an interesting avenue of
research, but outside the scope of this work.

##  4 Results

The results in this Section were computed by averaging a number of trials.
Each trial consisted of running 5000 episodes while in training mode, then 500
episodes in test mode where the learned policy is executed. For on-policy
methods, the test episodes included the exploration steps, and for off-policy,
the test policy was purely greedy.

###  4.1 Discounting Comparison

Figure 4 shows the effect of differing discounting amount on the training set
games for SARSA. The optimal discounting factor is not consistent between
games, with the best value differing quite substantially between Seaquest and
Asterix for example. A general trend is apparent; using extremely high
discounting ($0.5$-$0.9$) is ineffective, and too little discounting
($>0.9999$) also gives poor results. We encountered convergence problems when
no discounting was used, so a value of $\gamma=1$ is omitted from the plot.
The roughly convex dependence on discounting is consistent with the literature
on simpler environments.

Figure 4: Effect of varying discounting on training set games

###  4.2 Decay Comparison

Figure 5 shows the effect of varying decay on the training set games for
SARSA. There is no clear best decay, although values of both 0 and 1 are poor.
Decay in the range 0.5-0.9 seems most effective. There is less sensitivity to
the exact value of the decay parameter $\lambda$ than the discounting
parameter $\gamma$.

Figure 5: Effect of varying decay on training set games

###  4.3 Exploration Periods, on/off policy

We experimented with policies that extend the amount of exploration on
Seaquest. We tested a modified $\epsilon$-greedy approach, where whenever the
random policy was chosen, that action was then taken for more than a single
step. We varied this exploration length between 1 and 6 steps. We got somewhat
mixed results. Figure 6 shows that for SARSA and Q-learning, depending on the
value of $\epsilon$, length two exploration periods can be better than
standard $\epsilon$-greedy, but the difference is close to the noise floor.
Otherwise there is a downward trend in the average reward obtained, with a
small up-tick around length six.

Figure 6: Longer exploration periods with SARSA on Seaquest for three
$\epsilon$ values, for SARSA (top) and Q-Learning (bottom)

###  4.4 Algorithm Comparison

For each algorithm described in Section 3, we ran 5 trials on each of the
games where it could be applied. Two issues arose that needed to be addressed.
Several game & algorithm combinations exhibited behaviour where the agent
would make no progress and consistently hit the episode limit with zero
reward. This happened in particular on Zaxxon and Montezuma’s revenge. These
cases were excluded from the comparisons. We also encountered divergent
behaviour on Q-learning roughly 40% of the time, and on GQ a surprising 70% of
the time. Runs where divergence occured were excluded.

We consider SARSA as the baseline. First we compare the performance and
consistency of the methods. For each algorithm we computed the average reward
during the test episodes for each trail and game. We then computed the
relative performance of the method as the average over each game of the ratio
of the trial average against the baseline. The average was computed over the
middle 90% of games to remove outliers. We also computed the test reward
standard deviation (SD) across trails for each game as a measure of
consistency. The median (Q2) and 1st and 3rd quartiles of the SD across games
is given together with the relative performance in Table 1.

Algorithm | Rel. perf | SD Q1 | SD Q2 | SD Q3
---|---|---|---|---
SARSA | 1.00 | 0.26 | 0.41 | 0.55
AC | 0.99 | 0.23 | 0.45 | 0.58
ETTR | 1.03 | 0.32 | 0.44 | 0.58
GQ | 0.65 | 0.14 | 0.42 | 0.81
Q | 0.82 | 0.27 | 0.50 | 0.85
R | 0.96 | 0.26 | 0.42 | 0.59
Table 1: Algorithm performance and consistency comparison

We can see that SARSA, R, AC and ETTR all performed similarly in the relative
performance metric. This suggests that the additional complexity of AC and the
narrower focus of ETTR did not help. Our hypothesis that ETTR would perform
more consistently is also not supported by the standard deviation results. The
off-policy methods Q-learning and GQ-learning did not perform well here,
showing significantly worse results.

The large variances we see can potentially effect the relative performance
figures significantly. A more robust method of comparing a pair of algorithms
is to count the number of games for which the average test reward is higher
for one game than the other. Table 2 contains these comparisons. Cases where
one or both games failed to converge or timed-out are excluded from the table.

| SARSA | AC | ETTR | GQ | Q | R
---|---|---|---|---|---|---
SARSA | - | 28/25 | 22/21 | 49/5 | 44/10 | 30/23
AC | 25/28 | - | 18/25 | 49/4 | 38/17 | 26/25
ETTR | 21/22 | 25/18 | - | 40/1 | 36/8 | 22/20
GQ | 5/49 | 4/49 | 1/40 | - | 8/37 | 5/49
Q | 10/44 | 17/38 | 8/36 | 37/8 | - | 13/41
R | 23/30 | 25/26 | 20/22 | 49/5 | 41/13 | -
Table 2: Pairwise higher/lower test reward comparison

The results in this table support the inferences made using the relative
performance measure. SARSA appears better than R-learning by a larger margin,
possibly due to chance, but otherwise the on-policy methods show similar
results.

We also computed the correlation coefficient between each pair of algorithms,
considering only trials for which both methods finished. The results are shown
in Table 3.

| SARSA | AC | ETTR | GQ | Q | R
---|---|---|---|---|---|---
SARSA | 1.00 | 0.27 | -0.12 | -0.86 | -0.46 | 0.38
AC | 0.27 | 1.00 | -0.27 | -0.75 | -0.10 | -0.06
ETTR | -0.12 | -0.27 | 1.00 | -0.69 | -0.76 | -0.21
GQ | -0.86 | -0.75 | -0.69 | 1.00 | 0.71 | -0.79
Q | -0.46 | -0.10 | -0.76 | 0.71 | 1.00 | -0.07
R | 0.38 | -0.06 | -0.21 | -0.79 | -0.07 | 1.00
Table 3: Algorithm correlation

As would be expected, there is little correlation between SARSA and ETTR,
suggesting they perform well on differing problems. There is moderate
correlation between SARSA and AC, which would be expected given there
similarities. Q and GQ learning are highly correlated, whereas they are both
negatively correlated with R-learning. This suggests the behaviour of
R-learning is closer to the on-policy methods, given it’s correlation with
SARSA.

One of the most relevant properties of a learning algorithm is its speed of
convergence. For each method, we computed the average reward over the last 500
episodes of the training episodes, and compared it to the average across the
500 episodes preceding those. If the average was within 10%, we considered the
method to have converged. The percentage of converged trails out of those that
finished for each method was: SARSA: 85%; AC: 80%; ETTR: 84%; GQ: 80%; Q: 82%;
R: 85%. The convergence rates are very similar across the methods considered.
These results also suggests that the step size constants that we chose were
reasonable.

##  5 Discussion

###  5.1 Instability

We were surprised by the variability in the results we saw. The same agent
with the same hyper-parameters can exhibit learning curves that show little
resemblance to one another. On games with fairly stable behaviour, there is
often a $\pm 50\%$ swing in results between trials. On less stable games such
as Zaxxon, the agents could get stuck on degenerate low reward policies. This
makes comparisons between agents difficult. It is unsatisfying to average over
such effects, as it hides the true nature of what is being learned.

###  5.2 Divergence

Off-policy methods such as Q-learning are known to have convergence issues
when function approximation is used, even in the linear case that we
considered. This is normally considered a theoretical issue when
$\epsilon$-greedy policies are used, as the difference between
$\epsilon$-greedy and pure on-policy is minor. However we did see a high
probability of divergence in practice with Q-learning, as detailed in the
results section. Surprisingly, we also experienced divergence with
GQ($\lambda$), despite theoretical convergence results. Our implementation did
not use the projection step that they detail. Their theory requires the
projection, however they suggest it is not needed in practice. We believe that
it is the likely cause of the convergence issues, combined with perhaps a
sensitivity to the step parameters.

Interestingly, The off-policy R-learning method did not have the same
divergence issues, and it performed nearly as well as SARSA in our
experiments. It appears the best choice among the off-policy methods
considered.

###  5.3 Exploration

Even within the class of linear value function approximation, we clearly see
in our experiments that sub-optimal policies are being learned in many trials.
Plotting the reward per episode over time shows that the average reward per
episode stops improving, but at a level below the optimal reward level, and at
a level varying between trials. We speculate that the epsilon-greedy policy is
not resulting in much exploration, rather its effect is to introduce jitter
that helps prevents degenerate policies from being followed, such as repeated
ineffective actions, or behavioural loops. Based on the need for large
learning rates, it seems that the noise introduced into the value function by
taking large steps in parameter space is contributing far more to exploration.

###  5.4 Learned policies

Looking at raw point scores does not give a clear indication of what a policy
is actually doing. To get a better idea of the sorts of policies that were
being learned we examined videos of the each agent’s gameplay on a number of
games. For the majority of games we examined, the learned policies consisted
of a very simple sequence of movements together with holding down the fire
key. For example, with Seaquest, holding fire together with alternating left-
down and right-down actions appeared to be the policy learned by SARSA most of
the time. Similar behaviour was learned on Krull, where repeatedly jumping
gives a very high score.

We noticed that the linear algorithms we considered were capable of learning
to move to a particular location on the screen at the start of the episode,
then from there repeating a set of actions. For example, in Gopher, the AC
agent learned to move to a fixed point right of the starting location, then to
repeatedly trigger the fire action. This appeared to be effective due to a
simplistic enemy AI. For Zaxxon, you die after 10 seconds with a score of zero
unless you move to the center of the screen, away from the starting position
on the left. This turned out to be an extremely difficult policy to learn, as
the negative reinforcement (death) occurs long after the actions. All the
algorithms we looked at had some runs where they completely failed to learn
that policy, or discovered it then later abandoned it part way through
training.

###  5.5 Choosing the best algorithm for an environment

The comparison on diverse environments we have performed allows us to see
which environments each algorithm is best at and which it is worse at. First
we consider the off-policy algorithms. They tended to perform best on problems
where exploration actions could have major negative effect on the outcome,
which is consistent with our expectations. For example, for Zaxxon mentioned
above, they were more consistently able to stay in the safe region of the
screen. Looking in particular at Q-learning, we see a large improvement in
performance when switching from the training regime to the test regime (where
no epsilon randomness is used) for Seaquest, Assault, Time pilot, Asterix, and
smaller improvements for many other games. For some games, a sharp drop in
performance occurs, such as for Tutankham and Space Invaders.

Games that require a level of random left and right movement were particularly
a problem, such as the Crazy Climber game, where the player just has to move
forward most of the time, but occasional left or right movement is required to
go around obstacles. Potentially a soft-max policy could avoid these issues,
but as mentioned in Section 2.1, the temperature parameter needs to be fine-
tuned separately for each game due to the large differences in score
magnitudes.

The ETTR method was able to outperform the other algorithms for a number of
games, including Gopher, Q*bert and Road Runner. These games had a common
pattern where you need to react to movement in squares adjacent to the player
character. This pattern was shared by a number of other games also, so its not
clear if any conclusions can be drawn from that. The ETTR agent had
significantly less risk aversion than the other algorithms considered, which
was the expected behaviour. For example on Seaquest, instead of moving the
player’s submarine to the bottom of the screen which is the safest location,
it stayed near the top of the screen where more rewards are available. In
terms of actual scores, we didn’t see significant negative effects from the
lack of risk aversion in the games we looked at.

The AC agent was able to outperform all the other methods on a number of
problems also. It had significantly better scores on Boxing, Alien and
Frostbite. The AC method can also be tuned to a greater degree than SARSA, so
it is a good choice when tuning on a per-problem basis is used. Additional
tuning can also be a downside, although based on it’s consistent performance
that doesn’t appear to be the case here.

##  6 Related Work on the Arcade Learning Environment

The arcade learning environment is relatively new, so little work so far
directly targets it. (Bellemare et al., 2013) introduces the environment. They
consider the performance of several state representations with SARSA.

Naddaf (2010) presents some tables comparing decay and learning rates for 4 of
the training set games. Our plots in Section 4 are more comprehensive. Mnih et
al. (2013) consider the use of deep learning methods on a subset of 7 games,
showing super-human performance on 3 games.

## Conclusion

The results in this paper provide a guide other researchers working with the
Arcade Learning Environment. We give guidance on most effective decay,
discounting and epsilon-greedy constants. We also provide a comparison of
standard RL algorithms, showing that a set of commonly used linear on-policy
algorithms give similar performance. We also show that some common off-policy
methods have serious issues with the complexity of the ALE environment, and we
give recommendations based on our empirical results on the RL algorithms that
should used in different game environments.

## References

  * Argall et al. (2009) Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning.  A survey of robot learning from demonstration.  _Robotics and Autonomous Systems_ , 57(5):469–483, 2009.
  * Baird et al. (1995) Leemon Baird et al.  Residual algorithms: Reinforcement learning with function approximation.  In _ICML_ , pages 30–37, 1995.
  * Barto et al. (1983) A.G. Barto, R.S. Sutton, and C.W. Anderson.  Neuronlike adaptive elements that can solve difficult learning control problems.  _Systems, Man and Cybernetics, IEEE Transactions on_ , SMC-13(5):834–846, 1983.  ISSN 0018-9472.
  * Bellemare et al. (2013) M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling.  The arcade learning environment: An evaluation platform for general agents.  _Journal of Artificial Intelligence Research_ , 2013.
  * Gordon (1995) Geoffrey J Gordon.  Stable function approximation in dynamic programming.  Technical report, DTIC Document, 1995.
  * He et al. (2012) He He, Hal Daume III, and Jason Eisner.  Imitation learning by coaching.  In _Advances in Neural Information Processing Systems 25_ , pages 3158–3166, 2012.
  * Lagoudakis and Parr (2003) Michail G Lagoudakis and Ronald Parr.  Least-squares policy iteration.  _The Journal of Machine Learning Research_ , 4:1107–1149, 2003.
  * Maei and Sutton (2010) Hamid Reza Maei and Richard S Sutton.  Gq ($\lambda$): A general gradient algorithm for temporal-difference prediction learning with eligibility traces.  In _Proceedings of the Third Conference on Artificial General Intelligence_ , 2010.
  * McCallum (1996) Andrew Kachites McCallum.  _Reinforcement learning with selective perception and hidden state_.  PhD thesis, University of Rochester, 1996.
  * Mnih et al. (2013) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning.  Technical report, DeepMind Technologies, 2013.
  * Naddaf (2010) Yavar Naddaf.  _Game-Independent AI Agents for Playing Atari 2600 Console Games_.  Masters, University of Alberta, 2010.
  * Price and Boutilier (2003) Bob Price and Craig Boutilier.  Accelerating reinforcement learning through implicit imitation.  _J. Artif. Intell. Res.(JAIR)_ , 19:569–629, 2003.
  * Rummery and Niranjan (1994) Gavin A Rummery and Mahesan Niranjan.  _On-line Q-learning using connectionist systems_.  University of Cambridge, Department of Engineering, 1994.
  * Schwartz (1993) Anton Schwartz.  A reinforcement learning method for maximizing undiscounted rewards.  In _ICML_ , volume 93, pages 298–305, 1993.
  * Singh and Sutton (1996) Satinder P Singh and Richard S Sutton.  Reinforcement learning with replacing eligibility traces.  _Machine learning_ , 22(1-3):123–158, 1996.
  * Sutton et al. (2008) Richard S Sutton, Cs. Szepesvari, and H. R. Maei.  A convergent o(n) algorithm for off-policy temporal-difference learning with linear function approximation.  In _Advances in Neural Information Processing Systems_ , 2008.
  * Watkins (1989) Christopher John Cornish Hellaby Watkins.  _Learning from delayed rewards._ PhD thesis, University of Cambridge, 1989.


# Sums and products of symplectic eigenvalues

Tanvi Jain  Indian Statistical Institute, New Delhi 110016, India
<EMAIL_ADDRESS>
###### Abstract.

For every $2n\times 2n$ real positive definite matrix $A,$ there exists a real
symplectic matrix $M$ such that $M^{T}AM=\text{\rm diag}(D,D),$ where $D$ is
the $n\times n$ positive diagonal matrix with diagonal entries
$d_{1}(A)\leq\cdots\leq d_{n}(A).$ The numbers $d_{1}(A),\ldots,d_{n}(A)$ are
called the symplectic eigenvalues of $A.$ We derive analogues of Wielandt’s
extremal principle and multiplicative Lidskii’s inequalities for symplectic
eigenvalues.

###### Key words and phrases:

symplectic eigenvalues, Wielandt’s extremal principle, multiplicative
Lidskii’s inequalities

###### 2010 Mathematics Subject Classification:

15A45, 47A75, 49S05, 81P45

##  1\. Introduction

Let $J$ be the $2n\times 2n$ matrix

$J=\begin{bmatrix}O&I_{n}\\\ -I_{n}&O\end{bmatrix},$ (1)

where $I_{n}$ is the $n\times n$ identity matrix. Consider the antisymmetric
bilinear form $(\cdot,\cdot)$ on $\mathbb{R}^{2n}$ given by

$(x,y)=\langle x,Jy\rangle,\ x,y\in\mathbb{R}^{2n}.$

We call this bilinear form to be the symplectic inner product on
$\mathbb{R}^{2n}.$ A $2n\times 2n$ real matrix that preserves the symplectic
inner product is called a symplectic matrix, i.e., $M$ is a symplectic matrix
if

$\langle Mx,JMy\rangle=\langle x,Jy\rangle$

for all $x,y$ in $\mathbb{R}^{2n}.$ This is equivalent to saying that $M$ is
symplectic if and only if $M^{T}JM=J.$ We denote by $Sp(2n)$ the collection of
all $2n\times 2n$ real symplectic matrices. This is a Lie group under matrix
multiplication. A basic theorem in symplectic matrix theory, generally known
as Williamson’s theorem tells us that for every $2n\times 2n$ real positive
definite matrix $A,$ there exists a $2n\times 2n$ symplectic matrix $M$ such
that

$M^{T}AM=\begin{bmatrix}D&O\\\ O&D\end{bmatrix},$ (2)

where $D$ is the $n\times n$ positive diagonal matrix with diagonal entries
$d_{1}(A)\leq\cdots\leq d_{n}(A).$ These positive numbers are complete
invariants for $A$ under the action of the symplectic group. We call them the
symplectic eigenvalues of $A.$ Symplectic eigenvalues are important in
different areas of mathematics and physics such as symplectic geometry,
symplectic topology, both classical and quantum mechanics and quantum
information. See, for instance, [2, 9, 10, 11, 13, 15]. In particular,
applications of symplectic eigenvalues in the newer area of quantum
information has driven much of the recent work on this topic [2, 13, 15, 18,
20].

For over a century, there has been an extensive interest in the study of
variational principles for eigenvalues of Hermitian matrices and the
relationships between the eigenvalues of $A,$ $B$ and $A+B.$ We refer the
reader to Chapter III of [3], Chapter 5 of [12] and to the excellent
exposition [4] for detailed accounts of these topics. If $A$ is a $2n\times
2n$ real positive definite matrix, then $\imath A^{1/2}JA^{1/2}$ is a
Hermitian matrix. It can be seen that $A$ has symplectic eigenvalues
$d_{1},\ldots,d_{n}$ if and only if $\imath A^{1/2}JA^{1/2}$ has eigenvalues
$\pm d_{1},\ldots,\pm d_{n}.$ So, in principle, it could be possible to derive
the properties of symplectic eigenvalues from the well-known properties of
eigenvalues of Hermitian matrices. But due to the complicated form of the
matrix $\imath A^{1/2}JA^{1/2},$ it is often not feasible to do so. The theory
of symplectic eigenvalues also differs from the eigenvalue theory because of
the differences in the symplectic group and the orthogonal group. It is very
remarkable that inspite of some major differences, we can find symplectic
analogues to many of the classical results in eigenvalue theory [6, 10, 13,
14]. Two versions of symplectic maxmin principles, analogous to the Courant-
Fischer-Weyl minmax principle can be found in the literature. See [9, 6, 8].
These principles are in turn used to give symplectic analogues of an
interlacing theorem, monotonicity principle and Weyl’s inequalities. In [10],
Hiroshima proved the inequalities
$\sum_{j=1}^{k}d_{j}(A+B)\geq\sum_{j=1}^{k}\left(d_{j}(A)+d_{j}(B)\right),$
$1\leq k\leq n$; and these inequalities were generalised to Lidskii’s type
inequalities in [14] by using the analyticity of symplectic eigenvalues. A
symplectic analogue of the Ky Fan maximum principle was proved in [10]. An
important extremal principle that includes the Courant-Fischer-Weyl principle
as well as the Ky Fan maximum principle for eigenvalues of Hermitian matrices
is the Wielandt extremal principle. This played an important role in the
development of the celebrated Horn’s conjecture. It gives us an expression for
arbitrary sum of $k$ eigenvalues of Hermitian matrices. If $A$ is an $n\times
n$ Hermitian matrix with eigenvalues
$\lambda_{1}(A)\leq\cdots\leq\lambda_{n}(A),$ then for all $1\leq k\leq n$ and
$1\leq i_{1}<\cdots<i_{k}\leq n,$

$\sum\limits_{j=1}^{k}\lambda_{i_{j}}(A)={\underset{\underset{\text{\rm
dim}\mathcal{W}_{j}=n-i_{j}+1}{\mathcal{W}\supset\cdots\supset\mathcal{W}_{k}}}{\max}}{\underset{\underset{\textrm{orthonormal}}{x_{j}\in\mathcal{W}_{j}}}{\text{\rm
min}}}\sum\limits_{j=1}^{k}\langle x_{j},Ax_{j}\rangle.$

We give a symplectic analogue of the Wielandt principle. A subset
$\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}$ of $\mathbb{R}^{2n}$ is called
symplectically orthonormal if

$\langle x_{i},Jx_{j}\rangle=\langle y_{i},Jy_{j}\rangle=0\textrm{ and
}\langle x_{i},Jy_{j}\rangle=\delta_{ij}$

for all $i,j=1,\ldots,k.$

######  Theorem 1.

Let $A$ be a $2n\times 2n$ real positive definite matrix with symplectic
eigenvalues $d_{1}\leq\cdots\leq d_{n}.$ Let $1\leq k\leq n$ and $1\leq
i_{1}<\cdots<i_{k}\leq n.$ Then

$\sum\limits_{j=1}^{k}d_{i_{j}}={\underset{\underset{\text{\rm
dim}\,\mathcal{W}_{j}=2n-i_{j}+1}{\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}}}{\max}}{\underset{\underset{\underset{\textrm{orthonormal}}{\textrm{symplectically}}}{x_{j},y_{j}\in\mathcal{W}_{j}}}{\text{\rm
min}}}\sum\limits_{j=1}^{k}\frac{\langle x_{j},Ax_{j}\rangle+\langle
y_{j},Ay_{j}\rangle}{2}.$ (3)

This gives us an alternative proof of the Lidskii type inequalities for
symplectic eigenvalues that were first proved in [14].

######  Corollary 2.

Let $A$ and $B$ be two $2n\times 2n$ real positive definite matrices. Suppose
that $A,$ $B$ and $A+B$ have symplectic eigenvalues $d_{1}(A)\leq\cdots\leq
d_{n}(A),$ $d_{1}(B)\leq\cdots\leq d_{n}(B)$ and $d_{1}(A+B)\leq\cdots\leq
d_{n}(A+B),$ respectively. Then for all $k=1,\ldots,n$ and $1\leq
i_{1}<\cdots<i_{k}\leq n,$

$\sum\limits_{j=1}^{k}d_{i_{j}}(A+B)\geq\sum\limits_{j=1}^{k}d_{i_{j}}(A)+\sum\limits_{j=1}^{k}d_{j}(B).$
(4)

We also study the multiplicative versions of Lidskii’s inequalities for
symplectic eigenvalues. For two positive definite matrices $A$ and $B,$ their
geometric mean is the positive definite matrix

$A\\#B=A^{1/2}\left(A^{-1/2}BA^{-1/2}\right)^{1/2}A^{1/2}.$

This was introduced by Pusz and Woronowicz in [19] and have important
connections with various areas such as physics, computer science, matrix
analysis and Riemannian geometry [5]. Suppose $A$ and $B$ are two $2n\times
2n$ real positive definite matrices. Let $d_{1}(A)\leq\cdots\leq d_{n}(A),$
$d_{1}(B)\leq\cdots\leq d_{n}(B)$ and $d_{1}(A\\#B)\leq\cdots\leq
d_{n}(A\\#B)$ be the symplectic eigenvalues of $A,$ $B$ and $A\\#B,$
respectively. The following relationships were proved in Theorem 3 of [6]: For
all $k=1,\ldots,n,$

$\prod\limits_{j=1}^{k}d_{j}^{2}(A\\#B)\geq\prod\limits_{j=1}^{k}d_{j}(A)d_{j}(B)$

and

$\prod\limits_{j=k}^{n}d_{j}^{2}(A\\#B)\leq\prod\limits_{j=k}^{n}d_{j}(A)d_{j}(B).$

As a consequence of our analysis, we can extend these inequalities to
arbitrary products of symplectic eigenvalues analogous to the multiplicative
inequalities proved by Lidskii for eigenvalues of two positive definite
matrices [16].

######  Theorem 3.

Let $A$ and $B$ be two $2n\times 2n$ real positive definite matrices. Then for
all $k=1,\ldots,n$ and $1\leq i_{1}<\cdots<i_{k}\leq n$

$\prod\limits_{j=1}^{k}d_{i_{j}}(A)d_{j}(B)\leq\prod\limits_{j=1}^{k}d_{i_{j}}^{2}(A\\#B)\leq\prod\limits_{j=1}^{k}d_{i_{j}}(A)d_{n-j+1}(B).$
(5)

In Section 2, we recall some basic facts on symplectic spaces and symplectic
eigenvalues, and introduce some terminology and notation. We prove Theorem 1
and Corollary 2 in Section 3. In the process we give a third version of the
maxmin principle for symplectic eigenvalues. Theorem 3 is proved in Section 4.

##  2\. Preliminaries

A subspace $\mathcal{V}$ of $\mathbb{R}^{2n}$ is called a symplectic space if
for every $x\in\mathcal{V},$ there exists a $y\in\mathcal{V}$ such that
$\langle x,Jy\rangle\neq 0.$ It can be verified that every symplectic space
has even dimension. Let $\mathcal{V}$ be a symplectic subspace of
$\mathbb{R}^{2n}$ and let $S$ be a nonempty subset of $\mathcal{V}.$ Then the
set

$S^{\perp_{s}}=\\{y\in\mathcal{V}:\langle x,Jy\rangle=0\textrm{ for all }x\in
S\\}$

is called the symplectic complement of $S$ in $\mathcal{V}.$ For any given
nonempty set $S,$ $S^{\perp_{s}}$ is always a subspace of $\mathcal{V}.$ If
$S$ is a subspace of $\mathcal{V},$ then $\text{\rm dim}\,S+\text{\rm
dim}\,S^{\perp_{s}}=\text{\rm dim}\mathcal{V}.$ A subspace $\mathcal{U}$ of a
symplectic space $\mathcal{V}$ is symplectic if and only if
$\mathcal{U}\cap\mathcal{U}^{\perp_{s}}=\\{0\\}.$ In this case, $\mathcal{V}$
is the direct sum of $\mathcal{U}$ and its symplectic complement. Two vectors
$x$ and $y$ are said to be skew-orthogonal if their symplectic inner product
$\langle x,Jy\rangle=0.$ A nonempty set is called skew-orthogonal if all its
vectors are mutually skew-orthogonal. A skew-orthogonal subspace of
$\mathbb{R}^{2n}$ is called isotropic. A subspace $\mathcal{V}$ is isotropic
if and only if $\mathcal{V}^{\perp_{s}}\supseteq\mathcal{V}.$ Clearly, the
dimension of an isotropic subspace of a $2m$-dimensional symplectic space is
always less than or equal to $m.$ A pair of vectors $(x,y)$ is said to be
normalised symplectic pair if $\langle x,Jy\rangle=1,$ and a set
$\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}$ is said to be symplectically
orthonormal if

$\langle x_{i},Jx_{j}\rangle=0=\langle y_{i},Jy_{j}\rangle$

and

$\langle x_{i},Jy_{j}\rangle=\delta_{ij}$

for all $i,j=1,\ldots,k.$ Every symplectically orthonormal set is linearly
independent. If $\mathcal{V}$ is a $2k$-dimensional symplectic space and
$\\{x_{1},\ldots,$ $x_{k},y_{1},\ldots,y_{k}\\}$ is a symplectically
orthonormal subset of $\mathcal{V},$ we call it to be a symplectic basis of
$\mathcal{V}.$ The columns of a $2n\times 2n$ symplectic matrix form a
symplectic basis of $\mathbb{R}^{2n}.$ A basis of a symplectic space
$\mathcal{V}$ that is symplectic as well as orthonormal is called an
orthosymplectic basis of $\mathcal{V}.$ The standard basis
$\\{e_{1},\ldots,e_{2n}\\}$ of $\mathbb{R}^{2n}$ is an orthosymplectic basis
of $\mathbb{R}^{2n}.$ For more details on symplectic spaces we refer the
reader to [2, 9]. Let $\mathcal{V}$ be a symplectic space and let
$\mathcal{B}=\\{u_{1},\ldots,u_{m},v_{1},\ldots,v_{m}\\}$ be a symplectic
basis of $\mathcal{V}.$ We define a new inner product
$\langle\cdot,\cdot\rangle_{\mathcal{B}}$ on $\mathcal{V}$ as follows, (see
[8]): if $x$ and $y$ are two vectors of $\mathcal{V},$ we can write $x$ and
$y$ as

$x=\sum\limits_{i=1}^{m}\left(\alpha_{i}u_{i}+\beta_{i}v_{i}\right)\textrm{
and }y=\sum\limits_{i=1}^{m}\left(\gamma_{i}u_{i}+\delta_{i}v_{i}\right).$

Then

$\langle
x,y\rangle_{\mathcal{B}}=\sum\limits_{i=1}^{m}\left(\alpha_{i}\gamma_{i}+\beta_{i}\delta_{i}\right).$

We denote the norm of $x$ in this inner product by $\|x\|_{\mathcal{B}}.$
Clearly, $\mathcal{B}$ is an orthosymplectic basis of $\mathcal{V}$ under the
inner product $\langle\cdot,\cdot\rangle_{\mathcal{B}}$ and the symplectic
inner product $(\cdot,\cdot).$ We call this $\mathcal{B}$-orthosymplectic
basis of $\mathcal{V}.$ We define the $\mathcal{B}$-complement of $x$ to be
the vector

$x^{\prime}=\sum\limits_{i=1}^{m}\left(-\beta_{i}u_{i}+\alpha_{i}v_{i}\right).$

One can see that $x^{\prime\prime}=-x$ for every $x\in\mathcal{V},$ and
$u_{j}^{\prime}=v_{j}$ and $v_{j}^{\prime}=-u_{j},$ $1\leq j\leq m.$ Also

$\langle x,y\rangle_{\mathcal{B}}=\langle x,Jy^{\prime}\rangle=\langle
y,Jx^{\prime}\rangle=\langle x^{\prime},y^{\prime}\rangle_{\mathcal{B}}.$ (6)

So, if $\\{x_{1},\ldots,x_{k}\\}$ is a skew-orthogonal,
$\mathcal{B}$-orthonormal subset of $\mathcal{V},$ then
$\\{x_{1},\ldots,x_{k},x_{1}^{\prime},\ldots,x_{k}^{\prime}\\}$ is a
$\mathcal{B}$-orthosymplectic (symplectic as well as
$\mathcal{B}$-orthonormal) set. For a subspace $\mathcal{W}$ of $\mathcal{V},$
the $\mathcal{B}$-complement of $\mathcal{W}$ is the set

$\mathcal{W}^{\prime}=\\{x^{\prime}:x\in\mathcal{W}\\}.$

Clearly, $\mathcal{W}^{\prime}$ is also a subspace of $\mathcal{W}$ and
$\text{\rm dim}\mathcal{W}^{\prime}=\text{\rm dim}\mathcal{W}.$ We denote the
intersection $\mathcal{W}\cap\mathcal{W}^{\prime}$ by $\mathcal{W}^{\sharp}.$
It can be seen that

$\mathcal{W}^{\sharp}=\\{x\in\mathcal{W}:x^{\prime}\in\mathcal{W}\\},$

and $\text{\rm dim}\mathcal{W}>\frac{1}{2}\text{\rm dim}\mathcal{V}$ implies
$\text{\rm dim}\mathcal{W}^{\sharp}>0.$

Let $A$ be a $2n\times 2n$ real positive definite matrix and let $d$ be a
symplectic eigenvalue of $A.$ We call a pair of vectors $(u,v)$ to be a
symplectic eigenvector pair of $A$ corresponding to $d$ if

$Au=dJv\textrm{ and }Av=-dJu.$

If $d_{1}(A),\ldots,d_{n}(A)$ are the symplectic eigenvalues of $A,$ then the
set $\mathcal{B}=\\{u_{1},\ldots,u_{n},$ $v_{1},\ldots,v_{n}\\}$ is called a
symplectic eigenbasis of $A$ if $\mathcal{B}$ is a symplectic basis of
$\mathbb{R}^{2n}$ and each $(u_{i},v_{i})$ is a normalised symplectic
eigenvector pair of $A$ corresponding to $d_{i}(A).$ The columns of the
symplectic matrix $M$ in (2) form a symplectic eigenbasis of $\mathbb{R}^{2n}$
corresponding to the symplectic eigenvalues $d_{1}(A),\ldots,d_{n}(A)$ of $A.$
With this terminology, we can alternatively state Williamson’s theorem as
follows: Let $A$ be a $2n\times 2n$ real positive definite matrix. Then there
exist positive numbers $d_{1}\leq\cdots\leq d_{n}$ and a symplectic basis
$\mathcal{B}=\\{u_{1},\ldots,u_{n},v_{1},\ldots,v_{n}\\}$ of $\mathbb{R}^{2n}$
such that $d_{1},\ldots,d_{n}$ are the symplectic eigenvalues of $A$ and
$\mathcal{B}$ is a corresponding symplectic eigenbasis.

##  3\. Proof of Theorem 1

We start with an inequality on symplectic eigenvalues that is analogous to the
Poincaré inequality for eigenvalues. See Theorem III.1.1 of [3].

######  Proposition 4.

Let $A$ be a $2n\times 2n$ real positive definite matrix with symplectic
eigenvalues $d_{1}\leq\cdots\leq d_{n}.$ Let $1\leq k\leq n.$ Then for any
$2n-k+1$-dimensional subspace $\mathcal{M},$ there exist two vectors $u,v$ in
$\mathcal{M}$ such that $\langle u,Jv\rangle=1$ and

$d_{k}\geq\frac{\langle u,Au\rangle+\langle v,Av\rangle}{2}.$ (7)

###### Proof.

Let $\mathcal{B}=\\{u_{1},\ldots,u_{n},v_{1},\ldots,v_{n}\\}$ be a symplectic
eigenbasis of $\mathbb{R}^{2n}$ corresponding to the symplectic eigenvalues
$d_{1},\ldots,d_{n}$ of $A.$ Let $\mathcal{N}$ be the space spanned by
$\\{u_{1},\ldots,u_{n},$ $v_{1},\ldots,v_{k}\\}.$ Since $\text{\rm
dim}\mathcal{M}+\text{\rm dim}\mathcal{N}=3n+1,$ $\text{\rm
dim}\left(\mathcal{M}\cap\mathcal{N}\right)\geq n+1.$ Hence we can find a
vector $u$ in $\mathcal{M}\cap\mathcal{N}$ such that $\|u\|_{\mathcal{B}}=1$
and the $\mathcal{B}$-complement $u^{\prime}$ of $u$ also belongs to
$\mathcal{M}\cap\mathcal{N}.$ Since $u,u^{\prime}$ both belong to
$\mathcal{N}$ and $\mathcal{N}$ is spanned by $u_{1},\ldots,u_{n},$
$v_{1},\ldots,v_{k},$ we must have

$u=\sum\limits_{i=1}^{k}\left(\alpha_{i}u_{i}+\beta_{i}v_{i}\right),$

where $\sum_{i=1}^{k}\left(\alpha_{i}^{2}+\beta_{i}^{2}\right)=1.$ We thus
have

$u^{\prime}=\sum\limits_{i=1}^{k}\left(-\beta_{i}u_{i}+\alpha_{i}v_{i}\right).$

A straightforward calculation shows that

$\langle
u,Au\rangle=\sum\limits_{i=1}^{k}\left(\alpha_{i}^{2}+\beta_{i}^{2}\right)d_{i}\leq
d_{k}.$ (8)

Similarly, $\langle u^{\prime},Au^{\prime}\rangle\leq d_{k}.$ Since $\langle
u,Ju^{\prime}\rangle=1,$ $u$ and $u^{\prime}$ are the required vectors that
satisfy (7).

We next prove a maxmin principle for symplectic eigenvalues. Two versions of
this principle can be found in [9, 6] and [8]. Though our present version is
similar to that given in Theorem 1 of [8], the maxmin principle in its present
form plays a key role in proving Theorem 1.

######  Theorem 5.

Let $A$ be a $2n\times 2n$ real positive definite matrix, and let
$d_{1}\leq\cdots\leq d_{n}$ be the symplectic eigenvalues of $A.$ Then

$d_{k}={\underset{\underset{dim\mathcal{M}=2n-k+1}{\mathcal{M}\subseteq\mathbb{R}^{2n}}}{\max}}{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{M}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2},\ \ 1\leq k\leq n.$ (9)

###### Proof.

Let $\mathcal{M}$ be a $2n-k+1$-dimensional subspace of $\mathbb{R}^{2n}.$ By
Proposition 4 we can find a normalised symplectic pair $(u,v)$ that satisfy
(7). Then

$\displaystyle d_{k}$ $\displaystyle\geq$ $\displaystyle\frac{\langle
u,Au\rangle+\langle v,Av\rangle}{2}$ $\displaystyle\geq$
$\displaystyle{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{M}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2}.$

This shows that the right hand side of (9) is less than or equal to $d_{k}.$
Let $\mathcal{B}=\\{u_{1},\ldots,u_{n},v_{1},\ldots,v_{n}\\}$ be a symplectic
eigenbasis of $\mathbb{R}^{2n}$ corresponding to the symplectic eigenvalues
$d_{1},\ldots,d_{n}$ of $A.$ Let $\mathcal{M}_{k}$ be the subspace spanned by
$\\{u_{1},\ldots,u_{n},$ $v_{k},\ldots,v_{n}\\}.$ Clearly $\text{\rm
dim}\mathcal{M}_{k}=2n-k+1.$ Let
$x=\sum_{i=1}^{n}\alpha_{i}u_{i}+\sum_{i=k}^{n}\beta_{i}v_{i}$ and
$y=\sum_{i=1}^{n}\gamma_{i}u_{i}+\sum_{i=k}^{n}\delta_{i}v_{i}$ be any two
elements of $\mathcal{M}_{k}$ such that $\langle x,Jy\rangle=1.$ We have

$\displaystyle 1$ $\displaystyle=$ $\displaystyle\langle x,Jy\rangle$
$\displaystyle=$
$\displaystyle\sum\limits_{i=k}^{n}\left(\alpha_{i}\delta_{i}-\beta_{i}\gamma_{i}\right)$
$\displaystyle\leq$
$\displaystyle\sum\limits_{i=k}^{n}\frac{\alpha_{i}^{2}+\beta_{i}^{2}+\gamma_{i}^{2}+\delta_{i}^{2}}{2}.$

The last inequality follows from the arithmetic-geometric mean inequality. Now
we have

$\displaystyle\frac{\langle x,Ax\rangle+\langle y,Ay\rangle}{2}\ $
$\displaystyle=$
$\displaystyle\sum\limits_{i=1}^{n}\frac{\alpha_{i}^{2}+\gamma_{i}^{2}}{2}d_{i}+\sum\limits_{i=k}^{n}\frac{\beta_{i}^{2}+\delta_{i}^{2}}{2}d_{i}$
(10) $\displaystyle\geq$ $\displaystyle
d_{k}\sum\limits_{i=k}^{n}\frac{\alpha_{i}^{2}+\beta_{i}^{2}+\gamma_{i}^{2}+\delta_{i}^{2}}{2}$
$\displaystyle\geq$ $\displaystyle d_{k}.$

Thus

$d_{k}\leq{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{M}_{k}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2}.$ (11)

The two sides of (11) are equal for $x=u_{k}$ and $y=v_{k}.$ This gives the
equality (9).

######  Lemma 6.

Let $\mathcal{V}$ be a $2n$-dimensional symplectic space and let $\mathcal{B}$
be a symplectic basis of $\mathcal{V}.$ For $1\leq k\leq n,$ let
$\mathcal{W}_{1}\supseteq\cdots\supseteq\mathcal{W}_{k}$ be a decreasing chain
of subspaces of $\mathcal{V}$ with $\text{\rm dim}\mathcal{W}_{j}\geq
n+k-j+1,$ $1\leq j\leq k.$ Let $\\{w_{1},\ldots,w_{k-1}\\}$ be a
$\mathcal{B}$-orthonormal, skew-orthogonal set such that
$w_{j}\in\mathcal{W}_{j}^{\sharp},$ $j=1,\ldots,k-1,$ and let $\mathcal{U}$ be
the space spanned by
$\\{w_{1},\ldots,w_{k-1},w_{1}^{\prime},\ldots,w_{k-1}^{\prime}\\}.$ Then
there exists an element
$v\in\mathcal{W}_{1}^{\sharp}\cap\\{w_{1},\ldots,w_{k-1}\\}^{\perp_{s}}$ such
that $\mathcal{U}+\text{\rm span}\\{v,v^{\prime}\\}$ has a
$\mathcal{B}$-orthosymplectic basis
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\},$ where
$v_{j}\in\mathcal{W}_{j}^{\sharp}$ for all $j=1,\ldots,k.$

###### Proof.

Since $\text{\rm dim}\mathcal{W}_{j}\geq n+(k-j+1),$ $\text{\rm
dim}\mathcal{W}_{j}^{\sharp}\geq 2(k-j+1),$ $1\leq j\leq k.$ We prove the
result by induction on $k.$ Let $k=2,$ and let
$w_{1}\in\mathcal{W}_{1}^{\sharp}.$ Then $\mathcal{U}=\text{\rm
span}\\{w_{1},w_{1}^{\prime}\\}.$ We know that $\text{\rm
dim}\mathcal{W}_{1}^{\sharp}\geq 4$ and $\text{\rm
dim}\,\mathcal{U}^{\perp_{s}}=2n-2.$ So, if
$w_{1}\in\mathcal{W}_{2}^{\sharp},$ then there exists a
$v\in\mathcal{W}_{1}^{\sharp}\cap\mathcal{U}^{\perp_{s}}.$ The set
$\\{v,w_{1},v^{\prime},w_{1}^{\prime}\\}$ is the required
$\mathcal{B}$-orthosymplectic basis of $\mathcal{U}+\text{\rm
span}\\{v,v^{\prime}\\}.$ Now, suppose that
$w_{1}\notin\mathcal{W}_{2}^{\sharp}.$ Since $\text{\rm
dim}\mathcal{W}_{2}^{\sharp}\geq 2,$ we can find a
$v\in\mathcal{W}_{2}^{\sharp}\cap\\{w_{1}\\}^{\perp_{s}}.$ By the classical
Gram-Schmidt orthogonalisation on $\\{v,w_{1}\\}$ in the space
$(\mathcal{V},\langle\cdot,\cdot\rangle_{\mathcal{B}}),$ we can find a
$\mathcal{B}$-orthonormal, skew-orthogonal elements $v_{2},v_{1}$ in
$\mathcal{W}_{2}$ and $\mathcal{W}_{1},$ respectively, such that $\text{\rm
span}\\{w_{1},v\\}=\text{\rm span}\\{v_{1},v_{2}\\}.$ Hence
$\\{v_{1},v_{2},v_{1}^{\prime},v_{2}^{\prime}\\}$ forms the required
$\mathcal{B}$-orthosymplectic basis of $\mathcal{U}+\text{\rm
span}\\{v,v^{\prime}\\}.$

Assume that the result holds for $k-1.$ We prove it for $k.$ By the induction
hypotheses, there exists a vector $u$ in
$\mathcal{W}_{2}^{\sharp}\cap\\{w_{2},\ldots,w_{k-1}\\}^{\perp_{s}}$ such that

$\mathcal{S}=\text{\rm
span}\\{w_{2},\ldots,w_{k-1},u,w_{2}^{\prime},\ldots,w_{k-1}^{\prime},u^{\prime}\\}=\text{\rm
span}\\{x_{2},\ldots,x_{k},x_{2}^{\prime},\ldots,x_{k}^{\prime}\\}$

for some $\mathcal{B}$-orthonormal, skew-orthogonal
$x_{j}\in\mathcal{W}_{j}^{\sharp},$ $2\leq j\leq k.$ Suppose $\mathcal{U}$ is
the symplectic space spanned by
$w_{1},\ldots,w_{k-1},w_{1}^{\prime},\ldots,w_{k-1}^{\prime}.$ Two cases arise
depending on $u\in\mathcal{U}$ or $u\notin\mathcal{U}.$ Consider the case
$u\in\mathcal{U}.$ Since $\mathcal{S}\subseteq\mathcal{U}$ and $\text{\rm
dim}\mathcal{S}=\text{\rm dim}\,\mathcal{U}=2(k-1),$
$\mathcal{S}=\mathcal{U}.$ We know that $\text{\rm
dim}\mathcal{W}_{1}^{\sharp}\geq 2k$ and $\text{\rm
dim}\mathcal{U}^{\perp_{s}}=2(n-k+1).$ Hence there exists a vector $v$ in
$\mathcal{W}_{1}^{\sharp}\cap\mathcal{U}^{\perp_{s}}$ with
$\|v\|_{\mathcal{B}}=1.$ Since $v$ is skew-orthogonal to each element of
$\mathcal{U},$ it is $\mathcal{B}$-orthogonal and skew-orthogonal to all
$x_{j}$’s. Take $v_{1}=v$ and $v_{j}=x_{j},$ $2\leq j\leq k.$ Then
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ is the
required $\mathcal{B}$-orthosymplectic basis of $\mathcal{U}+\text{\rm
span}\\{v,v^{\prime}\\}.$

Now consider the case $u\notin\mathcal{U}.$ If $u\in\mathcal{U}^{\perp_{s}},$
then take $v=u.$ As in the preceding paragraph, we see that
$\\{v,x_{2},\ldots,x_{k},v^{\prime},x_{2}^{\prime},\ldots,x_{k}^{\prime}\\}$
forms the required $\mathcal{B}$-orthosymplectic basis of
$\mathcal{U}+\text{\rm span}\\{v,v^{\prime}\\}.$ So, let
$u\notin\mathcal{U}^{\perp_{s}}.$ We can write $u=u_{1}+u_{2},$ where
$u_{1}\in\mathcal{U}$ and $u_{2}\in\mathcal{U}^{\perp_{s}}.$ Since
$u,u_{1}\in\mathcal{W}_{1}^{\sharp},$ $u_{2}$ also belongs to
$\mathcal{W}_{1}^{\sharp}.$ Take $v=u_{2},$ and consider the space
$\mathcal{U}_{0}=\mathcal{U}+\text{\rm span}\\{v,v^{\prime}\\}.$ Clearly,
$u\in\mathcal{U}_{0}$ and $\\{x_{2},\ldots,x_{k}\\}$ is a
$\mathcal{B}$-orthonormal, skew-orthogonal set in $\mathcal{U}_{0}.$ Since
$\mathcal{U}_{0}$ is a symplectic space and $\text{\rm
dim}\mathcal{U}_{0}=2k>\text{\rm dim}\mathcal{S},$ we can find a vector
$v_{1}$ in $\mathcal{U}_{0}$ with $\|v_{1}\|_{\mathcal{B}}=1$ that is
$\mathcal{B}$-orthogonal and skew-orthogonal to $x_{2},\ldots,x_{k}.$ Taking
$v_{j}=x_{j}$ for $2\leq j\leq k,$ we can see that
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ is the
required $\mathcal{B}$-orthosymplectic basis of $\mathcal{U}_{0}.$

######  Theorem 7.

Let $\mathcal{B}$ be a symplectic basis of $\mathbb{R}^{2n}.$ Let
$\mathcal{V}_{1}\subset\cdots\subset\mathcal{V}_{k}$ be an increasing chain of
subspaces, and let $\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}$ be a
decreasing chain of subspaces of $\mathbb{R}^{2n}$ such that $\text{\rm
dim}\mathcal{V}_{j}=n+i_{j}$ and $\text{\rm dim}\mathcal{W}_{j}=2n-i_{j}+1$
for $j=1,\ldots,k,$ and $1\leq i_{1}<\cdots<i_{k}\leq n.$ Then we can find two
$\mathcal{B}$-orthosymplectic sets
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ and
$\\{w_{1},\ldots,w_{k},w_{1}^{\prime},\ldots,w_{k}^{\prime}\\}$ that have the
same span and are such that $v_{j}\in\mathcal{V}_{j}^{\sharp}$ and
$w_{j}\in\mathcal{W}_{j}^{\sharp}$ for all $j=1,\ldots,k.$

###### Proof.

We first note that for every $j=1,\ldots,k,$ $\text{\rm
dim}\mathcal{V}_{j}^{\sharp}\geq 2i_{j}$ and $\text{\rm
dim}\mathcal{W}_{j}^{\sharp}\geq 2(n-i_{j}+1).$ We use induction on $k$ to
prove the theorem. When $k=1,$ there exists an element
$x\in\mathcal{V}_{1}^{\sharp}\cap\mathcal{W}_{1}^{\sharp}$ with
$\|x\|_{\mathcal{B}}=1.$ Take $v_{1}=w_{1}=x$ to obtain the result for $k=1.$

Assume that the result holds for $k-1.$ Then there exist two
$\mathcal{B}$-orthosymplectic sets
$\\{v_{1},\ldots,v_{k-1},v_{1}^{\prime},\ldots,v_{k-1}^{\prime}\\}$ and
$\\{x_{1},\ldots,x_{k-1},x_{1}^{\prime},\ldots,x_{k-1}^{\prime}\\}$ that have
the same span $\mathcal{U}$ and are such that
$v_{j}\in\mathcal{V}_{j}^{\sharp}$ and $x_{j}\in\mathcal{W}_{j}^{\sharp}$ for
all $j=1,\ldots,k-1.$ We now show that there exist vectors
$v_{k}\in\mathcal{V}_{k}^{\sharp}$ and $w_{j}\in\mathcal{W}_{j}^{\sharp},$
$1\leq j\leq k,$ such that the vectors $v_{1},\ldots,v_{k}$ and
$w_{1},\ldots,w_{k}$ satisfy the required properties.

For $j=1,\ldots,k$ consider the subspace
$\mathcal{S}_{j}=\mathcal{V}_{k}\cap\mathcal{W}_{j}.$ Clearly $\text{\rm
dim}\mathcal{S}_{j}\geq n+i_{k}-i_{j}+1\geq n+k-j+1.$ Also for all
$j=1,\ldots,k-1,$ $x_{j}\in\mathcal{S}_{j}.$ By Lemma 6 we can find a
$v\in\mathcal{S}_{1}^{\sharp}\cap\\{x_{1},\ldots,x_{k-1}\\}^{\perp_{s}}$ such
that the space spanned by
$\\{x_{1},\ldots,x_{k-1},v,x_{1}^{\prime},\ldots,x_{k-1}^{\prime},v^{\prime}\\}$
has a $\mathcal{B}$-orthosymplectic basis
$\\{w_{1},\ldots,w_{k},w_{1}^{\prime},\ldots,w_{k}^{\prime}\\},$ where
$w_{j}\in\mathcal{S}_{j}^{\sharp}$ for all $j=1,\ldots,k.$ By the classical
Gram-Schmidt orthogonalisation on $x_{1},\ldots,x_{k-1},v$ in the space
$(\mathbb{R}^{2n},\langle\cdot,\cdot\rangle_{\mathcal{B}}),$ we get a
$\mathcal{B}$-unit vector
$v_{k}\in\mathcal{S}_{1}^{\sharp}\subseteq\mathcal{V}_{k}^{\sharp}$ that is
$\mathcal{B}$-orthogonal to all $x_{1},\ldots,x_{k-1},$ and is such that
$\text{\rm span}\\{x_{1},\ldots,x_{k-1},v\\}=\text{\rm
span}\\{x_{1},\ldots,x_{k-1},v_{k}\\}.$ Since $\\{x_{1},\ldots,x_{k-1},v\\}$
is a skew-orthogonal set, $v_{k}$ is also skew-orthogonal to all $x_{j}$’s,
and consequently to the space $\mathcal{U}.$ Now the set
$\\{v_{1},\ldots,v_{k-1},v_{1}^{\prime},\ldots,v_{k-1}^{\prime}\\}$ is a
$\mathcal{B}$-orthosymplectic basis of $\mathcal{U}.$ Thus
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ is a
$\mathcal{B}$-orthosymplectic set that has the same span as
$\\{w_{1},\ldots,w_{k},w_{1}^{\prime},\ldots,w_{k}^{\prime}\\}.$ This proves
the theorem.

######  Proposition 8.

Let $\mathcal{B}$ be a symplectic eigenbasis of $\mathbb{R}^{2n}$
corresponding to the symplectic eigenvalues $d_{1}\leq\cdots\leq d_{n}$ of a
$2n\times 2n$ real positive definite matrix $A.$ Let
$\\{x_{1},\ldots,x_{k},x_{1}^{\prime},\ldots,x_{k}^{\prime}\\}$ and
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ be two
$\mathcal{B}$-orthosymplectic sets in $\mathbb{R}^{2n}$that have the same
span. Then

$\sum\limits_{j=1}^{k}\left(\langle x_{j},Ax_{j}\rangle+\langle
x_{j}^{\prime},Ax_{j}^{\prime}\rangle\right)=\sum\limits_{j=1}^{k}\left(\langle
v_{j},Av_{j}\rangle+\langle v_{j}^{\prime},Av_{j}^{\prime}\rangle\right).$

###### Proof.

Let $\mathcal{B}=\\{w_{1},\ldots,w_{n},z_{1},\ldots,z_{n}\\}.$ We can write
each vector $x$ in $\mathbb{R}^{2n}$ as

$x=\sum\limits_{i=1}^{n}\left(\alpha_{i}w_{i}+\beta_{i}z_{i}\right).$

Then

$\langle
x,Ax\rangle=\sum\limits_{i=1}^{n}d_{i}\left(\alpha_{i}^{2}+\beta_{i}^{2}\right).$
(12)

Let $\widetilde{D}$ be the operator on $\mathbb{R}^{2n}$ defined as

$\widetilde{D}w_{i}=d_{i}w_{i}\textrm{ and }\widetilde{D}z_{i}=d_{i}z_{i},$

$1\leq i\leq n.$ Clearly, $\widetilde{D}$ is a diagonal operator on
$(\mathbb{R}^{2n},\langle\cdot,\cdot\rangle_{\mathcal{B}}).$ We have
$\widetilde{D}x=\sum\limits_{i=1}^{n}d_{i}\left(\alpha_{i}w_{i}+\beta_{i}z_{i}\right).$
By using the definition of $\langle\cdot,\cdot\rangle_{\mathcal{B}}$ and
equality (12), we get

$\langle
x,\widetilde{D}x\rangle_{\mathcal{B}}=\sum\limits_{i=1}^{n}d_{i}\left(\alpha_{i}^{2}+\beta_{i}^{2}\right)=\langle
x,Ax\rangle.$ (13)

Since $\\{x_{1},\ldots,x_{k},x_{1}^{\prime},\ldots,x_{k}^{\prime}\\}$ and
$\\{v_{1},\ldots,v_{k},v_{1}^{\prime},\ldots,v_{k}^{\prime}\\}$ are
$\mathcal{B}$-orthonormal and have the same span, we have

$\sum\limits_{i=1}^{k}\left(\langle
x_{i},\widetilde{D}x_{i}\rangle_{\mathcal{B}}+\langle
x_{i}^{\prime},\widetilde{D}x_{i}^{\prime}\rangle_{\mathcal{B}}\right)=\sum\limits_{i=1}^{k}\left(\langle
v_{i},\widetilde{D}v_{i}\rangle_{\mathcal{B}}+\langle
v_{i}^{\prime},\widetilde{D}v_{i}^{\prime}\rangle_{\mathcal{B}}\right).$ (14)

Thus by using the relations (13) and (14), we obtain the proposition.

Proof of Theorem 1: Let
$\mathcal{B}=\\{u_{1},\ldots,u_{n},v_{1},\ldots,v_{n}\\}$ be a symplectic
eigenbasis of $\mathbb{R}^{2n}$ corresponding to the symplectic eigenvalues
$d_{1},\ldots,d_{n}$ of $A.$ Let $\mathcal{M}_{j}$ be the
$2n-i_{j}+1$-dimensional subspace spanned by $\\{u_{1},\ldots,u_{n},$
$v_{i_{j}},\ldots,v_{n}\\}.$ Let $1\leq j\leq k,$ and let $(x,y)$ be any
normalised symplectic pair of vectors in $\mathcal{M}_{j}.$ By (10), we see
that

$d_{i_{j}}\leq\frac{\langle x,Ax\rangle+\langle y,Ay\rangle}{2}.$

Hence, for any symplectically orthonormal set
$\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}$ with
$x_{j},y_{j}\in\mathcal{M}_{j},$ we have

$\sum\limits_{j=1}^{k}d_{i_{j}}\leq\sum\limits_{j=1}^{k}\frac{\langle
x_{j},Ax_{j}\rangle+\langle y_{j},Ay_{j}\rangle}{2}.$

The two sides are equal when we take $x_{j}=u_{i_{j}}$ and $y_{j}=v_{i_{j}},$
$1\leq j\leq k.$ Thus

$\sum\limits_{j=1}^{k}d_{i_{j}}={\underset{\underset{\underset{\textrm{symp.
o.
n.}}{\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}}}{x_{j},y_{j}\in\mathcal{M}_{j}}}{\text{\rm
min}}}\sum\limits_{j=1}^{k}\frac{\langle x_{j},Ax_{j}\rangle+\langle
y_{j},Ay_{j}\rangle}{2}.$

This shows that the left hand side of (3) is less than or equal to its right
hand side.

Now let $\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}$ be a decreasing
chain of subspaces with $\text{\rm dim}\mathcal{W}_{j}=2n-i_{j}+1.$ Let
$\mathcal{V}_{j}$ be the subspaces spanned by $\\{u_{1},\ldots,u_{n},$
$v_{1},\ldots,$$v_{i_{j}}\\}.$ Clearly $\text{\rm
dim}\mathcal{V}_{j}=n+i_{j}.$ By Theorem 7 we can find two
$\mathcal{B}$-orthosymplectic sets
$\\{x_{1},\ldots,x_{k},$$x_{1}^{\prime},\ldots,x_{k}^{\prime}\\}$ and
$\\{w_{1},\ldots,w_{k},$$w_{1}^{\prime},\ldots,w_{k}^{\prime}\\}$ that have
the same span $\mathcal{U}$ and are such that
$x_{j},x_{j}^{\prime}\in\mathcal{W}_{j}$ and
$w_{j},w_{j}^{\prime}\in\mathcal{V}_{j}$ for all $j=1,\ldots,k.$ Since both
$w_{j},w_{j}^{\prime}$ belong to $\mathcal{V}_{j}$ and $\mathcal{V}_{j}$ is
spanned by $\\{u_{1},\ldots,u_{n},v_{1},\ldots,v_{i_{j}}\\},$ we must have

$w_{j}=\sum\limits_{m=1}^{i_{j}}\left(\alpha_{jm}u_{m}+\beta_{jm}v_{m}\right).$

Then by using the same argument as in (8) and by using Proposition 8 we have

$\displaystyle\sum\limits_{j=1}^{k}d_{i_{j}}$ $\displaystyle\geq$
$\displaystyle\sum\limits_{j=1}^{k}\frac{\langle w_{j},Aw_{j}\rangle+\langle
w_{j}^{\prime},Aw_{j}^{\prime}\rangle}{2}$ $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{k}\frac{\langle x_{j},Ax_{j}\rangle+\langle
x_{j}^{\prime},Ax_{j}^{\prime}\rangle}{2}.$

This gives

$\sum\limits_{j=1}^{k}d_{i_{j}}\geq{\underset{\underset{\underset{\textrm{symp.
o.
n.}}{\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}}}{x_{j},y_{j}\in\mathcal{W}_{j}}}{\text{\rm
min}}}\sum\limits_{j=1}^{k}\frac{\langle x_{j},Ax_{j}\rangle+\langle
y_{j},Ay_{j}\rangle}{2}.$

This proves equality (3).

Proof of Corollary 2: Let $\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}$
be a decreasing chain of subspaces with $\text{\rm
dim}\mathcal{W}_{j}=2n-i_{j}+1$ such that

$\sum\limits_{j=1}^{k}d_{i_{j}}(A)={\underset{\underset{\underset{\textrm{symp.
o.
n.}}{\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}}}{u_{j},v_{j}\in\mathcal{W}_{j}}}{\text{\rm
min}}}\sum\limits_{j=1}^{k}\frac{\langle u_{j},Au_{j}\rangle+\langle
v_{j},Av_{j}\rangle}{2}.$ (15)

By Theorem 5 of [6], we have

$\sum\limits_{j=1}^{k}d_{j}(B)\leq\sum\limits_{j=1}^{k}\frac{\langle
u_{j},Bu_{j}\rangle+\langle v_{j},Bv_{j}\rangle}{2}$ (16)

for all symplectically orthonormal vectors
$u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}.$ Now, by Theorem 1 and relations (15)
and (16), we obtain

$\displaystyle\sum\limits_{j=1}^{k}d_{i_{j}}(A+B)$ $\displaystyle\geq$
$\displaystyle{\underset{\underset{\underset{\textrm{symp. o.
n.}}{\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}}}{u_{j},v_{j}\in\mathcal{W}_{j}}}{\text{\rm
min}}}\frac{\langle u_{j},(A+B)u_{j}\rangle+\langle
v_{j},(A+B)v_{j}\rangle}{2}$ $\displaystyle\geq$
$\displaystyle\sum\limits_{j=1}^{k}d_{i_{j}}(A)+\sum\limits_{j=1}^{k}d_{j}(B).$

##  4\. Proof of Theorem 3

Let $\alpha=(\alpha_{1},\ldots,\alpha_{n})$ be a real vector. The vector
$\alpha^{\uparrow}$ denotes the vector
$(\alpha_{1}^{\uparrow},\ldots,\alpha_{n}^{\uparrow})$ obtained by arranging
the components of $x$ in increasing order. For any two vectors $\alpha,\beta,$
$\alpha\leq\beta$ if $\alpha_{i}^{\uparrow}\leq\beta_{i}^{\uparrow}$ for all
$i=1,\ldots,n.$ We say $\alpha$ is supermajorised by $\beta,$ in symbols
$\alpha\prec^{w}\beta,$ if for all $k=1,\ldots,n,$

$\sum_{j=1}^{k}\,\alpha_{j}^{\uparrow}\geq\sum_{j=1}^{k}\,\beta_{j}^{\uparrow}.$
(17)

Further $\alpha$ is majorised by $\beta$ (or $\beta$ majorises $\alpha$) if
the two sides in (17) are equal when $k=n.$

A function $\varphi:\mathbb{R}^{n}\to\mathbb{R}$ is called Schur concave if
for every $\alpha,\beta$ in $\mathbb{R}^{n},$

$\alpha\prec\beta\,\implies\,\varphi(\alpha)\geq\varphi(\beta).$

The function $\varphi$ is called monotonically increasing if $\alpha\leq\beta$
implies $\varphi(\alpha)\leq\varphi(\beta).$ We refer the reader to [17,
Theorem 8.8, p.87] for the following lemma.

######  Lemma 9.

Let $\varphi:\mathbb{R}^{n}_{+}\to\mathbb{R}$ be a Schur-concave and
monotonically increasing function. Then for any two vectors $x,$ $y$ in
$\mathbb{R}^{n}_{+}$ with $x\prec^{w}y,$ we have $\varphi(x)\geq\varphi(y).$

We now give a generalisation of Theorem 1.

######  Theorem 10.

Let $1\leq k\leq n,$ and let $\varphi:[0,\infty)^{k}\to\mathbb{R}_{+}$ be a
Schur-concave, permutation invariant and a monotonically increasing function.
Let $A$ be a $2n\times 2n$ real positive definite matrix with symplectic
eigenvalues $d_{1}\leq\cdots\leq d_{n}.$ Then for every $1\leq
i_{1}<\cdots<i_{k}\leq n,$

$\varphi((d_{i_{1}},\ldots,d_{i_{k}}))={\underset{\underset{\text{\rm
dim}\mathcal{M}_{j}=2n-i_{j}+1}{\mathcal{M}_{1}\supset\cdots\supset\mathcal{M}_{k}}}{\max}}{\underset{\underset{\underset{\textrm{symp.
o. n.}}{u_{j},v_{j}\in\mathcal{M}_{j}}}{\mathcal{M}=\text{\rm
span}\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}}}{\text{\rm
min}}}\varphi(d_{\mathcal{M}}).$ (18)

Here $d_{\mathcal{M}}$ denotes the $k$-vector of symplectic eigenvalues of the
real positive definite matrix $A_{\mathcal{M}}$ obtained by restricting $A$ to
$\mathcal{M}.$

###### Proof.

Let $\mathcal{B}=\\{w_{1},\ldots,w_{n},z_{1},\ldots,z_{n}\\}$ be a symplectic
eigenbasis of $\mathbb{R}^{2n}$ corresponding to the symplectic eigenvalues
$d_{1},\ldots,d_{n}$ of $A.$ For each $j=1,\ldots,k,$ let $\mathcal{W}_{j}$ be
the space spanned by $\\{w_{1},\ldots,w_{n},z_{1},\ldots,z_{i_{j}}\\},$ and
let $\mathcal{M}_{1}\supset\mathcal{M}_{2}\supset\cdots\supset\mathcal{M}_{k}$
be a decreasing chain of subspaces of $\mathbb{R}^{2n}$ such that $\text{\rm
dim}\mathcal{M}_{j}=2n-i_{j}+1.$ By Theorem 7, we can find a symplectic
subspace $\mathcal{U},$ and its two $\mathcal{B}$-orthosymplectic bases
$\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}$ and
$\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}$ such that
$u_{j},v_{j}\in\mathcal{M}_{j}$ and $x_{j},y_{j}\in\mathcal{W}_{j}$ for all
$j=1,\ldots,k.$ Then as in (8), we have

$d_{i_{j}}\geq\frac{\langle x_{j},Ax_{j}\rangle+\langle
y_{j},Ay_{j}\rangle}{2},\ 1\leq j\leq k.$ (19)

Let $\alpha$ be the $n$-vector $\left(\frac{\langle
x_{j},Ax_{j}\rangle+\langle y_{j},Ay_{j}\rangle}{2}\right).$ Let
$d_{\mathcal{U}}$ be the vector of symplectic eigenvalues of the real positive
definite matrix $A_{\mathcal{U}}$ obtained by restricting $A$ to
$\mathcal{U}.$ Then by Theorem 3 of [7], we know that
$\alpha\prec^{w}d_{\mathcal{U}}.$ By Lemma 9, we have
$\varphi(\alpha)\geq\varphi(d_{\mathcal{U}}).$ Since
$\alpha\leq(d_{i_{1}},\ldots,d_{i_{k}})$ and $\varphi$ is increasing,
$\varphi(\alpha)\leq\varphi((d_{i_{1}},\ldots,d_{i_{k}})).$ This proves that
the left hand side of (18) is greater than or equal to its right hand side.

To prove the equality, we consider the subspaces $\mathcal{M}_{j}$ spanned by
the vectors $\\{w_{1},\ldots,w_{n},z_{i_{j}},\ldots,z_{n}\\}.$ Clearly
$\mathcal{M}_{1}\supset\cdots\supset\mathcal{M}_{k}$ and $\text{\rm
dim}\mathcal{M}_{j}=2n-i_{j}+1.$ Let $\mathcal{M}$ be the span of any
symplectically orthonormal set $\\{x_{1},\ldots,x_{k},y_{1},\ldots,y_{k}\\}$
where $x_{j},y_{j}\in\mathcal{M}_{j},$ $1\leq j\leq k.$ For $j=1,\ldots,k,$
let $\mathcal{N}_{j}$ be the subspace of $\mathcal{M}$ spanned by
$\\{x_{1},\ldots,x_{k},y_{j},\ldots,y_{k}\\}.$ By (11), we can see that

$d_{i_{j}}\leq{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{M}_{j}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2}\leq{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{N}_{j}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2}.$ (20)

Let $\tilde{d_{1}}\leq\cdots\leq\tilde{d_{k}}$ be the symplectic eigenvalues
of $A_{\mathcal{M}}.$ By using arguments similar to those in Theorem 5 for
$A_{\mathcal{M}},$ we have

$\tilde{d_{j}}={\underset{\underset{dim\mathcal{N}=2k-j+1}{\mathcal{N}\subseteq\mathcal{M}}}{\max}}{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{N}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2},\ \ 1\leq k\leq n.$

This in turn gives

$\tilde{d_{j}}\geq{\underset{\underset{\langle
x,Jy\rangle=1}{x,y\in\mathcal{N}_{j}}}{\text{\rm min}}}\frac{\langle
x,Ax\rangle+\langle y,Ay\rangle}{2}.$ (21)

Combining (20) and (21), we get $d_{i_{j}}\leq\tilde{d_{j}}$ for all
$j=1,\ldots,k.$ Since $\varphi$ is monotonically increasing,
$\varphi((d_{i_{1}},\ldots,d_{i_{k}}))\leq\varphi((\tilde{d_{1}},\ldots,\tilde{d_{k}}))=\varphi(d_{\mathcal{M}}).$
Thus we have

$\varphi((d_{i_{1}},\ldots,d_{i_{k}}))\leq{\underset{\underset{\underset{\textrm{symp.
o. n.}}{u_{j},v_{j}\in\mathcal{M}_{j}}}{\mathcal{M}=\text{\rm
span}\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}}}{\text{\rm
min}}}\varphi(d_{\mathcal{M}}),$

which proves (18).

By taking
$\varphi((\alpha_{1},\ldots,\alpha_{k}))=\alpha_{1}\cdots\alpha_{k},$ we
obtain the following corollary. This extends Theorem 5(ii) of [6].

######  Corollary 11.

Let $A$ be a $2n\times 2n$ real positive definite matrix with symplectic
eigenvalues $d_{1}\leq\cdots\leq d_{n}.$ Then for all $k=1,\ldots,n$ and
$1\leq i_{1}<\cdots<i_{k}\leq n,$

$\prod\limits_{j=1}^{k}d_{i_{j}}^{2}(A)={\underset{\underset{\text{\rm
dim}\mathcal{W}_{j}=2n-i_{j}+1}{\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}}}{\max}}{\underset{\underset{\underset{\textrm{
symplectically
orthonormal}}{u_{j},v_{j}\in\mathcal{W}_{j}}}{\mathcal{M}=\text{\rm
span}\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}}}{\text{\rm
min}}}\det(A_{\mathcal{M}}).$ (22)

Proof of Theorem 3: Let $\mathcal{W}_{1}\supset\cdots\supset\mathcal{W}_{k}$
be a decreasing chain of subspaces of $\mathbb{R}^{2n}$ with $\text{\rm
dim}\mathcal{W}_{j}=2n-i_{j}+1.$ Suppose $u_{j},v_{j}\in\mathcal{W}_{j}$ are
such that $\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}$ is symplectically
orthonormal. Let $\mathcal{M}$ be the span of
$\\{u_{1},\ldots,u_{k},v_{1},\ldots,v_{k}\\}.$ For the convenience of
notation, we denote $v_{1},\ldots,v_{k}$ by $u_{k+1},\ldots,u_{2k},$
respectively. Let $U$ be the orthogonal matrix such that
$A\\#B=A^{1/2}UB^{1/2}.$ Now,

$\displaystyle\left|\det(A\\#B)_{\mathcal{M}}\right|^{2}$ $\displaystyle=$
$\displaystyle\left|\det\begin{bmatrix}\langle
u_{i}A^{1/2}UB^{1/2}u_{j}\rangle\end{bmatrix}\right|^{2}$ (23)
$\displaystyle=$ $\displaystyle\left|\det\begin{bmatrix}\langle
A^{1/2}u_{i},UB^{1/2}u_{j}\rangle\end{bmatrix}\right|^{2}$ $\displaystyle\leq$
$\displaystyle\left|\det\begin{bmatrix}\langle
A^{1/2}u_{i},A^{1/2}u_{j}\rangle\end{bmatrix}\right|\left|\det\begin{bmatrix}\langle
UB^{1/2}u_{i},UB^{1/2}u_{j}\rangle\end{bmatrix}\right|$ $\displaystyle=$
$\displaystyle\left|\det(A_{\mathcal{M}})\right|\left|\det(B_{\mathcal{M}})\right|$
$\displaystyle\leq$
$\displaystyle\det(A_{\mathcal{M}})\prod\limits_{j=1}^{2k}\lambda_{j}^{\downarrow}(B),$

where $\lambda_{1}^{\downarrow}(B),\ldots,\lambda_{2n}^{\downarrow}(B)$ denote
the eigenvalues of $B$ arranged in decreasing order. The last inequality in
(23) follows from Theorem III.1.5 of [3]. By using (22), we obtain

$\prod\limits_{j=1}^{k}d_{i_{j}}^{4}(A\\#B)\leq\prod\limits_{j=1}^{k}d_{i_{j}}^{2}(A)\prod\limits_{j=1}^{2k}\lambda_{j}(B).$

Let $M$ be a symplectic matrix such that $M^{T}BM=\text{\rm diag}(D(B),D(B)),$
where $D(B)$ is the positive diagonal matrix with diagonal entries
$d_{1}(B)\leq\cdots\leq d_{n}(B).$ By the congruence invariance property of
geometric means, $M^{T}(A\\#B)M=(M^{T}AM)\\#(M^{T}BM),$ and the fact that for
all positive definite matrices $X,$ $d_{j}(M^{T}XM)=d_{j}(X),$ $1\leq j\leq
n,$ we can assume that $B$ is the diagonal matrix $\text{\rm
diag}(D(B),D(B)).$ In this case,
$\prod\limits_{j=1}^{2k}\lambda_{j}^{\downarrow}(B)=\prod\limits_{j=1}^{k}d_{n-j+1}^{2}(B).$
This gives the second inequality in (5).

Take $G=A\\#B.$ Then we have $A^{1/2}=GB^{-1/2}U^{*}.$ Calculations similar to
those in (23) give us the relation

$\det(A_{\mathcal{M}})\leq\left(\det(G_{\mathcal{M}})\right)^{2}\det(B^{-1}_{U\mathcal{M}})\leq\left(\det(G_{\mathcal{M}})\right)^{2}\prod\limits_{j=1}^{2k}\lambda_{j}^{\downarrow}(B^{-1}).$

Again, as in the previous paragraph, we can assume that $B$ is the diagonal
matrix $\text{\rm diag}(D(B),D(B)),$ and by using (22) we can obtain

$\prod\limits_{j=1}^{k}d_{i_{j}}^{2}(A)\leq\prod\limits_{j=1}^{k}d_{i_{j}}^{4}(G)\prod\limits_{j=1}^{k}\frac{1}{d_{j}^{2}(B)}.$

This gives the first inequality of (5).

We end the paper with an example which shows that unlike eigenvalues, the
symplectic eigenvalues of $AA^{T}$ and $A^{T}A$ need not be equal. Let $A$ be
the $4\times 4$ matrix

$A=\begin{bmatrix}A_{1}&O\\\ O&A_{2}\end{bmatrix},$

where $A_{1}=\begin{bmatrix}1&0\\\ 0&2\end{bmatrix}$ and
$A_{2}=\begin{bmatrix}0&1\\\ 2&0\end{bmatrix}.$ Then straightforward
calculations show that the symplectic eigenvalues of $A^{T}A$ are $2$ and $2$;
whereas the symplectic eigenvalues of $AA^{T}$ are $1$ and $4.$ Further, it
can be easily seen from Matlab experiments that for any two $2n\times 2n$ real
positive definite matrices $A$ and $B,$ the symplectic eigenvalues of
$A^{1/2}BA^{1/2}$ and $B^{1/2}AB^{1/2}$ need not be the same.

Acknowledgement: The author acknowledges the financial support from SERB
MATRICS grant number MTR/2018/000554.

## Conflict of interest

The author declares that she has no conflict of interest.

## References

  * [1]
  * [2] Arvind, B. Dutta, N. Mukunda and R. Simon, The real symplectic groups in quantum mechanics and optics, Pramana, 45 (1995), 471-495.
  * [3] R. Bhatia, Matrix Analysis, Springer, 1997.
  * [4] R. Bhatia, Linear algebra to quantum cohomology: the story of Alfred Horn’s inequalities, American Mathematical Monthly, 108 (2001), 289-318.
  * [5] R. Bhatia, The Riemannian mean of positive matrices, in Matrix Information Geometry, eds. F. Nielsen and R. Bhatia, Springer, (2013), 35-51.
  * [6] R. Bhatia and T. Jain, On symplectic eigenvalues of positive definite matrices, J. Math. Phys., 56 (2015), 112201.
  * [7] R. Bhatia and T. Jain, A Schur-Horn theorem for symplectic eigenvalues, Linear Algebra Appl., 599 (2020), 133-139.
  * [8] R. Bhatia and T. Jain, Variational principles on symplectic eigenvalues, to appear in Canad. Math. Bulletin.
  * [9] M. de Gosson, Symplectic Geometry and Quantum Mechanics, Oper. Theor., 166, Birkhäuser, 2006.
  * [10] T. Hiroshima, Additivity and multiplicativity properties of some Gaussian channels for Gaussian inputs, Phys. Rev. A, 73 (2006), 012330.
  * [11] H. Hofer and E. Zehnder, Symplectic Invariants and Hamiltonian Dynamics, Birkhauser, 2011.
  * [12] R. Horn and C. Johnson, Matrix Analysis, Cambridge Univ. Press, 2011.
  * [13] J. Eisert, T. Tyc, T. Rudolph, B.C. Sanders, Gaussian quantum marginal problem, Commun. Math. Phys., 280 (2008), 263-280.
  * [14] T. Jain and H. K. Mishra, Derivatives of symplectic eigenvalues and a Lidskii type theorem, to appear in Canad. J. Math., (2020).
  * [15] M. Krbek, T. Tyc and J. Vlach, Inequalities for quantum marginal problems with continuous variables, J. Math. Phys., 55 (2014), 062201-7.
  * [16] A. S. Markus, The eigen- and singular values of the sum and product of linear operators, Russian Math. Surveys, 19 (1964), 92-120.
  * [17] A. W. Marshall, I. Olkin and B. C. Arnold, Inequalities: Theory of Majorization and Its Applications, Springer, 2011.
  * [18] K. R. Parthasarathy, Symplectic dilation, Gaussian states and Gaussian channels, Indian J. Pure Ap. Mat., 46 (2015), 419-439.
  * [19] W. Pusz and S. L. Woronowicz, Functional calculus for sesquilinear forms and the purification map, Rep. Math. Phys., 8 (1975), 159-170.
  * [20] D. S̆afránek, I. Fuentes, Optimal probe states for the estimation of Gaussian unitary channels, Phys. Rev. A, 94 (2016), 062313.