# The thermodynamic quantities of a black hole with an $f(R)$ global monopole

Jingyun Man Hongbo Cheng111E-mail address<EMAIL_ADDRESS>
Department of Physics, East China University of Science and Technology,
Shanghai 200237, China
The Shanghai Key Laboratory of Astrophysics, Shanghai 200234, China

###### Abstract

The thermodynamic quantities such as the local temperature, heat capacity,
off-shell free energy and the stability of a black hole involving a global
monopole within or outside the $f(R)$ gravity are examined. We compare the two
classes of results to show the influence from the generalization of the
general relativity. It is found that the $f(R)$ theory will modify the
thermodynamic properties of black holes, but the shapes of curves for
thermodynamic quantities with respect to the horizon are similar to the
results within the frame of general relativity. In both cases there will exist
a small black hole which will decay and a large stable black hole in the case
that the temperatures are higher than their own critical temperature.

PACS number(s): 04.70Bw, 14.80.Hv
Keywords: Black hole, Global monopole, $f(R)$ gravity

I. Introduction

Recently more contributions were paid to the thermodynamics of various kinds
of black holes. More than thirty years ago, Bekenstein pointed out that the
entropy of a black hole is proportional to its surface area [1-3]. Hawking
also discussed the particle creation around a black holes to show that the
black hole has a thermal radiation with the temperature subject to its surface
gravity [4]. The issues about phase transitions of black holes in the frame of
semiclassical gravity were listed in Ref. [5]. Further the thermodynamic
properties of modified Schwarszchild black holes have been investigated [6,
7]. The thermodynamic behaviours including phase transition in Born-Infeld-
anti-de Sitter black holes were explored by means of various ways [8, 9]. The
phase transition of the quantum-corrected Schwarzschild black hole was
discussed, which fosters the research on the quantum-mechanical aspects of
thermodynamic behaviours [10].

In the process of the vacuum phase transition in the early Universe the
topological defects such as domain walls, cosmic strings and monopoles were
generated from the breakdown of local or global gauge symmetries [11, 12].
Among these topological defects, a global monopole as a spherical symmetric
topological defect occurred in the phase transition of a system composed by a
self-coupling triplet of scalar field whose original global O(3) symmetry is
spontaneously broken to U(1). It has been shown that the metric outside a
monopole has a deficit solid angle [13]. We researched on the strong
gravitational lensing for a massive source with a global monopole and find
that the deficit angle associated with the monopole affect the lensing
properties [14]. H. A. Buchdahl proposed a modified gravity theory named as
$f(R)$ gravity to explain the accelerated expansion of the Universe instead of
adding unknown forms of dark energy or dark matter [15-18]. The metric outside
a gravitational object involving a global monopole in the context of $f(R)$
gravity theory has been studied [19]. Further the classical motion of a
massive test particle around the gravitational source with an $f(R)$ global
monopole is probed [20]. We also examine the gravitational lensing for the
same object in the strong field limit [21].

Here we plan to investigate the thermodynamics of a static and spherically
symmetric black hole swallowing a global monopole or an $f(R)$ global
monopole. It is significant to understand the $f(R)$ theory in a new
direction. We wish to find the influences from the modified gravity on the
thermal properties of black holes. First of all we introduce a black hole
containing a global monopole in the context of $f(R)$ gravity theory and
certainly the black hole metric will recover to be one of the metrics of black
hole with a deficit solid angle. We show the dependence of the horizon on the
mass and parameters describing the monopole and the modification from $f(R)$
theory. We exhibit the thermodynamic characteristics due to the parameters of
the black hole. The thermodynamic stability will also be checked. We are going
to discuss the results in the end.

II. The thermodynamics of black holes involving an $f(R)$ global monopole

We adopt the line element,

$ds^{2}=A(r)dt^{2}-B(r)dr^{2}-r^{2}(d\theta^{2}+\sin^{2}\theta d\varphi^{2})$
(1)

which is static and spherical. In the $f(R)$ gravity theory, the action
introduced is,

$I=\frac{1}{2\kappa}\int d^{4}x\sqrt{-g}f(R)+I_{m}$ (2)

where $f(R)$ is an analytical function of Ricci scalar $R$ and $\kappa=8\pi
G$, $G$ is the Newton constant, $g$ is the determinant of metric and $I_{m}$
is the action of matter fields which can be denoted as,

$I_{m}=\int d^{4}x\sqrt{-g}{\cal L}.$ (3)

Here the Lagrangian for global monopole is [13],

${\cal
L}=\frac{1}{2}(\partial_{\mu}\phi^{a})(\partial^{\mu}\phi^{a})-\frac{1}{4}\lambda(\phi^{a}\phi^{a}-\eta^{2})^{2}$
(4)

where $\lambda$ and $\eta$ are parameters. The ansatz for the triplet of field
configuration showing a monopole is $\phi^{a}=\eta h(r)\frac{x^{a}}{r}$ with
$x^{a}x^{a}=r^{2}$, where $a=1,2,3$. $h(r)$ is a dimensionless function to be
determined by the equation of motion. This model has a global O(3) symmetry,
which is spontaneously broken to U(1). The field equation reads,

$F(R)R_{\mu\nu}-\frac{1}{2}f(R)g_{\mu\nu}-\nabla_{\mu}\nabla_{\nu}F(R)+g_{\mu\nu}\Box
F(R)=\kappa T_{\mu\nu}$ (5)

with $F(R)=\frac{df(R)}{dR}$ and $T_{\mu\nu}$ is the minimally coupled energy-
momentum tensor. The field equation (5) was solved under the weak field
approximation that assumes the components of metric tensor like $A(r)=1+a(r)$
and $B(r)=1+b(r)$ with $|a(r)|$ and $|b(r)|$ being smaller than unity [19].
Here the modified theory of gravity corresponds to a small correction on the
general relativity like $F(R(r))=1+\psi(r)$ with $\psi(r)\ll 1$. It is clear
that $F(R)=1$ is equivalent to the conventional general relativity. Further
the modification can be taken as the simplest analytical function of the
radial coordinate $\psi(r)=\psi_{0}r$. In this case the factor $\psi_{0}$
reflects the deviation of standard general relativity. The external metric of
the black hole with a global monopole is found finally [19, 20],

$A=B^{-1}=1-8\pi G\eta^{2}-\frac{2GM}{r}-\psi_{0}r$ (6)

where $M$ is the mass parameter. It should be pointed out that the parameter
$\eta$ is of the order $10^{16}GeV$ for a typical grand unified theory, which
means $8\pi G\eta^{2}\approx 10^{-5}$. If we choose $\psi_{0}=0$ excluding the
modification from $f(R)$ theory, the metric (6) will recover to be the result
by M. Barriola et.al. [13] like,

$ds^{2}=(1-8\pi G\eta^{2}-\frac{2GM}{r})dt^{2}-\frac{dr^{2}}{1-8\pi
G\eta^{2}-\frac{2GM}{r}}-r^{2}(d\theta^{2}+\sin^{2}\theta d\varphi^{2}).$ (7)

The function $A(r)$ is plotted in Figure 1 for comparison of two metrics for a
global monopole and $f(R)$ monopole black hole. According to the behavior of
$f(R)$ curve, it is clear that the black hole has two horizons, one is the
inner horizon which is supposed to be the event horizon $r_{H}$, the other one
is the outer horizon. The event horizons of two metrics seem to be the same
from Figure 1, because the given parameters $8\pi G\eta^{2}$ is very small,
but the analytic expression of $r_{H}$ are quite distinct. Solving the
equation $A(r_{H})=0$, the event horizon of metric (6) is located at,

$r_{H}=\frac{(1-8\pi G\eta^{2})-\sqrt{(1-8\pi
G\eta^{2})^{2}-8\psi_{0}GM}}{2\psi_{0}}.$ (8)

It also gives the relation between the mass parameter $GM$ and the event
horizon $r_{H}$,

$GM=\frac{1}{2}r_{H}(1-8\pi G\eta^{2}-\psi_{0}r_{H})$ (9)

for $f(R)$ monopole metric. It is seen from Figure 2 that there is a maximun
$GM_{0}=\frac{(1-8\pi G\eta^{2})^{2}}{8\psi_{0}}$ at $r_{H0}=\frac{1-8\pi
G\eta^{2}}{2\psi_{0}}$ where the inner and outer horizon meets. The Hawking
temperature can be obtained,

$\displaystyle T_{H}^{M}=T_{H}^{M}(\eta,\psi_{0})\hskip 28.45274pt$
$\displaystyle=\frac{1}{4\pi}[-g^{tt}g^{rr}g^{\prime}_{tt}]|_{r=r_{H}}\hskip
2.84526pt$ $\displaystyle=\frac{1}{4\pi}(\frac{1-8\pi
G\eta^{2}}{r_{H}}-2\psi_{0})$ (10)

where the prime stands for the derivative with respect to the radial
coordinate $r$. The local temperature is given by [22],

$\displaystyle T_{loc}^{M}=\frac{T_{H}^{M}}{\sqrt{A(r)}}\hskip 284.52756pt$
$\displaystyle=\frac{1}{4\pi}(\frac{1-8\pi
G\eta^{2}}{r_{H}}-2\psi_{0})\sqrt{\frac{r}{\psi_{0}r_{H}^{2}-(1-8\pi
G\eta^{2})r_{H}+(1-8\pi G\eta^{2})r-\psi_{0}r^{2}}}.$ (11)

Having omitted the influence from the modified gravity, we obtain the local
temperature as,

$T_{loc}=\frac{1}{4\pi r_{H}}\sqrt{\frac{(1-8\pi G\eta^{2})r}{r-r_{H}}}.$ (12)

The local temperatures for a global monopole and an $f(R)$ global monopole are
shown respectively in the Figure 3. There is a minimal local temperature
$T_{c}$, which means if the local temperature is below $T_{c}$, no black hole
exists. When the temperatures for the two kinds of black holes are high
enough, there will both exist two black holes, one is small and the other is
large. From the calculation of the extrema of local temperature,
$(\frac{\partial T_{loc}}{\partial r_{H}})_{r}=0$, the minimal local
temperature can be obtained as,

$T_{c}^{M}=\frac{\sqrt{r}}{\pi}(\frac{\psi_{0}}{(1-8\pi
G\eta^{2})^{\frac{2}{3}}-(1-8\pi
G\eta^{2}-2\psi_{0}r)^{\frac{2}{3}}})^{\frac{3}{2}}.$ (13)

The modifying factor $\psi_{0}$ from $f(R)$ leads the minimum of local
temperature lower than the one for global monopole black hole,

$T_{c}=\frac{3\sqrt{3}}{8\pi r}\sqrt{1-8\pi G\eta^{2}}$ (14)

for $8\pi G\eta^{2}=10^{-5}$, $r=10$ and $\psi_{0}=0.02$, then
$T_{c}^{M}=0.01836$, $T_{c}=0.02067$. The Figure 4 shows that the critical
temperature is a decreasing function of the modifying factor $\psi_{0}$
denoting the influence from $f(R)$ gravity here.

According to Bekenstein’s opinion, the entropy is proportional to the area of
event horizon and denoted as [1-3],

$\displaystyle S=\frac{A_{0}}{4}$ $\displaystyle=\pi r_{H}^{2}$ (15)

leading $dS=2\pi r_{H}dr_{H}$.

From the first law of thermodynamics $dE_{loc}=T_{loc}dS$, the thermodynamical
local energy can be derived as,

$\displaystyle E_{loc}^{M}=E_{0}+\int_{S_{0}}^{S}T_{loc}^{M}dS\hskip
199.16928pt$ $\displaystyle=r\sqrt{(1-8\pi
G\eta^{2})-\psi_{0}r}-\sqrt{r(r-r_{H})}\sqrt{(1-8\pi
G\eta^{2})-\psi_{0}(r+r_{H})}$ (16)

Here $S_{0}$ represents that $M=0$. For simplicity $E_{0}=0$. The
thermodynamic local energy in the case not belonging to $f(R)$ theory becomes,

$E_{loc}=\sqrt{(1-8\pi G\eta^{2})r}(\sqrt{r}-\sqrt{r-r_{H}}).$ (17)

For the purpose of checking the stability of the black holes, we should
discuss their heat capacity,

$\displaystyle C^{M}=(\frac{\partial E_{loc}^{M}}{\partial
T_{loc}^{M}})_{r}\hskip 256.0748pt$ $\displaystyle=2\pi(r-r_{H})[(1-8\pi
G\eta^{2})-2\psi_{0}r_{H}][(1-8\pi G\eta^{2})-\psi_{0}(r+r_{H})]\hskip
28.45274pt$ $\displaystyle\times[(1-8\pi
G\eta^{2})(\frac{r^{2}}{r_{H}^{2}}-3)\psi_{0}+(1-8\pi
G\eta^{2})^{2}\frac{3r_{H}-2r}{2r_{H}^{2}}+2\psi_{0}^{2}r_{H}]^{-1}.$ (18)

Similarly we choose the modifying factor $\psi_{0}=0$, the heat capacity of
black holes leads,

$\displaystyle C=(\frac{\partial E_{loc}}{\partial T_{loc}})_{r}\hskip
14.22636pt$ $\displaystyle=4\pi\frac{r_{H}^{2}(r-r_{H})}{3r_{H}-2r}.$ (19)

According to Eq. (18) and Eq. (19), we compare the heat capacities of these
two kinds of black holes in the Figure 5. The shapes of the curves of heat
capacities are similar, but can be recognized explicitly. The expression of
heat capacity for a Schwarzschild black hole with a global monopole is exactly
the same as the one for original Schwarzschild black hole. In Figure 5, the
$f(R)$ curve shifts from the conventional one. In every case the relatively
smaller horizon leads the heat capacity to be negative and the positive
capacity is due to the larger horizon, which meaning that the larger black
holes are stable. It should be emphasized that only huge black holes can
survive for long time within the frame of $f(R)$ theory. When
$r_{H}>\frac{1-8\pi G\eta^{2}-[(1-8\pi G\eta^{2})(2\psi_{0}r-(1-8\pi
G\eta^{2}))^{2}]^{3/2}}{2\psi_{0}}$, the $f(R)$ heat capacity is positive and
a large black hole is stable. A small black hole appears unstable for
$0<r_{H}<\frac{1-8\pi G\eta^{2}-[(1-8\pi G\eta^{2})(2\psi_{0}r-(1-8\pi
G\eta^{2}))^{2}]^{3/2}}{2\psi_{0}}$ since the heat capacity is negative. A
large Schwarzchild black hole with a monopole is stable when
$r_{H}>\frac{2r}{3}$, while it is unstable on $0<r_{H}<\frac{2r}{3}$.

In order to explore the phase transition among the black holes, we should
derive their off-shell free energy. The off-shell free energy can be defined
as,

$F_{off}^{M}=E_{loc}^{M}-TS$ (20)

where $E_{loc}^{M}$ is thermodynamic local energy from Eq. (16), $S$ is the
entropy of the black hole from Eq. (15) and $T$ is an arbitrary temperature.
The off-shell free energy of black holes containing a global monopole governed
by $f(R)$ gravity theory can be calculated as,

$F_{off}^{M}=r\sqrt{(1-8\pi
G\eta^{2})-\psi_{0}r}-\sqrt{r(r-r_{H})}\sqrt{(1-8\pi
G\eta^{2})-\psi_{0}(r+r_{H})}-\pi r_{H}^{2}T.$ (21)

If we are not going to consider the deviation from $f(R)$ theory, the Eq. (21)
will become the expression of the off-shell free energy for the black holes
with only a deficit solid angle like,

$F_{off}=\sqrt{(1-8\pi G\eta^{2})r}(\sqrt{r}-\sqrt{r-r_{H}})-\pi r_{H}^{2}T.$
(22)

In Figure 6 the behaviour of the off-shell free energy of black hole involving
a global monopole proposed by Barriola and Vilenkin [13] is shown as a
function of the horizon under several temperatures. Having considered extrema
of off-shell free energy, $(\frac{\partial F_{off}}{\partial r_{H}})_{r}=0$ or
$(\frac{\partial F^{M}_{off}}{\partial r_{H}})_{r}=0$, we give out the
critical temperatures which has already been mentioned above. When the
temperature is lower than the critical value, no black hole will appear. For
the temperature above the critical one the large black holes are stable but
the smaller ones are unstable. The dependence of the free energy of black
holes with an $f(R)$ global monopole on the horizon is plotted in the Figure 7
when the temperature is chosen to be several values around their own critical
temperature. The shapes of the off-shell free energy are similar to those in
the Figure 6. When the temperature is lower than critical value, there is no
real root from the equation $(\frac{\partial F_{off}}{\partial r_{H}})_{r}=0$
or $(\frac{\partial F^{M}_{off}}{\partial r_{H}})_{r}=0$, therefore no black
hole will appear. There are two equal roots $r_{H1}=r_{H2}=\frac{1-8\pi
G\eta^{2}-[(1-8\pi G\eta^{2})(2\psi_{0}r-(1-8\pi
G\eta^{2}))^{2}]^{3/2}}{2\psi_{0}}$ at critical temperature. When $T>T_{c}$,
two physically meaningful roots exist. An unstable small black hole appears at
event horizon $r_{H}=r_{H1}$, and a stable large black hole appears at
$r_{H}=r_{H2}$.

III. Discussion

We search for the thermodynamic quantities of the black holes with a global
monopole in the context of $f(R)$ gravity theory. We also obtain the
thermodynamic quantities excluding the modifications from $f(R)$ theory. We
compare the two classes of results to show how the generalized gravity
corrects the original thermodynamic quantities such as local temperature, heat
capacity, off-shell free energy. It should be pointed out that the
$f(R)$-modifications on these quantities are manifest. It is interesting that
the shapes of these quantities of black holes controlled by two different
gravity theories are similar although the quantities’ expressions are more
different from each other. In both cases the small black holes are unstable
and the large ones are stable when the temperature is higher than the critical
temperature and no black hole can exist as the temperature is sufficiently
low. The critical temperature has something to do with the modified factor
from $f(R)$ theory. Here we open a new window to explore the $f(R)$ theory
revising the Einstein Gravity.

In this paper, in what we pay more attention to is the thermodynamic
behaviours of a black hole with an $f(R)$ global monopole, as the stablity of
the black hole is dependent on them. There are some methods proposed for
probing a gravitational source which has a solid deflict angle subject to
$f(R)$ global monopole. But one necessary condition is that the black hole has
to be stable enough to be observed. Through calculation of the heat capacity,
only when the event horizon is larger than $\frac{1-8\pi G\eta^{2}-[(1-8\pi
G\eta^{2})(2\psi_{0}r-(1-8\pi G\eta^{2}))^{2}]^{3/2}}{2\psi_{0}}$, the black
hole of an $f(R)$ global monopole can be stable. It is the condition for a
large and more stable black hole given by extrema of off-shell free energy as
well. Moreover, a stable black hole also requires the local temperature higher
than the critical temperature $T_{c}$.

Acknowledge

This work is supported by NSFC No. 10875043 and is partly supported by the
Shanghai Research Foundation No. 07dz22020.

## References

  * [1] J. D. Bekenstein, Lett. Nuovo Cim. 4(1972)737
  * [2] J. D. Bekenstein, Phys. Rev. D7(1973)2333
  * [3] J. D. Bekenstein, Phys. Rev. D9(1974)3292
  * [4] S. W. Hawking, Commun. Math. Phys. 43(1975)199
  * [5] G. J. Stephens, B. L. Hu, Int. J. Theor. Phys. 40(2001)2183
  * [6] W. Kim, E. J. Son, M. Yoon, JHEP 0804(2008)042
  * [7] R. G. Cai, L. M. Cao, N. Ohta, JHEP 1004(2010)082
  * [8] Y. S. Myung, Y. Kim, Y. Park, Phys. Rev. D78(2008)084002
  * [9] A. Lala, D. Roychowdhury, Phys. Rev. D86(2012)084027
  * [10] W. Kim, Y. Kim, arXiv: 1207.5318
  * [11] T. W. B. Kibble, J. Phys. A9(1976)1387
  * [12] A. Vilenkin, Phys. Rep. 121(1985)263
  * [13] M. Barriola, A. Vilenkin, Phys. Rev. Lett. 63(1989)341
  * [14] H. Cheng, J. Man, Class. Quantum Grav. 28(2011)015001
  * [15] H. A. Buchdahl, Non-linear Lagrangians and cosmological theory, MNRAS 150(1970)1
  * [16] S. Nojiri, S. D. Odintsov, Phys. Rev. D68(2003)125312
  * [17] S. M. Carroll, V. Duvvuri, M. Trodden, M. S. Turner, Phys. Rev. D70(2004)043528
  * [18] S. Fay, R. Tavakol, S. Tsujikawa, Phys. Rev. D74(2007)063509
  * [19] T. R. P. Carames, E. R. B. de Mello, M. E. X. Guimaraes, Int. J. Mod. Phys. A (2011)
  * [20] T. R. P. Carames, E. R. B. de Mello, M. E. X. Guimaraes, arXiv: 1111.1856
  * [21] J. Man, H. Cheng, arXiv: 1205.4857
  * [22] R. C. Tolman, Phys. Rev. 35(1930)904

Figure 1: The figure shows the function $A(r)$ of $f(R)$ monopole black hole
(solid line) and Schwarzchild black hole with a global monopole (dot line).
Here $8\pi G\eta^{2}\approx 10^{-5}$, $\psi_{0}=0.02$ and $GM=1$.

Figure 2: The figure show the mass parameter $GM$ for $f(R)$ monopole black
hole (solid line) and global monopole black hole (dash line). Here $8\pi
G\eta^{2}\approx 10^{-5}$, $\psi_{0}=0.02$.

Figure 3: The solid and dotted curves of the dependence of the local
temperatures on the horizon with $8\pi G\eta^{2}\approx 10^{-5}$, $r=10$ and
$\psi_{0}=0.02$ for the Schwarzschild black hole with an $f(R)$ global
monopole or a global monopole respectively.

Figure 4: The dependence of the critical temperature for $8\pi
G\eta^{2}\approx 10^{-5}$ and $r=10$ on the modifying factor $\psi_{0}$ from
$f(R)$ gravity.

Figure 5: The solid and dotted curves correspond to the dependence of the heat
capacities on the horizon with $8\pi G\eta^{2}\approx 10^{-5}$, $r=10$ and
$\psi_{0}=0.02$ for the Schwarzschild black hole with an $f(R)$ global
monopole or a global monopole respectively.

Figure 6: The dotted, solid and dashed curves correspond to the dependence of
the off-shell free energy of the Schwarzschild black hole with a global
monopole on the horizon with $8\pi G\eta^{2}\approx 10^{-5}$ and $r=10$ for
$T=0.018,0.021,0.03$ respectively.

Figure 7: The solid, dot and dashed curves correspond to the dependence of the
off-shell free energy of the Schwarzschild black hole with an $f(R)$ global
monopole on the horizon with $8\pi G\eta^{2}\approx 10^{-5}$, $r=10$ and
$\psi_{0}=0.02$ for $T=0.018,0.021,0.03$ respectively.


# Learning Memory-Based Control for Human-Scale Bipedal Locomotion

Jonah Siekmann, Srikar Valluri, Jeremy Dao, Lorenzo Bermillo, Helei Duan, Alan
Fern, and Jonathan Hurst  Collaborative Robotics and Intelligent Systems
Institute
Oregon State University
Corvallis, Oregon 97331
_{ siekmanj, valluris, jdao, bermillf, duanh, afern<EMAIL_ADDRESS>
###### Abstract

Controlling a non-statically stable biped is a difficult problem largely due
to the complex hybrid dynamics involved. Recent work has demonstrated the
effectiveness of reinforcement learning (RL) for simulation-based training of
neural network controllers that successfully transfer to real bipeds. The
existing work, however, has primarily used simple memoryless network
architectures, even though more sophisticated architectures, such as those
including memory, often yield superior performance in other RL domains. In
this work, we consider recurrent neural networks (RNNs) for sim-to-real biped
locomotion, allowing for policies that learn to use internal memory to model
important physical properties. We show that while RNNs are able to
significantly outperform memoryless policies in simulation, they do not
exhibit superior behavior on the real biped due to overfitting to the
simulation physics unless trained using dynamics randomization to prevent
overfitting; this leads to consistently better sim-to-real transfer. We also
show that RNNs could use their learned memory states to perform online system
identification by encoding parameters of the dynamics into memory.

##  I Introduction

Reinforcement learning has shown significant promise as a tool for solving
complex control problems such as manipulation and legged locomotion. Recent
work in transferring these trained controllers from simulation onto real
robots has also enjoyed encouraging results [1, 2, 3, 4, 5, 6], but many of
these approaches use simple memoryless policy architectures, which could limit
the extraction of useful information. Use of memory-based architectures has
the potential to yield better performance in partially-observed domains, which
has been observed in a variety of applications [7, 8, 9]. In this work, we
demonstrate, for the first time, the application of learned, memory-based
control to dynamic locomotion on the bipedal robot Cassie produced by Agility
Robotics.

Memory-based controllers, such as recurrent neural networks (RNN), are a
potentially powerful choice for solving highly dynamic nonlinear control
problems due to their ability to infer important information about systems
that is not directly observable [1, 10]. Though classical control methods [11,
12, 13] have made exceptional progress in bipedal control, these approaches
often require disturbance observers to account for model inaccuracies [14, 15]
and are generally memory-based, containing some sort of hidden state which is
updated in real-time. These can be seen as either predictive or history-
compressing mechanisms, usually requiring tedious hand-tuning of gains.
Conversely, RNNs have been shown to be able to infer the dynamics of systems
thanks to the memory contained in their hidden state, effectively performing
implicit system identification [1, 16], suggesting that they could fill the
role of a disturbance observer and make memory-based learned control policies
more robust than other, non-memory-based methods.

Figure 1: We use recurrent neural networks and dynamics randomization to
greatly improve the sim-to-real transfer rate and demonstrate learned, memory-
based control on the bipedal robot, Cassie. RNNs trained without dynamics
randomization are unable to consistently transfer to hardware (failures
darkened and overlaid with X), while the same RNNs, when trained with dynamics
randomization, are able to consistently transfer to the real world when
trained with dynamics randomization.

While the representational power offered by RNNs has the potential to solve
new classes of challenging control problems, it also increases the potential
for overfitting. This can be problematic when training occurs in simulated
environments, which do not necessarily correspond well to the real environment
[17]. In particular, an expressive RNN controller may learn to exploit details
of the simulation dynamics that are maladaptive in the real world, leading to
failure. As we demonstrate in our experiments, this is what we observe for our
bipedal locomotion problem. That is, despite achieving significantly higher
rewards than memoryless controllers in simulation, the same RNN controllers
suffer from poor performance in the real world.

A common way to help address this sim-to-real challenge is the use of dynamics
randomization during simulation-based training. In particular, by training
across different simulation environments with varying dynamics properties, the
learned controller is less likely to exploit idiosyncrasies of any one
dynamics setting. One of the main contributions of our work is to demonstrate
that this approach is highly effective for training RNN controllers for the
Cassie biped. We show that by randomizing a small number of dynamics
parameters over reasonable ranges, the RNNs can be consistently trained in
simulation and successfully transferred to the real world. We further give
insight into the success of the RNN controllers, by demonstrating that the
latent memory is able to effectively encode important dynamics parameters
based on the observation history.

##  II Background

###  II-A Reinforcement Learning

Reinforcement learning is a machine learning paradigm that seeks to train
agents to maximize an expected reward through trial-and-error [18].
Reinforcement learning problems are often formalized as an agent interacting
with a Markov decision process (MDP) in order to learn a behavior to maximize
expected returns. This problem is typically presented in a manner in which an
agent receives a state $s_{t}$ at timestep $t$ from the MDP, which the agent
acts on based on its policy $\pi(a_{t}|s_{t})$. The MDP’s transition function
receives the agent’s action $a_{t}$ and returns the next state $s_{t+1}$ and
reward $r_{t}$ based on the action taken. The policy is often a stochastic
policy, in which case it is a function $\pi(a|s)$ which takes in a state $s$
and outputs the parameters of a distribution, usually the mean and standard
deviation of a normal distribution. The reward $r=R(s,a)$ is a scalar signal
that expresses how good a particular state-action pair is. The agent’s goal is
then to find an optimal policy that maximizes expected return $J(\pi)$,

$J(\pi)=\mathbb{E}_{\pi}\left[\sum_{t=0}^{T}\gamma^{t}r_{t}\right]$

where $T$ is the horizon of an episode, and $\gamma\in[0,1]$ is the discount
factor.

###  II-B Proximal Policy Optimization (PPO)

An effective solution to many RL problems is the family of policy gradient
algorithms, in which the gradient of the expected return with respect to the
policy parameters is computed and used to update the parameters through
gradient ascent. PPO is a model-free policy gradient algorithm which samples
data through interaction with the environment and optimizes a “surrogate”
objective function. PPO introduces a modified objective function that adopts
clipped probability ratios which forms a pessimistic estimate of the policy’s
performance [19]. It also addresses the problem of excessive policy updates by
restricting changes that move the probability ratio,

$r_{t}(\theta)=\frac{\pi_{\theta}(a_{t}|s_{t})}{\pi_{\theta_{old}}(a_{t}|s_{t})}$

too far away from 1. The probability ratio is a measure of how different the
current policy is from the previous policy (the policy before the last
update). The smaller the ratio the greater the difference. The “surrogate”
objective function is then modified into the clipped objective:

$L(\theta)=\mathbb{E}_{t}\left[min\left(r_{t}(\theta)\hat{A_{t}},clip(r_{t}(\theta),1-\epsilon,1+\epsilon)\hat{A_{t}}\right)\right]$

where $\epsilon$ is a tunable hyperparameter that increases or decreases the
bounds which constrain the probability ratio $r_{t}(\theta)$. Clipping the
probability ratio discourages the policy from changing too much and taking the
minimum results in using the lower, pessimistic bound of the unclipped
objective. Thus any change in the probability ratio $r_{t}(\theta)$ is
included when it makes the objective worse, and otherwise is ignored [19].
This can prevent the policy from changing too quickly and leads to more stable
learning.

###  II-C RNNs for Control Problems

RNNs have been successfully applied to many control domains using
reinforcement learning, often resulting in performance superior to feedforward
networks. Singla et al. [20] shows that using deep recurrent Q networks (DRQN)
instead of conventional feedforward Q Networks in UAV navigation results in
less collisions and more energy-efficient performance [20]. Hausknecht and
Stone [21] also use DRQNs for Atari games, finding that DRQNs with a single
observation are a viable alternative to DQNs with a history of states, and
that DRQNs are more robust to partial observability that non-memory based
agents. It has been shown that RNNs can store and recollect information for
arbitrarily long amounts of time [22], as well as perform system
identification as noted in Heess et al. [16] and further explored in Peng et
al. [1]. Furthermore, RNNs can do so through gradient descent, without hand-
tuning of hyperparameters, in contrast to feedforward networks which require
access to hand-picked fixed window of state histories [23].

Thus, while RNNs have been shown to be extremely successful in achieving
superior performance to feedforward networks on a variety of robotic control
tasks, some even on hardware [1], they have not yet been demonstrated for the
task of real-world robotic bipedal locomotion. This task differs significantly
from other control tasks, such as robotic arm manipulation, due to the
significant underactuation of the system and complicated contact dynamics.

###  II-D Dynamics Randomization

Dynamics randomization [1] [5] is the practice of randomizing physics
parameters of the simulated environment in the hopes that training agents on a
variety of possible dynamics will lead to better performance in the real
world. Tan et al. [5] leverage this technique to learn quadruped locomotion
from scratch in a physics simulator and then deploy the learned controller
into hardware, while Peng et al. [1] use a similar system to train a robotic
arm to manipulate objects in the real world. They improve the robustness of
control policies by simulating latency as well as physical properties such as
mass, joint center of mass, joint damping, and other similar parameters of the
environment.

More formally, using notation from Peng et al. [1], the objective is to train
a memory-based agent to perform manipulation tasks under the conditions set by
the real world dynamics $p^{*}(s_{t+1}|s_{t},a_{t})$. However, sampling from
these dynamics is not very time-efficient. Instead, the agent is trained
across a wide range of possible dynamics by using a set of dynamics parameters
$\mu$, sampled from a multivariate uniform distribution, to parameterize the
simulation dynamics $\hat{p}_{\mu}(s_{t+1}|s_{t},a_{t})$, so the objective is
reframed as attempting to maximize the expected return over the distribution
of dynamics parameters $\rho_{\mu}$.

##  III Method

Figure 2: We provide an RNN with a clock input, a velocity command, and
information about the robot’s state. The RNN produces joint position commands
at 33Hz, which are translated into torque-level commands by a PD controller at
2kHz. Figure 3: A diagram of the RNN structure (left) and conventional NN
structure (right) we use in our experiments. The recurrent policy has
connections which loop back onto itself, so that information from previous
timesteps is available. This produces a memory-like mechanism, allowing the
RNN to encode things about the state history which may be useful for choosing
actions. Both networks have approximately the same number of parameters.

###  III-A State Space and Action Space

The policy’s input consists of:

$X_{t}=\left\\{\begin{array}[]{ll}\vskip 3.0pt plus 1.0pt minus
1.0ptf_{vel}&\text{desired forward speed}\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\sin(\frac{2\pi\omega}{L})&\text{clock input}\\\ \vskip 3.0pt plus 1.0pt
minus 1.0pt\cos(\frac{2\pi\omega}{L})&\text{clock input}\\\ \hat{q},\
\hat{\dot{q}}&\text{robot state}\end{array}\right.$

Where $f_{vel}$ is the desired forward speed, $\sin(\frac{2\pi\omega}{L})$ and
$\cos(\frac{2\pi\omega}{L})$ are clock inputs, $\omega$ is an integer that
increments by one every policy evaluation step, $L$ is the phase length of the
reference motion, and $\hat{q},\ \hat{\dot{q}}$ are a collection of sensor-
based estimates of robot orientation, joint positions, and joint velocities.
In total, the input dimension of the networks is 49. The outputs of the policy
are simply raw motor PD targets, much like Xie et al. [4].

The policy is evaluated every 30 milliseconds, or roughly 33Hz, while the PD
controller operates at 2kHz, as can be seen in Figure 2. The control rate was
chosen because it corresponds to an even 60 cycles of the PD controller. The
output of the policy is added to a constant “neutral position” offset and then
fed directly into the motor PD controllers. The “neutral position” corresponds
to the static motor positions of the robot standing at rest. For more details
of the system setup we refer readers to Xie et al. [4], who use a similar
system.

###  III-B Reward Design

Our learning process makes use of a reference trajectory produced by an expert
walking controller to help the policy learn in the initial stages of training.
The reference trajectory contains a sequence of states of the robot walking
forward at 1 m/s. Though the reference contains the full robot state at each
timestep, we only use the center of mass position, orientation, and velocity,
motor positions, and spring positions. All policies were trained to maximize
the following reward function:

$\displaystyle R=$ $\displaystyle 0.20\cdot\text{exp}(-q_{\text{err}})$
$\displaystyle+$ $\displaystyle 0.20\cdot\text{exp}(-\dot{x}_{\text{err}})$
(1) $\displaystyle+$ $\displaystyle 0.05\cdot\text{exp}(-x_{\text{err}})$
$\displaystyle+$ $\displaystyle 0.20\cdot\text{exp}(-\dot{y}_{\text{err}})$
$\displaystyle+$ $\displaystyle
0.30\cdot\text{exp}(-\text{orient}_{\text{err}})$ $\displaystyle+$
$\displaystyle 0.05\cdot\text{exp}(-\text{spring}_{\text{err}})$

$q_{\text{err}}$, $\dot{x}_{\text{err}}$, $x_{\text{err}}$, and
$\text{spring}_{\text{err}}$ are squared error terms between the robot’s joint
positions, forward velocity, forward position, and passive spring position and
that of the corresponding reference state. These terms encourage the policy to
follow the reference motion. To prevent long-term sideways drift, we penalize
the y velocity $\dot{y}_{\text{err}}$. To keep the robot facing straight, we
use $\text{orientation}_{\text{err}}$ which is the quaternion difference
between the robot’s orientation and an orientation which faces straight ahead.

It is important to note that while the reward function is partially based on a
reference motion, very similarly to Xie et al. [4], the agent does not
actually receive any information about the reference motion in the input
space, aside from a clock input which corresponds roughly to the walking cycle
of the reference motion. Though we believe that the recurrent policies do not
have any theoretical reason for needing a clock input, we were not able to
train any to walk on hardware without it.

###  III-C Dynamics Randomization

At the start of each episode during training, we randomize 61 dynamics
parameters. A description of these parameters can be found in Table I.

Parameter | Unit | Range
---|---|---
Joint damping | Nms/rad | $[0.5,1.5]\times\text{default values}$
Joint mass | kg | $[0.7,1.3]\times\text{default values}$
Pelvis CoM (x) | cm |  $[-25,6]$ from origin
Pelvis CoM (y) | cm |  $[-7,7]$ plus default
Pelvis CoM (z) | cm |  $[-4,4]$ plus default
TABLE I: We chose ranges for each parameter based on estimates of uncertainty.
In addition, when robot behaviors seemed sensitive to a particular parameter
and this parameter seemed to be wrong, we increased the randomization range to
reduce sensitivity.

We aggressively randomize the pelvis center of mass because we believe the
robot’s center of mass in simulation differs significantly from the center of
mass we see on hardware; the results we see on hardware appear to confirm this
phenomenon. While this is not as principled as randomizing the entire robot’s
center of mass, we find that it introduces enough noise into the simulation
dynamics to overcome the sim-to-real gap.

###  III-D Recurrent Proximal Policy Optimization

We trained all policies with PPO, a model-free reinforcement learning
algorithm. However, correctly calculating the gradient of an RNN requires the
use of the backpropagation through time (BPTT) algorithm, which necessitates
special measures when sampling from the replay buffer. Thus, we use PPO with
the exception that for recurrent policies, instead of sampling individual
timesteps from the replay buffer, we sample batches of entire trajectories. In
addition, we collect statistics including the mean state $s_{\mu}$ and
standard deviation $s_{\sigma}$ before training, and use these to normalize
states during training and during evaluation. We found this prenormalization
step to be very important for consistent performance. In addition, we used a
threshold to stop the KL divergence between the old policy and new policy from
growing too large by aborting the optimization step early if the threshold was
surpassed. Pseudocode of the recurrent version of the algorithm that we used
can be seen in Algorithm 1.

Algorithm 1 Recurrent PPO

0: $kl_{thresh}\geq 0\in\mathbb{R}$

0: $N>0\in\mathbb{Z},K>0\in\mathbb{Z}$

Collect state mean $s_{\mu}$ and standard deviation $s_{\sigma}$

for iteration=1,2,… do

buff = []

$\theta_{old}\leftarrow\theta$

Collect $N$ rollouts from environment into buff.

Compute advantage estimates $\hat{A}_{\text{buff}}$.

for epoch=1,2, … $K$ do

for batch=1,2, … do

$\hat{s},\hat{a},\hat{A}\leftarrow$ sample trajectories from buff.

Optimize surrogate $L$ wrt $\theta$, using $\hat{s},\hat{a},\hat{A}$

if
$KL(\pi_{\theta}(\hat{s},\hat{a}),\pi_{\theta_{old}}(\hat{s},\hat{a}))>kl_{thresh}$
then

break

end if

end for

end for

end for

###  III-E Network Architecture

All policies had an input dimension of length 49, and an output dimension of
size 10. The recurrent policy used was a Long Short-Term Memory [22] network
consisting of two LSTM layers of 128 units each, and a linear output layer.
Our conventional neural network, or feedforward (FF), policies had two hidden
layers of 300 units each, which was roughly equivalent in terms of parameter
count to the recurrent policy. During training, we used a fixed action
standard deviation with a value of $e^{-2}$.

The recurrent critic was also an LSTM network with two layers of 128 units
each, with a single output representing the value function. The feedforward
critic was similarly a network with two hidden layers of 300 units each, in
order to maintain parameter-count equivalence with the recurrent critic. No
information about the dynamics disturbances was provided to either the policy
or the critic in the input space.

##  IV Results

###  IV-A Simulation

Figure 4: A PCA projection of the hidden layer activations of a feedforward
policy (top) and of an LSTM policy (bottom) could imply that using an LSTM
policy better enables learning of cyclic behaviors. Each dot represents a two-
dimensional projection of all latent states for a single timestep. Dots become
lighter as a function of time.

We trained ten LSTM networks and ten FF networks with dynamics randomization,
and ten LSTM networks and ten FF networks without dynamics randomization, each
with separate random seeds. When training the recurrent policies, we used a
batch size of 64 trajectories and a maximum trajectory length of 300
timesteps, equal to 9 seconds of simulation time. When training the
feedforward policies, we used a batch size of 1024 timesteps. Each recurrent
network took about sixteen hours to train on a machine with 56 CPU cores and
112 threads, while feedforward networks took about six hours on the same
machine. All networks were trained for fifty million simulation timesteps, and
each iteration we sampled about fifty thousand timesteps from the simulated
environment. We used PyTorch [24] to train and execute the policy, and Adam
[25] as our optimizer. As can be seen in Figure 5, when trained without
dynamics randomization, LSTM networks attain a significantly higher reward
than feedforward networks, with surprisingly little variance. Feedforward
networks obtain a notably lower reward, with high variance. We attribute this
difference to the LSTM overfitting the dynamics of the simulation, as
feedforward networks appear to perform roughly at the same level as LSTM
networks in sim-to-real when both are trained without dynamics randomization.

Figure 5: Reward curve of LSTM and FF networks during training without
dynamics randomization. The LSTM achieves a much higher reward with remarkably
little variance, but both networks perform roughly the same on hardware.

We conducted a robustness test in simulation across ten chosen sets of
dynamics, taken from the range in Table I. The results can be seen in Table
II. We randomly sample ten sets of dynamics parameters and measure the average
time each type of policy is able to remain upright and walking for each of the
ten sets. As can be seen, dynamics randomization improves performance of both
policy types and LSTM with dynamics randomization performs the best.

We also include a dimension-reduced visualization of the latent spaces of both
a feedforward and LSTM policy using principal component analysis, seen in
Figure 4. The projection of the latent states is in both cases highly cyclic,
though the LSTM policy is much more neatly segmented into 28 distinct points,
matching the clock cycle of the reference trajectory. The shape of these
projections aligns with our intuition that a bipedal walking controller should
learn behaviors which are cyclical, and could indicate that LSTM networks have
an advantage in emulating this.

Parameter Set | LSTM | LSTM DR | FF | FF DR
---|---|---|---|---
$\mu_{1}$ | 1.3s |  $>40.0$s | 1.6s | 17.1s
$\mu_{2}$ |  $>40.0$s |  $>40.0$s |  $>40$s |  $>40.0$s
$\mu_{3}$ | 1.9s |  $>40.0$s | 6.7s | 14.3s
$\mu_{4}$ | 1.5s |  $>40.0$s | 1.6s | 1.93s
$\mu_{5}$ |  $>40.0$s |  $>40.0$s | 36.7s |  $>40.0$s
$\mu_{6}$ | 1.6s |  $>40.0$s | 1.7s | 35.2s
$\mu_{7}$ |  $>40.0$s |  $>40.0$s | 36.3s | 13.7s
$\mu_{8}$ |  $>40.0$s |  $>40.0$s | 4.8s | 19.2s
$\mu_{9}$ | 28.8s |  $>40.0$s | 12.2s | 23.6s
$\mu_{10}$ |  $>40.0$s |  $>40.0$s | 6.7s | 15.8s
Avg. | 23.5s | 40.0s | 14.8s | 22.1s
TABLE II: Average time (in seconds) that ten randomly seeded policies were
able to walk in simulation subject to the conditions of the randomly chosen
dynamics parameters $\mu_{i}$, sampled from the distributions specified in
Table I. The last row gives the averaged values over all 10 sets of dynamics
parameters.

###  IV-B Hardware

Results are best seen in our accompanying video
111https://youtu.be/V8_JVvdJt_I. All of the recurrent policies trained with
dynamics randomization are able to walk forward for at least forty seconds on
the real robot, while LSTMs and FF networks trained without randomization fail
to consistently transfer to hardware.

We were unable to get FF policies trained with dynamics randomization to work
on hardware, as they exhibited extremely shaky behavior and evaluating them
would have likely resulted in damage to our robot. We believe that the reason
for this shakiness is the aggressive amount of dynamics randomization we do in
simulation, which interferes with the ability of memoryless agents to learn.

We note that all of the LSTM policies trained with dynamics randomization are
able to successfully walk on Cassie, while only two LSTM policies trained
without randomization are able to enter a stable walking gait. Feedforward
networks perform slightly better than LSTM networks when both are trained
without dynamics randomization, though much like Xie et al. [4], we observe
significant shakiness and instability on FF networks, which is not present in
the LSTM networks.

Seed | LSTM | LSTM DR | FF
---|---|---|---
1 | 7s |  $>40$s | 0s
2 | 7s |  $>40$s | 16s
3 |  $>40$s |  $>40$s | $>40s$
4 | 11s |  $>40$s | 9s
5 | 3s |  $>40$s | 33s
6 | 3s |  $>40$s | 0s
7 | 2s |  $>40$s | 5s
8 | 0s |  $>40$s | 3s
9 |  $>40$s |  $>40$s | 7s
10 | 4s |  $>40$s | 10s
Avg. | 11.7s | 40s | 12.3s
TABLE III: Time (in seconds) that a policy resulting from a seed was able to
walk in the real world. We cut off experiments that run for more than 40
seconds.

Of particular difficulty to most policies was stepping in-place. We have
reason to believe that the robot’s true center of mass position is
significantly different from the center of mass position we simulate. As a
means of addressing this discrepancy, we randomize the pelvis center of mass x
location in order to move the center of mass of the full robot forward and
backward, exposing agents to potential differences between simulation and the
real world. None of the policies trained without dynamics randomization were
able to step in-place, though some were able to walk forward. This is likely
because the robot center of mass discrepancy is made worse by the fact that
low-speed or no-speed walking is much more dynamically unstable than walking
at speed. All of the recurrent policies trained with dynamics randomization
were able to transfer to hardware successfully and walk in a straight line for
over forty seconds.

###  IV-C Dynamics extraction

We assume that in order for a memory-based agent to succeed in an environment
with unknown dynamics, it is advantageous to maintain a compressed history of
states that includes encoded estimates about those dynamics to determine which
action to take. We hypothesize that it should be possible to extract these
embedded dynamics from an RNN trained with dynamics randomization by imagining
the hidden states of the RNN as the latent representation of an autoencoder,
and use supervised learning to decode these latent representations back into
the original dynamics.

Using these hidden state and ground truth dynamics pairs, we constructed
datasets of 40,000 training data points, and 10,000 testing data points for
each policy tested. We then trained a decoder network to predict the dynamics
from the recurrent hidden states.

For most of the dynamics parameters, we found that dynamics randomization did
not provide an advantage in allowing LSTMs to encode information about the
dynamics into their hidden states. This is probably due to the state-history
compression ability of RNNs, which contained enough information for the
decoder network to make approximate guesses at the true value, regardless of
whether the encoder had been trained with dynamics randomization.

However, we were able to consistently predict the position of the pelvis
center of mass in the sagittal axis with higher fidelity when it was encoded
using LSTMs trained with dynamics randomization, as can be seen in Table IV.
This is consistent with our expectations, as it was the dynamics parameter
that we most aggressively randomized. This suggests that in simulation, joint
damping and joint mass were simply not important enough to bother accounting
for individually, and instead policies just learned to be as robust as
possible to variations in these quantities instead of attempting to infer
information about their actual values. Since we chose such a large range for
pelvis center of mass, however, policies were unable to merely learn behavior
that was robust across the entire range of possible values.

Policy | Mean Absolute Error | Mean Percent Error
---|---|---
LSTM (randomization) | 1.77mm | 30.8%
LSTM (no randomization) | 2.99mm | 62.4%
TABLE IV: Mean Absolute Error and Mean Percent Error for the decoder network’s
prediction of the pelvis center of mass in the sagittal axis. Here, the
decoder networks are able to decode far more information about the pelvis
center of mass from the memory of the policies trained with dynamics
randomization than the policies trained without dynamics randomization,
implying that this was a useful quantity to encode into memory.

##  V Conclusion

In this work, we showed that we were able to successfully and consistently
train recurrent neural networks to control a Cassie robot. The policies were
learned and tested first in simulation, then transferred to the robot,
demonstrating the robustness and promise of this approach. We also demonstrate
that recurrent neural networks appear to require some type of dynamics
randomization to consistently transfer to hardware for our task, and offer the
explanation that recurrent neural networks tend to overfit the dynamics of the
simulation. We further show that recurrent networks are capable of encoding
important information about disturbances in the dynamics into their memory,
and that this information can be extracted using a decoder network; thus,
recurrent networks can perform, for our task, online system identification.

Our success in these areas has implications for those seeking to apply
reinforcement learning to complex dynamical systems. The ability of recurrent
neural networks to do online system identification as well as behave as
controllers means that they have the potential to reduce model complexity
while increasing robustness. Future work might include investigating how
drastic disturbances must be during training for a recurrent neural network to
account for them rather than just being robust to them. Further investigation
could also look into the limits of what a recurrent network is able to infer
from dynamics. RNNs could also give insight into which aspects of dynamical
systems are highly sensitive so users can decide which control areas require
more focus than others. We feel that our findings relating to memory-based
control here are not only interesting for practical reasons, but also
consistent with the motivation behind the creation and use of disturbance
observers in classical control methods.

## Acknowledgements

This work was partially supported by NSF grant IIS-1849343 and DARPA contract
W911NF-16-1-0002. Thanks to Kevin Green, Yesh Godse, and Pedro Morais for
insightful discussions, support, and feedback regarding this project. Thanks
to Stephen Offer and Intel for providing access to the vLab computing cluster.

## References

  * Peng et al. [2018] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 3803–3810, May 2018.  doi: 10.1109/ICRA.2018.8460528.  URL https://ieeexplore.ieee.org/document/8460528.
  * OpenAI et al. [2019] OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming Yuan, Wojciech Zaremba, and Lei Zhang.  Solving Rubik’s Cube with a Robot Hand.  _arXiv preprint_ , 2019.  URL https://arxiv.org/abs/1910.07113.
  * Andrychowicz et al. [2020] OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Józefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, and Wojciech Zaremba.  Learning dexterous in-hand manipulation.  _The International Journal of Robotics Research_ , 39(1):3–20, 2020.  doi: 10.1177/0278364919887447.  URL https://doi.org/10.1177/0278364919887447.
  * Xie et al. [2019] Zhaoming Xie, Patrick Clary, Jeremy Dao, Pedro Morais, Jonathan Hurst, and Michiel van de Panne.  Learning Locomotion Skills for Cassie: Iterative Design and Sim-to-Real.  In _3rd Conference on Robotic Learning (CORL)_ , 2019.  URL https://zhaomingxie.github.io/publications/corl_2019.pdf.
  * Tan et al. [2018] Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke.  Sim-to-Real: Learning Agile Locomotion For Quadruped Robots.  In _Proceedings of Robotics: Science and Systems_ , Pittsburgh, Pennsylvania, June 2018.  doi: 10.15607/RSS.2018.XIV.010.  URL http://www.roboticsproceedings.org/rss14/p10.html.
  * Hwangbo et al. [2019] Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter.  Learning agile and dynamic motor skills for legged robots.  _Science Robotics_ , 4(26):eaau5872, 2019.
  * Koul et al. [2019] Anurag Koul, Alan Fern, and Sam Greydanus.  Learning Finite State Representations of Recurrent Policy Networks.  In _International Conference on Learning Representations_ , 2019.  URL https://iclr.cc/Conferences/2019/Schedule?showEvent=1062.
  * Arras et al. [2017] Leila Arras, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek.  Explaining Recurrent Neural Network Predictions in Sentiment Analysis.  In _Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_ , pages 159–168, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.  doi: 10.18653/v1/W17-5221.  URL https://www.aclweb.org/anthology/W17-5221.
  * Cechin et al. [2003] Adelmo Luis Cechin, D Regina, P Simon, and K Stertz.  State automata extraction from recurrent neural nets using k-means and fuzzy clustering.  In _23rd International Conference of the Chilean Computer Science Society, 2003. SCCC 2003. Proceedings._ , pages 73–78. IEEE, 2003.  URL https://ieeexplore.ieee.org/document/1245447.
  * Wierstra et al. [2010] Daan Wierstra, Alexander Förster, Jan Peters, and Jürgen Schmidhuber.  Recurrent policy gradients.  _Logic Journal of the IGPL_ , 18(5):620–634, 2010\.  URL https://academic.oup.com/jigpal/article/18/5/620/751594.
  * Vukobratovic and Borovac [2004] Miomir Vukobratovic and Branislav Borovac.  Zero-Moment Point - Thirty Five Years of its Life. _I. J. Humanoid Robotics_ , 1:157–173, 03 2004.  doi: 10.1142/S0219843604000083.  URL https://www.worldscientific.com/doi/abs/10.1142/S0219843604000083.
  * Westervelt et al. [2003] Eric R Westervelt, Jessy W Grizzle, and Daniel E Koditschek.  Hybrid zero dynamics of planar biped walkers.  _IEEE transactions on automatic control_ , 48(1):42–56, 2003.  URL https://ieeexplore.ieee.org/document/1166523.
  * Rezazadeh et al. [2015] Siavash Rezazadeh, Christian Hubicki, Mikhail Jones, Andrew Peekema, Johnathan Van Why, Andy Abate, and Jonathan Hurst.  Spring-Mass Walking With ATRIAS in 3D: Robust Gait Control Spanning Zero to 4.3 KPH on a Heavily Underactuated Bipedal Robot.  Dynamic Systems and Control Conference, 10 2015.  doi: 10.1115/DSCC2015-9899.  URL https://doi.org/10.1115/DSCC2015-9899.  V001T04A003.
  * Kim et al. [2018] Mingon Kim, Jung Hoon Kim, Sanghyun Kim, Jaehoon Sim, and Jaeheung Park.  Disturbance observer based linear feedback controller for compliant motion of humanoid robot.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 403–410. IEEE, 2018.  URL https://ieeexplore.ieee.org/document/8460618.
  * Paine et al. [2015] Nicholas Paine, Joshua S Mehling, James Holley, Nicolaus A Radford, Gwendolyn Johnson, Chien-Liang Fok, and Luis Sentis.  Actuator control for the NASA-JSC valkyrie humanoid robot: A decoupled dynamics approach for torque control of series elastic robots.  _Journal of Field Robotics_ , 32(3):378–396, 2015\.  URL https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21556.
  * Heess et al. [2015] Nicolas Heess, Jonathan J. Hunt, Timothy P. Lillicrap, and David Silver.  Memory-based control with recurrent neural networks.  _CoRR_ , abs/1512.04455, 2015.  URL http://arxiv.org/abs/1512.04455.
  * Boeing and Bräunl [2012] A. Boeing and T. Bräunl.  Leveraging multiple simulators for crossing the reality gap.  In _2012 12th International Conference on Control Automation Robotics Vision (ICARCV)_ , pages 1113–1119, Dec 2012.  URL https://ieeexplore.ieee.org/document/6485313.
  * Sutton and Barto [2018] Richard S. Sutton and Andrew G. Barto.  _Reinforcement Learning: An Introduction_.  The MIT Press, second edition, 2018.  URL http://incompleteideas.net/book/the-book-2nd.html.
  * Schulman et al. [2017] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal Policy Optimization Algorithms, 2017.  URL https://arxiv.org/abs/1707.06347.
  * Singla et al. [2019] A. Singla, S. Padakandla, and S. Bhatnagar.  Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge.  _IEEE Transactions on Intelligent Transportation Systems_ , pages 1–12, 2019.  ISSN 1558-0016.  doi: 10.1109/TITS.2019.2954952.  URL https://ieeexplore.ieee.org/document/8917687.
  * Hausknecht and Stone [2015] Matthew Hausknecht and Peter Stone.  Deep Recurrent Q-Learning for Partially Observable MDPs.  In _AAAI Fall Symposium on Sequential Decision Making for Intelligent Agents (AAAI-SDMIA15)_ , November 2015.  URL http://www.cs.utexas.edu/~pstone/Papers/bib2html/b2hd-SDMIA15-Hausknecht.html.
  * Hochreiter and Schmidhuber [1997] Sepp Hochreiter and Jürgen Schmidhuber.  Long Short-Term Memory.  _Neural Comput._ , 9(8):1735–1780, November 1997\.  ISSN 0899-7667.  doi: 10.1162/neco.1997.9.8.1735.  URL https://doi.org/10.1162/neco.1997.9.8.1735.
  * Yu et al. [2017] Wenhao Yu, Jie Tan, C. Karen Liu, and Greg Turk.  Preparing for the Unknown: Learning a Universal Policy with Online System Identification.  In _Proceedings of Robotics: Science and Systems_ , Cambridge, Massachusetts, July 2017.  doi: 10.15607/RSS.2017.XIII.048.  URL http://www.roboticsproceedings.org/rss13/p48.html.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  PyTorch: An Imperative Style, High-Performance Deep Learning Library.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché Buc, E. Fox, and R. Garnett, editors, _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc., 2019.  URL https://papers.nips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.  URL https://arxiv.org/abs/1412.6980.


# Translation surfaces with large systoles

Peter Buser, Eran Makover and Bjoern Muetzel

###### Abstract

In this paper we continue to investigate the systolic landscape of translation
surfaces started in [CHMW]. We show that there is an infinite sequence of
surfaces $(S_{g_{k}})_{k}$ of genus $g_{k}$, where $g_{k}\to\infty$ with large
systoles. On the other hand we show that for hyperelliptic surfaces we can
find a suitable homology basis, where a large number of loops that induce the
basis are short.

Keywords: translation surfaces, systoles, maximal surfaces, hyperelliptic
involution.
Mathematics Subject Classification (2020): 30F10, 32G15 and 53C22.

##  1 Introduction

A translation surface $S$ is a connected surface obtained by identifying
pairwise by translations the sides of a set of plane polygons. It follows that
$S$ is a closed flat surface with singularities or cone points, where a cone
point $p$ has angle $2\pi\cdot(k+1)$ for $k\in{\mathbb{N}}$ ([Ma],[Wr] and
[Zo]).

Translation surfaces are $\operatorname{CAT}(0)$ spaces, where the cone points
can be seen as points of "concentrated" negative curvature. Therefore these
surfaces share many properties of surfaces with non-positive curvature and
especially the well-studied hyperbolic surfaces (see, for example, [Ak],
[AM1], [Ba], [Ge], [Pa], [Ra], [Sc1] and [Sc2]). In this article we are
interested in short curves on general and hyperelliptic translation surfaces.
The leitmotif is to carry over the corresponding results for compact
hyperbolic surfaces to the category of translation surfaces.

More precisely, we are interested in the systole of a translation surface $S$
which is a shortest simple closed geodesic. We denote by
$\operatorname{sys}(S)$ its length. To make this a scaling invariant we have
to normalize by the area $\operatorname{area}(S)$ of $S$ to obtain
$\operatorname{SR}(S)=\frac{\operatorname{sys}(S)^{2}}{\operatorname{area}(S)}$,
the systolic ratio of $S$. Let

$\operatorname{SR}^{tr}(g)=\sup\\{\frac{\operatorname{sys}(S)^{2}}{\operatorname{area}(S)}\mid
S\text{ translation surface of genus }g\\}$

be the supremal systolic ratio in genus $g$. The systolic ratio in genus $g$
can only be of order $\frac{\log^{2}(g)}{g}$ (see [AM2], [KS]). This estimate
is valid for any smooth Riemannian surface. It also applies in the case of
translation surfaces, as any translation surface can be approximated by smooth
Riemannian surfaces.

Concerning this invariant little is known in the case of translation surfaces.
Preceeding this article the best lower bound known to the authors was of order
$\frac{1}{g}$ (see [CHMW]). Furthermore for fixed genus only the genus one
case is clear. In the case of flat tori
$\operatorname{SR}^{tr}(1)=\frac{2}{\sqrt{3}}$. In this case the maximal
surface is the equilateral torus which has a hexagonal lattice. In genus two
Judge and Parlier conjecture in [JP] that the surface $\mathop{Hex}_{2}$
obtained by gluing parallel sides of two isometric regular hexagons is
maximal. It is furthermore known that for fixed genus the supremal systolic
ratio is a maximum (see [CHMW]). We call a surface $S_{max}$ maximal, if
$\operatorname{SR}^{tr}(g)$ is attained in this surface. Maximal surfaces have
the following interesting property:

######  Theorem 1.1 ([CHMW]).

Let $S_{max}$ be a maximal translation surface of genus $g\geq 1$. Then every
simple closed geodesic that does not run through a cone point is intersected
by a systole of $S_{max}$.

Concerning the lower bound of $\operatorname{SR}^{tr}(g)$ we show in this
article in Section 2:

######  Theorem 1.2 (Surfaces with large systoles).

There exist, for infinitely many $g\in{\mathbb{N}}$ translation surfaces
$S_{g}$ of genus $g$, such that

$\frac{\log(g)}{g\cdot\log(\log(g))}\leq\operatorname{SR}(S_{g})\leq\operatorname{SR}^{tr}(g).$

We believe that the lower bound is of order $\frac{\log(g)^{2}}{g}$ as
conjectured for $\operatorname{SR}^{tr}(g)$ in [CHMW]. Inspired by the
constructions of [Bu] and [PW] for hyperbolic surfaces, the idea for the lower
bound in Theorem 1.2 is to construct explicit example surfaces from graphs
with large girth.

On the other hand, in the case of hyperelliptic hyperbolic surfaces it can be
shown that these surfaces have many short loops. Such a characterization can
also be made in the case of translation surfaces: Given a hyperelliptic
translation surface $S$ of genus $g\geq 2$, we find homologically independent
loops on $S$, whose lengths are bounded from above by a constant.

######  Theorem 1.3 (Hyperelliptic surfaces).

Let $S$ be a hyperelliptic translation surface of genus $g\geq 2$ and area
$4\pi g$. Then there exist $\lceil\frac{2g+2}{3}\rceil$ geodesic loops
$(\alpha_{k})_{k=1,\ldots,\left\lceil\frac{2g+2}{3}\right\rceil}$ that can be
extended to a homology basis of $H_{1}(S,{\mathbb{Z}})$ such that

$\ell(\alpha_{k})\leq 4\cdot\sqrt{\frac{12g}{2g+5-3k}}\text{ \ and \
}\frac{\ell(\alpha_{k})^{2}}{\operatorname{area}(S)}\leq\frac{48}{\pi(2g+5-3k)}\text{
for all }k=1,\ldots,\left\lceil\frac{2g+2}{3}\right\rceil.$

This means that for a hyperelliptic surface $S$ of genus $g$ and area $4\pi g$
we can find almost $\frac{2g}{3}$ loops that induce a homology basis which are
shorter than a constant. These results will be presented in Section 3 and
extend the result from [BMM] for hyperbolic surfaces to the category of
translation surfaces. These bounds hold more generally for compact surfaces
endowed with a Riemannian metric of non-positive curvature that have finitely
many cone-like singularities all of whose cone angles are greater than $2\pi$.
This is stated in Theorem 3.2.

##  2 Translation surfaces with large systoles

To construct translation surfaces with large systoles we base our construction
on graphs with large girth which are well-known and studied. Here we will use
$k$-regular graphs. A graph is called $k$-regular if all of its vertices have
degree $k$. The idea is to identify the vertices with $k$-holed flat tori. The
edges of the graph then provide the pasting scheme on how to glue the holes of
the tori together. We then adjust the parameters, such that the shortest non-
trivial loop in one of the tori is as large as the shortest loop going through
the graph. This way we obtain:

######  Theorem 2.1 (Surfaces with large systoles).

There exist, for infinitely many $g\in{\mathbb{N}}$ translation surfaces
$S_{g}$ of genus $g$, such that

$\frac{\log(g)}{g\cdot\log(\log(g))}\leq\operatorname{SR}(S_{g}).$

This is Theorem 1.2 from the introduction.

###  2.1 Construction of surfaces with large systoles

We will start with the construction and the properties of the $k$-holed tori.
To this end we start by constructing a rectangle $R$ of length $L+2/L$ and
width $2/L$ with a slit of length $L\in{\mathbb{N}}$ (see Figure 1).

$1/L$ $1/L$ $1/L$ $1/L$ $L$ $L$ $1/L$ $1/L$ $1/L$ $1/L$

Figure 1: Construction of a rectangle $R$ with a slit of length $L$. Dashed
lines are glued together.

To construct $R$ we proceed as indicated in the figure. We take two rectangles
of length $L+2/L$ and width $1/L$ and glue them together by translational
identification, leaving a slit of length $L$ in the middle to obtain our slit
rectangle $R$.

Example 1 We illustrate the later procedure with the following simple example
of a translation surface $X$ of genus two with two cone points of angle
$4\pi$. This is a special case of the example given in [Wr, Figure 6]. To this
end we first construct a slit torus $P$ from $R$ by pasting together opposite
sides of the rectangle $R$ as can be seen in Figure 2. We then paste two
copies $P_{1}$ and $P_{2}$ of $P$ from Figure 2 in the following way. The
upper segment of the slit in $P_{1}$ is pasted to the lower segment of the
slit in $P_{2}$ and vice versa.

$P_{1}$ $P_{2}$

Figure 2: Dashed and full lines are identified in each of the two rectangles
to obtain the two tori $P_{1}$ and $P_{2}$. These are glued together along the
slit to obtain the surface $X$.

We now build a flat torus $T$ with $2L^{2}$ slits from $2L^{2}$ copies of $R$.
Such a torus will represent a vertex $v$ of our $k$-regular graph $\Gamma$,
where

$k=2L^{2}.$ (1)

$\,2L$ $2L+4/L$ $T$

Figure 3: Construction of a torus $T$ with $2L^{2}$ slits from copies of the
rectangle $R$.

This construction is illustrated in Figure 3. To this end we first glue
$L^{2}$ copies of $R$ together with a top to bottom pasting to obtain a column
of rectangles $C$ with $L^{2}$ slits. Then we take two such copies of $C$ and
paste them with a left to right identification to obtain the rectangle
$T^{\prime}$ with $2L^{2}$ slits. Finally we glue the top and the bottom of
$T^{\prime}$ and the left and right side of $T^{\prime}$ to obtain the torus
$T$ with $2L^{2}$ slits. $T^{\prime}$ has

$\text{length \ }2L+4/L\text{ \ and width \ }2L.$

We now contruct a surface $S$ with the help of a $k=2L^{2}$ regular graph
$\Gamma$. A copy $T_{i}$ of $T$ represents a vertex $v_{i}$ of our graph
$\Gamma$ and a slit of $T_{i}$ a half-edge of the vertex $v_{i}$. The pasting
pattern for the slits is provided by the edges of $\Gamma$. More precisely,
for each vertex $v_{i}$ of $\Gamma$ and a copy $T_{i}$ of $T$ we paste slit
number $s$ of $T_{i}$ to slit number $t$ of $T_{j}$ if and only if half-edge
$s$ of vertex $v_{i}$ is glued to half-edge $t$ of vertex $v_{j}$ thus forming
an edge $e=\\{s,t\\}$ of $\Gamma$. We call $\Gamma$ the underlying pasting
scheme or pasting pattern of $S$. The pasting of the slits is as in Example 1.
The upper segment of the first slit is pasted to the lower segment of the
second slit and vice versa. This way all cone points are of angle $4\pi$.
In order to get $S$ with a large systole we need a graph $\Gamma$ with a large
girth, that is, a large length of the shortest non-trivial closed reduced edge
path. An edge path is called reduced if it has no edge followed by its inverse
(and in the closed case, the last edge is not the inverse of the first). The
corresponding graphs with large girth will be provided in the following
subsection to complete the length estimates. We now identify
$\operatorname{sys}(S)$.

######  Lemma 2.2.

If our graph $\Gamma$ in the construction of $S$ satisfies

$\operatorname{girth}(\Gamma)\cdot\frac{2}{L}\geq 2L,\text{ \ then \
}\operatorname{sys}(S)=2L.$ (2)

This implies that there are systoles of $S$ on every copy $T_{i}$ of the torus
$T$.

###### Proof of Lemma 2.

Assume that our graph $\Gamma$ in the construction satisfies the condition of
the lemma.

$T_{i}$ $p$ $c$ $q$ $T_{i}$ $p$ $\,\,c$ $\,q$

Figure 4: An arc $c$ of the geodesic $\gamma$ connecting two cone points $p$
and $q$.

To any simple closed geodesic $\gamma$ on $S$ we associate an edge path
$\xi(\gamma)$ in the standard way: If $\gamma$ begins on copy $T_{i_{0}}$ of
$T$, then $\xi$ begins on vertex $v_{i_{0}}$. Then, successively, whenever
$\gamma$ leaves some $T_{i}$ at slit $s$ entering the adjacent $T_{j}$ at slit
$t$ we add edge $e=\\{s,t\\}$ to the edge path. For the case when some subarc
of $\gamma$ runs along a slit say of $T_{i}$ and thus is simultaneously
visible on $T_{i}$ and on the adjacent $T_{j}$ we must make precise at which
point $\gamma$ shall be considered entering $T_{j}$. Since $\gamma$ is a
geodesic it runs along an entire side of the slit from one singular point to
the other. Figure 4 shows two cases: on the left hand side subarc $c$ enters a
rectangle of $T_{i}$ arrives at the slit on the cone point $p$ goes along the
slit and leaves $T_{i}$ into $T_{j}$ at the other singular point $q$. In this
case $q$ will be the exit point. On the right hand side $c$ also goes from $p$
to $q$ but then continues in $T_{i}$. In this case $c$ is not considered being
entering $T_{j}$. A particular case occurs when $\gamma$ runs entirely along
the slit from $p$ to $q$ and then back to $p$. In this case $\xi(\gamma)$ is
the point path $\\{v_{i}\\}$. For $\xi(\gamma)$ there are therefore two cases:

  * Case 1.)

$\xi(\gamma)$ is homotopic in $\Gamma$ to a non-trivial cycle: In this case
$\xi(\gamma)$ has combinatorial length greater or equal to
$\operatorname{girth}(\Gamma)$ and $\gamma$ contains at least
$\operatorname{girth}(\Gamma)$ arcs with disjoint interiors each of which
connects two distinct slits of some $T_{i}$ . By the following Lemma 2.3 we
have

$\ell(\gamma)\geq\operatorname{girth}(\Gamma)\cdot\frac{2}{L}.$ (3)

######  Lemma 2.3.

Any arc in $T$ that connects two distinct slits with each other has length
greater or equal to $2/L$.

###### Proof.

In the base rectangle $R$ (see Figure 1) the distance from the slit to the
boundary equals $1/L$. ∎

  * Case 2.)

$\xi(\gamma)$ is homotopic to a point path:

    * Case 2.1)

$\xi(\gamma)$ itself is not a point path: This can, for example, happen, if a
loop goes through a few tori $T_{i}$ and then returns to its starting torus.
In this case $\gamma$ contains two subarcs $c$ and $c^{\prime}$ with disjoint
interiors, such that $c$ lies in some $T_{i}$ having its endpoints on a slit
$\sigma$ of $T_{i}$ and $c^{\prime}$ lies in some other $T_{j}$ having its
endpoints on a slit $\sigma^{\prime}$ of $T_{j}$. We claim that

$\ell(c)\geq L\text{ \ and \ }\ell(c^{\prime})\geq L.$ (4)

$\,c$ $\sigma$ $\,d$ $c$

Figure 5: The auxiliary curve $d$ on the slit $\sigma$ connects two arcs of
$c$.

###### Proof of Inequality (4).

Consider w.l.o.g. $c$ on $T_{i}$ with endpoints on the slit $\sigma$. If $c$
contains a subarc of positive length on one of the sides of $\sigma$ then,
being a geodesic $c$ covers the entire side and has length greater or equal to
$L$. Now assume that this is not the case. Then again, because $c$ is a
geodesic, it is not homotopic with fixed endpoints to an arc that lies
entirely on $\sigma$. Connect the endpoint of $c$ with the initial point by a
geodesic arc $d$ of length $\ell(d)\leq L$ that lies on $\sigma$ (Figure 5
shows a case). Then the curve $c$ followed by $d$ is essentially simple and
not homotopic to a point curve.

Essentially simple means that possible double points which may occur at the
cone points can be removed by small homotopies. The inequality $\ell(c)\geq L$
now follows from the next lemma.

######  Lemma 2.4.

Let $\eta$ be an essentially simple closed curve on the slit torus $T$ that is
not homotopic to a point. Then $\ell(\gamma)\geq 2L$.

$\overline{T}$ $D$ $\eta_{1}$ $a$ $b$ $\,\overline{T}$ $\eta_{2}$ $a$ $\,b$
$\overline{T}$ $\,\,\eta_{3}$ $a$ $b$

Figure 6: Non-trivial simple closed curves of $T$ in $\overline{T}$.

###### Proof.

Let $\overline{T}$ be the torus obtained from $T$ by closing the opened slits
again. We denote by $a$ and $b$ generators of the fundamental group of
$\overline{T}$ represented by closed geodesics of length $\ell(a)=2L+4/L$ and
$\ell(b)=2L$ (see Figure 6). In $\overline{T}$ the curve $\eta$ is either
homotopic to a point, or else homotopic to some $a^{m}\cdot b^{n}$ with
$|m|+|n|>0$. If it is homotopic to a point, then, by the Jordan curve theorem
it bounds a simply connected domain $D$ as in the example of $\eta_{1}$ in
Figure 6. Since $\eta$ is not contractible in $T$, $D$ contains at least one
of the sewed slits and so $\ell(\eta)\geq 2L$. If $\eta$ is homotopic to
$a^{m}\cdot b^{m}$, then its vertical projection with respect to the Euclidean
geometry of $T$ has length greater or equal to $|m|\cdot(2L+4/L)$ and its
horizontal projection to $b$ has length greater or equal to $|n|\cdot 2L$.
Since $|m|+|n|\neq 0$ this implies that $\ell(\eta)\geq 2L$. This completes
the proof of Lemma 2.4. ∎

Together with the lemma, inequality (4) follows. ∎

    * Case 2.2)

$\xi(\gamma)$ is a point path. In this case $\gamma$ runs entirely in some
$T_{i}$ and Lemma 2.4 implies that $\ell(\gamma)\geq 2L$.

In total we have shown that $\operatorname{sys}(S)=2L$ and this length is
realized by the systoles of the tori $T_{i}$ in $S$. This concludes the proof
of Lemma 2. ∎

###  2.2 Length estimates

In order to get $S$ with a large systole we need a regular graph $\Gamma$ with
a large girth. At the same time $S$ should have "small area". So $\Gamma$
should have a large girth and a relatively small number of vertices. Several
constructions are known ([LPS],[LS]). The best result for our purpose is from
[ES]. There, for any $k\geq 2$ and $l\geq 3$ a $k$-regular graph
$\Gamma_{k,l}$ of girth $l$ with at most $n(k,l)$ vertices is constructed
where

$n(k,l)\leq 4\sum_{t=1}^{l-2}(k-1)^{t}.$

For $k\geq 4$ we have the simplification

$n(k,l)\leq(k-1)^{l},\quad(k\geq 4).$

We recall, that by (2) our goal is to find $\Gamma$, such that

$\operatorname{girth}(\Gamma)\cdot\frac{2}{L}\geq 2L\text{ \ or \
}\operatorname{girth}(\Gamma)\geq L^{2}.$

Let now $L\in{\mathbb{N}}$, $L\geq 2$, take the above slit torus $T$ with
$k=2L^{2}$ slits (see (1)) and set $l=L^{2}$. The graph
$\Gamma_{L}:=\Gamma_{2L^{2},L^{2}}$ then has girth $L^{2}$ and
$n_{L}:=n(2L^{2},L^{2})\leq(2L^{2}-1)^{L^{2}}$ vertices. By Lemma 2 the
surface $S_{L}$ resulting from pasting $n_{L}$ copies of $T$ together with
pasting pattern $\Gamma_{L}$ has systole $\operatorname{sys}(S_{L})=2L$.

The genus of $S_{L}$ can be found using the Euler characteristic that relates
the genus to the cone angles. If a translation surface of genus $g$ has cone
points $(p_{i})_{i=1,\ldots,N}$ with cone angle $2\pi\cdot(k_{i}+1)$ at
$p_{i}$, where $k_{i}\in{\mathbb{N}}$, then

$\sum_{i=1}^{N}k_{i}=2g-2.$ (5)

By this equation and the fact that all cone points of $S_{L}$ are of angle
$4\pi$ we have

$\displaystyle\operatorname{area}(S_{L})$ $\displaystyle=$ $\displaystyle
n_{L}(4L^{2}+8)$ $\displaystyle g_{L}:=\operatorname{genus}(S_{L})$
$\displaystyle=$ $\displaystyle n_{L}L^{2}+1.$ (6)

Hence, for any $L\in{\mathbb{N}}$, $L\geq 2$ we have constructed a translation
surface $S_{L}$ with systole $\operatorname{sys}(S_{L})=2L$ and genus

$g_{L}\leq L^{2}(2L^{2}-1)^{L^{2}}+1.$ (7)

Furthermore, for large $L$ the area $\operatorname{area}({S_{L}})$ is close to
$4g$. Theorem 2.1 now follows from the next lemma. $\square$

######  Lemma 2.5.

If $4<y\leq x(2x)^{x}$ and $x>6$, then $\frac{\log(y)}{\log\log(y)}<x$.

###### Proof.

It suffices to prove the lemma for the case of equality $y=x(2x)^{x}$. Since
then $\log(y)=\log(x)+x\log(2x))$ the statement for $x$ is equivalent to

$\log(x)+x\log(2x)\leq x\log(\log(x)+x\log(2x)).$

For $6<x<12$ this inequality may be checked by plotting the two functions of
$x$. For $x\geq 12$ we argue that then

$\log(x)+x\log(2x)<x\log(3x)\quad\text{and}\quad 3x<\log(x)+x\log(2x).$

∎

We add that in Lemma 2.5 we also have the upper bound $x<\log(y)$, for $x>6$.

##  3 Short loops on hyperelliptic translation surfaces

In this section we apply the methods used in [BMM] in order to show that, in
the same way as for hyperbolic surfaces, large systoles on translation
surfaces do not occur if they are hyperelliptic. As only topological and area
arguments shall be used for this we consider, more generally, any compact
orientable surface endowed with a Riemannian metric of non-positive curvature
that has finitely many cone-like singularities all of whose cone angles are
greater than $2\pi$. We call them _cn(0) surfaces_ (short for “two dimensional
cone manifold of curvature $\leq 0$”). Any translation surface is a cn(0)
surface. The relevant property of a cn(0)-surface is that any embedded open
disk $B_{r}(q)$ of radius $r$ centered at some point $q$ has area

$\operatorname{area}(B_{r}(q))\geq\pi\,r^{2}.$ (8)

A cn(0) surface $S$ of genus $g$ is called _hyperelliptic_ if there exists an
isometry $\phi:S\to S$ of order two, called the _hyperelliptic involution_ ,
that has $2g+2$ fixed points. The fixed points are called the _Weierstrass
points_ ; some of the Weierstrass points may be at the same time cone points.

We first restrict ourselves to the systole for which the arguments are fairly
straightforward. The normalisation of the area to $4\pi g$ in the following
theorem is made to simplify the calculations. The theorem is stated for the
homological systole $\operatorname{sys}_{h}(S)$.

######  Theorem 3.1 (Homology systoles of hyperelliptic surfaces).

Let $S$ be a hyperelliptic cn(0) surface of genus $g\geq 2$ and area $4\pi g$.
Then

$\operatorname{sys}_{h}(S)\leq 4\sqrt{2}\text{ \ and \
}\frac{\operatorname{sys}_{h}(S)^{2}}{\operatorname{area}(S)}\leq\frac{8}{\pi
g}.$

###### Proof.

We formulate the proof in such a way that it also serves as Step 1 in the
proof of the next theorem. Let $p_{1}^{*},\ldots,p_{2g+2}^{*}$ be the
Weierstrass points fixed by the hyperelliptic involution $\phi:S\to S$. The
quotient $\Sigma=S/\phi$ under the action of $\phi$ is a topological sphere,
we denote by $p_{1},\ldots,p_{2g+2}$ the images of
$p_{1}^{*},\ldots,p_{2g+2}^{*}$ under the natural projection $\Pi:S\to\Sigma$
(see Figure 7). These too shall be called _Weierstrass points_. Since $\phi$
acts as an isometric involution on each $B_{r}(p_{i}^{*})$ the metric disks
$B_{r}(p_{i})$ on $\Sigma$ have half the areas of the $B_{r}(p_{i}^{*})$ on
$S$. Thus, by (8), as long as for given $i$ the radius $r$ is such that
$B_{r}(p_{i})$ is embedded,

$\operatorname{area}(B_{r}(p_{i}))=\frac{1}{2}\operatorname{area}(B_{r}(p_{i}^{*}))\geq\frac{\pi}{2}\cdot
r^{2},\quad i=1,\ldots,2g+2.$ (9)

$p_{1}$ $p_{1}^{*}$ $p_{2}$ $p_{2}^{*}$ $p_{10}$ $p_{9}$ $p_{8}$ $p_{7}$
$p_{6}$ $p_{5}$ $p_{4}$ $p_{3}$ $\beta$ $\alpha^{\prime}$
$\alpha^{\prime\prime}$ $\delta_{1}$ $q$ $\delta^{*}_{1}$ $S$ $\Sigma$

Figure 7: A hyperelliptic surface $S$ of genus four with ten Weierstrass
points.

The strategy is to find a short geodesic loop at some $p_{i}$ on $\Sigma$ and
lift it to a short homologically non-trivial closed curve on the covering
surface $S$. For this we look at the union of disks
$B_{r}(p_{1})\cup\ldots\cup B_{r}(p_{2g+2})$ on $\Sigma$ and let $r$ grow
continuously beginning with $r=0$ until one of the following two cases occurs
for the first time:

Case A) _the closure of a single disk comes to self-intersect._

Case B) _the closures of two different disks come to intersect._

Let $r_{1}$ be the smallest value of $r$ for which this occurs. Then the open
disks $B_{r_{1}}(p_{1}),\ldots,B_{r_{1}}(p_{2g+2})$ are embedded and pairwise
disjoint but not so all of the closed ones. Since the sum of the areas of the
disjoint disks is less than
$\operatorname{area}(\Sigma)=\frac{1}{2}\operatorname{area}(S)=2\pi g$ we have
by (9), $(2g+2)\frac{\pi}{2}r_{1}^{2}\leq 2\pi g$ which yields

$r_{1}\leq\sqrt{\frac{2g}{g+1}}<\sqrt{2}.$ (10)

Now suppose it is Case A) that is occurring for $r_{1}$, say for the disk at
$p_{1}$ (see Figure 7). Then there are two geodesic arcs of length $r_{1}$
emanating from $p$ meeting at their endpoints in some point $q$. This point
may be an ordinary point or a cone point. Since the open disk
$B_{r_{1}}(p_{1})$ is embedded the arcs meet at an angle $\pi$ provided $q$ is
an ordinary point respectively, in such a way that both angles at $q$ are
$\geq\pi$ if $q$ is a cone point. In either case the two arcs form a geodesic
loop $\delta_{1}$ (see Figure 7). It was shown in [Mu] that $\delta_{1}$ lifts
to a figure eight closed geodesic $\delta^{*}_{1}$ of length
$2\ell(\delta_{1})=4r_{1}$ with self intersection at the lift $p^{*}_{1}$ of
$p_{1}$. Splitting $\delta^{*}_{1}$ at $p^{*}_{1}$ into two loops we get two
non-separating simple closed geodesics $\alpha^{\prime}$ and
$\alpha^{\prime\prime}$ in the free homotopy classes of these loops, where

$\ell(\alpha^{\prime})=\ell(\alpha^{\prime\prime})\leq\ell(\delta_{1})=2r_{1}.$

Note that the homotopy classes of $\alpha^{\prime}$ and
$\alpha^{\prime\prime}$ may be identical. We also mention that the figure
eight geodesic can be split in another way so as to get a simple closed
homologically trivial geodesic $\beta$.

We turn to Case B). Here the closures of two distinct disks, say
$B_{r_{1}}(p_{1})$ and $B_{r_{1}}(p_{2})$ meet at some point $q$. By the same
argument as before the two geodesic arcs of length $r_{1}$ from $p_{1}$ to $q$
and from $p_{2}$ to $q$ together form a geodesic arc $\delta_{1}$ from $p_{1}$
to $p_{2}$. In this case $\delta_{1}$ lifts to a homotopically non-trivial
$\phi$-invariant simple closed geodesic $\alpha$ of length

$\ell(\alpha)=2\ell(\delta_{1})=4r_{1}.$

This completes the proof. ∎

######  Theorem 3.2 (Homology bases of hyperelliptic surfaces).

Let $S$ be a hyperelliptic cn(0) surface of genus $g\geq 2$ and area $4\pi g$.
Then there exist $\lceil\frac{2g+2}{3}\rceil$ geodesic loops
$(\alpha_{k})_{k=1,\ldots,\left\lceil\frac{2g+2}{3}\right\rceil}$ that can be
extended to a homology basis of $H_{1}(S,{\mathbb{Z}})$ such that

$\ell(\alpha_{k})\leq 4\cdot\sqrt{\frac{12g}{2g+5-3k}}\text{ \ and \
}\frac{\ell(\alpha_{k})^{2}}{\operatorname{area}(S)}\leq\frac{48}{\pi(2g+5-3k)}\text{
for all }k=1,\ldots,\left\lceil\frac{2g+2}{3}\right\rceil.$

This theorem implies Theorem 1.3 from the introduction.

###### Proof.

The proof has two parts. In the first part we construct a collection of
geodesic arcs on $\Sigma$ whose endpoints are Weierstrass points. This is
carried out by estimating areas of growing disks extending the procedure in
the proof of Theorem 3.1. The lifts of these arcs in $S$ form a family of
simple closed curves, however, this family may not be extendable to a homology
basis. In the second part of the proof we shall therefore restrict the family
to a subset so as to make the extension possible. The second part uses only
topological arguments.

_Part 1._ We let the common radius $r$ of the disks $B_{r}(p_{i})$ grow but
stepwise prevent certain disks from further growing. Step 1 has been carried
out in the proof of Theorem 3.1 at the end of which either Case A) or Case B)
has occurred for some radius $r=r_{1}$. In Case A) the open disk
$B_{r_{1}}(p_{1})$ is still embedded but its boundary has a tangential self
intersection and we have detected a geodesic loop $\delta_{1}$ of length
$2r_{1}$ with base point $p_{1}$. In this case we _freeze_ disk
$B_{r_{1}}(p_{1})$, meaning we now prevented it from growing further.

In Case B) two disjoint embedded open disks $B_{r_{1}}(p_{1})$,
$B_{r_{1}}(p_{2})$ are detected whose boundaries intersect each other
tangentially and a simple geodesic arc $\delta_{1}$ of length $2r_{1}$ from
$p_{1}$ to $p_{2}$ is detected. In this case we freeze $B_{r_{1}}(p_{1})$ and
$B_{r_{1}}(p_{2})$ i.e we prevent both of them from growing further.

We now continue the procedure letting the common radius of the non frozen
disks grow beyond $r_{1}$ until again Case A) or Case B) occurs, at which
moment we get a new geodesic arc, freeze the involved disks, and so on. If
several disks happen to come to touch at the same time, we can choose an
arbitrary order to turn such a case into a succession of individual steps with
only one new arc in each. We detail the general step for $m\geq 2$.

Step $\boldsymbol{m}$: Let $j_{m-1}$ denote the number of all disks that are
frozen at the end of Step $m{-}1$. If $j_{m-1}=2g+2$ the procedure has come to
halt at the end of the preceding step and we note $M:=m-1$. Otherwise, we
expand the remaining disks $(B_{r}(p_{i}))_{i=j_{m-1}+1,\ldots,2g+2}$, letting
the common radius $r$ grow further until again for some $r=r_{m}$ one of the
following occurs:

Case A) _the closure of a single disk comes to self-intersect._

Case B) _the closures of two different disks come to intersect._

In Case A) we we set $j_{m}=j_{m-1}+1$, assume, by adapting the notation, that
the center of the said disk is $p_{j_{m}}$, and get a simple geodesic loop
$\delta_{m}$ of length $\ell(\delta_{m})=2r_{m}$ based at $p_{j_{m}}$, where
$r_{m}$ is the radius currently reached. Disk $B_{r_{m}}(p_{j_{m}})$ gets
frozen.

In Case B) for $r=r_{m}$ two subcases may occur: B1) an expanding disk
w.l.o.g. centered at $p_{j_{m-1}+1}$ touches an already frozen disk in which
case we set $j_{m}=j_{m-1}+1$ and freeze $B_{r_{m}}(p_{j_{m}})$ or, B2) two
expanding disks w.l.o.g. centered at $p_{j_{m-1}+1}$ and $p_{j_{m-1}+2}$ come
to touch for $r=r_{m}$ in which case we set $j_{m}=j_{m-1}+2$ and freeze both,
$B_{r_{m}}(p_{j_{m}-1})$ and $B_{r_{m}}(p_{j_{m}})$. In either case we get a
simple geodesic arc $\delta_{m}$ from $p_{j_{m}}$ to $p_{j_{m}-k}$ for some
$k$.

The procedure stops at the end of Step $M$ at which moment it has produced
simple geodesic arcs and loops $\delta_{1},\ldots,\delta_{M}$ that pairwise
intersect each other at most at their endpoints, with $g+1\leq M\leq 2g+2$.

To estimate the lengths of the $\delta_{m}$ we note that by (9) we have for
the $2g+2-j_{m-1}$ growing disks in Step $m$ at the moment when $r=r_{m}$,

$(2g+2-j_{m-1})\frac{\pi
r_{m}^{2}}{2}\leq\sum_{i=j_{m-1}+1}^{2g+2}\operatorname{area}(B_{r_{m}}(p_{i}))\leq\operatorname{area}(\Sigma)=2\pi
g.$

This yields

$\frac{\ell(\delta_{m})}{2}=r_{m}\leq\sqrt{\frac{4g}{2g+2-j_{m-1}}},\quad
m=1,\ldots,M$ (11)

(with $j_{0}=0$). We finish Part 1 by analysing the combinatorial properties
of the graph formed by $\delta_{1},\ldots,\delta_{M}$. More precisely, for any
$m\in\\{1,\ldots,M\\}$, we let ${\mathcal{H}}_{m}$ be the graph embedded in
the sphere $\Sigma$ whose vertex set is the set of Weierstrass points
$p_{1},\ldots,p_{2g+2}$ and the edges are $\delta_{1},\ldots,\delta_{m}$. In
the following loop means closed edge.

######  Lemma 3.3.

Each ${\mathcal{H}}={\mathcal{H}}_{m}$ has the following properties.

  1. (i)

${\mathcal{H}}$ is embedded in $\Sigma$.

  2. (ii)

The vertices of ${\mathcal{H}}$ are the Weierstrass points
$p_{1},\ldots,p_{2g+2}$.

  3. (iii)

For any loop $\lambda$ of ${\mathcal{H}}$ each of the two open components of
$\Sigma\setminus\lambda$ contains at least one vertex of ${\mathcal{H}}$.

  4. (iv)

Each connected component of ${\mathcal{H}}$ is one of the following

     * –

an isolated vertex,

     * –

a non-trivial tree,

     * –

an isolated loop,

     * –

a non-trivial tree with exactly one loop attached to it.

Here _non-trivial_ means that the tree has at least one edge. In what follows,
_any_ graph ${\mathcal{H}}$ with properties (i) - (iv) will be called a _graph
of type W_ (the $W$ alludes to the Weierstrass points).

###### Proof.

Properties (i) and (ii) hold by the definition of the ${\mathcal{H}}_{m}$.
Property (iii) follows from the Gauss-Bonnet theorem since the curvature of
the metric we are dealing with is $\leq 0$ away from the Weierstrass points.
For the properties of the components in (iv) we proceed by induction on the
stepwise search process of the $\delta_{m}$.

At the end of Step 1 we have graph ${\mathcal{H}}_{1}$ whose components are
vertices plus one edge that is either closed or connects distinct vertices.

Now assume that the lemma holds for the graph ${\mathcal{H}}_{m-1}$ which is
given at the beginning of Step $m$. Let $\delta_{m}$ be the new geodesic arc
obtained at this step. By construction the initial point $p_{j_{m}}$ of
$\delta_{m}$ is a Weierstrass point to which no edge of ${\mathcal{H}}_{m-1}$
is attached. For the other endpoint of $\delta_{m}$ there are two
possibilities: 1.) it coincides with $p_{j_{m}}$ in which case $\delta_{m}$
together with $p_{j_{m}}$ is a component of ${\mathcal{H}}_{m}$, or 2.)
$\delta_{m}$ connects $p_{j_{m}}$ with a component ${\mathcal{T}}$ of
${\mathcal{H}}_{m-1}$ in which case
${\mathcal{T}}^{\prime}:={\mathcal{T}}+\delta_{m}+p_{j_{m}}$ is a tree in case
${\mathcal{T}}$ is an isolated vertex or a tree, respectively a tree with
exactly one closed attached edge in case ${\mathcal{T}}$ is a loop or a tree
with an attached loop. Hence, ${\mathcal{H}}_{m}$ too satisfies (i) – (iv). ∎

We add that, by construction, the final graph ${\mathcal{H}}_{M}$ has no
isolated vertices.

_Part 2._ The lifts of $\delta_{1},\ldots,\delta_{M}$ to $S$ under the
hyperelliptic covering $\Pi:S\to\Sigma$ are generally not extendable to a
basis of $H_{1}(S,{\mathbb{Z}})$. We therefore remove certain parts to remedy
this. The arguments make no use of metric geometry. They are carried out in
[BMM] and we just mention the main steps for clarity.

Two types of removals come into play. One is to remove edges from
${\mathcal{H}}_{M}$, the other takes place on $S$ and is as follows: Let
${\mathcal{H}}$ be a subgraph of ${\mathcal{H}}_{M}$ obtained by removing some
edges but leaving the vertex set unchanged. Then ${\mathcal{H}}$ is again of
type $W$. The inverse image ${\mathcal{H}}^{*}$ of ${\mathcal{H}}$ under the
covering map $\Pi:S\to\Sigma$ consists of simple closed curves (the lifts of
the edges of ${\mathcal{H}}$ that are not loops) and figure eight shaped
curves (the lifts of the loops of ${\mathcal{H}}$). We now modify
${\mathcal{H}}^{*}$ by removing from each figure eight shape one of the two
simple loops it consists of (it does not matter which). The thus modified lift
shall be denoted by ${\mathcal{H}}^{\\#}$. It consists of simple closed curves
that pairwise intersect each other in at most one point.

The goal is now to remove certain edges from ${\mathcal{H}}_{M}$, as few as
possible, so that ${\mathcal{H}}^{\\#}$ extends to a basis. For concise
expression we say that closed curves $a_{1},\ldots,a_{n}$ form a _partial
basis_ if their homology classes $[a_{1}],\ldots,[a_{n}]$ can be extended to a
homology basis. The following sufficient criterion is used [BMM, Lemma 3.3]:

######  Lemma 3.4.

Let $a_{1},\ldots,a_{n}$ be distinct simple closed curves on a compact
orientable surface $F$ of genus $g\geq 1$. If
$F\smallsetminus(a_{1}\cup\ldots\cup a_{n})$ is connected then the curves form
a partial basis of $\smash{H}_{1}(F,{\mathbb{Z}})$.

It suffices therefore to reduce ${\mathcal{H}}_{M}$ to ${\mathcal{H}}$ in such
a way that $S\setminus{\mathcal{H}}^{\\#}$ is connected. For this we have
another criterion [BMM, Lemma 3.2]:

######  Lemma 3.5.

${\mathcal{H}}^{\\#}$ in $S$ is non-separating if and only if any open
connected component of $\Sigma\smallsetminus{\mathcal{H}}$ admits a Jordan
curve $\mathcal{J}$ that separates $p_{1},\ldots,p_{2g+2}$ into two odd
subsets i.e. the number of Weierstrass points on either side of $\mathcal{J}$
is odd.

The property expressed in this lemma shall be called the _Weierstrass point
separation property_. It thus remains to delete edges from ${\mathcal{H}}_{M}$
so that the resulting graph ${\mathcal{H}}$ has this property. This is carried
out by a pruning algorithm in [BMM] which we do not reproduce here but merely
state its outcome in form of a lemma. At one point in the description of the
algorithm in [BMM] a curvature argument is made to show that on either side of
any loop of ${\mathcal{H}}_{M}$ on $\Sigma$ there lies at least one
Weierstrass point. Here we have shifted this property to item (iii) of Lemma
3.3 above and included it in the definition of a graph of type $W$ so as to
free the next lemma from metric hypotheses.

The term “edges” in the lemma englobes open and closed ones.

######  Lemma 3.6.

Let ${\mathcal{G}}$ be a graph of type $W$ embedded in the sphere $\Sigma$
with 2g+2 vertices and assume that none of the vertices is isolated. Then
${\mathcal{G}}$ contains a subgraph ${\mathcal{H}}$ of type $W$ with
$\left\lceil\frac{2g+2}{3}\right\rceil$ edges that has the Weierstrass point
separation property.

We now apply the lemma to ${\mathcal{G}}={\mathcal{H}}_{M}$, where we recall
that ${\mathcal{H}}_{M}$ has no isolated vertices. The edges $\delta_{m}$ of
${\mathcal{H}}_{M}$, for $m=1,\ldots,M$, have the length bounds (11). Lemma
3.6 yields a selection

$\delta_{m_{1}},\ldots,\delta_{m_{K}},\quad
K:=\left\lceil\frac{2g+2}{3}\right\rceil,$

that form a subgraphs ${\mathcal{H}}$ of type $W$ that has the Weierstrass
points separation property. We chose the order of the indices
$m_{1},\ldots,m_{K}$ in such a way that

$j_{m_{1}-1}<j_{m_{2}-1}<\ldots<j_{m_{K}-1}<2g+2.$

We then have

$j_{m_{k}-1}\leq 2g-K+1+k,\quad k=1,\ldots,K.$

By (11) and the definition of $K$ we get

$\ell(\delta_{m_{k}})\leq 2\sqrt{\frac{4g}{K+1-k}}\leq
2\sqrt{\frac{12g}{2g+5-3k}},\quad
k=1,\ldots,\left\lceil\frac{2g+2}{3}\right\rceil.$

The lifts of $\delta_{m_{1}},\ldots,\delta_{m_{K}}$ in ${\mathcal{H}}^{\\#}$
have at most twice the lengths and by Lemma 3.4 and 3.5 they form a partial
basis. This completes the proof. ∎

## References

  * [Ak] Akrout, H.: Singularités topologiques des systoles généralisées, Topology 42(2) (2003), 291–308.
  * [AM1] Akrout H. and Muetzel B. : Construction of Riemann surfaces with large systoles, Journal of Geometry 107 (2016), 187–205.
  * [AM2] Akrout, H. and Muetzel, B.: Construction of surfaces with large systolic ratio, arXiv:1311.1449 (2018).
  * [Ba] Bavard, C.: Systole et invariant d’Hermite J. Reine. Angew. Math. 482 (1997), 93–120.
  * [Bu] Buser, P.: Riemannsche Flächen mit grosser Kragenweite (German), Comment. Math. Helv. 53(1) (1978), 395–407.
  * [BMM] Buser, P., Makover, E. and Muetzel B.: Short homology basis for hyperelliptic hyperbolic surfaces, Isr. J. Math. (2023), https://doi.org/10.1007/s11856-023-2600-y.
  * [CHMW] Columbus T., Herrlich F., Muetzel B. and Weitze-Schmithüsen G.: Systolic geometry of translation surfaces, Exp. Math. (2022), doi:10.1080/10586458.2022.2108946.
  * [ES] Erdös P. and Sachs H.: Reguläre Graphen gegebener Taillenweite mit minimaler Knotenzahl, Wiss. Z. Martin-Luther-Univ. Halle-Wittenberg Math.-Naturwiss. Reihe 12 (1963), 251–257.
  * [Ge] Gendulphe, M.: Découpages et inégalités systoliques pour les surfaces hyperboliques à bord, Geometriae dedicata 142 (2009), 23–35.
  * [JP] Judge, Ch. and Parlier H.: The maximum number of systoles for genus two Riemann surfaces with abelian differentials, Comment. Math. Helv. 94(2) (2019), 399–437.
  * [KS] Katz, M. and Sabourau, S.: Entropy of systolically extremal surfaces and asymptotic bounds, Ergo. Th. Dynam. Sys. 25(4) (2005), 1209–1220.
  * [KZ] Kontsevich, M. and Zorich A.: Connected components of the moduli spaces of abelian differentials with prescribed singularities, Invent. Math. 153 (2003), 631–678.
  * [LPS] Lubotzky, A., Phillips, R. and Sarnak, P.: Ramanujan graphs, Combinatorica 8 (1988), 261–277.
  * [LS] Linial, N. and Simkin M.: A randomized construction of high girth regular graphs, Random Structures & Algorithms 58(2) (2021), 345–369.
  * [Ma] Massart, D.: A short introduction to translation surfaces, Veech surfaces, and Teich- mueller dynamics, In: Surveys in Geometry I, Springer (2022), 343–388.
  * [Mu] Muetzel, B.: On the second successive minimum of the Jacobian of a Riemann surface, Geom. Dedicata 161(1) (2012), 85–107.
  * [Pa] Parlier, H.: The homology systole of hyperbolic Riemann surfaces, Geom. Dedicata 157(1) (2012), 331–338.
  * [PW] Petri, B. and Walker A.: Graphs of large girth and surfaces of large systole, Math. Res. Lett. 25(6) (2018), 1937–1956.
  * [Ra] Rafi, K.: Thick-thin decomposition for quadratic differentials, Math. Res. Lett. 14(2) (2007), 333–341.
  * [Sc1] Schmutz, P.: Congruence subgroups and maximal Riemann surfaces, J. Geom. Anal. 4(2) (1994), 207–218.
  * [Sc2] Schmutz Schaller, P.: Riemann surfaces with shortest geodesic of maximal length, Geom. Funct. Anal. 3(6) (1993), 564–631.
  * [Wr] Wright, A.: Translation surfaces and their orbit closures: an introduction for a broad audience, EMS Surv. Math. Sci. 2(1) (2015), 63–108.
  * [Zo] Zorich, A.: Flat surfaces. Frontiers in number theory, physics, and geometry I, Springer, Berlin (2006), 437–583.

Peter Buser
Department of Mathematics, Ecole Polytechnique Fédérale de Lausanne
Station 8, 1015 Lausanne, Switzerland
e-mail<EMAIL_ADDRESS>

Eran Makover
Department of Mathematics, Central Connecticut State University
1615 Stanley Street, New Britain, CT 06050, USA
e-mail<EMAIL_ADDRESS>

Bjoern Muetzel
Department of Mathematics, Eckerd College
4200 54th avenue South, St. Petersburg, FL 33711, USA
e-mail<EMAIL_ADDRESS>