linktocpage

KINETIC ENERGY FLUCTUATION-DRIVEN LOCOMOTOR TRANSITIONS ON POTENTIAL ENERGY
LANDSCAPES OF BEAM OBSTACLE TRAVERSAL AND SELF-RIGHTING
by

Ratan Sadanand Othayoth Mullankandy
A dissertation submitted to Johns Hopkins University
in conformity with the requirements for the degree of
Doctor of Philosophy
Baltimore, Maryland

October, 2021

© 2021 by Ratan Sadanand Othayoth Mullankandy
All rights reserved

## Abstract

When moving in nature, animal contend with constraints imposed by their
environment, morphology, and physiology. Despite this seeming difficulty,
animals move amazingly well by making effective physical interaction with the
environment to use and transition between different modes of locomotion such
as walking, running, climbing, and self-righting. By contrast, robots struggle
to do so in real world. Understanding the principles of how locomotor
transitions emerge from constrained physical interaction is necessary for
robots to move robustly in nature by using and transitioning between locomotor
modes.

Recent studies of physical interaction with environment discovered that
discoid cockroaches use and transition between diverse locomotor modes to
traverse beams and self-right on ground. For both systems, animals
probabilistically transitioned between modes via multiple pathways, while its
self-propulsion created seemingly wasteful kinetic energy fluctuation. In this
dissertation, we seek mechanistic explanations for these observations by
adopting a physics-based approach that integrates biological and robotic
studies.

We discovered that animal and robot locomotor transitions during beam obstacle
traversal and ground self-righting are barrier-crossing transitions on
potential energy landscapes. Whereas animals and robot traversed stiff beams
by rolling their body between beam, they pushed across flimsy beams,
suggesting a concept of terradynamic favorability where modes with easier
physical interaction are more likely to occur. Robotic beam traversal revealed
that, system state either remains in a favorable mode or probabilistically
transitions to one when kinetic energy fluctuation is comparable to the
transition barrier. Robotic self-righting transitions occurred similarly and
additionally revealed that changing system parameters (wing opening) lowers
landscape barriers over which comparable kinetic energy fluctuation can induce
probabilistic transitions. Animals’ transitions in both systems mostly
occurred similarly, but sensory feedback may facilitate its beam traversal.
Finally, we developed a method to measure animal movement across large
spatiotemporal scales in an existing terrain treadmill.

## Dissertation Readers

###

Chen Li (Advisor)
Assistant Professor
Department of Mechanical Engineering
Johns Hopkins Whiting School of Engineering

Noah J. Cowan
Professor
Department of Mechanical Engineering
Johns Hopkins Whiting School of Engineering

Louis L. Whitcomb
Professor
Department of Mechanical Engineering
Johns Hopkins Whiting School of Engineering

###### Table of Contents

  1. 1 Introduction
    1. 1.1 Motivation and Significance
    2. 1.2 Background
      1. 1.2.1 Model system I: Beam obstacle traversal
      2. 1.2.2 Model system II: Winged self-righting on flat ground
    3. 1.3 Knowledge gap and challenges
    4. 1.4 Integrative approach for bridging the knowledge gap
      1. 1.4.1 Rationale and challenges in studying physical interaction in animals
      2. 1.4.2 Integrative approach to studying physical interaction during locomotion
    5. 1.5 Potential energy landscapes to model physical interaction
      1. 1.5.1 Inspiration: Free energy landscapes of protein folding transitions
      2. 1.5.2 Hypotheses
      3. 1.5.3 Drawbacks of previous potential energy landscapes
    6. 1.6 Organization of Chapters
  2. 2 Kinetic energy fluctuation from oscillatory self-propulsion facilitates barrier-crossing locomotor transitions during beam traversal
    1. 2.1 Summary
    2. 2.2 Author contributions
    3. 2.3 Introduction
    4. 2.4 Methods
      1. 2.4.1 Robotic physical model
      2. 2.4.2 Robot beam obstacles
      3. 2.4.3 Robot experiment imaging
      4. 2.4.4 Robot experiment protocol
      5. 2.4.5 Animals
      6. 2.4.6 Animal beam obstacles
      7. 2.4.7 Animal multi-camera imaging arena
      8. 2.4.8 Animal experiment protocol
      9. 2.4.9 High accuracy 3-D motion reconstruction
      10. 2.4.10 Definition of pitch and roll modes and pitch-to-roll transition
      11. 2.4.11 Data averaging
      12. 2.4.12 Kinetic energy fluctuation
      13. 2.4.13 Statistics
      14. 2.4.14 Potential energy landscape
      15. 2.4.15 Local minima and system state trajectories on potential energy landscape
      16. 2.4.16 Average potential energy landscape at each beam stiffness
      17. 2.4.17 Percentage of trials in which system is attracted to basin of observed mode on actual landscape
      18. 2.4.18 Energy barrier to escape from pitch local minimum
      19. 2.4.19 Robot system state velocity directions
      20. 2.4.20 Animal active body and limb adjustments
    5. 2.5 Results
    6. 2.6 Discussion
      1. 2.6.1 Towards understanding emergent behaviour in natural environments
  3. 3 Propelling and perturbing appendages together facilitate strenuous ground self-righting
    1. 3.1 Summary
    2. 3.2 Significance Statement
    3. 3.3 Author contributions
    4. 3.4 Introduction
    5. 3.5 Methods
    6. 3.6 Animals
      1. 3.6.1 Leg modification
      2. 3.6.2 Experiment protocol
      3. 3.6.3 Self-righting performance
      4. 3.6.4 Preference of self-righting strategies
      5. 3.6.5 Kinetic energy measurement
      6. 3.6.6 Relationship between wing opening and leg flailing
    7. 3.6.7 Design and actuation
    8. 3.6.8 Similarity to animal
    9. 3.6.9 Experiment protocol
    10. 3.6.10 Self-righting performance
    11. 3.6.11 Robot 3-D motion reconstruction
    12. 3.6.12 Kinetic energy measurements
    13. 3.7 Potential energy landscape modelling
      1. 3.7.1 Model definition
      2. 3.7.2 System state trajectories on potential energy landscape
      3. 3.7.3 Potential energy barrier measurements
      4. 3.7.4 Comparison of kinetic energy and potential energy barriers
      5. 3.7.5 Data analysis and statistics
    14. 3.8 Results
      1. 3.8.1 Leg flailing facilitates animal winged self-righting
      2. 3.8.2 Wing opening and leg flailing together facilitate robot self-righting
      3. 3.8.3 Robot self-righting resembles animal’s
      4. 3.8.4 Robot and animal have similar evolving potential energy landscapes
      5. 3.8.5 Self-righting transitions are destabilizing,  barrier-crossing transitions on landscape
      6. 3.8.6 Self-righting via rolling overcomes smaller barrier than via pitching
      7. 3.8.7 Barrier reduction by wing opening facilitates self-righting via rolling
    15. 3.9 Discussion
      1. 3.9.1 Stereotyped motion emerges from physical interaction constraint
      2. 3.9.2 Towards potential energy landscape theory of self-righting transitions
  4. 4 Tracking and reconstructing large obstacle interaction of small animals using a terrain treadmill
    1. 4.1 Summary statement
    2. 4.2 Author Contributions
    3. 4.3 Introduction
    4. 4.4 Methods
      1. 4.4.1 Experiment and data collection
      2. 4.4.2 Measuring animal movement in obstacle field
      3. 4.4.3 Unwrapped 2-D trajectory
    5. 4.5 Results
      1. 4.5.1 Free locomotion at large spatiotemporal scales
      2. 4.5.2 Animal-obstacle interaction
      3. 4.5.3 Multiple behaviors and behavioral transitions
    6. 4.6 Discussion
  5. 5 Conclusion
    1. 5.1 General remarks
      1. 5.1.1 Reasoning about terradynamic favorability during physical interaction
      2. 5.1.2 Criteria for barrier-crossing locomotor transitions
    2. 5.2 Stereotypy and Variations
    3. 5.3 Implications for animal and robot locomotion
    4. 5.4 Future directions
      1. 5.4.1 Neuromechanical interaction with during locomotor transitions
      2. 5.4.2 Decision making in natural environments
      3. 5.4.3 Detecting and leveraging terrain affordances
    5. 5.5 Closing remarks

###### List of Tables

  1. 2.1 Geometric dimensions, physical properties, and sample sizes for animal and robot experiments.
  2. 3.1  Mass distribution of the robot.
  3. 3.2  Comparison between animal and robot.

###### List of Figures

  1. 1.1 Physical interaction during terrestrial locomotion is constrained by environmental, morphological, and physiological constraints
  2. 1.2 Illustrative examples of animal and robot locomotor transitions.
  3. 1.3 Bioinspired approach in legged locomotion.
  4. 1.4 Model organism and model systems
  5. 1.5 Model system I : Beam obstacle traversal
  6. 1.6 Model system II : Ground self-righting
  7. 1.7 Neuromechanical feedback during locomotion
  8. 1.8 Integrative approach to understand physical interaction in different media
  9. 1.9 Free energy landscapes of protein folding transitions.
  10. 1.10 Minimal potential energy landscape of flexible beam traversal
  11. 1.11 Static potential energy landscape of ground self-righting.
  12. 2.1 A cockroach transitioning (orange arrow) from pitch to roll mode to traverse grass-like beam obstacles.
  13. 2.2  Robotic physical model.
  14. 2.3 Design of robotic physical model and rigid beams with torsional springs at base.
  15. 2.4 Robot beam stiffness characterization
  16. 2.5 Design of rigid beams with torsional springs at base for animal experiments.
  17. 2.6 Characterization of rigid beams with torsional springs at base for animal experiments
  18. 2.7 Animal locomotion arena with a layer of beam obstacles (green), with seven high-speed cameras.
  19. 2.8 Snapshot of animal traversing beam obstacles
  20. 2.9 Representative motion of body and beams and system potential energy during interaction and definition of traversal and pitch-to-roll transition.
  21. 2.10 Kinetic energy fluctuation
  22. 2.11 Potential energy landscape model, with definition of variables and parameters.
  23. 2.12 Pitch-to-roll transition probability of animal and robot as a function of beam stiffness K
  24. 2.13 Robot locomotor transitions on a potential energy landscape
  25. 2.14 Robot locomotor transitions are stochastic and become more likely as kinetic energy fluctuation increases
  26. 2.15 Robot tends to transition to roll basin when kinetic energy fluctuation is comparable to potential energy barrier to escape pitch local minimum and towards direction of lower barrier
  27. 2.16 Favorability measure for robot
  28. 2.17 Animal tends to transition to roll basin when kinetic energy fluctuation is comparable to potential energy barrier to escape pitch basin and when roll basin is more favorable.
  29. 2.18 Comparison of energy landscape between protein-folding transitions and locomotor transition.
  30. 3.1 Strenuous, leg-assisted, winged ground self-righting of discoid cockroach
  31. 3.2 Animal leg modification and robotic physical model
  32. 3.3 Robot wing and leg actuation and body orientation measurement.
  33. 3.4 Animal’s kinetic energy and self-righting probability
  34. 3.5 Comparison of average percentage of time spent on winged and legged self-righting attempts between animals with intact and modified legs
  35. 3.6 Animal kinetic energy calculation
  36. 3.7 Correlation between animal’s body and leg motion
  37. 3.8 Robot’s kinetic energy and self-righting performance.
  38. 3.9 Robot’s self-righting motion and potential energy landscape.
  39. 3.10 Animal’s potential energy landscape.
  40. 3.11 Robot state trajectories on potential energy landscape.
  41. 3.12 Robot’s stereotyped body motion during self-righting
  42. 3.13 Robot’s potential energy barriers for self-righting via pitch and roll modes.
  43. 3.14 Animal’s potential energy barriers for self-righting via pitch and roll modes.
  44. 3.15 Comparison between robot’s kinetic energy and potential energy barrier along roll direction.
  45. 3.16 Comparison between robot’s pitch kinetic energy and pitch potential energy barrier.
  46. 3.17 Dependence of potential energy landscape on ground geometry.
  47. 4.1 Terrain treadmill
  48. 4.2 Measuring motion of animal exploring sparse pillar fields.
  49. 4.3 Animal behavior and performance of the treadmill.
  50. 4.4 Representative metrics and 3-D reconstruction of animal exploring sparse pillar obstacle field.

##  Chapter 1 Introduction

###  1.1 Motivation and Significance

Movement is one of the most ubiquitous and conspicuous features of animals
(Alexander, 2006; Biewener, 2003; Tinbergen, 1955) and occurs across diverse
environments ranging from rainforests to deserts to flat plains, occurring
over large spatiotemporal scale. For effective terrestrial locomotion in such
natural environments, animals must generate necessary forces (Taylor and
Heglund, 1982) to both support and propel themselves, which requires
significant physical interaction of limbs and often the body with the terrain
(Alexander, 2006; Biewener, 2003; Dickinson et al., 2000). However, when
moving in real world, physical interaction between animal and environment
during is often complex (Dickinson et al., 2000; Holmes et al., 2006) even in
simple, homogeneous environments. Because the environment is spatially and
temporally variable, so is the physical interaction with it. In addition, to
the heterogeneity and variation in physical properties such as shape, size,
and stiffness of terrain, as well as continual animal-terrain contact animals
must operate under the constraints imposed by their morphology, physiology,
and environment (Figure 1.1. These may limit the propulsive that the animal
can generate (Dickinson et al., 2000; Taylor and Heglund, 1982) to move in
environment.

Figure 1.1: Physical interaction during terrestrial locomotion is constrained
by environmental, morphological, and physiological constraints. Examples of
physical interaction during locomotion. (A) American cockroach exiting a
crevice. (B) Mouse passing trough a vertical gap. (C) Search and Rescue dog
moving in rubble. (D) Lizards moving through flexible shrubs. Images courtesy
of (A) PolyPEDAL Lab, (B) Getty Images, (C) Ground Zero Emergency Canine
Training, (D) LLL Reptile and Supply Co Inc.

Despite these seeming difficulties during physical interaction, terrestrial
locomotion in biological organisms is robust and agile. Even with
perturbations from continual contact and instabilities in environment
(Sponberg and Full, 2008; Biewener and Daley, 2007) and the biomechanical
constraints (Biewener, 2003; Holmes et al., 2006), animals adjust their
physical interaction to maintain dynamic running (Jindrich and Full, 2002;
Full et al., 2006) and walking. Furthermore, in the extreme case of losing
foothold and flipping over on their back, animals self-right to get back on
their feet and continue moving (Ashe, 1970; Full et al., 1995; Li et al.,
2019). Often, running or walking alone cannot accomplish effective locomotion
in varying environments (e.g., cluttered forest floor, sparse branches in
canopy, etc.) and animals must transition (Lock et al., 2013; Low et al.,
2015) between different modes of locomotion such as climbing (Goldman et al.,
2006), rolling (Domokos and Várkonyi, 2008), burrowing (Winter et al., 2014),
or even self-righting (Li et al., 2019) (Figure 1.2).

Figure 1.2: Illustrative examples of animal and robot locomotor transitions.
Reproduced from (Othayoth et al., 2020)

By contrast, although advancements in robotics over the past decades
(Altendorfer et al., 2001; Raibert, 1986, 2008) have enabled robots to walk
and run stably even across surfaces that are rigid (Raibert, 1986, 2008),
rugged (Altendorfer et al., 2001), and yielding (Li et al., 2009, 2013;
Aguilar and Goldman, 2016), they either lack the ability or struggle to
robustly transition between modes beyond stable walking and running (Guizzo
and Ackerman, 2015; Yang et al., 2018). It is crucial to be able to robustly
transition between different modes of locomotion for robots to move
effectively in natural, artificial or extraterrestrial environments (which
often have physical constraints) to assist or autonomously perform tasks such
as search and rescue (Murphy et al., 2008), environmental monitoring (Dunbabin
and Marques, 2012), extraterrestrial exploration (Titus et al., 2021; Li and
Lewis, 2022), and home service (Forlizzi and DiSalvo, 2006).

A major hurdle towards this grand vision in robotics is that beyond stable
running and walking, we do not yet know how to effectively generate and use
forces to move in environment when physical interaction is constrained by the
robot’s environment and morphology. For example, how should the robot move its
body and legs to exert forces against its environment to enter a cave opening
(Titus et al., 2021) narrower than its body, or regain foothold when fallen
over? Without such capabilities, robots struggle or even fail to move in the
real world. To enable these capabilities, we need general physics models that
can inform us how to predict and generate the necessary forces (Koditschek,
2021) to interact with environment for transitioning to, avoiding, or escaping
from locomotor modes beyond walking and running. Improving robotic mobility in
extreme environments is a grand challenge in robotics (Yang et al., 2018), and
understanding the physics of locomotor transitions is an essential ingredient
for addressing this challenge.

Figure 1.3: Bioinspired approach in legged locomotion.

While the challenge is formidable, a bioinspired approach (Sharbafi et al.,
2017; Lynch et al., 2012; Pfeifer et al., 2007) proved tractable and
successful. In this approach (Figure 1.3), the principles learnt from
biological systems were translated to create engineering design rules and
general physics models that predicted forces and informed robot design to
improve their mobility. Such simplified physics models (Blickhan and Full,
1993; Dickinson et al., 1999; Hu et al., 2009; Li et al., 2013), either
derived analytically or synthesized from systematic experimental studies of
animals and their robotic physical models (Aguilar et al., 2016; Aydin et al.,
2019), have been successful in revealing principles of generating and
maintaining steady state locomotion in modes such as walking (Kuo, 2007),
running (Blickhan and Full, 1993), vertical climbing (Goldman et al., 2006).
For example, the simplest model of running on ground, spring-loaded inverted
pendulum model (SLIP (Blickhan and Full, 1993)) was inspired from studying
legged animals and have advanced the capability of robots to run stably and
autonomously in moderately rugged environments (Altendorfer et al., 2001). It
also described the fundamental dynamics of running and hopping in two-, four-,
six- and eight-legged animals (Blickhan and Full, 1993). In addition to
improving robotic mobility, these physics models help understand general
principles spanning different biological systems.

###  1.2 Background

Previous studies predominantly focused on generating (Blickhan and Full, 1993;
Goldman et al., 2006; Kuo, 2007; Li et al., 2012), stabilizing (Biewener and
Daley, 2007; Couzin-Fuchs et al., 2015; Revzen et al., 2013) or transitioning
between (Bramble and Lieberman, 2004; Diedrich and Warren, 1995; Hoyt and
Taylor, 1981; Li, 2000) steady-state between walking and running. But insights
from these studies do not translate to scenarios where the animal or robot
must make locomotor transitions using physical interaction while operating
under environmental or biomechanical constraints, which is often the case when
moving in real world. The studies in this dissertation are motivated by the
recent observations of physical interaction during traversal of flexible beam-
like obstacles (Li et al., 2015) and self-righting on flat ground (Li et al.,
2019) in discoid cockroaches (Blaberus discoidalis, Figure 1.4).

In both model systems, the animal displayed diverse, probabilistic locomotor
transitions that emerged via constrained physical interaction with its
environment. To traverse flexible beam obstacles or self-right, animals must
physically interact with the environment, which is often constrained or
strenuous. For example, traversing layers of adjacent beams with a gap
narrower than their body width is difficult for animals—so is pushing against
stiff beams which may not deflect easily due to large restoring forces.
Similarly, to self-right, animals must overcome potential energy barriers that
are seven times greater than the mechanical energy required per stride for
steady-state, medium speed running (Kram et al., 1997) or, exert ground
reaction forces eight times greater than that during steady-state medium speed
running (Full et al., 1995). In the next sections, we briefly summarize the
results from the previous studies of locomotor transitions in beam obstacle
traversal (Li et al., 2015) and ground self-righting of cockroaches (Li et
al., 2019).

Figure 1.4: Model organism and model systems. Discoid cockroach (Blaberus
discoidalis) (A) traversing flexible beams and (B) self-righting on flat
ground. Images courtesy of Will Kirk.

####  1.2.1 Model system I: Beam obstacle traversal

To begin to advance terradynamics, (the study of locomotor-terrain
interactions (Li et al., 2013)) into three-dimensions, beyond relatively
uniform granular media, Li et al. (2015) challenged cockroaches to traverse
flexible beams with gaps smaller than their body width. The study discovered
that during the physical interaction to traverse the beam obstacles,
cockroaches used different modes such as climbing up the beams, rolling in
between the beam gaps, pushing against the beams, falling forward, or moving
laterally (Figure 1.5A). Animals probabilistically transitioned between
different modes and did so via multiple pathways (Figure 1.5B) during
traversal attempts. Some modes and transitions were more probable than other,
with traversal most likely to occur via body rolling. This fact was attributed
to the streamlined ellipsoidal shape of the cockroach, which induced body
rolling from passive mechanical interactions. The animal also experienced
constant body vibrations due to intermittent ground contact. A minimal
potential energy landscape was used to speculate that during traversal, the
animal must overcome a potential energy barrier (which varied with modes) (see
Section 1.5 for details).

Figure 1.5: Model system I : Beam obstacle traversal. (A) Schematic of diverse
locomotor modes observed during beam traversal. (B) Multi-pathway locomotor
transitions and their probabilities during beam traversal. Reproduced from (Li
et al., 2015)

####  1.2.2 Model system II: Winged self-righting on flat ground

An interesting observation (Li et al., 2015) in beam obstacle traversal led to
a more detailed study on self-righting (Li et al., 2019). As the cockroach
traversed and exited the beam obstacle field, it occasionally became unstable,
lost its foothold, and fell on its back but almost always self-righted to
continue moving. Li et al. (2019) discovered that cockroaches self-right on
flat ground via diverse strategies such as by opening their wings against
ground (winged self-righting) or by pushing their legs against the ground
(legged self-righting) (Figure 1.6A). It was also observed that physical
interaction with flat ground resulted in probabilistic transitions between
self-righting modes via multiple pathways Figure 1.6. Certain modes and
transitions were more likely to occur. In addition, the animal frequently and
desperately flailed its legs during its self-righting attempts; it was
presumed that these created small kinetic energy fluctuation. Finally, a
static potential energy landscape model (that considered body rotation but not
wing opening; see Section 1.5 for details) observed that different modes
overcame varying potential energy barriers.

These common observations suggest that despite their differences, studying
both model systems together may help understand how the locomotor transitions
emerge from physical interactions in beam traversal and self-righting.

Figure 1.6: Model system II : Ground self-righting (A) Winged and legged modes
observed during self-righting. (B) Multi-pathway locomotor transitions and
their probabilities during beam traversal. Reproduced from (Li et al., 2019)

###  1.3 Knowledge gap and challenges

The common themes across both model systems such as probabilistic, multi-
pathway transitions, likelihood of some modes over others, could not be
satisfactorily explained by the potential energy landscape modelling. More
broadly, there exists a knowledge gap in our understanding of how animals make
direct physical interaction with 3-D terrain to transition between locomotor
modes, and how robots should do so too. For example, although the existing
models such as spring-loaded inverted pendulum (SLIP) are effective and
generalize over a broad range of locomotor-terrain parameters for dynamic
running or walking, they do not inform or extrapolate to locomotor
transitions. Bridging this knowledge gap, in addition to advancing to our
understanding of how biological organisms move (Dickinson et al., 2000;
Padilla et al., 2014), will also enable robots to move robustly in nature for
relevant societal applications (Yang et al., 2018).

Given the common observations across both model systems (see Section 1.2) it
is possible, as previous studies of beam traversal and self-righting have
posited, that the potential energy landscape could be a conceptual framework
for thinking about how to generate and control locomotor transitions. However,
it has not matured sufficiently to quantitatively reason about our
observations of locomotor transitions or provide answers to above questions.
This dissertation is a step towards advancing the potential energy landscape
approach to be able to provide explanations for the above questions.

However, such an effort has its challenges (Holmes et al., 2006)—physical
interaction during locomotor transitions often involves intermittent contact
of the animal or robot with the environment, and their combined degrees of
freedom are large. In addition, continual collisions and frictional contact
introduce nonlinearities. Although the physical interaction obeys Newton’s
laws of motion, solving (or even deriving) the equation for such complex
systems is often intractable. Given these complexities, and the fact that
transitions are probabilistic in both model systems, a statistical physics-
like approach may prove useful. A statistical physics treatment has advanced
understanding of complex, stochastic, macroscopic phase transitions in self-
propelled living systems, such as animal foraging (Viswanathan et al., 2011),
traffic (Helbing, 2001), and active matter (Fodor and Marchetti, 2018;
Ramaswamy, 2010).

Beginning to answer these questions requires an interdisciplinary approach
integrating biology, robotics, and physics.

###  1.4 Integrative approach for bridging the knowledge gap

####  1.4.1 Rationale and challenges in studying physical interaction in
animals

“The essential function of a robot is to perform work on its environment
specified by its user (Koditschek, 2021)”. In the same vein, at its most
fundamental level, animals are mechanical systems that physically interact
with the environment to propel themselves (Dickinson et al., 2000).
Investigating the mechanics of physical interaction with environment is a
seemingly obvious first step towards understanding how organism, and how robot
can, elicit (or avoid) a desired locomotor transition.

However, locomotion in biological systems emerges (Dickinson et al., 2000;
Anderson, 1972) not just from mechanical interactions; animals can also sense
and adjust its mechanical interaction in response to the sensed information
via feedback, both neural and mechanical (Dickinson et al., 2000) (Figure
1.7). For example, a when fetching balls, a dog adjusts its running speed and
direction based on the sensory information from its eyes. As a result, even if
our focus is to tease apart the role of passive physical interaction alone, we
must consider the possibility that effects of sensory feedback may not be
entirely avoidable.

Here, we first focus on understanding passive mechanical interaction, which
provides a foundation for understanding sensory feedback control. This
approach is inspired from early studies of aerodynamics of passive airfoils
(Cayley, 1876). Although airfoils were extremely simplified models of bird
wings, these studies a provided physics insights about flight control, which
were lacking in detailed anatomical studies and observation of birds at the
time. We follow a similar approach here by studying not just the animal, but
also its simplified robotic model and compare them to gain physical insights.
To make this comparison meaningful, we will study locomotor transitions of the
animal during its rapid, bandwidth-limited locomotion during which sensory
feedback is minimal due to delays in neuronal transmission speeds (Figure
1.7).

Figure 1.7: Neuromechanical feedback during locomotion. Neural sensory
feedback (blue lines) is minimal during rapid bandwidth-limited locomotion.
Image modified from Dickinson et al. (2000).

####  1.4.2 Integrative approach to studying physical interaction during
locomotion

Considering the challenges discussed in Section 1.3 and the coupled
neuromechanical interactions, studying physical interaction during locomotor
transitions seems daunting at first, but a physics-based approach that has
begun to advance terradynamics (Li et al., 2013, 2015) (the study of
locomotor-terrain interactions) may prove beneficial. Such an approach, rooted
in physics and integrating biology and robots has previously helped understand
fluid–structure physical interaction in aerial and aquatic locomotion of
animals (Dickinson et al., 1999; Lauder and EG, 2002) and robots (Teoh and
Wood, 2013; Zhu et al., 2019) , we understand fairly well their thanks to
well-established experimental, theoretical and computational tools, such as
wind tunnel and water channel, airfoil and hydrofoil, aero- and hydrodynamic
theories, and computational fluid dynamics techniques (Vogel, 1996)(Figure
1.8).

By creating controlled granular media testbeds, robotic physical models, and
theoretical and computational models, recent studies elucidated how animals
(and how robots should) use physical interaction with granular media to move
effectively both on and within sandy terrain. (see (Goldman, 2014) for a
review). The general physical principles (Goldman, 2014) and predictive
physics models (Goldman, 2014; Li et al., 2013) not only advanced
understanding of functional morphology (Li et al., 2012; Maladen et al., 2011;
Sharpe et al., 2015), muscular control (Ding et al., 2013; Sharpe et al.,
2013), and evolution (McInroe et al., 2016) of animals, but also led to new
design and control strategies (Aguilar et al., 2016; Goldman, 2014; Li et al.,
2009, 2010; Marvi et al., 2014; Shrivastava et al., 2020) that enabled a
diversity of robots to traverse granular environments.

Figure 1.8: Integrative approach to understand physical interaction in
different media.

Such an integrated approach offers several advantages.

  * •

Observations of model organisms inspire robot design and action. For example,
insights from studying walking and running of cockroaches inspired the design
of the RHex robot (Altendorfer et al., 2001).

  * •

Simplified robots serve as physical models for testing biological hypotheses
or generating new ones and allow control and variation of parameters to
discover general principles. For example, the RHex-like robot was used to
demonstrate that physical interaction of robots and animals with vertical
pillar obstacles depends sensitively on robot body shape but not the pillar
shape or geometry (Han et al., 2021).

  * •

Physical principles and predictive models from this empirical approach provide
mechanistic explanations for animal locomotion and design tools and action
strategies for robots. For example, a robotic testbed (Robofly) provided
mechanistic insights into how insects fly by flapping their wings (Dickinson
et al., 1999), which later informed design of miniature flying robot – Robobee
(Teoh and Wood, 2013).

Inspired by these successes, we will adopt a similar physics-based approach by
integrating biological and robotic studies of beam traversal and self-righting
with potential energy landscape modelling of the physical interaction.

###  1.5 Potential energy landscapes to model physical interaction

Figure 1.9: Free energy landscapes of protein folding transitions. Left:
Funnel shaped free energy landscapes of protein folding. As proteins fold,
their state moves transitions from a high energy state to a low energy state
by overcoming free energy barriers. Modified from (Dill and MacCallum, 2012).
Right: Schematic of stochastic, multi-pathway protein folding transitions.
Thicker arrows show transitions that are more probable. Modified from (Voelz
et al., 2012)

####  1.5.1 Inspiration: Free energy landscapes of protein folding transitions

The potential energy landscapes for modelling physical interaction during
locomotor transition are directly inspired by free energy landscapes of multi-
pathway protein folding transitions (Dill and Chan, 1997; Dill et al., 2008;
Onuchic and Wolynes, 2004; Wales, 2003). Free energy landscape is a function
of all possible confirmations (molecular structure) of a protein molecule
(Figure 1.9). Microscopic, near-equilibrium proteins have a three-dimensional
chain-like conformation which is initially unfolded and has a high free
energy. To achieve its biological function, the unfolded protein chain must
fold into a specific confirmation called the native state, which has the
lowest possible free energy. However, the transition from unfolded to native
state does not occur in a single step; instead, unfolded states progressively
transition to various intermediate states that are reaching the native state.
Lower free energy states are more stable and hence thermodynamically
favorable. When the protein folding problem is seen through the lens of free
energy landscapes (Figure 1.9), following observations emerge:

  1. 1.

Free energy landscapes have peaks (local free energy maxima) and basins (local
free energy minima).

  2. 2.

When proteins fold, they from higher energy states/basins to lower energy
states/basins. In other words, proteins transition towards thermodynamically
favorable states.

  3. 3.

The transitions from high energy state/basin to a low energy state/basin
requires overcoming a substantial free energy barrier separating the basins,
which is enabled by the random thermal energy fluctuations. Hence, transitions
are probabilistic.

  4. 4.

Proteins can probabilistically transition via multiple pathways. However, some
transitions are more likely than others depending on the thermodynamic
favorability, barrier height, and available thermal energy fluctuation.

  5. 5.

In addition to thermal energy fluctuation, transitions can also be enabled
modifying the landscape to lower the transition barrier.

Although our model systems (Li et al., 2015, 2019) of beam traversal and self-
righting are macroscopic, self-propelled, and far-from-equilibrium, their
locomotor transitions share several similarities such as diverse locomotor
modes, multi-pathway transitions between locomotor modes that occur
probabilistically, and preference of some modes over others Inspired by the
seeming similarities of our system to them, we contend that the potential
energy landscape approach helps understand how self-propelled, far-from-
equilibrium macroscopic animals’ and robots’ probabilistic locomotor
transitions during traversal of flexible beam obstacles and self-righting on
flat ground emerge from physical interaction, whose equations of motion are
unknown or intractable (Aguilar et al., 2016; Han et al., 2021).

####  1.5.2 Hypotheses

Having reasoned about the validity of using potential energy landscape
approach, we present the hypotheses that we seek to resolve in this
dissertation:

  1. 1.

Are locomotor transitions of beam traversal and self-righting system barrier-
crossing transitions on evolving potential energy landscapes?

  2. 2.

When it is comparable to the potential energy barriers between basins, can the
kinetic energy fluctuation observed during beam traversal and self-righting
help escape from a basin to make locomotor transitions for traversal and self-
righting?

  3. 3.

When kinetic energy fluctuation is not sufficient to escape barriers along
certain direction, is it possible to alter the landscape to lower the barriers
to be comparable to available kinetic energy fluctuation and induce
transitions?

  4. 4.

Analogous to thermodynamic favorability of protein states, do locomotor modes
have terradynamic favorability? If so, is the locomotor-terrain system more
likely to transition to a terradynamically favorable modes.

  5. 5.

How can we begin to quantify physical interaction and transitions at larger
spatiotemporal scales?

These hypotheses have only been speculated and not tested in previous studies
of beam traversal and self-righting.

####  1.5.3 Drawbacks of previous potential energy landscapes

Previous studies of beam obstacle traversal (Li et al., 2015) and self-
righting (Li et al., 2019) used a simple physics model to obtain the system
potential energy and visualize the potential energy landscape. To calculate
system potential energy, following approximations were used to obtain simple
physics models. In both model systems, the animal was approximated as a rigid
ellipsoid with uniform density while ignoring legs and wings. The lowest point
on the rigid ellipsoid was always assumed to be in contact with ground. In
addition, the beams were approximated as rigid plates attached to the ground
through torsional spring joints (Figure 1.10). For both systems, friction and
other non-conservative effects were not considered. System potential energy
for self-righting system was the gravitational potential energy of the body
(Figure 1.11), whereas that for beam traversal was the sum of body
gravitational potential energy and beam deflection elastic energy. While these
landscapes provided initial qualitative explanations, they had a few
shortcomings.

In both these simplified potential energy landscape models (Figures 1.10,
1.11), the systems potential energy depends on more than two parameters. The
previous beam traversal landscape was a minimal potential energy landscape; Li
et al. (2015) simply calculated the minimal potential energy over all
rotational degrees of freedom for positions in the vicinity of the beam
obstacles. Similarly in self-righting landscape, the animal’s wing opening,
which substantially changes potential energy, are not considered. Finally,
resolving these hypotheses will provide quantitative observations of how
system state behaves on potential energy landscapes during physical
interaction, which have been only hypothesized in previous studies.

Figure 1.10: Minimal potential energy landscape of flexible beam traversal.(A)
Simplified physics model to calculate system potential energy. (B) Potential
energy landscape in the space of forward ($X$) and lateral ($Y$) body center
of mass position. Colored arrows indicate hypothesized systems state
trajectories on landscape. Roll mode (red arrow) overcomes lower barrier on
landscape during traversal compared to climb mode (green arrow). Deflect mode
(black arrow) does not result in traversal. Reproduced from (Li et al., 2015).
Figure 1.11: Static potential energy landscape of ground self-righting. (A-C)
An ellipsoid approximating the animal body in contact with the ground,
pitching (A), rolling (B) or rotating diagonally (simultaneous pitching and
rolling; (D), Potential energy landscape, shown as the center of mass height
in the space of body pitch and roll. Colored arrows indicate hypothesized
system state trajectories for body rotations of modes shown in (A)-(C)
Reproduced from (Li et al., 2019)

###  1.6 Organization of Chapters

  * •

Chapter 2 details the biological, robotic, and physics studies of physical
interaction during beam traversal.

  * •

Chapter 3 details the biological, robotic, and physics studies of physical
interaction during self-righting on flat ground.

  * •

Chapter 4 discusses methods to track and analyze animal kinematics data
collected during movement over large spatiotemporal scales using a previously
built terrain treadmill.

  * •

Chapter 5 summarize the discoveries from Chapters 2-4 and their implications.

##  Chapter 2 Kinetic energy fluctuation from oscillatory self-propulsion
facilitates barrier-crossing locomotor transitions during beam traversal

††footnotetext: This chapter is a published paper by Ratan Othayoth, George
Thoms, and Chen Li in The Proceedings of the National Academy of Sciences
(2020) (Othayoth et al. (2020))

###  2.1 Summary

Effective locomotion in nature happens by transitioning across multiple modes
(e.g., walk, run, climb). Despite this, far more mechanistic understanding of
terrestrial locomotion has been on how to generate and stabilize around near-
steady-state movement in a single mode. We still know little about how
locomotor transitions emerge from physical interaction with complex terrain.
Consequently, robots largely rely on geometric maps to avoid obstacles, not
traverse them. Recent studies revealed that locomotor transitions in complex
3-D terrain occur probabilistically via multiple pathways. Here, we show that
an energy landscape approach elucidates the underlying physical principles. We
discovered that locomotor transitions of animals and robots self-propelled
through complex 3-D terrain correspond to barrier-crossing transitions on a
potential energy landscape. Locomotor modes are attracted to landscape basins
separated by potential energy barriers. Kinetic energy fluctuation from
oscillatory self-propulsion helps the system stochastically escape from one
basin and reach another to make transitions. Escape is more likely towards
lower barrier direction. These principles are surprisingly similar to those of
near-equilibrium, microscopic systems. Analogous to free energy landscapes for
multi-pathway protein folding transitions, our energy landscape approach from
first principles is the beginning of a statistical physics theory of multi-
pathway locomotor transitions in complex terrain. This will not only help
understand how the organization of animal behavior emerges from multi-scale
interactions between their neural and mechanical systems and the physical
environment, but also guide robot design, control, and planning over the
large, intractable locomotor-terrain parameter space to generate robust
locomotor transitions through the real world.

###  2.2 Author contributions

Ratan Othayoth designed study, developed robotic physical model, performed
animal and robot experiments, analyzed data, developed energy landscape model,
drafted and revised the paper; George Thoms developed robotic physical model
and performed preliminary robot experiments; Chen Li designed and oversaw
study, defined analyses, and wrote and revised the paper.

###  2.3 Introduction

To move about in the environment, animals can use many modes of locomotion
(e.g., walk, run, crawl, climb, fly, swim, jump, burrow) (Alexander, 2006;
Biewener, 2003; Dickinson et al., 2000) and must often transition across them
(Lock et al., 2013; Low et al., 2015) (e.g., Figure 1.2A). Despite this, far
more of our mechanistic understanding of terrestrial locomotion has been on
how animals generate (Blickhan and Full, 1993; Goldman et al., 2006; Hu et
al., 2009; Kuo, 2007; Li et al., 2012) and stabilize (Biewener and Daley,
2007; Couzin-Fuchs et al., 2015; Revzen et al., 2013) steady-state, limit-
cycle-like locomotion using a single mode.

Recent studies begin to reveal how terrestrial animals transition across
locomotor modes in complex environments. Locomotor transitions, like other
animal behavior, emerge from multi-scale interactions of the animal and
external environment across the neural, postural, navigational, and ecological
levels (Berman, 2018; Brown and Bivort, 2018; Nathan et al., 2008). At the
neural level, terrestrial animals can use central pattern generators
(Ijspeert, 2008) and sensory information (Blaesing, 2004; Kohlsdorf and
Biewener, 2006; Ritzmann et al., 2012) to switch locomotor modes to traverse
different media or overcome obstacles. At the ecological level, terrestrial
animals foraging across natural landscapes switch locomotor modes to minimize
metabolic cost (Shepard et al., 2013). At the intermediate level, terrestrial
animals also transition between walking and running to save energy (Bramble
and Lieberman, 2004). However, there remains a knowledge gap in how locomotor
transitions in complex terrain emerge from direct physical interaction (i.e.,
terradynamics (Li et al., 2013)) of an animal’s body and appendages with the
environment. In particular, we lack theoretical concepts for thinking about
how to generate and control locomotor transitions in complex terrain that are
on the same level of limit cycles for single-mode locomotion (25). For
example, locomotion in irregular terrain with repeated perturbations is rarely
near steady state and requires an animal to continually modify its behavior,
which cannot be well described by limit cycles (Spagna et al., 2007; Sponberg
and Full, 2008).

Understanding of how to make use of physical interaction with complex terrain
(environmental affordance (Gibson, 2014; Roberts et al., 2020)) to generate
and control locomotor transitions is also critical to advancing mobile
robotics. Similar to personal computers decades ago, mobile robots are on the
verge of becoming a part of society. Some robots (e.g., robot vacuums, self-
driving cars) already excel at navigating flat surfaces, by transitioning
across driving modes (e.g., forward drive, U-turn, stop, park (Thrun, 2010))
to avoid sparse obstacles using a geometric map of the environment (Latombe,
2012). However, many critical applications, such as search and rescue in
rubble, inspection and monitoring in buildings, extraterrestrial exploration
through rocks, and even drug delivery inside a human body, require robots to
transition across diverse locomotor modes to traverse unavoidable obstacles in
complex terrain (Hu et al., 2018; Lock et al., 2013; Low et al., 2015) (Figure
1.2B)). Yet, terrestrial robots still struggle to do so robustly (Guizzo and
Ackerman, 2015), because we do not understand well how locomotor transitions
(or lack thereof) emerge from physical interaction with complex terrain.

Figure 2.1: A cockroach transitioning (orange arrow) from pitch to roll mode
to traverse grass-like beam obstacles.

Our study is motivated by recent observations in a model system of insects
traversing complex 3-D terrain. The discoid cockroach, native to rainforest
floor, can traverse flexible, grass-like beam obstacles using many locomotor
modes, stochastically transitioning across them via multiple pathways (Li et
al., 2015). For simplicity, hereafter we focus on the transition between two
modes. The animal often first pushes against the beams, and beam elastic
restoring forces lead the animal body to pitch up (Figure 2.1), blue). After
this, though, the animal rarely pushes across (3% probability) but often rolls
(Figure 2.1), red) to maneuver through beam gaps (45% probability). We define
these as “pitch” and “roll” modes. Note that we use “locomotor mode” here in
the general sense, not confined to limit-cycle locomotor behavior. The pitch
mode is more challenging than the roll mode because the animal has to lift its
weight and deflect the beams more (this is only true when beams are stiff,
though; see Results). Thus, the animal appears to statistically transition
from less to more favorable modes. In addition, the animal’s body oscillates
as its legs continually pushed against the ground when trying to traverse.
Besides in obstacle traversal, similar multi-pathway locomotor transitions,
preference of some modes over others, and seemingly wasteful body oscillation
were observed in self-righting of insects (Li et al., 2019).

In the field of protein folding, adopting a statistical physics view and using
an energy landscape approach led researchers to recognize that proteins fold
via multiple pathways and understand the physical principles (Dill et al.,
2008; Onuchic and Wolynes, 2004; Wales, 2003). These near-equilibrium,
microscopic systems statistically transition from higher to lower energy
states (local minima) on a free energy landscape (increasing thermodynamic
favorability). Thermal fluctuation helps the system stochastically cross
energy barriers at transition states (saddle points between local minimum
basins). These physical principles operating on a rugged landscape leads to
the multi-pathway protein folding transitions. Inspired by the seeming
similarities of our system to them, we contend that an energy landscape
approach helps understand how self-propelled, far-from-equilibrium macroscopic
animals’ and robots’ probabilistic locomotor transitions in complex 3-D
terrain emerge from physical interaction, whose equations of motion are
unknown or intractable (Aguilar and Goldman, 2016; Han et al., 2021).
Specifically, we hypothesize that:

  1. 1.

The self-propelled system’s state is attracted to a local minimum basin on a
potential energy landscape; locomotor transition from one mode to another can
be viewed as the system state escaping from one basin and settling into
another. (What governs transition?)

  2. 2.

When it is comparable to the potential barrier, kinetic energy fluctuation
from oscillatory self-propulsion helps the system escape from a landscape
basin to make locomotor transitions. (When does transition happen?)

  3. 3.

Escape from a basin is more likely towards a direction along which the escape
barrier is lower. (How does transition happen?)

To begin to establish an energy landscape approach of locomotor transitions
across modes in complex 3-D terrain, we tested these hypotheses for the two
representative modes (pitch and roll) of the model body-beam interaction
system defined above. Although the previous study introduced an early energy
landscape model to qualitatively explain why locomotor shape affected physical
interaction and thus locomotion (Li et al., 2015), none of these hypotheses
were proposed or tested. We emphasize that our potential energy landscape
directly arises from locomotor-terrain interaction physics using first
principles. This is unlike artificially defined potential functions to explain
walk-to-run transition (Diedrich and Warren, 1995) and other non-equilibrium
biological phase transitions (Kelso, 2012), or metabolic energy landscapes
inferred from oxygen consumption measurements to explain behavioral switching
of locomotor modes (Shepard et al., 2013).

Because animal locomotion emerges from complex interactions of neural and
physical mechanisms (Dickinson et al., 2000), to observe the outcome of pure
physical interaction, we developed and tested a minimalistic robotic physical
model (Figure 2.2)) with feedforward control. The robot had an ellipsoid-like
body that was propelled forward at a constant speed and was free to pitch and
roll (achieved through a gyroscope mechanism) in response to interaction with
two beams. The body was constrained not to yaw or move laterally to simplify
energy landscape modeling. We also performed experiments with the discoid
cockroach traversing beams during escape response to study how physical
interaction affects the animal’s locomotor transitions when neural control is
bandwidth limited (Dickinson et al., 2000). Comparison of robot and animal
observations can reveal aspects of the transitions that likely involve neural
mechanisms.

Figure 2.2: Robotic physical model.

To test the first hypothesis, in both robot and animal experiments, we used
rigid “beams” with torsional joints at the base (Figures 2.3,2.4,2.5,2.6) as
one-degree-of-freedom 3-D terrain components to generate a simple potential
energy landscape. We then reconstructed the potential energy landscape and 3-D
motion of the robot or animal body and beams in high accuracy (as opposed to
visual examination in the previous study (Li et al., 2015)) (Figures 2.14,
2.15) for the entire traversal. This allowed us to quantify how the system
state behaved on the landscape during each observed locomotor mode and
transition between modes. To test the second hypothesis, for the robot, we
applied controlled oscillation with variable frequency f to vary kinetic
energy fluctuation (Figure 2.10). Because we could not vary the animal’s
naturally occurring body oscillation, in animal experiments we changed the
barrier relative to kinetic energy oscillation by varying beam torsional joint
stiffness K by over an order of magnitude in the range of natural flexible
terrain elements (Table S2). K was also varied by over an order of magnitude
for robot experiments and, together with animal experiments, helped elucidate
how transition depended on terrain properties. Because the potential energy
landscape consists of not only beam elastic energy but also body and beam
gravitational energy, variation of K also changed how escape barrier compared
in different directions, allowing the third hypothesis to be tested. See
Methods and Supplementary Methods for technical detail and Table 2.1 for
sample sizes.

###  2.4 Methods

####  2.4.1 Robotic physical model

To approximate the body shape of the discoid cockroach (Li et al., 2015), we
3-D printed an ellipsoid-like body, PLA plastic using UPBOX+, Tiertime, CA,
USA), whose top and bottom halves were slices of an ellipsoid. The body was
suspended (center of mass at 10 cm above the ground) via a custom gyroscope
mechanism that allowed free body pitching and rolling (Figure 2.3A). We added
mass to the body so that it is bottom heavy, with body center of mass at 1.1
cm below the pitch axis and 1.6 cm below the roll axis. Body pitch and roll at
static equilibrium for a freely suspended body without beam contact were near
zero (pitch = 3.3$\degree$ $\pm$ 0.4$\degree$, roll = 1.7$\degree$ $\pm$
0.8$\degree$; note that positive pitch is pitching downward). See Table 2.1
for geometric dimensions and physical properties of the body.

Figure 2.3: Design of robotic physical model and rigid beams with torsional
springs at base. (A) Photo of robot body and beams. Body is propelled forward
at a constant speed (white arrow) and can be oscillated vertically (yellow
arrows). Body can freely pitch and roll in response to interaction with beams.
(B) CAD model of body, showing design of pitch and roll joints and axes. Body
center of mass is below geometric center due to added weight. Pitch (blue) and
roll (red) axes cross geometric center. (C) CAD model of beam base, showing
design of torsional joint. Rigid beams rotate about an axis parallel to y-axis
(yellow arrow). K is varied by using different combinations of soft (red) and
stiff (cyan) springs.

We used a linear actuator (Firgelli FA-HF-100-12-12, Firgelli Automation, WA,
USA) to propel the body forward towards the obstacles. To introduce body
kinetic energy fluctuation, we oscillated the body vertically using two DC
servo motors (XM430-W350T, Dynamixel, CA, USA) via a five-bar linkage
mechanism 3-D printed from PLA plastic (UPBOX+, Tiertime, CA, USA). We varied
kinetic energy fluctuation by varying oscillation frequency. Our preliminary
experiments showed that body oscillation along different directions did not
qualitatively affect the outcome. Thus, we chose vertical oscillation to
better observe response in body pitch and roll.

The body oscillated vertically along the following triangular wave trajectory
(fitted from the measured $z$ position):

$\displaystyle z$ $\displaystyle=z_{0}+Aft+N(\mu,\sigma),$ $\displaystyle
0\leq t\leq\frac{T}{2}$ (2.1) $\displaystyle z$
$\displaystyle=z_{0}+A(1-ft)+N(\mu,\sigma),$ $\displaystyle\frac{T}{2}<t\leq
T$ (2.2)

where $z$ is the vertical position of the body geometric center, $f$ is
vertical oscillation frequency, $T=1/f$ is vertical oscillation period, $A$ =
23.4 mm is the vertical oscillation amplitude, and $z_{0}$ = 102.4 mm is the
average vertical position when there is no oscillation. To prevent the body
from being stuck against beams due to friction, we added a small noise, $N$,
which is normally distributed with a mean of $\mu$ = 0.7 mm and a standard
deviation of $\sigma$ = 1.2 mm. Kinetic energy fluctuation from this noise was
small compared to that from the vertical oscillation. The vertical oscillation
induced small lateral oscillation (12% of vertical oscillation amplitude). The
motor angles were commanded using a microcontroller (Open CM 0.94, Robotis,
CA, USA). We note that the animal’s body oscillation is much more complex,
variable, and less periodic than the robot’s. It was difficult to use a wave
oscillation with well-defined amplitude and frequency to approximate it.

Table 2.1: Geometric dimensions, physical properties, and sample sizes for animal and robot experiments. |  | Animal | Robot
---|---|---|---
Number of Individuals | 6 | N/A
Body | Mass $m_{\text{body}}$ (g) | 2.6 $\pm$ 0.3 | 233
Length (cm) | 5.3 $\pm$ 0.1 | 22.1
Width (cm) | 2.4 $\pm$ 0.1 | 15.8
Thickness (cm) | 0.8 $\pm$ 0.1 | 5.8
Beam | Lateral spacing(cm) | 1.0 | 12.7
Width (cm) | 1.0 | 2.8
Mass $m_{\text{beam}}$ (g) |  | 0.33 | 0.42 | 0.63 | 0.70 | 1.03 |  | 38
Inner layer thickness (mm) | 0.04 | 0.05 | 0.07 | 0.10 | 0.25 | N/A
Total thickness (mm) | 0.54 | 0.55 | 0.72 | 0.75 | 0.85 | 6
Length $L$ (cm) | 5.7 | 8.8 | 8.7 | 8.6 | 9.3 | 18
Torsional stiffness K (mN$\cdot$m/rad) | 0.1 | 0.2 | 0.7 | 1.7 | 11.4 | 28 | 55 | 255 | 344
Sample size | No. of trials | Ind. 1 | 11 | 10 | 9 | 11 | 10 | 0 Hz | 10 | 10 | 10 | 10
Ind. 2 | 10 | 10 | 10 | 7 | 10 | 1 Hz | 10 | 10 | 10 | 10
Ind. 3 | 11 | 10 | 10 | 11 | 11 | 2 Hz | 10 | 10 | 10 | 10
Ind. 4 | 13 | 10 | 11 | 11 | 13 | 3 Hz | 10 | 10 | 10 | 10
Ind. 5 | 10 | 10 | 10 | 12 | 10 | 4 Hz | 10 | 10 | 10 | 10
Ind. 6 | 9 | 10 | 10 | 10 | 10 | 5 Hz | 10 | 10 | 10 | 10
Total | 64 | 60 | 60 | 62 | 64 | 6 Hz | 10 | 10 | 10 | 10
Total no. of trials | 310 | 280

All data averages are mean $\pm$ s.d. $m_{\text{beam}}$ is the mass of one
beam.

####  2.4.2 Robot beam obstacles

For robot experiments, we mounted two rigid beams to a fixed base (Figure
2.3A) vertically using 3-D printed torsional spring joints (Figure 2.3). We
varied K by using different combinations of soft and stiff torsional springs
(McMaster Carr, NJ) (Figure 2.3C, red and cyan) in parallel. The rigid beams
were laser cut from acrylic plates (VLS60, Universal Laser & McMaster-Carr,
NJ, USA). We covered the beam edges using smooth plastic straw (6 mm diameter)
to reduce friction between them and the body during interaction.

Figure 2.4: Robot beam stiffness characterization. Beam restoring torque as a
function of bending angle $\Delta\theta$(defined in Figure 2.3C). Red and cyan
curves are data for soft and stiff spring. Dashed lines are linear fits
(through the origin) of data of each K, whose slope give K. (B) K for
different combinations of springs used (mean $\pm$ s.d., n = 3 springs, 3
loading cycles each).

We characterized torsional stiffness of the stiff and soft torsional springs
by measuring the restoring torque about the torsional joint as a function of
joint deflection angle (Figure 2.4A) using a 3-axis force sensor (Optoforce
OMD-20-FG, OnRobot, Denmark). Torsional stiffness was calculated from the
slope of the linear fit (across the origin) of torque as a function of
deflection angle (Figure 2.4B). By combining the stiff and soft torsional
springs, we varied K by over an order of magnitude ([28, 55, 255, 344]
mN$\cdot$m/rad). See Table 2.1 for geometric dimensions and physical
properties of the beams.

####  2.4.3 Robot experiment imaging

Robot experiments were recorded using three synchronized high-speed cameras
(IL5, Fastec Imaging, San Diego, CA) at 200 frames s-1 and a resolution of
1920 $\times$ 1080 pixels. To automatically track the body and beams over the
entire range of rotation, we attached BEEtags (Crall et al., 2015) (18 mm
$\times$ 18 mm) on the body (9 markers), vertical oscillation transmission (3
markers), right beam (2 markers), and left beam (5 markers). We used FasMotion
software (Fastec Imaging, San Diego, CA) to save the videos to storage drives
after recording for tracking and processing.

####  2.4.4 Robot experiment protocol

Before each trial, the body was positioned at a distance of 11 cm from the
beams, and the beams were set to be vertical. We started video recording and
body oscillation (for $f$ > 0), waited for 1 s, and then propelled the body
forward at a constant speed of 0.7 cm$\cdot$s-1 by a distance of 30 cm
(maximum possible by the linear actuator). Body oscillation was applied (for
$f$ > 0) until the end of forward translation. After forward translation
completed, we stopped body oscillation and video recording and moved the body
to its initial position for the next trial.

At each K, we varied kinetic energy fluctuation by varying $f$ from 0 Hz to 6
Hz with an increment of 1 Hz. At each K and each $f$, we performed 10 trials.
This resulted in a total of 280 trials, with 70 trials at each K across all
$f$. See Table 2.1 for detailed sample size.

####  2.4.5 Animals

We chose to study the discoid cockroach, Blaberus discoidalis, because it
dwells on the floor of tropical rainforests with dense vegetation and litter
and excels at traversing complex terrain (Li et al., 2015). We used adult male
discoid cockroaches (Pinellas County Reptiles, St Petersburg, FL, USA), as
females are often gravid and under different load bearing conditions. Prior to
experiments, we kept the cockroaches in individual plastic containers at room
temperature (24 $\degree$C) on a 12h:12h light:dark cycle. See Table 2.1 for
dimensions and mass of the animals tested.

####  2.4.6 Animal beam obstacles

We custom made rigid “beams” with torsional springs at the base (Figure 2.5A).
For each beam, we sandwiched a flexible layer between two stiff layers and
exposed a small portion of the flexible layer (Figure 2.5B), which acted as
torsional spring joint about which the beams deflect in the $x-z$ plane. We
varied the thickness of the flexible layer ([0.04, 0.05, 0.07, 0.10, 0.25] mm)
to vary the torsional stiffness K of the torsional joint by over two orders of
magnitude ([0.1, 0.2, 0.7, 1.7, 11.4] mN$\cdot$m/rad) in a similar range as
natural obstacles like leaves, stalks, and grass. Polyethylene terephthalate
plastic (McMaster Carr, NJ, USA) and cardstock (0.2 mm thickness, Neenah Inc.,
GA, USA) were used for the flexible and stiff layers and bonded using
thermally bonding glue (Therm-O-Web, IL, USA) and a laminating machine
(AmazonBasics, Amazon). The layer of 10 beams was laser cut (VLS60, Universal
Laser Systems, AZ, USA) to have identical geometry and spacing.

Figure 2.5: Design of rigid beams with torsional springs at base for animal
experiments.(A) Photo of a layer of animal beams. Inset shows a closer view of
torsional joint. (B) Side view schematic of beam design following (Haldane,
2015). Stiff outer layers (black) provide rigidity, and a small exposed
section of flexible inner layer (blue) acts as a torsional spring joint.
Dimensions not true to scale.

We characterized K by measuring the restoring torque about the torsional joint
as a function of beam deflection angle (Figure 2.6A) using a 6-axis force and
torque sensor (Nano 43, ATI Industrial Automation, NC, USA). K was calculated
from the slope of the linear fit (across the origin) of torque as a function
of deflection angle (Figure 2.6B). See Table 2.1 for geometric dimensions and
physical properties of the beams.

Figure 2.6: Characterization of rigid beams with torsional springs at base for
animal experiments. (A) Beam restoring torque as a function of bending angle
for different flexible layer thickness ([0.04, 0.05 0.07, 0.10, 0.25] mm).
Dashed lines are linear fits (through the origin) of data, whose slopes give
K. (B) K as a function of flexible layer thickness (mean $\pm$ s.d., n = 62,
37, 76, 38, 32 loading cycles).

####  2.4.7 Animal multi-camera imaging arena

We constructed an arena for animal experiments to measure locomotor
transitions (Figure 2.7). Previous studies showed that animals often laterally
explored beam obstacles before traversing (Li et al., 2015). To increase
experimental yield, we used 10 identical beams in an obstacle layer, which
presented nine gaps of 1 cm (narrower than animal body width of 2.4 cm, but
larger than body thickness of 0.8 cm) for the animal to traverse. All the
beams were vertical without external force from the animal. The beam obstacle
layer was inserted into a slit cut in the flat ground between two transparent
sidewalls made of acrylic sheets. A runway funneled the animal towards the
middle of beam obstacle layer to minimize the interaction with the sidewall.
To facilitate traversal with minimal body yaw (on average), we arranged the
beam obstacle layer to be perpendicular to the direction of animal movement.
The reduced body yaw allowed us to more accurately visualize how trials
evolved on the potential energy landscape (see section below), which was
calculated using the average body yaw from all trials. Paper cardstock covered
the ground surface. We placed a dark shelter with food and water on the exit
side of the obstacle layer for the animal to rest after each trial.

Figure 2.7: Animal locomotion arena with a layer of beam obstacles (green),
with seven high-speed cameras. $x$, $y$,$z$ axes show lab frame.

Animal experiments were recorded using seven synchronized high-speed cameras
(N5A-100, Adimec, Netherlands) at 100 frames s-1 and a resolution of 2592
$\times$ 2048 pixels. When interacting with the obstacles, animal body
orientation varied substantially. We carefully positioned the cameras around
the entire arena to cover the entire rotation range of motion, with two from
back views, two side views, two isometric views, and one top view (Figure
2.7A). We used the StreamPix software (Norpix Inc., Montreal, Canada) to
automatically save the videos to storage drives as they were being recorded,
after which they were converted to AVI format for tracking and processing.

Figure 2.8: Snapshot of animal traversing beam obstacles (view from the shaded
camera in Figure 2.7). Markers are attached to the animal body and beams to
track their 3-D motion (yellow and magenta trajectories). Red, green, blue
axes show body frame attached to markers.

To automatically track the animal and beams, we attached a 7 mm $\times$ 7 mm
BEEtag (Crall et al., 2015) to the animal body and 9 mm $\times$ 9 mm BEEtags
to the top and bottom ends of both sides of each beam (Figure 2.8). The animal
BEEtag was much lighter (< 0.15 g) than the animal itself (2.6 g). It was
printed onto a rounded oval cardboard to minimize interference with the
obstacle traversal and attached to the dorsal surface of the abdomen using
ultraviolet curing glue (Bondic, Aurora, Canada).

####  2.4.8 Animal experiment protocol

Before the experiment, the arena was illuminated and heated to about
43$\degree$C with six work lamps (Coleman Cable, Waukegan, IL, USA). Before
each trial, the animal was placed in the starting end of the arena and allowed
to settle down. We then started video recording and probed the animal with a
stick with a soft tip (made from paper tapes) to induce it to run towards the
obstacles. The animal did not always immediately traverse after running into
beam obstacles. Instead, it often made multiple failed attempts to traverse
and sometimes explored the obstacle layer laterally to attempt traversing at
different beam gaps, before eventually traversing. Once the animal traversed
and reached the shelter, we stopped video recording and allowed the animal to
rest for $\sim$10 minutes before the next trial.

We tested six animal individuals and beams of five different torsional
stiffness K and collected a total of 337 trials. The same six individuals were
tested across all K. We discarded trials in which any of the following were
observed: (1) the animal did not move within 10 s after it was probed; (2) the
animal moved back to the starting area or did not attempt to traverse; (3) the
animal used the sidewall to traverse; or (4) the animal climbed up the beams
and its body and all six legs lost contact with the ground. This resulted in a
total of 310 accepted trials, with approximately 10 trials for each animal at
each K. See Table 2.1 for detailed sample size.

####  2.4.9 High accuracy 3-D motion reconstruction

To calibrate the cameras over the working space for 3-D motion reconstruction,
for both robot and animal experiments, we built a calibration object with
multiple markers (47 for robot and 17 for animal) using Lego bricks (The Lego
Group, Denmark). We then used the direct linear transformation software
DLTcal5 (Hedrick, 2008) to obtain intrinsic and extrinsic camera parameters.
We used a custom MATLAB script to automatically track 2-D coordinates of the
markers in each camera view using the BEEtag code (Crall et al., 2015).

Using the tracked 2-D marker coordinates from multiple camera views and camera
calibration parameters, we obtained the 3-D position of the four corners of
each BEEtag markers using the direct linear transformation software DLTdv5
(Hedrick, 2008) , which was then used to obtain the marker frame (Figure 2.8).
For the animal, we translated and rotated the marker frame by the measured
translational ($\Delta x$ = 10 mm, $\Delta y$ = $-$0.2 mm, $\Delta z$ = $-$3
mm) and rotational (roll = 0$\degree$, pitch = 10$\degree$, yaw = 1$\degree$)
offsets to obtain 3-D position and orientation of the body frame at the body
geometric center, which nearly overlapped with body center of mass (Kram et
al., 1997). For the robot, we used a CAD model of the body to determine the
location of center of mass relative to the markers fixed to the body.
Depending on which body markers were reconstructed in each video frame, we
translated and rotated the reconstructed marker frame by its measured
translational and rotational offsets to obtain 3-D position and orientation of
the body frame at the center of mass. For both the robot and animal, we used
Euler angles (yaw $\alpha$, pitch $\beta$, and roll $\gamma$, Z$-$Y’$-$X”
Tait-Bryan convention) to define 3-D rotation. Note that with this convention,
when the body pitches upward, pitch angle is negative.

To quantify the accuracy of 3-D reconstruction using BEEtag tracking combined
with Direct Linear Transformation, we 3-D printed a high-precision calibration
object. The calibration object had nine BEEtag markers mounted on a horizontal
plate in a 3 $\times$ 3 grid with a 7 cm grid distance, each oriented at a
pitch and yaw angle of 0$\degree$, 30$\degree$, and 60$\degree$. We measured
the 3-D position and orientation of each marker from 3-D reconstruction
(described above) and compared them to the designed values. This demonstrated
that our imaging setup achieved high accuracy in 3-D position and orientation
reconstruction (s.d. of position error = 0.6 mm; s.d. of orientation error =
1.1$\degree$). We also verified that lens distortion was minimal (< 1%) using
the checkboard distortion measurement method.

For each trial, we calculated body translational ($v_{x}$, $v_{y}$, $v_{z}$)
and rotational ($\omega_{\alpha}$, $\omega_{\beta}$, $\omega_{\gamma}$)
velocities and beam deflection angles from vertical ($\Delta\theta_{i}$) as a
function of time. Beam angle was averaged from visible tags on each beam.
Considering lateral symmetry, to simplify analysis of the roll mode, we
flipped all trials in which the body rolled left to rolling right. For the
animal, we offset the measured lateral positions (y) of each trial so that y =
0 in the middle of the gap that the animal traversed during the final,
successful attempt.

####  2.4.10 Definition of pitch and roll modes and pitch-to-roll transition

We defined the robot to be in the roll mode if both beams lost contact with
the body and bounced back to vertical before the distal end of the body
crossed the beams ($x$ = 0), and we defined it to be in the pitch mode
otherwise. For the robot, body motion was highly repeatable from trial to
trial, and pitch-to-roll transition always resulted in a sharp decrease in
system potential energy. Thus, we defined transition to occur when system
potential energy reached a peak value (Figure 2.9E, vertical dashed line
($ii$)), after which it immediately reduced.

We defined the animal to be in the roll mode if its body roll (absolute value)
exceeded 62$\degree$, because from system geometry this was the minimal roll
for the body to move through the gap between two adjacent beams without
deflecting them. The animal was defined to be in pitch mode otherwise. For
trials in which the animal transitioned from the pitch to roll mode, we
defined transitions to occur when body roll (absolute value) exceeded
20$\degree$ (Figure 2.9B, vertical dashed line ($ii$)). We verified that
system potential energy (Figure 2.9F) decreased at this moment.

Figure 2.9: Representative motion of body and beams and system potential
energy during interaction and definition of traversal and pitch-to-roll
transition. (A, B) Body roll (red) and pitch (blue) as a function of time. (C,
D) Left (red) and right (blue) beam deflection angle as a function of time.
(E, F) System potential energy as a function of time. Data shown for a
representative pitch-to-roll transition at K = 255 mN$\cdot$m/rad for robot
and K = 0.7 mN$\cdot$m/rad for animal. For both the robot and animal, pitch-
to-roll transition resulted in a reduction in system potential energy. Note
that negative pitch is the body pitching head-up. Dashed lines (i) and (ii)
are when body first contacts beams and when pitch-to-roll transition occurs.
Dashed line (iii) is when the robot’s forward translation ends and when
animal’s distal end crosses the beam (x = 0).

####  2.4.11 Data averaging

Because the robot was propelled forward at a constant speed, its 3-D
kinematics, potential energy, and kinetic energy were a function of body
forward position $x$. To obtain average 3-D kinematics and potential energy as
a function of $x$, we interpolated the measured position, orientation, and
potential energy over $x$ and then averaged them across all trials at a given
K. For the robot, we averaged lateral position $y$ and body yaw $\alpha$ for
all the trials at each K for each $x$ and used this average trajectory of
measured $x$, $y$, and $\alpha$ to calculate an average potential energy
landscape. For the animal, because of the high variability in $y$ and
$\alpha$, for simplicity we set both to zero when calculating the average
potential energy landscape at each $x$.

Because we focused on the pitch-to-roll transition (see definition in the next
section), we considered only the animal’s final, successful attempt in which
such a transition may occur. For the final, successful attempt, we analyzed
the portion of the trial starting from five frames (0.05 s) before the
animal’s head contacted the beams (Figure 2.9, dashed vertical line ($i$)) to
ten frames (0.1 s) after the entire body crossed the obstacle layer (at $x$ =
0, Figure 2.9, dashed vertical line ($iii$)). Because the robot body was
translated with a constant forward speed and always crossed the beams, for it
we analyzed the portion of the trial starting from when the body first
contacted the beams (Figure 2.9, dashed vertical line ($i$)) until the end of
forward translation (Figure 2.9, dashed vertical line ($iii$)).

####  2.4.12 Kinetic energy fluctuation

For both the robot and animal, we defined body kinetic energy fluctuation as
the sum of kinetic energy due to translational and rotational velocity
components other than forward motion of the body ($v_{y}$, $v_{z}$,
$\omega_{\alpha}$, $\omega_{\beta}$, $\omega_{\gamma}$). To calculate moment
of inertia, we approximated the animal body as an ellipsoid with uniform mass
distribution, considering that legs only consist less than 15% of total mass
(Kram et al., 1997). For the robot, we calculated moment of inertia from a CAD
model of the body with accurate geometry and mass distribution.

Figure 2.10: Kinetic energy fluctuation. (A) Kinetic energy fluctuation of
robot as function of f. *** indicates a significant dependence (ANOVA, P <
0.0001, F = 520.99). (B) Kinetic energy fluctuation of animal as function of
K. n.s. indicate no significant difference (ANOVA, P = 0.3835, F = 0.9047).
See Table 2.1 for sample size.

For both the robot and animal, we calculated average kinetic energy
fluctuation from first beam contact (of the final successful attempt of each
trial for the animal) to when transition occurred using the trials in which
the body transitioned to the roll mode. This was because for the trials in
which the body was trapped in the pitch mode, it was difficult to define the
onset of pitching as can be readily done for the onset of rolling. Including
these trials would add the substantial kinetic energy of continuous body
pitching that resulted from the interaction, which was not part of the
fluctuation that induced the transition. We verified that kinetic energy
fluctuation differed little between before contact with the beams and from
first contact to when transition occurred. We then averaged kinetic energy
fluctuation over time for each trial, from when the body first contacted a
beam (Figure 2.9, dashed line ($i$)) (in the final successful attempt of each
trial for the animal), to when it transitioned to roll mode (Figure 2.9,
dashed line ($ii$)). For the robot, we then averaged these trial averages
across all trials at each $f$ in which the robot transitioned to the roll mode
to obtain average kinetic energy fluctuation at each $f$ (Figure 2.10A). For
the animal, we averaged these trial averages across all trials at each K to
obtain average kinetic energy fluctuation at each K (Figure 2.10B).

####  2.4.13 Statistics

All probabilities were calculated relative to the total number of accepted
trials of each treatment. All average data are reported as mean $\pm$ s.d. For
the robot, we used a chi-square test to test whether pitch-to-roll transition
probability depended on K, with K and $f$ as fixed factors. For the animal, we
used a chi-square test to test whether pitch-to-roll transition probability
depended on K, with K and individual as fixed factors and including their
crossed effect. For the robot, we used an ANOVA to test whether kinetic energy
fluctuation increased with $f$. To test whether the animal’s kinetic energy
fluctuation depended on K, we pooled data from all the trials in which pitch-
to-roll transition occurred (see section above for explanation) for each K and
performed a mixed-effect ANOVA with K as a fixed factor and individual as a
random factor. We used a Student’s t-test to test whether the robot’s system
state was attracted to the basin corresponding to the measured mode in all
trials. All statistical tests were performed using JMP Pro 13 (SAS Institute
Inc., NC, USA).

####  2.4.14 Potential energy landscape

In energy landscape modeling, we approximated the animal body as a rigid
ellipsoid and obtained the robot body shape from a CAD model used for 3-D
printing the body. The beams were modeled as rigid rectangular plates on
torsional joints (Figure 2.11A). Because the beams had a finite mass, forward
deflection lowered beam center of mass and thus beam gravitational potential
energy. Because the measured beam restoring torque was nearly proportional to
deflection angle for both the robot (Figure 2.4B) and animal (Figure 2.6A), we
approximated the torsional joint at the base of each beam as a perfect Hookean
torsional spring and assumed that there was no damping. Because the body only
pushed forward against the beams, in the model we only allowed forward beam
deflection ($\Delta\theta_{1,2}\geq 0$).

Figure 2.11: Potential energy landscape model, with definition of variables
and parameters. (A) Oblique view schematic of body (a rigid ellipsoid) and
beams (rigid rectangular plates with torsional joints at base) of torsional
stiffness K. Without body contact, both beams are vertical (light green). With
body contact, beams are deflected forward (dark green) by angles
$\Delta\theta_{1,2}$. (B, C) Side view of model for robot (B) and animal (C)
to show center of mass height changes with body pitching and beam deflection.
Solid and dashed ellipses show body in static equilibrium and pitched-up,
respectively. Center of mass of body and beams are shown.

For robot modeling, we set center of mass to be below the pitch and roll axes
as measured (Figure 2.11B). For animal modeling, we constrained the lowest
point of the body to always touch the ground (ground constraint, Figure
2.11C), because the animal maintained ground contact during traversal (we
rejected trials in which the animal climbed onto the beams) and we neglected
the animal’s legs. Thus, for both the robot and animal, body pitching and
rolling in response to interaction with the beams increased center of mass
height and thus body gravitational potential energy. In addition, because the
robot was suspended from and driven forward by a linear actuator, its center
of mass height was constrained to move within a measured range of $z$ = [9.9
cm, 11.8 cm]. Because the robot’s controlled vertical oscillation was modeled
as part of kinetic energy fluctuation, we used the average body center of mass
vertical position before contacting the beams ($z$ = 10.8 cm, vertical height
constraint) to calculate its initial body potential energy. We verified that
at any given $x$, landscape shape remained similar within the $z$ range in
which the robot was oscillated. For both the robot and animal, we offset
system potential energy to zero when the body was not in contact with beams
and in its static equilibrium (at zero pitch and zero roll) so system
potential energy shown on the landscapes were relative to this initial
equilibrium (2.11B, C).

The full potential energy landscape depended on body orientation (pitch, roll,
yaw) and forward and lateral positions ($x$, $y$), given the vertical height
and ground constraints on the robot and animal, respectively. Because we
focused on body pitch and roll motions, for a given body position ($x$, $y$)
and yaw, we varied body pitch and roll over [$-$180$\degree$, 180$\degree$] to
calculate system potential energy landscape over pitch-roll space. In Figures
2.13B, 2.14, 2.15A, and 2.17A, we only show the landscape over a part of the
entire pitch-roll space to better focus on the pitch and roll basins. We then
calculated beam deflection due to body contact (only allowing
$\Delta\theta_{1,2}\geq 0$) and center of mass height increase ($\Delta z$) to
obtain system potential energy as below.

$E=m_{body}g\Delta
z+\frac{1}{2}m_{beam}gL(\cos\Delta\theta_{1}+\cos\Delta\theta_{2}-2)+\frac{1}{2}K(\Delta\theta^{2}_{1}+\Delta\theta^{2}_{2})$
(2.3)

where where $m_{body}$ is body mass, $g$ is gravitational acceleration,
$\Delta z$ is body center of mass height increase from its equilibrium
configuration (at near zero pitch and zero roll), $m_{beam}$ is beam mass, $L$
is beam length, $K$ is beam torsional stiffness, and $\Delta\theta_{1}$ and
$\Delta\theta_{2}$ are beam deflection angles from vertical.

We note that our landscape did not model body-beam interaction after the beams
bounced back.

####  2.4.15 Local minima and system state trajectories on potential energy
landscape

For each forward position $x$ of the body relative to the beams, we examined
the landscape to determine the pitch and roll local minima and measured their
potential energies. Note that for the robot their potential energies did not
include height change due to controlled vertical oscillation (see section
above). To visualize how the measured state of the system behaved on the
landscape, we projected the measured body pitch and roll onto the landscape
for each $x$ (Figure 2.14A, 2.15, 2.17A, blue and red dots for trials in which
the system was trapped in the pitch mode and transitioned to the roll mode),
which formed a system state trajectory over time as traversal progressed. Note
that only the end points of the trajectory, which represent the current state,
showed the actual potential energy of the system at the corresponding $x$. The
rest of the visualized trajectory showed how body pitch and roll evolved but,
for visualization purpose, was simply projected on the landscape surface.
Because roll local minimum does not exist at K = 28 mN$\cdot$m/rad for the
robot, for comparison with other K, we defined it to be at (pitch, roll) =
(0$\degree$, $\pm$42$\degree$) based on the minimal body roll required to
traverse without beam deflection.

####  2.4.16 Average potential energy landscape at each beam stiffness

To facilitate observation of statistical trends, we calculated the average
potential energy landscape at each K and visualized all trials on it. Average
landscape calculation used the average measured lateral position $y$ and body
yaw for each $x$. For the robot, this average potential energy landscape was a
good approximation of the actual landscape for each trial, because the robot
was constrained by design to have minimal lateral motion or yawing. Despite
this, when projected onto the average potential energy landscape, in some
trials at high K, a portion of the system state trajectory appeared to
momentarily go out of the pitch basin and then re-entered it (Figure 2.15).
This was an artifact from landscape averaging. In those trials, the robot body
experienced larger yawing due to a slight lateral bending of the plastic pole
that suspended the robot resulting from high beam restoring forces. Because
such trials are rare in the robot experiment, the average landscape basin was
close to that without body yawing. Examination of the actual landscape for
each robot trial (see next section) verified that the state trajectory in the
pitch mode was almost always in the pitch basin. For the animal that freely
moved laterally and yawed, the average landscape was a much poorer
approximation of the actual landscape for each trial.

####  2.4.17 Percentage of trials in which system is attracted to basin of
observed mode on actual landscape

Because the average landscape did not account for trial-to-trial variation, to
better quantify how well the potential energy landscape explained the observed
locomotor modes, for both the robot and animal, we further calculated the
actual (not averaged) potential energy landscape for each trial using the
measured position ($x$, $y$) and body yaw of that trial. We then counted the
number of trials in which the system state either stayed in the pitch basin or
transitioned to the roll basin, in accord with the locomotor mode observed,
and we calculated the percentage of trajectories attracted to the
corresponding basin.

####  2.4.18 Energy barrier to escape from pitch local minimum

We measured the potential energy barrier that must be overcome to escape from
the pitch local minimum. First, at each body forward position $x$, we
considered imaginary straight paths away from the pitch local minimum (Figure
2.13B, $iii$, blue dot) in the full pitch-roll space ([$-180\degree$,
180$\degree$]), parameterized by an angle $\Psi$ relative to the negative
pitch direction (body pitched up). Along each imaginary straight path, we
obtained a cross section of the potential energy landscape (Figure 2.13B,
$iii$, inset). Then, we measured and defined the maximal increase in potential
energy in the cross section as the escape barrier along this imaginary
straight path, which was a function of $\Psi$, as shown by a polar plot
(Figure 2.15B). Then, we calculated how escape barrier along different
directions away from pitch local minimum changed as traversal progressed
(increasing $x$). We defined pitch-to-roll transition barrier as the lowest
escape barrier, which occurred at the saddle point between pitch and roll
basins. We measured how pitch-to-roll transition barrier and the location of
saddle point in the pitch-roll space changed as $x$ increased. For the robot,
we calculated pitch-to-roll transition barrier using the average landscape at
each K. For the animal, we used the average landscape with zero average
lateral position and body yaw for simplicity, considering its large trial-to-
trial variation in lateral position and body yaw.

####  2.4.19 Robot system state velocity directions

To measure the direction towards which the robot state trajectory was moving
in the pitch-roll space during transition, for each trial, we calculated the
velocity vector of the state trajectory in the pitch-roll space from the
measured body roll and pitch, low-pass filtered data using a sixth order
Butterworth filter. Then, we calculated the polar angle of this velocity
vector relative to the pitch-roll axes of the landscape. To focus on the
transition, for each trial in which pitch-to-roll transition occurred, we only
considered the portion of the trial occurring over the $x$ range from start of
beam contact to the onset of transition (Figure 2.9, vertical dashed lines
($i$)-($ii$)). For trials in which pitch-to-roll transition did not occur, we
considered the portion of the trial within the average $x$ range where
transition was observed at higher K ($x$ = [$-69$, $-39$] mm). For each K, we
pooled data of trials in which the system was trapped in the pitch mode and
those in which the system transitioned to the roll mode to calculate their
respective distribution (polar histogram) of velocity directions (Figure
2.15D, blue and red). We also measured the directions of the saddle point
between the pitch and roll basins and the local maximum along the pitch-up and
pitch-down directions, averaged over the $x$ range in which transition was
observed (Figure 2.15D, yellow and gray dashed lines).

####  2.4.20 Animal active body and limb adjustments

We observed high speed videos of animal experiments to search for evidence of
the animal using active adjustments to make transition. For each K, we counted
the percentage of trials in which the animal repeatedly flexed its head
relative to the body, differentially used its hind legs, or did both (Wang et
al., 2022).

###  2.5 Results

Before encountering the beams, both the robot and animal moved forward with a
near horizontal body posture. After beam contact, both the robot and animal
started traversing by pushing against the beams, with the body pitched up. As
beam stiffness K increased, pitch-to-roll transition probability increased for
both the robot and animal (Figure 2.12; P < 0.0001, mixed-design chi-squared
test). At low K, neither transitioned to the roll mode even with body
oscillation. At the highest K, both always transitioned, except for the robot
without oscillation. In addition, for the robot at high K (255
mN$\cdot$m/rad), pitch-to-roll transition probability increased with
oscillation frequency f (Figure 2.12B) and thus with kinetic energy
fluctuation (Figure 2.10A). At the highest K tested (344 mN$\cdot$m/rad),
pitch-to-roll transition probability reached one for all f > 0 tested. For
simplicity, below we first describe robot results followed by animal results.

Figure 2.12: Pitch-to-roll transition probability of animal (A) and robot (B)
as a function of beam stiffness K . For robot, we varied oscillation frequency
f to vary kinetic energy fluctuation. *** indicates a significant dependence
on K (animal: mixed-effects chi-squared test, P < 0.0001, $\chi^{2}$ = 297.4;
robot: chi-squared test, P < 0.0001, $\chi^{2}$ = 247.1). n = 64, 60, 60, 62,
64 trials for animal and n = 70 trials at each K for robot.

We tested the first hypothesis by reconstructing the robot’s potential energy
landscape and evaluating how its system state behaved on the landscape (Figure
2.13). Using the measured physical and geometric parameters of the body and
beams, we calculated the robot’s system potential energy (sum of body and beam
gravitational energy and beam elastic energy) as a function of body pitch,
roll, and forward position x relative to the beams. For simplicity, we first
examine results at K = 255 N$\cdot$m/rad. Before the body contacted the beams
(Figure 2.13A, i), pitching or rolling increased body gravitational energy
(because body center of mass was below rotation axes, Figure 2.11). Thus, the
potential energy landscape over body pitch-roll space had a global minimum at
zero pitch and zero roll, i.e., when the body was horizontal (Figure 2.13B,
i). As the body moved closer and interacted with the beams (Figure 2.13A, ii,
iii), the global minimum evolved into a “pitch” local minimum at a finite
pitch and zero roll (Figure 2.13B, ii, iii, blue). Meanwhile, two “roll” local
minima emerged at near zero pitch and a finite positive or negative roll
(Figure 2.13B, ii, iii, red, for rolling right or left), whose energies were
lower than the pitch local minimum. Hereafter, we refer to these local minimum
basins as pitch and roll basins ††A fourth basin also emerged with its local
minimum at a finite positive pitch and zero roll, corresponding to the body
pitching down against the beams. However, such a configuration was never
observed in the robot or animal..

Figure 2.13: Robot locomotor transitions on a potential energy landscape.
Results are shown at K = 255 mN$\cdot$m/rad. (A) Snapshots of body before and
during interaction with two beams in pitch (i, ii, iii) and roll (iii’) modes.
(B) Snapshots of landscape over body pitch-roll space before (i) and during
(ii, iii) interaction. Representative system state trajectories are shown for
being trapped in pitch basin (blue) and transitioning to roll basin (red).
Insets in (iii) define potential energy barriers to escape from pitch local
minimum in pitch-up and positive roll directions. Dashed gray curves on
landscape show boundaries between pitch and roll basins. Note that landscape
evolves as body moves forward (increasing x), and only part of landscape over
pitch-roll space is shown to focus on pitch and roll basins.

We discovered that the robot’s system state during the observed pitch and roll
modes were attracted to the pitch and roll basins, respectively. When the body
was far away from the beams, the system state in pitch and roll space settled
to the global minimum of the landscape (Figure 2.13B, i). During beam
interaction, without oscillation, the system state was trapped in the pitch
basin, leading to the body pushing across the beams in a pitched-up
orientation with little roll (Figures 2.13A, B, ii, iii). With oscillation,
the system stochastically escaped from the pitch basin and crossed a potential
energy barrier to reach the roll basin (Figure 2.13B, iii), thereby
transitioning from the pitch to the roll mode (Figure 2.13B, ii, iii’). We
examined system state trajectory on the landscape reconstructed for each
trial. Whether the robot was trapped in the pitch mode (blue trajectories) or
transitioned to the roll mode (red trajectories), its system state was
attracted to the corresponding basin in nearly all trials (99%, not
significantly different from 1, P > 0.15, Student’s t-test, Figure 2.15A,
iii). Because of this strong attraction, the measured system potential energy
closely matched the observed mode basin’s local minimum energy throughout
traversal (Figure 2.16iii, solid vs. dashed curves). All these findings held
true at other K (near 100%, Figures 2.15A, 2.16). Together, these robot
results supported our first hypothesis.

Next, we tested the second hypothesis. We first observed how kinetic energy
fluctuation affected the robot’s escape from a basin. Again, we examine
results at K = 255 mN$\cdot$m/rad first for simplicity. As f increased (which
increased kinetic energy fluctuation), the system was more likely to escape
from the pitch basin it was initially attracted to and reach the roll basin
(Figure 2.14), resulting in more likely pitch-to-roll transitions (Figure
2.12B, K = 255 mN$\cdot$m/rad).

Figure 2.14: Robot locomotor transitions are stochastic and become more likely
as kinetic energy fluctuation increases. Comparison of state trajectory
ensemble on average landscape (snapshot at x = -0.4 cm) across oscillation
frequencies: (A) f = 0 Hz; (B) f = 3 Hz; (C) f = 6 Hz. Results are shown at K
= 255 mN$\cdot$m/rad. Blue and red curves show trials trapped in pitch basin
and transitioning to roll basin, respectively. Trials in which body rolls left
are flipped to rolling right considering lateral symmetry. n = 10 trials at
each f. Only part of landscape over pitch-roll space is shown to focus on
pitch and roll basins. Blue trajectories exiting pitch basin is an artifact of
landscape averaging. Figure 2.15: Robot tends to transition to roll basin when
kinetic energy fluctuation is comparable to potential energy barrier to escape
pitch local minimum and towards direction of lower barrier. (A) Average
potential energy landscape over pitch-roll space (snapshot at x = 8 mm) with
ensemble of state trajectories. Blue and red curves show trials trapped in
pitch basin and transitioning to roll basin, respectively. Note that landscape
evolves as body moves forward (increasing x) and only part of the landscape
over pitch-roll space is shown to focus on the pitch and roll basins. Top
right number on each landscape shows percentage of trials in which system
state is attracted to pitch/roll basin corresponding to observed mode. Blue
trajectories exiting pitch basin is an artifact of landscape averaging. (B)
Polar plot of potential energy barrier to escape from pitch local minimum
(blue dot) along all directions in pitch-roll space (snapshot at x = -53 mm).
Pitch-to-roll transition barrier is defined as minimal escape barrier (arrows
in iv), which occurs at saddle point between pitch and roll basins (yellow
dot). (C) Pitch-to-roll transition barrier as a function of x. Gray band shows
x range in which pitch-to-roll transition is observed (mean $\pm$ s.d.). Green
circle/line in B, D shows measured average kinetic energy fluctuation of 2.3
mJ at highest f = 6 Hz tested (Figure 2.10). (D) Probability distribution of
state velocity directions in pitch-roll space in the x range where transition
is observed (gray band in C). Blue and red are data from trials trapped in
pitch basin and transitioning to roll basin, respectively. Trials in which
body rolls left are flipped to rolling right considering lateral symmetry.
Black dashed lines and gray shaded sectors show angular direction of maximal
escape barriers (mean $\pm$ s.d) along pitch up and down directions. Yellow
dashed line and shaded sector show angular direction of minimal escape barrier
(mean $\pm$ s.d), which occurs at saddle point. Columns i-iv are at K = 28,
55, 255, and 344 mN$\cdot$m/rad. Data shown in A, C and D are for all f tested
(n = 70 trials) at each K.

Then, we compared the minimal potential energy barrier to escape from the
pitch local minimum with the average kinetic energy fluctuation at $f$ = 6 Hz
(Figure 2.15C, iii). The escape barrier depended on both towards which
direction the system moved in the pitch-roll space (Figure 2.13B, iii, insets,
Figure 2.15B, iii) and body forward position x relative to the beams (Figure
2.15C, iii). Minimal escape barrier occurred at the saddle point between the
pitch and roll basins (Figure 2.15C, yellow dot), which we defined as pitch-
to-roll transition barrier. Only within a small range of x was average kinetic
energy fluctuation at $f$ = 6 Hz (Figure 2.15C, iii, green) sufficient for
overcoming pitch-to-roll transition barrier (Figure 2.15C, iii, black). This
range matched remarkably well with the x range over which pitch-to-roll
transition was observed with increasing likelihood with f (gray band showing
mean $\pm$ s.d. from all trials across f). All these findings held true at K =
344 N$\cdot$m/rad. At K = 28 N$\cdot$m/rad, minimal escape barrier far
exceeded kinetic energy fluctuation, consistent with the absence of
transition. Together, these robot results supported our second hypothesis.

Finally, we tested the third hypothesis by examining the direction towards
which the robot’s system state moved during interaction. At each K, when the
body was not in contact with the beams, the escape barrier was large along all
directions in the pitch-roll space (e.g., x = -80 mm). As the body moved
forward (increasing x), the escape barrier towards the direction of roll
basins reduced drastically, becoming comparable to or even smaller than
average kinetic energy fluctuation at $f$ = 6 Hz (green circle) at the saddle
point (yellow dot). By contrast, escape barrier in the direction of pitching
up or down was always greater than average kinetic energy fluctuation (Figure
2.15B). Examination of how the system state moved on the landscape and
probability distribution of system state velocity directions in the pitch-roll
space (Figure 2.15D) showed that escape was more aligned with the direction of
the saddle point between pitch and roll basins, i.e., escape was more likely
towards the direction of lower barrier. This is intuitive because in other
directions escape barrier was higher and often exceeded kinetic energy
fluctuation. Together, these robot observations supported our third
hypothesis.

Comparison of robot observations across K further suggested a concept of
favorability for locomotor transitions. As K increased, pitch-to-roll
transition became more likely (Figure 2.15A), saturating at one for all f > 0
tested at the highest K (Figure 2.12B). Intuitively, when the beams were
flimsy, the body pushed across (trapped in the pitch mode) as if nothing were
there; when the beams were rigid, the body could not push across and must
roll. Thus, the likelihood of pitch-to-roll transition is positively
correlated with how favorable transitioning to the roll mode is relative to
staying in the pitch mode.

Figure 2.16: Favorability measure for robot. Potential energy of measured
pitch and roll modes (solid, mean $\pm$ s.d.) and of pitch and roll local
minima (dashed) as a function of x. Measured data are for all f tested (n = 70
trials) at each K. Blue and red show trials trapped in pitch basin and
transitioning to roll basin, respectively. Columns i-iv are at K = 28, 55,
255, and 344 mN$\cdot$m/rad. Dotted line at x = 8 mm shows location of
snapshots in Figure 2.15A.

To provide a measure of favorability, we compared whether the pitch or roll
basin was lower during traversal, measured at their respective local minimum
(Figure 2.16). At low K (28 mN$\cdot$m/rad), the pitch basin remained the
global minimum basin throughout traversal (Figure 2.16i), indicating that the
pitch mode was more favorable. As K increased, the pitch basin became
increasingly higher than the roll basin (Figure 2.16ii-iv), indicating that
the roll mode became increasingly more favorable. At small K = 55
mN$\cdot$m/rad for x > 0, although the roll mode was more favorable (Figure
2.16ii), kinetic energy fluctuation was smaller than the transition barrier
(Figure 2.15C, ii); thus, transition did not occur (Figure 2.15A, ii). We
emphasize that the negative correlation between the probability of staying in
or transitioning to a mode and its relative basin height is only an emergent
outcome of the transition physics. The passive robot does not directly feel
how high or how low an adjacent basin is; whether it escapes and makes a
transition only depends on the basin in which it currently resides. Exactly
how favorability difference between basins emerges from the local dynamics of
escaping from a basin remains to be understood.

Figure 2.17: Animal tends to transition to roll basin when kinetic energy
fluctuation is comparable to potential energy barrier to escape pitch basin
and when roll basin is more favorable. (A) Potential energy landscape over
pitch-roll space (snapshot at x = 4 mm, dotted lines in B) with ensemble of
state trajectories. Dashed black curves on landscape show boundary of pitch
basin. Note that landscape evolves as body moves forward (increasing x) and
only part of landscape over pitch-roll space is shown to focus on pitch and
roll basins. We set color map scale to saturate at high energy to highlight
landscape basins. (B) Potential energy of pitch and roll local minima and
pitch-to-roll transition barrier as a function of x. Green line is measured
average kinetic energy fluctuation of 0.02 mJ. Columns i-v are at K = 0.1,
0.2, 0.7, 1.7, and 11.4 mN$\cdot$m/rad (n = 64, 60, 60, 62, and 64 trials).

Similar to the feedforward-controlled robot, the animal’s system state during
the observed pitch or roll mode was attracted to the corresponding basin of
the potential energy landscape (Figure 2.17A, 90% of trials at all K). In
addition, pitch-to-roll transition mostly occurred when both average kinetic
energy fluctuation became comparable to transition barrier and the roll mode
became more favorable than the pitch mode (Figure 2.17B). These similar
observations were remarkable because, for the animal that displayed larger
lateral motion and yawing, leg motion, and individual variation, the landscape
(which was averaged from all trials) provided a much coarser approximation of
the system than for the simpler, well-controlled robot. These animal results
supported our first and second hypotheses. We did not test the third
hypothesis in the animal, considering that the measured system state velocity
was noisy and the animal had higher lateral and yaw motion during traversal.

These results showed that physical interaction with the terrain also played a
major role in the animal’s probabilistic locomotor transitions, even when
active behavior was likely at play. In some trials, the animal transitioned
even when its average kinetic energy fluctuation was smaller than transition
barrier (Figure 2.17B). In addition, the animal occasionally transitioned to
the less favorable roll mode at low K (Figure 2.17A, i, ii, red trajectories).
Further, the animal often flexed its head relative to the body and used the
two hind legs differentially (Wang et al., 2022) during beam interaction (23%,
63%, 89%, 79%, and 85% of the trials at the five K’s). All these were evidence
that the animal’s transition involved active behavior (see discussion). Unlike
the robot that was pulled forward at a constant speed (pulling force always
exceeded beam resistive force), the animal had a finite ability to push
forward and may rely more on such active behavior to facilitate transition
(Wang et al., 2022).

###  2.6 Discussion

In summary, using a transition between two representative modes in a model
system, we demonstrated that an energy landscape approach helps understand how
stochastic transitions of animals and robots across locomotor modes
statistically emerge from physical interaction with complex 3-D terrain. We
discovered that kinetic energy fluctuation from oscillatory self-propulsion
helps the system cross barriers on a potential energy landscape to make
locomotor transitions. This provided compelling evidence about why variation
in movement can lead to stochastic outcome (Stephens et al., 2011) and can be
advantageous when locomotor behavior is separated into distinct modes. This
also explained early observations of surprising ability to traverse
unstructured terrain of bandwidth-limited, rapid-running insects (Sponberg and
Full, 2008) and feedforward-controlled legged robots (Altendorfer et al.,
2001), as both have substantial body oscillation during locomotion. However,
we view this way of “vibrate like a particle” as only one of a suite of
transition strategies. Animals and robots may use other strategies to make
transitions, such as plan anticipatory actions (Gart and Li, 2018) and use
random search (Xuan and Li, 2020a) to overcome barriers, use sensory feedback
adjustments to move towards lower barriers or reduce barriers (Wang et al.,
2022), or even change morphology to modify landscape topology to introduce or
eliminate certain modes (Han et al., 2021).

We posit that there is an “energy landscape dominated” regime of locomotion,
where along certain directions there exist large potential energy barriers
that are comparable to or exceed kinetic energy and/or mechanical work
generated by each propulsive cycle or motion. This may happen when propulsive
forces are either limited by physiological, morphological, and environmental
(e.g., low friction) constraints or do not well align with directions along
which large barriers occur. In complex terrain with many large obstacles (Gart
and Li, 2018; Han et al., 2021; Li et al., 2015; Wang et al., 2022) and even
during strenuous maneuvers (Li et al., 2019; Othayoth and Li, 2021; Xuan and
Li, 2020a, b), these situations are frequent. In this regime, not only does
energy landscape modeling provide a useful statistical physics approach for
understanding locomotor transitions across modes, but it may also allow
comparison across systems (different animal species, robots, terrain, and
modes) to discover general physical principles. Outside of this regime, energy
landscape modeling is not useful—for example, not for ballistic jumping over
small obstacles with kinetic energy far exceeding potential energy barriers.

We discovered that distinct attractive basins of the potential energy
landscape can lead to stereotyped locomotor modes and transitions in both the
animal and feedforward-controlled robot. Because our potential energy
landscape is directly derived from first principles (as opposed to fitting a
model to behavioral data (Mearns et al., 2020; Stephens et al., 2008;
Wiltschko et al., 2015), this result provided compelling evidence that
behavioral stereotypy of animals emerges from their neural and mechanical
systems directly interacting with the physical environment (Berman, 2018;
Brown and Bivort, 2018). In addition, our approach should inform how direct
physical interaction with the environment constrains behavioral hierarchy
(Berman, 2018; Brown and Bivort, 2018). For example, for grass-like obstacle
traversal, starting with our coarse-grained landscape here resulting from a
rigid body interacting with rigid “beams” on torsional springs, we can add
degrees of freedom describing head flexion (Wang et al., 2022), body bending
and twisting, articulated leg motions, and more realistic beam obstacles with
cantilever bending and spatial heterogeneity. This will reveal more nuanced
pathways of transitioning between fine-grained locomotor modes that have a
variety of body and appendage configuration and terrain responses (e.g.,
flexing the head and tucking the legs to roll into the gap (Wang et al.,
2022), separating beams laterally, etc.). Analyzing the disconnectivity
(Wales, 2003) of basins of such a more complete, high-dimensional energy
landscape will reveal the hierarchy (“treeness” (Berman et al., 2016)) of
locomotor modes in complex terrain.

####  2.6.1 Towards understanding emergent behaviour in natural environments

More broadly, these considerations suggest that our energy landscape approach
provides a means towards first-principle, physical understanding of the
organization of locomotor behavior, filling a critical knowledge gap. The
field of movement ecology (Nathan et al., 2008) makes field observations of
trajectories of animals as a point mass moving and making behavioral
transitions in natural environments (e.g., (Suraci et al., 2019)), whose
physical interactions are difficult to measure. Recent progress in
quantitative ethology has advanced understanding of the organization of
behavior (Berman et al., 2016; Berman, 2018; Brown and Bivort, 2018; Mearns et
al., 2020; Wiltschko et al., 2015), often by quantifying kinematics in
homogeneous, near featureless laboratory environments (Berman et al., 2016;
Cande et al., 2018; Mearns et al., 2020; Stephens et al., 2008). Our work
highlights the importance and feasibility of, and opens new avenues for,
studying how the organization of behavior is constrained by an animal’s direct
physical interaction with realistic environments (Li et al., 2013). Doing so
will help inform how animal behavior evolves in nature; it will also simplify
robot design, control, and planning to generate robust locomotor transitions
in complex terrain (Mi et al., 2022), which may be otherwise intractable in
the large locomotor-terrain parameter space. This is analogous to rugged free
energy landscapes allowing divide-and-conquer in protein folding (Dill and
MacCallum, 2012).

Our empirically discovered physical principles of locomotor transitions are
surprisingly similar to those of microscopic systems (Figure 2.18), especially
multi-pathway protein folding transitions where predictive energy landscape
theories have been very successful (Dill et al., 2008; Onuchic and Wolynes,
2004; Wales, 2003). Thus, we envision our energy landscape as the beginning of
a statistical physics theory that will quantitatively predict global
structures and emergent dynamics of multi-pathway locomotor transitions in the
energy landscape dominated regime. An immediate next step towards this is to
model conservative forces using potential energy landscape gradients and add
stochastic, non-conservative propulsive and dissipative forces that perturb
the system to “diffuse” across landscape barriers (analogous to (Bryngelson
and Wolynes, 1989; Socci et al., 1996)). Doing this will also elucidate how
escape dynamics from a basin locally leads to emergent favorability difference
between basins. These physical principles will help reveal how animals, and
how robots should, use local force sensing to control motion to facilitate
locomotor transitions on the landscape. Further, although it seems obvious
that near-equilibrium statistical thermodynamics does not directly apply here,
an energy landscape approach to locomotor transitions in complex terrain
provides opportunities to test and develop new theories of few-body active
matter (Savoie et al., 2019).

Figure 2.18: Comparison of energy landscape between protein-folding
transitions and locomotor transition. (A) Energy landscape theories help
understand physical principles and predict global structures and emergent
properties of probabilistic protein folding transitions via multiple pathways.
Image credits: A, left panel Reproduced from (Dill and MacCallum, 2012); A,
right panel reproduced from (Voelz et al., 2012). (B) We envision energy
landscape modeling as a beginning of a statistical physics approach for
understanding and predicting probabilistic, multi-pathway locomotor
transitions in complex terrain (Li et al., 2015).

Finally, our energy landscape approach provides a conceptual way of thinking
about locomotor modes beyond near-steady-state, limit-cycle-like behavior
(e.g., walk, run, climb (Blickhan and Full, 1993; Goldman et al., 2006; Kuo,
2007)) by adding metastable behavior (Byl and Tedrake, 2009) locally attracted
to landscape basins (e.g., pitch and roll modes here, which are far-from-
steady maneuvers). We foresee the creation of new dynamical systems theories
of terrestrial locomotion (Holmes et al., 2006) that produce transitions
across locally attractive landscape basins as well as between limit-cycle
attractors (Diederich et al., 2002; Geyer et al., 2006). They will enable
using physical interaction to design, control, and plan basins funneled into
one another to compose (Burridge et al., 1999) locomotor transitions to
perform high-level tasks in the real world. Terradynamics of locomotor-terrain
interaction starting from first principles (Li et al., 2013) such as
illustrated here will facilitate this progress.

##  Chapter 3 Propelling and perturbing appendages together facilitate
strenuous ground self-righting

††footnotetext: This chapter is a published paper by Ratan Othayoth and Chen
Li in eLife (2021) (Othayoth and Li (2021))

###  3.1 Summary

Terrestrial animals must self-right when overturned on the ground, but this
locomotor task is strenuous. To do so, the discoid cockroach often pushes its
wings against the ground to begin a somersault which rarely succeeds. As it
repeatedly attempts this, the animal probabilistically rolls to the side to
self-right. During winged self-righting, the animal flails its legs
vigorously. Here, we studied whether wing opening and leg flailing together
facilitate strenuous ground self-righting. Adding mass to increase hind leg
flailing kinetic energy increased the animal’s self-righting probability. We
then developed a robot with similar strenuous self-righting behavior and used
it as a physical model for systematic experiments. The robot’s self-righting
probability increased with wing opening and leg flailing amplitudes. A
potential energy landscape model revealed that, although wing opening did not
generate sufficient kinetic energy to overcome the high pitch potential energy
barrier to somersault, it reduced the barrier for rolling, facilitating the
small kinetic energy from leg flailing to probabilistically overcome it to
self-right. The model also revealed that the stereotyped body motion during
self-righting emerged from physical interaction of the body and appendages
with the ground. Our work demonstrated the usefulness of potential energy
landscape for modeling self-righting transitions.

###  3.2 Significance Statement

Animals can, and robots should, use different types of appendages together to
propel and perturb themselves to self-right when overturned, a strenuous yet
crucial locomotor task.

###  3.3 Author contributions

Ratan Othayoth designed robotic physical model, performed animal and robot
experiments, analyzed and validate results, and created visualizations. Chen
Li conceived and designed study, supervised the project, obtained funding and
reviewed and edited the manuscript for publication,

###  3.4 Introduction

Ground self-righting is a critical locomotor capability that animals must have
to survive (for a review, see Li et al. (2019)). The longer an animal is
flipped over and stranded, the more susceptible it is to risks like predation,
starvation, desiccation (Steyermark and Spotila, 2001), and limited mating
success (Penn and Brockmann, 1995). Thus, it is crucial for animals to be able
to self-right at a high probability because it can mean the difference between
life or death. Similarly, ground self-righting is critical for the continuous
operation of mobile robots (for a review, see Li et al. (2017)).

Ground self-righting is a strenuous task. For example, to self-right,
cockroaches must overcome potential energy barriers seven times greater than
the mechanical energy required per stride for steady-state, medium speed
running (8 body lengths s-1) (Kram et al., 1997) or, exert ground reaction
forces eight times greater than that during steady-state medium speed running
(5 body lengths-1) (Full et al., 1995). Often, animals struggle to self-right
quickly and needs multiple attempts (Brackenbury, 1990; Domokos and Várkonyi,
2008; Hoffman, 1980; Koppányi and Kleitman, 1927; Li et al., 2019; Silvey,
1973) to self-right due to constraints from morphology, actuation, and the
terrain (Domokos and Várkonyi, 2008; Faisal and Matheson, 2001; Golubović et
al., 2017; Li et al., 2019; Steyermark and Spotila, 2001).

Ground self-righting has been studied in a diversity of animals, including
insects (Brackenbury, 1990; Delcomyn, 1987; Faisal and Matheson, 2001;
Frantsevich and Mokrushov, 1980; Li et al., 2019; Sherman et al., 1977; Zill,
1986), crustaceans (Davis, 1968; Silvey, 1973) , mollusks (Hoffman, 1980;
Weldon and Hoffman, 1979; Zhang et al., 2020), and vertebrates (Ashe, 1970;
Bartholomew and Caswell, 1951; Creery and Bland, 1980; Domokos and Várkonyi,
2008; Golubović A, 2015; Koppányi and Kleitman, 1927; Malashichev, 2016;
Pellis et al., 1991; Robins et al., 1998; Vince, 1986; Winters et al., 1986).
A diversity of strategies have been described, including using appendages such
as legs, wings, tail, and neck and deforming the body substantially. Often,
rather than using a single type of appendages or just deforming the body
without using appendages, animals use them together to propel and perturb the
body to destabilize from the upside-down state ((Brackenbury, 1990); (Davis,
1968); (Domokos and Várkonyi, 2008); (Faisal and Matheson, 2001); (Hoffman,
1980); (Li et al., 2019)). In particular, vigorous appendage flailing is a
ubiquitous behavior observed across a diversity of species ((Ashe, 1970);
(Brackenbury, 1990); (Davis, 1968); (Delcomyn, 1987); (Domokos and Várkonyi,
2008); (Faisal and Matheson, 2001); (Hoffman, 1980); (Kleitman and Koppanyi,
1926); (Koppányi and Kleitman, 1927); (Li et al., 2019); (Zill, 1986)). Some
of these animals also use other appendages or the body to propel against the
ground ((Brackenbury, 1990); (Davis, 1968); (Domokos and Várkonyi, 2008);
(Faisal and Matheson, 2001); (Hoffman, 1980); (Li et al., 2019)), and such
vigorous appendage flailing appears to be a desperate, wasteful struggle.

Figure 3.1: Strenuous, leg-assisted, winged ground self-righting of discoid
cockroach. (A) Representative snapshots of animal successfully self-righting
by pitch (blue) and roll (red) modes after multiple failed attempts (black
arrow). See Figure 1—video 1 for a typical trial, in which the animal makes
multiple failed attempts to pitch over the head and eventually rolls to self-
right. (B) Schematic of metastable state with a triangular base of support
(dashed triangle) formed by ground contacts of head and two wing wedges, with
vigorous leg flailing. Red and blue curves show representative trajectories of
left and right hind leg tips from a trial. x-y-z is lab frame. (C, D)
Stereotyped body motion during successful (C) and failed (D) self-righting
attempts in body pitch, body roll, and center of mass height space. i, ii, and
iii in A, C, and D show upside-down (i), metastable (ii), and upright (iii,
iii’) states. Ellipsoids show means (center of ellipsoid) $\pm$ s.d.
(principal semi-axis lengths of ellipsoid) of body pitch, body roll, and
center of mass height at the beginning, highest center of mass height, and end
of the attempt. For failed attempts, the upside-down state at the end of the
attempt is not shown because it overlaps with the upside-down state at the
start of the attempts (i). Data from Li et al. (2019)

Here, we study how propulsive and perturbing appendages together contribute to
successful strenuous ground self-righting. Our model system is the discoid
cockroach’s strenuous ground self-righting using wings (Li et al., 2019))
(Figure 3.1). The overturned animal opens and pushes its wings against the
ground in an attempt to self-right, resulting in its body pitching forward
(Figure 3.1Ai). Because the two opened wings and head form a triangular base
of support, in which the center of mass projection falls (Figure 3.1Aii, B),
this intermediate state is metastable. However, wing pushing rarely pitches
the animal all the way over its head to self-right (the pitch mode, Figure
3.1Aii-iv, blue). Thus, the animal often opens and closes its wings (hereafter
referred to as an attempt ) multiple times, resulting in its body repeatedly
pitching up and down, but it fails to self-right (Figure 3.1A, black arrows).
Eventually, the animal almost always self-rights by rolling sideways over one
of the wings (the roll mode; Figure 3.1Aiii’-iv’, red). Although wings are the
primary propulsive appendages in this self-righting strategy, the animal also
vigorously flails its legs mediolaterally, even when body pitching nearly
prevents them from reaching the ground (Figure 3.1B, dashed curves). The legs
occasionally scrape the ground, the abdomen occasionally flexes and twists,
and the wings often deform passively under load (Li et al., 2019). For
simplicity, we focused on the perturbing effects of the more frequent leg
flailing (but see discussion of these other perturbing motions). Another
curious observation is that, although the animal can in principle rotate its
body in arbitrary trajectories to self-right, the observed body motion is
stereotyped (Figure 3.1) (Li et al., 2019).

A recent potential energy landscape approach to locomotor transitions
(Othayoth et al., 2020, 2021) provides a modeling framework to understand how
propelling and perturbing appendages together contribute to strenuous ground
self-righting. A previous study modeling ground self-righting of turtles in
two dimensions (the transverse plane in which the body rolls) suggested that,
when trapped in a gravitational potential energy well, modest kinetic energy
from perturbing appendages (legs and neck) helps overcome the small potential
energy barriers (Domokos and Várkonyi, 2008). A recent study of cockroaches
took an initial step in expanding potential energy landscape modeling of
ground self-righting to three dimensions (Li et al., 2019). However, due to
frequent camera occlusions, this study was unable to measure the complex 3-D
motions of appendages and only modeled the animal as a rigid body. For turtles
with a rigid shell interacting with the ground, modeling self-righting with a
rigid body is a good first-order approximation. However, this approximation is
no longer good for modeling winged self-righting of the discoid cockroach
because wing opening will change potential energy landscape.

Figure 3.2: Animal leg modification and robotic physical model. (A) Discoid
cockroach with modified hind legs with stainless steel spheres attached. (B)
Robotic physical model in metastable state with a triangular base of support
(dashed triangle), formed by ground contacts of head and two wing edges. Black
arrow shows body Z-axis, $\textit{Z}_{body}$. Figure 3.3: Robot wing and leg
actuation and body orientation measurement. (A) Schematic of leg-assisted,
winged self-righting robot from front and side views with geometric
dimensions. Front view illustrates wing rolling and leg oscillation and side
view illustrates wing pitching. Wing pitching and rolling are by the same
angle, synchronized, and together compose wing opening. (B) Motor angles of
wings (blue) and leg (red) as a function of time. Solid and dashed curves are
commanded and measured motor actuation profiles, respectively. (C) Projection
of gravitational acceleration vector "g"onto body Z-axis "Z" body as a
function of time, measured using onboard IMU. Vertical dashed line shows the
instant when the robot self-righted. In B, C, columns i and ii are for a
representative failed and successful trial, respectively.

Inspired by these insights and limitations, we hypothesized that the discoid
cockroach’s wing opening reduces the barriers to be sufficiently low for small
kinetic energy from leg flailing to overcome. This hypothesis predicted that
the greater the wing opening and leg flailing are, the more likely self-
righting is to occur. We first tested this prediction in the animal, by
directly modifying the hind leg inertia to increase kinetic energy from leg
flailing (Figure 3.2A) and studying how it impacted self-righting probability.
Then, we developed a robotic physical model (Figure 3.2B) to systematically