We hypothesize that maximizing the algorithm’s design freedom would be even
more important when designing robots with increased degrees of freedom, using
more sophisticated optimization algorithms that can operate in an
exponentially growing search space.

Overall, this sequence of experiments showed that automated search could
discover physically realistic shapes and controllers for our shape-changing
robot in one of the two environments we studied (flat ground). Although the
hand-designed controllers each performed comparably to the best discovered
controllers in a single environment, by changing shape, the robot had a better
combined average speed in both environments. Concretely, the best shape-
controller pair found by hill climbing locomoted at a speeds of 0.1888 BL/s on
flat ground and 0.0158 BL/s on incline, resulting in an average speed across
the two environments of 0.1023 BL/s, compared to the average speed of -0.1995
BL/s for the round shape with a rolling gait and 0.0424 BL/s for the flat
shape with an inchworm gait (Table 5.1).

#### 5.2.2 Transferring to a physical robot.

Transferring simulated robots to reality introduces many challenges. For
perfect transferal, the simulation and hardware need to have matching
characteristics, including: material properties, friction modeling, actuation
mechanisms, shape, geometric constraints, and range of motion. In practice,
hardware and software limitations preclude perfect transferal, so domain
knowledge must be used to achieve a compromise between competing
discrepancies. Here, we sought to maximize the transferal of useful behavior,
rather than strictly transferring all parameters.

In simulation, we found that the same actuators could be used to create
different locomotion gaits. When restricted to the cylindrical shape, we found
that sequential inflation of the bladders could be used to induce rolling. A
hand designed policy enabled the flatter robots to move with inchworm motion.
To transfer such shape change and gaits to a physical robot, we created a
robot which had an inflatable core, eight pneumatic surface-based actuators
for generating motion, and variable-friction feet on each edge to selectively
grip the environment (Fig. 5.1). This suite of features allowed the robot to
mirror the simulated robot’s gaits, including rolling and inching. The “hand-
designed controllers” from simulation were transferred to reality by sending
the same command sequence from a PC to digital pressure regulators [156] that
inflated the bladders, resulting in forward motion. However, it was found that
different bladders expanded at different rates and had slightly different
maximum inflation before failure, so in the experiments shown in this
manuscript, the robots were manually teleoperated to approximate the hand-
designed controllers with nonuniform timesteps between each actuation state.
Further details on the robot hardware are presented in the Methods section.

Mirroring simulation, rolling was achieved by inflating the trailing edge
bladder to push the robot forward, exposing new bladders that were then
inflated one at a time, sequentially (Fig. 5.1a and Fig. 5.4a). Each inflation
shifted the robot’s center of mass forward so the robot tipped in the desired
direction, allowing the robot to roll repeatedly. This motion was effective
for locomoting over flat ground (average speed 0.05 BL/s). When we attempted
to command the robot to roll up inclines, the slope of the incline and the
robot’s seam made it difficult for the robot to roll. These observations
suggest the existence of a transition regime on the physical robot, where the
ideal shape-locomotion pair switches from a rolling cylinder to a flat shape
with inchworm gait. However, the boundary is not cleanly defined on the
physical hardware: at increasing inflation levels approaching the strain limit
of the silicone, the robot could roll up increasingly steep inclines up to
${\sim}9{{}^{\circ}}$. After just a few such cycles, the bladders would
irreversibly rupture, causing the robot to roll backward to the start of the
incline.

By accessing multiple shapes and corresponding locomotion modes, shape-
changing robots can potentially operate within multiple sets of environments.
For example, when our robot encountered inclines, it could switch shapes (Fig.
5.4a-c and Supplementary Movie S1). To transition to a flattened state capable
of inchworm motion, the robot would deflate its inner bladder, going from a
diameter of 7 cm (width-to-thickness ratio $\gamma=1$) to an outer height of
${\sim}1.2$ cm ($\gamma{\sim}8.3$) (Fig. 5.4b). The central portions of the
robot flatten to ${\sim}7$ mm, which is approximately the thickness of the
robot’s materials, resulting in $\gamma{\sim}14$. During controlled tests, an
average flat-to-cylinder morphing operation at 50 kPa took 11.5 seconds, while
flattening with a vacuum ($-80$ kpa) took 4.7 seconds (see Methods for
additional details).

Flattening reduced the second moment of area of the robot’s cross-section,
allowing the bladders’ inflation to bend the robot in an arc (Fig. 5.4c). At a
first approximation, body curvature is given as $\kappa=\frac{M}{EI}$, where
$M$ is the externally-induced moment, $E$ is the effective modulus, and $I$ is
the axial cross-section’s second moment of area. Thus, flatter robots should
bend to higher curvatures for a given pressure. However, even for the flattest
shape, bending was insufficient to produce locomotion: on prototypes with
unbiased frictional properties, bending made the robot curl and flatten in-
place.

Variable-friction “feet” were integrated onto both ends of the robot and
actuated one at a time to alternate between gripping in front of the robot and
at its back, allowing the robot to inch forward (average speed of 0.01 BL/s on
flat wood). The feet consisted of a latex balloon inside unidirectionally
stretchable silicone lamina [157], wrapped with cotton broadcloth. When the
inner latex balloon was uninflated (-80 kPa), the silicone lamina was pulled
into its fabric sheath, thus the fabric was the primary contact with the
ground. When the balloon was inflated (50 kPa), it pushed the silicone lamina
outward and created a higher-friction contact with the ground (Fig. 5.5a). To
derive coefficients of static friction ($\mu$) for both the uninflated
($\mu_{u}$) and the inflated ($\mu_{i}$) cases, we slid the robot over various
surfaces including acrylic, wood, and gravel. As the robot slid over a
surface, it would typically exhibit an initial linear regime corresponding to
pre-slip deformation of the feet, followed by slip and a second linear kinetic
friction regime (Fig. 5.5b). From the pre-slip regime, we infer that on a wood
surface $\mu_{u}$ = 0.56 and $\mu_{i}$ = 0.70 — an increase of ${\sim}25\%$
(Fig. 5.5c). On acrylic, $\mu_{u}$ = 0.38 and $\mu_{i}$ = 0.51, which is an
increase of 35$\%$, yielding an inching speed of 0.007 BL/s.

When the difference in friction ($\Delta\mu=\mu_{i}-\mu_{u}$) for the
variable-friction feet was too low (such as on gravel), inchworm motion was
ineffective, as predicted by simulation (Fig. 5.5d). Similarly, when the
average friction ($\mu_{m}=(\mu_{i}+\mu_{u})/2$) was too high, it would
overpower the actuators and lead to negligible motion (Fig. 5.5e). On wood,
the inchworm gait was effective on inclines up to ${\sim}14{{}^{\circ}}$, at a
speed of 0.008 BL/s (Fig. 5.4 and the Supplementary Video). Thus, the robot
could quickly roll over flat terrain (0.05 BL/s) then flatten to ascend
moderate inclines, attaining its goal of maximizing total traveled distance.

Figure 5.4:  Shape change allowed the physical robot to operate in previously
inaccessible environments. a, While round, the robot’s actuators created a
rolling gait which was effective on flat ground. b, By deflating its inner
bladder, the robot could flatten. c, While flat, the outer bladders induced an
inchworm-like gait, allowing the robot to ascend inclines up to ${\sim}14$
degrees. d, The inchworm gait gripped the ground to crawl forward, making it
ineffective on granular surfaces. When faced with such a situation, the robot
could expand its inner bladder to begin rolling. For length-scale reference,
the robot is 10 cm by 15 cm while flattened, and 7 cm diameter by 15 cm while
round. Panels a-c correspond to times from a single trial, while panel d is
from a different trial and has a separate start time. Figure 5.5: The
variable-friction feet change their frictional properties when inflated. a,
When the robot’s feet are inflated, silicone bladders protrude from their
fabric sheath to contact the ground. Blue scale bar on inset represents 1 cm.
b, Force vs. displacement when the robot was slid over wood, acrylic, and
gravel. Each shaded region represents $\pm$one standard deviation about the
mean (solid line). c, Coefficient of static friction. The boxes denote 25th
and 75th percentiles, while the bars represent the median. d, Speed (in
simulation) as a function of the difference between friction values,
$\Delta\mu=\mu_{i}-\mu_{u}$ (where $\mu_{i}$ is friction while the foot is
inflated to 50 kPa, and $\mu_{u}$ is friction while uninflated at -80 kPa). e,
Speed (in simulation) as a function of average friction value,
$\mu_{m}=(\mu_{i}+\mu_{u})/2$. In d-e, the hand-tuned inchworm gait was used.

### 5.3 Discussion

In this study, we tested the hypothesis that adapting the shape of a robot, as
well as its control policy, can yield faster locomotion across environmental
transitions than adapting only the control policy of a single-shape robot. In
simulation, we found that a shape-changing robot traversed two test
environments faster than an equivalent but non-morphing robot. Then, we
designed a physical robot to utilize the design insights discovered through
the simulation, and found that shape change was a viable and physically-
realizable strategy for increasing the robot’s locomotion speed.

We have also shown progress toward an automated sim2real framework for
realizing metamorphosing soft robots capable of operating in different
environments. In such a pipeline, simulated shape-changing robots would be
designed to achieve a desired function in multiple environments, then
transferred to physical robots that could attain similar shapes and behaviors.
We demonstrated each component of the pipeline on a representative task and
set of environments: locomotion over flat ground and an incline. Starting with
an initial robot design, the search method sought valid shapes and control
policies which could succeed in each environment, however, it is notable that
the search found it difficult to find a successful inchworm gait. This is
likely due to a lack of gradient between control policies that do not produce
inching and those that do. Thus, a complete pipeline would need to include
more sophisticated search methods that are better able to search this space.
The effective shapes and gaits were then transferred to physical hardware.
However, the simulation was able to generate some non-transferable behavior by
exploiting inaccuracies of some simulation parameters. For example, when the
friction coefficient was too low, the robot would make unrealistic motions
such as sliding over the ground. Other parameters, such as modulus, timescale,
maximum inner bladder pressure, and resolution of the voxel simulation (i.e.,
the number of simulated voxels per bladder), and material density, could be
adjusted without causing drastic changes in behavior. Developing a unified
framework for predicting the sim2real transferability of multiple shapes and
behaviors to a single robot remains an unsolved problem.

Insights from early physical prototypes were used to improve the simulator’s
hyperparameters (such as physical constants), resulting in more effective
sim2real transferal. Pairing hardware advances with multiple cycles through
the sim2real pipeline, we plan to systematically close the loop such that data
generated by the physical robot can be used to train a more accurate
simulator, after which a new round of simulation to reality transfers can be
attempted. This iterative process will be used to reduce the gap between
simulation and reality in future experiments.

With advances such as increased control of the physical robots’ shape and more efficient, parallelized soft-robot simulators, the pipeline should be able to solve increasingly challenging robot design problems and discover more complicated shape-controller pairs. While the sim2real transfer reported in this manuscript primarily tested intermediate shapes between two extremal shapes | a fully-inflated cylinder and a flattened sheet | future robots may be able to morph between shapes embedded within a richer, but perhaps less intuitive morphospace. For example, robots could be automatically designed with a set $C$ of $N_{c}$ inflatable cores and corresponding constraining fabric outer layers. To transition between shapes, a different subset $C$ could be inflated, yielding $2^{N_{c}}$ distinct robot morphologies. Designing more sophisticated arrangements of actuators and inflatable cores could be achieved using a multilayer evolutionary algorithm, where the material properties of robots are designed along with their physical structure and control policies. [158]. Additionally, it is unclear how to properly embed sensors into the physical robot to measure its shape, actuator state, and environment. Although some progress has been made toward intrinsically sensing the shape of soft robots [159], and environmental sensing [160], it remains an open challenge for a robot to detect that it as encountered an unforeseen environment and edit its body morphology and behavioral control policy accordingly.

Future advances in hardware and search algorithms could be used to design
shape-changing robots that can operate across more challenging environmental
changes. For example, swimming or amphibious robots could be automatically
designed using underwater soft-robot simulation frameworks [161], and changing
shape within each gait cycle might allow robots to avoid obstacles [141] or
adapt to environmental transitions. We have begun extending our framework to
include underwater locomotion, where locomoting between terrestrial and
aquatic environments represents a more extreme environmental transition than
flat-to-inclined surface environments. Our preliminary results suggest that
multiple swimming shape-gait pairs can be evolved using the same pipeline and
robot presented herein (see Supplementary Information). While recent work has
shown the potential advantages of adapting robot limb shape and gait for
amphibious locomotion [162], closing the sim2real gap on shape-changing
amphibious robots remains largely unstudied.

Collectively, this work represents a step toward the closed-loop automated
design of robots that dynamically adjust their shape to expand their
competencies. By leveraging soft materials, such robots potentially could
metamorphose to attain multiple grasping modalities, adapt their dynamics to
intelligently interact with their environment, and change gaits to continue
operation in widely different environments.

### 5.4 Methods

#### 5.4.1 Simulation environment.

The robots were simulated with the multi-material soft robot simulator
Voxelyze [143], which represents robots as a collection of cubic elements
called voxels. A robot can be made to move via external forces or through
expansion of a voxel along one or more of its 3 dimensions.

Voxels were instantiated as a lattice of Euler-Bernoulli beams (Fig.S 5.6a).
Thus, adjacent voxels were represented as points connected by beams (Fig.S
5.6b). Each beam had length $l=0.01~{}m$, elastic modulus $E=400~{}kPa$,
density $\rho=3000~{}\frac{Kg}{m^{3}}$, coefficient of friction $\mu=2.0$ ,
and damping coefficient $\zeta=1.0$ (critically damped). For comparison,
silicone typically has a modulus of ${\sim}100-600~{}kPa$ and density of
${\sim}1000~{}\frac{Kg}{m^{3}}$. These parameters were initially set to
$E=100~{}kPa$ and $\rho=1000~{}\frac{Kg}{m^{3}}$, but were iteratively changed
to increase the speed and stability of the simulation while maintaining
physically realistic behavior. We simulated gravity as an external
acceleration ($g=9.80665~{}\frac{m}{s^{2}}$) acting on each voxel. For the
flat environment, gravity was in the simulation’s negative z direction. Since
changing the direction of gravity is physically equivalent to and
computationally simpler than rotating the floor plane, we simulated the slope
by changing the direction of gravity. The robot could change shape by varying
the force pushing outward, along to the interior voxels’ surface normals,
representing a discrete approximation of pressure (Fig.S 5.6c). The maximum
pressure was set at $12~{}kPa$ after comparison to prior results (for example,
the robotic skins introduced by Shah et al. inflated their pneumatic bladders
to under 20 kPa [141]) and after initial experiments with hardware that
suggested only 10${\sim}$35 kPa was necessary.

The robots’ external bladders were simulated via voxel expansion such that a
voxel expanded along the z-dimension of its local coordinate space at
$3e^{-4}~{}m$ per simulation step and $1.5e^{-5}~{}m$ along the x-dimension.
Expansion in the y-dimension created a bending force on the underlying skin
voxels. This value was changed on a sliding scale from $1.76e^{-4}~{}m$ to
$3e^{-5}~{}m$ based on the pressure of the robots’ core, such that bladder
expansion created minimal bending force when the robot was inflated,
simulating the expansion of physically-realizable soft robots. Concretely, the
y-dimension expansion was computing using a normalizing equation
$(b-a)*((P-P_{MIN})/(P_{MAX}-P_{MIN}))+a$ where $a=1.7$, $b=10$, $P_{MAX}$ is
the maximum outward force per voxel in the robot’s core ($1.4~{}N$), $P_{MIN}$
is the minimum outward force per voxel $0~{}N$ and $P$ is the current outward
force per voxel. These values were adjusted iteratively, until simulated and
physical robots with the same controllers exhibited similar behavior in both
the inclined and flat environments. Lastly, to prevent the robot from slipping
down the hill, and to enable other non-rolling gaits, the robot was allowed to
change the static and kinetic friction of its outer voxels between a low value
($\mu=1e{-4}$) when inactive and high value ($\mu=2.0$) when active.

#### 5.4.2 Optimization

The optimization algorithm searched over 3 adjustable aspects of the robots:
shape (parameterized as inner bladder pressure), orientation of the robot
relative to the incline, and actuation sequence. The algorithm searched over a
single number $p\in[0,12]$ ($kPa$) for shape and
$\theta\in[0^{\circ},90^{\circ}]$ for orientation (see Fig.S 5.3a for
illustrations of each parameter).

The robot’s actuation sequence $S$ over $T$ actuation steps was represented by
a binary $10\times T$ matrix where a $1$ corresponds to bladder expansion and
$0$ corresponds to bladder deflation. Each of the first eight rows
corresponded to one of the inflatable bladders, and the last two rows
controlled the variable friction feet. Each column represented the actuation
to occur during a discrete amount of simulation time steps $t$, resulting in a
total simulation length of $t*T$. $t$ was set such that an actuation achieved
full inflation, followed by a pause for the elastic material to settle.
Actuating in this manner minimizes many effects of the complex dynamics of
soft materials, reducing the likelihood of the robots exploiting idiosyncraies
of the simulation environment. In this study, we used $t=11670$ timesteps of
$\approx$0.000106 ($dt$) seconds each and $T=8$ for all simulations. We the
actuation matrix $T$ is repeated twice for each simulation thus doubling the
length of simulation for a total simulation time of $\approx$19.79 seconds
($(t*dt)*2T)$. To populate $S$, the algorithm searched over a set of
parameters (frequency $f$ and offset $\phi$) for each of the ten actuators.
Both of these parameters were kept in the range $0-T$ where in our case we set
$T=16$. $f$ determined the number of columns between successive actuator
activations, where $f=0$ created a row in the actuation matrix of all $1$’s,
$f=1$ created a row with every other column filled by a $1$, $f=2$ every two
columns filled by a $1$, and so on. $\phi$ specified the number of columns
before that actuator’s first activation.

We optimized the parameters of shape, orientation and actuation using a hill
climber method. This method was chosen for computational efficiency, since a
single robot simulation took considerable wall-clock time (approximately 2.5
minutes on a 2.9 GHz Intel Core i7 processor). The hill-climber algorithm
needs only one robot evaluation per optimization step, in contrast to more
advanced optimization algorithms that often require multiple evaluations per
optimization step. The current set of parameters $C$ was initialized to
randomly-generated values and evaluated in the simulation, where fitness was
defined as the distance traveled over flat ground, or distance traveled up the
incline. A variant $V$ was made by mutating each of the parameters by sampling
from a normal distribution centered around the current parameters of $C$. $V$
was then tested in the simulation, and if it traveled farther, the algorithm
replaced $C$ with $V$ and generated a new $V$. The process of generating
variations, evaluating fitness, and replacing the parameters was done for 200
generations. To determine the repeatability of such an algorithm, we ran 60
independent hill climbers for each of the 6 experiments, as described in the
Results section.

#### 5.4.3 Manufacturing the physical robot.

The physical robot was designed to enable transfer of function, shapes, and
control policies from simulation, while maximizing locomotion speed and ease
of manufacture. In summary, the inner bladder was silicone (Dragon Skin 10,
abbreviated here as DS10, Smooth-On Inc.), the cylindrical body was cotton
dropcloth, and the outer bladders were made with a stiffer silicone (Dragon
Skin 30, abbreviated here as DS30, Smooth-On Inc.) for higher force output.
The variable-friction feet were made out of latex balloons, unidirectionally
stretchable lamina (STAUD prepreg, described in [157]), and cotton dropcloth.
Complete manufacturing details follow.

First, the outer bladders were made (Fig.S 5.7a). Two layers of DS10 were rod
coated onto a piece of polyethylene terephthalate (PET). After curing, the
substrate was placed in a laser cutter (ULS 2.0), PET-side up, and an outline
of the eight bladders were cut into the PET layer. The substrate was removed
from the laser cutter and the PET not corresponding to the bladders (i.e., the
outer “negative” region) was removed. Two layers of DS30 were rod coated onto
the substrate. DS30 is stiffer than DS10, and was used to increase the outer
actuators’ bending force, while DS10 was used in all other layers to keep the
robot flexible. Using ethanol as a loosening agent, the encased PET was then
removed from all eight bladders. Finally, a layer of DS10 was cast over the
bladders’ DS10 side for attaching broadcloth to begin manufacturing of the
inner bladder.

The inner bladder was made by first soaking cotton broadcloth (15 cm by 20 cm)
with DS10, and placing it on the uncured layer on top of the outer bladders
(Fig.S 5.7b). PET was then laid on the robot, and the inner bladder outline
was lasercut into the PET. Again, the outer PET was removed, and DS10 was
rodcoated to complete the inner bladder. The PET was removed using ethanol and
tweezers, and silicone tubing (McMaster-Carr) was inserted into each bladder
and adhered with DS10.

To make the variable-friction feet, rectangular slits were lasercut into
broadcloth, and unidirectionally stretchable laminate [157] was attached using
Sil-Poxy (Smooth-On Inc.) (Fig.S 5.7c). Latex balloons were attached using
Sil-Poxy, and the feet were sealed in half with Sil-Poxy to make an enclosed
envelope for each foot. When at vacuum or atmospheric pressure, the fabric
would contact the environment, leading to a low-friction interaction. When the
feet were inflated, the silicone would contact the environment, allowing the
feet to increase their friction.

Finally, the robot was assembled by attaching the feet to the main robot body
using Sil-Poxy, and the robot was folded to bond the inner bladder to the
bladder-less half, using DS10 (Fig.S 5.7d).

#### 5.4.4 Experiments with the physical robot.

To test the robot’s locomotion capabilities, we ran the physical robots
through several tests on flat and inclined ground. The pressure in the robots’
bladders was controlled using pneumatic pressure regulators [156]. The robots
were primarily operated on wood (flat and tipped to angles up to
${\sim}15^{\circ}$), with additional experiments carried out on a flat acrylic
surface and a flat gravel surface (see Fig. 5.4 and the Supplementary Movie
S1).

The variable-friction feet were assessed by pulling the robot across three
materials (acrylic, wood, gravel) using a materials testing machine (Instron
3343). The robot was placed on a candidate material and dragged across the
surface at $100~{}mm/min$ for $130~{}mm$ at atmospheric conditions
($23^{\circ}~{}C$, $1~{}atm$). This process was repeated 10 times for each
material, at two feet inflation pressures: vacuum ($-80~{}kPa$) and inflated
($50~{}kPa$). The static coefficient of friction, $\mu_{s}$, was calculated by
dividing the force at the upper end of the linear regime by the weight of the
robot.

The robot’s shape-changing speed was assessed by manually inflating and
deflating the robot’s inner core for 20 cycles. For each cycle, the robot body
was inflated to a cylindrical shape with a line pressure of $50~{}kPa$, and
the time required to attain a diameter of ${\sim}7~{}cm$ was recorded. The
body was then deflated with a line pressure of $-80~{}kPa$, and the time
required to flatten to a height of ${\sim}1.2~{}cm$ was recorded.

### Supplementary Material

  * •

Movie S1. A soft robot that adapts to environments through shape change

  * •

Figure S1. The simulated robot

  * •

Figure S2. Manufacturing the physical robot

Figure 5.6: The simulated robot could switch between round and flat shapes, as
modeled by a shape-changing lattice. a, Visual rendering of the robot in both
its inflated and flat shapes, with the corresponding underlying Euler-
Bernoulli beams shown on the right, in red. b, A voxel is represented as a
point mass connected to its neighbors by beams, adapted from [143]. c, The
pressure ($P$) vectors (red) acting on each interior voxel (green) when fully
inflated. Figure 5.7:  Manufacturing the physical robot. a, First, the outer
bladders were made out of silicone. b, The outer bladders were bonded to
silicone-soaked cotton broadcloth, and the inner bladder was fabricated. c, To
make variable-friction feet, rectangular slits were lasercut into broadcloth,
and unidirectionally stretchable lamina [157] and latex balloons were attached
with silicone. d, The robot was assembled by attaching the feet to the main
robot body, and the robot was folded to bond the inner bladder to the bladder-
less half. Small squares to the right of each schematic depict a simplified
cross-section of the robot.

## Chapter 6 Arguments

### 6.1 Overview

In this dissertation, I have presented an embodied approach to improve the
multitask learning performance of neural networks in robotics. This work
introduces novel metrics that quantify how certain designs will perform across
multiple tasks and ultimately demonstrates how aspects of a robot’s design can
improve sample efficiency and resistance to catastrophic interference,
allowing for better multitask behavior. Furthermore, the discoveries in this
dissertation can easily be combined with other neural centric approaches to
these problems and are thus synergistic rather than competitive with other
current work in this domain.

In this chapter I will provide a discussion of the significance of the results
found in the preceding chapters as well as its potential impact on future
work. Thus, it is assumed that the reader is familiar with the main results in
the preceding chapters, each of which is a published paper (chapter 4 is in
review) that details how a robot’s design is a vital component of embodied
neural control, and specifically influences whether or not a specific neural
network and training algorithm will succumb to the effects of catastrophic
interference.

In Chapter 2 I showed, that across three equally capable robots, design
variations were the principle factor in determining the amount of catastrophic
interference they experienced during training. In Chapter 3 I developed novel
metrics that provide a theoretical foundation for measuring the learning
ability of a robot design as well as its likely hood to resist catastrophic
interference. In Chapter 4 I tested the metrics developed in Chapter 3 across
various robot designs and training algorithms, demonstrating that they were
predictive of a robot’s performance. I also showed how such designs could be
found via the co-optimization of morphology and control and that evolution
naturally found designs that exhibited sensor homeostasis. Lastly, in Chapter
5 I provided examples from a physical soft robot that was optimized based on
these principles to achieve locomotion in different environments.

### 6.2 Significance and Future Work

This work has implications for a variety of disciplines, thus here I describe
how my research compliments the fields of neural network based control,
robotics, embodied cognition, automated design, and morphological computation
as well as hypothesize how future work may include and improve on discoveries
from this dissertation. I also discuss homeostasis and the reality gap, two
topics that typically fall under evolutionary robotics research.

#### 6.2.1 Neural-Network-Based Control

In this dissertation, I have shown that a robot’s design (e.g. sensor
location) can fundamentally alter the optimal weight manifolds of a neural
network that controls that robot, thus influencing the controller’s
learnability and resistance to catastrophic interference. More specifically,
by changing sensor location, we observed changes in the number and placement
along the loss surface of control parameters suitable for individual
environments, and in how these optimal yet environment-specific parameters
overlap across different environments to produce generalist controllers which
resist catastrophic interference. Traditional methods for overcoming
catastrophic interference focus on finding better ways to navigate the loss
surface of task, while the methods here instead alter the loss surface itself.
Using this strategy, I have shown that regardless of the algorithm used, it is
possible to alleviate catastrophic interference by changing aspects of the
robot’s design, without increasing controller complexity. Specifically, design
changes can suppress or exacerbate the potential for catastrophic interference
by expanding or shrinking the overlap of performant controller parameters for
that body plan across different environments.

As mentioned, one of the reasons that neurocentric approaches dominate the
catastrophic interference literature is likely due to the _Universal
Approximation Theorem_ [26], which states that for any continuous function
there exists a set of parameters for a sufficiently large neural network that
can approximate that function. In these terms, standard approaches are
continually seeking new ways to find these parameters no matter the difficulty
of the task. The work in this dissertation seeks to change the function itself
from a more difficult function to an easier one; one that has a larger set of
parameters that can approximate the function. This work shows that the
physical design of a robot can influence the learning capabilities of a neural
controller, but it is a property that is often abstracted away as an
environmental component in other research that addresses this problem. Thus,
this research has the capability to transform the current thought paradigm in
this and other problems in machine learning research since it shows that
including design in the scope of the problem allows for the power to render
the loss landscape more benign for learning. This may even extend to currently
nonembodied fields such as vision tasks, and natural language processing
(NLP). For example, some recent work in NLP trained robots to act similarly to
semantically-similar word2vec [163] encoded commands. This work showed that
inducing an alignment between motoric and linguistic similarities (thus
improving performance) was facilitated or hindered by the mechanical structure
of the robot [164].

Similarly, state of the art methods in image classification rely solely on
neural network training and in particular the convolutional neural
architecture. However, certain types of classification tasks are still
difficult and unsolved. For instance, currently a neural network can
outperform a human on digit classification in the MNIST data set, yet the same
model cannot compete with humans in the real world; MNIST is a toy problem
when it comes to real world variation in digit appearance which vary in
writing medium, writing utensils, color, surroundings context, etc. This same
reasoning extends to networks that outperform humans on the ImageNet data set;
it is still not as challenging as everyday vision problems the human visual
system solves. It is unclear what would allow current systems to move from the
world of data sets to human level competence and generality. While, it is
likely that there are many advances that will need to be made, this work
provides new motivation for embodied approaches to vision tasks. For example,
a human classifies an object by much more that what is looks like in two
dimensions (2D). In reality objects are three dimensional (3D), they have
unique smells, tastes, and physical interactions with ourselves and the rest
of the environment. Many objects are classified by what they do more than what
they look like. Even when humans make classification in static 2D images,
these types of inferences and experiences are brought to bear on the task.
This is also true in any discussion of natural language processing; words are
more than the recognition of their characters. The word “fall” conveys not
only an action but conjures the sensory knowledge of how the stomach feels in
free fall. Work that includes embodiment as an element in these tasks was
initiated in the past [29, 165, 166], but has largely been overshadowed in
more recent research by rapid deep learning advances. Thus, the work in this
dissertation provides the motivation to embody neural networks in other
domains and to mate them with robot designs that have been optimized for the
network inside to facilitate neural training. However, finding such robot
designs may be difficult and unintuitive.

#### 6.2.2 Robotics

Conventional methods for building robots require intensive engineering steps
that still heavily rely on human intuition and individual expertise. This
design process requires a massive investment of time for both simulated and
real robots. Given these current time constraints, a robot progresses through
relatively few design and evaluation stages. Even then, it is not clear that
the final robot design is highly optimized for the task or its controller
[167, 168]. As stated previously, this is typically not seen as a problem in
light of Universal Approximation Theorem [26]. Despite this, current robot
systems tend to be limited to performing very specific and repetitive tasks
that can be programmed without the use of machine learning. This is
particularly the case for industrial robots which perform a single function on
a production line. Furthermore, using neural controllers in current robotic
systems would be inefficient due to the long training times of current neural
networks. Thus, neural control in robotics faces two problems: a fundamental
inability to compete with traditional hand designed control algorithms and
sample inefficiency.

In this dissertation, I have shown that, from a learning perspective, the best
robot designs are often configured into unintuitive designs that would likely
be overlooked by human designers. In contrast to human-derived designs, these
unintuitive designs have been shown to be easily attuned to their environment
and can be generalized to perform well across tasks. Thus, this work provides
a foundation for the creation of more general purpose robots, whose designs
can be found can be found automatically via optimization. I have also shown
that the process of co-optimization (optimization of both design and
controller) is more sample efficient than traditional controller optimization
alone. In light of the current computational constraints preventing the
adoption of neural controllers in robotics, these advances provide a path
towards automated design of trainable robots, ultimately costing less human
time, energy, and money. While most of this work has been conducted within the
domain of traditional rigid robots, the results of this dissertation may be
more immediately applicable to the emerging field of soft robotics.

One reason neural control is not used in rigid robots is because of their
predictable nature, i.e., we have good mathematical models to describe how
they behave given any internal forces or external forces in the environment.
Thus, they can usually be controlled with a simpler, and standard
proportional-integral-derivative (PID) controller [169]. Where it is often
easy to predict the cause and reaction cycle in rigid robots, soft robots tend
to exhibit complex physical properties and often chaotic motion patterns that
are difficult to predict, and thus, (PID) control has been infeasible for many
soft robotic platforms [170, 171]. Soft robots’ unintuitive nature results
from the vast possibilities of shapes, material properties, motors, and
actuators compared to traditional rigid robots. Thus, there is particular
interest to use automation in both the design and control of soft robots [146,
172, 173, 174], an example of which I have demonstrated in this work via the
optimization of a soft shape-changing robot.

Another reason soft robots are an ideal platform for design and control
optimization is that amorphous material properties allow them to change their
design (including shape, rigidity, sensors) over the course of performing some
task. Some robots are achieve this by using a subset of a lattice of sensors
[101] or using adjustable antenna[102]. Some soft robotics projects have
allowed for changes in other morphological attributes, such as geometry [88],
material properties [103], or the number and placement of actuators [30].
However, to date the motivation for creating such shape and design changing
robots has been scant. The work in this dissertation suggests that a robot
that can dynamically change its shape could continually provide a favorable
learning environment, from the point of view of its embodied controller, in a
variety of environments. Here I have shown a specific example of this by
utilizing both automated control and design to create a multi-environment soft
robot that can traverse different terrains. Yet, this work is in its infancy,
and the robot used in this work has a relatively small space for design
variation and actuation. Thus, future work in this area should focus on using
more complex robot designs as well as including the metrics developed in this
dissertation to measure the effects on the neural controller.

#### 6.2.3 Embodied Cognition

As has been mentioned, current neural centric approaches to multitask robotics
tend to ignore the consequences of a specific robot design, often touting the
generality of the neural control algorithm to apply equally well to any
platform. However, in this work I have shown that the robot design has a
distinct impact on the ability to train a neural control policy across various
training methods. If we think of the robot design as a part of the task
environment, this means that changing robot design before or during training
is akin to having the ability to alter the task to better suit the neural
controller. The success of this strategy in this work reinforces and is
reinforced by current trends in the examination of human intelligence in the
field of embodied cognition. Cognition is embodied when aspects beyond the
brain take a significant role in cognitive processing. In both traditional
machine learning and cognitive science, the body is considered as a peripheral
component of intelligent behavior [175]. Proponents of embodied cognition have
been able to show that many of the intelligent processes in nature are a
result of deep connections between both the body and brain of organisms [176].
For example it has been shown that in humans, many of our peripheral systems
work passively and only send signals to the brain when they themselves compute
something worth further cognitive computation [177, 178]. Other work mentioned
in the introduction has also shown that certain structures (such as the human
foot) allow the brain and limbs to perform a consistent function with minimal
changes in explicit computations [27, 177]. This work, thus, adds a
significant contribution to this field of research by showing that benefits of
an integrated view of intelligence work in a robotic system as well. This work
is also supported by evolutionary biology, which, at varying time scales, co-
optimizes both the neural and physical structures of an organism
simultaneously. However, this and other work has yet to reach the
sophistication of nature’s design algorithms.

#### 6.2.4 Automated Design

Although I was able to co-optimize control and design with success, in this
work, we relied on simple evolutionary methods (mostly due to the simplicity
of the design). For this work to take a larger role in other robotics
platforms, much work still needs to be done on the automated design of complex
systems.

This dissertation relies on the most simple form of design optimization,
called direct encoding, where an optimization algorithm directly manipulates
the physical properties of the robot. In biological terms this means that a
single gene alone determines a specific phenotypic trait. For instance the
optimization algorithm in Chapter 5 mutated one quantity that was used
directly as the inflation pressure of the robot core. In Chapters 3 and 4, the
evolved floating point numbers were the actual coordinates of the sensors on
the robot’s body. This method of optimization is not seen in biology and has
been shown to be less competent in the literature to date [179, 180].

Recently, methods have been created that utilize a more biologically plausible
indirect-encoding scheme where multiple genes may be used to determine a
single design trait and such genes are interpreted (an indirect step) to
determine the effect of a design aspect [181]. This later method is prevalent
in nature and, is often considered more powerful as it allows for regularity
in a design. Regularity is typically used to describe the compressibility of
the information required to produce a structure that involves symmetries and
repetition of design motifs [182]. Thus, a good design pattern the is produced
by a set of genes can be reused and varied more easily, for example like the
finger of a hand. The utility of indirect encoding has been proven by recent
work in automated design [183, 49, 31], which has been shown to outperform a
typical direct encoding scheme. However, much of this work is still performed
on relatively simple robot designs compared to those currently used in
reality, and work is still ongoing on working to improve encoding schemes for
real world complexity [183].

This research gives insight into how robotic design is a key element in the
goal of achieving general artificial intelligence and provides a basis to
continue to improve the state of automatic design algorithms. This work also
provides motivation to develop these methods in a manner consistent with
simultaneous co-optimization of both brain and body.

#### 6.2.5 Homeostasis

The work presented here provides more than a simple motivation to renew work
on automated design or co-optimization; it proposes a mechanism by which such
work may be judged. Here we have shown that the design of a robot is an
important factor in determining its learning performance, but this alone does
not allow one to determine how to measure the quality of a design during the
optimization process. Without such a mechanism, it is impossible to create a
purely design-oriented optimization problem seeing as it would lack any goal.
However, in chapter 4 I proposed a possible proxy for the measurement of body
designs resistant to catastrophic interference, namely, homeostasis. In that
chapter it was shown that during co-optimization of body and control, with the
sole goal being performance, the designs found followed a natural gradient
from those that experience “different” environments very differently to those
that experience them as nearly identical. Specifically, if we keep track of
the light signal over time, and measure the similarity between that signal
across different environments, we find that designs later in the optimization
have much more similar signals as measured by dynamic-time-warping.
Furthermore, these are the same designs that were found to have high scores
for learnability and resistance to catastrophic forgetting. Thus, signal
similarity could be used as a proxy metric for the more comprehensive
measurements of $M_{L}$ and $M_{CI}$. While this was not explicitly tried in
this work, future work could focus on using homeostasis between a robot’s
sensors to discover designs that are more favorable to learning. It is also
likely that there are other mechanisms that describe why certain designs
perform better than others, thus future work should also be focused on
expanding the scope of possible mechanisms beyond homeostasis.

#### 6.2.6 Morphological Computation

One branch of evolutionary robotics that stands to benefit from the
discoveries in this work is morphological computation. Research in this field
focuses on showing that the body of an organism or robot actually can and does
perform computations. However, this field has often struggled with two
distinct problems. The first is that research often simply finds new ways to
invent mechanical computers that are inspired by lifelike forms. Yet, these
structures become completely devoid of the functions found from their
counterparts in nature. For instance one paper showed how the waves produced
by an octopus-like arm could be used for computing [184], however, it does not
state that octopi can or would use their arms in this manner. My work does not
contribute to this type of morphological computation research. However, the
second type of work done in this field suffers from the problem of quantifying
what or how much computation a morphological component is performing. In this
line of reasoning, one of the first morphological computation experiments is
that of passive dynamic walkers [185, 186, 85]. These are bipedal structures
without motors that can walk with a natural human like gait down a slight
incline with perfect balance; with added motors these robots have a morphology
that allows for a simple controller to perform walking, thus, the morphology,
by its design, is said to be handling much of the computation necessary to
walk since the controller no longer performs many of the difficult
calculations for balance and joint control. There are many other such
experiments of this manner [187, 177], which all amount to describing ways
that a design can make the controller task easier or require less computation.
However, as we have discussed, they often fall short of quantifying just how
much computation their designs are performing. Furthermore, most research does
not specifically focus on improving designs for neural control.

The work I have shown here introduces two novel metrics $M_{L}$ and $M_{CI}$,
that could be used in morphological computation research to explicitly define
the type of computation being performed by a specific design. Specifically,
$M_{L}$ could be used to describe the computational work done by a design as
an increase in the optimal parameter space, thus quantifying what it means for
a design to make a control problem easier. This is especially true since the
metric was shown to be causally correlated with sample efficiency.

Figure 6.1: (Left) A hypothetical weight manifold for a robot trained in
simulation. (Right) The weight manifold for the same task but showing the true
optimal parameter space dictated by reality. We can imagine a shift in physics
as a shift in the location of the optimal weight manifold $\theta^{*}$ in the
total space $\theta$. Thus a robot design that creates a larger $\theta*$
would be more likely to successfully transfer its learned behavior to reality
transfer to reality.

#### 6.2.7 Reality Gap

The reality gap is the term given to the failure of systems trained and tested
in simulation to perform adequately when the system is transferred into a real
physical counterpart (when we move from simulation to reality). Here the gap
refers to the difference in performance which is often catastrophic, meaning
the system is not usable once transferred to reality. The reality gap problem
was first introduced by Jakobi in 1997 [144]. However, this problem has
resurfaced with the use of neural networks in robotics [147, 188, 189, 190].
Neural networks require many samples of data, which for robotics is most
easily obtained by running physics simulations that recreate the
characteristics of a real task. Again, when the trained neural network is
transferred to a real robot, the performance often varies to the degree that
the transfer is considered a failure. The most common way to combat this is to
either increase the generalization ability of the network [188], this can
involve introducing noise or varying physics in the simulation, or using
standard neurocentric methods: normalization, training schedules, different
architectures, rapid adjustment learning, etc [191, 192]. However, Future work
based on this dissertation may have the potential to shift the current
paradigm by focusing on finding robot designs that are robust to environmental
changes.

In this work, I found robot designs that were robust to differing environments
(locations of a light source); this principle could similarly be applied to
the reality gap by finding robot designs robust to other types of changes in
the environment such as physics. It is also possible the designs found in this
dissertation, whose robustness allows for multitask behavior, are not mutually
exclusive from the same designs that may allow for successful transfer to
reality. As described in Fig. 6.1, we could similarly compare the weight
manifold for one of the best phototaxis designs found in this dissertation
against the baseline human designs. We can imagine a shift in physics as a
shift in the location of the optimal weight manifold $\theta^{*}$ in the total
space $\theta$. When we look at the problem in this manner, we can see that
the hand designed robot is very unlikely to experience a shift that would
allow it to maintain the same parameters and experience the same behavior from
simulation to reality. This is because there are very few parameters that
allow for optimal behavior. On the other hand, the optimized design may
experience many types of shifts, and the parameters which were optimal in the
simulation would have a much higher probability of remaining in the shifted
optimal region in reality. The possible problems with this theory are that the
physics may also manipulate not just the location but the total size of the
optimal weight manifold as well. Testing this theory might also be difficult
by requiring a large number of robot to be fabricated. However, the work done
thus far provides the motivation to test this hypothesis in future and to
further apply the principles of design optimization to this and other areas in
robotics research.

### 6.3 Conclusion

In this dissertation, I have presented an embodied approach to improve the
multitask learning performance of neural networks in robotics. This work
introduces novel metrics that quantify how certain designs will perform across
multiple tasks and ultimately demonstrates how aspects of a robot’s design can
improve sample efficiency and resistance to catastrophic interference,
allowing for better multitask behavior. However, this work may also be
applicable to many other areas of machine learning, robotics, and engineering.
While the research in this dissertation has been mainly focused on examining
neural control in simple simulated robots in an arguably simple task, this
work strongly suggests that the conclusions drawn in this dissertation would
apply generally to more complex robots and tasks. Thus, future work should
focus on proving this explicitly by scaling up to more complex robots and
environments. This would be most helpful to the larger community by choosing
multitask environments that have been used in other robotics experiments to
compare the performance of design-optimized robots against a common benchmark.
This would also provide the ability to test synergies between the methods
described in this dissertation and the more common neurocentric approaches
described throughout this dissertation. Other avenues of future work should
also explore the relationship between design and other common robotics
problems. I have mentioned some of these problem here such as problems in soft
robotics, the reality gap, or morphological computation. Lastly, future work
should focus on deploying these techniques in real robotic platforms as some
designs that may be most beneficial in simulation could prove to be less
feasible from a manufacturing standpoint. The best designs in this
dissertation were often unintuitive, thus, learning to modify the space of
possible designs by realistic manufacturing capabilities could be necessary to
realize the benefits of design optimization in real robotic platforms.

## Bibliography

  * [1] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly and Neil Houlsby  “Big transfer (bit): General visual representation learning”  In _arXiv preprint arXiv:1912.11370_ 6.2, 2019, pp. 8
  * [2] Taco S Cohen, Mario Geiger, Jonas Köhler and Max Welling  “Spherical cnns”  In _arXiv preprint arXiv:1801.10130_ , 2018
  * [3] Bill Zhang  “Adversarial Examples that Fool both Computer Vision and Time-Limited Humans”  In _arXiv preprint arXiv:1802.08195_ , 2018
  * [4] Andrew Brock, Jeff Donahue and Karen Simonyan  “Large scale GAN training for high fidelity natural image synthesis”  In _arXiv preprint arXiv:1809.11096_ , 2018
  * [5] Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le and Ruslan Salakhutdinov  “Transformer-xl: Attentive language models beyond a fixed-length context”  In _arXiv preprint arXiv:1901.02860_ , 2019
  * [6] Wen Zhang, Yang Feng, Fandong Meng, Di You and Qun Liu  “Bridging the gap between training and inference for neural machine translation”  In _arXiv preprint arXiv:1906.02448_ , 2019
  * [7] Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova  “Bert: Pre-training of deep bidirectional transformers for language understanding”  In _arXiv preprint arXiv:1810.04805_ , 2018
  * [8] Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds and Petko Georgiev  “Grandmaster level in StarCraft II using multi-agent reinforcement learning”  In _Nature_ 575.7782 Nature Publishing Group, 2019, pp. 350–354
  * [9] Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell and Raphael Ribas  “Solving rubik’s cube with a robot hand”  In _arXiv preprint arXiv:1910.07113_ , 2019
  * [10] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis and Thore Graepel  “Mastering atari, go, chess and shogi by planning with a learned model”  In _Nature_ 588.7839 Nature Publishing Group, 2020, pp. 604–609
  * [11] Frank Rosenblatt  “The perceptron: a probabilistic model for information storage and organization in the brain.”  In _Psychological review_ 65.6 American Psychological Association, 1958, pp. 386
  * [12] Henry J Kelley  “Gradient theory of optimal flight paths”  In _Ars Journal_ 30.10, 1960, pp. 947–954
  * [13] Stuart Dreyfus  “The numerical solution of variational problems”  In _Journal of Mathematical Analysis and Applications_ 5.1 Academic Press, 1962, pp. 30–45
  * [14] Paul J Werbos  “Generalization of backpropagation with application to a recurrent gas market model”  In _Neural networks_ 1.4 Elsevier, 1988, pp. 339–356
  * [15] David E Rumelhart, Geoffrey E Hinton and Ronald J Williams  “Learning representations by back-propagating errors”  In _nature_ 323.6088 Nature Publishing Group, 1986, pp. 533–536
  * [16] Kunihiko Fukushima  “Neocognitron: A hierarchical neural network capable of visual pattern recognition”  In _Neural networks_ 1.2 Elsevier, 1988, pp. 119–130
  * [17] Yann Le Cun, Lionel D Jackel, Brian Boser, John S Denker, Henry P Graf, Isabelle Guyon, Don Henderson, Richard E Howard and William Hubbard  “Handwritten digit recognition: Applications of neural network chips and automatic learning”  In _IEEE Communications Magazine_ 27.11 IEEE, 1989, pp. 41–46
  * [18] Ian Goodfellow, Yoshua Bengio and Aaron Courville  “Deep Learning (Adaptive Computation and Machine Learning series)”  e MIT Press, Cambridge, England, 2016
  * [19] Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville and Yoshua Bengio  “An empirical investigation of catastrophic forgetting in gradient-based neural networks”  In _arXiv preprint arXiv:1312.6211_ , 2013  URL: https://arxiv.org/abs/1312.6211
  * [20] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho and Agnieszka Grabska-Barwinska  “Overcoming catastrophic forgetting in neural networks”  In _Proceedings of the national academy of sciences_ 114.13 National Acad Sciences, 2017, pp. 3521–3526  URL: https://www.pnas.org/content/114/13/3521
  * [21] Michael McCloskey and Neal J Cohen  “Catastrophic interference in connectionist networks: The sequential learning problem”  In _Psychology of learning and motivation_ 24 Elsevier, 1989, pp. 109–165
  * [22] Ken Wang  “DeepMind achieved StarCraft II GrandMaster Level, but at what cost?”  In _Medium_ Medium, 2020  URL: https://medium.com/swlh/deepmind-achieved-starcraft-ii-grandmaster-level-but-at-what-cost-32891dd990e4
  * [23] Bernard Ans and Stéphane Rousset  “Avoiding catastrophic forgetting by coupling two reverberating neural networks”  In _Comptes Rendus de l’Académie des Sciences-Series III-Sciences de la Vie_ 320.12 Elsevier, 1997, pp. 989–997
  * [24] Robert Coop, Aaron Mishtal and Itamar Arel  “Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting”  In _IEEE transactions on neural networks and learning systems_ 24.10 Ieee, 2013, pp. 1623–1634
  * [25] Chelsea Finn, Pieter Abbeel and Sergey Levine  “Model-agnostic meta-learning for fast adaptation of deep networks”  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , 2017, pp. 1126–1135  JMLR.org  URL: http://proceedings.mlr.press/v70/finn17a/finn17a.pdf
  * [26] George Cybenko  “Approximation by superpositions of a sigmoidal function”  In _Mathematics of control, signals and systems_ 2.4 Springer, 1989, pp. 303–314
  * [27] Jenny A Kent, Joel H Sommerfeld and Nicholas Stergiou  “Changes in human walking dynamics induced by uneven terrain are reduced with ongoing exposure, but a higher variability persists”  In _Scientific Reports_ 9.1 Nature Publishing Group, 2019, pp. 1–9
  * [28] Josh C Bongard  “Evolutionary robotics”  In _Communications of the ACM_ 56.8 ACM New York, NY, USA, 2013, pp. 74–83
  * [29] Karl Sims  “Evolving virtual creatures”  In _Proceedings of the 21st annual conference on Computer graphics and interactive techniques_ , 1994, pp. 15–22
  * [30] Hod Lipson and Jordan B Pollack  “Automatic design and manufacture of robotic lifeforms”  In _Nature_ 406.6799 Nature Publishing Group, 2000, pp. 974–978  URL: https://www.nature.com/articles/35023115
  * [31] Nick Cheney, Josh Bongard, Vytas SunSpiral and Hod Lipson  “Scalable co-optimization of morphology and control in embodied machines”  In _Journal of The Royal Society Interface_ 15.143 The Royal Society, 2018, pp. 20170937  URL: https://doi.org/10.1098/rsif.2017.0937
  * [32] Collin Cappelle, Anton Bernatskiy and Josh Bongard  “Reducing Training Environments in Evolutionary Robotics Through Ecological Modularity”  In _Conference on Biomimetic and Biohybrid Systems_ , 2017, pp. 95–106  Springer
  * [33] Gregory Hornby, Al Globus, Derek Linden and Jason Lohn  “Automated antenna design with evolutionary algorithms”  In _Space 2006_ pace 2006, 2006, pp. 7242
  * [34] Peter Cariani  “To evolve an ear. Epistemological implications of gordon pask’s electrochemical devices”  In _Systems research_ 10.3 Wiley Online Library, 1993, pp. 19–33
  * [35] Patricia A Vargas, Renan C Moioli, Fernando J Von Zuben and Phil Husbands  “Homeostasis and evolution together dealing with novelties and managing disruptions”  In _International Journal of Intelligent Computing and Cybernetics_ Emerald Group Publishing Limited, 2009
  * [36] William Ashby  “Design for a brain: The origin of adaptive behaviour”  Springer Science & Business Media, 2013
  * [37] James Dyke and Inman Harvey  “Hysteresis and the limits of homeostasis: from daisyworld to phototaxis”  In _European Conference on Artificial Life_ , 2005, pp. 241–251  Springer
  * [38] JG Dyke and IR Harvey  “Pushing up the daisies”, 2006
  * [39] Ezequiel A Di Paolo  “Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions”  In _From animals to animat_ 6 MIT Press, 2000, pp. 440–449
  * [40] Thierry Hoinville and Patrick Hénaff  “Comparative study of two homeostatic mechanisms in evolved neural controllers for legged locomotion”  In _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566)_ 3, 2004, pp. 2624–2629  IEEE
  * [41] John S Torday  “Homeostasis as the mechanism of evolution”  In _Biology_ 4.3 Multidisciplinary Digital Publishing Institute, 2015, pp. 573–590
  * [42] Mehdi Keramati and Boris Gutkin  “Collecting reward to defend homeostasis: A homeostatic reinforcement learning theory”  In _Biorxiv_ Cold Spring Harbor Laboratory, 2014, pp. 005140
  * [43] Tobias Morville, Karl Friston, Denis Burdakov, Hartwig R Siebner and Oliver J Hulme  “The homeostatic logic of reward”  In _bioRxiv_ Cold Spring Harbor Laboratory, 2018, pp. 242974
  * [44] Michael Levin, Alexis M Pietak and Johanna Bischof  “Planarian regeneration as a model of anatomical homeostasis: recent progress in biophysical and computational approaches”  In _Seminars in cell & developmental biology_ 87, 2019, pp. 125–144  Elsevier
  * [45] Yann LeCun, Yoshua Bengio and Geoffrey Hinton  “Deep learning”  In _Nature_ 521.7553 Nature Research, 2015, pp. 436–444
  * [46] Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Arshak Navruzyan, Nigel Duffy and Babak Hodjat  “Evolving Deep Neural Networks”  In _arXiv preprint arXiv:1703.00548_ , 2017
  * [47] Frederic Gruau and Darrell Whitley  “Adding learning to the cellular development of neural networks: Evolution and the Baldwin effect”  In _Evolutionary computation_ 1.3 MIT Press, 1993, pp. 213–233
  * [48] Josh C Bongard and Rolf Pfeifer  “Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny”  In _Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation_ , 2001, pp. 829–836  Morgan Kaufmann Publishers Inc.
  * [49] Kenneth O Stanley and Risto Miikkulainen  “Evolving neural networks through augmenting topologies”  In _Evolutionary computation_ 10.2 MIT Press, 2002, pp. 99–127
  * [50] Dario Floreano and Francesco Mondada  “Automatic creation of an autonomous agent: Genetic evolution of a neural network driven robot”  In _Proceedings of the third international conference on Simulation of adaptive behavior: From Animals to Animats 3_ , 1994, pp. 421–430  MIT Press
  * [51] Inman Harvey, Phil Husbands, Dave Cliff, Adrian Thompson and Nick Jakobi  “Evolutionary robotics: the Sussex approach”  In _Robotics and autonomous systems_ 20.2 Elsevier, 1997, pp. 205–224
  * [52] Josh C Bongard  “Evolutionary robotics”  In _Communications of the ACM_ 56.8 Acm, 2013, pp. 74–83
  * [53] K. Sims  “Evolving 3D morphology and behaviour by competition”  In _Artificial Life IV_ , 1994, pp. 28–39
  * [54] H. Lipson and J.. Pollack  “Automatic design and manufacture of artificial lifeforms”  In _Nature_ 406, 2000, pp. 974–978
  * [55] Nick Cheney, Robert MacCurdy, Jeff Clune and Hod Lipson  “Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding”  In _Proceedings of the Genetic and Evolutionary Computation Conference_ , 2013, pp. 167–174  Acm  URL: https://doi.org/10.1145/2463372.2463404
  * [56] Robert M French  “Catastrophic forgetting in connectionist networks”  In _Trends in cognitive sciences_ 3.4 Elsevier, 1999, pp. 128–135  URL: https://doi.org/10.1016/S1364-6613(99)01294-2
  * [57] Hod Lipson, Jordan B Pollack, Nam P Suh and P Wainwright  “On the origin of modular variation”  In _Evolution_ 56.8 BioOne, 2002, pp. 1549–1556
  * [58] KO Ellefsen, JB Mouret, J Clune and Josh C Bongard  “Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills”  In _PLoS Comput Biol_ 11.4, 2015, pp. e1004128
  * [59] Jeff Clune, Jean-Baptiste Mouret and Hod Lipson  “The evolutionary origins of modularity”  In _Proc. R. Soc. B_ 280, 2013, pp. 20122863  The Royal Society
  * [60] Carlos Espinosa-Soto and Andreas Wagner  “Specialization can drive the evolution of modularity”  In _PLoS Comput Biol_ 6.3 Public Library of Science, 2010, pp. e1000719
  * [61] Nadav Kashtan and Uri Alon  “Spontaneous evolution of modularity and network motifs”  In _Proceedings of the National Academy of Sciences of the United States of America_ 102.39 National Acad Sciences, 2005, pp. 13773–13778
  * [62] Sara Sabour, Nicholas Frosst and Geoffrey E Hinton  “Dynamic routing between capsules”  In _Advances in Neural Information Processing Systems_ , 2017, pp. 3859–3869
  * [63] Kai Olav Ellefsen, Jean-Baptiste Mouret and Jeff Clune  “Neural Modularity Reduces Catastrophic Forgetting”  In _The Evolution of Learning: Balancing Adaptivity and Stability in Artificial Agents_ Norwegian University of ScienceTechnology, 2014, pp. 111
  * [64] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu and Raia Hadsell  “Progressive neural networks”  In _arXiv preprint arXiv:1606.04671_ , 2016
  * [65] Sang-Woo Lee, Chung-Yeon Lee, Dong-Hyun Kwak, Jiwon Kim, Jeonghee Kim and Byoung-Tak Zhang  “Dual-Memory Deep Learning Architectures for Lifelong Learning of Everyday Human Behaviors.”  In _Ijcai_ , 2016, pp. 1669–1675
  * [66] Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel and Daan Wierstra  “Pathnet: Evolution channels gradient descent in super neural networks”  In _arXiv preprint arXiv:1701.08734_ , 2017
  * [67] Roby Velez and Jeff Clune  “Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks”  In _arXiv preprint arXiv:1705.07241_ , 2017
  * [68] Duc-Cuong Dang and Per Kristian Lehre  “Self-adaptation of mutation rates in non-elitist populations”  In _International Conference on Parallel Problem Solving from Nature_ , 2016, pp. 803–813  Springer
  * [69] Jason Teo, Asni Tahir, Norhayati Daut, Nordaliela Mohd Rusli and Norazlina Khamis  “Fixed vs. Self-Adaptive Crossover-First Differential Evolution”  In _Applied Mathematical Sciences_ 10.32 Hikari Ltd, 2016, pp. 1603–1610
  * [70] Joel Lehman, Jay Chen, Jeff Clune and Kenneth O Stanley  “Safe Mutations for Deep and Recurrent Neural Networks through Output Gradients”  In _arXiv preprint arXiv:1712.06563_ , 2017
  * [71] Leonardo Vanneschi, Mauro Castelli and Sara Silva  “A Survey of Semantic Methods in Genetic Programming”  In _Genetic Programming and Evolvable Machines_ 15.2 Hingham, MA, USA: Kluwer Academic Publishers, 2014, pp. 195–214
  * [72] Marcin Szubert, Anuradha Kodali, Sangram Ganguly, Kamalika Das and Josh C Bongard  “Semantic Forward Propagation for Symbolic Regression”  In _International Conference on Parallel Problem Solving from Nature_ , 2016, pp. 364–374  Springer
  * [73] Helmut Hauser, Auke J Ijspeert, Rudolf M Füchslin, Rolf Pfeifer and Wolfgang Maass  “Towards a theoretical foundation for morphological computation with compliant bodies”  In _Biological cybernetics_ 105.5-6 Springer, 2011, pp. 355–370  URL: https://doi.org/10.1007/s00422-012-0471-0
  * [74] Josh Bongard  “Morphological change in machines accelerates the evolution of robust behavior”  In _Proceedings of the National Academy of Sciences_ 108.4 National Acad Sciences, 2011, pp. 1234–1239
  * [75] Sam Kriegman, Nick Cheney and Josh Bongard  “How morphological development can guide evolution”  In _arXiv preprint arXiv:1711.07387_ , 2017
  * [76] Josh C Bongard, Anton Bernatskiy, Ken Livingston, Nicholas Livingston, John Long and Marc Smith  “Evolving Robot Morphology Facilitates the Evolution of Neural Modularity and Evolvability”  In _Proceedings of the 2015 on Genetic and Evolutionary Computation Conference_ , 2015, pp. 129–136  Acm
  * [77] Anton Bernatskiy and Josh Bongard  “Choice of robot morphology can prohibit modular control and disrupt evolution”  In _Proceedings of the 14th European Conference on Artificial Life_ , 2017, pp. 60–67  MIT Press
  * [78] Josh Bongard, Victor Zykov and Hod Lipson  “Resilient machines through continuous self-modeling”  In _Science_ 314.5802 American Association for the Advancement of Science, 2006, pp. 1118–1121
  * [79] Sara Lohmann, Jason Yosinski, Eric Gold, Jeff Clune, Jeremy Blum and Hod Lipson  “Aracna: An open-source quadruped platform for evolutionary robotics”  In _Artificial Life_ 13, 2012, pp. 387–392
  * [80] Dominik Belter, Piotr Skrzypczyñski, Krzysztof Walas and Donald Wlodkowic  “Affordable multi-legged robots for research and STEM education: a case study of design and technological aspects”  In _Progress in Automation, Robotics and Measuring Techniques_ Springer, 2015, pp. 23–34
  * [81] Robert T Schroer, Matthew J Boggess, Richard J Bachmann, Roger D Quinn and Roy E Ritzmann  “Comparing cockroach and Whegs robot body motions”  In _Robotics and Automation, 2004. Proceedings. ICRA’04. 2004 IEEE International Conference on_ 4, 2004, pp. 3288–3293  Ieee
  * [82] J-M Jehanno, Antoine Cully, Christophe Grand and J-B Mouret  “Design of a Wheel-Legged Hexapod Robot for Creative Adaptation”  In _Mobile Service Robotics_ World Scientific, 2014, pp. 267–276
  * [83] Yoo-Seok Kim, Gwang-Pil Jung, Haan Kim, Kyu-Jin Cho and Chong-Nam Chu  “Wheel transformer: A wheel-leg hybrid robot with passive transformable wheels”  In _IEEE Transactions on Robotics_ 30.6 Ieee, 2014, pp. 1487–1498
  * [84] Lukas Lichtensteiger and Peter Eggenberger  “Evolving the morphology of a compound eye on a robot”  In _Advanced Mobile Robots, 1999.(Eurobot’99) 1999 Third European Workshop on_ , 1999, pp. 127–134  Ieee  URL: https://ieeexplore.ieee.org/document/827631
  * [85] E Vaughan, Ezequiel A Di Paolo and I Harvey  “The evolution of control and adaptation in a 3D powered passive dynamic walker”  In _Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems, Artificial Life IX_ , 2004, pp. 139–145  URL: http://www.cs.uvm.edu/~jbongard/2015%5C%5FCS206/2004%5C%5FVaughan.pdf
  * [86] Eric Brown, Nicholas Rodenberg, John Amend, Annan Mozeika, Erik Steltz, Mitchell R Zakin, Hod Lipson and Heinrich M Jaeger  “Universal robotic gripper based on the jamming of granular material”  In _Proceedings of the National Academy of Sciences_ 107.44 National Acad Sciences, 2010, pp. 18809–18814  URL: https://www.pnas.org/content/107/44/18809
  * [87] Josh Bongard  “Morphological change in machines accelerates the evolution of robust behavior”  In _Proceedings of the National Academy of Sciences_ 108.4 National Academy Sciences, 2011, pp. 1234–1239  URL: https://www.pnas.org/content/108/4/1234
  * [88] Sam Kriegman, Stephanie Walker, Dylan Shah, Michael Levin, Rebecca Kramer-Bottiglio and Josh Bongard  “Automated shapeshifting for function recovery in damaged robots”  In _Proceedings of Robotics: Science and Systems_ , 2019  URL: http://www.roboticsproceedings.org/rss15/p28.pdf
  * [89] Ana Pervan and Todd Murphey  “Algorithmic materials: Embedding computation within material properties for autonomy”  In _Robotic Systems and Autonomous Platforms_ Woodhead Publishing, 2019, pp. 197–221  URL: http://www.sciencedirect.com/science/article/pii/B9780081022603000093
  * [90] Nicolas Y Masse, Gregory D Grant and David J Freedman  “Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization”  In _Proceedings of the National Academy of Sciences_ 115.44 National Academy Sciences, 2018, pp. E10467–e10475  URL: https://www.pnas.org/content/115/44/E10467
  * [91] Joshua Powers, Sam Kriegman and Josh Bongard  “The effects of morphology and fitness on catastrophic interference”  In _Artificial Life Conference Proceedings_ , 2018, pp. 606–613  MIT Press  URL: https://www.mitpressjournals.org/doi/pdf/10.1162/isal%5C%5Fa%5C%5F00111
  * [92] Valentino Braitenberg  “Vehicles: Experiments in synthetic psychology”  MIT press, 1986  URL: https://mitpress.mit.edu/books/vehicles
  * [93] David Hadka and Patrick Reed  “Borg: An Auto-Adaptive Many-Objective Evolutionary Computing Framework”  In _Evolutionary Computation_ 21.2, 2013, pp. 231–259  URL: https://ieeexplore.ieee.org/document/6793867
  * [94] Robert M French  “Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks”  In _Proceedings of the 13th annual cognitive science society conference_ , 1991, pp. 173–178  Erlbaum  URL: https://www.aaai.org/Papers/Symposia/Spring/1993/SS-93-06/SS93-06-007.pdf
  * [95] Anthony Robins  “Catastrophic forgetting, rehearsal and pseudorehearsal”  In _Connection Science_ 7.2 Taylor & Francis, 1995, pp. 123–146  URL: https://doi.org/10.1080/09540099550039318
  * [96] Xu He and Herbert Jaeger  “Overcoming Catastrophic Interference using Conceptor-Aided Backpropagation”  In _International Conference on Learning Representations_ , 2018  URL: https://openreview.net/pdf?id=B1al7jg0b
  * [97] Shawn LE Beaulieu, Sam Kriegman and Josh C Bongard  “Combating catastrophic forgetting with developmental compression”  In _Proceedings of the Genetic and Evolutionary Computation Conference_ , 2018, pp. 386–393  ACM  URL: https://dl.acm.org/citation.cfm?id=3205455.3205615
  * [98] Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu and Raia Hadsell  “Progress & Compress: A scalable framework for continual learning”  In _Proceedings of the 35th International Conference on Machine Learning_ 80 Pmlr, 2018, pp. 4528–4537  URL: http://proceedings.mlr.press/v80/schwarz18a/schwarz18a.pdf
  * [99] Michalis K Titsias, Jonathan Schwarz, Alexander G de G Matthews, Razvan Pascanu and Yee Whye Teh  “Functional Regularisation for Continual Learning”  In _arXiv preprint arXiv:1901.11356_ , 2019  URL: https://arxiv.org/abs/1901.11356
  * [100] Spyros Gidaris and Nikos Komodakis  “Dynamic few-shot visual learning without forgetting”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 4367–4375  URL: http://openaccess.thecvf.com/content%5C%5Fcvpr%5C%5F2018/papers/Gidaris%5C%5FDynamic%5C%5FFew-Shot%5C%5FVisual%5C%5FCVPR%5C%5F2018%5C%5Fpaper.pdf
  * [101] Rebecca K Kramer, Carmel Majidi and Robert J Wood  “Wearable tactile keypad with stretchable artificial skin”  In _IEEE International Conference on Robotics and Automation_ , 2011, pp. 1103–1107  Ieee  URL: https://ieeexplore.ieee.org/document/5980082
  * [102] Miriam Fend, Simon Bovet, Hiroshi Yokoi and Rolf Pfeifer  “An active artificial whisker array for texture discrimination”  In _IEEE/RSJ International Conference on Intelligent Robots and Systems_ 2, 2003, pp. 1044–1049  Ieee  URL: https://ieeexplore.ieee.org/document/1248782
  * [103] Yashraj S Narang, Alperen Degirmenci, Joost J Vlassak and Robert D Howe  “Transforming the Dynamic Response of Robotic Structures and Systems Through Laminar Jamming”  In _IEEE Robotics and Automation Letters_ 3.2 Ieee, 2018, pp. 688–695  URL: https://ieeexplore.ieee.org/document/8141952
  * [104] Miao Zhang, Huiqi Li, Shirui Pan, Xiaojun Chang, Zongyuan Ge and Steven W Su  “Differentiable neural architecture search in equivalent space with exploration enhancement”  In _Advances in Neural Information Processing Systems_ 33, 2020, pp. 6–12
  * [105] David Ha  “Reinforcement learning for improving agent design”  In _Artificial life_ 25.4 MIT Press, 2019, pp. 352–365
  * [106] Tamara G Kolda, Robert Michael Lewis and Virginia Torczon  “Optimization by direct search: New perspectives on some classical and modern methods”  In _SIAM review_ 45.3 SIAM, 2003, pp. 385–482
  * [107] Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters and Jürgen Schmidhuber  “Natural evolution strategies”  In _The Journal of Machine Learning Research_ 15.1 JMLR. org, 2014, pp. 949–980
  * [108] Kenneth V Price  “Differential evolution”  In _Handbook of optimization_ Springer, 2013, pp. 187–214
  * [109] David Hadka and Patrick Reed  “Borg: An auto-adaptive many-objective evolutionary computing framework”  In _Evolutionary computation_ 21.2 MIT Press, 2013, pp. 231–259
  * [110] Toni Giorgino  “Computing and visualizing dynamic time warping alignments in R: the dtw package”  In _Journal of statistical Software_ 31.7, 2009, pp. 1–24
  * [111] Christopher Rackauckas, Anshul Singhvi, Chris Graaf, Yingbo Ma, Michael Hatherly, Scott P. Jones, dextorious, Colin Caine, Elliot Saba, Julia TagBot, Kvaz1r, Max G, Sheehan Olver, The Gitter Badger, c123w and Maja Gwóźdź  “SciML/DifferentialEquations.jl: v6.17.1”  Zenodo, 2021  DOI: 10.5281/zenodo.4785349
  * [112] Peter Jager  “Cebrennus Simon, 1880 (Araneae: Sparassidae): a revisionary up-date with the description of four new species and an updated identification key for all species”  In _Zootaxa_ 3790.2, 2014, pp. 319–356  DOI: 10.11646/zootaxa.3790.2.4
  * [113] Sindya N. Bhanoo  “A Desert Spider With Astonishing Moves”  In _The New York Times_ , 2014  URL: https://www.nytimes.com/2014/05/06/science/a-desert-spider-with-astonishing-moves.html
  * [114] Rhodri H. Armour and Julian F.. Vincent  “Rolling in Nature and Robotics: A Review”  In _Journal of Bionic Engineering_ 3.4, 2006, pp. 195–208  DOI: 10.1016/s1672-6529(07)60003-1
  * [115] Huai-Ti Lin, Gary G. Leisk and Barry Trimmer  “GoQBot: a caterpillar-inspired soft-bodied rolling robot”  In _Bioinspiration & Biomimetics_ 6.2, 2011, pp. 026007  DOI: 10.1088/1748-3182/6/2/026007
  * [116] David Johan Christensen  “Evolution of shape-changing and self-repairing control for the ATRON self-reconfigurable robot”  In _Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on_ , 2006, pp. 2539–2545  Ieee
  * [117] Mark Yim, Wei-Min Shen, Behnam Salemi, Daniela Rus, Mark Moll, Hod Lipson, Eric Klavins and Gregory S Chirikjian  “Modular self-reconfigurable robot systems [grand challenges of robotics]”  In _IEEE Robotics & Automation Magazine_ 14.1 Ieee, 2007, pp. 43–52
  * [118] Christopher Parrott, Tony J Dodd and Roderich Groß  “HyMod: A 3-DOF Hybrid Mobile and Self-Reconfigurable Modular Robot and its Extensions”  In _Distributed Autonomous Robotic Systems_ Springer, 2018, pp. 401–414
  * [119] Chandana Paul, Francisco J Valero-Cuevas and Hod Lipson  “Design and control of tensegrity robots for locomotion”  In _IEEE Transactions on Robotics_ 22.5 Ieee, 2006, pp. 944–957
  * [120] Andrew P Sabelhaus, Jonathan Bruce, Ken Caluwaerts, Pavlo Manovi, Roya Fallah Firoozi, Sarah Dobi, Alice M Agogino and Vytas SunSpiral  “System design and locomotion of SUPERball, an untethered tensegrity robot”  In _2015 IEEE International Conference on Robotics and Automation (ICRA)_ , 2015, pp. 2867–2873  Ieee
  * [121] Ali Sadeghi, Alessio Mondini and Barbara Mazzolai  “Toward Self-Growing Soft Robots Inspired by Plant Roots and Based on Additive Manufacturing Technologies”  In _Soft Robotics_ 4.3, 2017, pp. 211–223  DOI: 10.1089/soro.2016.0080
  * [122] Shuhei Miyashita, Steven Guitron, Marvin Ludersdorfer, Cynthia R Sung and Daniela Rus  “An untethered miniature origami robot that self-folds, walks, swims, and degrades”  In _2015 IEEE International Conference on Robotics and Automation (ICRA)_ , 2015, pp. 1490–1496  Ieee
  * [123] Daniela Rus and Michael T Tolley  “Design, fabrication and control of origami robots”  In _Nature Reviews Materials_ 3 Nature Publishing Group, 2018, pp. 101–112
  * [124] Rolf Pfeifer, Max Lungarella and Fumiya Iida  “Self-Organization, Embodiment, and Biologically Inspired Robotics”  In _Science_ 318.5853, 2007, pp. 1088–1093  DOI: 10.1126/science.1145803
  * [125] Uluc Saranli, Martin Buehler and Daniel E Koditschek  “RHex: A simple and highly mobile hexapod robot”  In _The International Journal of Robotics Research_ 20.7 SAGE Publications, 2001, pp. 616–631
  * [126] Marc Raibert, Kevin Blankespoor, Gabriel Nelson and Rob Playter  “Bigdog, the rough-terrain quadruped robot”  In _IFAC Proceedings Volumes_ 41.2 Elsevier, 2008, pp. 10822–10825
  * [127] Scott Kuindersma, Robin Deits, Maurice Fallon, Andrés Valenzuela, Hongkai Dai, Frank Permenter, Twan Koolen, Pat Marion and Russ Tedrake  “Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot”  In _Autonomous Robots_ 40.3 Springer, 2016, pp. 429–455
  * [128] Auke Jan Ijspeert, Alessandro Crespi, Dimitri Ryczko and Jean-Marie Cabelguen  “From swimming to walking with a salamander robot driven by a spinal cord model”  In _Science_ 315.5817 American Association for the Advancement of Science, 2007, pp. 1416–1420
  * [129] Maoxun Li, Shuxiang Guo, Hideyuki Hirata and Hidenori Ishihara  “Design and performance evaluation of an amphibious spherical robot”  In _Robotics and Autonomous Systems_ 64 Elsevier, 2015, pp. 21–34
  * [130] Wan Cheol Myeong, Kwang Yik Jung, Sung Wook Jung, YH Jung and Hyun Myung  “Development of a drone-type wall-sticking and climbing robot”  In _Ubiquitous Robots and Ambient Intelligence (URAI), 2015 12th International Conference on_ , 2015, pp. 386–389  Ieee
  * [131] Richard J Bachmann, Frank J Boria, Ravi Vaidyanathan, Peter G Ifju and Roger D Quinn  “A biologically inspired micro-vehicle capable of aerial and terrestrial locomotion”  In _Mechanism and Machine Theory_ 44.3 Elsevier, 2009, pp. 513–526
  * [132] William RT Roderick, Mark R Cutkosky and David Lentink  “Touchdown to take-off: at the interface of flight and surface locomotion”  In _Interface focus_ 7.1 The Royal Society, 2017, pp. 20160094
  * [133] Moharam Habibnejad Korayem, Hami Tourajizadeh and Mahdi Bamdad  “Dynamic load carrying capacity of flexible cable suspended robot: robust feedback linearization control approach”  In _Journal of Intelligent & Robotic Systems_ 60.3-4 Springer, 2010, pp. 341–363
  * [134] Jiangping Li, Hongbin Ma, Chenguang Yang and Mengyin Fu  “Discrete-time adaptive control of robot manipulator with payload uncertainties”  In _Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2015 IEEE International Conference on_ , 2015, pp. 1971–1976  Ieee
  * [135] Antoine Cully, Jeff Clune, Danesh Tarapore and Jean-Baptiste Mouret  “Robots that can adapt like animals”  In _Nature_ 521.7553 Nature Publishing Group, 2015, pp. 503–507
  * [136] Konstantinos Chatzilygeroudis, Vassilis Vassiliades and Jean-Baptiste Mouret  “Reset-free trial-and-error learning for robot damage recovery”  In _Robotics and Autonomous Systems_ 100 Elsevier, 2018, pp. 236–250
  * [137] Andre Rosendo, Marco Atzigen and Fumiya Iida  “The trade-off between morphology and control in the co-optimized design of robots”  In _PLoS ONE_ 12.10, 2017, pp. e0186107  DOI: 10.1371/journal.pone.0186107
  * [138] Martin Garrad, Jonathan Rossiter and Helmut Hauser  “Shaping Behavior With Adaptive Morphology”  In _IEEE Robotics and Automation Letters_ 3.3, 2018, pp. 2056–2062  DOI: 10.1109/lra.2018.2807591
  * [139] Helmut Hauser  “Resilient machines through adaptive morphology”  In _Nature Machine Intelligence_ , 2019, pp. 1  DOI: 10.1038/s42256-019-0076-6
  * [140] Sehyuk Yim and Metin Sitti  “Shape-programmable soft capsule robots for semi-implantable drug delivery”  In _IEEE Transactions on Robotics_ 28.5 Ieee, 2012, pp. 1198–1202
  * [141] Dylan S Shah, Michelle Ching-Sum Yuen, Liana G Tilton, Ellen J Yang and Rebecca Kramer-Bottiglio  “Morphing Robots Using Robotic Skins That Sculpt Clay”  In _IEEE Robotics and Automation Letters_ 4 Ieee, 2019, pp. 2204–2211
  * [142] Dae-Young Lee, Sa-Reum Kim, Ji-Suk Kim, Jae-Jun Park and Kyu-Jin Cho  “Origami Wheel Transformer: A Variable-Diameter Wheel Drive Robot Using an Origami Structure”  In _Soft Robotics_ 4.2, 2017, pp. 163–180  DOI: 10.1089/soro.2016.0038
  * [143] Jonathan Hiller and Hod Lipson  “Dynamic simulation of soft multimaterial 3d-printed objects”  In _Soft Robotics_ 1.1 Mary Ann Liebert, Inc., NY 10801 USA, 2014, pp. 88–101
  * [144] Nick Jakobi, Phil Husbands and Inman Harvey  “Noise and the reality gap: The use of simulation in evolutionary robotics”  In _European Conference on Artificial Life_ , 1995, pp. 704–720  Springer
  * [145] Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun and Marco Hutter  “Learning agile and dynamic motor skills for legged robots”  In _Science Robotics_ 4.26, 2019, pp. eaau5872  DOI: 10.1126/scirobotics.aau5872
  * [146] Jonathan Hiller and Hod Lipson  “Automatic design and manufacture of soft robots”  In _IEEE Transactions on Robotics_ 28.2 Ieee, 2012, pp. 457–466
  * [147] Sylvain Koos, Jean-Baptiste Mouret and Stéphane Doncieux  “The transferability approach: Crossing the reality gap in evolutionary robotics”  In _IEEE Transactions on Evolutionary Computation_ 17.1 Ieee, 2013, pp. 122–145
  * [148] Nicholas W Bartlett, Michael T Tolley, Johannes TB Overvelde, James C Weaver, Bobak Mosadegh, Katia Bertoldi, George M Whitesides and Robert J Wood  “A 3D-printed, functionally graded soft robot powered by combustion”  In _Science_ 349.6244 American Association for the Advancement of Science, 2015, pp. 161–165
  * [149] Andrei A. Rusu, Matej Večerík, Thomas Rothörl, Nicolas Heess, Razvan Pascanu and Raia Hadsell  “Sim-to-Real Robot Learning from Pixels with Progressive Nets” ISSN: 2640-3498  In _Conference on Robot Learning_ PMLR, 2017, pp. 262–270  URL: http://proceedings.mlr.press/v78/rusu17a.html
  * [150] Y. Chebotar, A. Handa, V. Makoviychuk, M. Macklin, J. Issac, N. Ratliff and D. Fox  “Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience”  In _2019 International Conference on Robotics and Automation (ICRA)_ , 2019, pp. 8973–8979  DOI: 10.1109/icra.2019.8793789
  * [151] Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba and Pieter Abbeel  “Sim-to-real transfer of robotic control with dynamics randomization”  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , 2018, pp. 1–8  Ieee
  * [152] Melanie Mitchell, John H. Holland and Stephanie Forrest  “When will a Genetic Algorithm Outperform Hill Climbing”  In _Advances in Neural Information Processing Systems 6_ Morgan-Kaufmann, 1994, pp. 51–58  URL: http://papers.nips.cc/paper/836-when-will-a-genetic-algorithm-outperform-hill-climbing.pdf
  * [153] Joran W. Booth, Dylan Shah, Jennifer C. Case, Edward L. White, Michelle C. Yuen, Olivier Cyr-Choiniere and Rebecca Kramer-Bottiglio  “OmniSkins: Robotic skins that turn inanimate objects into multifunctional robots”  In _Science Robotics_ 3.22, 2018, pp. eaat1853  DOI: 10.1126/scirobotics.aat1853
  * [154] S.. Felton, M.. Tolley, C.. Onal, D. Rus and R.. Wood  “Robot self-assembly by folding: A printed inchworm robot”  In _2013 IEEE International Conference on Robotics and Automation_ , 2013, pp. 277–282  DOI: 10.1109/icra.2013.6630588
  * [155] D. Lee, S. Kim, Y. Park and R.. Wood  “Design of centimeter-scale inchworm robots with bidirectional claws”  In _2011 IEEE International Conference on Robotics and Automation_ , 2011, pp. 3197–3204  DOI: 10.1109/icra.2011.5980458
  * [156] J.. Booth, J.. Case, E.. White, D.. Shah and R. Kramer-Bottiglio  “An addressable pneumatic regulator for distributed control of soft robots”  In _2018 IEEE International Conference on Soft Robotics (RoboSoft)_ , 2018, pp. 25–30  DOI: 10.1109/robosoft.2018.8404892
  * [157] Sang Yup Kim, Robert Baines, Joran Booth, Nikolaos Vasios, Katia Bertoldi and Rebecca Kramer-Bottiglio  “Reconfigurable soft body trajectories using unidirectionally stretchable composite laminae”  In _Nature Communications_ 10.1, 2019, pp. 3464  DOI: 10.1038/s41467-019-11294-7
  * [158] David Howard, Agoston E. Eiben, Danielle Frances Kennedy, Jean-Baptiste Mouret, Philip Valencia and Dave Winkler  “Evolving embodied intelligence from materials to machines”  In _Nature Machine Intelligence_ 1.1, 2019, pp. 12–19  DOI: 10.1038/s42256-018-0009-9
  * [159] Gabor Soter, Andrew Conn, Helmut Hauser and Jonathan Rossiter  “Bodily Aware Soft Robots: Integration of Proprioceptive and Exteroceptive Sensors”  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , 2018, pp. 2448–2453  DOI: 10.1109/icra.2018.8463169
  * [160] Takuya Umedachi, Takeshi Kano, Akio Ishiguro and Barry A. Trimmer  “Gait control in a soft robot by sensing interactions with the environment using self-deformation”  In _Open Science_ 3.12, 2016, pp. 160766  DOI: 10.1098/rsos.160766
  * [161] Francesco Corucci, Nick Cheney, Francesco Giorgio-Serchi, Josh Bongard and Cecilia Laschi  “Evolving Soft Locomotion in Aquatic and Terrestrial Environments: Effects of Material Properties and Environmental Transitions”  In _Soft Robotics_ 5.4, 2018, pp. 475–495  DOI: 10.1089/soro.2017.0055
  * [162] Robert Baines, Simon Freeman, Frank Fish and Rebecca Kramer  “Variable stiffness morphing limb for amphibious legged robots inspired by chelonian environmental adaptations”  In _Bioinspiration & Biomimetics_ 15.2, 2020, pp. 025002  DOI: 10.1088/1748-3190/ab68e8
  * [163] Kenneth Ward Church  “Word2Vec”  In _Natural Language Engineering_ 23.1 Cambridge University Press, 2017, pp. 155–162
  * [164] David Matthews, Sam Kriegman, Collin Cappelle and Josh Bongard  “Word2vec to behavior: morphology facilitates the grounding of language in machines.”  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2019, pp. 4153–4160  IEEE
  * [165] Paul Fitzpatrick, Giorgio Metta, Lorenzo Natale, Sajit Rao and Giulio Sandini  “Learning about objects through action-initial steps towards artificial cognition”  In _2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422)_ 3, 2003, pp. 3140–3145  IEEE
  * [166] Giorgio Metta, Francesco Panerai, Riccardo Manzotti and Giului Sandini  “Babybot: an artificial developing robotic agent”  In _Proc. Sixth Int. Conf. on the Simulation of Adaptive Behaviors (SAB 2000), Paris, France_ , 2000
  * [167] Ankur M Mehta, Joseph DelPreto, Kai Weng Wong, Scott Hamill, Hadas Kress-Gazit and Daniela Rus  “Robot creation from functional specifications”  In _Robotics Research_ Springer, 2018, pp. 631–648
  * [168] Xiaobin Zhang, Yinhao Zhu and Hai Lin  “Performance guaranteed human-robot collaboration through correct-by-design”  In _2016 American Control Conference (ACC)_ , 2016, pp. 6183–6188  IEEE
  * [169] Stuart Bennett  “Development of the PID controller”  In _IEEE Control Systems Magazine_ 13.6 IEEE, 1993, pp. 58–62
  * [170] Morgan T Gillespie, Charles M Best, Eric C Townsend, David Wingate and Marc D Killpack  “Learning nonlinear dynamic models of soft robots for model predictive control with neural networks”  In _2018 IEEE International Conference on Soft Robotics (RoboSoft)_ , 2018, pp. 39–45  IEEE
  * [171] Cosimo Della Santina, Robert K Katzschmann, Antonio Bicchi and Daniela Rus  “Soft Robotic Modeling and Control: Bringing Together Articulated Soft Robots and Soft-Bodied Robots”  SAGE Publications Sage UK: London, England, 2021
  * [172] Javier Tapia, Espen Knoop, Mojmir Mutnỳ, Miguel A Otaduy and Moritz Bächer  “Makesense: Automated sensor design for proprioceptive soft robots”  In _Soft robotics_ 7.3 Mary Ann Liebert, Inc., publishers 140 Huguenot Street, 3rd Floor New …, 2020, pp. 332–345
  * [173] Guirec Maloisel, Espen Knoop, Christian Schumacher and Moritz Bächer  “Automated routing of muscle fibers for soft robots”  In _IEEE Transactions on Robotics_ 37.3 IEEE, 2021, pp. 996–1008
  * [174] Hod Lipson  “Challenges and opportunities for design, simulation, and fabrication of soft robots”  In _Soft Robotics_ 1.1, 2014, pp. 21–27  URL: https://www.liebertpub.com/doi/10.1089/soro.2013.0007
  * [175] Rolf Pfeifer and Josh Bongard  “How the body shapes the way we think: a new view of intelligence”  MIT press, 2006
  * [176] Fred Adams  “Embodied cognition”  In _Phenomenology and the Cognitive Sciences_ 9.4 Springer, 2010, pp. 619–628
  * [177] Vincent C Müller and Matej Hoffmann  “What is morphological computation? On how the body contributes to cognition and control”  In _Artificial life_ 23.1 MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info …, 2017, pp. 1–24
  * [178] František Baluška and Michael Levin  “On having no head: cognition throughout biological systems”  In _Frontiers in psychology_ 7 Frontiers, 2016, pp. 902
  * [179] Lauren E Gillespie, Gabriela R Gonzalez and Jacob Schrum  “Comparing direct and indirect encodings using both raw and hand-designed features in tetris”  In _Proceedings of the Genetic and Evolutionary Computation Conference_ , 2017, pp. 179–186
  * [180] Peter Eggenberger Hotz  “Comparing direct and developmental encoding schemes in artificial evolution: A case study in evolving lens shapes”  In _Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753)_ 1, 2004, pp. 752–757  IEEE
  * [181] Yohannes Kassahun, Mark Edgington, Jan Hendrik Metzen, Gerald Sommer and Frank Kirchner  “A common genetic encoding for both direct and indirect encodings of networks”  In _Proceedings of the 9th annual conference on Genetic and evolutionary computation_ , 2007, pp. 1029–1036
  * [182] Hod Lipson  “Principles of modularity, regularity, and hierarchy for scalable systems”  In _Journal of Biological Physics and Chemistry_ 7.4 Citeseer, 2007, pp. 125
  * [183] Jeff Clune, Kenneth O Stanley, Robert T Pennock and Charles Ofria  “On the performance of indirect encoding across the continuum of regularity”  In _IEEE Transactions on Evolutionary Computation_ 15.3 IEEE, 2011, pp. 346–367
  * [184] Kohei Nakajima, Helmut Hauser, Tao Li and Rolf Pfeifer  “Information processing via physical soft body”  In _Scientific reports_ 5.1 Nature Publishing Group, 2015, pp. 1–11
  * [185] Russ Tedrake, Teresa Weirui Zhang, Ming-fai Fong and H Sebastian Seung  “Actuating a simple 3D passive dynamic walker”  In _IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004_ 5, 2004, pp. 4656–4661  IEEE
  * [186] Steve Collins, Andy Ruina, Russ Tedrake and Martijn Wisse  “Efficient bipedal robots based on passive-dynamic walkers”  In _Science_ 307.5712 American Association for the Advancement of Science, 2005, pp. 1082–1085
  * [187] Helmut Hauser, Rudolf Marcel Fuechslin and Kohei Nakajima  “Morphological Computation: The Body as a Computational Resource”  In _Opinions and Outlooks on Morphological Computation_ Self-published, 2014, pp. 226–244
  * [188] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon and Stan Birchfield  “Training deep networks with synthetic data: Bridging the reality gap by domain randomization”  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , 2018, pp. 969–977
  * [189] Nicols Cruz and Javier Ruiz-del-Solar  “Closing the simulation-to-reality gap using generative neural networks: Training object detectors for soccer robotics in simulation as a case study”  In _2020 International Joint Conference on Neural Networks (IJCNN)_ , 2020, pp. 1–8  IEEE
  * [190] Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira and Stan Birchfield  “Structured domain randomization: Bridging the reality gap by context-aware synthetic data”  In _2019 International Conference on Robotics and Automation (ICRA)_ , 2019, pp. 7249–7255  IEEE
  * [191] Michael Lomnitz, Zigfried Hampel-Arias, Nina Lopatina and Felipe A Mejia  “A general approach to bridge the reality-gap”  In _arXiv preprint arXiv:2009.01865_ , 2020
  * [192] Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell and Konstantinos Bousmalis  “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks”  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 12627–12637
  * [193] Joshua Powers, Ryan Grindle, Sam Kriegman, Lapo Frati, Nick Cheney and Josh Bongard  “Morphology dictates learnability in neural controllers”  In _Artificial Life Conference Proceedings_ , 2020, pp. 52–59  MIT Press One Rogers Street, Cambridge, MA 02142-1209 USA journals-info mit
  * [194] Joshua Powers, Ryan Grindle, Lapo Frati and Josh Bongard  “A good body is all you need: avoiding catastrophic interference via agent architecture search”  In _arXiv preprint arXiv:2108.08398_ , 2021
  * [195] Dylan S Shah, Joshua P Powers, Liana G Tilton, Sam Kriegman, Josh Bongard and Rebecca Kramer-Bottiglio  “A soft robot that adapts to environments through shape change”  In _Nature Machine Intelligence_ 3.1 Nature Publishing Group, 2021, pp. 51–59