Dynamics of Darwinian versus Baldwinian versus Lamarckian evolution

Kristoffer Reinholt Thomsen & Steen Rasmussen

FKF
University of Southern Denmark
April 2023

###### Abstract

In recent years, studies in epigenetic inheritance in biological systems as
well as studies on evolution in non-biological systems e.g., machine learning
and robotics, have reopened the discussion of non-Darwinian methods of
evolutionary optimization. In this paper, the three most prominent classical
evolutionary strategies Lamarckian, Darwinian, and Baldwinian are implemented
and compared in an agent-based simulation. The dynamics of optimization and
learning are studied in constant as well as in dynamic environments, both as
single evolutionary strategy populations and as mixed evolutionary strategy
populations. The three different agent types are implemented as simple objects
that need to gather resources to replicate. The agent fitness is defined
through a bitstring match between the agent and the environment and to
replicate each agent needs to gather sufficient resources in the simulated
chemostat. Initially assuming constant replication costs for all strategies,
it is not surprising that for single strategy environments the Darwinian
strategy is superior (most quickly develops high fitness) for high learning
costs, while the Lamarckian strategy dominates for low learning costs with the
Baldwinian strategy placed in between. As expected, the Lamarckian strategy is
also performing best in ‘fast’ varying environments, followed by the
Baldwinian strategy with the Darwinian last. The opposite is true for ‘slowly’
varying environments, as the learning costs become a burden. To make a more
’fair’ comparison between the three different evolutionary methods when they
all interact, we can assume that both the ability to learn within an
individual’s lifetime and the learning process itself require additional
resources compared to an evolutionary process without lifetime learning. When
all three strategies coexist and we vary the replication costs for the
different strategies we find the following general pattern: The evolutionary
advantage decreases for Lamarckian and Baldwinian strategies as their learning
costs increase and/or as their relative replication costs increase compared to
the Darwinian strategies, and in most situations the Baldwinian strategy
performs in between the Darwinian and Lamarckian strategies. Finally, as our
investigations emphasize the superiority of the Lamarckian strategy in most
cases with low learning cost, it is interesting to note that a corresponding
biological inheritance mechanism did not evolve, while the Lamarckian strategy
is currently extensively used in technology applications.

###### Contents

  1. 1 Background
    1. 1.1 A Brief Historical Review of Lamarck’s, Darwin’s, and Baldwin’s fundamental ideas
      1. 1.1.1 Lamarck
      2. 1.1.2 Darwin
      3. 1.1.3 Baldwin
    2. 1.2 A Brief Summary of Modern Evolutionary and Learning Studies
      1. 1.2.1 Evolution
      2. 1.2.2 Cognition and Learning
      3. 1.2.3 Perspectives from the Past and the Future
  2. 2 Methods
    1. 2.1 General agent characteristics
    2. 2.2 Environment and Fitness
    3. 2.3 Darwinian agents
    4. 2.4 Lamarckian agents
    5. 2.5 Baldwinian agents
    6. 2.6 Temporally dependent Environment
  3. 3 Results
    1. 3.1 Mono agent type population fitness in constant environments
      1. 3.1.1 Comparison of mono agent type populations: constant environment
    2. 3.2 Mono agent type population fitness in temporally varying environment
    3. 3.3 Pairwise agent population competition
    4. 3.4 Three-way agent population competition
  4. 4 Discussion
  5. 5 Conclusions

##  1 Background

In this paper we use simulation to explore and compare evolutionary and
learning strategies originally proposed by Lamarck, Darwin, and Baldwin.
Because of the historical significance of the Lamarckian, Darwinian, and
Baldwinian ideas, we start with a brief historical review in Section 1.1. In
Section 1.2 we summarize some of the modern ideas on evolutionary and learning
as well as compares and contrasts ideas from the past with perspectives for
the future. In Section 2 we introduced the agent based simulation methods
used, while Section 3 is devoted to the results obtained when comparing and
contrasting Lamarckian, Darwinian, and Baldwinian evolutionary and learning
strategies. Section 4 is devoted to a critical discussion of the results
obtained, while Section 5 summarizes our findings.

###  1.1 A Brief Historical Review of Lamarck’s, Darwin’s, and Baldwin’s
fundamental ideas

####  1.1.1 Lamarck

Jean-Baptiste Lamarck (1744-1829) was a French naturalist and professor of
zoology, who in his classical work Philosophie Zoologique (1809) stated his
formulation of a theory of biological evolution. Lamarck was a strong
proponent of the naturalistic view of evolution and life as a process best
explained by natural and universal laws indistinguishable from the laws
governing the inorganic world [10, pp. 8-9][29, pp. 9-15]. Through his study
of zoology and paleontology, he also held a firm view of species being mutable
in time, and all species, including man, as having descended from other
species [10, pp. 8-9]. Lamarck was a pioneer in the study of simple organisms,
a field he felt was neglected by his peers, and he even coined the still-used
classification term invertebrate while working as ”Curator of Insecta and
Vermes” at the Muséum National d’Histoire Naturelle [2]. Much of Philosophie
Zoologique is dedicated to explaining his philosophy behind classification of
organisms.

Lamarck is credited with formulating the first consistent theory of biological
evolution which covered many levels of description in biology, ranging from
simple organisms to cognition. However, he is mostly known for formulating two
laws to which he attributed the modifications of the body structures of
animals. He summarizes both laws as follows in [29, p. 113]. The first law
states:

> In every animal which has not passed the limits of its development, a more
> frequent and continuous use of any organ gradually strengthens, develops and
> enlarges that organ, and gives it a power proportional to the length of time
> it has been used; while the permanent disuse of any organ imperceptibly
> weakens and deteriorates it, and progressively diminishes its functional
> capacity, until it finally disappears.

This first law is Lamarck’s famous law of use and disuse, which he saw as the
channel of biological adaption. The second law states:

> All the acquisitions or losses wrought by nature on individuals, through the
> influence of the environment in which their race has been placed, and hence
> through the influence of the predominant use or permanent disuse of any
> organ; All these are preserved by reproduction to the new individuals which
> arise, provided that the acquired modifications are common to both sexes, or
> at least to the individuals which produced the young.

The second law is what later has been condensed to the law of inheritance of
acquired traits. This is the inference for which Lamarck in modern times has
been remembered, though not always fondly, since Lamarck’s theory of
inheritance was deemed incompatible after Mendel discovered genetic
inheritance, and as science later discovered DNA as the molecular base for
inheritance. Hereafter, the theory of Natural Selection proposed by Darwin
progressively became the most supported theory of evolution among scientists.

In present times, however, Lamarck’s ideas of evolution have again become
relevant, both in biological and non-biological evolving systems. Epigenetic
inheritance has reopened the discussion in biological systems, as such factors
have been observed to allow transmission of different phenotypes from cells
with identical genotypes [25]. In non-biological systems, cultural evolution
and evolving computational programs in the fields of artificial intelligence
and artificial life can be seen as Lamarckian in nature, as evolution in such
systems is often not solely driven by replication and random mutations.
Furthermore, in non-biological evolution, replicating entities may be ill
defined or the evolving entities may not even replicate in a Darwinian manner
as we shall explore in the following sections.

####  1.1.2 Darwin

The Origin of Species, one of the most influential scientific writing ever
written, was published in 1859 by Charles Darwin. As much has been written
about Darwin’s works, the following section is highly influenced by the
English Wikipedia page on On the Origin of Species, as this is possibly the
most read text based on the original book, perhaps only second to the book
itself.

Darwin included both evidence that he had collected on the Beagle expedition
in the 1830s and his subsequent findings from research, correspondence, and
experimentation. As historical context it should be noted that Charles
Darwin’s grandfather Erasmus Darwin outlined a hypothesis of transmutation of
species in the 1790s; an idea Lamarck developed further in Philosophie
Zoologique [29]. Both envisaged that spontaneous generation produced simple
forms of life that progressively developed greater complexity, adapting to the
environment by inheriting changes in adults caused by use or disuse. In his
notes, Charles Darwin sketched a genealogical branching of a single
evolutionary tree, discarding Lamarck’s independent lineages progressing to
higher forms. The central idea in Darwin’s thinking is expressed in the
following quote:

> As many more individuals of each species are born than can possibly survive;
> and as, consequently, there is a frequently recurring struggle for
> existence, it follows that any being, if it vary however slightly in any
> manner profitable to itself, under the complex and sometimes varying
> conditions of life, will have a better chance of surviving, and thus be
> naturally selected. From the strong principle of inheritance, any selected
> variety will tend to propagate its new and modified form. [10, pp. 30-31]

In On the Origins of Species, Darwin discusses the production of variation
within species induced by selective breeding, and that this is central to
understanding the natural world, where such variation is naturally,
ubiquitously present [10, chap. 1].

In Darwin’s time there was no agreed-upon model of heredity. He accepted a
version of the inheritance of acquired characteristics (which after Darwin’s
death came to be called Lamarckism), and in Chapter V of On the Origins of
Species, Darwin discusses what he called the effects of use and disuse; he
wrote:

> I think there can be little doubt that use in our domestic animals has
> strengthed and enlarged certain parts, and disuse diminished them; and that
> such modifications are inherited. Under free nature we have no standard of
> comparison by which to judge of the effects of long-continued use or disuse,
> for we know not the parent-forms; but many animals possess structures which
> can be best explained by effects of disuse. [10, p. 235]

From this, it seems clear that Darwin did not think his theory of Natural
Selection was a complete theory, and that phenotypic changes during the
lifetimes of organisms were better explained by Lamarck’s theory. Still,
evolutionary research in the following centuries has been marked by competing
Darwinian and Lamarckian factions. Building on Darwin’s theory of Natural
Selection, James Mark Baldwin later introduced a new factor, attempting to
explain these ongoing phenotypic changes in organisms.

####  1.1.3 Baldwin

In his 1896 paper A New Factor in Evolution [6], the American philosopher and
psychologist James Mark Baldwin argued that the Lamarckian idea of inheritance
of acquired traits is redundant in evolution, as intelligence can explain the
plasticity of the phenotype. Baldwin’s main argument focused on the role of
intelligence in organisms as a means of plastically adapting to their
environments during their lifetimes, which gives the ability to increase
survival and chance of reproduction. He was partly inspired by observing the
natural mental development of his children, and his stepwise theory of
cognitive development was a major influence on the later, and much more widely
known, developmental theory of Jean Piaget [3, 9]. Baldwin argued that:

> The most plastic individuals would be preserved to do the advantageous
> things for which their variations show them to be the most fit, and the next
> generation would show an emphasis on just this direction in its variations
> [6, p. 447]

Baldwin’s ideas were formulated within the Darwinian framework as he knew they
had to be consistent with natural selection. Though he was hesitant to enter
the field of theoretical biology, Baldwin proposed that his theory could
explain discontinuities in fossil records [6].

As the popularity of Lamarckism diminished, Baldwins ideas of phenotypic
plasticity surfaced. A review of the role of phenotypic plasticity in
evolution can be found in [21] by Fusco and Minelli, where phenotypic
plasticity is defined as the ability of individual genotypes to produce
different phenotypes when exposed to different environmental conditions. This
includes the ability of an individual organism to change its phenotypic state
or activity (e.g. its metabolism) in response to variations in environmental
conditions. The current iteration of Baldwins new factor - the Baldwin effect
- has been largely accepted as a present factor in evolution [1].

###  1.2 A Brief Summary of Modern Evolutionary and Learning Studies

####  1.2.1 Evolution

The molecular mechanisms of inheritance were still unknown in the time of
Lamarck, Darwin, and Baldwin. After the rise of Mendelian genetics and the
discovery of the distinction between the germ line and the somatic line, and
later the discovery of DNA, Lamarck’s theory was declared incompatible with
biological evolution, as phenotypic changes during an organism’s lifetime had
no known means of being transmitted though inheritance [12, p. 322].
Therefore, Darwin’s idea of natural selection became the dominant theory of
biological evolution. However, the study of epigenetics has since revived the
fundamental ideas of Lamarckism in the field of biological evolution [25].

Modern theoretical and computational studies of biological evolution use
multiple mathematical tools including game theory [42, 30]; systems of
ordinary differential equations such as Manfred Eigen’s Quasi-species and
Hypercycle models [13, 14]; Stuart Kaufmann’s NK model, which defines a rugged
combinatorial fitness state space [27]; mathematical optimization, e.g., in
John Holland’s Genetic Algorithms [24]; random graphs [38]; cellular automata
[32]; information theory [45]; statistical models [18]; and agent based
simulations [37, 16, 28, 5, 20, 44, 33]. The Artificial Life and related
communities have been deeply engaged in these developments. See e.g., the
Artificial Life and European Conference for Artificial Life Proceedings from
the onset of conferences in 1987 (and proceedings in 1989) through today
(2022).

Though evolution traditionally is mainly perceived as referring to biological
evolution, the term is applicable to a much wider set of systems. The
evolutionary framework is useful for describing any system in which entities
are replicated or reproduced. For example, human made technologies also
undergo evolutionary processes [19, 43]. So far, mainly the evolution in
physical technologies has been explored quantitatively. Physical technologies
are the tools we use for transforming energy and materials in pursuit of our
goals. However, social technologies - tools for organizing people in pursuit
of our goals i.e., laws, moral values, money, religion, government, or
business - also undergo evolution. Importantly, our physical and social
technologies interact and undergo coevolution [15].

Cultural and technological evolution happens at a much shorter timescale than
biological evolution, where human evolution occurs over thousands of years,
while human culture, in particular human technology, may change over tens of
years or even faster. Emergence of social and physical technologies such as
the grammatical language, the written word, and the internet have increased
the transmission scale, speed and accuracy between human agents over time.
This ever-increasing communication and coordination capability is likely the
main underlying driver of humanity’s increasing fitness over the last 200,000+
years [41].

####  1.2.2 Cognition and Learning

It is obvious that an organism’s ability to react in response to environmental
changes has a potential for enhancing its survival. Some single celled
organisms have the ability to regulate the direction of their motility in
response to concentration gradients and thereby move towards locations with
more metabolic resources [46]. Therefore, intracellular information processing
and simple decision-making already occur at the single cellular level.

The neuron emerged as a specialized cell type capable of fast signal
transmission both locally and across the entire organism. In some organisms a
localized development of neuron-based information processing of sensory inputs
eventually evolved into a central nervous system (CNS) and a brain. A defining
feature of the brain is its capability of learning as a brain allows an
organism to adaptively change its behaviors. The brain is plastic, as neural
(and presumably intraneural) connections change during the lifetime of the
organism.

Can evolutionary processes be found in the neural networks of the brain? After
all, the brain is a complex adaptive system that relies on optimization for
continually and plastically improving the fitness of itself. A challenge for
applying the evolutionary idea for describing the brain is the non-replicating
neuron. In this context an interesting model using Darwinian channels in the
brain has been proposed by Fernando et al. [17] that instead considers
multiple groups of neurons that can act in parallel. The model outlines a
mechanism for copying patterns of synaptic connections between different
neuronal groups, where one group forms connections to another group, and
”hijacks” it to make a copy of its synaptic topology. Thereby an existing
neural connectivity can be copied (”replicated”) into a nearby neural group.
With a sufficient number of such groups operating in parallel, the system
composes a ”population” of neural groups, where the groups then define the
units of selection. Thus, Darwinian-style natural selection might also occur
in the brain [17].

It can be argued that the brain is fit for analysis using multiple (or even
all) of the previously mentioned classical evolutionary theories; Lamarckism,
Darwinism, and Baldwin’s new factor. The brain clearly exhibits Baldwinian
characteristics, as its development, function, and microscopic structure,
which are all phenotypic dimensions, are plastic. It is also clear that
cultural evolution affects the human brain, as the behavior (which is
generated by the brain) is highly influenced by verbal thoughts (thereby
language), learned morals, and many other culturally transmitted phenomena. It
has been argued that cultural evolution is partly Lamarckian [11] making a
full description of the brain partly Lamarckian, though this is not generally
agreed upon [23]. Finally, it is clear from the evolutionary history of the
development of the brain that genetic factors have a large influence on the
structure of the brain [31, 22, 26], making the evolution of the brain
Darwinian. Darwinian mechanisms may also be relevant for describing
neuroplasticity, as proposed by Fernando et al. (2012) [17].

####  1.2.3 Perspectives from the Past and the Future

Historically, evolution has been associated with generational changes while
learning has been associated with changes within the lifespan of an individual
[4]. This distinction makes sense as biological evolution has developed a
material mechanism that ensures inheritance of traits from parent to the
offspring based on genetic information. It should be noted that biological
evolution apparently has not developed a material mechanism that ensures
inheritance to offspring of behavior learned in the lifetime of a parent.

Biological evolution through Darwinian channels happens discretely through new
material reproduction, whereas learning may happen continually throughout the
entire lifespan of an organism through reorganization of an existing material
basis within the organism. For higher order biological organisms this material
reorganization happens within the brain, while for plants and some
microorganisms this material reorganization occurs through distributed
processes. In any event a hard distinction between evolution and learning is
dissolved for systems undergoing Lamarckian and Baldwinian evolution, where
adaption happens outside of replication/reproduction.

Evolution, biological as well as non-biological evolution, is sometimes an
optimization processes, where traits become better and better suited to the
environment, while qualitatively new traits do not emerge. However,
qualitatively new traits do also emerge during evolution although it is not
well understood how and under which conditions this occurs. Therefore, current
research on evolutionary processes is preoccupied with understanding the
hallmarks [8, 7] and mechanisms for open-endedness in such processes [34].
Thus, evolutionary processes come in two flavors [39]: optimization processes
and expansion processes. The former suffices in systems whose size and
interactions do not change substantially over time, while the latter is a key
property of open-ended evolution, where new entities and interaction types
enter the game. Examining data from systems in physics, biology, and
engineering, the authors in [39] argue that the evolutionary optimization
dynamics are the cumulative effect of multiple independent events, or quakes
in these systems. These quakes are uniformly distributed on a logarithmic time
scale and produce a decelerating fitness improvement when using the
appropriate independent variable, which is a signature of evolutionary
optimization. Finally, they argue that to enhance the evolutionary richness in
a system that only optimizes, the system has to be enriched with new
components and thereby interactions to expand its dynamics.

A fundamental similarity between evolution and learning is that both can be
characterized as processes that enhance outcomes over time. Further, evolution
and learning are caused by similar underlying processes. They both
fundamentally improve dynamics through trail and error. Even supervised
machine learning is at the base a trail and error process that continuously is
compared to a desired outcome.

As our technologies become increasingly more life-like and intelligent, often
implemented through Lamarckian and Baldwinian evolution, the lines between
evolution and learning will likely further blend together. Take e.g. a self-
driving car. The functionality/fitness of such a car is both dependent on its
physical components (engine, tires, sensors, etc.) and the software which
drives the car. The physical components cannot by updated - only exchanged
with new identical versions for repair - whereas the software can be
continually updated at any time. In this sense, the evolution of the physical
car is Darwinian in nature, while the software is Lamarckian in nature. The
environment of self-driving cars will also eventually be increasingly more
dynamic as self-driving cars remove the constant and limiting factor of human
driving capabilities, meaning future cars will optimise driving among other
self-driving cars potentially allowing large and frequent innovations in
transport. Note, the exact same logic of the self-driving car example can be
applied to modern smartphones.

##  2 Methods

To study the evolutionary dynamics and the role of learning in the three
classical theories outlined in the previous section we develop an agent based
model containing three agent types: Lamarckian, Darwinian, and Baldwinian
agents. The capabilities of each agent type is inspired by their corresponding
classical evolutionary theory. This model allows us to directly compare the
fitness optimization and population dynamics of each classical theory. In this
section, the model is described in detail, starting with the general
characteristics shared by all three agent types.

###  2.1 General agent characteristics

Our simulation is composed of a fixed number of agents competing for space in
a population of constant size $N$. The agents independently (no social
interactions and no spacial dependency) collect resources from a reservoir at
rates dictated only by their fitness (defined in Section 2.2), and replicate
after their total collected resources reach a threshold amount
$R_{replication}$. During replication, the mother agent and daughter agent
equally split the resources of the mother agent, as is typical with single-
celled organisms. The daughter agent replaces the space of a random agent
within the total population, thereby conserving the population size. Every
agent carries information in the form of a genotype $g$ and a phenotype $p$
both defined as bit strings. The genome can be decomposed into two genes: a
metabolic gene $M$ and a learning gene $Q$. The Darwinian genome only has the
metabolic gene $M$ of length $|M|=L$, whereas the Lamarckian and Baldwinian in
addition have the learning gene $Q$ also of length $|Q|=L$ that controls the
learning rate, making their total genomes $2L$ long. Each agent in the initial
population is ascribed a Family identifier (a number between 1 and $N$) which
is inherited by descendent agents. These characteristics are shared by all
three agent types.

###  2.2 Environment and Fitness

The environment is defined as a bit string $E$ of length $|E|=L$ that defines
the optimal phenotype (bit string sequence) of the metabolic gene $M$. The
normalized fitness of the $i$’th agent ($f_{i}$) is computed as the number of
matching bits between the environment and the agent phenotype divided by the
gene length:

$f_{i}^{p}(p_{i},E)=\frac{\sum_{j=1}^{L}\delta_{p}}{L},\,\,\,\,\,\,\textrm{where}\,\,\,\,\,\delta_{p}=\begin{cases}1\,\,\,\textrm{if}\,\,\,p_{i}(j)=E(j)\\\
0\,\,\,\textrm{if}\,\,\,p_{i}(j)\neq E(j)\\\ \end{cases}$ (1)

i.e., the fitness function is linearly dependent on the number of matching
bits. It also follows that the fitness landscape is smooth and contains a
single global maximum corresponding to the phenotype with $L$ matching bits.
The phenotypic fitness of an agent dictates the rate at which that agent gains
resources:

$r_{i}(t+\Delta t,f_{i}^{p})=r_{i}(t,f_{i}^{p})+f_{i}^{p}\Delta t.$ (2)

When all phenotype bits match the environment bits the agent has maximal
fitness, and therefore also maximum resource gain per time step. Since the
agents reproduce after having collected a threshold amount of resources, a
high fitness also correspond to a high rate of replication.

In the code, the genotypic fitness $f_{i}^{g}$ is also computed to study the
difference in dynamics of the genotype and phenotype. It is defined similarly
to the phenotypic fitness:

$f_{i}^{g}(g_{i},E)=\frac{\sum_{j=1}^{L}\delta_{g}}{L},\,\,\,\,\,\,\textrm{where}\,\,\,\,\,\delta_{g}=\begin{cases}1\,\,\,\textrm{if}\,\,\,g_{i}(j)=E(j)\\\
0\,\,\,\textrm{if}\,\,\,g_{i}(j)\neq E(j)\\\ \end{cases}$ (3)

The pseudo code for computation of agent phenotypic fitness, genotypic fitness
and resource gain is as follows:

Result: Computation of phenotypic fitness, genotypic fitness and resource gain

for  _$t=[1:T]$_ do

for  _$i=[1:N]$_ do

for  _$j=[1:L]$_ do

if  _$p_{i}(j)==E(j)$_ then

$f_{i}^{p}(t)=f_{i}^{p}(t)+1$;

end if

if  _$g_{i}(j)==E(j)$_ then

$f_{i}^{g}(t)=f_{i}^{g}(t)+1$;

end if

$r_{i}(t+1)=r_{i}(t)+f_{i}^{p}$;

end for

end for

end for

Algorithm 1 Fitness and resource gain

Note that the both the genotypical and the phenotypical fitness can be
calculated for all agent types, while it is only the phenotypical fitness that
determines the ability to collect resources.

###  2.3 Darwinian agents

When reproducing, the Darwinian agent transmits the genetic information of the
genotype bits with error rate $\alpha$. The genotype maps directly to the
phenotype at birth and the Darwinian agent has no means of changing its
phenotype during its lifetime. A Darwinian agent’s genotype and phenotype are
therefore equal and constant in time; $p_{i}^{D}=g_{i}^{D}$. This is a
simplification as biological systems do not have a direct map from the
genotype to the phenotype. Equation (3) and (2) therefore dictate that the
Darwinian agent fitness is constant in time if the environment is also
constant in time. The resource gain is modelled as follows:

$r_{i}(t+\Delta t,f_{i})=\begin{cases}\begin{aligned}
&r_{i}(t,f_{i}^{p})+f_{i}^{p}\Delta
t\,\,\,&\textrm{if}\,\,\,\,\,\,r_{i}(t)<R^{D}_{replication}\\\
&\frac{r_{i}(t,f_{i}^{p})}{2}\,\,\,&\textrm{if}\,\,\,\,\,\,r_{i}(t)\geq
R^{D}_{replication}\\\ \end{aligned}\end{cases}$ (4)

The resource gain of the Darwinian agent is described by Equation (2) when the
agent has not yet reached the threshold required for reproducing. When the
threshold is reached, the agent reproduces using half its current resources
(as described in Section 2.1).

Figure 1: Illustration of the Darwinian agent type. Note that
$g^{0}_{i}=p^{0}_{i}$ and that the agent only has a metabolic gene $M$ of
length $|M|=L$. The agent gathers resources for a while until a certain
threshold $R^{D}_{replication}$ is reached at which point agent $i$ divides.
The remaining available resources are divided equally between the mother and
the daughters agents, and a (set) of random genome mutations with frequency
$\alpha$ may occur at the daughter genome.

###  2.4 Lamarckian agents

The defining feature of the Lamarckian agent is its ability to transmit its
current phenotype at replication, i.e., the phenotype of the mother agent at
the time of replication becomes the genotype of the daughter agent. The
phenotype bit string is transmitted with no errors. The Lamarckian population
instead evolves through individual agents changing their phenotypes during
their lifetimes through so called learning events, in which agents switch the
state of a random bit in their phenotypic information. Learning events occur
with an agent-specific probability $c_{i}$ (denoted curiosity), and has a cost
of a set amount of resources $R_{learning}$. This is an important assumption.
Further, it is assumed that the material basis that enables the learning
mechanism to exist requires extra resources to replicate, so
$R^{L}_{replication}>R^{D}_{replication}$.

The rate at which Lamarckian agents gain resources is:

$r_{i}(t+\Delta t,f_{i}(t))=\begin{cases}\begin{aligned}
&r_{i}(t,f_{i}^{p}(t))+\left[f_{i}^{p}(t)-R^{L}_{learning}c_{i}\right]\Delta
t\,\,\,&\textrm{if}\,\,\,\,\,\,r_{i}(t)<R^{L}_{replication}\\\
&\frac{r_{i}(t,f_{i}^{p}(t))}{2}\,\,\,&\textrm{if}\,\,\,\,\,\,r_{i}(t)\geq
R_{replication}\\\ \end{aligned}\end{cases}$ (5)

Curiosity is a genetic trait added to the genome as a second gene $Q$ also of
length $L$, making the total Lamarckian genome a length of $2L$ bits. The
value of the $i$’th agent’s curiosity is calculated as follows

$c_{i}=\frac{\sum_{j=L+1}^{2L}p_{i}(j)}{L}.$ (6)

This yields $L$ possible values:

$c_{i}\in
C\cdot\left\\{\,\,0,\,\,\frac{1}{L},\,\,\frac{2}{L},\,\,\cdots\,\,,\,\,\frac{L-2}{L},\,\,\frac{L-1}{L},\,\,1\,\,\right\\},$
(7)

where $0\leq C\leq 1$. $C$ is a parameter defining the maximum curiosity of
agents. Learning events may change any bit in the phenotype; both in the
metabolic gene (from which the fitness is computed) and in the curiosity gene.

For the agent to be able to effectively learn from learning events in the
phenotypical gene, the agent memorizes the location of the previously switched
bit and its original state, as well as its resource gain rate during the
previous time step. If the agent has decreased its resource gain rate after
the bit switch, it switches the learning bit back, reversing its phenotype to
its original state before learning. This is described in the Algorithm 2
pseudo code. Note that the switched bit may be within either of the two genes
(metabolic and curiosity). If a learning event change the curiosity gene, the
fitness of the agent is not changed (in a constant environment). Agents will
therefore not reverse switches in the curiosity gene.

Result: Changes in phenotype due to learning

for  _$t=[1:T]$_ do

for  _$i=[1:N]$_ do

$learning\\_bit$ = random bit between $1$ and $2L$;

$p_{i}(learning\\_bit)$ = not($p_{i}(learning\\_bit)$);

Compute and store $r_{i}(t+1)$ using Equation 5;

$\Delta r_{i}(t+1)=r_{i}(t+1)-r_{i}(t)$;

if  _$\Delta r_{i}(t+1) <\Delta r_{i}(t)$_ then

$p_{i}(learning\\_bit)=$not($p_{i}(learning\\_bit)$);

end if

end for

end for

Algorithm 2 Learning algorithm Figure 2: Illustration of the Lamarckian agent
type. Note that the agent has two genes each of length $L$: one for metabolism
and one for learning. The agent gathers resources while it at the same time,
with a frequency determined by the curiosity parameter $C$, learns about its
environment. Each learning event requires a certain amount of resources
$R_{learning}$ and the learning process can occur at both phenotypical genes,
so the agent can both alter its metabolic abilities as well as it ’curiosity’
level during it’s lifetime. At a certain resource threshold
$R^{L}_{replication}$ the agent $i$ divides and its resources are divided
equally between the mother and the daughter agents. Note that the phenotypical
genes are inherited, not the genotypical genes.

###  2.5 Baldwinian agents

The Baldwinian agent may be viewed as the missing link between the Darwinian
agent and the Lamarckian agent. Like the Darwinian agent, the Baldwinian agent
only transmits the genotype with error rate $\alpha$ during replication.
However, like the Lamarckian agent the Baldwinian agent is capable of learning
during its lifetime, changing its full phenotype (both $M$ and $Q$ genes)
without being able to transmit these changes to its offspring. The ability to
learn via the curiosity trait can be seen as modelling phenotypic plasticity,
as Baldwinian (and Lamarckian) agents during their lifetime $\tau_{i}$ explore
approximately $c_{i}\cdot\tau_{i}$ new phenotypic states. The curiosity and
resource gain (fitness) of the Baldwinian agent is defined by Equations (5),
(6) and (7), like the Lamarckian agent type. If $C=0$ in Equation (7), the
Baldwinian agent type coincides with the Darwinian agent type.

Note that in the current implementation the Baldwinian agent type does not
mutate its curiosity gene during replication. A Baldwinian agent may change
its phenotypic curiosity during its lifetime through learning, but will not
transmit such changes to its daughter agents. Thus, a population of Baldwinian
agents have a limited pool of genotypic curiosity; meaning Baldwinian
populations will eventually have a homogeneous genotypic curiosity after
sufficiently many generations, though there may still be variations in their
phenotypic curiosity (if the genotypic curiosity of the population is non-
zero).

Figure 3: Illustration of the Baldwinian agent type. Like the Lamarckian agent
type, the Baldwinian agent type has both a metabolic and curiosity gene, and
can change the phenotype of both genes via learning events at the cost of
$R_{learning}$. The illustration shows agent $i$ gathering resources and
learning until it has reached a certain threshold $R^{B}_{replication}$ where
it replicates; its daughter taking agent $j$’s place in the population.
However, learned phenotypic changes are not transmitted during replication,
where the genome instead is mutated, as for the Darwinian agent type.

###  2.6 Temporally dependent Environment

The environment $E$ is previously defined by a bit string that is constant in
time, usually as a string of length $L$ containing only ones. However, the
environment may also be time dependent. How the environment is a function of
time may be defined in numerous ways, giving rise to different types of
dynamics. In our model, the time varying environment is defined as a semi-
periodic variation, such that time periods of length $\tau_{E}$ with constant
environment, is interrupted by sudden environmental changes given by an
environmental mutation (error) rate $\alpha_{E}$ (See Algorithm 3). This
loosely models the biological emergence of new feeding conditions in a
chemostat.

Result: Periodic mutations in the environment

for  _$t=[1:T]$_ do

if  _$remainder(t,\tau_{E})==0$_ then

for  _$j=[1:L]$_ do

Draw $random\\_number$;

if  _$random\\_number <\alpha_{E}$_ then

$E(j)$=not$(E(j))$

end if

end for

end if

end for

Algorithm 3 Temporal changes in the environment

##  3 Results

In this section simulations of the agent based model described in section 2
are run in MATLAB 2020a. The dynamics of agent replication and evolutionary
optimization of the phenotype are explored through these simulations. A set of
standard conditions listed in Table 1 has been used in simulations throughout
the results, while only varying one or two parameters per simulation.

General parameters:
---
Description | Symbol | Value
Population size | $N$ | $1000$
Initial mean fitness | $\bar{f}(t=0)$ | $0.01$
Initial mean curiosity | $\bar{c}(t=0)$ | $0.1$
Error rate | $\alpha$ | $0.01$
Replication threshold (for all agent types) | $R_{replication}$ | $10$
Learning cost | $R_{learning}$ | $0.1$
Gene length (both curiosity gene and metabolic gene) | $L$ | $10$
Maximum curiosity | $C$ | $1$
Additional parameters in time-dependent environment:
Description | Symbol | Value
Environment mutation rate | $\alpha_{E}$ | $0.5$
Environment mutation period | $\tau_{E}$ | $100$
Table 1: Standard conditions for simulations

###  3.1 Mono agent type population fitness in constant environments

In this section the temporal dynamics of the mean fitness of pure populations
in a constant environment are tested using simulations for all three agent
types. First the general dynamics at standard conditions (Table 1) are shown
for the three agent types, whereafter the dynamics of the agent types are
compared in simulations with chosen parameter variations.

Figure 4: Upper panel: The temporal dynamics of population mean fitness of a
pure Darwinian population plotted against $t$. Lower left panel: Family mean
fitness of agent families in the pure Darwinian population vs. $t$. The
individual agents in the initial population are ascribed a number from 1 to N
- called their family identifier - with a corresponding color in the used
colormap (blue $\rightarrow$ green $\rightarrow$ yellow). This identifier is
inherited by their daughter agents. Lower right panel: The number of family
members in each family vs. $t$. It should be noted that graphs may be hidden
by other graphs at small $t$, as families mean fitness graphs are degenerate
for Darwinian populations because of the small variance in phenotypic fitness
(Figure 7).

For Darwinian agents the mean phenotypic fitness graph and the mean genotypic
fitness graph overlays, as the genotype maps directly to the phenotype, as
explained in Section 2.3. Initially, the optimization is very slow as the
initial mean fitness of $0.01$ means only $\approx$10% of the initial
population have a non-zero fitness. At low fitness, the optimization of
Darwinian agents are slower than learning-based algorithms, since the rate of
optimization is dependent on replication rate, while At low mean fitness the
optimization is slow since the agents must first collect enough resources to
replicate. At high mean fitness, the optimization may be slowed by
coincidental replacement of agents with higher than mean fitness by agents
with lesser than mean fitness due to the high flux of new agents. For the used
simulation, the family with the identifier 970 (bright yellow) becomes the
dominating family due to its higher rate of replication caused by its high
fitness.

Figure 5: Upper panel: The temporal dynamics of population mean fitness of a
pure Lamarckian population plotted against $t$. Lower left panel: Family mean
fitness of agent families in the pure Lamarckian population vs. $t$. Lower
right panel: The number of family members in each family vs. $t$.

At standard conditions the mean fitness of the Lamarckian population increases
significantly quicker than the other agent types. Learning events efficiently
increase the phenotypic fitness (dotted red line) at small $t$ as the
likelihood of switches increasing the fitness is large. The genotypic fitness
(thin red line) starts increasing as the agents begin to replicate at
$t\approx 25$. A selection pressure for high curiosity is initially present,
resulting in a high mean curiosity (dashed black line). As the population mean
fitness increases, learning events become less efficient at increasing
phenotypic fitness, as agents become more likely to decrease their fitness
from switches. After a critical point at $t\approx 100$ we have
$\bar{f}_{p}=\bar{f}_{p}$, and the dynamics change as the high population mean
curiosity instead result in $\bar{f}_{p}(t>100)<\bar{f}_{g}(t>100)$, and a
selection pressure for lower curiosity is generated. At $t>300$ the mean
curiosity decreases as agents with low curiosity and high fitness emerge, and
the system finally reaches a constant final state of a population consisting
purely of agents with a single high fitness phenotype and zero curiosity. For
a genome length of $L=10$ \- which is the used standard condition - this is
likely to be the state of maximum fitness, but for larger genomes the final
agents are unlikely to have exactly maximum fitness.

Comparing the family dynamics of the Lamarckian population with the family
dynamics of the Darwinian population shown on Figure 4 middle panel reveals a
higher variation of phenotypic fitness within each family as learning events
increase the plasticity of the phenotype (also illustrated in Figure 7). At
$t\approx 500$, all agents in the population has maximal fitness. As seen on
Figure 5 Lower Panel, simulations may result in multiple competing families
with maximal fitness.

Figure 6: Upper panel: The temporal dynamics of population mean fitness of a
pure Baldwinian population plotted against $t$. Lower left panel: Family mean
fitness of agent families in the pure Baldwinian population vs. $t$. Lower
right panel: The number of family members in each family vs. $t$.

The Baldwinian population initially quickly increases its mean phenotypic
fitness via learning like the Lamarckian population. However, this only boosts
the fitness of Baldwinian agents during their lifetime, and replicated agents
do not benefit. The genotypic fitness can only increase through random
mutations during replication. Since Baldwinian agents spend resources on
learning, the replication rates of Baldwinian agents are lower than for
Darwinian agents, and the rise in mean genotypic fitness is therefore slower
than for the Darwinian population shown on Figure 4. Learning becomes less
efficient as phenotypic fitness increases, and like the Lamarckian population,
the Baldwinian population reaches a critical point at $t\approx 425$
whereafter $\bar{f}_{p}(t>100)<\bar{f}_{g}(t>100)$. Unlike the Lamarckian
population, the Baldwinian population is not able to find a final state
consisting of agents with high fitness and zero curiosity, as the random
mutations introduce noise into both the curiosity and metabolic gene pools.

Similar to the Lamarckian population on Figure 5, the Baldwinian population
have a high variation of phenotypic fitness within each family due to the
phenotypic plasticity generated by learning.

Figure 7: The variance of phenotypic fitness in the populations from the
simulations shown on Figures 4-6. The Lamarckian population (red) shows a very
high initial increase in phenotypic variance followed by a correspondingly
fast decrease, as the population has reached a mean fitness of $\sim 0.8$. At
$t\approx 490$, the Lamarckian phenotypic variance becomes zero, as all agents
in the population have optimal fitness and zero curiosity. The Darwinian
population (blue) has the lowest maximal phenotypic variance; and this maximum
also happens later in the dynamics than for the Lamarckian and Baldwinian
populations. The Baldwinian population (purple) shows higher phenotypic
variance compared to the Darwinian population at all times, as the Baldwinian
agent type was designed to do.

####  3.1.1 Comparison of mono agent type populations: constant environment

In this section, the fitness and curiosity dynamics of pure populations of all
three agent types are compared while changing a single parameter from its
standard value and keeping all other parameters as defined in Table 1. Here,
pure populations mean that three separate simulations with either 1000
Darwinian, Lamarckian or Baldwinian agents have been run with a given set of
parameters, and the results of these three simulations have been plotted in a
single figure for direct comparison of phenotypic and genotypic fitness
optimization dynamics as well as the mean curiosity dynamics. In the Figures
8-13 the parameters $L$, $\alpha$, $R_{replication}$, $C$, and $R_{learning}$
are varied consecutively. The simulation runtimes are also shown to gauge and
compare the computational cost of the simulations. Depending on the
simulation, the total number of $t$-steps ($T$) is either $T=1000$ or
$T=2500$, which should be regarded while comparing figures and runtimes.

Figure 8: Standard conditions. Darwinian computation time = 14 seconds,
Baldwinian computation time = 21 seconds, Lamarckian computation time = 23
seconds. Figure 9: Standard conditions with $L=30$. At larger gene lengths,
the Lamarckian population is less likely to enter the part of the dynamics
where the curiosity is down-regulated within the duration of the simulation.
Darwinian computation time = 39 seconds, Baldwinian computation time = 57
seconds, Lamarckian computation time = 86 seconds. Figure 10: Standard
conditions with $\alpha=0.001$. With a smaller error rate, the Darwinian and
Baldwinian optimization is slower, but the Darwinian population is able to
reach states of higher fitness likely due to the lesser noise introduced in
the population. The Baldwinian population is inhibited from reaching this
state of high fitness due to the noise introduced by learning events, though
given enough simulation time, the population may be able to down-regulate
curiosity. Darwinian computation time = 35 seconds, Baldwinian computation
time = 53 seconds, Lamarckian computation time = 55 seconds. Figure 11:
Standard conditions with $R_{replication}=5$. At a smaller replication
threshold, the Darwinian algorithm is able to search through more states. The
optimization is therefore faster. The non-constant fitness of the Lamarckian
population after reaching zero mean curiosity is due to the presence of
several competing families with zero curiosity at $t>520$. Darwinian
computation time = 20 seconds, Baldwinian computation time = 26 seconds,
Lamarckian computation time = 30 seconds. Figure 12: Standard conditions with
$C=0.01$. At low maximum curiosity, the optimization of the Lamarckian
population is slower than the Darwinian and Baldwinian. The Baldwinian
dynamics are nearly identical to the Darwinian dynamics, as the Baldwinian
agent type coincides with the Darwinian at zero curiosity, as explained in
Section 2.5. Darwinian computation time = 40 seconds, Baldwinian computation
time = 55 seconds, Lamarckian computation time = 55 seconds. Figure 13:
Standard conditions with $R_{learning}=9$. At learning cost comparable to the
replication cost, the optimization of the Lamarckian population is slower than
the Darwinian and Baldwinian. A selection pressure for lowering curiosity
emerges in the Baldwinian population. Darwinian computation time = 15 seconds,
Baldwinian computation time = 21 seconds, Lamarckian computation time = 21
seconds.

To summarize, at the chosen standard conditions the Lamarckian agent type
optimizes fitness better than both the Darwinian and Baldwinian agent types.
For larger gene length ($L=30$) all agent types optimize slower and reach a
lower final fitness within the allotted simulation time. Furthermore, the
Lamarckian agent type is less likely to find a state of near-maximal fitness
and zero curiosity, and the dynamics therefore does show the characteristic
second rise in fitness. At lower mutation rate ($\alpha=0.001$) yields slower
optimization for the Darwinian and Baldwinian agent types, but also a higher
optimal fitness due to the lesser noise introduced by mutations. If the
mutation rate instead increases, the Darwinian and Baldwinian agent type
optimize faster, but reach a lower optimal fitness. At standard conditions, we
do not observe a crossover - where the Darwinian and Baldwinian agent types
beat the Lamarckian agent type - by only varying the mutation rate $\alpha$.
Lower replication cost benefits the Darwinian and Baldwinian agent types, as
their optimization rates scale with replication rate, while the Lamarckian
optimization rate stays constant. At standard conditions, we do not observe a
crossover by only varying the replication cost. At low maximum curiosity
($C=0.01$) the optimization rate of the Lamarckian agent type is lowered
drastically, and a crossover in the dynamics is observed. Finally, increasing
the learning cost slows Lamarckian optimization. However, a crossover in the
dynamics is only observed at very high learning costs, i.e., when the learning
cost is comparable to the replication cost ($R_{learning}=9\approx
R_{replication}=10$).

###  3.2 Mono agent type population fitness in temporally varying environment

Now the fitness dynamics for pure populations of all three agent types are
investigated in a temporally dependent environment as defined in Section 2.6.
The relative effectiveness of the agent types is measured by comparing the
time-average of the mean fitness during steady state of the dynamics; steady
state being defined as $t>3000$.

Figure 14: A comparison of the fitness dynamics of pure populations in a
temporal environment defined in Section 2.6. The parameters are defined such
that the environment is scrambled ($\alpha_{E}=0.5$) every 100 t-steps
($\tau_{E}=100$). Parameter values used: Standard conditions except
$R_{replication}=50$ and $N=100$.

For the parameters used in the simulation on Figure 14, the Lamarckian agent
type has the highest time-averaged mean phenotypic fitness (horizontal gray
line), followed by the Baldwinian agent type, and finally the Darwinian agent
type has the lowest time-averaged mean fitness. The simulations show that when
the replication rate is on the same time scale as the mutations of the
environment, the Darwinian method of increasing fitness through replication
with mutation is not a viable strategy for evolutionary optimization.
Lamarckian and Baldwinian populations are simply able to explore more states
utilizing the curiosity-based learning algorithm before the environment
mutates. 20 simulations with the same parameters as used for the simulation on
Figure 14 but with varying rng-seeds show the general results for the fitness
dynamics at steady state ($3000<t<6000$): Time-averaged mean phenotypic
fitness
$\bar{f}_{p}^{L}=0.6701>\bar{f}_{p}^{B}=0.6013>\bar{f}_{p}^{D}=0.5166$; Time-
averaged mean genotypic fitness
$\bar{f}_{g}^{L}=0.6022>\bar{f}_{g}^{D}=0.5166>\bar{f}_{g}^{B}=0.5057$. The
highest mean genotypic fitness is reached by the Lamarckian population due to
transmission of the phenotypic changes to the genotype of the new agent, i.e.,
inheritance of acquired traits, and the genotypic mean fitness of the
Baldwinian and Darwinian populations are approximately equal, due to the lack
thereof. This is illustrated on Figure 14.

###  3.3 Pairwise agent population competition

In this section, the total population is allowed to be a mix of different
agent types, allowing for simulations of direct competition between agent
types. For such simulations with a finite total population size, the
population drifts towards homogeneity, meaning the total population will
consist of a single agent type within a finite amount of $t$-steps. This final
agent type is then considered to be winner.

Figure 15: Family dynamics of a mixed initial population consisting of 100
Darwinian agents (blue spectrum) and 100 Lamarckian agents (red spectrum). For
the used parameters and initial conditions, the Lamarckian agent type fills
the population at $t\approx 200$, and is therefore declared the winning agent
type. Furthermore, at $t\approx 1000$, only one Lamarckian family dominates
the population, as shown by the complete population being composed of a single
shade of red.

For a non-analytic model as ours, a complete analysis of the outcomes of such
competitions would be obtained by running simulations while varying every
parameter and initial condition, making a high dimensional phase space,
wherein each point has a single winning agent type. Furthermore, to make the
results rigorously statistically significant, multiple simulations would have
to be run for each point in the phase space, since the random number
generation influences the final outcome; especially close to the interfaces
between areas where one agent has a clear advantage. This is not possible in
practise, and the results would be equally hard to illustrate.

Instead, we have chosen to limit our simulations by varying two parameters at
a time, keeping all other parameters and initial conditions equal to the
standard conditions specified in Table 1, while only allowing two agent types
to compete in each simulation with their initial populations being equal. For
a constant environment the chosen parameters are the learning cost
$R_{learning}$ and the relative replication cost $R_{ij}$ between the agent
types $i$ and $j$, which is defined as
$R_{ij}=R^{i}_{replication}/R^{j}_{replication}$. A simulation is run and the
winning agent type is found for each sampled pair of parameter values
corresponding to the pixels in the subsequent figures. The colors of the
pixels in these figures correspond to the winning agent types using the
established color convention.

Figure 16: Winning agent types as a function of relative replication cost and
learning cost for mixed populations consisting of 100 Lamarckian and 100
Darwinian agents (left panel); 100 Baldwinian and 100 Darwinian agents (middle
panel); 100 Lamarckian and 100 Baldwinian agents (right panel).

It is found that learning gives a definite advantage at low learning cost;
even when learning agents have a high relative replication cost compared to
non-learning agents. Also note that the Baldwinian agent type has an advantage
over the Lamarckian agent type at low learning cost and $R_{LB}>1.8$, which is
further demonstrated in Figure 18.

###  3.4 Three-way agent population competition

We now investigate a system similar to the one in the previous section, but
with all three agent types in competition.

Figure 17: Family dynamics of a mixed initial population consisting of 100
Darwinian agents (blue spectrum), 100 Baldwinian agents (purple spectrum), and
100 Lamarckian agents (red spectrum). For the used parameters and initial
conditions, the Lamarckian agent type fills the population at $t\approx 400$.

For this, we limit our investigations to cases where $R^{L}_{replication}\leq
R^{B}_{replication}\leq R^{D}_{replication}$; more specifically, we define
$R_{LB}=R_{BD}$, e.g. for the rightmost column of pixels in Figure 18 we
define $R_{LB}=R_{BD}=2$, meaning
$R^{L}_{replication}=2R^{B}_{replication}=4R^{D}_{replication}$.

The reasoning behind this choice is as follows: In biology, the cost of
replication commonly increases with the complexity and amount of the wetware
that is replicated. We argue that Baldwinian organisms are typically more
complex than simple Darwinian organisms, as the capability of learning
requires additional wetware. It is then reasonable that Lamarckian organisms
would require even more wetware to also encode the learned phenotypic changes.

Figure 18: Investigation of three-way competition with an initial population
consisting of 100 agents of each agent type. Here, we have used
$R_{learning}=\\{0.01,...,1\\}$ and $R_{LB}=R_{LB}=\\{1.3,...,2.3\\}$.

It can be seen that all three agent types have regions in this parameter space
where their algorithms are advantageous. The Lamarckian agent type
predominantly wins at $R_{LB}=R_{LB}<1.8$ The Darwinian agent type wins at
high learning costs and high relative replication costs, while the Baldwinian
agent type frequently wins at $R_{LB}=R_{LB}>1.8$ and $R_{learning}<0.5$,
which corresponds with Figure 16.

So far, the competition dynamics have only been investigated in constant
environments. We now allow the environment to change, as described in Section
2.6, while also varying the learning cost and relative replication cost,
thereby forming a multiaxial plot of the winning agent type as can be seen on
Figure 19.

Figure 19: Each subplot shows the winning agent types in dynamic environments
described by pairs of values for the environment mutation period ($\tau_{E}$)
and the environment mutation rate ($\alpha_{E}$). Furthermore, these subplots
are made for 16 different pairs of values for the relative replication cost
($R_{LB}$ and $R_{BD}$, constrained by demanding $R_{LB}=R_{BD}$) and learning
cost ($R_{learning}$).

Much information of the model dynamics can be extracted from Figure 19. At
equal replication cost ($R_{LB}=R_{BD}=1$), the Lamarckian agent dominates in
the dynamic environments. At $R_{LB}=R_{BD}\geq 1.2$ the Darwinian and
Baldwinian agent types dominate in very dynamic environments, i.e., at high
$\alpha_{E}$ and low $\tau_{E}$. The fraction of Darwinian to Baldwinian
winners depend heavily on the learning cost, with only a small dependence on
$\alpha_{E}$ and $\tau_{E}$. A small $\alpha_{E}$ always benefits the
Lamarckian agent type. This indicates that the curiosity learning algorithm is
efficient at adapting phenotypes to minor changes in the environment. Finally,
at both high learning cost and high relative replication cost the Darwinian
agent type dominates in the dynamic environments, though the Lamarckian agent
type still frequently wins in a constant and near-constant environment. This
goes against our hypothesis - supported by our initial findings - of the
Lamarckian agent type being the most fit in a dynamic environment; at least
there is a large dependence on the relative replication cost and the specific
nature of the dynamic environment.

##  4 Discussion

Though the main focus during the design of the model has been conceptual
simplicity, the process of formulating the model into code has involved the
introduction of many variables. This has made the set of possible initial
conditions too large to make a complete investigation of the possible dynamics
of the system. The study of the Baldwinian agent type was mostly included to
satisfy a curiosity of the role of learning in biological systems, but the
small genome size used in simulation is far from biological genomes which are
on the scale of $10^{5}-10^{12}$ base pairs [35], and as mentioned in Section
2.3 the simple genotype-phenotype mapping also deviates from the complex
mapping in biological systems [36]. As shown in Figure 9, the Lamarckian
population is less likely to down-regulate curiosity when having large genomes
which limit evolutionary optimization at high fitness of the current
Lamarckian algorithm. Furthermore, the curiosity algorithm has a maximal
learning rate of a single bit per $t$-step, where the Darwinian algorithm
instead is maximally able to change $\alpha L$ bits per generation. This
proportionality gives the Darwinian algorithm a definite advantage at very
large genome sizes. The amount of inherited information of even the simplest
biological organisms capable of learning surely is much greater then the
amount of genetic information carried by the implemented agents in all
simulations, so the currently implemented curiosity algorithm is unequipped
for optimizing such systems. This is all specific to the learning algorithm,
and therefore no generic conclusions can be stated about the general dynamics
of other learning algorithms for systems of high information.

The reasoning behind the design of the learning algorithm used by the
Lamarckian and Baldwinian agents has been conceptional and computational
simplicity. A different learning algorithm would undoubtedly yield different
dynamics, which subsequently limits the utility of the specific findings
outlined in the Results section for general systems. Though this must be the
case, there are aspects of the found results which may be generic.
Particularly, the advantage of agents having the ability to learn during its
lifetime while in an environment which changes at a faster rate than the
replication rate of the agents. The ability of the Lamarckian agent to
transmit such learned changes during replication has yielded the fastest rise
in fitness for most of the investigated parameter conditions.

As the simulations fall under the category of mathematical optimization, it is
relevant to compare and discuss the computational cost of simulations for each
agent type. The main computational cost is due replication of agents and the
calculation of the fitness of each agent. In a constant environment the
Darwinian agents have constant fitness during their lifetime, their fitness is
therefore only calculated once; at birth. The Lamarckian and Baldwinian agent
types change their fitness during their lifetime at each learning event, and
their new fitness must be calculated whenever they learn adding a scaling
factor of $\bar{c}$ to the computation cost. In terms of the model-specific
parameters, it is mostly the maximum curiosity $C$ and the replication
threshold $R$ which determine the average computational cost of each $t$-step.
The stated runtimes on the figures in Section 3.1.1 show a high cost of the
Lamarckian algorithm, but for most parameters the Lamarckian populations
quickly reach a high fitness, and computational cost could be lowered by
gauging the runtime needed for reaching the desired fitness, or simply
terminating the simulation when a desired fitness has been reached.

Other fitness functions could also easily be implemented in the model for
investigation. The simple linear fitness landscape of the environment used in
this study could be exchanged with, e.g., a more complex NK model.

Further studies of Lamarckian and Baldwinian evolution and optimization could
implement more sophisticated learning algorithms based on artificial neural
networks or simulated annealing in more complex environments, perhaps with
defined temporal or spacial patterns.

A system consisting of agents utilizing neural networks to distinguish food
from poison in an environment of dynamically changing set of materials has
been studied by Sasaki and Tokoro (1999) [40]. By varying the heritability of
acquired characteristics, the authors find the Darwinian population to have
the highest level of adaption in dynamic environments. Our model shows similar
results (Figure 19), through not generally, as each agent type may perform
best in dynamic environments depending on the model parameter values.

##  5 Conclusions

Our work successfully implements the Darwinian, Lamarckian and Baldwinian
algorithms in a single agent based modelling framework, and all three
algorithms exhibit evolutionary optimization. Pure populations of all agent
types exhibit a region of exponential growth in agent replications if the
population is not yet filled with replicating agents (Figure 3 left panel).
When all agents in the population is able to replicate, the replication rate
instead increases slowly with population mean fitness (Figure 3 right panel).

Fitness dynamics generally deviate from standard optimization processes in
$log(t)$-plots at low and high fitness due to the fitness dependency of
replication rates (Figure 4). The speed of evolutionary optimization for pure
Darwinian populations is inversely proportional to the replication threshold,
meaning a high replication rate benefits the Darwinian agent type (Figure 11).
Optimization speeds for pure Lamarckian agent populations are proportional to
curiosity (learning rate), though the maximal fitness is inversely
proportional to curiosity (Figure 12). At large genome sizes pure Lamarckian
populations are less likely to reach a final state of zero curiosity via down-
regulation of the curiosity gene (Figure 9). The evolutionary optimization
speeds of pure Baldwinian populations are initially higher than pure Darwinian
populations, but become lower at high fitness as the curiosity learning
algorithm becomes less efficient since the resources expended on learning
instead inhibits the optimization of Baldwinian populations via the Darwinian
algorithm due to the lower replication rate compared to the Darwinian agent
type (Figure 8). For high maximum curiosity, optimization speeds of pure
Baldwinian populations are slower than those of pure Lamarckian populations
due to the loss of useful phenotypic changes at replication, but at low
curiosity the Baldwinian dynamics coincide with the dynamics of Darwinian
populations (Figure 12).

Both Lamarckian and Baldwinian populations reach higher phenotypic fitness
than the Darwinian population in a temporally dependent environment which
scrambles at a periodicity on the same timescale as the replication rate
(Figure 14).

Mixed populations simulations allowing for direct competition between agent
types (pairwise and three-way) show populations drift toward homogeneity.
These simulations show regions in parameter space that benefit the agent
types.

## References

  * [1] Wikipedia: Baldwin effect.  https://en.wikipedia.org/wiki/Baldwin_effect.
  * [2] Wikipedia: Invertebrate.  https://en.wikipedia.org/wiki/Invertebrate.
  * [3] Wikipedia: James mark baldwin.  https://en.wikipedia.org/wiki/James_Mark_Baldwin.
  * [4] Takaya Arita and Reiji Suzuki.  Interactions between learning and evolution: The outstanding strategy generated by the baldwin effect.  Artificial Life, 7:196–205, 2000.
  * [5] RJ Bagley, JD Farmer, and W Fontana.  Artificial life ii, 1991.
  * [6] J. Mark Baldwin.  A new factor in evolution.  The American Naturalist, 30(354):441–451, 1896.
  * [7] Mark A Bedau and C Titus Brown.  Visualizing evolutionary activity of genotypes.  Artificial Life, 5(1):17–35, 1999.
  * [8] Mark A Bedau and Norman H Packard.  Measurement of evolutionary activity, teleology, and life.  Artificial Life, 2:431–461, 1992.
  * [9] Emily D Cahan.  The genetic psychologies of james mark baldwin and jean piaget.  Developmental Psychology, 20(1):128, 1984.
  * [10] Charles Darwin.  On the Origin of Species By Means of Natural Selection 6th edition.  The Floating Press 2009, 1859.
  * [11] Daniel C Dennett.  From bacteria to Bach and back The evolution of minds.  WW Norton & Company, 2017.
  * [12] Daniel C Dennett and Daniel Clement Dennett.  Darwin’s Dangerous Idea Evolution and the Meanings of Life.  Number 39. Simon and Schuster, 1996.
  * [13] Manfred Eigen.  Selforganization of matter and the evolution of biological macromolecules.  Naturwissenschaften, 58(10):465–523, 1971.
  * [14] Manfred Eigen and Ruthild Winkler-Oswatitsch.  Transfer-rna, an early gene?  Naturwissenschaften, 68(6):282–292, 1981.
  * [15] Doyne Farmer, Fotini Markopoulou, Eric Beinhocker, and Steen Rasmussen.  Collaborators in creation.  https://aeon.co/essays/how-social-and-physical-technologies-collaborate-to-create.
  * [16] J Doyne Farmer, Stuart A Kauffman, and Norman H Packard.  Autocatalytic replication of polymers.  Physica D: Nonlinear Phenomena, 22(1-3):50–67, 1986.
  * [17] Chrisantha Thomas Fernando, Eors Szathmary, and Phil Husbands.  Selectionist and evolutionary approaches to brain function a critical appraisal.  Frontiers in computational neuroscience, 6:24, 2012.
  * [18] RA Fisher and EB Ford.  Variability of species.  Nature, 118(2971):515–516, 1926.
  * [19] Lee Fleming and Olav Sorenson.  Technology as a complex adaptive system evidence from patent data.  Research Policy, 30(7):1019 – 1039, 2001.
  * [20] Walter Fontana, Peter F Stadler, Erich G Bornberg-Bauer, Thomas Griesmacher, Ivo L Hofacker, Manfred Tacker, Pedro Tarazona, Edward D Weinberger, and Peter Schuster.  Rna folding and combinatory landscapes.  Physical review E, 47(3):2083, 1993.
  * [21] Giuseppe Fusco and Alessandro Minelli.  Phenotypic plasticity in development and evolution facts and concepts, 2010.
  * [22] Peter Godfrey-Smith.  Other minds The octopus, the sea, and the deep origins of consciousness.  Farrar, Straus and Giroux, 2016.
  * [23] Geoffrey M Hodgson and Thorbjorn Knudsen.  Dismantling lamarckism why descriptions of socio-economic evolution as lamarckian are misleading.  Journal of Evolutionary Economics, 16(4):343–366, 2006.
  * [24] John Henry Holland et al.  Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence.  MIT press, 1992.
  * [25] Eva Jablonka and Marion J Lamb.  Epigenetic inheritance and evolution the Lamarckian dimension.  Oxford University Press on Demand, 1999.
  * [26] Harry Jerison.  Evolution of the brain and intelligence.  Elsevier, 2012.
  * [27] Stuart Kauffman and Simon Levin.  Towards a general theory of adaptive walks on rugged landscapes.  Journal of theoretical Biology, 128(1):11–45, 1987.
  * [28] Stuart A Kauffman.  Autocatalytic sets of proteins.  Journal of theoretical biology, 119(1):1–24, 1986.
  * [29] Jean-Baptiste Lamarck.  1984, zoological philosophy.  Trans. by H Elliot.) Chicago, IL University of Chicago Press.[Google Scholar], 1809.
  * [30] Kristian Lindgren.  Evolutionary phenomena in simple dynamics.  In Artificial life II, pages 295–312, 1992.
  * [31] Peter Lund Madsen.  Dr. Zukaroffs testamente. Version 2.0. En bog om menneskehjernen.  Gyldendal AS, 2019.
  * [32] John S McCaskill and Norman H Packard.  Analysing emergent dynamics of evolving computation in 2d cellular automata.  In International Conference on Theory and Practice of Natural Computing, pages 3–40. Springer, 2019.
  * [33] Charles Ofria and Claus O Wilke.  Avida: A software platform for research in computational evolutionary biology.  Artificial life, 10(2):191–229, 2004.
  * [34] Norman Packard, Mark A. Bedau, Alastair Channon, Takashi Ikegami, Steen Rasmussen, Kenneth Stanley, and Tim Taylor.  Open-ended evolution and open-endedness editorial introduction to the open-ended evolution i special issue.  Artificial Life, 25(1):1–3, 2019.  PMID 30933628.
  * [35] Rob Phillips, Jane Kondev, Julie Theriot, Hernan G Garcia, and Nigel Orme.  Physical biology of the cell (2nd edition).  Garland Science, 2012.
  * [36] Massimo Pigliucci.  Genotype–phenotype mapping and the end of the ‘genes as blueprint’metaphor.  Philosophical Transactions of the Royal Society B: Biological Sciences, 365(1540):557–566, 2010.
  * [37] Steen Rasmussen.  En undersøgelse af instabiliteter og selvorganiserende processer i dynamiske systemer.  PhD thesis, Danmarks Tekniske Universitet, 1985.
  * [38] Steen Rasmussen.  Toward a quantitative theory of the origin of life.  In Christopher G. Langton, editor, Artificial Life, pages 79–81. Addison-Wesley Publishing Company, 1989.
  * [39] Steen Rasmussen and Paolo Sibani.  Two modes of evolution optimization and expansion.  Artificial life, 25(1):9–21, 2019.
  * [40] Takahiro Sasaki and Mario Tokoro.  Evolving learnable neural networks under changing environments with various rates of inheritance of acquired characters: comparison of darwinian and lamarckian evolution.  Artificial Life, 5(3):203–223, 1999.
  * [41] Paolo Sibani and Steen Rasmussen.  Human wealth evolution: Trends and fluctuations.  Physica A: Statistical Mechanics and its Applications, 558:124985, 2020.
  * [42] John Maynard Smith.  Game theory and the evolution of behaviour.  Proceedings of the Royal Society of London. Series B. Biological Sciences, 205(1161):475–488, 1979.
  * [43] Ricard V Solée, Sergi Valverde, Marti Rosas Casals, Stuart A Kauffman, Doyne Farmer, and Niles Eldredge.  The evolutionary ecology of technological innovations.  Complexity, 18(4):15–27, 2013.
  * [44] Shinpei Tanaka, Harold Fellermann, and Steen Rasmussen.  Structure and selection in an autocatalytic binary polymer model.  EPL (Europhysics Letters), 107(2):28004, 2014.
  * [45] Andreas Wagner.  Information theory, evolutionary innovations and evolvability.  Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1735):20160416, 2017.
  * [46] Daniel J Webre, Peter M Wolanin, and Jeffry B Stock.  Bacterial chemotaxis.  Current Biology, 13(2):R47–R49, 2003.


and an appropriate barycenter of $\partial\psi_{\delta}(x_{i})$.

Next, we prove the following result.

######  Theorem A.2.

With the notations above, and under the assumptions of Theorem A.1, assume
further than $\psi^{*}$ is $C^{1,\alpha}$. Let $[x_{1},x_{2}]$ be a line
segment in $\Omega$ with length $L$ and denote

$t=\max_{s\in[0,1]}(1-s)\psi_{\delta}(x_{1})+s\psi_{\delta}(x_{2})-\psi_{\delta}((1-s)x_{1}+sx_{2})$

Then there exists $C$ and $\alpha_{1},\alpha_{2}>0$ such that if

$\delta\leq C\min\\{L^{\alpha_{1}},\frac{L^{\alpha_{2}}}{\lambda}\\}$

then

$t\geq cL^{\alpha_{1}}.$

Note that when $t=0$, this theorem provides a bound on the diameter of the
flat parts of $\psi_{\delta}$ by some power of $\delta$. For a given $t>0$, it
implies a bound on the diameter of the sections of $\psi_{\delta}$. Indeed if
$y\in\partial S_{\delta}(x,p,t)$, namely
$\psi_{\delta}(y)=\psi_{\delta}(x)+p\cdot(y-x)+t$, then

$\begin{split}&(1-s)\psi_{\delta}(x)+s\psi_{\delta}(y)-\psi_{\delta}((1-s)x+sy)\\\
&\qquad=(1-s)(\psi_{\delta}(x)-p\cdot x)+s(\psi_{\delta}(y)-p\cdot
y)-(\psi_{\delta}((1-s)x+sy)-p\cdot((1-s)x+sy))\leq t.\end{split}$

Hence, taking $\delta\lesssim t$, the previous theorem implies that $|x-y|\leq
C\,t^{1/\alpha_{1}}$ or $|x-y|\leq C\,\delta^{1/\alpha_{2}}$. Consider now any
$\Omega^{\prime}\subset\Omega$ at a distance $d_{0}$ from $\partial\Omega$,
with $d_{0}\geq C\,\delta^{1/\alpha_{2}}$ and take
$\rho=c\,d_{0}^{\alpha_{1}}$; we necessarily have that
$S(x,p,\rho)\subset\Omega$ for any $x\in\Omega^{\prime}$, which is (14). All
the assumptions of Theorem 2.5 are thus satisfied in this case.

###### Proof of Theorem A.2.

We assume (without loss of generality) that $0=\frac{x_{1}+x_{2}}{2}$ and
$\psi^{\delta}(x_{1})=\psi^{\delta}(x_{2})$, $\nabla\psi(0)=0$. We also denote
by $e$ the unit vector in the direction of $x_{2}$. Given $\theta>0$, we
consider the cone of opening $\theta$ and axis $e$:

$C_{\theta}=\left\\{x\in\mathbb{R}^{n}\,\Big{|}\,\frac{x\cdot e}{|x|}\geq
1-\theta\right\\}$

and

$A=\left\\{x\in C_{\theta}\,|\,L/8\leq|x|\leq L/4\right\\}$

We have

$|A|\geq CL^{n}\theta^{n-1}$

and classical properties of convex functions (see Lemma 2.1 in [26] for
example) imply:

$|z\cdot e|\leq C\frac{\|\nabla\psi\|_{L^{\infty}}}{L}(L\theta+t)\qquad\forall
z\in\partial\psi^{\delta}(x),\quad\forall x\in A_{\theta}.$

Next, the $C^{1,\alpha}$ regularity of $\psi^{*}$ implies

$\nabla\psi(x)\cdot e\geq cL^{\beta}\quad\mbox{ for $x\in\Omega^{\prime}\cap
A$}$

for some $\beta>0$ if $\theta$ is small enough. Indeed, we have $\psi(x)\geq
c|x|^{1+\beta}$ and so $\nabla\psi(x)\cdot\frac{x}{|x|}\geq c|x|^{\beta}$. We
deduce $\nabla\psi(x)\cdot e\geq c\cos\theta
L^{\beta}-\sin\theta\|\nabla\psi\|_{\infty}$ in $A$, so we must take
$\theta=c\max\\{1,\frac{L^{\beta}}{\|\nabla\psi\|_{\infty}}\\}$

In particular, we have

$\displaystyle\mathrm{dist}\,(\nabla\psi(x_{i}),\partial\psi_{\delta}(x_{i}))$
$\displaystyle\geq\mathrm{dist}\,(\nabla\psi(x_{i})\cdot
e,\partial\psi_{\delta}(x_{i})\cdot e)$ $\displaystyle\geq
c(1-\theta)L^{\beta}-C(\theta+\frac{t}{L})\qquad\forall x\in A$

With our choice of
$\theta=c\max\\{1,\frac{L^{\beta}}{\|\nabla\psi\|_{\infty}}\\}$, we see that
if $t\leq cL^{1+\beta}$, then

$\displaystyle\mathrm{dist}\,(\nabla\psi(x_{i}),\partial\psi_{\delta}(x_{i}))$
$\displaystyle\geq cL^{\beta}\qquad\forall x\in A.$

Theorem A.1 implies in particular

(80) $c\left(\sum_{x_{i}\in
A_{\theta}}f_{i}\right)L^{2\beta}\leq\sum_{x_{i}\in
A_{\theta}}f_{i}\mathrm{dist}\,(\nabla\psi(x_{i}),\partial\psi_{\delta}(x_{i}))^{2}\leq
C\lambda\delta.$

It remains to see that

$\sum_{x_{i}\in A_{\theta}}f_{i}=\mu\left(\cup_{x_{i}\in
A}V_{i}\right)\geq\mu(\frac{1}{2}A)\geq cL^{n}\theta^{n-1}$

if $\delta$ is small compared to the lengths of $A$, that is $\delta\leq\theta
L\sim L^{1+\beta}$. Inequality (80) then implies

$L^{\alpha}\leq C\lambda\delta$

with $\alpha=n(1+\beta)+\beta$. If $\delta\leq c\frac{L^{\alpha}}{\lambda}$,
that’s a contradiction. We prove that if $\delta\leq
c\min\\{\frac{L^{\alpha}}{\lambda},L^{1+\beta}\\}$, then we must have $t\geq
cL^{1+\beta}$, which implies the result.

∎

##  Appendix B Proof of Proposition 3.2

We will prove the following proposition, which implies in particular
Proposition 3.2.

######  Proposition B.1.

Given a non-negative Radon measure $\mu$ on $\mathbb{R}^{n}$, the following
statements are equivalent

There exists $\lambda,\Lambda>0$ and $\beta$ such that

$\lambda|B_{r}|\leq\mu(B_{r})\leq\Lambda|B_{r}|,\quad\mbox{ for all ball
$B_{r}\subset\Omega$ with $r\geq\beta\delta$}$

There exists $\lambda,\Lambda>0$ and $\beta$ such that

$\lambda|K|\leq\mu(K)\leq\Lambda|K|,\quad\mbox{ for all convex Borel
$K\subset\Omega$ with $\ell(K)\geq\beta\delta$}$

There exists $\lambda,\Lambda>0$ and $\beta$ such that

$\lambda|A_{\delta}|\leq\mu(A_{\delta})\leq\Lambda|A_{\delta}|,$

for any Borel set $A$ and with $A_{\delta}=\cup_{x\in
A}(S_{\delta}(x))\subset\Omega$ where $\\{S_{\delta}(x)\\}_{x\in A}$ is a
family of convex sets satisfying $x\in S_{\delta}$,
$\ell(S_{\delta})\geq\beta\delta$, and for which Vitali’s covering lemma (with
constant $C_{*}$) can be applied.

Furthermore, the constant $\lambda$, $\Lambda$ and $\beta$ appearing in these
statements differ by a factor depending only on the dimension (and $C_{*}$ for
the last one).

###### Proof.

It is clear that

$(iii)\Rightarrow(ii)\Rightarrow(i).$

We will assume that $(i)$ holds with $\beta=1$ and prove that it implies
$(ii)$ (for some $\beta>1$). In view of John’s Lemma 3.1, it is enough to
prove the result when $K=E$ is an ellipsoid with smallest radius larger than
$\beta\delta$. For such an ellipsoid, we have

(81) $E+B_{\delta}:=\cup_{x\in E}B_{\delta}(x)\subset(1+\frac{1}{\beta})E.$

Indeed, we have
$E=\\{x\in\mathbb{R}^{n}\,;\,\left(\sum_{i=1}^{n}\left(\frac{x_{i}}{a_{i}}\right)^{2}\right)^{1/2}\leq
1\\}$ with $a_{i}\geq\beta\delta$. If $y\in E+B_{\delta}$, there exists $x\in
E$ such that $|x-y|\leq\delta$ and so Minkowski inequality yields

$\begin{split}\left(\sum_{i=1}^{n}\left(\frac{y_{i}}{a_{i}}\right)^{2}\right)^{1/2}&\leq\left(\sum_{i=1}^{n}\left(\frac{x_{i}}{a_{i}}\right)^{2}\right)^{1/2}+\left(\sum_{i=1}^{n}\left(\frac{y_{i}-x_{i}}{a_{i}}\right)^{2}\right)^{1/2}\leq
1+\frac{1}{\beta\delta}\left(\sum_{i=1}^{n}\left(y_{i}-x_{i}\right)^{2}\right)^{1/2}\\\
&\leq 1+\frac{1}{\beta}.\end{split}$

Since $E$ is compact, Vitali’s covering Lemma implies the existence of a
covering $E\subset\cup_{i=1}^{k}B_{3\delta}(x_{i})$ with $x_{i}\in E$ and such
that the balls $B_{\delta}(x_{i})$ are disjoint. We then have

$\mu(E)\leq\sum_{i=1}^{k}\mu(B_{3\delta}(x_{i}))\leq\Lambda\sum_{i=1}^{k}|B_{3\delta}(x_{i})|=\Lambda
3^{n}\sum_{i=1}^{k}|B_{\delta}(x_{i})|=\Lambda
3^{n}|\cup_{i=1}^{k}B_{\delta}(x_{i})|\leq\Lambda 3^{n}|E+B_{\delta}|.$

We emphasize that we may not have $B_{3\delta}(x_{i})\subset\Omega$, but the
upper bound $\mu(B_{3\delta}(x_{i}))\leq\Lambda C|B_{\delta}(x_{i})|$ still
holds). Finally (81) implies that

$\mu(E)\leq\Lambda 3^{n}(1+\frac{1}{\beta})^{n}|E|.$

To prove the other inequality, we consider the ellipsoid
$\tilde{E}=(1-\frac{1}{\beta})E=\frac{\beta-1}{\beta}E$ (here, we assume that
$\beta>1$). This ellipsoid satisfies $\ell(\tilde{E})\geq(\beta-1)\delta$ and
so (81) implies

(82)
$\tilde{E}+B_{\delta}\subset(1+\frac{1}{\beta-1})\tilde{E}\subset\frac{\beta}{\beta-1}\tilde{E}=E$

We now consider a covering of
$\tilde{E}\subset\cup_{i=1}^{k}B_{3\delta}(x_{i})$ with $x_{i}\in\tilde{E}$
and such that the balls $B_{\delta}(x_{i})$ are disjoint. We then have (using
(82), which guarantees in particular that $B_{\delta}(x_{i})\subset
E\subset\Omega$):

$\mu(E)\geq\mu(\tilde{E}+B_{\delta})\geq\mu(\cup_{i=1}^{k}B_{\delta}(x_{i}))=\sum_{i=1}^{k}\mu(B_{\delta}(x_{i}))\geq\lambda\sum_{i=1}^{k}|B_{\delta}(x_{i})|\geq\frac{\lambda}{3^{n}}\sum_{i=1}^{k}|B_{3\delta}(x_{i})|\geq\frac{\lambda}{3^{n}}|\tilde{E}|.$

The definition of $\tilde{E}$ implies

$\mu(E)\geq\frac{\lambda}{3^{n}}|(1-\frac{1}{\beta})E|=\frac{\lambda}{3^{n}}(1-\frac{1}{\beta})^{n}|E|.$

We conclude that $(ii)$ holds, with (for example) $\beta=2$.

We now prove that $(ii)$ implies $(iii)$ by using Vitali’s covering lemma:
There exists a universal constant $C_{*}$ and a set of points $x_{i}\in A$
such that the $S_{\delta}(x_{i})$ are disjoint and
$A_{\delta}\subset\cup_{i}C_{*}S_{\delta}(x_{i})$. We then have (using $(ii)$
since the $C_{*}S_{\delta}(x_{i})$ are convex):

$\mu(A_{\delta})\leq\sum_{i}\mu(C_{*}S_{\delta}(x_{i}))\leq\Lambda\sum_{i}|C_{*}S_{\delta}(x_{i})|\leq\Lambda
C_{*}^{n}\sum_{i}|S_{\delta}(x_{i})|=\Lambda
C_{*}^{n}|\cup_{i}S_{\delta}(x_{i})|\leq\Lambda C_{*}^{n}|A_{\delta}|.$

and

$\mu(A_{\delta})\geq\mu(\cup_{i}S_{\delta}(x_{i}))=\sum_{i}\mu(S_{\delta}(x_{i}))\geq\frac{\lambda}{C_{*}^{n}}\sum_{i}|C_{*}S_{\delta}(x_{i})|\geq\frac{\lambda}{C_{*}^{n}}|\cup_{i}C_{*}S_{\delta}(x_{i})|\geq\frac{\lambda}{C_{*}^{n}}|A_{\delta}|.$

∎

##  Appendix C  Restriction of the Kantorovich potential

Let $\psi:\mathbb{R}^{n}\to(-\infty,+\infty]$ be a Kantorovich potential given
by Theorem 1.1. That is $\psi$ is a convex lower-semi-continuous function and
there exists an optimal transference plan $\pi$ concentrated in the graph of
$\partial\psi$. We denote $\phi=\psi^{*}$ its Legendre transform. The
construction below is classical (see for instance [3]) and recalled here for
the reader’s convenience.

Given two sets $X$ and $Y$ such that $\mathrm{spt}\,\mu\subset
X\subset\mathbb{R}^{n}$ and $\mathrm{spt}\,\nu\subset Y\subset\mathbb{R}^{n}$,
we define

$\tilde{\phi}(y)=\begin{cases}\phi(y)&\mbox{ if there exists $x\in X$ such
that $y\in\partial\psi(x)$ }\\\ +\infty&\mbox{ otherwise}\end{cases}$

and

$\tilde{\psi}(x)=\sup_{y\in Y}(x\cdot y-\tilde{\phi}(y))$

(the function $\tilde{\phi}$ and $\tilde{\psi}$ are defined in
$\mathbb{R}^{n}$).

We then have the following:

######  Proposition C.1.

The function $\tilde{\psi}$ is a Kantorovich potential for the optimal
transference plan $\pi$ and satisfies

$\tilde{\psi}(x)=\psi(x)\mbox{ for all $x\in\mathrm{spt}\,\mu$}$

and $\partial\tilde{\psi}(\mathbb{R}^{n})\subset{\Gamma(Y)}$, the closed
convex envelop of $Y$.

If we take $Y=\mathrm{spt}\,\nu$, we get
$\tilde{\psi}(x)=\sup_{y\in\mathrm{spt}\,\nu}\left(\tilde{\phi}(y)+x\cdot
y\right)$ for some function $\phi:\mathrm{spt}\,\nu\to\overline{\mathbb{R}}$
and
$\partial\tilde{\psi}(\mathbb{R}^{n})\subset\Gamma(\mathrm{spt}\,\nu)\subset\overline{\mathcal{O}}$,
where $\Gamma(\mathrm{spt}\,\nu)$ denotes the closed convex envelop of
$\mathrm{spt}\,\nu$.

###### Proof.

The fact that $\tilde{\psi}$ is convex and lower-semicontinuous follows from
its definition and since $\tilde{\phi}(y)\geq\phi(y)$, we have
$\tilde{\psi}(x)\leq\psi(x)$.

Furthermore, if $x\in X$ is such that there exists $y\in Y$ such that
$y\in\partial\psi(x)$, then $\tilde{\phi}(y)=\phi(y)$ and so

$\tilde{\psi}(x)\geq x\cdot y-\tilde{\phi}(y)=x\cdot y-\phi(y)=x\cdot
y-\psi^{*}(y)=\psi(x)$

(the last equality follows from (13)).

Finally, given $(x,y)\in\mathrm{Graph}(\partial\psi)\cap(X\times Y)$, we now
have both $\tilde{\phi}(y)=\phi(y)$ and $\tilde{\psi}(x)=\psi(x)$ and so

$\tilde{\psi}(x)-x\cdot y=\psi(x)-x\cdot
y=-\psi^{*}(y)=-\phi(y)=-\tilde{\phi}(y)\leq\tilde{\psi}(z)-z\cdot y$

for all $z\in\mathbb{R}^{n}$. This inequality can also be written as
$\tilde{\psi}(z)\geq\tilde{\psi}(x)+y\cdot(z-y)$ which means
$y\in\partial\tilde{\psi}(x)$. We deduce

$\mathrm{Graph}(\partial\psi)\cap(X\times
Y)\subset\mathrm{Graph}(\partial\tilde{\psi}).$

Since $\pi\in\Pi(\mu,\nu)$ is concentrated in
$\mathrm{spt}\,\mu\times\mathrm{spt}\,\nu$, we have
$\mathrm{spt}\,(\pi)\subset\mathrm{Graph}(\partial\psi)\cap(X\times
Y)\subset\mathrm{Graph}(\partial\tilde{\psi})$ and so $\tilde{\psi}$ is a
Kantorovich potential (associated to the same transference plan $\pi$).

Finally, the fact that $\tilde{\psi}(x)=\psi(x)$ on the $x$-projection of
$\mathrm{Graph}(\partial\psi)\cap(X\times Y)$ implies that

$\tilde{\psi}(x)=\psi(x)\mbox{ in }\mathrm{spt}\,\mu.$

∎

## References

  * [1] Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyré.  Iterative Bregman projections for regularized transportation problems.  SIAM J. Sci. Comput., 37(2):A1111–A1138, 2015.
  * [2] Jean-David Benamou and Vincent Duval.  Minimal convex extensions and finite difference discretisation of the quadratic Monge-Kantorovich problem.  European J. Appl. Math., 30(6):1041–1078, 2019.
  * [3] Robert J. Berman.  Convergence rates for discretized Monge-Ampère equations and quantitative stability of optimal transport.  Found. Comput. Math., 21(4):1099–1140, 2021.
  * [4] J. Bourgain and V. D. Milman.  New volume ratio properties for convex symmetric bodies in ${\bf R}^{n}$.  Invent. Math., 88(2):319–340, 1987.
  * [5] Yann Brenier.  Décomposition polaire et réarrangement monotone des champs de vecteurs.  C. R. Acad. Sci. Paris Sér. I Math., 305(19):805–808, 1987\.
  * [6] Yann Brenier.  Polar factorization and monotone rearrangement of vector-valued functions.  Communications on pure and applied mathematics, 44(4):375–417, 1991\.
  * [7] L. Caffarelli.  The regularity of mappings with convex potentials.  In Partial differential equations and related subjects (Trento, 1990), volume 269 of Pitman Res. Notes Math. Ser., pages 53–58. Longman Sci. Tech., Harlow, 1992.
  * [8] Luis A. Caffarelli.  Interior $W^{2,p}$ estimates for solutions of the Monge-Ampère equation.  Ann. of Math. (2), 131(1):135–150, 1990.
  * [9] Luis A Caffarelli.  A localization property of viscosity solutions to the monge-ampère equation and their strict convexity.  Annals of Mathematics, 131(1):129–134, 1990.
  * [10] Luis A Caffarelli.  Some regularity properties of solutions of monge-ampère equation.  Communications on pure and applied mathematics, 44(8-9):965–969, 1991.
  * [11] Luis A. Caffarelli.  Some regularity properties of solutions of Monge Ampère equation.  Comm. Pure Appl. Math., 44(8-9):965–969, 1991.
  * [12] Luis A Caffarelli.  Boundary regularity of maps with convex potentials.  Communications on pure and applied mathematics, 45(9):1141–1151, 1992.
  * [13] Luis A Caffarelli.  The regularity of mappings with a convex potential.  Journal of the American Mathematical Society, 5(1):99–104, 1992\.
  * [14] Luis A Caffarelli.  Boundary regularity of maps with convex potentials–ii.  Annals of mathematics, 144(3):453–496, 1996.
  * [15] G. De Philippis, A. Figalli, and O. Savin.  A note on interior $W^{2,1+\varepsilon}$ estimates for the Monge-Ampère equation.  Math. Ann., 357(1):11–22, 2013.
  * [16] Guido De Philippis.  Regularity of optimal transport maps and applications, volume 17 of Tesi. Scuola Normale Superiore di Pisa (Nuova Series) [Theses of Scuola Normale Superiore di Pisa (New Series)].  Edizioni della Normale, Pisa, 2013.
  * [17] Guido De Philippis and Alessio Figalli.  $W^{2,1}$ regularity for solutions of the Monge-Ampère equation.  Invent. Math., 192(1):55–69, 2013.
  * [18] Guido De Philippis and Alessio Figalli.  Partial regularity for optimal transport maps.  Publ. Math. Inst. Hautes Études Sci., 121:81–112, 2015.
  * [19] Alessio Figalli.  Regularity properties of optimal maps between nonconvex domains in the plane.  Comm. Partial Differential Equations, 35(3):465–479, 2010.
  * [20] Alessio Figalli and Young-Heon Kim.  Partial regularity of Brenier solutions of the Monge-Ampère equation.  Discrete Contin. Dyn. Syst., 28(2):559–565, 2010.
  * [21] Liliana Forzani and Diego Maldonado.  Properties of the solutions to the Monge-Ampère equation.  Nonlinear Anal., 57(5-6):815–829, 2004.
  * [22] M. Goldman, F. Otto, and M. Huesmann.  A large-scale regularity theory for the monge-ampere equation with rough data and application to the optimal matching problem.  arXiv:1808.09250, 2018.
  * [23] Michael P Goldman and Felix Otto.  A variational proof of partial regularity for optimal transportation maps.  arXiv:1704.05339, 2017.
  * [24] Cristian E. Gutiérrez.  The Monge-Ampère equation, volume 44 of Progress in Nonlinear Differential Equations and their Applications.  Birkhäuser Boston, Inc., Boston, MA, 2001.
  * [25] Cristian E. Gutiérrez and Qingbo Huang.  Geometric properties of the sections of solutions to the Monge-Ampère equation.  Trans. Amer. Math. Soc., 352(9):4381–4396, 2000.
  * [26] P.-E. Jabin, A. Mellet, and M. Molina-Fructuoso.  Local regularity result for an optimal transportation problem with rough measures in the plane.  J. Funct. Anal., 281(2):Paper No. 109041, 43, 2021.
  * [27] Jun Kitagawa, Quentin Mérigot, and Boris Thibert.  Convergence of a Newton algorithm for semi-discrete optimal transport.  J. Eur. Math. Soc. (JEMS), 21(9):2603–2651, 2019.
  * [28] Bruno Lévy.  A numerical algorithm for $L_{2}$ semi-discrete optimal transport in 3D.  ESAIM Math. Model. Numer. Anal., 49(6):1693–1715, 2015.
  * [29] Wenbo Li and Ricardo H. Nochetto.  Quantitative stability and error estimates for optimal transport plans.  IMA J. Numer. Anal., 41(3):1941–1965, 2021.
  * [30] Quentin Mérigot, Alex Delalande, and Frédéric Chazal.  Quantitative stability of optimal transport maps and linearization of the 2-wasserstein space, 2019.
  * [31] Quentin Mérigot and Boris Thibert.  Optimal transport: discretization and algorithms.  In Geometric partial differential equations. Part II, volume 22 of Handb. Numer. Anal., pages 133–212. Elsevier/North-Holland, Amsterdam, [2021] ©2021.
  * [32] Quentin Mérigot.  A multiscale approach to optimal transport.  Computer Graphics Forum, 30(5):1583–1592, 2011.
  * [33] Adam M. Oberman and Yuanlong Ruan.  An efficient linear programming method for optimal transportation, 2015\.
  * [34] Elias M. Stein.  Harmonic analysis: real-variable methods, orthogonality, and oscillatory integrals, volume 43 of Princeton Mathematical Series.  Princeton University Press, Princeton, NJ, 1993.  With the assistance of Timothy S. Murphy, Monographs in Harmonic Analysis, III.
  * [35] Cédric Villani.  Topics in optimal transportation, volume 58 of Graduate Studies in Mathematics.  American Mathematical Society, Providence, RI, 2003.
  * [36] Cédric Villani.  Optimal transport: old and new, volume 338.  Springer Science & Business Media, 2008.
  * [37] Yifeng Yu.  Singular set of a convex potential in two dimensions.  Comm. Partial Differential Equations, 32(10-12):1883–1894, 2007\.