# Periodic orbits in deterministic discrete-time evolutionary game dynamics:
An information-theoretic perspective

Sayak Bhattacharjee<EMAIL_ADDRESS>Department of Physics, Indian Institute
of Technology Kanpur, Uttar Pradesh 208016, India.  Vikash Kumar Dubey
<EMAIL_ADDRESS>Department of Physics, Indian Institute of Technology
Kanpur, Uttar Pradesh 208016, India.  Archan Mukhopadhyay
<EMAIL_ADDRESS>School of Computer Science, University of
Birmingham, Birmingham B15 2TT, United Kingdom  Sagar Chakraborty
<EMAIL_ADDRESS>Department of Physics, Indian Institute of Technology
Kanpur, Uttar Pradesh 208016, India.

###### Abstract

Even though existence of non-convergent evolution of the states of populations
in ecological and evolutionary contexts is an undeniable fact, insightful
game-theoretic interpretations of such outcomes are scarce in the literature
of evolutionary game theory. As a proof-of-concept, we tap into the
information-theoretic concept of relative entropy in order to construct a
game-theoretic interpretation for periodic orbits in a wide class of
deterministic discrete-time evolutionary game dynamics, primarily
investigating the two-player two-strategy case. Effectively, we present a
consistent generalization of the evolutionarily stable strategy—the
cornerstone of the evolutionary game theory—and aptly term the generalized
concept: information stable orbit. The information stable orbit captures the
essence of the evolutionarily stable strategy in that it compares the total
payoff obtained against an evolving mutant with the total payoff that the
mutant gets while playing against itself. Furthermore, we discuss the
connection of the information stable orbit with the dynamical stability of the
corresponding periodic orbit.

Evolutionary games, evolutionarily stable strategy, replicator map, escort-
incentive dynamics, periodic orbits, relative entropy.

##  I  Introduction

Many ecological systems are known to exhibit cyclic evolution of the abundance
of the constituent species—well-studied examples include the snowshoe-
hare–lynx system [1], wolf-moose system [2] and lemming populations [3]. The
simplest explanation for such a cyclic behaviour is mostly attributed to
predation, which may be modelled by the corresponding Lotka–Volterra equation
[4]. Since the Lotka–Volterra equation is mathematically mappable [5] onto the
replicator equation [6, 7]—a paradigmatic evolutionary dynamic in the theory
of evolutionary games [8]—the relevance of oscillatory dynamics in
evolutionary game theory is worth pondering.

One of the central ideas in evolutionary game theory is that of evolutionarily
stable strategy (ESS) [9], which happens to be Nash equilibrium [10, 11] as
well. Through the folk theorem (and other similar theorems) [12] of
evolutionary game theory, the convergent outcomes—represented by a fixed point
in phase space—of the replicator dynamics can be interpreted as Nash
equilibrium and ESS whenever achievable. These game-theoretic interpretations
of the fixed points can be further supplemented with interesting information-
theoretic connection: A fixed point that is an ESS is locally asymptotically
stable and an appropriately constructed Kullback–Leibler (KL) divergence (also
called relative entropy) [13] is a Lyapunov function [14] for the fixed point,
i.e., the nonnegative function decreases with the passage of time [15] until
the dynamics converges onto the fixed point eventually.

Nevertheless, there is a dearth of literature about the game-theoretic and the
information-theoretic interpretations for non-convergent oscillatory outcomes.
In this paper, we contribute to this area. For pragmatic reasons, it is rather
convenient to work with discrete-time replicator equation (replicator map) as
even with a population with two strategies (phenotypes), one can witness
cyclic behaviour [16, 17]. In fact, using the replicator map, it was shown
earlier [18] that a game-theoretic interpretation of periodic orbits is
possible—an extension of ESS, called heterogeneity stable orbit (HSO), was
proposed. However, unlike the ESS, the concept of the HSO does not consider
fitness as the quantity to be optimized during natural selection; rather it
concerns itself with the optimization of a weighted fitness—the weight being
heterogeneity. The heterogeneity is the probability that two arbitrarily
chosen members of the population belong to two different phenotypes. Although
a justification for the HSO could be found in its connection with the stable
periodic orbits just as ESS is connected with stable fixed points, going
beyond the well-accepted notion of fitness to a weighted fitness appears
unconventional. Moreover, it is not clear if HSO conforms with the principle
of decreasing relative entropy [13].

A natural question emerges at this point: Using the notion of decreasing
relative entropy during the evolution of the system, does there exist a
natural extension of ESS for the case of periodic orbits such that fitness
(sans any weight) is the object of optimization? We answer this question
affirmatively in this paper by the introduction of the extension of ESS termed
appropriately as Information Stable Orbit (ISO). We also find an encouraging
connection of this game-theoretic concept—obtained though information-
theoretic ideas—with the dynamical system theoretic concept of stability of
periodic orbits. Our investigation is not restricted to only the replicator
map, rather we expand our investigations to encompass a much wider class of
evolutionary dynamics like incentive [19] and escort-incentive dynamics [20]
(which account for many well-studied dynamics like logit [21] and best-reply
[22]).

Given the rather technical nature of the results in our work, below we first
provide a succinct outline of the key aspects of our work and the main
results.

###  I.1 Outline of the paper

As noted in the introduction, the central result of this work is a
generalization of the game-theoretic notion of evolutionarily stable
strategies (ESS)—traditionally only defined for fixed points—to periodic
outcomes in discrete-time evolutionary dynamics. To do this, we revisit the
relative entropy minimization principle for evolutionary dynamics [15, 23,
24]. As per this principle, if an evolving population state is close to an
equilibrium point, the relative entropy (quantified using the Kullback-Leibler
(KL) divergence, defined in Eq. (1)) monotonically decreases. This heuristic
principle may be considered as an application of Kullback’s general proposal
of the principle of minimum discrimination information—which has been used
successfully in various contexts [25, 26, 27]—to evolutionary systems. In
fact, this may be interpreted quite physically: Darwinian evolution ensures
that the amount of biological information produced during evolution, as
compared to the equilibrium state, is optimized [24].

In Sec. II, we explicitly demonstrate how minimization of the KL-divergence
yields the well-known game-theoretic stability criteria (such as ESS) for
fixed points in evolutionary dynamics. While such results have been explored
recently for continuous-time dynamics, we extend this to discrete-time
dynamics. In addition to the paradigmatic replicator map (see Eq. (2)), we
also discuss two generalized classes of evolutionary dynamics as well—the
incentive and escort-incentive dynamics. The incentive dynamic (Eq. (3))
accommodates for various updation protocols of incentive in evolutionary
games, while the escort-incentive dynamic (Eq. (8)) additionally generalizes
the averaging procedure in the incentive map.

Our main results are in Sec. III, where we extend this program for periodic
orbits in discrete-time evolutionary dynamics. A previous (purely game-
theoretic) endeavour [18] to construct a generalization of ESS for periodic
orbits in such dynamics was the Heterogeneity Stable Orbit (HSO) (see Eq.
(13)), which was also accompanied by a generalization of the Nash equilibrium,
known as the Heterogeneity Orbit (HO) (Eq. (11)). From a game-theoretic
perspective, the hope was straightforward: Just as an ESS state yields the
maximum payoff against an arbitrary state in its neighbourhood as compared to
the state playing against itself, a generalization of ESS for periodic
orbits—which would be a sequence of states—would achieve this in a
periodically averaged sense. In other words, we intuit that the periodic orbit
would yield the maximum total payoff when played against a sequence of
evolving states as opposed to the evolving states playing against themselves.
The HSO condition achieves this in a little weak fashion because it involves a
weighted payoff, where the weight measures the heterogeneity in the evolving
population state. Quite elegantly, our information-theoretic technique,
adopted in this paper, achieves our intuition exactly: We do not require a
weighted payoff and simply compare the sum of payoffs over a period; thus,
generalizing the notion of ESS for periodic orbits in the discrete-time
evolutionary dynamics. We dub this generalization the Information Stable Orbit
(ISO) and formulate it for the replicator map (Eq. (14)), the incentive map
(Eq. (17)) and the escort-incentive map (Eq.(18)).

Finally, before we conclude the work in Section IV, we analytically indicate
how an ISO may correspond to dynamically stable periodic orbit (in Sec. III.3)
and verify the analysis for various evolutionary dynamics numerically (in Sec.
III.4). Having presented the brief outline of the body of this paper, we now
discuss the technical details in the body of this paper without further ado.

##  II Interpreting Fixed Points

With a view to keeping the presentation of the main ideas of the paper
succinct, we confine ourselves to the simplest yet conceptually non-trivial
set-up of two-person two-strategy games. Specifically, we consider an infinite
unstructured population of two types whose frequencies change over time under
some replication-selection rule. Between the individuals (or the players, in
game-theoretic terminology), the interactions are assumed random and of two-
player kind. The frequencies of the $i$-th type is denoted by $x_{i}$ and
thus,
$\mathbf{x}=\left(x_{1},x_{2}\right)=\left(x_{1},1-x_{1}\right)=(x,1-x)$, in
different equivalent notations, is the frequency (or state) vector of the
population. Thus, throughout the paper, the states are restricted to the unit
interval, i.e., $0\leq x\leq 1$, as discussed briefly in subsection below.
Many kinds of dynamics for the evolution of the frequencies are studied in the
literature.

First, let us recall the paradigmatic continuous-time replicator dynamic [6,
7] in the light of information-theoretic ideas. Of particular use, is the
relative entropy or the Kullback–Leibler (KL) divergence given by

$D_{\textrm{KL}}(\mathbf{p}||\mathbf{q})\coloneqq\sum_{y}p(y)\ln[{p(y)}/{q(y)}]$
(1)

where $y$ belongs to the support of the probability distributions $\mathbf{p}$
and $\mathbf{q}$. The KL-divergence helps to measure the distance between two
probability distributions and herein, we use it as a measure of the distance
of an evolving population state from a fixed (equilibrium) population state.
Note that the states can be interpreted as distributions, even though the
dynamics is not stochastic. If the fixed population state is chosen as the ESS
state, the aforementioned KL-divergence decreases with time [15]. Given that
the KL-divergence fits the criteria for being a valid Lyapunov function, one
may also interpret this as a statement for dynamical stability. There exists,
however, an important nuance to this fact: While the local asymptotic
stability and evolutionary stability of a fixed point state are synonymous
only for two-strategy games, for games with higher number of strategies, ESS
implies local asymptotic stability of the corresponding fixed point but the
converse is not necessarily true [6].

###  II.1 Incentive Dynamics

In this paper, we are motivated to carry out our investigations with discrete-
time dynamics because they are a convenient test-bed for investigating non-
convergent outcomes like periodic orbits and chaos. In particular, the
following is a version of the discrete-time replicator dynamic [28, 29, 30,
22, 31, 32, 16, 17, 18, 33], henceforth referred to as the replicator map,
that exhibits periodic orbits and chaos even in the simple two-strategy case:

$\Delta x_{i}^{(k)}=x_{i}^{(k)}\left[f_{i}(\mathbf{x}^{(k)})-\left\langle
f(\mathbf{x}^{(k)})\right\rangle\right].$ (2)

Here $\Delta x_{i}^{(k)}\coloneqq x_{i}^{(k+1)}-x_{i}^{(k)}$ represents the
difference between consecutive population states and $f$ is the fitness
function for the types, with the average population fitness being given by
$\left\langle
f(\mathbf{x}^{(k)})\right\rangle\coloneqq\sum_{j}x_{j}^{(k)}f_{j}(\mathbf{x}^{(k)})$.
The superscript denotes the time step. We also note that as per our notation,
$f(\mathbf{x})$ denotes a vector with components $f_{i}(\mathbf{x})$. For this
discrete map, it can be shown that a locally asymptotically stable fixed point
implies ESS for two-strategy games [17, 34], without its converse being
necessarily true. To ensure the solutions of the above discrete replicator map
(and all such subsequently introduced maps) are physical (i.e., $0\leq
x^{(k)}_{i}\leq 1$ for all $i$ and $k$), we must restrict the parameter space
of the map (here, captured by the function $f$; for details, see Eq. (23)) to
the strict physical region, as defined in Ref. [17, 35].

The replicator map in Eq. (2) phenomenologically models the replication-
selection dynamics in line with the Darwinian tenet of natural selection. We
know [17] that its fixed points correspond to the Nash equilibria [36] and the
evolutionarily stable strategies or states [37] are associated with the folk
theorems of the evolutionary game theory [38], just as in the continuous-time
case. However, we remark that this form of the replicator equation should not
necessarily be seen just as an Euler discretization of the continuous-time
equation: It has its own existence unrelated to its counterpart continuous
equation. One can motivate this replicator map from viability selection rules
[31]. Also, this replicator equation appears to model intergenerational
cultural transmission [30, 32], boundedly rational players, imitational
behaviour in bimatrix cyclic games [29], and reinforcement learning [28].
Thus, it has applicability even beyond usual evolutionary contexts in the
biological systems.

In fact, a unification of quite a few evolutionary dynamics is mathematically
possible through the incentive dynamic [19, 39, 40, 41, 20, 42, 43, 43] whose
discrete-time representation is given by

$\Delta
x_{i}^{(k)}=x_{i}^{(k)}(\delta_{\varphi})^{(k)}_{i}\equiv\varphi_{i}(\mathbf{x}^{(k)})-x_{i}^{(k)}\sum_{j}\varphi_{j}(\mathbf{x}^{(k)}),$
(3)

where $\bm{\varphi}(\mathbf{x}^{(k)})$ is the incentive vector. The notation
$(\delta_{\varphi})^{(k)}_{i}$ has been introduced for future convenience. The
incentive dynamic arises by generalizing the notion of incentive for the
players of a game, accounting for different updating procedures utilized by
the players. Choice of different incentive functions, $\varphi$, correspond to
different dynamics (such as replicator, projection [44], logit [21], and best
response [45, 46]) as exhibited in Table 1.

Table 1: Incentive functions for various evolutionary dynamics. Evolutionary Dynamic | Incentive ($\varphi_{i}(\mathbf{x})$)
---|---
Projection | $f_{i}(\mathbf{x})$
Replicator | $x_{i}f_{i}(\mathbf{x})$
Logit |  $\frac{\textrm{exp}(\beta f_{i}(\mathbf{x}))}{\sum_{i}\textrm{exp}(\beta f_{i}(\mathbf{x}))}$111Here, $\beta$ is called the rationality factor, see [21].
Best Reply |  $x_{i}\textrm{BR}_{i}(\mathbf{x})$222$\textrm{BR}(\mathbf{x})$ is the best response function, a detailed discussion is given in [22].

A state of the population is an incentive stable state (ISS) [20],
$\hat{\mathbf{x}}$, if for any $\mathbf{x}$ in the local deleted neighbourhood
of $\hat{\mathbf{x}}$, we have

$\sum_{i}\hat{x}_{i}\frac{\varphi_{i}(\mathbf{x})}{x_{i}}>\sum_{i}\varphi_{i}(\mathbf{x}).$
(4)

ISS reduces to ESS for the replicator equation, given explicitly by
$\hat{\mathbf{x}}\cdot f(\mathbf{x})>\mathbf{x}\cdot f(\mathbf{x})$.

For the two-strategy incentive map—as a generalization of the corresponding
result of replicator map [17]—one can quite readily show using linear
stability analysis that a locally asymptotically stable fixed point obeys the
ISS condition. Moreover, it is an easy exercise to show that just like for the
replicator dynamic, the ISS for the continuous incentive dynamic implies that
the time derivative of the KL-divergence,
$D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x})$, is negative. In what follows,
we observe that negative discrete-time derivative of the KL-divergence,
defined as

$\Delta D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})\coloneqq
D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k+1)})-D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})$
(5)

implies the ISS state. The discrete-time derivative measures the change in
relative entropy between an evolving state and an equilibrium (fixed) state.
Notice,

$\displaystyle\Delta D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})$
$\displaystyle=-\sum_{i=1}^{2}\hat{x}_{i}\ln{\left(x_{i}^{(k+1)}/x_{i}^{(k)}\right)}$
(6a)
$\displaystyle=-\sum_{i}^{2}\hat{x}_{i}\ln{\left[1+(\delta_{\varphi})_{i}^{(k)}\right]}$
(6b)
$\displaystyle\geq-\ln{\left[1+\sum_{i}^{2}\hat{x}_{i}(\delta_{\varphi})_{i}^{(k)}\right]}.$
(6c)

The last inequality is obtained using Jensen’s inequality [47, 13] for convex
functions. Now, for decreasing KL-divergence, we should have $\Delta
D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})<0$, or,
$\sum_{i}\hat{x}_{i}(\delta_{\varphi})^{(k)}_{i}>0$. This implies,
$\sum_{i}\varphi_{i}(\mathbf{x}^{(k)})-\sum_{i}\hat{x}_{i}\left(\varphi_{i}(\mathbf{x}^{(k)})/x_{i}^{(k)}\right)<0$
for all $\mathbf{x}^{(k)}$ in the deleted neighbourhood of $\hat{\mathbf{x}}$,
which is the ISS condition. Note that if one could interpret the KL-divergence
as the discrete-time Lyapunov function [48, 49] for this dynamic, this result
shows that the stable fixed point implies ISS for two-strategy games, which is
in line with the result known using linear stability analysis. Furthermore, an
extension to higher $(n>2)$ strategy games is immediate: the superscripts in
the summations can be replaced by $n$ in Eq. (6).

For the results on information-theoretic interpretations of stability
conditions in evolutionary dynamics in the subsequent parts of the paper, we
will follow a line of reasoning similar to Eq. (6), with the details of the
mathematical steps relegated to the Appendices.

###  II.2 Escort-Incentive Dynamics

There is yet another way the replicator dynamic can be generalized.
Specifically, consider the discrete $q$-deformed replicator dynamic [50] given
by

$\Delta x_{i}^{(k)}=(x_{i}^{(k)})^{q}(f_{i}(\mathbf{x}^{(k)})-\langle
f(\mathbf{x}^{(k)})\rangle_{q})$ (7)

where $q$ is a positive real number and $\langle
f(\mathbf{x}^{(k)})\rangle_{q}:=\sum_{j}(x_{j}^{(k)})^{q}f_{j}(\mathbf{x}^{(k)})/\sum_{j}(x_{j}^{(k)})^{q}$
is the $q$-generalized mean. Clearly, setting $q\rightarrow 1$ reduces this
dynamic to the standard replicator dynamic. Eq. (7) is, in turn, a special
case of a larger set of dynamics given by the following discrete-time escort-
incentive dynamic [20]:

$\Delta
x_{i}^{(k)}=\varphi_{i}(\mathbf{x}^{(k)})-\tilde{\sigma}_{i}(\mathbf{x}^{(k)})\sum_{j}\varphi_{j}(\mathbf{x}^{(k)}),$
(8)

where the escort distribution vector,
$\bm{\tilde{\sigma}}(\mathbf{x}^{(k)})=(\tilde{\sigma}_{1}(\mathbf{x}^{(k)}),\tilde{\sigma}_{2}(\mathbf{x}^{(k)}))$
is defined as
$(\sum_{i=1}^{2}\sigma(x_{i}))^{-1}\left(\sigma(x_{1}),\sigma(x_{2})\right)$.
Nondecreasing and strictly positive on $(0,1)$, the escort function, $\sigma$
(appropriately normalized), maps a discrete probability distribution into
itself. In our context, the escort distributions, which are often relevant in
nonextensive statistical systems [51], help further generalize the notion of
mean—used, e.g., in defining mean fitness of a population. Different choices
for different escort and incentive functions correspond to different dynamics,
such as replicator, $q$-deformed replicator, projection, and exponential
escort (see Table 2). It is interesting to note that this generalization of
the replicator dynamic is actually motivated through the information-theoretic
framework [20]. In fact, Eq. (3) is a special case of Eq. (8) (when
$\sigma_{i}(\mathbf{x})=x_{i}$).

Table 2: Escort and incentive function combinations for various evolutionary dynamics. Evolutionary dynamic | Incentive ($\varphi_{i}(\mathbf{x})$) | Escort ($\sigma_{i}(\mathbf{x})$)
---|---|---
Replicator | $x_{i}f_{i}(\mathbf{x})$ | $x_{i}$
Replicator with selection |  $\beta x_{i}f_{i}(\mathbf{x})$333Here, $\beta$ is the intensity of selection in the dynamic. | $\beta x_{i}$
$q$-deformed replicator | $x_{i}^{q}f_{i}(\mathbf{x})$ |  $x_{i}^{q}$444Here, $q=1$ will give the replicator dynamic.
Exponential escort | $e^{x_{i}}f_{i}(\mathbf{x})$ | $e^{x}$

It is natural that the concepts of ESS and ISS can be further extended for the
escort-incentive dynamics: A state of the population is an escort incentive
stable state (EISS) [20], $\hat{\mathbf{x}}$, if for any $\mathbf{x}$ in the
local deleted neighbourhood of $\hat{\mathbf{x}}$, we have

$\sum_{i}\hat{x}_{i}\frac{\varphi_{i}(\mathbf{x})}{\sigma_{i}(\mathbf{x})}>\sum_{i}x_{i}\frac{\varphi_{i}(\mathbf{x})}{\sigma_{i}(\mathbf{x})}.$
(9)

Again, for the two-strategy escort-incentive map, using linear stability
analysis, it is an easy exercise to show—as a generalization of the
corresponding result of replicator map [17]—that a locally asymptotically
stable fixed point of the two-strategy escort-incentive map is an EISS.

It stands proven in the literature [50] that the escort divergence is the
Lyapunov function for the continuous escort-replicator equation (which is an
alternate escort distribution motivated evolutionary dynamic). We recall that
the escort divergence, $D_{\sigma}(\mathbf{x}||\mathbf{x}^{\prime})$—a
generalization of the KL-divergence—is given by
$D_{\sigma}(\mathbf{x}||\mathbf{x}^{\prime})\coloneqq\sum_{i}\int_{x^{\prime}_{i}}^{x_{i}}\left(\log_{\sigma}(v)-\log_{\sigma}(x^{\prime}_{i})\right)\textrm{d}v$,
where $\log_{\sigma}(x)\coloneqq\int_{1}^{x}{\sigma(u)}^{-1}\textrm{d}u$ is
called the escort logarithm. Motivated by this result, it is natural to check
what negative discrete-time derivative of the escort divergence, defined as
$\Delta D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})\coloneqq
D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k+1)})-D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})$,
implies and how it is connected with a fixed point, $\hat{\mathbf{x}}$, that
is an EISS. Here, demanding $\Delta
D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})<0$ implies

$\sum_{i}\hat{x}_{i}\frac{\varphi_{i}(\mathbf{x})}{\sigma_{i}(\mathbf{x})}>\sum_{i}x_{i}\frac{\varphi_{i}(\mathbf{x})}{\sigma_{i}(\mathbf{x})}-C_{\sigma}(\hat{\textbf{x}},k)$
(10)

where $C_{\sigma}(\mathbf{\hat{x}},k)$ is a non-negative correction term
(specific to two types) that arises because of the choice of bounds of the
integrals. The details of this computation and the explicit form of
$C_{\sigma}(\mathbf{\hat{x}},k)$ are provided in Appendix A. Since the
correction term is non-negative, the obtained condition is automatically
satisfied by the EISS condition in Eq. (9).

In summary, we have understood how the decrease in an information-theoretic
divergence, namely, the escort divergence (of which KL-divergence is a special
case) corresponds to the fixed points that are ESS, ISS, and EISS of a wide
class of discrete-time evolutionary dynamics.

##  III Interpreting Periodic orbits

Non-convergent outcomes, like periodic orbits, are quite common in
evolutionary dynamics. In the discrete-time dynamics discussed in the
preceding section, such outcomes [16, 17, 52, 18] are possible even with only
two-strategy games, thus rendering the investigations about them analytically
tractable. If we represent a finite sequence of $m$ states representing
iterates of a map by $\\{\mathbf{x}^{(k)}\\}_{k=1}^{k=m}$ (where $k$ denotes
the time step), then a periodic orbit of orbit $m$ (i.e., an $m$-period orbit)
can be denoted by an infinite sequence $\\{\mathbf{x}^{(k)}\\}_{k\geq 1}$ such
that $\mathbf{x}^{(k)}=\mathbf{x}^{(k+m)}$, for any $k\geq 1$, where $m$ is
the least (also known as prime) period of the orbit. Since a periodic orbit
only has $m$ distinct states, it may be compactly denoted by a finite sequence
given by $\\{\mathbf{x}^{(k)}\\}_{k=1}^{k=m}$—a time-ordered collection of
$m$-period points. Now we intend to show that the information-theoretic
concept of KL-divergence (or more generally the escort divergence) presents a
unique viewpoint that helps to construct a game-theoretic interpretation for
the non-convergent dynamical equilibrium, specifically periodic orbit.

It is imperative to bring to the readers’ attention that an exercise of
imparting game-theoretic meaning to the periodic orbits was performed in Ref.
[18]. We first critically revisit that work and scrutinize the concepts
related to the periodic orbits within the game-theoretic framework. We confine
ourselves only to the standard replicator map in this discussion.

###  III.1 Scrutiny of Heterogeneity Stable Orbit

Two definitions were introduced [18] as extensions of Nash equilibrium and
evolutionarily stable strategy, namely, heterogeneity orbit (HO) and
heterogeneity stable orbit (HSO). We formally define them below.

Definition 1a (HO for replicator map). A sequence of $m$ distinct states
$\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$ is an HO of order $m$ if for all states
$\hat{\mathbf{x}}^{(j)}$ $(1\leq j\leq m)$ of this sequence,

$\sum_{k=1}^{m}H_{{\hat{\bf x}}^{(k)}}\Big{[}\hat{\mathbf{x}}^{(j)}\cdot
f(\mathbf{\hat{x}}^{(k)})\Big{]}=\sum_{k=1}^{m}H_{\hat{\bf
x}^{(k)}}\Big{[}\mathbf{x}\cdot f(\mathbf{\hat{x}}^{(k)})\Big{]}.$ (11)

Here, heterogeneity factor, $H_{\mathbf{x}}\equiv 2x(1-x)$, is the probability
that two arbitrarily chosen members of the population belong to two different
types. The condition of HO is equivalent to the condition of periodicity given
by

$\sum_{j=k}^{k+m-1}\mathbf{\hat{x}}^{(j)}(1-\mathbf{\hat{x}}^{(j)})\left[f_{1}(\mathbf{\hat{x}}^{(j)})-f_{2}(\mathbf{\hat{x}}^{(j)})\right]=0,$
(12)

where $\mathbf{\hat{x}}^{(k)}$ ($1\leq k\leq m$), is any one of the $m$ states
of an $m$-periodic orbit. For $m=1$, it reduces to the condition of the Nash
equilibrium—$\mathbf{\hat{x}}\cdot f(\mathbf{\hat{x}})=\mathbf{x}\cdot
f(\mathbf{\hat{x}})$ for any $\mathbf{x}$ in the interior of the simplex.

Definition 1b (HSO for replicator map). An HSO of order $m$ is a sequence of
$m$ distinct states, $\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$ such that

$\sum_{k=1}^{m}H_{{\bf{x}}^{(k)}}\big{[}\hat{\mathbf{x}}^{(1)}\cdot
f(\mathbf{{x}}^{(k)})\Big{]}>\sum_{k=1}^{m}H_{{\bf{x}}^{(k)}}\big{[}{\mathbf{x}}^{(1)}\cdot
f(\mathbf{{x}}^{(k)})\Big{]},$ (13)

for any sequence of states $\\{{\bf x}^{(k)}\\}_{k=1}^{k=m}$ of the map
starting in some infinitesimal deleted neighbourhood of $\hat{{\bf x}}^{(1)}$.
The HSO reduces to ESS for $m=1$ and extends the concept of ESS to periodic
orbits.

Using linear stability, one can show that all locally asymptotically stable
periodic orbits obey the HSO criterion, although the converse is not
necessarily true [18]. Moreover, HSO implies HO, just as ESS implies Nash
equilibrium. To truly interpret the condition game-theoretically, however, a
connection to underlying strategy space is necessary; and in this context, it
can be shown that just as the ESS condition is associated with the idea of
strong stability [5], HSO may also be associated with the extension of strong
stability for periodic orbits [18].

Two criticisms of the definition of the HSO are in order. First, the
introduction of the heterogeneity factor is somewhat ad hoc. Second, a close
inspection reveals that the HSO condition is a bit odd in the following sense:
The total payoff on the right hand side of Eq. (13) is not the sum of the
evolving mutant state playing with itself, but instead is the sum of payoffs
of the mutant playing against a temporally fixed mutant state chosen [note the
fixed $(1)$ superscript on the right hand side of Eq. (13)]. In this paper, we
circumvent the aforementioned two drawbacks by introducing a new
generalization of the ESS.

###  III.2 Information Stable Orbit

Let us generalize the concept of ESS to define information stable orbit (ISO)
as follows:

Definition 2a (ISO for replicator map). A sequence of $m$ distinct states,
$\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$, is ISO of order $m$ of the replicator
map if

$\sum_{k=1}^{m}\hat{\mathbf{x}}^{(1)}\cdot
f(\mathbf{x}^{(k)})>\sum_{k=1}^{m}\mathbf{x}^{(k)}\cdot f(\mathbf{x}^{(k)}),$
(14)

for any sequence of $m$ states, $\\{{\bf x}^{(k)}\\}_{k=1}^{k=m}$, of the map
starting in some infinitesimal deleted neighbourhood of $\hat{{\bf x}}^{(1)}$.

The definition of the ISO of order $m$ qualifies any given finite sequence of
$m$ states of a map, $\\{\mathbf{\hat{x}}^{(k)}\\}_{k=1}^{k=m}$, by comparing
it—using Eq. (14)—with a finite sequence of $m$ mutant states,
$\\{\mathbf{x}^{(k)}\\}_{k=1}^{k=m}$, also governed by the map. In doing so,
only the first element of the given sequence, $\mathbf{\hat{x}}^{(1)}$, is
explicitly used in the inequality, with the implicit knowledge that the
remaining elements of this sequence are specified by the map exactly. The
sequence for mutant states starts from the deleted neighbourhood of the first
element of the ISO, and the remaining elements of this sequence are also
determined exactly by the map. This formulation of a stability criterion for
periodic orbits is inspired by the previously defined HSO [18].

Note that explicitly, Eq. (14) is blind to the notion of a periodic orbit.
Thus, in order to qualify a periodic orbit by ISO, we must ensure that each of
the $m$ sequences of length $m$ starting from each of the $m$ distinct states
of an $m$-periodic orbit should obey the ISO condition. Here, all the $m$
sequences $\\{\hat{\mathbf{x}}^{(j)}\\}_{j=k}^{j=k+m-1}$ equivalently denote
the same $m$-periodic orbit and corresponding nearby sequences
$\\{{\mathbf{x}}^{(j)}\\}_{j=k}^{j=k+m-1}$ start in the infinitesimal deleted
neighbourbood of $\hat{\mathbf{x}}^{(k)}$. Hence, we can recast the ISO
condition (Eq. (14)) for periodic orbits as

$\sum_{j=k}^{k+m-1}\hat{\mathbf{x}}^{(k)}\cdot
f(\mathbf{x}^{(j)})>\sum_{j=k}^{k+m-1}\mathbf{x}^{(j)}\cdot
f(\mathbf{x}^{(j)}),$ (15)

for all $m$ states $\hat{\mathbf{x}}^{(k)}$ of the sequence
$\\{\hat{\mathbf{x}}^{(k)}\\}_{k=1}^{k=m}$. Note that the ISO reduces to the
ESS for $m=1$, which is consistent with the fact that a fixed point can be
understood as a 1-period orbit.

Since any state of an $m$-period orbit of a nonlinear map is a fixed point of
the $m$-th iterate of the map, it is very natural to ask if the condition of
decrease in the KL-divergence leads to an extension of ESS, either HSO or ISO;
after all, we have seen earlier that $\Delta
D_{\textrm{KL}}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})<0$ leads to ESS [see Eqs.
(6)a–c]. To this end, we introduce the notation,

$\Delta_{m}D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})\equiv
D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k+m)})-D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})$
(16)

which measures the change in the KL-divergence over $m$ time steps. Thus, it
is a measure of the total change in the relative entropy over a time period of
a periodic orbit of the replicator map. Using Jensen’s inequalities as used in
the fixed-point computation (see Eq. (6)), we find that demanding
$\Delta_{m}D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})<0$
implies the ISO condition as given in Eq. (15) and not the condition of HSO.
The details of this proof are provided in Appendix B.

Thus, this information-theoretic fact justifies the aptness of the name ISO.
One should observe the unidirectionality in the arguments presented in
Appendix B. In particular, just as there exist some unstable fixed points that
are ESS in the discrete replicator map, we may also find some unstable
periodic orbits that obey the ISO condition. This unidirectionality is also
true for the HSO condition and appears to be an artefact of the temporal
discretization.

As can be guessed, the generalization of the ISO for the incentive dynamics is
straightforward:

Definition 2b (ISO for incentive map). A sequence of $m$ distinct states,
$\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$, is ISO of order $m$ of the incentive
map if

$\displaystyle\sum_{k=1}^{m}\sum_{i}\hat{x}_{i}^{(1)}\frac{\varphi_{i}(\mathbf{x}^{(k)})}{x_{i}^{(k)}}>\sum_{k=1}^{m}\sum_{i}\varphi_{i}(\mathbf{x}^{(k)})$
(17)

for any sequence of states, $\\{{\bf x}^{(k)}\\}_{k=1}^{k=m}$, of the map
starting in some infinitesimal deleted neighbourhood of $\hat{{\bf x}}^{(1)}$.

We observe that this condition reduces to ISO on setting
$\varphi(\mathbf{x})=xf(\mathbf{x})$ as expected for the replicator map. Also,
$m=1$ reproduces the ISS condition (Eq. (4)) as required. We remark that for
the condition in Eq. (17) to hold for periodic orbits, we can recast it into a
more general form similar to Eq. (15). Then, just like for the ISO of the
replicator map, here one can show that the decreasing KL-divergence
constructed using a periodic orbit and its neighbouring orbit implies the ISO
criterion for incentive map—to prove so, all one has to do is to replace
$\delta_{xf}$ by more general $\delta_{\varphi}$ in the proof in Appendix B.

As in the discussion for fixed points, we now generalize ISO for the escort-
incentive dynamic.

_Definition 2c (ISO for escort-incentive map)_. A sequence of $m$ distinct
states, $\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$, is ISO of order $m$ of the
escort-incentive map if

$\displaystyle\sum_{k=1}^{m}\sum_{i}\hat{x}_{i}^{(1)}\frac{\varphi_{i}(\mathbf{x}^{(k)})}{\sigma_{i}(\mathbf{x}^{(k)})}>\sum_{k=1}^{m}\sum_{i}x_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(k)})}{\sigma_{i}(\mathbf{x}^{(k)})}$
(18)

for any sequence of states, $\\{{\bf x}^{(k)}\\}_{k=1}^{k=m}$, of the map
starting in some infinitesimal deleted neighbourhood of $\hat{{\bf x}}^{(1)}$.

The condition in Eq. (18) can also be recast to a form like in Eq. (15) for
periodic orbits. Here, in the context of the information-theoretic
interpretation of this condition that is an $m$-periodic orbit, the
appropriate information-theoretic concept is the escort divergence. Hence, we
introduce the notation,
$\Delta_{m}D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})\coloneqq
D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k+m)})-D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})$
which measures the change in the escort divergence over $m$ time steps. We
find that demanding
$\Delta_{m}D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})<0$, implies

$\displaystyle\sum_{j=k}^{k+m-1}\sum_{i}\hat{x}_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma_{i}(\mathbf{x}^{(j)})}>\sum_{j=k}^{k+m-1}\sum_{i}x_{i}^{(j)}\frac{\varphi_{i}(\mathbf{x}^{(j)}}{\sigma_{i}(\mathbf{x}^{(j)})}$
$\displaystyle-\sum_{j=k}^{k+m-1}C_{\sigma}(\hat{\textbf{x}}^{(k)},j).~{}~{}$
(19)

where $C_{\sigma}(\hat{\textbf{x}}^{(k)},j)$ is the non-negative correction
term introduced in Eq. (10), which arose due to the choice of bounds in the
computation. The derivation of this result is given in Appendix C. We observe
that if a periodic orbit satisfies the definition of ISO for escort-incentive
map (Eq. (18)), then it automatically satisfies the immediately preceding
inequality (III.2). Hence, the relation of the ISO for escort-incentive map,
that is a periodic orbit, with decreasing escort divergence is transparent.

###  III.3 Dynamical Stability and Game-Theoretic Interpretation

In subsection III.2, we have motivated the ISO condition from information-
theoretic grounds. We now illustrate the central idea of the paper through a
proposition that brings together the three perspectives—information-theoretic,
game-theoretic, and dynamic—whose interconnection is what this paper is all
about.

For simplicity, we consider the replicator map. As described in subsection
III.1, we note that a locally asymptotically stable periodic orbit of (two-
player two-strategy) replicator map is an HSO [18]. Does such a fact exist
between periodic orbit and ISO as well? We find that the answer is affirmative
(although, the mathematical proof is elusive for periodic orbit of general
period). We provide the following result for a two-period orbit:

Proposition 1. A locally asymptotically stable 2-period orbit of a replicator
map for two-player two-strategy game is an ISO.

Proof. Since we know that a locally asymptotically stable 2-periodic orbit
must be HSO [18], we rewrite the HSO condition (Eq. (13)) as follows

$\displaystyle\sum_{j=1}^{2}\left[\hat{x}^{(1)}\cdot
f(\mathbf{x}^{(j)})-x^{(j)}\cdot f(\mathbf{x}^{(j)})\right]>$
$\displaystyle\left[\frac{H_{\mathbf{x}^{(2)}}}{H_{\mathbf{x}^{(1)}}}\left((\mathbf{x}^{(1)}-\hat{\mathbf{x}}^{(1)})\cdot
f(\mathbf{x}^{(2)})\right)-(\mathbf{x}^{(2)}-\hat{\mathbf{x}}^{(1)})\cdot
f(\mathbf{x}^{(2)})\right].$ (20)

This way of writing makes the terms, required to define the ISO condition,
appear conveniently together on the left hand side; hence, to show that the
ISO condition holds, one has to simply show that the right hand side is
nonnegative. Now, the right hand side may be recast as

$\displaystyle\left[f_{1}(\mathbf{x}^{(2)})\\!-\\!f_{2}(\mathbf{x}^{(2)})\right]\\!\\!\left[\left(\hat{x}^{(1)}-x^{(2)}\right)\\!-\\!\frac{H_{\mathbf{x}^{(2)}}}{H_{\mathbf{x}^{(1)}}}\left(\hat{x}^{(1)}-x^{(1)}\right)\right]$
$\displaystyle=\\!2(H_{\mathbf{x}^{(2)}})^{-1}\Delta{x}^{(2)}\left[\left(\hat{x}^{(1)}-x^{(2)}\right)\\!-\\!\frac{H_{\mathbf{x}^{(2)}}}{H_{\mathbf{x}^{(1)}}}\left(\hat{x}^{(1)}-x^{(1)}\right)\right],$
(21)

where $\Delta x^{(2)}\coloneqq x^{(3)}-x^{(2)}$ (as in the notation introduced
in Eq. (2)). We are free to choose small enough neighbourhood of $\hat{\bf
x}^{(1)}$ so that ${\bf x}^{(1)}$ is close enough such that the term
containing $\hat{x}^{(1)}-x^{(1)}$ on the right-hand side of Eq. (21) can be
made arbitrarily small (note that its prefactor composed of heterogeneity
factors is finite). Furthermore, for stable periodic orbits and the
aforementioned small neighbourhood, $\Delta{x}^{(2)}$ and
$\hat{x}^{(1)}-x^{(2)}$ are of same sign. In conclusion, the right-hand side
of Eq. (20) is a positive quantity for some small neighbourhood of $\hat{\bf
x}^{(1)}$. Therefore, the right-hand side of Eq. (20) is greater than zero,
and the aforementioned proposition stands proven.

Thus, Proposition 1 illustrates a connection between the dynamical stability
of periodic outcomes and the ISO equilibrium. We conjecture that such results
may also be derived for higher period orbits although it appears to be
mathematically more involved and require the specific properties of the
dynamic in question.

To appreciate the game-theoretic point-of-view, we observe how the game-
theoretic idea of stability in terms of comparison between payoffs seamlessly
translates to the stability criterion for periodic orbits as well. There is no
explicit need for a heterogeneity-weighted payoff as considered in a prior
work [18]. The periodicity condition in Eq. (12) can be treated as the
extension of Nash equilibrium for periodic orbits. ISO serves as the extension
of ESS. An ESS state is a state such that its payoff, when played against any
other state in its neighbourhood, is greater than the payoff gained when the
other state plays with itself. Analogously, for the ISO, we see that the
relevant quantity for comparison is the sum of payoffs for a particular state
$\hat{\mathbf{x}}$ of the $m$-period orbit. Here we must look at a sequence of
$m$ states starting from the neighbourhood of the state $\hat{\mathbf{x}}$ and
compare the sum of the payoffs—as gained by the state $\hat{\mathbf{x}}$ while
in play with each of the states in the sequence—with the sum of the payoff
that the states in the sequence gain while playing against themselves. In
other words, we may say that the $m$-period orbit is evolutionarily stable
when a state in the orbit outperforms an evolving mutant state as compared to
when the mutant state keeps playing with itself.

Finally, we remark that the definitions of ISO (Definition 2a-c) are
constructed from a game-theoretic perspective, without invoking any notion of
(periodic) dynamics. Thus, although our entire analysis pertains to two-player
two-strategy games, these definitions also hold for two-player $n$-strategy
games (with $n>2$).

###  III.4 Numerical Verification

Figure 1: _Stable periodic orbits are ISO for replicator map:_ The figure
presents the bifurcation diagram for the replicator map for a two-player two-
strategy game. The payoff matrix, ${\sf\Pi}$, is chosen such that $T=1.5+S$,
$R=1$ and $P=0$. The dash-dotted line, $S=4.5$, intersects the bifurcation
diagram at 2-period points—$a\approx 0.73$ (inverted triangle) and $b\approx
0.26$ (diamond)—of the 2-period orbit. The dashed line, $S=5.7$, cuts the
bifurcation diagram at the 4-period points of the 4-period
orbit—$\\{c,d,e,f\\}\approx\\{0.74,0.63,0.20,0.14\\}$; the points are
respectively marked by circle, triangle, square and cross. Upper inset: It
depicts positivity of
$\delta^{2}\equiv\delta^{2}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
[calculated for the stable 2-period points $a$ and $b$; see Eq. (22)] for
small values of $\epsilon$. The curves for
$\delta^{2}_{\varphi,\sigma}((a,1-a),\epsilon)$ vs. $\epsilon$ and
$\delta^{2}_{\varphi,\sigma}((b,1-b),\epsilon)$ vs. $\epsilon$ are marked
respectively by inverted triangles and diamonds. Lower inset: Similar to what
is done in the upper inset, it showcases positivity of
$\delta^{4}\equiv\delta^{4}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
corresponding to the $4$-period points—$c,\,d,\,e,\,$ and $f$—represented by
the curves respectively marked by circles, triangles, squares and crosses.
Figure 2: _Stable periodic orbits are ISO for incentive map:_ The
figure—analogous to Fig. (1)—presents the bifurcation diagram for the logit
map for two-player two-strategy game. The payoff matrix, ${\sf\Pi}$, is chosen
such that $R=1$, $S=2$, $T=3$, and $P=1$. Upper inset: It depicts positivity
of
$\delta^{2}\equiv\delta^{2}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
for the two states of the stable 2-period orbit points—$a\approx 0.59$
(inverted triangle) and $b\approx 0.06$ (diamond)—corresponding to $\beta=3.8$
(see vertical dash-dotted line intersecting the bifurcation diagram). Lower
inset: It shows positivity of
$\delta^{4}\equiv\delta^{4}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
for the four states of the $4$-period orbit,
$\\{c,d,e,f\\}\approx\\{0.72,0.50,0.10,0.09\\}$, denoted by circle, triangle,
square and cross respectively corresponding to $\beta=4.5$ (see vertical
dashed line intersecting the bifurcation diagram). Figure 3: _Stable periodic
orbits are ISO for escort-incentive map:_ This figure—analogous to Fig.
(1)—showcases the bifurcation diagram for the $q$-replicator map for two-
player two-strategy games. The payoff matrix, ${\sf\Pi}$, is chosen such that
$R=1$, $P=0$, and $T=2.5+S$; $q$ has been chosen to be $3$. Upper inset: It
shows positivity of
$\delta^{2}\equiv\delta^{2}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
for the two states of the stable 2-period orbit points—$a\approx 0.56$
(inverted triangle) and $b\approx 0.36$ (diamond)—existing at $S=19.0$ (see
vertical dash-dotted line intersecting the bifurcation diagram). Lower inset:
It depicts positivity of
$\delta^{4}\equiv\delta^{4}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$
for the four states of the $4$-period orbit,
$\\{c,d,e,f\\}\approx\\{0.60,0.55,0.34,0.30\\}$, denoted by circle, triangle,
square and cross respectively corresponding to $S=25.0$ (see vertical dashed
line intersecting the bifurcation diagram).

We now undertake the exercise of numerically illustrating that the periodic
orbits indeed are ISO or its generalizations for the incentive and escort-
incentive maps. For this purpose, it helps to denote an $m$-period orbit as
$\\{\hat{\bf x}^{(k)}\\}_{k=1}^{k=m}$and so $\\{{\bf x}^{(k)}\\}_{k=1}^{k=m}$
is the neighbouring trajectory with $x^{(1)}=\hat{x}^{(1)}+\epsilon$.
Furthermore, the ISO condition for the escort-incentive map (it being the most
general version) may be compactly written as
$\delta^{m}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)})>0$ where

$\delta^{m}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)\coloneqq\sum_{j=k}^{k+m-1}\sum_{i}\left[\hat{x}_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma_{i}(\mathbf{x}^{(j)})}-x_{i}^{(j)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma_{i}(\mathbf{x}^{(j)})}\right].$
(22)

We want to show that the condition is satisfied at least for some small values
of $\epsilon$.

In addition to the replicator map, we take two more maps—logit map, and
$q$-replicator map—to respectively represent the classes of incentive and
escort-incentive maps. The fitness of $i$-th strategy (or type)
$f_{i}(\mathbf{x})$, is calculated using a payoff matrix,

${\sf{\Pi}}$ = $\begin{bmatrix}R&S\\\ T&P\end{bmatrix}$
$(R,S,T,P\in\mathbb{R})$,
---
(23)

so that $f_{i}(\mathbf{x})=({\sf{\Pi}}\mathbf{x})_{i}$. It should be remarked
that the elements of the payoff matrix must be chosen carefully so as to
achieve forward-invariability of the corresponding maps, as noted in Sec. II
(see Ref. [17, 35]).

For each of the dynamics, we first demonstrate the existence of periodic
orbits by generating their bifurcation diagrams, and subsequently choose
parameter values to obtain examples of 2-period and 4-period orbits. We then
verify the positivity of
$\delta^{m}_{\varphi,\sigma}(\hat{\mathbf{x}}^{(k)},\epsilon)$ for each of
these examples using small values of $\epsilon$. The stability of the periodic
orbit can be further verified using linear stability analysis and by observing
the asymptotic behaviour of the corresponding time series. We systematically
exhibit examples of stable $2$-period orbit ($\\{a,b\\}$) and $4$-period orbit
($\\{c,d,e,f\\}$) of replicator, logit and $q$-replicator dynamics (see Figs.
1, 2 and 3 respectively) to illustrate that they follow the ISO criterion.

##  IV Discussion and Conclusions

Our results in this paper lie in the exciting overlapping area of evolutionary
game theory, dynamical systems theory, and information theory. Specifically,
we have shown that an extension of the idea of ESS is possible for periodic
orbits in a large class of discrete-time evolutionary dynamics. We have termed
the extended concept as ISO (and its generalizations) because it can be
motivated through the information-theoretic idea of decreasing KL-divergence—a
rather general principle that extends the idea of entropy maximization [13] in
natural world. Thus, the concept of the ISO—in line with similar recent
developments [15, 53, 54]—highlights the tightly knit connections between the
fields of evolutionary dynamics and information theory. These connections
between evolutionary game dynamics and information theory is a promising
avenue of research, and we suspect that a lot of information-theoretic
concepts (such as Rényi entropy, Fisher information, and information geometry)
may be transported to this context to improve our understanding of
evolutionary systems. In particular, the possibility of an information-metric
under which the discrete-time replicator map is a gradient-like system is
worth pondering, and it could lead to tighter conditions of the stability for
periodic orbits in game-theoretic terms.

From the game-theoretic view point, the ISO is a more satisfying extension of
the ESS as compared to HSO [18] because it does not require the use of the ad
hoc heterogeneity factor in its definition. Furthermore, recall that ESS
condition is tied to the concept of strong stability [5] and it is not
surprising that HSO is connected to the concept of the strongly stable
strategy set (SSSS) [18]—the extension of strong stability for periodic
orbits. This is an important qualification, for the strong stability provides
perhaps the ‘best validation for the concept of evolutionary stability’ [5].
Since locally asymptotically stable $2$-period orbit—being HSO—is ISO, when
the orbit is SSSS, it is ISO as well.

Before we end, we would like to delve into some related subtlety that deserves
thorough future investigation. We wonder how the unstable periodic orbits
connect with the ISO. We know that some unstable orbits are HSO and some are
not [18]. Although we are presently unable to provide any rigorous proof but
we conjecture that a similar scenario is true for the case of ISO: A trivial
example is that of the replicator map for two-player two-strategy stag-hunt
game where there is an unstable interior fixed point ($1$-period orbit) that
is not an ESS (or ISO of order 1) [17] (cf. Appendix D).

Since the replicator dynamic in a structured population remains same as the
one for the unstructured population—only the payoff matrix gets modified [55,
56] to include the effects of the structure—our conclusions should carry over
to structured infinite populations. To end with an optimistic note, we
envisage useful extensions of the concepts developed in this paper for
asymmetric games and extensive games, as well as for games in the finite
populations. However, for finite populations [57], the notions of mixed ESS
and existence of periodic orbits have to be defined before an information-
theoretic perspective may be constructed. Probably, the idea of ISO can be
useful for the continuous-time evolutionary dynamics as well through the
construction of Poincaré maps—an avenue worth pursuing in future.

###### Acknowledgements.

SB acknowledges the SURGE Programme at IITK (where part of the work was
carried out) and support from the KVPY fellowship. SC acknowledges the support
from SERB (DST, govt. of India) through Project No. MTR/2021/000119.

##  Appendix A Escort-incentive stable state (EISS)

In this appendix, we show how demanding decreasing relative entropy gives a
condition consistent with the game-theoretic EISS condition for escort-
incentive dynamics. For this computation, we need the following two
inequalities obtained via usual bounds on Riemann integrals:

$\displaystyle\int_{a}^{b}\log_{\sigma}(v)\>\textrm{d}v$
$\displaystyle\geq(b-a)\log_{\sigma}(a),$ (24a)
$\displaystyle\int_{a}^{b}\frac{1}{\sigma(u)}\textrm{d}u$
$\displaystyle\geq\frac{b-a}{\sigma(b)}.$ (24b)

By making use of these inequalities, we compute the total change in relative
entropy over a time period as follows.

$\displaystyle\Delta
D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})=\sum_{i=1}^{2}\Big{[}-\left(\hat{x}_{i}-x_{i}^{(k+1)}\right)\log_{\sigma}(x_{i}^{(k+1)})$
$\displaystyle+\left(\hat{x}_{i}-x_{i}^{(k)}\right)\log_{\sigma}(x_{i}^{(k)})+\int_{x_{i}^{(k+1)}}^{x_{i}^{(k)}}\log_{\sigma}(v)\>\textrm{d}v\Big{]}$
(25a)
$\displaystyle\geq\sum_{i=1}^{2}\left[\left(\hat{x}_{i}-x_{i}^{(k)}\right)\left(\log_{\sigma}(x_{i}^{(k)})-\log_{\sigma}(x_{i}^{(k+1)})\right)\right]$
(25b)
$\displaystyle=\sum_{i=1}^{2}\left[\left(\hat{x}_{i}-x_{i}^{(k)}\right)\int_{x_{i}^{(k+1)}}^{x_{i}^{(k)}}\frac{1}{\sigma(u)}\textrm{d}u\right]$
(25c)
$\displaystyle\geq\sum_{i}^{2}\left[\left(\hat{x}_{i}-x_{i}^{(k)}\right)\frac{\left(x_{i}^{(k)}-x_{i}^{(k+1)}\right)}{\sigma(x_{i}^{(k)})}\right]+$
$\displaystyle(\hat{x}_{2}-x_{2}^{(k)})(x_{2}^{(k)}-x_{2}^{(k+1)})\\!\\!\left[\frac{1}{\sigma(x_{2}^{(k+1)})}-\frac{1}{\sigma(x_{2}^{(k)})}\right]\quad$
(25d)
$\displaystyle=-\\!\\!\sum_{i}^{2}\hat{x}_{i}\frac{\varphi_{i}(\mathbf{x}^{(k)})}{\sigma(x_{i}^{(k)})}+\\!\sum_{i}^{2}x_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(k)})}{\sigma(x_{i}^{(k)})}-\\!C_{\sigma}(\hat{\textbf{x}},k),$
(25e)

where, the correction term, $C_{\sigma}(\hat{\textbf{x}},k)$, is given by

$\displaystyle C_{\sigma}(\mathbf{\hat{x}},k)\coloneqq$
$\displaystyle\left|\hat{x}_{1}-x_{1}^{(k)}\right|\left(x_{2}^{(k)}-x_{2}^{(k+1)}\right)$
$\displaystyle\Big{[}{\sigma(x_{2}^{(k+1)})}^{-1}-{\sigma(x_{2}^{(k)})}^{-1}\Big{]}$
(26)

The first inequality in Eq. (25b) is obtained by using the bound in Eq. (24a),
while the second inequality in Eq. (25d) is obtained by using Eq. (24b).
However, to compare with the EISS expression, we notice that it is necessary
to separate out a correction term $C_{\sigma}(\hat{\textbf{x}},k)$ given
explicitly in Eq. (A). This expression is obtained from Eq. (25d) by noting
that $(\hat{x}_{2}-x_{2}^{(k)})=-(\hat{x}_{1}-x_{1}^{(k)})$ and that without
any loss of generality, one can fix $\hat{x}_{1}-x_{1}^{(k)}>0$. Further, we
note that since the escort function is nondecreasing, we have
$(x_{2}^{(k)}-x_{2}^{(k+1)})\left[{\sigma(x_{2}^{(k+1)})}^{-1}-{\sigma(x_{2}^{(k)})}^{-1}\right]\geq
0$. This implies that the $C_{\sigma}(\mathbf{\hat{x}},k)$ is non-negative.
Now, requiring the stability of fixed point, implies $\Delta
D_{\sigma}(\hat{\mathbf{x}}||\mathbf{x}^{(k)})<0$, which gives us Eq. (10),
and that is consistently satisfied by the EISS condition (Eq. (9)).

##  Appendix B Information stable orbit (ISO) for replicator map

Here, we show how the requirement of decreasing total relative entropy leads
to the ISO condition for the replicator map. We compute as follows.

$\displaystyle\Delta_{m}D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})=-\sum_{i=1}^{2}\hat{x}_{i}^{(k)}\ln{\frac{x_{i}^{(k+m)}}{x_{i}^{(k)}}}$
(27a)
$\displaystyle\qquad=-m\sum_{i}^{2}\hat{x}_{i}^{(k)}\sum_{j=k}^{k+m-1}\frac{1}{m}\ln{\left[1+(\delta_{xf})_{i}^{(j)}\right]}$
(27b)
$\displaystyle\qquad\geq-m\sum_{i}^{2}\hat{x}_{i}^{(k)}\ln{\left[1+\sum_{j=k}^{k+m-1}\frac{1}{m}(\delta_{xf})_{i}^{(j)}\right]}$
(27c)
$\displaystyle\qquad\geq-m\ln{\left[1+\sum_{i}^{2}\hat{x}_{i}^{(k)}\sum_{j=k}^{k+m-1}\frac{1}{m}(\delta_{xf})_{i}^{(j)}\right]},$
(27d)

where the last two inequalities are obtained using Jensen’s inequality. One
finds that demanding
$\Delta_{m}D_{\textrm{KL}}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})<0$
implies the ISO condition as given in Eq. (15). Also note that an extension to
higher strategy space may be possible here too (as in for the fixed point
analysis in Eq. (6)), we simply replace the superscripts on the summations by
$n$ ($n>2$).

##  Appendix C Information stable orbit (ISO) for escort-incentive map

Here, we show how the requirement of decreasing total relative entropy leads
to a condition consistent with the ISO condition for the escort-incentive map.
We compute this by observing that the total relative entropy can be written as
a telescoping sum of intermediate differences of relative entropies. Thus,

$\displaystyle\Delta_{m}D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})$
$\displaystyle=\sum_{j=k}^{k+m-1}\left[D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(j+1)})-D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(j)})\right]$
(28a) $\displaystyle=\sum_{j=k}^{k+m-1}\Delta
D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(j)})$ (28b)
$\displaystyle\geq-\sum_{j=k}^{k+m-1}\sum_{i=1}^{2}\hat{x}_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma(x_{i}^{(j)})}+\sum_{j=k}^{k+m-1}\sum_{i=1}^{2}x_{i}^{(j)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma(x_{i}^{(j)})}$
$\displaystyle\quad-\sum_{j=k}^{k+m-1}C_{\sigma}(\hat{\textbf{x}}^{(k)},j).$
(28c)

The last inequality is obtained by using Eq. (25e) and hence
$\sum_{j=k}^{k+m-1}C_{\sigma}(\hat{\textbf{x}}^{(k)},j)$—a sum of nonnegative
quantities—is nonnegative. This means that if the total escort divergence is
decreasing, i.e.,
$\Delta_{m}D_{\sigma}(\hat{\mathbf{x}}^{(k)}||\mathbf{x}^{(k)})<0$, then
following is implied:

$\displaystyle\sum_{j=k}^{k+m-1}\sum_{i}\hat{x}_{i}^{(k)}\frac{\varphi_{i}(\mathbf{x}^{(j)})}{\sigma_{i}(\mathbf{x}^{(j)})}>\sum_{j=k}^{k+m-1}\sum_{i}x_{i}^{(j)}\frac{\varphi_{i}(\mathbf{x}^{(j)}}{\sigma_{i}(\mathbf{x}^{(j)})}$
$\displaystyle-\sum_{j=k}^{k+m-1}C_{\sigma}(\hat{\textbf{x}}^{(k)},j),\,\,$
(29)

which is consistent with the ISO condition given in Eq. (18).

##  Appendix D A note on 2-period orbit and ISO

Here we demonstrate a curious result for the analytically tractable case of
$2$-period orbits in the replicator map. In particular, we prove the
following.

Proposition 2. A two-period orbit of the replicator map for two-player two-
strategy game is an ISO.

Proof. Consider a periodic orbit
$\\{\hat{\textbf{x}}^{(1)},\hat{\textbf{x}}^{(2)}\\}$ and note that, by
definition, $\Delta\hat{x}^{(1)}\Delta\hat{x}^{(2)}<0$. If we consider the two
points of the periodic orbit to be well-separated and a state
${\textbf{x}}^{(1)}$ in the infinitesimal small neighbourhood of
$\hat{\textbf{x}}^{(1)}$, then we expect
$\Delta{x}^{(1)}\Delta{x}^{(2)}<0$—even if the periodic orbit is unstable, as
long as ${\textbf{x}}^{(2)}$ (the state to which ${\textbf{x}}^{(1)}$ is
mapped) is in small neighbourhood of $\hat{\textbf{x}}^{(2)}$, far away from
${\textbf{x}}^{(1)}$. We should always be able to write

$\displaystyle\Delta x^{(1)}\Delta
x^{(2)}+\sum_{j={1}}^{2}\left(\frac{H_{\mathbf{x}^{(2)}}}{H_{\mathbf{x}^{(j)}}}\right)\Delta
x^{(j)}$ $\displaystyle\left(x^{(1)}-\hat{x}^{(1)}\right)<0\qquad$ (30)

because the second term in the left hand side can be made infinitesimally
small by taking $x^{(1)}\rightarrow\hat{x}^{(1)}$ as the heterogeneity factor
is a positive quantity. Rewriting this equation, we arrive at

$\sum_{j=1}^{2}\left(H_{\mathbf{x}^{(j)}}\right)^{-1}\Delta
x^{(j)}(x^{(j)}-\hat{x}^{(1)})<0,$ (31)

which may be further rewritten using the replicator map to yield

$\sum_{j=1}^{2}(x^{(j)}-\hat{x}^{(1)})\left[f_{1}(\mathbf{x}^{(j)})-f_{2}(\mathbf{x}^{(j)})\right]<0.$
(32)

This equation is equivalent to the ISO condition for $2$-period orbits.

Thus, it appears that a $2$-period orbit is ISO irrespective of its stability
property (and hence Proposition 1 naturally follows from Proposition 2).
However, the fraction of mutants required to invade ISO has to be
comparatively larger for the case of stable $2$-period orbits. It must be kept
in mind that the analytically tractable case of the $2$-period orbits of the
replicator map appears to be very special in this respect: It remains to be
proven that how much of this discussion goes over to the higher period orbits
in arbitrary escort-incentive dynamics.

## References

  * Stenseth  _et al._ [1997] N. C. Stenseth, W. Falck, O. N. Bjørnstad, and C. J. Krebs, Population regulation in snowshoe hare and canadian lynx: Asymmetric food web configurations between hare and lynx, Proceedings of the National Academy of Sciences 94, 5147 (1997).
  * Peterson  _et al._ [1984] R. O. Peterson, R. E. Page, and K. M. Dodge, Wolves, moose, and the allometry of population cycles, Science 224, 1350 (1984).
  * Fauteux  _et al._ [2015] D. Fauteux, G. Gauthier, and D. Berteaux, Seasonal demography of a cyclic lemming population in the canadian arctic, Journal of Animal Ecology 84, 1412 (2015).
  * Lotka [1920] A. J. Lotka, Analytical note on certain rhythmic relations in organic systems, Proceedings of the National Academy of Sciences 6, 410 (1920).
  * Hofbauer and Sigmund [1998] J. Hofbauer and K. Sigmund,  _Evolutionary Games and Population Dynamics_ (Cambridge University Press, 1998).
  * Taylor and Jonker [1978] P. D. Taylor and L. B. Jonker, Evolutionary stable strategies and game dynamics, Mathematical Biosciences 40, 145 (1978).
  * Schuster and Sigmund [1983] P. Schuster and K. Sigmund, Replicator dynamics, Journal of Theoretical Biology 100, 533 (1983).
  * Smith [1982] J. M. Smith,  _Evolution and the Theory of Games_ (Cambridge University Press, 1982).
  * Smith and Price [1973] J. M. Smith and G. R. Price, The logic of animal conflict, Nature 246, 15 (1973).
  * Nash [1950a] J. F. Nash, Equilibrium points in n-person games, Proceedings of the National Academy of Sciences 36, 48 (1950a).
  * Nash [1951] J. F. Nash, Non-cooperative games, The Annals of Mathematics 54, 286 (1951).
  * Cressman and Tao [2014a] R. Cressman and Y. Tao, The replicator equation and other game dynamics, Proc. Natl. Acad. Sci. U.S.A. 111, 10810 (2014a).
  * Cover and Thomas [2005] T. M. Cover and J. A. Thomas,  _Elements of Information Theory_ (Wiley, 2005).
  * Jordan and Smith [1999] D. W. Jordan and P. Smith,  _Nonlinear Ordinary Differential Equations: An Introduction to Dynamical Systems_ (Oxford University Press, 1999).
  * Baez and Pollard [2016] J. C. Baez and B. S. Pollard, Relative entropy in biological systems, Entropy 18, 10.3390/e18020046 (2016).
  * Vilone  _et al._ [2011] D. Vilone, A. Robledo, and A. Sánchez, Chaos and unpredictability in evolutionary dynamics in discrete time, Phys. Rev. Lett. 107, 038101 (2011).
  * Pandit  _et al._ [2018] V. Pandit, A. Mukhopadhyay, and S. Chakraborty, Weight of fitness deviation governs strict physical chaos in replicator dynamics, Chaos: An Interdisciplinary Journal of Nonlinear Science 28, 033104 (2018), https://doi.org/10.1063/1.5011955 .
  * Mukhopadhyay and Chakraborty [2020a] A. Mukhopadhyay and S. Chakraborty, Periodic orbit can be evolutionarily stable: Case study of discrete replicator dynamics, Journal of Theoretical Biology 497, 110288 (2020a).
  * Fryer [2013] D. E. A. Fryer, On the existence of general equilibrium in finite games and general game dynamics (2013), arXiv:1201.2384 [cs.GT] .
  * Harper and Fryer [2015] M. Harper and D. Fryer, Lyapunov functions for time-scale dynamics on riemannian geometries of the simplex, Dynamic Games and Applications 5, 318 (2015).
  * Wagner [2013] E. Wagner, The explanatory relevance of nash equilibrium: One-dimensional chaos in boundedly rational learning, Philosophy of Science 80, 783 (2013).
  * Cressman [2003] R. Cressman,  _Evolutionary dynamics and extensive form games_ (MIT Press, Cambridge, MA., 2003).
  * Bomze [1991] I. M. Bomze, Cross entropy minimization in uninvadable states of complex populations, Journal of Mathematical Biology 30, 73 (1991).
  * Karev [2010] G. P. Karev, Replicator equations and the principle of minimal production of information, Bulletin of mathematical biology 72, 1124 (2010).
  * Olivares and Paris [2007] S. Olivares and M. G. Paris, Quantum estimation via the minimum kullback entropy principle, Physical Review A 76, 042120 (2007).
  * Qian  _et al._ [1991] M. Qian, G. Gong, and J. W. Clark, Relative entropy and learning rules, Physical Review A 43, 1061 (1991).
  * Floerchinger and Haas [2020] S. Floerchinger and T. Haas, Thermodynamics from relative entropy, Physical Review E 102, 052117 (2020).
  * Börgers and Sarin [1997] T. Börgers and R. Sarin, Learning through reinforcement and replicator dynamics, J. Econ. Theory 77, 1 (1997).
  * Hofbauer and Schlag [2000] J. Hofbauer and K. H. Schlag, Sophisticated imitation in cyclic games, J. Evol. Econ. 10, 523 (2000).
  * Bisin and Verdier [2000] A. Bisin and T. Verdier, Beyond the melting pot: Cultural transmission, marriage, and the evolution of ethnic and religious traits, Q. J. Econ 115, 955 (2000).
  * McElreath and Boyd [2007] R. McElreath and R. Boyd,  _Mathematical Models of Social Evolution: A Guide for the Perplexed_ (Univ of Chicago Press, Chicago, 2007).
  * Montgomery [2010] J. D. Montgomery, Intergenerational cultural transmission as an evolutionary game, Am. Econ. J. Microecon. 2, 115 (2010).
  * Mukhopadhyay and Chakraborty [2020b] A. Mukhopadhyay and S. Chakraborty, Deciphering chaos in evolutionary games, Chaos 30, 121104 (2020b).
  * Pandit  _et al._ [2021] V. Pandit, A. Mukhopadhyay, and S. Chakraborty, Erratum: Weight of fitness deviation governs strict physical chaos in replicator dynamics [chaos 28, 033104 (2018)], Chaos: An Interdisciplinary Journal of Nonlinear Science 31, 049901 (2021), https://doi.org/10.1063/5.0050272 .
  * Mukhopadhyay  _et al._ [2021] A. Mukhopadhyay, S. Chakraborty, and S. Chakraborty, Chaos and coexisting attractors in replicator-mutator maps, Journal of Physics: Complexity 2, 035005 (2021).
  * Nash [1950b] J. F. Nash, Equilibrium points in n-person games, Proceedings of the National Academy of Sciences 36, 48 (1950b).
  * Maynard Smith [1982] J. Maynard Smith,  _Evolution and the theory of games_ (Cambridge University Press Cambridge ; New York, 1982) pp. viii, 224 p. :.
  * Cressman and Tao [2014b] R. Cressman and Y. Tao, The replicator equation and other game dynamics, Proceedings of the National Academy of Sciences 111, 10810 (2014b).
  * Fryer [2014] D. E. A. Fryer, The Kullback-Leibler divergence as a lyapunov function for incentive based game dynamics (2014), arXiv:1207.0036 [math.DS] .
  * Fryer [2012] D. E. A. Fryer, The uniform distribution in incentive dynamics (2012), arXiv:1207.0037 [cs.GT] .
  * Harper  _et al._ [2013] M. Harper, D. Fryer, and A. Vlasic, Mean evolutionary dynamics for stochastically switching environments (2013), arXiv:1306.2373 [math.DS] .
  * Harper and Fryer [2016] M. Harper and D. Fryer, Stationary stability for evolutionary dynamics in finite populations, Entropy 18, 10.3390/e18090316 (2016).
  * Harper and Fryer [2013] M. Harper and D. Fryer, Incentive processes in finite populations (2013), arXiv:1306.2389 [math.DS] .
  * Lahkar and Sandholm [2008] R. Lahkar and W. H. Sandholm, The projection dynamic and the geometry of population games, Games and Economic Behavior 64, 565 (2008).
  * Matsui [1992] A. Matsui, Best response dynamics and socially stable strategies, Journal of Economic Theory 57, 343 (1992).
  * Gilboa and Matsui [1991] I. Gilboa and A. Matsui, Social stability and equilibrium, Econometrica 59, 859 (1991).
  * Jensen [1906] J. L. W. V. Jensen, Sur les fonctions convexes et les inégalités entre les valeurs moyennes, Acta Mathematica 30, 175 (1906).
  * Bof  _et al._ [2018] N. Bof, R. Carli, and L. Schenato, Lyapunov theory for discrete time systems (2018).
  * Wiggins [2003] S. Wiggins,  _Introduction to Applied Nonlinear Dynamical Systems and Chaos_ (Springer-Verlag, 2003).
  * Harper [2011] M. Harper, Escort evolutionary game theory, Physica D: Nonlinear Phenomena 240, 1411 (2011).
  * Beck and Schögl [1995] C. Beck and F. Schögl,  _Thermodynamics of chaotic systems_ (1995).
  * Umezuki [2018] Y. Umezuki, Bifurcation analysis of the rock–paper–scissors game with discrete-time logit dynamics, Mathematical Social Sciences 95, 54 (2018).
  * Harper [2010] M. Harper, The replicator equation as an inference dynamic (2010), arXiv:0911.1763 [math.DS] .
  * Harper [2009] M. Harper, Information geometry and evolutionary game theory (2009), arXiv:0911.1383 [cs.IT] .
  * Ohtsuki and Nowak [2006] H. Ohtsuki and M. A. Nowak, The replicator equation on graphs, Journal of theoretical biology 243, 86 (2006).
  * Ohtsuki and Nowak [2008] H. Ohtsuki and M. A. Nowak, Evolutionary stability on graphs, Journal of Theoretical Biology 251, 698 (2008).
  * Nowak  _et al._ [2004] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Emergence of cooperation and evolutionary stability in finite populations, Nature 428, 646 (2004).