# An approach to non-equilibrium statistical physics using variational
Bayesian inference

Maxwell J D Ramstead  Wellcome Centre for Human Neuroimaging, University
College London, London, UK  Dalton A R Sakthivadivel
<EMAIL_ADDRESS>Department of Mathematics, CUNY Graduate Centre,
New York, NY 10016, USA  Karl J Friston  Wellcome Centre for Human
Neuroimaging, University College London, London, UK

(28th August 2024)

###### Abstract

We discuss an approach to mathematically modelling systems made of objects
that are coupled together, using generative models of the dependence
relationships between states (or trajectories) of the things comprising such
systems. This broad class includes open or non-equilibrium systems and is
especially relevant to self-organising systems. The ensuing variational free
energy principle (FEP) has certain advantages over using random dynamical
systems explicitly, notably, by being more tractable and offering a
parsimonious explanation of why the joint system evolves in the way that it
does, based on the properties of the coupling between system components. Using
the FEP allows us to model the dynamics of an object as if it were a process
of variational inference, because variational free energy (or surprisal) is a
Lyapunov function for its dynamics. In short, we argue that using generative
models to represent and track relations among subsystems leads us to a
particular statistical theory of interacting systems. Conversely, this theory
enables us to construct nested models that respect the known relations among
subsystems. We point out that the fact that a physical object conforms to the
FEP does not necessarily imply that this object performs inference in the
literal sense; rather, it is a useful explanatory fiction which replaces the
‘explicit’ dynamics of the object with an ‘implicit’ flow on free energy
gradients—a fiction that may or may not be entertained by the object itself.

###### Contents

  1. 1 Introduction
  2. 2 The free energy principle
    1. 2.1 Conceptually
    2. 2.2 Mathematically
      1. 2.2.1 An overview so far
    3. 2.3 Dynamically
      1. 2.3.1 Stochastic thermodynamics
  3. 3 Summary remarks
  4. 4 Some preliminary philosophical considerations
  5. 5 A map of that part of the territory that behaves as if it were a map
  6. 6 Some remarks on terminological idiosyncrasies of the FEP literature
    1. 6.1 Generative models and variational densities versus recognition densities
    2. 6.2 Generative models and generative processes
  7. 7 Conclusion

### Acknowledgements

This paper more fully develops the theory outlined in a previous manuscript
[1], and we are grateful to those listed there for their contributions to
preliminary discussions on the topic. The account here is largely based on a
series of talks given by the second named author at the Department of
Philosophy of the University of Vienna and the Laufer Centre for Physical and
Quantitative Biology, and the informal discussions that followed.

##  1 Introduction

In statistics, generative models are joint probability distributions that
model the relationships between different variables—most often, observations
and their causes, or data and a set of labels. In the study of open and
interacting systems, the use of statistical models—capable of describing such
relationships—seems to be particularly apt. One approach to making such
models, using variational (approximate Bayesian) inference [2, 3], describes
how a physical object reflects the properties of its environment and vice
versa. Bayesian mechanics under the free energy principle (FEP) [4, 5] aims to
describe the physical properties of a system in terms of dynamics on a
manifold of probability distributions (i.e., as variational inference),
resting on the useful property that physical objects generally look as if they
track or infer the statistics of things to which they are coupled.

In this paper, we argue that generative models provide an especially useful
foundation in scientific modelling generally, and especially in the study of
systems exhibiting adaptation, morphogenesis, and other non-equilibrium
phenomena. More specifically, we argue that generative modelling—premised on
the FEP—is particularly apt for the practice of modelling the activity of
modelling itself; for instance, in modelling sophisticated self-organising
systems, where we write down a representation of the model that we assume is
in play in prediction and action. We use the FEP to illustrate how
implementing models which (i) formalise how one object keeps track of another,
and (ii) codify these relationships, can increase the fidelity of our
mathematical modelling. We identify a particular issue, which the FEP is able
to uniquely address: the FEP keeps track of how a model depends on aspects of
the thing that is modelled in a particularly insightful way.

Interestingly, because it incorporates this information about things that are
coupled and their coupling, it has been suggested that the FEP conflates the
metaphorical map (our mathematical model of the thing modelled) and the
territory (the thing modelled). The so-called model reification or map-
territory fallacy is a general critique of the practice of scientific
modelling—in physics, principally due to Cartwright [6, 7]. Some have
referenced this fallacy in relation to the FEP [8, 9]. In particular, it has
been suggested that FEP-theoretic modelling conflates the metaphorical
“map”—(i.e., the scientific model that scientists use to make sense of some
real-world phenomenon)—and “territory,” i.e., the actual physical system that
is being modelled. (See [10, 11, 12, 3] for related critical discussions of
such claims.)

According to this line of thinking, using Bayesian mechanics to claim that we
can model objects as themselves engaging in inference about the statistics of
their environment constitutes a case of model reification. The allegation is
that, in describing the dynamics of physical objects as implementing a form of
inference—as opposed to considering the inferential aspects of the explanation
as pertaining to our scientific models of those objects—the FEP theorist
mistakenly conflates their metaphorical “map” of the territory (i.e., the
scientific FEP-theoretic or Bayesian mechanical model) with the “territory”
(the real-world system itself). This constitutes a reification of our model by
assuming some aspect of the model or “map” is a real feature of the physical
world or “territory” [10, 11]. By clarifying that this potential difficulty
does not directly and necessarily apply to FEP-theoretic modelling, we are
able to turn the story on its head: namely, we point out that the FEP provides
a set of ultimate explanatory constraints on what counts as a model (or map)
of anything whatsoever that exists in the physical world.

Here, we discuss the use of the FEP in statistics and statistical physics, and
what advantages it offers; we then argue that there is no conflation of map
and territory, i.e., of modelled system and scientific model, in the FEP—in
fact, we describe how they are carefully distinguished. We argue that
modelling the dynamics of an object such that it looks as if it models the
system it is coupled to is not an instance of the map-territory fallacy. This
argument gives way to a more general theme of the FEP, which is that capturing
interactions and dependence relations between things using generative models
is insightful. We discuss how using the FEP gives us a flexible model of the
properties of the object that look like they perform inference, but crucially,
without inherently reifying the inferential aspects of the model, attributing
them to the real-world object. That is to say, a model of an object can always
be written as modelling or inferring the structure of its environment. To make
this argument, we appeal to two distinctions: one between the generative model
(which is our scientific model) and the physical object being modelled (the
real ‘thing’ in the world) [3]; and the other between the generative model and
the variational density, which are often conflated in the secondary literature
[13, 14].

To sketch the argument: the FEP provides us with tools from dynamical systems
and information theory, allowing us to model any coupled random dynamical
systems as statistical estimators, i.e., as engaging in the statistical
estimation of some quantity pertaining to the set of objects with which it
interacts. Under the FEP, one object that is coupled to any other can thus be
read as a representation of the coupled or estimated object, albeit of a
deflationary, qualified sort (discussed in [15]; see also [13, 16, 3]). More
formally, the FEP applies to generative models that contain a partition of the
states or paths of a joint system (a so called “particular” partition, i.e.,
into objects or particles). This partition induces a statistical boundary
called a Markov blanket [17], which captures conditional independence
relations amongst subsets of the system [2, 18]. The FEP says that, when such
Markov blanketed subsets of the system exist, those subsystems track [19] each
others’ statistical properties (dynamically) across the blanket. Essentially,
the FEP is a modelling framework which captures the following fact: models of
coupled particles or factorised systems can be written as parameter
estimators, and estimating the parameters of a distribution is (a form of)
Bayesian inference. We can think of the FEP, metaphorically, as a map of the
part of the territory that behaves as if it were a map—or looks as such to an
external observer.

Accounts such as [10] have encouraged practitioners of the FEP to be explicit
about this point. Viewing the theory this way, not only is there no conflation
between our FEP-theoretic model of the system and the system itself, but in
fact we arrive at a novel modelling method for coupled random dynamical
systems. In particular, we do not claim that the objects in the system
themselves must literally be performing inference. Rather, we describe the
dynamics as doing inference, in the sense of being a statistical estimator
whose states parameterise a variational density.

We first present a conceptual, and then technical, introduction to the FEP and
Bayesian mechanics—and the way it leverages the relationships encoded by a
generative model to build a scientific model of the dynamics of physical
objects using free energy gradients. This is possible by modelling the
property of physical systems that they look as if they engage in modelling to
an external observer. We then argue that there is no ambiguity between model
and modelled real-world system in this formalism. We will examine possible
reasons for mistakenly invoking the map-territory fallacy in this context, due
mainly to the historically botched distinctions between the generative model
and the variational density, on the one hand, and between the generative model
and the so-called generative process, on the other.

We propose that the FEP and Bayesian mechanics provide a map of any possible
map whatsoever—of, or held by—a physical system. (This includes the use of the
FEP itself by a modeller.) We suggest that this leads, in a Wittgensteinian
manner, to constraints on all possible maps, or models of physical things.
This is meant in terms of logically necessary preconditions for sense-making,
and thus the parsing of sensory streams on which cognition is contingent [15].
In particular, these constraints arise from what it means to be a map or model
of a physical process at all.

We remark that we will not claim to invalidate the map-territory fallacy
itself. That is, we do not suggest that the map and the territory are never
conflated; they obviously are in cases of genuine model reification (as
discussed cogently by [11]). Models built using the FEP are still scientific
models, and the properties of FEP theoretic model also run the risk of
reification. Instead, we will address the claim that the FEP is limited in its
effectiveness or engages in model reification because real physical things
cannot be modelled as themselves modelling their environments by minimising
free energy.

##  2 The free energy principle

###  2.1 Conceptually

In this subsection we will provide an up-to-date overview of the concepts
behind the FEP. One can also see [3] for a discussion of the main ideas. See
the following subsection for more mathematical details.

Conceptually, the FEP starts from a partitions an overall “system” into three
component sets of states: the internal states $\mu$ of a ‘thing’ or
‘particle’, an environment (which possesses external states $\eta$, relative
to the thing in question), and an interface separating but coupling the two
(the so-called Markov blanket, $b$), which mediates the influence of $\eta$ on
$\mu$ and _vice-versa_. The states of the system will be tuples
$x\coloneqq(\eta,b,\mu)$, where $\eta$ and $\mu$ are states of two subsystems
and $b$ is some locus of states such that $p(\eta\mid\mu,b)=p(\eta\mid b)$ and
vice versa. “Particular states” are the internal states of each thing and
their respective Markov blanket. Of note, the particular partition is akin to
choosing a set of coarse-grained variables at a superordinate scale,
associating each state or path at the subordinate scale to a particle at the
superordinate scale (as either an internal or a blanket state), in terms of
the relations of conditional independence that obtain between states or paths
[20, 2].

More precisely, we will consider systems of interacting objects, such as an
open random dynamical system coupled to its environment. Fixing a final time
$\tau$, the joint system will be denoted by the time-indexed family of random
variables $\\{X\\}_{t:\tau}$ taking values $x\in\mathcal{X}$ at each $t$, with
a specific realisation of the system (meaning, a sample path or trajectory
beginning at some fixed $X_{0}=x_{0}$) being the sequence
$\gamma(x_{0})=\\{x_{0},x_{1},\ldots,x_{\tau}\\}$. We will define
$\gamma(x_{0})\mapsto\gamma_{t}(x_{0})$ as the projection to the state at time
$t$. Then $B_{t}$ is what we have called the Markov blanket in
$\\{X\\}_{t:\tau}$ at time $t$, separating it into statistically distinct
subsystems (which one can then argue do not mix into each other [18]). We will
repeat this notation for

$\displaystyle\gamma_{t}(\eta_{0})=N_{t}=\eta\in\mathcal{N}$
$\displaystyle\gamma_{t}(b_{0})=B_{t}=b\in\mathcal{B}$
$\displaystyle\gamma_{t}(\mu_{0})=M_{t}=\mu\in\mathcal{M}$

as well as for conditional trajectories such as

$\gamma_{t}(\eta_{0}\mid B_{t}=b)=\eta_{b}\in\mathcal{N}\crossproduct\\{b\\}.$

Although we will not go into the theory of random dynamical systems, when
equipped with a time evolution map acting on the sample space, these are all
the ingredients necessary to define a random dynamical system [21], and this
is implicit in our discussion; for details see the account in [14]. When
$\\{M\\}_{t:\tau}$ is an open system coupled to an environment with states
$\eta$, a value $\mu$ is called an internal state, and the subsystem
$\\{M\\}_{t:\tau}$ is variously called an object, a thing, or a particle in
the literature. This is the convention we will stick to.

We can guarantee that the dynamics of a system can be described as a flow on
surprisal, and thereby, cast as implementing a form of inference, whenever
there exists a “particular partition” of that system. At the core of the FEP
is a mathematical statement about the behaviour of such systems. For a
comprehensive overview, see [3]. In particular, the FEP allows us relate the
dynamics of states or paths that are internal to the object, $\mu$, to the
dynamics of external states, conditioned on boundary or blanket states
$p(\eta\mid b)$—that is, understanding what goes on inside a subsystem by
understanding what it is coupled to, as ‘seen’ via blanket states [5, 22].

The logic of the FEP can be summarised as follows. If there exists a
particular partition of a system, then there exists a coupling between
internal and external states or paths, given blanket states or paths; then,
since the internal dynamics of a given object are coupled to other (external)
subsets of the system—in such a way that they can be read as parameters for
probability densities over the external subset—the FEP describes these
interacting systems using variational inference. This is a core aspect of the
FEP that distinguishes it from a straightforward application of density
dynamics to the internal states per se.

This is where the interpretation of dynamics as a form of inference comes into
play. Indeed, one way to state the FEP is:

######  Claim 1.

If a random dynamical system (i) is subject to a specific kind of sparse
coupling or boundary condition, such that it possesses a Markov blanket, and
(ii) has a subsystem $\\{M\\}_{t:\tau}$ under that blanket with characteristic
states (in the sense that it occupies a pullback attractor), then
$\\{M\\}_{t:\tau}$ can always be described as if it minimises a variational
free energy functional of densities over states or trajectories of
$\\{N\\}_{t:\tau}$ [14, 22].

Another statement one can make is that Bayesian mechanics formalises the
truism that, on average, an object will do whatever it is that it does on
average, given the sort of object it is. Behind this tautological statement,
we find a formal apparatus that enables us to model a variety of dynamical
systems in a parsimonious manner. We hope this will allow us to take some
first steps towards a mechanistic description of how self-organisation
manifests in physical systems with such pullback attractors [3].

Bayesian mechanics explains why every physical thing looks as if it were
performing inference over the causes of its perturbations, forming (Bayes)
optimal beliefs about those causes [22]. Bayesian mechanics begins from the
minimisation of self-information (also called surprisal) or free energy, and
allows us to write down mechanical theories (i.e., laws of motion) yielding
the dynamics of systems on an attractor [19]. Surprisal is simply the negative
log-probability of an observed outcome. To say that an object minimises (or
follows a path of least) surprisal is to say that an object evolves towards
the outcomes which are most probable, given the sort of thing that it is
(meaning, its characteristic states of being, or characteristic trajectories
through its state space).

Interestingly, this means that surprisal is suited to act as a kind of
‘ontological potential’ for the object, in the sense that the surprisal is
equivalent to a constraint on what states and outcomes are accessible to the
object; the trajectories of a physical system are random curves through the
gradient of surprisal over the state space, which satisfy an equation of
motion derived from the surprisal itself [4, 23]. As has been asserted in
recent literature [19], one reading of the FEP would hold that the crux of the
ubiquity of the FEP is that every _thing_ which is coupled to another thing
holds a model of that thing to which it is coupled in the following, trivial
sense: in virtue of this coupling, one subsystem estimates the parameters of
the probability density over states (or trajectories) of the other subsystems
(and vice versa). Using this property, they can be written as performing
inference in the sense of variational Bayesian inference.

This coupling is sometimes discussed in terms of the ‘probabilistic beliefs’
that one object holds about its environment. Given that the term ‘belief’ is
quite theoretically loaded (especially in philosophy), we ought perhaps to say
‘estimator,’ instead. Once more, our contention is that anything which
estimates the parameters of a density is engaged in a form of Bayesian
inference, by mathematical definition; i.e., it is mathematically indisputable
that systems that estimate statistics represent, or encode, or otherwise hold
probabilistic beliefs—whether the practitioner likes the word ‘beliefs’ or
not, and irrespective of what commitments one has with respect to notions such
as encoding and representation (e.g., [24]).

Generative models and variational methods allow us to licence these claims
[25, 26]. Generative models are joint probability densities over causes and
consequences which, here, are used to capture the dependencies between the
variables constituting $\\{X\\}_{t:\tau}$, meaning the way that changes in
some variables cause changes in others. Writing down a generative model for
some real object in the world (the metaphorical territory) is building a
metaphorical map of that territory (a formalisation or codification of the
relations between the things in that territory).

Variational techniques comprise a family of inference techniques which allow a
modeller to approximate some (usually intractable) posterior probability
density under a (usually specified) generative model. In variational
inference, we write down a parameterised probability density function called a
variational density, which is used as an approximation to the posterior
probability density. The quantity called variational free energy scores the
difference between the variational density and the posterior over external
states—here, a conditional density, given a particular blanket state. Hence,
when some notion of the difference between them is zero, variational free
energy can be used as a proxy for surprise, and we can approximate the
posterior density; which is a form of parameterised Bayesian inference.

Note that any given object cannot change the generative model associated to
that object. This follows because the object can only change the variational
density encoded by its internal states. However, to explain these
changes—under a variational principle of least action—one needs to specify the
generative model under which the posterior is approximated. This means the
generative model explains the dynamics of something that is observed in a
particular way. This is usually expressed by saying that the generative model
is entailed by the thing in question. That an object entails a generative
model is consistent with statements like “an object is a model of its
environment”: e.g., in the sense of the good regulator theorem [19]). This
follows because the generative model just is a probabilistic description of
the joint states an object and its environment occupy.

###  2.2 Mathematically

With that conceptual overview being stated, we will now provide some further
mathematical details. Similar (contemporary) reviews can be found in [4, 5,
19, 27]. We will argue that coupled random dynamical systems partitioned under
a Markov blanket naturally have the form of variational Bayesian inference
problems, in the sense that we obtain the values of parameters from the
minimisation of free energy or surprisal. This is what licences the claim that
minimising free energy implements a form of inference, in which case the
_object_ models its environment. Ultimately this simply means that, in virtue
of being coupled via blanket states to its environment, the object reflects
the structure of, and data about (i.e., models), its environment—or, what it
‘believes’ that its environment is like (in the sense of being a statistical
estimator, as defined above).

Appealing to the Laplace approximation (that if a density has a unique
maximum, in a neighbourhood of the most likely state, then the density is
approximately Gaussian with that mean), we will suppose all random variables
and all trajectories are jointly Gaussian. We will call the conditionally
expected state—that is, the expected state given some fixed blanket state—the
conditional mode $\hat{\mu}_{b}$ or $\hat{\eta}_{b}$. Let
$\mathcal{L}(\mu,b)=-\log p(\mu,b)$ be the surprisal of internal and blanket
states, such that
$\hat{\mu}_{b}=\operatorname*{\mathrm{arg\,min}}_{\mu}\mathcal{L}(\mu,b)$ and
thus satisfies $\nabla_{\mu}\mathcal{L}(\mu,b)=0$. By the Freidlin–Wentzell
theorem, we can argue the same is true of paths (see [19]). In that setting,
the interpretation of surprisal as the fluctuations of a random dynamical
system away from its expected path plays a crucial role in stochastic
thermodynamics [28, 29].

The FEP states that, given the existence of a boundary, we can construct a map
relating (the dynamics of) internal states to (those of) the environment
across the boundary. In particular, if there exists a boundary in the form of
a Markov blanket, then there exists a function
$\sigma:\mathcal{M}\to\mathcal{N}$ such that
$\sigma(\hat{\mu}_{b})=\hat{\eta}_{b}$, sending conditional modes to
conditional modes [22, Lemma 4.3]. We construct this map by composing the
natural projections to boundary states, $\bm{\mu}:\mathcal{B}\to\mathcal{M}$
and $\bm{\eta}:\mathcal{B}\to\mathcal{N}$, into

$\bm{\eta}\circ\bm{\mu}^{-1}:\mathcal{M}\to\mathcal{N}.$

We have assumed $\bm{\mu}$ is at minimum an injection, hence bijective on its
image, from which we can take a left-cancellative pseudo-inverse. Under
hypotheses on these maps, for example, that two blanket states which map to
the same value of $\mu$ also map to the same value of $\eta$, this is a useful
function, in the sense that it associates a unique pair of conditional modes
to each blanket state.111One can see [4] for a proof looking at the properties
of solutions to certain linear systems of equations. Alternatively, note that
this is evident from taking a tensor-Hom adjunct of the composition,
$b\mapsto\xi(b,-)$, and asking that this map be an injection in the sense that
when $i=j$, the map takes $\hat{\mu}_{i}=\hat{\mu}_{j}$ to equal images
$\hat{\eta}_{i}=\hat{\eta}_{j}$ [22, pp 25]. Importantly, in the definition of
$\sigma$, we can pass an arbitrary internal state in; its image will simply
not be the conditional mode of the environment, and hence it will not
parameterise a density consistent with $p(\eta\mid b)$.

We then postulate the existence of a family of probability densities
parameterised by values of some known quantity, called the variational density
$q(\eta;\hat{\eta})$. Generically this is constructed by taking a family of
Gaussian densities with different modes. By the previous discussion, we can
interpret this as the likely configuration of the environment, knowing the
environment must be amenable to a certain internal state on average (given a
corresponding blanket state), $q(\eta;\sigma(\mu_{b}))$. It is immediate that
the probability density parameterised by the conditional mode $\hat{\eta}_{b}$
ought to be the same as the conditional probability $p(\eta\mid b)$. Moreover,
if we assume the system is coupled by a particular partition, we naturally
have $\sigma(\hat{\mu}_{b})=\hat{\eta}_{b}$. Together, we have supposed that
the object exists as the preimage of $\hat{\eta}_{b}$, or tends to do so, and
hence, that the environmental states that are likely under a given conditional
mode of the object ought to be equivalent to the probabilities over external
states conditioned on that blanket state.

Because we are identifying a parameter value which both (i) has the greatest
log-likelihood of being observed and (ii) fits a parameterised distribution
over unobserved states to a distribution over unobserved states given observed
states, this set up naturally has the structure of a variational inference
problem [30]. As discussed, the quantity known as variational free energy
scores the difference between the variational density and the true conditional
density in terms of the relative entropy from one to the other, known as the
KL divergence, added to the surprisal of internal and blanket states:

$F(\mu,b)\coloneqq\int q(\eta;\sigma(\mu))\log
q(\eta;\sigma(\mu))\differential{\eta}-\int q(\eta;\sigma(\mu))\log p(\eta\mid
b)\differential{\eta}-\log p(\mu,b).$

In the context of variational inference, we consider both terms in order to
obtain a parameter for which $q=p$ almost surely and identify the least
surprising (or most likely) parameter; the interpretation of the surprisal
term is obvious, whilst the entropy term prevents $q$ from concentrating about
the mode, spreading to match $p$.222Note that as before, we can tell this
story for the trajectories of a subsystem, where we take
$q(\gamma(\eta_{0});u_{b})$ and $p(\gamma(\eta_{0})\mid\gamma(b_{0}))$
instead, where $u_{b}$ is the expected path given an observed trajectory of
blanket states. The use of maximum entropy in the setting of sample paths is
reviewed in [31]. Again, in the context of variational inference, the least
surprising parameter is also the parameter with the most evidence—meaning
objects that are ‘good’ at occupying their characteristic states can be
described as self-evidencing. This has the interpretation of minimising the
surprisal of observations of $\mu,b$ by inferring probabilities of unobserved
latent states $\eta$ using the so-called discriminative model $p(\eta\mid
b,\mu)$.

Importantly, the variational free energy is always greater than or equal to
the surprisal, since the KL divergence is non-negative in general. Minimising
the free energy effectively implements a form of inference, where we have
inferred the best possible approximation to the true conditional or posterior
density. That is, when the KL term is zero, the variational density and the
density over external states given the state of the blanket are equal up to
sets of zero probability. We observe (by properties of $\sigma$) that this is
only possible when $\mu=\hat{\mu}_{b}$. In the limit of exact Bayesian
inference, when the functional form of posterior beliefs encoded by internal
states coincides with the posterior density over external states, the
variational free energy reduces to the self-information of internal and
blanket states. (Refer to e.g. [22, Lemma 4.2], or the variational free energy
lemma in [27] for more discussion.)

This returns us to our earlier discussion about parameter estimation in the
FEP. In this case (and again invoking the Laplace approximation), the
surprisal being non-zero has a neat interpretation as the uncertainty that the
object cannot resolve by being at the mode. Since the object is conditionally
independent of its environment, it cannot ‘see’ beyond the Markov
blanket—$N_{t}$ is an unobserved latent variable causing observed blanket
states—lending this inference process a physical interpretation appropriate
for self-organising and adaptive objects as well.

We will now give the following result: the object is at its most likely point
in state or path space if and only if it minimises variational free energy. We
will recall that the KL divergence is non-negative, implying

$D_{\mathrm{KL}}(q,p)+\mathcal{L}(\mu,b)\geq\mathcal{L}(\mu,b)$

saturating at $\mu=\hat{\mu}_{b}$ by construction. Since we defined
$\hat{\mu}_{b}=\operatorname*{\mathrm{arg\,min}}_{\mu}\mathcal{L}(\mu,b)$, and
any movement in parameter space $\hat{\mu}_{b}+v$ adds a non-zero KL term to
$\mathcal{L}(\hat{\mu}_{b},b)$, we conclude

$\nabla_{\mu}\mathcal{L}(\mu,b)=0\iff\nabla_{\mu}F(\mu,b)=0.$

What we have shown is there is a natural interpretation of the most likely
state as a parameter estimation, in the sense of Bayesian inference. This
argument, whilst logically straightforward, conceals a more subtle point: all
of this begins from the existence of a generative model. In particular, since
the joint probability $p(\eta,b,\mu)$ factorises into $p(\eta\mid
b,\mu)p(b,\mu)$ (see the probabilistic chain rule), logarithms of products
decompose additively, and we have $p(\eta\mid b,\mu)=p(\eta\mid b)$ by
assumption, we have

$\displaystyle\int$ $\displaystyle q(\eta;\sigma(\mu))\log
q(\eta;\sigma(\mu))\differential{\eta}-\int q(\eta;\sigma(\mu))\log
p(\eta,b,\mu)\differential{\eta}=$ $\displaystyle\int q(\eta;\sigma(\mu))\log
q(\eta;\sigma(\mu))\differential{\eta}-\int q(\eta;\sigma(\mu))\log p(\eta\mid
b)\differential{\eta}-\log p(\mu,b).$

In particular, if we begin with a generative model and a variational density,
we are led to an inference with a satisfactory physical interpretation and
explanatory power (we will dive into the latter in the section on dynamics).
Not only is this the case, but by elementary mathematical
considerations333Notice that the support of the generative model and
variational density differ. Whilst we may still take the KL divergence if the
variational density is absolutely continuous with respect to the generative
model, this should not be expected to be meaningful. we find we ought to take
this factorisation. This observation is the main point of our proposal. The
use of this form of the free energy has an even more straightforward
interpretation as using an evidence lower bound to infer the generative model,
such that the object can generate samples of how environmental states are
likely to vary with internal states. Our objective in using the FEP is to use
this mathematical fact to replace classic models of stochastic processes—which
use the minimisation of surprisal—with a more tractable and interpretable
quantity: the variational free energy.

####  2.2.1 An overview so far

All of this suggests that if a model of a subsystem occupies the states which
we expect it to, given what the subsystem is, then we can model that subsystem
as performing inference (see [19]). The idea is predicated on the tautology
that so long as we observe that subsystem as being ‘the thing that it is’,
then it must (by definition) be in a state that is amenable to its continued
existence as that kind of thing. Similarly, if an object remains identifiable
as a particular thing through time, it must be evolving in the way that we
expect of that thing. Aaronson refers to this idea—the existence of a system
in a certain state being conditioned on the result of a computation (and
therefore encoding that computation in its state)—as “anthropic computation”
[32]. One phrasing of our proposal is that using generative models in the
study of coupled random dynamical systems is a way of leveraging anthropic
computation to build models of systems in statistical physics, and that the
FEP gives a formal apparatus with which we can build such models.

Put a different way, the FEP provides us with tools that allow us to model any
particle, or “thing”—in the sense that it exists as a separable, identifiable
thing—as if it were inferring the external states causing its boundary states,
and (re)acting so as to remain in system-like configurations. This
interpretation of the dynamics as inference comes from the minimisation of a
free energy functional of beliefs about the causes of blanket states, as
parameterised by internal states, which guarantees—or describes—the
minimisation of the surprisal of $(\mu,b)$. Using this approach, we can
understand the dynamics or observable behaviour of things as realising a path
of minimal free energy [3, 27]. Since the object cannot ‘see’ past its Markov
blanket, it must infer the external states it is interacting with; moreover,
when it does so, it can be said to maximise self-evidence.

It is interesting to note that there is no guarantee the system’s model is
good; in fact, we expect simple systems with poor ability to self-sustain to
have simple models, meaning the FEP gives us concrete predictions about how
“anthropic computers” might behave when it is too difficult to minimise a
given potential. This has been discussed in the context of the FEP in [33,
34].

###  2.3 Dynamically

We will now explore what the FEP says about the dynamics of objects that
minimise variational free energy, focusing now on the writing of a dynamical
system in terms of free energy gradients—and how it relates physical
properties of an object to the dynamics of beliefs parameterised by the likely
states or trajectories of that object. For similar reviews one can see [2, 4,
19, 27]. We will give a quick example of how the heat dissipated into the
environment by an object behind a boundary maintaining a non-equilibrium
steady state can be written in terms of the variational free energy.

Assume the state space $\mathcal{N}$ is some open, flat Riemannian manifold of
dimension $n$ ($\dim\mathcal{B}=d$, $\dim\mathcal{M}=m$, respectively), where
we set $\ell=n+d+m$. Let $H_{t}$ be a one-dimensional standard Brownian
motion: $\\{H_{t}\\}_{t:\tau}$ has stationary and independent increments,
$H_{0}=0$ almost surely, and $t\mapsto H_{t}$ is continuous almost surely for
all $t\in[0,T]$. Now let $W_{t}$ be a $k$-vector of independent Brownian
motions (also called a $k$-dimensional Wiener process).

By elementary arguments in stochastic analysis, we know that we can write any
stochastic differential equation (SDE) with an equilibrium measure as

$\differential{X_{t}}=-\nabla_{x}\log p(X_{t})+\nu\differential{W_{t}}.$

Arguments in the literature imply this holds for non-equilibrium steady states
as well, with the matrices described in [35, 36]:

$\differential{X_{t}}=-(Q(X_{t})-\Gamma(X_{t}))\nabla_{x}\log
p(X_{t})+\nu(X_{t})\differential{W_{t}}$

where $\nu$ is an $\ell$-by-$k$ Lipshitz continuous matrix field with bounded
linear growth. In stochastic thermodynamics the surprisal of a path given a
particular ensemble of paths is often used—in our notation,

$\differential{X_{t}}=-\nabla\log_{\gamma_{t}(x_{0})}p(\gamma_{t}(x_{0}),t)+\nu(t)\differential{W_{t}}$

where the entries in the diffusion matrix may change in time. In each
situation we have a vector field—the gradient flow of surprisal—added to a
Gaussian distributed random variable with mean zero and variance $t$. (Note
that from here onwards we will fix all fluctuation- and curvature-related
constants to $+1$.)

Recall that $\nabla_{\mu}\mathcal{L}(\mu,b)=0$ if and only if
$\nabla_{\mu}F(\mu,b)=0$. An infinitesimal variation in the KL divergence is
the Fisher information; hence, using the Laplace approximation, in a
neighbourhood of the mode the KL term is a constant. In that case, for a given
blanket state $B_{t}=\beta$ (fixed in time) the conditional mode
evolves444With error bounded locally by some cubic polynomial in
$(\mu-\hat{\mu}_{\beta})$. Note that since $q_{\mu}(\eta)$ is exponential by
assumption, the Hessian of this free energy is non-singular and positive-
definite, such that (i) second-order terms suffice to describe its behaviour
in a neighbourhood of the minimum and (ii) first-order terms vanish at the
minimum. as

$\partial_{t}\hat{\mu}_{\beta}(t)=-\nabla_{\mu}F(\mu,\beta)$

coming from the following SDE on the state space
$\mathcal{M}\crossproduct\mathcal{B}\crossproduct\mathbf{R}^{2k}$:

$\differential{(M_{t},B_{t})}=-\nabla_{(\mu,b)}F(\mu,b)+\differential{(W_{t}^{M},W_{t}^{B})}$

and so on for the other equations above.

Here we recover our original notion of replacing the dynamics of the object
with an inference process: the dynamics of the most likely state of the object
approximately perform a minimisation of the variational free energy
functional. In the path setting this becomes even more apparent, since the
most likely path simply is the solution to the equations of motion minimising
free energy.

####  2.3.1 Stochastic thermodynamics

We may now ask whether the dynamics of beliefs, seen as performing variational
inference, can be related to the physical dynamics of states parameterising
those beliefs. We show that we can relate performing inference to dissipating
heat using this FEP.

We will consider a thermostatted system—an object embedded in a heat bath of
temperature $T$ called a medium or an environment—at non-equilibrium steady
state, such as a colloidal particle moving in one spatial dimension. The force
on the particle is a function of both its state and a time-independent control
parameter [28]. We set the blanket state as the control parameter, since this
determines the mode of the non-equilibrium steady state. In this case, since
we have set $\partial_{t}\hat{\mu}_{b}(t)=-\nabla F(\mu,b)$ in a neighbourhood
of the mode, under mild estimates on the expectation we have the Langevin
equation

$\partial_{t}\mu(t)=-\nabla_{\mu}F(\mu_{t},b)+\xi_{t}$

whose solutions for different realisations of noise are trajectories of the
particle, $\gamma(\mu_{0})$. As in Freidlin–Wentzell theory, such trajectories
are generated with probability

$p(\gamma(\mu_{0}))=\exp{-\frac{1}{T}\int_{0}^{t}(\partial_{s}\mu(s)+\nabla
F(\mu_{s},b))^{2}+\mathrm{div}\,\nabla F(\mu_{s},b)\differential{s}}$

which is simply the magnitude of fluctuations (with the extra term motivated
in [37]).

Let $\langle\ldots\rangle$ be the expectation of some quantity at a given
instant in time. We will denote by $\mathbf{q}[\gamma(\mu_{0})]$ the heat
dissipated along a trajectory, defined as

$\mathbf{q}[\gamma(\mu_{0})]=T\log\frac{p(\gamma(\mu_{0}))}{p(\tilde{\gamma}(\mu_{0}))}$
(1)

where $\tilde{\gamma}(\mu_{0})$ is a time-reversed path.

Computing the difference contained in (1), the heat dissipated into the
environment is

$T\int_{0}^{t}\nabla F(\mu_{s},b)\partial_{s}\mu(s)\differential{s}.$

Indeed, it is known that for a particle diffusion under Langevin dynamics, the
dissipated heat is $T\langle\partial_{t}\mu(t)\rangle\differential\mu$ [38].
Integrating the change in $\mathbf{q}$ along a trajectory, one has the same
expression for $\mathbf{q}[\gamma(\mu_{0})]$, implying the heat flowing out of
the object from the start of a path to its end is proportional to (the
negation of) the difference in free energy from start to finish. If we suppose
the system is maintaining a non-equilibrium steady state, then we have an
explicit form for how much energy is used to perform an inference. From this
we obtain a concrete description of how free energy reservoirs are channelled
from the environment to maintain beliefs, according with self-organisation in
this situation [39] and consistent with existing theories of stochastic
thermodynamics [28].

This construction could be read as an implicit thermodynamics of belief
updating; in the sense that it expresses the work done by a system in terms of
inference. This nicely recapitulates the formulation of thermodynamics in
terms of information length, based upon dynamics (of internal states) [40,
41]. See also: [4]. For example, [40] concludes that “Bennett’s classic
acceptance ratio method for measuring free energy differences also measures
thermodynamic length”. However, the free energy differences in this setting
refer to thermodynamic free energy differences of internal states. The
dénouement of the current treatment is that the same conclusions apply to
variational free energy and, implicitly, beliefs about external states held by
internal states.

##  3 Summary remarks

To summarise: formulated most generally, the FEP says that the average path
taken by an object that is sparsely coupled to its environment is
characterised by a Lyapunov function—a function that decreases monotonically
along the trajectories of a (deterministic) dynamical system. The utility of
Lyapunov functions is that they guarantee stability, allowing us to understand
the dynamical system as flowing on the gradient of the Lyapunov function. In
the FEP, this Lyapunov function is surprisal or variational free energy,
depending on the setting. In other words, to understand the claim that the FEP
models a system as doing inference, it is crucial to understanding the
‘inference’ component of surprisal minimisation. Surprisal is simply the
negative log probability. Variational free energy is a tractable (i.e., easily
computed) upper bound on surprisal, where minimising variational free energy
allows us to infer a target density by approximating it with a variational
density. The surprisal being considered is, much like probability, surprisal
given a model, in that it quantifies how surprising it would be for an
observer—equipped with a model—to find the system (such as itself) in some
non-system-like state or configuration. Minimising surprisal implies that the
variational free energy is minimised (i.e. the object-as-model is a good
estimator of its environment [19]), and likewise, minimising the variational
free energy places an upper bound on the surprisal.

The crucial idea is that the dynamics of a system does inference when written
as minimising a free energy functional, which is itself the consequence of
reading surprisal as a Lyapunov function for those dynamics. In turn, the
dynamics of a system minimise surprisal or free energy whenever that system is
equipped with a Markov blanket; and hence, the dynamics of a system with a
particular partition can always be written as a gradient flow on surprisal.
This gradient flow is what we identify as actually being the process (i.e.,
the performance) of inference. Explicit examples of this can be found in [23,
42].

More specifically, suppose there exists a conditional density over external
states, $q(\eta;\hat{\eta}_{b},\vartheta_{\eta\mid b})$, where we have
parameterised the variational density by the conditional mode $\hat{\eta}_{b}$
and variance $\vartheta_{\eta\mid b}$ of the environment. For convenience, we
assume a Laplace approximation to the posterior, meaning that in a
neighbourhood of its peak, the posterior is approximately Gaussian, and hence
uniquely determined by its mean and variance. This guarantees that finding the
right values of $\hat{\eta}_{b},\vartheta_{\eta\mid b}$ are sufficient for $q$
to equal the posterior almost surely. More generally, under the maximum
entropy assumption of a Gibbs density, any constraint function suffices.

The FEP says that if there exists a boundary in the form of a Markov blanket,
then there exists a function $\sigma$ relating (the dynamics of) internal
states to the environment across the boundary [22, Lemma 4.3]. In that case,
we can map these parameters to $\sigma(\hat{\mu}_{b})$ and
$[\eta-\sigma(\hat{\mu}_{b})]^{2}$, meaning that the _particle_ models its
environment. Ultimately this simply means that, in virtue of being coupled
(via blanket states) to its environment, the particle reflects data about its
environment—or, what it believes that its environment is like. Under the
assumption that a particle which exists in an environment models its
environment, free energy is implicitly a measurement of coherence. If the
particle stops modelling its environment in a particular way (i.e., for a
particular $\sigma$), it must have ceased to exist with the mean that $\sigma$
was constructed for; conversely, if a particle ceases to exist, it will stop
modelling its environment.

If free energy (or surprisal) remains irreducibly high, it is likely that the
system occupies surprising (i.e., implausible or uncharacteristic) states;
this would render it non-existent over some timescale, in the sense that it
would be occupying states that were not characteristic of the thing in
question (e.g., a fish out of water). These statements tie together the
reasoning that things which exist in an environment reflect the statistics of
their environment in a particular way, based on the way they are coupled to
their environment; or, they do not exist that way.

Similarly, note the direction of the implication in Claim 1. In the context of
modelling persistent systems, the FEP is a sort of inversion of the typical
reasoning one sees: instead of saying an object persists (as a thing) because
it is stable, we say an object is stable (as what it is) because it persists
(as that thing) [22]. We stipulate that the KL divergence vanishes—because it
is a tautology that $p$ should equal $q$ almost surely—and because physics
only ‘works’ in a given regime if this is the case (see [19, §IIB] where this
argument is spelt out). This happens in the background, so to speak. Then we
can use the FEP to describe the dynamics as a gradient flow on surprisal—which
can be done by a simple mathematical fact and independently of the FEP—as a
process which has performed inference in virtue of existing. Adding this
‘inflationary’ KL term, the non-equilibrium steady state density of the system
can be understood as a model of the environment under this stipulation. All
this is made possible by the existence of a particular partition. Hence, we
arrive at a deflationary account of intelligence through an inflationary
mathematical technique.

From the viewpoint of model building, since internal states are by
construction unobservable, we can in principle tell any story that we like
about them—provided that this story is supported well enough by observable
evidence. One such story is that those internal states look as if they
parameterise a variational density over external states. Taking this
explanation seriously, and leveraging it as an approach to modelling physical
systems, is applying the FEP.

##  4 Some preliminary philosophical considerations

We will now enter a more philosophical discussion concerning the use of the
FEP. The preceding technical arguments establish the FEP as an explanatory
principle that can be applied to any thing (particle, person, or population)
around which one can draw a boundary or Markov blanket. This means that the
FEP is what it says on the tin: it is a principle. In the physics, this means
that the FEP is the foundation of a method to be applied [3]. So, how might
one apply the FEP?

At the most general level of analysis, it could be argued that any scientific
process whatsoever is an application of the FEP: in the sense that any
scientific explanation or model involves resolving uncertainty about some
observed thing or dynamics, under some hypotheses about the observed world.

More specifically, how might one apply the FEP to a particular particle,
person or population? The answer is straightforward: the observer is trying to
infer the generative model—which contains unobservable internal states of an
object—that best explains the observed dynamics of some thing. What is
observed is simply the blanket states of some ‘thing’ that shield internal
states from direct observation. If a generative model that best explains an
object’s behaviour can be identified, then by construction the FEP affords the
modeller a complete (but not over-complete) description of the object’s
internal dynamics, even though they can never be observed. As noted above,
technical term for this relationship between real-world system and generative
model is “entailment”: where a system that conforms to the dependence
structure of the generative model is said to entail that generative model.

In short, if we equate the territory with the full joint system (crucially,
including the unobserved internal states or dynamics of the observed thing),
then the generative model plays the role of a map that best describes that
(partially observed) territory. Note here, that the map only exists in
relation to the observer—and, crucially, that the observer could be observing
itself. In addition, applications of the FEP enables one to map out the parts
of a territory that are unobservable in another sense; namely, in the sense
that one cannot observe every state a given thing has been, or will be in. And
even if one could, the internal states would be forever sequestered behind
blanket states.

In short, applications of the FEP place the generative model centre stage as
explanations for the behaviour of things—in terms of a map of some territory
that can only be observed through its impressions on its Markov blanket.

##  5 A map of that part of the territory that behaves as if it were a map

We argue that the FEP can be understood, metaphorically, a “map” of sorts: a
map of that part of the territory which behaves or looks as if it were a map.
In other words, the FEP provides us with tools to understand the
mathematically striking property of self-organising systems: that they look as
if they infer, track, reflect, or represent the statistics of their
environment. Even more simply, it gives us tools to model systems that look as
if they are modelling the world.

How does FEP-theoretic technology allow us to model self-organising systems as
modelling their world? The generative model or joint probability
$p(\eta,b,\mu)$ plays a central role. This generative model can be read as a
probabilistic specification of the states or paths characteristically occupied
by the joint particle-environment system. In other words, our scientific model
(or map) includes the generative model entailed by the system (i.e., the
territory). In this sense, the generative model we hypothesise—to account for
observations of an object—allows us to represent the constraints on the kinds
of states the joint system can be found in. In other words, the generative
model is a modeller’s map of the whole “territory” that characterises the
observed system and its environment (including the modeller, as an observer in
the environment).

The technology of the FEP allows us to say that certain components of the
system being considered look as if they track other components, and that this
is a feature of our map contained in a generative model under a particular
partition. In providing us with a calculus allowing us to model things as
modelling other things, the FEP provides us with a map (a scientific model) of
any possible map or tracking relationship between things (namely, a mapping
between internal and external states, mediated by blanket states).555Here, we
are assuming representationalist accounts of scientific practice. For
alternatives, see [10].

Does our ability to always describe some dynamics as a form of inference imply
that those dynamics are literally a form of inference, as opposed to being a
manner of describing those dynamics? Here, it is important to defuse a
possible (but in our view naïve) objection. It is a truism that physical
systems need not explicitly calculate their trajectories of motion, to be
modelled as pursuing such trajectories. In developing a Bayesian mechanical
account of the dynamics of systems, we need not assume that the particle
itself is literally performing inference.666Ways of defending stronger,
increasingly literal versions of this positions are available as well. For a
defence of the claim that physical systems are quite literally in the business
of inference, see [43, 44]. Related to this, views of physical systems
implementing computations has been articulated [45, 32], as well as views
where systems self-organise by computing predictions [46, 47, 42, 29]. What is
at stake with the FEP is an ‘as if’ description. We simply take the fact that
surprisal varies with the dynamics of the system, which implies free energy is
minimised; the minimisation of this quantity is mathematically equivalent to
inference, in the sense of being an estimator.

The fact that it is our model of the system which does inference and not
necessarily the system itself is both a boon and a bane. It enables us to
model the system using the more tractable gradient of variational free energy
and the mathematics of statistical inference, rather than studying the
possibly highly non-linear coupling between the dynamics of the two systems.
However, it is not necessarily a faithful representation of the system in the
literalist sense. See also [10].

In summary, our scientific, FEP-theoretic model is, metaphorically, a map that
allows us to say that some (internal) subset of the system looks as if it
possesses a map; which we interpret formally as tracking the statistics of
another (external) subset. Our map (as scientists and modellers) can be
identified in an unproblematic way as the generative model of the system,
which FEP-theoretic technology enables us to write down. It is by construction
that our model is a model of the modelling capacities of subsets of the real-
world system: that is, our map is precisely a map of how objects behave as if
they were maps, allowing us to construct scientific models of the maps encoded
(or looking as if they were encoded) by the internal subset of the system
considered. This is an answer to allegations that FEP-theoretic modelling
commits the map-territory fallacy: namely, it shows that there is no
conflation of the map and the territory, in the sense of the map we postulate
as being embodied by the system within the mathematical machinery of the FEP.

Taking all the above, we might say that the FEP ultimately provides us with a
map of any possible map whatsoever of, or held by, a physical system. This
echoes seminal work in philosophy by Wittgenstein [48]. In his famous
Tractatus Logico-Philosophicus, Wittgenstein set out to delimit the domain of
sensible propositions by determining the general form of a proposition, an
utterance able to carry a truth value. This set ultimate constraints on what
is sensible, and bounded what is meaningful “from inside” of language itself.
Similarly, the FEP provides ultimate constraints on possible maps or modelling
relations, which arise from what it means to be a map or model of a physical
process at all. Indeed, any modelling process consistent with the laws of
physics must conform to the FEP, such that the FEP sets ultimate limits on
what can count as a map or model—starting from within the technology of
mapping or modelling itself.

In sum, the generative model is a kind of scientific model, which harnesses
what we know about the dependency structure of the system. A variational
density, on the other hand, is a probability density over external states or
paths, implemented in terms of the sufficient statistics of external states
given blanket states. It is an explicitly inferential representation of the
manner in which subsets of a system with a particular partition track each
other: in which case, we are describing a part of the territory that, as seen
in our map, also behaves as if it were a map.

##  6 Some remarks on terminological idiosyncrasies of the FEP literature

In summary, we have argued that the map-territory fallacy, as it has been
leveraged against the FEP (e.g., by [8, 9]) does not apply to FEP-theoretic
modelling—that is, not in the sense that it conflates the model and the thing
modelled. This allegation constitutes a fallacy, which one might call the map-
territory fallacy fallacy.

Mathematically, there is no ambiguity or conflation of map and territory in
FEP-theoretic modelling. Although the language used to narrate the mathematics
of the FEP are at times ambiguous (which may have led to the confusion in
philosophical commentary on the FEP), the mathematical structure itself is
not. In the use of the FEP, an object is modelled as deploying a variational
density, which its internal states parameterise; its dynamics are given in our
generative model, which is meant to capture or represent the real (or most
likely) dependence relations of the real-world system [13]. The It is thus a
scientific principle, allowing us (as modellers) to construct scientific
models (implemented as a generative model of states or paths), the content of
which is precisely a mechanistic theory for the qualified representational
capacities of certain particular systems (the property that they look as if
they infer or track other things in the system, to which they are coupled)
[3].

Charitably, one might argue that the allegations that FEP-theoretic modelling
commits the map-territory fallacy may be due to several terminological
ambiguities and idiosyncrasies that have mired the FEP literature. In
fairness, these ambiguities and idiosyncrasies may lend themselves to
misinterpretation. We now discuss two such cases.

###  6.1 Generative models and variational densities versus recognition
densities

The first concerns the ambiguous discussion or description of two core
constructs in the FEP literature—namely, the generative model and the
variational (or recognition) density. It is noteworthy that the manner in
which these constructs are deployed in the FEP-theoretic formulation is
markedly different from previous Bayesian approaches. This has seldom been
noted in the literature (with a few exceptions, e.g., [13]); and even when it
is discussed, additional issues tend to undermine the clarity of the
exposition (as noted by [9]).

In more traditional machine learning approaches, e.g., in the literature on
the Helmholtz machine [49], the generative model and the recognition model
correspond to the feedforward and the feedback passes that together constitute
the process of data-driven Bayesian inference. In these settings, both the
generative model and the recognition model are straightforwardly encoded in
the connection weights of the Helmholtz machine: the generative model is
encoded in the weighting of the backwards connections, which establish an
information channel from high-level latent spaces of the neural network to its
sensory layers; whereas the recognition model is encoded in the feedforward
connections. In this setting, one assumes that any system that implements this
pair of functionally differentiated weightings (e.g., a brain) implements a
generative and recognition model pair.

Much of the literature on the FEP is based on this reading of the generative
and recognition models: it is often said within this literature that FEP-
theoretic generative models are encoded by the brain, much like these would be
in a Helmholtz machine. However, whilst generative and variational models are
also used in the FEP-theoretic formulation, they are deployed in a markedly
different way, which we discussed extensively above.

Overlooking these differences may have contributed to the impression that
theorists of the FEP conflate the metaphorical map and the territory. Recall
that a generative model, generically, is just a joint probability density
defined over the states (or paths) of a system. In other words, it is our map
or representation of the system being modelled and its mechanics; and more
particularly, it is our model of the dependencies between the variables of the
system, providing us with equations of motion for the whole system (i.e., it
is defined over internal, blanket, and external states/paths).

Technically, we say that the system that we are modelling entails a generative
model; whereas the internal states of the particle (look, to the modeller, as
if they) estimate the parameters of a conditional density over external
states. This difference is crucial to understanding what is at stake in FEP-
theoretic modelling: we say that one subset of variables over which the
generative model is defined (i.e., the internal states) encodes a variational
or recognition model over another subset (the external states). Entailment is
the property that, metaphorically, “the part of the territory that behaves
like a map” (the internal states that encode the parameters of the variational
density) is itself part of the territory described by our map (the generative
model) and so is subject to the mechanics of that territory, as described by
our map. Entailment means that the relationships that hold between variables,
as we represent in the generative model, are obtained as a description of the
dynamics of the real-world system.

The subtle but key point is that the generative model is not encoded by the
internal states of the particle being considered: it is not some kind of
internal model. Rather, the generative model is a description of the
dependence relations that obtain in the dynamics of a system: i.e., such-and-
such dynamics entail such-and-such equations of motion and dependencies. FEP-
theoretic modelling does not assume that the generative model is encoded in
the brain. Arguably, to claim this would indeed constitute a map-territory
fallacy. Under the FEP, parts of the system itself (i.e., by internal states)
look to an external observer as if, or can be modelled as if, they encode the
parameters of a variational model.

However, this raises a question: if it is our scientific model, how can the
generative model play a role in inference if only a conditional posterior
model is encoded by the system? To claim that a system uses or leverages its
generative model is, in some sense, a shorthand expressing the fact that the
system leverages its own causal structure, and the data patterns that it
elicits through action, to generate patterns of adaptive behaviour—all of
which are subject to the regularities that are captured in the generative
model. This is a statement of embodied and enactive cognition [50, 13]. The
system, in having sensory states, also has access to the causal structure in
the generative model.

So, the generative model is our model or map, but it denotes the causal
structure of the system being modelled (i.e., the territory). This causal
structure—and in particular, the presence of a Markov blanket or particular
partition—means that some parts of the system (the internal states) encode a
variational density over some other parts of the system (the external states);
that is, they “act like a map” or model the statistics of external states.

###  6.2 Generative models and generative processes

The second ambiguity concerns the generative model and the so-called
“generative process”. Some presentations of the FEP sometimes define the
generative model as a scientific model of some data-generating generative
process; see, for instance, [51, 13]). On this reading, the generative model
is our “map” of some real-world process, namely, the generative process—which
would then correspond to the metaphorical territory.

In practice, however, in FEP-theoretic modelling, this distinction does not
map onto that between a scientific model and real-world system, i.e., onto the
distinction between the map and territory. Perhaps confusingly, the generative
process does not denote (the structure of) the real data-generating process
that we are trying to model. In FEP-theoretic formulations, the generative
process is also part of our scientific model.

It should be noted that, practically, the distinction between generative model
and generative process only comes into play when we are implementing the
dynamics of a specific system in simulations. When we write code to simulate
the system of interest, typically, we need to specify (i) the dependencies
that hold between the variables of a system, which we call the generative
model; and (ii) the equations of motion that govern the time evolution of the
external states per se in our simulation. The latter is labelled the
‘generative process’, since it generates the dynamics of external states,
which are tracked by internal states. The key thing to note, however, is that
generative process is also part of our scientific model—it harnesses what we
know about the physics of the situation described by the generative model, in
which the thing is embedded and ‘lives’.

##  7 Conclusion

We argued that, in describing ‘things’ or ‘particles’ as estimators or
(deflated) representations of the systems to which they are coupled, the FEP-
theoretic apparatus does not commit the map territory fallacy, i.e., it is
false that applications the FEP necessarily reify aspects of the metaphorical
map (i.e., our scientific model), mistakenly taking them to be part of the
territory (i.e., the real-world system that we wish to model); although this
remains possible in principle. We have further argued that this allegation
itself constitutes a map-territory fallacy fallacy. In distinguishing the
generative model and the variational density—and the generative model and the
generative process from the real-world system that we aim to model—the
technology of FEP-theoretic modelling allows us to construct a map of that
part of the territory which looks to an external observer as if or behaves as
if it were a map, without committing model reification. The FEP is thus,
metaphorically, a map of any possible map whatsoever of, or held by, a
physical system. One ought to celebrate this ‘territory-map mapping,’ and the
FEP is ultimately, at its core, a principled approach to the formalisation of
this mapping.

## References

  * [1] Maxwell J D Ramstead, Dalton A R Sakthivadivel, and Karl J Friston.  On the map-territory fallacy fallacy.  2022\.  Preprint arXiv:2208.06924.
  * [2] Karl J Friston.  A free energy principle for a particular physics.  2019\.  Preprint arXiv:1906.10184.
  * [3] Maxwell J D Ramstead, Dalton A R Sakthivadivel, Conor Heins, Magnus Koudahl, Beren Millidge, Lancelot Da Costa, Brennan Klein, and Karl J Friston.  On Bayesian mechanics: a physics of and by beliefs.  Interface Focus, 13(3):20220029, 2023.
  * [4] Lancelot Da Costa, Karl J Friston, Conor Heins, and Grigorios A Pavliotis.  Bayesian mechanics for stationary processes.  Proceedings of the Royal Society A, 477(2256), 2021.
  * [5] Karl Friston, Lancelot Da Costa, Noor Sajid, Conor Heins, Kai Ueltzhöffer, Grigorios A Pavliotis, and Thomas Parr.  The free energy principle made simpler but not too simple.  Physics Reports, 1024:1–29, 2023.
  * [6] Nancy Cartwright.  How the Laws of Physics Lie.  Oxford University Press, 1984.
  * [7] Nancy Cartwright.  Nature’s Capacities and their Measurement.  Oxford University Press, 1989.
  * [8] Jelle Bruineberg, Krzysztof Dolega, Joe Dewhurst, and Manuel Baltieri.  The emperor’s new Markov blankets.  Behavioral and Brain Sciences, pages 1–63, 2020.
  * [9] Thomas van Es.  Living models or life modelled? On the use of models in the free energy principle.  Adaptive Behavior, 2020.
  * [10] Mel Andrews.  The math is not the territory: navigating the free energy principle.  Biology & Philosophy, 36(3):30, 2021.
  * [11] Mel Andrews.  Making reification concrete: a response to Bruineberg et al.  Behavioral and Brain Sciences, 45:e186, 2022.
  * [12] Michael D Kirchhoff, Julian Kiverstein, and Ian Robertson.  The literalist fallacy and the free energy principle: model-building, scientific realism, and instrumentalism.  The British Journal for the Philosophy of Science, 2022.
  * [13] Maxwell J D Ramstead, Michael D Kirchhoff, and Karl J Friston.  A tale of two densities: active inference is enactive inference.  Adaptive Behavior, 28(4):225–239, 2020.
  * [14] Karl J Friston.  A free energy principle for biological systems.  Entropy, 14(11):2100–2121, 2012.
  * [15] Maxwell J D Ramstead, Karl J Friston, and Inês Hipólito.  Is the free-energy principle a formal theory of semantics? From variational density dynamics to neural and phenotypic representations.  Entropy, 2020.
  * [16] Maxwell J D Ramstead.  The empire strikes back: some responses to Bruineberg and colleagues.  Behavioral and Brain Sciences, 45:e205, 2022.
  * [17] Judea Pearl.  Graphical Models for Probabilistic and Causal Reasoning, pages 367–389.  Springer, 1998.
  * [18] Dalton A R Sakthivadivel.  Weak Markov blankets in high-dimensional, sparsely-coupled random dynamical systems.  2022\.  Preprint arXiv:2207.07620.
  * [19] Dalton A R Sakthivadivel.  A worked example of the Bayesian mechanics of classical objects.  In Active Inference: Third International Workshop, IWAI 2022, Grenoble, France, September 19, 2022, Revised Selected Papers, pages 298–318. Springer, 2023.
  * [20] Karl J Friston, Erik D Fagerholm, Tahereh S Zarghami, Thomas Parr, Inês Hipólito, Loïc Magrou, and Adeel Razi.  Parcels and particles: Markov blankets in the brain.  Network Neuroscience, 5(1):211–251, 2021.
  * [21] Ludwig Arnold.  Random Dynamical Systems.  Springer Monographs in Mathematics. Springer Berlin Heidelberg, 1998.  2003 edition.
  * [22] Dalton A R Sakthivadivel.  Towards a geometry and analysis for Bayesian mechanics.  2022\.  Preprint arXiv:2204.11900.
  * [23] Thomas Parr, Lancelot Da Costa, and Karl Friston.  Markov blankets, information geometry and stochastic thermodynamics.  Philosophical Transactions of the Royal Society A, 378(2164):20190159, 2020.
  * [24] Romain Brette.  Is coding a relevant metaphor for the brain?  Behavioral and Brain Sciences, 42:e215, 2019.
  * [25] Matthew James Beal.  Variational algorithms for approximate Bayesian inference.  University of London, University College London (United Kingdom), 2003.
  * [26] Karl J Friston.  The free-energy principle: a unified brain theory?  Nature Reviews Neuroscience, 11(2):127–138, 2010.
  * [27] Karl Friston, Lancelot Da Costa, Dalton A R Sakthivadivel, Conor Heins, Grigorios A Pavliotis, Maxwell Ramstead, and Thomas Parr.  Path integrals, particular kinds, and strange things.  Physics of Life Reviews, 47:35–62, 2023.
  * [28] Udo Seifert.  Stochastic thermodynamics, fluctuation theorems and molecular machines.  Reports on Progress in Physics, 75(12):126001, 2012.
  * [29] Udo Seifert.  From stochastic thermodynamics to thermodynamic inference.  Annual Review of Condensed Matter Physics, 10:171–192, 2019.
  * [30] Radford M Neal and Geoffrey E Hinton.  A view of the EM algorithm that justifies incremental, sparse, and other variants.  In Learning in Graphical Models, pages 355–368. Springer, 1998.
  * [31] Steve Pressé, Kingshuk Ghosh, Julian Lee, and Ken A Dill.  Principles of maximum entropy and maximum caliber in statistical physics.  Reviews of Modern Physics, 85(3):1115, 2013.
  * [32] Scott Aaronson.  Guest column: NP-complete problems and physical reality.  ACM SIGACT News, 36(1):30–52, 2005.
  * [33] Dalton A R Sakthivadivel.  Regarding flows under the free energy principle: a comment on “How particular is the physics of the free energy principle?” by Aguilera, Millidge, Tschantz, and Buckley.  Physics of Life Reviews, 42:25–28, 2022.
  * [34] Karl J Friston.  Very particular: comment on “How particular is the physics of the free energy principle?”.  Physics of Life Reviews, 41:58–60, 2022.
  * [35] Ping Ao.  Potential in stochastic differential equations: novel construction.  Journal of physics A: Mathematical and General, 37(3):L25, 2004.
  * [36] Lancelot Da Costa and Grigorios A Pavliotis.  The entropy production of stationary diffusions.  Journal of Physics A: Mathematical and Theoretical, 56(36):365001, 2023.
  * [37] Leticia F Cugliandolo, Vivien Lecomte, and Frédéric Van Wijland.  Building a path-integral calculus: a covariant discretization approach.  Journal of Physics A: Mathematical and Theoretical, 52(50):50LT01, 2019.
  * [38] Ken Sekimoto.  Langevin equation and thermodynamics.  Progress of Theoretical Physics Supplement, 130:17–27, 1998.
  * [39] Kai Ueltzhöffer.  On the thermodynamics of prediction under dissipative adaptation.  arXiv:2009.04006, 2020.
  * [40] Gavin E Crooks.  Measuring thermodynamic length.  Physical Review Letters, 99(10):100602, 2007.
  * [41] Eun-jin Kim.  Investigating information geometry in classical and quantum systems through information length.  Entropy, 20(8):574, 2018.
  * [42] Kai Ueltzhöffer, Lancelot Da Costa, Daniela Cialfi, and Karl J Friston.  A drive towards thermodynamic efficiency for dissipative structures in chemical reaction networks.  Entropy, 23(9):1115, 2021.
  * [43] Alex B Kiefer.  Literal perceptual inference.  In Philosophy and Predictive Processing, 2017.
  * [44] Alex B Kiefer.  Psychophysical identity and free energy.  Journal of the Royal Society Interface, 17(169):20200370, 2020.
  * [45] Clare Horsman, Susan Stepney, Rob C Wagner, and Viv Kendon.  When does a physical system compute?  Proceedings of the Royal Society A, 470(2169):20140182, 2014.
  * [46] Susanne Still, David A Sivak, Anthony J Bell, and Gavin E Crooks.  Thermodynamics of prediction.  Physical Review Letters, 109(12):120604, 2012.
  * [47] Nikolay Perunov, Robert A Marsland, and Jeremy L England.  Statistical physics of adaptation.  Physical Review X, 6(2):021036, 2016.
  * [48] Ludwig Wittgenstein.  Tractatus Logico-Philosophicus.  1922\.  2013 edition, Routledge.
  * [49] Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel.  The Helmholtz machine.  Neural Computation, 7(5):889–904, 1995.
  * [50] Evan Thompson.  Mind in life.  Harvard University Press, 2010.
  * [51] Thomas Parr, Giovanni Pezzulo, and Karl J Friston.  Active Inference: The Free Energy Principle in Mind, Brain, and Behavior.  MIT Press, 2022.