# Collective behavior from surprise minimization

Conor Heins<EMAIL_ADDRESS>Department of Collective Behaviour, Max Planck
Institute of Animal Behavior, Konstanz D-78457, Germany  Centre for the
Advanced Study of Collective Behaviour, University of Konstanz, Konstanz
D-78457, Germany  Department of Biology, University of Konstanz, Konstanz
D-78457, Germany  VERSES Research Lab, Los Angeles, CA 90016, USA  Beren
Millidge  Medical Research Council Brain Networks Dynamics Unit, University of
Oxford, Oxford OX1 3TH, UK  Lancelot Da Costa  VERSES Research Lab, Los
Angeles, CA 90016, USA  Department of Mathematics, Imperial College London,
London SW7 2AZ, UK  Wellcome Centre for Human Neuroimaging, University College
London, London, WC1N 3AR, UK  Richard P. Mann  Department of Statistics,
School of Mathematics, University of Leeds, Leeds, LS2 9JT, UK  Karl Friston
VERSES Research Lab, Los Angeles, CA 90016, USA  Wellcome Centre for Human
Neuroimaging, University College London, London, WC1N 3AR, UK  Iain D. Couzin
<EMAIL_ADDRESS>Department of Collective Behaviour, Max Planck Institute of
Animal Behavior, Konstanz D-78457, Germany  Centre for the Advanced Study of
Collective Behaviour, University of Konstanz, Konstanz D-78457, Germany
Department of Biology, University of Konstanz, Konstanz D-78457, Germany

###### Abstract

Collective motion is ubiquitous in nature; groups of animals, such as fish,
birds, and ungulates appear to move as a whole, exhibiting a rich behavioral
repertoire that ranges from directed movement to milling to disordered
swarming. Typically, such macroscopic patterns arise from decentralized, local
interactions among constituent components (e.g., individual fish in a school).
Preeminent models of this process describe individuals as self-propelled
particles, subject to self-generated motion and ‘social forces’ such as short-
range repulsion and long-range attraction or alignment. However, organisms are
not particles; they are probabilistic decision-makers. Here, we introduce an
approach to modelling collective behavior based on active inference. This
cognitive framework casts behavior as the consequence of a single imperative:
to minimize surprise. We demonstrate that many empirically-observed collective
phenomena, including cohesion, milling and directed motion, emerge naturally
when considering behavior as driven by active Bayesian inference — without
explicitly building behavioral rules or goals into individual agents.
Furthermore, we show that active inference can recover and generalize the
classical notion of social forces as agents attempt to suppress prediction
errors that conflict with their expectations. By exploring the parameter space
of the belief-based model, we reveal non-trivial relationships between the
individual beliefs and group properties like polarization and the tendency to
visit different collective states. We also explore how individual beliefs
about uncertainty determine collective decision-making accuracy. Finally, we
show how agents can update their generative model over time, resulting in
groups that are collectively more sensitive to external fluctuations and
encode information more robustly.

## Significance Statement

We introduce a model of collective behavior, proposing that individual members
within a group, such as a school of fish or a flock of birds, act to minimize
surprise. This active inference approach naturally generates well-known
collective phenomena such as cohesion and directed movement without explicit
behavioral rules. Our model reveals intricate relationships between individual
beliefs and group properties, demonstrating that beliefs about uncertainty can
shape collective decision-making accuracy. As agents update their generative
model in real-time, groups become more sensitive to external perturbations and
more robust in encoding information. Our work provides fresh insights into
understanding collective dynamics and could inspire strategies in the study of
animal behavior, swarm robotics and distributed systems.

## Introduction

The principles underlying coordinated group behaviors in animals have inspired
research in disciplines ranging from zoology to engineering to physics [1, 2,
3]. Collective motion in particular has been a popular phenomenon to study,
due in part to its striking visual manifestation and ubiquity (e.g., swarming
locusts, schooling fish, flocking birds and herding ungulates), and in part to
the simplicity of models that can reproduce many of its qualitative features;
like cohesive, directed movement [98, 99, 100, 97]. Because of this,
collective motion is often cited as a canonical example of a self-organizing
complex system, wherein collective properties emerge from simple interactions
among distributed components.

Popular theoretical models cast collective motion as groups composed of self-
propelled particles (SPPs) that influence one another via simple ‘social
forces.’ Early models like the Vicsek model [100] consider only a simple
alignment interaction, where each particle aligns its direction of travel with
the average heading of its neighbors. While oversimplifying the biological
mechanisms in play, SPP models — like the Vicsek model — are useful for their
amenability to formal understanding, e.g., the computation of universal
quantities and relations through hydrodynamic and mean-field limits [8, 9, 10,
11].

Recent research has shifted towards more biologically-motivated approaches
that aim to model the specific behavioral circuits and decision-rules that
govern individual behaviors [12, 13, 14, 15]. While these models are less
analytically-tractable than SPP models, they are more appealing to domain
specialists like biologists, as they can generate predictions about sensory
features in an individual’s environment that are necessary and sufficient for
evoking behavior. Furthermore, these predictions can be tested experimentally
[16, 14]. This data-driven approach can thus provide mechanistic insights into
the biological and cognitive origins of decision-making [17, 13].

In this work, we propose a model class that blends the first-principles,
theoretical approach of physical models with biological-plausibility,
resulting in an ecologically-valid but theoretically-grounded agent-based
model of collective behavior. Our model class is based on active inference, a
framework for designing and describing adaptive systems where all aspects of
cognition — learning, planning, perception, and action — are viewed as a
process of inference [18, 19, 95, 21]. Active inference originated in
theoretical neuroscience as a normative account of self-organizing, biological
systems as constantly engaged in predictive exchanges with their sensory
environments [90, 23, 94, 25].

## Collective motion models: from self-propelled particles to Bayesian agents

In popular self-propelled particle models, an individual’s movement is
described as driven by a combination of social and environmental forces. These
forces are often treated as vectors that capture various tendencies seen in
biological collective motion, such as repulsion, attraction (to neighbors or
external targets), and alignment. These forces can then be combined with
various nonlinearities and weights to capture mechanisms of interaction.

In contrast, the active inference approach forgoes specifying explicit
vectorial forces, and instead starts by modelling all behavior as the solution
to an inference problem, namely the problem of inferring the latent causes of
sensations. Perception and action strive to improve the agent’s predictions of
sensory inputs, based on its internal model of its world (see Figure 1A). By
equipping this internal model with expectations about the environment’s
underlying tendencies, ‘social forces’ can emerge naturally as agents attempt
to suppress sensory data that are mismatched with their expectations. This
perspective-shift offers a unifying modelling ontology for describing adaptive
behavior, while also resonating with cybernetic principles like homeostatic
regulation and process theories of neural function like predictive coding [26,
93, 28, 29].

Active inference blends the construct validity of cognitivist approaches with
the first-principles elegance of physics-based approaches by invoking
minimization of a single, all-encompassing objective function that explains
behavior: surprise, or, under certain assumptions, prediction error. As an
example of this perspective shift, in this work we investigate a specific
class of generative models that can be used to account for the types of
collective behaviors exhibited by animal groups. In doing so, we hope to
showcase the benefits of the framework, while also proposing a testable model
class for use in studies of biological collective motion.

Figure 1: A: Schematic illustrating the Bayesian perspective in the context of
our single agents, where the hidden states of the environment are segregated
from a focal agent by means of sensory data $y_{t}$ (right panel of A). This
contrasts with classic self-propelled particle models (left panel of A), where
environmental or social information manifests in terms of social forces on the
focal individual, who emits its own actions based on hand-crafted decision-
rules (e.g., changes to heading direction). B: Schematic illustration of the
sector-specific distance tracking. The left panel shows a Bayesian network
representation of a dynamic generative model (i.e., a time-series model), that
represents the time-evolution of a latent variable $x_{1,...,T}$ and
simultaneous observations $y_{1,...T}$. Shown are both a standard time-series
representation (lower left) and its equivalent representation as generalized
coordinates of motion
$\tilde{x}_{t}=(x_{t},x_{t}^{\prime},x_{t}^{\prime\prime},...)$ (right). We
show the orders of differentiation used for our model in practice (3 orders of
motion for $\tilde{x}$ and 2 orders of motion for $\tilde{y}$). The middle
panel of B shows how each component of the vectorial hidden state
$\mathbf{x}=(x_{h,1},...,x_{h,L})$ is computed as the average nearest-neighbor
distance for the neighbors within each visual sector. Observations are
generated as noisy, Gaussian samples centered on the sector-wise distance
hidden state (right panel of B). This requires the agent to estimate the true
hidden state $x_{t}$ by performing inference with respect to a generative
model of how sensory data are generated
$p(\tilde{\mathbf{y}},\tilde{\mathbf{x}})$.

## Active inference and generative models of behavior

A common pipeline in the quantitative study of animal behavior involves
selecting a candidate behavioral algorithm or decision rule that may explain a
given behavior, and then fitting the parameters of the candidate model to
experimental or observational data [30, 16]. While these approaches often
yield strong quantitative fits to data, the explanatory power of the models
reduces to the interpretation of hard-coded parameters, which often have
opaque relationships to real biological mechanisms or constructs [31].

In the active inference framework we rather ask: what is the minimal model an
organism might have of its environment that is sufficient to explain its
behavior? Behavior is cast as the process by which the agent minimizes
surprise or prediction error, with respect to this model of the world [90,
32]. The principle of prediction-error minimization enjoys empirical support
in neuroscience [26, 33] and a theoretical basis in the form of the Free
Energy Principle [90, 23, 25], an account of all self-organizing systems that
casts them as implicit models of their environments, ultimately in the service
of minimizing the surprise (a.k.a., self-information) associated with sensory
states [34, 35, 36].

What states-of-affairs count as surprising hinges on a generative model that
can assign a likelihood to sensory data. When it comes to modelling behavior
driven by this principle, the challenge then becomes specifying a generative
or world model, whereby a particular pattern of behavior simply emerges by
minimizing surprise.

According to active inference, agents minimize surprise by changing their
beliefs about the world (changing which observations are considered
surprising) or by acting on the world to avoid surprising sensory data. The
former strategy is thought to correspond to passive processes such as
perception and learning, whereas the latter corresponds to processes like
active sensing and movement. Action is thus motivated by the desire to
generate sensations that are as least surprising as possible.

In this paper, we describe the motion of mobile, mutually-sensing agents as
emerging from a process of collective active inference, whereby agents both
estimate the hidden causes of their sensations, while also actively changing
their position in space in order to minimize prediction error. In contrast to
models that use pre-specified behavioral rules for generating behavior,
generative models entail collective behavior by appealing to a probabilistic
representation of how an organism’s sensory inputs are generated.

## A generative model for a (social) particle

We now consider a sufficient generative model for an individual in a moving
group. We equip this individual, hereafter referred as the focal agent, with a
representation of a simple random variable: the local distance $x$ between
itself and its neighbors. For generality, we can expand this into a
multivariate random variable to describe a set of distances
$\mathbf{x}=(x_{1},x_{2},...,x_{L})$ that track the distance between the focal
agent and its neighbors within $L$ different sensory sectors (see Figure 1B).
We analogize these $L$ sectors to adjacent visual fields of an agent’s field
of view [37, 38].

The focal agent possesses a model of the distance(s) $\mathbf{x}$ and its
sensations thereof $\mathbf{y}$. In particular, our focal agent represents the
dynamics of $\mathbf{x}$ using a stochastic differential equation (a.k.a., a
state-space model) defined by a drift $\mathbf{f}$ and some stochastic forcing
$\omega$ — we refer to this component of the generative model as the dynamics
model. The stochastic term $\omega$ captures the agent’s uncertainty about
paths of $\mathbf{x}$ over time. The agent also believes it can sense
$\mathbf{x}$ via observations $\mathbf{y}$, mediated by a sensory map, which
we call the observation model. This is defined by some (possibly non-linear)
function $\mathbf{g}$ with additive noise $z$. The agent’s generative model is
then fully described by a pair of equations that detail 1) the time-evolution
of the distance and 2) the simultaneous generation of sensory samples of the
distance:

$\displaystyle D\tilde{\mathbf{x}}$
$\displaystyle=\tilde{\mathbf{f}}+\tilde{\boldsymbol{\omega}}$
$\displaystyle\tilde{\mathbf{y}}$
$\displaystyle=\tilde{\mathbf{g}}+\tilde{\boldsymbol{z}}$ (1)

All random variables are described using generalized coordinates of motion
with the convention
$\tilde{\mathbf{q}}=\\{\mathbf{q},\mathbf{q}^{\prime},\mathbf{q}^{\prime\prime},...\\}$.
Generalized coordinates allow us to represent the trajectory of a random
variable using a vector of local time derivatives (position, velocity,
acceleration, etc.). The matrix $D$ is a generalized derivative operator that
moves a vector of generalized coordinates up one order of motion
$D(x,x^{\prime},x^{\prime\prime},...)^{\top}=(x^{\prime},x^{\prime\prime},x^{\prime\prime\prime},...)^{\top}$.
The generalized functions $\tilde{\mathbf{f}}$ and $\tilde{\mathbf{g}}$
therefore operate on vectors of generalized coordinates (see SI Appendix,
Section A for details on generalized coordinates and filtering).

## Generalized filtering and active inference

An agent equipped with this dynamic generative model then performs active
inference by updating its beliefs (state estimation, or filtering) and control
states (action) to minimize surprise.

Inference entails updating a probabilistic belief over hidden states
$\tilde{\mathbf{x}}$ in the face of sensory data $\tilde{\mathbf{y}}$. Our
agents solve this filtering problem using generalized filtering [83, 40], an
algorithm for approximate Bayesian inference and parameter estimation on
dynamic state-space models. This is achieved by minimizing the variational
free energy $F$, a tractable upper bound on surprise (i.e., negative log
evidence or marginal likelihood). The agent minimizes the free energy with
respect to a belief distribution $q(\tilde{\mathbf{x}})$ with parameters
$\nu$; this approximates the true posterior
$q_{\nu}(\tilde{\mathbf{x}})\approx p(\tilde{\mathbf{x}}|\tilde{\mathbf{y}})$,
which is the optimal solution in the context of Bayesian inference. The true
posterior $p(\tilde{\mathbf{x}}|\tilde{\mathbf{y}})$ is difficult to compute
for many generative models due to the difficult calculation of the marginal
(log) likelihood $\ln p(\tilde{\mathbf{y}})$. Variational methods circumvent
this intractable marginalization problem by replacing it with a tractable
optimization problem: namely, adjusting an approximate posterior to match the
true posterior by minimizing $F$ with respect to its (variational) parameters
$\nu$.

We parameterize $q(\tilde{\mathbf{x}})$ as a Gaussian with mean-vector
$\tilde{\boldsymbol{\mu}}$, which is a natural choice for this generative
model since the assumption of normally-distributed noises
$\tilde{\mathbf{z}},\tilde{\boldsymbol{\omega}}$ imply that the true posterior
will be Gaussian near the posterior mode $\arg\max
p(\tilde{\mathbf{x}}|\tilde{\mathbf{y}})$. The implicit Gaussian (i.e.,
Laplace) assumption is ubiquitous in the modelling and signal processing
literature [41] and can be regarded as a ‘minimal’ assumption, by appeal to
things like the central limit theorem and related principles (e.g., Jaynes’
maximum entropy principle). According to generalized filtering,
$\tilde{\boldsymbol{\mu}}$ is updated using a sum of prediction errors:

$\displaystyle\frac{d\tilde{\boldsymbol{\mu}}}{dt}$
$\displaystyle\propto-\nabla_{\tilde{\boldsymbol{\mu}}}F(\tilde{\boldsymbol{\mu}},\tilde{\mathbf{y}})$
$\displaystyle\propto\tilde{\boldsymbol{\varepsilon}}_{z}-\tilde{\boldsymbol{\varepsilon}}_{\omega}$
$\displaystyle\ \textrm{where}\hskip
5.69054pt\tilde{\boldsymbol{\varepsilon}}_{z}$
$\displaystyle=\tilde{\mathbf{y}}-\tilde{\mathbf{g}}(\tilde{\boldsymbol{\mu}})$
$\displaystyle\ \tilde{\boldsymbol{\varepsilon}}_{\omega}$
$\displaystyle=D\tilde{\boldsymbol{\mu}}-\tilde{\mathbf{f}}(\boldsymbol{\tilde{\mu}})$
(2)

The ensuing evidence accumulation can be regarded as a natural generalization
of predictive coding [26, 42, 43], where beliefs about local trajectories
$\tilde{\boldsymbol{\mu}}$ are updated using a running assimilation of sensory
and model prediction errors: $\tilde{\boldsymbol{\varepsilon}}_{z}$ and
$\tilde{\boldsymbol{\varepsilon}}_{\omega}$, respectively. For notational
clarity, we have omitted terms that weigh these prediction errors; the so-
called generalized sensory and model precisions
$\tilde{\Pi}^{\textbf{z}},\tilde{\Pi}^{\boldsymbol{\omega}}$, which encode the
agent’s assumptions about the magnitude and correlation structure of noise.
The importance of these precisions will become clear later, when understanding
the relationship between precision-weighted prediction errors and social
forces.

While inference entails changing the approximate posterior means
$\tilde{\boldsymbol{\mu}}$ to best explain sensory data, action entails
changing the data itself to better match the data to one’s current beliefs.
Similar to the update scheme in (A.21), actions are also updated by minimizing
free energy:

$\displaystyle\frac{da}{dt}$
$\displaystyle=-\nabla_{a}F(\tilde{\boldsymbol{\mu}},\tilde{\mathbf{y}}(a))$
$\displaystyle=-\nabla_{\tilde{\mathbf{y}}}F(\tilde{\boldsymbol{\mu}},\tilde{\mathbf{y}}(a))\nabla_{a}\tilde{\mathbf{y}}(a)$
$\displaystyle\propto-\tilde{\boldsymbol{\varepsilon}}_{z}^{\top}\nabla_{a}\tilde{\mathbf{y}}(a)$
(3)

Actions thus are updated using a product of sensory prediction errors
$\tilde{\boldsymbol{\varepsilon}}_{z}$ and a ‘sensorimotor contingency’
$\nabla_{a}\tilde{\mathbf{y}}(a)$ or reflex arc. This sort of ‘reflexive
action’ — where control is simply targeted at minimizing sensory prediction
errors — underlies active inference accounts of motor control [93, 44], and
can be formally related to proportional-integral-derivative (PID) control
[96]. These prediction errors measure how far an agent’s observations are from
its expectations; the agent then acts using (3) to minimize this deviation.
Active inference agents are thus driven to act in a way that aligns with their
(biased) expectations about the world [88]. In the next section, we will see
how building a particular type of bias into each agent’s generative model
leads to the appearance terms in (3) that resemble social forces.

## Social forces as a consequence of predictive control

In particular, we take the agent’s action to be its heading direction
$a=\mathbf{v}$, and examine the case where the agent observes the distance to
its neighbors within a single sensory sector, i.e., $L=1$,
$\mathbf{x}=(x_{1})$. We distinguish the agent’s representation of the
distance $\mathbf{x}$ from the actual distance using the subscript $h$.
Therefore $\mathbf{x}_{h}=(x_{h,1},x_{h,2},...,x_{h,L})$ denotes the average
distances (and corresponding sensory samples $\mathbf{y}_{h}$) calculated
using the actual positions of other agents. For the case of $L=1$, and
assuming the agent observes both the distance and its rate of change
$y^{\prime}_{h,1}$, this is:

$\displaystyle x_{h,1}$ $\displaystyle=\frac{1}{K}\sum_{j\in
N_{in}}\|\mathbf{r}_{j}-\mathbf{r}\|$ $\displaystyle y_{h,1}$
$\displaystyle=x_{h,1}+z_{h,1}$ $\displaystyle x^{\prime}_{h,1}$
$\displaystyle=\frac{dx_{h,1}}{dt}$ $\displaystyle y^{\prime}_{h,1}$
$\displaystyle=x^{\prime}_{h,1}+z^{\prime}_{h,1}$ (4)

$N_{in}$ is the set of neighbors within the agent’s single sensory sector, $K$
is the size of this set, $\mathbf{r}$ is the focal agent’s position vector,
and $\mathbf{r}_{j}$ is the position vectors of neighbor $j$. The sensory
observation of the generalized distance
$\tilde{y}_{h}=(y_{h,1},y^{\prime}_{h,1})$ is a sample of the hidden state,
perturbed by some additive noises $\tilde{z}=(z_{h,1},z^{\prime}_{h,1})$. By
expanding the active inference control rule in (3), we arrive at the following
differential equation for the heading vector:

$\displaystyle\frac{d\mathbf{v}}{dt}$
$\displaystyle=\xi^{\prime}_{z}\Delta\hat{\mathbf{r}}$
$\displaystyle\xi^{\prime}_{z}$
$\displaystyle=\pi^{\prime}_{z,1}(y^{\prime}_{h,1}-\mu^{\prime}_{h,1})$
$\displaystyle\Delta\hat{\mathbf{r}}$ $\displaystyle=\frac{1}{K}\sum_{j\in
N_{in}}\frac{\Delta\mathbf{r}_{j}}{\|\Delta\mathbf{r}_{j}\|},\Delta\mathbf{r}_{j}=\mathbf{r}_{j}-\mathbf{r}$
(5)

The average vector $\Delta\hat{\mathbf{r}}$ is exactly the (negative)
‘sensorimotor contingency’ term $\nabla_{a}\tilde{\mathbf{y}}(a)$ from (3)
(see SI Appendix, Section A for detailed derivations):

$\displaystyle\nabla_{\mathbf{v}}\tilde{y}(\mathbf{v})$
$\displaystyle=\nabla_{\mathbf{r}}y=\frac{1}{K}\sum_{j\in
N_{in}}\frac{\mathbf{r}-\mathbf{r}_{j}}{\|\mathbf{r}-\mathbf{r}_{j}\|}=-\Delta\hat{\mathbf{r}}$
(6)

The simple action update in (5) means that the focal agent moves along a
vector pointing towards the average position of its neighbors. Whether this
movement is attractive or repulsive is determined by the sign of the
precision-weighted prediction error
$\xi^{\prime}_{z}=\pi^{\prime}_{z,1}(y^{\prime}_{h,1}-\mu^{\prime}_{h,1})$,
and its magnitude depends on two factors: 1) the sensory precision or
‘reliability’ $\pi^{\prime}_{z,1}$ that the agent affords observations of the
rate-of-change of $y_{h,1}$; and 2) the degree to which these rate-of-change
observations deviate from their predicted value
$y^{\prime}_{h,1}-\mu^{\prime}_{h,1}$.

The presence of both attractive and repulsive forces depends on the agent’s
model of the distance dynamics, captured by the functional form of
$\tilde{\mathbf{f}}$. In particular, consider forms of $\tilde{\mathbf{f}}$
that relax $\mathbf{x}$ to some attracting fixed point $\eta>0$. Equipped with
such a stationary model of the local distance, inference dynamics (c.f.,
(A.21)) will constantly bias its predictions $\mu$ according to the prior
belief that the distance is pulled to $\eta$. Given this biased dynamics model
and the action update in (3), such an agent will move to ensure that distance
observations $\tilde{y}_{h}$ are equal to the fixed point $\eta$.

This action update shows immediate resemblance to the attractive and repulsive
vectors common to social force-based models [97, 98, 99], which often share
the following general form:

$\displaystyle F_{attr}$ $\displaystyle\propto\sum_{j\in
Z_{A}}\frac{\mathbf{r}_{ij}}{\|\mathbf{r}_{ij}\|}$ $\displaystyle F_{repul}$
$\displaystyle\propto-\frac{1}{K}\sum_{j\in
Z_{R}}\frac{\mathbf{r}_{ij}}{\|\mathbf{r}_{ij}\|}$

where $Z_{A},Z_{R}$ refer to distance-defined zones of attraction or
repulsion, respectively. In the active inference framework, these social
forces emerge as the derivative of the observations with respect to action
$\nabla_{a}\tilde{y}$, where the sign and magnitude of the precision-weighted
sensory prediction error $\xi^{\prime}_{z}$ determines whether the vector is
attractive (towards neighbors) or repulsive (away from neighbors). The
transition point between attraction and repulsion is therefore given by
$\eta$, the point at which prediction errors switch sign.

An important consequence of this formulation is that, unlike the action rule
used in social force-based models, the ‘steady-state’ solution occurs when all
social forces disappear (when prediction errors vanish). In this case, the
agent ceases to change its heading direction and adopts its previous velocity.
This occurs when the agent’s sensations align with its (biased) predictions
$y_{h,1}\approx\eta$. In classic SPP models, this is equivalent to the
different social force vectors exactly cancelling each other.

We can therefore interpret social force-based models as limiting cases of
distance-inferring active inference agents, because one can conceive of social
forces as just those forces induced by free energy gradients; namely, the
forces that drive belief-updating. In the case of our active inference agents,
attractive and repulsive forces emerge naturally when we assume A) agents
model the local distance dynamics as an attractor with some positive-valued
fixed point $\eta$; B) agents can act by changing their heading direction and
C) agents observe at least the first time derivative of their observations
(e.g., $y^{\prime}_{h,1}$, but see SI Appendix, Section A for detailed
derivations).

It is worth highlighting the absence of an explicit, vectorial alignment force
in this model, consistent with experimental findings in two species of fish
[12, 17]. The heading vectors of neighbors is nevertheless implicitly
incorporated into the calculation of first-order prediction errors
$\xi^{\prime}_{z}$ via the first order hidden state $x^{\prime}_{h,1}$ (c.f.,
(Social forces as a consequence of predictive control) and SI Appendix,
Eq.(A.40)). In particular, the $x^{\prime}_{h,1}$ (from which the observations
$y^{\prime}_{h,1}$ are sampled) is equivalent to the ‘relative velocity’ term
used in so-called selective attraction and repulsion models, where the
instantaneous rate at which neighbours approach or move away, is used to drive
movement [101]. However, explicit alignment forces as seen in the Vicsek model
[100] and 2- and 3-Zone Couzin models [97, 48] can also be recovered if we
assume agents have a generative model of the average angle between their
heading vector and those of their neighbors (see SI Appendix, Section B for
derivations of alignment forces).

## Multivariate sensorimotor control

Having recovered social forces as free energy gradients in the case of a
single sensory sector ($L=1$), we now revisit the general formulation of the
generative model’s state-space, where the hidden variable $x$ is treated as an
$L$-dimensional vector state: $\mathbf{x}=(x_{1},x_{2},...,x_{L})$, with
correspondingly $L$-dimensional observations
$\mathbf{y}=(y_{1},y_{2},...,y_{L})$.

Specifically, we consider each $x_{l}$ to represent the average distance-to-
neighbors within one of a subset of adjacent sensory sectors, where each
sector is offset from the next by a fixed inter-sector angle (see Figure 1B
for a schematic of the multi-sector set-up). The rest of the generative model
is identical; the agents estimate these distances (and their temporal
derivatives $x_{l}^{\prime},x_{l}^{\prime\prime},...$) while changing their
heading direction to minimize free energy. Following the same steps as in the
case of a single sector, the resulting update rule for $\mathbf{v}$ is a
weighted sum of ‘sector-vectors’, where generalized observations from each
sector-specific modality $\tilde{y}_{l}$ are used to compute the prediction
errors that scale the corresponding sector-vector. This generalizes the
scalar-vector product in (5) to a matrix-vector product:

$\displaystyle\frac{d\mathbf{v}}{dt}$
$\displaystyle=\tilde{\boldsymbol{\xi}}_{z}^{\top}\Delta\hat{\mathbf{R}}$
$\displaystyle\Delta\hat{\mathbf{R}}$
$\displaystyle=-\begin{bmatrix}\nabla_{\mathbf{v}}\tilde{y}_{1}\\\
\nabla_{\mathbf{v}}\tilde{y}_{2}\\\ \vdots\\\
\nabla_{\mathbf{v}}\tilde{y}_{L}\end{bmatrix}$ (8)

where now the (negative) sensorimotor contingency
$-\nabla_{a}\tilde{\mathbf{y}}=\Delta\hat{\mathbf{R}}$ is a matrix whose rows
contain the partial derivatives $\nabla_{\mathbf{v}}\tilde{y}_{l}$ (i.e. the
‘sector-vectors’). Each sector-vector is a vector pointing towards the average
neighbor position within sector $l$.

Figure 2: A: Example snapshots of different collective states in schools of
$N=50$ active inference agents. Each line represents the trajectory of one
individual, and color gradient represents time, from earliest (light blue) to
latest (purple). The polarized regime in the left panel was simulated with the
default parameters listed in supplementary Table E1. The milling regime
(middle panel) was achieved by increasing the variance of velocity
fluctuations (encoded in $\sigma^{2}_{z^{\prime},h}$) from $0.01$ to $0.05$
(relative to the default configuration) and increasing $\lambda_{z}$ from
$1.0$ to $1.2$. The disordered regime was achieved by increasing the sensory
smoothness parameter to $2.0$ and decreasing $\eta$ from $1.0$ to $0.5$ and
$\alpha$ from $0.5$ to $0.1$ (relative to the default configuration). B:
Average polarization (left) and milling probability (right) shown as a
function of the two factorized components of the sensory precision,
$\Gamma_{z}$ (log-transformed) and $\lambda_{z}$. For each combination of
precision parameters, we ran $500$ independent trials of ‘free schooling,’ and
then averaged the quantities of interest across trials. Each ‘free schooling’
trial lasted $15$ seconds ($1500$ time steps with $dt=0.01s$); the time-
averaged metrics (polarization and milling probability, respectively, were
computed from the last $10$ seconds of the trial.

## Numerical results

Given a group of active inference agents — equipped with the generative models
described in previous sections — it is straightforward to generate
trajectories of collective motion by integrating each agent’s heading vector
over time: $\dot{\mathbf{r}}_{i}=\mathbf{v}_{i},i\in\\{1,2,...,N\\}$ where $N$
is the number of agents. We update all heading directions
$\\{\mathbf{v}_{i}\\}_{i=1}^{N}$ and beliefs
$\\{\tilde{\boldsymbol{\mu}}_{i}\\}_{i=1}^{N}$ in parallel via a joint
gradient descent on their respective free energies:

$\displaystyle\dot{\mathbf{v}}_{1}$
$\displaystyle=-\nabla_{\mathbf{v}_{1}}F(\tilde{\boldsymbol{\mu}}_{1},\tilde{\mathbf{y}}_{1})$
$\displaystyle\dot{\tilde{\boldsymbol{\mu}}}_{1}$
$\displaystyle=-\nabla_{\tilde{\boldsymbol{\mu}}_{1}}F(\tilde{\boldsymbol{\mu}}_{1},\tilde{\mathbf{y}}_{1})$
$\displaystyle\dot{\mathbf{v}}_{2}$
$\displaystyle=-\nabla_{\mathbf{v}_{2}}F(\tilde{\boldsymbol{\mu}}_{2},\tilde{\mathbf{y}}_{2})$
$\displaystyle\dot{\tilde{\boldsymbol{\mu}}}_{2}$
$\displaystyle=-\nabla_{\tilde{\boldsymbol{\mu}}_{2}}F(\tilde{\boldsymbol{\mu}}_{2},\tilde{\mathbf{y}}_{2})$
$\displaystyle\vdots$ $\displaystyle\vdots$
$\displaystyle\dot{\mathbf{v}}_{N}$
$\displaystyle=-\nabla_{\mathbf{v}_{N}}F(\tilde{\boldsymbol{\mu}}_{N},\tilde{\mathbf{y}}_{N})$
$\displaystyle\dot{\tilde{\boldsymbol{\mu}}}_{N}$
$\displaystyle=-\nabla_{\tilde{\boldsymbol{\mu}}_{N}}F(\tilde{\boldsymbol{\mu}}_{N},\tilde{\mathbf{y}}_{N})$
(9)

For the simulation results shown here, each agent tracks the average distance
$x_{l}$ within a total of $L=4$ sensory sectors that each subtend $60^{\circ}$
(starting at $-120^{\circ}$ and ending at $+120^{\circ}$, relative to the
focal agent’s heading direction) and observe the sector-specific distances
calculated using all neighbors lying within $5.0$ units of the focal agent’s
position. Each agent represents the vector of local distances as a generalized
state with 3 orders of motion:
$\tilde{\mathbf{x}}=\\{\mathbf{x},\mathbf{x}^{\prime},\mathbf{x}^{\prime\prime}\\}$,
$\tilde{\boldsymbol{\mu}}=\\{\boldsymbol{\mu},\boldsymbol{\mu}^{\prime},\boldsymbol{\mu}^{\prime\prime}\\}$.
Agents can observe the first and second orders of the distance
$\tilde{\mathbf{y}}=\\{\mathbf{y},\mathbf{y}^{\prime}\\}$, i.e. the distance
itself and its instantaneous rate-of-change. In the numerical results to
follow, we use active inference to study the relationship between the
properties of individual cognition (e.g., the parameters of agent-level
generative models) and collective phenomenology.

### Collective regimes

Simulated groups of these distance-inferring agents display robust, cohesive
collective motion (see Figure 2A and Supplemental Movies S1-S5). Figure 2A
displays examples of different types of group phenomena exhibited in groups of
active inference agents, whose diversity and types resemble those observed in
animal groups [49, 50] and in other collective motion models [100, 97, 51].
These range from directed, coherent movement with strong inter-agent velocity
correlations (‘polarized motion’) to group rotational patterns, like milling,
which features high angular momentum around the group’s center-of-mass.

### Relating individual beliefs to collective outcomes

In all but the most carefully constructed systems [31, 52, 53], the
relationship between individual and collective representations is often
opaque. In particular, the relationship between individual level uncertainty
or ‘risk’ and collective behavior is an open area of research. For instance,
some research has indicated that increased risk-sensitivity at the level of
the individual may lead to decreased risk-encoding at the collective level
[54]. Inspired by these observations, we use active inference to examine the
quantitative relationship between uncertainty at the individual level and
collective phenomenology. We begin by examining common metrics of group motion
like polarization and angular momentum [97]. In Figure 2B we explore how
polarization and angular momentum are affected by two components of agent-
level sensory uncertainty (i.e., inverse sensory precision): 1) the absolute
precision that agents associate to sensory noise and 2) the autocorrelation or
‘smoothness’ associated to that noise.

These components are encoded in each agent’s observation model, which assumes
generalized distance observations $\tilde{\mathbf{y}}$ are normally-
distributed around the generalized state $\tilde{\mathbf{x}}$:

$\displaystyle
P(\tilde{\mathbf{y}}|\tilde{\mathbf{x}})=\mathcal{N}(\tilde{\mathbf{y}};\tilde{\mathbf{x}},\tilde{\Sigma}^{\boldsymbol{z}})$
(10)

Where we focus on the parameterization of the inverse of the covariance
matrix, a.k.a., the precision matrix
$\tilde{\Pi}^{\boldsymbol{z}}=\left(\tilde{\Sigma}^{\boldsymbol{z}}\right)^{-1}$.
This precision matrix factorizes into two sub-matrices, one encoding the
amplitude of random fluctuations $\mathbf{z}$ and one encoding their temporal
smoothness, i.e., the inverse of the covariance between different derivatives
of random fluctuations (e.g., between $z$ and $z^{\prime}$):

$\displaystyle\tilde{\Pi}^{\mathbf{z}}$
$\displaystyle=\Pi^{\mathbf{z}}\otimes\tilde{\Pi}^{z}$
$\displaystyle\text{where}\hskip 7.11317pt\Pi^{\mathbf{z}}$
$\displaystyle=\begin{bmatrix}\Gamma_{z,1}&0&\ldots&0\\\ 0&\Gamma_{z,2}&&\\\
\vdots&&\ddots\\\ 0&&&\Gamma_{z,L}\end{bmatrix}$ (11)
$\displaystyle\tilde{\Pi}^{z}$ $\displaystyle=\begin{bmatrix}1&0\\\
0&2\lambda_{z}^{2}\end{bmatrix}$ (12)

Intuitively, $\Gamma_{z}$ encodes the variance or amplitude that the agent
associates with the noise in each of its $L$ sensory sectors $z_{l}$, and
$\lambda_{z}$ encodes how ‘smooth’ the agent believes the noise is [40, 55]. A
higher value of $\lambda_{z}$ implies that the agent believes sensory noise is
more serially-correlated (e.g., random fluctuations in optical signals caused
by smooth variations in refraction due to turbulence in water). Section C.1 of
the SI Appendix shows how the smoothness parameter $\lambda_{z}$ can be
derived from a noise process with a Gaussian autocorrelation function. The
consequences of this parameterization can be mapped back to the first-order
prediction errors $\boldsymbol{\xi}^{\prime}_{z}$ that drive action in (5) and
(8):

$\displaystyle\boldsymbol{\xi}^{\prime}_{z}$
$\displaystyle=\begin{bmatrix}2\Gamma_{z,1}\lambda_{z}^{2}(y^{\prime}_{h,1}-\mu^{\prime}_{h,1})\\\
2\Gamma_{z,2}\lambda_{z}^{2}(y^{\prime}_{h,2}-\mu^{\prime}_{h,2})\\\ \vdots\\\
2\Gamma_{z,L}\lambda_{z}^{2}(y^{\prime}_{h,L}-\mu^{\prime}_{h,L})\end{bmatrix}$
(13)

Here, we have simply written the precision assigned to noise $z_{h,l}$ in a
particular sensory sector as a product of the amplitude and smoothness
parameters: $\pi^{\prime}_{z,l}=2\Gamma_{z,l}\lambda_{z}^{2}$.

Figure 2B shows how the different components (amplitude and smoothness) of the
agent’s beliefs about uncertainty determine group behavior, as quantified by
average polarization and milling probability. Average polarization is defined
here as the time average of the polarization of the group, where the
polarization at a given time $p(t)$ measures the alignment of velocities of
agents comprising the group [97, 56]:

$\hat{p}=\frac{1}{T-t_{0}}\sum_{t=t_{0}}^{T}p(t)\hskip
28.45274ptp(t)=\frac{1}{N}\|\sum_{i=1}^{N}\mathbf{v}_{i}(t)\|$ (14)

Note that the time average is calculated once steady-state has been reached,
where the beginning of this state is indicated by $t_{0}$ (for the heatmaps
shown in 2B, we calculate these average metrics with $t_{0}=5s$). High average
polarization indicates directed, coherent group movement. The left panel of
Figure 2B shows how $\Gamma_{z}$ and $\lambda_{z}$ contribute to the average
polarization of the group. An increase in either parameter causes polarization
to decrease and angular momentum to increase, reflecting the transition from
directed motion to a milling regime, where the group rotates around its center
of mass. We calculate the milling probability (c.f. right panel of Figure 2B)
as the proportion of trials where the time-averaged angular momentum surpassed
$0.5$. The average angular momentum can be used to quantify the degree of
rotational motion, and is calculated as the time- and group-average of the
individual angular momenta around the groups’ center of mass $\mathbf{c}$:

$\hat{m}=\frac{1}{T-t_{0}}\sum_{t=t_{0}}^{T}m(t)\hskip
28.45274ptm(t)=\frac{1}{N}\|\sum_{i=1}^{N}\mathbf{r}_{ic}(t)\times\mathbf{v}_{i}(t)\|$
(15)

where $\mathbf{r}_{ic}$ is a relative position vector for agent $i$, defined
as the vector pointing from the group center $\mathbf{c}$ to agent $i$’s
position: $\mathbf{r}_{i}-\mathbf{c}$. We observed a large range of
$\Gamma_{z}$ and $\lambda_{z}$ for which the milling regime (high average
angular momentum) was stable (Figure 2B, right side). This stands in contrast
to earlier self-propelled particular models like the original 3-zone Couzin
model, where milling was only stable under a relatively limited range of
parameters [97].

These collective changes can be understood by recalling how first-order
prediction errors $\boldsymbol{\xi}^{\prime}_{z}$ (and thus the velocity
update) depend on $\Gamma_{z}$ and $\lambda_{z}$:

$\displaystyle\boldsymbol{\xi}^{\prime}_{z}$ $\displaystyle\propto
2\Gamma_{z}\lambda_{z}^{2}$ (16)

In practice, this means that as the group believes in more predictable (less
rough) first-order sensory information $\mathbf{y}^{\prime}_{z}$, the group as
a whole is more likely to enter rotational, milling-like regimes. However, the
enhancing effect of these first-order prediction errors
$\boldsymbol{\xi}^{\prime}_{z}$ on rotational motion is bounded; if prediction
errors are over-weighted (e.g. high $\Gamma_{z}$ and/or $\lambda_{z}$), the
group becomes more polarized again and likely to fragment (see SI Appendix,
Fig. E1). This fragmentation probability occurs at both low and high levels of
$\Gamma_{z}$ and $\lambda_{z}$, implying that there is an optimal range of
individual-level sensory precision where cohesive group behavior (whether
polarized or milling) is stable. Thus, our model predicts that assuming one’s
sensory information is highly-precise is neither required, or in fact even
desirable, for animals in order to facilitate collective motion.

We have seen how one can use active inference to relate features of
individual-level beliefs (in this case, beliefs about sensory precision) to
collective patterns, focusing in the present case on common metrics for
studying collective motion like polarization and the tendency to mill.

In the following sections, we move from looking at group-level patterns that
occur during free movement, to studying the consequences of individual-level
uncertainty for collective information-processing. We begin by investigating
how collective information transfer depends on individual-level beliefs about
the relative precisions associated with different types of sensory
information.

Figure 3: A: Collective accuracy as a function of proportion informed or
$p_{inf}$ for differing values of the sensory precision assigned to social
observations $\Gamma_{z-\text{Social}}$. Average accuracy for each condition
(combination of $p_{inf},\Gamma_{z-\text{Social}},\Gamma_{z-\text{Target}}$)
was computed as the proportion of successful hits across $500$ trials. Here,
the average accuracy is further averaged across all the values of the
$\Gamma_{z-\text{Target}}$ parameter, meaning each accuracy here is computed
as the average of $15000$ total trials ($500$ trials per condition $\times$
$30$ different values of $\Gamma_{z-\text{Target}}$). B: Collective accuracy
as a function of both the social and target precisions
($\Gamma_{z-Social},\Gamma_{z-Target}$, shown in log-scale) averaged across
values of $p_{inf}$ ranging from $p_{inf}=0.15$ to $p_{inf}=0.40$. Each
condition’s accuracy was computed as the proportion of accurate decisions from
$500$ trials.

### Collective information transfer

In this section, we take inspiration from the collective leadership and
decision-making literature to investigate how individuals in animal groups can
collectively navigate to a distant target [48, 57, 58, 59]. This phenomenon is
an example of effective leadership through collective information transfer and
is remarkable for a number of reasons; one that speaks to its emergent nature,
is the fact that these collective decisions are possible despite — and indeed
even because of — the presence of uninformed individuals in the group [57].
Figure 3A shows that active inference agents engaged in this task reproduce a
result from earlier work [48] on the relationship between the proportion of
uninformed individuals and collective accuracy. Namely, as the proportion of
informed individuals increases, so does the accuracy of reaching the majority-
preferred target. In the same vein as earlier sections, we also investigated
the dependence of this effect, as well as the average target-reaching
accuracy, on individual-level beliefs.

We operationalize the notion of an agent being ‘informed’ (about an external
target) by introducing a new latent variable to its generative model; this
variable $x_{\text{target}}$ represents the distance between the informed
agent’s position $\mathbf{r}$ and a point-mass-like target with position
vector $\mathbf{T}=[T_{1},T_{2}]$. We thus define this new hidden state and
observation as follows: $x_{\text{target}}=\|\mathbf{T}-\mathbf{r}\|$,
$y_{\text{target}}=x_{\text{target}}+z_{\text{target}}$. Just like the
‘social’ distance observations $\mathbf{y}_{h}$, this target distance
observation $y_{\text{target}}$ represents a (potentially-noisy) observation
of the true distance $x_{\text{target}}$. As before, the agent represents both
the target distance $x_{\text{target}}$ and its observations
$y_{\text{target}}$ using generalized coordinates of motion. Each informed
agent has a dynamics model of $\tilde{x}_{\text{target}}$, whereby they assume
the target-distance is driven by some drift function
$f_{\text{target}}(x_{\text{target}})=-\alpha_{t}x_{\text{target}}$ which
relaxes to $0$. As with the social distances, we truncate the agent’s
generalized coordinates embedding of the target distance to three orders of
motion and the generalized observations to two orders of motion.

Each informed agent maintains a full posterior belief
$\tilde{\boldsymbol{\mu}}=(\tilde{\mu}_{1},\tilde{\mu}_{2},...,\tilde{\mu}_{L},\tilde{\mu}_{\text{target}})$
about the local distances $\tilde{x}_{1},\tilde{x}_{2},...,\tilde{x}_{L}$ as
well as the target distance $\tilde{x}_{\text{target}}$.

Using identical reasoning to arrive at the action updates in (5) and (8), one
can augment the matrix-vector product in (8) with an extra sensorimotor
contingency and prediction error that represents target-relevant information:

$\displaystyle\frac{d\mathbf{v}}{dt}$
$\displaystyle=\tilde{\boldsymbol{\xi}}_{z}^{\top}\begin{bmatrix}\Delta\hat{\mathbf{R}}\\\
\Delta\mathbf{T}\end{bmatrix}$ $\displaystyle\Delta\mathbf{T}$
$\displaystyle=-\nabla_{\mathbf{v}}\tilde{y}_{\text{target}}=\frac{\mathbf{T}-\mathbf{r}}{\|\mathbf{T}-\mathbf{r}\|}$
(17)

This matrix-vector product can then be seen as a weighted combination of
social and target vectors, with the weights afforded to each equal to their
respective precision-weighted prediction errors:

$\displaystyle\frac{d\mathbf{v}}{dt}$
$\displaystyle=\underbrace{\xi_{\text{social}}\Delta\hat{\mathbf{R}}}_{\text{Social
vector}}+\underbrace{\xi_{\text{target}}\Delta{\mathbf{T}}}_{\text{Target
vector}}$ (18)

This expression is analogous to the velocity update in Equation (3) of Ref.
[48], where a ‘preferred direction’ vector is integrated into the agent’s
action update with some pre-determined weight. This weight is described as
controlling the relative strengths of non-social vs. social information. For
active inference agents, the weighting of target-relevant information emerges
naturally as a precision-weighted prediction error (here represented as
$\xi_{\text{target}}$), and the target-vector itself is equivalent to a
sensorimotor reflex arc, that represents the agent’s assumptions about how the
local flow of the target distance $y^{\prime}_{\text{target}}$ changes as a
function of the agent’s heading direction $\mathbf{v}$. An important
consequence of this construction, is that, unlike in previous models where
this weight is ‘baked-in’ as a fixed parameter, the weight assigned to the
target vector is dynamic, and fluctuates according to how much the agent’s
expectations about the target distance $\tilde{\mu}_{\text{target}}$ predict
the sensed target distance $y_{\text{target}}$.

Using this new construction, we can simulate a group of active inference
agents, in which some proportion $p_{inf}$ of agents represent this extra set
of target-related variables as described above. To generate
$\tilde{y}_{\text{target}}$ observations for these informed individuals, we
placed a spatial target at a fixed distance away from the group’s center-of-
mass and then allowed the informed individuals to observe the generalized
target distance
$\tilde{y}_{\text{target}}=(y_{\text{target}},y^{\prime}_{\text{target}})$. We
then integrated the collective dynamics over time and measured the accuracy
with which the group was able to navigate to the target (see Materials and
Methods for details). By performing hundreds of these trials for different
values of $p_{inf}$, we reproduced the results of Ref. [48] in Figure 3. We
see that as the number of informed individuals increases, collective accuracy
increases. However, this performance gain depends on the agents‘ beliefs about
sensory precision, which we now dissociate into two components:
$\Gamma_{z\text{-Social}}$ ( the precision assigned to the social distance
observations) and $\Gamma_{z\text{-Target}}$ (the precision assigned to target
distance observations). By varying these two precisions independently, which
respectively scale $\xi_{\text{social}}$ and $\xi_{\text{target}}$ in (18), we
can investigate the dependence of collective accuracy on the beliefs of
individual agents about the uncertainty attributed to different sources of
information.

Figure 3A shows the average collective accuracy as a function of $p_{inf}$,
for different levels of the social distance precision
$\Gamma_{z\text{Social}}$. The pattern that emerges is that the social
precision, that optimizes collective decision-making, sits within a bounded
range. The general effect of social precision is to essentially balance the
amplification of target-relevant information throughout the school, with the
need for the group to maintain cohesion. When social precision is too high,
agents over-attend to social information and are not sensitive to the
information provided by informed individuals; when it is too low, the group is
likely to fragment and will not accurately share target-relevant information;
meaning only the informed individuals will successfully reach the target.
Figure 3B shows that a similar optimal precision-balance exists for
$\Gamma_{z\text{Target}}$. Here, we show average collective accuracy (averaged
across values of $p_{inf}$ as a function of social- and target-precision.
Maximizing collective accuracy appears to rely on agents balancing the sensory
precision they assign to different sources of information; under the active
inference model proposed here, this balancing act can be exactly formulated in
terms of the variances (inverse precisions) afforded to different types of
sensory cues.

### Online plasticity through parameter learning

The ability of groups to tune their response to changing environmental
contexts, such as rapid perturbations or informational changes, is a key
feature of natural collective behavior [54, 15]. However, many self-propelled
particle models lack a generic way to incorporate this behavioral sensitivity
[48] and exhibit damped, ‘averaging’-like responses to external inputs [60].
This results from classical models usually equipping individuals with fixed
interaction rules and constant weights for integrating different information
sources. While online weight-updating rules and evolutionary algorithms have
been used to adaptively tune single-agent parameters in some cases [48, 59,
61], these approaches are often not theoretically principled and driven by
specific use-cases [with notable exceptions [62, 63, 64]].

Active inference offers an account of tune-able sensitivity, using the same
principle used to derive action and belief-updating in previous sections:
minimizing surprise. In practice, this sensitivity emerges when we allow
agents to update their generative models per se in real-time. Updating
generative model parameters over time is often referred to as “learning” in
the active inference literature [102], since it invokes the notion of updating
beliefs about parameters rather than states, where parameters and states
distinguish themselves by fast and slow timescales of updating, respectively.
We leverage this idea to allow agents to adapt their generative models and
thus adapt their behavioral rules, referring to this process as plasticity,
in-line with the notion of short-term plasticity in neural circuits [66]. To
enable agents to update generative model parameters, we can simply augment the
coupled gradient descent in (Numerical results) with an additional dynamical
equation, this time by minimizing free energy with respect to model
parameters, which we subsume into a set $\theta$:

$\displaystyle\dot{\theta}$
$\displaystyle=-\nabla_{\theta}F(\tilde{\boldsymbol{\mu}},\tilde{\boldsymbol{y}},\theta)$
(19)

The generative model parameters $\theta$ represent the statistical
contingencies or regularities agents believes govern their sensory world; this
includes the various precisions associated with sensory and process noises
$\tilde{\Pi}^{z},\tilde{\Pi}^{\omega}$ and the parameters of the dynamics and
observation models, $\tilde{\mathbf{f}},\tilde{\mathbf{g}}$. Since the free
energy is a smooth function of all the generative model parameters, in theory
learning can be done with respect to any parameter using procedure entailed by
(19).

Figure 4: A: Schematic of the sensory perturbation protocol. The ‘pseudo-
motion’ stimulus consists of repetitively perturbing the agent’s sensory
sectors with a moving wave of prediction errors in the agent’s velocity-
observation modality $\mathbf{y}^{\prime}_{h}$. The top panel shows stimulus
pattern as a heatmap over (amplitude over time) with two repetitions, starting
from negative (red, sectors $1$ and $2$) and transitioning to positive (blue,
sectors $3$ and $4$) prediction errors. The sign-switch in the stimulus (from
negative to positive) mimics a moving object that first moves towards focal
individual and then moves away. The temporal order of the stimulus across the
sectors can be used to selectively emulate a right-moving vs. left-moving
object, relative to the focal individual’s heading-direction. The bottom panel
shows how the stimulated agent’s beliefs about the distance hidden state
$\boldsymbol{\mu}$ changes over the course of the motion stimulus, with these
beliefs being analogized to hypothetical neural activity. B: Response
magnitude to a perturbation in presence or absence of parameter learning. Left
panel: example pair of 2-D trajectories of active inference agents with
matched pre-perturbation histories, in response to an individual perturbation.
The ability to perform parameter-learning is left on in one stochastic
realization (green) and turned off in the other (blue), following the
perturbation. Right panel: initialization-averaged collective responses (group
turning angle) to perturbation of active inference agents when learning is
enabled or disabled. The perturbation response of a 2-zone self-propelled
particle model (purple line) based on [48] is also shown for reference. C:
Collective response as a function of the number of perturbed individuals,
comparing simulations where parameter-learning is enabled to those where it’s
disabled. Shown is the mean response with highest density regions (HDRs) of
integrated turning magnitude within 500-1000 ms of the perturbation (left) and
response probability (right) computed from $N_{i}=200$ independent
initializations of each condition. For each initialization, the average metric
is computed across $N_{r}=50$ independent realizations that were run forward
from the same point in time, following a sensory prediction error perturbation
(to a randomly-chosen set of perturbed agents). Response probability is
computed as the proportion of independent realizations, per initialization,
where the group turning rate exceeded $\pi$ radians within the first 10
seconds of the perturbation.

In practice, combining parameter-learning with active inference usually
implies a separation of timescales, whereby learning or plasticity occurs
concurrently to state inference and action but at a slower update rate. In all
the results shown here, agents update parameters an order of magnitude more
slowly than they update beliefs or actions. To furnish a interpretable example
of plasticity, in the simulations described here, we enabled agents to update
their beliefs about the sensory smoothness parameter $\lambda_{z}$. We chose
sensory smoothness due to its straightforward relationship to the magnitude of
sensory prediction errors (c.f. the relation in (16) and SI Appendix, Section
C). As agents tune $\lambda_{z}$ to minimize free energy, belief updating and
action will at the same time become quadratically more or less responsive to
sensory information.

One example of where behavioral plasticity is crucial for collective
information processing is a group’s ability to rapidly amplify behaviorally-
relevant information, e.g., detecting the presence of a predator [67, 68, 69].
To study the effect of behavioral plasticity on collective responsiveness, we
perturbed single agents in groups of active inference agents while enabling or
disabling online plasticity. We perturbed groups by inducing transient
‘phantom’ prediction errors in random subsets of agents and measuring the
resulting turning response of the group (see Materials and Methods for
details). These prediction errors were structured (see Figure 4A) to mimic a
transient visual stimulus, e.g., a loom stimulus or approaching predator [70],
which reliably induces a sustained turning response in the chosen individual
[60]. Figure 4 shows the effect of enabling plasticity on the size and
sensitivity of collective responses to these perturbations. Not only do
plasticity-enabled groups respond more strongly to perturbations of single-
agents, compared to their plasticity-disabled counterparts (4B), but the
magnitude of the collective response is also more sensitive to the size of the
perturbation (4C). As has been measured in biological collectives [71], the
plasticity-enabled groups collectively encode the size of perturbations with
higher dynamic range than plasticity-disabled controls.

The active inference framework provides a flexible and theoretically-
principled approach to modeling adaptive, collective behavior with tuneable
sensitivity, that eschews ad-hoc update rules or expensive evolutionary
simulations. The plasticity mechanism proposed here is not limited to updating
beliefs about sensory smoothness: it can be extended to update beliefs about
any model parameter using the same principle. The ability to adapt generative
model parameters in real-time represents a promising avenue for future
research in active inference and collective behavior, and may lead to more
biologically-plausible hypotheses about the mechanisms underlying adaptive
responses in the natural world.

## Discussion

We have proposed active inference as a flexible, cognitively-inspired model
class that can be used in the theoretical study of collective motion, as well
as in empirical settings as an individual-level model of behavior. By framing
behavior as the consequence of prediction-error minimization — with respect to
an individual’s world model — we offer examples of how naturalistic collective
motion emerges in, where individual behavior is driven by the imperative to
minimize the surprisal associated with sensory signals. Under mild
distributional assumptions, this surprise is scored by an interpretable proxy;
namely, prediction error. In the particular case of collective motion, a group
of active inference agents equipped with a simple generative model of local
social information can recover and generalize the social forces that have been
the core mechanism in classical SPP models of collective motion. The active
inference framework also provides a probabilistic interpretation of ad-hoc
‘weight’ parameters that are often used in these models, in terms of the
precisions that agents associate to different types of sensory information.

We have also shown how the active inference framework can be used to
characterize the relationship between generative model parameters and emergent
information-processing capacities, as measured by collective information
transfer and responsiveness to external perturbations. Active inference’s
generality allows us to relax the typically-static behavioral rules of SPP
models, by enabling agents to flexibly tune their sensitivity to prediction
errors. This is achieved via principled processes like parameter learning
(i.e., ‘plasticity’), and can be used to model naturalistic features of
collective behavior, such as the tendency to amplify salient (i.e., precise)
information, that have largely evaded modelling in the SPP paradigm, except in
cases where adaptation rules are explicitly introduced [48, 59]. However, when
we simply allow agents to update parameters, in addition to beliefs and
agents, using the principle of surprise-minimization, many hallmarks of these
naturalistic behaviors can be easily obtained.

The surprise minimization approach adopted here is both theoretically grounded
in fundamental physical, cybernetic and informational principles [72, 23, 73,
74] while also biologically-inspired, due to the scalability of the belief and
action update rules, which are hypothesized to be implementable on neuronal
circuits [43]. Our approach thus also harmonizes with modern ‘data-driven’
approaches in behavioral biology, that aim to quantitatively estimate the
behavioral algorithms used by different biological systems directly from
experimental data [13, 14, 15].

By providing a flexible modeling approach that casts perception, action, and
learning as manifestations of the single drive to minimize surprise, we have
highlighted active inference as a toolbox for studying collective behavior in
natural systems. Future work in this area could explore how the framework can
be used to investigate other forms of collective behavior (not just collective
motion), like multi-choice decision-making, social foraging and communication
[75, 76]. The results shown in the current work serve primarily as a proof of
concept: we started by writing down a specific, hypothetical active inference
model of agents engaged in group movement, and then generated naturalistic
behaviors by integrating the resulting equations of motion (i.e., free energy
gradients) for this particular model. Taking inspiration from fields like
computational psychiatry [77, 78], we emphasize the ability to move from
simple forward modelling of behavior to data-driven model inversion, whereby
one hopes to infer the values of parameters that best explain empirical data
(of e.g., behavioral movement data). Instead of using ‘force mapping’
techniques to estimate social forces from behavioral measurements [79, 80],
our approach would instead frame the problem as one of computational
phenotyping, where alternative generative models that a particular animal
might be equipped with, could be estimated from behavioral or neural data
acquired from that animal. The resulting ‘social forces’ or interaction rules
would then emerge as those behaviors that minimize surprise, relative to the
generative model that best explains the animal’s behavior. Both the estimation
of model parameters and alternative model structures can be achieved through
Bayesian model inversion and system identification methods like Bayesian model
selection, averaging or reduction [81].

## Materials and Methods

For all simulations we randomly initialized the positions and (unit-magnitude)
velocities of $N$ particles, and integrated the equations of motion for active
inference and generalized filtering using a forwards Euler-Maruyama scheme
with an integration window of $\Delta t=0.01s$ (see SI Appendix, Section E for
details). We varied group size $N$ and the length of the simulation $T$ (in
seconds) depending on the experiment. Detailed background on generalized
filtering, active inference, and derivations specific to the generative model
we used for collective motion can be found in the SI Appendix, Section A. All
other parameters used for simulations, unless stated otherwise, are listed in
SI Appendix, Table E.1. The code (written in JAX and Julia) used to perform
simulations can be found in the following open-source repository:
https://github.com/conorheins/collective_motion_actinf [82].

### Quantifying fragmented groups

For all experiments, we excluded trials where the group failed to maintain
cohesion (or fragmented) to a sufficient degree. We deemed any given trial
fragmented, when at least one individual was further than 2.0 dimensionless
units away from all other individuals for at least 3 of the last 10 seconds of
the trial. For the perturbation experiments, groups were excluded if this
criterion was reached during the last 5 seconds of the 20 second post-
perturbation period.

### Collective information transfer experiments

For each trial of collective target-navigation, we initialized a group of
$N=30$ agents with random positions and velocities (centered on the origin)
and augmented the generative models of a fixed proportion $p_{inf}$ of the
total number of agents, where $p_{inf}$ ranged from $0.05$ to $1.0$, with
extra latent and observed variables representing the distance to the target
with position vector $\mathbf{T}$. The distance to the target was always $10$
units from the origin. We measured collective accuracy as follows: we count a
given trial as successful if the group is able to navigate to within $0.25$
units of the target without losing cohesion within $T=15$ seconds (the length
of each trial). The accuracy for a given experimental condition was then
computed as the proportion of successes observed in $500$ total trials.

### Perturbation experiments

For the perturbation experiments, we simulated $N_{i}=200$ randomly-
initialized independent runs of $N=50$ agents, which we term independent
initializations. We ran each initialization forward for $T=100$ seconds, a
point at which metrics like average polarization, angular momentum, and median
nearest-neighbor distance were highly likely to have stopped changing and
fluctuate around a stationary value. Starting at $T=100$ we then split each
initialization into two further sets of $N_{r}=50$ parallel realizations. Each
realization used a different random seed used to A) generate the action- and
observation-noises; and B) select the candidate agent(s) for perturbation.
Note that the splitting of seeds at $T=100$ means that each realization has an
identical history up until that point. We enabled parameter learning of
$\lambda_{z}$ in one set of realizations and we left it disabled in the other.
We then perturbed random subsets of agents in both learning-enabled and
-disabled realizations (2% - 50% of the group, i.e., 1 to 25 agents), by
transiently inducing first-order prediction errors
$\boldsymbol{\xi}^{\prime}_{z}$ in the perturbed individuals. We computed the
relative group turning angle after the perturbation for $20s$ to generate the
plots in Figure 4B and C.

##### Acknowledgements:

The authors would like to thank Brennan Klein, Jake Graving, Armin Bahl,
Dimitrije Markovic, Thomas Parr, Pawel Romanczuk, and Manuel Baltieri for
discussions during the writing of this manuscript, and Maya Polovitskaya for
creating the fish schematic used in the figures. CH and IDC acknowledge
support from the Office of Naval Research Grant N0001419-1-2556, Germany’s
Excellence Strategy-EXC 2117-422037984 (to IDC), the Max Planck Society, the
European Union’s Horizon 2020 research and Innovation Programme under the
Marie Skłodowska-Curie Grant agreement (to IDC; #860949), the PathFinder
European Innovation Council Work Programme (to IDC; #101098722), and the John
Templeton Foundation (to CH; #61780). LD is supported by the Fonds National de
la Recherche, Luxembourg (Project code: 13568875) and the Engineering and
Physical Sciences Research Council Centre for Doctoral Training in Mathematics
of Random Systems: Analysis, Modelling and Simulation (EP/S023925/1). RPM is
supported by UK Research and Innovation Future Leaders Fellowship MR/S032525/1
and the Templeton World Charity Foundation Inc. TWCF-2021-20647. KF is
supported by funding for the Wellcome Centre for Human Neuroimaging (Ref:
205103/Z/16/Z), a Canada-UK Artificial Intelligence Initiative (Ref:
ES/T01279X/1) and the European Union’s Horizon 2020 Framework Programme for
Research and Innovation under the Specific Grant Agreement No. 945539 (Human
Brain Project SGA3).

## References

  * [1] Peter F Major and Lawrence M Dill  “The three-dimensional structure of airborne bird flocks”  In _Behavioral Ecology and Sociobiology_ 4.2 Springer, 1978, pp. 111–122
  * [2] Scott Camazine et al.  “Self-organization in biological systems”  Princeton university press, 2003
  * [3] Michael Rubenstein, Christian Ahler and Radhika Nagpal  “Kilobot: A low cost scalable robot system for collective behaviors”  In _2012 IEEE International Conference on Robotics and Automation_ , 2012, pp. 3293–3298  IEEE
  * [4] Ichiro AOKI  “A Simulation Study on the Schooling Mechanism in Fish”  In _NIPPON SUISAN GAKKAISHI_ 48.8, 1982, pp. 1081–1088  DOI: 10.2331/suisan.48.1081
  * [5] Craig W Reynolds  “Flocks, herds and schools: A distributed behavioral model”  In _Proceedings of the 14th annual conference on Computer graphics and interactive techniques_ , 1987, pp. 25–34
  * [6] Tamás Vicsek et al.  “Novel type of phase transition in a system of self-driven particles”  In _Physical review letters_ 75.6 APS, 1995, pp. 1226
  * [7] Iain D Couzin et al.  “Collective memory and spatial sorting in animal groups”  In _Journal of theoretical biology_ 218.1 London, New York, Academic Press., 2002, pp. 1–12
  * [8] John Toner and Yuhai Tu  “Flocks, herds, and schools: A quantitative theory of flocking”  In _Physical review E_ 58.4 APS, 1998, pp. 4828
  * [9] David JT Sumpter  “The principles of collective animal behaviour”  In _Philosophical transactions of the royal society B: Biological Sciences_ 361.1465 The Royal Society London, 2006, pp. 5–22
  * [10] Eric Bertin, Michel Droz and Guillaume Grégoire  “Boltzmann and hydrodynamic description for self-propelled particles”  In _Physical Review E_ 74.2 APS, 2006, pp. 022101
  * [11] Pierre Degond and Sébastien Motsch  “Continuum limit of self-driven particles with orientation interaction”  In _Mathematical Models and Methods in Applied Sciences_ 18.supp01 World Scientific, 2008, pp. 1193–1215
  * [12] James E Herbert-Read et al.  “Inferring the rules of interaction of shoaling fish”  In _Proceedings of the National Academy of Sciences_ 108.46 National Acad Sciences, 2011, pp. 18726–18731
  * [13] Daniel S Calovi et al.  “Swarming, schooling, milling: phase diagram of a data-driven fish school model”  In _New journal of Physics_ 16.1 IOP Publishing, 2014, pp. 015026
  * [14] Andrew M Hein et al.  “An algorithmic approach to natural behavior”  In _Current Biology_ 30.11 Elsevier, 2020, pp. R663–R675
  * [15] Ashkaan K Fahimipour et al.  “Wild animals suppress the spread of socially transmitted misinformation”  In _Proceedings of the National Academy of Sciences_ 120.14 National Acad Sciences, 2023, pp. e2215428120
  * [16] Jacques Gautrais et al.  “Deciphering interactions in moving animal groups”  In _PLoS computational biology_ 8.9 Public Library of Science San Francisco, USA, 2012, pp. e1002678
  * [17] Yael Katz et al.  “Inferring the structure and dynamics of interactions in schooling fish”  In _Proceedings of the National Academy of Sciences_ 108.46 National Acad Sciences, 2011, pp. 18720–18725
  * [18] Karl J Friston, Jean Daunizeau and Stefan J Kiebel  “Reinforcement learning or active inference?”  In _PloS one_ 4.7 Public Library of Science, 2009, pp. e6421
  * [19] Karl Friston et al.  “Active inference: a process theory”  In _Neural computation_ 29.1 MIT Press, 2017, pp. 1–49
  * [20] Thomas Parr, Giovanni Pezzulo and Karl J Friston  “Active inference: the free energy principle in mind, brain, and behavior”  MIT Press, 2022
  * [21] Lancelot Da Costa et al.  “Active inference on discrete state-spaces: a synthesis”  In _Journal of Mathematical Psychology_ 99 Elsevier, 2020, pp. 102447
  * [22] Karl Friston  “A theory of cortical responses”  In _Philosophical transactions of the Royal Society B: Biological sciences_ 360.1456 The Royal Society London, 2005, pp. 815–836
  * [23] Karl Friston, James Kilner and Lee Harrison  “A free energy principle for the brain”  In _Journal of Physiology-Paris_ 100.1-3 Elsevier, 2006, pp. 70–87
  * [24] Karl Friston  “What is optimal about motor control?”  In _Neuron_ 72.3 Elsevier, 2011, pp. 488–498
  * [25] Karl Friston et al.  “The Free Energy Principle Made Simpler but Not Too Simple”  In _Physics Reports_ 1024, The Free Energy Principle Made Simpler but Not Too Simple, 2023, pp. 1–29  DOI: 10.1016/j.physrep.2023.07.001
  * [26] Rajesh PN Rao and Dana H Ballard  “Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects”  In _Nature neuroscience_ 2.1 Nature Publishing Group, 1999, pp. 79–87
  * [27] Rick A Adams, Stewart Shipp and Karl J Friston  “Predictions not commands: active inference in the motor system”  In _Brain Structure and Function_ 218.3 Springer, 2013, pp. 611–643
  * [28] Abdullahi Ali et al.  “Predictive coding is a consequence of energy efficiency in recurrent neural networks”  In _Patterns_ 3.12 Elsevier, 2022
  * [29] Charlotte Caucheteux, Alexandre Gramfort and Jean-Rémi King  “Evidence of a predictive coding hierarchy in the human brain listening to speech”  In _Nature human behaviour_ 7.3 Nature Publishing Group UK London, 2023, pp. 430–441
  * [30] Kevin N Laland  “Social learning strategies”  In _Animal Learning & Behavior_ 32.1 Springer, 2004, pp. 4–14
  * [31] Peter M Krafft, Erez Shmueli, Thomas L Griffiths and Joshua B Tenenbaum  “Bayesian collective learning emerges from heuristic social learning”  In _Cognition_ 212 Elsevier, 2021, pp. 104469
  * [32] Manuel Baltieri and Christopher L Buckley  “Generative models as parsimonious descriptions of sensorimotor loops”  In _arXiv preprint arXiv:1904.12937_ , 2019
  * [33] Cem Uran et al.  “Predictive coding of natural images by V1 firing rates and rhythmic synchronization”  In _Neuron_ 110.7, 2022, pp. 1240–1257.e8  DOI: https://doi.org/10.1016/j.neuron.2022.01.002
  * [34] Karl Friston  “The free-energy principle: a rough guide to the brain?”  In _Trends in cognitive sciences_ 13.7 Elsevier, 2009, pp. 293–301
  * [35] Jakob Hohwy  “The self-evidencing brain”  In _Noûs_ 50.2 Wiley Online Library, 2016, pp. 259–285
  * [36] Karl Friston  “A free energy principle for a particular physics”  In _arXiv preprint arXiv:1906.10184_ , 2019
  * [37] Bertrand Collignon, Axel Séguret and José Halloy  “A stochastic vision-based model inspired by zebrafish collective behaviour in heterogeneous environments”  In _Royal Society open science_ 3.1 The Royal Society Publishing, 2016, pp. 150473
  * [38] Renaud Bastien and Pawel Romanczuk  “A model of collective behavior based purely on vision”  In _Science advances_ 6.6 American Association for the Advancement of Science, 2020, pp. eaay0792
  * [39] Karl Friston, Klaas Stephan, Baojuan Li and Jean Daunizeau  “Generalised filtering”  In _Mathematical Problems in Engineering_ 2010 Hindawi, 2010
  * [40] Karl Friston et al.  “Variational free energy and the Laplace approximation”  In _Neuroimage_ 34.1 Elsevier, 2007, pp. 220–234
  * [41] David JC MacKay and David JC Mac Kay  “Information theory, inference and learning algorithms”  Cambridge university press, 2003
  * [42] Janneke FM Jehee, Constantin Rothkopf, Jeffrey M Beck and Dana H Ballard  “Learning receptive fields using predictive feedback”  In _Journal of Physiology-Paris_ 100.1-3 Elsevier, 2006, pp. 125–132
  * [43] Michael W Spratling  “A review of predictive coding algorithms”  In _Brain and cognition_ 112 Elsevier, 2017, pp. 92–97
  * [44] Antonella Maselli, Pablo Lanillos and Giovanni Pezzulo  “Active inference unifies intentional and conflict-resolution imperatives of motor control”  In _PLoS computational biology_ 18.6 Public Library of Science San Francisco, CA USA, 2022, pp. e1010095
  * [45] Manuel Baltieri and Christopher L Buckley  “PID control as a process of active inference with linear generative models”  In _Entropy_ 21.3 Multidisciplinary Digital Publishing Institute, 2019, pp. 257
  * [46] Christopher L Buckley, Chang Sub Kim, Simon McGregor and Anil K Seth  “The free energy principle for action and perception: A mathematical review”  In _Journal of Mathematical Psychology_ 81 Elsevier, 2017, pp. 55–79
  * [47] Pawel Romanczuk, Iain D Couzin and Lutz Schimansky-Geier  “Collective motion due to individual escape and pursuit response”  In _Physical Review Letters_ 102.1 APS, 2009, pp. 010602
  * [48] Iain D Couzin, Jens Krause, Nigel R Franks and Simon A Levin  “Effective leadership and decision-making in animal groups on the move”  In _Nature_ 433.7025 Nature Publishing Group, 2005, pp. 513–516
  * [49] Ch Becco, Nicolas Vandewalle, Johann Delcourt and Pascal Poncin  “Experimental evidences of a structural and dynamical transition in fish school”  In _Physica A: Statistical Mechanics and its Applications_ 367 Elsevier, 2006, pp. 487–493
  * [50] Kolbjørn Tunstrøm et al.  “Collective states, multistability and transitional behavior in schooling fish”  In _PLoS computational biology_ 9.2 Public Library of Science San Francisco, USA, 2013, pp. e1002915
  * [51] Irene Giardina  “Collective behavior in animal groups: theoretical models and empirical studies”  In _HFSP journal_ 2.4 Taylor & Francis, 2008, pp. 205–219
  * [52] Rafael Kaufmann, Pranav Gupta and Jacob Taylor  “An active inference model of collective intelligence”  In _Entropy_ 23.7 MDPI, 2021, pp. 830
  * [53] Conor Heins et al.  “Spin glass systems as collective active inference”  In _Active Inference: Third International Workshop, IWAI 2022, Grenoble, France, September 19, 2022, Revised Selected Papers_ , 2023, pp. 75–98  Springer
  * [54] Matthew MG Sosna et al.  “Individual and collective encoding of risk in animal groups”  In _Proceedings of the National Academy of Sciences_ 116.41 National Acad Sciences, 2019, pp. 20556–20561
  * [55] Thomas Parr, Jakub Limanowski, Vishal Rawji and Karl Friston  “The computational neurology of movement under active inference”  In _Brain_ , 2021
  * [56] Jerome Buhl et al.  “From disorder to order in marching locusts”  In _Science_ 312.5778 American Association for the Advancement of Science, 2006, pp. 1402–1406
  * [57] Iain D Couzin et al.  “Uninformed individuals promote democratic consensus in animal groups”  In _science_ 334.6062 American Association for the Advancement of Science, 2011, pp. 1578–1580
  * [58] Ariana Strandburg-Peshkin, Damien R Farine, Iain D Couzin and Margaret C Crofoot  “Shared decision-making drives collective movement in wild baboons”  In _Science_ 348.6241 American Association for the Advancement of Science, 2015, pp. 1358–1361
  * [59] Vivek H Sridhar et al.  “The geometry of decision-making in individuals and collectives”  In _Proceedings of the National Academy of Sciences_ 118.50 National Acad Sciences, 2021
  * [60] Allison Kolpas et al.  “How the spatial position of individuals affects their influence on swarms: a numerical comparison of two popular swarm dynamics models”  In _PloS one_ 8.3 Public Library of Science, 2013, pp. e58525
  * [61] Anastasia Bizyaeva, Alessio Franci and Naomi Ehrich Leonard  “Nonlinear opinion dynamics with tunable sensitivity”  In _IEEE Transactions on Automatic Control_ IEEE, 2022
  * [62] Heiko Hamann  “Evolution of collective behaviors by minimizing surprise”  In _Artificial Life Conference Proceedings_ , 2014, pp. 344–351  MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info …
  * [63] Tanja Katharina Kaiser and Heiko Hamann  “Innate Motivation for Robot Swarms by Minimizing Surprise: From Simple Simulations to Real-World Experiments”  In _IEEE Transactions on Robotics_ 38.6 IEEE, 2022, pp. 3582–3601
  * [64] Daniela Gandolfi et al.  “Emergence of associative learning in a neuromorphic inference network”  In _Journal of Neural Engineering_ 19.3 IOP Publishing, 2022, pp. 036022
  * [65] Karl Friston et al.  “Active inference and learning”  In _Neuroscience & Biobehavioral Reviews_ 68 Elsevier, 2016, pp. 862–879
  * [66] Matthias H Hennig  “Theoretical models of synaptic short term plasticity”  In _Frontiers in computational neuroscience_ 7 Frontiers Media SA, 2013, pp. 45
  * [67] Ashley JW Ward, James E Herbert-Read, David JT Sumpter and Jens Krause  “Fast and accurate decisions through collective vigilance in fish shoals”  In _Proceedings of the National Academy of Sciences_ 108.6 National Acad Sciences, 2011, pp. 2312–2315
  * [68] Ariana Strandburg-Peshkin et al.  “Visual sensory networks and effective information transfer in animal groups”  In _Current Biology_ 23.17 Elsevier, 2013, pp. R709–R711
  * [69] Jacob D Davidson et al.  “Collective detection based on visual information in animal groups”  In _Journal of the Royal Society Interface_ 18.180 The Royal Society, 2021, pp. 20210142
  * [70] Roy Harpaz, Minh Nguyet Nguyen, Armin Bahl and Florian Engert  “Precise visuomotor transformations underlying collective behavior in larval zebrafish”  In _Nature communications_ 12.1 Nature Publishing Group UK London, 2021, pp. 6578
  * [71] Luis Gómez-Nava et al.  “Fish shoals resemble a stochastic excitable system driven by environmental perturbations”  In _Nature Physics_ Nature Publishing Group UK London, 2023, pp. 1–7
  * [72] Roger C Conant and W Ross Ashby  “Every good regulator of a system must be a model of that system”  In _International journal of systems science_ 1.2 Taylor & Francis, 1970, pp. 89–97
  * [73] Alexander D Wissner-Gross and Cameron E Freer  “Causal entropic forces”  In _Physical review letters_ 110.16 APS, 2013, pp. 168702
  * [74] Hannes Hornischer, Stephan Herminghaus and Marco G Mazza  “Structural transition in the collective behavior of cognitive agents”  In _Scientific reports_ 9.1 Nature Publishing Group UK London, 2019, pp. 12477
  * [75] Daniel Ari Friedman et al.  “Active Inferants: an active inference framework for ant colony behavior”  In _Frontiers in behavioral neuroscience_ 15 Frontiers, 2021, pp. 647732
  * [76] Mahault Albarracin, Daphne Demekas, Maxwell JD Ramstead and Conor Heins  “Epistemic communities under active inference”  In _Entropy_ 24.4 MDPI, 2022, pp. 476
  * [77] P Read Montague, Raymond J Dolan, Karl J Friston and Peter Dayan  “Computational psychiatry”  In _Trends in cognitive sciences_ 16.1 Elsevier, 2012, pp. 72–80
  * [78] Ryan Smith, Paul Badcock and Karl J Friston  “Recent advances in the application of predictive coding and active inference models within clinical neuroscience”  In _Psychiatry and Clinical Neurosciences_ Wiley Online Library, 2020  URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/pcn.13138
  * [79] R Escobedo et al.  “A data-driven method for reconstructing and modelling social interactions in moving animal groups”  In _Philosophical Transactions of the Royal Society B_ 375.1807 The Royal Society, 2020, pp. 20190380
  * [80] Rajnesh K Mudaliar and Timothy M Schaerf  “An examination of force maps targeted at orientation interactions in moving groups”  In _Plos one_ 18.9 Public Library of Science San Francisco, CA USA, 2023, pp. e0286810
  * [81] William D Penny, Jérémie Mattout and N Trujillo-Barreto  “Bayesian model selection and averaging”  In _Statistical Parametric Mapping: The analysis of functional brain images. London: Elsevier_ , 2006
  * [82] Conor Heins  “Collective Motion ActInf” Deposited on July 27, 2023, 2023  URL: https://github.com/conorheins/collective_motion_actinf

## References

  * [83] Karl Friston, Klaas Stephan, Baojuan Li and Jean Daunizeau  “Generalised filtering”  In _Mathematical Problems in Engineering_ 2010 Hindawi, 2010
  * [84] Karl Friston  “Hierarchical models in the brain”  In _PLoS computational biology_ 4.11 Public Library of Science, 2008
  * [85] Bhashyam Balaji and Karl Friston  “Bayesian state estimation using generalized coordinates”  In _Signal processing, sensor fusion, and target recognition XX_ 8050 SPIE, 2011, pp. 716–727
  * [86] Karl J Friston  “Variational filtering”  In _NeuroImage_ 41.3 Elsevier, 2008, pp. 747–766
  * [87] Karl J Friston, N Trujillo-Barreto and Jean Daunizeau  “DEM: a variational treatment of dynamic systems”  In _Neuroimage_ 41.3 Elsevier, 2008, pp. 849–885
  * [88] Christopher L Buckley, Chang Sub Kim, Simon McGregor and Anil K Seth  “The free energy principle for action and perception: A mathematical review”  In _Journal of Mathematical Psychology_ 81 Elsevier, 2017, pp. 55–79
  * [89] Karl Friston et al.  “The free energy principle made simpler but not too simple”  In _Physics Reports_ 1024 Elsevier, 2023, pp. 1–29
  * [90] Karl Friston  “A theory of cortical responses”  In _Philosophical transactions of the Royal Society B: Biological sciences_ 360.1456 The Royal Society London, 2005, pp. 815–836
  * [91] Karl Friston and Stefan Kiebel  “Predictive coding under the free-energy principle”  In _Philosophical Transactions of the Royal Society B: Biological Sciences_ 364.1521 The Royal Society London, 2009, pp. 1211–1221
  * [92] Yanping Huang and Rajesh PN Rao  “Predictive coding”  In _Wiley Interdisciplinary Reviews: Cognitive Science_ 2.5 Wiley Online Library, 2011, pp. 580–593
  * [93] Rick A Adams, Stewart Shipp and Karl J Friston  “Predictions not commands: active inference in the motor system”  In _Brain Structure and Function_ 218.3 Springer, 2013, pp. 611–643
  * [94] Karl Friston  “What is optimal about motor control?”  In _Neuron_ 72.3 Elsevier, 2011, pp. 488–498
  * [95] Thomas Parr, Giovanni Pezzulo and Karl J Friston  “Active inference: the free energy principle in mind, brain, and behavior”  MIT Press, 2022
  * [96] Manuel Baltieri and Christopher L Buckley  “PID control as a process of active inference with linear generative models”  In _Entropy_ 21.3 Multidisciplinary Digital Publishing Institute, 2019, pp. 257
  * [97] Iain D Couzin et al.  “Collective memory and spatial sorting in animal groups”  In _Journal of theoretical biology_ 218.1 London, New York, Academic Press., 2002, pp. 1–12
  * [98] Ichiro AOKI  “A Simulation Study on the Schooling Mechanism in Fish”  In _NIPPON SUISAN GAKKAISHI_ 48.8, 1982, pp. 1081–1088  DOI: 10.2331/suisan.48.1081
  * [99] Craig W Reynolds  “Flocks, herds and schools: A distributed behavioral model”  In _Proceedings of the 14th annual conference on Computer graphics and interactive techniques_ , 1987, pp. 25–34
  * [100] Tamás Vicsek et al.  “Novel type of phase transition in a system of self-driven particles”  In _Physical review letters_ 75.6 APS, 1995, pp. 1226
  * [101] Pawel Romanczuk, Iain D Couzin and Lutz Schimansky-Geier  “Collective motion due to individual escape and pursuit response”  In _Physical Review Letters_ 102.1 APS, 2009, pp. 010602
  * [102] Karl Friston et al.  “Active inference and learning”  In _Neuroscience & Biobehavioral Reviews_ 68 Elsevier, 2016, pp. 862–879
  * [103] Lancelot Da Costa et al.  “Active Inference on Discrete State-Spaces: A Synthesis”  In _Journal of Mathematical Psychology_ 99, 2020, pp. 102447  DOI: 10.1016/j.jmp.2020.102447

##  Appendix A An active inference model of collective motion

Each agent within our model of collective motion maintains an internal model
of its local environment represented by average distances to its neighbours.
These distances are partitioned into $L$ sensory sectors
$\mathbf{x}={x_{1},x_{2},...,x_{L}}$, with each agent observing noisy versions
of these distances through a corresponding sensory channel
$\mathbf{y}={y_{1},y_{2},...,y_{L}}$. Each agent estimates the hidden distance
variable(s) $\mathbf{x}$ over time using its observed sensory states
$\mathbf{y}$. In practice, each agent implements this through a form of
variational Bayesian inference developed for continuous data-assimilation in
dynamic environments called generalized filtering, which can be seen as a
variational, more flexible version of Kalman filters. This dynamic inference
process entails updating posterior beliefs about $\mathbf{x}$ using a gradient
descent on variational free energy. In the case of Gaussian assumptions about
observation and state noise, these free energy gradients resemble a precision-
weighted average of sensory and state prediction errors. This comprises the
state-estimation component of active inference and is unpacked in detail in
Section A.1.

In addition to estimating the hidden distance variable with generalized
filtering, each agent also changes its heading direction $\mathbf{v}$ in order
to minimize the same variational free energy functional. When the agent’s
model of the distance dynamics is strongly ‘biased’ by a prior belief that the
steady-state value of the distance variable(s) $\tilde{\mathbf{x}}$ hovers
around a particular value $\boldsymbol{\eta}$, then agents will change their
heading in a way that appears like they ‘want’ to maintain this target
distance between them and their neighbours. Concretely, this means they move
closer to neighbors when the sensed distance $\mathbf{y}$ is larger than
expected, and move away from neighbors when $\mathbf{y}$ is smaller than
expected.

This symmetry between belief updating and action, as both following the
gradients of the same loss function, is what theoretically distinguishes
active inference from other continuous control schemes, which often use
different objectives for estimation and control. In the following sections we
detail the processes of state-estimation and action under active inference.

###  A.1 Generalized filtering overview

Agents estimate hidden states $x$ as the variational solution to a Bayesian
inference problem; they achieve this in practice using an online-filtering
algorithm known as generalized filtering [83, 84]. Generalized filtering is a
generic Bayesian filtering scheme for non-linear state-space models formulated
in generalized coordinates of motion [85]. It subsumes, as special cases,
variational filtering [86], dynamic expectation maximization [87] and
generalized predictive coding [88]. This inversion scheme relies on a simple
dynamical generative specification of hidden states $x$ and how they relate to
observations $y$. The generative model starts by postulating that the time
evolution of a variable $x$ is given by a stochastic differential equation
with the following form:

$\displaystyle\frac{dx_{t}}{dt}$ $\displaystyle=f(x_{t})+\omega_{t}$ (A.1)

where $f$ is some deterministic flow function (i.e., a vector field) that
depends on the current state $x_{t}$, and $\omega_{t}$ is a (smooth) additive
Gaussian noise process. Under generalized filtering, we successively
differentiate (A.1), to finesse the difficult computation of the paths or
trajectories of $x_{t}$ locally in time, by instead focusing on the much
easier problem of computing the serial derivatives of $x_{t}$. This allows one
to express a local trajectory of $\vec{x}=\\{x_{t},x_{t+1},\ldots,x_{t+T}\\}$
in terms of the derivatives of $x_{t}$, i.e.,
$\tilde{x}_{t}=(x_{t}^{\prime},x_{t}^{\prime\prime},x_{t}^{\prime\prime\prime},\ldots,x^{[n]}_{t},\ldots)$,
where $x^{[n]}_{t}:=\frac{d^{n}}{dt^{n}}x_{t}$. We used the notation
$\tilde{x}_{t}$ to denote a vector of these higher orders of motion at time
$t$, a representation known as generalized coordinates. The equivalence
between generalized coordinates and paths locally in time follows from
Taylor’s theorem, where the path of $x$ around some time $t$ can be expressed
as a combination of its higher order derivatives:

$\displaystyle x_{t+h}$
$\displaystyle=x_{t}^{[0]}+\sum_{n=1}^{\infty}\frac{x^{[n]}}{n!}h^{n}$ (A.2)

Note that the (local in time) equality between a path $\vec{x}$ and its Taylor
series only holds when the sample paths of $x_{t}$ are analytic functions,
which itself requires $f$ to be analytic and the noise process $\omega_{t}$ to
be analytic (in particular non-white noise fluctuations) [89]. Successively
differentiating the base equation in (A.1) (and ignoring contributions of the
flow of order higher than one) yields a series of stochastic differential
equations that describe the evolution of each order of motion $x_{t}^{[n]}$ as
depending on its own state and the $n$th derivative of the noise [85]:

$\displaystyle\dot{x}$ $\displaystyle=f(x)+\omega$
$\displaystyle\dot{x}^{\prime}$
$\displaystyle=f_{x}x^{\prime}+\omega^{\prime}$
$\displaystyle\dot{x}^{\prime\prime}$
$\displaystyle=f_{x}x^{\prime\prime}+\omega^{\prime\prime}$
$\displaystyle\vdots$ $\displaystyle\Rightarrow D\tilde{x}$
$\displaystyle=\tilde{f}+\tilde{\omega}$

where, following the notation used in [84, 83, 85], we use the notation
$f_{x}$ for the Jacobian (i.e., matrix of first order partial derivatives) of
the flow function $f$ evaluated at $x$, i.e., $Jf(x)$, and omit the time
variable from our notation for conciseness. Note that the above construction
assumes a local linearization of $f$ around $x$, in the sense that it ignores
the contribution of higher order derivatives of the flow [85]. The $D$ is the
time derivative operator in generalised coordinates, with identity matrices
along the first leading (block) diagonal and $\tilde{f},\tilde{\omega}$ are
the generalized flow function and generalized noises, respectively:

$D=\begin{bmatrix}0&I&\\\ &\ddots&\ddots&\\\ &&\ddots&I\\\
&&&0\end{bmatrix}\hskip 28.45274pt\tilde{f}=\begin{bmatrix}f(x^{[0]})\\\
f_{x}x^{[1]}\\\ \vdots\\\ f_{x}x^{[n]}\end{bmatrix}\hskip
28.45274pt\tilde{\omega}=\begin{bmatrix}\omega^{[0]}\\\ \omega^{[1]}\\\
\vdots\\\ \omega^{[n]}\end{bmatrix}\\\ $

Here, $n$ is some chosen order at which to truncate the derivatives. This
truncation means that the Taylor expansion of a path $\vec{x}$ in (A.2) is
rendered an approximation – valid locally in time. Having specified a dynamics
over $x$ (and its reformulation in generalized coordinates), we are in a
position to specify the observation model. In generalized filtering, the
generative model of state dynamics is supplemented with an observation model
that maps hidden states $x$ to their sensory consequences $y$ via some
(differentiable) sensory map $g(x)$ and additive Gaussian smooth fluctuations
$z$:

$\displaystyle y_{t}$ $\displaystyle=g(x_{t})+z_{t}$ (A.3)

Like the states, we can similarly express observations in generalized
coordinates by successively differentiating (A.3) to obtain a similar single
expression for the generalized observation equation:

$\displaystyle y$ $\displaystyle=g(x)+z$ $\displaystyle y^{\prime}$
$\displaystyle=g_{x}x^{\prime}+z^{\prime}$ $\displaystyle y^{\prime\prime}$
$\displaystyle=g_{x}x^{\prime\prime}+z^{\prime\prime}$ $\displaystyle\vdots$
$\displaystyle\Rightarrow\tilde{y}$ $\displaystyle=\tilde{g}+\tilde{z}$

where here the $i$th motion of observations $y^{[i]}$ is not a function of
itself but rather that of the motion of the (generalized) hidden states
$x^{[i]}$ and fluctuations $z^{[i]}$. In other words, the motion of
observations tracks the simultaneous motion of the states, subject to any
nonlinearities in the sensory map $g$ and the motion of the noise $z$. Given
Gaussian assumptions on the generalised noises $\tilde{\omega}$ and
$\tilde{z}$, we can then write down the full hidden state and observation
model $p(\tilde{y},\tilde{x})$ in terms of Gaussian densities:

$\displaystyle D\tilde{x}$ $\displaystyle=\tilde{f}+\tilde{\omega}$
$\displaystyle\tilde{\omega}$
$\displaystyle\sim\mathcal{N}(\tilde{\omega};\mathbf{0},\tilde{\Sigma}^{\omega})$
$\displaystyle\tilde{y}$ $\displaystyle=\tilde{g}+\tilde{z}$
$\displaystyle\tilde{z}$
$\displaystyle\sim\mathcal{N}(\tilde{z};\mathbf{0},\tilde{\Sigma}^{z})$
$\displaystyle\implies p(\tilde{y},\tilde{x})$
$\displaystyle=p(\tilde{y}|\tilde{x})p(D\tilde{x}|\tilde{x})$
$\displaystyle=\mathcal{N}(\tilde{y};\tilde{g},\tilde{\Sigma}^{z})\mathcal{N}(D\tilde{x};\tilde{f},\tilde{\Sigma}^{\omega})$
(A.4)

This Gaussian specification of the generative model licenses efficient, online
update rules for the sufficient statistics of approximate posterior beliefs
that track the expected value of the generalised hidden state $\tilde{x}$.
This relies on a simple expression for the variational free energy of this
state-space model; as we will see in the following sections, this not only
enables efficient state estimation (a.k.a, updating beliefs about hidden
states $\tilde{x}$), but also algorithms for inferring generative model
parameters.

###  A.2 State estimation under generalized filtering

Generalized filtering relies on optimizing posterior beliefs in order to
minimize variational free energy $F$, an upper bound on the surprise
associated with observations $y$ under some generative model $m$:

$F\geq\underbrace{-\ln p(y;m)}_{\textrm{surprise}}$ (A.5)

where the model $m$ defines a joint distribution over observations and latent
variables $p(y,\vartheta)$. The latent variables themselves $\vartheta$ are
often split into hidden states $x$ and parameters $\theta$. Exact Bayesian
inference entails obtaining the posterior distribution over latent variables
$p(\vartheta|y)$, which can be expressed using Bayes rule:

$\displaystyle p(\vartheta|y)$ $\displaystyle=\frac{p(y,\vartheta)}{p(y)}$
(A.6) $\displaystyle\ p(y)$ $\displaystyle\triangleq\int
p(y,\vartheta)d\vartheta$ (A.7)

where hereafter we leave out the dependence on the model $m$.

In order to compute the posterior exactly, one has to compute the marginal
probability of observations $p(y)$, also known as the marginal likelihood or
model evidence. Computing the marginal likelihood is often intractable or
difficult in practice, motivating the introduction of the variational bound,
the free energy $F$, also known as the (negative) evidence lower-bound or
ELBO. This can be shown by writing $F$ as the Kullback-Leibler divergence
between some "variational" distribution $q(\vartheta;\nu)$ over latent
variables with parameters $\nu$ and the true posterior $p(\vartheta|y)$:

$\displaystyle F$ $\displaystyle=\mathbb{E}_{q}\left[\ln q(\vartheta)-\ln
p(y,\vartheta)\right]$
$\displaystyle=D_{KL}\left(q(\vartheta;\nu)||p(\vartheta|y)\right)\underbrace{-\ln
p(y)}_{\text{surprise}}$ (A.8) $\displaystyle\ \implies F$
$\displaystyle\geq-\ln p(y)$ (A.9)

The upper bound holds because the Kullback-Leibler divergence is always non-
negative $D_{KL}(p||q)\geq 0$. Intuitively, as the variational distribution
$q(\vartheta;\nu)$ better approximates the true posterior distribution
$p(\vartheta|y)$, where the (in)accuracy of the approximation is measured by
the KL divergence, then the tighter the free energy bounds the surprise. This
decomposition also makes clear why minimizing $F$ with respect to variational
parameters $\nu$ is a way to update the variational distribution $q$ to
approximate the true posterior $p(\vartheta|y)$. The variational distribution
is thus often referred to as an approximate posterior, where the exact
posterior obtained by applying Bayesian rule as in Equation (A.6) corresponds
to the variational posterior that minimises $F$.

Now we turn to deriving the Laplace-approximation to the variational free
energy (VFE) for the Gaussian state-space models used in generalised
filtering. The Laplace approximation is an analytically tractable way to
approximate the true posterior with a Gaussian distribution, which simplifies
inference to an online filtering algorithm that corresponds to minimizing a
sum of squared prediction errors.

Recall that our goal is to perform inference on the latent variables
$\vartheta$ by optimizing an approximate posterior distribution
$q(\vartheta;\nu)$. In our case, we let $\vartheta=\\{x,\theta\\}$ where $x$
are hidden states and $\theta$ encompass other generative model parameters
(e.g., hyperparameters of the generative model like
$\tilde{f},\tilde{g},\tilde{\Sigma}^{z},\tilde{\Sigma}^{\omega}$). For now we
focus on inference over hidden states $x$ and treat parameter inference later.
Under the Laplace approximation we use a Gaussian distribution for the
approximate posterior distribution $q(x;\nu)$:

$\displaystyle q(x;\nu)$
$\displaystyle=\mathcal{N}(x;\underbrace{\mu,\Sigma^{\nu}}_{\nu})$ (A.10)

where the variational parameters $\nu$ are comprised of the sufficient
statistics of a Gaussian distribution: the mean $\mu$ and covariance
$\Sigma^{\nu}$. We add the subscript $\nu$ to the variational variance to
distinguish it from generative model covariances, e.g.
$\tilde{\Sigma}^{z},\tilde{\Sigma}^{\omega}$.

We can now arrive at a more specific expression for the variational free
energy using the Gaussian form of the variational distribution. We start by
decomposing the free energy into the sum of an expected energy term and a
(negative) entropy, where the energy is defined as the negative log joint
density over states and observations: $-\ln p(x,y)$ and the negative entropy
is that of the variational posterior i.e., $\mathbb{E}_{q}[\ln q(x;\nu)]$:

$\displaystyle F=\mathbb{E}_{q}\left[-\ln
p(x,y)\right]-\frac{1}{2}\left[\ln\left|\Sigma\right|+d\ln 2\pi e\right]$
(A.11)

where $d$ is the dimensionality of $x$ and the full term on the right follows
from the entropy of a multivariate Gaussian:
$\operatorname{H}[\mathcal{N}(x;\mu,\Sigma)]=\frac{1}{2}\left[\ln\left|\Sigma\right|+d\ln
2\pi e\right]$.

Additional assumptions allow one to further simplify the expected energy term
$\mathbb{E}_{q}\left[-\ln p(x,y)\right]$; namely, if we assume that the
posterior is tightly peaked around the mean $\mu$ and that $p(x,y)$ is twice-
differentiable in $x$, we can motivate a 2nd-order Taylor expansion of the
expected energy term around its mode, i.e. when $x=\mu$:

$\displaystyle\mathbb{E}_{q}\left[-\ln p(x,y)\right]$
$\displaystyle\approx\mathbb{E}_{q}\left[-\ln p(\mu,y)-\nabla_{x}\ln
p(x,y)\biggr{\rvert}_{x=\mu}(x-\mu)-\frac{1}{2}(x-\mu)^{\top}\nabla^{2}_{x}\ln
p(x,y)\biggr{\rvert}_{x=\mu}(x-\mu)\right]$ $\displaystyle=-\ln
p(\mu,y)-\frac{1}{2}\operatorname{tr}\left(\Sigma\nabla^{2}_{x}\ln
p(x,y)\biggr{\rvert}_{x=\mu}\right)$ (A.12)

Combining this approximation of the expected energy with the remaining terms
in the variational free energy, we can now write the full expression of the
Laplace-approximated free energy $F_{L}$:

$\displaystyle F_{L}=-\ln
p(\mu,y)-\frac{1}{2}\operatorname{tr}\left(\Sigma\nabla^{2}_{x}\ln
p(x,y)\biggr{\rvert}_{x=\mu}\right)-\frac{1}{2}\left(\ln\left|\Sigma\right|+d\ln
2\pi e\right)$ (A.13)

A useful feature of this expression is that the optimal variational covariance
$\Sigma^{\nu}$ can obtained by setting the derivative of $F_{L}$ with respect
to the covariance $\Sigma$ equal to 0 and solving for $\Sigma$, i.e. finding
the values of the covariance that minimize the $F_{L}$:

$\displaystyle\frac{\partial F_{L}}{\partial\Sigma}=0$
$\displaystyle\iff\Sigma^{\nu}=-\left(\nabla^{2}_{x}\ln
p(x,y)\biggr{\rvert}_{x=\mu}\right)^{-1}$ (A.14)

i.e., the optimal variance of the variational distribution is the curvature of
the Laplace energy around its mode. Substituting this expression back into the
full free energy, we can then write an expression that only depends on the
mean vector $\mu$ of the variational density, since the variatonal variance
$\Sigma^{\nu}$ is now expressed as a function of the mean:

$\displaystyle F_{L}$ $\displaystyle=-\ln
p(\mu,y)+\frac{1}{2}\underbrace{\operatorname{tr}\left(\Sigma^{\nu}(\Sigma^{\nu})^{-1}\right)}_{=d}-\frac{1}{2}\left(\ln\left|\Sigma^{\nu}\right|+d\ln
2\pi e\right)$ $\displaystyle=-\ln
p(\mu,y)-\frac{1}{2}\left(\ln\left|\Sigma^{\nu}\right|+d\ln 2\pi\right)$
(A.15)

This means that the Laplace approximation to the variational free energy is a
function of only the variational mean $\mu$ and sensory observations $y$,
because the variational variance $\Sigma^{\nu}$ is itself a function of $\mu$.
Belief updating then consists in minimizing the Laplace-approximated free
energy $F_{L}$ with respect to $\mu$:

$\displaystyle\dot{\mu}\propto-\nabla_{\mu}F_{L}(\mu,y)$ (A.16)

Which consists of descending the gradient of the energy and log determinant
terms with respect to $\mu$. When the generative model $p(x,y)$ is Gaussian,
the energy term is quadratic in $\mu$ and $y$. This means that its gradient
can be written in terms of precision-weighted prediction errors, which score
the difference between the expected observations (given the current value of
$\mu$) and the actual observations $y$. This notion of using prediction errors
to estimate hidden quantities is also known as predictive coding [90, 91, 92].
The log determinant term — in all of our cases of interest — turns out to have
a vanishing gradient with respect to $\mu$. In summary, the free energy
gradient is a sum of precision-weighted prediction errors, and $\mu$ evolves
to minimize those prediction errors.

To illustrate this, we take the simplest example — that of a linear, static,
joint Gaussian generative model, where the prior over hidden states $p(x)$ is
a Gaussian density with mean $\eta$ and covariance $\Sigma^{\omega}$, and the
observation model $p(x|y)$ is a Gaussian density with mean $g(x)$, which is
some linear function of the hidden state:

$\begin{split}y\sim\mathcal{N}(g(x),\Sigma^{z}),\quad
x\sim\mathcal{N}(\eta,\Sigma^{\omega}).\end{split}$ (A.17)

For this linear Gaussian generative model, the variational mean $\mu$ only
influences the expected energy term of $F_{L}$, because the optimal covariance
$\Sigma^{\nu}$ is independent of $\mu$. Thus we ignore the constant entropy
term and write out the energy as a sum of precision-weighted prediction
errors:

$\displaystyle-\ln p(\mu,y)$ $\displaystyle=-\ln p(y|\mu)-\ln p(\mu)$
$\displaystyle=\frac{1}{2}\left[\varepsilon_{z}^{T}\Pi^{z}\varepsilon_{z}+\varepsilon_{\omega}^{T}\Pi^{\omega}\varepsilon_{\omega}\right]$
where $\displaystyle\Pi^{z}=(\Sigma^{z})^{-1},\hskip
5.69054pt\Pi^{\omega}=(\Sigma^{\omega})^{-1}$ and
$\displaystyle\varepsilon_{z}=y-g(\mu),\hskip
5.69054pt\varepsilon_{\omega}=\mu-\eta$ (A.18)

We can write out gradients of this quadratic energy function to yield the
update equation for the means $\mu$ as in (A.16), and see that $\mu$ changes
as a precision-weighted sum of ‘sensory‘ and ‘model’ prediction errors (up to
additive constants):

$\displaystyle\dot{\mu}$ $\displaystyle=-\nabla_{\mu}F_{L}(\mu,y)$
$\displaystyle=-\nabla_{\mu}\left[\frac{1}{2}\left(\varepsilon_{z}^{T}\Pi^{z}\varepsilon_{z}+\varepsilon_{\omega}^{T}\Pi^{\omega}\varepsilon_{\omega}\right)\right]$
$\displaystyle=-\left[(g_{\mu})^{T}\Pi^{z}\varepsilon_{z}+\Pi^{\omega}\varepsilon_{\omega}\right]$
(A.19)

Note that the variational means only depend on the terms of $F_{L}$ containing
$\varepsilon_{z}$ and $\varepsilon_{\omega}$, so that the update reduces to a
gradient descent on a sum of squared prediction errors. This belief update
scheme illustrates the key principles of predictive coding under the Laplace
approximation: conditional means, denoted as $\mu$, change as a function of
precision-weighted prediction errors. The concept of precision-weighting in
belief updating is intuitive: if the generative model attributes higher
variance to sensory fluctuations as compared to state variance (i.e.,
$\Pi^{z}<\Pi^{\omega}$), then sensory data is relatively unreliable and
consequently makes a smaller impact on posterior beliefs. Therefore, the
adjustment to the posterior mean $\mu$ in (A.19) is primarily influenced by
the state prediction error term $\Pi^{\omega}\varepsilon_{\omega}$ or the
prior. Conversely, when sensory information is allocated higher precision
(lower variance) relative to prior beliefs (i.e., $\Pi^{z}>\Pi^{\omega}$),
belief updates will strongly rely on sensory data.

We apply the above steps to derive the Laplace-approximated free energy with a
Gaussian posterior $q(x;\nu)$ to the dynamical generative model in (A.4),which
is constructed from Gaussian densities. Note that we use the tilde notation to
now indicate that all variables are vectors of generalised coordinates, e.g.,
$\tilde{y},\tilde{x}$, etc. Proceeding exactly as above, the Laplace free
energy is a sum of the energy, a log determinant term, and a constant term,
i.e. (A.15). Unlike in the linear Gaussian case, the potential non-linearity
in the flows make that the log determinant term varies with respect to $\mu$.
However, as it turns out its gradient is approximately zero under the local
linear approximation. Therefore, the only term that matters in the free energy
as its gradient does not vanish is the energy term. In summary, we write:

$\displaystyle F_{L}$
$\displaystyle\propto\tilde{\varepsilon}_{z}^{T}\tilde{\Pi}^{z}\tilde{\varepsilon}_{z}+\tilde{\varepsilon}_{\omega}^{T}\tilde{\Pi}^{\omega}\tilde{\varepsilon}_{\omega}$
(A.20) $\displaystyle\tilde{\varepsilon}_{z}$
$\displaystyle\triangleq\tilde{y}-\tilde{g}$
$\displaystyle\tilde{\varepsilon}_{\omega}$ $\displaystyle\triangleq
D\tilde{\mu}-\tilde{f}$

Here, the so-called ‘generalised errors’ $\tilde{\varepsilon}_{z}$ and
$\tilde{\varepsilon}_{\omega}$ encapsulate sensory and state prediction errors
across orders of motion. Belief updating is again performed using a gradient
descent on free energy, but the dynamic nature of inference necessitates an
additional ’motion’ term:

$\displaystyle\frac{d\tilde{\mu}}{dt}$
$\displaystyle=D\tilde{\mu}-\nabla_{\tilde{\mu}}F_{L}$
$\displaystyle=D\tilde{\mu}+{g}_{\tilde{\mu}}^{\top}\tilde{\xi}_{z}+{f}_{\tilde{\mu}}^{\top}\tilde{\xi}_{\omega}-D^{\top}\tilde{\xi}_{\omega}$
$\displaystyle\textrm{where}\hskip 5.69054pt\tilde{\xi}_{z}$
$\displaystyle=\tilde{\Pi}^{z}\tilde{\varepsilon}_{z}$
$\displaystyle\tilde{\xi}_{\omega}$
$\displaystyle=\tilde{\Pi}^{\omega}\tilde{\varepsilon}_{\omega}$ (A.21)

The additional term $D\tilde{\mu}$ places the gradient descent within the
context of the expected movement of the conditional means $\tilde{\mu}$, and
hence of the free energy minimum. This concept has been referred to as
’gradient descent in a moving frame of reference’ [83]. This implies that free
energy minimization does not occur when the beliefs cease moving, but rather
when the belief update rate $\frac{d\tilde{\mu}}{dt}$ is identical to the
beliefs about the motion itself $D\tilde{\mu}$, in other words when
$\frac{\partial F}{\partial\tilde{\mu}}=0\iff
D\tilde{\mu}=\frac{d\tilde{\mu}}{dt}$. This additional temporal correction
proves beneficial in a dynamic data assimilation regime, where incoming
observations are integrated online with beliefs that are evolving according to
their own prior dynamics [83].

###  A.3 Active inference for continuous control

Active inference casts action or control as issuing from the same process of
free energy minimization as used for state estimation; the only difference is
that we now have an additional set of variables, actions $a$, that can be
changed to minimize free energy as well. The update equation for actions $a$
closely resembles that used to update the variational mean $\mu$, i.e., a
gradient descent on the (Laplace-encoded) variational free energy:

$\displaystyle\frac{da}{dt}$ $\displaystyle=-\frac{\partial
F_{L}(\mu,y(a))}{\partial a}$ $\displaystyle=-\frac{\partial F_{L}}{\partial
y(a)}\frac{\partial y(a)}{\partial a}$ (A.22)

where we have now introduced a dependence between of observations $y$ on
actions $a$. This allows us to express the free energy gradient with respect
to action as the product of the derivative of the free energy with respect to
observations $\nabla_{y}F_{L}(\mu,y(a))$ and the derivative of the function
mapping from actions to observations $\frac{\partial y(a)}{\partial a}$. The
free energy gradient with respect to observations is exactly the sensory
prediction error $\nabla_{y}F_{L}(\mu,y(a))=\xi_{z}=\Pi(y-g(x))$. This assumed
dependence of observations on actions underwrites the notion that active
inference agents cannot directly measure how their actions affect hidden
states, but may only do so via their sensory consequences. This has been
speculated to explain the architecture of descending motor pathways in
corticospinal systems, where motor commands are ‘unpacked’ into proprioceptive
predictions at the level of spinal circuits and other lower motor nuclei.
Action is thus realized by minimizing proprioceptive prediction errors via
classical reflex arcs [93]. The reflex arc term $\frac{\partial y(a)}{\partial
a}$ of (A.22) is analogous to a forward model in motor control [94], because
it reflects the agent’s implicit assumptions about how the agent’s own actions
lead to their (anticipated) sensory consequences. This sort of update rule
leads active inference agents to minimize sensory prediction errors via these
‘baked-in’ sensorimotor contingencies. In this way active inference has been
referred to as ‘action by self-fulfilling prophecy’ [88]. In other words, the
agent generates top-down expectations of ‘preferred’ sensory inputs, which
then generates prediction errors which can then be suppressed through low-
level motoric reflexes [93].

###  A.4 Filtering and control for a self-propelled particle

Having derived a routine for state estimation and action through a generalized
gradient flow on the Laplace-approximated variational free energy $F_{L}$, we
can now apply this to the simulation of collective motion. In what follows, we
write down a sufficient generative model for a single self-propelled agent and
unpack the corresponding free energy gradients ((A.20) and (A.22)) using the
structure and parameters of the chosen generative model. In this section we
unpack the per-agent generative model of local distances described in the main
text and demonstrate how a more parametric, unconstrained version of social
forces are reproduced by minimizing free energy with respect to the distance-
tracking generative model.

####  A.4.1 A generalised filter for local distances and their time evolution

As described in the main text, each agent represents a an $L$-dimensional
vector $\mathbf{x}$ where $\mathbf{x}=(x_{1},x_{2},...,x_{L})$.111We use the
bold notation $\mathbf{x}$ to represent a vector-valued variable The agent not
only represents the instantaneous value (or ‘position’) of $\mathbf{x}$ but
also its generalized motion, which we truncate at 3rd order:

$\displaystyle\dot{\mathbf{x}}$
$\displaystyle=f(\mathbf{x})+\boldsymbol{\omega}$
$\displaystyle\dot{\mathbf{x}}^{\prime}$
$\displaystyle=f_{\mathbf{x}}\mathbf{x}^{\prime}+\boldsymbol{\omega}^{\prime}$
$\displaystyle\dot{\mathbf{x}}^{\prime\prime}$
$\displaystyle=f_{\mathbf{x}}\mathbf{x}^{\prime\prime}+\boldsymbol{\omega}^{\prime\prime}$
$\displaystyle\Rightarrow D\tilde{\mathbf{x}}$
$\displaystyle=\tilde{\mathbf{f}}+\tilde{\boldsymbol{\omega}}$

The flow at the first order $f$ is a linear dynamical system with drift matrix
$A$ and fixed point with value $\boldsymbol{\eta}$:

$\displaystyle f(\mathbf{x})$
$\displaystyle=-\mathbf{A}(\mathbf{x}-\boldsymbol{\eta})$ (A.23)

The eigenvalues of the $L\times L$ matrix $\mathbf{A}$ determine the rate at
which the hidden states $\mathbf{x}$ are assumed to relax to their expected
value of $\boldsymbol{\eta}$. In general, this matrix can be parameterized
arbitrarily to encode different kinds of linear couplings among the different
hidden states $x_{1},x_{2},...,x_{L}$. In the present work we parameterize
$\mathbf{A}$ simply as a diagonal matrix with a single diagonal value
$\alpha>0$, which can also be expressed as an $\alpha$-scaled version of the
identity matrix $L\times L$ identity matrix $I_{L}$:

$\displaystyle\mathbf{A}$ $\displaystyle=-\alpha I_{L}$ (A.24)

In combination with the amplitude of random fluctuations $\Sigma^{\omega}$,
$\alpha$ determines how quickly the hidden states relax to their mean value of
$\boldsymbol{\eta}$.222Heuristically, it is an exponential decay rate. The
generalised flow function $\tilde{\mathbf{f}}$ can thus be written as a linear
function of the generalised state $\tilde{\mathbf{x}}$:

$\displaystyle\tilde{\mathbf{f}}=\begin{bmatrix}f(\mathbf{x})\\\
f_{\mathbf{x}}\mathbf{x}^{\prime}\\\
f_{\mathbf{x}}\mathbf{x}^{\prime\prime}\end{bmatrix}$
$\displaystyle=-\begin{bmatrix}\mathbf{A}&\mathbf{0}&\mathbf{0}\\\
\mathbf{0}&\mathbf{A}&\mathbf{0}\\\
\mathbf{0}&\mathbf{0}&\mathbf{A}\end{bmatrix}\begin{bmatrix}\mathbf{x}-\boldsymbol{\eta}\\\
\mathbf{x}^{\prime}\\\ \mathbf{x}^{\prime\prime}\end{bmatrix}$
$\displaystyle=\begin{bmatrix}-\alpha I_{L}&\mathbf{0}&\mathbf{0}\\\
\mathbf{0}&-\alpha I_{L}&\mathbf{0}\\\ \mathbf{0}&\mathbf{0}&-\alpha
I_{L}\end{bmatrix}\begin{bmatrix}\mathbf{x}-\boldsymbol{\eta}\\\
\mathbf{x}^{\prime}\\\
\mathbf{x}^{\prime\prime}\end{bmatrix}=-\alpha\begin{bmatrix}\mathbf{x}-\boldsymbol{\eta}\\\
\mathbf{x}^{\prime}\\\ \mathbf{x}^{\prime\prime}\end{bmatrix}$ (A.25)

where $\mathbf{0}$ are $L\times L$ matrices of zeros. We assume a multivariate
Gaussian form for the generalized noises $\tilde{\boldsymbol{\omega}}$,
meaning the density over the generalized motion $D\tilde{\mathbf{x}}$ is a
Gaussian density, which we hereafter refer to as the ‘dynamics model’ or
‘dynamical prior’:

$\displaystyle P(D\tilde{\mathbf{x}}|\tilde{\mathbf{x}})$
$\displaystyle=\mathcal{N}(D\tilde{\mathbf{x}};\tilde{\mathbf{f}},\tilde{\Sigma}^{\boldsymbol{\omega}})$
(A.26)

Consistent with the block diagonal form of the generalised flow function
$\tilde{\mathbf{f}}$, we also assume the covariance of the generalized noises
$\tilde{\Sigma}^{\boldsymbol{\omega}}$ factorizes into a Kronecker product of
‘spatial’ and ‘temporal’ covariance matrices, i.e.,

$\displaystyle\tilde{\Sigma}^{\boldsymbol{\omega}}$
$\displaystyle=\Sigma^{\boldsymbol{\omega}}\otimes\tilde{\Sigma}^{\omega}$
(A.27)

where the spatial covariance $\Sigma^{\boldsymbol{\omega}}$ (note the bold
superscript $\boldsymbol{\omega}$) represents covariance between $L$ noise
processes at the zero-th order $\boldsymbol{\omega}^{[0]}$, i.e.,
$\Sigma^{\boldsymbol{\omega}}=\mathbb{E}[\boldsymbol{\omega}^{[0]}\otimes\boldsymbol{\omega}^{[0]}]$,
and $\tilde{\Sigma}^{\omega}$ encodes covariance between different derivatives
of the first order noise, i.e.,
$\forall{m,n}:\left(\tilde{\Sigma}^{\omega}\right)_{nm}=\mathbb{E}[\omega^{[n]}\cdot\omega^{[m]}]$.
The entries of this covariance matrix can be written in terms of the
derivatives of the autocorrelation function of the random fluctuations
evaluated at lag $0$, $\rho(0)$:

$\displaystyle\rho(h)$
$\displaystyle\triangleq(\Sigma^{\boldsymbol{\omega}})^{-1}\mathbb{E}[\omega^{[0]}(\tau)\cdot\omega^{[0]}(\tau+h)]$
$\displaystyle\Rightarrow\tilde{\Sigma}^{\omega}$
$\displaystyle=\begin{bmatrix}1&0&\ddot{\rho}(0)&\\\ 0&-\ddot{\rho}(0)&0&\\\
\ddot{\rho}(0)&0&\ddot{\ddot{\rho}}(0)&\\\ &&&\ddots\end{bmatrix}$ (A.28)

The checkerboard structure in the matrix reflects the fact that fluctuations
at the first order are orthogonal to their motion (first derivative), but
anti-correlated with their 2nd, 4th, …, etc. derivatives. A derivation of the
temporal covariance matrix from the autocorrelation function of the first-
order fluctuations can be found in Appendix A.5.3 of [95]. In the generative
models of our agents, we assume a Gaussian autocorrelation function with
"smoothness" parameter $\lambda_{\omega}$, which yields a simple
parameterization of $\tilde{\Sigma}^{\omega}$:

$\displaystyle\rho(h)$ $\displaystyle=e^{-\frac{h}{2\lambda_{\omega}}^{2}}$
(A.29) $\displaystyle\Rightarrow\tilde{\Sigma}^{\omega}$
$\displaystyle=\begin{bmatrix}1&0&-\frac{1}{2\lambda^{2}_{\omega}}&\ldots\\\
0&\frac{1}{2\lambda^{2}_{\omega}}&0&\\\
-\frac{1}{2\lambda^{2}_{\omega}}&0&\frac{3}{4\lambda^{4}_{\omega}}&\\\
\vdots&&&\ddots\end{bmatrix}$ (A.30)

A higher value of $\lambda_{\omega}$ dampens the variance of the generalised
fluctuations at higher orders of differentiation. The correspondence of
increasing $\lambda_{\omega}$ to an increasingly-autocorrelated process at the
first order becomes intuitive once we consider the case of standard white
noise, i.e., the derivative of the Wiener process, whose higher orders of
motion have infinite variance (the state of the process at a given time
changes infinitely quickly). This ability to handle differentiable noise goes
beyond the usual Markovian assumptions made in standard state space models
(e.g., Kalman-Bucy filters), which assume that the driving noise is white.

We parameterize the $L\times L$ spatial covariance
$\Sigma^{\boldsymbol{\omega}}$ through its precision matrix
$\Pi^{\boldsymbol{\omega}}$, as a diagonal matrix whose entries are given by a
single precision (inverse variance) $\Gamma_{\omega}$:

$\displaystyle\Sigma^{\boldsymbol{\omega}}$
$\displaystyle=(\Pi^{\boldsymbol{\omega}})^{-1}=\begin{bmatrix}\Gamma_{\omega}&0&0&\ldots\\\
0&\Gamma_{\omega}&0&\\\ 0&0&\Gamma_{\omega}&\\\
\vdots&&&\ddots\end{bmatrix}^{-1}$ (A.31)

The observation likelihood describes sensory observations
$\mathbf{y}=\\{y_{1},y_{2},...,y_{L}\\}$ as noise-perturbed copies of the
hidden states $\mathbf{x}$. We truncate generalized observations at second
order, i.e., agents can sense the first order hidden state $\mathbf{x}$ and
its motion $\mathbf{x}^{\prime}$:

$\displaystyle\mathbf{y}$ $\displaystyle=\mathbf{x}+\mathbf{z}$
$\displaystyle\mathbf{y}^{\prime}$
$\displaystyle=\mathbf{x}^{\prime}+\mathbf{z}^{\prime}$ (A.32)

This can be equivalently expressed as a linear function $\tilde{\mathbf{g}}$
of the full generalised state
$\tilde{\mathbf{x}}=\\{\mathbf{x},\mathbf{x}^{\prime},\mathbf{x}^{\prime\prime}\\}$,
where $\tilde{\mathbf{g}}$ represents multiplication with a non-invertible
matrix that discards acceleration information $\mathbf{x}^{\prime\prime}$:

$\displaystyle\tilde{\mathbf{y}}$
$\displaystyle=\tilde{\mathbf{g}}+\tilde{\mathbf{z}}$
$\displaystyle\begin{bmatrix}\mathbf{y}\\\ \mathbf{y}^{\prime}\end{bmatrix}$
$\displaystyle=\begin{bmatrix}I_{L}&\mathbf{0}&\mathbf{0}\\\
\mathbf{0}&I_{L}&\mathbf{0}\end{bmatrix}\begin{bmatrix}\mathbf{x}\\\
\mathbf{x}^{\prime}\\\
\mathbf{x}^{\prime\prime}\end{bmatrix}+\begin{bmatrix}\mathbf{z}\\\
\mathbf{z}^{\prime}\end{bmatrix}$ (A.33)

We leverage the same assumptions about the sensory noises $\tilde{\mathbf{z}}$
as we did for the state noises $\tilde{\mathbf{\omega}}$ to end up with the
following multivariate Gaussian form for the observation model:

$\displaystyle p(\tilde{\mathbf{y}}|\tilde{\mathbf{x}})$
$\displaystyle=\mathcal{N}(\tilde{\mathbf{y}};\tilde{\mathbf{g}},\tilde{\Sigma}^{\mathbf{z}})$
(A.34)

We parameterize the likelihood model’s sensory noises $\tilde{\mathbf{z}}$
identically to the state noises $\tilde{\boldsymbol{\omega}}$, namely using a
spatial precision parameter $\Gamma_{z}$ and temporal smoothness parameter
$\lambda_{z}$.

Having specified the dynamics and observation models in terms of Gaussian
distributions, we can write out the full generative model as a joint Gaussian
density over (generalized) hidden states and observations. We can furthermore
define an approximate posterior over the hidden states $\tilde{\mathbf{x}}$
that has a multivariate Gaussian form
$Q(\tilde{\mathbf{x}})=\mathcal{N}(\tilde{\mathbf{x}};\tilde{\boldsymbol{\mu}};\Sigma^{\nu})$,
which can be summarized entirely in terms of its posterior mean vector
$\tilde{\boldsymbol{\mu}}$, due to the fact that under the Laplace
approximation the variational covariance depends directly on the mean. From
here, we can define the Laplace-approximated variational free energy for this
generative model as proportional to a sum of squared prediction errors:

$\displaystyle p(\tilde{\mathbf{y}},\tilde{\mathbf{x}})$
$\displaystyle=p(\tilde{\mathbf{y}}|\tilde{\mathbf{x}})p(D\tilde{\mathbf{x}}|\tilde{\mathbf{x}})$
$\displaystyle=\mathcal{N}(\tilde{\mathbf{y}};\tilde{\mathbf{g}},\tilde{\Sigma}^{\mathbf{z}})\mathcal{N}(D\tilde{\mathbf{x}};\tilde{\mathbf{\mathbf{missing}}{f}},\tilde{\Sigma}^{\boldsymbol{\omega}})$
(A.35) $\displaystyle F_{L}$
$\displaystyle=\frac{1}{2}\left[\tilde{\boldsymbol{\varepsilon}}_{z}^{\top}\tilde{\Pi}^{\boldsymbol{z}}\tilde{\boldsymbol{\varepsilon}}_{z}+\tilde{\boldsymbol{\varepsilon}}_{\omega}^{\top}\tilde{\Pi}^{\boldsymbol{\omega}}\tilde{\boldsymbol{\varepsilon}}_{\omega}-\ln\left(|\tilde{\Pi}^{\mathbf{z}}||\tilde{\Pi}^{\boldsymbol{\omega}}||\Pi^{\nu}|\right)+3L\ln
2\pi\right]$ $\displaystyle\textrm{where }\Pi^{\nu}$
$\displaystyle\triangleq(\Sigma^{\nu})^{-1}$
$\displaystyle\tilde{\boldsymbol{\varepsilon}}_{z}$
$\displaystyle=\tilde{\mathbf{y}}-\tilde{\mathbf{g}}({\tilde{\boldsymbol{\mu}}})=\begin{bmatrix}\mathbf{y}-\boldsymbol{\mu}\\\
\mathbf{y}^{\prime}-\boldsymbol{\mu}^{\prime}\end{bmatrix},\hskip
5.69054pt\tilde{\boldsymbol{\varepsilon}}_{\omega}=D\tilde{\boldsymbol{\mu}}-\tilde{\mathbf{f}}(\tilde{\boldsymbol{\mu}})=\begin{bmatrix}\boldsymbol{\mu}^{\prime}+\alpha(\boldsymbol{\mu}-\boldsymbol{\eta})\\\
\boldsymbol{\mu}^{\prime\prime}+\alpha\boldsymbol{\mu}^{\prime}\\\
\alpha\boldsymbol{\mu}^{\prime\prime}\end{bmatrix}$ (A.36)

where the sensory prediction errors $\tilde{\boldsymbol{\varepsilon}}_{z}$
score the difference between the generalized observations
$\mathbf{y},\mathbf{y}^{\prime}$ and their expected values
$\boldsymbol{\mu},\boldsymbol{\mu}^{\prime}$, and the model or process
prediction errors $\tilde{\boldsymbol{\varepsilon}}_{\omega}$ score the
difference between the motion of the generalized means
$D\tilde{\boldsymbol{\mu}}$ and their expected motion
$\tilde{\mathbf{f}}(\tilde{\boldsymbol{\mu}})$, which has been expanded above
using the linear form of the flow function detailed in (A.25). Note that here,
due to the Laplace approximation, the generative model’s expectation functions
$\tilde{\mathbf{g}},\tilde{\mathbf{f}}$ are evaluated at the variational mean
$\tilde{\boldsymbol{\mu}}$, rendering the variational beliefs a moving point-
estimate of the hidden states $\tilde{\mathbf{x}}$.

Filtering consists of updating $\tilde{\boldsymbol{\mu}}$ as a generalized
gradient flow on this energy functional $F_{L}$ as in
(LABEL:eq:PC_update_dynamic_gm_suppl). To be explicit, below we expand these
free energy gradients using the particular forms of
$\tilde{\mathbf{g}},\tilde{\mathbf{f}}$ used by our self-propelled particle
agent:

$\displaystyle\frac{d\tilde{\boldsymbol{\mu}}}{dt}$
$\displaystyle=D\tilde{\boldsymbol{\mu}}-\nabla_{\tilde{\boldsymbol{\mu}}}F_{L}$
$\displaystyle=D\tilde{\boldsymbol{\mu}}+\nabla_{\tilde{\boldsymbol{\mu}}}\tilde{\mathbf{g}}^{\top}\tilde{\boldsymbol{\xi}}_{z}+\nabla_{\tilde{\boldsymbol{\mu}}}\tilde{\mathbf{f}}^{\top}\tilde{\boldsymbol{\xi}}_{\omega}-D^{\top}\tilde{\boldsymbol{\xi}}_{\omega}$
$\displaystyle\textrm{where}\hskip 5.69054pt\tilde{\boldsymbol{\xi}}_{z}$
$\displaystyle=\tilde{\Pi}^{\mathbf{z}}\begin{bmatrix}\mathbf{y}-\boldsymbol{\mu}\\\
\mathbf{y}^{\prime}-\boldsymbol{\mu}^{\prime}\end{bmatrix}$
$\displaystyle\tilde{\boldsymbol{\xi}}_{\omega}$
$\displaystyle=\tilde{\Pi}^{\boldsymbol{\omega}}\begin{bmatrix}\boldsymbol{\mu}^{\prime}+\alpha(\boldsymbol{\mu}-\boldsymbol{\eta})\\\
\boldsymbol{\mu}^{\prime\prime}+\alpha\boldsymbol{\mu}^{\prime}\\\
\alpha\boldsymbol{\mu}^{\prime\prime}\end{bmatrix}$
$\displaystyle\nabla_{\tilde{\boldsymbol{\mu}}}\tilde{\mathbf{g}}$
$\displaystyle=\begin{bmatrix}I_{L}&\mathbf{0}&\mathbf{0}\\\
\mathbf{0}&I_{L}&\mathbf{0}\end{bmatrix},\hskip
5.69054pt\nabla_{\tilde{\boldsymbol{\mu}}}\tilde{\mathbf{f}}=\begin{bmatrix}-\alpha
I_{L}&\mathbf{0}&\mathbf{0}\\\ \mathbf{0}&-\alpha I_{L}&\mathbf{0}\\\
\mathbf{0}&\mathbf{0}&-\alpha I_{L}\end{bmatrix}$ (A.37)

This sort of filtering scheme means that the agent’s beliefs