# Exact minimax entropy models of large–scale neuronal activity

Christopher W. Lynn  Corresponding author<EMAIL_ADDRESS>Initiative for the Theoretical Sciences, The Graduate Center, City University
of New York, New York, NY 10016, USA  Joseph Henry Laboratories of Physics,
Princeton University, Princeton, NJ 08544, USA  Department of Physics,
Quantitative Biology Institute, and Wu Tsai Institute, Yale University, New
Haven, CT 06520, USA  Qiwei Yu  Lewis–Sigler Institute for Integrative
Genomics, Princeton University, Princeton, NJ 08544, USA  Rich Pang  Princeton
Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
Stephanie E. Palmer  Department of Organismal Biology and Anatomy, University
of Chicago, Chicago, IL 60637, USA  Department of Physics, University of
Chicago, Chicago, IL 60637, USA  William Bialek  Joseph Henry Laboratories of
Physics, Princeton University, Princeton, NJ 08544, USA  Lewis–Sigler
Institute for Integrative Genomics, Princeton University, Princeton, NJ 08544,
USA  Center for Studies in Physics and Biology, Rockefeller University, New
York, NY 10065 USA

###### Abstract

In the brain, fine–scale correlations combine to produce macroscopic patterns
of activity. However, as experiments record from larger and larger
populations, we approach a fundamental bottleneck: the number of correlations
one would like to include in a model grows larger than the available data. In
this undersampled regime, one must focus on a sparse subset of correlations;
the optimal choice contains the maximum information about patterns of activity
or, equivalently, minimizes the entropy of the inferred maximum entropy model.
Applying this “minimax entropy” principle is generally intractable, but here
we present an exact and scalable solution for pairwise correlations that
combine to form a tree (a network without loops). Applying our method to over
one thousand neurons in the mouse hippocampus, we find that the optimal tree
of correlations reduces our uncertainty about the population activity by 14%
(over 50 times more than a random tree). Despite containing only 0.1% of all
pairwise correlations, this minimax entropy model accurately predicts the
observed large–scale synchrony in neural activity and becomes even more
accurate as the population grows. The inferred Ising model is almost entirely
ferromagnetic (with positive interactions) and exhibits signatures of
thermodynamic criticality. These results suggest that a sparse backbone of
excitatory interactions may play an important role in driving collective
neuronal activity.

##  I Introduction

Understanding how collective behaviors emerge from webs of fine–scale
interactions is a central goal in statistical mechanics approaches to networks
of neurons Wiener (1958); Cooper (1973); Little (1974); Hopfield (1982); Amit
(1989); Hertz  _et al._ (1991). At the same time, exploration of the brain has
been revolutionized by experimental methods that monitor, simultaneously, the
electrical activity of hundreds or even thousands of neurons Segev  _et al._
(2004); Litke  _et al._ (2004); Chung  _et al._ (2019); Dombeck  _et al._
(2010); Tian  _et al._ (2012); Demas  _et al._ (2021); Steinmetz  _et al._
(2021). One approach to connecting these new data with statistical physics
models is maximum entropy, in which we construct the maximally disordered
model that is consistent with measured expectation values Jaynes (1957). In
particular, is seems natural to build models that match the mean activity of
individual neurons and the correlations between pairs of neurons. These
pairwise maximum entropy models have been strikingly successful in describing
collective behavior not only in networks of real neurons, but also in the
evolution of protein families, the dynamics of genetic networks, flocks of
birds, and social networks Schneidman  _et al._ (2006); Nguyen  _et al._
(2017); Meshulam  _et al._ (2017, 2021); Tkačik  _et al._ (2015); Lezon  _et
al._ (2006); Weigt  _et al._ (2009); Marks  _et al._ (2011); Lapedes  _et al._
(2012); Bialek  _et al._ (2012); Russ  _et al._ (2020); Lynn  _et al._ (2019).

But as experiments progress to record from larger and larger numbers of
neurons, we face a combinatorial explosion. Even if we focus on pairwise
correlations, the number of correlations approaches the number of independent
samples in modern experiments Segev  _et al._ (2004); Litke  _et al._ (2004);
Chung  _et al._ (2019); Dombeck  _et al._ (2010); Tian  _et al._ (2012); Demas
_et al._ (2021); Steinmetz  _et al._ (2021). In this undersampled regime, one
is forced to select only a sparse subset of the correlations to include in any
model. While constructing an accurate model with only a small number of
correlations may seem hopeless, one can draw inspiration from statistical
physics, where effective descriptions of macroscopic phenomena can often
ignore many of the microscopic details.

Here, given restrictions on the number and structure of correlations we can
include in a model, we seek to identify the ones that contain the maximum
information about system activity. We demonstrate that the optimal
correlations are those that induce the maximum entropy model with minimum
entropy Zhu  _et al._ (1997). Solving this minimax entropy problem is
generally infeasible. But for pairwise correlations that form a tree (a
network without loops), the entropy reduction decomposes into a sum over
connected pairs; the advantages of tree structure in models of neural activity
have been appreciated in other contexts Prentice  _et al._ (2016). This
decomposition reduces the minimax entropy problem to a minimum spanning tree
problem, which can be solved exactly and efficiently Chow and Liu (1968);
Nguyen  _et al._ (2017). The result is a framework for uncovering the
maximally informative tree of correlations in very large systems Lynn  _et
al._ (2023).

We apply our method to investigate the collective activity of $N\sim 1500$
neurons in the mouse hippocampus Gauthier and Tank (2018). While most pairs of
neurons are only weakly correlated, some rare pairs have mutual information
orders of magnitude larger than average. By focusing on these exceptionally
strong correlations, our minimax entropy model captures 50 times more
information than a random tree and, despite containing only 0.1% of all
pairwise correlations, produces realistic large–scale synchrony in activity.
Moreover, the model becomes even more accurate as the population grows,
providing hope for statistical physics descriptions of even larger systems.

The paper is organized as follows. In §II, we define the minimax entropy
problem and present a solution for trees of pairwise correlations. In §III, we
review a relatively recent experiment on large–scale recordings of neuronal
activity in the mouse hippocampus. In §IV, we demonstrate that the optimal
tree of correlations produces realistic patterns of synchronized activity. In
§V, we investigate the structural properties of the optimal tree and the
functional properties of the induced Ising model. In §VI, we show that the
minimax entropy model becomes more accurate for larger populations, and then
in §VII we investigate the thermodynamic properties of the minimax entropy
model, finding that the real system is poised at a special point in its phase
diagram. Finally, in §VIII we provide conclusions and outlook.

##  II Minimax entropy trees

###  II.1 Maximum entropy models

Consider a system of $N$ elements $i=1,\ldots,N$ with states
$\bm{x}=\\{x_{i}\\}$, where $x_{i}$ is the state of element $i$. From
experiments, we have access to $M$ samples of the system activity
$\bm{x}^{(m)}$, where $m=1,\ldots,M$. Our knowledge about the system is
defined by observables, which can be represented as expectation values

$\left<f(\bm{x})\right>_{\text{exp}}=\frac{1}{M}\sum_{m=1}^{M}f(\bm{x}^{(m)}),$
(1)

where $f(\bm{x})$ is an arbitrary function of the state $\bm{x}$. For example,
one could measure the average states of individual elements
$\left<x_{i}\right>_{\text{exp}}$ or the correlations among multiple elements
$\left<x_{i}x_{j}\right>_{\text{exp}}$,
$\left<x_{i}x_{j}x_{k}\right>_{\text{exp}}$, and so on. Given a set of $K$
observables $\mathcal{O}=\\{f_{\nu}(\bm{x})\\}$, where $\nu=1,\ldots,K$, the
most unbiased prediction for the distribution over states is the maximum
entropy model Shannon (1948); Jaynes (1957)

$P_{\mathcal{O}}(\bm{x})=\frac{1}{Z}\text{exp}\Bigg{[}-\sum_{\nu=1}^{K}\lambda_{\nu}f_{\nu}(\bm{x})\Bigg{]},$
(2)

where $Z$ is the normalizing partition function, and the parameters
$\lambda_{\nu}$ ensure that the model matches the experimental observations,
such that

$\left<f_{\nu}(\bm{x})\right>=\left<f_{\nu}(\bm{x})\right>_{\text{exp}}.$ (3)

To have control over errors in the $K$ expectation values, we must have $K\ll
MN$. But as experiments record from larger systems, one is confronted with an
explosion of possible observables. The total number of correlations grows
exponentially with $N$, and even the $K\propto N^{2}$ pairwise correlations
violate the good sampling condition as $N$ grows large. Thus, to avoid
sampling problems, one must focus on a sparse subset of correlations. Here we
arrive at the central question: Among a large set of observables, which should
we choose to include in a model?

###  II.2 Minimax entropy principle

Suppose we want to find the set of observables $\mathcal{O}$ that yields the
most accurate description of the system. We can choose $\mathcal{O}$ to
maximize the log–likelihood of the model $P_{\mathcal{O}}$ or, equivalently,
minimize the KL divergence with respect to the data
$D_{\text{KL}}(P_{\text{exp}}||P_{\mathcal{O}})$. Due to the form of
$P_{\mathcal{O}}$ in Eq. (2), the KL divergence simplifies to a difference in
entropies

$\displaystyle D_{\text{KL}}(P_{\text{exp}}||P_{\mathcal{O}})$
$\displaystyle=\left<\log\frac{P_{\text{exp}}(\bm{x})}{P_{\mathcal{O}}(\bm{x})}\right>_{\text{exp}}$
$\displaystyle=\log Z+\frac{1}{\ln
2}\sum_{\nu}\lambda_{\nu}\left<f_{\nu}(\bm{x})\right>_{\text{exp}}-S_{\text{exp}}$
$\displaystyle=S_{\mathcal{O}}-S_{\text{exp}},$ (4)

where the final equality follows from Eq. (3), and entropies $S$ are measured
in bits. We therefore find that the optimal observables $\mathcal{O}$ are the
ones that minimize the entropy of the maximum entropy model $S_{\mathcal{O}}$.
This is the “minimax entropy” principle, which was proposed 25 years ago but
has received relatively little attention Zhu  _et al._ (1997).

In addition to providing the best description of the data, the optimal
observables $\mathcal{O}$ can also be viewed as containing the maximum
information about the system. If we begin by observing each element
individually, then we only have access to the marginal distributions
$P_{i}(x_{i})$; in this case, the maximum entropy model is the independent
distribution $P_{\text{ind}}(\bm{x})=\prod_{i}P_{i}(x_{i})$ with entropy
$S_{\text{ind}}$. If, in addition to the marginals, we also observe some of
the correlations between elements Schneidman  _et al._ (2003), this knowledge
reduces our uncertainty about the system by an amount
$I_{\mathcal{O}}=S_{\text{ind}}-S_{\mathcal{O}}\geq 0$. Thus, by minimizing
$S_{\mathcal{O}}$, the optimal observables $\mathcal{O}$ also maximize the
information $I_{\mathcal{O}}$ contained in the observed correlations.

In practice, applying the minimax entropy principle poses two distinct
challenges. First, for each set of observables $\mathcal{O}=\\{f_{\nu}\\}$,
one must solve the traditional maximum entropy problem; that is, one must
compute the parameters $\lambda_{\nu}$ such that the model $P_{\mathcal{O}}$
matches the expectations $\left<f_{\nu}(\bm{x})\right>_{\text{exp}}$ in the
data. Second, one must repeat this calculation for all sets of observables
$\mathcal{O}$ to find the one that minimizes the entropy $S_{\mathcal{O}}$.
This search process is generally intractable. In what follows, we study a
class of observables that admits an exact and efficient solution, enabling
statistical physics models of very large systems.

###  II.3 Trees of pairwise correlations

For simplicity, we focus on binary variables $x_{i}=0,1$, for which the
marginals $P_{i}(x_{i})$ are defined by the averages $\left<x_{i}\right>$. In
the search for sources of order in a system, one might begin with the simplest
correlations: those between pairs of elements $\left<x_{i}x_{j}\right>$. In
populations of $N\sim 100$ neurons, one often has sufficient data to fit all
the pairwise correlations, which can be very effective in capturing key
features of the collective activity Schneidman  _et al._ (2006); Meshulam  _et
al._ (2017, 2021). But this corresponds to $K\propto N^{2}$ constraints, and
at large $N$ we will violate the good sampling condition $K\ll NM$. To avoid
undersampling, we are forced to select a sparse subset of pairwise
correlations, which can be visualized as a network $\mathcal{G}$ with edges
defining the observed correlations between variables. Each network induces a
maximum entropy model

$P_{\mathcal{G}}(\bm{x})=\frac{1}{Z}\text{exp}\Bigg{[}\sum_{(ij)\in\mathcal{G}}J_{ij}x_{i}x_{j}+\sum_{i}h_{i}x_{i}\Bigg{]},$
(5)

where the parameters $h_{i}$ and $J_{ij}$ enforce the constraints on
$\left<x_{i}\right>$ and $\left<x_{i}x_{j}\right>$ in $\mathcal{G}$,
respectively. The minimax entropy principle tells us that we should find the
network $\mathcal{G}$ (within some allowed set) that produces the maximum
entropy model $P_{\mathcal{G}}$ with minimum entropy $S_{\mathcal{G}}$.

In statistical physics, calculations are difficult in part due to feedback
loops. By eliminating loops, many statistical physics models become tractable,
as in one–dimensional systems or on Bethe lattices Sethna (2021). In the Ising
model—which is equivalent to Eq. (5)—if the interactions $J_{ij}$ lie on a
tree $\mathcal{T}$ (or a network without loops), then one can efficiently
compute the partition function $Z$ and all statistics of interest (see
Appendix A). Inverting this procedure, one can begin with the averages
$\left<x_{i}\right>$ and the correlations $\left<x_{i}x_{j}\right>$ on a tree
$\mathcal{T}$ and analytically derive the maximum entropy parameters Chow and
Liu (1968); Nguyen  _et al._ (2017):

$\displaystyle J_{ij}$
$\displaystyle=\ln\left[\frac{\left<x_{i}x_{j}\right>\left(1-\left<x_{i}\right>-\left<x_{j}\right>+\left<x_{i}x_{j}\right>\right)}{\left(\left<x_{i}\right>-\left<x_{i}x_{j}\right>\right)\left(\left<x_{j}\right>-\left<x_{i}x_{j}\right>\right)}\right],$
(6) $\displaystyle h_{i}$
$\displaystyle=\ln\frac{\left<x_{i}\right>}{1-\left<x_{i}\right>}$ (7)
$\displaystyle\quad\quad+\sum_{j\in\mathcal{N}_{i}}\ln\left[\frac{\left(1-\left<x_{i}\right>\right)\left(\left<x_{i}\right>-\left<x_{i}x_{j}\right>\right)}{\left<x_{i}\right>\left(1-\left<x_{i}\right>-\left<x_{j}\right>+\left<x_{i}x_{j}\right>\right)}\right],$

where $\mathcal{N}_{i}$ represents the neighbors of $i$ in $\mathcal{T}$ (see
Appendix B). Since each tree contains $N-1$ correlations, the total number of
observables is $K=2N-1$, and so we are well sampled if the number of
independent samples obeys $M\gg 2$.

Equations (6) and (7) solve the maximum entropy problem for the distribution
$P_{\mathcal{T}}$, but we still need to search over all of the $N^{N-2}$ trees
to find the one that minimizes the entropy $S_{\mathcal{T}}$. This search
simplifies significantly by noticing that the information $I_{\mathcal{T}}$
decomposes into a sum over the connections $(ij)$ in $\mathcal{T}$,

$I_{\mathcal{T}}=S_{\text{ind}}-S_{\mathcal{T}}=\sum_{(ij)\in\mathcal{T}}I_{ij},$
(8)

where $I_{ij}$ is the mutual information between $i$ and $j$ (see Appendix C)
Chow and Liu (1968); Nguyen  _et al._ (2017). Note that for pairs
$(ij)\in\mathcal{T}$, the mutual information $I_{ij}$ is the same in the model
and the data, so we can compute the entropy $S_{\mathcal{T}}$ directly from
the data without constructing the model itself.

Equation (8) tells us that the tree with the minimum entropy $S_{\mathcal{T}}$
is the one with the largest total mutual information. Identifying this optimal
tree is a minimum spanning tree problem Nguyen  _et al._ (2017), which can be
solved efficiently using a number of different algorithms Moore and Mertens
(2011). To begin, one computes the mutual information $I_{ij}$ between all
elements [Fig. 1(a)]. One can then grow the optimal tree by greedily
connecting the element $i$ in the tree to the new element $j$ with the largest
mutual information $I_{ij}$ [Fig. 1(b)]; this is Prim’s algorithm, which runs
in $O(N^{2})$ time [Fig. 1(c)]. Thus, by restricting to trees of pairwise
correlations, we can solve the minimax entropy problem exactly, even at very
large $N$.

Figure 1: Constructing the optimal tree. (a) Visualization of the mutual
information $I_{ij}$ (edges) between elements in a system (nodes), with
darker, thicker edges reflecting larger $I_{ij}$. (b) Illustration of Prim’s
algorithm. At each step, we consider the mutual information $I_{ij}$ between
elements in the tree and those not yet connected (matrix). We then connect the
two elements with the largest $I_{ij}$ (dashed edge) and repeat until all
elements have been added. (c) Optimal tree that maximizes minimizes
$S_{\mathcal{T}}$ and maximizes $I_{\mathcal{T}}$.

##  III Large–scale neuronal activity

We ultimately seek to explain the collective behaviors of very large networks.
However, each tree only contains a vanishingly small fraction $2/N$ of all
pairwise correlations; and even if we have access to all of the pairwise
statistics, there’s still no guarantee of success. Can such a sparse set of
observations capture something important about the system as a whole?

To answer this question, we consider patterns of electrical activity in
$N=1485$ neurons in the hippocampus of a mouse, recorded in a recent
experiment Gauthier and Tank (2018). Mice are genetically engineered so that
their neurons express a protein whose fluorescence is modulated by calcium
concentration, which in turn follows the electrical activity of the cells.
This fluorescence is recorded using a scanning two–photon microscope as the
mouse runs in a virtual environment. The signal from each cell consists of a
quiet background punctuated by short bursts of activity Meshulam  _et al._
(2017), providing a natural binarization into active ($x_{i}=1$) or silent
($x_{i}=0$) within each video frame [Fig. 2(a)]. Capturing images at
$30\,\text{Hz}$ for $39\,\text{min}$ yields $M\sim 7\times 10^{4}$ samples of
the collective state $\bm{x}=\\{x_{i}\\}$, but these are not all independent.
Nonetheless, we can still estimate the mutual information $I_{ij}$ with small
errors after correcting for finite data effects (see Appendix D).

Figure 2: Collective activity in a large population of neurons. (a) Time
series of neuronal activity in the mouse hippocampus, where each dot
represents an active neuron (see Gauthier and Tank (2018) for experimental
details). States $\bm{x}=\\{x_{i}\\}$ represent the population activity within
a single window of width $\Delta t=1/30\,\text{s}$. (b) Ranked order of
significant mutual information $I_{ij}$ in the population. Solid line and
shaded region reflect estimates and errors (two standard deviations) after
correcting for finite data (see Appendix D). (c) Distribution of correlation
coefficients over neuron pairs, with percentages indicating the fraction of
positively and negatively correlated pairs. (d) Mutual information $I_{ij}$
versus correlation coefficient, where each point represents a distinct neuron
pair. Estimates and errors are the same in (b). (e) Distribution of physical
distances between neurons. (f) Average mutual information $I_{ij}$ as a
function of physical distance, computed in bins that contain $500$ pairs each;
note that individual pairs vary widely around this average.

Among all $\sim 10^{6}$ pairs of neurons, only $9\%$ exhibit significant
mutual information with values shown in Fig. 2(b). We see that a small number
of pairs contain orders of magnitude more information than average
($\bar{I}=2.9\times 10^{-4}$ bits). This heavy–tailed distribution provides
hope for a tree of correlations that contains much more information than
typical $I_{\mathcal{T}}\gg(N-1)\bar{I}$. Additionally, while most pairs of
cells are negatively correlated [Fig. 2(c)], the strongest mutual information
corresponds to positive correlations [Fig. 2(d)]. And while most neurons are
far from one another [Fig. 2(e)], larger values of $I_{ij}$ are concentrated
among pairs of cells that are close to one another, as can be seen by plotting
the mean mutual information as a function of distance [Fig. 2(f)]. Together,
these observations suggest that a backbone of positively correlated and
physically proximate neurons may provide a large amount of information about
the collective neural activity.

##  IV Predictions of minimax entropy model

Constructing the minimax entropy tree (Fig. 1), we find that that it captures
$I_{\mathcal{T}}=26.2$ bits of information. This reduces our uncertainty about
the population activity by $I_{\mathcal{T}}/S_{\text{ind}}=14.4\%$, which is
equivalent to freezing the states of 214 randomly selected neurons. For
comparison, we consider two additional networks: (i) a random tree, which
represents a typical collection of correlations, and (ii) the tree of minimum
physical distances, which reflects the fact that neighboring neurons are more
likely to be strongly correlated [Fig. 2(f)]. The optimal tree captures over
twice as much information as the minimum distance tree and over 50 times more
than random.

Figure 3: Predicting pairwise statistics. (a) Ranked order of significant
mutual information in the population (black), two–standard–deviation errors
(shaded region), and prediction of the minimax entropy model (red). (b)
Correlation coefficients predicted in the model versus those in the data, with
dashed lines indicating equality. All pairs are divided evenly into bins along
the x–axis, with solid lines and shaded regions reflecting means and errors
(standard deviations) within bins.

While each model $P_{\mathcal{T}}$ is defined to match a sparse subset of the
observed correlations $\left<x_{i}x_{j}\right>$ (and thus mutual information
$I_{ij}$) for $(ij)\in\mathcal{T}$, we can ask what $P_{\mathcal{T}}$ predicts
for all pairs of neurons (see Appendix E). We note that the optimal tree does
not simply match the largest $N-1$ values of $I_{ij}$; in general, these will
form loops. Yet we find that the minimax entropy model still predicts the
distribution of $I_{ij}$ within experimental error for the top $\sim N$ values
[Fig. 3(a)]. Indeed, we find that the model captures the strong correlations
in the population [Fig. 3(b)]; this accuracy decreases significantly for the
minimum distance and random trees (see Appendix F). As expected, the optimal
tree underpredicts the strengths of weak and negative correlations [Fig.
3(b)]. Although these correlations may seem unimportant individually, we note
that they comprise the vast majority of neuron pairs [Fig. 2(c)].

Figure 4: Predicting synchronized activity. Distribution $P(K)$ of the number
of simultaneously active neurons $K$ in the data (black), predicted by the
minimax entropy tree (red), and the Gaussian distribution for independent
neurons with mean and variance $\langle K\rangle_{\text{exp}}$ (dashed). We
confirm that the Gaussian matches the data after shuffling the activity of
each neuron in time. To estimate $P(K)$ and error bars (two standard
deviations), we first split the data into 1–minute blocks to preserve
dependencies between consecutive samples. We then select one third of these
blocks at random and repeat 100 times. For each subsample, we compute the
optimal tree $\mathcal{T}$ and predict $P(K)$ using a Monte Carlo simulation
of the model $P_{\mathcal{T}}$.

With knowledge of only $2/N\sim 0.1\%$ of the pairwise correlations, can the
optimal tree capture collective behavior in the system? In neuronal
populations (and other complex systems), one key collective property is
synchronized activity Schneidman  _et al._ (2006); Tkačik  _et al._ (2015);
Meshulam  _et al._ (2021); Lynn  _et al._ (2019), which is characterized by
the probability $P(K)$ that $K$ out of the $N$ neurons are simultaneous
active. If the neurons were independent, this distribution would be
approximately Gaussian at large $N$ (Fig. 4, dashed). But in real populations,
the dependencies among neurons leads to a much broader distribution (Fig. 4,
black), with moments of extreme synchrony in both activity (large $K$) and
silence (small $K$). If one builds a model from pairwise correlations chosen
at random, then the distribution $P(K)$ is almost indistinguishable from that
of an independent system (see Appendix F). By contrast, the optimal tree
captures most of this collective behavior Lynn  _et al._ (2023), correctly
predicting $\gtrsim 100$–fold increases in the probabilities that $K\gtrsim
50$ neurons are active in the same small time bin (Fig. 4, red). Although the
detailed patterns of activity in the system are shaped by competing
interactions that are missing from our optimal tree, this shows that
large–scale synchrony can emerge from a sparse network of the strongest
correlations.

##  V Structure of minimax entropy model

###  V.1 Ising structure

Figure 5: Maximum entropy models of large–scale activity. (a,b) Distributions
of Ising interactions $J^{\text{I}}_{ij}$ (a) and local fields
$h^{\text{I}}_{i}$ (b) in the optimal tree (red), the minimum distance tree
(blue), and a random tree (cyan). (c) Average activities
$\left<\sigma_{i}\right>$ versus local fields $h^{\text{I}}_{i}$, where each
point represents an individual neuron. Dashed line illustrates the independent
prediction $\left<\sigma_{i}\right>=\tanh h^{\text{I}}_{i}$. (d) Average
interaction fields
$h^{\text{int}}_{i}=\sum_{j}J^{\text{I}}_{ij}\left<\sigma_{j}\right>$ versus
local fields $h^{\text{I}}_{i}$. Percentages indicate the proportion of
neurons for which $h_{i}^{\text{int}}<h_{i}^{\text{I}}$ (dashed line indicates
equality).
Figure 6: Network structure of optimal tree. (a) Illustrations of the minimax
entropy tree (left), the minimum distance tree (middle), and a random tree
(right). In all networks, the central neuron has the largest degree (number of
connections), and those with one connection are located on the perimeter with
distance from the central neuron decreasing in the clockwise direction. (b)
Distributions of degrees in the optimal and minimum distance trees. Dashed
line indicates the Poisson distribution of random trees. (c) Distributions of
physical distances among the connections in each tree. (d) Distributions of
topological distances (or the lengths of shortest paths) across all pairs of
neurons in each tree.

To understand the nature of the optimal tree, we can study the minimax entropy
model $P_{\mathcal{T}}$ itself, which, as discussed above, is equivalent to an
Ising model from statistical physics. This mapping is made concrete by
considering a system of spins $\sigma_{i}=2x_{i}-1\in\pm 1$ with Ising
interactions $J^{\text{I}}_{ij}=J_{ij}/4$ and local fields
$h^{\text{I}}_{i}=h_{i}/2+\sum_{j}J^{\text{I}}_{ij}$, where $J_{ij}$ and
$h_{i}$ are defined in Eqs. (6) and (7). If the interaction
$J^{\text{I}}_{ij}$ is positive (negative), then activity in neuron $i$ leads
to activity (silence) in neuron $j$, and vice versa. For random trees, the
interactions $J^{\text{I}}_{ij}$ are nearly evenly split between positive and
negative [Fig. 5(a)]; this is consistent with previous investigations of
fully–connected models in populations of $N\sim 100$ neurons Schneidman  _et
al._ (2006); Tkačik  _et al._ (2015); Meshulam  _et al._ (2021). Meanwhile, we
recall that the largest mutual information in the population belongs to
positively correlated neurons [Fig. 2(d)]. Accordingly, the optimal tree has
interactions that are almost exclusively positive [Fig. 5(a)]. We have
arrived, perhaps surprisingly, at a traditional Ising ferromagnet.

While the interactions $J^{\text{I}}_{ij}$ define effective influences between
neurons, the local fields represent individual biases toward activity
($h^{\text{I}}_{i}>0$) or silence ($h^{\text{I}}_{i}<0$). For random trees,
all of the local fields are negative [Fig. 5(b)], reflecting the fact that
neurons are more likely to be silent than active. But in the optimal tree, we
see that some neurons are counterintuitively biased toward activity with
$h^{\text{I}}_{i}>0$ [Fig. 5(b)]. These positive biases stand in competition
with the positive interactions in the model, which, because neurons favor
silence, tend to induce silence in the population.

To understand the effects of interactions on individual cells, we note that
the average activity of an independent neuron $i$ is fully defined by
$h^{\text{I}}_{i}$ through the relation $\left<\sigma_{i}\right>=\tanh
h^{\text{I}}_{i}$. Since random trees contain only weak correlations, the
neuronal activity closely tracks this independent prediction [Fig. 5(c)]. As
interactions become increasingly positive in the minimum distance and optimal
trees, the alignment of neighboring neurons produces average activities that
are significantly lower than one would expect from local fields alone [Fig.
5(c)]. For each neuron $i$, the competition between internal biases and
interactions is made clear by comparing the local field $h^{\text{I}}_{i}$ to
the average influence due to interactions
$h^{\text{int}}_{i}=\sum_{j}J^{\text{I}}_{ij}\left<\sigma_{j}\right>$. In
random trees, only $1\%$ of neurons are dominated by interactions, such that
$h^{\text{int}}_{i}<h^{\text{I}}_{i}$ [Fig. 5(d)]; this proportion increases
to $43\%$ in the minimum distance tree and $57\%$ in the optimal tree [Fig.
5(d)]. So despite the fact that the tree structure constrains each cell to
only interact with two others in the entire population (on average), most
neurons in the optimal tree are driven more strongly by interactions than
internal biases.

###  V.2 Network structure

In addition to the functional properties of the model $P_{\mathcal{T}}$, we
can also study the graph structure of the optimal tree $\mathcal{T}$. To
visualize each tree, we place the cell with the most connections (or largest
degree $k_{i}=|\mathcal{N}_{i}|$) at the center and all of the cells with
single connections ($k_{i}=1$) around the perimeter [Fig. 6(a)]. For random
trees, the distribution of degrees is Poisson [Fig. 6(b)], preventing the
emergence of high–degree hub nodes. Degrees are even more sharply peaked in
the minimum distance tree, such that we do not observe a single neuron with
more than four connections [Fig. 6(b)]. By contrast, the optimal tree has a
much broader degree distribution, with a central neuron that connects to 29
other cells in the population [Fig. 6(b)]. Such hub nodes are frequently
observed in the brain’s physical connectivity Song  _et al._ (2005); Lynn and
Bassett (2019); Lin  _et al._ (2023), and are thought to play an important
role in facilitating communication Albert and Barabási (2002).

By maximizing information about the population, one might hope that the
optimal tree captures features of the true interactions between neurons. In
the brain, demands on communication are constrained by energetic costs Harris
_et al._ (2012). Networks have evolved to balance efficient communication
(minimizing the number of steps between cells in the network, known as
topological distance) with energetic efficiency (minimizing the physical
lengths of connections) Laughlin and Sejnowski (2003); Lynn and Bassett
(2019). These pressures are in direct competition: Networks with physically
local connections form lattice–like structures with long topological
distances, and networks with short topological distances (known as the
small–world property Watts and Strogatz (1998)) require physically long–range
connections. Indeed, in the minimum distance tree, which is composed of the
physically shortest connections [Fig. 6(c)], communication between two neurons
requires $\sim 100$ intermediate cells on average [Fig. 6(d)]; and random
trees, which are known to produce short topological distances [Fig. 6(d)], are
mostly composed of long–range connections [Fig. 6(c)]. Meanwhile, the minimax
entropy model identifies connections that are much shorter than average [Fig.
6(c)] while simultaneously maintaining small–world structure [Fig. 6(d)], just
as observed in real neuronal networks Laughlin and Sejnowski (2003).

##  VI Scaling with population size

Thus far, we have focused on a single population of $N\sim 1500$ neurons. But
as experiments advance to record from even larger populations, how does the
minimax entropy model scale with $N$? To answer this question, in the spirit
of Ref. Meshulam  _et al._ (2021) one can imagine growing a contiguous
population centered at a single neuron [Fig. 7(a)], and computing the optimal
tree for increasing population sizes. Due to the efficiency of our model, we
can repeat this process starting from each of the different neurons and
average over the results. As the population grows, the independent entropy
$S_{\text{ind}}$ must increase extensively (that is, linearly with $N$) on
average [Fig. 7(b)]. Since each tree contains $N-1$ correlations, one might
also expect the information $I_{\mathcal{T}}$ of any tree to scale
extensively. However, we find that the scaling of $I_{\mathcal{T}}$ with
population size depends critically on which correlations we use in building
the tree [Fig. 7(b)].

Figure 7: Scaling of the minimax entropy model. (a) Illustration of our growth
process superimposed on a fluorescence image of the $N=1485$ neurons in the
mouse hippocampus. Starting with a single neuron $i$, we grow the population
of $N$ neurons closest to $i$ (red to yellow). We then repeat this process for
each neuron and average the results. (b) Information $I_{\mathcal{T}}$
captured by different trees as a function of population size $N$. Dashed line
indicates the independent entropy $S_{\text{ind}}$. (c) Fraction of the
independent entropy $I_{\mathcal{T}}/S_{\text{ind}}$ explained by different
trees as a function of $N$. (d) Average Ising local fields $h^{\text{I}}_{i}$
(solid) and interaction fields $h^{\text{int}}_{i}$ (dashed) for the maximum
entropy models on different trees $P_{\mathcal{T}}$.

If the information $I_{\mathcal{T}}$ grows extensively, then the model
$P_{\mathcal{T}}$ explains a constant proportion of the independent entropy
$I_{\mathcal{T}}/S_{\text{ind}}$ across different population sizes. Indeed,
because the properties of the closest neurons do not change as the population
grows (on average), the minimum distance tree captures a nearly constant $\sim
6\%$ of the independent entropy [Fig. 7(c)]. By contrast, since the mutual
information $I_{ij}$ between neurons tends to decrease with physical distance
[Fig. 2(f)], the average mutual information $\bar{I}$ in a spatially
contiguous population decreases with $N$. Thus, the typical information in a
random tree grows subextensively with the population size [Fig. 7(b)], and the
fractional information $I_{\mathcal{T}}/S_{\text{ind}}$ vanishes [Fig. 7(c)].

But even though the average mutual information $\bar{I}$ decreases, as the
population grows we uncover more of the exceptionally large mutual information
$I_{ij}$ in the tail of the distribution [Fig. 2(b)]. By identifying these
highly informative correlations, the optimal tree accumulates a superextensive
amount of information $I_{\mathcal{T}}$ [Fig. 7(b)], thus capturing a greater
proportion of the independent entropy as $N$ increases [Fig. 7(c)]. This
increased explanatory power is underpinned by stronger interactions and weaker
local fields in the Ising network [Fig. 7(d)]. There is no sign that the trend
in Fig. 7(c) is saturating at $N\sim 10^{3}$, suggesting that our minimax
entropy framework may become even more effective for larger populations.

##  VII Thermodynamics and signatures of criticality

Figure 8: Thermodynamics of the minimax entropy model. (a) Average activity
$\frac{1}{N}\sum_{i}\left<x_{i}\right>_{T}$ as a function of temperature $T$
in the optimal tree $\mathcal{T}$ for populations of increasing size $N$ and
for independent neurons (dashed). (b) Normalized susceptibility $\chi/N$
versus temperature $T$ for the same models as (a). Inset shows the increase in
$\chi/N$ with population size $N$ for the true minimax entropy models ($T=1$).
(c) Normalized susceptibility versus $T$ for the minimum distance tree (blue)
and random trees (cyan) across different population sizes. (d) Maximum value
of the normalized susceptibility $\chi/N$ (top) and peak temperature $T$
(bottom) as functions of the population size $N$ for the optimal tree (red),
the minimum distance tree (blue), random trees (cyan), and independent neurons
(dashed). (e,f) Specific heat $C/N$ (e) and information fraction
$I_{\mathcal{T}}/S_{\text{ind}}$ (f) versus temperature $T$ for the minimax
entropy models in (a) and (b). Inset in (e) shows the increase in $C/N$ with
population size $N$ for the true minimax entropy models ($T=1$). In all
panels, darker lines reflect populations of increasing size $N$, constructed
using the method in Fig. 7(a) and averaged over 100 random initializations. In
(a)–(e), dashed lines represent independent neurons.

As discussed above, each tree of observed correlations $\mathcal{T}$ generates
a maximum entropy model $P_{\mathcal{T}}$ [Eq. (5)], which in turn is
equivalent to a system of Ising spins. This mapping from experimental
observations to statistical physics gives us the opportunity to ask whether
the model $P_{\mathcal{T}}$ occupies a special place in the space of possible
models. In statistical mechanics, equilibrium systems are described by the
Boltzmann distribution,

$P(\bm{x})=\frac{1}{Z}\exp\left[-\frac{1}{T}E(\bm{x})\right],$ (9)

where $T$ is the temperature of the system and $E(\bm{x})$ is the Hamiltonian,
which defines the energy of state $\bm{x}$. For a given tree $\mathcal{T}$, we
notice that $P_{\mathcal{T}}$ defines a Boltzmann distribution with
temperature $T=1$ and energy

$E(\bm{x})=-\sum_{(ij)\in\mathcal{T}}J_{ij}x_{i}x_{j}-\sum_{i}h_{i}x_{i},$
(10)

where $J_{ij}$ and $h_{i}$ are defined in Eqs. (6) and (7). Note that we do
not assume the experimental system itself is in equilibrium; this
correspondence is purely mathematical.

By perturbing the temperature away from $T=1$, we can probe at least one slice
through the space of possible networks Tkačik  _et al._ (2015). For each value
of $T$, we arrive at a hypothetical system $P_{\mathcal{T}}(\bm{x};T)$ with
average activities $\left<x_{i}\right>_{T}$ and correlations
$\left<x_{i}x_{j}\right>_{T}$ that are no longer constrained to match
experimental observations. Consider the minimax entropy tree, which (as
discussed in §V) produces a ferromagnetic Ising model with nearly all positive
interactions $J^{\text{I}}_{ij}$ [Fig. 5(a)]. At high temperatures $T\gg 1$,
fluctuations destroy the preference for silence over activity, and the system
approaches the average activity
$\frac{1}{N}\sum_{i}\left<x_{i}\right>_{T}=0.5$ [Fig. 8(a)]. Meanwhile, at low
temperatures $T\ll 1$, activity vanishes as the network freezes into the
all–silent ground state $\bm{x}=0$ [Fig. 8(a)]. In both limits, all of the
information contained in correlations is lost.

As the temperature decreases, most systems experience a gradual transition
from disorder to order. But for certain combinations of parameters $J_{ij}$
and $h_{i}$, a small change in the temperature $T$ can lead to a large change
in the behavior of the system, and as $N$ becomes large, this transition
becomes sharp Sethna (2021); Tkačik  _et al._ (2015); Schnabel  _et al._
(2011). Such phase transitions mark a critical point in the space of possible
systems, with the Ising ferromagnet as the canonical example Peierls (1936).
In the optimal tree, as the temperature increases just above $T=1$, the
positive interactions lead to a much steeper increase in activity than an
independent system; and this transition grows even sharper for larger
populations [Fig. 8(a)].

We emphasize that at any finite $N$ there is no true critical point, but
$N\sim 1000$ may be large enough that the idealization $N\rightarrow\infty$ is
useful. Since we are studying models defined on trees, there are also
subtleties about how one would construct the thermodynamic limit, since such a
large fraction of sites are on the boundary Baxter (2016). For our purposes,
the interesting question is whether real networks of neurons are in any sense
at special points in the space of possible networks. One way in which this
could happen is if parameters are set so that simple macroscopic quantities
have near–extremal values.

One example of a macroscopic quantity that provides a global measure of
collective behavior is the total susceptibility of the mean activity to
changes in the bias fields,

$\chi=\sum_{ij}\frac{d\left<x_{i}\right>_{T}}{dh_{j}}=\frac{1}{T}\sum_{ij}\left(\left<x_{i}x_{j}\right>_{T}-\left<x_{i}\right>_{T}\left<x_{j}\right>_{T}\right),$
(11)

where the rewriting in terms of connected correlations can be derived from the
Boltzmann distribution [Eq. (9)]. We recall that at conventional critical
points we would see a divergence of $\chi/N$ as $N\rightarrow\infty$. At both
high and low temperatures, correlations are destroyed, and the susceptibility
vanishes [Fig. 8(b)]. However, at intermediate temperatures, the
susceptibility exhibits a peak that becomes sharper as the system grows, even
after normalizing by the population size $N$ [Fig. 8(b)]. Moreover, as $N$
increases, the peak temperature decreases toward $T=1$, corresponding to the
true minimax entropy model $P_{\mathcal{T}}$. By contrast, the minimum
distance and random trees undergo smooth transitions from disorder to order
[Fig. 8(c)], with the maximum susceptibility and peak temperatures remaining
approximately constant across all population sizes $N$ [Fig. 8(d)].

In addition to the susceptibility $\chi$, we also observe a dramatic peak in
the specific heat $C/N$ [Fig. 8(e)], where

$C=\frac{d\left<E(\bm{x})\right>_{T}}{dT}$ (12)

is the heat capacity (see Appendix G). Although there is no meaning to “heat”
in this system, because the specific heat is related to the variance in
energy, we can think of the peak in $C$ as being a peak in the dynamic range
of (log) probabilities across the states of the network. These divergences in
the susceptibility and heat capacity also align with a sharp peak in the
information fraction $I_{\mathcal{T}}/S_{\text{ind}}$ [Fig. 8(f)], with larger
systems becoming even more strongly correlated. Together, these results
indicate that the true minimax entropy model $P_{\mathcal{T}}$ is poised near
a special point in the space of models $P_{\mathcal{T}}(\bm{x};T)$, where
small changes in parameter values can produce large changes in the collective
behavior of the system.

##  VIII Conclusions

The maximum entropy principle provides the most unbiased mapping from
experimental observations to statistical physics models. Over the past two
decades, this link has proven useful in understanding the emergence of
collective behaviors in populations of neurons and other complex living
systems Schneidman  _et al._ (2006); Nguyen  _et al._ (2017); Meshulam  _et
al._ (2017, 2021); Tkačik  _et al._ (2015); Lezon  _et al._ (2006); Weigt  _et
al._ (2009); Marks  _et al._ (2011); Lapedes  _et al._ (2012); Bialek  _et
al._ (2012); Russ  _et al._ (2020); Lynn  _et al._ (2019). Less widely
emphasized is the fact that there is not a single maximum entropy model, but
rather a landscape of possible models depending on what features of the system
we choose to constrain. Quite generally, we should choose the features that
are most informative—the ones that minimize the entropy of the maximum entropy
model—leading to the minimax entropy principle Zhu  _et al._ (1997). As
experiments record from larger and larger populations of neurons Segev  _et
al._ (2004); Litke  _et al._ (2004); Chung  _et al._ (2019); Dombeck  _et al._
(2010); Tian  _et al._ (2012); Demas  _et al._ (2021); Steinmetz  _et al._
(2021), we enter an undersampled regime in which selecting a limited number of
maximally informative features is not only conceptually appealing, but also
practically necessary.

While the minimax entropy problem is generally intractable, here we make
progress in two steps. First, we build upon previous work by constraining mean
activities and pairwise correlations, resulting in models that are equivalent
to systems of Ising spins. Second, taking inspiration from the Bethe lattice,
we focus only on trees of correlations, or sparse networks without loops.
Under these restrictions, we solve the minimax entropy problem exactly,
identifying the optimal tree in quadratic time Chow and Liu (1968); Nguyen
_et al._ (2017). The result is a non–trivial family of statistical physics
models that can be constructed very efficiently for large neuronal
populations.

It is far from obvious that these models can capture any of the essential
collective behavior in real networks. To answer this question, we study a
population of $N\sim 1500$ neurons in the mouse hippocampus Gauthier and Tank
(2018), identifying the maximally informative tree of pairwise correlations
(Figs. 5 and 6). Despite containing only one correlation per neuron, this
minimax entropy model accounts for $14\%$ of the independent entropy (over
$50$ times more than random trees) and predicts the distribution of
large–scale synchrony in activity (Fig. 4). Moreover, the model becomes more
effective as the population grows (Fig. 7) and exhibits hints of critical
behavior (Fig. 8). The success of such a sparse model hinges on the fact that
the distribution of mutual information between neurons is heavy–tailed [Fig.
2(b)], such that a few rare correlations carry much more information than
average. In fact, the physical connections between neurons are now understood
to be heavy–tailed across a range of animals Lynn  _et al._ (2022), suggesting
that our approach my prove effective in other neural systems. While these
minimax entropy models cannot capture all of a system’s collective properties,
they provide at least a starting point for simplified descriptions of the much
larger systems becoming accessible in modern experiments.

###### Acknowledgements.

We thank L. Meshulam and J.L. Gauthier for guiding us through the data of Ref.
Gauthier and Tank (2018), and C.M. Holmes and D.J. Schwab for helpful
discussions. This work was supported in part by the National Science
Foundation through the Center for the Physics of Biological Function
(PHY–1734030), by the National Institutes of Health through the BRAIN
initiative (R01EB026943), by the James S McDonnell Foundation through a
Postdoctoral Fellowship Award (C.W.L.), and by Fellowships from the Simons
Foundation and the John Simon Guggenheim Memorial Foundation (W.B.).

##  Appendix A Ising calculations on a tree

To establish notation, we begin by reviewing well known ideas about
statistical mechanics for models without loops. We then proceed, here and in
subsequent Appendices, to technical points needed for the main text.

###  A.1 Partition function

Consider a system of $N$ binary variables $x_{i}\in\\{0,1\\}$,
$i=1,2,\ldots,N$, defined by fields $h_{i}$ and interactions $J_{ij}$ that lie
on a tree $\mathcal{T}$. The Boltzmann distribution [Eq. (5)] takes the form

$P_{\mathcal{T}}(\bm{x})=\frac{1}{Z}\text{exp}\Bigg{[}\sum_{(ij)\in\mathcal{T}}J_{ij}x_{i}x_{j}+\sum_{i}h_{i}x_{i}-F\Bigg{]},$
(13)

where $F=0$ is the zero–point energy, which will become useful. To begin, we
seek to compute the partition function,

$Z=\sum_{\bm{x}}\text{exp}\Bigg{[}\sum_{(ij)\in\mathcal{T}}J_{ij}x_{i}x_{j}+\sum_{i}h_{i}x_{i}-F\Bigg{]}.$
(14)

To do so, imagine summing over one variable, and finding a new system of $N-1$
variables with the same partition function $Z$. If we can repeat this process
until no variables remain, then computing $Z$ will be trivial.

We label the nodes $i$ based on the order that they are removed, and we let
$h^{(i)}_{i}$ and $F^{(i)}$ denote the updated parameters at step $i$, while
the interactions $J_{ij}$ stay fixed. Consider summing over a variable $i$
with only one connection in the network, say to variable $j$. We note that
such a node is always guaranteed to exist in a tree. To keep the partition
function fixed, the new system with $i$ removed must satisfy the equations

$e^{h_{j}^{(i)}x_{j}-F^{(i)}}\big{(}e^{J_{ij}x_{j}+h^{(i)}_{i}}+1\big{)}=e^{h_{j}^{(i+1)}x_{j}-F^{(i+1)}}.$
(15)

This is a system of two equations (one for each value of $x_{j}$), which we
can solve for the new parameters

$\displaystyle h_{j}^{(i+1)}$
$\displaystyle=h_{j}^{(i)}+\ln[\frac{e^{J_{ij}+h_{i}^{(i)}}+1}{e^{h_{i}^{(i)}}+1}\Bigg{]},$
(16) $\displaystyle F^{(i+1)}$
$\displaystyle=F^{(i)}-\ln[e^{h_{i}^{(i)}}+1\big{]}.$ (17)

After removing $i$, the new system still forms a tree, so we can repeat the
above procedure. When all nodes have been removed, we are left with a single
parameter $\mathcal{F}=F^{(N+1)}$, which is the free energy of the system, and
the partition function is given by

$Z=e^{-\mathcal{F}}.$ (18)

###  A.2 Average activities and correlations

To compute population statistics, one simply needs to take derivatives of the
partition function,

$\displaystyle\langle x_{i}\rangle$ $\displaystyle=\frac{d\ln
Z}{dh_{i}}=-\frac{d\mathcal{F}}{dh_{i}},$ (19) $\displaystyle\langle
x_{i}x_{j}\rangle$ $\displaystyle=\frac{d\ln
Z}{dJ_{ij}}=-\frac{d\mathcal{F}}{dJ_{ij}},$ (20)

where $\frac{d}{dh_{i}}$ and $\frac{d}{dJ_{ij}}$ represent total derivatives,
which account for indirect dependencies via Eqs. (16) and (17). Since
$\frac{d\mathcal{F}}{dF^{(i+1)}}=1$ and $\frac{dh_{i}^{(i)}}{dh_{i}}=1$, the
above procedure yields

$\langle x_{i}\rangle=-\frac{\partial F^{(i+1)}}{\partial
h^{(i)}_{i}}-\frac{d\mathcal{F}}{dh^{(i+1)}_{j}}\frac{\partial
h^{(i+1)}_{j}}{\partial h^{(i)}_{i}}.$ (21)

Noticing that

$-\frac{d\mathcal{F}}{dh^{(i+1)}_{j}}=-\frac{d\mathcal{F}}{dh_{j}}=\langle
x_{j}\rangle,$ (22)

and taking derivatives of Eqs. (16) and (17), we have

$\displaystyle\langle x_{i}\rangle$
$\displaystyle=\frac{1}{1+e^{-h_{i}^{(i)}}}$ (23)
$\displaystyle\quad\quad+\langle
x_{j}\rangle\Bigg{(}\frac{1}{1+e^{-J_{ij}-h_{i}^{(i)}}}-\frac{1}{1+e^{-h_{i}^{(i)}}}\Bigg{)}.$

The correlation follows analogously,

$\langle x_{i}x_{j}\rangle=-\frac{d\mathcal{F}}{dh^{(i+1)}_{j}}\frac{\partial
h^{(i+1)}_{j}}{\partial J_{ij}}=\frac{\langle
x_{j}\rangle}{1+e^{-J_{ij}-h_{i}^{(i)}}}.$ (24)

Thus, by proceeding in the opposite order from which the nodes were removed,
we can compute the average activities $\langle x_{i}\rangle$ and correlations
$\langle x_{i}x_{j}\rangle$ for $(ij)\in\mathcal{T}$. For the correlations
$\langle x_{i}x_{j}\rangle$ off the tree (that is, for
$(ij)\not\in\mathcal{T}$), see Appendix E.

##  Appendix B Maximum entropy on a tree

We now solve the inverse problem for the parameters $h_{i}$ and $J_{ij}$ given
the observations $\langle x_{i}\rangle$ and $\langle x_{i}x_{j}\rangle$ on a
tree $\mathcal{T}$. Inverting Eqs. (23) and (24) yields

$\displaystyle h_{i}^{(i)}$ $\displaystyle=\ln\left[\frac{\langle
x_{i}\rangle-\langle x_{i}x_{j}\rangle}{1-\langle x_{i}\rangle-\langle
x_{j}\rangle+\langle x_{i}x_{j}\rangle}\right]$ (25) $\displaystyle J_{ij}$
$\displaystyle=\ln\left[\frac{\langle x_{i}x_{j}\rangle}{\langle
x_{j}\rangle-\langle x_{i}x_{j}\rangle}\right]-h_{i}^{(i)}.$ (26)

Combining the above equations, we can solve for the interaction $J_{ij}$ in
Eq. (6). To compute the local field $h_{i}$, we note that we can repeat the
procedure in Appendix A ending at any node; this is equivalent to choosing the
root of the tree. If we choose $i$ to be the final node, then we have

$h_{i}^{(N)}=\ln\frac{\langle x_{i}\rangle}{1-\langle x_{i}\rangle}.$ (27)

Additionally, for each neighbor $j\in\mathcal{N}_{i}$, Eq. (16) tells us that
we receive a contribution to $h_{i}^{(N)}$ of the form

$\displaystyle h_{i}^{(j+1)}-h_{i}^{(j)}$
$\displaystyle=\ln[\frac{e^{J_{ij}+h_{j}^{(j)}}+1}{e^{h_{j}^{(j)}}+1}\Bigg{]}$
(28) $\displaystyle=\ln\left[\frac{\langle x_{i}\rangle\left(1-\langle
x_{i}\rangle-\langle x_{j}\rangle+\langle
x_{i}x_{j}\rangle\right)}{\left(1-\langle x_{i}\rangle\right)\left(\langle
x_{i}\rangle-\langle x_{i}x_{j}\rangle\right)}\right].$

Combining these contributions yields

$\displaystyle h_{i}$
$\displaystyle=h_{i}^{(N)}-\sum_{j\in\mathcal{N}_{i}}\big{(}h_{i}^{(j+1)}-h_{i}^{(j)}\big{)}$
(29) $\displaystyle=\ln\frac{\langle x_{i}\rangle}{1-\langle x_{i}\rangle}$
(30)
$\displaystyle\quad\quad+\sum_{j\in\mathcal{N}_{i}}\ln\left[\frac{\left(1-\langle
x_{i}\rangle\right)\left(\langle x_{i}\rangle-\langle
x_{i}x_{j}\rangle\right)}{\langle x_{i}\rangle\left(1-\langle
x_{i}\rangle-\langle x_{j}\rangle+\langle x_{i}x_{j}\rangle\right)}\right].$

We have thus arrived at an analytic solution to the maximum entropy problem on
a tree.

##  Appendix C Information in a tree of correlations

Our ability to efficiently construct the optimal tree $\mathcal{T}$ depends
critically on the decomposition of the information $I_{\mathcal{T}}$ into the
sum of mutual information $I_{ij}$ over pairs $(ij)\in\mathcal{T}$ [Eq. (8)].
To derive this result, we note that for each connection $(ij)\in\mathcal{T}$,
the observables $\langle x_{i}\rangle$, $\langle x_{j}\rangle$, and $\langle
x_{i}x_{j}\rangle$ fully define the marginal distribution
$P_{ij}(x_{i},x_{j})$. Now consider a new tree
$\mathcal{T^{\prime}}=\mathcal{T}/(ij)$ with the connection $(ij)$ removed,
such that we do not observe $\langle x_{i}x_{j}\rangle$. Since $\mathcal{T}$
has no loops, after removing $(ij)$ the two elements $i$ and $j$ become
independent. Meanwhile, the dependence of the rest of the system on $i$ and
$j$ remains fixed. Thus, observing the correlation $\langle x_{i}x_{j}\rangle$
leads to a drop in entropy

$S_{\mathcal{T^{\prime}}}-S_{\mathcal{T}}=S(P_{i})+S(P_{j})-S(P_{ij})=I_{ij},$
(31)

where $I_{ij}$ is the observed mutual information between $i$ and $j$.
Repeating the above argument for every correlation in $\mathcal{T}$, we arrive
at Eq. (8).

##  Appendix D Estimating mutual information

In order to estimate the mutual information between neurons $I_{ij}$, one must
correct for finite–data effects Strong  _et al._ (1998). To do so, we
subsample the data hierarchically for different data fractions
$\\{1,0.9,\ldots,0.2,0.1\\}$, such that each subsample is contained within the
larger subsamples. Additionally, to preserve the dependencies between
consecutive data points $\bm{x}^{(m)}$ and $\bm{x}^{(m+1)}$, we sample
temporally contiguous fractions of the data. To ensure that each point is
sampled with equal probability, we allow subsamples that span the beginning
and end of the recording.

For each subsample, we estimate the mutual information between neurons $i$ and
$j$ using the equation

$I_{ij}=\sum_{x_{i},x_{j}}\tilde{P}_{ij}(x_{i},x_{j})\log\frac{\tilde{P}_{ij}(x_{i},x_{j})}{\tilde{P}_{i}(x_{i})\tilde{P}_{j}(x_{j})},$
(32)

where

$\displaystyle\tilde{P}_{ij}(x_{i},x_{j})$
$\displaystyle=\frac{1}{M+1}\Big{(}1+\sum_{m=1}^{M}\delta_{x_{i},x_{i}^{(m)}}\delta_{x_{j},x_{j}^{(m)}}\Big{)},$
(33) $\displaystyle\tilde{P}_{i}(x_{i})$
$\displaystyle=\sum_{x_{j}}\tilde{P}_{ij}(x_{i},x_{j}).$ (34)

The pseudo–counts in Eq. (33) ensure that the mutual information estimates do
not diverge. After estimating $I_{ij}$ for each data fraction, following Ref.
Strong  _et al._ (1998) we extrapolate to the infinite–data limit using a
linear fit with respect to the inverse data fraction. Repeating this process
100 times, we arrive at a distribution of infinite–data estimates for
$I_{ij}$, from which we can compute a mean and standard deviation (Fig. 9). To
check the above procedure, we note that shuffling the activity of each neuron
in time should destroy the mutual information $I_{ij}$. Indeed, for
time–shuffled data, we do not observe a single significant mutual information
in the population.

Figure 9: Correcting for finite–data effects on mutual information. For a
given pair of neurons $i$ and $j$, we plot the estimated mutual information
$I_{ij}$ [Eq. (32)] versus the inverse data fraction for individual subsamples
of the data (grey). Repeating 100 times, we plot the average $I_{ij}$ for each
data fraction (black), a linear fit (dashed line), and the infinite–data
estimate (red). Data points and error bars reflect means and standard
deviations over the 100 repetitions.

##  Appendix E Computing all correlations

Given a maximum entropy model with parameters $h_{i}$ and $J_{ij}$ on a tree
$\mathcal{T}$, in Appendix A we showed how to compute the averages $\langle
x_{i}\rangle$ and correlations $\langle x_{i}x_{j}\rangle$ on the tree.
Specifically, we computed the partition function $Z$ by summing over variables
$x_{i}$ in the order $i=1,2,\cdots,N$, and then computed statistics in the
reverse order. Here, we show how to compute the correlations $\langle
x_{i}x_{j}\rangle$ not on the tree; that is, for $(ij)\not\in\mathcal{T}$. To
begin, we assume that we have computed the correlations $\langle
x_{j}x_{k}\rangle$ for all nodes $k>i>j$. Then, if we compute $\langle
x_{i}x_{j}\rangle$, the procedure will follow by induction.

From the Boltzmann distribution in Eq. (5), we have

$\frac{d\langle x_{i}\rangle}{dh_{j}}=\langle x_{i}x_{j}\rangle-\langle
x_{i}\rangle\langle x_{j}\rangle.$ (35)

We already know how to compute the averages $\langle x_{i}\rangle$ and
$\langle x_{j}\rangle$, so all that remains is to calculate the above
derivative. Let $p(i)$ denote the parent of $i$ (that is, the final neighbor
when $i$ is removed) and likewise for $p(j)$. Differentiating Eq. (23) with
respect to $h_{j}$, we have

$\frac{d\langle x_{i}\rangle}{dh_{j}}=\frac{\partial\langle
x_{i}\rangle}{\partial
h_{i}^{(i)}}\frac{dh_{i}^{(i)}}{dh_{j}}+\frac{\partial\langle
x_{i}\rangle}{\partial\langle x_{p(i)}\rangle}\frac{d\langle
x_{p(i)}\rangle}{dh_{j}}.$ (36)

We note that

$\frac{d\langle x_{p(i)}\rangle}{dh_{j}}=\langle x_{p(i)}x_{j}\rangle-\langle
x_{p(i)}\rangle\langle x_{j}\rangle,$ (37)

which we have already computed by assumption, since $p(i)>i$. From Eq. (23) we
have,

$\displaystyle\frac{\partial\langle x_{i}\rangle}{\partial h_{i}^{(i)}}$
$\displaystyle=\frac{e^{-h_{i}^{(i)}}}{\big{(}1+e^{-h_{i}^{(i)}}\big{)}^{2}}$
(38) $\displaystyle\quad+\langle
x_{p(i)}\rangle\Bigg{(}\frac{e^{-J_{ip(i)}-h_{i}^{(i)}}}{\big{(}1+e^{-J_{ip(i)}-h_{i}^{(i)}}\big{)}^{2}}-\frac{e^{-h_{i}^{(i)}}}{\big{(}1+e^{-h_{i}^{(i)}}\big{)}^{2}}\Bigg{)},$

and

$\frac{\partial\langle x_{i}\rangle}{\partial\langle
x_{p(i)}\rangle}=\frac{1}{1+e^{-J_{ip(i)}-h_{i}^{(i)}}}-\frac{1}{1+e^{-h_{i}^{(i)}}}.$
(39)

Finally, we note that the dependence of $h_{i}^{(i)}$ on $h_{j}$ runs only
through $h^{(j+1)}_{p(j)}$, such that

$\frac{dh_{i}^{(i)}}{dh_{j}}=\frac{dh_{i}^{(i)}}{dh^{(j+1)}_{p(j)}}\frac{\partial
h^{(j+1)}_{p(j)}}{\partial h_{j}}.$ (40)

Since $p(j)>j$, we can assume that we have already computed
$\frac{dh_{i}^{(i)}}{dh^{(j+1)}_{p(j)}}$. Finally, Eq. (16) yields

$\frac{\partial h^{(j+1)}_{p(j)}}{\partial
h_{j}}=\frac{1}{1+e^{-J_{jp(j)}-h_{j}^{(j)}}}-\frac{1}{1+e^{-h_{j}^{(j)}}}.$
(41)

Plugging everything into Eq. (36), and inducting on $i>j$, one can compute the
correlations $\langle x_{i}x_{j}\rangle$ between all variables.

##  Appendix F Predictions of random and minimum distance trees

Figure 10: Pairwise statistics in minimum distance and random trees. (a)
Ranked order of significant mutual information in the population (black),
two–standard–deviation errors (shaded region), and predictions of the minimum
distance tree (blue) and a random tree (cyan). (b) Correlation coefficients
predicted in the minimum distance and random tree models versus those in the
data, with dashed lines indicating equality. All pairs are divided evenly into
bins along the x–axis, with solid lines and shaded regions reflecting means
and errors (standard deviations) within bins.

In §IV, we studied the predictions of the minimax entropy model
$P_{\mathcal{T}}$ corresponding to the optimal tree $\mathcal{T}$. For
comparison, here we consider the predictions of the minimum distance and
random trees. While the optimal tree captures
$I_{\mathcal{T}}=26.2\,\text{bits}$ of information
($I_{\mathcal{T}}/S_{\text{ind}}=14.4\%$ of the independent entropy), the
minimum distance tree only captures $11.9\,\text{bits}$ of information
($6.5\%$ of the independent entropy), and a typical random tree only captures
$(N-1)\bar{I}=0.4\,\text{bits}$ ($0.2\%$ of the independent entropy).

Figure 11: Synchronized activity in minimum distance and random trees.
Distribution $P(K)$ of the number of simultaneously active neurons $K$ in the
data (black), the Gaussian distribution for independent neurons with mean and
variance $\langle K\rangle_{\text{exp}}$ (dashed), and the predictions of the
minimum distance tree (blue) and a random tree (cyan). As in Fig. 4, to
estimate $P(K)$ and error bars (two standard deviations), we first split the
data into 1–minute blocks to preserve dependencies between consecutive
samples. We then select one third of these blocks at random and repeat 100
times. For each subsample of the data, we fit the maximum entropy model
$P_{\mathcal{T}}$ for each tree $\mathcal{T}$ and predict $P(K)$ using a Monte
Carlo simulation.

For each tree, we can predict the mutual information $I_{ij}$ and correlation
coefficients between all pairs of neurons using the procedure in Appendix E.
Since the minimum distance tree includes some of the largest mutual
information in the population, it is able to match the distribution of
$I_{ij}$ (within errors) out to $N\sim 100$ neurons [Fig. 10(a)]. However, the
minimum distance tree fails to predict the observed correlations across most
of the dynamic range of the data [Fig. 10(b)]. Meanwhile, random trees
typically include only weak mutual information [Fig. 10(a)], such that their
predictions are nearly indistinguishable from a population of independent
neurons [Figs. 10(b)].

For the minimax entropy model, a backbone of strong positive interactions
combine to produce accurate predictions for the distribution $P(K)$ of
population–wide synchrony $K$ (Fig. 4). By contrast, random trees predict a
Gaussian distribution consistent with independent neurons, and the minimum
distance tree only produces a slightly broader distribution (Fig. 11). In both
models, large–scale synchrony in activity ($K\gtrsim 50$) and silence
($K\lesssim 10$) occurs significantly less frequently than observed the data.

##  Appendix G Thermodynamic quantities

Consider a system with fields $h_{i}$, interactions $J_{ij}$ that lie on a
tree $\mathcal{T}$, and temperature $T$. The Boltzmann distribution
$P(\bm{x})$ takes the form in Eq. (9) with energy $E(\bm{x})$ defined in Eq.
(10). Here, we denote averages over $P(\bm{x})$ by $\langle\cdot\rangle$,
while dropping the subscript $T$. The susceptibility $\chi$ [Eq. (11)], can be
computed using the results of Appendix E.

To compute the heat capacity $C$ [Eq. (12)], we begin with the average energy

$\langle E(\bm{x})\rangle=T^{2}\frac{d\ln
Z}{dT}=-T^{2}\frac{d(\mathcal{F}/T)}{dT}=\mathcal{F}-T\frac{d\mathcal{F}}{dT}.$
(42)

To calculate the free energy $\mathcal{F}$, we proceed as in Appendix A. After
including the temperature $T$, Eqs. (16) and (17) take the form

$\displaystyle h_{j}^{(i+1)}$
$\displaystyle=h_{j}^{(i)}+T\ln[\frac{e^{\frac{1}{T}(J_{ij}+h_{i}^{(i)})}+1}{e^{\frac{1}{T}h_{i}^{(i)}}+1}\Bigg{]},$
(43) $\displaystyle F^{(i+1)}$
$\displaystyle=F^{(i)}-T\ln[e^{\frac{1}{T}h_{i}^{(i)}}+1\big{]}.$ (44)

Iteratively summing over each variable, we arrive at the free energy
$\mathcal{F}=F^{(N+1)}$. To compute $\frac{d\mathcal{F}}{dT}$, we take
derivatives of Eqs. (45) and (46), yielding

$\displaystyle\frac{dh_{j}^{(i+1)}}{dT}$
$\displaystyle=\frac{dh_{j}^{(i)}}{dT}+\ln[\frac{e^{\frac{1}{T}(J_{ij}+h_{i}^{(i)})}+1}{e^{\frac{1}{T}h_{i}^{(i)}}+1}\Bigg{]}-\frac{1}{1+e^{-\frac{1}{T}h_{i}^{(i)}}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{h^{(i)}_{i}}{T}\Bigg{)}+\frac{1}{1+e^{-\frac{1}{T}(J_{ij}+h_{i}^{(i)})}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{J_{ij}+h^{(i)}_{i}}{T}\Bigg{)},$
(45) $\displaystyle\frac{dF^{(i+1)}}{dT}$
$\displaystyle=\frac{dF^{(i)}}{dT}-\ln[e^{\frac{1}{T}h_{i}^{(i)}}+1\big{]}-\frac{1}{1+e^{-\frac{1}{T}h_{i}^{(i)}}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{h^{(i)}_{i}}{T}\Bigg{)}.$
(46)

Iterating the above equations, we arrive at the derivative
$\frac{d\mathcal{F}}{dT}=\frac{dF^{(N+1)}}{dT}$, which completes our
calculation of the average energy [Eq. (42)].

The heat capacity is given by

$C=\frac{d\langle E(\bm{x})\rangle}{dT}=-T\frac{d^{2}\mathcal{F}}{dT^{2}}.$
(47)

Taking derivatives of Eqs. (45) and (46), we have

$\displaystyle\frac{d^{2}h_{j}^{(i+1)}}{dT^{2}}$
$\displaystyle=\frac{d^{2}h_{j}^{(i)}}{dT^{2}}-\frac{1}{1+e^{-\frac{1}{T}h_{i}^{(i)}}}\frac{d^{2}h^{(i)}_{i}}{dT^{2}}-\frac{1}{T}\frac{e^{-\frac{1}{T}h^{(i)}_{i}}}{\big{(}1+e^{-\frac{1}{T}h^{(i)}_{i}}\big{)}^{2}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{h^{(i)}_{i}}{T}\Bigg{)}^{2}$
(48)
$\displaystyle\quad\quad\quad\quad\,+\frac{1}{1+e^{-\frac{1}{T}(J_{ij}+h_{i}^{(i)})}}\frac{d^{2}h^{(i)}_{i}}{dT^{2}}+\frac{1}{T}\frac{e^{-\frac{1}{T}(J_{ij}+h^{(i)}_{i})}}{\big{(}1+e^{-\frac{1}{T}(J_{ij}+h^{(i)}_{i})}\big{)}^{2}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{J_{ij}+h^{(i)}_{i}}{T}\Bigg{)}^{2},$
$\displaystyle\frac{d^{2}F^{(i+1)}}{dT^{2}}$
$\displaystyle=\frac{d^{2}F^{(i)}}{dT^{2}}-\frac{1}{1+e^{-\frac{1}{T}h_{i}^{(i)}}}\frac{d^{2}h^{(i)}_{i}}{dT^{2}}-\frac{1}{T}\frac{e^{-\frac{1}{T}h^{(i)}_{i}}}{\big{(}1+e^{-\frac{1}{T}h^{(i)}_{i}}\big{)}^{2}}\Bigg{(}\frac{dh^{(i)}_{i}}{dT}-\frac{h^{(i)}_{i}}{T}\Bigg{)}^{2}.$
(49)

Finally, after computing
$\frac{d^{2}\mathcal{F}}{dT^{2}}=\frac{d^{2}F^{(N+1)}}{dT^{2}}$ iteratively,
we have arrived at the heat capacity $C$.

## References

  * Wiener (1958) Norbert Wiener,  _Nonlinear Problems in Random Theory_ (MIT Press, Cambridge MA, 1958).
  * Cooper (1973) Leon N Cooper, “A possible organization of animal memory and learning,” in  _Collective Properties of Physical Systems: Proceedings of Nobel Symposium 24_ , edited by B Lundqvist and S Lundqvist (Academic Press, New York, 1973) pp. 252–264.
  * Little (1974) William A Little, “The existence of persistent states in the brain,” Math. Biosci. 19, 101–120 (1974).
  * Hopfield (1982) John J Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
  * Amit (1989) Daniel J Amit,  _Modeling Brain Function: The World of Attractor Neural Networks_ (Cambridge University Press, 1989).
  * Hertz  _et al._ (1991) John Hertz, Anders Krogh, and Richard G Palmer,  _Introduction to the Theory of Neural Computation_ (Addison–Wesley, Redwood City, 1991).
  * Segev  _et al._ (2004) Ronen Segev, Joe Goodhouse, Jason Puchalla, and Michael J Berry II, “Recording spikes from a large fraction of the ganglion cells in a retinal patch,” Nat. Neurosci. 7, 1155–1162 (2004).
  * Litke  _et al._ (2004) AM Litke, N Bezayiff, EJ Chichilnisky, W Cunningham, W Dabrowski, AA Grillo, M Grivich, P Grybos, P Hottowy, S Kachiguine, _et al._ , “What does the eye tell the brain?: Development of a system for the large-scale recording of retinal output activity,” IEEE Trans. Nucl. Sci. 51, 1434–1440 (2004).
  * Chung  _et al._ (2019) Jason E Chung, Hannah R Joo, Jiang Lan Fan, Daniel F Liu, Alex H Barnett, Supin Chen, Charlotte Geaghan-Breiner, Mattias P Karlsson, Magnus Karlsson, Kye Y Lee, _et al._ , “High-density, long-lasting, and multi-region electrophysiological recordings using polymer electrode arrays,” Neuron 101, 21–31 (2019).
  * Dombeck  _et al._ (2010) Daniel A Dombeck, Christopher D Harvey, Lin Tian, Loren L Looger, and David W Tank, “Functional imaging of hippocampal place cells at cellular resolution during virtual navigation,” Nat. Neurosci. 13, 1433–1440 (2010).
  * Tian  _et al._ (2012) Lin Tian, Jasper Akerboom, Eric R Schreiter, and Loren L Looger, “Neural activity imaging with genetically encoded calcium indicators,” Prog. Brain Res. 196, 79–94 (2012).
  * Demas  _et al._ (2021) Jeffrey Demas, Jason Manley, Frank Tejera, Kevin Barber, Hyewon Kim, Francisca Martínez Traub, Brandon Chen, and Alipasha Vaziri, “High-speed, cortex-wide volumetric recording of neuroactivity at cellular resolution using light beads microscopy,” Nat. Methods 18, 1103–1111 (2021).
  * Steinmetz  _et al._ (2021) Nicholas A Steinmetz, Cagatay Aydin, Anna Lebedeva, Michael Okun, Marius Pachitariu, Marius Bauza, Maxime Beau, Jai Bhagat, Claudia Böhm, Martijn Broux, _et al._ , “Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings,” Science 372, eabf4588 (2021).
  * Jaynes (1957) Edwin T Jaynes, “Information theory and statistical mechanics,” Phys. Rev. 106, 620 (1957).
  * Schneidman  _et al._ (2006) Elad Schneidman, Michael J Berry II, Ronen Segev, and William Bialek, “Weak pairwise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007 (2006).
  * Nguyen  _et al._ (2017) H Chau Nguyen, Riccardo Zecchina, and Johannes Berg, “Inverse statistical problems: from the inverse Ising problem to data science,” Adv. Phys. 66, 197–261 (2017).
  * Meshulam  _et al._ (2017) Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank, and William Bialek, “Collective behavior of place and non-place neurons in the hippocampal network,” Neuron 96, 1178–1191 (2017).
  * Meshulam  _et al._ (2021) Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank, and William Bialek, “Successes and failures of simplified models for a network of real neurons,” Preprint: arxiv.org/abs/2112.14735 (2021).
  * Tkačik  _et al._ (2015) Gašper Tkačik, Thierry Mora, Olivier Marre, Dario Amodei, Stephanie E Palmer, Michael J Berry II, and William Bialek, “Thermodynamics and signatures of criticality in a network of neurons,” Proc. Natl. Acad. Sci. U.S.A. 112, 11508–11513 (2015).
  * Lezon  _et al._ (2006) Timothy R Lezon, Jayanth R Banavar, Marek Cieplak, Amos Maritan, and Nina V Fedoroff, “Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns,” Proc. Natl. Acad. Sci. U.S.A. 103, 19033–19038 (2006).
  * Weigt  _et al._ (2009) Martin Weigt, Robert A White, Hendrik Szurmant, James A Hoch, and Terence Hwa, “Identification of direct residue contacts in protein–protein interaction by message passing,” Proc. Natl. Acad. Sci. U.S.A. 106, 67–72 (2009).
  * Marks  _et al._ (2011) Debora S Marks, Lucy J Colwell, Robert Sheridan, Thomas A Hopf, Andrea Pagnani, Riccardo Zecchina, and Chris Sander, “Protein 3D structure computed from evolutionary sequence variation,” PLoS One 6, e28766 (2011).
  * Lapedes  _et al._ (2012) Alan Lapedes, Bertrand Giraud, and Christopher Jarzynski, “Using sequence alignments to predict protein structure and stability with high accuracy,” arXiv preprint arXiv:1207.2484 (2012).
  * Bialek  _et al._ (2012) William Bialek, Andrea Cavagna, Irene Giardina, Thierry Mora, Edmondo Silvestri, Massimiliano Viale, and Aleksandra M Walczak, “Statistical mechanics for natural flocks of birds,” Proc. Natl. Acad. Sci. U.S.A. 109, 4786–4791 (2012).
  * Russ  _et al._ (2020) William P Russ, Matteo Figliuzzi, Christian Stocker, Pierre Barrat-Charlaix, Michael Socolich, Peter Kast, Donald Hilvert, Remi Monasson, Simona Cocco, Martin Weigt, _et al._ , “An evolution-based model for designing chorismate mutase enzymes,” Science 369, 440–445 (2020).
  * Lynn  _et al._ (2019) Christopher W Lynn, Lia Papadopoulos, Daniel D Lee, and Danielle S Bassett, “Surges of collective human activity emerge from simple pairwise correlations,” Phys. Rev. X 9, 011022 (2019).
  * Zhu  _et al._ (1997) Song Chun Zhu, Ying Nian Wu, and David Mumford, “Minimax entropy principle and its application to texture modeling,” Neural Comput. 9, 1627–1660 (1997).
  * Prentice  _et al._ (2016) Jason S. Prentice, Olivier Marre, Mark L. Ioffe, Adrianna R. Loback, Gašper Tkačik, and Michael J. Berry II, “Error–robust modes of the retinal population code,” PLoS Comput. Biol. 12, e1005148 (2016).
  * Chow and Liu (1968) CKCN Chow and Cong Liu, “Approximating discrete probability distributions with dependence trees,” IEEE Trans. Inf. Theory 14, 462–467 (1968).
  * Lynn  _et al._ (2023) Christopher W Lynn, Qiwei Yu, Rich Pang, William Bialek, and Stephanie E Palmer, “Exactly solvable statistical physics models for large neuronal populations,” Preprint: arxiv.org/abs/2310.10860 (2023).
  * Gauthier and Tank (2018) Jeffrey L Gauthier and David W Tank, “A dedicated population for reward coding in the hippocampus,” Neuron 99, 179–193 (2018).
  * Shannon (1948) Claude E Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J. 27, 379–423 (1948).
  * Schneidman  _et al._ (2003) Elad Schneidman, Susanne Still, Michael J Berry II, and William Bialek, “Network information and connected correlations,” Phys. Rev. Lett. 91, 238701 (2003).
  * Sethna (2021) James P Sethna,  _Statistical Mechanics: Entropy, Order Parameters, and Complexity_ , Vol. 14 (Oxford University Press, USA, 2021).
  * Moore and Mertens (2011) Cristopher Moore and Stephan Mertens,  _The Nature of Computation_ (OUP Oxford, 2011).
  * Song  _et al._ (2005) Sen Song, Per Jesper Sjöström, Markus Reigl, Sacha Nelson, and Dmitri B Chklovskii, “Highly nonrandom features of synaptic connectivity in local cortical circuits,” PLoS Biol. 3, e68 (2005).
  * Lynn and Bassett (2019) Christopher W Lynn and Danielle S Bassett, “The physics of brain network structure, function and control,” Nat. Rev. Phys. 1, 318 (2019).
  * Lin  _et al._ (2023) Albert Lin, Runzhe Yang, Sven Dorkenwald, Arie Matsliah, Amy R Sterling, Philipp Schlegel, Szi-chieh Yu, Claire E McKellar, Marta Costa, Katharina Eichler, _et al._ , “Network statistics of the whole-brain connectome of Drosophila,” Preprint: biorxiv.org/content/10.1101/2023.07.29.551086 (2023).
  * Albert and Barabási (2002) Réka Albert and Albert-László Barabási, “Statistical mechanics of complex networks,” Rev. Mod. Phys. 74, 47 (2002).
  * Harris  _et al._ (2012) Julia J Harris, Renaud Jolivet, and David Attwell, “Synaptic energy use and supply,” Neuron 75, 762–777 (2012).
  * Laughlin and Sejnowski (2003) Simon B Laughlin and Terrence J Sejnowski, “Communication in neuronal networks,” Science 301, 1870–1874 (2003).
  * Watts and Strogatz (1998) Duncan J Watts and Steven H Strogatz, “Collective dynamics of ‘small–world’ networks,” Nature 393, 440–442 (1998).
  * Schnabel  _et al._ (2011) Stefan Schnabel, Daniel T Seaton, David P Landau, and Michael Bachmann, “Microcanonical entropy inflection points: Key to systematic understanding of transitions in finite systems,” Phys. Rev. E 84, 011127 (2011).
  * Peierls (1936) Rudolf Peierls, “On Ising’s model of ferromagnetism,” in  _Mathematical Proceedings of the Cambridge Philosophical Society_ , Vol. 32 (Cambridge University Press, 1936) pp. 477–481.
  * Baxter (2016) Rodney J Baxter,  _Exactly solved models in statistical mechanics_ (Elsevier, 2016).
  * Lynn  _et al._ (2022) Christopher W Lynn, Caroline M Holmes, and Stephanie E Palmer, “Heavy–tailed neuronal connectivity arises from Hebbian self–organization,” Preprint: biorxiv.org/content/10.1101/2022.05.30.494086 (2022).
  * Strong  _et al._ (1998) Steven P Strong, Roland Koberle, Rob R de Ruyter van Steveninck, and William Bialek, “Entropy and information in neural spike trains,” Phys. Rev. Lett. 80, 197 (1998).