# History-dependence shapes causal inference of brain-behaviour relationships

Brandon Caie  Centre for Neuroscience Studies, Queen’s University  Gunnar
Blohm  Centre for Neuroscience Studies, Queen’s University

## Abstract

Behavioural and neural time series are often correlated with the past. This
history-dependence may represent a fundamental property of the measured
variables, or may arise from how confounding variables change over time. Here
we argue that undecidability about the ground-truth of history-dependence is a
general computational property of systems that exchange information with its
environment, and show that the resulting uncertainty has a direct impact on
causal inference. We first argue that uncertainty in the ground truth of
history-dependence is an inherent property of open systems that cannot be
explicitly falsified. Simple model systems are then simulated to show how
different assumptions about history-dependence can lead to spurious
correlations and statistical properties of data distributions that are
typically unaccounted for. We then consider this problem from an
interventionist perspective, showing that interventions can only be guaranteed
to remedy the spurious correlation problem when the latent dynamics between
the intervention and measured processes are known a priori, and the effect of
the intervention is invariant at the chosen level of analysis. We conclude
that uncertainty about history-dependence is a fundamental property of the
study of neural systems, and in light of this discuss how causality should be
assessed in neuroscience.

## Introduction

Behaviour and its neural basis can be understood in terms of change ([93, 123,
72, 114]). Linking behaviour to neural activity therefore requires the
identification of correlational or causal relationships between neural and
behavioural processes that are themselves in flux ([90, 86, 102, 79]).
Generally, how does one determine the statistical properties of how a system
changes over time? In isolated systems, where there is no exchange of
information between the external world, history dependence may be answered by
systematically manipulating the inputs and recording outputs – if one can
change the stimulus eliciting a behavioural response, or electrically
stimulate a neuron, one can test how the output at one timepoint is dependent
on the input from a previous timepoint.

When neural and behavioural processes are measured, even with careful
controls, there is irreducible variability in observed responses ([73, 77,
126, 82, 68, 118, 130, 70, 71]). Because of this, models accounting for
behaviour and/or neuronal activity often introduce a random process at some
level in order to explain empirical variability (e.g. [76, 74, 97]). Once a
random event is required to explain uncertainty in any sort of measurement,
one may then be interested in where this randomness originates, and what
purpose it serves 111In The Foundation of Statistics, [121] wrote ”It is
unanimously agreed that statistics depends somehow on probability. But, as to
what probability is and how it is connected with statistics, there has seldom
been such complete disagreement and breakdown of communication since the Tower
of Babel.” 222Bertrand Russell said that ”Probability is the most important
concept in modern science, especially as nobody has the slightest notion what
it means.” [69]. A random event may be interpreted as an inherent property of
the system – alternatively, a random process may describe variability that
arises from a collection of unobserved, but knowable, causes ([98, 95])). In
the former case, variability is often considered to be irreducible, and as
such a fundamental property of the system, while in the latter case it may
arise from dependencies that while theoretically knowable can be sufficiently
explained by a collection of random events.

One way in which observed variability may be explained through a knowable
cause, in lieu of appealing to inherent randomness, is history-dependence
([108, 99, 78, 104, 84]). In neuroscience, it is common to collect many trials
of a similar behaviour, upon which one will find variability ([89, 94]) – the
study of history-dependence asks to what degree previous trials are predictive
of future states, and as such a predictor of future variance. Even in cases
where there is no apparent benefit to treating trials as independent, it is
common to observe statistical dependencies between neural activity and
behaviour across-trials ([83, 101, 87, 125, 100, 128, 104, 105]). When
deconstructing the history-dependence of data that is traditionally aligned to
a common event in a repeated task and treated as an ensemble, one is
essentially asking to what degree observed variability can be explained by
considering a given process not as a collection of independent events, but as
different elements of unique sequences. However, in order to test many of
these hypotheses about what may be thought of as the essentialism of
variability, that is to consider variability as a natural kind, the ability to
distinguish stationary randomness from history-dependence needs to be a
computationally tractable problem 333Ernst Mayr describes essentialism as a
belief system wherein ” a limited number of fixed, unchangeable ”ideas”
underlying the observed variability [in nature], with the eidos (idea) being
the only thing that is fixed and real, while the observed variability has no
more reality than the shadows of an object on a cave wall” (quoted in [124]) .

Here, we argue that falsifying the ground-truth history-dependence of neural
systems is computationally undecidable. In doing so, we propose a fundamental
limitation on the ability to falsify the distinction between inherent
randomness and variability arising from history dependence. Importantly,
rather than simply providing a limitation on our understanding of a generative
model of a living organism, we claim that uncertainty about the ground-truth
history-dependence of a system has material consequences on causal inference.
By studying simple model systems where history-dependence can be quantified,
we arrive at the following primary claims:

1. There is no free lunch for the ontological nature of variability in joint observations of neural and behavioural data – no agnostic vantage from which one can avoid making assumptions about whether a repeated measurement is inherently random or a product of history-dependence. 2. Inductive biases about variability have direct effects on hypothesis testing and causal inference.

### Undecidability of history-dependence in open systems

Experimentalists observe statistical dependencies between neural activity and
behaviour of previous and current trials ([83, 101, 87, 125, 100, 128, 104,
105]). Given this knowledge, one may then be interested in using information
derived from statistical dependencies to deduce general properties about the
systems that produce observations of behaviour and neural activity.
Fundamentally, this is a systems identification approach that seeks to uncover
the dynamical properties of the system in question from repeated measurements.
However, systems identification requires the ability to control the initial
conditions of the system under different controlled inputs. Neural systems, on
the other hand, are rarely (if ever) able to be isolated under fixed
conditions. Thus, the ability to perform systems identification in
neuroscience requires that the dynamics under question are sufficiently
stationary and isomorphic across different individuals and/or experimental
demarcations (be them time points, trials, sessions, etc) such that
fundamental properties can be uncovered. It can be debated if, when and to
what degree this may be possible in different biological systems ([109, 112,
80, 120]). But how would one go about verifying the stationarity hypothesis in
even the simplest of model systems?

Consider a simple systems identification problem. Here, an experimenter is
trying to determine if a binary sequence generator is governed by a random
process, or a deterministic dynamical system (Fig 1). The experimenter decides
to run 100 experiments, where in each 100 outcomes are recorded. In each of
these 100 experiments, a different sequence is observed each time. After
averaging all of the outcomes, the probability of observing a 0 is close to
that of a 1, so it may be concluded that the underlying process is random.
However, for every possible experimental outcome, it would be trivial to
construct a deterministic machine, the length equalling the sum of the
experiments producing the data, that yields an identical output. Following
this, one may repeat another 100 experiments, upon which one could falsify the
previous machine, and then propose a new deterministic machine to account for
all of the experiments observed thus far, and so on ad infinitum.

Figure 1: Undecidability of binary outcomes. A: A program that continues if a
deterministic sequence can be constructed to explain the outcome of a random
event, upon which a new random event is draw, will never halt B: Two distinct
generative process for a binary sequence (random and deterministic) are
depicted in the left column, with two identical empirical samples shown in the
right column.

This recursive logic (Fig 1) is the basis for a few famous computability and
logical problems, such as the halting problem, the incompleteness theorem, and
the diagonal argument ([129, 117, 103]). Essentially, this class of arguments
relies on demonstrating a set of logical statements that are contradictory and
result in a recursive chain of plausible justifications. Consider generally
that the goal of reasoning about a system is to improve a belief over it, Q.
The argument posits that while no reason can be Q itself (i.e. rejection of
circular logic), no reason is sufficiently justified in the absence of a
further reason. Applying this logic to deciding whether a system is inherently
random or exhibiting history-dependence, explaining variability in data
through a random process may always be contradicted by a suitably chosen
epistemic source, i.e. a past sequence of inputs that must be retested to
falsify, and vice versa.

So when assessing history-dependence in biological systems, where precise
test-retesting under a fixed history of inputs may not be realizable, one must
begin with the premise that establishing the ground-truth history-dependence
of a system is computationally undecidable. While a statistical relationship
between past and present inputs to the system may be deduced, we should be
wary of positioning the recovery of a ground-truth understanding of system
dynamics as a goal, even with the ability to sample from it infinitely.
Instead, we argue it should be expected that an irreducible uncertainty exists
in our understanding of any governing dynamics under observation.

What are the consequences of this fundamental limitation in the accuracy of
systems identification on open systems, where there is a fundamental asymmetry
between past and future, such as those likely to explain the neural control of
behaviour? Our view is that, beyond providing a barrier to a complete ontology
of a living thing, uncertainty over the history-dependence in a system has a
tangible and quantifiable effect on statistical inference of causal
interventions to neural circuits. Specifically, we will show that statistical
testing of brain-behaviour relationships, whether through correlational
studies or causal interventions, is dependent on the history-dependence of the
system, which is only testable by expanding the timescale over which possible
dynamics are considered, thus requiring new experiments to falsify a new set
of dynamics that cover the expanded hypothesis space, that in turn require
retesting, ad infinitum.

## Results

### History-dependence in simple model systems

To build an understanding of the statistical challenges of history-dependent
data, we will focus on the study of simple model systems where history-
dependence can be quantified and controlled. Maybe the simplest example of a
history-dependent system is a random walk. Random walks are a fundamental
concept in mathematics and statistics, serving as a useful model for various
biological phenomena ([75]). Essentially, a random walk is a mathematical
process that describes the trajectory of a particle as it takes successive,
random steps in space. In the simplest case, being a one-dimensional random
walk, a particle starts at an initial position $x_{0}$, and at each time step
moves left or right by some fixed distance. The direction of the step is
determined by a random process, such as a binary outcome in a discrete
setting, where 1 represents a step to the right, and 0 a step to the left. The
position of the particle at any given time $t$ is denoted as $x_{t}$, and can
be expressed as the cumulative sum of the random steps taken up to that point

$x_{t}=x_{0}+\sum_{i=1}^{t}Z_{i}$ (1)

Where:

$\displaystyle x_{t}$ $\displaystyle\text{ is the position at time }t$
$\displaystyle x_{0}$ $\displaystyle\text{ is the initial position at time }0$
$\displaystyle Z_{i}$ are independent and identically distributed random
variables representing the steps taken at each time step.

Now consider that this process is repeated many times, such that we have a
collection of random processes $X$ composed of these individual walks. $X_{t}$
can be described as a collection of states at each time point:

$\displaystyle X_{t}$
$\displaystyle=(X_{t}^{(1)},X_{t}^{(2)},\ldots,X_{t}^{(N)})$ (2)
$\displaystyle=(X_{0}^{(1)},X_{0}^{(2)},\ldots,X_{0}^{(N)})+\left(\sum_{j=1}^{t}Z_{j}^{(1)},\sum_{j=1}^{t}Z_{j}^{(2)},\ldots,\sum_{j=1}^{t}Z_{j}^{(N)}\right)$
(3)

Here:

$\displaystyle X_{t}$ $\displaystyle\text{ is the state vector representing
the entire random process at time }t$ $\displaystyle N$ is the total number of
individual random walks in the collection.

The expected value of a collection of random walks is always zero, because
random walks consist of equally likely steps in both positive and negative
directions. This symmetrical nature ensures that, on average, the positive
steps cancel out the negative steps, resulting in an expected value of zero.
Mathematically,

$\mathbb{E}[X_{t}]=\sum_{i=1}^{t}\mathbb{E}[Z_{i}]=0$ (4)

However, the variance of the random walk does not stay constant across time.
The variance of the random walk is given by:

$\text{Var}[X_{t}]=\sum_{i=1}^{t}\text{Var}[Z_{i}]=t\cdot\sigma^{2}$ (5)

Where $\text{Var}[X_{t}]$ represents the variance of the random walk at time
$t$, and $\sigma^{2}$ is the variance of each individual step. Because the
variance of the ensemble $X$ grows with time, while the expected value remains
zero, we can deduce something important about its properties: a random walk is
non-ergodic. An ergodic process is one wherein the statistical properties
derived from one long realization are the same as those obtained from an
ensemble of many independent realizations. In short, an ergodic process allows
you to learn about the ensemble behavior by observing a single, infinitely
long sample path, and vice versa. From an experimentalist’s perspective,
knowing a given measurement comes from an ergodic process can be important,
because it allows for knowledge about many measurements of a system collected
at one moment in time to generalize to how single measurements change with
time, and vice versa.

A random walk is non-ergodic because it does not satisfy this property. In a
random walk, the statistical properties obtained from a single trajectory can
differ from those derived from an ensemble of independent realizations. This
discrepancy arises due to the inherent randomness in individual steps and the
accumulation of that randomness over time. As a result, any given random walk
may exhibit a wide range of possible trajectories, while the behavior observed
over a finite time span may not necessarily reflect the entire ensemble (Fig
2a). Thus, while the expected mean of the process is unchanging across time,
individual trajectories vary considerably, such that the expected variance of
the ensemble continues to increase over time, with more and more processes
accumulating sequences of events that are individually unlikely to occur. In
the language of ergodic theory, the time-average does not equal the ensemble
average. This distinction is visualized in Fig 2A (ensemble average depicted
as the cross-section in purple, and the time average the longitudinal-section
in green).

Figure 2: Non-Ergodicity in a Simple Model of History-Dependence. A:, The
ensemble average vs the time average of a set of time series is depicted. In
ergodic processes, these properties are equivalent. B:, depiction of a set of
random walks with a constant starting position. Non-ergodicity is due to the
increase in variance over time with constant mean. C: Plate notation showing a
hypothetical problem where an experimenter is sampling from a physical process
at different time points. D: Theoretical distribution Z for random walk
sampling over different lengths of time; note the non-Gaussian heavy tails
that increase with experiment length

Consider now the study of a system that is governed by a random walk (Fig 2B).
At every time-point imagine they can sample from a single realization of the
process with some variability. For simplicity, assume that the true value of
the process is the mean of a normal distribution, where the variance $\sigma$
of that distribution is the precision with which one may directly sample from
the underlying random walk. They may then repeat the experiment N times, such
that $y_{t}$ represents samples from the $N$ experiments at time $t$. If the
experimentalist could repeat this sampling process over the same random walks
an infinite number of times, the theoretical distribution $Z$ could be
recovered by sampling from Y

$\begin{split}Y_{t}^{(n)}&\leftarrow N(X_{t}^{(n)},\sigma)\\\
Z_{t}^{(n)}&\leftarrow Y_{t}^{(n)}\end{split}$ (6)

The theoretical distribution is thus akin to a mixture model, where the
probability density of each random walk over Gaussian means represents the
theoretical distribution of one process, equally weighted with all other
random walks to give the theoretical distribution $Z$. Figure 2C depicts this
process in plate notation, where each variable within a ’plate’ is repeated
’N’ number of times. For this, we simulated the theoretical distribution Z for
experiments with different numbers of trials (100, 200, 300, 400) in Fig 2D.

Two things should be observed in the $Z$ distributions in Fig 2D. First, a
greater proportion of the probability density occurs in the tails of the
distribution than would be expected from a Gaussian. Secondly, the proportion
of the density occupied by the tails is dependent on the length of the
experiment. Put simply, this means that the chances of single trajectories
showing extreme deviations from the average when accumulating random events
increases over time. Thus, while the average across many instances of a random
walk – the ensemble average– remains constant, any given realization is not
predictive of the ensemble behaviour. Further, the probability of observing a
realization that has accumulated far away from the ensemble average increases
with the length of an experiment, such that collecting more data from a single
trajectory decreases, rather than increases, its predictive power of the
ensemble statistics.

What are some of the consequences of studying non-ergodic systems? Consider a
study that correlates neural activity with a behavioural time series. In
absence of knowledge about the true generative model of a system, we can
leverage simulations to describe what should be expected in simple systems
where we have control over the process. First, consider two independent time
series, one a measurement of a behaviour $B$, and one a measurement of a
neural response $N$. It has been known since the Monte Carlo simulations of
[92] (and see [116] for an asymptotic proof) that if these two processes were
to follow independent random walks,

$\begin{split}B_{t}=B_{0}+\sum_{i=1}^{t}Z_{i}\\\
N_{t}=N_{0}+\sum_{i=1}^{t}N_{i}\\\ \end{split}$ (7)

while the ensemble of each process have identical means, spurious correlations
tend to be found between individual comparisons of the two processes.
Informally, this occurs because of the non-ergodicity of random walks – in a
2D space, one realization of two independent random walks will not cover the
space on average equally, so on any given sample a relationship between the
two is likely to occur. This has a major consequence for statistical testing
that assumes increasing the sample size improves the reliability of rejecting
the null hypothesis. In the case of independent random walks, as the number of
samples grows, the distribution of correlations remains unchanged, but the
distribution under the null hypothesis shrinks – because of this, collecting
more samples from two independent processes increases the probability of
observing false positives.

Further, the false positive rate between two independent time series can be
shown to be dependent on the strength of history-dependence. To show this, we
consider an autoregressive (AR) model. An AR model is a stochastic process in
which each value in the series is a linear combination of its past values,
with random noise added independently at each time step. Mathematically, an AR
process of order $p$ can be expressed as:

$X_{t}=c+\phi_{1}X_{t-1}+\phi_{2}X_{t-2}+\ldots+\phi_{p}X_{t-p}+\varepsilon_{t}$

Here:

$\displaystyle X_{t}$ represents the current value in the time series.
$\displaystyle c$ is a constant term.
$\displaystyle\phi_{1},\phi_{2},\ldots,\phi_{p}$ are the autoregressive
coefficients $\displaystyle X_{t-1},X_{t-2},\ldots,X_{t-p}$ are the lagged
values of the time series. $\displaystyle\varepsilon_{t}$ is a white noise
error term with mean zero and constant variance. $\displaystyle p$ is the
order of the process

An AR process is therefore a simple model system where the strength of
history-dependence can be adjusted parametrically. An AR process is equivalent
to a random walk when the autoregressive coefficient $p$ and the coefficient
$\phi_{1}$ are equal to 1. In this case, the AR process becomes a first-order
autoregressive process, often denoted AR(1), and can be fully described by a
random walk. In contrast, an AR(0) process simplifies to a constant term plus
white noise, as the series simplifies to a sequence of independent random
variables.

Figure 3: Spurious correlations in independent processes with history-
dependence. A: Graphical depiction of an autoregressive AR(1) model equivalent
to a random walk, where the value of the current timestep is dependent only on
the previous value. The strength of the dependency on the previous timestep is
controlled by $\phi$. B: False positive rate for two independent AR(1)
processes as a function of the length of the time series. C: Visualization of
two independent AR(1) processes as a 2D random walk. 10 random realizations
are plotted in black. Regression lines are plotted in red. D: False positive
rate for a fixed experiment length of 400 timesteps is plotted against the
AR(1) parameter $\phi$, showing a gradual change in spurious correlations
between white noise ($\phi$ = 0) and a random walk ($\phi$ = 1).

To demonstrate the impact of this history-dependence on false positive rates
between two independent time series, we simulated two AR(1) processes while
varying $\phi_{1}$, and plotted the resulting false positive rate, assuming a
significance threshold $\alpha=.05$. The probability of inferring a spurious
relationship between the independent time series was dependent on the strength
of the relationship between the previous time-step, as controlled by the
$\phi_{1}$ coefficient. Fig 3 summarizes these results. An AR(1) model
describing a behavioural and neural process (respectively $B$, $N)$) is
depicted in Fig 3A, with the false positive rate for different lengths of the
time series (here the rate is calculated as the average across 1000
simulations) in Fig 3B. In Fig 3C, 10 random realizations of the independent
$B$ and $N$ processes are depicted as a 2D random walk, with the best-fitting
linear regression between each shown in red, highlighting the tendency to find
a relationship between independent series on any given realization. Fig 3D
plots the average false positive rate for a fixed time series length (T = 400)
across 1000 simulations while varying the $\phi$ parameter. By systematically
varying $\phi$, we could adjust the AR process to behave like white noise
($\phi=0$), a random walk ($\phi=1$), and the space in between. In doing so,
we can see that the false positive rate for two independent time series is
continuously dependent on the strength of history-dependence. Thus, the
strength of history-dependence can control the expected correlational
structure under a null hypothesis, i.e. two measured processes being
independent.

### Mediation analysis over latent intervention dynamics

Issues surrounding spurious correlations in the data used in neuroscience have
been raised previously. Because of this, there have been growing calls for
hypothesis testing in neuroscience to be grounded in causal inference ([122,
131, 111, 96, 119]). Causal inference methods that seek to discover global
causal structures from data are used in neuroscience ([110]), such as Granger
causality, structural equation models, state-space models, and Bayesian
networks. Each of these data-driven approaches have different limitations, and
often different understandings of causality, but generally operate by making
assumptions about the global structure of an underlying graphical model in
order to identify local causal relationships ([115]). For the remainder of
this work, we will focus on the somewhat distinct interventionist approach to
establishing causation, which begins with the problem of identifying local
causes in order to uncover an underlying graphical model ([81]).

It is first worth mentioning that the logic underpinning causality and
experimental interventions in neuroscience is not homogeneous. Some
experimental interventions have a great deal of accumulated evidence to act on
a certain neural process, and thus become a way to manipulate the neural
process to estimate an effect on a behaviour – in this way, a strong a priori
understanding of an interventions effect on a biophysical process that we can
(indirectly) measure is used to infer the relationship between that
biophysical process and an unknown, such as a behaviour. This is what we may
call a neural-deductive approach, where the path between intervention and
neural process is taken as a given in order to deduce a cause between neural
process and behaviour. In contrast, one may have an accumulation of evidence
that a certain intervention affects a given behaviour, and use measurements of
different physical processes to form inferences about how the behaviour is
generated – we may call this a behavioural-deductive approach.

Unfortunately, a strong a-priori understanding of a given intervention is not
always realistic. In reality, how a given intervention in cognitive
neuroscience is understood is usually based on an accumulation of knowledge
across many different levels of analysis. Because of this, the intervention in
context of the measured processes often becomes the process of study itself,
rather than the intervention existing as a known variable that can be taken as
a given ([67]). For this reason, and because we believe this issue has been
underappreciated in cognitive neuroscience, we will now use model simulations
to address the following question: how does adding an intervention to our
previous problem of spurious correlations in jointly-measured brain-
behavioural data improve our ability to assess causality between jointly
measured processes, if there is uncertainty about what the path this
intervention takes to a measured variable. Or rephrased, what should our
expectations for the null hypothesis be when testing the interaction between
new experimental interventions and jointly measured neural and behavioural
processes when the underlying graphical model is unknown. And does this depend
on the properties of the system(s) it is influencing, i.e. history-dependence,
we are measuring? How does uncertainty about these properties change our
expectations under the null hypothesis?

Figure 4: Latent dynamics between null interventions and measured variables
influence mediation analysis. A: Directed graph between an intervention and a
neural and behavioural time series through latent dynamics (black circle).
Regressions performed for mediation analysis are denoted a, b, c. B: Sample
time series for N, B, and I under static and auto-correlated latent
intervention dynamics. C: False positive rates simulated for 1000 experiments
for regression c (solid black), a (large-dash), and b (small-dash), and the
mediation sobel test (red). The plots are shown for static (left) and auto-
correlated latent dynamics (right) for 100 trials (top) and 1000 trials
(bottom) as a function of the $\phi$ parameter (set to be equal for $N$ and
$B$).

We now describe a causal intervention model. Typically, an intervention is
thought of as an experimentally controlled variable that is known a priori.
However, when studying complex systems the underlying graphical model between
a controlled intervention and a measured process is usually hypothesized,
rather than known. While an experiment may have one intervention, and the
measurements of one or several neural and behavioural processes, it is not
always the case that it is known that the intervention directly acts on a
measured variable. In the case of high-dimensional systems, such as those
dealt with in neuroscience, there are typically many possible paths between an
intervention and a causal effect on the measured variable(s). Thus, an
experimentally controlled intervention may have a causal effect on a measured
process, but often through intermediary processes that may have unknown
dynamical properties.

For this reason, we ask how statistical analysis of causal interventions may
change when different latent dynamics between the intervention as defined by
the experimenter and the measured variables are assumed. To do so, we consider
the intervention as a dynamical system of the same form as the autoregressive
models we have considered previously, and ask what would be expected in the
null case where independent neural and behavioural processes are measured over
interventions to latent dynamics that do not causally affect each process. We
consider a neural $N(\phi)$ and behavioural $B(\phi)$ processes that are each
generated by an autoregressive model again parameterized by $\phi$. We then
model a causal intervention over a latent process $I(\phi)$ that is itself an
autoregressive model, where the $\phi$ parameter can be independently
manipulated. It should be re-emphasized that for this null intervention, there
is no causal effect between the independently generated $N$ and $B$, and $I$ –
we instead generate each and ask how causal relationships between $I$, $N$,
and $B$ may be assessed in the null case where different latent dynamics
between an experimentally-defined intervention and the measured processes are
assumed.

We next require a method to infer causality from the intervention. Mediation
analysis is a statistical method to infer a causal path between an independent
variable and a dependent variable through the inclusion of a mediating
variable ([106]). Mediation analysis first confirms the significance of a
regression $c$ between an independent and dependent variable, then performing
a regression $a$ between the independent and mediating variable, and a
regression $b$ between the mediating and dependent variable (Fig 4A).
Significance testing for mediation analyses can then be performed in several
ways, for example a Sobel test, where a mediation effect is significant if
both $c$ supports a significant relationship and a mediation score is
significant, using a z-score computed as

$z=\frac{a*b}{\sqrt{b^{2}sa*a^{2}sb}}$ (8)

We use this approach to deducing causal relationships to ask if, and under
what conditions, knowledge about the latent dynamics between an experimentally
controlled intervention and a measured process can reduce the impact of
spurious correlations in autocorrelated data. To this end, we simulate $N$ and
$B$ independently for 1000 experiments, and compute the correlation as a
function of $\phi$, as before, to get a baseline regression for each
experiment. However, here we then consider $N$ as a mediator between $I$ and
$B$; thus, we also compute two additional regressions $I-->N$ and $I-->B$, in
order to compute the Sobel test for the presence of a mediating relationship
between $I$, $N$, and $B$. Fig 4A visualizes this model graphically, and Fig
4B plots sample traces of each process.

Fig 4c then shows the false-positive rate for each individual regression, and
the mediation test, as a function of the $\phi$ parameter for two types of
null interventions: one generated by an AR($\phi=0$), which we denote as
static latent dynamics and one AR($\phi=1$), which we denote as autocorrelated
latent dynamics. The false positive rates are then shown for four cases:
static and autocorrelated latent dynamics over 100 and 1000 trials. In the
static intervention case, it can be seen that while false positives in
individual correlations can be seen to rise as a function of $\phi$, the
mediation analysis reduces the false positive rate dramatically. Thus, by the
additional knowledge of the latent dynamics between the intervention and
measured variables being independently sampled, the spurious relationships
found in correlational analysis can be avoided altogether by instead testing
the hypothesis of a relationship between $N$ and $B$, given the latent
dynamics $I$.

However, when the latent dynamics between the intervention and $N$ and $B$ are
assumed to be time-dependent, here done by simulating the intervention
regression time series as a random walk, the false positive rate rises as a
function of $phi$, such that when all three processes are generated
independently, a significant relationship is concluded in about 40% of cases
when the experiments were simulated with 100 trials. As such, if an
experimentally controlled intervention passes through latent dynamics $I$
prior to causally effecting $N$ and $B$, the expected false positive rate for
significant mediation tests is dependent on the latent dynamics, which may or
may not be observable and/or falsifiable. Additionally, as can be seen in Fig
4C, when the experiment length is increased to 1000 trials, the number of
significant mediation results increases, as was the case when correlating two
independent random walks. As such, for single time-series the confounding
influence of latent dynamics between independent and dependent variables may
be exaggerated, rather than mitigated, by collecting more trials. From this
analysis, it can be seen that the inclusion of an experimentally controlled
variable can reduce the problem of spurious correlations, but only if this
variable (or any variables between the intervention and the causal effect on
the measured variables) influences the measured processes through a path in
the system that is itself not autocorrelated. When the latent dynamics between
intervention and measured variables are assumed to be autocorrelated, the null
analysis suggests that the problem of spurious correlation may persist.

Figure 5: Latent dynamics control mediation significance testing in a causal
effect model. A: Graphical model of the causal effect model. $B$ and $N$ are
AR-processes auto-correlated according to $\phi$, and $N$ effects $B$
according to a gain factor $g_{N}$. The latent dynamics of the intervention in
turn effect $B$ and $N$ by a gain factor $g_{I}$.B: sample time-series for
$B$, $N$, and $I$ for white noise and random walk interventions, where $g_{I}$
is either 0 (acausal) or non-zero (causal). C: Distribution of mediation
p-values for 1000 experiments for causal white noise (top left), acausal white
noise (top right), causal random walk (bottom left) and acausal random walk
(top right). D: Mean and standard deviation of regression coefficients (left)
and p-values are plotted against the intervention to behaviour (regression c
in figure 4) in 7 bins. E: Proportion of significant tests are plotted as a
function of latent intervention dynamics variance ($sigma_{I}$) and gain
($g_{I}$). Plots are organized as in B. F: Intervention variance-gain plots
for the causal random walk model, for different distributions of $g_{I}$

Defining the latent dynamics of the null intervention allows us to simulate a
false positive distribution, but this does not allow us to compare the
distribution of mediation p-values to what may be expected in the case where a
true effect exists. To address this, we developed a causal mediation model
where i) the neural process directly influences the behavioural process at
each time-point and ii) the intervention time series directly influences both
the behavioural and neural processes. We consider an intervention as a latent
dynamic that we specify with an autoregressive process, that in turn
influences a neural and behavioural time-series with additive gain $g_{I}$. To
focus on the role of the latent intervention dynamics, we model the neural and
behavioural processes as history-dependent, that is a random walk. $N$
provides additive gain to $B$, controlled by $g_{N}$, thus providing a causal
effect on the behavioural time series.

$\begin{gathered}I_{t}=\phi_{1}I_{t-1}+\varepsilon_{t}\\\
N_{t}=N_{0}+\sum_{i=1}^{t}N_{i}+I_{t}*g_{I}+\varepsilon_{t}\\\
B_{t}=B_{0}+\sum_{i=1}^{t}B_{i}+I_{t}*g_{I}+N_{t}*g_{N}+\varepsilon_{t}\\\
\end{gathered}$ (9)

We illustrate this model graphically in Fig 5A, and plot example processes for
single experiments in Fig 5B, where the latent dynamics of the intervention
and gain are changed. We depict four scenarios: a white noise latent dynamic
that causally influences $N$ and $B$ (that is, $g_{I}~{}=0$) denoted $W_{c}$,
a null white noise latent dynamic $W_{N}$ ($g_{I}=0$) , a random walk
intervention that causally influences $N$ and $B$, denoted $R_{C}$, and a
random walk null intervention $R_{N}$ ($g_{I}=0$). We then perform a false
positive analysis on the mediation statistic for each model. Fig 5C plots the
distribution of mediation p-values for each model (top left $W_{C}$, top right
$W_{N}$, bottom right $R_{C}$, bottom left $R_{N}$). Here, it can be seen that
for white noise latent dynamics, a similar distribution of mediation p-values
is observed for both the causal and null model, suggesting that assessing
causality between $N$ and $B$ through $I$ may lead to similar conclusions
regardless of whether or not $I$ actually influences $N$ and $B$. Further, the
distribution of p-values is approximately uniform, suggesting that the null
hypothesis would likely not be rejected if these experiments could be repeated
many times, regardless of whether or not the intervention had a causal effect.

For random walk interventions, we see a different trend. Mediation p-values
are not uniform, but are heavily skewed towards zero, indicating that over
most experiments a significant mediation effect would be reported – regardless
of whether or not an actual effect was observed. To confirm that the reported
p-values were representative of the actual regression coefficients in the
mediation test, we plotted the mean and standard deviations of the direct
regression coefficient $c$ (intervention onto behaviour) as a function of the
variance explained in the regression ($r^{2}$) in order to confirm that the
coefficients were related to variance explained, and then the mediation
p-value compared to the direct regression ($r^{2}$), in order to confirm that
mediation significance was correlated with variance explained in the direct
regression. Overall, this tells us that when using mediation analysis to infer
causality between two measured processes $N$ and $B$ through the use of an
intervention, different latent dynamics in the path $I$ can obscure true
effects when the dynamics are time-independent, ie generated from a white
noise process, bu can generate false positives when the latent dynamics of the
intervention are history dependent, ie generated from a random walk process.
To more fully explore the state space of the model for different
interventions, we then systematically varied the intervention gain $g_{I}$ and
intervention variance $\varepsilon_{I}$ for each of the four intervention
latent dynamics.

Fig 5E plots a heatmap of mediation p-values for each model variation (top
left: acausal-white noise, top right: causal-white noise, bottom left:
acausal-random walk, bottom right: causal random walk.) Here, it can be seen
that for the acausal-white noise model, significant mediation results are
infrequent, consistent with the latent dynamics of the intervention not
influence either $N$ or $B$. In this model, the effect of the gain parameter
is irrelevant, since no effect is observed, but the variance of the process
determines the time-series the regressions are performed on. In the causal
white noise model, the proportion of significant mediation tests was dependent
on both the variance and the gain of the intervention. Thus, distinguishing
between a null and causal intervention was dependent on the statistical
properties of the latent intervention dynamics. For random walk latent
intervention dynamics (bottom left), the proportion of false positive
mediation results for the acausal model increased relative to the acausal
white noise model. This suggests that the expectation of different latent
dynamics (white noise vs random walk) can determine the expected false
positive rate. In the causal random walk model (bottom right), again the gain
and variance parameters of the intervention determined the proportion of
significant tests. Because of this, for different parameters of the latent
intervention dynamics, the causal and acausal model may or may not be able to
be distinguished by repeated testing.

In the causal random walk model, we observed an interesting nonlinearity in
the state-space for very low $g_{I}$ (<.3), as shown in the black box over Fig
5E, bottom right. To explore this further, we simulated the causal random walk
model across smaller discretizations of $g_{I}$. We show the results for this
in Fig 5F. From here, it can be seen that there is a progressive increase,
followed by a decrease, in the proportion of significant mediation tests in
this parameter range. Thus, even a simple causal model like this can have
interesting nonlinearities in the state-space of latent intervention dynamics,
where small changes may elicit different expectations for mediation analysis
that would require an a priori knowledge of the graphical model and latent
dynamics in order to predict.

## Discussion

In this paper, we have argued for the computational undecidability of ground-
truth history-dependence, and used model systems where history-dependence can
be controlled in order to detail the assumptions about history-dependence
required to make causal statements about the relationship between neural and
behavioural processes. Given this, a discussion on hypothesis testing in
neuroscience is warranted – rather than provide a barrier to using
interventions for understanding the neural basis of behaviour, we argue that
explicitly beginning from the starting axiom of irreducible uncertainty would
permit a more holistic understanding of the systems under study, and our place
in intervening on them.

### Hypothesis testing in absence of ground-truth understanding

Perhaps unsurprisingly, the problem of spurious regression was first analyzed
in the context of econometrics ([92]), where trends are not isolated and
repeated in a laboratory, but analyzed and forecast as the system, which we
concede cannot be controlled experimentally, evolves. A neuroscientist, on the
other hand, may object to the characterization of spurious correlation arising
from history-dependence as a major concern for their experimental approaches.
In doing so, they may explain the distinction between regressing single time
series that only occur once, as is typical in economics, and repeatedly
running controlled experiments – such an appeal would rely on the Central
Limit Theorem (Fig 6). The central limit theorem guarantees that the
distribution of sample means of a sufficiently large and independently drawn
random sample from any population, regardless of its original probability
distribution, will tend to approximate a normal distribution. This
approximation becomes increasingly accurate as the sample size grows larger.
For the central limit theorem to be valid, samples must be independent of each
other. When this condition is met, it allows us to use the properties of the
normal distribution for making statistical inferences and hypothesis testing
without requiring the data to arise from a process with known and verifiable
statistical properties.

As such, an experimentalist may be unconcerned with the undecidability of
history-dependence, and the resulting uncertainty over the statistical
properties of the processes they are sampling within individuals, if the goal
is to discern the effect of an experimental intervention at the population
level 444In doing so, one is essentially conceding an inability to identify
ground-truth processes in favour of statistical testing of group or
individual-level changes in the face of external interventions. It may be
optimistic to believe that this is the default (or even dominant)
understanding when performing statistical testing in neuroscience experiments
– the conclusions as stated often undermine this belief. More likely, the
dominance of statistical philosophies that focus on inference from sample to
population are rooted in a confluence of historical developments and
hybridizations of competing theories – the entrenchment of these trajectories
into accepted paradigms has been suggested as a sort of ritualistic belief
system ([88]). Regardless, we will operate under the assumption that this is
generally understood so as to better clarify what sorts of hypotheses are
justifiable under the appeal to the central limit theorem.. By collecting
enough participants and enough data, they can perform statistical tests on the
sample means across different conditions under assumptions of normality, as
the distribution of the sampling mean converges to a normal distribution. To
be sure, many variables follow a normal distribution at the population level,
despite being caused in all likelihood by a complex set of history-dependent
factors that may be individually described by probability distributions of
arbitrary form. So in appealing to the central limit theorem, an experimenter
may argue that, unlike time series that are generally accepted to be non-
repeatable, such as economic trends, in the laboratory they can isolate
specific situations and repeatedly measure them. Although this requires
conceding a general inability to converge on identifying a ground-truth
generative process, it allows for sampling to be exploited in order to test
hypotheses about experimental interventions at population levels.

Figure 6: Central limit theorem allows for test-retesting of effects in non-
ergodic processes. A: Data matrix of a hypothetical non-ergodic process, where
each row is a single ’trial’, and each column represents a time point within
the trial. For arguments sake, each data matrix here is collected from an
independent process. B: resulting distributions of activity averaged across
all trials for each independent matrix. These are cartoons, the point is they
are non-Gaussian. With a sufficient number of independent sessions, the
distribution over sample means across sessions approaches a normal
distribution under the central limit theorem.

Given the inability to falsify the dynamics of open systems from repeated
sampling, it is worth being explicit in what hypotheses are actually testable
in absence of a ground-truth understanding of a system. Take for example the
testing of how an external intervention changes the group-level relationship
between a measured neural and behavioural variable. For this class of
question, one would typically design an experiment such that a behaviour could
be atomized into independent units, often called trials, where a continuously
measured neural variable is aligned to and discretized alongside ([94, 132,
127]). The goal of statistical analysis for such an experiment is to test
whether an intervention changes the correlation structure between neural
activity and behaviour at the population-level. For such a claim to hold,
spurious correlations arising from history-dependence in independent time
series are not necessarily a problem – while this may introduce substantial
across-subject variability, by considering each subject as an unknown data-
generating process, a statistical difference in the pooled means of each
process should be detectable by repeatedly sampling the population on and off
a given intervention. From this, causal statements about population effects of
an intervention can be made without having to be concerned about underlying
generative models or spurious correlations – with the important caveat that
statements made about this intervention may have little to no predictive power
over individual outcomes.

However, many other legitimate hypotheses may be made that are more specific
than asking if an intervention changes a population-level phenomenon. In
neuroscience and psychology, it is increasingly appreciated that inter and
intra-individual differences are a dominant source of variability in data that
corrupt many assumptions of traditional statistical analyses ([107]). Indeed,
these sources of variability may not just be noise that an experimenter should
seek to eliminate, but a reflection of the specificity of experimental
interventions to inter and intra-individual differences that may yield
misleading mixtures of effects when inappropriately aggregated. The basic
example is that of an intervention that can have different effects depending
on some intervening variable. If the intervening variable is not taken into
account, averaging across distinct contexts can produce inappropriate null
results. Thus, it is important to generate hypotheses about interventions that
do not ask whether it has an effect at a population level, but instead in a
subset of individuals, a subset of contexts, or both.

In refining our hypothesis space to include conditional-dependencies, we can
sometimes sacrifice the power to be agnostic about the functional form of the
distributions we sample from, as we end up further removed from data that can
be taken to be reasonably independent, and into cases where data being
combined come quite clearly from dependent sources. At each level of
granularity of hypothesis, we therefore require additional "independent" test-
retests to be able to abstract over potentially dependent sources of
information in order to avoid the issues with spurious regressions, thus
permitting the testing of these more granular hypotheses.

We illustrate hypotheses of different granularity in Fig 7: general-group,
where the effect of an intervention is tested at a population level;
conditional-group, where a group-level effect is hypothesized to only be
observed in a subset of contexts in a given dataset; general-individual, where
only certain individuals are hypothesized to respond to the intervention; and
conditional-individual, where certain individuals only respond to the
intervention in certain contexts. In these hypotheses, different assumptions
need to be made about independence for conventional statistical testing to be
valid. For general-group assessments, as we have suggested, the underlying
distribution of each individual, or any other post-hoc splitting of a given
data-set, need not follow any distribution in particular, so long as it has
some theoretical bound. But consider instead a conditional-group assessment,
where an experimenter collecting some joint neural and behavioural data and
employing the following logic:

>   1. 1.
>
> Intervention $X$ only influences neural measure $Y$ under conditions where
> neural modulator $T$ is high
>
>   2. 2.
>
> Neural modulator $T$ is only observed to be elevated when a subset of the
> behaviour $B_{s}$ is observed
>
>   3. 3.
>
> Intervention $X$ should only influence correlation between $Y$ and $B_{s}$,
> and not $Y$ and $B_{\sim s}$
>
>

Figure 7: Granularity of hypotheses on brain-behaviour interventions. A:
General-group hypothesis, where a causal intervention is hypothesized to
influence a behavioural process through a neural process on average, measured
across a population. B: Conditional-group hypothesis, where the same causal
intervention is hypothesized to only influence a subset of data in the joint
neural-behavioural space. C: General individual, where a causal intervention
is hypothesized to only influence a subset of people in a population. D:
Conditional individual, where a causal intervention is hypothesized to
influence specific people under specific subsets of a larger dataset.

This may be a perfectly valid hypothesis to generate. However, in performing
hypothesis testing on a subset of the data, we rely on that subset having a
stationary distribution that can be sampled from independently of the other
subsets. In short, it introduces the hypothesis of a data subset being a
discrete and independent category, without necessarily verifying that
statistical inference of differences under this assumption can be justified.
To be sure, these sorts of hypotheses may represent reasonable claims, perhaps
justified by some prior knowledge about the system. Returning to the example
of random walks, imagine studying two independent processes where it is
hypothesized that successive repetitions of the same outcome result from a
conditionally-independent state than successive alternations. This may be a
perfectly valid hypothesis to hold – as such, one may deconstruct a time
series into categories of different sequences, building a sort of library of
hypothetical states with different associations to the generative statistics
of a task. But how would one go about verifying the ontological validity of
such a categorization? In an alternative case of the generative models being
described by a purely random process, very similar subsets could likely be
demarcated post-hoc, yet in this case we know that they come from a process
where i) these classes are not a natural kind, and ii) data in these classes
are not conditionally-independent.

Individual variability represents a different level of granularity than
conditional-group hypotheses. Many causal interventions to brain-behaviour
relationships are thought to be specific to individual physiology and
circumstance, but hypotheses in neuroscience and psychology are typically
tested by collecting large participant pools ([113, 85]). In the case where a
hypothesis about an intervention is specific to an individual, a new
granularity of independence must be proposed in order to avoid the spurious
correlation problem of regressing single time series. Here, one may again rely
on the atomization of the trial, such that many independent measures can be
collected within a single session. In this case, one could formulate
hypotheses for specific individuals, or sets of individuals that follow some
pattern in a population, once again relying on the assumption that these
trials are in fact independent.

Given a survey of the literature on sequential dependencies in behaviour, a
researcher may well deem this to be an unacceptable assumption, and conclude
that a history-dependence in the behaviour may introduce the problem of
spurious correlation we observed in analyzing independent random walks.
Instead, they may conclude that in order to study person-specific (sometimes
called low-N designs) brain-behaviour relationships, many different repeated
measures of sessions are required (as B.F. Skinner said, “It is more useful to
study one animal for 1000 hours than to study 1000 animals for one hour
([91])). By performing such a study, they would hope to abstract over
dependencies between trials by getting an estimate of the session-session
variability. However, such a study would pose an additional hypothesis: are
there dependencies between sessions? It is easy to see here how recursive
logic could be employed to iteratively question the claims for independence,
propose an experiment that requires additional data at a timescale further
removed, then question its independence or propose a correction ad infinitum.
The result is a never-ending chain of plausible hypotheses for the functional
role of a behaviour spanning the summed length of an experiment, requiring an
experiment of an additional length to treat statistically, and so on.

Practically, at some point we have to choose whether data are functionally
independent enough for statistical testing. Although truly demonstrating
independence may be computationally undecidable, we may be content in saying
that some things are more dependent than others. For example, ignoring the
statistical challenges of sequential dependencies across trials in an
experiment with a learning effect, or averaging the effects of a neural
intervention across individuals with different relevant physiology, may be
deemed to be unreasonable. On the other hand, the dependencies of seasonal
variables on test-retesting within an individual over many months may not be
so important in determining the effect of an intervention that acts over
minutes. However, in between such extremes lies much room for interpretation.
What principles should guide our decision-making when assessing the relative
independence of brain-behaviour data in response to interventions? If we are
forced to concede that explicit falsification of ground-truth internal
dynamics is undecidable, we have shown here that true effects can be masked,
and false effects exaggerated, depending on our knowledge of the underlying
graphical model. Uncertainty about the context of an intervention with respect
to the ongoing internal dynamics of the system under study thus poses a
challenge to the notion that collecting more data leads to more certainty
about the effect of an independently controlled variable, particularly if the
system under study is non-ergodic.

In this paper, we have shown that this uncertainty is a fundamental property
of open-systems that cannot be isolated from exchanging information with the
world, and analyzed the statistical properties of stochastic processes with
different causal structures in order to demonstrate the ease with which
different latent dynamics may obscure real effects or amplify the relationship
between independent signals. As a result, we conclude that causal inference in
neuroscience should necessarily begin with uncertainty over possible causal
models, which although are almost certainty unobservable, may be more or less
likely at different levels of analysis.

## Code Availability

Code to run the simulations is available at
https://github.com/BJCaie/Ergodicity/

## Acknowledgements

This research was undertaken thanks in part to funding from the Natural
Sciences and Engineering Research Council of Canada (NSERC), the Canada
Foundation for Innovation (CFI), and the Connected Minds Program, supported by
Canada First Research Excellence Fund, Grant CFREF-2022-00010.

## References

  * [1] Odd O Aalen, Kjetil Røysland, Jon Michael Gran and Bruno Ledergerber  “Causality, mediation and time: a dynamic viewpoint”  In _Journal of the Royal Statistical Society. Series A, (Statistics in Society)_ 175.4, 2012, pp. 831–861  DOI: 10.1111/j.1467-985X.2011.01030.x
  * [2] C Allen and C F Stevens  “An evaluation of causes for unreliability of synaptic transmission.”  In _Proceedings of the National Academy of Sciences of the United States of America_ 91.22, 1994, pp. 10380–10383  DOI: 10.1073/pnas.91.22.10380
  * [3] E.T Bell  “The Development Of Mathematics”, 1945  URL: http://archive.org/details/in.ernet.dli.2015.133966
  * [4] Heather Christine Bell  “Behavioral Variability in the Service of Constancy”  In _International Journal of Comparative Psychology_ 27.2, 2014  DOI: 10.46867/ijcp.2014.27.02.02
  * [5] Tiago Branco and Kevin Staras  “The probability of neurotransmitter release: variability and feedback control at single synapses”  In _Nature Reviews Neuroscience_ 10.5, 2009, pp. 373–383  DOI: 10.1038/nrn2634
  * [6] Jerome Bruner  “A Short History of Psychological Theories of Learning”  In _Daedalus_ 133.1, 2004, pp. 13–20  URL: https://www.jstor.org/stable/20027892
  * [7] R… Carpenter  “A neural mechanism that randomises behaviour”  In _Journal of Consciousness Studies_ 6.1, 1999, pp. 13–22
  * [8] Anne K. Churchland et al.  “Variance as a signature of neural computations during decision making”  In _Neuron_ 69.4, 2011, pp. 818–831  DOI: 10.1016/j.neuron.2010.12.037
  * [9] Edward A Codling, Michael J Plank and Simon Benhamou  “Random walk models in biology”  In _Journal of The Royal Society Interface_ 5.25, 2008, pp. 813–834  DOI: 10.1098/rsif.2008.0014
  * [10] Gustavo Deco and Ranulfo Romo  “The role of fluctuations in perception”  In _Trends in Neurosciences_ 31.11, 2008, pp. 591–598  DOI: 10.1016/j.tins.2008.08.007
  * [11] Gustavo Deco, Edmund T. Rolls and Ranulfo Romo  “Stochastic dynamics as a principle of brain function”  In _Progress in Neurobiology_ 88.1, 2009, pp. 1–16  DOI: 10.1016/j.pneurobio.2009.01.006
  * [12] Ashesh K. Dhawale, Maurice A. Smith and Bence P. Ölveczky  “The Role of Variability in Motor Learning”  In _Annual Review of Neuroscience_ 40.1, 2017, pp. 479–498  DOI: 10.1146/annurev-neuro-072116-031548
  * [13] Nadine Dijkstra and Leon Bruin  “Cognitive Neuroscience and Causal Inference: Implications for Psychiatry”  In _Frontiers in Psychiatry_ 7, 2016, pp. 129  DOI: 10.3389/fpsyt.2016.00129
  * [14] Gerald M. Edelman and Joseph A. Gally  “Degeneracy and complexity in biological systems”  In _Proceedings of the National Academy of Sciences_ 98.24, 2001, pp. 13763–13768  DOI: 10.1073/pnas.231499798
  * [15] Peter W. Evans and Sally Shrapnel  “The Two Sides of Interventionist Causation”, 2017  URL: https://philsci-archive.pitt.edu/12906/
  * [16] A. Faisal, Luc P.. Selen and Daniel M. Wolpert  “Noise in the nervous system”  In _Nature Reviews Neuroscience_ 9.4, 2008, pp. 292–303  DOI: 10.1038/nrn2258
  * [17] Jillian H. Fecteau, Crystal Au, Irene T. Armstrong and Douglas P. Munoz  “Sensory biases produce alternation advantage found in sequential saccadic eye movement tasks”  In _Experimental Brain Research_ 159.1, 2004, pp. 84–91  DOI: 10.1007/s00221-004-1935-9
  * [18] Jason Fischer and David Whitney  “Serial dependence in visual perception”  In _Nature Neuroscience_ 17.5, 2014, pp. 738–743  DOI: 10.1038/nn.3689
  * [19] Aaron J. Fisher, John D. Medaglia and Bertus F. Jeronimus  “Lack of group-to-individual generalizability is a threat to human subjects research”  In _Proceedings of the National Academy of Sciences_ 115.27, 2018, pp. E6106–E6115  DOI: 10.1073/pnas.1711978115
  * [20] Michael D. Fox, Abraham Z. Snyder, Justin L. Vincent and Marcus E. Raichle  “Intrinsic Fluctuations within Cortical Systems Account for Intertrial Variability in Human Behavior”  In _Neuron_ 56.1, 2007, pp. 171–184  DOI: 10.1016/j.neuron.2007.08.023
  * [21] Juan Gao et al.  “Sequential Effects in Two-Choice Reaction Time Tasks: Decomposition and Synthesis of Mechanisms”  In _Neural Computation_ 21.9, 2009, pp. 2407–2436  DOI: 10.1162/neco.2009.09-08-866
  * [22] Gerd Gigerenzer  “Statistical rituals: The replication delusion and how we got there”  In _Advances in Methods and Practices in Psychological Science_ 1.2, 2018, pp. 198–218  DOI: 10.1177/2515245918771329
  * [23] Joshua I. Gold and Michael N. Shadlen  “The Neural Basis of Decision Making”  In _Annual Review of Neuroscience_ 30.1, 2007, pp. 535–574  DOI: 10.1146/annurev.neuro.29.051605.113038
  * [24] Alex Gomez-Marin  “Causal Circuit Explanations of Behavior: Are Necessity and Sufficiency Necessary and Sufficient?”  In _Decoding Neural Circuit Structure and Function_ Cham: Springer International Publishing, 2017, pp. 283–306  DOI: 10.1007/978-3-319-57363-2_11
  * [25] James E. Graham, Amol M. Karmarkar and Kenneth J. Ottenbacher  “Small Sample Research Designs for Evidence-based Rehabilitation: Issues and Methods”  In _Archives of physical medicine and rehabilitation_ 93.8, 2012, pp. S111–S116  DOI: 10.1016/j.apmr.2011.12.017
  * [26] C… Granger and P. Newbold  “Spurious regressions in econometrics”  In _Journal of Econometrics_ 2.2, 1974, pp. 111–120  DOI: 10.1016/0304-4076(74)90034-7
  * [27] Donald Hebb  “The Organization of Behaviour”, 1949
  * [28] Alexander Huk, Kathryn Bonnen and Biyu J. He  “Beyond Trial-Based Paradigms: Continuous Behavior, Ongoing Neural Activity, and Natural Stimuli”  In _The Journal of Neuroscience_ 38.35, 2018, pp. 7551–7558  DOI: 10.1523/JNEUROSCI.1920-17.2018
  * [29] Eyke Hüllermeier and Willem Waegeman  “Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods”  In _Machine Learning_ 110.3, 2021, pp. 457–506  DOI: 10.1007/s10994-021-05946-3
  * [30] Eric Jonas and Konrad Paul Kording  “Could a Neuroscientist Understand a Microprocessor?”  In _PLOS Computational Biology_ 13.1, 2017  DOI: doi.org/10.1371/journal.pcbi.1005268
  * [31] Tatjana Kanashiro, Gabriel Ocker, Marlene Cohen and Brent Doiron  “Attentional modulation of neuronal variability in circuit models of cortex”  In _eLife_ 6.e23978, 2017  DOI: 10.7554/eLife.23978
  * [32] Armen Der Kiureghian and Ove Ditlevsen  “Aleatory or epistemic? Does it matter?”  In _Structural Safety_ 31.2, Risk Acceptance and Risk Communication, 2009, pp. 105–112  DOI: 10.1016/j.strusafe.2008.06.020
  * [33] Konrad P. Kording, Joshua B. Tenenbaum and Reza Shadmehr  “The dynamics of memory as a consequence of optimal adaptation to a changing body”  In _Nature Neuroscience_ 10.6, 2007, pp. 779–786  DOI: 10.1038/nn1901
  * [34] Floris P. Lange, Dobromir A. Rahnev, Tobias H. Donner and Hakwan Lau  “Prestimulus Oscillatory Activity over Motor Cortex Reflects Perceptual Expectations”  In _Journal of Neuroscience_ 33.4, 2013, pp. 1400–1410  DOI: 10.1523/JNEUROSCI.1094-12.2013
  * [35] Brian Lau and Paul W Glimcher  “Dynamic Response-by-Response Models of Matching Behavior in Rhesus Monkeys”  In _Journal of the Experimental Analysis of Behavior_ 84.3, 2005, pp. 555–579  DOI: 10.1901/jeab.2005.110-04
  * [36] Klaus Linkenkaer-Hansen, Vadim V. Nikouline, J. Palva and Risto J. Ilmoniemi  “Long-Range Temporal Correlations and Scaling Behavior in Human Brain Oscillations”  In _Journal of Neuroscience_ 21.4, 2001, pp. 1370–1377  DOI: 10.1523/JNEUROSCI.21-04-01370.2001
  * [37] Richard J. Lipton and Kenneth W. Regan  “Georg Cantor: Diagonal Method”  In _People, Problems, and Proofs: Essays from Gödel’s Lost Letter: 2010_ Berlin, Heidelberg: Springer, 2013, pp. 83–86  DOI: 10.1007/978-3-642-41422-0_15
  * [38] Jan-Matthis Lueckmann, Jakob H. Macke and Hendrikje Nienborg  “Can Serial Dependencies in Choices and Neural Activity Explain Choice Probabilities?”  In _Journal of Neuroscience_ 38.14, 2018, pp. 3495–3506  DOI: 10.1523/JNEUROSCI.2225-17.2018
  * [39] Jakob H Macke and Hendrikje Nienborg  “Choice (-history) correlations in sensory cortex: cause or consequence?”  In _Current Opinion in Neurobiology_ 58, 2019, pp. 148–154  DOI: 10.1016/j.conb.2019.09.005
  * [40] David P. MacKinnon, Amanda J. Fairchild and Matthew S. Fritz  “Mediation Analysis”  In _Annual review of psychology_ 58, 2007, pp. 593  DOI: 10.1146/annurev.psych.58.110405.085542
  * [41] Madhur Mangalam and Damian G. Kelty-Stephen  “Point estimates, Simpson’s paradox, and nonergodicity in biological sciences”  In _Neuroscience & Biobehavioral Reviews_ 125, 2021, pp. 98–107  DOI: 10.1016/j.neubiorev.2021.02.017
  * [42] Encarni Marcos et al.  “Neural Variability in Premotor Cortex Is Modulated by Trial History and Predicts Behavioral Performance”  In _Neuron_ 78.2, 2013, pp. 249–255  DOI: 10.1016/j.neuron.2013.02.006
  * [43] Eve Marder  “Variability, compensation, and modulation in neurons and circuits”  In _Proceedings of the National Academy of Sciences_ 108.supplement_3, 2011, pp. 15542–15548  DOI: 10.1073/pnas.1010674108
  * [44] Ioana E. Marinescu, Patrick N. Lawlor and Konrad P. Kording  “Quasi-experimental causality in neuroscience and behavioural research”  In _Nature Human Behaviour_ 2.12, 2018, pp. 891–898  DOI: 10.1038/s41562-018-0466-5
  * [45] David Mehler and Konrad Kording  “The lure of causal statements: Rampant mis-inference of causality in estimated connectivity”  In _Arxiv_ , 2018  URL: https://arxiv.org/abs/1812.03363
  * [46] Peter C.. Molenaar  “A Manifesto on Psychology as Idiographic Science: Bringing the Person Back Into Scientific Psychology, This Time Forever”  In _Measurement: Interdisciplinary Research & Perspective_ 2.4, 2004, pp. 201–218  DOI: 10.1207/s15366359mea0204_1
  * [47] Peter C.M. Molenaar and Cynthia G. Campbell  “The New Person-Specific Paradigm in Psychology”  In _Current Directions in Psychological Science_ 18.2, 2009, pp. 112–117  URL: https://www.jstor.org/stable/20696008
  * [48] Yael Niv  “Reinforcement learning in the brain”  In _Journal of Mathematical Psychology_ 53.3, Special Issue: Dynamic Decision Making, 2009, pp. 139–154  DOI: 10.1016/j.jmp.2008.12.005
  * [49] Judea Pearl  “Causal inference in statistics: An overview”  In _Statistics Surveys_ 3.0, 2009, pp. 96–146  DOI: 10.1214/09-SS057
  * [50] P… Phillips  “Understanding spurious regressions in econometrics”  In _Journal of Econometrics_ 33.3, 1986, pp. 311–340  DOI: 10.1016/0304-4076(86)90001-1
  * [51] Mikhail Prokopenko et al.  “Self-referential basis of undecidable dynamics: From the Liar paradox and the halting problem to the edge of chaos”  In _Physics of Life Reviews_ 31, Physics of Mind, 2019, pp. 134–156  DOI: 10.1016/j.plrev.2018.12.003
  * [52] Alfonso Renart and Christian K. Machens  “Variability in neural activity and behavior”  In _Current Opinion in Neurobiology_ 25, 2014, pp. 211–220  DOI: 10.1016/j.conb.2014.02.013
  * [53] Lauren N. Ross and Dani S. Bassett  “Causation in neuroscience: keeping mechanism meaningful”  In _Nature Reviews. Neuroscience_ 25.2, 2024, pp. 81–90  DOI: 10.1038/s41583-023-00778-7
  * [54] Mostafa Safaie et al.  “Preserved neural dynamics across animals performing similar behaviour”  In _Nature_ 623.7988, 2023, pp. 765–771  DOI: 10.1038/s41586-023-06714-0
  * [55] Leonard J. Savage  “The Foundations of Statistics”  Wiley Publications in Statistics, 1954
  * [56] Shan H. Siddiqi, Konrad P. Kording, Josef Parvizi and Michael D. Fox  “Causal mapping of human brain function”  In _Nature Reviews Neuroscience_ 23.6, 2022, pp. 361–375  DOI: 10.1038/s41583-022-00583-8
  * [57] B.. Skinner  “Science and human behavior”, Science and human behavior  Oxford, England: Macmillan, 1953
  * [58] Elliott Sober  “Evolution, Population Thinking, and Essentialism”  In _Philosophy of Science_ 47.3, 1980, pp. 350–383  URL: https://www.jstor.org/stable/186950
  * [59] Elexa St.-Saaltink, Peter Kok, Hakwan C. Lau and Floris P. Lange  “Serial Dependence in Perceptual Decisions Is Reflected in Activity Patterns in Primary Visual Cortex”  In _Journal of Neuroscience_ 36.23, 2016
  * [60] Richard B. Stein, E. Gossen and Kelvin E. Jones  “Neuronal variability: noise or part of the signal?”  In _Nature Reviews Neuroscience_ 6.5, 2005, pp. 389–397  DOI: 10.1038/nrn1668
  * [61] Mark Stokes and Eelke Spaak  “The Importance of Single-Trial Analyses in Cognitive Neuroscience”  In _Trends in Cognitive Sciences_ 20.7, 2016, pp. 483–486  DOI: 10.1016/j.tics.2016.05.008
  * [62] Leo P. Sugrue, Greg S. Corrado and William T. Newsome  “Matching Behavior and the Representation of Value in the Parietal Cortex”  In _Science_ 304.5678, 2004, pp. 1782–1787  DOI: 10.1126/science.1094765
  * [63] Alan Turing  “On Computable Numbers, with an Application to the Entscheidungsproblem”  In _Proceedings of the London Mathematical Society_ 42.1, 1937, pp. 230–265  DOI: doi.org/10.1112/plms/s2-42.1.230
  * [64] Leonhard Waschke, Niels A. Kloosterman, Jonas Obleser and Douglas D. Garrett  “Behavior needs neural variability”  In _Neuron_ 109.5, 2021, pp. 751–766  DOI: 10.1016/j.neuron.2021.01.023
  * [65] Sebastian Weichwald and Jonas Peters  “Causality in Cognitive Neuroscience: Concepts, Challenges, and Distributional Robustness”  In _Journal of Cognitive Neuroscience_ 33.2, 2021, pp. 226–247  DOI: 10.1162/jocn_a_01623
  * [66] Alex H. Williams and Scott W. Linderman  “Statistical neuroscience in the single trial limit”  In _Current Opinion in Neurobiology_ 70, Computational Neuroscience, 2021, pp. 193–205  DOI: 10.1016/j.conb.2021.10.008

## References

  * [67] Odd O Aalen, Kjetil Røysland, Jon Michael Gran and Bruno Ledergerber  “Causality, mediation and time: a dynamic viewpoint”  In _Journal of the Royal Statistical Society. Series A, (Statistics in Society)_ 175.4, 2012, pp. 831–861  DOI: 10.1111/j.1467-985X.2011.01030.x
  * [68] C Allen and C F Stevens  “An evaluation of causes for unreliability of synaptic transmission.”  In _Proceedings of the National Academy of Sciences of the United States of America_ 91.22, 1994, pp. 10380–10383  DOI: 10.1073/pnas.91.22.10380
  * [69] E.T Bell  “The Development Of Mathematics”, 1945  URL: http://archive.org/details/in.ernet.dli.2015.133966
  * [70] Heather Christine Bell  “Behavioral Variability in the Service of Constancy”  In _International Journal of Comparative Psychology_ 27.2, 2014  DOI: 10.46867/ijcp.2014.27.02.02
  * [71] Tiago Branco and Kevin Staras  “The probability of neurotransmitter release: variability and feedback control at single synapses”  In _Nature Reviews Neuroscience_ 10.5, 2009, pp. 373–383  DOI: 10.1038/nrn2634
  * [72] Jerome Bruner  “A Short History of Psychological Theories of Learning”  In _Daedalus_ 133.1, 2004, pp. 13–20  URL: https://www.jstor.org/stable/20027892
  * [73] R… Carpenter  “A neural mechanism that randomises behaviour”  In _Journal of Consciousness Studies_ 6.1, 1999, pp. 13–22
  * [74] Anne K. Churchland et al.  “Variance as a signature of neural computations during decision making”  In _Neuron_ 69.4, 2011, pp. 818–831  DOI: 10.1016/j.neuron.2010.12.037
  * [75] Edward A Codling, Michael J Plank and Simon Benhamou  “Random walk models in biology”  In _Journal of The Royal Society Interface_ 5.25, 2008, pp. 813–834  DOI: 10.1098/rsif.2008.0014
  * [76] Gustavo Deco, Edmund T. Rolls and Ranulfo Romo  “Stochastic dynamics as a principle of brain function”  In _Progress in Neurobiology_ 88.1, 2009, pp. 1–16  DOI: 10.1016/j.pneurobio.2009.01.006
  * [77] Gustavo Deco and Ranulfo Romo  “The role of fluctuations in perception”  In _Trends in Neurosciences_ 31.11, 2008, pp. 591–598  DOI: 10.1016/j.tins.2008.08.007
  * [78] Ashesh K. Dhawale, Maurice A. Smith and Bence P. Ölveczky  “The Role of Variability in Motor Learning”  In _Annual Review of Neuroscience_ 40.1, 2017, pp. 479–498  DOI: 10.1146/annurev-neuro-072116-031548
  * [79] Nadine Dijkstra and Leon Bruin  “Cognitive Neuroscience and Causal Inference: Implications for Psychiatry”  In _Frontiers in Psychiatry_ 7, 2016, pp. 129  DOI: 10.3389/fpsyt.2016.00129
  * [80] Gerald M. Edelman and Joseph A. Gally  “Degeneracy and complexity in biological systems”  In _Proceedings of the National Academy of Sciences_ 98.24, 2001, pp. 13763–13768  DOI: 10.1073/pnas.231499798
  * [81] Peter W. Evans and Sally Shrapnel  “The Two Sides of Interventionist Causation”, 2017  URL: https://philsci-archive.pitt.edu/12906/
  * [82] A. Faisal, Luc P.. Selen and Daniel M. Wolpert  “Noise in the nervous system”  In _Nature Reviews Neuroscience_ 9.4, 2008, pp. 292–303  DOI: 10.1038/nrn2258
  * [83] Jillian H. Fecteau, Crystal Au, Irene T. Armstrong and Douglas P. Munoz  “Sensory biases produce alternation advantage found in sequential saccadic eye movement tasks”  In _Experimental Brain Research_ 159.1, 2004, pp. 84–91  DOI: 10.1007/s00221-004-1935-9
  * [84] Jason Fischer and David Whitney  “Serial dependence in visual perception”  In _Nature Neuroscience_ 17.5, 2014, pp. 738–743  DOI: 10.1038/nn.3689
  * [85] Aaron J. Fisher, John D. Medaglia and Bertus F. Jeronimus  “Lack of group-to-individual generalizability is a threat to human subjects research”  In _Proceedings of the National Academy of Sciences_ 115.27, 2018, pp. E6106–E6115  DOI: 10.1073/pnas.1711978115
  * [86] Michael D. Fox, Abraham Z. Snyder, Justin L. Vincent and Marcus E. Raichle  “Intrinsic Fluctuations within Cortical Systems Account for Intertrial Variability in Human Behavior”  In _Neuron_ 56.1, 2007, pp. 171–184  DOI: 10.1016/j.neuron.2007.08.023
  * [87] Juan Gao et al.  “Sequential Effects in Two-Choice Reaction Time Tasks: Decomposition and Synthesis of Mechanisms”  In _Neural Computation_ 21.9, 2009, pp. 2407–2436  DOI: 10.1162/neco.2009.09-08-866
  * [88] Gerd Gigerenzer  “Statistical rituals: The replication delusion and how we got there”  In _Advances in Methods and Practices in Psychological Science_ 1.2, 2018, pp. 198–218  DOI: 10.1177/2515245918771329
  * [89] Joshua I. Gold and Michael N. Shadlen  “The Neural Basis of Decision Making”  In _Annual Review of Neuroscience_ 30.1, 2007, pp. 535–574  DOI: 10.1146/annurev.neuro.29.051605.113038
  * [90] Alex Gomez-Marin  “Causal Circuit Explanations of Behavior: Are Necessity and Sufficiency Necessary and Sufficient?”  In _Decoding Neural Circuit Structure and Function_ Cham: Springer International Publishing, 2017, pp. 283–306  DOI: 10.1007/978-3-319-57363-2_11
  * [91] James E. Graham, Amol M. Karmarkar and Kenneth J. Ottenbacher  “Small Sample Research Designs for Evidence-based Rehabilitation: Issues and Methods”  In _Archives of physical medicine and rehabilitation_ 93.8, 2012, pp. S111–S116  DOI: 10.1016/j.apmr.2011.12.017
  * [92] C… Granger and P. Newbold  “Spurious regressions in econometrics”  In _Journal of Econometrics_ 2.2, 1974, pp. 111–120  DOI: 10.1016/0304-4076(74)90034-7
  * [93] Donald Hebb  “The Organization of Behaviour”, 1949
  * [94] Alexander Huk, Kathryn Bonnen and Biyu J. He  “Beyond Trial-Based Paradigms: Continuous Behavior, Ongoing Neural Activity, and Natural Stimuli”  In _The Journal of Neuroscience_ 38.35, 2018, pp. 7551–7558  DOI: 10.1523/JNEUROSCI.1920-17.2018
  * [95] Eyke Hüllermeier and Willem Waegeman  “Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods”  In _Machine Learning_ 110.3, 2021, pp. 457–506  DOI: 10.1007/s10994-021-05946-3
  * [96] Eric Jonas and Konrad Paul Kording  “Could a Neuroscientist Understand a Microprocessor?”  In _PLOS Computational Biology_ 13.1, 2017  DOI: doi.org/10.1371/journal.pcbi.1005268
  * [97] Tatjana Kanashiro, Gabriel Ocker, Marlene Cohen and Brent Doiron  “Attentional modulation of neuronal variability in circuit models of cortex”  In _eLife_ 6.e23978, 2017  DOI: 10.7554/eLife.23978
  * [98] Armen Der Kiureghian and Ove Ditlevsen  “Aleatory or epistemic? Does it matter?”  In _Structural Safety_ 31.2, Risk Acceptance and Risk Communication, 2009, pp. 105–112  DOI: 10.1016/j.strusafe.2008.06.020
  * [99] Konrad P. Kording, Joshua B. Tenenbaum and Reza Shadmehr  “The dynamics of memory as a consequence of optimal adaptation to a changing body”  In _Nature Neuroscience_ 10.6, 2007, pp. 779–786  DOI: 10.1038/nn1901
  * [100] Floris P. Lange, Dobromir A. Rahnev, Tobias H. Donner and Hakwan Lau  “Prestimulus Oscillatory Activity over Motor Cortex Reflects Perceptual Expectations”  In _Journal of Neuroscience_ 33.4, 2013, pp. 1400–1410  DOI: 10.1523/JNEUROSCI.1094-12.2013
  * [101] Brian Lau and Paul W Glimcher  “Dynamic Response-by-Response Models of Matching Behavior in Rhesus Monkeys”  In _Journal of the Experimental Analysis of Behavior_ 84.3, 2005, pp. 555–579  DOI: 10.1901/jeab.2005.110-04
  * [102] Klaus Linkenkaer-Hansen, Vadim V. Nikouline, J. Palva and Risto J. Ilmoniemi  “Long-Range Temporal Correlations and Scaling Behavior in Human Brain Oscillations”  In _Journal of Neuroscience_ 21.4, 2001, pp. 1370–1377  DOI: 10.1523/JNEUROSCI.21-04-01370.2001
  * [103] Richard J. Lipton and Kenneth W. Regan  “Georg Cantor: Diagonal Method”  In _People, Problems, and Proofs: Essays from Gödel’s Lost Letter: 2010_ Berlin, Heidelberg: Springer, 2013, pp. 83–86  DOI: 10.1007/978-3-642-41422-0_15
  * [104] Jan-Matthis Lueckmann, Jakob H. Macke and Hendrikje Nienborg  “Can Serial Dependencies in Choices and Neural Activity Explain Choice Probabilities?”  In _Journal of Neuroscience_ 38.14, 2018, pp. 3495–3506  DOI: 10.1523/JNEUROSCI.2225-17.2018
  * [105] Jakob H Macke and Hendrikje Nienborg  “Choice (-history) correlations in sensory cortex: cause or consequence?”  In _Current Opinion in Neurobiology_ 58, 2019, pp. 148–154  DOI: 10.1016/j.conb.2019.09.005
  * [106] David P. MacKinnon, Amanda J. Fairchild and Matthew S. Fritz  “Mediation Analysis”  In _Annual review of psychology_ 58, 2007, pp. 593  DOI: 10.1146/annurev.psych.58.110405.085542
  * [107] Madhur Mangalam and Damian G. Kelty-Stephen  “Point estimates, Simpson’s paradox, and nonergodicity in biological sciences”  In _Neuroscience & Biobehavioral Reviews_ 125, 2021, pp. 98–107  DOI: 10.1016/j.neubiorev.2021.02.017
  * [108] Encarni Marcos et al.  “Neural Variability in Premotor Cortex Is Modulated by Trial History and Predicts Behavioral Performance”  In _Neuron_ 78.2, 2013, pp. 249–255  DOI: 10.1016/j.neuron.2013.02.006
  * [109] Eve Marder  “Variability, compensation, and modulation in neurons and circuits”  In _Proceedings of the National Academy of Sciences_ 108.supplement_3, 2011, pp. 15542–15548  DOI: 10.1073/pnas.1010674108
  * [110] Ioana E. Marinescu, Patrick N. Lawlor and Konrad P. Kording  “Quasi-experimental causality in neuroscience and behavioural research”  In _Nature Human Behaviour_ 2.12, 2018, pp. 891–898  DOI: 10.1038/s41562-018-0466-5
  * [111] David Mehler and Konrad Kording  “The lure of causal statements: Rampant mis-inference of causality in estimated connectivity”  In _Arxiv_ , 2018  URL: https://arxiv.org/abs/1812.03363
  * [112] Peter C.. Molenaar  “A Manifesto on Psychology as Idiographic Science: Bringing the Person Back Into Scientific Psychology, This Time Forever”  In _Measurement: Interdisciplinary Research & Perspective_ 2.4, 2004, pp. 201–218  DOI: 10.1207/s15366359mea0204_1
  * [113] Peter C.M. Molenaar and Cynthia G. Campbell  “The New Person-Specific Paradigm in Psychology”  In _Current Directions in Psychological Science_ 18.2, 2009, pp. 112–117  URL: https://www.jstor.org/stable/20696008
  * [114] Yael Niv  “Reinforcement learning in the brain”  In _Journal of Mathematical Psychology_ 53.3, Special Issue: Dynamic Decision Making, 2009, pp. 139–154  DOI: 10.1016/j.jmp.2008.12.005
  * [115] Judea Pearl  “Causal inference in statistics: An overview”  In _Statistics Surveys_ 3.0, 2009, pp. 96–146  DOI: 10.1214/09-SS057
  * [116] P… Phillips  “Understanding spurious regressions in econometrics”  In _Journal of Econometrics_ 33.3, 1986, pp. 311–340  DOI: 10.1016/0304-4076(86)90001-1
  * [117] Mikhail Prokopenko et al.  “Self-referential basis of undecidable dynamics: From the Liar paradox and the halting problem to the edge of chaos”  In _Physics of Life Reviews_ 31, Physics of Mind, 2019, pp. 134–156  DOI: 10.1016/j.plrev.2018.12.003
  * [118] Alfonso Renart and Christian K. Machens  “Variability in neural activity and behavior”  In _Current Opinion in Neurobiology_ 25, 2014, pp. 211–220  DOI: 10.1016/j.conb.2014.02.013
  * [119] Lauren N. Ross and Dani S. Bassett  “Causation in neuroscience: keeping mechanism meaningful”  In _Nature Reviews. Neuroscience_ 25.2, 2024, pp. 81–90  DOI: 10.1038/s41583-023-00778-7
  * [120] Mostafa Safaie et al.  “Preserved neural dynamics across animals performing similar behaviour”  In _Nature_ 623.7988, 2023, pp. 765–771  DOI: 10.1038/s41586-023-06714-0
  * [121] Leonard J. Savage  “The Foundations of Statistics”  Wiley Publications in Statistics, 1954
  * [122] Shan H. Siddiqi, Konrad P. Kording, Josef Parvizi and Michael D. Fox  “Causal mapping of human brain function”  In _Nature Reviews Neuroscience_ 23.6, 2022, pp. 361–375  DOI: 10.1038/s41583-022-00583-8
  * [123] B.. Skinner  “Science and human behavior”, Science and human behavior  Oxford, England: Macmillan, 1953
  * [124] Elliott Sober  “Evolution, Population Thinking, and Essentialism”  In _Philosophy of Science_ 47.3, 1980, pp. 350–383  URL: https://www.jstor.org/stable/186950
  * [125] Elexa St.-Saaltink, Peter Kok, Hakwan C. Lau and Floris P. Lange  “Serial Dependence in Perceptual Decisions Is Reflected in Activity Patterns in Primary Visual Cortex”  In _Journal of Neuroscience_ 36.23, 2016
  * [126] Richard B. Stein, E. Gossen and Kelvin E. Jones  “Neuronal variability: noise or part of the signal?”  In _Nature Reviews Neuroscience_ 6.5, 2005, pp. 389–397  DOI: 10.1038/nrn1668
  * [127] Mark Stokes and Eelke Spaak  “The Importance of Single-Trial Analyses in Cognitive Neuroscience”  In _Trends in Cognitive Sciences_ 20.7, 2016, pp. 483–486  DOI: 10.1016/j.tics.2016.05.008
  * [128] Leo P. Sugrue, Greg S. Corrado and William T. Newsome  “Matching Behavior and the Representation of Value in the Parietal Cortex”  In _Science_ 304.5678, 2004, pp. 1782–1787  DOI: 10.1126/science.1094765
  * [129] Alan Turing  “On Computable Numbers, with an Application to the Entscheidungsproblem”  In _Proceedings of the London Mathematical Society_ 42.1, 1937, pp. 230–265  DOI: doi.org/10.1112/plms/s2-42.1.230
  * [130] Leonhard Waschke, Niels A. Kloosterman, Jonas Obleser and Douglas D. Garrett  “Behavior needs neural variability”  In _Neuron_ 109.5, 2021, pp. 751–766  DOI: 10.1016/j.neuron.2021.01.023
  * [131] Sebastian Weichwald and Jonas Peters  “Causality in Cognitive Neuroscience: Concepts, Challenges, and Distributional Robustness”  In _Journal of Cognitive Neuroscience_ 33.2, 2021, pp. 226–247  DOI: 10.1162/jocn_a_01623
  * [132] Alex H. Williams and Scott W. Linderman  “Statistical neuroscience in the single trial limit”  In _Current Opinion in Neurobiology_ 70, Computational Neuroscience, 2021, pp. 193–205  DOI: 10.1016/j.conb.2021.10.008