# The Stochastic-Quantum Correspondence

Jacob A. Barandes<EMAIL_ADDRESS>Jefferson Physical Laboratory,
Harvard University, Cambridge, MA 02138

(September 3, 2024)

###### Abstract

This paper introduces an exact correspondence between a general class of
stochastic systems and quantum theory. This correspondence provides a new
framework for using Hilbert-space methods to formulate highly generic, non-
Markovian types of stochastic dynamics, with potential applications throughout
the sciences. This paper also uses the correspondence in the other direction
to reconstruct quantum theory from physical models that consist of
trajectories in configuration spaces undergoing stochastic dynamics. The
correspondence thereby yields a new formulation of quantum theory, alongside
the Hilbert-space, path-integral formulations, and quasiprobability
formulations. In addition, this reconstruction approach opens up new ways of
understanding quantum phenomena like interference, decoherence, entanglement,
noncommutative observables, and wave-function collapse.

##  I Introduction

The theory of stochastic processes describes the phenomenological behavior of
systems with definite configurations that evolve probabilistically in time.
Quantum theory is a comprehensive mathematical apparatus for making
measurement predictions when taking into account the microscopic constituents
of various kinds of physical systems, from subatomic particles to
superconductors. At an empirical level, both theories involve probabilities,
and at the level of formalism, both employ vectors and matrices.

There have been a number of previous attempts in the research literature to
identify a fundamental relationship connecting stochastic-process theory and
quantum theory [1, 2, 3, 4, 5, 6, 7]. The most well-known of these approaches
are due to Bopp [8, 9, 10], Fényes [11], and Nelson [12, 13]. Altogether
different are stochastic-collapse models [14, 15], in which a quantum system’s
wave function or density matrix is assumed to experience stochastic
fluctuations through time.

The present paper, which is not continuous with those earlier developments,
introduces an exact correspondence between a highly general class of
stochastic systems and quantum theory. This _stochastic-quantum
correspondence_ takes the form of a simple ‘dictionary’ expressing any time-
dependent stochastic matrix in terms of a suitable combination of Hilbert-
space ingredients.

From a practical standpoint, the stochastic-quantum correspondence provides a
systematic framework for constructing highly generic forms of stochastic
dynamics, much as the Lagrangian or Hamiltonian formulations of classical
mechanics provide systematic frameworks for constructing deterministic
dynamics. Potential applications range from turbulence to finance, to name
just two examples. Importantly, the stochastic-quantum correspondence does not
require assuming that the stochastic dynamics in question can be modeled as a
Markov chain, nor does it require making a number of other frequently invoked
approximations.

Taking a more foundational perspective, this paper also uses the stochastic-
quantum correspondence to show that physical models based on configuration
spaces combined with stochastic dynamics can replicate all the empirical
predictions of textbook quantum theory—including interference, decoherence,
entanglement, noncommutative observables, and wave-function collapse—without
relying on the austere and metaphysically opaque Dirac-von Neumann axioms [16,
17]. In this alternative approach, a given system moves stochastically along a
physical trajectory in a classical-looking configuration space, and the
mathematical objects of the Hilbert-space formulation play a functional role
akin to gauge-dependent variables.

At the very least, this approach therefore yields a new formulation of quantum
theory, one that is based on a picture of stochastic systems evolving in
configuration spaces. This new formulation therefore joins a list of ways to
formulate quantum systems that include the traditional Hilbert-space
formulation [16, 17], the path-integral formulation [18, 19, 20], and the
quasi-probability formulation [21, 22]. As noted in [20], “there is a pleasure
in recognizing old things from a new point of view,” and “there is always the
hope that the new point of view will inspire an idea for the modification of
present theories, a modification necessary to encompass present experiments.”

In addition to establishing these new results, this paper identifies several
forms of gauge invariance that have not been described in the research
literature thus far, analyzes the measurement process in detail, and describes
the implications of the stochastic-quantum correspondence for dynamical
symmetries and for formal enlargements or dilations of a system’s Hilbert
space. Taking advantage of having a concrete model of stochastic physical
variables in hand, this paper also revisits and clarifies a number of
important questions related to the status of nonlocality in quantum theory.

Considering the mathematical simplicity of this stochastic-quantum
correspondence, it is surprising that it has apparently not shown up in the
research literature before. To the author’s knowledge, the only previous
example that bears a suggestive resemblance to the approach taken in this
paper, at least at the level of some of its equations, is the unpublished
draft [23].111The author thanks Logan McCarty for finding this reference.
Although that reference argues that some stochastic processes can be modeled
using a formalism similar to that of quantum theory, it does not establish
that the resulting Hilbert-space representation is fully general. Nor does it
attempt to show that the correspondence is bidirectional, so that quantum
systems can be modeled by stochastic processes in configuration spaces.

Section II will start with the definition of a _generalized stochastic system_
, and then introduce the key distinction between _divisible_ and _indivisible_
dynamics. Section III will construct the Hilbert-space representation for a
given generalized stochastic system, formulate the dictionary that will
ultimately connect generalized stochastic systems with quantum systems, and
identify an important new class of gauge transformations that have not yet
been described in the research literature. Section IV will establish the
stochastic-quantum correspondence, which will involve reviewing the definition
of _unistochastic matrices_ , introducing the concept of a _division event_ ,
and showing how some of the most characteristic features of any quantum
system—like interference, entanglement, and decoherence—can be understood from
the perspective of the underlying generalized stochastic system. Section V
will provide a detailed treatment of the measurement process, which will
entail introducing the notion of an _emergeable_ , and then turn to a larger
discussion of the measurement problem and the uncertainty principle. Section
VI will describe further implications of the stochastic-quantum correspondence
for quantum theory, focusing on symmetries, Hilbert-space dilations, and
nonlocality. Section VII will conclude the paper with a brief discussion, as
well as with several open questions to be addressed in future work.

##  II Stochastic Processes

###  II.1 Generalized Stochastic Systems

The most general kind of stochastic process requires only a sample space, an
initial probability distribution, and a time-dependent random variable. (For
pedagogical treatments of the theory of stochastic processes, see [24, 25, 26,
27].) Stochastic processes defined in this way lack an ingredient that plays
the role of a _dynamical law_. This paper will be concerned with a slightly
more narrowly defined construction that includes the notion of a dynamical
law, and that can still describe a wide variety of physical systems.

A _generalized stochastic system_ will be defined to consist of a
_configuration space_ $\mathcal{C}$ together with a dynamical law in the form
of a _stochastic map_ $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ that
acts linearly on probability distributions over $\mathcal{C}$ at an initial
time $0$ to yield corresponding probability distributions over $\mathcal{C}$
at other times $t\neq 0$. For the purposes of this paper, the set of allowed
times will be assumed to be isomorphic to a subset of the real line
$\mathbb{R}$ containing $0$, up to a choice of measurement units.

The formalism for a generalized stochastic system is easiest to express in the
case in which $\mathcal{C}$ has a finite number $N$ of configurations labeled
by positive integers $1,\dots,N$:

$\mathcal{C}\equiv\mathopen{}\mathclose{{}\left\\{1,\dots,N}\right\\}.$ (1)

In that case, the system’s _standalone probabilities_ at the initial time $0$
can be denoted by

$p_{j}\mathopen{}\mathclose{{}\left(0}\right)\quad\mathopen{}\mathclose{{}\left[\textrm{for
}j=1,\dots,N}\right],$ (2)

the system’s standalone probabilities at $t\neq 0$ can be denoted by

$p_{i}\mathopen{}\mathclose{{}\left(t}\right)\quad\mathopen{}\mathclose{{}\left[\textrm{for
}i=1,\dots,N}\right],$ (3)

and the stochastic map consists of _conditional probabilities_

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)\equiv
p\mathopen{}\mathclose{{}\left(i,t|j,0}\right)\quad\mathopen{}\mathclose{{}\left[\textrm{for
}i,j=1,\dots,N}\right],$ (4)

where $p\mathopen{}\mathclose{{}\left(i,t|j,0}\right)$ denotes the conditional
probability for the system to be in its $i$th configuration at the time $t$,
given that the system is in its $j$th configuration at the initial time $0$.
(Note that no assumption is made here about whether $t>0$ or $t<0$.) Being
probabilities, these quantities satisfy the usual non-negativity and
normalization conditions

$p_{j}\mathopen{}\mathclose{{}\left(0}\right),p_{i}\mathopen{}\mathclose{{}\left(t}\right)\geq
0,\quad\sum_{j=1}^{N}p_{j}\mathopen{}\mathclose{{}\left(0}\right)=\sum_{i=1}^{N}p_{i}\mathopen{}\mathclose{{}\left(t}\right)=1,$
(5)

and

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)\geq
0,\qquad\sum_{i=1}^{N}\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=1.$
(6)

Then from _Bayesian marginalization_ , one has the _linear_ relationship

$p_{i}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{j=1}^{N}\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)p_{j}\mathopen{}\mathclose{{}\left(0}\right),$
(7)

where the initial standalone probabilities
$p_{j}\mathopen{}\mathclose{{}\left(0}\right)$ are assumed to be arbitrary and
can therefore be freely adjusted without altering the conditional
probabilities $\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)$.

Letting $p\mathopen{}\mathclose{{}\left(0}\right)$ denote an $N\times 1$
_probability vector_ whose entries are given by the standalone probabilities
$p_{j}\mathopen{}\mathclose{{}\left(0}\right)$, letting
$p\mathopen{}\mathclose{{}\left(t}\right)$ denote the analogous $N\times 1$
probability vector with entries given by
$p_{i}\mathopen{}\mathclose{{}\left(t}\right)$, and letting
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ denote the $N\times N$ time-
dependent _transition matrix_ consisting of the conditional probabilities
$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)$, one can naturally recast
the linear Bayesian marginalization relationship (7) in matrix form as

$p\mathopen{}\mathclose{{}\left(t}\right)=\Gamma\mathopen{}\mathclose{{}\left(t}\right)p\mathopen{}\mathclose{{}\left(0}\right).$
(8)

The conditions (6) on the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ identify it, mathematically
speaking, as a _(column) stochastic matrix_. On physical grounds,
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ will be assumed to satisfy the
continuity condition that in the limit $t\to 0$, it approaches its initial
value $\Gamma\mathopen{}\mathclose{{}\left(0}\right)$, which will be taken to
be the $N\times N$ identity matrix $\mymathbb{1}$:

$\lim_{t\to
0}\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\Gamma\mathopen{}\mathclose{{}\left(0}\right)\equiv\mymathbb{1}\equiv\mathrm{diag}\mathopen{}\mathclose{{}\left(1,\dots,1}\right).$
(9)

Next, consider a _random variable_ $A\mathopen{}\mathclose{{}\left(t}\right)$
with (not necessarily distinct) real-valued _magnitudes_
$a_{1}\mathopen{}\mathclose{{}\left(t}\right),\dots,a_{N}\mathopen{}\mathclose{{}\left(t}\right)$
determined by the system’s configuration $i=1,\dots,N$, and possibly also
depending explicitly on the time $t$. Then the _expectation value_
$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle$ is defined as the
statistical average or mean of the magnitudes of
$A\mathopen{}\mathclose{{}\left(t}\right)$ over the system’s standalone
probability distribution at $t$:

$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle\equiv\sum_{i=1}^{N}a_{i}\mathopen{}\mathclose{{}\left(t}\right)p_{i}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{i=1}^{N}\sum_{j=1}^{N}a_{i}\mathopen{}\mathclose{{}\left(t}\right)\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)p_{j}\mathopen{}\mathclose{{}\left(0}\right).$
(10)

One can go on to define the standard deviation and various statistical moments
of $A\mathopen{}\mathclose{{}\left(t}\right)$ by appropriate generalizations
of this basic definition.

All these formulas can be extended to systems with continuous configuration
spaces. For a system with a continuous configuration space $\mathcal{C}$, one
uses standalone probability densities
$p\mathopen{}\mathclose{{}\left(y,0}\right)$ at the initial time $0$ and
$p\mathopen{}\mathclose{{}\left(x,t}\right)$ at $t\neq 0$, where $x$ and $y$
each symbolically denotes a set of real-valued coordinates. The Bayesian
marginalization relationship (7) then becomes

$p\mathopen{}\mathclose{{}\left(x,t}\right)=\int_{\mathcal{C}}d\mu\mathopen{}\mathclose{{}\left(y}\right)\,\Gamma\mathopen{}\mathclose{{}\left(x,y,t}\right)p\mathopen{}\mathclose{{}\left(y,0}\right),$
(11)

where $d\mu\mathopen{}\mathclose{{}\left(y}\right)$ is a suitable integral
measure over $\mathcal{C}$ and where the conditional probability density
$\Gamma\mathopen{}\mathclose{{}\left(x,y,t}\right)$ naturally serves as an
integral kernel. A random variable $A\mathopen{}\mathclose{{}\left(t}\right)$
then has magnitudes $a\mathopen{}\mathclose{{}\left(x,t}\right)$ labeled by
$x$ and $t$, and its expectation value (10) becomes

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle\equiv\int_{\mathcal{C}}d\mu\mathopen{}\mathclose{{}\left(x}\right)\,a\mathopen{}\mathclose{{}\left(x,t}\right)p\mathopen{}\mathclose{{}\left(x,t}\right)\\\
&\quad=\int_{\mathcal{C}}d\mu\mathopen{}\mathclose{{}\left(x}\right)\int_{\mathcal{C}}d\mu\mathopen{}\mathclose{{}\left(y}\right)\,a\mathopen{}\mathclose{{}\left(x,t}\right)\Gamma\mathopen{}\mathclose{{}\left(x,y,t}\right)p\mathopen{}\mathclose{{}\left(y,0}\right).\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(12)

For ease of exposition, the discrete case will be assumed going forward.

Equations like (7), (9), and (11) may appear to single out the initial time
$0$ as a special time. Subsection IV.6, however, will show that for systems in
sufficiently strong contact with a repeatedly eavesdropping environment, the
initial time $0$ need not actually be a unique time, but will typically be
only one of many times that play a similar role.

###  II.2 Traditional Approximations

In textbook treatments of stochastic processes, one often introduces various
approximations or simplifications in defining a system’s time-dependent
transition matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ to make it
easier to construct and describe. A typical such approximation is to assume
that the system is a _discrete-time Markov chain_ , meaning that for some
small but finite time scale $\delta t$, one can express the time-dependent
transition matrix $\Gamma\mathopen{}\mathclose{{}\left(n\,\delta t}\right)$ at
any integer number $n\geq 1$ of steps of duration $\delta t$ as $n$ powers of
a constant stochastic matrix $\Gamma$:

$\Gamma\mathopen{}\mathclose{{}\left(n\,\delta t}\right)=\Gamma^{n}.$ (13)

Somewhat more generally, a convenient simplification is to assume that for any
two times $t$ and $t^{\prime}$ satisfying $t>t^{\prime}>0$, one has the
composition law

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(14)

This simplification is known as _divisibility_ [28], a term that seems to have
first appeared in the research literature in a 2008 paper [29] by Wolf and
Cirac in the context of quantum channels.222Note that this notion of
divisibility is unrelated to the much older concept of _infinite divisibility_
, which refers to a probability distribution that can be expressed as the
probability distribution of a sum of any integer number of independent and
identically distributed random variables. Here
$\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)$ is
likewise required to satisfy the mathematical requirements of being a
stochastic matrix, in the sense that its entries are all non-negative and its
columns each sum to $1$, as in (6).

An even more special simplification is to take the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ to be a time-dependent
_permutation matrix_ , meaning a matrix whose columns are a permutation of the
columns of the $N\times N$ identity matrix $\mymathbb{1}$. In that case,
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ contains only $1$s and $0$s,
so it does not contain nontrivial probabilities at all, and the system
transitions deterministically from one configuration to another in its
configuration space $\mathcal{C}$. In a suitable continuum limit $N\to\infty$,
the time evolution reduces to smooth, deterministic dynamics.

Absent these sorts of approximations or simplifications, one is confronted
with the task of constructing an $N\times N$ time-dependent, generically non-
Markovian, _indivisible_ transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ for a given configuration
space $\mathcal{C}$, ideally in a systematic way. For small configuration
spaces, it is easy to devise smoothly time-dependent, non-Markovian,
indivisible examples, like the $2\times 2$ transition matrix

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)\equiv\begin{pmatrix}e^{-t^{2}/\tau^{2}}&1-e^{-t^{2}/\tau^{2}}\\\
1-e^{-t^{2}/\tau^{2}}&e^{-t^{2}/\tau^{2}}\end{pmatrix},$ (15)

where $\tau$ is a constant with units of time, or

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)\equiv\begin{pmatrix}\cos^{2}\omega
t&\sin^{2}\omega t\\\ \sin^{2}\omega t&\cos^{2}\omega t\end{pmatrix},$ (16)

where $\omega$ is a constant with units of inverse-time. However, it may not
seem obvious how to construct smoothly time-dependent transition matrices
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ systematically beyond the
$2\times 2$ case, especially in the case of large ($N\gg 1$) configuration
spaces.

A sufficiently general approach for accomplishing this task could have
numerous practical applications in many scientific and technical fields. This
paper will develop one such approach, and use it to provide a self-contained
theoretical justification for why the Markov and divisibility approximations
work so well in many real-world cases.

##  III The Hilbert-Space Formulation

###  III.1 The Schur-Hadamard Factorization

One of the goals of this paper will be to introduce a new and highly general
framework for formulating time-dependent transition matrices
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$, conceptually akin to the
Lagrangian or Hamiltonian frameworks for formulating deterministic dynamics
for mechanical systems.

The starting place will be to ‘solve’ the non-negativity condition
$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)\geq 0$ on the individual
entries of the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ by expressing them in the
following way:

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=|\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)|^{2}.$
(17)

This equation is not a postulate—it is a mathematical identity.

The $N\times N$ matrix $\Theta\mathopen{}\mathclose{{}\left(t}\right)$
introduced in (17) is guaranteed to exist, although it is not unique. Its
entries $\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)$ could be taken to
be the real square roots of the corresponding quantities
$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)$, but they could also
include complex numbers, quaternions, or even the elements of a more general
normed algebra (although associativity is a very helpful property to require).
To keep things simple, and with the eventual goal of reproducing the usual
formalism of quantum theory, this paper will choose
$\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)$ to involve only at most
the complex numbers.

On account of the normalization condition on the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ specified in (6), note that
the matrix $\Theta\mathopen{}\mathclose{{}\left(t}\right)$ must satisfy the
summation condition

$\sum_{i=1}^{N}|\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)|^{2}=1.$
(18)

For now, no further conditions, such as unitarity, will be imposed on
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$, whose significance will soon
become more clear.

There are several helpful ways to re-express the identity (17). To begin, one
can introduce the _Schur-Hadamard product_ $\odot$, which is defined for
arbitrary $N\times N$ matrices $X$ and $Y$ as entry-wise multiplication [30,
31, 32]:

$\mathopen{}\mathclose{{}\left(X\odot Y}\right)_{ij}\equiv X_{ij}Y_{ij}.$ (19)

One can then regard (17) as expressing the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ as a _Schur-Hadamard
factorization_ of the complex-conjugated matrix
$\overline{\Theta\mathopen{}\mathclose{{}\left(t}\right)}$ with
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ itself:

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\overline{\Theta\mathopen{}\mathclose{{}\left(t}\right)}\odot\Theta\mathopen{}\mathclose{{}\left(t}\right).$
(20)

Schur-Hadamard products are not widely used in linear algebra, in part because
they are basis-dependent. For the purposes of analyzing a given generalized
stochastic system, however, this basis-dependence is unimportant, because the
system’s configuration space $\mathcal{C}$ naturally singles out a specific
basis, to be defined momentarily.

###  III.2 The Dictionary

As an alternative approach that will turn out to have significant
ramifications, one starts by defining an $N$-member collection of $N\times N$
constant, diagonal projection matrices $P_{1},\dots,P_{N}$, which will be
called _configuration projectors_. For each $i=1,\dots,N$, the configuration
projector $P_{i}$ consists of a single $1$ in its $i$th row, $i$th column, and
$0$s in all its other entries. That is, $P_{i}$ is defined as

$P_{i}\equiv\mathrm{diag}(0,\dots,0,\underset{\mathclap{\begin{subarray}{c}\uparrow\\\
i\textrm{th entry}\end{subarray}}}{1},0,\dots,0),$ (21)

with individual entries

$P_{i,jk}=\delta_{ij}\delta_{ik},$ (22)

where $\delta_{ij}$ is the usual Kronecker delta:

$\delta_{ij}\equiv\begin{cases}1&\textrm{for }i=j,\\\ 0&\textrm{for }i\neq
j.\end{cases}$ (23)

It follows immediately that the configuration projectors satisfy the
conditions of _mutual exclusivity_ ,

$P_{i}P_{j}=\delta_{ij}P_{i},$ (24)

and _completeness_ ,

$\sum_{i=1}^{N}P_{i}=\mymathbb{1},$ (25)

where again $\mymathbb{1}$ is the $N\times N$ identity matrix. These two
conditions are the defining features of a _projection-valued measure (PVM)_
[33, 34], so the configuration projectors $P_{1},\dots,P_{N}$ naturally
constitute a PVM.

Letting $\mathrm{tr}\mathopen{}\mathclose{{}\left(\cdots}\right)$ denote the
usual matrix trace, one can then recast the mathematical identity (17)
relating the entries of $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ with
the entries of $\Theta\mathopen{}\mathclose{{}\left(t}\right)$ as

$\boxed{\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)P_{i}\Theta\mathopen{}\mathclose{{}\left(t}\right)P_{j}).}$
(26)

This equation is a new result. It will turn out to serve as an important
_dictionary_ that translates between the formalism of generalized stochastic
processes, as symbolized by
$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)$ on the left-hand side,
and an expansive set of mathematical tools for constructing stochastic
dynamics, as embodied by the right-hand side.333Similar-looking formulas
appear incidentally in equations (3)–(6) of [35] as an intermediate step in
proving a lemma that the authors use for conceptually different purposes.

###  III.3 The Hilbert Space

To understand what these mathematical tools are, the next step will be to
introduce a set of $N\times 1$ column vectors $e_{1},\dots,e_{N}$, where
$e_{i}$ has a $1$ in its $i$th component and $0$s in all its other components:

$e_{1}\equiv\mathopen{}\mathclose{{}\left(\begin{smallmatrix}1\\\ 0\\\
\vdots\\\ 0\\\ 0\end{smallmatrix}}\right),\quad\dots,\quad
e_{N}\equiv\mathopen{}\mathclose{{}\left(\begin{smallmatrix}0\\\ 0\\\
\vdots\\\ 0\\\ 1\end{smallmatrix}}\right).$ (27)

That is, $e_{i}$ has components

$e_{i,j}=\delta_{ij}.$ (28)

It follows that the column vectors $e_{1},\dots,e_{N}$ form an orthonormal
basis for the vector space of all $N\times 1$ column vectors, and
$e_{1},\dots,e_{N}$ will be called the system’s _configuration basis_. In
particular,

$e_{i}^{\dagger}e_{j}=\delta_{ij},\quad e_{i}e_{i}^{\dagger}=P_{i},$ (29)

where $P_{i}$ is the $i$th configuration projector, as defined in (21).

Hence, the right-hand side of the dictionary (26) is a trace over a _Hilbert
space_ , meaning a complete inner-product space over the complex numbers. More
explicitly, the dictionary picks out a Hilbert space $\mathcal{H}$ that is
isomorphic to the vector space $\mathbb{C}^{N}$ of $N\times 1$ column vectors
with complex-valued entries, under the inner product $v^{\dagger}w$:

$\mathcal{H}\cong\mathbb{C}^{N}.$ (30)

The dictionary therefore provides a Hilbert-space formulation for constructing
highly generic forms of stochastic dynamics.

Substituting the right-hand side of the dictionary (26) into the linear
Bayesian marginalization relationship (7) between the system’s standalone
probabilities $p_{j}\mathopen{}\mathclose{{}\left(0}\right)$ at the initial
time $0$ and the standalone probabilities
$p_{i}\mathopen{}\mathclose{{}\left(t}\right)$ at $t\neq 0$, one finds that

$p_{i}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(P_{i}\rho\mathopen{}\mathclose{{}\left(t}\right)),$
(31)

where $\rho\mathopen{}\mathclose{{}\left(t}\right)$ is an $N\times N$ time-
dependent, self-adjoint, unit-trace, generically non-diagonal matrix defined
as

$\mathopen{}\mathclose{{}\left.\begin{aligned}
\rho\mathopen{}\mathclose{{}\left(t}\right)&\equiv\Theta\mathopen{}\mathclose{{}\left(t}\right)\mathopen{}\mathclose{{}\left[\sum_{j=1}^{N}p_{j}\mathopen{}\mathclose{{}\left(0}\right)P_{j}}\right]\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)\\\
&=\Theta\mathopen{}\mathclose{{}\left(t}\right)\mathrm{diag}\mathopen{}\mathclose{{}\left(\dots,p_{j}\mathopen{}\mathclose{{}\left(0}\right),\dots}\right)\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right),\\\
&\qquad\qquad\textrm{with}\quad\rho^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)=\rho\mathopen{}\mathclose{{}\left(t}\right)\\\
&\qquad\qquad\textrm{and}\quad\mathrm{tr}(\rho\mathopen{}\mathclose{{}\left(t}\right))=1.\end{aligned}\qquad}\right\\}$
(32)

Crucially, notice how the linearity of the Bayesian marginalization
relationship (7) is ultimately responsible for the linearity of the
relationship between the matrix $\rho\mathopen{}\mathclose{{}\left(t}\right)$
and its value $\rho\mathopen{}\mathclose{{}\left(0}\right)$ at the initial
time $0$, as expressed in (32).

Similarly, by substituting the formula (31) for
$p_{i}\mathopen{}\mathclose{{}\left(t}\right)$ into the definition (10) of the
expectation value of a random variable
$A\mathopen{}\mathclose{{}\left(t}\right)$, one obtains

$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle=\mathrm{tr}(A\mathopen{}\mathclose{{}\left(t}\right)\rho\mathopen{}\mathclose{{}\left(t}\right)).$
(33)

Here $A\mathopen{}\mathclose{{}\left(t}\right)$ is now understood to be the
$N\times N$ time-dependent, diagonal matrix whose entries are the random
variable’s individual magnitudes
$a_{1}\mathopen{}\mathclose{{}\left(t}\right),\dots,a_{N}\mathopen{}\mathclose{{}\left(t}\right)$:

$A\mathopen{}\mathclose{{}\left(t}\right)\equiv\sum_{i=1}^{N}a_{i}\mathopen{}\mathclose{{}\left(t}\right)P_{i}=\mathrm{diag}\mathopen{}\mathclose{{}\left(\dots,a_{i}\mathopen{}\mathclose{{}\left(t}\right),\dots}\right).$
(34)

In the special case in which the system’s standalone probability distribution
at the initial time $0$ is pure, meaning that one of the system’s
configurations $j$ is occupied with probability $1$, the system’s probability
vector at the initial time $0$ is equal to the $j$th vector $e_{j}$ in the
configuration basis (28):

$p\mathopen{}\mathclose{{}\left(0}\right)=e_{j}=\mathopen{}\mathclose{{}\left(\begin{smallmatrix}0\\\
\vdots\\\ 0\\\ 1\\\ 0\\\ \vdots\\\
0\end{smallmatrix}}\right)\negthickspace\negthickspace\negthickspace\negthickspace\leftarrow
j\textrm{th entry}\quad\mathopen{}\mathclose{{}\left[\textrm{pure}}\right].$
(35)

One can then define an $N\times 1$ column vector

$\Psi\mathopen{}\mathclose{{}\left(t}\right)\equiv\Theta\mathopen{}\mathclose{{}\left(t}\right)e_{j},$
(36)

which is ultimately just the $j$th column of
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$. Due to the summation
condition (18) on $\Theta\mathopen{}\mathclose{{}\left(t}\right)$, this column
vector $\Psi\mathopen{}\mathclose{{}\left(t}\right)$ automatically has unit
unit norm, in the sense that

$\Psi^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)\Psi\mathopen{}\mathclose{{}\left(t}\right)=1.$
(37)

Moreover, the $i$th component
$\Psi_{i}\mathopen{}\mathclose{{}\left(t}\right)$ of
$\Psi\mathopen{}\mathclose{{}\left(t}\right)$ is equal to the specific
complex-valued matrix entry
$\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)$:

$\Psi_{i}\mathopen{}\mathclose{{}\left(t}\right)=\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right).$
(38)

This component $\Psi_{i}\mathopen{}\mathclose{{}\left(t}\right)$ is a purely
law-like quantity, in the sense of being just another name for a part of
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$, which is itself just a way of
encoding the system’s dynamical law, as embodied by the system’s transition
matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$.

It follows from a short calculation that when the purity condition (35) holds
at the initial time $0$, the self-adjoint matrix
$\rho\mathopen{}\mathclose{{}\left(t}\right)$ defined in (32) is rank-one and
has factorization

$\rho\mathopen{}\mathclose{{}\left(t}\right)=\Psi\mathopen{}\mathclose{{}\left(t}\right)\Psi^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)\quad\mathopen{}\mathclose{{}\left[\textrm{pure}}\right].$
(39)

The probability formula (31) then simplifies to

$p_{i}\mathopen{}\mathclose{{}\left(t}\right)=|\Psi_{i}\mathopen{}\mathclose{{}\left(t}\right)|^{2},$
(40)

and the formula (33) for the expectation value of a random variable
$A\mathopen{}\mathclose{{}\left(t}\right)$ becomes

$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle=\Psi^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)A\mathopen{}\mathclose{{}\left(t}\right)\Psi\mathopen{}\mathclose{{}\left(t}\right).$
(41)

Looking at all these results, one notices a striking resemblance to
mathematical objects and formulas that are familiar from textbook quantum
theory. Specifically, one sees that
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ plays the role of a _time-
evolution operator_ , $\rho\mathopen{}\mathclose{{}\left(t}\right)$ is a
_density matrix,_ $\Psi\mathopen{}\mathclose{{}\left(t}\right)$ is a _state
vector_ or _wave function_ , and $A\mathopen{}\mathclose{{}\left(t}\right)$
represents an _observable_.444Note that for the purposes of this paper, the
terms ‘operator’ and ‘matrix’ will be used interchangeably, as will the terms
‘state vector’ and ‘wave function.’ The probability formulas (31) and (40)
have the same form as the _Born rule_ , and (33) and (41) have the same form
as the standard expressions for quantum expectation values. (For pedagogical
treatments of quantum theory, see [36, 37, 38, 39, 40].)

These formulas are all expressed in what would conventionally be called the
_Schrödinger picture_. It will also end up being useful to introduce the
_Heisenberg picture_ , which is defined according to

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\rho^{H}\equiv\rho\mathopen{}\mathclose{{}\left(0}\right),\quad\Psi^{H}\equiv\Psi\mathopen{}\mathclose{{}\left(0}\right),\\\
&\quad
A^{H}\mathopen{}\mathclose{{}\left(t}\right)\equiv\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)A\mathopen{}\mathclose{{}\left(t}\right)\Theta\mathopen{}\mathclose{{}\left(t}\right),\end{aligned}\qquad}\right\\}$
(42)

where $A^{H}\mathopen{}\mathclose{{}\left(t}\right)$ now includes both a
possible _explicit_ dependence on time through its magnitudes
$a_{i}\mathopen{}\mathclose{{}\left(t}\right)$ as well as an _implicit_
dependence on time through the time-evolution operator
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$. In the Heisenberg picture,
the probability formula (31) becomes555Note that for a generic time-evolution
operator $\Theta\mathopen{}\mathclose{{}\left(t}\right)$, the Heisenberg-
picture version
$P_{i}^{H}\mathopen{}\mathclose{{}\left(t}\right)\equiv\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)P_{i}\Theta\mathopen{}\mathclose{{}\left(t}\right)$
of a projector $P_{i}$ will not likewise be a projector.

$p_{i}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(P^{H}\mathopen{}\mathclose{{}\left(t}\right)\rho^{H}),$
(43)

and the formula (33) for expectation values becomes

$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle=\mathrm{tr}(A^{H}\mathopen{}\mathclose{{}\left(t}\right)\rho^{H}).$
(44)

Despite the similarity to expressions found in quantum theory, as well as the
appearance of non-diagonal matrices, it is important to keep in mind that the
system under investigation here is always fundamentally in a specific
configuration $i=1,\dots,N$ in its configuration space $\mathcal{C}$ at any
given time, and that the system’s dynamics is completely captured by the
transition matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$, whose
entries are conditional probabilities
$p\mathopen{}\mathclose{{}\left(i,t|j,0}\right)$, in accordance with (4). The
mathematical objects $\Theta\mathopen{}\mathclose{{}\left(t}\right)$,
$\rho\mathopen{}\mathclose{{}\left(t}\right)$,
$\Psi\mathopen{}\mathclose{{}\left(t}\right)$, and
$A\mathopen{}\mathclose{{}\left(t}\right)$, despite being extremely useful,
are difficult to assign direct physical meanings, in part because they are not
uniquely defined by $\mathcal{C}$ or
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$.

###  III.4 Gauge Transformations

To make this non-uniqueness more manifest, it will be helpful to introduce an
analogy with the _Maxwell theory of classical electromagnetism_. (For
pedagogical treatments of classical electromagnetism, see [41, 42, 43].)

In classical electromagnetism, the electric and magnetic fields are physically
meaningful quantities, but it is often very convenient to work instead in
terms of scalar and vector potentials, which are not uniquely defined. All
choices for the potentials that yield the same electric and magnetic fields
are said to be related by _gauge transformations_. Any one such choice for the
potentials is called a _gauge choice_ , and the scalar and vector potentials
themselves are called _gauge potentials_ or _gauge variables_.

Making a suitable gauge choice can greatly simplify many calculations, such as
using Lorenz gauge to compute the electric and magnetic fields for delayed
boundary conditions. Ultimately, however, no gauge choice is fundamentally
more physically correct than any other gauge choice, and all calculations of
physical predictions in classical electromagnetism must conclude with
expressions that are written in terms of gauge-invariant quantities.

To set up the claimed analogy with electromagnetic gauge transformations, one
starts by observing from the basic relationship
$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=|\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)|^{2}$
in (17) that the Schur-Hadamard product (19) of the time-evolution operator
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ and a matrix of time-dependent
phases $\exp(i\theta_{ij}(t))$ is a transformation of
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ with no physical effects, and
therefore corresponds to a genuine form of gauge invariance:

$\Theta\mathopen{}\mathclose{{}\left(t}\right)\mapsto\Theta\mathopen{}\mathclose{{}\left(t}\right)\odot\begin{pmatrix}e^{i\theta_{11}\mathopen{}\mathclose{{}\left(t}\right)}&e^{i\theta_{12}\mathopen{}\mathclose{{}\left(t}\right)}\\\
e^{i\theta_{21}\mathopen{}\mathclose{{}\left(t}\right)}&\ddots\\\
&&e^{i\theta_{NN}\mathopen{}\mathclose{{}\left(t}\right)}\end{pmatrix}.$ (45)

This gauge transformation can be written equivalently at the level of
individual matrix entries as

$\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)\mapsto\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)e^{i\theta_{ij}\mathopen{}\mathclose{{}\left(t}\right)}.$
(46)

To the author’s knowledge, this kind of gauge invariance, which will be called
a _Schur-Hadamard gauge transformation_ , has not yet been described in the
research literature, and is therefore a new result. It will turn out to play a
key role in the analysis of dynamical symmetries that will be presented in
Subsection VI.1, and will be extended in an interesting way in the context of
Hilbert-space dilations in Subsection VI.2.

The Hilbert-space formulation has another form of gauge invariance, which
appears to have first been written down in [44] in the context of
transformations of the Schrödinger equation between inertial and non-inertial
reference frames. Letting $V\mathopen{}\mathclose{{}\left(t}\right)$ be a
time-dependent unitary matrix, the following transformation is also a gauge
invariance of the Hilbert-space formulation, leaving all probabilities
$p_{i}\mathopen{}\mathclose{{}\left(t}\right)$, expectation values
$\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle$, and the transition
matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ as a whole
unchanged:666One should be mindful of the appearance of the initial time $0$
in $V^{\dagger}\mathopen{}\mathclose{{}\left(0}\right)$ in the transformation
rule for $\Theta\mathopen{}\mathclose{{}\left(t}\right)$.

$\mathopen{}\mathclose{{}\left.\begin{aligned}
\rho\mathopen{}\mathclose{{}\left(t}\right)&\mapsto\rho_{V}\mathopen{}\mathclose{{}\left(t}\right)\equiv
V\mathopen{}\mathclose{{}\left(t}\right)\rho\mathopen{}\mathclose{{}\left(t}\right)V^{\dagger}\mathopen{}\mathclose{{}\left(t}\right),\\\
\Psi\mathopen{}\mathclose{{}\left(t}\right)&\mapsto\Psi_{V}\mathopen{}\mathclose{{}\left(t}\right)\equiv
V\mathopen{}\mathclose{{}\left(t}\right)\Psi\mathopen{}\mathclose{{}\left(t}\right),\\\
A\mathopen{}\mathclose{{}\left(t}\right)&\mapsto
A_{V}\mathopen{}\mathclose{{}\left(t}\right)\equiv
V\mathopen{}\mathclose{{}\left(t}\right)A\mathopen{}\mathclose{{}\left(t}\right)V^{\dagger}\mathopen{}\mathclose{{}\left(t}\right),\\\
\Theta\mathopen{}\mathclose{{}\left(t}\right)&\mapsto\Theta_{V}\mathopen{}\mathclose{{}\left(t}\right)\equiv
V\mathopen{}\mathclose{{}\left(t}\right)\Theta\mathopen{}\mathclose{{}\left(t}\right)V^{\dagger}\mathopen{}\mathclose{{}\left(0}\right).\end{aligned}\qquad}\right\\}$
(47)

If the unitary matrix $V\mathopen{}\mathclose{{}\left(t}\right)$ is taken to
be time-_independent_ , then the gauge transformation (47) is merely a change
of orthonormal basis. However, if $V\mathopen{}\mathclose{{}\left(t}\right)$
depends nontrivially on time, and if one regards the system’s Hilbert space at
each moment in time as a _fiber_ over a one-dimensional _base manifold_
parameterized by the time coordinate $t$, then
$V\mathopen{}\mathclose{{}\left(t}\right)$ represents a local-in-time, unitary
transformation of each individual Hilbert-space fiber. In particular, any
given time-dependent state vector
$\Psi\mathopen{}\mathclose{{}\left(t}\right)$, regarded as a trajectory
through the Hilbert space $\mathcal{H}$, can be mapped to any other trajectory
by a suitable choice of time-dependent unitary matrix
$V\mathopen{}\mathclose{{}\left(t}\right)$, so trajectories in $\mathcal{H}$
do not describe gauge-invariant facts.

##  IV The Stochastic-Quantum Correspondence

###  IV.1 Kraus Decompositions

In the most general case, a time-evolution operator
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ may not satisfy any nontrivial
constraints apart from the summation condition (18). It will turn out to be
helpful to find alternative ways of representing the $N\times N$ matrix
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ in terms of more tightly
constrained mathematical objects.

For $\beta=1,\dots,N$, let $K_{\beta}\mathopen{}\mathclose{{}\left(t}\right)$
be the $N\times N$ matrix defined to share its $\beta$th column with
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$, but with $0$s in all its
other entries:

$K_{\beta}\mathopen{}\mathclose{{}\left(t}\right)\equiv\begin{pmatrix}0&\cdots&0&\Theta_{1\beta}\mathopen{}\mathclose{{}\left(t}\right)&0&\cdots&0\\\
\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots\\\
0&\cdots&0&\Theta_{N\beta}\mathopen{}\mathclose{{}\left(t}\right)&0&\cdots&0\end{pmatrix}\quad\mathopen{}\mathclose{{}\left[\beta=1,\dots,N}\right].$
(48)

The entries of $K_{\beta}\mathopen{}\mathclose{{}\left(t}\right)$ are given
explicitly by

$K_{\beta,ij}\mathopen{}\mathclose{{}\left(t}\right)=\delta_{\beta
j}\Theta_{ij}\mathopen{}\mathclose{{}\left(t}\right).$ (49)

Then the summation condition (18) on
$\Theta\mathopen{}\mathclose{{}\left(t}\right)$ becomes the statement that the
matrices
$K_{1}\mathopen{}\mathclose{{}\left(t}\right),\dots,K_{N}\mathopen{}\mathclose{{}\left(t}\right)$
satisfy the _Kraus identity_ :

$\sum_{\beta=1}^{N}K_{\beta}^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)K_{\beta}\mathopen{}\mathclose{{}\left(t}\right)=\mymathbb{1}.$
(50)

These matrices are therefore called _Kraus operators_ [45]. One can then write
the dictionary (26) in an alternative form called a _Kraus decomposition_
:777Conditional probabilities similar in form to (51) were studied in [46].

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{\beta=1}^{N}\mathrm{tr}(K_{\beta}^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)P_{i}K_{\beta}\mathopen{}\mathclose{{}\left(t}\right)P_{j}).$
(51)

Like all the other mathematical objects in the Hilbert-space formulation, the
Kraus operators
$K_{1}\mathopen{}\mathclose{{}\left(t}\right),\dots,K_{N}\mathopen{}\mathclose{{}\left(t}\right)$
are not unique. Notice also that any number of $N\times N$ matrices satisfying
the Kraus identity (50) are guaranteed to yield a valid transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ via the Kraus decomposition
(51). Said in another way, the preceding argument establishes the existence
but not the uniqueness of Kraus operators for any given time-evolution
operator $\Theta\mathopen{}\mathclose{{}\left(t}\right)$.

Kraus operators and Kraus decompositions play an important role in quantum
information theory. In particular, they provide (non-unique) expressions for
generalizations of unitary time evolution known as _quantum channels_ , or
_completely positive trace-preserving (CPTP) maps_.

###  IV.2 Unistochastic Matrices

In general, Kraus operators need not have the specific form (48), and there
need not be $N$ of them. In the most minimal case in which a system’s
transition matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ is
determined by just a single Kraus operator
$K_{1}\mathopen{}\mathclose{{}\left(t}\right)$, that Kraus operator will be
denoted instead by $U\mathopen{}\mathclose{{}\left(t}\right)$. In that case,
the general Schur-Hadamard factorization (20) specializes to

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\overline{U\mathopen{}\mathclose{{}\left(t}\right)}\odot
U\mathopen{}\mathclose{{}\left(t}\right).$ (52)

That is,

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=|U_{ij}\mathopen{}\mathclose{{}\left(t}\right)|^{2}.$
(53)

Equivalently, in dictionary form (26), one has

$\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(U^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)P_{i}U\mathopen{}\mathclose{{}\left(t}\right)P_{j}).$
(54)

The Kraus identity (50), meanwhile, reduces to the statement that
$U\mathopen{}\mathclose{{}\left(t}\right)$ is unitary,

$U^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)=U^{-1}\mathopen{}\mathclose{{}\left(t}\right),$
(55)

and the system’s transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ is then said to be a
_unistochastic matrix_. That is, a unistochastic matrix is a square matrix
whose individual entries are the modulus-squares of the corresponding entries
of a unitary matrix.

Note that a unitary time-evolution operator
$U\mathopen{}\mathclose{{}\left(t}\right)$ will not generically remain unitary
under arbitrary Schur-Hadamard gauge transformations (45). Hence, writing a
unistochastic transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ in terms of a unitary time-
evolution operator $U\mathopen{}\mathclose{{}\left(t}\right)$ corresponds to
making a gauge choice—or, somewhat more precisely, to partially fixing the
gauge freedom (45).

Unistochastic matrices were first introduced in 1954 by Horn [47], who
originally called them ‘ortho-stochastic’ matrices. The modern term
‘unistochastic matrix’ was introduced by Thompson in 1989 [48, 49]. The term
_orthostochastic matrix_ now refers to a square matrix whose entries are the
modulus-squares of the corresponding entries of a _real orthogonal_ matrix.

Every orthostochastic matrix is unistochastic. Importantly, however, the
reverse statement is not generally true, meaning that the complex numbers
generically play a necessary role in formulating a unistochastic transition
matrix $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ in terms of a unitary
time-evolution operator $U\mathopen{}\mathclose{{}\left(t}\right)$. Even when
the complex numbers are not strictly necessary for writing down a unitary
time-evolution operator $U\mathopen{}\mathclose{{}\left(t}\right)$, such as if
the time-evolution operator can be taken to be real and orthogonal, it is
still very convenient to employ the complex numbers for a given Hilbert-space
representation, so that one can take advantage of the many highly useful
constructs that show up in standard treatments of quantum theory.

It follows immediately from the dictionary formula (54) that every
unistochastic transition matrix is _doubly stochastic_ , or _bistochastic_ ,
which means that summing over any of its rows _or_ any of its columns always
yields $1$:

$\sum_{i=1}^{N}\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{j=1}^{N}\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=1.$
(56)

###  IV.3 Unistochastic Systems

A generalized stochastic system whose transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ is a unistochastic matrix will
be called a _unistochastic system_.

To provide a simple example, note that every permutation matrix is, in
particular, a unitary matrix. Moreover, because the entries of a permutation
matrix are all $1$s and $0$s, they are individually invariant when one
computes their modulus-squares, so every permutation matrix is _also_ a
unistochastic matrix. It follows that a discrete, deterministic system whose
dynamics is defined by a permutation matrix is a special case of a
unistochastic system.

Remarkably, as will be shown in Subsection VI.2 _, every_ stochastic map can
be expressed in terms of a unitary time-evolution operator on a suitably
enlarged or _dilated_ Hilbert space. As a consequence, _every_ generalized
stochastic system can be regarded as a subsystem of a unistochastic system.
This statement can be formalized as a theorem, called the _stochastic-quantum
theorem_ [50], and implies that the study of generalized stochastic systems
can essentially be reduced to the study of unistochastic systems. Hence,
assuming a unistochastic system is not as special a condition as it might
initially seem.

Assuming a unistochastic system based on a unitary time-evolution operator
$U\mathopen{}\mathclose{{}\left(t}\right)$ that is a differentiable function
of the time $t$, one can define a corresponding self-adjoint generator
$H\mathopen{}\mathclose{{}\left(t}\right)$, called the system’s _Hamiltonian_
, according to

$H\mathopen{}\mathclose{{}\left(t}\right)\equiv i\hbar\frac{\partial
U\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}U^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)=H^{\dagger}\mathopen{}\mathclose{{}\left(t}\right).$
(57)

Here the factor of $i$ ensures that the $N\times N$ matrix
$H\mathopen{}\mathclose{{}\left(t}\right)$ is self-adjoint, and, for present
purposes, the _reduced Planck constant_ $\hbar$ is a fixed quantity introduced
for purposes of measurement units. Ultimately, the specific numerical value of
$\hbar$ in any given set of units must be determined empirically by comparison
with experiments.

In terms of the Hamiltonian, the system’s density matrix
$\rho\mathopen{}\mathclose{{}\left(t}\right)$ then evolves in time according
to the _von Neumann equation_ ,

$i\hbar\frac{\partial\rho\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}=[H\mathopen{}\mathclose{{}\left(t}\right),\rho\mathopen{}\mathclose{{}\left(t}\right)],$
(58)

its state vector $\Psi\mathopen{}\mathclose{{}\left(t}\right)$ (if it exists)
evolves according to the _Schrödinger equation_ ,

$i\hbar\frac{\partial\Psi\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}=H\mathopen{}\mathclose{{}\left(t}\right)\Psi\mathopen{}\mathclose{{}\left(t}\right),$
(59)

its Heisenberg-picture random variables
$A^{H}\mathopen{}\mathclose{{}\left(t}\right)$ evolve according to the
_Heisenberg equation of motion_ ,

$\frac{dA^{H}\mathopen{}\mathclose{{}\left(t}\right)}{dt}=\frac{i}{\hbar}[H^{H}\mathopen{}\mathclose{{}\left(t}\right),A^{H}\mathopen{}\mathclose{{}\left(t}\right)]+\mathopen{}\mathclose{{}\left(\frac{\partial
A\mathopen{}\mathclose{{}\left(t}\right)}{\partial t}}\right)^{H},$ (60)

and its expectation values $\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle$ evolve according to
the _Ehrenfest equation_ ,

$\frac{d\langle
A\mathopen{}\mathclose{{}\left(t}\right)\rangle}{dt}=\frac{i}{\hbar}\mathrm{tr}([H\mathopen{}\mathclose{{}\left(t}\right),A\mathopen{}\mathclose{{}\left(t}\right)]\rho\mathopen{}\mathclose{{}\left(t}\right))+\mathopen{}\mathclose{{}\left\langle\frac{\partial
A\mathopen{}\mathclose{{}\left(t}\right)}{\partial t}}\right\rangle.$ (61)

The matrix $H^{H}\mathopen{}\mathclose{{}\left(t}\right)$ appearing in the
Heisenberg equation of motion (60) is the Hamiltonian expressed in the
Heisenberg picture (42). Note also that the brackets $[X,Y]$ that naturally
show up in these equations are genuine _commutators_ $XY-YX$, not Poisson
brackets, and involve products of non-diagonal matrices that do not generally
commute with each other under multiplication.

The emergence of these famous equations from a physical model based on a
stochastically evolving trajectory in a configuration space $\mathcal{C}$ is a
surprising new result.

Intriguingly, if the system’s time-evolution operator
$\Theta\mathopen{}\mathclose{{}\left(t}\right)=U\mathopen{}\mathclose{{}\left(t}\right)$
is indeed unitary, then under the unitary gauge transformation defined by
(47), the Hamiltonian transforms precisely as a _non-Abelian gauge potential_
:

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&H\mathopen{}\mathclose{{}\left(t}\right)\mapsto
H_{V}\mathopen{}\mathclose{{}\left(t}\right)\\\
&\qquad=V\mathopen{}\mathclose{{}\left(t}\right)H\mathopen{}\mathclose{{}\left(t}\right)V^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)-i\hbar
V\mathopen{}\mathclose{{}\left(t}\right)\frac{\partial
V^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}.\end{aligned}\negmedspace\negmedspace\negmedspace\negmedspace\negmedspace\negmedspace\qquad}\right\\}$
(62)

This transformation behavior makes clear that a Hamiltonian is not a gauge-
invariant observable, even though it may happen to coincide with various
observables according to particular gauge choices. (For pedagogical treatments
of non-Abelian gauge theories, see [51, 52].)

Furthermore, one can write the Schrödinger equation (59) as

$\mathcal{D}\mathopen{}\mathclose{{}\left(t}\right)\Psi\mathopen{}\mathclose{{}\left(t}\right)=0.$
(63)

Here $\mathcal{D}\mathopen{}\mathclose{{}\left(t}\right)$ is a _gauge-
covariant derivative_ defined according to

$\mathcal{D}\mathopen{}\mathclose{{}\left(t}\right)\equiv\mymathbb{1}\frac{\partial}{\partial
t}+\frac{i}{\hbar}H\mathopen{}\mathclose{{}\left(t}\right),$ (64)

which maintains its form under the unitary gauge transformations (47), in the
sense that

$V\mathopen{}\mathclose{{}\left(t}\right)\mathopen{}\mathclose{{}\left[\mymathbb{1}\frac{\partial}{\partial
t}+\frac{i}{\hbar}H\mathopen{}\mathclose{{}\left(t}\right)}\right]\mathopen{}\mathclose{{}\left(\cdots}\right)=\mathopen{}\mathclose{{}\left[\mymathbb{1}\frac{\partial}{\partial
t}+\frac{i}{\hbar}H_{V}\mathopen{}\mathclose{{}\left(t}\right)}\right]\mathopen{}\mathclose{{}\left[V\mathopen{}\mathclose{{}\left(t}\right)\mathopen{}\mathclose{{}\left(\cdots}\right)}\right].$
(65)

These formulas make manifest that the Hilbert-space formulation of a
generalized stochastic system is ultimately a collection of gauge-dependent
quantities, or gauge variables. Hence, although a Hilbert-space formulation
may be extremely useful for constructing stochastic dynamics or for carrying
out calculations, one might rightly be suspicious about trying to assign
direct physical meanings to its mathematical ingredients.

Notice that if one picks the gauge-transformation matrix
$V\mathopen{}\mathclose{{}\left(t}\right)$ to be the adjoint of the
unistochastic system’s time-evolution operator
$U\mathopen{}\mathclose{{}\left(t}\right)$,

$V\mathopen{}\mathclose{{}\left(t}\right)\equiv
U^{\dagger}\mathopen{}\mathclose{{}\left(t}\right),$ (66)

then the Hamiltonian precisely vanishes:

$H_{V}\mathopen{}\mathclose{{}\left(t}\right)=0.$ (67)

This choice of gauge is nothing other than the definition (42) of the
Heisenberg picture. Unitary gauge transformations (47) can therefore be viewed
as generalized changes of time-evolution picture.888The fact that one can set
$H_{V}\mathopen{}\mathclose{{}\left(t}\right)=0$ for all $t$ is a
manifestation of the fact that the _fiber bundle_ in this case, consisting of
copies of the system’s Hilbert space fibered over a one-dimensional base
manifold parameterized by the time $t$, has vanishing curvature.

###  IV.4 Interference

The appearance of the Schrödinger equation (59) is an important signal that
the dictionary (26) is more than just a tool for using Hilbert-space methods
to craft highly general forms of stochastic dynamics. It also suggests that
generalized stochastic systems might have the resources to replicate the
features of quantum theory more broadly.

As another hint pointing in this direction, one starts by noting that an
arbitrary time-dependent transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ is generically _indivisible_ ,
in the sense that it does not satisfy the divisibility property (14) at
arbitrary times. To see what goes wrong with divisibility, suppose that at
some time $t^{\prime}$, the transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ has a matrix inverse
$\Gamma^{-1}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$, and define a
new $N\times N$ matrix
$\tilde{\Gamma}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)$
according to

$\tilde{\Gamma}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\equiv\Gamma\mathopen{}\mathclose{{}\left(t}\right)\Gamma^{-1}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(68)

As an immediate consequence, one then has

$\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\tilde{\Gamma}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right),$
(69)

which resembles the divisibility property (14). However, it follows from an
elementary theorem of linear algebra that the inverse of a stochastic matrix
can only be stochastic if both matrices are permutation matrices, and
therefore do not involve nontrivial probabilities.999Proof: Let $X$ and $Y$ be
$N\times N$ matrices with only non-negative entries and with $Y=X^{-1}$, so
that $XY=\mymathbb{1}$. Then, in particular, the first row of $X$ must be
orthogonal to the second-through-$N$th columns of $Y$. Because $Y$ is
invertible, the columns of $Y$ must all be linearly independent, so the first
row of $X$ must be orthogonal to the
$\mathopen{}\mathclose{{}\left(N-1}\right)$-dimensional subspace spanned by
the second-through-$N$th columns of $Y$. Because the entries of $X$ and $Y$
are all non-negative by assumption, the only way that this orthogonality
condition can hold is if precisely one of the entries in the first row of $X$
is nonzero, with a $0$ in the corresponding entry in each of the second-
through-$N$th columns of $Y$. Repeating this argument for the other rows of
$X$, one sees that $X$ can only have a single nonzero entry in each row. If
$X$ is a stochastic matrix, then each of these nonzero entries must be the
number $1$, so $X$ must be a permutation matrix. Because the inverse of a
permutation matrix is again a permutation matrix, it follows that $Y$ must
likewise be a permutation matrix. QED Hence, the matrix
$\tilde{\Gamma}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)$
defined in (68) is not generically stochastic, so (69) does not express a
genuine form of divisibility.

There is an alternative—and far-reaching—way to understand the generic
indivisibility of a time-dependent transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$. To this end, suppose that
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ happens to be unistochastic,
and let $U\mathopen{}\mathclose{{}\left(t}\right)$ be a unitary time-evolution
operator for $\Gamma\mathopen{}\mathclose{{}\left(t}\right)$. Then for any two
times $t$ and $t^{\prime}$, one can define a _relative_ time-evolution
operator

$U\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)\equiv
U\mathopen{}\mathclose{{}\left(t}\right)U^{\dagger}\mathopen{}\mathclose{{}\left(t^{\prime}}\right),$
(70)

which yields the composition law

$U\mathopen{}\mathclose{{}\left(t}\right)=U\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$ (71)

At the level of the unistochastic transition matrix
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$, one has from the Schur-
Hadamard factorization (52) that

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\overline{U\mathopen{}\mathclose{{}\left(t}\right)}\odot
U\mathopen{}\mathclose{{}\left(t}\right)\\\
&=\overline{\mathopen{}\mathclose{{}\left[U\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}\right]}\odot\mathopen{}\mathclose{{}\left[U\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}\right],\end{aligned}\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(72)

which cannot generally be expressed in the form
$\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ for
any stochastic matrix $\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$, due to the presence of cross terms.

Indeed, examining individual matrix entries, one finds more explicitly that

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{k=1}^{N}|U_{ik}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2}|U_{kj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}\\\
&+\sum_{k\neq l}\overline{U_{ik}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U_{kj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}U_{il}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U_{lj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(73)

With $\Gamma_{kj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ defined
according to (53) as usual,

$\Gamma_{kj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=|U_{kj}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2},$
(74)

and defining

$\Gamma_{ik}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\equiv|U_{ik}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2},$ (75)

which is manifestly unistochastic, one sees that the discrepancy between
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)$ and its would-be division
$\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ is
given by

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Gamma_{ij}\mathopen{}\mathclose{{}\left(t}\right)-\mathopen{}\mathclose{{}\left[\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}\right]_{ij}\\\
&=\sum_{k\neq l}\overline{U_{ik}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{k}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}U_{il}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{l}\mathopen{}\mathclose{{}\left(t^{\prime}}\right),\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(76)

where
$\Psi\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\equiv\Theta\mathopen{}\mathclose{{}\left(t^{\prime}}\right)e_{j}$
is the system’s state vector at the time $t^{\prime}$, in keeping with the
general definition of state vectors in (36).

Remarkably, the right-hand side of (76) gives the general mathematical formula
for quantum interference, despite the absence of manifestly quantum-theoretic
assumptions. One sees from this analysis that interference is a direct
consequence of stochastic dynamics not generally being divisible. More
precisely, interference is nothing more than a generic discrepancy between
_actual_ indivisible stochastic dynamics and _hypothetically_ divisible
stochastic dynamics.

In particular, quantum-mechanical interference does not imply that matter has
a physically wavelike nature, contrary to frequent claims in textbook
treatments like [53]. Indeed, from the perspective of the present discussion,
the notion that quantum-mechanical interference ever necessitated assigning
matter a physically wavelike quality was merely an unfortunate accident of
history, arising from the fact that many early empirical examples of
interference in quantum systems happened to resemble the behavior of
interfering waves propagating in three-dimensional physical space.

These historical examples were clearly special cases. _Multiparticle_ systems
have Schrödinger waves that propagate through high-dimensional configuration
spaces, as was noted by Schrödinger himself in his early work on wave
mechanics [54]. For more abstract systems, like _qubits_ , there fail to exist
continuous configuration spaces for Schrödinger waves altogether.

This new way of thinking about quantum-mechanical interference has
implications for the interpretation of the famous _double-slit experiment_.
Recall that in the double-slit experiment, an emitter sends particles one at a
time toward a wall with two slits in it, and a detection screen on the other
side of the wall records the particle’s eventual landing site. In the usual
‘classical’ description of the experiment, one asks first which slit the
particle enters, and then, _conditioning_ _on the answer_ , one then
_restarts_ the dynamics with that slit as the new initial condition. Over many
repetitions of the experiment, the detection screen records a blend consisting
of the statistical distribution of landing sites from particles passing
through the _upper_ slit with the statistical distribution of landing sites
from particles passing through the _lower_ slit. In the case of quantum-
mechanical particles like electrons, however, one instead finds that the
landing sites form a wavelike interference pattern, and the conclusion is
supposedly that each particle is really a Schrödinger wave of some kind, or
that the particle fails to go through one slit or the other.101010The
exposition in [53] ends up at precisely such a conclusion: “It is _not_ true
that the electrons go _either_ through hole 1 or hole 2.” [Emphasis in the
original.] This conclusion, however, does not logically follow from the
empirical appearance of interference effects, but also implicitly depends on
the hidden assumption that the behavior of an electron in a double-slit
experiment can be described by divisible dynamics.

According to the approach laid out in this paper, the particle really does go
through a specific slit in each run of the experiment. The final interference
pattern on the detection screen is due to the generic indivisibility of time
evolution for quantum systems. One cannot divide up the particle’s evolution
into, firstly, its transit from the emitter to the slits, and then secondly,
conditioned on which slit the particle enters, the particle’s transit from the
slits to the detection screen. The interference that shows up in the double-
slit experiment may be surprising, but that is only because indivisible
stochastic dynamics can be highly nonintuitive. In the historical absence of a
sufficiently comprehensive framework for describing indivisible stochastic
dynamics, it was difficult to recognize just how nonintuitive such dynamics
could be, or what sorts of empirically appearances it could produce.

In response to this last point, one might suggest that Schrödinger waves
nonetheless offer a superior means of explaining why the double-slit
experiment yields the results that it does. Unfortunately, such hopes are
dashed as soon as one considers sending _two_ particles toward the slits on
each run of the experiment. A two-particle system’s Schrödinger wave evolves
in a _six-dimensional_ configuration space, which is arguably not more
intuitive than indivisible stochastic dynamics.

Of course, if one regards the quantum-mechanical particles that make up matter
as arising more fundamentally from underlying _quantum fields_ , then the
wavelike properties of those quantum fields ensure that particles of matter
have wavelike properties as well, and therefore exhibit _wave-particle
duality_. That said, there is nothing about the analysis of the double-slit
experiment alone that calls for positing quantum fields. The necessity of
quantum field theory comes from other theoretical and empirical
considerations. (For a modern motivation, see [55].) One should also keep in
mind that quantum fields are conceptually distinct from Schrödinger wave
functions.

###  IV.5 Implications of Interference

The fact that interference shows up in a sufficiently generic stochastic model
means that relative phase factors in state vectors have clear empirical
signatures, even in the absence of the usual axioms of textbook quantum
theory. These empirical manifestations of relative phases are strong evidence
that it should be possible to carry out measurements on a much wider set of
observables than those that are represented by diagonal matrices (34) in a
generalized stochastic system’s configuration basis. Indeed, Subsection V.2
will show that non-diagonal, self-adjoint matrices will turn out to be
candidate observables as well.

Thinking more broadly, this overall analysis means that if one is given an
indivisible generalized stochastic system, then there will generically be a
quantitative discrepancy between the system’s actual behavior—as predicted
theoretically or measured empirically—and predictions made for the system
based on the nearest divisible or Markovian approximation to the system’s
stochastic dynamics. Again, this discrepancy is precisely interference.

One way to understand this discrepancy is to note that under a divisibility
approximation, one can assign definite probabilities to each of the system’s
possible trajectories by iteratively applying transition matrices, according
to the composition law
$\Gamma\mathopen{}\mathclose{{}\left(t}\right)=\Gamma\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ from
(14). Because iteratively applying transition matrices is not possible for a
generalized stochastic system with indivisible dynamics, such systems do not
generically have well-defined probabilities for entire possible trajectories.

In the Hilbert-space formulation of a generalized stochastic system, one can
assign complex-valued quantities called _amplitudes_ to the system’s possible
trajectories, using the fact that unitary time-evolution operators can be
composed or applied iteratively, as in
$U\mathopen{}\mathclose{{}\left(t}\right)=U\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ from
(71). These amplitudes form the conceptual basis for the _path-integral
formulation_ of quantum theory [18, 19, 20]. From the standpoint of the
stochastic-quantum correspondence, which gives an alternative formulation of
quantum theory, the fact that these amplitudes ‘interfere’ with each other
does not mean that they all physically occur in some sort of literal
superposition, or that the system simultaneously takes all such paths in
reality, but is merely an artifact of the indivisible dynamics of the
underlying generalized stochastic system.

Collectively, the foregoing observations lead to the concrete prediction that
interference should arise in a much broader class of contexts than just for
quantum systems. Indeed, given any probabilistically evolving system with
indivisible or non-Markovian dynamics, one can now interpret any discrepancies
between the behavior of such a system and the behavior of its nearest
divisible or Markovian approximation as manifestations of interference. One
could therefore imagine experimentally measuring interference effects for
essentially any system that can be modeled using indivisible or non-Markovian
stochastic dynamics.

###  IV.6 Division Events and the Markov Approximation

Why do discrete-time Markov chains (13) provide such a good approximation to
so many stochastic processes in the real world? One intuitively reasonable
explanation is that when a system is not isolated from a noisy and intrusive
environment, delicate correlations from one time to another ‘wash out’ over
short time scales as those correlations leak out into the environment.

Deriving this intuitive picture from first principles in a more precise way
might appear to be a difficult task. Indeed, such a derivation would seem to
require finding a more general framework for describing a non-Markovian
process, and then showing that such a process becomes approximately Markovian
in the appropriate physical circumstances. Fortunately, this paper provides
just such a framework.

To set things up, one starts by introducing a composite system
$\mathcal{S}\mathcal{E}$ consisting of a _subject system_ $\mathcal{S}$
together with an _environment_ $\mathcal{E}$. The configurations of the
subject system’s configuration space $\mathcal{C}_{\mathcal{S}}$ will be
labeled by $i=1,\dots,N$, and the configurations of the environment’s
configuration space $\mathcal{C}_{\mathcal{E}}$ will be labeled by
$e=1,\dots,M$, where $M\geq N$. The configuration space of the composite
system is then the Cartesian product111111The right-hand side of (77) is
indeed a Cartesian product, not a tensor product, because this equation is
solely a statement about the composite system’s configuration space, not its
dynamics or its Hilbert-space representation.

$\mathcal{C}_{\mathcal{S}\mathcal{E}}=\mathcal{C}_{\mathcal{S}}\times\mathcal{C}_{\mathcal{E}},$
(77)

meaning that each element of $\mathcal{C}_{\mathcal{S}\mathcal{E}}$ is a
simple ordered pair of the form $\mathopen{}\mathclose{{}\left(i,e}\right)$.
One then singles out $N$ configurations of the environment by labeling them as
$e\mathopen{}\mathclose{{}\left(1}\right),\dots,e\mathopen{}\mathclose{{}\left(N}\right)$.

For the dynamics, suppose for simplicity that the composite system evolves
according to an overall unistochastic transition matrix

$\Gamma^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=\overline{U^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)}\odot
U^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right),$ (78)

or, in terms of individual entries,

$\Gamma_{ie,i_{0}e_{0}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=|U_{ie,i_{0}e_{0}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)|^{2}.$
(79)

Furthermore, suppose that the subject system and the environment interact up
to a time $t^{\prime}>0$ in such a way that they end up with joint
probabilities of the form

$p_{i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\delta_{e^{\prime}e(i^{\prime})},$
(80)

which describe an idealized _statistical correlation_ between the
configuration $i^{\prime}$ of the subject system at $t^{\prime}$ and the
corresponding configuration $e(i^{\prime})$ of the environment.

If there is to be any possibility of the two subsystems evolving independently
for times $t>t^{\prime}$ after the interaction has concluded, then it should
be possible to factorize the composite system’s relative time-evolution
operator $U^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ between the two subsystems for $t>t^{\prime}$ as the
following tensor product:121212Note the natural appearance of a tensor product
in (81), which is a statement about the composite system’s dynamics in the
system’s Hilbert-space representation.

$\mathopen{}\mathclose{{}\left.\begin{aligned}
U^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)&=U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\otimes
U^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)\\\
&\qquad\qquad\textrm{for
}t>t^{\prime}.\end{aligned}\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(81)

In terms of individual entries, one has

$\mathopen{}\mathclose{{}\left.\begin{aligned}
U_{ie,i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)&=U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U_{ee^{\prime}}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\ &\qquad\qquad\textrm{for
}t>t^{\prime},\end{aligned}\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(82)

meaning that each entry
$U_{ie,i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ of the composite system’s relative time-evolution operator
is the product of corresponding entries
$U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ and
$U_{ee^{\prime}}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ of the relative time-evolution operators for the two
subsystems individually.

In light of the Born rule (40), the joint probabilities (80) correspond to a
wave function131313If necessary, one can easily write down idealized examples
of unitary time-evolution operators for the composite system that produce the
wave function (83). For instance, one could use
$U^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\equiv\sum_{i^{\prime}}P_{i^{\prime}}^{\mathcal{S}}\otimes
R_{e(i^{\prime})}^{\mathcal{E}}$, where $P_{i^{\prime}}^{\mathcal{S}}$ is the
$i^{\prime}$th configuration projector (21) for the subject system, and where
$R_{e(i^{\prime})}^{\mathcal{E}}$ is a unitary transformation that takes the
environment’s initial configuration to the configuration $e(i^{\prime})$.

$\Psi_{i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=\Psi_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\delta_{e^{\prime}e(i^{\prime})}.$
(83)

The composite system’s wave function at later times $t>t^{\prime}$ after the
interaction is therefore given in terms of the relative time-evolution
operator (82) according to

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Psi_{ie}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{i^{\prime},e^{\prime}}U_{ie,i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{i^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\\\
&=\sum_{i^{\prime}}U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)U_{ee(i^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right).\end{aligned}\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(84)

From the Born rule (40), one sees that the joint probabilities for
$t>t^{\prime}$ are given by

$p_{ie}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=\mathopen{}\mathclose{{}\left|\Psi_{ie}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)}\right|^{2}.$
(85)

Marginalizing over the configuration $e$ of the environment and invoking the
unitarity of the environment’s relative time-evolution operator
$U^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)$,
one obtains the standalone probabilities
$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for the subject
system alone for $t>t^{\prime}$:

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{e}p_{ie}^{\mathcal{S}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)\\\
&=\sum_{i_{1}^{\prime},i_{2}^{\prime}}\overline{U_{ii_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{i_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)}U_{ii_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{i_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\\\
&\qquad\times\sum_{e}\overline{U_{ee(i_{1}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ee(i_{2}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&=\sum_{i^{\prime}}|U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2}|\Psi_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}.\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(86)

Taking the limit $t\to t^{\prime}$ in (86) and referring back to the Born rule
(40) again, one sees that the subject system’s standalone probabilities at the
time $t^{\prime}$ are

$p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=|\Psi_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}.$
(87)

One also sees from (86) that, as in (75), one can identify

$\Gamma_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\equiv|U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2}.$ (88)

Hence, (86) simplifies to a genuinely linear relationship that precisely
mirrors the Bayesian marginalization formula (7), with $t^{\prime}$ now
effectively serving as a new ‘initial time’:

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{i^{\prime}}\Gamma_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(89)

Applying the original Bayesian marginalization formula (7) from the actual
initial time $0$ to the time $t^{\prime}$, one also has the equation

$p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=\sum_{j}\Gamma_{i^{\prime}j}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)p_{j}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(0}\right).$
(90)

Combining (89) with (90) immediately yields

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{j}\Gamma_{ij}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)p_{j}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(0}\right),$
(91)

where $\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ is a
manifestly _divisible_ transition matrix:

$\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)\equiv\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(92)

Thus, the interaction between the subject system $\mathcal{S}$ and the
environment $\mathcal{E}$ up to the time $t^{\prime}$ has led to a transition
matrix $\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for the
subject system that is momentarily divisible at $t^{\prime}$.

It is natural to refer to $t^{\prime}$ as a _division event_. An important
corollary is that the initial time $0$ is not a unique or special time, but is
instead only one of many division events inevitably experienced by a system in
sufficiently strong contact with a repeatedly eavesdropping environment—in the
sense that the interactions with the environment lead to correlations that
look approximately like those in (80). Division events will play a crucial
role in the rest of this paper.

Suppose that these kinds of division events can be approximated as occurring
regularly over a characteristic time scale $\delta t$. Suppose, moreover, that
the unistochastic dynamics is homogeneous in time, in the sense that
$U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t+\delta t\leftarrow
t}\right)=U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(\delta t}\right)$ for
all times $t$. Then the subject system’s transition matrix after any integer
number $n\geq 1$ of time steps $\delta t$ is given by

$\Gamma^{\mathcal{S}}\mathopen{}\mathclose{{}\left(n\,\delta
t}\right)=\mathopen{}\mathclose{{}\left(\Gamma^{\mathcal{S}}}\right)^{n},$
(93)

where

$\Gamma_{ij}^{\mathcal{S}}\equiv|U_{ij}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(\delta
t}\right)|^{2}.$ (94)

The stochastic dynamics therefore takes the form of a discrete-time Markov
chain (13). This analysis therefore provides a theoretical explanation for the
ubiquity of Markovian stochastic dynamics in so many real-world cases, and
represents another new result.

###  IV.7 Decoherence

Had the environment not interacted with the subject system, then the subject
system’s density matrix
$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ at the
time $t^{\prime}$ would have generically been non-diagonal, in accordance with
the general definition (32):

$\mathopen{}\mathclose{{}\left.\begin{aligned}
\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)&=U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\mathopen{}\mathclose{{}\left[\sum_{j}p_{j}\mathopen{}\mathclose{{}\left(0}\right)P_{j}}\right]U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\\\
&=U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\mathrm{diag}\mathopen{}\mathclose{{}\left(\dots,p_{j}\mathopen{}\mathclose{{}\left(0}\right),\dots}\right)U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(95)

By contrast, suppose that the environment indeed interacts with the subject
system to produce a division event (92) at $t^{\prime}$. In that case, the
standalone probability
$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for the subject
system to occupy its $i$th configuration at $t>t^{\prime}$ is given by the
linear marginalization relationship (89), which can be written instead as

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(P_{i}\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)),$
(96)

where

$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)\equiv
U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right),$ (97)

and where, in turn,

$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\equiv\sum_{i^{\prime}}p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)P_{i^{\prime}}^{\mathcal{S}}=\mathrm{diag}\mathopen{}\mathclose{{}\left(\dots,p_{i^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right),\dots}\right),$
(98)

which is diagonal.

On comparing the two expressions (95) and (98) for the subject system’s
density matrix $\rho\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$ at
$t^{\prime}$, one sees that the interaction with the environment has
effectively eliminated the off-diagonal entries, or _coherences_ , in the
subject system’s density matrix. This phenomenon is called _decoherence_ , and
the foregoing analysis makes clear that decoherence is nothing more than the
mundane leakage of correlations into the environment when viewed through the
lens of the Hilbert-space formulation.

This analysis also sheds new light on the meaning of coherences in density
matrices, as well as on _superpositions_ in state vectors, where
superpositions are related to coherences in the case of a rank-one density
matrix through the formula
$\rho_{i_{1}i_{2}}\mathopen{}\mathclose{{}\left(t}\right)=\Psi_{i_{1}}\mathopen{}\mathclose{{}\left(t}\right)\overline{\Psi_{i_{2}}\mathopen{}\mathclose{{}\left(t}\right)}$,
in accordance with (39). From the standpoint of this analysis, superpositions
and coherences are merely indications that one is catching a given system when
it is in the midst of an indivisible stochastic process, between division
events, rather than implying that the system is literally in ‘multiple states
at once.’

These results may also help explain why the precise connection between quantum
theory and stochastic processes has historically remained unclear for so long.
If one assumes a Markov approximation, as is often the case in the research
literature on stochastic processes, then coherences and superposition do not
show up, meaning that density matrices remain diagonal, state vectors remain
trivial, and nontrivial unistochastic dynamics cannot arise.141414See [56] for
an analysis of the connection between decoherence and Markovian dynamics
within the standard Hilbert-space formulation of quantum theory.

###  IV.8 Entanglement

Consider next a composite system $\mathcal{A}\mathcal{B}$ consisting of a pair
of subsystems $\mathcal{A}$ and $\mathcal{B}$. Suppose that the two subsystems
do not interact from the initial time $0$ up to some later time
$t^{\prime}>0$, but then begin interacting at $t^{\prime}$.

For times $t$ between $0$ and $t^{\prime}$, the absence of interactions means
that the composite system’s transition matrix
$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)$
factorizes into the tensor product of a transition matrix
$\Gamma^{\mathcal{A}}\mathopen{}\mathclose{{}\left(t}\right)$ for
$\mathcal{A}$ and a separate transition matrix
$\Gamma^{\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)$ for
$\mathcal{B}$:

$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)=\Gamma^{\mathcal{A}}\mathopen{}\mathclose{{}\left(t}\right)\otimes\Gamma^{\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)\quad\textrm{for
}0\leq t<t^{\prime}.$ (99)

Starting at the time $t^{\prime}$, however, the composite system’s transition
matrix
$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)$,
which encodes cumulative statistical information and therefore correlations,
will fail to factorize between the two subsystems, in the sense that

$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)\neq\Gamma^{\mathcal{A}}\mathopen{}\mathclose{{}\left(t}\right)\otimes\Gamma^{\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)\quad\textrm{for
}t>t^{\prime},$ (100)

for any possible transition matrices
$\Gamma^{\mathcal{A}}\mathopen{}\mathclose{{}\left(t}\right)$ and
$\Gamma^{\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)$ that properly
capture the respective dynamics of the two subsystems. (It is worth noting
that this loss of factorization gives a highly general, model-independent way
to define an interaction.) Even if the two subsystems have a notion of
localizability in space, and are eventually placed at a large separation
distance at some time $t>t^{\prime}$, the composite system’s transition matrix
will still fail to factorize between the two subsystems, thereby leading to
the appearance of what looks like nonlocal stochastic dynamics across that
separation distance.151515Questions about nonlocality will be addressed in
detail in Subsection VI.3.

However, if the composite system exhibits a division event of the form (92) at
some later time $t^{\prime\prime}>t^{\prime}$, perhaps due to interactions
between one of the subsystems and the larger environment, then the composite
system’s transition matrix
$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)$ will
divide at $t^{\prime\prime}$:

$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t}\right)=\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime\prime}}\right)\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t^{\prime\prime}}\right)\quad\textrm{for
}t>t^{\prime\prime}>t^{\prime}.$ (101)

If the two subsystems $\mathcal{A}$ and $\mathcal{B}$ do not interact with
each other after $t^{\prime}$, then the _relative_ transition matrix
$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime\prime}}\right)$ appearing in (101) will factorize between them,

$\Gamma^{\mathcal{A}\mathcal{B}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime\prime}}\right)=\Gamma^{\mathcal{A}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime\prime}}\right)\otimes\Gamma^{\mathcal{B}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime\prime}}\right),$ (102)

so the two subsystems will cease exhibiting what had looked like nonlocal
stochastic dynamics.

This analysis precisely captures the quantum-theoretic notion of
_entanglement_. Systems that interact with each other start to exhibit what
appears to be a nonlocal kind of stochastic dynamics, even if the systems are
moved far apart in physical space, and decoherence by the environment
effectively causes a breakdown in that apparent dynamical nonlocality.

This stochastic picture of entanglement and nonlocality provides a new way to
understand why they occur in the first place. The indivisible nature of
generic stochastic dynamics could be viewed as a form of nonlocality in time,
which then leads to an apparent nonlocality across space. A division event
leads to an instantaneous restoration of locality in time, which then leads to
a momentary restoration of manifest locality across space.

##  V Measurements

###  V.1 Emergeables

The preceding sections have shown that a generalized stochastic system—that
is, a physical model with kinematics based on a configuration space and
dynamics based on a suitable stochastic law—is capable of accounting for
signature features of quantum theory, like superposition, interference,
decoherence, and entanglement. In addition, the Hilbert-space side of the
dictionary (26) contains many expressions and equations that are identical to
those found in quantum theory.

However, an actual quantum system also includes observables beyond those that
are diagonal, as in (34), in a single basis. Indeed, the existence of
noncommuting observables represented by non-diagonal, self-adjoint matrices is
another hallmark feature of quantum theory.

Remarkably, a generalized stochastic system will generically contain such
observables as well. Specifically, Subsection V.2 will establish that non-
diagonal, self-adjoint matrices represent candidate observables that naturally
satisfy the usual probabilistic rules of quantum theory, including the Born
rule, all without the need to introduce any new fundamental axioms. In so
doing, the analysis ahead will demonstrate that the dictionary (26) is not
merely a tool for studying a broad class of stochastic processes, but truly
defines a comprehensive stochastic-quantum correspondence.

As motivation, let $A$ be a time-independent (diagonal) random variable (34),
and consider the time derivative of its Heisenberg-picture counterpart
$A^{H}\mathopen{}\mathclose{{}\left(t}\right)$, as defined for a generic time-
evolution operator $\Theta\mathopen{}\mathclose{{}\left(t}\right)$ by (42):

$\frac{dA^{H}\mathopen{}\mathclose{{}\left(t}\right)}{dt}=\frac{\partial\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}A\Theta\mathopen{}\mathclose{{}\left(t}\right)+\Theta^{\dagger}\mathopen{}\mathclose{{}\left(t}\right)A\frac{\partial\Theta\mathopen{}\mathclose{{}\left(t}\right)}{\partial
t}.$ (103)

Evaluating this matrix in the limit $t\to 0$ gives an $N\times N$ self-
adjoint, generically non-diagonal matrix $\dot{A}$ at the initial time $0$:

$\dot{A}\equiv\lim_{t\to
0}\frac{dA^{H}\mathopen{}\mathclose{{}\left(t}\right)}{dt}=\dot{A}^{\dagger}.$
(104)

This matrix will not generally commute with the original random variable $A$
itself:

$[A,\dot{A}]\neq 0.$ (105)

Nonetheless, the matrix $\dot{A}$ has physical uses. For example, one has

$\mathrm{tr}(\dot{A}\rho\mathopen{}\mathclose{{}\left(0}\right))=\lim_{t\to
0}\frac{d\mathopen{}\mathclose{{}\left\langle
A\mathopen{}\mathclose{{}\left(t}\right)}\right\rangle}{dt},$ (106)

which is a perfectly meaningful physical quantity, even though the time
derivative of an expectation value is not necessarily the expectation value of
something physical.

The matrix $\dot{A}$ therefore resembles a random variable in some ways, but
incorporates stochastic dynamics directly into its definition (104) through
the time-evolution operator $\Theta\mathopen{}\mathclose{{}\left(t}\right)$,
and does not have a transparent interpretation at the level of the generalized
stochastic system’s underlying configuration space $\mathcal{C}$ alone.
Instead, $\dot{A}$ is an emergent amalgam of kinematical and dynamical
ingredients, so it will be called an _emergeable_.161616There is a sense in
which emergeables are not an entirely new idea, but are closely related to
emergent physical properties like temperatures or pressures. This terminology
is intended to contrast $\dot{A}$ with the system’s genuine random variables,
which could be called _beables_ —that is, ‘be-ables’—to invoke a term coined
by Bell in [57].

As a concrete example, consider a particle whose underlying _position_ is
regarded as a physical configuration, corresponding to some random variable
$A$. If the particle’s dynamics is stochastic, in the sense that the particle
can be described as a generalized stochastic system, then the particle’s
_velocity_ (or, equivalently, the particle’s _momentum_) will not generally
have a well-defined value at all times, and will naturally be representable as
an emergeable $\dot{A}$ along the lines described here.

###  V.2 The Measurement Process

With all the requisite conceptual background now in place, one can model the
measurement of a generic observable as a physical process. To start, consider
a composite system $\mathcal{S}\mathcal{D}\mathcal{E}$ consisting of three
subsystems that will be called a _subject system_ $\mathcal{S}$, a _measuring
device_ $\mathcal{D}$, and an _environment_ $\mathcal{E}$. Note that one of
the additional goals ahead will be to identify the criteria for a subsystem
like $\mathcal{D}$ to be regarded as a genuine measuring device.

Focusing momentarily on the subject system $\mathcal{S}$, consider an $N\times
N$ self-adjoint matrix

$\tilde{A}^{\mathcal{S}}=\tilde{A}^{\mathcal{S}\dagger},$ (107)

which may or may not be one of the subject system’s diagonal random
variables.171717More generally, one could take $\tilde{A}^{\mathcal{S}}$ to be
a _normal matrix_ , meaning a matrix that commutes with its adjoint
$\tilde{A}^{\mathcal{S}\dagger}$. As a concrete example,
$\tilde{A}^{\mathcal{S}}$ could be an emergeable like (104).

By the _spectral theorem_ , $\tilde{A}^{\mathcal{S}}$ has a _spectral
decomposition_ of the form

$\tilde{A}^{\mathcal{S}}=\sum_{\alpha}\tilde{a}_{\alpha}\tilde{P}_{\alpha}^{\mathcal{S}},$
(108)

where $\tilde{a}_{\alpha}$ are the eigenvalues of $\tilde{A}^{\mathcal{S}}$
and where $\tilde{P}_{\alpha}^{\mathcal{S}}$ are its eigenprojectors. These
eigenprojectors $\tilde{P}_{\alpha}^{\mathcal{S}}$ are not generically
diagonal, but they nonetheless satisfy the analogues of the mutual exclusivity
condition (24),

$\tilde{P}_{\alpha}^{\mathcal{S}}\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}=\delta_{\alpha\alpha^{\prime}}\tilde{P}_{\alpha}^{\mathcal{S}},$
(109)

and the completeness relation (25),

$\sum_{\alpha}\tilde{P}_{\alpha}^{\mathcal{S}}=\mymathbb{1}^{\mathcal{S}},$
(110)

where $\mymathbb{1}^{\mathcal{S}}$ is the identity matrix for the subject
system. These eigenprojectors therefore constitute a projection-valued measure
(PVM) of their own. Letting $\tilde{e}_{\alpha}^{\mathcal{S}}$ be the
corresponding orthonormal basis, one has, in analogy with (29), the following
statements:

$\tilde{e}_{\alpha}^{\mathcal{S}\dagger}\tilde{e}_{\alpha^{\prime}}^{\mathcal{S}}=\delta_{\alpha\alpha^{\prime}},\quad\tilde{e}_{\alpha}^{\mathcal{S}}\tilde{e}_{\alpha}^{\mathcal{S}\dagger}=\tilde{P}_{\alpha}^{\mathcal{S}}.$
(111)

If $\tilde{A}^{\mathcal{S}}$ happens to be one of the subject system’s random
variables (34), then the eigenvalues $\tilde{a}_{\alpha}$ are the random
variable’s magnitudes, and the eigenprojectors
$\tilde{P}_{\alpha}^{\mathcal{S}}$ are the configuration projectors (21). More
generally, however, the eigenvalues $\tilde{a}_{\alpha}$ and the
eigenprojectors $\tilde{P}_{\alpha}^{\mathcal{S}}$ do not yet have an obvious
physical meaning.

Suppose that the measuring device $\mathcal{D}$ has configurations $d(\alpha)$
that can be labeled by the same index $\alpha$ that appears in the spectral
decomposition (108). Similarly, suppose that the environment $\mathcal{E}$ has
configurations $e\mathopen{}\mathclose{{}\left(\alpha}\right)$ that can also
be labeled by $\alpha$.

Generalizing (79) from the earlier analysis of the decoherence process,
suppose, moreover, that the composite system
$\mathcal{S}\mathcal{D}\mathcal{E}$ evolves according to an overall
unistochastic transition matrix:

$\Gamma_{ide,i_{0}d_{0}e_{0}}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=|U_{ide,i_{0}d_{0}e_{0}}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)|^{2}.$
(112)

Generalizing (83) and letting
$\tilde{e}_{\alpha^{\prime},i^{\prime}}^{\mathcal{S}}$ denote the
$i^{\prime}$th component of the basis vector
$\tilde{e}_{\alpha^{\prime}}^{\mathcal{S}}$ with respect to the subject
system’s configuration basis $e_{i^{\prime}}^{\mathcal{S}}$, suppose that the
three subsystems interact up to a time $t^{\prime}>0$ in such a way that they
end up with the overall wave function181818It is straightforward to write down
idealized examples of suitable unitary time-evolution operators for the
composite system. One choice is
$U^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\equiv\sum_{\alpha^{\prime}}\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}\otimes
R_{d(\alpha^{\prime})}^{\mathcal{D}}\otimes
R_{e(\alpha^{\prime})}^{\mathcal{E}}$, where
$\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}$ is the $\alpha^{\prime}$th
eigenprojector appearing in the spectral decomposition (108), and where
$R_{d(\alpha^{\prime})}^{\mathcal{D}}$ and
$R_{e(\alpha^{\prime})}^{\mathcal{E}}$ are unitary transformations for the
measuring device and the environment that respectively put them in the
configurations $d(\alpha^{\prime})$ and $e(\alpha^{\prime})$.

$\Psi_{i^{\prime}d^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=\sum_{\alpha^{\prime}}\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha^{\prime},i^{\prime}}^{\mathcal{S}}\delta_{d^{\prime}d(\alpha^{\prime})}\delta_{e^{\prime}e(\alpha^{\prime})},$
(113)

and that, mirroring (81), the composite system’s relative time-evolution
operator factorizes between the three subsystems for later times
$t>t^{\prime}$:

$\mathopen{}\mathclose{{}\left.\begin{aligned}
U^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)&=U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\otimes
U^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)\\\
&\qquad\otimes U^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\ &\qquad\qquad\qquad\textrm{for
}t>t^{\prime}.\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(114)

Then the composite system’s wave function for times $t>t^{\prime}$ after the
interaction is

$\mathopen{}\mathclose{{}\left.\begin{aligned}
\Psi_{ide}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)&=\sum_{i^{\prime},e^{\prime},d^{\prime}}U_{ide,i^{\prime}d^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Psi_{i^{\prime}d^{\prime}e^{\prime}}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\\\
&=\sum_{i^{\prime}}\sum_{\alpha^{\prime}}U_{ii^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha^{\prime},i^{\prime}}^{\mathcal{S}}\\\
&\qquad\times
U_{dd(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)U_{ee(\alpha^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right).\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(115)

Invoking the Born rule (40), it follows from the explicit expression (115) for
the composite system’s wave function that the joint probabilities for
$t>t^{\prime}$ are given by

$p_{ide}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)=\mathopen{}\mathclose{{}\left|\Psi_{ide}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)}\right|^{2}.$
(116)

Marginalizing over the configuration $i$ of the subject system $\mathcal{S}$
as well as the configuration $e$ of the environment $\mathcal{E}$, and
invoking the unitarity of both the subject system’s relative time-evolution
operator $U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ and the environment’s relative time-evolution operator
$U^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow t^{\prime}}\right)$,
one obtains the standalone probabilities
$p_{d}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t}\right)$ for the measuring
device $\mathcal{D}$ alone for $t>t^{\prime}$:

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&p_{d}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{i,e}p_{ide}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)\\\
&=\sum_{i_{1}^{\prime},i_{2}^{\prime}}\sum_{\alpha_{1}^{\prime},\alpha_{2}^{\prime}}\overline{U_{dd(\alpha_{1}^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{\Psi}_{\alpha_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha_{1}^{\prime},i_{1}^{\prime}}^{\mathcal{S}}}\\\
&\qquad\times
U_{dd(\alpha_{2}^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{\Psi}_{\alpha_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha_{2}^{\prime},i_{2}^{\prime}}^{\mathcal{S}}\\\
&\qquad\times\sum_{i}\overline{U_{ii_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ii_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&\qquad\times\sum_{e}\overline{U_{ee(\alpha_{1}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ee(\alpha_{2}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&=\sum_{\alpha^{\prime}}|U_{dd(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2}|\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}.\end{aligned}\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(117)

In the limit $t\to t^{\prime}$, the last line of (117) implies that

$p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)=|\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}.$
(118)

Hence, the measuring device $\mathcal{D}$ has a standalone probability
$|\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}$
of ending up in its configuration $d(\alpha^{\prime})$, exactly as predicted
by the textbook version of the Born rule. One can then naturally define an
expectation value
$\langle\tilde{A}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\rangle$
for $\tilde{A}^{\mathcal{S}}$ at $t^{\prime}$ as the usual kind of statistical
average:

$\langle\tilde{A}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\rangle\equiv\sum_{\alpha}\tilde{a}_{\alpha}p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(119)

This analysis establishes that as long as there exists a form of unistochastic
time evolution (112) for the composite system
$\mathcal{S}\mathcal{D}\mathcal{E}$ that arrives at the wave function (113),
the matrix $\tilde{A}^{\mathcal{S}}$ represents a genuine observable, in the
sense that the time evolution (112) leads to the measuring device ending up in
the correct configuration with the correct Born-rule probability.

For times $t>t^{\prime}$ after the interaction, the last line of (117) implies
that the time $t^{\prime}$ is a division event for the measuring device:

$\Gamma^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t}\right)=\Gamma^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\Gamma^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\quad\textrm{for
}t>t^{\prime}.$ (120)

Here the measuring device’s dynamics for times $t>t^{\prime}$ is given by the
relative unistochastic transition matrix

$\Gamma_{dd(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\equiv|U_{dd(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)|^{2}.$ (121)

By contrast, if the observable $\tilde{A}^{\mathcal{S}}$ is an emergeable, as
opposed to one of the subject system’s (diagonal) random variables (34), then
the subject system $\mathcal{S}$ does not experience a division event at
$t^{\prime}$. Instead, the subject system remains mired in indivisible time
evolution at $t^{\prime}$, with some stochastically evolving underlying
configuration. Moreover, if indeed $\tilde{A}^{\mathcal{S}}$ is an emergeable,
then the measurement result obtained by the measuring device is an emergent
effect of the interaction between the subject system and the measuring device,
in a sense suggested by Bohr in 1935 [58, 59], rather than transparently
revealing a physical aspect of the configuration of the subject system alone.

Despite $t^{\prime}$ not necessarily being a division event for the subject
system $\mathcal{S}$, one can nevertheless compute the standalone probability
$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for the subject
system to be in its $i$th configuration for times $t>t^{\prime}$ by
marginalizing over the measuring device $\mathcal{D}$ and the environment
$\mathcal{E}$:

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{d,e}p_{ide}^{\mathcal{S}\mathcal{D}\mathcal{E}}\mathopen{}\mathclose{{}\left(t}\right)\\\
&=\sum_{i_{1}^{\prime},i_{2}^{\prime}}\sum_{\alpha_{1}^{\prime},\alpha_{2}^{\prime}}\overline{U_{ii_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{\Psi}_{\alpha_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha_{1}^{\prime},i_{1}^{\prime}}^{\mathcal{S}}}\\\
&\qquad\times
U_{ii_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{\Psi}_{\alpha_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{e}_{\alpha_{2}^{\prime},i_{2}^{\prime}}^{\mathcal{S}}\\\
&\qquad\times\sum_{d}\overline{U_{dd(\alpha_{1}^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{dd(\alpha_{2}^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&\qquad\times\sum_{e}\overline{U_{ee(\alpha_{1}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ee(\alpha_{2}^{\prime})}^{\mathcal{E}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&=\sum_{\alpha^{\prime}}\mathopen{}\mathclose{{}\left[\sum_{i_{1}^{\prime},i_{2}^{\prime}}\overline{U_{ii_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ii_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{e}_{\alpha^{\prime},i_{2}^{\prime}}^{\mathcal{S}}\overline{\tilde{e}_{\alpha^{\prime},i_{1}^{\prime}}^{\mathcal{S}}}}\right]\\\
&\qquad\times|\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}.\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(122)

Recognizing
$|\tilde{\Psi}_{\alpha^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)|^{2}$
from (118) as the standalone probability
$p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$
for the measuring device $\mathcal{D}$ to end up in its configuration
$d(\alpha^{\prime})$ at the time $t^{\prime}$, and recalling both the
configuration projectors $P_{i}^{\mathcal{S}}$ defined in (21) as well as the
eigenprojectors $\tilde{P}_{\alpha}^{\mathcal{S}}$ appearing in the spectral
decomposition (108) for $\tilde{A}^{\mathcal{S}}$, one can write (122) more
succinctly as

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(P_{i}^{\mathcal{S}}\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)).$
(123)

Here the subject system’s density matrix
$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for $t>t^{\prime}$
is given by

$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)\equiv
U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\mathopen{}\mathclose{{}\left[\sum_{\alpha^{\prime}}p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}}\right]U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right).$ (124)

One can therefore recast the expectation value (119) for
$\tilde{A}^{\mathcal{S}}$ as

$\langle\tilde{A}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)\rangle=\mathrm{tr}(\tilde{A}^{\mathcal{S}}\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)),$
(125)

which precisely mirrors the formula (33) for the expectation value of a
(diagonal) random variable.

Furthermore, (122) yields a _linear_ relationship between the standalone
probabilities
$p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$
for the measuring device $\mathcal{D}$ at $t^{\prime}$ and the standalone
probabilities $p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for
the subject system $\mathcal{S}$ at $t>t^{\prime}$:

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\sum_{\alpha^{\prime}}\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(126)

The entries
$\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)$ of the _hybrid_ relative transition matrix appearing here
are given explicitly by

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&\equiv\sum_{i_{1}^{\prime},i_{2}^{\prime}}\overline{U_{ii_{1}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)}U_{ii_{2}^{\prime}}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{e}_{\alpha^{\prime},i_{2}^{\prime}}^{\mathcal{S}}\overline{\tilde{e}_{\alpha^{\prime},i_{1}^{\prime}}^{\mathcal{S}}}.\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(127)

Because these matrix entries do not depend on the measuring device’s
standalone probabilities
$p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$,
they naturally serve as _conditional probabilities_ for the subject system
$\mathcal{S}$ to be in its $i$th configuration at the time $t>t^{\prime}$,
given that the measuring device $\mathcal{D}$ is in its configuration
$d(\alpha^{\prime})$ at $t^{\prime}$:

$p^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(i,t|d(\alpha^{\prime}),t^{\prime}}\right)\equiv\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right).$ (128)

###  V.3 Wave-Function Collapse

Importantly, notice that one can write the hybrid transition matrix (127) in
an overall form that resembles the dictionary (26):

$\mathopen{}\mathclose{{}\left.\begin{aligned}
&\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\\\
&\qquad=\mathrm{tr}(U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)P_{i}^{\mathcal{S}}U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{P}_{\alpha}^{\mathcal{S}}).\end{aligned}\negthickspace\negthickspace\negthickspace\negthickspace\qquad}\right\\}$
(129)

Rearranging the right-hand side gives the equation

$\Gamma_{i,d(\alpha^{\prime})}^{\mathcal{S}\mathcal{D}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)=\mathrm{tr}(P_{i}^{\mathcal{S}}\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)),$
(130)

with a _conditional_ density matrix
$\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)$
for the subject system $\mathcal{S}$ at the time $t>t^{\prime}$ naturally
defined by time-evolving the eigenprojector
$\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}$ from $t^{\prime}$ to $t$:

$\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)\equiv
U^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{P}_{\alpha^{\prime}}^{\mathcal{S}}U^{\mathcal{S}\dagger}\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right).$ (131)

Thus, the calculation (122) reduces to the statement that the standalone
probabilities $p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ for
the subject system at $t>t^{\prime}$ are given by

$p_{i}^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)=\mathrm{tr}(P_{i}^{\mathcal{S}}\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)),$
(132)

where the subject system’s density matrix
$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$, which was
originally defined in (124), can equivalently be expressed as a probabilistic
mixture of the conditional density matrices
$\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)$
defined in (131), statistically weighted by the measurement probabilities
$p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right)$:

$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)\equiv\sum_{\alpha^{\prime}}\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)p_{d(\alpha^{\prime})}^{\mathcal{D}}\mathopen{}\mathclose{{}\left(t^{\prime}}\right).$
(133)

Taking stock of these results, one sees that to make future predictions for
$t>t^{\prime}$ about the subject system $\mathcal{S}$, conditioned on the
measuring device’s result $d(\alpha^{\prime})$ at $t^{\prime}$, one uses the
conditional probabilities (130), in which the subject system’s density matrix
has been effectively replaced with the conditional density matrix
$\rho^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)$.
This conditional density matrix corresponds to a _collapsed_ state vector or
wave function defined as

$\Psi^{\mathcal{S}|\alpha^{\prime},t^{\prime}}\mathopen{}\mathclose{{}\left(t}\right)\equiv
U\mathopen{}\mathclose{{}\left(t\leftarrow
t^{\prime}}\right)\tilde{e}_{\alpha}^{\mathcal{S}}.$ (134)

The phenomenon of _wave-function collapse_ therefore reduces to a prosaic
example of conditioning, a conclusion that represents another new result.

By contrast, for an observer who does not know the specific measurement result
$d(\alpha^{\prime})$, the correct density matrix
$\rho^{\mathcal{S}}\mathopen{}\mathclose{{}\left(t}\right)$ to use is the one
defined in (133). Again, this density matrix consists of an appropriate
probabilistic mixture of conditional or collapsed density matrices that are
statistically weighted over the measurement results.

###  V.4 The Measurement Problem

According to the foregoing treatment of the measurement process, a measuring
device is an ordinary physical system that can carry out a measurement of an
observable, and then ends up in a final configuration that reflects a definite
measurement outcome. The probabilities for a measuring device’s various
possible measurement outcomes are given by the textbook Born rule (118), and
conditioning on the specific measurement outcome leads to the textbook formula
(134) for wave-function collapse. Hence, the picture of quantum theory
presented in this paper arguably has the resources to solve the measurement
problem [60].

The stochastic-quantum correspondence is also helpful for understanding the
measurement process in another important way. Textbook treatments of quantum
theory typically regard measuring devices as axiomatic primitives or posits,
without providing clear principles for deciding which kinds of systems merit
being called measuring devices and which do not. The approach taken toward the
measurement process in this paper not only gives a candidate resolution of the
measurement problem, but also yields a natural set of criteria for defining
what counts as a good measuring device in the first place, without the need to
regard measuring devices as special among all other systems in any truly
fundamental way.

Based on the approach in this paper, one sees that a good measuring device
should be a physical system with at least as many configurations as possible
outcomes for the observable to be measured (at least up to the desired level
of experimental resolution), it should admit an overall form of dynamics that
results in the correct final correlations, and it should be in sufficiently
strong contact with a noisy and intrusive environment to generate a robust
division event at the conclusion of the measurement interaction. It is worth
noting that the first two of these three criteria would be standard
requirements for a measuring device even without worrying about indivisible
stochastic dynamics or quantum theory.191919Without the third criterion—strong
contact with an environment—one obtains a “latent measurement” in the language
of [61, 56].

###  V.5 The Uncertainty Principle

Again, the foregoing treatment of the measurement process leads to the
textbook Born rule (118) and the textbook formula (134) for wave-function
collapse. As a consequence, any pair of observables $\tilde{A},\tilde{B}$ and
their respective standard deviations $\Delta\tilde{A},\Delta\tilde{B}$ will
satisfy the _Heisenberg-Robertson uncertainty principle_ [62, 63],

$\Delta\tilde{A}\,\Delta\tilde{B}\geq\frac{1}{2}|\mathrm{tr}(i[\tilde{A}\tilde{B}-\tilde{B}\tilde{A}]\rho)|,$
(135)

as follows from any of the standard proofs.202020Here is one proof, which is
adapted from [64]: Because density matrices are positive semidefinite, it
follows from the spectral theorem that for any $N\times N$ matrix $X$ and any
$N\times N$ density matrix $\rho$, one has the inequality
$\mathrm{tr}(X^{\dagger}X\rho)\geq 0$. Let $\tilde{A},\tilde{B}$ be any pair
of observables, assumed without any real loss of generality to have vanishing
expectation values $\langle\tilde{A}\rangle=\langle\tilde{B}\rangle=0$. Let
$x$ be a variable real number and let $X\equiv x\tilde{A}+i\tilde{B}$. Then
$\mathrm{tr}(X^{\dagger}X\rho)\geq 0$ becomes the inequality
$f\mathopen{}\mathclose{{}\left(x}\right)\geq 0$, where
$f\mathopen{}\mathclose{{}\left(x}\right)\equiv ax^{2}+bx+c$ is a quadratic
function with coefficients $a\equiv(\Delta\tilde{A})^{2}$,
$b\equiv\mathrm{tr}(i[\tilde{A}\tilde{B}-\tilde{B}\tilde{A}]\rho)$, and
$c\equiv(\Delta\tilde{B})^{2}$. The minimum value of
$f\mathopen{}\mathclose{{}\left(x}\right)$ is $-b^{2}/4a+c$, so one arrives at
the inequality $-b^{2}/4a+c\geq 0$, which one can rearrange to give the
Heisenberg-Robertson uncertainty principle. QED

The stochastic-quantum correspondence goes beyond replicating the uncertainty
principle by painting a clearer picture of what the uncertainty principle
physically means. Consider for simplicity the case in which $\tilde{A}=A$ is a
random variable, or beable, and $\tilde{B}$ is an emergeable, in the language
of Subsection V.1. Then $A$ has a direct interpretation solely in terms of the
subject system’s configuration space, whereas $\tilde{B}$ encodes an emergent
pattern in the subject system’s dynamics that can nonetheless show up in the
measurement outcomes of a measuring device.

Suppose that $A$ has a definite value or magnitude at some initial time $0$.
Then the subject system must be in a specific configuration with probability
$1$ at the initial time $0$. The overall stochastic dynamics will then lead to
uncertainty in the outcome of any measurement of $\tilde{B}$.

Suppose that one goes ahead and measures $\tilde{B}$, so that a definite
measurement outcome emergently shows up in the configuration of a measuring
device at some time $t^{\prime}>0$. Then the analysis in Subsection V.3
implies that there is an inevitable disturbance in the subject system that
leads its density matrix to end up effectively as a non-diagonal matrix equal
to an eigenprojector of $\tilde{B}$. A non-diagonal density matrix signifies
that the system is in the midst of an indivisible stochastic process, as
explained in Subsection IV.7. In the present circumstances, that indivisible
stochastic process is precisely one that would ensure that if $\tilde{B}$ were
measured again shortly after $t^{\prime}$, then the measuring device would
obtain the same outcome for $\tilde{B}$ as before. However, being in the s of
an indivisible stochastic process also implies uncertainty in the subject
system’s underlying configuration, thereby rendering the value of $A$
uncertain.

##  VI Further Implications

###  VI.1 Symmetries

The stochastic-quantum correspondence developed in this paper provides new
ways to think about _dynamical symmetries_ in quantum theory, meaning
transformations that leave the dynamics invariant. Going in the other
direction, the stochastic-quantum correspondence also makes it more
straightforward to impose dynamical symmetries systematically as constraints
in the construction of the dynamics for a given generalized stochastic system.

Classically, any invertible transformation of a system’s configurations
$i=1,\dots,N$ is a permutation transformation of the configuration projectors
(21):

$\mathopen{}\mathclose{{}\left.\begin{aligned} &P_{i}\mapsto
P_{\sigma\mathopen{}\mathclose{{}\left(i}\right)},\\\ &\quad\textrm{with
}\mathopen{}\mathclose{{}\left\\{\sigma\mathopen{}\mathclose{{}\left(1}\right),\dots,\sigma\mathopen{}\mathclose{{}\left(N}\right)}\right\\}=\mathopen{}\mathclose{{}\left\\{1,\dots,N}\right\\}.\end{aligned}\qquad}\right\\}$
(136)

More generally, a transformation between two PVMs $P_{1},\dots,P_{N}$ and
$\tilde{P}_{1},\dots,\tilde{P}_{N}$ is always a similarity transformation of
the form

$P_{i}\mapsto\tilde{P}_{i}\equiv V^{\dagger}P_{i}V,$ (137)

where $V$ is some unitary operator.212121Proof: Let $e_{1},\dots,e_{N}$ be the
orthonormal configuration basis (27), with $e_{i}^{\dagger}e_{j}=\delta_{ij}$
and $e_{i}e_{i}^{\dagger}=P_{i}$ as in (29), and let
$\tilde{e}_{1},\dots,\tilde{e}_{N}$ be an orthonormal basis related to the new
projectors $\tilde{P}_{i}$ in the analogous way, with
$\tilde{e}_{i}^{\dagger}\tilde{e}_{j}=\delta_{ij}$ and
$\tilde{e}_{i}\tilde{e}_{i}^{\dagger}=\tilde{P}_{i}$. Then the $N\times N$
matrix defined by $V\equiv\sum_{i}e_{i}\tilde{e}_{i}^{\dagger}$ is unitary and
satisfies $V^{\dagger}P_{i}V=\tilde{P}_{i}$. Going the other way, if $V$ is an
$N\times N$ unitary matrix, then the $N\times N$ matrices defined for
$i=1,\dots,N$ by $\tilde{P}_{i}\equiv V^{\dagger}P_{i}V$ are guaranteed to
constitute a PVM. QED This similarity transformation reduces to the
configurational transformation (136) if and only if $V$ is a permutation
matrix.