# An Exact Theory of Causal Emergence for Linear Stochastic Iteration Systems

Kaiwei Liu  School of Systems Science, Beijing Normal University, 100875,
Beijing, China  Bing Yuan  Swarma Research, 102300, Beijing, China  Jiang
Zhang Email<EMAIL_ADDRESS>School of Systems Science, Beijing Normal
University, 100875, Beijing, China  Swarma Research, 102300, Beijing, China

###### Abstract

After coarse-graining a complex system, the dynamics of its macro-state may
exhibit more pronounced causal effects than those of its micro-state. This
phenomenon, known as causal emergence, is quantified by the indicator of
effective information. However, two challenges confront this theory: the
absence of well-developed frameworks in continuous stochastic dynamical
systems and the reliance on coarse-graining methodologies. In this study, we
introduce an exact theoretic framework for causal emergence within linear
stochastic iteration systems featuring continuous state spaces and Gaussian
noise. Building upon this foundation, we derive an analytical expression for
effective information across general dynamics and identify optimal linear
coarse-graining strategies that maximize the degree of causal emergence when
the dimension averaged uncertainty eliminated by coarse-graining has an upper
bound. Our investigation reveals that the maximal causal emergence and the
optimal coarse-graining methods are primarily determined by the principal
eigenvalues and eigenvectors of the dynamic system’s parameter matrix, with
the latter not being unique. To validate our propositions, we apply our
analytical models to three simplified physical systems, comparing the outcomes
with numerical simulations, and consistently achieve congruent results.

Keywords:  Causal Emergence, Effective Information, Linear Stochastic
Iteration System, Coarse-graining.

##  1 Introduction

Many complex systems [1] in reality, such as cities [2, 3, 4], companies [5,
6, 7], bird flocks [8, 9], perception systems [10, 11, 12], brains [13, 14,
15], cells [16, 17], molecules, etc., all exhibit emergent behaviors or
statistical laws on a macro-level that cannot be simply derived from micro-
level properties or dynamics. However, how to quantitatively characterize
emergence has only recently garnered widespread attention [18, 19, 20, 21, 22,
23, 24]. The theories of causal emergence attempt to capture the conception of
emergence from the point of view of causality. Intuitively, causal emergence
refers to the phenomenon in dynamical systems where stronger causal effects
can be obtained on macro-states by coarse-graining the micro-states [25].
There are three different ways to quantify the idea of causal emergence, which
are based on effective information [22, 23], partial information decomposition
[26, 27, 21, 15, 24], and dynamical independence [20, 14, 10], respectively.
In this paper, we mainly focus on the first way.

Erik Hoel introduced the initial quantitative theory for causal emergence,
founded on effective information ($EI$) [22, 23, 25]. Nevertheless, the
original framework is limited to quantifying discrete Markov chains in both
the time domain and state space [28, 29], overlooking continuous state spaces.
To extend the causal emergence theory in continuous spaces, P. Chvykov and E.
Hoel put forth the theory of causal geometry [30], wherein they devised a
method for calculating $EI$ in functional mappings across continuous state
spaces [31, 6]. Nonetheless, this theory solely explores general functional
mappings and neglects the multi-step dynamical evolution, rendering it
inapplicable to dynamical systems in continuous state spaces. Furthermore, all
of Hoel’s theories [22, 23, 25, 32, 30, 33] necessitate a predefined coarse-
graining strategy to discern instances of causal emergence. While this coarse-
graining strategy can be derived by maximizing $EI$ for macro-dynamics,
solving the optimization problem for continuous variables proves challenging
[34, 6, 35].

There exist various approaches to coarse-grain the micro-states of complex
systems, including manually designed renormalization methods [36], traditional
dimensionality reduction techniques [37, 38, 39, 40, 41], and machine
learning-based methods for coarse-graining or renormalization [34, 6, 42]. To
automatically discover coarse-graining strategies that optimize causal
emergence, J. Zhang and K. Liu introduced a machine-learning framework known
as the Neural Information Squeezer (NIS) [43], employing reversible neural
networks. This framework facilitates the automatic extraction of effective
coarse-graining strategies and macro-state dynamics, enabling the direct
identification of causal emergence from diverse time series data.
Subsequently, the research team proposed the enhanced NIS+ framework [35],
which integrates the optimization of coarse-graining through $EI$ maximization
into machine learning. NIS and NIS+ have successfully addressed the challenge
of identifying causal emergence in data by optimizing coarse-graining
techniques and macro-dynamics within continuous stochastic dynamical systems
through $EI$ maximization. Nonetheless, machine learning-based methods [44,
45, 46] rely heavily on data adequacy and the level of neural network
training. While they can provide numerical solutions, they lack a ground truth
for assessing the quality of model training and the reliability of outcome
indicators.

To overcome the limitations identified in previous studies, this paper will
first derive analytical solutions for $EI$ and causal emergence in linear
stochastic iterative systems. It will then determine the explicit expression
for maximal causal emergence and the corresponding optimal coarse-graining
strategies, under the condition that the dimension-averaged uncertainty
eliminated by coarse-graining is constrained by a specified threshold.
Subsequent to this, numerical simulation experiments will be conducted on
simplified physical systems to validate the theoretical findings.
Additionally, we will explore challenges such as maximizing causal emergence
under an alternative condition involves constructing the coarse-graining map
through a combination of dimensionality reduction projection and rotation,
showcasing the interplay between determinism and degeneracy, examining the
connection between maximizing causal emergence and minimizing prediction
errors in dynamics, and addressing nonlinear dynamics. The primary objective
of this research is to address deficiencies in $EI$ related to Transition
Probability Matrices (TPM) [22, 25] and causal geometry within a toy model
context [30]. Furthermore, it aims to lay a theoretical groundwork to enhance
the identification of causal emergence through machine learning [43, 35].

##  2 Basic Notions and Problems Formulation

###  2.1 Continuous linear stochastic iteration system

A stochastic iteration system [47] is a sequence of random variables $x_{t}$
at different times $t$ which usually represent the dynamic evolution of a
certain quantity, such as stock prices, temperature changes, traffic flows,
etc. This paper mainly focuses on linear high-dimensional stochastic iteration
systems [48] to reduce our calculation.

For variable $x_{t}\in\mathcal{R}^{n}$ at time $t$ in the stochastic iteration
system, the evolution of $x_{t}$ follows the stochastic iterative equation

$\displaystyle x_{t+1}=Ax_{t}+\varepsilon_{t}.$ (2.1)

We define matrix $A\in\mathcal{R}^{n\times n}$ as a dynamical parameter
matrix. In Equation (2.1), $\varepsilon_{t}\sim\mathcal{N}(0,\Sigma)$
represents Gaussian noises with zero means and the covariance matrix
$\Sigma\in\mathcal{R}^{n\times n}$. We specify the ranks of matrices ${\rm
rk}(A)={\rm rk}(\Sigma)=n$. ${\rm rk}(\cdot)$ means the rank of a matrix. We
can easily observe that the probability of $x_{t+1}$ falling in a region
centered with $Ax_{t}$.

###  2.2 Coarse-graining

In the realm of linear stochastic iterative systems, we confine coarse-
graining strategies to linear maps. Therefore, a direct approach entails
employing the relatively straightforward projection matrix coarse-graining
method.

Generally, the coarse-graining strategy can be represented as
$\phi(x_{t}):\mathbb{R}^{n}\to\mathbb{R}^{m},m<n$. It is designed to map
micro-states to a lower-dimensional space and to convert micro-states into
macro-states.

######  Definition 2.1.

(Coarse-graining strategy): For variable $x_{t}$ at time $t$ in the linear
stochastic iteration system, we define linear mapping

$\displaystyle\phi(x_{t})=Wx_{t},$ (2.2)

where, $W\in\mathcal{R}^{k\times n}$, ${\rm rk}(W)=k$, to map $x_{t}$ in high-
dimensional space $\mathcal{R}^{n}$ to $y_{t}$ in low-dimensional space
$\mathcal{R}^{k}$, achieving coarse-graining of variables

$\displaystyle y_{t}=\phi(x_{t})=Wx_{t}$ (2.3)

in the stochastic iteration system.

###  2.3 Macro dynamical system

Coarse-graining strategy can map a set of micro-states to specific macro-
states, thus can naturally derive the expressions from micro-state parameter
matrices $A$ and $\Sigma$ for dynamical models on macro-state space which can
be described by macro-state parameter matrices $A_{M}$ and $\Sigma_{M}$. Since
$W\in\mathcal{R}^{n\times k}$ in linear coarse-graining strategy is
irreversible, so we need Moore-Penrose generalized inverse matrix [49]
$W^{\dagger}$ in matrix theory [50] to derive $A_{M}$.

According to Equations (2.3) and (2.1), we can get that

$\displaystyle\begin{aligned}
y_{t+1}&=Wx_{t+1}=WAx_{t}+W\varepsilon_{t}.\end{aligned}$ (2.4)

Due to the fact that both the micro-dynamical model itself and the coarse-
graining strategies are linear transformations, and based on the properties of
linear algebra, there is also a set of equations for the macro-dynamics. Thus,
we hope that there is a matrix $A_{M}$ satisfying the following equation:

$\displaystyle\begin{aligned}
y_{t+1}&=A_{M}y_{t}+\varepsilon_{M,t},\end{aligned}$ (2.5)

And according to Equations (2.5), (2.4) and $y_{t}=Wx_{t}$, we can get that

$\displaystyle\begin{cases}WA=A_{M}W,\\\
W\varepsilon_{t}=\varepsilon_{M,t}.\end{cases}$ (2.6)

Since $\varepsilon_{t}\sim\mathcal{N}(0,\Sigma)$,
$\varepsilon_{m,t}=W\varepsilon_{t}\sim\mathcal{N}(0,\Sigma_{M})$.
$W\in\mathcal{R}^{n\times k}$ is irreversible, so we require the Moore-Penrose
generalized inverse matrix $W^{\dagger}\in\mathcal{R}^{n\times
k},WW^{\dagger}=I_{k}$, to solve Equation (2.6). Multiply both sides of the
first equation of Equation (2.6) by $W^{\dagger}$, we can obtain the macro-
state parameter matrix

$A_{M}=WAW^{\dagger},$ (2.7)

and the macro dynamics in the Definition 2.2 as follows.

######  Definition 2.2.

(Macro dynamics): For the linear stochastic iteration system in Equation
(2.1), the variables $x_{t}\in\mathcal{R}^{n},t=0,1,\dots$, are mapped to
macro state variables $y_{t}\in\mathcal{R}^{k},t=0,1,\dots$, and there exists
a new evolutionary dynamics between macro state variables, which we call macro
dynamics. Macro dynamics can be represented by macro iterative equations

$\displaystyle y_{t+1}=A_{M}y_{t}+\varepsilon_{M,t},$ (2.8)

in which $A_{M}=WAW^{\dagger}\in\mathcal{R}^{k\times k}$,
$\varepsilon_{m,t}\sim\mathcal{N}(0,\Sigma_{M}),\Sigma_{m}=W\Sigma W^{T}$.
$W^{\dagger}$ is the Moore-Penrose generalized inverse matrix of $W$.

###  2.4 Inverse-coarse-graining

To evaluate the information loss caused by coarse-graining and derive macro
dynamics, we also need to transform the macro-state back to the micro-state,
which is called inverse-coarse-graining. Due to the difference in dimensions
$n$ and $k$ of the vectors before and after transformation, coarse-graining is
not a reversible process. For linear transformations, irreversibility is
manifested as $W\in\mathcal{R}^{n\times k}$ in linear coarse-graining is
irreversible.

######  Definition 2.3.

(Inverse-coarse-graining): As $W^{\dagger}$ is the Moore-Penrose generalized
inverse matrix of $W$, we define

$\displaystyle\phi^{\dagger}(y_{t})=W^{\dagger}y_{t},$ (2.9)

$W^{\dagger}\in\mathcal{R}^{n\times k}$, ${\rm rk}(W^{\dagger})=k$ as the
inverse-coarse-graining mapping of variables in the stochastic iteration
system. By applying the inverse-coarse-graining

$\displaystyle\hat{x}_{t}=W^{\dagger}y_{t},$ (2.10)

we can map the variable $y_{t}\in\mathcal{R}^{k}$ after dimensionality
reduction back to a high-dimensional space $\mathcal{R}^{n}$.

With inverse-coarse-graining, we can then estimate the prediction loss
function of the dynamics after coarse-graining.

##  3 Effective information and causal emergence

With the definitions and expressions of micro dynamics and macro dynamics, we
can study the phenomenon of causal emergence. We start with the calculation of
effective information and then use the method of difference of effective
information between micro-dynamics and macro-dynamics to calculate the degree
of causal emergence.

###  3.1 Effective information

We refer to Hoel’s effective information ($EI$) metric to measure the
magnitude of causal effects in dynamical systems. In general, $EI$ can be
understood as the mutual information between the probability distributions of
the states of variables at two different time points when the state at the
earlier time is intervened artificially and keeps the causal mechanism, i.e.,
the transitional probability $Pr(x_{t+1}|x_{t})$ unchanged, which is

$EI=I(\tilde{x}_{t+1},x_{t}|do(x_{t}\sim U(X))),$ (3.1)

where $x_{t}\in X$, and $U(X)$ means the uniform distribution or maximum
entropy distribution on $X$. The intervention is formalized using Judea
Pearl’s theory [51] of causality, particularly through $do(\cdot)$ operations,
which means artificially defining the probability distribution space of a
variable. Consequently, the distribution of $x_{t+1}$ will be indirectly
altered by the intervention on $x_{t}$ through the causal mechanism
$Pr(x_{t+1}|x_{t})$. Therefore, $\tilde{x}_{t+1}$ denotes the random variable
representing $x_{t+1}$ following the intervention on $x_{t}$. To be noticed,
Equation (3.1) is only an operational definition and the intervention of $do$
operator is just an imaginary operation to calculate $EI$ but has no real
physical meanings.

The most classic applications of $EI$ are in the analysis of transitional
probability matrix (TPM) for Markov chains. Assuming our probability
transition matrix is $M=(M_{ij})_{n\times
n}=(M_{1}^{T},\dots,M_{i}^{T},\dots,M_{n}^{T})^{T}\in\mathcal{R}^{n\times n}$,
$M_{i}=(M_{i1},\dots,M_{ij},\dots,M_{in})\in\mathcal{R}^{1\times n}$, $1\leq
i,j\leq n$. Intervention on Markov transition matrix at $t$, let
$P(X_{t}=x_{t})=1/n$, then the state at $t+1$ will follow the effect
distribution
$E_{D}=(\sum_{i}M_{i1}/n,\dots,\sum_{i}M_{ij}/n,\dots,\sum_{i}M_{in}/n)\in\mathcal{R}^{1\times
n}$. So, $EI$ is [18]:

$\displaystyle
EI(M)=\frac{1}{n}\sum_{i=1}^{n}D_{KL}(M_{i}||E_{D})=\frac{1}{n}\sum_{i,j}M_{ij}\log_{2}\left(\frac{nM_{ij}}{\sum_{l}M_{lj}}\right),$
(3.2)

where $D_{KL}(\cdot||\cdot)$ is the KL-divergence between two probability
distributions. The benefit of expressing Equation (3.1) to the new form of
Equation (3.2) is to stress that $EI$ is only the function of the
dynamics($M$) and independent of the distribution of $x_{t}$. This calculation
method requires both variable time and state space to be discrete.

According to previous works [22, 25, 29], effective information $EI(M)$ can be
divided into two parts, determinism $-\left<H(M_{i}))\right>_{i}$ and
degeneracy $H(E_{D})$, as

$\displaystyle\begin{aligned} EI(M)&=-\left<H(M_{i}))\right>_{i}+H(E_{D})\\\
&=\mathop{\frac{1}{n}\sum_{i,j}M_{ij}\log_{2}\left(M_{ij}\right)}_{Determinism}+\mathop{\sum_{j}\left[-\frac{\sum_{l}M_{lj}}{n}\log_{2}\left(\frac{\sum_{l}M_{lj}}{n}\right)\right]}_{Degeneracy}.\end{aligned}$
(3.3)

Determinism measures how reliably $x_{t}$ leads to the future state $x_{t+1}$
of the system. Degeneracy measures to what degree there is deterministic
convergence (not due to noise) from other states onto the future states
$x_{t+1}$ specified by ${x_{t}}$, degeneracy refers to multiple ways of
deterministically achieving the same effect. Two values can be calculated
separately to obtain $EI$.

Further, we can generalize $EI$ to a continuous state space. We should replace
summations with integrals, and change the intervened distribution of $x_{t}$
from a discrete equal probability distribution $P(x_{t})=1/n$ to a continuous
uniform distribution $U([-L/2,L/2]^{n})$ in which probability density function
is $p(x_{t})=1/L^{n}$. Since the intervened distribution can not be defined on
an infinite space such as $\mathcal{R}^{n}$, we set the intervention on a
limited sub-region of $\mathcal{R}^{n}$ which is $[-L/2,L/2]^{n}$, where $L$
is a hyperparameter with a value being at least larger than the maximum of
$x_{t}$ within finite time steps.

We can refer to Hoel’s framework [22, 32] to derive an expression for the
effective information of linear stochastic iteration system (Equation 2.1).
Actually, according to Equation 2.1, the conditional distribution of $x_{t+1}$
under given $x_{t}$ is a Gaussian distribution,
$p(x_{t+1}|x_{t})=\mathcal{N}(Ax_{t},\Sigma)$. Therefore, we can compute the
$EI$ for this Gaussian distribution [43] as $EI=I(x_{t+1},x_{t}|do(x_{t}\sim
U))$, drawing an analogy to mutual information and information entropy within
a Gaussian distribution [52] as

$\displaystyle EI(A,\Sigma)=\ln\displaystyle\frac{|\det(A)|L^{n}}{(2\pi
e)^{\frac{n}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2}}}.$ (3.4)

Similar to $EI(M)$ of discrete systems, $EI(A,\Sigma)$ of continuous systems
can also be decomposed into two terms: determinism
$-\left<H(p(x_{t+1}|do(x_{t})))\right>$ and degeneracy $H(E_{D}(x_{t+1}))$ as

$\displaystyle\begin{aligned}
EI(A,\Sigma)&=-\left<H(p(x_{t+1}|x_{t}))\right>+H(E_{D}(x_{t+1}))\\\
&=\mathop{-\ln\left[(2\pi
e)^{\frac{n}{2}}\det(\Sigma)^{\frac{1}{2}}\right]}_{Determinism}+\mathop{\ln\left(|det(A)|L^{n}\right)}_{Degeneracy}.\end{aligned}$
(3.5)

The higher the determinism or the lower the degeneracy, the stronger the
causal effect of the stochastic iteration system [22]. The specific derivation
process can refer to the calculation process in Appendix A.

Since the calculation of $EI$ involves multiple integrals, $L^{n}$ appears in
the expression of $EI$. When the value of $L$ is high, the value of $EI$
becomes higher after $n$ powers of $L$. So we stipulate that when calculating
causal emergence, the $EI$ in different dimensions is dimensionally averaged
to eliminate the impact of power growth of $L$. Another reason for taking the
average on the dimension is that for the comparison of $EI$ from different
dimensions, dimension averaged $EI$ can eliminate $L$, which is shown in
Section 3.2. Here, in Theorem 3.1, we define a new index that is effective
information for stochastic iterating systems ($\mathcal{J}$).

######  Definition 3.1.

(Effective information for stochastic iterating systems): For the linear
stochastic iteration systems like Equation (2.1), the effective information of
the dynamical system is calculated as

$\displaystyle\mathcal{J}(A,\Sigma)\equiv\frac{EI(A,\Sigma)}{n}=\frac{1}{n}\ln\displaystyle\frac{|\det(A)|L^{n}}{(2\pi
e)^{\frac{n}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2}}}=\ln\displaystyle\frac{|\det(A)|^{\frac{1}{n}}L}{(2\pi
e)^{\frac{1}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2n}}}$ (3.6)

$\det(\cdot)$ represents the determinant value corresponding to a matrix
$\cdot$. $|\cdot|$ represents the absolute value of $\cdot$. $L$ represents
the interval size of $do(\cdot)$ intervention.

The effective information $\mathcal{J}$ can also be decomposed into two parts,
determinism and degeneracy as

$\displaystyle\begin{aligned} \mathcal{J}(A,\Sigma)&=\mathop{-\ln\left[(2\pi
e)^{\frac{1}{2}}\det(\Sigma)^{\frac{1}{2n}}\right]}_{Determinism}+\mathop{\ln\left(|det(A)|^{\frac{1}{n}}L\right)}_{Degeneracy}.\end{aligned}$
(3.7)

The proof of Theorem 3.1 can refer to Appendix A. $\mathcal{J}(A,\Sigma)$ can
be used to measure the causal effects of micro dynamics and macro dynamics
then calculate causal emergence. To be noticed, the scope $L$ of $do$
intervention in macro-state dynamics $EI$ is related to
$W\in\mathcal{R}^{n\times k}$. So we need to limit the range of $W$ to ensure
the $do$ intervention range of macro-state $y_{t}\in\mathcal{R}^{k}$ and
micro-state $x_{t}\in\mathcal{R}^{n}$ as $do(y_{t})\sim U([-L/2,L/2]^{k})$ and
$do(x_{t})\sim U([-L/2,L/2]^{n})$, which we will conduct more analysis in
Subsection 3.4.

###  3.2 Causal emergence

After obtaining effective information for stochastic iterating systems
$\mathcal{J}$ at different dimensions $k$ of variables $x_{t}$, we still have
a problem to solve. Except for the influence of indices, the value of
hyperparameter $L$ itself, which is artificially assumed, also has a great
influence on the results of $\mathcal{J}$. This hyperparameter can be
eliminated by calculating causal emergence for stochastic iterating systems as
Definition 3.2.

######  Definition 3.2.

(Causal emergence for stochastic iterating systems) Hyperparameter $L$ can be
eliminated by subtraction as

$\displaystyle\Delta\mathcal{J}=\mathcal{J}_{M}-\mathcal{J}_{m},$ (3.8)

where $\mathcal{J}_{m}\equiv\mathcal{J}(A,\Sigma)$ is the $EI$ for the micro-
dynamics (Equation 2.1, and
$\mathcal{J}_{M}\equiv\mathcal{J}(A_{M},\Sigma_{M})$ is the $EI$ for the
macro-dynamics (Equation 2.8), and $\Delta\mathcal{J}$ is the degree of causal
emergence for stochastic iterating systems.

Through the calculations in the Appendix A, we obtain the following theorem.
In this theorem, we not only calculate the analytical solution for causal
emergence but also ensure that the degree of the analytical solution is only
related to the parameters of stochastic iterating systems without the
influence of $L$.

######  Theorem 3.1.

(Analytical solution for causal emergence): For the linear stochastic
iteration systems like Equation (2.1), causal emergence of the system after
coarse-graining $y_{t}=\phi(x_{t})=Wx_{t}$, $W\in\mathcal{R}^{k\times n}$, is
calculated as

$\displaystyle\Delta\mathcal{J}=\mathop{\ln\frac{|\det(WAW^{\dagger})|^{\frac{1}{k}}}{|\det(A)|^{\frac{1}{n}}}}_{DegeneracyEmergence}+\mathop{\ln\frac{|\det(\Sigma)|^{\frac{1}{2n}}}{|\det(W\Sigma
W^{T})|^{\frac{1}{2k}}}}_{DeterminismEmergence}$ (3.9)

$W\in\mathcal{R}^{k\times n}$, $x_{t}\in\mathcal{R}^{n}$,
$y_{t}\in\mathcal{R}^{k}$. Causal Emergence can be also divided into two
terms, and we name them as degeneracy emergence and determinism emergence,
separately. Both terms depend on the coarse-grained results of the overall
derivative and noise indicators, respectively.

The premise for the Theorem 3.1 to hold is that both macro and micro states
can be represented by $L$ as the range of values, and the premise is that the
two cannot differ by too much order of magnitude. When the intervention
interval size for each dimension of micro and macro variables is consistent
and equal to $L$, $L$ can be directly eliminated when calculating causal
emergence, so $\Delta\mathcal{J}$ is independent with $L$.

###  3.3 Some Bounds related with Causal Emergence

After inferring the calculation method of $\Delta\mathcal{J}$, we can
determine whether there is causal emergence in the macro states after coarse-
graining. Next, we will find an optimal matrix $W$ to maximize the degree of
causal emergence $\Delta\mathcal{J}$ [22, 43]. To solve this problem, we will
firstly discuss the upper bounds related with the two terms, the determinism
emergence and degeneracy emergence, in Equation 3.9, separately. Here we have
lemmas below.

The first lemma is to analyze degeneracy emergence by optimizing
$A_{M}=WAW^{\dagger}$.

######  Lemma 3.1.

For matrix $A$, its eigenvalues can be real numbers or complex numbers, while
the range of absolute values(modulus) for $|\det(WAW^{\dagger})|$ is bounded:

$\displaystyle
0\leq\left|\det(WAW^{\dagger})\right|\leq\prod_{i=1}^{k}|\lambda_{i}|,$ (3.10)

where, $A\in\mathcal{R}^{n\times n}$, $W\in\mathcal{R}^{k\times n}$,
$W^{\dagger}\in\mathcal{R}^{n\times k}$ is the Moore-Penrose generalized
inverse matrix of $W$, and $k$ is the rank of the matrix $W$.

  1. (1)

When the eigenvalues are all real numbers,
$|\lambda_{1}|\geq|\lambda_{2}|\geq\dots\geq|\lambda_{n}|\geq 0$ are $n$
absolute values of eigenvalues of matrix $A$ sorted from largest to smallest,
$\lambda_{1},\lambda_{2},\dots,\lambda_{n}\in R$.

  2. (2)

When the eigenvalues contain imaginary numbers, $|\cdot|$ represents the
modulus of complex numbers. Note that since the eigenvalues of imaginary
numbers appear together with their conjugate complex numbers, both eigenvalues
must have equal moduli and must be discarded or retained simultaneously.

  3. (3)

When $A$ is a symmetric matrix and $A>0$ is a positive definite matrix, since
$\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{n}\geq 0$, we can just write
the equation as

$\displaystyle\displaystyle\prod_{i=n-k+1}^{n}\lambda_{i}\leq\det(WAW^{\dagger})\leq\prod_{i=1}^{k}\lambda_{i}.$
(3.11)

The proof of the lemma can be referred to in Appendix B. In addition, we can
verify this lemma through numerical simulations. For example, for matrix $A$
with eigenvalues of
$\lambda_{1}=2.540,\lambda_{2}=1.380,\lambda_{3}=-0.4899,\lambda_{4}=0.1149$,
we randomly generate the elements of the matrix $W$. From Fig.1a We can see
that there is an upper bound $\lambda_{1}\lambda_{2}$ on the corresponding
$\det(A_{M})$ value of each $W$. If $A$ is also randomly generated, we can see
that all the scatter points are below the diagonal line $y=x$ in Fig.1b which
means Lemma 3.1 is satisfied.

After the lemma about $|\det(A_{M})|=|\det(WAW^{\dagger})|$, the next lemma is
about $\det(\Sigma_{M})=\det(W\Sigma W^{T})$, which is related with the
determinism emergence.

######  Lemma 3.2.

We define the singular values of $W$ are $s_{W,1}\geq s_{W,2}\geq\dots\geq
s_{W,k}$, then $\det(W\Sigma W^{T})$ satisfies

$\displaystyle\left(\prod_{i=1}^{k}s_{W,i}\kappa^{\frac{1}{2}}_{n-i+1}\right)^{\frac{1}{k}}\leq\det(W\Sigma
W^{T})^{\frac{1}{2k}}\leq\left(\prod_{i=1}^{k}s_{W,i}\kappa^{\frac{1}{2}}_{i}\right)^{\frac{1}{k}},$
(3.12)

$\kappa_{1}\geq\kappa_{2}\geq\dots\geq\kappa_{n}$ are the eigenvalues of
$\Sigma$. When $\Sigma=\sigma^{2}I$, the equal sign holds and

$\displaystyle\det(W\Sigma
W^{T})^{\frac{1}{2k}}=\sigma\left(\prod_{i=1}^{k}s_{W,i}\right)^{\frac{1}{k}}.$
(3.13)

In Fig.1c, we validate the inequalities using randomly generated $\Sigma$ and
$W$. Therefore, we can control the range of $\det(W\Sigma W^{T})$ by adjusting
the magnitude of the singular values of $W$. We further can adjust the
magnitude of the singular value of $W$ to weaken the noise and enhance the
causal emergence as we know in Lemma 3.2. As shown in Fig.1d, the smaller the
mean singular values of $W$ we generate, the greater the degree of causal
emergence $\Delta\mathcal{J}$.

###  3.4 Causal Emergence Maximization under Given Information Loss

In this subsection, we will officially discuss the problem of causal emergence
maximization. Before proceeding, we will introduce a condition on uncertainty
elimination to constrain $W$, such that zeros in denominator are excluded.

####  3.4.1 The Constraint on the uncertainty elimination by coarse-graining

According to Lemma 3.2, when the singular value of $W$, $s_{W,1}\to 0$,
$\det(W\Sigma W^{T})\to 0$. This means if $\phi(x_{t})=Wx_{t}$ is a completely
unconstrained map, the values of $x_{t}$ and the noise $\epsilon_{t}$ may be
scaled in an unbounded manner and the information entropy
$H(y_{t})\to-\infty$, leading to $y_{t}$ having an excessively narrow range.
To prevent the macro variables from being excessively compressed by the
coarse-graining map which may result in excessive information content, it is
essential to impose the following restriction,

$\displaystyle\frac{1}{n}H(p(x_{t}))-\frac{1}{k}H(p(y_{t}))\leq\eta,$ (3.14)

where, $n$ and $k$ are the dimensionalities of micro- and macro- variables,
and $\eta>0$ is a given constant, and $H(p(x_{t}))=(n/2)\ln(2\pi
e)+(1/2)\ln(\det(\Sigma))$ and $H(p(y_{t}))=(k/2)\ln(2\pi
e)+(1/2)\ln(\det(W\Sigma W^{T}))$. The inequality can be understood as the
dimension averaged uncertainty eliminated (or information gained) by the
coarse-graining strategy is under a given threshold $\eta$. According to the
inequality (3.14), we can further get that

$\displaystyle\det(W\Sigma
W^{T})^{\frac{1}{k}}\geq\epsilon\det(\Sigma)^{\frac{1}{n}},$ (3.15)

where, $W^{T}$ represents the transpose matrix of matrix $W$ and
$\epsilon=\exp(-2\eta)$. The covariance matrix affects the range of values of
random variables, the limitations here can also ensure the effectiveness of
the $do$ intervention scope $L$ mentioned in Section 3.1. It will be clear in
the subsequent sections that this constraint could also significantly
influence the scope of causal emergence.

With the help of Lemmas 3.1 and 3.2, we can obtain the optimal solution for
causal emergence $\Delta\mathcal{J}^{*}$.

(a)

(b)

(c)

(d)

Figure 1: Experimental results of the simulation about the lemma and the
theorem presented in Section 3. (a) For matrix $A$ with eigenvalues of
$\lambda_{1}=2.540,\lambda_{2}=1.380,\lambda_{3}=-0.4899,\lambda_{4}=0.1149$,
we randomly generate matrix $W$. We can see that there is an upper bound
$\lambda_{1}\lambda_{2}$ on the corresponding $\det(A_{M})$ value of each $W$.
(b) If $A$ is also randomly generated, we can see the scatter plot that
conforms to Lemma.3.1. (c) We validate the inequality using randomly generated
$\Sigma$ and $W$. We can control the range of $\det(W\Sigma W^{T})$ by
adjusting the magnitude of the singular value of $W$. (d) The smaller the mean
singular value of $W$ we generate, the greater the degree of causal emergence
$\Delta\mathcal{J}$.

######  Theorem 3.2.

For the linear stochastic iteration systems like Equation (2.1), after coarse-
graining $y_{t}=\phi(x_{t})=Wx_{t}$, $W\in\mathcal{R}^{k\times n}$, under the
constraint of Equation (3.15), the maximum of the degree of causal emergence
that the system can achieve is

$\displaystyle\Delta\mathcal{J}^{*}=\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\lambda_{i}|-\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\lambda_{i}|+\eta$
(3.16)

$|\lambda_{1}|\geq|\lambda_{2}|\geq\dots\geq|\lambda_{n}|\geq 0$ are $n$
modulus of eigenvalues of matrix $A$ sorted from the largest to the smallest,
$\lambda_{1},\lambda_{2},\dots,\lambda_{n}\in C$.

###### Proof.

We will discuss the maximization of causal emergence from determinism
emergence and degeneracy emergence, separately.

  1. (1)

Degeneracy emergence ($\Delta\mathcal{J}_{1}$)

Based on Theorem 3.1, we know that

$\displaystyle\begin{aligned}
\Delta\mathcal{J}_{1}&=\ln\frac{|\det(WAW^{\dagger})|^{\frac{1}{k}}}{|\det(A)|^{\frac{1}{n}}}\leq\ln\frac{\left(\displaystyle\prod_{i=1}^{k}|\lambda_{i}|\right)^{\frac{1}{k}}}{\left(\displaystyle\prod_{i=1}^{n}|\lambda_{i}|\right)^{\frac{1}{n}}}=\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\lambda_{i}|-\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\lambda_{i}|.\end{aligned}$
(3.17)

If and only if the maximum $k$ eigenvalues are preserved, the equality holds.
At this point,
$A_{M}=WAW^{\dagger}=(WV)\Lambda(WV)^{\dagger}=\tilde{W}\Lambda\tilde{W}^{\dagger}$,
$\Lambda={\rm diag}(\lambda_{1},\dots,\lambda_{n})\in\mathcal{R}^{n\times n}$,
it is satisfied that $\tilde{W}=(\tilde{W}_{k},O_{k\times{(n-k)}})$,
$\tilde{W}_{k}\in\mathcal{R}^{k\times k}$ can be any invertible matrix.
$V=(v_{1},\dots,v_{n})$, $v_{i}$ is the eigenvector corresponding to the
eigenvalues $\lambda_{i}$ of matrix $A$, $i=1,\dots,n$,
$\lambda_{1}\geq\dots\geq\lambda_{n}$. Since $\tilde{W}=WV$, We can obtain the
expression for $W$ as

$\displaystyle W=(\tilde{W}_{k},O_{k\times{(n-k)}})V^{-1}$ (3.18)

  2. (2)

Determinism emergence ($\Delta\mathcal{J}_{2}$)

Based on Equation (3.9), and the inequalities (3.12), (3.14) and (3.15), we
know that

$\displaystyle\begin{aligned}
\Delta\mathcal{J}_{2}&=\ln\frac{|\det(\Sigma)|^{\frac{1}{2n}}}{|\det(W\Sigma
W^{T})|^{\frac{1}{2k}}}=\frac{1}{2}\ln\frac{|\det(\Sigma)|^{\frac{1}{n}}}{|\det(W\Sigma
W^{T})|^{\frac{1}{k}}}\\\
&\leq\frac{1}{2}\ln\frac{|\det(\Sigma)|^{\frac{1}{n}}}{|\det(\Sigma)|^{\frac{1}{n}}\epsilon}=\frac{1}{2}\ln\frac{1}{\epsilon}=\eta\end{aligned}$
(3.19)

when the information loss reaches the upper bound, the determinism emergence
reaches the maximum. Determinism is enhanced while ensuring that randomness is
not excessively reduced.

By taking the maximum value of $\Delta\mathcal{J}_{1}$ and
$\Delta\mathcal{J}_{2}$, we can obtain

$\displaystyle\begin{aligned}
\Delta\mathcal{J}^{*}&=\Delta\mathcal{J}_{1}^{*}+\Delta\mathcal{J}_{2}^{*}=\ln\frac{\left(\displaystyle\prod_{i=1}^{k}|\lambda_{i}|\right)^{\frac{1}{k}}}{\left(\displaystyle\prod_{i=1}^{n}|\lambda_{i}|\right)^{\frac{1}{n}}}+\frac{1}{2}\ln\frac{1}{\epsilon}=\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\lambda_{i}|-\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\lambda_{i}|+\eta\end{aligned}$
(3.20)

∎

Suppose the solution set of the optimal coarse-grained matrix $W^{*}$ which
satisfying $\Delta\mathcal{J}(W^{*})=\Delta\mathcal{J}^{*}$, is
$\mathcal{W}^{*}$. Although it is difficult for us to find all the elements in
set $\mathcal{W}^{*}$, we will give methods for calculating special solutions
for different types of matrices in later sections.

####  3.4.2 Optimal solution set $\mathcal{W}^{*}$

After optimizing determinism emergence and degeneracy emergence, we can find
two solution sets corresponding to the optimal solutions. Thus, the
intersection of the two solution sets is the solution set of $W$ corresponding
to the maximum degree of causal emergence.

######  Corollary 3.1.

When $\varepsilon_{t}\sim N_{\mathcal{N}}(0,\Sigma)$ and the degree of causal
emergence reaches its maximum value, $W$ needs to satisfy

$\displaystyle\begin{cases}WV=(\tilde{W}_{k},O_{k\times{(n-k)}}),\\\
\det{(W\Sigma
W^{T})}^{\frac{1}{k}}=\epsilon\det{(\Sigma)}^{\frac{1}{n}},\end{cases}$ (3.21)

in which $V=(v_{1},\dots,v_{n})$, $v_{i}$ is the eigenvector corresponding to
the eigenvalues $\lambda_{i}$ of matrix $A$, $i=1,\dots,n$,
$\lambda_{1}\geq\dots\geq\lambda_{n}$. $\tilde{W}_{k}\in\mathcal{R}^{k\times
k}$ can be any invertible matrix. The solution set of the optimal coarse-
grained matrix $W^{*}$ satisfying
$\Delta\mathcal{J}(W^{*})=\Delta\mathcal{J}^{*}$ as $\epsilon=\exp(-2\eta)$ is

$\displaystyle\mathcal{W}^{*}=\left\\{W\bigg{|}W=(\tilde{W}_{k},O_{k\times{(n-k)}})V^{-1},\det(W\Sigma
W^{T})=\epsilon\det(\Sigma)^{\frac{k}{n}}\right\\}.$ (3.22)

We can find the proof process in the previous Subsection 3.4.

####  3.4.3 A case study of $\mathcal{W}^{*}$

To understand the physical meaning of the solution set, we can use a simple
example in the three-dimensional space to visualize our optimal solution set.
In specific cases, when $k=2,n=3$, the projection of $W$ in three-dimensional
space is a circle. In this case, matrix
$W=(w_{1}^{T},w_{2}^{T})^{T}\in\mathcal{R}^{2\times 3}$ can be split into two
row-vectors, $w_{i}=(w_{i1},w_{i2},w_{i3}),i=1,2$, in $\mathcal{R}^{3}$, The
eigenvector matrix $V=(v_{1},v_{2},v_{3})\in\mathcal{R}^{3\times 3}$ can be
regarded as the combining of three vectors,
$v_{j}=(v_{1j},v_{2j},v_{3j})^{T}\in\mathcal{R}^{3},j=1,2,3$. To visualize the
solution set of $W$, we introduce the following case as
$\epsilon=\exp(-2\eta)$.

######  Example 3.1.

In order to present the results more intuitively, we specify
$w_{1}w_{2}^{T}=0$, $\Sigma=\sigma^{2}I_{3}$. According to Equation (3.22), we
can get

$\displaystyle\begin{cases}w_{1}v_{3}=0,\\\ w_{2}v_{3}=0,\\\
(w_{1}w_{1}^{T})(w_{2}w_{2}^{T})=\epsilon,\end{cases}$ (3.23)

Although $W\in\mathcal{R}^{2\times 3}$ is in a six-dimensional space, we can
get the solution set of $w_{1}$ for specified $w_{2}$

$\displaystyle\begin{cases}w_{11}v_{13}+w_{12}v_{23}+w_{13}v_{33}=0,\\\
w_{11}^{2}+w_{12}^{2}+w_{13}^{2}=R^{2},\end{cases}$ (3.24)

in which $R^{2}=\epsilon/(w_{2}w_{2}^{T})$, $R$ is the radius of the sphere.
Similarly, we can also obtain the range of values for $w_{2}$.

###### Proof.

To meet the conditions
$WV=(Wv_{1},Wv_{2},Wv_{3})=(\tilde{W}_{k},O_{k\times{(n-k)}})$, we only need
to specify $w_{1}v_{3}=w2v_{3}=0$. This means that $w_{1}$ and $w_{2}$ are
contained in the plane perpendicular to $v_{3}$ and passing through the
origin, which is $w_{11}v_{13}+w_{12}v_{23}+w_{13}v_{33}=0$. When $w_{1}$ and
$w_{2}$ are in this plane, the degeneracy emergence $\Delta\mathcal{J}_{1}$
reaches its minimum value.

For determinism emergence $\Delta\mathcal{J}_{2}$,

$\displaystyle\begin{aligned} \det{(W\Sigma W^{T})}&=(w_{1}\Sigma
w_{1}^{T})(w_{2}\Sigma w_{2}^{T})-(w_{1}\Sigma w_{2}^{T})(w_{2}\Sigma
w_{1}^{T})\\\
{}_{(\Sigma=\sigma^{2}I_{3})}&=\sigma^{4}\left[(w_{1}w_{1}^{T})(w_{2}w_{2}^{T})-(w_{1}w_{2}^{T})(w_{2}w_{1}^{T})\right]\\\
{}_{w_{1}w_{2}^{T}=0}&=\sigma^{4}\left[(w_{1}w_{1}^{T})(w_{2}w_{2}^{T})\right]=\epsilon\det{(\Sigma)}^{\frac{k}{n}}\\\
&=\epsilon\sigma^{4},\end{aligned}$ (3.25)

When $w_{2}$ is fixed, the solution set of $w_{1}$ is a sphere in three-
dimensional space
$w_{11}^{2}+w_{12}^{2}+w_{13}^{2}=\mathcal{R}^{2},\mathcal{R}^{2}=\epsilon/(w_{2}w_{2}^{T})$.
If we fix $w_{1}$, the result is the same. ∎

From this example, we can derive the following proposition

######  Proposition 3.1.

In three-dimensional space $\mathcal{R}^{3}$, if the noise is a white noise
sequence $\Sigma=\sigma^{2}I_{3}$ and $w_{1}$ and $w_{2}$ are perpendicular to
each other, when causal emergence
$\Delta\mathcal{J}(W^{*})=\Delta\mathcal{J}^{*}$, the solution set of $w_{i}$
for $i=1,2$ is the intersection of a plane and a sphere in the space, which is
a circle.

In Fig.2, we visualize the solution set of $w_{1}=(w_{11},w_{12},w_{13})$. At
the same time, we can extend the scope of application of Example 3.1 for
general form of covariance matrix $\Sigma$, the solution set becomes an
ellipsoid. If $w_{1}w_{2}^{T}\neq 0$, then the spherical or ellipsoidal
surface undergoes translation. Therefore, the final solution sets for $w_{1}$
and $w_{2}$ are the intersection lines of ellipsoids or spheres with planes.
Two spatial curves form the solution set of $W$. So we can get the proposition
below.

######  Proposition 3.2.

In three-dimensional space $\mathcal{R}^{3}$, $\Sigma>0$ the coarse-graining
parameter matrix $W$ can be regarded as composed of $k$ row vectors $w_{i}$,
$k=1,2,3$, $i=1,\dots,k$, the solution space of each vector $w_{i}$ is the
intersection of a plane and an ellipsoid, which is an ellipse.

With a visualized solution set in three-dimensional space, we can infer the
optimal solution of $W$ in high-dimensional space.

######  Proposition 3.3.

For any $n>1$, a coarse-graining matrix $W\in\mathcal{R}^{n\times k}$ can be
split into $k$ row vectors $w_{i}\in\mathcal{R}^{n}$ in the same way, and the
solution set of $\mathcal{W}^{*}$ can be understood as the intersection of
hyperplanes and hyperellipsoids in an $\mathcal{R}^{n}$.

So after obtaining the relevant parameters of dynamics, we first derive the
expression for a high-dimensional ellipse, where any point on the high-
dimensional ellipse can maximize the causal emergence. At the same time, when
the model is more complex, we can also choose to find points close to the
ellipse, even if it is not the optimal solution, which can increase the
likelihood of causal emergence.

Figure 2: The visualization of the set for the optimal solutions of the coase-
graining strategy $W$. When $w_{1}w_{2}^{T}=0$, $\Sigma=\sigma^{2}I_{3}$.
Although $W\in\mathcal{R}^{2\times 3}$ is in a six-dimensional space, we can
draw the range of values for $w_{1}$ while limiting $w_{2}$,
$w_{11}v_{13}+w_{12}v_{23}+w_{13}v_{33}=0$ and
$w_{11}^{2}+w_{12}^{2}+w_{13}^{2}=R^{2},$ in which
$R^{2}=\epsilon/(w_{2}w_{2}^{T})$. When causal emergence
$\Delta\mathcal{J}(W^{*})=\Delta\mathcal{J}^{*}$, the solution set of $w_{i}$
is the intersection of a plane (blue) and a sphere (red) in 3D space
$\mathcal{R}^{3}$, which is a circle

###  3.5 The conditions for causal emergence

After knowing the solution method for the maximum causal emergence of the
system, we need to analyze the conditions under which causal emergence occurs
as $\Delta\mathcal{J}>0$. Two scenarios are being examined: the presence of
causal emergence in typical situations and its manifestation under optimized
conditions.

First, in general cases, causal emergence $\Delta\mathcal{J}$ is determined by
two factors: degeneracy $\Delta\mathcal{J}_{1}$ and determinism
$\Delta\mathcal{J}_{2}$. The sufficient condition for $\Delta\mathcal{J}>0$ is
that $\Delta\mathcal{J}_{1},\Delta\mathcal{J}_{2}>0$ at the same time.

The prerequisite for $\Delta\mathcal{J}_{1}>0$ is that there is a significant
difference between the eigenvalues $\lambda_{1}\geq\dots\geq\lambda_{n}$ of
parameter matrix $A\in\mathcal{R}^{n\times n}$. Assuming the macro-state
$y_{t}\in\mathcal{R}^{k}$, the greater the difference between
$\lambda_{1},\dots,\lambda_{k}$ and $\lambda_{k+1},\dots,\lambda_{n}$, the
more obvious the causal emergence. The best condition is that
$\lambda_{k}\gg\lambda_{k+1}$, if the latter $n-k$ term is discarded, there
will be a significant causal emergence.

The condition for $\Delta\mathcal{J}_{2}>0$ is that there is significant
randomness in the system as $\Sigma$ is a positive definite matrix and
$H(p(x_{t}))>-\infty$. If $\Sigma=0$ and $H(p(x_{t}))=-\infty$ as there is no
random noise in the system, effective information is also invalid. In theory,
the larger the system noise, the larger the space that can be compressed
during coarse-graining, and the stronger the causal emergence can appear.

Second, when causal emergence reaches its optimal solution, we can also
determine the condition for $\Delta\mathcal{J}>0$ based on the result of
$\Delta\mathcal{J}^{*}$. From Equation (3.20), we can know that when

$\displaystyle\begin{aligned}
\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\lambda_{i}|-\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\lambda_{i}|<\eta\end{aligned}$
(3.26)

as the difference of the average singular value of $A$ to the average singular
value of $A_{M}$ after taking the logarithm can be smaller than the bound of
the information entropy $\eta$ we specify, $\Delta\mathcal{J}^{*}>0$. If this
result holds, we can assert that there must exist a $W$ to make
$\Delta\mathcal{J}>0$.

##  4 Results

After knowing how to optimize causal emergence, we can use the theorems we
derived to analyze several cases of linear stochastic iteration systems. We
will attempt to search for the maximum of causal emergence in systems with
known dynamics. We provide three cases, focusing on the applications of
determinism emergence, degeneracy emergence, and the manifestation of causal
emergence in $\mathcal{R}^{3}$ space.

###  4.1 Random walk

Our first case is random walk [53], and our analysis focuses on the noise
$\varepsilon_{t}$ and the covariance matrix $\Sigma$, which mainly impact
determinism emergence. The random walk model is a mathematical model used to
describe the random movement of objects in a specific space, in which, a
walker moves between a series of locations, with the direction and distance of
each movement being random. This model can study various phenomena, such as
changes in asset prices in financial markets, the diffusion of particles in
fluids, etc. In the random walk model, the walker can only move a certain
distance to the left or right each time, and the distance obeys a normal
distribution. When multiple wanderers coexist, a changing parameter matrix can
be formed as an identity matrix, $A=I_{n}$,

$\displaystyle
x_{t+1}=x_{t}+\varepsilon_{t},\varepsilon_{t}\sim\mathcal{N}(0,\Sigma),$ (4.1)

which forms a linear stochastic series mainly affected by noise sequences
$\varepsilon_{t}$.

We set an example as $n=4$, $k=1$ while

$\displaystyle\begin{gathered}\Sigma=\begin{pmatrix}0.4782&-0.1967&-0.0287&0.0419\\\
-0.1967&0.6711&0.0233&-0.1067\\\ -0.0287&0.0233&0.3154&0.0738\\\
0.0419&-0.1067&0.0738&0.4211\end{pmatrix}.\end{gathered}$ (4.3)

The motion trend of each dimension of $x_{t}$ is shown in Fig.3a. From
Equation (3.15) we already know that the ratio between $\det{(W\Sigma
W^{T})}^{1/k}$ and $\det{(\Sigma)}^{1/n}$ should be greater than the lower
bound $\epsilon$. The constant for uncertainty loss is $\eta=0.3466$ and the
corresponding $\epsilon=0.5$. This value selected guarantee that when the
singular values of $W$ are all $1$, the determinism emergence
$\Delta\mathcal{J}_{2}=0.2439>0$ as $W=(-0.0819,0.1432,-0.8421,0.5135)$
satisfies $\det(W\Sigma
W^{T})^{\frac{1}{k}}/\det(\Sigma)^{\frac{1}{n}}=0.614>\epsilon$.

We can plot probability density graphs for the four dimensions of micro-state
noise $\varepsilon_{t}$ and macro-state noise $\varepsilon_{M,t}$ in Fig.3b,
respectively. We can find that the variance of macro-state is smaller than
that of micro-state, so the determinism emergence of macro-state dynamical
systems is larger. Meanwhile, in Fig.3c, we can see that under the condition
that the singular values of $W$ are all 1, the higher $k$, the stronger the
uncertainty of the system, and the smaller the degree of causal emergence.

Reducing the average noise of the system and increasing its determinism is an
important condition for the occurrence of causal emergence. The main reason is
that we can reduce the dimensions with weaker determinism. The premise for
this phenomenon is that there are significant differences in noise between
different dimensions. As shown in Fig.3d, we randomly generate $\Sigma$ and
$W$ and calculate the value of $\Delta\mathcal{J}$. We found that the larger
the variance of the eigenvalues $\kappa_{i}$ of $\Sigma$, $i=1,\dots,n$, the
more likely the occurrence of causal emergence.

(a)

(b)

(c)

(d)

Figure 3: Experimental results for random walks model. (a) The trajectories
$x_{t}$ of a random walker in different dimensions. (b) The probability
density functions for the four dimensions of micro-state noises
$\varepsilon_{t}$ and the macro-state noise $\varepsilon_{M,t}$. (c) The
degree of causal emergence under different macro-state dimension $k$. Under
the condition that the singular values of $W$ are all 1, the higher $k$, the
stronger the uncertainty of the system, and the smaller the degree of causal
emergence. (d) $\Delta\mathcal{J}$ and $std(\kappa)$. The larger the variance
of the eigenvalues $\kappa_{i}$ of $\Sigma$, $i=1,\dots,n$, the more likely
the occurrences of causal emergence with higher degrees.

###  4.2 Heat dissipation

The first case focuses on determinism and noise, while the second case will
focus on degeneracy and parameter matrix $A$. This case is called the
discretized [54] heat conduction model [55, 56]. In this discretized case, the
conduction of heat is mainly reflected in the temperature changes over time
[57] at the corresponding observation nodes [58]. We can use matrix $A$ to
represent the changes in temperature, which includes information on the rate
of change. This matrix usually corresponds to a positive definite sparse
matrix, which can be used to describe a system’s temperature change process.
For example

$\displaystyle
x_{t+1}=Ax_{t}+\varepsilon_{t},\varepsilon_{t}\sim\mathcal{N}(0,\sigma^{2}I_{n}),\sigma=0.01,$
(4.4)

describes the variation of temperature $x_{t}$ over time $t$. When $n=4$,
$x_{t}=(x_{1,t},x_{2,t},x_{3,t},x_{4,t})^{T}$, $x_{1,t}$, $x_{2,t}$,
$x_{3,t}$, and $x_{4,t}$ represent the temperatures at four nodes at time $t$
as shown in Fig.4b. The parameter matrix is,

$\displaystyle\begin{gathered}A=\begin{pmatrix}0.6&0.2&0&0\\\ 0.2&0.7&0.1&0\\\
0&0.1&0.4&0.1\\\ 0&0&0.1&0.3\end{pmatrix},\end{gathered}$ (4.6)

where diagonal elements represent the rates of heat conservation at each node,
and the off-diagonals represent the energy transfer between different nodes as
shown in Fig.4a. Assuming $\epsilon=1$ as $\eta=0$, then the uncertainties in
both the dynamics at micro- and macro- levels are the same according to
Equation (3.14). As a result, only the effect of degeneracy emergence can
influence the causal emergence according to Equation (3.20).

We can simulate the diffusion process by setting the initial state to
$x_{0}=(10,10,10,10)^{T}$ and we iterate the equation
$x_{t+1}=Ax_{t}+\varepsilon_{t}$ for $t=1,2,\dots$. The results are shown in
Fig4b. We can directly coarse-grain the micro-state data to obtain
$y_{t}=Wx_{t}$ for $t=0,1,2,\dots$. At the same time, we can use the macro-
state dynamics to iterate the initial macro-state $y_{0}$. In order to
distinguish from the previous $y_{t}$ data, we use $\hat{y}_{t}$ to represent
the data generated by macro-state dynamical iterations here.
$\hat{y}_{0}=y_{0}$ and
$\hat{y}_{t+1}=WAW^{\dagger}\hat{y}_{t}+W\varepsilon_{t}$ for $t=0,1,2,\dots$.
By comparison in Fig.4c, we can find that the dynamics of $y_{t}$ and
$\hat{y}_{t}$ are close to each other, macro-state dynamics can represent the
dynamic evolution law of data after the coarse-graining.

By reducing the low correlation and dimensions that are difficult to reverse
between $x_{t}$ and $x_{t+1}$, degeneracy can be increased, resulting in
stronger causal emergence. For example, in the case shown in Fig.4d, when
$k=1$, $\sigma=0.01$ and $\eta=0$ as Shannon entropy of $x_{t}$ and $y_{t}$
remains stable, causal emergence occurs with the degree
$\Delta\mathcal{J}=0.6656$ and it is greater than the case when $k>1$. We can
take any element from $\mathcal{W}^{*}$, for example,
$W=(0.5856,0.7910,0.1748,0.03065)\in\mathcal{R}^{1\times k}$ which satisfies
our condition of $W$ in Equation (3.21). The macro-state dynamic parameter
matrix is $A_{M}\in\mathcal{R}^{1\times 1}$ with an eigenvalue
$\lambda_{M,1}=0.8702$. The eigenvalues of the micro-state matrix $A$ are
$\lambda_{1}=0.8702,\lambda_{2}=0.5000,\lambda_{3}=0.4000$, and
$\lambda_{4}=0.2298$. It can be seen that retaining the maximum eigenvalue
will result in significant causal emergence. In physical terms, it can be
understood as a region with slower dissipation, and the causal relationship
between the temperature at time $t+1$ and time $t$ is stronger, making it more
suitable for analyzing the temperature evolution law of the entire system.

To verify our theoretic result about the exact expression for causal
emergence, we can compare the obtained analytical solution in Equation (3.16)
with the numerical results which can be obtained by the following steps. We
first randomly generate numeric samples of $x_{t}\sim U([-1,1]^{n})$ and
$y_{t}\sim U([-1,1]^{k})$, then perform a one-step iteration for micro and
macro-dynamics as $x_{t+1}=Ax_{t}+\varepsilon_{t}$ and
$y_{t+1}=A_{M}y_{t}+\varepsilon_{M,t}$,
$\varepsilon_{t}\sim\mathcal{N}_{n}(0,\sigma^{2}I_{n})$ and
$\varepsilon_{M,t}\sim\mathcal{N}_{k}(0,\sigma^{2}I_{k})$. We then directly
calculate the mutual information $I(x_{t+1},x_{t}|do(x_{t})\sim
U([-1,1]^{n}))$ for micro-dynamics and $I(y_{t+1},y_{t}|do(y_{t})\sim
U([-1,1]^{k}))$ between the generated samples of $x_{t+1}$ at time $t+1$ and
the samples of $x_{t}$ at time $t$ to obtain the numerical solution of
effective information and calculate causal emergence as

$\displaystyle\Delta\mathcal{I}=\frac{I(y_{t+1},y_{t}|do(y_{t})\sim
U([-1,1]^{k}))}{k}-\frac{I(x_{t+1},x_{t}|do(x_{t})\sim U([-1,1]^{n}))}{n}.$
(4.7)

To distinguish from analytical solutions $\Delta\mathcal{J}$, we use
$\Delta\mathcal{I}$ to represent the causal emergence computed by the
mentioned numeric method. From Fig.4e we found that when $n=4,k=1$ and
$\sigma^{2}=0.01$, as the sample size increases, the numerical solutions for
causal emergence gradually approach the analytical solutions.

(a)

(b)

(b)

(d)

(e)

Figure 4: Experimental results for heat dissipation model. (a) The
illustration of heat transfer and dissipation for a one-dimensional chain with
four vertices. (b) The micro-state data of temperature at different times $t$
for 4 nodes. (c) The comparison between the macro-state data directly obtained
by coarsening the micro-state data, $y_{t}$ to obtain $y_{t}=Wx_{t}$ for
$t=0,1,2,\dots$, and $\hat{y}_{t}$ which is generated by the macro-state
dynamics to iterate the initial macro-state $y_{0}$. (d) The comparisons for
the degrees of causal emergence under different dimensions of macro-states
$k$. when $k=1$ and $\epsilon=1$, maximized degree of causal emergence occurs
as $\Delta\mathcal{J}=0.6656$. (e) The comparison between $\Delta\mathcal{I}$
and $\Delta\mathcal{J}$. As the sample size increases, the numerical solutions
for causal emergence $\Delta\mathcal{I}$ gradually approach the analytical
solutions $\Delta\mathcal{J}$.

###  4.3 Spiral rotating

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5: Experimental results of spiral rotating. (a) When $\Psi={\rm
diag}(0.94,0.94,0.99)$ and $x_{0}=(1,1,3)^{T}$, $x_{t}$ represents a point in
space that rotates around the axis of rotation and contracts towards the axis
of rotation. By the optimal coarse-graining $W^{*}$, we can obtain macro-
states that move along the axis of rotation. (b) The degree of causal
emergence and its dependence on the dimension of macro-states for the example
in (a), which reaches its maximum value as $\Delta\mathcal{J}=0.0341$ when
$k=1$. (c) The spiral curve when $\Psi={\rm diag}(0.99,0.97,0.2)$ and
$x_{0}=(1,1,1)^{T}$, the trajectory of $x_{t}$ is compressed to a plane
perpendicular to $u$ at the initial stage. (d) We can project $x_{t}$ on the
plane as a macro-state, where $k=2$, $\Delta\mathcal{J}=0.5295$. (e) When
$\pi/2<\theta<3\pi/2$, $x_{t}$ tends to oscillate more than rotate, while
$\theta<\pi/2$ or $\theta>3\pi/2$ $x_{t}$ tends to be a rotate model. (f) When
$\theta$ approaches $\pi$, the degree of causal emergence is smaller, while
when it approaches $0$ or $2\pi$, the degree of causal emergence is larger.

In addition to the results obtained for specific models in sections 4.1 and
4.2, we need a more intuitive understanding of the meaning of coarse-graining
and causal emergence. Here, we take a spiral rotating model [59] from
analytical geometry [60, 61] as an example to visualize our model system in
$\mathcal{R}^{3}$ space and analyze how causal emergence is reflected. A
common example is a model in which a rigid body rotates about an axis. Suppose
we have a vector in three-dimensional space that represents the position of a
point $x_{0}\in\mathcal{R}^{3}$, and we want to rotate it about some straight
line through the origin. This operation can be represented by the rotation
matrix $R$. For rotation around the straight line represented by the unit
vector $u=(a,b,c)$, $||u||=\sqrt{a^{2}+b^{2}+c^{2}}=1$, a rotation matrix can
be used to describe the rotation operation:

$\displaystyle\begin{gathered}R=\begin{pmatrix}{\rm cos}\theta+a^{2}(1-{\rm
cos}\theta)&ab(1-{\rm cos}\theta)-c{\rm sin}\theta&ac(1-{\rm cos}\theta)+b{\rm
sin}\theta\\\ ab(1-{\rm cos}\theta)+c{\rm sin}\theta&{\rm
cos}\theta+b^{2}(1-{\rm cos}\theta)&bc(1-{\rm cos}\theta)-a{\rm sin}\theta\\\
ac(1-{\rm cos}\theta)-b{\rm sin}\theta&bc(1-{\rm cos}\theta)+a{\rm
sin}\theta&{\rm cos}\theta+c^{2}(1-{\rm
cos}\theta)\end{pmatrix}.\end{gathered}$ (4.9)

Where $\theta$ represents the angle of the rotation. This matrix describes a
rotation about the straight line represented by the unit vector $u$. Multiply
the vector $x_{t}$ by the matrix $R$ to get the new rotated vector $x_{t+1}$.
This rotation matrix describes a model of rotation around a certain straight
line and can be used to describe rotation operations around a specified axis
in three-dimensional space. Since $R$ doesn’t change the modulus of $x_{t}$,
we can add an adjustment matrix $\Psi={\rm diag}(\psi_{1},\psi_{2},\psi_{3})$
to adjust the modulus of the variable $x_{t}$. Then the linear stochastic
iteration systems in $\mathcal{R}^{3}$ is

$\displaystyle
x_{t+1}=Ax_{t}+\varepsilon_{t},A=R\Psi,\varepsilon_{t}\sim\mathcal{N}(0,\sigma^{2}I_{n}),\sigma=0.01,$
(4.10)

when $\psi_{i}<1,i=1,2,3$, the model is attenuated. Adjusting $\psi_{i}$ can
cause the system to produce different values of causal emergence at different
macro-state dimensions.

Assuming the direction vector of the rotation axis is $u_{0}=(0,0.1,1)^{T}$,
after normalization as $u=u_{0}/||u_{0}||$, the rotation axis $u$ can be
obtained. At the same time, we specify the rotation angle $\theta=\pi/16$ and
$\eta=0$ as the dimension averaged Shannon entropy of both $x_{t}$ and $y_{t}$
remains the same. Through $\theta$ and $u$, we can get the rotation matrix
$R$. By changing $\Psi$, we will obtain causal emergence in different forms.

The occurrence of causal emergence in three-dimensional space can be
understood as the stronger causal effect of system evolution when the path of
$x_{t}$ is projected onto a plane or a straight line. As shown in Fig.5a, when
$\Psi={\rm diag}(0.94,0.94,0.99)$ and $x_{0}=(1,1,3)^{T}$, $x_{t}$ contracts
towards the axis of rotation. In Fig.5b, the causal emergence reaches its
maximum value as $\Delta\mathcal{J}=0.0341$ when $k=1$. The eigenvalues of the
micro-state matrix $A$ are $\lambda_{1}=0.9895$,$\lambda_{2}=0.9222+0.1834i$
and $\lambda_{3}=0.9222-0.1834i$, the macro-state parameter matrix $A_{M}$
only retains the eigenvalues with the maximum modulus $\lambda_{1}=0.9895$.
This result is obtained by setting the coarse-graining mapping $W$ to compress
the rotation trajectory into a one-dimensional straight line as the macro-
states.

In the second scenario in Fig.5c, when $\Psi={\rm diag}(0.99,0.97,0.2)$ and
$x_{0}=(1,1,1)^{T}$, the path of $x_{t}$ will first shrink to a plane
perpendicular to $u$. In 5d, we project $x_{t}$ onto the plane as the macro-
state, where $k=2$, $\Delta\mathcal{J}=0.5295$. We retain the two eigenvalues
$\lambda_{1}=0.9612+0.1900i$ and $\lambda_{2}=0.9612-0.1900i$ with the largest
and identical module lengths. The reason why we do not take $k=1$ here is that
the more dimensionality we reduce, the greater the information loss and
$|\lambda_{1}|=|\lambda_{2}|$. Therefore, in the absence of further
improvement in causal emergence, we do not need to continue the dimensionality
reduction of the system.

Besides $\Psi$, rotation angle $\theta$ also significantly impacts causal
emergence. In Fig.5f, when $\theta$ approaches $\pi$, the degree of causal
emergence is smaller, while when it approaches $0$ or $2\pi$, it is larger.
This is because when $\pi/2<\theta<3\pi/2$, $x_{t}$ tends to oscillate more
than rotate, as shown in the Fig.5e. The correlation between $x_{t}+1$ and
$x_{t}$ in the model is relatively small, leading to a weakening of causal
emergence.

##  5 Discussion

###  5.1 Optimal causal emergence when $W^{\dagger}=W^{T}$

(a)

(b)

Figure 6: (a) We randomly generate matrices $A$, $\Sigma$ and $W$ with a
singular value of 1 when $n=4$, $k=2$, we can see the upper bound of
$|\det(WAW^{\dagger})|/\det(W\Sigma W^{T})^{1/2}$ is
$\tilde{d}_{1}\tilde{d}_{2}$. (b) When
$\lambda_{1}=0.8,\lambda_{1}=0.2,\lambda_{3}=0.4,\lambda_{4}=0.2$ and randomly
arrange $(0.2,0.4,0.6,0.8)$ to generate $(\kappa_{1},\dots,\kappa_{n})$.
$C(\Delta\mathcal{J}_{1},\Delta\mathcal{J}_{2})$ and $C(\kappa,\lambda)$ shows
a negative slope after drawing the scatter plot.

In previous sections, we have obtained the explicit expression for the maximum
causal emergence and the optimal coarse-graining under the condition that the
dimension averaged uncertainty loss by coarse-graining is bounded. However,
under this condition, the competition between the degeneracy term which is
solely determined by $A$ and the determinism term which is determined by $A$
and $\Sigma$ in the optimal causal emergence can not be observed as shown in
Equation (3.20).

Therefore, we will discuss the maximization of causal emergence under a
different condition, that is,

$W^{\dagger}=W^{T},$ (5.1)

in this section. Equation (5.1) means the coarse-graining map $W$ can be
decomposed as a projection by discarding the information in $n-k$ dimensions
and a rotation in a $k$-dimensional space without information loss. This also
means the $k$ first singular values are units, and the norm of $W$ and
$\Sigma$ will not be excessively enlarged or reduced. In this case, as
$W^{\dagger}=W^{T}$, the coarse-graining strategy will simultaneously filter
the eigenvalues of $A$ and $\Sigma$. According to Equation (3.6) and (3.9), we
can get that $|\det(A)|/\det(\Sigma)^{1/2}=|\det(A\Sigma^{-1/2})|$. So
$A\Sigma^{-1/2}$ determines the causal emergence of the system and is
necessary to be analyzed below.

######  Theorem 5.1.

When $W^{\dagger}=W^{T}$, the causal emergence of linear stochastic iterative
systems satisfies

$\displaystyle\Delta\mathcal{J}\leq\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\tilde{d}_{i}|-\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\tilde{d}_{i}|,$
(5.2)

where, $|\tilde{d}_{1}|\geq\dots\geq|\tilde{d}_{n}|$ are $n$ eigenvalues of
$A\Sigma^{-1/2}$. The equal sign holds when $A$ and $\Sigma$ share the same
$n$ eigenvectors as

$A\Sigma^{-1/2}=V\Lambda K^{-1/2}V^{T}=V\tilde{D}V^{T},$ (5.3)

where $\tilde{D}=\Lambda K^{-1/2}={\rm
diag}(\tilde{d}_{1},\dots,\tilde{d}_{n})$ is also a diagonal matrix, in which
$|\tilde{d}_{1}|\geq\dots\geq|\tilde{d}_{n}|$, and $V$ is the matrix formed by
the shared eigenvectors of $A$ and $\Sigma$.

When $A$ and $\Sigma$ do not share the eigenvector matrix, we can’t directly
calculate the correlation between the eigenvalue matrix of $A\Sigma^{-1/2}$
and $K$ or $\Lambda$ within the current framework of matrix theory, so it is
difficult to find an analytical solution for causal emergence. But we can
still obtain numerical solutions through optimization of $W$ and confirm that
the upper bound of causal emergence is to retain the maximum $k$ eigenvalues
of $A\Sigma^{-1/2}$. We have the inequality (5.2) in Theorem 5.1. To verify
this inequality we randomly generate matrices $A$, $\Sigma$ and $W$ with a
singular value of 1 when $n=4$, $k=2$, we can see the upper bound of
$|\det(WAW^{\dagger})|/\det(W\Sigma W^{T})^{1/2}$ is
$\tilde{d}_{1}\tilde{d}_{2}$ in Fig.6b. Unlimited the singular value of $W$
means that it can reduce the macro-state noise and information entropy in the
system. This also means that the locally effective information we retain can
be adjusted arbitrarily by adjusting $W$. This is also a balance point between
determinism and degeneracy.

Then, we can derive an analytical expression for the maximized causal
emergence when $A$ and $\Sigma$ share the same $n$ eigenvectors, that is to
say, $\Sigma=VKV^{T}$ and $A=V\Lambda V^{T}$, and Equation (5.3) holds.
Rearranging the order of $\tilde{d_{i}}$ can result in diagonal matrix
$\tilde{D}={\rm diag}(\tilde{d}_{1},\dots,\tilde{d}_{n})$ in which
$|\tilde{d}_{1}|\geq\dots\geq|\tilde{d}_{n}|$. Then, according to Equation
(3.9) we can obtain an analytical solution for the maximized causal emergence,
which is also the case where Equation (5.2) takes the equal sign. The above
proof process can refer to Appendix B.

Next, we will study when the relationship between the
determinism($\Delta\mathcal{J}_{2}$) and the
degeneracy($\Delta\mathcal{J}_{1}$) is cooperative such that the causal
emergence can be maximized under the condition of the simultaneous
diagonalization for $A$ and $\Sigma$. This problem is equivalent to ask when
the quantity expressed in Equation 5.2 which is determined by the eigenvalues
of $A\Sigma^{-1/2}$ will be maximized if the eigenvalues in $A$ and $\Sigma$
are unchanged. We found that the orders of the eigenvalues in both $A$ and
$\Sigma$, which can be characterized by the correlation between
$\kappa=(\kappa_{1},\dots,\kappa_{n})$ and
$\lambda=(\lambda_{1},\dots,\lambda_{n})$ denoted as $C(\kappa,\lambda)$, play
an important role. If $\lambda$ and $\kappa$ are positive correlated, the
effects of determinism and degeneracy contributed to the causal emergence
cancel each other. For example, when $\lambda=(0.8,0.6,0.4,0.2)$,
$\kappa=(0.8,0.6,0.4,0.2)$, and $k=2$, the optimal coarse-graining strategy
for $\Delta\mathcal{J}_{1}$ should retain the largest $k=2$ eigenvalues
$\lambda_{1}=0.8,\lambda_{2}=0.6$ of $A$ according to Equation (3.17), but for
$\kappa$, the largest $k$ eigenvalues $\kappa_{1}=0.8,\kappa_{2}=0.6$ are also
retained, which implies small determinism and large degeneracy. We can see
that the determinism and the degeneracy are mutually inhibiting leading to a
small causal emergence: $\Delta\mathcal{J}=0$.

On the other hand, if $\lambda$ and $\kappa$ are negative correlated then
causal emergence can be increased because determinism and degeneracy reinforce
each other. For example, when $\lambda=(0.8,0.6,0.4,0.2)$ and
$\kappa=(0.2,0.4,0.6,0.8)$, the optimal coarse-graining strategy for
$\Delta\mathcal{J}_{1}$ should retain the largest $k=2$ eigenvalues
$\lambda_{1}=0.8,\lambda_{2}=0.6$ of $A$, this also means to retain the
smallest $k$ eigenvalues of $\Sigma$, then determinism and degeneracy are
mutually reinforced, and a larger causal emergence can be obtained:
$\Delta\mathcal{J}=0.8959$.

In Fig.6a, we visualize how the correlation between $\lambda$ and $\kappa$ can
influence the cooperative relationship between determinism and degeneracy
which can be characterized by the correlation between $\Delta\mathcal{J}_{1}$
and $\Delta\mathcal{J}_{2}$ denoted as
$C(\Delta\mathcal{J}_{1},\Delta\mathcal{J}_{2})$ in a small example. First, we
fix the values and the orders of $\Lambda$ as
$\lambda_{1}=0.8,\lambda_{1}=0.2,\lambda_{3}=0.4,\lambda_{4}=0.2$. Second, we
randomly permute $(0.2,0.4,0.6,0.8)$ to generate $\kappa$. Finally, we
randomly sample the invertible matrix $V$ such that $A$ and $\Sigma$ can be
obtained. The results are shown in Fig.6a,
$C(\Delta\mathcal{J}_{1},\Delta\mathcal{J}_{2})$ and $C(\kappa,\lambda)$ shows
a negative slope after drawing the scatter plot, this means that the two
values are negatively correlated. In general, maximizing causal emergence
$\Delta\mathcal{J}$ is also a process of finding optimal orders to arrange
$\lambda_{i}$ and $\kappa_{i}$ for all $i$ to achieve a balance between
determinism $\Delta\mathcal{J}_{2}$ and degeneracy $\Delta\mathcal{J}_{1}$.

###  5.2 About the information loss in dynamics prediction

Our entire paper discusses the changes in causal emergence of systems after
coarse-graining. For linear stochastic iteration systems, deleting smaller
eigenvalues in the parameter matrix $A$ or larger eigenvalues in $\Sigma$ can
both increase the value of causal emergence. Coarse graining and data
dimensionality reduction [37, 38, 39, 41, 40] are intricately linked. Once
data dimensionality is reduced, it will be accompanied by information loss,
and the transformation from micro-states to macro-states is no exception.
Therefore, we also need to briefly analyze the relationship between the loss
function in dynamics prediction and the effective information of the dynamics.
For linear stochastic iteration systems, we define the dynamical loss in the
following way:

######  Definition 5.1.

(Dynamical loss): For the variable $x_{t}$ at time $t$, we define the
dynamical loss as

$\displaystyle L_{D}=||x_{t+1}-\tilde{x}_{t+1}||,$ (5.4)

in which $x_{t+1}=Ax_{t}+\varepsilon_{t}$,
$\tilde{x}_{t+1}=\phi^{\dagger}(y_{t+1})=W^{\dagger}y_{t+1}$.

In Definition 5.1, $y_{t+1}$ is derived from $y_{t}$ and the macro iterative
Equation (2.8) and $y_{t}=Wx_{t}$. Therefore,

$\displaystyle\tilde{x}_{t+1}=W^{\dagger}WAW^{\dagger}Wx_{t}+W^{\dagger}W\varepsilon_{t},$
(5.5)

the dynamical loss then should be:

$\displaystyle
L_{D}=||(W^{\dagger}WAW^{\dagger}W-A)x_{t}+(W^{\dagger}W-I)\varepsilon_{t}||,$
(5.6)

it is the difference between the micro-state variables
$x_{t}\in\mathcal{R}^{n}$ that are then mapped into micro-states after one
iteration in the macro dynamics, and the actual micro-state $x_{t+1}$, which
reflects the information loss caused by coarse-graining on dynamics.

Due to the dependence between $L_{D}$ and variables $x_{t}$ and
$\varepsilon_{t}$, in order to directly analyze the relationship between the
causal emergence of the system $\Delta\mathcal{J}$ and the dynamical loss, we
consider the upper bound of $L_{D}$ to eliminate the influence of random
variables.

######  Lemma 5.1.

(Supremum of dynamical loss): For the variable $x_{t}$ at time $t$, when
$||x_{t}||^{*}=\mathop{\sup}_{t}||x_{t}||<\infty$, we define the supremum of
dynamical loss $L_{D}$ as

$\displaystyle L_{D}\leq
S_{D}=\mathop{||A-\hat{A}||_{F}||x_{t}||^{*}}_{DegeneracySupremum}+\mathop{(n-k)||\varepsilon_{t}||}_{DeterminismSupremum},$
(5.7)

in which $\hat{A}=W^{\dagger}WAW^{\dagger}W$. $S_{D}$ can be decomposed into
determinism supremum and degeneracy supremum. $||\cdot||_{F}$ is the Frobenius
norm of matrix $\cdot$.

We can directly determine the relationship between the upper bound and
$\Delta\mathcal{J}_{1}$. When $k$ is a constant, the minimum value of $S_{D}$
is equivalent to the maximum value of degeneracy emergence,
$\Delta\mathcal{J}_{1}$.

######  Theorem 5.2.

($\Delta\mathcal{J}_{1}$ and $S_{D}$): For the known variables $x_{t}$, the
micro-state parameter matrix $A$ and the noise $\varepsilon_{t}$, $t=0,1,...$,

$\displaystyle\arg\max\limits_{W}\Delta\mathcal{J}_{1}=\arg\min\limits_{W}S_{D}=W^{*}.$
(5.8)

The proof of this theorem is referred to Appendix Appendix C. After obtaining
the relationship between the supremum $S_{D}$ and the degeneracy emergence
$\Delta\mathcal{J}_{1}$. It can be observed that after minimizing $S_{D}$, the
solution set of $W$, $\mathcal{W}_{L}$, satisfies
$\mathcal{W}_{L}\in\mathcal{W}^{*}$. It can be seen that reducing loss can
only increase the degeneracy of the system, but further search for $W$ is
needed for determinism. This explains why we still need to train for a few
steps to find the maximum causal emergence after the loss function converges
in machine learning for causal emergence identification [43, 35].

###  5.3 Nonlinear form

Although the effective information and causal emergence in this paper are
based on linear stochastic iterative systems, we can still apply them to
nonlinear models under certain conditions. Nonlinear stochastic iterative
systems like $x_{t+1}=f(x_{t})+\varepsilon_{t}$ do not have the same known
parameter matrix $A$ as linear stochastic iterative systems. However, when
$f:\mathcal{R}^{n}\to\mathcal{R}^{n}$, we can replace $A$ with the gradient
matrix of $\nabla f(x_{t})\in\mathcal{R}^{n\times n}$ at $x_{t}$

Therefore, near $x_{t}$, when the noise is small, we can obtain an expression
for the effective information of the nonlinear iterative system

$\displaystyle\mathcal{J}(x_{t})=\ln\displaystyle\frac{|\det(\nabla
f(x_{t}))|^{\frac{1}{n}}L}{(2\pi
e)^{\frac{1}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2n}}},$ (5.9)

In this way, we can get the expression of causal emergence as

$\displaystyle\Delta\mathcal{J}(x_{t})=\ln\frac{|\det(\nabla
f_{M}(y_{t})|^{\frac{1}{k}}}{|\det(\nabla
f(x_{t})|^{\frac{1}{n}}}+\ln\frac{|\det(\Sigma)|^{\frac{1}{2n}}}{|\det(\Sigma_{M})|^{\frac{1}{2k}}}.$
(5.10)

For some simple differentiable nonlinear functions $f(x_{t})$, we can use this
method to calculate the magnitude of causal emergence.

##  6 Conclusion

In this article, we first propose the coarse-graining strategy $y_{t}=Wx_{t}$
of linear stochastic iterative systems like
$x_{t+1}=Ax_{t}+\varepsilon_{t},\varepsilon_{t}\sim\mathcal{N}(0,\Sigma)$,
that map micro-state $x_{t}\in\mathcal{R}^{n}$ into macro-states
$y_{t}\in\mathcal{R}^{k},k<n$. Coarse-graining can naturally derive macro
dynamical models $y_{t+1}=A_{M}y_{t}+\varepsilon_{M,t}$ for
$y_{t}\in\mathcal{R}^{k}$, $A_{M}=WAW^{\dagger}$,
$\varepsilon_{M,t}\sim\mathcal{N}(0,\Sigma_{M})$, $\Sigma_{M}=W\Sigma W^{T}$.
For $x_{t+1}=Ax_{t}+\varepsilon_{t}$, we can use $\mathcal{J}$ and
$\Delta\mathcal{J}$ to represent the effective information of the system and
causal emergence in different dimensions $k$ and $n$. To ensure that the macro
dynamical information entropy $H(p(y_{t}))$ is not excessively reduced during
coarse-graining, we need to ensure $\frac{H(x_{t})}{n}-\frac{H(y_{t})}{k}$ is
bounded by $\eta$.

To directly determine the causal emergence in the system without being
affected by coarse-graining parameter matrix $W$, we optimize $W$ to obtain
the maximum causal emergence $\Delta\mathcal{J}^{*}$ that the system can
achieve. When obtaining the optimal solution, the iterative parameter matrix
$A_{M}$ only retains the maximum $k$ eigenvalues of $A$, and the determinant
value of $\Sigma_{M}$ is compressed to the lowest allowed lower limit
$\epsilon\det(\Sigma)^{\frac{k}{n}}$. The obtained solution set can be
understood as the intersection of $k$ hyperplanes and hyperellipsoids, an
ellipse in three-dimensional space. We search for causal emergence in three
systems that are known models, focusing on the application of determinism,
degeneracy and the manifestation of causal emergence in $\mathcal{R}^{3}$
space. In random walks, energy dissipation, and accurate selection models,
varying degrees of causal emergence can be detected.

We conclude that there are two influencing factors for causal emergence, one
is the iterative parameter matrix $A$, and the other is the covariance matrix
$\Sigma$ of random noise, which respectively affect the degeneracy and
determinism of the system. Smaller eigenvalues in the parameter matrix
indicate weak causal relationships between variables in certain dimensions.
Suppose the coarse-grained strategy precisely discards these dimensions. In
that case, the causal effect of the macro-state dynamics is stronger than that
of the micro-state, and the optimal causal emergence will occur.

The causal emergence calculation method in this article has several
advantages. Firstly, we present the causal emergence of a known model in a
linear stochastic iterative system as an analytical solution. This is also the
first time that causal emergence has been calculated as an analytical solution
in a continuous-state dynamic system. Secondly, in maximizing causal
emergence, we found the solution set of the coarse-graining strategy. We found
that the eigenvalues of the parameter matrix in a linear system determine the
value of causal emergence. The third is the final result we obtain, which is
only related to the system itself and not to specific data. This to some
extent solves the problem of numerical solutions relying on data and the need
for artificial search for coarse-graining strategies.

However, our model still has many shortcomings. One is that the current model
is only applicable to linear systems, and there is still a lack of unified
analysis and judgment for nonlinear systems. On the other hand, the time of
the linear stochastic iterative system we study is discrete, and the
calculation method of causal emergence is not applicable to continuous time
dynamic systems.

There are two important research directions in the future. One is to consider
how to correctly calculate causal emergence for continuous-time Markov
processes with Discrete probability space such as birth and death processes or
queuing processes. The other direction is about the causal emergence of
stochastic differential equations such as option pricing equation and Langevin
equation. We will continue to conduct in-depth research on these contents in
the future.

Data ailability

All the codes and data are available at:
https://github.com/kilovoltage/An_Exact_Causal_Emergence_Theory.

##  Appendix Appendix A Calculation effective information and causal emergence

In this appendix, we will prove the theorems related to effective information
and causal emergence, and demonstrate the specific process of formula
derivation. The first process to be demonstrated is the inference of effective
information related to linear stochastic iterative systems $\mathcal{J}$.

Proof of Theorem 3.1

###### Proof.

For the linear stochastic iteration system like Equation (2.1), $x_{t+1}$
follows a normal distribution about $x_{t}$, so
$x_{t+1}\sim\mathcal{N}(Ax_{t},\Sigma)$,

$\displaystyle
p(x_{t+1}|x_{t})=\displaystyle\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}}\exp\left\\{-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right\\}$
(A.1)

Following the calculation method of causal geometry, assuming
$\varepsilon_{t}$ is small, $do(x_{t}\sim U([-L/2,L/2]^{n}))\equiv
do(x_{t}\sim U)$, then $p(x_{t})=1/L^{n},L>0$, we can calculate the effect
distribution

$\displaystyle\begin{aligned}
E_{D}(x_{t+1})&=\displaystyle\int_{\mathcal{X}}p(x_{t+1}|x_{t})p(x_{t})d^{n}x_{t}\\\
&=\displaystyle\int_{\mathcal{X}}\displaystyle\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}}\exp\left\\{-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right\\}\frac{1}{L^{n}}d^{n}x_{t}\\\
{}_{(x_{t}\approx
A^{-1}x_{t+1})}&\approx\displaystyle\int_{\mathcal{X}}\displaystyle\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}}\exp\left\\{-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right\\}\frac{|\det(A^{-1})|}{L^{n}}d^{n}x_{t+1}\\\
&=\frac{1}{|\det(A)|L^{n}}\end{aligned}$ (A.2)

Afterward, we can calculate the effective information as

$\displaystyle\begin{aligned}
EI&=\displaystyle\int_{\mathcal{X}}D_{KL}[p(x_{t+1}|x_{t})|E_{D}(x_{t+1})]d^{n}x_{t+1}\\\
&=\displaystyle\int_{\mathcal{X}}\frac{d^{n}x_{t}}{L^{n}}\displaystyle\int_{\mathcal{X}}\displaystyle\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}}\exp\left\\{-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right\\}\\\
&\left[-\ln\left(\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}\right)-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})-\ln\frac{1}{|\det(A)|L^{n}}\right]d^{n}x_{t+1}\\\
&=-\frac{1}{2}\ln\left(\det(\Sigma)(2\pi)^{n}\right)-\frac{n}{2}-\ln\frac{1}{|\det(A)|L^{n}}\\\
&=\ln\displaystyle\frac{|\det(A)|L^{n}}{\displaystyle\det(\Sigma)^{\frac{1}{2}}(2\pi
e)^{\frac{n}{2}}}.\end{aligned}$ (A.3)

According to the properties of information entropy, determinism information

$\displaystyle\begin{aligned}
-\left<H(p(x_{t+1}|x_{t}))\right>&=-\left<-\int_{\mathcal{X}}p(x_{t+1}|x_{t})\ln(p(x_{t+1}|x_{t}))d^{n}x_{t+1}\right>\\\
&=\displaystyle\int_{\mathcal{X}}\frac{d^{n}x_{t}}{L^{n}}\displaystyle\int_{\mathcal{X}}\displaystyle\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}}\exp\left\\{-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right\\}\\\
&\left[-\ln\left(\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}\right)-\frac{1}{2}(x_{t+1}-Ax_{t})^{{}^{\prime}}\Sigma^{-1}(x_{t+1}-Ax_{t})\right]d^{n}x_{t+1}\\\
&=\left[-\ln\left(\det(\Sigma)^{\frac{1}{2}}(2\pi)^{\frac{n}{2}}\right)-\frac{n}{2}\right]\\\
&=\ln\frac{1}{\det(\Sigma)^{\frac{1}{2}}(2\pi e)^{\frac{n}{2}}},\end{aligned}$
(A.4)

degeneracy information

$\displaystyle\begin{aligned}
H(E_{D}(x_{t+1}))&=-\int_{\mathcal{X}}E_{D}(x_{t+1})\ln(E_{D}(x_{t+1}))d^{n}x_{t+1}\\\
{}_{(x_{t}\approx
A^{-1}x_{t+1})}&\approx-\displaystyle\int_{\mathcal{X}}E_{D}(x_{t+1})\ln\frac{1}{|det(A)|L^{n}}d^{n}x_{t+1}\\\
&=\ln\left(|det(A)|L^{n}\right).\end{aligned}$ (A.5)

It’s obvious that

$\displaystyle-\left<H(p(x_{t+1}|x_{t}))\right>+H(E_{D}(x_{t+1}))=\ln\displaystyle\frac{|\det(A)|L^{n}}{\displaystyle\det(\Sigma)^{\frac{1}{2}}(2\pi
e)^{\frac{n}{2}}}=EI.$ (A.6)

Then we can easily get

$\displaystyle\mathcal{J}\equiv\frac{EI}{n}=\frac{1}{n}\ln\displaystyle\frac{|\det(A)|L^{n}}{(2\pi
e)^{\frac{n}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2}}}=\ln\displaystyle\frac{|\det(A)|^{\frac{1}{n}}L}{(2\pi
e)^{\frac{1}{2}}\displaystyle\det(\Sigma)^{\frac{1}{2n}}}.$ (A.7)

If the system is nonlinear, it can be understood that $A=\nabla f(x_{t})$ will
change with $x_{t}$. Then Equations (5.9) and (5.10) can also be proven. ∎

After obtaining effective information, we can calculate causal emergence.

Proof of Theorem 3.1

###### Proof.

For the micro dynamical systems like Equation (2.1) and macro dynamical
systems like Equation (2.8) after coarse-graining, we can calculate the micro
effective information $\mathcal{J}_{m}$ and macro effective information
$\mathcal{J}_{M}$ of two kinds of dynamical systems separately. In this way,
causal emergence

$\displaystyle\begin{aligned}
\Delta\mathcal{J}&=\mathcal{J}_{M}-\mathcal{J}_{m}=\frac{EI_{M}}{k}-\frac{EI_{m}}{n}\\\
&=\frac{1}{k}\ln\displaystyle\frac{|\det(A_{M})|L^{k}}{\displaystyle\det(\Sigma_{M})^{\frac{1}{2}}(2\pi
e)^{\frac{k}{2}}}-\frac{1}{n}\ln\displaystyle\frac{|\det(A)|L^{n}}{\displaystyle\det(\Sigma)^{\frac{1}{2}}(2\pi
e)^{\frac{n}{2}}}\\\
&=\frac{1}{k}\mathop{\left(\ln\displaystyle\frac{1}{\displaystyle\det(\Sigma_{M})^{\frac{1}{2}}(2\pi
e)^{\frac{k}{2}}}+\ln\displaystyle\left(|\det(A_{M})|L^{k}\right)\right)}_{DeterminismInformation\quad
DegeneracyInformation}\\\
&-\frac{1}{n}\left(\ln\displaystyle\frac{1}{\displaystyle\det(\Sigma)^{\frac{1}{2}}(2\pi
e)^{\frac{n}{2}}}+\ln\displaystyle\left(|\det(A)|L^{n}\right)\right)\\\
&=\mathop{\ln\frac{|\det(WAW^{\dagger})|^{\frac{1}{k}}}{|\det(A)|^{\frac{1}{n}}}}_{DegeneracyEmergence}+\mathop{\ln\frac{|\det(\Sigma)|^{\frac{1}{2n}}}{|\det(W\Sigma
W^{T})|^{\frac{1}{2k}}}}_{DeterminismEmergence}.\end{aligned}$ (A.8)

∎

##  Appendix Appendix B Optimization of Macrodynamic Parameter Matrix

To maximize causal emergence, we need to use relevant content from matrix
theory to optimize coarse-graining strategies.

Proof of Lemma 3.1

###### Proof.

Perform eigendecomposition on matrix $A$, $A=V\Lambda V^{-1}$, $\Lambda={\rm
diag}(\lambda_{1},\dots,\lambda_{n})\in\mathcal{R}^{n\times n}$ is the
diagonal matrix of eigenvalues, $V\in\mathcal{R}^{n\times n}$ is the
eigenvector matrix corresponding to the eigenvalue matrix. Determinant

$\displaystyle\det(WAW^{\dagger})=\det(WV\Lambda
V^{-1}W^{\dagger})=\det(\tilde{W}\Lambda\tilde{W}^{\dagger})=\displaystyle\sum_{m}\left(\gamma_{m}\prod_{\lambda_{i}\in
L_{m}}\lambda_{i}\right),$ (B.1)

in which
$L_{m}=\\{\lambda_{N_{m,1}},\lambda_{N_{m,2}},\dots,\lambda_{N_{m,k}}\\}\subseteq\\{\lambda_{1},\dots,\lambda_{n}\\}$
are new sets formed by arbitrarily choosing $k$ eigenvalues from $n$
eigenvalues of $A$. The sorting of these sets is based on the sum of the
elements in the set, in descending order. $N_{m}\subseteq\\{1,\dots,n\\}$, the
number of elements in sets $|N_{m}|,|L_{m}|=k$, $m=1,2,\dots,C_{n}^{k}$.
$\gamma_{m}$ are coefficients determined by $W$, $\gamma_{m}>0$,
$\sum_{m}\gamma_{m}=1$. It is not difficult to find that when $\gamma_{m}=1$,
matrix $A$ only retains the eigenvalues in $L_{m}$ after transformation.

The calculation of $\gamma_{1},\dots,\gamma_{C_{n}^{k}}$ are as follows. Set
$WV=\tilde{W}=(w_{ij})_{k\times n}$, then
$W^{\dagger}V^{-1}=\tilde{W}^{\dagger}=(v_{ij})_{n\times k}$. According to the
principle of determinant calculation, we can divide $\gamma_{m}$ into two
parts

$\displaystyle\gamma_{m}=\gamma_{m,1}-\gamma_{m,2},$ (B.2)

$\gamma_{m,1},\gamma_{m,2}$ correspond to the sum of terms with coefficients
of $1$ and $-1$ in the determinant calculation, respectively. So

$\displaystyle\gamma_{m,1}=\displaystyle\sum_{l=0}^{k-1}\left[\sum_{s\in\\{s\\}_{m}}\left(\prod_{i=1}^{k}w_{i,s_{i}}v_{s_{i},j}\right)\right],j=\begin{cases}i+l,i+l&\leq
k,\\\ i+l-k,i+l&>k.\end{cases}$ (B.3)
$\displaystyle\gamma_{m,2}=\displaystyle\sum_{l=2}^{k+1}\left[\sum_{s\in\\{s\\}_{m}}\left(\prod_{i=1}^{k}w_{i,s_{i}}v_{s_{i},j}\right)\right],j=\begin{cases}l-i,l-i&>0,\\\
l-i+k,l-i&\geq 0.\end{cases}$ (B.4)

$s=\\{s_{1},s_{2},...s_{k}\\}$ is a new sequence formed by randomly arranging
all elements in $N_{m}$, $\\{s\\}_{m}$ is a set composed of such elements $s$.
Number of elements in the set $|\\{s\\}_{m}|=k!$.

If all eigenvalues of $A$ satisfy
$\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{n}$, we can simultaneously
find the maximum and minimum values of $\det(WAW^{\dagger})$. When
$\gamma_{1}=1$, only the maximum $k$ eigenvalues are retained, and
$\det(WAW^{\dagger})$ takes the maximum value $\prod_{i=1}^{k}\lambda_{i}$. On
the contrary, when $\gamma_{m}=1$, only the smallest $k$ eigenvalues are
retained, and the $\det(WAW^{\dagger})$ takes the minimum value
$\prod_{i=n-k+1}^{n}\lambda_{i}$. So

$\displaystyle\displaystyle\prod_{i=n-k+1}^{n}\lambda_{i}\leq\det(WAW^{\dagger})\leq\prod_{i=1}^{k}\lambda_{i}.$
(B.5)

From the proof of B.5, we know that $\gamma_{m}$ are coefficients determined
by $W$ and when $\gamma_{m}=1$, matrix $A$ only retains the eigenvalues in
$L_{m}$. So when the eigenvalues of matrix $A$ are not specified to be all
positive real numbers, the modulus length satisfies
$|\lambda_{1}|\geq|\lambda_{2}|\geq\dots\geq|\lambda_{n}|$, then

$\displaystyle\begin{aligned}
|\det(WAW^{\dagger})|=\left|\displaystyle\sum_{m}\left(\gamma_{m}\prod_{\lambda_{i}\in
L_{m}}\lambda_{i}\right)\right|\leq\displaystyle\sum_{m}\gamma_{m}\left|\prod_{\lambda_{i}\in
L_{m}}\lambda_{i}\right|=\displaystyle\sum_{m}\gamma_{m}\left(\prod_{\lambda_{i}\in
L_{m}}\left|\lambda_{i}\right|\right)\leq\displaystyle\prod_{i=1}^{k}|\lambda_{i}|,\end{aligned}$
(B.6)

We directly study the situation of eigenvalues in the complex field. When
$L_{m}={\lambda_{1},\dots,\lambda_{k}}$, the determinant
$|\det(WAW^{\dagger})|$ reach its maximum value. The decrease of $\gamma_{m}$,
the intervention of smaller absolute values of eigenvalues
$|\lambda_{i}|,i\geq k$ leads to the decrease of $|\det(WAW^{\dagger})|$. ∎

Proof of Theorem 5.1

###### Proof.

According to (B.1) and Jensen’s inequality, we can obtain inequalities

$\displaystyle\begin{aligned} |\det(W\Sigma
W^{\dagger})|^{-\frac{1}{2}}=\left[\sum\left(\gamma_{m}\prod\kappa_{i}\right)\right]^{-\frac{1}{2}}\leq\left[\sum\gamma_{m}\left(\prod\kappa_{i}\right)^{-\frac{1}{2}}\right]=|\det(W\Sigma^{-\frac{1}{2}}W^{\dagger})|,\end{aligned}$
(B.7)

if and only if the eigenvalues are equal, the equal sign holds. By utilizing
this inequality and combining it with the relevant properties of linear
algebra, we can conclude that

$\displaystyle\begin{aligned}
\Delta\mathcal{J}&=\ln\frac{|\det(WAW^{\dagger})|^{\frac{1}{k}}}{|\det(W\Sigma
W^{\dagger})|^{\frac{1}{2k}}}-\ln\frac{|\det(A)|^{\frac{1}{n}}}{|\det(\Sigma)|^{\frac{1}{2n}}}\\\
&=\frac{1}{k}\ln(|\det(W\Sigma
W^{\dagger})|^{-\frac{1}{2}}|\det(WAW^{\dagger})|)-\frac{1}{n}\ln|\Sigma^{-\frac{1}{2}}A|\\\
&\leq\frac{1}{k}\ln(|\det(W\Sigma^{-\frac{1}{2}}W^{\dagger})||\det(WAW^{\dagger})|)-\frac{1}{n}\ln|\Sigma^{-\frac{1}{2}}A|\\\
&=\frac{1}{k}\ln|\det(W\Sigma^{-\frac{1}{2}}W^{\dagger}WAW^{\dagger})|-\frac{1}{n}\ln|\Sigma^{-\frac{1}{2}}A|\\\
{}_{(\Sigma=VKV^{T},A=V\Lambda
V^{T})}&\leq\frac{1}{k}\ln|\det(W\Sigma^{-\frac{1}{2}}AW^{\dagger})|-\frac{1}{n}\ln|\Sigma^{-\frac{1}{2}}A|\\\
{}_{(W=(\tilde{W}_{k},O_{k\times{(n-k)}})V^{T})}&\leq\frac{1}{k}\sum_{i=1}^{k}\ln\displaystyle|\tilde{d}_{i}|-\frac{1}{n}\sum_{i=1}^{n}\ln\displaystyle|\tilde{d}_{i}|\end{aligned}$
(B.8)

When $A$ and $\Sigma$ share the same $n$ eigenvectors, that is to say,
$\Sigma=VKV^{T}$ and $A=V\Lambda V^{T}$, where $V$ is the matrix formed by the
shared eigenvectors, and $K$ and $\Lambda$ are the eigenvalues for $A$ and
$\Sigma$, respectively. The eigenvalues of $\Sigma^{-1/2}A$ are equal to the
product of the eigenvalues of $A$ and $\Sigma^{-1/2}$.
$W=(\tilde{W}_{k},O_{k\times{(n-k)}})V^{T}$ represents that both $A$ and
$\Sigma^{-1/2}$ retain the eigenvalues with the highest norm, the second and
third unequal signs can also be taken as equal signs. ∎

##  Appendix Appendix C Loss function and causal emergence

In this section, we first prove that $S_{D}$ is the supremum of $L_{D}$.

Proof of Lemma 5.1

###### Proof.

Based on the theory of matrix modules, when
$||x_{t}||^{*}=\mathop{\sup}_{t}||x_{t}||<\infty$,

$\displaystyle\begin{aligned}
L_{D}&=||Ax_{t}+\varepsilon_{t}-(W^{\dagger}WAW^{\dagger}Wx_{t}+W^{\dagger}W\varepsilon_{t})||\\\
&=||(A-W^{\dagger}WAW^{\dagger}W)x_{t}+(I_{n}-W^{\dagger}W)\varepsilon_{t}||\\\
&\leq||(A-W^{\dagger}WAW^{\dagger}W)x_{t}||+||(I_{n}-W^{\dagger}W)\varepsilon_{t}||\\\
&\leq||(A-W^{\dagger}WAW^{\dagger}W)||_{F}||x_{t}||+||(I_{n}-W^{\dagger}W)||_{F}||\varepsilon_{t}||\\\
&=||(A-W^{\dagger}WAW^{\dagger}W)||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&\leq||(A-W^{\dagger}WAW^{\dagger}W)||_{F}||x_{t}||^{*}+(n-k)||\varepsilon_{t}||=S_{D}.\end{aligned}$
(C.1)

∎

Next, we prove that $W$ at the minimum value of the supremum $S_{D}$ can
maximize the degeneracy emergence.

Proof of Lemma 5.2

###### Proof.

$|\lambda_{1}|\geq|\lambda_{2}|\geq\dots\geq|\lambda_{n}|\geq 0$ are $n$
modulus of eigenvalues of matrix $A$ sorted from largest to smallest.
$\Lambda={\rm diag}(\lambda_{1},\dots,\lambda_{n})\in\mathcal{R}^{n\times n}$.
$\tilde{W}=WV$, $\tilde{W}^{\dagger}\tilde{W}$ and
$I-\tilde{W}^{\dagger}\tilde{W}$ are both symmetric idempotent matrices.
$rk(\tilde{W}^{\dagger}\tilde{W})=k$,
$I-\tilde{W}^{\dagger}\tilde{W}=(r_{ij})_{n\times n}$ has the following
properties:

  1. (1)

$\displaystyle\sum_{l=1}^{n}r_{il}r_{lj}=r_{ij}$.

  2. (2)

$||I-\tilde{W}^{\dagger}\tilde{W}||_{F}=\displaystyle\sqrt{\sum_{i,j=1}^{n,n}r_{ij}^{2}}=k$.

  3. (3)

$\forall l,0<l\leq
n,\displaystyle\sqrt{\sum_{i,j=1}^{n,n}(r_{il}r_{lj})^{2}}=1$

Based on the theory of matrix modules,

$\displaystyle\begin{aligned}
S_{D}&=||(A-W^{\dagger}WAW^{\dagger}W)||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
{}_{(\tilde{W}=WV)}&=||V(\Lambda-\tilde{W}^{\dagger}\tilde{W}\Lambda\tilde{W}^{\dagger}\tilde{W})V^{-1}||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&=||\Lambda-\tilde{W}^{\dagger}\tilde{W}\Lambda\tilde{W}^{\dagger}\tilde{W}||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&=||(I_{n}-\tilde{W}^{\dagger}\tilde{W})\Lambda(I_{n}-\tilde{W}^{\dagger}\tilde{W})+2(I_{n}-\tilde{W}^{\dagger}\tilde{W})\Lambda\tilde{W}^{\dagger}\tilde{W}||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&\geq||(I_{n}-\tilde{W}^{\dagger}\tilde{W})\Lambda(I_{n}-\tilde{W}^{\dagger}\tilde{W})||_{F}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&=\displaystyle\sqrt{\sum_{0<i,j\leq
n}\left|\sum_{l=1}^{n}r_{il}r_{lj}\lambda_{l}\right|^{2}}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&\geq\displaystyle\sqrt{\sum_{0<i,j\leq
n}\sum_{l=1}^{n}\left|r_{il}r_{lj}\lambda_{l}\right|^{2}}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&\geq\displaystyle\sqrt{\sum_{0<i,j\leq
n}\sum_{l=k+1}^{n}\left|r_{il}r_{lj}\lambda_{l}\right|^{2}}||x_{t}||+(n-k)||\varepsilon_{t}||\\\
&\geq\displaystyle\sqrt{\sum_{l=k+1}^{n}\left|\lambda_{l}\right|^{2}}||x_{t}||+(n-k)||\varepsilon_{t}||.\end{aligned}$
(C.2)

If and only if the maximum $k$ eigenvalues are preserved in $\hat{A}$, only
the $n-k$ eigenvalues with the smallest modulus are retained $S_{D}$, the
equal sign holds. $W=(\tilde{W}_{k},O_{k\times{(n-k)}})V^{-1}$, exactly the
same solution set as the degeneracy emergence at its maximum
$\Delta\mathcal{J}_{1}^{*}$. ∎

## References

  * [1] J. H. Holland, Emergence: From chaos to order.  OUP Oxford, 2000.
  * [2] G. West, Scale: The universal laws of life, growth, and death in organisms, cities, and companies.  Penguin, 2018.
  * [3] R. Li, L. Dong, J. Zhang, X. Wang, W.-X. Wang, Z. Di, and H. E. Stanley, “Simple spatial scaling rules behind complex cities,” Nature communications, vol. 8, no. 1, p. 1841, 2017.
  * [4] L. Dong, Z. Huang, J. Zhang, and Y. Liu, “Understanding the mesoscopic scaling patterns within cities,” Scientific reports, vol. 10, no. 1, p. 21201, 2020.
  * [5] J. Zhang, C. P. Kempes, M. J. Hamilton, R. Tao, and G. B. West, “Scaling laws and a general theory for the growth of public companies,” arXiv preprint arXiv:2109.10379, 2021.
  * [6] Y. Zhang, K. Liu, and W. Gui, “Bayesian and e-bayesian estimations of bathtub-shaped distribution under generalized type-i hybrid censoring,” Entropy, vol. 23, no. 8, p. 934, 2021.
  * [7] J. Xu, X. Chen, L. Wen, and J. Zhang, “Company scaling and its deviations: New indicators for enterprise evaluation and bankruptcy prediction,” Plos one, vol. 18, no. 10, p. e0287105, 2023.
  * [8] R. T. Wicks, S. C. Chapman, and R. Dendy, “Mutual information as a tool for identifying phase transitions in dynamical complex systems with limited data,” Physical Review E, vol. 75, no. 5, p. 051125, 2007.
  * [9] C. Hartman and B. Benes, “Autonomous boids,” Computer Animation and Virtual Worlds, vol. 17, no. 3-4, pp. 199–206, 2006.
  * [10] D. Liu, W. Liu, X. Yuan, and Y. Jiang, “Conscious and unconscious processing of ensemble statistics oppositely modulate perceptual decision-making.,” American Psychologist, vol. 78, no. 3, p. 346, 2023.
  * [11] L. Wang and Y. Jiang, “Action observation network: domain-specific or domain-general?,” Trends in Cognitive Sciences, 2023.
  * [12] R. Wang, V. Gates, Y. Shen, P. Tino, and Z. Kourtzi, “Flexible structure learning under uncertainty,” Available at SSRN 4014051.
  * [13] S. Jingnan, J. He, and X. Gao, “Neurofeedback training of control network improves ssvep-based bci performance in children,” 2021.
  * [14] O. Sporns, J. Faskowitz, A. S. Teixeira, S. A. Cutts, and R. F. Betzel, “Dynamic expression of brain functional systems disclosed by fine-scale analysis of edge time series,” Network Neuroscience, vol. 5, no. 2, pp. 405–433, 2021.
  * [15] T. F. Varley, Uncovering Higher-Order Structures in Complex Systems with Multivariate Information Theory.  Indiana University, 2023.
  * [16] Z. Zhao, Y. Zhou, B. Liu, J. He, J. Zhao, Y. Cai, J. Fan, X. Li, Z. Wang, Z. Lu, et al., “Two-photon synthetic aperture microscopy for minimally invasive fast 3d imaging of native subcellular behaviors in deep tissue,” Cell, vol. 186, no. 11, pp. 2475–2491, 2023.
  * [17] T. Dong, J. He, S. Wang, L. Wang, Y. Cheng, and Y. Zhong, “Inability to activate rac1-dependent forgetting contributes to behavioral inflexibility in mutants of multiple autism-risk genes,” Proceedings of the National Academy of Sciences, vol. 113, no. 27, pp. 7644–7649, 2016.
  * [18] B. Yuan, J. Zhang, A. Lyu, J. Wu, Z. Wang, M. Yang, K. Liu, M. Mou, and P. Cui, “Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies,” Entropy, vol. 26, no. 2, p. 108, 2024.
  * [19] J. T. Oden, T. Belytschko, I. Babuska, and T. Hughes, “Research directions in computational mechanics,” Computer Methods in Applied Mechanics and Engineering, vol. 192, no. 7-8, pp. 913–922, 2003.
  * [20] A. K. Seth, “Measuring emergence via nonlinear granger causality.,” in alife, vol. 2008, pp. 545–552, 2008.
  * [21] F. E. Rosas, P. A. M. Mediano, H. J. Jensen, A. K. Seth, A. B. Barrett, R. L. Carhart-Harris, and D. Bor, “Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data,” PLOS Computational Biology, vol. 16, p. e1008289, Dec 2020.
  * [22] E. P. Hoel, L. Albantakis, and G. Tononi, “Quantifying causal emergence shows that macro can beat micro,” Proceedings of the National Academy of Sciences, vol. 110, no. 49, pp. 19790–19795, 2013.
  * [23] E. P. Hoel, L. Albantakis, W. Marshall, and G. Tononi, “Can the macro beat the micro? integrated information across spatiotemporal scales,” Neuroscience of Consciousness, vol. 2016, no. 1, p. niw012, 2016.
  * [24] F. E. Rosas, B. C. Geiger, A. I. Luppi, A. K. Seth, D. Polani, M. Gastpar, and P. A. Mediano, “Software in the natural world: A computational approach to emergence in complex multi-level systems,” arXiv preprint arXiv:2402.09090, 2024.
  * [25] E. P. Hoel, “When the map is better than the territory,” Entropy, vol. 19, no. 5, p. 188, 2017.
  * [26] P. L. Williams and R. D. Beer, “Nonnegative decomposition of multivariate information,” arXiv preprint arXiv:1004.2515, 2010.
  * [27] B. Rassouli, F. E. Rosas, and D. Gündüz, “Data disclosure under perfect sample privacy,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 2012–2025, 2019.
  * [28] K. Liu, X. Lü, F. Gao, and J. Zhang, “Expectation-maximizing network reconstruction and most applicable network types based on binary time series data,” Physica D: Nonlinear Phenomena, vol. 454, p. 133834, 2023.
  * [29] J. Zhang, R. Tao, and B. Yuan, “Dynamical reversibility and a new theory of causal emergence,” arXiv preprint arXiv:2402.15054, 2024.
  * [30] P. Chvykov and E. Hoel, “Causal geometry,” Entropy, vol. 23, no. 1, p. 24, 2020.
  * [31] M. K. Transtrum, B. B. Machta, K. S. Brown, B. C. Daniels, C. R. Myers, and J. P. Sethna, “Perspective: Sloppiness and emergent theories in physics, biology, and beyond,” The Journal of chemical physics, vol. 143, no. 1, 2015.
  * [32] B. Klein and E. Hoel, “The emergence of informative higher scales in complex networks,” Complexity, vol. 2020, pp. 1–12, 2020.
  * [33] R. Comolatti and E. Hoel, “Causal emergence is widespread across measures of causation,” arXiv preprint arXiv:2202.01854, 2022.
  * [34] K. Liu and Y. Zhang, “The e-bayesian estimation for lomax distribution based on generalized type-i hybrid censoring scheme,” Mathematical Problems in Engineering, vol. 2021, pp. 1–19, 2021.
  * [35] M. Yang, Z. Wang, K. Liu, Y. Rong, B. Yuan, and J. Zhang, “Finding emergence in data: causal emergence inspired dynamics learning,” arXiv preprint arXiv:2308.09952, 2023.
  * [36] L. P. Kadanoff, “Scaling laws for ising models near t c,” Physics Physique Fizika, vol. 2, no. 6, p. 263, 1966.
  * [37] C. d. Villemagne and R. E. Skelton, “Model reductions using a projection formulation,” International Journal of Control, vol. 46, no. 6, pp. 2141–2169, 1987.
  * [38] D. L. Boley, “Krylov space methods on state-space control models,” Circuits, Systems and Signal Processing, vol. 13, pp. 733–758, 1994.
  * [39] K. Gallivan, E. Grimme, and P. Van Dooren, “Asymptotic waveform evaluation via a lanczos method,” Applied Mathematics Letters, vol. 7, no. 5, pp. 75–80, 1994.
  * [40] S. Gugercin, “An iterative svd-krylov based method for model reduction of large-scale dynamical systems,” Linear Algebra and its Applications, vol. 428, no. 8-9, pp. 1964–1986, 2008.
  * [41] A. C. Antoulas, “An overview of approximation methods for large-scale dynamical systems,” Annual reviews in Control, vol. 29, no. 2, pp. 181–190, 2005.
  * [42] H.-Y. Hu, S.-H. Li, L. Wang, and Y.-Z. You, “Machine learning holographic mapping by neural network renormalization group,” Physical Review Research, p. 023369, 2020.
  * [43] J. Zhang and K. Liu, “Neural information squeezer for causal emergence,” Entropy, vol. 25, no. 1, p. 26, 2022.
  * [44] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio, “Toward causal representation learning,” Proceedings of the IEEE, vol. 109, no. 5, pp. 612–634, 2021.
  * [45] K. A. Murphy and D. S. Bassett, “Information decomposition to identify relevant variation in complex systems with machine learning,” arXiv preprint arXiv:2307.04755, 2023.
  * [46] K. A. Murphy and D. S. Bassett, “Information decomposition in complex systems via machine learning,” Proceedings of the National Academy of Sciences, vol. 121, no. 13, p. e2312988121, 2024.
  * [47] W. Dunsmuir and E. J. Hannan, “Vector linear time series models,” Advances in Applied Probability, vol. 8, no. 2, pp. 339–364, 1976.
  * [48] E. J. Hannan and L. Kavalieris, “Multivariate linear time series models,” Advances in Applied Probability, vol. 16, no. 3, pp. 492–561, 1984.
  * [49] J. C. A. Barata and M. S. Hussein, “The moore–penrose pseudoinverse: A tutorial review of the theory,” Brazilian Journal of Physics, vol. 42, pp. 146–165, 2012.
  * [50] R. A. Horn and C. R. Johnson, Matrix analysis.  Cambridge university press, 2012.
  * [51] J. Pearl, “Causal inference in statistics: An overview,” 2009.
  * [52] P. Venkatesh, C. Bennett, S. Gale, T. Ramirez, G. Heller, S. Durand, S. Olsen, and S. Mihalas, “Gaussian partial information decomposition: Bias correction and application to high-dimensional data,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  * [53] G. F. Lawler and V. Limic, Random walk: a modern introduction, vol. 123.  Cambridge University Press, 2010.
  * [54] D. W. Peaceman and H. H. Rachford, Jr, “The numerical solution of parabolic and elliptic differential equations,” Journal of the Society for industrial and Applied Mathematics, vol. 3, no. 1, pp. 28–41, 1955.
  * [55] T. L. Bergman, Fundamentals of heat and mass transfer.  John Wiley & Sons, 2011.
  * [56] S. Patankar, Numerical heat transfer and fluid flow.  CRC press, 2018.
  * [57] R. J. LeVeque, Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems.  SIAM, 2007.
  * [58] N. R. Roy and P. Chandra, “Energy dissipation model for wireless sensor networks: a survey,” International Journal of Information Technology, vol. 12, no. 4, pp. 1343–1353, 2020.
  * [59] E. W. Weisstein, “Rotation matrix,” https://mathworld. wolfram. com/, 2003.
  * [60] W. Qiu, Analytic Geometry.  Peking University Press, 1996.
  * [61] C. B. Boyer, History of analytic geometry.  Courier Corporation, 2012.