Towards an extended taxonomy of information dynamics via Integrated
Information Decomposition

Pedro A.M. Mediano1*, Fernando E. Rosas2,3,4*, Andrea I. Luppi5,6,7, Robin L.
Carhart-Harris2, Daniel Bor1, Anil K. Seth8, Adam B. Barrett8,9

1 Department of Psychology, University of Cambridge, Cambridge, UK

2 Centre for Psychedelic Research, Imperial College London, London, UK

3 Data Science Institute, Imperial College London, London, UK

4 Centre for Complexity Science, Imperial College London, London, UK

5 University Division of Anaesthesia, University of Cambridge, Cambridge, UK

6 Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK

7 Leverhulme Centre for the Future of Intelligence, University of Cambridge,
Cambridge, UK

8 Sackler Center for Consciousness Science, University of Sussex, Brighton, UK

9 The Data Intensive Science Centre, University of Sussex, Brighton, UK

* These authors contributed equally to this work.

* Correspondence<EMAIL_ADDRESS><EMAIL_ADDRESS>
## Abstract

Complex systems, from the human brain to the global economy, are made of
multiple elements that interact in such ways that the behaviour of the ‘whole’
often seems to be more than what is readily explainable in terms of the ‘sum
of the parts.’ Our ability to understand and control these systems remains
limited, one reason being that we still don’t know how best to describe – and
quantify – the higher-order dynamical interactions that characterise their
complexity. To address this limitation, we combine principles from the
theories of Information Decomposition and Integrated Information into what we
call Integrated Information Decomposition, or $\Phi$ID. $\Phi$ID provides a
comprehensive framework to reason about, evaluate, and understand the
information dynamics of complex multivariate systems. $\Phi$ID reveals the
existence of previously unreported modes of collective information flow,
providing tools to express well-known measures of information transfer and
dynamical complexity as aggregates of these modes. Via computational and
empirical examples, we demonstrate that $\Phi$ID extends our explanatory power
beyond traditional causal discovery methods – with profound implications for
the study of complex systems across disciplines.

## Introduction

How can we best characterise the plethora of dynamical phenomena that can
emerge in a system of interacting components? Progress on this question seems
critical to support advances in our ability to understand, engineer, and
control complex systems such as the central nervous system [1], the global
climate [2], macroeconomics [3], and many others. The predominant approach for
analysing such systems is in terms of cause-effect pairs, seeking to link each
cause to its individual effect via a ‘causal arrow’ (see e.g. [4, 5]).
However, such approaches have an important limitation that is rarely
acknowledged: they neglect higher-order relationships that cannot be expressed
in terms of such arrows, which – nonetheless – have been shown to be
ubiquitous in complex systems as varied as genetic networks [6], financial
markets [7], baroque music [8], and the human brain [9, 10, 11].

As an alternative approach, various one-dimensional metrics have been proposed
to assess the dynamical complexity of processes (e.g. [12, 13]). An
interesting case of this is found in the neuroscience literature, where it has
been proposed that a key feature of the neural dynamics underpinning advanced
cognition, flexible behaviour, and even the presence of consciousness, can be
captured by a single number that accounts for the brain’s ability to
‘integrate information.’ There have been several attempts to operationalise
this notion, including the various $\Phi$ measures in Integrated Information
Theory (IIT) [14, 15, 16] and Causal Density (CD) [17]; however, these
measures have been shown to behave inconsistently [18, 19, 8], making
empirical applications difficult to interpret.

Here we argue that the two research programs described above are limited
because they are rooted in the intuition that information can only be
transferred or stored between parts of a system. However, it has recently
become apparent that high-order interactions that go beyond transfer and
storage can be captured via the framework of Partial Information Decomposition
(PID) [20], which demonstrates how the information that two or more sources
provide about a given target can be decomposed into redundant, unique, and
synergistic components [21]. Specifically, redundancy refers to information
held simultaneously by both sources, unique information is that held by one
source but not the other, and synergy is the information conveyed by both
sources together but none of them in isolation. PID has been usefully applied
to systems such as cellular automata [22, 23], artificial neural networks [24,
25], socioeconomic data [26], and gene interactions [27]. However, the
information taxonomy introduced by PID is only valid in scenarios with a
single target variable, being unable to discriminate between different ways in
which two or more target variables can be affected collectively. This
important limitation prevents PID from providing an encompassing view of the
dynamics of complex systems, where the past of multiple variables affects the
future of multiple variables.

In this paper we introduce the _Integrated Information Decomposition_
framework ($\Phi$ID), which combines principles from the theories of
Information Decomposition and Integrated Information to overcome PID’s
critical limitation and enables a complete information decomposition on groups
of time series. The $\Phi$ID framework introduces a novel information
taxonomy, revealing the existence of modes of information dynamics that have
not been previously reported. Furthermore, it allows us to show precisely how
measures of transfer entropy and integrated information are aggregates of
several qualitatively distinct modes. As proof of concept, we use two example
datasets (simultaneous recordings of heart rate and respiratory volume in
healthy subjects; and a neurobiologically realistic simulation of whole-brain
activity) to show that the high-order effects discussed here are not merely
theoretical speculations, but can have substantial effects on real-world
analyses and their interpretation.

## Results

### Integrated information decomposition: $\Phi$ID

#### Decomposing multivariate information

Consider a system composed of two interdependent elements that co-evolve over
time. If the system’s future state depends only on the preceding state (i.e.
if its dynamics are Markovian), then the total amount of information carried
from past to future is known as the time-delayed mutual information111In non-
Markovian systems the corresponding quantity is known as excess entropy [13].
Information decomposition in non-Markovian systems will be covered in a
subsequent publication. [19], and can be quantified as the mutual information
between past and future states of the system:

$\mathrm{TDMI}=I(\bm{X}_{t};\bm{X}_{t+1})=I(X^{1}_{t},X^{2}_{t}\;;\;X^{1}_{t+1},X^{2}_{t+1})~{}.$
(1)

Above, the superscripts $1$ and $2$ refer to the two elements of the system,
and $\bm{X}_{t}=(X^{1}_{t},X^{2}_{t})$ is a shorthand notation for the
system’s state at time $t$.

One can analyse the total information flow using the Partial Information
Decomposition (PID) framework, which decomposes the mutual information between
multiple sources and a target variable into unique (Un), redundant (Red), and
synergistic (Syn) contributions – also known as ‘information atoms’ [21].
However, just as the great strength of PID is its capacity to account for
multiple sources of information, its main limitation is that it is restricted
to considering only a single (potentially multivariate) target. Therefore, a
direct application of PID to the TDMI would have to consider the past states
of the system’s elements $X_{t}^{1}$ and $X_{t}^{2}$ as sources and the joint
future state of the system $\bm{X}_{t+1}$ as target. Specifically, focusing on
how information flows from past to future, this account decomposes the
information provided by past states $X_{t}^{1}$ and $X_{t}^{2}$ about the
joint future state $\bm{X}_{t+1}$, as

$\displaystyle\mathrm{TDMI}=\texttt{Red}(X_{t}^{1},X_{t}^{2};\bm{X}_{t+1})+\texttt{Un}(X_{t}^{1};\bm{X}_{t+1}|X_{t}^{2})+\texttt{Un}(X_{t}^{2};\bm{X}_{t+1}|X_{t}^{1})+\texttt{Syn}(X_{t}^{1},X_{t}^{2};\bm{X}_{t+1}).$

Here, the first term corresponds to the redundant information provided by both
$X_{t}^{1}$ and $X_{t}^{2}$ about the joint future state of the system
$\bm{X}_{t+1}$; the second and third terms refer to the unique information
that only the past state $X_{t}^{1}$ provides about the joint future state
$\bm{X}_{t+1}$ (and likewise for $X_{t}^{2}$); and finally, the last term
accounts for the synergistic information that the two elements’ past states
provide about the system’s joint future, only when they are considered
together. Unfortunately, this approach neglects the fact that the parts of the
system are distinct not only in the past, but also in the future – in other
words, it can tell where the information is coming from, but not where it is
going to. One naive solution would be to consider the time-reverse of the
equation above, with both future states as sources and the joint past state as
target (what we refer to as the ‘backwards’ PID). However, this leaves
unsolved the underlying problem that PID cannot provide a unified
decomposition of information across multiple sources and multiple targets
simultaneously.

In order to obtain an encompassing description of the system’s dynamics, one
must extend the PID approach to enable multi-target information decomposition.
To address this issue, our strategy is to take a temporal perspective on PID
itself, focusing on how the information encoded by the PID atoms may evolve
over time. For instance, information that was uniquely encoded by one element
of the system in the past may become redundantly encoded by two in the future,
or synergistic information may subsequently become uniquely encoded by one of
the elements – and so on. This intuition suggests that when decomposing
information flow between past and future in a system of two elements there are
not four, but rather 16 distinct information atoms: each corresponding to a
pair of the original four PID atoms evolving from past to future. Thus, we
denote each $\Phi\mathrm{ID}$ atom as a pair of PID atoms: e.g. the
information that was carried redundantly in the past and becomes synergistic
in the future corresponds to $\texttt{Red}\\!\rightarrow\\!\texttt{Syn}$; and
the synergistic information in the past that becomes unique to $X^{1}_{t+1}$
in the future corresponds to $\texttt{Syn}\\!\rightarrow\\!\texttt{Un}^{1}$;
and so on.

Like the original PID atoms, the $\Phi\mathrm{ID}$ atoms are structured in a
_lattice_ , depicted in Fig. 1 (for a formal derivation see Methods). As with
PID, several extra ingredients need to be specified to compute the numerical
value of these atoms: in $\Phi\mathrm{ID}$, it can be shown that standard
Shannon theory and PID together specify a system of 15 equations for the 16
$\Phi\mathrm{ID}$ atoms, yielding an underdetermined system. To compute the
atoms one must provide one extra constraint, which can be done by introducing
a _double-redundancy_ function – a multi-target extension of PID’s redundancy
function.222For PID, it is often the redundancy function that provides the
needed constraint to allow computation of numerical values for the PID atoms.
Note that the $\Phi$ID framework does not impose a particular functional form
for the double-redundancy function, and hence different functional forms can
be explored. A formal development of these ideas, and its extension to systems
of more than two elements, is provided in the Methods.

Fig 1: Lattice representation of information atoms from PID to $\Phi$ID. a)
System of two elements co-evolving and interacting over time, decomposed
either according to a _forward PID_ (left) or a _backward PID_ (middle).
Integrated Information Decomposition ($\Phi\mathrm{ID}$; right) unifies and
extends both PIDs, providing an encompassing framework of information dynamics
in complex systems. b) Redundant (Red), unique (Un) and synergistic (Syn)
atoms in the bivariate PID lattice. c) $\Phi\mathrm{ID}$ lattice for a system
of two time series, where each $\Phi\mathrm{ID}$ atom corresponds to a pair of
two PID atoms that indicate how information evolves from past to future.

#### A new taxonomy for information dynamics in complex systems

Based on $\Phi$ID, we propose an extended taxonomy of information dynamics
according to six disjoint and qualitatively distinct phenomena (Fig. 2):

Fig 2: Taxonomy of information dynamics in complex systems. Six qualitatively
different modes of information dynamics, represented in terms of their
constituent atoms in the $\Phi\mathrm{ID}$ lattice.

Storage

: Information that remains in the same element or set of elements (even if it
includes collective effect). Comprises
$\texttt{Red}\\!\rightarrow\\!\texttt{Red}$,
$\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{1}$,
$\texttt{Un}^{2}\\!\rightarrow\\!\texttt{Un}^{2}$, and
$\texttt{Syn}\\!\rightarrow\\!\texttt{Syn}$.

Copy

: Information that becomes duplicated. Comprises
$\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Red}$, and
$\texttt{Un}^{2}\\!\rightarrow\\!\texttt{Red}$.

Transfer

Information that moves between elements. Comprises
$\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{2}$ and
$\texttt{Un}^{2}\\!\rightarrow\\!\texttt{Un}^{1}$.

Erasure

Duplicated information that is pruned. Comprises
$\texttt{Red}\\!\rightarrow\\!\texttt{Un}^{1}$ and
$\texttt{Red}\\!\rightarrow\\!\texttt{Un}^{2}$.

Downward causation

Collective properties that define individual futures. Comprises
$\texttt{Syn}\\!\rightarrow\\!\texttt{Un}^{1}$,
$\texttt{Syn}\\!\rightarrow\\!\texttt{Un}^{2}$, and
$\texttt{Syn}\\!\rightarrow\\!\texttt{Red}$.

Upward causation

Collective properties that are defined by individuals. Comprises
$\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Syn}$,
$\texttt{Un}^{2}\\!\rightarrow\\!\texttt{Syn}$, and
$\texttt{Red}\\!\rightarrow\\!\texttt{Syn}$.

While the downward causation mode has been discussed in the past [20], upward
causation and synergistic storage
($\texttt{Syn}\\!\rightarrow\\!\texttt{Syn}$) have, to our knowledge, not been
reported in the literature. Importantly, traditional methods of causal
discovery cannot capture the type of modes that involves interactions between
multiple target variables. In effect, while methods such as PID or
multivariate Granger causality can effectively deal with multivariate target
variables, they cannot untangle how each component of the target may be
differently affected, and — more importantly — how sources may affect the
target as a whole, without (or in addition to) affecting its parts. Overall,
this new taxonomy leads to less ambiguous and fully quantifiable descriptions
of information dynamics in complex systems, in addition to grounding abstract
concepts such as upward and downward causation,333The relation between
$\Phi$ID and causal emergence [28] can be found in a separate publication
[29]. and notions such as integrated information – as we discuss below.

#### A simple example of information decomposition with $\Phi$ID

As a first example of the kind of insight that Integrated Information
Decomposition can provide, let us focus on the decomposition of a variable’s
so-called ‘active information storage’ (AIS) [30], which is defined as the
TDMI of an individual part of the system (i.e. the mutual information between
the present of one variable, $X^{1}_{t}$, and its own future, $X^{1}_{t+1}$).
To decompose AIS, consider that in PID the mutual information of a single
source variable with the target is decomposed as the sum of redundancy (which
is information that each source has about the target) and that source’s unique
information. Similarly, in $\Phi\mathrm{ID}$ AIS is decomposed in terms of
redundancy and unique information, but now taking into account _both_ past and
future:

$\displaystyle\begin{split}\mathrm{AIS}(X^{1})=I(X^{1}_{t};X^{1}_{t+1})=\texttt{Red}\\!\rightarrow\\!\texttt{Red}\,+\,\texttt{Red}\\!\rightarrow\\!\texttt{Un}^{1}\,+\,\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Red}\,+\,\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{1}~{}.\end{split}$
(2)

Here, $\texttt{Red}\\!\rightarrow\\!\texttt{Red}$ corresponds to redundant
information in the past of both parts that is present in the future of both
parts; $\texttt{Red}\\!\rightarrow\\!\texttt{Un}^{1}$ is the redundant
information in the past that is eliminated from the second element and hence
is only conserved in $X^{1}_{t+1}$; and similarly for the remaining atoms.

Even with this simple example, $\Phi\mathrm{ID}$ already yields new insights
into the system’s information dynamics: note that, although
$X^{2}_{t},X^{2}_{t+1}$ are not in this mutual information,
$I(X^{1}_{t};X^{1}_{t+1})$ shares the
$\texttt{Red}\\!\rightarrow\\!\texttt{Red}$ term with
$I(X^{2}_{t};X^{2}_{t+1})$ by virtue of them being considered part of the same
multivariate stochastic process. Therefore, if one uses simple mutual
information as a measure of storage one may include information that is not
stored exclusively in a given variable, resulting in a ‘double-counting’ of
the $\texttt{Red}\\!\rightarrow\\!\texttt{Red}$ atom which can lead to
paradoxical conclusions – such as the sum of individual information storages
being greater than the total information flow (TDMI).

More generally, $\Phi\mathrm{ID}$ can be used decompose many other quantities
of interest for complex systems analysis (Fig. 3), and their decompositions
can help us both to understand existing measures and design new ones. In the
following sections we apply this line of reasoning and the $\Phi\mathrm{ID}$
framework to two prominent scenarios in complex systems analysis: the
assessment of causal interactions between system components, and the
quantification of system-wide integrated information.

Fig 3: Common information-theoretic measures decomposed into integrated
information atoms. Constituent $\Phi\mathrm{ID}$ atoms of active information
storage, transfer entropy (from $X_{t}^{1}$ to $X_{t+1}^{2}$), causal density
(sum of transfer entropies), and whole-minus-sum integrated information,
highlighted in blue. Dark blue indicates double-counting in causal density,
and red indicates a negative contribution of redundancy to integrated
information (see text for details).

### Theoretical implications

#### Different types of integration

Measures of integrated information, usually denoted by $\Phi$, aim to quantify
the degree to which a temporal evolution of a dynamical system depends on the
interdependencies between its parts [15],444The parts being chosen so as to
have the weakest overall informational link between them. see [19] for a
review. Integrated information measures have been applied widely, most notably
in the neuroscience of consciousness, but also to studies of diverse complex
systems [31, 32, 33]. In this section we investigate the concept of integrated
information through the lens of $\Phi\mathrm{ID}$.

The key insight that $\Phi$ID delivers is that there are multiple
qualitatively different ways in which a multivariate dynamical process can
integrate information – through different combinations of redundant, unique,
and synergistic effects. To illustrate this, let’s focus on the so-called
”whole-minus-sum” empirical integrated information metric [34], which for a
simple 2 component system is calculated as555There is only one possible
partitioning of a 2 component system, so here we don’t need to search for the
minimum information partition.

$\displaystyle\Phi^{\mathrm{WMS}}=I(\bm{X}_{t};\bm{X}_{t+1})-\sum_{i}I(X_{t}^{i};X_{t+1}^{i})~{}.$
(3)

which reflects a balance between the information contained within the whole
system ($I(\bm{X}_{t};\bm{X}_{t+1})$) and the information contained within the
parts ($I(X_{t}^{1};X_{t+1}^{1})$ and $I(X_{t}^{2};X_{t+1}^{2})$). We apply
this metric to the following elementary examples of 2 binary variables (Fig.
4):

  * •

A copy transfer system, in which $X_{t}^{1},X_{t}^{2},X_{t+1}^{1}$ are i.i.d.
fair coin flips, and $X_{t+1}^{2}=X_{t}^{1}$ (i.e. the information of
$X_{t}^{1}$ is copied to $X_{t+1}^{2}$).

  * •

The downward XOR, in which $X_{t}^{1},X_{t}^{2},X_{t+1}^{2}$ are i.i.d. fair
coin flips, and $X_{t+1}^{1}\equiv X_{t}^{1}+X_{t}^{2}~{}\mathrm{(mod~{}2)}$.

  * •

The parity-preserving random (PPR), in which $X_{t}^{1},X_{t}^{2}$ are i.i.d.
fair coin flips, and $X_{t+1}^{1}+X_{t+1}^{2}\equiv
X_{t}^{1}+X_{t}^{2}~{}\mathrm{(mod~{}2)}$ (i.e. $\bm{X}_{t+1}$ is a random
string of the same parity as $\bm{X}_{t}$).

A direct calculation shows that these three systems are ‘equally integrated’:
$\Phi^{\mathrm{WMS}}=1$ for all of them, which implies that the degree to
which the dynamics of the whole cannot be perfectly predicted from the parts
alone is equivalent [19, 34]. However, a more nuanced analysis using $\Phi$ID
reveals that these systems integrate information in qualitatively different
ways. In effect, the integration in the copy system is entirely due to
transfer dynamics ($\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{2}$); the
downward XOR integrates information by transforming synergistic into unique
information ($\texttt{Syn}\\!\rightarrow\\!\texttt{Un}^{1}$); and PPR due to
information synergistic in the past and future
($\texttt{Syn}\\!\rightarrow\\!\texttt{Syn}$). All the other $\Phi\mathrm{ID}$
atoms in each of these systems are zero (proofs in the Appendix).

Fig 4: Example systems of logic gates. While these three systems have the same
integrated information (measured with $\Phi^{\mathrm{WMS}}$), their
information dynamics are radically different. The idiosyncrasy of each type of
dynamic is captured by the $\Phi\mathrm{ID}$ formalism, which shows that in
each system there is only one non-zero atom, different for each system.

#### Measures of integrated information capture multiple $\Phi\mathrm{ID}$
atoms

Within the IIT literature, researchers have proposed multiple measures aimed
at quantifying to what extent a system is integrated as a whole, in terms of
its parts influencing each other’s evolution over time [35]. These measures,
though superficially similar, are known to behave inconsistently, for reasons
that are not always clear [19]. Here we use $\Phi$ID to dissect and compare
three existing measures of integrated information ($\Phi^{\mathrm{WMS}}$,
$\psi$, $\Phi_{G}$) and causal density (CD), bringing to light their
similarities and differences.666We provide definitions of each measure in the
Supplementary Material – for details see Section 2.2 of Ref. [19] and the
original references [15, 36, 37].

As a systematic exploration, one can determine which measures are sensitive to
which modes of information dynamics by calculating whether each measure is
zero, positive, or negative for a system consisting of only one particular
$\Phi$ID atom (Table 1; proofs in the Appendix). Strikingly, each proposed
measure of integration captures a distinct combination of $\Phi$ID atoms:
although generally most of them capture synergistic effects and avoid (or
penalise) redundant effects, they differ substantially.

The conclusion of this analysis is that these measures are not simply
different approximations of a single concept of integration, but rather they
are capturing intrinsically different aspects of the system’s information
dynamics. While aggregate measures like these can be empirically useful, one
should keep in mind that they are measuring combinations of different effects
within the system’s information dynamics. Echoing the conclusions of Ref.
[19]: these measures behave differently not only in practice, but also _in
principle_.

Table 1: Sensitivity of integrated information measures to $\Phi$ID atoms. For each measure, entries indicate whether the value is positive (+), negative (-) or 0 in a system in which the given $\Phi$ID atom is the only non-zero atom. Atom colour code taken from Fig. 1. $\Phi$ID atoms | Measures
---|---
| $\Phi^{\mathrm{WMS}}$ | $\Phi_{G}$ | $\psi$ | CD
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn | + | 0 | + | 0
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni | + | + | + | +
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed | + | + | + | +
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn | + | 0 | 0 | 0
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=SynCol, border-width=0pt]SyiSyn | + | 0 | 0 | 0
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni | 0 | 0 | 0 | 0
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un2Col, border-width=0pt]SyiUnj | + | + | 0 | +
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed | 0 | + | 0 | +
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=Un1Col, border-width=0pt]SyiUni | 0 | 0 | 0 | 0
[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed$\rightarrow$[width=0.55cm, padding-break-top=0.1em, padding-break-bottom=0.1em, rounded, background-color=RedCol, border-width=0pt]SyiRed | $-$ | 0 | 0 | 0

#### A $\Phi$ID account of information transfer

Most methods of statistical causal discovery use the conditional mutual
information777Or linear variants of it, to which our conclusions also apply.
as their main building block. Here we illustrate how one of such approaches,
transfer entropy (TE) [4], can be decomposed in terms of $\Phi$ID, showing
that it conflates qualitatively distinct effects in non-straightforward ways,
and fails to capture high-order modes of information flow. The TE from the
system’s first to its second element, defined as $\text{TE}(1\rightarrow
2):=I(X^{1}_{t};X^{2}_{t+1}|X^{2}_{t})$, can be decomposed via $\Phi$ID as

$\displaystyle\begin{split}\text{TE}(1\rightarrow
2)=\texttt{Syn}\\!\rightarrow\\!\texttt{Red}\,+\,\texttt{Syn}\\!\rightarrow\\!\texttt{Un}^{2}\,+\,\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Red}\,+\,\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{2}~{}.\end{split}$
(4)

Note that, of these, $\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{2}$ is the
only ‘genuine’ transfer term – all others correspond to redundant or
synergistic effects involving both variables in past or future, which do not
imply any kind of transfer phenomena. This complicates the interpretation of
TE as a fully general measure of information transfer.888Similar concerns
about TE have been raised in Ref. [20]. In contrast, $\Phi$ID can isolate the
part of the transfer entropy that corresponds to information transfer through
the $\texttt{Un}^{1}\\!\rightarrow\\!\texttt{Un}^{2}$ term.

Additionally, Eq. (4) implies that the atom
$\texttt{Syn}\\!\rightarrow\\!\texttt{Red}$ is accounted for in both
$\text{TE}(1\rightarrow 2)$ and $\text{TE}(2\rightarrow 1)$. This has an
important consequence: if one quantifies the total causal influence within the
system via adding up both TEs (a quantity known as unnormalised causal density
(uCD) [19]), this may overestimate the effective interdependencies by double-
counting this atom (Fig. 3). In fact, the double-counting of
$\texttt{Syn}\\!\rightarrow\\!\texttt{Red}$ implies that uCD can potentially
be even larger than TDMI. While this overestimation of uCD has been noted
before [37], $\Phi$ID not only reveals the precise reason for this
overestimation, but also provides a practical solution: one can correct uCD by
substracting the double-counted $\Phi$ID atom. This problem – and its solution
– may have important consequences in fields such as computational
neuroscience, where the total TE of a brain region is a popular metric of its
relevance for the brain’s hierarchical organization [38, 11].

The decomposition of uCD via $\Phi$ID reveals another important limitation of
traditional causal discovery methods: they do not account for modes of
information flow that involve synergy in the targets. Therefore, while these
methods account for possible interactions of source variables, they neglect
interactions in the targets – which are never considered jointly. As an
example, take the parity-preserving system in Fig. 4: this system has zero CD
and zero AIS, yet it clearly has dynamical structure (as picked up by
$\Phi^{\mathrm{WMS}}$). This is an example of information being carried purely
in high-order effects, in a way that common measures like AIS and TE are
unable to capture. More generally, we expect this to be particularly prevalent
in systems with distinct micro- and macro-scale behaviour [28, 29].

### Numerical examples

In this section we showcase three applications of $\Phi\mathrm{ID}$ to
simulated and real data, illustrating the capabilities of $\Phi\mathrm{ID}$ to
yield new insights and solve practical problems. As stated above, the
numerical calculation of $\Phi\mathrm{ID}$ atoms depends on a choice of
double-redundancy function – which, as in the case of PID, gives room to a
range of options (see e.g. Refs [21, 39, 40, 41]). In all examples below we
use a multi-target extension of the Common Change in Surprisal (CCS) measure
by Ince [39]; furthermore, we show that our all the results replicate with a
multi-target extension of Barrett’s Minimum Mutual Information (MMI) measure
[40] (see Supplementary Material).

#### Why whole-minus-sum $\Phi$ can be negative

$\Phi\mathrm{ID}$ can be further leveraged to explain certain behaviours of
integrated information and dynamical complexity measures. In particular,
$\Phi^{\mathrm{WMS}}$ can sometimes take negative values, which could suggest
a counter-intuitive notion of a ‘negatively integrated’ system. In fact, this
behaviour has been used as an argument to discard $\Phi^{\mathrm{WMS}}$ as a
suitable measure of integrated information [36, 37]. $\Phi\mathrm{ID}$ can
provide an explanation of this otherwise paradoxical behaviour, and furnishes
a simple solution.

By applying $\Phi\mathrm{ID}$ to the definition of $\Phi^{\mathrm{WMS}}$ in
Eq. (3), one finds that $\Phi^{\mathrm{WMS}}$ accounts for all the synergies
in the system, the unique information transferred between parts of the system
and, importantly, the negative of the
$\texttt{Red}\\!\rightarrow\\!\texttt{Red}$ atom (Fig. 3; see Supplementary
Material for details). The presence of this negative double-redundancy term
shows that $\Phi^{\mathrm{WMS}}$ can be negative in highly redundant systems,
in which $\texttt{Red}\\!\rightarrow\\!\texttt{Red}$ is larger than all other
atoms that constitute $\Phi^{\mathrm{WMS}}$. This is akin to Williams and
Beer’s [21] explanation of the negativity of the interaction information [42],
applied to multivariate processes. Based on this insight, one can formulate a
‘revised’ measure of integrated information, $\Phi^{\mathrm{R}}$, by adding
back the double-redundancy, which includes only synergistic and unique
transfer terms.

We computed $\Phi^{\mathrm{R}}$ numerically for a simple two-node
autoregressive (AR) system, mimicking the setting in Ref. [19]. The system
consists of two continuous variables with dynamics such that
$\bm{X}_{t+1}\sim\mathcal{N}(A\bm{X}_{t};\Sigma)$, with $A$ being a $2\times
2$ matrix with all entries set to 0.4, and $\Sigma$ a noise (or _innovations_)
covariance matrix with 1’s along the diagonal and a given noise correlation
($c$ in the notation of Ref. [19]) in the off-diagonal entries. We calculated
$\Phi^{\mathrm{WMS}}$ and $\Phi^{\mathrm{R}}$ with respect to the system’s
stationary distribution, which can be shown to be a multivariate Gaussian.
Plots of both quantities are shown in Fig. 5, and details of the computation
can be found in the Appendix.

Fig 5: Standard and revised $\Phi^{\mathrm{WMS}}$ in a two-component noisy
autoregressive system. As the noise injected to both components becomes more
correlated, $\Phi^{\mathrm{WMS}}$ drops below zero while $\Phi^{\mathrm{R}}$
remains positive.

As expected, $\Phi^{\mathrm{WMS}}$ drops below zero as synergy decreases and
redundancy increases with noise correlation. However, after adding back the
double-redundancy term, the revised version, $\Phi^{\mathrm{R}}$, tends to 0
for high noise correlation, which is more consistent with some of the other
measures highlighted in Ref. [19], e.g. CD and $\Phi^{*}$.

#### Information decomposition in simulated whole-brain activity

We next analysed simulated whole-brain activity via the well-known Dynamic
Mean Field (DMF) model introduced by Deco _et al._ [43]. This model represents
cortical regions as macroscopic neural fields, whose local dynamics are
described by a set of coupled differential equations. The DMF model
incorporates realistic aspects of neurophysiology such as synaptic dynamics
and membrane potential [44, 45, 46], and is informed by a network of
anatomical connections obtained e.g. from diffusion tensor imaging (DTI) -
while having the advantage of being free from physiological noise confounds.
An additional biophysical haemodynamic model [47] enables the firing rates
generated by the DMF model to be transformed into BOLD signals similar to
resting-state fMRI data from humans, and that have been subject of study in
applications of the DMF to model the neural effects of sleep [48], anaesthesia
[49], and psychedelic drugs [50].

Fig 6: Measures of integrated information in a whole-brain computational
model. a) Schematic diagram of the Dynamic Mean-Field (DMF) model [43, 49],
which combines a DTI-based connectome and a whole-brain parcellation to
simulate realistic BOLD signals. b) As the global coupling parameter $G$ is
increased, the mean firing rate exhibits a sharp increase at approximately
$G=2$. Importantly, $\Phi^{\mathrm{WMS}}$ shows a downward peak, suggesting a
conceptually problematic negative value of integration – while the revised
measure $\Phi^{\mathrm{R}}$ shows a strong positive peak.

We simulate the DMF equations using a DTI-based connectome obtained from the
public Human Connectome Project data [51] using the same model settings as
Herzog _et al._ [50], and compute $\Phi^{\text{WMS}}$ and $\Phi^{\text{R}}$
for all pairs of brain regions.999The choice of analysing pairs of regions is
only for convenience, as the theory is defined to systems of any size. These
values were calculated for varying values of a global coupling parameter
(denoted by $G$) and the resulting average values (over all pairs of brain
regions) were then analysed. The details of the model, the simulation
procedure, and the computation of integrated information measures can be found
in the Appendix.

As shown in Fig. 6, for values of $G$ close to 2 the mean firing rate of the
model increases sharply, reminiscent of a phase transition. At this point
$\Phi^{\mathrm{WMS}}$ shows a marked decrease, suggesting that the system is
least integrated in the transition region. In fact, the value of
$\Phi^{\mathrm{WMS}}$ is often less than zero, which would correspond to the
conceptually problematic notion of a system that is ’negatively integrated.’
Crucially, however, when the double-counting of redundancy is corrected and
$\Phi^{\mathrm{R}}$ is used instead, a completely different picture appears:
integration (understood as synergy plus transfer) is always positive, and it
strongly _increases_ and peaks in the transition region. This result aligns
well with prior literature [52, 50] showing that the point $G=2$ corresponds
to the model’s optimal fit to data from awake subjects, and that a high level
of integration is required for the normal operation of the brain in healthy,
conscious individuals [14].

The strong discrepancy between $\Phi^{\mathrm{WMS}}$ and $\Phi^{\mathrm{R}}$
confirms that the concerns regarding the conflation of multiple information
effects highlighted throughout this article are not a mere theoretical issue,
but can trigger misleading interpretations in the analysis of neuroscientific
data. Hence, this example illustrates the capability of $\Phi\mathrm{ID}$ to
disambiguate between qualitatively different dynamical phenomena in time
series data.

####  $\Phi$ID sheds new light on empirical results

To provide an empirical demonstration of the capabilities of $\Phi$ID, we used
it to study the dynamical relationship that exists between heart rate and
respiration in healthy human subjects. This choice was motivated by the well-
known influence of respiration on heart rate, which can be captured in terms
of transfer entropy between respiratory volume and heart rate time series [53,
54, 55]. Therefore, we sought to investigate how the $\Phi$ID framework could
be used to decompose this effect into its constituent informational elements.

For this purpose, we analysed the Fantasia database [56], an openly available
dataset that contains data from 40 healthy subjects while watching the Disney
movie ‘Fantasia.’ Following the preprocessing pipeline outlined in Ref. [53],
we extracted synchronised time series for inter-beat intervals from the ECG
timeseries, and detrended the respiratory volume. Using the resulting data, we
calculated both the transfer entropy from heart to breath and from breath to
heart, and then proceeded to decompose these quantities in terms of their
$\Phi$ID constituents (Fig. 7).

Fig 7: Decomposition of information transfer between heart rate and
respiratory volume. Transfer entropy decomposed into its four constituent
$\Phi\mathrm{ID}$ atoms, calculated in both directions (heart to breath [H2B]
and breath to heart [B2H]).

As expected based on previous work, the transfer entropy from breath to heart
was significantly higher than from heart to breath. Crucially, our analysis
revealed that this effect is driven by two distinct $\Phi$ID atoms, out of the
four that comprise transfer entropy. The TE result is dominated by the
$\texttt{Syn}\\!\rightarrow\\!\texttt{Un}$ atom, while the transfer atom
itself ($\texttt{Un}\\!\rightarrow\\!\texttt{Un}$) shows no significant
differences. Importantly, however, $\texttt{Un}\\!\rightarrow\\!\texttt{Red}$
shows a significant effect in the _opposite_ direction to the main TE result.
The standard TE analysis is unable to resolve these modes of information
dynamics, and thus misses the heart’s unique contribution to the heart-breath
joint dynamics.

In summary, $\Phi$ID shows that the effect seen in the transfer entropy
towards the heart is not transfer but synergistic, and that there is a smaller
unique effect originating at the heart that is overshadowed by the
former.101010The study of the physiological implications of these findings
will be investigated in a separate publication. These results illustrate the
type of advantages that $\Phi$ID can bring beyond standard transfer entropy
and Granger causality analyses.

## Discussion

This paper introduces $\Phi$ID as a formal framework to study high-order
interactions in the dynamics of multivariate complex systems. By bringing
together aspects of integrated information theory (IIT) and partial
information decomposition (PID), the $\Phi$ID framework allows us to decompose
multivariate information flow into interpretable, distinct parts. This
decomposition brings two important outcomes. First, it allows us to better
understand and refine existing metrics of information exchange and dynamical
complexity. Second, it enables systematic analyses of previously unexplored
modes of information dynamics, which are not captured by previous analysis
methods.

### Towards multi-dimensional accounts of dynamical complexity

$\Phi$ID provides principled tools to inspect existing measures of information
dynamics and overcome some of their shortcomings. In particular, we have shown
that both the widely used transfer entropy and what is referred to as
‘integrated information’ in the context of IIT are in fact aggregates of
several distinct information effects, typically including transfer and synergy
phenomena. In addition, our analysis shows that different measures of
integrated information actually capture different $\Phi$ID atoms in various
proportions, which provides a formal explanation for the heterogeneity among
existing measures reported in Ref. [19].

Supporting our theoretical results, we showed in empirical physiological data
that significant differences in transfer entropy can be observed even in the
absence of genuine information transfer phenomena, highlighting the practical
relevance of our framework. Likewise, our analysis based on whole-brain
modelling showed how the conflation of fundamentally distinct information
phenomena in existing measures of integrated information can introduce
substantial confusion in the results and interpretation of neuroscientific
analysis. $\Phi$ID provides tools to identify these problems, and also to fix
them, by tailoring measures that target the specific kinds of information
dynamics one wishes to analyse.

As well as providing a new taxonomy of information dynamics phenomena,
$\Phi$ID establishes that there are fundamental limitations to any purported
all-encompassing scalar measure of dynamical complexity, in line with Feldman
and Crutchfield [57]. The space of possible complex dynamics is,
unsurprisingly, vast and complex, and while scalar measures might still have
great practical value in specific contexts,111111For example, measures that
accurately discriminate between neural configurations corresponding to
conscious and unconscious states in a particular experimental paradigm [58].
$\Phi$ID clarifies that a general theory of complex systems (biological or
otherwise) cannot be reduced to a single, one-size-fits-all measure, but
rather needs to embrace this richness.

### Limitations and future extensions

$\Phi$ID is a general tool to decompose multivariate mutual information, and
the nature of the resulting decomposition critically depends on how the
underlying joint distribution has been constructed. In particular, note that
the $\Phi\mathrm{ID}$ framework depends only on a joint probability
distribution $p(\bm{X}_{t},\bm{X}_{t+1})$, and thus its results can be
interpreted as causality in the Pearl or Granger sense, depending on whether
the distribution comes from intervention or observation, respectively. If the
distribution is built on observational data then the decomposition generally
should be understood in the Granger-causal sense (i.e. as referring to
predictive ability). Similarly, if the conditional distribution
$p(\bm{X}_{t+1}|\bm{X}_{t})$ is equivalent to a do() distribution in Pearl’s
sense [5], and the system satisfies the faithfulness and causal Markov
conditions, then the results of $\Phi$ID are to be interpreted in a
counterfactual causal sense. In either case, the formalism developed here
applies directly.

Naturally, $\Phi\mathrm{ID}$ inherits some characteristics of PID. In
particular, $\Phi\mathrm{ID}$ specifies the relationships between information
atoms, but does not prescribe a particular functional form for them. To
compute numerical values of $\Phi\mathrm{ID}$ atoms one needs a double-
redundancy function, a multi-target extension of PID’s redundancy function. In
PID, several distinct redundancy functions have been proposed, and while they
have been shown to agree in various scenarios [59], they may differ in other
cases and there is not yet a consensus on one that is universally preferable
[60]. The formulation of multiple double-redundancy functions, and a thorough
comparison in simulated systems is an important line for future work.

Finally, it is important to remark that the framework presented in this paper
focuses on decomposing the mutual information between two time points. While
this captures all the information carried from past to future in Markovian
systems, it might miss relevant phenomena in systems with non-Markovian
dynamics – which typically arise in experimental data of scenarios with many
non-observable variables. As an important extension, future work should
investigate the effect of unobserved variables on the proposed decomposition,
which could be done e.g. leveraging Taken’s embedding theorem [61, 62] or
other methods [63].

## Methods

This section establishes the mathematical bases of our framework. The aim is
to build a decomposition of the TDMI, as defined in Eq. (1), that
differentiates the role of each source and each target (possibly multivariate)
variable – hence accounting for both cause (forward) and effect (backward)
information simultaneously. To do this, our proposed decomposition brings
together the partitions enabled by a forward PID (where variables at time $t$
and $t+1$ are sources and targets, respectively) and a backward PID (where the
assignment of sources and targets is reversed), as illustrated in Figure 1a.
By doing this, we overcome PID’s limitation of having only one single target
variable and formulate a multi-target information decomposition.

Note that throughout this section we switch from the Red/Un/Syn notation above
to the more standard (and more general) ‘curly bracket’ notation introduced by
Williams and Beer [21].

### Double-redundancy lattice

Let us start by reviewing the construction of the redundancy lattice that is
employed in PID to formalise our intuitive understanding of redundancy [21].
This lattice is built over the set $\mathcal{A}$, which for the case of two
time series can be expressed as

$\mathcal{A}:=\\{\\{1\\},\\{2\\},\\{1,2\\},\\{\\{1\\},\\{2\\}\\}\\},$ (5)

which correspond to all the sets of subsets of $\\{1,2\\}$ where no element is
contained in another.121212For a case of $N$ variables, then $\mathcal{A}$ is
the set of antichains on the lattice $(\mathcal{P}(\\{1,...,N\\}),\subseteq)$,
discussed in Ref. [21]. We focus on the bivariate case for clarity, although
the $\Phi$ID formalism developed here can be applied to any $N$. Then, the
lattice is built using a natural (partial) order relationship that exists
between the elements of $\mathcal{A}$ [21]: for
$\bm{\alpha},\bm{\beta}\in\mathcal{A}$, one says that

$\bm{\alpha}\preceq\bm{\beta}\quad\text{if}\quad\text{for all
}b\in\bm{\beta}\text{ there exists }a\in\bm{\alpha}\text{ such that }a\subset
b.$ (6)

The lattice that encodes the relationship $\preceq$ is known as the redundancy
lattice (Fig. 1b), and guides the construction of the four terms in the PID.

Our first step in building the foundations of $\Phi$ID is to build a product
lattice over $\mathcal{A}\times\mathcal{A}$, in order to extend the notion of
redundancy from PID to the case of multiple source and target variables (here
$X_{t}^{1}$, $X_{t}^{2}$ and $X_{t+1}^{1}$, $X_{t+1}^{2}$ respectively).
Intuitively, $\Phi\mathrm{ID}$ is the ‘product’ of two complementary single-
target PIDs, one decomposing the information carried by the past about the
future, and the other decomposing the information carried by the future about
the past (Fig. 1a). To formalise this intuition, we extend Williams and Beer’s
[21] notation, and denote sets of sources and targets using their indices only
with an arrow going from past to future. Hence, the nodes of the product
lattice are denoted as $\bm{\alpha}\rightarrow\bm{\beta}$ for
$\bm{\alpha},\bm{\beta}\in\mathcal{A}$.

A natural partial ordering relationship can be establish over the product
lattice as follows:

$\bm{\alpha}\rightarrow\bm{\beta}\preceq\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}\quad\text{iff}\quad\bm{\alpha}\preceq\bm{\alpha}^{\prime}\>\>\text{and}\>\>\bm{\beta}\preceq\bm{\beta}^{\prime}.$
(7)

This relationship establishes a lattice structure131313A proof of this is
provided in the Appendix., which for the case of a bipartite system consists
of 16 nodes (Fig. 1c).

### Redundancies and atoms

The other ingredient in the PID recipe – besides the redundancy lattice – is a
_redundancy function_ , $I_{\cap}$, that quantifies the amount of
‘overlapping’ information about the target that is common to a set of sources
$\bm{\alpha}\in\mathcal{A}$ [21]. The redundancy function in a PID,
$I_{\cap}^{\bm{\alpha}}$, encompasses the following terms in the case of two
source variables:

  * •

$I_{\cap}^{\\{1\\}\\{2\\}}$ is the information about the target that is in
either source,

  * •

$I_{\cap}^{\\{i\\}}$ is the information in source $i$, and

  * •

$I_{\cap}^{\\{12\\}}$ is the information that is in both sources when
considered together.

This subsection extends the notion of overlapping information to the multi-
target setting.

For a given $\bm{\alpha}\rightarrow\bm{\beta}\in\mathcal{A}\times\mathcal{A}$,
the overlapping information that is common to sources $\bm{\alpha}$ and can be
seen in targets $\bm{\beta}$ is denoted as
$I_{\cap}^{\bm{\alpha}\rightarrow\bm{\beta}}$ and referred to as the _double-
redundancy function_. In the following, we assume that the double-redundancy
function satisfies two axioms:

  * •

Axiom 1 (compatibility): if $\bm{\alpha}=\\{\alpha_{1},\dots,\alpha_{J}\\}$
and $\bm{\beta}=\\{\beta_{1},\dots,\beta_{K}\\}$ with
$\bm{\alpha},\bm{\beta}\in\mathcal{A}$ and $\alpha_{j},\beta_{k}$ non-empty
subsets of $\\{1,\dots,N\\}$, then the following cases can be reduced to the
redundancy of PID or the mutual information:141414Here we use the shorthand
notation $\bm{X}_{t}^{\alpha}:=(X_{t}^{i_{1}},\dots,X_{t}^{i_{K}})$ for
$\alpha=\\{i_{1},\dots,i_{K}\\}$.

$I_{\cap}^{\bm{\alpha}\rightarrow\bm{\beta}}=\begin{cases}\texttt{Red}(\bm{X}_{t}^{\alpha_{1}},\dots,\bm{X}_{t}^{\alpha_{J}};\bm{X}_{t^{\prime}}^{\beta_{1}})&\text{if}\quad
K=1,\\\
\texttt{Red}(\bm{X}_{t^{\prime}}^{\beta_{1}},\dots,\bm{X}_{t^{\prime}}^{\beta_{K}};\bm{X}_{t}^{\alpha_{1}})&\text{if}\quad
J=1,\\\
I(\bm{X}_{t}^{\alpha_{1}};\bm{X}_{t^{\prime}}^{\beta_{1}})&\text{if}\quad
J=K=1.\end{cases}$

  * •

Axiom 2 (partial ordering): if
$\bm{\alpha}\rightarrow\bm{\beta}\preceq\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}$
then $I_{\cap}^{\bm{\alpha}\rightarrow\bm{\beta}}\leq
I_{\cap}^{\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}}$.

Intuitively, the first axiom guarantees that any double-redundancy function in
$\Phi$ID reduces to a PID-type redudancy function when evaluated in certain
atoms, and the second encapsulates the basic desideratum of the double-
redudancy being in agreement with the partial ordering given by the product
lattice.

By exploiting these two axioms, one can define ‘atoms’ that belong to each of
the nodes via the Moebius inversion formula. Concretely, the $\Phi$ID atoms
$I_{\partial}^{\bm{\alpha}\rightarrow\bm{\beta}}$ are defined as the
quantities that guarantee the following condition for all
$\bm{\alpha}\rightarrow\bm{\beta}\in\mathcal{A}\times\mathcal{A}$:

$\displaystyle
I_{\cap}^{\bm{\alpha}\rightarrow\bm{\beta}}=\sum_{\begin{subarray}{c}\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}\preceq\bm{\alpha}\rightarrow\bm{\beta}\end{subarray}}I_{\partial}^{\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}}~{}.$
(8)

In other words, $I_{\partial}^{\bm{\alpha}\rightarrow\bm{\beta}}$ corresponds
to the information contained in node $\bm{\alpha}\rightarrow\bm{\beta}$ and
not in any node below it in the lattice. These are analogues to the redundant,
unique, and synergistic atoms in standard PID, but using the product lattice
as a scaffold. By inverting this relationship, one can find a recursive
expression for calculating $I_{\partial}$ as

$\displaystyle
I_{\partial}^{\bm{\alpha}\rightarrow\bm{\beta}}=I_{\cap}^{\bm{\alpha}\rightarrow\bm{\beta}}-\sum_{\begin{subarray}{c}\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}\prec\bm{\alpha}\rightarrow\bm{\beta}\end{subarray}}I_{\partial}^{\bm{\alpha}^{\prime}\rightarrow\bm{\beta}^{\prime}}~{}.$
(9)

With all the tools at hand, we can deliver the promised decomposition of the
TDMI in terms of atoms of integrated information, as established in the next
definition.

######  Definition 1.

The Integrated Information Decomposition ($\Phi\mathrm{ID}$) of a system with
Markovian dynamics is the collection of atoms $I_{\partial}$ defined from the
redundancies $I_{\cap}$ via Eq. (9), which satisfy

$\displaystyle\mathrm{TDMI}=I(\bm{X}_{t};\bm{X}_{t^{\prime}})=\sum_{\begin{subarray}{c}\bm{\alpha},\bm{\beta}\in\mathcal{A}\end{subarray}}I_{\partial}^{\bm{\alpha}\rightarrow\bm{\beta}}~{}.$
(10)

In this way, the $\Phi$ID of two time series gives 16 atoms that correspond to
the lattice shown in Figure 1c, which are computed via a linear transformation
over the 16 redundancies. Importantly, Axioms 1 and 2 allow us to compute all
the $I_{\cap}$ terms once a single-target PID redundancy function
$\texttt{Red}(\cdot)$ has been chosen with the sole exception of
$I_{\cap}^{\\{1\\}\\{2\\}\rightarrow\\{1\\}\\{2\\}}$.151515This can be
verified directly by studying the coefficients of the linear system of
equations that relate redundancies and atoms. All this is summarised in the
following result.

######  Proposition 1 (15-for-free).

Axioms 1 and 2 provide unique values for the 16 atoms of the product lattice
after one defines (i) a single-target redundancy function
$\texttt{Red}(\cdot)$, and (ii) an expression for
$I_{\partial}^{\\{1\\}\\{2\\}\rightarrow\\{1\\}\\{2\\}}$.

Therefore, in the same way as in PID the definition of $\texttt{Red}(\cdot)$
gives 3 other terms (unique and synergy) as side-product, Proposition 1 shows
that in $\Phi\text{ID}$ the addition of the double-redundancy function
$I_{\partial}^{\\{1\\}\\{2\\}\rightarrow\\{1\\}\\{2\\}}$ gives the 15 other
terms for free. In the Supplementary Material we describe $\Phi\mathrm{ID}$
extensions of two common PID redundancy functions (Ince’s CCS [39] and
Barrett’s MMI [40] measures), which we use for all numerical examples in this
paper.

## Supporting information

##### S1 Appendix.

Proofs and technical details.

## Acknowledgments

The authors thank Julian Sutherland for valuable discussions. P.A.M.M. was
supported by the Wellcome Trust (grant number 210920/Z/18/Z). F.E.R. was
supported by the Ad Astra Chandaria Foundation, and by the European Union’s
H2020 research and innovation programme under the Marie Skłodowska-Curie grant
agreement No. 702981. A.I.L. is supported by a Gates Cambridge Scholarship.
A.K.S. and A.B.B. acknowledge support from the Dr Mortimer and Theresa Sackler
Foundation. A.K.S. also acknowledges support from the Canadian Institute for
Advanced Research (CIFAR) Program on Brain, Mind, and Consciousness.

## References

  * 1. Kelso JS.  Dynamic Patterns: The Self-organization of Brain and Behavior.  MIT Press; 1995.
  * 2. Runge J, Bathiany S, Bollt E, Camps-Valls G, Coumou D, Deyle E, et al.  Inferring causation from time series in Earth system sciences.  Nature Communications. 2019;10(1):2553.
  * 3. Dosi G, Roventini A.  More is different … and complex! The case for agent-based macroeconomics.  J Evol Econ. 2019;29(1).
  * 4. Bressler SL, Seth AK.  Wiener–Granger causality: A well established methodology.  NeuroImage. 2011;58(2):323–329.
  * 5. Pearl J, Mackenzie D.  The Book of Why: The New Science of Cause and Effect.  Basic Books; 2018.
  * 6. Chan TE, Stumpf MP, Babtie AC.  Gene regulatory network inference from single-cell data using multivariate information measures.  Cell systems. 2017;5(3):251–267.
  * 7. Scagliarini T, Faes L, Marinazzo D, Stramaglia S, Mantegna RN.  Synergistic information transfer in the global system of financial markets.  Entropy. 2020;22(9):1000.
  * 8. Rosas FE, Mediano PA, Gastpar M, Jensen HJ.  Quantifying high-order interdependencies via multivariate extensions of the mutual information.  Physical Review E. 2019;100(3):032305.  doi:10.1103/PhysRevE.100.032305.
  * 9. Wibral M, Phillips WA, Lizier JT, Priesemann V.  Partial information decomposition as a unified approach to the characterization and design of neural goal functions.  BMC neuroscience. 2015;16(1):1–2.
  * 10. Luppi AI, Mediano PA, Rosas FE, Holland N, Fryer TD, O’Brien JT, et al.  A synergistic core for human brain evolution and cognition.  bioRxiv. 2020;.
  * 11. Luppi AI, Mediano PA, Rosas FE, Allanson J, Pickard JD, Carhart-Harris RL, et al.  A Synergistic Workspace for Human Consciousness Revealed by Integrated Information Decomposition.  bioRxiv. 2020;.
  * 12. Grassberger P.  Toward a quantitative theory of self-generated complexity.  International Journal of Theoretical Physics. 1986;25(9):907–938.
  * 13. Crutchfield JP, Feldman DP.  Regularities unseen, randomness observed: Levels of entropy convergence.  Chaos: An Interdisciplinary Journal of Nonlinear Science. 2003;13(1):25–54.
  * 14. Tononi G, Sporns O, Edelman G.  A measure for brain complexity: Relating functional segregation and integration in the nervous system.  Proceedings of the National Academy of Sciences. 1994;91(11):5033–5037.
  * 15. Balduzzi D, Tononi G.  Integrated information in discrete dynamical systems: Motivation and theoretical framework.  PLoS Computational Biology. 2008;4(6):1–18.
  * 16. Oizumi M, Albantakis L, Tononi G.  From the phenomenology to the mechanisms of consciousness: Integrated information theory 3.0.  PLoS Computational Biology. 2014;10(5).
  * 17. Seth AK, Barrett AB, Barnett L.  Causal density and integrated information as measures of conscious level.  Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2011;369(1952):3748–3767.
  * 18. van Walsum AMvC, Pijnenburg Y, Berendse H, van Dijk B, Knol D, Scheltens P, et al.  A neural complexity measure applied to MEG data in Alzheimer’s disease.  Clinical Neurophysiology. 2003;114(6):1034–1040.
  * 19. Mediano P, Seth A, Barrett A.  Measuring integrated information: Comparison of candidate measures in theory and simulation.  Entropy. 2019;21(1):17.
  * 20. James RG, Barnett N, Crutchfield JP.  Information flows? A critique of transfer entropies.  Physical Review Letters. 2016;116(23):238701.
  * 21. Williams PL, Beer RD. Nonnegative decomposition of multivariate information; 2010\.
  * 22. Finn C, Lizier JT.  Quantifying information modification in cellular automata using pointwise partial information decomposition.  In: Artificial Life Conference Proceedings. MIT Press; 2018. p. 386–387.
  * 23. Rosas F, Mediano P, Ugarte M, Jensen H.  An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems.  Entropy. 2018;20(10):793.
  * 24. Beer RD, Williams PL.  Information processing and dynamics in minimally cognitive agents.  Cognitive Science. 2015;39(1):1–38.  doi:10.1111/cogs.12142.
  * 25. Tax TMS, Mediano PAM, Shanahan M.  The Partial Information Decomposition of Generative Neural Network Models.  Entropy. 2017;19(9).  doi:10.3390/e19090474.
  * 26. Varley TF.  Intersectional synergies: Untangling irreducible effects of intersecting identities via information decomposition.  arXiv:210610338. 2021;.
  * 27. Cang Z, Nie Q.  Inferring spatial and signaling relationships between cells from single cell transcriptomic data.  Nature Communications. 2020;11(1):1–13.  doi:10.1038/s41467-020-15968-5.
  * 28. Seth AK.  Measuring autonomy and emergence via Granger causality.  Artificial Life. 2010;16(2):179–196.
  * 29. Rosas FE, Mediano PA, Jensen HJ, Seth AK, Barrett AB, Carhart-Harris RL, et al.. Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data; 2020.
  * 30. Lizier J.  The Local Information Dynamics of Distributed Computation in Complex Systems.  University of Sydney; 2010.
  * 31. Mediano PAM, Farah JC, Shanahan M.  Integrated Information and Metastability in Systems of Coupled Oscillators.  ArXiv e-prints. 2016;.
  * 32. Aguilera M, Di Paolo E.  Integrated information in the thermodynamic limit.  Neural Networks. 2019;.
  * 33. Mediano PA, Rosas FE, Farah JC, Shanahan M, Bor D, Barrett AB.  Integrated information as a common signature of dynamical and information-processing complexity.  arXiv preprint arXiv:210610211. 2021;.
  * 34. Barrett AB, Seth AK.  Practical measures of integrated information for time-series data.  PLoS Computational Biology. 2011;7(1):1–18.
  * 35. Tegmark M.  Improved measures of integrated information.  PLoS Computational Biology. 2016;12(11):e1005123.  doi:10.1371/journal.pcbi.1005123.
  * 36. Griffith V. A principled infotheoretic $\varphi$-like measure; 2014.
  * 37. Oizumi M, Tsuchiya N, Amari Si.  Unified framework for information integration based on information geometry.  Proceedings of the National Academy of Sciences. 2016;113(51):14817–14822.
  * 38. Deco G, Vidaurre D, Kringelbach ML.  Revisiting the global workspace orchestrating the hierarchical organization of the human brain.  Nature Human Behaviour. 2021;5(4):497–511.  doi:10.1038/s41562-020-01003-6.
  * 39. Ince RAA.  Measuring Multivariate Redundant Information with Pointwise Common Change in Surprisal.  Entropy. 2017;19(7).  doi:10.3390/e19070318.
  * 40. Barrett AB.  Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems.  Phys Rev E. 2015;91:052802.
  * 41. Finn C, Lizier JT.  Pointwise partial information decomposition using the specificity and ambiguity lattices.  Entropy. 2018;20(4):297.  doi:10.3390/e20040297.
  * 42. McGill WJ.  Multivariate information transmission.  Psychometrika. 1954;19(2):97–116.
  * 43. Deco G, Ponce-Alvarez A, Hagmann P, Romani GL, Mantini D, Corbetta M.  How local excitation-inhibition ratio impacts the whole brain dynamics.  Journal of Neuroscience. 2014;34(23):7886–7898.
  * 44. Deco G, Jirsa VK.  Ongoing cortical activity at rest: Criticality, multistability, and ghost attractors.  Journal of Neuroscience. 2012;32(10):3366–3375.  doi:10.1523/JNEUROSCI.2523-11.2012.
  * 45. Deco G, Ponce-Alvarez A, Mantini D, Romani GL, Hagmann P, Corbetta M.  Resting-state functional connectivity emerges from structurally and dynamically shaped slow linear fluctuations.  Journal of Neuroscience. 2013;33(27):11239–11252.  doi:10.1523/JNEUROSCI.1091-13.2013.
  * 46. Hansen EC, Battaglia D, Spiegler A, Deco G, Jirsa VK.  Functional connectivity dynamics: Modeling the switching behavior of the resting state.  NeuroImage. 2015;105:525–535.  doi:10.1016/j.neuroimage.2014.11.001.
  * 47. Friston KJ, Mechelli A, Turner R, Price CJ.  Nonlinear responses in fMRI: The Balloon model, Volterra kernels, and other hemodynamics.  NeuroImage. 2000;12(4):466–477.  doi:10.1006/nimg.2000.0630.
  * 48. Ipiña IP, Kehoe PD, Kringelbach M, Laufs H, Ibañez A, Deco G, et al.  Modeling regional changes in dynamic stability during sleep and wakefulness.  NeuroImage. 2020;215:116833.  doi:10.1016/j.neuroimage.2020.116833.
  * 49. Luppi AI, Mediano PA, Rosas FE, Allanson J, Pickard JD, Williams GB, et al.  Paths to oblivion: Common neural mechanisms of anaesthesia and disorders of consciousness.  bioRxiv. 2021;doi:10.1101/2021.02.14.431140.
  * 50. Herzog R, Mediano PAM, Rosas FE, Carhart-Harris R, Sanz Y, Tagliazucchi E, et al.  A mechanistic model of the neural entropy increase elicited by psychedelic drugs.  Scientific Reports. 2020;10(17725).
  * 51. Glasser MF, Sotiropoulos SN, Wilson JA, Coalson TS, Fischl B, Andersson JL, et al.  The minimal preprocessing pipelines for the Human Connectome Project.  NeuroImage. 2013;80:105–124.
  * 52. Deco G, Cruzat J, Cabral J, Knudsen GM, Carhart-Harris RL, Whybrow PC, et al.  Whole-brain multimodal neuroimaging model using serotonin receptor maps explains non-linear functional effects of LSD.  Current Biology. 2018; p. 1–10.
  * 53. Nemati S, Edwards BA, Lee J, Pittman-Polletta B, Butler JP, Malhotra A.  Respiration and heart rate complexity: effects of age and gender assessed by band-limited transfer entropy.  Respiratory Physiology & Neurobiology. 2013;189(1):27–33.
  * 54. Pini N, Lucchini M, Fifer WP, Burtchen N, Signorini MG, et al.  Lagged transfer entropy analysis to investigate cardiorespiratory regulation in newborns during sleep.  In: BIOSTEC-BIOSIGNALS. SCITEPRESS; 2019. p. 139–146.
  * 55. de Abreu RM, Catai AM, Cairo B, Rehder-Santos P, da Silva CD, Signini ÉDF, et al.  A transfer entropy approach for the assessment of the impact of inspiratory muscle training on the cardiorespiratory coupling of amateur cyclists.  Frontiers in Physiology. 2020;11.
  * 56. Iyengar N, Peng C, Morin R, Goldberger AL, Lipsitz LA.  Age-related alterations in the fractal scaling of cardiac interbeat interval dynamics.  American Journal of Physiology – Regulatory, Integrative and Comparative Physiology. 1996;271(4):R1078–R1084.
  * 57. Feldman DP, Crutchfield JP.  Measures of statistical complexity: Why?  Physics Letters A. 1998;238(4-5):244–252.
  * 58. Casali AG, Gosseries O, Rosanova M, Boly M, Sarasso S, Casali KR, et al.  A theoretically based index of consciousness independent of sensory processing and behavior.  Science Translational Medicine. 2013;5(198):198ra105.
  * 59. Rosas FE, Mediano PA, Rassouli B, Barrett AB.  An operational information decomposition via synergistic disclosure.  Journal of Physics A: Mathematical and Theoretical. 2020;53(48):485001.  doi:10.1088/1751-8121/abb723.
  * 60. James RG, Emenheiser J, Crutchfield JP.  Unique information via dependency constraints.  Journal of Physics A: Mathematical and Theoretical. 2018;52(1):014002.
  * 61. Takens F.  Detecting strange attractors in turbulence.  In: Dynamical Systems and Turbulence. Springer; 1981. p. 366–381.
  * 62. Cliff OM, Prokopenko M, Fitch R.  An information criterion for inferring coupling of distributed dynamical systems.  Frontiers in Robotics and AI. 2016;3:71.
  * 63. Wilting J, Priesemann V.  Inferring collective dynamical states from widely unobserved systems.  Nature Communications. 2018;9(1):1–7.