11institutetext: School of Electrical and Computer Engineering, University of
Tehran, College of Engineering, Tehran, Iran
11email<EMAIL_ADDRESS>22institutetext: Department of
Anaesthesia, Harvard Medical School, Boston, Massachusetts, USA
**footnotetext: Abdolmahdi Bagheri and Mahdi Dehshiri contributed equally.

# Algorithmic Identification of Essential Exogenous Nodes for Causal
Sufficiency in Brain Networks

Abdolmahdi Bagheri∗(🖂) 11 Mahdi Dehshiri∗ 11 Babak Nadjar Araabi 11 Alireza
Akhondi-Asl 22

###### Abstract

In the investigation of any causal mechanisms, such as the brain’s causal
networks, the assumption of causal sufficiency plays a critical role. Notably,
neglecting this assumption can result in significant errors, a fact that is
often disregarded in the causal analysis of brain networks. In this study, we
propose an algorithmic identification approach for determining essential
exogenous nodes that satisfy the critical need for causal sufficiency to
adhere to it in such inquiries. Our approach consists of three main steps:
First, by capturing the essence of the Peter-Clark (PC) algorithm, we conduct
independence tests for pairs of regions within a network, as well as for the
same pairs conditioned on nodes from other networks. Next, we distinguish
candidate confounders by analyzing the differences between the conditional and
unconditional results, using the Kolmogorov-Smirnov test. Subsequently, we
utilize Non-Factorized identifiable Variational Autoencoders (NF-iVAE) along
with the Correlation Coefficient index (CCI) metric to identify the
confounding variables within these candidate nodes. Applying our method to the
Human Connectome Project’s (HCP) movie-watching task data, we demonstrate that
while interactions exist between dorsal and ventral regions, only dorsal
regions serve as confounders for the visual networks, and vice versa. These
findings align consistently with those resulting from the neuroscientific
perspective. Finally, we show the reliability of our results by testing 30
independent runs for NF-iVAE initialization.

###### Keywords:

Brain networks Causal sufficiency assumption, Causal discovery, Non-Factorized
identifiable Variational Autoencoders (NF-iVAE), Human Connectome Project
(HCP).

##  1 Introduction

Probing a causal model and extracting its links necessitates upholding certain
prerequisites, among them, the causal sufficiency assumption is the primary
one [2]. With various definitions for this assumption presented in [1], it
essentially asserts that in exploring causal relations, all potential
confounders or their ancestors must be accounted for the model under
investigation. Violating this assumption often results in extracting spurious
connections, suggesting causal links where none exist. Despite the importance
of this assumption, it is often overlooked in studying causality in brain
networks. In these studies, some focused on only one network such as the
visual, working memory, or attention network, as exemplified by those in [3,
4, 5, 6, 7], or two or more networks together such as those in [8, 9], working
memory and attention networks in [10, 11, 12], dorsal and ventral attention
networks in [13, 14, 15] and the dorsal attention, ventral attention, and
visual presented in [16, 17, 18]. However, in many cases, the confounding
variables can be out of the considered networks, for instance, in the finger-
tapping task, the information flows from the brainstem implying that
subcortical regions can act as confounding variables for the cortex,
highlighting that these regions must be included in extracting causality to
uphold this assumption. While studies such as those by [19, 20] have made
strides by incorporating subcortical regions alongside the cortex in causal
discovery, leveraging a fusion of methodologies outlined in [21, 22, 23, 24],
their findings still suffer from unreliability due to the low performance of
existing causal discovery methods in extracting causality in large networks.
Conversely, the approaches presented in [25, 26, 27, 28] have demonstrated
progress in addressing causal discovery in the presence of both measured and
unmeasured confounders, enabling the study of smaller networks without the
concern of causal sufficiency. However, they also suffer from low performance
in real-world scenarios. This suggests that focusing solely on a small network
without causal sufficiency may not be practical with the existing methods. An
alternative approach could involve identifying and exclusively including
confounders in the network under study, followed by employing existing methods
that extract causality with high accuracy in smaller networks.

In this paper, we present an algorithmic identification approach to extract
the essential exogenous nodes for causal sufficiency in brain Networks. This
enables the use of existing causal methods that accurately extract causal
structures for smaller networks by including only the confounding variables,
thus mitigating concerns about the low performance of existing methods with a
large number of variables. Our proposed approach performs in three sequential
steps. Firstly, similar to the Peter-Clark (PC) algorithm [2], initiates by
conducting unconditional and conditional independence tests between pairs of
regions within a network. Then, it carries out conditional dependency tests on
these same pairs conditioned on regions from other networks. This procedure is
uniformly applied across all participants. By comparing the distribution of
conditional and unconditional independence tests for all subjects with the
Kolmogorov-Smirnov test, the candidate confounders are distinguished. Finally,
the results from applying Non-Factorized Identifiable Variational Autoencoders
(NF-iVAE) [4] to the candidate regions are computed and based on the
Correlation Coefficient index (CCI) metric, we identify confounding variables
among the candidates. The proposed algorithm is population-based, implying
that it is applicable in problems involving a cohort of individuals, such as
brain network analysis.

Our contributions are threefold:

  * •

We introduce an algorithmic identification method for discovering essential
exogenous nodes for causal sufficiency.

  * •

We apply our approach to the Human Connectome Project’s (HCP) movie-watching
task data and demonstrate that the identified regions align consistently with
those discovered from a neuroscientific perspective.

  * •

we show the reliability of our results by testing 30 independent runs for NF-
iVAE initialization.

The codes associated with this study are publicly available at github.

##  2 Proposed method

To present and discuss the design of the proposed identification method, we
first present definitions and preliminaries. Then the algorithm for
identifying essential exogenous nodes for causal efficiency is presented.
Finally, we describe the data used and the preprocessing steps. In this paper,
$v\in V$ shows all the variables. $z\in Z$ are the variables in the network
under the study. $s\in S$ is variables out of the networks under study, i.e.,
candidate confounders and $c\in C$ are the confounders. $j\in J$ denotes the
index of the subjects in the set $J$.

###  2.1 Definitions and preliminaries

####  2.1.1 Causal sufficiency

A general definition of causal sufficiency assumption is presented in [1] as
follows,

######  Definition 1 (A general definition of causal sufficiency)

The set of variables in $S\subseteq V$ is causally sufficient for all the
variables in $Z\subseteq V$, if, for every pair of variable $z_{1}$ and
$z_{2}$, all the common causes or at least the ancestor of common causes are
in $S$, i.e., $C\subseteq S\subseteq V$.

The details on the PC algorithm, an (un)conditional independence test, and the
NF-iVAE are presented in supplementary materials.

###  2.2 Identification of Essential exogenous nodes for causal sufficiency

Our proposed identification algorithm runs in three main steps:

  1. 1.

Our approach, similar to the PC algorithm, initiates by conducting
unconditional and conditional independence tests between all pairs of regions,
$Z$. Complementary to the PC algorithm, it then carries out conditional
independence tests on these same pairs conditioned on regions from other
networks, $S$ is the number of possible confounders. This procedure is
uniformly applied across all participants.

  2. 2.

By comparing the distribution of conditional and unconditional independence
tests of each pair for all subjects with the Kolmogorov-Smirnov (KS) test, a
subset of candidate confounding variables are distinguished. These candidates
can represent both a confounder and a mediator.

  3. 3.

Measuring the Correlation coefficient index (CCI) between the latent variables
generated by NF-iVAE and the real values of candidates proposed in the
previous step, we identify the nodes that are confounding variables of the
network under study.

The proposed 3-step approach for the identification of confounders is given in
Algorithm 1.

Input: Subset of Nodes $Z\subset V$, Subset of potential confounding variables
$L\subset V\setminus Z$, List of subjects $J$, Correlation Coefficient
Threshold $T_{h}$, $\alpha$: Threshold for $H_{0}$.
Initialize: Confounding candidate variables set $S\leftarrow\\{\\}$,
Confounding variables set $C\leftarrow\\{\\}$.
Step 1:

1: for $l\in L$ do

2: for $z_{i}\in Z$ do

3: for $z_{k}\in Z\setminus\\{z_{i}\\}$ do

4: Initialize unconditional independence set $U_{\text{unc}}\leftarrow\\{\\}$

5: Initialize conditional independence set $U_{\text{cond}}\leftarrow\\{\\}$

6: for $j\in J$ do

7: $U_{\text{unc}}\leftarrow U_{\text{unc}}\cup\\{z_{i}^{j}\perp z_{k}^{j}\\}$

8: $U_{\text{cond}}\leftarrow U_{\text{cond}}\cup\\{z_{i}^{j}\perp
z_{k}^{j}\mid l^{j}\\}$

9: end forStep 2:

10: $pval\leftarrow\text{\\{PValue of KS-test on }U_{\text{cond}}\text{ and
}U_{\text{unc}}\\}$

11: if $pval<\alpha$ and mean($U_{\text{unc}}$) $<$ mean($U_{\text{con}}$)
then

12: $S\leftarrow S\cup\\{l\\}$

13: end ifend Step2

14: end for

15: end for

16: end forend Step1

Step 3:

1: Apply NF-iVAE on $Z$ and estimate $L^{\prime}$.

2: $CCI\leftarrow\text{\\{Correlation Coefficient Index between $L$ and
$L^{\prime}$ pairs\\}}$

3: for $s\in S$ do

4: if $CCI[s]>T_{h}$ then

5: $C\leftarrow C\cup\\{s\\}$

6: end if

7: end for

end Step3
Output: $C$.

Algorithm 1 Identification of Essential Exogenous Nodes for Causal Sufficiency
Figure 1: a.The true underlying network with variables $z_{1},...,z_{5}$, and
confounders, $c_{1}$ and $c_{2}$ b. starting PC algorithm with fully connected
graph, c. result of PC algorithm applying d. Identifying essential confounders
based on the proposed algorithm i.e., $s_{2}=c_{1}$ and $s_{4}=c_{2}$, e.
applying PC algorithm to $z_{1},...,z_{5}$ and $c_{1}$ and $c_{2}$, F. result
of applying causal discovery method with confounders

Fig. 1 illustrates the result of employing algorithmic identification of
confounding variables. In this figure, the true graph is presented in (a). In
(b), the set of variables within the dashed rectangle represents the network
under study. In (c), the result of the PC algorithm is shown, which includes
spurious connections. In (d), the essential confounders are extracted. In (e),
all possible links from the confounders and variables of the network under
study are added. Finally, in (f), the result of applying PC to this set of
variables is illustrated in which the spurious links are omitted.

###  2.3 Data, preprocessing, and parcellation

In this study, we utilized task-based fMRI data obtained from the HCP.
Specifically, we focused on the movie-watching task data of 172 participants,
which is part of the HCP’s tasks designed to elicit a broad range of neural
activity. Preprocessing of the fMRI data was conducted using established
pipelines provided by the HCP. For this study, the brain was divided into 100
distinct regions using the Schaefer parcellation [30] scheme, and 16
subcortical regions are added based on the Destrieux parcellation [31] to
ensure causal sufficiency.

##  3 Experiments

###  3.1 Experiment design

The relationship between the visual and attention networks during movie-
watching tasks is a subject of interest within neuroscience, given that both
networks are highly engaged in processing the complex, dynamic stimuli
presented in films. The visual and attention networks comprise 13 and 27
nodes, respectively. We explore the interaction between these networks by
identifying nodes from the attention (visual) network that act as confounding
variables for at least two nodes in the visual (attention) network to
demonstrate our method’s performance and discuss the consistency of our
results with studies such as [32].

###  3.2 Results

Table 1 shows the candidate confounder variables of the visual network pairs
within the attention network derived from steps 1 and 2 of our proposed
algorithm.

Table 1: The candidate confounders in the Attention network and the corresponding variables in the Visual network. V1, V2, V3, V4, V5, and V6 denote LH_VisCent_ExStr_2, LH_VisCent_ExStr_3, RH_VisCent_ExStr_1, RH_VisCent_ExStr_3, RH_VisPeri_ExStrInf_1, LH_VisPeri_StriCal_1, respectively. | Confounding candidates
---
in the Attention network
Visual network pairs
LH_DorsAttnA_TempOcc_1 | (V1, V2), (V1, V3), (V1, V4)
LH_DorsAttnA_SPL_1 | (V4, V5)
RH_DorsAttnA_TempOcc_1 | (V1, V2), (V1, V4)
RH_DorsAttnA_SPL_1 | (V6, V4), (V4, V5)

Table 2 shows the candidate confounder variables of the attention network
pairs within the visual network derived from the same process as in Table 1.

Table 2: The candidate confounders in the Visual network and the corresponding variables in the Attention network. A1, A2, A3, A4, A5, A6, A7, and A8 denote LH_DorsAttnA_ParOcc_1, RH_DorsAttnA_SPL_1, LH_DorsAttnA_TempOcc_1, LH_DorsAttnB_FEF_1, RH_DorsAttnB_FEF_1, RH_DorsAttnA_TempOcc_1, RH_DorsAttnA_ParOcc_1, LH_DorsAttnA_SPL_1, respectively. | Confounding candidates
---
in the Visual network
Attention network pairs
LH_VisCent_ExStr_1 | (A1, A2)
LH_VisCent_ExStr_3 |  | (A3,A4), (A3, A5), (A1, A8), (A1, A2), (A8, A6),
---
(A8, A7), (A4, A6), (A6, A2), (A6, A5), (A7, A2)
RH_VisCent_ExStr_1 | (A1, A8), (A1, A2)
RH_VisCent_ExStr_3 |  | (A1, A8), (A1, A2), (A8, A6), (A8, A7), (A6, A2),
---
(A7, A2)
RH_VisCent_ExStrSup_1 | (A1, A2)

To identify the confounders among the candidates, by considering the index of
the subjects $J$ as the conditioning variable, we apply NF-iVAE model and
estimate the CCI between the latents sampled from the model and the
confounding variable candidates. These two Tables demonstrate that mostly the
confounding candidates with a higher number of KS-test rejections, acquire
higher CCI. In Figs 2.a and b, we include the algorithm’s results using the
NF-iVAE and iVAE instead of the NF-iVEA. According to these results, NF-iVAE
acquires noticeably higher CCI than iVAE, since the NF-iVAE can account for
the cases where there is dependence between latent variables. Based on the
computed results, the identified confounding variables in attention networks
are among the Dorsal regions. Additionally, the regions that the visual
networks are confounding are in dorsal regions as well.

Figure 2:  The CCI values for identified regions based on NF-iVAE and iVAE. a.
results for the case that attention is confounder b. results for the case that
visual is confounder c. the relation between dorsal, ventral and visual
regions

Fig 2.c, shows the interaction between attention and visual networks. These
findings closely mirror previous research such as [32], in which, it is shown
that the regions in the dorsal and ventral attention have influenced each
other, however, only the Dorsal regions have causal interactions with visual
networks. Additionally, similar to the same study’s results, the visual
regions also affect the dorsal attention. It is worth noting that in the
context of movie-watching, one might hypothesize that the visual network’s
processing of visual stimuli causally influences the attention network’s
allocation of attentional resources. However, the reverse could also be true,
where the attention network’s prioritization influences how visual information
is processed.

To demonstrate the reliability of the identification algorithm, in Fig 3, we
present the probability at which each node of the visual network was ranked
among the top 5 for having the highest CCI across all 30 runs. In each, the
NF-iVAE is initialized with a unique random seed to ensure the diversity of
model conditions. In each run, we estimated the CCI between the latent
variables extracted by the NF-iVAE and the original nodes of the visual
cortex. This selection process highlighted the nodes that are most effectively
captured by the NF-iVAE, reflecting the approach’s capacity to consistently
identify the most relevant and representative nodes across different initial
conditions.

Figure 3:  Probability of Visual Cortex Nodes Ranked Among Top 5 by CCI Across
30 Runs. The y-axis denotes the frequency of inclusion in the top 5 rankings,
and the x-axis lists the nodes.

This probability distribution elucidates our method’s ability to reliably
identify the essential confounding nodes within the visual cortex,
underscoring its effectiveness in identifying confounding nodes from proposed
candidates.

##  4 Conclusion

This study introduced a novel algorithmic approach for identifying essential
exogenous nodes to uphold the assumption of causal sufficiency in the analysis
of brain networks. Through an experimental setup involving the visual and
attention networks during movie-watching tasks, our method has demonstrated a
significant capacity to identify confounding variables that align with
neuroscientific studies and highlight the reliability of the proposed method.
Moreover, the consistency of iVAE, and NF-iVAE for different random
initialization along with high CCI for the proposed candidates driven by
independence tests, bolsters the validity of our findings from a
neuroscientific perspective.

## References

  * [1] Spirtes P, Glymour CN, Scheines R. Causation, prediction, and search. MIT Press; 2000.
  * [2] Beckers S. Causal sufficiency and actual causation. Journal of Philosophical Logic. 2021 Dec;50(6):1341-74.
  * [3] Nauhaus I, Busse L, Carandini M, Ringach DL. Stimulus contrast modulates functional connectivity in visual cortex. Nature neuroscience. 2009 Jan;12(1):70-6.
  * [4] Harris KD, Thiele A. Cortical state and attention. Nature reviews neuroscience. 2011 Sep;12(9):509-23.
  * [5] Hofer SB, Ko H, Pichler B, Vogelstein J, Ros H, Zeng H, Lein E, Lesica NA, Mrsic-Flogel TD. Differential connectivity and response dynamics of excitatory and inhibitory neurons in visual cortex. Nature neuroscience. 2011 Aug;14(8):1045-52.
  * [6] Koshino H, Carpenter PA, Minshew NJ, Cherkassky VL, Keller TA, Just MA. Functional connectivity in an fMRI working memory task in high-functioning autism. Neuroimage. 2005 Feb 1;24(3):810-21.
  * [7] Hampson M, Driesen NR, Skudlarski P, Gore JC, Constable RT. Brain connectivity related to working memory performance. Journal of Neuroscience. 2006 Dec 20;26(51):13338-43.
  * [8] Ungerleider SK. Mechanisms of visual attention in the human cortex. Annual review of neuroscience. 2000 Mar;23(1):315-41.
  * [9] Luo TZ, Maunsell JH. Neuronal modulations in visual cortex are associated with only one of multiple components of attention. Neuron. 2015 Jun 3;86(5):1182-8.
  * [10] Oberauer K. Working memory and attention–A conceptual analysis and review. Journal of cognition. 2019;2(1).
  * [11] Panichello MF, Buschman TJ. Shared mechanisms underlie the control of working memory and attention. Nature. 2021 Apr 22;592(7855):601-5.
  * [12] Awh E, Vogel EK, Oh SH. Interactions between attention and working memory. Neuroscience. 2006 Apr 28;139(1):201-8.
  * [13] Vossel S, Geng JJ, Fink GR. Dorsal and ventral attention systems: distinct neural circuits but collaborative roles. The Neuroscientist. 2014 Apr;20(2):150-9.
  * [14] Suo X, Ding H, Li X, Zhang Y, Liang M, Zhang Y, Yu C, Qin W. Anatomical and functional coupling between the dorsal and ventral attention networks. Neuroimage. 2021 May 15;232:117868.
  * [15] Vossel S, Weidner R, Driver J, Friston KJ, Fink GR. Deconstructing the architecture of dorsal and ventral attention systems with dynamic causal modeling. Journal of Neuroscience. 2012 Aug 1;32(31):10637-48.
  * [16] Zhao J, Wang J, Huang C, Liang P. Involvement of the dorsal and ventral attention networks in visual attention span. Human brain mapping. 2022 Apr 15;43(6):1941-54.
  * [17] Ahrens MM, Veniero D, Freund IM, Harvey M, Thut G. Both dorsal and ventral attention network nodes are implicated in exogenously driven visuospatial anticipation. Cortex. 2019 Aug 1;117:168-81.
  * [18] Majerus S, Attout L, D’Argembeau A, Degueldre C, Fias W, Maquet P, Martinez Perez T, Stawarczyk D, Salmon E, Van der Linden M, Phillips C. Attention supports verbal short-term memory via competition between dorsal and ventral attention networks. Cerebral Cortex. 2012 May 1;22(5):1086-97.
  * [19] Bagheri A, Dehshiri M, Bagheri Y, Akhondi-Asl A, Nadjar Araabi B. Brain effective connectome based on fMRI and DTI data: Bayesian causal learning and assessment. Plos one. 2023 Aug 18;18(8):e0289406.
  * [20] Bagheri A, Pasande M, Bello K, Araabi BN, Akhondi-Asl A. Discovering Dynamic Effective Connectome of Brain with Bayesian Dynamic DAG Learning. arXiv preprint arXiv:2309.07080. 2024 Sep 7.
  * [21] Zheng X, Aragam B, Ravikumar PK, Xing EP. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems. 2018;31.
  * [22] Ng I, Ghassami A, Zhang K. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems. 2020;33:17943-54.
  * [23] Bello K, Aragam B, Ravikumar P. Dagma: Learning dags via m-matrices and a log-determinant acyclicity characterization. Advances in Neural Information Processing Systems. 2022 Dec 6;35:8226-39.
  * [24] Andrews B, Ramsey J, Sanchez Romero R, Camchong J, Kummerfeld E. Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow Shrink Trees. Advances in Neural Information Processing Systems. 2024 Feb 13;36.
  * [25] Forré P, Mooij JM. Constraint-based causal discovery for non-linear structural causal models with cycles and latent confounders. arXiv preprint arXiv:1807.03024. 2018 Jul 9.
  * [26] Gerhardus A, Runge J. High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems. 2020;33:12615-25.
  * [27] David Kaltenpoth and Jilles Vreeken. 2023. Nonlinear causal discovery with latent confounders. In Proceedings of the 40th International Conference on Machine Learning (ICML’23), Vol. 202. JMLR.org, Article 638, 15639–15654.
  * [28] Wang YS, Drton M. Causal discovery with unobserved confounding and non-Gaussian data. Journal of Machine Learning Research. 2023;24(271):1-61.
  * [29] Lu C, Wu Y, Hernández-Lobato JM, Schölkopf B. Invariant causal representation learning for out-of-distribution generalization. In International Conference on Learning Representations 2021 Oct 6.
  * [30] Schaefer A, Kong R, Gordon EM, Laumann TO, Zuo XN, Holmes AJ, Eickhoff SB, Yeo BT. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI. Cerebral cortex. 2018 Sep 1;28(9):3095-114.
  * [31] Destrieux C, Fischl B, Dale A, Halgren E. Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage. 2010 Oct 15;53(1):1-5.
  * [32] Ptak R. The frontoparietal attention network of the human brain: action, saliency, and a priority map of the environment. The Neuroscientist. 2012 Oct;18(5):502-15.

Supplementary for “Algorithmic Identification of Essential Exogenous Nodes for
Causal Sufficiency in Brain Networks”

##  Appendix 0.A Definitions and preliminaries required for the proposed
method

To present and discuss the design of the proposed identification method, we
first present definitions and preliminaries, including the definition of
causal sufficiency, the PC algorithm, an (un)conditional independence test,
and the NF-iVAE. Then the algorithm for identifying essential exogenous nodes
for causal efficiency is presented. Finally, we describe the data used and the
preprocessing steps. In this paper, $v\in V$ shows all the variables. $z\in Z$
are the variables in the network under the study. $s\in S$ is variables out of
the networks under study, i.e., candidate confounders and $c\in C$ are the
confounders. $j\in J$ denotes the index of the subjects in the set $J$.

###  0.A.1 Definitions and preliminaries

####  0.A.1.1 Causal sufficiency

A general definition of causal sufficiency assumption is presented in [1] as
follows,

######  Definition 2 (A general definition of causal sufficiency)

The set of variables in $S\subseteq V$ is causally sufficient for all the
variables in $Z\subseteq V$, if, for every pair of variable $z_{1}$ and
$z_{2}$, all the common causes or at least the ancestor of common causes are
in $S$, i.e., $C\subseteq S\subseteq V$.

####  0.A.1.2 Peter-Clark algorithm

The Peter-Clark (PC) algorithm [2] provides a search architecture under i.i.d.
sampling. This algorithm performs as follows, 1- PC starts with a complete
undirected graph. 2- It removes edges between variables that are
unconditionally independent of each other. This step eliminates edges
representing direct relationships between variables that are not supported by
the data. 3- For each pair of variables with an edge between them, and for
each variable connected to either of them: If the pair of variables remains
independent when conditioned on the connecting variable, the algorithm removes
the edge between them. This step eliminates edges where the relationship
between variables can be explained by a third variable. 4- If the pair of
variables remains independent when conditioned on both pairs of connecting
variables, remove the edge between them.

####  0.A.1.3 Unconditional and conditional independence test

######  Lemma 1 (Independence test)

[3] Under the null hypothesis that $v_{1}$ and $v_{2}$ are statistically
independent, the statistic

$\hat{T}_{UI}=\frac{1}{n}\text{Tr}(\hat{K}_{v_{1}}\hat{K}_{v_{2}})$ (1)

has the same asymptotic distribution as

$\hat{T}_{UI}=\frac{1}{n^{2}}\sum_{i,j=1}^{n}\lambda_{{v_{1}},i}\lambda_{{v_{2}},j}\zeta_{ij}^{2},$
(2)

i.e., $T_{UI}\overset{d}{=}\hat{T}_{UI}$ as $n\to\infty$. In this lemma,
$\hat{K}_{v_{1}}$ and $\hat{K}_{v_{2}}$ are the corresponding centralized
kernel matrices. Given the samples $v_{1}$ and $v_{2}$, $\lambda_{v_{1},i}$
and $\lambda_{v_{2},j}$ are the eigenvalues of $\hat{K}_{v_{1}}$ and
$\hat{K}_{v_{2}}$, respectively. $\zeta_{i}j$ are i.i.d. standard Gaussian
variables (i.e., $zeta_{ij}^{2}$ are i.i.d. $\chi_{1}^{2}$-distributed
variables).

######  Lemma 2 (Conditional independence test)

[3] Under the null hypothesis $H_{0}$ ($v_{1}$ and $v_{2}$ are conditionally
independent given z), we have that the statistic

$T_{CI}=\frac{1}{n}\text{Tr}(\hat{K}_{v_{1}|z}\hat{K}_{v_{2}|z})$ (3)

has the same asymptotic distribution as

$\hat{T}_{CI}=\frac{1}{n}\sum_{k=1}^{n^{2}}\hat{\lambda}_{k}\cdot\zeta_{k}^{2},$
(4)

where $\hat{\lambda}_{k}$ are eigenvalues of $\hat{w}\hat{w}^{T}$ and
$\hat{w}=[\hat{w}_{1},\ldots,\hat{w}_{n}]$, with the vector $\hat{w}_{t}$
obtained by stacking
$M_{t}=[\psi_{v_{1}|z,1}(\hat{v_{1}}_{t}),...,\psi_{v_{2}|z,n}(\hat{v_{2}}_{t})][\psi_{v_{2}|z,1}(\hat{v_{2}}_{t}),...,\psi_{v_{2}|z,n}(\hat{v_{2}}_{t})]^{T}.$
The eigenvalue decompositions (EVD) of the centralized kernel matrices are
$\tilde{K}_{v_{1}}=V_{v_{1}}A_{v_{1}}V_{v_{1}}^{T}$ and
$\tilde{K}_{v_{2}}=V_{v_{2}}A_{v_{2}}V_{v_{2}}^{T}$, where $A_{v_{1}}$ and
$A_{v_{2}}$ are the diagonal matrices containing the non-negative eigenvalues
$\lambda_{v_{2},i}$ and $\lambda_{y,i}$, respectively. Then,
$\psi_{v_{1}}=[\psi_{1}(v_{1}),\ldots,\psi_{n}(v_{1})]V_{v_{1}}A_{v_{1}}^{1/2}$
and
$\phi_{y}=[\phi_{1}(v_{2}),\ldots,\phi_{n}(v_{2})]V_{v_{2}}A_{v_{2}}^{1/2}$.
I.e., $\psi_{i}(v_{1})=\sqrt{\lambda_{v_{1},i}}V_{v_{1},i}$, where
$V_{v_{1},i}$ denotes the $i$th eigenvector of $K_{v_{1}}$.

####  0.A.1.4 NF-iVAE

NF-iVAE can be used to identify latent variables for a model in non-stationary
environments [4], contrary to the VAE’s that are frequently used for the
identification of latent confounding variables, [5, 6]. NF-iVAE posits a more
general setting than iVAE [7] for which the dependence between latent
variables is considered in Prior the distribution. It assumes the conditional
prior $p(S|J)$ belongs to a general exponential family formalized as follows:
$p(S|J)$ belongs to a general exponential family with a parameter vector given
by an arbitrary function $U$ and sufficient statistics
$T(S)=[T_{f}(S)^{T},T_{NN}(S)^{T}]^{T}$ given by the concatenation of: a) the
sufficient statistics
$T_{f}(S)=[T_{1}(s_{1})^{T},\ldots,T_{n}(s_{n})^{T}]^{T}$ of a factorized
exponential family, where $s_{i}\in S$ and for all the $T_{i}(s_{i})$ have
dimension larger or equal to $2$, and b) the output $T_{NN}(S)$ of a neural
network with ReLU activations. The resulting density function is thus given by

$p_{T,\lambda}(S|J)=\frac{Q(S)}{C^{\prime}(J)}\exp\left[T(S)^{T}\lambda(J)\right],$
(5)

where $Q$ is the base measure and $C^{\prime}$ is the normalizing constant.

###  0.A.2 Identification of Essential exogenous nodes for causal sufficiency

Our proposed identification algorithm runs in three main steps:

  1. 1.

Our approach, similar to the PC algorithm, initiates by conducting
unconditional and conditional independence tests between all pairs of regions,
$Z$. Complementary to the PC algorithm, it then carries out conditional
independence tests on these same pairs conditioned on regions from other
networks, $S$ is the number of possible confounders. This procedure is
uniformly applied across all participants.

  2. 2.

By comparing the distribution of conditional and unconditional independence
tests of each pair for all subjects with the Kolmogorov-Smirnov (KS) test, a
subset of candidate confounding variables are distinguished. These candidates
can represent both a confounder and a mediator.

  3. 3.

Measuring the Correlation coefficient index (CCI) between the latent variables
generated by NF-iVAE and the real values of candidates proposed in the
previous step, we identify the nodes that are confounding variables of the
network under study.

It is important to note that while NF-iVAE can identify all confounding
variables, it necessitates the fulfillment of certain stringent assumptions,
such as the condition (iv) in Theorem 1 [4], which stipulates the existence of
$k+1$ distinct points $\\{U^{0},U^{1},\ldots,U^{k}\\}$ where the matrix
$L=[\lambda(U)^{1}-\lambda(U)^{0},\ldots,\lambda(U)^{k}-\lambda(U)^{0}]$ of
size $k\times k$ is invertible, with $k$ representing the dimension of $T$.
Furthermore, in NF-iVAE method, the number of confounding variables and
topological order has to be known. This imposes limitations in scenarios where
the number of nodes in the target network, whose causal structure is to be
determined, is fewer than the number of confounding variables. Conversely,
relying on candidates discovered in the first two steps offers an alternative
approach for identifying potential confounding variables in situations where
the availability of distinct points $\\{U^{0},U^{1},\ldots,U^{k}\\}$ and
knowledge of the topological order of nodes are insufficient. This results in
more effectiveness even when the number of confounding variables exceeds the
number of nodes for which causal structure identification is sought as we only
apply the NF-iVAE on the nodes of interest.

## References

  * [1] Beckers S. Causal sufficiency and actual causation. Journal of Philosophical Logic. 2021 Dec;50(6):1341-74.
  * [2] Spirtes P, Glymour CN, Scheines R. Causation, prediction, and search. MIT Press; 2000.
  * [3] Zhang K, Peters J, Janzing D, Schölkopf B. Kernel-based conditional independence test and application in causal discovery. InProceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence 2011 Jul 14 (pp. 804-813).
  * [4] Lu C, Wu Y, Hernández-Lobato JM, Schölkopf B. Invariant causal representation learning for out-of-distribution generalization. International Conference on Learning Representations 2021 Oct 6.
  * [5] Tran D, Ranganath R, Blei DM. The variational Gaussian process. In4th International Conference on Learning Representations, ICLR 2016 2016.
  * [6] Harada, Shonosuke, and Hisashi Kashima. ”InfoCEVAE: treatment effect estimation with hidden confounding variables matching.” Machine Learning (2022): 1-19.
  * [7] Khemakhem, Ilyes, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. ”Variational autoencoders and nonlinear ica: A unifying framework.” In International Conference on Artificial Intelligence and Statistics, pp. 2207-2217. PMLR, 2020.