# FED-CD: Federated Causal Discovery from Interventional and Observational
Data

Amin Abyaneh
McGill University, Montreal, Canada
<EMAIL_ADDRESS>
&Nino Scherrer
ETH Zürich, Zürich, Switzerland
<EMAIL_ADDRESS>
&Patrick Schwab
GlaxoSmithKline, Artificial Intelligence
and Machine Learning
Zug, Switzerland
<EMAIL_ADDRESS>
&Stefan Bauer
KTH Royal Institute of Technology &
CIFAR Azrieli Global Scholar
Tübingen, Germany
<EMAIL_ADDRESS>
&Bernhard Schölkopf
Max Planck Institute for Intelligent Systems
Tübingen, Germany
<EMAIL_ADDRESS>
&Arash Mehrjou
GlaxoSmithKline, Artificial Intelligence
and Machine Learning &
Max Planck Institute for Intelligent Systems
Tübingen, Germany &
ETH Zürich, Zürich, Switzerland
<EMAIL_ADDRESS>

###### Abstract

Existing causal discovery methods typically require the data to be available
in a centralized location. However, many practical domains, such as
healthcare, limit access to the data gathered by local entities, primarily for
privacy and regulatory constraints. To address this, we propose FED-CD, a
federated framework for inferring causal structures from distributed datasets
containing observational and interventional data. By exchanging updates
instead of data samples, FED-CD ensures privacy while enabling decentralized
discovery of the underlying directed acyclic graph (DAG). We accommodate
scenarios with shared or disjoint intervened covariates, and mitigate the
adverse effects of interventional data heterogeneity. We provide empirical
evidence for the performance and scalability of FED-CD for decentralized
causal discovery using synthetic and real-world DAGs.

_Keywords_ Causal Discovery, Federated Learning, Decentralized Data, Bayesian
Networks

##  1 Introduction

The fundamental and often challenging problem of discovering cause-effect
relationships has long captivated the attention of statisticians (Pearl,
2009), and other disciplines such as epidemiology (Vandenbroucke et al., 2016)
and computational biology (Sachs et al., 2005). Inferring causal relationships
is critical for understanding the underlying mechanisms behind random
variables, and predicting the effect of externally imposed manipulations on
causal dynamics of systems (Spirtes, 2010). Conventional approaches to causal
discovery primarily rely on centralized data. However, when dealing with
sensitive information like patients’ medical records, preserving privacy is
essential and protected by strict regulations. As a result, institutions often
refrain from disclosing sensitive data required to infer causal structures.
Privacy concerns extend beyond the medical domain, arising in economics,
biology, and engineering (Glancy, 2012; Papernot et al., 2016).

Consequently, researchers have turned to privacy-preserving causal discovery
to leverage decentralize data. Most notably, federated learning (Li et al.,
2020) is employed to learn causal structures while avoiding privacy
violations. Yet, previous efforts tend to suffer from limitations. For
instance, DARLS framework (Ye et al., 2022) is limited to data generated from
a linear underlying causal mechanism, whereas DS-FCD (Gao et al., 2021) offers
no priority to clients with more valuable contributions. Critically, previous
attempts solely utilize observational data gathered from unperturbed systems,
which restricts the discovered structures to Markov equivalence classes (Yang
et al., 2018; Spirtes and Zhang, 2016). Gathering interventional data by
applying controlled perturbations alleviates such shortcomings through
improving identifiability (Yang et al., 2018; Ke et al., 2019).

We propose FED-CD, a federated approach for causal discovery from
decentralized observational and interventional data. We learn a global causal
structure from a federation of clients, without direct access to each client’s
local data. To this end, clients adopt a neural causal discovery method, for
instance Lippe et al. (2021), to discover a local structure from their private
data. These local findings are then aggregated on a server while adhering to
two important criteria. Firstly, clients with more reliable knowledge about
intervened variables have a larger impact on the discovery process. And
secondly, no data samples are communicated between the clients, or to the
server, since only probabilistic updates are exploited in the aggregation
process.

Conforming to the mentioned criteria, FED-CD can confidentially aggregate
clients’ updates based on the reliability of their local dataset. Clients’
contributions are rated by the extent of their access to interventional data
and the location of interventions in the overall causal structure, owing to
our novel knowledge aggregation method. Therefore, a client presents a
stronger vote on the existence of a cause-effect relationship in the vicinity
of its interventional data. Moreover, our approach poses few restrictive
assumptions about the data generation mechanism, and endures a level of
heterogeneity in interventional data. We empirically demonstrate that FED-CD
performs similar to the centralized approaches, without directly sharing
clients’ private data, for synthetic and real-world causal structures. An
overview of FED-CD is presented in Figure 1.

Figure 1: Overview of FED-CD and its federated setup. We only represent a
single client, as the rest perform similar operations in parallel.

##  2 Related Work

Attempts to discover causal structures from distributed data were already in
effect before the rise of federated learning. For instance, Gou et al. (2007)
proposes a two-step approach: they obtain a local Bayesian network for each
partition of the dataset and then, utilizing conditional independence tests,
they combine local DAGs into a final global model. Moreover, in Na and Yang
(2010), the authors propose a voting-based approach that accounts for sending
local models to a central site where the final structure is determined by
looking for frequently occurring parts among the local DAGs. Both these
methods operate for only a single round; thus, no iterative improvement
happens, and the discovered DAG is decided by the majority of the clients.

There has been a surge in research at the intersection of causal discovery and
federated learning. Among these, DS-FCD (Gao et al., 2021) is a gradient-based
method where at each round, clients learn an adjacency matrix to approximate
the causal graph and neural network representation to approximate the causal
mechanisms based on their local data. Another related work is developed by Ng
and Zhang (2021), where they convert a centralized optimization problem –
earlier proposed by Zheng et al. (2018) – to a distributed one and solve it
through inter-client collaboration. DARLS (Ye et al., 2022) is another method
that counts on generalized linear models and distributed optimization to find
an optimal DAG without violating clients’ privacy. Authors in Mian et al.
(2023) enhance the previous work by avoiding the exchange of model parameters.
Yet, generally, federated causal discovery methods are either limited to
linear causal mechanisms, or only consider homogeneous data. Moreover, the
rich information provided by interventional data is not exploited in any
federated work to the best of our knowledge.

Finally, due to relevancy to our topic, we present the background of
continuous optimization-based causal discovery methods. Unlike score-based
methods, this class of algorithms avoids the combinatorial greedy search over
DAGs by employing gradient-based approaches. DAGs with NO TEARS (Zheng et al.,
2018) is considered the first to reformulate graph search problem as a
continuous optimization problem (Vowels et al., 2021). DAG-GNN (Yu et al.,
2019) extends NO TEARS by incorporating deep learning to capture complex
nonlinear mappings, by proposing a deep generative model and applying a
variant of the structural constraint to learn the underlying DAG.

When it comes to neural causal discovery methods, SDI (Ke et al., 2019) and
DCDI (Brouillard et al., 2020), are known to discover causal DAG leveraging
the combination of observational and interventional data for discrete and
continuous data, respectively. Other approaches within this context, such as
SAM (Kalainathan et al., 2018) only consider observational data. ENCO (Lippe
et al., 2021) presents an efficient DAG discovery method for directed, acyclic
causal graphs utilizing both observational and interventional data. They
present the graph search as optimization of independent edge likelihoods, and
reliably recover DAGs for large feature spaces.

##  3 Problem Setup and assumptions

_Causal graphical models_ (CGM) provide mathematical abstraction to
quantitatively describe causal relations between random variables. A CBN is a
pair $(G,P)$, where $G=(X,E)$ is a _directed acyclic graph_ (DAG), and $P$ is
a distribution faithful to $G$ (Spirtes, 2010). Within $G$, each node,
$X_{i}\in X\;\forall i\in\\{1,...,N\\}$, represents a random variable, and a
directed edge $X_{i}\rightarrow X_{j}$ denotes the existence of a cause-effect
relationship from $X_{i}$ to $X_{j}$ (Pearl, 2009). We denote the binary
adjacency matrix of $G$ by $adj(G)_{N\times N}$, and assume causal sufficiency
of the CGM, i.e., all common causes of variables are included and observable.

An _intervention_ on a random variable $X$ is an applied perturbation on the
system, such that it overrides the natural values of $X$ that affects its
probability distribution and (possibly) alters the DAG. We call the gathered
data from an unperturbed system _observational_ , $D_{\mathcal{O}}$, and we
use the term _interventional_ , $D_{\mathcal{I}}$, when the data is gathered
from the perturbed system. The collection of these two datasets is denoted by
$D:=\\{D_{\mathcal{O}},\;D_{\mathcal{I}}\\}$, that is a centralized dataset
generated by perturbed and unperturbed causal mechanisms described by $(G,P)$.
Moreover, interventions within $D_{\mathcal{I}}$ are perfect,
$p_{\mathcal{I}}(X_{i}\mid\text{Pa}(X_{i}))=p_{\mathcal{I}}(X_{i})$, where
$p_{\mathcal{I}}$ is the distribution of intervened $X_{i}$.

Our proposed federated setup consists of a central node acting as a server and
$K$ other nodes as clients. We denote the server with $S$ and the clients
with, $C_{k}\;\forall k\in\\{1,...,K\\}$. Each client is an independent
processing unit, and can only communicate with the server. The centralized
dataset $D$ is distributed among the clients, where each client $C_{k}$ has
access to a (not necessarily disjoint) subset of $D$, called
$D^{k}:=\\{D_{\mathcal{O}}^{k},\;D_{\mathcal{I}}^{k}\\}$. To utilize the
entire dataset for structure discovery, we assume these subsets to span the
entire $D$, such that $D=\bigcup^{K}_{k=1}D^{k}$.

Any dataset can be distributed either horizontally or vertically. Horizontal
distribution leads to clients having access to the entire set of features, but
only a fraction of samples. A vertical split, however, provides access to only
a subset of features in the original centralized data. We consider a
horizontal split for $D_{\mathcal{O}}$ and different setups with either
vertical or horizontal distribution of $D_{\mathcal{I}}$. Therefore, we assume
that clients are aware of all the dataset features, but might not have access
to interventional data corresponding to each random variable, $X_{i}$. The
accessible set of intervened variables for each client $C_{k}$ is represented
by $X^{k}_{\mathcal{I}}\subset X$.

#### Problem statement

Within the described federated setting, we focus on recovering the DAG, $G$,
rather than the underlying probability distribution, $P$. We enforce a strict
restriction on direct exchange of $D^{k}$ samples either between two arbitrary
clients, $C_{i}$ and $C_{j}$, or to the server, $S$.

##  4 Federated Causal Discovery From Decentralized Data

Following the federated learning paradigm, clients participating in FED-CD
collaborate through an iterative discovery process. Each federated round
fundamentally consists of two stages:

  1. 1.

Local discovery (Section 4.1): Each client applies a local causal discovery
method (LCDM) to yield a _prior belief_ about $G$ while considering existing
belief from previous rounds. Therefore, a client, $C_{k}$, can securely share
its knowledge of the causal structure with the server, without disclosing
samples or model parameters.

  2. 2.

Global aggregation (Section 4.3): The server employs an aggregation method to
find the updated belief from a collection of clients’ communicated beliefs.
Then, the server broadcasts the aggregated belief back to all clients, which
they utilize for local discovery in the next round.

We start with the first step in Section 4.1 by discussing the nature of the
prior belief knowledge and LCDMs. Next, we present two aggregation methods
based on the characteristics of clients’ local data. In Section 4.2, we assume
a horizontal distribution of $D_{\mathcal{I}}$, that is, every client has
access to the same set of interventions. Yet, assuming identical intervened
covariates in interventional data is a naive assumption for real-world
scenarios, where clients oftentimes hold interventional data on different
features. Consequently, in Section 4.3, we propose a proximity-based belief
aggregation applicable to clients with non-identical sets of intervened
variables, i.e., vertical distribution of $D_{\mathcal{I}}$. It simulates a
level of heterogeneity between clients in interventional data, since the
underlying distributions vary across clients.

###  4.1 Local discovery of prior beliefs

The goal in this stage is for each client to infer a belief about the global
DAG. To this end, clients apply an LCDM to their local datasets, $D^{k}$. We
expect the LCDM to produce and later incorporate prior knowledge about the
structure, and favor this knowledge when uncertain about the existence of any
$X_{i}\rightarrow X_{j}\in E$. We define clients’ beliefs with matrices of the
same dimensions as $adj(G)$.

###### Definition 4.1

(Belief Matrix) Let $\psi\in\mathbb{R}^{N\times N}$ be the matrix with
elements $\psi_{ij}$, where $\psi_{ij}$ is the parameter of an independent
Bernoulli distribution, i.e., Ber($\psi_{ij}$)$\;\forall i,j$. We call $\psi$
the belief matrix, and each element, $\psi_{ij}$, corresponds to the existence
probability of $X_{i}\to X_{j}$, i.e., $P(X_{i}\to
X_{j})\sim\text{Ber}(\mathcal{B}_{ij})$. The matrix $\psi$ results in an
adjacency matrix by using an element-wise binary step function (threshold =
0.5).

We carefully pick neural LCDMs that learn similar belief matrices through
continuous-optimization. For instance, ENCO (Lippe et al., 2021) uses
$\gamma,\;\theta\in\mathbb{R}^{N\times N}$ matrices to represent the existence
and orientation of edges in a graph, respectively. This formulation is not
unique to ENCO and other methods, such as SDI (Ke et al., 2019) and DCDI
(Brouillard et al., 2020) use similar belief matrices. We mostly consider ENCO
as clients’ LCDM, owing to its efficient implementation and scalability.

Algorithm 1 FED-CD: federated causal discovery from observational and
interventional data. We omit the description of basic functions for simplicity
of presentation.

Data: $D:=\\{D_{\mathcal{O}},\;D_{\mathcal{I}}\\}$; $\\{C_{1},\ldots
C_{K}|K:=\text{n\\_clients}\in\mathbb{N}\\}$;
$T:=\text{n\\_rounds}\in\mathbb{N}$

Return: $\tilde{G}:=\text{estimation of the underlying causal structure
(DAG)}$

for $T\;iterations$ do $\triangleright$ federated rounds

for $k=1$ to $K$ do

$\psi^{k}\leftarrow local\\_causal\\_discovery\\_method(D^{k},\;\psi)$
$\triangleright$ local learning (asynchronous and in parallel)

end for

$\psi\leftarrow
proximity\\_based\\_aggregation(\psi,\;\\{\psi^{1},\ldots,\psi^{k}\\})$
$\triangleright$ model aggregation stage

end for

$\tilde{G}\leftarrow binary\\_step(\psi^{*},\;threshold=0.5)$ $\triangleright$
estimated causal DAG

function proximity_based_aggregation$(\psi,\;\\{\psi^{1},\ldots,\psi^{k}\\})$

$adj\\_mat\leftarrow belief\\_to\\_adj(\psi)$ $\triangleright$ latest
structure from previous round’s $\psi$

for $X_{i}\to X_{j}$ in $edges(adj\\_mat)$ do $\triangleright$ calculate
reliability scores

for $k=1$ to $K$ do

for $X_{s}$ in $X_{C_{k}}^{\mathcal{I}}$ do

$\psi^{k}_{sp},\ldots,\psi^{k}_{ij}\leftarrow
get\\_path\\_beliefs(X_{s},X_{i}\rightarrow X_{j})$ $\triangleright$ retrieves
beliefs for all paths: $X_{s}\to X_{p}\to...\to X_{i}\to X_{j}$

$r_{ij}^{k}\leftarrow max(\psi^{k}_{sp}\cdot\ldots\cdot\psi^{k}_{ij}\cdot
m_{s}^{k},\;r_{ij}^{k})$ $\triangleright$ over paths like $X_{s}\to\ldots
X_{j}\to X_{i}$

end for

$\hat{r}_{ij}^{k}=e^{\beta r_{ij}^{k}}/{\sum_{k\in[K]}e^{\beta r_{ij}^{k}}}$
$\triangleright$ reliability scores to probabilities

end for

$\psi^{*}_{ij}=\sum_{k\in[K]}\hat{r}_{ij}^{k}\psi_{ij}^{k}$ $\triangleright$
final aggregation

end for

return $\psi^{*}$

The LCDM needs to consider the updated belief from the server into its local
discovery process. We achieve this by directly initializing the LCDM belief
matrices, $\gamma,\;\theta$ in the case of ENCO, by sampling from Bernoulli
distributions with $\psi$ parameters, and enforcing this knowledge as a
Lagrangian on LCDM’s internal optimization loss function. More specifically,
ENCO alternates between _distribution fitting_ and _graph fitting_ stages.
Distribution fitting trains a neural network $f_{\phi_{i}}$ to model $X_{i}$’s
conditional data distribution, $p(X_{i}\mid\mathbf{X_{-i}})$, on observational
data. Then, the graph fitting stage learns $\gamma$ and $\theta$ by minimizing

$\mathcal{L}=\mathbb{E}_{\hat{I}\sim
p_{I(I)}}\mathbb{E}_{\tilde{p}_{\hat{I}}(X)}\mathbb{E}_{p_{\gamma,\theta}(\zeta)}\left[\sum_{i=1}^{N}\mathcal{L}_{\zeta}(X_{i})\right]+\lambda_{sparse}\sum_{i=1}^{N}\sum_{j=1}^{N}\sigma(\gamma_{ij})\cdot\sigma(\theta_{ij}),$
(1)

where $p_{I}(I)$ and $\tilde{p}_{\hat{I}}(X)$ represent interventional data
distribution and $p_{\gamma,\theta}(\zeta)$ is the distribution over adjacency
matrices with
$\zeta_{ij}\sim\text{Ber}(\sigma(\gamma_{ij})\sigma(\theta_{ij}))$. Note that
$\sigma$ is the sigmoid function. Moreover, $\mathcal{L}_{\zeta}(X_{i})=-\log
f_{\phi_{i}}(X_{i};\zeta_{\cdot,i}\odot X_{-i})$, that is the negative log-
likelihood estimate of the variable $X_{i}$ with the distribution learned from
observational data. The last term is for sparsity regularization 111Details of
the optimization process and its tractability are provided in Lippe et al.
(2021). We integrate an extra differentiable regularization term to consider
the prior belief in Equation 1:

$\tilde{\mathcal{L}}=\mathcal{L}+\lambda_{prior}\sum_{i=1}^{N}\sum_{j=1}^{N}-\left[\zeta_{ij}\log(\psi_{ij})+(1-\zeta_{ij})\log(1-\psi_{ij})\right],$
(2)

where the log loss measures the agreement between the observed values in
$\zeta$ and the probabilities represented by $\psi$. Lower values of the log
loss indicate a better agreement between $\zeta$ and $\psi$, encouraging the
optimizer to find solutions that align with the prior belief. Hence, the
optimization process of LCDM is guided by the belief, especially when facing
uncertainty about the existence of an arbitrary $X_{i}\rightarrow X_{j}$. Such
uncertainties may arise when $D^{k}_{\mathcal{I}}$ lacks interventional
samples.

###  4.2 Naive belief aggregation

In a naive scenario, we assume $X^{k}_{\mathcal{I}}=X,\;\forall
k\in\\{1,...,K\\}$. Let $\psi^{k}$ be the prior belief obtained by each
client, after transmission of $\psi^{k}$ to the server, the aggregated belief
is calculated by a weighted average over $\psi^{k}$ matrices through

$\displaystyle\psi^{*}=\sum_{k}w_{k}\psi^{k},\quad
w_{k}=\frac{size(D^{k})}{size(D)},\quad\forall k\in\\{1,...,K\\},$ (3)

where $\psi_{ij}^{*}$ is the aggregated belief. Because
$w_{1}+w_{2}+\ldots,w_{K}=1$, each element of the weighted sum $\psi_{ij}$
represents the parameter of a Bernoulli distribution. However, without access
to same intervened covariates across clients’ interventional dataset, this
method fails to take client’s expertise into account. For instance, a client
with access to direct interventions on $X_{i}$ is typically more confident
about the edge probability of $X_{i}\rightarrow X_{j}$ than a client without
such knowledge.

###  4.3 Proximity-based belief aggregation

We know that $D^{k}$, contains interventions on the set of variables indicated
by $X^{\mathcal{I}}_{C_{k}}$. After applying LCDM to the local data,
$\psi^{k}_{ij}$ is the posterior probability of $X_{i}\rightarrow X_{j}$’s
existence, calculated by the client $C_{k}$. The question we are interested in
is how to aggregate posteriors $\psi^{k}_{ij}\;\forall k\in\\{1,...,K\\}$
while considering the clients’ knowledge of interventions.

###### Proposition 4.2

Consider $X_{i},X_{j}\in X$ to be two arbitrary variables in $G$. Then, for a
client $C_{k}$ admitting an intervened variable, $X_{s}\in
X^{k}_{\mathcal{I}}$, the reliability of $\psi^{k}_{ij}$, denoted by
$r^{k}_{ij}\in[0,1]$, is higher if $X_{i}\to X_{j}$ belongs to a path in $G$
descending from $X_{s}$.

Sketch of the proof. Note that if the described path exists, then every
variable $X_{i}$ in the described path is a descendant of $X_{s}$. The proof
is straightforward through causal factorization (Pearl, 2009),

$P(X)=\prod_{i=1}^{N}P(X_{i}\mid\text{Pa}(X_{i}))\Rightarrow
P(X_{i},X_{j})=P(X_{j}\mid\text{Pa}(X_{j}))\cdot
P(X_{i}\mid\text{Pa}(X_{i})).$ (4)

Assuming an intervention on $X_{s}$ and the truncated factorization formula
(causal factorization with applied intervention (Pearl, 2009)), we write
Equation 4 for $X_{i}\to X_{j}$ as

$\displaystyle P(X\backslash X_{s}\mid\text{{do}}(X_{s}=x))=\prod_{i\mid
X_{i}\neq X_{s}}P(X_{i}\mid\text{{pa}}_{i})\Big{|}_{x}$ (5)
$\displaystyle\Rightarrow
P(X_{i},X_{j}\mid\text{{do}}(X_{s}=x))=P(X_{i}\mid\text{{pa}}_{i})\Big{|}_{x}P(X_{j}\mid\text{{pa}}_{i})\Big{|}_{x},$
(6)

where it becomes clear that altering $X_{s}$’s distribution directly affects
its descendants.

In order to find $r_{ij}^{k}$ for each client, we inject hypothetical mass
$m_{s}^{k}$ to the node $X_{s}$ and let the mass flow in the DAG without its
magnitude being divided by the fan-out of each node. When passing through any
edge $X_{q}\to X_{p}$ on the way, the mass value is multiplied by $\psi_{qp}$
where $\psi_{qp}$ is the current aggregated belief in the network. The new
mass then continues to flow in the graph in the same manner. For instance, the
reliability score of the edge $X_{s}\to X_{p}$ is $\psi_{sp}^{k}m_{s}^{k}$.
The score becomes $\psi_{sp}^{k}\psi_{pq}^{k}m_{s}^{k}$ for $X_{p}\to X_{q}$.
Should different mass values reach the same node from multiple paths, we pick
the maximum regardless of the source variables. After a sufficient number of
steps, the mass flows through the entire DAG.

All clients perform the same process only once at each round, based on the
latest aggregated belief, $\psi$. Let the mass passing through an edge be the
reliability score for that edge. The process yields a set of reliability
scores, $\\{r^{1}_{ij},r^{2}_{ij},\ldots,r^{K}_{ij}\\}$, assigned to each edge
$X_{i}\to X_{j}$. We cannot directly use $r^{k}_{ij}$ values for aggregating
the local beliefs $\\{\psi_{ij}^{1},\psi_{ij}^{2},\ldots,\psi_{ij}^{K}\\}$ as
$\sum_{k}r_{ij}^{k}\neq 1$. To turn $r^{k}_{ij}$ values into probabilities, we
apply a softmax function and write the new aggregated belief as

$\psi_{ij}^{*}=\sum_{k\in[K]}\hat{r}_{ij}^{k}\psi_{ij}^{k},\quad\hat{r}_{ij}^{k}=\frac{e^{\beta
r_{ij}^{k}}}{\sum_{k\in[K]}e^{\beta r_{ij}^{k}}},\quad\forall
k\in\\{1,...,K\\},$ (7)

where $\psi_{ij}^{*}$ is the aggregated belief and $\beta>0$ is the
temperature parameter of the softmax function. The rate parameter $\beta$
controls the sensitivity of the distribution of $\hat{r}_{ij}^{k}$ to the
difference of the reliability scores calculated by different clients. This
procedure has the following parameters:

  * •

$m_{i}^{k},\;\forall i\in\\{i\mid X_{i}\in X^{k}_{\mathcal{I}}\\}$, the
initial mass of $C_{k}$. We set this parameter proportionate to the number of
interventional samples for $X_{i}$ available in $D^{k}$.

  * •

$\beta$, the softmax temperature, optimized by a grid search over a reasonable
range.

Algorithm 1 presents FED-CD and our novel proximity-based aggregation. We
conclude this section by providing a toy example to illustrate the proposed
belief aggregation.

###### Example 4.2.1

Let’s consider a 2-client setup. $C_{1}$ has interventional samples on $X_{1}$
(left), whereas $C_{2}$ holds interventional data on $X_{3}$ (right). We
assume the clients have an equal number of samples, $m_{1}^{1}=m_{2}^{3}=1$,
and $\beta=2$ and $\psi_{ij}$ acquired from last aggregation are written on
the edges. Edges with $\psi_{ij}<0.5$ are omitted for simplicity of
visualization.

X1X2X3X4X5$C_{1}:m_{1}^{1}=1$X60.60.60.50.90.80.7X1X2X3X4X5$C_{2}:m_{2}^{3}=1$X60.50.50.80.90.50.5

The flow of mass for $C_{1}$ and $C_{2}$ yields:

$\displaystyle r_{5,6}^{1}=0.6\cdot 0.9\cdot 0.8\cdot m_{1}^{1}=0.432,\quad
r_{5,6}^{2}=0.9\cdot 0.5\cdot m_{3}^{2}=0.540\Rightarrow
R_{5,6}=\\{0.432,0.540\\}.$

Then, the final reliability score of $C_{1}$ and $C_{2}$ for $X_{5}\to X_{6}$
is calculated by:

$\displaystyle\hat{r}_{5,6}^{1}=\frac{e^{(2\times 0.432)}}{e^{(2\times
0.432)}+e^{(2\times 0.540)}}\approx
0.446,\quad\hat{r}_{5,6}^{2}=\frac{e^{(2\times 0.540)}}{e^{(2\times
0.540)}+e^{(2\times 0.432)}}\approx 0.554.$

##  5 Experiments and Evaluation

We construct multiple federated setups to demonstrate the efficacy of FED-CD.
We compare FED-CD results with the centralized approach, and the performance
of a non-collaborating client. Next, we analyze naive averaging-based and
proximity-based aggregation methods, especially elaborating on the
shortcomings of the former facing vertical data distributions. We conclude by
comparing FED-CD to methods in the literature on real-world and synthetic
graphs, and analyzing its scalability.

###  5.1 Experimental setup

#### DAGs

For synthetic data, we employ the random Erdős–Rényi (ER) model to pick
experimental DAGs of size $d=20$, where each edge is added with an independent
pre-determined probability from the others (Erdös and Rényi, 2011).
Experiments are conducted for $ER-n$ graphs where $n\in\\{1,2,4,6\\}$. Based
on the definition, the value of $n.d$ is equal to the expected number of
edges. For the real-world graphs in the last section, we turn to _bnlearn_
package (Scutari, 2009), and pick Sachs (Sachs et al., 2005), Alarm (Beinlich
et al., 1989), and Asia (Lauritzen and Spiegelhalter, 1988) DAGs from the
dataset.

#### Dataset

The global dataset, $D$, is generated based on the previously sampled DAGs,
and by employing the synthetic sample generation mechanism similar to Ke et
al. (2019). The process utilizes randomly initialized neural networks – MLPs
with two layers with categorical inputs – to model the ground-truth
conditional distributions. We sample categorical data with 10 categories in
different sizes for both interventional and observational sets. For all
experiments, the observational data is horizontally distributed among clients.
However, a client may receive interventional samples for all intervened
variables (horizontal) or only a subset of those variables (vertical). Further
details about the data generation process appear on Appendix B.

#### Evaluation

We compare the outcome of each setup to the ground truth by calculating the
Structural Hamming Distance (SHD) between the two. Note that the outcome for
FED-CD is calculated by applying a binary step function (threshold = 0.5) to
the most recent belief matrix. We then run each experiment for 20 random
seeds, and report the average and confidence interval of the SHD results in
all the plots and tables.

Figure 2: Our method is rigorously compared against a centralized approach and
an isolated client, taking into account the size of the interventional data.
In the centralized approach, the entire dataset is available, while in both
FED-CD and the isolated client, each client is limited to only half of $D$
distributed horizontally. This comprehensive comparison shows promising
insights into the performance of FED-CD against a centralized approach. Figure
3: We apply the proximity-based and naive aggregation methods to vertically
distributed $D_{\mathcal{I}}$. Each curve corresponds to a 5-client setup,
where clients have access to disjoint and covering subsets of vertically
distributed interventional dataset, i.e., local datasets contain
interventional samples on only 4 out of 20 dataset features. The x-axis shows
the size of $D_{\mathcal{I}}$.

###  5.2 Horizontal data distribution

We start with a scenario where both $D_{\mathcal{O}}$ and $D_{\mathcal{I}}$
are split horizontally. Figure 2 presents multiple experiments with our method
while employing the proximity-based aggregation (Section 4.3). The results
demonstrate that two collaborative clients clearly outperform an isolated
client. Moreover, it is evident that FED-CD can mostly discover a DAG with the
same precision as the centralized approach. Both comparisons suggest that the
aggregation method is effectively and privately exchanging the knowledge of
local dataset, causing improvement even in a simple horizontal distribution.
Note that as the size of $D_{\mathcal{I}}$ increases in Figure 2, the results
of the isolated client come closer to FED-CD (and to the centralized
approach), as each client now holds adequate samples to find the underlying
DAG in a non-collaborative and centralized fashion.

###  5.3 Vertical distribution of interventional data

To highlight the critical role of interventional data in federated discovery,
we turn to vertical distribution of $D_{\mathcal{I}}$, which limits clients’
intervened variables to a subset of dataset features. From this angle, Figure
3 compares naive averaging-based aggregation method to the more sophisticated
proximity-based aggregation. The results suggest that averaging-based
approaches fail to discover the true structure, as they disregard clients’
unique access to interventions.

Figure 4: We compare three setups: non-collaborative clients A and B with
access to vertical subsets of $D_{\mathcal{I}}$, and FED-CD where the same A
and B clients now share their knowledge of the underlying structure. The
results demonstrate the advantage of FED-CD in leveraging insights from both
clients, highlighting its ability to harness diverse information, even from
clients with access to fewer intervened covariates.

We further showcase the capability of proximity-based aggregation through in
Figure 4, where two isolated clients are compared against a 2-client federated
setup, where these two clients start to collaborate through FED-CD. We observe
a meaningful improvement as a result of collaboration. Moreover, Figure 4
illustrates that a client with access to the majority of interventional data
still benefits from collaborating with the less significant client. This is
justified as the second client holds key interventional samples critical for a
more precise DAG discovery, and demonstrates the way our aggregation stage
manages to reflect the opinion of such clients.

###  5.4 Performance with more collaborating clients

To assess the effect of increasing the number of clients, we performed two
series of experiments in Figure 5. For the first setup, new clients join FED-
CD by bringing their own local dataset. Therefore, the size of
$D_{\mathcal{I}}$ increases upon adding a new client, and more information
becomes available in the network, while $D^{k}$ have a fixed size. Then, we
keep the $size(D_{\mathcal{I}})$ constant, and then increase the number of
clients. Each new client receives a share by joining the setup, leaving the
others with fewer samples.

The results implicate that FED-CD experiences higher precision when clients
with new information are added to the collaborative environment. Each client
brings fresh samples, which in turn helps the aggregation stage to learn more
accurate existence probabilities. However, even in the case of
$D_{\mathcal{I}}$’s further division by adding new clients, FED-CD manages to
perform consistently. Such results demonstrate that our aggregation technique
is capable of efficiently gathering data from an increasing number of clients
with less information, without having a detrimental effect on the overall
efficiency. The only breaking point happens when clients are left with so few
samples, that even the LCDM itself fails to converge to a local graph (see
convergence conditions in Lippe et al. (2021)).

Figure 5: Effect of having more clients participate in FED-CD for different
$D_{\mathcal{I}}$ sizes. The discovered DAG is more accurate where
$size(D_{\mathcal{I}})$ increases with the addition of each new client (first
row). FED-CD manages to sustain the knowledge when the fixed-size
$D_{\mathcal{I}}$ is further divided by the increase in number of clients
(second row).

###  5.5 Baselines and generalization

We compare our setup against some well-known methods for DAG discovery by
leveraging both interventional and observational data. From the methods listed
in Guo et al. (2020), we pick GIES (Hauser and Bühlmann, 2012), IGSP (Wang et
al., 2017), and DCDI (Brouillard et al., 2020), apply them in a centralized
manner to $\\{D_{\mathcal{O}},D_{\mathcal{I}}\\}$, and report the results. The
size of global datasets is the same for all methods, and is divided vertically
between clients in FED-CD. The results indicate the capability of FED-CD in
efficient knowledge aggregation, considering that each client has access to
less information compared to centralized approaches, yet still the setup as a
whole manages to yield precise discoveries. The last three graphs are adopted
from _bnlearn_ (Scutari, 2010).

Table 1: Comparison between FED-CD and centralized baselines for synthetic and
real-world graphs

DAG Type | ER-1 | ER-2 | ER-4 | ER-6 | Sachs | Alarm | Asia
---|---|---|---|---|---|---|---
GIES | $14.4\pm 2.1$ | $46.9\pm 0.8$ | $127.3\pm 1.6$ | $168.8\pm 0.7$ | $21.3\pm 4.3$ | $43.2\pm 7.5$ | $13.8\pm 3.1$
IGSP | $11.8\pm 2.8$ | $43.7\pm 0.7$ | $118.5\pm 0.4$ | $152.8\pm 0.1$ | $14.4\pm 3.4$ | $25.1\pm 1.4$ | $12.6\pm 2.7$
DCDI | $3.0\pm 1.1$ | $24.3\pm 2.1$ | $31.4\pm 8.6$ | $43.0\pm 12.3$ | $6.9\pm 1.2$ | $36.8\pm 2.3$ | $6.1\pm 1.6$
ENCO | $1.9\pm 0.1$ | $4.3\pm 0.1$ | $5.7\pm 0.2$ | $7.8\pm 0.3$ | $0.5\pm 0.4$ | $13.8\pm 2.5$ | $1.9\pm 0.1$
FED-CD (2-clients) | $2.6\pm 0.6$ | $3.9\pm 0.4$ | $4.1\pm 0.1$ | $5.5\pm 0.3$ | $0.9\pm 0.3$ | $14.2\pm 0.8$ | $1.5\pm 0.6$
FED-CD (4-clients) | $2.3\pm 0.4$ | $3.3\pm 0.7$ | $4.6\pm 0.6$ | $5.7\pm 0.4$ | $0.8\pm 0.5$ | $13.9\pm 3.1$ | $1.7\pm 0.4$

##  6 Limitations and Future Work

#### Limited data

We observed that in scenarios where each client has a limited local dataset,
the DAG discovered by FED-CD is still better than individual estimates of
clients but is not as promising as a centralized approach. Shortage in data
samples could appear devastating when crossing a certain threshold. The
threshold varies across different LCDMs and underlying data generation
processes; therefore, future work could propose a generic method to yield this
threshold.

#### Computational complexity

Although running clients’ LCDM in parallel potentially reduces the overall
computational time compared to a centralized approach, it can pose a higher
computational load depending on the nature of the learning method. For
instance, if the computational cost of the LCDM is linear regarding data and
the overheads are negligible, then the computational load remains the same as
a centralized approach. On the other hand, having a large overhead aside from
dataset size leads to higher computational load compared to centralized
approaches.

#### Acyclicity

One must ensure that the adjacency matrix obtained after several rounds of
aggregation is still a DAG. Pruning the adjacency matrix corresponding to the
final estimation is one option. Yet, future work could integrate extra
constraints in the aggregation process to ensure acyclicity.

##  7 Conclusion

We proposed a novel federated DAGs discovery from a distributed dataset to
avoid sharing local and private samples to preserve clients’ privacy.
Utilizing interventional data on top of observational samples, we can recover
the underlying causal structure with high precision, and handle both vertical
and horizontal distribution of interventional data without direct exchange of
information. Through comprehensive experiments, we showcased FED-CD’s
scalability and performance compared to centralized methods. Most notably, we
employ a naive aggregation approach to highlight the advantage of aggregation
with attention to clients’ interventions. It is evident that FED-CD can be
employed to effectively learn causal structures from decentralized data, while
adhering to privacy concerns.

##  8 Reproducibility

To ensure the reproducibility of the results presented earlier in the paper,
we have published the FED-CD’s source code
222https://www.github.com/aminabyaneh/fed-cd, accompanied by the essential
reproducibility instructions that yield the plots depicted in the paper,
provided in the main README.md file.

## References

  * Beinlich et al. (1989) Ingo A Beinlich, Henri Jacques Suermondt, R Martin Chavez, and Gregory F Cooper.  The alarm monitoring system: A case study with two probabilistic inference techniques for belief networks.  In _AIME 89_ , pages 247–256. Springer, 1989.
  * Brouillard et al. (2020) Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin.  Differentiable causal discovery from interventional data.  _Advances in Neural Information Processing Systems_ , 33:21865–21877, 2020.
  * Erdös and Rényi (2011) Paul Erdös and Alfréd Rényi.  On the evolution of random graphs.  In _The structure and dynamics of networks_ , pages 38–82. Princeton University Press, 2011.
  * Gao et al. (2021) Erdun Gao, Junjia Chen, Li Shen, Tongliang Liu, Mingming Gong, and Howard Bondell.  Federated causal discovery.  _arXiv preprint arXiv:2112.03555_ , 2021.
  * Glancy (2012) Dorothy J Glancy.  Privacy in autonomous vehicles.  _Santa Clara L. Rev._ , 52:1171, 2012.
  * Gou et al. (2007) Kui Xiang Gou, Gong Xiu Jun, and Zheng Zhao.  Learning bayesian network structure from distributed homogeneous data.  In _Eighth acis international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing (snpd 2007)_ , volume 3, pages 250–254. IEEE, 2007.
  * Guo et al. (2020) Ruocheng Guo, Lu Cheng, Jundong Li, P Richard Hahn, and Huan Liu.  A survey of learning causality with data: Problems and methods.  _ACM Computing Surveys (CSUR)_ , 53(4):1–37, 2020\.
  * Hauser and Bühlmann (2012) Alain Hauser and Peter Bühlmann.  Characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs.  _The Journal of Machine Learning Research_ , 13(1):2409–2464, 2012.
  * Kalainathan et al. (2018) Diviyan Kalainathan, Olivier Goudet, Isabelle Guyon, David Lopez-Paz, and Michèle Sebag.  Structural agnostic modeling: Adversarial learning of causal graphs.  _arXiv preprint arXiv:1803.04929_ , 2018.
  * Ke et al. (2019) Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio.  Learning neural causal models from unknown interventions.  _arXiv preprint arXiv:1910.01075_ , 2019.
  * Lauritzen and Spiegelhalter (1988) Steffen L Lauritzen and David J Spiegelhalter.  Local computations with probabilities on graphical structures and their application to expert systems.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 50(2):157–194, 1988.
  * Li et al. (2020) Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith.  Federated learning: Challenges, methods, and future directions.  _IEEE Signal Processing Magazine_ , 37(3):50–60, 2020.
  * Lippe et al. (2021) Phillip Lippe, Taco Cohen, and Efstratios Gavves.  Efficient neural causal discovery without acyclicity constraints.  _arXiv preprint arXiv:2107.10483_ , 2021.
  * Mian et al. (2023) Osman Mian, David Kaltenpoth, Michael Kamp, and Jilles Vreeken.  Nothing but regrets—privacy-preserving federated causal discovery.  In _International Conference on Artificial Intelligence and Statistics_ , pages 8263–8278. PMLR, 2023.
  * Na and Yang (2010) Yongchan Na and Jihoon Yang.  Distributed bayesian network structure learning.  In _2010 IEEE International Symposium on Industrial Electronics_ , pages 1607–1611. IEEE, 2010.
  * Ng and Zhang (2021) Ignavier Ng and Kun Zhang.  Towards federated bayesian network structure learning with continuous optimization.  _arXiv preprint arXiv:2110.09356_ , 2021.
  * Papernot et al. (2016) Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman.  Towards the science of security and privacy in machine learning.  _arXiv preprint arXiv:1611.03814_ , 2016.
  * Pearl (2009) Judea Pearl.  _Causality_.  Cambridge university press, 2009.
  * Sachs et al. (2005) Karen Sachs, Omar Perez, Dana Pe’er, Douglas A Lauffenburger, and Garry P Nolan.  Causal protein-signaling networks derived from multiparameter single-cell data.  _Science_ , 308(5721):523–529, 2005.
  * Scutari (2009) Marco Scutari.  Learning bayesian networks with the bnlearn r package.  _arXiv preprint arXiv:0908.3817_ , 2009.
  * Scutari (2010) Marco Scutari.  Learning bayesian networks with the bnlearn R package.  _Journal of Statistical Software_ , 35(3):1–22, 2010.  doi: 10.18637/jss.v035.i03.
  * Spirtes (2010) Peter Spirtes.  Introduction to causal inference.  _Journal of Machine Learning Research_ , 11(5), 2010.
  * Spirtes and Zhang (2016) Peter Spirtes and Kun Zhang.  Causal discovery and inference: concepts and recent methodological advances.  In _Applied informatics_ , volume 3, pages 1–28. SpringerOpen, 2016\.
  * Vandenbroucke et al. (2016) Jan P Vandenbroucke, Alex Broadbent, and Neil Pearce.  Causality and causal inference in epidemiology: the need for a pluralistic approach.  _International journal of epidemiology_ , 45(6):1776–1786, 2016.
  * Vowels et al. (2021) Matthew J Vowels, Necati Cihan Camgoz, and Richard Bowden.  D’ya like dags? a survey on structure learning and causal discovery.  _arXiv preprint arXiv:2103.02582_ , 2021.
  * Wang et al. (2017) Yuhao Wang, Liam Solus, Karren Dai Yang, and Caroline Uhler.  Permutation-based causal inference algorithms with interventions.  _arXiv preprint arXiv:1705.10220_ , 2017.
  * Yang et al. (2018) Karren Yang, Abigail Katcoff, and Caroline Uhler.  Characterizing and learning equivalence classes of causal dags under interventions.  In _International Conference on Machine Learning_ , pages 5541–5550. PMLR, 2018.
  * Ye et al. (2022) Qiaoling Ye, Arash A Amini, and Qing Zhou.  Distributed learning of generalized linear causal networks.  _arXiv preprint arXiv:2201.09194_ , 2022.
  * Yu et al. (2019) Yue Yu, Jie Chen, Tian Gao, and Mo Yu.  Dag-gnn: Dag structure learning with graph neural networks.  In _International Conference on Machine Learning_ , pages 7154–7163. PMLR, 2019.
  * Zheng et al. (2018) Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric P Xing.  Dags with no tears: Continuous optimization for structure learning.  _arXiv preprint arXiv:1803.01422_ , 2018.

##  Appendix A Supporting Experiments

We present a comprehensive set of additional experiments with the FED-CD
framework, aimed to put the proposed framework to test from a variety of
angles.

###  A.1 Performance per federated Rounds

Figure 6 experiments demonstrate the way FED-CD performs after each federated
round. These experiments are conducted for two different dataset sizes, and
are a special case of the plots shown in the main experiments. The plots
indicate the framework’s improvement at the end of each round (aggregation
step), and the resulting final convergence after a few rounds.

Figure 6: We provide a closer look into the gradual improvement at each
federated round. Our method is compared against a centralized approach and an
isolated client for two $D_{\mathcal{I}}$ sizes: 4 (first row) and 20 (second
row). The centralized approach has access to the entire dataset, while each
client in FED-CD and the isolated client all have access to half of the
dataset. The plots demonstrate that FED-CD is closely following a centralized
approach at each federated round. Figure 7: Evaluation of our approach when
using SDI as the LCDM, instead of ENCO. We use structured graphs of size 10
here, and the improvement caused by the federated setup is still observable
for all structured graph types. We need to reduce the graph size due to SDI’s
computational time to achieve tractable experiments.

###  A.2 Generalization to other LCDM candidates

We believe that other continuous-optimization based structure discovery
methods can be employed as the LCDM in our approach if modifications explained
in our Methodology are applicable. For instance, in Figure 7, we switch the
LCDM from ENCO to SDI, a work on causal discovery investigated in our
literature review. The results still indicate an improvement when employing a
federated setup. Note that due to the heavy computational load of SDI, we are
only experimenting on structured graphs with 10 nodes. For details about the
underlying data generator graphs of these experiments, please refer to Section
A.4.

###  A.3 Investigating the average edge entropy

Consensus refers to the collective agreement among processes in a distributed
setup to converge on a specific value, achieved through the voting of each
process. Consensus plays a crucial role in federated learning. In this
decentralized setting, achieving consensus ensures that all participating
clients contribute to the model update in a synchronized manner, leading to a
consistent and accurate global model. Consensus helps address the challenge of
model divergence, where variations in local data and training conditions
across clients can cause the models to drift apart. By reaching consensus,
federated learning can mitigate the effects of data heterogeneity and ensure
that the aggregated model represents a shared understanding of the data.

Figure 8: Average edge entropy for all the elements in the clients’ adjacency
matrix for a 10-client federated setup is plotted here. The results reveal a
substantial decrease in the values of edge entropy after each round, which
serves as an empirical sign of convergence and indicates that the federated
setup is progressing towards reaching a consensus.g clients over the rounds.
Figure 9: Average edge entropy for all the elements in the clients’ adjacency
matrix for a 10-client federated setup plotted for structured graphs. The
results reveal the same levels of decrease in the values of edge entropy after
each round as Figure 8.

In our study, we assess the consensus achieved by calculating the average
entropy (AE) for each edge. This measure is based on the information exchanged
between clients and the server at the end of each round. The empirical results
depicted in Figure 8 and Figure 9 showcase the dynamics of average entropy
reduction across different causal graphs as the rounds progress. Remarkably,
our proposed approach demonstrates the ability to attain consensus in most
cases within 10 rounds, although in certain scenarios, we observe a
substantial reduction in average entropy without reaching complete consensus.
These findings highlight the effectiveness of our approach in reducing
uncertainty and aligning the distributed processes towards a common
understanding.

###  A.4 Experiments on structured graphs

We conduct further experiments on a range of structured graphs to further
evaluate FED-CD. These graphs include the chain, bidiag, full, collider, and
jungle graphs. In the bidiag graphs, the probability of a node $X_{i}$ being
connected to its immediate parents $X_{i-1}$ and $X_{i-2}$ is defined as
$P(X_{i}=1|X_{i-1}=1,X_{i-2}=1)$. Similarly, the chain graph represents a
simplified version, where the probability of a node $X_{i}$ being connected to
its immediate parent $X_{i-1}$ is defined as $P(X_{i}=1|X_{i-1}=1)$. Collider
graphs involve a specific node $X$ having all other nodes as its parents, with
the probability of a node $X_{i}$ being connected to its parents being
$P(X_{i}=1|Pa(X_{i}))$. The full graph represents the densest connected graph
possible, where the probability of a node $X_{i}$ being connected to all
preceding nodes $X_{1},X_{2},\ldots,X_{i-1}$ is defined as
$P(X_{i}=1|X_{1}=1,X_{2}=1,\ldots,X_{i-1}=1)$. Lastly, the jungle graph
introduces a binary tree structure with additional connections to a node’s
parent’s parent. These experiments provide valuable insights into how our
approach performs under different graph configurations, capturing diverse
causal relationships and their complexities.

Figure 10 and Figure 11 show FED-CD performing consistently on datasets
generated by structured graphs. Note that all the graphs here have the same
size 20 as the random graphs, and the datasets are produced with the same
process of Lippe et al. [2021] and Ke et al. [2019] with the same sizes as the
main paper. The next plot, Figure 12, depicts the experimental results for
vertical distribution of $D_{\mathcal{I}}$ where the underlying data
generation process is built upon structured graphs, further verifying the
results in the main paper.

Figure 10: We provide a detailed analysis of the incremental improvement
achieved in each federated round for _structured graphs_. Our method is
compared against a centralized approach and an isolated client setting using
two different sizes of $D_{\mathcal{I}}$: 4 (first row) and 20 (second row).
In the centralized approach, the entire dataset is available, while in both
FED-CD and the isolated client, each client has access to only half of the
dataset. The plots clearly illustrate that FED-CD consistently tracks the
performance of the centralized approach in every federated round. Figure 11:
Our method is compared against a centralized approach and an isolated client
based on the size of interventional data. The centralized approach has access
to the entire dataset, while each client in FED-CD and the isolated client all
have access to half of the dataset. The results of comparison to the
centralized approach with structured graphs are aligned with that of the
random ER graphs. Figure 12: We compare three different setups with vertical
distribution of $D_{\mathcal{I}}$, this time considering structured graphs.
The setups are non-collaborative clients A and B with access to only a
vertical subset of $D_{\mathcal{I}}$, and a federated setup where both clients
collaborate through FED-CD. Evidently, FED-CD can take advantage of clients’
insight about the global structure, even from the less significant clients.

##  Appendix B Experiment Setup and Details

###  B.1 Dataset size and distribution

For the entire experiments, aside from the baselining table, we use $10k$
observational samples as $D_{\mathcal{O}}$ and $2k$ samples for
$D_{\mathcal{I}}$. For the baselining, however, we increase the size of
$D_{\mathcal{O}}$, $D_{\mathcal{I}}$ to $25k$ and $5k$ to make the experiments
compatible with other methods such as IGSP. These numbers correspond to the
size of $D$, that is, the global dataset yet to be distributed among several
clients. For example, in a 2-client setup, each client typically gets half of
the observational data. Depending on the vertical or horizontal split of the
interventional data, a client might get access to samples from all or part of
dataset features.

###  B.2 Graph and dataset generation

We adhere to the random Erdős–Rényi [Erdös and Rényi, 2011] model to generate
experimental DAGs of size $d=20$. Erdős–Rényi graphs, named after
mathematicians Paul Erdős and Alfréd Rényi, are a type of random graph model
that provides a simple and fundamental framework for studying random network
structures. These graphs are characterized by their random and independent
edge formation. Each edge is added with an independent pre-determined
probability from the others, and is modeled by $G(d=20,\;p)$. Hence, each edge
is included in the graph with probability $p$. The probability for generating
a graph that has $d$ nodes and $n$ edges is $p^{n}(1-p)^{{\binom{n}{2}}-n}$.

Our experiments are conducted for $ER-n$ graphs where $n\in\\{1,2,4,6\\}$. The
value of $m=n.d$ (where $n$ is the parameter in $ER-n$) is equal to the
expected number of edges. For example, as $d=20$ throughout the experiments,
an $ER-2$ DAG has an expected number of $m=40$ edges. Further, to determine
the orientation of the edges, we assume the causal ordering of $(X_{i},X_{j})$
to be oriented $X_{i}\rightarrow Xj$ if $i<j$, otherwise $X_{j}\rightarrow
X_{i}$.

Same as Lippe et al. [2021] and Ke et al. [2019], we employ categorical
variables of 10 categories each. The process utilizes randomly initialized
neural networks – MLPs with two layers with categorical inputs – to model the
ground-truth conditional distributions. For instance, when a variable $X_{i}$
has $M$ parents, $M$ embedding vectors are stacked and passed as an input. The
hidden size of the layers is 48, and their architecture uses a LeakyReLU
activation functions between layers and a softmax activation to find a
distribution for the output. More information about the process could be found
in the appendix of Lippe et al. [2021].