Abstract

Sibling fixed effects (FE) models are useful for estimating causal treatment
effects while offsetting unobserved sibling-invariant confounding. However,
treatment estimates are biased if an individual’s outcome affects their
sibling’s outcome. We propose a robustness test for assessing the presence of
outcome-to-outcome interference in linear two-sibling FE models. We regress a
gain-score—the difference between siblings’ continuous outcomes—on both
siblings’ treatments and on a pre-treatment observed FE. Under certain
restrictions, the observed FE’s partial regression coefficient signals the
presence of outcome-to-outcome interference. Monte Carlo simulations
demonstrated the robustness test under several models. We found that an
observed FE signaled outcome-to-outcome spillover if it was directly
associated with a sibling-invariant confounder of treatments and outcomes,
directly associated with a sibling’s treatment, or directly and equally
associated with both siblings’ outcomes. However, the robustness test
collapsed if the observed FE was directly but differentially associated with
siblings’ outcomes or if outcomes affected siblings’ treatments.

Acknowledgements: This work was supported by the Eunice Kennedy Shriver
National Institute for Child Health and Human Development through the
Population Research Center at the University of Texas-Austin (P2CHD042849) and
through the Center for Demography at the University of Wisconsin-Madison (T32
HD007014-42). This content is solely the responsibility of the author and does
not necessarily represent the official views of the National Institutes of
Health. I thank Felix Elwert for comments and discussions that informed this
manuscript. Conflicts of interest: none.

###### Contents

  1. 1 Introduction
  2. 2 Methods
    1. 2.1 Baseline model
    2. 2.2 Gain-score regression
    3. 2.3 Robustness test
  3. 3 Simulations
  4. 4 Discussion
  5. A Derivations for Graphical Models
    1. A.1 Baseline model without outcome-to-outcome interference
    2. A.2 Baseline model with outcome-to-outcome interference
  6. B Full Simulation Results
  7. C Post-Hoc Simulation
  8. D Main Simulation Code
  9. E Post-Hoc Simulation Code

##  1 Introduction

Sibling fixed effects (FE) models are commonplace in observational
epidemiologic research for estimating causal treatment effects while
controlling for unobserved sibling-invariant confounding, such as shared
genetic risk factors.1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 These models rely on
strong assumptions—in particular, an individual’s outcome cannot affect their
sibling’s treatment nor outcome.1, 5, 10, 11 Such assumptions are precarious
because siblings’ health are interdependent.12, 13 One can prevent outcome-to-
treatment interference by measuring all treatments before all outcomes within
sibships. However, study design cannot circumvent outcome-to-outcome
interference, thereby threatening the validity of any sibling FE analysis.
Statistical tests like regressions with negative controls can assess potential
bias,14, 15 although there is little guidance for their use in gauging
outcome-to-outcome interference in sibling comparison studies.

We explain a robustness test for assessing outcome-to-outcome interference in
linear two-sibling FE models. The method involves regressing a gain-score—the
difference in siblings’ outcomes—on both siblings’ treatments and on a pre-
treatment observed FE that is related to siblings’ treatments or outcomes.
Using graphical causal modeling and Monte Carlo simulations, we demonstrate
when the partial regression coefficient for the observed FE signals outcome-
to-outcome interference across several models.

##  2 Methods

###  2.1 Baseline model

Our baseline sibling FE model (Figure 1A) reflects a typical sibling
comparison design.5 Subscripts $i=\left\\{1,\ldots,N\right\\}$ and
$j=\left\\{1,2\right\\}$ denote cluster and sibling, respectively. $T_{ij}$ is
a binary or continuous treatment, $Y_{ij}$ is a continuous outcome, $U_{i}$ is
an unobserved FE (e.g., family health history), and $D_{i}=Y_{i2}-Y_{i1}$ is
the gain-score. Arrows and adjacent Greek letters denote causal effects:
$\delta$ is the treatment effect ($T_{ij}\rightarrow Y_{ij}$); $\psi$ is the
unobserved confounding effect on outcomes ($U_{i}\rightarrow Y_{ij}$); and
$\chi$ ($U_{i}\rightarrow T_{i1}$) and $\gamma$ ($U_{i}\rightarrow T_{i2}$)
are unobserved confounding effects on treatments. This model embeds two
assumptions. First, the treatment effect $\delta$ and the confounding effect
$\psi$ are equal for both siblings. Second, all effects are linear and
homogeneous. Both assumptions align with conventional linear FE models.5, 16,
17, 18, 19 For accessibility, we omitted sibling-specific covariates, which
are compatible with the baseline model and all subsequent models as long as
one can condition on them without loss of generality.

###  2.2 Gain-score regression

Following the baseline model, $\delta$ is our estimand. Regressing $Y_{ij}$ on
$T_{ij}$ will not identify $\delta$, covariate adjustment notwithstanding,
because $U_{i}$ confounds both variables. This motivates the utility of FE
estimators that isolate treatment effects while controlling for certain types
of unobserved confounding.5, 11, 16, 17, 18, 19 The gain-score is one such
estimator, and we demonstrate its mechanics and susceptibility to outcome-to-
outcome interference in simple linear regression. We regress the gain-score on
siblings’ treatments,

$D_{i}=b_{1}T_{i1}+b_{2}T_{i2}+e_{i},$ (1)

where $e_{i}$ is the residual. Through outcome differencing, the gain-score
estimator offsets unobserved confounding bias such that the treatment-
conditional covariance between $U_{i}$ and $D_{i}$ ($\sigma_{U_{i}D_{i}\cdot
T_{i1}T_{i2}}$) is zero, thereby isolating the treatment effect.11, 16, 17 The
baseline model includes four pathways that “transmit” bias from $U_{i}$ into
$D_{i}$:

  1. 1.

$U_{i}\rightarrow T_{i1}\rightarrow Y_{i1}\rightarrow D_{i}$: $-\chi\delta$

  2. 2.

$U_{i}\rightarrow T_{i2}\rightarrow Y_{i2}\rightarrow D_{i}$: $\gamma\delta$

  3. 3.

$U_{i}\rightarrow Y_{i1}\rightarrow D_{i}$: $-\psi$

  4. 4.

$U_{i}\rightarrow Y_{i2}\rightarrow D_{i}$: $\psi$

Covariate adjustment on $T_{i1}$ and $T_{i2}$ closes paths 1 and 2, and paths
3 and 4 cancel each other out exactly because the association that flows along
path 3 ($-\psi$) is equal to the negative value of the association that flows
along path 4 ($\psi$). Thus, $\sigma_{U_{i}D_{i}\cdot T_{i1}T_{i2}}=0$, and
$b_{2}=\delta$ identifies the treatment effect precisely (likewise,
$b_{1}=-\delta$). See Appendix for details.

The estimator’s validity relies on absent outcome-to-outcome interference.
Otherwise, we cannot identify $\delta$. The model in Figure 1B introduces
outcome-to-outcome interference $\eta$ ($Y_{i1}\rightarrow Y_{i2}$). There are
now two additional pathways from $U_{i}$ to $D_{i}$:

  1. 5.

$U_{i}\rightarrow T_{i1}\rightarrow Y_{i1}\rightarrow Y_{i2}\rightarrow
D_{i}$: $\chi\delta\eta$

  2. 6.

$U_{i}\rightarrow Y_{i1}\rightarrow Y_{i2}\rightarrow D_{i}$: $\psi\eta$

Covariate adjustment on $T_{i1}$ blocks path 5. However, path 6 is open as a
result of outcome-to-outcome interference, so $\sigma_{U_{i}D_{i}\cdot
T_{i1}T_{i2}}\neq 0$, thereby preventing treatment identification (i.e.,
$b_{2}\neq\delta$ and $b_{1}\neq-\delta$). See Appendix for details.

###  2.3 Robustness test

Given that outcome-to-outcome interference in sibling FE models biases
treatment identification, we may consider statistical tests for gauging this
source of bias. An ideal test is a gain-score regression that adjusts for
$U_{i}$:

$D_{i}=b_{1}T_{i1}+b_{2}T_{i2}+b_{U}U_{i}+e_{i}.$ (2)

Under the models in Figures 1A and 1B, $b_{U}=0$ indicates no outcome-to-
outcome spillover ($\eta=0$) because $\sigma_{U_{i}D_{i}\cdot
T_{i1}T_{i2}}=0$, whereas $b_{U}\neq 0$ indicates outcome-to-outcome spillover
($\eta\neq 0$) because $\sigma_{U_{i}D_{i}\cdot T_{i1}T_{i2}}\neq 0$. This
hypothetical test is straightforward but impossible; $U_{i}$ is unobserved, so
we cannot adjust for it. Indeed, adjustment for $U_{i}$ would render sibling
FE analysis obsolete—simply regress $Y_{ij}$ on $T_{ij}$ and $U_{i}$ to
identify $\delta$.

Still, we may measure cluster-level characteristics, raising the possibility
of adjusting for an observed FE in the gain-score regression. The model in
Figure 1C introduces $C_{i}$, a binary or continuous observed pre-treatment
FE, and $U_{i}^{\prime}$, an unobserved FE that is equivalent to $U_{i}$
without $C_{i}$. The dashed line between $U_{i}^{\prime}$ and $C_{i}$
indicates a direct and linear association that is equal to $\pi$. An example
of a potential observed FE is maternal nativity, which is a sibling-invariant
characteristic that is associated with family health history and pregnancy-
related outcomes in the United States.20, 21 Additionally, $C_{i}$ should
precede siblings’ treatments to prevent bias from conditioning on a mediator
variable between the treatment and outcome or from conditioning on a post-
treatment collider.22, 23

Under the model in Figure 1C, we repeat the gain-score regression and adjust
for $C_{i}$:

$D_{i}=b_{1}T_{i1}+b_{2}T_{i2}+b_{C}C_{i}+e_{i}.$ (3)

There are four pathways from $C_{i}$ to $D_{i}$:

  1. 1.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
T_{i1}\rightarrow Y_{i1}\rightarrow D_{i}$: $-\pi\chi\delta$

  2. 2.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
T_{i2}\rightarrow Y_{i2}\rightarrow D_{i}$: $\pi\gamma\delta$

  3. 3.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
Y_{i1}\rightarrow D_{i}$: $-\pi\psi$

  4. 4.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
Y_{i2}\rightarrow D_{i}$: $\pi\psi$

Covariate adjustment on siblings’ treatments blocks paths 1 and 2, and outcome
differencing offsets the associations from paths 3 and 4. Therefore,
$\sigma_{C_{i}D_{i}\cdot T_{i1}T_{i2}}=0$, so $b_{C}=0$.

We then consider the model in Figure 1D, which includes $C_{i}$ and outcome-
to-outcome interference. Two additional pathways link $C_{i}$ to $D_{i}$:

  1. 5.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
T_{i1}\rightarrow Y_{i1}\rightarrow Y_{i2}\rightarrow D_{i}$: $\chi\delta\eta$

  2. 6.

$C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}U^{\prime}_{i}\rightarrow
Y_{i1}\rightarrow Y_{i2}\rightarrow D_{i}$: $\pi\psi\eta$

While covariate adjustment closes path 5, path 6 remains open, so
$\sigma_{C_{i}D_{i}\cdot T_{i1}T_{i2}}\neq 0$ and $b_{C}\neq 0$.
Theoretically, this robustness test signals outcome-to-outcome interference in
a basic sibling FE model. We now turn to simulation analyses to exercise this
test in multiple settings.

##  3 Simulations

Monte Carlo simulations demonstrated the robustness test’s performance in the
model of Figure 1D and across its subsequent variations in Figure 2.24 Model
alterations included treatment-to-treatment interference, direct associations
between $C_{i}$ and siblings’ treatments or outcomes, treatment-to-outcome
interference, or outcome-to-treatment interference.

Our simulation model follows:

$U^{\prime}_{i},\upsilon_{c},\upsilon_{i1},\upsilon_{i2}\sim N(0,1)$
$\boldsymbol{\alpha}=\\{\lambda,\omega\\}$ $C_{i}=\begin{cases}0\ \text{if}\
\pi U^{\prime}_{i}+\upsilon_{c}\leq 1\\\ 1\ \text{if}\ \pi
U^{\prime}_{i}+\upsilon_{c}>1\end{cases}$ $T_{i1}=\begin{cases}0\ \text{if}\
\tau C_{i}+\chi U^{\prime}_{i}\leq-0.2\\\ 1\ \text{if}\ \tau C_{i}+\chi
U^{\prime}_{i}>-0.2\end{cases}$ $T_{i2}=\begin{cases}0\ \text{if}\ \phi
T_{i1}+\mu Y_{i1}+\nu C_{i}+\gamma U^{\prime}_{i}\leq 1\\\ 1\ \text{if}\ \phi
T_{i1}+\mu Y_{i1}+\nu C_{i}+\gamma U^{\prime}_{i}>1\end{cases}$ $Y_{i1}=\delta
T_{i1}+\kappa T_{i2}+\lambda C_{i}+\psi U^{\prime}_{i}+\upsilon_{i1}$
$Y_{i2}=\delta T_{i2}+\theta T_{i1}+\eta Y_{i1}+\boldsymbol{\alpha}C_{i}+\psi
U^{\prime}_{i}+\upsilon_{i2}$ $D_{i}=Y_{i2}-Y_{i1}$

For each sibling FE model, we ran two simulation sets, each consisting of two
simulations with 1,000 runs of 5,000 observations (sibling pairs). Simulations
varied on outcome-to-outcome interference ($\eta$) and on the association
between $C_{i}$ and $U_{i}^{\prime}$ ($\pi$):

  1. 1.

Simulation 1.1: $\eta=0$ and $\pi=0.5$;

  2. 2.

Simulation 1.2: $\eta=0.3$ and $\pi=0.5$;

  3. 3.

Simulation 2.1: $\eta=0$ and $\pi=0$;

  4. 4.

Simulation 2.2: $\eta=0.3$ and $\pi=0$.

Baseline parameters were fixed across simulations: $\delta=3$, $\chi=1$,
$\gamma=2$, and $\psi=5$. Unless specified, we set additional parameters to
zero. Within samples, we ran the gain-score regression in equation (3). We
conducted simulations in Stata Statistical Software: Release 16.25 See
Appendix for simulation code.

Table 1 displays simulation inputs and results for the observed FE, and
Appendix Table 1 contains full results. We first consider simulations 1.1 and
1.2. Under the model of Figure 1D, $\widehat{b_{C}}=0$ (95% confidence
interval: -0.105, 0.105) without outcome-to-outcome interference, and the
coverage probability—the percent of samples with confidence intervals that
overlapped zero—exceeded 95%. Conversely, $\widehat{b_{C}}=0.264$ (95%
confidence interval: 0.159, 0.370) with outcome-to-outcome interference, and
the coverage probability was <1%. These results persisted in the presence of
treatment-to-treatment interference (Figure 2A), direct associations between
$C_{i}$ and siblings’ treatments (Figure 2B), equivalent and direct
associations between $C_{i}$ and siblings’ outcomes (Figure 2C), and
treatment-to-outcome interference (Figure 2D). However, $\widehat{b_{C}}\neq
0$ with direct non-equivalent associations between $C_{i}$ and siblings’
outcomes (Figure 2E) or with outcome-to-treatment interference (Figure 2F),
regardless if outcome-to-outcome interference was present.

Results from simulations 2.1 and 2.2 indicate that the robustness test was
only valid under the models in Figures 2B and 2C if $\pi=0$; if $C_{i}$ and
$U_{i}^{\prime}$ were not directly associated, then $C_{i}$ must have been
directly associated with a treatment or directly and equally associated with
both outcomes for the robustness test to work. Otherwise, the test was invalid
because $C_{i}$ was not associated with any other variable (Figures 1D, 2A,
2D, and 2F) or because $C_{i}$ was differentially associated with siblings’
outcomes (Figure 2E).

In a post-hoc analysis, we re-ran all simulation sets under the model in
Figure 2A such that $C_{i}$ was directly associated with only one sibling’s
treatment: $\nu=1.5$ ($C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i2}$) and
$\tau=0$ ($C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i1}$), or $\nu=0$ and
$\tau=0.7$. Results were consistent with main findings, so $C_{i}$ is a valid
observed FE in these settings (see Appendix).

##  4 Discussion

In this paper, we demonstrated a simple robustness test for assessing bias
from outcome-to-outcome interference in linear two-sibling FE models. The test
involves regressing a gain-score on a pre-treatment observed FE and on both
siblings’ treatments. For the test to work, the observed FE must be directly
associated with an sibling-invariant confounder on treatments and outcomes,
directly associated with at least one sibling’s treatment, or directly and
equally associated with both siblings’ outcomes. Further, the robustness test
collapses if the observed FE is directly and differentially associated with
siblings’ outcomes or if an individual’s outcome affects their sibling’s
treatment, although the latter violates the treatment estimation in a sibling
FE model regardless.1, 5, 10, 11 Failure to meet this criterion does not
necessarily warrant a variable’s exclusion from a FE regression model. For
example, the regression should adjust for an observed cluster-level variable
that differentially affects siblings’ treatments and outcomes because it is a
confounder, but it is not a valid observed FE for our robustness test.

Finding an observed FE for the robustness test is data dependent. Birth
registries are commonplace in sibling FE analyses2, 3, 4, 7, 8, 11 and may
provide several candidate variables. For instance, United States birth records
record maternal demographic information that are fixed and related to family
health history, such as nativity and ethnicity.20, 21, 26 Other sources may
not have a similarly wide breadth of sibling-shared characteristics.
Additionally, these shared characteristics may have complex links to siblings’
characteristics, and researchers must interrogate how candidate observed FEs
relate the treatments and outcomes. For this reason, we considered only pre-
treatment variables as candidate observed FEs for the robustness test. Recent
methodological guidance discourages conditioning on post-treatment variables
to minimize bias in treatment estimates.22, 23 In our sibling FE model,
conditioning on an observed FE that is causally affected by both siblings’
treatments would induce such bias in the treatment estimate, so it is also
possible that it would invalidate the observed FE’s performance in the
robustness test. Using an observed FE that is measured prior to siblings’
treatments alleviates this concern. More broadly, we recommend graphical
modeling to determine a variable’s usefulness for the robustness test27, 28
and, when possible, running multiple robustness tests with valid observed FEs
to thoroughly screen possible outcome-to-outcome interference.

We also highlight the flexibility of this robustness test. Despite relying on
gain-scores, it is applicable to studies with different FE strategies, such as
dummy score regression17 or conditional maximum likelihood estimation.5 The
data may require reshaping such that each observation consists of a dyadic
pair. Furthermore, the robustness test may be used in studies of other types
of dyadic pairs and not only in sibling comparison analyses.

There are some limitations to this paper. We focused on settings with linear
homogenous effects. Our test may be invalid in models with nonlinear
relationships, including those with binary outcomes. Furthermore, we
restricted our attention to dyadic sibling models, although this test may work
in data with clusters of three or more siblings—for example, one could run the
robustness test on randomly selected pairs from sibships. Finally, we omitted
models with shared mediator or collider variables, which can induce bias in
sibling FE models.29 These factors may also invalidate our robustness test.

In sum, our gain-score robustness test is a useful tool for validating two-
sibling FE models. Researchers who conduct sibling comparison studies should
take advantage of this test to evaluate potential bias from outcome-to-outcome
interference.

## References

  * 1 Frisell T, Öberg S, Kuja-Halkola R, Sjölander A. Sibling comparison designs: bias from non-shared confounders and measurement error. Epidemiol. 2012;23:713-720.
  * 2 Khashan AS, Kenny LC, Lundholm C, Kearney PM, Gong T, Almqvist C. Mode of obstetrical delivery and type 1 diabetes: a sibling design study. Pediatrics. 2014;134:e806-813.
  * 3 Khashan AS, Kenny LC, Lundholm C, et al. Gestational age and birth weight and the risk of childhood type 1 diabetes: a population-based cohort and sibling design study. Diabetes Care. 2015;38:2308-2315.
  * 4 Hvolgaard Mikkelsen S, Olsen J, Bech BH, Obel C. Parental age and attention-deficit/hyperactivity disorder (adhd). Int J Epidemiol. 2016;46:409-420.
  * 5 Sjölander A, Frisell T, Kuja-Halkola R, Öberg S, Zetterqvist J. Carryover effects in sibling comparison designs. Epidemiol. 2016;27:852-858.
  * 6 Hanley GE, Hutcheon JA, Kinniburgh BA, et al. Interpregnancy interval and adverse pregnancy outcomes: an analysis of successive pregnancies. Obstet Gynecol. 2017;129:408-415.
  * 7 Axelsson PB, Clausen TD, Petersen AH, et al. Relation between infant microbiota and autism?: results from a national cohort sibling design study. Epidemiol. 2019;30:52-60.
  * 8 Mallinson DC, Larson A, Berger LM, Grodsky E, Ehrenthal DB. Estimating the effect of Prenatal Care Coordination in Wisconsin: a sibling fixed effects analysis. Health Serv Res. 2020;55:82-93.
  * 9 Petersen AH, Lange T. What is the causal interpretation of sibling comparison designs? Epidemiol. 2020;31:75-81.
  * 10 Sibling-comparison designs, are they worth the effort? Am J Epidemiol. 2021;190:738-741.
  * 11 Mallinson DC, Elwert FE. Estimating sibling spillover effects with unobserved confounding using gain-scores. arXiv, doi:2102.11150v3, 6 May 2021, preprint: not peer reviewed.
  * 12 Feinberg ME, Solmeyer AR, McHale SM. The third rail of family systems: sibling relationships, mental and behavioral health, and preventive intervention in childhood and adolescence. Clin Child Fam Psychol Rev. 2012;15:43–57.
  * 13 De Neve JW, Kawachi I. Spillovers between siblings and from offspring to parents are understudied: a review and future directions for research. Soc Sci Med. 2017;183:56-61.
  * 14 Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiol. 2010;21:383-388.
  * 15 Cunningham S. Causal Inference: The Mixtape. New Haven: Yale University Press, 2021.
  * 16 Kim Y, Steiner PM. Gain scores revisited: a graphical models perspective. Sociol Methods Res. 2021;50:1353-1375.
  * 17 Kim Y, Steiner PM. Causal graphical views of fixed effects and random effects models. Br J Math Stat Psychol. 2021;74:165-183.
  * 18 Gunasekara FI, Richardson K, Carter K, Blakely T. Fixed effects analysis of repeated measures data. Int J Epidemiol. 2014;43:264-269.
  * 19 Imai K, Kim IS. When should we use unit fixed effects regression models for causal inference with longitudinal data? Am J Pol Sci. 2019;63:467-490.
  * 20 Qin C, Gould JB. Maternal nativity status and birth outcomes in Asian immigrants. J Immigrant Minority Health. 2010;12:798-805.
  * 21 Creanga AA, Berg CJ, Syverson C, Seed K, Bruce FC, Callaghan WM. Race, ethnicity, and nativity differentials in pregnancy-related mortality in the United States. Obstet Gynecol. 2012;120:361-268.
  * 22 Elwert F, Winship C. Endogenous selection bias: the problem of conditioning on a collider variable. Annu Rev Soc. 2014;40:31-53.
  * 23 VanderWeele TJ. Explanation in Causal Inference: Methods for Mediation and Interaction. New York: Oxford University Press, 2015.
  * 24 Adkins LC, Gade MN. Monte Carlo experiments using Stata: a primer with examples. Adv Econ. 2012;30:429-77.
  * 25 StataCorp. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC; 2019.
  * 26 National Center for Health Statistics. Revisions of the U.S. Standard Certificates and Reports. https://www.cdc.gov/nchs/nvss/revisions-of-the-us-standard-certificates-and-reports.htm. Accessed September 3, 2021.
  * 27 Pearl J. Linear models: a useful “microscope” for causal analysis. J Causal Inference. 2013;1:155-170.
  * 28 Tennant PWG, Murray EJ, Arnold KF, et al. Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations. Int J Epidemiol. 2021;50:620-632.
  * 29 Sjölander A, Zetterqvist J. Confounders, mediators, or colliders. Epidemiol. 2017;28:540-547.

Assessing Outcome-to-Outcome Interference in Sibling Fixed Effects Models
Appendix

##  Appendix A Derivations for Graphical Models

###  A.1 Baseline model without outcome-to-outcome interference

We derive the gain-score regression for the baseline model in Figure 1A to
demonstrate how the regression identifies $\delta$ using Wright’s path-tracing
rules.27 The method following applies to settings with variables that are
standardized to mean zero and unit variance: between any two variables, the
covariance ($\sigma$) is equal to the sum of products of path coefficients
along unconditionally open paths that connect the variables. We can use these
values to compute partial regression coefficients from linear graphical
models. This method is applicable to settings with non-standardized variables
with additional computation. For ease of demonstration, we assume models with
standardized variables. See Pearl (2013) for additional details.27

Our gain-score regression follows:

$D_{i}=b_{1}T_{i1}+b_{2}T_{i2}+e_{i}.$

We want to derive $b_{1}$ (the partial regression coefficient for $T_{i1}$)
and $b_{2}$ (the partial regression coefficient for $T_{i2}$). To do this, we
use the following formulas:

$\displaystyle b_{1}$
$\displaystyle=\frac{\sigma_{D_{i}T_{i1}}-\sigma_{D_{i}T_{i2}}*\sigma_{T_{i1}T_{i2}}}{1-{\sigma_{T_{i1}T_{i2}}}^{2}}$
$\displaystyle b_{2}$
$\displaystyle=\frac{\sigma_{D_{i}T_{i2}}-\sigma_{D_{i}T_{i1}}*\sigma_{T_{i1}T_{i2}}}{1-{\sigma_{T_{i1}T_{i2}}}^{2}}$

Thus, we need to compute $\sigma_{D_{i}T_{i1}}$, $\sigma_{D_{i}T_{i2}}$, and
$\sigma_{T_{i1}T_{i2}}$. Following Wright’s path-tracing rules, we calculate
the following:

$\displaystyle\sigma_{D_{i}T_{i1}}$
$\displaystyle=-\psi\chi-\delta+\delta\chi\gamma+\psi\chi$
$\displaystyle=-\delta+\delta\chi\gamma$ $\displaystyle\sigma_{D_{i}T_{i2}}$
$\displaystyle=-\psi\gamma-\delta\chi\gamma+\delta+\psi\gamma$
$\displaystyle=-\delta\chi\gamma+\delta$ $\displaystyle\sigma_{T_{i1}T_{i2}}$
$\displaystyle=\chi\gamma$

When we compute $b_{1}$, we identify the negative value of the treatment
effect:

$\displaystyle b_{1}$
$\displaystyle=\frac{-\delta+\delta\chi\gamma-(-\delta\chi\gamma+\delta)(\chi\gamma)}{1-{(\chi\gamma)}^{2}}$
$\displaystyle=-\delta$

Likewise, when we compute $b_{2}$, we identify the treatment effect precisely:

$\displaystyle b_{2}$
$\displaystyle=\frac{-\delta\chi\gamma+\delta-(-\delta+\delta\chi\gamma)(\chi\gamma)}{1-{(\chi\gamma)}^{2}}$
$\displaystyle=\delta$

###  A.2 Baseline model with outcome-to-outcome interference

Following the steps of the prior subsection, we derive the partial regression
coefficients $b_{1}$ and $b_{2}$ given the model in Figure 1B. Unlike that of
Figure 1A, this model includes outcome-to-outcome interference. We compute
$\sigma_{D_{i}T_{i1}}$, $\sigma_{D_{i}T_{i2}}$, and $\sigma_{T_{i1}T_{i2}}$
with Wright’s path-tracing rules:

$\displaystyle\sigma_{D_{i}T_{i1}}$
$\displaystyle=-\psi\chi-\delta+\psi\chi\eta+\delta\eta+\delta\chi\gamma+\psi\chi$
$\displaystyle=-\delta+\psi\chi\eta+\delta\eta+\delta\chi\gamma$
$\displaystyle\sigma_{D_{i}T_{i2}}$
$\displaystyle=-\psi\gamma-\delta\chi\gamma+\psi\gamma\eta+\delta\chi\gamma\eta+\delta+\psi\gamma$
$\displaystyle=-\delta\chi\gamma+\psi\gamma\eta+\delta\chi\gamma\eta+\delta$
$\displaystyle\sigma_{T_{i1}T_{i2}}$ $\displaystyle=\chi\gamma$

We compute $b_{1}$, which does not identify (the negative value of) the
treatment effect, nor a simple function of it:

$\displaystyle b_{1}$
$\displaystyle=\frac{-\delta+\psi\chi\eta+\delta\eta+\delta\chi\gamma-(-\delta\chi\gamma+\psi\gamma\eta+\delta\chi\gamma\eta+\delta)(\chi\gamma)}{1-{(\chi\gamma)}^{2}}$
$\displaystyle=-\delta+\delta\eta+\frac{\psi\chi\eta(1-{\gamma}^{2})}{1-{(\chi\gamma)}^{2}}$

As anticipated, $b_{2}$ does not identify the treatment effect nor a simple
function of it:

$\displaystyle b_{2}$
$\displaystyle=\frac{-\delta\chi\gamma+\psi\gamma\eta+\delta\chi\gamma\eta+\delta-(\delta+\psi\chi\eta+\delta\eta+\delta\chi\gamma)(\chi\gamma)}{1-{(\chi\gamma)}^{2}}$
$\displaystyle=\delta+\frac{\psi\gamma\eta(1-{\chi}^{2})}{1-{(\chi\gamma)}^{2}}$

##  Appendix B Full Simulation Results

Appendix Table 1 includes additional findings from the main Monte Carlo
simulation analyses, specifically the estimated partial regression
coefficients and accompanying 95% confidence intervals for both siblings’
treatments. These additional results corroborate the primary findings that we
show in the main text. Still, the additional results for the simulations of
the treatment-to-outcome interference model (Figure 2D) are noteworthy. In
Simulation 1.1 (no outcome-to-outcome interference and the observed FE is
related to the unobserved FE), we find that neither $b_{1}$ nor $b_{2}$
identify (a simple function of) the focal treatment effect, yet $b_{C}$
correctly signals the absence of outcome-to-outcome interference. Mallinson
and Elwert (2021) demonstrate that we cannot precisely identify treatment
effects in this model, but we can identify lower bounds of one of the
treatment-to-outcome interference effects (which is more commonly known as a
“spillover effect”).11

##  Appendix C Post-Hoc Simulation

Appendix Table 2 displays results from a post-hoc Monte Carlo simulation
analysis. We re-ran all simulation sets as described in the main text for the
model in Figure 2B, where the observed FE $C_{i}$ has a direct association
with both siblings’ treatments. However, we varied the simulations such that
the observed FE had a direct association with only one sibling’s
treatment—that is, $\nu=1.5$
($C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i2}$) and $\tau=0$
($C_{i}\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i1}$), or $\nu=0$ and
$\tau=0.7$. These results were consistent with those of the main analysis,
indicating that $C_{i}$ could be a valid observed FE even if it was directly
associated with a single sibling’s treatment and with no other variable in the
data generating process.

##  Appendix D Main Simulation Code

We conducted Monte Carlo simulations in Stata Statistical Software: Release
16.25

##  Appendix E Post-Hoc Simulation Code

We conducted Monte Carlo simulations in Stata Statistical Software: Release
16.25


11institutetext: Inria111All affiliations: Inria, Univ. Rennes, CNRS, Irisa,
France
11email<EMAIL_ADDRESS>
# Effects of Locality and Rule Language on Explanations for Knowledge Graph
Embeddings

Luis Galárraga 11 0000-0002-0241-5379

###### Abstract

Knowledge graphs (KGs) are key tools in many AI-related tasks such as
reasoning or question answering. This has, in turn, propelled research in link
prediction in KGs, the task of predicting missing relationships from the
available knowledge. Solutions based on KG embeddings have shown promising
results in this matter. On the downside, these approaches are usually unable
to explain their predictions. While some works have proposed to compute post-
hoc rule explanations for embedding-based link predictors, these efforts have
mostly resorted to rules with unbounded atoms, e.g.,
$\textit{bornIn}(x,y)\Rightarrow\textit{residence}(x,y)$, learned on a global
scope, i.e., the entire KG. None of these works has considered the impact of
rules with bounded atoms such as
$\textit{nationality}(x,\textit{England})\Rightarrow\textit{speaks}(x,\textit{English})$,
or the impact of learning from regions of the KG, i.e., local scopes. We
therefore study the effects of these factors on the quality of rule-based
explanations for embedding-based link predictors. Our results suggest that
more specific rules and local scopes can improve the accuracy of the
explanations. Moreover, these rules can provide further insights about the
inner-workings of KG embeddings for link prediction.

###### Keywords:

knowledge graph embeddings, explainable AI

##  1 Introduction

The continuous advances in information extraction on the Web have given rise
to large repositories of machine-friendly statements modeled as knowledge
graphs (KGs). These are collections of facts of the form
$\textit{p}(\textit{s},\textit{o})$ that describe real-world entities, e.g.,
$\textit{capital}(\textit{Italy},\textit{Rome})$. In this formalism, the
predicate $p$ in a statement $\textit{p}(\textit{s},\textit{o})$ can be seen
as a directed labeled edge that connects the subject $s$ to the object $o$.
KGs allow computers to “understand” the real world, and find applications in
multiple AI-related tasks such as entity-centric IR, reasoning, question
answering, smart assistants, etc. Since KGs usually suffer from
incompleteness, a central task in KGs is _link prediction_ , where the goal is
to infer new facts from the available knowledge. Link prediction constitutes a
fundamental step towards proper knowledge graph completion.

Approaches for link prediction in KGs abound and fall mainly into two
paradigms. On the one hand, _symbolic methods_ [16, 19, 12] mine explicit
patterns on the graph, e.g., the rule
$\textit{capital}(\textit{x},\textit{y})$
$\Rightarrow\textit{inCountry}(\textit{y},\textit{x})$, and use those patterns
to infer new relationships between entities. On the other hand, approaches
based on latent factors [22, 4, 21, 34, 31, 29] embed predicates $p$ and
entities $s,o$ in a latent space driven by a score function that ranks true
facts better than false ones. For example, TransE [4] learns $d$-dimensional
embeddings (in bold) for predicates and entities such that
$\bm{s}+\bm{p}\approx\bm{o}$, if $\textit{p}(\textit{s},\textit{o})$ holds in
reality. TransE’s score function for facts is then
$-\lVert\bm{s}+\bm{p}-\bm{o}\rVert_{l}$ ($l=\\{1,2\\}$).

Embedding-based methods have exhibited promising performance for link
prediction, however their main downside is that they operate as black boxes:
one cannot obtain an explanation of the logic behind a predicted fact
$\textit{p}(\textit{s},\textit{o})$ from the latent representations of $s$,
$p$, and $o$. This has therefore motivated some works on mining rule-based
explanations for KG embeddings [9, 7, 27, 26]. Those explanations can help us,
for instance, verify if the embeddings meet expected reasoning guarantees such
as transitivity, i.e.,
$\textit{p}(\textit{x},\textit{z})\land\textit{p}(\textit{z},\textit{y})\Rightarrow\textit{p}(\textit{x},\textit{y})$,
or detect biases in the data. It is known that redundancy in the form of
inverse predicates, e.g., $\textit{hyponym}(\textit{feline},\textit{cat})$,
$\textit{hypernym}(\textit{cat},\textit{feline})$ in benchmark datasets, led
to over-estimated accuracies for state-of-the-art embedding-based link
predictors [18, 3]. Had a mechanism to understand that the embeddings mainly
captured patterns such as $\textit{hyponym}(\textit{x},\textit{y})\Rightarrow$
$\textit{hypernym}(\textit{y},\textit{x})$, this issue could have been
detected in advance.

A limitation of existing explanations for KG embeddings is that they only
capture global inference patterns. This is tantamount to mining explanations
in the language of unbounded atoms, i.e., rules with no constants in the
arguments such as
$\textit{bornIn}(\textit{x},\textit{z})\land\textit{officialLang}(\textit{z},\textit{y})\Rightarrow\textit{speaks}(\textit{x},\textit{y})$,
that hold _globally_ , that is, on the entire KG. However, such rules cannot
express specific entity associations such as
$\textit{nationality}(\textit{x},\textit{USA})\Rightarrow\textit{speaks}(\textit{x},\textit{English})$,
presumably captured by link predictors. On those grounds, Section 4 addresses
the following research question (RQ1): what is the impact of specific rules in
the quality of the explanations for embedding-based predictors?. Moreover, and
in line with existing works in interpretable AI [24, 23], we also study a
second research question (RQ2): how does learning explanation rules on
specific regions of the KG, i.e., local explanations, impact the quality of
the resulting rules?. Before answering these questions, we discuss basic
concepts and related work in Section 2, and explain how to compute rule-based
explanations for link predictors in Section 3.

##  2 Preliminaries

###  2.1 Background Concepts

####  2.1.1 Knowledge Graphs.

A knowledge graph $\mathcal{K}$ = ($\mathcal{V}$, $\mathcal{E}$, $l_{v}$,
$l_{e}$) is a directed labeled graph with sets of vertices $\mathcal{V}$ and
edges $\mathcal{E}$, where the injective functions
$l_{v}:\mathcal{V}\rightarrow\mathcal{I}$,
$l_{e}:\mathcal{E}\rightarrow\mathcal{P}$ assign labels to the vertices and
edges. The sets $\mathcal{I}$ and $\mathcal{P}$ contain entity and predicate
labels. An edge labeled _capital_ departing from a vertex labeled _France_ to
a vertex labeled _Paris_ denotes the _statement_ or _fact_
$\textit{capital}(\textit{France},\textit{Paris})$, i.e., France has capital
Paris. Hence, a KG
$\mathcal{K}\subset\mathcal{I}\times\mathcal{P}\times\mathcal{I}$ is also a
set of facts $\textit{p}(\textit{s},\textit{o})$ with subject $s$, predicate
$p$, and object $o$. Usually, standard KGs store only facts believed to be
true.

We define the _potential set_ $\Omega(\mathcal{K})$ of a KG as the universe of
facts that could be constructed from the entities and predicates in
$\mathcal{K}$. More formally,
$\Omega(\mathcal{K})=\mathcal{D}_{v}(\mathcal{K})\times\mathcal{D}_{e}(\mathcal{K})\times\mathcal{D}_{v}(\mathcal{K})$
where

$\mathcal{D}_{v}(\mathcal{K})=\\{l_{v}(v):v\in\mathcal{V}\\},$

$\mathcal{D}_{e}(\mathcal{K})=\\{l_{e}(e):e\in\mathcal{E}\\}$

are the _entity and predicate domains_ of $\mathcal{K}$. Furthermore, we
define the _potential set_ of a predicate $p$ as
$\Omega(\mathcal{K})\supseteq\Omega^{p}(\mathcal{K})=\\{p(s,o):(s,o)\in\mathcal{D}^{p}(\mathcal{K})\times\bar{\mathcal{D}}^{p}(\mathcal{K})\\}$
with
$\mathcal{D}^{p}(\mathcal{K})=\\{s:\exists o:p(s,o)\in\mathcal{K}\\},$
$\bar{\mathcal{D}}^{p}(\mathcal{K})=\\{o:\exists s:p(s,o)\in\mathcal{K}\\}.$

$\Omega^{p}(\mathcal{K})$ therefore defines the set of all possible facts that
could be constructed with the known subjects and objects of predicate $p$.

####  2.1.2 Horn rules.

An _atom_ $A$ is a statement with constant predicate such that its subject and
object arguments can be variables $v\in\mathbb{V}$ with
$\mathbb{V}\cap\mathcal{I}=\emptyset$. If $A$ has only variable arguments, we
say $A$ is _unbounded_ , otherwise it is _bounded_. A _Horn rule_ $R$ is a
statement of the form $\bm{B}\Rightarrow H$ where the _body_ $\bm{B}$ is a
conjunction of atoms $\bigwedge_{1\leq i\leq n}{A_{i}}$, and $H$ is the head
atom. For instance, the rule
$\textit{parent}(\textit{x},\textit{z})\land\textit{nationality}(\textit{z},\textit{y})\Rightarrow\textit{nationality}(\textit{x},\textit{y})$
states that parents and children have the same nationality. These rules
usually come with scores that quantify their precision. It is common to
require atoms in rules to have at least one variable, be transitively
connected, and form _safe_ rules, that is, ensure that the head variables
occur also in the body. This condition guarantees that the head variables are
universally quantified, allowing for concrete predictions via _substitutions_.
A substitution $\sigma:\mathbb{V}\rightarrow\mathcal{I}$ is a partial mapping
from variables to constants, such that its application to atoms or rules
replaces each variable with its corresponding constant in the mapping. For
example, applying the substitution
$\sigma=\\{x\rightarrow\mathit{\textit{Marie
Curie}},y\rightarrow\mathit{France}\\}$ to the atom
$A:\mathit{nationality}(x,y)$, gives a new atom
$\sigma(A):\mathit{nationality}(\textit{Marie Curie},\mathit{France})$. We say
a rule $R:\bm{B}\Rightarrow H$ _predicts a fact_ $A^{\prime}$ in a KG
$\mathcal{K}$, denoted by $R\land\mathcal{K}\vdash A^{\prime}$, iff
$\exists\;\sigma:(\forall
B\in\bm{B}:\sigma(B)\in\mathcal{K})\land\sigma(H)=A^{\prime}$. Put differently
a rule predicts a fact $A^{\prime}$ if there exist a substitution $\sigma$
that (i) maps each atom in the rule’s body to a known KG fact, and (ii) maps
the head atom to $A^{\prime}$. If $R$ predicts a statement $A^{\prime}$ and
$A^{\prime}\in\mathcal{K}$, we say that $R$ _predicts $A^{\prime}$ correctly_,
i.e., the prediction is a known fact, and we use the notation
$R\land\mathcal{K}\vDash A^{\prime}$.

####  2.1.3 Link Predictors.

A _link predictor_ $f:\Omega(\mathcal{K})\rightarrow\mathbb{R}$ is a function
that scores the facts in the potential set of a KG, usually assigning higher
values to true facts. Link predictors are mostly used to answer queries of the
forms $\textit{p}(\textit{s},\textit{?})$ or
$\textit{p}(\textit{?},\textit{o})$, in other words, queries that ask for the
most likely subject or object of a statement given the other two components.
Embedding-based link predictors operate on latent representations for
entities, predicates, and facts in $\Omega(\mathcal{K})$. Hence, they actually
have the form $f=\hat{f}\circ h$, where
$\hat{f}:\mathbb{C}^{k}\rightarrow\mathbb{R}$222Most methods embed the
entities in real spaces, i.e., in $\mathbb{R}^{k}$, but a few, e.g.,[31]
resort to vectors of complex numbers.  is a function defined on a
$k$-dimensional representation for facts, and
$h:\Omega(\mathcal{K})\rightarrow\mathbb{C}^{k}$ maps facts to $k$-dimensional
vectors. If the semantics of the vector components are not understandable to
humans, we say that $f$ is a black box. That is the case for pure embedding-
based link predictors such as TransE [4] or ComplEx [31].

####  2.1.4 Explanations.

An explanation $E=\langle\mathcal{R},g\rangle$ for a black-box link predictor
$f:\Omega(\mathcal{K})\rightarrow\mathbb{R}$ consists of a set $\mathcal{R}$
of Horn rules and a function $g:\mathcal{R}\rightarrow\mathbb{R}$ that
attributes higher scores to rules that “agree” with $f$. A rule
$R:\bm{B}\Rightarrow H$ agrees with $f$, if $R$ predicts a fact
$A\in\Omega(\mathcal{K})$ also predicted by $f$. This definition assumes the
existence of a threshold $\theta$ such that $f(A)\geq\theta$ is interpreted as
the black box also “thinking” that $A$ is true. Explanations can be of
different scope, namely _global_ when they are learned on the potential set
$\Omega^{p}(\mathcal{K})$ of a predicate $p$, or _local_ when they are learned
on smaller regions of $\Omega^{p}(\mathcal{K})$ as explained in Section 3.

###  2.2 Related Work

####  2.2.1 Link Prediction.

This problem has received a lot of attention in the last 10 years with
approaches lying on a spectrum from symbolic methods to embedding-based
techniques. We refer the reader to [14] for a comprehensive survey. Symbolic
techniques learn explicit patterns, e.g., arbitrary subgraphs, paths,
association rules, Horn rules, etc., from KGs and use those patterns as
features to predict missing links between entities [16, 19, 12]. In contrast,
the common principle of embedding-based methods is to model entities and
predicates as elements in a latent space, where predicates characterize
interactions between entity embeddings. Those interactions are modeled as
geometrical operations, e.g., translation in TransE [4] where
$\bm{s}+\bm{p}\approx\bm{o}$ for true facts
$\textit{p}(\textit{s},\textit{o})$ ($\bm{s},\bm{p},\bm{o}\in\mathbb{R}^{d}$),
or rotation in RotatE [30]. More recent methods resort to neural architectures
[20, 28] that exploit the vicinity of entities in the graph to learn proper
latent representations for both entities and predicates.

In all cases, a scoring function – implemented by minimizing a loss function –
guides the training of the embeddings, which are learned to yield high scores
for true facts and low scores for false facts. The latter are obtained by
corrupting the true facts in the KG – a task of utter importance for the
quality of the embeddings [13, 36].

Other methods combine the strengths of symbolic patterns and embeddings [6].
In [17], the authors improve the accuracy of different state-of-the-art
embedding-based link predictors by removing those predictions that are not
backed up by any of the Horn rules learned on the data. This strategy is
complemented with a combined ranking that takes into account the individual
rankings given by the rules and the embeddings. Some approaches [1, 35, 13]
propose iterative algorithms that use rules and embeddings to produce better
examples for subsequent training. In contrast, other methods [11, 25] instruct
the embeddings to comply to explicit reasoning patterns, e.g., transitivity,
$\textit{p}(\textit{x},\textit{z})\land\textit{p}(\textit{z},\textit{y})\Rightarrow\textit{p}(\textit{x},\textit{y})$.

####  2.2.2 Explaining the Black Box.

Unlike symbolic approaches, link predictors based on embeddings are black
boxes. Hence, there have been some efforts to explain their logic by mining
explicit patterns [7, 27, 26] with attribution scores. Among these patterns,
Horn rules are the most popular. The rules are extracted using state-of-the-
art rule or path mining algorithms [15, 16, 2, 8], whereas the attribution
scores are learned via machine learning, e.g., linear or logistic regression
in the spirit of standard explanation techniques such as LIME [24].
Nevertheless, none of these approaches exploits the power of Horn rules at its
best. For instance, [7, 27] mine rule explanations of up to two atoms, e.g.,
$\textit{bornIn}(\textit{x},\textit{y})\Rightarrow\textit{livesIn}(\textit{x},\textit{y})$,
whereas DistMult [34] can only learn _pure_ paths such as
$\textit{bornIn}(\textit{x},\textit{z})\land\textit{inCountry}(\textit{z},\textit{y})\Rightarrow\textit{nationality}(\textit{x},\textit{y})$.
Hence, none of these methods can induce explanations in the language of
bounded atoms such as
$\textit{nationality}(\textit{x},\textit{UK})\Rightarrow\textit{speaks}(\textit{x},\textit{English})$.
Furthermore, all these endeavors mine global explanations. Embedding-based
models can, though, be very complex and therefore hard to approximate in the
general sense. Thus we explore the effects of bounded atoms and locality in
the quality of the explanations.

##  3 Explaining KG Embeddings for Link Prediction

Algorithm 1 describes a generic procedure to compute rule-based explanations
for a black-box link predictor $f$ trained on a KG $\mathcal{K}$, containing
both true ($\mathcal{K}^{+}$) and corrupted facts ($\mathcal{K}^{-}$), in line
with existing approaches [7, 27, 26]. The rules are learned on a context $C$
consisting of facts of a given predicate $p$. We elaborate on the stages of
the algorithm and the different ways to define the learning context.

Input: link predictor $f:\Omega(\mathcal{K})\rightarrow\mathbb{R}$ trained on
$\mathcal{K}=\mathcal{K}^{+}\cup\mathcal{K}^{-}$, context
$C\subset\Omega^{p}(\mathcal{K})$

Output: an explanation $E=\langle\mathcal{R},g\rangle$ with set of rules
$\mathcal{R}$, $g:\mathcal{R}\rightarrow\mathbb{R}$

1

2$\hat{\mathcal{K}}:=\emptyset$

3 foreach  _$A:=p(s,o)\in C$_ do

4 if  _$f(A)\geq\theta$_ then

5 $\hat{\mathcal{K}}:=\hat{\mathcal{K}}\cup\\{p^{f}(s,o)\\}$

6

7 else

8 $\hat{\mathcal{K}}:=\hat{\mathcal{K}}\cup\\{\neg p^{f}(s,o)\\}$

9

10

11$\mathcal{R}:=\text{rule mining
on}\;\mathcal{K}\cup\hat{\mathcal{K}}\;\text{for predicates}\;{p^{f},\neg
p^{f}}$

12 return
$\mathit{build\text{-}rule\text{-}based\text{-}surrogate(\mathcal{R},\mathcal{K},\hat{\mathcal{K}},f)}$

13

Algorithm 1 Build Explanation

Binarizing the black box. To learn Horn rules that mimic a black-box link
predictor $f$, we need to convert $f$’s scores for facts into true or false
verdicts. To this end, lines 2–6 label each fact in the context $C$ by
computing $f$’s score and then applying a threshold to decide whether the fact
is deemed true or not by $f$. This set $\hat{\mathcal{K}}$ of annotated facts
is represented by the surrogate predicates $p^{f}$, $\neg p^{f}$. For instance
the fact $\textit{speaks${}^{f}$}(\textit{A. Einstein},\textit{English})$
means that $f$ “thinks” that Einstein speaks English.

Rule Mining. Line 7 in Alg. 1 learns a set $\mathcal{R}$ of Horn rules of the
forms $\bm{B}\Rightarrow p^{f}(s,o)$ and $\bm{B}\Rightarrow\neg p^{f}(s,o)$
with confidence scores from the original KG $\mathcal{K}$ and the black-box
annotated context $\hat{\mathcal{K}}$.

Learning the explanation.  Finally, line 8 uses the rules in $\mathcal{R}$ as
features to learn a surrogate model
$f_{s}:\mathbb{R}^{|\mathcal{R}|}\rightarrow\mathbb{R}$ that mimics the
binarized $f$ and provides importance scores for the mined rules. Given a
statement $A=\bar{p}^{f}(s,o)\in\hat{\mathcal{K}}$ with
$\bar{p}^{f}\in\\{p^{f},\neg p^{f}\\}$, we encode $A$ as a vector
$x_{A}\in\mathbb{R}^{|\mathcal{R}|}$ such that its $i$-th entry is set as
follows:

$x_{A}[i]=\begin{cases}\mathit{sgn}(A)\times\mathit{conf}(R_{i})&R_{i}\land(\mathcal{K}\cup\hat{\mathcal{K}})\vDash
A\\\
-\mathit{sgn}(A)\times\mathit{conf}(R_{i})&R_{i}\land(\mathcal{K}\cup\hat{\mathcal{K}})\vdash
A^{\prime}\;\text{with}\;A^{\prime}\neq A\\\ 0&\text{otherwise}\end{cases}$

Here $\mathit{sgn}(A)=1$ if $A=\bar{p}^{f}(s,o)$, otherwise
$\mathit{sgn}(A)=-1$. If a rule $R_{i}\in\mathcal{R}$ _predicts correctly_ a
statement $A\in\hat{\mathcal{K}}$, the $i$-th component of $x_{A}$ holds a
value equals the confidence of $R_{i}$ (reported by the rule mining phase)
with the same polarity of $f$’s prediction. In that case, the rule $R_{i}$
agrees with $f$ and is a potential explanation for $f$’s answer on $A$. If
$R_{i}$ is a potential explanation for some other fact $A^{\prime}$, we change
the sign of confidence value. In any other case, we assign a score of 0 to the
entry. We use the $x_{A}$ vectors and the binarized labels – given by
$\mathit{sgn}(A)$ – to train a surrogate logistic regression classifier
$f_{s}$, whose coefficients define an attribution mapping
$g:\mathcal{R}\rightarrow\mathbb{R}$ for rules – our explanation. The
surrogate $f_{s}$ can provide both binary labels and probability scores for
facts, and its coefficients can be used to rank the rules predicting true and
false verdicts $p^{f}(s,o),\neg p^{f}(s,o)$.

Explanation Context. Existing explanation approaches for KG embeddings [26,
27] mine global explanations, where the context $C$ given as input to Alg. 1
contains a large sample of true and false statements. The latter are obtained
by corrupting the true facts, so that for each fact
$\textit{p}(\textit{s},\textit{o})$ we also add
$\\{p(s^{\prime},o),p(s,o^{\prime})\\}$ ($s\neq s^{\prime},o\neq o^{\prime}$).
The resulting surrogate $f_{s}$ approximates $f$’s general logic when
predicting $p$-labeled links.

A drawback of explanations based on global surrogates is that they assume that
rules have always the same importance for all $p$-labeled predictions. Such a
simplistic assumption can make explanation mining uninformative, if for
example, the black box has a fine-grained behavior, i.e., it implements
different logics for different regions of the KG. On those grounds, we propose
to mine explanations within a _local_ scope obtained by calling Alg. 1 on
different sub-contexts $C^{\prime}\subseteq C$ with triples that are close to
each other in the latent space. These sub-contexts are obtained by applying
agglomerative hierarchical clustering on $\bm{s}\oplus\bm{o}$, i.e., on the
latent representation of pairs $s,o$ for true facts
$\textit{p}(\textit{s},\textit{o})\in C$333$\oplus$ denotes concatenation;
sub-contexts are corrupted to obtain counter-examples.. We can also define
_per-instance_ contexts around a target fact $A=p(s,o)$ by calling Alg. 1 on a
sub-context $C^{\prime}=\\{A^{\prime}=p(s^{\prime},o):A^{\prime}\in
C\\}\cup\\{A^{\prime}=p(s,o^{\prime}):A^{\prime}\in C\\}\cup\\{A\\}$, that is,
on statements that share at least one argument with $A$.

##  4 Evaluation

To answer our research questions, we study the impact of bounded atoms (RQ1)
and locality (RQ2) on the fidelity of rule explanations for embedding-based
link predictors through a quantitative and an anecdotal evaluation.

###  4.1 Experimental Setup

####  4.1.1 Datasets and Link Predictors.

We use the benchmark datasets fb15k-237, wn18rr, and yago3-10, on which we
trained the bilinear methods ComplEx [31] and HolE [21], and the translational
approach TransE [4]. We used the implementations and data offered by the
Torch-KGE library [5].

####  4.1.2 Rule Mining.

We mine Horn rules with AMIE [15], a state-of-the-art rule miner for large
KGs. By default, AMIE mines closed Horn rules444These are safe rules where
each variable occurs in at least 2 atoms of up to 3 unbounded atoms, but it
can be instructed to mine longer rules, as well as to allow bounded atoms.
Longer rules in combination with bounded atoms increase significantly the
search space for rules, therefore we did not experiment with more than 3 atoms
to avoid prohibitive runtimes [10]. AMIE does not support explicit counter-
examples to estimate the precision of rules, as required by Alg. 1, hence we
extended the system to support explicit false facts in the precision
computation. These counter-examples were generated through a variant of
Bernoulli sampling that accounts for predicate domains [33]. We use all rules
making at least 2 correct predictions with a precision of at least 10% to
learn the surrogate model (see Section 3).

####  4.1.3 Explanations.

We compute rule-based explanations for the studied link predictors using the
test instances of the experimental datasets to construct contexts $C$ of
different scopes, i.e., global, local, and per-instance as explained in
Section 3. For each call to Alg. 1, we split $C$ into train and test sets
$C_{\mathit{train}}$ and $C_{\mathit{test}}$ (30%), so that we learn the
explanations on $C_{\mathit{train}}$ and evaluate them on $C_{\mathit{test}}$.
Local clusters are computed using agglomerative hierarchical clustering
instructed to return $k$ clusters. We chose the most performing value of $k$
between 2 and 6.

Link predictors are mainly used for two tasks: fact classification (true vs.
false) and subject/object prediction for queries
$\textit{p}(\textit{?},\textit{o})$ and $\textit{p}(\textit{s},\textit{?})$
where potential candidates are ranked by their score. We quantify the fidelity
of our surrogate models (their ability to approximate the link predictors) for
these two tasks via standard metrics, namely the ROC-AUC score and the mean
reciprocal rank (MRR). The threshold $\theta$ to binarize $f$’s scores (line 6
in Alg. 1) is chosen via logistic regression as follows: we learned a logistic
regression classifier $f_{c}:C\rightarrow[0,1]$ using $f$’s scores on $C$ as
input features and the real truth value of the facts as label – corrupted
facts are assumed false. In that spirit, $f_{c}$ estimates the probability
that a given statement is true. $\theta$ is then chosen so that
$f_{c}(\theta)=0.5$.

###  4.2 Results

| ROC-AUC | S-MRR | O-MRR
---|---|---|---
| Unbounded | Bounded | Unbounded | Bounded | Unbounded | Bounded
|  B |  L |  PI |  G |  L |  PI |  B |  L |  PI |  G |  L |  PI |  B |  L |  PI |  G |  L |  PI
complex |  0.71 |  0.68 |  0.64 |  0.93 |  0.93 |  0.95 |  0.13 |  0.16 |  0.19 |  0.31 |  0.35 |  0.44 |  0.97 |  1.00 |  0.97 |  0.97 |  0.98 |  0.93
transe |  0.72 |  0.70 |  0.64 |  0.95 |  0.92 |  0.95 |  0.12 |  0.20 |  0.19 |  0.22 |  0.47 |  0.45 |  0.97 |  0.99 |  0.98 |  0.98 |  0.93 |  0.91
hole |  0.66 |  0.63 |  0.60 |  0.98 |  0.99 |  0.99 |  0.08 |  0.16 |  0.22 |  0.27 |  0.36 |  0.50 |  0.98 |  0.98 |  0.97 |  0.98 |  1.00 |  0.97
Table 1: Fidelity on fb15k-237. Best performances are in bold; best locality
results are underlined. The baseline B are global explanations with unbounded
atoms. G, PI, and L stand for global, per-instance, and local explanations.

| ROC-AUC | S-MRR | O-MRR
---|---|---|---
|  G |  L |  PI |  G |  L |  PI |  G |  L |  PI
complex |  0.55 |  0.64 |  0.68 |  0.93 |  0.60 |  0.38 |  0.92 |  0.93 |  1.00
transe |  0.51 |  0.55 |  0.69 |  0.71 |  0.87 |  0.32 |  0.93 |  0.92 |  1.00
hole |  – |  0.65 |  0.73 |  – |  0.42 |  0.38 |  – |  1.00 |  1.00
(a) wn18RR

ROC-AUC | S-MRR | O-MRR
---|---|---
G |  L |  PI |  G |  L |  PI |  G |  L |  PI
0.55 |  0.75 |  0.00 |  0.94 |  0.39 |  0.17 |  0.87 |  1.00 |  1.00
0.66 |  0.63 |  0.63 |  0.73 |  0.50 |  0.50 |  0.94 |  0.97 |  1.00
0.71 |  0.81 |  0.65 |  0.90 |  0.38 |  0.26 |  0.93 |  1.00 |  0.99
(b) yago3-10

Table 2: Fidelity of rule-based explanations with bounded atoms.

####  4.2.1 Quantitative Evaluation.

Tables 1 and 2 report the average ROC-AUC and MRR for the different
explanation setups, namely unbounded vs. bounded rules learned on global (G),
local (L), and per-instance (PI) scopes. The scores are computed by averaging
the fidelity obtained for each call to Alg. 1 weighted by the size of the
corresponding test set, i.e., $|C_{\mathit{test}}|$. We disaggregate the MRR
into S-MRR and O-MRR – when the task is to predict the subject or object given
the other two components.

Our baseline setting (denoted by B) are global unbounded rules as mined by
existing approaches [7, 27, 26]. We highlight that we could not mine
explanations with such a setting for wn18RR and yago3-10 on any of the studied
link predictors – not even for local or per-instance scopes. This happens
because unbounded rules can only be extracted when the training KG contains
very prevalent and general regularities in the interactions between the
predicates. The datasets wn18RR and yago3-10, however, have much fewer
predicates than fb15k-237: 11 and 37 for the former versus 237 for the latter.
Bounded atoms also increase the coverage explanation for fb15k-237. While the
baseline provides explanations for 18 different predicates for ComplEx on
fb15k-237, allowing bounded rules increases the coverage to 58 predicates
(HolE and TransE exhibit comparable increases). Moreover the results in Table
1 suggest that bounded atoms in rules generally increase fidelity.

It is important to remark that allowing constants in the rule atoms comes at
the expense of longer runtimes and many more, potentially noisy, rules. On
fb15k-237 with global scopes, for example, the number of unique rules mined
from TransE increases from 1k to 193k. That said, only 134k of those rules get
non-zero coefficients during the attribution phase – implemented via logistic
regression. This phase is indeed designed to identify the rules with actual
explanation power w.r.t the veredicts of the link predictor.

We also observe that rule-based surrogates tend to be better at mimicking the
link predictors for object prediction. This is explained by the nature of KG
predicates, which are usually defined in a subject-oriented manner, e.g.,
$\textit{nationality}(\textit{J. Biden},\textit{USA})$ and not hasCitizen(USA,
J. Biden). This makes subject prediction generally harder to mimic, because,
e.g., it is easier to predict the nationalities of J. Biden than to predict
all USA citizens. Besides, this phenomenon is corroborated by the actual
performances of the link predictors. For instance, ComplEx exhibits an average
S-MRR of 0.29 on wn18RR, whereas the average O-MRR reaches 0.46. That said,
Table 2(a) suggests that S-MRR fidelity can still be high even in the presence
of subject-oriented predicates.

When we look at the effects of locality on fidelity, we notice mixed effects.
On fb15k-237, locality hurts ROC-AUC performance for unbounded rules and
brings moderate performance gains for the MRR. This is probably because good
general unbounded rules require more “diverse” data. The situation is
different for bounded rules, for which locality boosts fidelity in most cases.
These results suggest that locality and bounded rules are complementary. A
similar behavior can also be observed for coverage. For example, local scopes
combined with bounded rules allow mining 507k unique rules (with non-zero
attribution) for 130 different predicates for ComplEx on fb15k-237 vs. 112k
rules/58 predicates and 1505 rules/62 predicates when only one of the features
is enabled (the baseline mines 730 predicates covering 18 predicates). For
per-instance scopes we can compute rule explanations for up to 2782 individual
facts (out of 20k) covering 83 predicates (HolE on fb15k-237).

fb15k-237
---
(1)
$\textit{place\\_of\\_birth}(\textit{x},\textit{Chicago})\Rightarrow\textit{nationality}(\textit{x},\textit{USA})$
[TransE]
(2)
$\textit{has\\_lived\\_in}(\textit{x},\textit{Brooklyn})\Rightarrow\textit{nationality}(\textit{x},\textit{USA})$
[ComplEx]
(3)
$\textit{profession}(\textit{x},\textit{Author})\Rightarrow\textit{gender}(\textit{x},\textit{M})$
[ComplEx]
(4)
$\textit{impersonates}(\textit{z},\textit{x})\land\textit{gender}(\textit{z},\textit{y})\Rightarrow\textit{gender}(\textit{x},\textit{y})$
[ComplEx, HolE]
(5)
$\textit{country}(\textit{z},\textit{y})\land\textit{birth\\_place}(\textit{x},\textit{z})\Rightarrow\textit{nationality}(\textit{x},\textit{y})$
(6)
$\textit{company}(\textit{z},\textit{x})\land\textit{athlete:sport}(\textit{z},\textit{y})\Rightarrow\textit{sport}(\textit{x},\textit{y})^{\circ}$
[HolE]
(7)
$\textit{fwc:club}(\textit{z},\textit{x})\land\textit{sport}(\textit{z},\textit{y})\Rightarrow\textit{sport}(\textit{x},\textit{y})^{\circ}$
[HolE]
yago3-10
(8) $\textit{affiliation}(\textit{x},\textit{Ume\r{a}
IK})\Rightarrow\textit{gender}(\textit{x},\textit{F})\dagger$ [TransE,
ComplEx]
(9) $\textit{wonPrize}(\textit{x},\textit{O. Orange-
Nassau})\Rightarrow\textit{wonPrize}(\textit{x},\textit{D.S. Medal})$
[ComplEx]
wn18rr
(10)
$\textit{meronym\\_mb}(\textit{Insecta},\textit{x})\Rightarrow\textit{hypernym}(\textit{x},\textit{Animal})\dagger$
[ComplEx, HolE]
Table 3: Some rule explanations. ∘, $\dagger$ denote local and per-instance
explanations.

####  4.2.2 Anecdotal Evaluation.

Table 3 shows a few examples of rule-based explanations for our experimental
link predictors. These correspond to some of the best ranked rules according
to the coefficients of the surrogate classifiers. The rules illustrate
regularities preserved by the link predictors, since the body of the rules
defines conditions satisfied by the facts of the KG, in contrast to the head
that matches statements predicted by our black boxes (see Alg 1). Rules with
bounded atoms offer legible insights about the information that the link
predictors may be capturing to make predictions.

A key observation is that the different link predictors do not seem to rely on
the same information – as suggested by rules (1) and (2) for ComplEx and
TransE on fb15k-237. This is supported by the fact that among the 47
predicates for which ComplEx finds global explanations with bounded atoms,
only 3 have common rules with TransE. We bring our attention to rule (3),
which suggests that embeddings do reproduce the biases in the source
data555Rule (8), on the other hand, refers to a women’s football team.. Recall
that fb15k-237 was mainly extracted from Wikipedia, known to have gender
biases [32]. Those biases are easier to spot with rules with bounded atoms,
which are a complement to more general explanations such as (4) and (5). We
also highlight that local contexts can illustrate the semantics captured by
the embeddings. This is exemplified by rules (6) and (7) that were learned on
the same predicate but on two fact clusters. As we can see, our mining routine
learned semantically equivalent rules, defined on different thematic domains,
namely _athletes_ and _fwc_ ; the latter refers to the 2010 FIFA World Cup.

##  5 Conclusion

We have studied the effects of specific rules with bounded atoms and local
scopes on the quality of explanations for embedding-based link predictors on
knowledge graphs. Our results suggest a rather positive impact on the
explanation fidelity and the coverage of the explanations. Moreover, specific
rules and local scopes exhibit a symbiotic relationship.

Even though rule-based explanations reflect regularities preserved by black-
box link predictors, they do not shed light on causality. In this line of
though, we envision to compute causal explanations that help us understand the
role of the different entities, predicates, and latent components of KG
embeddings in the resulting predictions. We have also planned to elaborate
more on the relationship between link prediction performance and explanation
fidelity, in particular at the level of the individual predicates. The source
code and experimental data of our work is available at
https://gitlab.inria.fr/glatour/geebis.

Acknowledgment. This research was supported by TAILOR, a project funded by EU
Horizon 2020 research and innovation programme under GA No. 952215.

## References

  * [1] UniKER: A Unified Framework for Combining Embedding and Horn Rules for Knowledge Graph Inference.  In ICML Workshop on Graph Representation Learning and Beyond (GRL+), 2020.
  * [2] Naser Ahmadi, Viet-Phi Huynh, Vamsi Meduri, Stefano Ortona, and Paolo Papotti.  Mining Expressive Rules in Knowledge Graphs.  Journal of Data and Information Quality, 1(1), 2019.
  * [3] Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li.  Realistic Re-Evaluation of Knowledge Graph Completion Methods: An Experimental Study.  In ACM SIGMOD Conference, 2020.
  * [4] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko.  Translating Embeddings for Modeling Multi-relational Data.  In Advances in Neural Information Processing Systems 26. 2013.
  * [5] Armand Boschin.  TorchKGE: Knowledge Graph Embedding in Python and PyTorch.  In International Workshop on Knowledge Graphs, 2020.
  * [6] Armand Boschin, Nitisha Jain, Gurami Keretchashvili, and Fabian Suchanek.  Combining Embeddings and Rules for Fact Prediction.  In Int. Research School in AI in Bergen. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022.
  * [7] Ivan Sanchez Carmona and Sebastian Riedel.  Extracting Interpretable Models from Matrix Factorization Models.  In International Conference on Cognitive Computation, 2015.
  * [8] Yang Chen, Daisy Zhe Wang, and Sean Goldberg.  ScaLeKB: Scalable Learning and Inference over Large Knowledge Bases.  VLDB Journal, 25(6), 2016.
  * [9] Mohamed H Gad-Elrab, Daria Stepanova, Trung-Kien Tran, Heike Adel, and Gerhard Weikum.  ExCut: Explainable Embedding-Based Clustering over Knowledge Graphs.  In International Semantic Web Conference, 2020.
  * [10] Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek.  Fast Rule Mining in Ontological Knowledge Bases with AMIE+.  VLDB Journal, 24(6), 2015.
  * [11] Shu Guo, Lin Li, Zhen Hui, Lingshuai Meng, Bingnan Ma, Wei Liu, Lihong Wang, Haibin Zhai, and Hong Zhang.  Knowledge Graph Embedding Preserving Soft Logical Regularity.  In International Conference on Knowledge Management, 2020.
  * [12] Zhongni Hou, Xiaolong Jin, Zixuan Li, and Long Bai.  Rule-Aware Reinforcement Learning for Knowledge Graph Reasoning.  In ACL/IJCNLP (Findings), 2021.
  * [13] Nitisha Jain, Trung-Kien Tran, Mohamed H. Gad-Elrab, and Daria Stepanova.  Improving Knowledge Graph Embeddings with Ontological Reasoning.  In International Semantic Web Conference, 2021.
  * [14] Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu.  A Survey on Knowledge Graphs: Representation, Acquisition, and Applications.  IEEE Transactions on Neural Networks and Learning Systems, 33(2), 2022.
  * [15] Jonathan Lajus, Luis Galárraga, and Fabian Suchanek.  Fast and Exact Rule Mining with AMIE 3.  In Extended Semantic Web Conference, 2020.
  * [16] Ni Lao, Tom Mitchell, and William W. Cohen.  Random Walk Inference and Learning in A Large Scale Knowledge Base.  In Conference on Empirical Methods in Natural Language Processing, 2011.
  * [17] Christian Meilicke, Patrick Betz, and Heiner Stuckenschmidt.  Why a Naive Way to Combine Symbolic and Latent Knowledge Base Completion Works Surprisingly Well.  In 3rd Conference on Automated Knowledge Base Construction, 2021\.
  * [18] Christian Meilicke, Manuel Fink, Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, and Heiner Stuckenschmidt.  Fine-Grained Evaluation of Rule and Embedding-Based Systems for Knowledge Graph Completion.  In International Semantic Web Conference, pages 3–20, 2018.
  * [19] Changping Meng, Reynold Cheng, Silviu Maniu, Pierre Senellart, and Wangda Zhang.  Discovering Meta-Paths in Large Heterogeneous Information Networks.  In The Web Conference, 2015.
  * [20] Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network.  In Conference of the North American Chapter of the Association for Computational Linguistics, 2018.
  * [21] Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio.  Holographic Embeddings of Knowledge Graphs.  In AAAI Conference on Artificial Intelligence, 2016.
  * [22] Maximilian Nickel and Volker Tresp.  Tensor Factorization for Multi-relational Learning.  In Machine Learning and Knowledge Discovery in Databases, 2013.
  * [23] Georgina Peake and Jun Wang.  Explanation Mining: Post Hoc Interpretability of Latent Factor Models for Recommendation Systems.  In ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, 2018.
  * [24] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  Why should I trust you?: Explaining the Predictions of any Classifier.  In ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, 2016.
  * [25] Tim Rocktäschel and Sebastian Riedel.  End-to-end Differentiable Proving.  In Conference on Neural Information Processing Systems, 2017.
  * [26] Andrey Ruschel, Arthur Colombini Gusmão, Gustavo Padilha Polleti, and Fábio Gagliardi Cozman.  Explaining Completions Produced by Embeddings of Knowledge Graphs.  In European Conf. on Symbolic and Quantitative Approaches with Uncertainty, 2019.
  * [27] Ivan Sanchez, Tim Rocktaschel, Sebastian Riedel, and Sameer Singh.  Towards Extracting Faithful and Descriptive Representations of Latent Variable Models.  In AAAI Spring Symposium on Knowledge Representation and Reasoning, 2015.
  * [28] Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou.  End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion.  In AAAI Conference on Artificial Intelligence, 2019.
  * [29] Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng.  Reasoning with Neural Tensor Networks for Knowledge Base Completion.  In Conference on Neural Information Processing Systems, 2013.
  * [30] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.  In International Conference on Learning Representations, 2019.
  * [31] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard.  Complex Embeddings for Simple Link Prediction.  In International Conference on Machine Learning, 2016.
  * [32] Claudia Wagner, Eduardo Graells-Garrido, David Garcia, and Filippo Menczer.  Women through the Glass Ceiling: Gender Asymmetries in Wikipedia.  EPJ Data Science, 5:1–24, 2016.
  * [33] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen.  Knowledge Graph Embedding by Translating on Hyperplanes.  AAAI Conference on Artificial Intelligence, 28(1), 2014.
  * [34] Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases.  In International Conference on Learning Representations, 2015.
  * [35] Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, and Huajun Chen.  Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning.  In The Web Conference, 2019.
  * [36] Yongqi Zhang, Quanming Yao, and Lei Chen.  Efficient, Simple and Automated Negative Sampling for Knowledge Graph Embedding, 2020.