# Proactive Pseudo-Intervention:
Contrastive Learning For Interpretable Vision Models

Dong Wang1, Yuewei Yang1, Chenyang Tao1, Zhe Gan2, Liqun Chen1,
Fanjie Kong1, Ricardo Henao1, Lawrence Carin1
1 Duke University, 2 Microsoft Corporation
{dong.wang363, yuewei.yang, chenyang.tao, liqun.chen, ricardo.henao,
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Deep neural networks excel at comprehending complex visual signals, delivering
on par or even superior performance to that of human experts. However, ad-hoc
visual explanations of model decisions often reveal an alarming level of
reliance on exploiting non-causal visual cues that strongly correlate with the
target label in training data. As such, deep neural nets suffer compromised
generalization to novel inputs collected from different sources, and the
reverse engineering of their decision rules offers limited interpretability.
To overcome these limitations, we present a novel contrastive learning
strategy called Proactive Pseudo-Intervention (PPI) that leverages proactive
interventions to guard against image features with no causal relevance. We
also devise a novel causally informed salience mapping module to identify key
image pixels to intervene, and show it greatly facilitates model
interpretability. To demonstrate the utility of our proposals, we benchmark on
both standard natural images and challenging medical image datasets. PPI-
enhanced models consistently deliver superior performance relative to
competing solutions, especially on out-of-domain predictions and data
integration from heterogeneous sources. Further, our causally trained saliency
maps are more succinct and meaningful relative to their non-causal
counterparts.

##  1 Introduction

Figure 1:  Interpretation for the bird-classification models using saliency
maps generated by LRP (layer-wise relevance propagation) and our model PPI.
LRP shows that naively trained deep model makes decisions based on the
background cues (habitat, e.g., rocks, bulrushes) that are spuriously
correlated with the bird species, while our causally informed PPI mostly
focuses on the bird anatomy, that generalizes beyond the natural habitat.

Deep neural networks hold great promise in applications requiring the analysis
and comprehension of complex imagery. Recent advances in hardware, network
architectures, and model optimization, along with the increasing availability
of large-scale annotated datasets [31, 12, 11], have enabled these models to
match and sometimes outperform human experts on a number of tasks, including
natural image classification [32], objection recognition [20], disease
diagnosis [48], and autonomous driving [8], among others.

While deep learning solutions have been positively recognized for their
ability to learn black-box models in a purely data driven manner, their very
nature makes them less credible for their inability to communicate the
reasoning for making predictions in a way that is comprehensible to humans
[26, 43]. This denies consequential applications where the reliability and
trustworthiness of a prediction are of primary concern and require expert
audit, e.g., in healthcare [48]. To stimulate widespread use of deep learning
models, a means of interpreting predictions is necessary. However, model
interpretation techniques often reveal a concerning fact, that deep learning
models tend to assimilate spurious correlations that do not necessarily
capture the causal relationship between the input (image) and output (label)
[63]. This issue is particularly notable in small-sample-size (weak
supervision) scenarios or when the sources of non-informative variation are
overwhelming, thus likely to cause severe overfitting. These can lead to
catastrophic failures on deployment [19, 64].

A growing recognition of the issues associated with the lack of interpretable
predictions is well documented in recent years [1, 26, 43]. Such phenomenon
has energized researchers to actively seek creative solutions. Among these,
two streams of work, namely saliency mapping [68, 54, 10] and causal
representation learning (CRL) [28, 65, 2], stand out as some of the most
promising directions. Specifically, saliency mapping encompasses techniques
for post hoc visualizations on the input (image) space to facilitate the
interpretation of model predictions. This is done by projecting the key
features used in prediction back to the input space, resulting in the commonly
known saliency maps. Importantly, these maps do not directly contribute to
model learning. Alternatively, CRL solutions are built on the principles of
establishing invariance from the data, and it entails teasing out sources of
variation that are spuriously associated with the model output (labels). CRL
models, while emphasizing the differences between causation and correlation,
are not subject to the rigor of causal inference approaches, because their
goal is not to obtain accurate causal effect estimates but rather to produce
robust models with better generalization ability relative to their naively
learned counterparts [2].

In this work, we present Proactive Pseudo-Intervention (PPI), a solution that
accounts for the needs of causal representation identification and visual
verification. Our key insight is the derivation of causally-informed saliency
maps, which facilitate visual verification of model predictions and enable
learning that is robust to (non-causal) associations. While true causation can
only be established through experimental interventions, we leverage tools from
contrastive representation learning to synthesize pseudo-interventions from
observational data. Our procedure is motivated by the causal argument:
perturbing the non-causal features will not change the target label.

To motivate, in Figure 1 we present an example to illustrate the benefits of
producing causally-informed saliency maps. In this scenario, the task is to
classify two bird species (A and B) in the wild. Due to the differences in
their natural habitats, A-birds are mostly seen resting on rocks, while
B-birds are more commonly found among bulrushes. A deep model, trained
naively, will tend to associate the background characteristics with the
labels, knowing these strongly correlate with the bird species (labels) in the
training set. This is confirmed by the saliency maps derived from the layer-
wise relevance propagation (LRP) techniques [4]: the model also attends
heavily on the background features, while the difference in bird anatomy is
what causally determines the label. If we were provided with an image of a
bird in an environment foreign to the images in the training set, the model
will be unable to make a reliable prediction, thus causing robustness
concerns. This generalization issue worsens with a smaller training sample
size. On the other hand, saliency maps from our PPI-enhanced model
successfully focus on the bird anatomy, and thus will be robust to
environmental changes captured in the input images.

PPI addresses causally-informed reasoning, robust learning, and model
interpretation in a unified framework. A new saliency mapping method, named
Weight Back Propagation (WBP), is also proposed to generate more concentrated
intervention mask for PPI training. The key contributions of this paper
include:

  * •

An end-to-end contrastive representation learning strategy PPI that employs
proactive interventions to identify causally relevant features.

  * •

A fast and architecture-agnostic saliency mapping module WBP that delivers
better visualization and localization performance.

  * •

Experiments demonstrating significant performance boosts from integrating PPI
and WBP relative to competing solutions, especially on out-of-domain
predictions, data integration with heterogeneous sources and model
interpretation.

##  2 Background

Visual Explanations Saliency mapping collectively refers to a family of
techniques to understand and interpret black-box image classification models,
such as deep neural networks [1, 26, 43]. These methods project the model
understanding of the targets, i.e., labels, and their predictions back to the
input space, which allows for the visual inspection of automated reasoning and
for the communication of predictive visual cues to the user or human expert,
aiming to shed model insights or to build trust for deep-learning-based
systems.

In this study, we focus on post hoc saliency mapping strategies, where
saliency maps are constructed given an arbitrary prediction model, as opposed
to relying on customized model architectures for interpretable predictions
[19, 64], or to train a separate module to explicitly produce model
explanations [19, 21, 6, 18, 53]. Popular solutions under this category
include: activation mapping [69, 51], input sensitivity analysis [53], and
relevance propagation [4]. Activation mapping based methods fail at
visualizing fine-grained evidence, which is particularly important in
explaining medical classification models [14, 51, 60]. Input sensitivity
analysis based methods produce fine-grained saliency maps. However, these maps
are generally less concentrated [10, 18] and less interpretable. Relevance
propagation based methods, like LRP and its variants, use complex rules to
prioritize positive or large relevance, making the saliency maps visually
appealing to human. However, our experiments demonstrate that LRP and its
variants highlight spuriously correlated features (boarderlines and
backgrounds). By contrast, our WBP backpropagates the weights through layers
to compute the contributions of each input pixel, which is truly faithful to
the model, and WBP tends to highlight the target objects themselves rather
than the background. At the same time, the simplicity and efficiency makes WBP
easily work with other advanced learning strategies for both model diagnosis
and improvements during training.

Our work is in a similar spirit to [18, 10, 6, 60], where meaningful
perturbations have been applied to the image during model training, to improve
prediction and facilitate interpretation. Poineering works have relied on user
supplied “ground-truth” explainable masks to perturb [46, 35, 45], however
such manual annotations are costly and hence rarely available in practice.
Alternatively, perturbations can be computed by solving an optimization for
each image. Such strategies are costly in practice and also do not effectively
block spurious features. Very recently, exploratory effort has been made to
leverage the tools from counterfactual reasoning [21] and causal analysis [40]
to derive visual explanations, but do not lend insights back to model
training. Our work represents a fast, principled solution that overcomes the
above limitations. It automatically derives explainable masks faithful to the
model and data, without explicit supervision from user-generated explanations.

Contrastive Learning.  There has been growing interest in exploiting
contrastive learning (CL) techniques for representations learning [39, 9, 24,
29, 59]. Originally devised for density estimation [23], CL exploits the idea
of learning by comparison to capture the subtle features of data, i.e.,
positive examples, by contrasting them with negative examples drawn from a
carefully crafted noise distribution. These techniques aim to avoid
representation collapse, or to promote representation consistency, for
downstream tasks. Recent developments, both empirical and theoretical, have
connected CL to information-theoretic foundations [59, 22], thus establishing
them as a suite of de facto solutions for unsupervised representation learning
[9, 24].

The basic form of CL is essentially a binary classification task specified to
discriminate positive and negative examples. In such a scenario, the binary
classifier is known as the critic function. Maximizing the discriminative
power wrt the critic and the representation sharpens the feature encoder.
Critical to the success of CL is the choice of appropriate noise distribution,
where the challenging negatives, i.e., those negatives that are more similar
to positive examples, are often considered more effective contrasts. In its
more generalized form, CL can naturally repurpose the predictor and loss
functions without introducing a new critic [59]. Notably, current CL methods
are not immune to spurious associations, a point we wish to improve in this
work.

Causality and Interventions. From a causality perspective, humans learn via
actively interacting with the environment. We intervene and observe changes in
the outcome to infer causal dependencies. Machines instead learn from static
observations that are unable to inform the structural dependencies for causal
decisions. As such, perturbations to the external factors, e.g., surroundings,
lighting, viewing angles, may drastically alter machine predictions, while
human recognition is less susceptible to such nuisance variations. Formally,
such difference is best explained with the $do$-notation [41]:
$\mathbb{P}(Y|do(x))=\sum_{z}\mathbb{P}(Y|X=x,z)\mathbb{P}(z)$, where we
identify $x$ as the features, e.g., an object in the image, and $z$ as the
confounders, e.g., background in the example above. Note that
$\mathbb{P}(Y|do(x))$ is fundamentally different from the conditional
likelihood $\mathbb{P}(Y|X=x)=\sum_{z}\mathbb{P}(Y|X=x,z)\mathbb{P}(z|X=x)$,
which machine uses for associative reasoning.

Unfortunately, carrying out real interventional studies, i.e., randomized
control trials, to intentionally block non-causal associations, is oftentimes
not a feasible option for practical considerations, e.g., due to cost and
ethics. This work instead advocates the application of synthetic interventions
to uncover the underlying causal features from observational data.
Specifically, we proactively edit $x$ and its corresponding label $y$ in a
data-driven fashion to encourage the model to learn potential causal
associations. Our proposal is in line with the growing appreciation for the
significance of establishing causality in machine learning models [49]. Via
promoting invariance [2], such causally inspired solutions demonstrate
superior robustness to superficial features that do not generalize [62]. In
particular, [57, 67] showed the importance and effectiveness of accounting for
interventional perspectives. Our work brings these causal views to construct a
simple solution that explicitly optimizes visual interpretation and model
robustness.

Figure 2: Illustration of the proposed PPI learning strategy. Input images are
intervened by removing the saliency map based masks, which alters the input
label (e.g., negative control). For positive contrast, we use the original
input as well as an input masked with a random slaiency map. We use WBP for
the generation of saliency maps.

##  3 Proactive Pseudo-Intervention

Below we describe the construction of Proactive Pseudo-Intervention (PPI), a
causally-informed contrastive learning scheme that seeks to simultaneously
improve the accuracy, robustness, generalization and interpretability of deep-
learning-based computer vision models.

The PPI learning strategy, schematically summarized in Figure 2, consists of
three main components: ($i$) a saliency mapping module that highlights
causally relevant features; ($ii$) an intervention module that synthesizes
contrastive samples; and ($iii$) the prediction module, which is standard in
recent vision models, e.g., VGG [55], ResNet [25], and Inception Net [58].
Motivated by the discussions from our introduction, PPI establishes a feedback
loop between the saliency map module and the prediction module, which is
interfaced by the synthesized contrastive examples in the intervention module.
Under this configuration, the prediction module is encouraged to modify its
predictions only when provided with causally-relevant synthetic interventions.
Note that components ($i$) and ($ii$) do not involve any additional parameters
or neural network modules, which makes our strategy readily applicable to the
training of virtually any computer vision task without major customization.
Details of these building blocks are given below.

###  3.1 Synthetic causal interventions for contrasts

Key to our formulation is the design of a synthetic intervention strategy that
generates contrastive examples to reinforce causal relevance during model
training. Given a causal saliency map ${\bm{s}}_{m}({\bm{x}})$ for an input
${\bm{x}}$ wrt label $y=m$, where $m=1,\ldots,M$, and $M$ is the number of
classes, the synthetic intervention consists of removing (replacing with zero)
the causal information from ${\bm{x}}$ contained in ${\bm{s}}_{m}({\bm{x}})$,
and then using it as the contrastive learning signal.

For now, let us assume the causal salience map ${\bm{s}}_{m}({\bm{x}})$ is
known; the procedure to obtain the saliency map will be addressed in the next
section. For notational clarity, we use subscript $i$ to denote entities
associated with the $i$-th training sample, and omit the dependency on
learnable parameters. To remove causal information from ${\bm{x}}_{i}$ and
obtain a negative contrast ${\bm{x}}_{i}^{*}$, we apply the following soft-
masking

${\bm{x}}_{i}^{*}={\bm{x}}_{i}-T({\bm{s}}_{m}({\bm{x}}_{i}))\odot{\bm{x}}_{i},$
(1)

where $T(\cdot)$ is a differentiable masking function and $\odot$ denotes
element-wise (Hadamard) multiplication. Specifically, we use the thresholded
sigmoid for masking:

$T({\bm{s}}_{m}({\bm{x}}_{i}))=\frac{1}{1+\exp(-\omega({\bm{s}}_{m}({\bm{x}}_{i})-\mathbf{\sigma}))},$
(2)

where $\sigma$ and $\omega>0$ are the threshold and scaling parameters,
respectively. We set the scaling $\omega$ so that $T({\bm{s}})$ will result in
a sharp transition from $0$ to $1$ near $\sigma$. Using (1) we define the
contrastive loss as

$L_{con}(\theta)=\sum_{i}\ell({\bm{x}}_{i}^{*},\neg y;f_{\theta}),$ (3)

where $f_{\theta}$ is the prediction module, $\ell({\bm{x}},y;f_{\theta})$ is
the loss function we wish to optimize, e.g. cross entropy, and $\neg$ is used
to denote that the original class label has been flipped. In the binary case,
$\neg y=1-y$, and in the multi-class case it can be interpreted accordingly,
e.g., using a one vs. others cross entropy loss. In practice, we set
$\ell({\bm{x}},y;f_{\theta})=-\ell({\bm{x}},y;f_{\theta})$. We will show in
the experiments that this simple and intuitive causal masking strategy works
well in practice (see Tables 2 and 4, and Figure 4). Alternatively, we also
consider a hard-masking approach in which a minimal bounding box covering the
thresholded saliency map is removed. See the Appendix for details.

Note that we are making the implicit assumption that the saliency map is
uniquely determined by the prediction module $f_{\theta}$. While optimizing
(3) explicitly attempts to improve the fit of the prediction module
$f_{\theta}$, it also implicitly informs the causal saliency mapping. This is
sensible because if a prediction is made using non-causal features, which
implies the associated saliency map ${\bm{s}}_{m}({\bm{x}})$ is also non-
causal, then we should expect that after applying ${\bm{s}}_{m}({\bm{x}})$ to
${\bm{x}}$ using (1), we can still expect to make the correct prediction,
i.e., the true label, for both positive (the original) and negative (the
intervened) samples.

Saliency map regularization. Note that naively optimizing (3) can lead to
degenerate solutions for which any saliency map that satisfies the causal
sufficiency, i.e., encompassing all causal features, is a valid causal
saliency map. For example, a trivial solution where the saliency map covers
the entire image may be considered causal. To protect against such degeneracy,
we propose to regularize the $L_{1}$-norm of the saliency map to encourage
succinct (sparse) representations, i.e., $L_{reg}=\|{\bm{s}}_{m}\|_{1}$, for
$m=1,\ldots,M$.

Adversarial positive contrasts. Another concern with solely optimizing (3) is
that models can easily overfit to the intervention, i.e., instead of learning
to capture causal relevance, the model learns to predict interventional
operations. For example, the model can learn to change its prediction when it
detects that the input has been intervened, regardless of whether the image is
missing causal features. So motivated, we introduce adversarial positive
contrasts:

${\bm{x}}^{\prime}_{i}={\bm{x}}_{i}-T({\bm{s}}_{m}({\bm{x}}_{j}))\odot{\bm{x}}_{i},\quad
i\neq j,$ (4)

where we intervene with a false saliency map, i.e.,
${\bm{s}}_{m}({\bm{x}}_{j})$ is the saliency map from a different input
${\bm{x}}_{j}$, while still encouraging the model to make the correct
prediction via

$L_{ad}(\theta)=\sum_{i}\ell({\bm{x}}_{i}^{\prime},y;f_{\theta})\,,\vspace{-3mm}$
(5)

where ${\bm{x}}^{\prime}_{i}$ is the adversarial positive contrast. The
complete loss for the proposed model, $L=L_{cls}+L_{con}+L_{reg}+L_{ad}$,
consists of the contrastive loss in (3), the regularization loss, $L_{reg}$,
and the adversarial loss in (5).

###  3.2 Saliency Weight Backpropagation

In order to generate saliency maps that inform decision-driving features in
the (raw) pixel space, we describe Weight Back Propagation (WBP), a novel
computationally efficient scheme for saliency mapping applicable to arbitrary
neural architectures. WBP evaluates individual contributions from each pixel
to the final class-specific prediction, and we empirically find the results to
be more causally-relevant relative to competing solutions based on human
judgement.

Table 1: WBP update rules for common transformations. Transformation | $G(\cdot)$
---|---
Activation Layer | $\tilde{W}^{l}=h\circ{\tilde{W}^{l+1}}$
FC Layer | $\tilde{W}^{l}=\tilde{W}^{l+1}W^{l}$
Convolutional Layer | $\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]^{T_{0,1}}_{flip_{2,3}}}$
BN Layer | $\tilde{W}^{l}=\frac{\tilde{W}^{l+1}}{\sigma}\gamma$
Pooling Layer | Relocate/Distribute $\tilde{W}^{l+1}$

To simplify our presentation, we first consider a vector input and a linear
mapping. Let ${\bm{x}}^{l}$ be the internal representation of the data at the
$l$-th layer, with $l=0$ being the input layer, i.e., ${\bm{x}}^{0}={\bm{x}}$,
and $l=L$ being the penultimate logit layer prior to the softmax
transformation, i.e., $\mathbb{P}(y|{\bm{x}})={\rm softmax}({\bm{x}}^{L})$. To
assign the relative importance to each hidden unit in the $l$-th layer, we
notationally collapse all transformations after $l$ into an operator denoted
by $\tilde{W}^{l}$, which we call the saliency matrix, satisfying,

${\bm{x}}^{L}=\tilde{W}^{l}{\bm{x}}^{l},\quad\forall l\in[0,\ldots,L],$ (6)

where ${\bm{x}}^{L}$ is an $M$-dimensional vector corresponding to the $M$
distinct classes in $y$. Though presented in a matrix form in a slight abuse
of notation, i.e., the instantiation of the operator $\tilde{W}^{l}$
effectively depends on the input ${\bm{x}}$, thus all nonlinearities have been
effectively absorbed into it. We posit that for an object associated with a
given label $y=m$, its causal features are subsumed in the interactions
between the $m$-th row of $\tilde{W}^{0}$ and input ${\bm{x}}$, i.e.,

$[{\bm{s}}_{m}({\bm{x}})]_{k}=[\tilde{W}^{0}]_{mk}[{\bm{x}}]_{k},$ (7)

where $[{\bm{s}}_{m}({\bm{x}})]_{k}$ denotes the $k$-th element of the
saliency map ${\bm{s}}_{m}({\bm{x}})$ and $[\tilde{W}^{0}]_{mk}$ is a single
element of $\tilde{W}^{0}$. A key observation for computation of
$\tilde{W}^{l}$ is that it can be done recursively. Specifically, let
$g_{l}({\bm{x}}^{l})$ be the transformation at the $l$-th layer, e.g., an
affine transformation, convolution, activation, normalization, etc., then it
holds that

$\tilde{W}^{l+1}{\bm{x}}^{l+1}=\tilde{W}^{l+1}g_{l}({\bm{x}}^{l})=\tilde{W}^{l}{\bm{x}}^{l}.$
(8)

This allows for recursive computation of $\tilde{W}^{l}$ via

$\tilde{W}^{l}=G(\tilde{W}^{l+1},g_{l}),\quad\tilde{W}^{L}=1,$ (9)

where $G(\cdot)$ is the update rule. We list the update rules for common
transformations in deep networks in Table 1, with corresponding derivations
detailed below.

Fully-connected (FC) layer. The FC transformation is the most basic operation
in deep neural networks. Below we omit the bias term as it does not directly
interact with the input. Assuming $g_{l}({\bm{x}}^{l})=W^{l}{\bm{x}}^{l}$, it
is readily seen that

$\tilde{W}^{l+1}{\bm{x}}^{l+1}=\tilde{W}^{l+1}g_{l}({\bm{x}}^{l})=(\tilde{W}^{l+1}W^{l}){\bm{x}}^{l},$
(10)

so $\tilde{W}^{l}=\tilde{W}^{l+1}W^{l}$. Graphical illustration with standard
affine mapping and ReLU activation can be found in the appendix.

Figure 3: Visualization of the inferred saliency maps. Left: CUB dataset.
Right: ImageNet dataset.

Nonlinear activation layer. Considering that an activation layer simply
rescales the saliency weight matrices, i.e.,
${\bm{x}}^{l+1}=g_{l}({\bm{x}}^{l})=h^{l}\circ{\bm{x}}^{l}$, where $\circ$ is
the composition operator, we obtain $\tilde{W}^{l}=h\circ\tilde{W}^{l+1}$.
Using the ReLU activation as a concrete example, we have
$h({\bm{x}}^{l})=\mathbbm{1}\\{{\bm{x}}^{l}\geq 0\\}$.

Convolutional layer. The convolution is a generalized form of linear mapping.
In practice, convolutions can be expressed as tensor products of the form
$\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]^{T_{0,1}}_{flip_{2,3}}}$, where
$W^{l}\in{\mathbb{R}^{D_{2}\times{D_{1}}\times{(2S+1)}\times{(2S+1)}}}$ is the
convolution kernel, $T_{0,1}$ is the transpose in dimensions 0 and 1 and
$flip_{2,3}$ is an exchange in dimensions 2 and 3. See the Appendix for
details.

Pooling and normalization layer. Summarization and standardization are two
other essential operations for the success of deep neural networks, achieved
by pooling and batch normalization (BN) techniques, respectively. They too can
be considered as special instantiations of linear operations. Here we
summarize the two most popular operations in Table 1.

##  4 Experiments

To validate the utility of our approach, we consider both natural and medical
image datasets, and compare it to existing state-of-the-art solutions. All the
experiments are implemented in PyTorch. The source code will be available at
https://github.com/author_name/PPI. Due to space limitation, details of the
experimental setup and additional analyses are deferred to the Appendix.

Datasets. We present our findings on five representative datasets: ($i$)
CIFAR-10 [31]; ($ii$) ImageNet (ILSVRC2012) [47]; ($iii$) CUB [61], a natural
image dataset with over $12k$ photos for classification of $200$ bird species
in the wild, heavily confounded by the background characteristics; ($iv$) GA
[34], a new medical image dataset for the prediction of geographic atrophy
(GA) using 3D optical coherence tomography (OCT) image volumes, characterized
by small sample size ($275$ subjects) and highly heterogeneous (collected from
$4$ different facilities); and ($v$) LIDC-IDRI [33], a public medical dataset
of $1,085$ lung lesion CT images annotated by $4$ radiologists. Detailed
specifications are described in the Appendix.

Baselines. The following set of popular saliency mapping schemes are
considered as comparators for the proposed approach: ($i$) Gradient: standard
gradient-based salience mapping; ($ii$) GradCAM [51]: gradient-weighted class
activation mapping; ($iii$) LRP [4]: layer-wise relevance propagation and its
variants.

Hyperparameters. The final loss of the proposed model is a weighted summation
of four losses: $L=L_{cls}+w_{1}L_{con}+w_{2}L_{reg}+w_{3}L_{ad}$. The weights
are simply balanced to match the magnitude of $L_{cls}$, _i.e._ , $w_{3}=1$,
$w_{2}=0.1$ and $w_{1}=1$ (CUB), $=1$ (GA), and $=10$ (LIDC). See Appendix Sec
B for details about the masking parameters $\sigma$ and $\omega$.

###  4.1 Natural Image Datasets

Classification Gains In this experiment, we investigate how the different
pairings of PPI and saliency mapping schemes (i.e., GradCAM, LRP, WBP) affect
performance. In Table 2, the first row represents VGG11 model trained with
only classification loss, and the following rows represent VGG11 trained with
PPI with different saliency mapping schemes. We see consistent performance
gains in accuracy via incorporating PPI training on both CUB and CIFAR-10
datasets. The gains are mostly significant when using our WBP for saliency
mapping (improving the accuracy from $0.662$ to $0.696$ on CUB, and from
$0.881$ to $0.901$ on CIFAR-10.

Table 2: Performance improvements achieved by training with PPI on CUB, CIFAR-10, and GA dataset. We report means and standard deviations (SDs) from 5-fold cross-validation for GA prediction. Models | CUB | Cifar-10 | GA
---|---|---|---
| (Acc) | (Acc) | (AUC)
Classification | $0.662$ | $0.881$ | $0.877\pm 0.040$
+$\text{PPI}_{Gradient}$ | ${0.673}$ | $0.885$ | $0.890\pm 0.035$
+$\text{PPI}_{LRP}$ | ${0.680}$ | $0.891$ | $0.895\pm 0.037$
+$\text{PPI}_{GradCAM}$ | ${0.683}$ | $0.895$ | $0.908\pm 0.036$
+$\text{PPI}_{WBP}$ | 0.696 | 0.901 | $0.925\pm 0.023$
Figure 4: Saliency maps on GA dataset based on models trained with PPI and
without PPI. Maps of models trained with PPI are more clinically relevant by
focusing on retinal layers likely to contain abnormalities or lesions, and
more concentrated.

Model Interpretability In this task, we want to qualitatively and
quantitatively compare the causal relevance of saliency maps generated by our
proposed model and its competitors. In Figure 3, we show the saliency maps
produced by different approaches for a VGG11 model trained on CUB. Visually,
gradient-based solutions (Grad and GradCAM) tend to yield overly dispersed
maps, indicating a lack of specificity. LRP gives more appealing saliency
maps. However, these maps also heavily attend to the spurious background cues
that presumably help with predictions. When trained with PPI, the saliency
maps attend to birds body, and with WBP, the saliency maps focus on the causal
related pixels.

To quantitatively evaluate the causal relevance of competing saliency maps, we
adopt the evaluation scheme proposed in [26], consisting of masking out the
contributing saliency pixels and then calculating the reduction in prediction
score. A larger reduction is considered better for accurately capturing the
pixels that ‘cause’ the prediction. Results are summarized in Figure 5(a),
where we progressively remove the top-$k$ saliency points, with
$k=100,500,1000,5000,10000$ ($10000\approx 6.6\%$ of all pixels), from the CUB
test input images. Our PPI consistently outperforms its counterparts, with its
lead being most substantial in the low-$k$ regime. Notably, for large $k$, PPI
removes nearly all predictive signal. This implies PPI specifically targets
the causal features. Quantitative evaluation with additional metrics are
provided in the Appendix.

To test the performance of WBP itself (without being trained with PPI), we
compare WBP with different approaches for a VGG11 model trained on ImageNet
from PyTorch model zoo. Figure 3(left) shows that saliency maps generated by
WBP more concentrate on objects themselves. Also, thanks to the fine
resolution of WBP, the model pays more attention to the patterns on the fur to
identify the leopard (row 1). This is more visually consistent with human
judgement. Figure 5(b) demonstrates WBP identifies more causal pixels on
ImageNet validation images.

(a) CUB

(b) ImageNet

Figure 5: Quantitative evaluations of causal relevance of competing saliency
maps (higher is better).

###  4.2 OCT-GA: Geographic Atrophy Classification

Next we show how the proposed PPI handles the challenges of small training
data and heterogeneity in medical image datasets. In this experiment (with our
new dataset, that we will make public), each OCT volume image consists of
$100$ scans of a $512\times 1000$ sized image [5]. We use a multi-view CNN
model [56] to process such 3D OCT inputs, and use it as our baseline solution
(see the Appendix for details). We investigate how the different saliency
mapping schemes (i.e., Grad, GradCAM, LRP, WBP) work with PPI. For WBP, we
also tested the bounding box variant, denoted as WBP (box) (see the Appendix
for details). In Table 2, we see consistent performance gains in AUC score via
incorporating PPI training (from $0.877$ to $0.925$, can be improve to $0.937$
by PPI with WBP(box)), accompanied by the reductions in model variation
evaluated by the standard deviations of AUC from the five-fold cross-
validation. The gains are most significant when using our WBP for saliency
mapping. We further compare the saliency maps generated by these different
combinations. We see that without the additional supervision from PPI,
competing solutions like Grad, GradCAM and LRP sometimes yield non-sensible
saliency maps (attending to image corners). Overall, PPI encourages more
concentrated and less noisy saliency maps. Also, different PPI-based saliency
maps agree with each other to a larger extent. Our findings are also verified
by experts (co-authors, who are ophthalmologists specializing in GA)
confirming that the PPI-based saliency maps are clinically relevant by
focusing on retinal layers likely to contain abnormalities or lesions. These
results underscore the practical value of the proposed proactive
interventions.

Cross-domain generalization. Common to medical image applications is that
training samples are usually integrated from a number of healthcare facilities
(i.e., domains), and that predictions are sometimes to be made on subjects at
other facilities. Despite big efforts to standardize the image collection
protocols, with different imaging systems operated by technicians with varying
skills, apparent domain shifts are likely to compromise the cross-domain
performance of these models. We show this phenomenon on the GA dataset in
Table 3, where source samples are collected from four different hospitals in
different health systems (A, B, C and D, see the Appendix for details). Each
cell contains the AUC of the model trained on site X (row) and tested on site
Y (column), with same-site predictions made on hold-out samples. A significant
performance drop is observed for cross-domain predictions (off-diagonals)
compared to in-domain predictions (diagonals). With the application of PPI,
the performance gaps between in-domain and cross-domain predictions are
considerably reduced. The overall accuracy gains of PPI further justify the
utility of causally-inspired modeling. Notably, site D manifests strong
spurious correlation that help in-domain prediction but degrades out-of-site
generalization, which is partly resolved by the proposed PPI.

Table 3: AUC results for GA prediction with or without PPI. Models are trained
on one site and cross-validated on the other sites. Darker color indicates
better performance.

With PPI | A | B | C | D | Mean | STD
---|---|---|---|---|---|---
A | 1.000 | 0.906 | 0.877 | 0.865 | 0.912 | 0.061
B | 0.851 | 0.975 | 0.863 | 0.910 | 0.900 | 0.056
C | 0.954 | 0.875 | 0.904 | 0.931 | 0.916 | 0.034
D | 0.824 | 0.846 | 0.853 | 0.904 | 0.857 | 0.034
No PPI | A | B | C | D | Mean | STD
A | 1.000 | 0.854 | 0.832 | 0.827 | 0.878 | 0.082
B | 0.810 | 0.874 | 0.850 | 0.906 | 0.860 | 0.040
C | 0.860 | 0.779 | 0.873 | 0.862 | 0.843 | 0.043
D | 0.748 | 0.792 | 0.836 | 0.961 | 0.834 | 0.092

###  4.3 LIDC-IDRI: Lung Lesions Classification

Table 4: LIDC-IDRI classification AUC results.

Models | AUC
---|---
Tensor Net-X [15] | 0.823
DenseNet [27] | 0.829
LoTeNet [50] | 0.874
Inception_v3 [58] | 0.921
+$\text{PPI}_{GradCAM}$ | 0.933
+$\text{PPI}_{Gradient}$ | 0.930
+$\text{PPI}_{LRP}$ | 0.931
+$\text{PPI}_{WBP}$ | 0.935
+$\text{PPI}_{WBP(box)}$ | 0.941

To further examine the practical advantages of the proposed PPI in real-world
applications, we benchmark its utility on LIDC-IDRI; a public lung CT scan
dataset [3]. We followed the preprocessing steps outlined in [30] to prepare
the data, and adopted the experimental setup from [50] to predict lesions. We
use Inception_v3 [58] as our base model for both standard classification and
PPI-enhanced training with various saliency mapping schemes. See the Appendix
for details.

Lesion classification. We first compare PPI to other specialized SOTA network
architectures. Table 4 summarizes AUC scores of Tensor Net-X [15], DenseNet
[27], LoTeNet [50], Inception_v3 [58], as well as our Inception_v3 trained
with and without $\text{PPI}_{WBP}$. The proposed $\text{PPI}_{WBP(box)}$
leads the performance chart by a considerable margin, improving Inception_v3
from $0.92$ to $0.94$.

Figure 6: Saliency maps on LIDC-IDR. Saliency maps of PPI+WBP are mostly
consistent with the ground truths.

Weakly-supervised image segmentation. In Figure 6, we compare saliency maps
generated by GradCAM, WBP, WBP (box) to the ground truth lesion masks from
expert annotations. Note that we have only supplied patch-label labels during
training, not the pixel-level expert segmentation masks, which constitute a
challenging task of weakly-supervised image segmentation. In line with the
observations from the GA experiment, our PPI-training enhanced WBP saliency
maps are mostly consistent with the expert segmentations. Together with Table
4, Figure 6 confirms that the proposed PPI+WBP improves both the
classification performance and model interpretability.

##  5 Conclusions

We have presented Proactive Pseudo-Intervention (PPI), a novel interpretable
computer vision framework that organically integrates saliency mapping, causal
reasoning, synthetic intervention and contrastive learning. PPI couples
saliency mapping with contrastive training by creating artificially intervened
negative samples absent of causal features. To communicate model insights and
facilitate causal-informed reasoning, we derived an architecture-agnostic
saliency mapping scheme called Weight Back Propagation (WBP), which faithfully
captures the causally-relevant pixels/features for model prediction. Visual
inspection of the saliency maps show that WBP, is more robust to spurious
features compared to competing approaches. Empirical results on natural and
medical datasets verify the combination of PPI and WBP consistently delivers
performance boosts across a wide range of tasks relative to competing
solutions, and the gains are most significant where the application is
complicated by small sample size, data heterogeneity, or confounded with
spurious correlations.

## References

  * [1] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim.  Sanity checks for saliency maps.  In Advances in Neural Information Processing Systems, pages 9505–9515, 2018.
  * [2] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz.  Invariant risk minimization.  arXiv preprint arXiv:1907.02893, 2019.
  * [3] Samuel G Armato III, Geoffrey McLennan, Luc Bidaut, Michael F McNitt-Gray, Charles R Meyer, Anthony P Reeves, Binsheng Zhao, Denise R Aberle, Claudia I Henschke, Eric A Hoffman, et al.  The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans.  Medical physics, 38(2):915–931, 2011.
  * [4] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  PloS one, 10(7):e0130140, 2015.
  * [5] David S Boyer, Ursula Schmidt-Erfurth, Menno van Lookeren Campagne, Erin C Henry, and Christopher Brittain.  The pathophysiology of geographic atrophy secondary to age-related macular degeneration and the complement pathway as a therapeutic target.  Retina (Philadelphia, Pa.), 37(5):819, 2017.
  * [6] Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud.  Explaining image classifiers by counterfactual generation.  In International Conference on Learning Representations, 2018.
  * [7] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian.  Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.  In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 839–847. IEEE, 2018.
  * [8] Chenyi Chen, Ari Seff, Alain Kornhauser, and Jianxiong Xiao.  Deepdriving: Learning affordance for direct perception in autonomous driving.  In Proceedings of the IEEE International Conference on Computer Vision, pages 2722–2730, 2015.
  * [9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  arXiv preprint arXiv:2002.05709, 2020.
  * [10] Piotr Dabkowski and Yarin Gal.  Real time image saliency for black box classifiers.  In Advances in Neural Information Processing Systems, pages 6967–6976, 2017.
  * [11] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [12] Li Deng.  The mnist database of handwritten digit images for machine learning research [best of the web].  IEEE Signal Processing Magazine, 29(6):141–142, 2012.
  * [13] Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das.  Explanations based on the missing: Towards contrastive explanations with pertinent negatives.  In Advances in Neural Information Processing Systems, pages 592–603, 2018.
  * [14] Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu.  Towards explanation of dnn-based prediction with guided feature inversion.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1358–1367, 2018.
  * [15] Stavros Efthymiou, Jack Hidary, and Stefan Leichenauer.  Tensornetwork for machine learning.  arXiv preprint arXiv:1906.06329, 2019.
  * [16] Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Visualizing higher-layer features of a deep network.  University of Montreal, 1341(3):1, 2009.
  * [17] Ruth Fong, Mandela Patrick, and Andrea Vedaldi.  Understanding deep networks via extremal perturbations and smooth masks.  In Proceedings of the IEEE International Conference on Computer Vision, pages 2950–2958, 2019.
  * [18] Ruth C Fong and Andrea Vedaldi.  Interpretable explanations of black boxes by meaningful perturbation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 3429–3437, 2017.
  * [19] Hiroshi Fukui, Tsubasa Hirakawa, Takayoshi Yamashita, and Hironobu Fujiyoshi.  Attention branch network: Learning of attention mechanism for visual explanation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10705–10714, 2019.
  * [20] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  * [21] Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee.  Counterfactual visual explanations.  In ICML, 2019.
  * [22] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent-a new approach to self-supervised learning.  Advances in Neural Information Processing Systems, 33, 2020.
  * [23] Michael Gutmann and Aapo Hyvärinen.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models.  In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 297–304, 2010.
  * [24] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2020.
  * [25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [26] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim.  A benchmark for interpretability methods in deep neural networks.  In Advances in Neural Information Processing Systems, pages 9737–9748, 2019.
  * [27] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [28] Fredrik Johansson, Uri Shalit, and David Sontag.  Learning representations for counterfactual inference.  In International conference on machine learning, pages 3020–3029, 2016.
  * [29] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan.  Supervised contrastive learning.  arXiv preprint arXiv:2004.11362, 2020.
  * [30] Simon Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R Ledsam, Klaus Maier-Hein, SM Ali Eslami, Danilo Jimenez Rezende, and Olaf Ronneberger.  A probabilistic u-net for segmentation of ambiguous images.  In Advances in Neural Information Processing Systems, pages 6965–6975, 2018.
  * [31] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [32] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Communications of the ACM, 60(6):84–90, 2017.
  * [33] Curtis P Langlotz, Bibb Allen, Bradley J Erickson, Jayashree Kalpathy-Cramer, Keith Bigelow, Tessa S Cook, Adam E Flanders, Matthew P Lungren, David S Mendelson, Jeffrey D Rudie, et al.  A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 nih/rsna/acr/the academy workshop.  Radiology, 291(3):781–791, 2019.
  * [34] Jessica N Leuschen, Stefanie G Schuman, Katrina P Winter, Michelle N McCall, Wai T Wong, Emily Y Chew, Thomas Hwang, Sunil Srivastava, Neeru Sarin, Traci Clemons, et al.  Spectral-domain optical coherence tomography characteristics of intermediate age-related macular degeneration.  Ophthalmology, 120(1):140–150, 2013.
  * [35] Kunpeng Li, Ziyan Wu, Kuan-Chuan Peng, Jan Ernst, and Yun Fu.  Tell me where to look: Guided attention inference network.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9215–9223, 2018.
  * [36] Aravindh Mahendran and Andrea Vedaldi.  Salient deconvolutional networks.  In European Conference on Computer Vision, pages 120–135. Springer, 2016.
  * [37] Grégoire Montavon.  Gradient-based vs. propagation-based explanations: an axiomatic comparison.  In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pages 253–265. Springer, 2019.
  * [38] Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller.  Layer-wise relevance propagation: an overview.  In Explainable AI: interpreting, explaining and visualizing deep learning, pages 193–209. Springer, 2019.
  * [39] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, 2018.
  * [40] Matthew O’Shaughnessy, Gregory Canal, Marissa Connor, Christopher Rozell, and Mark Davenport.  Generative causal explanations of black-box classifiers.  Advances in Neural Information Processing Systems, 33, 2020.
  * [41] Judea Pearl.  Causality.  Cambridge university press, 2009.
  * [42] Vitali Petsiuk, Abir Das, and Kate Saenko.  Rise: Randomized input sampling for explanation of black-box models.  arXiv preprint arXiv:1806.07421, 2018.
  * [43] Sylvestre-Alvise Rebuffi, Ruth Fong, Xu Ji, and Andrea Vedaldi.  There and back again: Revisiting backpropagation saliency methods.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8839–8848, 2020.
  * [44] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ” why should i trust you?” explaining the predictions of any classifier.  In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  * [45] Laura Rieger, Chandan Singh, William Murdoch, and Bin Yu.  Interpretations are useful: penalizing explanations to align neural networks with prior knowledge.  In International Conference on Machine Learning, pages 8116–8126. PMLR, 2020.
  * [46] Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez.  Right for the right reasons: training differentiable models by constraining their explanations.  In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 2662–2670, 2017.
  * [47] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  International Journal of Computer Vision (IJCV), 115(3):211–252, 2015.
  * [48] Paul Sajda.  Machine learning for detection and diagnosis of disease.  Annu. Rev. Biomed. Eng., 8:537–565, 2006.
  * [49] Bernhard Schölkopf.  Causality for machine learning.  arXiv preprint arXiv:1911.10500, 2019.
  * [50] Raghavendra Selvan and Erik B Dam.  Tensor networks for medical image classification.  In Medical Imaging with Deep Learning, 2020.
  * [51] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE International Conference on Computer Vision, pages 618–626, 2017.
  * [52] Dasom Seo, Kanghan Oh, and Il-Seok Oh.  Regional multi-scale approach for visually pleasing explanations of deep neural networks.  IEEE Access, 8:8572–8582, 2019.
  * [53] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning important features through propagating activation differences.  In International Conference on Machine Learning, pages 3145–3153, 2017.
  * [54] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  arXiv preprint arXiv:1312.6034, 2013.
  * [55] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [56] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller.  Multi-view convolutional neural networks for 3d shape recognition.  In Proceedings of the IEEE international conference on computer vision, pages 945–953, 2015.
  * [57] Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, and Stefan Bauer.  Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness.  In International Conference on Machine Learning, pages 6056–6065. PMLR, 2019.
  * [58] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
  * [59] Yonglong Tian, Dilip Krishnan, and Phillip Isola.  Contrastive multiview coding.  arXiv preprint arXiv:1906.05849, 2019.
  * [60] Jorg Wagner, Jan Mathias Kohler, Tobias Gindele, Leon Hetzel, Jakob Thaddaus Wiedemer, and Sven Behnke.  Interpretable and fine-grained visual explanations for convolutional neural networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9097–9107, 2019.
  * [61] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie.  The Caltech-UCSD Birds-200-2011 Dataset.  Technical Report CNS-TR-2011-001, California Institute of Technology, 2011\.
  * [62] Haohan Wang, Zexue He, Zachary C Lipton, and Eric P Xing.  Learning robust representations by projecting superficial statistics out.  InInternational Conference on Learning Representations, 2019.
  * [63] Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu.  Score-cam: Score-weighted visual explanations for convolutional neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 24–25, 2020.
  * [64] Lezi Wang, Ziyan Wu, Srikrishna Karanam, Kuan-Chuan Peng, Rajat Vikram Singh, Bo Liu, and Dimitris N Metaxas.  Sharpen focus: Learning with attention separability and consistency.  In Proceedings of the IEEE International Conference on Computer Vision, pages 512–521, 2019.
  * [65] Tan Wang, Jianqiang Huang, Hanwang Zhang, and Qianru Sun.  Visual commonsense representation learning via causal inference.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 378–379, 2020.
  * [66] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In European conference on computer vision, pages 818–833. Springer, 2014.
  * [67] Cheng Zhang, Kun Zhang, and Yingzhen Li.  A causal view on robustness of neural networks.  arXiv preprint arXiv:2005.01095, 2020.
  * [68] Yitian Zhao, Yalin Zheng, Yifan Zhao, Yonghuai Liu, Zhili Chen, Peng Liu, and Jiang Liu.  Uniqueness-driven saliency analysis for automated lesion detection with applications to retinal diseases.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 109–118. Springer, 2018.
  * [69] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.

Figure 7: Illustration of the Weight Backpropagation (WBP) through a fully
connected layer with ReLU activation layer.

##  Appendix A Weight Backpropagation (WBP)

###  A.1 Graphical illustration of WBP

See Figure 7 for a graphical illustration with standard affine mapping and
ReLU activation.

###  A.2 Derivation of Convolutional Weight Backpropagation

Let’s denote the input variable as $I\in{\mathbb{R}^{H\times{W}}}$, the
convolutional filter weight as $W\in{\mathbb{R}^{(2S+1)\times{(2S+1)}}}$, the
output variable as $O\in{\mathbb{R}^{H\times{W}}}$, and the weight
backpropagate to $O$ as $\hat{W}\in{\mathbb{R}^{H\times{W}}}$. We omit the
bias here because it does not directly interact with the input variables. We
denote $\otimes$ as the convolutional operator. We have  $\displaystyle O$
$\displaystyle=I\otimes{W}$ $\displaystyle O_{i,j}$
$\displaystyle=\sum_{i^{\prime}=-S}^{S}\sum_{j^{\prime}=-S}^{S}I_{i+i^{\prime},j+j^{\prime}}W_{i^{\prime}+S,j^{\prime}+S}$
$\displaystyle\sum_{i}\sum_{j}O_{i,j}\tilde{W}_{i,j}$
$\displaystyle=\sum_{i}\sum_{j}\sum_{i^{\prime}=-S}^{S}\sum_{j^{\prime}=-S}^{S}I_{i+i^{\prime},j+j^{\prime}}W_{i^{\prime}+S,j^{\prime}+S}\tilde{W}_{i,j}$
$\displaystyle\sum_{i}\sum_{j}O_{i,j}\tilde{W}_{i,j}$
$\displaystyle=\sum_{i}\sum_{j}I_{i,j}\sum_{i^{\prime}=-S}^{S}\sum_{j^{\prime}=-S}^{S}\tilde{W}_{i+i^{\prime},j+j^{\prime}}W_{-i^{\prime}+S,-j^{\prime}+S}$
$\displaystyle\sum_{i}\sum_{j}O_{i,j}\tilde{W}_{i,j}$
$\displaystyle=\sum_{i}\sum_{j}I_{i,j}(\tilde{W}\otimes{[W]_{flip_{i,j}}})_{i,j}$

Hence the weight backpropagate through a convolutional layer is
$\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]_{flip}}$. For the 3D cases,
$I^{l}\in{\mathbb{R}^{D_{1}\times{H}\times{W}}}$,the weight back propagates to
$O^{l}$ is $\tilde{W}^{l+1}\in{\mathbb{R}^{D_{2}\times{H}\times{W}}}$ and the
convolutional weight is
$W^{l}\in{\mathbb{R}^{D_{2}\times{D_{1}}\times{(2S+1)}\times{(2S+1)}}}$. To
match the depth of $\tilde{W}^{l+1}$, the $W^{l}$ is transposed in the first
two dimensions. So
$\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]^{T_{0,1}}_{flip_{2,3}}}$. If the
convolutional layer is downsizing the input variable (i.e., strides), the
$\tilde{W}^{l+1}_{ijk}$ is padded with zeros around the weights
(left,right,up, and down) to for the input elements that the convolutional
filter strides over. The number of padding zeros is equal to the number of
strides minus 1.

##  Appendix B Details on Causal Masking

In this work, we consider three types of causal masking: ($i$) the point-wise
soft causal masking defined by Equation (2) in the main text, ($ii$) hard
masking, and ($iii$) box masking. For the hard masking, for each image, we
keep points with WBP weight larger than $k$ times of the standard deviation of
WBP weights of the whole image. We test $k$ from $1$ to $7$ and achieve
similar results. As $k=7$ performs slightly better, we set $k$ as $7$ for all
experiments. For the box masking, we use the center of mass for these kept
points as the center to draw a box. The height and width of this box is
defined as $center_{h/w}\pm{1.2std_{h/w}}$. In this way at least $90\%$ of
filtered points are contained in the box. For the soft masking, we set
$\omega$ to $100$ and $\sigma$ to $0.25$. We have also experimented with
image-adaptive thresholds instead of a fixed $\sigma$ for all inputs, i.e.,
set the threshold as mean value plus $k$ times of the standard deviation of
WBP weights of the whole image. We repeat the experiments a few times and the
results are consistent. The experiment comparison of these masking methods
mention above is conducted on LIDC dataset.

Table 5: Different causal masking methods on LIDC Models | AUC
---|---
WBP-soft (fixed $\sigma$) | 0.931
WBP-soft (adaptive $\sigma$) | 0.941
WBP-hard (point) | 0.935
WBP-hard (box) | 0.941

##  Appendix C Related Work

In this work, we propose a contrastive causal representation learning
strategy, i.e., Proactive Pseudo-Intervention (PPI), that leverages proactive
interventions to identify causally-relevant image features. This approach is
complemented with a novel causal salience map visualization module, i.e.,
Weight Back Propagation (WBP), that identifies important pixels in the raw
input image, which greatly facilitates interpretability of predictions.

Prior related works will be discussed in this section. Compared with
alternative post-hot saliency mapping methods, WBP outperforms these methods
as both a standalone causal saliency map and a trainable model for model
interpretation. Compared with other trainable interpretation models, the
proposed PPI+WBP improves both model performance and model interpretations.

###  C.1 Post-hoc Saliency Maps

We compare WBP with other post-hoc saliency mapping methods to show why WBP is
able to target the causal features, and generate more succinct and reliable
saliency maps.

Perturbation Based Methods These methods make perturbations to individual
inputs or neurons and monitor the impact on output neurons in the network.
[66] occludes different segments of an input image and visualized the change
in the activations of subsequent layers. Several methods follow a similar
idea, but use other importance measures or occlusion strategies [42, 44, 52].
More complicated works aim to generate an explanation by optimizing for a
perturbed version of the image [18, 17, 10, 14]. [60] proposes a new
adversarial defense technique which filters gradients during optimization to
achieve fine-grained explanation. However, such perturbation based methods are
computationally intensive and involve sophisticated model designs, which make
it extremely hard to be integrated with other advance learning strategies.

Backpropagation Based Methods Backpropagation based methods (BBM) propagate an
importance signal from an output neuron backwards through the layers to the
input. These methods are usually fast to compute and produce fine-grained
importance/relevancy maps. WBP is one of such method.

The pioneer methods in this category backpropagate a gradient to the image,
and branches of studies extend this work by manipulating the gradient. These
methods are discussed and compared in [36, 16]. However, these maps are
generally less concentrated [10, 18] and less interpretable. Other BBMs such
as Layer-wise Relevance Propagation [4], DeepLift [53] employ top-down
relevancy propagation rules. DeepLift is sensitive to the reference inputs,
which needs more human efforts and background knowledge to produce appealing
saliency maps. The nature of depending on reference inputs limits its ability
on model diagnosis and couple with learning strategies to continuously
improving models’ performance. LRP decomposes the relevance, $R$, from a
neuron, $k$, in the upper layer to every connected neurons, $j$, in the lower
layer. The decomposition is distributed through gradients under the suggested
implementation [38]. Our experiments on GA and CUB datasets show that vanilla
LRP performs similar to gradient based methods, which is also demonstrated in
[37]. The variants of LRP use complex rules to prioritize positive or large
relevance, making the saliency map visually appealing to human. However, our
experiments demonstrate the unfaithfulness of LRP and its variants as they
highlight spuriously correlated features (boarderlines and backgrounds). By
contrast, our WBP backpropagates the the weights of through layers to compute
the contributions of each input pixel, which is truly faithful to the model,
and WBP tends to highlight the target objects themselves rather than the
background. At the same time, the simplicity and efficiency makes WBP easily
work with other advanced learning strategies for both model diagnosis and
improvements during training.

Table 6: A list of commonly used LRP rules.[38] Rules | Formula
---|---
$LRP$ | $R_{j}=\sum_{k}\frac{a_{j}w_{jk}}{\sum_{0,j}a_{j}w_{jk}}R_{k}$
$LRP_{\epsilon}$ | $R_{j}=\sum_{k}\frac{a_{j}w_{jk}}{\epsilon+\sum_{0,j}a_{j}w_{jk}}R_{k}$
$LRP_{\gamma}$ | $R_{j}=\sum_{k}\frac{a_{j}(w_{jk}+\gamma w^{+}_{jk})}{\sum_{0,j}a_{j}(w_{jk}+\gamma w^{+}_{jk})}R_{k}$
$LRP_{\alpha\beta}$ | $R_{j}=\sum_{k}(\alpha\frac{(a_{j}w_{jk})^{+}}{\sum_{0,j}(a_{j}w_{jk})^{+}}-\beta\frac{(a_{j}w_{jk})^{-}}{\sum_{0,j}(a_{j}w_{jk})^{-}})R_{k}$
$LRP_{flat}$ | $R_{j}=\sum_{k}\frac{1}{\sum_{j}1}R_{k}$
$LRP_{w^{2}}$ | $R_{j}=\sum_{j}\frac{w_{ij}^{2}}{\sum_{i}w^{2}_{ij}}R_{j}$
$LRP_{Z^{\beta}}$ | $R_{j}=\sum_{j}\frac{x_{i}w_{ij}-l_{i}w^{+}_{ij}-h_{i}w^{-}{ij}}{\sum_{i}x_{i}w_{ij}-l_{i}w^{+}_{ij}-h_{i}w^{-}{ij}}R_{j}$

Activation Based Methods Methods under this category (such as CAM, Grad-CAM,
guided Grad-CAM, Grad-CAM++) use a linear combination of class activation maps
from convolutional layers to derive a saliency map. The main difference
between them is how to the linear combination weights are computed. The
generation of saliency maps is easy and these methods can be coupled with
advanced training strategies to improve training [35]. However, they fail at
visualizing fine-grained evidence, which is particularly important in
explaining medical classification models. Additionally, it is not guaranteed
that the resulting explanations are faithful and reflect the decision making
process of the model [14, 51, 60]. Grad-CAM++ [7] proposes to introduce
higher-order derivatives to capture pixel-level importance, while its high
computational cost in calculating the second- and third-order derivatives
makes it impractical for training purposes.

Figure 8: Saliency maps on CUB dataset.

###  C.2 Interpretable Models

Unlike the post hoc saliency map generation described above, an alternative
approach is to train a separate module to explicitly produce model
explanations [19, 21, 6, 18, 53]. Such post hoc causal explanations can be
generated with black-box classifiers based on a learned low-dimensional
representation of the data [40] . Related to our work is adversarial-based
visual explanation method is developed in [60], highlighting the key features
in the input image for a specific prediction. Contrastive explanations are
produced in [13] to justify the predictions from a deep neural network. Also
in [21] the authors generate counterfactual visual explanations that highlight
what and how regions of an image would need to change in order for the model
to predict a distractor class $c^{\prime}$ instead of the predicted class $c$.
The main differences to our construction are two fold: ($i$) they rely on a
separate module to be trained, and ($ii$) these approaches only produce
explanations, but such explanations are not exploited to provide feedback for
model improvement.

Striking the goal of both good explanation and good performance is more
challenging. One promising direction is to inject model-dependent
perturbations to the input images as strategic augmentations [18, 10, 6]. In
such examples, parts of the image have been masked and replaced with various
references such as mean pixel values, blurred image regions, random noise,
outputs of generative models, etc. However, these pixel-level perturbations
are very costly and difficult to craft. [64] propose new learning objectives
for attention separability and cross-layer consistency, which result in
improved attention discriminability and reduced visual confusion. However, it
generates heat-map style attention maps, which fail in fine-resolution model
explanations which is important in medical related tasks. In [19] an
additional attention branch is learned to generate attention map, and then
applies the attention map to the original image or feature map; they achieve
compelling attention maps on natural images. However, as the attention maps
are not derived directly from the classification model, there is no guarantee
for their faithfulness. Further, having an additional attention network
results in increased network size, which raises concerns for the risk of over-
fitting, particularly on datasets with a limited sample size.

Figure 9: Saliency maps on ImageNet dataset.

##  Appendix D Natural Image Datasets

###  D.1 CUB Experiments

CUB dataset descriptions and experiment settings. CUB has 11,788 images of 200
bird spices. To train a VGG11 network, we use 8,190 training images and
validate the model on 2,311 validation images, with the accuracy are reported
on the remaining 1,227 testing images. The network is trained for 100 epochs
with a learning rate decay of 0.1 every 30 epochs. The batch size is 32. The
optimizer is SGD with initial learning rate at 0.01.

Classification performance improvement with PPI. We compare classification
performance among models trained with different schemes. The baseline is VGG11
classification without PPI. Three different saliency mapping methods are
tested within our PPI framework: Gradient, LRP, Grad-CAM, and WBP. Top 1k
points with the largest saliency scores in all saliency maps are used to
generate the soft mask so the comparison is fair. During training, since only
a small portion of points are used to generate the mask, the scale of
$L_{con}$ is about 100 times smaller than other losses. To fix this imbalance,
$L_{con}$ is weighted by a factor of 100$\times$ after the first 20 epochs.
The results are shown in Table 2.

Model Interpretability More saliency maps on CUB dataset are shown in Figure.
8. Visually, gradient-based solutions (Grad and GradCAM) tend to yield overly
dispersed maps, indicating a lack of specificity. LRP gives more appealing
saliency maps. However, these maps also heavily attend to the spurious
background cues that presumably help with predictions. When trained with PPI,
the saliency maps attend to birds body, and with WBP, the saliency maps focus
on the causal related pixels. Quantitative Test. We also measure the quality
of the generated saliency maps through their localization ability. Extending
from pointing game which uses only the maximum point (energy-based pointing
game [63]), we care about how much energy of the saliency map falls into the
target object bounding box. Evaluation result is shown in Table 7. We see our
WBP outperforms prior arts by a large margin, more than $93\%$ energy of the
top $100$ saliency points fall into the ground truth bounding box of the
target object.

Table 7: Energry based test on CUB dataset (higher is better). Point Num | LRP | LRPflat | LRPϵ | LRPγ | LRP${}_{w^{2}}$ | LRPαβ | Gradient | GradCAM | WBP
---|---|---|---|---|---|---|---|---|---
100 | 0.819 | 0.671 | 0.819 | 0.685 | 0.684 | 0.588 | 0.796 | 0.730 | 0.935
500 | 0.772 | 0.639 | 0.772 | 0.632 | 0.644 | 0.542 | 0.744 | 0.731 | 0.890
1000 | 0.740 | 0.622 | 0.740 | 0.607 | 0.624 | 0.517 | 0.706 | 0.723 | 0.855
5000 | 0.624 | 0.575 | 0.624 | 0.549 | 0.579 | 0.449 | 0.568 | 0.637 | 0.720
10000 | 0.555 | 0.542 | 0.555 | 0.519 | 0.554 | 0.418 | 0.489 | 0.553 | 0.642
All points | 0.396 | 0.339 | 0.396 | 0.401 | 0.399 | 0.262 | 0.331 | 0.275 | 0.484

###  D.2 ImageNet Experiments

To verify the effectiveness of WBP to find the relevance pixels on a large
scale natural images, we generate saliency maps using an ImageNet pretrained
VGG11 on ImageNet 2012 Validation set. Training is not part of this
experiment. The saliency mapping methods evaluated with ImageNet are Gradient,
Grad-CAM, LRP, and WBP. More saliency maps on ImageNet dataset are shown in
Figure. 9. In the pointing game experiment, for each saliency method, we
remove the top k saliency pixels and measure the drop in prediction scores are
measured. Intuitively, a larger drop is considered a stronger evidence that
the identified features are causal.

###  D.3 CIFAR-10 Experiments

To verify the effectiveness of PPI on improving the classification
performance, we evaluate PPI with 4 different saliency mapping methods, namely
Gradient, Grad-CAM, LRP, and WBP, during training. For all training including
without PPI, all loss terms are weighted such that they are at a similar
magnitude. The initial learning rate is 0.01 and reduce by $1/10$ every 30
epochs and we train the model for 100 epochs. The backbone is VGG11 with
random initialized weights.

##  Appendix E Geographic Atrophy (GA) Experiments

GA dataset descriptions Our GA dataset is derived from the A2A SD-OCT Study
(http://ClinicalTrials.gov identifier NCT00734487), which was an ancillary
observational prospective study of a subset of eyes from the AREDS2 conducted
at four sites (National Eye Institute, Duke Eye Center, Emory Eye Center, and
Devers Eye Institute) [34]. In this experiment (with our new dataset, that we
will make public), each OCT volume image consists of $100$ scans, each of
which being a $512\times 1000$ pixel image [5]. $1,085$ OCT images are
collected from $275$ subjects during $5$ years. An example of 3D OCT images is
shown in Figure. 10.

Figure 10: Illustration of a 3D OCT image example.

###  E.1 Image differences between $4$ sites

Images in GA dataset are collected from $4$ different sites, hereafter denoted
as A, B, C, and D respectively. There are $315$ images ($101$ positive
samples) from site A, $334$ images ($73$ positive samples) from site B, $260$
images ($131$ positive samples) from site A, and $176$ images ($59$ positive
samples) from site D. We show typical example images from $4$ sites separately
in Figure. 11. As the dataset is collected during $7$ years, some images in
site D are of smaller image size as they are sampled with different type of
machine. We paddle these images by repeating the left and right areas, as show
in the right bottom example.

Figure 11: OCT slice examples from $4$ site.

###  E.2 Multi-view CNN Variation

We use a variant of the multi-view CNN model [56] to process the 3D OCT
inputs, and use it as our baseline solution. The architecture of this model is
outlined in Figure 12. For each slice, the model feed it into a CNN network,
and get the feature $f_{i}$ of slice $i$ ($f_{i}=CNN(x_{i})$), followed by a
fully connected layer and a Sigmoid activation to get a probability score
$p_{i}=sigmoid(FC_{1}(f_{i}))$. We observe that slices in different slices
contributes differently to the identification of GA, which motivates us to
implement a location-aware view pooling, illustrated in the right part of Fig.
12. Each slice is assigned to a position id, ranging from $1$ to $100$. The
model first uses an embedding layer to embed the position id to a six
dimension position feature vector $e_{i}$. Then, we combine the feature vector
$f_{i}$ extracted from the slice image with the corresponding $e_{i}$
together. The combined feature vector is fed into a fully connected layer to
get the logit score $a_{i}$.

$a_{i}=FC_{2}([f_{i},e_{i}])$ (11)

To reduce computational burden during training, we randomly sample $10$ out of
the $100$ slices (with an abuse of notation, denoted by
$a_{1},a_{2},\dots,a_{10}$) and send them into a Softmax function to get the
attention weights for the $10$ sampled slices, using the following equation

$w_{i}=\frac{exp((ReLU(a_{i})+\delta)/\tau)}{\sum_{k=1}^{10}exp((ReLU(a_{i})+\delta)/\tau)}$
(12)

Here $\delta$ is a trainable bias term parameter, initialized to a high value
to stabilize the training, and gradually attenuated to a small number during
training. $\tau$ is the temperature parameter, which is set to a small value
to sharpen the attention weight, which helps us to find out the most important
slices for GA diagnosis. The get final predicted probability of GA (GA score)
for an image $x$ at inference time, we compute the weighted summation of the
probabilities $w_{i}$ of all $100$ slices $GA=\sum_{i}w_{i}p_{i}$.

Figure 12: Illustration of multi-view CNN based 3D OCT image classification
model.

###  E.3 Experiment settings

The CNN network is an Inception_v3, which is pre-trianed on ImageNet. For
training all models, we use the Adam optimizer with a learning rate of
$5\times 10^{-5}$ with a learning rate decay of $0.5$ every $10$ epochs for
the pre-trained CNN network, and the Adam optimizer with a learning rate of
$5\times 10^{-3}$ with a learning rate decay of $0.2$ every $10$ epochs for
the other layers in the model. We use a batch size of $2$ because of the large
size of 3D OCT images and our multi-view CNN model (will be illustrated in
E.2). Random horizontal flips, and Gaussian noise are used for data
augmentations during training.

##  Appendix F LIDC Experiment Details

We also test the proposed method on a public medical CT scan dataset LIDC-IDRI
[3]. We follow the settings in [30, 50] that crops the original images into
$128\times 128$ patches centered on a lesion for which at least one
radiologist has annotated. In our experiment, we focus on the classification
task of predicting the presence of a lesions, which is consistent with the
setup of [50]. There are four radiologists annotates each patch with both
lesion label and lesion mask. A patch in the dataset is labeled as positive if
more than two (i.e. $\geq 3$) radiologists have annotated presence of a
lesion, otherwise negative. The ground-truth mask is the pixel-level union set
of the four masks. We use Inception-v3 [58] as our base model for both
standard classification and PPI-enhanced training with various saliency
mapping schemes. To match the receptive field of an Inception-v3 model, we
resize the input patches to $299\times 299$. For training all models, we use
the Adam optimizer with a learning rate of $10^{-4}$ with a learning rate
decay of $0.3$ every $10$ epochs after epoch $50$, and a batch size of $64$.
Random horizontal flips, vertical flips and rotations within $20$ degrees are
used for data augmentations during training.

##  Appendix G PPI Improving Model Interpretability

Extra comparison examples of saliency maps on GA and LIDC dataset based on
model trained without PPI (odd rows) and model trained with PPI (even rows)
are shown in Figure. 13, and 14. We find that saliency maps of model trained
in PPI learning strategy are clinically relevant by focusing on retinal layers
(GA) or falling on ground-truth segmentations (LIDC).

Figure 13: Comparison examples of saliency maps on GA dataset based on model
trained with and without PPI.  Figure 14: Comparison examples of saliency maps
on LIDC dataset based on model trained with and without PPI.