11institutetext: Department of Artificial Intelligence, University of
Groningen, 9747AG Groningen, The Netherlands 11email:
<EMAIL_ADDRESS>22institutetext: Bonn-Rhein-Sieg University of
Applied Sciences, 53757 Sankt Augustin, Germany
22email<EMAIL_ADDRESS>
# Sanity Checks for Explanation Uncertainty

Matias Valdenegro-Toro 22 0000-0001-5793-9498 Mihir Mulye 11
0009-0001-8043-8752

###### Abstract

Explanations for machine learning models can be hard to interpret or be wrong.
Combining an explanation method with an uncertainty estimation method produces
explanation uncertainty. Evaluating explanation uncertainty is difficult. In
this paper we propose sanity checks for uncertainty explanation methods, where
a weight and data randomization tests are defined for explanations with
uncertainty, allowing for quick tests to combinations of uncertainty and
explanation methods. We experimentally show the validity and effectiveness of
these tests on the CIFAR10 and California Housing datasets, noting that
Ensembles seem to consistently pass both tests with Guided Backpropagation,
Integrated Gradients, and LIME explanations.

###### Keywords:

Explanation Uncertainty Uncertainty Estimation Sanity Checks.

##  1 Introduction

Neural networks are used for many tasks, producing impressive performance, but
still they are black-box models that are very difficult to interpret for
humans. There is increasing interest on obtaining explanations from neural
networks, due to their increasing use and for legal and ethical requirements
[2].

There are many ways to attempt to obtain interpretations for a neural network
prediction, including gradient-based saliency methods [4], interpretable
models, and gray-box models [12]. Evaluating explanations is particularly
difficult since they are ambiguous and there is no ground truth explanation
[13].

An explanation is often assumed to be correct, but as any prediction,
explanations can also be misleading or incorrect, which motivates the need to
quantify explanation uncertainty, as show in Figure 1.

There is previous work about producing explanations with uncertainty, like
Slack et al. [14] that use a Bayesian framework to model explanation
uncertainty in LIME and KernelSHAP, Bykov et al. [5] that combines Bayesian
Neural Networks and Layer-wise Relevance Propagation to produce explanation
uncertainty, and Zhang et al. [18] that find sources of uncertainty in LIME:
sampling randomness, sampling proximity, and explanation variation across data
points.

Uncertainty estimation methods are often approximations, to a Bayesian Neural
Network (BNN) [8], or using non-Bayesian methods like Ensembles [7].
Quantifying the quality of explanation uncertainty is an open problem, as
standard metrics for explanations cannot be directly applied, motivating the
need for alternative testing methods.

In this paper, we explore the idea of extending sanity checks [1] to evaluate
the quality of explanation uncertainty in gradient-based explanation methods.
We combine several uncertainty estimation method (MC-Dropout, MC-DropConnect,
Ensembles, Flipout) with saliency explanations like Guided Backpropagation,
Integrated Gradients, and LIME, and propose sanity checks based on data and
weight randomization, to validate if explanation uncertainty does behave
according to some basic expectations: that when the model is manipulated to
lose information (randomization), this loss of information is reflected in the
explanation uncertainty, by increasing explanation uncertainty as
randomization progresses.

This work aims to increase understanding on how explanations behave and if
explanation uncertainty follows some basic rules, as uncertainty in an
explanation should be somehow proportional to correctness, and there is no
direct way to evaluate explanation correctness. Sanity checks can be used as
basic test when developing new explanation methods with uncertainty
estimation.

The contributions of this work are: we propose weight and data randomization
tests for explanations with uncertainty, setting some basic expectations as
explanation uncertainty should increase as tests progress, and we provide an
initial evaluation of several uncertainty estimation methods and saliency
explanation methods on CIFAR10 and California Housing datasets.

(a) Input
Image

(b) Explanation
Mean

(c) Explanation
Uncertainty

Figure 1: Example explanation with uncertainty on a FER dataset image using
Guided Backpropagation and Ensembles.

##  2 Sanity Checks for Uncertain Explanations

###  2.1 Explanation Uncertainty

This section introduces the basic notation and concept about explanations with
uncertainty.

Given a trained machine learning model $f_{\theta}$ with parameters $\theta$,
we assume that the model has integrated some method for uncertainty
quantification, such as MC-Dropout, Ensembling, Variational Inference, etc.
Quantifying uncertainty usually implies multiple stochastic forward passes
(MC-Dropout, Variational Inference), or multiple models (Ensembles):

$\mu(x)=T^{-1}\sum_{i}f_{\theta}(x)$ (1)
$\sigma^{2}(x)=T^{-1}\sum_{i}(f_{\theta}(x)-\mu(x))^{2}$ (2)

With $T$ the number of stochastic forward passes or ensemble members. To
obtain a gradient-based saliency explanation $\text{expl}(x)$, it is often a
function of model gradients and predictions $\hat{y}=f_{\theta}(x)$:

$\text{expl}(x)=F\left(\hat{y},\frac{\partial f_{\theta}(x)}{\partial
x}\right)$ (3)

To produce explanations with uncertainty, we combine the two previous
formulations, noting that with uncertainty model, the gradient is also
stochastic, and across multiple forward passes or ensemble members, a
explanation mean $\text{expl}_{\mu}(x)$ and standard deviation
$\text{expl}_{\sigma}(x)$ can be computed:

$\text{expl}_{\mu}(x)=T^{-1}\sum_{i}\text{expl}(x)=T^{-1}\sum_{i}F\left(\hat{y},\frac{\partial
f_{\theta}(x)}{\partial x}\right)$ (4)
$\text{expl}_{\sigma}(x)=T^{-1}\sum(\text{expl}(x)-\text{expl}_{\mu}(x))^{2}$
(5)

The standard deviation $\text{expl}_{\sigma}(x)$ can be interpreted as
multiple forward passes will produce different predictions, together with
different explanations. If the explanations are all very different, this will
produce a high standard deviation, and if explanations are similar, then
standard deviation will be low. Then $\text{expl}_{\sigma}(x)$ can be used to
assess if the combination of model, uncertainty quantification method, and
gradient explanation method, produce explanations that are robust to multiple
forward passes (low $\text{expl}_{\sigma}(x)$, or the explanation is low
quality and not clear to the user (high $\text{expl}_{\sigma}(x)$).

A Bayesian interpretation is that a posterior predictive distribution for the
explanation can be computed through a gradient explanation, and in this paper
we use monte carlo approximations.

Explanations computed with the formulation above, combining uncertainty
quantification methods with explanation ones, are denoted as explanation
uncertainty. Explanations without uncertainty quantification can be named as
point-wise explanations, in consistency with the uncertainty quantification
literature.

Note that explanation uncertainty is explicitly about modelling uncertainty in
the explanation, while uncertainty explanation is a different concept, where
the aim is to explain the uncertainty of a prediction.

For visualization purposes, it is possible to integrate the mean and standard
deviation of an explanation into a single explanation, using the coefficient
of variation $\text{CV}(x)$:

$\text{CV}(x)=\frac{\text{expl}_{\sigma}(x)}{\text{expl}_{\mu}(x)}$ (6)

###  2.2 Sanity Checks

This section introduces sanity checks for explanation uncertainty.

Sanity checks for saliency explanations were originally introduced by Adebayo
et al. [1], and they were defined for point-wise explanations. In this section
we define how these sanity checks can be expanded to consider explanation
uncertainty, and how they can help validate it. There are two basic sanity
checks:

Weight Randomization. Model weights are randomized progressively, layer by
layer, starting by the layers closest to the input. The concept is that by
destroying information stored in weights, through randomization, the
explanation should also be close to random, and explanation uncertainty
$\text{expl}_{\sigma}(x)$ should increase, and keep increasing as more layers
are randomized.

Explanation uncertainty $\text{expl}_{\sigma}(x)$ should be a measure of
correctness for a specific explanation, so naturally a correct explanation
should have low uncertainty, and a wrong explanation should have high
uncertainty. By destroying information in the model, the explanation
uncertainty $\text{expl}_{\sigma}(x)$ should reflect this loss of information
in the model, in a way it is a form of epistemic uncertainty.

We propose to evaluate the weight randomization sanity check by training a
model on a given dataset, create explanations with uncertainty on the test
set, and randomize weights progressively, layer by layer, and compare the
explanation uncertainty $\text{expl}_{\sigma}(x)$ on the test set as layers
are randomized, and we expect to see increasing $\text{expl}_{\sigma}(x)$
compared to the explanation uncertainty on the trained model on the test set.

Data Randomization. Training set labels are randomized, and a model is trained
on randomized labels, which should lead the model to overfit and not
generalize to a validation or test set. Explanations with uncertainty are
made, and explanation uncertainty $\text{expl}_{\sigma}(x)$ should be higher
for the model trained on random labels than for the model trained on the real
labels.

The concept of label randomization is to train an improper model, where the
learned relationship between inputs and outputs does not generalize outside
the training set. Explanations in this case are expected to not be
informative, and this should be reflected in the explanation uncertainty
$\text{expl}_{\sigma}(x)$.

Both sanity checks for explanation uncertainty allow for basic evaluation of
explanation uncertainty methods, as there are no other methods to specifically
evaluate explanation uncertainty. These checks test the specific combination
of uncertainty estimation and explanation methods, as both interact, as will
be shown in our results.

Description |  |  Expected Outcome
---|---|---
Weight Randomization Test |
Randomize weights of the model progressively, destroying task information stored on weights |  |  Explanation uncertainty increases as weights are progressively randomized.
Data Randomization Test |
Train a model on randomized labels, which produces a model unable to generalize |  |  Explanation uncertainty is higher compared to a model trained on proper labels.
Table 1: Summary of sanity checks for explanation uncertainty and proposed
expectations when performing these tests. This small guide can be used to
decide if a XAI + UQ method passes or fails a test.

##  3 Experiments

This section describes our evaluation of several combination of saliency
explanation methods and uncertainty quantification methods. We experimentally
assess if these combinations pass each test or not. The aim of our evaluation
is not to provide a definite answer if these methods pass our sanity checks or
not, but to validate that the proposed sanity checks do work and provide a way
to test and evaluate explanation uncertainty methods.

We present results on CIFAR10 Image Classification, and California Housing
price regression. First we describe the saliency methods we evaluate, and then
the uncertainty estimation methods, then present our actual experimental
results, and then close with some discussion and summary.

###  3.1 Saliency Methods

This section describes the saliency methods we used to evaluate explanation
uncertainty.

Guided Backpropagation. Gradient-based saliency methods produce explanations
by computing $\frac{\partial\hat{y}}{\partial x}$, the gradient of network
output $\hat{y}$ (for a particular output neuron) with respect to input $x$,
which measures impact of the network output as the input changes, which is a
form of explanation.

GBP [15] modifies the gradient of the ReLU activation to reduce explanation
noise and enhance human interpretability. The activation $f$ after applying
ReLU activation relu at layer $l$ is:

$f_{i}^{l+1}=\operatorname{\textit{relu}}\left(f_{i}^{l}\right)=\max\left(f_{i}^{l},0\right)$
(7)

Where $i$ is the feature dimension. GBP handles backpropagation through ReLU
non-linearity by combining vanilla backpropagation and DeconvNets as:

$R_{i}^{l}=\left(f_{i}^{l}>0\right)\cdot\left(R_{i}^{l+1}>0\right)\cdot
R_{i}^{l+1}$ (8)

The reconstructed image $R$ at any layer $l$ is generated by the positive
forward pass activations $f_{i}^{l}$ and the positive error signal activations
$R_{i}^{l+1}$. This aids in guiding the gradient by both positive input and
positive error signals. The negative gradient flow is prevented in GBP,
providing more importance to neurons increasing the activation at higher
neurons.

Integrated Gradients. (IG) [16] aims to improve explanations by setting axioms
and expectations, and designing a method that fulfills them. IG performs an
integral along a linear path from a baseline input $x^{\prime}$ to given input
$x$ of the local gradient. IG computation for feature $i$ is:

$\text{IG}_{i}(x,F)=\left(x_{i}-x_{i}^{\prime}\right)\times\int_{\alpha=0}^{1}\frac{\partial
F\left(x^{\prime}+\alpha\times\left(x-x^{\prime}\right)\right)}{\partial
x_{i}}d\alpha$ (9)

$\alpha$ is the interpolation constant for feature perturbation the straight
path between baseline and input. $F(x)$ is the model function mapping features
to predictions. The solution is obtained using numerical approximation because
calculating definite integral for Equation 10 is difficult.

$\text{IG}_{i}(x,F)=\left(x_{i}-x_{i}^{\prime}\right)\times
m^{-1}\sum_{k=1}^{m}\frac{\partial
F\left(x^{\prime}+\frac{k}{m}\times\left(x-x^{\prime}\right)\right)}{\partial
x_{i}}$ (10)

LIME. This method [11] produces global explanations using a local method
(linear regression), by first sampling around the desired input $x$ to be
explained and then fitting a surrogate method that can produce a local
explanation. The core idea of LIME is to locally approximate the decision
boundary of the model and then make a local explanation from this
approximation.

###  3.2 Uncertainty Quantification Methods

This section describes the uncertainty quantification methods we used to
evaluate explanation uncertainty. We first cover three stochastic methods,
where the model becomes stochastic and each forward pass produces a sample of
a predictive distribution, estimating model uncertainty.

Dropout. Monte Carlo Dropout [6] is a well known method for uncertainty
estimation, corresponding to enabling Dropout at inference time, the model
becoming stochastic, and each forward pass producing one sample of the
predictive posterior distribution. We use dropout layers with $p=0.5$.

DropConnect. Monte Carlo DropConnect [9] is similar to MC-Dropout, but weights
are dropped (set to zero) instead of activations, and this behavior is enabled
at inference time, with forward passes also producing a sample of the
predictive posterior distribution.

Flipout. This method is a Bayesian Neural Network [3], approximating weights
$W$ with a Gaussian distribution $q_{\theta}(W)$, using a loss that
approximates the Evidence Lower Bound (ELBO):

$L(\theta)=\text{KL}(\mathbb{P}(W),q_{\theta}(W))-\mathbb{E}_{w\sim
q_{\theta}(W)}[\log\mathbb{P}(y\,|\,x,w)]$ (11)

These loss terms are the KL-divergence between the prior $\mathbb{P}(W)$ and
the approximate weight posterior distribution $q_{\theta}(W)$, plus the
negative log-likelihood $-\log\mathbb{P}(y\,|\,x,w)$ computed as an
expectation by sampling weights from the approximate weight posterior $w\sim
q_{\theta}(W)$. The negative log-likelihood is the loss function for the task,
mean squared error for regression, and cross-entropy for classification.

In particular Flipout [17] stabilizes training by allowing different sampled
weights to be used in a batch of predictions, speeding up training and
improving predictive performance.

We also use one deterministic method, ensembling:

Ensembles. Ensembling [7] trains $T$ copies of the same model architecture, on
the same dataset, and then combines their predictions as shown below.
Differences between models come from random weight initialization. Ensembles
are often one of the best performing uncertainty estimation methods, and are
very easy to implement.

As all uncertainty quantification methods we use produce samples or use
multiple models, those samples/predictions have to be combined to estimate
model uncertainty. For classification, the predicted probability vectors are
averaged, and for regression, the mean $\mu(x)$ and variance $\sigma^{2}(x)$
across samples/predictions is computed:

$\mu(x)=T^{-1}\sum f(x)\qquad\qquad\sigma^{2}(x)=T^{-1}\sum(f(x)-\mu(x))^{2}$
(12)

Where $T$ is the number of samples/models. The standard deviation $\sigma(x)$
is a measure of uncertainty in this case.

###  3.3 CIFAR10 Classification

In this section we perform experiments using the well known CIFAR 10 dataset.
We train a small CNN on this dataset using MC-Dropout for uncertainty
estimation, and perform sanity checks on Guided Backpropagation (GBP) and
Integrated Gradients (IG).

As visually validating image explanations is not easy, in order to compare
explanations produced by a modified model (data and weight randomization
tests) with the explanations made by the original (unrandomized) model, we use
the Structural Similarity Index (SSIM).

Weight randomization test. Our main results using SSIM are shown in Figure 2,
where we compare the SSIM for $\text{expl}_{\mu}(x)$ and
$\text{expl}_{\sigma}(x)$ separately. The mean behaves as expected, decreasing
as more weights are randomized in both explanation methods, but GBP decreases
to a minimum SSIM of around 0.1. The standard deviation
$\text{expl}_{\sigma}(x)$ also decreases as expected in both explanation
methods, but does not decrease at the same level as the mean, and its not
close to being a minimum. We believe these results show that Dropout + GBP and
Dropout + IG both pass the weight randomization test.

Figures 3 and 4 show an individual sample and how their explanation mean,
standard deviation, and coefficient of variation, vary with weight
randomization. These particular examples show how these method pass the weight
randomization test, as the mean and standard deviation explanations become
more noisy with more layers in the network being randomized (as expected),
with varying level of noise. It is clear how GBP explanations (Figure 4)
become more noisy with more random weights, while IG (Figure 3) produces an
almost blank image, which explains the behavior presented in the SSIM results.
Additionally these results show how differently GBP and IG work, producing
very different behavior on the explanation and its uncertainty
$\text{expl}_{\sigma}(x)$.

(a) SSIM for the mean $\text{expl}_{\mu}(x)$

(b) SSIM for the std $\text{expl}_{\sigma}(x)$

Figure 2: SSIM values for the weight randomization test on CIFAR 10 using
Dropout. These values are computed between the explanation representation
generated with no weight randomization and with incremental weight
randomization. As the amount of weight randomization increases, the similarity
between explanation representations decreases.

Input |
---|---

|  0% |  20% |  40% |  60% |  80% |  100%
---|---|---|---|---|---|---

Mean
---
Std
Coef. Var.

Figure 3: Visualization of weight randomization effect on the explanations
obtained using Dropout and Integrated Gradients on a CIFAR10 sample. Note how
the both mean and standard deviation explanation become almost blank with
increasing random weights.

Input |
---|---

|  0% |  20% |  40% |  60% |  80% |  100%
---|---|---|---|---|---|---

Mean
---
Std
Coef. Var.

Figure 4: Visualization of weight randomization effect on the explanations
obtained using Dropout and Guided Backpropagation on a CIFAR10 sample. Note
how the mean and standard deviation explanation become noisy with increasing
random weights.

Data randomization test. Main results for this test are shown in Figure 2,
where we present SSIM values for $\text{expl}_{\mu}(x)$ and
$\text{expl}_{\sigma}(x)$ separately. These values should be compared to a
theoretical SSIM maximum value of 1.0 when no randomization is applied, and
when randomizing labels, we obtain similar results compared to the weight
randomization tests, where GBP decreases SSIM considerably more than IG, but
overall both methods pass the data randomization test.

Figure 5 shows one individual sample and the predicted explanations and their
uncertainties for each stage of the data randomization test (model trained on
true labels vs random labels). It is clear that both examples pass the data
randomization test, as in both cases training on random labels produces a more
noisy mean and standard deviation explanation, which is reflected into a noisy
coefficient of variation. But it can be visually seen that GBP produces higher
$\text{expl}_{\sigma}(x)$ values, indicating its further decrease of SSIM over
the whole dataset.

Explanation Method |  Mean |  Standard Deviation
---|---|---
|  $\text{expl}_{\mu}(x)$ |  $\text{expl}_{\sigma}(x)$
IG |  0.401 |  0.894
GBP |  0.252 |  0.510
Table 2: SSIM values for the data randomization on CIFAR10 using MC-Dropout.
These values are computed between the explanation representation with no data
randomization and with data randomization. These SSIM values should be
compared to a theoretical maximum of 1.0 when no data randomization is
performed. Note how GBP has a larger drop compared to IG.

(a) Input Image

|  True Labels |  Random Labels
---|---|---

Mean
---
Std
Coef. Var.

(b) Integrated Gradients

|  True Labels |  Random Labels
---|---|---

(c) Guided Backpropagation

Figure 5: Visualization of data randomization effect on explanations obtained
using Dropout and GBP and IG. Note how the explanation for the model with
random labels is considerably more noisy than explanation from a model with
true labels.

###  3.4 California Housing Regression

In this section we evaluate a tabular regression dataset: the california
housing dataset [10]. We train a multi-layer perceptron with four fully
connected layers, 8-8-8-1 neurons, and ReLU activation. Uncertainty estimation
methods are applied to the last layer only, when applicable.

As tabular features are not directly humanly interpretable as images are, we
compare the explanation uncertainty $\text{expl}_{\sigma}(x)$ directly for
each test, using bar plots.

Weight randomization test. Figure 6 presents the weight randomization results.
Overall only ensembles consistently passes this test by producing higher
explanation uncertainty for increasing randomized layer weights for both GBP
and LIME. In the case of LIME, ensembles produced a constant higher
explanation uncertainty, and we consider this to be passing the test.

Other uncertainty estimation methods produce inconsistent results with both
GBP and LIME, like Flipout consistently producing lower explanation
uncertainty as layers are randomized, which is counterintuitive to proposed
interpretation of the weight randomization test.

Data randomization test. Figure 7 presents data randomization results. In this
case we only compare the explanation uncertainty for a model trained with the
true labels in the training set, versus another model trained with permuted
labels, breaking the relationship between inputs and labels, and producing an
overfitted model that does not generalize to the validation/test set.

Results are overall similar than with the weight randomization test, with
ensembles clearly producing higher explanation uncertainty for random labels
for both GBP and LIME, while other methods like Flipout and Dropout only
sometimes produce this behavior, depending on the explanation method being
used.

| Guided Backprop | LIME
---|---|---
Dropout |  |
DropConnect |  |
Flipout |  |
Ensembles |  |
Figure 6: Weight randomization test comparison across two explanation methods and four uncertainty methods on the California Housing Dataset (Tabular Regression). Dropout + GBP and Ensembles with GBP and LIME pass the test, while other combinations fail the test. | Guided Backprop | LIME
---|---|---
Dropout |  |
DropConnect |  |
Flipout |  |
Ensembles |  |
Figure 7: Data randomization test comparison across two explanation methods
and four uncertainty methods on the California Housing Dataset (Tabular
Regression). Most combinations fail this test, with only Dropout + LIME,
Flipout + LIME, and Ensembles with GBP and LIME passing the test.

###  3.5 Discussion

Table 3 provides as summary of all our results over multiple datasets,
explanation methods, and uncertainty quantification methods. From our combined
evaluation, it is clear that both MC-Dropout and Ensembles are the best
uncertainty methods across all combinations of explanation method (GBP, IG,
LIME), while other uncertainty estimation methods do not always pass both
kinds of tests.

We believe that our results show that explanation methods and uncertainty
quantification methods do have interacting effects, due to some UQ methods
like ensembles producing consistent passing of both weight and data
randomization tests, while other combinations do not consistently pass both
tests, and only pass them in specific combinations with a explanation method.

Overall we believe that our evaluation clearly shows the usefulness of sanity
checks for explanation uncertainty, which adds a useful tool to the
explainable AI researchers’ toolbox, in particular for explanation methods
that can produce explanation uncertainty.

|  | GBP |  | IG |  | LIME
---|---|---|---|---|---|---
Dataset | UQ Method | Weight | Data |  | Weight | Data |  | Weight | Data
CIFAR10 | Dropout | ✓ | ✓ |  | ✓ | ✓ |  |  |
Calif Housing | Dropout | ✓ | ✗ |  |  |  |  | ✗ | ✓
| DropConnect | ✗ | ✗ |  |  |  |  | ✗ | ✗
| Flipout | ✗ | ✗ |  |  |  |  | ✗ | ✓
| Ensembles | ✓ | ✓ |  |  |  |  | ✓ | ✓
Table 3: Summary of sanity checks for uncertainty explanations. A blank cell
indicates that this combination of dataset/uncertainty method/explanation
method was not tested.

##  4 Conclusions and Future Work

This work extends the common sanity checks for saliency explanations into
explanation uncertainty methods, where a uncertainty estimation method is
combined with an explanation method, to produce explanations with uncertainty,
with aims to evaluate correctness of each explanations.

Our proposed weight and data randomization tests come with specific
expectations, that increasing randomization in a model should be reflected on
the explanation uncertainty $\text{expl}_{\sigma}(x)$.

We evaluate and showcase our tests on CIFAR10 and California Housing datasets,
with multiple uncertainty estimation and explanation methods. Overall we find
that Dropout and Ensembles with Guided Backpropagation, Integrated Gradients,
or LIME, always pass our tests, while other combinations fail these tests.

We expect that our proposed tests for explanation uncertainty can be used to
validate new developments in uncertainty estimation and saliency explaination
methods.

Our work is limited by the selection of uncertainty estimation and saliency
explanation methods, and by the selection of datasets used for evaluation.
Additionally there could be other ways to compare explanation uncertainty in
each test (we used SSIM and direct $\text{expl}_{\sigma}(x)$ comparisons),
which needs to be adapted to specific use cases and input modalities.

## Broader Impact Statement

Explanations can be legally and ethically required, so users expect them to be
correct or at least informative, but that not guaranteed with current
explainable AI methods. We believe explanation uncertainty can have a positive
societal impact, by quantifying uncertainty in explanations, and our proposed
method can help evaluate and set basic expectations on explanations with
uncertainty.

Uncertainty estimation is only approximate and can also produce misleading
predictions, and when combined with explanation methods, it can produce
misleading explanation uncertainty. There are no guarantees about correctness
for both prediction, uncertainty, and explanation.

## References

  * [1] Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. Advances in neural information processing systems 31 (2018)
  * [2] Atkinson, K., Bench-Capon, T., Bollegala, D.: Explanation in ai and law: Past, present and future. Artificial Intelligence 289, 103387 (2020)
  * [3] Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: International conference on machine learning. pp. 1613–1622. PMLR (2015)
  * [4] Borys, K., Schmitt, Y.A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C.M., Nensa, F.: Explainable ai in medical imaging: An overview for clinical practitioners–saliency-based xai approaches. European journal of radiology p. 110787 (2023)
  * [5] Bykov, K., Höhne, M.M.C., Creosteanu, A., Müller, K.R., Klauschen, F., Nakajima, S., Kloft, M.: Explaining bayesian neural networks. arXiv preprint arXiv:2108.10346 (2021)
  * [6] Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: international conference on machine learning. pp. 1050–1059. PMLR (2016)
  * [7] Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv:1612.01474 (2016)
  * [8] Lampinen, J., Vehtari, A.: Bayesian approach for neural networks—review and case studies. Neural networks 14(3), 257–274 (2001)
  * [9] Mobiny, A., Yuan, P., Moulik, S.K., Garg, N., Wu, C.C., Van Nguyen, H.: Dropconnect is effective in modeling uncertainty of bayesian deep networks. Scientific reports 11(1), 1–14 (2021)
  * [10] Pace, R.K., Barry, R.: Sparse spatial autoregressions. Statistics & Probability Letters 33(3), 291–297 (1997)
  * [11] Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you? explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. pp. 1135–1144 (2016)
  * [12] Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1(5), 206–215 (2019)
  * [13] Samek, W., Binder, A., Montavon, G., Lapuschkin, S., Müller, K.R.: Evaluating the visualization of what a deep neural network has learned. IEEE transactions on neural networks and learning systems 28(11), 2660–2673 (2016)
  * [14] Slack, D., Hilgard, A., Singh, S., Lakkaraju, H.: Reliable post hoc explanations: Modeling uncertainty in explainability. Advances in neural information processing systems 34, 9391–9404 (2021)
  * [15] Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 (2014)
  * [16] Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International conference on machine learning. pp. 3319–3328. PMLR (2017)
  * [17] Wen, Y., Vicol, P., Ba, J., Tran, D., Grosse, R.: Flipout: Efficient pseudo-independent weight perturbations on mini-batches. arXiv preprint arXiv:1803.04386 (2018)
  * [18] Zhang, Y., Song, K., Sun, Y., Tan, S., Udell, M.: ” why should you trust my explanation?” understanding uncertainty in lime explanations. arXiv preprint arXiv:1904.12991 (2019)


# AI for Anticipatory Action: Moving Beyond Climate Forecasting

Benjamin Q. Huynh
Department of Environmental Health & Engineering
Johns Hopkins University
<EMAIL_ADDRESS>
&Mathew V. Kiang
Department of Epidemiology & Population Health
Stanford University
<EMAIL_ADDRESS>
###### Abstract

> Disaster response agencies have been shifting from a paradigm of climate
> forecasting towards one of anticipatory action: assessing not just what the
> climate will be, but how it will impact specific populations, thereby
> enabling proactive response and resource allocation. Machine learning (ML)
> models are becoming exceptionally powerful at climate forecasting, but
> methodological gaps remain in terms of facilitating anticipatory action.
> Here we provide an overview of anticipatory action, review relevant
> applications of machine learning, identify common challenges, and highlight
> areas where ML can uniquely contribute to advancing disaster response for
> populations most vulnerable to climate change.

## Background

Anticipatory action – proactive measures taken in response to forecasted risks
and disasters before they occur – has emerged as a crucial framework for
disaster response (Figure 1) (?; ?). Central to anticipatory action is impact-
based forecasting, an approach that focuses on predicting the specific impacts
of an impending disaster, such as health outcomes, disruption of services, or
economic loss (?; ?; ?; ?; ?). By contrast, climate forecasting typically
reports environmental conditions, such as wind speeds or rainfall. Impact-
based forecasting is typically more accessible to humanitarian practitioners,
but harder to predict; climate forecasting is generally more accurate, but may
not adequately communicate potential hazards. However, the two approaches are
not mutually exclusive – in fact, we examine here areas in which machine
learning can uniquely be applied to leverage the predictive power of climate
forecasting as well as the applied perspective of impact-based forecasting.

Despite the promise of increased resilience from anticipatory action, major
gaps remain in terms of developing robust, trustworthy systems. Forecast-based
financing, a system where financing is allocated proactively based on risk
estimates for models, has become increasingly used within United Nations
agencies and non-governmental organizations. However, because regions that
receive higher-risk forecasts receive more aid, poor predictive performance
and causal validity of such forecasts may lead to misallocation of resources
(?). The gap between the promise and current reality of anticipatory action
may be in part aided by advances in AI research; we examine here potential
areas of opportunity where AI can be used to improve anticipatory action.

Pre-crisis: Disaster risk reduction Post-forecast, pre-onset: Anticipatory
action Post-onset: Response, recovery, reconstruction _Early warning_
_Disaster onset_ Figure 1: Typical timeline of disaster preparedness,
anticipatory action, and response. _Disaster risk reduction_ refers to
strengthening resilience before crises occur, and _anticipatory action_ refers
to proactive response and resource allocation after an early warning system
flags an incoming disaster.

## Challenges and Opportunities

Figure 2 shows a typical undertaking of the forecasting portion of
anticipatory action. A model is developed based on spatiotemporal
environmental data and used to make population-specific impact forecasts. From
there, the forecasts are used to enable proactive resource allocation. We
broadly identify two strategies to forecast disaster impacts: (1) a
correlational forecasting approach where the impacts of the disaster are
directly modeled; and (2) a causal approach wherein the disaster is
meteorologically modeled, then outputs are run through an exposure-response
function to ascertain the impact of the disaster. A subsequent step is taken
after either approach to apply the model to a target population for a
population-specific forecast. Table 1 shows examples of how impact-based
forecasts can be used.

Raw data Modeling strategy? Impact forecast Climate forecast Exposure-response
function Population-specific impact forecast
CorrelationalCausal$P=g(C|C=c)$$Y=\phi(P,V|P=p,V=v)$$Y=f(C,V|C=c,V=v)$ Figure
2: Typical workflow for the forecasting portion of anticipatory action. After
population-specific impact forecasts are provided, targeted resource
allocation can be conducted.

### Problem Formulation

Formally, let $C=c_{1},c_{2},...c_{k}$ be a set of climate-related features
and let $V=v_{1},v_{2},...v_{k}$ be a set of population-specific features. The
correlational forecasting approach to directly model disaster impacts is
represented by

$Y_{s,t+n}=f(C,V|C=c_{s,t},V=v_{s,t})$

where $Y$ is the outcome of interest, $s$ and $t$ are spatiotemporal indices,
and $n$ is the forecasting lead time. Machine learning approaches have been
used for impact-based forecasting for outcomes related to disasters, such as
modeling displacement, food insecurity, disease outbreaks, economic loss, or
infrastructure assessment (?; ?; ?; ?; ?; ?). Most ML approaches for impact-
based forecasting directly model the impact without using climate forecasting
as an intermediary; to the extent climate variables are used, they are
typically just more predictors for the model.

In terms of climate forecasting, such models can be represented as

$P_{s,t+n}=g(C|C=c_{s,t})$

where $P$ is a modeled climate event, such as a risk score or climate
condition, and $g$ is the climate forecasting function. Machine learning
models for climate forecasting, while historically inferior to global
simulation methods like numerical weather prediction, have since reached
comparable or even superior performance to simulation-based methods for a
variety of forecasting applications, ranging from tropical cyclones, heat
waves, rainfall, floods, wildfires, and more (?; ?; ?; ?; ?; ?; ?). ML models
are far less computationally expensive than global simulation models, and can
be used to supplement them by emulating their predictions. Because such models
are emulating mechanistic physical processes, predictive performance is often
very high compared to other forecasting domains. Many ML models frame their
outputs as risk scores, providing spatiotemporal risk estimates for a given
type of natural disaster.

Causally forecasting disaster impacts is represented by

$Y_{s,t+n}=f(C,V|do(C=c_{s,t}),V=v_{s,t})$

where the $do$ operator refers to causally setting the climate variables $C$
to be specific values, rather than using a correlational relationship (?). In
practice, causal forecasting is facilitated by an _exposure-response function_
:

$Y=\phi(P,V|P=p,V=v)$

where $\phi$ represents the relationship between the climate event and the
impact of interest. Exposure-response functions are typically used in
environmental epidemiology, where the relationship between an exposure and a
health outcome is identified, then that relationship is used to simulate
health outcomes based on known exposure levels (?). Depending on how they are
identified, exposure-response functions can vary in levels of causal validity,
ranging from results from randomized controlled trials to associations
identified from observational data using causal inference. In other fields,
this relationship may also be known as a _damage function_ (?).

Table 1: Example scenarios of disasters and their potential forecasts. Hazard |  Climate Forecast |  Exposure-response function(s) |  Impact-based Forecast
---|---|---|---
Tropical cyclone |  A category 4 tropical cyclone with windspeed 220km/h is anticipated to make landfall in 36-48 hours. |  Relationship between tropical cyclone and building damage/displacement |  40-60% of housing in Region A will be damaged, with an anticipated 20,000-40,000 displaced individuals.
Wildfire smoke |  Wildfire smoke will reach City B tomorrow morning and cover the city for 48-72 hours. |  Relationship between smoke exposure and hospitalizations; relationship between excess hospitalizations and drug shortages |  200-400 excess hospitalizations are expected, and extra inhalers will be necessary to prevent shortages.
Drought |  Precipitation far below average is likely for the summer in Province C. |  Relationship between precipitation and crop failure/food insecurity, relationship between food insecurity and malnourishment |  Food insecurity will increase, and prices for affected crops will spike by 55-75%. Incidence of child malnourishment will increase by 20-30%.

### Open Areas for Improvement

The decision to use a causal or correlational modeling strategy is context-
specific, and both approaches have their own advantages and disadvantages. For
causal forecasting, exposure-response functions can be difficult to identify,
depending on the exposure. Correlational forecasting is subject to
confounders, threatening causal validity (?). Both approaches face challenges
of dataset shift and algorithmic fairness. Forecasting for anticipatory action
thus remains an open problem with multiple potential areas for improvement,
and we discuss here how AI can be used to address such issues.

#### Exposure-response functions

Exposure-response functions are difficult to obtain, and especially so for
disaster-related impacts. Disasters vary in magnitude, location, and
population characteristics, with relatively infrequent occurrence – estimating
a valid association between a disaster and an impact typically requires high-
resolution data and a plausible empirical strategy (?). Even air pollution,
which is continuously monitored with high-resolution exposure and health
outcomes data, drives contention over what the exposure-response function
should be (?). Machine learning can be useful in improving estimation of
exposure-response relationships by producing better characterizations of
disaster impacts. For example, satellite imagery in conjunction with AI has
been used to assess building damage from disasters (?; ?; ?); such information
could be used to develop exposure-response relationships between disaster
occurrence and building damage, potentially enabling forecasts of building
damage.

As another example, AI can be used to improve estimates of health outcomes
such as mortality or disease prevalence after a disaster. In low-resource
environments, post-disaster estimates of mortality or disease prevalence are
conducted by taking surveys across clusters of households. Results from such
surveys can vary drastically due to sampling bias, which can be difficult to
overcome through randomization due to poor access to households post-disaster.
The survey approach remains the gold-standard, as official national or
international statistics are generally too slow to capture the magnitude of a
disaster shortly after it has occurred (?; ?). AI can be used in conjunction
with post-stratification, a technique typically used in the social sciences to
learn from biased samples, to estimate adjusted mortality or disease
prevalence rates (?; ?; ?).

Additionally, mortality or disease prevalence surveys typically obtain
population-wide estimates by extrapolating from survey results:

$\frac{E_{s}}{N_{s}}\approx\frac{E_{f}}{N_{f}}$

where $E$ and $N$ represent events and population size respectively, and $s$
and $f$ indices indicate observations from the survey data and the full
population at large, respectively. Typically, $E_{f}$ is estimated by assuming
a value for $N_{f}$ (as well as adjusting for any covariates), but there is
often uncertainty regarding the value of $N_{f}$ as populations tend to
migrate in response to disasters, potentially affecting estimates of $E_{f}$.
Any AI methods to estimate $N_{f}$, whether through migration or population
modeling, can thus be valuable in improving identification of exposure-
response functions (?; ?; ?).

#### Dataset Shift

All forecasting approaches are subject to dataset shift, where there is a
difference in distribution between training data and deployment-time testing
data. This is particularly true for disaster-related forecasts: first,
unprecedented events happen regularly under climate change, and models may not
provide valid results for environmental conditions that have never been seen
before (?); secondly, because disasters happen relatively infrequently, there
will not be sufficient historical data regarding how a disaster of any given
magnitude affects any given region or subpopulation of a country. Taken
together, high-resolution country-wide forecasts will require targeted
population-specific forecasts in a manner that addresses dataset shift.

One approach to handling dataset shift is leveraging _transportability_ of
causal effects (?; ?). Transportability provides a framework to assess whether
causal effects can be applied from one environment to another. As an example,
suppose we have a plausible causal effect estimate saying that wildfire smoke
caused an $X\%$ increase in hospitalizations per $\mathrm{\SIUnitSymbolMicro
g}$/${\mathrm{m}}^{3}$ of fine particulate matter in City A. We want to know
how many additional hospitalizations we’d expect to see if smoke reached City
B, but City B has a different age distribution than City A, and age is a
confounder in that it affects both exposure levels to smoke and likelihood of
hospitalization. Transportability theory determines that if we can identify
both the full causal effect:

$P(hospitalization|do(smoke))$

and the age-specific causal effect:

$P(hospitalization|do(smoke),age)$

then it is feasible to transport causal estimates from City A to City B.
Transportability has been applied in various complex scenarios in health (?;
?), and could also be useful for improving causal validity of disaster
forecasts.

Relatedly, another approach to handle dataset shift for impact forecasting
involves _invariant prediction_ (?; ?; ?). Briefly, approaches leveraging
invariant prediction allow for selection of causal features from a model by
identifying model configurations with high prediction invariance, enabling
more robust predictions less likely to be influenced by spurious correlations.
Using approaches based on invariant prediction would improve the causal
validity of correlational impact forecasting without requiring the
intermediary step of calculating an exposure-response function, thereby
improving the reliability of impact predictions in relationship to varying
climate variables. Although these approaches require the strong assumption of
there being no unobserved confounders present, sensitivity analyses for
potentially unobserved confounders could be used to provide uncertainty
intervals for predictions (?; ?; ?). Applications of invariant causal
prediction have recently been gaining traction within medicine (?; ?; ?) and
earth sciences (?; ?), suggesting that applications to climate disasters and
anticipatory action may be feasible.

#### Data Scarcity and Algorithmic Bias

Climate data are inequitably collected, with marginalized populations
typically having poorer and lower-quality data (?). 60% of countries lack
basic water information services that can be used for flood preparedness, and
roughly half of all countries do not have multi-hazard early warning systems
(?). The World Meteorological Organization roughly estimates that countries
without such early warning systems have over eight times the disaster-related
mortality rates than countries with early warning systems (?). Data scarcity
degrades the performance of predictive models for regions that are most
vulnerable to disasters in the first place, making anticipatory action more
difficult within low-resource countries. Typical numerical weather prediction
models rely on data collected globally to make better localized predictions,
but AI efforts to ameliorate data scarcity such as localized predictions via
transfer learning, or up/downscaling climate predictions may help circumvent
such requirements and improve impact-based forecasts (?; ?; ?).

Careful consideration needs to be given to the ethical implications of AI
models for impact forecasting, particularly as forecast-based financing
becomes increasingly used (?). In particular, practitioners should exercise
caution as to how the choice of metric affects equity considerations. Treating
economic loss as an absolute metric for impact assessment favors wealthier
regions; considering risk and loss as relative has been shown to provide more
equitable results in risk assessment (?). Additionally, considering how
disasters impact regions beyond economic loss, and instead using metrics like
health or wellness, may provide a more nuanced characterization of disaster
impacts (?). Algorithmic audits and assessments commonly used for composite
indicators or risk scoring machine learning models should be used to assess
the algorithmic fairness of impact-based forecasts (?; ?; ?; ?; ?).

## Discussion

In this paper, we surveyed how AI advances can be used to improve anticipatory
action in terms of current methodological gaps and potential areas for
improvement. There are several limitations to our paper. First, this is not an
exhaustive review of all relevant literature: we refer to other review papers
where applicable, and subfields such as AI for climate forecasting or causal
prediction are sufficiently deep and nuanced that reviewing all relevant works
is out of scope. Secondly, there are other crucial areas of improvement that
are not mentioned here, such as model explainability or uncertainty
quantification – our subjective judgment is that they represent long-standing
challenges of fundamental importance that are not sufficiently specific to
anticipatory action to be in scope of this survey.

Amidst intensifying climate change and increased frequency of natural
disasters, the need for improved anticipatory action will only rise. Some of
the AI-related challenges and opportunities we raise may appear too
intractable or open-ended. We argue that since anticipatory action is already
being widely used and deployed, any improvements to existing systems, however
marginal, could be consequential and life-saving. Such a perspective could be
used to foster a long-term vision for a future where we have better data and
tools, enabling protection of the most vulnerable from disasters before they
occur.

## References

  * [Arjovsky et al. 2019] Arjovsky, M.; Bottou, L.; Gulrajani, I.; and Lopez-Paz, D.  2019\.  Invariant risk minimization.  arXiv preprint arXiv:1907.02893.
  * [Aunan and Pan 2004] Aunan, K., and Pan, X.-C.  2004\.  Exposure-response functions for health effects of ambient air pollution applicable for china–a meta-analysis.  Science of the total environment 329(1-3):3–16.
  * [Bareinboim and Pearl 2012] Bareinboim, E., and Pearl, J.  2012\.  Transportability of causal effects: Completeness results.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, 698–704.
  * [Bhagwat, Ferryman, and Gibbons 2023] Bhagwat, A. M.; Ferryman, K. S.; and Gibbons, J. B.  2023\.  Mitigating algorithmic bias in opioid risk-score modeling to ensure equitable access to pain relief.  Nature medicine 29(4):769–770.
  * [Bi et al. 2023] Bi, K.; Xie, L.; Zhang, H.; Chen, X.; Gu, X.; and Tian, Q.  2023\.  Accurate medium-range global weather forecasting with 3D neural networks.  Nature 619(7970):533–538.  Number: 7970 Publisher: Nature Publishing Group.
  * [Broniecki, Leemann, and Wüest 2020] Broniecki, P.; Leemann, L.; and Wüest, R.  2020\.  Improved multilevel regression with post-stratification through machine learning.
  * [Chen, Zhang, and Wang 2020] Chen, R.; Zhang, W.; and Wang, X.  2020\.  Machine learning in tropical cyclone forecast modeling: A review.  Atmosphere 11(7):676.
  * [Cifuentes et al. 2020] Cifuentes, J.; Marulanda, G.; Bello, A.; and Reneses, J.  2020\.  Air temperature forecasting using machine learning techniques: a review.  Energies 13(16):4215.
  * [Coughlan de Perez et al. 2015] Coughlan de Perez, E.; van den Hurk, B.; Van Aalst, M.; Jongman, B.; Klose, T.; and Suarez, P.  2015\.  Forecast-based financing: an approach for catalyzing humanitarian action based on extreme weather and climate forecasts.  Natural Hazards and Earth System Sciences 15(4):895–904.
  * [Cullmann et al. 2021] Cullmann, J.; Dilley, M.; Egerton, P.; Grasso, V. F.; Honoré, C.; Lúcio, F.; Luterbacher, J.; Nullis, C.; Power, M.; Rea, A.; Repnik, M.; and Stander, J.  2021\.  The state of climate services 2021: Water.
  * [Dickerman and Hernán 2020] Dickerman, B. A., and Hernán, M. A.  2020\.  Counterfactual prediction is not only for causal inference.  European journal of epidemiology 35:615–617.
  * [Doocy, Cherewick, and Kirsch 2013] Doocy, S.; Cherewick, M.; and Kirsch, T.  2013\.  Mortality following the haitian earthquake of 2010: a stratified cluster survey.  Population health metrics 11(1):1–9.
  * [Foini et al. 2023] Foini, P.; Tizzoni, M.; Martini, G.; Paolotti, D.; and Omodei, E.  2023\.  On the forecastability of food insecurity.  Scientific Reports 13(1):2793.
  * [Gang, O’Keeffe, and Roberts 2023] Gang, K. B. A.; O’Keeffe, J.; and Roberts, L.  2023\.  Cross-sectional survey in central african republic finds mortality 4-times higher than un statistics: how can we not know the central african republic is in such an acute humanitarian crisis?  Conflict and Health 17(1):21.
  * [Gelman et al. 2016] Gelman, A.; Goel, S.; Rivers, D.; Rothschild, D.; et al.  2016\.  The mythical swing voter.  Quarterly Journal of Political Science 11(1):103–130.
  * [Gerl et al. 2016] Gerl, T.; Kreibich, H.; Franco, G.; Marechal, D.; and Schröter, K.  2016\.  A review of flood loss models as basis for harmonization and benchmarking.  PloS one 11(7):e0159791.
  * [Hino and Nance 2021] Hino, M., and Nance, E.  2021\.  Five ways to ensure flood-risk research helps the most vulnerable.  Nature 595(7865):27–29.
  * [Honore, Kumar, and Speck 2022] Honore, C.; Kumar, A.; and Speck, R. M.  2022\.  Global status of multi-hazard early warning systems: Target g.
  * [Huynh and Basu 2019] Huynh, B. Q., and Basu, S.  2019\.  Forecasting Internally Displaced Population Migration Patterns in Syria and Yemen.  Disaster Medicine and Public Health Preparedness 1–6.  Publisher: Cambridge University Press.
  * [Huynh et al. 2021] Huynh, B. Q.; Kwong, L. H.; Kiang, M. V.; Chin, E. T.; Mohareb, A. M.; Jumaan, A. O.; Basu, S.; Geldsetzer, P.; Karaki, F. M.; and Rehkopf, D. H.  2021\.  Public health impacts of an imminent red sea oil spill.  Nature sustainability 4(12):1084–1091.
  * [Huynh et al. 2023] Huynh, B. Q.; Chin, E. T.; Koenecke, A.; Ouyang, D.; Ho, D. E.; Kiang, M. V.; and Rehkopf, D. H.  2023\.  Potential for allocative harm in an environmental justice data tool.  arXiv preprint arXiv:2304.05603.
  * [Jain et al. 2020] Jain, P.; Coogan, S. C.; Subramanian, S. G.; Crowley, M.; Taylor, S.; and Flannigan, M. D.  2020\.  A review of machine learning applications in wildfire science and management.  Environmental Reviews 28(4):478–505.
  * [Jung et al. 2017] Jung, J.; Concannon, C.; Shroff, R.; Goel, S.; and Goldstein, D. G.  2017\.  Simple rules for complex decisions.  arXiv preprint arXiv:1702.04690.
  * [Kind, Botzen, and Aerts 2020] Kind, J.; Botzen, W. W.; and Aerts, J. C.  2020\.  Social vulnerability in cost-benefit analysis for flood risk management.  Environment and Development Economics 25(2):115–134.
  * [Kishore et al. 2018] Kishore, N.; Marqués, D.; Mahmud, A.; Kiang, M. V.; Rodriguez, I.; Fuller, A.; Ebner, P.; Sorensen, C.; Racy, F.; Lemery, J.; et al.  2018\.  Mortality in puerto rico after hurricane maria.  New England journal of medicine 379(2):162–170.
  * [Li et al. 2021] Li, Q.; Wang, Z.; Shangguan, W.; Li, L.; Yao, Y.; and Yu, F.  2021\.  Improved daily smap satellite soil moisture prediction over china using deep learning model with transfer learning.  Journal of Hydrology 600:126698.
  * [Liu 2012] Liu, J.  2012\.  Progresses and challenges on qpe/qpf utilization in hydrology.  Tropical Cyclone Research and Review 1(2):194–197.
  * [Mayson 2019] Mayson, S. G.  2019\.  Bias in, bias out.  The Yale Law Journal 2218–2300.
  * [Merz et al. 2020] Merz, B.; Kuhlicke, C.; Kunz, M.; Pittore, M.; Babeyko, A.; Bresch, D. N.; Domeisen, D. I.; Feser, F.; Koszalka, I.; Kreibich, H.; et al.  2020\.  Impact forecasting to support emergency management of natural hazards.  Reviews of Geophysics 58(4):e2020RG000704.
  * [Mosavi, Ozturk, and Chau 2018] Mosavi, A.; Ozturk, P.; and Chau, K.-w.  2018\.  Flood prediction using machine learning models: Literature review.  Water 10(11):1536.
  * [Novikov et al. 2018] Novikov, G.; Trekin, A.; Potapov, G.; Ignatiev, V.; and Burnaev, E.  2018\.  Satellite imagery analysis for operational damage assessment in emergency situations.  In Business Information Systems: 21st International Conference, BIS 2018, Berlin, Germany, July 18-20, 2018, Proceedings 21, 347–358.  Springer.
  * [Obermeyer et al. 2021] Obermeyer, Z.; Nissan, R.; Stern, M.; Eaneff, S.; Bembeneck, E. J.; and Mullainathan, S.  2021\.  Algorithmic bias playbook.  Center for Applied AI at Chicago Booth.
  * [Oshri et al. 2018] Oshri, B.; Hu, A.; Adelson, P.; Chen, X.; Dupas, P.; Weinstein, J.; Burke, M.; Lobell, D.; and Ermon, S.  2018\.  Infrastructure quality assessment in africa using satellite imagery and deep learning.  In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 616–625.
  * [Parks et al. 2022] Parks, R. M.; Benavides, J.; Anderson, G. B.; Nethery, R. C.; Navas-Acien, A.; Dominici, F.; Ezzati, M.; and Kioumourtzoglou, M.-A.  2022\.  Association of tropical cyclones with county-level mortality in the us.  JAMA 327(10):946–955.
  * [Paulus and Kent 2020] Paulus, J. K., and Kent, D. M.  2020\.  Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities.  NPJ digital medicine 3(1):99.
  * [Pearl and Bareinboim 2022] Pearl, J., and Bareinboim, E.  2022\.  External validity: From do-calculus to transportability across populations.  In Probabilistic and causal inference: The works of Judea Pearl.  451–482.
  * [Pearl 2009] Pearl, J.  2009\.  Causality.  Cambridge university press.
  * [Peters, Bühlmann, and Meinshausen 2016] Peters, J.; Bühlmann, P.; and Meinshausen, N.  2016\.  Causal inference by using invariant prediction: identification and confidence intervals.  Journal of the Royal Statistical Society Series B: Statistical Methodology 78(5):947–1012.
  * [Pfister, Bühlmann, and Peters 2019] Pfister, N.; Bühlmann, P.; and Peters, J.  2019\.  Invariant causal prediction for sequential data.  Journal of the American Statistical Association 114(527):1264–1276.
  * [Pham and Luengo-Oroz 2022] Pham, K. H., and Luengo-Oroz, M.  2022\.  Predictive modeling of movements of refugees and internally displaced people: Towards a computational framework.
  * [Proma et al. 2022] Proma, A. M.; Islam, S.; Ciko, S.; Baten, R. A.; and Hoque, E.  2022\.  NADBenchmarks - a compilation of Benchmark Datasets for Machine Learning Tasks related to Natural Disasters.
  * [Prosperi et al. 2020] Prosperi, M.; Guo, Y.; Sperrin, M.; Koopman, J. S.; Min, J. S.; He, X.; Rich, S.; Wang, M.; Buchan, I. E.; and Bian, J.  2020\.  Causal inference and counterfactual prediction in machine learning for actionable healthcare.  Nature Machine Intelligence 2(7):369–375.
  * [Robinson, Hohman, and Dilkina 2017] Robinson, C.; Hohman, F.; and Dilkina, B.  2017\.  A deep learning approach for population estimation from satellite imagery.  In Proceedings of the 1st ACM SIGSPATIAL Workshop on Geospatial Humanities, 47–54.
  * [Rolnick et al. 2022] Rolnick, D.; Donti, P. L.; Kaack, L. H.; Kochanski, K.; Lacoste, A.; Sankaran, K.; Ross, A. S.; Milojevic-Dupont, N.; Jaques, N.; Waldman-Brown, A.; et al.  2022\.  Tackling climate change with machine learning.  ACM Computing Surveys (CSUR) 55(2):1–96.
  * [Rosenbaum and Rubin 1983] Rosenbaum, P. R., and Rubin, D. B.  1983\.  Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome.  Journal of the Royal Statistical Society: Series B (Methodological) 45(2):212–218.
  * [Rudolph et al. 2018] Rudolph, K. E.; Schmidt, N. M.; Glymour, M. M.; Crowder, R.; Galin, J.; Ahern, J.; and Osypuk, T. L.  2018\.  Composition or context: using transportability to understand drivers of site differences in a large-scale housing experiment.  Epidemiology (Cambridge, Mass.) 29(2):199.
  * [Runge et al. 2019] Runge, J.; Bathiany, S.; Bollt, E.; Camps-Valls, G.; Coumou, D.; Deyle, E.; Glymour, C.; Kretschmer, M.; Mahecha, M.; Muñoz-Marí, J.; et al.  2019\.  Inferring causation from time series in earth system sciences, nat. commun., 10, 2553.
  * [Sheth et al. 2022] Sheth, P.; Shah, R.; Sabo, J.; Candan, K. S.; and Liu, H.  2022\.  Stcd: A spatio-temporal causal discovery framework for hydrological systems.  In 2022 IEEE International Conference on Big Data (Big Data), 5578–5583.  IEEE.
  * [Subbaswamy, Chen, and Saria 2022] Subbaswamy, A.; Chen, B.; and Saria, S.  2022\.  A unifying causal framework for analyzing dataset shift-stable learning algorithms.  Journal of Causal Inference 10(1):64–89.
  * [Subbaswamy, Schulam, and Saria 2019] Subbaswamy, A.; Schulam, P.; and Saria, S.  2019\.  Preventing failures due to dataset shift: Learning predictive models that transport.  In The 22nd International Conference on Artificial Intelligence and Statistics, 3118–3127.  PMLR.
  * [Thalheimer, Simperingham, and Jjemba 2022] Thalheimer, L.; Simperingham, E.; and Jjemba, E. W.  2022\.  The role of anticipatory humanitarian action to reduce disaster displacement.  Environmental Research Letters 17(1):014043.
  * [Van der Heyden et al. 2014] Van der Heyden, J.; Demarest, S.; Van Herck, K.; De Bacquer, D.; Tafforeau, J.; and Van Oyen, H.  2014\.  Association between variables used in the field substitution and post-stratification adjustment in the belgian health interview survey and non-response.  International journal of public health 59:197–206.
  * [Vosper et al. 2023] Vosper, E.; Watson, P.; Harris, L.; McRae, A.; Santos-Rodriguez, R.; Aitchison, L.; and Mitchell, D.  2023\.  Deep learning for downscaling tropical cyclone rainfall to hazard-relevant spatial scales.  Journal of Geophysical Research: Atmospheres e2022JD038163.
  * [Wang, Chen, and Marathe 2019] Wang, L.; Chen, J.; and Marathe, M.  2019\.  Defsi: Deep learning based epidemic forecasting with synthetic information.  In Proceedings of the AAAI conference on artificial intelligence, volume 33, 9607–9612.
  * [Weingärtner and Wilkinson 2019] Weingärtner, L., and Wilkinson, E.  2019\.  Anticipatory crisis financing and action: concepts, initiatives and evidence.
  * [Xie et al. 2016] Xie, M.; Jean, N.; Burke, M.; Lobell, D.; and Ermon, S.  2016\.  Transfer learning from deep features for remote sensing and poverty mapping.  In Proceedings of the AAAI conference on artificial intelligence, volume 30.
  * [Yadlowsky et al. 2022] Yadlowsky, S.; Namkoong, H.; Basu, S.; Duchi, J.; and Tian, L.  2022\.  Bounds on the conditional and average treatment effect with unobserved confounding factors.  The Annals of Statistics 50(5):2587–2615.
  * [Yeh et al. 2020] Yeh, C.; Perez, A.; Driscoll, A.; Azzari, G.; Tang, Z.; Lobell, D.; Ermon, S.; and Burke, M.  2020\.  Using publicly available satellite imagery and deep learning to understand economic well-being in africa.  Nature communications 11(1):2583.
  * [Yu et al. 2022] Yu, J.; Liu, J.; Baek, J.-W.; Fong, C.; and Fu, M.  2022\.  Impact-based forecasting for improving the capacity of typhoon-related disaster risk reduction in typhoon committee region.  Tropical Cyclone Research and Review 11(3):163–173.
  * [Zhang et al. 2023] Zhang, Y.; Long, M.; Chen, K.; Xing, L.; Jin, R.; Jordan, M. I.; and Wang, J.  2023\.  Skilful nowcasting of extreme precipitation with NowcastNet.  Nature 619(7970):526–532.  Number: 7970 Publisher: Nature Publishing Group.
  * [Zhao and Zhang 2020] Zhao, F., and Zhang, C.  2020\.  Building damage evaluation from satellite imagery using deep learning.  In 2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI), 82–89.  IEEE.
  * [Zigler and Dominici 2014] Zigler, C. M., and Dominici, F.  2014\.  Point: clarifying policy evidence with potential-outcomes thinking—beyond exposure-response estimation in air pollution epidemiology.  American journal of epidemiology 180(12):1133–1140.