# Generalized Balancing Weights via Deep Neural Networks

Yoshiaki Kitazawa
NTT DATA Mathematical Systems Inc.
Data Mining Division
1F Shinanomachi Rengakan, 35, Shinanomachi
Shinjuku-ku, Tokyo, 160-0016, Japan
<EMAIL_ADDRESS>
###### Abstract

Estimating causal effects from observational data is a central problem in many
domains. A general approach is to balance covariates with weights such that
the distribution of the data mimics randomization. We present generalized
balancing weights, Neural Balancing Weights (NBW), to estimate the causal
effects of an arbitrary mixture of discrete and continuous interventions. The
weights were obtained through direct estimation of the density ratio between
the source and balanced distributions by optimizing the variational
representation of $f$-divergence. For this, we selected $\alpha$-divergence as
it presents efficient optimization because it has an estimator whose sample
complexity is independent of its ground truth value and unbiased mini-batch
gradients; moreover, it is advantageous for the vanishing-gradient problem. In
addition, we provide the following two methods for estimating the balancing
weights: improving the generalization performance of the balancing weights and
checking the balance of the distribution changed by the weights. Finally, we
discuss the sample size requirements for the weights as a general problem of a
curse of dimensionality when balancing multidimensional data. Our study
provides a basic approach for estimating the balancing weights of
multidimensional data using variational $f$-divergences.

##  1 Introduction

Estimating causal effects from observational data is a central problem in many
application domains, including public health, social sciences, clinical
pharmacology, and clinical decision-making. One standard approach is balancing
covariates with weights that are the same as the density ratios between the
source and balanced distributions, such that their distribution mimics
randomization. Many methods have been developed to estimate the balancing
weights, such as inverse propensity weighting (IPW) Rosenbaum and Rubin [25],
augmented inverse propensity weighting (AIPW) [23], generalized propensity
score (GPS) [10], covariate balancing propensity score (CBPS) [9], overlap
weighting [14], and entropy balancing (EB) [8, 31]. However, these methods are
limited to categorical or continuous interventions.

In this study, we propose generalized balancing weights to estimate the causal
effects of an arbitrary mixture of discrete and continuous interventions. To
the best of our knowledge, no causal inference method focusing on the
balancing weights exists for this problem. We approach this problem by
directly estimating the density ratio, more precisely, the Radon–Nikodým
derivatives, between the source and balanced distributions using a neural
network algorithm by optimizing a variational representation of a
$f$-divergence. $f$-divergences, whose values are greater than or equal to
zero and considered zero if the two distributions are equal, are the
statistics used to measure the closeness of the two distributions. The optimal
functions for the variational representations derived from $f$-divergences
with the Legendre transform correspond to the density ratio between the
distributions [17]. An approach to estimate the density ratio by optimizing a
variational representation of a $f$-divergence was developed in the domain
adaptation region [30].

However, optimizing the $f$-divergences, including estimating the density
ratio, is challenging. This is due to the following reasons. First, for KL-
divergence, the dominant $f$-divergence, the requirements for sample size
increase exponentially with the true amount of the divergence [15, 29].
Second, a naive gradient estimate over mini-batch samples leads to a biased
estimate of the full gradient [4]. Third, gradients of neural networks often
vanish when the estimated probability ratios are close to zero [2].

To avoid the first problem, we focus on $\alpha$-divergence, which is a
subgroup of $f$-divergence. $\alpha$-divergence has an estimator whose sample
complexity is independent of its ground truth value and unbiased mini-batch
gradients. In addition, by selecting $\alpha$ from a particular interval, we
avoid vanishing gradients of neural networks when the neural networks reach
extreme local minima.

In addition, we provide two techniques for estimating the balancing weights.
First, we propose a validation method using test data and an early stopping
method to improve the generalization performance of balancing. The
generalization performance of the weights worsens as the dimensions of the
data increase, and the sample size requirements of the weights increase
exponentially with the dimensions. Next, we present a method for measuring the
performance of balancing weights by estimating the $\alpha$-divergence
information to check the balance of the distribution,

This study is divided into seven parts. First, we introduce the background of
the study. Second, we review related studies. Third, we define the terminology
and concepts for causal inferences. Fourth, we present our novel method for
estimating balancing weights. Fifth, we provide techniques for estimating the
weights. Sixth, we discuss the sample requirements for the weights. Finally,
we conclude this paper. All the numerical experiments and proofs are described
in the appendix.

##  2 Related Work

#### Balancing weight.

Many methods have been proposed to estimate the balancing weights. The
following methods are proposed for binary intervention: inverse propensity
weighting (IPW) [25], augmented inverse propensity weighting (AIPW) [23],
covariate balancing propensity score (CBPS) [9] and overlap weighting [14].
The following methods have been proposed for continuous intervention:
generalized propensity score (GPS) [10] and entropy balancing (EB) [8, 31].
Recently, Lee, Ma, and de Luna (2022) proposed balancing weights available for
discrete or continuous intervention but only one of them at a time.

#### Statistical divergences and density ratio estimation.

Despite the abundance of classic studies [16, 30], we focused on studies that
directly estimate density ratios or optimize statistical divergences using
neural networks. In this review, these studies have beenclassified into four
groups. First is the estimation of KL-divergence or mutual information [3, 19,
22]; the second is density ratio estimation [11]; the third is generative
adversarial networks (GANs) [18, 32, 6, 33] (statistical divergences were used
as discriminators for GANs); and the fourth is domain generation [28, 6, 37,
1]. In addition to these application studies, divergences were improved [5].

##  3 Terminologies and Definitions

Here, we briefly introduce the terminology and definitions used in this study.

#### Notations and Terminologies.

Random variables are denoted by capital letters; for example, $A$. Small
letters are used for the values of random variables of the corresponding
capital letters; $a$ is the value of the random variable $A$. Bold letters
$\mathbf{A}$ or $\mathbf{a}$ represent a set of variables or random variable
values. In particular, $\mathbf{V}=\\{V_{1},\dots,V_{n}\\}$ are used for the
observed random variables and $\mathbf{U}=\\{U_{1},\dots,U_{m}\\}$ are used as
unobserved random variables. For example, the domain of the variable $A$ is
denoted by $\mathcal{X}_{A}$, and
$\mathcal{X}_{A_{1}}\times\cdots\times\mathcal{X}_{A_{n}}$ is denoted by
$\mathcal{X}_{\mathbf{A}}$ for $\mathbf{A}=A_{1}\times\cdots\times A_{n}$.
$\mathbf{V}\cup\mathbf{U}$ are assumed to be semi-Markovian models and
$G=G_{\mathbf{V}\mathbf{U}}$ denotes the causal graph for
$\mathbf{V}\cup\mathbf{U}$. $Pa(\mathbf{A})_{G}$, $Ch(\mathbf{A})_{G}$,
$An(\mathbf{A})_{G}$, and $De(\mathbf{A})_{G}$ represent parents, children,
ancestors, and descendants of the observed variables in $G$, respectively, for
$\mathbf{A}\subset\mathbf{V}$. In this study, $Pa(\mathbf{A})_{G}$,
$Ch(\mathbf{A})_{G}$, $An(\mathbf{A})_{G}$, and $De(\mathbf{A})_{G}$ do not
include $\mathbf{A}$. $P$ and $Q$ are used as the probability measures on
$(\mathbb{R}^{d},\mathscr{F})$, where $\mathscr{F}$ denotes the
$\sigma$-algebra of subsets of $\mathbb{R}^{d}$. $E_{P}[\cdot]$ and
$E_{P}[\cdot|\cdot]$ denote expectation and conditional expectation under the
distribution $P$, respectively. For example,
$E_{P}[\mathbf{X}]=\int_{\mathcal{X}_{\mathbf{X}}}dP$ and
$E_{P}[\mathbf{Y}|\mathbf{X}]=\int_{\mathcal{X}_{\mathbf{Y}}}dP(\mathbf{Y}|\mathbf{X})$.
$\hat{E}_{P}[\cdot]$ denotes the empirical expectation under $P$; that is, the
sample mean of the finite observations drawn from $P$. $P$ is called _absolute
continuous_ with respect to $Q$, $P(A)=0$ whenever $Q(A)=0$ for any
$A\in\mathscr{F}$, which is represented as $P\ll Q$. $\frac{dP}{dQ}$ denotes
the Radon–Nikodým derivative of $P$ with respect to $Q$ for $P$ and $Q$ with
$P\ll Q$. In this study, we refer to density ratios as the Radon–Nikodým
derivatives. $\mu$ denotes a probability measure on $\mathbb{R}^{d}$ with
$P\ll\mu$ and $Q\ll\mu$.
$\mathbf{X}^{(N)}=\\{\mathbf{X}^{1},\ldots,\mathbf{X}^{N}\\}$ denotes $N$
i.i.d. random variables from $\mu$.
$\mathbf{X}_{P}^{(N)}=\allowbreak\\{\mathbf{X}^{1}_{\sim
P},\allowbreak\ldots,\allowbreak\mathbf{X}^{N}_{\sim P}\\}$ and
$\mathbf{X}_{Q}^{(N)}=\allowbreak\\{\mathbf{X}^{1}_{\sim
Q},\allowbreak\ldots,\allowbreak\mathbf{X}^{N}_{\sim Q}\\}$ denote variables
defined as $P(\mathbf{X}^{i}_{\sim
P}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X}^{i}\leq\mathbf{x})$ and
$Q(\mathbf{X}^{i}_{\sim
Q}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X}^{i}\leq\mathbf{x})$,
$\forall\mathbf{x}\in\mathbb{R}^{d}$, for $1\leq i\leq N$. We represent
$f\lesssim g$ when $\lim\sup_{n\rightarrow\infty}f(n)/g(n)<\infty$ holds. The
notation $f\gtrsim g$ is defined similarly.

###  3.1 Definitions

In this study, we considered the causal effects of joint and multidimensional
interventions. For clarity, we used different notations, “$do$” and
“$\overline{do}$”, for single-dimensional and multidimensional interventions,
respectively. 111The values of the variables in the parentheses for both
symbols can be dropped if not necessary in the context. For example, we
sometimes represent $do(\mathbf{X}=\mathbf{x})$ or
$\overline{do}(\mathbf{X}=\mathbf{x})$ as $do(\mathbf{X})$ or
$\overline{do}(\mathbf{X})$, respectively. For a single-dimensional
intervention, a $do$ symbol is used, which is the same as Pearl’s
$do$-calculation.

######  Definition 3.1 ($do$-calculation, Pearl(2009)).

For the two given disjoint sets of variables
$\mathbf{X},\mathbf{Y}\subset\mathbf{V}$, the causal effect on $\mathbf{Y}$
for intervention in $\mathbf{X}$ with values $\mathbf{x}$, denoted by
$P(\mathbf{Y}|do(\mathbf{X}=\mathbf{x}))$, is defined as the probability
distribution, such that

$P(\mathbf{Y}|do(\mathbf{X}=\mathbf{x}))=\sum_{\begin{subarray}{c}\mathbf{v}^{\prime}\in\mathcal{X}_{\mathbf{V}^{\prime}}\\\
pa_{\mathbf{x}}\in\mathcal{X}_{Pa(\mathbf{X})_{G}}\end{subarray}}\frac{P(\mathbf{Y},\mathbf{X}=\mathbf{x},Pa(\mathbf{X})_{G}=pa_{\mathbf{x}},\mathbf{V}^{\prime}=\mathbf{v}^{\prime})}{P(\mathbf{X}=\mathbf{x}|Pa(\mathbf{X})_{G}=pa_{\mathbf{x}})},$
(1)

where $\mathbf{V}^{\prime}=\mathbf{V}\setminus(\mathbf{X}\cup
Pa(\mathbf{X})_{G}\cup\mathbf{Y})$. The causal effect of $\mathbf{X}$ on
$\mathbf{Y}$ under the conditions $\mathbf{Z}$ denoted by
$P(\mathbf{Y}=\mathbf{y}|do(\mathbf{X}=\mathbf{x}),\mathbf{Z}=\mathbf{z})$ is
defined as the probability distribution, such that

$P(\mathbf{Y}=\mathbf{y}|do(\mathbf{X}=\mathbf{x}),\mathbf{Z})=\frac{P(\mathbf{Y}=\mathbf{y},\mathbf{Z}|do(\mathbf{X}=\mathbf{x}))}{P(\mathbf{Z}|do(\mathbf{X}=\mathbf{x}))}.$
(2)

Notably, from Definition 3.1, a $do$-calculation for a set of variables
coincides with the simultaneous interventions for each variable:

$P(\mathbf{Y}|do(\mathbf{X}))=P(\mathbf{Y}|do(X_{1}),do(X_{2}),\ldots,do(X_{n})),$
(3)

where $\mathbf{X}=\\{X_{1},X_{2},\dots,X_{n}\\}$. Here, we refer to each
intervention in Eq. (3) as a “single-dimensional intervention” .

Furthermore, we use the $\overline{do}$ symbol for multidimensional
intervention. Intuitively, a $\overline{do}$ symbol represents the
intervention of the variables that preserves the functional relationship
within the variables.

######  Definition 3.2 ($\overline{do}$ symbol).

For disjoint sets of variables
$\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Y}\subset\mathbf{V}$,
$\overline{do}$ symbol defines the following probability distribution over
them:

$P(\mathbf{Y},\overline{do}(\mathbf{X_{1}}),\overline{do}(\mathbf{X_{2}}),\ldots,\overline{do}(\mathbf{X_{n}}))=P(\mathbf{Y}|do(\mathbf{X}))\times
P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}}),$ (4)

where
$\mathbf{X}=\mathbf{X}_{1}\cup\mathbf{X}_{2}\cup\dots\cup\mathbf{X}_{n}$.

$\overline{do}$ symbols are useful, particularly when we consider
interventions in a multivalued discrete variable expressed using one-hot
encoding. In this case, we cannot express the causal effect effectively using
$do$ symbols. For example, let us consider the case of an intervention in the
ternary variable $X$, $\mathcal{X}_{X}=\\{x_{1},x_{2},x_{3}\\}$ and let $X$ be
expressed by
$\mathbf{X}^{\prime}=(X^{\prime}_{1},X^{\prime}_{2},X^{\prime}_{3})$, such
that $X^{\prime}_{i}=1$ if $X=x_{i}$ otherwise $X^{\prime}_{i}=0$ for
$i=1,2,3$. Then, $P(\mathbf{Y}|do(X=x_{3}))$ is the same as
$P(\mathbf{Y},\overline{do}(\mathbf{X}^{\prime}=(0,0,1)))$, which differs from
$P(\mathbf{Y}|do(\mathbf{X}^{\prime}=(0,0,1)))$. We refer to this type of
intervention as a “multidimensional intervention” .

Next, we provide definitions of the $f$-divergence and $f$-divergence
information.

######  Definition 3.3 ($f$-divergence).

The $f$-divergence $D_{f}$ between the two probability measures $P$ and $Q$
with $Q\ll P$ induced by a convex function $f$ satisfying $f(1)=0$ is defined
by $D_{f}(Q||P)=E_{P}[f(dQ/dP)]$.

Many divergences are specific cases obtained by selecting a suitable generator
function $f$. For example, $f(u)=u\log u$ corresponds to the KL-divergence. In
particular, we focus on $\alpha$-divergence, which is expressed as follows:

$D_{\alpha}(Q||P)=E_{P}\left[\frac{1}{\alpha(\alpha-1)}\left\\{\left(\frac{dQ}{dP}\right)^{1-\alpha}-1\right\\}\right],$
(5)

where $\alpha\in\mathbb{R}\setminus\\{0,1\\}$. From Eq. (5), Hellinger
divergence is obtained as $\alpha=1/2$, and $\chi^{2}$ divergence by
$\alpha=-1$.

From $f$-divergence, the $f$-divergence information is defined as the mutual
information if we choose the KL-divergence as the $f$-divergence. Here, we
present a definition of $f$-divergence information for multi-variables.

######  Definition 3.4 ($f$-divergence information).

For disjoint sets of variables
$\mathbf{X}=\\{\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{n}\\}\subset\mathbf{V}$,
let $P_{\mathbf{X}}$ be the joint probability measure for $\mathbf{X}$. For
each $i=1,2,\ldots,n$,
$P_{\mathbf{X}_{i}}=\int_{\mathcal{X}_{\mathbf{X}\setminus\mathbf{X}_{i}}}dP_{\mathbf{X}}$
is a measure of the marginal distribution of $P_{\mathbf{X}}$ for
$\mathbf{X}_{i}$. The $f$-divergence information for
$\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{n}$ under $P_{\mathbf{X}}$
and a convex function $f$ satisfying $f(1)=0$ is defined as the $f$-divergence
between $P_{\mathbf{X}}$ and $P_{\mathbf{X}_{1}}\times
P_{\mathbf{X}_{2}}\times\cdots\times P_{\mathbf{X}_{n}}$:

$\displaystyle
I_{f}(\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{n};P_{\mathbf{X}})$
$\displaystyle=$ $\displaystyle
E_{P_{\mathbf{X}}}\left[f\left(\frac{dP_{\mathbf{X}_{1}}\times
dP_{\mathbf{X}_{2}}\times\cdots\times
dP_{\mathbf{X}_{n}}}{dP_{\mathbf{X}}}\right)\right].$ (6)

##  4 Problem Set Up

Before describing the details of the problem, we provide a notation for the
probability distribution, which is the goal of balancing. Hereafter, $P$
denotes the probability distribution of observational data. For the given
disjoint sets
$\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Y},\mathbf{Z}\subset\mathbf{V}$,
let $\widetilde{P}$ be a probability distribution, as follows:

$\displaystyle\widetilde{P}$ $\displaystyle=$ $\displaystyle
P(\mathbf{Y},\overline{do}(\mathbf{X_{1}}),\overline{do}(\mathbf{X_{2}}),\ldots,\overline{do}(\mathbf{X_{n}}),\mathbf{Z})$
(7) $\displaystyle=$ $\displaystyle
P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})\times P(\mathbf{X_{1}})\times
P(\mathbf{X_{2}})\times\cdots\times P(\mathbf{X_{n}})\times P(\mathbf{Z}),$

where
$\mathbf{X}=\mathbf{X}_{1}\cup\mathbf{X}_{2}\cup\dots\cup\mathbf{X}_{n}$.
$\widetilde{P}$ is the probability distribution of the counterfactual data
from simultaneous (multidimensional) interventions in
$\mathbf{X_{1}},\mathbf{X_{2}},\ldots,\mathbf{X_{n}}$ under the condition
$\mathbf{Z}$.

#### Objective.

The objective of this study is to obtain the balancing weights that transform
$P(\mathbf{Y},\mathbf{X},\mathbf{Z})$ into
$\widetilde{P}(\mathbf{Y},\mathbf{X},\mathbf{Z})$. More precisely, given the
i.i.d. observational data
$\\{(\mathbf{x}^{i},\mathbf{z}^{i})|i=1,2,\ldots,N\\}$, we aim to estimate the
weights $\mathit{BW}(\mathbf{X},\mathbf{Z})$, such that

$E_{\widetilde{P}}\left[f(\mathbf{X},\mathbf{Z})\right]=E_{P}\left[f(\mathbf{X},\mathbf{Z})\cdot\mathit{BW}(\mathbf{X},\mathbf{Z})\right]$
(8)

holds for any measurable function $f$ on $\mathbb{R}^{d}$. If we obtain the
weights, we estimate the conditional average causal effect (CACE) for
$P(\mathbf{Y},\overline{do}(\mathbf{X_{1}}),\overline{do}(\mathbf{X_{2}}),\ldots,\allowbreak\overline{do}(\mathbf{X_{n}}),\mathbf{Z})$,
that is $E_{\widetilde{P}}[\mathbf{Y}|\mathbf{X},\mathbf{Z}]$, using state-of-
the-art supervised machine learning algorithms, with the weights assigned as
the individual weights for each sample.

#### Assumptions.

We assumed the following to achieve our objective:

  * •

Assumption 1. The causal effect $P(\mathbf{Y}|do(\mathbf{X}))$ is
identifiable, or equivalently, $\widetilde{P}$ from Eq. (7) can be identified.
222The simplest case that satisfies Assumption 1 is that no confounding exists
among the data ([21], P78, Theorem 3.2.5). 333If certain unobserved data are
assumed to exist, the identifiability of the causal effect is determined by
the structure of the causal diagram for $P$. One criterion for the
identifiability of a causal effect is expressed by [27]. The discussion of the
identifiability of the causal effect is beyond the scope of this study.

  * •

Assumption 2. Let $\mathbb{P}$ =
$P(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})$ and let
$\mathbb{Q}$ = $P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}})\times P(\mathbf{Z})$. Subsequently, we assume that
$\mathbb{Q}\ll\mathbb{P}$.

Assumption 2 is the same as the _overlap_ assumption if we consider this a
single-dimensional intervention. Here, we propose overlapped assumptions for
joint and multidimensional interventions.

##  5 Estimation of Balancing Weights

In this section, we present the way to effectively estimate the probability
density ratios by optimizing $f$-divergence.

#### Density Ratios as Balancing Weights.

We first note that the density ratios, which are referred to as the
Radon–Nikodým derivative in this paper, are equal to the balancing weight of
the target. For a density ratio of $P$ to $\widetilde{P}$, that is
$\frac{d\widetilde{P}}{dP}$, it holds that

$E_{\widetilde{P}}[f]=\int f\cdot\frac{d\widetilde{P}}{dP}\cdot
dP=E_{P}\left[f\cdot\frac{d\widetilde{P}}{dP}\right],$ (9)

for any measurable function $f$ in $\mathbb{R}^{d}$. Then, Eq. (8) and Eq. (9)
are equivalent. As an example of the aforementioned density ratio, let $X$ be
a binary variable with $\mathcal{X}_{X}=\\{1,0\\}$ and let $\mathbf{Z}$ be
covariates. Using propensity score
$e(\mathbf{z})=P(X=1|\mathbf{Z}=\mathbf{z})$, we observe that
$\frac{d\widetilde{P}}{dP}(X=1,\mathbf{z})=P(X=1)/e(\mathbf{z})$ and
$\frac{d\widetilde{P}}{dP}(X=0,\mathbf{z})=P(X=0)/(1-e(\mathbf{z}))$. That is,
$\frac{d\widetilde{P}}{dP}$ is the stabilized inverse probability of the
treatment weighting [24].

###  5.1 Our Approach

Our approach involves obtaining the density ratios as an optimal function for
a variational representation of an $f$-divergence. This approach is based on
the fact that the optimal function is connected to density ratios [16].

#### Variational representation.

Using the Legendre transform of the convex conjugate of a twice differentiable
convex function $f$, $f^{*}(\psi)=\sup_{r\in\mathbb{R}}\\{\psi\cdot
r-f(r)\\}$, we obtain a variational representation of $f$-divergence:

$D_{f}(Q||P)=\sup_{\phi\geq
0}\\{E_{Q}[f^{\prime}(\phi)]-E_{P}[f^{*}(f^{\prime}(\phi))]\\},$ (10)

where supremum is considered over all measurable functions with
$E_{Q}[f^{\prime}(\phi)]<\infty$ and $E_{P}[f^{*}(f^{\prime}(\phi))]<\infty$.
The maximum value is achieved at $\phi=dQ/dP$. 444Eq. (10) holds only for
differentiable convex functions. For a general statement of a variational
representation of $f$-divergence, for example, see Nguyen et al.(2007).

We obtained the optimal function for Eq. (10) by replacing $\phi$ in the
equation with a neural network model $\phi_{\theta}$ and training it through
back-propagation with a loss function, such that

$\mathcal{L}(\theta)=-\left\\{\hat{E}_{Q}[f^{\prime}(\phi_{\theta})]-\hat{E}_{P}[f^{*}(f^{\prime}(\phi_{\theta}))]\right\\}.$
(11)

#### Selecting $\alpha$-divergence for Optimization.

We select $\alpha$-divergence for the following reasons. First, the sample
size requirements for $\alpha$-divergence is independent of its ground truth
value; second, it has unbiased mini-batch gradients; third, it can avoid a
vanishing gradient problem.

The variational representation of $\alpha$-divergence is as follows (Lemma C.1
in Appendix C):

$\displaystyle D_{\alpha}(Q||P)=\sup_{\phi\geq
0}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[\phi^{-\alpha}\right]-\frac{1}{1-\alpha}E_{P}\left[\phi^{1-\alpha}\right]\right\\}.$
(12)

#### Sample size requirements for $\alpha$-divergence.

The $\alpha$-divergence has an estimator with sample complexity $O(1)$
(Corollary 1 in Birrell et al., 2022, P19; Corollary C.10 in Appendix C).
Conversely, the sample complexity of KL-divergence is $O(e^{KL(Q||P)})$ [15,
29]:

$\lim_{N\rightarrow\infty}\frac{N\cdot\mathrm{Var}\Big{[}\widehat{KL^{N}}(Q||P)\Big{]}}{KL(Q||P)^{2}}\geq\frac{e^{KL(Q||P)}-1}{KL(Q||P)^{2}},$
(13)

where $\widehat{KL^{N}}(Q||P)$ is the KL-divergence estimator for sample size
N using a variational representation of the divergence, and $KL(Q||P)$ is the
ground truth value.

#### Unbiasedness for mini-batch gradients.

$\phi$ in Eq. (12) can be expressed in a Gibbs density form (Proposition C.2
in Appendix C). Then, we observe that

$D_{\alpha}(Q||P)=\sup_{T}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot
T}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot T}\right]\right\\},$
(14)

where supremum is considered over all measurable functions
$T:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with $E_{P}[e^{(\alpha-1)\cdot
T}]<\infty$ and $E_{Q}[e^{\alpha\cdot T}]<\infty$.

From this equation, we obtain our loss function, which has unbiasedness for
mini-batch gradients (Proposition C.8 in Appendix C), as follows :

$\mathcal{L}_{\alpha}(\theta)=\frac{1}{\alpha}\hat{E}_{Q}\left[e^{\alpha\cdot
T_{\theta}}\right]+\frac{1}{1-\alpha}\hat{E}_{P}\left[e^{(\alpha-1)\cdot
T_{\theta}}\right].$ (15)

#### Advantage in vanishing gradients problem.

By setting $\alpha$ within $(0,1)$, we can avoid vanishing gradients of neural
networks when they reach the extreme local minima. The vanishing-gradient
problem for optimizing divergence is known in GANs [2]. Now, we consider the
case where the probability ratio $e^{T_{\theta}(\mathbf{x})}$ in Eq. (15) is
nearly zero or large for some point $\mathbf{x}$, corresponding to cases in
which the probabilities for $P$ or $Q$ at some points are much smaller than
those for the other.

To show the relation between $e^{T_{\theta}(\mathbf{x})}$ and the learning of
the neural networks, we obtain gradient of Eq. (15):

$\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)=\hat{E}_{Q}\Big{[}\nabla_{\theta}T_{\theta}\cdot
e^{\alpha\cdot
T_{\theta}}\Big{]}-\hat{E}_{P}\Big{[}\nabla_{\theta}T_{\theta}\cdot
e^{(\alpha-1)\cdot T_{\theta}}\Big{]}.$ (16)

The behavior of $\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)$ when
$E_{Q}[e^{T_{\theta}}]\rightarrow 0$ or
$E_{Q}[e^{T_{\theta}}]\allowbreak\rightarrow\allowbreak\infty$, under some
regular conditions for $T_{\theta}$ and an assumption that $P\ll Q$, can be
summarized as follows: Let $E[\ \cdot\ ]$ denote $E_{P}[E_{Q}[\ \cdot\ ]]$,
then

$\alpha>1$: $E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{0}$
(as $E_{Q}[e^{T_{\theta}}]\rightarrow 0$), and
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{\infty}-\vec{\infty}$
(as $E_{Q}[e^{T_{\theta}}]\rightarrow\infty$).

$\alpha<0$: $E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{0}$
(as $E_{Q}[e^{T_{\theta}}]\rightarrow\infty$), and
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{\infty}-\vec{\infty}$
(as $E_{Q}[e^{T_{\theta}}]\rightarrow 0$).

$0<\alpha<1$:
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow-\vec{\infty}$ (as
$E_{Q}[e^{T_{\theta}}]\rightarrow 0$), and
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{\infty}$ (as
$E_{Q}[e^{T_{\theta}}]\rightarrow\infty$).

Notably, $E_{Q}[e^{T_{\theta}}]\rightarrow 0\Leftrightarrow
E_{P}[e^{T_{\theta}}]\rightarrow 0$ and
$E_{Q}[e^{T_{\theta}}]\rightarrow\infty\Leftrightarrow
E_{P}[e^{T_{\theta}}]\rightarrow\infty$, because $Q\ll P$ and $P\ll Q$.

For $\alpha>1$ and $\alpha<0$, cases exist where
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\allowbreak\rightarrow\vec{0}$.
This implies the possibility that the neural networks reach extreme local
minima such that their estimations for density ratios are $0$ or $\infty$.
However, this problem can be avoided by selecting $\alpha$ from interval
$(0,1)$. We note that the selecting of $\alpha$ does not cause instability in
numerical calculations for cases where
$E[\nabla_{\theta}\mathcal{L}_{\alpha}(\theta)]\rightarrow\vec{\infty}-\vec{\infty}$.
In Appendix D.1, we present numerical experimental results for different
values of $\alpha$.

##  6 Method

In this section, we first present the main theorem that summarizes the new
balancing weight method proposed herein. Next, we present the balancing weight
method.

###  6.1 Main Theorem

Here, we present the main theorem that summarizes the new balancing weight
method proposed herein.

######  Theorem 6.1.

Given disjoint sets of
$\mathbf{X}=\\{\mathbf{X}_{1},\allowbreak\mathbf{X}_{2},\allowbreak\dots,\allowbreak\mathbf{X}_{n}\\},\allowbreak\mathbf{Y},\allowbreak\mathbf{Z}\subset\allowbreak\mathbf{V}$
satisfying

$\mathbf{X}=\\{\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n}\\}\subset
An(\mathbf{Y})_{G}\hskip 12.91663pt\text{and}\hskip 12.91663pt\mathbf{Z}\cap
De(\mathbf{Y})_{G}=\phi.$ (17)

Let
$\mathbb{P}=P(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})$
and $\mathbb{Q}=P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}})\times P(\mathbf{Z})$, and
$\widetilde{P}=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})\times
P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}})\times P(\mathbf{Z})$. We assume that $P$ satisfies
Assumptions 1 and 2 in the aforementioned setting, and it holds that
$E_{\mathbb{P}}\left[\left(d\mathbb{Q}/d\mathbb{P}\right)^{1-\alpha}\right]<\infty$
for some $0<\alpha<1$, then, for the optimal function $T^{*}$, such that

$\displaystyle
T^{*}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})=\arg\inf_{T\in\mathcal{T}^{\alpha}}\left\\{\frac{1}{\alpha}E_{\mathbb{Q}}\left[e^{\alpha\cdot
T}\right]+\frac{1}{1-\alpha}E_{\mathbb{P}}\left[e^{(\alpha-1)\cdot
T}\right]\right\\},$ (18)

it holds that

$\frac{d\widetilde{P}}{dP}=e^{-T^{*}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})}.$
(19)

Here, $\mathcal{T}^{\alpha}$ denotes the set of all non-constant functions
$T(\mathbf{x}):\mathbb{R}^{d}\rightarrow\mathbb{R}$ with
$E_{\mathbb{P}}[e^{(\alpha-1)\cdot T(\mathbf{X})}]<\infty$.

###### Proof.

See Appendix C. ∎

Here, we mention that the assumption (17) is necessary for the Eq. (19) to
hold, which is derived from our Theorem C.15 in Appendix C.

###  6.2 Balancing Weight Method

We present the implementation of training a neural balancing weights (NBW)
model in Algorithm 1. It is important to consider the stopping time $K$ for
neural network model $T_{\theta_{K}}$ in Algorithm 1, which is discussed in
the next section. To obtain the sample mean under $\mathbb{Q}$, that is, the
estimator for $E_{\mathbb{Q}}\left[e^{\alpha\cdot T_{\theta}}\right]$ in Eq.
(18), a shuffling operation can be used for the samples. Now, we define neural
balancing weights (NBW). 555We distinguish the notation of
$\mathit{BW}(\cdot)$ by the expression of the variables in the parentheses.
For example, for three variables $X_{1},X_{2},X_{3}\subset\mathbf{V}$, let
$\mathbf{X}=\\{X_{1},X_{2}\\}$. Then,
$\mathit{BW}(\mathbf{X},X_{3};T_{\theta})$ is used to indicate the balancing
weights for $dP(X_{1},X_{2})\times dP(X_{3})/dP(X_{1},X_{2},X_{3})$.
Conversely, $\mathit{BW}(X_{1},X_{2},X_{3};T_{\theta})$ denotes the balancing
weights for $dP(X_{1})\times dP(X_{2})\times dP(X_{3})/dP(X_{1},X_{2},X_{3})$.
666However, we drop the variables in the parentheses and write
$\mathit{BW}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z};T_{\theta})$
as $\mathit{BW}_{\theta}$ if not necessary in the context.

######  Definition 6.2 (Neural Balancing Weights).

Let $T_{\theta_{K}}$ be a neural networks obtained from Algorithm 1. Then, the
NBW of $T_{\theta_{K}}$, expressed as
$\mathit{BW}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z};T_{\theta_{K}})$,
are defined as

$\mathit{BW}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z};T_{\theta_{K}})=\frac{1}{Z}e^{-T_{\theta_{K}}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})},$
(20)

where
$Z=\hat{E}_{\mathbb{P}}\left[e^{-T_{\theta_{K}}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})}\right]$.

We estimate $E_{\widetilde{P}}[\mathbf{Y}|\mathbf{X},\mathbf{Z}]$, that is the
CACE for
$P(\mathbf{Y},\overline{do}(\mathbf{X_{1}}),\allowbreak\overline{do}(\mathbf{X_{2}}),\allowbreak\ldots,\allowbreak\overline{do}(\mathbf{X_{n}}),\allowbreak\mathbf{Z})$,
using
$\mathit{BW}(\mathbf{X}_{1},\allowbreak\mathbf{X}_{2},\allowbreak\dots,\allowbreak\mathbf{X}_{n},\allowbreak\mathbf{Z};\allowbreak
T_{\theta_{K}})$ as the sample weights of the supervised algorithm:

$\widehat{E}_{\widetilde{P}}\left[\mathbf{Y}|\mathbf{X},\mathbf{Z}\right]=\widehat{E}_{P}\left[\mathbf{Y}\cdot\mathit{BW}_{\theta_{K}}\
|\ \mathbf{X},\mathbf{Z}\right].$ (21)

Here, $\widehat{E}_{P}$ corresponds to the model of a supervised machine
learning algorithm. As an example, we demonstrate a back-propagation algorithm
using balancing weights for the mean squared error (MSE) loss in Algorithm 3
in Appendix E.

Algorithm 1 Training a Neural Balancing Weight model

Train Data
$(\mathbf{x}_{1},\\{(\mathbf{x}_{1}^{i},\ldots,\mathbf{x}_{n}^{i},\allowbreak\mathbf{z}^{i})\\}_{i=1}^{N}$

A Neural Balancing Weight Model $T_{\theta_{K}}$

$\sigma_{1}^{\mathbf{x}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

$\vdots$

$\sigma_{n}^{\mathbf{x}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

$\sigma^{\mathbf{z}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

for $t=1$ to $K$ do

$\hat{E}_{\mathbb{P}}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{(\alpha-1)\cdot
T_{\theta_{t}}(\mathbf{x}_{1}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})}$

$\hat{E}_{\mathbb{Q}}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{\alpha\cdot
T_{\theta_{t}}(\mathbf{x}_{1}^{\sigma_{1}^{\mathbf{x}}(i)},\ldots,\mathbf{x}_{n}^{\sigma_{n}^{\mathbf{x}}(i)},\mathbf{z}^{\sigma^{\mathbf{z}}(i)})}$

$\mathcal{L}_{\alpha}({\theta_{t}})\leftarrow\hat{E}_{\mathbb{Q}}/\alpha+\hat{E}_{\mathbb{P}}/(1-\alpha)$

$\theta_{t+1}\leftarrow\theta_{t}-\nabla_{\theta_{t}}\mathcal{L}_{\alpha}({\theta_{t}})$

end for

##  7 Techniques for NBW

We propose two techniques for estimating balancing weights: ($\mathrm{i}$)
improves generalization performance of the balancing weights. ($\mathrm{ii}$)
measures the performance of the balancing weights by estimating the
$\alpha$-divergence information.

###  7.1 Improving the Generalization Performance of the Balancing Weights

In this section, we first present an overfitting problem for balancing
distributions. We then present two methods for improving the generalization
performance of the weights: a validation method using test data and an early
stopping method. Herein, let $T_{\theta_{t}}$ denote an NBW model at step $t$
in Algorithm 1. Let
$\hat{\mathbf{X}}_{{Q}}^{(N)}(t)=e^{-T_{\theta_{t}}}\cdot\mathbf{X}_{{P}}^{(N)}$,
that is, the data balanced by the weights of $e^{-T_{\theta_{t}}}$.
Subsequently, let $\hat{{Q}}^{(N)}_{t}$ and $\hat{{Q}}^{(N)}$ denote the
probability distributions of $\hat{\mathbf{X}}_{{Q}}^{(N)}(t)$ and
$\hat{\mathbf{X}}_{{Q}}^{(N)}$, respectively, which correspond to the
estimated and true distributions for balancing.

#### An overfitting problem for balancing distributions.

From Corollary C.12 in Appendix C, we observe
$\hat{\mathbf{X}}_{{Q}}^{(N)}(t)\xrightarrow{\ \ \text{d}\ \
}\mathbf{X}_{{Q}}^{(N)}$ as $t\rightarrow\infty$. Then, Theorem 1 in [35]
shows that

$\lim_{t\rightarrow\infty}W_{1}({Q},\hat{{Q}}^{(N)}_{t})=W_{1}({Q},\hat{{Q}}^{(N)})\gtrsim
N^{-1/(d-\delta)}\quad(\,\forall\delta>0\,),$ (22)

where $W_{1}$ is the Wasserstein distance of order $1$ and $d$ is the lower
Wasserstein dimension defined in [35]. Eq. (22) implies that, for balancing
finite data, the destination of the balanced distribution is an empirical
distribution, and the generalization performance of balancing worsens
exponentially when the dimension of the data is larger. In view of
optimizations of GANs, [36] referred to this phenomenon the “momorization” and
proposed an early stopping method.

#### Validation method using test data.

We can use a validation method using test data. Because $\hat{{Q}}^{(N)}$ and
$\hat{{P}}^{(N)}$ are empirical probability distributions, we observe that
$d\hat{{Q}}^{(N)}/d\hat{{P}}^{(N)}(x)=\allowbreak dQ/dP(x)$ if
$x\in\mathbf{X}^{(N)}$, otherwise $d\hat{{Q}}^{(N)}/d\hat{{P}}^{(N)}(x)=0$
(Proposition C.17 in Appendix C). Then, the optimal function of Eq. (15) for
both distributions, that is $T_{*}^{(N)}=\allowbreak-
log(d\hat{{Q}}^{(N)}/d\hat{{P}}^{(N)})$, is infinite except for the
observations, and the loss of the $T_{*}^{(N)}$ is infinite for data
independent of the observations. This implies that the loss of $T_{t}^{(N)}$
for the test data turns to increase from the middle of the training period,
and we can determine the training step at which the generalization performance
of the weights begins to worsen. In Section D.2 in Appendix D, we provide
numerical experimental results to confirm the relationship between dimensions
of data ($d$) and steps in training ($K$).

#### Early stopping method.

In addition, we present an early stopping method for estimating the balancing
weights as follows, which is inspired by the method developed in [36]
(Corollary C.24 in Appendix C): for some $\delta>0$, let

$K_{0}=C\cdot N^{2/(d+\delta)},$ (23)

where $C>0$ is constant. Then, we have
$W_{1}({Q},\hat{{Q}}^{(N)}_{K_{0}})\lesssim N^{-1/(d+\delta)}$. Unfortunately,
the curse of dimensionality remains in the proposed method. This will be
discussed in the next section.

###  7.2 Measuring the Performance of the Balancing Weights

Let us assume that we obtain an NBW model $T_{\theta_{0}}$ and let
$\mathit{BW}_{\theta_{0}}=\mathit{BW}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z};T_{\theta_{0}})$
be the balancing weights of $T_{\theta_{0}}$. If $\mathit{BW}_{\theta_{0}}$
successfully estimates $\frac{d{Q}}{d{P}}$, then the $\alpha$-divergence
between ${Q}$ and ${P}_{0}$ will be nearly zero. Conversely, if
$\mathit{BW}_{\theta_{0}}$ fails to estimate $\frac{d{Q}}{d{P}}$, the
$\alpha$-divergence between ${Q}$ and ${P}_{0}$ is significantly different
from zero. This implies that we can measure the performance of the balancing
weights using the $\alpha$-divergence information for ${P}_{0}$.

Next, we present the definition of an $\alpha$-divergence information
estimator using neural networks.

######  Definition 7.1 (Neural $\alpha$-divergence Information Estimator).

For disjoint sets of variables
$\mathbf{X}_{1},\allowbreak\mathbf{X}_{2},\allowbreak\ldots,\allowbreak\mathbf{X}_{n}\allowbreak\subset\allowbreak\mathbf{V}$,
the neural $\alpha$-divergence information estimator for ${P}$ is defined as

$\displaystyle\widehat{I}_{\alpha}(\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{n};T_{\theta_{*}})=\frac{1}{\alpha(1-\alpha)}-\inf_{\theta\in\Theta}\left\\{\frac{1}{\alpha}\hat{E}_{{Q}}\left[e^{\alpha\cdot
T_{\theta}}\right]+\frac{1}{1-\alpha}\hat{E}_{{P}}\left[e^{(\alpha-1)\cdot
T_{\theta}}\right]\right\\}.$ (24)

To measure the performance of balancing the weights from the NBW model, we
estimate the $\alpha$-divergence information for balanced distribution from
the weights. That is, we use the sample mean under a balanced distribution,
despite the sample mean under ${P}$ for Eq. (24). For example, we assume that
we have certain weights $\mathit{BW^{\prime}}=\\{bw^{i}:i=1,2,...,N\\}$, where
$bw^{i}$ denotes the weight of sample $i$ of $N$. The balanced distribution
from the weights is

$d{P}^{\prime}=\mathit{BW}^{\prime}\cdot d{P}.$ (25)

The $\alpha$-divergence information for ${P}^{\prime}$ is estimated by
replacing ${P}$ with ${P}^{\prime}$ for Eq. (24) in the following manner:
despite the sample mean $\hat{E}_{{P}}[e^{(\alpha-1)\cdot T_{\theta}}]$ for
these equations, we use the weighted sample mean, such that

$\hat{E}_{{P}^{\prime}}[e^{(\alpha-1)\cdot
T_{\theta}}]=\frac{1}{N}\Sigma_{i=1}^{N}bw^{i}\cdot e^{(\alpha-1)\cdot
T_{\theta}(\mathbf{x}_{1}^{i},\mathbf{x}_{2}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})}.$
(26)

Details on the implementation for measuring the performance of balancing
weights from an NBW model are provided in Algorithm 2, which includes the
validation method for the overfitting problem in Section 7.1.

Algorithm 2 Algorithm for checking the balance

Train Data
$\\{(\mathbf{x}_{1}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})\\}_{i=1}^{N}$,
Test Data
$\\{(\widetilde{\mathbf{x}}_{1}^{i},\ldots,\widetilde{\mathbf{x}}_{n}^{i},\widetilde{\mathbf{z}}^{i})\\}_{i=1}^{N}$,
A Neural Balancing Weight Model $T_{\theta}$

The estimated $\alpha$-divergence information $\widehat{I}_{\alpha}$ for the
balanced distribution with the balancing weights from $T_{\theta}$

$\sigma_{1}^{\mathbf{x}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

$\vdots$

$\sigma_{n}^{\mathbf{x}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

$\sigma^{\mathbf{z}}\leftarrow\texttt{SHUFFLE}(\\{1:N\\})$

$\\{bw^{i}\\}_{i}^{N}\leftarrow\frac{e^{-T_{\theta}(\mathbf{x}_{1}^{i},\mathbf{x}_{2}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})}}{\sum\\{e^{-T_{\theta}(\mathbf{x}_{1}^{i},\mathbf{x}_{2}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})}\\}}$

$\widehat{\mathbf{I}}_{\alpha}\leftarrow\\{\\}$

for $t=1$ to $K$ do

$\hat{E}_{\mathbb{P}_{0}}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{(\alpha-1)\cdot
T_{\psi}(\mathbf{x}_{1}^{i},\ldots,\mathbf{x}_{n}^{i},\mathbf{z}^{i})}\cdot
bw^{i}$

$\hat{E}_{\mathbb{Q}}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{\alpha\cdot
T_{\psi}(\mathbf{x}_{1}^{\sigma_{1}^{\mathbf{x}}(i)},\ldots,\mathbf{x}_{n}^{\sigma_{n}^{\mathbf{x}}(i)},\mathbf{z}^{\sigma^{\mathbf{z}}(i)})}$

$\mathcal{L}_{\alpha}({\psi})\leftarrow\hat{E}_{\mathbb{Q}}/\alpha+\hat{E}_{\mathbb{P}_{0}}/(1-\alpha)$

$\psi\leftarrow\psi-\nabla_{\psi}\mathcal{L}_{\alpha}({\psi})$

$\hat{E}_{\mathbb{P}_{0}}^{te}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{(\alpha-1)\cdot
T_{\psi}(\widetilde{\mathbf{x}}_{1}^{i},\ldots,\widetilde{\mathbf{x}}_{n}^{i},\widetilde{\mathbf{z}}^{i})}\cdot
bw^{i}$

$\hat{E}_{\mathbb{Q}}^{te}\leftarrow\frac{1}{N}\Sigma_{i=1}^{N}e^{\alpha\cdot
T_{\psi}(\widetilde{\mathbf{x}}_{1}^{\sigma_{1}^{\mathbf{x}}(i)},\ldots,\widetilde{\mathbf{x}}_{n}^{\sigma_{n}^{\mathbf{x}}(i)},\widetilde{\mathbf{z}}^{\sigma^{\mathbf{z}}(i)})}$

$\widehat{I}_{\alpha}^{t}\leftarrow 1/\\{\alpha\cdot(1-\alpha)\\}$

$\qquad\qquad\qquad-\
\hat{E}_{\mathbb{Q}}^{te}/\alpha\,-\,\hat{E}_{\mathbb{P}_{0}}^{te}/(1-\alpha)$

$\widehat{\mathbf{I}}_{\alpha}\leftarrow\widehat{\mathbf{I}}_{\alpha}\cup\\{\widehat{I}_{\alpha}^{t}\\}$

end for

$\widehat{I}_{\alpha}\leftarrow\max_{t}\widehat{\mathbf{I}}_{\alpha}$

##  8 Limitations: Sample Size Requirements.

In Section 7.1, we noted that our method has a curse of dimensionality. The
sample size requirement of the proposed method is
$N>\left(\frac{1}{\varepsilon}\right)^{d+\delta}$ for
$W_{1}({Q},\hat{{Q}}^{(N)}_{K_{0}})<\varepsilon$ (Corollary C.25 in Appendix
C). However, the curse of dimensionality is an essential problem when
balancing multivariate data owing to the following factors. Because the
optimal balancing weights defined as Eq. (8) for (finite) observational data
are the density ratios of the empirical distributions, the distribution of the
data balanced by them is the empirical distribution. Subsequently, owing to
the balancing of the weights, the curse of dimensionality of the empirical
distribution occurs, which is the same as that described in Section 7.1.
Therefore, to achieve high generalization performance, we need to obtain
weights that differ from the ideal density ratio between the source and target
of the empirical distribution. Further research is required to address this
problem. In Appendix D.3, we present the numerical examination results in
which the causal effects of joint and multidimensional interventions were
estimated with different sample sizes.

##  9 Conclusion

We propose generalized balancing weights to estimate the causal effects of an
arbitrary mixture of discrete and continuous interventions. Three methods for
training the weights were provided: an optimization method to learn the
weights, a method to improve the generalization performance of the balancing
weights, and a method to measure the performance of the weights. We showed the
sample size requirements for the weights and then discussed the curse of
dimensionality that occurs as a general problem when balancing
multidimensional data. Although the curse of dimensionality remains in our
method, we believe that this study provides a basic approach for estimating
the balancing weights of multidimensional data using variational
$f$-divergence.

## References

  * Acuna et al. [2021] David Acuna, Guojun Zhang, Marc T Law, and Sanja Fidler.  f-domain adversarial learning: Theory and algorithms.  In _International Conference on Machine Learning_ , pages 66–75. PMLR, 2021.
  * Arjovsky and Bottou [2017] Martin Arjovsky and Léon Bottou.  Towards principled methods for training generative adversarial networks.  _arXiv preprint arXiv:1701.04862_ , 2017.
  * Belghazi et al. [2018] Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm.  Mutual information neural estimation.  In _International conference on machine learning_ , pages 531–540. PMLR, 2018.
  * Bellemare et al. [2017] Marc G Bellemare, Ivo Danihelka, Will Dabney, Shakir Mohamed, Balaji Lakshminarayanan, Stephan Hoyer, and Rémi Munos.  The cramer distance as a solution to biased wasserstein gradients.  _arXiv preprint arXiv:1705.10743_ , 2017.
  * Birrell et al. [2022] Jeremiah Birrell, Markos A Katsoulakis, and Yannis Pantazis.  Optimizing variational representations of divergences and accelerating their statistical estimation.  _IEEE Transactions on Information Theory_ , 2022.
  * Ganin et al. [2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  _The journal of machine learning research_ , 17(1):2096–2030, 2016.
  * Gilardoni [2010] Gustavo L Gilardoni.  On pinsker’s and vajda’s type inequalities for csiszár’s $f$-divergences.  _IEEE Transactions on Information Theory_ , 56(11):5377–5386, 2010.
  * Hainmueller [2012] Jens Hainmueller.  Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies.  _Political analysis_ , 20(1):25–46, 2012.
  * Imai and Ratkovic [2014] Kosuke Imai and Marc Ratkovic.  Covariate balancing propensity score.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 76(1):243–263, 2014.
  * Imai and Van Dyk [2004] Kosuke Imai and David A Van Dyk.  Causal inference with general treatment regimes: Generalizing the propensity score.  _Journal of the American Statistical Association_ , 99(467):854–866, 2004.
  * Kato and Teshima [2021] Masahiro Kato and Takeshi Teshima.  Non-negative bregman divergence minimization for deep direct density ratio estimation.  In _International Conference on Machine Learning_ , pages 5320–5333. PMLR, 2021.
  * Lacoste-Julien et al. [2012] Simon Lacoste-Julien, Mark Schmidt, and Francis Bach.  A simpler approach to obtaining an o (1/t) convergence rate for the projected stochastic subgradient method.  _arXiv preprint arXiv:1212.2002_ , 2012.
  * Lee et al. [2022] Seong-ho Lee, Yanyuan Ma, and Xavier de Luna.  Covariate balancing for causal inference on categorical and continuous treatments.  _Econometrics and Statistics_ , 2022.
  * Li et al. [2018] Fan Li, Kari Lock Morgan, and Alan M Zaslavsky.  Balancing covariates via propensity score weighting.  _Journal of the American Statistical Association_ , 113(521):390–400, 2018.
  * McAllester and Stratos [2020] David McAllester and Karl Stratos.  Formal limitations on the measurement of mutual information.  In _International Conference on Artificial Intelligence and Statistics_ , pages 875–884. PMLR, 2020.
  * Nguyen et al. [2007] XuanLong Nguyen, Martin J Wainwright, and Michael Jordan.  Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization.  _Advances in neural information processing systems_ , 20, 2007.
  * Nguyen et al. [2010] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan.  Estimating divergence functionals and the likelihood ratio by convex risk minimization.  _IEEE Transactions on Information Theory_ , 56(11):5847–5861, 2010.
  * Nowozin et al. [2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka.  f-gan: Training generative neural samplers using variational divergence minimization.  _Advances in neural information processing systems_ , 29, 2016.
  * Oord et al. [2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  _arXiv preprint arXiv:1807.03748_ , 2018.
  * Pearl [1995] Judea Pearl.  Causal diagrams for empirical research.  _Biometrika_ , 82(4):669–688, 1995.
  * Pearl [2009] Judea Pearl.  Causality: Models, reasoning and inference, 2009.
  * Poole et al. [2019] Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker.  On variational bounds of mutual information.  In _International Conference on Machine Learning_ , pages 5171–5180. PMLR, 2019.
  * Robins et al. [1994] James M Robins, Andrea Rotnitzky, and Lue Ping Zhao.  Estimation of regression coefficients when some regressors are not always observed.  _Journal of the American statistical Association_ , 89(427):846–866, 1994.
  * Robins et al. [2000] James M Robins, Miguel Angel Hernan, and Babette Brumback.  Marginal structural models and causal inference in epidemiology, 2000\.
  * Rosenbaum and Rubin [1984] Paul R Rosenbaum and Donald B Rubin.  Reducing bias in observational studies using subclassification on the propensity score.  _Journal of the American statistical Association_ , 79(387):516–524, 1984.
  * Shiryaev [1995] Albert Nikolaevich Shiryaev.  Probability, 1995.
  * Shpitser and Pearl [2012] Ilya Shpitser and Judea Pearl.  Identification of conditional interventional distributions.  _arXiv preprint arXiv:1206.6876_ , 2012.
  * Si et al. [2009] Si Si, Dacheng Tao, and Bo Geng.  Bregman divergence-based regularization for transfer subspace learning.  _IEEE Transactions on Knowledge and Data Engineering_ , 22(7):929–942, 2009.
  * Song and Ermon [2019] Jiaming Song and Stefano Ermon.  Understanding the limitations of variational mutual information estimators.  _arXiv preprint arXiv:1910.06222_ , 2019.
  * Sugiyama et al. [2012] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori.  Density-ratio matching under the bregman divergence: a unified framework of density-ratio estimation.  _Annals of the Institute of Statistical Mathematics_ , 64(5):1009–1044, 2012.
  * Tübbicke [2022] Stefan Tübbicke.  Entropy balancing for continuous treatments.  _Journal of Econometric Methods_ , 11(1):71–89, 2022.
  * Uehara et al. [2016] Masatoshi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, and Yutaka Matsuo.  Generative adversarial nets from a density ratio estimation perspective.  _arXiv preprint arXiv:1610.02920_ , 2016.
  * Uppal et al. [2019] Ananya Uppal, Shashank Singh, and Barnabás Póczos.  Nonparametric density estimation & convergence rates for gans under besov ipm losses.  _Advances in neural information processing systems_ , 32, 2019.
  * Vegetabile et al. [2021] Brian G Vegetabile, Beth Ann Griffin, Donna L Coffman, Matthew Cefalu, Michael W Robbins, and Daniel F McCaffrey.  Nonparametric estimation of population average dose-response curves using entropy balancing weights for continuous exposures.  _Health Services and Outcomes Research Methodology_ , 21(1):69–110, 2021.
  * Weed and Bach [2019] Jonathan Weed and Francis Bach.  Sharp asymptotic and finite-sample rates of convergence of empirical measures in wasserstein distance.  2019\.
  * Yang and Weinan [2022] Hongkang Yang and E Weinan.  Generalization and memorization: The bias potential model.  In _Mathematical and Scientific Machine Learning_ , pages 1013–1043. PMLR, 2022.
  * Zhang et al. [2019] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan.  Bridging theory and algorithm for domain adaptation.  In _International Conference on Machine Learning_ , pages 7404–7413. PMLR, 2019.

##  Appendix A Organization of the supplementary

The organization of this supplementary document is as follows: In Section B,
we lists definitions of the notations used in the proofs; Section C presents
the theorems and propositions referenced in this study and their proofs; In
Section D, the numerical experimental results related to the discussion in
this study are provided; In Section E, the backpropagation algorithm referred
to in Section 6 is presented. In addition to this material, the code used in
the numerical experiments is also submitted as supplementary material.

##  Appendix B Notations

Table 1 lists the notations and definitions used in the proofs of Section C.

Table 1: Notations and definitions used in the proofs Notations | Definitions, Meanings
---|---
$\mathbf{1}(\cdot)$ | A propositional function: $\mathbf{1}(\textit{cond})=1$ if cond is true, and
|  $\mathbf{1}(\textit{cond})=0$ otherwise.
$id_{A}$ | The identity function of a set $A$: $id_{A}(x)=1$ if $x\in A$, and
|  $id_{A}(x)=0$ otherwise.
$\|\cdot\|$ | the Euclidean norm.
$O_{a}(x)$ | A term such that $\lim_{x\rightarrow 0}O(x)/x=C_{a}<\infty$, where $C_{a}$ is a scalar value
| determined by $a$.
$O(x)$ | A term such that $\lim_{x\rightarrow 0}O(x)/x<C$, where $C$ is constant.
$f\lesssim g$ | A relationship between two functions $f$ and $g$ such that
|  $\lim\sup_{n\rightarrow\infty}f(n)/g(n)<\infty$.
$P\ll Q$ |  $P$ is absolutely continuous with respect to $Q$.
$P$, $Q$ | A pair of probability measures with $P\ll Q$ and $Q\ll P$.
$\mu$ | A probability measure with $P\ll\mu$ and $Q\ll\mu$.
$\frac{dP}{dQ}$ | The Radon–Nikodým derivative of $P$ with respect to $Q$.
| When $\frac{dQ}{d\mu}(\mathbf{x})=0$, this is defined as
$\frac{dP}{dQ}(\mathbf{x})=0$.
$\mathbf{X}$ | A random variable with a probability distribution $\mu$.
$\mathbf{X}_{\sim P}$ | A random variable obtained from $\mathbf{X}$ by changing the probability
| distributions from $\mu$ to $P$: $P(\mathbf{X}_{\sim
P}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X}\leq\mathbf{x})$,
$\forall\mathbf{x}\in\mathbb{R}^{d}$.
| Intuitively, an observed value of $\mathbf{X}$ in $P$.
$\mathbf{X}_{\sim Q}$ | A random variable obtained from $\mathbf{X}$ by changing the probability
| distributions from $\mu$ to $Q$: $Q(\mathbf{X}_{\sim
Q}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X}\leq\mathbf{x})$,
$\forall\mathbf{x}\in\mathbb{R}^{d}$.
| Intuitively, an observed value of $\mathbf{X}$ in $Q$.
$\mathbf{X}^{(N)}$ |  $N$ i.i.d. random variables from $\mu$: $\mathbf{X}^{(N)}=\\{\mathbf{X}^{1},\mathbf{X}^{2},\ldots,\mathbf{X}^{N}\\}$, $\mathbf{X}^{i}\overset{\mathrm{iid}}{\sim}\mu$.
$\mathbf{X}_{P}^{(N)}$ | Random variables obtained from $\mathbf{X}^{(N)}$ by changing the probability
| distributions from $\mu$ to $P$:
$\mathbf{X}_{P}^{(N)}=\allowbreak\\{\mathbf{X}^{1}_{\sim
P},\allowbreak\mathbf{X}^{2}_{\sim P},\ldots,\mathbf{X}^{N}_{\sim P}\\}$,
|  $P(\mathbf{X}_{\sim
P}^{i}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X^{i}}\leq\mathbf{x})$,
$\forall\mathbf{x}\in\mathbb{R}^{d}$, ($1\leq i\leq N$).
| Intuitively, observed values of $\mathbf{X}^{(N)}$ in $P$.
$\mathbf{X}_{Q}^{(N)}$ | Random variables obtained from $\mathbf{X}^{(N)}$ by changing the probability
| distributions from $\mu$ to $Q$:
$\mathbf{X}_{Q}^{(N)}=\allowbreak\\{\mathbf{X}^{1}_{\sim
Q},\allowbreak\mathbf{X}^{2}_{\sim Q},\ldots,\mathbf{X}^{N}_{\sim Q}\\}$,
|  $Q(\mathbf{X}_{\sim
Q}^{i}\leq\mathbf{x})=\allowbreak\mu(\mathbf{X^{i}}\leq\mathbf{x})$,
$\forall\mathbf{x}\in\mathbb{R}^{d}$, ($1\leq i\leq N$).
| Intuitively, observed values of $\mathbf{X}^{(N)}$ in $Q$.
$\hat{P}^{(N)}$ | The (empirical) distributions of $\mathbf{X}_{P}^{(N)}$: $\hat{P}^{(N)}(x)=\frac{1}{N}\sum_{i}\mathbf{1}(\mathbf{X}^{i}_{\sim P}=x)$.
$\hat{Q}^{(N)}$ | The (empirical) distributions of $\mathbf{X}_{Q}^{(N)}$: $\hat{Q}^{(N)}(x)=\frac{1}{N}\sum_{i}\mathbf{1}(\mathbf{X}^{i}_{\sim Q}=x)$.
$\mathcal{T}^{\alpha}$ | The set of functions defined in Theorem 6.1.
$\mathbf{U}=\\{U_{1},\dots,U_{m}\\}$ | Unobserved random variables.
$\mathbf{V}=\\{V_{1},\dots,V_{n}\\}$ | Observed random variables.
$\mathcal{X}_{\mathbf{A}}$ | The domain of variables $\mathbf{A}$.
$G=G_{\mathbf{V}\mathbf{U}}$ | The causal graph for $\mathbf{V}\cup\mathbf{U}$.
$Pa(\mathbf{A})_{G}$ | All the parents of the observed variables in $G$ for for $\mathbf{A}\subset\mathbf{V}$.
$Ch(\mathbf{A})_{G}$ | All the children of the observed variables in $G$ for for $\mathbf{A}\subset\mathbf{V}$.
$An(\mathbf{A})_{G}$ | All the ancestors of the observed variables in $G$ for for $\mathbf{A}\subset\mathbf{V}$.
$De(\mathbf{A})_{G}$ | All the descendants of the observed variables in $G$ for for $\mathbf{A}\subset\mathbf{V}$.
$W_{p}$ | The Wasserstein distance of order $p$.

##  Appendix C Proofs

###  C.1 Proofs for Section 5

In this Section, we provide propositions and their proofs, as referred to in
Section 5.

######  Lemma C.1.

A variational representation of $\alpha$-divergence is given as

$\displaystyle D_{\alpha}(Q||P)$ $\displaystyle=$ $\displaystyle\sup_{\phi\geq
0}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[\phi^{-\alpha}\right]-\frac{1}{1-\alpha}E_{P}\left[\phi^{1-\alpha}\right]\right\\},$
(27)

where supremum is considered over all measurable functions with
$E_{P}[\phi^{1-\alpha}]<\infty$ and $E_{Q}[\phi^{-\alpha}]<\infty$. The
maximum value is achieved at $\phi=dQ/dP$.

###### proof of Lemma C.1.

Let $f_{\alpha}(t)=\\{t^{1-\alpha}-(1-\alpha)\cdot
t-\alpha\\}/\\{\alpha(\alpha-1)\\}$ for $\alpha\neq 0,1$, then

$\displaystyle E_{P}\left[f_{\alpha}\left(\frac{dQ}{dP}\right)\right]$
$\displaystyle=$ $\displaystyle
E_{P}\left[\frac{1}{\alpha(\alpha-1)}\left(\frac{dQ}{dP}\right)^{1-\alpha}+\frac{1}{\alpha}\left(\frac{dQ}{dP}\right)+\frac{1}{1-\alpha}\right]$
(28) $\displaystyle=$
$\displaystyle\frac{1}{\alpha(\alpha-1)}E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\right]+\frac{1}{\alpha}+\frac{1}{1-\alpha}$
$\displaystyle=$ $\displaystyle D_{\alpha}(Q||P).$

Note that, the Legendre transform for
$g_{\alpha}(x)=\mathbf{x}^{1-\alpha}/(1-\alpha)$ is obatined as

$g_{\alpha}^{*}(x)=\frac{\alpha}{\alpha-1}x^{1-\frac{1}{\alpha}}.$ (29)

In addition, note that, for the Legendre transforms of any fuction $h(x)$, it
hold that

$\\{C\cdot h(x)\\}^{*}=C\cdot
h^{*}\left(\frac{x}{C}\right)\quad\text{and}\quad\\{h(x)+C\cdot
t+D\\}^{*}=h^{*}(x-C)-D.$ (30)

Here, $A^{*}$ denotes the the Legendre transform of $A$.

From (29) and (30), we have

$\displaystyle f_{\alpha}^{*}(t)$ $\displaystyle=$
$\displaystyle\left\\{\frac{1}{(-\alpha)}g_{\alpha}(t)+\frac{1}{\alpha}t+\frac{1}{1-\alpha}\right\\}^{*}$
(31) $\displaystyle=$
$\displaystyle\frac{1}{(-\alpha)}g_{\alpha}^{*}\left(-\alpha\cdot\left\\{t-\frac{1}{\alpha}\right\\}\right)-\frac{1}{1-\alpha}$
$\displaystyle=$ $\displaystyle-\frac{1}{\alpha}g_{\alpha}^{*}\left(1-\alpha
t\right)+\frac{1}{\alpha-1}$ $\displaystyle=$
$\displaystyle-\frac{1}{\alpha}\left\\{\frac{\alpha}{\alpha-1}\left(1-\alpha
t\right)^{1-\frac{1}{\alpha}}\right\\}+\frac{1}{\alpha-1}$ $\displaystyle=$
$\displaystyle\frac{1}{1-\alpha}\left(1-\alpha
t\right)^{1-\frac{1}{\alpha}}+\frac{1}{\alpha-1}.$

By differentiating $f_{\alpha}(t)$, we obtain

$f_{\alpha}^{\prime}(t)=-\frac{1}{\alpha}t^{-\alpha}+\frac{1}{\alpha}.$ (32)

Thus, we have

$\displaystyle
E_{Q}\left[f_{\alpha}^{\prime}(\phi)\right]=E_{Q}\left[-\frac{1}{\alpha}\phi^{-\alpha}+\frac{1}{\alpha}\right].$
(33)

From (31) and (32), we obtain

$\displaystyle E_{P}\left[f_{\alpha}^{*}(f_{\alpha}^{\prime}(\phi))\right]$
$\displaystyle=$ $\displaystyle
E_{P}\left[\frac{1}{1-\alpha}\left\\{1-\alpha\cdot\left(-\frac{1}{\alpha}\phi^{-\alpha}+\frac{1}{\alpha}\right)\right\\}^{1-\frac{1}{\alpha}}+\frac{1}{\alpha-1}\right]$
(34) $\displaystyle=$ $\displaystyle
E_{P}\left[\frac{1}{1-\alpha}\phi^{1-\alpha}+\frac{1}{\alpha-1}\right].$

In additionm, from (33) and (34), we see for both
$E_{P}[\phi^{1-\alpha}]<\infty$ and $E_{Q}[\phi^{-\alpha}]<\infty$ to hold is
equivalent for both
$E_{P}\left[f_{\alpha}^{*}(f_{\alpha}^{\prime}(\phi))\right]<\infty$ and
$E_{Q}\left[f_{\alpha}^{\prime}(\phi)\right]<\infty$ to hold.

Finally, substituting (33) and (34) for (10), we have

$\displaystyle D_{\alpha}(Q||P)$ $\displaystyle=$ $\displaystyle\sup_{\phi\geq
0}\\{E_{Q}[f_{\alpha}^{\prime}(\phi)]-E_{P}[f_{\alpha}^{*}(f_{\alpha}^{\prime}(\phi))]\\}$
$\displaystyle=$ $\displaystyle\sup_{\phi\geq
0}\left\\{E_{Q}\left[-\frac{1}{\alpha}\phi^{-\alpha}+\frac{1}{\alpha}\right]-E_{P}\left[\frac{1}{1-\alpha}\phi^{1-\alpha}+\frac{1}{\alpha-1}\right]\right\\}$
$\displaystyle=$ $\displaystyle\sup_{\phi\geq
0}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[\phi^{-\alpha}\right]-\frac{1}{1-\alpha}E_{P}\left[\phi^{1-\alpha}\right]\right\\}.$

This completes the proof. ∎

######  Proposition C.2.

$\alpha$-divergence can be written as follows:

$D_{\alpha}(Q||P)=\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot
T}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot T}\right]\right\\},$
(35)

where supremum is considered over all measurable functions
$T:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with $E_{P}[e^{(\alpha-1)\cdot
T}]<\infty$ and $E_{Q}[e^{\alpha\cdot T}]<\infty$. The equality holds for
$T^{*}$ satisfying

$\frac{dQ}{dP}=e^{-T^{*}}.$ (36)

###### proof of Proposition C.2.

Substituting $e^{-T}$ for $\phi$ in (27), we have

$\displaystyle D_{\alpha}(Q||P)$ $\displaystyle=$ $\displaystyle\sup_{\phi\geq
0}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[\phi^{-\alpha}\right]-\frac{1}{1-\alpha}E_{P}\left[\phi^{1-\alpha}\right]\right\\}$
(37) $\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[\left\\{e^{-T}\right\\}^{-\alpha}\right]-\frac{1}{1-\alpha}E_{P}\left[\left\\{e^{-T}\right\\}^{1-\alpha}\right]\right\\}$
$\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot
T}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot T}\right]\right\\}.$

Finally, from Lemma C.1, the equality for (37) holds if and only if

$\frac{dQ}{dP}=e^{-T^{*}}.$ (38)

This completes the proof. ∎

######  Proposition C.3.

For $T\in\mathcal{T}^{\alpha}$, let

$\displaystyle l_{\alpha}\left(\mathbf{X}_{\sim Q},\mathbf{X}_{\sim
P};T\right)$ $\displaystyle=$ $\displaystyle\frac{1}{\alpha}e^{\alpha\cdot
T(\mathbf{X}_{\sim Q})}+\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T(\mathbf{X}_{\sim P})}.$ (39)

Then the optimal function $T^{*}$ for
$\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}\left(\mathbf{X}_{\sim
Q},\mathbf{X}_{\sim P};T\right)$ is obtained as $T^{*}=-\log dQ/dP$,
$\mu$-almost everywhere.

###### proof of Proposition C.3.

From the definition for $\mathbf{X}_{\sim Q}$ and $\mathbf{X}_{\sim P}$, we
see

$e^{\alpha\cdot T(\mathbf{X}_{\sim Q}=\mathbf{x})}=e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x}),$

and

$e^{(\alpha-1)\cdot T(\mathbf{X}_{\sim P}=\mathbf{x})}=e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x}).$

Subsequently, we obtain

$l_{\alpha}(\mathbf{X}_{\sim Q}=\mathbf{x},\mathbf{X}_{\sim
P}=\mathbf{x};T)=\frac{1}{\alpha}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})+\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x}).$

Note that, from Jensen’s inequality, it holds that

$\log(p\cdot X+q\cdot Y)\geq p\cdot\log(X)+p\cdot\log(Y),$ (40)

for $X,Y>0$ and $p,q>0$ with $p+q=1$, and the equality holds when $X=Y$.

From this equation, by letting $X=e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})$,
$Y=e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x})$, $p=1-\alpha$ and
$q=\alpha$, we observe that

$\log(p\cdot X+q\cdot Y)=\log\left(\frac{1}{\alpha\cdot(1-\alpha)}\cdot
l_{\alpha}(\mathbf{X}_{\sim Q}=\mathbf{x},\mathbf{X}_{\sim
P}=\mathbf{x};T)\right),$

and $\log\left(\frac{1}{\alpha\cdot(1-\alpha)}\cdot
l_{\alpha}(\mathbf{X}_{\sim Q}=\mathbf{x},\mathbf{X}_{\sim
P}=\mathbf{x};T)\right)$ is minimized when $e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})=e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x})$, $\mu$-almost
everywhere.

Then, we see that
$\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}_{\sim
Q}=\mathbf{x},\mathbf{X}_{\sim P}=\mathbf{x};T)$ is achieved at
$e^{-T^{*}(\mathbf{x})}=\frac{dQ}{dP}$, $\mu$-almost everywhere. Hence, we
have $T^{*}=-\log dQ/dP$, $\mu$-almost everywhere.

∎

######  Proposition C.4.

For $T\in\mathcal{T}^{\alpha}$, let $T^{+k}=T+k$. Then the optimal function
$T^{*}$ for $\inf_{k\in\mathbb{R}}l_{\alpha}\left(\mathbf{X}_{\sim
Q},\mathbf{X}_{\sim P};T^{+k}\right)$ is satisfying that
$E_{P}\left[e^{-T^{*}}\right]=1$, where $l_{\alpha}\left(\mathbf{X}_{\sim
Q},\mathbf{X}_{\sim P};T\right)$ is defined as (39).

###### proof of Proposition C.4.

From the definition for $\mathbf{X}_{\sim Q}$ and $\mathbf{X}_{\sim P}$, we
see

$e^{\alpha\cdot T^{+k}(\mathbf{X}_{\sim Q}=\mathbf{x})}=e^{\alpha\cdot
T^{+k}(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})=e^{\alpha\cdot
k}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x}),$

and

$e^{(\alpha-1)\cdot T(\mathbf{X}_{\sim P}=\mathbf{x})}=e^{(\alpha-1)\cdot
T^{+k}(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x})=e^{(\alpha-1)\cdot
k}\cdot e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x}).$

Subsequently, we obtain

$l_{\alpha}(\mathbf{X}_{\sim Q}=\mathbf{x},\mathbf{X}_{\sim
P}=\mathbf{x};T)=e^{\alpha\cdot k}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})+e^{\alpha\cdot
k}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x}).$

For Jensen’s inequality (40), let $X=e^{\alpha\cdot k}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})$,
$Y=e^{(\alpha-1)\cdot k}\cdot e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x})$, $p=1-\alpha$ and
$q=\alpha$. Then, $l_{\alpha}\left(\mathbf{X}_{\sim Q},\mathbf{X}_{\sim
P};T^{+k}\right)$ is minimized when $e^{\alpha\cdot k_{*}}\cdot e^{\alpha\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dQ}{d\mu}(\mathbf{x})=e^{(\alpha-1)\cdot
k_{*}}\cdot e^{(\alpha-1)\cdot
T(\mathbf{X}=\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x})$, $\mu$-almost
everywhere.

Then, we see

$e^{-k_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})=e^{-T(\mathbf{x})}\cdot\frac{dP}{d\mu}(\mathbf{x}).$

By integrating both sides of the above equality over $\mathbb{R}^{d}$ with
$\mu$, we obtain

$e^{-k_{*}}=E_{P}\left[e^{-T}\right].$

From this, we have

$e^{k_{*}}\cdot E_{P}\left[e^{-T}\right]=1.$

Now, since $T_{*}=T+k_{*}$, we see

$E_{P}\left[e^{-T_{*}}\right]=E_{P}\left[e^{-T+k_{*}}\right]=e^{k_{*}}\cdot
E_{P}\left[e^{-T}\right]=1.$

This completes the proof. ∎

######  Proposition C.5.

For a fixed point $\mathbf{x}_{0}\in\mathbb{R}^{d}$, suppose that
$\frac{dQ}{d\mu}(\mathbf{x}_{0})>0$ and $\frac{dP}{d\mu}(\mathbf{x}_{0})>0$.
For a constant $L>0$, let $I_{L}$ denote an interval
$[-L-\log\frac{dQ}{dP}(\mathbf{x}_{0}),L-\log\frac{dQ}{dP}(\mathbf{x}_{0})]$.
Subsequently, let $f:I_{L}\rightarrow\mathbb{R}$ be a function as follows:

$f(t)=\frac{1}{\alpha}\cdot e^{\alpha\cdot
t}\cdot\frac{dQ}{d\mu}(\mathbf{x}_{0})+\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot t}\cdot\frac{dP}{d\mu}(\mathbf{x}_{0}).$ (41)

Let
$\frac{\lambda}{2}=e^{-L}\cdot\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{0})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{0})\right\\}^{\alpha}$.
Then, $f(t)$ satisfies $f^{\prime\prime}(t)\cdot
t^{2}\geq\frac{\lambda}{2}\cdot t^{2}$ for all $t\in I_{L}$, that is, $f(t)$
is $\lambda$-strongly convex. In addition, $\left|f^{\prime}(t)\right|\leq
2\cdot
e^{L}\cdot\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{0})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{0})\right\\}^{\alpha}$
holds for all $t\in I_{L}$, and $f(t)$ is minimized at
$t_{*}=-\log\frac{dQ}{dP}(\mathbf{x}_{0})$.

###### proof of Proposition C.5.

By repeating the derivative of $f(t)$, we obtain

$f^{\prime}(t)=e^{\alpha\cdot
t}\cdot\frac{dQ}{d\mu}(\mathbf{x})-e^{(\alpha-1)\cdot
t}\cdot\frac{dP}{d\mu}(\mathbf{x}),$ (42)

and

$f^{\prime\prime}(t)=\alpha\cdot e^{\alpha\cdot
t}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot e^{(\alpha-1)\cdot
t}\cdot\frac{dP}{d\mu}(\mathbf{x}).$ (43)

First, we see that $f^{\prime\prime}(t)\geq\frac{\lambda}{2}$ holds for all
$t\in I_{L}$. From (43), we have

$\displaystyle f^{\prime\prime}(t)$ $\displaystyle\geq$
$\displaystyle\alpha\cdot
e^{\alpha\cdot(t_{*}-L)}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot
e^{(\alpha-1)\cdot(t_{*}+L)}\cdot\frac{dP}{d\mu}(\mathbf{x})$ (44)
$\displaystyle\geq$ $\displaystyle\alpha\cdot e^{-\alpha\cdot L}\cdot
e^{\alpha\cdot t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot
e^{(\alpha-1)\cdot L}\cdot e^{(\alpha-1)\cdot
t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})$ $\displaystyle\geq$
$\displaystyle\alpha\cdot e^{-L}\cdot e^{\alpha\cdot
t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot e^{-L}\cdot
e^{(\alpha-1)\cdot t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})$ $\displaystyle=$
$\displaystyle e^{-L}\cdot\left\\{\alpha\cdot e^{\alpha\cdot
t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot e^{(\alpha-1)\cdot
t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})\right\\}.$

Note that, from (40), we see

$p\cdot X+q\cdot Y\geq X^{p}\cdot Y^{q},$ (45)

for $X,Y>0$ and $p,q>0$ with $p+q=1$, and the equality holds when $X=Y$. By
letting $X=e^{\alpha\cdot t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})$,
$Y=e^{(\alpha-1)\cdot t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})$, $p=\alpha$, and
$q=1-\alpha$ in the above equality, we obtain

$\displaystyle\alpha\cdot e^{\alpha\cdot
t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})+(1-\alpha)\cdot e^{(\alpha-1)\cdot
t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})$ $\displaystyle\geq$
$\displaystyle\left\\{e^{\alpha\cdot
t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{\alpha}\left\\{e^{(\alpha-1)\cdot
t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}$ (46)
$\displaystyle=$ $\displaystyle
e^{\left\\{\alpha^{2}-(1-\alpha)^{2}\right\\}\cdot
t_{*}}\left\\{\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}$
$\displaystyle=$
$\displaystyle\left\\{\frac{dQ}{dP}(\mathbf{x})\right\\}^{1-2\alpha}\left\\{\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}$
$\displaystyle=$
$\displaystyle\left\\{\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x})\right\\}^{\alpha}.$

Thus, from (44) and (46), we see $f^{\prime\prime}(t)\geq\frac{\lambda}{2}$
holds for all $t\in I_{L}$.

Next, we obtain $\left|f^{\prime}(t)\right|\leq
2e^{L}\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{0})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{0})\right\\}^{\alpha}$.
From (42), we have

$\displaystyle\left|f^{\prime}(t)\right|$ $\displaystyle\leq$ $\displaystyle
e^{\alpha\cdot(t_{*}+L)}\cdot\frac{dQ}{d\mu}(\mathbf{x})+e^{(\alpha-1)\cdot(t_{*}-L)}\cdot\frac{dP}{d\mu}(\mathbf{x})$
(47) $\displaystyle=$ $\displaystyle e^{\alpha\cdot t_{*}}\cdot e^{\alpha\cdot
L}\cdot\frac{dQ}{d\mu}(\mathbf{x})+e^{(\alpha-1)\cdot t_{*}}\cdot
e^{(1-\alpha)\cdot L}\cdot\frac{dP}{d\mu}(\mathbf{x})$ $\displaystyle\leq$
$\displaystyle e^{L}\left\\{e^{\alpha\cdot
t_{*}}\cdot\frac{dQ}{d\mu}(\mathbf{x})+e^{(\alpha-1)\cdot
t_{*}}\cdot\frac{dP}{d\mu}(\mathbf{x})\right\\}$ $\displaystyle=$
$\displaystyle
e^{L}\left[\left\\{\frac{dQ}{dP}(\mathbf{x})\right\\}^{-\alpha}\frac{dQ}{d\mu}(\mathbf{x})+\left\\{\frac{dQ}{dP}(\mathbf{x})\right\\}^{1-\alpha}\frac{dP}{d\mu}(\mathbf{x})\right]$
$\displaystyle=$ $\displaystyle
e^{L}\left[\left\\{\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x})\right\\}^{\alpha}+\left\\{\frac{dQ}{d\mu}(\mathbf{x})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x})\right\\}^{\alpha}\right].$

Here, we see $f^{\prime}(t)\leq 2\cdot
e^{L}\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{0})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{0})\right\\}^{\alpha}$.

The rest of the proposition statement follows from Lemma C.3.

This completes the proof. ∎

######  Corollary C.6.

For $N$ fixed points
$\left\\{\mathbf{x}_{i}\right\\}_{i=1}^{N}\subset\mathbb{R}^{d}$, suppose that
$\frac{dQ}{d\mu}(\mathbf{x}_{i})>0$ and $\frac{dP}{d\mu}(\mathbf{x}_{i})>0$
($1\leq\forall\leq N$). For a constant $L>0$, let $I_{L}^{i}$ denote an
interval
$[-L-\log\frac{dQ}{dP}(\mathbf{x}_{i}),L-\log\frac{dQ}{dP}(\mathbf{x}_{i})]$.
Subsequently, let $f^{(N)}:I_{L}^{1}\times I_{L}^{2}\times\cdots\times
I_{L}^{N}\rightarrow\mathbb{R}$ be a function as follows:

$f^{(N)}(\mathbf{t})=f^{(N)}(t_{1},t_{2},\ldots,t_{N})=\frac{1}{\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
t_{i}}\cdot\frac{dQ}{d\mu}(\mathbf{x}_{i})+\frac{1}{1-\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
t_{i}}\cdot\frac{dP}{d\mu}(\mathbf{x}_{i}),$ (48)

and let

$\frac{\lambda}{2}=\frac{1}{N}\cdot\min_{1\leq i\leq
N}\left\\{e^{-L}\cdot\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{i})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{i})\right\\}^{\alpha}\right\\}.$
(49)

Then, $f^{(N)}(\mathbf{t})$ satisfies

$\mathbf{t}^{T}\cdot\nabla^{2}f^{(N)}(\mathbf{t})\cdot\mathbf{t}=\sum_{1\leq
i,j\leq N}t_{i}\cdot t_{j}\cdot\frac{\partial^{2}f^{(N)}}{\partial
t_{i}t_{j}}\geq\frac{\lambda}{2}\cdot\|\mathbf{t}\|^{2},$ (50)

that is, $f^{(N)}(\mathbf{t})$ is $\lambda$-strongly convex.

In addition, let $D_{i}=\allowbreak 2\cdot
e^{L}\cdot\allowbreak\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{i})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{i})\right\\}^{\alpha}$,
and let $D=\max\left\\{D_{1},\allowbreak D_{2},\allowbreak\ldots,\allowbreak
D_{N}\right\\}^{2}$. Then,

$\left\|\nabla
f^{(N)}(\mathbf{t})\right\|^{2}=\left\|\left(\frac{\partial}{\partial
t_{1}}f^{(N)}(\mathbf{t}),\frac{\partial}{\partial
t_{2}}f^{(N)}(\mathbf{t}),\ldots,\frac{\partial}{\partial
t_{N}}f^{(N)}(\mathbf{t})\right)\right\|^{2}\leq D^{2},$ (51)

for all $\mathbf{t}\allowbreak\in\allowbreak
I_{L}^{1}\allowbreak\times\allowbreak
I_{L}^{2}\allowbreak\times\allowbreak\cdots\allowbreak\times\allowbreak
I_{L}^{N}$, and $f^{(N)}(\mathbf{t})$ is minimized at
$\mathbf{t}_{*}=\allowbreak(t_{*}^{1},\allowbreak
t_{*}^{2},\allowbreak\ldots,\allowbreak
t_{*}^{N})=\allowbreak(-\log\frac{dQ}{dP}(\mathbf{x}_{1}),\allowbreak-\log\frac{dQ}{dP}(\mathbf{x}_{2}),\allowbreak\ldots,\allowbreak-\log\frac{dQ}{dP}(\mathbf{x}_{N}))$.

###### proof of Corollary C.6.

Let

$f_{i}(t)=\frac{1}{\alpha}e^{\alpha\cdot
t}\cdot\frac{dQ}{d\mu}(\mathbf{x}_{i})+\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
t}\cdot\frac{dP}{d\mu}(\mathbf{x}_{i}),$ (52)

and let
$\frac{\lambda_{i}}{2}=e^{-L}\cdot\left\\{\frac{dQ}{d\mu}(\mathbf{x}_{i})\right\\}^{1-\alpha}\left\\{\frac{dP}{d\mu}(\mathbf{x}_{i})\right\\}^{\alpha}$.
From Proposition C.5, for each $1\leq i\leq N$, $f_{i}(t)$ satisfies that
$f^{\prime\prime}_{i}(t)\geq\frac{\lambda_{i}}{2}\cdot t$ and
$f^{\prime}_{i}(t)\leq D_{i}$ holds for all $t\in I_{L}^{i}$, and $f_{i}(t)$
is minimized at $t_{*}^{i}=-\log\frac{dQ}{dP}(\mathbf{x}_{i})$.

Note that,

$\displaystyle\nabla^{2}f^{(N)}(\mathbf{t})$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{\partial^{2}}{\partial
t_{1}t_{1}}f^{(N)}(\mathbf{t})&\frac{\partial^{2}}{\partial
t_{1}t_{2}}f^{(N)}(\mathbf{t})&\cdots&\frac{\partial^{2}}{\partial
t_{1}t_{N}}f^{(N)}(\mathbf{t})\\\ \frac{\partial^{2}}{\partial
t_{2}t_{1}}f^{(N)}(\mathbf{t})&\frac{\partial^{2}}{\partial
t_{2}t_{2}}f^{(N)}(\mathbf{t})&&\\\ \vdots&&\ddots&&\\\
\frac{\partial^{2}}{\partial
t_{N}t_{j}}f^{(N)}(\mathbf{t})&\frac{\partial^{2}}{\partial
t_{i}t_{j}}f^{(N)}(\mathbf{t})&\cdots&\frac{\partial^{2}}{\partial
t_{N}t_{N}}f^{(N)}(\mathbf{t})\end{pmatrix}$ (53) $\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{1}{N}\cdot f^{\prime\prime}_{1}(t_{1})\\\
&\frac{1}{N}\cdot f^{\prime\prime}_{2}(t_{2})&&\text{\huge{0}}\\\ &&\ddots\\\
&\text{\huge{0}}&&\ddots\\\ &&&&\frac{1}{N}\cdot
f^{\prime\prime}_{N}(t_{N})\end{pmatrix}.$

From this, we see

$\displaystyle\mathbf{t}^{T}\cdot\nabla^{2}f^{(N)}(\mathbf{t})\cdot\mathbf{t}$
$\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}f^{\prime\prime}_{i}(t_{i})\cdot
t_{i}^{2}$ (54) $\displaystyle\geq$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}\frac{\lambda_{i}}{2}\cdot t_{i}^{2}$
$\displaystyle=$
$\displaystyle\sum_{i=1}^{N}\frac{1}{N}\cdot\frac{\lambda_{i}}{2}\cdot
t_{i}^{2}$ $\displaystyle\geq$
$\displaystyle\sum_{i=1}^{N}\frac{\lambda}{2}\cdot t_{i}^{2}$ $\displaystyle=$
$\displaystyle\frac{\lambda}{2}\cdot\|\mathbf{t}\|^{2}.$

In addition, since $f^{(N)}(t)=\frac{1}{N}\sum_{i=1}^{N}f_{i}(t_{i})$, we have

$\displaystyle\left\|\nabla f^{(N)}(\mathbf{t})\right\|^{2}$ $\displaystyle=$
$\displaystyle\left\|\left(\frac{\partial}{\partial
t_{1}}f^{(N)}(\mathbf{t}),\frac{\partial}{\partial
t_{2}}f^{(N)}(\mathbf{t}),\ldots,\frac{\partial}{\partial
t_{N}}f^{(N)}(\mathbf{t})\right)\right\|^{2},$ (55) $\displaystyle=$
$\displaystyle\left\|\left(\frac{1}{N}\cdot
f_{1}^{\prime}(t_{1}),\frac{1}{N}\cdot
f_{2}^{\prime}(t_{2}),\ldots,\frac{1}{N}\cdot
f_{N}^{\prime}(t_{N})\right)\right\|^{2}$ $\displaystyle\leq$
$\displaystyle\left\|\left(\frac{1}{N}\cdot D_{1},\frac{1}{N}\cdot
D_{2},\ldots,\frac{1}{N}\cdot D_{N}\right)\right\|^{2}$ $\displaystyle\leq$
$\displaystyle\left\|\left(\frac{1}{N}\cdot D,\frac{1}{N}\cdot
D,\ldots,\frac{1}{N}\cdot D\right)\right\|^{2}$ $\displaystyle=$
$\displaystyle D^{2},$

and $f^{(N)}(\mathbf{t})$ is minimized at
$\mathbf{t}_{*}=\allowbreak(t_{*}^{1},\allowbreak
t_{*}^{2},\allowbreak\ldots,\allowbreak
t_{*}^{N})\allowbreak=\allowbreak(-\log\frac{dQ}{dP}(\mathbf{x}_{1}),\allowbreak-\log\frac{dQ}{dP}(\mathbf{x}_{2}),\allowbreak\ldots,\allowbreak-\log\frac{dQ}{dP}(\mathbf{x}_{N}))$.

This completes the proof. ∎

######  Lemma C.7.

For $T\in\mathcal{T}^{\alpha}$, let

$\displaystyle l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim P};T)$
$\displaystyle=$ $\displaystyle\frac{1}{\alpha}\cdot e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim Q})}+\frac{1}{1-\alpha}\cdot e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})},$ (56) $\displaystyle L_{\alpha}(Q,P;T)$
$\displaystyle=$ $\displaystyle\frac{1}{\alpha}\cdot E_{Q}\left[e^{\alpha\cdot
T(\mathbf{X})}\right]+\frac{1}{1-\alpha}\cdot E_{P}\left[e^{(\alpha-1)\cdot
T(\mathbf{X})}\right],$ (57)

and let

$\displaystyle l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim P})$
$\displaystyle=$
$\displaystyle\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T),$ (58) $\displaystyle L_{\alpha}(Q,P)$
$\displaystyle=$
$\displaystyle\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}L_{\alpha}(Q,P;T),$
(59)

where the infimums of (58) and (59) are considered over all measurable
functions with $T:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with
$E_{P}[e^{(\alpha-1)\cdot T}]<\infty$ and $E_{Q}[e^{\alpha\cdot T}]<\infty$.

In addition, let

$\displaystyle\hat{L}_{\alpha}^{(N)}(Q,P;T)$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim
Q})}+\frac{1}{1-\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})},$ (60) $\displaystyle\hat{L}_{\alpha}^{(N)}(Q,P)$
$\displaystyle=$
$\displaystyle\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\hat{L}_{\alpha}^{(N)}(Q,P;T).$
(61)

Then, it holds that

$E_{\mu}\left[\hat{L}_{\alpha}^{(N)}(Q,P;T)\right]=E_{\mu}\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right]=L_{\alpha}(Q,P;T),$ (62)
$E_{\mu}\left[\hat{L}_{\alpha}^{(N)}(Q,P)\right]=E_{\mu}\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]=L_{\alpha}(Q,P).$ (63)

###### proof of Lemma C.7.

We first show the last equality in (63) holds. Now, we consider the following
integral:

$\displaystyle
E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]$ $\displaystyle=$
$\displaystyle\int\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right\\}d\mu$ $\displaystyle=$
$\displaystyle\int\left\\{\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right\\}d\mu$ $\displaystyle=$
$\displaystyle\int\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right\\}d\mu.$

Let $T^{*}$ be the opimal function for (35) in Lemma C.2. Let $T_{k}=-\log
dQ/dP+1/k$, then from Proposition C.3, we have

$\lim_{k\rightarrow\infty}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{k})=\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T).$ (65)

From this, we obtain

$\displaystyle\lim_{k\rightarrow\infty}E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{k})\right]$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\lim_{k\rightarrow\infty}E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{k})\right]$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)$ $\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\Big{\\{}E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\ \right]\Big{\\}}.$

Now, from Lemma C.2, we see

$\displaystyle\left|\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right|$ $\displaystyle=$
$\displaystyle\left|\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right\\}\right|$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}e^{\alpha\cdot
T^{*}(\mathbf{X}^{i}_{\sim Q})}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T^{*}(\mathbf{X}^{i}_{\sim P})}$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\left(\frac{dQ}{dP}(\mathbf{X}^{i}_{\sim
Q})\right)^{\alpha}-\frac{1}{1-\alpha}\left(\frac{dQ}{dP}(\mathbf{X}^{i}_{\sim
P})\right)^{\alpha-1}.$

Let $\phi(\mathbf{X})$ denote the term on the right hand side of
(LABEL:Lemma_loss_not_biased_lemma_integral_is_finte). Then, we observe that

$\left|\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right|\leq\phi(\mathbf{X})\quad\text{and}\quad
E[\phi(\mathbf{X})]<\infty.$

That is, we see that the following sequence is uniformaly integrable for
$\mu$:

$\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{k})\right\\}_{k=1}^{N}.$

Thus, from the property of the Lebesgue integral (26, P188, Theorem 4), we
obtain

$E\left[\lim_{k\rightarrow\infty}\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{k})\right\\}\right]=\lim_{k\rightarrow\infty}E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{k})\right].$ (68)

Finaly, from (LABEL:Eq_Lemma_loss_not_biased_lemma_N_left_1) and (68), we have

$\displaystyle\frac{1}{\alpha(1-\alpha)}-E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]$ $\displaystyle=$ $\displaystyle
E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]$ (69) $\displaystyle=$ $\displaystyle
E\left[\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\Big{\\{}\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\Big{\\}}\ \right]$ $\displaystyle=$
$\displaystyle
E\left[\lim_{k\rightarrow\infty}\left\\{\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{k})\right\\}\right]$ $\displaystyle=$
$\displaystyle\lim_{k\rightarrow\infty}E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{k})\right]\qquad\therefore(\ref{exchange_integral_sup})$
$\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\Big{\\{}E\left[\frac{1}{\alpha(1-\alpha)}-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\
\right]\Big{\\}}\qquad\therefore(\ref{Eq_Lemma_loss_not_biased_lemma_N_left_1})$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\qquad$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-L_{\alpha}(Q,P).$

Here, we see

$E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim
P})\right]=L_{\alpha}(Q,P).$ (70)

Next, we show the first equality in (63) holds. Note that, it holds that

$\frac{1}{N}\sum_{i=1}^{N}\inf_{T_{i}:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{i})\leq\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T)\leq\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T^{*}).$ (71)

Since
$\inf_{T_{i}:\mathbb{R}^{d}\rightarrow\mathbb{R}}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{i})=l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{*})$ from Proposition C.3, we have

$\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T^{*})\leq\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T)\leq\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T^{*}).$ (72)

Therefore,

$\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T)=\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T^{*}).$ (73)

From this, we see

$\displaystyle\hat{L}_{\alpha}^{(N)}(Q,P)$ $\displaystyle=$
$\displaystyle\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)$ (74) $\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T^{*})$ $\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})$ $\displaystyle=$ $\displaystyle
l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim P}).$

Subsequently, by integrating both sides of the above equation, we have

$E\Big{[}\hat{L}_{\alpha}^{(N)}(Q,P)\Big{]}=E\Big{[}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\Big{]}.$ (75)

Here, we have (63) from (70) and (75).

To see (62), note that, it holds that

$\hat{L}_{\alpha}^{(N)}(Q,P;T)=\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T).$ (76)

By integrating both sides of the above equation, we have

$\displaystyle E\Big{[}\hat{L}_{\alpha}^{(N)}(Q,P;T)\Big{]}$ $\displaystyle=$
$\displaystyle
E\Big{[}\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\Big{]}$ (77) $\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}E\Big{[}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\Big{]}$ $\displaystyle=$ $\displaystyle
E\Big{[}l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim P};T)\Big{]}$
$\displaystyle=$ $\displaystyle L_{\alpha}(Q,P;T).$

Here, we see that (62) holds.

This completes the proof. ∎

######  Proposition C.8.

Let $T_{\theta}(\mathbf{x}):\mathbb{R}^{d}\rightarrow\mathbb{R}$ be a function
such that the map
$\theta=(\theta_{1},\theta_{2},\ldots,\theta_{p})\in\Theta\mapsto
T_{\theta}(\mathbf{x})$ is differentiable for all $\theta$ and $\mu$-almost
every $\mathbf{x}\in\mathbb{R}^{d}$. Assume that, for a point
$\bar{\theta}\in\Theta$, it holds that $E_{P}[e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X})}]<\infty$ and $E_{Q}[e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X})}]<\infty$, and there exist a compact neighborhood
of the $\bar{\theta}$, which is denoted by $B_{\bar{\theta}}$, and a constant
value $L$, such that
$|T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x})|<L\|\psi-\bar{\theta}\|$
holds. Then, for $l_{\alpha}(\mathbf{X}^{i}_{\sim Q}$, $\mathbf{X}^{i}_{\sim
P};T)$ and $\hat{L}_{\alpha}^{(N)}(Q,P;T),L_{\alpha}(Q,P;T)$ in Proposition
C.12, it holds that

$E\left[\nabla_{\theta}\,L_{\alpha}(\hat{Q}^{(N)},\hat{P}^{(N)};T_{\theta})|_{\theta=\bar{\theta}}\right]=E\Big{[}\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{\theta})|_{\theta=\bar{\theta}}\Big{]}=\nabla_{\theta}\,L_{\alpha}(Q,P;T_{\theta})|_{\theta=\bar{\theta}}.$
(78)

Here, $E[\ \cdot\ ]$ denotes $E_{P}[E_{Q}[\ \cdot\ ]]$.

###### proof of Proposition C.8.

We now consider the values, as $\psi\rightarrow\bar{\theta}$, of the following
two integrals:

$\int\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}}-\frac{1}{\alpha}e^{\alpha\cdot T_{\bar{\theta}}}\right\\}dQ,$ (79)

and

$\int\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot T_{\bar{\theta}}}\right\\}dP.$
(80)

Note that, it follows from the intermediate value theorem that

$\left|\frac{1}{\alpha}e^{\alpha\cdot x}-\frac{1}{\alpha}e^{\alpha\cdot
y}\right|=|x-y|\cdot
e^{\alpha\cdot\\{y+\gamma\cdot(x-y)\\}}\quad(\,\exists\gamma\in[0,1]\,).$ (81)

By using the above equation as $x=T_{\psi}(\mathbf{x})$ and
$y=T_{\bar{\theta}}(\mathbf{x})$ for the integrand of (79), we see

$\displaystyle\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{x})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{x})}\right\\}\right|$ (82) $\displaystyle=$
$\displaystyle\frac{1}{\|\psi-\bar{\theta}\|}|T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x})|\cdot
e^{\alpha\cdot\\{T_{\bar{\theta}}(\mathbf{x})+\gamma_{\mathbf{x}}\cdot(T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x}))\\}}\quad\qquad(\,\gamma_{\mathbf{x}}\in[0,1]\,)$
$\displaystyle=$
$\displaystyle\frac{1}{\|\psi-\bar{\theta}\|}|T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x})|\cdot
e^{\alpha\cdot\gamma_{\mathbf{x}}\cdot(T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x}))}\cdot
e^{\alpha\cdot T_{\bar{\theta}}(\mathbf{x})}$ $\displaystyle\leq$
$\displaystyle\frac{1}{\|\psi-\bar{\theta}\|}|T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x})|\cdot
e^{\alpha\gamma_{\mathbf{x}}|T_{\psi}(\mathbf{x})-T_{\bar{\theta}}(\mathbf{x})|}\cdot
e^{\alpha\cdot T_{\bar{\theta}}(\mathbf{x})}$ $\displaystyle\leq$
$\displaystyle L\cdot e^{\alpha L\cdot\|\psi-\bar{\theta}\|}\cdot
e^{\alpha\cdot T_{\bar{\theta}}(\mathbf{x})},$

for all $\psi\in B_{\bar{\theta}}$.

Thus, integrating the term on the left hand side of (82) by $Q$, we see

$\displaystyle\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{X})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dQ$ (83) $\displaystyle\leq$
$\displaystyle\int L\cdot e^{\alpha L\cdot\|\psi-\bar{\theta}\|}\cdot
e^{\alpha\cdot T_{\bar{\theta}}(\mathbf{X})}dQ$ $\displaystyle=$
$\displaystyle L\cdot e^{\alpha
L\cdot\|\psi-\bar{\theta}\|}E_{Q}\left[e^{\alpha\cdot
T_{\bar{\theta}}}\right].$

Considering the supremum for $\psi\in B_{\bar{\theta}}$ of (83), it holds that

$\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\Bigg{\\{}\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}}\right\\}\right|dQ\Bigg{\\}}$ (84) $\displaystyle\leq$
$\displaystyle\sup_{\psi\in B_{\bar{\theta}}}\Big{\\{}L\cdot e^{\alpha
L\cdot\|\psi-\bar{\theta}\|}\,E_{Q}\left[e^{\alpha\cdot
T_{\bar{\theta}}}\right]\Big{\\}}$ $\displaystyle=$ $\displaystyle
E_{Q}\left[e^{\alpha\cdot T_{\bar{\theta}}}\right]\cdot\sup_{\psi\in
B_{\bar{\theta}}}L\cdot e^{\alpha L\cdot\|\psi-\bar{\theta}\|}<\infty,$

since $B_{\bar{\theta}}$ is compact.

Similarly, as for (80), we see

$\displaystyle\sup_{\psi\in B_{\bar{\theta}}}\hbox
to0.0pt{$\displaystyle\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}}(\mathbf{X})-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dP$\hss}$ (85)
$\displaystyle\leq$ $\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\Big{\\{}L\cdot
e^{(1-\alpha)L\cdot\|\psi-\bar{\theta}\|}\,E_{P}\left[e^{(1-\alpha)\cdot
T_{\bar{\theta}}}\right]\Big{\\}}$ $\displaystyle=$ $\displaystyle
E_{P}\left[e^{(1-\alpha)\cdot T_{\bar{\theta}}}\right]\cdot\sup_{\psi\in
B_{\bar{\theta}}}L\cdot e^{(1-\alpha)L\cdot\|\psi-\bar{\theta}\|}<\infty.$

From (84) and (85), we obtain

$\displaystyle\sup_{\psi\in B_{\bar{\theta}}}\hbox
to0.0pt{$\displaystyle\int\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}\right|dP\,dQ$\hss}$ (86)
$\displaystyle=$ $\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\int\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{X}^{i}_{\sim Q})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X}^{i}_{\sim Q})}\right\\}\right.$
$\displaystyle\qquad\qquad\quad+\left.\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}(\mathbf{X}^{i}_{\sim P})}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X}^{i}_{\sim P})}\right\\}\right|dP\,dQ$
$\displaystyle\leq$ $\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\int\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{X}^{i}_{\sim Q})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X}^{i}_{\sim Q})}\right\\}\right|$
$\displaystyle\qquad\quad\quad+\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}(\mathbf{X}^{i}_{\sim P})}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X}^{i}_{\sim P})}\right\\}\right|dP\,dQ$
$\displaystyle=$ $\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\left\\{\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{X})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dQ\right.$
$\displaystyle\qquad\quad\quad+\left.\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}(\mathbf{X})}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dP\right\\}$ $\displaystyle\leq$
$\displaystyle\sup_{\psi\in
B_{\bar{\theta}}}\left\\{\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{\alpha}e^{\alpha\cdot
T_{\psi}(\mathbf{X})}-\frac{1}{\alpha}e^{\alpha\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dQ\right\\}$
$\displaystyle\qquad+\sup_{\psi\in
B_{\bar{\theta}}}\left\\{\int\left|\frac{1}{\|\psi-\bar{\theta}\|}\left\\{\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\psi}(\mathbf{X})}-\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T_{\bar{\theta}}(\mathbf{X})}\right\\}\right|dP\right\\}$ $\displaystyle<$
$\displaystyle\infty.$

Therefore, the following set is uniformaly integrable for $\mu$:

$\left\\{\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}\,:\,\psi\in
B_{\bar{\theta}}\right\\}.$ (87)

Then, from the property of the Lebesgue integral (26, P188, Theorem 4), the
integral $\int\int(\cdot)\,dPdQ$ and the limitation
$\lim_{\psi\rightarrow\bar{\theta}}\,$ for the above term are exchangeable.

Hence, we have

$\displaystyle\lim_{\psi\rightarrow\bar{\theta}}\hbox
to0.0pt{$\displaystyle\int\int\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}dP\,dQ$\hss}$ (88)
$\displaystyle=$
$\displaystyle\int\int\lim_{\psi\rightarrow\bar{\theta}}\left[\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}\right]dP\,dQ$
$\displaystyle=$
$\displaystyle\int\int\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\theta})|_{\theta=\bar{\theta}}\,dP\,dQ$
$\displaystyle=$ $\displaystyle
E\Big{[}\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\theta})|_{\theta=\bar{\theta}}\Big{]}.$

On the other hand, for the term on the left hand side of (88), we obtain

$\displaystyle\lim_{\psi\rightarrow\bar{\theta}}\hbox
to0.0pt{$\displaystyle\int\int\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}dP\,dQ$\hss}$ (89)
$\displaystyle=$
$\displaystyle\lim_{\psi\rightarrow\bar{\theta}}\frac{1}{\|\psi-\bar{\theta}\|}\hbox
to0.0pt{$\displaystyle\int\int\Big{\\{}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\psi})-l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\bar{\theta}})\Big{\\}}dP\,dQ$\hss}$
$\displaystyle=$
$\displaystyle\lim_{\psi\rightarrow\bar{\theta}}\frac{1}{\|\psi-\bar{\theta}\|}\Big{\\{}L_{\alpha}(Q,P;T_{\psi})-L_{\alpha}(Q,P;T_{\bar{\theta}})\Big{\\}}$
$\displaystyle=$
$\displaystyle\nabla_{\theta}\,L_{\alpha}(Q,P;T_{\theta})|_{\theta=\bar{\theta}}.$

From (88) and (89), we obtain

$E\left[\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim
P};T_{\theta})|_{\theta=\bar{\theta}}\right]=\nabla_{\theta}\,L_{\alpha}(Q,P;T_{\theta})|_{\theta=\bar{\theta}}.$
(90)

From this, we also have

$\displaystyle
E\left[\nabla_{\theta}\,l_{\alpha}(\hat{Q}^{(N)},\hat{P}^{(N)};T_{\theta})|_{\theta=\bar{\theta}}\right]$
$\displaystyle=$ $\displaystyle
E\left[\nabla_{\theta}|_{\theta=\bar{\theta}}\,\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\theta})\right]$ (91) $\displaystyle=$
$\displaystyle
E\left[\frac{1}{N}\sum_{i=1}^{N}\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\theta})|_{\theta=\bar{\theta}}\right]$
$\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}E\Big{[}\nabla_{\theta}\,l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T_{\theta})|_{\theta=\bar{\theta}}\Big{]}$
$\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}\nabla_{\theta}\,L_{\alpha}(Q,P;T_{\theta})|_{\theta=\bar{\theta}}$
$\displaystyle=$
$\displaystyle\nabla_{\theta}\,L_{\alpha}(Q,P;T_{\theta})|_{\theta=\bar{\theta}}.$

Here, we see (78) from (90) and (91).

This completes the proof. ∎

######  Proposition C.9.

Let

$\hat{D}_{\alpha}^{(N)}(Q||P)=\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left[\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim
Q})}\right\\}-\frac{1}{1-\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})}\right\\}\right],$ (92)

where supremum is considered over all measurable functions
$T:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with $E_{P}[e^{(\alpha-1)\cdot
T}]<\infty$ and $E_{Q}[e^{\alpha\cdot T}]<\infty$.

Then, it holds that if $\alpha\neq 1/2$,

$\displaystyle\sqrt{N}\left\\{\hat{D}_{\alpha}^{(N)}(Q||P)-D_{\alpha}(Q||P)\right\\}$
(93) $\displaystyle\xrightarrow{\ \ d\ \ }$ $\displaystyle\mathcal{N}\Big{(}\
0,\ C^{1}_{\alpha}\cdot D_{2\alpha-1}(Q||P)+C^{2}_{\alpha}\cdot
D_{\alpha}(Q||P)+C^{3}_{\alpha}\cdot D_{\alpha}(Q||P)^{2}\ \Big{)},$

where

$\displaystyle C^{1}_{\alpha}$ $\displaystyle=$
$\displaystyle\left(\frac{1}{\alpha^{2}}+\frac{1}{(1-\alpha)^{2}}\right)\cdot(2\alpha-1)\cdot(2\alpha-2),$
(94) $\displaystyle C^{2}_{\alpha}$ $\displaystyle=$
$\displaystyle\frac{2\\{\alpha^{2}+(1-\alpha)^{2}\\}}{\alpha\cdot(1-\alpha)}\quad\text{and}\quad
C^{3}_{\alpha}=-\alpha^{2}-(1-\alpha)^{2},$ (95)

and if $\alpha=1/2$,

$\displaystyle\sqrt{N}\left\\{\hat{D}_{\alpha}^{(N)}(Q||P)-D_{\alpha}(Q||P)\right\\}$
(96) $\displaystyle\xrightarrow{\ \ d\ \ }$ $\displaystyle\mathcal{N}\Big{(}\
0,\ 4\,D_{\alpha}(Q||P)-\frac{1}{2}\,D_{\alpha}(Q||P)^{2}\ \Big{)}.$

###### proof of Proposition C.9.

First, we note that

$\displaystyle\hat{D}_{\alpha}^{(N)}(Q||P)$ $\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left[\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim
Q})}\right\\}-\frac{1}{1-\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})}\right\\}\right]$ (97) $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left[\frac{1}{\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim
Q})}\right\\}+\frac{1}{1-\alpha}\left\\{\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})}\right\\}\right]$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}\left[\frac{1}{\alpha}e^{\alpha\cdot
T(\mathbf{X}^{i}_{\sim Q})}+\frac{1}{1-\alpha}e^{(\alpha-1)\cdot
T(\mathbf{X}^{i}_{\sim P})}\right]$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\frac{1}{N}\sum_{i=1}^{N}\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P};T)\right]$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{N}\sum_{i=1}^{N}l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P}).$

On the other hand, from Lemma C.7, it holds that

$\displaystyle D_{\alpha}(Q||P)$ $\displaystyle=$
$\displaystyle\sup_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot
T}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot T}\right]\right\\}$
(98) $\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\inf_{T:\mathbb{R}^{d}\rightarrow\mathbb{R}}\left\\{\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot
T}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot T}\right]\right\\}$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{N}\sum_{i=1}^{N}L_{\alpha}(Q,P)$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{N}\sum_{i=1}^{N}E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right].$

Subtracting (98) from (97), we have

$\hat{D}_{\alpha}^{(N)}(Q||P)-D_{\alpha}(Q||P)=\frac{1}{N}\sum_{i=1}^{N}\left\\{l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})-E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]\right\\}.$ (99)

Let $L_{i}=l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim
P})-E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim
P})\right]$. Then $\\{L_{i}\\}_{i=1}^{N}$ are independently identically
distributed variables whose means and variances are as follows:

$\displaystyle E\left[L_{i}\right]$ $\displaystyle=$ $\displaystyle 0,$ (100)
$\displaystyle\mathrm{Var}\left[L_{i}\right]$ $\displaystyle=$ $\displaystyle
E\left[\left\\{l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim
P})-E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim Q},\mathbf{X}^{i}_{\sim
P})\right]\right\\}^{2}\right]$ (101) $\displaystyle=$ $\displaystyle
E_{P}\left[\ E_{Q}\left[\
\frac{1}{\alpha}\left\\{\left(\frac{dQ}{dP}\right)^{-\alpha}(\mathbf{X}^{i}_{\sim
Q})-E_{Q}\left[\left(\frac{dQ}{dP}\right)^{-\alpha}\right]\right\\}\right.\right.$
$\displaystyle\qquad+\left.\left.\frac{1}{1-\alpha}\left\\{\left(\frac{dQ}{dP}\right)^{1-\alpha}(\mathbf{X}^{i}_{\sim
P})-E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\ \right]\right\\}\
\right]^{2}\right]$ $\displaystyle=$ $\displaystyle\frac{1}{\alpha^{2}}\cdot
E_{Q}\left\\{\left(\frac{dQ}{dP}\right)^{-\alpha}(\mathbf{X}^{i})-E_{Q}\left[\left(\frac{dQ}{dP}\right)^{-\alpha}\
\right]\right\\}^{2}$ $\displaystyle\qquad+\frac{1}{(1-\alpha)^{2}}\cdot
E_{P}\left\\{\left(\frac{dQ}{dP}\right)^{1-\alpha}(\mathbf{X}^{i})-E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\right]\right\\}^{2}$
$\displaystyle=$ $\displaystyle\frac{1}{\alpha^{2}}\cdot
E_{P}\left\\{\frac{dQ}{dP}\cdot\left(\frac{dQ}{dP}\right)^{-\alpha}(\mathbf{X}^{i})-E_{P}\left[\frac{dQ}{dP}\cdot\left(\frac{dQ}{dP}\right)^{-\alpha}\
\right]\right\\}^{2}$ $\displaystyle\qquad+\frac{1}{(1-\alpha)^{2}}\cdot
E_{P}\left\\{\left(\frac{dQ}{dP}\right)^{1-\alpha}(\mathbf{X}^{i})-E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\right]\right\\}^{2}$
$\displaystyle=$
$\displaystyle\left\\{\frac{1}{\alpha^{2}}+\frac{1}{(1-\alpha)^{2}}\right\\}\cdot
E_{P}\left\\{\left(\frac{dQ}{dP}\right)^{1-\alpha}-E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\right]\right\\}^{2}$
$\displaystyle=$
$\displaystyle\left\\{\frac{1}{\alpha^{2}}+\frac{1}{(1-\alpha)^{2}}\right\\}\Biggr{\\{}E_{P}\left[\left(\frac{dQ}{dP}\right)^{2\cdot(1-\alpha)}\right]-\left\\{E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}\right]\right\\}^{2}\Biggr{\\}}$
$\displaystyle=$
$\displaystyle\left\\{\frac{1}{\alpha^{2}}+\frac{1}{(1-\alpha)^{2}}\right\\}$
$\displaystyle\ \times\Bigg{[}\
(2\alpha-1)(2\alpha-2)\left\\{\frac{1}{(2\alpha-1)(2\alpha-2)}E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-(2\alpha-1)}-1\right]\right\\}\
+\ 1$
$\displaystyle\quad\quad-\alpha^{2}(1-\alpha)^{2}\left\\{\frac{1}{\alpha\cdot(\alpha-1)}E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}-1\
\right]\right\\}^{2}$
$\displaystyle\quad\quad+\alpha^{2}(1-\alpha)^{2}\left\\{\frac{2}{\alpha\cdot(\alpha-1)}E_{P}\left[\left(\frac{dQ}{dP}\right)^{1-\alpha}-1\
\right]\right\\}\quad-\ 1\ \Bigg{]}.$

From this, if $\alpha\neq 1/2$, we have

$\displaystyle\mathrm{Var}\left[L_{i}\right]$ $\displaystyle=$ $\displaystyle
C^{1}_{\alpha}\cdot D_{2\alpha-1}(Q||P)+C^{2}_{\alpha}\cdot
D_{\alpha}(Q||P)+C^{3}_{\alpha}\cdot D_{\alpha}(Q||P)^{2},$ (102)

where

$\displaystyle C^{1}_{\alpha}$ $\displaystyle=$
$\displaystyle\left(\frac{1}{\alpha^{2}}+\frac{1}{(1-\alpha)^{2}}\right)\cdot(2\alpha-1)\cdot(2\alpha-2),$
$\displaystyle C^{2}_{\alpha}$ $\displaystyle=$
$\displaystyle\frac{2\\{\alpha^{2}+(1-\alpha)^{2}\\}}{\alpha\cdot(1-\alpha)}\quad\text{and}\quad
C^{3}_{\alpha}=-\alpha^{2}-(1-\alpha)^{2},$

and if $\alpha=1/2$, we obtain

$\displaystyle\mathrm{Var}\left[L_{i}\right]$ $\displaystyle=$ $\displaystyle
4\,D_{\alpha}(Q||P)-\frac{1}{2}\,D_{\alpha}(Q||P)^{2}.$ (103)

Therefore, by the central limit theorem, we see

$\frac{1}{N}\sum_{i=1}^{N}\left\\{l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})-E\left[l_{\alpha}(\mathbf{X}^{i}_{\sim
Q},\mathbf{X}^{i}_{\sim P})\right]\right\\}\xrightarrow{\ \ d\ \
}\mathcal{N}\left(0,\ast\right).$ (104)

Here the “$\ast$” is (102) or (103), which corresponds to the cases that
$\alpha\neq 1/2$ or $\alpha=1/2$, respectively.

This completes the proof. ∎

We mention that the statement of the following corollary is the same as
Corollary 1 in Birrell et al.(2022).

######  Corollary C.10 (Birrell et al.(2022), P19, Corollary 1).

For $\alpha=1/2$, it holds that

$\lim_{N\rightarrow\infty}\frac{N\cdot\mathrm{Var}\Big{[}\
\hat{D}_{1/2}^{(N)}(Q||P)\Big{]}}{D_{1/2}(Q||P)^{2}}=\frac{8\,D_{1/2}(Q||P)-D_{1/2}(Q||P)^{2}}{2\,D_{1/2}(Q||P)^{2}}.$
(105)

Thus, the sample complexity of $D_{\alpha}$ for $\alpha=1/2$ is $O(1)$.

###### proof of Corollary C.10.

The statement of the corollary follows from (96) in Proposition C.9. ∎

######  Proposition C.11.

Let $\\{T_{k}\\}_{k=1}^{\infty}$ be a sequence of functions in
$\mathcal{T}^{\alpha}$ with $E_{P}[e^{-T_{k}(\mathbf{X})}]=1$ such that
$\lim_{k\rightarrow\infty}T_{k}=-\log dQ/dP$, $P$-almost everywhere.
Subsequently, let $\\{\mathbf{X}^{Q}_{k}\\}_{k=1}^{\infty}$ be a sequence of
variables on $\mathbb{R}^{d}$ defiened as follows:

$\mathbf{X}^{Q}_{k}=e^{-T_{k}(\mathbf{X}^{P})}\cdot\mathbf{X}^{P}.$

Then, it holds that

$\mathbf{X}^{Q}_{k}\xrightarrow{\ \ \text{d}\ \
}\mathbf{X}^{Q},\quad\text{as}\quad k\longrightarrow\infty.$ (106)

###### proof of Proposition C.11.

Let $Q_{k}$ denote the probability distribution of $\mathbf{X}^{Q}_{k}$:
$Q_{k}(A)=P(\mathbf{X}^{Q}_{k}\in A)$ for all $A\in\mathscr{F}$. Then, since
$\frac{dQ_{k}}{dP}=e^{-T_{k}(\mathbf{X})}$, we see

$\frac{dQ_{k}}{dQ}=e^{-T_{k}(\mathbf{X})}\cdot\frac{dP}{dQ}.$ (107)

Now, from Corollary 6 in [7], for probalility measures $A$ and $B$ with
$A\ll\mu$ and $B\ll\mu$, it holds that

$\frac{1}{2}\left\\{E_{\mu}\left|\frac{dA}{d\mu}-\frac{dB}{d\mu}\right|\right\\}^{2}\leq
D_{\alpha}(A||B).$ (108)

By substituting $A=Q_{k}$ and $B=Q$ into (108), we have

$\displaystyle\frac{1}{2}\left\\{E_{\mu}\left|\frac{dQ_{k}}{d\mu}-\frac{dQ}{d\mu}\right|\right\\}^{2}$
$\displaystyle\leq$ $\displaystyle D_{\alpha}(Q_{k}||Q)$ (109)
$\displaystyle=$
$\displaystyle\int\frac{1}{\alpha(\alpha-1)}\left\\{\left(\frac{dQ_{k}}{dQ}\right)^{1-\alpha}-1\right\\}dQ$
$\displaystyle=$
$\displaystyle\int\frac{1}{\alpha(\alpha-1)}\int\left(\frac{dQ_{k}}{dQ}\right)^{1-\alpha}dQ-\frac{1}{\alpha(\alpha-1)}$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(\alpha-1)}\int\left(\frac{dP}{dQ}\right)^{1-\alpha}\cdot
e^{(\alpha-1)\cdot T_{k}}\frac{dQ}{dP}dP-\frac{1}{\alpha(\alpha-1)}$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(\alpha-1)}\int\left(\frac{dQ}{dP}\right)^{\alpha}\cdot
e^{(\alpha-1)\cdot T_{k}}dP-\frac{1}{\alpha(\alpha-1)}.$

Now, from Hölder’s inequality, we have

$\displaystyle\int\left|\left(\frac{dQ}{dP}\right)^{\alpha}\cdot
e^{(\alpha-1)\cdot T_{k}}\right|dP$ $\displaystyle\leq$
$\displaystyle\left\\{\int\left(\frac{dP}{dQ}\right)^{\frac{-\alpha}{\alpha}}dP\right\\}^{\alpha}$
(110) $\displaystyle\quad\times\ \left\\{\int\left(e^{(\alpha-1)\cdot
T_{k}}\right)^{\frac{1}{1-\alpha}}dP\right\\}^{1-\alpha}$ $\displaystyle=$
$\displaystyle\left(E_{P}\left[e^{-T_{k}}\right]\right)^{1-\alpha}$
$\displaystyle=$ $\displaystyle 1<\infty.$

Hence, we see the following sequence is uniformaly integrable for $P$:

$\left\\{\left(\frac{dQ}{dP}\right)^{\alpha}\cdot e^{(\alpha-1)\cdot
T_{k}}\right\\}_{k=1}^{N}.$ (111)

Then, for (109) as $k\rightarrow\infty$, we see

$\displaystyle\lim_{k\rightarrow\infty}\frac{1}{2}\left\\{E_{\mu}\left|\frac{dQ_{k}}{d\mu}-\frac{dQ}{d\mu}\right|\right\\}^{2}$
$\displaystyle\leq$
$\displaystyle\lim_{k\rightarrow\infty}\frac{1}{\alpha(\alpha-1)}\int\left(\frac{dQ}{dP}\right)^{\alpha}\cdot
e^{(\alpha-1)\cdot T_{k}}dP-\frac{1}{\alpha(\alpha-1)}$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha(\alpha-1)}\int\left(\frac{dQ}{dP}\right)^{\alpha}\cdot\left(\frac{dQ}{dP}\right)^{1-\alpha}dP-\frac{1}{\alpha(\alpha-1)}$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha(\alpha-1)}\int\frac{dQ}{dP}dP-\frac{1}{\alpha(\alpha-1)}$
$\displaystyle=$ $\displaystyle 0.$

Thus, $Q_{k}$ converges to $Q$ in total variation.

The statement (106), the convergence of $\hat{\mathbf{X}}^{Q}_{k}$ to
$\mathbf{X}^{Q}$ in distribution, is derived from the convergence of $Q_{k}$
to $Q$ in total variation.

This completes the proof. ∎

######  Corollary C.12.

Let $\\{T_{k}\\}_{k=1}^{\infty}$ be a sequence of functions in
$\mathcal{T}^{\alpha}$ such that

$\displaystyle D_{\alpha}(\hat{Q}^{(N)}||\hat{P}^{(N)})$ (112)
$\displaystyle=$
$\displaystyle\lim_{k\rightarrow\infty}\left\\{\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{\alpha\cdot
T_{k}(\mathbf{X}^{i}_{\sim
Q})}+\frac{1}{1-\alpha}\frac{1}{N}\sum_{i=1}^{N}e^{(\alpha-1)\cdot
T_{k}(\mathbf{X}^{i}_{\sim P})}\right\\}.$

Subsequently, let
$\\{\hat{\mathbf{X}}_{\mathbb{Q}}^{(N)}(k)\\}_{k=1}^{\infty}$ be a sequence of
variables on $\mathbb{R}^{d}$ defiened as follows:

$\hat{\mathbf{X}}_{\mathbb{Q}}^{(N)}(k)=e^{-T_{k}}\cdot\mathbf{X}_{\mathbb{P}}^{(N)}$
(113)

Then, it holds that, as $k\longrightarrow\infty$,

$\hat{\mathbf{X}}_{\mathbb{Q}}^{(N)}(k)\xrightarrow{\ \ \text{d}\ \
}\hat{\mathbf{X}}_{\mathbb{Q}}^{(N)}.$ (114)

###### proof of Corollary C.12.

Let $\nu$ be the countable measure on
$\left\\{\mathbf{X}_{1},\mathbf{X}_{2},\ldots,\mathbf{X}_{N}\right\\}$:

$\nu(\mathbf{x})=\left\\{\begin{array}[]{ll}1&\text{if}\ \ 1\leq\exists i\leq
N\ \text{s.t.}\ \mathbf{X}_{i}=\mathbf{x},\\\
0&\text{otherwise.}\end{array}\right.$ (115)

Then, $\hat{P}^{(N)}\ll\nu$ and $\hat{Q}^{(N)}\ll\nu$.

For Proposition C.11 and its proof, substituting $\hat{P}^{(N)}$ for $P$,
$\hat{Q}^{(N)}$ for $Q$, and $\nu$ for $\mu$, we see that the statement of the
corollary holds.

This completes the proof. ∎

######  Proposition C.13.

For $\hat{T}\in\mathcal{T}^{\alpha}$, let $\hat{Q}$ and $\hat{P}$ be two
probalities defined as

$d\hat{Q}=e^{-\hat{T}}\cdot dP\quad\text{and}\quad d\hat{P}=e^{\hat{T}}\cdot
dQ,$

and let $T_{*}=-\log dQ/dP$.

Then, it hols that

$\displaystyle\frac{\alpha}{2}\left\\{E_{\mu}\left|\hat{Q}-Q\right|\right\\}^{2}+\frac{1-\alpha}{2}\left\\{E_{\mu}\left|\hat{P}-P\right|\right\\}^{2}$
(116) $\displaystyle\leq$
$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}E_{Q}\left[e^{\alpha\cdot(T_{*}-\hat{T})}\right]-\frac{1}{1-\alpha}E_{P}\left[e^{(\alpha-1)\cdot(T_{*}-\hat{T})}\right]$
$\displaystyle=$ $\displaystyle
L_{\alpha}(Q,P;\hat{T})-L_{\alpha}(Q,P;T_{*}).$ (117)

Here, $L_{\alpha}(Q,P;\cdot)$ in (117) is defined as (57) in Lemma C.7

###### proof of Proposition C.13.

First, we see (116). Note that, it holds that

$\frac{\hat{dQ}}{dQ}=e^{-\hat{T}}\cdot\frac{dP}{dQ}\qquad\text{and}\qquad\frac{d\hat{P}}{dP}=e^{\hat{T}}\cdot\frac{dQ}{dP}.$
(118)

By using (108), we have

$\frac{\alpha}{2}\left\\{E_{\mu}\left|\hat{Q}-Q\right|\right\\}^{2}+\frac{1-\alpha}{2}\left\\{E_{\mu}\left|\hat{P}-P\right|\right\\}^{2}\leq(1-\alpha)\cdot
D_{1-\alpha}(\hat{Q}||Q)+\alpha\cdot D_{\alpha}(\hat{P}||P).$

Thus, we obtain

$\displaystyle\frac{\alpha}{2}\left\\{E_{\mu}\left|\hat{Q}-Q\right|\right\\}^{2}+\frac{1-\alpha}{2}\left\\{E_{\mu}\left|\hat{P}-P\right|\right\\}^{2}$
(119) $\displaystyle\leq$ $\displaystyle(1-\alpha)\cdot
D_{1-\alpha}(\hat{Q}||Q)+\alpha\cdot D_{\alpha}(\hat{P}||P)$ $\displaystyle=$
$\displaystyle\frac{1-\alpha}{\alpha(\alpha-1)}\left\\{\int\left(\frac{d\hat{Q}}{dQ}\right)^{\alpha}dQ-1\right\\}$
$\displaystyle\qquad+\
\frac{\alpha}{\alpha(\alpha-1)}\left\\{\int\left(\frac{d\hat{P}}{dP}\right)^{1-\alpha}dP-1\right\\}$
$\displaystyle=$ $\displaystyle-\ \frac{1}{\alpha}\left\\{\int
e^{-\alpha\cdot\hat{T}}\left(\frac{dP}{dQ}\right)^{\alpha}dQ-1\right\\}$
$\displaystyle\qquad-\ \frac{1}{1-\alpha}\left\\{\int
e^{(1-\alpha)\cdot\hat{T}}\left(\frac{dQ}{dP}\right)^{1-\alpha}dP-1\right\\}$
$\displaystyle=$ $\displaystyle-\ \frac{1}{\alpha}\left\\{\int
e^{-\alpha\cdot\hat{T}}e^{\alpha\cdot T_{*}}dQ-1\right\\}$
$\displaystyle\qquad-\ \frac{1}{1-\alpha}\left\\{\int
e^{-(\alpha-1)\cdot\hat{T}}e^{(\alpha-1)\cdot T_{*}}dP-1\right\\}$
$\displaystyle=$ $\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\int
e^{\alpha\cdot(T_{*}-\hat{T})}dQ-\frac{1}{1-\alpha}\int
e^{(\alpha-1)\cdot(T_{*}-\hat{T})}dP.$

Here, we see (116).

To obtain (117), we have

$\displaystyle\frac{1}{\alpha(1-\alpha)}-\frac{1}{\alpha}\int
e^{\alpha\cdot(T_{*}-\hat{T})}dQ-\frac{1}{1-\alpha}\int
e^{(\alpha-1)\cdot(T_{*}-\hat{T})}dP$ (120) $\displaystyle=$
$\displaystyle\frac{1}{\alpha}+\frac{1}{1-\alpha}-\frac{1}{\alpha}\int
e^{\alpha\cdot(T_{*}-\hat{T})}dQ-\frac{1}{1-\alpha}\int
e^{(\alpha-1)\cdot(T_{*}-\hat{T})}dP$ $\displaystyle=$
$\displaystyle\int\left\\{\frac{1}{\alpha}\cdot\frac{dQ}{d\mu}-\frac{1}{\alpha}\cdot
e^{\alpha\cdot(T_{*}-\hat{T})}\frac{dQ}{d\mu}\right\\}d\mu$
$\displaystyle\qquad\qquad+\int\left\\{\frac{1}{1-\alpha}\cdot\frac{dP}{d\mu}-\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot(T_{*}-\hat{T})}\frac{dP}{d\mu}\right\\}d\mu.$

By replacing the measures $\mu$ of the two integrals in (120) with

$\nu=e^{-\alpha\cdot\hat{T}}d\mu\quad\text{and}\quad\tau=e^{-(\alpha-1)\cdot\hat{T}}d\mu,$
(121)

respectively, we obtain

$\displaystyle\int\left\\{\frac{1}{\alpha}\cdot\frac{dQ}{d\nu}-\frac{1}{\alpha}\cdot
e^{\alpha\cdot(T_{*}-\hat{T})}\frac{dQ}{d\nu}\right\\}e^{\alpha\cdot\hat{T}}\cdot
d\nu$ (122)
$\displaystyle\qquad\qquad+\int\left\\{\frac{1}{1-\alpha}\cdot\frac{dP}{d\tau}-\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot(T_{*}-\hat{T})}\frac{dP}{d\tau}\right\\}e^{(\alpha-1)\cdot\hat{T}}\cdot
d\tau$ $\displaystyle=$ $\displaystyle\int\left\\{\frac{1}{\alpha}\cdot
e^{\alpha\cdot\hat{T}}\frac{dQ}{d\nu}-\frac{1}{\alpha}\cdot e^{\alpha\cdot
T_{*}}\frac{dQ}{d\nu}\right\\}d\nu$
$\displaystyle\qquad\qquad+\int\left\\{\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot\hat{T}}\frac{dP}{d\tau}-\frac{1}{1-\alpha}\cdot
e^{(\alpha-1)\cdot T_{*}}\frac{dP}{d\tau}\right\\}d\tau$ $\displaystyle=$
$\displaystyle\left\\{\frac{1}{\alpha}\int
e^{\alpha\cdot\hat{T}}dQ+\frac{1}{1-\alpha}\int
e^{(\alpha-1)\cdot\hat{T}}dP\right\\}$ $\displaystyle\qquad\qquad\qquad-\
\left\\{\frac{1}{\alpha}\int e^{\alpha\cdot T_{*}}dQ+\frac{1}{1-\alpha}\int
e^{(\alpha-1)\cdot T_{*}}dP\right\\}$ $\displaystyle=$ $\displaystyle
L_{\alpha}(Q,P;\hat{T})-L_{\alpha}(Q,P;T_{*}).$

This completes the proof. ∎

###  C.2 Proofs for Section 6

In this Section, we present two theorems for the proposed method in Section 6.
Before presenting the first theorem, we briefly review Pearl’s $do$-calculus
(Pearl(1995)) used in the proof of the first theorem.

######  Theorem C.14 ($do$-calculus, Pearl(1995)).

Causal effects can be transformed by following rules R1-R3:

  1. R1.

$P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z},\mathbf{W})=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{W})$,
if
$(\mathbf{Y}\mathrel{\perp\mspace{-10.0mu}\perp}\mathbf{Z}|\mathbf{X},\mathbf{W})_{\overline{G}(\mathbf{X})}$.

  2. R2.

$P(\mathbf{Y}|do(\mathbf{X}),do(\mathbf{Z}),\mathbf{W})=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z},\mathbf{W})$,
if
$(\mathbf{Y}\mathrel{\perp\mspace{-10.0mu}\perp}\mathbf{Z}|\mathbf{X},\mathbf{W})_{\overline{\underline{G}}(\mathbf{X},\mathbf{Z})}$.

  3. R3.

$P(\mathbf{Y}|do(\mathbf{X}),do(\mathbf{Z}),\mathbf{W})=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{W})$,
if
$(\mathbf{Y}\mathrel{\perp\mspace{-10.0mu}\perp}\mathbf{Z}|\mathbf{X},\mathbf{W})_{\overline{G}(\mathbf{X},\mathbf{Z}^{*})}$,
where $\mathbf{Z}^{*}=\mathbf{Z}\setminus
An(\mathbf{W})_{\overline{G}(\mathbf{X})}$.

Here, $\overline{G}(\mathbf{A})$ denotes a graph obtained from $G$ by deleting
all arrows emerging from variables to $\mathbf{A}$, and
$\underline{\overline{G}}(\mathbf{A},\mathbf{B})$ denotes a graph obtained
from $G$ by deleting both of all arrows emerging from any variables to
$\mathbf{A}$ and all arrows emerging from $\mathbf{B}$ to any variables, and
$(\mathbf{A}\mathrel{\perp\mspace{-10.0mu}\perp}\mathbf{B})_{G}$ represents
that there is no path between $\mathbf{A}$ and $\mathbf{B}$ in $G$.

We now provide the first theorem, which presents a sufficient condition for
explanatory variables to be available for estimating causal effects.

######  Theorem C.15.

Let $G$ be a DAG for $\mathbf{V}$ and $\mathbf{U}$. For disjoint sets
$\mathbf{X},\mathbf{Y},\mathbf{Z}\subset\mathbf{V}$, suppose that
$P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})$ is identifiable in $G$, and
$\mathbf{X}\subset An(\mathbf{Y})_{G}$. Let $\mathbf{Z}_{De}=\mathbf{Z}\cap
De(\mathbf{Y})_{G}$. Then,

$\displaystyle P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})$ (123) $\displaystyle=$
$\displaystyle\begin{cases}P(\mathbf{Y}|\mathbf{X},\mathbf{Z})&\text{if
$\mathbf{Z}_{De}=\phi$},\\\
\frac{P(\mathbf{Y}|\mathbf{X},\mathbf{Z}\setminus\mathbf{Z}_{De})P(\mathbf{Z}_{De}|\mathbf{Y},\mathbf{X},\mathbf{Z}\setminus\mathbf{Z}_{De})}{P(\mathbf{Z}_{De}|\mathbf{X},\mathbf{Z}\setminus\mathbf{Z}_{De})}&\text{if
$\mathbf{Z}_{De}\neq\phi$}.\end{cases}$

###### proof of Theorem C.15.

We note that each $Z_{i}\in\mathbf{Z}$ can be assumed to be that either
$Z_{i}\in An(\mathbf{Y})_{G}$ or $Z_{i}\in De(\mathbf{Y})_{G}$. To see this,
suppose that there exist some $Z_{i}\in\mathbf{Z}$ such that $Z_{i}\notin
An(\mathbf{Y})_{G}$ and $Z_{i}\notin De(\mathbf{Y})_{G}$. Let
$\mathbf{V}^{\prime}=\mathbf{V}\setminus(\mathbf{Y}\cup\mathbf{X}\cup\mathbf{Z})$.
Since
$(\mathbf{Y}\mathrel{\perp\mspace{-10.0mu}\perp}Z_{i}|\mathbf{X},\mathbf{V}^{\prime},\mathbf{U})_{\overline{G}(\mathbf{X})}$
holds for the $Z_{i}$, by applying $do$-calculus R1 in Theorem C.14, we have

$P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z}\setminus\\{Z_{i}\\},Z_{i},\mathbf{V}^{\prime},\mathbf{U})=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z}\setminus\\{Z_{i}\\},\mathbf{V}^{\prime},\mathbf{U}).$
(124)

By marginalizing both sides of (124) for
$\mathcal{X}_{\mathbf{V}^{\prime}\cup\mathbf{U}}$, we obtain

$P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z}\setminus\\{Z_{i}\\},Z_{i})=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z}\setminus\\{Z_{i}\\}).$
(125)

Thus, after repeating the above calculation,
$P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})$ finally includes only
$Z_{i}\in\mathbf{Z}$ such that $Z_{i}\in An(\mathbf{Y})_{G}$ or $Z_{i}\in
De(\mathbf{Y})_{G}$.

Therefore, in this proof, we assume that

$\mathbf{Z}=An(\mathbf{Y})_{G}\cup De(\mathbf{Y})_{G}.$ (126)

Next, we note that $\mathbf{Z}\cap An(\mathbf{X})_{G}\cap
De(\mathbf{Y})_{G}=\phi$. To see this, suppose $\mathbf{Z}\cap
An(\mathbf{X})_{G}\cap De(\mathbf{Y})_{G}\neq\phi$. Let
$\dotarrow{\mathbf{V}^{\prime}}$ denote a path through only variables of
$\mathbf{V}^{\prime}$. Then there exists a directed path such that
$\mathbf{Y}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}\dotarrow{\mathbf{V}^{\prime}}\mathbf{X}$,
which contradicts the assumption $\mathbf{X}\subset An(\mathbf{Y})_{G}$.

From the above discussion, $\mathbf{Z}$ can be divided into the three disjoint
sets as follows:

$\displaystyle\mathbf{Z}$ $\displaystyle=$
$\displaystyle\mathbf{Z}_{1}\cup\mathbf{Z}_{2}\cup\mathbf{Z}_{3},$
$\displaystyle\mathbf{Z}_{1}$ $\displaystyle=$
$\displaystyle(\mathbf{Z}\setminus De(\mathbf{X})_{G})\cap
An(\mathbf{Y})_{G},$ $\displaystyle\mathbf{Z}_{2}$ $\displaystyle=$
$\displaystyle\mathbf{Z}\cap De(\mathbf{X})_{G}\cap An(\mathbf{Y})_{G},$
$\displaystyle\mathbf{Z}_{3}$ $\displaystyle=$
$\displaystyle(\mathbf{Z}\setminus An(\mathbf{X})_{G})\cap
De(\mathbf{Y})_{G}.$

Then, each of the paths between $\mathbf{Z}_{1}$, $\mathbf{Z}_{2}$ and
$\mathbf{Z}_{3}$ is one of the following P1, P2 and P3:

  1. P1.

$\mathbf{Z}_{1}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{2}$,

  2. P2.

$\mathbf{Z}_{2}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{3}$,

  3. P3.

$\mathbf{Z}_{1}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{3}$.

In fact, if there exists a directed path in the opposite direction of P1, that
is $\mathbf{Z}_{2}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{1}$, then there
exists a path such that
$X_{i}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{2}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{1}$.
This contradicts the assumption $\mathbf{Z}_{1}\subset\mathbf{Z}\setminus
De(\mathbf{X})_{G}$. Similarly, if there exists a directed path in the
opposite direction of P2, that is
$\mathbf{Z}_{3}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{2}$, then there
exists a path such that
$Y_{i}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{3}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{2}$,
which contradicts the assumption $\mathbf{Z}_{2}\subset An(\mathbf{Y})_{G}$.
In addition, if there exists a directed path in the opposite direction of P3,
that is $\mathbf{Z}_{3}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{1}$, then
there exists a path such that
$Y_{i}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{3}\dotarrow{\mathbf{V}^{\prime}}\mathbf{Z}_{1}$,
which contradicts the assumption $\mathbf{Z}_{1}\subset An(\mathbf{Y})_{G}$.
Therefore, all paths expect P1, P2 and P3 are denied.

Hence, by marginalizing $P(\mathbf{V})$ for
$\mathcal{X}_{\mathbf{V}^{\prime}}$, we obtain

$\displaystyle P(\mathbf{Y},\mathbf{X},\mathbf{Z})$ $\displaystyle=$
$\displaystyle\sum_{\mathcal{X}_{\mathbf{V}^{\prime}}}P(\mathbf{V})$
$\displaystyle=$ $\displaystyle
P(\mathbf{Y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})\cdot
P(\mathbf{X}|\mathbf{Z}_{1})\cdot P(\mathbf{Z}_{1})$ $\displaystyle\times
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{3}|\mathbf{Y},\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2}).$

In additon, from (1), we have

$\displaystyle P(\mathbf{Y},\mathbf{Z}|do(\mathbf{X}))$ $\displaystyle=$
$\displaystyle P(\mathbf{Y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})\cdot
P(\mathbf{Z}_{1})\cdot P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})$ (127)
$\displaystyle\times
P(\mathbf{Z}_{3}|\mathbf{Y},\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2}).$

In the case that $\mathbf{Z}_{3}=\phi$, by marginalizing out $\mathbf{Y}$ of
(127), we have

$\displaystyle P(\mathbf{Z}|do(\mathbf{X}))$ $\displaystyle=$
$\displaystyle\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Y}=\mathbf{y},\mathbf{Z}|do(\mathbf{X}))$
$\displaystyle=$
$\displaystyle\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Y}=\mathbf{y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})\cdot
P(\mathbf{Z}_{1})\cdot P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})$
$\displaystyle=$ $\displaystyle P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Y}=\mathbf{y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})$
$\displaystyle=$ $\displaystyle P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1}).$

On the other hand, in the case that $\mathbf{Z}_{3}\neq\phi$, we obtain

$\displaystyle P(\mathbf{Z}|do(\mathbf{X}))$ $\displaystyle=$
$\displaystyle\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Y}=\mathbf{y},\mathbf{Z}|do(\mathbf{X}))$
$\displaystyle=$
$\displaystyle\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Y}=\mathbf{y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})\cdot
P(\mathbf{Z}_{1})$ $\displaystyle\times
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{3}|\mathbf{Y}=\mathbf{y},\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})$
$\displaystyle=$ $\displaystyle P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})$
$\displaystyle\qquad\times\sum_{\mathbf{y}\in\mathcal{X}_{\mathbf{Y}}}P(\mathbf{Z}_{3}|\mathbf{Y}=\mathbf{y},\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})\cdot
P(\mathbf{Y}=\mathbf{y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})$
$\displaystyle=$ $\displaystyle P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{3}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2}).$

Summarizing the above results, we have

$P(\mathbf{Z}|do(\mathbf{X}))=\begin{cases}P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1}),&\text{if
$\mathbf{Z}_{3}=\phi$},\\\ P(\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{2}|\mathbf{X},\mathbf{Z}_{1})\cdot
P(\mathbf{Z}_{3}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})&\text{if
$\mathbf{Z}_{3}\neq\phi$}.\end{cases}$ (128)

Inserting (127) and (128) into (2), we see

$\displaystyle P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})$ $\displaystyle=$
$\displaystyle\frac{P(\mathbf{Y},\mathbf{Z}|do(\mathbf{X}))}{P(\mathbf{Z}|do(\mathbf{X}))}$
(129) $\displaystyle=$
$\displaystyle\begin{cases}P(\mathbf{Y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})&\text{if
$\mathbf{Z}_{3}=\phi$},\\\
\frac{P(\mathbf{Y}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})P(\mathbf{Z}_{3}|\mathbf{Y},\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})}{P(\mathbf{Z}_{3}|\mathbf{X},\mathbf{Z}_{1},\mathbf{Z}_{2})}&\text{if
$\mathbf{Z}_{3}\neq\phi$}.\end{cases}$

Note that, $\mathbf{Z}_{3}=\mathbf{Z}\cap De(\mathbf{Y})_{G}$, since
$\mathbf{Z}\cap An(\mathbf{X})_{G}\cap De(\mathbf{Y})_{G}=\phi$.

Therefore, by rewriting $\mathbf{Z}_{3}$ as $\mathbf{Z}_{De}$ and
$\mathbf{Z}_{1}\cup\mathbf{Z}_{2}$ as $\mathbf{Z}\setminus\mathbf{Z}_{De}$ for
(129), we obtain (123).

This completes the proof. ∎

Next, we provide the main theorem presented in Section 6.

######  Theorem C.16 (Theorem 6.1 restated).

Given disjoint sets of
$\mathbf{X}=\\{\mathbf{X}_{1},\allowbreak\mathbf{X}_{2},\allowbreak\dots,\allowbreak\mathbf{X}_{n}\\},\allowbreak\mathbf{Y},\allowbreak\mathbf{Z}\subset\allowbreak\mathbf{V}$
satisfying

$\mathbf{X}=\\{\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n}\\}\subset
An(\mathbf{Y})_{G},$ (130)

and

$\mathbf{Z}\cap De(\mathbf{Y})_{G}=\phi.$ (131)

Let
$\mathbb{P}=P(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})$
and $\mathbb{Q}=P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}})\times P(\mathbf{Z})$, and
$\widetilde{P}=P(\mathbf{Y}|do(\mathbf{X}),\mathbf{Z})\times
P(\mathbf{X_{1}})\times P(\mathbf{X_{2}})\times\cdots\times
P(\mathbf{X_{n}})\times P(\mathbf{Z})$.

Suppose $P$ satisfies Assumptions 1 and 2 in the above setting, and it holds
that
$E_{\mathbb{P}}\left[\left(d\mathbb{Q}/d\mathbb{P}\right)^{1-\alpha}\right]<\infty$
for some $0<\alpha<1$, then, for the optimal function $T^{*}$, such that

$\displaystyle
T^{*}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})$ (132)
$\displaystyle=$
$\displaystyle\arg\inf_{T\in\mathcal{T}^{\alpha}}\left\\{\frac{1}{\alpha}E_{\mathbb{Q}}\left[e^{\alpha\cdot
T}\right]\right.$
$\displaystyle\qquad\qquad\quad\left.+\frac{1}{1-\alpha}E_{\mathbb{P}}\left[e^{(\alpha-1)\cdot
T}\right]\right\\},$

it holds that

$\frac{d\widetilde{P}}{dP}=e^{-T^{*}(\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{n},\mathbf{Z})}.$
(133)

Here, $\mathcal{T}^{\alpha}$ denotes the set of all non-constant functions
$T(\mathbf{x}):\mathbb{R}^{d}\rightarrow\mathbb{R}$ with
$E_{\mathbb{P}}[e^{(\alpha-1)\cdot T(\mathbf{X})}]<\infty$.