# Posterior Covariance Information Criterion

Yukito Iba and Keisuke Yano  The Institute of Statistical Mathematics, 10-3
Midori cho, Tachikawa City, Tokyo, 190-8562, Japan

###### Abstract.

We introduce an information criterion, pcic, for predictive evaluation based
on quasi-posterior distributions. It is regarded as a natural generalisation
of the widely applicable information criterion (waic) and can be computed via
a single Markov chain Monte Carlo run. pcic is useful in a variety of
predictive settings that are not well dealt with in waic, including weighted
likelihood inference and quasi-Bayesian prediction.
Keywords. Bayesian statistics; Markov chain Monte Carlo; Predictive model
selection; covariate-shift adaptation; causal inference; quasi-Bayesian
prediction

##  1\. Introduction

Predictive model selection is a central topic in statistical science. In the
early 1970s, Akaike’s information criterion (aic; Akaike, 1971, 1973, 1974)
and Mallow’s $C_{p}$ (Mallows, 1973) initiated a vast area of researches in
the following fifty years. Various information criteria have been proposed to
assess the quality of out-of-sample prediction, for example, Takeuchi’s
information criterion (tic; Takeuchi, 1976), generalised information criterion
(gic; Konishi & Kitagawa, 1996), focused information criterion (fic; Claeskens
& Hjort, 2003), and criteria based on bootstrap samples (eic and “optimism”;
Ishiguro et al., 1997, Efron, 2004). There are also various types of cross-
validatory assessments proposed for this purpose (e.g., Gelfand & Dey, 1994;
Vehtari et al., 2017; Fong & Holmes, 2020).

Bayesian predictive model assessment is a problem of recent interest in this
field (Gelman et al., 2013). Because Markov chain Monte Carlo (mcmc) becomes a
common tool for Bayesian inference, it is highly convenient to evaluate model
performance using posterior samples generated by mcmc. Pioneering proposals in
this direction include the deviance information criterion (dic; Spiegelhalter
et al., 2002), Bayesian leave-one-out cross validation (Bayesian-loocv;
Gelfand & Dey, 1994; Vehtari et al., 2017), and widely applicable information
criterion (waic; Watanabe, 2010b, 2018b). Bayesian-loocv enables a direct
evaluation of the predictive distribution using samples from a single run of
mcmc with original data; it is not necessary to perform extra simulations with
“leave-one-out” data. In addition, waic has a smaller variance in the Monte
Carlo estimator (Watanabe, 2018b) even when the posterior density is sensitive
to a leave-one-out procedure (Peruggia, 1997), whereas it is asymptotically
equivalent to Bayesian-loocv (Watanabe, 2010a, 2018a).

The derivation of waic (Watanabe, 2018b) implies that the information required
for evaluating the predictive ability of a model is contained in posterior
distributions and can be extracted in the form of posterior covariance, at
least in the lowest order of an asymptotic expansion with respect to the
sample size. This principle is also seen in sensitivity estimation from mcmc
outputs (e.g., Pérez et al., 2006; Giordano et al., 2018). This study aims to
further pursue this idea and generalise waic for a wide range of predictive
settings, including covariate-shift adaptation (e.g., Shimodaira, 2000;
Sugiyama et al., 2007), counterfactual prediction (e.g., Platt et al., 2013;
Baba et al., 2017), and quasi-Bayesian prediction (e.g., Konishi & Kitagawa,
1996; Yin, 2009).

To treat these applications in a unified manner, we develop posterior
covariance information criterion (pcic) and present the theoretical property
(Theorem 2.1) in §2. pcic allows predictive settings where a score function
for the training is not necessarily the full likelihood. In such cases, the
working posterior is known as a quasi-posterior distribution (Chernozhukov &
Hong, 2003), a Gibbs posterior distribution (Zhang, 2006; Jiang & Tanner,
2008), or generalised posterior distribution (Bissiri et al., 2016), and has
been growing attentions in a variety of applied fields (e.g., Wang et al.,
2015; Guedj, 2019; Loaiza-Maya et al., 2021).

In §3, we illustrate the applications of pcic to several diversified
predictive scenarios, which is the core of this study. In §4 we prove Theorem
2.1, highlighting the connection between pcic and loocv. §5 concludes the
paper.

##  2\. Posterior Covariance Information Criterion

In this section, we formally present pcic and its basic properties in a
generic form. In the subsequent section, we present how to utilise it in
various predictive settings.

Let us begin with a quasi-Bayesian framework for pcic. Suppose that we have
$n$ independent and identically distributed observations
$X=(X_{1},\ldots,X_{n})$ from a sample space $\mathcal{X}$ and weights
$\\{w_{i}>0:i=1,\ldots,n\\}$ for observations. On the basis of these
observations, we define a quasi-posterior density on $\Theta$ in
$\mathbb{R}^{d}$ associated with arbitrary observation-wise score functions
$\\{s_{i}(\cdot,\cdot):i=1,\ldots,n\\}$ and a prior density $\pi(\theta)$ as:

$\displaystyle\pi(\theta\,;\,X)=\frac{\exp\\{\sum_{i=1}^{n}s_{i}(X_{i},\theta)\\}\pi(\theta)}{\int\exp\\{\sum_{i=1}^{n}s_{i}(X_{i},\theta^{\prime})\\}\pi(\theta^{\prime})d\theta^{\prime}}.$

Examples of observation-wise score functions are illustrated in §3. Using
this, we define the quasi-Bayesian predictive densities of $X_{i}$s as:

$\displaystyle h_{i,\pi}(\widetilde{X}_{i}\mid X):=\int
h_{i}(\widetilde{X}_{i}\mid\theta)\pi(\theta\,;\,X)d\theta,\quad
i=1,\ldots,n,$

where $\\{h_{i}(\cdot\mid\theta)\,:\,i=1,\ldots,n\\}$ is a set of the
observation-wise probability densities parameterised by $\theta\in\Theta$. Our
goal is to estimate the weighted quasi-Bayesian generalisation error

$\displaystyle
G_{n}:=\mathbb{E}_{\widetilde{X}}\left[-\frac{1}{n}\sum_{i=1}^{n}w_{i}\log
h_{i,\pi}(\widetilde{X}_{i}\mid X)\right],$

where $\widetilde{X}=(\widetilde{X}_{1},\ldots,\widetilde{X}_{n})$ is an
independent copy of $X$ and $\mathbb{E}_{\widetilde{X}}$ is the expectation
with respect to $\widetilde{X}$. To this end, we introduce the posterior
covariance information criterion (pcic) defined by

pcic
$\displaystyle=-\sum_{i=1}^{n}\frac{w_{i}}{n}\log\mathbb{E}_{\mathrm{pos}}[h_{i}(X_{i}\,\mid\,\theta)]\,+\,\sum_{i=1}^{n}\frac{w_{i}}{n}\,\mathrm{Cov}_{\mathrm{pos}}\left[\log
h_{i}(X_{i}\,\mid\,\theta)\,,\,s_{i}(X_{i}\,,\,\theta)\right],$ (1)

where let operations $\mathbb{E}_{\mathrm{pos}}$ and
$\mathrm{Cov}_{\mathrm{pos}}$ denote expectation and covariance with respect
to the quasi-posterior distribution, respectively. An important feature of
pcic is that the training phase does not necessarily employ the full
likelihood as the validation phase, which enable us to treat a wide range of
predictive settings, as illustrated in §3.

The following theorem shows that pcic is an asymptotically unbiased estimator
of the quasi-Bayesian generalisation error $G_{n}$.

######  Theorem 2.1.

Under regularity conditions in Appendix A, we have

$\displaystyle\mathbb{E}_{X}[G_{n}]-\mathbb{E}_{X}[\textsc{pcic}]=o(n^{-1}),$

where $\mathbb{E}_{X}$ is the expectation with respect to $X$.

Remark. pcic is a natural generalisation of waic

$\displaystyle\textsc{waic}:=-\frac{1}{n}\sum_{i=1}^{n}\log\mathbb{E}_{\mathrm{pos}}[h(X_{i}\mid\theta)]+\frac{1}{n}\sum_{i=1}^{n}\mathbb{V}_{\mathrm{pos}}[\log
h(X_{i}\mid\theta)],$

where $\log h(\cdot\mid\cdot)$ is a log likelihood, and
$\mathbb{V}_{\mathrm{pos}}$ denotes variance with respect to the posterior
distribution. This is verified by setting all score functions to the log
likelihood
$s_{1}(\cdot\,,\,\cdot)=s_{2}(\cdot\,,\,\cdot)=\cdots=s_{n}(\cdot\,,\,\cdot)=\log
h(\cdot\,\mid\,\cdot)$ and setting $w_{1}=\cdots=w_{n}=1$. Theorem 2.1 may
hold for some singular models, that is, statistical models without some of
regularity conditions in Appendix A, as is the case with waic, while in this
paper we restrict ourselves to regular cases satisfying the conditions in
Appendix A.

##  3\. Applications

This section gives three applications of pcic: The first two scenarios deal
with weighted likelihood inference; the last example deals with model
selection for quasi-Bayesian prediction.

###  3.1. Covariate shift adaptation

In a regression scenario, the distributions of covariates are often changed
between the training and test phases in real-world problems; such situation is
known as covariate shift (Shimodaira (2000)) and has attracted considerable
attention in a wide range of applications such as bioinformatics, spam
filtering, brain-computer interfacing, and econometrics (e.g., Sugiyama et
al., 2007; Yamazaki et al., 2007).

Applying pcic to the Bayesian prediction under covariate shift enables the
evaluation of the predictive performance of models with a single mcmc run
based on the original data. This has a considerable advantage over the
conventional cross-validatory assessment (Sugiyama et al., 2007), where
training should be conducted repeatedly for each training set. We are also
free from model-by-model analytical calculations of information quantities
appearing in the bias correction term, which potentially distracts domain
users from using information criteria to test their own models.

Let $\\{(Y_{i},X_{i}):i=1,\ldots,n\\}$ be i.i.d. pairs of a response variable
and a covariate, and we assume a parametric model $\\{h(Y\mid
X\,,\,\theta):\theta\in\Theta\\}$ for a response variable $Y$ given a
covariate $X$. We introduce a pair of distributions $p_{\mathrm{train}}(x)$
and $p_{\mathrm{test}}(x)$ that express the distributions of the covariate
$X_{i}$s in the training and test phases, respectively. For simplicity’s sake,
we assume that the ratio $r(x):=p_{\mathrm{test}}(x)/p_{\mathrm{train}}(x)$ is
known. Then, the quasi-posterior distribution associated with $r(X_{i})$s is
given by

$\displaystyle\pi_{\lambda}(\theta\,;\,\\{Y_{i},X_{i}\\})\propto\exp\left\\{\sum_{i=1}^{n}r^{\lambda}(X_{i})\,\log
h(Y_{i}\mid X_{i}\,,\,\theta)\right\\}\pi(\theta),$

where $\pi(\theta)$ denotes the prior density. According to the previous
studies (Shimodaira, 2000; Sugiyama et al., 2007), we introduce tilting
parameter $\lambda\geq 0$ to control the trade-off between consistency and
stability; if $\lambda=1$, the working quasi-likelihood in the training phase
is unbiased to the full likelihood in the test phase, but the variance of the
estimated model can be large (Shimodaira (2000)). Meanwhile, setting
$\lambda=0$ corresponds to the Bayesian inference using the unweighted
likelihood, which usually leads to inconsistent but stable results. pcic in
this setup is derived from the generic expression (1) by setting
$w(X_{i})=r(X_{i})$, $s_{i}(Y_{i},X_{i},\theta)=r^{\lambda}(X_{i})\,\log
h(Y_{i}\mid X_{i}\,,\,\theta)$, which is given as

$\displaystyle\textsc{pcic}=-\frac{1}{n}\sum_{i=1}^{n}r(X_{i})\log\mathbb{E}_{\mathrm{pos}}[h(Y_{i}\mid
X_{i}\,,\,\theta)]\,+\,\frac{1}{n}\sum_{i=1}^{n}r^{1+\lambda}(X_{i})\,\mathbb{V}_{\mathrm{pos}}\left[\log
h(Y_{i}\mid X_{i}\,,\,\theta)\right].$

We can select the value of the tilting parameter $\lambda$ by minimising pcic.

Figure 1.  An example of the use of pcic in covariate shift adaptation: (a)
The posterior mean of $f_{\theta}(x)$ and the $95\%$ credible interval for
pcic-best $\lambda$ are shown by a blue solid line and two dashed lines,
respectively, while those for waic-best $\lambda$ are shown by red solid and
dashed lines, respectively. The training and test data are shown as black and
green circles, respectively. (b) pcic, waic, and the generalisation error for
$\lambda\in\\{0.01i\,:\,i=1,\ldots,200\\}$. (c) Enlarged plot of pcic curve.

We illustrate the proposed method by selecting the tilting parameter $\lambda$
in a regression problem described in Sugiyama et al. (2007); a set of
artificial data by

$\displaystyle Y_{i}=\mathrm{sinc}(X_{i})+\varepsilon_{i},\quad
i=1,\ldots,n,n+1,\ldots,n+m,$

where $\varepsilon_{i}$s follows $N(0,\sigma^{2})$ with
$\sigma^{2}=(0.25)^{2}$. The first $n=50$ covariates
$\\{X_{i}:i=1,\ldots,n\\}$ from $N(0,1)$ and the latter $m=50$ covariates
$\\{X_{i+n}:i=1,\ldots,m\\}$ from $N(0.5,(0.3)^{2})$ are used as the training
and test data, respectively. We assume a linear regression model
$f_{\theta}(x)=\theta_{1}+\theta_{2}x$ and the prior distribution
$\pi(\theta)=N(0,I_{2})$, where $I_{2}$ is the $2\times 2$ identity matrix.

Figure 1 shows the comparison of pcic and waic; here waic uses the unweighted
likelihood, and both are defined using quasi-posterior samples. Figure 1 (a)
displays means and their uncertainties of the pcic and waic-best quasi-
posterior distribution with the sets of training and test data. Figure 1 (b)
showcases the values of pcics, waics, and the quasi-Bayesian generalisation
error $(1/m)\sum_{i=n+1}^{n+m}\left\\{-\log h_{i,\pi_{\lambda}}(Y_{i}\mid
X_{i})\right\\}$. Figure 1 (c) provides an enlarged plot of pcic curve which
emphasizes the bias-variance trade-off of choosing $\lambda$. pcic is close to
the quasi-Bayesian generalisation error, whereas waic cannot accommodate the
change in $\lambda$ (Figure 1 (b)). These features are reflected in the
resultant predictions shown in Figure 1 (a).

###  3.2. Causal inference

Causal inference and predictive model assessment have long been developed as
almost separate fields. However, there is a growing interest in predictive
model evaluation in counterfactual situations. Several criteria for selecting
the marginal structural model (Robins et al., 2000) have been developed, for
example, cross-validatory assessment (Brookhart & der Laan, 2006), focused
information criterion (Vansteelandt et al., 2012), quasi-likelihood
information criterion (Platt et al., 2013, Taguri & Matsuyama, 2013), and a
$C_{p}$ criterion (Baba et al., 2017).

Here, we develop a quasi-Bayesian version of the $C_{p}$ criterion in Baba et
al. (2017) by pcic, and show that pcic successfully works in this setting. We
focus on the case of inverse probability weighted (ipw) estimation using the
propensity score (Rosenbaum & Rubin, 1983), and utilise a Bayesian framework
developed in recent studies (Hoshino, 2008; Saarela et al., 2015; Roy et al.,
2017).

We use the following notation: $H$ and $n$ denote the number of treatments and
sample size, respectively. The observed data consist of a set of quartets
$\\{(Y_{i},X_{i},T_{i},Z_{i})\,:\,i=1,\ldots,n\\}$, where $i$ denotes an
individual, $Y_{i}$ represents an observed outcome,
$X_{i}=(X_{i}^{(1)},\ldots,X_{i}^{(H)})$ are the set of baseline covariates,
$T_{i}=(T_{i}^{(1)},\ldots,T_{i}^{(H)})\in\\{0,1\\}^{H}$ are treatment
assignments ($T_{i}^{(h)}=1$ means treatment $h$ is applied and $0$
otherwise), and $Z_{i}$ is a confounder vector.

To represent a counterfactual setting, we introduce a potential outcome
$Y^{(h)}=\\{Y_{i}^{(h)}:i=1,\ldots,n\\}$ with the treatment $h$ being applied;
an observed outcome $Y_{i}$ equals to $Y_{i}^{(h)}$ when the treatment $h$ is
applied to the individual $i$. Then, $h(\cdot\mid X,Z,\theta)$ is assumed to
be a conditional parametric density of a potential outcome given a covariate
and a confounder. Assuming that the propensity score
$e_{i}^{(h)}:=\mathbb{P}(T_{i}^{(h)}=1\mid Z_{i})$ is known, the quasi-
posterior distribution is defined as

$\displaystyle\pi(\theta\,;\,\\{(Y_{i},X_{i},T_{i},Z_{i}):i=1,\ldots,n\\})\propto\exp\left\\{\sum_{i=1}^{n}\sum_{h=1}^{H}\frac{T_{i}^{(h)}}{e_{i}^{(h)}}\log
h(Y_{i}^{(h)}\mid X_{i}^{(h)}\,,\,Z_{i}\,,\,\theta)\right\\}\pi(\theta),$

where $\pi(\theta)$ is a prior density.

Here we adapt the prediction in the original population and consider a
weighted loss

wloss
$\displaystyle:=\mathbb{E}_{\widetilde{Y}}\left[-\sum_{i=1}^{n}\sum_{h=1}^{H}\frac{T_{i}^{(h)}}{e_{i}^{(h)}}\log
h_{\pi}(\widetilde{Y}_{i}^{(h)}\mid
Y_{1},\ldots,Y_{n}\,,\,X_{i}^{(h)}\,,\,Z_{i})\right]$

with $\widetilde{Y}_{i}$ being an independent copy of $Y_{i}$. Expectation of
wloss corresponds to a Bayesian predictive version of wrisk in Baba et al.
(2017).

Figure 2. Application of pcic to the causal inference: (a) the correspondence
between ${\textsc{pcic}}_{\mathrm{w}}$ and $\mathrm{w}C_{p}$. (b) Comparison
of the bias correction terms.

Now, we can apply the generic theory in §2, which offers an information
criterion using the weighted loss for quasi-Bayesian prediction:

$\displaystyle{\textsc{pcic}}_{\mathrm{w}}=$
$\displaystyle-\frac{1}{n}\sum_{i=1}^{n}\sum_{h=1}^{H}\frac{T_{i}^{(h)}}{e_{i}^{(h)}}\log\mathbb{E}_{\mathrm{pos}}[h(Y^{(h)}_{i}\mid
X^{(h)},Z_{i},\theta)]$
$\displaystyle\,+\,\frac{1}{n}\sum_{i=1}^{n}\sum_{h=1}^{H}\left(\frac{T_{i}^{(h)}}{e_{i}^{(h)}}\right)^{2}\,\mathbb{V}_{\mathrm{pos}}\left[\log
h(Y^{(h)}_{i}\mid X^{(h)}_{i}\,,\,Z_{i}\,,\,\theta)\right].$

We compare ${\textsc{pcic}}_{\mathrm{w}}$ to $\mathrm{w}C_{p}$ of the ipw
estimators given in Baba et al. (2017), where $\mathrm{w}C_{p}$ is
asymptotically unbiased to wrisk of ipw estimators for linear regression
models with variance known. Figure 2 compares ${\textsc{pcic}}_{\mathrm{w}}$
and $\mathrm{w}C_{p}$ when we employ the linear regression setup described in
Baba et al. (2017). We create samples as

$\displaystyle
Y^{(h)}_{i}=1+x^{(h)}+0.5(x^{(h)})^{2}+z+\epsilon,\,i=1,\ldots,n=50,\,h=1,\ldots,H=6,$

where $z$ and $\epsilon$ are independently distributed according to
$\mathrm{Un}(-3^{1/2},3^{1/2})$ and $N(0,1)$, respectively. Here we set the
density of $N(\theta_{1}+\theta_{2}x^{(h)}+\theta_{3}(x^{(h)})^{2},2)$ to
$h(Y\mid x^{(h)},Z,\theta)$, and set $N(0,1000\cdot I_{3\times 3})$ to the
prior distribution, where $I_{3\times 3}$ is the $3\times 3$ identity matrix.
The values of ${\textsc{pcic}}_{\mathrm{w}}$ and $\mathrm{w}C_{p}$ (and their
bias-correction terms) are very similar as shown in Figure 2.

###  3.3. Quasi-Bayesian prediction

The usage of a surrogate score function instead of the full likelihood has
been discussed when it is difficult or reluctant to work with the full
likelihood in the training step, or when robustness against contamination,
data heterogeneity, or model misspecifications are considered (e.g., Yu &
Moyeed, 2001; Yin, 2009; Sriram et al., 2013; Giummolè et al., 2019). Model
selection in this context has been discussed in several ways; for example, see
Konishi & Kitagawa (1996); Heritier et al. (2009); Dawid & Musio (2015).

Here, we aim to compare several models such as $M$ candidates
$\\{h^{(m)}(\cdot\,\mid\,\theta):\theta\in\Theta_{m}\\}$, $m=1,\ldots,M$, when
working with a quasi-posterior distribution on the basis of surrogate score
functions $\\{s^{(m)}(\cdot\,,\,\theta):\theta\in\Theta_{m}\\}$,
$m=1,\ldots,M$. We propose the model selection method of minimising pcic given
by

$\displaystyle\textsc{pcic}=-\frac{1}{n}\sum_{i=1}^{n}\log\mathbb{E}_{\mathrm{pos}}[h^{(m)}(X_{i}\mid\theta)]+\frac{1}{n}\sum_{i=1}^{n}\mathrm{Cov}_{\mathrm{pos}}[\log
h^{(m)}(X_{i}\mid\theta),s^{(m)}(X_{i},\theta)],$

where $X_{i}$s are i.i.d. observations. Below, we illustrate the application
of pcic and compare it with the model selection method of minimising waic:

$\displaystyle\textsc{waic}=-\frac{1}{n}\sum_{i=1}^{n}\log\mathbb{E}_{\mathrm{pos}}[h^{(m)}(X_{i}\mid\theta)]+\frac{1}{n}\sum_{i=1}^{n}\mathbb{V}_{\mathrm{pos}}[\log
h^{(m)}(X_{i}\mid\theta)].$

For the comparison, we create samples from $Y_{i}=\theta+\varepsilon_{i}$,
$i=1,\ldots,N$, with the common distribution of $\varepsilon_{i}$s either the
standard normal distribution or the Cauchy distribution. We work with the
quasi-posterior distribution based on the Laplace distribution. We varied the
sample size $N$ from $10$ to $100$ and repeated the experiment of selecting
the model among the normal, the Laplace, and the Cauchy distributions 100
times. Table 1 shows that pcic works better than waic especially when the
sample size is small. Figure 3 presents boxplots of the generalization errors,
pcics, and waics for the normal, Laplace, and Cauchy models when the true
error distribution is the standard normal distribution and the sample size is
$10$. For each model, the mean of pcic is closer to that of the generalisation
error than that of waic. In addition, for the normal model, the variance of
pcic is smaller than that of waic. These are considered to result in better
model selection when the sample size is small.

Table 1. Selection frequencies for the models selected by waic and pcic among the normal, the Laplace, and the Cauchy distributions. True distribution of $\varepsilon_{i}$ | $N$ | Criterion | Normal | Laplace | Cauchy
---|---|---|---|---|---
Normal | 10 | waic | 81 | 18 | 1
|  | pcic | 90 | 8 | 2
Normal | 20 | waic | 87 | 13 | 0
|  | pcic | 90 | 9 | 1
Normal | 100 | waic | 99 | 1 | 0
|  | pcic | 99 | 1 | 0
Cauchy | 10 | waic | 5 | 40 | 55
|  | pcic | 9 | 22 | 69
Cauchy | 20 | waic | 2 | 17 | 81
|  | pcic | 2 | 14 | 84
Cauchy | 100 | waic | 0 | 3 | 97
|  | pcic | 0 | 2 | 98
Figure 3. Application of pcic to the qausi-Bayesian prediction when the true
error distribution is the standard normal distribution: (a) Boxplots and the
means (black points) of the generalisation error, pcic, waic for the normal
model. The dash line denotes the mean of the generalisation error. (b) Those
for the Laplace model. (c) Those for the Cauchy model.

Table 1 implies that waic also works when the sample size is large. The
following proposition shows that if the surrogate score function forms an
unbiased estimating function (Godambe, 1976), pcic is asymptotically
equivalent to waic in expectation. The proof is provided in Appendix C.

######  Proposition 3.1.

Assume that the regularity conditions in Appendix A hold. If the specified
model $\\{h(\cdot\mid\theta):\theta\in\Theta\\}$ contains the true
distribution and the gradient of the surrogate score function
$s(\cdot,\theta)$ is an unbiased estimating function, that is,

$\displaystyle\int\nabla_{\theta}s(X,\theta)h(X\mid\theta)dX=0\,\,\text{for
all $\theta\in\Theta$},$

then the bias correction term of pcic is reduced to that of waic:

$\displaystyle\mathbb{E}_{X}\left[\frac{1}{n}\sum_{i=1}^{n}\mathrm{Cov}_{\mathrm{pos}}[\log
h(X_{i}\mid\theta),s(X_{i},\theta)]\right]=\mathbb{E}_{X}\left[\frac{1}{n}\sum_{i=1}^{n}\mathbb{V}_{\mathrm{pos}}[\log
h(X_{i}\mid\theta)]\right]+o(n^{-1}).$

Because the surrogate score function in the numerical experiment satisfies the
condition in the above proposition, the difference between waic and pcic is
asymptotically negligible for the correctly specified model. However, it
considerably affects the model selection with a finite sample size, as shown
in the above example.

For predictive model evaluation based on surrogate score functions, Konishi &
Kitagawa (1996) proposes

$\displaystyle\textsc{gic}=-\frac{1}{n}\sum_{i=1}^{n}\log
h(X_{i}\mid\widehat{\theta}_{S})+\mathrm{tr}\Bigg{\\{}J_{S}^{-1}\mathbb{E}_{X}[\nabla_{\theta_{S}}s(X,\theta)\\{\nabla_{\theta_{S}}\log
h(X,\theta)\\}^{\top}]\Bigg{\\}},$

where $\widehat{\theta}_{S}$ is an $M$-estimate
$\sum_{i=1}^{n}s(X_{i},\widehat{\theta}_{S})=\max_{\theta\in\Theta}\sum_{i=1}^{n}s(X_{i},\theta)$,
$\theta_{S}$ is the point
$\mathbb{E}_{X}[s(X\,,\,\theta_{S})]=\max_{\theta\in\Theta}\mathbb{E}_{X}[s(X\,,\,\theta)]$,
$\nabla_{\theta_{S}}$ is the gradient with respect to $\theta$ evaluated at
$\theta=\theta_{S}$, and $J_{S}$ is
$-\mathbb{E}_{X}[\nabla^{2}_{\theta_{S}}s(X,\theta)]$. gic relaxes the
underlying assumption of aic that estimation is by maximum likelihood. An
advantage of our approach is that pcic can circumvent model-by-model analytic
derivative calculations required for gic.

##  4\. Derivation

This section provides a derivation of pcic. Here, we use a weighted quasi-
Bayesian version of importance sampling loocv (hereafter, $\textsc{is-
cv}_{\mathrm{wq}}$) as the stepping stone of the proof.

We first define $\textsc{is-cv}_{\mathrm{wq}}$ by:

$\displaystyle\textsc{is-
cv}_{\mathrm{wq}}=-(1/n)\sum_{i=1}^{n}w_{i}\log\mathbb{E}_{\mathrm{pos}}[h_{i}(X_{i}\mid\theta)\mid
X_{-i}],$

where $\mathbb{E}_{\mathrm{pos}}[\cdot\mid X_{-i}]$ means the
leave-$X_{i}$-out posterior expectation. It is also expressed as

$\displaystyle\textsc{is-
cv}_{\mathrm{wq}}=-\frac{1}{n}\sum_{i=1}^{n}w_{i}\log\mathbb{E}_{\mathrm{pos}}\left[\mathrm{e}^{\log
h_{i}(X_{i})-s_{i}(X_{i},\theta)}\right]+\frac{1}{n}\sum_{i=1}^{n}w_{i}\log\mathbb{E}_{\mathrm{pos}}\left[\mathrm{e}^{-s_{i}(X_{i},\theta)}\right].$
(2)

Then, the following theorem holds. The proof of this theorem is given in
Appendix B.

######  Theorem 4.1.

Assume the regularity conditions in Appendix A hold. Then the difference
between $\textsc{is-cv}_{\mathrm{wq}}$ and $G_{n}$ is negligible in
expectation: $\mathbb{E}_{X}[G_{n}]-\mathbb{E}_{X}[\textsc{is-
cv}_{\mathrm{wq}}]=o(n^{-1})$.

Next, we prove the following theorem stating that pcic and $\textsc{is-
cv}_{\mathrm{wq}}$ are asymptotically equivalent in probability. The following
proof is a generalisation of the equivalence of loocv and waic (Watanabe,
2010b, 2018a) that utilises a functional cumulant expansion.

######  Theorem 4.2.

Assume that the regularity conditions in Appendix A hold. Then $\textsc{is-
cv}_{\mathrm{wq}}$ and pcic are asymptotically equivalent in probability:
$\textsc{pcic}=\textsc{is-cv}_{\mathrm{wq}}+o_{P}(n^{-1})$.

###### Proof of Theorem 4.2.

The proof employs the functional cumulant expansion developed in Watanabe
(2010a) with the proof presented in Appendix C:

######  Lemma 4.3.

Assume that the regularity conditions in Appendix A hold. Then, we have the
cumulant expansion

$\displaystyle\log\mathbb{E}_{\mathrm{pos}}\left[\mathrm{e}^{f(X_{i},\theta)}\right]=\mathbb{E}_{\mathrm{pos}}[f(X_{i},\theta)]+(1/2)\mathbb{V}_{\mathrm{pos}}[f(X_{i},\theta)]+o_{P}(n^{-1}),\,\,i=1,\ldots,n,$

where $f(X_{i},\theta)$ is either $\log
h_{i}(X_{i}\mid\theta)-s_{i}(X_{i},\theta)$, $-s_{i}(X_{i},\theta)$, or $\log
h_{i}(X_{i}\mid\theta)$.

From Lemma 4.3 with $f(X_{i},\theta)=\log
h_{i}(X_{i}\mid\theta)-s_{i}(X_{i},\theta)$ and
$f(X_{i},\theta)=-s_{i}(X_{i},\theta)$, the right hand side of (2) is equal to

$\displaystyle-\frac{1}{n}\sum_{i}^{n}w_{i}\mathbb{E}_{\mathrm{pos}}[\log
h_{i}(X_{i}\mid\theta)]-\frac{1}{2n}\sum_{i}^{n}w_{i}\mathbb{V}_{\mathrm{pos}}[\log
h_{i}(X_{i}\mid\theta)]$
$\displaystyle\quad\quad\quad+\frac{1}{n}\sum_{i=1}^{n}w_{i}\mathrm{Cov}_{\mathrm{pos}}[\log
h_{i}(X_{i}\mid\theta),s_{i}(X_{i},\theta)]+o_{P}(n^{-1}).$ (3)

Using Lemma 4.3 with $f(X_{i},\theta)=\log h_{i}(X_{i}\mid\theta)$, the first
line of the formula (3) is expressed as

$\displaystyle-\frac{1}{n}\sum_{i=1}^{n}w_{i}\log\mathbb{E}_{\mathrm{pos}}[h_{i}(X_{i}\mid\theta)]+o_{P}(n^{-1}).$
(4)

By substituting (4) in (3), the resultant expression shows $\textsc{is-
cv}_{\mathrm{wq}}$ is asymptotically equal to pcic. ∎

###### Proof of Theorem 2.1.

Combining Theorem 4.1 with Theorem 4.2 completes the proof. ∎

##  5\. Conclusion

We have proposed an information criterion, pcic, which generalises waic to
diversified predictive settings including covariate shift adaptation, causal
inference, and predictions with surrogate score functions. pcic shares the
favorable features of waic: it can be computed with a single run of mcmc and
is asymptotically unbiased to the Bayesian generalisation error.

A principle elucidated by this study is that information necessary for
predictive evaluation of Bayesian methods is contained in the posterior
distribution and is represented as a form of the posterior covariance. As
shown in our examples, this observation is ubiquitously applicable in various
predictive settings.

## Acknowledgement

The authors would like to thank Yoshiyuki Ninomiya and Yusaku Ohkubo for
fruitful discussions. The authors would also like to thank Shintaro Hashimoto
for his helpful comments on the early version of the manuscript. We used an R
package bayesQR (Benoit et al., 2017). This work was partly supported by JST
KAKENHI 19K20222, 21K12067, and JST CREST JPMJCR1763.

## References

  * Akaike (1971) Akaike, H. (1971).  Determination of the number of factors by an extended maximum likelihood principle.  Research Memo. 44, Institute of Statistical Mathematics.
  * Akaike (1973) Akaike, H. (1973).  Information theory and an extension of the maximum likelihood principle.  In Proceedings of the 2nd Intertnational Symposium on Information Theory, B. Petrov & F. Csáki, eds.
  * Akaike (1974) Akaike, H. (1974).  A new look at the statistical model identification.  IEEE Transaction on Automatic Control 19, 716–723.
  * Baba et al. (2017) Baba, T., Kanamori, T. & Ninomiya, Y. (2017).  A $C_{p}$ criterion for semiparametric causal inference.  Biometrika 104, 845–861.
  * Benoit et al. (2017) Benoit, D., Al-Hamzawi, R., Yu, K. & den Poel, D. V. (2017).  bayesQR: Bayesian Quantile Regression.  R package https://CRAN.R-project.org/package=bayesQR.
  * Bissiri et al. (2016) Bissiri, P., Holmes, C. & Walker, S. (2016).  A general framework for updating brief distributions.  Journal of the Royal Statistical Society: Series B (Statistical Methodology) 78, 1103–1130.
  * Brookhart & der Laan (2006) Brookhart, M. & der Laan, M. V. (2006).  A semiparametric model selection criterion with applications to the marginal structural model.  Computational Statistics $\&$ Data Analysis 50, 475–498.
  * Chernozhukov & Hong (2003) Chernozhukov, V. & Hong, H. (2003).  An MCMC approach to classical estimation.  Journal of Econometrics 115, 293–346.
  * Claeskens & Hjort (2003) Claeskens, G. & Hjort, N. (2003).  The focused information criterion.  Journal of the American Statistical Association 98, 879–899.
  * Dawid & Musio (2015) Dawid, P. & Musio, M. (2015).  Bayesian model selection based on proper scoring rules.  Bayesian Analysis 10, 479–499.
  * Efron (2004) Efron, B. (2004).  The estimation of prediction error: Covariance penalties and cross-validation.  Journal of the American Statistical Association 99, 619–632.
  * Fong & Holmes (2020) Fong, E. & Holmes, C. (2020).  On the marginal likelihood and cross-validation.  Biometrika 107, 489–496.
  * Gelfand & Dey (1994) Gelfand, A. & Dey, D. (1994).  Bayesian model choice: Asymptotics and exact calculations.  Journal of the Royal Statistical Society: Series B 56, 501–514.
  * Gelman et al. (2013) Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A. & Rubin, D. (2013).  Bayesian Data Analysis, 3rd Edition.  Chapman & Hall/CRC.
  * Giordano et al. (2018) Giordano, R., Broderick, T. & Jordan, M. (2018).  Covariances, robustness, and variational Bayes.  Journal of Machine Learning Research 19, 1–49.
  * Giummolè et al. (2019) Giummolè, F., Mameli, V., Ruli, E. & Ventura, L. (2019).  Objective Bayesian inference with proper scoring rules.  TEST 28, 728–755.
  * Godambe (1976) Godambe, P. (1976).  Conditional likelihood and unconditional optimum estimating equations.  Biometrika 63, 277–284.
  * Guedj (2019) Guedj, B. (2019).  A primer on PAC-Bayesian learning.  arXiv:1901.05353.
  * Heritier et al. (2009) Heritier, S., Cantoni, E., Copt, S. & Victoria-Feser, M.-P. (2009).  Robust methods in biostatistics.  John Wiley & Sons Ltd.
  * Hoshino (2008) Hoshino, A. (2008).  A Bayesian propensity score adjustment for latent variable modeling and MCMC algorithm.  Computational Statistics & and Data Analysis 52, 1413–1429.
  * Ishiguro et al. (1997) Ishiguro, M., Sakamoto, Y. & Kitagawa, G. (1997).  Bootstrapping log likelihood and EIC, an extension of AIC.  Annals of the Institute of Statistical Mathematics 49, 411–434.
  * Jiang & Tanner (2008) Jiang, W. & Tanner, M. (2008).  Gibbs posterior for variable selection in high-dimensional classification and data mining.  The Annals of Statistics 36, 2207–2231.
  * Konishi & Kitagawa (1996) Konishi, S. & Kitagawa, G. (1996).  Generalised information criteria in model selection.  Biometrika 83, 875–890.
  * Loaiza-Maya et al. (2021) Loaiza-Maya, R., Martin, G. & Frazier, D. (2021).  Focused Bayesian prediction.  Journal of Applied Econometrics .
  * Mallows (1973) Mallows, C. (1973).  Some comments on $C_{p}$.  Technometrics 15, 661–675.
  * Pérez et al. (2006) Pérez, C., Martin, J. & Rufo, M. (2006).  Mcmc-based local parametric sensitivity estimations.  Computational Statistics & Data Analysis 51, 823–835.
  * Peruggia (1997) Peruggia, M. (1997).  On the variability of case-deletion importance sampling weights in the Bayesian linear model.  Journal of the Ametican Statistical Association 92, 199–207.
  * Platt et al. (2013) Platt, R., Brookhart, M., Cole, S., Westreich, D. & Schisterman, E. (2013).  An information criterion for marginal structural models.  Statistics in Medicine 32, 1383–1393.
  * Robins et al. (2000) Robins, J. M., Hernan, M. A. & Brumback, B. (2000).  Marginal structural models and causal inference in epidemiology.  Epidemiology 11, 550–560.
  * Rosenbaum & Rubin (1983) Rosenbaum, P. & Rubin, D. (1983).  The central role of the propensity score in observational studies for causal effects.  Biometrika 70, 41–55.
  * Roy et al. (2017) Roy, J., Lum, K. & Daniels, M. (2017).  A Bayesian nonparametric approach to marginal structural models for point treatments and a continuous or survival outome.  Biostatistics 18, 32–47.
  * Saarela et al. (2015) Saarela, O., Stephens, D., Moodie, E. & Klein, M. (2015).  On Bayesian estimation of marginal structural models.  Biometrics 71, 279–288.
  * Shimodaira (2000) Shimodaira, H. (2000).  Improving predictive inference under covariate shift by weighting the log-likelihood function.  Journal of Statistical Planning and Inference 90, 227–244.
  * Spiegelhalter et al. (2002) Spiegelhalter, D., Best, N., Carlin, B. & van der Linde, A. (2002).  Bayesian measures of model complexity and fit.  Journal of the Royal Statistical Society: Series B 64, 583–639.
  * Sriram et al. (2013) Sriram, K., Ramamoorthi, R. & Ghosh, P. (2013).  Posterior consistency of Bayesian quantile regression based on the misspecified asymmetric Laplace density.  Bayesian Analysis 8, 479–504.
  * Sugiyama et al. (2007) Sugiyama, M., Krauledat, M. & Müller, K.-R. (2007).  Covariate shift adaptation by importance weighted cross validation.  Journal of Machine Learning Research 8, 985–1005.
  * Taguri & Matsuyama (2013) Taguri, M. & Matsuyama, Y. (2013).  Comments on an information criterion for marginal structural models by R. W. Platt, M. A. Brookhart, S. R. Cole, D. Westreich, and E. F. Schisterman.  Statistics in medicine 32, 3590–3591.
  * Takeuchi (1976) Takeuchi, K. (1976).  Distribution of informational statistics and a criterion of model fitting (in japanese).  Suri-Kagaku (Mathematic Sciences) 153, 12–18.
  * Vansteelandt et al. (2012) Vansteelandt, S., Bekaert, M. & Claeskens, G. (2012).  On model selection and model misspecification in causal inference.  Statistical Methods in Medical Research 21, 7–30.
  * Vehtari et al. (2017) Vehtari, A., Gelman, A. & Gabry, J. (2017).  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC.  Statistics and Computing 27, 1413––1432.
  * Wang et al. (2015) Wang, Y.-X., Fienberg, S. & Smola, A. (2015).  Privacy for free: Posterior sampling and stochastic gradient monte carlo.  Proceeding of the 32nd International Conference on Machine Learning (ICML’15) , 2493–2502.
  * Watanabe (2010a) Watanabe, S. (2010a).  Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory.  Journal of Machine Learning Research 11, 3571–3594.
  * Watanabe (2010b) Watanabe, S. (2010b).  Equations of states in singular statistical estimation.  Neural Networks 23, 20–34.
  * Watanabe (2018a) Watanabe, S. (2018a).  Higher order equivalence of Bayes cross validation and WAIC.  In Proceedings of Mathmatics and Statistics, Information geometry and its applications, vol. 4.
  * Watanabe (2018b) Watanabe, S. (2018b).  Mathematical Theory of Bayesian Statistics.  Chapman & Hall/CRC.
  * Yamazaki et al. (2007) Yamazaki, K., Kawanabe, M., Watanabe, S., Sugiyama, M. & Müller, K.-R. (2007).  Asymptotic Bayesian generalization error when training and test distributions are different.  Proceedings of the 24th International Conference on Machine Learning (ICML’07) , 1079–1086.
  * Yin (2009) Yin, G. (2009).  Bayesian generalized method of moments.  Bayesian Analysis 4, 191–208.
  * Yoshida (2011) Yoshida, N. (2011).  Polynomial type large deviation inequalities and quasi-likelihood analysis for stochastic differential equations.  Annals of the Institute of Statistical Mathematics 63, 431–479.
  * Yu & Moyeed (2001) Yu, K. & Moyeed, R. (2001).  Bayesian quantile regression.  Statistics & Probability Letters 54, 437–447.
  * Zhang (2006) Zhang, T. (2006).  From $\varepsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation.  The Annals of Statistics 34, 2180–2210.

## Appendix

##  Appendix A Regularity conditions

We make the following assumptions. For $i=1,\ldots,d$, let $\theta_{i}$ denote
the $i$-th component of $\theta$ and let $\partial_{i}$ denote the partial
derivative with respect to $\theta^{i}$. Let $X_{1},\ldots,X_{n+1}$ be
independent observations, and let $\mathbb{E}$ denote the expectation with
respect to $X_{1},\ldots,X_{n+1}$. Let $\theta_{S}$ denote the point such that

$\displaystyle\mathbb{E}\left[\sum_{i=1}s_{i}(X_{i},\theta_{S})\right]=\max_{\theta\in\Theta}\mathbb{E}\left[\sum_{i=1}^{n}s_{i}(X_{i},\theta)\right]$

and let $\widehat{\theta}_{S}$ be defined by

$\displaystyle\sum_{i=1}^{n}s_{i}(X_{i},\widehat{\theta}_{S})=\max_{\theta\in\Theta}\sum_{i=1}^{n}s_{i}(X_{i},\theta),$

respectively. Let $\nabla_{\theta_{S}}$ denote the gradient with respect to
$\theta$ evaluated at $\theta=\theta_{S}$.

Let four information matrices be defined by

$\displaystyle I_{H}$
$\displaystyle:=\sum_{i=1}^{n}w_{i}\mathbb{E}[\\{\nabla_{\theta_{S}}\log
h_{i}(X_{i}\mid\theta)\\}\\{\nabla_{\theta_{S}}\log
h_{i}(X_{i}\mid\theta)\\}^{\top}]/n,$ $\displaystyle J_{H}$
$\displaystyle:=\sum_{i=1}^{n}w_{i}\mathbb{E}[-\\{\nabla^{2}_{\theta_{S}}\log
h_{i}(X_{i}\mid\theta)\\}]/n,$ $\displaystyle I_{S}$
$\displaystyle:=\sum_{i=1}^{n}\mathbb{E}[\\{\nabla_{\theta_{S}}s_{i}(X_{i}\mid\theta)\\}\\{\nabla_{\theta_{S}}s_{i}(X_{i}\mid\theta)\\}^{\top}]/n,\,\text{and}$
$\displaystyle J_{S}$
$\displaystyle:=\sum_{i=1}^{n}\mathbb{E}[-\\{\nabla^{2}_{\theta_{S}}s_{i}(X_{i}\mid\theta)\\}]/n,$

respectively. Let $\pi_{\mathrm{simp}}(\theta;X)$ be defined by

$\displaystyle\pi_{\mathrm{simp}}(\theta;X)=\frac{1}{\sqrt{(2\pi)^{d}\mathrm{det}(n^{-1}J^{-1}_{S})}}\exp\left\\{-\frac{n}{2}(\theta-\widehat{\theta}_{S})^{\top}J_{S}(\theta-\widehat{\theta}_{S})\right\\}.$

  1. R1

All derivatives of $\log h_{i}(x\mid\theta)$ and $s_{i}(x,\theta)$ up to order
3 exist for each observation $x$ and for $\theta$ in a neighbourhood of
$\theta_{S}$.

  2. R2

All moments exist for these derivatives in a neighborhood of $\theta_{S}$.

  3. R3

Four information matrices $I_{H},I_{S},J_{H},J_{S}$ are positive definite.

  4. R4

For the $M$-estimator $\widehat{\theta}_{S}$, we have

$\displaystyle\widehat{\theta}_{S}-\theta_{S}=(1/\sqrt{n})J_{S}^{-1}\sum_{i=1}^{n}\nabla_{\theta_{S}}s_{i}(X_{i},\theta)/\sqrt{n}+o_{P}(1),$

where the residual remains its order after taking the expectation.

  5. R5

Moments up to order 8 of the quasi posterior distribution are approximated by
those of $\pi_{\mathrm{simp}}(\theta;X)$ as

$\displaystyle\mathbb{E}_{\mathrm{pos}}\left[\prod_{j=1}^{8}(\theta-\theta_{S})_{i_{j}}\right]=\int\prod_{j=1}^{8}(\theta-\theta_{S})_{i_{j}}\pi_{\mathrm{simp}}(\theta;X)d\theta+o_{P}\left(n^{-(\sum_{j=1}^{8}i_{j})/2}\right),$

where $(i_{1},\ldots,i_{8})\in\mathbb{N}^{8}$ such that
$i_{1}+\cdots+i_{8}\leq 8$, and the $o_{P}$ terms remain their order after
taking the expectation.

  6. R6

For $i,j,k\in\\{1,\ldots,d\\}$ and for $a\in\\{1,\ldots,n\\}$, quantities

$\displaystyle\mathbb{E}\left\\{\mathbb{E}_{\mathrm{pos}}\left[\left\\{\int(1-u)^{2}(\partial_{ijk}\log
h_{a}(X_{n+1}\mid\theta))_{\theta=\theta_{S}+u(\theta-\theta_{S})}du\right\\}^{2}\right]\right\\}^{1/2}\quad\text{and}$
$\displaystyle\mathbb{E}\left\\{\mathbb{E}_{\mathrm{pos}}\left[\left\\{\int(1-u)(\partial_{ij}\log
h_{a}(X_{n+1}\mid\theta))_{\theta=\theta_{S}+u(\theta-\theta_{S})}du\right\\}^{2}\right]\right\\}^{1/2}$

are finite.

  7. R7

Let $g(X_{j};i,\theta)$ be either $\log h_{i}(X_{j})-s_{i}(X_{j},\theta)$,
$-s_{i}(X_{j},\theta)$, or $\log h_{i}(X_{j})$ for $i=1,\ldots,n$ and
$j=1,\ldots,n+1$. For each $j=1,\ldots,n+1$ and $i=1,\ldots,n$, and for
$\alpha\in[-1,1]$,

$\displaystyle\mathbb{E}\left[\int_{0}^{1}(1-u)^{2}\left(\frac{d^{3}}{d\beta^{3}}\log\mathbb{E}_{\mathrm{pos}}\left[\left(\frac{g(X_{j};i,\theta)}{g(X_{j};i,\theta_{S})}\right)^{\beta}\right]\right)_{\beta=u\alpha}du\right]=o(n^{-1}).$

  8. R8

For the weights, we have $\sup_{i}w_{i}=O(1)$.

We explain about these assumptions. Assumptions R1-R3 indicate that the target
statistical model is regular (Watanabe, 2018b). Assumption R4 means that the
$M$-estimator $\widehat{\theta}_{S}$ has an asymptotic linear form in the
sense of expectation, which can be shown by using the polynomial-type large
deviation inequality for $\widehat{\theta}_{S}-\theta_{S}$ (Yoshida, 2011).
Assumption R5 means that the quasi-posterior distribution can be approximated
by the normal distribution in the sense of moments, which is expected from the
Laplace expansion. Assumptions R6 and R7 are necessary for the mathematical
convenience (Watanabe, 2018a).

##  Appendix B Proof of Theorem 4.1

It suffices to analyse the order of the quasi-Bayesian generalisation error
$G_{n}$. We shall show under conditions in Appendix A, we have

$\displaystyle\mathbb{E}[G_{n}]$
$\displaystyle=\mathcal{L}(\theta_{S})+\frac{\mathrm{tr}(J_{H}J_{S}^{-1}I_{S}J_{S}^{-1})+\mathrm{tr}(J_{H}J_{S}^{-1})-\mathrm{tr}(I_{H}J_{S}^{-1})}{2n}+o(n^{-1}),$
(5)

where $\mathcal{L}(\theta):=\mathbb{E}[-(1/n)\sum_{i=1}^{n}w_{i}\log
h_{i}(X_{n+1}\mid\theta)]$.

We begin by applying the similar argument in Lemma 4.3 to obtain

$\displaystyle\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{n}\log\mathbb{E}_{\mathrm{pos}}[h_{i}(X_{n+1}\mid\theta)]\right]$
$\displaystyle=-\mathcal{L}(\theta_{S})+\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{n}\mathbb{E}_{\mathrm{pos}}\left[\log\frac{h_{i}(X_{n+1}\mid\theta)}{h_{i}(X_{n+1}\mid\theta_{S})}\right]\right]$
$\displaystyle\quad+\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{2n}\mathbb{V}_{\mathrm{pos}}\left[\log\frac{h_{i}(X_{n+1}\mid\theta)}{h_{i}(X_{n+1}\mid\theta_{S})}\right]\right]+o(n^{-1}).$
(6)

Observe that from Conditions R4-R5, we have

$\displaystyle\mathbb{E}_{\mathrm{pos}}[\theta-\theta_{S}]$
$\displaystyle=\widehat{\xi}/\sqrt{n}+o_{P}(n^{-1/2})\quad\text{and}$
$\displaystyle\mathbb{E}_{\mathrm{pos}}[(\theta-\theta_{S})(\theta-\theta_{S})^{\top}]$
$\displaystyle=J_{S}^{-1}/n\widehat{\xi}\widehat{\xi}^{\top}/n+o_{P}(n^{-1}),$

where
$\widehat{\xi}:=J_{S}^{-1}\sum_{i=1}^{n}\nabla_{\theta_{S}}s_{i}(X_{i},\theta)/\sqrt{n}$,
and the $o_{P}(n^{-1/2})$ and $o_{P}(n^{-1})$ terms remain their orders after
taking the expectation. With notations $\delta_{S}:=\theta-\theta_{S}$,
$\mathcal{H}^{(1)}_{i}(\cdot):=\nabla_{\theta_{S}}\log
h_{i}(\cdot\mid\theta)$, and
$\mathcal{H}^{(2)}_{i}(\cdot):=\nabla^{2}_{\theta_{S}}\log
h_{i}(\cdot\mid\theta)$, the Taylor expansion together with this observation
gives

$\displaystyle\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{n}\mathbb{E}_{\mathrm{pos}}\left[\log\frac{h_{i}(X_{n+1}\mid\theta)}{h_{i}(X_{n+1}\mid\theta_{S})}\right]\right]$
$\displaystyle=\sum_{i=1}^{n}\frac{w_{i}}{n}\mathbb{E}\mathbb{E}_{\mathrm{pos}}\left[\delta_{S}^{\top}\mathcal{H}^{(1)}_{i}(X_{n+1})\right]$
$\displaystyle\quad\quad+\sum_{i=1}^{n}\frac{w_{i}}{2n}\mathbb{E}\mathbb{E}_{\mathrm{pos}}\left[\delta_{S}^{\top}\mathcal{H}^{(2)}_{i}(X_{n+1})\delta_{S}\right]+o(n^{-1})$
$\displaystyle=-\frac{1}{2n}\mathrm{tr}(J_{H}J_{S}^{-1})-\frac{1}{2n}\mathrm{tr}\left(J_{H}J_{S}^{-1}I_{S}J_{S}^{-1}\right)+o(n^{-1}).$

Similarly, we have

$\displaystyle\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{n}\mathbb{V}_{\mathrm{pos}}\left[\log\frac{h_{i}(X_{n+1}\mid\theta)}{h_{i}(X_{n+1}\mid\theta_{S})}\right]\right]$
$\displaystyle=\mathbb{E}[\mathbb{E}_{\mathrm{pos}}[\delta_{S}^{\top}I_{H}\delta_{S}]-\mathbb{E}_{\mathrm{pos}}[\delta_{S}]^{\top}I_{H}\mathbb{E}_{\mathrm{pos}}[\delta_{S}]]+o(n^{-1})$
$\displaystyle=\frac{\mathrm{tr}(I_{H}J_{S}^{-1})}{n}+o(n^{-1}),$

which completes the proof. ∎

##  Appendix C Proofs of lemmas and propositions

###  C.1. Proof of Lemma 4.3

Let $f(X_{j};i,\theta)=g(X_{j};i,\theta)/g(X_{j};i,\theta_{S})$. Since we have

$\displaystyle\frac{d}{d\alpha}\log\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]$
$\displaystyle=\frac{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)\log
f(X_{j};i,\theta)]}{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{i};\theta)]}\quad\text{and}$
$\displaystyle\frac{d^{2}}{d\alpha^{2}}\log\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]$
$\displaystyle=\frac{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)\\{\log
f(X_{j};i,\theta)\\}^{2}]\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]}{\\{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]\\}^{2}}$
$\displaystyle\quad-\frac{\\{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)\log
f(X_{j};i,\theta)]\\}^{2}}{\\{\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]\\}^{2}},$

the Taylor expansion of
$\log\mathbb{E}_{\mathrm{pos}}[f^{\alpha}(X_{j};i,\theta)]$ around $\alpha=0$
completes the proof.

###  C.2. Proofs of Propositions 3.1

Let
$\mathcal{T}^{(1)}_{S,i}(\cdot):=\\{J_{S}^{-1}\nabla_{\theta_{S}}s_{i}(\cdot,\theta)\\}^{\top}$
and $\mathcal{H}^{(1)}_{i}(\cdot):=\nabla_{\theta_{S}}\log
h_{i}(\cdot\mid\theta)$, respectively. Under conditions in Appendix A, the
Taylor expansion gives

$\displaystyle\sum_{i=1}^{n}w_{i}\mathrm{Cov}_{\mathrm{pos}}[\log
h_{i}(X_{i}\mid\theta),s_{i}(X_{i},\theta)]=\mathrm{tr}\left(\mathbb{E}\left[\sum_{i=1}^{n}\frac{w_{i}}{n}\mathcal{T}^{(1)}_{S,i}(X_{i})\mathcal{H}^{(1)}_{i}(X_{i})\right]\right)+o(n^{-1}).$

Together with the above expression, the identity

$\displaystyle\nabla_{\theta}\int\nabla^{\top}_{\theta}s(X,\theta)h(X\mid\theta)dX=\int\nabla^{2}s(X,\theta)h(X\mid\theta)dX+\int\nabla_{\theta}s(X,\theta)\nabla^{\top}_{\theta}h(X\mid\theta)dX$

completes the proof of Proposition 3.1.