# Generalized Posterior Calibration via Sequential Monte Carlo Sampler

Masahiro Tanaka<EMAIL_ADDRESS>0000-0002-2269-8468 Faculty of
Economics, Fukuoka University,8-19-1, Nanakuma,
JonanFukuokaFukuokaJapan814-0180

(2018)

###### Abstract.

For robustness toward model misspecification, the generalized posterior
inference approach modifies the likelihood term by raising it to the power of
a learning rate, thereby adjusting the spread of the posterior. This paper
proposes a computationally efficient strategy for selecting an appropriate
learning rate. The proposed approach builds upon the generalized posterior
calibration (GPC) algorithm introduced by Syring and Martin (2019)
[Biometrika, Volume 106, Issue 2, pp. 479-486], which is designed to select
the learning rate to achieve the nominal frequentist coverage. This algorithm,
which evaluates the coverage probability based on bootstrap samples, suffers
from high computational costs due to the need for repeated posterior
simulations for bootstrap samples. To address this limitation, the study
proposes an algorithm that combines elements of the GPC algorithm with the
sequential Monte Carlo (SMC) sampler. By leveraging the similarity between the
learning rate in generalized posterior inference and the inverse temperature
in SMC sampling, the proposed algorithm efficiently calibrates the posterior
distribution with less computational cost. For demonstration, the proposed
algorithm was applied to several statistical learning models.

generalized posterior, Gibbs posterior, generalized posterior calibration,
sequential Monte Carlo sampler

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Mathematics of computing
Probabilistic algorithms††ccs: Mathematics of computing Probabilistic
inference problems††ccs: Computing methodologies Uncertainty
quantification††ccs: Mathematics of computing Bayesian computation

##  1\. Introduction

As the amount of available data and the complexity of workhorse models
increase, the need for robustness in statistical learning becomes more
pressing. Given the trend, this paper focuses on two strands of robust
Bayesian methods. The first strand is generalized Bayesian inference (Martin
and Syring, 2022; Nott et al., ming). Let $\mathcal{D}$ denote a dataset with
a sample size $N$ and consider it to be independent,
$\mathcal{D}=\left\\{\mathcal{D}_{i}\right\\}_{i=1}^{N}$. In standard Bayesian
inference, the posterior of a $K$-dimensional unknown parameter vector
$\boldsymbol{\theta}$ is composed of a likelihood
$p\left(\mathcal{D}|\boldsymbol{\theta}\right)$ and prior
$p\left(\boldsymbol{\theta}\right)$,

$\pi\left(\boldsymbol{\theta}\right)\propto
p\left(\mathcal{D}|\boldsymbol{\theta}\right)p\left(\boldsymbol{\theta}\right),$

In contrast, the generalized Bayesian inference is based on a generalized
posterior that is obtained by equipping a learning rate $\eta\left(>0\right)$
(also called a scaling parameter) with the following likelihood:

$\pi_{\eta}^{*}\left(\boldsymbol{\theta}\right)\propto
p\left(\mathcal{D}|\boldsymbol{\theta}\right)p\left(\boldsymbol{\theta}\right)$

By setting $\eta<1$, we can increase the spread of the posterior, making the
inference robust to model misspecifications (e.g., (Walker and Hjort, 2001;
Grünwald, 2012; Miller and Dunson, 2019; Miller, 2021)). The second class of
robust Bayesian methods is Gibbs posterior inference (Zhang, 2006b, a; Jiang
and Tanner, 2008; Bissiri et al., 2016; Atchadé, 2017; Syring and Martin,
2023). This approach formulates an inferential problem using a generic loss
function $r_{i}\left(\boldsymbol{\theta};\mathcal{D}_{i}\right)$ instead of a
probabilistic model:

$\pi_{\eta}^{*}\left(\boldsymbol{\theta}\right)\propto\exp\left\\{-Nr\left(\boldsymbol{\theta};\mathcal{D}\right)\right\\}^{\eta}p\left(\boldsymbol{\theta}\right),$

where $r\left(\boldsymbol{\theta};\mathcal{D}\right)$ denotes an empirical
risk function defined as

$r\left(\boldsymbol{\theta};\mathcal{D}\right)=\frac{1}{N}\sum_{i=1}^{N}r_{i}\left(\boldsymbol{\theta};\mathcal{D}_{i}\right).$

The algorithm proposed in the paper can be applied to both Gibbs and
generalized posterior inferences. Subsequently,
$q\left(\boldsymbol{\theta};\mathcal{D}\right)$ denotes a likelihood
$p\left(\mathcal{D}|\boldsymbol{\theta}\right)$ or pseudo-likelihood
$\exp\left\\{-Nr\left(\boldsymbol{\theta};\mathcal{D}\right)\right\\}$,

(1) $\pi_{\eta}^{*}\left(\boldsymbol{\theta}\right)\propto
q\left(\boldsymbol{\theta};\mathcal{D}\right)^{\eta}p\left(\boldsymbol{\theta}\right).$

There are several approaches to select $\eta$ ((Grünwald and van Ommen, 2017;
Holmes and Walker, 2017; Lyddon et al., 2019; Syring and Martin, 2019)), and
each with a different focus111See (Wu and Martin, 2023) for a comparison.. We
intend to improve the generalized posterior calibration (GPC) algorithm
proposed by Syring and Martin (2019). In the GPC, the coverage probability
under a specific value of $\eta$ is evaluated using bootstrap samples, and
$\eta$ is numerically chosen to achieve the nominal frequentist coverage
probability. A notable limitation of the GPC is its computational cost. We
must repeatedly run a posterior simulator for bootstrap samples until
convergence.

The contribution of this study is that it develops a new computational
strategy for choosing the learning rate $\eta$. Essentially, the algorithm we
propose is a fusion of the GPC and sequential Monte Carlo (SMC) samplers (Del
Moral et al., 2006; Dai et al., 2022). SMC samplers are a class of algorithms
that apply importance sampling to intermediate distributions that bridge from
a prior to a posterior. The learning rate in generalized posterior inference
and the inverse temperature in SMC samplers play the same role–powering the
likelihood. By exploiting this similarity, we transform the target
distribution gradually to achieve the target credible/confidence level.
Indeed, Syring and Martin (2019) mentioned a related idea in their
supplementary material.222“In our implementation of the algorithm, the
posterior is sampled every time $\omega$ [note by the author: learning rate]
is updated. However, it may be faster to sample the posterior $M$ times for
$\omega_{0}$ and subsequently use importance sampling to update the posterior
samples each time $\omega$ is updated.” (p. 2 of the Supplementary Material).
However, to the best knowledge of the author, no practical implementation of
this idea has been studied. Furthermore, it is ineffective to apply importance
sampling directly to the problem because its quality critically depends on the
disparity between the proposal and target distributions (e.g., (Liu, 2001));
when the consecutive learning rates, i.e., the proposal and target
distributions, are not close enough, importance sampling would not work well.
Therefore, the use of an SMC sampler is of critical importance.

The remainder of this paper is structured as follows. Section 2 introduces the
proposed algorithm. In Section 3, we apply the algorithm to synthetic and real
data for demonstration. Section 4 concludes the paper.

##  2\. Method

###  2.1. Generalized posterior calibration

This section describes the GPC (Syring and Martin, 2019). We refer to it as
the GPC-MCMC to distinguish it from the proposed algorithm. Let
$\mathcal{C}_{\alpha}^{\eta}\left(\mathcal{D}\right)$ denote the generalized
posterior $100\left(1-\alpha\right)\%$ credible set for $\boldsymbol{\theta}$
with $\eta$. The coverage probability function is represented as follows:

$c_{\alpha}\left(\eta|\mathbb{P}\right)=\mathbb{P}\left\\{\boldsymbol{\theta}^{\dagger}\left(\eta\right)\in\mathcal{C}_{\alpha}^{\eta}\left(\mathcal{D}\right)\right\\},$

where $\boldsymbol{\theta}^{\dagger}\left(\eta\right)$ denotes the Kullback-
Leibler minimizer in the model obtained with $\eta$. As the true data
distribution $\mathbb{P}$ is unknown, we replace it with the empirical
distribution $\mathbb{P}_{N}$,

$c_{\alpha}\left(\eta|\mathbb{P}_{N}\right)=\mathbb{P}_{N}\left\\{\hat{\boldsymbol{\theta}}\left(\eta\right)\in\mathcal{C}_{\alpha}^{\eta}\left(\mathcal{D}\right)\right\\},$

where $\hat{\boldsymbol{\theta}}\left(\eta\right)$ represents a point estimate
of $\boldsymbol{\theta}$ obtained with $\eta$. Even with this modification,
$c_{\alpha}\left(\eta|\mathbb{P}_{N}\right)$ cannot be evaluated because
enumeration of all $N^{N}$ possible with-replacement samples from
$\mathcal{D}$ is needed. Therefore, we approximate $\mathbb{P}_{N}$ using a
bootstrap method. With $B$ bootstrap samples
$\left\\{\breve{\mathcal{D}}^{\left[b\right]}\right\\}_{b=1}^{B}$, the
coverage probability can be estimated as

$\hat{c}_{\alpha}\left(\eta|\mathbb{P}_{N}\right)=\frac{1}{B}\sum_{b=1}^{B}\mathbb{I}\left\\{\hat{\boldsymbol{\theta}}\left(\eta\right)\in\mathcal{C}_{\alpha}^{\eta}\left(\breve{\mathcal{D}}^{\left[b\right]}\right)\right\\},$

where $\mathbb{I}\left\\{\cdot\right\\}$ denotes the indicator function.
$\eta$ is chosen by solving
$\hat{c}_{\alpha}\left(\eta|\mathbb{P}_{N}\right)=1-\alpha$ via a stochastic
approximation (Robbins and Monro, 1951). At the $s$th iteration, a single step
of the stochastic approximation recursion is given as

(2)
$\eta_{\left(s+1\right)}\leftarrow\eta_{\left(s\right)}+\varsigma_{l}\left[\hat{c}_{\alpha}\left(\eta_{\left(s\right)}|\mathbb{P}_{N}\right)-\left(1-\alpha\right)\right],$

where $\left\\{\varsigma_{l}\right\\}$ denotes a non-increasing sequence such
that $\sum_{l}\varsigma_{l}=\infty$ and $\sum_{l}\varsigma_{l}^{2}<\infty$.
While Syring and Martin (2019) specify $\varsigma_{s}=s^{-0.51}$ in their
study, we used a variant of Keston’s (Kesten, 1958) rule:
$\varsigma_{s}\left(l\right)=l^{-0.51}$ where $l$ increases by 1 only when
there is a directional change in the trajectory of $\eta_{\left(s\right)}$ and
$\hat{c}_{\alpha}\left(\eta_{\left(s\right)}|\mathbb{P}_{N}\right)<1$. This
modification shortens the convergence time significantly. Let $R$ denote the
number of posterior draws used for analysis after discarding initial draws as
warmup. Figure 1 summarizes the GPC-MCMC.

Figure 1. Generalized posterior calibration

input:observed dataset $\mathcal{D}$, target kernel
$\pi_{\eta}^{*}\left(\cdot\right)$, initial guess $\eta_{1}$, target credible
level $\alpha$, termination threshold $\epsilon$.

Generate $B$ bootstrap samples
$\left\\{\breve{\mathcal{D}}^{\left[b\right]}\right\\}_{b=1}^{B}$ from
$\mathcal{D}$.

Set $s\leftarrow 1$ and $l\leftarrow 1$.

while converge

Compute $\hat{\boldsymbol{\theta}}$ with $\eta_{\left(s\right)}$.

for $b=1,...,B$:

Simulate $R$ posterior draws for $\breve{\mathcal{D}}^{\left[b\right]}$ using
a MCMC

sampler with $\eta_{\left(s\right)}$.

Compute the credible set
$\mathcal{C}_{\alpha}^{\eta_{\left(s\right)}}\left(\breve{\mathcal{D}}^{\left[b\right]}\right)$.

end for

Compute the coverage probability
$\hat{c}_{\alpha}\left(\eta_{\left(s\right)}|\mathbb{P}_{N}\right)$.

if
$\left|\hat{c}_{\alpha}\left(\eta_{\left(s\right)}|\mathbb{P}_{N}\right)-\left(1-\alpha\right)\right|<\epsilon$:

Set $\hat{\eta}\leftarrow\eta_{\left(s\right)}$.

break

else

Set a new learning rate $\eta_{\left(s+1\right)}$ according to (2).

Set $s\leftarrow s+1$.

end if

end while

return: $\hat{\eta}$

###  2.2. Sequential Monte Carlo sampler

SMC samplers (Del Moral et al., 2006) are a class of Monte Carlo simulation
algorithms which repeatedly apply importance sampling to a sequence of
synthetic intermediate distributions $\left\\{\pi_{t}\right\\}_{t=0}^{T}$ to
obtain collections of weighted particles
$\left\\{w_{t}^{\left[m\right]},\boldsymbol{\theta}_{t}^{\left[m\right]}\right\\}_{m=1}^{M}$.
The initial distribution $\pi_{0}$ is the prior,
$\pi_{0}\left(\boldsymbol{\theta}\right)=p\left(\boldsymbol{\theta}\right)$,
and the terminal distribution $\pi_{T}$ is the target distribution, that is,
the posterior,
$\pi_{T}\left(\boldsymbol{\theta}\right)=p\left(\mathcal{D}|\boldsymbol{\theta}\right)p\left(\boldsymbol{\theta}\right)$.
An intermediate distribution is specified as a likelihood-tempered posterior:

(3) $\pi_{t}\left(\boldsymbol{\theta}\right)\propto
p\left(\mathcal{D}|\boldsymbol{\theta}\right)^{\phi_{t}}p\left(\boldsymbol{\theta}\right),$

where $\left\\{\phi_{t}\right\\}_{t=1}^{T}$ denotes an increasing sequence
with $\phi_{0}=0$ and $\phi_{T}=1$. $\phi_{t}$ can be interpreted as inverse
temperature. $p$ $\left\\{\tilde{\pi}_{t}\right\\}_{t=0}^{T}$ denotes a
sequence of auxiliary distributions:

$\tilde{\pi}_{t}\left(\boldsymbol{\theta}_{0:t}\right)=\pi_{t}\left(\boldsymbol{\theta}_{t}\right)\prod_{s=0}^{t-1}\mathcal{L}_{s}\left(\boldsymbol{\theta}_{s+1},\boldsymbol{\theta}_{s}\right),$

where $\mathcal{L}_{t}\left(\cdot,\cdot\right)$ denotes a Markov kernel which
is also called a backward kernel, where it moves back from
$\boldsymbol{\theta}_{t+1}$ to $\boldsymbol{\theta}_{t}$. $\tilde{\pi}_{t}$ is
approximated using a system of weighted particles,
$\left\\{w_{t}^{\left[m\right]},\boldsymbol{\theta}_{t}^{\left[m\right]}\right\\}_{m=1}^{M}$.
The particles are mutated via a Markov kernel
$\mathcal{K}_{t}\left(\cdot,\cdot\right)$ which is termed a forward kernel.
Let $\gamma_{t}\left(\boldsymbol{\theta}\right)$ denote the unnormalized
posterior density with $\phi_{t}$

$\gamma_{t}\left(\boldsymbol{\theta}\right)=p\left(\mathcal{D}|\boldsymbol{\theta}\right)^{\phi_{t}}p\left(\boldsymbol{\theta}\right).$

The unnormalized weights are represented as

$\displaystyle W_{t}^{\left[m\right]}$ $\displaystyle\propto$
$\displaystyle\frac{\tilde{\pi}_{t}\left(\boldsymbol{\theta}_{t}^{\left[m\right]}\right)\prod_{s=1}^{t-1}\mathcal{L}_{s}\left(\boldsymbol{\theta}_{s+1}^{\left[m\right]},\boldsymbol{\theta}_{s}^{\left[m\right]}\right)}{\zeta_{1}\left(\boldsymbol{\theta}_{1}^{\left[m\right]}\right)\prod_{s^{\prime}=1}^{t-1}\mathcal{K}_{s^{\prime}}\left(\boldsymbol{\theta}_{s^{\prime}-1}^{\left[m\right]},\boldsymbol{\theta}_{s^{\prime}}^{\left[m\right]}\right)}$
$\displaystyle\propto$
$\displaystyle\widetilde{W}_{t}^{\left[m\right]}W_{t-1}^{\left[m\right]},$

where $\widetilde{W}_{t}^{\left[m\right]}$ represents the unnormalized
incremental weight given by

$\widetilde{W}_{t}^{\left[m\right]}=\frac{\gamma_{t}\left(\boldsymbol{\theta}_{t}^{\left[m\right]}\right)\mathcal{L}_{t-1}\left(\boldsymbol{\theta}_{t}^{\left[m\right]},\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)}{\gamma_{t-1}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)\mathcal{K}_{t}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]},\boldsymbol{\theta}_{t}^{\left[m\right]}\right)}.$

Following the literature, we employ an MCMC kernel for the forward kernel
$\mathcal{K}_{t}$, leaving it $\pi_{t}$-invariant. This choice is optimal, as
the variance of the unnormalized weights is approximately minimized (Del Moral
et al., 2006). The backward kernel is represented as

$\mathcal{L}_{t-1}\left(\boldsymbol{\theta}_{t}^{\left[m\right]},\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)=\frac{\pi_{t}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)\mathcal{K}_{t}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]},\boldsymbol{\theta}_{t}^{\left[m\right]}\right)}{\pi_{t}\left(\boldsymbol{\theta}_{t}^{\left[m\right]}\right)}.$

Under this specification, the unnormalized incremental weights are reduced to

$\widetilde{W}_{t}^{\left[m\right]}=\frac{\gamma_{t}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)}{\gamma_{t-1}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)}=p\left(\mathcal{D}|\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)^{\phi_{t}-\phi_{t-1}},$

and thus the unnormalized weights are updated as

$W_{t}^{\left[m\right]}=w_{t-1}^{\left[m\right]}q\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]};\mathcal{D}\right){}^{\phi_{t}-\phi_{t-1}}.$

As $\pi_{t}$ evolves, the variance of the weights tends to increase, inducing
the weighted particles to degenerate. The standard metric of particle
degeneracy is the effective sample size (ESS) (Kong et al., 1994),

$ESS_{t}=\frac{1}{\sum_{m=1}^{M}\left(w_{t}^{\left[m\right]}\right)^{2}}=\frac{\left(\sum_{m=1}^{M}W_{t-1}^{\left[m\right]}\widetilde{W}_{t}^{\left[m\right]}\right)^{2}}{\sum_{m=1}^{M}\left(W_{t-1}^{\left[m\right]}\widetilde{W}_{t}^{\left[m\right]}\right)^{2}}.$

This study used stratified resampling algorithm (Kitagawa, 1996) if $ESS_{t}$
is below a prespecified threshold $\overline{ESS}=\psi M$,
$\psi\in\left(0,1\right)$.

The adaptive SMC sampler is applicable to generalized posterior. Exploiting
the similarity between pseudo-posterior (1) in generalized/Gibbs posterior
inference and likelihood tempered posterior (3) in SMC sampling, we treat the
learning rate $\eta_{t}$ analogously to the inverse temperature $\phi_{t}$ as

$\gamma_{t}^{*}\left(\boldsymbol{\theta}\right)=q\left(\boldsymbol{\theta};\mathcal{D}\right)^{\eta_{t}}p\left(\boldsymbol{\theta}\right).$

The temperature schedule is critically important in the SMC sampler. If the
temperature schedule is too coarse, the consecutive intermediate distributions
are too different to obtain a good particle approximation, while if it is too
fine, it incurs unnecessary computational load. In this study, we use the
adaptive SMC sampler (Jasra et al., 2011; Beskos et al., 2016; Whiteley et
al., 2016; Huggins and Roy, 2019). $\eta_{t+1}$ is chosen by keeping the ESS
above a target level, $\widetilde{ESS}_{t}=\xi ESS_{t-1}$,
$\xi\in\left(0,1\right)$. The ESS can be represented as a function of
$\eta_{t}$ as follows:

$ESS\left(\eta_{t}\right)=\frac{\left(\sum_{m=1}^{M}W_{t-1}^{\left[m\right]}q\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]};\mathcal{D}\right)^{\eta_{t}-\eta_{t-1}}\right)^{2}}{\sum_{m^{\prime}=1}^{M}\left(W_{t-1}^{\left[m^{\prime}\right]}q\left(\boldsymbol{\theta}_{t-1}^{\left[m^{\prime}\right]};\mathcal{D}\right)\right)^{2}}.$

$\phi_{t}$ is chosen by solving $ESS\left(\eta_{t}\right)=\widetilde{ESS}_{t}$
via bisection search. The choice of $\xi$ involves a trade-off between
sampling quality and computational cost. The weighted particles are resampled
if $ESS\left(\phi_{1}\right)<\overline{ESS}$. Figure 2 summarizes the adaptive
SMC sampler for the generalized posterior. Let
$\textrm{ASMC}^{*}\left(\eta_{0},\eta_{T};\left\\{w_{0}^{\left[m\right]},\boldsymbol{\theta}_{0}^{\left[m\right]}\right\\}_{m=1}^{M}\right)$
denote the adaptive SMC sampler for the generalized posterior with initial and
terminal learning rates $\eta_{0},\eta_{T}$, respectively, and initial
weighted particles
$\left\\{w_{0}^{\left[m\right]},\boldsymbol{\theta}_{0}^{\left[m\right]}\right\\}_{m=1}^{M}$.

Figure 2. Adaptive SMC sampler for generalized/Gibbs posterior

input: dataset $\mathcal{D}$, target kernel
$\pi_{\eta}^{*}\left(\cdot\right)$, initial and terminal learning rate
$\eta_{1}$, $\eta$, target ESS $\widetilde{ESS}$, resampling threshold
$\overline{ESS}$, initial weighted particles
$\left\\{w_{0}^{\left[m\right]},\boldsymbol{\theta}_{0}^{\left[m\right]}\right\\}_{m=1}^{M}$.

Set $t\leftarrow 0$.

while converge:

Set $t\leftarrow t+1$

if $ESS\left(\eta\right)>\widetilde{ESS}$ then:

Set $T\leftarrow t$ and $\eta_{T}\leftarrow\eta$ and

else:

Solve $ESS\left(\eta_{t}\right)=\widetilde{ESS}$ in
$\eta_{t}\in\left(\eta_{t-1},\eta\right]$.

end if

for $m=1,...,M$:

Compute the unnormalized weights

$W_{t}^{\left[m\right]}=w_{t-1}^{\left[m\right]}q\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]};\mathcal{D}\right)^{\eta_{t}-\eta_{t-1}}.$

Normalize the weights

$w_{t}^{\left[m\right]}=W_{t}^{\left[m\right]}\left(\sum_{m^{\prime}=1}^{M}W_{t}^{\left[m^{\prime}\right]}\right)^{-1}.$

end for

Resample the particles if $ESS_{t}<\overline{ESS}$.

for $m=1,...,M$:

Sample the particles
$\boldsymbol{\theta}_{t}^{\left[m\right]}\sim\mathcal{K}_{t}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]},\cdot\right)$.

end for

if $\eta_{t}=\eta$ then:

break

end if

end while

return
$\left\\{w_{T}^{\left[m\right]},\boldsymbol{\theta}_{T}^{\left[m\right]}\right\\}_{m=1}^{M}$

###  2.3. Generalized posterior calibration via sequential Monte Carlo sampler

This paper proposes the GPC via SMC sampler (GPC-SMC) (Figure 3), which, as
its name suggests, is a fusion of the two algorithms. Instead of an MCMC
sampler, we apply the adaptive SMC sampler to bootstrap samples. The weighted
particles are re-used as the initial states for the next iteration. At the
$s$th iteration, the adaptive SMC sampler runs with a temperature schedule
$\left\\{\eta_{\left(s\right)},...,\eta_{\left(s+1\right)}\right\\}$ and
initial weighted particles
$\left\\{w_{\left(s\right)}^{\left[b,m\right]},\boldsymbol{\theta}_{\left(s\right)}^{\left[b,m\right]}\right\\}_{m=1}^{M}$.
As the iterations proceed, a temperature schedule is likely to become shorter,
drastically reducing the computational cost per iteration. In contrast, the
GPC-MCMC has a constant computational cost per iteration; even when
$\eta_{\left(s\right)}$ and $\eta_{\left(s+1\right)}$ are close during the
last phase of the optimization, it needs to simulate the full length of
chains.

Though we can initialize particles using an SMC sampler with a long
temperature schedule $\left\\{0,...,\eta_{\left(1\right)}\right\\}$, we
suggest using an MCMC sampler with $\eta_{\left(1\right)}$ to generate initial
particles. The corresponding weights are set according to the posterior
densities evaluated at the particles.

$\displaystyle W_{\left(1\right)}^{\left[b,m\right]}$ $\displaystyle=$
$\displaystyle
q\left(\boldsymbol{\theta}^{\left[b,m\right]};\breve{\mathcal{D}}^{\left[b\right]}\right)^{\eta_{\left(1\right)}}p\left(\boldsymbol{\theta}^{\left[b,m\right]}\right),$
$\displaystyle w_{\left(1\right)}^{\left[b,m\right]}$ $\displaystyle=$
$\displaystyle
W_{\left(1\right)}^{\left[b,m\right]}\left(\sum_{m^{\prime}=1}^{M}W_{\left(1\right)}^{\left[b,m^{\prime}\right]}\right)^{-1}.$

In the GPC-SMC, a temperature schedule can be decreasing if
$\eta_{\left(s\right)}<\eta_{\left(s+1\right)}$. This might seem peculiar, but
the SMC sampler effectively works as long as the temperature schedule is
selected finely enough, that is, $\xi$ is close to one, making the proposal
and target distributions sufficiently similar.

Figure 3. Generalized posterior calibration via SMC sampler

input: observed dataset $\mathcal{D}$, target kernel
$\pi_{\eta}^{*}\left(\cdot\right)$, initial guess $\eta_{\left(1\right)}$,
target credible level $\alpha$, termination threshold $\epsilon$.

Set $s\leftarrow 1$ and $l\leftarrow 1$.

Compute the posterior estimate $\hat{\boldsymbol{\theta}}$ with
$\eta_{\left(1\right)}$.

Generate $B$ bootstrap samples
$\left\\{\breve{\mathcal{D}}^{\left[b\right]}\right\\}_{b=1}^{B}$ from
$\mathcal{D}$.

for $b=1,...,B$:

Simulate $M$ posterior draws
$\left\\{\boldsymbol{\theta}_{\left(1\right)}^{\left[b,m\right]}\right\\}_{m=1}^{M}$
using a MCMC

sampler with $\eta_{\left(1\right)}$.

Initialize the weights
$\left\\{w_{\left(1\right)}^{\left[b,m\right]}\right\\}_{m=1}^{M}$.

Compute the credible set
$\mathcal{C}_{\alpha}^{\eta_{\left(1\right)}}\left(\breve{\mathcal{D}}^{\left[b\right]}\right)$.

end for

while converge:

if
$\left|\hat{c}_{\alpha}\left(\eta_{\left(s\right)}|\mathbb{P}_{N}\right)-\left(1-\alpha\right)\right|<\epsilon$
then:

Set $\hat{\eta}\leftarrow\eta_{\left(s\right)}$.

break

else:

Set a new learning rate $\eta_{\left(s+1\right)}$ according to (2).

Compute the posterior estimate $\hat{\boldsymbol{\theta}}$with
$\eta_{\left(s+1\right)}$.

for $b=1,...,B$:

Simulate the posterior draws

$\left\\{w_{\left(s+1\right)}^{\left[b,m\right]},\boldsymbol{\theta}_{\left(s+1\right)}^{\left[b,m\right]}\right\\}_{m=1}^{M}$
using

$\textrm{ASMC}^{*}\left(\eta_{\left(s\right)},\eta_{\left(s+1\right)};\left\\{w_{\left(s\right)}^{\left[b,m\right]},\boldsymbol{\theta}_{\left(s\right)}^{\left[b,m\right]}\right\\}_{m=1}^{M}\right)$.

Compute the credible set
$\mathcal{C}_{\alpha}^{\eta_{\left(s+1\right)}}\left(\breve{\mathcal{D}}^{\left[b\right]}\right)$.

end for

Set $s\leftarrow s+1$.

end if

end while

return: $\hat{\eta}$

In our context, each of an MCMC sampler and SMC sampler has its advantages and
disadvantages. The computational cost of an SMC sampler is generally larger
than that of a MCMC sampler. Therefore, if an efficient MCMC sampler is
available, there may be no strong reason to use an SMC sampler. On the
contrary, an SMC sampler works well for distributions that are difficult to
simulate, e.g., posterior distributions with multimodality or discontinuity.
The advantage of an SMC sampler is more pronounced for Gibbs posterior
inference because no closed-form expression of the conditional posterior
distributions is available, making posterior simulations difficult, and
because for many cases a Gibbs posterior is discontinuous as in the subsequent
section. While Hamiltonian Monte Carlo is a state-of-the-art solution to
sampling from arbitrary distributions, it is only applicable to continuous
target distributions. Thus, as seen below, there is no feasible option for
posterior simulation except classical and sub-optimal algorithms such as the
random-walk Metropolis-Hastings algorithm. Indeed, the choice of an MCMC
kernel $\mathcal{K}_{t}$ is also important in SMC sampling, but it is not as
sensitive an issue as in MCMC sampling.

##  3\. Application

###  3.1. Quantile regression

For demonstration, we applied the GPC-SMC and GPC-MCMC to quantile regression
with synthetic data. Following Section 4 of (Syring and Martin, 2019), the DGP
is specified as

$y_{i}=\boldsymbol{\theta}^{\top}\boldsymbol{x}_{i}+\varepsilon_{i},\quad\varepsilon_{i}\sim\mathcal{N}\left(0,\sigma^{2}\right),$

where $\boldsymbol{x}_{i}=\left(1,x_{1,i}\right)^{\top}$,
$x_{1,i}+2\sim\chi_{4}^{2}$, $\varepsilon_{i}\sim\mathcal{N}\left(0,1\right)$,
$\boldsymbol{\theta}=\left(\theta_{1},\theta_{2}\right)^{\top}=\left(2,1\right)^{\top}$,
and $\sigma^{2}=1$. We consider three cases of the sample size
$N\in\left\\{100,400,1600\right\\}$. We infer $\boldsymbol{\theta}$ using a
Gibbs posterior approach. $\boldsymbol{\theta}$ is inferred for the
$\left(100\times\tau\right)$-th percentile based on an empirical risk function
given by

$r\left(\boldsymbol{\theta};\mathcal{D}\right)=\frac{1}{N}\sum_{i=1}^{N}\left|\left(y_{i}-\boldsymbol{\theta}^{\top}\boldsymbol{x}_{i}\right)\left(\tau-\mathbb{I}\left\\{y_{i}<\boldsymbol{\theta}^{\top}\boldsymbol{x}_{i}\right\\}\right)\right|.$

We fix $\tau=0.5$. The prior for $\boldsymbol{\theta}$ is a normal
distribution with mean zero and variance $\varsigma^{2}$,
$\boldsymbol{\theta}\sim\mathcal{N}\left(\boldsymbol{0}_{2},\varsigma^{2}\boldsymbol{I}_{2}\right)$.
We choose a fairly diffuse prior, $\varsigma^{2}=100^{2}$.

For the GPC-SMC, we employed the adaptive random-walk Metropolis-Hastings
(RWMH) algorithm (Haario et al., 2001) as the MCMC kernel. At the $t$th
iteration of SMC sampling, a proposal $\boldsymbol{\theta}^{\prime}$ is
generated from a multivariate normal distribution,
$\boldsymbol{\theta}^{\prime}\sim\mathcal{N}\left(\boldsymbol{\theta}_{t}^{\left[m\right]},\zeta_{t}\boldsymbol{\Sigma}_{t}\right)$,
where $\zeta_{t}\left(>0\right)$ denotes a scaling parameter and
$\boldsymbol{\Sigma}_{t-1}$ represents a covariance matrix.
$\boldsymbol{\theta}^{\prime}$ is accepted with probability:

$\alpha\left(\boldsymbol{\theta}^{\prime},\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)=\frac{\gamma_{t}^{*}\left(\boldsymbol{\theta}^{\prime}\right)}{\gamma_{t}^{*}\left(\boldsymbol{\theta}_{t-1}^{\left[m\right]}\right)}.$

$\boldsymbol{\Sigma}_{t}$ is chosen based on the covariance matrix of the
current particles as

$\boldsymbol{\Sigma}_{t+1}\leftarrow\frac{\eta_{t}}{\eta_{t+1}}\frac{1}{M}\sum_{m=1}^{M}w_{t}^{\left[m\right]}\boldsymbol{\theta}_{t}^{\left[m\right]}\left(\boldsymbol{\theta}_{t}^{\left[m\right]}\right)^{\top}.$

$\zeta_{t}$ is adaptively tuned on the fly according to the following updating
rule:

$\log\zeta_{t+1}\leftarrow\log\zeta_{t}+\varphi_{t}\left(\varrho^{*}-\varrho_{t}\right),$

where $\varrho_{t}$ represents the average acceptance rate at the $t$th
iteration, $\varrho^{*}$ denotes a prespecified target acceptance rate, and
$\left\\{\varphi_{t}\right\\}$ indicates a decreasing sequence such that
$\sum_{t}\varphi_{t}=\infty$ and $\sum_{t}\varphi_{t}^{2}<\infty$. We fixed
$\varrho^{*}=0.25$ and $\varphi_{t}=\left(t+1\right)^{-0.51}$.

We evaluated the numerical efficiency of the two algorithms based on wall
clock computation time. It is impossible to exactly compare the two
algorithms, because the relative performance critically depends on the
implementation details, in particular, the number of MCMC draws $R$ and number
of particles $M$. We choose $M=1,000$ and $R=20,000$ so that the multivariate
effective sample size (multiESS) (Vats et al., 2019) is approximately the same
as $M$. Although multiESS for MCMC sampling, which reflects the degrees of
autocorrelations in the generated chain, is conceptually different from ESS
for SMC sampling, they have similar implications for practitioners. As a
metric of the efficiency of an MCMC sampler, a minimum of chain-wise effective
sample sizes (minESS) (Geyer, 1992) is also used in the literature. As minESS
is generally smaller than multiESS (roughly a half in our case), using
multiESS means that the comparison is more favorable to the GPC-MCMC than
using minESS.

The other tuning parameters are selected as follows. The target credible level
is $\alpha=0.05$, meaning a 95% credible set is calibrated. The number of
bootstrap samples is $B=500$. The stopping criterion for finding the
appropriate $\eta$ is $\epsilon=0.005$. The initial guess of the learning rate
is fixed to $\eta=1$. The threshold for resampling is $\psi=0.5$, following
the standard practice in the literature. The tuning parameter of the target
ESS for choosing a new learning rate is $\xi=0.999$. While there is neither
consensus nor a principled guideline regarding the choice of $\xi$, our choice
may be one of the largest values among those used in the literature 333While
Jasra et al. (2011) uses $\xi=0.95$, there is considerable latitude in setting
$\xi$: from 1/3 (Kantas et al., 2014) to 0.999 (Zhou et al., 2016)..Thus, our
choice is rather unfavorable to the GPC-SMC in comparison to the GPC-MCMC.

All the computations are conducted using MATLAB (R2023b) on an Ubuntu desktop
(22.04.4 LTS) running on an AMD Ryzen Threadripper 3990X 2.9GHz 64-core
processor. The computation using different bootstrap samples is parallelized.
If a multi-machine parallel system is available, the GPC-SMC would run even
faster, as the particle-wise computation in an SMC sampler is also
parallelizable. For the GPC-MCMC, we use an algorithm proposed by Vihola
(2012) which is a variant of Haario et al. (2001), because the adaptive RWMH
algorithm does not work well, resulting in strongly autocorrelated chains.
Vihola’s (Vihola, 2012) adaptive algorithm is designed to estimate the shape
of the target distribution while coercing the acceptance rate. Therefore,
again, our comparison is unfavorable to the GPC-SMC.

Table 1 reports the coverage probabilities based on 200 synthetic datasets.
For all three cases, the coverage probabilities are close to 95%, which
implies that both algorithms effectively achieved the target
credible/confidence level. The medians of the calibrated learning rate are
$\eta\approx 1.6$ for $N=100$, $\eta\approx 1.4$ for $N=400$, and $\eta\approx
1.1$ for $N=1600$.

Table 1. Coverage probability

$N$ | Coverage probability (%)
---|---
| GPC-MCMC | GPC-SMC
100 | 97.0 | 96.0
400 | 93.5 | 94.0
1600 | 95.0 | 94.5

Note: $R$ is the number of draws generated by an MCMC sampler. $M$ is the
number of particles generated by an SMC sampler. Coverage probability of
ground truth is evaluated based on 200 synthetic datasets.

Figure 4 displays the distributions of the computation time. The GPC-SMC is
generally faster than the GPC-MCMC. When the initial value $\eta=1$ is close
to the desirable value, the difference in computation time between the two
algorithms is pronounced. This happens because the advantage of the GPC-SMC is
significant for the final phase of the stochastic approximation optimization.
For cases with $N=100$, $\eta=1$ is too small and during the initial phase,
long temperature schedules are used for SMC sampling, incurring non-negligible
computational cost. Therefore, with a good guess of $\eta$, the GPC-SMC would
further overwhelm the GPC-MCMC.

Figure 4. Comparison of computational cost

Note: Boxplot displays the distribution of computation time (in second) for
200 synthetic datasets.

###  3.2. Support vector machine

We consider support vector machine classification with the South African Heart
Disease dataset (see Section 4.4.2 of (Hastie et al., 2009)), as in Section 5
of (Syring and Martin, 2019). We have a binary outcome
$y_{i}\in\left\\{-1,1\right\\}$ and a $K$-dimensional vector of predictors
$\boldsymbol{x}_{i}=\left(1,x_{1,i},...,x_{K-1,i}\right)^{\top}$. Stacking
them yields $\boldsymbol{y}=\left(y_{1},...,y_{N}\right)^{\top}$ and
$\boldsymbol{X}=\left(\boldsymbol{x}_{1},...,\boldsymbol{x}_{N}\right)^{\top}$.
The support vector machine seeks to find
$\boldsymbol{\theta}=\left(\theta_{1},...,\theta_{k}\right)^{\top}$ that
minimizes the following objective function:

$r\left(\boldsymbol{\theta}\right)=\frac{1}{N}\sum_{i=1}^{N}2\max\left(0,1-y_{i}\boldsymbol{\theta}^{\top}\boldsymbol{x}_{i}\right).$

We assigned an independent Laplace-type prior to $\boldsymbol{\theta}$. The
log pseudo-posterior is represented as

$\pi_{\eta}^{*}\left(\boldsymbol{\theta}\right)\propto-\eta\sum_{i=1}^{N}2\max\left(0,1-y_{i}\boldsymbol{x}_{i}^{\top}\boldsymbol{\theta}\right)-\nu^{-1}\sum_{k=1}^{K}\left|\frac{\theta_{k}}{\sigma_{k}}\right|,$

where $\sigma_{k}$ denotes the standard deviation of the $k$th predictor
$x_{k,1},...,x_{k,N}$ with $\sigma_{1}=1$ and $\nu\left(>0\right)$ represents
a tuning parameter. We fixed $\nu=10$. (Polson and Scott, 2011) developed a
Gibbs sampler for Bayesian inference of this model based on a data
augmentation technique, although a resulting credible set ($\eta=1$) is not
well-calibrated (Syring and Martin, 2019). We use the Gibbs sampler of (Polson
and Scott, 2011) for the GPC-MCMC, while the adaptive RWMH for the GPC-SMC.
The sample size is $N=462$ and the dimension of $\boldsymbol{\theta}$ is
$K=8$. We choose $R=20,000$ and $M=4,000$ so that the multiESS from MCMC
sampling and ESS from SMC sampling are roughly matched. The other tuning
parameters are the same as in Section 3.1.

We executed the two algorithms 20 times with different random seeds. Both
algorithms consistently yielded $\eta\approx 0.09$, which aligns with the
result of (Syring and Martin, 2019); the Gibbs sampling algorithm of (Polson
and Scott, 2011) without calibration of $\eta$ brings excessively optimistic
uncertainty quantification. As shown in Figure 5, the GPC-SMC outperforms the
GPC-MCMC in terms of computing speed. The medians of computation time for the
GPC-MCMC and GPC-SMC are approximately 18.0 and 10.7 minutes, respectively. In
addition, the variation in time taken for computation is much smaller for the
GPC-SMC; the coefficients of variation for the GPC-MCMC and GPC-SMC are 0.335
and 0.017, respectively.

Figure 5. Comparison of computational cost

Note: Boxplot displays the distribution of computation time (in second) for 20
runs with different random seeds.

##  4\. Conclusion

In conclusion, this paper presented a novel and computationally efficient
strategy for selecting an appropriate learning rate in generalized posterior
inference. By building upon the GPC algorithm (Syring and Martin, 2019), which
aims to achieve nominal frequentist coverage, we devised an algorithm that
combined elements of the GPC with the SMC sampler. This integration harnessed
the similarity between the learning rate in generalized posterior inference
and the inverse temperature in SMC sampling, enabling the calibration of the
posterior distribution with reduced computational costs. The proposed approach
addresses the limitation of high computational costs associated with existing
methods by leveraging the efficiency of SMC sampling. Through empirical
demonstrations on statistical learning models, we illustrated the
effectiveness and practicality of our proposed algorithm in selecting an
appropriate learning rate for robustness toward model misspecification.

Overall, this research contributes to advancing the methodology of generalized
posterior inference by providing a scalable and efficient solution for
learning rate selection. Future work may explore further refinements and
extensions of the proposed algorithm, as well as its application in other
domains requiring robust Bayesian inference under model uncertainty.

## References

  * (1)
  * Atchadé (2017) Yves A. Atchadé. 2017\.  On the Contraction Properties of Some High-dimensional Quasi-posterior Distributions.  _Annals of Statistics_ 45, 5 (2017), 2248–2273.  https://doi.org/10.1214/16-AOS1526
  * Beskos et al. (2016) Alexandros Beskos, Ajay Jasra, Nikolas Kantas, and Alexandre Thiery. 2016.  On the Convergence of Adaptive Sequential Monte Carlo Methods.  _Annals of Applied Probability_ 26, 2 (2016), 1111–1146.  https://doi.org/10.1214/15-AAP1113
  * Bissiri et al. (2016) P. G. Bissiri, C. C. Holmes, and S. G. Walker. 2016\.  A General Framework for Updating Belief Distributions.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 78, 5 (02 2016), 1103–1130.  https://doi.org/10.1111/rssb.12158
  * Dai et al. (2022) Chenguang Dai, Pierre E. Jacob Jeremy Heng, and Nick Whiteley. 2022\.  An Invitation to Sequential Monte Carlo Samplers.  _J. Amer. Statist. Assoc._ 117, 539 (2022), 1587–1600.  https://doi.org/10.1080/01621459.2022.2087659 arXiv:https://doi.org/10.1080/01621459.2022.2087659
  * Del Moral et al. (2006) Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. 2006\.  Sequential Monte Carlo Samplers.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 68, 3 (05 2006), 411–436.  https://doi.org/10.1111/j.1467-9868.2006.00553.x
  * Geyer (1992) Charles J. Geyer. 1992\.  Practical Markov Chain Monte Carlo.  _Statist. Sci._ 7, 4 (1992), 473 – 483.  https://doi.org/10.1214/ss/1177011137
  * Grünwald (2012) Peter Grünwald. 2012\.  The Safe Bayesian. In _Algorithmic Learning Theory_ , Nader H. Bshouty, Gilles Stoltz, Nicolas Vayatis, and Thomas Zeugmann (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 169–183.
  * Grünwald and van Ommen (2017) Peter Grünwald and Thijs van Ommen. 2017.  Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It.  _Bayesian Analysis_ 12, 4 (2017), 1069–1103.  https://doi.org/10.1214/17-BA1085
  * Haario et al. (2001) Heikki Haario, Eero Saksman, and Johanna Tamminen. 2001\.  An Adaptive Metropolis Algorithm.  _Bernoulli_ 7, 2 (2001), 223–242.  https://doi.org/10.2307/3318737
  * Hastie et al. (2009) Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, and Jerome H. Friedman. 2009.  _The Elements of Statistical Learning: Data Mining, Inference, and Prediction_. Vol. 2.  Springer.
  * Holmes and Walker (2017) C. C. Holmes and S. G. Walker. 2017.  Assigning a Value to a Power Likelihood in a General Bayesian Model.  _Biometrika_ 104, 2 (03 2017), 497–503.  https://doi.org/10.1093/biomet/asx010
  * Huggins and Roy (2019) Jonathan H. Huggins and Daniel M. Roy. 2019.  Sequential Monte Carlo As Approximate Sampling: Bounds, Adaptive Resampling via $\infty$-ESS, and an Application to Particle Gibbs.  _Bernoulli_ 25, 1 (2019), 584–622.  https://doi.org/10.3150/17-BEJ999
  * Jasra et al. (2011) Ajay Jasra, David A. Stephens, Arnaud Doucet, and Theodoros Tsagaris. 2011.  Inference for Lévy-Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo.  _Scandinavian Journal of Statistics_ 38, 1 (2011), 1–22.  https://doi.org/10.1111/j.1467-9469.2010.00723.x
  * Jiang and Tanner (2008) Wenxin Jiang and Martin A. Tanner. 2008.  Gibbs Posterior for Variable Selection in High-dimensional Classification and Data Mining.  _Annals of Statistics_ 36, 5 (2008), 2207–2231.  https://doi.org/10.1214/07-AOS547
  * Kantas et al. (2014) Nikolas Kantas, Alexandros Beskos, and Ajay Jasra. 2014\.  Sequential Monte Carlo Methods for High-dimensional Inverse Problems: A Case Study for the Navier–Stokes Equations.  _SIAM/ASA Journal on Uncertainty Quantification_ 2, 1 (2014), 464–489.  https://doi.org/10.1137/130930364
  * Kesten (1958) Harry Kesten. 1958\.  Accelerated Stochastic Approximation.  _Annals of Mathematical Statistics_ 29, 1 (1958), 41–59.
  * Kitagawa (1996) Genshiro Kitagawa. 1996\.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models.  _Journal of Computational and Graphical Statistics_ 5, 1 (1996), 1–25.  https://doi.org/10.1080/10618600.1996.10474692
  * Kong et al. (1994) Augustine Kong, Jun S. Liu, and Wing Hung Wong. 1994\.  Sequential Imputations and Bayesian Missing Data Problems.  _J. Amer. Statist. Assoc._ 89, 425 (1994), 278–288.  https://doi.org/10.1080/01621459.1994.10476469
  * Liu (2001) Jun S. Liu. 2001\.  _Monte Carlo Strategies in Scientific Computing_.  Springer.
  * Lyddon et al. (2019) S. P. Lyddon, C. C. Holmes, and S. G. Walker. 2019\.  General Bayesian Updating and the Loss-likelihood Bootstrap.  _Biometrika_ 106, 2 (03 2019), 465–478.  https://doi.org/10.1093/biomet/asz006
  * Martin and Syring (2022) Ryan Martin and Nicholas Syring. 2022.  Direct Gibbs Posterior Inference on Risk Minimizers: Construction, Concentration, and Calibration.  In _Advancements in Bayesian Methods and Implementation_ , Arni S.R. Srinivasa Rao, G. Alastair Young, and C.R. Rao (Eds.). Handbook of Statistics, Vol. 47. Elsevier, Chapter 1, 1–41.  https://doi.org/10.1016/bs.host.2022.06.004
  * Miller (2021) Jeffrey W. Miller. 2021\.  Asymptotic Normality, Concentration, and Coverage of Generalized Posteriors.  _Journal of Machine Learning Research_ 22, 168 (2021), 1–53.
  * Miller and Dunson (2019) Jeffrey W. Miller and David B. Dunson. 2019.  Robust Bayesian Inference via Coarsening.  _J. Amer. Statist. Assoc._ 114, 527 (2019), 1113–1125.  https://doi.org/10.1080/01621459.2018.1469995 PMID: 31942084.
  * Nott et al. (ming) David J. Nott, Christopher Drovandi, and David T. Frazier. forthcoming.  Bayesian Inference for Misspecified Generative Models.  _Annual Review of Statistics and Its Application_ (forthcoming).  https://doi.org/10.1146/annurev-statistics-040522-015915
  * Polson and Scott (2011) Nicholas G. Polson and Steven L. Scott. 2011.  Data Augmentation for Support Vector Machines.  _Bayesian Analysis_ 6, 1 (2011), 43–48.
  * Robbins and Monro (1951) Herbert Robbins and Sutton Monro. 1951.  A Stochastic Approximation Method.  _Annals of Mathematical Statistics_ 22, 3 (1951), 400–407.  http://www.jstor.org/stable/2236626
  * Syring and Martin (2019) Nicholas Syring and Ryan Martin. 2019.  Calibrating General Posterior Credible Regions.  _Biometrika_ 106, 2 (12 2019), 479–486.  https://doi.org/10.1093/biomet/asy054
  * Syring and Martin (2023) Nicholas Syring and Ryan Martin. 2023.  Gibbs Posterior Concentration Rates under Sub-exponential Type Losses.  _Bernoulli_ 29, 2 (2023), 1080–1108.  https://doi.org/10.3150/22-BEJ1491
  * Vats et al. (2019) Dootika Vats, James M. Flegal, and Galin L. Jones. 2019\.  Multivariate Output Analysis for Markov Chain Monte Carlo.  _Biometrika_ 106, 2 (04 2019), 321–337.  https://doi.org/10.1093/biomet/asz002
  * Vihola (2012) Matti Vihola. 2012\.  Robust Adaptive Metropolis Algorithm with Coerced Acceptance Rate.  _Statistics and Computing_ 22 (2012), 997–1008.  https://doi.org/10.1007/s11222-011-9269-5
  * Walker and Hjort (2001) Stephen Walker and Nils Lid Hjort. 2001.  On Bayesian Consistency.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 63, 4 (2001), 811–821.  https://doi.org/10.1111/1467-9868.00314
  * Whiteley et al. (2016) Nick Whiteley, Anthony Lee, and Kari Heine. 2016.  On the Role of Interaction in Sequential Monte Carlo Algorithms.  _Bernoulli_ 22, 1 (2016), 494–529.  https://doi.org/10.3150/14-BEJ666
  * Wu and Martin (2023) Pei-Shien Wu and Ryan Martin. 2023.  A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference.  _Bayesian Analysis_ 18, 1 (2023), 105–132.  https://doi.org/10.1214/21-BA1302
  * Zhang (2006a) Tong Zhang. 2006a.  From $\varepsilon$-entropy to KL-entropy: Analysis of Minimum Information Complexity Density Estimation.  _Annals of Statistics_ 34, 5 (2006), 2180–2210.  https://doi.org/10.1214/009053606000000704
  * Zhang (2006b) Tong Zhang. 2006b.  Information-theoretic Upper and Lower Bounds for Statistical Estimation.  _IEEE Transactions on Information Theory_ 52, 4 (2006), 1307–1321.  https://doi.org/10.1109/TIT.2005.864439
  * Zhou et al. (2016) Yan Zhou, Adam M. Johansen, and John A.D. Aston. 2016\.  Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach.  _Journal of Computational and Graphical Statistics_ 25, 3 (2016), 701–726.  https://doi.org/10.1080/10618600.2015.1060885 arXiv:https://doi.org/10.1080/10618600.2015.1060885