lemmatheorem lemma corollarytheorem corollary propositiontheorem proposition
definitiontheorem definition remarktheorem remark

# On quantitative Laplace-type convergence results for some exponential
probability measures, with two applications

Valentin De Bortoli 111corresponding author<EMAIL_ADDRESS>Department of Statistics, University of Oxford, UK  Agnès Desolneux
<EMAIL_ADDRESS>Centre Borelli, CNRS and ENS Paris-
Saclay, France

###### Abstract

Laplace-type results characterize the limit of sequence of measures
$(\pi_{\varepsilon})_{\varepsilon>0}$ with density w.r.t the Lebesgue measure
$(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)\propto\exp[-U(x)/\varepsilon]$
when the temperature $\varepsilon>0$ converges to $0$. If a limiting
distribution $\pi_{0}$ exists, it concentrates on the minimizers of the
potential $U$. Classical results require the invertibility of the Hessian of
$U$ in order to establish such asymptotics. In this work, we study the
particular case of norm-like potentials $U$ and establish quantitative bounds
between $\pi_{\varepsilon}$ and $\pi_{0}$ w.r.t. the Wasserstein distance of
order $1$ under an invertibility condition of a generalized Jacobian. One key
element of our proof is the use of geometric measure theory tools such as the
coarea formula. We apply our results to the study of maximum entropy models
(microcanonical/macrocanonical distributions) and to the convergence of the
iterates of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm at low
temperatures for non-convex minimization.

###### Contents

  1. 1 Introduction
  2. 2 Limit theorems
  3. 3 Applications
    1. 3.1 Maximum entropy distribution
      1. 3.1.1 Maximum entropy models
      2. 3.1.2 From macrocanonical to microcanonical
      3. 3.1.3 Some simple experiments
    2. 3.2 Non-convex minimization
      1. 3.2.1 Non-convex setting and related work
      2. 3.2.2 Quantitative convergence for SGLD
      3. 3.2.3 The importance of the _thermodynamic barrier_
  4. 4 Proofs
    1. 4.1 Proof of Theorem 2
      1. 4.1.1 The case $d\geq p$
      2. 4.1.2 The case $d\leq p$
      3. 4.1.3 Proof of Theorem 2
    2. 4.2 Proofs of Section 3.1
      1. 4.2.1 The case $d\geq p$
      2. 4.2.2 The case $d\leq p$
      3. 4.2.3 Main result
    3. 4.3 Proofs of Section 3.2
      1. 4.3.1 Proof of Section 3.2.2
      2. 4.3.2 Proof of Section 3.2.2
      3. 4.3.3 Proof of Section 3.2.2
  5. A Technical results for Section 2
    1. A.1 From normal Hessian to generalized Jacobian
    2. A.2 Truncation and lower bounds
    3. A.3 Quantitative control of the derivative
    4. A.4 Regularity results
  6. B Technical results for Section 4.3.1
    1. B.1 Parametric lower and truncation bounds
    2. B.2 Quantitative Morse lemma and parametric Laplace-type results
    3. B.3 Control of the moments
  7. C Basics on flows and geometric measure theory

###### Contents

  1. 1 Introduction
  2. 2 Limit theorems
  3. 3 Applications
    1. 3.1 Maximum entropy distribution
      1. 3.1.1 Maximum entropy models
      2. 3.1.2 From macrocanonical to microcanonical
      3. 3.1.3 Some simple experiments
    2. 3.2 Non-convex minimization
      1. 3.2.1 Non-convex setting and related work
      2. 3.2.2 Quantitative convergence for SGLD
      3. 3.2.3 The importance of the _thermodynamic barrier_
  4. 4 Proofs
    1. 4.1 Proof of Theorem 2
      1. 4.1.1 The case $d\geq p$
      2. 4.1.2 The case $d\leq p$
      3. 4.1.3 Proof of Theorem 2
    2. 4.2 Proofs of Section 3.1
      1. 4.2.1 The case $d\geq p$
      2. 4.2.2 The case $d\leq p$
      3. 4.2.3 Main result
    3. 4.3 Proofs of Section 3.2
      1. 4.3.1 Proof of Section 3.2.2
      2. 4.3.2 Proof of Section 3.2.2
      3. 4.3.3 Proof of Section 3.2.2
  5. A Technical results for Section 2
    1. A.1 From normal Hessian to generalized Jacobian
    2. A.2 Truncation and lower bounds
    3. A.3 Quantitative control of the derivative
    4. A.4 Regularity results
  6. B Technical results for Section 4.3.1
    1. B.1 Parametric lower and truncation bounds
    2. B.2 Quantitative Morse lemma and parametric Laplace-type results
    3. B.3 Control of the moments
  7. C Basics on flows and geometric measure theory

##  1 Introduction

Asymptotic expansions of integrals are ubiquitous in probability theory with
applications in simulated annealing [34, 58], Bayesian inference [41, 65, 63],
statistical physics [25] or chaos expansion [48]. In this paper, we are
interested in families of Gibbs probability measures $\pi_{\varepsilon}$ given
by the density w.r.t the Lebesgue measure
$(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)\propto\exp[-\|F(x)\|^{k}/\varepsilon]$,
for any $x\in\mathbb{R}^{d}$, where $k\in\mathbb{N}$ and $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ is some function with $F(0)=0$. In
particular, we derive non asymptotic bounds between
$(\pi_{\varepsilon})_{\varepsilon>0}$ and $\pi_{0}$ when $\varepsilon\to 0$.

Taking a step back, one way to deal with expansion of integrals of the form
$\int_{\mathbb{R}^{d}}\exp[-U(x)/\varepsilon]\mathrm{d}x$ where $U:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$ is to rely on Laplace-type techniques, see
[6, 57, 22, 28, 26, 30, 5, 70] for instance. Assuming that $U$ admits a unique
minimizer $U(0)=0$ and under additional regularity conditions, it can be shown
that if the Hessian of $U$ evaluated at $0$ is invertible then
$\int_{\mathbb{R}^{d}}\exp[-U(x)/\varepsilon]\mathrm{d}x\sim
C_{U}\varepsilon^{d/2}$, with explicit constant $C_{U}>0$ (series expansions
are also available under similar conditions). This result on asymptotic
integrals can be immediately used to prove the convergence of probability
measures with density w.r.t. the Lebesgue measure
$(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)\propto\exp[-U(x)/\varepsilon]$,
see [44] for instance. However the invertibility of the Hessian is a
restricting condition which is not satisfied in the setting of this paper
where $U(x)=\|F(x)\|^{k}$ in the case $p\leq d$.

When the Hessian $\nabla^{2}U(0)$ is not invertible, the asymptotic integral
expansion becomes _degenerate_ and is more difficult to analyze. However,
several approaches have been proposed in order to tackle this issue. For
instance in [64] a multidimensional version of the Watson lemma is established
under restrictive conditions on the potential $U$ near its singularities.
Another approach consists in integrating over the manifold of minimizers and
apply local change of variables [48], see also [4, 11]. In particular, [48]
derives an asymptotic integral expansion similar to the one obtained in the
non-degenerate case, under some invertibility condition of the minors of the
Hessian. Similarly, [44] uses the tubular neighborhood theorem to obtain that
$\pi_{\varepsilon}$ converges to a limiting measure under the assumption that
the minimizers of $U$ can be partitioned into a collection of manifolds.
Closer to the method introduced in the present paper, [18, 3] propose to use
the so-called _Gelfand-Leray forms_ to tackle the non-degeneracy problem. By
integrating over the level sets it can be shown that

$\textstyle{\int_{\mathsf{V}}\varphi(x)\exp[-U(x)/\varepsilon]\mathrm{d}x=\int_{0}^{+\infty}\exp[-t/\varepsilon](\int_{U^{-1}(t)\cap\mathsf{V}}\varphi(x)\omega_{U}(x))\mathrm{d}t\;,}$
(1)

where $\mathsf{V}$ is an open set,
$\varphi\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R})$ a test function and
$\omega_{U}$ the so-called Gelfand-Leray form associated with $U$. Then by
establishing regularity properties for the functional
$t\mapsto\int_{U^{-1}(t)}\varphi(x)\omega_{U}(x)$ one can recover asymptotic
integral expansion in the case where $U$ is analytic, see [18, 3]. This theory
does not rely on any invertibility condition on the Hessian of $U$ but the
exponents appearing in the asymptotic expansion are usually not available in
closed form in general since their derivation relies on resolution of
singularities [42].

In this paper, we consider a different approach which also does not rely on
invertibility conditions on the Hessian of $U$ at singularity points but
allows us to derive quantitative expansions. To do so, we restrict the set of
functions $U:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ to the set of norm-like
functions, i.e. $U$ is norm-like if there exists $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ and $k\in\mathbb{N}$ such that for any
$x\in\mathbb{R}^{d}$, $U(x)=\|F(x)\|^{k}$ (however our results can be extended
to general potentials $U$ under classical invertibility conditions on the
Hessian of $U$). Instead of invertibility conditions of the Hessian of $U$ we
require invertibility conditions on the generalized Jacobian of $F$ to be
defined below. Our approach relies on tools from the geometric measure theory
and in particular the coarea formula, see [1] for instance. Indeed, using this
formula we are able to establish a similar result as (1) where the Gelfand-
Leray form is replaced by a twisted Hausdorff measure $\nu$. By establishing
the Lipschitz regularity of the mapping
$t\mapsto\int_{U^{-1}(t)\cap\mathsf{V}}\varphi(x)\mathrm{d}\nu(x)$ we are able
to provide quantitative bounds for the integral expansion. This expansion is
the key to our main result which establishes quantitative bounds w.r.t. the
Wasserstein distance of order $1$ between $\pi_{\varepsilon}$ and a limit
probability measure $\pi_{0}$ for $\varepsilon>0$ small enough. Concurrently
to this work, [10] establishes similar quantitative results in the case where
the Hessian of $U$ is invertible.

One of our main motivation for this study is the application of Laplace-type
results to maximum entropy models. In particular, using our quantitative
bounds, we are able to provide a link between two possible maximum entropy
models commonly used in statistical physics and image processing [12, 21].
Given a reference measure $\mu$ and constraints $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ it is possible to define the _microcanonical
model_ , which corresponds to the probability measure $\pi_{\mathrm{mic}}$
with the minimum Kullback-Leibler divergence w.r.t $\mu$ and such that
$\pi_{\mathrm{mic}}$ is supported on the set
$\\{x\in\mathbb{R}^{d}\,:\;F(x)=0\\}$. Similarly, one can define the
_macrocanonical model_ with level $\varepsilon>0$ denoted
$\pi_{\mathrm{mac}}^{\varepsilon}$ such that
$\pi_{\mathrm{mac}}^{\varepsilon}$ minimizes the Kullback-Leibler divergence
w.r.t $\mu$ and satisfies the integrability condition
$\int_{\mathbb{R}^{d}}\|F(x)\|^{2}\mathrm{d}\pi_{\mathrm{mac}}^{\varepsilon}(x)=\varepsilon$.
Using results from information geometry [19] and under mild regularity
conditions, $\pi_{\varepsilon}$ can be written as a Gibbs measure and its
limiting behavior when $\varepsilon\to 0$ can be investigated in the context
of asymptotic expansions of integrals. In this work, we show that the limit of
the macrocanonical models when $\varepsilon\to 0$ is a twisted microcanonical
model and propose an algorithm to asymptotically recover the original
microcanonical distribution.

Another application we consider in this paper is the non-asymptotic study of
Stochastic Gradient Langevin Dynamics (SGLD) [68], a popular algorithm used to
approximate the minimizers of non-convex functions in a machine learning
setting. Given a potential (or population risk) $U:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$ which can be written for any
$x\in\mathbb{R}^{d}$ as $U(x)=\int_{\mathsf{Z}}u(x,z)\mathrm{d}\mu(z)$, we
assume that we have access to an empirical version of this risk for any
$n\in\mathbb{N}$ and $z^{1:n}=\\{z_{i}\\}_{i=1}^{n}\in\mathsf{Z}^{n}$ given by
$U_{n}(x,z^{1:n})=(1/n)\sum_{i=1}^{n}u(x,z_{i})$ as well as an unbiased
estimator of its gradient denoted $g$. Conditionally to the samples $z^{1:n}$,
SGLD corresponds to the recursion associated with an unadjusted Langevin
dynamics [61, 24, 20] with target $U_{n}(\cdot,z^{1:n})/\varepsilon$ for a
small value of $\varepsilon>0$ and where the gradient of
$U_{n}(\cdot,z^{1:n})$ is replaced by its unbiased estimator $g$. Since the
invariant measure of the underlying Langevin diffusion is given by
$\pi_{\varepsilon}$ with density w.r.t. the Lebesgue measure
$(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)\propto\exp[-U_{n}(x,z^{1:n})/\varepsilon]$
we obtain that for small values of the parameter $\varepsilon>0$, the samples
$(X_{k})_{k\in\mathbb{N}}$ are approximately concentrated around the
minimizers of $U_{n}(\cdot,z^{1:n})$. In existing works on SGLD [60, 14, 32,
27, 55, 71] all quantitative theoretical guarantees are obtained w.r.t. the
sequence of $(\mathbb{E}[U(X_{k})])_{k\in\mathbb{N}}$. On the contrary, we
characterize the limiting distribution of SGLD and establish non-asymptotic
quantitative bounds between SGLD and this limit. Finally, we show that the
support of the limiting distribution is included into (but not necessarily
equal to) the set of minimizers of $U$ when $n$ is large. This last result
relies on the notion of _thermodynamic barrier_ which we introduce in the
context of integral expansion. To our knowledge this is the first time that
the importance of such a barrier to derive parametric Laplace-type results is
highlighted (as opposed to the _kinetic barrier_ which is a well-known
quantity in simulated annealing [40]).

To summarize, our main contributions are three-fold:

  1. (i)

We establish quantitative bounds between $\pi_{\varepsilon}$ and $\pi_{0}$
under the assumption that the potential $U$ is norm-like and additional
regularity conditions. In particular we provide an upper-bound between
$\pi_{\varepsilon}$ and $\pi_{0}$ w.r.t. the Wasserstein distance of order
$1$. We emphasize that our results also hold under the more classical
invertibility condition on the Hessian of $U$.

  2. (ii)

We apply this result to show that under mild conditions the limit of a natural
class of macrocanonical models is _not_ the original microcanonical model.
However, we prove that a twisted sequence of macrocanonical models converges
to the microcanonical distribution. This observation allows us to construct a
Langevin-based algorithm in order to sample from this distribution. We
illustrate our method in low-dimensional settings.

  3. (iii)

Finally, we apply our theory to study the behavior of the iterates of SGLD in
the context of non-convex optimization. In particular, we characterize the
limiting distribution of SGLD at low temperature and show that it concentrates
on the minimizers of the population loss for a large number of samples using
the notion of _thermodynamic barrier_. Note that in this study, we no longer
assume that $U$ is norm-like but instead rely on invertibility conditions on
the Hessian of $U$.

The rest of the paper is organized as follows. In Section 2 we present our
main results, i.e. quantitative bounds between $\pi_{\varepsilon}$ and the
limiting distribution $\pi_{0}$. In Section 3, we present our two main
applications: the links between the macrocanonical and microcanonical maximum
entropy models in Section 3.1, and a study of the convergence of SGLD for non-
convex minimization in Section 3.2. The proofs of our results are gathered in
Section 4.

## Notation

Let $d\in\mathbb{N}^{\star}$. We denote $\\{e_{i}\\}_{i=1}^{d}$ the canonical
basis of $\mathbb{R}^{d}$. Let $\langle\cdot,\cdot\rangle$ be the Euclidean
scalar product over $\mathbb{R}^{d}$, and $\|\cdot\|$ be the corresponding
norm. We denote $\operatorname{B}(x,r)$ the ball with center
$x\in\mathbb{R}^{d}$ and radius $r>0$ w.r.t. the norm $\|\cdot\|$. Similarly,
we denote $\operatorname{B}_{\infty}(x,r)$ the ball with center
$x\in\mathbb{R}^{d}$ and radius $r>0$ w.r.t the norm $\|\cdot\|_{\infty}$
given for any $x=(x_{1},\dots,x_{d})\in\mathbb{R}^{d}$ by
$\|x\|_{\infty}=\sup_{i\in\\{1,\dots,d\\}}\left|x_{i}\right|$.

Let $\mathsf{A}\subset\mathbb{R}^{d}$, we denote
$\operatorname{diam}(\mathsf{A})=\sup_{x,y\in\mathsf{A}}\|x-y\|$ its diameter
and $\mathsf{A}^{\mathrm{c}}$ its complementary set. Let
$\mathcal{B}(\mathbb{R}^{d})$ denote the Borel $\sigma$-field of
$\mathbb{R}^{d}$. Let $\mathsf{U}$ be an open set of $\mathbb{R}^{d}$,
$n\in\mathbb{N}^{\star}$ and let $\mathrm{C}^{n}(\mathsf{U},\mathbb{R}^{p})$
be the set of the $n$-differentiable $\mathbb{R}^{p}$-valued functions defined
over $\mathsf{U}$. If $p=1$ we simply denote $\mathrm{C}^{n}(\mathsf{U})$. Let
$f\in\mathrm{C}^{1}(\mathsf{U})$, we denote by $\nabla f$ its gradient.
Furthermore, if $f\in\mathrm{C}^{2}(\mathsf{U})$ we denote $\nabla^{2}f$ its
Hessian and $\Delta f$ its Laplacian. By convention we denote $\nabla^{0}f=f$.
We also denote $\mathrm{C}(\mathsf{U},\mathbb{R}^{p})$ the set of continuous
functions defined over $\mathsf{U}$. Let $f:\mathsf{A}\to\mathbb{R}^{p}$ with
$p\in\mathbb{N}^{\star}$ and $\mathsf{A}\subset\mathbb{R}^{d}$. The function
$f$ is said to be $\mathtt{L}$-Lipschitz with $\mathtt{L}\geq 0$ if for any
$x,y\in\mathsf{A}$,
$\left\|f(x)-f(y)\right\|\leq\mathtt{L}\left\|x-y\right\|$. Let $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ such that $F$ is differentiable at
$x\in\mathbb{R}^{d}$ and denote
$\mathrm{D}F(x)=(\partial_{j}F_{i}(x))_{i\in\\{1,\dots,p\\},j\in\\{1,\dots,d\\}}$
the Jacobian matrix of $F$ which is a $p\times d$ matrix. We define the
generalized Jacobian $\mathrm{J}F:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ given for
any $x\in\mathbb{R}^{d}$ by

$\mathrm{J}F(x)=\left\\{\begin{matrix}\textstyle{\det(\mathrm{D}F(x)\mathrm{D}F(x)^{\top})^{1/2}\;,\qquad\text{if
$d\geq p$,}}\\\
\textstyle{\det(\mathrm{D}F(x)^{\top}\mathrm{D}F(x))^{1/2}\;,\qquad\text{if
$d\leq p$.}}\end{matrix}\right.$ (2)

Finally, if $F\in\mathrm{C}^{\ell}(\mathbb{R}^{d},\mathbb{R}^{p})$ with
$\ell\in\mathbb{N}$, we define $\mathrm{D}^{j}F$ for any
$j\in\\{1,\dots,\ell\\}$, recursively by
$\mathrm{D}^{j+1}F=\mathrm{D}(\mathrm{D}^{j}F)$. Note that for any
$j\in\\{0,\dots,\ell\\}$, $\mathrm{D}^{j}F$ can be represented as $p\times
d\times\dots\times d$ tensor (where $d$ appears $j$ times) with symmetric last
$j$ coordinates. Hence, we can define $\mathrm{D}^{j+1}F^{\top}$ the tensor
where the first and last dimension have been exchanged, which is a
$d\times\dots d\times p$ tensor. If $p=1$ then, we write
$\nabla^{j}F=\mathrm{D}^{j}F$ for any $j\in\\{1,\dots,\ell\\}$.

Let $(\mathsf{X},\mathcal{X})$ be a measurable space. We denote by
$\mathbb{F}(\mathsf{X})$ the set of the
$\mathcal{X}/\mathcal{B}(\mathbb{R})$-measurable real functions over
$\mathsf{X}$. Let $\mathbb{M}(\mathcal{X})$ be the set of finite signed
measures over $\mathcal{X}$ and let $\mu\in\mathbb{M}(\mathcal{X})$. For
$f\in\mathbb{F}(\mathsf{X})$ a $\mu$-integrable function we denote

$\textstyle{\mu[f]=\int_{\mathsf{X}}f(x)\mathrm{d}\mu(x)\;,}$ (3)

the integral of $f$ w.r.t. to $\mu$ when it is well-defined. We also define
$\mathscr{P}(\mathsf{X},\mathcal{X})$ the set of probability measures over
$\mathcal{X}$ and when there is no ambiguity on the sigma-field we simply
denote it by $\mathscr{P}(\mathsf{X})$. We denote by $\lambda$ the Lebesgue
measure on $\mathbb{R}^{d}$. Let $(\mathsf{X},\mathcal{X})$ and
$(\mathsf{Y},\mathcal{Y})$ be two measurable spaces. A Markov kernel
$\mathrm{K}$ is a mapping $\mathrm{K}:\
\mathsf{X}\times\mathcal{Y}\to\left[0,1\right]$ such that for any
$x\in\mathsf{X}$, $\mathrm{K}(x,\cdot)\in\mathscr{P}(\mathsf{Y},\mathcal{Y})$
and for any $\mathsf{A}\in\mathcal{Y}$, $\mathrm{K}(\cdot,\mathsf{A})$ is
measurable.

Let $\mu,\nu\in\mathscr{P}(\mathsf{X},\mathcal{X})$. A probability measure
$\zeta\in\mathscr{P}(\mathsf{X}\times\mathsf{X},\mathcal{X}\otimes\mathcal{X})$
is said to be a transference plan between $\mu$ and $\nu$ if for any
$\mathsf{A}\in\mathcal{X}$,
$\zeta(\mathsf{A}\times\mathcal{X})=\mu(\mathsf{A})$ and
$\zeta(\mathcal{X}\times\mathsf{A})=\nu(\mathsf{A})$. We denote by
$\mathbf{T}(\mu,\nu)$ the set of all transference plans between $\mu$ and
$\nu$. We define the Wasserstein metric/distance of order $\ell\geq 1$
$\mathbf{W}_{\ell}(\mu,\nu)$ between $\mu$ and $\nu$ by

$\textstyle{\mathbf{W}_{\ell}(\mu,\nu)^{\ell}=\inf_{\zeta\in\mathbf{T}(\mu,\nu)}\int_{\mathsf{X}^{2}}\|x-y\|^{\ell}\mathrm{d}\zeta(x,y)\;.}$
(4)

We denote $\mathcal{H}^{r}$ the Hausdorff measure of order $r>0$, given for
any $\mathsf{A}\subset\mathbb{R}^{d}$ by
$\mathcal{H}^{r}(\mathsf{A})=\lim_{\delta\to
0}\mathcal{H}^{r,\delta}(\mathsf{A})$, where for any $\delta>0$ we have

$\textstyle{\mathcal{H}^{r,\delta}(\mathsf{A})=\inf\\{\sum_{i\in\mathbb{N}}\mathrm{diam}(\mathsf{U}_{i})^{r}\,:\;\mathsf{A}\subset\cup_{i\in\mathbb{N}}\mathsf{U}_{i}\;,\
\mathrm{diam}(\mathsf{U}_{i})\leq\delta\\}\;.}$ (5)

Basic facts on the Hausdorff measure are gathered in Appendix C. Finally, we
denote by $\mathtt{Poly}(k,\mathsf{A})$ the set of polynomials of
$k\in\mathbb{N}$ variables with coefficients in $\mathsf{A}\subset\mathbb{R}$.

##  2 Limit theorems

In this section, we state our main theorem and draw links with previous
approximation results for probability integrals. Let $k\in\mathbb{N}$,
$\bar{\varepsilon}>0$, $F:\ \mathbb{R}^{d}\to\mathbb{R}^{p}$ with
$d,p\in\mathbb{N}^{\star}$ such that for any
$\varepsilon\in(0,\bar{\varepsilon})$,
$\int_{\mathbb{R}^{d}}\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x<+\infty$.
For any $\varepsilon\in(0,\bar{\varepsilon})$, we define
$\pi_{\varepsilon}\in\mathscr{P}(\mathbb{R}^{d})$ such that for any
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$

$\textstyle{\pi_{\varepsilon}(\mathsf{A})=\left.\int_{\mathsf{A}}\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x\middle/\int_{\mathbb{R}^{d}}\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x\right.\;.}$
(6)

Our goal is to study the behavior of $\pi_{\varepsilon}$ when $\varepsilon\to
0$. In particular, we identify a limit $\pi_{0}\in\mathscr{P}(\mathbb{R}^{d})$
and derive non-asymptotic convergence bounds. For any
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ we let (when it is well-defined)

$\textstyle{\pi_{0}(\mathsf{A})=\left.\int_{F^{-1}(0)\cap\mathsf{A}}\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-\min(d,p)}(x)\middle/\int_{F^{-1}(0)}\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-\min(d,p)}(x)\right.\;.}$
(7)

In what follows we consider the following assumption on $F$ which implies that
$\pi_{\varepsilon}$ is well-defined for any $\varepsilon\geq 0$.

######  H 1.

$F\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R}^{p})$ and $F(0)=0$. For any
$x\in F^{-1}(0)$, $\mathrm{J}F(x)\neq 0$. There exist $\mathtt{m},\upalpha>0$
and $R\geq 0$ such that for any $x\in\mathbb{R}^{d}$ with
$\left\|x\right\|\geq R$,
$\left\|F(x)\right\|\geq\mathtt{m}\left\|x\right\|^{\upalpha}$.

A few remarks are in order. First, the assumption $F(0)=0$ is only technical
and can be replaced by $F^{-1}(0)\neq\emptyset$. Similarly, the smoothness
assumption $F\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R}^{p})$ allows us
to avoid some technicalities in the proofs but can be relaxed. Second, we
assume that $x\mapsto\|F(x)\|$ grows at least polynomially when
$\left\|x\right\|\to+\infty$. This condition can also be relaxed to handle
sub-polynomial growth at infinity. However, changing the rate of growth might
affect the quantitative convergence properties of
$(\pi_{\varepsilon})_{\varepsilon>0}$ towards $\pi_{0}$. A study of sub-
polynomial growth is left for future work. Finally the assumption that for any
$x\in F^{-1}(0)$, $\mathrm{J}F(x)\neq 0$ is necessary in order to define
$\pi_{0}$.

######  Theorem 1.

Assume H 1. Then for any $\varepsilon\geq 0$, $\pi_{\varepsilon}$ is well-
defined. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open and bounded such that
$F^{-1}(0)\subset\mathsf{U}$. Let
$\varphi\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R})$ and $C_{\varphi}\geq 0$ such
that for any $x\in\mathbb{R}^{d}$

$\textstyle{\left|\varphi(x)\right|\leq
C_{\varphi}\exp[C_{\varphi}\left\|x\right\|^{\upalpha k}]\;,}$ (8)

with $\upalpha>0$ defined in H 1 and $k\in\mathbb{N}^{\star}$ in (6). Then
$\lim_{\varepsilon\to
0}\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|=0$. In addition, if
$\varphi$ is $M_{1,\varphi}$-Lipschitz on $\mathsf{U}$ with $M_{1,\varphi}\geq
0$, then there exist
$A\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+}),\bar{\varepsilon}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+})$
such that for any $\varepsilon\in(0,\bar{\varepsilon}(C_{\varphi}))$ we have

$\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|\leq
A(C_{\varphi})(1+M_{0,\varphi}+M_{1,\varphi})\varepsilon^{1/k}\;,$ (9)

with $M_{0,\varphi}=\sup\\{|\varphi(x)|\,:\;x\in\mathsf{U}\\}$,
$A,\bar{\varepsilon}$ independent functions from $\varphi$, $A$ is non-
decreasing and $\bar{\varepsilon}$ is non-increasing.

###### Proof.

We provide a sketch of the proof. The whole proof is postponed to Section 4.1.
We first start by showing that we can restrict our study to versions of
$\pi_{\varepsilon}$ and $\pi_{0}$ with truncated support. Then our study
differs depending on if $d\geq p$ or $d\leq p$. If $d\geq p$ then we use tools
from geometric measure theory, and in particular the coarea formula in
combination with the Lipschitz property of applications of the form
$t\mapsto\int_{F^{-1}(t)}\varphi(x)\mathrm{d}\mathcal{H}^{d-p}(x)$. In the
case $d\leq p$ we adapt arguments from Morse theory and Laplace theory to
derive quantitative bounds on
$\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|$. These analyses are
conducted with smooth test functions
$\varphi\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$ and we relax this
hypothesis using smoothing arguments. ∎

We highlight a few key points from this theorem and draw links with the
existing literature.

  1. (a)

Our result is related to the Laplace-type convergence results of [70, 48, 4,
11, 11, 6, 57, 22, 28, 26, 30, 5]. In these works, the authors study integrals
of the form
$\int_{\mathbb{R}^{d}}\varphi(x)\exp[-U(x)/\varepsilon]\mathrm{d}x$ in the
limiting case where $\varepsilon\to 0$ under regularity assumptions on $U$ and
$\varphi$, and the non-degeneracy condition that for any
$x\in\operatorname*{arg\,min}_{\mathbb{R}^{d}}U$, $\nabla^{2}U(x)$ is
invertible. We say that $U:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ is norm-like
with exponent $2$ if there exist $p\in\mathbb{N}^{\star}$ and $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ such that $U(x)=\left\|F(x)\right\|^{2}$. In
this case, and under additional regularity assumptions, we have for any
$x\in\mathbb{R}^{d}$

$\nabla^{2}U(x)=2\mathrm{D}^{2}F(x)^{\top}F(x)+2\mathrm{D}F(x)^{\top}\mathrm{D}F(x)\;.$
(10)

In particular for any $x\in F^{-1}(0)$, $\nabla^{2}U(x)$ is invertible if and
only if $\mathrm{J}F(x)\neq 0$ which is precisely the non-degeneracy condition
imposed in H 1 (note that this directly implies that $d\leq p$). For any
$U\in\mathrm{C}^{3}(\mathbb{R}^{d},\mathbb{R}_{+})$ with Lipschitz third order
derivatives, there exist $p\in\mathbb{N}^{\star}$ and
$F\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R}^{p})$ with Lipschitz derivatives
such that for any $x\in\mathbb{R}^{d}$, $U(x)=\left\|F(x)\right\|^{2}$, see
[7, 31]. Hence, our restriction to functions $U:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$ of the form $U=\left\|F\right\|^{k}$ for some
$F:\ \mathbb{R}^{d}\to\mathbb{R}^{p}$ with $p,k\in\mathbb{N}^{\star}$ is not
too constraining under additional regularity assumptions.

  2. (b)

We emphasize that our results can be extended to a general potential
$U\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R}_{+})$ if the Hessian of $U$
is invertible on the set $U^{-1}(0)$, thus recovering the usual setting of
Laplace-type convergence results.

  3. (c)

As highlighted previously, we aim at providing quantitative bounds for
$\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|$. In the case where
$d\leq p$ we adapt classical arguments from Morse and Laplace theory. Our main
contribution is the use of geometric measure theory to cover the case $d\geq
p$ which cannot be treated with classical Laplace arguments. The case $d\geq
p$ is of particular interest in machine learning applications requiring to
sample from distributions of the form
$(\mathrm{d}\pi/\mathrm{d}\lambda)(x)\propto\exp[-\left\|F(x)\right\|^{2}]$,
where $F:\ \mathbb{R}^{d}\to\mathbb{R}^{p}$ is an encoder neural network with
$d\geq p$. In Section 3.1 we investigate this situation in details in the case
of generative modeling with macrocanonical and microcanonical models.

As a by-product of Theorem 1 we obtain the following corollary which
establishes quantitative bounds for the Wasserstein distance of order $1$
between $\pi_{\varepsilon}$ and $\pi_{0}$.

######  Corollary .

Assume H 1. Then for any $\varepsilon\geq 0$, $\pi_{\varepsilon}$ is well-
defined. In addition, there exist $A_{1}\geq 0$ and $\bar{\varepsilon}>0$ such
that for any $\varepsilon\in\left[0,\bar{\varepsilon}\right)$ we have

$\mathbf{W}_{1}(\pi_{\varepsilon},\pi_{0})\leq A_{1}\varepsilon^{1/k}\;,$ (11)

where we recall that $k\in\mathbb{N}^{\star}$ is defined in (6).

###### Proof.

Denote $\mathrm{Lip}=\\{\varphi:\
\mathbb{R}^{d}\to\mathbb{R}\,:\;\left|\varphi(x)-\varphi(y)\right|\leq\left\|x-y\right\|\;,\
\text{for any }x,y\in\mathbb{R}^{d}\\}$. In addition, denote
$\mathrm{Lip}_{0}=\\{\varphi:\
\mathbb{R}^{d}\to\mathbb{R}\,:\;\left|\varphi(x)-\varphi(y)\right|\leq\left\|x-y\right\|\;,\
\text{for any }x,y\in\mathbb{R}^{d},\ \varphi(0)=0\\}$. First, using [66,
Theorem 5.10] we have for any $\varepsilon>0$.

$\displaystyle\mathbf{W}_{1}(\pi_{\varepsilon},\pi_{0})=\sup\\{\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\,:\;\varphi\in\mathrm{Lip}\\}=\sup\\{\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|\,:\;\varphi\in\mathrm{Lip}_{0}\\}\;.$
(12)

Let $r=\upalpha k$ and $\beta_{r}=\left\lceil 1/r\right\rceil$. For any
$\varphi\in\mathrm{Lip}_{0}$ and $x\in\mathbb{R}^{d}$ with
$\left\|x\right\|\geq 1$ we have

$\beta_{r}\exp[\beta_{r}\left\|x\right\|^{\upalpha
k}]\geq\beta_{r}^{\beta_{r}}/(\beta_{r}!)\left\|x\right\|^{r\beta_{r}}\geq\left\|x\right\|\geq\left|\varphi(x)\right|\;.$
(13)

Similarly, if $\left\|x\right\|\leq 1$ we have that
$\left|\varphi(x)\right|\leq 1\leq\beta_{r}\exp[\beta_{r}\left\|x\right\|]$.
Hence for any $x\in\mathbb{R}^{d}$,
$\left|\varphi(x)\right|\leq\beta_{r}\exp[\beta_{r}\left\|x\right\|]$. For any
$\varphi\in\mathrm{Lip}_{0}$ we have $M_{1,\varphi}=1$. In addition, the set
$F^{-1}(0)$ is compact since
$\lim_{\left\|x\right\|\to+\infty}\left\|F(x)\right\|=+\infty$ and
$F\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R}^{p})$. Therefore there exists $R>0$
such that $F^{-1}(0)\subset\operatorname{B}(0,R)$. In particular, note that
$\mathsf{U}=\operatorname{B}(0,R)$ is open and bounded. We have that for any
$\varphi\in\mathrm{Lip}_{0}$ and $x\in\mathsf{U}$

$\left|\varphi(x)\right|\leq\left\|x\right\|\leq R\;.$ (14)

Therefore we get that $M_{0,\varphi}=R$. Combining these results with Theorem
1 we have for any $\varphi\in\mathrm{Lip}_{0}$ and
$\varepsilon\in\left[0,\bar{\varepsilon}(\left\lceil 1/r\right\rceil)\right)$

$\left|\pi_{\varepsilon}[\varphi]-\pi_{0}[\varphi]\right|\leq
A(C_{\varphi})(1+M_{0,\varphi}+M_{1,\varphi})\varepsilon^{1/k}\leq
2A(\left\lceil 1/r\right\rceil)(1+R)\varepsilon^{1/k}\;.$ (15)

We conclude the proof upon combining this result and (12). ∎

Concurrently to our work, [10] establishes similar quantitative bounds w.r.t
the Wasserstein distance of order $1$ without the norm-like assumption on the
potential $U$ but assuming that the Hessian of $U$ is invertible on
$\operatorname*{arg\,min}\\{U(x)\,:\;x\in\mathbb{R}^{d}\\}$, see [10, Lemma
4.6]. To do so, the authors derive estimates of the form
$\mathbf{W}_{1}(\pi_{\varepsilon_{1}},\pi_{\varepsilon_{2}})\leq
C|\varepsilon_{2}^{1/2}-\varepsilon_{1}^{1/2}|$ with $C\geq 0$ for
$\varepsilon_{1},\varepsilon_{2}>0$ using a coupling lemma and results from
Morse theory. The final bound is obtained by letting $\varepsilon_{2}\to 0$ in
the previous inequality and the fact that $\lim_{\varepsilon\to
0}\pi_{\varepsilon}=\pi_{0}$ weakly. Section 2 extends these bounds to the
case where the Hessian of $U$ is no longer invertible under a norm-like
condition on the potential.

Also, note that [44] establishes that $\lim_{\varepsilon\to
0}\pi_{\varepsilon}=\pi_{0}$ weakly without any norm-like assumption on $U$
under the condition that
$\mathsf{C}^{\star}=\operatorname*{arg\,min}\\{U(x)\,:\;x\in\mathbb{R}^{d}\\}$
can be partitioned into a collection of manifolds and that for any
$x\in\mathsf{C}^{\star}$, the Hessian of $U$ along the normal plan to the
manifold associated with $x$ is invertible (we denote
$\nabla_{\mathrm{N}}^{2}U(x)$ this quantity). In Section A.1, we show that if
$U=\|F\|^{2}$ then we have that
$\det(\nabla_{\mathrm{N}}^{2}U(x))=\mathrm{J}F(x)^{2}$ for any $x\in
F^{-1}(0)$. Hence, the invertibility condition can be written as: for any
$x\in F^{-1}(0)$, $\mathrm{J}F(x)\neq 0$ similarly to H 1 and the limiting
measure identified by [44] is exactly $\pi_{0}$.

The next result is an extension of Theorem 1 where, given
$k\in\mathbb{N}^{\star}$, $\Psi:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$,
$\pi_{\varepsilon}$ is replaced by $\pi_{\varepsilon}^{\Psi}$ defined for any
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ by

$\textstyle{\pi_{\varepsilon}^{\Psi}(\mathsf{A})=\left.\int_{\mathsf{A}}\Psi(x)\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x\middle/\int_{\mathbb{R}^{d}}\Psi(x)\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x\right.\;.}$
(16)

Similarly, for any $\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ we let (when it
is well-defined)

$\textstyle{\pi_{0}^{\Psi}(\mathsf{A})=\left.\int_{F^{-1}(0)\cap\mathsf{A}}\Psi(x)\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-\min(d,p)}(x)\middle/\int_{F^{-1}(0)}\Psi(x)\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-\min(d,p)}(x)\right.\;.}$
(17)

We consider the following assumption on $\Psi$ which ensures that
$\pi_{\varepsilon}^{\Psi}$ is well-defined for $\varepsilon\geq 0$ when
combined with H 1.

######  H 2.

$\Psi\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R}_{+})$, there exists $C_{\Psi}$
such that for any $x\in\mathbb{R}^{d}$, $\Psi(x)\leq
C_{\Psi}\exp[C_{\Psi}\left\|x\right\|^{\upalpha k}]$ and $\Psi(x)>0$ for any
$x\in F^{-1}(0)$.

Note that in H 2, $\upalpha>0$ is given in H 1 and $k\in\mathbb{N}^{\star}$ is
given in (6) Under this assumption, we derive the following quantitative
bounds.

######  Theorem 2.

Assume H 1 and H 2. Then for any $\varepsilon\geq 0$, $\pi_{\varepsilon}$ is
well-defined. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open and bounded such that
$F^{-1}(0)\subset\mathsf{U}$. Let
$\varphi\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R})$ and $C_{\varphi}\geq 0$ such
that for any $x\in\mathbb{R}^{d}$

$\left|\varphi(x)\right|\leq
C_{\varphi}\exp[C_{\varphi}\left\|x\right\|^{\upalpha k}]\;,$ (18)

with $\upalpha>0$ defined in H 1 and $k\in\mathbb{N}^{\star}$ in (6). Then
$\lim_{\varepsilon\to
0}\left|\pi_{\varepsilon}^{\Psi}[\varphi]-\pi_{0}^{\Psi}[\varphi]\right|=0$.
In addition, if $\varphi$ and $\Psi$ are respectively
$M_{1,\varphi}$-Lipschitz and $M_{1,\Psi}$-Lipschitz on $\mathsf{U}$ with
$M_{1,\varphi},M_{1,\Psi}\geq 0$, then there exist
$A\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+})$,
$\bar{\varepsilon}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+}^{\star})$ such
that for any $\varepsilon\in(0,\bar{\varepsilon}(C_{\varphi}))$ we have

$\textstyle{|\pi_{\varepsilon}^{\Psi}[\varphi]-\pi_{0}^{\Psi}[\varphi]|\leq
A(C_{\varphi})(1+M_{0,\varphi}+M_{1,\varphi})\varepsilon^{1/k}\;,}$ (19)

with $M_{0,\varphi}=\sup\\{|\varphi(x)|\,:\;x\in\mathsf{U}\\}$,
$A,\bar{\varepsilon}$ independent functions from $\varphi$, $A$ non-decreasing
and $\bar{\varepsilon}$ non-increasing.

###### Proof.

The proof is postponed to Section 4.1. ∎

The same remarks formulated after Theorem 1 hold for this extension. In
particular the norm-like assumption on $U$ can be omitted and replaced by a
classical invertibility condition on the Hessian at $U^{-1}(0)$. Note that
Theorem 1 is a direct consequence of Theorem 2 upon letting $\Psi=1$. An
important consequence of Theorem 2 is the case where for any
$x\in\mathbb{R}^{d}$, $\Psi(x)=\mathrm{J}F(x)$. Indeed, doing so we obtain
that $\pi_{0}^{\Psi}$ is the uniform distribution on $F^{-1}(0)$, i.e. the
maximum entropy distribution with support $F^{-1}(0)$. We discuss this special
case in Section 3.1. Similarly to Section 2 we can also establish the
following corollary.

######  Corollary .

Assume H 1 and H 2. Then for any $\varepsilon\geq 0$, $\pi_{\varepsilon}$ is
well-defined. In addition, assume that $\Psi$ is Lipschitz continuous. Then
there exist $A_{2}\geq 0$ and $\bar{\varepsilon}>0$ such that for any
$\varepsilon\in\left(0,\bar{\varepsilon}\right)$ we have

$\mathbf{W}_{1}(\pi_{\varepsilon}^{\Psi},\pi_{0}^{\Psi})\leq
A_{2}\varepsilon^{1/k}\;,$ (20)

where we recall that $k\in\mathbb{N}^{\star}$ is defined in (6).

###### Proof.

The proof is similar to the one of Section 2. ∎

##  3 Applications

In this section, we present two applications of our results. First, we draw
links between two different maximum entropy models in Section 3.1. Then, we
study the convergence of SGLD for non-convex minimization in Section 3.2.

###  3.1 Maximum entropy distribution

####  3.1.1 Maximum entropy models

Let $\mathsf{X}\subset\mathbb{R}^{d}$ be a compact space with
$\mathrm{int}(\mathsf{X})\neq\emptyset$ and $G:\mathsf{X}\to\mathbb{R}^{p}$
measurable. The maximum entropy model with constraint $G$ is denoted
$\pi_{\mathrm{Ent}}$ and is given by

$\textstyle{\pi_{\mathrm{Ent}}=\operatorname*{arg\,max}\\{H(\pi)\,:\;\pi\in\mathscr{P}_{\lambda}(\mathsf{X}),\
\pi[\|G\|]<+\infty,\ \pi[G]=0\\}}\;,$ (21)

where $\mathscr{P}_{\lambda}(\mathsf{X})$ is the set of probability measures
on $\mathsf{X}$ which admit a density w.r.t the Lebesgue measure,
$\pi[G]=\int_{\mathsf{X}}G(x)\mathrm{d}\pi(x)$ and
$H(\pi)=-\int_{\mathsf{X}}\log(\mathrm{d}\pi/\mathrm{d}\lambda)(x)\mathrm{d}\pi(x)$
for any $\pi\in\mathscr{P}_{\lambda}(\mathsf{X})$, where $\lambda$ is the
Lebesgue measure on $\mathbb{R}^{d}$. Note that $H(\pi)$ is well-defined (but
can be infinite) since $\mathsf{X}$ is compact and in this case
$\operatorname{KL}(\pi|\mu)=-H(\pi)+\log(\lambda(\mathsf{X}))$, where $\mu$ is
the uniform measure on $\mathsf{X}$ and we recall that $\lambda$ is the
Lebesgue measure.

Such maximum entropy models naturally arise in many areas such as statistical
physics [46, 12], econometrics [38], generative modeling [59, 52, 21],
reinforcement learning [76] or image processing [35, 23]. In what follows, we
introduce two extensions of (21). More precisely, we use the analogy between
the maximization of the entropy and the minimization of an appropriate
Kullback-Leibler divergence to extend (21) on non-compact spaces, following
[21].

##### Macrocanonical distributions

Given $\mu\in\mathscr{P}(\mathbb{R}^{d})$ and $G:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ measurable, we say that
$\pi^{\star}\in\mathscr{P}(\mathbb{R}^{d})$ is a macrocanonical distribution
[12] with reference measure $\mu$ and constraint $G$ if it satisfies

$\textstyle{\pi^{\star}\in\operatorname*{arg\,min}\\{\operatorname{KL}(\pi|\mu)\,:\;\pi\in\mathscr{P}(\mathbb{R}^{d}),\
\pi[\|G\|]<+\infty,\ \pi[G]=0\\}}\;.$ (22)

Using results from information geometry [19] any macrocanonical model can be
written as an exponential distribution under mild assumptions.

######  Proposition  ([21]).

Assume that $G\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R}^{p})$ and that there
exist $\upalpha,\upbeta,\eta>0$ with $\upbeta>\upalpha$ such that

$\textstyle{\sup\\{\|G(x)\|(1+\|x\|^{\upalpha})^{-1}\,:\;x\in\mathbb{R}^{d}\\}<+\infty\;,\qquad\int_{\mathbb{R}^{d}}\exp[\eta\|x\|^{\upbeta}]\mathrm{d}\mu(x)<+\infty\;,}$
(23)

In addition, assume that for any $\theta\in\mathbb{R}^{p}$ with
$\|\theta\|=1$, we have
$\mu(\\{x\in\mathbb{R}^{d}\,:\;\langle\theta,G(x)\rangle<0\\})>0$. Then there
exists a unique macrocanonical distribution with constraint $G$ and reference
measure $\mu$ denoted $\pi^{\star}$. In addition, there exists
$\theta^{\star}\in\mathbb{R}^{p}$ such that for any $x\in\mathbb{R}^{d}$

$\textstyle{(\mathrm{d}\pi^{\star}/\mathrm{d}\mu)(x)=\exp[-\langle\theta^{\star},G(x)\rangle-L(\theta^{\star})]\;,\qquad
L(\theta^{\star})=\log\int_{\mathbb{R}^{d}}\exp[-\langle\theta^{\star},G(x)\rangle]\mathrm{d}\mu(x)\;.}$
(24)

We refer to [21] for a relaxation of these conditions and a detailed study of
the macrocanonical model. Section 3.1.1 ensures that macrocanonical
distributions can be written as Gibbs measures. We emphasize that these
distributions satisfy a constraint in expectation. An alternative way to
define maximum entropy models under constraints is to ensure almost sure
equality. This tighter constraint yields new maximum entropy models called
microcanonical distributions.

##### Microcanonical distributions

A microcanonical model $\pi^{\star}\in\mathscr{P}(\mathbb{R}^{d})$ with
reference measure $\mu$ and constraint $G$ can be defined as

$\textstyle{\pi^{\star}\in\operatorname*{arg\,min}\\{\operatorname{KL}(\pi|\mu)\,:\;\pi\in\mathscr{P}(\mathbb{R}^{d}),\
\pi[\|G\|]=0\\}}\;.$ (25)

We emphasize that imposing (25) is equivalent to impose that $\pi^{\star}$ has
minimum Kullback-Leibler divergence w.r.t $\mu$ among all distributions which
satisfy $G=0$ almost surely. Note that if there exists a microcanonical model
$\pi^{\star}$ with $\operatorname{KL}(\pi^{\star}|\mu)<+\infty$ then
$\mu(\mathsf{A})>0$ where $\mathsf{A}=\\{x\in\mathbb{R}^{d}\,:\;G(x)=0\\}$ and
in this case $\pi^{\star}=\mu(\cdot\cap\mathsf{A})/\mu(\mathsf{A})$. In what
follows, we will say that $\pi^{\star}$ is the uniform microcanonical
distribution with constraint $G$ if $G^{-1}(0)$ is compact with
$\mathcal{H}^{d-\min(d,p)}(G^{-1}(0))<+\infty$ and $\pi^{\star}$ is the
uniform distribution on $G^{-1}(0)$, i.e. for any
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$, we have

$\textstyle{\pi^{\star}(\mathsf{A})=\int_{G^{-1}(0)\cap\mathsf{A}}\mathrm{d}\mathcal{H}^{d-\min(d,p)}(x)/\mathcal{H}^{d-\min(d,p)}(G^{-1}(0))\;.}$
(26)

Note that contrary to macrocanonical distributions which admit a
representation as a Gibbs measure, see Section 3.1.1, the microcanonical
models are concentrated on the set $G^{-1}(0)$. In the next section, we draw
links between these two maximum entropy distributions.

####  3.1.2 From macrocanonical to microcanonical

In order to draw links between macrocanonical and microcanonical models we
consider a specific sequence of macrocanonical models associated with the
constraint $G_{\varepsilon}=\|F\|^{2}-\varepsilon$ for $\varepsilon>0$. Let
$\mu\in\mathscr{P}(\mathbb{R}^{d})$ be a reference probability measure. Under
the conditions of Section 3.1.1, we define a family of measures
$\\{\rho_{\varepsilon}\\}_{\varepsilon>0}$ such that for any $\varepsilon>0$,
$\rho_{\varepsilon}$ is the macrocanonical distribution associated with
$G_{\varepsilon}$ and $\mu$. More precisely, for any $\varepsilon>0$, there
exists $\theta_{\varepsilon}\in\mathbb{R}$ such that for any
$x\in\mathbb{R}^{d}$

$\textstyle{(\mathrm{d}\rho_{\varepsilon}/\mathrm{d}\mu)(x)=\exp[-\theta_{\varepsilon}\|F(x)\|^{2}-L_{\varepsilon}]\;,\quad
L_{\varepsilon}=\log\int_{\mathbb{R}^{d}}\exp[-\theta_{\varepsilon}\|F(x)\|^{2}]\mathrm{d}\mu(x)\;.}$
(27)

Our goal is to study the behavior of the family
$\\{\rho_{\varepsilon}\\}_{\varepsilon>0}$ when $\varepsilon\to 0$. In
particular, we show that there exists a reference measure $\mu$ such that the
limit $\pi_{0}=\lim_{\varepsilon\to 0}\rho_{\varepsilon}$ exists where
$\pi_{0}$ is the uniform microcanonical distribution (26). We start with the
following proposition which ensures that $\lim_{\varepsilon\to
0}\theta_{\varepsilon}=+\infty$ with linear rate.

######  Proposition .

Let $\mu\in\mathscr{P}(\mathbb{R}^{d})$ and $F:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$. Assume that the conditions of Section 3.1.1
with $G_{\varepsilon}=\|F\|^{2}-\varepsilon$ for any $\varepsilon>0$ are
satisfied and let $\rho_{\varepsilon}$ be the macrocanonical distribution with
constraint $G_{\varepsilon}$ and reference measure $\mu$. Assume that there
exists $\Psi:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ such that $\mu$ admits a
density w.r.t. the Lebesgue measure given by $\Psi$. In addition, assume that
H 1 and H 2 hold. Then, we have that $\theta_{\varepsilon}\sim_{\varepsilon\to
0}p/(2\varepsilon)$.

###### Proof.

We provide a sketch of the proof. The whole proof is postponed to Section 4.2.
In the case where $d\geq p$ we use tools from geometric measure theory to
derive an equivalent of
$\int_{\mathbb{R}^{d}}\Psi(x)\|F(x)\|^{2}\exp[-\|F(x)\|^{2}/\varepsilon]\mathrm{d}x$
when $\varepsilon\to 0$ akin to Section 4.1.1. In the case $d\leq p$, using
Morse lemma and classical Laplace analysis we show a similar result. We
conclude upon combining these results and that
$\rho_{\varepsilon}[\|F\|^{2}]=\varepsilon$. ∎

Note that the equivalent in Section 3.1.2 is independent from the reference
measure $\mu$. In Section 4.2, we prove an extension of Section 3.1.2, see
Section 4.2.3. In particular we show that
$G_{\varepsilon}=\|F\|^{2}-\varepsilon$ can be replaced by
$G_{\varepsilon}=\|F\|^{k}-\varepsilon$ with $k\in\mathbb{N}^{\star}$ under
similar conditions. In particular, we get that for any
$k\in\mathbb{N}^{\star}$, $\theta_{\varepsilon}\sim_{\varepsilon\to
0}\mathcal{C}_{k}/\varepsilon$ with $\mathcal{C}_{k}$ explicit if $k=2$.
Finally note that under the same conditions as Section 3.1.2, we have that
$\pi_{\varepsilon}^{\Psi}[\|F\|^{2}]\sim_{\varepsilon\to 0}p/(2\varepsilon)$,
where we recall that for any $\varepsilon>0$ and
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$

$\textstyle{\pi_{\varepsilon}^{\Psi}(\mathsf{A})=\int_{\mathsf{A}}\Psi(x)\exp[-\|F(x)\|^{2}/\varepsilon]\mathrm{d}x/\int_{\mathbb{R}^{d}}\Psi(x)\exp[-\|F(x)\|^{2}/\varepsilon]\mathrm{d}x\;.}$
(28)

This comes from the facts that
$\rho_{\varepsilon}=\pi_{\theta_{\varepsilon}}^{\Psi}$ and
$\rho_{\varepsilon}[\|F\|^{2}]=\varepsilon$.

The following proposition establishes quantitative bounds w.r.t. the
Wasserstein of order $1$ between $\\{\rho_{\varepsilon}\\}_{\varepsilon>0}$
and a limiting measure. We obtain this result upon combining Section 3.1.2 and
Section 2.

######  Proposition .

Assume the same conditions as Section 3.1.2. Then there exist
$\bar{\varepsilon}>0$ and $A_{3}\geq 0$ such that for any
$\varepsilon\in\left(0,\bar{\varepsilon}\right]$ we have

$\mathbf{W}_{1}(\rho_{\varepsilon},\pi_{0}^{\Psi})\leq
A_{3}\varepsilon^{1/2}\;,$ (29)

where $\pi_{0}^{\Psi}$ is given in (17).

Section 3.1.2 establishes quantitative bounds between the family of
macrocanonical models $\\{\rho_{\varepsilon}\\}_{\varepsilon>0}$ and a
limiting distribution $\pi_{0}^{\Psi}$. Note that this limiting distribution
is _not_ the uniform microcanonical distribution $\pi^{\star}$ defined in (26)
in general, even though it is supported on the set $F^{-1}(0)$. However, it is
possible to sample from this $\pi^{\star}$ by choosing $\mu$ such that for any
$x\in F^{-1}(0)$,
$\Psi(x)=(\mathrm{d}\mu/\mathrm{d}\lambda)(x)=\mathrm{J}F(x)$.

####  3.1.3 Some simple experiments

##### Methodology

In this experimental section we consider two simple examples of functions
$F:\mathbb{R}^{d}\to\mathbb{R}^{p}$ with $d=1$ or $2$ and $p=1$. We consider
the associated models $\\{\pi_{\varepsilon}\\}_{\varepsilon>0}$ and
$\\{\pi_{\varepsilon}^{\Psi}\\}_{\varepsilon>0}$ and their limit as
$\varepsilon$ goes to $0$, showing in particular that for $\Psi=\mathrm{J}F$,
it converges to the uniform microcanonical model that is the uniform
distribution on $F^{-1}(0)$. We also check experimentally the scaling relation
of Proposition 3.1.2.

##### Zeros of a polynomial

In this first example, let $P:\mathbb{R}\to\mathbb{R}$ be a polynomial. We are
interested in the zeros of $P$, and, more precisely in sampling in a uniform
way on the set of the zeros of $P$. Using the notations of the previous
sections, we set $F=P$. For $\varepsilon>0$, let us define the two
distributions, with respective densities with respect to the Lebesgue measure
on $\mathbb{R}$ given for any $x\in\mathbb{R}$ by

$\displaystyle\textstyle{(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)=\exp[-F(x)^{2}/\varepsilon]/\int_{\mathbb{R}}\exp[-F(\tilde{x})^{2}/\varepsilon]\mathrm{d}\tilde{x}}\;,$
(30)
$\displaystyle\textstyle{(\mathrm{d}\pi_{\varepsilon}^{\Psi}/\mathrm{d}\lambda)(x)=\mathrm{J}F(x)\exp[-F(x)^{2}/\varepsilon]/\int_{\mathbb{R}}\mathrm{J}F(\tilde{x})\exp[-F(\tilde{x})^{2}/\varepsilon]\mathrm{d}\tilde{x}}\;,$
(31)

where for any $x\in\mathbb{R}$, $\Psi(x)=\mathrm{J}F(x)=|P^{\prime}(x)|$.

In Figure 1, we display the polynomial $P$ where for any $x\in\mathbb{R}$,
$P(x)=x(x-0.5)(x-1.7)(x-2.5)$. We also check numerically the scaling relation
of Proposition 3.1.2. Then, in Figure 2, we illustrate the different behaviors
of $\pi_{\varepsilon}$ and $\pi_{\varepsilon}^{\Psi}$ when $\varepsilon$ is
small ($\varepsilon=10^{-3}$ in our experiments). We also present the
histograms obtained using of $N=10^{4}$ independent samples of
$\pi_{\varepsilon}$ and of $\pi_{\varepsilon}^{\Psi}$. The stars indicate the
target limit distributions (that is the uniform distribution on the zeros of
$P$ for the limit of $\pi_{\varepsilon}^{\Psi}$).

Figure 1: Left: graph of the polynomial $x\mapsto P(x)$, that has $4$ zeros.
Right: verifying the scaling relation of Section 3.1.2 by plotting
$\varepsilon\mapsto 2\pi_{\varepsilon}[P^{2}]$, which is equivalent to
$\varepsilon$ as $\varepsilon$ goes to $0$.

Figure 2: Distributions $\pi_{\varepsilon}$ and $\pi_{\varepsilon}^{\Psi}$
(first line), and histogram of their samples from them (second line). This
experiment shows that the limit distribution of $\pi_{\varepsilon}^{\Psi}$ as
$\varepsilon$ goes to $0$ is the uniform microcanonical model given by the
uniform distribution on the zeros of $P$.

##### Two-dimensional ellipse

In this second example, we consider the function
$F:\mathbb{R}^{2}\to\mathbb{R}$ given for any
$x=(x_{1},x_{2})\in\mathbb{R}^{2}$ by $F(x)=a_{1}x_{1}^{2}+a_{2}x_{2}^{2}-1$
with $a_{1},a_{2}>0$. For $\varepsilon>0$, we define $\pi_{\varepsilon}$ and
$\pi_{\varepsilon}^{\Psi}$ whose densities w.r.t the Lebesgue measure are
given for any $x\in\mathbb{R}^{2}$ by

$\displaystyle\textstyle{(\mathrm{d}\pi_{\varepsilon}/\mathrm{d}\lambda)(x)=\exp[-\|F(x)\|^{2}/\varepsilon]/\int_{\mathbb{R}^{2}}\exp[-\|F(\tilde{x})\|^{2}/\varepsilon]\mathrm{d}\tilde{x}}\;,$
(32)
$\displaystyle\textstyle{(\mathrm{d}\pi_{\varepsilon}^{\Psi}/\mathrm{d}\lambda)(x)=\mathrm{J}F(x)\exp[-\|F(x)\|^{2}/\varepsilon]/\int_{\mathbb{R}^{2}}\mathrm{J}F(\tilde{x})\exp[-\|F(\tilde{x})\|^{2}/\varepsilon]\mathrm{d}\tilde{x}}\;,$
(33)

To sample from $\pi_{\varepsilon}$ and $\pi_{\varepsilon}^{\Psi}$, we use two
Markov chains given by the Unadjusted Langevin Algorithm [61, 24, 20], given
respectively by $X_{0},Y_{0}\in\mathbb{R}^{2}$ and the recursions

$\displaystyle
X_{n+1}=X_{n}-\gamma(\nabla\|F\|^{2}(X_{n})/\varepsilon)+\sqrt{2\gamma}Z_{n+1}\;,$
(34) $\displaystyle
Y_{n+1}=Y_{n}-\gamma(\nabla\|F\|^{2}(Y_{n})/\varepsilon-\nabla\log\mathrm{J}F(Y_{n}))+\sqrt{2\gamma}Z_{n+1}\;,$
(35)

where $\gamma>0$ is a stepsize and $\\{Z_{n}\\}_{n\in\mathbb{N}}$ is a family
of independent Gaussian random variables with zero mean and identity
covariance matrix. For any $x\in\mathbb{R}^{2}$ we have

$\displaystyle\textstyle{\mathrm{J}F(x)=2(a_{1}^{2}x_{1}^{2}+a_{2}^{2}x_{2}^{2})^{1/2}\;,}\quad\textstyle{\nabla\|F\|^{2}(x)=2F(x)\nabla
F(x)=4(a_{1}x_{1}^{2}+a_{2}x_{2}^{2}-1)\left(\begin{array}[]{l}a_{1}x_{1}\\\
a_{2}x_{2}\end{array}\right)\;,}$ (38)
$\displaystyle\textstyle{\nabla\log\mathrm{J}F(x)=(a_{1}^{2}x_{1}^{2}+a_{2}^{2}x_{2}^{2})^{-1}\left(\begin{array}[]{l}a_{1}^{2}x_{1}\\\
a_{2}^{2}x_{2}\end{array}\right)}\;.$ (41)

The set $F^{-1}(0)$ is an ellipse, of cartesian equation
$a_{1}x_{1}^{2}+a_{2}x_{2}^{2}=1$ and has a polar parametrization given for
any $\theta\in[0,2\uppi)$ by

$\textstyle{r(\theta)=(a_{2}+(a_{1}-a_{2})\cos^{2}\theta)^{1/2}\;.}$

This is not an arc-length parametrization, and the infinitesimal length of the
curve element between $\theta$ and $\theta+d\theta$ is given by
$\ell(\theta)d\theta$ where for any $\theta\in\left(0,2\uppi\right]$ we have

$\ell(\theta)=(r(\theta)^{2}+r^{\prime}(\theta)^{2})^{1/2}=(a_{2}^{2}+(a_{1}^{2}-a_{2}^{2})\cos^{2}\theta)^{1/2}(a_{2}+(a_{1}-a_{2})\cos^{2}\theta)^{-3/2}\;.$

In Figure 3, we set $a_{1}=1$, $a_{2}=4$, $\varepsilon=10^{-3}$,
$\gamma=10^{-5}$, $N=9\times 10^{7}$ iterations in the Markov chains, and
histograms with bins of size $0.05\uppi$. In Figure 3, we compare two
different histograms: the ones of the angle values ($\theta$) for the points
of the chain $X_{n}$ (diamonds), of the chain $Y_{n}$ (stars), and the two
distributions with density
$\theta\mapsto\ell(\theta)/\int_{-\pi}^{\pi}\ell(\theta)\mathrm{d}\theta$
(black curve) and
$\theta\mapsto\ell(\theta)\mathrm{J}F(\theta)^{-1}/\int_{-\pi}^{\pi}\ell(\theta)\mathrm{J}F(\theta)^{-1}\mathrm{d}\theta$
(red curve). Note that
$\theta\mapsto\ell(\theta)/\int_{-\pi}^{\pi}\ell(\theta)\mathrm{d}\theta$
corresponds to the Hausdorff measure on the ellipse pushed by the polar
parametrisation $\theta\mapsto r(\theta)$, i.e. the uniform microcanonical
model pushed by $\theta\mapsto r(\theta)$. We observe that the Markov chain
corrected with the generalized Jacobian indeed achieves the uniform
microcanonical model on the set $F^{-1}(0)$. Finally, we also check
experimentally the scaling relation of Proposition 3.1.2.

|  |
---|---|---
Figure 3: Left: histogram of angles of the samples $(X_{n})_{n\in\mathbb{N}}$
and $(Y_{n})_{n\in\mathbb{N}}$, showing that $\pi_{\varepsilon}^{\Psi}$ is
close to the uniform distribution on the ellipse for small values of
$\varepsilon>0$. Middle: the trajectory of the first $10^{6}$ samples of
$Y_{n}$ (with a burnin of $10^{3}$ iterates). Right: checking the scaling
relation of Proposition 3.1.2.

###  3.2 Non-convex minimization

####  3.2.1 Non-convex setting and related work

In this section, we consider the following minimization problem:

$\text{find
}x^{\star}\in\operatorname*{arg\,min}\\{U(x)\,:\;x\in\mathbb{R}^{d}\\}\;,$
(42)

with $U\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$ and
$\operatorname*{arg\,min}\\{U(x)\,:\;x\in\mathbb{R}^{d}\\}\neq\emptyset$.
Here, we do not assume that $U$ is convex. Hence, classical first-order
optimization schemes such as gradient descent might get trapped in saddle
points. Adding isotropic Gaussian noise to this dynamics circumvents this
issue, see [9, 58] for instance. The algorithm is then given by the following
recursion: $X_{0}\in\mathbb{R}^{d}$ and for any $k\in\mathbb{N}$

$X_{k+1}=X_{k}-\gamma_{k}\nabla U(X_{k})+Z_{k+1}\;,$ (43)

where $(Z_{k})_{k\in\mathbb{N}}$ is a sequence of independent Gaussian random
variables with zero mean and covariance matrix
$\sigma_{k}^{2}\operatorname{Id}$, with
$(\gamma_{k})_{k\in\mathbb{N}}\in(\mathbb{R}_{+})^{\mathbb{N}}$ a sequence of
stepsizes and $(\sigma_{k})_{k\in\mathbb{N}}\in(\mathbb{R}_{+})^{\mathbb{N}}$.
If $\sigma_{k}^{2}=\gamma_{k}^{2}$ for any $k\in\mathbb{N}$,
$\sum_{k\in\mathbb{N}}\gamma_{k}=+\infty$ and
$\sum_{k\in\mathbb{N}}\gamma_{k}^{2}<+\infty$ then the algorithm is in the
_weakly disturbed_ regime and under additional assumptions on $U$ one can show
that $(X_{k})_{k\in\mathbb{N}}$ converges almost surely to a local minimizer
of $U$, see [58] for instance. However,
$(U(X_{k})_{k\in\mathbb{N}})_{k\in\mathbb{N}}$ does not necessarily converges
to the global minimum of $U$. The intuition behind this behavior is that the
variance of the noise decreases too quickly for the sequence to explore
efficiently the landscape of $U$.

In order to perform global optimization, one can consider _simulated
annealing_ algorithms where $\sigma_{k}^{2}=\gamma_{k}T_{k}$ for any
$k\in\mathbb{N}$, with
$(T_{k})_{k\in\mathbb{N}}\in(\mathbb{R}_{+})^{\mathbb{N}}$ a sequence of
temperatures which slowly decrease. These algorithms were introduced in the
context of discrete optimization in [47] and have been thoroughly investigated
in [37] (discrete state-space), [33, 34] (discrete-time algorithm), [36, 15,
43] (diffusion), [58] (CLT type results), [62] (constrained optimization),
[72] (non Gaussian noise) and [2] (control of the sequence in total variation)
for instance. In [40, 39] a sufficient and necessary condition is given on the
rate of decrease $(T_{k})_{k\in\mathbb{N}}$ so that
$U(X_{k})_{k\in\mathbb{N}}$ converges towards the minimum of $U$, more
precisely the condition reads
$\sum_{k\in\mathbb{N}}\exp[-b^{\star}/T_{k}]=+\infty$, where $b^{\star}>0$ is
a parameter which depends only on $U$ called the _kinetic barrier_ (or depth
in [40, 39]). One of the main limiting factor of simulated annealing is the
slow rate of convergence of $(T_{k})_{k\in\mathbb{N}}$ towards $0$, which is
often set as $T_{k}=C/\log(k+1)$ for any $k\in\mathbb{N}$ and for some
constant $C\geq 0$. Note that recently the convergence of modifications of the
simulated annealing algorithms under faster cooling rates have been
investigated in discrete spaces by either changing the Markov chain
transitions [16] or the energy landscape [17], see also [13]. Finally, we
emphasize that most of the results regarding the convergence of (43) can be
extended to the case where $\nabla U$ is replaced by an unbiased estimator
under additional conditions.

With the advent of neural networks, numerous schemes exploiting the annealing
structure have been proposed to minimize the non-convex losses which arise in
deep learning applications, see [73] for instance. Drawing connections with
unadjusted Langevin algorithms, and in particular Stochastic Gradient Langevin
Dynamics (SGLD) [68], [60, 75] replace $\nabla U$ by an unbiased estimator and
let $\gamma_{k}=\gamma>0$, $\sigma_{k}^{2}=2\gamma\varepsilon$ for any
$k\in\mathbb{N}$ with $\varepsilon>0$ in (43). Under curvature and regularity
assumptions on the potential $U$, the authors derive quantitative bounds on
$(\mathbb{E}[U(X_{k})-\min_{\mathbb{R}^{d}}U])_{k\in\mathbb{N}}$. Since then
several accelerations have been proposed in the literature [14, 32, 27, 55,
74, 71].

In the next section, we improve the results of [60] by providing upper bounds
w.r.t the first order Wasserstein distance between the distribution of
$(X_{k})_{k\in\mathbb{N}}$ given by SGLD and an explicit limiting
distribution. Our results complete the ones of [60] which deals with the
behavior of $(\mathbb{E}[U(X_{k})-\min_{\mathbb{R}^{d}}U])_{k\in\mathbb{N}}$.
To the best of our knowledge, this is the first result establishing
quantitative bounds on the distance between the iterates of SGLD and a
limiting measure concentrated on the minimizers of the target potential $U$.

####  3.2.2 Quantitative convergence for SGLD

In this section we start by recalling the setting considered in [60]. We
assume that there exist a topological space
$(\mathsf{Z},\mathcal{B}(\mathsf{Z}))$, a probability measure
$\mu\in\mathscr{P}(\mathsf{Z},\mathcal{B}(\mathsf{Z}))$ and $u:\
\mathbb{R}^{d}\times\mathsf{Z}\to\mathbb{R}_{+}$ such that for any
$z\in\mathsf{Z}$,
$u(\cdot,z)\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R}_{+})$, and define for
any $x\in\mathbb{R}^{d}$

$\textstyle{U(x)=\int_{\mathsf{Z}}u(x,z)\mathrm{d}\mu(z)\;.}$ (44)

We also denote $U^{\star}$ the global minimum of $U$. We do not have access to
$\mu$ and $U$ directly but instead consider an empirical version of the
target. Let $n\in\mathbb{N}$. We define $U_{n}:\
\mathbb{R}^{d}\times\mathsf{Z}^{n}\to\mathbb{R}_{+}$ given for any
$z^{1:n}=\\{z_{i}\\}_{i=1}^{n}\in\mathsf{Z}^{n}$ and $x\in\mathbb{R}^{d}$ by

$\textstyle{U_{n}(x,z^{1:n})=(1/n)\sum_{i=1}^{n}u(x,z_{i})\;.}$ (45)

Let $(\mathsf{Y},\mathcal{Y})$ be a measurable space, $\mathrm{R}:\
\mathsf{Z}^{n}\times\mathcal{Y}\to\left[0,1\right]$ a Markov kernel and $g:\
\mathbb{R}^{d}\times\mathsf{Y}\to\mathbb{R}^{d}$ such that for any
$x\in\mathbb{R}^{d}$ and $z^{1:n}\in\mathsf{Z}^{n}$ we have

$\textstyle{\nabla_{x}U_{n}(x,z^{1:n})=\int_{\mathsf{Y}}g(x,y)\mathrm{R}(z^{1:n},\mathrm{d}y)\;.}$
(46)

Let $Z$ be a random variable on $\mathsf{Z}^{n}$ with distribution
$\mu^{\otimes n}$. Let $\\{Y_{k}\\}_{k\in\mathbb{N}}$ be a family of
independent random variables on $\mathsf{Y}$ such that conditionally to $Z$ we
have for any $k\in\mathbb{N}$ that $Y_{k}$ has distribution
$\updelta_{Z}\mathrm{R}$. Finally, let $\varepsilon>0$. We consider the
Stochastic Gradient Langevin Dynamics (SGLD) sequence
$(X_{k})_{k\in\mathbb{N}}$ given by the following recursion:
$X_{0}\in\mathbb{R}^{d}$ and for any $k\in\mathbb{N}$

$X_{k+1}=X_{k}-\gamma g(X_{k},Y_{k})+\sqrt{2\gamma\varepsilon}G_{k+1}\;,$ (47)

where $(G_{k})_{k\in\mathbb{N}}$ is a sequence of independent Gaussian random
variables with zero mean and identity covariance matrix and $\gamma>0$ is a
stepsize. We also assume that $\\{Y_{k}\\}_{k\in\mathbb{N}}$ and
$\\{G_{k}\\}_{k\in\mathbb{N}}$ are independent. For any $k\in\mathbb{N}$, we
denote $\mathrm{Q}_{k}:\
\mathsf{Z}^{n}\times\mathcal{B}(\mathbb{R}^{d})\to\left[0,1\right]$, the
Markov kernel such that for any random variable $Z$ on $\mathsf{Z}^{n}$ with
distribution $\mu^{\otimes n}$, we have that $X_{k}$ has distribution
$\updelta_{Z}\mathrm{Q}_{k}$ conditionally to $Z$. We consider the following
assumption, which is similar to the one of [60].

######  H 3 ($n$).

For any $z\in\mathsf{Z}$,
$u(\cdot,z)\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R})$ and the
following hold:

  1. (a)

There exist $A,B\geq 0$ such that for any $z\in\mathsf{Z}$,
$\left|u(0,z)\right|\leq A$ and $\|\nabla_{x}u(0,z)\|\leq B$.

  2. (b)

There exists $\mathtt{M}\geq 0$ such that for any
$x_{1},x_{2}\in\mathbb{R}^{d}$, $z\in\mathsf{Z}$,
$\|\nabla_{x}^{k}u(x_{1},z)-\nabla_{x}^{k}u(x_{2},z)\|\leq\mathtt{M}\|x_{1}-x_{2}\|$
for any $k\in\\{1,2,3\\}$.

  3. (c)

There exist $\mathtt{m}>0$, $\mathtt{c}\geq 0$ such that for any
$x\in\mathbb{R}^{d}$, $z\in\mathsf{Z}$,
$\langle\nabla_{x}u(x,z),x\rangle\geq\mathtt{m}\|x\|^{2}-\mathtt{c}$.

  4. (d)

There exists $C\geq 0$ such that for any $x\in\mathbb{R}^{d}$ and
$z^{1:n}\in\mathsf{Z}^{n}$ we have

$\textstyle{\int_{\mathsf{Z}^{n}}\int_{\mathsf{Y}}\|\nabla_{x}U_{n}(x,z^{1:n})-g(x,y)\|^{2}\mathrm{R}(z^{1:n},\mathrm{d}y)\mathrm{d}\mu^{\otimes
n}(z^{1:n})\leq C(1+\|x\|^{2})\;.}$ (48)

Similar to [60] we define the uniform spectral gap

$\textstyle{\lambda^{\star}=\inf\\{\updelta_{z^{1:n}}\mathrm{R}[\|\nabla
h\|^{2}]/\updelta_{z^{1:n}}\mathrm{R}[|h|^{2}]\,:\;z^{1:n}\in\mathsf{Z}^{n},\
h\in\mathrm{C}^{1}(\mathbb{R}^{d})\cap\mathrm{L}^{2}(\updelta_{z^{1:n}}\mathrm{R}),\
h\neq 0,\ \updelta_{z^{1:n}}\mathrm{R}[h]=0\\}\;,}$ (49)

and note that under H 3($n$), $\lambda^{\star}>0$. Finally, we also consider
the following assumption which ensures that the limiting measures we consider
are well-defined.

######  H 4 ($n$).

$u\in\mathrm{C}(\mathbb{R}^{d}\times\mathsf{Z},\mathbb{R})$ and the following
hold:

  1. (a)

$\mathsf{Z}$ is compact.

  2. (b)

There exist $N\in\mathbb{N}$,
$\\{x_{0}^{k}\\}_{k=1}^{N}\in\mathrm{C}(\mathsf{Z}^{n},\mathbb{R}^{d})$ such
that for any $z^{1:n}\in\mathsf{Z}^{n}$,
$\mathsf{C}(z^{1:n})\subset\\{x_{0}^{k}(z)\\}_{k=1}^{N}$ and
$\mathsf{C}(z^{1:n})\neq\emptyset$, where
$\mathsf{C}(z^{1:n})=\operatorname*{arg\,min}\\{U_{n}(x,z^{1:n})\,:\;x\in\mathbb{R}^{d}\\}$.

  3. (c)

For any $\upbeta>0$,
$\int_{\mathsf{Z}^{n}}\sigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$, where

$\textstyle{\sigma_{\upbeta}^{\star}(z^{1:n})=\sum_{x\in\mathsf{C}(z^{1:n})}\det(\nabla_{x}^{2}U(x,z^{1:n}))^{-\upbeta}\;.}$
(50)

We also introduce the _thermodynamic barrier_ $c^{\star}:\
\mathsf{Z}^{n}\to\mathbb{R}$ such that for any $z^{1:n}\in\mathsf{Z}^{n}$,
$c^{\star}(z^{1:n})=+\infty$ if $U_{n}(\cdot,z^{1:n})$ does not admit a local
minimizer which is not a global minimizer and

$c^{\star}(z^{1:n})=\inf\\{U_{n}(x,z^{1:n})\,:\;\text{$x$ is a local minimizer
of $U_{n}(\cdot,z)$ but not a global minimizer}\\}-U_{n}^{\star}(z^{1:n})\;,$
(51)

with
$U_{n}^{\star}(z^{1:n})=\inf\\{U_{n}(x,z^{1:n})\,:\;x\in\mathbb{R}^{d}\\}$. We
refer to Section 3.2.3 for a discussion on the _thermodynamic barrier_ and its
importance in non-convex optimization. Under H 3($n$) and H 4($n$), we derive
quantitative bounds for the sequence $(X_{k})_{k\in\mathbb{N}}$.

######  Proposition .

Let $n\in\mathbb{N}$. Assume H 3($n$) and H 4($n$). Then there exist $C\geq
0$, $\bar{\varepsilon},\bar{\gamma},\upbeta>0$ and $k_{0}\in\mathbb{N}$ such
that for any $\varepsilon\in(0,\bar{\varepsilon}]$,
$\gamma\in(0,\bar{\gamma}]$ and $k\in\mathbb{N}$ with $k\geq k_{0}$ we have

$\displaystyle\mathbf{W}_{1}(\mu^{\otimes n}\mathrm{Q}_{k},\mu^{\otimes
n}\mathrm{S}_{0})$ $\displaystyle\leq
C(1/\varepsilon+d)^{2}(\gamma^{1/4}\log(1/\gamma)+\gamma)/(\lambda^{\star}\varepsilon)$
(52)
$\displaystyle\qquad\qquad\textstyle{+C(1+D_{n})(\varepsilon^{1/2}+\varepsilon^{-d/2}\int_{\mathsf{Z}^{1:n}}\exp[-c^{\star}(z^{1:n})/\varepsilon]\mathrm{d}\mu^{\otimes
n}(z^{1:n}))\;,}$ (53)

with
$D_{n}=\int_{\mathsf{Z}^{n}}\sigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$ and $\mathrm{S}_{0}:\
\mathsf{Z}^{n}\times\mathcal{B}(\mathbb{R}^{d})\to\left[0,1\right]$ the Markov
kernel given for any $z^{1:n}\in\mathsf{Z}^{n}$ and
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ by

$\textstyle{\mathrm{S}_{0}(z^{1:n},\mathsf{A})=\left.\int_{\mathsf{C}(z^{1:n})\cap\mathsf{A}}\det(\nabla_{x}^{2}U_{n}(x,z^{1:n}))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x)\middle/\int_{\mathsf{C}(z^{1:n})}\det(\nabla_{x}^{2}U_{n}(x,z^{1:n}))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x)\right.\;,}$
(54)

with
$\mathsf{C}(z^{1:n})=\operatorname*{arg\,min}\\{U_{n}(x,z^{1:n})\,:\;x\in\mathbb{R}^{d}\\}$.

###### Proof.

We provide a sketch of the proof. The whole proof is postponed to Section
4.3.1. First, we assess the convergence of $(\mu^{\otimes
n}\mathrm{Q}_{k})_{k\in\mathbb{N}}$ by splitting the error in two parts. A
first part is bounded using the geometric ergodicity of SGLD as in [60] and
controls the distance between $\mu^{\otimes n}\mathrm{Q}_{k}$ and
$\mu^{\otimes n}\mathrm{S}_{\varepsilon}$, where for any $z\in z^{1:n}$,
$\updelta_{z}^{1:n}\mathrm{S}_{\varepsilon}$ is the Gibbs distribution given
for any $\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$

$\textstyle{\updelta_{z^{1:n}}\mathrm{S}_{\varepsilon}(\mathsf{A})=\left.\int_{\mathsf{A}}\exp[-U_{n}(x,z^{1:n})/\varepsilon]\mathrm{d}x\middle/\int_{\mathbb{R}^{d}}\exp[-U_{n}(x,z^{1:n})/\varepsilon]\mathrm{d}x\right.\;.}$
(55)

Then, using a parametric version of Theorem 1, see Section B.2, we bound the
distance between $\mu^{\otimes n}\mathrm{S}_{\varepsilon}$ and $\mu^{\otimes
n}\mathrm{S}_{0}$. ∎

A few remarks are in order:

  1. (a)

The condition that
$\int_{\mathsf{Z}^{n}}\upsigma_{1}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$ is necessary to ensure that $\mu^{\otimes
n}\mathrm{S}_{0}$ is well-defined. In Section 3.2.2 we assume the slightly
stronger condition that
$\int_{\mathsf{Z}^{n}}\upsigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$ for any $\upbeta>0$. In fact the condition could be
relaxed to
$\int_{\mathsf{Z}^{n}}\upsigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$ for any $\upbeta\in\left(0,\upgamma\right]$ with
$\upgamma>0$ an explicit constant. Obtaining such a result requires to derive
explicit bounds in the quantitative Morse lemma [51, Theorem 3.2], see also
Section B.2. We leave this analysis to future works. Note also that this
condition can be satisfied upon regularizing the function $u$ and assuming
that the number of global minimizers is bounded. Indeed, if we replace
$u(x,z)$ by $u(x,z)+(\alpha/2)\|x\|^{2}$ (with $\alpha>0$ some regularization
parameter) then we get that for any $\upbeta>0$,
$\int_{\mathsf{Z}^{n}}\upsigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})\leq N_{0}\alpha^{-\upbeta d}$ where $N_{0}$ is an upper-bound on
the number of global minimizers.

  2. (b)

The upper-bound in (52) also depends on the _thermodynamic barrier_. This
constant quantifies how close the local minima which are not global minima are
to the global minima and is crucial to establish quantitative parametric
Laplace-type results. We illustrate this situation in Section 3.2.3 with a
simple example.

  3. (c)

Section 3.2.2 ensures that for a given precision level $r>0$, there exist
$\varepsilon,\gamma>0$ small enough and $k\in\mathbb{N}$ large enough such
that $\mathbf{W}_{1}(\mu^{\otimes n}\mathrm{Q}_{k},\mu^{\otimes
n}\mathrm{S}_{0})\leq r$. However, note that $\mu^{\otimes n}\mathrm{S}_{0}$
is not concentrated on the minimizers of $U$. This highlights the fact that
the minimization of the empirical risk does not guarantee that the population
risk is small.

In order to verify that the population risk is small in expectation w.r.t. the
target measure $\mu^{\otimes n}\mathrm{S}_{0}$ we use stability tools to
establish the following proposition.

######  Proposition .

Assume that H 3($n$) and H 4($n$) hold uniformly w.r.t $n\in\mathbb{N}$. In
addition, assume that there exist $C_{0},\alpha,\bar{\varepsilon}>0$ such that
for any $\varepsilon\in(0,\bar{\varepsilon}]$

$\textstyle{\varepsilon^{-d/2}\int_{\mathsf{Z}^{1:n}}\exp[-c^{\star}(z^{1:n})/\varepsilon]\mathrm{d}\mu^{\otimes
n}(z^{1:n})}\leq C_{0}\varepsilon^{\alpha}\;.$ (56)

Then for any $\eta\in\left(0,1\right)$, there exist $C_{\eta}\geq 0$ and
$n_{0}\in\mathbb{N}$ such that for any $n\geq n_{0}$ we have

$\mu^{\otimes n}\mathrm{S}_{0}[U]-U^{\star}\leq C_{\eta}n^{-s\eta/(1+\eta
s)}\;,$ (57)

with $s=\min(1/4,\alpha/2)$ and where we recall that $U^{\star}$ is the global
minimum of $U$.

###### Proof.

The proof is postponed to Section 4.3.2. ∎

Note that Section 3.2.2 implies that the sequence $(\mu^{\otimes
n}\mathrm{S}_{0}[U])_{n\in\mathbb{N}}$ is tight since under H 3($n$) we have
that there exist $R\geq 0$ and $\upalpha>0$ such that for any
$x\in\mathbb{R}^{d}$ with $\|x\|\geq R$ $U(x)\geq\mathtt{m}\|x\|^{\upalpha}$.
Furthermore, Section 3.2.2 implies that each limiting point of the sequence
$(\mu^{\otimes n}\mathrm{S}_{0}[U])_{n\in\mathbb{N}}$ is concentrated on the
minimizers of $U$ when $n\to+\infty$, i.e. when the number of training points
$\\{z_{i}\\}_{i=1}^{n}$ grows to infinity. However, we do not necessarily have
that $(\mu^{\otimes n}\mathrm{S}_{0}[U])_{n\in\mathbb{N}}$ converges towards
$\pi_{0}$ given for any $\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ by

$\textstyle{\pi_{0}(\mathsf{A})=\int_{\mathsf{A}\cap\mathsf{C}}\det(\nabla^{2}U(x))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x)/\int_{\mathsf{C}}\det(\nabla^{2}U(x))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x)}\;,$
(58)

where $\mathsf{C}=\operatorname*{arg\,min}\\{U(x)\,:\;x\in\mathbb{R}^{d}\\}$.

In what follows, we describe a counter-example for which $(\mu^{\otimes
n}\mathrm{S}_{0})_{n\in\mathbb{N}}$ does not converges weakly towards
$\pi_{0}$. Let $\mathsf{X}=\mathbb{R}$, $\mathsf{Z}=\left[-1/2,1/2\right]$ and
$u:\ \mathsf{X}\times\mathsf{Z}\to\mathbb{R}$ such that for any
$x\in\mathbb{R}$ and $z\in\left[-1/2,1/2\right]$ we have

$\textstyle{u(x,z)=\left\\{\begin{matrix}[l]&(x+\uppi)^{4}/(1+(x+\uppi)^{2})+\cos(3x)+zx\;,&\text{if
$x<-\uppi\;,$}\\\ &\cos(3x)+zx\;,&\text{if $-\uppi\leq x\leq\uppi\;,$}\\\
&(x-\uppi)^{4}/(1+(x-\uppi)^{2})+\cos(3x)+zx\;,&\text{if $x>\uppi\;.$}\\\
\end{matrix}\right.}$ (59)

We also let $\mu$ to be the uniform measure on $[-1/2,1/2]$. We obtain that
for any $n\in\mathbb{N}$, $z^{1:n}\in\mathsf{Z}^{n}$ and $x\in\mathsf{X}$

$\textstyle{U(x)=u(x,0)\;,\qquad
U_{n}(x,z^{1:n})=u(x,(1/n)\sum_{k=1}^{n}z^{k})\;.}$ (60)

Note that for any $z\neq 0$, $x\mapsto u(x,z)$ admits a unique global
minimizer, see the proof of Section 3.2.2, whereas if $z=0$, $x\mapsto u(x,z)$
admits four global minimizers
$\\{x_{i}^{\star}\\}_{i=1}^{4}=\\{-\uppi,-\uppi/3,\uppi/3,\uppi\\}$, see
Figure 4 for an illustration. In the next proposition, we show that
$\lim_{n\to+\infty}\mu^{\otimes n}\mathrm{S}_{0}[U]=U^{\star}$ but that
$(\mu^{\otimes n}\mathrm{S}_{0})_{n\in\mathbb{N}}$ does _not_ converge weakly
towards $\pi_{0}$.

######  Proposition .

Let $u$ be given by (59) and $\mu$ be the uniform measure on
$\left[-1/2,1/2\right]$. Then, we have that for any $n\in\mathbb{N}$

$\textstyle{\lim_{n\to+\infty}\mu^{\otimes
n}\mathrm{S}_{0}=(\updelta_{-\uppi}+\updelta_{\uppi})/2\;,\qquad\mu^{\otimes
n}\mathrm{S}_{0}[U]-U^{\star}\leq(\uppi/(6\sqrt{3}))n^{-1/2}\;.}$ (61)

###### Proof.

The proof is postponed to Section 4.3.3. ∎

$x$$y$

$x$$y$

Figure 4: Left: the function $x\mapsto u(x,0)$ with $u$ given in (59). Right:
the function $x\mapsto u(x,0.5)$. The global minimizers are given by the
dotted red lines.

In particular, we have that $(\mu^{\otimes n}\mathrm{S}_{0})_{n\in\mathbb{N}}$
does not converge weakly towards $\pi_{0}$ but converges towards a limiting
probability measure supported on the set of minimizers of $U$.

####  3.2.3 The importance of the _thermodynamic barrier_

To conclude this section, we investigate the role of the _thermodynamic
barrier_ in order to establish quantitative parametric Laplace-type results.
This quantity should not be confused with the concept of _kinetic barrier_
which has been investigated in the context of simulated annealing, see [40,
39] for instance. We refer to [67] for an introduction to the concept of
_thermodynamic barrier_ and _kinetic barrier_ in the context of chemistry, see
also Figure 5 for an illustration.

In a general setting, we consider a function $f:\
\mathsf{X}\times\mathsf{Z}\to\mathbb{R}$ where $\mathsf{X}$ and $\mathsf{Z}$
are topological spaces and $f(\cdot,z)$ admits a global minimizer for any
$z\in\mathsf{Z}$. Let $z\in\mathsf{Z}$, if $f(\cdot,z)$ admits a local
minimizer which is not a global minimizer we recall that the _thermodynamic
barrier_ $c^{\star}(z)$ is given by

$c^{\star}(z)=\inf\\{f(x,z)\,:\;\text{$x$ is a local minimizer of $f(\cdot,z)$
but not a global minimizer}\\}-f^{\star}(z)\;,$ (62)

with $f^{\star}(z)=\inf\\{f(x,z)\,:\;x\in\mathsf{X}\\}$. The _thermodynamic
barrier_ quantifies how close the values of the local minima are to the global
ones. Let $\mathsf{X}=\mathbb{R}^{d}$ and for any $\varepsilon>0$,
$z\in\mathsf{Z}$ and $\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$, define

$\displaystyle\textstyle{\updelta_{z}\mathrm{S}_{\varepsilon}(\mathsf{A})=\int_{\mathsf{A}}\exp[-f(x,z)/\varepsilon]\mathrm{d}x/\int_{\mathbb{R}^{d}}\exp[-f(x,z)/\varepsilon]\mathrm{d}x\;,}$
(63)
$\displaystyle\textstyle{\updelta_{z}\mathrm{S}_{0}(\mathsf{A})=\int_{\mathsf{A}\cap\mathsf{C}(z)}\det(\nabla_{x}^{2}f(x,z))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x)/\int_{\mathsf{C}(z)}\det(\nabla_{x}^{2}f(x,z))^{-1/2}\mathrm{d}\mathcal{H}^{0}(x),}$
(64)

with
$\mathsf{C}(z)=\operatorname*{arg\,min}\\{f(x,z)\,:\;x\in\mathbb{R}^{d}\\}$.
In Section B.2 we show (under assumptions on $f$) that for any
$z\in\mathsf{Z}$ and for any $\varphi\in\mathrm{C}(\mathbb{R}^{d},\mathbb{R})$
which satisfies the conditions of Section B.2 there exist $A,\upbeta\geq 0$
and $\bar{\varepsilon}>0$ such that for any
$\varepsilon\in\left(0,\bar{\varepsilon}\right)$

$\left|\updelta_{z}\mathrm{S}_{\varepsilon}[\varphi]-\updelta_{z}\mathrm{S}_{0}[\varphi]\right|\leq
A(1+\sigma_{\upbeta}^{\star}(z))\\{\varepsilon^{1/2}+\varepsilon^{-d/2}\exp[-c^{\star}(z)/\varepsilon]\\}\;,$
(65)

with $A,\upbeta$ independent from $z$. The dependency of the right-hand side
w.r.t $\sigma_{\upbeta}^{\star}(z)$ comes from the fact that
$\updelta_{z}\mathrm{S}_{0}$ is well-defined if and only if
$\nabla_{x}^{2}f(x,z)$ is invertible at the global minimizers of $f(\cdot,z)$.

We now investigate the dependency w.r.t the _thermodynamic barrier_
$c^{\star}$. We are going to build a simple example for which the
_thermodynamic barrier_ plays a crucial role. In particular, we will show that
the dependency of the form $\exp[-c^{\star}(z)/\varepsilon]$ is tight in (65).
Let $k\in\mathbb{N}$, $\mathsf{X}=\\{0,1\\}$, $\mathsf{Z}=\mathbb{R}$ and $f:\
\mathsf{X}\times\mathsf{Z}\to\mathbb{R}$ such that for any $x\in\mathsf{X}$
and $z\in\mathsf{Z}$, $f(x,z)=xz^{2k+1}$. For any $\varepsilon>0$ and
$z\in\mathbb{R}$ we define $\updelta_{z}\mathrm{S}_{\varepsilon}$ by

$\displaystyle\updelta_{z}\mathrm{S}_{\varepsilon}$
$\displaystyle=(\updelta_{0}\exp[-f(0,z)/\varepsilon]+\updelta_{1}\exp[-f(1,z)/\varepsilon])/(1+\exp[-f(1,z)/\varepsilon])$
(66)
$\displaystyle=\updelta_{0}\mathrm{sigm}(z^{2k+1}/\varepsilon)+\updelta_{1}\mathrm{sigm}(-z^{2k+1}/\varepsilon)\;,$
(67)

where $\mathrm{sigm}:\ \mathbb{R}\to\mathbb{R}$ is the sigmoid function given
for any $t\in\mathbb{R}$ by $\mathrm{sigm}(t)=(1+\exp[-t])^{-1}$. When $z>0$
the minimum of $f(\cdot,z)$ is $0$ and is attained at $x=0$. When $z=0$, we
have $f=0$ (and the minimum is therefore attained at $x=0$ and $x=1$). When
$z<0$ the minimum of $f(\cdot,z)$ is $-z^{2k+1}$ and is attained at $x=1$.
Therefore we have that $\updelta_{z}\mathrm{S}_{0}=\updelta_{0}$ if $z>0$,
$\updelta_{z}\mathrm{S}_{0}=\updelta_{1}$ if $z<0$ and
$\updelta_{z}\mathrm{S}_{0}=(\updelta_{0}+\updelta_{1})/2$ if $z=0$. Using
that the Wasserstein distance of order $1$ between two Bernoulli distribution
with parameter $p_{1}$ and $p_{2}$ is given by $|p_{1}-p_{2}|$ we get that for
any $z\in\mathsf{Z}$ and $\varepsilon>0$

$\mathbf{W}_{1}(\updelta_{z}\mathrm{S}_{\varepsilon},\updelta_{z}\mathrm{S}_{0})=\mathrm{sigm}(-\left|z\right|^{2k+1}/\varepsilon)\;.$
(68)

Hence, for a _fixed_ value of $z\in\mathsf{Z}$, we get that
$\mathbf{W}_{1}(\updelta_{z}\mathrm{S}_{\varepsilon},\updelta_{z}\mathrm{S}_{0})$
is of order $\mathcal{O}(\exp[-\left|z\right|^{2k+1}/\varepsilon])$. In
particular, we get that for any $z\in\mathsf{Z}$, $\limsup_{\varepsilon\to
0}\mathbf{W}_{1}(\updelta_{z}\mathrm{S}_{\varepsilon},\updelta_{z}\mathrm{S}_{0})/\varepsilon$
is bounded. In what follows, we show that $\liminf_{\varepsilon\to
0}\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})/\varepsilon=+\infty$,
for some probability measure $\mu\in\mathscr{P}(\mathbb{R})$.

Let $\chi:\ \mathsf{X}\to\mathbb{R}$ with $\chi(0)=1$ and $\chi(1)=0$. First,
note that for any $z>0$ we have

$\updelta_{z}\mathrm{S}_{\varepsilon}[\chi]-\updelta_{z}\mathrm{S}_{0}[\chi]=\mathrm{sigm}(-z^{2k+1}/\varepsilon)\;.$
(69)

Hence, using that $c^{\star}(z)=\left|z\right|^{2k+1}$ for any
$z\in\mathbb{R}$ and that $\mathrm{sigm}(-t)\geq\exp[-2t]$ for any $t>0$ we
have for any $z>0$,

$\updelta_{z}\mathrm{S}_{\varepsilon}[\chi]-\updelta_{z}\mathrm{S}_{0}[\chi]\geq\exp[-2z^{2k+1}/\varepsilon]\geq\exp[-2c^{\star}(z)/\varepsilon]\;.$
(70)

Let $\mu\in\mathscr{P}(\mathbb{R}_{+})$ such that $\mu(\\{0\\})=0$. Using (70)
and that $\chi$ is $1$-Lipschitz, we have

$\textstyle{\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})\geq\int_{0}^{+\infty}\exp[-2c^{\star}(z)/\varepsilon]\mathrm{d}\mu(z)\;.}$
(71)

Assume that $\mu$ is the uniform distribution on $\left[0,1\right]$. Then, we
have for any $\varepsilon\in\left(0,1\right)$

$\displaystyle\textstyle{\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})}\geq\textstyle{\int_{0}^{1}\exp[-2z^{2k+1}/\varepsilon]\mathrm{d}x\geq
2^{-1/(2k+1)}(\int_{0}^{1}\exp[-z^{2k+1}]\mathrm{d}z)\varepsilon^{1/(2k+1)}\;.}$
(72)

This shows that the order of
$\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})$ is at most
$\mathcal{O}(\varepsilon^{1/(2k+1)})$. This is in stark contrast with the
order identified for
$\mathbf{W}_{1}(\updelta_{z}\mathrm{S}_{\varepsilon},\updelta_{z}\mathrm{S}_{0})$.

This latter observation highlights the crucial role of the _thermodynamic
barrier_ when establishing uniform Laplace-type results w.r.t some parameter
$z\in\mathsf{Z}$. Note that we recover that
$\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})$ is of order at
least $\mathcal{O}(\varepsilon)$ if $\mu$ is supported on
$\left[z_{0},+\infty\right)$ with $z_{0}>0$. Similar conclusions hold if we
show that $c^{\star}(z)\geq c_{0}$ for any $z\in\mathbb{R}$ with $c_{0}>0$.
Hence, (assuming that $c^{\star}$ is continuous) the discrepancy between the
order of $\mathbf{W}_{1}(\mu\mathrm{S}_{\varepsilon},\mu\mathrm{S}_{0})$ and
the one of
$\mathbf{W}_{1}(\updelta_{z}\mathrm{S}_{\varepsilon},\updelta_{z}\mathrm{S}_{0})$
might arise if:  (a) At least one of the local minima (which is not a global
minimum) converges towards a global minimum when $z\to z^{\star}$ for some
value of $z^{\star}\in\mathsf{Z}$,  (b) $z^{\star}$belongs to the support of
$\mu$.  If these two conditions are fulfilled then a more careful study of
$\int_{\mathsf{Z}}\exp[-c^{\star}(z)/\varepsilon]\mathrm{d}\mu(z)$ is needed
in order to obtain quantitative bounds.

$x$$y$ Figure 5: Difference between the _thermodynamic barrier_ (blue) and the
_kinetic barrier_ (red).

##  4 Proofs

In this section, we gather the proofs of the previous sections. In Section 4.1
we prove Theorem 2. Then, in Section 4.2 we provide the proofs of the results
of Section 3.1. Finally, the proofs of the results of Section 3.2 are given in
Section 4.3.

###  4.1 Proof of Theorem 2

In this section, we prove Theorem 2. We recall that Theorem 1 is a
straightforward consequence of Theorem 2 upon letting $\Psi=1$. We let
$k\in\mathbb{N}^{\star}$, $F:\ \mathbb{R}^{d}\to\mathbb{R}^{p}$ and $\Psi:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$. For any $\varphi:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$ and $\varepsilon>0$ we define

$\displaystyle\textstyle{\mathcal{I}_{\varepsilon}(\varphi)=C_{\varepsilon}^{-1}\int_{\mathbb{R}^{d}}\varphi(x)\Psi(x)\exp[-\left\|F(x)\right\|^{k}/\varepsilon]\mathrm{d}x\;,\quad\mathcal{J}_{\varepsilon}=\mathcal{I}_{\varepsilon}(1)\;,}$
(73)
$\displaystyle\textstyle{C_{\varepsilon}=\int_{\mathbb{R}^{d}}\exp[-\left\|x\right\|^{k}/\varepsilon]\mathrm{d}x=\varepsilon^{d/k}\int_{\mathbb{R}^{d}}\exp[-\left\|x\right\|^{k}]\mathrm{d}x=\varepsilon^{d/k}C_{1}\;.}$
(74)

In addition, we define

$\textstyle{\mathcal{I}_{0}(\varphi)=\int_{F^{-1}(0)}\varphi(x)\Psi(x)\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-\hat{d}}(x)\;,\quad\mathcal{J}_{0}=\mathcal{I}_{0}(1)\;,}$
(75)

where $\hat{d}=\min(d,p)$ and we recall that for any $x\in\mathbb{R}^{d}$,
$\mathrm{J}F(x)=\det(\mathrm{D}F(x)\mathrm{D}F(x)^{\top})^{1/2}$ if $d\geq p$
and $\mathrm{J}F(x)=\det(\mathrm{D}F(x)^{\top}\mathrm{D}F(x))^{1/2}$
otherwise. If $\varphi:\ \mathbb{R}^{d}\to\mathbb{R}$ and
$I_{\varepsilon}(\left|\varphi\right|)<+\infty$ for some $\varepsilon\geq 0$
we define $\mathcal{I}_{\varepsilon}(\varphi)$ similarly as in (73) and (75).
Note that for any $\varepsilon\geq 0$ and $\varphi:\
\mathbb{R}^{d}\to\mathbb{R}^{p}$ such that it is defined we have
$\pi_{\varepsilon}[\varphi]=\mathcal{I}_{\varepsilon}(\varphi)/\mathcal{J}_{\varepsilon}$.

The rest of this section is organized as follows. In Section 4.1.1, we prove
our main result , i.e. a quantitative Laplace-type result in the case $d\geq
p$ using the coarea formula. In Section 4.1.2, we prove similar results in the
case $d\leq p$ using Laplace’s method and Morse theory. Finally, we conclude
with the proof of Theorem 2 in Section 4.1.3. Additional technical results are
postponed to Appendix A.

####  4.1.1 The case $d\geq p$

In what follows, we assume that $d\geq p$ and for any $\varphi:\
\mathbb{R}^{d}\to\mathbb{R}_{+}$, $t\in\mathbb{R}^{p}$ we define
$\mathcal{L}_{t}(\varphi)$ by

$\textstyle{\mathcal{L}_{t}(\varphi)=\int_{F^{-1}(t)}\varphi(x)\Psi(x)\mathrm{J}F(x)^{-1}\mathrm{d}\mathcal{H}^{d-p}(x)\;.}$
(76)

Note that $\mathcal{L}_{0}(\varphi)=\mathcal{I}_{0}(\varphi)$. Let $\varphi:\
\mathbb{R}^{d}\to\mathbb{R}$. Then, if
$\mathcal{L}_{t}(\left|\varphi\right|)<+\infty$, we define
$\mathcal{L}_{t}(\varphi)$ similarly to (76). The following proposition
establishes that $t\mapsto\mathcal{L}_{t}(\varphi)$ is Lipschitz under mild
regularity conditions. We emphasize that this proposition is no longer true if
$d\leq p$. Indeed, let us consider the following counterexample. Let $F:\
\mathbb{R}\to\mathbb{R}^{2}$ given for any $x\in\mathbb{R}$ by
$F(x)=((1-x^{2})/(1+x^{2}),x(1-x^{2})/(1+x^{2}))$. The set $F(\mathbb{R})$
defines a right strophoid. Then, for any $t\neq 0$ we have
$\mathcal{H}^{0}(F^{-1}(t))=1$ or $0$, but $\mathcal{H}^{0}(F^{-1}(0))=2$ and
therefore $\mathcal{L}_{t}(\mathrm{J}F/\Psi)=1$ or $0$ near $t=0$ but
$\mathcal{L}_{0}(\mathrm{J}F/\Psi)=2$ with $\Psi=1$. Hence
$t\mapsto\mathcal{L}_{t}(\mathrm{J}F/\Psi)$ is not even continuous.

######  Proposition .

Assume H 1, H 2 and that $d\geq p$. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open
such that $F^{-1}(0)\subset\mathsf{U}$ and
$\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$. Then $\lim_{t\to
0}\mathcal{L}_{t}(\varphi)=\mathcal{L}_{0}(\varphi)$. In addition, assume that
$\Psi,\varphi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$. Then there exist
$B_{0}\geq 0$ and $\eta>0$ such that for any
$t\in\bar{\operatorname{B}}(0,\eta)$,

$\left|\mathcal{L}_{t}(\varphi)-\mathcal{L}_{0}(\varphi)\right|\leq
B_{0}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\left\|t\right\|\;,$
(77)

with for any $i\in\\{0,1\\}$ and
$f\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$,
$M_{i,f}=\sup\\{\|\nabla^{i}f(x)\|\,:\;x\in
F^{-1}(\bar{\operatorname{B}}(0,\eta))\\}$, $B_{0}$ and $\eta$ are independent
from $\varphi$ and $\Psi$ and
$F^{-1}(\bar{\operatorname{B}}(0,\eta))\subset\mathsf{U}$.

###### Proof.

First, we show that there exists an explicit diffeomorphism between
$F^{-1}(t)$ and $F^{-1}(0)$ for $\|t\|$ small enough. Then we use the coarea
formula to express $\mathcal{L}_{t}$ as an integral over $F^{-1}(0)$ and use
the dominated convergence theorem to conclude the first part of the proof. For
the second part of the proof we differentiate the diffeomorphism w.r.t the
parameter $t$ and provide explicit bounds for the derivative.

  1. (a)

The set $F^{-1}(0)$ is compact since
$\lim_{\left\|x\right\|\to+\infty}\left\|F(x)\right\|=+\infty$. First, there
exists $\eta_{0}>0$ such that
$F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta_{0}))\subset\mathsf{U}$.
Indeed, since $F^{-1}(0)$ is compact, there exists $\varepsilon_{0}>0$ such
that
$F^{-1}(0)+\bar{\operatorname{B}}_{\infty}(0,\varepsilon_{0})\subset\mathsf{U}$.
We now show that for any $\varepsilon>0$, there exists $\eta_{\varepsilon}>0$
such that
$F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta_{\varepsilon}))\subset
F^{-1}(0)+\bar{\operatorname{B}}_{\infty}(0,\varepsilon)$. If this is false,
we let $\varepsilon>0$ such that for any $\eta>0$,
$F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta))\not\subset
F^{-1}(0)+\bar{\operatorname{B}}_{\infty}(0,\varepsilon)$. Hence there exists
a sequence $(x_{k})_{k\in\mathbb{N}}\in(\mathbb{R}^{d})^{\mathbb{N}}$ such
that $\|F(x_{k})\|_{\infty}\leq 1/(k+1)$ and
$d(x_{k},F^{-1}(0))\geq\varepsilon$. But, up to extraction, there exists
$x^{\star}\in F^{-1}(\bar{\operatorname{B}}_{\infty}(0,1))$ such that
$\lim_{k\to+\infty}x_{k}=x^{\star}$. Then, we have $F(x^{\star})=0$ and
$d(x^{\star},F^{-1}(0))>\varepsilon$, which is absurd. Hence for any
$\varepsilon>0$, there exists $\eta_{\varepsilon}>0$ such that
$F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta_{\varepsilon}))\subset
F^{-1}(0)+\bar{\operatorname{B}}_{\infty}(0,\varepsilon)$. We let
$\eta_{0}=\eta_{\varepsilon_{0}}$.

Second, there exists $\eta_{1}>0$ such that for any $x\in
F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta_{1}))$, $\mathrm{J}F(x)>0$.
Indeed, if this is not the case then there exists $(x_{k})_{k\in\mathbb{N}}$
with $\lim_{k\to+\infty}F(x_{k})=0$ and $\mathrm{J}F(x_{k})=0$. Since
$F^{-1}(\bar{\operatorname{B}}_{\infty}(0,1))$ is compact there exists
$x^{\star}$ such that, up to extraction, $\lim_{k\to+\infty}x_{k}=x^{\star}$.
Then $F(x^{\star})=0$ and $\mathrm{J}F(x^{\star})=0$, which is absurd. We
define $\mathsf{K}_{0}=F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta_{1}))$
and $\mathsf{K}_{1}=F^{-1}(\bar{\operatorname{B}}_{\infty}(0,\eta))$ with
$\eta=\min(\eta_{1}/2,\eta_{0})$. Note that $\mathsf{K}_{1}\subset\mathsf{U}$
and $\mathsf{K}_{1}\subset\mathrm{int}(\mathsf{K}_{0})$.

In what follows for any $x\in\mathbb{R}^{d}$ we define
$G(x)=\mathrm{D}F(x)\mathrm{D}F(x)^{\top}$. Note that for any
$x\in\mathbb{R}^{d}$, $\det(G(x))^{1/2}=\mathrm{J}F(x)$ since $d\geq p$. We
also have that for any $x\in\mathsf{K}_{0}$, $G(x)$ is invertible since
$\mathrm{J}F(x)>0$. In addition, we have that for any $i,j\in\\{1,\dots,p\\}$
and $x\in\mathbb{R}^{d}$

$G_{i,j}(x)=\langle\nabla F_{i}(x),\nabla F_{j}(x)\rangle\;.$ (78)

We define $\\{f_{i}\\}_{i=1}^{p}$ such that for any $i\in\\{1,\dots,p\\}$,
$f_{i}:\ \mathbb{R}^{d}\to\mathbb{R}^{d}$ with for any $x\in\mathsf{K}_{0}$

$\textstyle{f_{i}(x)=\sum_{k=1}^{p}h_{i,k}(x)\nabla F_{k}(x)\;,}$ (79)

with $\\{h_{i,j}(x)\\}_{1\leq i,j\leq p}=G(x)^{-1}$. For any
$x\in\mathsf{K}_{0}$ and $i,j\in\\{1,\dots,p\\}$ we have

$\textstyle{\langle f_{i}(x),\nabla
F_{j}(x)\rangle=\sum_{k=1}^{p}h_{i,k}(x)\langle\nabla F_{k}(x),\nabla
F_{j}(x)\rangle=\updelta_{i}(j)\;,}$ (80)

where $\updelta_{i}$ is the Dirac mass at $i$. In what follows, we let
$\\{g_{i}\\}_{i=1}^{p}$ such that for any $i\in\\{1,\dots,p\\}$, $g_{i}:\
\mathbb{R}^{d}\to\mathbb{R}^{d}$ and
$g_{i}\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R}^{d})$ such that
$g_{i}(x)=f_{i}(x)$ for any $x\in\mathsf{K}_{1}$, and $g_{i}(x)=0$ for
$x\in\mathrm{int}(\mathsf{K}_{0})^{\mathrm{c}}$, such functions exist using
Whitney extension theorem for instance, see [69]. In what follows, we fix
$t=(t_{1},\dots,t_{p})\in\bar{\operatorname{B}}_{\infty}(0,\eta)$. For any
$i\in\\{1,\dots,p\\}$ let $\Phi_{i}:\
\mathbb{R}\times\mathbb{R}^{d}\to\mathbb{R}^{d}$ given by $\Phi_{i}(0,x)=x$
for any $x\in\mathbb{R}^{d}$ and for any $s\in\mathbb{R}$ and
$x\in\mathbb{R}^{d}$

$\partial_{s}\Phi_{i}(s,x)=-g_{i}(\Phi_{i}(s,x))\;.$ (81)

For any $i\in\\{1,\dots,p\\}$, $\Phi_{i}$ is well-defined using Appendix C.
Therefore, we have for any $i\in\\{1,\dots,p\\}$ and $s\in\mathbb{R}$,
$x\in\mathbb{R}^{d}$ such that $\Phi_{i}(s,x)\in\mathsf{K}_{1}$

$\partial_{s}F(\Phi_{i}(s,x))=-(\langle g_{i}(\Phi_{i}(s,x)),\nabla
F_{1}(\Phi_{i}(s,x))\rangle,\dots,\langle g_{i}(\Phi_{i}(s,x)),\nabla
F_{p}(\Phi_{i}(s,x))\rangle)=-e_{i}\;,$ (82)

where we recall that $\\{e_{i}\\}_{i=1}^{p}$ is the canonical basis of
$\mathbb{R}^{p}$. We define $\bar{\Phi}_{t}:\ \mathbb{R}^{d}\to\mathbb{R}^{d}$
such that for any $x\in\mathbb{R}^{d}$, $\bar{\Phi}_{t}(x)=x^{(p)}$ with
$x^{(0)}=x$ and for any $i\in\\{0,\dots,p-1\\}$,
$x^{(i+1)}=\Phi_{i+1}(t_{i+1},x^{(i)})$. Note that
$\bar{\Phi}_{t}\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$ and is a
diffeomorphism, see Appendix C. Using (81) we have that
$\bar{\Phi}_{t}(F^{-1}(t))=F^{-1}(0)$. In addition, $F^{-1}(t)$ is
$\mathcal{H}^{d-p}$ countably rectifiable using Appendix C. Using this result
and the coarea formula, see Theorem C.1, we have

$\textstyle{\mathcal{L}_{t}(\varphi)=\int_{F^{-1}(0)}\varphi(\bar{\Phi}_{t}^{-1}(x))\Psi(\bar{\Phi}_{t}^{-1}(x))\mathrm{J}F(\bar{\Phi}_{t}^{-1}(x))^{-1}|\det(\mathrm{D}\bar{\Phi}_{t}^{-1}(x))|\mathrm{d}\mathcal{H}^{d-p}(x)\;.}$
(83)

Since $F^{-1}(0)\times\operatorname{B}_{\infty}(0,\eta)$ is compact and
$(t,x)\mapsto\bar{\Phi}_{t}^{-1}(x)$ and
$(t,x)\mapsto\mathrm{D}\bar{\Phi}_{t}^{-1}(x)$ are continuous with for any
$x\in\mathbb{R}^{d}$, $\bar{\Phi}_{0}^{-1}(x)=x$ and
$\mathrm{D}\bar{\Phi}_{0}^{-1}(x)=\operatorname{Id}$, we get that $\lim_{t\to
0}\mathcal{L}_{t}(\varphi)=\mathcal{L}_{0}(\varphi)$ using the dominated
convergence theorem.

  2. (b)

For the second part of the proof we control the derivative of
$t\mapsto\chi(t,x)$ where for any $x\in\mathbb{R}^{d}$ and
$t\in\mathbb{R}^{p}$ we have

$\chi(t,x)=\varphi(\bar{\Phi}_{t}^{-1}(x))\Psi(\bar{\Phi}_{t}^{-1})\mathrm{J}F(\bar{\Phi}_{t}^{-1}(x))^{-1}|\det(\mathrm{D}\bar{\Phi}_{t}^{-1}(x))|\;.$
(84)

Using Section A.3, there exists $P\in\mathtt{Poly}(4,\mathbb{R}_{+})$ such
that for $x\in F^{-1}(0)$ and $t\in\operatorname{B}_{\infty}(0,\eta)$ we have

$\displaystyle\left\|\partial_{t}\chi(t,x)\right\|$
$\displaystyle\leq(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})$
(85) $\displaystyle\quad\times
P(M_{1,F},M_{2,F},M_{3,F},1/m_{1,F})\exp[P(M_{1,F},M_{2,F},M_{3,F},1/m_{1,F})]\;,$
(86)

with $P$ independent from $\varphi$ and $\Psi$. Hence we have that for any
$t\in\operatorname{B}_{\infty}(0,\eta)$

$\displaystyle\left|\mathcal{L}_{t}(\varphi)-\mathcal{L}_{0}(\varphi)\right|$
$\displaystyle\leq\textstyle{\int_{F^{-1}(0)}\left|\chi(t,x)-\chi(0,x)\right|\mathrm{d}\mathcal{H}^{d-p}(x)}$
(87)
$\displaystyle\leq(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})P(M_{1,F},M_{2,F},M_{3,F},1/m_{1,F})$
(88)
$\displaystyle\qquad\exp[P(M_{1,F},M_{2,F},M_{3,F},1/m_{1,F})]\mathcal{H}^{d-p}(F^{-1}(0))\left\|t\right\|\;,$
(89)

which concludes the proof since $\mathcal{H}^{d-p}(F^{-1}(0))<+\infty$ by
Appendix C.

∎

######  Proposition .

Assume H 1, H 2 and that $d\geq p$. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open
and bounded such that $F^{-1}(0)\subset\mathsf{U}$ and
$\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$. Then there exists
$\eta>0$ such that $\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{0}(\varphi)$,
where for any $\varepsilon>0$,
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{F^{-1}(\operatorname{B}(0,\eta))})$.
In addition, assume that
$\varphi,\Psi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$, then there
exists $A_{2}\geq 0$ such that for any $\varepsilon>0$

$\textstyle{\left|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)\right|\leq
A_{2}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,}$
(90)

with for any $i\in\\{0,1\\}$ and
$f\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$,
$M_{i,f}=\sup\\{\|\nabla^{i}f(x)\|\,:\;x\in
F^{-1}(\bar{\operatorname{B}}(0,\eta))\\}$, $A_{2}$ and $\eta$ are independent
from $\varphi$ and $\Psi$ and
$F^{-1}(\bar{\operatorname{B}}(0,\eta))\subset\mathsf{U}$.

###### Proof.

First, note that $\mathcal{I}_{0}(\varphi)=\mathcal{L}_{0}(\varphi)$, see (75)
and (76). In what follows, we let $\eta>0$ be given by Section 4.1.1 and
define
$\mathcal{I}_{0}^{\mathrm{in}}(\varphi)=C_{\varepsilon}^{-1}\int_{\operatorname{B}(0,\eta)}\exp[-\left\|t\right\|^{k}/\varepsilon]\mathcal{L}_{0}(\varphi)\mathrm{d}t$.
We have

$\textstyle{|\mathcal{I}_{0}(\varphi)-\mathcal{I}_{0}^{\mathrm{in}}(\varphi)|=\left|\mathcal{I}_{0}(\varphi)\right|C_{\varepsilon}^{-1}\int_{\operatorname{B}(0,\eta)^{\mathrm{c}}}\exp[-\left\|t\right\|^{k}/\varepsilon]\mathrm{d}t\leq
2^{d/k}\left|\mathcal{I}_{0}(\varphi)\right|\exp[-\eta^{k}/(2\varepsilon)]\;.}$
(91)

Using the coarea formula, see Theorem C.1, we have for any $\varepsilon>0$,

$\textstyle{\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=C_{\varepsilon}^{-1}\int_{F^{-1}(\operatorname{B}(0,\eta))}\Psi(x)\varphi(x)\exp[-\left\|F(x)\right\|^{k}/\varepsilon]=C_{\varepsilon}^{-1}\int_{\operatorname{B}(0,\eta)}\exp[-\left\|t\right\|^{k}/\varepsilon]\mathcal{L}_{t}(\varphi)\mathrm{d}t\;.}$
(92)

Therefore, using this result and the change of variable
$t\mapsto\varepsilon^{1/k}t$ we have

$\displaystyle|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}^{\mathrm{in}}(\varphi)|$
$\displaystyle\textstyle{\leq
C_{\varepsilon}^{-1}\int_{\operatorname{B}(0,\eta)}\exp[-\left\|t\right\|^{k}/\varepsilon]|\mathcal{L}_{t}(\varphi)-\mathcal{L}_{0}(\varphi)|\mathrm{d}t}$
(93)
$\displaystyle\leq\textstyle{C_{1}^{-1}\int_{\operatorname{B}(0,\eta/\varepsilon^{1/k})}\exp[-\left\|t\right\|^{k}]|\mathcal{L}_{t\varepsilon^{1/k}}(\varphi)-\mathcal{L}_{0}(\varphi)|\mathrm{d}t\;.}$
(94)

Hence, we get that $\lim_{\varepsilon\to
0}|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}^{\mathrm{in}}(\varphi)|=0$
using the dominated convergence theorem and that $\lim_{\varepsilon\to
0}|\mathcal{L}_{t\varepsilon^{1/k}}(\varphi)-\mathcal{L}_{0}(\varphi)|=0$
according to Section 4.1.1. This concludes the first part of the proof upon
combining this result and (91). In addition, assume that
$\varphi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$ then using the second
part of Section 4.1.1 and (94) we have

$\left|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}^{\mathrm{in}}(\varphi)\right|\leq\textstyle{B_{0}C_{1}^{-1}\int_{\mathbb{R}^{p}}\left\|t\right\|\exp[-\left\|t\right\|^{k}]\mathrm{d}t(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,}$
(95)

which concludes the proof upon combining this result and (91). ∎

####  4.1.2 The case $d\leq p$

We now turn to the case $d\leq p$. The proof of this result is more classical
and does not rely on geometric measure theory. Instead we build on the Morse
theory approach for Laplace approximation, see [70] for example. The following
proposition is a quantitative extension of [70, Theorem 3, p.495].

######  Proposition .

Assume H 1, H 2 and that $d\leq p$. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open
and bounded such that $F^{-1}(0)\subset\mathsf{U}$ and
$\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$. Then
$\lim_{\varepsilon\to
0}|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)|=0$,
with
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{\mathsf{V}})$
and $\mathsf{V}$ open such that $F^{-1}(0)\subset\mathsf{V}\subset\mathsf{U}$.
In addition, assume that
$\varphi,\Psi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$. Then there
exists $B_{1}\geq 0$ such that for any $\varepsilon>0$ we have

$|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)|\leq
B_{1}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$
(96)

with $B_{1}$ independent from $\varphi$ and $\Psi$ and for any $i\in\\{0,1\\}$
and $f\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$,
$M_{i,f}=\sup\\{\|\nabla^{i}f(x)\|\,:\;x\in\mathsf{U}\\}$.

###### Proof.

Let $\\{x_{0}^{\ell}\\}_{\ell=1}^{N}$ and
$\\{\mathsf{W}_{\ell}\\}_{\ell=1}^{N}$ be given by Section A.2 such that
$F^{-1}(0)=\cup_{\ell=1}^{N}\\{x_{0}^{\ell}\\}$ and $\mathrm{d}F(x)$ is
injective for any $\ell\in\\{1,\dots,N\\}$ and $x\in\mathsf{W}_{\ell}$. In
addition, for any $\ell,m\in\\{1,\dots,N\\}$,
$\mathsf{W}_{\ell}\cap\mathsf{W}_{m}=\emptyset$. Let $\ell\in\\{1,\dots,N\\}$
and $U:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ such that for any
$x\in\mathbb{R}^{d}$, $U(x)=\|F(x)\|^{2}$. Since
$F\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R}^{p})$ we have that
$U\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R})$. We divide the rest of
the proof into two parts.

  1. (a)

First, we have that
$\nabla^{2}U(x_{0}^{\ell})=2\mathrm{D}F(x_{0}^{\ell})^{\top}\mathrm{D}F(x_{0}^{\ell})$
which is invertible since $\mathrm{J}F(x_{0}^{\ell})>0$. Therefore, we can
apply Morse’s lemma [56, Theorem 3.1.1] and there exists a diffeormorphism
$\Phi_{\ell}\in\mathrm{C}^{1}(\bar{\Omega}_{\ell},\bar{\mathsf{W}}_{\ell})$
with $0\in\Omega_{\ell}$, $x_{0}^{\ell}\in\mathsf{W}_{\ell}$ and
$\Omega_{\ell}\subset\mathbb{R}^{d}$ open such that for any
$x\in\Omega_{\ell}$, $U(\Phi_{\ell}(x))=\left\|x\right\|^{2}$,
$\Phi_{\ell}(0)=x_{0}^{\ell}$ and
$\mathrm{D}\Phi_{\ell}(0)=(\mathrm{D}F(x_{0}^{\ell})^{\top}\mathrm{D}F(x_{0}^{\ell}))^{-1/2}$.
Note that $\det(\mathrm{D}\Phi_{\ell}(0))=\mathrm{J}F(x_{0}^{\ell})^{-1}$.

Let $r_{\ell}>0$ such that
$\bar{\operatorname{B}}(0,r_{\ell})\subset\Omega_{\ell}$. We have

$\textstyle{1-\int_{\Omega_{\ell}/\varepsilon^{1/k}}\exp[-\left\|x\right\|^{k}]\mathrm{d}x/C_{1}\leq\int_{\mathbb{R}^{d}}\exp[-\left\|x\right\|^{k}/2]\mathrm{d}x\exp[-r_{\ell}^{k}/(2\varepsilon)]/C_{1}\;.}$
(97)

In what follows, we no longer consider $\ell\in\\{1,\dots,N\\}$ to be fixed.
Let $\mathsf{V}=\cup_{\ell=1}^{N}\Phi_{\ell}(\Omega_{\ell})$ and
$\mathcal{I}_{0,\varepsilon}^{\mathrm{in}}(\varphi)=\sum_{\ell=1}^{N}\int_{\Omega_{\ell}/\varepsilon^{1/k}}\exp[-\left\|x\right\|^{k}]\mathrm{d}x\varphi(x_{0}^{\ell})\Psi(x_{0}^{\ell})\mathrm{J}F(x_{0}^{\ell})^{-1}/C_{1}$.
We recall that we have

$\textstyle{\mathcal{I}_{0}(\varphi)=\sum_{\ell=1}^{N}\varphi(x_{0}^{\ell})\Psi(x_{0}^{\ell})\mathrm{J}F(x_{0}^{\ell})^{-1}\;.}$
(98)

Combining this result and (97) we get

$\textstyle{\left|\mathcal{I}_{0,\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)\right|\leq
NM\int_{\mathbb{R}^{d}}\exp[-\left\|x\right\|^{k}/2]\mathrm{d}x\exp[-r_{\mathrm{min}}^{k}/(2\varepsilon)]/C_{1}\;,}$
(99)

where $r_{\mathrm{min}}=\min\\{r_{\ell}\,:\;\ell\in\\{1,\dots,N\\}\\}$ and
$M=\max\\{|\varphi(x_{0}^{\ell})|\Psi(x_{0}^{\ell})\mathrm{J}F(x_{0}^{\ell})^{-1}\,:\;\ell\in\\{1,\dots,N\\}\\}$.
Using for any $\ell\in\\{1,\dots,N\\}$ the change of variable
$x\mapsto\Phi_{\ell}(x)$ and $x\mapsto\varepsilon^{1/k}x$ we have

$\displaystyle\left|\mathcal{I}_{0,\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)\right|$
$\displaystyle\textstyle{\leq\sum_{\ell=1}^{N}\int_{\Omega_{\ell}/\varepsilon^{1/k}}|\varphi(\Phi_{\ell}(\varepsilon^{1/k}x))\Psi(\Phi_{\ell}(\varepsilon^{1/k}x))\det(\mathrm{D}\Phi_{\ell}(\varepsilon^{1/k}x))}$
(100)
$\displaystyle\qquad\qquad-\varphi(x_{0}^{\ell})\Psi(x_{0}^{\ell})\mathrm{J}F(x_{0}^{\ell})^{-1}|\exp[-\left\|x\right\|^{k}]\mathrm{d}x/C_{1}\;,$
(101)

For any $\ell\in\\{1,\dots,N\\}$, let $\chi_{\ell}:\
\Omega_{\ell}\to\mathbb{R}$ given for any $x\in\Omega_{\ell}$ by

$\chi_{\ell}(x)=\varphi(\Phi_{\ell}(x))\Psi(\Phi_{\ell}(x))\det(\mathrm{D}\Phi_{\ell}(x))\;.$
(102)

We conclude the first part of the proof using (99), the dominated convergence
theorem in (101) and that for any $\ell\in\\{1,\dots,N\\}$,
$\chi\in\mathrm{C}(\Omega_{\ell},\mathbb{R})$.

  2. (b)

For the second part of the proof, since
$\varphi,\Psi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$ and for any
$\ell\in\\{1,\dots,N\\}$,
$\Phi_{\ell}\in\mathrm{C}^{2}(\bar{\Omega}_{\ell},\bar{\mathsf{W}}_{\ell})$
since $F\in\mathrm{C}^{\infty}(\mathbb{R}^{d},\mathbb{R}^{p})$ using [56,
Theorem 3.1.1], we have that for any $\ell\in\\{1,\dots,N\\}$,
$\chi_{\ell}\in\mathrm{C}^{1}(\bar{\Omega}_{\ell},\mathbb{R})$ and there
exists $B_{1}^{\prime}\geq 0$ (independent from $\varphi$ and $\Psi$) such
that for any $\ell\in\\{1,\dots,N\\}$ and $x\in\Omega_{\ell}$ we have

$\left\|\mathrm{D}\chi_{\ell}(x)\right\|\leq
B_{1}^{\prime}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\;.$
(103)

Using this result and (101) we get that

$|\mathcal{I}_{0,\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)|\leq\textstyle{NB_{1}^{\prime}(1+M_{0,\varphi}+M_{1,\varphi})\varepsilon^{1/k}\int_{\mathbb{R}^{d}}\left\|x\right\|\exp[-\left\|x\right\|^{k}]\mathrm{d}x/C_{1}\;.}$
(104)

Combining this result and (99) concludes the proof.

∎

####  4.1.3 Proof of Theorem 2

We start by proving the results of Theorem 2 in a smooth setting, then we
deduce the general case using a smoothing lemma.

######  Proposition .

Assume H 1 and H 2. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open and bounded
such that $F^{-1}(0)\subset\mathsf{U}$ and
$\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$ which satisfies (18).
Then, $\lim_{\varepsilon\to
0}\left|\pi_{\varepsilon}^{\Psi}[\varphi]-\pi_{0}^{\Psi}[\varphi]\right|=0$.
In addition, assume that
$\varphi,\Psi\in\mathrm{C}^{1}(\bar{\mathsf{U}},\mathbb{R})$. Then there
exists $A\in\mathrm{C}(\mathbb{R}_{+}^{3},\mathbb{R}_{+})$ such that for any
$\varepsilon\in(0,\mathtt{m}^{k}/(4\\{C_{\varphi}+C_{\Psi}+C_{\varphi}C_{\Psi}+1\\}))$

$\left|\pi_{\varepsilon}^{\Psi}[\varphi]-\pi_{0}^{\Psi}[\varphi]\right|\leq
A(C_{\varphi},C_{\Psi},m_{0,\Psi})(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$
(105)

with for any $i\in\\{0,1\\}$ and
$f\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$,
$M_{i,f}=\sup\\{\|\nabla^{i}f(x)\|\,:\;x\in\mathsf{U}\\}$,
$m_{0,\Psi}=\inf\\{\Psi(x)\,:\;x\in F^{-1}(0)\\}$ and $A$ independent function
from $\varphi$ and $\Psi$. Finally, $A$ is non-decreasing w.r.t its first two
variables and non-increasing w.r.t its last variable.

###### Proof.

For any $\varepsilon\in\left(0,\bar{\varepsilon}\right)$ we have

$\displaystyle\left|\pi_{\varepsilon}^{\Psi}[\varphi]-\pi_{0}^{\Psi}[\varphi]\right|$
$\displaystyle\leq\left|\mathcal{I}_{\varepsilon}(\varphi)/\mathcal{J}_{\varepsilon}-\mathcal{I}_{0}(\varphi)/\mathcal{J}_{0}\right|$
(106)
$\displaystyle\leq\mathcal{J}_{\varepsilon}^{-1}\left|\mathcal{I}_{\varepsilon}(\varphi)-\mathcal{I}_{0}(\varphi)\right|+\mathcal{I}_{0}(\varphi)/(\mathcal{J}_{0}\mathcal{J}_{\varepsilon})\left|\mathcal{I}_{\varepsilon}(1)-\mathcal{I}_{0}(1)\right|\;.$
(107)

Let $\eta>0$ be given by Section 4.1.1 and let
$\mathsf{V}=F^{-1}(\operatorname{B}(0,\eta))$ if $d\geq p$, and $\mathsf{V}$
given by Section 4.1.2 otherwise. Note that $F^{-1}(0)\subset\mathsf{V}$. For
any $\varepsilon\in\left(0,\bar{\varepsilon}\right)$ we define
$\mathcal{I}_{\varepsilon}^{\mathrm{out}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{\mathsf{V}^{\mathrm{c}}})$
and
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{\mathsf{V}})$.
Let $\bar{\varepsilon}=\mathtt{m}^{k}/(4+4C_{\varphi,\Psi})$, with
$C_{\varphi,\Psi}=(C_{\Psi}+1)(C_{\varphi}+1)$ and note that we have

$\bar{\varepsilon}<\min(\mathtt{m}^{k}/(1+C_{\varphi,\Psi}),\mathtt{m}^{k}/(1+C_{\varphi,1}))\;.$
(108)

We divide the rest of the proof into two parts. First, we control
$\mathcal{I}_{\varepsilon}^{\mathrm{out}}(\varphi)$ using the technical bounds
of Section A.2. Then, we control
$|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)|$
using either Section 4.1.1 if $d\geq p$ or Section 4.1.2 if $d\leq p$. We
conclude upon combining these results.

  1. (a)

Using (108) and Section A.2 we get that for any
$\varepsilon\in(0,\bar{\varepsilon})$ we have

$\mathcal{I}_{\varepsilon}^{\mathrm{out}}(\varphi)\leq
A_{1}(C_{\varphi,\Psi})\varepsilon^{-d/k}\exp[-\beta_{1}/\varepsilon]\;,\qquad\mathcal{I}_{\varepsilon}^{\mathrm{out}}(1)\leq
A_{1}(C_{1,\Psi})\varepsilon^{-d/k}\exp[-\beta_{1}/\varepsilon]\;,$ (109)

with $\beta_{1}>0$ and $A_{1}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+})$
independent from $\varphi$ and $\Psi$ and non-decreasing.

  2. (b)

Using either Section 4.1.1 if $d\geq p$ or Section 4.1.2 if $d\leq p$ we have
that for any $\varepsilon\in(0,\bar{\varepsilon})$

$\displaystyle\left|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)-\mathcal{I}_{0}(\varphi)\right|\leq
A_{2}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$
(110)
$\displaystyle\left|\mathcal{I}_{\varepsilon}^{\mathrm{in}}(1)-\mathcal{I}_{0}(1)\right|\leq
2A_{2}(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$ (111)

with for any $i\in\\{0,1\\}$ and
$f\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$,
$M_{i,f}=\sup\\{\|\nabla^{i}f(x)\|\,:\;x\in\mathsf{U}\\}$, $A_{2}$ independent
from $\varphi$ and $\Psi$.

Combining (109) and (110) we get that for any
$\varepsilon\in(0,\bar{\varepsilon})$

$\displaystyle\left|\mathcal{I}_{\varepsilon}(\varphi)-\mathcal{I}_{0}(\varphi)\right|\leq
A_{2}(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}+A_{1}(C_{\varphi,\Psi})\varepsilon^{-d/k}\exp[-\beta_{1}/\varepsilon]\;,$
(112)
$\displaystyle\left|\mathcal{I}_{\varepsilon}(1)-\mathcal{I}_{0}(1)\right|\leq
2A_{2}(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}+A_{1}(C_{1,\Psi})\varepsilon^{-d/k}\exp[-\beta_{1}/\varepsilon]\;.$
(113)

Since for any $\varepsilon>0$,
$\varepsilon^{-d/k}\exp[-\beta/\varepsilon]\leq(1/(k\beta)+k/(d\beta))^{1/k+d/k}\varepsilon^{1/k}$
there exists $\tilde{A}\in\mathrm{C}(\mathbb{R}_{+}^{2},\mathbb{R}_{+})$ such
that for any $\varepsilon\in(0,\bar{\varepsilon})$ we have

$\displaystyle\left|\mathcal{I}_{\varepsilon}(\varphi)-\mathcal{I}_{0}(\varphi)\right|\leq\tilde{A}(C_{\varphi},C_{\Psi})(1+M_{0,\varphi}+M_{1,\varphi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$
(114)
$\displaystyle\left|\mathcal{I}_{\varepsilon}(1)-\mathcal{I}_{0}(1)\right|\leq
2\tilde{A}(1,C_{\Psi})(1+M_{0,\Psi}+M_{1,\Psi})\varepsilon^{1/k}\;,$ (115)

with $\tilde{A}$ independent from $\Psi$ and $\varphi$ and $\tilde{A}$ non-
decreasing w.r.t. to each of its variables. Using Section A.2, there exists
$A_{0}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+}^{\star})$ such that for any
$\varepsilon\in\left[0,\bar{\varepsilon}\right]$,
$\mathcal{J}_{\varepsilon}\geq A_{0}(m_{0,\Psi})$ with
$m_{0,\Psi}=\inf\\{\Psi(x)\,:\;x\in F^{-1}(0)\\}$ and $A_{0}$ independent from
$\Psi$ and non-increasing. Finally, note that
$\mathcal{I}_{0}(\varphi)\leq\inf\\{\mathrm{J}F(x)^{-1}\,:\;x\in
F^{-1}(0)\\}M_{0,\varphi}\mathcal{H}^{d-p}(F^{-1}(0))$ with
$\mathcal{H}^{d-p}(F^{-1}(0))<+\infty$ by Appendix C. Combining these results,
(106) and (114) concludes the proof. ∎

Using this proposition along with a smoothing lemma, see Section A.4, we
conclude the proof of Theorem 2.

###### Proof of Theorem 2.

We begin by introducing the families
$\\{\varphi^{\delta}\,:\;\delta\in\left(0,\bar{\delta}\right)\\}$ and
$\\{\Psi^{\delta}\,:\;\delta\in\left(0,\bar{\delta}\right)\\}$ which are
smooth approximations of $\varphi$ and $\Psi$ respectively. Since $F^{-1}(0)$
is compact and $\mathsf{U}^{\mathrm{c}}$ is closed there exists $r>0$ such
that $F^{-1}(0)+\operatorname{B}(0,r)\subset\mathsf{U}$. Let
$\mathsf{U}_{0}=F^{-1}(0)+\operatorname{B}(0,r/2)$ and remark that
$F^{-1}(0)\subset\mathsf{U}_{0}$ and
$\mathsf{U}_{0}+\operatorname{B}(0,r/2)\subset\mathsf{U}$. Let
$\\{\varphi^{\delta}\,:\;\delta\in\left(0,\bar{\delta}\right)\\}$ and
$\\{\Psi^{\delta}\,:\;\delta\in\left(0,\bar{\delta}\right)\\}$ with
$\bar{\delta}>0$ given by Section A.4, $\mathsf{V}\leftarrow\mathsf{U}$ and
$\mathsf{U}\leftarrow\mathsf{U}_{0}$. Then using the dominated convergence
theorem and the fact that $F^{-1}(0)$ is compact we have that $\lim_{\delta\to
0}\pi_{0}^{\Psi^{\delta}}[\varphi^{\delta}]=\pi_{0}^{\Psi}[\varphi]$.
Similarly, using the dominated convergence theorem we get that there exists
$\bar{\varepsilon}_{0}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+})$ such that
for any $\varepsilon\in(0,\bar{\varepsilon}_{0}(C_{\varphi}))$ we have
$\lim_{\delta\to
0}\pi^{\Psi^{\delta}}_{\varepsilon}[\varphi^{\delta}]=\pi^{\Psi}_{\varepsilon}[\varphi]$.
We conclude upon using Section 4.1.3 and Section A.4.

∎

###  4.2 Proofs of Section 3.1

In this section, we prove Section 3.1.2. Similarly to the proof of Theorem 2
we divide the proof into two parts depending on whether $d\geq p$ in Section
4.2.1 or $d\leq p$ in Section 4.2.2. Our main result, which is a
generalization of Section 3.1.2 is presented in Section 4.2.3. We define
$\mathcal{I}_{\varepsilon}$ and $\mathcal{I}_{0}$ as in (73) and (75).

####  4.2.1 The case $d\geq p$

Our first result corresponds to an adaptation of Section 4.1.1 to the case
where $\varphi=\|F\|^{k}$. Indeed, in this case we have that
$\mathcal{I}_{0}(\varphi)=0$ and we can tighten our previous results.

######  Proposition .

Assume H 1, H 2 and that $d\geq p$. Then there exists $\eta>0$ with
$\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})/\varepsilon=\mathcal{C}_{k}\mathcal{I}_{0}(1)$,
where for any $\varepsilon>0$,
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{F^{-1}(\operatorname{B}(0,\eta))})$
for any $\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$ and

$\textstyle{\mathcal{C}_{k}=\left.\int_{\mathbb{R}^{p}}\|t\|^{k}\exp[-\|t\|^{k}]\mathrm{d}t\middle/\int_{\mathbb{R}^{p}}\exp[-\|t\|^{k}]\mathrm{d}t\right.}\;.$
(116)

###### Proof.

Let $\eta>0$ be given by Section 4.1.1 with $\varphi=1$. We recall that for
any $\varepsilon>0$ we have that

$\textstyle{\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})=\int_{F^{-1}(\operatorname{B}(0,\eta))}\|F(x)\|^{k}\Psi(x)\exp[-\|F(x)\|^{k}/\varepsilon]\mathrm{d}x/C_{\varepsilon}\;.}$
(117)

Using the coarea formula, see Theorem C.1, we have for any $\varepsilon>0$,

$\textstyle{\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})/\varepsilon=\int_{\operatorname{B}(0,\eta)}(\left\|t\right\|^{k}/\varepsilon)\exp[-\left\|t\right\|^{k}/\varepsilon]\mathcal{L}_{t}(1)\mathrm{d}t/C_{\varepsilon}\;,}$
(118)

where $\mathcal{L}_{t}(1)$ is defined in (76) for any
$t\in\operatorname{B}(0,\eta)$. Using the change of variable
$t\mapsto\varepsilon^{1/k}t$ we have for any $\varepsilon>0$

$\textstyle{\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})/\varepsilon=\int_{\operatorname{B}(0,\eta/\varepsilon^{1/k})}\left\|t\right\|^{k}\exp[-\left\|t\right\|^{k}]\mathcal{L}_{t\varepsilon^{1/k}}(1)\mathrm{d}t/C_{1}\;.}$
(119)

For any $\varepsilon>0$ let $g_{\varepsilon}:\ \mathbb{R}^{p}\to\mathbb{R}$
such that for any $t\in\mathbb{R}^{p}$,
$g_{\varepsilon}(t)=\mathcal{L}_{t\varepsilon^{1/k}}(1)\mathbbm{1}_{\operatorname{B}(0,\eta/\varepsilon^{1/k})}(t)$.
Note that for any $t\in\mathbb{R}^{d}$ and $\varepsilon>0$,
$\left|g_{\varepsilon}(t)\right|\leq\sup_{\bar{\operatorname{B}}(0,\eta)}\left|\mathcal{L}_{t}(1)\right|$.
In addition, we have that for any $t\in\mathbb{R}^{d}$, $\lim_{\varepsilon\to
0}g_{\varepsilon}(t)=\mathcal{L}_{0}(1)$ using Section 4.1.1. Therefore, we
get that

$\textstyle{\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})/\varepsilon=\mathcal{L}_{0}(1)\int_{\mathbb{R}^{p}}\left\|t\right\|^{k}\exp[-\left\|t\right\|^{k}]\mathrm{d}t/C_{1}\;.}$
(120)

We conclude the proof upon noting that $\mathcal{I}_{0}(1)=\mathcal{L}_{0}(1)$
and that
$\mathcal{C}_{k}=\int_{\mathbb{R}^{p}}\left\|t\right\|^{k}\exp[-\left\|t\right\|^{k}]\mathrm{d}t/C_{1}$.
∎

####  4.2.2 The case $d\leq p$

We now adapt Section 4.1.2 to the case where $\varphi=\|F\|^{k}$.

######  Proposition .

Assume H 1, H 2 and $d\leq p$. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open and
bounded such that $F^{-1}(0)\subset\mathsf{U}$. Then $\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})/\varepsilon=\mathcal{C}_{k}\mathcal{I}_{0}(1)$,
where for any $\varepsilon>0$,
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\varphi)=\mathcal{I}_{\varepsilon}(\varphi\mathbbm{1}_{\mathsf{V}})$
for any $\varphi\in\mathrm{C}(\bar{\mathsf{U}},\mathbb{R})$, with $\mathsf{V}$
open such that $F^{-1}(0)\subset\mathsf{V}\subset\mathsf{U}$, and

$\textstyle{\mathcal{C}_{k}=\left.\int_{\mathbb{R}^{p}}\|t\|^{k}\exp[-\|t\|^{k}]\mathrm{d}t\middle/\int_{\mathbb{R}^{p}}\exp[-\|t\|^{k}]\mathrm{d}t\right.\;.}$
(121)

###### Proof.

Let $\\{x_{0}^{\ell}\\}_{\ell=1}^{N}$, $\\{\mathsf{W}_{\ell}\\}_{\ell=1}^{N}$
and $\\{\Phi_{\ell}\\}_{\ell=1}^{N}$ be given as in the proof of Section
4.1.2. Let $\mathsf{V}=\cup_{\ell=1}^{N}\Phi_{\ell}(\Omega_{\ell})$. Using for
any $\ell\in\\{1,\dots,N\\}$ the change of variable $x\mapsto\Phi_{\ell}(x)$
and $x\mapsto\varepsilon^{1/k}x$ we have

$\textstyle{\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})\varepsilon=\sum_{\ell=1}^{N}\int_{\Omega_{\ell}/\varepsilon^{1/k}}\|x\|^{k}\exp[-\left\|x\right\|^{k}]\Psi(\Phi_{\ell}(\varepsilon^{1/k}x))\det(\mathrm{D}\Phi_{\ell}(\varepsilon^{1/k}x))\mathrm{d}x/C_{1}\;.}$
(122)

For any $\ell\in\\{1,\dots,N\\}$ and $\varepsilon>0$, let
$g_{\ell,\varepsilon}:\ \mathbb{R}^{p}\to\mathbb{R}$ such that for any
$\ell\in\\{1,\dots,N\\}$, $\varepsilon>0$ and $x\in\mathbb{R}^{p}$ we have

$g_{\ell,\varepsilon}(x)=\Psi(\Phi_{\ell}(\varepsilon^{1/k}x))\det(\mathrm{D}\Phi_{\ell}(\varepsilon^{1/k}x))\mathbbm{1}_{\Omega_{\ell}/\varepsilon^{1/k}}(x)\;.$
(123)

Note that for any $\ell\in\\{1,\dots,N\\}$, $\varepsilon>0$ and
$x\in\mathbb{R}^{p}$ we have

$|g_{\ell,\varepsilon}(x)|\leq\sup\\{\left|\Psi(\Phi_{\ell}(x))\right|\left|\det(\mathrm{D}\Phi_{\ell}(x))\right|\,:\;\ell\in\\{1,\dots,N\\},\
\ x\in\Omega_{\ell}\\}\;.$ (124)

In addition, we have that for any $\ell\in\\{1,\dots,N\\}$ and
$x\in\mathbb{R}^{p}$, $\lim_{\varepsilon\to
0}g_{\ell,\varepsilon}(x)=\Psi(x_{0}^{\ell})\mathrm{J}F(x_{0}^{\ell})^{-1}$.
We conclude upon using the dominated convergence theorem. ∎

####  4.2.3 Main result

######  Proposition .

Assume H 1 and H 2. Let $\mathsf{U}\subset\mathbb{R}^{d}$ open and bounded
such that $F^{-1}(0)\subset\mathsf{U}$. Then $\lim_{\varepsilon\to
0}\pi_{\varepsilon}^{\Psi}(\|F\|^{k})/\varepsilon=\mathcal{C}_{k}$, where

$\textstyle{\mathcal{C}_{k}=\left.\int_{\mathbb{R}^{p}}\|t\|^{k}\exp[-\|t\|^{k}]\mathrm{d}t\middle/\int_{\mathbb{R}^{p}}\exp[-\|t\|^{k}]\mathrm{d}t\right.\;.}$
(125)

###### Proof.

Let $\eta>0$ be given by Section 4.1.1 with $\varphi=1$ and let
$\mathsf{V}=F^{-1}(\operatorname{B}(0,\eta))$ if $d\geq p$. If $d\leq p$, let
$\mathsf{V}$ be given by Section 4.2.2. We have that
$F^{-1}(0)\subset\mathsf{V}$. For any
$\varepsilon\in\left(0,\bar{\varepsilon}\right)$ we define
$\mathcal{I}_{\varepsilon}^{\mathrm{out}}(\|F\|^{k})=\mathcal{I}_{\varepsilon}(\|F\|^{k}\mathbbm{1}_{\mathsf{V}^{\mathrm{c}}})$
and
$\mathcal{I}_{\varepsilon}^{\mathrm{in}}(\|F\|^{k})=\mathcal{I}_{\varepsilon}(\|F\|^{k}\mathbbm{1}_{\mathsf{V}})$.
Using Section A.2, we have that $\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}^{\mathrm{out}}(\|F\|^{k})/\varepsilon=0$. Hence,
using Section 4.2.1 if $d\geq p$ and Section 4.2.2 if $d\leq p$, we get that
$\lim_{\varepsilon\to
0}\mathcal{I}_{\varepsilon}(\|F\|^{k})/\varepsilon=\mathcal{C}_{k}\mathcal{I}_{0}(1)$.
Similarly, using Section A.2 and Section 4.1.1 we have that
$\lim_{\varepsilon\to 0}\mathcal{I}_{\varepsilon}(1)=\mathcal{I}_{0}(1)$,
which concludes the proof upon remarking that for any $\varepsilon>0$,
$\pi_{\varepsilon}^{\Psi}(\|F\|^{k})=\mathcal{I}_{\varepsilon}(\|F\|^{k})/\mathcal{I}_{\varepsilon}(1)$.
∎

We are now ready to prove a generalization of Section 3.1.2.

######  Proposition .

Let $\mu\in\mathscr{P}(\mathbb{R}^{d})$, $F:\ \mathbb{R}^{d}\to\mathbb{R}^{p}$
and $k\in\mathbb{N}^{\star}$. Assume that the conditions of Section 3.1.1 with
$G_{\varepsilon}=\|F\|^{k}-\varepsilon$ for any $\varepsilon>0$ are satisfied
and for any $\varepsilon>0$, let $\rho_{\varepsilon}$ the macrocanonical
distribution with constraint $G_{\varepsilon}$ and reference measure $\mu$.
Assume that there exists $\Psi:\ \mathbb{R}^{d}\to\mathbb{R}_{+}$ such that
$\mu$ admits a density w.r.t. the Lebesgue measure given by $\Psi$. In
addition, assume that H 1 and H 2 hold. Then, we have that
$\theta_{\varepsilon}\sim_{\varepsilon\to 0}\mathcal{C}_{k}/\varepsilon$,
where

$\textstyle{\mathcal{C}_{k}=\left.\int_{\mathbb{R}^{p}}\|t\|^{k}\exp[-\|t\|^{k}]\mathrm{d}t\middle/\int_{\mathbb{R}^{p}}\exp[-\|t\|^{k}]\mathrm{d}t\right.\;.}$
(126)

###### Proof.

Recall that using Section 3.1.1 we have that for any $\varepsilon>0$ and
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$

$\textstyle{\rho_{\varepsilon}(\mathsf{A})=\left.\int_{\mathsf{A}}\Psi(x)\exp[-\theta_{\varepsilon}\|F(x)\|^{k}]\mathrm{d}x\middle/\int_{\mathbb{R}^{d}}\Psi(x)\exp[-\theta_{\varepsilon}\|F(x)\|^{k}]\mathrm{d}x\right.\;.}$
(127)

Hence, using Section 4.2.3 we have that $\lim_{\varepsilon\to
0}\rho_{\varepsilon}(\|F\|^{k})\theta_{\varepsilon}=\mathcal{C}_{k}$. Since
$\rho_{\varepsilon}[G_{\varepsilon}]=0$ we have also have that
$\rho_{\varepsilon}[\|F\|^{k}]=\varepsilon$, which concludes the proof. ∎

Note that Section 3.1.2 is obtained upon noting that $\mathcal{C}_{2}=p/2$.

###  4.3 Proofs of Section 3.2

In Section 4.3.1, we establish Section 3.2.2. In Section 4.3.2 we use
stability results from [60, 8] to obtain Section 3.2.2. Finally, we prove
Section 3.2.2 in Section 4.3.3. Additional technical results are postponed to
Appendix B.

####  4.3.1 Proof of Section 3.2.2

In this section, we prove Section 3.2.2 which is an application of a
parametric version of the results presented in Section 4.1.2. We refer to
Appendix B for a detailed presentation of these results. We will apply them in
the context of the non-convex minimization setting presented in Section 3.2
which we recall here.

We aim at minimizing $U:\ \mathbb{R}^{d}\to\mathbb{R}$. We assume that there
exist a topological space $(\mathsf{Z},\mathcal{B}(\mathsf{Z}))$, a
probability measure $\mu\in\mathscr{P}(\mathsf{Z},\mathcal{Z})$ and $u:\
\mathbb{R}^{d}\times\mathsf{Z}\to\mathbb{R}_{+}$ such that for any
$x\in\mathbb{R}^{d}$

$\textstyle{U(x)=\int_{\mathsf{Z}}u(x,z)\mathrm{d}\mu(z)\;.}$ (128)

For any $n\in\mathbb{N}$ we define
$U_{n}:\mathbb{R}^{d}\times\mathsf{Z}^{n}\to\mathbb{R}$ such that for any
$x\in\mathbb{R}^{d}$ and $z^{1:n}=\\{z_{i}\\}_{i=1}^{n}\in\mathsf{Z}^{n}$

$\textstyle{U_{n}(x,z^{1:n})=(1/n)\sum_{i=1}^{n}u(x,z_{i})\;.}$ (129)

For all $\varepsilon>0$, when it is well-defined we denote
$\mathrm{S}_{\varepsilon}:\
\mathsf{Z}^{n}\times\mathcal{B}(\mathbb{R}^{d})\to\left[0,1\right]$ the Markov
kernel such that for any $z^{1:n}\in\mathsf{Z}^{n}$ and
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ we have

$\textstyle{\updelta_{z^{1:n}}\mathrm{S}_{\varepsilon}(\mathsf{A})=\left.\int_{\mathsf{A}}\exp[-U_{n}(x,z^{1:n})/\varepsilon]\mathrm{d}x\middle/\int_{\mathbb{R}^{d}}\exp[-U_{n}(x,z^{1:n})/\varepsilon]\mathrm{d}x\right.\;.}$
(130)

Similarly, when it is well-defined, we denote $\mathrm{S}_{0}:\
\mathsf{Z}^{n}\times\mathcal{B}(\mathbb{R}^{d})\to\left[0,1\right]$ the Markov
kernel such that for any $z^{1:n}\in\mathsf{Z}^{n}$ and
$\mathsf{A}\in\mathcal{B}(\mathbb{R}^{d})$ we have

$\textstyle{\updelta_{z^{1:n}}\mathrm{S}_{0}(\mathsf{A})=\left.\int_{\mathsf{A}\cap\mathsf{C}_{n}(z^{1:n})}\det(\nabla_{x}^{2}U_{n}(x,z^{1:n}))^{-1}\mathrm{d}\mathcal{H}^{0}(x)\middle/\int_{\mathsf{C}_{n}(z^{1:n})}\det(\nabla_{x}^{2}U_{n}(x,z^{1:n}))^{-1}\mathrm{d}\mathcal{H}^{0}(x)\right.\;,}$
(131)

where $\mathsf{C}_{n}(z^{1:n})=\operatorname*{arg\,min}U_{n}(\cdot,z^{1:n})$.
We recall that for any $\upbeta>0$, $\sigma_{\upbeta}^{\star}$ is defined in
(50). We begin with the following proposition.

######  Proposition .

Let $n\in\mathbb{N}$ and assume H 3$(n)$ and H 4$(n)$. Then there exist $C\geq
0$ and $\upbeta,\bar{\varepsilon}>0$ such that for any
$\varepsilon\in\left(0,\bar{\varepsilon}\right)$

$\textstyle{\int_{\mathsf{Z}^{n}}\mathbf{W}_{1}(\updelta_{z^{1:n}}\mathrm{S}_{\varepsilon}^{n},\updelta_{z^{1:n}}\mathrm{S}_{0}^{n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})\leq
C(1+D_{n})(\varepsilon^{1/2}+\varepsilon^{-d/2}\int_{\mathsf{Z}^{1:n}}\exp[-c^{\star}(z^{1:n})/\varepsilon]\mathrm{d}\mu^{\otimes
n}(z^{1:n}))\;,}$ (132)

with
$D_{n}=\int_{\mathsf{Z}^{n}}\sigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})<+\infty$ and $C,\bar{\varepsilon},\upbeta$ independent from $n$.

###### Proof.

The proof of this result is a direct application of Section B.2 which is a
parametric version of Theorem 2. In order to apply Section B.2, we check that
H 5 is satisfied for $\mathsf{Z}\leftarrow\mathsf{Z}^{n}$ and $u\leftarrow
U_{n}$. We only need to check that there exists
$\mathtt{m}_{0},\upalpha_{0}>0$ and $R_{0}$ such that for any
$x\in\mathbb{R}^{d}$ with $\|x\|\geq R_{0}$ and $z^{1:n}\in\mathsf{Z}^{n}$,
$U_{n}(x,z^{1:n})\geq\mathtt{m}_{0}\left\|x\right\|^{\upalpha_{0}}$. We have
that for any $x\in\mathbb{R}^{d}$ and $z\in\mathsf{Z}$

$\displaystyle\textstyle{u(x,z)=u(0,z)+\int_{0}^{1}\langle\nabla_{x}u(tx,z),x\rangle\mathrm{d}t\geq-A-\mathtt{c}+(\mathtt{m}/2)\|x\|^{2}\;.}$
(133)

Hence, for any $x\in\mathbb{R}^{d}$ with $\|x\|\geq
2((A+\mathtt{c})/\mathtt{m})^{1/2}$ we have that for any
$z^{1:n}\in\mathsf{Z}^{n}$, $U_{n}(x,z^{1:n})\geq(\mathtt{m}/4)\|x\|^{2}$.
Hence, combining this result, H 3$(n)$ and H 4$(n)$, we can apply Section B.2
which states that for any $\varphi:\ \mathbb{R}^{d}\to\mathbb{R}$
$M_{1,\varphi}$-Lipschitz function with $M_{1,\varphi},C_{\varphi}\geq 0$ such
that for any $x\in\mathbb{R}^{d}$, $\left|\varphi(x)\right|\leq
C_{\varphi}\exp[C_{\varphi}\left\|x\right\|^{\upalpha}]$ then, there exist
$B_{2}\in\mathrm{C}(\mathbb{R}_{+},\mathbb{R}_{+})$ and $\upbeta>0$ such that

$\left|\updelta_{z^{1:n}}\mathrm{S}_{\varepsilon}[\varphi]-\updelta_{z^{1:n}}\mathrm{S}_{0}[\varphi]\right|\leq
B_{2}(C_{\varphi})(1+M_{0,\varphi}+M_{1,\varphi})(1+\sigma_{\upbeta}^{\star}(z^{1:n}))\\{\varepsilon^{1/2}+\varepsilon^{-d/2}\exp[-c^{\star}(z^{1:n})/\varepsilon]\\}\;,$
(134)

with $M_{0,\varphi}=\sup\\{|\varphi(x)|\,:\;x\in\mathsf{K}\\}$, $\mathsf{K}$,
$B_{2}$ and $\upbeta$ independent from $z$ and $B_{2}$ non-decreasing. The
rest of the proof is similar to the one of Section 2 upon replacing Theorem 1
by (134). ∎

The proof of Section 3.2.2 is then a direct application of [60, Proposition
3.3] and Section 4.3.1.

####  4.3.2 Proof of Section 3.2.2

In this section, we prove Section 3.2.2. We start by recalling a proposition
from [60, Proposition 3.5] about the uniform stability of the exponential
measure with potential $U_{n}$, see [8] for a definition of the uniform
stability.

######  Lemma .

Assume that H 3$(n)$ holds uniformly w.r.t $n$. Then for any $n\in\mathbb{N}$,
$\varepsilon>0$, $z_{0}^{1:n},z_{1}^{1:n}\in\mathsf{Z}^{n}$ which only differs
along one index, we have

$\textstyle{|\updelta_{z^{1:n}_{0}}\mathrm{S}_{\varepsilon}u(\cdot,z)-\updelta_{z^{1:n}_{1}}\mathrm{S}_{\varepsilon}u(\cdot,z)|\leq
4(\mathtt{M}^{2}(\mathtt{c}+d\varepsilon)/\mathtt{m}+B^{2})\mathtt{c_{LS}}/(n\varepsilon)\;,}$
(135)

where $\mathtt{c_{LS}}\geq 0$ is such that for any $z^{1:n}\in\mathsf{Z}^{n}$,
$\updelta_{z^{1:n}}\mathrm{S}_{\varepsilon}$ satisfies the logarithmic Sobolev
inequality with constant $\mathtt{c_{LS}}$, see [60, Proposition 3.2].

For completeness, we recall that a probability measure
$\nu\in\mathscr{P}(\mathbb{R}^{d})$ is said to satisfy the logarithmic Sobolev
inequality with constant $\mathtt{c_{LS}}$ if for any
$\pi\in\mathscr{P}(\mathbb{R}^{d})$ with positive density w.r.t $\nu$ given by
$g\in\mathrm{C}^{1}(\mathbb{R}^{d},\mathbb{R})$ we have

$\textstyle{\operatorname{KL}\left(\pi|\nu\right)\leq
2\mathtt{c_{LS}}\int_{\mathbb{R}^{d}}\|\nabla\log(g(x))\|^{2}\mathrm{d}\pi(x)\;.}$
(136)

We are now ready to show that the limiting measures are stable.

######  Proposition .

Assume that H 3$(n)$ and H 4$(n)$ hold uniformly w.r.t $n\in\mathbb{N}$. In
addition, assume that there exist
$C_{0},C_{\sigma},\alpha,\bar{\varepsilon}_{0}>0$ such that for any
$n\in\mathbb{N}$ and $\varepsilon\in(0,\bar{\varepsilon}_{0}]$

$\textstyle{\varepsilon^{-d/2}\int_{\mathsf{Z}^{n}}\exp[-c^{\star}(z^{1:n})/\varepsilon]\mathrm{d}\mu^{\otimes
n}(z^{1:n})\leq
C_{0}\varepsilon^{\alpha}\;,}\qquad\textstyle{\int_{\mathsf{Z}^{n}}\sigma_{\upbeta}^{\star}(z^{1:n})\mathrm{d}\mu^{\otimes
n}(z^{1:n})\leq C_{\sigma}\;.}$ (137)

Then there exists $n_{0}\in\mathbb{N}$ such that for any
$\eta\in\left(0,1\right)$, there exists $C\geq 0$ such that for any
$z\in\mathsf{Z}$, $n\in\mathbb{N}$ with $n\geq n_{0}$ and
$j\in\\{1,\dots,n\\}$, we have

$\textstyle{\int_{\mathsf{Z}^{n+1}}|\updelta_{z^{1:n}_{0}}\mathrm{S}_{0}u(\cdot,z)-\updelta_{z^{1:n}_{1}}\mathrm{S}_{0}u(\cdot,z)|\mathrm{d}\mu^{\otimes
n}(z_{0}^{1:n})\mathrm{d}\mu(z_{1,j})\leq Cn^{-\eta s/(1+\eta s)}\;,}$ (138)

where $s=\min(\alpha/2,1/4)$ and for any $z_{0}^{1:n}\in\mathsf{Z}^{n}$ we let
$z_{1}^{1:n}\in\mathsf{Z}^{n}$ with $z_{0,i}=z_{1,i}$ for any
$i\in\\{1,\dots,N\\}$ such that $i\neq j$.

###### Proof.

Let $\eta\in\left(0,1\right)$, $n\in\mathbb{N}$ with $n\geq n_{0}$ and
$n_{0}^{-1/(1+\eta
s)}<\bar{\varepsilon}=\min(\bar{\varepsilon}_{0},\bar{\varepsilon}_{1})$
(where $\bar{\varepsilon}_{1}$ is given by
$\bar{\varepsilon}_{1}\leftarrow\bar{\varepsilon}$ in Section 4.3.1) with
$s=\min(\alpha/2,1/4)$, $\varepsilon\in\left(0,\bar{\varepsilon}\right)$ and
$z_{0}^{1:n},z_{1}^{1:n},z\in\mathsf{Z}^{n}$ where there exists
$j\in\\{1,\dots,n\\}$ such that for any $i\in\\{1,\dots,n\\}$, $i\neq j$,
$z_{0,i}=z_{1,i}$. Using the triangle inequality we have for any
$\varepsilon>0$

$\displaystyle|\updelta_{z^{1:n}_{0}}\mathrm{S}_{0}u(\cdot,z)-\updelta_{z^{1:n}_{1}}\mathrm{S}_{0}u(\cdot,z)|\leq|\updelta_{z^{1:n}_{0}}\mathrm{S}_{0}u(\cdot,z)-\updelta_{z^{1:n}_{0}}\mathrm{S}_{\varepsilon}u(\cdot,z)|$
(139)
$\displaystyle\qquad+|\updelta_{z^{1:n}_{0}}\mathrm{S}_{\varepsilon}u(\cdot,z)-\updelta_{z^{1:n}_{1}}\mathrm{S}_{\varepsilon}u(\cdot,z)|+|\updelta_{z^{1:n}_{1}}\mathrm{S}_{\varepsilon}u(\cdot,z)-\updelta_{z^{1:n}_{1}}\mathrm{S}_{0}u(\cdot,z)|\;.$
(140)

Using [60, Lemma 3.5] and Section B.3, there exists $C_{0}\geq 0$ (independent