least $1-\zeta$ over $\mu_{z}^{\otimes n}$, we have

$\displaystyle G_{S}(T)\leq
2s\sqrt{\frac{M(\sigma_{1},\sigma_{2},d,\alpha)}{n}\int_{0}^{T}\mathds{E}_{U}\left[\left\|\widehat{F}_{S}(W_{t}^{S})\right\|^{2}\right]dt+\frac{\log(3/\zeta)+\Lambda}{n}}$

with $\Lambda=\text{{{KL}}}\left(\rho_{0}||\pi\right)$, and the noise mixing
constant $M(\sigma_{1},\sigma_{2},d)$ is given by:

$\displaystyle
M(\sigma_{1},\sigma_{2},d,\alpha):=\frac{1}{4\sigma^{2}+\frac{\sigma_{1}^{\alpha}\alpha
2^{\alpha}\Gamma\left(\frac{d+\alpha}{2}\right)R^{2-\alpha}}{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)d\Gamma\left(\frac{d}{2}\right)}}.$

Proof  The proof follows by the exact same computations than the proofs of
Corollary 8 and Theorem 10.

Based on the above theorem, we can make an interesting observation. The
constant $M(\sigma_{1},\sigma_{2},d,\alpha)$ clearly shows the relative
contribution of both noises. Therefore, we can argue that they have the same
contribution when the following condition is fulfilled:

$\displaystyle 4\sigma_{2}^{2}=\frac{\sigma_{1}^{\alpha}\alpha
2^{\alpha}\Gamma\left(\frac{d+\alpha}{2}\right)R^{2-\alpha}}{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)d\Gamma\left(\frac{d}{2}\right)}.$

Using the asymptotic developments discussed in Section 5, and proven in
Appendix D, in the limit $d\to\infty$, we can write this condition as:

$\displaystyle
4\sigma_{2}^{2}\underset{d\to\infty}{\sim}\frac{1}{R^{2-\alpha}}{P_{\alpha}}\frac{\sigma_{1}^{\alpha}d^{\frac{\alpha}{2}}}{d},$

where the pre-factor $P_{\alpha}$ has been defined and studied in Section 5.
It is now meaningful to write it as:

$\displaystyle(\sigma_{2}\sqrt{d_{0}})^{2}=\frac{1}{4P_{\alpha}}(\sigma_{1}\sqrt{d_{0}})^{\alpha},$
(33)

with $d_{0}:=d/(R^{2})$, the ”reduced dimension introduced in Section 5. This
is the equal noise contribution condition. What is particularly noticeable in
Equation 33 is that, as was the case in the study of the phase transition, in
Section 5, the meaningful quantities, for both noises, is (scale of the noise)
$\times$ $\sqrt{\textbf{dimension}}$.

###  C.8 Time-uniform bounds deduced from a generalized Poincaré inequality -
Omitted proofs of Section 4.3

In this section, we fix the convex function $\Phi$ to be
$\Phi(x)=\frac{1}{2}x^{2}$. In that case, the Bregman divergence becomes
symmetric and satisfies:

$\displaystyle D_{\Phi}(a,b)=D_{\Phi}(b,a)=\frac{1}{2}(a-b)^{2}.$

Combined with the logarithmic Sobolev inequality, given by Theorem 29, this
allows us to prove a time uniform bound on the chi-squared distance between
the prior and the posterior.

We denote, as above:

$\displaystyle
V_{S}(w):=\widehat{F}_{S}(w)+\frac{\eta}{2}\left\|w\right\|^{2},\quad\text{and:
}V(w):=\frac{\eta}{2}\left\|w\right\|^{2},$

with $\eta>0$ the regularization coefficient.

Before bounding the $\Phi$-entropy, we first need the following lemma. It is
close to remarks already made in (Gentil and Imbert, 2008) and (Tristani,
2013), for instance.

###### Lemma 49

In this setting, $\bar{u}_{\infty}$ is the density of an infinitely divisible
probability distribution, whose characteristic exponent is given by:

$\displaystyle\forall\xi\in{\mathds{R}^{d}},\leavevmode\nobreak\
\psi(\xi)=\frac{\sigma_{2}^{2}}{2\eta}\left\|\xi\right\|^{2}+\frac{\sigma_{1}^{\alpha}}{\alpha\eta}\left\|\xi\right\|^{\alpha}.$

Proof  Let us first recall some notations. $\pi$ is the (prior) density
distribution whose density with respect to the Lebesgue measure is
$\bar{u}_{\infty}$. The existence of such the steady state $\bar{u}_{\infty}$
was obtained in (Gentil and Imbert, 2008; Tristani, 2013). Moreover, Gentil
and Imbert (2008) showed that $\pi$ is infinitely divisible, therefore, it
makes sense to introduce its characteristic exponent, denoted $\psi$, and
defined by:

$\displaystyle\forall\xi\in{\mathds{R}^{d}},\leavevmode\nobreak\
\mathds{E}_{X\sim\pi}\left[e^{i\xi\cdot X}\right]=e^{-\psi(\xi)}.$

Following computations of Gentil and Imbert (2008); Tristani (2013). We
quickly show how the expression of the characteristic exponent can easily be
obtained in our particular case. First, because of the symmetry of Equation 7,
both $\bar{u}_{\infty}$ and $\psi$ must be symmetric (i.e. even functions).
Therefore, if we define the Fourier transform by:

$\displaystyle\mathcal{F}(u)(\xi)=\int_{\mathds{R}^{d}}e^{-iw\cdot\xi}u(w)dw,$

then we have:

$\displaystyle\mathcal{F}(\bar{u}_{\infty})(\xi)=e^{-\psi(x)}.$

We can take, at least formally (Tristani, 2013), the Fourier transform of
Equation 7 and write that, in the sense of distributions:

$\displaystyle\mathcal{F}(\bar{u}_{\infty})(\xi)\left(-\sigma_{1}^{\alpha}\left\|\xi\right\|^{\alpha}-\sigma_{2}^{2}\left\|\xi\right\|^{2}\right)+\eta\mathcal{F}(\nabla\cdot(w\bar{u}_{\infty}(w)))(\xi)=0.$

In the above, we used the expression of Fourier transform for the Laplacian
and the fractional Laplacian. Using properties of the Fourier transform, this
leads to:

$\displaystyle\mathcal{F}(\bar{u}_{\infty})(\xi)\left(-\sigma_{1}^{\alpha}\left\|\xi\right\|^{\alpha}-\sigma_{2}^{2}\left\|\xi\right\|^{2}\right)-\eta\nabla\mathcal{F}(\bar{u}_{\infty})(\xi).$

Therefore:

$\displaystyle\eta\xi\cdot\nabla\psi(\xi)=\sigma_{1}^{\alpha}\left\|\xi\right\|^{\alpha}+\sigma_{2}^{2}\left\|\xi\right\|^{2}.$

Given that $\psi$ is symmetric, we can write it as
$P(\left\|\xi\right\|^{2})$, we can then rewrite the previous equation as (for
$a>0$):

$\displaystyle
P^{\prime}(a)=\frac{\sigma_{1}^{\alpha}a^{\alpha/2-1}+\sigma_{2}^{2}}{2\eta}.$

Integrating, and noting that we have $P(0)=0$, leads to the result.

In this setting, the $\Phi$-entropy inequality, i.e. Theorem 29, becomes the
following statement.

######  Corollary 50 (Generalized Poincaré’s inequality)

Let $\mu$ be an infinitely divisible law on ${\mathds{R}^{d}}$, with
associated triplet denoted $(b,Q,\nu)$, then, for every smooth enough function
$v$, we have:

$\displaystyle\frac{1}{2}\int v^{2}d\mu-\frac{1}{2}\left(\int
vd\mu\right)^{2}\leq\int\nabla v\cdot Q\nabla vd\mu+\iint
D_{\Phi}(v(x),v(x+z))d\nu(z)d\mu(x).$

where we used the symmetry of the Bregman divergence, for
$\Phi(x)=\frac{1}{2}x^{2}$, to revert the arguments of this divergence,
appearing in Theorem 29.

If $v$ is chosen to be the Radon-Nykodym derivative of another measure $\nu$,
with respect to $\mu$, we recognize the chi-squared distance, defined by:

$\displaystyle\chi^{2}\left(\nu\right|\left|\mu\right):=\int\left(\frac{d\nu}{d\mu}\right)^{2}d\mu-1.$
(34)

We also remind the definition of Renyi entropies, for $\beta>1$:

$\displaystyle\text{{{D}}}_{\alpha}\left(\nu||\mu\right)=\frac{1}{\beta-1}\log\int\left(\frac{d\nu}{d\mu}\right)^{\beta}d\mu.$
(35)

Note that, by convention, we also set
$\text{{{D}}}_{1}\left(\nu||\mu\right)=\text{{{KL}}}\left(\nu||\mu\right)$,
see (van Erven and Harremoës, 2014) for an extensive review of the properties
of those entropies.

The following inequalities are clear, the first one being proven in (van Erven
and Harremoës, 2014), it is a direct consequence of Jensen’s inequality:

$\displaystyle\text{{{KL}}}\left(\nu||\mu\right)\leq\text{{{D}}}_{2}\left(\nu||\mu\right)=\log(\chi^{2}\left(\nu\right|\left|\mu\right)+1)\leq\chi^{2}\left(\nu\right|\left|\mu\right).$
(36)

####  C.8.1 Warmu -up: the case $\sigma_{2}>0$

In order to mimic the resoning of Sections 4.1 and 4.2, we first handle the
case where $\sigma_{2}>0$. This is an additional theoretical result.

###### Theorem 51

We make Assumptions 5 and 6, accordingly to this choice of convex function.
Then, with probability at least $1-\zeta$ over $S\sim\mu_{z}^{\otimes n}$ and
$w\sim\rho_{T}^{S}$, we have, for any $\lambda>0$:

$\displaystyle G_{S}(w)\leq
2s\sqrt{\frac{1}{n\sigma_{2}}\bar{I}(T,S)+\frac{2e^{-\eta
T}\Lambda+\log\frac{24}{\zeta^{3}}}{n}},$

with $\Lambda=\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(\rho_{0}\right)$, and:

$\displaystyle\bar{I}(T,S):=\int_{0}^{T}e^{-\eta(T-t)}\mathds{E}_{\pi}\left[(v_{t}^{S})^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\right]dt.$

Proof  As before, we fix $t$ and $S$ and ease the notations by denoting $u$
instead of $u_{t}^{S}$, and similarly for $v$. For our particular choice of
function $\Phi$,by Lemma 7, the entropy flow is equal to:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)=-\sigma_{2}^{2}\int\left\|\nabla
v\right\|^{2}\bar{u}_{\infty}$ $\displaystyle-
C_{\alpha,d}\sigma_{1}^{\alpha}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx$ $\displaystyle+\int
v\bar{u}_{\infty}\nabla v\cdot\nabla\widehat{F}_{S}.$

By Young’s inequality, we have, for any $C>0$ and $\gamma\in(0,1]$:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)\leq-\left(\sigma_{2}^{2}-\frac{C}{2}\right)\int\left\|\nabla
v\right\|^{2}\bar{u}_{\infty}$ $\displaystyle-\gamma
C_{\alpha,d}\sigma_{1}^{\alpha}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx$
$\displaystyle+\frac{1}{2C}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}.$

Thanks to Lemma 49, we know that the Levy triplet of $\bar{u}_{\infty}$ is
given by
$(0,\sigma_{2}^{2}/(\eta),(C_{\alpha,d}\sigma_{1}^{\alpha}/(\alpha\eta))\nu_{\alpha})$.
Therefore, the Poincaré’s inequality of Corollary 50 is given by:

$\displaystyle\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)\leq\frac{\sigma_{2}^{2}}{2\eta}\int\left\|\nabla
v\right\|^{2}\bar{u}_{\infty}+\frac{C_{\alpha,d}\sigma_{1}^{\alpha}}{\alpha\eta}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx.$

Therefore, we have:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\gamma\alpha\eta\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)-\left(\sigma_{2}^{2}-\frac{C}{2}-\frac{\alpha\gamma\sigma_{2}^{2}}{2}\right)\int\left\|\nabla
v\right\|^{2}\bar{u}_{\infty}+\frac{1}{2C}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}.$ (37)

We make the choice $\gamma=1/\alpha$ and $C=\sigma_{2}^{2}$, this gives:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)\leq-\eta\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)+\frac{1}{2\sigma_{2}^{2}}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}.$

Solving this differential inequality implies that, with
$\Lambda:=\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(\rho_{0}\right)$ and
$T>0$,

$\displaystyle\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v_{T}^{S}\right)\leq
e^{-\eta T}\Lambda+\frac{1}{2\sigma_{2}^{2}}\bar{I}(T,S),$

with:

$\displaystyle\bar{I}(T,S):=\int_{0}^{T}e^{-\eta(T-t)}\mathds{E}_{\pi}\left[(v_{t}^{S})^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\right]dt.$

From Equation (34), this implies that:

$\displaystyle\frac{1}{2}\chi^{2}\left(\rho_{T}^{S}\right|\left|\pi\right)\leq
e^{-\eta
T}\Lambda+\frac{1}{2\sigma_{2}^{2}}\int_{0}^{T}e^{-\eta(T-t)}I(t,S)dt.$

We conclude by applying Theorem 34 and noting that, in our notations, we have:

$\displaystyle\text{{{D}}}_{2}\left(\rho_{T}^{S}||\pi\right)\leq
2\chi^{2}\left(\rho_{S}^{t}\right|\left|\pi\right)=2\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right).$

In order to make the above theorem more interpretable, we present, as an
additional theoretical result, the following corollary. It states that, if the
loss is Lipschitz in $w$ and if the regularization coefficient is big enough,
then we have a simpler time-uniform bound. The main interest for the following
result is that it it fully comparable with our other results. This is an
additional result, it was not presented in the main part of the document.

###### Corollary 52

We make the same assumptions than in Theorem 51 and further assume that the
loss $f(w,z)$ is $L_{f}$-Lipschitz in $w$, uniformly with respect to $z$.
Then, if

$\displaystyle a:=\eta-\frac{L_{f}^{2}}{2\sigma_{2}^{2}}>0,$ (39)

then, with probability at least $1-\zeta$ over $S\sim\mu_{z}^{\otimes n}$ and
$w\sim\rho_{T}^{S}$, we have, for any $\lambda>0$:

$\displaystyle
G_{S}(w)\leq\frac{2s}{\sqrt{n}}\left\\{\frac{L_{f}^{2}}{2\sigma_{2}^{2}}\frac{1-e^{-aT}}{a}+\frac{3}{2}\log\left(\frac{2}{\zeta}\right)+\frac{\Lambda}{e^{\eta
T}}\right\\}^{\frac{1}{2}}.$

with $\Lambda=\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(\rho_{0}\right)$.

Proof

We follow the exact same step than the proof of Theorem 51, up to Equation
(37). We make the same choices for $\gamma$ and $C$ and obtain:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\eta\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)+\frac{1}{2\sigma_{2}^{2}}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}.$

By the Lipschitz assumption, this is:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\eta\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)+\frac{L_{f}^{2}}{2\sigma_{2}^{2}}\int
v^{2}\bar{u}_{\infty}.$
$\displaystyle=-\left(\eta-\frac{L_{f}^{2}}{2\sigma_{2}^{2}}\right)\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)+\frac{L_{f}^{2}}{2\sigma_{2}^{2}}.$

The result is deduced by the exact same reasoning than the last steps of the
proof of Theorem 51. By applying Theorem 1, this gives that, with probability
at least $1-\zeta$ over $S\sim\mu_{z}^{\otimes n}$ and $w\sim\rho_{T}^{S}$, we
have, for any $\lambda>0$:

$\displaystyle
G_{S}(w)\leq\frac{1}{\lambda}\left(\frac{L_{f}^{2}}{2\sigma_{2}^{2}}\frac{1-e^{-aT}}{a}+\frac{3}{2}\log(2/\zeta)+e^{-\eta
T}\Lambda\right)+\lambda\frac{s^{2}}{n}.$

with $\Lambda=\text{{{KL}}}\left(\rho_{0}||\bar{u}_{\infty}\right)$.
Therefore, we can optimize the value of the parameter $\lambda>0$ to get the
result.

####  C.8.2 Time uniform bounds in the case $\sigma_{2}=0$ \- Proof of Theorem
11

In the previous subsection, we used heavily the presence of a non-trivial
Brownian part in the operator $I$. However, we face the same problem that we
had with Theorem 51: while providing a generalization bound for multifractal
dynamics, which is our main goal, this theorem is not very informative about
the impact of the tail-index $\alpha$ on the generalization performance. This
is why, in this section, we focus, like in Section 4.2, on the case
$\sigma_{2}=0$. This follows the steps presented in Section 4.2, with a change
in the choice of the convex function $\Phi$.

To overcome this issue, we show how it is possible to derive a bound, even
without the Brownian part, by using a reasoning similar to Section C.3. Under
Assumption 5, we can apply the exact same reasoning with the choice of
function: $\Phi(x)=x^{2}/2$.

With this change in the convex function $\Phi$, the relevant quantity is no
more the Fisher information, but the following $L^{2}$-norm on the gradient of
$v$, this is the term that appears in Poincaré’s inequality:

$\displaystyle\mathbf{G}_{v}:=\int\left\|\nabla
v\right\|^{2}\bar{u}_{\infty}dx.$ (40)

This quantity, $\mathbf{G}_{v}$, correspond to the $\Phi$-information term,
$I_{\Phi}(v)$, introduced in Lemma 7, for the particular choice of convex
function that we make in this section, namely $\Phi(x)=x^{2}/2$

The following lemma is proven by the exact same lines than Lemma 37, only the
function $\Phi$, and the corresponding $\Phi$-regularity assumption, changes.

###### Lemma 53

Assume that $v$ is $\Phi$-regular, for $\Phi(x)=x^{2}/2$. Then, we have the
following representation:

$\displaystyle\iint
D_{\Phi}(v(x),v(x+z))d\nu_{\alpha}(z)\bar{u}_{\infty}(x)dx=\frac{\sigma_{d-1}}{2d}\int_{0}^{\infty}G_{v}(r)\frac{dr}{r^{\alpha-1}},$

where the function $G_{v}$ is non-negative and defined by:

$\displaystyle
G_{v}(r):=\frac{2d}{r^{2}\sigma_{d-1}}\int_{0}^{r}\int_{s}^{r}\int_{{\mathds{R}^{d}}}\int_{\mathds{S}^{d-1}}\theta\cdot\nabla
v(x+s\theta)\theta\cdot\nabla v(x+u\theta)\bar{u}_{\infty}(x)d\theta dxduds.$

This function can be continuously extended at $0$ by:

$\displaystyle G_{v}(0)=\mathbf{G}_{v}.$

This function, $G_{v}:[0,\infty)\longrightarrow[0,\infty)$, corresponds to the
function $J_{\Phi,v}$, introduced in the main part of the paper, for the
particular choice of convex function $\Phi(x)=\left\|x\right\|^{2}/2$.

The following assumption is the equivalent of Assumption 9 for the function
$G_{v}$, instead of $J_{v}$.

###### Assumption 54

There exists an absolute constant $R>0$ such that, for $\mu_{z}^{\otimes
n}$-almost all $S$ and all $t\geq 0$, we have:

$\displaystyle\forall r\in[0,R],\leavevmode\nobreak\
G_{v}(r)\geq\frac{1}{2}G_{v}(0).$

This assumption is exactly a reformulation of 9, rewritten with
$\Phi(x)=\frac{1}{2}x^{2}$. Using the denominations of Section 4.3, it is
9-$\Phi_{2}$.

We can now prove Theorem 11, which is a time-uniform generalization bound
obtained in the pure heavy-tailed case ($\sigma_{1}=0$).

See 11

Proof  We start, again, with the expression of the entropy flow.

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle=-C_{\alpha,d}\sigma_{1}^{\alpha}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx+\int
v\bar{u}_{\infty}\nabla v\cdot\nabla\widehat{F}_{S}.$

We split in two the Bregman integral term and apply the generalized Poincaré
inéquality, along with Young’s inequality on the last term, for any $C>0$:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\frac{\alpha\eta}{2}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)-\frac{C_{\alpha,d}\sigma_{1}^{\alpha}}{2}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx+\int
v\bar{u}_{\infty}\nabla v\cdot\nabla\widehat{F}_{S}.$
$\displaystyle\leq-\frac{\alpha\eta}{2}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)-\frac{C_{\alpha,d}\sigma_{1}^{\alpha}}{2}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx$
$\displaystyle\quad\quad+\frac{C}{2}\mathbf{G}_{v}+\frac{1}{2C}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}dx.$

We now use Lemma 53 and Assumption 54, this gives:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\frac{\alpha\eta}{2}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)-\frac{C_{\alpha,d}\sigma_{1}^{\alpha}}{2}\frac{\sigma_{d-1}}{2d}\int_{0}^{R}G_{v}(r)\frac{dr}{r^{\alpha-1}}+\frac{C}{2}\mathbf{G}_{v}+\frac{1}{2C}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}dx$
$\displaystyle\leq-\frac{\alpha\eta}{2}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)-\frac{C_{\alpha,d}\sigma_{1}^{\alpha}\sigma_{1}^{\alpha}R^{2-\alpha}}{8d(2-\alpha)}\mathbf{G}_{v}+\frac{C}{2}\mathbf{G}_{v}+\frac{1}{2C}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}dx.$

Therefore, we make the choice:

$\displaystyle
C:=\frac{C_{\alpha,d}\sigma_{1}^{\alpha}\sigma_{1}^{\alpha}R^{2-\alpha}}{4d(2-\alpha)},$

and we get:

$\displaystyle\frac{d}{dt}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)$
$\displaystyle\leq-\frac{\alpha\eta}{2}\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)+\frac{2d(2-\alpha)}{C_{\alpha,d}\sigma_{1}^{\alpha}\sigma_{1}^{\alpha}R^{2-\alpha}}\int
v^{2}\left\|\nabla\widehat{F}_{S}\right\|^{2}\bar{u}_{\infty}dx.$

We introduce the same constant as the one introduced in Theorem 10, namely:

$\displaystyle
K_{\alpha,d}=\frac{d(2-\alpha)}{C_{\alpha,d}\sigma_{d-1}R^{2-\alpha}}.$

Therefore, solving the above differential inequality gives:

$\displaystyle\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)\leq
e^{-\frac{\alpha\eta
T}{2}}\Lambda+\frac{2K_{\alpha,d}}{\sigma_{1}^{\alpha}}\bar{I}(T,S),$

where $\bar{I}(T,S)$ and $\Lambda$ are defined in the same way than in Theorem
51.

We conclude by applying Theorem 34 and noting that
$\text{{{D}}}_{2}\left(\rho_{S}^{T}||\pi\right)\leq
2\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v^{S}_{T}\right)$.

####  C.8.3 Discussion of potential extensions to the discrete case

##  Appendix D Asymptotics of the constants - Omitted proofs of Section 5

This section is devoting to studying the limit behavior of the constants
introduced in our generalization bounds, in terms of $\alpha$ and $d$. This
corresponds to the proof of the theoretical results of Section 5.

We first study the asymptotic behavior of the constant when $d\to\infty$.

See 12

Proof  By the definitions of Sections 4.2 and 5, we have:

$\displaystyle\bar{K}_{\alpha,d}=\frac{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)}{\alpha
2^{\alpha}}\frac{d\Gamma\left(\frac{d}{2}\right)}{\Gamma\left(\frac{d+\alpha}{2}\right)}.$

By Lemma 31, we have:

$\displaystyle\frac{d\Gamma\left(\frac{d}{2}\right)}{\Gamma\left(\frac{d+\alpha}{2}\right)}$
$\displaystyle\underset{d\to\infty}{\sim}\frac{d\Gamma\left(\frac{d}{2}\right)}{\Gamma\left(\frac{d}{2}\right)\left(\frac{d}{2}\right)^{\alpha/2}}$
$\displaystyle=2^{\alpha/2}d^{1-\frac{\alpha}{2}},$

from which we immediately deduce the result.

We now prove Lemma 13, which study the low-tail limit of our generalization
bounds for heavy-tailed dynamics.

See 13

Proof  By the definitions of Sections 4.2 and 5, we have:

$\displaystyle\bar{K}_{\alpha,d}=\frac{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)}{\alpha
2^{\alpha}}\frac{d\Gamma\left(\frac{d}{2}\right)}{\Gamma\left(\frac{d+\alpha}{2}\right)}.$

We have $\alpha\in[1,2)$, therefore, we know that
$\frac{\alpha}{2}\notin\mathds{Z}$. We can then apply Euler’s reflection
formula, Lemma 30, and get that:

$\displaystyle\Gamma\left(1-\frac{\alpha}{2}\right)\Gamma\left(\frac{\alpha}{2}\right)$
$\displaystyle=\frac{\pi}{\sin\left(\frac{\pi\alpha}{2}\right)}$
$\displaystyle=\frac{\pi}{\sin\left(\pi-\frac{\pi\alpha}{2}\right)}$
$\displaystyle\underset{\alpha\to 2^{-}}{\sim}\frac{1}{1-\frac{\alpha}{2}}.$

On the other hand, by continuity of the $\Gamma$ function and using the
identity $\Gamma(x+1)=x\Gamma(x)$, we also have:

$\displaystyle\Gamma\left(\frac{d+\alpha}{2}\right)\underset{\alpha\to
2^{-}}{\sim}\Gamma\left(\frac{d}{2}+1\right)=\frac{d}{2}\Gamma\left(\frac{d}{2}\right).$

Putting everything together, and using that $\Gamma(1)=1$, we get that:

$\displaystyle\bar{K}_{\alpha,d}\underset{\alpha\to 2^{-}}{\sim}\frac{1}{2}.$

As $K_{\alpha,d}=R^{\alpha-2}\bar{K}_{\alpha,d}$, with $R>0$, this implies the
result.

Finally, the next lemma presents the main properties of the function
$\alpha\longmapsto P_{\alpha}$, introduced in Section 5. As a reminder, we
have:

$\displaystyle
P_{\alpha}:=\frac{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)}{\alpha
2^{\alpha/2}}$

###### Lemma 55

The function $\alpha\longmapsto P_{\alpha}$ is, on the interval $[1,2)$,
continuous, decreasing and satisfies:

$\displaystyle\forall\alpha\in[1,2),\leavevmode\nobreak\ \frac{1}{2}\leq
P_{\alpha}\leq\sqrt{\frac{\pi}{2}}.$

Proof  The continuity follows from the continuity of the $\Gamma$ function on
$(0,\frac{1}{2}]$. Moreover, we have, using the particular values of the
$\Gamma$ function recalled in A.4:

$\displaystyle
P_{1}=\frac{\Gamma\left(\frac{1}{2}\right)}{\sqrt{2}}=\frac{\sqrt{\pi}}{\sqrt{2}}.$

Using the proof of the Lemma 13, we also have:

$\displaystyle P_{\alpha}\underset{\alpha\to
2^{-}}{\longrightarrow}\frac{1}{2}.$

Now we fix $\alpha\in(1,2)$, we use Euler’s reflection formula, Lemma 30, and
the formula $\Gamma(1+z)=z\Gamma(z)$ to get:

$\displaystyle P_{\alpha}$
$\displaystyle=\frac{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)}{\alpha
2^{\alpha/2}}$ $\displaystyle=\frac{2-\alpha}{\alpha
2^{\alpha/2}}\frac{\pi}{\sin\left(\frac{\pi\alpha}{2}\right)\Gamma\left(\frac{\alpha}{2}\right)}$
$\displaystyle=\frac{\pi\left(1-\frac{\alpha}{2}\right)}{\sin\left(\frac{\pi\alpha}{2}\right)}\frac{1}{2^{\alpha/2}\Gamma\left(1+\frac{\alpha}{2}\right)}$

From the general properties of the $\Gamma$ function, we know that
$\alpha\longmapsto\Gamma\left(1+\frac{\alpha}{2}\right)$ is increasing on
$(1,2)$, and therefore it is enough to show that the function

$\displaystyle
g(\alpha)=\frac{2-\alpha}{\sin\left(\frac{\pi\alpha}{2}\right)},$

is decreasing, as both $\Gamma\left(1+\frac{\alpha}{2}\right)$ and $g$ are
positive on $(1,2)$. A quick calculation reveals that:

$\displaystyle\sin^{2}\left(\frac{\pi\alpha}{2}\right)g^{\prime}(\alpha)=-\cos\left(\frac{\pi\alpha}{2}\right)\left\\{\tan\left(\pi\left(\frac{\alpha}{2}-1\right)\right)-\pi\left(1-\frac{\alpha}{2}\right)\right\\},$

which implies the desired result, by noting that $\tan(x)\leq x$ on
$(-\pi/2,0)$.

The function $\alpha\longmapsto P_{\alpha}$ is represented on Figure 4. This
plot confirms the calculations of Lemma 55.

Figure 4: Graphical representation of the function $\alpha\longmapsto
P_{\alpha}$. The leftmost value is $P_{1}=\sqrt{\frac{\pi}{2}}\simeq 1.2533$
and the right limit is $P_{2^{-}}=\frac{1}{2}$.

###  D.1 Precisions on the low noise-regime

In Section 5, we concluded the existence of two regimes predicted by our
bounds. We first defined the low noise regime by the following condition:

$\displaystyle\frac{\sigma_{1}\sqrt{d}}{R}<1.$

However, the pre-factor $\alpha\longmapsto P_{\alpha}$ is decreasing.
Therefore, we need to take it into account to accurately describe the regime
where the generalization error is increasing with $\alpha$. This happens when
we have:

$\displaystyle\frac{P_{1}R}{\sigma_{1}\sqrt{d}}<\frac{P_{2}R^{2}}{d\sigma_{1}^{2}}.$

Using Lemma 55, we get the condition that was mentioned briefly in Section 5:

$\displaystyle\frac{\sigma_{1}\sqrt{d}}{R}<\frac{1}{\sqrt{2\pi}}.$

###  D.2 Comparison with the constants appearing in other works

Figure 5: Comparison of the dimension dependence of our bounds and that of Raj
et al. (2023b), under the assumption that $\ell$ is Lipschitz continuous, for
both bounds.

In(Raj et al., 2023b), the authors prove expected generalization bounds for
heavy-tail dynamics. The constant appearing in their bounds has a complex
dependence on various constants defined in their assumptions, but we can sum
up the dependence in $\alpha$ and $d$ as follows (by Applying Euler’s
reflection formula to their Lemma $7$):

$\displaystyle
R_{\alpha,d}=\mathcal{O}\left(1+\frac{\sqrt{d}\Gamma\left(\frac{\alpha+d}{2}\right)}{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)\Gamma\left(\frac{d}{2}\right)}\right)$

By the proof of Lemma 12 and 13, we see that:

$\displaystyle R_{\alpha,d}=\mathcal{O}\left(d^{\frac{1+\alpha}{2}}\right).$

It is to be noted that our bound has a rate in $n$ of $1/2$, i.e. our bound is
proportional to $1/\sqrt{n}$, while the bound of Raj et al. (2023b) has a rate
of $1$, i.e. the bound is proportional to $1/n$. If we denote by $\Xi$ the
ratio of the rate in $d$ to the rate in $n$ of the bound, we find (under a
Lipschitz assumption for both bounds):

$\displaystyle\Xi^{\text{(Ours)}}=1-\frac{\alpha}{2},\quad\Xi^{\text{\cite[citep]{(\@@bibref{AuthorsPhrase1Year}{raj_algorithmic_2023}{\@@citephrase{,
}}{})}}}=\frac{1+\alpha}{2}.$ (41)

The meaning of the coefficient $\Xi$ is that, in an overparameterized regime
where $d>n$, the bound is non-vacuous only if $\Xi<1$. We graphically
represented this coefficient in Figure 5.

##  Appendix E Discussion of the time dependence of the bounds

While Theorem 10 is the first high probability generalization bound for heavy-
tailed dynamics, one may notice the time dependence of the bound, coming from
the integral term, i.e.

$\displaystyle\int_{0}^{T}\mathds{E}_{U}\left[\widehat{F}_{S}(W_{t})\right]dt.$
(42)

Because of this term, as explained in Section 5, the light-tail limit of this
bound leads to an informal bound of the form:

$\displaystyle G_{S}(W_{t})\underset{\alpha\to
2^{-}}{\lesssim}\sqrt{\frac{1}{n\sigma^{2}}\int_{0}^{T}\mathds{E}_{U}\left[\widehat{F}_{S}(W_{t})\right]dt},$

which is worse than the existing bounds on Langevin dynamics Mou et al.
(2017); Li et al. (2020); Farghly and Rebeschini (2021), which achieve time-
independent bounds. More explicitly, Mou et al. (2017) obtained a similar
bound, but with an additional exponential time-decay in the integral term,
i.e., informally,

$\displaystyle\int_{0}^{T}e^{-a(T-t)}\mathds{E}_{U}\left[\widehat{F}_{S}(W_{t})\right]dt,$

where $a>0$ is a constant dependent of the problem.

To the best of our knowledge, there is no obvious theoretical argument
affirming that the limit $\alpha\to 2^{-}$ should exactly recover the bounds
for Langevin dynamics, while it is quite clear that this limit should not be
infinite, as in Raj et al. (2023b). That being said, it is worth discussing
why this time dependence happens in our bound, how it could be improved, and,
most importantly, why a time independent bound is currently beyond the reach
of the theory.

Section 4.3 presents one step in the direction of time-uniform bounds.
However, the integral term appearing in Theorem 11 is not satisfactory,
because of its lack of interpretability, compared to Equation 42.

On the other hand, a more detailed analysis of the results of Mou et al.
(2017); Li et al. (2020) shows that time-independence is a direct consequence
of the use of a logarithmic Sobolev inequality (LSI, see Section A.3). If we
translate these arguments in our setting, such an inequality would need to be
satisfied by the prior distribution $\pi$. Therefore, it is natural to ask,
whether we could apply Theorem 29 in our proofs.

A similar computation has been proposed by Gentil and Imbert (2008), in their
study of the rate of convergence to equilibrium of an equation of the form of
Equation 6. However, we argue that we cannot apply this reasoning in our
setting. Let’s explain it briefly: in the proof of Theorem 10, the crucial
term that appears in the computation of the entropy flow is what we called the
Bregman integral, it is given by:

$\displaystyle B_{\Phi}^{\alpha}(v)=C_{\alpha,d}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}d\nu_{\alpha}(z)dx.$

On the other hand, Theorem 29 uses the following term, if we apply it to $\pi$
(in a similar way to how it is done in Section C.8.1):

$\displaystyle\iint D_{\Phi}(v(x+z),v(x))\bar{u}_{\infty}d\nu_{\alpha}(z)dx.$

Unfortunately, the Bregman divergence is not commutative in general, and, to
the best of our knowledge, there is no obvious way of linking the two
integrals appearing above. This discussion is the motivation behind Section
4.3, where we applied Theorem 29 and Lemma 7 to the case where the Bregman
divergence becomes symmetric, i.e. when $\Phi(x)\propto x^{2}$.

Therefore, we conclude that we do not have the LSI that we would need to make
our bounds time-independent. Let’s end this short discussion by writing down
the result that we would obtain with such an inequality.

###### Theorem 56

We consider the same setting than in Theorem 10, but we additionally assume
that the prior $\pi$ satisfies the following LSI, with reversed Bregman
divergence compared to Theorem 29, for all $v$ smooth enough:

$\displaystyle\text{Ent}_{\bar{u}_{\infty}}^{\Phi}\left(v\right)\leq\frac{C_{\alpha,d}\sigma_{1}^{\alpha}}{\alpha\eta}\iint
D_{\Phi}(v(x),v(x+z))\bar{u}_{\infty}(x)d\nu_{\alpha}(z)dx.$

Then, with probability at least $1-\zeta$ over $\mu_{z}^{\otimes n}$, we have:

$\displaystyle\mathds{E}_{U}\left[G_{S}(W_{T}^{S})\right]\leq\sqrt{\frac{2K_{\alpha,d}}{\sigma_{1}^{\alpha}}\int_{0}^{T}e^{-\frac{\alpha\eta}{2}(T-t)}\mathds{E}_{U}\left[\left\|\nabla\widehat{F}_{S}(W^{S}_{t})\right\|^{2}\right]dt+\frac{\Lambda+\log(3/\zeta)}{n}}$

with $\Lambda=\text{{{KL}}}\left(\rho_{0}||\bar{u}_{\infty}\right)$ and:

$\displaystyle
K_{\alpha,d}=\frac{(2-\alpha)\Gamma\left(1-\frac{\alpha}{2}\right)d\Gamma\left(\frac{d}{2}\right)}{\alpha
2^{\alpha}\Gamma\left(\frac{d+\alpha}{2}\right)R^{2-\alpha}},$ (43)

##  Appendix F Additional experimental details

###  F.1 Hyperparameters details

In this section, we give a list of the exact hyperparameters used to obtain
our main experiments. In general, those hyperparameters were chosen to make
our experimental setting as close as possible to our theoretical setting and
assumptions.

Linear model trained on MNIST (Figures 8 and 9) : We use a linear predictor,
without bias, trained with a (multivariate cross-entropy loss). All $10$
classes of the MNIST dataset were used but we randomly subsample $10\%$ of the
training set, to lower the computational cost of the experiments. We simulate
Equation 16 with $T=5.10^{3}$, $\gamma=10^{-2}$, $\eta=10^{-3}$. The last
$2000$ iterations were used to estimate the accuracy error, as described in
Section F.2. During those experiments, we let $\alpha$ vary in $[1.6,2]$,
using a linear scale of $10$ values. The parameter $\sigma$ varies in a
logarithmic scale of $10$ values, such that $\sigma\sqrt{d}\in[0.5,40]$, where
$d$ is the number of parameters in the model, this is inspired by the analysis
of Section 5. This range was chosen to ensure both that the training is stable
enough and that the heavy tail of the noise has an impact on the accuracy
(i.e. $\sigma$ not too small).

$2$-layers FCN trained on MNIST: This experiment was used to produce Figures
1, 3 and 2. We use a $2$-layer neural network with ReLU activation, without
bias, trained with a (multivariate cross-entropy loss). All $10$ classes of
the MNIST dataset were used but we randomly subsample $10\%$ of the training
set, to lower the computational cost of the experiments. We simulate Equation
16 with $T=10^{4}$, $\gamma=10^{-2}$, $\eta=10^{-3}$. The last $2000$
iterations were used to estimate the accuracy error, as described in Section
F.2. During those experiments, we let $\alpha$ vary in $[1.6,2]$, using a
linear scale of $10$ values. The width of the network varies in a linear scale
of $10$ values, between $40$ and $200$.

FashionMNIST experiments: The same model was also used on the FashionMNIST
dataset (a $10\%$ subsample of it), to obtain Figures 11, 10(a) and 10(b).

$5$-layers FCN trained on MNIST: We use a $5$-layer neural network with ReLU
activation, without bias, trained with a (multivariate cross-entropy loss).
$100\%$ of the MNIST dataset were used. We experimented using a batch-size in
this case, i.e. Equation 16 is replaced by:

$\displaystyle\hat{W}^{S}_{k+1}=\hat{W}^{S}_{k}-\frac{\gamma}{b}\sum_{j=1}^{b}\nabla
f(\hat{W}^{S}_{k},z_{i_{j}})-\eta\gamma\hat{W}^{S}_{k}+\gamma^{\frac{1}{\alpha}}\sigma_{1}L_{1}^{\alpha},$
(44)

where $b\in\mathds{N}^{\star}$ is the batch size and $(i_{1},\dots,i_{b})$ are
random indices drawn at each iteration. We simulate Equation 16 with
$T=10^{5}$, $\gamma=10^{-2}$, $\eta=10^{-3}$. The last $100$ iterations were
used to estimate the accuracy error, as described in Section F.2. We used
batch sizes $b\in\left\\{64,128,256,512\right\\}$. During those experiments,
we let $\alpha$ vary in $[1.7,2]$, using a linear scale of $10$ values. In
order for the training to be stable enough for the experiment to converge, we
noted that we had to use smaller values, those values were chosen on a
logarithmic scale of $6$ values so that $\sigma\sqrt{d}\in[0.1,3]$. It is
probable that only the low noise regime, of Section 5, is then observable.
Note that, in this case, i.e. when using mini-batches, we estimated the
gradient norms $\left\|\nabla\widehat{F}_{S}\right\|$ using the batch
gradients, appearing in Equation 44, instead of the whole empirical risk
$\widehat{F}_{S}$. This makes sense as one of the goals of using batches is to
reduce the computational cost of the experiments.

###  F.2 Accuracy evolution and robust mean estimation

Figures 6 and 7 show the evolution of both the train and test accuracy, during
the first $2000$ iterations of training, for models and datasets similar to
the one used for our main experiments (but with less data to make it easier to
visualize). We show these curves for different values of the tail index
$\alpha$ and of the quantity $\sigma\sqrt{d}$, which we argue in Section 5 has
a strong impact on the training dynamics.

This allows us to observe two phenomenons, first, at least in these two
experiments, there seems to be a transition, around the value
$\sigma\sqrt{d}=1$. Indeed, for $\sigma\sqrt{d}\lesssim 1$, we observe that
the tail of the noise has little impact on the evolution of both accuracies
and therefore on the generalization error. On the other hand, for higher
values of $\sigma\sqrt{d}$, we see the impact of the tail index $\alpha$.
Indeed, the heavier the tail (i.e. the smaller $\alpha$), the more jumps are
observed on both accuracies, which necessarily have an impact on the observed
generalization. This seems to follow our theory developed in Section 5.

More importantly, the presence of those jumps makes the estimation of the
generalization error, $G_{S}(W_{T}^{S})$, extremely noisy. To avoid this
issue, we use the following two techniques:

  1. 1.

The generalization error is averaged among the $2000$ last iterations (recall
that we use $N\geq 5000$ in all our experiments so that the model is close to
converence when we estimate the generalization error).

  2. 2.

As the random jumps introduce values of the generalization error that are
completely random and often much higher than the values between the jumps, we
avoid them to bias the estimation by removing the $15\%$ upper quantile of the
generalization error, in the last $2000$ iterations, before computing the
mean.

(a) $\sigma\sqrt{d}=0.1$

(b) $\sigma\sqrt{d}=1$

(c) $\sigma\sqrt{d}=10$

Figure 6: Evolution of the test and training accuracies, during the first
$2000$ iterations, for the simulation of (3) with a linear model trained with
a cross-entropy loss, on the MNIST dataset.

(a) $\sigma\sqrt{d}=1$

(b) $\sigma\sqrt{d}=10$

Figure 7: Evolution of the test and training accuracies, during the first
$2000$ iterations, for the simulation of (3) with a $2$-layer neural network,
trained with a cross-entropy loss, on the MNIST dataset.

###  F.3 Experimental procedures details

In this section, we quickly give more details on how the figures of Section 6
were obtained.

####  F.3.1 Bound estimation - Figures 2, 8 and 11

To get Figures 2, 8 and 11, we estimated the computable part of Equation 14.
To do so, the gradient norm of all iterations where averaged, to get an
estimate of the integral term:

$\displaystyle\widehat{I}:=\gamma\sum_{k=1}^{N}\left\|\nabla\widehat{F}_{S}(\hat{W}_{k}^{S})\right\|^{2}\simeq\int_{0}^{T}\left\|\nabla\widehat{F}_{S}(W_{t}^{S})\right\|^{2}dt,$
(45)

where $\gamma$ is the learning rate and $N$ the number of iterations. The
quantity that is plotted on the $y$-axis of those figures is:

$\displaystyle\widehat{G}:=\sqrt{\frac{P_{\alpha}d^{1-\frac{\alpha}{2}}}{n\sigma_{1}^{\alpha}R^{2-\alpha}}\widehat{I}}.$
(46)

Moreover, this quantity is averaged over $10$ random seeds, with the same
values of the hyperparameters $(\sigma,d,\alpha)$. Unless mentioned otherwise,
the constant $R$, which is unknown a priori, is taken to be $R=1$ in the
figures.

####  F.3.2 Correlation experiments - Figures 1 and 10(b)

Figures 1 and 10(b): In this experiment, we train a $2$-layers neural network
on MNIST. The tail-index $\alpha$ varies in $[1.6,2]$ and the width of the
network in $[40,200]$. To obtain the green curve, we proceed as follows:

  * •

For each value fixed $w$ of the width, we compute Kendall’s correlation
coefficient, denoted $\tau$, between the accuracy error and the tail-index
$\alpha$.

  * •

This gives us the correlation in terms of the width, i.e. a map $\tau(w)$.

  * •

We repeat this procedure for $10$ random seeds.

  * •

The green curve represents the mean value and standard deviation of $\tau(w)$.

For the dot black curve, to get a less noisy coefficient, we directly used,
for each width $w$, the average of the accuracy error across all the random
seeds, and we computed the correlation between $\alpha$ and this mean accuracy
error. This is why there are no error bars in that case.

###  F.4 Additional experiments

In this subsection, we present a few additional experiments to further support
the findings of Section 6. These additional experiments are organized as
follows:

  1. 1.

Additional experiments conducted on the FashionMNIST dataset (the experiments
presented in Section 6 were all conducted on the MNIST dataset).

  2. 2.

Additional experiments using a linear model.

  3. 3.

More details figures regarding the use of mini-batches.

  4. 4.

Interesting investigations regarding the value of the parameter $R$ appearing
in Equation 14.

####  F.4.1 Additional experiment with linear models

We conducted experiments similar to that presented in Section 6 with a linear
model, instead of a $2$-layers FCN. The results are presented in Figures 8 and
9. More specifically, in these experiments, we let both the noise scale
$\sigma_{1}$ and the tail-index $\alpha$ vary in a fixed grid of values (see
Section F.1). Note that, in this case, it is not possible to let the number of
parameters vary without affecting the data, but we can act on $\sigma_{1}$
instead, and still test our theory. That being said, this model has the
advantage, when using the cross-entropy as a surrogate loss, to fit nicely in
our theoretical setting.

(a) $R=1$

(b) $R=25$

Figure 8: Linear model on MNIST Figure 9: Correlation (Kendall’s $\tau$)
between the tail index $\alpha$ and the accuracy gap for different values of
the noise scale $\sigma_{1}$.

More specifically, we can see on Figure 8 that the estimated bound, according
to Equation 46, correlates very well with the generalization error. On Figure
9, we show how the correlation between $\alpha$ and the accuracy gap varies
when $\sigma_{1}$ is varying, see Section F.3. The phase transition, predicted
in Section 5 is visible in this figure. An additional interesting phenomenon
is observed on Figure 9. Indeed, on the left of the correlation plots (both
the black and green curves), we can see that the Kendall’s $\tau$ coefficient,
measuring the correlation between $\alpha$ and $G_{S}$, decreases. We
interpret this as a third regime, when $\sigma_{1}$ is too small for our
theory to hold and the algorithm starts behaving more like a deterministic
gradient flow.

As we did on previous experiments, we can use Figure 9 and the analysis of
Section 5 to estimate the value that the parameter $R$ takes in those
experiments. Given that $d=7840$ in this experiment and that the phase
transition seems to be happening around $\sigma_{1}\simeq 0.3$, we can
estimate $R\simeq\sigma_{1}\sqrt{d}\simeq 26$. On Figure 8, we plotted the
estimated bound both for $R=1$ (our default choice) along with the estimated
bound for $R=25$. We observe on that second figure a much improved
correlation, hence supporting the experimental results of Section 6.

###### Remark 57

With this linear model experiment, it is not possible to let the number of
parameters vary (at least not without affecting the data), it is therefore not
possible to test the dimension dependence, as in Figure 3, in this case.

####  F.4.2 Additional experiments on the FashionMNIST dataset

(a) Regression of the parameter $\alpha$ from the accuracy error, for a
$2$-layers network trained on $10\%$ of the FashionMNIST dataset.

(b) Correlation between $\alpha$ and the accuracy error, for a $2$-layers
network trained on $10\%$ of the FashionMNIST dataset, with varying width.

Figure 10: Experiments with a $2$-layers FCN and the FshionMNIST dataset.

We conducted the same experiments as those presented on Figures 3 and 1 on the
Fashion MNIST dataset. The results are presented in Figures 10(a) and 10(b).
We observe the exact same behavior than the experiments conducted on the MNIST
dataset: the regression of the tail index shows a remarkable correlation with
the ground truth $\alpha$, with the expected monotonicity, even though the
value of $\alpha$ is underestimated. Moreover, the phase transition, between
the two regimes predicted in Section 5, is observable in Figure 10(b). It
seems to happen at a higher value of $d$ than in the MNIST experiment.

(a) $R=4$

(b) $R=11$

Figure 11: FasionMNIST

Additionally, we present in Figure 11 the value of the estimated bound, based
on Equation 46, compared to the accuracy gap, for different values of th
parameter $R$. For the default choice $R=1$, we already observe a very good
correlation. Based on Figure 10(b), using the same procedure than in our
previous experiments, we can estimate the value of $R$ to be about $R\simeq
11$. As we observe in all our previous experiments, in Sections 6 and F.4.1,
we observe that taking into account this value of $R$ improves the observed
correlation betzeen the bound and the accuracy gap, hence supporting the
theory.

####  F.4.3 Additional experiments with the full MNIST dataset and mini-
batches

As already mentioned in Section 6, in order to argue that our bounds may also
be pertinent in more practical settings than the figure presented in the rest
of the paper, we computed our bound using a FCN$5$, on the full MNIST dataset,
using mini-batches. This is to be compared eith our other experiments, were,
according to the SDE we study, we take the full batch at each iteration to
compute $\nabla\widehat{F}_{S}$. The exact procedure and hyperparameters are
detailed, as for other experiments, in Section F.1. These results are shown on
Figure 12, were we observe that, for several values of the batch size, we
still observe a very good correlation between the estimated bound and the
generalization error. Unfortunately, our theroy does not predict the behavior
of the accuracy gap with respect to the batch-size, but, from this experiment,
we understand that it is still capturing relatively well the behavior of the
accuracy gap.

(a) Batch size $64$

(b) Batch size $128$

(c) Batch size $256$

Figure 12: Estimated bound, computed with Equation 46, versus accuracy gap,
for a $5$-layers FCN on the full MNIST dataset, for different value of batch
size.


# Mitigating quantum decoherence in force sensors by internal squeezing

M. Korobko<EMAIL_ADDRESS>Institut für Laserphysik und
Zentrum für Optische Quantentechnologien der Universität Hamburg,
Luruper Chaussee 149, 22761 Hamburg, Germany  J. Südbeck  Institut für
Laserphysik und Zentrum für Optische Quantentechnologien der Universität
Hamburg,
Luruper Chaussee 149, 22761 Hamburg, Germany  S. Steinlechner  Faculty of
Science and Engineering, Maastricht University, Duboisdomein 30, 6229 GT
Maastricht, The Netherlands  Nikhef, Science Park 105, 1098 XG Amsterdam, The
Netherlands  R. Schnabel  Institut für Laserphysik und Zentrum für Optische
Quantentechnologien der Universität Hamburg,
Luruper Chaussee 149, 22761 Hamburg, Germany

###### Abstract

The most efficient approach to laser interferometric force sensing to date
uses monochromatic carrier light with its signal sideband spectrum in a
squeezed vacuum state. Quantum decoherence, i.e. mixing with an ordinary
vacuum state due to optical losses, is the main sensitivity limit. In this
work, we present both theoretical and experimental evidence that quantum
decoherence in high-precision laser interferometric force sensors enhanced
with optical cavities and squeezed light injection can be mitigated by a
quantum squeeze operation inside the sensor’s cavity. Our experiment shows an
enhanced measurement sensitivity that is independent of the optical readout
loss in a wide range. Our results pave the way for quantum improvements in
scenarios where high decoherence previously precluded the use of squeezed
light. Our results hold significant potential for advancing the field of
quantum sensors and enabling new experimental approaches in high-precision
measurement technology.

Introduction. Quantum-correlated light, such as squeezed light, has been
successfully used to boost the sensitivity of a broad range of force sensors:
from gravitational-wave detectors [1, 2, 3, 4, 5], optomechanical devices [6]
and dark matter detectors [7, 8], biological sensors [9, 10] and magnetometers
[11]. Reaching high sensitivity in these quantum-noise limited devices
requires maximizing the carrier light’s power to a level where the device
remains operational. A conventional method for accomplishing this is by
employing optical cavities, which resonantly enhance both the power and
signal. The combination of cavity enhancement with quantum squeezed light
allows achieving even higher sensitivity. Yet, further improvement is hindered
by quantum decoherence, caused by the loss of purity in squeezed state due to
quantum vacuum being mixed in at lossy optical components.

The optimal design of quantum sensors relies on the ability to extract
information out of the system. This ability is quantified by dedicated
metrics, such as quantum Cramer-Rao bound (QCRB), discovering new sensing
approaches [12, 13]. In an ideal lossless system, QCRB defines the best
possible sensitivity of a device at every angular signal frequency $\Omega$
for a given optical power [14, 15, 16]. In reality, decoherence always
prevents the (lossless) QCRB from being reached [17, 18, 19, 20]. For example,
the best phase sensitivity for a lossless Michelson interferometer for a given
energy per measurement time, is achieved with N00N states, where $N$ stands
for the total number of photons [21]. However, it is known that quantum
decoherence renders N00N states ineffective. Instead, the preferred method for
achieving high sensitivity involves using monochromatic light combined with a
vacuum state, squeezed in a relevant signal spectral band. [18]. In this
example, the use of a decoherence-induced quantum bound instead of a QCRB
allows to find an efficient experimental design. Such bounds are especially
relevant when the loss in a sensor is high. The major source of decoherence in
all cavity-enhanced sensors is the readout loss that occurs on the light after
outcoupling from the cavity: e.g. due to propagation losses inside the
waveguides of chip-based sensors or mode mismatch in the mode-filtered readout
of a gravitational wave detector. So far, readout loss has limited quantum
enhancement from injected squeezing in force sensors.

Here, we show that quantum squeezing of the light inside the cavity-enhanced
force sensor allows to mitigate a substantial portion of decoherence due to
loss of optical quanta. We experimentally mitigate up to 20% of readout loss
and show 4 dB of sensitivity enhancement independent of the readout loss
across a wide range. Further, we prove theoretically, that our internal
squeezing approach is the most effective strategy for boosting the sensitivity
in the presence of loss. We generalize previous studies of internal squeezing
[22, 23, 24, 25, 26, 20, 27] and establish the decoherence-induced limit for
practical lossy cavity-enhanced sensors, demonstrating that this limit is
ultimately dictated exclusively by the losses inside the detector cavity.

Figure 1:  Conceptual representation of a cavity-enhanced sensor with internal
and external squeezing. The detector cavity is used to measure the
displacement $x$ of the movable end mirror. This displacement imprints a phase
modulation on the light field, which is amplified by the cavity. Internal
squeezing is generated by a nonlinear crystal inside the cavity by pumping it
with second harmonic pump. Depending on the phase of the pump, the signal is
either amplified or deamplified. External squeezing is generated independently
and is injected into the detector cavity through a Faraday isolator (FI). The
output signal with external and internal squeezing $S_{x}$ is detected on a
phase-sensitive detector. The impact of the readout loss $\epsilon_{\rm read}$
can be reduced by optimal choice of the internal gain $q$, defined by the pump
strength. Optimal internal squeeze operation allows to reach the fundamental
limit, defined by internal loss $\epsilon_{\rm int}$.

Internal squeezing for quantum sensors. In internal squeezing, a second-order
nonlinear crystal is placed inside the cavity and pumped with second harmonic
light. The pump phase determines which quadrature of the optical field is
parametrically attenuated (squeezed), while the orthogonal quadrature is then
amplified. If the signal corresponds to a displacement of the quadrature being
squeezed, it is accordingly deamplified. Despite this, the sensitivity,
defined by the signal-to-noise ratio (SNR) is still improved [22, 24].

A simple example for a force sensor is an optical cavity with a movable end
mirror. The external oscillating force displaces the movable mirror, creating
a phase modulation signal on the reflected light field, which is detected with
a balanced homodyne detector. Here, we consider such a sensor employing quasi-
monochromatic light with an externally injected squeezed sideband spectrum as
well as a squeeze operation inside the sensor cavity (Fig. 1). Optical loss in
the system influences the sensitivity in different ways, depending on whether
the loss occurs (i) before the measurement interaction, i.e. on the meter
alone, (ii) after the measurement, i.e. on the meter that carries the full
signal information, or (iii) during the time when the meter is accumulating
the signal. In the first case, the loss sets an upper bound on the external
squeeze factor. In the second case, the loss occurs after the information has
been imprinted on the meter. The third case is central to this work and more
complex. The loss occurs during the measurement process itself, thus changing
the SNR on the out-coupled light.

We use the input-output formalism [28, 29, 30] to derive the noise spectrum
$S_{sn}(\Omega)$ and the power of signal transfer function
$\mathcal{T}_{x}(\Omega)$, as well as the noise-to-signal ratio (sensitivity)
$S_{x}(\Omega)$. They take the form:

$\displaystyle S_{\rm sn}(\Omega)=1-\frac{1-\epsilon_{\rm
read}}{(T_{c}+\epsilon_{\rm int}+q)^{2}+\Omega^{2}}$
$\displaystyle\times\left[4T_{c}q+\left(1-\beta^{-1}\right)\left((T_{c}-\epsilon_{\rm
int}-q)^{2}+\Omega^{2}\right)\right],$ (1)
$\displaystyle|\mathcal{T}_{x}(\Omega)|^{2}=\frac{8\pi P_{c}}{\hbar\lambda
c}\times\frac{T_{c}(1-\epsilon_{\rm read})}{(T_{c}+\epsilon_{\rm
int}+q)^{2}+\Omega^{2}},$ (2) $\displaystyle S_{x}(\Omega)=\frac{S_{\rm
sn}(\Omega)}{|\mathcal{T}_{x}(\Omega)|^{2}},$ (3)

where $T_{c}$ is the coupling mirror power transmission, $\epsilon_{\rm int}$
is the internal power loss, and $q$ is the roundtrip parametric power gain,
$\epsilon_{\rm read}$ is the readout power loss, see Fig. 1,
$\beta^{-1}=e^{-2r_{\rm ext}}$ is the external squeezing, with $r_{\rm ext}$
the corresponding squeeze parameter [31], $c$ is the speed of light, $\lambda$
is the central wavelength and $P_{c}$ is intra-cavity light power. In deriving
these equations we used the single-mode approximation, where only one optical
mode acquires the signal, and $\\{T_{c},\epsilon_{\rm int},q\\}\ll 1$ [32]. In
this model we neglected quantum back-action effects such as photon radiation
pressure, since they can be circumvented by quantum back-action evasion
techniques [33]. The sensitivity of our sensor is limited only by quantum shot
noise. We fixed the average intra-cavity power $P_{c}$ and assumed no loss or
depletion on the second harmonic pump power.

In order to simplify the conceptual explanation, we focus on the case of the
peak sensitivity, which occurs at $\Omega=0$, and leave out the frequency
dependence due to the cavity linewidth. From the input-output relation in the
lossless case we obtain the QCRB:

$S_{\rm QCRB}(0)=\frac{\hbar\lambda c}{8\pi
P_{c}}\times\frac{(T_{c}-q)^{2}}{\beta T_{c}},$ (4)

which turns to zero in the limit of infinite input squeezing
$\beta\rightarrow\infty$ or at the (lossless) parametric threshold for
internal squeezing $q=T_{c}$, resulting in the well-known theoretical limit of
infinite SNR in lossless systems. In [20], it was proposed that the loss-
induced sensitivity limit is defined by the noise added to QCRB due to losses,
when internal squeezing operates at threshold $q=q^{\rm
th}:=T_{c}+\epsilon_{\rm int}$. However, it is possible to optimize internal
squeezing to evade part of decoherence in the system and achieve even higher
sensitivity.

Optimized internal squeezing enhances sensitivity in one of two ways:
(i) With low readout loss or external squeezing, it attenuates the signal
quadrature. The signal is maximally deamplified by 6 dB, while quantum noise
squeezing can be arbitrary high in the absence of loss [34, 35, 24],
increasing the overall SNR. This increase is possible due to a different path
for the signal and the noise in the sensor. Maximal signal deamplification
occurs at the pump power that corresponds to the optical oscillation threshold
of the $\chi^{(2)}$ process. In practice, there is an optimal parametric pump
power value below threshold, which depends on the nonzero optical loss value.
(ii) With high readout loss or external squeezing, the nonlinear process
amplifies both signal and quantum noise in signal quadrature. In this case,
the impact of the readout loss is mitigated, similarly to the output
amplification discussed earlier in this section. An important difference is
that here amplification is realised inside the cavity, i.e. already during the
time when the signal is accumulated, which provides direct access to tuning
the SNR of the out-coupled light.

The optimal internal cavity roundtrip gain $q$ depends on the quantities
$\epsilon_{\rm int}$, $T_{c}$, $\epsilon_{\rm read}$, and $\beta$. It is
computed by minimizing the value of the sensitivity $S_{x}(0)$ in Eq. (3),
which results in the optimal sensitivity:

$\displaystyle S^{\rm opt}_{1/\beta}(0)$ $\displaystyle=\frac{\hbar\lambda
c}{8\pi P_{c}}\times 4\left(\epsilon_{\rm int}+\frac{T_{c}\epsilon_{\rm
read}}{\epsilon_{\rm read}\beta+(1-\epsilon_{\rm read})}\right),$ (5)
$\displaystyle q^{\rm opt}$ $\displaystyle=T_{c}\left(1-\frac{2\epsilon_{\rm
read}}{\beta(1-\epsilon_{\rm read})-\epsilon_{\rm read}}\right)-\epsilon_{\rm
int}.$ (6)

The expression for $S^{\rm opt}_{1/\beta}$ is strictly lower than the lowest
noise limit proposed in Ref. [20] for non-zero readout loss. A highly squeezed
beam possesses substantial photon power. However, by assuming a suitably small
measurement bandwidth, we can ensure that the power within this specific
bandwidth remains small. The fundamental sensitivity limit of our work is
achieved for infinite external squeezing, $\beta\rightarrow\infty$. In this
case the optimal internal gain maximally amplifies the signal quadrature,
$q=-q_{\rm th}$, and the limit becomes defined solely by the internal loss:

$S^{\rm lim}_{0}(0)=4\epsilon_{\rm int}\times\frac{\hbar\lambda c}{8\pi
P_{c}}.$ (7)

The equation formalizes the main theoretical result our paper: for a fixed
power in the carrier field, if the external squeeze factor approaches
infinity, the noise-to-signal ratio becomes independent on the readout loss
and approaches zero when the cavity internal loss approaches zero. Injection
loss limits the achievable level of external squeezing, thus the optimized
sensitivity given by Eq.(6) represents the practical decoherence-induced
limit.

Figure 2: Schematic representation of the experimental setup. Phase modulation
signal from the electro-optic modulator (EOM) was injected from the back of
the detector cavity and was observed on the balanced homodyne detector by
overlapping with local oscillator (LO). The phase of the LO enable the
selection of the quadrature of the light to observe. Pump light was used to
create internal amplification in the detector cavity. The internal gain was
adjusted by tuning the pump power with a variable optical attenuator (VOA).
External squeezing was injected into the ISC through a Faraday Isolator (FI).
The readout loss was adjusted with a VOA.

Experimental decoherence mitigation. We demonstrated the mitigation of the
readout loss and the existence of optimal internal squeezing in a table-top
experimental setup, see Fig. 2. Our internal squeezing cavity (ISC) was a
Fabry-Perot cavity with a nonlinear periodically poled KTP crystal inside
acting as an optical-parametric amplifier. Parametric gain of $q$ was set by
the power and the phase of the second harmonic pump. The phase of the pump was
actively controlled to keep the amplification phase stable.

Figure 3: Demonstration of optimal sensitivity with internal and external
squeezing approaching the optimal limit in Eq. (5). Each plot shows the
improvement in the SNR with respect to the case without internal squeezing
(with external squeezing). This improvement was observed as a function of
internal gain in the detector cavity: negative gain means deamplification
(squeezing), positive – amplification. Solid curves are theoretical
predictions based on the independently measured parameters. Plot (a) shows a
theoretical prediction for optimal operation in the absence of readout loss.
Plots(b-d) demonstrate the different regimes of internal squeezing for
different level of external squeezing: 5.4 dB, 8.6 dB and 10.5 dB. For low
values of external squeezing, peak SNR is achieved when squeezing is generated
also internally. For high values of external squeezing it becomes optimal to
amplify the signal quadrature. Plots (d-f) demonstrate the independence of the
peak SNR enhancement on the readout loss. The enhancement of $\sim$4 dB is
achieved for 10%, 20% and 30% of readout loss, for different levels of
internal gain. In all data sets, for the case of high squeezing (close to gain
equal to $-1$) [36], the effect of phase noise played a significant role: due
to the jitter in the phase of injected external squeezing, some part of anti-
squeezed noise coupled into the readout quadrature, which further degraded the
sensitivity (seen as strong degradation of the SNR close to gain equal to
$-1$). The error bars on the experimental data are not shown, see the
discussion in the main text.

The cavity output field was analysed with a balanced-homodyne detector (BHD).
The phase of the BHD’s local oscillator was actively controlled to keep the
readout quadrature stable. We injected a weak field carrying a 5 MHz phase
modulation signal from the back of the ISC, which emulated the measurement
signal. Depending on the phase of the pump, we could observe amplification or
deamplification of the signal, as well as anti-squeezing or squeezing of the
noise. By taking the spectrum of the signal and the noise, we could compute
the change in the SNR compared to the case without the parametric process.
Further, we injected external squeezing from a squeeze laser [36]. We kept the
external squeeze field without any bright carrier field and periodically
varied its phase. On the output of the homodyne detector, we recorded the
spectrogram (spectrum as a function of time) of the observed signal. As a
result, we consecutively measured two orthogonal quadratures of the injected
squeeze field, and used it to extract optical parameters of the setup: the
transmission of the incoupling mirror $T_{c}=11$%, the internal loss
$\epsilon_{\rm int}=1.2$%, the injection loss of 8% and the readout loss
$\epsilon_{\rm read}=10$% [37]. Initial injected external squeezing was
inferred to be 10.5 dB. Phase noise of $\approx 50$ mrad was inferred from the
measurements with different pump strength [38]. We observed less phase noise
for smaller values of external squeezing: 40 mrad for 8.6 dB and 15 mrad for
5.4 dB. We conclude that the main contribution came from the phase noise of
the external squeezing interacting with the internal squeezing process. We
used these independently measured parameters to compute expected SNR gain
based on our theoretical model.

We changed internal squeezing gain and recorded squeezing spectra together
with amplified or deamplified signals. In the first stage of the experiment,
we gradually increased the injected external squeezing value from 5.4 dB to
10.5 dB. When both the readout loss and the external squeezing were small, the
optimal internal gain corresponded to an actual squeezing process, see Fig.
3(a-b). As we increased external squeezing, the optimal internal gain
approached zero, see Fig. 3(c), and then it became optimal to amplify inside
the cavity, Fig. 3(d).

We further artificially increased the readout loss from 10% to 30%, by dumping
part of the light on a polarizing beam splitter. By taking the full range from
maximal deamplification to maximal amplification we could find the optimal
point where the SNR was the highest. In the case of 10% of readout loss, the
optimal internal gain was close to zero, see Fig. 3(d). For higher loss, as
expected from our theory in Eq. (6), it became optimal to amplify the signal
inside the cavity, see Fig. 3(e-f). We observed the maximal sensitivity
improvement of $\sim 4$ dB, independent of the readout loss. This way, we
demonstrated for the first time the mitigation of quantum decoherence. Our
results show good match with theory and demonstrate the significant
enhancement over the case without decoherence mitigation (i.e. with maximal
internal squeezing), which was considered in [20].

We could not deduce meaningful error bars for Fig. 3. Most of the source data
were averaged directly on the spectrum analyser, which did not allow for the
extraction of variances. However, since all data points are statistically
independent, the good match between the data and the theoretically predicted
SNR gain allows to be confident in the significance of our observed results in
Fig. 3.

Discussion and outlook. We have demonstrated for the first time how readout
loss can be optimally mitigated in quantum-enhanced force sensors. We used an
internal squeezing approach that complemented injected external squeezing,
reaching the newly established fundamental limit. The ultimate value of this
limit is determined exclusively by internal loss. This loss is fundamentally
different from the readout loss, since it occurs during the measurement
process itself, thus changing the SNR on the out-coupled light. Any consequent
quantum operation on this light would not be able to improve the SNR,
therefore internal loss sets the fundamental limit.

Our result can be placed in more general context of studying the impact of
state impurity on the QCRB [39, 40, 41], which has not been done for cavity-
enhanced sensors. While we do not derive the QCRB for our setup from the first
principles, our argument follows the same spirit as Refs. [39, 40, 41]. There
are two general conditions for achieving the QCRB: (i) the detector should be
in a pure state, and (ii) the back-action of the meter should be evaded [15,
16]. The state’s impurity not only prevents achieving an arbitrary low
uncertainty in the signal quadrature, but also obstructs efficient back-action
evasion, which relies on quantum correlations between the two quadratures
[29]. As a result, the primary constraint on attainable sensitivity is
determined by the state’s purity during the interaction between the meter and
the object. This is directly manifested in our fundamental limit in Eq. (7),
via its proportionality to cavity internal loss.

We also note that the output amplification proposed in Ref. [42] also evades a
part of the readout loss: the loss upon photodetection. Both signal and noise
are amplified, and thus the detrimental effect of vacuum mixed into the
quantum state can become negligible. This approach was recently explored in
other contexts [43, 44, 45, 46, 47]. In the output amplification approach, the
signal is amplified after it has exited the detector cavity. Output
amplification could only approach the fundamental limit if the inefficiency of
the photodetection was the only readout loss. In reality, the efficiency of
the photodiodes can reach very high values [37] and the main readout loss
occurs at the outcoupling from the sensor. This is the case, for example, with
imperfect transverse mode shapes such as those found in gravitational-wave
detectors and usually in chip-based optical waveguide sensors. In the
companion paper [32] we provide more details on the comparison between the two
approaches.

Our results are readily applicable to force sensors that are limited by
quantum shot noise, and whose principle schemes can be mapped to a single
cavity. This approach is especially promising for the cavities that naturally
have nonlinear materials in them, such as on-chip devices [48, 49], or
whispering-gallery-mode sensors [50, 51, 52, 53]. For these devices squeezing
injection might be challenging, and the readout is often subject to
significant losses. There, internal squeezing can become a useful tool for
mitigating these losses and achieving further quantum improvement to the
sensitivity. Even in the systems with several cavities internal squeezing
allows for the enhancement of the sensitivity, by quantum-expanding the
linewidth [26, 27] or creating PT-symmetric configurations [54, 55]. Our
approach paves the way for implementing quantum enhancement in lossy
experiments in which it was previously impractical.

The new fundamental sensitivity limit contributes to the detailed
understanding of quantum limits on realistic sensors, and serves as a valuable
guide for designing such experiments. The use of optimal internal squeezing
will enable the use of squeezed light in quantum technology beyond laboratory
environments, opening new paths for technological progress.

Acknowledgements This work was supported by the Deutsche
Forschungsgemeinschaft (DFG) under Germany’s Excellence Strategy EXC 2121
“Quantum Universe”-390833306.

## References

  * Abadie _et al._ [2011] J. Abadie _et al._ , A gravitational wave observatory operating beyond the quantum shot-noise limit, Nature Physics 7, 962 (2011).
  * Acernese _et al._ [2019] F. Acernese _et al._ , Increasing the astrophysical reach of the advanced virgo detector via the application of squeezed vacuum states of light, Physical Review Letters 123, 231108 (2019).
  * Tse _et al._ [2019] M. Tse _et al._ , Quantum-enhanced advanced LIGO detectors in the era of gravitational-wave astronomy, Physical Review Letters 123, 231107 (2019).
  * Yu _et al._ [2020] H. Yu _et al._ , Quantum correlations between light and the kilogram-mass mirrors of LIGO, Nature 583, 43 (2020).
  * Lough  _et al._ [2021] J. Lough, E. Schreiber, F. Bergamin, H. Grote, M. Mehmet, H. Vahlbruch, C. Affeldt, M. Brinkmann, A. Bisht, V. Kringel, H. Lück, N. Mukund, S. Nadji, B. Sorazu, K. Strain, M. Weinert, and K. Danzmann, First demonstration of 6 dB quantum noise reduction in a kilometer scale gravitational wave observatory, Physical Review Letters 126, 041102 (2021).
  * Yap  _et al._ [2020] M. J. Yap, J. Cripe, G. L. Mansell, T. G. McRae, R. L. Ward, B. J. J. Slagmolen, P. Heu, D. Follman, G. D. Cole, T. Corbitt, and D. E. McClelland, Broadband reduction of quantum radiation pressure noise via squeezed light injection, Nature Photonics 14, 19 (2020).
  * Backes  _et al._ [2021] K. M. Backes, D. A. Palken, S. A. Kenany, B. M. Brubaker, S. B. Cahn, A. Droster, G. C. Hilton, S. Ghosh, H. Jackson, S. K. Lamoreaux, A. F. Leder, K. W. Lehnert, S. M. Lewis, M. Malnou, R. H. Maruyama, N. M. Rapidis, M. Simanovskaia, S. Singh, D. H. Speller, I. Urdinaran, L. R. Vale, E. C. van Assendelft, K. van Bibber, and H. Wang, A quantum enhanced search for dark matter axions, Nature 590, 238 (2021).
  * Carney  _et al._ [2021] D. Carney, G. Krnjaic, D. C. Moore, C. A. Regal, G. Afek, S. Bhave, B. Brubaker, T. Corbitt, J. Cripe, N. Crisosto, A. Geraci, S. Ghosh, J. G. E. Harris, A. Hook, E. W. Kolb, J. Kunjummen, R. F. Lang, T. Li, T. Lin, Z. Liu, J. Lykken, L. Magrini, J. Manley, N. Matsumoto, A. Monte, F. Monteiro, T. Purdy, C. J. Riedel, R. Singh, S. Singh, K. Sinha, J. M. Taylor, J. Qin, D. J. Wilson, and Y. Zhao, Mechanical quantum sensing in the search for dark matter, Quantum Science and Technology 6, 024002 (2021).
  * Taylor  _et al._ [2013] M. A. Taylor, J. Janousek, V. Daria, J. Knittel, B. Hage, H.-A. Bachor, and W. P. Bowen, Biological measurement beyond the quantum limit, Nature Photonics 7, 229 (2013).
  * Taylor and Bowen [2016] M. A. Taylor and W. P. Bowen, Quantum metrology and its application in biology, Physics Reports 615, 1 (2016), arXiv:1409.0950 .
  * Li  _et al._ [2018] B.-B. Li, J. Bílek, U. B. Hoff, L. S. Madsen, S. Forstner, V. Prakash, C. Schäfermeier, T. Gehring, W. P. Bowen, and U. L. Andersen, Quantum enhanced optomechanical magnetometry, Optica 5, 850 (2018), arXiv:1802.09738 .
  * Tsang  _et al._ [2016] M. Tsang, R. Nair, and X.-M. Lu, Quantum theory of superresolution for two incoherent optical point sources, Physical Review X 6, 031033 (2016).
  * Bentley  _et al._ [2023] J. Bentley, H. Nurdin, Y. Chen, X. Li, and H. Miao, Designing Optimal Linear Detectors: A Bottom-Up Approach, Physical Review Applied 19, 034009 (2023).
  * Braginsky  _et al._ [2000] V. B. Braginsky, M. L. Gorodetsky, K. S. Thorne, and F. Y. Khalili, Energetic Quantum Limit in Large-Scale Interferometers, in  _Gravitational Waves. Third Edoardo Amaldi Conference, Pasadena, California, 12-16 July_ , edited by S.Meshkov (Melville NY:AIP Conf. Proc. 523, 2000) pp. 180–189.
  * Tsang  _et al._ [2011] M. Tsang, H. M. Wiseman, and C. M. Caves, Fundamental Quantum Limit to Waveform Estimation, Physical Review Letters 106, 090401 (2011), arXiv:1006.5407 .
  * Miao  _et al._ [2017] H. Miao, R. X. Adhikari, Y. Ma, B. Pang, and Y. Chen, Towards the Fundamental Quantum Limit of Linear Measurements of Classical Signals, Physical Review Letters 119, 050801 (2017), arXiv:1608.00766 .
  * Dorner  _et al._ [2009] U. Dorner, R. Demkowicz-Dobrzański, B. J. Smith, J. S. Lundeen, W. Wasilewski, K. Banaszek, and I. A. Walmsley, Optimal Quantum Phase Estimation, Physical Review Letters 102, 040403 (2009), arXiv:0807.3659 .
  * Demkowicz-Dobrzański  _et al._ [2013] R. Demkowicz-Dobrzański, K. Banaszek, and R. Schnabel, Fundamental quantum interferometry bound for the squeezed-light-enhanced gravitational wave detector GEO 600, Physical Review A 88, 041802(R) (2013).
  * Demkowicz-Dobrzański  _et al._ [2015] R. Demkowicz-Dobrzański, M. Jarzyna, and J. Kołodynski, Quantum Limits in Optical Interferometry, Progress in Optics 60, 345 (2015).
  * Miao  _et al._ [2019] H. Miao, N. D. Smith, and M. Evans, Quantum Limit for Laser Interferometric Gravitational-Wave Detectors from Optical Dissipation, Physical Review X 9, 011053 (2019).
  * Dowling [2008] J. P. Dowling, Quantum optical metrology – the lowdown on high-N00N states, Contemporary Physics 49, 125 (2008), arXiv:0904.0163 .
  * Rehbein  _et al._ [2005] H. Rehbein, J. Harms, R. Schnabel, and K. Danzmann, Optical Transfer Functions of Kerr Nonlinear Cavities and Interferometers, Physical Review Letters 95, 193001 (2005).
  * Peano  _et al._ [2015] V. Peano, H. G. L. Schwefel, C. Marquardt, and F. Marquardt, Intracavity Squeezing Can Enhance Quantum-Limited Optomechanical Position Detection through Deamplification, Physical Review Letters 115, 243603 (2015), arXiv:1502.06423 .
  * Korobko  _et al._ [2017] M. Korobko, L. Kleybolte, S. Ast, H. Miao, Y. Chen, and R. Schnabel, Beating the Standard Sensitivity-Bandwidth Limit of Cavity-Enhanced Interferometers with Internal Squeezed-Light Generation, Physical Review Letters 118, 143601 (2017).
  * Korobko  _et al._ [2018] M. Korobko, F. Y. Khalili, and R. Schnabel, Engineering the optical spring via intra-cavity optical-parametric amplification, Physics Letters A 382, 2238 (2018), arXiv:1709.03055 .
  * Korobko  _et al._ [2019] M. Korobko, Y. Ma, Y. Chen, and R. Schnabel, Quantum expander for gravitational-wave observatories, Light: Science & Applications 8, 118 (2019), arXiv:1903.05930 .
  * Adya  _et al._ [2020] V. B. Adya, M. J. Yap, D. Töyrä, T. G. McRae, P. A. Altin, L. K. Sarre, M. Meijerink, N. Kijbunchoo, B. J. J. Slagmolen, R. L. Ward, and D. E. McClelland, Quantum enhanced kHz gravitational wave detector with internal squeezing, Classical and Quantum Gravity 37, 07LT02 (2020).
  * Caves and Schumaker [1985] C. M. Caves and B. L. Schumaker, New formalism for two-photon quantum optics. I. Quadrature phases and squeezed states, Physical Review A 31, 3068 (1985).
  * Danilishin and Khalili [2012] S. L. Danilishin and F. Y. Khalili, Quantum Measurement Theory in Gravitational-Wave Detectors, Living Reviews in Relativity 15, 10.12942/lrr-2012-5 (2012).
  * Schumaker and Caves [1985] B. L. Schumaker and C. M. Caves, New formalism for two-photon quantum optics. II. Mathematical foundation and compact notation, Physical Review A 31, 3093 (1985).
  * Stoler [1970] D. Stoler, Equivalence Classes of Minimum Uncertainty Packets, Physical Review D 1, 3217 (1970).
  * Korobko  _et al._ [2023] M. Korobko, J. Südbeck, S. Steinlechner, and R. Schnabel, Fundamental sensitivity limit in cavity-enhanced interferometers with external and internal squeezing, to be published (2023).
  * Kimble  _et al._ [2001] H. J. Kimble, Y. Levin, A. B. Matsko, K. S. Thorne, and S. P. Vyatchanin, Conversion of conventional gravitational-wave interferometers into quantum nondemolition interferometers by modifying their input and/or output optics, Physical Review D 65, 022002 (2001).
  * Milburn and Walls [1981] G. Milburn and D. Walls, Production of squeezed states in a degenerate parametric amplifier, Optics Communications 39, 401 (1981).
  * Collett and Gardiner [1984] M. J. Collett and C. W. Gardiner, Squeezing of intracavity and traveling-wave light fields produced in parametric amplification, Physical Review A 30, 1386 (1984).
  * Schnabel [2017] R. Schnabel, Squeezed states of light and their applications in laser interferometers, Physics Reports 684, 1 (2017).
  * Vahlbruch  _et al._ [2016] H. Vahlbruch, M. Mehmet, K. Danzmann, and R. Schnabel, Detection of 15 dB squeezed states of light and their application for the absolute calibration of photoelectric quantum efficiency, Physical Review Letters 117, 110801 (2016).
  * Mehmet  _et al._ [2011] M. Mehmet, S. Ast, T. Eberle, S. Steinlechner, H. Vahlbruch, and R. Schnabel, Squeezed light at 1550 nm with a quantum noise reduction of 12.3 dB., Optics Express 19, 25763 (2011), 22273968 .
  * Tsang [2013] M. Tsang, Quantum metrology with open dynamical systems, New Journal of Physics 15, 073005 (2013).
  * Alipour  _et al._ [2014] S. Alipour, M. Mehboudi, and A. T. Rezakhani, Quantum metrology in open systems: dissipative cramér-rao bound, Physical review letters 112, 120405 (2014).
  * Fiderer  _et al._ [2019] L. J. Fiderer, J. M. E. Fraïsse, and D. Braun, Maximal quantum fisher information for mixed states, Physical review letters 123, 250502 (2019).
  * Caves [1981] C. M. Caves, Quantum-mechanical noise in an interferometer, Physical Review D 23, 1693 (1981).
  * Colombo  _et al._ [2022] S. Colombo, E. Pedrozo-Peñafiel, A. F. Adiyatullin, Z. Li, E. Mendez, C. Shu, and V. Vuletić, Time-reversal-based quantum metrology with many-body entangled states, Nature Physics , 1 (2022).
  * Manceau  _et al._ [2017] M. Manceau, G. Leuchs, F. Khalili, and M. Chekhova, Detection Loss Tolerant Supersensitive Phase Measurement with an SU(1,1) Interferometer, Physical Review Letters 119, 223604 (2017).
  * Ou [2012] Z. Y. Ou, Enhancement of the phase-measurement sensitivity beyond the standard quantum limit by a nonlinear interferometer, Physical Review A 85, 023815 (2012).
  * Knyazev  _et al._ [2019] E. Knyazev, F. Y. Khalili, and M. V. Chekhova, Overcoming inefficient detection in sub-shot-noise absorption measurement and imaging, Optics Express 27, 7868 (2019).
  * Frascella  _et al._ [2021] G. Frascella, S. Agne, F. Y. Khalili, and M. V. Chekhova, Overcoming detection loss and noise in squeezing-based optical sensing, npj Quantum Information 7, 1 (2021).
  * Ramelow  _et al._ [2019] S. Ramelow, A. Farsi, Z. Vernon, S. Clemmen, X. Ji, J. E. Sipe, M. Liscidini, M. Lipson, and A. L. Gaeta, Strong Nonlinear Coupling in a Si3N4 Ring Resonator, Physical Review Letters 122, 153906 (2019).
  * Strekalov  _et al._ [2016] D. V. Strekalov, C. Marquardt, A. B. Matsko, H. G. L. Schwefel, and G. Leuchs, Nonlinear and quantum optics with whispering gallery resonators, Journal of Optics 18, 123002 (2016).
  * V.B.Braginsky, M.L.Gorodetsky, V.S.Ilchenko [1993] V.B.Braginsky, M.L.Gorodetsky, V.S.Ilchenko, Optical whispering-gallery microresonators, Proceedings of SPIE 2097, 283 (1993).
  * M.L.Gorodetsky, V.S.Ilchenko [1994] M.L.Gorodetsky, V.S.Ilchenko, High-Q optical whispering-gallery microresonators: Precession approach for spherical mode analysis and emission patterns with prism couplers, Optics Communications 113, 133 (1994).
  * Foreman  _et al._ [2015] M. R. Foreman, J. D. Swaim, and F. Vollmer, Whispering gallery mode sensors, Advances in Optics and Photonics 7, 168 (2015).
  * Schliesser and Kippenberg [2010] A. Schliesser and T. J. Kippenberg, Cavity optomechanics with whispering-gallery-mode optical micro-resonators, arXiv , 121 (2010), arXiv:1003.5922 .
  * Dmitriev  _et al._ [2022] A. Dmitriev, H. Miao, and D. Martynov, Enhancing the sensitivity of interferometers with stable phase-insensitive quantum filters (2022), arXiv:2110.15354 [gr-qc, physics:quant-ph] .
  * Gardner  _et al._ [2022] J. W. Gardner, M. J. Yap, V. Adya, S. Chua, B. J. J. Slagmolen, and D. E. McClelland, Nondegenerate internal squeezing: An all-optical, loss-resistant quantum technique for gravitational-wave detection, Physical Review D 106, L041101 (2022).