# Entanglement Devised Barren Plateau Mitigation

Taylor L. Patti<EMAIL_ADDRESS>Department of Physics, Harvard
University, Cambridge, Massachusetts 02138, USA  Khadijeh Najafi  Department
of Physics, Harvard University, Cambridge, Massachusetts 02138, USA  IBM
Quantum, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 USA  Xun
Gao  Department of Physics, Harvard University, Cambridge, Massachusetts
02138, USA  Susanne F. Yelin  Department of Physics, Harvard University,
Cambridge, Massachusetts 02138, USA

(December 22nd, 2020)

###### Abstract

Hybrid quantum-classical variational algorithms are one of the most propitious
implementations of quantum computing on near-term devices, offering classical
machine learning support to quantum scale solution spaces. However, numerous
studies have demonstrated that the rate at which this space grows in qubit
number could preclude learning in deep quantum circuits, a phenomenon known as
barren plateaus. In this work, we implicate random entanglement as the source
of barren plateaus and characterize them in terms of many-body entanglement
dynamics, detailing their formation as a function of system size, circuit
depth, and circuit connectivity. Using this comprehension of entanglement, we
propose and demonstrate a number of barren plateau ameliorating techniques,
including: initial partitioning of cost function and non-cost function
registers, meta-learning of low-entanglement circuit initializations,
selective inter-register interaction, entanglement regularization, the
addition of Langevin noise, and rotation into preferred cost function
eigenbases. We find that entanglement limiting, both automatic and engineered,
is a hallmark of high-accuracy training, and emphasize that as learning is an
iterative organization process while barren plateaus are a consequence of
randomization, they are not necessarily unavoidable or inescapable. Our work
forms both a theoretical characterization and a practical toolbox; first
defining barren plateaus in terms of random entanglement and then employing
this expertise to strategically combat them.

##  I Introduction

The rapid development of noisy quantum devices Preskill (2018) has lead to
great interest in hybrid quantum-classical variational algorithms, through
which classical machine learning techniques are employed to prepare, sample,
and optimize states on noisy quantum hardware McClean  _et al._ (2016); Farhi
_et al._ (2014); Peruzzo  _et al._ (2014); Yung  _et al._ (2014); Wecker  _et
al._ (2015). Not only do these algorithms show potential for a variety of
near-term applications Kandala  _et al._ (2017), they are inherently robust
against certain coherent errors and are free to minimize decoherence effects
through the exploration of unconventional gate sequences. Of particular
interest are quantum neural networks (QNNs) Farhi and Neven (2018), in which
quantum input states are transformed into output states by a parametrized
quantum circuit (PQC). The output states then undergo a series of
measurements, collectively referred to as a cost function, and the measurement
results are used to optimize the circuit.

Although QNNs offer a straight-forward approach, their implementation can be
quite challenging. Among the greatest of these difficulties are barren
plateaus McClean  _et al._ (2018): regions of the cost function’s parameter
space where it is rather constant, varying too little for successful gradient-
based optimization. While in shallow circuits these barren landscapes are cost
function-dependent Cerezo  _et al._ (2020); Uvarov and Biamonte (2020), the
effect is cost function-independent for circuits that are sufficiently deep.
Moreover, even gradient-free algorithms can be impacted Arrasmith  _et al._
(2020); Anand  _et al._ (2020). While certain restricted subsets of PQCs are
somewhat resilient to barren plateaus Sharma  _et al._ (2020); Wiersema  _et
al._ (2020), the most general implementation, known as the “hardware efficient
ansatz”, becomes exponentially barren with increasing qubit number. Numerous
techniques have been suggested for the amelioration of barren plateaus,
including layerwise and symmetry based training Skolik  _et al._ (2020);
Fontana  _et al._ (2020), correlated and identity-esq circuit initialization
Volkoff and Coles (2020); Grant  _et al._ (2019), and quantum convolutional
neural network protocols Pesah  _et al._ (2020), but they have yet to form a
complete toolbox that is suitable for large-scale, general purpose QNNs.

Likewise, our understanding of barren plateaus is extensive yet far from
complete. For some years, it has been understood that barren plateaus are a
consequence of concentration of measure Bremner  _et al._ (2009); Gross  _et
al._ (2009), stemming from the effects of randomness on the exponential
dimension of quantum state space. More recently, the relationship between
entanglement and barrenness has been explored by quantum scrambling studies
Holmes  _et al._ (2020) and in terms of visible and hidden units Marrero  _et
al._ (2020). However, we still lack comprehensive understanding of how
entanglement induces barren plateaus with respect to cost function register
size, qubit connectivity, and circuit depth. As a result, barren plateau
mitigation strategies that rely on these insights have yet to be developed.

In this work, we give a detailed account of how random entanglement leads to
barren plateau formation. While barren plateaus can be both noise-independent
and noise-induced Wang  _et al._ (2020), we here consider only the former
variety. In particular, we derive the relationship between cost function
barrenness and qubit entanglement, including the rate at which barrenness
scales with circuit depth. As our findings quantify barrenness via the
entanglement of specific qubit subsets, we develop partitioning methods that
initially or continuously restrict such entanglement. This generates non-
barren cost function landscapes and thus improves circuit learning. We find
that initially partitioned circuits not only learn faster, but often produce
less entangled solutions. As entangled states are more sensitive to
decoherence, this factorizability can decrease the number of measurements
required to accurately estimate the cost function, potentially reducing the
problematic number of expectation values required for each circuit iteration
Verteletskyi  _et al._ (2020); Gokhale  _et al._ (2019); Izmaylov  _et al._
(2019).

In order to verify and exploit these findings, we design a classical meta-
learning protocol that avoids barren plateaus while generating an arbitrary
circuit with rich entanglement structure. In contrast to other QNN meta-
learning proposals Verdon  _et al._ (2019), which address specific problem
classes, ours is suitable for general PQCs. Moreover, as our meta-learning
technique does not pre-train circuit output, it is itself immune to barren
plateaus. Furthermore, we model a real-time regularization process that
penalizes forms of entanglement that are potentially problematic and show that
this method ameliorates barren landscapes, decreasing both training time and
error. We also make the novel identification of barren plateaus as a form of
Langevin noise in the circuit parameter space and demonstrate the
effectiveness of injecting additional Langevin noise into the training
process, a technique that has been used to combat overfitting in deep
classical neural networks Welling and Teh (2011). Finally, we draw a parallel
between entanglement dynamics and the improved performance of QNNs in certain
measurement bases.

Figure 1:  (a) Diagram of a linear circuit with $n_{C}=3$, $n_{N}=2$, and
$L=4$. Qubits $q_{i}$ and $q_{j}$ interact in layers $k$ through two-qubit
unitaries $u_{ij}^{k}$, which comprise layer unitaries $U_{k}$, and ultimately
form total unitary $U$. The qubits of $\mathcal{R}_{C}$ are then readout by
the cost function operator $\mathcal{M}_{C}$. (b) Ground state compressor for
randomly generated $9$-qubit long-range interaction Hamiltonian (Eq. 8) ground
states. The circuit learns to represent the ground states $|\Psi_{g}\rangle$
as $3$-qubit representations $|\psi_{g}\rangle$. (c) An illustration of barren
plateaus with $\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\rangle$
($n_{C}=2$) and $n=3,5,7,9$ (blue, orange, green, red). The variance
$\sigma_{\mathcal{O}}^{2}$ of the partial cost function derivative
$\mathcal{O}$ is known to decrease rapidly with increasing circuit layers $L$
until ultimately reaching barren plateau magnitude $\sigma_{B}^{2}\propto
2^{-n}$.

##  II Variational Algorithms in Layered 1D Quantum Circuits

Before characterizing the relationship between entanglement and barren
plateaus, we provide a brief overview of hybrid quantum-classical variational
algorithms in $1$D circuits. Examples of such circuits are shown in Figs. 1
(a) and (b). These circuits have a total number of $n$ qubits partitioned into
two registers: the cost function register $\mathcal{R}_{C}$, whose qubits are
measured with some observable $\mathcal{M}_{C}$, and the non-cost function
register $\mathcal{R}_{N}$, with qubits that are not directly measured. These
registers have $n_{C}$ and $n_{N}$ qubits, respectively, such that
$n=n_{C}+n_{N}$. In a $1$D system, the qubits interact only with their nearest
neighbors via two-qubit unitaries, denoted $u_{ij}^{k}$ for interactions
between the $i$th and $j$th qubit in layer $k$. As this work considers pure
states, each $u_{ij}^{k}$ can be fully describe with six rotation angles as
Dita (2003)

$u_{ij}^{k}=R_{34}(\theta_{6})R_{23}(\theta_{5})R_{12}(\theta_{4})R_{34}(\theta_{3})R_{23}(\theta_{2})R_{34}(\theta_{1}),$
(1)

where $R_{ij}(\theta)$ is a sinusoidal rotation matrix on axes $i$ and $j$
that can be expressed as $R_{ij}(\theta)=\exp(-i\theta K_{ij})$. Here,
$K_{ij}$ is a Hermitian matrix that is equal to $\pm i$ at elements $ij$ and
$ji$ and $0$ elsewhere. The universal nature of this parametrization
distinguishes our work from studies that impose a restricted unitary structure
Sharma  _et al._ (2020); Wiersema  _et al._ (2020) and ensures that, for
sufficient depth, our unitaries are random enough to generate barren plateaus
McClean  _et al._ (2018); Puchala and Miszczak (2017). We note that the
$\theta_{i}$ which correspond to $u_{n_{C},n_{C}+1}^{k}$ are especially
significant, as they entangle registers $\mathcal{R}_{C}$ and
$\mathcal{R}_{N}$, and we denote them $\theta_{i}^{E}$ when relevant.

These two-qubit interactions are then organized into full layer unitaries

$U_{k}=\prod_{m=0}^{(n-1)/2}u_{q+2m,q+2m+1}^{k},$ (2)

where $q$ is the remainder of $k/2$. As all interactions are pairwise, the
$u_{ij}^{k}$ in each single-layer unitary $U_{k}$ commute.

We describe the unitary of the full system as

$U=\prod_{i=1}^{L}U_{i},$ (3)

with total number of gate layers $L$. Fig. 1 (a) illustrates a generic example
of such a circuit for $n=5$, $n_{C}=3$ and $L=4$.

In hybrid quantum-classical variational algorithms, circuit training is
described by a cost function $\mathcal{L}$, which is some function $f$ of
expectation value

$\langle\mathcal{M}_{C}\rangle=\langle\psi_{out}|\mathcal{M}_{C}|\psi_{out}\rangle,$
(4)

such that $\mathcal{L}=f\left[\langle\mathcal{M}_{C}\rangle\right]$ and where
$|\psi_{out}\rangle=U|\psi_{in}\rangle$ is the output of the quantum circuit
$U$ with input state $|\psi_{in}\rangle$. Unless otherwise specified, we take
$|\psi_{in}\rangle=|0\rangle$. The classical learning algorithm then minimizes
$\mathcal{L}$ by updating the parameters $\theta_{i}$ through use of the
partial derivatives

$\mathcal{O}_{i}=\frac{\partial\mathcal{L}}{\partial\theta_{i}}=\frac{\partial
f}{\partial\langle\mathcal{M}_{C}\rangle}\frac{\partial\langle\mathcal{M}_{C}\rangle}{\partial\theta_{i}}.$
(5)

Fig. 1 (b) illustrates a specific learning example: a ground state compressor.
The ground state compressor is a circuit that takes $n=9$ qubit ground states
$|\Psi_{i}^{g}\rangle$ and their average $z$-axis magnetization

$\langle
M_{i}\rangle=\langle\Psi_{i}^{g}|\frac{1}{n}\sum_{i}^{n}\sigma_{i}^{z}|\Psi_{i}^{g}\rangle,$
(6)

as training data, where $\sigma_{i}^{b}$ is the Pauli operator along axis $b$
acting on qubit $i$. The circuit then learns to compress
$|\Psi_{i}^{g}\rangle$ into $n_{C}=3$ qubit equivalents $|\psi_{i}^{g}\rangle$
in the $x$-basis by using their average $x$-axis magnetization

$\langle
m_{i}\rangle=\langle\psi_{i}^{g}|\frac{1}{n_{C}}\sum_{i}^{n_{C}}\sigma_{i}^{x}|\psi_{i}^{g}\rangle$
(7)

as training labels. Here, we generate $N_{g}$ different $|\Psi_{i}^{g}\rangle$
from randomly parametrized long-range interaction Hamiltonians

$H=\sum_{i,j=1}^{9}\left(J_{ij}^{z}\sigma_{i}^{z}\sigma_{j}^{z}+J_{ij}^{x}\sigma_{i}^{x}\sigma_{j}^{x}\right)+\sum_{i=1}^{9}(w_{i}\sigma_{i}^{x}+v\sigma^{z}),$
(8)

where $J^{z}_{ij}$, $J^{x}_{ij}$, $w_{i}$, and $v$ are all random constants.
In this case, $\langle\mathcal{M}_{C}\rangle$ is a series of $\langle
m_{i}\rangle$ and we choose $\mathcal{L}$ as the L1 loss between the training
output and labels

$\mathcal{L}_{g}=\sum_{i}^{N_{g}}|\langle m_{i}\rangle-\langle M_{i}\rangle|.$
(9)

This circuit is an extension of that used in Shen  _et al._ (2020). We remark
that this task is inherently global, requiring magnetization information from
both $\mathcal{R}_{C}$ qubits $4$-$6$ and $\mathcal{R}_{N}$ qubits $1$-$3$ and
$7$-$9$.

##  III The Effect of Entanglement on Barren Plateaus

Barren plateaus are a manifestation of concentration of measure Walters ,
meaning that they arise from the tendency of high-dimensional, random
distributions to cluster about their mean. In a PQC, the measurement
expectation value $\langle\mathcal{M}_{C}\rangle$ of the quantum circuit is
determined by parameters $\theta_{i}$. For random circuit initialization, as
the number of these parameters grows, the impact of the individual parameter
uncertainties becomes small and, for the vast majority of parameter sets
$\theta_{i}$, $\langle\mathcal{M}_{C}\rangle$ approaches its mean with very
low variance such that
$\frac{\partial\langle\mathcal{M}_{C}\rangle}{\partial\theta_{i}}\rightarrow
0$. In the interest of building intuition, we can draw an analogy between the
collective effects of parameters $\theta_{i}$ on
$\langle\mathcal{M}_{C}\rangle$ and the behavior of an average of $N$ Gaussian
distributions $X=\frac{1}{N}\sum_{i}^{N}\mathcal{N}_{i}(\mu,\sigma^{2})$,
where $\mathcal{N}_{i}$ are Gaussian distributions with mean $\mu$ and
variance $\sigma^{2}$. Assuming that all $\mathcal{N}_{i}$ are independent,
the uncertainty of individual $\mathcal{N}_{i}$ are washed out and
$X=\mathcal{N}(\mu,\sigma^{2}/N)$. That is, the probability that $X$ deviates
from $\mu$ vanishes exponentially in $N$.

We emphasize that both concentration of measure and the barren plateaus that
they produce are a product of randomness in large dimensional systems, not
large dimensionality alone. For this reason, barren plateaus are typically
discussed in the context of random PQCs and quantified in terms of unitary
$t$-designs Renes  _et al._ (2004); Dankert  _et al._ (2009); Harrow and Low
(2009), or probability distributions that approximate the average of
polynomial functions of degree $\leq t$. Fig. 1 (c) illustrates the
characteristic behavior of these features, as detailed in McClean  _et al._
(2018). As the circuits are randomly parametrized, their statistical behavior
is described by the Haar distribution. Assuming that $U$ is at least as random
as a quantum $1$-design, the mean of $\mathcal{O}_{i}$ over the probability
distribution of all Haar random unitary matrices $U$ is
$\mu_{\mathcal{O}_{i}}=0$, and the training dynamics rely solely on the
variance of this quantity. For relatively shallow circuit depth $L$, it is
known that the unitary approaches a quantum $2$-design Harrow and Low (2009).
As such, the variance of the gradient with respect to this unitary ensemble
$\sigma_{\mathcal{O}_{i}}^{2}=\text{var}(\mathcal{O}_{i})$ decreases rapidly
in $L$, ultimately reaching the steady-state $2$-design value $\sim 2^{-n}$
McClean  _et al._ (2018). As $n$ becomes large, randomly initialized circuit
parameters cease to update and training fails. In what follows, we use omitted
subscript $\mathcal{O}$ to refer in general to arbitrary parameters
$\theta_{i}$, using the subscripted version $\mathcal{O}_{i}$ to specify only
when the distinction is relevant. For instance, our numerical data is
calculated with $\mathcal{O}_{1}$.

Figure 2:  (a) Variance $\sigma_{\mathcal{O}}^{2}$ of cost function derivative
$\mathcal{O}$ for
$\mathcal{L}=\langle\prod_{i=1}^{n_{C}}\sigma_{i}^{z}\rangle$ in units of its
barren plateau value $\sigma_{B}^{2}$ vs number of gate layers $L$ for $n=9$
and various $n_{C}$. (Inset) The change in entanglement entropy $S$ vs circuit
depth $L$ for a $4$-$5$ qubit bipartition, as defined in Eq. 11 and
illustrated in the inset of (b). Whereas $S$ describes the entanglement growth
analytically for $n_{C}=4$, it is only an approximation for $n_{C}=1$, $2$,
$3$, which could be improved by using an initial $n_{C}$-$n_{N}$ bipartition.
(b) Variance $\sigma_{\mathcal{O}}^{2}$ of cost function derivative
$\mathcal{O}$ in units of its barren plateau value $\sigma_{B}^{2}$ vs
normalized change in entropy $S$ for
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\rangle$ ($n_{C}=2$) and
$n=3,5,7,9$ (blue, orange, green, and red). Larger values of $n$ experience
greater relative suppression of $\sigma_{\mathcal{O}}^{2}$ as
$\sigma_{\mathcal{O}}^{2}/\sigma_{B}^{2}\propto S_{B}=2^{-S}$, which is the
analytical solution for $n=5$ (orange) and an approximation for other $n$.
(Inset) Schematic of bi-partition entropy of entanglement $S$ for $n=5$. Full
density matrix $\rho$ broken into two subsets $\mathcal{R}_{\alpha}$ and
$\mathcal{R}_{\beta}$, where $\mathcal{R}_{\alpha}$ always contains as much of
$\mathcal{R}_{C}$ as possible.

To intuitively understand how random entanglement causes barren plateaus, we
point out that for a randomly initialized parameter $\theta_{i}$ to contribute
to the concentration of $\langle\mathcal{M}_{C}\rangle$ and thus to the
vanishing of $\mathcal{O}$, it must have some form of influence over the
qubits of $\mathcal{R}_{C}$. For the qubits of $\mathcal{R}_{N}$, this
interaction occurs via $U$ and results in entanglement between the two
registers. According to this reasoning, barren plateau emergence should be
proportional to the spread of random entanglement. Fig. 2 (a) shows the
emergence of barren plateaus vs circuit depth $L$. As $L$ increases,
$\sigma_{\mathcal{O}}^{2}$ decreases exponentially with $\sqrt{L}$ until
approaching its asymptotic limit $\sigma_{B}^{2}$. While shallow circuits with
smaller cost function registers $\mathcal{R}_{C}$ initially enjoy greater
$\sigma_{\mathcal{O}}^{2}$, $n$ determines $\sigma_{B}^{2}$ for deep circuits
and we will later conjecture that this asymptote corresponds to entanglement
saturation between all qubits on the random circuit. As circuit depth is a
form of discretized interaction time $\tau$, this scaling is equivalent to the
$\tau$ dependence of entanglement growth of two-level quantum systems in $1$D
Žndarič (2020).

To describe these entanglement dynamics quantitatively, we consider the
density matrix of the output qubits

$\rho=|\psi\rangle\langle\psi|=U|0\rangle\langle 0|U^{\dagger}.$ (10)

In a compromise between simplicity and generality, in this work we describe
the spread of circuit entanglement with the bipartite entanglement entropy

$S=-\text{Tr}[\rho_{\alpha}\log_{2}\rho_{\alpha}]$ (11)

where $\rho_{\alpha}$ is the reduced density matrix of $(n-1)/2$ connected
qubits of register $\mathcal{R}_{\alpha}$, taken so as to contain as many cost
function qubits as possible. The remaining $(n+1)/2$ qubits are in
$\mathcal{R}_{\beta}$, such that $\rho_{\alpha}=\text{Tr}_{\beta}[\rho]$, as
illustrated in the inset of Fig. 2 (b). For pure states, this entropy is
symmetric and $S=-\text{Tr}[\rho_{\beta}\log_{2}\rho_{\beta}]$ is equivalent.
As this entanglement approximation assumes full entanglement within
$\mathcal{R}_{\alpha}$, it is most precise when
$\mathcal{R}_{C}=\mathcal{R}_{\alpha}$. Fig. 2 (a) displays
$\sigma_{\mathcal{O}}^{2}$ vs circuit depth for a variety of $n_{C}$ in an
$n=9$ system. While all $n_{C}$ scale roughly as $2^{-S}$ (inset), $n_{C}=4$
is most accurately characterized as, for that case,
$\mathcal{R}_{C}=\mathcal{R}_{\alpha}$.

Thus, while a single such partitioning is adequate for describing entanglement
spread in configurations with
$|\mathcal{R}_{C}|\sim|\mathcal{R}_{\alpha}|=(n-1)/2$, various such partitions
may be used to track short-term entanglement growth when
$|\mathcal{R}_{C}|\ll|\mathcal{R}_{\alpha}|$ or long-term entanglement growth
when $|\mathcal{R}_{C}|\gg|\mathcal{R}_{\alpha}|$, such that the entanglement
entropy does not temporarily stagnate, like Fig. 2 (b) $n=9$ (red) or rapidly
saturate, such as Fig. 2 (b) $n=3$ (blue). The plot is scaled from initial
entanglement $S_{0}$ and normalized to asymptotic difference $S_{B}-S_{0}$. In
particular, $n=3$ (blue) is initially saturated as it is nearly fully
entangled with the minimal number of gates $L=2$, while as
$|\mathcal{R}_{C}|<|\mathcal{R}_{\alpha}|$, $n=7$, $9$ (green, red) have
superlogarithmic scaling for $S\rightarrow S_{B}$. At the expense of
computational simplicity, more general metrics could be adopted, such as a
$n_{N}$-fold sum of bipartite mutual information $I_{2}$

$S_{N}=\sum_{q\in\mathcal{R}_{N}}I_{2}(\mathcal{R}_{C},\mathcal{R}_{q}),$ (12)

where $\mathcal{R}_{q}$ is the single-qubit subspace for each qubit
$q\in\mathcal{R}_{N}$.

We now derive the relationship between $S$ and $\sigma_{\mathcal{O}}^{2}$. In
particular, we consider $\mathcal{R}_{E}$, the subspace of qubits that are
entangled with (or causal to) the cost function. We begin by proving that
$\sigma_{\mathcal{O}}^{2}$ is dependent on the dimension $d_{E}$ of
$\mathcal{R}_{E}$ and then establish the link between $d_{E}$ and $S$. Let us
assume that the circuit input is a product state, here specifically
$|0\rangle$. Then the output state $\rho=|\psi\rangle\langle\psi|$ can be
written as $\rho=\rho^{E}\otimes\rho^{D}$, where $\rho^{E}$ belongs to
$\mathcal{R}_{E}$ (is entangled with $\mathcal{M}_{C}$) and $\rho^{D}$ does
not. This factorization then implies that

$\rho^{E}\otimes\rho^{D}=U|0\rangle\langle 0|U^{\dagger}\rightarrow
U=U_{E}\otimes U_{D},$ (13)

where $U_{E}|0_{E}\rangle\langle 0_{E}|U_{E}^{\dagger}=\rho^{E}$ and
$U_{D}|0_{D}\rangle\langle 0_{D}|U_{D}^{\dagger}=\rho^{D}$. Then, the
expectation value of our observable $\langle\mathcal{M}_{C}\rangle$ becomes

$\langle 0_{E}|U_{E}^{\dagger}\mathcal{M}_{C}U_{E}|0_{E}\rangle\cdot\langle
0_{D}|U_{D}^{\dagger}U_{D}|0_{D}\rangle=\langle
0_{E}|U_{E}^{\dagger}\mathcal{M}_{C}U_{E}|0_{E}\rangle,$ (14)

rendering its derivative
$\frac{\partial\langle\mathcal{M}_{C}\rangle}{\partial\theta_{i}}$ for
$\theta_{i}$ in layer $l$

$\frac{\partial\langle\mathcal{M}_{C}\rangle}{\partial\theta_{i}}=i\langle
0_{E}|U_{R}^{\dagger}\left[K,U_{L}^{\dagger}\mathcal{M}_{C}U_{L}\right]U_{R}|0_{E}\rangle,$
(15)

where $U_{R}$ and $U_{L}$ are the products of unitaries $U_{k}$ for $k<l$ and
$k\geq l$, respectively and where $K$ is the rotation generator for
$\theta_{i}$. Assuming a randomly initialized circuit, this reduces the
problem to that of McClean  _et al._ (2018), where using the Haar measure it
is shown that $\mu_{\mathcal{O}_{i}}=0$ with respect to any $\theta_{i}$, with
variance $\sigma_{\mathcal{O}}^{2}\sim 1/d_{E}$, where $d_{E}=2^{n_{E}}$ is
the dimensionality of the entangled subspace $\rho^{E}$. We contrast this with
the barrenness of a fully entangled circuit $\sim
1/d=2^{-(n_{E}+n_{D})}=2^{-n}$, which can, for many applications, be numerous
orders of magnitude smaller.

To implicate $S$ in this barren plateau process, we note that for
$\rho=\rho^{E}\otimes\rho^{D}$

$S=-\text{Tr}\left[\text{Tr}_{\beta}[\rho^{E}\otimes\rho^{D}]\cdot\log_{2}(\text{Tr}_{\beta}[\rho^{E}\otimes\rho^{D}])\right].$
(16)

Given that if $|\rho^{E}|<|\rho_{\alpha}|$ we need to describe early
entanglement spread with a smaller bipartition of $S$, we can assume that
$\rho^{D}$ is fully contained in $\rho_{\beta}$ such that
$\text{Tr}_{\beta}[\rho^{E}\otimes\rho^{D}]=\text{Tr}_{\beta_{E}}[\rho^{E}]\cdot\text{Tr}[\rho^{D}]=\text{Tr}_{\beta_{E}}[\rho^{E}]$,
where $\beta_{E}$ is the entangled portion of $\mathcal{R}_{\beta}$. Then

$S=-\text{Tr}\left[\text{Tr}_{\beta_{E}}[\rho^{E}]\cdot\log_{2}\text{Tr}_{\beta_{E}}[\rho^{E}]\right]\\\
=-\text{Tr}\left[(\rho^{E})_{\alpha}\cdot\log_{2}(\rho^{E})_{\alpha}\right]=n_{\beta_{E}}=n_{E}-n_{\alpha}$
(17)

as this is precisely the definition of the number of entangled qubits shared
between $\rho_{\alpha}$ and $\rho_{\beta}$. Then, the total number of
$\mathcal{R}_{C}$ entangled qubits is $n_{\beta_{E}}+n_{\alpha}=n_{E}$ such
that $d_{E}=2^{n_{E}}$. Therefore, $\sigma_{\mathcal{O}}^{2}\propto
2^{-n_{E}}$ and changes proportionally to $2^{-S}$.

For simplicity, in the above proofs we assumed that a given qubit is either
completely entangled or disentangled. A similar result for the more general
case of partial entanglement follows straight-forwardly by taking general
$|\psi\rangle=\sum_{i}c_{i}|\psi_{E}^{i}\rangle|\psi_{D}^{i}\rangle$,
following the above steps, and repartitioning each component of the sum

$\rho=\sum_{i,j}c_{i}c_{j}^{*}|\psi_{E}^{i}\rangle|\psi_{D}^{i}\rangle\langle\psi_{E}^{j}|\langle\psi_{D}^{j}|$
(18)

into
$|\psi_{E^{\prime}}^{k}\rangle\langle\psi_{E^{\prime}}^{k}|\otimes|\psi_{D^{\prime}}^{k}\rangle\langle\psi_{D^{\prime}}^{k}|$
such that the dimension of $|\psi_{E^{\prime}}^{k}\rangle$, is maximal under
the factorization constraint.

Figure 3:  (a) Partitioned cost function derivative variance
$\sigma_{\mathcal{OP}}^{2}$ for
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\rangle$ in units of its non-
partitioned value $\sigma_{\mathcal{O}}^{2}$ vs number of gate layers $L$ for
$n=3,5,7,9$ (blue, orange, green, and red) and $n_{C}=2$. As expected,
$\sigma_{\mathcal{OP}}^{2}$ is approximately a factor of $2^{n_{N}}$ larger
than $\sigma_{\mathcal{O}}^{2}$ due to the absence of random
$\mathcal{R}_{C}$-$\mathcal{R}_{N}$ entanglement. (b) Training loss
$\mathcal{L}=\mathcal{L}_{g}$ of Eq. 9 for ground state compressor in Fig. 1
(b) (gray) and
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$ (red)
vs training epochs for $L=200$. Solid lines are initially partitioned circuits
and dashed lines are fully random initializations, with increased learning
performance in the latter. The corresponding evolution of $S$ is shown in the
(inset).

This mapping between the number and degree of cost function-entangled qubits
and plateau barrenness highlights that circuit connectivity, and not simply
overall circuit depth, is an accurate indicator for the barrenness of the
training landscape. Fig. 3 (a) is a proof of principle illustration of this
point. We remove the register connecting the $u_{2,3}^{k}$ (unitaries between
qubits $q_{2}$ and $q_{3}$) gates from each layer $k$ for circuits where
$n_{C}=2$ and $\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\rangle$,
permanently separating, or partitioning, the registers $\mathcal{R}_{C}$ and
$\mathcal{R}_{N}$. As circuit depth grows, entanglement with the qubits of
$\mathcal{R}_{N}$ suppresses $\sigma_{\mathcal{O}}^{2}$ much faster than its
partitioned counterpart $\sigma_{\mathcal{OP}}^{2}$, which never exceeds the
variance of circuit of total $n=2$ and is therefore numerous orders of
magnitude larger than the variance $\sigma_{\mathcal{O}}^{2}$ of the fully
entangled system. While insightful, permanent partitioning is clearly not a
practical solution for barren plateaus, as it limits not only the barrenness,
but also the expressibility of the circuit to that of only $n_{C}$ qubits.

##  IV Initialization Techniques for Barren Plateau Mitigation

While permanent partitioning is tantamount to simply employing a circuit of
smaller $n$, initial parameter restrictions can improve circuit trainability
without reducing circuit expressibility. Intuitively, the advantages of this
method stem from the role of entropy as a thermodynamic arrow that drives
statistical processes forward. As a toy example, let us imagine a classical
machine learning protocol where we would like to create a gaseous mixture with
optimized concentrations of two gases. If we initially partition the gases,
the learning algorithm can simply allow the gases to mix themselves by passing
through a vent in the partition, sealing the vent when the ideal concentration
is reached on one side. This process occurs independently, driven forward by
entropic considerations. If, however, the gases are initially mixed and
therefore have a maximum entropy configuration, the learning algorithm cannot
succeed by simply unsealing a vent. The problem has been complicated and
learning will fail unless more heroic measures are taken.

In this section, we explore methods for quantum equivalents of such entropy
limiting initialization schemes and in Sec. V we detail some of these “more
heroic” measures.

###  A Initial Entanglement Partitioning

One way to avoid barren plateaus without suppressing expressibility is to
initially partition the circuit, like in Fig. 3 (a), but then to allow
$\mathcal{R}_{C}$-$\mathcal{R}_{N}$ entanglement throughout the training
process. This method is fundamentally distinct from Grant  _et al._ (2019);
Wiersema  _et al._ (2020), as we only initialize a subset of two-qubit gates
$u_{ij}^{k}$ to the identity and have devised a cost function and
entanglement-based strategy to motivate this choice. Furthermore, our
treatment applies to universal PQCs of potentially great depth, not restricted
subspaces of $U$ Wiersema  _et al._ (2020). Fig. 3 (b) displays the $n=9$
ground state compressor loss $\mathcal{L}=\mathcal{L}_{g}$ of Eq. 9 (gray) and
training of
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$ (red)
for both initially partitioned (solid lines) and fully random (dashed)
initializations with $L=200$. Throughout this work, the AMSGrad gradient
descent algorithm is used for circuit parameter update Reddi  _et al._ (2018).
The corresponding bipartite entanglement entropies $S$ are in the inset.

At first, initially partitioned circuits suffer a bout of decreased accuracy,
which corresponds to a period of low yet rapidly growing entanglement $S$
(inset) that is either insufficient to express the target state of interest
(gray) or simply lower than those of many of the degenerate solutions (red).
Later, initially partitioned circuits can produce lower error and require
fewer training epochs, which we hypothesize stems from the responsiveness of
the gradient during the initial phase of low entanglement, enabling the system
to train unfettered by barren plateaus and driving interactions forward
through entropy growth. Furthermore, strategic initializations may be
sufficient to avoid barren plateaus throughout training. We emphasize the
relationship between plateau barrenness and $S$ (or alternatively, in other
works, $n$ and $L$) is a product of circuit parameter randomness of at least a
quantum $2$-design and can thus only be assumed for random circuit
initializations. That is, such randomness cannot generally be assumed
throughout the training process, as this represents an inherently structured
organizing of circuit parameters.

When initially partitioned cost functions are learned with high accuracy
(here, ground state compression, but also in both the partitioned training of
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$ in Fig.
5 (c)), $S$ peaks towards the end of the rapid training period before dropping
down to a lower steady-state value. This indicates that the initially
partitioned circuit identifies an appropriate solution that is less entangled
with the unmeasured qubits of $\mathcal{R}_{N}$, a potentially desirable
quality as widespread entanglement can lower the coherence time of qubits.
Moreover, an extension of this technique could be used to partially factor the
cost function registers themselves, resulting generally in fewer required
readouts for a given cost function determination and ameliorating the so-
called “measurement problem” Verteletskyi  _et al._ (2020); Gokhale  _et al._
(2019); Izmaylov  _et al._ (2019).

We indicate that this ultimate drop in bipartite entanglement is reminiscent
of the late-stage decrease in tripartite mutual information noted in Shen  _et
al._ (2020). The two phenomena are not in conflict, however, with the former
indicating the disentanglement of measured and unmeasured qubits after the
necessary information from those qubits had been collected, while the later
signals that the global features of the input information are learned towards
the end of the training process. In fact, both observations suggest that
information locality is the salient feature of late-stage hybrid quantum-
classical learning algorithms. Finally, we indicate that a reduction in $S$
also occurs for non-partitioned circuits that can be learned with high
accuracy (dashed gray in Fig. 3 (b) and dashed black Fig. 5 (c)), whereas it
is absent from low-accuracy circuits (red). Indeed, some degree of automatic
$\mathcal{R}_{C}$-$\mathcal{R}_{N}$ factorization appears to be a natural
feature of high-accuracy QNN training.

Finally, we comment that the cost function
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$ trains
rapidly when initialized to a barren plateau, and neither its time nor
accuracy are improved by mitigating these barren plateaus via initial
partitioning. This suggests that certain classes of cost functions (in this
case observables that target one of their eigenstates, as will be discussed in
Sec. V) may be naturally resistant to barren plateaus. This could potentially
be due to a rapidly accelerating ordering process, wherein even small
$\sigma_{\mathcal{O}}^{2}$ quickly navigate $\mathcal{L}$ to a non-barren
region of its landscape.

###  B Entanglement Meta-Learning as Circuit Pre-Training

Figure 4:  (a) A deep circuit ($L=100$) pre-training procedure that minimizes
collective entanglement $S_{C}$ (Eq. 21), the entanglement entropy between
both the input and output registers of $\mathcal{R}_{C}$ and $\mathcal{R}_{N}$
for $n=3$, $5$ (blue, orange). As random entanglement decreases with $S_{C}$,
$\sigma_{\mathcal{O}}^{2}$ increases. Crucially, this pre-training procedure
is unique from partitioned initialization as it permits non-trivial
interaction at the level of individual circuit layers (inset), with magnitude
of inter-register interaction remaining $2/\pi\approx 0.637$, which is
consistent with that of random $\theta_{i}$. (b) Derivative variance
$\sigma_{\mathcal{O}}^{2}$ for
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\rangle$ ($n_{C}=2$) and
initially partitioned registers $\mathcal{R}_{C}$ and $\mathcal{R}_{N}$ in
units of its non-partitioned value $\sigma_{B}^{2}$ vs number of register
entangling gate layers $L_{E}$. Here, $L=200$ and $n=3,5,7,9$ (blue, orange,
green, and red).

A similar yet more sophisticated solution for non-barren initializations is
classical pre-training of the circuit gates to control
$\mathcal{R}_{C}$-$\mathcal{R}_{N}$ entanglement. This process is a form of
meta-learning Thrun and Pratt (1998), a branch of machine learning algorithms
directed at optimizing the learning process of other algorithms.

We must be careful, however, in our choice of pre-training cost function.
Simply minimizing $S$ would itself be a form of randomly parametrized gradient
descent algorithm on the output qubits and would thus, like previous meta-
learning techniques, tend to generate the concentration of parameters that
lead to barren plateaus Verdon  _et al._ (2019). We reiterate that this
observation is not in conflict with our claim that $\sigma_{\mathcal{O}}^{2}$
vanishes $\propto 2^{-S}$ for randomly initiated PQCs, as this relation is not
universal, but rather applies to circuit unitaries that are Haar distributed,
and can therefore only be assumed in random, not pre-trained, circuits.

As an alternative to $S$, we can combat barrenness by minimizing the
collective entanglement $S_{C}$ of registers $\mathcal{R}_{C}$ and
$\mathcal{R}_{N}$, which considers the $2n$-qubit space of both input and
output registers. To define $S_{C}$, we boost into the $2n$-qubit pure state

$|\Psi\rangle=\sum_{i,j=0}^{2^{n}-1}\frac{\langle\psi_{i}|U|\psi_{j}\rangle}{\sqrt{2^{n}}}|\psi_{j}\rangle|\psi_{i}\rangle$
(19)

where $|\psi_{i}\rangle$ represent some set basis vectors in the
$n$-dimensional Hilbert space. We can then define the density matrix operator
of the full $2n$ collective qubit system

$\mathrm{P}=|\Psi\rangle\langle\Psi|.$ (20)

Now the reduced density matrix and corresponding entropy of entanglement of
$\mathcal{R}_{C}$ are defined over its $2n_{C}$ input and output qubits as

$\mathrm{P}_{C}=\text{Tr}_{N}[\mathrm{P}],\hskip
5.69046ptS_{C}=-\text{Tr}[\mathrm{P}_{C}\log_{2}\mathrm{P}_{C}],$ (21)

where $\text{Tr}_{N}$ is a trace over the $2n_{N}$ input and output qubits of
$\mathcal{R}_{N}$. Fig. 4 (a) shows the growth of $\sigma_{\mathcal{O}}^{2}$
for $n=3$, $5$ (blue, orange), $n_{C}=2$, and $L=100$. As the entanglement is
minimized, $\sigma_{\mathcal{O}}^{2}$ draws closer to its partitioned value
$\sigma_{n\approx n_{C}}^{2}$, reducing the initialization problem of the
barren plateau from $O(2^{-n})$ to approximately $O(2^{-n_{C}})$ as the
initialization $S_{C}$ of the circuit is minimized through training. This
comparison is approximate because the pre-training method, while quite
general, implies some inherent ordering that may distinguish it from a
bipartition of $2$-designs. As $S_{C}\rightarrow 0$ and $\mathcal{R}_{C}$ and
$\mathcal{R}_{N}$ become factorized, $\sigma_{\mathcal{O}}^{2}$ grows more
similar to the variance of an $n_{C}$ qubit system, reducing the barren
plateau effect by $\approx 2^{n-n_{C}}$ orders of magnitude. We note that as
$S_{C}\leq 2\text{min}(n_{C},n_{N})$, $\sigma_{\mathcal{O}}^{2}$ of $n=3$ is
constant for $S_{C}>2$.

Critically, the average magnitude of $\mathcal{R}_{C}$-$\mathcal{R}_{N}$
interaction on a given layer $k$ is not reduced. To see this, consider a rough
metric of inter-register mixing

$\langle|\sin(\theta_{i}^{E})|\rangle=\frac{1}{3L}\sum_{\theta_{i}^{E}}|\sin(\theta_{i}^{E})|,$
(22)

where $\theta_{i}^{E}$ are the $3L$ rotation angles of the
$u_{n_{C},n_{C}+1}^{k}$ gates that entangle the registers $\mathcal{R}_{C}$
and $\mathcal{R}_{N}$. $\langle|\sin(\theta_{i}^{E})|\rangle$ describes these
interactions because it is the average of off-diagonal (or rotating) elements
in the two-qubit rotation matrices $u_{n_{C},n_{C}+1}^{k}$. The inset of Fig.
4 demonstrates that this quantity remains at its uniformly distributed value
$2/\pi$, even as the collective registers become increasingly factored. This
indicates that while the total entanglement of collective $\mathcal{R}_{C}$
and $\mathcal{R}_{N}$ is reduced, the average inter-register interaction at
any given layer $k$ remains unaffected, providing a highly non-trivial circuit
initialization. This method is distinct from Grant  _et al._ (2019) as it does
not produce a network of identity-producing blocks, but rather a nearly
arbitrary initialization with the sole yet crucial constraint of adjustable
factorizability along a single connection. This distinction may be
particularly important for deep circuits Campos  _et al._ (2020). What is
more, this method assumes no specification of problem structure Verdon  _et
al._ (2019), making it generally applicable.

Although classical pre-training is untenable for circuits of large $n$, it can
serve as meta-learning for the fundamental effect of entanglement on PQCs and
their optimal initializations. As such, it may lead to some scalable
generalization of the procedure. It could also be applied iteratively on
subsets of large circuits, i.e., on the qubits which form the border between
$\mathcal{R}_{C}$ and $\mathcal{R}_{N}$. Most promisingly, recent advances in
efficient subsystem entanglement measuring techniques, such as random
measurements Brydges  _et al._ (2019) and fidelity out-of-time correlators
Lewis-Swan  _et al._ (2019), may pave the way for on-hardware hybrid quantum-
classical variational minimization of collective entanglement $S_{C}$, or some
analogous measure, enabling full-circuit, high-variance gradient
initializations for arbitrary size PQCs.

##  V Dynamic Control of Barren Plateaus

As the difficulties of training the relatively simple cost function
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$ for
deep circuits in Fig. 3 (b) alludes, initialization techniques can be
insufficient for complete mitigation of barren plateaus. To combat this, we
now propose a variety of methods to directly manage long-term entanglement of
the $\mathcal{R}_{C}$ and $\mathcal{R}_{N}$ output registers. Returning to the
analogy of optimal mixing of a classical bipartite gas in Sec. IV, this
section details quantum analogies to the more “heroic” methods that we can
take to dynamically control the gaseous mixture’s entropy, such as
regularization of (penalization of the learning algorithm) or the introduction
of additional dynamics into the system.

###  A Hard Limit on $\mathcal{R}_{C}$-$\mathcal{R}_{N}$ Entangling Gates

The simplest of these dynamic methods is imposing a hard limit on the number
of $\mathcal{R}_{C}$-$\mathcal{R}_{N}$ entangling layers $L_{E}$ in an
otherwise deep circuit. This is explored in Fig. 4 (b). Although total depth
$L=200$, relatively large gradients can still be achieved while still
permitting a considerable number of $\mathcal{R}_{C}$-$\mathcal{R}_{N}$
interactions. As $L_{E}$ grows, $\sigma_{\mathcal{O}}^{2}$ decays with a
similar scaling in $L_{E}$ as unrestricted circuits do in total gate number
$L$, corroborating that barren plateaus indeed arise with the spread of cost
function entanglement, not circuit depth itself. This method could be
particularly fruitful when using a reinforcement learning algorithm Zhang  _et
al._ (2019); Verdon  _et al._ (2019), as the circuit could learn to process
and extract the most relevant portions of $\mathcal{R}_{N}$ before ultimately
transferring them to $\mathcal{R}_{C}$ in a limited number of $L_{E}$.

###  B Entanglement Regularization

Figure 5:  Loss function vs epochs for entanglement regularized learning of
(a) ground state compressor $\mathcal{L}=\mathcal{L}_{g}$ (Eq. 9), (b)
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$, and
(c) $\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$
with $L=200$. In all cases, non-zero regularization terms lead to equal (c) or
faster and more accurate (a and b) learning with decreased $S$, mitigating the
effects of barren plateaus. Moreover, the behavior of $S$ is reflective of the
overall training difficulty (insets): whereas the cost functions of (a) and
(c) can be learned rather rapidly and accurately and do so with naturally
lower levels of bipartite entanglement $S$ that are more responsive to
regularization, that of (b) trains slower and with less accuracy, with
entanglement growth that is controlled very little by regularization. The
black dashed line in (c) corresponds to $S$ for unregularized, unpartitioned
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$, which
learns with equal effectiveness as its partitioned and regularized
counterparts, highlighting the increased learnability eigenstate learning.

A yet more dynamic method for limiting entanglement is with regularization of
$\mathcal{R}_{C}$-$\mathcal{R}_{N}$ gates. Regularization adds a penalizing
term with adjustable scale parameter $\lambda$

$\eta=\lambda\sum_{i}|\sin(\theta_{i}^{E})|\mathcal{L}$ (23)

to the original cost function $\mathcal{L}$ in order to implicitly limit the
amount of cross-register entanglement. $\sum_{i}|\sin(\theta_{i}^{E})|$ is
proportional to the inter-register mixing measure
$\langle|\sin(\theta_{i}^{E})|\rangle$ of Sec. IV and serves to limit
entanglement generating interactions. Moreover as the values $\theta_{i}$ are
already stored within the classical learning algorithm, this metric does not
require additional queries to the quantum hardware. Scaling $\eta$ by
$\mathcal{L}$ results in an adaptive regularization process that resists
entanglement in regions of poor solutions while relaxing to the original
learning problem as $\mathcal{L}$ approaches zero. This adaptivity can be even
more fruitful by making $\lambda$ a decreasing function of $\mathcal{L}$, such
that $\eta$ disturbs the learning process even less near optimal solutions.

The regularized gradient is then

$\mathcal{O}_{i}=\left(1+\lambda\sum_{i}|\sin(\theta_{i}^{E})|\right)\frac{\partial\mathcal{L}}{\partial\theta_{i}}+\lambda\cos(\theta_{i})\text{sign}(\sin(\theta_{i}))\mathcal{L}.$
(24)

During portions of the training process that are still largely random, the
average of $\frac{\partial\mathcal{L}}{\partial\theta_{i}}$ over all Haar
random unitaries $\mu_{\mathcal{O}_{i}}=0$, as we can assume by concentration
of measure for deep circuits that $\sum_{i}|\sin(\theta_{i}^{E})|$ is
approximately constant. The variance, however, does increases to

$\sigma_{\mathcal{O}_{i}}^{2}\rightarrow\left(1+\lambda\frac{6L}{\pi}\right)^{2}\sigma_{\mathcal{O}_{i}}^{2}.$
(25)

We highlight that although regularization only directly augments the variance
of parameters $\theta_{i}^{E}$ upon which it acts, we observe a similar
increase on the unregularized angles, indicating that its mitigation of barren
plateaus is a system-wide effect. Furthermore, we note that $\lambda$ is
adjustable and that the regularized variance grows quadratically in circuit
depth, whereas $\sigma_{\mathcal{O}}^{2}$ is constant for deep circuits with a
given number of qubits $n$.

Fig. 5 displays this learning process for an initially partitioned circuit
trained with a $\lambda$ which is piecewise-adaptive in (solid line) in
comparison with an algorithm using only initial partitioning, that is,
$\lambda=0$ (dashed) for $L=200$. Three different loss functions are used: (a
- gray) ground state compressor $\mathcal{L}=\mathcal{L}_{g}$ (Eq. 9), (b -
red) $\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$,
and (c -green)
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$. Ground
state compression can be achieved both faster and with greater factorization
of the output solution, while solutions to
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$ have
greatly improved accuracy. Although
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$ is
rapidly learned both with and without regularization and/or initial
partitioning, its regularized solutions still benefit from the increased
factorizability, with the dashed black line in Fig. 5 (c - inset) showing its
unpartitioned $S$.

As discussed in Sec. IV, we again comment on the seeming resilience from
barren plateaus of
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle$ and
other cost function measurements that target their eigenstates. While still
beginning in a barren landscape with overwhelming probability for random
circuit initializations, these algorithms learn equally well as without barren
plateau mitigation.

###  C Langevin Noise as Gradient Supplement

Figure 6:  Preparing a state $|\psi\rangle$ under barren plateau conditions
(random initialization of $L=200$) for $n=9$ and cost function
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$
through both (a) the addition of Langevin noise on a subset of parameters and
(b) substitution of measurement basis for which target state is an eigenstate.
(a) Additional Langevin noise term $\lambda\sum_{i}^{N}|\phi_{i}|\mathcal{L}$
increases gradient variance $\sigma_{\mathcal{O}_{i}}^{2}\rightarrow(1+\lambda
N\pi)^{2}\sigma_{\mathcal{O}_{i}}^{2}$ with respect to parameters $\phi_{i}$,
thus helping to navigate barren plateau landscapes. This can be viewed as an
increase in diffusion constant $2\mathcal{D}$. (b) By substituting the cost
function $\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{x}\rangle$, for which
our target state $|\psi\rangle$ is an eigenstate (or alternatively, choosing a
“natural” cost function basis), we obtain a faster, more accurate learning
process.

The results of Fig. 5 (b) raise an interesting point: the addition of
regularization terms to the cost function can improve accuracy without
significantly decreasing entanglement. We hypothesize that this is because
$\eta$, while sometimes successfully limiting entanglement, is always
providing additional perturbation in the form of noise. Langevin noise in
particular has proven fruitful in classical machine learning and has been used
to prevent overfitting in classical neural networks Welling and Teh (2011).

To motivate this hypothesis, we make the observation that the cost function
gradient in barren plateaus can be conceptualized as a form of Langevin noise
in circuit parameter space. Typically, Langevin noise is defined for functions
$g$ that vary with time $\tau$. Then $g(\tau)$ satisfies the conditions that
$\langle g(\tau)\rangle_{\textbf{Lan}}\equiv\int g(\tau)d\tau=0$ and $\langle
g(\tau)g(\tau^{\prime})\rangle_{\textbf{Lan}}=2\mathcal{D}\delta(\tau-\tau^{\prime})$,
where $\delta$ is the Dirac-delta function and $\mathcal{D}$ is some finite,
non-zero diffusion constant Coffey  _et al._ (1996). In the case of
$\mathcal{O}$, the moments are not integrals over time, but rather over the
parameters $\theta_{i}$ as described by the Haar measure, such that
$\mathcal{D}=\sigma_{\mathcal{O}}^{2}/2$.

Under this Langevin noise formulation, barren plateaus can be framed as an
entanglement-induced diminution of $\mathcal{D}$. To examine the utility of
Langevin noise, we can add an additional noise term
$\lambda\sum_{i}^{N}|\phi_{i}|\mathcal{L}$ to the original loss function such
that

$G=(1+\lambda\sum_{i}^{N}|\phi_{i}|)\mathcal{L},$ (26)

with derivatives $g_{i}=\frac{\partial G}{\partial\phi_{i}}$ and where
$\phi_{i}$ are an arbitrarily chosen subset of circuit parameters of size $N$.
For uniformly distributed $\phi_{i}\in\hat{\phi}$ on the interval $[0,2\pi)$,
this yields the equivalent relation

$\langle
g_{i}(\hat{\phi})g_{i}(\hat{\phi})\rangle_{\textbf{Lan}\mathbf{\phi}}\equiv
2\mathcal{D}=(1+\lambda N\pi)^{2}\sigma_{\mathcal{O}_{i}}^{2}.$ (27)

Fig. 6 (a) illustrates the effectiveness of such Langevin noise in barren
landscapes, producing a high-accuracy solution for
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$
despite fully random initialization for $n=9$ on a deep circuit ($L=200$). We
note that, like the increased variance of entanglement regularization, angles
that are not directly perturbed by added Langevin noise still enjoy an
increase in variance from the system-wide effect of the technique.

###  D Natural Cost Function Bases

Finally, we discuss our repeated observation that successful learning in
initially barren landscapes is greatly facilitated when the target output is
an eigenstate of the cost function observables, a basis choice that we refer
to as “natural”. This observation of a natural basis has been made for other
product state PQC objective functions, such as in the basis transformations of
electronic structure reference states in quantum chemistry McClean  _et al._
(2016). Not only do such configurations learn more rapidly, their training
rate and accuracy are not impacted by otherwise successful barren plateau
mitigation techniques.

We have suggested a potential link between this trainability of natural basis
cost functions and the tendency of these circuits to limit their own
entanglement, navigating out of the barren landscapes of random matrices and
into a tractable configuration. In particular, the variational preparation of
measurement eigenstates has several entropy-based advantages, such as a
vanishing gradient variance when circuit approaches an optimal solution. As
discussed in Sec. IV, this effect may also be due to a rapidly accelerating
ordering process, wherein even small $\sigma_{\mathcal{O}}^{2}$ lead to
$\mathcal{L}$ efficiently escaping into non-barren regions of its landscape.

Regardless of the origin of its effect, rotating cost function measurements
into natural bases can be a strong barren plateau mitigating strategy. Fig. 6
(b) illustrates that by substituting
$\mathcal{L}=\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{x}\rangle$ for
$\mathcal{L}=|\langle\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}\rangle|$, we
can obtain a desired solution $|\psi\rangle$ such that
$\langle\psi|\sigma_{1}^{z}\sigma_{2}^{z}\sigma_{3}^{z}|\psi\rangle=0$ much
more effectively. The replacement of a single $\sigma^{z}$ with the operator
$\sigma^{x}$ reduces the problem to an eigenstate optimization and results in
a learning process that trains quickly and automatically limits entanglement,
(entanglement behavior analogous to the black dashed line in Fig. 5 (c)), in
contrast to the difficulties of the original problem (red dashed line in Fig.
3 (b)).

##  VI Conclusion

We have demonstrated the relationship between total qubit-cost function random
entanglement and the barrenness of a learning landscape both analytically and
numerically and oriented these findings within the context of many-body
entanglement dynamics. Based on these results, we established various metrics
for barren plateau prediction, both in terms of entanglement and, for a $1$D
system, circuit depth. We also proposed an input-output entanglement metric,
whose minimization we suggest is key to circuit learnability. Using this
knowledge, we went on to propose various mitigation schemes, including initial
partitioning of cost function and non-cost function registers, meta-learning
of low-entanglement high-interaction PQC initializations, limiting inter-
register interaction, entanglement regularization, the addition of Langevin
noise, and utilizing natural cost function bases. We demonstrated the
effectiveness of these techniques, elucidating the role that entanglement
minimization plays in both the assisted and unassisted training of QNNs and
emphasizing that, as existing barren plateau proofs assume sufficiently random
parametrizations which do not apply under all circumstance, barren plateaus
can potentially be avoided or escaped in generic PQCs.

While these findings imply that QNN learning must strike a non-trivial balance
between randomness, expressibility, and barrenness, they lay the groundwork
for numerous mitigation techniques that may facilitate large-scale quantum
circuit learning. Furthermore, these methods furnish various secondary
benefits, such as solution factorization, novel paradigms of quantum meta-
learning, and increased understanding of circuit optimization, to name a few.
Furthermore, they suggest that the growth of circuit entanglement could
potentially be harnessed to drive the learning process.

Oftentimes, the presence of barren plateaus in PQCs is interpreted as an
absolute impasse, as it is typically believed to preclude learning. However,
this work emphasizes that not only can barren plateaus be ameliorated through
entanglement considerations, they should be understood as manifestations of
circuit randomness that have not been proved to apply to more organized
configurations, such as those which may manifest during the learning process.
To understand the relationship between cost function barrenness and total
circuit learnability, the evolution of circuit parameter distributions
throughout the learning process should be characterized. Such a statistical
characterization will also shed further light on the viability of barren
plateau mitigation methods.

###### Acknowledgements.

S.F.Y. and T.L.P. would like to thank the AFOSR and the NSF for funding
through the CUA-PFC grant. T.L.P. acknowledges that this material is based
upon work supported by the National Science Foundation Graduate Research
Fellowship under Grant No. DGE-1745303. X.G. is supported by the Postdoctoral
Fellowship in Quantum Science of the Harvard-MPQ Center for Quantum Optics,
the Templeton Religion Trust grant TRT 0159, and by the Army Research Office
under Grant W911NF1910302 and MURI Grant W911NF-20-1-0082.

## References

  * Preskill (2018) J. Preskill, Quantum Computing in the NISQ era and beyond, Quantum 2, 79 (2018).
  * McClean  _et al._ (2016) J. R. McClean, J. Romero, R. Babbush, and A. Aspuru-Guzik, The theory of variational hybrid quantum-classical algorithms, New Journal of Physics 18, 023023 (2016).
  * Farhi  _et al._ (2014) E. Farhi, J. Goldstone, and S. Gutmann, A quantum approximate optimization algorithm, arXiv (2014), arXiv:1411.4028 .
  * Peruzzo  _et al._ (2014) A. Peruzzo, J. McClean, P. Shadbolt, M.-H. Yung, X.-Q. Zhou, P. J. Love, A. Aspuru-Guzik, and J. L. O’Brien, A variational eigenvalue solver on a photonic quantum processor, Nature Communications 5, 4213 (2014).
  * Yung  _et al._ (2014) M.-H. Yung, J. Casanova, A. Mezzacapo, J. McClean, L. Lamata, A. Aspuru-Guzik, and E. Solano, From transistor to trapped-ion computers for quantum chemistry, Scientific Reports 4, 10.1038/srep03589 (2014).
  * Wecker  _et al._ (2015) D. Wecker, M. B. Hastings, and M. Troyer, Progress towards practical quantum variational algorithms, Phys. Rev. A 92, 042303 (2015).
  * Kandala  _et al._ (2017) A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J. M. Chow, and J. M. Gambetta, Hardware-efficient Quantum Optimizer for Small Molecules and Quantum Magnets, Nature Publishing Group 549, 242 (2017), arXiv:1704.05018 .
  * Farhi and Neven (2018) E. Farhi and H. Neven, Classification with Quantum Neural Networks on Near Term Processors,  (2018), arXiv:1802.06002 .
  * McClean  _et al._ (2018) J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, Barren plateaus in quantum neural network training landscapes, Nature Communications 9, 4812 (2018).
  * Cerezo  _et al._ (2020) M. Cerezo, A. Sone, T. Volkoff, L. Cincio, and P. J. Coles, Cost-Function-Dependent Barren Plateaus in Shallow Quantum Neural Networks,  (2020), arXiv:2001.00550 .
  * Uvarov and Biamonte (2020) A. Uvarov and J. Biamonte, On barren plateaus and cost function locality in variational quantum algorithms,  (2020), arXiv:2011.10530 .
  * Arrasmith  _et al._ (2020) A. Arrasmith, M. Cerezo, P. Czarnik, L. Cincio, and P. J. Coles, Effect of barren plateaus on gradient-free optimization,  (2020), arXiv:2011.12245 .
  * Anand  _et al._ (2020) A. Anand, M. Degroote, and A. Aspuru-Guzik, Natural Evolutionary Strategies for Variational Quantum Computation,  (2020), arXiv:2012.00101 .
  * Sharma  _et al._ (2020) K. Sharma, M. Cerezo, L. Cincio, and P. J. Coles, Trainability of dissipative perceptron-based quantum neural networks, arXiv (2020), arXiv:2005.12458 .
  * Wiersema  _et al._ (2020) R. Wiersema, C. Zhou, Y. de Sereville, J. F. Carrasquilla, Y. B. Kim, and H. Yuen, Exploring entanglement and optimization within the hamiltonian variational ansatz, PRX Quantum 1, 020319 (2020).
  * Skolik  _et al._ (2020) A. Skolik, J. R. McClean, M. Mohseni, P. van der Smagt, and M. Leib, Layerwise learning for quantum neural networks,  (2020), arXiv:2006.14904 .
  * Fontana  _et al._ (2020) E. Fontana, M. Cerezo, A. Arrasmith, I. Rungger, and P. J. Coles, Optimizing parametrized quantum circuits via noise-induced breaking of symmetries,  (2020), arXiv:2011.08763 .
  * Volkoff and Coles (2020) T. Volkoff and P. J. Coles, Large gradients via correlation in random parameterized quantum circuits,  (2020), arXiv:2005.12200 .
  * Grant  _et al._ (2019) E. Grant, L. Wossnig, M. Ostaszewski, and M. Benedetti, An initialization strategy for addressing barren plateaus in parametrized quantum circuits, Quantum 3, 214 (2019).
  * Pesah  _et al._ (2020) A. Pesah, M. Cerezo, S. Wang, T. Volkoff, A. T. Sornborger, and P. J. Coles, Absence of Barren Plateaus in Quantum Convolutional Neural Networks,  (2020), arXiv:2011.02966 .
  * Bremner  _et al._ (2009) M. J. Bremner, C. Mora, and A. Winter, Are random pure states useful for quantum computation?, Phys. Rev. Lett. 102, 190502 (2009).
  * Gross  _et al._ (2009) D. Gross, S. T. Flammia, and J. Eisert, Most quantum states are too entangled to be useful as computational resources, Phys. Rev. Lett. 102, 190501 (2009).
  * Holmes  _et al._ (2020) Z. Holmes, A. Arrasmith, B. Yan, P. J. Coles, A. Albrecht, and A. T. Sornborger, Barren plateaus preclude learning scramblers,  (2020), arXiv:2009.14808 .
  * Marrero  _et al._ (2020) C. O. Marrero, M. Kieferová, and N. Wiebe, Entanglement Induced Barren Plateaus,  (2020), arXiv:2010.15968 .
  * Wang  _et al._ (2020) S. Wang, E. Fontana, M. Cerezo, K. Sharma, A. Sone, L. Cincio, and P. J. Coles, Noise-Induced Barren Plateaus in Variational Quantum Algorithms,  (2020), arXiv:2007.14384 .
  * Verteletskyi  _et al._ (2020) V. Verteletskyi, T.-C. Yen, and A. F. Izmaylov, Measurement optimization in the variational quantum eigensolver using a minimum clique cover, J. Chem. Phys. 152, 10.1063/1.5141458 (2020).
  * Gokhale  _et al._ (2019) P. Gokhale, O. Angiuli, Y. Ding, K. Gui, T. Tomesh, M. Suchara, M. Martonosi, and F. T. Chong, Minimizing State Preparations in Variational Quantum Eigensolver by Partitioning into Commuting Families,  (2019), arXiv:1907.13623 .
  * Izmaylov  _et al._ (2019) A. F. Izmaylov, T.-C. Yen, R. A. Lang, and V. Verteletskyi, Unitary partitioning approach to the measurement problem in the Variational Quantum Eigensolver method,  (2019), arXiv:1907.09040 .
  * Verdon  _et al._ (2019) G. Verdon, M. Broughton, J. R. McClean, K. J. Sung, R. Babbush, Z. Jiang, H. Neven, and M. Mohseni, Learning to learn with quantum neural networks via classical neural networks,  (2019), arXiv:1907.05415 .
  * Welling and Teh (2011) M. Welling and Y. W. Teh, Bayesian learning via stochastic gradient langevin dynamics, Proceedings of the 28th International Conference on International Conference on Machine Learning (2011).
  * Dita (2003) P. Dita, Factorization of unitary matrices, Journal of Physics A: Mathematical and General 36, 2781 (2003).
  * Puchala and Miszczak (2017) Z. Puchala and J. Miszczak, Symbolic integration with respect to the haar measure on the unitary groups, Bulletin of the Polish Academy of Sciences: Technical Sciences 65, 21 (2017).
  * Shen  _et al._ (2020) H. Shen, P. Zhang, Y.-Z. You, and H. Zhai, Information scrambling in quantum neural networks, Phys. Rev. Lett. 124, 200504 (2020).
  * (34) M. Walters, Ph.d., University of Rochester, 2015.
  * Renes  _et al._ (2004) J. M. Renes, R. Blume-Kohout, A. J. Scott, and C. M. Caves, Symmetric informationally complete quantum measurements, Journal of Mathematical Physics 45, 2171 (2004), https://doi.org/10.1063/1.1737053 .
  * Dankert  _et al._ (2009) C. Dankert, R. Cleve, J. Emerson, and E. Livine, Exact and approximate unitary 2-designs and their application to fidelity estimation, Phys. Rev. A 80, 012304 (2009).
  * Harrow and Low (2009) A. W. Harrow and R. A. Low, Random quantum circuits are approximate 2-designs, Communications in Mathematical Physics 291, 257 (2009).
  * Žndarič (2020) M. Žndarič, Entanglement growth in diffusive systems, Communications Physics 3, 100 (2020).
  * Reddi  _et al._ (2018) S. J. Reddi, S. Kale, and S. Kumar, On the convergence of adam and beyond, International Conference on Learning Representations (2018).
  * Thrun and Pratt (1998) S. Thrun and L. Pratt,  _Learning to Learn: Introduction and Overview_ (1998).
  * Campos  _et al._ (2020) E. Campos, A. Nasrallah, and J. Biamonte, Abrupt Transitions in Variational Quantum Circuit Training,  (2020), arXiv:2010.09720 .
  * Brydges  _et al._ (2019) T. Brydges, A. Elben, P. Jurcevic, B. Vermersch, C. Maier, B. P. Lanyon, P. Zoller, R. Blatt, and C. F. Roos, Probing rényi entanglement entropy via randomized measurements, Science 364, 260 (2019).
  * Lewis-Swan  _et al._ (2019) R. J. Lewis-Swan, A. Safavi-Naini, J. J. Bollinger, and A. M. Rey, Unifying scrambling, thermalization and entanglement through measurement of fidelity out-of-time-order correlators in the dicke model, Nat Commun 10, 1581 (2019).
  * Zhang  _et al._ (2019) X.-M. Zhang, Z. Wei, R. Asad, X.-C. Yang, and X. Wang, When does reinforcement learning stand out in quantum control? A comparative study on state preparation, npj Quantum Information 1, 85 (2019).
  * Coffey  _et al._ (1996) W. Coffey, Y. P. Kalmykov, and J. T. Waldron,  _The Langevin Equation: With Applications in Physics, Chemistry and Electrical Engineering_ (1996).


# Evaluating Predictive Uncertainty and Robustness to Distributional Shift
Using Real World Data

Kumud Lakara
Dept. of Computer Science and Engineering
Manipal Institute of Technology, MAHE
Manipal, India
<EMAIL_ADDRESS>
Akshat Bhandari 11footnotemark: 1
Dept. of Computer Science and Engineering
Manipal Institute of Technology, MAHE
Manipal, India
<EMAIL_ADDRESS>Pratinav Seth 11footnotemark: 1
Dept. of Computer Applications
Manipal Institute of Technology, MAHE
Manipal, India
<EMAIL_ADDRESS>
Ujjwal Verma
Dept. of Electronics and Comm. Engineering
Manipal Institute of Technology, MAHE
Manipal, India
<EMAIL_ADDRESS>
Equal contribution.

###### Abstract

Most machine learning models operate under the assumption that the training,
testing and deployment data is independent and identically distributed
(i.i.d.). This assumption does not generally hold true in a natural setting.
Usually, the deployment data is subject to various types of distributional
shifts. The magnitude of a model’s performance is proportional to this shift
in the distribution of the dataset. Thus it becomes necessary to evaluate a
model’s uncertainty and robustness to distributional shifts to get a realistic
estimate of its expected performance on real-world data. Present methods to
evaluate uncertainty and model’s robustness are lacking and often fail to
paint the full picture. Moreover, most analysis so far has primarily focused
on classification tasks. In this paper, we propose more insightful metrics for
general regression tasks using the Shifts Weather Prediction Dataset. We also
present an evaluation of the baseline methods using these metrics.

##  1 Introduction

Recent times have seen the growing deployment of machine learning models and
deep neural networks in many mission-critical tasks such as medical image
diagnosis Ker et al. [2017], banking systems Boobier [2020] and autonomous
vehicles Nascimento et al. [2018]. A prevalent assumption in machine learning
is that the data the model sees after the development phase is independent and
identically distributed (i.i.d). This implies that if a model performs well
during the development phase it will be able to interpolate that performance
to deployment. This however is seldom the case. Most data available to models
in the real world is often out of the distribution of the data on which the
models were trained and tested. When dealing with critical applications, the
model’s inability to generalize to the out of distribution deployment data can
have disastrous consequences. Therefore, such tasks require not only for
models to make accurate predictions but also a precise quantification of
predictive uncertainty Amodei et al. [2016]. Thus, it is imperative to
properly assess a model’s robustness to distribution shift and its estimation
of predictive uncertainty.

While the problem is apparent, yet there are limited resources and datasets
that allow for proper evaluation of uncertainty estimates and robustness to
distributional shift emulating real-world data. Most of the available datasets
such as Imagenet - C Hendrycks and Dietterich [2019], A Hendrycks et al.
[2021], R Hendrycks et al. [2020], O Hendrycks et al. [2021] and WILDS Koh et
al. [2021] focus primarily on image classification tasks. The recently
introduced Shifts Dataset Malinin et al. [2021] provides a favourable data
setting. It is composed of three parts each corresponding to a different data
modality: tabular weather prediction data, machine translation data and self-
driving car data. All the data modalities are affected by distributional shift
and pose challenges with respect to evaluating uncertainty predictions and
robustness against distributional shift.

In this work, we present novel evaluation metrics for predictions on the
tabular data related to weather prediction in the Shifts Dataset Malinin et
al. [2021]. We introduce metrics for effectively evaluating both uncertainty
predictions and robustness to distributional shift for regression tasks. We
believe these metrics will be a much needed addition to the scarce pool of
metrics available presently for regression tasks. In addition to this, we also
validate the proposed metrics by evaluating the performance of baselines for
predicting the temperature at a particular latitude/longitude and time, given
all available measurements and climate model predictions.111
https://github.com/kumudlakara/Shifts_evaluation_metrics_regression

##  2 Evaluation Metrics

We believe a model with lower degradation in performance on shifted data is
more robust to distribution shift. The quality of predictive uncertainty is
governed by the model’s ability to discriminate between i.i.d. and out of
distribution data. We present metrics for the joint evaluation of predictive
uncertainty and robustness to distributional shift. We validate our proposed
metrics using the baseline Gradient Boosted Decision Trees (GBDT) models as
used in Malinin et al. [2021].

To build baseline gradient boosted decision trees for the temperature
prediction regression task, the open-source CatBoost gradient boosting library
Prokhorenkova et al. [2017] is used. Similar to Malinin et. al Malinin et al.
[2021], we consider an ensemble-based approach to uncertainty estimation for
GBDT models Ustimenko et al. [2021]. We use a pre-trained ensemble of ten
models on the train data from the canonical partition of the Weather
Prediction dataset with different random seeds. The models are optimized with
the loss function `RMSEWithUncertainty` Ustimenko et al. [2021] which predicts
mean and variance of the normal distribution by optimizing the negative log-
likelihood. Hyperparameter tuning is performed on the in-domain development
data. Each model is constructed with a depth of 8 and is then trained for
20,000 iterations with a learning rate of 0.3.

###  2.1 RMSE/RMV Ratio

The root mean squared error (RMSE) and the root mean variance (RMV) can be
viewed as indicators of the model accuracy and uncertainty respectively.

$RMSE=\sqrt{\frac{1}{|B|}\sum_{b\epsilon B}(y_{b}-\hat{y_{b}})^{2}}$ (1)
$RMV=\sqrt{\frac{1}{|B|}\sum_{b\epsilon B}\sigma_{b}^{2}}$ (2)

where $\sigma_{b}^{2}$ is the variance, $|B|$ is the bin size and
$y_{b}-\hat{y_{b}}$ is the error between the model’s prediction and the ground
truth value for example b $\epsilon$ B. The ratio of the two should be as
close to one as possible. We reason this inference by considering the
following paradigm:

_If the error (RMSE) for a prediction is high then the model’s uncertainty
(RMV) about that prediction should also be high._

When plotting the retention curve Malinin [2019]Lakshminarayanan et al. [2016]
for the RMSE/RMV ratio, an ideal scenario would be a horizontal line close to
one. This would indicate that RMSE is roughly equal to RMV. Figure 1(a) shows
the RMSE/RMV retention curves (R3-curves) for single and ensemble models. The
ensemble model clearly displays a more linear relationship between RMSE and
RMV compared to the single model. The RMSE/RMV ratio is also much closer to
one for the ensemble model compared to the single model.

###  2.2 Logarithmic Expected Normalized Calibration Error (LENCE)

Levi et al. Levi et al. [2019] propose a definition for calibration for
regression by replacing mis-classification probability with mean squared
error. This is called Expected Normalized Calibration Error (ENCE). This is an
improvement over the existing definition of calibrated regression uncertainty
Kuleshov et al. [2018]. To calculate ENCE we arrange the data in increasing
order of variance. Then we create bins of size S such that S divides the total
number of examples into N bins. Each resulting bin represents an interval in
the standard deviation axis. These intervals are non-overlapping and their
boundary values are in increasing order Levi et al. [2019]. This measure is
now analogous to the Expected Calibration Error (ECE)Naeini et al. [2015] used
in the case of classification problems. For finding the error in calibration
of the model, we use the following formulation for ENCE:

$ENCE=\frac{1}{N}\sum_{b=1}^{N}\frac{|RMV(b)-RMSE(b)|}{RMV(b)}$ (3)

where $N$ is the total number of bins. A shortcoming of ENCE is that if the
model were to predict a homogeneous uncertainty for each example and that
happened to match the model’s empirical uncertainty for the entire population
then ENCE would be zero. However, it would not be a correct evaluation of the
models uncertainty or generalization ability. Levi et. al Levi et al. [2019]
propose using the Variation Coefficient of Standard Deviation, $C_{v}$ as a
secondary diagnostic tool with ENCE.

$C_{v}=\frac{\sqrt{\frac{\sum_{t=1}^{T}(\sigma_{t}-\mu_{\sigma})^{2}}{T-1}}}{\mu_{\sigma}}$
(4)

where $\mu_{\sigma}=\frac{1}{T}\sum_{t=1}^{T}\sigma_{t}$ and T is the number
of training examples. Evaluating two separate metrics and drawing relative
inferences is complex and hence we propose the Logarithmic ENCE (LENCE). We
incorporate $C_{v}$ as a measure of dispersion of the uncertainty predictions
by the model and ENCE as a measure of the calibration error into a single
evaluation metric. We define LENCE as:

$LENCE=log|ENCE+\frac{1}{C_{v}}|$ (5)

Ideally, the uncertainty values will be well dispersed and $C_{v}$ would be
high. Then LENCE will approximately be $log|ENCE|$. In the absolute worst case
when $ENCE=0$ and $C_{v}\rightarrow 0$; $LENCE\rightarrow\infty$. ENCE,
$C_{v}$ and LENCE values for single and ensemble models are presented in Table
1. A lower LENCE value for the ensemble model indicates its superiority.

Table 1: ENCE, $C_{v}$ and LENCE for baseline GBDT ensemble and single models Model | ENCE | $C_{v}$ | LENCE
---|---|---|---
Ensemble | 1.084 | 0.387 | 1.2968
Single | 3.551 | 0.550 | 1.6804
Figure 1: (a) RMSE/RMV retention curve (b) MWSE retention curve

###  2.3 LL-Fisher Uncertainty (LL-FU)

We introduce the LL-Fisher metric as a joint measure of uncertainty and
robustness to distributional shift. The LL-Fisher metric measures the
localisation of a probability distribution function Villani [2002]. This
metric can be used to evaluate how well a prediction represents the
distribution of data that the model used to make the prediction itself.
Inspired by log-loss used for calculating Fisher Information Ly et al. [2017]
for a prediction _x_ with mean µ and variance $\sigma^{2}$ following a
continuous distribution.

Figure 2: (a) LL-FU vs error scatter plot (b) LL-FU F1 retention curve (c) MSE
- LLFU retention curve

We propose the LL-Fisher Uncertainty as follows:

$LL\scalebox{0.8}[1.0]{-}FU=max(0,\frac{1}{2}log(2\pi\sigma^{2}))+\frac{(x-\mu)^{2}}{2\sigma^{2}}$
(6)

We plot the F1-retention curve Figure 2(b), the MSE-retention curve Figure
2(c) and calculate the summary statistics: F1-AUC and R-AUC for the evaluation
of our metric. We use F1@95 score to jointly evaluate uncertainty and
robustness. A good uncertainty measure should achieve low R-AUC, high F1-AUC
and high F1@95 scores Malinin et al. [2021]. We calculated these metrics for
various uncertainty measures of prediction using a GBDT model as well as an
ensemble of these models.

Table 2: Ensemble model F1-AUC, F1@95 and R-AUC scores for uncertainty measures on the dev set of the Shifts Weather Prediction Dataset | Uncertainty Metric
---|---
| mvar | tvar | varm | epkl | LL-FU
F1-AUC | 0.4951 | 0.5220 | 0.5012 | 0.5051 | 0.7009
F1@95 | 0.6581 | 0.6583 | 0.6579 | 0.6551 | 0.6778
R-AUC | 1.4186 | 1.2708 | 1.2868 | 1.2266 | 0.5838
Table 3: Single model F1-AUC, F1@95 and R-AUC scores for uncertainty measures on the dev set of the Shifts Weather Prediction Dataset. | Uncertainty Metric
---|---
| mvar | tvar | varm | epkl | LL-FU
F1-AUC | 0.4425 | 0.4425 | 0.3851 | 0.4425 | 0.6873
F1@95 | 0.6276 | 0.6276 | 0.6115 | 0.6276 | 0.6478
R-AUC | 1.8868 | 1.8868 | 1.9526 | 1.8868 | 0.7740

Results using ensemble and single models for various uncertainty measures are
depicted in Table 2 and Table 3 showing superiority of our uncertainty
measure. We compare both scores for ensemble and single model for LLFU. The
scores for ensemble are better than the single model indicating the
superiority of the ensembling approach. These results are presented in Table 4
.

We observe that for out of distribution data, as the error of prediction
increases, the lower bound of uncertainty (LL-FU) also increases. This
validates the idea that with increasing shift in data distribution, not only
does the likelihood of an error increase but so does the uncertainty
corresponding to the prediction. We believe that LL-FU is unique in capturing
this property when compared to other uncertainty metrics Malinin et al. [2021]
as reflected in Figure 2(a).

Table 4: LL-FU for baseline GBDT ensemble and single models Model | F1-AUC | F1@95 | R-AUC
---|---|---|---
Ensemble | 0.7009 | 0.6778 | 0.5838
Single | 0.6873 | 0.6478 | 0.7740

###  2.4 Mean Weighted Squared Error (MWSE)

Intuitively, a model should be increasingly uncertain about it’s prediction as
the magnitude of distributional shift increases and hence will make more
erroneous decisions. The error-retention curves Malinin [2019]
Lakshminarayanan et al. [2016] do not directly evaluate the uncertainty
estimates of the model and only take into account the correlation between
error and uncertainty estimates. To more resiliently evaluate robustness and
also take into account the magnitude of estimated uncertainty, we introduce
MWSE. We propose MWSE as a secondary metric only to gain more insight into the
models’ behaviour by analysing their relative divergence.

$MWSE=\frac{1}{N}\sum_{i=1}^{i=N}(\hat{y_{i}}-y)^{2}*(UNC)_{i}$ (7)

where _$\hat{y_{i}}$_ and y is the predicted and true value respectively and
UNC is model’s uncertainty estimate for each prediction.

Ideally datapoints with low MSE should have low uncertainty and those with
high MSE should have high uncertainty which would mean their product would be
very low or very high respectively. In Figure 1(c) the ensemble diverges more
than the single model. As the retention fraction increases, the ensemble shows
a greater divergence from the single model. From this observation we infer
that the ensemble is comparatively more responsive to distributional shift
than the single model. MWSE assesses the sensitivity of a model to
distribution shift by using error and uncertainty in conjunction with one
another. The MWSE-retention curve when studied in tandem with the MSE-
retention curve gives more insight into the quality of estimated uncertainty.

##  3 Conclusion and Future Work

This work presents three primary metrics (RMSE/RMV ratio, LENCE and LL-FU) to
quantify and evaluate predictive uncertainty under dataset shift. We used the
Shifts Weather Prediction DatasetMalinin et al. [2021] for validation of the
proposed metrics. Through these metrics we aimed to jointly evaluate
predictive uncertainty and robustness of a model and its performance under
dataset shift. While we focused only on regression tasks, we believe there is
potential for these metrics to be extrapolated to classification as well as
machine translation tasks. Another avenue of future research can be to
incorporate the memory and computational efficiency of different methods into
the evaluation metrics. We hope our work helps the community and inspires
further research on evaluation metrics for shifted multi-modal data.

## Acknowledgments and Disclosure of Funding

We would like to thank _Mars Rover Manipal_ for providing the necessary
resources for our research.

## References

  * Amodei et al. [2016] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané.  Concrete problems in ai safety.  _arXiv preprint arXiv:1606.06565_ , 2016.
  * Boobier [2020] T. Boobier.  _AI and the Future of Banking_.  John Wiley & Sons, 2020.
  * Hendrycks and Dietterich [2019] D. Hendrycks and T. Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hendrycks et al. [2020] D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  _arXiv preprint arXiv:2006.16241_ , 2020.
  * Hendrycks et al. [2021] D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song.  Natural adversarial examples.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 15262–15271, 2021.
  * Ker et al. [2017] J. Ker, L. Wang, J. Rao, and T. Lim.  Deep learning applications in medical image analysis.  _Ieee Access_ , 6:9375–9389, 2017.
  * Koh et al. [2021] P. W. Koh, S. Sagawa, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, T. Lee, et al.  Wilds: A benchmark of in-the-wild distribution shifts.  In _International Conference on Machine Learning_ , pages 5637–5664. PMLR, 2021.
  * Kuleshov et al. [2018] V. Kuleshov, N. Fenner, and S. Ermon.  Accurate uncertainties for deep learning using calibrated regression.  In _International Conference on Machine Learning_ , pages 2796–2804. PMLR, 2018.
  * Lakshminarayanan et al. [2016] B. Lakshminarayanan, A. Pritzel, and C. Blundell.  Simple and scalable predictive uncertainty estimation using deep ensembles.  _arXiv preprint arXiv:1612.01474_ , 2016.
  * Levi et al. [2019] D. Levi, L. Gispan, N. Giladi, and E. Fetaya.  Evaluating and calibrating uncertainty prediction in regression tasks.  _arXiv preprint arXiv:1905.11659_ , 2019.
  * Ly et al. [2017] A. Ly, M. Marsman, J. Verhagen, R. P. Grasman, and E.-J. Wagenmakers.  A tutorial on fisher information.  _Journal of Mathematical Psychology_ , 80:40–55, 2017.
  * Malinin [2019] A. Malinin.  _Uncertainty estimation in deep learning with application to spoken language assessment_.  PhD thesis, University of Cambridge, 2019.
  * Malinin et al. [2021] A. Malinin, N. Band, G. Chesnokov, Y. Gal, M. J. Gales, A. Noskov, A. Ploskonosov, L. Prokhorenkova, I. Provilkov, V. Raina, et al.  Shifts: A dataset of real distributional shift across multiple large-scale tasks.  _arXiv preprint arXiv:2107.07455_ , 2021.
  * Naeini et al. [2015] M. P. Naeini, G. Cooper, and M. Hauskrecht.  Obtaining well calibrated probabilities using bayesian binning.  In _Twenty-Ninth AAAI Conference on Artificial Intelligence_ , 2015\.
  * Nascimento et al. [2018] A. M. Nascimento, L. F. Vismari, P. S. Cugnasca, J. Camargo, J. de Almeida, R. Inam, E. Fersman, A. Hata, and M. Marquezini.  Concerns on the differences between ai and system safety mindsets impacting autonomous vehicles safety.  In _International Conference on Computer Safety, Reliability, and Security_ , pages 481–486. Springer, 2018.
  * Prokhorenkova et al. [2017] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin.  Catboost: unbiased boosting with categorical features.  _arXiv preprint arXiv:1706.09516_ , 2017.
  * Ustimenko et al. [2021] A. Ustimenko, L. Prokhorenkova, and A. Malinin.  Uncertainty in gradient boosting via ensembles.  _ArXiv_ , abs/2006.10562, 2021.
  * Villani [2002] C. Villani.  A review of mathematical topics in collisional kinetic theory.  _Handbook of mathematical fluid dynamics_ , 1(71-305):3–8, 2002.