radius $r>0$ such that, for every $\hat{y}\in Y$ and $\mu\in(0,\mu_{g})$, the
function $\mathcal{L}_{\mu}(\cdot,\hat{y})$ admits a local minimizer
$x(\mu,\hat{y})$ which lies in $\operatorname{\mathbb{B}}_{r}(\bar{x})$.
Moreover, $x(\mu,\hat{y})\to\bar{x}$ uniformly on $Y$ as $\mu\downarrow 0$.

######  Proof 5.11.

Let $r>0$ be as in Lemma 5.8. For $\mu\in(0,\mu_{g})$, the Moreau envelope
$g^{\mu}$ is a continuous function, and this extends to
$\mathcal{L}_{\mu}(\cdot,\hat{y})$ for arbitrary $\hat{y}\in Y$. Hence, this
function possesses a global minimizer over
$\operatorname{\mathbb{B}}_{r}(\bar{x})$ which we denote by $x(\mu,\hat{y})$.
As $\mu\downarrow 0$, we find $x(\mu,\hat{y})\to\bar{x}$ from Lemma 5.8, and
this convergence is uniform for $\hat{y}\in Y$.

Let us note that Lemma 5.8 and Theorem 5.10 are based on WSOSC from Definition
4.8, which does not demand uniqueness of the underlying Lagrange multiplier.

####  5.3.2 Rates of convergence

Our rates-of-convergence analysis of Algorithm 5.1 is based on a primal-dual
pair $(\bar{x},\bar{y})\in\mathbb{R}^{n}\times\mathbb{R}^{m}$ which solves the
M-stationarity system (4.3) associated with (P) such that an (upper) error
bound condition is valid.

For brevity of notation, we will partially make use of the following
assumptions.

######  Assumption 7 (Rates of convergence).

The following hold.

  1. (i)

Let $\bar{x}\in\mathbb{R}^{n}$ be an M-stationary point of (P), and let
$\bar{y}\in\Lambda(\bar{x})$ be chosen such that there are a constant
$\varrho_{\mathrm{u}}>0$ and a neighborhood $U$ of
$(\bar{x},c(\bar{x}),\bar{y})$ such that the upper error bound condition
(4.17) holds for each triplet $(x,z,y)\in
U\cap(\mathbb{R}^{n}\times\operatorname{dom}g\times\mathbb{R}^{m})$.

  2. (ii)

Let $\\{(x^{k},z^{k},y^{k})\\}$ be a sequence generated by Algorithm 5.1 with
$\varepsilon_{k}\to 0$.

  3. (iii)

The primal-dual sequence $\\{(x^{k},y^{k})\\}$ converges to
$(\bar{x},\bar{y})$.

  4. (iv)

For each $k\in\mathbb{N}$ large enough, $\hat{y}^{k}=y^{k-1}$ is valid.

Note that we already know, by Theorem 5.10, that the AL admits approximate
local minimizers and stationary points in a neighborhood of some M-stationary
point $\bar{x}\in\mathbb{R}^{n}$ which satisfies WSOSC. We shall now see that,
under the error bound conditions from Section 4.3 involving a fixed multiplier
$\bar{y}\in\Lambda(\bar{x})$, if the algorithm chooses these local minimizers
(or any other points sufficiently close to $\bar{x}$), then we automatically
obtain the convergence $(x^{k},z^{k},y^{k})\to(\bar{x},c(\bar{x}),\bar{y})$.
In this case, the sequence $\\{y^{k}\\}$ is necessarily bounded, so it is
reasonable to assume that the safeguarded multipliers are eventually chosen as
$\hat{y}^{k}=y^{k-1}$. Let us recall that even validity of SSOSC at $\bar{x}$
for $\bar{y}$ may not be sufficient for the error bound condition, see Remark
4.23. However, Section 4.3 provides a number of sufficient conditions which
still can be checked in terms of initial problem data, so we will abstain here
from postulating any more specific assumptions on the upper error bound.
Furthermore, we do not stipulate any lower error bound conditions, deviating
from all other related papers, where the lower estimate was never problematic,
see Remark 4.26.

The following result, which is motivated by [58, Prop. 4.29], can be
considered as (retrospective) justification for Assumption 7(iii)–(iv) in the
presence of Assumption 7(i)–(ii). Besides the error bound condition, a CQ is
needed. As we require an M-stationary point of (P), this is not too
restrictive.

######  Proposition 5.12.

Let Assumption 7(i)–(ii) hold and suppose that (at least) one of the following
conditions is valid:

  1. (a)

$c^{\prime}(\bar{x})$ possesses full row rank $m$;

  2. (b)

condition (4.6) is valid, $\operatorname{dom}g$ is closed, and $g$ is
continuous relative to its domain.

Then there exists a radius $r>0$ such that, if
$x^{k}\in\operatorname{\mathbb{B}}_{r}(\bar{x})$ for all sufficiently large
$k\in\mathbb{N}$, then we have the convergences $\Theta(x^{k},z^{k},y^{k})\to
0$ and $(x^{k},z^{k},y^{k})\to(\bar{x},c(\bar{x}),\bar{y})$ as $k\to\infty$.

######  Proof 5.13.

Let $r>0$ be small enough so that the upper estimate in the error bound
condition (4.17) holds for all
$(x,z,y)\in\mathbb{R}^{n}\times\operatorname{dom}g\times\mathbb{R}^{m}$ with
$x\in\operatorname{\mathbb{B}}_{r}(\bar{x})$. Assume now that
$x^{k}\in\operatorname{\mathbb{B}}_{r}(\bar{x})$ for all $k\in\mathbb{N}$
sufficiently large. The proof is divided into multiple steps.

We first show that $c(x^{k})-z^{k}\to 0$ as $k\to\infty$. Consider two cases.
If $\\{\mu_{k}\\}$ remains bounded away from zero, this assertion readily
follows from the penalty updating scheme at Algorithm 5.1. Instead, if
$\mu_{k}\downarrow 0$, then we can argue from Proposition 5.1 that

(5.5)
$c^{\prime}(x^{k})^{\top}\left[c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\right]\to 0$

as $\mu_{k}\downarrow 0$, by boundedness of $\\{\nabla f(x^{k})\\}$,
$\\{\hat{y}^{k}\\}$, and $\\{\varepsilon_{k}\\}$. Let us now show
$c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\to 0$, which readily yields
$c(x^{k})-z^{k}\to 0$ since $\\{\hat{y}^{k}\\}$ is bounded and
$\mu_{k}\downarrow 0$. In case (a) where $c^{\prime}(\bar{x})$ has full row
rank, the matrices $c^{\prime}(x)c^{\prime}(x)^{\top}$ are uniformly
invertible on $\operatorname{\mathbb{B}}_{r}(\bar{x})$, potentially after
shrinking $r$, and (5.5) gives $c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\to 0$. Next,
for case (b), assume that (4.6) holds while $\operatorname{dom}g$ is closed
and $g$ is continuous on its domain. Note that, for each $k\in\mathbb{N}$, we
even have
$\mu_{k}^{-1}(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k})\in\widehat{\partial}g(z^{k})$
by definition of the prox-operator and compatibility of the regular
subdifferential with smooth additions, and this also gives
$(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k},-\mu_{k})\in\widehat{N}_{\operatorname{epi}g}(z^{k},g(z^{k}))$.
Recall that (4.6) is equivalent to the metric regularity of $\Xi$ from (4.1)
at $((\bar{x},g(c(\bar{x}))),(0,0))$. Lemma 2.6 now yields the existence of
$s>0$ such that, for all sufficiently large $k\in\mathbb{N}$, we have

(5.6)
$\operatorname{\mathbb{B}}_{s}(0,0)\subseteq\begin{bmatrix}c^{\prime}(x^{k})&0\\\
0&1\end{bmatrix}\operatorname{\mathbb{B}}_{1}(0,0)+\operatorname{conv}\bigl{(}T_{\operatorname{epi}g}(z^{k},g(z^{k}))\cap\operatorname{\mathbb{B}}_{1}(0,0)\bigr{)}$

as $g$ is continuous on $\operatorname{dom}g$. In order to see this, we need
to make sure that $(z^{k},g(z^{k}))$ is sufficiently close to
$(c(\bar{x}),g(c(\bar{x})))$ for large enough $k\in\mathbb{N}$, and due to the
continuity of $g$, this boils down to showing that $z^{k}$ is sufficiently
close to $c(\bar{x})$ for large enough $k\in\mathbb{N}$.

Along the tail of the sequence (without relabeling), we have that
$\\{x^{k}\\}$ is close to $\bar{x}$. For every $k$, the optimality of $z^{k}$
in the proximal minimization subproblem reads

$\forall
z\in\mathbb{R}^{m}\colon\qquad\mu_{k}g(z^{k})+\frac{1}{2}\|c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\|^{2}\leq\mu_{k}g(z)+\frac{1}{2}\|c(x^{k})+\mu_{k}\hat{y}^{k}-z\|^{2}.$

Taking the specific choice $z\coloneqq c(\bar{x})\in\operatorname{dom}g$ and
dividing both sides by $\mu_{k}>0$ results in

$g(z^{k})+\frac{1}{2\mu_{k}}\|c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\|^{2}\leq
g(c(\bar{x}))+\frac{1}{2\mu_{k}}\|c(x^{k})+\mu_{k}\hat{y}^{k}-c(\bar{x})\|^{2}<\infty.$

By invoking the triangle, Cauchy–Schwarz, and Young’s inequalities, this
implies that

$\displaystyle\|z^{k}-c(\bar{x})\|^{2}{}={}$
$\displaystyle\|z^{k}-[c(x^{k})+\mu_{k}\hat{y}^{k}]-c(\bar{x})+[c(x^{k})+\mu_{k}\hat{y}^{k}]\|^{2}$
$\displaystyle{}\leq{}$
$\displaystyle\|z^{k}-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|^{2}$
$\displaystyle\qquad+2\|z^{k}-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|\|c(\bar{x})-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|$
$\displaystyle\qquad+\|c(\bar{x})-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|^{2}$
$\displaystyle{}\leq{}$ $\displaystyle
2\|z^{k}-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|^{2}+2\|c(\bar{x})-[c(x^{k})+\mu_{k}\hat{y}^{k}]\|^{2}$
$\displaystyle{}\leq{}$ $\displaystyle
4\left[\mu_{k}g(c(\bar{x}))-\mu_{k}g(z^{k})+\|c(x^{k})+\mu_{k}\hat{y}^{k}-c(\bar{x})\|^{2}\right].$

Rearranging gives

$\mu_{k}g(z^{k})+\frac{1}{4}\|z^{k}-c(\bar{x})\|^{2}\leq\mu_{k}g(c(\bar{x}))+\|c(x^{k})+\mu_{k}\hat{y}^{k}-c(\bar{x})\|^{2}.$

Since $c(\bar{x})\in\operatorname{dom}g$, the term $\mu_{k}g(c(\bar{x}))$
vanishes as $\mu_{k}\downarrow 0$, and so does $\mu_{k}\hat{y}^{k}$.
Therefore, possibly shrinking the neighborhood considered around $\bar{x}$,
the right-hand side remains bounded by some arbitrarily small $C>0$ for all
large $k\in\mathbb{N}$, by continuous differentiability of $c$, i.e.,

$\mu_{k}g(z^{k})+\frac{1}{4}\|z^{k}-c(\bar{x})\|^{2}\leq C$

holds for all $k\in\mathbb{N}$ large enough. By virtue of the prox-boundedness
of $g$, [19, Lem. 4.1] yields boundedness of $\\{z^{k}\\}$ and, thus, of
$\\{g(z^{k})\\}$ by continuity of $g$ on its domain which is assumed to be
closed (Heine’s theorem yields uniform continuity of $g$ on closed, bounded
subsets of $\operatorname{dom}g$). As $\mu_{k}\downarrow 0$ and $C>0$ can be
made arbitrarily small if only $r>0$ is chosen small enough, it follows that
$\\{z^{k}\\}$ is arbitrarily close to $c(\bar{x})$ for all large enough
$k\in\mathbb{N}$.

Pick $w\in\operatorname{\mathbb{B}}_{s}(0)$ arbitrary. Then, for each
sufficiently large $k\in\mathbb{N}$, we can rely on (5.6) to find
$(u^{k},\alpha_{k})\in\operatorname{\mathbb{B}}_{1}(0,0)$ and
$(v^{k},\beta_{k})\in\operatorname{conv}\bigl{(}T_{\operatorname{epi}g}(z^{k},g(z^{k}))\cap\operatorname{\mathbb{B}}_{1}(0,0)\bigr{)}$
such that $(w,0)=(c^{\prime}(x^{k})u^{k}+v^{k},\alpha_{k}+\beta_{k})$. From
$(v^{k},\beta_{k})\in\operatorname{conv}T_{\operatorname{epi}g}(z^{k},g(z^{k}))$,
we find
$\langle(v^{k},\beta_{k}),(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k},-\mu_{k})\rangle\leq
0$ due to (2.5). Thus

$\displaystyle\langle w,c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\rangle$
$\displaystyle=\langle
c^{\prime}(x^{k})u^{k}+v^{k},c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\rangle$
$\displaystyle=\langle
u^{k},c^{\prime}(x^{k})^{\top}(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k})\rangle+\mu_{k}\beta_{k}$
$\displaystyle\qquad+\langle(v^{k},\beta_{k}),(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k},-\mu_{k})\rangle$
$\displaystyle\leq\langle
u^{k},c^{\prime}(x^{k})^{\top}(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k})\rangle+\mu_{k}\beta_{k}\to
0,$

where we used boundedness of $\\{u^{k}\\}$ and $\\{\beta_{k}\\}$ as well as
$\mu_{k}\downarrow 0$ and (5.5) Testing this with $w\coloneqq\pm
s(c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k})/\|c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\|$
gives $c(x^{k})+\mu_{k}\hat{y}^{k}-z^{k}\to 0$.

We now demonstrate that $\Theta(x^{k},z^{k},y^{k})\to 0$ as $k\to\infty$.
Observe that

$\nabla_{x}\mathcal{L}(x^{k},y^{k})=\nabla_{x}\mathcal{L}^{\mathrm{S}}(x^{k},z^{k},y^{k})=\nabla_{x}\mathcal{L}^{\mathrm{S}}_{\mu_{k}}(x^{k},z^{k},\hat{y}^{k})$

holds for all $k\in\mathbb{N}$ by construction of the dual update rule. Then
the first summand in $\Theta(x^{k},z^{k},y^{k})$ satisfies

$\|\nabla_{x}\mathcal{L}(x^{k},y^{k})\|=\|\nabla_{x}\mathcal{L}^{\mathrm{S}}_{\mu_{k}}(x^{k},z^{k},\hat{y}^{k})\|\leq\varepsilon_{k},$

which converges to zero by Assumption 7(ii). Hence, as $\|c(x^{k})-z^{k}\|\to
0$ was obtained previously, the second term in (4.15) vanishes, too. For the
third and last term, it remains to show that
$\operatorname{dist}(y^{k},\partial g(z^{k}))\to 0$. This, however, readily
follows from Proposition 5.1.

Finally, recall that $x^{k}\in\operatorname{\mathbb{B}}_{r}(\bar{x})$ for all
$k\in\mathbb{N}$ and that $\Theta(x^{k},z^{k},y^{k})\to 0$. Hence, the
convergence $(x^{k},z^{k},y^{k})\to(\bar{x},c(\bar{x}),\bar{y})$ is an
immediate consequence of (4.17).

Subsequently, we will prove convergence rates for the sequence
$\\{(x^{k},z^{k},y^{k})\\}$ in the presence of Assumption 7. Since the
distance of $(x^{k},z^{k},y^{k})$ to $(\bar{x},c(\bar{x}),\bar{y})$ admits an
estimate relative to the residual terms
$\Theta_{k}\coloneqq\Theta(x^{k},z^{k},y^{k})$ by (4.17), we will largely base
our analysis on the sequence $\\{\Theta_{k}\\}$, and the results on the
sequence $\\{(x^{k},z^{k},y^{k})\\}$ will follow directly. However, this
correspondence heavily relies on a _two-sided_ error bound, see the proof of
Theorem 5.18 below. In stark contrast to Remark 4.26, the following Lemma 5.14
shows that, along a sequence generated by Algorithm 5.1, a lower error bound
holds. This exploits the fact that, as a consequence of Proposition 5.1, the
distance-to-subdifferential in $\Theta$ does not play a role for the error
bound _at the iterates_. Therefore, complementing the upper estimate of
Assumption 7(i), a two-sided error bound becomes _algorithmically_ available,
enabling the derivation of convergence rates.

######  Lemma 5.14.

Let $\bar{x}\in\mathbb{R}^{n}$ be an M-stationary point of (P) and
$\bar{y}\in\Lambda(\bar{x})$ be arbitrary. Suppose Assumption 7(ii)–(iv) hold.
Then there are a constant $\varrho_{\mathrm{l}}>0$ and a neighborhood $U$ of
$(\bar{x},c(\bar{x}),\bar{y})$ such that, for each triplet
$(x^{k},z^{k},y^{k})\in
U\cap(\mathbb{R}^{n}\times\operatorname{dom}g\times\mathbb{R}^{m})$, we have

(5.7)
$\varrho_{\mathrm{l}}\,\Theta(x^{k},z^{k},y^{k})\leq\|x^{k}-\bar{x}\|+\|z^{k}-c(\bar{x})\|+\|y^{k}-\bar{y}\|.$

######  Proof 5.15.

For each triplet
$(x^{k},z^{k},y^{k})\in\mathbb{R}^{n}\times\operatorname{dom}g\times\mathbb{R}^{m}$,
we can exploit (4.3) and the triangle inequality to obtain

$\displaystyle\Theta(x^{k},z^{k},y^{k}){}={}$
$\displaystyle\|\nabla_{x}\mathcal{L}(x^{k},y^{k})\|+\|c(x^{k})-z^{k}\|$
$\displaystyle{}\leq{}$
$\displaystyle\|\nabla_{x}\mathcal{L}(x^{k},y^{k})-\nabla_{x}\mathcal{L}(\bar{x},\bar{y})\|+\|c(x^{k})-c(\bar{x})\|+\|z^{k}-c(\bar{x})\|,$

where the equality is due to Assumption 7(ii) and Proposition 5.1, which imply
$\operatorname{dist}(y^{k},\partial g(z^{k}))=0$ for all $k\in\mathbb{N}$.
Then, noting that $\nabla_{x}\mathcal{L}$ and $c$ are locally Lipschitz
continuous by Section 1(i), the claim follows.

Our next result has been inspired by [58, Lem. 4.30].

######  Lemma 5.16.

Let Assumption 7 hold and set $\Theta_{k}\coloneqq\Theta(x^{k},z^{k},y^{k})$
for each $k\in\mathbb{N}$. Then

$(1-\varrho_{\mathrm{u}}\mu_{k})\Theta_{k}\leq\varepsilon_{k}+\varrho_{\mathrm{u}}\mu_{k}\Theta_{k-1}$

for all $k\in\mathbb{N}$ large enough, where $\varrho_{\mathrm{u}}>0$ is the
constant from (4.17).

######  Proof 5.17.

Due to Proposition 5.1, $y^{k}\in\partial g(z^{k})$ holds for all
$k\in\mathbb{N}$. Then, by (4.15) and Algorithm 5.1, we have

$\Theta_{k}\leq\varepsilon_{k}+\|c(x^{k})-z^{k}\|=\varepsilon_{k}+\mu_{k}\|y^{k}-y^{k-1}\|,$

where the equality is due to the update rule at Algorithm 5.1 and Assumption
7(iv). Using the triangle inequality yields

$\Theta_{k}\leq\varepsilon_{k}+\mu_{k}\|y^{k}-\bar{y}\|+\mu_{k}\|y^{k-1}-\bar{y}\|.$

By Assumption 7(i), since $x^{k}\to\bar{x}$ and, due to Proposition 5.12,
$z^{k}\to c(\bar{x})$, we find
$\|y^{k}-\bar{y}\|\leq\varrho_{\mathrm{u}}\Theta_{k}$ for all $k\in\mathbb{N}$
large enough. Hence,

$\Theta_{k}\leq\varepsilon_{k}+\varrho_{\mathrm{u}}\mu_{k}(\Theta_{k}+\Theta_{k-1})$

holds for all $k\in\mathbb{N}$ large enough, and reordering gives the
assertion.

With the above lemma and the two-sided error bound enabled by Assumption 7(i)
and Lemma 5.14, one can deduce convergence rates for the sequence
$\\{(x^{k},z^{k},y^{k})\\}$, see [58, Thm 4.31] as well. Notice that the
condition $\varepsilon_{k}\in\mathpzc{o}(\Theta_{k-1})$ can be easily
guaranteed in practice. Exemplary, one could compute the next iterate
$(x^{k},z^{k},y^{k})$ with a precision
$\varepsilon_{k}\leq\nu_{k}\Theta_{k-1}$ where $\\{\nu_{k}\\}$ is a given null
sequence. It should be mentioned also that the value $\Theta_{k}$ from (4.15)
becomes algorithmically available thanks to Proposition 5.1, and can readily
be obtained by the dual update rule at Algorithm 5.1.

######  Theorem 5.18.

Let Assumption 7 hold and set $\Theta_{k}\coloneqq\Theta(x^{k},z^{k},y^{k})$
for each $k\in\mathbb{N}$. Assume that
$\varepsilon_{k}\in\mathpzc{o}(\Theta_{k-1})$. Then

  1. (a)

for every $q\in(0,1)$, there exists $\bar{\mu}(q)$ such that, if
$\mu_{k}\leq\bar{\mu}(q)$ for sufficiently large $k\in\mathbb{N}$, then
$(x^{k},z^{k},y^{k})\to(\bar{x},c(\bar{x}),\bar{y})$ Q-linearly with rate $q$.

  2. (b)

If $\mu_{k}\downarrow 0$, then
$(x^{k},z^{k},y^{k})\to(\bar{x},c(\bar{x}),\bar{y})$ Q-superlinearly.

######  Proof 5.19.

Let $k\in\mathbb{N}$ be large enough so that $\hat{y}^{k}=y^{k-1}$. By Lemma
5.16, if $\mu_{k}$ is small enough so that $1-\varrho_{\mathrm{u}}\mu_{k}>0$,
then

$\frac{\Theta_{k}}{\Theta_{k-1}}\leq\frac{\varrho_{\mathrm{u}}\mu_{k}}{1-\varrho_{\mathrm{u}}\mu_{k}}+\mathpzc{o}(1).$

The desired rates for $\\{(x^{k},z^{k},y^{k})\\}$ are an easy consequence of
the upper and lower estimates in (4.17) and Lemma 5.14, as these give

$\displaystyle\frac{\|x^{k}-\bar{x}\|+\|z^{k}-c(\bar{x})\|+\|y^{k}-\bar{y}\|}{\|x^{k-1}-\bar{x}\|+\|z^{k-1}-c(\bar{x})\|+\|y^{k-1}-\bar{y}\|}\leq\frac{\varrho_{\mathrm{u}}}{\varrho_{\mathrm{l}}}\frac{\varrho_{\mathrm{u}}\mu_{k}}{1-\varrho_{\mathrm{u}}\mu_{k}}+\mathpzc{o}(1)$

for all $k\in\mathbb{N}$ large enough.

The following result is analogous to [58, Cor. 4.32].

######  Corollary 5.20.

Let Assumption 7 hold and assume that the subproblems occurring at Algorithm
5.1 of Algorithm 5.1 are solved exactly, i.e., that $\varepsilon_{k}=0$ for
all $k\in\mathbb{N}$. Then $\\{\mu_{k}\\}$ remains bounded away from zero.

######  Proof 5.21.

For each $k\in\mathbb{N}$, we make use of $V_{k}\coloneqq\|c(x^{k})-z^{k}\|$
and $\Theta_{k}\coloneqq\Theta(x^{k},z^{k},y^{k})$. Let $k\in\mathbb{N}$ be
large enough so that $\hat{y}^{k}=y^{k-1}$. Arguing as in the proof of Lemma
5.16, we have for all $k\in\mathbb{N}$ that $\Theta_{k}\leq V_{k}$ since
$\varepsilon_{k}=0$. Furthermore, using the triangle inequality, the
convergences, $x^{k}\to\bar{x}$, $z^{k}\to c(\bar{x})$, and Assumption 7(iv),
we obtain

$V_{k}=\mu_{k}\|y^{k}-y^{k-1}\|\leq\varrho_{\mathrm{u}}\mu_{k}(\Theta_{k}+\Theta_{k-1})$

from (4.17). Combining these inequalities yields

$\frac{V_{k}}{V_{k-1}}\leq\varrho_{\mathrm{u}}\mu_{k}\left(\frac{\Theta_{k}}{\Theta_{k-1}}+1\right).$

Assuming now by contradiction that $\mu_{k}\downarrow 0$, we deduce from the
proof of Theorem 5.18 that $\Theta_{k}/\Theta_{k-1}\to 0$, and then
$V_{k}/V_{k-1}\to 0$ follows. Hence, $V_{k}/V_{k-1}\leq\theta$ for all
$k\in\mathbb{N}$ sufficiently large, where $\theta\in(0,1)$ is a fixed
parameter of Algorithm 5.1, so that Algorithm 5.1 gives a contradiction, thus
proving the assertion.

In summary, local fast convergence of Algorithm 5.1, even for nonconvex
functions $g$, can be obtained in the presence of suitable second-order
conditions (one ensuring the existence of minimizers of the subproblems and
another one to guarantee validity of an upper error bound) and a first-order
CQ which, in principle, gives us the full convergence of the primal-dual
sequence.

In comparison with the noteworthy results from [23, 28], these assumptions may
seem quite strong. However, let us mention that in the settings discussed in
these papers, the (convex) function $g$ under consideration is chosen in such
a way that the aforementioned two second-order conditions can already be
merged into one, see Remark 4.18. Furthermore, it is likely that the
additional postulation of a first-order CQ could be avoided in these papers
too, since $g$ (or at least its derivative) is convex and/or polyhedral
_enough_ and, for convex functions, the proximal operator is well-behaved. It
remains a question of future research whether, exemplary, a generalized
polyhedral structure of $g$ (where its domain and epigraph are unions of
finitely many convex polyhedral sets) makes the additional assumption of a
first-order CQ superfluous.

##  6 Some exemplary settings

In light of our theoretical findings for the general problem (P), this section
examines two notable illustrative settings: sparsity-promoting and
complementarity-constrained optimization.

###  6.1 Sparsity-promoting optimization

Here, we take a closer look at the sparsity-promoting optimization problem
(4.9) which has been already discussed in Example 4.7.

Let us fix some point $\bar{x}\in\mathbb{R}^{n}$. For
$\bar{y}\in\partial\|\cdot\|_{0}(c(\bar{x}))$, we make use of

$I^{00}(\bar{x},\bar{y})\coloneqq\\{i\in
I^{0}(\bar{x})\,|\,\bar{y}_{i}=0\\},\quad
I^{0\pm}(\bar{x},\bar{y})\coloneqq\\{i\in I^{0}(\bar{x})\,|\,\bar{y}_{i}\neq
0\\}$

where $I^{0}(\bar{x})$ has been defined in Example 4.7. With the definition of
$I^{\pm}(\bar{x})$ therein, one obtains that

(6.1)
$T_{\operatorname{gph}\partial\|\cdot\|_{0}}(c(\bar{x}),\bar{y})=\left\\{(v,\eta)\in\mathbb{R}^{m}\times\mathbb{R}^{m}\,\middle|\,\begin{aligned}
&\forall i\in I^{\pm}(\bar{x})\colon&&\eta_{i}=0\\\ &\forall i\in
I^{0\pm}(\bar{x},\bar{y})\colon&&v_{i}=0\\\ &\forall i\in
I^{00}(\bar{x},\bar{y})\colon&&v_{i}\eta_{i}=0\end{aligned}\right\\}.$

This can be used to see that (4.19) reduces to the linear independence of the
family $(\nabla c_{i}(\bar{x}))_{i\in I^{0}(\bar{x})}$.

For $u\in\mathbb{R}^{n}\setminus\\{0\\}$ and $\eta\in
D(\partial\|\cdot\|_{0})(c(\bar{x}),\bar{y})(c^{\prime}(\bar{x})u)$, we easily
find

$\langle\eta,c^{\prime}(\bar{x})u\rangle=\sum_{i\in
I^{\pm}(\bar{x})}\underbrace{\eta_{i}}_{=0}c^{\prime}_{i}(\bar{x})u+\sum_{i\in
I^{0}(\bar{x})}\underbrace{\eta_{i}c^{\prime}_{i}(\bar{x})u}_{=0}=0,$

so that (4.24) reduces to

(6.2) $\forall u\in\\{u^{\prime}\in\mathbb{R}^{n}\,|\,\forall i\in
I^{0\pm}(\bar{x},\bar{y})\colon\,c^{\prime}_{i}(\bar{x})u^{\prime}=0\\}\setminus\\{0\\}\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},\bar{y})[u,u]>0,$

and this corresponds to the classical second-order sufficient condition for
the nonlinear program

$\operatorname*{minimize}_{x}{}\quad{}f(x){}\quad{}\operatorname{subject~{}to}{}\quad{}c_{i}(x)=0\quad
i\in I^{0\pm}(\bar{x},\bar{y}).$

From [9, Ex. 6.3], we find

$\mathcal{C}(\bar{x})=\\{u\in\mathbb{R}^{n}\,|\,f^{\prime}(\bar{x})u\leq
0,\,\forall i\in I^{0}(\bar{x})\colon\,c^{\prime}_{i}(\bar{x})u=0\\}.$

However, for $u\in\mathbb{R}^{n}$ satisfying $c^{\prime}_{i}(\bar{x})u=0$ for
all $i\in I^{0}(\bar{x})$, we already have

$f^{\prime}(\bar{x})u=-\langle\bar{y},c^{\prime}(\bar{x})u\rangle=-\sum_{i\in
I^{\pm}(\bar{x})}\underbrace{\bar{y}_{i}}_{=0}c^{\prime}_{i}(\bar{x})u-\sum_{i\in
I^{0}(\bar{x})}\bar{y}_{i}\underbrace{c^{\prime}_{i}(\bar{x})u}_{=0}=0,$

i.e., $u\in\mathcal{C}(\bar{x})$ due to
$\bar{y}\in\partial\|\cdot\|(c(\bar{x}))$, and this particularly holds for
$\bar{y}\in\Lambda(\bar{x})$ which exists whenever $\bar{x}$ is M-stationary.
In the latter case, we thus obtain the simplified representation

(6.3) $\mathcal{C}(\bar{x})=\\{u\in\mathbb{R}^{n}\,|\,\forall i\in
I^{0}(\bar{x})\colon\,c^{\prime}_{i}(\bar{x})u=0\\}.$

Noting that $\bar{y}_{i}=0$ holds for each $i\in I^{\pm}(\bar{x})$, [9, Ex.
6.3] shows that WSOSC is implied by

$\forall u\in\mathcal{C}(\bar{x})\setminus\\{0\\},\,\exists
y\in\Lambda(\bar{x})\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},y)[u,u]>0,$

while

$\exists\bar{y}\in\Lambda(\bar{x}),\,\forall
u\in\mathcal{C}(\bar{x})\setminus\\{0\\}\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},\bar{y})[u,u]>0$

is sufficient for SSOSC, and these correspond to certain second-order
sufficient optimality conditions for the optimization problem

$\operatorname*{minimize}_{x}{}\quad{}f(x){}\quad{}\operatorname{subject~{}to}{}\quad{}c_{i}(x)=0\quad
i\in I^{0}(\bar{x}).$

Clearly, due to (6.3), both conditions are implied by (6.2). The following
example shows that (6.2) can, indeed, be stronger than SSOSC.

######  Example 6.1.

We consider (4.9) for the functions $f\colon\mathbb{R}^{2}\to\mathbb{R}$ and
$c\colon\mathbb{R}^{2}\to\mathbb{R}^{2}$ given by

$f(x)\coloneqq\frac{1}{2}(x_{1}-x_{2})^{2}+x_{1}-x_{2},\quad
c(x)\coloneqq\begin{pmatrix}x_{1}-x_{2}\\\ x_{1}+x_{2}\end{pmatrix},$

and choose $\bar{x}$ to be the origin in $\mathbb{R}^{2}$. Note that
$I^{0}(\bar{x})=\\{1,2\\}$ and $\Lambda(\bar{x})=\\{(-1,0)\\}$, i.e.,
$\bar{y}\coloneqq(-1,0)$ is the uniquely determined multiplier in this
situation. As the critical cone $\mathcal{C}(\bar{x})$ reduces to the origin,
SSOSC at $\bar{x}$ for $\bar{y}$ is trivially satisfied.

Observe that we have

$\nabla_{xx}^{2}\mathcal{L}(\bar{x},\bar{y})=\begin{pmatrix}1&-1\\\
-1&1\end{pmatrix},$

and for $\bar{u}\coloneqq(1,1)$ and $\bar{\eta}\coloneqq(0,0)$, we find
$\bar{\eta}\in
D(\partial\|\cdot\|_{0})(c(\bar{x}),\bar{y})(c^{\prime}(\bar{x})\bar{u})$ from
(6.1). Furthermore,
$\nabla_{xx}^{2}\mathcal{L}(\bar{x},\bar{y})\bar{u}+c^{\prime}(\bar{x})^{\top}\bar{\eta}=0$
is valid. Hence, (4.20) does not hold, and this also shows that the stronger
condition (4.24) fails—the latter being equivalent to (6.2) in the present
setting.

###  6.2 Complementarity-constrained optimization

Let $m\coloneqq 2p$ for some $p\in\mathbb{N}$ and consider the special
situation $g\coloneqq\operatorname{\delta}_{C_{\textup{cc}}}$ where
$C_{\textup{cc}}\subseteq\mathbb{R}^{2p}$ is given by

$C_{\textup{cc}}\coloneqq\\{z\in\mathbb{R}^{2p}\,|\,\forall
i\in\\{1,\ldots,p\\}\colon\,0\leq z_{i}\perp z_{p+i}\geq 0\\},$

i.e., $C_{\textup{cc}}$ is the standard complementarity set. Problem (P),
thus, reduces to

(MPCC)
$\operatorname*{minimize}_{x}{}\quad{}f(x){}\quad{}\operatorname{subject~{}to}{}\quad{}c(x)\in
C_{\textup{cc}},$

a _mathematical problem with complementarity constraints_ (MPCC), see the
classical monographs [40, 46]. Note that standard inequality and equality
constraints can be added without any difficulty and are omitted here for
brevity of presentation.

For a feasible point $\bar{x}\in\mathbb{R}^{n}$ of (MPCC), we make use of the
index sets

$\displaystyle I^{+0}(\bar{x})$
$\displaystyle\coloneqq\\{i\in\\{1,\ldots,p\\}\,|\,c_{i}(\bar{x})>0,\,c_{p+1}(\bar{x})=0\\},$
$\displaystyle I^{0+}(\bar{x})$
$\displaystyle\coloneqq\\{i\in\\{1,\ldots,p\\}\,|\,c_{i}(\bar{x})=0,\,c_{p+i}(\bar{x})>0\\},$
$\displaystyle I^{00}(\bar{x})$
$\displaystyle\coloneqq\\{i\in\\{1,\ldots,p\\}\,|\,c_{i}(\bar{x})=0,\,c_{p+i}(\bar{x})=0\\},$

which provide a disjoint partition of $\\{1,\ldots,p\\}$. As we have

$\displaystyle\partial g(c(\bar{x}))=\partial^{\infty}g(c(\bar{x}))$
$\displaystyle=N_{C_{\textup{cc}}}(c(\bar{x}))$
$\displaystyle=\left\\{y\in\mathbb{R}^{2p}\,\middle|\,\begin{aligned} &\forall
i\in I^{+0}(\bar{x})\colon&&y_{i}=0\\\ &\forall i\in
I^{0+}(\bar{x})\colon&&y_{p+i}=0\\\ &\forall i\in
I^{00}(\bar{x})\colon&&(y_{i}\leq 0\,\land\,y_{p+i}\leq
0)\,\lor\,y_{i}y_{p+i}=0\end{aligned}\right\\},$

we can specify the precise meaning of the CQ (4.6). Note that, as
$\operatorname{\delta}_{C_{\textup{cc}}}$ is continuous on its closed domain
$C_{\textup{cc}}$, (4.6) can be used in Proposition 5.12. We also note that

$\displaystyle\widehat{N}_{C_{\textup{cc}}}(c(\bar{x}))=\left\\{y\in\mathbb{R}^{2p}\,\middle|\,\begin{aligned}
&\forall i\in I^{+0}(\bar{x})\colon&&y_{i}=0\\\ &\forall i\in
I^{0+}(\bar{x})\colon&&y_{p+i}=0\\\ &\forall i\in
I^{00}(\bar{x})\colon&&y_{i}\leq 0,\,y_{p+i}\leq 0\end{aligned}\right\\}.$

Let us now assume that $\bar{x}$ is an M-stationary point of (MPCC). Some
calculations show that the associated critical cone is given by

$\displaystyle\mathcal{C}(\bar{x})=\left\\{u\in\mathbb{R}^{n}\,\middle|\,\begin{aligned}
&&&f^{\prime}(\bar{x})u\leq 0\\\ &\forall i\in
I^{+0}(\bar{x})\colon&&c^{\prime}_{p+i}(\bar{x})u=0\\\ &\forall i\in
I^{0+}(\bar{x})\colon&&c^{\prime}_{i}(\bar{x})u=0\\\ &\forall i\in
I^{00}(\bar{x})\colon&&0\leq c^{\prime}_{i}(\bar{x})u\perp
c^{\prime}_{p+i}(\bar{x})u\geq 0\end{aligned}\right\\},$

see [9, Sec. 5.1]. If
$\Lambda(\bar{x})\cap\widehat{N}_{C_{\textup{cc}}}(c(\bar{x}))$ is nonempty,
i.e., if $\bar{x}$ is so-called strongly stationary, a simplified
representation of the critical cone is available which does not involve the
$\nabla f(\bar{x})$ anymore but depends on a multiplier
$y\in\Lambda(\bar{x})\cap\widehat{N}_{C_{\textup{cc}}}(c(\bar{x}))$ and is
given by

$\displaystyle\mathcal{C}(\bar{x})=\left\\{u\in\mathbb{R}^{n}\,\middle|\,\begin{aligned}
&&&f^{\prime}(\bar{x})u\leq 0\\\ &\forall i\in
I^{+0}(\bar{x})\colon&&c^{\prime}_{p+i}(\bar{x})u=0\\\ &\forall i\in
I^{0+}(\bar{x})\colon&&c^{\prime}_{i}(\bar{x})u=0\\\ &\forall i\in
I^{00}_{--}(\bar{x},y)\colon&&c^{\prime}_{i}(\bar{x})u=0,\,c^{\prime}_{p+i}(\bar{x})=0\\\
&\forall i\in
I^{00}_{-0}(\bar{x},y)\colon&&c^{\prime}_{i}(\bar{x})u=0,\,c^{\prime}_{p+i}(\bar{x})u\geq
0\\\ &\forall i\in I^{00}_{0-}(\bar{x},y)\colon&&c^{\prime}_{i}(\bar{x})u\geq
0,\,c^{\prime}_{p+i}(\bar{x})u=0\\\ &\forall i\in
I^{00}_{*}(\bar{x},y)\colon&&0\leq c^{\prime}_{i}(\bar{x})u\perp
c^{\prime}_{p+i}(\bar{x})u\geq 0\end{aligned}\right\\},$

see [41, Lem. 4.1]. Here, we used

$\displaystyle I^{00}_{--}(\bar{x},y)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,y_{i}<0,\,y_{p+i}<0\\},$ $\displaystyle
I^{00}_{-0}(\bar{x},y)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,y_{i}<0,\,y_{p+i}=0\\},$ $\displaystyle
I^{00}_{0-}(\bar{x},y)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,y_{i}=0,\,y_{p+i}<0\\},$ $\displaystyle
I^{00}_{*}(\bar{x},y)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,y_{i}=0,\,y_{p+i}=0\\},$

which provide a disjoint partition of $I^{00}(\bar{x})$.

Following the arguments provided at the end of [9, Sec. 3.1], we find

$\displaystyle\widehat{N}^{\textup{p}}_{C_{\textup{cc}}}(c(\bar{x}),c^{\prime}(\bar{x})u)$
$\displaystyle=\widehat{N}_{T_{C_{\textup{cc}}}(c(\bar{x}))}(c^{\prime}(\bar{x})u)$
$\displaystyle=\left\\{y\in\mathbb{R}^{2p}\,\middle|\,\begin{aligned} &\forall
i\in I^{+0}(\bar{x})\cup I^{00}_{+0}(\bar{x},u)\colon&&y_{i}=0\\\ &\forall
i\in I^{0+}(\bar{x})\cup I^{00}_{0+}(\bar{x},u)\colon&&y_{p+i}=0\\\ &\forall
i\in I^{00}_{00}(\bar{x},u)\colon&&y_{i}\leq 0,\,y_{p+1}\leq
0\end{aligned}\right\\}$

for each $u\in\mathcal{C}(\bar{x})$, where we made use of a disjoint partition
of $I^{00}(\bar{x})$ given by

$\displaystyle I^{00}_{+0}(\bar{x},u)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,c^{\prime}_{i}(\bar{x})u>0,\,c^{\prime}_{p+i}(\bar{x})u=0\\},$
$\displaystyle I^{00}_{0+}(\bar{x},u)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,c^{\prime}_{i}(\bar{x})u=0,\,c^{\prime}_{p+i}(\bar{x})u>0\\},$
$\displaystyle I^{00}_{00}(\bar{x},u)$ $\displaystyle\coloneqq\\{i\in
I^{00}(\bar{x})\,|\,c^{\prime}_{i}(\bar{x})u=0,\,c^{\prime}_{p+i}(\bar{x})u=0\\}.$

Hence, we have

$\Lambda(\bar{x},u)=\left\\{y\in\Lambda(\bar{x})\,\middle|\,\begin{aligned}
&\forall i\in I^{00}_{+0}(\bar{x},u)\colon&&y_{i}=0\\\ &\forall i\in
I^{00}_{0+}(\bar{x},u)\colon&&y_{p+i}=0\\\ &\forall i\in
I^{00}_{00}(\bar{x},u)\colon&&y_{i}\leq 0,\,y_{p+1}\leq
0\end{aligned}\right\\}.$

Thus, due to [9, Thm 5.4], WSOSC can be stated in the form

$\forall u\in\mathcal{C}(\bar{x})\setminus\\{0\\},\,\exists
y\in\Lambda(\bar{x},u)\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},y)[u,u]>0.$

As remarked directly after Definition 4.8, any multiplier
$\bar{y}\in\Lambda(\bar{x})$ suitable to appear in SSOSC necessarily belongs
to
$\bigcap_{u\in\mathcal{C}(\bar{x})\setminus\\{0\\}}\widehat{N}^{\textup{p}}_{C_{\textup{cc}}}(c(\bar{x}),c^{\prime}(\bar{x})u)$,
and for any such multiplier $\bar{y}$,
$\mathrm{d}^{2}\operatorname{\delta}_{C_{\textup{cc}}}(c(\bar{x}),\bar{y})(c^{\prime}(\bar{x})u)$
vanishes, see [9, Lem. 3.2, Prop. 3.6]. Hence, SSOSC takes the form

$\exists\bar{y}\in\Lambda(\bar{x})\cap\bigcap_{u\in\mathcal{C}(\bar{x})\setminus\\{0\\}}\widehat{N}^{\textup{p}}_{C_{\textup{cc}}}(c(\bar{x}),c^{\prime}(\bar{x})u),\,\forall
u\in\mathcal{C}(\bar{x})\setminus\\{0\\}\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},\bar{y})[u,u]>0.$

It follows from [24, Lem. 3.2] that this is a less restrictive assumption than
the standard second-order sufficient condition for (MPCC) which takes the form

$\exists\bar{y}\in\Lambda(\bar{x})\cap\widehat{N}_{C_{\textup{cc}}}(c(\bar{x})),\,\forall
u\in\mathcal{C}(\bar{x})\setminus\\{0\\}\colon\quad\nabla^{2}_{xx}\mathcal{L}(\bar{x},\bar{y})[u,u]>0$

and is based on a strongly stationary point. A detailed study on the
relationship between WSOSC and SSOSC as well as other MPCC-tailored second-
order optimality conditions is beyond the scope of this paper, see e.g. [24,
26] for an overview.

The graphical derivative of the limiting normal cone mappings associated with
$C_{\textup{cc}}$ has been computed recently in [8, Sec. 5.2], and the
obtained formulas can be used to specify the CQs (4.16), (4.19), (4.20), and
(4.24) in the recent setting.

##  7 Concluding remarks

The results in this paper could be extended to cover the extra feature in (P)
of a geometric _convex_ constraint $x\in X$, which was not included here for
reasons of exposition. It remains unclear, instead, how to address such
additional constraint with nonconvex $X$, if not reformulating into (P) and
accepting $x\in X$ as a soft constraint, see [17, Rem. 4.1].

Another challenging question is whether it is possible to dispose the
additional constraint qualification in the nonconvex polyhedral case (i.e.,
$\operatorname{epi}g$ being the union of finitely many convex polyhedra) in
the analysis of Section 5. Such a result would yield convergence rates merely
via some second-order sufficient conditions and the (upper) error bound,
generalizing [23]. In specific situations, this should be possible even in the
nonpolyhedral setting, as [28] has shown for the case of convex linear-
quadratic $g$. As already pointed out in Section 1, such a generalization does
not seem to be available in nonpolyhedral settings.

Future research may also focus on the relationships between the proximal point
algorithm and the augmented Lagrangian method in the fully nonconvex setting,
in the vein of [51, 55], and investigate saddle-point properties of the
augmented Lagrangian function in primal-dual terms as in [58].

## References

  * [1] B.Bank, J.Guddat, D.Klatte, B.Kummer, and K.Tammer, _Non-Linear Parametric Optimization_ , Birkhäuser, Basel, 1983, doi:10.1007/978-3-0348-6328-5.
  * [2] H. H.Bauschke and P. L.Combettes, _Convex Analysis and Monotone Operator Theory in Hilbert Spaces_ , Springer, New York, 2011, doi:10.1007/978-1-4419-9467-7.
  * [3] M.Benko, On inner calmness*, generalized calculus, and derivatives of the normal-cone mapping, _Journal of Nonsmooth Analysis and Optimization_ 2 (2021), 5881, doi:10.46298/jnsao-2021-5881.
  * [4] M.Benko, H.Gfrerer, and J. V.Outrata, Calculus for directional limiting normal cones and subdifferentials, _Set-Valued and Variational Analysis_ 27 (2019), 713–745, doi:10.1007/s11228-018-0492-5.
  * [5] M.Benko, H.Gfrerer, J. J.Ye, J.Zhang, and J.Zhou, Second-order optimality conditions for general nonconvex optimization problems and variational analysis of disjunctive systems, 2022, arXiv:2203.10015v2.
  * [6] M.Benko and P.Mehlitz, On implicit variables in optimization theory, _Journal of Nonsmooth Analysis and Optimization_ 2 (2021), 7215, doi:10.46298/jnsao-2021-7215.
  * [7] M.Benko and P.Mehlitz, Calmness and calculus: two basic patterns, _Set-Valued and Variational Analysis_ 30 (2022), 81–117, doi:10.1007/s11228-021-00589-x.
  * [8] M.Benko and P.Mehlitz, On the directional asymptotic approach in optimization theory Part A: approximate, M-, and mixed-order stationarity, _arXiv_ (2022), arXiv:2204.13932.
  * [9] M.Benko and P.Mehlitz, Why second-order sufficient conditions are, in a way, easy — or — revisiting calculus for second subderivatives, _Journal of Convex Analysis_ 30 (2023), 541–589, https://www.heldermann.de/JCA/JCA30/JCA302/jca30031.htm.
  * [10] D. P.Bertsekas, _Constrained Optimization and Lagrange Multiplier Methods_ , Athena Scientific, 1996.
  * [11] E. G.Birgin and J. M.Martínez, _Practical Augmented Lagrangian Methods for Constrained Optimization_ , SIAM, Philadelphia, 2014, doi:10.1137/1.9781611973365.
  * [12] J.Bolte, S.Sabach, and M.Teboulle, Nonconvex Lagrangian-based optimization: monitoring schemes and global convergence, _Mathematics of Operations Research_ 43 (2018), 1210–1232, doi:10.1287/moor.2017.0900.
  * [13] J. F.Bonnans and A.Shapiro, _Perturbation Analysis of Optimization Problems_ , Springer, New York, 2000, doi:10.1007/978-1-4612-1394-9.
  * [14] E.Börgens, C.Kanzow, and D.Steck, Local and global analysis of multiplier methods for constrained optimization in Banach spaces, _SIAM Journal on Control and Optimization_ 57 (2019), 3694–3722, doi:10.1137/19m1240186.
  * [15] X.Chen, L.Guo, Z.Lu, and J. J.Ye, An augmented Lagrangian method for non-Lipschitz nonconvex programming, _SIAM Journal on Numerical Analysis_ 55 (2017), 168–193, doi:10.1137/15m1052834.
  * [16] A. R.Conn, N. I. M.Gould, and P. L.Toint, A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds, _SIAM Journal on Numerical Analysis_ 28 (1991), 545–572, doi:10.1137/0728030.
  * [17] A.De Marchi, Implicit augmented Lagrangian and generalized optimization, _arXiv_ (2023), arXiv:2302.00363v2.
  * [18] A.De Marchi, X.Jia, C.Kanzow, and P.Mehlitz, Constrained composite optimization and augmented Lagrangian methods, _Mathematical Programming_ 201 (2023), 863–896, doi:10.1007/s10107-022-01922-4.
  * [19] A.De Marchi and A.Themelis, Proximal gradient algorithms under local Lipschitz gradient continuity, _Journal of Optimization Theory and Applications_ 194 (2022), 771–794, doi:10.1007/s10957-022-02048-5.
  * [20] S.Dempe, J.Dutta, and B. S.Mordukhovich, New necessary optimality conditions in optimistic bilevel programming, _Optimization_ 56 (2007), 577–604, doi:10.1080/02331930701617551.
  * [21] N. K.Dhingra, S. Z.Khong, and M. R.Jovanović, The proximal augmented Lagrangian method for nonsmooth composite optimization, _IEEE Transactions on Automatic Control_ 64 (2019), 2861–2868, doi:10.1109/tac.2018.2867589.
  * [22] A. L.Dontchev and R. T.Rockafellar, _Implicit Functions and Solution Mappings_ , Springer, Heidelberg, 2014, doi:10.1007/978-0-387-87821-8.
  * [23] D.Fernández and M.Solodov, Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficient optimality condition, _SIAM Journal on Optimization_ 22 (2012), 384–407, doi:10.1137/10081085x.
  * [24] H.Gfrerer, Optimality conditions for disjunctive programs based on generalized differentiation with application to mathematical programs with equilibrium constraints, _SIAM Journal on Optimization_ 24 (2014), 898–931, doi:10.1137/130914449.
  * [25] H.Gfrerer and J. V.Outrata, On computation of generalized derivatives of the normal-cone mapping and their applications, _Mathematics of Operations Research_ 41 (2016), 1535–1556, doi:10.1287/moor.2016.0789.
  * [26] L.Guo, G. H.Lin, and J. J.Ye, Second-order optimality conditions for mathematical programs with equilibrium constraints, _Journal of Optimization Theory and Applications_ 158 (2013), 33–64, doi:10.1007/s10957-012-0228-x.
  * [27] N. T. V.Hang, B. S.Mordukhovich, and M. E.Sarabi, Augmented Lagrangian method for second-order cone programs under second-order sufficiency, _Journal of Global Optimization_ 82 (2022), 51–81, doi:10.1007/s10898-021-01068-1.
  * [28] N. T. V.Hang and M. E.Sarabi, Local convergence analysis of augmented Lagrangian methods for piecewise linear-quadratic composite optimization problems, _SIAM Journal on Optimization_ 31 (2021), 2665–2694, doi:10.1137/20m1375188.
  * [29] N. T. V.Hang and M. E.Sarabi, Convergence of augmented Lagrangian methods for composite optimization problems, _arXiv_ (2023), arXiv:2307.15627v1.
  * [30] M. R.Hestenes, Multiplier and gradient methods, _Journal of Optimization Theory and Applications_ 4 (1969), 303–320, doi:10.1007/bf00927673.
  * [31] A. F.Izmailov, On the analytical and numerical stability of critical Lagrange multiplier, _Computational Mathematics and Mathematical Physics_ 45 (2005), 930–946, https://www.mathnet.ru/php/archive.phtml?wshow=paper&jrnid=zvmmf&paperid=636&option_lang=eng.
  * [32] A. F.Izmailov, A. S.Kurennoy, and M. V.Solodov, Local convergence of the method of multipliers for variational and optimization problems under the noncriticality assumption, _Computational Optimization and Applications_ 60 (2015), 111–140, doi:10.1007/s10589-014-9658-8.
  * [33] A. F.Izmailov and M. V.Solodov, Stabilized SQP revisited, _Mathematical Programming_ 133 (2012), 93–120, doi:10.1007/s10107-010-0413-3.
  * [34] X.Jia, C.Kanzow, P.Mehlitz, and G.Wachsmuth, An augmented Lagrangian method for optimization problems with structured geometric constraints, _Mathematical Programming_ 199 (2023), 1365–1415, doi:10.1007/s10107-022-01870-z.
  * [35] C.Kanzow, F.Krämer, P.Mehlitz, G.Wachsmuth, and F.Werner, A nonsmooth augmented Lagrangian method and its application to Poisson denoising and sparse control, _arXiv_ (2023), arXiv:2304.06434.
  * [36] C.Kanzow and D.Steck, On error bounds and multiplier methods for variational problems in Banach spaces, _SIAM Journal on Control and Optimization_ 56 (2018), 1716–1738, doi:10.1137/17m1146518.
  * [37] C.Kanzow and D.Steck, Improved local convergence results for augmented Lagrangian methods in $C^{2}$-reducible constrained optimization, _Mathematical Programming_ 177 (2019), 425–438, doi:10.1007/s10107-018-1261-9.
  * [38] C.Kanzow, D.Steck, and D.Wachsmuth, An augmented Lagrangian method for optimization problems in Banach spaces, _SIAM Journal on Control and Optimization_ 56 (2018), 272–291, doi:10.1137/16m1107103.
  * [39] A. B.Levy, Implicit multifunction theorems for the sensitivity analysis of variational conditons, _Mathematical Programming_ 74 (1996), 333–350, doi:10.1007/bf02592203.
  * [40] Z. Q.Luo, J. S.Pang, and D.Ralph, _Mathematical Programs with Equilibrium Constraints_ , Cambridge University Press, Cambridge, 1996, doi:10.1017/cbo9780511983658.
  * [41] P.Mehlitz, On the linear independence constraint qualification in disjunctive programming, _Optimization_ 69 (2020), 2241–2277, doi:10.1080/02331934.2019.1679811.
  * [42] A.Mohammadi, B. S.Mordukhovich, and M. E.Sarabi, Variational analysis of composite models with applications to continuous optimization, _Mathematics of Operations Research_ 47 (2022), 397–426, doi:10.1287/moor.2020.1074.
  * [43] B. S.Mordukhovich, _Variational Analysis and Applications_ , Springer, Cham, 2018, doi:10.1007/978-3-319-92775-6.
  * [44] B. S.Mordukhovich and M. E.Sarabi, Critical multipliers in variational systems via second-order generalized differentiation, _Mathematical Programming_ 169 (2018), 605–648, doi:10.1007/s10107-017-1155-2.
  * [45] B. S.Mordukhovich and M. E.Sarabi, Criticality of Lagrange multipliers in variational systems, _SIAM Journal on Optimization_ 29 (2019), 1524–1557, doi:10.1137/18m1206862.
  * [46] J. V.Outrata, M.Kočvara, and J.Zowe, _Nonsmooth Approach to Optimization Problems with Equilibrium Constraints_ , Kluwer Academic, Dordrecht, 1998, doi:10.1007/978-1-4757-2825-5.
  * [47] J. P.Penot, Second-order conditions for optimization problems with constraints, _SIAM Journal on Control and Optimization_ 37 (1998), 303–318, doi:10.1137/s0363012996311095.
  * [48] M. J. D.Powell, A method for nonlinear constraints in minimization problems, in _Optimization_ , Academic Press, 1969, 283–298.
  * [49] S. M.Robinson, Some continuity properties of polyhedral multifunctions, in _Mathematical Programming at Oberwolfach_ , H.König, B.Korte, and K.Ritter (eds.), Springer, Berlin, 1981, 206–214, doi:10.1007/bfb0120929.
  * [50] R. T.Rockafellar, A dual approach to solving nonlinear programming problems by unconstrained optimization, _Mathematical Programming_ 5 (1973), 354–373, doi:10.1007/bf01580138.
  * [51] R. T.Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming, _Mathematics of operations research_ 1 (1976), 97–116, doi:10.1287/moor.1.2.97.
  * [52] R. T.Rockafellar, Lagrange multipliers and optimality, _SIAM Review_ 35 (1993), 183–238, doi:10.1137/1035044.
  * [53] R. T.Rockafellar, Extended nonlinear programming, in _Nonlinear Optimization and Related Topics_ , G.Di Pillo and F.Giannessi (eds.), volume 36 of Applied Optimization, Kluwer, 2000, 381–399, doi:10.1007/978-1-4757-3226-9_20.
  * [54] R. T.Rockafellar, Augmented Lagrangians and hidden convexity in sufficient conditions for local optimality, _Mathematical Programming_ 198 (2023), 159–194, doi:10.1007/s10107-022-01768-w.
  * [55] R. T.Rockafellar, Convergence of augmented Lagrangian methods in extensions beyond nonlinear programming, _Mathematical Programming_ 199 (2023), 375–420, doi:10.1007/s10107-022-01832-5.
  * [56] R. T.Rockafellar and R. J. B.Wets, _Variational Analysis_ , Springer, 1998, doi:10.1007/978-3-642-02431-3.
  * [57] S.Sabach and M.Teboulle, Lagrangian methods for composite optimization, in _Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2_ , R.Kimmel and X. C.Tai (eds.), volume 20 of Handbook of Numerical Analysis, Elsevier, 2019, 401–436, doi:10.1016/bs.hna.2019.04.002.
  * [58] D.Steck, _Lagrange Multiplier Methods for Constrained Optimization and Variational Problems in Banach Spaces_ , PhD thesis, University of Würzburg, 2018, https://nbn-resolving.org/urn:nbn:de:bvb:20-opus-174444.
  * [59] G.Wachsmuth, Strong stationarity for optimization problems with complementarity constraints in absence of polyhedricity, _Set-Valued and Variational Analysis_ 25 (2017), 133–175, doi:10.1007/s11228-016-0370-y.
  * [60] D. W.Walkup and R. J. B.Wets, A Lipschitz characterization of convex polyhedra, _Proceedings of the American Mathematical Society_ 23 (1969), 167–173, doi:10.2307/2037511.
  * [61] J. J.Ye and J.Zhou, Verifiable sufficient conditions for the error bound property of second-order cone complementarity problems, _Mathematical Programming_ 171 (2018), 361–395, doi:10.1007/s10107-017-1193-9.
  * [62] J.Zowe and S.Kurcyusz, Regularity and stability for the mathematical programming problem in Banach spaces, _Applied Mathematics and Optimization_ 5 (1979), 49–62, doi:10.1007/bf01442543.


# Fine-tuning Wav2vec for Vocal-burst Emotion Recognition

Dang-Khanh Nguyen  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>Ngoc-Huynh Ho  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>Sudarshan Pant  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>Guee-Sang Lee  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>Soo-Huyng Kim  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>Hyung-Jeong Yang  Department of AI Convergence
Chonnam National University
Gwangju, South Korea
<EMAIL_ADDRESS>
###### Abstract

The ACII Affective Vocal Bursts (A-VB) competition introduces a new topic in
affective computing, which is understanding emotional expression using the
non-verbal sound of humans. We are familiar with emotion recognition via
verbal vocal or facial expression. However, the vocal bursts such as laughs,
cries, and signs, are not exploited even though they are very informative for
behavior analysis. The A-VB competition comprises four tasks that explore non-
verbal information in different spaces. This technical report describes the
method and the result of SclabCNU Team for the tasks of the challenge. We
achieved promising results compared to the baseline model provided by the
organizers.

###### Index Terms:

Vocal Burst, Emotion Recognition, Wav2vec

##  I Introduction

Language and facial expression are strong indicators for behavior analysis.
There is numerous research trying to solve the emotion recognition problem
based on these data. However, the non-linguistic vocalizations are
understudied even though they are very useful information. Analyzing and
applying these signals are interesting topics and require more attention from
researchers. The A-VB competition was conducted for that reason and is
expected to explore advanced improvements in emotion science. The competition
[2] consists of 4 individual tasks as below:

  * •

High-dimension task (A-VB-HIGH): a multi-output regression task generating 10
values in the range of [0,1] corresponding to levels of Awe, Excitement,
Amusement, Awkwardness, Fear, Horror, Distress, Triumph, Sadness, and
Surprise.

  * •

Two-dimension task (A-VB-TWO): a multi-output regression task generating 2
values in the range of [0,1] corresponding to levels of Valence and Arousal.

  * •

Culture task (A-VB-CULTURE): a multi-output regression task generating 40
values in the range of [0,1] corresponding to levels of culture including
China, US, South Africa, and Venezuela combined with levels of emotion
including Awe, Excitement, Amusement, Awkwardness, Fear, Horror, Distress,
Triumph, Sadness, Surprise.

  * •

Type task (A-VB-TYPE): a multi-class classification task predicting the type
of expressive vocal including Gasp, Laugh, Cry, Scream, Grunt, Groan, Pant,
Other.

The evaluation metric for three regression tasks is the Concordance
Correlation Coefficient (CCC) and the metric for the categorical task is the
Unweighted Average Recall (UAR). The detail is listed below:

  * •

A-VB-HIGH: The metric is the average CCC score of 10 emotions.

  * •

A-VB-TWO: The metric is the average CCC score of valence-arousal.

  * •

A-VB-CULTURE: The metric is the average CCC score of 40 culture-emotion
levels.

  * •

A-VB-TYPE: The metric is the UAR score of 8 classes of vocalizations.

All of the above metrics are in the range of [0,1]. The greater the score is,
the better the model performs. We should also note that all the results in
this paper are in percentage.

In this paper, we propose a straightforward approach using a pre-trained
Wav2vec network to resolve the problem. The model accomplishes a noticeable
improvement compared to the baseline provided by the organizers. Because of
its simplicity, our model can be considered a new baseline for all tasks in
the competition.

##  II Related work

In the baseline paper [2], the authors introduce two different approaches,
which are feature-based and end-to-end approaches. In the feature-based
option, the OpenSMILE toolkit [11] is leveraged to extract the COMPARE
(COMputational PARalinguistics ChallengE [12]) feature and EGEMAPS (The
extended Geneva Minimalistic Acoustic Parameter Set [13]) feature from an
input sample. The features are then fed to a 3-layer fully-connected neural
network. Mean Squared Error (MSE) loss is used for regression tasks while the
classification task applies the Cross-entropy (CE) loss function.

In the end-to-end manner, the baseline model uses End2You [14], a multimodal
profiling toolkit that is capable of training and evaluating models from raw
input. Particularly, Emo-18 architecture [15] is chosen for the competition.
The model includes 1-D Convolutional Neural Network (CNN) layers to derive the
features from audio frames and a Recurrent Neural Network to learn the
temporal information.

For the speaker recognition task, Nik Vaessen and David A. van Leeuwen [4]
conducted fine-tuning the Wav2vec2 model by using a shared fully-connected
layer. Their model and ours have one thing in common: exploiting the pre-
trained Wav2vec. However, there are considerable differences between the two
methods. Basically, speaker recognition is a classification problem so the
authors optimize the model with CE or Binary Cross-entropy (BCE) loss. In our
method, we consider two loss options for the regression tasks, which are MSE
and CCC loss. Additionally, besides using a shared fully-connected layer, we
also take advantage of the RNN to explore the temporal behavior.

##  III Method

The sequence embeddings are obtained from the waveform signal by the audio
extractor. They are then fed into an RNN to enrich the sequence information.
Afterward, a fully connected layer changes the embeddings’ dimension to the
output sizes depending on the particular task. Finally, a pooling layer is
used to reduce variable-length sequence embeddings to fix-size speaker
embedding. The dimension of the final prediction would be 2, 10, 40, or 8
corresponding to A-VB-TWO, A-VB-HIGH, A-VB-CULTURE or A-VB-TYPE task,
respectively. Figure 1 describes the architecture of our method.

Figure 1: Block diagram of our proposed model.

###  III-A Audio extractor

We take advantage of the Pre-trained Wav2vec 2.0 models [3] provided by
Pytorch. They are trained with large unlabeled audio corpora so they can
effectively capture the audio features. We conducted experiments with 4
versions of the Wav2vec2 model described below:

  * •

BASE: use the base configuration of transformer trained with 960 hours of
unlabeled audio from LibriSpeech [5].

  * •

LARGE: use the large configuration of transformer trained with 960 hours of
unlabeled audio from LibriSpeech [5].

  * •

LARGE-LV60K: use the large configuration of transformer trained with 60,000
hours of unlabeled audio from Libri-Light [6].

  * •

XLSR53: use base configuration of transformer [10] trained with 56,000 hours
of unlabeled audio from multiple datasets (Multilingual LibriSpeech [7],
CommonVoice [8] and BABEL [9]).

###  III-B Pooling Method

Inspired by the model of [4], we use 4 options of pooling to fix the length of
embedding: first (take the first sequence embedding to be the speaker
embedding), last (take the last sequence embedding to be the speaker
embedding), max, and average pooling. The performance of models with various
types of pooling layers is recorded to study their impact on the result. The
operation of the pooling can be described as:

$s_{i}=Pooling(e_{1},e_{2},...,e_{m_{i}})$ (1)

where $s_{i}$ is the speaker embedding of $i^{th}$ sample;
$e_{1},e_{2},...,e_{m_{i}}$ are the temporal embeddings and $m_{i}$ is the
sequence length of corresponding sample.

###  III-C Loss function

We use the CE loss for the classification task. In the remaining tasks, we
want to test the effect of the loss function on the performance of the model
so we did the experiments and compared the result of the model using Mean
Square Error (MSE) and Concordance Correlation Coefficient (CCC) loss. The CCC
loss is formulated as below:

$\mathcal{L}=1-CCC=1-\frac{2s_{xy}}{s^{2}_{x}+s^{2}_{y}+\left(\overline{x}-\overline{y}\right)^{2}}$
(2)

where $\overline{x}$ and $\overline{y}$ are the mean values of ground truth
and predicted values, respectively, $s_{x}$ and $s_{x}$ are corresponding
variances and $s_{xy}$ is the covariance value.

##  IV Dataset and Experiments

###  IV-A Dataset

The database used for the competition is the HUME-VB dataset [1] which
consists of 59201 audio files and is split into 3 sets (training, validation,
and test) of similar size. Each file has 53 labels corresponding to 4 tasks,
one label is a categorical label that is used in the classification task and
the remains are values in the range [0,1] representing the emotional level.
The organizers provide 2 versions of the HUME-VB dataset: the raw version
sampled at 48kHz and the processed version where audio files are converted to
16kHz and normalized to -3 decibels. In our experiments, we take the processed
version as the input of our model.

###  IV-B Experiments

Our model was implemented using the Pytorch framework. The experiments were
conducted on a machine with NVIDIA RTX 2080 Ti GPU. All scenarios were run in
20 epochs, and the model with the best performance on the validation set was
recorded. The batch size is 16 and the initial learning rate is $1e-4$. We
used Adam optimizer with the weight decay coefficient of $0.0625$.

In our setting, we take the output from the $12^{th}$ layer of the Wav2vec
network to be the sequence embeddings. The input size of the RNN network
depends on the configuration of the pre-trained audio extractor, which is 768
for base configuration and 1024 for large configuration. It includes 2 LSTM
layers and the hidden size is fixed to 512.

##  V Discussion

We tested the performance of the model with various audio extractors to
explore their effect. Table I shows the performance on four tasks of the
competition with 4 pre-trained Wav2vec extractors. As a result, the XLSR53
pre-trained model achieves the best performance in A-VB-TWO and A-VB-TYPE when
LARGE-LV60K attains the highest scores in A-VB-HIGH and A-VB-CULTURE. In the
meanwhile, the BASE model produces the lowest score in A-VB-TWO and A-VB-TYPE
due to its simple architecture.

TABLE I: Evaluation score on the HUME-VB validation set with different extractors. Experimented with CCC loss and Last Pooling Model | TWO | HIGH | CULTURE | TYPE
---|---|---|---|---
End2You Baseline | 49.88 | 56.38 | 43.59 | 41.66
BASE | 54.65 | 58.69 | 47.18 | 41.65
LARGE | 55.42 | 58.00 | 47.12 | 43.96
LARGE-LV60K | 61.59 | 65.41 | 53.39 | 47.22
XLSR53 | 61.94 | 65.32 | 52.50 | 49.89

Regarding the pooling method, we examined their influence on the results in
A-VB-HIGH. As shown in Table II, in both LARGE-LV60K and XLSR53 model, the
Last pooling outperforms the other options while the First pooling gets the
lowest CCC score among the four methods. The result of Avg pooling is slightly
better than Max pooling in both LARGE-LV60K and XLSR53 scenarios. It can be
inferred that the last embedding of the sequence contains the most useful
information for the prediction when using other embeddings or combining them
may degrade the accuracy.

TABLE II: Evaluation score on the HUME-VB validation set with different pooling methods. Experimented on A-VB-HIGH with CCC loss Pooling | LARGE-LV60K | XLSR53
---|---|---
First | 53.56 | 58.20
Max | 60.08 | 61.41
Avg | 61.49 | 62.40
Last | 65.41 | 65.32

Next, we conducted the training processes with MSE and CCC to explore their
advantage. As a consequence, the model trained with CCC loss gives better
performance on the validation set compared to the one trained with MSE loss.
The detail is shown in Table III.

TABLE III: Evaluation score on the HUME-VB validation set with different loss functions. Experimented on A-VB-HIGH with Last pooling Loss function | LARGE-LV60K | XLSR53
---|---|---
MSE | 63.77 | 63.94
CCC | 65.41 | 65.32

In addition, we carried out the ablation study to analyze the role of the RNN.
According to Table IV, using the RNN can significantly boost the accuracy of
the model in all four tasks. It can be explained by the capability of learning
temporal information of the LSTM, which can enhance the overall operation of
the model.

TABLE IV: Evaluation score on the HUME-VB validation set with and without RNN. Experimented on A-VB-HIGH with LARGE-LV60K model, CCC loss, and Last pooling Model | TWO | HIGH | CULTURE | TYPE
---|---|---|---|---
Without RNN | 47.38 | 50.70 | 40.20 | 39.10
With RNN | 61.59 | 65.41 | 53.39 | 47.22

After conducting the above experiments, we concluded that the best
configuration of our model is combining either LARGE-LV60K or XLSR53 pre-
trained model with last pooling method and utilizing CCC loss. This setting
was used to train separated model for each task in order to obtain unbiased
evaluation on test set. We decided to choose LARGE-LV60K for A-VB-HIGH and
A-VB-CULTURE, XLSR53 for A-VB-TWO and A-VB-TYPE. This time we trained each
model for 50 epochs and applied early stopping by monitoring the validation
result. Our best models and their evaluations on test set and validation set
are listed in Table V.

TABLE V: Evaluation score on the HUME-VB validation and test sets. Experimented with CCC loss and Last pooling for 50 epochs Task | Pre-trained | Performance
---|---|---
name | Audio Extractor | Val set | Test set
TWO | XLSR53 | 61.94 | 62.02
HIGH | LARGE-LV60K | 66.76 | 66.77
CULTURE | LARGE-LV60K | 54.93 | 54.95
TYPE | XLSR53 | 49.89 | 49.70

##  VI Conclusion

This paper presents our proposed method for all sub-challenges of the A-VB
competition. Particularly, we fine-tuned the pre-trained Wav2vec and combined
it with basic neural networks and a proper pooling method. The CCC loss and
Last pooling show the best performance on four tasks among the other options.
Our model outperforms the baseline of the organizer on the test set, with the
CCC score of 62.02 for A-VB-TWO, 66.77 for A-VB-HIGH, 54.95 for A-VB-CULTURE
and UAR metric of 49.70 for A-VB-TYPE.

## Acknowledgment

This work was supported by a National Research Foundation of Korea (NRF) grant
funded by the Korean government (MSIT). (NRF-2020R1A4A1019191).

## References

  * [1] Cowen, Alan and Baird, Alice and Tzirakis, Panagiotis and Opara, Michael and Kim, Lauren and Brooks, Jeff and Metrick, Jacob, “The hume vocal burst competition dataset (H-VB) — raw data [exvo: updated 02.28.22] [data set],” Zenodo, 2022.
  * [2] Baird, Alice and Tzirakis, Panagiotis and Batliner, Anton and Schuller, Björn and Keltner, Dacher and Cowen, Alan, “The ACII 2022 Affective Vocal Bursts Workshop & Competition: Understanding a critically understudied modality of emotional expression,” arXiv preprint arXiv:2207.03572 (2022).
  * [3] Baevski, Alexei, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in Neural Information Processing Systems 33 (2020): 12449-12460.
  * [4] Vaessen, Nik, and David A. Van Leeuwen, “Fine-tuning wav2vec2 for speaker recognition,” In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7967-7971. IEEE, 2022.
  * [5] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), volume, 5206–5210. 2015.
  * [6] J. Kahn, M. Rivière, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazaré, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-light: a benchmark for asr with limited or no supervision,” In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7669–7673. 2020.
  * [7] Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert, “Mls: a large-scale multilingual dataset for speech research,” Interspeech 2020, Oct 2020.
  * [8] Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber, “Common voice: a massively-multilingual speech corpus,” 2020.
  * [9] Mark John Francis Gales, Kate Knill, Anton Ragni, and Shakti Prasad Rath, “Speech recognition and keyword spotting for low-resource languages: babel project research at cued,” In SLTU. 2014.
  * [10] Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli, “Unsupervised cross-lingual representation learning for speech recognition,” 2020.
  * [11] F. Eyben, M. Wöllmer, and B. Schuller, “Opensmile: the Munich versatile and fast open-source audio feature extractor,” in Proceedings of the 18th ACM International Conference on Multimedia, 2010, pp. 1459–1462.
  * [12] B. Schuller, S. Steidl, A. Batliner, J. Hirschberg, J. K. Burgoon, A. Baird, A. Elkins, Y. Zhang, E. Coutinho, K. Evanini et al., “The interspeech 2016 computational paralinguistics challenge: Deception, sincerity & native language,” in Proceedings of INTERSPEECH, 2016, pp. 2001–2005.
  * [13] F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. André, C. Busso, L. Y. Devillers, J. Epps, P. Laukka, S. S. Narayanan et al., “The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing,” IEEE Transactions on Affective Computing, vol. 7, no. 2, pp. 190–202, 2015.
  * [14] P. Tzirakis, S. Zafeiriou, and B. W. Schuller, “End2you–the imperial toolkit for multimodal profiling by end-to-end learning,” arXiv preprint arXiv:1802.01115, 2018.
  * [15] P. Tzirakis, J. Zhang, and B. W. Schuller, “End-to-end speech emotion recognition using deep neural networks,” In Proc. ICASSP, 2018, pp. 5089-5093.


# Supercurrent parity-meter in a nanowire Cooper-pair transistor

Ji-Yin Wang  QuTech and Kavli Institute of NanoScience, Delft University of
Technology, 2600 GA Delft, The Netherlands  These authors contributed equally
Constantin Schrade  Department of Physics, Massachusetts Institute of
Technology, 77 Massachusetts Avenue, Cambridge, Massachusetts 02139 USA  These
authors contributed equally  Vukan Levajac  David van Driel  Kongyi Li  QuTech
and Kavli Institute of NanoScience, Delft University of Technology, 2600 GA
Delft, The Netherlands  Sasa Gazibegovic  Ghada Badawy  Roy L.M. Op het Veld
Department of Applied Physics, Eindhoven University of Technology, 5600 MB
Eindhoven, The Netherlands  Joon Sue Lee  California NanoSystems Institute,
University of California Santa Barbara, Santa Barbara, California 93106, USA
Current address: Department of Physics and Astronomy, University of Tennessee,
Knoxville, Tennessee 37996, USA  Mihir Pendharkar  Electrical and Computer
Engineering, University of California Santa Barbara, Santa Barbara, California
93106, USA  Connor P. Dempsey  Electrical and Computer Engineering, University
of California Santa Barbara, Santa Barbara, California 93106, USA  Chris J.
Palmstrøm  California NanoSystems Institute, University of California Santa
Barbara, Santa Barbara, California 93106, USA  Electrical and Computer
Engineering, University of California Santa Barbara, Santa Barbara, California
93106, USA  Materials Department, University of California Santa Barbara,
Santa Barbara, California 93106, USA  Erik P.A.M. Bakkers  Department of
Applied Physics, Eindhoven University of Technology, 5600 MB Eindhoven, The
Netherlands  Liang Fu  Department of Physics, Massachusetts Institute of
Technology, 77 Massachusetts Avenue, Cambridge, Massachusetts 02139 USA  Leo
P. Kouwenhoven  QuTech and Kavli Institute of NanoScience, Delft University of
Technology, 2600 GA Delft, The Netherlands  Microsoft Station Q Delft, 2600 GA
Delft, The Netherlands  Jie Shen<EMAIL_ADDRESS>QuTech and Kavli
Institute of NanoScience, Delft University of Technology, 2600 GA Delft, The
Netherlands  Beijing National Laboratory for Condensed Matter Physics,
Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China

###### Abstract

We study a Cooper-pair transistor realized by two Josephson weak links that
enclose a superconducting island in an InSb-Al hybrid nanowire. When the
nanowire is subject to a magnetic field, isolated subgap levels arise in the
superconducting island and, due to the Coulomb blockade, mediate a
supercurrent by coherent co-tunneling of Cooper pairs. We show that the
supercurrent resulting from such co-tunneling events exhibits, for low to
moderate magnetic fields, a phase offset that discriminates even and odd
charge ground states on the superconducting island. Notably, this phase offset
persists when a subgap state approaches zero energy and, based on theoretical
considerations, permits parity measurements of subgap states by supercurrent
interferometry. Such supercurrent parity measurements could, in a new series
of experiments, provide an alternative approach for manipulating and
protecting quantum information stored in the isolated subgap levels of
superconducting islands.

When two superconducting (SC) leads couple via a Coulomb-blockaded quantum dot
(QD), the isolated energy levels on the dot mediate a supercurrent by coherent
co-tunneling of Cooper pairs bib:Spivak1991 . For the particular case of a
single-level QD, a control knob for the direction of the supercurrent is given
by the charge parity of dot electrons bib:Spivak1991 . Such a parity-
controlled supercurrent has been observed in a nanowire (NW) QD Josephson
junction (JJ) bib:vanDam2006 ; bib:Razmadze2020 . It is described by the
Josephson relation, $I=(-1)^{n_{0}}I_{c}\sin(\varphi)$, where $I_{c}$ is the
critical current, $\varphi$ is the SC phase difference, and $n_{0}$ is the
number of dot electrons. In general, the Josephson relation can also acquire a
phase offset, $\varphi\rightarrow\varphi+\varphi_{0}$ with $\varphi_{0}\neq
0,\pi$, when time-reversal and mirror symmetry are broken bib:Zazunov2009 .
This breaking occurs, for example, if a spin-orbit coupled QD is subject to a
magnetic field bib:Zazunov2009 ; bib:Brunetti2013 ; bib:Szombati2016 ;
bib:Schrade2017 .

A different possibility of coupling two SC leads is via an intermediate SC
island with finite charging energy; a ‘Cooper-pair transistor’ (CPT)
bib:Geerligs1990 ; bib:Tuominen1992 ; bib:Fulton1989 ; bib:Woerkom2015 ;
bib:Veen2018 ; bib:Proutski2019 ; bib:Schrade20184 . Unlike in the QD JJ, the
SC island carries, within its parity lifetime, an even number of electrons in
the ground state, as signified by a charging energy that is a $2e$-periodic
function of the island gate charge (e, elementary charge) bib:Tuominen1992 ;
bib:Woerkom2015 ; bib:Veen2018 . In particular, since the odd charge states
are energetically unfavorable for a conventional CPT, the Josephson relation
is not expected to exhibit a parity-controlled phase offset.

Figure 1: (Color online) a, False-color micrograph of the measured NbTiN
(green) SQUID device comprising an InSb-Al NW CPT in the right arm and an InSb
nanowire reference junction in the left arm. Top gates (L, R, REF) define
tunable JJs, and a plunger gate (P) controls the electron number on the SC
island of the nanowire CPT. b, Cross-sections along the lines shown in (a).
Gates are made of Ti/Au ($10/120$ nm) and are separated from the InSb-Al
nanowire by $30$ nm sputtered SiNx. The substrate, p-doped Si covered with
$285$ nm SiO2, acts as a global back gate.

Recently, a CPT has been realized with an Indium Arsenide-Aluminium (Al)
hybrid NW bib:Veen2018 ; bib:Proutski2019 . In this case, upon increasing a
magnetic field parallel to the NW, a transition from a $2e$-periodic switching
current to a switching current with even-odd pattern has been observed
bib:Veen2018 . The interpretation is that a low-energy subgap state arises in
the SC island, and, depending on its occupancy, the charge ground state
carries an even or an odd number of electrons. An open question is if the
Josephson relation of a NW CPT exhibits in the presence of subgap states a
parity-controlled phase offset?

Here, we address this question with a NW CPT integrated in a superconducting
quantum interference device (SQUID). We investigate the previously described
situation when the NW CPT is subject to a parallel magnetic field so that
subgap levels arise in the SC island and mediate a supercurrent by coherent
co-tunneling of Cooper pairs. We show that supercurrent resulting from Cooper
pair co-tunneling exhibits a phase offset, which distinguishes even and odd
charge ground states on the SC island. This phase offset persists when a
subgap state approaches zero energy and, based on theoretical considerations,
enables parity readout of low-energy subgap states. Such supercurrent parity
readout could provide a new approach for the manipulation bib:Beenakker2004 ;
bib:Mao2004 ; bib:Engel2005 ; bib:Lalumiere2010 ; bib:Ionicioiu2007 ;
bib:Pfaff2013  and protection bib:Andersen2019 ; bib:Bultink2020  of quantum
information stored in the isolated subgap levels of SC islands bib:Aasen2016 ;
bib:Plugge2017 ; bib:Karzig2017 ; bib:Schrade20182 ; bib:Schrade20183 .

The device geometry of our experiment is shown in Fig. 1. For realizing the
CPT, we use a shadow-grown bib:Gazibegovich2017  Al SC island on an Indium
Antimonide (InSb) NW, which couples to two SC Al leads via gate-tunable
tunneling barriers. A plunger gate is used for controlling the electron number
on the SC island. As we intend to study the full Josephson relation of the NW
CPT, we integrate our setup in a SQUID loop made of niobium-titanium nitride
(NbTiN) and a second InSb NW reference junction. The tunnel coupling of the
reference junction is adjustable by a local gate electrode.

Initially, we pinch off the reference junction and characterize the NW CPT by
measuring the differential conductance $\mathrm{d}I/\mathrm{d}V$ versus the
source-drain voltage $V$ and the plunger gate voltage $V_{P}$. Our results are
shown in Fig. 2a for zero and finite parallel magnetic fields $B_{\parallel}$.

At zero magnetic field, we observe a pattern of Coulomb diamonds with sharp
edges due to the weak island-lead coupling. Besides the Coulomb diamonds,
which signify the importance of charging effects on the SC island, we also
measure conductance peaks at zero-bias around the SC island’s charge
degeneracy points (insert curve in Fig. 2a). These peaks exhibit a
$2e$-periodicity and indicate that, in addition to charging effects, the
tunneling of Cooper pairs is also relevant in the presented transport regime.
Furthermore, above a onset voltage $V_{onset}$, a $1e$-periodic modulation of
the differential conductance appears, which marks the onset of quasiparticle
transport.

At finite magnetic fields, the aforementioned onset voltage for quasiparticle
transport persists. However, below the onset voltage, the Coulomb diamonds
split, resulting in an even-odd pattern. We attribute the appearance of this
even-odd pattern to low-energy subgap states that form on the SC island. More
specifically, the magnetic field induces a Zeeman splitting of spinful, odd-
parity states and, thereby, reduces the minimum single-particle excitation
energy in the NW CPT. As a result, odd-parity states that partially reside in
the InSb nanowire can detach from the quasiparticle continuum and, because of
their enhanced effective $g$-factor in comparison to the Al shell, form
isolated levels below the SC gap bib:Veen2018 ; bib:Vaitiekenas2018 .

Figure 2: (Color online) a, Differential conductance, $\text{d}I/\text{d}V$,
versus source-drain voltage $V$ and plunger gate voltage $V_{P}$. At zero
parallel magnetic field, the differential conductance shows a Coulomb diamond
pattern with a $2e$-periodicity. At $B_{\parallel}=100\,$mT, the
$2e$-periodicity of the Coulomb diamonds lifts due to the appearance of an
odd-parity charge ground state on the SC island. Inset curves show the
differential conductance at zero bias. b, Top panel: Switching current,
$I_{sw}$, versus parallel magnetic field $B_{\parallel}$ and plunger gate
voltage $V_{P}$. Bottom panel: Magnetic field dependence of the normalized
even and odd peak spacings, $S_{e}/(S_{e}+S_{o})$ and $S_{o}/(S_{e}+S_{o})$,
showing a transition from a $2e$-periodicity to an even-odd pattern. Such a
transition in the peak spacing is consistent with a state detaching from the
SC island’s quasiparticle continuum and forming an isolated subgap level
bib:Veen2018 .

Next, we investigate the subgap levels on the SC island in more detail. We
lower the island-lead tunneling barriers and, with the reference junction
still pinched off, measure the switching current $I_{sw}$ as a function of the
parallel magnetic field $B_{\parallel}$ and plunger gate voltage $V_{P}$. Our
results are depicted in Fig. 2b. At zero magnetic field, the switching current
exhibits a $2e$-periodic peak spacing implying that the SC island always
carries an even number of electrons in its charge ground state (see also Fig.
S1a in supplementary material). The situation changes upon applying a parallel
magnetic field. The magnetic field induces a splitting of the $2e$-periodic
peaks, and, as a result, the switching current exhibits a peak-spacing with an
even-odd pattern (see also Fig. S1b in supplementary material). Similar to the
differential conductance, we attribute the appearance of this even-odd pattern
to charge ground states with even and odd fermion parity on the SC island.
Moreover, the extracted peak spacing oscillates as a function of applied
magnetic field (bottom panel in fig.2b), as well as the plunger gate voltage,
indicating either the anticrossing or the crossing of the lowest-energy subgap
state with a second subgap state at higher energybib:Jie2018 ; bib:Jie2020 .

Figure 3: (Color online) a, Voltage drop $V$ across the NW CPT versus applied
bias current $I_{b}$ and plunger gate voltage $V_{P}$, showing an even-odd
pattern consistent with the appearance of low-energy subgap states at a
parallel magnetic field $B_{\parallel}=170\,$mT. b, Voltage drop $V$ as a
function of the applied bias current $I_{b}$ and the flux $\Phi$ that pierces
through the SQUID loop for the plunger gate voltages $V_{P}$ marked in (a).
The fitted switching current $I_{sw}$ (yellow) displays a phase offset
$\varphi_{0}$ that discriminates the even and odd charge parity sectors of the
SC island. c, Phase offset $\varphi_{0}$ (relative to the even Coulomb
valleys) versus plunger gate voltage $V_{P}$. Dashed lines are a guide to the
eye. In the range of plunger gate voltages shown here, the phase offset is
insensitive.

We now open the reference junction and measure the NW CPT’s full Josephson
relation in the presence of low-energy subgap states. For the results
presented here, we focus on the magnetic field strength $B_{\parallel}=170$
mT, and adopt a highly-asymmetric SQUID configuration so that the phase drop
occurs primarily across the NW CPT. Under these conditions, we apply a bias
current $I_{b}$ and measure the voltage drop $V$ across the SQUID as a
function of the plunger gate voltage $V_{P}$ and the flux $\Phi$ piercing
through the SC loop. Fig. 3 shows our measurement data, which we will now
discuss in more detail:

The main result of our measurements is that the Josephson relation of the NW
CPT exhibits a substantial relative phase offset $\varphi_{0}$ between Coulomb
valleys of opposite charge parity. To determine this phase offset for the
Coulomb valleys marked in Fig. 3a, we fit the switching current $I_{sw}$ as a
function of the flux $\Phi$. The fitted curves, shown in Fig. 3b, allow us to
extract $\varphi_{0}\sim-1.24\pi$ and $\varphi_{0}\sim-1.31\pi$ for the first
and second pair of Coulomb valleys, respectively. For the remaining pairs, we
find similar values for the phase offset, as summarized in Fig. 3c. Notably,
the leftmost pair of data points in Fig. 3c shows that phase offset persists
when the Coulomb peaks are approximately $1e$-spaced (see detailed analysis in
Fig. S2 in supplementary materials). Therefore, the phase offset facilitates
charge parity readout even if a subgap state approaches zero energy, and the
behavior of the SC island’s addition energy is identical for opposite charge
parity sectors.

Motivated by our measurements, we now discuss a mechanism for a parity-
dependent phase offset. We introduce a model for the NW CPT, which comprises a
mesoscopic SC island coupled to a pair of $s$-wave SC leads. In our model, we
focus on the two lowest isolated subgap levels in the SC island, $\pm E_{a}$
and $\pm E_{b}$,indicated by the peak spacing oscillation as a function of
magnetic field and plunger gate in 2b. These two levels form an effective spin
degree of freedom if the Coulomb blockade fixes the SC island’s total fermion
parity. In particular, the effective spin mediates a supercurrent, similar to
the electron spin in a QD JJ, through the co-tunneling of Cooper pairs. We
distinguish between two types of co-tunneling sequences:

(1) In the first type of sequence, shown in Fig. 4a, the Cooper pair splits so
that one electron tunnels via $\pm E_{a}$ while the other electron tunnels via
$\pm E_{b}$. For such a two-level sequence, the corresponding supercurrent
contribution acquires a prefactor given by the SC island charge parity,
$(-1)^{n_{0}}$. This parity prefactor is analogous to the parity prefactor
appearing in the Josephson relation of a QD JJ, where Cooper pairs tunnel via
two dot levels with opposite spin polarization bib:Spivak1991 .

(2) In the second type of sequence, shown in Fig. 4b, both Cooper pair
electrons tunnel via either $\pm E_{a}$ or $\pm E_{b}$. For such a single-
level sequence, each of the two electrons contributes a prefactor given by the
parity of $\pm E_{a}$ or $\pm E_{b}$. In particular, since the same parity
prefactor appears twice in the sequence, it squares to one. Consequently, in
the single-level supercurrent contribution a parity prefactor is absent.

Figure 4: (Color online) a, A typical sequence of intermediate states in which
a Cooper pair tunnels between the SC leads (left, right) via the two lowest
isolated subgap levels $a,b$ in the intermediate SC island (center). Such a
sequence yields a contribution to the supercurrent proportional to the joint
parity of the two subgap levels. In the illustration, numbers indicate the
sequence of tunneling events, and solid/empty dots represent filled/empty
subgap levels. b, A typical sequence of intermediate states that involves
Cooper pair transport via a single subgap level yielding no parity-dependent
prefactor.

If we collect all co-tunneling sequences of the first and second type, we
obtain the Josephson relation (see details in supplementary material section
4),

$I=(-1)^{n_{0}}I_{ab}\sin(\varphi+\varphi_{ab})+\sum_{\ell=a,b}I_{\ell}\sin(\varphi+\varphi_{\ell}).$
(1)

Here, $I_{ab}$ and $I_{\ell}$ are amplitudes, which are $1e$-periodic in the
gate charge if the lowest subgap level is at zero energy. Furthermore,
$\varphi_{ab},\varphi_{\ell}$ are phase offsets due to the simultaneous
breaking of time-reversal and mirror symmetry. In our setup, time-reversal
symmetry breaking arises from the magnetic field, inducing different tunneling
probabilities for spin-up and spin-down electrons. The breaking of mirror
symmetry results from asymmetric tunneling contacts so that Cooper pairs
tunneling across the left/right JJ acquire different tunneling phases.

At this point, it is worth highlighting two differences between the NW CPT and
a QD JJ: First, the island which mediates the Josephson current is in a SC
state, not a normal state as for a QD JJ. Consequently, not only conventional
tunneling events can occur, but also anomalous tunneling events in which an
electron is created/destroyed on both the SC island and the leads. Second, for
a QD JJ, the wavefuntions on the dot are highly localized which justifies a
point-like tunneling contact. In comparison, for a NW CPT, the subgap level
wavefunctions can be extended, which induces longer-range island-lead tunnel
couplings. In particular, such longer-range couplings can break the mirror
symmetry, due to the combined effect of spin-orbit coupling and magnetic field
in the tunneling region, and lead to additional contributions to
$\varphi_{ab},\varphi_{\ell}$ .

Returning to Eq. (1), the total phase offset is
$\varphi_{0}\equiv\text{arg}[-I_{ab}e^{i\varphi_{ab}}+\sum_{\ell}I_{\ell}e^{i\varphi_{\ell}}]-\text{arg}[I_{ab}e^{i\varphi_{ab}}+\sum_{\ell}I_{\ell}e^{i\varphi_{\ell}}]$.
In this expression, the parity prefactor flips upon tuning the gate charge of
the SC island between different charge parity sectors. As a result of these
parity-flips, the phase offset does not exhibit a $1e$-periodicity in the gate
charge even if one of the subgap states is at zero energy. Instead, if
$I_{ab}\neq 0$, $\varphi_{0}$ is always $2e$-periodic and permits the
measurement of the parity of the lowest subgap level. To practically enable
such parity measurements, the two-level contribution should be sizable,
$I_{ab}\gg I_{\ell}$. Also, to avoid thermal excitations, the temperature $T$
should be small compared to the level separation $|E_{a}-E_{b}|$.
Interestingly though, if $|E_{a}-E_{b}|\gtrsim T$, the parity prefactor
measures the joint parity of $\pm E_{a}$ and $\pm E_{b}$. Such joint parity
measurements could be leveraged for entangling qubits stored in the subgap
levels of SC islands bib:Beenakker2004 ; bib:Mao2004 ; bib:Engel2005 ;
bib:Lalumiere2010 ; bib:Ionicioiu2007 ; bib:Pfaff2013 .

So far, we have discussed a regime with substantial $\varphi_{0}$ for parity
read-out with maximal resolution. However, such an ideal situation is not
always realized. In Fig. 5a, we display the phase offset versus plunger gate
voltage for multiple magnetic field values. For a selection of data points, we
also show the fitted switching current $I_{sw}$ in Fig. 5b. Detailed analysis
is shown in Fig.S3-Fig.S5 in supplementary material. In comparison, there is
another regime in which NW CPT exhibiting phase independence on its parity
(see details in Fig.S6-Fig.S7 in supplementary material). In Fig. 5, our
findings are two-fold: First, we observe that the phase offset for subsequent
Coulomb valley pairs is tunable by the magnetic field and the plunger gate
voltage. Such a tunability arises because both control parameters change the
support of the subgap level wavefunction and, thereby, alter the lead-island
Josephson couplings. Second, we find that the phase offset decreases upon
increasing the magnetic field. This decrease suggests that the level
seperation between the lowest-energy and higher-energy subgap states increases
so that the supercurrent contribution with the parity-dependent prefactor
becomes energetically unfavorable.

In summary, we have studied the Josephson relation of an InSb-Al NW CPT. We
have demonstrated that upon applying a magnetic field, subgap levels arise in
the SC island and mediate a supercurrent with a parity-dependent phase offset.
We have shown that the phase offset persists when the subgap state approaches
zero energy and enables parity readout of the lowest energy subgap state. Such
a supercurrent parity readout could be useful for the manipulation
bib:Beenakker2004 ; bib:Mao2004 ; bib:Engel2005 ; bib:Lalumiere2010 ;
bib:Ionicioiu2007 ; bib:Pfaff2013  and protection bib:Andersen2019 ;
bib:Bultink2020  of qubits stored in the isolated subgap levels of SC islands
bib:Aasen2016 ; bib:Plugge2017 ; bib:Karzig2017 ; bib:Schrade20182 ;
bib:Schrade20183 .

Figure 5: (Color online) a, Phase offset $\varphi_{0}$ versus plunger gate
voltage $V_{P}$ for various parallel magnetic fields $B_{\parallel}$. Dashed
lines are a guide to the eye. The phase offset is sensitive to both plunger
gate voltage and magnetic field variations. b, Voltage drop $V$ as a function
of the applied bias current $I_{b}$ and the SQUID flux $\Phi$ for a parallel
magnetic field $B_{\parallel}=160\,$mT. The switching current $I_{sw}$
(yellow) displays a phase offset $\varphi_{0}$ between even (e) and odd (o)
Coulomb valleys of the SC island that is tunable by the plunger gate voltage
$V_{P}$.

##  I Method

Device fabrication. The InSb NWs used in the experiment were grown on an
Indium phosphide substrate by metalorganic vapor phase epitaxy. In the
molecular beam epitaxy chamber, Al flux was deposited along a specific
direction to form Al shadows on InSb NWs by neighboring NWs
bib:Gazibegovich2017 . InSb-Al NWs with shadows were transferred onto a doped
Si/SiOx substrate using a nano-manipulator installed inside an SEM. NbTiN
superconductor was sputter deposited right after Ar etching dedicated to
removing the oxidized layer. Subsequently, 30 nm SiNx was sputter deposited to
work as a dielectric layer, and 10/120 nm Ti/Au was used as a top gate.

Transport measurement. The sample was measured at a base temperature of
$\sim$20 mK in an Oxford dry dilution refrigerator equipped with a vector
magnet. Differential conductance was measured by applying small AC lock-in
excitation superimposed on a DC voltage and then measuring AC and DC current
through the device. Typically, low frequency of $\sim$ 27 Hz and AC excitation
amplitude of $\sim$10 $\mu$V were used for lock-in measurement. In current
bias measurement, current was applied through the device while monitoring
voltage drop on device. The direction of the magnetic field was aligned with
respect to the InSb-Al island arm by detecting the supercurrent of Cooper-pair
transistor while rotating the magnetic field direction.

Data taking. In this article, all data comes from one device.

##  II Data availability

The authors declare that all of the raw data together with analysis files are
available at https://zenodo.org/record/5075186$\\#$.YOQqW$\\_$kzaUk.

##  III Acknowledgement

We are grateful to Roman Lutchyn, Bernard van Heck, Michiel de Moor, Chun-Xiao
Liu and Marta Pita Vidal for fruitful discussion. This work has been supported
by the Dutch Organization for Scientific Research (NWO), the Foundation for
Fundamental Research on Matter (FOM) and Microsoft Corporation Station Q. J.S
acknowledges support from Chinese Academy of Sciences under Grant No.
XDB33000000, National Science Foundation of China under Grant No. 92065203,
and the Synergic Extreme Condition User Facility.

##  IV Author contribution

J.W. and J.S. conceived the experiment. J.W. and J.S. fabricated the device.
J.W., J.S. and D.V.D. performed the measurements. C.S. and L.F. did the theory
simulations. K.L. helped transfer the nanowires. S.G., G.B., R.L.M.O.H.V.,
J.S.L., M.P., C.P.D., C.J.P. and E.P.A.M.B. carried out the material growth.
J.W., J.S., C.S. and V.L. analyzed the data. C.S., J.S. and J.W. wrote
manuscript. J.S. supervised the project.

##  V Competing Interests

The authors declare no competing interests.

## References

  * (1) Spivak, B. I. & Kivelson, S. A. Negative local superfluid densities: The difference between dirty superconductors and dirty Bose liquids. Phys. Rev. B 43, 3740 (1991).
  * (2) van Dam, J. A. et al. Supercurrent reversal in quantum dots. Nature (London) 442, 667 (2006).
  * (3) Razmadze, D. et al. Quantum Dot Parity Effects in Trivial and Topological Josephson Junctions. Phys. Rev. Lett. 125, 116803 (2020).
  * (4) Zazunov, A. et al. Anomalous Josephson Current through a Spin-Orbit Coupled Quantum Dot. Phys. Rev. Lett. 103, 147004 (2009).
  * (5) Brunetti, A. et al. Anomalous Josephson current, incipient time-reversal symmetry breaking, and Majorana bound states in interacting multilevel dots. Phys. Rev. B 88, 144515 (2013).
  * (6) Szombati, D. B. et al. Josephson $\phi$0-junction in nanowire quantum dots. Nat. Phys. 12, 568 (2016).
  * (7) Schrade, C. et al. Detecting topological superconductivity with $\varphi$0 Josephson junctions. Phys. Rev. B 95, 195421 (2017).
  * (8) Fulton, T. A. et al. Observation of combined Josephson and charging effects in small tunnel junction circuits. Phys. Rev. Lett. 63, 1307 (1989).
  * (9) Geerligs, L. J. et al. Single Cooper-pair tunneling in small-capacitance junctions. Phys. Rev. Lett. 65, 377 (1990).
  * (10) Tuominen, M. T. et al. Experimental evidence for parity-based 2e periodicity in a superconducting single-electron tunneling transistor. Phys. Rev. Lett. 69, 1997 (1992).
  * (11) van Woerkom, D. J. et al. One minute parity lifetime of a NbTiN Cooper-pair transistor. Nature Physics 11, 547 (2015).
  * (12) van Veen, J. et al. Magnetic-field-dependent quasiparticle dynamics of nanowire single-Cooper-pair transistors. Phys. Rev. B 98, 174502 (2018).
  * (13) Proutski, A. et al. Broadband microwave spectroscopy of semiconductor nanowire-based Cooper-pair transistors. Phys. Rev. B 99, 220504 (2019)
  * (14) Schrade, C. & Fu, L. Andreev or Majorana, Cooper finds out. arXiv:1809.06370.
  * (15) Beenakker, C. W. J. et al. Charge Detection Enables Free-Electron Quantum Computation. Phys. Rev. Lett. 93, 20501 (2004).
  * (16) Mao, W. et al. Mesoscopic Quadratic Quantum Measurements. Phys. Rev. Lett. 93, 56803 (2004).
  * (17) Engel, H.-A. & Loss, D. Fermionic Bell-State Analyzer for Spin Qubits. Science 309, 586 (2005).
  * (18) Lalumière, K. et al. Tunable joint measurements in the dispersive regime of cavity QED. Phys. Rev. A 81, 40301 (2010).
  * (19) Ionicioiu, R. Entangling spins by measuring charge: A parity-gate toolbox. Phys. Rev. A 75, 32339 (2007).
  * (20) Pfaff, W. et al. Demonstration of entanglement-by-measurement of solid-state qubits. Nature Phys. 9, 29-33 (2013).
  * (21) Andersen, C. K. et al. Entanglement stabilization using ancilla-based parity detection and real-time feedback in superconducting circuits. npj Quantum Information 5, 69 (2019).
  * (22) Bultink, C. C. et al. Protecting quantum entanglement from leakage and qubit errors via repetitive parity measurements. Science Advances 20 Mar 2020: Vol. 6, no. 12.
  * (23) Aasen, D. et al. Milestones Toward Majorana-Based Quantum Computing. Phys. Rev. X 6, 031016 (2016).
  * (24) Plugge, S. et al. Majorana box qubits. New Journal of Physics 19, 012001 (2017).
  * (25) Karzig, T. et al. Scalable designs for quasiparticle-poisoning-protected topological quantum computation with Majorana zero modes. Phys. Rev. B 95, 235305 (2017).
  * (26) Schrade, C. & Fu, L. Majorana Superconducting Qubit. Phys. Rev. Lett 121, 267002 (2018).
  * (27) Schrade, C. & Fu, L. Quantum Computing with Majorana Kramers Pairs. arXiv:1807.06620.
  * (28) Gazibegovich, S. et al. Epitaxy of advanced nanowire quantum devices. Nature 548, 434 (2017)
  * (29) Vaitiekenas, S. et al. Effective g Factor of Subgap States in Hybrid Nanowires. Phys. Rev. Lett. 121, 037703 (2018).
  * (30) Shen, J. et al. Parity transitions in the superconducting ground state of hybrid InSb–Al Coulomb islands. Nat. Commun. 9, 4801 (2018).
  * (31) Shen, J. et al. A full parity phase diagram of a Majorana island. arXiv:2021.10118 (2020).