# The effects of strain in multi-orbital superconductors: the case of Sr2RuO4

Sophie Beck  Center for Computational Quantum Physics, Flatiron Institute, 162
5th Avenue, New York, NY 10010, USA  Alexander Hampel  Center for
Computational Quantum Physics, Flatiron Institute, 162 5th Avenue, New York,
NY 10010, USA  Manuel Zingl  Center for Computational Quantum Physics,
Flatiron Institute, 162 5th Avenue, New York, NY 10010, USA  Carsten Timm
Institute of Theoretical Physics, Technische Universität Dresden, 01062
Dresden, Germany  Würzburg-Dresden Cluster of Excellence ct.qmat, Technische
Universität Dresden, 01062 Dresden, Germany  Aline Ramires  Paul Scherrer
Institut, CH-5232 Villigen PSI, Switzerland

###### Abstract

Uniaxial strain experiments have become a powerful tool to unveil the
character of unconventional phases of electronic matter. Here we propose a
combination of the superconducting fitness analysis and density functional
theory (DFT) calculations in order to dissect the effects of strain in complex
multi-orbital quantum materials from a microscopic perspective. We apply this
framework to the superconducting state of Sr2RuO4, and argue that the recently
proposed orbitally anti-symmetric spin-triplet (OAST) order parameter
candidate has unique signatures under strain which are in agreement with
recent observations. In particular, we can account for the asymmetric
splitting of the critical temperatures for compressive strain along the
$\langle 100\rangle$ direction, and the reduction of the critical temperatures
for compressive strain along the $\langle 001\rangle$ and $\langle 110\rangle$
directions with a single free parameter.

The recent development of strain devices suitable for a variety of
experimental probes has opened a new direction of investigation of complex
quantum materials Hicks  _et al._ (2014a). The application of uniaxial strain
along different directions allows for the selective reduction of spatial
symmetries, which is key to uncover the character of the underlying phases of
matter in a variety of magnetic and superconducting materials Brodsky  _et
al._ (2017); Park  _et al._ (2018); Böhmer  _et al._ (2017); Kim  _et al._
(2018); Sun  _et al._ (2019). Strain can tune material parameters such as
orbital occupation Pfau  _et al._ (2019), Fermi surface geometry Hsu  _et al._
(2016), and topological properties Mutch  _et al._ (2019). Given the
complexity of most of the functional materials available today, a clear
understanding of the effects of strain from a microscopic perspective is
highly desirable.

Here we take as an example Sr2RuO4, a material whose superconducting order
parameter has been the focus of debate for more than 25 years Mackenzie  _et
al._ (2017). This system has recently been investigated by thermodynamic
Watson  _et al._ (2018); Steppke  _et al._ (2017), transport Barber  _et al._
(2018), angle-resolved photoemission spectroscopy (ARPES) Sunko  _et al._
(2019), nuclear magnetic resonance (NMR) Pustogow  _et al._ (2019), and muon
spin relaxation ($\mu$SR) Grinenko  _et al._ (2021) experiments under strain,
giving us important hints on the nature of the superconducting state in this
material.

The best contenders for the superconducting state of Sr2RuO4 are chiral order
parameters Rice and Sigrist (1995); Kallin and Berlinsky (2009); Kallin
(2012); Kallin and Berlinsky (2016). Chiral superconductivity is supported by
several experimental probes: polar Kerr rotation experiments reveal time-
reversal symmetry breaking (TRSB) at the superconducting critical temperature
$T_{c}$ Xia  _et al._ (2006), ultrasound attenuation experiments indicate a
two-component order parameter Lupien  _et al._ (2001); Ghosh  _et al._ (2020),
and the study of junctions suggest the presence of superconducting domains
Kidwingira  _et al._ (2006); Anwar  _et al._ (2013). Also, recent $\mu$SR
measurements under compressive strain along the $\langle 100\rangle$ direction
show a clear splitting between the superconducting critical temperature
$T_{c}$ and the temperature below which TRSB is observed ($T_{\text{TRSB}}$)
Grinenko  _et al._ (2021). A splitting is expected for a chiral
superconductor, but some of the features observed experimentally cannot be
accounted for by a simple phenomenological Ginzburg-Landau theory. First, the
transition seems to be rather asymmetric concerning the evolution of the two
temperatures: $T_{c}$ is strongly enhanced, while $T_{\text{TRSB}}$ remains
almost unchanged up to uniaxial strains of about $1\%$ Grinenko  _et al._
(2021). Second, the evolution of $T_{c}$ is strongly non-linear, and there is
no observable cusp of $T_{c}$ around zero strain Watson  _et al._ (2018). In
addition, recent experiments indicate that compressive strain along the
$\langle 001\rangle$ direction causes a reduction of $T_{c}$ Jerzembeck  _et
al._ (2021). This behavior is in contradiction with the expected enhancement
based on a weak-coupling scenario with an increased density of states (DOS)
given the proximity to the van Hove singularity. Furthermore, experiments
under compressive strain along the $\langle 110\rangle$ direction indicate a
mild suppression of the critical temperature Hicks  _et al._ (2014b).

The original proposal of a chiral superconducting state for Sr2RuO4 suggested
a $p$-wave triplet state with d-vector along the $z$-direction Rice and
Sigrist (1995). This state is hard to reconcile with new NMR experiments
Pustogow  _et al._ (2019): the reduction of the Knight shift for in-plane
fields is expected only for singlet or triplet states with an in-plane
d-vector. Furthermore, the standard chiral $p$-wave state has difficulties
accounting for thermodynamic Kittaka  _et al._ (2018) and transport Hassinger
_et al._ (2017) experiments indicating the presence of gap nodes. In this
context, some of us have recently proposed a new order parameter candidate: an
even-parity pseudospin-singlet chiral superconducting state with horizontal
line nodes Suh  _et al._ (2020). In the microscopic orbital basis, this
superconducting state in the $E_{g}$ channel is an $s$-wave orbital-
antisymmetric spin-triplet (OAST) which can be stabilized by local
interactions in the presence of a large effective Hund coupling and realistic
three-dimensional spin-orbit coupling Suh  _et al._ (2020).

Here we investigate the effects of strain on local order parameters with
$E_{g}$ symmetry. We conclude that the concept of _superconducting fitness_
Ramires and Sigrist (2016, 2017); Ramires  _et al._ (2018); Ramires (2021) in
conjunction with density functional theory (DFT) calculations can clarify the
origin of the unusual features of the evolution of $T_{c}$ and
$T_{\text{TRSB}}$ under strain along different directions in a consistent
manner. For that, we use the evolution of the normal state band structure from
DFT and a single free parameter. Our analysis stems from a microscopic
perspective and captures qualitatively new effects beyond a naive Ginzburg-
Landau construction. In particular, we discuss the microscopic origin of the
asymmetry of $T_{c}$ and $T_{\text{TRSB}}$ under strain along the $\langle
100\rangle$ direction, and how strain along the $\langle 001\rangle$ direction
reduces $T_{c}$, even though the DOS is enhanced. Moreover, we discuss the
behavior of the transition temperatures under strain along the $\langle
110\rangle$ direction.

The simplest form of superconducting fitness measures was introduced in the
context of two-orbital models Ramires  _et al._ (2018), therefore here we
focus on reduced models for Sr2RuO4 along the $k_{x}k_{z}$ and $k_{y}k_{z}$
planes, both with $D_{2h}$ symmetry (from now on we refer to these planes as
$XZ$ and $YZ$, respectively). Note that there are no two-dimensional
irreducible representations (irreps) in $D_{2h}$, so at first sight, it seems
that we have lost the discussion about the splitting of the degenerate
superconducting transitions in $E_{g}$ for the complete model with $D_{4h}$
symmetry. Here, the reduced models along the $XZ$ and $YZ$ planes should be
seen as degenerate. Their inequivalence under strain is a manifestation of the
symmetry breaking.

Sufficiently far from the Brillouin-zone diagonals ($k_{y}=\pm k_{x}$), the
bands close to the Fermi energy are dominated by only two of the Ru
$t_{2g}$-orbitals. For concreteness, here we consider the $XZ$ plane,
dominated by the $d_{xz}$ and $d_{xy}$ orbitals. Projecting into this
subspace, we obtain the following effective two-orbital Hamiltonian

$\displaystyle\mathcal{H}_{0}^{XZ}=\sum_{{\mathbb{k}}}\Psi^{\dagger}_{\mathbb{k}}\hat{H}_{0}^{XZ}({\mathbb{k}})\Psi_{\mathbb{k}},$
(1)

in the basis
$\Psi^{\dagger}_{\mathbb{k}}=(c_{{\mathbb{k}},xz{\uparrow}}^{\dagger},\allowbreak
c_{{\mathbb{k}},xz{\downarrow}}^{\dagger},\allowbreak
c_{{\mathbb{k}},xy{\uparrow}}^{\dagger},\allowbreak
c_{{\mathbb{k}},xy{\downarrow}}^{\dagger})$, with

$\displaystyle\hat{H}_{0}^{XZ}({\mathbb{k}})=\sum_{a,b=0}^{3}\tilde{h}_{ab}^{XZ}({\mathbb{k}})\,\hat{\tau}_{a}\otimes\hat{\sigma}_{b},$
(2)

where the $\tilde{h}_{ab}^{p}({\mathbb{k}})$ are real even functions of
momentum labelled by indexes $(a,b)_{p}$, with $a$ and $b$ corresponding to
$\hat{\tau}_{a}$ and $\hat{\sigma}_{b}$, Pauli matrices encoding the orbital
and the spin degrees of freedom, respectively ($\hat{\sigma}_{0}$ and
$\hat{\tau}_{0}$ are identity matrices), and $p=\\{XZ,YZ\\}$ corresponds to
the plane. There are, in principle, 16 functions labelled as $(a,b)_{XZ}$, but
in presence of time-reversal and inversion symmetries these are constrained to
only six: $(0,0)_{XZ}$, $(3,0)_{XZ}$, and $(2,1)_{XZ}$ in $A_{g}$, the first
two associated with intra-orbital hopping and the last with atomic spin-orbit
coupling (SOC); $(2,2)_{XZ}$ in $B_{1g}$ and $(2,3)_{XZ}$ in $B_{2g}$, both
associated with momentum-dependent SOC; and $(1,0)_{XZ}$ in $B_{3g}$
associated with inter-orbital hopping. Analogously, we parametrize the local
gap matrices in the orbital basis as

$\displaystyle\hat{\Delta}^{XZ}=d_{0}\,\hat{\tau}_{a}\otimes\hat{\sigma}_{b}\,(i\hat{\sigma}_{2}),$
(3)

where $d_{0}$ is the order parameter amplitude. Similar construction holds for
the $YZ$ plane. The derivation of the terms in the normal state Hamiltonian,
labelled as $(a,b)_{p}$, and order parameters, labelled as $[a,b]_{p}$, are
given in detail in the Supplemental Material (SM) SM , which also includes
references Ramires and Sigrist (2019); Vogt and Buttrey (1995); Barber  _et
al._ (2019); Blaha  _et al._ ; Perdew  _et al._ (1996); Marzari and Vanderbilt
(1997); Souza  _et al._ (2001); Kuneš  _et al._ (2010); Pizzi  _et al._
(2020); Parcollet  _et al._ (2015); Giannozzi  _et al._ (2009); Garrity  _et
al._ (2014); Min  _et al._ (2006); Sigrist and Ueda (1991).

From here on, we focus on even parity two-component order parameters with
$E_{g}$ symmetry, connecting to the OAST order parameter proposed in Suh  _et
al._ (2020), which reconciles several experimental observations. In the
complete description of Sr2RuO4 as a three-dimensional three-orbital model
(see details in the SM SM ), the order parameter is dominated by the basis
matrices $\\{[5,3],[6,3]\\}$. Once we project the complete model into the $XZ$
or $YZ$ planes, we identify $[5,3]\rightarrow[2,3]_{YZ}$ and
$[6,3]\rightarrow[2,3]_{XZ}$. A subdominant contribution to this symmetry
channel comes from $\\{[3,0],-[2,0]\\}$, which in the projected models are
identified as $[2,0]\rightarrow[1,0]_{YZ}$ and $[3,0]\rightarrow[1,0]_{XZ}$.
These are the only $E_{g}$ order parameters we can discuss within the
projected models. The quantitative analysis below is done within the two-
orbital models, and we use the correspondences summarized in Fig. 1 to connect
the results to the original 3-orbital problem with $D_{4h}$ symmetry.

Figure 1: The central panel schematically indicates the orbital content over
the Fermi surfaces of Sr2RuO4 in the $k_{z}$ = 0 plane. Highlighted in gray
are the XZ and YZ planes discussed in this work. The color red (blue, green)
encode the orbital $d_{xy}$ ($d_{yz}$, $d_{xz}$) content as low (bright) or
high (dark) color in each Fermi surface sheet (labelled as $\alpha$, $\beta$,
and $\gamma$). The $\lambda$-matrices make explicit the orbital configuration
of the pairs for the $\\{[5,3],[6,3]\\}$ and $\\{[3,0],-[2,0]\\}$ order
parameters with $E_{g}$ symmetry. These are written in the orbital basis
$(d_{yz},d_{xz},d_{xy})$. The long arrows indicate the mapping from the
superconducting order parameter components $[a,b]$ in $E_{g}$ for the three-
dimensional model with $D_{4h}$ symmetry to $[a,b]_{p}$ for the two-
dimensional models along the $XZ$ and $YZ$ planes.

Within the standard weak-coupling assumptions, the superconducting critical
temperature for a two-orbital system under strain can be written as Ramires
_et al._ (2018)

$\displaystyle T_{c}(s)$ $\displaystyle\approx$
$\displaystyle\frac{4e^{\gamma}}{\pi}\frac{\omega_{C}}{2}\exp{\left[\left(-\frac{1}{2|v|}-\delta(s)\right)\frac{1}{\alpha(s)}\right]},$
(4)

where $\gamma$ is the Euler constant, $\omega_{C}$ is a characteristic energy
cutoff, $|v|$ is the magnitude of the attractive interaction in the symmetry
channel of interest (the last two assumed to be strain independent). The
superconducting fitness functions are written as

$\displaystyle\alpha(s)=\frac{1}{16}\sum_{a}N_{a}(0,s)\langle||\hat{F}_{A}({\mathbb{k}}_{Fa},s)||^{2}\rangle_{\text{FS}_{a}},$
(5)

and

$\displaystyle\delta(s)=\frac{\omega_{C}^{2}}{32}\sum_{a}N_{a}(0,s)\left\langle\frac{||\hat{F}_{C}({\mathbb{k}}_{Fa},s)||^{2}}{q({\mathbb{k}}_{Fa})^{2}}\right\rangle_{\\!\\!\text{FS}_{a}},$
(6)

where $q({\mathbb{k}})=\epsilon_{a}({\mathbb{k}})-\epsilon_{b}({\mathbb{k}})$
corresponds to the energy difference between the two bands. The sum over $a$
corresponds to the sum over the Fermi surfaces with DOS $N_{a}(0,s)$ at the
Fermi level for strain $s$, and $\langle...\rangle_{\text{FS}_{a}}$
corresponds to the average over the respective Fermi surface.
$||\hat{M}||^{2}=\text{Tr}[\hat{M}\hat{M}^{\dagger}]$ corresponds to the
Frobenius norm of the matrix $\hat{M}$. The superconducting fitness matrices
are defined as

$\displaystyle\hat{F}_{A,C}({\mathbb{k}},s)(i\hat{\sigma}_{2})=\tilde{H}_{0}({\mathbb{k}},s)\tilde{\Delta}({\mathbb{k}})\pm\tilde{\Delta}({\mathbb{k}})\tilde{H}_{0}^{*}(-{\mathbb{k}},s),$
(7)

where
$\tilde{H}_{0}({\mathbb{k}},s)=[\hat{H}_{0}({\mathbb{k}},s)-\tilde{h}_{00}({\mathbb{k}},s)\hat{\sigma}_{0}\otimes\hat{\tau}_{0}]/|\tilde{h}({\mathbb{k}},s)|$,
with
$|\tilde{h}({\mathbb{k}},s)|^{2}=\sum_{(a,b)\neq(0,0)}|\tilde{h}_{ab}({\mathbb{k}},s)|^{2}$,
is the normalized normal state Hamiltonian, and
$\tilde{\Delta}({\mathbb{k}})=\hat{\Delta}({\mathbb{k}})/d_{0}$ is the
normalized gap matrix. The index $C$ ($A$) in $\hat{F}_{C(A)}({\mathbb{k}},s)$
corresponds to the (anti-)commutator nature of this quantity. A finite
$\hat{F}_{A}({\mathbb{k}},s)$ corresponds to the weight of intra-band pairing,
guaranteeing a robust weak coupling instability. Conversely, a finite
$\hat{F}_{C}({\mathbb{k}},s)$ corresponds to the weight of inter-band pairing,
which is detrimental to the superconducting instability and reduces the
critical temperature. The evaluation of $\hat{F}_{A}({\mathbb{k}})$ for the
local order parameters in $E_{g}$ within the two-orbital models is summarized
in Table 1.

For small strain, we can determine the evolution of the superconducting
fitness functions using the evolution of the hopping amplitudes and the total
DOS evaluated by DFT calculations (see SM SM  for quantitative estimates).
Given the proximity of the two Fermi surfaces at the XZ and YZ planes, we
assume that the superconducting fitness functions for each Fermi surface are
going to be approximately the same. We define

$\displaystyle\langle||\hat{F}_{A}({\mathbb{k}},s)||^{2}\rangle$
$\displaystyle\approx$
$\displaystyle\langle||\hat{F}_{A}({\mathbb{k}},0)||^{2}\rangle(1+F_{1}s+F_{2}s^{2}),$
(8)

for a representative Fermi vector, and the total density of states

$\displaystyle N(0,s)$ $\displaystyle\approx$ $\displaystyle
N(0,0)(1+N_{1}s+N_{2}s^{2}).$ (9)

The coefficients $F_{1,2}$ and $N_{1,2}$ are summarized in Table 2 for
compressive strain along different directions.

| $(3,0)_{p}$ | $(2,1)_{p}$ | $(2,2)_{p}$ | $(2,3)_{p}$ | $(1,0)_{p}$
---|---|---|---|---|---
$[2,3]_{p}$ | 0 | 0 | 0 | 1 | 0
$[1,0]_{p}$ | 0 | 0 | 0 | 0 | 1
Table 1:  Superconducting fitness analysis for the effective two-orbital
models. Each line corresponds to an order parameter with matrix structure
$[a,b]_{p}$. The numerical entries correspond to
$||\hat{F}_{A}({\mathbb{k}},s)||^{2}=4\sum_{cd}(\text{table
entry})\,|\tilde{h}_{cd}({\mathbb{k}},s)|^{2}/|\tilde{h}({\mathbb{k}},s)|^{2}$,
for each term $(c,d)_{p}$ in the normal-state Hamiltonian. The corresponding
table for $||\hat{F}_{C}({\mathbb{k}},s)||^{2}$ is obtained by exchanging
$0\leftrightarrow 1$.

In order to get a simpler form for the evolution of the critical temperature
with strain, we use the relation
$||\hat{F}_{A}({\mathbb{k}})||^{2}+||\hat{F}_{C}({\mathbb{k}})||^{2}=4$ and
write

$\displaystyle\alpha(s)$
$\displaystyle\approx\alpha(0)(1+N_{1}s+N_{2}s^{2})(1+F_{1}s+F_{2}s^{2}),$
(10) $\displaystyle\delta(s)$
$\displaystyle\approx\delta(0)(1+N_{1}s+N_{2}s^{2})\left[1-A(F_{1}s+F_{2}s^{2})\right],$
(11)

where
$A=\langle||\hat{F}_{A}(0)||^{2}/q^{2}\rangle_{\text{FS}a}/\langle||\hat{F}_{C}(0)||^{2}/q^{2}\rangle_{\text{FS}a}$,
with an implicit ${\mathbb{k}}$ dependence. For the $[2,3]_{XZ/YZ}$ order
parameters, we estimate $A\approx 10^{-5}$, and for the $[1,0]_{XZ/YZ}$ order
parameters $A=0$ (see quantitative discussion in the SM SM ). The small value
of $A$ allows us to neglect the dependence of $\delta(s)$ on strain through
the $F_{1,2}$ coefficients. Furthermore, within the assumption that $1\gg
2|v|\delta(0)$, we take $\delta(s)\approx\delta(0)$, such that the dependence
of the critical temperature on strain is carried only by the fitness function
$\alpha(s)$. Within these considerations, the closed form equation for the
evolution of $T_{c}$ can be cast as

$\displaystyle\frac{T_{c}(s)}{T_{c}(0)}$ $\displaystyle\approx$
$\displaystyle\exp{\left[g\left(1-\frac{\alpha(0)}{\alpha(s)}\right)\right]},$
(12)

where $g=\left[1/(2|v|)+\delta(0)\right]/\alpha(0)$, the only free parameter
in this analysis, is chosen such that our results are in good agreement with
the experimentally observed value of $T_{c}(s=-0.5\%)\approx 2.9$K for strain
along the $\langle 100\rangle$ direction Steppke  _et al._ (2017). Our results
for strain along different directions are summarized in Fig. 2. Note that for
strain along the $\langle 100\rangle$ direction, the transition temperatures
for $[2,3]_{XZ}$ and $[2,3]_{YZ}$ evolve differently, indicating the splitting
of the two originally degenerate components $\\{[5,3],[6,3]\\}$ in $E_{g}$. We
associate the higher temperature with $T_{c}$ and the lower temperature with
$T_{\text{TRSB}}$. As expected, for strain along the $\langle 001\rangle$
direction $T_{c}$ and $T_{\text{TRSB}}$ are the same as there is no symmetry
breaking. Finally, for strain along the $\langle 110\rangle$ direction there
is symmetry breaking, but its effect cannot be captured by our approach
considering only the $XZ$ and $YZ$ planes since these planes remain equivalent
(see further discussion in the SM SM ).

Figure 2: Evolution of the critical temperatures under compressive uniaxial
strain along the $\langle 100\rangle$, $\langle 001\rangle$ and $\langle
110\rangle$ directions. (a) Order parameter $\\{[5,3],[6,3]\\}$ with $g=5.8$.
(b) Order parameter $\\{[3,0],-[2,0]\\}$ with $g=5$. The horizontal dashed
line is a guide to the eye corresponding to the unstrained critical
temperature. The black (red) circles at $s=-0.5\%$ correspond to the
experimental value for strain along the $\langle 100\rangle$ ($\langle
001\rangle$) direction. For both plots we set the representative $k_{F}=2.67$
to be between the $\beta$ and $\gamma$ Fermi surfaces.

In order to qualitatively understand the evolution of the critical
temperatures as a function of strain along different directions, we summarize
here some of the properties of the parameters in the normal state Hamiltonian
along the $XZ$ plane. The dominant term within $(3,0)_{XZ}$ is associated with
the imbalance in the intra-orbital nearest-neighbour hopping amplitudes for
the $d_{xz}$ and $d_{xy}$ orbitals. $(2,1)_{XZ}$ is associated with the
correlation-enhanced effective atomic SOC, taken as $\eta\approx 0.2$ eV Tamai
_et al._ (2019). $(2,3)_{XZ}$ is associated with momentum-dependent SOC in the
$B_{2g}$ irrep. As an estimate, here we take its value to be equal to one
tenth the hopping amplitude with the same inter-orbital structure Suh  _et
al._ (2020), $t_{xy/xz}^{\text{SOC}}(\hat{d}_{d})\approx
t_{xy/xz}(\hat{d}_{d})/10\approx 0.68$ meV. By symmetry, $(2,2)_{XZ}$ and
$(1,0)_{XZ}$ are zero along the $XZ$ plane. We then focus on the evolution of
the terms $(3,0)_{XZ}$ and $(2,3)_{XZ}$ with strain. The analysis for the $YZ$
plane leads to a similar conclusion. The evolution of the hopping amplitudes
and DOS as a function of strain is determined by DFT calculations, performed
without SOC. The correlation-enhanced effective atomic SOC is taken by its
agreement with the experimental Fermi surfaces Tamai  _et al._ (2019) and was
verified not to change significantly with strain (this result will be
published elsewhere).

For compressive strain along the $\langle 100\rangle$ direction, we see from
Table 2 that the dominant coefficients defining the evolution of the critical
temperature with strain are associated with the evolution of the DOS
($|N_{1,2}|>|F_{1,2}|$). The effect of the fitness measure
$||\hat{F}_{A}({\mathbb{k}},s)||^{2}$ is simply to slow down the evolution of
the critical temperatures, as $F_{1,2}$ and $N_{1,2}$ have opposite sign. To
note here is that at the linear level the evolution of the DOS, and therefore
of the critical temperatures, is already rather asymmetric. Furthermore, for
the $[5,3]$ component there is a very large quadratic contribution to the
evolution of the DOS due to the proximity to a van Hove singularity, which
reflects on the evolution of the critical temperature, as shown in Fig. 2 (a)
(black curves). Given these coefficients obtained for low strain, we would
expect an upturn of $T_{\text{TRSB}}$ for $s=-1.7\%$. While this is a
relatively high strain value, the upturn can be accessed in experiments by
deviations from linear behavior at low strains.

Compressive strain along the $\langle 001\rangle$ direction does not reduce
the symmetry group, and as a consequence there is no splitting of the
superconducting transition temperatures. For this strain direction, we find
$|F_{1}|>|N_{1}|$, what indicates that the behavior of the critical
temperature as a function of strain is dominated by the evolution of the
superconducting fitness measure $||\hat{F}_{A}({\mathbb{k}},s)||^{2}$. This is
in contrast to the discussion above for strain along the $\langle 100\rangle$,
with the evolution of the critical temperature dominated by the DOS. In order
to understand the evolution of the fitness measure under compressive strain,
we can look more carefully at how each term in the normal state Hamiltonian
evolves with strain. The superconducting fitness analysis in Table 1 tells us
that, for the order parameter $[2,3]_{XZ}$, the normal state term $(3,0)_{XZ}$
contributes to a finite $\hat{F}_{C}({\mathbb{k}})$ and $(2,3)_{XZ}$
contributes to a finite $\hat{F}_{A}({\mathbb{k}})$. For compressive strain
along the $\langle 001\rangle$ direction, the magnitude of $(3,0)_{XZ}\propto
t_{xz}(\hat{x})-t_{xy}(\hat{x})$ [here $t_{\gamma}(\hat{n})$ stands for the
intra-orbital hopping amplitude for orbital $\gamma$ along the $\hat{n}$
direction] is enhanced, leading to an increase of $\hat{F}_{C}({\mathbb{k}})$,
and $(2,3)_{XZ}$ is reduced, leading to a reduction in
$\hat{F}_{A}({\mathbb{k}})$, such that the coefficient $F_{1}$ is negative.
Interestingly, at the quadratic level the evolution of the critical
temperature is dominated by the DOS, as $|N_{2}|>|F_{2}|$ and we predict an
upturn of the critical temperature for strain values of approximately
$-2.2\%$. The results for the $\langle 001\rangle$ direction are shown in Fig.
2 (a) (red curve).

We also consider the behavior of the critical temperatures under compressive
strain along the $\langle 110\rangle$ direction. Strain along this direction
reduces the point group symmetry, but as we are modelling only the $XZ$ and
$YZ$ planes, we cannot capture the symmetry breaking and our results show the
same strain evolution for both critical temperatures. In this case the
behavior is dominated by the evolution of the DOS, this time in agreement with
the evolution of the superconducting fitness function, as $F_{1,2}$ and
$N_{1,2}$ have the same sign. We observe an overall reduction of the critical
temperature for both components. These results are summarized in Fig. 2 (a)
(blue curve). For this direction an upturn of the critical temperatures would
also be expected for $s\approx-2.9\%$. In a more complete treatment with a
three dimensional Fermi surface, we expect to see a splitting of the critical
temperatures away from the line we currently obtained. This suggests that if
the splitting is smaller than the absolute reduction of the critical
temperatures we have observed here, both $T_{c}$ and $T_{\text{TRSB}}$ could
be suppressed as a function of compressive strain, as preliminary results
indicate Klauss .

Concerning the other two component order parameter in $E_{g}$,
$\\{[3,0],-[2,0]\\}$, the strain dependence of $\hat{F}_{A}({\mathbb{k}})$ is
defined by the evolution of $\tilde{h}_{10}({\mathbb{k}})$, but, by symmetry,
this term is zero along the $XZ$ and $YZ$ planes. As consequence, within our
approach, the evolution of the critical temperature is controlled only by the
evolution of the DOS through the $N_{1,2}$ coefficients in Table 2. These
results are summarized in Fig. 2 (b). Note that even though the behavior for
strain along the $\langle 100\rangle$ direction is similar to the
$\\{[5,3],[6,3]\\}$ scenario (black curves), now compressive strain along the
$\langle 001\rangle$ direction leads to an enhancement of $T_{c}$ (red curve),
while strain along the $\langle 110\rangle$ reduces the critical temperature
(blue curve).

Here we would like to highlight that the reduction of the critical temperature
for strain along the $\langle 001\rangle$ direction seems to be a unique
feature for the order parameter in $E_{g}$ dominated by $\\{[5,3],[6,3]\\}$,
associated with the recently proposed OAST superconducting state. From a naive
weak-coupling scenario, we would expect an increase in $T_{c}$ based on the
enhanced DOS under strain, as recently discussed in Ref. Jerzembeck  _et al._
(2021).

In conclusion, this work proposes a microscopic perspective for the
understanding of the effects of strain in superconductors with multiple
degrees of freedom based on the superconducting fitness analysis. This
construction is particularly useful in order to go beyond the predictions of
effective Ginzburg-Landau theories. Assuming an OAST superconducting state
with $E_{g}$ symmetry, we are able to account for the evolution of the
critical temperatures under strain along different directions with a single
free parameter $g$, associated with the strength of the interactions in this
specific pairing channel. In particular, we find that compressive strain along
the $\langle 100\rangle$ direction can lead to an asymmetric splitting of
$T_{c}$ and $T_{\text{TRSB}}$, and that compressive strain along the $\langle
001\rangle$ direction reduces $T_{c}$, even though the DOS at the Fermi
surface is enhanced. We also find that up to splitting, both $T_{c}$ and
$T_{\text{TRSB}}$ are reduced under compressive strain along the $\langle
110\rangle$ direction. These results indicate that the phenomenology of the
OAST order parameter satisfies important constraints imposed by recent
experiments. We believe that this kind of analysis can be extremely useful for
understanding the behavior of other complex superconductors under strain and
to ultimately determine the symmetry of the superconducting order parameter in
these materials.

Direction | SC Component | $N_{1}(\%s)^{-1}$ | $N_{2}(\%s)^{-2}$ | $F_{1}(\%s)^{-1}$ | $F_{2}(\%s)^{-2}$
---|---|---|---|---|---
$\langle 100\rangle$ | $[5,3]$ | $-0.0928$ | $+0.4143$ | $+0.0333$ | $-0.0125$
$[6,3]$ | $+0.2536$ | $+0.0797$ | $-0.0091$ | $-0.0042$
$\langle 001\rangle$ | $\\{[5,3],[6,3]\\}$ | $-0.0503$ | $+0.0072$ | $+0.0589$ | $-0.0024$
$\langle 110\rangle$ | $\\{[5,3],[6,3]\\}$ | $+0.0296$ | $+0.0074$ | $+0.0129$ | $-0.0004$
Table 2: Coefficients for the evaluation of the evolution of the critical
temperature as a function of strain for the $\\{[5,3],[6,3]\\}$ order
parameter inferred from the analysis of the two-orbital models along the $XZ$
and $YZ$ planes. $N_{1}$ and $N_{2}$: Coefficients determining the evolution
of the density of states at the Fermi energy. $F_{1}$ and $F_{2}$:
Coefficients determining the evolution of
$||\hat{F}_{A}({\mathbb{k}},s)||^{2}$ (as in Eq. 8), obtained by selecting a
representative $k_{F}=2.67$ to be between the $\beta$ and $\gamma$ Fermi
surfaces. Note that compressive strain correspond to negative values.

###### Acknowledgements.

The authors would like to thank S. Ghosh, V. Grinenko, C. Hicks, F.
Jerzembeck, and H.-H. Klauss for interesting discussions. The Flatiron
Institute is a division of the Simons Foundation. C.T. acknowledges support by
the DFG through the Collaborative Research Center SFB 1143, project A04, the
Research Training Group GRK 1621, and the Cluster of Excellence on Complexity
and Topology in Quantum Matter ct.qmat (EXC 2147). A.R. acknowledges the
financial support of the Swiss National Science Foundation through an
Ambizione Grant No. 186043.

## References

  * Hicks  _et al._ (2014a) Clifford W. Hicks, Mark E. Barber, Stephen D. Edkins, Daniel O. Brodsky, and Andrew P. Mackenzie, “Piezoelectric-based apparatus for strain tuning,” Review of Scientific Instruments 85, 065003 (2014a).
  * Brodsky  _et al._ (2017) Daniel O. Brodsky, Mark E. Barber, Jan A. N. Bruin, Rodolfo A. Borzi, Santiago A. Grigera, Robin S. Perry, Andrew P. Mackenzie, and Clifford W. Hicks, “Strain and vector magnetic field tuning of the anomalous phase in Sr3Ru2O7,” Science Advances 3, e1501804 (2017).
  * Park  _et al._ (2018) Joonbum Park, Hideaki Sakai, Onur Erten, Andrew P. Mackenzie, and Clifford W. Hicks, “Effect of applied orthorhombic lattice distortion on the antiferromagnetic phase of CeAuSb2,” Phys. Rev. B 97, 024411 (2018).
  * Böhmer  _et al._ (2017) A. E. Böhmer, A. Sapkota, A. Kreyssig, S. L. Bud’ko, G. Drachuck, S. M. Saunders, A. I. Goldman, and P. C. Canfield, “Effect of biaxial strain on the phase transitions of Ca(Fe1-xCox)As2,” Phys. Rev. Lett. 118, 107002 (2017).
  * Kim  _et al._ (2018) H.-H. Kim, S. M. Souliou, M. E. Barber, E. Lefrançois, M. Minola, M. Tortora, R. Heid, N. Nandi, R. A. Borzi, G. Garbarino, A. Bosak, J. Porras, T. Loew, M. König, P. J. W. Moll, A. P. Mackenzie, B. Keimer, C. W. Hicks, and M. Le Tacon, “Uniaxial pressure control of competing orders in a high-temperature superconductor,” Science 362, 1040–1044 (2018).
  * Sun  _et al._ (2019) Dan Sun, Dmitry A. Sokolov, Jack M. Bartlett, Jhuma Sannigrahi, Seunghyun Khim, Pallavi Kushwaha, Dmitry D. Khalyavin, Pascal Manuel, Alexandra S. Gibbs, Hidenori Takagi, Andrew P. Mackenzie, and Clifford W. Hicks, “Magnetic frustration and spontaneous rotational symmetry breaking in PdCrO2,” Phys. Rev. B 100, 094414 (2019).
  * Pfau  _et al._ (2019) H. Pfau, C. R. Rotundu, J. C. Palmstrom, S. D. Chen, M. Hashimoto, D. Lu, A. F. Kemper, I. R. Fisher, and Z.-X. Shen, “Detailed band structure of twinned and detwinned BaFe2As2 studied with angle-resolved photoemission spectroscopy,” Phys. Rev. B 99, 035118 (2019).
  * Hsu  _et al._ (2016) Yi-Ting Hsu, Weejee Cho, Alejandro Federico Rebola, Bulat Burganov, Carolina Adamo, Kyle M. Shen, Darrell G. Schlom, Craig J. Fennie, and Eun-Ah Kim, “Manipulating superconductivity in ruthenates through fermi surface engineering,” Phys. Rev. B 94, 045118 (2016).
  * Mutch  _et al._ (2019) Joshua Mutch, Wei-Chih Chen, Preston Went, Tiema Qian, Ilham Zaky Wilson, Anton Andreev, Cheng-Chien Chen, and Jiun-Haw Chu, “Evidence for a strain-tuned topological phase transition in ZrTe5,” Science Advances 5, eaav9771 (2019).
  * Mackenzie  _et al._ (2017) Andrew P. Mackenzie, Thomas Scaffidi, Clifford W. Hicks, and Yoshiteru Maeno, “Even odder after twenty-three years: the superconducting order parameter puzzle of Sr2RuO4,” npj Quantum Materials 2, 40 (2017).
  * Watson  _et al._ (2018) Christopher A. Watson, Alexandra S. Gibbs, Andrew P. Mackenzie, Clifford W. Hicks, and Kathryn A. Moler, “Micron-scale measurements of low anisotropic strain response of local ${T}_{c}$ in Sr2RuO4,” Phys. Rev. B 98, 094521 (2018).
  * Steppke  _et al._ (2017) Alexander Steppke, Lishan Zhao, Mark E. Barber, Thomas Scaffidi, Fabian Jerzembeck, Helge Rosner, Alexandra S. Gibbs, Yoshiteru Maeno, Steven H. Simon, Andrew P. Mackenzie, and Clifford W. Hicks, “Strong peak in tc of Sr2RuO4 under uniaxial pressure,” Science 355, eaaf9398 (2017).
  * Barber  _et al._ (2018) M. E. Barber, A. S. Gibbs, Y. Maeno, A. P. Mackenzie, and C. W. Hicks, “Resistivity in the vicinity of a van hove singularity: Sr2RuO4 under uniaxial pressure,” Phys. Rev. Lett. 120, 076602 (2018).
  * Sunko  _et al._ (2019) Veronika Sunko, Edgar Abarca Morales, Igor Marković, Mark E. Barber, Dijana Milosavljević, Federico Mazzola, Dmitry A. Sokolov, Naoki Kikugawa, Cephise Cacho, Pavel Dudin, Helge Rosner, Clifford W. Hicks, Philip D. C. King, and Andrew P. Mackenzie, “Direct observation of a uniaxial stress-driven lifshitz transition in Sr2RuO4,” npj Quantum Materials 4, 46 (2019).
  * Pustogow  _et al._ (2019) A. Pustogow, Yongkang Luo, A. Chronister, Y. S. Su, D. A. Sokolov, F. Jerzembeck, A. P. Mackenzie, C. W. Hicks, N. Kikugawa, S. Raghu, E. D. Bauer, and S. E. Brown, “Constraints on the superconducting order parameter in Sr2RuO4 from oxygen-17 nuclear magnetic resonance,” Nature (London) 574, 72–75 (2019).
  * Grinenko  _et al._ (2021) Vadim Grinenko, Shreenanda Ghosh, Rajib Sarkar, Jean-Christophe Orain, Artem Nikitin, Matthias Elender, Debarchan Das, Zurab Guguchia, Felix Brückner, Mark E Barber, Joonbum Park, Naoki Kikugawa, Dmitry A Sokolov, Jake S Bobowski, Takuto Miyoshi, Yoshiteru Maeno, Andrew P Mackenzie, Hubertus Luetkens, Clifford W Hicks, and Hans-Henning Klauss, “Split superconducting and time-reversal symmetry-breaking transitions in Sr2RuO4 under stress,” Nature Physics 17, 748–754 (2021).
  * Rice and Sigrist (1995) T M Rice and M Sigrist, “Sr2RuO4: an electronic analogue of 3He?” Journal of Physics: Condensed Matter 7, L643–L648 (1995).
  * Kallin and Berlinsky (2009) C Kallin and A J Berlinsky, “Is Sr2RuO4 a chiral p-wave superconductor?” Journal of Physics: Condensed Matter 21, 164210 (2009).
  * Kallin (2012) Catherine Kallin, “Chiral p-wave order in Sr2RuO4,” Reports on Progress in Physics 75, 042501 (2012).
  * Kallin and Berlinsky (2016) Catherine Kallin and John Berlinsky, “Chiral superconductors,” Reports on Progress in Physics 79, 054502 (2016), arXiv:1512.01151 [cond-mat.supr-con] .
  * Xia  _et al._ (2006) Jing Xia, Yoshiteru Maeno, Peter T. Beyersdorf, M. M. Fejer, and Aharon Kapitulnik, “High resolution polar kerr effect measurements of Sr2RuO4: Evidence for broken time-reversal symmetry in the superconducting state,” Phys. Rev. Lett. 97, 167002 (2006).
  * Lupien  _et al._ (2001) C. Lupien, W. A. MacFarlane, Cyril Proust, Louis Taillefer, Z. Q. Mao, and Y. Maeno, “Ultrasound attenuation in Sr2RuO4: An angle-resolved study of the superconducting gap function,” Phys. Rev. Lett. 86, 5986–5989 (2001).
  * Ghosh  _et al._ (2020) Sayak Ghosh, Arkady Shekhter, F. Jerzembeck, N. Kikugawa, Dmitry A. Sokolov, Manuel Brando, A. P. Mackenzie, Clifford W. Hicks, and B. J. Ramshaw, “Thermodynamic Evidence for a Two-Component Superconducting Order Parameter in Sr2RuO4,” Nature Physics (2020).
  * Kidwingira  _et al._ (2006) Francoise Kidwingira, J. D. Strand, D. J. Van Harlingen, and Yoshiteru Maeno, “Dynamical superconducting order parameter domains in Sr2RuO4,” Science 314, 1267–1271 (2006).
  * Anwar  _et al._ (2013) M. S. Anwar, Taketomo Nakamura, S. Yonezawa, M. Yakabe, R. Ishiguro, H. Takayanagi, and Y. Maeno, “Anomalous switching in Nb/Ru/Sr2RuO4 topological junctions by chiral domain wall motion,” Scientific Reports 3, 2480 (2013).
  * Jerzembeck  _et al._ (2021) Fabian Jerzembeck, Henrik S. Røising, Alexander Steppke, Helge Rosner, Dmitry A. Sokolov, Naoki Kikugawa, Thomas Scaffidi, Steven H. Simon, Andrew P. Mackenzie, and Clifford W. Hicks, “The Superconductivity of Sr2RuO4 Under c-Axis Uniaxial Stress,” arXiv e-prints , arXiv:2111.06228 (2021).
  * Hicks  _et al._ (2014b) Clifford W. Hicks, Daniel O. Brodsky, Edward A. Yelland, Alexandra S. Gibbs, Jan A. N. Bruin, Mark E. Barber, Stephen D. Edkins, Keigo Nishimura, Shingo Yonezawa, Yoshiteru Maeno, and Andrew P. Mackenzie, “Strong increase of ¡i¿t¡/i¿¡sub¿c¡/sub¿ of Sr2RuO4 under both tensile and compressive strain,” Science 344, 283–285 (2014b).
  * Kittaka  _et al._ (2018) Shunichiro Kittaka, Shota Nakamura, Toshiro Sakakibara, Naoki Kikugawa, Taichi Terashima, Shinya Uji, Dmitry A. Sokolov, Andrew P. Mackenzie, Koki Irie, Yasumasa Tsutsumi, Katsuhiro Suzuki, and Kazushige Machida, “Searching for gap zeros in Sr2RuO4 via field-angle-dependent specific-heat measurement,” Journal of the Physical Society of Japan 87, 093703 (2018).
  * Hassinger  _et al._ (2017) E. Hassinger, P. Bourgeois-Hope, H. Taniguchi, S. René de Cotret, G. Grissonnanche, M. S. Anwar, Y. Maeno, N. Doiron-Leyraud, and Louis Taillefer, “Vertical line nodes in the superconducting gap structure of Sr2RuO4,” Phys. Rev. X 7, 011032 (2017).
  * Suh  _et al._ (2020) Han Gyeol Suh, Henri Menke, P. M. R. Brydon, Carsten Timm, Aline Ramires, and Daniel F. Agterberg, “Stabilizing even-parity chiral superconductivity in Sr2RuO4,” Phys. Rev. Research 2, 032023 (2020).
  * Ramires and Sigrist (2016) Aline Ramires and Manfred Sigrist, “Identifying detrimental effects for multiorbital superconductivity: Application to Sr2RuO4,” Phys. Rev. B 94, 104501 (2016).
  * Ramires and Sigrist (2017) Aline Ramires and Manfred Sigrist, “A note on the upper critical field of Sr2RuO4 under strain,” J. Phys.: Conf. Ser. 807 (2017).
  * Ramires  _et al._ (2018) Aline Ramires, Daniel F. Agterberg, and Manfred Sigrist, “Tailoring ${T}_{c}$ by symmetry principles: The concept of superconducting fitness,” Phys. Rev. B 98, 024501 (2018).
  * Ramires (2021) Aline Ramires, “Nodal gaps from local interactions in Sr2RuO4,” arXiv e-prints , arXiv:2110.10621 (2021), arXiv:2110.10621 [cond-mat.supr-con] .
  * (35) See Supplemental Material .
  * Ramires and Sigrist (2019) Aline Ramires and Manfred Sigrist, “Superconducting order parameter of Sr2RuO4: A microscopic perspective,” Phys. Rev. B 100, 104501 (2019).
  * Vogt and Buttrey (1995) T. Vogt and D. J. Buttrey, “Low-temperature structural behavior of Sr2RuO4,” Phys. Rev. B 52, R9843–R9846 (1995).
  * Barber  _et al._ (2019) Mark E. Barber, Frank Lechermann, Sergey V. Streltsov, Sergey L. Skornyakov, Sayak Ghosh, B. J. Ramshaw, Naoki Kikugawa, Dmitry A. Sokolov, Andrew P. Mackenzie, Clifford W. Hicks, and I. I. Mazin, “Role of correlations in determining the van hove strain in Sr2RuO4,” Phys. Rev. B 100, 245139 (2019).
  * (39) Peter Blaha, Karlheinz Schwarz, Georg K HMadsen, Dieter Kvasnicka, Joachim Luitz, Robert Laskowsk, Fabien Tran, Laurence Marks, and Laurence Marks,  _WIEN2k: An Augmented Plane Wave Plus Local Orbitals Program for Calculating Crystal Properties_.
  * Perdew  _et al._ (1996) John P. Perdew, Kieron Burke, and Matthias Ernzerhof, “Generalized gradient approximation made simple,” Phys. Rev. Lett. 77, 3865–3868 (1996).
  * Marzari and Vanderbilt (1997) Nicola Marzari and David Vanderbilt, “Maximally localized generalized wannier functions for composite energy bands,” Phys. Rev. B 56, 12847–12865 (1997).
  * Souza  _et al._ (2001) Ivo Souza, Nicola Marzari, and David Vanderbilt, “Maximally localized wannier functions for entangled energy bands,” Phys. Rev. B 65, 035109 (2001).
  * Kuneš  _et al._ (2010) Jan Kuneš, Ryotaro Arita, Philipp Wissgott, Alessandro Toschi, Hiroaki Ikeda, and Karsten Held, “Wien2wannier: From linearized augmented plane waves to maximally localized wannier functions,” Computer Physics Communications 181, 1888 – 1895 (2010).
  * Pizzi  _et al._ (2020) Giovanni Pizzi, Valerio Vitale, Ryotaro Arita, Stefan Blügel, Frank Freimuth, Guillaume Géranton, Marco Gibertini, Dominik Gresch, Charles Johnson, Takashi Koretsune, Julen Ibañez-Azpiroz, Hyungjun Lee, Jae-Mo Lihm, Daniel Marchand, Antimo Marrazzo, Yuriy Mokrousov, Jamal I Mustafa, Yoshiro Nohara, Yusuke Nomura, Lorenzo Paulatto, Samuel Poncé, Thomas Ponweiser, Junfeng Qiao, Florian Thöle, Stepan S Tsirkin, Małgorzata Wierzbowska, Nicola Marzari, David Vanderbilt, Ivo Souza, Arash A Mostofi, and Jonathan R Yates, “Wannier90 as a community code: new features and applications,” Journal of Physics: Condensed Matter 32, 165902 (2020).
  * Parcollet  _et al._ (2015) Olivier Parcollet, Michel Ferrero, Thomas Ayral, Hartmut Hafermann, Igor Krivenko, Laura Messio, and Priyanka Seth, “Triqs: A toolbox for research on interacting quantum systems,” Computer Physics Communications 196, 398 – 415 (2015).
  * Giannozzi  _et al._ (2009) Paolo Giannozzi, Stefano Baroni, Nicola Bonini, Matteo Calandra, Roberto Car, Carlo Cavazzoni, Davide Ceresoli, Guido L Chiarotti, Matteo Cococcioni, Ismaila Dabo, Andrea Dal Corso, Stefano de Gironcoli, Stefano Fabris, Guido Fratesi, Ralph Gebauer, Uwe Gerstmann, Christos Gougoussis, Anton Kokalj, Michele Lazzeri, Layla Martin-Samos, Nicola Marzari, Francesco Mauri, Riccardo Mazzarello, Stefano Paolini, Alfredo Pasquarello, Lorenzo Paulatto, Carlo Sbraccia, Sandro Scandolo, Gabriele Sclauzero, Ari P Seitsonen, Alexander Smogunov, Paolo Umari, and Renata M Wentzcovitch, “QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials,” Journal of Physics: Condensed Matter 21, 395502 (2009).
  * Garrity  _et al._ (2014) Kevin F. Garrity, Joseph W. Bennett, Karin M. Rabe, and David Vanderbilt, “Pseudopotentials for high-throughput dft calculations,” Computational Materials Science 81, 446 – 452 (2014).
  * Min  _et al._ (2006) Hongki Min, J. E. Hill, N. A. Sinitsyn, B. R. Sahu, Leonard Kleinman, and A. H. MacDonald, “Intrinsic and rashba spin-orbit interactions in graphene sheets,” Phys. Rev. B 74, 165310 (2006).
  * Sigrist and Ueda (1991) Manfred Sigrist and Kazuo Ueda, “Phenomenological theory of unconventional superconductivity,” Rev. Mod. Phys. 63, 239–311 (1991).
  * Tamai  _et al._ (2019) A. Tamai, M. Zingl, E. Rozbicki, E. Cappelli, S. Riccò, A. de la Torre, S. McKeown Walker, F. Y. Bruno, P. D. C. King, W. Meevasana, M. Shi, M. Radović, N. C. Plumb, A. S. Gibbs, A. P. Mackenzie, C. Berthod, H. U. R. Strand, M. Kim, A. Georges, and F. Baumberger, “High-resolution photoemission on Sr2RuO4 reveals correlation-enhanced effective spin-orbit coupling and dominantly local self-energies,” Phys. Rev. X 9, 021048 (2019).
  * (51) H.-H. Klauss, private communication .

See pages ,,1,,2,,3,,4,,5,,6,,7,,8,,9,,10,,11,,12,,13,,14,,15,,16,,17,,18,,19
of SROStrain.SM.V12.pdf


11institutetext: E&E Engineering, Stellenbosch University, Stellenbosch, South
Africa
11email<EMAIL_ADDRESS>
# StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource
Contexts††thanks: This work is supported in part by the National Research
Foundation of South Africa (grant number: 120409) and a Google Faculty Award
for HK.

Matthew Baas  0000-0003-3001-6292 Herman Kamper  0000-0003-2980-3475

###### Abstract

Voice conversion is the task of converting a spoken utterance from a source
speaker so that it appears to be said by a different target speaker while
retaining the linguistic content of the utterance. Recent advances have led to
major improvements in the quality of voice conversion systems. However, to be
useful in a wider range of contexts, voice conversion systems would need to be
(i) trainable without access to parallel data, (ii) work in a zero-shot
setting where both the source and target speakers are unseen during training,
and (iii) run in real time or faster. Recent techniques fulfil one or two of
these requirements, but not all three. This paper extends recent voice
conversion models based on generative adversarial networks (GANs), to satisfy
all three of these conditions. We specifically extend the recent StarGAN-VC
model by conditioning it on a speaker embedding (from a potentially unseen
speaker). This allows the model to be used in a zero-shot setting, and we
therefore call it StarGAN-ZSVC. We compare StarGAN-ZSVC against other voice
conversion techniques in a low-resource setting using a small 9-minute
training set. Compared to AutoVC—another recent neural zero-shot approach—we
observe that StarGAN-ZSVC gives small improvements in the zero-shot setting,
showing that real-time zero-shot voice conversion is possible even for a model
trained on very little data. Further work is required to see whether scaling
up StarGAN-ZSVC will also improve zero-shot voice conversion quality in high-
resource contexts.

###### Keywords:

speech processing voice conversion generative adversarial networks zero-shot.

##  1 Introduction

Voice conversion is a speech processing task where speech from a source
speaker is transformed so that it appears to come from a different target
speaker while preserving linguistic content. A fast, human-level voice
conversion system has significant applications across several industries, from
those in privacy and identity protection [16] to those of voice mimicry and
disguise [37, 10]. It can also be essential for addressing downstream speech
processing problems in low-resource contexts where training data is limited:
it could be used to augment training data by converting the available
utterances to novel speakers—effectively increasing the diversity of training
data and improving the quality of the resulting systems.

Recent techniques have improved the quality of voice conversion significantly,
in part due to the Voice Conversion Challenge (VCC) and its efforts to
concentrate disparate research efforts [36]. Some techniques are beginning to
achieve near human-level quality in conversion outputs. However, much of the
advances and improvements in quality are limited in their practical usefulness
because they fail to satisfy several requirements that would be necessary for
practical use, particularly in low-resource settings.

First, a practical voice conversion system should be trainable on non-parallel
data. That is, training data should not need to contain utterances from
multiple speakers saying the same words – such a setting is known as a
parallel data setting. Non-parallel data is the converse, where the different
utterances used to train the model do not contain the same spoken words.
Parallel data is difficult to collect in general, and even more so for low-
resource language (those which have limited digitally stored corpora). Second,
a practical system should be able to convert speech to and from speakers which
have not been seen during training. This is called zero-shot voice conversion.
Without this requirement, a system would need to be retrained whenever speech
from a new speaker is desired. Finally, for a number of practical
applications, a voice conversion system needs to run at least in real-time.
For data augmentation in particular, having the system run as fast as possible
is essential for it to be practical in the training of a downstream speech
model.

With these requirements in mind, we look to extend existing state-of-the-art
voice conversion techniques. We specifically extend the recent StarGAN-VC2
[13] approach to the zero-shot setting, proposing the new StarGAN-ZSVC model.
StarGAN-ZSVC achieves zero-shot prediction by using a speaker encoding network
to generate speaker embeddings for potentially unseen speakers; these
embeddings are then used to condition the model at inference time.

Through objective and human evaluations, we show that StarGAN-ZSVC performs
better than simple baseline models and similar or better than the recent
AutoVC zero-shot voice conversion approach [24] across a range of evaluation
metrics. More specifically, it gives similar or better performance in all
zero-shot settings considered, and does so more than five times faster than
AutoVC.

##  2 Related Work

A typical voice conversion system operates in the frequency domain, first
converting an input utterance into a spectrogram and then using some model to
map the spectrogram spoken by a source speaker to that of one spoken by a
target speaker. The output spectrogram is then converted to a waveform in the
time-domain using a vocoder [28]. In this paper, we denote spectrogram
sequences as $X=[\mathbf{x}_{1},\mathbf{x}_{2},...,\mathbf{x}_{T}]$, where the
spectrogram contains $T$ frames, and each frame $\mathbf{x}_{i}$ consists of
some number of frequency bins. In our case, we use 80-dimensional Mel-scale
frequency bins, i.e. $\mathbf{x}_{i}\in\mathbb{R}^{80}$.

Some models [32, 11] use parametric algorithms like the WORLD vocoder [20] to
convert the output spectrogram back to a time-domain waveform. Others use
neural vocoders, which can be divided into autoregressive models, such as
those of the WaveNet family [21], and non-autoregressive models, such as
MelGAN [15].

Voice conversion models themselves can be classified on several levels. Older
techniques rely on rule-based techniques [27, 6] while newer models rely on
statistical techniques and often make extensive use of deep neural networks
[28]. Models can also be classified into traditional models that can only
perform one-to-one voice conversion, such as the recurrent DBLSTM-RNN [31] and
Gaussian mixture based models [33, 30], to newer models like those using
variational autoencoders [24, 12] and vector quantized neural networks [3] to
allow for many-to-many conversion where a single model can convert between
several possible source-target speaker pairings.

Finally, the recent AutoVC model [24] (Section 2.2) emerged as the first model
to be able to perform zero-shot voice conversion where either the source or
target speaker is unseen during training. For our new model, we also take
inspiration from recent work on speaker encoding networks trained for speaker
verification [35], as well as the StarGAN-VC and -VC2 [11, 13] models (Section
2.1). Concretely, we attempt to combine these in a new zero-shot voice
conversion model.

###  2.1 StarGAN and Voice Conversion

Generative adversarial networks (GANs) train two separate networks: a
generator and a discriminator. The generator is trained to produce realistic
outputs (i.e. it should aim to accurately approximate some function), while
the discriminator is trained to discern true outputs from ones produced by the
generator. Part of the generator’s objective is to fool the discriminator and
to essentially maximize its loss metric, while the discriminator is trained to
do the opposite.

One set of successful voice conversion techniques relies on re-purposing the
StarGAN image-to-image translation technique [2] for voice conversion. In
particular, StarGAN-VC2 [13] extends upon StarGAN-VC [11] by training a single
generator model to perform many-to-many voice conversion using speaker-
dependent modulation factors in so-called conditional instance normalization
layers [5]. The model’s generator
$G(X_{\text{src}},\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}})$ converts a
spectrogram $X_{\text{src}}$ from a source speaker to a target speaker,
producing the converted output $X_{\text{src}\rightarrow\text{trg}}$. The
source and target speaker identities are given as one-hot vectors,
$\mathbf{s}_{\text{src}}$ and $\mathbf{s}_{\text{trg}}$, respectively. The
model’s discriminator $D(X,\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}})$
takes an input spectrogram $X$ and returns a scalar. Intuitively, the
generator is trained to force the discriminator’s output when given converted
spectrograms to be high, while the discriminator is trained to make its output
low when given converted outputs and high when given original spectrograms.

More formally, the generator $G$ is trained to minimize the loss
$\mathcal{L}=\lambda_{\text{id}}\mathcal{L}_{\text{id}}+\lambda_{\text{cyc}}\mathcal{L}_{\text{cyc}}+\mathcal{L}_{G-\text{adv}}$.
The first term, $\mathcal{L}_{\text{id}}$ is an identity loss term. It aims to
minimize the difference between the input and output spectrogram when the
model is made to keep the same speaker identity, i.e. convert from speaker A
to speaker A. It is defined by the $L_{2}$ loss:

$\mathcal{L}_{\text{id}}=||G(X_{\text{src}},\mathbf{s}_{\text{src}},\mathbf{s}_{\text{src}})-X_{\text{src}}||_{2}$
(1)

Next, many-to-many voice conversion systems like StarGAN-VC2 can perform
cyclic mappings, whereby the model is made to convert an input utterance from
a source speaker to a target speaker, and then convert the output utterance
back to the source speaker. The second term of the loss aims to minimize the
cyclic reconstruction error between the cyclic mapping and original
spectrogram [2]:

$\mathcal{L}_{\text{cyc}}=||X_{\text{src}}-G(G(X_{\text{src}},\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}}),\mathbf{s}_{\text{trg}},\mathbf{s}_{\text{src}})||_{1}$
(2)

Finally, the adversarial loss term $\mathcal{L}_{G-\text{adv}}$ is added based
on the LSGAN [18] loss. It defines two constants $a$ and $b$, whereby $G$’s
loss tries to push $D$’s output for converted utterances closer to $a$, while
$D$’s loss function tries to push $D$’s output for converted utterances closer
to $b$ and its output for real outputs closer to $a$. Concretely, $G$’s
adversarial loss is defined as

$\mathcal{L}_{G-\text{adv}}=\left(D(G(X_{\text{src}},\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}}),\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}})-a\right)^{2}.$
(3)

while, the discriminator $D$ is trained to minimize the corresponding LSGAN
discriminator loss:

$\mathcal{L}_{D-\text{adv}}=\left(D(G(X_{\text{src}},\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}}),\mathbf{s}_{\text{src}},\mathbf{s}_{\text{trg}})-b\right)^{2}+\left(D(X_{\text{src}},\mathbf{s}_{\text{trg}},\mathbf{s}_{\text{src}})-a\right)^{2}$
(4)

In [13], the authors set the scalar coefficients to be
$\lambda_{\text{id}}=5$, and $\lambda_{\text{cyc}}=10$. The original study
[13] does not mention how $a$ and $b$ are set (despite these greatly affecting
training); we treat them as hyperparameters. Note that the true target
spectrogram $X_{\text{trg}}$ does not appear in any of the equations – this is
what allows StarGAN-VC2 to be trained with non-parallel data where the source
utterance $X_{\text{src}}$ has no corresponding utterance from the target
speaker.

StarGAN-VC2 uses a specially designed 2-1-2D convolutional architecture for
the generator, as well as a projection discriminator [19] which comprises of a
convolutional network (to extract features) followed by an inner product with
an embedding corresponding to the source/target speaker pair. For the
generator, a new form of modulation-based conditional instance normalization
was introduced in [13]. This allows the speaker identity (which is provided as
a one-hot vector) to multiplicatively condition the channels of an input
feature. According to [13], this special layer is a key component in achieving
high performance in StarGAN-VC2.

We use these building blocks for our new zero-shot approach. Concretely, the
one-hot speaker vectors in StarGAN-VC2 are replaced with continuous embedding
vectors obtained from a separate speaker encoding network (which can be
applied to arbitrary speakers), as outlined in Section 3.

###  2.2 AutoVC

Zero-shot voice conversion was first introduced in 2019 with the AutoVC model
[24], which remains one of only a handful of models that can perform zero-shot
conversion (see e.g. [25] for a very recent other example). For AutoVC, zero-
shot conversion is achieved by using an autoencoder with a specially designed
bottleneck layer which forces the network’s encoder to only retain linguistic
content in its encoded latent representation. The model then uses a separate
recurrent speaker encoder model $E(X)$, originally proposed for speaker
identification [35], to extract a speaker embedding $\mathbf{s}$ from an input
spectrogram. These speaker embeddings are then used to supply the missing
speaker identity information to the decoder which, together with the
linguistic content from the encoder, allows the decoder to synthesize an
output spectrogram for an unseen speaker.

Formally, the full encoder-decoder model is trained to primarily minimize two
terms. The first term is an $L_{2}$ reconstruction loss between the decoder
output spectrogram $X_{\text{src}\rightarrow\text{src}}$ and input spectrogram
$X_{\text{src}}$, with the source speaker’s encoding (from the speaker
encoder) provided to both the encoder and decoder. The second term is an
$L_{1}$ loss between the speaker embedding of the decoder output
$E(X_{\text{src}\rightarrow\text{src}})$ and the original speaker embedding
$\mathbf{s}_{\text{src}}=E(X_{\text{src}})$. The encoder and decoder consists
of convolutional and Long Short-Term Memory (LSTM) [8] recurrent layers which
are carefully designed to ensure that no speaker identity information is
present in the encoder output.

As with StarGAN-VC and StarGAN-VC2, a corresponding parallel target utterance
$X_{\text{trg}}$ does not appear in any of the loss terms, allowing AutoVC to
be trained without parallel data. Zero-shot inference is performed by using
the speaker encoder to obtain embeddings for new utterances from unseen
speakers, which is then provided to the decoder instead of the embedding
corresponding to the source speaker, causing the decoder to return a converted
output. We use this same idea of using an encoding network to obtain
embeddings for unseen speakers in our new GAN-based approach, which we
describe next.

##  3 StarGAN-ZSVC

While StarGAN-VC and StarGAN-VC2 allows training with non-parallel data and
runs sufficiently fast, it is limited in its ability to only perform voice
conversion for speakers seen during training: while parallel $X_{\text{src}}$
and $X_{\text{trg}}$ utterance pairs are not required, the model can only
synthesize speech for target speaker identities (specified as one-hot vectors)
seen during training. This could preclude the use of these models in many
practical situations where zero-shot conversion is required between unseen
speakers. Conversely, AutoVC allows for such zero-shot prediction and is
trained on non-parallel data, but it is implemented with a slow vocoder and
its performance suffers when trained on very little data. Combining the
strengths of both of these methods, we propose the StarGAN zero-shot voice
conversion model – StarGAN-ZSVC.

###  3.1 Overcoming the Zero-shot Barrier

To achieve voice conversion between multiple speakers, the original StarGAN-
VC2 model creates an explicit embedding vector for each source-target speaker
pairing, which is incorporated as part of the generator $G$ and discriminator
$D$. This requires that each source-target speaker pairing is seen during
training so that the corresponding embedding exists and has been trained –
prohibiting zero-shot voice conversion. To overcome this hurdle, we instead
infer separate source and target speaker embeddings, $\mathbf{s}_{\text{src}}$
and $\mathbf{s}_{\text{trg}}$, using a speaker encoder network $E$ – similar
to the approach followed in AutoVC (Section 2.2).

This framework is shown in Figure 1. Utterances from unseen speakers (i.e.
$X_{\text{src}}$ and $Y_{\text{trg}}$) are fed to the speaker encoder $E$,
yielding embeddings for these new speakers, which are then used to condition
$G$ and $D$, thereby enabling zero-shot conversion. The generator uses these
embeddings to produce a converted Mel-spectrogram
$X_{\text{src}\rightarrow\text{trg}}$ from a given source utterance’s Mel-
spectrogram $X_{\text{src}}$.

Figure 1: The StarGAN-ZSVC system framework. The speaker encoder network $E$
and the WaveGlow vocoder are pretrained on large speech corpora, while the
generator $G$ and discriminator $D$ are trained on a 9-minute subset of the
VCC dataset. During inference, arbitrary utterances for the source and target
speaker are used to obtain source and target speaker embeddings,
$\mathbf{s}_{\textrm{src}}$ and $\mathbf{s}_{\textrm{trg}}$.

$E$ is trained on a large corpus using a GE2E loss [35] which aims to
simultaneously maximize distances between embeddings from different speakers
while minimizing the distances between embeddings from utterances of the same
speaker. NVIDIA’s WaveGlow [23] is used, which does not require any speaker
information for vocoding and thus readily allows zero-shot conversion.

###  3.2 Overcoming the Speed Barrier

The speed of the full voice conversion system during inference is bounded by
(a) the speed of the generator $G$; (b) the speed of converting the utterance
between time and frequency domains, consisting of the initial conversion from
time-domain waveform to Mel-spectrogram and the speed of the vocoder; and (c)
the speed of the speaker encoder $E$. To ensure that the speed of the full
system is at least real-time, each subsystem needs to be faster than real-
time.

####  3.2.1 (a) Generator Speed.

For the generator $G$ to be sufficiently fast, we design it to only include
convolution, linear, normalization, and upscaling layers as opposed to a
recurrent architecture like those used in AutoVC [24]. By ensuring that the
majority of layers are convolutions, we obtain better-than real-time speeds
for the generator.

####  3.2.2 (b) Vocoder and Mel-spectrogram Speed.

The choice of vocoder greatly affects computational cost. Higher-quality
methods, such as those from the WaveNet family [21], are typically much slower
than real-time, while purely convolutional methods such as MelGAN [15] are
much faster but has poorer quality. Often the main difference between the
slower and faster methods is again the presence of traditional recurrent
layers in the vocoder architecture.

We opt for a reasonable middle-ground choice with the WaveGlow vocoder, which
does have recurrent connections but does not use any recurrent layers with
dense multiplications such as LSTM or Gated Recurrent Unit (GRU, another kind
of recurrent cell architecture [1]) layers. We specifically use a pretrained
WaveGlow network, as provided with the original paper [23]. Furthermore, the
speed of the Mel-spectrogram transformation for the input audio is well faster
than real-time due to the efficient nature of the fast Fourier transform and
the multiplication by Mel-basis filters.

####  3.2.3 (c) Speaker Encoder Speed.

The majority of research efforts into obtaining speaker embeddings involve
models using slower recurrent layers, often making these encoder networks the
bottleneck. We also make use of a recurrent stacked-GRU network as our speaker
embedding network $E$. However, we only need to obtain a single speaker
embedding to perform any number of conversions involving that speaker. We
therefore treat this as a preprocessing step where we apply $E$ to a few
arbitrary utterances from the target and source speakers, averaging the
results to obtain target and source speaker embeddings, and use those same
embeddings for all subsequent conversions.

We also design the speaker embeddings to be 256-dimensional vectors of unit
length. If we were to use StarGAN-ZSVC downstream for data augmentation (where
we want speech from novel speakers), we could then simply sample random unit-
length vectors of this dimensionality to use with the generator.

Figure 2: StarGAN-ZSVC’s network architectures. The speaker encoder $E$ is a
recurrent network similar to that used in the original GE2E paper, while the
generator $G$ and discriminator $D$ are modified versions from the original
StarGAN-VC2 architecture. Within layers, k and s represent kernel size and
stride (for convolutions), f is the scaling factor (for pixel shuffle), and h
and c are the height and channels of the output (for reshape layers). A number
alongside a layer indicates the number of output channels (for convolutions),
or output units (for linear and GRU layers). GLU layers split the input tensor
in half along the channels dimension. GSP, GLU, and SELU indicate global sum
pooling, gated linear units, and scaled exponential linear units,
respectively.

###  3.3 Architecture

With the previous considerations in mind, we design the generator $G$,
discriminator $D$, and encoder network $E$, as shown in Figure 2. The
generator and discriminator are adapted from StarGAN-VC2 [13], while the
speaker encoder is adapted from the original model proposed for speaker
identification [35]. Specifically, for $E$ we use a simple stacked GRU model,
while for $D$ we use a projection discriminator [19]. For $G$, we use the
2-1-2D generator from StarGAN-VC2 with a modified central set of layers,
denoted by the Conditional Block in the figure.

These conditional blocks are intended to provide the network with a way to
modulate the channels of an input spectrogram, with modulation factors
conditioned on the specific source and target speaker pairing. They utilize a
convolutional layer followed by a modified conditional instance normalization
layer [5] and a gated linear unit [4].

The modified conditional instance normalization layer performs the following
operation on an input feature vector $\mathbf{f}$:

$\text{CIN}(\mathbf{f},\gamma,\beta)=\gamma\left(\frac{\mathbf{f}-\mu(\mathbf{f})}{\sigma(\mathbf{f})}\right)+\beta$
(5)

where $\mu(\mathbf{f})$ and $\sigma(\mathbf{f})$ are respectively the scalar
mean and standard deviation of vector $\mathbf{f}$, while $\gamma$ and $\beta$
are computed using two linear layers which derive their inputs from the
speaker embeddings, as depicted in Figure 2. The above is computed separately
for each channel when the input feature contains multiple channels.

For the discriminator, the source and target speaker embeddings are also fed
through several linear layers and activation functions to multiply with the
pooled output of $D$’s main branch.

##  4 Experimental Setup

We compare StarGAN-ZSVC to other voice conversion models using the voice
conversion challenge (VCC) 2018 dataset [17], which contains parallel
recordings of native English speakers from the United States. Importantly, we
do not train StarGAN-ZSVC or the AutoVC model (to which we compare) using
parallel input-output examples. However, the traditional baseline models
(below) do require parallel data. All training and speed measurements are
performed on a single NVIDIA RTX 2070 SUPER GPU.

###  4.1 Dataset

The VCC 2018 dataset was recorded from 8 speakers, each speaking 81 utterances
from the same transcript. 4 speakers are used for training and 4 for testing.
To emulate a low-resource setting, we use a 9-minute subset of the VCC 2018
training dataset for StarGAN-ZSVC and AutoVC. This corresponds to 90% of the
utterances from two female (F) and two male (M) speakers (VCC2SF1, VCC2SF2,
VCC2SM1, and VCC2SM2). This setup is in line with existing evaluations on VCC
2018 [13], allows for all combinations of inter- and intra-gender conversions,
and allows for zero-shot evaluation on the 4 remaining unseen speakers.

In contrast to StarGAN-ZSVC and AutoVC, some of the baseline models only allow
for one-to-one conversions, i.e. they are trained on parallel data and can
only convert from seen speaker A to seen speaker B. We therefore train the
baseline models on a single source-target speaker mapping (from VCC2SF1 to
VCC2SM2), using 90% of the parallel training utterances for this speaker pair.

All utterances are resampled to 22.05 kHz and then converted to log Mel-
spectrograms with a window and hop length of 1024 and 256 samples,
respectively. During training, for each batch we randomly sample a $k$-frame
sequence from each spectrogram, where $k$ is randomly sampled from multiples
of 32 between 96 to 320 (inclusive). This is done for all models to make it
robust to utterance length, with the exception of StarGAN-ZSVC, which requires
fixed-size input for its discriminator. This leads to slightly worse
performance for StarGAN-ZSVC on long or silence-padded sequences. For a fair
comparison, we therefore only consider non-silent frames of the target
utterance.

###  4.2 Speaker Encoder and Vocoder Setup

The same WaveGlow vocoder is used to produce output waveforms for all networks
to ensure a fair comparison. The WaveGlow model is pretrained on a large
external corpus, as provided by the original paper [23]. However, since all
models use log Mel-spectrogram inputs and produce log Mel-spectrogram outputs,
we rather perform all quantitative comparisons on the spectrograms of each
utterance (instead of waveforms), in order to minimize the vocoder’s
confounding effect.

Our WaveGlow implementation takes approximately 240 ms to produce one second
of vocoded audio (taking a spectrogram as input). For the full voice
conversion system to be faster than real-time, this means that the combined
inference speed of the remaining sub-networks needs to be well under 700ms/s,
or preferably significantly faster if used for data augmentation.

The speaker encoder is trained on 90% of the utterances from a combined set
consisting of the VCTK [34], VCC 2018 [17], LibriSpeech [22], and the English
CommonVoice 2020-06 datasets.111Available under a CC-0 license at
https://commonvoice.mozilla.org/en/datasets. It is trained with the Adam
optimizer [14] for 8 epochs with 8 speakers per batch, and 6 utterances per
speaker in each batch. We start with a learning rate of $4\times 10^{-4}$ and
adjust it down to $3\times 10^{-7}$ in the final epoch. Embeddings for
speakers are precomputed by taking the average embedding over 4 arbitrary
utterances for each speaker.

###  4.3 Baseline Models

We train 4 baseline models for comparison, all using the Adam optimizer. The
first three are traditional one-to-one conversion models, consisting of a
simple linear model, a DBLSTM model [31], and a UNet model [26]. The final
model that we compare to is the AutoVC model, which is able to do zero-shot
many-to-many conversion (Section 2.2). We compare AutoVC to StarGAN-ZSVC on
all variants of seen/unseen source/target pairings. Each network is trained
according to the method developed by Smith [29] by evaluating the decrease in
loss every few hundred epochs for different learning rates, and updating the
learning rate to correspond to the largest decrease in loss. This process is
repeated until the validation loss begins to increase, after which training is
terminated.

All one-to-one models are trained in the same way, taking the source
spectrogram $X_{\text{src}}$ as input and trained with an $L_{1}$ loss (which
we found to produce better results than the $L_{2}$ loss) to predict the
ground-truth target spectrogram $X_{\text{trg}}$. The linear model consists of
4 convolutional layers with output channels and kernel sizes of (200, 5),
(200, 5), (100, 3), and (1, 3), respectively. The DBLSTM model is based on the
original paper [31], but we do not use any time-alignment techniques (such as
dynamic time warping). The model consists of 4 stacked bidirectional LSTM
layers with a hidden size of 256 and a dropout of 0.3, followed by a final
linear projection layer to bring the output dimension back to 80. The network
is trained with a batch size of 8. The UNet model is built based on the
structure of XResNet [7] using the method defined in the Fast.ai library [9].

Finally, AutoVC is trained using the same loss function as in the original
paper [24]. It is trained with a batch size of 4 for 4700 epochs. The speaker
encoder used is the pretrained encoder $E$ described above.

###  4.4 StarGAN-ZSVC Training

We train StarGAN-ZSVC using the same Adam optimizer and learning rate
scheduling technique of Smith [29]. Furthermore, we employ several tricks for
successfully training GANs: (i) gradients in $G$ and $D$ are clipped to have a
maximum norm of 1; (ii) the discriminator’s learning rate is made to always be
half of the generators learning rate; (iii) the number of iterations training
the discriminator versus generator is updated every several hundred epochs to
ensure that the discriminator’s loss is always roughly a factor of 10 lower
than the adversarial term of the generator’s loss; and (iv) dropout with a
probability of 0.3 is added to the input of $D$ after the first 3000 epochs
(if added earlier it causes artifacts and destabilizes training).

The loss function used is the same as that of StarGAN-VC2, with the exception
that the term $\mathcal{L}_{\text{cyc}}$ (see Section 2.1) is squared in our
model, which we found to give superior results. We set $a=1$, and $b=0$ for
the LSGAN constants, and $\lambda_{\text{cyc}}=10$, $\lambda_{\text{id}}=5$
for loss coefficients, being adjusted downwards during training in the same
manner as in [13].

###  4.5 Evaluation

We compare converted output spectrograms to their ground-truth target
spectrograms on the test dataset using several objective metrics. To account
for different speaking rates, we first use dynamic time warping (DTW) to align
the converted spectrogram to the target spectrogram, and then perform
comparisons over non-silent regions of the target utterance. Non-silent
regions are defined as those 80-dimensional spectrogram frames where the mean
vector element value is greater than -10dB.

In addition to computing the mean absolute error (MAE) and mean square error
(MSE) between spectrograms, we also compute a cosine similarity by finding the
cosine distance between each 80-dimensional source/target frame pair of the
Mel-spectrograms and then computing the mean cosine distance over all non-
silent frames (after DTW alignment). This metric, denoted as $\cos(\theta)$,
gives an additional measure of conversion quality.

Finally, to quantitatively measure speaker similarity (i.e. determining
whether the generated spectrogram sounds like the target speaker), we define a
new metric using the speaker encoder. We compute speaker embeddings for the
target and output converted spectrograms using the speaker encoder $E$. The
norm of the difference between these vectors,
$||E(X_{\text{src}\rightarrow\text{trg}})-E(X_{\text{trg}})||=e_{\text{norm}}$,
is then used as a measure of speaker similarity, with a smaller norm
corresponding to greater similarity between the converted and target
spectrogram.

We also perform a subjective listening test with 12 proficient English
speakers to assess how well StarGAN-ZSVC compares to AutoVC across various
zero-shot settings. Each participant rated the naturalness of 144 utterances
from 1 (bad) to 5 (excellent) where the utterance order and naming is
randomized. The 144 utterances consist of 8 converted utterances for each
seen/unseen source/target speaker pairing, for both AutoVC and StarGAN-ZSVC. A
further 14 utterances are included to find a baseline rating for raw and
vocoded audio. The ratings for each subset are averaged to find a mean opinion
score (MOS), which serves as a measure of conversion quality.

##  5 Experiments

We perform two sets of experiments. First we perform an evaluation on seen
speakers, where we compare StarGAN-ZSVC to all other models to obtain an
indication of both speed and performance. We then compare StarGAN-ZSVC with
AutoVC for zero-shot voice conversion, looking at both the output and cyclic
reconstruction error. We encourage the reader to listen to the demo
samples222https://rf5.github.io/sacair2020/ for the zero-shot models.

###  5.1 Seen-to-seen Conversion

In the first set of comparisons, we evaluate performance for test utterances
where other utterances from both the source and target speaker have been seen
during training. I.e., while the models have not been trained on these exact
test utterances, they have seen the speakers during training. There is,
however, a problem in directly comparing the one-to-one models (traditional
baselines) to the many-to-many models (AutoVC and StarGAN-ZSVC). The one-to-
one models are trained on parallel data, always taking in utterances from one
speaker as input (VCC2SF1 in our case) and always producing output from a
different target speaker (VCCSM2).

Table 1:  Objective evaluation results when converting between speakers where both the source and target speaker are seen during training. For all metrics aside from cosine similarity, lower is better. Speed is measured as the time (in milliseconds) required to convert one second of input audio. The first StarGAN-ZSVC and AutoVC entries correspond to evaluations on the one-to-one test utterances, while the final two starred entries correspond to metrics computed when using test utterances from all seen training speakers for the many-to-many models. Model | MAE | MSE | $\cos(\theta)$ | $e_{\text{norm}}$ | Speed (ms/s)
---|---|---|---|---|---
Linear | 1.277 | 2.689 | 0.980 | 0.860 | 0.15
DBLSTM | 1.329 | 3.102 | 0.982 | 0.496 | 12.52
UNet | 1.370 | 3.347 | 0.980 | 0.545 | 100.5
AutoVC | 0.993 | 1.756 | 0.987 | 0.259 | 10.99
StarGAN-ZSVC | 1.092 | 2.101 | 0.977 | 0.513 | 1.88
AutoVC* | 1.000 | 1.783 | 0.987 | 0.321 | 10.99
StarGAN-ZSVC* | 1.008 | 1.863 | 0.983 | 0.321 | 1.88

In contrast, the many-to-many models are trained without access to parallel
data, taking in input utterances from several speakers (4 speakers, including
VCC2SF1 and VCCSM2 in our case, as explained in Section 4.1). This means that
the one-to-one and many-to-many models observe very different amounts of data.
Moreover, while the data for both the one-to-one and many-to-many models are
divided into a 90%-10% train-test split, the same exact splits aren’t used in
both setups; this is because the former requires parallel utterances, and the
split is therefore across utterance pairs and not just individual utterances.
To address this, we evaluate the many-to-many models in two settings: on the
exact same test utterances as those from the test split of the one-to-one
models, as well as on all possible source/target speaker utterance pairs where
the source utterance is in the test utterances for the 4 seen training
speakers. In the former case, it could happen that the many-to-many model
actually observes one of the test utterances during training. Nevertheless,
reporting scores for both settings allows for a meaningful comparison.

The results of this evaluation on seen speakers are given in Table 1. The
results indicate that AutoVC appears to be the best in this evaluation on seen
speakers. However, this comes at a computational cost: the linear and StarGAN-
ZSVC models are a factor of 5 or more faster than the models relying on
recurrent layers like DBLSTM and AutoVC.

Table 2: Objective evaluation results for zero-shot voice conversion for AutoVC and StarGAN-ZSVC. The prediction metrics compare the predicted output to the ground truth target, while the reconstruction metrics compare the cyclic reconstruction $X_{\text{src}\rightarrow\text{trg}\rightarrow\text{src}}$ with the original source spectrogram. $e_{\text{norm}}$ indicates the vector norm of the speaker embeddings for the compared spectrograms, with lower values indicating closer speaker identities. |  | Prediction | Reconstruction
---|---|---|---
Setting | Model | MAE | $\cos(\theta)$ | $e_{\text{norm}}$ | MAE | $\cos(\theta)$ | $e_{\text{norm}}$
Seen-to-unseen | AutoVC | 1.246 | 0.982 | 0.742 | 1.178 | 0.976 | 0.392
StarGAN-ZSVC | 1.030 | 0.982 | 0.705 | 0.197 | 0.997 | 0.124
Unseen-to-seen | AutoVC | 1.014 | 0.986 | 0.328 | 1.201 | 0.975 | 0.753
StarGAN-ZSVC | 0.974 | 0.985 | 0.380 | 0.921 | 0.986 | 0.760
Unseen-to-unseen | AutoVC | 1.238 | 0.981 | 0.746 | 1.340 | 0.968 | 0.827
StarGAN-ZSVC | 1.079 | 0.982 | 0.742 | 0.921 | 0.986 | 0.760

###  5.2 Zero-shot Conversion

Next we compare StarGAN-ZSVC and AutoVC in zero-shot settings, where either
the source, target, or both source and target speaker are unseen during
training. Many-to-many models can also be used in a cyclic manner; we use such
cyclic reconstruction as another objective evaluation metric, where we compare
how well the original spectrogram is reconstructed when performing this
cyclical mapping of converting from the source speaker to the target speaker
and back again. Results for zero-shot conversion are shown in Table 2.

The performance for AutoVC and StarGAN-ZSVC are similar on most metrics for
the unseen-to-seen case. But for the seen-to-unseen case and the unseen-to-
unseen case (where both the target and source speakers are new) StarGAN-ZSVC
achieves both better prediction and reconstruction scores. This, coupled with
its fast inference speed (Section 5.1), enables it to be used efficiently and
effectively for downstream data augmentation purposes.

The results of the subjective evaluation are given in Figure 3. To put the
values into context, the MOS for the raw source utterances and vocoded source
utterances included in the analysis are 4.86 and 4.33 respectively – these
serve as an upper bound for the MOS values for both models. Figure 3 largely
supports the objective evaluations, providing further evidence that StarGAN-
ZSVC outperforms AutoVC in zero-shot settings. Interestingly, it would appear
that StarGAN-ZSVC also appears more natural in the traditional seen-to-seen
case. This evaluation indicates that, for human listeners, StarGAN-ZSVC
appears more natural in the low-resource context considered in this paper.

Figure 3: Mean opinion score for naturalness for AutoVC and StarGAN-ZSVC in
various source/target seen/unseen speaker pairings with 95% confidence
intervals shown.

##  6 Conclusion

This paper aimed to improve recent voice conversion methods in terms of speed,
the use of non-parallel training data, and zero-shot prediction capability. To
this end, we adapted the existing StarGAN-VC2 system by using a speaker
encoder to generate speaker embeddings which are used to condition the
generator and discriminator network on the desired source and target speakers.
The resulting model, StarGAN-ZSVC, can perform zero-shot inference and is
trainable with non-parallel data. In a series of experiments comparing
StarGAN-ZSVC to the existing zero-shot voice conversion method AutoVC, we
demonstrated that StarGAN-ZSVC is at least five times faster than AutoVC,
while yielding better scores on objective and subjective metrics in a low-
resource zero-shot voice conversion setting.

For future work, we plan to investigate whether scaling StarGAN-ZSVC up to
larger datasets yields similar performance to existing high-resource voice
conversion systems, and whether the system could be applied to other tasks
aside from pure voice conversion (such as emotion or pronunciation
conversion).

## References

  * [1] Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. EMNLP (2014)
  * [2] Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In: IEEE CVPR (2018)
  * [3] Chorowski, J., Weiss, R.J., Bengio, S., van den Oord, A.: Unsupervised speech representation learning using WaveNet autoencoders. arXiv e-prints arXiv:1901.08810 (2019)
  * [4] Dauphin, Y.N., Fan, A., Auli, M., Grangier, D.: Language Modeling with Gated Convolutional Networks. In: Precup, D., Teh, Y.W. (eds.) PMLR (2017)
  * [5] Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. In: ICLR (2017)
  * [6] Erro, D., Moreno, A.: Weighted frequency warping for voice conversion. In: Interspeech (2007)
  * [7] He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of Tricks for Image Classification with Convolutional Neural Networks. In: IEEE CVPR (2019)
  * [8] Hochreiter, S., Schmidhuber, J.: Long Short-term Memory. Neural Computation 9, 1735–80 (12 1997)
  * [9] Howard, J., Gugger, S.: DynamicUnet: create a U-Net from a given architecture (2020), https://docs.fast.ai/vision.models.unet#DynamicUnet, Last accessed 8 Aug 2020
  * [10] Huang, C., Lin, Y.Y., Lee, H., Lee, L.: Defending Your Voice: Adversarial Attack on Voice Conversion. arXiv e-prints arXiv:2005.08781 (2020)
  * [11] Kameoka, H., Kaneko, T., Tanaka, K., Hojo, N.: StarGAN-VC: Non-parallel many-to-many voice conversion using star generative adversarial networks. In: IEEE SLT Workshop (2018)
  * [12] Kameoka, H., Kaneko, T., Tanaka, K., Hojo, N.: ACVAE-VC: Non-Parallel Voice Conversion With Auxiliary Classifier Variational Autoencoder. IEEE Transactions on Audio, Speech, and Language Processing 27(9), 1432–1443 (2019)
  * [13] Kaneko, T., Kameoka, H., Tanaka, K., Hojo, N.: StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion. In: Interspeech (2019)
  * [14] Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv e-prints arXiv:1412.6980 (2014)
  * [15] Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W.Z., Sotelo, J., de Brébisson, A., Bengio, Y., Courville, A.C.: MelGAN: Generative adversarial networks for conditional waveform synthesis. In: NeurIPS (2019)
  * [16] Lal Srivastava, B.M., Vauquier, N., Sahidullah, M., Bellet, A., Tommasi, M., Vincent, E.: Evaluating Voice Conversion-Based Privacy Protection against Informed Attackers. In: ICASSP (2020)
  * [17] Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., Ling, Z.: The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods . In: Odyssey Speaker and Language Recognition Workshop (2018)
  * [18] Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least Squares Generative Adversarial Networks. ICCV (2017)
  * [19] Miyato, T., Koyama, M.: cGANs with Projection Discriminator. In: ICLR (2018)
  * [20] Morise, M., Yokomori, F., Ozawa, K.: WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications. IEICE Transactions on Information and Systems E99.D(7), 1877–1884 (2016)
  * [21] van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., Kavukcuoglu, K.: WaveNet: A Generative Model for Raw Audio. arXiv e-prints arXiv:1609.03499 (2016)
  * [22] Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: An ASR corpus based on public domain audio books. In: IEEE ICASSP (2015)
  * [23] Prenger, R., Valle, R., Catanzaro, B.: Waveglow: A Flow-based Generative Network for Speech Synthesis. In: IEEE ICASSP (2019)
  * [24] Qian, K., Zhang, Y., Chang, S., Yang, X., Hasegawa-Johnson, M.: AutoVC: Zero-shot voice style transfer with only autoencoder loss. In: PMLR (2019)
  * [25] Rebryk, Y., Beliaev, S.: ConVoice: Real-Time Zero-Shot Voice Style Transfer with Convolutional Network. arXiv e-prints arXiv:2005.07815 (2020)
  * [26] Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: MICCAI. Springer International Publishing (2015)
  * [27] Shuang, Z.W., Bakis, R., Shechtman, S., Chazan, D., Qin, Y.: Frequency warping based on mapping formant parameters. In: Interspeech (2006)
  * [28] Sisman, B., Yamagishi, J., King, S., Li, H.: An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning. arXiv e-prints arXiv:2008.03648 (2020)
  * [29] Smith, L.N.: Cyclical Learning Rates for Training Neural Networks. In: IEEE WACV (2017)
  * [30] Stylianou, Y., Cappe, O., Moulines, E.: Continuous probabilistic transform for voice conversion. IEEE Transactions on Speech and Audio Processing 6(2), 131–142 (1998)
  * [31] Sun, L., Kang, S., Li, K., Meng, H.: Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks. In: IEEE ICASSP (2015)
  * [32] Suundermann, D., Strecha, G., Bonafonte, A., Höge, H., Ney, H.: Evaluation of vtln-based voice conversion for embedded speech synthesis. In: Interspeech (2005)
  * [33] Toda, T., Black, A.W., Tokuda, K.: Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory. IEEE Transactions on Audio, Speech, and Language Processing 15(8), 2222–2235 (2007)
  * [34] Veaux, C., Yamagishi, J., Macdonald, K.: CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (2017), http://homepages.inf.ed.ac.uk/jyamagis/page3/page58/page58.html, Last accessed 1 Sep 2020
  * [35] Wan, L., Wang, Q., Papir, A., Moreno, I.L.: Generalized End-to-End Loss for Speaker Verification. ICASSP (2018)
  * [36] Zhao, Y., Huang, W.C., Tian, X., Yamagishi, J., Das, R.K., Kinnunen, T., Ling, Z., Toda, T.: Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion. arXiv e-prints arXiv:2008.12527 (2020)
  * [37] Zhizheng, W., Haizhou, L.: Voice conversion versus speaker verification: an overview. APSIPA Transactions on Signal and Information Processing 3, e17 (2014)