# Entanglement generation via power-of-SWAP operations between dynamic
electron-spin qubits

=Hugo V. Lepage  Cavendish Laboratory, Department of Physics, University of
Cambridge, Cambridge CB3 0HE, United Kingdom  =Aleksander A. Lasek  Cavendish
Laboratory, Department of Physics, University of Cambridge, Cambridge CB3 0HE,
United Kingdom  Hitachi Cambridge Laboratory, J. J. Thomson Avenue, CB3 0HE,
Cambridge, United Kingdom  David R. M. Arvidsson-Shukur  Cavendish Laboratory,
Department of Physics, University of Cambridge, Cambridge CB3 0HE, United
Kingdom  Department of Mechanical Engineering, Massachusetts Institute of
Technology, Cambridge, Massachusetts 02139, USA  Crispin H. W. Barnes
Cavendish Laboratory, Department of Physics, University of Cambridge,
Cambridge CB3 0HE, United Kingdom

###### Abstract

Surface acoustic waves (SAWs) can create moving quantum dots in piezoelectric
materials. Here we show how electron-spin qubits located on dynamic quantum
dots can be entangled. Previous theoretical and numerical models of quantum-
dot entanglement generation have been insufficient to study quantum dynamics
in realistic experimental devices. We utilize state-of-the-art graphics
processing units to simulate the wave function dynamics of two electrons
carried by a SAW through a 2D semiconductor heterostructure. We build a
methodology to implement a power-of-SWAP gate via the Coulomb interaction. A
benefit of the SAW architecture is that it provides a coherent way of
transporting the qubits through an electrostatic potential. This architecture
allows us to avoid problems associated with fast control pulses and guarantees
operation consistency, providing an advantage over static qubits. For inter-
dot barrier heights where the double occupation energy is sufficiently greater
than the double-dot hopping energy, we find that parameters based on
experiments in GaAs/AlGaAs heterostructures can produce a high-fidelity root-
of-SWAP operation. Our results provide a methodology for a crucial component
of dynamic-qubit quantum computing.

††preprint: APS/123-QED

##  I Introduction

The development of a universal semiconductor quantum computer hinges on the
ability to entangle qubits. One promising method is to use the exchange
interaction between electron-spins. This concept was first introduced by Loss
_et al._ , for static qubits (Loss and DiVincenzo, 1998; Burkard  _et al._ ,
1999), and Barnes _et al._ , for dynamic, also called flying, qubits carried
by surface acoustic waves (SAWs) (Barnes  _et al._ , 2000). The use of flying
qubits trapped in SAWs is a particularly favorable platform for quantum
computation for two reasons. First, the dynamic nature of the qubits enables
on-chip operations to be controlled by static electric and magnetic fields
from surface gates and magnetic microstructures (Furuta  _et al._ , 2004;
McNeil  _et al._ , 2010). Not having to vary surface gate potentials reduces
associated errors. Second, the confinement caused by the SAW potential
prevents spatial dispersion of the fermionic wave packets (Arvidsson-Shukur
_et al._ , 2017). The framework is especially promising for building a
universal quantum transducer—a bus that transports entangled qubits between
spatially separated parts of a quantum computer that could itself be
implemented in a different technology (Schuetz  _et al._ , 2015; Salari and
Darvazehban, 2019).

The last decade has seen significant developments in the achievement of SAW
technologies (DiVincenzo  _et al._ , 2000; Malinowski  _et al._ , 2018).
Advances include the reliable control of single-electron transport (McNeil
_et al._ , 2011; Hermelin  _et al._ , 2011; Ford, 2017; Edlbauer  _et al._ ,
2017) and the increase in electron-qubit coherence times (Stotz  _et al._ ,
2005; Sanada  _et al._ , 2013; Lazić  _et al._ , 2014; Petta  _et al._ ,
2005). The SAW framework for manipulating electron-spin qubits has shown
promise for realizing optics-like quantum processes with readily interacting
particles. Experimentally, single-qubit operations (Edlbauer  _et al._ ,
2017), beam-splitters (Rodriquez  _et al._ , 2005; Takada  _et al._ , 2019)
and spin-polarization readout devices (Kosaka  _et al._ , 2009) have been
realized in GaAs heterostructures, and a spin-qubit toolkit for the
implementation of generalized measurements has been presented (Arvidsson-
Shukur  _et al._ , 2017).

Previous works on flying electron systems have been restricted to either
single particle scenarios (Buscemi  _et al._ , 2010; Bordone  _et al._ ,
2004), or analytical models with a limited number of sites (Burkard  _et al._
, 1999; Barnes  _et al._ , 2000) and simplified simulations in 1D for two
particles(Owen  _et al._ , 2012; Giavaras  _et al._ , 2007).

In the latter case, it was suggested that entanglement generation could be
achieved using a single-shot root-of-SWAP operation in which two electrons
collide in a harmonic potential. See Appendix A for a definition of a logic
power-of-SWAP operation. Attempts simulate realistic devices in layered 3D
systems have faced problems owing to the space- and time-domain scaling
associated with solutions to the many-particle time-dependent Schrödinger
equation (TDSE). However, recent advances in graphics-processing-unit (GPU)
performance (Arunkumar  _et al._ , 2017; S.  _et al._ , 2016) have made
previously demanding problems readily solvable.

In this paper, we utilize state-of-the-art GPU hardware to run a customized
staggered-leapfrog algorithm (Goldberg  _et al._ , 1967; Askar and Cakmak,
1978; Maestri  _et al._ , 2000). Using a combination of two previous time
steps to solve the next, alongside iterative updates of the real and imaginary
parts of the wave function, enables us to simulate the dynamics of two
interacting electrons efficiently and accurately. In particular, we study SAW-
based flying qubits, interacting via the Coulomb interaction in a 2D double-
dot potential. The combination of our custom-built GPU hardware and tailored
software allows us to simulate time-dependent quantum dynamics in a
computation time on the order of days rather than years. Our results
demonstrate the experimental viability of entanglement generation via root-of-
SWAP operations. Furthermore, we show that the single-shot method (Owen  _et
al._ , 2012) is not experimentally feasible. Not only are our simulations
useful to gain insight into quantum logic operations, they also shed new light
on simpler analytical models. Specifically, we compare our simulation results
to two commonly used two-site models: the Hubbard approach (Barnes  _et al._ ,
2000), and the Hund-Mulliken method for molecular orbitals (Burkard  _et al._
, 1999). We establish the limits and applicability of these models. We use
experimentally realistic parameters for the interaction duration, the device
potential, and geometry. To ensure parameter realism, we calculate the
potential profile of the heterostructure with voltages applied to the metallic
gates. We use a Poisson-Schrödinger self-consisted solver to calculate the
range of values that are possible with current semiconductor technologies
Takada  _et al._ (2019). Since this work demonstrates a proof-of-concept for
the SAW-driven entangling operation, we use analytical equations to reproduce
the potentials calculated by our solver. In doing so, we avoid simulating a
specific device implementation and ensure that these simulations are
reproducible and adaptable to different experimental needs. Our work is a
vital step towards constructing the fundamental building blocks of a SAW-based
quantum computer. The simulations we present are based on the GaAs/AlGaAs SAW-
based heterostructure but our methodology, results, and conclusions are
applicable to other semiconductor quantum systems, including static quantum
dots.

This paper is structured as follows. In Sec. II, we describe in detail the
semiconductor device we use as a model in our simulations of SAW-driven
electron dynamics. In Sec. III, we provide an analytical description of a
device potential, as well as calculations of expected evolution of the two-
particle wave function during the root-of-SWAP operation. In Sec. IV, we
describe our numerical techniques and compare two methods for generating
entanglement: the collision method and Coulomb tunneling method. We also
compare these results to simple analytical two-site models. Sec. V contains a
study of the sensitivity of the logic operation as a function of changes in
the experimental parameters. Finally, in Sec. VI, we conclude with a
discussion of our results.

##  II Device Description

Here we describe the device structure and the electron dynamics that allow us
to model an entangling operation between two spin-qubits. The physical spin-
qubits are electrons, and their spatial dynamics are controlled by SAWs. In
each operation, two qubits travel through channels separated by a potential
barrier. At the locus of the entangling operation, this barrier is lower,
allowing the electron qubits to swap via the exchange interaction.

Fig. 1 shows a SAW device designed to carry out a power-of-SWAP entangling
operation on two electrons. The device is an adaptation of the one presented
in (Barnes  _et al._ , 2000). Sinusoidal SAWs are generated by interdigitated
transducers and propagate as transverse plane waves in the positive
$y$-direction. The SAWs modulate the electric potential of a piezoelectric
substrate to produce a train of quantum dots propagating along channels
defined by metallic gates. The SAWs trap pairs of electrons from a two-
dimensional electron gas in the same minimum (Robinson and Barnes, 2001), with
one electron in each channel (separated in the $x$-direction by the tunnel
barrier). The SAW then carries the electrons through their respective
channels. In the center of the device, where the barrier is lower, the
tunneling rate can be controlled by voltages on
$\text{TB}_{\text{L}},\text{TB}_{\text{R}}$.

The quantum dynamics of the system are generated by the travelling SAWs,
therefore the voltages on the surface gates can be held constant throughout
the entangling operation. This gives the SAW-based system a significant
advantage over static qubit systems, which are controlled by generating
voltage pulses that introduce charge noise and can induce stray SAWs, causing
decoherence.

Figure 1:  (a) Cross section of the potential layout in the region of high
barrier alongside a trace of the initial state of the wave function along the
$x$-dimension. (b) Schematic of a SAW-based power-of-SWAP device. Electrons
are carried by SAWs from bottom to top (positive $y$-coordinate) along two
channels, undergoing a power-of-SWAP operation in the central gate region.
Green dotted lines show the path electrons can take through the device.

In what follows we will consider a device consisting of a GaAs/AlGaAs
heterostructure containing a single layer of two-dimensional electron gas
trapped in a quantum well. On the top surface, a pattern of Schottky gates
creates the two channels running in the $y$-direction, separated by a central
barrier. We define the barrier’s center as the origin of the $x$-direction and
label the two channels with subscript L (left) and R (right) for negative or
positive $x$, respectively. Negative voltages on the gates labelled by
$\text{S}_{\text{L}},~{}\text{S}_{\text{R}},~{}\text{SG}_{\text{L}},~{}\text{SG}_{\text{R}},~{}\text{D}_{\text{L}},~{}\text{D}_{\text{R}}$
generate the outer walls of the two channels. Voltages on the gates labelled
CB, $\text{TB}_{\text{L}}$, and $\text{TB}_{\text{R}}$ control the profile of
the central barrier, and ensure that it strongly separates the two channels,
except at the middle part of the device in the $y$-direction, where the
barrier is lower. It is in the region of lower central barrier that the
entangling operation occurs. $\text{TB}_{\text{L}}$ and $\text{TB}_{\text{R}}$
are sufficiently close that they only produce a single potential maximum, in
the $x$-direction, between the two channels (Fig. 1(a)).

##  III Analytical Model

To describe quantum dynamics in this device, we use a two-particle Hamiltonian
of the form

$\hat{H}=\sum_{i=1,2}\left(\frac{\hat{p}_{i}^{2}}{2m_{i}}+\hat{V}_{\text{D}}(r_{i})+\hat{V}_{\text{SAW}}(t,r_{i})\right)+\hat{V}_{\text{C}}(r_{1},r_{2}),$
(1)

where $\hat{V}_{\text{C}}(r_{1},r_{2})$ is the two-particle Coulomb potential,
$\hat{V}_{\text{SAW}}(t,r)$ is the SAW potential carrying the electrons along
the channels and $\hat{V}_{\text{D}}(r)$ is the device potential. This
potential is made up of two parallel harmonic channels running along the
$y$-dimension, coupled in the central gate region by a Gaussian tunnel
barrier, forming a double quantum dot with harmonic confinement perpendicular
to the channels, along the $x-$dimension. An explicit expression is given in
Appendix B. By boosting our reference frame to match the velocity of the SAW,
which is constant, we can treat $\hat{V}_{\text{SAW}}(t,r)$ as a time-
independent confining potential along the channel direction. Finding the
eigenstates of the boosted time-independent Hamiltonian using a number basis
derived from second quantization allows us to obtain the two-particle wave
functions when the barrier between both channels is static. Since the
potential does not have any explicit spin dependence, because of a weak
Lorentz term, single-qubit spin rotations do not occur.

We assume that the electrons in both channels of the device in Fig. 1(b) are
in a separable spin state initially. At this stage, there is a high potential
barrier between the channels and they are too far apart to interact. We also
assume they are in eigenstates of the $z$-axis spin. The spin part of the wave
function can thus be labelled $\ket{s_{1}}\\!\ket{s_{2}}$, meaning that the
first electron is in spin state $s_{1}$, and the second one is in spin state
$s_{2}$. For a double-dot potential, the two-particle ground state is
symmetric in spatial coordinates, described by a spatial wave function
$\ket{\Psi^{\text{S}}(r_{1},r_{2})}$, while the first excited state is anti-
symmetric, with a spatial wave function $\ket{\Psi^{\text{A}}(r_{1},r_{2})}$.
We call the spin-antisymmetric combination a singlet state $\ket{\text{S}}$,
which corresponds to the ground state with energy $E_{\text{S}}$, and the
symmetric state a triplet state $\ket{\text{T}}$, which is the first excited
state with energy $E_{\text{T}}$ (see Fig. 2):

$\ket{\text{S}}=\frac{1}{\sqrt{2}}\ket{\Psi^{\text{S}}(r_{1},r_{2})}\left(\ket{\uparrow}\\!\ket{\downarrow}-\ket{\downarrow}\\!\ket{\uparrow}\right)$
(2)
$\ket{\text{T}}=\frac{1}{\sqrt{2}}\ket{\Psi^{\text{A}}(r_{1},r_{2})}\left(\ket{\uparrow}\\!\ket{\downarrow}+\ket{\downarrow}\\!\ket{\uparrow}\right)$
(3)

We choose the double-dot potential of the gate region such that an equal
linear combination of these states has both particles well localized in
different channels. This results in the eigenstates of initial high tunnel
barrier and those of the gate region having a high overlap. The disturbance
introduced by adiabatically changing the tunnel barrier in the SAW reference
frame is thus minimized. We can write down combined space and spin states as
$\ket{s_{1}s_{2}}_{\text{LR}}$, with particle 1 being in the left channel with
spin $s_{1}$ and particle 2 being in the right channel with spin $s_{2}$. They
are linear combinations of the triplet and singlet states:

$\begin{split}\ket{\downarrow\uparrow}_{\text{LR}}&=\frac{1}{\sqrt{2}}\left(\ket{\text{T}}+\ket{\text{S}}\right)\\\
&=\frac{1}{\sqrt{2}}\left(\ket{\Psi^{\text{RL}}(r_{1},r_{2})}\ket{\uparrow}\\!\ket{\downarrow}-\ket{\Psi^{\text{LR}}(r_{1},r_{2})}\ket{\downarrow}\\!\ket{\uparrow}\right),\end{split}$
(4)

$\begin{split}\ket{\uparrow\downarrow}_{\text{LR}}&=\frac{1}{\sqrt{2}}\left(\ket{\text{T}}-\ket{\text{S}}\right)\\\
&=\frac{1}{\sqrt{2}}\left(\ket{\Psi^{\text{LR}}(r_{1},r_{2})}\ket{\uparrow}\\!\ket{\downarrow}-\ket{\Psi^{\text{RL}}(r_{1},r_{2})}\ket{\downarrow}\\!\ket{\uparrow}\right),\end{split}$
(5)

where $\ket{\Psi^{\text{LR}}(r_{1},r_{2})}$ denotes a spatial state with
particle 1 in the left channel (negative $x$) and particle 2 in the right
channel (positive $x$). These take the form

$\ket{\Psi^{\text{RL}}(r_{1},r_{2})}=\frac{1}{\sqrt{2}}\left(\ket{\Psi^{\text{S}}(r_{1},r_{2})}+\ket{\Psi^{\text{A}}(r_{1},r_{2})}\right),$
(6)
$\ket{\Psi^{\text{LR}}(r_{1},r_{2})}=\frac{1}{\sqrt{2}}\left(\ket{\Psi^{\text{S}}(r_{1},r_{2})}-\ket{\Psi^{\text{A}}(r_{1},r_{2})}\right).$
(7)

Figure 2: Two-particle spatial wave functions. (a) Ground state
$\ket{\Psi^{\text{S}}}$ ($y_{1}=0$ and $y_{2}=0$). (b) First excited state
$\ket{\Psi^{\text{A}}}$ ($y_{1}=0$ and $y_{2}=0$). (c) Combination of the
ground state and first excited state $\ket{\Psi^{\text{LR}}}$. The first
particle is localized in the left channel and the second particle is localized
in the right channel ($y_{1}=0$ and $y_{2}=0$). (d) Gaussian spread of both
particles in the $y$-dimension ($x_{1}=0$ and $x_{2}=0$). All four panels show
the wave function divided by its extremum, with the z-axis in arbitrary units.

A system placed in such a linear superposition oscillates coherently with the
period, $~{}{2\pi\hbar}/{(E_{\text{T}}-E_{\text{S}})}$, determined by the
energy difference between the ground state and first excited state. A full
SWAP operation takes half of this period whilst the root-of-SWAP operation
takes a quarter of it, i.e. half the duration of a SWAP. In the limit where
the on-site Coulomb energy is much greater than the hopping energy, the
doubly-occupied states have vanishingly small probability amplitudes and can
be ignored (Barnes  _et al._ , 2000). The state during the time evolution is

$\ket{\psi(t)}=\frac{1}{\sqrt{2}}\left\\{\ket{\text{T}}+\exp\left(\frac{-it}{\hbar}\Delta
E\right)\ket{\text{S}}\right\\},$ (8)

where $\Delta E=E_{\text{T}}-E_{\text{S}}$. This description of the power-of-
SWAP operation allows us to calculate the probabilities of observing spin-up
(spin-down) particles in the left (right) channels after the operation. The
probability of measuring a swapped state, assuming an initial state
$\ket{\uparrow\downarrow}_{\text{LR}}$ and a fixed time of interaction $\tau$,
is given by

$\displaystyle\begin{split}P_{\text{SWAP}}\left(J\right)={\mid\braket{\psi(t=\tau)}{\downarrow\uparrow}_{\text{LR}}\mid}^{2}\\\
=\text{sin}^{2}\left(\frac{1}{2}{J\cdot\tau}\right),\end{split}$ (9)

where $J=\Delta E/(2\pi\hbar)$. This probability, given an input state,
depends only on the energy difference between the triplet and singlet states,
which in turn is a function of the device potential.

##  IV Simulations and results

Numerical simulations of the two-particle dynamics over two dimensions are
computationally expensive. However, the ability to model a complete set of
energy eigenstates reveals a more complicated behavior, in contrast with
analytical two-site models or one-dimensional simulations. In this section, we
present the numerical results of entanglement generation via two different
root-of-SWAP implementations (Burkard  _et al._ , 1999; Barnes  _et al._ ,
2000; Owen  _et al._ , 2012). In both cases, we find that the realistic
dynamics deviates from the simpler models.

###  IV.1 Numerical methods

The eigenstates of the double-dot system are obtained by numerically solving
the Hamiltonian built using the allowed two-particle basis states (Appendix
C). To reduce the size of our matrix representation, we can find the initial
state of the two-particle system efficiently by using a momentum-space (rather
than a position-space) eigensolver (Appendix D). Since the system of interest
is very close to the ground state, only a small number of momentum basis
states are needed. The ground and first excited spatial states found using
this method are presented in Fig. 2.

These time-independent methods are sufficient to find the initial state of the
system and to describe its time evolution in a constant potential. However,
when the potential varies as the electron travels across the device, time-
dependent simulations need to be used. We evolve the TDSE iteratively using
the staggered-leapfrog algorithm (Goldberg  _et al._ , 1967; Askar and Cakmak,
1978; Maestri  _et al._ , 2000).

Specific details on our GPU implementation of the eigensolver and staggered-
leapfrog simulator can be found in Appendix E.

###  IV.2 Coulomb tunneling entanglement generation

Building on a proposal from Ref. (Barnes  _et al._ , 2000), we explore an
exchange-interaction based method for the generation of entanglement between
two electrons in a SAW system. As described in Section III, the two electrons
occupy adjacent channels separated by a tunneling barrier, suppressing any
wave function overlap. When the electrons enter the low-barrier gate region,
they can tunnel through to the other channel at a rate that is determined in
part by the barrier height and in part by the Coulomb force, thus allowing for
the control of the power-of-SWAP gate by tuning the appropriate Schottky
gates. Fig. 4 shows snapshots of the wave function during an entangling
operation with realistic experimental parameters. When the potential barrier
is low, the two-particle state undergoes coherent oscillations between the
initial state and the fully swapped state. The duration of the two-particle
operation is determined by the length of the tunnel-coupled region. Since the
speed of a SAW is constant in a given material, the operation is identical for
all incoming electron pairs.

Starting with Eq. 9, and assuming that $J$ is exponentially dependent on the
tunnel barrier height $A_{\text{TB}}$, and time of interaction $\tau$ is
fixed, the probability of the final state being swapped with respect to the
initial state has the following dependence on the tunnel barrier:

$\displaystyle\begin{split}P_{\text{SWAP}}\left(A_{\text{TB}},\tau\right)=\text{sin}^{2}\left(\frac{1}{2}{J_{0}\cdot
e^{-b\cdot A_{\text{TB}}}\cdot\tau}\right),\end{split}$ (10)

where $J_{0}$ and $b$ are numerically determined parameters. Fig. 3 shows a
fit of our time-dependent numerical simulation data (See Appendix B for
parameter values used) with the analytical prediction from Eq. 10. It is
important to note that although Eq. 10 can describe the behaviour of a power-
of-SWAP under ideal conditions, a numerical approach is required to account
for more realistic scenarios. These can include the presence of impurities in
the quantum channels as well as a finite transition length between the low and
high tunnel barrier heights. The inset in Fig. 3 shows the probability
amplitude of each computational basis state as the electrons travel through a
root-of-SWAP gate. Interactions between the electrons are initially prohibited
by the high potential barrier separating them. As they are carried through the
interaction region, the electrons become entangled. Upon leaving the region of
low potential barrier, the particles can no longer interact and the
probability amplitudes become constant. We find that the SWAP probability
around $P_{\text{SWAP}}=0.5$ varies with the tunnel barrier height at a rate
of $8.07\times 10^{-4}~{}\mu\text{eV}^{-1}$. This allows for an experimentally
viable tunability of the quantum gate via the control of the tunnel barrier
height. Assuming a device temperature of 300 mK, tunnel barrier variations due
to thermal fluctuations will decrease the root-of-SWAP fidelity by $<0.1\%$.
This error could be reduced by increasing the height of the tunnel barrier, at
the cost of extending the operation time.

Figure 3: Probability of SWAP as a function of tunnel barrier height for fixed
interaction duration. Time evolution simulation results (red circles) are fit
using Equation 9 (solid blue line). The parameters $J_{0}=2.888\text{ps}^{-1}$
and $b=0.933\text{meV}^{-1}$ were found numerically. The inset figure
illustrates the occupation of the computational basis states as well as the
double occupancy states. In this example, the input state
$\ket{\uparrow\downarrow}_{\text{LR}}$ undergoes a root-of-SWAP operation with
finite tunnel barrier potential ramps. Figure 4: Entanglement generation using
the Coulomb tunneling method. Top and middle row: trace over the $x$-dimension
and $y$-dimension respectively for the initial state (left), root-of-SWAP
state (centre), and SWAP state (right) of the wave function. Bottom row: trace
over the second particle for the initial state (left), root-of-SWAP state
(centre), and SWAP state (right) of the wave function. Coordinates are chosen
to be in the SAW frame of reference with $y=0$ corresponding to a SAW minimum
and $x=0$ the peak of the tunnel barrier.

###  IV.3 Comparison to Models

To solve the dynamics of the power-of-SWAP operation in our heterostructure
SAW-based device, including the 2D spatial extent of the wave function and a
time-dependent potential, numerical simulations must be used. However, to
avoid lengthy and complicated computations, $\Delta E$ can be estimated using
simplified two-site models, thus getting an approximation for the power-of-
SWAP extent via Eq. 9.

Assuming a tight-binding-like model, where electrons can tunnel between the
quantum dots, we can estimate the full 2D time evolution by applying the Hund-
Mulliken model for molecular orbitals (Burkard  _et al._ , 1999) (see Appendix
F). Alternatively, the evolution of the two-particle state can also be
modelled with the Hubbard approach for short range Coulomb interaction Barnes
_et al._ (2000) (see Appendix G). Solving the Hund-Mulliken Hamiltonian, we
find the eigenenergies associated with the singlet and triplet states and
define the SWAP frequency in terms of the on-site energy $U$ and the hopping
term $t_{\text{h}}$:

$J=\frac{1}{2\pi\hbar}\left[V_{-}-V_{+}+\frac{1}{2}\left(\sqrt{U_{\text{h}}^{2}+16t_{\text{h}}^{2}}-U_{\text{h}}\right)\right]=\frac{\Delta
E}{2\pi\hbar},$ (11)

where $U_{\text{h}}=U-V_{+}+X$.

For the simplified Hubbard Hamiltonian, this expression reduces to

$J=\frac{1}{4\pi\hbar}\left(-U+\sqrt{U^{2}+16t_{\text{LR}}^{2}}\right)=\frac{\Delta
E}{2\pi\hbar}.$ (12)

For realistic Hamiltonians, it is impossible to obtain $U$ analytically.
Instead, we numerically calculate this parameter. To avoid unphysical results
introduced by the $1/r$ factor, a softened Coulomb potential is used (Owen
_et al._ , 2012) both in the models and the numerical simulations throughout
this work. We implement this softening by assuming that the wave function has
a Gaussian spread in the third dimension, with a standard deviation of
$\Delta_{z}$.

We compare both the Hubbard model and the Hund-Mulliken method described above
to our simulation results for a range of $\Delta_{z}$. We find a close match
between the frequency of the SWAP operation as calculated by our time-
dependent numerical solver and these obtained by solving the eigenvalue
problem directly. Both models (Eq. 11 and Eq. 12) show significant discrepancy
for most values of $\Delta_{z}$. Moreover, the Hund-Mulliken model predicts
negative frequencies for $\Delta_{z}<1$ nm. We conclude that although both
models provide a reasonable qualitative prediction of the two-particle
dynamics for Gaussian spread of $\Delta_{z}\sim 10-100$ nm, a more
sophisticated numerical approach, such as the one used in this work, is
required to obtain precise quantitative dynamics. A comparison of both
analytical models with our simulations can be seen in Fig. 5.

Figure 5: Comparison to analytical models. Power-of-SWAP frequency as a
function of effective wave function spread in the $z$-dimension. Coulomb
softening accounts for the finite $z$-dimension and plays an important role in
determining the rate of the exchange interaction.

###  IV.4 Entanglement generation via electron collisions

In a previously suggested root-of-SWAP scheme (Owen  _et al._ , 2012), two
electrons are travelling in individual channels separated by a high potential
barrier, such that there is no wave-function overlap. The potential barrier
abruptly (or diabatically) changes in the SAW reference frame such that the
two channels are joined to create a global potential minimum between them.
Without the presence of the barrier, both electrons fall towards one another
in a harmonic oscillator potential and interact via the Coulomb force. Once
the operation is completed, the central barrier is reintroduced, causing the
reappearance of separate decoupled channels. As the quantum states of
particles in layered semiconductor technologies are confined in the dimension
perpendicular to the quantum wells, which has a constant potential throughout
the device, the third dimension does not significantly affect the operation.
However, we find that the previous reduction to 1D is an oversimplification,
as the possible spatial dynamics in the second dimension strongly affects the
electron-electron interactions.

Here, we simulate this single-shot (i.e. in a single collision) entanglement
generation, and we find that under current experimentally realistic
parameters, it is impossible to generate a root-of-SWAP, or any significant
entanglement over the $x$-dimension.

Figure 6: Entanglement generation via the collision of two electrons. Top and
middle row: trace over the $x$-dimension and $y$-dimension respectively for
the initial state (left), root-of-SWAP state (centre), and SWAP state (right)
of the wave function. Bottom row: trace over the second particle for the
initial state (left), root-of-SWAP state (centre), and SWAP state (right) of
the wave function. Coordinates are chosen to be in the SAW frame of reference
with $y=0$ corresponding to a SAW minimum and $x=0$ the middle of the harmonic
channel.

Fig. 6 shows snapshots of the two-electron wave function undergoing a single
collision in two dimensions. The wave function remains fully separable along
the $x$-dimension. However, in the $y$-dimension, it transitions from a
Gaussian-like low-energy state to a more spread-out entangled state. This is
conflicting with the desired outcome of generating a maximally entangled state
in the $x$-dimension. The operation is effectively a SWAP instead of a root-
of-SWAP, with the additional downside of exciting higher-energy states in the
$y$-dimension. These unwanted spatial excitations of the wave function lead to
lowering the spatial fidelity of the operation and thus it is not possible to
concatenate multiple operations for useful quantum information processing.
This also prevents the restoration of the wave function to its original state
by applying the SWAP twice, a fundamental property of this operation. We find
that increasing the $y$ confinement does not prevent this behaviour until the
SAW amplitude is increased by a factor of order $10^{3}$, where the problem
effectively reduces to 1D. However, this would require SAW amplitudes on the
order of $10^{4}$ meV, which is experimentally unrealistic (Talyanskii  _et
al._ , 1997). Varying the $x$ confinement over a wide range also does not
solve the issue. Therefore we conclude that the collision method is unable to
produce the root-of-SWAP operation in a realistic 2D scenario.

##  V Experimental Sensitivity

Here, we investigate the power-of-SWAP operation’s sensitivity to disturbances
in $A_{\text{TB}}$ and $\tau$, which the output probabilities depend on. From
an information-theoretical perspective, an experiment’s sensitivity to an
unknown physical parameter, $\theta$, is quantified by the Fisher information:

$F(\theta)=\sum_{i}P(\mathcal{M}_{i}|\theta)\bigg{[}\frac{\partial}{\partial\theta}\ln
P(\mathcal{M}_{i}|\theta)\bigg{]}^{2},$ (13)

where $\mathcal{M}_{i}$ denotes the $i^{\mathrm{th}}$ measurement’s outcome
(Helstrom, 1969). Given $N$ experimental runs, the precision of an estimate
$\hat{\theta}$ of $\theta$ is bounded by the Cramér-Rao inequality:
$\mathrm{Var}(\hat{\theta})\geq[NF(\theta)]^{-1}$, such that the greater the
Fisher information, the smaller the estimator’s variance can be (Frieden,
2004). Using the output probabilities from Sec. III, we find that
$F(A_{\text{TB}})=b^{2}\tau^{2}J_{0}^{2}e^{-2bA_{\text{TB}}}$: the ability to
estimate $A_{\text{TB}}$ decreases exponentially with $A_{\text{TB}}$ itself.
However, for parameters that yield the root-of-SWAP operation
($A_{\text{TB}}\approx 3.86\;\rm{meV}$) we find that $F(A_{\text{TB}})\approx
2.15\;(\rm{meV})^{-2}$. This value of $F(A_{\text{TB}})$ lower bounds the
standard deviation of $A_{\text{TB}}$: $\sigma_{A_{\text{TB}}}\gtrapprox
0.012\;\rm{meV}$ in an experiment with $N=3000$ trials. Despite the
exponentially decreasing sensitivity of tunnel-barrier heights, the relevant
values for a root-of-SWAP operation are within the experimentally viable
regime (Takada  _et al._ , 2019) specified in Appendix B. The Fisher
information about $\tau$ is $F(\tau)=J_{0}^{2}e^{-2bA_{\text{TB}}}$ is
constant with respect to $\tau$ itself. For the ideal root-of-SWAP parameters,
we find that $F(\tau)\approx 6.17\times 10^{-3}\;\rm{ps}^{-2}$, which gives a
lower bound on the standard deviation of $\tau$: $\sigma_{\tau}\gtrapprox
0.23\;\rm{ps}$, when $N=3000$.

##  VI Discussion And Conclusion

The two-qubit entangling operation is an essential building block of a quantum
information processor. We have shown that surface-gate-controlled flying
electron-spin qubits are able to generate entanglement through the power-of-
SWAP operation in a reliable and stable fashion. We show that the problem of
wave function dispersion can be solved through the use of SAWs, which generate
the potential confinement needed to preserve the wave function’s profile. We
present accurate numerical solutions to the time-dependent Schrödinger
equation using a staggered-leapfrog method and we investigate previously
proposed schemes for generating entanglement between electron-spin qubits. We
find that realising the power-of-SWAP operation via electron collision (Owen
_et al._ , 2012) suffers from significant problems, whilst an implementation
based on tunnelling (Barnes  _et al._ , 2000; Burkard  _et al._ , 1999) is
shown to be realizable with high fidelity even when experimental control of
the tunneling barrier is imperfect. We find that this entangling operation
governed by the exchange interaction and coherent tunneling of electrons
offers a more stable approach and makes high gate fidelities possible.

Our two-particle simulations use experimentally realistic parameters and
potential layouts and show that such devices are readily realizable using
current semiconductor fabrication techniques. While the behaviour of an ideal
system can be predicted exactly by solving the Hamiltonian and assuming that
the electrons are initialized to and remain in a combination of triplet and
singlet states, the advantage of our numerical methods is to simulate
realistic entangling operations. Although these simulations were focused on
the experimental parameters of GaAs-based devices, the same behaviour is
expected in other SAW-based semiconductor devices. Moreover, our findings can
be generalized to systems that do not include SAWs. Static quantum dots,
confined in every dimension and separated by a tunnel barrier interact in the
same way. Such a tunnel barrier can be modulated using fast microwave pulses
(Brunner  _et al._ , 2011) to reproduce the two-particle dynamics presented in
this paper. A static root-of-SWAP gate was recently realised with high
fidelity using phosphorus donors in silicon (He  _et al._ , 2019), proving
that such systems are achievable experimentally. Coherent spin state SWAP
operations between electron-spin qubits in a quadruple array of semiconductor
quantum dots were also lately achieved (Kandel  _et al._ , 2019).

Our results provide new evidence that an entangling root-of-SWAP gate based on
the exchange interaction is experimentally viable in SAW-based semiconductor
heterostructures.

##  VII Acknowledgements

A.A.L. and H.V.L. have contributed equally to this work. This project has
received funding from the European Union’s Horizon 2020 research and
innovation programme under the Marie Skłodowska-Curie grant agreement No
642688. This work was supported by the Project for Developing Innovation
Systems of the Ministry of Education, Culture, Sports, Science and Technology
(MEXT), Japan. A.A.L. acknowledges support from EPSRC and Hitachi via CASE
studentships RG 94632. D.R.M.A.S. acknowledges support from the EPSRC, Hitachi
Cambridge, Lars Hierta’s Memorial Foundation and the Sweden-America
Foundation. The authors would also like to thank Prof. Chris Ford for valuable
discussions and comments.

##  Appendix A Power-of-SWAP quantum logic gate

In the two-qubit basis $\ket{00},\ket{01},\ket{10},\ket{11}$, the Power-of-
SWAP operation for $n-th$ power is represented by the matrix:

$SWAP^{n}=\begin{pmatrix}1&0&0&0\\\ 0&\frac{1}{2}(1+e^{i\pi
n})&\frac{1}{2}(1-e^{i\pi n})&0\\\ 0&\frac{1}{2}(1-e^{i\pi
n})&\frac{1}{2}(1+e^{i\pi n})&0\\\ 0&0&0&1\\\ \end{pmatrix},$ (14)

For root-of-SWAP, $n=\frac{1}{2}$ and the matrix representation is:

$\sqrt{SWAP}=\begin{pmatrix}1&0&0&0\\\
0&\frac{1}{2}(1+i)&\frac{1}{2}(1-i)&0\\\
0&\frac{1}{2}(1-i)&\frac{1}{2}(1+i)&0\\\ 0&0&0&1\\\ \end{pmatrix},$ (15)

##  Appendix B Parameter values

Parameter | Value range
---|---
Distance between channels | 80 nm
Tunnel coupled region start |  $y_{d}$ = 36 nm
Tunnel coupled region end |  $y_{u}$ = 144 nm
Interaction time |  $\tau=$ 36 ps
SAW amplitude |  $A_{\text{SAW}}$ = 25 meV
SAW wavelength |  $\lambda=1\upmu$m
SAW velocity |  $v$ = 3 nm / ps
Harmonic channel confinement |  $\omega_{x}^{2}$ = 0.002 $\frac{\text{meV}}{{\text{nm}}^{2}m_{e}}$
Electron effective mass | 0.067 $m_{e}$
Relative permitivitty (GaAs) | 13.1
Gaussian tunneling barrier amplitude |  $A_{1}$ = 15.3 meV
Gaussian tunneling barrier width |  $\sigma_{1}$ = 30 - 40 nm
Gaussian barrier amplitude |  $A_{2}$ = 510 meV
Gaussian barrier width |  $\sigma_{2}$ = 0.8 nm
Transition between barrier heights |  $\sigma_{y}$ = 10 nm
Coulomb softening |  $\Delta_{z}$ = 10 - 100 nm
Table 1: Ranges of parameter values used in simulations.

Explicit form of potentials used in Eq. 1, in terms of the parameters above,
in reference frame of the SAW:

$\displaystyle\begin{split}&V_{\text{D}}(x,y)=\frac{m_{e}}{2}\omega_{x}^{2}x^{2}+A_{1}\exp\left(\frac{-x^{2}}{2\sigma_{1}^{2}}\right)+\frac{A_{2}}{2}\exp\left(\frac{-x^{2}}{2\sigma_{2}^{2}}\right)\\\
&\times\left(2-\tanh\left(\frac{y-y_{d}}{\sigma_{y}}\right)-\tanh\left(-\frac{y-y_{u}}{\sigma_{y}}\right)\right).\end{split}$
(16)

$V_{\text{SAW}}(x,t)=\frac{A_{\text{SAW}}}{2}\left(1-\cos\left(\frac{x-tv}{\lambda}\right)\right)$
(17)

$V_{C}(r)=\frac{e^{2}}{4\sqrt{2}\pi\epsilon\Delta_{z}U(-\frac{1}{2},0,\frac{r^{2}}{2\Delta_{z}^{2}})}$
(18)

where $U$ is the confluent hypergeometric function of the second kind, which
encapsulates a Gaussian spread with standard deviation $\Delta_{z}$ in the
$z$-dimension:

$U(a,b,z)=\frac{1}{\Gamma(a)}\int_{0}^{\infty}e^{-zt}t^{a-1}(1+t)^{b-a-1}dt$
(19)

##  Appendix C Second Quantization basis

The full set of basis states of the system using second quantization, with two
spin-$\nicefrac{{1}}{{2}}$ fermions occupying $i$-th and $j$-th out of $N$
spatial sites respectively, is given by

$\displaystyle\begin{split}\ket{\uparrow\uparrow}_{ij}=c^{\dagger}_{i\uparrow}c^{\dagger}_{j\uparrow}\ket{0},i\neq
j\\\
\ket{\downarrow\downarrow}_{ij}=c^{\dagger}_{i\downarrow}c^{\dagger}_{j\downarrow}\ket{0},i\neq
j\\\
\ket{\uparrow\downarrow}_{ij}=c^{\dagger}_{i\uparrow}c^{\dagger}_{j\downarrow}\ket{0},\\\
\ket{\downarrow\uparrow}_{ij}=c^{\dagger}_{i\downarrow}c^{\dagger}_{j\uparrow}\ket{0}.\end{split}$
(20)

These fermionic creation operators obey the anticommutation relation
$\\{c^{\dagger}_{is_{1}},c^{\dagger}_{js_{2}}\\}=0$. Therefore, the basis
states are also related by
$\ket{\downarrow\uparrow}_{ij}=-\ket{\uparrow\downarrow}_{ji}$. With $N=2$,
the basis states of the Hubbard two-site model as in Appendix G are obtained.

##  Appendix D Momentum Space Eignesolver

A real-space wave function can be written in the discrete case as a sum of the
momentum eigenfunctions:

$\ket{\psi(\boldsymbol{r})}=\sum_{k}^{N_{k}}\phi_{k}\ket{\psi_{k}(\boldsymbol{r})}$
(21)

where $\ket{\psi_{k}(\boldsymbol{r})}$ are the momentum eigenfunctions.
Owing to the canonical relation between momentum and position operators,

$\hat{p}=-i\hbar\frac{\partial}{\partial r},$ (22)

the eigenfunctions are

$\ket{\psi_{k}\left(\boldsymbol{r}\right)}={(2\pi)^{\frac{d}{2}}}\ket{e^{i\boldsymbol{k}\cdot\boldsymbol{r}}}$
(23)

in $d$ dimensions. Therefore, the real space wave function can be written as:

$\ket{\psi(\boldsymbol{r})}={(2\pi)^{\frac{d}{2}}}\sum_{k}^{N_{k}}\phi_{k}\ket{e^{i\boldsymbol{k}\cdot\boldsymbol{r}}}$
(24)

This is just a Discrete Fourier Transform:

$\ket{\psi(\boldsymbol{r})}=\mathcal{F}\left(\ket{\phi(\boldsymbol{k})}\right)$
(25)

Similarly, the inverse is also true:

$\ket{\phi(\boldsymbol{k})}=\mathcal{F}^{-1}\left(\ket{\psi(\boldsymbol{r})}\right)$
(26)

where $\mathcal{F}^{-1}$ denotes the inverse Discrete Fourier transform.

Momentum space can be discretised in multiples of $\frac{2\pi}{L}$, where $L$
is a real space extent over which the amplitude of the wave function should
have decayed to 0 near the edges. Defining some cutoff $N_{k}$, momentum will
take the values:

${k}_{n}=n\frac{2\pi}{L},$ (27)

where $n=[-N_{k},-N_{k}+1,...,0,1,...,N_{k}]$.

When we Fourier transform the Schrödinger equation into momentum space, the
kinetic term $-\frac{\hbar^{2}}{2m}\nabla^{2}$ will become
$\frac{\hbar^{2}}{mL^{2}}\left(\text{cosh}(kL)-1\right)$ owing to discreteness
and the canonical momentum-position relation (Eq. 22). The Fourier-transformed
potential matrix element becomes

$\displaystyle\begin{split}\hat{V}_{kq}={(2\pi)}^{\frac{d}{2}}\sum_{r}^{N}e^{-i\boldsymbol{q}\cdot\boldsymbol{r}}V(\boldsymbol{r})e^{i\boldsymbol{k}\cdot\boldsymbol{r}}=\\\
{(2\pi)}^{\frac{d}{2}}\sum_{r}^{N}V(\boldsymbol{r})e^{i(\boldsymbol{k-q})\cdot\boldsymbol{r}}=\mathcal{F}\left(V(\boldsymbol{r})\right)(k-q)\end{split}$
(28)

The elements of the Hamiltonian matrix to be solved by diagonalisation in this
method take the form

$H_{kq}=\frac{\hbar^{2}}{mL^{2}}\left[\text{cosh}(kL)-1\right]\delta_{kq}+\hat{V}_{kq}$
(29)

Note that this matrix is not sparse as was the case in real space. This could
be thought of as reducing the memory space (smaller matrix) at the cost of
computational complexity. However owing to the efficiency of Fast Fourier
Transform algorithms, the problem is faster to solve as opposed to a real-
space solver. This method extends readily to two or more particles.

##  Appendix E GPU Implementation

###  E.1 Eigensolver

For the double-quantum-dot system investigated here, time-independent
solutions converge for 10 or more momentum basis states in each dimension. The
momentum wave function is zero-padded to the required real-space number of
points, which should be at least 100 in each dimension for the real-space
time-dependent solver. An inverse Discrete Fourier Transform (DFT) is applied.
Using the momentum basis makes the two-particle problem tractable in two
dimensions, as the Hamiltonian is reduced from $\sim 10^{16}$ to $\sim 10^{8}$
elements. Such a matrix of complex floating point numbers reaches the
limitations of random access memory (RAM) normally available to modern
computers. Our eigensolver can also run on a GPU for increased speed, however
GPU Video RAM tends to be smaller than CPU RAM. Memory is a limiting factor,
and only state-of-the-art devices are able to solve the problem accurately,
i.e. using at least 10 momentum modes in each dimension. A two-particle 2D
problem with 10 points in each dimension can be solved in tens of minutes on a
modern desktop computer and gives a result accurate enough to be used as a
starting point for the time-dependent solver while keeping the normalization
errors below 1%.

###  E.2 Staggered-lepfrog

We run our time-dependent iterative solver on GPU hardware to accelerate the
computation by parallelizing updates for each spatial site. The wave function
can be evolved either in real or momentum space, using DFTs to transform
between the two. We find that if the wave function does not contain
significant contributions from high-energy eigenstates, employing the momentum
basis allows us to reduce the wave-function size compared to real space,
whilst keeping the same accuracy. We use about 100 real-space, or 20 momentum-
space points, per dimension. However, the momentum solver tends to perform
poorly when sudden and strong interactions take place. We find it optimal to
use real space for simulating the collision method, and momentum space for the
Coulomb tunneling method.

For improved simulation speed and accuracy, we time-evolve only the
$\ket{\uparrow\downarrow}_{ij}$ part as a spin-independent wave function
$\ket{\psi}_{ij}$. Using such a state is justified, as the root-of-SWAP
operation only has an effect on $\ket{\uparrow\downarrow}_{ij}$ and
$\ket{\downarrow\uparrow}_{ij}$ states, which are related by a spatial index
exchange $(\ket{\downarrow\uparrow}_{ij}=-\ket{\uparrow\downarrow}_{ji})$.
Furthermore, $\ket{\uparrow\uparrow}_{ij}$ and
$\ket{\downarrow\downarrow}_{ij}$ states are invariant under the power-of-SWAP
operation, and thus irrelevant in this work.

##  Appendix F Hund-Mulliken model for molecular orbitals

The Hund-Mulliken model for molecular orbitals (Burkard  _et al._ , 1999)
builds a two-particle basis from right- and left-localized single-particle
states $\ket{\phi_{\pm}}$. These states are orthonormalised to
$\ket{\Phi_{\pm}}=(\ket{\phi_{\pm}}-g\ket{\phi_{\mp}})/(\sqrt{1-2Sg+g^{2}})$,
where $S=\braket{\phi_{\pm}\mid{\phi_{\mp}}}$ is the wave function overlap and
$g=(1-\sqrt{1-S^{2}})/S$. The singly- and doubly-occupied two-particle basis
is constructed with direct products:

$\begin{split}&\ket{\Psi^{s}_{\mp}}=\frac{1}{\sqrt{2}}\left(\ket{\Phi_{+}}\\!\ket{\Phi_{-}}\mp\ket{\Phi_{-}}\\!\ket{\Phi_{+}}\right),\\\
&\ket{\Psi^{d}_{\mp}}=\ket{\Phi_{\mp}}\\!\ket{\Phi_{\mp}}.\end{split}$ (30)

The Hamiltonian in this basis has the form:

$\hat{H}=\begin{pmatrix}V_{-}&0&-\sqrt{2}t_{\text{h}}&0\\\
0&V_{+}&-\sqrt{2}t_{\text{h}}&0\\\ 0&-\sqrt{2}t_{\text{h}}&U&X\\\
0&-\sqrt{2}t_{\text{h}}&X&U\\\ \end{pmatrix},$ (31)

where each entry is defined as:

$U=\frac{e^{2}}{4\pi\epsilon}\bra{\Psi^{d}_{\pm}}\frac{1}{r}\ket{\Psi^{d}_{\pm}},$
(32)
$X=\frac{e^{2}}{4\pi\epsilon}\bra{\Psi^{d}_{\pm}}\frac{1}{r}\ket{\Psi^{d}_{\mp}},$
(33)
$V_{+}=\frac{e^{2}}{4\pi\epsilon}\bra{\Psi^{s}_{+}}\frac{1}{r}\ket{\Psi^{s}_{+}},$
(34)
$V_{-}=\frac{e^{2}}{4\pi\epsilon}\bra{\Psi^{s}_{-}}\frac{1}{r}\ket{\Psi^{s}_{-}},$
(35)

and $t_{\text{h}}$ is the hopping term

$t_{\text{h}}=\bra{\Phi_{\pm}}\frac{\hat{p}^{2}}{2m}\ket{\Phi_{\mp}}-\frac{e^{2}}{4\sqrt{2}\pi\epsilon}\bra{\Psi^{s}_{+}}\frac{1}{r}\ket{\Psi^{d}_{\pm}}.$
(36)

Solving the Hund-Mulliken Hamiltonian in Eq. 31, we find the eigenenergies
associated with the singlet and triplet states and define the SWAP frequency
in terms of $U$ and $t_{\text{h}}$:

$J=\frac{1}{2\pi\hbar}\left[V_{-}-V_{+}+\frac{1}{2}\left(\sqrt{U_{\text{h}}^{2}+16t_{\text{h}}^{2}}-U_{\text{h}}\right)\right]=\frac{E_{\text{T}}-E_{\text{S}}}{2\pi\hbar},$
(37)

where $U_{\text{h}}=U-V_{+}+X$.

##  Appendix G Hubbard model

The evolution of the two-particle state can be modelled with the Hubbard
approach for short range Coulomb interaction without magnetic fields Barnes
_et al._ (2000). The simplified 2-site Hamiltonian in the second quantization
basis from Eq. 20 then has the form

$\hat{H}=\begin{pmatrix}V&0&0&0&0&0\\\ 0&V&0&0&0&0\\\
0&0&0&0&-t_{\text{LR}}&-t_{\text{LR}}\\\
0&0&0&0&t_{\text{LR}}&t_{\text{LR}}\\\ 0&0&-t_{\text{LR}}&t_{\text{LR}}&U&0\\\
0&0&-t_{\text{LR}}&t_{\text{LR}}&0&U\\\ \end{pmatrix},$ (38)

where $t_{\text{LR}}$ is the hopping term

$t_{\text{LR}}=\tensor{[}_{\text{L}}]{\bra{\uparrow}}{}\frac{\hat{p}^{2}}{2m}\ket{\uparrow}_{\text{R}}=\tensor{[}_{\text{L}}]{\bra{\downarrow}}{}\frac{\hat{p}^{2}}{2m}\ket{\downarrow}_{\text{R}},$
(39)

$U$ is the on-site energy

$U=\frac{e^{2}}{4\pi\epsilon}\tensor{[}_{\text{{LL}}}]{\bra{\uparrow\downarrow}}{}\frac{1}{r}\ket{\uparrow\downarrow}_{\text{{LL}}}=\tensor{[}_{\text{{RR}}}]{\bra{\uparrow\downarrow}}{}\frac{1}{r}\ket{\uparrow\downarrow}_{\text{{RR}}},$
(40)

and

$V=\frac{e^{2}}{4\pi\epsilon}\tensor{[}_{{\text{LR}}}]{\bra{\uparrow\uparrow}}{}\frac{1}{r}\ket{\uparrow\uparrow}_{\text{{LR}}}=\tensor{[}_{\text{{LR}}}]{\bra{\downarrow\downarrow}}{}\frac{1}{r}\ket{\downarrow\downarrow}_{\text{{LR}}}.$
(41)

Solving this Hamiltonian, we find the eigenenergies associated with the
singlet and triplet states and define the SWAP frequency in terms of $U$ and
$t_{\text{LR}}$:

$J=\frac{1}{4\pi\hbar}\left(-U+\sqrt{U^{2}+16t_{\text{LR}}^{2}}\right)=\frac{E_{\text{T}}-E_{\text{S}}}{2\pi\hbar}.$
(42)

## References

  * Loss and DiVincenzo (1998) D. Loss and D. P. DiVincenzo, Physical Review A 57, 120 (1998).
  * Burkard  _et al._ (1999) G. Burkard, D. Loss, and D. P. DiVincenzo, Physical Review B 59, 2070 (1999).
  * Barnes  _et al._ (2000) C. H. W. Barnes, J. M. Shilton, and A. M. Robinson, Physical Review B 62, 8410 (2000).
  * Furuta  _et al._ (2004) S. Furuta, C. H. W. Barnes, and C. J. L. Doran, Physical Review B 70, 205320 (2004).
  * McNeil  _et al._ (2010) R. P. G. McNeil, R. J. Schneble, M. Kataoka, C. J. B. Ford, T. Kasama, R. E. Dunin-Borkowski, J. M. Feinberg, R. J. Harrison, C. H. W. Barnes, D. H. Y. Tse, T. Trypiniotis, J. A. C. Bland, D. Anderson, G. A. C. Jones, and M. Pepper, Nano Letters 10, 1549 (2010).
  * Arvidsson-Shukur  _et al._ (2017) D. R. M. Arvidsson-Shukur, H. V. Lepage, E. T. Owen, T. Ferrus, and C. H. W. Barnes, Physical Review A 96, 052305 (2017).
  * Schuetz  _et al._ (2015) M. J. A. Schuetz, E. M. Kessler, G. Giedke, L. M. K. Vandersypen, M. D. Lukin, and J. I. Cirac, Phys. Rev. X 5, 031031 (2015).
  * Salari and Darvazehban (2019) A. Salari and A. Darvazehban, Journal of Low Temperature Physics 195, 138 (2019).
  * DiVincenzo  _et al._ (2000) D. P. DiVincenzo, D. Bacon, J. Kempe, G. Burkard, and K. B. Whaley, Nature 408, 339 (2000).
  * Malinowski  _et al._ (2018) F. K. Malinowski, F. Martins, T. B. Smith, S. D. Bartlett, A. C. Doherty, P. D. Nissen, S. Fallahi, G. C. Gardner, M. J. Manfra, C. M. Marcus, and F. Kuemmeth, Phys. Rev. X 8, 011045 (2018).
  * McNeil  _et al._ (2011) R. P. G. McNeil, M. Kataoka, C. J. B. Ford, C. H. W. Barnes, D. Anderson, G. A. C. Jones, I. Farrer, and D. A. Ritchie, Nature 477, 439 (2011).
  * Hermelin  _et al._ (2011) S. Hermelin, S. Takada, M. Yamamoto, S. Tarucha, A. D. Wieck, L. Saminadayar, C. Bäuerle, and T. Meunier, Nature 477, 435 (2011).
  * Ford (2017) C. J. B. Ford, Physica Status Solidi (b) 254 (2017).
  * Edlbauer  _et al._ (2017) H. Edlbauer, S. Takada, G. Roussely, M. Yamamoto, S. Tarucha, A. Ludwig, A. D. Wieck, T. Meunier, and C. Bäuerle, Nature communications 8, 1710 (2017).
  * Stotz  _et al._ (2005) J. A. H. Stotz, R. Hey, P. V. Santos, and K. H. Ploog, Nature materials 4, 585 (2005).
  * Sanada  _et al._ (2013) H. Sanada, Y. Kunihashi, H. Gotoh, K. Onomitsu, M. Kohda, J. Nitta, P. V. Santos, and T. Sogawa, Nature Physics 9, 280 (2013).
  * Lazić  _et al._ (2014) S. Lazić, A. Violante, K. Cohen, R. Hey, R. Rapaport, and P. V. Santos, Physical Review B 89, 085313 (2014).
  * Petta  _et al._ (2005) J. R. Petta, A. C. Johnson, J. M. Taylor, E. A. Laird, A. Yacoby, M. D. Lukin, C. M. Marcus, M. P. Hanson, and A. C. Gossard, Science 309, 2180 (2005).
  * Rodriquez  _et al._ (2005) R. Rodriquez, D. K. L. Oi, M. Kataoka, C. H. W. Barnes, T. Ohshima, and A. K. Ekert, Physical Review B 72, 085329 (2005).
  * Takada  _et al._ (2019) S. Takada, H. Edlbauer, H. V. Lepage, J. Wang, P.-A. Mortemousque, G. Georgiou, C. H. W. Barnes, C. J. B. Ford, M. Yuan, P. V. Santos, X. Waintal, A. Ludwig, A. D. Wieck, M. Urdampilleta, T. Meunier, and C. Bäuerle, Nature Communications 10, 4557 (2019).
  * Kosaka  _et al._ (2009) H. Kosaka, T. Inagaki, Y. Rikitake, H. Imamura, Y. Mitsumori, and K. Edamatsu, Nature 457, 702 (2009).
  * Owen  _et al._ (2012) E. T. Owen, M. C. Dean, and C. H. W. Barnes, Physical Review A 85, 022319 (2012).
  * Giavaras  _et al._ (2007) G. Giavaras, J. H. Jefferson, M. Fearn, and C. J. Lambert, Physical Review B 76, 245328 (2007).
  * Buscemi  _et al._ (2010) F. Buscemi, P. Bordone, and A. Bertoni, Phys. Rev. B 81, 045312 (2010).
  * Bordone  _et al._ (2004) P. Bordone, A. Bertoni, M. Rosini, S. Reggiani, and C. Jacoboni, Semiconductor Science and Technology 19, S412 (2004).
  * Arunkumar  _et al._ (2017) A. Arunkumar, E. Bolotin, B. Cho, U. Milic, E. Ebrahimi, O. Villa, A. Jaleel, C. J. Wu, and D. Nellans, SIGARCH Comput. Archit. News 45, 320 (2017).
  * S.  _et al._ (2016) S. S., W. Q., X. P., and C. X., in  _2016 7th International Conference on Cloud Computing and Big Data (CCBD)_ (2016) pp. 99–104.
  * Goldberg  _et al._ (1967) A. Goldberg, H. M. Schey, and J. L. Schwartz, American Journal of Physics 35, 177 (1967).
  * Askar and Cakmak (1978) A. Askar and A. S. Cakmak, The Journal of Chemical Physics 68, 2794 (1978).
  * Maestri  _et al._ (2000) J. J. V. Maestri, R. H. Landau, and M. J. Páez, American Journal of Physics 68, 1113 (2000).
  * Robinson and Barnes (2001) A. M. Robinson and C. H. W. Barnes, Phys. Rev. B 63, 165418 (2001).
  * Talyanskii  _et al._ (1997) V. I. Talyanskii, J. M. Shilton, M. Pepper, C. G. Smith, C. J. B. Ford, E. H. Linfield, D. A. Ritchie, and G. A. C. Jones, Phys. Rev. B 56, 15180 (1997).
  * Helstrom (1969) C. W. Helstrom, Journal of Statistical Physics 1, 231 (1969).
  * Frieden (2004) B. R. Frieden,  _Science from Fisher information: a unification_ (Cambridge University Press, 2004).
  * Brunner  _et al._ (2011) R. Brunner, Y.-S. Shin, T. Obata, M. Pioro-Ladrière, T. Kubo, K. Yoshida, T. Taniyama, Y. Tokura, and S. Tarucha, Phys. Rev. Lett. 107, 146801 (2011).
  * He  _et al._ (2019) Y. He, S. K. Gorman, D. Keith, L. Kranz, J. G. Keizer, and M. Y. Simmons, Nature 571, 371 (2019).
  * Kandel  _et al._ (2019) Y. P. Kandel, H. Qiao, S. Fallahi, G. C. Gardner, M. J. Manfra, and J. M. Nichol, Nature 573, 553 (2019).


[affiliation=1]Yi-ChengLin [affiliation=1]Tzu-QuanLin [affiliation=1]Hsi-
CheLin [affiliation=1,2]Andy T.Liu [affiliation=1]Hung-yiLee

# On the social bias of speech self-supervised models

###### Abstract

Self-supervised learning (SSL) speech models have achieved remarkable
performance in various tasks, yet the biased outcomes, especially affecting
marginalized groups, raise significant concerns. Social bias refers to the
phenomenon where algorithms potentially amplify disparate properties between
social groups present in the data used for training. Bias in SSL models can
perpetuate injustice by automating discriminatory patterns and reinforcing
inequitable systems. This work reveals that prevalent SSL models inadvertently
acquire biased associations. We probe how various factors, such as model
architecture, size, and training methodologies, influence the propagation of
social bias within these models. Finally, we explore the efficacy of debiasing
SSL models through regularization techniques, specifically via model
compression. Our findings reveal that employing techniques such as row-pruning
and training wider, shallower models can effectively mitigate social bias
within SSL model.

###### keywords:

self-supervised learning, bias, fairness

##  1 Introduction

Self-supervised learning (SSL) models have proven outstanding performance in
various tasks [1, 2, 3, 4, 5], also called foundation models. The SSL
framework includes two stages. The first stage pretrains a shared foundation
model model on many unlabeled data with pretext objectives. In the second
stage, the foundation models are adapted to downstream tasks supervised by
extracting meaningful representations or fine-tuning the whole model. However,
we observed SSL representation exhibiting bias and fairness problems that
propagate from pretraining to downstream tasks, raising concerns about
fairness.

Machine learning models may inadvertently reinforce stereotypes by associating
characteristics like emotion and hate more strongly with specific demographic
attributes of the speaker, such as age, gender, or accent [6]. For example, if
a model consistently associates anger with young speakers and sadness with
aged speakers, it could perpetuate stereotypes about emotional expression.
Rectifying bias in SSL models poses considerable challenges due to the
substantial economic and environmental costs involved [7].

Most previous works evaluate social bias by the parity of downstream task
performance between different social groups. Comparison of performance
differences between social groups for speech SSL models exists in Automated
Speech Recognition (ASR) [8, 9], Speech Translation (ST) [8], and Emotion
Recognition (ER) [10, 11]. Meng et al. [12] show that training data don't need
to be gender balanced to reach the best downstream performance, and pretrain
on data with lower speech rate has better downstream performance. There is
also work on assessing classification accuracy difference for binary gender on
ER models [13]. These works have studied downstream test performance
differences, neglecting bias in the SSL model representation. Juliette et al.
[14] demonstrated the self-supervised speech model's inclination towards the
language it was trained on within the representation space, and SpEAT [15]
revealed embedding-level bias between social groups in speech SSL models.
However, whether the SSL features are more biased than the acoustic features
is still unknown, and the contributing factors to social bias, such as model
architecture and size, have not been discussed in previous works.

While significant efforts have been devoted to improving the performance of
SSL models, there has been a notable lack of research on the bias present in
speech SSL models and the debiasing methods for social bias. Recognizing these
research gaps for speech, we aim to answer the following research questions:
(1) Do representations generated by SSL models amplify social bias? (2) Does
model size, pertaining step, and architecture affect social bias? (3) Research
in NLP shows a debiasing effect on compressed large language models [16, 17,
18]. Do model compression techniques affect social bias in speech SSL models?

To answer our research questions, we evaluate three well-known SSL models:
HuBERT [19], Wav2Vec2 [20], and MelHuBERT [21]. To quantify the social bias of
SSL model embedding in our research, we adopt SpEAT [15] to evaluate the
gender, age, and nationality bias.

Compared to prior works, our research provides more solid evidence of SSL
model's amplification of social bias and a more comprehensive exploration of
various factors that might affect social bias. Notably, we pioneer the
investigation into the efficacy of compression techniques in mitigating social
bias within speech SSL models. The following discoveries and insights emerge
from our investigation:

  * •

SSL model representation amplifies social bias compared to classic acoustic
features such as spectrogram or Mel-Frequency Cepstral Coefficients (MFCC).

  * •

Model size has little effect on social bias. Also, among models with equal
parameter counts, wider and shallower architectures typically exhibit reduced
social bias compared to narrower and deeper models.

  * •

Longer pretraining steps lead to higher social bias. Besides, models with
identical training objectives and architectural designs exhibit consistent
trends in bias during pretraining.

  * •

Among the compression methods, row pruning offers effective mitigation
strategies for all social bias in SSL speech models, and all compression
methods can reduce Age bias.

##  2 Experiment setup

###  2.1 Speech Self-supervised models

In recent years, speech self-supervised models have demonstrated strong
capabilities in extracting general features [22, 23]. Among those speech SSL
models, Wav2Vec 2.0 [20] discretizes the latent feature space and learns
contextualized representations through contrastive learning among masked time
stamps. HuBERT's [19] loss function is inherited from wav2vec 2.0, but it
moves the discretization process offline, using k-means clustering on either
MFCC features or hidden representations to generate discrete targets.
MelHuBERT [21] simplifies HuBERT's loss function to a simple cross-entropy
loss and removes the convolutional layer, using Mel Spectrogram as the input
directly, significantly reducing the computational cost of the model. Since
these three SSL models have similar model architectures, they are easy to
compare. In this paper, we will primarily focus on these three SSL models.

We train HuBERT, Wav2Vec2, and MelHuBERT models at smaller sizes in two
different architectures: small and slim, with nearly the same number of
parameters. Model details are shown in Table 2. To ensure alignment with [19],
HuBERT and Wav2Vec2 with configuration slim and small are trained using
fairseq [24] with a maximum batch size of 87.5 seconds of audio per GPU across
400k steps, while for MelHuBERT we prepare with batch size 12 and 421k steps
to ensure all models are trained with same amount of data instances. All
models are trained with 960 hours of LibriSpeech [25] data.

We also investigate models with larger sizes, including HuBERT
Base111https://dl.fbaipublicfiles.com/hubert/hubert_base_ls960.pt, Wav2Vec2
Base222https://dl.fbaipublicfiles.com/fairseq/wav2vec/wav2vec_small.pt,
Wav2Vec2
Large333https://dl.fbaipublicfiles.com/fairseq/wav2vec/libri960_big.pt,
MelHuBERT-20ms [26]. These models use the same dataset, LibriSpeech 960hr, for
pretraining.

###  2.2 Bias evaluation for Speech SSL models

We use SpEAT to evaluate bias in Speech SSL models. SpEAT evaluates the
alignment between two target concept embeddings, such as female ($X$) and male
($Y$), corresponding to its attribute concept embeddings, such as positive
($A$) and negative ($B$) valence. An effect size $d$ is calculated to show
which of the target concepts $X$ or $Y$ is closer to attribute $A$ than to
$B$. First, the difference of mean cosine similarity between a target
embedding $w$ and each embedding $a\in A$, $b\in B$ is calculated:

$\displaystyle s(w,A,B)$ $\displaystyle=mean_{a\in A}cos(w,a)-mean_{b\in
B}cos(w,b).$ (1)

And the effect size $d$ [27] is then calculated by:

$\displaystyle d$ $\displaystyle=\frac{mean_{x\in X}s(x,A,B)-mean_{y\in
Y}s(y,A,B)}{std\\_dev_{w\in X\cup Y}s(w,A,B)}.$ (2)

SpEAT $d$ values larger than 0.20, 0.50, and 0.80 are viewed as small, medium,
and large biases, respectively. Conversely, a negative $d$ value indicates a
reverse bias.

We follow the SpEAT setup closely, employing attribute concepts of positive
(A) and negative (B) valence. We turn to the Morgan Emotional Speech Set [28]
to gather speech samples, following the setup in SpEAT. Our target concepts
are drawn from the Speech Accent Archive [29], with gender, age, and
nationality filtered to create diverse bias categories. Table 1 provides an
overview of these categories. The embeddings for speech segments are
calculated by averaging the embeddings within each layer along time dimension,
followed by aggregating these averaged embeddings across all layers.

Table 1: Bias categories in our experiment. count represents the number of stimuli per target group. Category | target (X) | target (Y) | count
---|---|---|---
Gender | Female | Male | 60
Native | US | Korean | 55
Age | Young | Old | 58

00.5k1k2k3k4k6k8k10k20k30k40k50k60k70k80k100k120k160k200k240k280k320k360k400k$0$$0.5$$1$$1.5$SpEAT
$d$ (a) Gender

00.5k1k2k3k4k6k8k10k20k30k40k50k60k70k80k100k120k160k200k240k280k320k360k400k$-0.5$$0$$0.5$Wav2Vec2-slimHuBERT-
slimMelHuBERT-slimWav2Vec2-smallHuBERT-smallMelHuBERT-small (b) Native

00.5k1k2k3k4k6k8k10k20k30k40k50k60k70k80k100k120k160k200k240k280k320k360k400k$-0.5$$0$$0.5$
(c) Age

Figure 1: Model training steps versus SpEAT effective size.

###  2.3 Compression methods for Speech SSL models

There are many different ways to compress speech SSL models. In this work, we
follow Lin et al. [26], focusing on four basic compression methods: weight
pruning, head pruning, row pruning, and knowledge distillation. Besides
knowledge distillation, the other three methods can be considered iterative
pruning. In iterative pruning, we prune the network with two steps
iteratively. First, we prune blocks of weights and then perform finetuning on
the remaining weights. Based on the definition of a block, pruning can be
categorized into weight pruning, which is done at the level of individual
weights; head pruning, which targets attention heads; and row pruning, which
involves the rows and columns of the Feed-Forward Network (FFW).

Pruning: Among the three SSL models compared, HuBERT training has two stages;
stage 2 requires the acoustic unit clustered by the latent representation of
stage 1. Due to the unavailability of the stage-1 model and the resource
constraints preventing us from training a new HuBERT model, we opt not to
include HuBERT in the pruning experiments.

To perform head-pruning, we adopt a strategy of pruning the same number of
heads per layer based on their L1 norm, as higher layers typically exhibit
more significant L1 norms. This pruning occurs every 25k steps until 72 heads
remain, followed by further pruning every 40k steps until only 12 heads
remain. A learning rate of $5\times 10^{-5}$ is employed throughout this
process.

To implement row pruning, we target the rows of FFW1 and the columns of FFW2,
with a pruning objective of the sum of their L1 norms. Specifically, we prune
128 dimensions every 25,000 steps until 512 rows remain by a learning rate of
$10^{-5}$.

For weight pruning, we prune MelHuBERT first because the contrastive loss of
Wav2Vec2 cannot indicate convergence of the model. We conduct iterative
pruning on individual weights based on their L1 norm for all weights and
biases in the linear layers of Transformers. We maintain an exponential moving
average of the loss with a decay rate of 0.9998. Pruning is initiated if the
loss remains within 0.001 compared to the loss 15,000 steps prior. Our pruning
schedule varies in aggressiveness depending on the network's density: we prune
20% until 20% sparsity, 10% until 50% sparsity, 5% until 65% sparsity, 2.5%
until 70% sparsity, 1% until 80% sparsity. We use a learning rate of
$10^{-5}$. We recorded MelHuBERT's pruning schedule and applied it to
Wav2Vec2.

All the pruned models are trained with Adam optimizer [30], with a batch size
of 4 for MelHuBERT and 12 for Wav2Vec2, which aligns with previous works [26,
24].

Distillation: For distillation, we apply the same setup as DistilHubert [31]
in S3PRL [32]. We distill the models to 2, 4, and 6-layer student networks,
with an objective function to predict $4^{th}$ and $12^{th}$ layer of the
original SSL model's representation. We use a learning rate of $2\times
10^{-4}$ with Adam optimizer, batch size 24 for HuBERT, Wav2Vec2, and
MelHuBERT.

Table 2: Model details of MelHuBERT (Mel), HuBERT (HuB), and Wav2Vec2 (Wav)
architectures. $n_{l}$ is the number of transformer layers. $d_{m}$ is the
hidden size of encoder. $f_{s}$ is the dimensionality of the feed-forward
layer. $n_{h}$ is the number of heads.

architecture | configuration | parameters
---|---|---
$n_{l}$ | $d_{m}$ | $f_{s}$ | $n_{h}$ | Mel | Wav | HuB
Small | 3 | 640 | 2048 | 8 | 16.5M | 21.3M | 20.9M
Slim | 12 | 384 | 768 | 8 | 15.6M | 20.4M | 20.0M
Base | 12 | 768 | 3072 | 12 | 90.2M | 95.0M | 94.7M

GenderNativeAgeGender baselineNative baselineAge baseline (a)

1321201089684726048362412$0.5$$1$count of remaining headsSpEAT $d$ (a) Head
pruning

1321201089684726048362412$0$$0.5$$1$count of remaining heads (b) Head pruning

2560204815361024512$0$$0.5$$1$count of remaining rowsSpEAT $d$ (c) Row pruning

2560204815361024512$-0.5$$0$$0.5$count of remaining rows (d) Row pruning

00.20.30.40.50.60.70.8$0.5$$1$sparsitySpEAT $d$ (e) Weight pruning

00.20.30.40.50.60.70.8$0$$0.5$sparsity (f) Weight pruning

Figure 2: SpEAT $d$ versus parameters removed after applying 3 pruning methods
on Wav2Vec2 (a, c, e) and MelHuBERT (b, d, f). The closer to the right of the
graphs, the more parameters have been removed. The dashed lines are the SpEAT
$d$ measured on the unpruned model.

GenderNativeAge$0$$1$SpEAT $d$ (a) Wav2Vec2

GenderNativeAge$0$$0.5$$1$$1.5$246base (b) HuBERT

GenderNativeAge$0$$0.5$$1$$1.5$ (c) MelHuBERT

Figure 3: Effect of model distillation on (a) Wav2Vec2 (b) HuBERT (c)
MelHuBERT. 2, 4, 6 stands for the number of layers in the distilled model.
"base" stands for the model before distillation.

##  3 Result and Analysis

###  3.1 Effect of model architecture on bias

Table 3 illustrates that compared to traditional features, the features
extracted by SSL models have a more significant effect size on three bias
categories, indicating these models learned social bias from the training
process. MelHuBERT, for instance, trained on Mel spectrograms, exemplifies
increased bias. Notably, most SSL models have the highest SpEAT d on Gender
across all bias categories.

Moreover, an increase in model size doesn't necessarily correlate with
increased bias, contrasting with findings on compressing large language models
[16]. For example, among the Wav2Vec2 series, large model only has the
greatest SpEAT d on Native bias. In contrast, the greatest SpEAT d of Age and
Gender appears in base model. Additionally, although small models have
comparable parameters to slim models, they tend to have smaller effect sizes
overall, showing the effect of model architecture design on bias.

###  3.2 Effect of training steps on bias

Figure 1 shows that models with the same training objective and architecture
design show a similar trend in SpEAT $d$ during training. Moreover, biases
tend to be more pronounced in the later stages of pretraining. Upon random
initialization, MelHuBERT models consistently exhibit a significantly larger
SpEAT $d$ than other models. Regardless of the initial SpEAT $d$, convergence
to a positive value typically occurs within approximately 30k-60k steps. Among
the three bias categories analyzed, Gender emerges with the highest SpEAT $d$,
suggesting a more significant embedding-level bias.

###  3.3 Effect of compression on bias

To ensure the utility of compressed models, we first evaluate the upstream
models on downstream task Phoneme Recognition (PR). MelHuBERT uses a much
smaller batch size for pretraining, so the performance cannot be directly
compared with Wav2Vec2 and HuBERT base [33]. We follow the evaluation
configurations in the S3PRL toolkit. Table 4 shows all compressed models still
exhibit reasonable performance on downstream tasks, compared with [26].

Table 3: SpEAT effect size $d$ measured on speech SSL model representation or classical features with different sizes. The bold text denotes a model with less bias than any traditional features. Name | Age | Native | Gender
---|---|---|---
Wav2Vec2 slim | 0.49 | 0.52 | 0.85
Wav2Vec2 small | 0.40 | 0.31 | 0.07
Wav2Vec2 base | 0.51 | 0.21 | 1.21
Wav2Vec2 large | 0.46 | 0.69 | 0.58
HuBERT slim | 0.52 | 0.62 | 0.91
HuBERT small | 0.50 | 0.33 | 0.54
HuBERT base | 0.52 | 0.68 | 1.04
MelHuBERT slim | 0.44 | 0.55 | 1.33
MelHuBERT small | 0.14 | 0.57 | 0.92
MelHuBERT base | 0.16 | 0.62 | 0.92
STFT | 0.39 | -0.07 | -0.04
Spectrogram | -0.32 | -0.12 | 0.17
Mel Spectrogram | -0.32 | -0.07 | 0.03
MFCC | -0.05 | 0.13 | 0.50

####  3.3.1 Head pruning

We can see in Figure 2(a) and 2(b) that head pruning reduces SpEAT $d$ in
category Age, showing an apparent debias effect. However, for Native and
Gender bias, SpEAT $d$ fluctuates around the original model, suggesting a lack
of significant debiasing effect in these aspects.

####  3.3.2 Row pruning

In Figure 2(c) and 2(d), we observe a general reduction in SpEAT $d$ across
pruned models. Specifically, for the Gender bias in MelHuBERT and for all bias
categories in Wav2Vec2, there is a decreased bias. However, in the case of
Native and Age bias in MelHuBERT, the SpEAT d turns negative after pruning,
indicating the anti-bias characteristics in these categories at the onset of
pruning. Subsequently, SpEAT d gradually increases as more parameters are
pruned, leading to lower bias.

####  3.3.3 Weight pruning

Based on Figure 2(e) and 2(f), it's evident that weight pruning generally
lowers SpEAT $d$ in Age but shows a varied result across other bias categories
and models. Specifically for MelHuBERT, SpEAT $d$ initially decreases with
increasing sparsity but then rises afterward.

####  3.3.4 Distillation

Figure 3 vividly illustrates that model distillation does not universally
alleviate bias but sometimes increases bias. In the case of distilled
Wav2Vec2, we notice an increase in SpEAT $d$ for Gender and Native bias, yet a
decrease for most models on Age bias. Similarly, for distilled HuBERT, SpEAT
$d$ escalates for Gender but diminishes for Age and Native. Meanwhile, in the
case of distilled MelHuBERT, there's an increase in SpEAT $d$ for Gender while
a decrease is observed for Native and Age. Distillation generally increases
Gender bias while decreases Age bias. The distilled model size does not
significantly affect the embedding level bias measured.

Table 4: Downstream performance of the compressed models. dist. represent distillation. For pruned models, we report the performance of models with middle sparsity (72 heads for head pruning, 1536 rows for row pruning, sparsity 0.4 for weight pruning) and the sparsest model, separated by "/". For distilled models, we report the performance of the 2-layered model.  Model | Method | PR (PER$\downarrow$)
---|---|---
base | compressed
Wav2Vec2 | dist. | 5.7 | 21.3
head | 11.5 / 18.8
row | 8.9 / 17.3
weight | 6.6 / 8.9
HuBERT | dist. | 5.41 | 18.6
MelHuBERT | dist. | 8.2 | 27.2
head | 7.0 / 11.6
row | 8.8 / 14.1
weight | 8.1 / 8.7

##  4 Conclusion

This study offers critical insights into the impact of various factors within
speech SSL models on the embedding-level social bias. Our findings reveal that
SSL representation amplifies social bias compared to classic acoustic
features. When holding parameter counts constant, the small model exhibits
lower bias than the slim model. Longer pretraining steps lead to higher social
bias. Furthermore, row pruning demonstrates a capacity to reduce social bias
among the compression techniques examined, whereas weight pruning, head
pruning, and distillation show limited effectiveness. Lastly, all these
pruning methods effectively decrease Age bias in upstream models. Our research
serves as a roadmap toward developing more equitable and efficient SSL models.

##  5 Limitation

The analysis in our work focuses primarily on speaker-specific traits such as
$Gender$, $Native$, and $Age$, potentially overlooking other dimensions of
bias that could exist in semantic components of speech data such as political
ideology and religion. Also, our analysis focuses on models trained with
mainly English datasets. Further research is required on whether these results
can apply to other languages or multilingual settings.

##  6 Acknowledgement

We thank to National Center for High-performance Computing (NCHC) of National
Applied Research Laboratories (NARLabs) in Taiwan for providing computational
and storage resources.

## References

  * [1] A. T. Liu, S.-W. Li, and H.-y. Lee, ``Tera: Self-supervised learning of transformer encoder representation for speech,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 2021.
  * [2] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao _et al._ , ``Wavlm: Large-scale self-supervised pre-training for full stack speech processing,'' _IEEE Journal of Selected Topics in Signal Processing_ , 2022.
  * [3] M. Rivière, A. Joulin, P.-E. Mazaré, and E. Dupoux, ``Unsupervised pretraining transfers well across languages,'' in _Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020.
  * [4] A. Baevski, S. Schneider, and M. Auli, ``vq-wav2vec: Self-supervised learning of discrete speech representations,'' in _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_ , 2020.
  * [5] A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, ``Data2vec: A general framework for self-supervised learning in speech, vision and language,'' in _Proc. International Conference on Machine Learning (ICML)_ , 2022.
  * [6] R. Steed and A. Caliskan, ``Image representations learned with unsupervised pre-training contain human-like biases,'' in _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , 2021.
  * [7] R. Chen, J. Yang, H. Xiong, J. Bai, T. Hu, J. Hao, Y. FENG, J. T. Zhou, J. Wu, and Z. Liu, ``Fast model debias with machine unlearning,'' in _Advances in Neural Information Processing Systems_ , 2023.
  * [8] M. Z. Boito, L. Besacier, N. Tomashenko, and Y. Estève, ``A study of gender impact in self-supervised models for speech-to-text systems,'' 2022.
  * [9] P. Dheram, M. Ramakrishnan, A. Raju, I.-F. Chen, B. King, K. Powell, M. Saboowala, K. Shetty, and A. Stolcke, ``Toward fairness in speech recognition: Discovery and mitigation of performance disparities,'' _arXiv preprint arXiv:2207.11345_ , 2022.
  * [10] A. Derington, H. Wierstorf, A. Özkil, F. Eyben, F. Burkhardt, and B. W. Schuller, ``Testing speech emotion recognition machine learning models,'' 2023.
  * [11] J. Wagner, A. Triantafyllopoulos, H. Wierstorf, M. Schmitt, F. Burkhardt, F. Eyben, and B. W. Schuller, ``Dawn of the transformer era in speech emotion recognition: closing the valence gap,'' _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.
  * [12] Y. Meng, Y.-H. Chou, A. T. Liu, and H.-y. Lee, ``Don't speak too fast: The impact of data bias on self-supervised speech models,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022.
  * [13] C. Gorrostieta, R. Lotfian, K. Taylor, R. Brutti, and J. Kane, ``Gender De-Biasing in Speech Emotion Recognition,'' in _Proc. Interspeech 2019_ , 2019.
  * [14] J. Millet and E. Dunbar, ``Do self-supervised speech models develop human-like perception biases?'' in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2022.
  * [15] I. Slaughter, C. Greenberg, R. Schwartz, and A. Caliskan, ``Pre-trained speech processing models contain human-like biases that propagate to speech emotion recognition,'' in _The 2023 Conference on Empirical Methods in Natural Language Processing_ , 2023.
  * [16] G. Gonçalves and E. Strubell, ``Understanding the effect of model compression on social bias in large language models,'' in _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , 2023.
  * [17] G. Xu and Q. Hu, ``Can model compression improve nlp fairness,'' 2022.
  * [18] K. Ramesh, A. Chavan, S. Pandit, and S. Sitaram, ``A comparative study on the impact of model compression techniques on fairness in language models,'' in _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2023.
  * [19] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, ``Hubert: Self-supervised speech representation learning by masked prediction of hidden units,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 2021.
  * [20] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, ``wav2vec 2.0: A framework for self-supervised learning of speech representations,'' _Advances in neural information processing systems_ , vol. 33, pp. 12 449–12 460, 2020.
  * [21] T.-Q. Lin, H. yi Lee, and H. Tang, ``Melhubert: A simplified hubert on mel spectrograms,'' 2023.
  * [22] A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløe _et al._ , ``Self-supervised speech representation learning: A review,'' _IEEE Journal of Selected Topics in Signal Processing_ , 2022.
  * [23] E. Morais, R. Hoory, W. Zhu, I. Gat, M. Damasceno, and H. Aronowitz, ``Speech emotion recognition using self-supervised features,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022.
  * [24] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, ``fairseq: A fast, extensible toolkit for sequence modeling,'' _arXiv preprint arXiv:1904.01038_ , 2019.
  * [25] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, ``Librispeech: An ASR corpus based on public domain audio books,'' in _ICASSP 2015 - 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2015.
  * [26] T.-Q. Lin, T.-H. Yang, C.-Y. Chang, K.-M. Chen, T.-h. Feng, H.-y. Lee, and H. Tang, ``Compressing Transformer-based self-supervised models for speech processing,'' _arXiv preprint arXiv:2211.09949_ , 2022.
  * [27] J. Cohen, _Statistical power analysis for the behavioral sciences_. Academic press, 2013.
  * [28] S. D. Morgan, ``Categorical and dimensional ratings of emotional speech: Behavioral findings from the morgan emotional speech set,'' _Journal of Speech, Language, and Hearing Research_ , 2019.
  * [29] S. H. Weinberger and S. A. Kunath, ``The speech accent archive: towards a typology of english accents,'' in _Corpus-based studies in language use, language learning, and language documentation_ , 2011.
  * [30] D. P. Kingma and J. Ba, ``Adam: A method for stochastic optimization,'' _arXiv preprint arXiv:1412.6980_ , 2014.
  * [31] H.-J. Chang, S.-w. Yang, and H.-y. Lee, ``Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022.
  * [32] A. T. Liu, S.-w. Yang, P.-H. Chi, P.-c. Hsu, and H.-y. Lee, ``Mockingjay: Unsupervised speech representation learning with deep bidirectional transformer encoders,'' _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020.
  * [33] N. Vaessen and D. A. van Leeuwen, ``The effect of batch size on contrastive self-supervised speech representation learning,'' 2024.