# Single- and two-particle finite size effects in interacting lattice systems

Sergei Iskakov  Department of Physics, University of Michigan, Ann Arbor, MI
48109, USA  Hanna Terletska  Department of Physics and Astronomy, Middle
Tennessee State University, Murfreesboro, TN 37123, USA  Emanuel Gull
Department of Physics, University of Michigan, Ann Arbor, MI 48109, USA

(August 28, 2024)

###### Abstract

Simulations of extended quantum systems are typically performed by
extrapolating results of a sequence of finite-system-size simulations to the
thermodynamic limit. In the quantum Monte Carlo community, twist-averaging was
pioneered as an efficient strategy to eliminate one-body finite size effects.
In the dynamical mean field community, cluster generalizations of the
dynamical mean field theory were formulated to study systems with non-local
correlations. In this work, we put the twist-averaging and the dynamical
cluster approximation variant of the dynamical mean field theory onto equal
footing, discuss commonalities and differences, and compare results from both
techniques to the standard periodic boundary technique. At the example of
Hubbard-type models with local, short-range and Yukawa-like longer range
interactions we show that all methods converge to the same limit, but that the
convergence speed differs in practice. We show that embedding theories are an
effective tool for managing both one-body and two-body finite size effects, in
particular if interactions are averaged over twist angles.

##  I Introduction

Understanding the physics of periodic solids and periodic lattice model
systems requires describing their properties in the ‘thermodynamic’ or
macroscopic limit of infinite system size. Most numerical methods approach
this limit with results for a sequence of successively larger finite-size
systems, which are then extrapolated to the thermodynamic limit using a known
scaling behavior. Since the numerical effort grows either polynomially or
exponentially with system size, constructing reliable finite size
extrapolations from small systems is important.

Approximation errors may result from the truncation of the Hamiltonian to the
finite system and from the introduction of an artificial boundary or surface.
Periodic boundary conditions (PBC), which are the standard choice for the
simulation of periodic systems, eliminate boundary effects. Finite size
effects, which result from the truncation of the Hamiltonian and its solution
to the finite system (or, equivalently, to a discrete set of momentum points
in the Brillouin zone), remain. Two strategies to overcome such effects have
been proposed.

First, one may introduce a twist to the boundary conditions, resulting in a
shift of the corresponding momentum points [1, 2, 3, 4]. In the calculation of
local observables, simulations for multiple ‘twisted’ systems are then
averaged over twist angles. This ‘twist-averaged boundary condition’ (TABC)
technique is predominantly used in Monte Carlo calculations, since the average
can be taken during the simulation at no additional cost.

Second, one may construct a quantum embedding method. This is the rationale
behind cluster dynamical mean field methods [5, 6, 7, 8] such as the dynamical
cluster approximation (DCA) [5, 8], which constructs a periodic embedding. DCA
can be viewed as a generalization dynamical mean field theory (DMFT) to
‘patches’ in momentum space, where propagators, interactions, and correlations
are chosen to be identical in a patch in momentum space [9] but correlations
retain their full frequency dependence. The numerical effort of this technique
is typically $\sim 10\times$ the effort of the PBC solution, since the DCA
equations have to be solved self-consistently. In self-consistent diagrammatic
calculations, the DCA and the diagrammatic self-consistency can be performed
concurrently at no additional cost.

Twist-averaging and DCA embedding are remarkably similar. Both perform an
average over shifted momentum points, and both have the same asymptotic
convergence scaling, since for local observables and local interactions, all
of these strategies converge to the exact result with a quadratic convergence
in the linear system size but with different prefactors [5, 10] (faster
convergence may be observed under special circumstances [11]). They differ in
the place where the average is taken, which has consequences for the numerical
solution methods and for the prefactor of the convergence speed.

In this paper, we examine the convergence of these three strategies (PBC, DCA
and TABC). We show that, as expected, they converge to the same solution in
practice, and we examine convergence speed for different types of finite size
effects at the example of three different lattice models. In particular, we
examine the case of longer range interactions and highlight the importance of
averaging interactions in addition to the one-body Hamiltonian in such
systems.

The remainder of this paper is as follows. In Sec. II we introduce the models
studied, the finite size correction methods, and the solution methodology. In
Sec. III we present results, and Sec. IV contains conclusions.

##  II Model and Methods

###  II.1 Hubbard models with local and non-local interactions

We illustrate finite size effects at the example of the fermionic single-band
Hubbard-like Hamiltonian [12, 13] with non-local density-density interactions,

$\displaystyle
H=-t\sum_{\left<i,j\right>,\sigma}c^{\dagger}_{i,\sigma}c_{j,\sigma}+$
$\displaystyle\frac{1}{2}\sum_{i,j,\sigma\sigma^{\prime}}\tilde{U}_{ij}n_{i,\sigma}n_{j,\sigma^{\prime}}.$
(1)

$\sigma$ labels the spin index, $i$ and $j$ are site indices, and
$\left<i,j\right>$ denotes a sum over nearest neighbors.
$c^{\dagger}_{i,\sigma}$ ($c_{j,\sigma}$) are creation (annihilation)
operators and $n_{i,\sigma}=c^{\dagger}_{i,\sigma}c_{i,\sigma}$ describe the
particle number operators. $t$ denotes the nearest-neighbor hopping and
$\tilde{U}_{ij}$ the density-density interaction strength. We write
$H=H_{0}+V$, where $H_{0}$ denotes the quadratic hopping terms and $V$ denotes
the interactions. We restrict ourselves to the model on a three-dimensional
simple cubic lattice.

We separate the interaction into two parts,

$\displaystyle\tilde{U}_{ij}=U\delta_{ij}+V_{ij},$ (2)

where $U$ is the onsite Hubbard interaction and $V_{ij}$ the non-local part,
and discuss three different choices of $V_{ij}$.

For $V_{ij}\equiv 0$ we obtain the standard three-dimensional Hubbard model.
Due to its relevance to cold atomic gases [14] and a second-order phase
transition at high temperature, this model has become a fruitful testbed for
many-body methods and numerically exact results are available from several
reliable techniques [15, 16, 17, 10, 18, 19, 20, 21, 22], especially for
interactions near the maximal critical temperature.

For

$\displaystyle V_{ij}=V_{12}\delta_{\langle ij\rangle},$ (3)

with $V_{12}$ the nearest neighbor interaction strength and $\delta_{\langle
ij\rangle}=1$ if $i$ and $j$ are neighbors, and zero otherwise, we obtain the
extended Hubbard model. The nearest-neighbor interactions in this model induce
a charge-order phase transition which has been extensively studied with DCA
[23, 24, 25, 26], extended DMFT [27, 28, 29, 30], DMFT+GW [31, 32, 33, 34],
and dual boson techniques [35, 36, 37, 38, 39].

For

$\displaystyle
V_{ij}=V_{0}\frac{e^{-\alpha|\textbf{r}_{ij}|}}{|\textbf{r}_{ij}|}$ (4)

we obtain a Yukawa Hubbard model. $V_{0}=\frac{V_{12}}{e^{-\alpha}}$ denotes
the interaction strength, $\alpha$ the Yukawa decay constant, and
$\textbf{r}_{ij}=\textbf{r}_{i}-\textbf{r}_{j}$ the real space distance
between site $i$ and $j$. This model is chosen to mimic aspects of the long-
range interactions occurring in electronic structure problems without
introducing the complication of a divergent long-range potential.

In momentum space, we write

$\displaystyle\tilde{U}(\textbf{q})$ $\displaystyle=U+V(\textbf{q}),$ (5)
$\displaystyle V(\textbf{q})$
$\displaystyle=\sum_{i,j}V_{ij}e^{-i\textbf{q}\textbf{r}_{ij}},$ (6)

with q the bosonic momentum transfer vector. Note that general four-fermion
interactions, which we do not study in this work, depend on three momenta and
four unit cell indices.

We discuss results in the finite-temperature Matsubara formalism, where
single-particle quantities can be expressed in terms of Matsubara Green’s
functions, self-energies, and interaction vertices. Matsubara Green’s
functions are given as [40]

$\displaystyle
G(\omega_{n},\textbf{k})=\left[(i\omega_{n}+\mu)\mathbb{1}-F_{\textbf{k}}-{\Sigma}(\omega_{n},\textbf{k})\right]^{-1}.$
(7)

Here $\omega_{n}=(2n+1)\pi T$ denotes Matsubara frequencies at temperature
$T$, $\mu$ the chemical potential,
$F_{\textbf{k}}=H^{0}_{\textbf{k}}+\Sigma^{\infty}(\textbf{k})$ the Fock
matrix and $\Sigma$ the self-energy, which is uniquely defined if
$F_{\textbf{k}}$ is chosen such that $\Sigma(i\omega_{n}\rightarrow
i\infty)\rightarrow 0.$ The exact self-energy $\Sigma$ is a functional of the
momentum-dependent Green’s function and the momentum-dependent interaction
and, in diagrammatic theories, is obtained by adding all so-called skeleton
diagrams to infinite order [41, 40]. Approximations, such as the $GW$
approximation [42], can be constructed by considering a subset of all
diagrams.

###  II.2 Finite size effects

The approximation of an infinite periodic system by an auxiliary finite size
system introduces finite size approximations to the Hamiltonian $H=H_{0}+V$.
The approximation of the non-interacting Hamiltonian $H_{0}$ on a discrete set
of momentum points results in so-called ‘independent-particle’, ‘single-
particle’, or ‘one-body’ finite size effects [43, 44, 45, 46].

The situation for the interaction $V$ is more complex. The Hubbard interaction
is already constant in momentum space (local in real space) and therefore
remains exact when approximated on a finite system. The extended Hubbard
interaction is short-range and captured exactly on all but the smallest
systems. In contrast, the Yukawa interaction of Eq. 4 will become approximate
when truncated to the size of a finite system; the approximation leads to a
non-divergent but non-zero correction. In more general systems with long-
ranged interactions, such as those generally considered in the electronic
structure problem, the truncation of the interaction to a finite system may
lead to a divergent contribution which needs to be carefully compensated [47,
48, 43], and which may dominate the finite size convergence [49].

In the following, we denote continuous momenta in the infinite system by
$\textbf{k},\textbf{k}^{\prime},$ and q and discrete momenta in the finite
auxiliary system by $\textbf{K},\textbf{K}^{\prime}$, and Q,

A finite size approximation $G(\omega_{n},\textbf{k})\rightarrow
G(\omega_{n},\textbf{K})$ then introduces ‘one-body’ finite size errors via
the discretization of the $H^{(0)}_{\textbf{K}}$ contribution in the Fock
matrix on a finite number of K-points, along with approximations to the
interaction term. These approximations result in finite-size errors in the
self-energy contributions $\Sigma^{\infty}(\textbf{K})$ and
$\Sigma(\omega_{n},\textbf{K})$, which are generated from the approximations
of $H_{0}$ and $V$ in a highly non-linear manner.

Figure 1: Left panel: Scattering vertex, scattering an electron of momentum k
to momentum $\textbf{k}+\textbf{q}$ while absorbing momentum q. Right panel:
‘twisted’ version of the same vertex with twist $\zeta$, leaving the bosonic
momentum invariant.

###  II.3 Finite size corrections

####  II.3.1 Standard methodology

We start our discussion from local observables, such as the energy, the
double-occupancy or the density. These quantities can be expressed as
integrals over the Brillouin zone.

Most many-body methods are restricted to uniform momentum grids with regularly
spaced momenta $\textbf{K},\textbf{K}^{\prime}$ and Q, since all incoming and
outgoing momenta of a scattering vertex (see Fig. 1) have to lie on the
momentum grid. Integrals of smooth functions over such regularly spaced,
periodic grids with uniform weights converge quadratically in the number of
points in each dimension, leading to a quadratic convergence of Brillouin zone
integrals and thus to a quadratic convergence of local quantities. The
exception to this rule are long-range interactions with a singularity at the
$\Gamma$ point, see e.g. Refs. [47, 48, 43, 49].

In the standard methodology, simulations are performed on regularly spaced
Monkhorst-Pack grids [50] which, for Q, include the $\Gamma$ point with zero
momentum transfer. The sum $\textbf{K}+\textbf{Q}$ of a fermionic momentum K
and a bosonic momentum Q then lies on the fermionic grid, such that momentum
conservation can be respected.

Local quantities are evaluated as a sum over the discrete K-points. Momentum-
dependent quantities are only known at the discrete K points. If a more
precise momentum resolution is desired, additional momentum points have to be
obtained via interpolation. In this work, momentum-dependent quantities, such
as the momentum-dependent Matsubara Green’s function, are evaluated by
performing a Wannier interpolation [51] of the self-energy, from which the
Green’s function is recovered with an exactly evaluated non-interacting part
of the Hamiltonian. We note that interplations of the so-called cumulant [52]
and of the Green’s function are also possible, along with spline
interpolations of the self-energy [53]. The interpolation scheme can have a
large effect on the results if systems are small and self-energies are
strongly momentum dependent [54, 55, 56]. However, we find that, in the
systems analyzed here, results become independent of the interpolation scheme
as the system size is sufficiently increased.

One may also break some of the lattice symmetries, rather than performing
simulations on lattices that respect the full space-group symmetry of the
infinite system, and symmetrize results. Breaking symmetries gives access to
additional finite systems, which can then be used to perform finite size
extrapolations from a dense set of points. The technique was pioneered in the
context of exact diagonalization [57, 58] and is standard in cluster dynamical
mean field theory [8] (see Ref. [22] for detailed results for the systems
analyzed here).

####  II.3.2 Twisted boundary conditions and twist averaging

While the momenta $\textbf{K},\textbf{K}^{\prime}$ and Q have to be on
momentum grids with regularly spaced momenta, it is possible to shift the
‘fermionic’ momentum grids for K and $\textbf{K}^{\prime}$ by an arbitrary
momentum twist $\zeta$ while leaving the ‘bosonic’ one for Q invariant. The
difference of two fermionic momenta
$\textbf{K}+\zeta,\textbf{K}^{\prime}+\zeta$ then still lies on the bosonic
grid Q, and the sum of fermionic and bosonic momenta lies on the fermionic
grid, such that momentum conservation can be respected. The resulting shifted
scattering vertex is illustrated in the right panel of Fig. 1.

Momentum points can be shifted by introducing a phase factor or ‘twist’ into
the wave function or the hopping, and the technique of performing simulations
for many twists and averaging the results leads to the twist-averaged boundary
conditions [3, 4]. Twist averaging is commonly used in QMC simulations [43,
44, 59], where twists are either chosen randomly [4] or with uniformly spaced
twist angles [3].

Each twist $\zeta$ samples a different part of the one-body Hamiltonian
$H^{0}_{\textbf{K}+\zeta}$, while the two-body interaction $V(\textbf{q})$
remains approximated on the original Q-grid, $V(\textbf{Q})$. Observables are
evaluated for each twist angle, and the corresponding local quantities are
obtained by averaging over all twists. In Monte Carlo methods, the average
over twists is often obtained by sampling over $\zeta$, thereby incurring no
additional cost. In deterministic methods, or if momentum-dependent quantities
are desired, the numerical effort increases proportionally to the number of
twists.

In terms of a diagrammatic skeleton series, the relation between twisted
propagators $G^{t}$, self-energies $\Sigma^{t}$, and interactions is

$\displaystyle
G^{t}(\omega_{n},\textbf{K}+\zeta)=[(i\omega_{n}+\mu)\mathbb{1}-F_{\textbf{K}+\zeta}-\Sigma^{t}(\omega_{n},\textbf{K}+\zeta)]^{-1}$
(8)

where
$\Sigma^{t}(\omega_{n},\textbf{K}+\zeta)=\Sigma^{t}[G^{t}(\omega_{n},\textbf{K}+\zeta),U+V(\textbf{Q})]$
is a functional of the twisted interacting Green’s function for a given twist
$\zeta$ and the interactions evaluated at momentum Q.

Figure 2: From left to right. First panel: DCA approximation scheme in the
Brillouin zone of the 2D square lattice. An arbitrary lattice momentum k is
represented as the sum of a momentum $\zeta$ within a patch and a
corresponding cluster momentum K to the center of the patch. Second panel:
Scattering vertex in DCA with interactions $V^{c,a}$, scattering an electron
of momentum $\textbf{K}+\zeta$ to momentum
$\textbf{K}+\textbf{Q}+\zeta^{\prime}$ while absorbing a momentum
$\textbf{Q}+\zeta^{\prime\prime}$, violating momentum conservation on a scale
comparable to the patch size. Third panel: same, for interactions $V^{c,i}$
with momentum Q. Fourth panel: cluster scattering diagram, respecting cluster
momentum conservation. Straight lines denote lattice Green’s functions with
momenta indicated, wiggly lines momentum transfer, and vertices the
interaction vertex. Double lines denote cluster quantities.

####  II.3.3 Dynamical Cluster Approximation

The Dynamical Cluster Approximation [5, 8] was introduced as a cluster
generalization [5, 7, 6] of the single-site DMFT [60, 61, 62]. The method
simplifies to the single-site DMFT for system size $N_{c}=1$, becomes exact
for system size $N_{c}\rightarrow\infty$, and respects causal and conserving
properties [63]. We refer the reader to the original literature [5] and a
review [8] for a detailed derivation but repeat some of the important aspects
here.

DCA approximates the continuous lattice problem by an auxiliary discrete
cluster problem with momenta $\textbf{K},\textbf{K}^{\prime},\textbf{Q}$ and
modified cluster propagators $G^{c}(\omega_{n},\textbf{K})$, cluster self-
energies $\Sigma^{c}(\omega_{n},\textbf{K})$, and cluster interactions
$U+V^{c}(\textbf{Q})$. The momentum points of the discrete cluster are
embedded in the continuous Brillouin zone of the lattice, such that for every
lattice momentum k we write $\textbf{k}=\textbf{K}+\zeta$, with K a cluster
momentum point close to k (see left panel of Fig. 2). The regions over which
the average is taken, the so-called ‘momentum patches’, are centered around a
cluster momentum K and have equal size. The attribution of $k$-points to K
patches not unique [64, 65, 66].

The DCA propagators are given by the ‘coarse-grained’ patch averages

$\displaystyle
G^{c}(\omega_{n},\textbf{K})=\frac{1}{N}\sum_{\zeta}^{N}[(i\omega_{n}+\mu)\mathbb{1}-F_{\textbf{K}+\zeta}-\Sigma^{c}(\omega_{n},\textbf{K})]^{-1}$
(9)

the non-local DCA interactions are either left invariant,

$\displaystyle V^{c,i}(\textbf{Q})=V(\textbf{Q}),$ (10)

or obtained by coarse-graining the interaction over patches,

$\displaystyle
V^{c,a}(\textbf{Q})=\frac{1}{N}\sum_{\zeta}^{N}V(\textbf{Q}+\zeta).$ (11)

Both $V^{c,i}(\textbf{Q})$ and $V^{c,a}(\textbf{Q})$ have been used in
practice [67, 68, 24, 69]. The self-energies
$\Sigma^{c}(\omega_{n},\textbf{K})$ are obtained by evaluating a diagrammatic
skeleton series with propagators $G^{c}$ and interactions $U+V^{c,a/i}$ to
infinite order, which can be achieved via the solution of a quantum impurity
problem [62, 70]. Because Eq. 9 implicitly depends on $\Sigma^{c}$, the
quantum impurity problem needs to be solved self-consistently until
convergence.

As the cluster size is increased, the area over which $\zeta$ is averaged
decreases and the method converges to the solution of the infinite lattice
system. As in the case of periodic boundary conditions, local quantities, such
as the energy, converge quadratically in the linear extent of the system [8].

####  II.3.4 Commonalities and differences between DCA and twist-averaging

Twist-averaging and the DCA technique appear remarkably similar when written
in this form (Eqs. 8 and 9). Both rely on a summation over shifted momenta
$\zeta$ which accounts for strong variations in the one-body part of the
Hamiltonian, and both exhibit the same quadratic convergence to the infinite
system size limit. The major difference is in the place of the average. In
twist-averaging, every diagram is obtained for a definite twist angle $\zeta$.
In the DCA, the averaging takes place inside the propagator (and, in the case
of $V^{c,a}$, inside the interaction vertex). The system is then solved for
all twist angles at once, but the summation over twist angles leads to an
implicit dependence that requires the DCA to be solved self-consistently.

The implicit averaging over momentum patches in the Brillouin zone in Eq. 9
has a further consequence: momentum conservation is only guaranteed within
momentum patches located around K, rather than for each scattering vertex
individually. This is illustrated in the two middle panels of Fig. 2, where
DCA diagrams violate momentum conservation at each vertex within
$\zeta,\zeta^{\prime}$, and $\zeta^{\prime\prime}$. On the level of the
averaged cluster quantities, the momentum conservation is restored (right
panel of Fig. 2). Since the momentum conservation violation occurs on the
scale of the grid spacing, the approximation will scale to zero at the same
rate as the grid discretization errors as the system size is increased.

###  II.4 Solution of the finite system

The Hubbard model can be studied with numerically exact methods on moderate-
to-large systems [71]. However, since this work predominantly discusses finite
size methodologies, we limit this study to the fully self-consistent $GW$
method [42]. Results from GW are expected to show similar finite size behavior
as results from numerically exact methods but, because of the low polynomial
scaling [72], allow access to the large system sizes needed to rigorously show
finite size convergence. Where numerically exact results are available for
finite size extrapolations [10, 17, 73, 22], we have validated our conclusions
against these results.

The GW self-energy in the Matsubara frequency domain is

$\displaystyle\Sigma_{\sigma}(\omega_{n},\textbf{k})$
$\displaystyle=\Sigma_{\sigma}^{\infty}(\textbf{k})$
$\displaystyle-\frac{1}{\beta N_{k}}$
$\displaystyle\sum_{\textbf{q},m,\sigma^{\prime}}G_{\sigma^{\prime}}(\omega_{n}+\Omega_{m},\textbf{k}+\textbf{q})W(\Omega_{m},\textbf{q}),$
(12)

where $\beta$ is the inverse temperature, $\omega_{n}=(2n+1)\frac{\pi}{\beta}$
denotes fermionic Matsubara frequencies and $\Omega_{n}=2n\frac{\pi}{\beta}$
bosonic Matsubara frequencies. $W(\Omega_{m},\textbf{q})$ is the dynamical
screened interaction, $G(\omega_{n},\textbf{k})$ the Matsubara frequency
Green’s function, and the static part of the self-energy is

$\displaystyle\Sigma^{\infty}(\textbf{k})=\frac{1}{N_{k}}\sum_{\textbf{q},n}(U+V(\textbf{q}))\rho_{\textbf{k}+\textbf{q}},$
(13)

with $\rho_{\textbf{k}}=\frac{1}{\beta}\sum_{n}G(\omega_{n},\textbf{k})$ the
single-particle density matrix. Starting from the non-interacting Green’s
function:

$\displaystyle
G^{0}(\omega_{n},\textbf{k})=\left(i\omega_{n}+\mu-H^{0}_{\textbf{k}}\right)^{-1}$
(14)

we solve Eq. 12 self-consistently to obtain the GW approximation. This
requires solving the Dyson equation

$\displaystyle
G(\omega_{n},\textbf{k})=G^{0}(\omega_{n},\textbf{k})+G^{0}(\omega_{n},\textbf{k})\Sigma(\omega_{n},\textbf{k})G(\omega_{n},\textbf{k}).$
(15)

For a detailed description of the GW formalism used in this work see Ref.
[72].

Figure 3: Top left: Fermi surface of the non-interacting 3D Hubbard model at
half-filling in the simple cubic Brillouin zone, with special points $X$ and
$M$ indicated. Top right: Special points and standard path $\Gamma XM\Gamma
RX|RM$. Bottom: Density of the non-interacting model (orange, left axis) and
lowest Matsubara frequency of the real part of the Green’s function (blue,
right axis) at $\beta=6.25$ and $U=4$ along the full high-symmetry path. For
clarity the remainder of this paper shows momentum dependent quantities only
along the path from $X$ to $M$, indicated by a blue square.

Figure 4: Momentum dependence of the real part of the Green’s function at the
lowest Matsubara frequency for the Hubbard model at $U=4$ and $\beta=6.25$ for
different system sizes. Left panels: Results for clusters with periodic
boundary conditions. Right panels: DCA. Top row: Real part of the Green’s
function at lowest Matsubara frequency along a chosen region of the high
symmetry path. Bottom row: Imaginary part of the Green’s function at lowest
Matsubara frequency along a chosen region of the high symmetry path. Bottom
panels: Deviation from DCA results on a 12$\times$12$\times$12 lattice.
Symbols correspond to actual data points. Smooth momentum dependence is
obtained by Wannier interpolation.

##  III Results

We consider three different models in this paper: The 3D Hubbard model, the
extended Hubbard model and the extended Hubbard model with Yukawa
interactions. Each model is considered at a weak-to-intermediate interaction
strength where single-particle finite-size effects and correlation lengths are
expected to be large (see Ref. [22] for the Hubbard case).

Since much of this work discusses momentum-dependent quantities, we choose a
standard way of plotting a representative part of the momentum dependence,
rather than the full path in the Brillouin zone. The top left panel of Fig. 3
shows the Fermi surface of the half-filled 3D Hubbard model. The top right
panel illustrates the standard high symmetry path in the irreducible wedge,
along with a segment of the Brillouin zone, and the bottom panel shows the
density and the lowest frequency of the real part of the Green’s function
along this standard path. Interesting physics with large finite size
corrections occurs where the path intersects the Fermi surface, such as in the
segment from $X$ to $M$. We therefore choose $X$ to $M$ as a representative
path for display in the remainder of the paper (blue dashed section).

Figure 5: Imaginary part of the GW self-energy for the Hubbard model at
$U=4.0$ and $\beta=6.25$ between the $X$ and $M$ points, for system sizes
indicated. Left panels: Cluster with periodic boundary conditions. Right
panels: DCA. Bottom insets show the deviation from DCA results on a
12$\times$12$\times$12 lattice.

###  III.1 Local interaction: Hubbard model

We first analyze finite size effects in the half-filled Hubbard model in three
dimensions. Due to the locality of the interactions, all finite-size effects
originate from the one-body part of the Hamiltonian. Finite size effects
consist of one-body effects (or Brillouin zone approximation errors) and of
errors due to the truncation of the non-local part of the frequency-dependent
self-energy to the finite system. The Hartree contribution is an analytically
known constant at particle-hole symmetry; the Fock contribution is identically
zero. We also note that in addition to finite size errors there is a
substantial (and often dominant) method-specific $GW$ approximation error in
all the results we present in this section. As noted in Sec. II.4, we do not
discus this error here (see e.g. Refs. [22, 71]), focusing instead exclusively
on finite size effects.

As an example system, we choose a repulsive interaction strength of $U=4.0$
and set the overall hopping parameter to $t=1.0$. The temperature is
$T=1/\beta=1/6.25$, which puts the system into the vicinity of the second-
order phase transition for the GW method [22].

We briefly comment on the choice of interaction strength. For small $U$,
correlations are typically long range but weak. For large $U$, correlations
are often short-range but much stronger [12]. Thus, to analyze finite size
corrections, we choose an intermediate case of $U=4$ where finite size effects
are long range but large enough in magnitude to have a noticeable effect.

The main panel of Fig. 4 shows the momentum dependence of the Green’s function
at the lowest Matsubara frequency for systems of size 4$\times$4$\times$4,
6$\times$6$\times$6 and 8$\times$8$\times$8\. The bottom panel additionally
shows the difference to results obtained with DCA on a 12$\times$12$\times$12
lattice. The top panels show the real parts, the bottom ones the imaginary
parts. Results for periodic boundary conditions (PBC) are shown in the left
panels, those obtained with DCA in the right panels. Data points K are
indicated with symbols; lines are generated from interpolated points obtained
via Wannier interpolation. PBC results are mostly converged by size $8\times
8\times 8$. As shown in the right panels, DCA eliminates all system finite
size effects for all the system sizes considered.

Figure 6: Momentum dependence of the real (top) and imaginary (bottom) part of
the Green’s function at the lowest Matsubara frequency for the extended
Hubbard model at $U=4.0$, $V=0.2$ and $\beta=6.25$ in the GW approximation.
For system sizes indicated. Left panels: Clusters with periodic boundary
conditions. Right panels: DCA self-consistency. Bottom insets show the
deviation from the DCA results on 12$\times$12$\times$12 lattice. Symbols
correspond to actual data points. Smooth momentum dependence is obtained by
Wannier interpolation.

Figure 7: Imaginary part of the GW self-energy for the extended Hubbard model
at $U=4.0$, $V=0.2$ and $\beta=6.25$ at the half-way point between $X$ and
$M$, for system sizes indicated. Left panels: Clusters with periodic boundary
conditions. Right panels: DCA. Bottom insets show the deviation from DCA
results on a 12$\times$12$\times$12 lattice.

Fig. 5 shows the finite size convergence of the frequency-dependence of the
imaginary part of the self-energy. We choose a point halfway between $X$ and
$M$, lying on the non-interacting Fermi surface. As the high-frequency limit
for the self-energy converges to an analytically known [74] constant, all
finite size corrections occur at low frequency. At the lowest Matsubara
frequency, PBC self-energies show deviations on systems of size $4\times
4\times 4$ but converge as the system size is increased towards $8\times$
8$\times 8$. DCA results, due to complete elimination of the BZ discretization
error, converge for much smaller system size.

The almost complete absence of finite size errors in DCA can be understood as
follows. In the regime studied here, correlations are weak and almost all
finite-size effects in the GW solution of Hubbard model are due to the single-
particle finite size effects or discretization of $H_{0}$. The DCA average
over single-particle propagators eliminates these finite-size effects with its
summation over all momenta. Non-local self-energy contributions have a shorter
range than the smallest system size and do not result in visible finite size
effects. On the other hand, PBC results visibly suffer from single-particle
finite size effects.

###  III.2 Nearest-neighbor interaction: extended Hubbard model

We next consider the extended Hubbard model at intermediate local interaction
strength $U=6.0$ and nearest neighbor interaction $V_{12}=0.2$. Fig. 6 shows
the momentum dependence of the Green’s function at the lowest Matsubara
frequency. Both PBC (left panel) and DCA (right panel) show only small finite-
size effects at the fully occupied ($X$ point) and empty ($M$ point) regions
of the Brillouin zone. However, close to the partially occupied momentum
points, where correlation effects are expected to be most pronounced, PBC
solutions exhibit a much slower finite size convergence in comparison to DCA.
From the analysis of the Matsubara self-energy for a half-filled K-point (Fig.
7) we see that finite size errors are mostly restricted to the low-frequency
behavior of the dynamical self-energy.

#### Energetics and interaction coarse-graining

Finite size approximations to the Hamiltonian result in approximation errors
in the self-energy and the Green’s function, which propagate to finite size
errors in the energy. We calculate the correlation energy $\text{Tr}\Sigma G$,
with $\Sigma$ the dynamical part of the self-energy, and show relative errors
$\Delta$ in comparison to the DCA result $E_{\text{DCA}}^{L=12}$ on a
12$\times$12$\times$12 lattice, which we believe to be our most accurate
result. Table 1 shows $\Delta_{\text{PBC}}$ for naive PBC,
$\Delta_{\text{TABC}_{5}}$ for twist-averaging with $5\times 5\times 5=125$
and $\Delta_{\text{TABC}_{7}}$ for $7\times 7\times 7=343$ twists, and for the
two variants of DCA discussed in Sec. II.3.3: non-coarse-grained interactions
analogous to those used in the PBC and TABC methods, Eq. 10
($\Delta_{\text{DCA}_{i}}$), and coarse-grained interactions according to Eq.
11 ($\Delta_{\text{DCA}_{a}}$).

Lc |  $\Delta_{\text{PBC}}$ |  $\Delta_{\text{TABC}_{5}}$ |  $\Delta_{\text{TABC}_{7}}$ |  $\Delta_{\text{DCA}_{i}}$ |  $\Delta_{\text{DCA}_{a}}$
---|---|---|---|---|---
2 |  0.1796 |  0.1897 |  0.0810 |  0.0120 |  0.0106
4 |  0.0614 |  0.0016 |  0.0067 |  0.0007 |  0.0007
6 |  0.0040 |  0.0027 |  0.0017 |  0.0002 |  0.0002
8 |  0.0100 |  0.0004 |  0.0011 |  0.0001 |  0.0001
Table 1: Relative correlation energy error, $\Delta
E=|E-E^{DCA}_{Nc=12}|/E^{DCA}_{Nc=12}$, for the extended Hubbard model at
$U=4$, $V=0.2$ and $\beta=6.25$ for different methods and system sizes.

All finite size correction methods converge to the same result as a function
of system size. Twist-averaging is clearly beneficial and leads to errors that
are substantially smaller than those obtained with PBC on all but the smallest
system. These results clearly illustrate the power of twist-averaging for
energy calculations.

Little difference is evident between the two DCA methods, i.e. interaction
coarse-graining and no interacting coarse-graining. This is consistent with
previous results for the extended Hubbard model [68, 24, 69] and generally
expected for interactions that have little momentum dependence.

In addition to twist-avering over $5\times 5\times 5$ twists, we performed
simulations for $3\times 3\times 3$ (not shown) and $7\times 7\times 7$
twists. While differences between $3\times 3\times 3$ and $5\times 5\times 5$
twists are substantial, results only marginally change between $5\times
5\times 5$ and $7\times 7\times 7$ twists.

Figure 8: Momentum dependence of the real (top) and imaginary (bottom) part of
the Green’s function at lowest Matsubara frequency for the extended Hubbard
model with Yukawa interaction at $U=4.0$, $V_{0}=0.2$, $\alpha=0.25$ and
$\beta=6.25$, for system sizes indicated. Left panels: Clusters with periodic
boundary conditions. Right panels: DCA self-consistency. Bottom insets show
the deviation from DCA results on a 12$\times$12$\times$12 lattice. Symbols
correspond to actual data points. Smooth momentum dependence is obtained by
Wannier interpolation.

###  III.3 Longer-range interaction: Yukawa Hubbard model

The Yukawa-Hubbard model is chosen to mimic long-range interactions without a
divergent $1/r$ contribution. Unlike in the extended Hubbard model, the
interaction range may be larger than the finite size system considered.

To construct a model with interactions beyond the system size, we choose
$\alpha=0.25$ in Eq. 4. In this case, the interactions with the nearest
neighbor outside a 4$\times$4$\times$4 cluster is approximately $10\%$ of the
nearest-neighbor interaction. We then normalize the strength $V_{0}$ of the
Yukawa potential to obtain a nearest-neighbor interaction $V_{0}=0.2$,
comparable to one chosen for extended Hubbard model.

Figure 9:  Frequency dependence of the imaginary part of the local self-energy
for the extended Hubbard model with Yukawa interaction at $U=4.0$,
$V_{0}=0.2$, $\alpha=0.25$ and $\beta=6.25$, for system sizes indicated. Top
left: PBC. Rop right: DCA with invariant interactions. Bottom left: TABC (note
that the $8\times 8\times 8$ system with 125 twists is beyond our
capabilities). Bottom right: DCA with interaction coarse-graining. Bottom
insets show the deviation from DCA results on a 12$\times$12$\times$12
lattice.

As above, we consider the weak-to-intermediate interaction strength $U=4.0$
and $\beta=6.25$. Fig. 8 shows results for the Green’s function at the lowest
Matsubara frequency. The overall behavior is similar to the one in Fig. 6 for
the extended Hubbard model. DCA results converge quickly at low frequency.

Figure 10: Static GW self-energy for the extended Hubbard model with Yukawa
interaction at $U=4.0$, $V_{0}=0.2$, $\alpha=0.25$ and $\beta=6.25$ at the
$\Gamma$ point as a function of system size. Red open squares: Clusters with
periodic boundary conditions. Red filled squares: DCA with original
interactions. Orange filled squares: DCA with coarse-grained interactions.

Fig. 9 shows the frequency dependence of the local Matsubara self-energy. At
the lowest Matsubara frequency, finite size effects persist in PBC (top left
panel), while DCA with the original interactions (top right panel) achieves
fast system-size convergence. TABC results (bottom left panel) similarly show
fast convergence at low frequencies but deviations are evident at higher
frequencies. These high Matsubara frequency deviations are also evident in the
DCA results with original interactions. The high-frequency behavior of the
Yukawa Hubbard model with longer range interactions therefore differs
substantially from the extended Hubbard model with short-range interactions.

In the bottom right panel of Fig. 9, we show DCA simulations with coarse-
grained interactions (see Eq. 11). We average the interaction on 27 uniformly
spaced twist angles $\zeta$ for each K-patch (we cross-checked with $4\times
4\times 4$ twist angles at $U=4$ that $3\times 3\times 3$ were sufficient).
Interaction coarse-graining almost completely eliminates the high-frequency
finite size errors.

A high-frequency expansion of the self-energy [75, 76, 74] shows that this
high-frequency behavior is directly related to the strength of an effective
interaction [74]. The slow convergence at high frequency may therefore be
viewed as a consequence of the approximation to the interaction. To estimate
the finite-size convergence of these interaction finite size effects, we
examine the first moment of the self-energy high frequency expansion [75, 76,
74],

$\displaystyle\Sigma^{(1)}_{ii}=\sum_{kl}U_{ik}U_{ki}(\langle
n_{k}n_{l}\rangle-\langle n_{k}\rangle\langle n_{l}\rangle),$ (16)

in Table 2, as a function of system-size. Without interaction correction, PBC,
TABC and DCA converge slowly with system-size. Coarse-graining (or twist-
averaging) the interaction in the DCA coarse-graining procedure significantly
improves this behavior.

Lc | $\Sigma_{1}$
---|---
|  $PBC$ |  $TABC$ |  $DCA_{1}$ |  $DCA_{2}$
4 |  10.55 |  10.18 |  11.71 |  7.78
6 |  8.37 |  8.37 |  8.59 |  7.70
8 |  7.91 |  7.91 |  8.00 |  7.68
10 |  7.78 |  |  7.80 |  7.67
Table 2: First high-frequency moment of the GW self-energy for the Yukawa-
Hubbard model at $U=4$, $V_{0}=0.2$, $\alpha=0.25$, and $\beta=6.25$ for
various system sizes.

The static part of the self-energy also directly depends on interactions in
the system. Fig. 10 shows convergence of the static part of GW self-energy at
the $\Gamma$ point. There is a fast convergence of the DCA with coarse-grained
interactions.

Table 3 shows the convergence of the correlation energy for all four methods.
For system sizes where interaction finite-size effects are not important, DCA
with invariant lattice interactions achieves faster convergence than PBC.
However, on small systems, i.e. in the presence of the strong interaction
finite-size effects, coarse-graining is essential.

The twist-averaged correlation energy converges very fast (we also show data
for $7\times 7\times 7$ twists). This can be understood from the fact that the
correlation energy is dominated by the low-frequency behavior of the self-
energy (high-frequency terms are suppressed by the multiplication with the
Green’s function, which scales $\propto 1/\omega_{n}$). This low-energy self-
energy is captured well by the twisted simulation (Fig. 9).

As the system-size is increased, truncation errors disappear due to the
exponential decay of the Yukawa interaction.

Lc |  $\Delta_{\text{PBC}}$ |  $\Delta_{\text{TABC}_{5}}$ |  $\Delta_{\text{TABC}_{7}}$ |  $\Delta_{\text{DCA}_{i}}$ |  $\Delta_{\text{DCA}_{a}}$
---|---|---|---|---|---
4 |  0.134 |  0.0167 |  0.0015 |  0.134 |  0.0112
6 |  0.0026 |  0.0021 |  0.0035 |  0.0194 |  0.0028
8 |  0.0082 |  0.0019 |  0.00012 |  0.0064 |  0.0019
10 |  0.0062 |  |  |  0.0015 |  0.000004
Table 3: Relative correlation energy error for the Yukawa-Hubbard model at
$U=4$, $V_{0}=0.2$, $\alpha=0.25$, and $\beta=6.25$ for systems of size 4-10.

##  IV Discussion and Conclusions

How should one approach the thermodynamic limit with a sequence of finite-size
systems? This paper discussed two established techniques which were developed
in different communities: the technique of twist-averaging boundary
conditions, and the dynamical cluster approximation variant of cluster DMFT.

We find that TABC and DCA can be put on a similar theoretical footing, since
both attempt to average over areas of the Brillouin zone that would not be
considered in a naive Brillouin zone sampling. However, the precise way of
performing the average is different, as is the numerical effort. Twist-
averaging is particularly efficient in Monte Carlo simulations, where the
average can be taken at no additional cost during the Monte Carlo sampling.
DCA is particularly efficient in semi-analytic self-consistent methods such as
GW, since the DCA self-consistency can be converged during the diagrammatic
self-consistency at no additional cost.

We showed that TABC provides excellent energy estimates for interactions that
are contained within the finite size system. We also showed that averaging the
interactions, as is done in certain flavors of DCA, is essential if their
range extends beyond the system size. Both techniques achieve far faster
system size convergence than standard PBC.

Cluster DMFT results are often interpreted in terms of the historic connection
to the infinite coordination number limit taken in single-site DMFT. Given the
parallels between cluster DMFT and periodic lattice calculations, we find that
it is more convenient to discuss large-cluster DCA results in terms of finite-
size-corrected lattice calculations, rather than in terms of ‘mean field’
methods. This point of view has also been taken in numerous previous works,
including the large-cluster studies of two-dimensional superconductivity [77,
78] and three-dimensional Hubbard model physics [16, 79, 17], and is the main
reason for the success of the method in explaining the physics of the
pseudogap in the two-dimensional Hubbard model [53, 80, 81, 64], for which
recent twist-averaged calculations on small clusters [82] show remarkably
similar results.

Our results suggest that averaging longer-range interactions is a promising
route for controlling finite size effects, and that this technique should be
attempted for long range interactions, such as the Coulomb interactions
occurring in electronic structure theory.

###### Acknowledgements.

This work is supported by Simons Foundation via the Simons Collaboration on
the Many Electron Problem. H. T. has been supported by NSF DMR-1944974 grant.
This research used resources of the National Energy Research Scientific
Computing Center, a DOE Office of Science User Facility supported by the
Office of Science of the U.S. Department of Energy under Contract No. DE-
AC02-05CH11231 using NERSC award BES-ERCAP0020359.

## References

  * Valentí  _et al._ [1991] R. Valentí, C. Gros, P. J. Hirschfeld, and W. Stephan, Phys. Rev. B 44, 13203 (1991).
  * Gros [1992] C. Gros, Zeitschrift fü”r Physik B Condensed Matter 86, 359 (1992).
  * Gros [1996] C. Gros, Phys. Rev. B 53, 6865 (1996).
  * Lin  _et al._ [2001] C. Lin, F. H. Zong, and D. M. Ceperley, Phys. Rev. E 64, 016702 (2001).
  * Hettler  _et al._ [1998] M. H. Hettler, A. N. Tahvildar-Zadeh, M. Jarrell, T. Pruschke, and H. R. Krishnamurthy, Phys. Rev. B 58, R7475 (1998).
  * Lichtenstein and Katsnelson [2000] A. I. Lichtenstein and M. I. Katsnelson, Phys. Rev. B 62, R9283 (2000).
  * Kotliar  _et al._ [2001] G. Kotliar, S. Y. Savrasov, G. Pálsson, and G. Biroli, Phys. Rev. Lett. 87, 186401 (2001).
  * Maier  _et al._ [2005a] T. A. Maier, M. Jarrell, T. Prushke, and M. Hettler, Rev. Mod. Phys. 77, 1027 (2005a).
  * Fuhrmann  _et al._ [2007] A. Fuhrmann, S. Okamoto, H. Monien, and A. J. Millis, Phys. Rev. B 75, 205118 (2007).
  * Fuchs  _et al._ [2011a] S. Fuchs, E. Gull, M. Troyer, M. Jarrell, and T. Pruschke, Phys. Rev. B 83, 235113 (2011a).
  * Biroli and Kotliar [2002] G. Biroli and G. Kotliar, Phys. Rev. B 65, 155112 (2002).
  * Qin  _et al._ [2022] M. Qin, T. Schäfer, S. Andergassen, P. Corboz, and E. Gull, Annual Review of Condensed Matter Physics 13, 275 (2022), https://doi.org/10.1146/annurev-conmatphys-090921-033948 .
  * Arovas  _et al._ [2022] D. P. Arovas, E. Berg, S. A. Kivelson, and S. Raghu, Annual Review of Condensed Matter Physics 13, 239 (2022), https://doi.org/10.1146/annurev-conmatphys-031620-102024 .
  * Hart  _et al._ [2015] R. A. Hart, P. M. Duarte, T.-L. Yang, X. Liu, T. Paiva, E. Khatami, R. T. Scalettar, N. Trivedi, D. A. Huse, and R. G. Hulet, Nature 519, 211 (2015).
  * Staudt, R.  _et al._ [2000] Staudt, R., Dzierzawa, M., and Muramatsu, A., Eur. Phys. J. B 17, 411 (2000).
  * Kent  _et al._ [2005] P. R. C. Kent, M. Jarrell, T. A. Maier, and T. Pruschke, Phys. Rev. B 72, 060411(R) (2005).
  * Fuchs  _et al._ [2011b] S. Fuchs, E. Gull, L. Pollet, E. Burovski, E. Kozik, T. Pruschke, and M. Troyer, Phys. Rev. Lett. 106, 030401 (2011b).
  * De Leo  _et al._ [2011] L. De Leo, J.-S. Bernier, C. Kollath, A. Georges, and V. W. Scarola, Phys. Rev. A 83, 023606 (2011).
  * Paiva  _et al._ [2011] T. Paiva, Y. L. Loh, M. Randeria, R. T. Scalettar, and N. Trivedi, Phys. Rev. Lett. 107, 086401 (2011).
  * Kozik  _et al._ [2013] E. Kozik, E. Burovski, V. W. Scarola, and M. Troyer, Phys. Rev. B 87, 205102 (2013).
  * Duarte  _et al._ [2015] P. M. Duarte, R. A. Hart, T.-L. Yang, X. Liu, T. Paiva, E. Khatami, R. T. Scalettar, N. Trivedi, and R. G. Hulet, Phys. Rev. Lett. 114, 070403 (2015).
  * Iskakov and Gull [2022] S. Iskakov and E. Gull, Phys. Rev. B 105, 045109 (2022).
  * Terletska  _et al._ [2017] H. Terletska, T. Chen, and E. Gull, Phys. Rev. B 95, 115149 (2017).
  * Terletska  _et al._ [2018] H. Terletska, T. Chen, J. Paki, and E. Gull, Phys. Rev. B 97, 115117 (2018).
  * Paki  _et al._ [2019] J. Paki, H. Terletska, S. Iskakov, and E. Gull, Phys. Rev. B 99, 245146 (2019).
  * Terletska  _et al._ [2021] H. Terletska, S. Iskakov, T. Maier, and E. Gull, Phys. Rev. B 104, 085129 (2021).
  * Chitra and Kotliar [2000] R. Chitra and G. Kotliar, Phys. Rev. Lett. 84, 3678 (2000).
  * Camjayi  _et al._ [2008] A. Camjayi, K. Haule, V. Dobrosavljević, and G. Kotliar, Nature Physics 4, 932 (2008).
  * Amaricci  _et al._ [2010] A. Amaricci, A. Camjayi, K. Haule, G. Kotliar, D. Tanasković, and V. Dobrosavljević, Phys. Rev. B 82, 155102 (2010).
  * Kapcia  _et al._ [2017] K. J. Kapcia, S. Robaszkiewicz, M. Capone, and A. Amaricci, Phys. Rev. B 95, 125112 (2017).
  * Sun and Kotliar [2002] P. Sun and G. Kotliar, Phys. Rev. B 66, 085120 (2002).
  * Ayral  _et al._ [2012] T. Ayral, P. Werner, and S. Biermann, Phys. Rev. Lett. 109, 226401 (2012).
  * Ayral  _et al._ [2013] T. Ayral, S. Biermann, and P. Werner, Phys. Rev. B 87, 125149 (2013).
  * Huang  _et al._ [2014] L. Huang, T. Ayral, S. Biermann, and P. Werner, Phys. Rev. B 90, 195114 (2014).
  * Rubtsov  _et al._ [2012] A. Rubtsov, M. Katsnelson, and A. Lichtenstein, Annals of Physics 327, 1320 (2012).
  * van Loon  _et al._ [2014] E. G. C. P. van Loon, A. I. Lichtenstein, M. I. Katsnelson, O. Parcollet, and H. Hafermann, Phys. Rev. B 90, 235135 (2014).
  * Stepanov  _et al._ [2016] E. A. Stepanov, E. G. C. P. van Loon, A. A. Katanin, A. I. Lichtenstein, M. I. Katsnelson, and A. N. Rubtsov, Phys. Rev. B 93, 045107 (2016).
  * Peters  _et al._ [2019] L. Peters, E. G. C. P. van Loon, A. N. Rubtsov, A. I. Lichtenstein, M. I. Katsnelson, and E. A. Stepanov, Phys. Rev. B 100, 165128 (2019).
  * Vandelli  _et al._ [2020] M. Vandelli, V. Harkov, E. A. Stepanov, J. Gukelberger, E. Kozik, A. Rubio, and A. I. Lichtenstein, Phys. Rev. B 102, 195109 (2020).
  * Mahan [2000] G. Mahan,  _Many-Particle Physics_, Physics of Solids and Liquids (Springer US, 2000).
  * Luttinger and Ward [1960] J. M. Luttinger and J. C. Ward, Phys. Rev. 118, 1417 (1960).
  * Hedin [1965] L. Hedin, Phys. Rev. 139, A796 (1965).
  * Kent  _et al._ [1999] P. R. C. Kent, R. Q. Hood, A. J. Williamson, R. J. Needs, W. M. C. Foulkes, and G. Rajagopal, Phys. Rev. B 59, 1917 (1999).
  * Drummond  _et al._ [2008] N. D. Drummond, R. J. Needs, A. Sorouri, and W. M. C. Foulkes, Phys. Rev. B 78, 125106 (2008).
  * Azadi and Foulkes [2015] S. Azadi and W. M. C. Foulkes, The Journal of Chemical Physics 143, 102807 (2015).
  * Dagrada  _et al._ [2016] M. Dagrada, S. Karakuzu, V. L. Vildosola, M. Casula, and S. Sorella, Phys. Rev. B 94, 245108 (2016).
  * Fraser  _et al._ [1996] L. M. Fraser, W. M. C. Foulkes, G. Rajagopal, R. J. Needs, S. D. Kenny, and A. J. Williamson, Phys. Rev. B 53, 1814 (1996).
  * Williamson  _et al._ [1997] A. J. Williamson, G. Rajagopal, R. J. Needs, L. M. Fraser, W. M. C. Foulkes, Y. Wang, and M.-Y. Chou, Phys. Rev. B 55, R4851 (1997).
  * Wilhelm and Hutter [2017] J. Wilhelm and J. Hutter, Phys. Rev. B 95, 235123 (2017).
  * Monkhorst and Pack [1976] H. J. Monkhorst and J. D. Pack, Phys. Rev. B 13, 5188 (1976).
  * Marzari  _et al._ [2012] N. Marzari, A. A. Mostofi, J. R. Yates, I. Souza, and D. Vanderbilt, Rev. Mod. Phys. 84, 1419 (2012).
  * Stanescu and Kotliar [2006] T. D. Stanescu and G. Kotliar, Phys. Rev. B 74, 125110 (2006).
  * Macridin  _et al._ [2006] A. Macridin, M. Jarrell, T. Maier, P. R. C. Kent, and E. D’Azevedo, Phys. Rev. Lett. 97, 036401 (2006).
  * Sakai  _et al._ [2009a] S. Sakai, Y. Motome, and M. Imada, Physica B: Condensed Matter 404, 3183 (2009a).
  * Sakai  _et al._ [2009b] S. Sakai, Y. Motome, and M. Imada, Phys. Rev. Lett. 102, 056404 (2009b).
  * Sakai  _et al._ [2012] S. Sakai, G. Sangiovanni, M. Civelli, Y. Motome, K. Held, and M. Imada, Phys. Rev. B 85, 035102 (2012).
  * Betts and Stewart [1997] D. D. Betts and G. E. Stewart, Canadian Journal of Physics 75, 47 (1997).
  * Betts  _et al._ [1999] D. D. Betts, H. Q. Lin, and J. S. Flynn, Canadian Journal of Physics 77, 353 (1999).
  * Spink  _et al._ [2013] G. G. Spink, R. J. Needs, and N. D. Drummond, Phys. Rev. B 88, 085121 (2013).
  * Georges  _et al._ [1996] A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Rev. Mod. Phys. 68, 13 (1996).
  * Metzner and Vollhardt [1989] W. Metzner and D. Vollhardt, Phys. Rev. Lett. 62, 324 (1989).
  * Georges and Kotliar [1992] A. Georges and G. Kotliar, Phys. Rev. B 45, 6479 (1992).
  * Hettler  _et al._ [2000] M. H. Hettler, M. Mukherjee, M. Jarrell, and H. R. Krishnamurthy, Phys. Rev. B 61, 12739 (2000).
  * Gull  _et al._ [2010] E. Gull, M. Ferrero, O. Parcollet, A. Georges, and A. J. Millis, Phys. Rev. B 82, 155101 (2010).
  * Ferrero  _et al._ [2010] M. Ferrero, O. Parcollet, A. Georges, G. Kotliar, and D. N. Basov, Phys. Rev. B 82, 054502 (2010).
  * Staar  _et al._ [2016] P. Staar, M. Jiang, U. R. Hähner, T. C. Schulthess, and T. A. Maier, Phys. Rev. B 93, 165144 (2016).
  * Arita  _et al._ [2004] R. Arita, S. Onari, K. Kuroki, and H. Aoki, Phys. Rev. Lett. 92, 247006 (2004).
  * Wu and Tremblay [2014] W. Wu and A.-M. S. Tremblay, Phys. Rev. B 89, 205128 (2014).
  * Jiang  _et al._ [2018] M. Jiang, U. R. Hähner, T. C. Schulthess, and T. A. Maier, Phys. Rev. B 97, 184507 (2018).
  * Zgid and Gull [2017] D. Zgid and E. Gull, New Journal of Physics 19, 023047 (2017).
  * LeBlanc  _et al._ [2015] J. P. F. LeBlanc, A. E. Antipov, F. Becca, I. W. Bulik, G. K.-L. Chan, C.-M. Chung, Y. Deng, M. Ferrero, T. M. Henderson, C. A. Jiménez-Hoyos, E. Kozik, X.-W. Liu, A. J. Millis, N. V. Prokof’ev, M. Qin, G. E. Scuseria, H. Shi, B. V. Svistunov, L. F. Tocchio, I. S. Tupitsyn, S. R. White, S. Zhang, B.-X. Zheng, Z. Zhu, and E. Gull (Simons Collaboration on the Many-Electron Problem), Phys. Rev. X 5, 041041 (2015).
  * Yeh  _et al._ [2022] C.-N. Yeh, S. Iskakov, D. Zgid, and E. Gull, Fully self-consistent finite-temperature $gw$ in gaussian bloch orbitals for solids (2022).
  * Gull  _et al._ [2011a] E. Gull, P. Staar, S. Fuchs, P. Nukala, M. S. Summers, T. Pruschke, T. C. Schulthess, and T. Maier, Phys. Rev. B 83, 075122 (2011a).
  * Rusakov  _et al._ [2014] A. A. Rusakov, J. J. Phillips, and D. Zgid, J. Chem. Phys. 141, 194105 (2014).
  * Nolting [1972] W. Nolting, Z. Phys. Hadrons Nucl. 255, 25 (1972).
  * Knecht [2006] C. Knecht,  _Numerical and analytical approaches to strongly correlated electron systems_ , Ph.D. thesis, Johannes Gutenberg-Universität in Mainz (2006).
  * Maier  _et al._ [2005b] T. A. Maier, M. Jarrell, T. C. Schulthess, P. R. C. Kent, and J. B. White, Phys. Rev. Lett. 95, 237001 (2005b).
  * Gull  _et al._ [2013] E. Gull, O. Parcollet, and A. J. Millis, Phys. Rev. Lett. 110, 216405 (2013).
  * Gull  _et al._ [2011b] E. Gull, P. Staar, S. Fuchs, P. Nukala, M. S. Summers, T. Pruschke, T. C. Schulthess, and T. Maier, Phys. Rev. B 83, 075122 (2011b).
  * Werner  _et al._ [2009] P. Werner, E. Gull, O. Parcollet, and A. J. Millis, Phys. Rev. B 80, 045120 (2009).
  * Gull  _et al._ [2009] E. Gull, O. Parcollet, P. Werner, and A. J. Millis, Phys. Rev. B 80, 245102 (2009).
  * Huang [2020] E. W. Huang, Strong-coupling mechanism of the pseudogap in small hubbard clusters (2020).


# Libri-Adapt: A new Speech Dataset for Unsupervised Domain Adaptation

###### Abstract

This paper introduces a new dataset, Libri-Adapt, to support unsupervised
domain adaptation research on speech recognition models. Built on top of the
LibriSpeech corpus, Libri-Adapt contains 7200 hours of English speech recorded
on mobile and embedded-scale microphones, and spans 72 different domains that
are representative of the challenging practical scenarios encountered by ASR
models. More specifically, Libri-Adapt facilitates the study of domain shifts
in ASR models caused by a) different acoustic environments, b) variations in
speaker accents, c) previously unexplored factors such as heterogeneity in the
hardware and platform software of the microphones, and d) a combination of the
aforementioned three shifts. We also provide a number of baseline results
quantifying the impact of these domain shifts on the Mozilla DeepSpeech2 ASR
model.

Index Terms—  Unsupervised Domain Adaptation, Deep Learning, Microphone
Variability, Speech Recognition

##  1 Introduction

Recent advancements in deep neural networks have led to significant
improvements in the performance of automatic speech recognition (ASR).
However, it is also known that minor deviations between the training and test
domains (known as _domain shift_) can significantly degrade the
generalizability of deep learning models [1]. In the case of speech
recognition, such perturbations could be introduced in the data by several
real-world factors including ambient noise [2], variations in acoustic
environments [3, 4], accents of speakers [5], or even the microphones used to
record the speech [6, 7] – and they represent a major challenge to the
generalizability of ASR models in unconstrained settings. Clearly, the
collection of labeled training data across all these heterogeneous scenarios
is prohibitively expensive, therefore _unsupervised domain adaptation_ has
emerged as a promising technique towards adapting deep neural networks to
different but related domains, even in the absence of labeled data from the
target domain [8].

There are plethora of datasets to study and develop unsupervised domain
adaptation techniques for visual recognition (e.g., [9, 1, 10]), however in
speech recognition research, there is still a lack of large-scale realistic
datasets which contains examples of various forms of _domain shifts_ that are
expected in real-world scenarios. In this work, we present a new dataset named
_Libri-Adapt_ , primarily developed to study unsupervised domain adaptation,
which contains domain shifts caused by a) variability in recording
microphones, b) variability in speaker accents and c) variability in acoustic
environments. Libri-Adapt is mainly recorded on consumer-grade, embedded-scale
microphones (please refer Table 1) that are becoming popular due to their
easy-integration capabilities with microcomputers such as the Raspberry Pi to
enable developers to create speech applications for wearable and embedded
devices.

Libri-Adapt is built on top of the Librispeech-clean-100 dataset [11] which
contains 100 hours of US English speech from users reading public-domain audio
books. In Libri-Adapt, we record the Librispeech-clean-100 training corpus on
six different microphones (five are embedded-scale or smartphone microphones
and one is a Shure microphone to establish a higher-quality baseline), in
three different accents (US English, British English, and Indian English) and
under four synthesized background noise conditions (Clean, Rain, Wind,
Laughter). As such, Libri-Adapt has a total of 7,200 hours (6 microphones x 3
accents x 4 environments x 100 hours) of data for training ASR models and can
be used to study domain adaptation for potentially 72 different domains (6
microphones x 3 accents x 4 environments). For each domain, we also collect a
5.4 hour held-out test set derived from Librispeech test-clean corpus [11].

In §3, we describe the methodology used for collecting the dataset followed by
an overview of how the dataset is structured. In §4, we demonstrate the
potential of doing unsupervised domain adaptation research on this dataset by
training a Mozilla DeepSpeech2 [12] ASR model on each domain in the dataset
and testing it on another domains. Our results show that even state-of-the-art
ASR models such as DeepSpeech2 suffer accuracy degradation due to domain
shifts – e.g., the word error rate of a model trained on the Shure microphone
increases from 10.19% (when it is tested on the same microphone) to 24.95%
when it is tested on a Matrix Voice microphone.

In summary, Libri-Adapt is the largest public dataset targeted for consumer-
grade embedded-scale microphones to facilitate the study and development of
robust ASR models using domain adaptation techniques. The dataset is released
publicly under the CC BY 4.0
license111https://creativecommons.org/licenses/by/4.0/ and can be accessed
from https://bit.ly/2UK1McG222Updated baseline results with a newer release of
DeepSpeech2 model (0.8.2) are also available at this URL..

##  2 Related Work

Our work follows a rich tradition of making speech data public to support
research on speech and audio recognition. For example, Gemmeke et al. [13]
released AudioSet at ICASSP 2017 for the task of audio event recognition.
Similarly, a number of speech and keyword datasets have been released,
including LibriSpeech [11], Common Voices [14] and Speech Commands [15].
Researchers have also developed simulated datasets focused on the major
challenges in ASR, e.g., handling reverberations [16, 17], noisy speech [2],
speech separation and distant ASR [17, 2]. A prominent example has been the
CHiME challenge [2, 3, 4] which has released datasets containing real and
simulated multi-channel speech data in noisy real-world conditions as well as
in multi-speaker scenarios. For the purpose of studying accent variability,
prior works have used the Speecon [5] family of speech corpora recorded in
multiple languages.

To the best of our knowledge, no prior work has developed a large-scale speech
corpus for embedded-scale microphones (refer Table 1 for details) that are
gaining popularity nowadays. Further, recent research [6] has shown that
embedded microphones exhibit significant differences in their outputs owing to
the variations in their hardware and on-board signal processing pipelines. As
such, the primary goal of this paper is to collect speech datasets – from
embedded-scale microphones – that are large enough to support training of deep
ASR models. The variability across microphones can be interpreted as a form of
domain shift, thus allowing for development and evaluation of unsupervised
domain adaptation algorithms on this dataset. In addition, we are also
interested in understanding how microphone variations interact with other
challenges for ASR, namely variations in speaker accents and acoustic
environments – as such, we record speech data from 3 speaker accents in 4
acoustic environments on 6 different embedded microphones.

Unsupervised Domain Adaptation (UDA). At a high-level, the idea behind UDA is
as follows: given a pre-trained deep classifier for a source domain and an
unlabeled dataset from a target domain, how can we adapt the weights of the
source model such that it shows better performance in the target domain. This
class of algorithms has been used recently to adapt ASR models from clean to
noisy speech, for speech dereverberation and for speaker gender adaptation
[18, 19, 20]. Our work aims to advance this line of research by introducing
the large-scale multi-domain Libri-Adapt dataset.

##  3 Methodology and Dataset Overview

In this section, we describe the different domains in the Libri-Adapt dataset
and the methodology adopted to create them.

Microphones. One of the important goals for Libri-Adapt is to support ASR
research on consumer-grade embedded-scale microphones. In the past few years,
a number of embedded-scale microphones (both single- and multi-channel) have
been released which can be used in conjunction with microcomputers such as
Raspberry Pi and cloud-based speech models to develop custom speech analysis
systems [21]. However, research [6] has shown that these microphones could
potentially have significant variations in their hardware and software
processing pipelines, which in turn could induce domain shift in the speech
data obtained from them. Therefore, in order to develop robust ASR systems, we
believe it is critical to collect speech datasets from various microphones and
evaluate the generalizability of ASR models across them. In this vein, we
collect training data (derived from Librispeech-clean-100) from six different
microphones as listed in Table 1. Matrix
Voice333https://www.matrix.one/products/voice and
ReSpeaker444https://respeaker.io/usb_6+1_mic_array/ are circular 7-channel
microphone arrays that include on-device acoustic processing algorithms for
de-reverberation and delay-and-sum beamforming to combine the outputs of the
different channels. PlayStation-Eye is a digital camera used in Sony
PlayStation gaming consoles and consists of a 4-channel microphone array used
for speech user interaction with the gaming console. Next, we use a Google
Nexus 6 as a representative smartphone microphone with which users are likely
to interact while using speech applications. Further, we use a single-channel
USB microphone as an example of low-end microphones popularly used with
Raspberry Pi. Finally, to establish a higher-quality microphone baseline, we
employ a Shure MV5 Condenser microphone555https://www.shure.com/en-
GB/products/microphones/mv5/ which can also be used as a plug-and-play
microphone with mobile and wearable devices.

We use a Raspberry Pi as the host device for Matrix, ReSpeaker, USB and
PlayStation Eye, while the Shure microphone uses a Macbook Pro as the host
device. Each microphone in the dataset can be considered as a unique domain,
which allows researchers to perform a variety of domain adaptation
experiments.

Device | Channels |  | Maximum
---
Sampling
Rate (kHz)
Advertised Signal Processing Capabilities
| Matrix
---
Voice
7 | 8 to 96 |  Beamforming, dereverberation, noise cancellation
Respeaker | 7 | 8 to 48 |  Voice activity detection, beamforming, background noise suppression
| PlayStation
---
Eye
4 | 8 to 48 |  Voice location tracking, echo cancellation, background noise suppression.
USB Mic | 1 | 8 to 48 |  -
Google Nexus 6 | 3 | 8 to 48 |  -
Shure MV5 | 1 | 8 to 48 |  Auto gain, Equalization
Table 1: Technical specification of the microphones used in the Libri-Adapt
dataset.

Speaker Accents. Libri-Adapt consists of speech data in three English accents,
namely US English, British English and Indian English. To facilitate domain
adaptation experiments for countering the domain shift due to variations in
speaker accents, we made a design decision to create accented speech that
share the same underlying text corpus. In other words, the accented speech
subsets have exactly the same sentences being spoken, but in different English
accents. For US English, we use the original Librispeech-clean-100 dataset
that was recorded with 251 US speakers. To create its variants in British
English and Indian English accents, we employed the text-to-speech (TTS) API
666https://cloud.google.com/text-to-speech/ from Google Cloud Platform which
uses a pretrained WaveNet model to generate accented speech in multiple
voices. More specifically, we passed the transcripts of the Librispeech-
clean-100 corpus to the Google TTS API along with the target accent (British
and Indian) and obtained two accented versions of the Librispeech-clean-100
dataset, while ensuring that the underlying spoken sentences remain same in
each accent.

Data Recording Setup. We adopted the methodology of replay-and-record to
collect the Libri-Adapt dataset. Each accented dataset (US English, British
English and Indian English) was played on a JBL LSR305 monitor speaker
connected to a laptop, and recorded simultaneously on all six microphones in a
quiet room. The microphones were kept at a distance of 15cm from the speaker.
To ensure that the recordings are time-synchronized across microphones, we
implemented a network-based synchronization scheme based on MQTT [22]
messaging protocol. Before a speech file is played, the laptop sends a MQTT
message to the recording device hosting each microphone to initiate an audio
recording process for the duration of the speech file. Thereafter, the speech
file is played on the JBL LSR305 monitor speaker and is recorded
simultaneously by all six microphones. After the recording completes, each
device sends a confirmation message to the laptop, which proceeds to play the
next file in the dataset.

Indeed, this approach of replay-and-record has its pros and cons. It allows
for collecting large-scale datasets (recall that Libri-Adapt has 7200 hours of
data) with various domain shifts cheaper than recruiting human subjects –
however on the downside, as the speech is replayed through a speaker, it is
impacted by the transfer function of the speaker. To counter this issue, we
use a high-quality monitor speaker with a reasonably flat frequency response
to replay the audios, which ensures that there is minimal impact on the replay
audio quality due to the speaker. More importantly, we argue that the replayed
dataset still allows for studying the domain shift problem caused by different
microphones because the replay process could be interpreted as a form of
covariate shift on the input distribution, that is shared across all
microphones.

Ambient Noise Augmentation. Finally, we augment the speech datasets collected
from all microphones with three types of background noise, namely ‘Wind’,
‘Rain’ and ‘Laughter’. To this end, we use the audio samples for ‘Wind’,
‘Rain’ and ‘Laughter’ from the Environmental Sound Classification (ESC-50)
[23] dataset and record them on each of the microphone following our replay-
and-record methodology. In total, we record 20 noise samples for each noise
type on each microphone. Finally, to create the noisy variants of the dataset,
each speech file in the original (Clean) dataset is overlaid with a randomly
picked noise sample from the same microphone (at SNR of 20dB). As such, we end
up with 4 background noise conditions in the dataset: Clean, Rain, Wind,
Laughter.

Dataset Overview. Table 2 provides an overview of the Libri-Adapt dataset. In
total, the dataset has 6 microphone domains, 3 accent domains and 4 background
noise domains, resulting in a total of 6 x 3 x 4 = 72 domains. Each domain
consists of 28,540 speech files (sampling rate 16KHz) with a duration of 100.6
hours as labeled training data, and 2600 speech files with a duration of 5.4
hours as labeled test data. The dataset facilitates a wide variety of domain
adaptation experiments, both supervised and unsupervised. For example, one
could train an ASR model with {_US English_ speakers on a _Nexus 6_ microphone
in _Clean_ background noise} and test its performance on {_Indian English_
speakers using a _ReSpeaker_ microphone with _Rain_ background noise}.

Domain Types | Count |  Domain Names
---|---|---
Microphones | 6 |  Matrix Voice, ReSpeaker, PlayStation Eye, USB, Shure, Nexus 6
Accents | 3 |  US English, Indian English, British English
Background Noise | 4 |  Clean (or None), Rain, Wind, Laughter
Table 2: Overview of the Libri-Adapt dataset

##  4 Effect of Domain Shift on ASR Models

In this section, we analyze the Libri-Adapt dataset to demonstrate its
feasibility in supporting unsupervised domain adaptation research for ASR
models. To this end, we perform a number of experiments where we train an ASR
model on a particular domain and evaluate its performance on another domain.
More specifically, we use the Mozilla DeepSpeech2 pre-trained model [12]
(release 0.5.0) as the base model which has a low word error rate (WER) of
8.22% on the LibriSpeech dataset. We fine-tune the base model on the training
data from each domain and thereafter, the fine-tuned model for each domain is
tested on the held-out test set from other domains to compute the WER.

WER across Microphones. First, we analyze the impact of domain shift caused by
microphone variability on the accuracy of ASR models. Table 3 shows the WER of
the DeepSpeech2 model when it is fine-tuned for different ‘training’
microphones (in columns) and tested on other microphones (in rows). As
expected, in most cases, it can be observed that when data from the same
microphone is used for training and testing the model, it achieves the
smallest WER, for e.g., 11.39% in the case of PlayStation Eye. However, in
cases of domain shift caused by microphone mismatch, the WER increases to
24.18% when the PlayStation Eye model is tested on Matrix Voice. We also note
that the WER obtained when models from other microphones are tested on Matrix
Voice is significantly high (above 20% in all cases), however when the model
is fine-tuned on Matrix Voice, we can achieve a reasonable WER of 13.05%. This
highlights the opportunity for domain adaptation algorithms to adapt or align
the speech feature representations across microphones when ASR models are
deployed in scenarios with microphone mismatch.

| Training Microphones
---|---
|  | Matrix
---
Voice
ReSpeaker | USB | Shule | Nexus 6 |  | PlayStation
---
Eye
| Matrix
---
Voice
13.05 | 28.0 | 20.64 | 24.95 | 26.21 | 24.18
ReSpeaker | 16.06 | 12.66 | 14.49 | 16.09 | 15.93 | 15.21
USB | 13.30 | 14.44 | 11.40 | 14.41 | 15.39 | 15.38
Shule | 12.58 | 13.10 | 11.61 | 10.19 | 12.96 | 13.07
Nexus 6 | 13.09 | 12.81 | 11.99 | 12.73 | 12.27 | 15.28
| PlayStation
---
Eye
14.01 | 13.65 | 14.24 | 16.16 | 16.52 | 11.39
Table 3: WER of a fine-tuned DeepSpeech2 model trained and tested on various
microphone pairs for Clean, US English speech. The columns correspond to the
training microphone domain and rows correspond to the test microphone domain.

WER across Accents. Next, we evaluate the impact of accent changes on ASR
models. For this experiment, we set the microphone domain to _ReSpeaker_ and
background noise to _Clean_ , which enables us to isolate the impact of accent
variability on the model. As shown in Figure 1, there is a significant impact
on WER caused by variations in speaker accents – in particular, the
generalizability of US and British English accent models to Indian English
accent is quite poor, for instance, the WER of a model trained on US accent
increases from 11.4% to 29.5% when it is tested on data with an Indian accent.
Interestingly, we also observe that the WER of the British and Indian accent
models on their own domains is very low ($\sim$5%) as compared to the US
accent model (11.4%) – a possible explanation of this result is that British
and Indian accent datasets were generated by the Google WaveNet TTS model
which supports a limited number (5-6) of user voices. As such, it was easier
to fine-tune the model for these accents, as compared to US accent which had a
total of 251 unique users. From a domain adaptation perspective, this
represents a practical, albeit challenging, scenario: how to adapt a model
trained on a small user group to a larger user group with different accents.

Fig. 1: WER of a fine-tuned DeepSpeech2 model trained and tested on various accent datasets recorded on a _ReSpeaker_ microphone in a _Clean_ background. | Training Domains
---|---
| Clean | Rain | Wind | Laughter
Clean | 11.70 | 19.07 | 16.22 | 13.84
Rain | 30.32 | 21.63 | 23.45 | 23.56
Wind | 17.20 | 22.35 | 14.62 | 15.34
Laughter | 22.14 | 21.93 | 21.13 | 14.52
Table 4: WER of a fine-tuned DeepSpeech2 model trained and tested on various
background noise domains.

WER across Background Noise. In Table 4, we show the cross-domain results when
ASR models are trained and tested on different background noise pairs. In this
experiment, we set the microphone domain to USB and accent to US English,
which helps in isolating the effect of background noise on model performance.
The results in Table 4 confirm that domain shifts caused by ambient noise can
degrade the performance of state-of-the-art ASR models, hence making it a
pertinent problem for domain adaptation research.

Studying multi-domain shifts. So far, we have analyzed the effect of a single
form of domain shift on ASR models. However, Libri-Adapt can also be utilized
to study the effect of multiple forms of domain shifts on ASR models. As shown
in Table 5, we can combine microphones, accents and noise conditions in
various ways to form a domain; and then evaluate the cross-domain
generalizability of ASR models. For instance, we observe in Table 5 that when
a model trained on US accented clean speech collected on a ReSpeaker
microphone (_Domain 1_) is tested on Indian accented speech with Rain
background noise recorded on a PlayStation Eye (_Domain 2_), it suffers a WER
increase of nearly 29.8%. In summary, Libri-Adapt allows researchers to create
various complex adaptation scenarios to test the generalizability of their
domain adaptation algorithms.

| Training Domains
---|---
| Domain 1 | Domain 2 | Domain 3 | Domain 4
Domain 1 | 12.86 | 47.01 | 37.23 | 15.64
Domain 2 | 42.68 | 6.37 | 59.25 | 43.05
Domain 3 | 27.09 | 42.73 | 4.80 | 25.14
Domain 4 | 19.07 | 52.64 | 46.18 | 14.26
Table 5: Libri-Adapt facilitates a variety of domain adaptation experiments by
combining microphones, accents and noise conditions. This table shows the WER
for various domain adaptation tasks. _Domain 1_ = ReSpeaker-US-Clean; _Domain
2_ = PlayStationEye-Indian-Rain; _Domain 3_ = Shule-British-Laughter; _Domain
4_ = USB-US-Wind

##  5 Conclusion

We presented a speech dataset to facilitate the study and development of
unsupervised domain adaptation algorithms for ASR models. The dataset, derived
from the Librispeech corpus, contains 7,200 hours of training data from 72
different domains spanning across 6 microphones, 3 speaker accents and 4
background noise conditions. One limitation of this work is that in order to
create a large-scale and multi-domain dataset without incurring significant
costs, we adopted the methodology of replay-and-record which may introduce
minor speaker-related effects in the dataset. As a future work, we will
develop unsupervised domain adaptation algorithms to tackle the domain shifts
in the dataset. The Libri-Adapt dataset is released publicly under the CC BY
4.0 license and can be accessed from https://bit.ly/2UK1McG. Updated baseline
results with a newer release of DeepSpeech2 model (0.8.2) are also available
at this URL.

## References

  * [1] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell,  “Adversarial discriminative domain adaptation,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7167–7176.
  * [2] Jon Barker, Ricard Marxer, Emmanuel Vincent, and Shinji Watanabe,  “The third ‘chime’speech separation and recognition challenge: Dataset, task and baselines,”  in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE, 2015, pp. 504–511.
  * [3] Emmanuel Vincent, Shinji Watanabe, Aditya Arie Nugraha, Jon Barker, and Ricard Marxer,  “An analysis of environment, microphone and data simulation mismatches in robust speech recognition,”  Computer Speech & Language, vol. 46, pp. 535–557, 2017.
  * [4] Jon Barker, Shinji Watanabe, Emmanuel Vincent, and Jan Trmal,  “The fifth’chime’speech separation and recognition challenge: Dataset, task and baselines,”  in Interspeech, 2018.
  * [5] Dorota Iskra, Beate Grosskopf, Krzysztof Marasek, H Heuvel, Frank Diehl, and Andreas Kiessling,  “Speecon-speech databases for consumer devices: Database specification and validation,”  2002\.
  * [6] Akhil Mathur, Anton Isopoussu, Fahim Kawsar, Nadia Berthouze, and Nicholas D. Lane,  “Mic2mic: Using cycle-consistent generative adversarial networks to overcome microphone variability in speech systems,”  in Proceedings of the 18th International Conference on Information Processing in Sensor Networks, New York, NY, USA, 2019, IPSN ’19, pp. 169–180, ACM.
  * [7] Akhil Mathur, Anton Isopoussu, Fahim Kawsar, Robert Smith, Nicholas D. Lane, and Nadia Berthouze,  “On robustness of cloud speech apis: An early characterization,”  in Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, New York, NY, USA, 2018, UbiComp ’18, p. 1409–1413, Association for Computing Machinery.
  * [8] Hal Daume III and Daniel Marcu,  “Domain adaptation for statistical classifiers,”  Journal of artificial Intelligence research, vol. 26, 2006.
  * [9] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell,  “Cycada: Cycle-consistent adversarial domain adaptation,”  in International Conference on Machine Learning, 2018, pp. 1994–2003.
  * [10] Akhil Mathur, Anton Isopoussu, Fahim Kawsar, Nadia Berthouze, and Nicholas D. Lane,  “Flexadapt: Flexible cycle-consistent adversarial domain adaptation,”  2019\.
  * [11] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur,  “Librispeech: an asr corpus based on public domain audio books,”  in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 5206–5210.
  * [12] “Pre-trained Mozilla DeepSpeech2 ASR model,” https://github.com/mozilla/DeepSpeech/releases/tag/v0.5.0, 2019.
  * [13] Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter,  “Audio set: An ontology and human-labeled dataset for audio events,”  in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017, pp. 776–780.
  * [14] “Mozilla Voice,” https://voice.mozilla.org/en, 2019,  Accessed: .
  * [15] Pete Warden,  “Speech commands: A dataset for limited-vocabulary speech recognition,”  arXiv preprint arXiv:1804.03209, 2018.
  * [16] Keisuke Kinoshita, Marc Delcroix, Takuya Yoshioka, Tomohiro Nakatani, Emanuël Habets, Reinhold Haeb-Umbach, Volker Leutnant, Armin Sehr, Walter Kellermann, Roland Maas, et al.,  “The reverb challenge: A common evaluation framework for dereverberation and recognition of reverberant speech,”  in 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. IEEE, 2013, pp. 1–4.
  * [17] Mirco Ravanelli, Piergiorgio Svaizer, and Maurizio Omologo,  “Realistic multi-microphone data simulation for distant speech recognition,”  in Interspeech 2016, 2016.
  * [18] Daniel Michelsanti and Zheng-Hua Tan,  “Conditional generative adversarial networks for speech enhancement and noise-robust speaker verification,”  Proc. Interspeech 2017, pp. 2008–2012, 2017.
  * [19] Ehsan Hosseini-Asl, Yingbo Zhou, Caiming Xiong, and Richard Socher,  “A multi-discriminator cyclegan for unsupervised non-parallel speech domain adaptation,”  arXiv preprint arXiv:1804.00522, 2018.
  * [20] Ke Wang, Junbo Zhang, Sining Sun, Yujun Wang, Fei Xiang, and Lei Xie,  “Investigating generative adversarial networks based speech dereverberation for robust speech recognition,”  arXiv preprint arXiv:1803.10132, 2018.
  * [21] “DIY hardware to emulate Amazon Echo,” http://www.instructables.com/id/Build-DIY-Amazon-Alexa-With-a-MATRIX-Creator-on-Ha/, 2015\.
  * [22] “MQTT Messaging Protocol,” http://mqtt.org/, 2019.
  * [23] Karol J Piczak,  “Esc: Dataset for environmental sound classification,”  in Proceedings of the 23rd ACM international conference on Multimedia. ACM, 2015, pp. 1015–1018.