# RemixIT: Continual self-training of speech enhancement models via
bootstrapped remixing

Efthymios Tzinis , Yossi Adi , Vamsi K. Ithapu ,
Buye Xu , Paris Smaragdis , Anurag Kumar
E. Tzinis is with the University of Illinois at Urbana-Champaign, Urbana,
Illinois, USA (email: etzinis2@illinois.edu). Part of the work was done while
E. Tzinis was an intern at Meta Reality Labs Research. Y. Adi is with Meta AI
Research, Tel Aviv, Israel (email: adiyoss@fb.com). V. Ithapu and B.Xu are
with Meta Reality Labs Research, Redmond, WA, USA (email: ithapu, xub@fb.com).
P. Smaragdis is with the University of Illinois at Urbana-Champaign, Urbana,
Illinois, USA (email: paris@illinois.edu). A. Kumar is with Meta Reality Labs
Research, Redmond, WA, USA (email: anuragkr@fb.com).

###### Abstract

We present RemixIT, a simple yet effective self-supervised method for training
speech enhancement without the need of a single isolated in-domain speech nor
a noise waveform. Our approach overcomes limitations of previous methods which
make them dependent on clean in-domain target signals and thus, sensitive to
any domain mismatch between train and test samples. RemixIT is based on a
continuous self-training scheme in which a pre-trained teacher model on out-
of-domain data infers estimated pseudo-target signals for in-domain mixtures.
Then, by permuting the estimated clean and noise signals and remixing them
together, we generate a new set of bootstrapped mixtures and corresponding
pseudo-targets which are used to train the student network. Vice-versa, the
teacher periodically refines its estimates using the updated parameters of the
latest student models. Experimental results on multiple speech enhancement
datasets and tasks not only show the superiority of our method over prior
approaches but also showcase that RemixIT can be combined with any separation
model as well as be applied towards any semi-supervised and unsupervised
domain adaptation task. Our analysis, paired with empirical evidence, sheds
light on the inside functioning of our self-training scheme wherein the
student model keeps obtaining better performance while observing severely
degraded pseudo-targets.

###### Index Terms:

Self-supervised learning, speech enhancement, semi-supervised self-training,
zero-shot domain adaptation.

††publicationid: pubid: 0000–0000/00$00.00 © 2022 IEEE

##  I Introduction

One of the most fundamental problems in audio processing is speech
enhancement, where the goal is to isolate and reconstruct the clean speech
component from a noisy input recording [1]. Several studies have shown that
employing such denoising models as front-ends is key for building robust
automatic speech recognition (ASR) systems [2, 3, 4]. The universal
applicability of neural networks has proven to be beneficial for a variety of
signal processing problems, including speech enhancement. Sophisticated
architectures such as convolutional networks [5, 6, 7, 8], recurrent
processing [9] self-attention [10, 11, 12], generative adversarial networks
[13, 14] as well as variational auto-encoders [15], to name a few. Despite the
effectiveness of the aforementioned approaches in cases where large amounts of
in-domain training paired data are available, real-world applications
necessitate the need for developing robust self-supervised learning (SSL)
algorithms to train these models with in-the-wild mixtures.

Recent studies have shown that speech representations could be self-learned
and be used later for other downstream audio processing tasks [16, 17, 18].
However, in real-world settings, the speech recordings are degraded with
additive noise, thus, self-learning robust embeddings becomes particularly
challenging and demands the adaptation to the input noise distribution [19].
Several unsupervised speech denoising algorithms have been proposed by
identifying and training with relatively clean segments of the noisy speech
mixture [20, 21], using ASR losses [22, 23] exploiting visual cues [24], and
harnessing the spatial separability of the sources using mic-arrays [25, 26].
Mixture invariant training (MixIT) [27] enables unsupervised training of
separation models only with real-world single-channel recordings by generating
artificial mixtures of mixtures and estimating the independent sources.
Although MixIT has been proven successful for various speech enhancement tasks
[27, 28, 29], MixIT assumes access to in-domain noise samples which restricts
its universal applicability. Overcoming the latter constraint by injecting
additional out-of-domain (OOD) noise sources to the input mixture of mixtures
[30] further alters the input signal-to-noise ratio (SNR) distribution and its
performance depends heavily on the distribution shift between the injected and
real noise distributions. Thus, developing a SSL algorithm which does not
depend on external modality information nor assumptions about in-domain data
remains a challenging problem.

On the other hand, several self-training strategies have emerged and showed
promising results in classification tasks using convex combinations of labeled
and unlabeled data (e.g. Mixup [31]) but have also been successfully applied
to several audio tasks [32, 33]. In [34], a student model with a smaller
number of estimated sources has been trained on a subset of outputs of a pre-
trained MixIT model to solve the input SNR distribution mismatch. Furthermore,
a student model could also perform test-time adaptation by using the teacher’s
estimated waveforms as targets [35]. However, those approaches enforce only
the consistency of the student’s predictions over a frozen teacher’s output
pseudo-targets whereas other studies have shown that one can obtain
significant gains using unsupervised data augmentation [36], averages of
losses over multiple predictions [37], or their combination [38].

The student-teacher framework for singing-voice separation in [39] bears the
closest similarity to our work. The proposed setup assumes teacher pre-
training on supervised OOD data, performing inference on the in-domain noisy
dataset and storing the new pseudo-labeled dataset. At a second step, a
student network is trained on randomly mixed estimated sources that score
above a pre-defined confidence quality threshold. Unfortunately, if the
teacher’s estimates have low SNR and/or the threshold is not picked wisely
then the student model would also perform poorly. In contrast, some of the
most successful self-training approaches propose to iteratively update the
teacher’s weights using an exponential moving average scheme [40, 41, 42] or
sequentially update the teacher with the weights from a more expressive noisy
student [43].

In this work, we propose RemixIT which is based on several aforementioned
state-of-the-art SSL strategies for pseudo-labeling and continual training
while also providing a novel technique for training speech enhancement models
with OOD data. Our method trains a student model using self-augmented mixtures
generated by permuting and remixing the teacher’s estimates and using them as
pseudo-targets for regular regression. Moreover, RemixIT treats self-training
as a lifelong process while continually updating the teacher model using the
student’s weights that consequently leads to faster and more robust
convergence. RemixIT is the first method that:

  * •

Performs self-supervised learning using only in-domain mixture datasets and
OOD noise sources (e.g. MixIT pre-trained teacher with an OOD dataset).

  * •

Yields state-of-the-art results on several unsupervised and semi-supervised
denoising tasks without the need of clean speech waveforms or ad-hoc filtering
procedures.

  * •

Has strong theoretical and empirical evidence of why it works under various
noise levels.

  * •

Is able to leverage huge amounts of unsupervised data and generalize in
diverse training and adaptation scenarios.

Figure 1: RemixIT self-training procedure with a batch size of 4 noisy
mixtures. A teacher speech enhancement model $f_{\mathcal{T}}$ is pre-trained
in a supervised or unsupervised way on out-of-domain (OOD) data and performs
inference on a batch of noisy mixtures sampled from the in-domain noisy speech
dataset $\mathbf{m}\sim{\cal D}_{m}$. The randomly permuted teacher’s noise
estimates $\textbf{P}\widetilde{\mathbf{n}}$ are added together with the
teacher’s speech estimates $\widetilde{\mathbf{s}}$ to form the bootstrapped
mixtures $\widetilde{\mathbf{m}}$ which are fed to the student speech
enhancement network $f_{\mathcal{S}}$. The student is trained by regressing
over the teacher’s estimated sources which are now used as pseudo-targets
under a specified signal-level loss function. After repeating the overall
process for $K$ optimization steps, the teacher model may be updated using the
student’s weights in a continuous self-training scheme.

##  II RemixIT Method

RemixIT trains a speech enhancement model to isolate the clean speech signal
from its noisy observation. In general, we train a separation model $f$ which
outputs $M$ source waveforms for each input noisy speech recording with $T$
time-domain samples. Thus, given as input a batch of $B$ input waveforms
${\mathbf{x}}\in\mathbb{R}^{B\times T}$ the network estimates all sound
sources:

$\begin{gathered}\widehat{\mathbf{s}},\widehat{\mathbf{n}}=f({\mathbf{x}};{\bm{\theta}}),\enskip{\mathbf{x}}=\mathbf{s}+{\textstyle\sum}_{i=1}^{M-1}[\mathbf{n}]_{i}=\widehat{\mathbf{s}}+{\textstyle\sum}_{i=1}^{M-1}[\widehat{\mathbf{n}}]_{i},\end{gathered}$
(1)

where $\widehat{\mathbf{s}},\,\mathbf{s}\in\mathbb{R}^{B\times T}$,
$\widehat{\mathbf{n}},\,\mathbf{n}\in\mathbb{R}^{(M-1)\times B\times T}$,
${\bm{\theta}}$ are: the estimated speech signal, the clean speech target, the
estimated noise signal, the noise target and the parameters of the model,
respectively. We force the estimated sources $\widehat{\mathbf{s}}$ and
$\widehat{\mathbf{n}}$ to add up to the initial input mixtures ${\mathbf{x}}$
by using a mixture consistency projection layer [44]. We portray the inference
and self-training aspects of RemixIT in Figure 1, summarize it in Algorithm 1
and analyze it in depth in Section II-C. For completion, we highlight how
RemixIT differs from fully supervised training (assumes access to clean in-
domain speech) and previous state-of-the-art semi-supervised training methods
(MixIT assumes access to isolated in-domain noise recordings) in Sections II-A
and II-B, respectively.

${\bm{\theta}}^{(0)}_{\mathcal{T}}\leftarrow\textsc{Pretrain\\_Teacher}(f_{\mathcal{T}},{\cal
D}^{\prime})$

${\bm{\theta}}_{\mathcal{S}}\leftarrow\textsc{Initialize\\_Student}(f_{\mathcal{S}})$

for  _$k=0$ ; $k{+}{+}$; while $k<=K$_ do

for  _Sample_Batch $\mathbf{m}\in{\cal
D}_{m},\enskip\mathbf{m}\in\mathbb{R}^{B\times T}$_ do

$\widetilde{\mathbf{s}},\widetilde{\mathbf{n}}\leftarrow
f_{\mathcal{T}}(\mathbf{m};{\bm{\theta}}^{(k)}_{\mathcal{T}})$ // Teacher’s
estimates

$\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\textbf{P}\widetilde{\mathbf{n}}$
// Bootstrapped remixing

$\widehat{\mathbf{s}},\widehat{\mathbf{n}}\leftarrow
f_{\mathcal{S}}(\widetilde{\mathbf{m}};{\bm{\theta}}^{(k)}_{\mathcal{S}})$ //
Student’s estimates

$\mathcal{L}_{\operatorname{RemixIT}}=\sum_{b=1}^{B}\left[{\cal
L}(\widehat{s}_{b},\widetilde{s}_{b})+{\cal
L}(\widehat{n}_{b},\left[\textbf{P}\widetilde{\mathbf{n}}\right]_{b})\right]$

${\bm{\theta}}_{\mathcal{S}}\leftarrow\textsc{Update\\_Student}({\bm{\theta}}_{\mathcal{S}},\nabla_{{\bm{\theta}}_{\mathcal{S}}}\mathcal{L}_{\operatorname{RemixIT}})$

end for

${\bm{\theta}}^{(k+1)}_{\mathcal{T}}\leftarrow\textsc{Update\\_Teacher}({\bm{\theta}}^{(k)}_{\mathcal{T}},{\bm{\theta}}_{\mathcal{S}})$

end for

Algorithm 1 RemixIT for the noisy dataset ${\cal D}_{m}$.

###  II-A Supervised training

Supervised training is the straightforward way of training speech enhancement
models. It assumes access to both in-domain clean speech recordings,
$\mathbf{s}\sim{\cal D}_{s}$, as well as noise sources drawn from
$\mathbf{n}\sim{\cal D}_{n}$. Synthetic mixtures are generated at each
training step $\mathbf{m}=\mathbf{s}+\mathbf{n}$, by sampling a batch of clean
speech recordings $\mathbf{s}\sim{\cal D}_{s}$ and a batch of isolated noise
samples $\mathbf{n}\sim{\cal D}_{n}$, which are then fed to the separation
model $f$. For a sampled batch of $B$ input mixtures, the model predicts $M=2$
sources for each input mixture
($\widehat{\mathbf{s}},\,\widehat{\mathbf{n}}=f({\mathbf{x}};{\bm{\theta}})$)
and the following targeted loss function is minimized:

$\displaystyle\mathcal{L}_{\operatorname{Supervised}}=\textstyle\sum_{b=1}^{B}\left[{\cal
L}(\widehat{s}_{b},s_{b})+{\cal L}(\widehat{n}_{b},n_{b})\right],$ (2)

where ${\cal L}$ is any desired signal-level level loss function used to
penalize the reconstruction error between the estimates and their
corresponding targets. However, this training process is completely dependent
on the availability of clean speech and noise sources to capture the real-
world mixture distribution, making the model vulnerable to a performance
decline under unseen test conditions. This necessitates the development of SSL
and adaptation techniques for speech enhancement.

###  II-B Mixture invariant training (MixIT)

MixIT [27] is a simple yet effective idea for training a separation model
using artificial mixtures of mixtures (MoMs). In essence, MixIT assumes
availability of two sources of data during training, ${\cal D}_{m}$ which
consists of mixtures of speech and a noise source and ${\cal D}_{n}^{\prime}$
which contains noise recordings from a single noise source. The training
process boils down to sampling a batch of noisy speech recordings
$\mathbf{m}\sim{\cal D}_{m}$ (where $\mathbf{m}=\mathbf{s}+\mathbf{n}^{(1)}$),
and mixing them with another batch of clean noise samples
$\mathbf{n}^{(2)}\sim{\cal D}_{n}^{\prime}$. Note that the true noise
distribution of the real-world ${\cal D}_{n}^{*}$ ($\mathbf{n}^{(1)}\sim{\cal
D}_{n}^{*}$) is unknown and not necessarily same as the one available ${\cal
D}_{n}^{\prime}$. The separation model $f_{\mathcal{M}}$ is trained using the
synthetic batch of input-MoMs
${\mathbf{x}}=\mathbf{s}+\mathbf{n}_{1}+\mathbf{n}_{2}$ and tries to
reconstruct $M=3$ sources
$\widehat{\mathbf{s}},\,\widehat{\mathbf{n}}^{(1)},\,\widehat{\mathbf{n}}^{(2)}=f_{\mathcal{M}}({\mathbf{x}};{\bm{\theta}}_{\mathcal{M}})$,
by minimizing the following permutation invariant [45] loss function:

$\displaystyle\mathcal{L}_{\operatorname{MixIT}}^{(b)}=\underset{\bm{\pi}\in\mathcal{P}}{\min}\left[{\cal
L}(\widehat{s}_{b}+\widehat{n}_{b}^{(\pi_{1})},m_{b})+{\cal
L}(\widehat{n}_{b}^{(\pi_{2})},n_{b})\right],$ (3)

where $b$ is the batch’s index and $\mathcal{P}:=\\{(1,2),(2,1)\\}$ is the set
of permutations between the model’s noise output slots. If the noise sources
are independent from each other and the clean speech component, then the model
can learn to minimize this loss by reconstructing the mixture using its first
estimated slot and either one of the two noise slots available. Although MixIT
has been proven effective for various simulated speech enhancement setups [28,
29], the assumption about having access to a diverse set of in-domain noise
recordings from ${\cal D}_{n}^{\prime}$ which aptly captures the true
distribution of the present background noises ${\cal D}_{n}^{*}$ make it
impractical for many real-world settings. To this end, other works [30, 23]
have tried to deal with the distribution shift between the on-hand noise
dataset ${\cal D}_{n}$ and the clean noise distribution ${\cal D}_{n}^{*}$ in
order to avoid the need of in-domain noise samples. Specifically, [30]
proposes to use extra noise injection from an OOD distribution and in [23] ASR
and disentanglement losses have been proposed. However, the performance of the
former method still depends heavily the level of distribution shift between
the actual noise distribution ${\cal D}_{n}^{*}$ and ${\cal D}_{n}$ while the
latter method is more restrictive since it requires large pre-trained ASR
models.

###  II-C RemixIT: Self-training with bootstrapped remixing

In contrast to the aforementioned two training procedures which require in-
domain ground truth signals (e.g. supervised training requires clean speech
samples from $\mathbf{s}\sim{\cal D}_{s}$ as well as access to clean noise
sources drawn from $\mathbf{n}\sim{\cal D}_{n}$ while MixIT requires only
isolated in-domain noise waveforms), RemixIT does not depend on any other in-
domain information besides the mixture dataset ${\cal D}_{m}$. Specifically,
our method utilizes a student-teacher framework where the teacher’s noise
estimates are randomly permuted in a mini-batch sense and remixed with the
teacher’s speech estimates to create bootstrapped mixtures. A student model is
trained using as input the bootstrapped mixtures and regressing over the
teacher’s pseudo-target signals using a regular supervised loss at every
optimization step (for a succinct description of the training procedure please
see Algorithm 1). RemixIT also enjoys a continual refinement of the noisy
pseudo-target signals, after a few optimization steps, where the student model
weights are used to update the teacher network as it is illustrated in Figure
1.

####  II-C1 RemixIT’s teacher-student framework

For the initial teacher model, RemixIT can use any speech enhancement model
pre-trained on an OOD dataset ${\cal D}^{\prime}$ which outputs the speech
component and one or more noise estimated waveforms (see specification in
Equation 1). To this end, RemixIT materializes into semi-supervised domain
adaptation if the teacher was trained using a supervised loss and into a SSL
training scheme if the teacher was trained using MixIT.

Formally, given an batch of in-domain noisy mixtures
$\mathbf{m}=\mathbf{s}+\mathbf{n}\in\mathbb{R}^{B\times
T},\enskip\mathbf{m}\sim{\cal D}_{m}$, the teacher model estimates the speech
and the noise components as follows:

$\begin{gathered}\widetilde{\mathbf{s}},\,\widetilde{\mathbf{n}}=f_{\mathcal{T}}({\mathbf{x}};{\bm{\theta}}^{(k)}_{\mathcal{T}}),\enskip\mathbf{m}=\widetilde{\mathbf{s}}+{\textstyle\sum}_{i=1}^{M-1}[\textbf{n}^{\prime}]_{i}\\\
\end{gathered}$ (4)

where ${\bm{\theta}}^{(k)}_{\mathcal{T}}$ denotes the parameters of the
teacher model at the $k$-th optimization step. The second equation holds
because we enforce mixture consistency. A MixIT pre-trained model would
estimate $M=3$ sources and we can easily get a consolidated noise estimate by
summing the two latter noise estimated waveforms, namely,
$\widetilde{\mathbf{n}}=\sum_{i=1}^{M-1}[\textbf{n}^{\prime}]_{i}$. Notice
that the teacher model $f_{\mathcal{T}}$ does not need to be identical through
the whole training process and could be updated using any user-specified
protocol which results in a separation model that respects the constraints
defined in Equation 4. The teacher’s estimates within a batch of size $B$ are
used to generate the bootstrapped mixtures $\widetilde{\mathbf{m}}$ by
remixing the estimated speech and noise sources in a random order:

$\begin{gathered}\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\widetilde{\mathbf{n}}^{(\textbf{P})}\in\mathbb{R}^{B\times
T},\enskip\widetilde{\mathbf{n}}^{(\textbf{P})}=\textbf{P}\widetilde{\mathbf{n}},\enskip\textbf{P}\sim\bm{\Pi}_{B\times
B},\end{gathered}$ (5)

where P is drawn uniformly from the set of all $B\times B$ permutation
matrices and is used to produce the permuted noise sources
$\widetilde{\mathbf{n}}^{(\textbf{P})}$. The original teacher’s speech
estimates $\widetilde{\mathbf{s}}$ and the permuted noise sources
$\widetilde{\mathbf{n}}^{(\textbf{P})}$ are now used as target pairs to train
the student model $f_{\mathcal{S}}$ on the newly generated batch of
bootstrapped mixtures $\widetilde{\mathbf{m}}$ as shown below:

$\begin{gathered}\widehat{\mathbf{s}},\,\widehat{\mathbf{n}}=f_{\mathcal{S}}(\widetilde{\mathbf{m}};{\bm{\theta}}^{(k)}_{\mathcal{S}}),\enskip\widehat{\mathbf{s}},\,\widehat{\mathbf{n}}\in\mathbb{R}^{B\times
T}\\\ \mathcal{L}_{\operatorname{RemixIT}}^{(b)}={\cal
L}(\widehat{\mathbf{s}}_{b},\widetilde{\mathbf{s}}_{b})+{\cal
L}(\widehat{\mathbf{n}}_{b},\widetilde{\mathbf{n}}^{(\textbf{P})}_{b}),\enskip
b\in\\{1,\dots,B\\}.\end{gathered}$ (6)

The loss function used is similar to a supervised setup (see Equation 2) but
instead of ground-truth clean source waveforms, we use the noisy estimates,
$\widetilde{\mathbf{s}}$ and $\widetilde{\mathbf{n}}^{(\textbf{P})}$, provided
by the teacher network. If the signal-level loss function ${\cal L}$ also
minimizes the Euclidean norm between the estimated signals and the target
signals, the proposed cost function $\mathcal{L}_{\operatorname{RemixIT}}$
enjoys several convergence properties which enable our method to learn in a
robust SSL fashion even in cases where the teacher’s estimates are not close
to the ground-truth source waveforms (see Section II-C2).

Lastly, RemixIT refines the estimates of the teacher network $f_{\mathcal{T}}$
using the weights from the latest available student models. The continual
update protocols used in this study are the sequential and the running moving
average update protocols which are explained in detail in Section III-C.

####  II-C2 Error analysis under the Euclidean norm

In each optimization step, RemixIT tries to minimize a signal-level loss
function between the student’s estimates and the teacher’s pseudo-targets.
Since we are mostly interested in denoising, we focus on the speech estimates
of the teacher and the student networks with initial mixtures $\mathbf{M}$ and
the bootstrapped mixtures $\widetilde{\mathbf{M}}$ as inputs, respectively.
These estimates can also be expressed in the following way as random
variables:

$\begin{gathered}\widetilde{\mathbf{S}}=f^{(\widetilde{s})}_{\mathcal{T}}(\mathbf{M}=\mathbf{S}+\mathbf{N};\,{\bm{\theta}}^{(k)}_{\mathcal{T}}),\enskip\mathbf{M}\sim{\cal
D}_{m}\\\
\widehat{\mathbf{S}}=f^{(\widehat{s})}_{\mathcal{S}}\,(\widetilde{\mathbf{M}}=\widetilde{\mathbf{S}}+\widetilde{\mathbf{N}}^{(\textbf{P})};{\bm{\theta}}^{(k)}_{\mathcal{T}}).\end{gathered}$
(7)

Now, the teacher’s $\widetilde{\mathbf{R}}_{\cal T}$ and student’s
$\widehat{\mathbf{R}}_{\cal S}$ errors w.r.t. the initial clean targets
$\mathbf{S}$ are the following conditional probabilities:

$\begin{gathered}\widetilde{\mathbf{R}}_{\cal
T}=\widetilde{\mathbf{S}}-\mathbf{S},\enskip\enskip\widetilde{\mathbf{R}}_{\cal
S}=\widehat{\mathbf{S}}-\mathbf{S}\\\ \widetilde{\mathbf{R}}_{\cal T}\sim
P(\widetilde{\mathbf{R}}_{\cal
T}|\mathbf{S},\mathbf{N}),\enskip\widehat{\mathbf{R}}_{\cal S}\sim
P(\widehat{\mathbf{R}}_{\cal
S}|\widetilde{\mathbf{S}},\widetilde{\mathbf{N}},\textbf{P}).\end{gathered}$
(8)

Using a signal-level loss ${\cal L}$ that minimizes the squared error between
the estimated and the target signals in Equation 6 and assuming unit-norm
estimated and target signals $||s||=||\widetilde{s}||=||\widehat{s}||=1$,
RemixIT loss function becomes equivalent to minimizing the following
expression:

$\begin{gathered}\mathcal{L}_{\operatorname{RemixIT}}\propto{\mathbb{E}}[||\widehat{\mathbf{S}}-\widetilde{\mathbf{S}}||^{2}_{2}]={\mathbb{E}}[||(\widehat{\mathbf{S}}-\mathbf{S})-(\widetilde{\mathbf{S}}-\mathbf{S})||^{2}_{2}]\\\
=\underbrace{{\mathbb{E}}\left[||\widehat{\mathbf{R}}_{\cal
S}||^{2}_{2}\right]}_{\text{Supervised
Loss}}+\underbrace{{\mathbb{E}}\left[||\widetilde{\mathbf{R}}_{\cal
T}||^{2}_{2}\right]}_{\text{Constant w.r.t. }{\bm{\theta}}_{\cal
S}}-2\underbrace{{\mathbb{E}}\left[\langle\widehat{\mathbf{R}}_{\cal
S},\widetilde{\mathbf{R}}_{\cal T}\rangle\right]}_{\text{Errors'
correlation}}\end{gathered}$ (9)

Ideally, this loss could lead to the same optimization objective with a
supervised setup if the last inner-product term was zero since the middle term
becomes zero when computing the gradient w.r.t. the student’s parameters
${\bm{\theta}}_{\cal S}$. $\langle\widehat{\mathbf{R}}_{\cal
S},\widetilde{\mathbf{R}}_{\cal T}\rangle=0$ could be achieved if the teacher
produced outputs indistinguishable from the clean target signals or the
conditional error distributions in Equation 8 were independent. Intuitively,
as we continually update the teacher model and refine its estimates, we
minimize the norm of the teacher error which leads to higher fidelity
reconstruction from the student (for further analysis of how the student
learns to perform better than its teacher and for experimental validation of
this claim we refer the reader to Section IV-D).

Additionally, the bootstrapped remixing process forces the errors to be more
uncorrelated since the student tries to reconstruct the same clean speech
signals $\mathbf{s}$, similar to its teacher, but under a different mixture
distribution. Formally, the student tries to reconstruct $\mathbf{s}$ when
observing the bootstrapped mixtures
$\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\widetilde{\mathbf{n}}^{(\textbf{P})}$
while the teacher tries to reconstruct $\mathbf{s}$ only from the initial
input mixtures $\mathbf{m}=\mathbf{s}+\mathbf{n}$. This phenomenon becomes
apparent if we focus on the reconstruction of a single speech signal $s^{*}$
from the teacher and the student networks. In essence, we use the teacher
network to provide an estimated $\widetilde{s}^{*}$ from the corresponding
mixture $m^{*}$ and some perturbed noise sources
$\widetilde{n}_{b}^{\prime},\enskip\forall b\in\\{1,\dots,B\\}$ to create
bootstrapped mixtures:

$\begin{gathered}\widetilde{s}^{*},\widetilde{n}^{*}=f_{\mathcal{T}}(m^{*}=s^{*}+n^{*};{\bm{\theta}}_{\mathcal{T}})\\\
\widetilde{s}_{b}^{\prime},\widetilde{n}_{b}^{\prime}=f_{\mathcal{T}}(m_{b}^{\prime}=s_{b}^{\prime}+n_{b}^{\prime};{\bm{\theta}}_{\mathcal{T}})\\\
\widetilde{m}_{b}^{\prime}=\widetilde{s^{*}}+\widetilde{n}_{b}^{\prime},\enskip\forall
b\in\\{1,\dots,B\\}.\end{gathered}$ (10)

In the student-training phase, we perform inference using the student network
$f_{\cal S}$ on the batch of the aforementioned bootstrapped mixtures
$\widetilde{m}_{b}^{\prime}$. Because RemixIT’s loss is computed under
expectation (Equation 9), we can rearrange the order of batches that the
student network sees. Thus, we focus on the learning aspect of the student
network for the batch of bootstrapped mixtures above (Equation 10) and rewrite
the last error correlation term as follows:

$\begin{gathered}{\mathbb{E}}\left[\langle\widehat{\mathbf{R}}_{\cal
S},\widetilde{\mathbf{R}}_{\cal
T}\rangle\right]\approx\frac{1}{B}{\textstyle\sum}_{b=1}^{B}\left(\widehat{s}_{b}-s^{*}\right)^{\mathrm{T}}\left(\widetilde{s}^{*}-s^{*}\right)\\\
=\left(\widetilde{s}^{*}-s^{*}\right)^{\mathrm{T}}\underbrace{\frac{1}{B}{\textstyle\sum}_{b=1}^{B}\left(f^{(\widehat{s})}_{\mathcal{S}}(\widetilde{s}^{*}+\widetilde{n}_{b}^{\prime};{\bm{\theta}}_{\mathcal{S}})-s^{*}\right)}_{\text{\small
Empirical mean student error}}\end{gathered}$ (11)

The premise is that if the student sees a wide variety of bootstrapped
mixtures which have been generated using the same teacher’s speech estimate
$\widetilde{s}^{*}$, then the mean interference error produced by injecting
noisy teacher’s estimates $\widetilde{n}_{b}^{\prime}$ would go to zero under
expectation. Thus, the student could still minimize the errors’ correlation
term and still be able to learn from mixtures when the teacher performs poorly
(Section IV-E gives empirical analysis of our claim).

##  III Experimental Framework

###  III-A Datasets

DNS-Challenge (DNS): The DNSChallenge 2020 benchmark dataset [46] consists of
a large collection of clean speech recordings which are mixed with a wide
variety of noisy speech samples with $64{,}649$ and $150$ pairs of clean
speech and noise recordings for training and testing, respectively. DNS is
used for showing the effectiveness of the proposed self-training scheme where
large amounts of unsupervised training data is available and one needs to
improve the performance of a model trained only on limited OOD supervised
data.

LibriFSD50K (LFSD): This data collection includes $45{,}602$ and $3{,}081$
mixtures for training and testing, correspondingly. The clean speech samples
are drawn from the LibriSpeech [47] corpus and the noise recordings are taken
from FSD50K [48] representing a set of almost $200$ classes of background
noises after excluding all the human-made sounds from the AudioSet ontology
[49]. A detailed recipe of the dataset generation process is presented in
[29]. LFSD becomes an ideal candidate for semi-supervised/SSL teacher pre-
training on OOD data given its mixture diversity.

WHAM!: The generation process for this dataset produces $20{,}000$ training
noisy-speech pairs and $3{,}000$ test mixtures from the initial WHAM! [50]
dataset and has been identical to the procedure followed in [29]. The set of
background noises in WHAM! is limited to 10 classes of urban sounds.

VCTK: The VCTK dataset proposed in [51] includes $586$ synthetically generated
noisy test mixtures, where a speech sample from the VCTK speech corpus [52] is
mixed with an isolated noise recording from the DEMAND [53]. The VCTK and DNS
test partitions are used to illustrate the effectiveness of RemixIT under a
restrictive scenario zero-shot domain adaptation with limited data to perform
self-training.

###  III-B Speech enhancement model

In the supervised and RemixIT training recipes, the student model has $M=2$
output slots and always estimates the speech component and the noise source.
For the models which are trained with MixIT, we increase the number of output
slots to $M=3$ to enable the estimation of the additional noise component.
RemixIT is independent of the choice of the speech-enhancement model
architecture as long as the latter estimates both speech and noise components
of the input mixture. Our model’s choice was based on obtaining adequate
quality of speech reconstruction with low computational and memory
requirements (see Table I for a head-to-head comparison in a supervised in-
domain training setup with the previous state-of-the-art model). To this end,
we used the Sudo rm -rf [54] architecture with the more sparse computation
blocks using shared sub-band processing via group communication [55]. The
selected network has shown to provide high-quality source estimates under
speech enhancement [29] as well as sound separation [56] tasks while
significantly reducing the model’s size. We consider the selected architecture
with a default depth of $U=8$ U-ConvBlocks and the same parameter
configurations as used in [29]. In the sequential update protocol we increase
the depth of the new student networks every $20$ epochs from $8$ to $16$ and
finally $32$.

###  III-C RemixIT’s teacher update protocols configurations

RemixIT refines the estimates of the student network based on unsupervised and
semi-supervised teachers pre-trained on an OOD dataset but also has the
capability of repeatedly updating the teacher network to learn from higher-
quality source estimates. In our experiments, we evaluate the proposed method
under various online teacher updating protocols after $k$ training epochs.
Specifically, we consider the following:

Static teacher: The teacher is frozen throughout the training process, for all
optimization steps
${\bm{\theta}}^{(k)}_{\mathcal{T}}={\bm{\theta}}^{(0)}_{\mathcal{T}},\enskip\forall
k$.

Sequentially updated teacher: Every 20 epochs or equivalently
$K=\nicefrac{{20\times|{\cal D}_{m}|}}{{B}}$ optimization steps, where
$\nicefrac{{|{\cal D}_{m}|}}{{B}}$ is the number of batches per-training
epoch, we replace the teacher with the latest student, namely,
$\mathring{{\bm{\theta}}}^{(k\operatorname{mod}K)}_{\mathcal{T}}\coloneqq{\bm{\theta}}^{(k\operatorname{mod}K)}_{\mathcal{S}}$.

Exponentially moving average teacher: The teacher is gradually updated after
every epoch using an exponential moving average scheme
$\bar{{\bm{\theta}}}^{(j+1)}_{\mathcal{T}}\coloneqq\gamma{\bm{\theta}}^{(j)}_{\mathcal{S}}+(1-\gamma)\bar{{\bm{\theta}}}^{(j)}_{\mathcal{T}},\enskip\forall
k$ with $\gamma=0.01$, where $j$ is a multiple of $\nicefrac{{|{\cal
D}_{m}|}}{{B}}$.

###  III-D Training and evaluation details

For the semi-supervised and unsupervised RemixIT’s teachers we pre-train the
corresponding models following the supervised training process (Section II-A)
and MixIT (Section II-B), respectively. Although RemixIT can work with any
valid signal-level loss functions (Equations 3, 6), we choose the negative
scale-invariant signal to distortion ratio (SI-SDR) [57] for training all
models:

$\begin{gathered}\mathcal{L}(\widehat{y},y)=-\text{SI-
SDR}(\widehat{y},y)=-20\log_{10}(\nicefrac{{\|\alpha y\|}}{{\|\alpha
y-\widehat{y}\|}}).\end{gathered}$ (12)

$\alpha=\widehat{y}^{\top}y/\|y\|^{2}$ makes the loss invariant to the scale
of the estimated source $\widehat{y}$ and the target signal $y$. By setting
$\alpha=1$, SI-SDR becomes equivalent with SNR. We train all models using the
Adam optimizer [58] with a batch size of $B=2$ and an initial learning rate of
$10^{-3}$ which is divided by $2$ every $6$ epochs. For all experiments,
during training we assume that we do not have access to the input SNR
distribution and thus, we mix a clean and a noise source without altering
their corresponding power ratio. However, for the in-domain supervised
training setup with DNS we randomly mix clean speech and noise recordings with
SNR from a uniform distribution of $[-2,20]dB$, which has been shown to be
effective for multiple sound separation setups [59, 60, 61]. Finally, we
normalize all input mixture waveforms by subtracting their mean and dividing
by their standard deviation before feeding them to each model. We train and
test models which operate at a $16$kHz sampling rate.

The robustness of all speech enhancement models is measured using the SI-SDR
[57], the short-time objective intelligibility (STOI) [62] and the perceptual
evaluation of speech quality (PESQ) [63]. We evaluate the model checkpoints
after $100$ epochs for the pre-trained teachers and the supervised models and
after $60$ epochs for all the other configurations.

Training method and model details |  $\\#$Model | Available Training Data (%) |  Mean evaluation metrics
---|---|---|---
| Params | Clean Speech ${\cal D}_{s}$ | Clean Noise ${\cal D}_{n}$ | Mixture ${\cal D}_{m}$ | SISDR | PESQ | STOI
| ($10^{6}$) | DNS | LFSD | DNS | LFSD | DNS | LFSD | (dB)
Input Noisy Mixture | - |  |  |  |  |  |  | 9.2 | 1.58 | 0.915
MixIT with student ($U=8$) | In-domain | $0.79$ |  |  | 20% |  | 80% |  | 14.4 | 2.13 | 0.933
OOD noise | $0.79$ |  |  |  | 20% | 100% |  | 14.3 | 2.02 | 0.933
Extra noise [30] | $0.79$ |  |  |  | 50% | 100% |  | 14.5 | 2.03 | 0.930
Unsupervised RemixIT (ours) | Teacher ($U=8$) | $0.79$ |  |  |  | 20% |  | 80% | 14.8 | 2.15 | 0.940
Student ($U:8\rightarrow 16\rightarrow\mathbf{32}$) | $0.97$ |  |  |  |  | 100% |  | 16.0 | 2.34 | 0.952
Semi-supervised RemixIT (ours) | Teacher ($U=8$) | $0.56$ |  | 100% |  | 100% |  |  | 17.6 | 2.61 | 0.958
Student ($U:8\rightarrow 16\rightarrow\mathbf{32}$) | $0.73$ |  |  |  |  | 100% |  | 18.0 | 2.60 | 0.959
Supervised | FullSubNet∗ [8] | $5.6$ | 100% |  | 100% |  |  |  | 17.3 | 2.78 | 0.961
in-domain | Sudo rm -rf [56] ($U=8$) | $0.56$ | 100% |  | 100% |  |  |  | 18.6 | 2.69 | 0.962
training | Sudo rm -rf [56] ($U=32$) | $0.73$ | 100% |  | 100% |  |  |  | 19.7 | 2.95 | 0.971
TABLE I: Evaluation results for the speech enhancement task on the DNS test
set using the proposed RemixIT method, MixIT approaches [27, 30] and
supervised in-domain training with the Sudo rm -rf model [56] as well as the
previous state-of-the-art FullSubNet [8] supervised model (∗as it was
presented in the paper). All teacher and student networks follow the same Sudo
rm -rf model [56] architecture with the specified number of U-ConvBlocks
($U=8$ or $U=32$). $U:8\rightarrow 16\rightarrow\mathbf{32}$ denotes that we
double the depth of the student network every $20$ epochs and sequentially
update the teacher with the latest available student, the reported number
refers to the performance of the student with $U=32$.

##  IV Results and Discussion

###  IV-A The need for continual refinement of teacher’s estimates

Figure 2: SI-SDR (dB) performance on the test partition of the DNS dataset as
training progresses using different teacher update protocols. The initial
teacher network is shared across the various protocols (a Sudo rm -rf model
with $U=8$ Conv-blocks) and was pre-trained in a supervised way on the WHAM!
dataset. The orange solid line shows significant gains over the other
protocols (gray and black dashed lines) and denotes the mean SI-SDR obtained
by a student network with a sequentially updated teacher protocol where every
$20$ epochs we initialize a new student model and replace the teacher model
with the latest available student.

In Figure 2, we show the speech enhancement performance of the student models
produced by a static or a sequentially updated teacher every $20$ epochs.
Unsurprisingly, all protocols behave similarly until the $20$th epoch since
they use the exact same initial teacher. In contrast to the frozen teacher
protocols, after the $20$th epoch, the old teacher is replaced with the newly
trained student with $U$=8 and a new student, with twice as much depth (8
$\rightarrow$ 16) is initialized. Surprisingly, the sequentially updated
teacher protocol keeps teaching a better student separation model, even after
the $40$th epoch, compared to both models produced by the static teachers
which saturate for the same number of training steps. Comparing between the
students with $U$=8 and $U$=16 produced by static teacher models, it is
evident that the more expressive student performs better but not on par with
the same depth student produced by the sequentially updated teacher protocol.
Thus, the combination of the bootstrapped remixing and the continual
refinement of the teacher’s estimates is key for the significant improvement
that RemixIT yields. As a result, we have chosen the sequentially updated
teacher protocol as the default strategy for RemixIT, except for the zero-shot
adaptation where we use the exponential average teacher updating scheme
because the number of available training mixtures could make the student prone
to overfitting if trained from scratch.

###  IV-B Self-supervised and semi-supervised speech enhancement

Table I summarizes the mean speech enhancement performance of RemixIT against
in-domain and cross-domain supervised and SSL baselines with the same
architecture on the DNS test set. Notice that in both semi-supervised and
unsupervised cases, the learned RemixIT’s student does not assume access to
in-domain clean speech nor to noise samples like the previous state-of-the-art
SSL speech enhancement algorithms. For instance, SSL RemixIT’s teacher pre-
training is performed with OOD MixIT by using $80\%$ of the LFSD noisy
recordings ${\cal D}_{m}^{\prime}$ and rest $20\%$ to simulate the isolated
noise recordings ${\cal D}_{n}^{\prime}$, whereas the student is trained
solely on the vast amount of training mixtures in the DNS dataset.

Despite the fact that RemixIT makes no assumptions about the in-domain
distribution of mixtures nor it assumes access to in-domain ground truth
source waveforms, it significantly outperforms all the previous state-of-the-
art MixIT-like approaches. The unsupervised student learned using the proposed
method yields an improvement over the second-best unsupervised method of more
than ($14.5$dB $\rightarrow 16.0$dB in terms of SI-SDR and $0.02$ in terms of
STOI) compared against in-domain MixIT and the recently proposed extra noise
augmentation where an extra noise source is injected [30]. In the semi-
supervised domain adaptation setup, we show that RemixIT’s student still
provides noticeable improvement over its initial teacher pre-trained in a
supervised way assuming access to a smaller but diverse dataset like LFSD.
Although we have used the same separation model architecture across our
experiments, our method is independent of the model’s choice and could be used
with models that produce higher quality estimates. However, the bottom three
rows in Table I show that the model used in this study achieves state-of-the-
art speech enhancement results when trained with in-domain ground-truth
sources.

(a) Semi-supervised RemixIT with initial teacher pre-trained on WHAM! in a
supervised way.

(b) Unsupervised RemixIT with initial teacher pre-trained on LFSD using MixIT.

Figure 3: SI-SDR (dB) performance improvement on the training portion of the
DNS dataset that a RemixIT’s student with a sequentially updated teacher every
20 epochs yields as the training progresses over the initial teacher’s
estimates. We show that similar learning patterns emerge for different initial
teachers pre-trained in a semi-supervised way (top) and an unsupervised way
(bottom). The median and the mean $\Delta$SI-SDR are denoted with a solid
green line and an orange star, respectively.  Figure 4: SI-SDR performance
improvement that RemixIT’s student yields over its initial OOD pre-trained
teacher model for various low-resource adaptation datasets (e.g. DNS, LFSD and
WHAM!, from left to right). Both teacher and student models have the exact
same Sudo rm -rf architecture ($U=8$ ConvBlocks) and we use the running mean
teacher update protocol. RemixIT shows significant improvements against all
teacher models used in this study, namely, MixIT pre-training on LFSD
(blue/leftmost) and supervised training on LFSD (yellow/middle) and as well as
on WHAM! (green/rightmost).

###  IV-C Zero-shot domain adaptation

In low-resource training scenarios, the training mixtures in-hand might not be
sufficient to train a model from scratch, thus, we show how RemixIT can be
used as a zero-shot unsupervised domain adaptation algorithm. We perform
teacher pre-training on larger OOD datasets and fine-tune a student model
using limited in-domain mixtures. At the start of the adaptation process, the
student is initialized using the pre-trained teacher’s weights
${\bm{\theta}}_{\mathcal{S}}^{(0)}:={\bm{\theta}}_{\mathcal{T}}^{(0)}$ and we
perform RemixIT while periodically updating the teacher using the moving
average protocol (see Section III-C). The cross-dataset adaptation results are
illustrated in Figure 4. The proposed method delivers consistent improvements
across datasets and pre-training techniques, up to $0.8$dB in terms of SI-SDR
over the non-calibrated models. Unsurprisingly, one can notice that the level
of improvement is directly impacted by the amount of available noisy mixtures.
We postulate that this is the main reason that our method obtains larger
(smaller) gains for the adaptation on WHAM! (DNS) test partition which has
$3{,}000$ (only $150$) mixtures, respectively. However, RemixIT performs
adequately even in cases where there is a large distribution shift between the
training and the adaptation-test sets (e.g. WHAM! contains only 10 classes of
urban background noises while the DNS dataset is very diverse). Specifically,
the significant improvement after adapting a supervised pre-trained model on
WHAM! to the $150$ mixtures of the very diverse DNS set, indicates the
effectiveness of RemixIT under really challenging zero-shot learning
conditions.

(a) Semi-supervised RemixIT with initial teacher pre-trained on WHAM! in a
supervised way.

(b) Unsupervised RemixIT with initial teacher pre-trained on LFSD using MixIT.

Figure 5: Distribution of SNR improvement (dB) on the DNS training set that
the empirical mean RemixIT’s student after 10 training epochs ( Equation 11)
yields over its initial teacher in regions where the latter performs poorly.
The solid orange line denotes the mean SNR improvement for each number of
bootstrapped mixtures which are considered under expectation
$\frac{1}{B}\sum_{b=1}^{B}f^{(\widehat{s})}_{\mathcal{S}}(\widetilde{s}+\widetilde{n}_{b}^{\prime};{\bm{\theta}}_{\mathcal{S}})$.
We show that as a fixed student network sees more input bootstrapped mixtures,
the mean student performance becomes better on average than its teacher even
early in training and in regions where the teacher performs poorly.

###  IV-D Student learning progression

We analyze how a student speech enhancement model trained with RemixIT on the
DNS train set refines its estimates as the training progresses and how it
compares against its initial teacher. In Figure 3, we showcase the improvement
obtained in terms of SI-SDR for various teachers and their performance
brackets under a sequentially updated teacher every 20 epochs using the
parameters from the student network. Note that the student is gradually
learning to perform better than the initial teacher network in the regions
where the latter performs better (rightmost plots row-wise) even if producing
improvement over really good estimates (e.g. higher than $15$dB) becomes
harder. Thus, it becomes evident that the continual self-training scheme of
RemixIT where the teacher network is updated using the latest student’s
weights is key to a larger performance boost. The result holds for both OOD
supervised and MixIT teachers and is on par-with our theoretical analysis in
Section II-C2 where we show how a better teacher helps the error correlation
term of RemixIT’s loss function to diminish and resemble supervised training.
In contrast, for the low performing brackets ($[-30,-10]$dB in terms of
teacher SI-SDR (dB)), the student does not learn how to further increase its
performance, even if it regresses over the estimated waveforms of updated
teachers. The emergence of this learning pattern necessitates the discovery of
more robust self-training algorithms which can recover from cases where the
teacher network provides extremely noisy estimates.

###  IV-E Robust learning with very noisy teacher’s estimates

We investigate the robustness of RemixIT in cases where the teacher model
outputs a low quality speech estimate $\widetilde{s}$. Building upon the
analysis performed in Section II-C2, we reiterate on how important is for the
student to be trained on multiple bootstrapped mixtures
$\widetilde{m}_{b}^{\prime}=\widetilde{s}+\widetilde{n}_{b}^{\prime},\enskip\forall
b\in\\{1,\dots,B\\}$ produced using the same teacher’s speech estimate
$\widetilde{s}$ and independent teacher’s noise estimates
$\widetilde{n}_{b}^{\prime}$ (see Equations 10, 11). The distribution of the
SNR performance improvement that the empirical mean student yields over the
initial teacher after 10 training epochs is displayed in Figure 5 while
sweeping the number of input bootstrapped mixtures. For both cases of
supervised and MixIT teachers we see that the mean SNR improvement is around
$2$ dB when increasing the number of bootstrapped mixtures $B$ from $1$ to
$64$. Notably, this result holds for really bad teacher estimates, namely,
less than $5$ dB and is obtained by simply performing inference over more
augmentations of $\widetilde{s}$ without refining the student parameters.
Assuming that all speech estimates and the ground-truth signals have unit-norm
$\|s\|=\|\widetilde{s}\|=\|\widehat{s}_{b}\|=1$, the maximization of SNR
becomes equivalent to minimizing the $l2$ norm
$\operatorname{SNR}(\widehat{y},y)\propto-\|\widehat{y}-y\|$. As a result, the
mean SNR improvement of the empirical mean student leads the term
${\mathbb{E}}[\langle\widehat{\mathbf{R}}_{\cal
S},\widetilde{\mathbf{R}}_{\cal T}\rangle]$ closer to zero (Equation 11) and
consequently, the student to learn in a more robust way, even in cases where
the teacher’s error term $\|\widetilde{\mathbf{R}}_{\cal T}\|$ is far from
zero.

###  IV-F Cross-domain generalization

A performance analysis for cross-domain speech enhancement generalization is
displayed in Table II. MixIT and its variants fail to generalize in cases
where the noise distribution ${\cal D}_{n}$ does not closely resemble the true
in-domain distribution ${\cal D}_{n}^{*}$. However, RemixIT outperforms all
MixIT methods without having access to in-domain datasets. For instance, in
the case where one only has access to mixtures from the WHAM! (W!) dataset and
noise sources from LFSD (L), the best noise augmented MixIT model obtains only
$1.6$ dB of SI-SDR improvement on the adaptation WHAM! dataset. In stark
contrast, RemixIT with a pre-trained MixIT teacher on LFSD yields an
improvement of 5.3dB ($1.6\rightarrow 6.9$) over the best cross-dataset MixIT
and 0.7 dB ($6.2\rightarrow 6.9$) over the teacher model. Moreover, RemixIT
can also improve the performance of a teacher model in the source dataset
test-set by leveraging other target mixture dataset. Notice that RemixIT
yields an improvement of 1.4dB ($8.5\rightarrow 9.9$) on the LFSD test-set
over the pre-trained teacher model on LFSD by using self-training over the
diverse unsupervised DNS mixture dataset. Surprisingly, RemixIT also
outperforms its teacher by a large margin ($6.2\rightarrow 8.2$ dB) on the
WHAM! test set even though it has not seen any data from this datset which
shows how the proposed model can provide a seamless solution to generalizing
denoising models to unseen data.

Method | $U$ | Available Training Data | Mean test-set
---|---|---|---
Speech | Noise | Mixtures |  $\Delta\operatorname{SI-SDR}$ (dB)
${\cal D}_{s}$ | ${\cal D}_{n}$ | ${\cal D}_{m}$ | D | L | W!
| 8D | D | D |  | 9.4 | 10.3 | 8.4
Super- | 32 |  | 10.5 | 12.2 | 10.2
vised | 8W! | W! | W! |  | 6.1 | 4.8 | 11.3
| 32 |  | 7.1 | 6.2 | 12.8
In- | 8 |  | D | D | 5.2 | 6.3 | 6.6
domain | 8L |  | L | L | 5.6 | 8.5 | 6.2
MixIT | 8 |  | W! | W! | 4.5 | 2.3 | 9.8
MixIT | 8 |  | L | D | 5.1 | 3.6 | 5.3
with |  | W! | 1.2 | -0.2 | 1.7
OOD |  | D | W! | -0.8 | -6.3 | 1.8
noise |  | L | 1.7 | -1.7 | 1.2
MixIT | 8 |  | L+L | D | 5.3 | 1.3 | 4.7
w extra |  | W!+W! | 5.3 | 5.0 | 2.9
noise |  | D+D | W! | -3.5 | -9.8 | 3.0
[30] |  | L+L | -1.6 | -10.2 | 1.6
RemixIT | 8 | W! | W! | D | 6.7 | 5.7 | 11.3
w sup. | 32 | 7.3 | 6.4 | 10.9
teachers | 8 | D | D | W! | 6.9 | 5.6 | 9.0
(ours) | 32 | 6.9 | 5.4 | 9.8
RemixIT | 8 |  | L | L$\rightarrow$D | 6.3 | 9.3 | 7.5
w MixIT | 32 |  | 6.8 | 9.9 | 8.2
teacher | 8 |  | L | L$\rightarrow$W! | 5.3 | 7.5 | 6.8
(ours) | 32 |  | 4.9 | 7.2 | 6.9
TABLE II: Mean $\Delta$SI-SDR (dB) improvement for the models obtained using
RemixIT, MixIT and supervised training when evaluated on the test partitions
of the DNS (D), LFSD (L), and WHAM! (W!) datasets. $U$ defines the number of
UConvBlocks in the corresponding Sudo rm -rf [56] model. The supervised pre-
trained teacher models on DNS and WHAM! and the MixIT pre-trained teacher on
LFSD, used by RemixIT, are denoted with D, W! and L, respectively. The
evaluated RemixIT’s student models only see the bolded mixture dataset. The
bolded (blue-colored) performance numbers denote the best performing model
per-dataset without access to clean speech (clean corresponding in-domain
data). Gray background colored cells denote that the model had access to clean
in-domain training data for the corresponding dataset. The mean noisy input
mixture SI-SDR performance is $9.2$, $6.3$ and $-7.4$ dB for DNS, LFSD, and
WHAM! datasets, respectively.

##  V Conclusion

We have presented a self-training scheme for speech enhancement models which
is based on a lifelong bi-directional parameter update between a teacher and a
student network. The proposed framework aptly transfers the knowledge of a
pre-trained model on out-of-domain data using bootstrapped remixing and
through the continual refinement of the teacher’s outputs. We have
experimentally shown that our method significantly outperforms all previous
state-of-the-art self-supervised methods while being more general and without
the dependence on in-domain data. Moreover, our results illustrated that
RemixIT can also perform semi-supervised and zero-shot domain adaptation
setups with limited in-domain mixtures. Furthermore, our theoretical analysis
is backed by empirical results and instrumental to the understanding of the
teacher-student learning dynamics, especially in where our method can still
learn with extremely noisy pseudo-target signals. In the future, we aim to
strengthen the robustness of our algorithm by estimating a confidence-based
proxy for the quality of the pseudo-targets [38] as well as widen the
applicability of our method by applying it to different domains.

## References

  * [1] Jacob Benesty, Shoji Makino, and Jingdong Chen,  Speech enhancement,  Springer Science & Business Media, 2006.
  * [2] Felix Weninger, Hakan Erdogan, Shinji Watanabe, Emmanuel Vincent, Jonathan Le Roux, John R Hershey, and Björn Schuller,  “Speech enhancement with lstm recurrent neural networks and its application to noise-robust asr,”  in Latent Variable Analysis and Signal Separation. Springer, 2015, pp. 91–99.
  * [3] Tara N Sainath, Ron J Weiss, Kevin W Wilson, Bo Li, Arun Narayanan, Ehsan Variani, Michiel Bacchiani, Izhak Shafran, Andrew Senior, Kean Chin, et al.,  “Multichannel signal processing with deep neural networks for automatic speech recognition,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 5, pp. 965–979, 2017.
  * [4] Hassan Taherian, Zhong-Qiu Wang, Jorge Chang, and DeLiang Wang,  “Robust speaker recognition based on single-channel and multi-channel speech enhancement,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 1293–1302, 2020.
  * [5] Hyeong-Seok Choi, Jang-Hyun Kim, Jaesung Huh, Adrian Kim, Jung-Woo Ha, and Kyogu Lee,  “Phase-aware speech enhancement with deep complex u-net,”  in International Conference on Learning Representations, 2018.
  * [6] Yan Zhao, Zhong-Qiu Wang, and DeLiang Wang,  “Two-stage deep learning for noisy-reverberant speech enhancement,”  IEEE/ACM transactions on audio, speech, and language processing, vol. 27, no. 1, pp. 53–62, 2018.
  * [7] Alexandre Défossez, Gabriel Synnaeve, and Yossi Adi,  “Real time speech enhancement in the waveform domain,”  in Proc. Interspeech 2020, 2020, pp. 3291–3295.
  * [8] Xiang Hao, Xiangdong Su, Radu Horaud, and Xiaofei Li,  “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,”  in Proc. ICASSP, 2021, pp. 6633–6637.
  * [9] Han Zhao, Shuayb Zarar, Ivan Tashev, and Chin-Hui Lee,  “Convolutional-recurrent neural networks for speech enhancement,”  in Proc. ICASSP, 2018, pp. 2401–2405.
  * [10] Yuma Koizumi, Kohei Yatabe, Marc Delcroix, Yoshiki Masuyama, and Daiki Takeuchi,  “Speech enhancement using self-adaptation and multi-head self-attention,”  in Proc. ICASSP, 2020, pp. 181–185.
  * [11] Umut Isik, Ritwik Giri, Neerad Phansalkar, Jean-Marc Valin, Karim Helwani, and Arvindh Krishnaswamy,  “Poconet: Better speech enhancement with frequency-positional embeddings, semi-supervised conversational data, and biased loss,”  in Proc. Interspeech, 2020, pp. 2487–2491.
  * [12] Ashutosh Pandey and DeLiang Wang,  “Dense cnn with self-attention for time-domain speech enhancement,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1270–1279, 2021.
  * [13] Santiago Pascual, Antonio Bonafonte, and Joan Serra,  “Segan: Speech enhancement generative adversarial network,”  arXiv preprint arXiv:1703.09452, 2017.
  * [14] Szu-Wei Fu, Chien-Feng Liao, Yu Tsao, and Shou-De Lin,  “Metricgan: Generative adversarial networks based black-box metric scores optimization for speech enhancement,”  in Proc. ICML, 2019, pp. 2031–2041.
  * [15] Yoshiaki Bando, Masato Mimura, Katsutoshi Itoyama, Kazuyoshi Yoshii, and Tatsuya Kawahara,  “Statistical speech enhancement based on probabilistic integration of variational autoencoder and non-negative matrix factorization,”  in Proc. ICASSP, 2018, pp. 716–720.
  * [16] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,”  in Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 12449–12460.
  * [17] Weiran Wang, Qingming Tang, and Karen Livescu,  “Unsupervised pre-training of bidirectional speech encoders via masked reconstruction,”  in Proc. ICASSP, 2020, pp. 6889–6893.
  * [18] Andy T Liu, Shang-Wen Li, and Hung-yi Lee,  “Tera: Self-supervised learning of transformer encoder representation for speech,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2351–2366, 2021.
  * [19] Chien-Feng Liao, Yu Tsao, Hung-Yi Lee, and Hsin-Min Wang,  “Noise adaptive speech enhancement using domain adversarial training,”  arXiv preprint arXiv:1807.07501, 2018.
  * [20] Ryandhimas E Zezario, Tassadaq Hussain, Xugang Lu, Hsin-Min Wang, and Yu Tsao,  “Self-supervised denoising autoencoder with linear regression decoder for speech enhancement,”  in Proc. ICASSP, 2020, pp. 6669–6673.
  * [21] Aswin Sivaraman, Sunwoo Kim, and Minje Kim,  “Personalized speech enhancement through self-supervised data augmentation and purification,”  in Proc. Interspeech, 2021, pp. 2676–2680.
  * [22] Aswin Shanmugam Subramanian, Xiaofei Wang, Murali Karthick Baskar, Shinji Watanabe, Toru Taniguchi, Dung Tran, and Yuya Fujita,  “Speech enhancement using end-to-end speech recognition objectives,”  in Proc. WASPAA, 2019, pp. 234–238.
  * [23] Viet Anh Trinh and Sebastian Braun,  “Unsupervised speech enhancement with speech recognition embedding and disentanglement losses,”  arXiv preprint arXiv:2111.08678, 2021.
  * [24] Ying Cheng, Mengyu He, Jiashuo Yu, and Rui Feng,  “Improving multimodal speech enhancement by incorporating self-supervised and curriculum learning,”  in Proc. ICASSP, 2021, pp. 4285–4289.
  * [25] Xiaofei Li and Radu Horaud,  “Multichannel speech enhancement based on time-frequency masking using subband long short-term memory,”  in Proc. WASPAA, 2019, pp. 298–302.
  * [26] Kazuki Shimada, Yoshiaki Bando, Masato Mimura, Katsutoshi Itoyama, Kazuyoshi Yoshii, and Tatsuya Kawahara,  “Unsupervised speech enhancement based on multichannel nmf-informed beamforming for noise-robust automatic speech recognition,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 5, pp. 960–971, 2019.
  * [27] Scott Wisdom, Efthymios Tzinis, Hakan Erdogan, Ron J Weiss, Kevin Wilson, and John R Hershey,  “Unsupervised sound separation using mixture invariant training,”  in Proc. NeurIPS, 2020, vol. 33, pp. 3846–3857.
  * [28] Takuya Fujimura, Yuma Koizumi, Kohei Yatabe, and Ryoichi Miyazaki,  “Noisy-target training: A training strategy for dnn-based speech enhancement without clean speech,”  arXiv preprint arXiv:2101.08625, 2021.
  * [29] Efthymios Tzinis, Jonah Casebeer, Zhepei Wang, and Paris Smaragdis,  “Separate but together: Unsupervised federated learning for speech enhancement from non-iid data,”  in Proc. WASPAA, 2021, pp. 46–50.
  * [30] Koichi Saito, Stefan Uhlich, Giorgio Fabbro, and Yuki Mitsufuji,  “Training speech enhancement systems with noisy speech datasets,”  arXiv preprint arXiv:2105.12315, 2021.
  * [31] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz,  “Mixup: Beyond empirical risk minimization,”  in Proc. ICLR, 2018.
  * [32] Ryo Aihara, Toshiyuki Hanazawa, Yohei Okato, Gordon Wichern, and Jonathan Le Roux,  “Teacher-student deep clustering for low-delay single channel speech separation,”  in Proc. ICASSP, 2019, pp. 690–694.
  * [33] Wei-Ning Hsu, Yao-Hung Hubert Tsai, Benjamin Bolte, Ruslan Salakhutdinov, and Abdelrahman Mohamed,  “Hubert: How much can a bad teacher benefit asr pre-training?,”  in Proc. ICASSP, 2021, pp. 6533–6537.
  * [34] Jisi Zhang, Cătălin Zorilă, Rama Doddipatla, and Jon Barker,  “Teacher-Student MixIT for Unsupervised and Semi-Supervised Speech Separation,”  in Proc. Interspeech, 2021, pp. 3495–3499.
  * [35] Sunwoo Kim and Minje Kim,  “Test-time adaptation toward personalized speech enhancement: Zero-shot learning with knowledge distillation,”  in Proc. WASPAA, 2021, pp. 176–180.
  * [36] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le,  “Unsupervised data augmentation for consistency training,”  Proc. NeurIPS, vol. 33, pp. 6256–6268, 2020.
  * [37] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel,  “Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring,”  in Proc. ICLR, 2019.
  * [38] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li,  “Fixmatch: Simplifying semi-supervised learning with consistency and confidence,”  in Proc. NeurIPS, 2020, vol. 33, pp. 596–608.
  * [39] Zhepei Wang, Ritwik Giri, Umut Isik, Jean-Marc Valin, and Arvindh Krishnaswamy,  “Semi-supervised singing voice separation with noisy self-training,”  in Proc. ICASSP, 2021, pp. 31–35.
  * [40] Antti Tarvainen and Harri Valpola,  “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,”  in Proc. NeurIPS, 2017, vol. 30.
  * [41] Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, and Takaaki Hori,  “Momentum pseudo-labeling for semi-supervised speech recognition,”  in Proc. Interspeech, 2021, pp. 726–730.
  * [42] Qiantong Xu et al.,  “Iterative pseudo-labeling for speech recognition,”  in Proc. Interspeech, 2020, pp. 1006–1010.
  * [43] Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le,  “Self-training with noisy student improves imagenet classification,”  in Proc. CVPR, 2020, pp. 10684–10695.
  * [44] Scott Wisdom, John R Hershey, Kevin Wilson, Jeremy Thorpe, Michael Chinen, Brian Patton, and Rif A Saurous,  “Differentiable consistency constraints for improved deep speech enhancement,”  in Proc. ICASSP, 2019, pp. 900–904.
  * [45] Dong Yu, Morten Kolbæk, Zheng-Hua Tan, and Jesper Jensen,  “Permutation invariant training of deep models for speaker-independent multi-talker speech separation,”  in Proc. ICASSP, 2017, pp. 241–245.
  * [46] Chandan KA Reddy et al.,  “The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,”  in Proc. Interspeech, 2020, pp. 2492–2496.
  * [47] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur,  “Librispeech: an asr corpus based on public domain audio books,”  in Proc. ICASSP, 2015, pp. 5206–5210.
  * [48] Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra,  “Fsd50k: an open dataset of human-labeled sound events,”  arXiv preprint arXiv:2010.00475, 2020.
  * [49] Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter,  “Audio set: An ontology and human-labeled dataset for audio events,”  in Proc. ICASSP, 2017, pp. 776–780.
  * [50] Gordon Wichern, Joe Antognini, Michael Flynn, Licheng Richard Zhu, Emmett McQuinn, Dwight Crow, Ethan Manilow, and Jonathan Le Roux,  “WHAM!: Extending Speech Separation to Noisy Environments,”  in Proc. Interspeech, 2019, pp. 1368–1372.
  * [51] Ritwik Giri, Umut Isik, and Arvindh Krishnaswamy,  “Attention wave-u-net for speech enhancement,”  in Proc. WASPAA, 2019, pp. 249–253.
  * [52] Junichi Yamagishi, Christophe Veaux, and Kirsten MacDonald,  “CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),” 2019.
  * [53] Joachim Thiemann, Nobutaka Ito, and Emmanuel Vincent,  “The diverse environments multi-channel acoustic noise database (demand): A database of multichannel environmental noise recordings,”  in Proc. ICA, 2013.
  * [54] Efthymios Tzinis, Zhepei Wang, and Paris Smaragdis,  “Sudo rm-rf: Efficient networks for universal audio source separation,”  in Proc. MLSP, 2020, pp. 1–6.
  * [55] Yi Luo, Cong Han, and Nima Mesgarani,  “Ultra-lightweight speech separation via group communication,”  in Proc. ICASSP, 2021, pp. 16–20.
  * [56] Efthymios Tzinis, Zhepei Wang, Xilin Jiang, and Paris Smaragdis,  “Compute and memory efficient universal sound source separation,”  Journal of Signal Processing Systems, 2021.
  * [57] Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R Hershey,  “Sdr–half-baked or well done?,”  in Proc. ICASSP, 2019, pp. 626–630.
  * [58] Diederik P Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [59] Efthymios Tzinis, Shrikant Venkataramani, Zhepei Wang, Cem Subakan, and Paris Smaragdis,  “Two-step sound source separation: Training on learned latent targets,”  in Proc. ICASSP, 2020, pp. 31–35.
  * [60] Neil Zeghidour and David Grangier,  “Wavesplit: End-to-end speech separation by speaker clustering,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2840–2849, 2021.
  * [61] Cem Subakan, Mirco Ravanelli, Samuele Cornell, Mirko Bronzi, and Jianyuan Zhong,  “Attention is all you need in speech separation,”  in Proc. ICASSP, 2021, pp. 21–25.
  * [62] Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen,  “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,”  IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 7, pp. 2125–2136, 2011.
  * [63] Antony W Rix, John G Beerends, Michael P Hollier, and Andries P Hekstra,  “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,”  in Proc. ICASSP, 2001, pp. 749–752.


Corresponding author: Thomas L. R. Brien (envelope-o<EMAIL_ADDRESS>phone +44 (0) 29 20876269)

# Pre-deployment Verification and Predicted Mapping Speed of MUSCAT

T. L. R. Brien  School of Physics & Astronomy, Cardiff University, The Parade,
CF24 3AA, Cardiff, United Kingdom  P. A. R. Ade  School of Physics &
Astronomy, Cardiff University, The Parade, CF24 3AA, Cardiff, United Kingdom
P. S. Barry  Argonne National Laboratory, 9700 S. Cass Ave., Lemont IL, United
States of America  University of Chicago, 5640 S. Ellis Ave., Chicago IL,
United States of America  E. Castillo-Domínguez  SRON-Netherlands Institute
for Space Research, Landleven 12, 9747 AD Groningen, Netherlands  D. Ferrusca
Instituto Nacional de Astrofísica, Óptica y Electrónica, Luis Enrique Erro 1,
Santa María Tonantzintla 72840 Puebla, Mexico  V. Gómez-Rivera  Instituto
Nacional de Astrofísica, Óptica y Electrónica, Luis Enrique Erro 1, Santa
María Tonantzintla 72840 Puebla, Mexico  P. Hargrave  School of Physics &
Astronomy, Cardiff University, The Parade, CF24 3AA, Cardiff, United Kingdom
J. L. Hernández Rebollar  Instituto Nacional de Astrofísica, Óptica y
Electrónica, Luis Enrique Erro 1, Santa María Tonantzintla 72840 Puebla,
Mexico  A. Hornsby  School of Physics & Astronomy, Cardiff University, The
Parade, CF24 3AA, Cardiff, United Kingdom  D. H. Hughes  Instituto Nacional de
Astrofísica, Óptica y Electrónica, Luis Enrique Erro 1, Santa María
Tonantzintla 72840 Puebla, Mexico  J. M. Jáuregui-García  Instituto Nacional
de Astrofísica, Óptica y Electrónica, Luis Enrique Erro 1, Santa María
Tonantzintla 72840 Puebla, Mexico  P. Mauskopf  Arizona State University,
Tempe, Arizona, United States of America  D. Murias  Instituto Nacional de
Astrofísica, Óptica y Electrónica, Luis Enrique Erro 1, Santa María
Tonantzintla 72840 Puebla, Mexico  A. Papageorgiou  School of Physics &
Astronomy, Cardiff University, The Parade, CF24 3AA, Cardiff, United Kingdom
E. Pascale  Dipartimiento di Fisica, La Sapienza Universitá di Roma, Piazzale
Aldo Moro 5, 00185 Roma, Italy  A. Pérez  Instituto Nacional de Astrofísica,
Óptica y Electrónica, Luis Enrique Erro 1, Santa María Tonantzintla 72840
Puebla, Mexico  S. Rowe  School of Physics & Astronomy, Cardiff University,
The Parade, CF24 3AA, Cardiff, United Kingdom  M. W. L. Smith  School of
Physics & Astronomy, Cardiff University, The Parade, CF24 3AA, Cardiff, United
Kingdom  M. Tapia  Instituto Nacional de Astrofísica, Óptica y Electrónica,
Luis Enrique Erro 1, Santa María Tonantzintla 72840 Puebla, Mexico  C. Tucker
School of Physics & Astronomy, Cardiff University, The Parade, CF24 3AA,
Cardiff, United Kingdom  M. Velázquez  Instituto Nacional de Astrofísica,
Óptica y Electrónica, Luis Enrique Erro 1, Santa María Tonantzintla 72840
Puebla, Mexico  S. Ventura  Instituto Nacional de Astrofísica, Óptica y
Electrónica, Luis Enrique Erro 1, Santa María Tonantzintla 72840 Puebla,
Mexico  S. Doyle  School of Physics & Astronomy, Cardiff University, The
Parade, CF24 3AA, Cardiff, United Kingdom

###### Abstract

The Mexico-UK Submillimetre Camera for AsTronomy (MUSCAT) is a
$1.1\text{\,}\mathrm{mm}$ receiver consisting of 1,500 lumped-element kinetic
inductance detectors (LEKIDs) for the Large Millimeter Telescope (LMT; Volcán
Sierra Negra in Puebla, México). MUSCAT utilises the maximum field of view of
the LMT’s upgraded 50-metre primary mirror and is the first México-UK
collaboration to deploy a millimetre/sub-mm receiver on the Large Millimeter
Telescope. Using a simplistic simulator, we estimate a predicted mapping speed
for MUSCAT by combining the measured performance of MUSCAT with the observed
sky conditions at the LMT. We compare this to a previously calculated
bolometric-model mapping speed and find that our mapping speed is in good
agreement when this is scaled by a previously reported empirical factor.
Through this simulation we show that signal contamination due to sky
fluctuations can be effectively removed through the use of principle component
analysis. We also give an overview of the instrument design and explain how
this design allows for MUSCAT to be upgraded and act as an on-sky
demonstration testbed for novel technologies after the facility-class TolTEC
receiver comes online.

##  1 Introduction

The Mexico-UK Submillimetre Camera for AsTronomy (MUSCAT) is the first of a
suite of new receivers scheduled to be commissioned at the Large Millimeter
Telescope (LMT) on Volcán Sierra Negra in Puebla, México. The principal
application of MUSCAT is to carry out high spatial resolution observations
with class-leading mapping speeds. The focal plane of MUSCAT has 1,500 single-
colour, $1.1\mbox{-}$\mathrm{mm}$$ lumped-element kinetic inductance
detectors[1] (LEKIDs) operating at the background limit determined by the
atmospheric loading.

(a)

(b)

Figure 1: LABEL:sub@fig:LMT_map Location of the Large Millimeter Telescope.
Prepared with Cartopy[2]. LABEL:sub@fig:h-atlas_fields The 5 _H_ -ATLAS fields
as viewed from the LMT on an arbitrary night during the good weather season.
The solid-shaded regions show the passage of the fields across the sky during
astronomical night and the hatched areas show the instantaneous area covered
by each field. In descending order of azimuth, the three GAMA fields shown are
GAMA9, GAMA12 and GAMA15. Positions are calculated using the coordinates
module of Astropy.[3]

In its first semester on sky, MUSCAT will carry out high-resolution follow-up
observations of sources identified in the _Herschel_ -ATLAS[4] (_H_ -ATLAS)
survey. Of the 113,990 sources identified by _H_ -ATLAS in the GAMA9, GAMA12
and GAMA15 fields, only $39~{}$\mathrm{\char 37\relax}$$ of those with a
signal-to-noise ratio (SNR) of greater than four have an optical counterpart
allocated.[4, 5] In the north galactic plane (NGP), only
$36.4~{}$\mathrm{\char 37\relax}$$ of the 112,155 sources meeting the same SNR
threshold have optical counterparts assigned.[6, 4] The LMT’s site on Volcán
Sierra Negra ($$$~{}\mathrm{N}$ $$$~{}\mathrm{W}$) offers good coverage of
both of these fields as seen in Figure 1b which also shows the south galactic
plane (SGP). The three GAMA fields are each $12\times 3~{}$\deg$$, the NGP is
$15\times 10~{}$\deg$$.

A major limiting factor in the assignment of counterparts to faint sources
detected by _H_ -ATLAS is the confusion noise[4, 5]. This limit is due to
spatially over-lapping emission of extra-galactic sources due to the limited
resolution as the wavelength increases into the far-infrared/sub-mm regime.
This problem can however be greatly reduced with improved angular resolution,
with the confusion limit scaling as $\left(\lambda/D\right)^{2.5}$, where
$\lambda$ is the observation wavelength and $D$ is the diameter of the
telescope’s primary aperture.[7] For the case of _Herschel_ , the angular
resolution is defined by its $3.5~{}$\mathrm{m}$$ primary mirror,[8] giving an
angular resolution of $None$ at a wavelength of
$250~{}$\mathrm{\SIUnitSymbolMicro m}$$ and a confusion limit of $\sim
3~{}$\mathrm{m}\mathrm{Jy}\mathrm{/}\mathrm{b}\mathrm{e}\mathrm{a}\mathrm{m}$$.[9]
The LMT and its recently upgraded $50~{}$\mathrm{m}$$ primary mirror, offers
an angular resolution of $None$ allowing for substantial lowering of the
confusion limit. The optical design of MUSCAT maximises the $None$ diameter
field of view of LMT, making MUSCAT particularly powerful for high-resolution
mapping and follow-up surveys.

##  2 Instrument Design

The MUSCAT receiver is based on a classic Matryoshka doll (Russian doll)
cryostat, consisting of concentric radiation shields at
$450\text{\,}\mathrm{mK}$, $4\text{\,}\mathrm{K}$, and
$50\text{\,}\mathrm{K}$, all nested inside an outer vacuum can (OVC). Figure
2a shows a cross-sectioned CAD model of the MUSCAT cryostat and Figure 2b
shows the completed MUSCAT instrument during lab verification with the OVC and
radiation shields removed.

(a)

(b)

Figure 2: LABEL:sub@fig:MUSCAT_xSection Cross section of the MUSCAT CAD model
showing the nested radiation shields and the cold mirrors mounted to the
$450\text{\,}\mathrm{mK}$ stage. LABEL:sub@fig:MUSCAT_photo The built system
in the lab with the vacuum can and radiation shields removed, the focal plane
can be seen to the right of the photo about two-thirds down.

MUSCAT uses a fully reflective optics train which provides high-quality,
diffraction-limited beams in MUSCAT’s $1.1\text{\,}\mathrm{mm}$ band and also
at $850\text{\,}\mathrm{\SIUnitSymbolMicro m}$ as described in Brien et
al.[10] In Figure 2 the final two mirrors (M7 and M8) can be seen mounted to
the $450\text{\,}\mathrm{mK}$ plate.

The design of MUSCAT’s cooling system has been previously described[11] and
utilises four different cooling systems to produce a continuous temperature,
under the final loading, of $133\text{\,}\mathrm{mK}$ at the focal plane.
Initial cooling of the 50- and 4-kelvin stages is performed using a Cryomech
PT-420-RM which provides $2.0\text{\,}\mathrm{W}$ of cooling at
$4.2\text{\,}\mathrm{K}$. A 1-kelvin stage is cooled by a Chase Research
Cryogenics ^4He continuous sorption cooler (CC4); this stage does not include
a radiation shield and serves to buffer colder stages from thermal loading via
the mechanical supports to the $4\text{\,}\mathrm{K}$ stage. A further Chase
Research Cryogenics continuous sorption cooler using ^3He evaporators in
addition to ^4He pre-cooling stages (CC7), is used to cool the
$450\text{\,}\mathrm{mK}$ stage. Finally, a miniature dilution refrigerator
(MDR) is used to cool the focal plane to $\sim$130\text{\,}\mathrm{mK}$$. It
should be noted that while the CC7 stage runs continuously at a much higher
temperature than the performance reported by Klemencic et al.,[12] this is due
to operating under a substantial thermal load, approximately
$650\text{\,}\mathrm{\SIUnitSymbolMicro W}$, which is due to thermal loading
from the $4\text{\,}\mathrm{K}$ and $1\text{\,}\mathrm{K}$ stages and also
from heating the ^3He evaporator of the miniature dilution refrigerator.

The first-generation MUSCAT focal plane design contains 1,500 LEKIDs which are
distributed across three $100\text{\,}\mathrm{mm}$ silicon wafers with two
sub-arrays (250 pixels) on each wafer. Each sub-array is read out on a single
RF channel (MUX ratio 250:1). The detectors have been shown[13] to operate at
the background limit for source temperatures above $10\text{\,}\mathrm{K}$.
The focal plane has been shown to have a science-grade yield of in excess of
$75\text{\,}\mathrm{\char 37\relax}$.[14]

###  2.1 Focal Plane Vibration Mitigation

As described in Section 2, the MUSCAT focal plane is distributed across three
$100\text{\,}\mathrm{mm}$ silicon wafers. In order to minimise the gap between
wafers, the diced wafers are only clamped along their short edges as seen in
Figure 3a. Early testing of MUSCAT showed a substantial level of vibrational
pickup in the noise power spectrum which is illustrated by the red trace in
Figure 3c where a cluster of lines are seen from approximately
$500\text{\,}\mathrm{Hz}$ and the spectrum is dominated by $1/f$ noise below
$30\text{\,}\mathrm{Hz}$. To rectify this, a set of four spring-loaded pins
were added to each detector module. A cross section of a single pin is shown
in Figure 3b. The springs were custom made from beryllium copper and designed
to apply a force of $1\text{\,}\mathrm{N}$ each when compressed by
$3\text{\,}\mathrm{mm}$. The pin itself was made from G10 glass epoxy selected
to minimise the risk of damaging the detector wafer. The pin is compressed by
an aluminium cap which is bolted to the detector block (also aluminium) and
incorporates a short post to ensure the spring is only free to move in the
compression direction.

(a)

(b)

(c)

Figure 3: LABEL:sub@fig:detecotrModule View of one of the three focal plane
modules (with the back short removed). LABEL:sub@fig:pogoPin Cross sectioned
CAD model of a vibration suppressing pin, shown in uncompressed state. When
compressed, the top cap is secured to the back short by three equally-spaced
bolts, the pin contacts directly to the detector wafer.
LABEL:sub@fig:vibrationPSDs Power spectral densities without (red, background)
and with (black, foreground) the vibration suppressing pins. For a complete
discussion of the noise spectra, we direct readers to Tapia et al.[14]

The black trace in Figure 3c shows the noise spectrum of the same detector
measured after the spring-loaded pins were added. All of the high-frequency
noise due to vibration has been removed except a single line feature at
$7.5\text{\,}\mathrm{kHz}$ which is well outside of the MUSCAT readout band
($500\text{\,}\mathrm{Hz}$), the $1/f$ noise has also been removed with the
spectrum now flat to $\sim$1\text{\,}\mathrm{H}\mathrm{z}$$.

##  3 Instrument Platform Verification & Performance

The cryogenic platform, as described in Section 2, has shown excellent
performance in terms of stability and minimum temperature as well as
usability. Figure 4 shows the cool down of the system from room temperature to
a minimum detector temperature of $\sim$130\text{\,}\mathrm{mK}$$ after 80
hours. The oscillations in the $4\text{\,}\mathrm{K}$ and
$450\text{\,}\mathrm{mK}$ stages after $t=$60\text{\,}\mathrm{h}$$ is due to
the operation of the CC4 and CC7 systems. Circulation of ^3He in the MDR
commences at approximately $t=$75\text{\,}\mathrm{h}$.$

Figure 4: Cooldown of the MUSCAT instrument platform from room temperature to
continuous operation temperature ($130\text{\,}\mathrm{mK}$). The operating
temperature is achieved after approximately 80 hours. The oscillations in
temperature seen in some stages after $t=$60\text{\,}\mathrm{h}$$ are due to
the continuous recycling of the $1\text{\,}\mathrm{K}$ and
$450\text{\,}\mathrm{mK}$ stage coolers. The four stages shown (in descending
order of temperature at $t=$100\text{\,}\mathrm{h}$$) are the
$50\text{\,}\mathrm{K}$, $4\text{\,}\mathrm{K}$ and $450\text{\,}\mathrm{mK}$
stages and the final (coldest) trace is the detector temperature.

During prolonged lab-based verification work, MUSCAT maintained a focal-plane
temperature of $\left(132.6\pm 0.7\right)~{}$\mathrm{mK}$$ over a period of in
excess of seven weeks. This shows the cooling platform offers stable
temperatures on timescales in excess of those required for observatory
operations. The oscillations seen in the $4\text{\,}\mathrm{K}$ stage (green
trace) after $t=60~{}$\mathrm{h}$$ in Figure 4 are a result of the heating and
rapid cooling via gas-gap heat switches of the sorption pumps in the
continuous coolers. The similar oscillations seen in the
$450\text{\,}\mathrm{mK}$ are more extreme examples of the fluctuations
already reported for this type of continuous sorption cooler when operating
without a PID controlled stabilisation heater.[12] The fluctuations seen in
MUSCAT have a higher amplitude than those reported previously for this type of
fridge, this is a result of the high thermal load the
$450\text{\,}\mathrm{mK}$ stage is subjected to. This also causes the elevated
temperature at which this stage operates compared to the ultimate performance
of the continuous sorption coolers. These oscillations are not present in the
focal plane (purple trace in Figure 4) as the circulation rate of ^3He in the
miniature dilution refrigerator is much slower than the oscillation period of
the the fluctuations and effectively acts to dampen the fluctuation at the
focal plane.

##  4 Preliminary Estimation of Mapping Speed

To support the development of software for science operations with MUSCAT and
to help us optimise scan strategies, we have developed a simplistic mapping
simulator. This simulator combines the measured detector and focal plane
performance, obtained through a thorough programme of lab-based verification
as reported by Gómez-Rivera et al.[13] and Tapia et al.[14] as well as the
known sky conditions at the LMT site.[15, 16]

(a)

(b)

Figure 5: Results from Tapia et al.[14] LABEL:sub@fig:beammap Stacked and
centred beam maps of two sub arrays. LABEL:sub@fig:NEP_PSD Noise equivalent
power (NEP) of a typical pixel in the MUSCAT focal plane. The faint blue trace
in the background shows the raw NEP spectrum prior to cleaning with principal
component analysis (PCA), the black trace shows the NEP after PCA cleaning.
With PCA the $1/f$-noise knee frequency is typically reduced from
$1.6\text{\,}\mathrm{Hz}$ to $0.2\text{\,}\mathrm{Hz}$. The measured NEP
spectrum is in good agreement with the expected photon noise level (dashed
line) of the room-temperature black body observed during these measurements.
_Inset:_ Reduction in $1/f$-noise knee frequency with number of PCA
components. (Colour online)

Figure 5 shows some of the key results from lab-verification used to build a
mapping simulator. Figure 5a shows the combined beam maps for two sub-arrays
($\sim 500$ pixels) and Figure 5b shows a typical noise spectrum, with and
without cleaning via principal component analysis (PCA). For the purpose of
estimating the mapping speed we note that the measured beams can be
approximately circular down to a level of $-15~{}$\mathrm{dB}$$ and that the
noise spectra have a $1/f$ knee frequency of $0.2~{}$\mathrm{Hz}$$ when
cleaned using PCA. Furthermore, while Tapia et al.[14] describe the
performance of MUSCAT observing a $>$300\text{\,}\mathrm{K}$$ black body, we
note that the single-pixel results reported by Gómez-Rivera et al.[13] show
that the $1.6\text{\,}\mathrm{Hz}$ $1/f$ knee (prior to PCA cleaning) and
photon-noise limited are also observed for black body temperatures in the
range $10\mbox{--}70~{}$\mathrm{K}$$ which is representative of the expected
effective sky temperature at the the LMT’s site.

###  4.1 Description of Observation Simulation

The simulated astronomical scene is a collection of $5\text{\,}\mathrm{mJy}$
sources arranged in a uniform two-dimensional grid pattern against a noiseless
background. These dummy point sources have been modelled by convolving a delta
function with the measured beam profile (see Figure 5a) and sampled on a grid
spacing of $0.05\times\mathit{FWHM}_{\mathrm{beam}}$. Figure 6a shows this
initial scene. A single brighter source is located close to the centre of the
map for use as a datum.

The scene is mapped using a raster-scan pattern at a rate achievable by the
LMT ($None$${\mathrm{s}}^{-1}$) as shown in Figure 6a. Beam offsets are taken
from the lab-measured data and only the pixels found by Tapia et al. are used.
This scan strategy produces a mean integration time of
$0.18\text{\,}\mathrm{s}\mathrm{/}\mathrm{p}\mathrm{i}\mathrm{x}$ (see Figure
6b) in a $$$\times$$$ square in the centre of the map and takes
$29.7\text{\,}\mathrm{s}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{s}$.

(a)

(b)

Figure 6: LABEL:sub@fig:scene Example astronomical scene used in the
simulation consisting of regularly spaced $5\text{\,}\mathrm{mJy}$ sources
which are produced by convolving delta functions with the beam profile shown
in Figure 5a. This figure is the recovered map from a simulation without the
addition of instrument or sky noise. The transparent grey line shows the
boresight path of the raster scan. LABEL:sub@fig:sim_int Integration time, per
pixel, across the mapped region.

The response to our simulated sources is calculated using the measured
responsivity curves (see Gómez-Rivera et al.[13]) and produces the clean
timestream shown by the black trace in Figure 7. This timestream is then
contaminated by a typical sky-noise spectrum (D. Sánchez-Arguelles 2019, LMT
internal communication) for $\tau=0.06$ and the measured detector and system
noise of MUSCAT for each individual pixel[14], this produces the red trace in
Figure 7. Finally, the contaminated timestream is cleaned at the sub-array
level using principle component analysis (green trace in Figure 7).

###  4.2 Simulation Assumptions

In order to covert the noise power spectra and responsivity reported by Tapia
et al.[14] and Gómez-Rivera et al.[13] into flux density units, we use the
following:

$\displaystyle F_{$\mathrm{Jy}$}$ $\displaystyle=\frac{P\cdot
10^{26}}{A_{\mathrm{tel}}\cdot\delta\nu\cdot\eta_{\mathrm{opt}}}\,,$ (1)
$\displaystyle S_{$\mathrm{Hz}\mathrm{/}\mathrm{Jy}{}$}$
$\displaystyle=S_{$\mathrm{Hz}\mathrm{/}\mathrm{p}\mathrm{W}{}$}\cdot
A_{\mathrm{tel}}\cdot\delta\nu\cdot\eta_{\mathrm{opt}}\cdot 10^{-14}\,.$ (2)

Where $F_{$\mathrm{Jy}$}$ is the flux density in Janskys, $P$ is the optical
power, $A_{\mathrm{tel}}$ is the collecting area of the telescope, $\delta\nu$
is the receiver’s optical bandwidth, $\eta_{\mathrm{opt}}$ is the observation
optical efficiency, and $S$ is the responsivity in the units indicated. The
factor of $10^{-14}$ in Equation 2 combines the conversion from Watts to
Janskys and also from picowatts to Watts.

We assume that the entire collecting area of the LMT is utilised, giving
$A_{\mathrm{tel}}=1960~{}$\mathrm{m}^{2}$$; an optical bandwidth of
$\delta\nu=50~{}$\mathrm{GHz}$$ according to Tapia et al.[14] We use a
constant value of $\eta_{\mathrm{opt}}=0.7$ which is dominated by the sky—the
instrumental efficiency is accounted for in responsivity. As mentioned in
Section 4.1, we assume a constant opacity of $\tau=0.06$ during the
observation, this lies within the reported interquartile range for the good-
weather season at the LMT.[15, 16, 10] Finally, we assume that all detectors
see the same random sky noise.

###  4.3 Simulation Results

Figure 7 shows the single-pixel timestream for a single azimuth scan across
the scene. The flux due to the scene (black trace) is almost entirely lost in
the contaminated response (red trace), however it is well recovered after PCA
cleaning of the signal (green trace). Combining these timestreams for each
azimuth scan with the known pointing information produces the map seen in
Figure 8 where the regular pattern of sources can be seen.

Figure 7: Example of a simulation timestream while mapping the simulated
astronomical scene without noise (black trace), after contamination by sky and
system noise (red trace) and after PCA cleaning (green trace). Figure 8:
Reconstructed map of evenly-spaced, $5\text{\,}\mathrm{mJy}$ sources after
contamination by instrument and sky noise. Smoothed with a first-order
Gaussian filter for clarity.

To calculate the map sensitivity, we first calculate the RMS of the mapped
regions known not to contain a source, correcting for the known integration
time at each point (see Figure 6b). This produces a map sensitivity of
$0.95~{}$\mathrm{m}\mathrm{Jy}\mathrm{/}\mathrm{s}^{0.5}$$. From this
calculated mapping sensitivity, we find a simulated mapping speed of
$0.63~{}$\deg^{2}\mathrm{/}\mathrm{m}\mathrm{Jy}^{2}\mathrm{/}\mathrm{h}$$ for
the scan rate of $None$${\mathrm{s}}^{-1}$.

This simulated mapping speed is comparable to the theoretically calculated
mapping speed of
$3.0~{}$\deg^{2}\mathrm{/}\mathrm{m}\mathrm{Jy}^{2}\mathrm{/}\mathrm{h}$$
reported by Castillo-Dominguez et al.[17] using a purely bolometric model
scaled by the factor $\sim 7$ found by comparing the bolometric mapping speed
of the AzTEC camera to the observed performance on sky at the James Clark
Maxwell (JCMT) telescope.[18] Scaling Castillo-Dominguez et al.’s mapping
speed by this factor gives a mapping speed of
$0.42~{}$\deg^{2}\mathrm{/}\mathrm{m}\mathrm{Jy}^{2}\mathrm{/}\mathrm{h}$$.

##  5 Status of MUSCAT

At the time of publication, MUSCAT has completed its programme of lab-based
pre-deployment verification with the results described Gómez-Rivera et
al.,[13] Tapia et al.,[14] as well as those herein. Based on these results,
the MUSCAT collaboration have determined that the instrument is functional and
fit for purpose and MUSCAT is currently awaiting shipping to Mexico with no
further lab-based studies required. Upon arrival at the LMT, MUSCAT will
complete a commissioning programme to confirm that the on-sky performance is
in agreement with the results of the lab-based verification. MUSCAT will then
complete its 100-hour initial science run.

Figure 9: Model of the LMT receiver cabin showing the space that will be
occupied by the MUSCAt receiver and key infrastructure. The SuperSpec and
TolTEC instruments are also shown immediately to the left and right of MUSCAT
respectively.

The LMT receiver cabin has been made ready for MUSCAT with the installation of
warm mirror mounts and clearing of cabling and pulse tube line routes. Figure
9 shows a model of how MUSCAT will be located in the LMT receiver cabin and
how MUSCAT can cohabit this space with the TolTEC[19, 20] and SuperSpec[21]
instruments.

##  6 The Future of MUSCAT

TolTEC is the second-generation, facility-class continuum receiver for the LMT
and offers multi-colour and polarisation capabilities and will surpass the
capabilities of MUSCAT in the $1.1\text{\,}\mathrm{mm}$ band upon its arrival.
MUSCAT however is designed to provide a versatile platform for the
demonstration of innovative technologies, such as a new class of mm-wave on-
chip spectrometers, as well as novel KID-based readout platforms that promise
significant increases in multiplexing densities. With a set of straightforward
upgrades, MUSCAT is well placed to extend the capabilities offered at the LMT.
As a concrete example, the design and quality of the optics make MUSCAT a
platform capable of extending continuum observations at the LMT out to the
$850\text{\,}\mathrm{\SIUnitSymbolMicro m}$ band. Through a relatively simple
replacement of the focal plane and band-defining filters, MUSCAT could be
upgraded to offer mapping capabilities in this band—a facility the LMT does
not currently have.

##  7 Conclusion

We have shown that development of the MUSCAT instrument for the LMT has been
completed, that the system provides a stable and reliable cryogenic platform
and that the results of a thorough programme of lab-based verification studies
show that MUSCAT is capable of carrying out high-quality astronomical
observations with an anticipated mapping speed of order
$0.63\text{\,}\deg^{2}\mathrm{/}\mathrm{m}\mathrm{Jy}^{2}\mathrm{/}\mathrm{h}$
which is in broad agreement with with a previously reported bolometric model
of the the mapping speed when scaled by an accepted empirical factor. Finally,
we have explained how MUSCAT can be upgraded in the future to offer further
and novel astronomical capabilities at the LMT.

## Acknowledgements

We acknowledge RCUK and CONACYT through the Newton Fund (grant no.
ST/P002803/1), CONACYT for supporting the fellowship for the instrument
scientist (grant no. 053), Chase Research Cryogenics for the development of
the sub-kelvin coolers and to XILINX Inc. for the donation of the FPGAs used
for the ROACH2 boards.

## References

  * [1] S. Doyle, P. Mauskopf, J. Naylon, A. Porch, and C. Duncombe, “Lumped element kinetic inductance detectors,” Journal of Low Temperature Physics 151(1-2), pp. 530–536, 2008.
  * [2] Met Office, Cartopy: a cartographic python library with a Matplotlib interface.  Exeter, Devon, 2010 - 2015.
  * [3] Astropy Collaboration, “The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package,” AJ 156, p. 123, Sept. 2018.
  * [4] E. Valiante, M. W. L. Smith, S. Eales, S. J. Maddox, E. Ibar, R. Hopwood, L. Dunne, P. J. Cigan, S. Dye, E. Pascale, E. E. Rigby, N. Bourne, C. Furlanetto, and R. J. Ivison, “The Herschel–ATLAS data release 1–I. Maps, catalogues and number counts,” Monthly Notices of the Royal Astronomical Society 462, pp. 3146–3179, 07 2016.
  * [5] N. Bourne, L. Dunne, S. Maddox, S. Dye, C. Furlanetto, C. Hoyos, D. Smith, S. Eales, M. W. L. Smith, E. Valiante, et al., “The Herschel-ATLAS data release 1–ii. multi-wavelength counterparts to submillimetre sources,” Monthly Notices of the Royal Astronomical Society 462(2), pp. 1714–1734, 2016.
  * [6] C. Furlanetto, S. Dye, N. Bourne, S. Maddox, L. Dunne, S. Eales, E. Valiante, M. Smith, D. J. B. Smith, R. J. Ivison, et al., “The second Herschel-ATLAS data release–iii. optical and near-infrared counterparts in the north galactic plane field,” Monthly Notices of the Royal Astronomical Society 476(1), pp. 961–978, 2018.
  * [7] G. Helou and C. A. Beichman, “The confusion limits to the sensitivity of submillimeter telescopes,” in Liege International Astrophysical Colloquia, B. Kaldeich, ed., Liege International Astrophysical Colloquia 29, pp. 117–123, Dec. 1990.
  * [8] G. Pilbratt, J. Riedinger, T. Passvogel, G. Crone, D. Doyle, U. Gageur, A. Heras, C. Jewell, L. Metcalfe, S. Ott, et al., “Herschel Space Observatory-an ESA facility for far-infrared and submillimetre astronomy,” Astronomy & Astrophysics 518, p. L1, 2010.
  * [9] M. W. Smith, E. Ibar, S. J. Maddox, E. Valiante, L. Dunne, S. Eales, S. Dye, C. Furlanetto, N. Bourne, P. Cigan, et al., “The Herschel–ATLAS data release 2, paper i. submillimeter and far-infrared images of the south and north galactic poles: the largest herschel survey of the extragalactic sky,” The Astrophysical Journal Supplement Series 233(2), p. 26, 2017\.
  * [10] T. L. Brien, P. A. Ade, P. S. Barry, E. Castillo-Domìnguez, D. Ferrusca, T. Gascard, V. Gómez, P. C. Hargrave, A. L. Hornsby, D. Hughes, et al., “MUSCAT: the mexico-uk sub-millimetre camera for astronomy,” in Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy IX, 10708, p. 107080M, International Society for Optics and Photonics, 2018.
  * [11] T. L. Brien, E. Castillo-Dominguez, S. Chase, and S. M. Doyle, “A continuous 100-mK helium-light cooling system for muscat on the lmt,” Journal of Low Temperature Physics 193(5-6), pp. 805–812, 2018.
  * [12] G. Klemencic, P. Ade, S. Chase, R. Sudiwala, and A. Woodcraft, “A continuous dry 300 mK cooler for THz sensing applications,” Review of Scientific Instruments 87(4), p. 045107, 2016.
  * [13] V. Gómez-Rivera, “Design and characterization of the MUSCAT detectors,” Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy X , International Society for Optics and Photonics, 2020.
  * [14] M. Tapia, “MUSCAT focal plane verifcation,” Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy X , International Society for Optics and Photonics, 2020.
  * [15] D. Ferrusca et al., “Weather monitor station and 225 GHz radiometer system installed at sierra negra: the large millimeter telescope site,” in Ground-based and Airborne Instrumentation for Astronomy V, 9147, p. 914730, International Society for Optics and Photonics, 2014.
  * [16] M. Zeballos, D. Ferrusca, D. Hughes, et al., “Reporting the first 3 years of 225-GHz opacity measurements at the site of the large millimeter telescope alfonso serrano,” in Ground-based and Airborne Telescopes VI, 9906, p. 99064U, International Society for Optics and Photonics, 2016\.
  * [17] E. Castillo-Dominguez, P. Ade, P. Barry, T. Brien, S. Doyle, D. Ferrusca, V. Gomez-Rivera, P. Hargrave, A. Hornsby, D. Hughes, et al., “Mexico-uk sub-millimeter camera for astronomy,” Journal of Low Temperature Physics 193(5-6), pp. 1010–1015, 2018.
  * [18] G. Wilson, J. Austermann, T. Perera, K. Scott, P. A. Ade, J. Bock, J. Glenn, S. Golwala, S. Kim, Y. Kang, et al., “The AzTEC mm-wavelength camera,” Monthly Notices of the Royal Astronomical Society 386(2), pp. 807–818, 2008.
  * [19] The TolTEC Collaboration, “The TolTEC Camera.” Available at https://toltec.astro.umass.edu/ (2020/11/30).
  * [20] G. W. Wilson, P. Ade, I. Aretxaga, J. E. Austermann, J. Bardin, P. Barry, J. Beall, M. Berthoud, A. Braeley, S. A. Bryan, et al., “The TolTEC project: a millimeter wavelength imaging polarimeter (Conference Presentation),” in Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy IX, 10708, p. 107080I, International Society for Optics and Photonics, 2018.
  * [21] E. Shirokoff, P. Barry, C. Bradford, G. Chattopadhyay, P. Day, S. Doyle, S. Hailey-Dunsheath, M. Hollister, A. Kovács, H. Leduc, et al., “Design and performance of SuperSpec: an on-chip, kid-based, mm-wavelength spectrometer,” Journal of Low Temperature Physics 176(5-6), pp. 657–662, 2014.