# Unsupervised Domain Adaptation of Black-Box Source Models

Haojian Zhang1, Yabin Zhang2, Kui Jia1, Lei Zhang2
1South China University of Technology 2The Hong Kong Polytechnic University
eehjzhang<EMAIL_ADDRESS>
kuijia<EMAIL_ADDRESS>
###### Abstract

Unsupervised domain adaptation (UDA) aims to learn models for a target domain
of unlabeled data by transferring knowledge from a labeled source domain. In
the traditional UDA setting, labeled source data are assumed to be available
for adaptation. Due to increasing concerns for data privacy, source-free UDA
is highly appreciated as a new UDA setting, where only a trained source model
is assumed to be available, while labeled source data remain private. However,
trained source models may also be unavailable in practice since source models
may have commercial values and exposing source models brings risks to the
source domain, e.g., problems of model misuse and white-box attacks. In this
work, we study a subtly different setting, named Black-Box Unsupervised Domain
Adaptation (B2UDA), where only the application programming interface of source
model is accessible to the target domain; in other words, the source model
itself is kept as a black-box one. To tackle B2UDA, we propose a simple yet
effective method, termed Iterative Learning with Noisy Labels (IterLNL). With
black-box models as tools of noisy labeling, IterLNL conducts noisy labeling
and learning with noisy labels (LNL), iteratively. To facilitate the
implementation of LNL in B2UDA, we estimate the noise rate from model
predictions of unlabeled target data and propose category-wise sampling to
tackle the unbalanced label noise among categories. Experiments on benchmark
datasets show the efficacy of IterLNL. Given neither source data nor source
models, IterLNL performs comparably with traditional UDA methods that make
full use of labeled source data.

##  1 Introduction

Although deep models have achieved success on various tasks, it is difficult
to generalize the model learned from labeled training data to a target domain
of slightly shifted data distribution. At the same time, it is expensive to
collect a new target dataset with a large number of labeled training data.
Therefore, unsupervised domain adaptation (UDA) [30, 24, 5] is introduced to
learn the target model by transferring knowledge from the labeled source
domain to the unlabeled target domain. Motivated by seminal theories [3, 49],
popular UDA methods [24, 5, 36, 25, 50] target at learning domain invariant
feature representations. The underlying motivation is that the source
classifier could be safely applied to the target domain once domain invariant
feature representations are achieved. In UDA, labeled source data are assumed
to be available for target domain.

Although remarkable success has been achieved in UDA, increasing concerns for
data privacy post new challenges to this task. Specifically, data of source
and target domains are typically captured and stored on different devices and
contain private information. Thus it is risky to expose source data to the
target domain and vice versa. In other words, labeled source data may be not
available for the target domain, impeding the application of popular UDA
methods [24, 5, 36, 25, 50]. For this reason, a novel task, source-free UDA,
is introduced [45, 13, 23] to facilitate the model adaptation and protect the
source data privacy simultaneously.

Unlike the vanilla UDA, a well-trained source model, instead of labeled source
data, is provided to unlabeled target domain in the source-free UDA [45, 23].
Specifically, a white-box source model is available for the target domain;
thus, we term this task as white-box unsupervised domain adaptation (WBUDA) to
distinguish it from our investigated one in later paragraphs. In WBUDA, the
adaptation could be achieved by fine-tuning the source model on unlabeled
target data with well-designed objectives [45, 23].

However, the white-box source model is not always given in practice. Most
valuable models on cloud services (e.g., Google Cloud) are sealed as
application programming interface (API), where only the input-output interface
of a model is available and the model itself is kept as a black-box one. As
stated in [29], releasing an API instead of a white-box model could
commercialize the technology, reduce model misuse and make the model use
conveniently for the public; the white-box attacks [38, 39] may be also
avoided. Due to all reasons mentioned above, white-box source models are
probably unavailable in practice, which hinders the application of WBUDA
methods.

In this work, we study a subtly different setting of source-free UDA, where
only the API of the source model is accessible for the target domain. In other
words, the source model itself is kept as a black-box one; thus, we term this
task as black-box unsupervised domain adaptation (B2UDA). A few recent
attempts [46, 4, 27] have been made to tackle the B2UDA problem, but achieving
less satisfactory results. In this work, we propose a simple yet effective
algorithmic framework, termed Iterative Learning with Noisy Labels (IterLNL).
With black-box models as tools of noisy labeling, IterLNL conducts noisy
labeling and LNL iteratively. Specifically, we first get model predictions of
target data based on the black-box model and obtain their noisy labels as the
category with the maximum prediction probability. We note that the label noise
via the black-box model is highly unbalanced among categories (cf. Figure
2(c)), which is significantly different from the simulated and balanced ones
in LNL [40, 10]; such unbalanced label noise hinders the application of state-
of-the-art LNL methods [15, 10], inspiring the category-wise sampling
strategy. To facilitate the implementation of LNL in B2UDA, we also estimate
the noise rate from model predictions of unlabeled target data. Experiments on
benchmark datasets confirm the efficacy of our method. Notably, our IterLNL
performs comparably with methods of traditional UDA setting where labeled
source data are fully available.

##  2 Related Work

Source Free UDA. Traditional UDA [24, 5] assumes that labeled source data are
available for the target domain. Due to increasing concerns for data privacy,
source-free UDA [23, 22, 13, 17, 18, 45] is highly appreciated as a new UDA
setting, where only a source model is available for the target domain while
labeled source data remain private. Source free UDA methods typically fine-
tune the source model for the target domain with unlabeled target data [23,
17, 45, 22]. Specifically, Liang _et al._ [23] fine-tune the source model with
pseudo-labeling and information maximization between target data and their
predictions; a weight constraint is adopted in [22] to encourage similarity
between the source model and adapted target model. Additionally, source data
and source-style target data are respectively generated in [18] and [13] using
the statistics information stored in source model. The white-box source model
is required in the methods above, but it may be unavailable due to the
commercial and/or safety consideration [29]. To this end, we study a subtly
different B2UDA setting, where only the API of source model is accessible for
the target domain; in other words, the source model itself is kept as a black-
box one. We note that several attempts have been made on the B2UDA problem
recently. Based on pre-trained features, a denoising auto-encoder is used for
prediction of target labels in [4] and an encoder-decoder framework is used in
[46] where encoded target features are aligned to reference Gaussian
distributions; however, both of the two methods obtain less satisfactory
results on benchmark datasets. Morerio et al. [27] first train a conditional
Generative Adversarial Network (cGAN) with unlabeled target data and their
source predictions, and then learn the target model with samples generated by
cGAN; its performance is conditioned on the high-quality samples generated by
cGAN, thus limiting its general usage in UDA tasks. In general, the B2UDA
problem is not well addressed yet. In the present work, we propose IterLNL and
conduct thorough experiments on popular UDA benchmarks; results show that our
proposed method works successfully for B2UDA.

Figure 1: An illustration of different UDA settings. Source data and source
model are respectively required in the traditional UDA and WBUDA settings. In
contrast, B2UDA requires a black-box access to the source model only, which is
the least restrictive condition to apply domain adaptation to the unsupervised
target data.

Learning with Noisy Labels (LNL). LNL aims to learn models with noisy labeled
data robustly. Seminal LNL methods include estimating noise transition matrix
to transfer observed noisy labels to latent clean ones [37, 41], refining the
objective function [7, 51], and avoiding overfitting noisy labels with
memorization effects of neuron networks [15, 10], as summarized in [9].
Although we also adopt LNL technical tools for the B2UDA problem, the task
divergence between LNL and B2UDA raises new problems. Specifically, as
predictions of black-box models are adopted as noisy labels, the noise rate is
unknown and the label noise is unbalanced among categories in B2UDA (cf.
Figure 2(c)), while simulated and balanced noisy labels with known noise rate
are typically adopted in LNL [40, 8, 10]. Besides, noisy labels are given and
fixed in LNL, while we obtain noisy labels from predictions of black-box
models and could update noisy labels via updating models, inspiring the
iterative learning strategy.

Hypothesis Transfer Learning (HTL). HTL [19] utilizes source hypotheses (i.e.,
source models) to assist learning in the target domain. Different from the
B2UDA task, at least a small number of labeled target samples are demanded and
source hypotheses are typically required to be white-box ones in HTL methods
[19, 47, 1], hindering their applications in B2UDA tasks.

##  3 Problems and the proposed Method

Given unlabeled target data
$\mathcal{T}=\\{\mathbf{x}^{t}_{i}\\}_{i=1}^{n^{t}}$ sampled from a
distribution $\mathcal{Q}$, our problem of interest is to learn a model
$F:\mathcal{X}^{t}\to[0,1]^{K}$ such that the empirical target risk
$\frac{1}{n^{t}}\sum_{i=1}^{n^{t}}[\mathcal{L}(F(\mathbf{x}_{i}^{t}),y_{i}^{t})]$
(or ideally, the expected risk
$\mathbb{E}_{(\mathbf{x}^{t},y^{t})\in\mathcal{Q}}[\mathcal{L}(F(\mathbf{x}^{t}),y^{t})]$)
could be minimized, where $K$ is the category number, $\mathcal{L}$ is the
loss function of the task, and $y^{t}_{i}\in\\{1,\dots,K\\}$,
$i=1,\dots,n^{t}$, is the target label to be estimated. Depending on how much
knowledge one may have from a source domain, the problem can fall in different
established realms of unsupervised learning [42], unsupervised domain
adaptation (UDA) [24, 5], and source-free UDA [45, 23]. While the first one
assumes no the source knowledge and is of machine learning foundations, in
this work, we focus on different problem settings of UDA.

Unsupervised Domain Adaptation. Labeled source data
$\mathcal{S}=\\{\mathbf{x}^{s}_{i},y^{s}_{i}\\}_{i=1}^{n^{s}}$ sampled from a
distribution $\mathcal{P}$ are assumed to be available to help the learning
with unlabeled target data $\mathcal{T}$ in UDA, leading to the following
objective:

$F^{t}\Leftarrow\mathcal{O}_{\textrm{UDA}}(\mathcal{T},\mathcal{S}),$ (1)

where $F^{t}$ is the expected target model. The most popular methods [24, 5]
for UDA (1) are to learn domain invariant feature representations, then the
classifier learned from labeled source data $\mathcal{S}$ could be safely
applied to target data.

White-box Unsupervised Domain Adaptation. Source-free UDA [22, 23] is proposed
recently due to increasing concerns for data privacy. Indeed, we are in an era
of cloud computing, and source and target data are usually captured and
privately stored on different devices; it is thus risky to expose source data
for transferable use to the target domain. Source-free UDA typically proposes
to use a trained white-box source model $F^{s}$, instead of the labeled source
data $\mathcal{S}$, to accomplish the UDA objective, which is formalized as:

$\displaystyle
F^{t}\Leftarrow\mathcal{O}_{\textrm{WBUDA}}(\mathcal{T},F^{s}),$ (2)

where $F^{s}=\operatorname*{arg\,min}_{F}\mathcal{L}_{task}(F,\mathcal{S})$
and the task loss $\mathcal{L}_{task}$ is typically instantiated as:

$\mathcal{L}_{task}(F,\mathcal{S})=\frac{1}{n^{s}}\sum_{i=1}^{n^{s}}-\log(F_{y_{i}^{s}}(\mathbf{x}_{i}^{s})),$
(3)

where $F_{k}(\mathbf{x})$ stands for the $k^{th}$ entry of $F(\mathbf{x})$.
Since white-box source model $F^{s}$ is required in (2), we term it as white-
box unsupervised domain adaptation (WBUDA) to distinguish it from our
investigated one in follow paragraphs. In WBUDA, the target model is typically
achieved by fine-tuning the white-box source model $F^{s}$ on unlabeled target
data $\mathcal{T}$ using proper objectives, e.g., the pseudo labeling and
information maximization losses in [23].

(a) Pair ($\epsilon$=0.45)

(b) Symmetry ($\epsilon$=0.5)

(c) VisDA-2017

(d) Rescale curve (9)

Figure 2: (a)-(c): Transition matrices [31, 34] of different noise types,
where the simulated (a) pair flipping [10] and (b) symmetry flipping [40] are
widely adopted in LNL works [10, 44], and (c) presents the realistic noise
matrix in the VisDA-2017 dataset based on the black-box source model
$\widehat{F}^{s}$. The value in row $r$, column $c$ represents the probability
with which samples of category $r$ are assigned with label $c$. In all
figures, deeper colour indicates a larger value and all values are rounded to
the level of $0.01$. (Zoom in to see the exact values.). The label noise in
(c) VisDA-2017 is significantly unbalanced among categories, where the noise
rates of $4$-th, $7$-th and $9$-th categories are less than 0.2 while the
noise rates of $6$-th and $12$-th categories are more than 0.96. (d)
Illustration of the rescale curve (9) with different $\kappa$.

Black-box Unsupervised Domain Adaptation. Although source data remain private
in the WBUDA mentioned above [22, 23], the required white-box source model may
be not available in practice. Most valuable models on cloud services (e.g.,
Google Cloud) are sealed as APIs, where only input-output interfaces are
available. As stated in [29], releasing an API instead of a white-box model
could commercialize the technology, reduce model misuse and make the model use
conveniently for the public; white-box attacks [38, 39] could also be avoided.

Considering above advantages of releasing APIs, we investigate a subtly
different setting of source-free UDA, where only the API of a source model is
accessible for the target domain and the source model itself is kept as a
black-box one. We denote this task as black-box unsupervised domain adaptation
(B2UDA), which is formulated as:

$\displaystyle
F^{t}\Leftarrow\mathcal{O}_{\textrm{B}^{2}\textrm{UDA}}(\mathcal{T},\widehat{F}^{s}),$
(4)

where $\widehat{F}^{s}$ is the API of $F^{s}$; specifically, we could get the
output of $F^{s}(\mathbf{x})$ with respect to any sample $\mathbf{x}$ via
$\widehat{F}^{s}(\mathbf{x})$ and the source model $F^{s}$ itself is kept as a
black-box one. As there are many model APIs on cloud services, B2UDA is a
promising way to improve their adaptabilities, presenting broad practical
values.

Labeled source data $\mathcal{S}$ and white-box source model $F^{s}$ are
respectively required in UDA and WBUDA methods, impeding their applications in
the B2UDA task. To tackle the challenging B2UDA, we propose an Iterative
Learning with Noisy Labels (IterLNL) framework by conducting noisy labeling
and LNL iteratively, which are introduced as follows.

Figure 3: Framework of our proposed Iterative Learning with Noisy Labels
(IterLNL), where we conduct noisy labeling and LNL iteratively. Note that we
introduce noisy labels based on predictions of the black-box model.

###  3.1 Noisy Labeling

Given a black-box model $\widehat{F}$ (e.g., the black-box source model
$\widehat{F}^{s}$) in B2UDA, we could get label predictions of target data
$\\{\mathbf{x}_{i}^{t}\\}_{i=1}^{n^{t}}$ with $\widehat{F}$ as:

$\widehat{\mathcal{Y}}=\\{\widehat{F}(\mathbf{x}_{i}^{t})\\}_{i=1}^{n^{t}}.$
(5)

The corresponding pseudo label of target sample $\mathbf{x}_{i}^{t}$ is
defined as
$\widehat{y}_{i}^{t}=\operatorname*{arg\,max}_{k}\widehat{F}_{k}(\mathbf{x}_{i}^{t})$.
The pseudo labels $\\{\widehat{y}_{i}^{t}\\}_{i=1}^{n^{t}}$ could be highly
noisy due to the divergence across source and target domains. Furthermore, we
emphasize that the label noise in $\\{\widehat{y}_{i}^{t}\\}_{i=1}^{n^{t}}$
could be significantly unbalanced among categories; for example, the noise
rate could be extremely high for some categories and extremely low for the
others, as illustrated in Figure 2(c). Such unbalanced label noise via domain
shift is substantially different from the simulated ones [40, 10] in many LNL
works, as compared in Figure 2. In addition, the noise rate, which is usually
required in LNL algorithms, is unknown in B2UDA, while it is usually assumed
to be given in LNL [40, 10]. In the next paragraph, we propose strategies to
estimate the noise rate and tackle unbalanced label noise, which support the
successful target model learning with noisy labels.

###  3.2 Learning with Noisy Labels

Give noisy target label predictions $\widehat{\mathcal{Y}}$ in Section 3.1, we
resort to LNL to learn the target model. State-of-the-art LNL methods [15, 10,
12] usually combat noisy labels by selecting ‘clean’ samples from each mini-
batch for training, which is achieved by utilizing the memorization effects of
neuron networks [2]. Before going into detail, we denote $R(n)$ as the
percentage of instances selected for training in the mini-batch of $n$-th
iteration. LNL methods [15, 10, 12] typically keep more instances in the mini-
batch (i.e., $R(n)$ is large) at the beginning, and then gradually drop noisy
samples (i.e., $R(n)$ becomes smaller) as training proceeds; by using more
training instances at the beginning, a relatively reliable model could be
achieved since deep models learn clean and easy patterns at the beginning [2];
with the reliable model, noisy instances could be filtered out by gradually
dropping instances with larger losses.

We also adopt the aforementioned LNL strategy in our method since it presents
high robustness even with extremely noisy labels [10]. In LNL [10, 44], the
selecting percentage $R(n)$ is depending on the noise rate, which is either
assumed to be known in advance [10] or estimated with few labeled clean data
[48, 33]. However, realistic noisy labels are introduced by domain shift in
B2UDA, where the unavailable of labeled target data and unknown noise rate
impede the design of $R(n)$.

To this end, we propose a simple yet efficient strategy to estimate noise
rate. We first follow [10] to define $R(n)$ as:

$R(n)=1-\min{(\frac{n}{n_{k}}\epsilon,\epsilon)},$ (6)

where $n_{k}$ is a hyperparameter and $\epsilon$ is the noise rate.

Noise rate Estimation. We first present the empirical noise rate $\epsilon$
as:

$\epsilon=1-\frac{1}{n^{t}}\sum_{i=1}^{n^{t}}\mathcal{I}[\mathbf{x}^{t}_{i},\widehat{y}_{i}^{t}],$
(7)

where $\mathcal{I}[\mathbf{x}^{t}_{i},\widehat{y}_{i}^{t}]\in\\{0,1\\}$ is a
binary indicator; $\mathcal{I}[\mathbf{x}^{t}_{i},\widehat{y}_{i}^{t}]=1$ if
$\widehat{y}_{i}^{t}$ is the correct label of $\mathbf{x}^{t}_{i}$ and 0
otherwise. It is obvious that the empirical noise rate $\epsilon$ (7) is close
correlated to the classification accuracy of the black-box model
$\widehat{F}$. In the meantime, there is a correlation between the
classification accuracy and maximum prediction probability, as observed in
[21, 52, 12]. Although the prediction probability may be overconfident and
misleading viewed in isolation, the probability statistics is often sufficient
to reflect on the overall classification accuracy [12, 6], and also the noise
rate $\epsilon$.

To estimate the noise rate $\epsilon$, we calculate the proportion of target
data $\mathcal{T}$ with high prediction probability as:

$\rho^{\prime}=\frac{1}{n^{t}}\sum_{i=1}^{n^{t}}\mathbb{I}[\max(\widehat{F}(\mathbf{x}_{i}^{t}))>\gamma],$
(8)

where $\gamma\in[0,1]$ is the threshold and
$\mathbb{I}[var]=\begin{cases}1,&var=True\\\ 0,&Otherwise\end{cases}$. One may
opt for approximating noise rate $\epsilon$ with $1-\rho^{\prime}$. However,
considering that deep models are prone to make over-confident predictions [6],
we rescale $\rho^{\prime}\in[0,1]\to\rho\in[0,1]$ as:

$\rho=\begin{cases}0.5(2\rho^{\prime})^{1/\kappa},&\rho^{\prime}<0.5\\\
-0.5(2-2\rho^{\prime})^{1/\kappa}+1,&Otherwise.\end{cases}$ (9)

Although the equation (9) seems complicated, there is only one hyperparameter
$\kappa$ controlling the curve degree and it degenerates to
$\rho=\rho^{\prime}$ if $\kappa=1$, as illustrated in Figure 2(d). The design
of (9) is further investigated in Section 4.1. We finally approximate noise
rate $\epsilon$ as:

$\epsilon=1-\rho.$ (10)

Although the estimated noise rate $\epsilon$ (10) is not precise, we find that
our method is robust to the noise rate $\epsilon$; such an estimation of noise
rate works well and achieves good results close to that using the grounding
truth noise rate, as presented in Section 4.1.

Category-wise Sampling. Given the estimated noise rate $\epsilon$ (10), we
could conduct LNL by selecting $R(n)$ (6) percent samples with smaller loss
for training in the mini-batch of $n$-th iteration. However, as we state in
Section 3.1, the label noise is unbalanced among categories in B2UDA (cf.
Figure 2(c)); thus samples in categories with higher noise rate are prone to
present larger loss and be rejected for training, leading to worse results for
these categories, as presented in Table 1.

To this end, we propose to individually sample the $R(n)$ (6) percent samples
with smaller loss for each category. Technically, we introduce a probability
queue buffer $\mathbf{u}_{k}\in[0,1]^{h}$ for category $k\in[1,\dots,K]$,
where $\mathbf{u}_{k}$ is initialized as a vector filled with positive
infinity values and $h$ is the buffer length. For any instance
$\mathbf{x}^{t}$ in the $n$-th iteration, we obtain its corresponding noisy
label
$\widehat{y}^{t}=\operatorname*{arg\,max}_{k}\widehat{F}_{k}(\mathbf{x}^{t})$
and loss
$\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t})=-\log(F_{\widehat{y}^{t}}(\mathbf{x}^{t}))$,
where $F$ is the current model in learning. We propose an indicator
$\mathbf{I}(\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t}),\mathbf{u}_{\widehat{y}^{t}},n)$
to decide whether $\mathbf{x}^{t}$ should be adopted in training, which is
formulated as:

$\mathbf{I}(\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t}),\mathbf{u}_{\widehat{y}^{t}},n)=\begin{cases}1,&\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y})\leq
L_{R(n)}(\mathbf{u}_{\widehat{y}^{t}})\\\ 0,&Otherwise,\end{cases}$ (11)

where $L_{R(n)}(\mathbf{u}_{\widehat{y}^{t}})$ is the $R(n)$-th largest value
in $\mathbf{u}_{\widehat{y}^{t}}$. We utilize the loss
$\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t})$ to update current model $F$
if
$\mathbf{I}(\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t}),\mathbf{u}_{\widehat{y}^{t}},n)=1$
and drop it otherwise.

We also update the queue buffers $\\{\mathbf{u}_{k}\\}_{k=1}^{K}$ with all
samples in the $n$-th iteration. Specifically, given an instance
$\mathbf{x}^{t}$ and its corresponding noisy label $\widehat{y}^{t}$ and loss
$\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t})$ defined above, we push
$\mathcal{L}(F(\mathbf{x}^{t}),\widehat{y}^{t})$ into the queue buffer
$\mathbf{u}_{\widehat{y}^{t}}$ and pop the oldest value from
$\mathbf{u}_{\widehat{y}^{t}}$ simultaneously. In this way, we adopt samples
with the $R(n)$ percent smallest losses in each category for training in
$n$-th iteration.

Remarks. Note that our method is fundamentally different from existing DA
methods based on pseudo labels [26, 53]. Although we also adopt the categories
with maximum prediction probability on the black-box model as noisy labels, we
do not use them directly as accurate ones. We only use samples with noisy
labels to update the model if these samples present small losses based on the
current model; in other words, we treat noisy labels as accurate ones only if
they are consistent with the current model’s predictions. Moreover, source
data are not involved in model training, avoiding the misleading of source
data on target predictions.

###  3.3 Iternative Learning Strategy

With the noisy labeling in Section 3.1 and LNL in Section 3.2, we could get a
more reliable target model over the original black-box one (i.e., the one
introduces noisy labels), as illustrated in Figure 4(a). In other words, the
achieved target model could produce improved noisy labels over the original
noisy labels. It is a natural idea to conduct noisy labeling with the achieved
target model (or the black-box counterpart of the achieved target model), and
conduct LNL on the new noisy labels again. Finally, we define our algorithmic
framework by conducting noisy labeling and LNL iteratively, leading to the
Iterative Learning with Noisy Labels (IterLNL). We summarize IterLNL in
Algorithm 1 and illustrate its framework in Figure 3.

Remarks. The black-box source model and noisy labels are introduced
respectively in B2UDA and LNL tasks to help the learning with unlabeled data,
making the two tasks fundamentally different. Although we utilize the black-
box source model to assign noisy labels and learn the target model in a LNL
manner, there are three differences. First, the noise rate is unknown and to
be estimated in B2UDA and no labeled target data are provided for the noise
rate estimation. Second, the label noise is quite unbalanced among categories
in B2UDA. Third, different from the given and fixed noisy labels in LNL, the
noisy labels in B2UDA are introduced by black-box models and could be updated
as the model update, inspiring the iterative learning strategy.

Algorithm 1 Iterative Learning with Noisy Labels.

1:Black-box source model $\widehat{F}^{s}$, target data $\mathcal{T}$

2:Target model $F$

3:Initialize $\widehat{F}$ with $\widehat{F}^{s}$

4:for $m=1$ to $M$ do $\triangleright$ For each iterative step

5: Acquire noisy labels $\widehat{\mathcal{Y}}$ with $\mathcal{T}$ and
$\widehat{F}$ using (5)

6: Estimate noise rate $\epsilon$ with (8), (9) and (10)

7: Initialize target model $F$ and buffers $\\{\mathbf{u}_{k}\\}_{k=1}^{K}$

8: for $n=1$ to $N$ do $\triangleright$ For each iteration

9: Acquire $R(n)$ with (6)

10: Update model $F$ using data selected with (11)

11: Update buffers $\\{\mathbf{u}_{k}\\}_{k=1}^{K}$

12: end for

13: Update $\widehat{F}$ as the API (i.e., black-box model) of $F$

14:end for

Methods |  plane |  bcycl |  bus |  car |  horse |  knife |  mcycl |  person |  plant |  sktbrd |  train |  truck | Avg
---|---|---|---|---|---|---|---|---|---|---|---|---|---
Source Model [16] |  76.4 |  26.4 |  60.2 |  80.2 |  73.5 |  2.3 |  89.4 |  8.2 |  79.7 |  25.6 |  69.4 |  3.8 | 49.6
IterLNL (w/o Iter) |  91.4 |  68.9 |  69.6 |  89.9 |  88.0 |  25.1 |  92.8 |  26.1 |  93.1 |  59.1 |  83.3 |  16.3 | 67.0
IterLNL (w/o CateS) |  96.7 |  88.3 |  89.0 |  94.1 |  96.7 |  0.0 |  89.4 |  48.4 |  96.0 |  93.3 |  84.8 |  0.0 | 73.1
IterLNL (w/o Rescale) |  91.7 |  84.8 |  86.6 |  88.7 |  79.4 |  77.9 |  92.8 |  46.3 |  90.1 |  89.1 |  88.2 |  40.5 | 79.7
IterLNL |  77.0 |  84.6 |  85.1 |  92.0 |  92.1 |  74.1 |  92.6 |  49.1 |  89.1 |  91.7 |  84.5 |  49.4 | 80.1
IterLNL (with Val) |  89.0 |  79.5 |  84.3 |  81.0 |  87.7 |  88.1 |  92.5 |  38.7 |  87.1 |  96.9 |  78.8 |  67.0 | 80.9
Table 1: Ablation study on VisDA-2017 dataset, where all experiments are based
on a ResNet-101 model. Please refer to the main text for definitions.

##  4 Experiment

Office-31 [35] is the most popular benchmark dataset for UDA. There are
$4,110$ samples shared by three domains: amazon (A), webcam (W), and dslr (D).
VisDA-2017 [32] aims to transfer knowledge from synthetic images to real-world
ones, which is a challenging task with significant domain divergence. There
are $152K$ synthetic images and $55K$ real-world images shared by $12$
classes. Datasets of MNIST [20], Street View House Numbers (SVHN) [28], and
USPS [14] constitute the Digits task, which includes $10$ classes. There are
$50,000$ training samples, $10,000$ validation samples and $10,000$ test
samples in the MNIST dataset (M), where all images are black-and-white
handwritten digits; the SVHN dataset (S) contains $73,257$ training and
$26,032$ test images with colored backgrounds; the USPS dataset (U) contains
$7,291$ training and $2,007$ test images with black backgrounds.

Implementation Details. For experiments on datasets of Office-31 and
VisDA-2017, we employ the pre-trained ResNet model [11] as the backbone and
replace the last fully connected (FC) layer with a task-specific FC classifier
following [24, 5, 23]. We introduce the source model $F^{s}$ by fine-tuning
the constructed model on source data following [16] and then seal $F^{s}$ as
the black-box $\widehat{F}^{s}$, i.e., only the input-output interface of
$F^{s}$ is available. For experiments on Digits dataset, we follow [36] to
introduce the source model $F^{s}$ with convolutional layers and FC layers.
Following [5], we utilize the SGD optimizer and adopt the learning rate
strategy as $\eta_{n}=\frac{\eta_{0}}{(1+10\zeta)^{0.75}}$, where
$\eta_{0}=0.01$ and $\zeta$ is the process of training iterations linearly
changing from $0$ to $1$. We set the batch size as $64$, $\kappa=2$ in (9),
$\gamma=0.9$ in (8), buffer length $h=100$, and $n_{k}=0.5N$ ($N$ is the total
number of training iterations defined in Algorithm 1) in (6) for all
experiments; these hyperparameters are analyzed in Section 4.2.

(a) Acc. in iterative process

(b) Noise rate $\epsilon$

Figure 4: Illustration of (a) the accuracy of noisy labels
$\\{\widehat{y}_{i}^{t}\\}_{i=1}^{n^{t}}$ and target model $F$ via LNL, and
(b) the estimated noise rate in different iterative steps. ‘GT’, ‘with Val’,
‘IterLNL’ and ‘w/o Rescale’ indicate the noise rate calculated by all labeled
target data (7), labeled validation data, our strategy (10), and
$1-\rho^{\prime}$, respectively. Note that the noise rate estimation and
accuracy of noisy labels with $m=1$ are based on predictions of the black-box
source model $\widehat{F}^{s}$.

###  4.1 Ablation Study

We introduce several variants of IterLNL to investigate the individual
components in IterLNL. Specifically, following [10], we replace the category-
wise sampling (11) by simply using the $R(n)$ percent samples with smaller
losses in the $n$-th iteration for model learning, leading to ‘IterLNL (w/o
CateS)’. We also present the results of IterLNL by conducting noisy labeling
and learning with noisy labels only once (i.e., setting $M=1$ in Algorithm 1),
resulting in ‘IterLNL (w/o Iter)’. To investigate the noise rate estimation
strategy, we remove the rescale strategy (9) and set
$\epsilon=1-\rho^{\prime}$, leading to ‘IterLNL (w/o Rescale)’; we also
introduce a labeled target validation set by randomly selecting $30$ samples
per class (less than $1$% of the entire target data in VisDA-2017); we
estimate the noise rate $\epsilon$ by calculating the classification accuracy
$\alpha$ of $\widehat{F}$ on the validation set and approximating the noise
rate $\epsilon$ as $1-\alpha$, which is termed as ‘IterLNL (with Val)’.

As illustrated in Table 1, IterLNL improves over the IterLNL (w/o CateS) and
IterLNL (w/o Iter) significantly, justifying the efficacy of category-wise
sampling (11) and iterative learning strategy. Specifically, without the
category-wise sampling (11), the accuracy of samples in the knife and truck
categories drops to zero due to the initial high noise rate (i.e., low
accuracy) in Source Model. We also intuitively visualize the accuracy
improvement of model $F$ in the iterative learning process (i.e., with
different $m\in[1,M]$). As illustrated in Figure 4(a), the accuracy of $F$ via
LNL indeed improves over that of initial noisy labels in the beginning; the
improvement is gradually reduced as the iterative step $m$ increases, leading
to the final convergence. Additionally, IterLNL improves over IterLNL (w/o
Rescale) and approximate to results of IterLNL (with Val); this is intuitively
explained in Figure 4(b), where IterLNL (with Val) presents the most accurate
noise rate estimation, and IterLNL achieves more accurate estimation than
IterLNL (w/o Rescale) in most cases. The source model (i.e., $m$=1) tends to
make un-confident predictions on target data due to the domain divergence
while target models (i.e., $m$>1) are prone to make over-confident
predictions, motivating us to introduce the rescale curve (9) in Figure 2(d).

Settings | Methods |  U$\to$M |  S$\to$M |  M$\to$U
---|---|---|---|---
B2UDA | Source Model |  82.0 |  69.4 |  79.4
sMDA [4] |  83.4 |  69.9 |  81.2
PLR [27] |  91.8 |  97.3 |  89.3
IterLNL |  96.7$\pm$0.3 |  98.0$\pm$0.7 |  97.4$\pm$0.1
WBUDA | SDDA [18] |  – |  75.5 |  89.9
3C-GAN [22] |  99.3$\pm$0.1 |  99.4$\pm$0.1 |  97.3$\pm$0.2
SHOT [23] |  98.4 |  98.9 |  98.0
UDA | DANN [5] |  86.3$\pm$0.3 |  85.5$\pm$0.4 |  84.9$\pm$0.6
MCD [36] |  – |  96.2$\pm$0.4 |  96.5$\pm$0.3
CDAN [25] |  98.0 |  89.2 |  95.6
RWOT [43] |  97.5$\pm$0.2 |  98.8$\pm$0.1 |  98.5$\pm$0.2
Table 2: Results on Digits dataset. Tasks | Method | A$\to$D | A$\to$W | D$\to$A | D$\to$W | W$\to$A | W$\to$D | Avg
---|---|---|---|---|---|---|---|---
B2UDA | Source Model [16] | 79.7 | 78.1 | 64.9 | 96.0 | 65.4 | 99.2 | 80.1
sMDA [4] | 80.5 | 79.3 | 65.7 | 96.3 | 67.3 | 99.2 | 81.4
IterLNL |  91.3$\pm$0.1 |  89.8$\pm$0.7 |  74.3$\pm$0.8 |  98.7$\pm$0.2 |  73.7$\pm$0.1 |  99.3$\pm$0.3 | 87.9
WBUDA | SDDA [18] | 85.3 | 82.5 | 66.4 | 99.0 | 67.7 | 99.8 | 83.5
SHOT [23] | 94.0 | 90.1 | 74.7 | 98.4 | 74.3 | 99.9 | 88.6
3C-GAN [22] | 92.7$\pm$0.4 | 93.7$\pm$0.2 | 75.3$\pm$0.5 | 98.5$\pm$0.1 | 77.8$\pm$0.1 | 99.8$\pm$0.2 | 89.6
UDA | DANN [5] | 79.7$\pm$0.4 | 82.0$\pm$0.4 | 68.2$\pm$0.4 | 96.9$\pm$0.2 | 67.4$\pm$0.5 | 99.1$\pm$0.1 | 82.2
CDAN [25] | 92.9$\pm$0.2 | 94.1$\pm$0.1 | 71.0$\pm$0.3 | 98.6$\pm$0.1 | 69.3$\pm$0.3 | 100.0$\pm$.0 | 87.7
SymNets [50] | 93.9$\pm$0.5 | 90.8$\pm$0.1 | 74.6$\pm$0.6 | 98.8$\pm$0.3 | 72.5$\pm$0.5 | 100.0$\pm$.0 | 88.4
RWOT [43] | 94.5$\pm$0.2 | 95.1$\pm$0.2 | 77.5$\pm$0.1 | 99.5$\pm$0.2 | 77.9$\pm$0.3 | 100.0$\pm$.0 | 90.8
Table 3: Results on Office31 dataset, where all methods are based on a ResNet-50 model.  Tasks |  Methods |  plane |  bcycl |  bus |  car |  horse |  knife |  mcycl |  person |  plant |  sktbrd |  train |  truck |  Avg
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
B2UDA | Source Model [16] |  76.4 |  26.4 |  60.2 |  80.2 |  73.5 |  2.3 |  89.4 |  8.2 |  79.7 |  25.6 |  69.4 |  3.8 | 49.6
sMDA [4] |  77.8 |  39.3 |  66.1 |  73.7 |  74.3 |  4.2 |  87.9 |  16.7 |  79.8 |  36.9 |  71.4 |  9.8 | 53.1
SoFA [46] |  – |  – |  – |  – |  – |  – |  – |  – |  – |  – |  – |  – | 60.4
IterLNL |  77.0 |  84.6 |  85.1 |  92.0 |  92.1 |  74.1 |  92.6 |  49.1 |  89.1 |  91.7 |  84.5 |  49.4 | 80.1
WBUDA | 3C-GAN [22] |  94.8 |  73.4 |  68.8 |  74.8 |  93.1 |  95.4 |  88.6 |  84.7 |  89.1 |  84.7 |  83.5 |  48.1 | 81.6
SHOT [23] |  94.3 |  88.5 |  80.1 |  57.3 |  93.1 |  94.9 |  80.7 |  80.3 |  91.5 |  89.1 |  86.3 |  58.2 | 82.9
UDA | DANN [5] |  81.9 |  77.7 |  82.8 |  44.3 |  81.2 |  29.5 |  65.1 |  28.6 |  51.9 |  54.6 |  82.8 |  7.8 | 57.4
MCD [36] |  87.0 |  60.9 |  83.7 |  64.0 |  88.9 |  79.6 |  84.7 |  76.9 |  88.6 |  40.3 |  83.0 |  25.8 | 71.9
RWOT [43] |  95.1 |  80.3 |  83.7 |  90.0 |  92.4 |  68.0 |  92.5 |  82.2 |  87.9 |  78.4 |  90.4 |  68.2 | 84.0
Table 4: Results on VisDA-2017 dataset, where all methods are based on a
ResNet-101 model.

(a) Results with various $\gamma$

(b) Results with various $\kappa$

Figure 5: IterLNL’s results with various values of $\gamma$ (8) and $\kappa$
(9).

###  4.2 Analysis

Analyses on $\gamma$ and $\kappa$. We investigate the hyper-parameters
$\gamma$ (8) and $\kappa$ (9) on the Digits datasets considering the
experimental speed. As illustrated in Figure 5, IterLNL performs robustly
under a wide range of values. We adopt $\gamma=0.9$ and $\kappa=2.0$ in all
experiments; such simple settings work well but not necessarily lead to the
best results. Similar results are also observed in the study of buffer length
$h$ in Section 3.2 and $n_{k}$ (6), as presented in the appendices.

Comparison with LNL Method. We conduct the LNL method Co-teaching [10] with
our estimated noise rate $\epsilon$ (10) and noisy labels
$\\{\widehat{y}_{i}^{t}\\}_{i=1}^{n^{t}}$ introduced by the black-box source
model $\widehat{F}^{s}$ on the S$\to$M task. The result of $91.0\%$ with Co-
teaching is lower than $98.0\%$ with IterLNL, justifying the advantage of
IterLNL over vanilla LNL method on B2UDA.

###  4.3 Results

The results of IterLNL on datasets of Digits, Office31, and VisDA-2017 are
illustrated in Table 2, Table 3, and Table 4, respectively. Most comparable
results are directly reported from their original papers, except the Source
Model [16] and sMDA [4], which are implemented by ourselves. Taking advantages
of feature learning, our IterLNL improves over existing methods of B2UDA [4,
46] significantly; for example, IterLNL improves over sMDA [4] and SoFA [46]
by $27.0\%$ and $19.7\%$ on the VisDA-2017 dataset respectively; our IterLNL
also improves over [27] by presenting good generalization on various benchmark
datasets. Although we only use the black-box source model for the transfer use
in target domain, IterLNL achieves comparable results to methods of WBUDA,
where the white-box source model is required, and traditional UDA, where
labeled source data are required.

##  5 Conclusion and Broader Impact

Considering that the source model itself may not be available due to
commercial and/or safety considerations [29], we investigate the B2UDA task,
and propose a baseline algorithm of IterLNL. We verify its efficacy on popular
DA benchmarks with thorough experiments. The B2UDA task is of broad practical
value since it could further improve the utilities of APIs as cloud services,
pushing the DA research closer to practical applications.

## References

  * [1] Sk Miraj Ahmed, Aske R Lejbolle, Rameswar Panda, and Amit K Roy-Chowdhury.  Camera on-boarding for person re-identification using hypothesis transfer learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12144–12153, 2020.
  * [2] Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al.  A closer look at memorization in deep networks.  arXiv preprint arXiv:1706.05394, 2017.
  * [3] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan.  A theory of learning from different domains.  Machine learning, 79(1-2):151–175, 2010.
  * [4] Boris Chidlovskii, Stephane Clinchant, and Gabriela Csurka.  Domain adaptation in the absence of source domain data.  In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 451–460, 2016.
  * [5] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  The Journal of Machine Learning Research, 17(1):2096–2030, 2016\.
  * [6] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  arXiv preprint arXiv:1706.04599, 2017.
  * [7] Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama.  Sigua: Forgetting may make learning with noisy labels more robust.  In International Conference on Machine Learning, pages 4006–4016. PMLR, 2020.
  * [8] Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama.  Masking: A new perspective of noisy supervision.  NIPS, 2018.
  * [9] Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W Tsang, James T Kwok, and Masashi Sugiyama.  A survey of label-noise representation learning: Past, present and future.  arXiv preprint arXiv:2011.04406, 2020.
  * [10] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama.  Co-teaching: Robust training of deep neural networks with extremely noisy labels.  In Advances in neural information processing systems, pages 8527–8537, 2018.
  * [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [12] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  ICLR, 2017.
  * [13] Yunzhong Hou and Liang Zheng.  Source free domain adaptation with image translation.  arXiv preprint arXiv:2008.07514, 2020.
  * [14] Jonathan J. Hull.  A database for handwritten text recognition research.  IEEE Transactions on pattern analysis and machine intelligence, 16(5):550–554, 1994.
  * [15] Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei.  Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels.  In International Conference on Machine Learning, pages 2304–2313. PMLR, 2018.
  * [16] Mingsheng Long Junguang Jiang, Bo Fu.  Transfer-learning-library.  https://github.com/thuml/Transfer-Learning-Library, 2020.
  * [17] Youngeun Kim, Sungeun Hong, Donghyeon Cho, Hyoungseob Park, and Priyadarshini Panda.  Domain adaptation without source data.  arXiv preprint arXiv:2007.01524, 2020.
  * [18] Vinod K Kurmi, Venkatesh K Subramanian, and Vinay P Namboodiri.  Domain impression: A source data free domain adaptation method.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 615–625, 2021.
  * [19] Ilja Kuzborskij and Francesco Orabona.  Stability and hypothesis transfer learning.  In International Conference on Machine Learning, pages 942–950. PMLR, 2013.
  * [20] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.
  * [21] Dong-Hyun Lee.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In Workshop on challenges in representation learning, ICML, volume 3, page 2, 2013.
  * [22] Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu.  Model adaptation: Unsupervised domain adaptation without source data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [23] Jian Liang, Dapeng Hu, and Jiashi Feng.  Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation.  arXiv preprint arXiv:2002.08546, 2020.
  * [24] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan.  Learning transferable features with deep adaptation networks.  In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, pages 97–105. JMLR.org, 2015.
  * [25] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  In Advances in Neural Information Processing Systems, pages 1640–1650, 2018.
  * [26] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Deep transfer learning with joint adaptation networks.  arXiv preprint arXiv:1605.06636, 2016.
  * [27] Pietro Morerio, Riccardo Volpi, Ruggero Ragonesi, and Vittorio Murino.  Generative pseudo-label refinement for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3130–3139, 2020.
  * [28] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * [29] OpenAI.  Why did openai choose to release an api instead of open-sourcing the models?  [EB/OL].  https://openai.com/blog/openai-api/ Accessed March 4, 2021.
  * [30] Sinno Jialin Pan, Qiang Yang, et al.  A survey on transfer learning.  IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2010.
  * [31] Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu.  Making deep neural networks robust to label noise: A loss correction approach.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1944–1952, 2017.
  * [32] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko.  Visda: The visual domain adaptation challenge.  arXiv preprint arXiv:1710.06924, 2017.
  * [33] Harish Ramaswamy, Clayton Scott, and Ambuj Tewari.  Mixture proportion estimation via kernel embeddings of distributions.  In International conference on machine learning, pages 2052–2060. PMLR, 2016.
  * [34] Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich.  Training deep neural networks on noisy labels with bootstrapping.  arXiv preprint arXiv:1412.6596, 2014.
  * [35] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  In European conference on computer vision, pages 213–226. Springer, 2010.
  * [36] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada.  Maximum classifier discrepancy for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3723–3732, 2018.
  * [37] Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus.  Training convolutional networks with noisy labels.  ICLRW, 2015.
  * [38] Simen Thys, Wiebe Van Ranst, and Toon Goedemé.  Fooling automated surveillance cameras: adversarial patches to attack person detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  * [39] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel.  Ensemble adversarial training: Attacks and defenses.  arXiv preprint arXiv:1705.07204, 2017.
  * [40] Brendan Van Rooyen, Aditya Krishna Menon, and Robert C Williamson.  Learning with symmetric label noise: The importance of being unhinged.  NIPS, 2015.
  * [41] Brendan Van Rooyen and Robert C Williamson.  A theory of learning with corrupted labels.  J. Mach. Learn. Res., 18(1):8501–8550, 2017.
  * [42] Markus Weber, Max Welling, and Pietro Perona.  Unsupervised learning of models for recognition.  In European conference on computer vision, pages 18–32. Springer, 2000.
  * [43] Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang.  Reliable weighted optimal transport for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4394–4403, 2020.
  * [44] Hansi Yang, Quanming Yao, Bo Han, and Gang Niu.  Searching to exploit memorization effect in learning from corrupted labels.  arXiv preprint arXiv:1911.02377, 2019.
  * [45] Shiqi Yang, Yaxing Wang, Joost van de Weijer, and Luis Herranz.  Unsupervised domain adaptation without source data by casting a bait.  arXiv preprint arXiv:2010.12427, 2020.
  * [46] Hao-Wei Yeh, Baoyao Yang, Pong C Yuen, and Tatsuya Harada.  Sofa: Source-data-free feature alignment for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 474–483, 2021.
  * [47] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson.  How transferable are features in deep neural networks?  arXiv preprint arXiv:1411.1792, 2014.
  * [48] Xiyu Yu, Tongliang Liu, Mingming Gong, Kayhan Batmanghelich, and Dacheng Tao.  An efficient and provable approach for mixture proportion estimation using linear independence assumption.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4480–4489, 2018.
  * [49] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan.  Bridging theory and algorithm for domain adaptation.  In International Conference on Machine Learning, pages 7404–7413, 2019.
  * [50] Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan.  Domain-symmetric networks for adversarial domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5031–5040, 2019.
  * [51] Zhilu Zhang and Mert R Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  arXiv preprint arXiv:1805.07836, 2018.
  * [52] Tianyi Zhou, Shengjie Wang, and Jeff Bilmes.  Time-consistent self-supervision for semi-supervised learning.  In International Conference on Machine Learning (ICML), 2020.
  * [53] Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang.  Unsupervised domain adaptation for semantic segmentation via class-balanced self-training.  In Proceedings of the European conference on computer vision (ECCV), pages 289–305, 2018.


# Weak Supervision with Incremental Source Accuracy Estimation

Richard Correro

###### Abstract

Motivated by the desire to generate labels for real-time data we develop a
method to estimate the dependency structure and accuracy of weak supervision
sources incrementally. Our method first estimates the dependency structure
associated with the supervision sources and then uses this to iteratively
update the estimated source accuracies as new data is received. Using both
off-the-shelf classification models trained using publicly-available datasets
and heuristic functions as supervision sources we show that our method
generates probabilistic labels with an accuracy matching that of existing off-
line methods.

###### Index Terms:

Weak Supervision, Transfer Learning, On-line Algorithms.

##  I Introduction

Weak supervision approaches obtain labels for unlabeled training data using
noiser or higher level sources than traditional supervision [1]. These sources
may be heuristic functions, off-the-shelf models, knowledge-base-lookups, etc.
[2]. By combining multiple supervision sources and modeling their dependency
structure we may infer the true labels based on the outputs of the supervision
sources.

### Problem Setup

In the weak supervision setting we have access to a dataset
$X=\\{x_{1},\dots,x_{n}\\}$ associated with unobserved labels
$Y=\\{y_{1},\dots,y_{n}\\},\ \ y_{i}\in\\{1,\dots,k\\}$ and a set of weak
supervision sources $p_{i}(y|x),i=1,\dots,m$.

We denote the outputs of the supervision sources by
$\lambda_{1},\dots,\lambda_{m}$ and let $\mathbf{\lambda_{j}}=[\lambda_{1}\
\lambda_{2}\ \dots\ \lambda_{m}]^{T}$ denote the vector of labels associated
with example $x_{j}$. The objective is to learn the joint density

$f(y,\mathbf{\lambda})$

over the sources and the latent label. Using this we may estimate the
conditional density

$\displaystyle
f_{Y\mid\Lambda}(y|\mathbf{\lambda})=\frac{f_{Y,\Lambda}(y,\mathbf{\lambda})}{f_{\Lambda}(\lambda)},\quad
f_{\Lambda}(\lambda)>0.$ (1)

These sources may take many forms but we restrict ourselves to the case in
which $\lambda_{i}\in\\{0,\dots,k\\}$ and thus the label functions generate
labels belonging to the same domain as $Y$. Here $\lambda_{i}=0$ indicates the
$i^{th}$ source has not generated a label for this example. Such supervision
sources may include heuristics such as knowledge base lookups, or pre-trained
models.

##  II Related Work

Varma et. al. [3] and Ratner, et. al. [4] model the joint distribution of
$\lambda_{1},\dots,\lambda_{m},Y$ in the classification setting as a Markov
Random Field

$f_{G}(\lambda_{1},\dots,\lambda_{m},y)=\frac{1}{Z}\exp\left(\sum_{\lambda_{i}\in
V}\theta_{i}\lambda_{i}+\sum_{(\lambda_{i},\lambda_{j})\in
E}\theta_{i,j}\lambda_{i}\lambda_{j}+\theta_{Y}y+\sum_{\lambda_{i}\in
V}\theta_{Y,y}y\lambda_{i}\right)$

associated with graph $G=(V,E)$ where $\theta_{i,j}\ 1\leq i,j\leq m+1$ denote
the canonical parameters associated with the supervision sources and $Y$, and
$Z$ is a partition function [here
$V=\\{\lambda_{1},\dots,\lambda_{m}\\}\cup\\{Y\\}$]. If $\lambda_{i}$ is not
independent of $\lambda_{j}$ conditional on $Y$ and all sources $\lambda_{k},\
k\in\\{1,\dots,m\\}\setminus\\{i,j\\}$, then $(\lambda_{i},\lambda_{j})$ is an
edge in $E$.

Let $\Sigma$ denote the covariance matrix of the supervision sources and $Y$.
To learn $G$ from the labels

$O=\\{\lambda_{i}:\lambda_{i}=[\lambda_{1},\dots,\lambda_{m}]^{T};i=1,\dots,n\\}$

and without the ground truth labels, Varma et. al. assume that $G$ is sparse
and therefore that the inverse covariance matrix $\Sigma^{-1}$ associated with
$\lambda_{1},\dots,\lambda_{m},Y$ is graph-structured. Since $Y$ is a latent
variable the full covariance matrix $\Sigma$ is unobserved. We may write the
covariance matrix in block-matrix form as follows:

$Cov[O\cup S]:=\Sigma=\begin{bmatrix}\Sigma_{O}&\Sigma_{OS}\\\
\Sigma_{OS}^{T}&\Sigma_{S}\end{bmatrix}$

Inverting $\Sigma$, we write

$\Sigma^{-1}=\begin{bmatrix}K_{O}&K_{OS}\\\ K_{OS}^{T}&K_{S}\end{bmatrix}$

$\Sigma_{O}$ may be estimated empirically:

$\hat{\Sigma}_{O}=\frac{\Lambda\Lambda^{T}}{n}-\nu\nu^{T}$

where
$\Lambda=[\mathbf{\lambda}_{1}\mathbf{\lambda}_{2},\dots,\mathbf{\lambda}_{n}]$
denotes the $m\times n$ matrix of labels generates by the sources and
$\nu=\hat{E}[O]\in\mathbb{R}^{m}$ denotes the observed labeling rates.

Using the block-matrix inversion formula, Varma et. al. show that

$K_{O}=\Sigma_{O}^{-1}+c\Sigma_{O}^{-1}\Sigma_{OS}\Sigma_{OS}^{T}\Sigma_{O}^{-1}$

where
$c=(\Sigma_{S}-\Sigma_{OS}^{T}\Sigma_{O}^{-1}\Sigma_{OS})^{-1}\in\mathbb{R}^{+}$.
Letting $z=\sqrt{c}\Sigma_{O}^{-1}\Sigma_{OS}$, they write

$\Sigma_{O}^{-1}=K_{O}-zz^{T}$

where $K_{O}$ is sparse and $zz^{T}$ is low-rank positive semi definite.
Because $\Sigma_{O}^{-1}$ is the sum of a sparse matrix and a low-rank matrix
we may use Robust Principal Components Analysis [5] to solve the following:

$\displaystyle(\hat{S},\hat{L})=\text{argmin}_{(S,L)}||L||_{*}+\gamma||S||_{1}$
$\displaystyle s.t.\quad\quad S-L=\hat{\Sigma}^{-1}_{O}$

Varma et. al. then show that we may learn the structure of $G$ from $K_{O}$
and we may learn the accuracies of the sources from $z$ using the following
algorithm:

Result: $\hat{G}=(V,\hat{E}),\ \hat{L}$

$\mathbf{Inputs:}$ Estimate of covariance matrix $\hat{\Sigma}_{O}$, parameter
$\gamma$, threshold $T$

$\mathbf{Solve:}\quad(\hat{S},\hat{L})=\text{argmin}_{(S,L)}||L||_{*}+\gamma||S||_{1}$

s.t. $S-L=\hat{\Sigma}^{-1}_{O}$

$\hat{E}\xleftarrow{}\\{(i,j):i<j,\hat{S}_{i,j}>T\\}$

Algorithm 1 Weak Supervision Structure Learning and Source Estimation Using
Robust PCA (From [3])

Note that $\hat{L}=zz^{T}$.

Ratner, et. al. [4] show that we may estimate the source accuracies
$\hat{\mu}$ from $z$ and they propose a simpler algorithm for estimating $z$
if the graph structure is already known: If $E$ is already known we may
construct a dependency mask $\Omega=\\{(i,j):(\lambda_{i},\lambda_{j})\not\in
E\\}$. They use this in the following algorithm:

Result: $\hat{\mu}$

$\mathbf{Inputs:}$ Observed labeling rates $\hat{\mathbb{E}}[O]$ and
covariance $\hat{\Sigma}_{O}$; class balance $\hat{\mathbb{E}}[Y]$ and
variance $\hat{\Sigma}_{S}$; dependency mask $\Omega$

$\hat{z}\xleftarrow{}\text{argmin}_{Z}||\hat{\Sigma}_{O}^{-1}+zz^{T}||_{\Omega}$

$\hat{c}\xleftarrow{}\Sigma_{S}^{-1}(1+\hat{z}^{T}\hat{\Sigma}_{O}\hat{z})$

$\hat{\Sigma}_{OS}\xleftarrow{}\hat{\Sigma}_{O}\hat{z}/\sqrt{\hat{c}}$

$\hat{\mu}\xleftarrow{}\hat{\Sigma}_{OS}+\hat{\mathbb{E}}[Y]\hat{\mathbb{E}}[O]$

Algorithm 2 Source Estimation for Weak Supervision (From [4])

Snorkel, an open-source Python package, provides an implementation of
algorithm 2 [6].

##  III Motivating Our Approach

Although the algorithm proposed by Varma et. al. may be used determine the
source dependency structure and source accuracy, it requires a robust
principal components decomposition of the matrix $\hat{\Sigma}_{O}$ which is
equivalent to a convex Principal Components Pursuit (PCP) problem [5]. Using
the current state-of-the-art solvers such problems have time complexity
$O(\epsilon^{-2})$ where $\epsilon$ denotes the solver convergence tolerance
[5]. For reasonable choices of $\epsilon$ this may be a very expensive
calculation.

In the single-task classification setting, algorithm 2 may be solved by least-
squares and is therefore much less expensive to compute than algorithm 1. Both
algorithms, however, require the observed labeling rates and covariance
estimates of the supervision sources over the entire dataset and therefore
cannot be used in an on-line setting.

We therefore develop an on-line approach which estimates the structure of $G$
using algorithm 1 on an initial ”minibatch” of unlabeled examples and then
iteratively updates the source accuracy estimate $\hat{\mu}$ using using a
modified implementation of algorithm 2.

##  IV Methods

Given an initial batch $b_{1}$ of unlabeled examples
$X_{b_{1}}=\\{x_{1},\dots,x_{k}\\}$ we estimate $G$ by first soliciting labels
$\mathbf{\lambda}_{1},\dots,\mathbf{\lambda}_{k}$ for $X_{b_{1}}$ from the
sources. We then calculate estimated labeling rates $\hat{E}[O]$ and
covariances $\hat{\Sigma}_{Ob_{1}}$ which we then input to algorithm 1,
yielding $\hat{G}=(V,\hat{E})$ and $\hat{L}$. From $\hat{E}$ we create the
dependency mask
$\hat{\Omega}=\\{(i,j):(\lambda_{1},\lambda_{j})\not\in\hat{E}\\}$ which we
will use with future data batches. Using the fact that $\hat{L}=zz^{T}$ we
recover $\hat{z}$ by first calculating

$|\hat{z}|=\sqrt{diag(\hat{L})}$

We then break the symmetry using the method in [4]. Note that if a source
$\lambda_{i}$ is conditionally independent of the others then the sign of
$z_{i}$ determines the sign of all other elements of $z$.

Using $\hat{z},\ \hat{E}[O],\ \hat{\Sigma}_{Ob_{1}}$, class balance prior
$\hat{E}[Y]$ and class variance prior $\hat{\Sigma}_{S}$ we calculate
$\hat{\mu}$, an estimate of the source accuracies [if we have no prior beliefs
about the class distribution then we simply substitute uninformative priors
for $\hat{E}[O]$ and $\hat{\Sigma}_{Ob_{1}}$].

For each following batch $b_{p}$ of unlabeled examples $X_{b_{p}}$ we estimate
$\Sigma_{Ob_{p}}$ and $E[O]_{b_{p}}$. Using these along with $\hat{E}[O]$ and
$\hat{\Sigma}_{Ob_{1}}$ we calculate $\hat{\mu}_{b_{p}}$, an estimate of the
source accuracies over the batch. We then update $\hat{\mu}$ using the
following update rule:

$\hat{\mu}:=(1-\alpha)\hat{\mu}+\alpha\mu_{b_{p}}$

where $\alpha\in[0,1]$ denotes the mixing parameter. Our method thus models
the source accuracies using an exponentially-weighted moving average of the
estimated per-batch source accuracies.

Using the estimated source accuracies and dependency structure we may estimate
$p(y,\mathbf{\lambda})$ which we may then use to estimate
$p(y|\mathbf{\lambda})$ by (1).

Result: $\hat{\mu}$

$\mathbf{Inputs:}$ Observed labeling rates $\hat{\mathbb{E}}[O]_{b}$ and
covariance $\hat{\Sigma}_{Ob}$; class balance $\hat{\mathbb{E}}[Y]$ and
variance $\hat{\Sigma}_{S}$

for  _each batch b_ do

if  _is initial batch_ then

Use algorithm 1 to calculate $\hat{G}$ and $\hat{L}$

$|\hat{z}|\xleftarrow{}\sqrt{diag(\hat{L})}$

Determine the sign of the entries of $z$ using method from [4]

else

$\hat{z}\xleftarrow{}\text{argmin}_{z}||\hat{\Sigma}_{Ob}^{-1}+zz^{T}||_{\Omega}$

end if

$\hat{c}\xleftarrow{}\Sigma_{S}^{-1}(1+\hat{z}^{T}\hat{\Sigma}_{Ob}\hat{z})$

$\hat{\Sigma}_{OS}\xleftarrow{}\hat{\Sigma}_{Ob}\hat{z}/\sqrt{\hat{c}}$

$\hat{\mu}_{b}\xleftarrow{}\hat{\Sigma}_{OS}+\hat{\mathbb{E}}[Y]\hat{\mathbb{E}}[O]_{b}$

if  _is initial batch_ then

$\hat{\mu}\xleftarrow{}\hat{\mu}$

else

$\hat{\mu}\xleftarrow{}(1-\alpha)\hat{\mu}+\alpha\hat{\mu}_{b}$

end if

end for

Algorithm 3 Incremental Source Accuracy Estimation

##  V Tests

### Supervision Sources

We test our model in an on-line setting using three supervision sources. Two
of the sources are off-the-shelf implementations of Naïve Bayes classifiers
trained to classify text by sentiment. Each was trained using openly-available
datasets. The first model was trained using a subset of the IMDB movie reviews
dataset which consists of a corpus of texts labeled by perceived sentiment
[either ”positive” or ”negative”]. Because the labels associated with this
dataset are binary the classifier generates binary labels.

The second classifier was trained using another openly-available dataset, this
one consisting of a corpus of text extracted from tweets associated with air
carriers in the United States and labeled according to sentiment. These labels
in this dataset belong to three seperate classes [”positive”, ”neutral”, and
”negative”] and therefore the model trained using this dataset classifies
examples according to these classes.

The final supervision source is the Textblob Pattern Analyzer. This is a
heuristic function which classifies text by polarity and subjectivity using a
lookup-table consisting of strings mapped to polarity/subjectivity estimates.
To generate discrete labels for an example using this model we threshold the
polarity/subjectivity estimates associated with the label as follows:

  * •

If polarity is greater than 0.33 we generate a positive label

  * •

If polarity is less than or equal to 0.33 but greater than -0.33 we generate a
neutral label

  * •

If polarity is less than or equal to 0.33 we generate a negative label

### Test Data

We test our incremental model using a set of temporally-ordered text data
extracted from tweets associated with a 2016 GOP primary debate labeled by
sentiment [”positive”, ”neutral”, or ”negative”]. We do so by solicting labels
$\mathbf{\lambda}_{1},\dots,\mathbf{\lambda}_{n}$ associated with the $n$
examples from the three supervision sources.

### Weak Supervision as Transfer Learning

Note that this setting is an example of a transfer learning problem [7].
Specifically, since we are using models pre-trained on datasets similar to the
target dataset we may view the Naive Bayes models as transferring knowledge
from those two domains [Tweets associated with airlines and movie reviews,
respectively] to provide supervision signal in the target domain [7]. The
Pattern Analyzer may be viewed through the same lens as it uses domain
knowledge gained through input from subject-matter experts.

### Test Setup

Because our model is generative we cannot use a standard train-validation-test
split of the dataset to determine model performance. Instead, we compare the
labels generated by the model with the ground-truth labels over separate folds
of the dataset.

#### Data Folding Procedure

We split the text corpus into five folds. The examples are not shuffled to
perserve temporal order within folds. Using these folds we perform 5 separate
tests, each using four of the five folds in order. For example, the fifth test
uses the fold 5 and folds 1—3, in that order.

#### Partition Tests

For each set of folds we further partition the data into $k=100$ batches of
size $q$ which we refer to as ”minibatches” [as they are subsets of the
folds]. For each minibatch we solicit labels
$\mathbf{\lambda}_{1},\dots,\mathbf{\lambda}_{q},\
\mathbf{\lambda}_{i}\in\mathbf{R^{3}}$ from the two pretrained models and the
Pattern Analyzer. Note that both pretrained classifiers first transform the
text by tokenizing the strings and then calculating the term-frequency to
inverse document frequency (Tf-idf) for each token. We store these labels in
an array $\mathbf{L}$ for future use. We then calculate $\hat{E}[O]_{b}$ and
$\hat{\Sigma}_{Ob}$ for the minibatch, which we use with algorithm 3 to
generate $\hat{\mu}_{b}$ and the dependency graph $\hat{G}$. Using these we
generate labels corresponding to the examples contained within the minibatch.

Using the ground-truth labels associated with the examples contained within
the minibatch we calculate the accuracy of our method by comparing the
generated labels $\mathbf{\hat{y}}$ with the ground-truth labels $\mathbf{y}$:

$\texttt{accuracy}(\mathbf{y},\mathbf{\hat{y}})=\frac{1}{q}\sum_{i=0}^{q-1}\mathbf{1}(\mathbf{\hat{y}}_{i}=\mathbf{y}_{i})$

We then average the accuracy scores associated with each minibatch over the
number of minibatches used in each test to calculate the average per-test
accuracy [calculated using four of the five folds of the overall dataset].

We then compare the average accuracies of the labels produced using our
incremental method to the accuracies of the labels produced by an existing
off-line source accuracy estimation method based on algorithm 2 [6]. Since
this method works in an off-line manner it requires access to the entire set
$\mathbf{L}$ of labels generated by the supervision sources. Using these this
method generates its own set of generated labels $\mathbf{\hat{y}}_{baseline}$
with which we then calculate the baseline accuracy using the accuracy metric
above.

Finally, we compare the accuracy of the labels generated by our method with
the accuracy of the labels generated by each of the supervision sources.

#### Comparing Values of $\alpha$

We then follow the same procedure as above to generate labels for our method,
except this time we use different values of $\alpha$.

##  VI Results

Our tests demonstrate the following:

  1. 1.

Our model generates labels which are more accurate than those generated by the
baseline [when averaged over all 5 tests].

  2. 2.

Both our method and the baseline generate labels which are more accurate than
those generated by each of the supervision sources.

  3. 3.

Our tests of the accuracy of labels generated by our method using different
values of $\alpha$ yields an optimal values $\alpha=0.05$ and shows convexity
over the values tested.

Theses tests show that the average accuracy of the incremental model
qualitatively appears to increase as the number of samples seen grows. This
result is not surprising as we would expect our source accuracy estimate
approaches the true accuracy $\hat{\mu}\xrightarrow{}{\mu}$ as the number of
examples seen increases. This implies that the incremental approach we propose
generates more accurate labels as a function of the number of examples seen,
unlike the supervision sources which are pre-trained and therefore do not
generate more accurate labels as the number of labeled examples grows.

These tests also suggest that an optimal value for $\alpha$ for this problem
is approximately $0.05$ which is in the interior of the set of values tested
for $\alpha$. Since we used $100$ minibatches in each test of the incremental
model this implies that choosing an $\alpha$ which places greater weight on
more recent examples yields better performance, although more tests are
necessary to make any stronger claims.

Finally, we note that none of the models here tested are in themselves highly-
accurate as classification models. This is not unexpected as the supervision
sources were intentionally chosen to be ”off-the-shelf” models and no feature
engineering was performed on the underlying text data, neither for the
datasets used in pre-training the two classifier supervision sources nor for
the test set [besides Tf-idf vectorization]. The intention in this test was to
compare the relative accuracies of the two generative methods, not to design
an accurate discriminative model.

##  VII Conclusion

We develop an incremental approach for estimating weak supervision source
accuracies. We show that our method generates labels for unlabeled data which
are more accurate than those generated by pre-existing non-incremental
approaches. We frame our specific test case in which we use pre-trained models
and heuristic functions as supervision sources as a transfer learning problem
and we show that our method generates labels which are more accurate than
those generated by the supervision sources themselves.

Figure 1: Comparison of incremental and non-incremental model accuracy over minibatches. Figure 2: Average model accuracy over minibatches. Figure 3: Average per-batch accuracies for different values of $\alpha$. TABLE I: Average Accuracy of Incremental Model For Different Alpha Values Alpha | 0.001 | 0.01 | 0.025 | 0.05 | 0.1 | 0.25
---|---|---|---|---|---|---
Accuracy | 0.61245 | 0.61287 | 0.61832 | 0.61709 | 0.61498 | 0.61473

## References

  * [1] Alexander Ratner. Stephen Bach. Paroma Varma. Chris Ré. (2017) ”Weak Supervision: The New Programming Paradigm for Machine Learning”. Snorkel Blog.
  * [2] Mayee Chen. Frederic Sala. Chris Ré. ”Lecture Notes on Weak Supervision”. CS 229 Lecture Notes. Stanford University.
  * [3] Paroma Varma. Frederic Sala. Ann He. Alexander Ratner. Christopher Ré. (2019) ”Learning Dependency Structures for Weak Supervision Models”. Preprint.
  * [4] Alexander Ratner. Braden Hancock. Jared Dunnmon. Frederic Sala. Shreyash Pandey. Christopher Ré. (2018) ”Training Complex Models with Multi-Task Weak Supervision”. Preprint.
  * [5] Emmanuel J. Candès. Xiaodong Li. Yi Ma. John Wright. ”Robust principal component analysis?” Journal of the ACM. Vol 58. Issue 11.
  * [6] Alexander Ratner. Stephen H. Bach. Henry Ehrenberg. Jason Fries. Sen Wu. Christopher Ré. ”Snorkel: Rapid Training Data Creation with Weak Supervision.” Preprint.
  * [7] Sinno Jialin Pan. Qiang Yang (2009) ”A Survey on Transfer Learning”. IEEE Transactions on Knowledge and Data Engineering. Vol 22. Issue 10.


# On the Suppression and Enhancement of Thermal Chemical Rates in a Cavity

Jing Sun<EMAIL_ADDRESS>[  Oriol Vendrell
<EMAIL_ADDRESS>[

###### Abstract

The observed modification of thermal chemical rates in Fabry-Perot cavities
remains a poorly understood effect theoretically. Recent breakthroughs explain
some of the observations through the Grote-Hynes theory, where the cavity
introduces friction with the reaction coordinate, thus reducing the
transmission coefficient and the rate. The regime of rate enhancement, the
observed sharp resonances at varying cavity frequencies, and the survival of
these effects in the collective regime remain mostly unexplained. In this
paper, we consider the _cis_ -_trans_ isomerization of HONO atomistically
using an _ab-initio_ potential energy surface. We evaluate the transmission
coefficient using the reactive flux method and identify the conditions for
rate acceleration. In the underdamped, low-friction regime of the reaction
coordinate, the cavity coupling enhances the rate with increasing coupling
strength until reaching the Kramers turnover point. Sharp resonances in this
regime are related to cavity-enabled energy redistribution channels.

Heidelberg University] Theoretische Chemie, Physikalisch-Chemisches Institut,
Universität Heidelberg,
69120 Heidelberg, Germany Heidelberg University] Theoretische Chemie,
Physikalisch-Chemisches Institut, Universität Heidelberg,
69120 Heidelberg, Germany

Vibrational strong coupling (VSC) has emerged as a very active front in the
field of polaritonic chemistry since the pioneering demonstrations of Rabi
splitting in the infrared domain in Fabry-Perot configurations 1, 2, 3. At the
core of this sub-discipline lies the promise of modifying 4, and ultimately
controlling 5 the mechanisms and rates of thermal chemical reactions using the
vacuum fields of cavities 6. Besides important breakthroughs in the linear and
non-linear spectroscopy of VSC systems 4, 6, 5, 7, the more spectacular
results remain the experiments reporting the modification of chemical rates in
cavities by the Ebbesen group and others 8, 9, 10, 11, 12, 13. These
experiments have triggered the proposal of several theoretical models to
explain how the cavity modifies the ground electronic state structure 14 and
spectroscopy 15 and, more recently, how it modifies reaction rates 16, 17, 18,
19, 20, 21.

Theoretical models based on the Grote-Hynes theory 22 predict the suppression
of the transmission coefficient with increasing cavity coupling due to
increased friction at the top of the reaction barrier 19, 20, 21. How cavities
can enhance chemical reactions 9, how sharp resonances of the cavity with
vibrational modes affect the mechanism 9, 10, and how these effects survive in
the collective VSC regime, have remained poorly understood questions.

Here, we simulate the rate of a realistic isomerization reaction atomistically
using an _ab initio_ potential, both for one and several HONO molecules, in
the VSC regime. Our simulations explain how the cavity enhances chemical rates
in the underdamped regime and capture the turnover from the underdamped to the
damped regime as a function of the cavity coupling strength. Moreover, we
explain how, in the underdamped regime, sharp resonances of the cavity with
vibrational modes can strongly affect the reaction rate. Finally, our results
show how, in the collective VSC regime, the strong direct coupling to the
reaction coordinate ceases to be the determining factor in the cavity effect.

Figure 1:  _cis_ -_trans_ isomerization reaction in HONO. The axes indicate
the body-fixed frame of the molecules in the simulation. The presence of the
cavity is indicated schematically and is not to scale. HONO is characterized
by 6 vibrational coordinates: $3$ stretching modes, O$-$H, O$-$N and N$=$O;
$2$ bending modes, H$-$O$-$N and O$-$N$=$O; $1$ torsion mode $\tau$, the
isomerization reaction coordinate.

Our starting point is the Hamiltonian for a molecular ensemble coupled to one
or several cavity modes

$\displaystyle\hat{H}$
$\displaystyle=\sum_{l=1}^{N}\hat{H}_{mol}^{(l)}+\hat{H}_{cav}$ (1)

with

$\displaystyle\hat{H}_{mol}^{(l)}$
$\displaystyle=\sum_{j_{l}=1}^{F}\frac{\hat{P}_{j_{l}}^{2}}{2M_{j_{l}}}+\hat{V}(R_{1_{l}}\ldots
R_{F_{l}}),$ (2) $\displaystyle\hat{H}_{cav}$
$\displaystyle=\frac{1}{2}\left[\hat{p}_{cav}^{2}+\omega_{cav}^{2}\left(\hat{q}_{cav}+\frac{\boldsymbol{\lambda}}{\omega_{cav}}\cdot\sum^{N}_{l=1}\hat{\boldsymbol{\mu}}^{(l)})\right)^{2}\right],$
(3)

and where $V(R_{1_{l}}\ldots R_{F_{l}})$ denotes the ground electronic state
potential energy surface (PES) of the $l$-th molecule with momenta $P_{j_{l}}$
and positions $R_{j_{l}}$. Hence, the Born-Oppenheimer (BO) approximation is
assumed within each molecule, and
$\boldsymbol{\mu}^{(l)}\equiv\boldsymbol{\mu}^{(l)}(R_{1_{l}}\ldots
R_{F_{l}})$ is the permanent dipole vector of the l-th molecule.
$\hat{H}_{cav}$ can be reached from the Coulomb-gauge light-matter interaction
Hamiltonian by taking the long wave approximation followed by a unitary
transformation to the length form 23, 24, 17. For later convenience, we write
$\hat{H}_{cav}$ in its position-momentum representation ($\hat{q}_{cav}$,
$\hat{p}_{cav}$). $\omega_{cav}$ corresponds to the cavity mode frequency.
This form of the light-matter interaction has become standard in most
theoretical studies of VSC 17, 25, 19, 20, and details on its derivation 26,
23 and properties 27 can be found elsewhere. The parameter
$\boldsymbol{\lambda}$ equals the coupling strength
$\lambda=\sqrt{1/\epsilon_{0}V}$ times the unit polarization vector
$\boldsymbol{\epsilon}$ of the cavity mode, and $V$ represents the cavity
volume 23. Similarly to other studies and to facilitate comparisons, we
introduce the coupling parameter $g=\lambda\sqrt{\hbar\omega_{cav}/2}$, which
has units of electric field (using this relation and
$\hat{q}_{cav}=\sqrt{\hbar/2\omega_{cav}}(\hat{a}^{\dagger}+\hat{a})$, the
linear coupling term in $\hat{H}_{cav}$ reads
$g\,\boldsymbol{\epsilon}\cdot\boldsymbol{\mu}\,(\hat{a}^{\dagger}+\hat{a})$).

The simplicity of the unimolecular reaction mechanism in HONO makes it an
ideal benchmark system to understand how dynamical cavity effects can modify
chemical rates as compared, e.g., to bimolecular reactions in solution 10, 28.
We base our study on the CCSD(T)-quality _ab initio_ potential energy surface
(PES) of Richter et al. 29, which features a reaction barrier height of about
0.51 eV (49 kJ/mol) and where the _trans_ isomer is 11 meV more stable than
the _cis_ one. Quantum dynamics studies of the HONO isomerization triggered by
strong laser pulses have been based on this PES 30, 31. Despite its
simplicity, this chemical reaction constitutes a fully coupled and rich
dynamical system. Similarly to other isomerization reactions, e.g. involving
hydrocarbons 32, 33, 34, it takes place in the underdamped regime. Throughout
this work, the molecules are kept at a fixed orientation with respect to the
polarization direction of the cavity mode. In this way, we focus on the
coupling of the $\mu_{x}$ dipole component to the cavity polarizaton. As shown
in the SI, this component of the molecular dipole has the strongest modulation
at the isomerization barrier configuration.

The molecule-cavity system is considered within the classical approximation,
thus transforming all coordinate operators in Hamiltonians 2 and 3 to
classical functions. A classical description of the VSC regime is not new and
has been successfully applied to bulk systems described by force-field
potentials 35 and model Hamiltonians 19, 36. The _cis_ -_trans_ reaction rate
is described with the reactive flux method for the classical rate constant 32,
33, 37, 38, 34

$\displaystyle K(t)$
$\displaystyle=x_{cis}^{-1}\langle\dot{\tau}(0)\,\delta[\tau(0)-\tau^{{\ddagger}}]\,\theta[\tau(t)]\rangle,$
(4)

where $x_{cis}$ is the equilibrium fraction of HONO at the _cis_ geometry,
$\dot{\tau}(0)$ is the initial velocity of a phase-space point perpendicular
to the dividing surface between reactants and products, and
$\tau^{{\ddagger}}$ is the torsion angle corresponding to the transition state
(TS) geometry. The brackets indicate the canonical ensemble average over
trajectories, where we considered a temperature of 300 K throughout. The
Heaviside function $\theta[\tau]$ is defined to be one for the _trans_
configurations, and zero otherwise. The exact reactive flux is obtained in the
limit $t\to\infty$, in practice when the plateau for $K(t)$ is reached 38.
This occurs when all classical trajectories starting from the dividing surface
become trapped at either the reactants or products side. For example, for the
isomerization reaction of $n$-butane in the low-friction environment of a van
der Waals liquid this relaxation time is about 1 ps 33. Now, since 37

$\displaystyle\lim_{t\to 0^{+}}K(t)=K_{TST},$ (5)

one can introduce a transmission coefficient $\kappa(t)=K(t)/K_{TST}$ as the
quotient of the numerically exact reactive flux and the reactive flux without
recrossing, i.e. the TST assumption. $K_{TST}$ can be evaluated conveniently
using Eyring’s equation 39, 40, while $\kappa(t)$ is obtained from classical
trajectories. As shown in the SI, and as has been discussed in other works 17,
18, $K_{TST}$ is, to a very good approximation, insensitive to cavity effects.
Therefore, we consider $K_{TST}$ to be completely cavity-independent and
describe the cavity effect on the rate as

$\displaystyle K_{cav}=\kappa_{cav}\kappa_{0}K_{TST},$ (6)

where $K_{0}\equiv\kappa_{0}K_{TST}$ is the formally exact classical rate
outside the cavity. Here and in the following, transmission coefficients and
rate constants without a time argument refer to their plateau value. Clearly,
both $\kappa$ and $\kappa_{0}$ lie in the $[0,1]$ range but $\kappa_{cav}$ can
be both larger or smaller than one, corresponding to a chemical rate
enhancement or suppression, respectively.

In the following, we theoretically demonstrate that both enhancement and
suppression of reaction rates are possible within a cavity for realistic
chemical processes. Although we rely on classical rate theory 40, we note that
tunneling corrections for hydrogen abstraction reactions at 300 K result in
variations of the rate within the same order of magnitude 41. For reactions
involving heavier elements, quantum corrections to the rates are even more
insignificant. Along these lines, there is no reason to assume, a priori, that
photonic modes with frequencies similar to the atomic vibrations, and in
thermal equilibrium, shall result in significant quantum effects that affect
the general conclusions derived from classical rate theories for VSC systems.
This does not exclude situations where quantum effects may be important for
quantitative descriptions of cavity-modified rates in reactions involving
light atoms, as it is sometimes the case for rates outside cavities 42, 43,
41.

Figure 2:  a) Transmission coefficient $\kappa(t)$ for various cavity-coupling
strengths $g$ (V/nm) for $N=1$ and the cavity polarization aligned with HONO’s
$x$-axis. b) same as a) but with the HONO molecules coupled to a bath (see
SI). The shaded area on top of the solid lines indicates the standard
deviation of the average over trajectory runs. c) Asymptotic
$\kappa_{cav}=\kappa/\kappa_{0}$ for the curves in a) (red) and b) (green). d)
$\kappa_{cav}$ for increasing number of molecules at constant total
polaritonic coupling (see text for details).

Let us consider a single HONO molecule coupled to a cavity mode with $x$
polarization with respect to the molecular frame. In this case, the variation
of the permenent dipole is largest at the transition state (TS) of the
reaction coordinate, $\tau^{\ddagger}\approx\pi/2$. Outside the cavity,
$\kappa_{0}\approx 0.35$ at 300 K, the plateau value of the black curve in
Fig. 2a. This relatively low transmission is caused by a slow rate of intra-
molecular vibrational energy redistribution (IVR) of the activated
trajectories in the underdamped regime. As the cavity-coupling increases, one
sees how the plateau value stabilizes at a larger total transmission $\kappa$
in the red, blue and green curves. The cavity accelerates the chemical
reaction by increasing the total transmission coefficient compared to
$\kappa_{0}$, i.e. $\kappa_{cav}>1$. This is illustrated by the red trace in
Fig. 2c in the coupling regime where $\kappa_{cav}$ increases, and it is well
understood within our theoretical framework: the cavity provides an extra
energy redistribution pathway for a system with a low-friction reaction
coordinate. Recrossing events are increasingly suppressed and the transmission
increases. Nonetheless, as the cavity coupling to the torsion coordinate
further increases, a turning point is reached for $g>3$ V/nm. The amount of
recrossing at the barrier keeps increasing as well, thus finally reverting the
trend and decreasing the transmission again. This is the well-known Kramers
turning point 22, which, e.g., was predicted long ago for the isomerization of
cyclohexane as a function of solvent viscosity 34. Figure 2a illustrates its
origin in the quick drop of $\kappa(t)$ at short times for the strongest
cavity coupling.

When adding an external bath to HONO (see SI for details), the regime of
validity of the GH theory is restored. Figure 2b shows how now $\kappa(t)$
quickly reaches the plateau value within a few tens of femtoseconds, meaning
that activated trajectories visit the region of the TS only once or twice.
Since the plateau is reached quickly, the cavity can only have a short-time
effect close to the top of the barrier, where it can increase the amount of
recrossing thus reduce the transmission coefficient. As illustrated in Fig.2c
by the green trace, now $\kappa_{cav}<1$ and the chemical rate is reduced for
all coupling strengths. This is the regime captured in Refs. 19, 20, 21.

Figure 3:  Potential energy surface cut for a) $1$ and b) $100$ HONO molecules
as a function of the reaction coordinate $\tau$ and the cavity displacement
$q_{cav}-q_{cav}^{{\ddagger}}$. For $N=1$ the light-matter coupling is $g=8$
V/nm. The cavity coupling in b) is scaled by ${1/\sqrt{N}}$ to keep a constant
overall light-matter interaction. The color levels start at $0$ for the
lightest tone and increase in steps of $0.2$ eV. The red line indicates the
minimum energy path. The vertical dashed line separates the _cis_ and _trans_
configurations.

A question mark remains still in connection with the collective VSC regime,
where most experiments reporting modifications of chemical rates in Fabry-
Perot configurations operate. To shed some light into this issue, we have
performed trajectory calculations of the transmission coefficient for an
increasing number of molecules $N$ coupled to the cavity, again without an
extra bath. The coupling per molecule is scaled as usual by a factor
${N}^{-1/2}$ as a means to keep the overall light-matter coupling constant 44.
Starting from $N=1$, $g=1$ V/nm, and $\omega_{cav}=852$ cm$-1$, one sees in
Fig. 2d how, for increasing $N$, the cavity effect gradually fades away.
Responsible for the gradual trend $\kappa_{cav}\to 1$ is the decoupling of the
reaction coordinate from the cavity displacement with increasing $N$, as seen
by comparing the curvature of the minimum energy path (MEP) in Figs. 3a,
$N=1$, and 3b, with $N=100$. This reduction of the MEP curvature as $N$
increases, and thus the reduced friction caused by the cavity, implies that in
the large $N$ limit the cavity is not able to “cage” the TS and induce a
decrease of the transmission coefficient through this mechanism.

Figure 4:  Transmission coefficient $\kappa$ for various coupling strengths
$g_{\omega}=g\mu_{x}^{{\ddagger}}/\omega_{cav}$ as a function of
$\omega_{cav}$. Vertical bars represent standard deviations over the run
trajectories. $\omega_{cav}$ is chosen to be resonant with fundamental modes
of molecule: $\omega_{ONO}=609(cm^{-1})$, $\omega_{\tau}=640(cm^{-1})$,
$\omega_{ON}=852(cm^{-1})$, $\omega_{HON}=1263(cm^{-1})$,
$\omega_{NO}=1641(cm^{-1})$, $\omega_{OH}=3426(cm^{-1})$ and with the average
of every consecutive pair. The black dashed line indicates $\kappa_{0}$
outside the cavity with its standard deviation.

Finally, we address the question of sharp resonant effects, meaning when the
modification of chemical rates is particularly pronounced at specific cavity
frequencies. Through trajectory calculations it has been observed that the
outcome of reactive events can depend on the resonance between the cavity and
vibrational modes of the molecule, but a link to the actual modification of
chemical rates has not been established 25, 45. Our simulations of the
transmission coefficient in the underdamped, slow IVR regime reveal sharp
resonances in the rate constant effect as a function of $\omega_{cav}$. As
already discussed, in this regime the effect of the cavity is to introduce
extra energy redistribution pathways, whereby the effect at short times while
passing the TS barrier region is not so important. Thus, when the cavity is
resonant with a vibrational mode that happens to be strongly coupled to the
reaction coordinate, the enhancement of the rate is more prominent. As seen in
Fig. 4, $\kappa_{cav}\approx 2$ when $\omega_{cav}$ is resonant with the O-N
stretching mode at 852 cm-1. This is not surprising. It is well-known that the
O-N stretch is strongly coupled to the torsion coordinate in HONO 29, 30:
Selective laser excitations of this mode result in an enhanced probability of
isomerization out of equilibrium 31. This brings us to the observed cavity
catalysis of a unimolecular dissociation in solution by the Ebbesen group 9. A
plausible explanation is that the strong resonance of the cavity mode with a
carbonyl group could stabilize the hot nascent products inside the solvent
pocket, in this way preventing recrossing events at early times after passage
over the TS. Further studies will be required to test our hypothesis.

Let us recapitulate and place our findings in the context of the current
literature on thermal rate models in cavities. The regime considered in model
studies is the one in which the cavity is strongly coupled to the reaction
coordinate, while reactants and products are strongly damped by a bath 19 or
effectively by a short propagation time that hinders recrossing 14. In this
case, the reaction pathway becomes curved, as seen in the Shin-Metieu and
other 1D models 14, 19, 21. The regime in which the coupling is strong at the
barrier top and the reactants and products are damped by a bath is well
captured by the Grote-Hynes (GH) theory 22 and has been the basis of
theoretical proposals for the reduction of chemical rates in cavities 19, 21,
20. In this regime there are no sharp resonance effects, although there is a
continuous dependence of $\kappa$ on $\omega_{cav}$ related to the frequency
of the inverted reaction barrier 21. Similarly to our findings, existing
theories report a reduction of the transmission coefficient in the collective
regime as $N$ increases (cf. Fig. 2a in Ref. 21). Reference 20 attributes the
persistence of the modification of chemical rates in the large $N$ collective
regime to the assumption that “the polariton state is thermally activated to
yield collective barrier crossing” 20. While a coherent superposition of
activated complexes may display this behavior, a physical mechanism by which a
polaritonic system spontaneously results in coherence in the dark and under
thermal equilibrium of the material and photonic modes, has not been put
forward.

Concluding, our main contribution has been to identify dynamical effects
played by the cavity in the low-friction (or underdamped) regime of the
reaction coordinate. In this regime, the cavity effect is twofold: (1) It
_accelerates_ the chemical rate by increasing the friction compared to the
cavity-free system. This reduces the recrossing due to trajectories that,
otherwise, would visit reactants and products several times, thus increasing
the transmission coefficient compared to $\kappa_{0}$. As the cavity coupling
keeps increasing, the overall increased friction can introduce again more
coupling at the barrier and the trend can overturn. This is the well-known
Kramers turnover situation, and this regime can exist in the condensed phase
22, 34. (2) In the low-friction regime, sharp resonant effects are possible.
These are related to the new IVR pathways offered by the cavity. In the
products-side, they dissipate energy from the nascent hot products. In the
reactants-side, the resonances funnel energy into the nascent activated
complexes. Numerically, the former is captured by trajectories starting
towards the products-side and being effectively captured there. The latter is
captured by the trajectories initially moving towards reactants and being
effectively captured as well. If this reactants-side capture would be
ineffective, they would be counted as products but with a negative
contribution to the flux, in this way lowering the transmission (cf. Eq. 4).
Finally, when a bath is added to the HONO molecule and the overall friction is
sufficiently increased, the model reverts to the already known GH regime where
the cavity only affects the recrossings at the top of the reaction barrier.
Our findings shed important new light onto the question of cavity-modified
reactivity. However, it still remains for future work to better understand how
these cavity effects can survive in actual liquid phases and in the collective
regime for truly macroscopic numbers of molecules. It is plausible that the
more detailed answers lie beyond models of independent molecules and may
require studies of the transmission coefficient with full consideration of
environmental effects in the bulk 46.

## References

  * Hutchison et al. 2012 Hutchison, J. A.; Schwartz, T.; Genet, C.; Devaux, E.; Ebbesen, T. W. _Angew. Chem._ 2012, _124_ , 1624–1628
  * Shalabney et al. 2015 Shalabney, A.; George, J.; Hutchison, J.; Pupillo, G.; Genet, C.; Ebbesen, T. W. _Nat Commun_ 2015, _6_ , 5981
  * Ebbesen 2016 Ebbesen, T. W. _Acc. Chem. Res._ 2016, _49_ , 2403–2412
  * Dunkelberger et al. 2016 Dunkelberger, A. D.; Spann, B. T.; Fears, K. P.; Simpkins, B. S.; Owrutsky, J. C. _Nat. Commun._ 2016, _7_ , 13504
  * Yang et al. 2020 Yang, Z.; Xiang, B.; Xiong, W. _ACS Photonics_ 2020, _7_ , 919–924
  * Dunkelberger et al. 2018 Dunkelberger, A. D.; Davidson, R. B.; Ahn, W.; Simpkins, B. S.; Owrutsky, J. C. _J. Phys. Chem. A_ 2018, _122_ , 965–971
  * Fassioli et al. 2021 Fassioli, F.; Park, K. H.; Bard, S. E.; Scholes, G. D. _J. Phys. Chem. Lett._ 2021, _12_ , 11444–11459
  * Thomas et al. 2016 Thomas, A. et al. _Angew. Chem. Int. Ed._ 2016, _55_ , 11462–11466
  * Lather et al. 2019 Lather, J.; Bhatt, P.; Thomas, A.; Ebbesen, T. W.; George, J. _Angew. Chem. Int. Ed._ 2019, _58_ , 10635–10638
  * Thomas et al. 2019 Thomas, A. et al. _Science_ 2019, _363_ , 615–619
  * Vergauwe et al. 2019 Vergauwe, R. M. A.; Thomas, A.; Nagarajan, K.; Shalabney, A.; George, J.; Chervy, T.; Seidel, M.; Devaux, E.; Torbeev, V.; Ebbesen, T. W. _Angew. Chem. Int. Ed._ 2019, _58_ , 15324–15328
  * Thomas et al. 2020 Thomas, A.; Jayachandran, A.; Lethuillier-Karl, L.; Vergauwe, R. M. A.; Nagarajan, K.; Devaux, E.; Genet, C.; Moran, J.; Ebbesen, T. W. _Nanophotonics_ 2020, _9_ , 249–255
  * Imperatore et al. 2021 Imperatore, M. V.; Asbury, J. B.; Giebink, N. C. _J. Chem. Phys._ 2021, _154_ , 191103
  * Galego et al. 2019 Galego, J.; Climent, C.; Garcia-Vidal, F. J.; Feist, J. _Phys. Rev. X_ 2019, _9_ , 021057
  * del Pino et al. 2015 del Pino, J.; Feist, J.; Garcia-Vidal, F. J. _New J. Phys._ 2015, _17_ , 053040
  * Campos-Gonzalez-Angulo et al. 2019 Campos-Gonzalez-Angulo, J. A.; Ribeiro, R. F.; Yuen-Zhou, J. _Nat Commun_ 2019, _10_ , 4685
  * Li et al. 2020 Li, T. E.; Nitzan, A.; Subotnik, J. E. _J. Chem. Phys._ 2020, _152_ , 234107
  * Vurgaftman et al. 2020 Vurgaftman, I.; Simpkins, B. S.; Dunkelberger, A. D.; Owrutsky, J. C. _J. Phys. Chem. Lett._ 2020, 3557–3562
  * Li et al. 2021 Li, X.; Mandal, A.; Huo, P. _Nat Commun_ 2021, _12_ , 1315
  * Yang and Cao 2021 Yang, P.-Y.; Cao, J. _J. Phys. Chem. Lett._ 2021, _12_ , 9531–9538
  * Mandal et al. 2022 Mandal, A.; Li, X.; Huo, P. _J. Chem. Phys._ 2022, _156_ , 014101
  * Hynes 1986 Hynes, J. _J. Stat. Phys._ 1986, _42_ , 149–168
  * Flick et al. 2017 Flick, J.; Ruggenthaler, M.; Appel, H.; Rubio, A. _PNAS_ 2017, _114_ , 3026–3034
  * Haugland et al. 2021 Haugland, T. S.; Schäfer, C.; Ronca, E.; Rubio, A.; Koch, H. _J. Chem. Phys._ 2021, _154_ , 094113
  * Sidler et al. 2020 Sidler, D.; Ruggenthaler, M.; Appel, H.; Rubio, A. _J. Phys. Chem. Lett._ 2020, _11_ , 7525–7530
  * Power et al. 1959 Power, E. A.; Zienau, S.; Massey, H. S. W. _Philos. Trans. R. Soc. Lond. Ser. Math. Phys. Sci._ 1959, _251_ , 427–454
  * Schäfer et al. 2020 Schäfer, C.; Ruggenthaler, M.; Rokaj, V.; Rubio, A. _ACS Photonics_ 2020, _7_ , 975–990
  * Climent and Feist 2020 Climent, C.; Feist, J. _Phys. Chem. Chem. Phys._ 2020, _22_ , 23545–23552
  * Richter et al. 2004 Richter, F.; Hochlaf, M.; Rosmus, P.; Gatti, F.; Meyer, H.-D. _J. Chem. Phys._ 2004, _120_ , 1306–1317
  * Richter et al. 2004 Richter, F.; Rosmus, P.; Gatti, F.; Meyer, H.-D. _J. Chem. Phys._ 2004, _120_ , 6072–6084
  * Richter et al. 2007 Richter, F.; Gatti, F.; Léonard, C.; Le Quéré, F.; Meyer, H.-D. _J. Chem. Phys._ 2007, _127_ , 164315
  * Montgomery et al. 1979 Montgomery, J. A.; Chandler, D.; Berne, B. J. _J. Chem. Phys._ 1979, _70_ , 4056–4066
  * Rosenberg et al. 1980 Rosenberg, R. O.; Berne, B. J.; Chandler, D. _Chemical Physics Letters_ 1980, _75_ , 162–168
  * Kuharski et al. 1988 Kuharski, R. A.; Chandler, D.; Montgomery, J. A.; Rabii, F.; Singer, S. J. _J. Phys. Chem._ 1988, _92_ , 3261–3267
  * Li et al. 2020 Li, T. E.; Subotnik, J. E.; Nitzan, A. _Proc. Natl. Acad. Sci._ 2020, _117_ , 18324–18331
  * Wang et al. 2021 Wang, D. S.; Neuman, T.; Yelin, S. F.; Flick, J. _ArXiv210906631 Nlin Physicsphysics Physicsquant-Ph_ 2021
  * Chandler 1987 Chandler, D. _Introduction to Modern Statistical Mechanics_ ;  Oxford University Press: New York, 1987
  * Berne et al. 1988 Berne, B. J.; Borkovec, M.; Straub, J. E. _J. Phys. Chem._ 1988, _92_ , 3711–3725
  * Eyring 1935 Eyring, H. _J. Chem. Phys._ 1935, _3_ , 107–115
  * Hänggi et al. 1990 Hänggi, P.; Talkner, P.; Borkovec, M. _Rev. Mod. Phys._ 1990, _62_ , 251–341
  * Masgrau et al. 2002 Masgrau, L.; González-Lafont, À.; Lluch, J. M. _J. Phys. Chem. A_ 2002, _106_ , 11760–11770
  * Miller 1979 Miller, W. H. _J. Am. Chem. Soc._ 1979, _101_ , 6810–6814
  * Matzkies and Manthe 1998 Matzkies, F.; Manthe, U. _J. Chem. Phys._ 1998, _108_ , 4828–4836
  * Vendrell 2018 Vendrell, O. _Phys. Rev. Lett._ 2018, _121_ , 253001
  * Schäfer et al. 2021 Schäfer, C.; Flick, J.; Ronca, E.; Narang, P.; Rubio, A. _ArXiv210412429 Phys. Physicsquant-Ph_ 2021
  * Li et al. 2021 Li, T. E.; Nitzan, A.; Subotnik, J. E. _Angew. Chem. Int. Ed._ 2021, _60_ , 15533–15540