11institutetext: The University of Texas at Dallas, Richardson, USA
11email: {skm200005, bxz200006<EMAIL_ADDRESS>
# PALM: Pushing Adaptive Learning Rate Mechanisms for Continual Test-Time
Adaptation

Sarthak Kumar Maharana  Baoming Zhang  Yunhui Guo

###### Abstract

Real-world vision models in dynamic environments face rapid shifts in domain
distributions, leading to decreased recognition performance. Continual test-
time adaptation (CTTA) directly adjusts a pre-trained source discriminative
model to these changing domains using test data. A highly effective CTTA
method involves applying layer-wise adaptive learning rates, and selectively
adapting pre-trained layers. However, it suffers from the poor estimation of
domain shift and the inaccuracies arising from the pseudo-labels. In this
work, we aim to overcome these limitations by identifying layers through the
quantification of model prediction uncertainty without relying on pseudo-
labels. We utilize the magnitude of gradients as a metric, calculated by
backpropagating the KL divergence between the softmax output and a uniform
distribution, to select layers for further adaptation. Subsequently, for the
parameters exclusively belonging to these selected layers, with the remaining
ones frozen, we evaluate their sensitivity in order to approximate the domain
shift, followed by adjusting their learning rates accordingly. Overall, this
approach leads to a more robust and stable optimization than prior approaches.
We conduct extensive image classification experiments on CIFAR-10C,
CIFAR-100C, and ImageNet-C and demonstrate the efficacy of our method against
standard benchmarks and prior methods.

##  1 Introduction

In a real-world setting, machine-perception systems [1] function in a rapidly
changing environment. In such contexts, pre-trained vision models are
susceptible to performance degradation caused by potential distribution
shifts. For instance, while these models may have been trained on clean
images, images captured by sensors could be corrupted by various weather
conditions.

In the realm of addressing domain shifts due to image corruptions, test-time
adaptation (TTA) has been gaining a lot of traction lately [6, 44]. Here, a
pre-trained source model, with no access to its training (source) data due to
privacy, is adapted to unlabeled test data through online training. One
strategy in TTA involves mitigating domain shifts by adjusting the source
model parameters using pseudo labels and minimizing the entropy of the model
predictions [44]. While this approach effectively adapts the model to a single
type of corruption, it faces two constraints when continually applied across a
sequence of test tasks featuring different corruptions. Firstly, there is the
issue of error accumulation: since pseudo labels are generated for unlabeled
test data, errors accumulate over time. Secondly, catastrophic forgetting
occurs, resulting in a long-term loss of the model’s pre-trained knowledge due
to continuous adaptations and large parameter updates across tasks.

Continual test-time adaptation (CTTA) has been recently introduced to tackle
these challenges. In [32, 8, 44, 37], stable optimization is achieved by
updating the batch normalization (BN) parameters or re-formulating its
statistics. CoTTA [45] includes full model updates, with a teacher-student
network setup, to handle the above issues. To further improve the performance
of CoTTA, EcoTTA [41] introduces the attachment of "meta networks", _i.e_., a
BN layer and a convolution block, to the frozen source model for efficient
adaptation.

More recently, the authors in [21] demonstrated that based on the type of
distributional shift of the test data, a certain subset of layers of a pre-
trained model can be "surgically" or heuristically chosen to fine-tune.
Inspired by this finding, Layer-wise Auto-Weighting (LAW) [33] was introduced
recently. LAW utilizes the Fisher Information Matrix (FIM) [36] to estimate
domain shifts, automatically weighting trainable parameters and adjusting
their learning rates accordingly. While LAW achieves state-of-the-art results
for CTTA, it still encounters two significant limitations: 1) It presupposes
that layer-wise importance can be accurately estimated via FIM with initially
generated pseudo-labels. Nevertheless, it is widely recognized that pseudo-
labels are inherently noisy and unreliable. 2) Relying on directly
accumulating such approximated importance from noisy labels to calculate
domain shifts can lead to inaccuracies.

In this paper, we present a novel method called PALM to push the limit of
adaptive learning rate methods for CTTA. Our approach begins by identifying a
subset of layers that significantly contribute to the uncertainty of model
predictions. Rather than relying on pseudo-labels, we determine layer-wise
importance by leveraging information from the gradient space, specifically by
calculating the KL divergence between the model output and a uniform
distribution. This approach eliminates dependence on pseudo-labels and enables
a more accurate estimation of layer-wise importance. Subsequently, the
learning rates of the parameters of these layers are adjusted based on the
degree of domain shift observed in the test data. We utilize parameter
sensitivity as an indicator of the degree of distribution shift and employ
weighted moving averages to aggregate sensitivity from previous data batches.
Compared to existing adaptive learning rate methods, our proposed approach
offers a better estimation of domain shift for adjusting learning rates.

To summarise, our contributions are as follows:

  * •

We present PALM, a novel approach to continual test-time adaptation.
Specifically, we demonstrate improved schemes for adaptive learning rate
methods for CTTA.

  * •

Considering the prediction uncertainty of the model at test-time, we design an
approach to automatically select layers that would require further adaptation
and push the learning rates of parameters of these layers for model updates,
by estimating the domain shift based on parameter sensitivity.

  * •

Our method delivers promising results in both continual and gradual TTA across
three standard benchmark datasets. In addition, we conduct and demonstrate
ablation experiments on the different components of our work.

##  2 Related Works

Test-time adaptation (TTA): The overarching target of TTA is to adapt source
models to unlabelled test data in real-time in a source-free online training
fashion [16, 42, 44]. TENT [44] optimizes the Batch Normalisation (BN)
parameters by minimizing the Shannon entropy [40] of the pseudo labels. In
[37], the statistics of the BN parameters are replaced by the ones of the
input corrupted data. Adacontrast [3] combines contrastive learning and
utilizes the memory module to update the entire model’s parameters. While
previous TTA methods showed significant improvements, they were demonstrated
for a single domain shift at a time [4, 26, 37, 44] and did not consider
continual distribution shifts.

Continual TTA (CTTA): CTTA poses an adaptation challenge wherein the model
must continually adjust to evolving target data, accommodating a sequence of
dynamic domain shifts. The first work, CoTTA [45], considers addressing this
challenging task by setting up a teacher-student network, with full model
updates. The teacher model is updated using an exponential moving average
(EMA) to deal with forgetting and accumulation of errors with time. EcoTTA
[41] introduces "meta networks" _i.e_. a BN layer and a convolution block are
attached to the frozen source model. At every stage, an $L_{1}$ regularisation
loss between the outputs of the meta network and that of the source model is
computed to handle catastrophic forgetting.

Efficient fine-tuning approaches: Freezing model parameters to learn
information from downstream data has witnessed a good body of work [39, 27,
51, 22, 7, 9]. Depending upon the distribution shift of the target task, in
[21], the authors demonstrated that heuristically selecting a subset of
convolutional blocks for training while freezing the remaining blocks works
better than full fine-tuning.

Adaptive-learning rate methods: Regarding the studies on the importance of
each layer in training, [47, 49] concluded that deeper layers extract task-
specific features, as opposed to the shallow layers. Based on this finding,
AutoLR [35] design an adapted learning rate (LR) scheme based on the weight
variations between layers. Despite attempts to set different initial learning
rates for each layer, these rates remain mostly constant throughout training.
In the context of CTTA, LAW [33] used the FIM [36] to calculate the domain
shift and automatically adjust the learning rates of parameters. However, to
obtain the layer importance, the pseudo-labels are used to compute the FIM,
which would be inaccurate. In addition, a direct accumulation of FIMs across
domains would lead to error accumulation. Furthermore, the LR "weight" of each
layer is upper-bounded by the base LR that is used, which ignores the relative
"sensitivity" of parameters towards adaptation of the current batch and
domain.

Parameter sensitivity: In the literature about neural network pruning,
understanding the sensitivity of parameters towards training has been studied
well [19, 28, 43, 29, 23]. The overall idea is that parameters with lower
sensitivity need to be adapted more with an increased learning rate, and vice-
versa. [23], in addition, introduces a temporal variation of sensitivity, due
to uncertainty, as another factor during the training of models for natural
language understanding and machine translation tasks.

##  3 Background

###  3.1 Problem Setting

The objective of continual test-time adaptation (CTTA) is to adapt a fixed
discriminative pre-trained source model, parameterized by ${\theta_{s}}$ and
trained on the source dataset ($\mathcal{X}^{source}$,
$\mathcal{Y}^{source}$), to multiple incoming tasks $\mathcal{T}_{i}$ =
{$x_{k}$}${}_{k=1}^{n}$ where $\mathcal{T}_{i}$ refers to the $i^{th}$ task
and n is the total number of batches of $\mathcal{T}_{i}$, with changes in
domain distributions between tasks. At each time step, ${\theta_{s}}$ is
gradually adapted to each incoming batch ($x_{k}$) of task $\mathcal{T}_{i}$
in an online manner, where the model parameters are updated to ${\theta}$. In
this entire process of adaptation, the source dataset is unavailable for use,
due to privacy and storage constraints. As evident, the source model is first
parameterized by ${\theta_{s}}$ before the adaptation begins. With the
continuum of tasks, at time step t, the complete model parameters
${\theta}^{t}$ consists of a matrix of parameters, given by ${\theta}^{t}$ =
{${\theta^{t}_{1}}$, $\theta^{t}_{2}$,…, ${\theta^{t}_{n}}$,….,
${\theta^{t}_{N}}$}, where ${\theta^{t}_{n}}$, at time t, denotes the
parameters of the $n^{th}$ trainable layer out of a total of N.

###  3.2 Analysis of Adaptive Learning Rate Schemes for CTTA

In this subsection, we provide a detailed view and analysis of surgical fine-
tuning [21], AutoLR [35], and LAW [33]. This is essential to understand the
design principles and reasons for having an adaptive learning rate scheme for
CTTA, which also includes the motivation of our work.

To preserve the information during the source pre-training phase while fine-
tuning on a target task of a different distribution, surgical fine-tuning [21]
empirically show that heuristically fine-tuning only the first convolutional
block results in a better performance over full fine-tuning. Even though it is
a desirable outcome for efficient fine-tuning, in this work, we close the gap
by automatically selecting the layers that would adapt well to the target
task.

Inspired by [47, 49], AutoLR [35] claims that the weight changes across layers
should be in increasing order, where the weight change $\delta w_{n}$ in the
$n^{th}$ layer with weights $w_{n}$, at epoch e, is defined as,

$\delta w_{n}=\|w_{n}^{e}\ -w_{n}^{(e-1)}\|$ (1)

They hypothesize that weight changes in the initial layers should be small
because they extract general features whereas the deeper layers extract task-
specific features. A constrained adaptive LR optimization scheme is set up to
force the weight variations to increase with the layers without studying the
importance of each layer during adaptation. However, the updated LRs, for each
layer, do not deviate too much and remain constant throughout.

As the first adaptive LR work for CTTA, LAW [33] hypothesizes that identifying
and controlling the adaptation rate of the different layers is useful [21]. To
understand the loss landscape and layer importance, the authors compute the
gradient from the log-likelihood as,

$s(\theta_{n}^{t};x_{k})=\nabla_{\theta_{n}^{t}}\log(p_{\theta^{t}}(x_{k}))$
(2)

where $x_{k}$ and $\theta_{n}^{t}$ refer to the current test batch and
parameters of layer n at time t. Then, the Fisher Information Matrix (FIM)
[36], as a domain-shift indicator, is computed as a first-order approximation
of the Hessian matrix. As it can be seen, the layer-wise importance (Eq. 2)
completely relies on the pseudo-labels to compute the log-likelihood, which is
highly unreliable and uncertain. Due to the distribution shift, the label
quality would begin to decrease for a continuum of tasks. Furthermore, the
authors propose to capture "domain-level" FIM ($\hat{F_{n}^{t}}$), for each
layer during the adaptation, by a direct accumulation of the current layer-
wise FIM (${F_{n}^{t}}$) as,

$\hat{F}_{n}^{t}=\hat{F}_{n}^{t-1}+F_{n}^{t}$ (3)

Based on Eq. 3, this means that LAW [33] puts equal emphasis on
$\hat{F}_{n}^{t}$ and ${F_{n}^{t}}$, for each layer. An issue arises when
${F}_{n}^{t}$ of the current batch is small but gets outweighed by a
"relatively" larger $\hat{F}_{n}^{t}$. This raises questions on the amount of
adaptation required by the $n^{th}$ layer for batch $x_{k}$ as the FIMs
dynamically change for each test batch. This leads to an imbalance between the
accumulated FIMs $\hat{F}_{n}^{t}$ and the current FIM ${F_{n}^{t}}$.

Drawing from these primary observations, our method focuses on addressing the
following: 1) Based on the prediction uncertainty of the model, we
automatically select a subset of layers that would require further adaptation
to a task, without relying on the pseudo-labels. 2) As an indicator of the
domain shift, we leverage the idea of parameter sensitivity of these layers
only and adaptively alter their learning rates. We freeze the layers that are
not selected.

##  4 Proposed Method

---
Figure 1: PALM framework: At time t, input batch $x_{k}$ is processed by the
model, parameterized by $\theta^{t}$. The KL-divergence between the softmax
predictions and a uniform distribution is backpropagated to select the layers
with the gradient norm $\leq$ $\eta$, to quantify the uncertainty. The
parameter sensitivities of these layers, as an indicator of domain shift
meters, are computed to update their learning rates (Eq. 11). Finally, with
the optimization objective in Eq. 16, we update the model with the adjusted
learning rates of the parameters.

In this section, we introduce and formalize our proposed method for CTTA to
address the key limitations of the works discussed above. Fig. 1
diagrammatically illustrates our entire approach.

###  4.1 Layer Selection by Quantifying Prediction Uncertainty

CTTA maximizes model performance in the current domain and is vital to assess
prediction certainty amidst domain shifts. In the literature on continual
learning and TTA, several works utilize FIM [18, 38, 33]. However, the
unavailability of true labels, with an unknown distribution, motivates us to
capture the prediction uncertainty. Prior literature proposed methods for
uncertainty from the classifier output or feature space [12, 14, 24, 25, 20].
Motivated by [15], which proposed an uncertainty scoring function from the
gradient space, we compute the gradients of each layer. In essence, we
backpropagate the KL divergence (KL) between the softmax probabilities and a
uniform distribution u = [$\frac{1}{C}$, $\frac{1}{C}$, …., $\frac{1}{C}$]
$\in$ $\mathbb{R}^{C}$, where C is the total number of classes. This is a
measurement of how far the distribution of the model’s prediction is from a
uniform distribution. Intuitively, due to the distribution shift, the KL
divergence should be smaller. Let h denote the classifier of the model
parameterized by ${\theta^{t}}$. We compute the loss, denoting it as
$\mathcal{L}(\theta^{t})$ at time stamp t, as follows,

$\mathcal{L}(\theta^{t})=\text{KL}(\text{softmax}(\hat{h}(x_{k}))||\text{{u}})$
(4)

where $\hat{h}(x_{k})$ is the classifier’s output logits ($h(x_{k})$) smoothed
by a temperature coefficient T _i.e_. $\hat{h}(x_{k})$ = $h(x_{k})$/T. Our
intuition is that batches within the same task are of the same distribution
and so, smoothing the logits with large values of T will lead to them being
indistinguishable. With model parameters $\theta^{t}$, the gradients
(${{\nabla\mathcal{L}({\theta}^{t})}}$ =
$\frac{\partial\mathcal{L}(\theta^{t})}{\partial{\theta}^{t}}$) can be
simplified to,

${{\nabla\mathcal{L}({\theta}^{t})}}=\frac{1}{C}\sum_{j=1}^{C}\frac{\partial\mathcal{L_{\text{CE}}}(\hat{h}(x_{k}),j)}{\partial\theta^{t}}$
(5)

On the whole, the gradients boil down to the averaged cross-entropy loss
between the smoothed output logits and a uniform label (Eq. 5). In test-time
adaptation, since ground-truth labels are not present, the gradients are more
indicative of familiarity of the model with the batch of data. For batches
within the same task, that share the same distribution, the vector norm of Eq.
5 would be higher because of a higher KL divergence since the softmax
predictions would be less uniformly distributed. However, if the norm is
smaller, this would mean that the softmax outputs are closer to u. Our key
idea is to select all the parameters of layers that have the norm of gradients
to be less than or equal to a threshold $\eta$. Layers with small gradient
norms would require more adaptation for the model to make a confident
prediction. Since we freeze the remaining layers, this also helps to alleviate
catastrophic forgetting to a large extent by preserving the prior knowledge.
And so the scoring function, for each layer, is defined as,

$\mathcal{Z}_{\theta_{n}}=\|{{\nabla\mathcal{L}({\theta}_{n}^{t})}}\|_{1}$ (6)

As mentioned earlier, $\mathcal{Z}_{\theta_{n}}$ $\leq$ $\eta$ allows the
model to adaptively determine which specific layers need further adaptation.
We freeze the layers with $\mathcal{Z}_{\theta_{n}}$ $>$ $\eta$ _i.e_. set
their learning rate to 0. This way, in contrast to surgical fine-tuning [21],
we automatically select a subset of layers for adaptation to a batch of data.

###  4.2 Parameter Sensitivity as an Indicator of Domain Shift

New studies have found that model parameters demonstrate levels of criticality
[2, 50] across different positions. Moreover, not all parameters adapt
uniformly, and removing some can lower test generalization errors [31, 34].
Inspired by this observation, we claim that parameters have different
contributions during test-time adaptation. From Sec. 4.1, we adaptively select
layers that are uncertain in their predictions. For parameters of these layers
only, we compute their sensitivity _i.e_. an approximation of an error when it
is removed.

In our work, motivated from [19, 30, 29], we compute the parameter sensitivity
of a parameter $\theta_{n}^{t}$ based on the loss (Eq. 4) as follows,

$S_{n}^{t}=\mathcal{L}({\theta}^{t})-\mathcal{L}({\theta}^{t}|\theta_{n}^{t}=0)$
(7)

Precisely, $S_{n}^{t}$ approximates the change in the loss landscape at
$\theta^{t}$ when $\theta_{n}^{t}$ is removed. Sensitivity, computed as a
"weighting factor" of model weights [30, 29, 23], determines the level of
adaptation needed for model parameters.

We utilize the gradients of parameters from the selected layers, computed by
Eq. 5 to compactly derive Eq. 7 from the first-order Taylor series expansion
of $\mathcal{L}(\theta^{t})$ as,

$S_{n}^{t}=|\theta_{n}^{t}\nabla\mathcal{L}({\theta_{n}}^{t})|$ (8)

Our rationale is as follows: parameters with low sensitivity are pushed to
have higher learning rates owing to more adaptation. However, $S_{n}^{t}$ of
$\theta_{n}^{t}$ can be highly uncertain and would have a large variation. We
introduce a method to capture this "domain-level" uncertainty by computing a
weighted moving average of the sensitivity $S_{n}^{t}$. This is formulated as,

$\hat{S}_{n}^{t}=\alpha S_{n}^{t}+(1-\alpha)\hat{S}_{n}^{(t-1)}$ (9)

where $\hat{S}_{n}^{t}$ denotes the "domain-level" sensitivity and $\alpha$
$\in$ [0,1] is a smoothing factor. Specifically, $\alpha$ decides the
contribution of the current sensitivity $S_{n}^{t}$ to $\hat{S}_{n}^{t}$.
Since CTTA happens at batch level, it is critical to achieve a good balance
between $S_{n}^{t}$ and $\hat{S}_{n}^{t}$, with a slightly larger focus on
$S_{n}^{t}$. As it can be seen, we do not directly accumulate the "domain-
level" information, unlike LAW [33].

In addition, due to large distribution shifts happening because of the
continuum of tasks, larger deviations of $S_{n}^{t}$ from $\hat{S}_{n}^{t}$
are bound to happen. To quantify these statistics, an absolute difference
between $\hat{S}_{n}^{t}$ and $S_{n}^{t}$ can be computed as,

$\hat{D}_{n}^{t}=|S_{n}^{t}-\hat{S}_{n}^{t}|$ (10)

We emphasize that since Eq. 9 leverages sensitivity from previously computed
batches and domains, $\hat{D}_{n}^{t}$ would act as a good indicator of how
uncertain $S_{n}^{t}$ is to the previous batches and domains, for
$\theta_{n}$. This would take care of error accumulation too.

To automatically alter the learning rates of these parameters, the learning
rate "importance" ($i_{n}^{t}$) is calculated as follows,

$i_{n}^{t}=(\hat{D}_{n}^{t})/(\hat{S}_{n}^{t})$ (11)

Then, the final learning rate to update $\theta_{n}^{t}$ would be

$\hat{\kappa}_{n}^{t}=\kappa*i_{n}^{t}$ (12)

where $\kappa$ is the base fixed learning rate. The intuition behind the
formulation of Eq. 11 is as follows - smaller values of $\hat{S}_{n}^{t}$
indicate lower sensitivity and hence larger parameter updates are required via
promoting the learning rate. At the same time, large uncertainties by
$\hat{D}_{n}^{t}$ would also require larger updates. To avoid any
overflow/underflow, we add $\epsilon$ (0<$\epsilon$$\ll$1) to the numerator
and the denominator in Eq. 11.

###  4.3 Optimization Objective

Though our method is robust, it is necessary to avoid miscalibrated labels.
TENT [44] minimized the Shannon entropy on the output logits, defined as,

$\mathcal{H}(\hat{y})=-\sum_{c}p(y_{c})log(p(y_{c}))$ (13)

where $p(y_{c})$ refers to the softmax probability of class c _i.e_.
$p(y_{c})$ = softmax($h(x_{k})$). Following previous CTTA work [41], we use
entropy minimization as the main loss objective for adaptation where Eq. 13 is
minimized for samples below a certain threshold.

$\mathcal{L}^{entropy}=\mathds{1}_{\mathcal{H}(\hat{y})\leq\mathcal{H}_{0}}\mathcal{H}(\hat{y})$
(14)

where $\mathds{1}$ is the indicator function and $\mathcal{H}_{0}$ is a fixed
threshold set to $0.4*lnC$, consistently following [41]. To be fair with prior
works [45, 33], we use the consistency loss between the model predictions on
$x_{k}$ and its augmented version $\hat{x}_{k}$ to regularize the model as
follows,

$\mathcal{L}^{const}=-\sum_{c}p(y_{c})log(p(\hat{y}_{c}))$ (15)

where, $\hat{y}_{c}$ = $h(\hat{x}_{k})$. Our overall optimization objective is
as follows,

$\mathcal{L}^{total}=\mathcal{L}^{entropy}+\lambda\mathcal{L}^{const}$ (16)

where $\lambda$ indicates a regularisation coefficient to balance the
consistency loss. In summary, upon calculating the parameter-specific learning
rates (Eq. 12), we optimize the continually fine-tuned model to minimize Eq.
16.

##  5 Experiments

We draw a contrast of our proposed method against other state-of-the-art works
in a continual test-time adaptation (CTTA) setting. However, to have a broader
study and analysis, we also report results in a gradual test-time adaptation
(GTTA) setting, following [45, 33]. In the following subsections, we describe
the implementation details of both the challenging adaptation settings. For
additional results and analysis, please refer to the Supplementary Material.

###  5.1 Experimental Settings and Baselines

Datasets and tasks. Following the standard benchmarks in CTTA, set by CoTTA
[45], we evaluate our proposed method on datasets - CIFAR-10C, CIFAR-100C, and
ImageNet-C, based on image corruption schemes as set in [11]. Each dataset
contains a set of 15 corruption styles (_e.g_. gaussian noise, shot noise,
impulse noise, defocus blur) with 5 severity levels. In line with established
practices outlined in [45, 6], we report the mean classification error as the
evaluation metric.

CTTA implementation details. We evaluate our approach using frameworks from
prior state-of-the-art methods [44, 45, 3, 41, 37, 33]. To maintain fairness
regarding the usage of pre-trained vision models, we employ WideResNet-28 [48]
for CIFAR-10C, ResNeXt-29 [46] for CIFAR-100C, and ResNet-50 [10] for
ImageNet-C, all available on RobustBench [5]. We optimize using an Adam
optimizer [17], setting base learning rates ($\kappa$) to 5e-4 for CIFAR-10C
and CIFAR-100C, and 5e-5 for ImageNet-C. For balancing parameter
sensitivities, we set $\alpha$ to 0.5, 0.9, and 0.5 respectively, with
temperature coefficients T set to 50, 100, and 1000 respectively. We set
$\eta$ to 1, 0.5, and 0.3, and $\lambda$ to 0.01 throughout. Batch sizes are
set to 200, 200, and 64 for each dataset, following CoTTA [45], and results
are reported for the highest severity level of 5 for each task.

GTTA implementation details. In the standard CTTA setup, each task receives
images with the highest severity. However, in GTTA, following [45, 41, 33],
for each task the severity levels are gradually changed as
…3$\rightarrow$2$\rightarrow$1 $\xrightarrow[change]{task}$
1$\rightarrow$2$\rightarrow$3$\rightarrow$4$\rightarrow$5$\rightarrow$4$\rightarrow$3$\rightarrow$2$\rightarrow$1
$\xrightarrow[change]{task}$ 1$\rightarrow$2$\rightarrow$3… We maintain
consistency with the experimental details outlined in the CTTA setting.

Baselines. We compare our work against several CTTA baselines, including
Source, BN Stats Adapt [37], TENT Continual [44], CoTTA [45], EcoTTA [41],
Adacontrast [3], and LAW [33]. In addition, we also contrast our work against
surgical fine-tuning [21]. For EcoTTA, we report the results on CIFAR-10C and
ImageNet-C only since they do not use ResNeXt-29 [46] for CIFAR-100C.
Following the findings of surgical fine-tuning [21], we continually fine-tune
only the first convolutional block of the respective models, denoted as
Surgical fine-tuning (continual). We have mentioned the training details in
the Supplementary. For the reported results of LAW [33], we reproduce their
results strictly following their official implementation
111https://github.com/junia3/LayerwiseTTA/tree/main with parameters as
mentioned in their paper.

Table 1: Mean errors (in %) on CIFAR-10C \- We report the CTTA mean errors of
the 15 corruptions at a severity level of 5. We report the error of a single
update (online) only using WideResNet-28 [48] for all the reported baselines.

Method | Gaussian | Shot | Impulse | Defocus | Glass | Motion | Zoom | Snow | Frost | Fog | Brightness | Contrast |  | Elastic
---
Transform
Pixelate | JPEG | Mean
Source | 72.3 | 65.7 | 72.9 | 46.9 | 54.3 | 34.8 | 42.0 | 25.1 | 41.3 | 26.0 | 9.3 | 46.7 | 26.6 | 58.5 | 30.3 | 43.5
BN Stats Adapt [37] | 28.1 | 26.1 | 36.3 | 12.8 | 35.3 | 14.2 | 12.1 | 17.3 | 17.4 | 15.3 | 8.4 | 12.6 | 23.8 | 19.7 | 27.3 | 20.4
TENT Continual [44] | 24.8 | 20.6 | 28.6 | 14.4 | 31.1 | 16.5 | 14.1 | 19.1 | 18.6 | 18.6 | 12.2 | 20.3 | 25.7 | 20.8 | 24.9 | 20.7
| CoTTA [45]
---
24.3 | 21.3 | 26.6 | 11.6 | 27.6 | 12.2 | 10.3 | 14.8 | 14.1 | 12.4 | 7.5 | 10.6 | 18.3 | 13.4 | 17.3 | 16.2
| ECoTTA [41]
---
23.8 | 18.7 | 25.7 | 11.5 | 29.8 | 13.3 | 11.3 | 15.3 | 15.0 | 13.0 | 7.9 | 11.3 | 20.2 | 15.1 | 20.5 | 16.8
| Adacontrast [3]
---
29.1 | 22.5 | 30.0 | 14.0 | 32.7 | 14.1 | 12.0 | 16.6 | 14.9 | 14.4 | 8.1 | 10.0 | 21.9 | 17.7 | 20.0 | 18.5
| Surgical
---
fine-tuning [21]
(continual)
27.71 | 24.14 | 33.79 | 12.18 | 33.07 | 13.09 | 10.97 | 15.94 | 15.61 | 13.8 | 7.84 | 10.77 | 21.17 | 16.62 | 22.88 | 18.64
| LAW [33]
---
24.71 | 18.91 | 25.56 | 12.96 | 26.78 | 15.09 | 11.8 | 15.17 | 14.78 | 15.89 | 10.13 | 13.89 | 19.41 | 14.74 | 18.35 | 17.21
| Ours
---
25.89 | 18.1 | 22.71 | 12.38 | 25.3 | 13.18 | 10.79 | 13.5 | 13.15 | 12.21 | 8.53 | 11.85 | 17.98 | 12.02 | 15.47 | 15.54

Table 2: Mean errors (in %) on CIFAR-100C \- We report the CTTA mean errors of
the 15 corruptions at a severity level of 5. We report the error of a single
update (online) only using ResNeXt-29 [46] for all the reported baselines.

Method | Gaussian | Shot | Impulse | Defocus | Glass | Motion | Zoom | Snow | Frost | Fog | Brightness | Contrast |  | Elastic
---
Transform
Pixelate | JPEG | Mean
| Source
---
73.0 | 68 | 39.4 | 29.3 | 54.1 | 30.8 | 28.8 | 39.5 | 45.8 | 50.3 | 29.5 | 55.1 | 37.2 | 74.7 | 41.2 | 46.4
| BN Stats Adapt [37]
---
42.1 | 40.7 | 42.7 | 27.6 | 41.9 | 29.7 | 27.9 | 34.9 | 35.0 | 41.5 | 26.5 | 30.3 | 35.7 | 32.9 | 41.2 | 35.4
| TENT Continual [44]
---
37.2 | 35.8 | 41.7 | 37.9 | 51.2 | 48.3 | 48.5 | 58.4 | 63.7 | 71.1 | 70.4 | 82.3 | 88.0 | 88.5 | 90.4 | 60.9
| CoTTA [45]
---
40.1 | 37.7 | 39.7 | 26.9 | 38.0 | 27.9 | 26.4 | 32.8 | 31.8 | 40.3 | 24.7 | 26.9 | 32.5 | 28.3 | 33.5 | 32.5
| Adacontrast [3]
---
42.3 | 36.8 | 38.6 | 27.7 | 40.1 | 29.1 | 27.5 | 32.9 | 30.7 | 38.2 | 25.9 | 28.3 | 33.9 | 33.3 | 36.2 | 33.4
| Surgical
---
fine-tuning [21]
(continual)
40.75 | 36.36 | 37.89 | 25.53 | 39.02 | 27.65 | 25.27 | 32.26 | 30.45 | 37.35 | 23.78 | 26.82 | 32.37 | 27.64 | 37.20 | 32.02
| LAW [33]
---
41.06 | 36.73 | 38.38 | 25.67 | 37.02 | 27.82 | 25.22 | 30.79 | 30.04 | 37.35 | 24.41 | 27.69 | 31.25 | 27.84 | 34.92 | 31.75
| Ours
---
37.33 | 32.5 | 34.94 | 26.20 | 35.28 | 27.54 | 24.64 | 28.85 | 29.15 | 34.14 | 23.52 | 26.99 | 31.15 | 26.56 | 34.1 | 30.19

###  5.2 CTTA Classification Results

Results on CIFAR-10C and CIFAR-100C. We contrast our proposed method against
several baselines including surgical fine-tuning [21]. Tables 1 and 2 report
the CTTA mean classification errors across all the domains, for each of the
methods. The results reveal that our method performs the best across all
training setups and baselines for both datasets. In particular, we achieve
large improvements of 4.86% and 5.16% for CIFAR-10C and 5.21% and 30.71% for
CIFAR-100C, over BN Stats Adapt[3] and TENT Continual[44] respectively.
Concerning recent work LAW [33] that uses an adaptive LR scheme, our method
achieves an improvement of 1.67% and 1.56% on CIFAR-10C and CIFAR-100C,
respectively. It’s noteworthy that TENT Continual initially excels in the
early tasks but experiences error accumulation over time, leading to an
increase in mean error. In the study by CoTTA [45], despite achieving good
performance, the approach’s reliance on full model updates renders it
computationally expensive and unsuitable for deployment. LAW [33] uses the
uncertain pseudo labels, of each batch, to calculate the domain shift
approximated via the FIM computation. The minor error that is introduced for
each batch begins to propagate across all the domains. In addition, the
imbalance caused by the "domain-level" FIM and the current batch’s FIM,
results in a higher error rate for the initial tasks. In comparison, the
computation of the model prediction uncertainty to select layers is useful in
our work (Eq. 4). Another interesting observation is that with a drastic
reduction in the number of trainable parameters, as in Surgical fine-tuning
(continual), the mean errors in Table 2 is better than most CTTA benchmarks,
if not the best. This affirms the findings in [21] that concentrated
adaptation, for image corruptions, in the initial layers is sufficient.

Results on Imagenet-C In Table 3, we report the CTTA results on the ImageNet-C
dataset, with ResNet-50 pre-trained on ImageNet. As shown, our method
outperforms all the previous CTTA baselines and Surgical fine-tuning
(continual). We observe an improvement of 1% over LAW [33]. In addition, we
achieve 2.31%, 5.38%, and 30.71% improvements over CoTTA [45], Adacontrast
[3], and TENT [44], respectively.

Table 3: Mean errors (in %) on ImageNet-C \- We report the CTTA mean errors of
the 15 corruptions at a severity level of 5. We report the error of a single
update (online) only using ResNet-50 [10] for all the reported baselines.

Method | Gaussian | Shot | Impulse | Defocus | Glass | Motion | Zoom | Snow | Frost | Fog | Brightness | Contrast |  | Elastic
---
Transform
Pixelate | JPEG | Mean
| Source
---
97.8 | 97.1 | 98.2 | 81.7 | 89.8 | 85.2 | 78.0 | 83.5 | 77.1 | 75.9 | 41.3 | 94.5 | 82.5 | 79.3 | 68.6 | 82.0
| BN Stats Adapt [37]
---
85.0 | 83.7 | 85.0 | 84.7 | 84.3 | 73.7 | 61.2 | 66.0 | 68.2 | 52.1 | 34.9 | 82.7 | 55.9 | 51.3 | 59.8 | 68.6
| TENT Continual [44]
---
81.6 | 74.6 | 72.7 | 77.6 | 73.8 | 65.5 | 55.3 | 61.6 | 63.0 | 51.7 | 38.2 | 72.1 | 50.8 | 47.4 | 53.3 | 62.6
| CoTTA [45]
---
84.7 | 82.1 | 80.6 | 81.3 | 79.0 | 68.6 | 57.5 | 60.3 | 60.5 | 48.3 | 36.6 | 66.1 | 47.2 | 41.2 | 46.0 | 62.7
| ECoTTA [41]
---
- | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 63.4
| Adacontrast [3]
---
82.9 | 80.9 | 78.4 | 81.4 | 78.7 | 72.9 | 64.0 | 63.5 | 64.5 | 53.5 | 38.4 | 66.7 | 54.6 | 49.4 | 53.0 | 65.5
| Surgical
---
fine-tuning [21]
(continual)
83.42 | 79.5 | 77.28 | 86.1 | 82.4 | 72.28 | 61.18 | 63.1 | 64.76 | 51.26 | 35.92 | 72.9 | 56.24 | 46.72 | 50.2 | 65.55
| LAW [33]
---
81.86 | 75.04 | 72.12 | 77.42 | 73.22 | 63.96 | 54.6 | 57.92 | 61.06 | 50.1 | 36.36 | 68.66 | 49.08 | 46.22 | 49.12 | 61.12
| Ours
---
81.06 | 73.32 | 70.88 | 77.00 | 71.88 | 62.34 | 53.94 | 56.7 | 60.76 | 50.36 | 36.26 | 65.96 | 48.08 | 45.28 | 48.0 | 60.12

Table 4: Mean errors (in %) on CIFAR-10C, CIFAR-100C, and ImageNet-C \- We
report the gradual TTA (GTTA) mean errors averaged across all the severity
levels (1$\rightarrow$2$\rightarrow$3$\rightarrow$4$\rightarrow$5).

Dataset | Source |  | BN Stats
---
Adapt
| TENT
---
Continual
CoTTA | Adacontrast |  | Surgical
---
fine-tuning
(continual)
LAW | Ours
CIFAR-10C | 24.7 | 13.7 | 20.4 | 10.9 | 12.1 | 11.33 | 10.87 | 9.56
CIFAR-100C | 33.6 | 29.9 | 74.8 | 26.3 | 33.0 | 25.31 | 27.19 | 26.11
ImageNet-C | 58.4 | 48.3 | 46.4 | 38.8 | 66.3 | 41.36 | 40.52 | 38.49

###  5.3 GTTA Classification Results

Results on CIFAR-10C, CIFAR-100C, and Imagenet-C. This subsection presents the
gradual test-time adaptation (GTTA) results on CIFAR-10C, CIFAR-100C, and
Imagenet-C. For each task, the corruption severity of the inputs is gradually
increased from levels 1 to 5, as discussed in the implementation details in
Sec. 5.1. Table 4 reports the mean classification error for GTTA across all
severities. We observe that, for all the datasets, our approach outperforms or
performs comparably well across all the baselines [37, 44, 45, 3, 33]. With
most parameters frozen during the adaptation on each batch, our method is more
robust and inexpensive than CoTTA [45], which also demonstrates competitive
improvements in the mean classification error.

###  5.4 Ablation Studies

We conduct ablation experiments to validate our approach and gain insights
into its underlying mechanisms and components. Specifically, we conduct these
experiments and report results on the three CTTA benchmark datasets:
CIFAR-10C, CIFAR-100C, and ImageNet-100C.

WideResNet-28 [48] |  ResNeXt-29 [46] |  ResNet-50 [10]
---|---|---
|  |
|  |
Figure 2: Illustrations of the learning rate importances of WideResNet-28
[48], ResNeXt-29 [46], and ResNet-50 [10] during CTTA, for different
convolutional blocks/stages/layers. [Top row] - Variations across respective
datasets for domain glass_blur. [Bottom row] - Variations across respective
datasets for domain snow.

Analysis of the learning rate importance. The prime objective of our work is
to adaptively alter the learning rates of parameters of selected layers based
on their sensitivity. In Fig. 2, we illustrate how the learning rate (LR)
importance (Eq. 11) of WideResNet-28 [48], ResNeXt-29 [46], and ResNet-50
[10], averaged for a task, change during adaptation on their respective
datasets. For brevity, we show two domains - glass_blur and snow. As we can
see, the importance varies across domains and models. Across all the models,
we notice that generally, the deeper layers have smaller associated LR
importances, which means that the initial layers are adapted more to the
target domain. This result completely aligns with the findings of surgical
fine-tuning [21]. In addition to that, we also notice more sparsity in the
deeper layers. This further confirms the necessity of freezing parameters
automatically selected based on the target domain for adaptation.

Nullifying prediction uncertainty. In LAW [33], the authors make use of the
pseudo-labels to automatically decide on the importance of layers for
adaptation. In this, we study the effectiveness of such a design for our
method _i.e_. we investigate if layer importance computed using pseudo-labels
is helpful. We compute the gradients from the log-likelihood (Eq. 2) and
estimate the sensitivity of the parameters of the chosen layers. Table 5.4
reports the mean classification errors for all the datasets. We observe that
the gradients computed using the pseudo-labels are an unreliable indicator of
prediction uncertainty. The scoring function (Eq. 6) computed using these
gradients carries insufficient information. We achieve improvements of 2.44%,
3.69%, and 8.5% over CIFAR-10C, CIFAR-100C, and ImageNet-C, respectively.

|
---|---
a) Ablation on smoothing factor $\alpha$. | b) Ablation on temperature T.
Figure 3: Ablation results on smoothing factor $\alpha$ and temperature T for
CIFAR-10C, CIFAR-100C, and ImageNet-C.

Table 5: Prediction uncertainty
using pseudo-labels vs our method.

Dataset | Pseudo-labels | Ours
---|---|---
CIFAR-10C | 17.98 | 15.54
CIFAR-100C | 33.88 | 30.19
ImageNet-C | 68.62 | 60.12

Table 6: Ablation results on benchmark datasets with varying $\lambda$.

$\lambda$ | 0 | 0.01 | 0.05 | 0.1 | 0.5 | 1.0
---|---|---|---|---|---|---
CIFAR-10C | 16.84 | 15.54 | 16.45 | 17.03 | 17.65 | 17.93
CIFAR-100C | 30.44 | 30.19 | 30.51 | 30.57 | 30.63 | 30.7
ImageNet-C | 63.98 | 60.12 | 60.38 | 60.4 | 60.4 | 60.42

Contribution of smoothing factor $\alpha$. An important component of our
method is the computation of the moving average of sensitivities across data
batches, smoothed by a parameter $\alpha$ (Eq. 9). Fig. 3(a) illustrates the
variations of the mean errors, for the benchmark datasets, for varying
$\alpha$. We observe that the "domain-level" sensitivity across past batches
of data is indeed useful. For $\alpha$ = 0 _i.e_. when the current sensitivity
of parameters of selected layers is not considered, the mean classification
error for the respective datasets drastically spikes up. This suggests that
the effect of the current sensitivity of selected layers is indeed required.
For CIFAR-10C, we achieve the best performance when $\alpha$ = 0.5 with an
improvement of 1.67% over LAW [33]. Similarly, for CIFAR-100C and ImageNet-C,
we see improvements of 1.56% and 1% over LAW [33], at $\alpha$ = 0.9 and 0.5
respectively. With higher values of $\alpha$, the results do not vary too much
and are stable. Hence, attaining a balance between sensitivities from past
domains ($\hat{S}_{n}^{t}$) and current sensitivity ($S_{n}^{t}$) is critical,
as seen in the figure.

Logit smoothing via temperature coefficient T. In our method, we compute the
KL divergence between a uniform distribution u and a smoothed classifier’s
output logits (Eq. 4), denoted by $\hat{h}(x_{k})$. To study its influence, we
choose T $\in$ {0.5, 1, 10, 50, 100, 200, 1000}. We illustrate the results for
the three datasets in Fig. 3(b). Ideally, the temperature is an indicator of
the entropy of the predictions with large values pushing the distributions
closer to u [13]. Larger values of T spread the probability among inputs. In
CTTA, since we deal with domain shifts with no labels, we argue that higher
values of T are indeed useful to capture the uncertainty. With the output
distribution tending towards u, the KL divergence reduces (Eq. 4), indicating
more uncertainty of the model and hence, adaptation. As can be seen for a
difficult dataset like ImageNet-C with 1000 classes, larger values of T are
useful with the error reducing as T increases. For the CIFAR datasets, T = 50
and 100, respectively give the best performances. However, the errors begin to
rise for T $\geq$ 100 which could be due to extremely large spreads of the
distributions.

Robustness to the regularisation coefficient $\lambda$. As an important
ablation step, we understand the effect of the parameter $\lambda$ on the
final optimization objective, as in Eq. 16. We set $\lambda$ $\in$ {0, 0.01,
0.05, 0.1, 0.5, 1.0}. We study the contributions of the consistency loss to
entropy minimization. Table 5.4 reports the average classification errors on
CIFAR-10C, CIFAR-100C, and ImageNet-C, with varying $\lambda$. For $\lambda$ =
0.01, we achieve the best results across all the datasets. In addition, we
observe that our method is very robust to $\lambda$ _i.e_. with subtle
variations to differing $\lambda$, across the datasets. In particular, for
CIFAR-10C, with $\lambda$ = 0 _i.e_. with no consistency loss, we achieve
improvements of 0.37% and 3.86% over LAW [33] and TENT Continual [44]. For
CIFAR-100C, with no consistency loss ($\lambda$ = 0), we notice improvements
of 1.31%, 2.06%, 4.96%, and 30.46% over LAW [33], CoTTA [45], BN Stats Adapt
[37] and TENT Continual [44]. As for ImageNet-C, the consistent best setting
of $\lambda$ = 0.01 is better than LAW [33] by 1% and by 5.38% over
Adacontrast [3]. However, with $\lambda$ = 0 _i.e_. no consistency loss, we
see a spike in the error rate by about 3.86% over the best setting ($\lambda$
= 0.01).

##  6 Conclusion

In this paper, we propose a novel approach called PALM, to continual test-time
adaptation. Our approach automatically decides on the importance of a layer by
leveraging information from the gradient space _i.e_. by backpropagating the
KL divergence between a uniform distribution and the softmax outputs. This
effectively eliminates the need for pseudo-labels. For the parameters of these
layers, selected based on the norm of the gradients, the learning rates are
adjusted based on their sensitivity to domain shift. Extensive experiments are
conducted on benchmark datasets to demonstrate the efficacy of our method
against different baselines. The results reveal that PALM achieves superior
performance and pushes the state-of-the-art on several standard baselines.

## References

  * [1] Arnold, E., Al-Jarrah, O.Y., Dianati, M., Fallah, S., Oxtoby, D., Mouzakitis, A.: A survey on 3d object detection methods for autonomous driving applications. IEEE Transactions on Intelligent Transportation Systems 20(10), 3782–3795 (2019)
  * [2] Chatterji, N.S., Neyshabur, B., Sedghi, H.: The intriguing role of module criticality in the generalization of deep networks. arXiv preprint arXiv:1912.00528 (2019)
  * [3] Chen, D., Wang, D., Darrell, T., Ebrahimi, S.: Contrastive test-time adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 295–305 (2022)
  * [4] Choi, S., Yang, S., Choi, S., Yun, S.: Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes. In: European Conference on Computer Vision. pp. 440–458. Springer (2022)
  * [5] Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., Hein, M.: Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670 (2020)
  * [6] Döbler, M., Marsden, R.A., Yang, B.: Robust mean teacher for continual and gradual test-time adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7704–7714 (2023)
  * [7] Evci, U., Dumoulin, V., Larochelle, H., Mozer, M.C.: Head2toe: Utilizing intermediate representations for better transfer learning. In: International Conference on Machine Learning. pp. 6009–6033. PMLR (2022)
  * [8] Gong, T., Jeong, J., Kim, T., Kim, Y., Shin, J., Lee, S.J.: Robust continual test-time adaptation: Instance-aware bn and prediction-balanced memory. arXiv preprint arXiv:2208.05117 (2022)
  * [9] Guo, Y., Shi, H., Kumar, A., Grauman, K., Rosing, T., Feris, R.: Spottune: transfer learning through adaptive fine-tuning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4805–4814 (2019)
  * [10] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [11] Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019)
  * [12] Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
  * [13] Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
  * [14] Hsu, Y.C., Shen, Y., Jin, H., Kira, Z.: Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10951–10960 (2020)
  * [15] Huang, R., Geng, A., Li, Y.: On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems 34, 677–689 (2021)
  * [16] Jain, V., Learned-Miller, E.: Online domain adaptation of a pre-trained cascade of classifiers. In: CVPR 2011. pp. 577–584. IEEE (2011)
  * [17] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [18] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017)
  * [19] LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. Advances in neural information processing systems 2 (1989)
  * [20] Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems 31 (2018)
  * [21] Lee, Y., Chen, A.S., Tajwar, F., Kumar, A., Yao, H., Liang, P., Finn, C.: Surgical fine-tuning improves adaptation to distribution shifts. arXiv preprint arXiv:2210.11466 (2022)
  * [22] Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)
  * [23] Liang, C., Jiang, H., Zuo, S., He, P., Liu, X., Gao, J., Chen, W., Zhao, T.: No parameters left behind: Sensitivity guided adaptive learning rate for training large transformer models. arXiv preprint arXiv:2202.02664 (2022)
  * [24] Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690 (2017)
  * [25] Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. Advances in neural information processing systems 33, 21464–21475 (2020)
  * [26] Liu, Y., Kothari, P., Van Delft, B., Bellot-Gurlet, B., Mordan, T., Alahi, A.: Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems 34, 21808–21820 (2021)
  * [27] Long, M., Zhu, H., Wang, J., Jordan, M.I.: Unsupervised domain adaptation with residual transfer networks. Advances in neural information processing systems 29 (2016)
  * [28] Luo, J.H., Wu, J., Lin, W.: Thinet: A filter level pruning method for deep neural network compression. In: Proceedings of the IEEE international conference on computer vision. pp. 5058–5066 (2017)
  * [29] Molchanov, P., Mallya, A., Tyree, S., Frosio, I., Kautz, J.: Importance estimation for neural network pruning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11264–11272 (2019)
  * [30] Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J.: Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016)
  * [31] Mozer, M.C., Smolensky, P.: Skeletonization: A technique for trimming the fat from a network via relevance assessment. Advances in neural information processing systems 1 (1988)
  * [32] Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P., Tan, M.: Efficient test-time model adaptation without forgetting. In: International conference on machine learning. pp. 16888–16905. PMLR (2022)
  * [33] Park, J., Kim, J., Kwon, H., Yoon, I., Sohn, K.: Layer-wise auto-weighting for non-stationary test-time adaptation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1414–1423 (2024)
  * [34] Rasmussen, C., Ghahramani, Z.: Occam’s razor. Advances in neural information processing systems 13 (2000)
  * [35] Ro, Y., Choi, J.Y.: Autolr: Layer-wise pruning and auto-tuning of learning rates in fine-tuning of deep networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 2486–2494 (2021)
  * [36] Sagun, L., Evci, U., Guney, V.U., Dauphin, Y., Bottou, L.: Empirical analysis of the hessian of over-parametrized neural networks. arXiv preprint arXiv:1706.04454 (2017)
  * [37] Schneider, S., Rusak, E., Eck, L., Bringmann, O., Brendel, W., Bethge, M.: Improving robustness against common corruptions by covariate shift adaptation. Advances in neural information processing systems 33, 11539–11551 (2020)
  * [38] Schwarz, J., Czarnecki, W., Luketina, J., Grabska-Barwinska, A., Teh, Y.W., Pascanu, R., Hadsell, R.: Progress & compress: A scalable framework for continual learning. In: International conference on machine learning. pp. 4528–4537. PMLR (2018)
  * [39] Sener, O., Song, H.O., Saxena, A., Savarese, S.: Learning transferrable representations for unsupervised domain adaptation. Advances in neural information processing systems 29 (2016)
  * [40] Shannon, C.E.: A mathematical theory of communication. The Bell system technical journal 27(3), 379–423 (1948)
  * [41] Song, J., Lee, J., Kweon, I.S., Choi, S.: Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11920–11929 (2023)
  * [42] Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., Hardt, M.: Test-time training with self-supervision for generalization under distribution shifts. In: International conference on machine learning. pp. 9229–9248. PMLR (2020)
  * [43] Theis, L., Korshunova, I., Tejani, A., Huszár, F.: Faster gaze prediction with dense networks and fisher pruning. arXiv preprint arXiv:1801.05787 (2018)
  * [44] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 (2020)
  * [45] Wang, Q., Fink, O., Van Gool, L., Dai, D.: Continual test-time domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7201–7211 (2022)
  * [46] Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1492–1500 (2017)
  * [47] Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? Advances in neural information processing systems 27 (2014)
  * [48] Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016)
  * [49] Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13. pp. 818–833. Springer (2014)
  * [50] Zhang, C., Bengio, S., Singer, Y.: Are all layers created equal? The Journal of Machine Learning Research 23(1), 2930–2957 (2022)
  * [51] Zintgraf, L., Shiarli, K., Kurin, V., Hofmann, K., Whiteson, S.: Fast context adaptation via meta-learning. In: International Conference on Machine Learning. pp. 7693–7702. PMLR (2019)

##  7 Supplementary Material

In this work, we propose a continual test-time adaptation method that
adaptively changes the learning rates of selected parameters of the model.
This involves two core steps: 1) automatically selecting a subset of layers,
for further adaptation, based on the model’s prediction uncertainty. 2) as an
indicator of domain shift, we utilize the sensitivity of the parameters of the
selected layers, to the loss landscape, to weigh the learning rate importance.
In this supplement to the main paper, we offer further elaboration and
experimental findings on the following:

  * •

We critically analyze the effectiveness of different vector norms on the
gradients computed by backpropagating the KL divergence between the predicted
probabilities and a uniform distribution in Sec. 7.1.

  * •

We present an ablation study, in Sec. 7.2, on the threshold that upper bounds
the scoring function of gradients.

  * •

We study the different formulations of the learning rate "importance" in Sec.
7.3.

  * •

We evaluate our proposed approach in a more realistic and extremely
challenging setting _i.e_. Mixed-Domain Test-Time Adaptation (MDTTA) in Sec.
7.4.

  * •

We draw a contrast between our work and AutoLR [35] in Sec. 7.5, an adaptive
learning rate method for efficient vision model fine-tuning on downstream
tasks.

  * •

We validate our method across various pre-trained source models in Sec. 7.6.

###  7.1 Effectiveness of p-norms to Compute the Scoring Function
($\mathcal{Z}_{\theta_{n}}$)

Table 7: Mean errors (in %) on CIFAR-10C, CIFAR-100C, and ImageNet-C - We
report the CTTA mean errors for different p-norms that are used for the
scoring function.

p | 0 | 0.5 | 1 | 2 | 3 | 4 |  | 5
---
$\infty$
CIFAR-10C | 20.35 | 16.32 | 15.54 | 86.61 | 83.42 | 82.59 | 82.08 | 81.83
CIFAR-100C | 35.5 | 30.41 | 30.19 | 69.57 | 69.53 | 70.32 | 69.88 | 74.48
ImageNet-C | 68.61 | 60.23 | 60.12 | 97.06 | 96.90 | 96.85 | 96.84 | 96.90

In our proposed approach, we decide upon the usage of the $L_{1}$ vector norm
to bound the gradients backpropagated from the KL divergence between a uniform
distribution u = [$\frac{1}{C}$, $\frac{1}{C}$, …., $\frac{1}{C}$] $\in$
$\mathbb{R}^{C}$, where C is the total number of classes, and the model’s
softmax outputs. This helped in selecting the model layers that would require
further adaptation since smaller norm values translate to the fact that the
softmax outputs are closer to u. In this experiment, we justify the usage of
the $L_{1}$ norm for such a computation. To be specific, we compute a variety
of p-norms _i.e_. p $\in$ {0, 0.5, 1, 2, 3, 4, 5, $\infty$}. Table 7 reports
the mean classification errors on the benchmark datasets CIFAR-10C,
CIFAR-100C, and ImageNet-C, upon varying the scoring function. Mathematically,
for a vector $\vec{r}$, the p-norm is defined as:

$||\vec{r}||_{p}=(\sum_{i}|r_{i}|^{p})^{\frac{1}{p}}$ (17)

When p>1, the emphasis is on the larger components of $\vec{r}$ leading to
even sparser representations. In the gradient space, the scoring functions
with large values of p do not get a good representation of the divergence.
This can be seen from the reported results with the errors getting amplified
for all the datasets. For p<1, even though we do not observe large errors,
they do not outperform p = 1 because of the emphasis on smaller components.
Interestingly, with p$\rightarrow$1, the error gap begins to close.

|  |
---|---|---
a) CIFAR-10C | b) CIFAR-100C | c) ImageNet-C
Figure 4: Classification mean errors for different values of $\eta$. We select
the subset of layers with the scoring function $\mathcal{Z}_{\theta_{n}}$
$\leq$ $\eta$ while freezing the rest by not updating them.

###  7.2 Ablation on $\eta$

In the previous subsection, we discussed the analysis and motivation of using
an $L_{1}$ norm on the computed gradients to select the subset of layers that
would require further adaptation, an automatic alternative to surgical fine-
tuning [21]. In Fig. 4 we illustrate the effects of $\mathcal{Z}_{\theta_{n}}$
$\leq$ $\eta$ on the mean errors of CIFAR-10C, CIFAR-100C, and ImageNet-C. In
the main paper, we argue that larger values of temperature T are required to
capture the prediction uncertainty since the output distribution would largely
tend towards u. Since the temperature plays a large role in smoothing the
logits, we report $\eta$ $\in$ {0, 0.05, 0.1, 0.3, 0.5, 0.7, 1.0, 1.5, 2, 3,
4, 5}. From the figure, it is apparent that smaller values of $\eta$, with
their corresponding temperature are favored, leading to not updating the
majority of parameters of the subset of layers that were chosen. This reduces
the computational cost while simultaneously enhancing classification
performance and adaptation through our approach.

###  7.3 Understanding the Formulation of the Learning Rate Importance

The scale or "importance" of the learning rates of parameters in selected
layers, is formulated as:

$i_{n}^{t}=\hat{D}_{n}^{t}/\hat{S}_{n}^{t}$ (18)

where $\hat{D}_{n}^{t}$ indicates how uncertain $\hat{S}_{n}^{t}$ is to
previous batches of different domains. We increase the learning rate of less
sensitive and more uncertain parameters. In this ablation, we focus on the
different alternatives that Eq. 11 can take. In Table 8 we outline the
different alternatives.

Table 8: Different alternatives of Eq. 11 to compute the learning rate
importance for our work.

Alternative | Description
---|---
1\. $\hat{D}_{n}^{t}$/$\hat{S}_{n}^{t}$ | Proposed.
2\. $\hat{D}_{n}^{t}$*$\hat{S}_{n}^{t}$ | Increase the learning rates for highly sensitive/uncertain parameters.
3\. $\hat{S}_{n}^{t}$ | Exclude the influence of the uncertainty of $\hat{S}_{n}^{t}$ _i.e_. $\hat{D}_{n}^{t}$.
4\. 1/$\hat{S}_{n}^{t}$ | Study the influence of excluding $\hat{D}_{n}^{t}$.
5\. $\hat{D}_{n}^{t}$ | Study the influence of excluding $\hat{S}_{n}^{t}$.
6\. $\hat{S}_{n}^{t}$/$\hat{D}_{n}^{t}$ | Lower the learning rates of highly uncertain sensitive parameters.

Fig. 5 illustrates the mean errors of the datasets using the different
alternatives of Eq. 11. In alternatives 2, 3, and 6, the focus is on
increasing the learning rates for highly sensitive parameters _i.e_. In 4 and
5, we seek to analyze performance while specifically excluding the influence
of each factor one at a time. As can be seen, for alternatives 2, 3, and 5, we
see a small spike in mean error over the best (alternative 1). For alternative
6, the uncertainty captured by $\hat{D}_{n}^{t}$ decreases the learning rate,
leading to a sharp rise in errors. A similar conclusion can be drawn about
alternative 4 _i.e_. the learning rates are elevated for parameters with low
sensitivity. Hence, modeling the uncertainty and sensitivity of parameters is
crucial with both complementing each other, as in our proposed scheme.

|  |
---|---|---
a) CIFAR-10C | b) CIFAR-100C | c) ImageNet-C
Figure 5: Illustration of mean errors on CIFAR-10C, CIFAR-100C, and ImageNet-C
based on the alternative formulations of the learning rate "importance" (Eq.
11). Table 9: We report the Mixed-Domain TTA (MDTTA) mean errors (in %) with a
test batch of corruption severity 5.

Dataset | LAW | Ours
---|---|---
CIFAR-10C | 49.94 | 40.96
CIFAR-100C | 43.01 | 36.15
ImageNet-C | 91.01 | 91.38

###  7.4 Mixed-Domain Test-Time Adaptation

In the main paper, we perform and report experimental results on two test-time
adaptation settings - continual and gradual. However, to challenge current
approaches to a larger extent, we perform adaptation in a very "realistic"
setting where consecutive batches of data could originate from different
distributions. We call this as Mixed-Domain Test-Time Adaptation (MDTTA). In
standard CTTA, with the idea of tasks and distributions, we adapt the model to
a series of tasks where each task has batches from a single distribution.
There is no awareness of when a new task arrives. However, in MDTTA, with no
concept of a task, consecutive batches most likely have different image
corruptions/distributions. Such a realistic TTA setting should be critically
studied since in real-world scenarios, the model is never aware of how quickly
the distribution changes.

Table 9 reports the MDTTA results using LAW [33] and our method on CIFAR-10C,
CIFAR-100C, and ImageNet-C. Adaptive learning rate TTA methods like LAW [33]
claim to capture the domain shift of a batch by computing the Fisher
Information Matrix (FIM) [36], based on the pseudo-labels, to decide on the
layer importance. As seen from the results, such an approach is not very
useful for MDTTA. Along similar lines, our approach, which automatically
decides on the required subset of layers for further adaptation based on the
model’s current prediction uncertainty and computes their parameter
sensitivity, achieves staggering improvements of 8.98% and 6.86% over LAW [33]
for CIFAR-10C and CIFAR-100C, respectively. On ImageNet-C, we achieve
comparable performances.

Table 10: Fine-grained classification errors (in %) on CIFAR-10C and
CIFAR-100C by AutoLR and ours.

| CIFAR-10C
---|---
Method | Gaussian | Shot | Impulse | Defocus | Glass | Motion | Zoom | Snow | Frost | Fog | Brightness | Contrast |  | Elastic
---
Transform
Pixelate | JPEG | Mean
| AutoLR [35]
---
26.98 | 23.9 | 33.54 | 13.23 | 33.26 | 15.84 | 13.58 | 18.59 | 17.29 | 16.64 | 11.51 | 16.08 | 26.37 | 23.02 | 31.72 | 21.44
| Ours
---
25.89 | 18.1 | 22.71 | 12.38 | 25.3 | 13.18 | 10.79 | 13.5 | 13.15 | 12.21 | 8.53 | 11.85 | 17.98 | 12.02 | 15.47 | 15.54
| CIFAR-100C
| AutoLR [35]
---
39.51 | 35.11 | 37.09 | 27.55 | 37.91 | 30.62 | 28.74 | 34.61 | 34.50 | 40.76 | 30.51 | 35.47 | 36.79 | 34.09 | 41.31 | 34.97
| Ours
---
37.33 | 32.5 | 34.94 | 26.20 | 35.28 | 27.54 | 24.64 | 28.85 | 29.15 | 34.14 | 23.52 | 26.99 | 31.15 | 26.56 | 34.1 | 30.19

###  7.5 Comparison with AutoLR [35]

Ro _et al_. proposed AutoLR [35], an adaptive learning rate method, for
efficient fine-tuning, based on the weight variations between layers of a
model. With the intended increment in weight change across layers, the authors
propose a target weight change based on manually chosen hyper-parameters that
decide the range of weight change. With the desired weight variation being
very sensitive to the hyper-parameters, the learning rates change very little.
In this subsection, we contrast our work against AutoLR [35] by adapting their
work to the CTTA setting. We optimize our model using SGD with a learning rate
of 1e-3 and a momentum factor of 0.9. The final optimization objective is the
same as ours _i.e_. entropy minimization and consistency loss.

In Table 10, we report results for CIFAR-10C and CIFAR-100C to demonstrate the
effectiveness of our work. We observe improvements of 5.9% and 4.78% over
AutoLR [35] for CIFAR-10C and CIFAR-100C respectively. For CIFAR-100C, an
interesting observation is that for the initial tasks, the error gap between
AutoLR [35] and our approach is not too much. However, towards the later
stages, the error gap rises. A plausible explanation for this is that AutoLR
[35] does not consider or utilize the distributional shift between tasks to
update the learning rates of layers. In our work, due to the computation of
the parameter sensitivity, we approximate the amount of adaptation that is
required by the batch and update its learning rate accordingly.

Table 11: Classification mean error (in %) on CIFAR-10C using different source
models.

Method | WideResNet-28 [48] | ResNeXt-29 [46] | WideResNet-40 [5] | ResNet-50 [26]
---|---|---|---|---
Source | 43.5 | 18.0 | 18.3 | 48.8
BN Stats Adapt [37] | 20.4 | 13.3 | 14.6 | 16.1
TENT Continual [44] | 20.7 | 14.8 | 12.5 | 14.8
CoTTA [45] | 16.2 | 11.0 | 12.7 | 13.1
Adacontrast [3] | 18.5 | 11.0 | 11.9 | 14.5
LAW [33] | 17.21 | 10.64 | 11.79 | 12.62
Ours | 15.54 | 9.05 | 11.03 | 11.42

Table 12: Classification mean error (in %) on CIFAR-100C using different
source models.

Method | ResNeXt-29 [46] | WideResNet-40 [5] | ResNet-50 [26]
---|---|---|---
Source | 46.4 | 46.8 | 73.8
BN Stats Adapt [37] | 35.4 | 39.3 | 43.7
TENT Continual [44] | 60.9 | 36.9 | 44.2
CoTTA [45] | 32.5 | 38.2 | 37.6
Adacontrast [3] | 33.4 | 37.1 | 41.3
LAW [33] | 31.75 | 35.82 | 39.3
Ours | 30.19 | 33.93 | 37.78

Table 13: Classification mean error (in %) on ImageNet-C using different
source models.

Method | ResNet-50 [10] | ResNeXt-50 [46] | WideResNet-50 [48]
---|---|---|---
Source | 82.0 | 78.9 | 78.9
BN Stats Adapt [37] | 68.6 | 67.1 | 66.2
TENT Continual [44] | 62.6 | 58.7 | 57.7
CoTTA [45] | 62.7 | 59.8 | 57.9
Adacontrast [3] | 65.5 | 63.1 | 63.3
LAW [33] | 61.12 | 57.7 | 56.25
Ours | 60.12 | 56.88 | 54.66

###  7.6 CTTA Performance with Other Models

Following LAW [33], we carry out additional experiments to confirm the
adaptable effectiveness of our approach. To be fair and consistent with
respect to the source models, in addition to WideResNet-28 [48], we report
results on ResNeXt-29 [46], WideResNet-40 from [5], and ResNet-50 from [26]
for CIFAR-10C and CIFAR-100C benchmark datasets. For ImageNet-C, we use
ResNeXt-50 [46], WideResNet-50 [48], and ResNet-50 [10]. For each, we document
the mean classification error over 5 rounds. The goal of these experiments is
to eliminate any bias regarding the selection of models discussed in the main
paper. Tables 11, 13, and 13 report our results against Source, BN Stats Adapt
[37], TENT Continual [44], CoTTA [45], Adacontrast [3], and LAW [33] with
their respective models.

In Table 11, we report the mean classification errors on the CIFAR-10C dataset
for different baselines using WideResNet-28 [48], ResNeXt-29 [46],
WideResNet-40 from [5], and ResNet-50 from [26]. Our method clearly
outperforms all the baselines using the different source models. In
particular, we obtain performance improvements of 1.67%, 1.59%, 0.76%, and
1.2% for the respective models, over LAW [33].

Similarly, Tables 13 and 13 report the mean errors on CIFAR-100C and
ImageNet-C. For CIFAR-100C, we observe that TENT Continual [44] does not scale
well with change in models _i.e_. the error increases by 24% and 16.7& for
ResNeXt-29 over WideResNet-40 [5] and ResNet-50 [26]. The performance of CoTTA
[45] with ResNet-50 [10] demonstrates remarkable comparability to the outcomes
achieved by our approach. Adaptive learning rate methods, especially ours,
establish a new baseline for CTTA performance across all models with mean
errors. On the ImageNet-C dataset, we observe comparable performances between
TENT Continual [44] and CoTTA [45]. Across all the respective models on
ImageNet-C, our work achieves mean errors of 60.12%, 56.88%, and 54.66%
respectively.

Overall, our method demonstrates robust scalability across diverse models and
consistently outperforms various baselines. Our results unequivocally
establish the superiority of our approach, showcasing its dominance regardless
of the pre-trained parameters of the source model.