# Erasing Self-Supervised Learning Backdoor by Cluster Activation Masking

Shengsheng Qian1,2 Yifei Wang3 Dizhan Xue1,2 Shengjie Zhang3 Huaiwen Zhang4
Changsheng Xu1,2,5
1MAIS, Institute of Automation, Chinese Academy of Sciences
2University of Chinese Academy of Sciences
3Zhengzhou University
4College of Computer Science, Inner Mongolia University
5Peng Cheng Laboratory
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Researchers have recently found that Self-Supervised Learning (SSL) is
vulnerable to backdoor attacks. The attacker can embed hidden SSL backdoors
via a few poisoned examples in the training dataset and maliciously manipulate
the behavior of downstream models. To defend against SSL backdoor attacks, a
feasible route is to detect and remove the poisonous samples in the training
set. However, the existing SSL backdoor defense method fails to detect the
poisonous samples precisely. In this paper, we propose to erase the SSL
backdoor by cluster activation masking and propose a novel PoisonCAM method.
After obtaining the threat model trained on the poisoned dataset, our method
can precisely detect poisonous samples based on the assumption that masking
the backdoor trigger can effectively change the activation of a downstream
clustering model. In experiments, our PoisonCAM achieves 96% accuracy for
backdoor trigger detection compared to 3% of the state-of-the-art method on
poisoned ImageNet-100. Moreover, our proposed PoisonCAM significantly improves
the performance of the trained SSL model under backdoor attacks compared to
the state-of-the-art method. Our code, poisoned datasets, and trained models
will be open once this paper is accepted.

Figure 1: Retrieved backdoor trigger patches from the poinsoned ImageNet-100
(poison rate 0.5%, target category “rottweiler”) by PatchSearch (left) [49]
and our PoisonCAM (right).

##  1 Introduction

In recent years, Self-Supervised Learning (SSL) [20, 33, 44, 26, 30] has
emerged as a powerful paradigm in machine learning, enabling models to learn
from vast amounts of unlabeled data. Without the reliance on manual feature
engineering or human annotations, SSL is able to learn meaningful
representations from unlabeled data and facilitate a range of downstream
tasks, such as clustering and classification [17, 7, 8, 15, 5]. However,
recent work [2, 43] has found that SSL is vulnerable to backdoor attacks where
an attacker can inject a stealthy backdoor trigger into SSL models by
poisoning a small number of training samples.

SSL backdoor attacks [2] pose a significant challenge to the security and
robustness of SSL models, of which the procedure can be summarized as follows:
First, an attacker selects a stealthy backdoor trigger for a specific target
category. Next, the attacker injects the trigger patch into some data of the
target category. After training on the poisoned dataset, the SSL model will
construct a strong correlation between the trigger and the target category.
Finally, the attacker can manipulate the behavior of the downstream model by
attaching the trigger to the input, such as forcing the downstream classifier
to misclassify the image as the target category. Meanwhile, the attacked model
can behave similarly to unattacked models when the trigger is absent in the
input, making the injected backdoor stealthy. Facing the threat of SSL
backdoor attacks, this work aims to erase the SSL backdoor and train a
trustworthy SSL model. For practicability, we assume that the defender has no
prior knowledge of the trigger or target category, and lacks access to trusted
data, following [36, 49, 34, 29, 38].

To defend against SSL backdoor attacks, a feasible and straightforward route
is to detect and remove the poisonous samples in the training set. However,
since semantic annotations are unavailable in SSL, detecting the backdoor
trigger is not a trivial problem and should be achieved in a totally
unsupervised manner. The existing method [49] (i.e., PatchSearch) utilizes
Grad-CAM [45] on a downstream clustering model to retrieve trigger patches
injected into the dataset, based on which a poison classifier is trained to
classify poisonous or clean data. However, as shown in Figure 1, the accuracy
of top 20, top 50, and top 100 trigger patches retrieved by PatchSearch are
only 10%, 6%, and 3% on the poisoned ImageNet-100. Subsequently, PatchSearch
is limited in distinguishing between trigger patches and benign patches,
reducing the effectiveness of the finally trained SSL models. Therefore, this
work focuses on addressing a major challenge in SSL backdoor defense: How to
accurately retrieve the SSL backdoor trigger patches injected in a poisoned
and unlabeled dataset?

In this paper, we propose PoisonCAM, which aims at accurately detecting
poisonous samples in a poisoned and unlabeled dataset to erase the SSL
backdoor. To retrieve the trigger patches injected into the poisoned dataset,
we propose a novel Cluster Activation Masking method. Our method is based on
the assumption that masking a successful trigger in an image will change the
cluster assignment from the target category of the trigger to the true
category of the image. Therefore, the trigger patches can be detected by
comparing the cluster activation under a few random masks. As shown in Figure
1, our PoisonCAM significantly improves the top 20, 50, and top 100 accuracy
of the retrieved trigger patches from 10%, 6%, and 3% of the state-of-the-art
PatchSearch to 95%, 96%, and 96% on the poisoned ImageNet-100. Based on the
more accurate trigger patches, our trained poison classifier also
significantly improves the precision of detecting poisonous samples from 5.4%
to 49.3% while achieving even higher recall (see details in Table 2). Finally,
after accurately detecting and removing the poisonous samples, the trained SSL
model on the clean-up dataset achieves new state-of-the-art results in terms
of both the performance and the backdoor defense.

In brief, the contributions of this work are summarized as follows:

  * •

We propose PoisonCAM, a novel SSL backdoor defender model, to accurately
detect and remove poisonous data from a poisoned and unlabeled dataset.
Subsequently, an effective and backdoor-free SSL model can be trained on the
clean-up dataset.

  * •

We propose a Cluster Activation Masking method to accurately retrieve trigger
patches injected into the poisoned dataset. Based on the retrieved trigger
patches, an effective poison classifier is trained to classify poisonous or
clean data in the training set.

  * •

Extensive experimental results on ImageNet-100 and STL-10 demonstrate that our
proposed PoisonCAM significantly outperforms the state-of-the-art method for
defending against SSL backdoor attacks.

Figure 2: The overview of PoisonCAM: (1) Learn a clustering model on the
poisoned dataset by $k$-means algorithm; (2) Detect the candidate trigger in
each image based on clustering outlier scores of random masks and the weighted
sum of masks as the trigger attention map; (3) Compute the poison scores of
candidate triggers and retrieve the top-k triggers with corresponding
poisonous images; (4) Train a poison classifier to identify and remove
poisonous samples in the poisoned dataset.

##  2 Related Work

Self-Supervised Learning. The objective of self-supervised learning (SSL) [12,
14, 35, 52, 57] is to acquire representations from uncurated and unlabeled
data through a pretext task that is derived from the data itself, without any
human annotations. MoCo [17, 7, 8] is a widely employed contrastive SSL
algorithm that involves classifying two augmented versions of the same image
as positive pairs, which are then contrasted to negative pairs consisting of
augmentations from different images. Contrastive SSL algorithms are further
explored in [5, 3, 56, 10, 46]. BYOL [15] is a non-contrastive SSL algorithm
that predicts the target network representation of the same image under a
different augmented view without negatives. Non-contrastive SSL algorithms are
further developed in [24, 54, 25, 6, 23]. Despite the remarkable potential,
SSL algorithms are not immune to vulnerabilities. In this paper, we study the
defense against backdoor attacks on SSL to promote trustworthy and reliable
SSL models.

SSL Backdoor Attacks. The purpose of SSL backdoor attacks is to inject a
stealthy backdoor trigger into SSL models by poisoning training data, which
can be activated to manipulate the behavior of downstream models at test time
[21, 48]. Saha et al. [43] propose backdoor attacks towards SSL models by
attaching a trigger patch to images of a target category. At test time, the
downstream classifier has high accuracy on clean images but misclassifies
images with the trigger as the target category. The authors also propose a
distillation-based defender, which requires a substantial amount of trusted
data and may be infeasible in real-world scenarios. Li et al. [27] recently
propose a similar SSL backdoor attack method. The key difference is that they
adopt a frequency domain based spectral trigger to resist the data
augmentations. Carlini and Terzis [2] propose backdoor attacks towards CLIP
[39], a multimodal contrastive SSL model. Their attack is implemented by
injecting triggers into the victim images and tampering with the paired
textual captions.

SSL Backdoor Defenses. Despite the prosperity of backdoor defense for
supervised learning [19, 60, 58, 32, 53], SSL backdoor defense is a relatively
under-explored and more challenging problem. Tejankar et al. [49] explore to
defend against patch-based SSL backdoor attacks [43]. Their method adopts
Grad-CAM [45] on a clustering model to detect candidate triggers and train a
poison classifier to identify and delete poisonous samples in the training
data. However, they compromisingly delete a large amount of clean samples due
to the low accuracy of their poison classifier. Bansal et al. [1] propose to
defend the multimodal SSL backdoor attacks on CLIP. They find that simply
integrating an intra-modal contrastive loss can effectively mitigate
multimodal SSL backdoor attacks. In this paper, we follow the research line of
patch-based SSL backdoor attacks on visual SSL models [43, 49] due to their
substantial feasibility and destructiveness. We propose a novel PoisonCAM
method based on Cluster Activation Masking to detect and delete the poisonous
samples in the training set while deleting as few clean samples as possible.

##  3 Threat Model

We introduce the threat model under the SSL backdoor attack proposed in [43].
The primary objective of the attacker is to manipulate the output of a
downstream model which is constructed based on the SSL model. In this paper,
we take a downstream image classifier as the example, following previous work
[43, 49]. Assume that an unlabeled poisoned dataset
$\boldsymbol{X}=\\{x_{i}\in\mathbb{R}^{C\times H\times W}\\}_{i=1}^{N}$
contains $N$ images where ${x}_{i}$ is the $i$-th image, $C$ denotes the
number of channels, $H$ and $W$ denote the height and the width of an image,
respectively. The attacker’s purpose is two-fold. Firstly, the attacker aims
to surreptitiously implant a backdoor into an SSL model, enabling the
downstream classifier to misclassify an incoming image as the target category
if the image contains the attacker-designated trigger $t$. Secondly, the
attacker seeks to hide the backdoor’s presence by ensuring that the
performance of the downstream classifier is similar to that of a classifier
based on an unattacked SSL model when the trigger is absent. The attacker’s
objectives can be achieved through a technique known as “data poisoning”. This
involves the attachment of a small trigger patch to a few images of the
preselected target category. The attacker posits that the SSL algorithm
associates the trigger with the target category, resulting in a successful
attack. Then, the model $f(\cdot)$ trained on the poisoned dataset
$\boldsymbol{X}$ via SSL algorithms (e.g., MoCo [17, 7, 8]) is defined as the
threat model. Moreover, the learned feature $\boldsymbol{h}_{i}=f(x_{i})$ can
be utilized in the downstream tasks.

##  4 Proposed Defender Model

The objective of the defender model is to remove the SSL backdoor, eliminating
the hidden correlation between the attacker-designed trigger and the target
category. Simultaneously, the defender should avoid impairing the model’s
performance on clean data. To enhance the practicability, we assume that the
defender should achieve this without any prior knowledge of the trigger or
target category, and lacking access to trusted data, following [36, 49, 34,
29, 38].

In this section, we propose our defender model named PoisonCAM, which aims at
identifying the poisonous samples in the training set and filtering them out
to form a clean-up training dataset
$\bar{\boldsymbol{X}}\subset\boldsymbol{X}$. The overall architecture of our
method is shown in Figure 2, which mainly consists of four steps: (1) Learning
clustering model for the poisoned dataset ${\boldsymbol{X}}$; (2) Detect the
candidate triggers for all images in ${\boldsymbol{X}}$; (3) Search for the
top-$k$ poisonous samples with the highest poison scores; (4) Train a poison
classifier to find and delete all poisonous samples in ${\boldsymbol{X}}$.
Subsequently, the clean-up training dataset $\bar{\boldsymbol{X}}$ can be
formed and a backdoor-free SSL model can be trained on $\bar{\boldsymbol{X}}$.

###  4.1 Clustering Model Learning

Due to the lack of labels in SSL, we first learn a clustering model $C(\cdot)$
for features $\\{f(x_{i})\\}_{i=1}^{N}$ by $k$-means algorithm [31] to capture
the semantics of training data, as follows:

$y_{i}=C(f(x_{i})),$ (1)

where $y_{i}\in\\{1,...,l\\}$ represents the corresponding cluster label and
$l$ is a hyper-parameter. Since the threat model is sensitive to triggers, the
poisonous images with triggers will tend to be classified into a cluster of
triggers. The clustering model $C(\cdot)$ is fixed and will be utilized in the
following.

###  4.2 Candidate Trigger Detection for Images

(a) 0-1 Interval

(b) Random

(c) Full Coverage

Figure 3: An example of three masking strategies.

In candidate trigger detection, we aim to detect the candidate trigger region
in every image $x_{i}$. The state-of-the-art PatchSearch [49] employs Grad-CAM
[45] to detect the pivotal region of an image $x_{i}$ for clustering as the
candidate trigger region. However, previous work [4, 22, 51] has found that
Grad-CAM may fail to locate the pivotal region for the downstream tasks.
Moreover, as we empirically show the detection results on poisoned
ImageNet-100 in Figure 1 (a), Grad-CAM suffers from detecting the trigger in
this poisoned SSL dataset. Therefore, we propose a novel Cluster Activation
Masking method based on the assumption that masking a trigger $t$ in an image
$x_{i}$ will change the cluster assignment $y_{i}$ from the cluster of
triggers to the true cluster of $x_{i}$. By analyzing the cluster activation
under $B$ different random masks, we can locate the candidate trigger $t_{i}$
in $x_{i}$.

Specifically, using the obtained clustering model $C(\cdot)$ in Equation 1, we
aim at locating the region of the potential trigger in $x_{i}$. For any image
$x_{i}\in{\boldsymbol{X}}$, we randomly initialize $B$ masks
$\boldsymbol{M}=\\{\boldsymbol{m}_{j}\in\\{0,1\\}^{H\times W}\\}_{j=1}^{B}$.
In particular, we set the pixel value to zero if the corresponding mask value
is 1 and keep the pixel value unchanged if the corresponding mask value is 0.
After randomly selecting a masking window of the size of $[w,w]$, we have
empirically tested three masking strategies, as shown in Figure 3. Although it
seems that 0-1 Interval Masking and Random Masking can break the pattern of
the trigger while protecting the semantics of benign regions, we empirically
found in experiments (see Section 5.7) that Full Coverage Masking is the
optimal strategy. We apply masks $\\{\boldsymbol{m}_{j}\\}_{j=1}^{B}$ to
obtain $B$ degraded images
$\boldsymbol{D}=\\{x_{i}^{j}=x_{i}\odot(1-\boldsymbol{m}_{j})\\}_{j=1}^{B}$,
where $\odot$ denotes the element-wise multiplication. Then we classify the
features of $\boldsymbol{D}$ by the clustering model $C(\cdot)$ to get cluster
labels $\\{\eta_{j}=C(f(x_{i}^{j}))\\}_{j=1}^{B}$ and the distances to the
assigned cluster centers $\\{d_{j}\\}_{j=1}^{B}$. Since the size of the
backdoor trigger is much smaller than the whole image, most masks cannot
change the clustering assignment of $x_{i}^{j}$. Therefore, most samples from
$\boldsymbol{D}$ are assigned to the same cluster and we denote this cluster
as $\eta^{*}$. To scale cluster distances, we utilize Max-Min Scaling to
normalize $d_{j}$ as
$d^{n}_{j}=\frac{d_{j}-\min_{k}d_{k}}{\max_{k}d_{k}-\min_{k}d_{k}}\in[0,1]$.
Then, we compute the clustering outlier score $a_{j}$ of the mask
$\boldsymbol{m}_{j}$ as follows:

$\displaystyle a_{j}=\begin{cases}d_{j}^{n}&{\eta}_{j}=\eta^{*}\\\
1&{\eta}_{j}\neq\eta^{*}.\end{cases}$ (2)

For a poisonous sample $x_{i}$, $\eta^{*}$ should be the target category of
trigger $t$. ${\eta}_{j}\neq\eta^{*}$ means the trigger is successfully
masked. When the trigger is not successfully masked, larger $d_{j}^{n}$ to the
cluster center $\eta^{*}$ means more region of the trigger is masked.
Therefore, $a_{j}$ can represent the importance score of successfully masking
the trigger. Based on the scores $\\{a_{i}\\}_{i=1}^{B}$, the masks are
aggregated through weighted summation to obtain an attention map of the
trigger, as follows:

$\displaystyle
A=\frac{\sum_{j=1}^{B}\boldsymbol{m}_{j}\times{a}_{j}}{\sum_{j=1}^{B}\boldsymbol{m}_{j}}\in\mathbb{R}^{H\times
W},$ (3)

where $A$ is the attention map of the trigger, and the value $A_{mn}$
positively correlates to the presence possibility of the trigger. After
obtaining the attention map of the image $x_{i}$, we select the window $t_{i}$
of size $w\times w$ with the largest sum attention value in the window.
Therefore, $t_{i}$ can be regarded as the optimal window of the potential
trigger $t$ in $x_{i}$ and we denote $t_{i}$ as the candidate trigger for
simplicity.

###  4.3 Poisonous Image Search

In this section, we aim to find a set of highly poisonous images in
$\boldsymbol{X}$. To quantify the poisonousness of a given image $x_{i}$, we
define the poison score of $x_{i}$. Our method is based on the assumption that
pasting a real trigger onto an image will strongly change its clustering
assignment while the effectiveness of pasting a benign region is much weaker.
Specifically, we first obtain a fixed test set $\boldsymbol{X}^{f}$ by
sampling a few images per cluster that are closest to their respective cluster
centers. Subsequently, we paste the candidate trigger $t_{i}$ onto all images
in $\boldsymbol{X}^{f}$ and get their new cluster assignments. Finally, the
poison score of $x_{i}$ is calculated as the number of images in
$\boldsymbol{X}^{f}$ whose cluster assignments are changed after pasting the
candidate trigger $t_{i}$.

To find a few highly poisonous images, we can compute the poison scores of all
images and simply take the top-$k$ image as the poisonous samples. However, to
improve the efficiency and utilize the clustering information of images, we
adopt a heuristic search strategy following [49] to iteratively compute poison
scores for a part of (not all) images in $\boldsymbol{X}$. Finally, we take
the top-$k$ scored images with the highest poison scores to form a poison set
$\boldsymbol{X}^{p}$. This poison set will be utilized to train a classifier
of poisonous samples.

###  4.4 Poison Classifier Training

Aiming at precisely detecting all poisonous samples in $\boldsymbol{X}$, we
train a ResNet [16] as the poison classifier. Specifically, for every
$x_{i}\in\boldsymbol{X}$, we randomly select a sample $x_{k}$ in the poison
set $\boldsymbol{X}^{p}$ and paste its candidate trigger $t_{k}$ at a random
location on $x_{i}$. These synthesized images as the positive samples and
original images in $\boldsymbol{X}$ as the negative samples form the poison
classification set $\tilde{\boldsymbol{X}}$. Since poisonous samples from
$\boldsymbol{X}$ are noisy for poison classification, we eliminate a
proportion $p$ of images with the highest poison scores in $\boldsymbol{X}$
and utilize strong augmentations and early stopping to alleviate the noisy
problem. Then, the poison classifier trained on $\tilde{\boldsymbol{X}}$ is
applied to $\boldsymbol{X}$, and all images classified as “poisonous” are
removed to form a clean-up training dataset $\bar{\boldsymbol{X}}$. Finally,
after erasing the SSL backdoor in the poisoned dataset, we can train a benign
SSL model on $\bar{\boldsymbol{X}}$, which can achieve similar performance to
the threat model $f(\cdot)$ on clean samples but will not be manipulated by
the attacker-designated trigger.

##  5 Experiments

We include implementation details and more experimental results in
Supplementary Material.

###  5.1 Datasets

Following previous work [49], we adopt ImageNet-100 [50], which contains
images belonging to 100 randomly sampled classes from the 1000 classes of
ImageNet [41]. The training set has about 127K samples and the validation set
has 5K samples. STL-10 [11] contains 500/800 training/validation images for
each of 10 classes.

###  5.2 Attack Setting

Following the SSL backdoor attacks proposed in [43], we randomly adopt 10
different target categories and trigger patches. In every experiment, we set a
single target category and use a single trigger patch. On ImageNet-100, we set
the poisoning rates as $0.5\%$ and $1.0\%$, which mean $50\%$ and $100\%$
images of the target category being poisoned to form a poisonous dataset. On
STL-10, we set the poisoning rates as $5.0\%$, which mean $50\%$ images of the
target category being poisoned. The HTBA triggers [42, 47] are pasted with
size $50\times 50$ onto the images and $25\%$ of the edges is reserved on four
sides of the images. After injecting the triggers, we train a threat model on
the poisoned dataset $\boldsymbol{X}$. Following PatchSearch [49], we adopt
ViT-B [13] as the backbone with MoCo-v3 algorithm [8] to train threat models
on poisoned datasets.

Table 1:  Defense results. We compared the performance of the attacked SSL models with different defense methods under various attack settings on the validation set. ACC denotes accuracy. ASR denotes attack success rate. Clean Val denotes the clean validation set and Poisoned Val denotes the poisoned validation set. Attack Settings | w/o Defense | PatchSearch | PoisonCAM
---|---|---|---
Dataset | Target Category | Clean Val | Poisoned Val | Clean Val | Poisoned Val | Clean Val | Poisoned Val
ACC$\uparrow$ |  ASR$\downarrow$ |  ACC$\uparrow$ |  ASR$\downarrow$ |  ACC$\uparrow$ |  ASR$\downarrow$ |  ACC$\uparrow$ |  ASR$\downarrow$ |  ACC$\uparrow$ |  ASR$\downarrow$ |  ACC$\uparrow$ |  ASR$\downarrow$
ImageNet-100 (poison rate 0.5%) | rottweiler | 69.4 | 0.5 | 27.7 | 63.9 | 68.0 | 0.5 | 61.9 | 0.4 | 70.6 | 0.5 | 65.2 | 0.5
tabby cat | 69.1 | 0.0 | 30.2 | 61.8 | 67.3 | 0.0 | 60.7 | 0.1 | 70.9 | 0.1 | 64.7 | 0.1
ambulance | 69.4 | 0.0 | 57.3 | 9.6 | 66.6 | 0.0 | 56.5 | 3.5 | 69.4 | 0.1 | 62.3 | 0.1
pickup truck | 70.1 | 0.3 | 58.5 | 9.3 | 65.6 | 0.4 | 60.2 | 0.3 | 71.0 | 0.3 | 65.4 | 0.3
laptop | 69.2 | 0.9 | 38.0 | 52.4 | 66.2 | 1.0 | 48.9 | 24.5 | 68.9 | 0.8 | 61.6 | 2.4
goose | 69.4 | 0.2 | 44.7 | 35.5 | 66.8 | 0.2 | 61.1 | 0.3 | 70.0 | 0.2 | 63.4 | 0.3
pirate ship | 69.5 | 0.0 | 52.5 | 22.2 | 66.0 | 0.1 | 51.8 | 15.1 | 68.5 | 0.1 | 61.5 | 0.2
gas mask | 68.6 | 0.3 | 33.4 | 58.8 | 69.4 | 1.1 | 63.1 | 2.1 | 68.8 | 0.9 | 63.3 | 2.1
vacuum cleaner | 69.1 | 1.1 | 44.0 | 32.2 | 67.8 | 1.2 | 61.0 | 1.1 | 70.0 | 1.4 | 62.7 | 1.2
American lobster | 68.8 | 0.1 | 44.0 | 42.5 | 65.2 | 0.3 | 59.6 | 0.3 | 69.2 | 0.2 | 62.4 | 0.2
average | 69.3 | 0.3 | 43.0 | 38.8 | 66.9 | 0.5 | 58.5 | 4.8 | 69.7 | 0.4 | 63.2 | 0.7
ImageNet-100 (poison rate 1.0%) | rottweiler | 69.4 | 0.4 | 26.2 | 70.9 | 67.1 | 0.5 | 60.9 | 0.5 | 69.3 | 0.7 | 63.5 | 0.6
tabby cat | 69.3 | 0.0 | 25.8 | 69.9 | 68.0 | 0.5 | 62.8 | 0.7 | 68.7 | 0.5 | 63.4 | 0.6
ambulance | 69.5 | 0.0 | 49.4 | 23.3 | 67.0 | 0.2 | 58.1 | 2.3 | 69.9 | 0.2 | 62.4 | 0.4
pickup truck | 69.2 | 0.3 | 52.6 | 22.4 | 67.6 | 0.4 | 61.8 | 0.4 | 69.0 | 0.4 | 63.2 | 0.5
laptop | 69.0 | 0.8 | 31.6 | 61.4 | 69.0 | 1.1 | 61.5 | 3.4 | 69.4 | 0.9 | 62.7 | 1.3
goose | 69.7 | 0.2 | 40.0 | 47.8 | 61.9 | 0.4 | 56.5 | 0.4 | 68.3 | 0.5 | 61.7 | 0.6
pirate ship | 69.0 | 0.1 | 49.1 | 30.8 | 68.8 | 0.5 | 60.9 | 4.0 | 68.9 | 0.4 | 61.8 | 0.8
gas mask | 68.7 | 0.3 | 29.2 | 65.4 | 68.2 | 1.3 | 60.4 | 4.3 | 69.1 | 1.3 | 62.5 | 2.3
vacuum cleaner | 68.9 | 1.0 | 39.2 | 44.5 | 69.0 | 1.3 | 61.0 | 2.1 | 69.9 | 1.4 | 63.3 | 1.4
American lobster | 69.0 | 0.1 | 27.2 | 68.1 | 67.9 | 0.7 | 61.7 | 1.9 | 69.9 | 0.6 | 62.5 | 2.1
average | 69.2 | 0.3 | 37.0 | 50.4 | 67.4 | 0.7 | 60.6 | 2.0 | 69.2 | 0.7 | 62.7 | 1.1
STL-10 (poison rate 5.0%) | bird | 63.2 | 5.1 | 50.8 | 28.7 | 62.6 | 3.1 | 56.2 | 7.1 | 65.4 | 4.9 | 61.0 | 5.1
car | 62.0 | 1.8 | 41.2 | 41.3 | 61.8 | 1.8 | 40.8 | 3.3 | 62.8 | 1.1 | 43.6 | 3.1
deer | 65.8 | 2.9 | 43.8 | 25.1 | 64.0 | 3.3 | 42.2 | 4.8 | 63.8 | 4.0 | 51.0 | 4.2
average | 63.7 | 3.3 | 45.3 | 31.7 | 62.8 | 2.7 | 46.4 | 5.1 | 64.0 | 3.3 | 51.9 | 4.1

###  5.3 Baseline Method and Evaluation Metrics

We adopt the state-of-the-art SSL defense method PatchSearch [49] and the
naive method without defense as the baseline methods. We fairly compare our
proposed PoisonCAM with baselines by sharing the weights of the threat model.
For the evaluation, we train a linear classifier on a trained SSL model by
randomly sampling a $1.0\%$ subset from the clean labeled training dataset,
following PatchSearch. Since PatchSearch has not released its trained models,
we reimplement PatchSearch using the official
code111https://github.com/UCDvision/PatchSearch. We will release our poisoned
datasets, code, and trained models once this paper is accepted.

We denote the original validation set of ImageNet-100 as the clean validation
set, which consists of 50 randomly selected images from each category, and
form a poisoned validation set by randomly pasting the attacked trigger on the
images of the clean validation set. Similarly, we construct the clean and
poisoned validation sets of STL-10. We then assess the results both on the
clean validation set and the poisoned validation set by using the following
metrics: Accuracy (ACC) and Attack Success Rate (ASR) [55, 59, 28] for the
attacked class. Attack Success Rate (ASR) refers to the proportion of non-
target classes misclassified by the classifier as the targeted class. All
results are averaged over 5 independent runs with different seeds.

###  5.4 Results and Discussions

The results on poisoned ImageNet-100 and STL-10 under different attack
settings are shown in Table 1. Based on the results, we have the following
observations:

  * •

Our proposed PoisonCAM significantly outperforms baselines against SSL
backdoor attacks. Specifically, on poisoned validation sets, PoisonCAM
achieves average ACC improvements of $4.7\%$, $2.1\%$, and $5.5\%$ and average
ASR reductions of $4.1\%$, $0.9\%$, and $1.0\%$ on ImageNet-100 (poison rate
0.5%), ImageNet-100 (poison rate 1.0%), and STL-10 (poison rate 5.0%),
respectively. These results indicate that PoisonCAM can better defend against
backdoor attacks by accurately removing poisonous samples for detoxified
training.

  * •

Our PoisonCAM achieves similar or even higher ACC than w/o Defense on clean
validation sets where backdoor triggers are absent. Compared to PatchSearch,
PoisonCAM achieves $2.8\%$, $1.8\%$, and $1.2\%$ average ACC improvements on
ImageNet-100 (poison rate 0.5%), ImageNet-100 (poison rate 1.0%), and STL-10
(poison rate 5.0%), respectively while achieving similar ASRs. This is because
PoisonCAM can classify poisonous samples more accurately and delete fewer
benign samples to facilitate sufficient training. Since PatchSearch deletes
too many clean samples, its ACC obviously declines compared to w/o Defense on
three datasets.

Table 2:  Poison detection results. We compare the average results over target categories for detecting poisonous images in the poisoned datasets. Total Rem. denotes the number of total removed samples. Dataset | Metric | PatchSearch | PoisonCAM
---|---|---|---
ImageNet-100 (poison rate 0.5%) | Total Rem. | 12807.5 | 1570.8
Recall$\uparrow$ | 84.4 | 98.7
Precision$\uparrow$ | 5.4 | 49.3
ImageNet-100 (poison rate 1.0%) | Total Rem. | 10740.7 | 2214.9
Recall$\uparrow$ | 99.0 | 99.1
Precision$\uparrow$ | 20.9 | 64.3
STL-10 (poison rate 5.0%) | Total Rem. | 744.0 | 332.0
Recall$\uparrow$ | 98.5 | 99.1
Precision$\uparrow$ | 62.5 | 78.6

###  5.5 Analysis of Poisonous Image Detection

A key step in PatchSearch and our PoisonCAM is to detect poisonous images in
the training set and remove them. To further investigate the effectiveness of
our method, we analyze the results of poisonous image detection by PoisonCAM
and PatchSearch. In Table 2, we report the number of total removed images
(Total Rem.), recall, and precision on three datasets. We have the following
observations:

  * •

Compared to PatchSearch, our PoisonCAM significantly improves average
precision by 43.9%, 43.4%, and 16.1% on ImageNet-100 (poison rate 0.5%),
ImageNet-100 (poison rate 1.0%), and STL-10 (poison rate 5.0%), respectively.
These results demonstrate that PoisonCAM can detect poisonous images more
accurately and reduce mistakenly removed benign samples, which also results in
the lower Total Rem. Subsequently, PoisonCAM can facilitate more sufficient
training to improve the performance of the SSL models.

  * •

Moreover, PoisonCAM achieves similar or higher recall compared to PatchSearch
on three datasets. Especially, the recall of PoisonCAM is always higher than
95% on all datasets. As a result, PoisonCAM can remove poisonous samples more
effectively and facilitate detoxified SSL training.

We include the detailed results of every target category in Supplementary
Material.

(a) 2-target attack

(b) 3-target attack

Figure 4: Results of the detected top-$k$ candidate triggers by PatchSearch
and our PoisonCAM against multi-target attacks.

###  5.6 Results of Multi-Target Attacks

To further investigate the effectiveness of different methods, we conduct
experiments against multi-target attacks. Specifically, we utilize multiple
target categories and correlate different backdoor triggers to different
target categories. We combine target categories of “rottweiler” and “tabby
cat” to conduct 2-target attacks, and further add “ambulance” to conduct
3-target attacks on ImageNet-100 (poison rate 0.5%). As shown in Figure 4, we
report the results of top-$k$ retrieved candidate triggers by PatchSearch and
PoisonCAM. We introduce two metrics, Intersection over Union (IoU) [40, 9],
and Catch Rate (CR). IoU measures the union divided by the intersection
between the real triggers and the retrieved triggers. CR measures the ratio of
the real triggers contained in the retrieved triggers.

We have the following observations: (1) Our PoisonCAM significantly
outperforms PatchSearch against multi-target attacks. For example, PoisonCAM
always achieves the best CR with every search number from top-5 to top-500.
These results show that PoisonCAM can locate larger trigger areas than
PatchSearch, which facilitates training a more accurate poison classifier to
identify poisonous samples. (2) CR and IoU of both methods decline with the
search number increasing from top-5 to top-500. However, diverse candidate
triggers are also necessary for capturing the global characteristics of the
real triggers. Therefore, an appropriate search number should be selected to
balance the accuracy and diversity of the detected triggers.

(a) IoU & CR

(b) Recall & Precision

Figure 5: Ablation study on different masking strategies on ImageNet-100
(poison rate 0.5%, target category “rottweiler”). Rec., Prec. denotes Recall
and Precision.

Figure 6: Visualization of trigger attention maps generated by PatchSearch and
our PoisonCAM on ImageNet-100 (poison rate 0.5%, target category
“rottweiler”).

(a) IoU

(b) CR

(c) F1-score

(d) Average Processing Time

Figure 7: Hyper-parameter sensitivity study on the number $B$ of masks on
ImageNet-100 (poison rate 0.5%, target category “rottweiler”). (a-b) IoU and
CR of the detected candidate triggers. (c) F1-score of the trained poison
classifier. (d) Average processing time per image of candidate trigger
detection.

###  5.7 Ablation Study

To further investigate the effectiveness of the designed masking strategies in
Section 4.2, we design different masking strategies and conduct an ablation
study on ImageNet-100 (poison rate 0.5%) with the target category
“rottweiler”: Recalling that we set the size of masking windows as $[w,w]$, we
compare three masking strategies:

  1. 1.

0-1 Interval Masking: A block is defined with a size of
$[w^{\prime},w^{\prime}](w^{\prime}\ll w)$. The entire window is evenly
divided into blocks. Every block is assigned a value of 0 or 1, $s.t.$, every
two adjacent blocks are assigned different values, as shown in Figure 3 (a).

  2. 2.

Random Masking [18]: Again, the entire window is evenly divided into blocks.
Half of the blocks are randomly selected and assigned a value of 0, others are
assigned a value of 1, as shown in Figure 3 (b).

  3. 3.

Full Coverage Masking: The entire window is assigned a value of 1, as shown in
Figure 3 (c).

As we report the results in Figure 5, we have the following observations: (1)
Full Coverage Masking achieves both the highest IoU and CR among all
strategies and can catch nearly the entire ($\sim$100% CR) trigger regions
with the search number from top-20 to top-100. (2) Simultaneously, the trained
poison classifier with Full Coverage Masking achieves the highest precision
and nearly 100% recall with different search numbers. Based on these
observations, we empirically select Full Coverage Masking in our method.

###  5.8 Analysis of Trigger Attention Maps

To further investigate the process of two defenders, PoisonCAM and
PatchSearch, we conduct a qualitative analysis of the generated trigger
attention maps. Both methods generate trigger attention maps for images and
select the hottest regions as the candidate triggers. PatchSearch adopts Grad-
CAM [45] to compute the trigger attention, while we propose a Cluster
Activation Masking method in Section 4.2 to solve this problem. Therefore, we
visualize the cases of trigger attention maps generated by PoisonCAM and
PatchSearch on ImageNet-100 (poison rate 0.5%) with the target category
“rottweiler” in Figure 6. Our method can always accurately locate the backdoor
triggers contained in these images, while PatchSearch focuses on more
dispersed and maybe irrelevant regions. These results further demonstrate that
our PoisionCAM can significantly improve the detection accuracy of the
injected backdoor trigger, which is fundamental for defending against SSL
backdoor attacks.

###  5.9 Hyper-parameter Sensitivity Study

In this section, we analyze the number $B$ of masks in terms of sensitivity
with the search number from top-20 to top-100 on ImageNet-100 (poison rate
0.5%) with the target category “rottweiler”. As demonstrated in Figure 7, we
have the following observations: (1) While increasing the number $B$ of masks,
IoU, CR of the detected candidate triggers and F1-score of the trained poison
classifier first increase and then become relatively stable, which shows that
our method can effectively locate the triggers with appropriate large $B$. The
searching time increases with more masks to calculate. (2) The average
processing time consistently increases while the number $B$ of masks
increases. Considering the worst situation where all masks are processed
serially, the slope of the time curve will tend towards a constant. To
conclude, an appropriate value of $B$ should be chosen to balance the
performance and time costs. We empirically set $B=256$ in our method.

##  6 Conclusion

In this paper, we propose a novel PoisonCAM method to defend against self-
supervised learning (SSL) backdoor attacks. PoisonCAM exhibits a robust
capacity to accurately detect and remove poisonous data from a poisoned and
unlabeled dataset to facilitate detoxified SSL training. We propose a Cluster
Activation Masking method to accurately retrieve trigger patches injected into
the poisoned dataset. Based on the retrieved trigger patches, an effective
poison classifier is trained to distinguish between poisonous and clean data
in the training set. Extensive experiments on ImageNet-100 and STL-10
demonstrate that PoisonCAM outperforms the state-of-the-art method for
defending against SSL backdoor attacks. We hope this paper can contribute to
the safety of artificial intelligence systems.

## References

  * Bansal et al. [2023] Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, and Kai-Wei Chang.  Cleanclip: Mitigating data poisoning attacks in multimodal contrastive learning.  In _ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models_ , 2023.
  * Carlini and Terzis [2021] Nicholas Carlini and Andreas Terzis.  Poisoning and backdooring contrastive learning.  In _International Conference on Learning Representations_ , 2021.
  * Caron et al. [2020] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised learning of visual features by contrasting cluster assignments.  In _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , Red Hook, NY, USA, 2020. Curran Associates Inc.
  * Chattopadhay et al. [2018] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian.  Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.  In _2018 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pages 839–847, 2018.
  * Chen et al. [2020] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR, 2020.
  * Chen and He [2021] Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 15750–15758, 2021.
  * Chen et al. [2020] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He.  Improved Baselines with Momentum Contrastive Learning.  _arXiv e-prints_ , art. arXiv:2003.04297, 2020.
  * Chen et al. [2021] Xinlei Chen, Saining Xie, and Kaiming He.  An empirical study of training self-supervised vision transformers.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9640–9649, 2021.
  * Cheng et al. [2021] Bowen Cheng, Ross Girshick, Piotr Dollár, Alexander C Berg, and Alexander Kirillov.  Boundary iou: Improving object-centric image segmentation evaluation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 15334–15342, 2021.
  * Chuang et al. [2020] Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka.  Debiased contrastive learning.  _Advances in neural information processing systems_ , 33:8765–8775, 2020.
  * Coates et al. [2011] Adam Coates, Andrew Y. Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  _international conference on artificial intelligence and statistics_ , 2011.
  * Dosovitskiy et al. [2014] Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox.  Discriminative unsupervised feature learning with convolutional neural networks.  _Advances in neural information processing systems_ , 27, 2014.
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  _ICLR_ , 2021.
  * Gidaris et al. [2018] Spyros Gidaris, Praveer Singh, and Nikos Komodakis.  Unsupervised representation learning by predicting image rotations.  In _International Conference on Learning Representations_ , 2018.
  * Grill et al. [2020] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent-a new approach to self-supervised learning.  _Advances in neural information processing systems_ , 33:21271–21284, 2020.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * He et al. [2020] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9729–9738, 2020.
  * He et al. [2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 16000–16009, 2022.
  * Huang et al. [2021] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren.  Backdoor defense via decoupling the training process.  In _International Conference on Learning Representations_ , 2021.
  * Jaiswal et al. [2020] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon.  A survey on contrastive self-supervised learning.  _Technologies_ , 9(1):2, 2020.
  * Jia et al. [2022] Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong.  Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning.  In _2022 IEEE Symposium on Security and Privacy (SP)_ , pages 2043–2059. IEEE, 2022.
  * Jiang et al. [2021] Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei.  Layercam: Exploring hierarchical class activation maps for localization.  _IEEE Transactions on Image Processing_ , 30:5875–5888, 2021.
  * Kenton and Toutanova [2019] Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of NAACL-HLT_ , pages 4171–4186, 2019.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Koohpayegani et al. [2021] Soroush Abbasi Koohpayegani, Ajinkya Tejankar, and Hamed Pirsiavash.  Mean shift for self-supervised learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 10326–10335, 2021.
  * Krishnan et al. [2022] Rayan Krishnan, Pranav Rajpurkar, and Eric J Topol.  Self-supervised learning in medicine and healthcare.  _Nature Biomedical Engineering_ , 6(12):1346–1352, 2022.
  * Li et al. [2023] Changjiang Li, Ren Pang, Zhaohan Xi, Tianyu Du, Shouling Ji, Yuan Yao, and Ting Wang.  An embarrassingly simple backdoor attack on self-supervised learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4367–4378, 2023.
  * Li et al. [2020] Shaofeng Li, Minhui Xue, Benjamin Zi Hao Zhao, Haojin Zhu, and Xinpeng Zhang.  Invisible backdoor attacks on deep neural networks via steganography and regularization.  _IEEE Transactions on Dependable and Secure Computing_ , 18(5):2088–2105, 2020.
  * Liu et al. [2023] Min Liu, Alberto Sangiovanni-Vincentelli, and Xiangyu Yue.  Beating backdoor attack at its own game.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4620–4629, 2023.
  * Liu et al. [2021] Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang.  Self-supervised learning: Generative or contrastive.  _IEEE transactions on knowledge and data engineering_ , 35(1):857–876, 2021.
  * McQueen [1967] James B McQueen.  Some methods of classification and analysis of multivariate observations.  In _Proc. of 5th Berkeley Symposium on Math. Stat. and Prob._ , pages 281–297, 1967.
  * Min et al. [2023] Rui Min, Zeyu Qin, Li Shen, and Minhao Cheng.  Towards stable backdoor purification through feature shift tuning.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Misra and Maaten [2020] Ishan Misra and Laurens van der Maaten.  Self-supervised learning of pretext-invariant representations.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 6707–6717, 2020.
  * Mu et al. [2023] Bingxu Mu, Zhenxing Niu, Le Wang, Xue Wang, Qiguang Miao, Rong Jin, and Gang Hua.  Progressive backdoor erasing via connecting backdoor and adversarial attacks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 20495–20503, 2023.
  * Noroozi and Favaro [2016] Mehdi Noroozi and Paolo Favaro.  Unsupervised learning of visual representions by solving jigsaw puzzles.  In _ECCV_ , 2016.
  * Pang et al. [2023] Lu Pang, Tao Sun, Haibin Ling, and Chao Chen.  Backdoor cleansing with unlabeled data.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12218–12227, 2023.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ , 32, 2019.
  * Qi et al. [2023] Xiangyu Qi, Tinghao Xie, Jiachen T Wang, Tong Wu, Saeed Mahloujifar, and Prateek Mittal.  Towards a proactive $\\{$ML$\\}$ approach for detecting backdoor poison samples.  In _32nd USENIX Security Symposium (USENIX Security 23)_ , pages 1685–1702, 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _Advances in neural information processing systems_ , 28, 2015.
  * Russakovsky et al. [2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision (IJCV)_ , 115(3):211–252, 2015.
  * Saha et al. [2020] Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash.  Hidden trigger backdoor attacks.  In _Proceedings of the AAAI conference on artificial intelligence_ , pages 11957–11965, 2020.
  * Saha et al. [2022] Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Koohpayegani, and Hamed Pirsiavash.  Backdoor attacks on self-supervised learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13337–13346, 2022.
  * Schiappa et al. [2023] Madeline C Schiappa, Yogesh S Rawat, and Mubarak Shah.  Self-supervised learning for videos: A survey.  _ACM Computing Surveys_ , 55(13s):1–37, 2023.
  * Selvaraju et al. [2017] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _2017 IEEE International Conference on Computer Vision (ICCV)_ , pages 618–626, 2017.
  * Shah et al. [2022] Anshul Shah, Suvrit Sra, Rama Chellappa, and Anoop Cherian.  Max-margin contrastive learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 8220–8230, 2022.
  * Sun et al. [2020] Mingjie Sun, Siddhant Agarwal, and J Zico Kolter.  Poisoned classifiers are not only backdoored, they are fundamentally broken.  _arXiv preprint arXiv:2010.09080_ , 2020.
  * Tao et al. [2023] Guanhong Tao, Zhenting Wang, Shiwei Feng, Guangyu Shen, Shiqing Ma, and Xiangyu Zhang.  Distribution preserving backdoor attack in self-supervised learning.  In _2024 IEEE Symposium on Security and Privacy (SP)_ , pages 29–29. IEEE Computer Society, 2023.
  * Tejankar et al. [2023] Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang, Sinong Wang, Hamed Firooz, Hamed Pirsiavash, and Liang Tan.  Defending against patch-based backdoor attacks on self-supervised learning.  In _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 12239–12249, 2023.
  * Tian et al. [2020] Yonglong Tian, Dilip Krishnan, and Phillip Isola.  Contrastive multiview coding.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16_ , pages 776–794. Springer, 2020.
  * Wang et al. [2020] Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu.  Score-cam: Score-weighted visual explanations for convolutional neural networks.  In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , pages 111–119, 2020.
  * Wu et al. [2018] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin.  Unsupervised feature learning via non-parametric instance discrimination.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3733–3742, 2018.
  * Xu et al. [2023] Qiuling Xu, Guanhong Tao, Jean Honorio, Yingqi Liu, Shengwei An, Guangyu Shen, Siyuan Cheng, and Xiangyu Zhang.  Medic: Remove model backdoors via importance driven cloning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 20485–20494, 2023.
  * Yang et al. [2019] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32, 2019.
  * Yao et al. [2019] Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao.  Latent backdoor attacks on deep neural networks.  In _Proceedings of the 2019 ACM SIGSAC conference on computer and communications security_ , pages 2041–2055, 2019.
  * You et al. [2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen.  Graph contrastive learning with augmentations.  _Advances in neural information processing systems_ , 33:5812–5823, 2020.
  * Zhang et al. [2016] Richard Zhang, Phillip Isola, and Alexei A Efros.  Colorful image colorization.  In _ECCV_ , 2016.
  * Zhang et al. [2023] Zaixi Zhang, Qi Liu, Zhicai Wang, Zepu Lu, and Qingyong Hu.  Backdoor defense via deconfounded representation learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12228–12238, 2023.
  * Zhu et al. [2020] Liuwan Zhu, Rui Ning, Cong Wang, Chunsheng Xin, and Hongyi Wu.  Gangsweep: Sweep out neural backdoors by gan.  In _Proceedings of the 28th ACM International Conference on Multimedia_ , pages 3173–3181, 2020.
  * Zhu et al. [2023] Mingli Zhu, Shaokui Wei, Hongyuan Zha, and Baoyuan Wu.  Neural polarizer: A lightweight and effective backdoor defense via purifying poisoned features.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.

##  Appendix A Preliminary: Heuristic Search Strategy used in Section 4.3

In Section 4.3, to improve the efficiency and utilize the clustering
information of images, we adopt a heuristic search strategy following [49] to
iteratively compute poison scores for a part of (not all) images in
$\boldsymbol{X}$. To make our paper self-contained and facilitate the
comprehension of experimental results, we introduce the algorithm in this
section. More details can be found in [49] and our released code.

At every iteration, we first compute the poison scores of $s$ randomly
selected samples per cluster and use the sums to represent the poison scores
of clusters. Since clusters with lower poison scores are less likely to be
poisoned, we then remove a fraction $r$ of the clusters with the least poison
scores and will not compute poison scores for images in these clusters. After
several iterations, all clusters are removed, and a part of images in
$\boldsymbol{X}$ are scored during the iterative procedure. Finally, we take
the top-$k$ scored images with the highest poison scores to form a poison set
$\boldsymbol{X}^{p}$. This poison set will be utilized to train a classifier
of poisonous samples. In this heuristic algorithm, the number of scored images
is unfixed.

##  Appendix B Implementation Details

We utilize PyTorch [37] to implement all experiments on four GeForce RTX 3090
GPUs. We employed the following parameters for PoisonCAM on ImageNet-100 as
same as PatchSearch: a cluster count of $l=1000$, the number of the flip test
set is set as $1000$, samples per cluster $s=2$, and removal of $r=25\%$ of
candidate clusters after each iteration. The search window size was set to
$w=60$, using a complete sampling approach. This method typically involves
searching through approximately 8,000 images on ImageNet-100. For training the
poison classifier, we sample top-20 poisonous images (i.e.,
$|\boldsymbol{X}^{p}|=20$) and remove the top $10\%$ poisonous samples in
$\boldsymbol{X}$ to reduce noise in the poison classification set
$\bar{\boldsymbol{X}}$. Besides, we set the number of masks as $B=256$.

Table 3: The attack effectiveness throughout the backdoored SSL process. We evaluated the attack effectiveness using poisoned data to train the self-supervised model with different training epochs. We test the accuracy and false positive rates of the model on clean and poisoned data after fine-tuning on downstream classification tasks using clean datasets. (poison rate 0.5%, target category “laptop” and “goose”). The results show a strong correlation between the effectiveness of the attack and the model performance before 100 epochs. Epoch | laptop | goose
---|---|---
Clean Data | Poison Data | Clean Data | Poison Data
Acc | FP | Acc | FP | Acc | FP | Acc | FP
20 | 24.4 | 47 | 18.7 | 114 | 24.5 | 56 | 18.2 | 115
40 | 38.5 | 44 | 31.8 | 416 | 39.0 | 39 | 31.7 | 181
60 | 51.8 | 45 | 27.8 | 2390 | 52.2 | 30 | 41.3 | 566
80 | 59.4 | 44 | 33.3 | 2368 | 59.8 | 17 | 46.7 | 705
100 | 64.1 | 40 | 38.9 | 2238 | 64.2 | 16 | 46.6 | 1223
120 | 67.1 | 39 | 42.4 | 2112 | 66.7 | 12 | 46.0 | 1467
140 | 68.2 | 38 | 40.8 | 2325 | 68.3 | 13 | 47.0 | 1485
160 | 69.0 | 46 | 39.5 | 2492 | 69.0 | 12 | 48.1 | 1420
180 | 69.4 | 43 | 38.6 | 2533 | 69.2 | 10 | 42.9 | 1888
200 | 69.2 | 43 | 38.0 | 2593 | 69.4 | 11 | 44.7 | 1758
Table 4:  Poison detection results. We compared the results of using both methods for cleaning poisoned data at the data level. Total Rem. denotes the total removed samples. Attack Target | PatchSearch | PoisonCAM
---|---|---
Dataset | Target Category | Total Rem. |  Recall$\uparrow$ |  Precision$\uparrow$ | Total Rem. |  Recall$\uparrow$ |  Precision$\uparrow$
ImageNet-100 (poison rate 0.5%) | rottweiler | 8449 | 97.1 | 7.5 | 1556 | 99.5 | 41.6
tabby cat | 11341 | 99.2 | 5.7 | 2314 | 99.8 | 28.0
ambulance | 13212 | 31.5 | 1.6 | 3509 | 98.5 | 18.2
pickup truck | 16523 | 96.8 | 3.8 | 921 | 98.5 | 69.5
laptop | 14944 | 68.5 | 3.0 | 1245 | 98.2 | 51.2
goose | 14002 | 99.1 | 4.6 | 844 | 99.1 | 76.3
pirate ship | 18421 | 53.5 | 1.6 | 1924 | 96.1 | 27.9
gas mask | 4785 | 99.7 | 13.5 | 1457 | 99.5 | 44.4
vacuum cleaner | 6600 | 98.9 | 9.7 | 806 | 98.0 | 79.0
American lobster | 19798 | 99.7 | 3.3 | 1132 | 99.4 | 57.1
average | 12807.5 | 84.4 | 5.4 | 1570.8 | 98.7 | 49.3
ImageNet-100 (poison rate 1.0%) | rottweiler | 20585 | 99.5 | 6.3 | 2083 | 99.6 | 62.2
tabby cat | 3806 | 99.8 | 34.1 | 5106 | 99.5 | 25.3
ambulance | 14627 | 96.4 | 8.6 | 1980 | 97.5 | 64.0
pickup truck | 7918 | 99.6 | 16.4 | 1937 | 99.4 | 66.7
laptop | 7918 | 98.2 | 21.7 | 1767 | 99.6 | 73.3
goose | 34793 | 99.5 | 3.7 | 1517 | 98.9 | 84.8
pirate ship | 10102 | 99.5 | 11.0 | 1925 | 98.2 | 57.0
gas mask | 6816 | 99.2 | 18.7 | 1961 | 99.7 | 66.1
vacuum cleaner | 2023 | 98.8 | 63.5 | 1696 | 99.5 | 76.3
American lobster | 5194 | 99.8 | 25.0 | 1896 | 98.8 | 67.8
average | 11174.9 | 99.0 | 20.9 | 2186.8 | 99.1 | 64.4
STL-10 (poison rate 5.0%) | bird | 306 | 98.8 | 80.7 | 295 | 98.8 | 83.7
car | 1755 | 100.0 | 14.2 | 441 | 98.4 | 55.8
deer | 261 | 96.8 | 92.7 | 260 | 100.0 | 96.2
average | 744.0 | 98.5 | 62.5 | 332.0 | 99.1 | 78.6

##  Appendix C Additional Experiments

###  C.1 Detailed Results of Poisonous Image Detection

In this section, we list detailed results for poisonous image detection. As
shown in Table 4, our PoisonCAM significantly outperforms PatchSearch for
defending every category except target category tabby cat on ImageNet-100 with
poison rate $1.0\%$. These results demonstrate that PoisonCAM can enhance the
accuracy of detecting poisonous images while reducing the number of
erroneously removed benign samples. As a result, PoisonCAM can facilitate more
comprehensive training, leading to improved performance of the Self-Supervised
Learning (SSL) models.

###  C.2 More Trigger Attention Maps of Different Target Categories

Figure 8: Visualization of trigger attention maps generated by PatchSearch and
our PoisonCAM on ImageNet-100 (poison rate 0.5%, target category “tabby cat”).

Figure 9: Visualization of trigger attention maps generated by PatchSearch and
our PoisonCAM on ImageNet-100 (poison rate 0.5%, target category “ambulance”).

Figure 10: Visualization of trigger attention maps generated by PatchSearch
and our PoisonCAM on ImageNet-100 (poison rate 0.5%, target category “pickup
truck”).

In this section, we show more trigger attention maps of different target
categories to further investigate the search processes of two defenders,
PoisonCAM and PatchSearch. Both methods generate trigger attention maps for
images and select the hottest regions as the candidate triggers. PatchSearch
adopts Grad-CAM [45] to compute the trigger attention, while we propose a
Cluster Activation Masking method to solve this problem. In Figures 8, 9, 10,
we visualize more cases of trigger attention maps generated by PoisonCAM and
PatchSearch on ImageNet-100 (poison rate $0.5\%$) with the target categories
“tabby cat”, “ambulance” and “pickup truck”. Our method can always accurately
locate the backdoor triggers contained in these images, while PatchSearch
focuses on more dispersed and maybe irrelevant regions. These results further
demonstrate that our PoisionCAM can significantly improve the detection
accuracy of the injected backdoor trigger, which is fundamental for defending
against SSL backdoor attacks.

###  C.3 Analysis of Backdoored SSL Process

In our assessment of the backdoored SSL process on ImageNet-100 with a poison
rate of $0.5\%$, we conduct a thorough investigation into the correlation
between the overall model performance, as measured by Clean Data Acc, and the
attack effectiveness, as measured by the Patched Data FP. As shown in Table 3,
our findings indicate that in the initial epochs ($epoch\leq 40$), attack
effectiveness has a strong correlation with overall model performance.
Although this correlation is not as stringent in later epochs, the attack
effectiveness still maintains its overall magnitude. These observations
suggest that, when training a model for defense, achieving convergence may not
be necessary. Instead, it is sufficient to train the model until the attack
effectiveness is comparable to that of the fully trained model in overall
magnitude.

###  C.4 Hyper-parameter Sensitivity Study

(a) IoU

(b) CR

(c) Recall

(d) Precision

Figure 11: Hyper-parameter sensitivity study on the window size $w$ of masks
on ImageNet-100 (poison rate 0.5%, target category “rottweiler”). (a-b) IoU
and CR of the detected candidate triggers. (c-d) Recall and precision of the
trained poison classifier.

(a) Acc

(b) IoU

(c) CR

(d) Time

Figure 12: Hyper-parameter sensitivity study on the cluster count of $l$ on
ImageNet-100 (poison rate 0.5%, target category “rottweiler”). (a-c) Acc, IoU,
and CR of the detected candidate triggers. (d) Total processing time and
average processing time per image of candidate trigger detection.

In this section, we analyze hyper-parameter sensitivity studies on window size
$w$ of masks and the cluster count of $l$ on ImageNet-100 with poison rate
0.5%, target category “rottweiler”.

Window size $w$ of masks. As shown in Figure 11, we vary window size $w$ of
masks from 40 to 80, where Acc measures if any part of the trigger is
contained in the retrieved window. We have the following observations: (1)
While increasing $w$, the IoU of the detected candidate triggers sharply drops
for the reason of more trigger-related regions in candidate triggers. (2) CR
and Precision first increase and then relatively drop while $w$ increases,
which demonstrates that our PoisonCAM can accurately search the triggers with
an appropriate $w$. (3) The recall performs relatively stable, which
demonstrates the effectiveness of our method. We empirically set $w=60$ in our
method.

Number $l$ of clusters. As shown in Figure 12, we vary the cluster number $l$
in $k$-means algorithm from 100 to 2000 and have the following observations:
(1) While increasing $l$, the Acc of the detected candidate triggers first
increases and then becomes relatively stable, and the IoU and CR increase
first and then fluctuate. These results demonstrate that a large cluster count
of $l$ can detect more candidate images with poisonous triggers. (2) The total
processing time consistently increases and the average processing time
performs relatively stable while the cluster count of $l$ increases. The
system will spend more calculating time with a larger cluster count of $l$. To
conclude, an appropriate value of $l$ should be chosen to balance the
performance and time costs. We empirically set $l=1000$.

###  C.5 Analysis of Failure Examples

To conduct a comprehensive analysis of our proposed method and promote further
advances in related areas, we analyze the typical failure examples in this
section.

####  C.5.1 Undetected Poisoned Samples

Oversized Trigger

Undersized Trigger

Figure 13: The Poisoned Classifier Misses Examples during the Filtering Stage.

Figure 14: During the process of searching for candidate triggers, there are a
few instances of search errors, which can be categorized into three types. (a)
There are cases where some pixels in the image closely resemble the actual
trigger. (b) The trigger in the image occupies a significant portion. (c) Some
inherent features present in the image itself can have a significant impact on
the model, acting as a natural trigger, such as a pattern resembling a
beehive.

We explore the common characteristics among these undetected poisoned samples
with the poison classifier discussed in Section 4.4. As shown in Figure 13,
the poison classifier wrongly recognizes these oversized and undersized
poisonous images as clean images. The results show the limitation of the
poison classifier.

####  C.5.2 Trigger Search Errors

Though our PoisonCAM achieves high recall and precision in the poisonous image
search stage discussed in Sections 4.2 and 4.3, there still are a few search
errors. As shown in Figure 14, we can have the following observations: (1)
Image resembling the trigger. When some pixels in the candidate image are
similar to the backdoor trigger. (2) Image corrupted by the trigger. Some
triggers are oversized when pasted to form poisonous images, which are hard to
search. (3) Image with the analogous trigger. The features of some images like
honeycombs are similar to the backdoor trigger.