# Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning
and Backdoor Attacks

Wenhan Yang
Department of Computer Science
University of California, Los Angeles
Los Angeles, CA 90024
<EMAIL_ADDRESS>
&Jingdong Gao
Department of Computer Science
University of California, Los Angeles
Los Angeles, CA 90024
<EMAIL_ADDRESS>
Baharan Mirzasoleiman
Department of Computer Science
University of California, Los Angeles
Los Angeles, CA 90024
<EMAIL_ADDRESS>
###### Abstract

Contrastive Language-Image Pre-training (CLIP) on large image-caption datasets
has achieved remarkable success in zero-shot classification and enabled
transferability to new domains. However, CLIP is extremely more vulnerable to
targeted data poisoning and backdoor attacks, compared to supervised learning.
Perhaps surprisingly, poisoning 0.0001% of CLIP pre-training data is enough to
make targeted data poisoning attacks successful. This is four orders of
magnitude smaller than what is required to poison supervised models. Despite
this vulnerability, existing methods are very limited in defending CLIP models
during pre-training. In this work, we propose a strong defense, SafeClip, to
safely pre-train CLIP against targeted data poisoning and backdoor attacks.
SafeClip warms up the model by applying unimodal contrastive learning (CL) on
image and text modalities separately. Then, it carefully divides the data into
safe and risky subsets. SafeClip trains on the risky data by applying unimodal
CL to image and text modalities separately, and trains on the safe data using
the CLIP loss. By gradually increasing the size of the safe subset during the
training, SafeClip effectively breaks targeted data poisoning and backdoor
attacks without harming the CLIP performance. Our extensive experiments show
that SafeClip decrease the attack success rate of targeted data poisoning
attacks from 93.75% to 0% and that of the backdoor attacks from 100% to 0%,
without harming the CLIP performance on various datasets.

##  1 Introduction

Pre-training large vision-language models on enormous amount of paired image-
caption data crawled from the internet has achieved remarkable success in
zero-shot classification and robustness to distribution shift. CLIP learns
image and text representations in a shared space by maximizing the agreement
between the paired image-text representations, and minimizing the aggreement
between the unpaired ones. This alleviates the need for high-quality
annotations and allows scaling up the pre-training data to millions (Radford
et al., 2021) and billions of examples (Jia et al., 2021). Despite the
superior performance, CLIP is extremely vulnerable to targeted data poisoning
and backdoor attacks, where an adversary injects a subset of malicious
examples in the training data to change the prediction of particular examples
at test time. Perhaps surprisingly, poisoning only 0.0001% and 0.01% of the
pre-training data is enough to make targeted data poisoning and backdoor
attacks successful, respectively (Radford et al., 2021). Considering that the
large pre-training data of CLIP is often crawled from the internet, such
attacks are very easy to perform in practice.

Despite this vulnerability, protecting CLIP against targeted data poisoning
and backdoor attacks during pre-training has remained unaddressed. The only
recently proposed method, RoCLIP, aims to disassociate the poisoned image-
caption pairs during pre-training by matching the image representations with
the nearest neighbors of their captions, and matching the caption
representations with the nearest neighbors of their images (Yang &
Mirzasoleiman, 2023). However, this method can only defend CLIP against a
relatively small number of poisons. Two other methods proposed to clean a
poisoned pre-trained CLIP, by fine-tuning on a clean data of the same scale as
pre-training (Yang et al., 2023), or fine-tuning on a clean subset of pre-
training data using CL on image and text modalities (Bansal et al., 2023). The
first method is clearly not applicable to pre-training, and the second one
even increases the attack success rate if applied to poisoned pre-training
data, as we will confirm experimentally.

Protecting CLIP against targeted data poisoning and backdoor attacks during
pre-training is indeed very challenging. This is because training only once on
the poisoned pairs can make the attack successful. In contrast, in the
supervised setting the model should be trained on the poisoned data for
several epochs before the attack succeeds (Biggio et al., 2012; Turner et al.,
2019). Thus, to protect CLIP during pre-training, it is cruical to entirely
exclude the poisoned examples from the pre-training pipeline.

In this work, we propose the first effective defense, SafeClip, against strong
targeted data poisoning and backdoor attacks during pre-training CLIP.
SafeClip first warms up the model by applying unimodal CL on the image and
text modalities separately. This initializes the model in a way that poisoned
image-caption representations have a low similarity initially. Then, it
applies the CLIP loss with a low learning rate to associate image-caption
representations, while maintaining a low similarity for poisoned pairs.
Subsequently, SafeClip divides the data into a small safe set and a large
risky set based on the similarity of their image-caption representations.
SafeClip pre-trains the model by applying the CLIP loss only to the safe set
and applying CL to the image and text modalities of the risky set separately.
The safe and risky sets are updated during the training and the size of the
safe set is gradually increased. In doing so, SafeClip effectively excludes
the vast majority of the poisoned exaples from the safe set and prevents CLIP
loss to associate their poisoned image-captions. This effectively breaks the
attack. SafeClip ensures a superior performance by increasing the size of the
safe set applying data augmentation to its examples during pre-training.

We conduct extensive experiments on the Conceptual Caption (CC) 1M to evaluate
the effectiveness of SafeClip. We show that SafeClip effectively breaks state-
of-the-art targeted data poisoning and backdoor attacks during pre-training,
by decreasing the success rate of targeted data poisoning attacks from 93.75%
to 0% and that of backdoor attacks from 54.3% to 0%, without harming the zero-
shot and linear prob performance of CLIP on various datasets.

##  2 Related Work

Unimodal Contrastive Learning (CL) Unimodal contrastive learning is among the
most successful methods for representation learning (Chen et al., 2020; Caron
et al., 2020; Chen & He, 2021). CL maximizes the agreement between different
augmented views of the same example (positive pairs) while minimizing the
agreement between augmented views of different examples (negative pairs). A
recent body of work aimed to further improve the performance of CL, by
improving the consistency of the representations via a momentum encode (He et
al., 2020), eliminating the need for negative pairs (Grill et al., 2020), or
removing redundancy between components of the representation vectors (Zbontar
et al., 2021). Most relevant to our work is NNCLR, which enriches the learned
representations by keeping a memory bank of augmented representations and use
the nearest neighbor of every example in the pool as its positive pair
(Dwibedi et al., 2021).

Contrastive Language-Image pre-training (CLIP) Large vision-language models
like CLIP (Radford et al., 2021) and ALIGN (Jia et al., 2021) achieved a
remarkable success by contrastive pre-training on 400M and 1B image-caption
pairs crawled from the web. Recent work tried to improve the data efficiency
and performance of CLIP. Specifically, DeCLIP (Li et al., 2021) uses SimSiam
(Chen & He, 2021) and Masked Language Modeling (Devlin et al., 2018) to match
the augmented views of the image representations and the augmented views of
the text representations, to improve the data efficiency of CLIP. SLIP (Mu et
al., 2022) improves the performance by including in-modal contrastive learning
on images using SimCLR, which maximizes the agreement between different views
of the same augmented image while minimizing agreement between augmented views
of different images. CyCLIP (Goel et al., 2022) emphasizes the importance of
in-modal consistency, where the difference in similarities of image-image
pairs should be close to that of text-text pairs, and cross-modal consistency,
where the difference in similarities of each image-text pair should be
similar.

Targeted Data Poisoning and Backdoor Attacks on CLIP  CLIP is highly
susceptible to various types of targeted data poisoning and backdoor attacks
(Carlini & Terzis, 2021; Yang et al., 2023). Targeted data poisoning attacks
(TDPA) aim to deceive the model into misclassifying a specific test example by
modifying the captions of a small subset of the training data. Backdoor
attacks (BA) involve embedding a backdoor trigger into a small subset of
examples in the training data, with the goal of causing the model to
misclassify any test images with the same trigger. A backdoor trigger can be
either visible, like a distinguishable patch, or invisible, like patterned
noise points or patterned image deformation (Chen et al., 2017; Gu et al.,
2017; Nguyen & Tran, 2021). Adding trigger to only $0.01\%$ of the pre-
training data can cause the model to misclassify the backdoored examples. TPDA
is even more effective, requiring only 0.0001% of the data to be poisoned
(Carlini & Terzis, 2021).

Targeted Data Poisoning and Backdoor Defense on CLIP  Despite the
vulnerability of CLIP to TDPA and BA, existing defense methods are very
limited. RoCLIP (Yang & Mirzasoleiman, 2023) is the only proposed defense for
protecting CLIP during pre-training. RoCLIP first augments image-caption pairs
using techniques such as random cropping and color jittering. Subsequently, it
matches each image with the nearest neighbor of its caption, and vice versa.
These nearest-neighbors are drawn from a representation pool, which is updated
at the end of every epoch. However, RoCLIP is effective only against a limited
range of poisons and fails to defend the model when trained on datasets with a
poison rate higher than 0.0015%.

Two recent works proposed data cleansing for fine-tuning CLIP, or cleaning a
poisoned pre-trained CLIP during fine-tuning. Yang et al. (2023) proposed
dropping examples that have a low image-caption similarity based on a clean
pre-trained CLIP, to cleanse the fine-tuning data. This method requires a
clean pre-trained model, and a proper threshold to filter the poisons without
discarding a large amount of clean data. This threshold varies for different
attack types and is difficult to pre-compute. To clean a poisoned CLIP with
TDPA, Yang et al. (2023) proposed fine-tuning on a clean dataset of the same
size as the pre-training data. Moreover, to clean a poisoned CLIP with BA,
Bansal et al. (2023) proposed CleanCLIP, which fine-tunes the model on a clean
subset of the pre-training data with CLIP loss and CL loss on image and text
modalities. The first method is clearly not applicable to pre-training and the
second one increases the attack success rate when applied to the poisoned
data. This is because CL cluster the backdoored images and their cpations, and
the CLIP loss can even better associate the backdoored images with the
poisoned captions.

In this work, we propose the first effective defense for protecting CLIP
against strong TDPA (0.05%) and BA (0.05%) during pre-training, without
compromising the model’s performance.

##  3 Preliminary

###  3.1 Contrastive Language-Image Pre-training (CLIP)

Consider a dataset
$\mathcal{D}=\\{(\boldsymbol{x}_{i}^{\mathcal{I}},\boldsymbol{x}_{i}^{\mathcal{T}})\\}_{i=1}^{n}$
of $n$ image-captions pairs, where $\boldsymbol{x}_{i}^{\mathcal{I}}$ and
$\boldsymbol{x}_{i}^{\mathcal{T}}$ are the image and caption of the $i^{th}$
pair. The CLIP architecture consists of an image encoder
$f_{I}\\!:\\!\mathcal{I}\rightarrow\mathbb{R}^{d}$ and a text encoder
$f_{T}\\!:\\!\mathcal{T}\rightarrow\mathbb{R}^{d}$ to encode images and
captions. The encoded representations are projected into the same space and
are normalized to have unit $\ell_{2}$-norm. We denote the resulting image and
text representations by
$\boldsymbol{z}_{i}^{\mathcal{I}},\boldsymbol{z}_{i}^{\mathcal{T}}$. To create
the multi-modal interaction, the InfoNCE loss is applied to pull the projected
representations of every image-caption pair together while pushing apart the
projected representations of unpaied images and captions in the same mini-
batch. Formally, for a mini-batch of $N$ pairs, the CLIP loss is defined as:

$\displaystyle\mathcal{L}_{\text{CLIP}}\\!=\\!-\frac{1}{2N}\\!\sum_{j=1}^{N}\log\\!\left[\frac{\exp\left(\left<\boldsymbol{z}_{j}^{\mathcal{I}},\boldsymbol{z}_{j}^{\mathcal{T}}\right>/\tau\right)}{\sum_{k=1}^{N}\exp\left(\left<\boldsymbol{z}_{j}^{\mathcal{I}},\boldsymbol{z}_{k}^{\mathcal{T}}\right>/\tau\right)}\right]\\!\\!-\frac{1}{2N}\\!\sum_{k=1}^{N}\log\\!\left[\frac{\exp\left(\left<\boldsymbol{z}_{k}^{\mathcal{I}},\boldsymbol{z}_{k}^{\mathcal{T}}\right>/\tau\right)}{\sum_{j=1}^{N}\exp\left(\left<\boldsymbol{z}_{j}^{\mathcal{I}},\boldsymbol{z}_{k}^{\mathcal{T}}\right>/\tau\right)}\right],$
(1)

where $\tau$ is a trainable temperature parameter, and $\left<.,.\right>$ is
the inner product between two representations. The performances of CLIP is
evaluated with zero-shot or linear-probe, as we discuss next.

Zero-shot classification. Zero-shot classification assess the generalizability
and transferability of the model to unseen tasks. It transforms the downstream
labels into natural language captions using the provided engineered prompt
templates, such as "A photo of a {label}" (Radford et al., 2021). Then, it
calculates the cosine similarity between the representations of a given image
and each prompt, and predicts the label with the highest image-prompt
similarity.

Linear probe classification. Linear probe classification refers to evaluating
the extracted representations from the pre-trained image encoder for training
a linear classifier on the downstream labeled data.

###  3.2 Targeted Data Poisoning and Backdoor Attacks

Targeted data poisoning and backdoor attacks poison CLIP by injecting a set of
poisoned image-caption pairs to the pre-training data. Let
$\mathcal{D}_{p}=\\{(x_{i}^{\mathcal{I}},x_{c}^{\mathcal{T}})|x_{i}^{\mathcal{I}}\in\mathcal{I}_{t},x_{c}^{\mathcal{T}}\in\mathcal{T}_{adv}\\}$
be the injected poisoned pairs, where $\mathcal{I}_{t}$ is the poisoned
image(s) and $\mathcal{T}_{adv}$ is the set of adversarial caption related to
the adversarial label $y_{adv}$. To construct the poisoned caption set, one
can search the training dataset for all captions that contain the adversarial
label and use these captions as the adversarial captions. Another approach is
to use CLIP’s set of 80 different prompt-engineered text descriptions (Radford
et al., 2021) to construct captions for the adversairal label, and then either
use a subset of them or repeat them as necessary. In our work, we construct
$\mathcal{T}_{adv}$ from the training dataset, which is consistent with the
construction methods used in (Carlini & Terzis, 2021; Yang et al., 2023; Yang
& Mirzasoleiman, 2023; Bansal et al., 2023).

Targeted data poisoning attacks aim to misclassify a particular test example,
$x_{i}^{\mathcal{I}}$, as $y_{adv}$. Hence,
$D_{p}=\\{(x_{i}^{\mathcal{I}},x_{c}^{\mathcal{T}})|x_{c}^{T}\in\mathcal{T}_{adv}\\}$.

Backdoor attacks introduce a trigger patch to a set of poisoned images. The
goal is to misclassify any test examples with the trigger patch,
$x_{i}^{\mathcal{I}}\oplus\text{patch}$, as $y_{adv}$. Hence,
$D_{p}=\\{(x_{i}^{\mathcal{I}}\oplus\text{patch},x_{c}^{\mathcal{T}})|x_{i}^{\mathcal{I}}\in\mathcal{I},x_{c}^{T}\in\mathcal{T}_{adv}\\}$.
In contrast to targeted data poisoning attacks which target a particular test
example, backdoor attacks inject random images with the backdoor trigger,
paired with the adversarial captions.

Adversary Objective The primary objective of the adversary is to manipulate
the output representations of CLIP, such that certain images are misclassified
into adversarial categories instead of their true categories, while the other
images are classified correctly.

Adversary Capabilities We assume that the adversary has limited control over
the pre-training data, and can inject a small number ($\leq 0.05\%$ of the
dataset size) of poisoned examples into the training dataset. Adversary also
has the knowledge of the model structure, the training algorithm, and the
hyperparameter used by their victim, but they cannot modify the training
process directly.

##  4 Method

Next, we motivate and present SafeClip for robust pre-training of CLIP against
TDPA and BAs.

Figure 1: Cosine similarity matrices between image-caption representations,
obtained by CLIP and SafeClip. Training with CLIP loss on the poisoned data
associates the poisoned image-caption pairs. In contrast, SafeClip prevents
association of the poisoned image-caption pairs, by clustering images and
captions in the same category via an inmodal CL loss before pre-training.

###  4.1 Motivation

Targeted data poisoning and backdoor attacks can succeed extremely fast when
pre-training CLIP models. For example, when pre-training on a dataset with
0.01% poison rate, only 1 epoch of CLIP is enough to poison the model. Thus,
to prevent the model from being poisoned, it is essential to filter out the
majority of poisoned pairs before the pre-training starts, and keep them out
throughout the pre-training. If the model avoids training on or is exposed to
only a limited amount of the poisoned data, the representations of poisoned
images and captions do not get close during pre-training, and the attack
fails. However, as shown in Fig. 2, the poisoned pairs become inseparable from
the clean pairs after 1 pre-training epochs. To filter out the poisoned pairs,
SafeClip first warms up the model by a few epochs of unimodal CL on image and
text modalities, separately. The unimodal CL clusters simliar images and
similar texts together. In doing so, it effectively pushes the poisoned image-
caption representations apart, thus slowing down the attacks from taking
effect. Then, SafeClip runs 1 epoch of CLIP with small learning rate and
evaluates the cosine similarities of all examples. As shown in Fig. 2, after
the initial warmup epochs, the poisoned pairs can be better separated from the
clean pairs. Since poisoned pairs are less similar compared to clean pairs,
SafeClip only trains with the CLIP loss on examples with high cosine
similarity (safe data), while training on other examples (risky data) with
unimodal CL loss. Throughout the pre-training, SafeClip gradually increases
the amount of data used for training with CLIP loss. Since majority of the
poisoned pairs are never trained on, SafeClip can successfully defend the
model from strong targeted data poisoning and backdoor attacks. Fig. 2(c)
shows that the poison ratio in the pre-training data remains low throughout
pre-training with SafeClip, while more clean pairs are being used to pre-train
with the CLIP loss.

(a) CLIP

(b) SafeClip

(c) SafeClip

Figure 2: Distribution of image-caption cosine similarity after 1 epoch of
pre-training with (a) CLIP and (b) SafeClip. The data to the right of the red
line represents the portion of the (safe) data used for training with CLIP
loss. SafeClip almost exclusively applies the CLIP loss to clean data. (c)
Fraction of remaining poisons throughout pre-training with SafeClip. After the
warmup, the poison ratio drops from 0.08% to 0.018%, and gradually goes to 0%
as the pre-training continues. This shows the effectiveness of SafeClip in
filtering out the poisoned pairs and defending the model.

In summary, to prevent the model from being poisoned, SafeClip consists of
three steps: (1) A few epochs of unimodal CL warmup. (2) One epoch of slow-
paced CLIP warmup. (3) A combination of unimodal CL training and CLIP training
with data updating. The effect of SafeClip on image and text encoders is shown
in Fig 1. CLIP focuses on aligning the paired image-caption representations,
which renders it easy to poison. On the other hand, SafeClip clusters images
and captions in the same category. In doing so, it reduces the similarity of
poisoned image-caption representations, This allows SafeClip to successfully
defend strong poisoning and backdoor attacks. The pseudocode of the SafeClip
is illustrated in Alg. 1. Next, we will discuss each step in more details.

####  4.1.1 unimodal self-supervised Warmup

SafeClip leverages unimodal CL on both image and text modalities separately.
Since unimodal CL does not match poisoned images with captions, it does not
risk poisoning the models. The unimodal CL clusters similar images and similar
captions together. In doing so, poisoned pairs can be better separated from
the clean pairs. Effectively, since the poisoned image(s) and adversarial
captions are from different categories, poisoned images and adversarial
captions cluster with their respective true representation clusters during
unimodal CL warmup, and move further away from each other in representation
space. For example, to poison an image of “cat" with a “plane" caption, the
image needs to move closer to the “plane" text cluster and away from the “cat"
image cluster in the representation space. The closer the image is to its true
“cat" representation cluster at the beginning of training, the more
challenging it becomes to poison the image. Same argument applies to captions.

Nearest-Neighbors When the poison rate is high, poisoned images, which are
either identical images (TDPA) or images sharing the backdoor patch (BA)
cluster tightly together in the representation space. This prevent them from
getting close to the cluster of their true category. To avoid this issue and
enrich the quality of the representations, we extend our unimodal CL training
by using a nearest neighbor pool to find positive pairs (Dwibedi et al.,
2021). That is, rather than directly matching differently augmented views of
the same image or caption $(\boldsymbol{z}_{i},\boldsymbol{z}_{i}^{+})$, we
maintain a pool of image and caption representations, and match each image or
caption representation with their nearest neighbor in the pool rather than its
augmented views. The pool is implemented as a queue, initially initialized
with representations of random examples and is updated by including the
representations of examples in the current mini-batch, while excluding the
oldest representations in the queue. By exposing examples to more diverse
positive pairs, SafeClip prevents clustering of poisoned images and
adversarial captions, and can separate the poisoned pairs more effectively. We
will explore the impact of using the nearest neighbor approach in our ablation
study. The unimodal CL loss is defined as:

$\displaystyle\mathcal{L}_{\text{unimodal\\_NN}}=-\log\\!\left[\frac{\exp\left(\left<{\text{NN}(\boldsymbol{z}_{i},\mathcal{P}),\boldsymbol{z}_{i}^{+}}\right>/\tau\right)}{\sum_{k=1}^{N}\exp\left(\left<\text{NN}(\boldsymbol{z}_{i},\mathcal{P}),\boldsymbol{z}_{k}^{+}\right>/\tau\right)}\right]$
(2)

where $\boldsymbol{z}_{i}$ is the output image/text representation and
$\boldsymbol{z}_{i}^{+}$ is the augmented view of the image/text
representation, and $\text{NN}(\boldsymbol{z}_{i},\mathcal{P})$ is the nearest
neighbor operator defined as:

$\displaystyle
NN(\boldsymbol{z}_{i})=\text{argmin}_{\boldsymbol{p}\in\mathcal{P}}\|\boldsymbol{z}_{i}-\boldsymbol{p}\|_{2}.$
(3)

Slow-paced CLIP Warmup Epoch Although unimodal CL brings similar images and
captions closer in the image and text representation spaces, the images and
their corresponding captions remain relatively distant from each other. Thus,
to associate the image-caption representations and effectively distinguish
between poisoned and clean pairs, one epoch of CLIP warmup becomes essential
before the filtering step. Thus, following the unimodal CL warmup, we proceed
with one additional epoch of training with the CLIP loss. Crucially, to
mitigate the risk of poisoning, as the CLIP loss directly matches the image-
caption pairs, including the potentially poisoned ones, we employ a lower
learning rate. This slow-paced CLIP epoch helps prevent the model from
learning poisoned image-caption pairs while enabling SafeClip to filter out
the majority of poisoned pairs before pre-training.

As shown in Fig 2, the warmup results in a significant separation between
poisoned and clean pairs. In addition, as we will discuss in Sec. 5.2.1, only
a few epochs of unimodal CL is sufficient. SafeClip applies $r=5$ epochs of
unimodal warmup. We will show that this is not sensitive to tuning and can
apply to a wide range of poisons in Sec. 5.1.

####  4.1.2 Mixed training with data updating

After the warm-up phase, we evaluate the cosine similarities of all the
examples and divide them into the safe and risky sets. The top $k\%$ of the
data, characterized by high cosine similarity, is considered safe data, while
the remaining portion is deemed risky data. SafeClip applies CLIP loss to the
safe data, directly matching images with their captions. To ensure the
trustworthiness of the safe data, we select a small value for $k$ (e.g.,
$k=15$). On the other hand, instead of discarding the risky data, we continue
training on it with unimodal CL. However, two concerns still remain: (1) Some
poisoned pairs may still be in the safe data, (2) The model’s performance may
suffer as the CLIP loss is not applied to majority of the examples. To address
these concerns, at the end of each epoch, we assess the cosine similarity of
all examples and update $k=k+i$, to chose a larger fraction of the data with
highest cosine similarity as the new safe set, with $i$ being a small number
(e.g., $i=1$). To further boost the performance, we apply data augmentation to
the examples in the safe set used in the CLIP loss.

With the above update strategy, only a small number of poisoned pairs may
temporarily enter the safe data and cannot poison the model. At the same time,
more training on clean data with CLIP loss and on risky data with unimodal CL
loss allows the model to learn better representations and better distinguish
and discard the poisoned pairs during the training. Additionally, since we
progressively increase the proportion of safe data during training, by the end
of the training, the majority of the data will be part of the safe data and
will be trained on with CLIP loss, thereby resolving the performance issue. To
reduce the computational cost of updating the safe and risky sets, instead of
calculating the cosine similarities of all examples at every epoch, we
recompute all the similarities every $m$ epochs (e.g. $m=5$). In other epochs,
we only update the cosine similarity for $s\%>k\%$ of examples with the
highest similarities, and update the safe and risky set accordingly.

The loss of the mixed training is defined as:

$\displaystyle\mathcal{L}_{\text{{SafeClip}}}(\mathcal{D})=\mathcal{L}_{\text{unimodal\\_NN}}(\mathcal{D}_{\text{risky}})+\mathcal{L}_{\text{CLIP}}(\mathcal{D}_{\text{safe\\_aug}}).$
(4)

Note that, during mixed training, we still apply nearest-neighbors for
unimodal CL.

1 Input: Image encoder $f_{I}$, text encoder $f_{T}$, image pool $P_{I}$, text
pool $P_{T}$, unimodal warmup epochs $r$, training epochs $T$, safe set
initial size $k$, increment ratio $i$

Data: Dataset of image-caption pairs
$\mathcal{D}=\\{(x_{i}^{\mathcal{I}},x_{i}^{\mathcal{T}})\\}_{i=1}^{n},\mathcal{X}_{I}=\\{x_{i}^{\mathcal{I}}\\}_{i=1}^{n},\mathcal{X}_{T}=\\{x_{i}^{\mathcal{T}}\\}_{i=1}^{n}$

2 for  _$t\leftarrow 1$ to $r$_ do

3

4 Train $f_{I}$ with
$\mathcal{L}_{\text{unimodal\\_NN}}(\mathcal{X}_{I},P_{I})$ in Eq 2

5 Train $f_{T}$ with
$\mathcal{L}_{\text{unimodal\\_NN}}(\mathcal{X}_{T},P_{T})$ in Eq. 2

6

7Update $f_{I},f_{T}$ by training on $\mathcal{D}$ with $\mathcal{L}_{CLIP}$
in Eq. 1 using a small learning rate

8for  _$t\leftarrow 1$ to $T$_ do

9 Sort
$\\{\left<f_{I}(x_{i}^{\mathcal{I}}),f_{I}(x_{i}^{\mathcal{T}})\right>\\}_{i=1}^{n}$
in a decreasing manner

10 $\mathcal{D}_{\text{safe}}\leftarrow$ top $k\%$ of data according to the
above ordering

11
$\mathcal{D}_{\text{risky}}\leftarrow\mathcal{D}\setminus\mathcal{D}_{\text{safe}}$

12 $\mathcal{D}_{\text{safe\\_aug}}\leftarrow$ augmented examples in
$\mathcal{D}_{\text{safe}}$

13 Train $f_{I},f_{T}$ with
$\mathcal{L}_{\text{{SafeClip}}}(\mathcal{D})=\mathcal{L}_{\text{unimodal\\_NN}}(\mathcal{D}_{\text{risky}})+\mathcal{L}_{\text{CLIP}}(\mathcal{D}_{\text{safe\\_aug}})$

14 $k\leftarrow k+i$

Algorithm 1 SafeClip

##  5 Experiments

In this section, we evaluate the effectiveness of SafeClip against strong TDPA
and BA. We start by introducing the experimental setup. Then we present our
main results. Finally, we conduct an ablation study on different components of
SafeClip.

Training We used an open-source implementation of CLIP as our base model.
Similar to the setup in (Radford et al., 2021), we utilize a ResNet-50 as the
image encoder and a transformer as the text encoder. Due to computational
constraints and consistent with (Yang & Mirzasoleiman, 2023), we randomly
selected 1 million image-caption pairs from the Conceptual Captions 3M (CC3M)
dataset as our training dataset (Sharma et al., 2018). In each experiment, the
model is trained from scratch for 32 epochs with a batch size of 256, using
the AdamW optimizer (Loshchilov & Hutter, 2017).

Downstream Datasets To evaluate the downstream performance of our model, we
conduct linear probe and zero-shot classifications, as introduced in Sec. 3.1,
on 10 widely used datasets (Radford et al., 2021; Li et al., 2021; Yang &
Mirzasoleiman, 2023) listed in Table 4.

Adversarial Attacks To evaluate the effectiveness of our defense, we consider
two different attack baselines: targeted data poisoning attacks (TDPA)
(Carlini & Terzis, 2021) and backdoor attacks (BA) with visible patch
triggers.

For TPDAs, we randomly select 16 different images from the CC3M validation set
as our target images. For each target image, we choose a random class from the
ImageNet1K dataset (Deng et al., 2009), and construct an adversarial caption
set related to the label as discussed in Sec. 3.2. For each target image, we
generated 100 and 500 poisons.

For BA, we randomly select 200 and 500 images from the CC3M pre-training data
and apply the backdoor trigger. We use the same backdoor triggers as proposed
by (Gu et al., 2017). We choose a random class from the ImageNet1K dataset
(Deng et al., 2009) and construct the adversarial caption set related to the
label as discussed in Sec. 3.2. Each backdoored image is paired with a random
poisoned caption from the adversarial caption set. In our experiment, we used
the class “mushroom"

Defense Baselines We consider two defense baselines against the TDPA and the
BA. First is the only pre-training defense, namely RoCLIP (Yang &
Mirzasoleiman, 2023), that maintains a pool of representations and matches
every image with the nearest neighbor of its caption in the pool, and vice
versa. Second is adaptation of CleanCLIP (Bansal et al., 2023) to pre-
training, which applies CL and CLIP loss on all the pre-training examples.

We measure the effectiveness of attacks using the attack success rate. For
TPDA, Poison Success Rate (PSR) is the fraction of target images that are
classified as the adversarial label. For BA, Backdoor Success Rate (BSR) it is
the fraction of test images containing the backdoor triggers that are
classified as the adversarial label.

###  5.1 Main Result

Here, we evaluate the performance of SafeClip against TDPA and BA. We compare
SafeClip with baselines, based on both attack success rates and downstream
performance.

SafeClip defense First, we evaluate the effectiveness of SafeClip in breaking
TDPA and BA. Table 1 shows that for TPDA, SafeClip effectively reduces PSR of
CLIP from 93.75% to 0% and 6.25% for 100 and 500 poisons, respectively. For
BA, it reduces BSR of CLIP from 54.3% and 100% to 0%. This indicates that
SafeClip can manage to filter the vast majority of the poisoned pairs during
the warmup and throughout the training, and successfully defend CLIP during
pre-training.

Table 1: Effectiveness of SafeClip in defending targeted data poisoning attacks (TPDAs) and backdoor attacks (BAs) vs. baselines. The models are trained on 1M data. Model | # Poison (TPDA) | PSR (TPDA) | # Backdoor (BA) | BSR (BA)
---|---|---|---|---
CLIP | 100 | 93.75% | 200 | 54.33%
500 | 93.75% | 500 | 100%
SafeClip | 100 | 0% | 200 | 0%
500 | 6.25% | 500 | 0%
RoCLIP | 100 | 93.75% | 200 | 0.33%
500 | 75% | 500 | 2.33%
CleanCLIP | 100 | 87.5% | 200 | 66.67%
500 | 93.75% | 500 | 63.33%
Table 2: Downstream linear probe and zero-shot (top 1) accuracy of SafeClip vs CLIP. Method |  Task |  F102 |  Fd101 |  I1K |  Pet |  Cars |  Cal101 |  C10 |  C100 |  DTD |  Air.
---|---|---|---|---|---|---|---|---|---|---|---
|  0-shot |  0.62 | 6.29 | 9.87 | 5.51 | 0.75 | 40.69 | 39.7 | 10.41 | 3.14 | 0.48
SafeClip |  lin-prb |  99.38 | 45.58 | 24.53 | 51.02 | 15.35 | 74.4 | 71.9 | 47.32 | 56.01 | 27.63
|  0-shot |  1.0 | 7.1 | 9.6 | 3.4 | 0.8 | 34.9 | 34.9 | 7.3 | 3.7 | 0.8
CLIP |  lin-prb |  99.5 | 44.9 | 22.2 | 48.2 | 12.9 | 70.4 | 70.5 | 45.8 | 48.2 | 24.9

On the other hand, CleanCLIP and RoCLIP exhibit poor defense performances
against such strong attacks, and can even increase the attack success rate.
RoCLIP fails because the nearest neighbor of the poisoned image and caption is
more likely to be another poisoned image and caption when the number of
poisons is overwhelming. Similarly, CleanCLIP fails as it applies CLIP loss to
all examples including the poisoned pairs. Importantly, this confirms that CL
loss is not effective when applied with the CLIP loss to the poisoned data. We
see that CleanCLIP increases BSR from 54% to 66.67%, when 200 backdoored
images exist in the pre-training data.

SafeClip downstream performance Next, we evaluate SafeClip in terms of the
downstream performance, by reporting linear probe and zero-shot performance on
10 downstream datasets in Table 4. As shown in Table 1, SafeClip provides a
comparable or even superior performance compared to CLIP, which is due to the
increasing size of the safe set and using data augmentation.

###  5.2 Ablation Studies

Finally, we analyze the effect of SafeClip’s different components on its
robustness and performance.

####  5.2.1 Ablation on warmup

SafeClip’s warmup consist of two essential components: unimodal warmup and a
slow-paced CLIP warmup epoch. As introduced in Sec. 4.1.1, during unimodal
warmup, SafeClip applies CL to image and text modalities separately, while
during slow-paced CLIP warmup, SafeClip applies 1 epoch of CLIP loss to all
examples with a lower learning rate. Here, we illustrate the necessity of each
of these components. We randomly select 16 different images as our target
images, and each target image is paired with 100 poisoned captions related to
the adversarial labels. In total, there are 1,600 posioned pairs injected into
the dataset.

Unimodal CL Warmup Unimodal CL is essential to the success of SafeClip. As
shown in Table 3, row 1,2: 5 epochs of unimodal CL warmup enables the model to
filter out 7% more of the total poisons from the top 30% of examples with
highest cosine similarities, compared to 1 epoch of unimodal CL.

(a) Effect of nearest-neighbors

(b) Effect of low learning rate

(c) Effect of unimodal CL on $\mathcal{D}_{\text{risky}}$

Figure 3: Effect of using (a) near-neighbors during unimodal CL warmup and (b)
low learning rate during CLIP warmup. (c) unimodal CL on risky data during
mixed training. (a) and (b) show the remaining fraction of poison in the top
30% data after the warmup, for various initial poison rates, while (c) shows
the PSR at the end of the training.

However, as shown in row 3, the benefit of unimodal CL is rather diminishing,
and 5 more epochs of the unimodal training before filtering can only filter
out 0.1 % more poisons in the top 30%.

Table 3: Effect of number of CL and slow-paced CLIP warmup epochs on poison rate (PSR) # CL epochs | # CLIP epochs | PSR in top 30%
---|---|---
5 | 1 | 0.8%
1 | 1 | 7.8%
10 | 1 | 0.7%
5 | 0 | 27.5%
5 | 2 | 6.8%

Figure 4: Effect of using low increment ratio when updating
$\mathcal{D}_{\text{safe}}$

Slow-paced CLIP warmup Although CLIP training on full data exposed the model
to the poisoned pairs, to correlate the image representation with the caption
representation and better filtering the poisons out, it is essential to train
with CLIP loss for 1 epoch at a low learning rate. As shown in Table 3, row
1,4: without any CLIP training, there are 26.7% more poisons in the top 30% of
training data. At the same time, it is crucial to not train more with CLIP
loss on the full data before filtering out the poisoned pairs. As shown in row
5, applying 1 more CLIP epoch with low learning rate to the poisoned data
introduces 6% more poisons in the top 30%.

In addition, it is important to keep the learning rate low during the CLIP
warmup. As shown in Fig. 3(b), with a lower lr, 0.9% less poisons are found in
the top 30% when the dataset has a 0.01% poison rate, and 3.3% less poisons
are found in the top 30% when the dataset has a 0.05% poison rate.

Ablation on nearest-neighbors To study the effect of using nearest-neighbors
in the unimodal CL warmup, we randomly select 16 different images as target,
and each target image is paired with 100 or 500 captions related to the
adversarial labels. Fig. 3(a) shows when the dataset is lightly poisoned, the
effect of NN is not obvious, and the total poison rate in the top 30% of the
data is similar when using or not using NN. But, when the poison rate is high,
NN contributes significantly to decreasing the cosine similarities of the
poisoned pairs, marking a 7.3% drop in total poison rate in the top 30% data.

####  5.2.2 Ablation on mixed training

SafeClip’s mixed training epochs gradually incorporate increasing amounts of
data as safe, allowing them to be trained with the CLIP loss, while keeping
poisoned pairs away throughout the training process. This is essential for
both the high performance of SafeClip, despite initially being trained with a
small amount of data, and for achieving a low attack success rate.
Specifically, two design choices contribute significantly to our mixed
training: (1) unimodal CL training on risky data. (2)the small increment ratio
$i$. Here, we illustrate the necessity of each of these design choices. We run
our experiments on 100K data from CC3M. We randomly select 16 different images
as our target images, and pair each with 30 adversarial captions. In total,
480 poisoned pairs are injected into the dataset.

Unimodal CL on risky data Similar to unimodal warmup epochs as discussed in
Sec. 4.1.1, training with unimodal CL on risky data can improve SafeClip’s
defense effectiveness significantly. As shown in Fig. 3(c), unimodal CL helps
on risky data decrease the PSR by 12.5%.

Small increment ratio Larger increment ratio for updating safe data improves
CLIP’s performance, as it allows more data to be trained with CLIP loss
earlier. However, this can put the model into risk if any poisoned pairs enter
the safe data. As shown in Fig. 4, when we choose a larger $i=5$, the PSR is
increased by 37.5%.

##  6 Conclusion

We proposed SafeClip, an effective method for safely pre-train CLIP against
targeted data poisoning and backdoor attacks. Using inmodal CL warmup and
slow-paced CLIP warmup, SafeClip filters out majority of the poisons before
pre-training and continue to exclude them during pre-training with mixed
training strategy. Through extensive experiments, we demonstrated that
SafeClip drops the attack success rate down to 0% for backdoor attacks and
6.25 % for targeted data poisoning attack with poisoning ratio of 0.05%, while
preserving the CLIP performances on various downstream datasets.

## References

  * Bansal et al. (2023) Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, and Kai-Wei Chang.  Cleanclip: Mitigating data poisoning attacks in multimodal contrastive learning.  _arXiv preprint arXiv:2303.03323_ , 2023.
  * Biggio et al. (2012) Battista Biggio, Blaine Nelson, and Pavel Laskov.  Poisoning attacks against support vector machines.  _arXiv preprint arXiv:1206.6389_ , 2012.
  * Carlini & Terzis (2021) Nicholas Carlini and Andreas Terzis.  Poisoning and backdooring contrastive learning.  _arXiv preprint arXiv:2106.09667_ , 2021.
  * Caron et al. (2020) Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised learning of visual features by contrasting cluster assignments.  _Advances in neural information processing systems_ , 33:9912–9924, 2020.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pp. 1597–1607. PMLR, 2020.
  * Chen & He (2021) Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 15750–15758, 2021.
  * Chen et al. (2017) Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song.  Targeted backdoor attacks on deep learning systems using data poisoning.  _arXiv preprint arXiv:1712.05526_ , 2017.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Dwibedi et al. (2021) Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman.  With a little help from my friends: Nearest-neighbor contrastive learning of visual representations.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 9588–9597, 2021.
  * Goel et al. (2022) Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan Rossi, Vishwa Vinay, and Aditya Grover.  Cyclip: Cyclic contrastive language-image pretraining.  _Advances in Neural Information Processing Systems_ , 35:6704–6719, 2022.
  * Grill et al. (2020) Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent-a new approach to self-supervised learning.  _Advances in neural information processing systems_ , 33:21271–21284, 2020.
  * Gu et al. (2017) Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg.  Badnets: Identifying vulnerabilities in the machine learning model supply chain.  _arXiv preprint arXiv:1708.06733_ , 2017.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 9729–9738, 2020.
  * Jia et al. (2021) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.  Scaling up visual and vision-language representation learning with noisy text supervision.  In _International conference on machine learning_ , pp. 4904–4916. PMLR, 2021.
  * Li et al. (2021) Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, and Junjie Yan.  Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm.  _arXiv preprint arXiv:2110.05208_ , 2021.
  * Loshchilov & Hutter (2017) Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * Mu et al. (2022) Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie.  Slip: Self-supervision meets language-image pre-training.  In _European Conference on Computer Vision_ , pp. 529–544. Springer, 2022.
  * Nguyen & Tran (2021) Anh Nguyen and Anh Tran.  Wanet–imperceptible warping-based backdoor attack.  _arXiv preprint arXiv:2102.10369_ , 2021.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pp. 8748–8763. PMLR, 2021.
  * Sharma et al. (2018) Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut.  Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 2556–2565, 2018\.
  * Turner et al. (2019) Alexander Turner, Dimitris Tsipras, and Aleksander Madry.  Label-consistent backdoor attacks.  _arXiv preprint arXiv:1912.02771_ , 2019.
  * Yang & Mirzasoleiman (2023) Wenhan Yang and Baharan Mirzasoleiman.  Robust contrastive language-image pretraining against adversarial attacks.  _arXiv preprint arXiv:2303.06854_ , 2023.
  * Yang et al. (2023) Ziqing Yang, Xinlei He, Zheng Li, Michael Backes, Mathias Humbert, Pascal Berrang, and Yang Zhang.  Data poisoning attacks against multimodal encoders.  In _International Conference on Machine Learning_ , pp. 39299–39313. PMLR, 2023.
  * Zbontar et al. (2021) Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  In _International Conference on Machine Learning_ , pp. 12310–12320. PMLR, 2021.

##  7 Appendix

###  7.1 Benchmark Datasets

To evaluate the downstream performance of our model, we conduct linear probe
and zero-shot classifications, as introduced in Sec. 3.1, on 10 widely used
datasets (Radford et al., 2021; Li et al., 2021; Yang & Mirzasoleiman, 2023)
listed in Table 4.

Table 4: Details of downstream datasets. Dataset | Classes | Train Size | Test Size
---|---|---|---
CIFAR10 | 10 | 50,000 | 10,000
CIFAR100 | 100 | 50,000 | 10,000
Food-101 | 101 | 75,750 | 25,250
DTD | 47 | 3,760 | 1,880
FGVC Aircraft | 100 | 6,667 | 3,333
Flowers-102 | 102 | 2,040 | 6,149
Caltech-101 | 102 | 3,060 | 6,085
OxfordIIITPet | 37 | 3,680 | 3,669
Stanford Cars | 196 | 8,144 | 8,041
ImageNet1K | 1000 | 50,000 | 50,000


* Leifer and Spekkens (2013) Matthew S Leifer and Robert W Spekkens.  Towards a formulation of quantum theory as a causally neutral theory of bayesian inference.  _Physical Review A_ , 88(5):052130, 2013.
  * Li and Vitányi (2019) Ming Li and Paul Vitányi.  _An introduction to Kolmogorov complexity and its applications: Fourth edition_.  Springer, 2019.
  * Linden et al. (2009) Noah Linden, Sandu Popescu, Anthony J Short, and Andreas Winter.  Quantum mechanical evolution towards thermal equilibrium.  _Physical Review E_ , 79(6):061103, 2009.
  * Loewer (2012) Barry Loewer.  Two accounts of laws and time.  _Philosophical Studies_ , 160(1):115–137, 2012\.
  * Malabarba et al. (2014) Artur SL Malabarba, Luis Pedro García-Pintos, Noah Linden, Terence C Farrelly, and Anthony J Short.  Quantum systems equilibrate rapidly for most observables.  _Physical Review E_ , 90(1):012121, 2014.
  * Maudlin (2007) Tim Maudlin.  _The Metaphysics within Physics_.  Oxford University Press, 2007.
  * Meek (1995) Chris Meek.  Strong-completeness and faithfulness in belief networks.  In _Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence_ , 1995.
  * Mishima et al. (2004) K Mishima, M Hayashi, and SH Lin.  Entanglement in scattering processes.  _Physics Letters A_ , 333(5-6):371–377, 2004\.
  * Mooij et al. (2016) Joris M Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf.  Distinguishing cause from effect using observational data: methods and benchmarks.  _The Journal of Machine Learning Research_ , 17(1):1103–1204, 2016.
  * Mora and Briegel (2005) Caterina E Mora and Hans J Briegel.  Algorithmic complexity and entanglement of quantum states.  _Physical Review Letters_ , 95(20):200503, 2005\.
  * Mora and Briegel (2006) Caterina E Mora and Hans J Briegel.  Algorithmic complexity of quantum states.  _International Journal of Quantum Information_ , 4(04):715–737, 2006.
  * Mora et al. (2007) Caterina E Mora, Hans J Briegel, and Barbara Kraus.  Quantum kolmogorov complexity and its applications.  _International Journal of Quantum Information_ , 5(05):729–750, 2007.
  * Mueller (2007) Markus Mueller.  Quantum kolmogorov complexity and the quantum turing machine.  _arXiv preprint:0712.4377_ , 2007.
  * Myrvold (2020) Wayne C Myrvold.  Explaining thermodynamics: What remains to be done?  In Valia Allori, editor, _Statistical Mechanics And Scientific Explanation: Determinism, Indeterminism And Laws Of Nature_ , pages 113–143. World Scientific, 2020.
  * Nielsen and Chuang (2010) Michael Nielsen and Isaac Chuang.  _Quantum Computation and Quantum Information: 10th Anniversary Edition_.  Cambridge University Press, 2010.
  * Oreshkov et al. (2012) Ognyan Oreshkov, Fabio Costa, and Časlav Brukner.  Quantum correlations with no causal order.  _Nature communications_ , 3(1):1–8, 2012.
  * Pearl (2009) Judea Pearl.  _Causality: 2nd Edition_.  Cambridge University Press, 2009.
  * Peschanski and Seki (2016) Robi Peschanski and Shigenori Seki.  Entanglement entropy of scattering particles.  _Physics Letters B_ , 758:89–92, 2016.
  * Peters et al. (2017) Jonas Peters, Dominik Janzing, and Bernhard Schölkopf.  _Elements of Causal Inference_.  The MIT Press, 2017.
  * Popescu et al. (2006) Sandu Popescu, Anthony J Short, and Andreas Winter.  Entanglement and the foundations of statistical mechanics.  _Nature Physics_ , 2(11):754–758, 2006.
  * Preskill (1998) John Preskill.  Lecture notes for physics 229: Quantum information and computation.  _Available at http://theory.caltech.edu/ preskill/ph229/_ , 1998.
  * Price (2007) Huw Price.  Causal perspectivalism.  In Huw Price and Richard Corry, editors, _Causation, physics, and the constitution of reality: Russell’s republic revisited_. Oxford University Press, Oxford, 2007.
  * Reimann (2008) Peter Reimann.  Foundation of statistical mechanics under experimentally realistic conditions.  _Physical review letters_ , 101(19):190403, 2008\.
  * Ried et al. (2015) Katja Ried, Megan Agnew, Lydia Vermeyden, Dominik Janzing, Robert W Spekkens, and Kevin J Resch.  A quantum advantage for inferring causal structure.  _Nature Physics_ , 11(5):414–420, 2015.
  * Roberts (2017) Bryan W Roberts.  Three myths about time reversal in quantum theory.  _Philosophy of Science_ , 84(2):315–334, 2017\.
  * Roberts (2019) Bryan W Roberts.  Time reversal.  _philsci-archive preprint:15033_ , 2019.
  * Rubino et al. (2017) Giulia Rubino, Lee A Rozema, Adrien Feix, Mateus Araújo, Jonas M Zeuner, Lorenzo M Procopio, Časlav Brukner, and Philip Walther.  Experimental verification of an indefinite causal order.  _Science advances_ , 3(3):e1602589, 2017.
  * Rudin (1987) Walter Rudin.  _Real and Complex Analysis_.  McGraw-Hill, 1987.
  * Russell (1912) Bertrand Russell.  On the notion of cause.  _Proceedings of the Aristotelian society_ , 13:1–26, 1912\.
  * Sachs (1987) Robert G Sachs.  _The physics of time reversal_.  University of Chicago Press, 1987.
  * Sakurai and Napolitano (2011) J.J. Sakurai and Jim Napolitano.  _Quantum Mechanics: Second Edition_.  Addison-Wesley, 2011.
  * Sanpera et al. (2001) Anna Sanpera, Dagmar Bruß, and Maciej Lewenstein.  Schmidt-number witnesses and bound entanglement.  _Physical Review A_ , 63(5):050301, 2001.
  * Schmid et al. (2020) David Schmid, John H Selby, and Robert W Spekkens.  Unscrambling the omelette of causation and inference: The framework of causal-inferential theories.  _arXiv preprint arXiv:2009.03297_ , 2020.
  * Schroeder (2017) Daniel V Schroeder.  Entanglement isn’t just for spin.  _American Journal of Physics_ , 85(11):812–820, 2017.
  * Short and Farrelly (2012) Anthony J Short and Terence C Farrelly.  Quantum equilibration in finite time.  _New Journal of Physics_ , 14(1):013063, 2012\.
  * Sklar (1993) Lawrence Sklar.  _Physics and Chance_.  Cambridge University Press, 1993.
  * Sperling and Vogel (2011) J Sperling and W Vogel.  The schmidt number as a universal entanglement measure.  _Physica Scripta_ , 83(4):045002, 2011.
  * Spirtes et al. (2000) Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman.  _Causation, prediction, and search_.  The MIT Press, 2000.
  * Struyve (2020) Ward Struyve.  Time-reversal invariance and ontology.  _philsci-archive preprint:17682_ , 2020.
  * Terhal and Horodecki (2000) Barbara M Terhal and Paweł Horodecki.  Schmidt number for density matrices.  _Physical Review A_ , 61(4):040301, 2000.
  * Thompson et al. (2018) Jayne Thompson, Andrew JP Garner, John R Mahoney, James P Crutchfield, Vlatko Vedral, and Mile Gu.  Causal asymmetry in a quantum world.  _Physical Review X_ , 8(3):031013, 2018.
  * Timpson (2013) Christopher G Timpson.  _Quantum information theory and the foundations of quantum mechanics_.  OUP Oxford, 2013.
  * Van den Nest (2013) Maarten Van den Nest.  Universal quantum computation with little entanglement.  _Physical Review Letters_ , 110(6):060504, 2013\.
  * Vitányi (2001) Paul MB Vitányi.  Quantum kolmogorov complexity based on classical descriptions.  _IEEE Transactions on Information Theory_ , 47(6):2464–2479, 2001.
  * Wallace (2012) David Wallace.  _The Emergent Multiverse: Quantum theory according to the Everett interpretation_.  Oxford University Press, 2012.
  * Wallace (2015) David Wallace.  Recurrence theorems: a unified account.  _Journal of Mathematical Physics_ , 56(2):022105, 2015.
  * Weinberger (2018) Naftali Weinberger.  Faithfulness, coordination and causal coincidences.  _Erkenntnis_ , 83(2):113–133, 2018.
  * Wood and Spekkens (2015) Christopher J Wood and Robert W Spekkens.  The lesson of causal discovery algorithms for quantum correlations: Causal explanations of bell-inequality violations require fine-tuning.  _New Journal of Physics_ , 17(3):033002, 2015\.
  * Woodward (2005) James Woodward.  _Making things happen: A theory of causal explanation_.  Oxford university press, 2005.
  * Woodward (2014) James Woodward.  A functional account of causation; or, a defense of the legitimacy of causal thinking by reference to the only standard that matters—usefulness (as opposed to metaphysics or agreement with intuitive judgment).  _Philosophy of Science_ , 81(5):691–713, 2014\.
  * Woodward (2020) James Woodward.  Flagpoles anyone? causal and explanatory asymmetries.  _philsci-archive preprint:17419_ , 2020.
  * Woodward (2007) Jim Woodward.  Causation with a human face.  In Huw Price and Richard Corry, editors, _Causation, Physics, and the Constitution of Reality: Russell’s Republic Revisited_ , pages 66–105. Oxford University Press, Oxford, 2007.
  * Zanardi et al. (2004) Paolo Zanardi, Daniel A Lidar, and Seth Lloyd.  Quantum tensor product structures are observable induced.  _Physical review letters_ , 92(6):060402, 2004\.
  * Zurek (1989) Wojciech H Zurek.  Algorithmic randomness and physical entropy.  _Physical Review A_ , 40(8):4731, 1989.
  * Życzkowski et al. (1998) Karol Życzkowski, Paweł Horodecki, Anna Sanpera, and Maciej Lewenstein.  Volume of the set of separable states.  _Physical Review A_ , 58(2):883, 1998.


# Training Stronger Spiking Neural Networks with Biomimetic Adaptive Internal
Association Neurons

###### Abstract

As the third generation of neural networks, spiking neural networks (SNNs) are
dedicated to exploring more insightful neural mechanisms to achieve near-
biological intelligence. Intuitively, biomimetic mechanisms are crucial to
understanding and improving SNNs. For example, the associative long-term
potentiation (ALTP) phenomenon suggests that in addition to learning
mechanisms between neurons, there are associative effects within neurons.
However, most existing methods only focus on the former and lack exploration
of the internal association effects. In this paper, we propose a novel
Adaptive Internal Association (AIA) neuron model to establish previously
ignored influences within neurons. Consistent with the ALTP phenomenon, the
AIA neuron model is adaptive to input stimuli, and internal associative
learning occurs only when both dendrites are stimulated at the same time. In
addition, we employ weighted weights to measure internal associations and
introduce intermediate caches to reduce the volatility of associations.
Extensive experiments on prevailing neuromorphic datasets show that the
proposed method can potentiate or depress the firing of spikes more
specifically, resulting in better performance with fewer spikes. It is worth
noting that without adding any parameters at inference, the AIA model achieves
state-of-the-art performance on DVS-CIFAR10 (83.9%) and N-CARS (95.64%)
datasets.

Index Terms—  Adaptive Internal Association Neuron, Spiking Neural Networks,
Bionic Learning, Neuromorphic Data

##  1 Introduction

Inspired by the learning mechanisms of the mammalian brain, spiking neural
networks (SNNs) are considered a promising model for artificial intelligence
(AI) and theoretical neuroscience [1]. In theory, as the third generation of
neural networks, SNNs are computationally more powerful than traditional
artificial neural networks (ANNs) [2, 3].

Essentially, SNNs are dedicated to exploring more insightful neural mechanisms
to achieve near-biological intelligence. The most representative ones are the
leaky integrate-and-fire (LIF) model [4, 5] and spike-timing-dependent
plasticity (STDP) rules [1]. The LIF neuron model is a trade-off between
biomimicry and computability. It can reflect most properties of biological
neurons, while the calculation is relatively simple. STDP is a temporally
asymmetric form of Hebbian learning that arises from the close temporal
correlation between the spikes of two neurons, presynaptic and postsynaptic
[3]. Both of them inspire us to understand and improve SNNs from biological
mechanisms.

Fig. 1: The associative long-term potentiation phenomenon in hippocampal
slices of rats. (A) Two different groups of Schaffer collateral presynaptic
fibers (stimulation I and stimulation II) are stimulated while performing
intracellular recording of CA1 (cornu Ammonis) pyramidal cells. (B) The
average response of CA1 pyramidal cells to a stimulus from point II is
observed. The curve depicts the membrane voltage change after point I and
point II are stimulated separately and simultaneously for ten minutes under
control conditions [6].

The interesting associative long-term potentiation (ALTP) phenomenon [6] has
recently attracted our attention, as shown in Fig. 1 (A). When points $I$ and
$II$ were stimulated at the same time, the response of point $II$ was
enhanced, but not when stimulated separately. The ALTP phenomenon suggests
that in addition to learning mechanisms between neurons, there are associative
effects within neurons.

However, most biologically inspired methods mainly focus on the learning
between neurons and lack the exploration of associative effects within
neurons. For example, spatiotemporal backpropagation based method [7] with
surrogate gradient, time surfaces based method [8] to process the temporal
information of SNNs, membrane potential based method [5] to rectify the
distribution, membrane time constant based method [9] to avoid manual
adjustment of neuron parameters, axonal delay based method [10] to simulate a
short term memory, and other novel method [11, 4].

In this paper, we propose an adaptive internal associative (AIA) neuron model
to establish neglected associations within neurons. Consistent with the ALTP
phenomenon, the AIA neuron model is adaptive to input stimuli, and internal
associative learning occurs only when both dendrites are stimulated
simultaneously. Furthermore, we measure the strength of internal associations
by weighting gradients passed between neurons with input weights. Therefore,
the effect is positively correlated with input weights, as described in Eq 7.
In addition, we introduce intermediate caching to reduce the volatility of
associations. Extensive experiments demonstrate that the AIA neuron model can
potentiate or depress the firing of spikes more specifically. This biomimetic
model performs better with fewer spikes, which is closer to the efficiency of
the brain. It is worth noting that without adding any parameters at inference,
the AIA model achieves state-of-the-art performance on DVS-CIFAR10 (83.9%) and
N-CARS (95.64%) datasets.

The closest works to the AIA model are the learning rules of Hebbian and STDP
[1]. However, they both only focus on the learning mechanism between neurons,
while AIA establishes the associations inside neurons.

##  2 Method

###  2.1 LIF Neuron Model

For spiking neurons, the neuronic membrane potential increases with the
accumulation of weighted spikes, and an output spike is generated once the
membrane potential exceeds a threshold. The membrane potential of widely used
LIF model [4] is formulated as:

$\displaystyle\tau_{m}\frac{du}{dt}=-\left(u-V_{rest}\right)+R_{m}\cdot
I(t),\quad u<V_{th}$ (1) $\displaystyle\left.\begin{aligned} u=V_{rest}&\\\
o=\delta(t-t_{i})&\end{aligned}\right\\},\quad u\geq V_{th}$

where $\tau_{m}=R_{m}C_{m}$ is the membrane time constant, $R_{m}$ and $C_{m}$
are resistance constant and capacitance constant, respectively. $u$ is the
membrane potential, $I(t)$ is the input current, $V_{th}$ and $V_{rest}$ are
the spiking threshold and resting potential. Once $u$ reaches $V_{th}$ at
$t_{i}$, a spike is generated and $u$ is reset to $V_{rest}$, which is usually
taken as 0. The output spike $o$ is described by the Dirac delta function
$\delta(x)$. The input will be summed to $I(t)$ by the dendrite response
weight $w$. By absorbing the $1-dt/\tau_{m}$ and $1/C_{m}$ constants into
response weights $w$ and leakage coefficient $\lambda$ respectively [7], the
discrete form of Eq. 1 is described as:

$\displaystyle u_{i}^{n+1,t+1}=$ $\displaystyle\lambda
u_{i}^{n+1,t}(1-o_{i}^{n+1,t})+x_{i}^{n+1,t+1}$ (2) $\displaystyle
x_{i}^{n+1,t+1}=$ $\displaystyle\sum_{j}w_{ij}^{n+1}o_{j}^{n,t+1}$
$\displaystyle o_{i}^{n+1,t+1}=$ $\displaystyle H(u_{i}^{n+1,t+1}-V_{th})$

where n denotes the $n$-th layer and $w_{ij}$ is the synaptic weight from the
$j$-th neuron in pre-layer $n$ to the $i$-th neuron in the post-layer $n+1$.
$x_{i}$ denotes the weighted input, and $H(x)$ is the Heaviside step function,
whose gradient is given by the surrogate method [7].

Fig. 2: The network Framework. Data and gradients flow along space and time,
and $\Delta w^{AIA}$ is determined by the input weights.

###  2.2 Adaptive Internal Association Neuron Model

#### Motivation

The phenomenon of associative long-term potentiation [6] in Fig. 1 and the
associated long-term depressive phenomenon [12] illustrates that there is an
associative effect within neurons when stimulated at the same time. And the
effect is positively correlated with input weights. These observations inspire
us to explore the corresponding associative learning mechanisms within
neurons.

#### AIA Neuron Model

As shown in Fig. 1, between different input synapses within a single neuron,
associative learning occurs only when both input synapses are stimulated.
Since the biological mechanism of the ALTP phenomenon is unclear, we
hypothesized that a function $f(x)$ acting on dendrites could establish
stimulus-adaptive associative learning. We will discuss the form of this
function in Sec. 2.3. Adding the associative learning term $f(x)$ to Eq. 2,
the adaptive internal associative (AIA) neuron model can be formulated as:

$\displaystyle u_{i}^{n+1,t+1}=$ $\displaystyle\lambda
u_{i}^{n+1,t}(1-o_{i}^{n+1,t})+f(x_{i}^{n+1,t+1})$ (3) $\displaystyle
x_{i}^{n+1,t+1}=$ $\displaystyle\sum_{j}w_{ij}^{n+1}o_{j}^{n,t+1}$
$\displaystyle o_{i}^{n+1,t+1}=$ $\displaystyle H(u_{i}^{n+1,t+1}-V_{th})$

###  2.3 Derivation of Gradients

For the LIF model, the gradient of the weights is given by:

$\displaystyle\Delta w_{ij}^{LIF}$ $\displaystyle=\frac{\partial L}{\partial
w_{ij}^{n+1}}=\frac{\partial L}{\partial u_{i}^{n+1,t+1}}\frac{\partial
u_{i}^{n+1,t+1}}{\partial x_{i}^{n+1,t+1}}\frac{\partial
x_{i}^{n+1,t+1}}{\partial w_{ij}^{n+1}}$ (4) $\displaystyle=\frac{\partial
L}{\partial u_{i}^{n+1,t+1}}o_{j}^{n,t+1}$

where L is the loss function. For the AIA neuron model, the gradient of the
weights is given by:

$\displaystyle\Delta w_{ij}^{AIA}=\frac{\partial L}{\partial w_{ij}^{n+1}}$
$\displaystyle=\frac{\partial f(x_{i}^{n+1,t+1})}{\partial
x_{i}^{n+1,t+1}}\frac{\partial L}{\partial u_{i}^{n+1,t+1}}o_{j}^{n,t+1}$ (5)

Combining the observations in motivation, we can set the function
$f^{\prime}(x)=x$ to construct a stimulus-adaptive internal associative neuron
model. The $\Delta w_{ij}^{AIA}$ is formulated as:

$\displaystyle\Delta w_{ij}^{AIA}$
$\displaystyle=\left(\sum_{k}w_{ik}^{n+1}o_{k}^{n,t+1}\right)\frac{\partial
L}{\partial u_{i}^{n+1,t+1}}o_{j}^{n,t+1}$ (6)

where $k$ represents the neuron connected to neuron $i$. Note that
$o\in\\{0,1\\}$, Eq. 6 can be expressed as:

$\displaystyle\Delta w_{ij}^{AIA}$
$\displaystyle=o_{j}^{n,t+1}*\sum_{k}o_{k}^{n,t+1}w_{ik}^{n+1}\Delta
w_{ik}^{LIF}$ (7)

When the neuron $j$ is in the resting state($o_{j}=0$), the weight $w_{ij}$
remains unchanged. When neuron $j$ inputs a stimulus ($o_{j}=1$), all neurons
that simultaneously input a stimulus ($o_{k}=1$) affect $w_{ij}$. In other
words, only when neuron $i$ is stimulated by both the $j$-th and $k$-th
neurons will the corresponding associative effects be established within the
neuron.

In addition, Eq. 7 measures the association influence with minimum cost by
multiplexing the weights and the gradients between neurons. It turns out that
the greater the synaptic response to the stimulus, the stronger the effect of
associative learning, which is also consistent with the observations. Fig. 2
shows the network framework, and Eq. 3 and Eq. 7 represent computations in the
forward and backward respectively.

###  2.4 Intermediate Cache

In practice, the gradient swings between $0$ and $w_{ik}^{n+1}\Delta
w_{ik}^{LIF}$, which may cause the problem of “dead neurons”, so we introduce
an intermediate cache variable $\beta$. Specifically, let
$f^{\prime}(x)=\beta$, the gradients of $\Delta w_{ij}^{AIA}$ and $\beta$ are:

$\Delta w_{ij}^{AIA}=\frac{\partial L}{\partial
w_{ij}^{n+1}}=o_{j}^{n,t+1}*\beta\Delta w_{ij}^{LIF}$ (8) $\frac{\partial
L}{\partial\beta}=\sum_{k}o_{k}^{n,t+1}w_{ik}^{n+1}\Delta w_{ik}^{LIF}$ (9)

It can be seen that, after adding the intermediate cache, $\beta$ accumulates
gradients in Eq. 7 overall stimulated synapses and applies this association
effect only to stimulated synapses. It is worth noting that the intermediate
cache $\beta$, as an auxiliary tool for saving gradients, can be merged into
$w$ for calculation at inference time, so it will not bring additional
parameters.

Table 1: Performance of AIA neuron model and the SOTAs on CIFAR10-DVS, N-Caltech101 and N-CARS datasets.  Network | Method | Reference | Model | CIFAR10-DVS | N-Caltech101 | N-CARS
---|---|---|---|---|---|---
CNNs-based | RG-CNNs [13] | TIP 2020 | Graph-CNN | 54.00 | 61.70 | 91.40
SlideGCN [14] | ICCV 2021 | Graph-CNN | 68.00 | 76.10 | 93.10
ECSNet [15] | T-CSVT 2022 | LEE$\rightarrow$MA | 72.70 | 69.30 | 94.60
SNNs-based | HATS[8] | CVPR 2018 | HATS-SVM | 52.40 | 64.20 | 81.0
Dart[16] | TPAMI 2020 | SPM-SVM | 65.80 | 66.80 | -
STBP [7] | AAAI 2021 | Resnet-19 | 67.80 | - | -
PLIF [9] | ICCV 2021 | 7-layer | 74.80 | - | -
Dspike [17] | NeurIPS 2021 | ResNet-18 | 75.40 | - | -
AutoSNN [18] | ICML 2022 | - | 72.50 | - | -
RecDis [5] | CVPR 2022 | Resnet-19 | 72.42 | - | -
DSR [4] | CVPR 2022 | VGG-11 | 77.27 | - | -
NDA [19] | ECCV 2022 | VGG-11 | 81.70 | 78.20 | 90.1
AIA | - | VGG-9 | 83.90 | 80.07 | 95.64

##  3 Experiments

Extensive experiments are conducted to demonstrate the superiority of the AIA
neuron model. For a fair comparison, a LIF neuron model with the same
parameters is used as the baseline. Specifically, the weights are initialized
by kaiming distribution. The threshold and leakage coefficients $\lambda$ of
neurons are 1 and 0.5 respectively. Adam optimizer is introduced to adjust the
learning rate, which is initially set to $1\times 10^{-3}$.

###  3.1 Comparison with the State-of-the-Art

As shown in Tab. 1, we surpass previous state-of-the-art methods on DVS-
CIFAR10 [20], N-Caltech101 [21], N-CARS [8] neuromorphic datasets. DVS-CIFAR10
contains 10,000 samples, carrying noise and blur caused by event cameras. We
resize the samples to $64\times 64$ and achieve 83.9% accuracy. It is worth
noting that NDA is the best performer in the previous work, but its sample
size is $128\times 128$. To the best of our knowledge, the AIA neuron model
achieves state-of-the-art performance on DVS-CIFAR10 dataset among all SNNs.
N-Caltech101 is a spiking version of the original frame-based Caltech101
dataset. N-CARS is a large real-world event-based car classification dataset
extracted from various driving courses. On these two datasets, we resize the
samples to $48\times 48$, reaching the accuracy of 80.07% and 95.64%
respectively. In conclusion, the AIA model achieves leading performance by
establishing associations within neurons, which in turn illustrates the
effectiveness of biomimetic internal associations.

###  3.2 Analysis of AIA Neuron Model

Table 2: The performance of different neuron models on N-Caltech101 datasets and CIFAR10-DVS datasets. Neurons | N-Caltech101(%) | CIFAR10-DVS(%)
---|---|---
LIF | 78.13 | 82.5
IF | 75.94 | 81.9
PLIF | 75.58 | 79.0
AIA | 79.59 | 83.5
Cached AIA | 80.07 | 83.9

#### Compared with other neuron models

Ablation experiments are performed on CIFAR10-DVS and N-Caltech101 datasets to
further compare the effects of AIA and other neuronal models. We employ a
publicly available implementation of PLIF neurons [9], which sets the membrane
time constant as a trainable variable. As shown in Tab. 2, the AIA neuron
model achieves the accuracy of 79.59% and 83.5%, surpassing the most commonly
used neuron models under the same parameters. In addition, the intermediate
cache also improves the AIA model (Cached AIA).

#### Insightful variations of the AIA model

To gain further insight into the workings of AIA neurons, we perform some
analysis on the N-Caltech101 dataset. As shown in Fig. 3 (a), we calculate the
weight distribution of the AIA model and the LIF model respectively, and
obtain the normalized changes in different intervals. Red indicates that the
number of weights in the interval is increasing, and blue indicates that it is
decreasing. It turns out that the weights escape from intervals close to 0,
implying a larger change in membrane voltage. In other words, AIA neurons can
more effectively potentiate or depress the firing of spikes, resulting in
better extraction of key features.

In addition, the change in the number of spikes is shown in Fig. 3 (b). The
AIA model significantly affects the number of fired spikes, especially in
feature extraction layers. Notably, the number of fired spikes is
significantly reduced overall, implying that the AIA neuron model performs
better with fewer spikes. It indicates that the biomimetic internal
association mechanism inherently improves the efficiency and biological
plausibility of the model.

(a) Weight distribution.

(b) Number of spikes.

Fig. 3: Variations in weight distribution and number of spikes after using AIA
neurons compared to LIF neurons.

##  4 Conclusion

Inspired by the ALTP phenomenon, we propose a novel adaptive internal
association (AIA) neuron model. This biomimetic model achieves state-of-the-
art performance on neuromorphic datasets. In addition, the intermediate cache
is introduced to reduce volatility. The insightful analysis shows that AIA
neurons can more effectively potentiate or depress the firing of spikes and
perform better with fewer spikes. And the proposed neuron model can provide
the basis for more efficient and biologically plausible spiking neural
networks.

## References

  * [1] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda,  “Towards spike-based machine intelligence with neuromorphic computing,”  Nature, vol. 575, 2019.
  * [2] Wolfgang Maass,  “Networks of spiking neurons: The third generation of neural network models,”  Neural Networks, vol. 10, no. 9, pp. 1659–1671, 1997.
  * [3] Duzhen Zhang, Tielin Zhang, Shuncheng Jia, Qingyu Wang, and Bo Xu,  “Recent advances and new frontiers in spiking neural networks,”  in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, Luc De Raedt, Ed. 2022, pp. 5670–5677, ijcai.org.
  * [4] Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo,  “Training high-performance low-latency spiking neural networks by differentiation on spike representation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12444–12453.
  * [5] Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang,  “Recdis-snn: Rectifying membrane potential distribution for directly training spiking neural networks,”  in CVPR, 2022, pp. 326–335.
  * [6] G Barrionuevo and T H Brown,  “Associative long-term potentiation in hippocampal slices,”  Proceedings of the National Academy of Sciences, vol. 80, no. 23, pp. 7347–7351, 1983.
  * [7] Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li,  “Going deeper with directly-trained larger spiking neural networks,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 11062–11070.
  * [8] Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman,  “Hats: Histograms of averaged time surfaces for robust event-based object classification,”  in CVPR, June 2018.
  * [9] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian,  “Incorporating learnable membrane time constant to enhance learning of spiking neural networks,”  in ICCV, 2021, pp. 2661–2671.
  * [10] Pengfei Sun, Longwei Zhu, and Dick Botteldooren,  “Axonal delay as a short-term memory for feed forward deep spiking neural networks,”  in IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23-27 May 2022. 2022, pp. 8932–8936, IEEE.
  * [11] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Bo Xu,  “Brain-inspired balanced tuning for spiking neural networks,”  in IJCAI, 7 2018, pp. 1653–1659.
  * [12] Patric K. Stanton and Terrence J. Sejnowski,  “Associative long-term depression in the hippocampus induced by hebbian covariance,”  Nature, vol. 339, 1989.
  * [13] Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos,  “Graph-based spatio-temporal feature learning for neuromorphic vision sensing,”  IEEE Trans. Image Process., vol. 29, pp. 9084–9098, 2020.
  * [14] Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang,  “Graph-based asynchronous event processing for rapid object recognition,”  in 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. 2021, pp. 914–923, IEEE.
  * [15] Zhiwen Chen, Jinjian Wu, Junhui Hou, Leida Li, Weisheng Dong, and Guangming Shi,  “Ecsnet: Spatio-temporal feature learning for event camera,”  IEEE Transactions on Circuits and Systems for Video Technology, pp. 1–1, 2022.
  * [16] Bharath Ramesh, Hong Yang, Garrick Orchard, and et al.,  “Dart: Distribution aware retinal transform for event-based cameras,”  TPAMI, vol. 42, no. 11, pp. 2767–2780, 2020.
  * [17] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu,  “Differentiable spike: Rethinking gradient-descent for training spiking neural networks,”  Advances in Neural Information Processing Systems, vol. 34, pp. 23426–23439, 2021.
  * [18] Byunggook Na, Jisoo Mok, Seongsik Park, Dongjin Lee, Hyeokjun Choe, and Sungroh Yoon,  “Autosnn: Towards energy-efficient spiking neural networks,”  arXiv preprint arXiv:2201.12738, 2022.
  * [19] Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda,  “Neuromorphic data augmentation for training spiking neural networks,”  arXiv preprint arXiv:2203.06145, 2022.
  * [20] Hongmin Li, Hanchao Liu, Xiangyang Ji, and et al.,  “Cifar10-dvs: An event-stream dataset for object classification,”  Frontiers in Neuroscience, vol. 11, pp. 309, 2017.
  * [21] Garrick Orchard, Ajinkya Jayawant, Gregory Cohen, and Nitish V. Thakor,  “Converting static image datasets to spiking neuromorphic datasets using saccades,”  CoRR, vol. abs/1507.07629, 2015.