# Class Gradient Projection For Continual Learning

Cheng Chen  Center for Future Media, University of Electronic Science and
Technology of ChinaChengduChina<EMAIL_ADDRESS>,  Ji Zhang  Center for
Future Media, University of Electronic Science and Technology of
ChinaChengduChina<EMAIL_ADDRESS>,  Jingkuan Song1,2 1Shenzhen
Institute for Advanced Study, University of Electronic Science and Technology
of China
2Peng Cheng LaboratoryShenzhenChina<EMAIL_ADDRESS>and  Lianli Gao
Shenzhen Institute for Advanced Study, University of Electronic Science and
Technology of ChinaShenzhenChina<EMAIL_ADDRESS>
(2022)

###### Abstract.

Catastrophic forgetting is one of the most critical challenges in Continual
Learning (CL). Recent approaches tackle this problem by projecting the
gradient update orthogonal to the gradient subspace of existing tasks. While
the results are remarkable, those approaches ignore the fact that these
calculated gradients are not guaranteed to be orthogonal to the gradient
subspace of each class due to the class deviation in tasks, e.g.,
distinguishing “Man” from “Sea” v.s. differentiating “Boy” from “Girl”.
Therefore, this strategy may still cause catastrophic forgetting for some
classes. In this paper, we propose Class Gradient Projection (CGP), which
calculates the gradient subspace from individual classes rather than tasks.
Gradient update orthogonal to the gradient subspace of existing classes can be
effectively utilized to minimize interference from other classes. To improve
the generalization and efficiency, we further design a Base Refining (BR)
algorithm to combine similar classes and refine class bases dynamically.
Moreover, we leverage a contrastive learning method to improve the model’s
ability to handle unseen tasks. Extensive experiments on benchmark datasets
demonstrate the effectiveness of our proposed approach. It improves the
previous methods by 2.0% on the CIFAR-100 dataset. The code is available at
https://github.com/zackschen/CGP.

Continual learning, Gradient projection, Lifelong learning, Contrastive
learning.

††journalyear: 2022††copyright: acmcopyright††conference: Proceedings of the
30th ACM International Conference on Multimedia; October 10–14, 2022; Lisboa,
Portugal.††booktitle: Proceedings of the 30th ACM International Conference on
Multimedia (MM ’22), Oct. 10–14, 2022, Lisboa, Portugal††price: 15.00††isbn:
978-1-4503-9203-7/22/10††doi: 10.1145/3503161.3548054††submissionid:
1278††ccs: Computing methodologies Image representations

##  1\. Introduction

Figure 1. (a) In the standard gradient projection methods for CL, the actual
parameters update is $Final\nabla_{W}L$, which ensures that the network does
not forget the knowledge learned from the previous task. (b) The class
deviation can simultaneously affect the bases calculation and the gradient
projection, as a consequence, causing catastrophic forgetting.

The learn update projection.

Although the modern deep learning network (DNN) has shown good performance on
the task at hand (Cai et al., 2020; Wang et al., 2020; Zhang et al., 2022;
Deng and Liu, 2020; Li et al., 2020a; Zhang et al., 2021), they always
violently forget the previous knowledge when they learn new tasks. Such a
phenomenon is known as catastrophic forgetting, which is the research hotspot
in Continual Learning (CL) (McCloskey and Cohen, 1989; Ratcliff, 1990; Delange
et al., 2021). In the training phase of CL, the model needs to learn on a
stream of tasks sequentially. This requires the network to have plasticity,
that is, the ability to learn fresh knowledge from unseen tasks. At the same
time, it is critical for the network to retain previously learned knowledge,
i.e., the stability ability. However, “you can’t have your cake and eat it
too”: the dilemma of stability-plasticity is what the continual learning
networks need to solve primarily.

Many approaches have been proposed to deal with this dilemma, including i)
regularization-based methods (Kirkpatrick et al., 2017; Aljundi et al., 2018),
ii) memory-based methods (Lopez-Paz and Ranzato, 2017; Chaudhry et al., 2018),
and iii) expansion-based methods (Abati et al., 2020; Li et al., 2019). To be
more specific, regularization-based methods focus on penalizing the
modification of the most important weights of the network so that the previous
knowledge can be preserved. To maintain the performance on previous tasks,
memory-based approaches replay the data of old tasks or the synthetic data
from generative models to jointly train with present task samples. In
contrast, expansion-based methods seek to expand the architecture of the
network to learn new knowledge.

Figure 2. An overview of our Class Gradient Projection (CGP) network. First,
to alleviate the class deviation, CGP constructs the gradient subspace for
each class with a base refining module. Second, CGP projects the gradient of
the new task orthogonal to the constructed gradient subspace. Moreover, we
develop a contrastive loss to guide continuous optimization of the model when
its gradient is projected by more and more constructed gradient subspace.

Framework

Recently, the gradient projection based paradigms (Farajtabar et al., 2020;
Zeng et al., 2019; Saha et al., 2021) have achieved remarkable performance in
tackling the catastrophic forgetting problem in CL. Existing gradient
projection based methods put explicit constraints on the gradient directions
that the optimizer takes. For example, GPM (Saha et al., 2021) computes the
bases of these gradient subspace based on the representations learned by each
learning task. In the learning phase of the next task, new gradient steps in
the orthogonal direction to these gradient subspace would be taken. A simple
illustration is shown in Fig. 1 (a). Despite the impressive progress, there is
a critical problem for those gradient projection based methods. Considering a
simple scenario where the task at hand is to classify the given image as “Man”
or “Sea”. The class deviation between them is significantly large. In other
words, their gradient subspace could be so different from each other.
Therefore, if we directly perform Singular Value Decomposition (SVD) (Lee et
al., 2018) on the representations from all classes of a task, some bases for
“apple” or “car” may be missed. When the gradient projection is performed in
the subsequent tasks, the projected gradient is greatly disturbed; as a
consequence, the model forgets the knowledge learned previously, as
illustrated in Fig. 1 (b).

In this paper, we propose Class Gradient Projection (CGP) to address the side-
effect of class deviation in gradient projection. CGP calculates the gradient
subspace with the representations from individual classes rather than tasks,
such that the gradient update steps orthogonal to the constructed classes
subspace can be effectively utilized to minimize the interference from other
classes. Based on this framework, two effective variants are developed to
further alleviate catastrophic forgetting. Specifically, to construct a more
informative gradient subspace for each incoming task, we design a Base
Refining (BR) algorithm to refine class bases dynamically. BR adopts an inter-
class similarity estimation module to guide the partition of classes into the
built bases. In addition, considering that the update steps are orthogonal to
the gradient subspace of previous classes, the model may lack optimization
space for unseen classes. Therefore, we introduce a supervised contrastive
loss to learn more representative and robust features by pulling embedding
from positive samples closer and pushing embedding from negative samples
apart. The contrastive learning can guide continuous optimization of the model
when its gradient is projected by more and more classes subspace. The designed
two variants can complement each other to enhance the plasticity and stability
of the model simultaneously. In summary, the main contributions can be
summarized as follows:

  * •

We propose Class Gradient Projection (CGP), which effectively utilizes the
gradient update steps orthogonal to the constructed class subspace to minimize
the negative interference between classes.

  * •

Based on the CGP framework, we introduce a Base Refining (BR) algorithm as
well as a contrastive loss to further alleviate catastrophic forgetting. The
two components can complement each other to enhance the plasticity and
stability of the model simultaneously.

  * •

We conduct extensive experiments on several benchmark datasets and various
network architectures. The achieved results demonstrate the effectiveness of
our method.

##  2\. Related Work

###  2.1. Continual Learning

Prevalent approaches to address the catastrophic forgetting problem in
continual learning can be grouped into three broad categories: regularization-
based approaches (Aljundi et al., 2018; Zenke et al., 2017; Nguyen et al.,
2017; Lee et al., 2017), expansion-based approaches (Rusu et al., 2016; Veniat
et al., 2021; Hung et al., 2019; Li et al., 2019; Rajasegaran et al., 2019;
Serra et al., 2018), and memory-based approaches (Ayub and Wagner, 2021;
Lopez-Paz and Ranzato, 2017; Sprechmann et al., 2018; Rebuffi et al., 2017).

Regularization-based methods aim at avoiding excessive changes in the
parameters learned on old tasks when learning a new task. Typically, these
methods (Aljundi et al., 2018; Zenke et al., 2017) estimate importance weights
for each model parameter. Then the changes of the important parameters are
penalized by a regularizer for previous tasks. Under the Bayesian framework,
VCL and IMM (Nguyen et al., 2017; Lee et al., 2017) take the posterior
distribution of network parameters learned from previous tasks as the prior
distribution of network parameters on the current task, which implicitly
penalizes the changes of network parameters under the Bayesian framework.
However, our method works in putting constraints on gradient descent rather
than ascribing importance to parameters and penalizing the changes.

The basic idea of expansion-based approaches is to directly add or modify the
model structure. Some methods (e.g. (Rusu et al., 2016)) add a network to each
task and lateral connections to the network of the previous task. MNTDP
(Veniat et al., 2021) proposes a modular layer network approach, whose modules
represent atomic skills that can be composed to perform a certain task and
provides a learning algorithm to search the modules to combine with. These
strategies may work well, but they are computationally expensive and memory
intensive. Finally, other approaches (Rajasegaran et al., 2019; Serra et al.,
2018) provide an alternative approach that assigns the different sub-networks
or weights. In contrast to these methods, our method works within a fixed
network architecture.

For memory-based methods, catastrophic forgetting is avoided by storing data
from previous tasks and training them together with data from the current
task. Some methods (Lopez-Paz and Ranzato, 2017; Sprechmann et al., 2018;
Rebuffi et al., 2017) use replayed samples from previous tasks to constrain
the parameters’ update when learning the new task. They do not need to store
data from previous tasks, but their performance is significantly affected by
the quality of generated data, especially for complex natural images. Instead
of saving the raw data or generated data, our method saves the subspace bases
for future task learning, avoiding the privacy problem.

Recently, a series of continual learning methods combined with orthogonal
projection have been proposed (Farajtabar et al., 2020; Saha et al., 2021).
Gradient projection methods update the model with gradients in the orthogonal
directions of old tasks, without access to old task data. GPM (Saha et al.,
2021) find the bases of these subspaces by analyzing network representations
after learning each task with Singular Value Decomposition (SVD) in a single
shot manner and store them in the memory as gradient projection memory. In
this paper, to address the class deviation in gradient projection, we propose
class gradient projection. Through such class gradient projection, the network
achieves better stability and preserve more learned knowledge.

###  2.2. Contrastive Learning

In recent years, contrastive learning has shown superior performance, even in
competition with supervised training. Supervised contrastive learning (SCL) is
extended from the standard contrastive loss by incorporating the label
information to construct positive and negative pairs (Sohn et al., 2020).
Prototypical Contrastive Learning (PCL) (Li et al., 2020b) uses the centroids
of clusters as prototypes, and pulls the image embedding closer to its
prototypes. Noise-Contrastive estimation (Gutmann and Hyvärinen, 2010) is the
seminal work that estimates the latent distribution by contrasting it with
artificial noises. In addition, CPC (van den Oord et al., 2018) tries to learn
representations from visual inputs by leveraging an auto-regressive model to
predict the future in an unsupervised manner. These studies (Chen et al.,
2020; Chen and He, 2021) have resolved practical limitations that have
previously made learning difficult such as negative sample pairs, large batch
size, and momentum encoders. Meanwhile, it has been shown that supervised
learning can also enjoy the benefits of contrastive representation learning by
simply using labels to extend the definition of positive samples (Khosla et
al., 2020). Another method (Cha et al., 2021) combines the contrastive
learning with continual learning. They use samples of the current task as
anchors and samples of previous tasks as negative samples. Different from this
method, our method does not need to replay the previous samples. Instead, we
perform augmentation on input samples. The sample and the augmented sample
make up the positive pairs. The rest samples serve as negative samples. Then
we introduce a contrastive loss to pull embedding from positive samples closer
and to push embedding from negative samples apart. This contrastive learning
encourages the network to learn more representative and robust features of
tasks.

##  3\. Preliminaries

###  3.1. Continual Learning

In the setup of supervised continual learning, a series of $T$ tasks are
learned sequentially. We denote the task by its task descriptor,
$\tau\in\\{1,2,...,T\\}$ and its corresponding dataset
$\textit{D}_{\tau}=\\{{(x_{\tau,i},\mathit{y}_{\tau,i})}_{i=1}^{N_{\tau}}\\}$
which has $n_{\tau}$ example pairs. The $x_{\tau,i}$ ($\in\textit{X}$ ) is the
input vector and $\mathit{y}_{\tau,i}$ ($\in\textit{Y}$ ) is the target
vector. A DNN model parameterized with $\Phi=\\{\mathbb{W},\varphi\\}$ is used
to learn a mapping in the form
$\hat{\mathit{y}_{\tau}}=\mathit{f_{map}}(x_{\tau};\Phi)$. Here,
$\mathbb{W}_{\tau}$ = $\\{\mathit{W^{l}_{\tau}}\\}_{l=1}^{L}$ represents a $L$
layer neural network, where $\mathit{W}^{l}_{\tau}$ is the layer-wise weight
for layer $\mathit{l}$ and task $\tau$. In each layer, the layer network
computes the output $\mathit{x_{\tau,i}^{l+1}}$ for next layer:

$\mathit{O_{\tau,i}^{l}}=\mathit{f}(x^{l}_{\tau,i};\mathit{W_{\tau}^{l}}),\quad\mathit{x_{\tau,i}^{l+1}}=\sigma_{l}(\mathit{O_{\tau,i}^{l}}),$

with $\sigma_{l}$ is a non-linear function for layer $l$ and $\mathit{f}$ is
the linear function for layer. Following (Saha et al., 2021), at the first
layer, the $\mathit{x_{\tau,i}^{1}}$ = $\mathit{x_{\tau,i}}$ represents the
raw input data from task $\tau$. Whereas in the subsequent layers we define
$\mathit{x_{\tau,i}^{l}}$ as the representations of $\mathit{x_{\tau,i}}$ at
layer $l$. The output of final neural network $\mathit{x_{\tau,i}^{L+1}}$ is
then passed through a classifier parameterized by $\varphi$ to produce the
prediction $\hat{\mathit{y}}=\mathit{f}(\mathit{x_{\tau,i}^{L+1}};\varphi)$.
The model is trained by minimizing the loss function for task $\tau$, e.g.
cross-entropy loss

$\Phi^{*}=\underset{\Phi}{\textit{minimize}}\sum_{n=1}^{N_{t}}\ell(f(\mathit{x_{\tau,i}},\Phi),\mathit{y_{\tau,i}}),$

where $\Phi^{*}$ denotes the optimal model for task $\tau$.

###  3.2. Gradient Projection

Recently, a series of continual learning methods combined with orthogonal
gradient projection have been proposed (Farajtabar et al., 2020; Saha et al.,
2021). These methods update the model with gradients in the orthogonal
directions of old tasks, without access to old task data. After learning the
task $\tau$ completely, they construct the gradient space $\mathit{S_{\tau}}$
using the samples of task $\tau$. When learning the task $\tau+1$, the
gradient of model $\mathbb{W}_{\tau+1}$ is projected to the gradient subspace
$\mathit{S}_{\tau}$ of previous tasks to get the
$\textit{Proj}_{\nabla_{\mathbb{W}_{\tau+1}L}}$. Then the
$\textit{Proj}_{\nabla_{\mathbb{W}_{\tau+1}L}}$ is substracted out from the
origin gradient $\mathbb{W}_{t+1}$ so that remaining gradient updates lie in
the space orthogonal to $S_{\tau}$ (Saha et al., 2021).

##  4\. Method

As illustrated in the preliminaries, the gradient projection approaches
construct the gradient subspace by the samples from all the classes of task
$\tau$. However, we argue that class deviation in tasks may cause the
calculated gradient space too biased to represent the task accurately, leading
to the degradation of the performance. Moreover, it is critical to explore the
optimization space for the new tasks. In this section, we propose a novel
Class Gradient Projection (CGP) for continual learning. Fig. 2 illustrates the
pipeline of our proposed continual learning approach. We are going to show how
the CGP works at a high level. On the one hand, CGP constructs the gradient
subspace with individual classes and projects the gradient update of new tasks
to the direction orthogonal to the subspace of old classes. On another hand,
CGP learns the representations with supervised contrastive learning to explore
the optimization space for the new tasks. The training is done on the compound
loss:

(1) $\mathcal{L}=\mathcal{L}_{ce}+\lambda\cdot\mathcal{L}_{con},$

where $\mathcal{L}_{ce}$ is the cross-entropy loss and $\mathcal{L}_{con}$ is
the supervised contrastive learning.

###  4.1. Class Gradient Projection

In this section, we introduce the bases construction and class gradient
projection to show how it enables the network to learn continually without
forgetting.

##### When learning Task $\tau$ = 1:

We use the training process as shown in Eq. 1 without any constraint. After
the network learns completely, the Singular Value Decomposition (SVD) is
performed on the representations to construct the bases. Specifically, for
layer $l$ in $\mathbb{W}$, we construct a representation matrix
$\mathit{R_{1}^{l}}=[x_{1,1}^{l},x_{1,2}^{l},...,x_{1,r}^{l}]$ from the
samples of a certain number $r$. We separate the matrix $\mathit{R_{1}^{l}}$
by the target label:

$\mathit{R_{1}^{l}}=[C_{1,1}^{l},C_{1,2}^{l},...,C_{1,c}^{l},...,C_{1,c_{\tau}}^{l}],$

with

$C_{1,c}^{l}=[x_{1,c,1}^{l},x_{1,c,2}^{l},...,x_{1,c,n_{c}}^{l}],$

where $c_{\tau}$ represents the class number in task $\tau$ and $n_{c_{\tau}}$
is the number of samples in class $c_{\tau}$. Next, we perform SVD on
$C_{1,1}^{l}\in\mathbb{R}^{m\times n}$:

(2)
$\mathit{U}_{1,1}^{l},\Sigma_{1,1}^{l},(\mathit{V}_{1,1}^{l})^{T}=SVD(C_{1,1}^{l}),$

where $\mathit{U}_{1,1}^{l}$ is a $m\times m$ complex unitary matrix, and
$\mathit{V}_{1,1}^{l}$ is a $n\times n$ complex unitary matrix.
$\Sigma_{1,1}^{l}$ is a $m\times n$ rectangular diagonal matrix with non-
negative singular values $\\{\delta\\}$ on the diagonal in a descending order.
By applying the k-rank approximation on $C_{1,1}^{l}$ for the given threshold
( $\epsilon^{l}_{\tau}$ ):

(3)
$\left\|(\mathit{C_{1,1}^{l}})_{k}\right\|_{F}^{2}\geq\epsilon_{\tau}^{l}\left\|\mathit{C_{1,1}^{l}}\right\|_{F}^{2},$

where $\left\|\cdot\right\|_{F}$ is the Frobenius norm, we construct the bases
$S_{1}^{l}$ by picking the first $k$ vectors in $\mathit{U}_{1,1}^{l}$.

When learning the rest classes, e.g. class $j$, we need to remove the common
bases contained in $S_{1}^{l}$ so that the bases constructed by $C_{1,j}^{l}$
are orthogonal to the existing bases in $S_{1}^{l}$:

(4)
$\hat{C}_{1,j}^{l}=C_{1,j}^{l}-S_{1}^{l}({S_{1}^{l}})^{T}(C_{1,j}^{l})=C_{1,j}^{l}-\textit{Proj}_{C_{1,j}^{l}}.$

After applying the SVD on $\hat{C}_{1,j}^{l}$, we follow the criteria to
construct the bases for class $j$:

(5)
$\left\|\textit{Proj}_{C_{1,j}^{l}}\right\|_{F}^{2}+\left\|(\mathit{\hat{C}_{1,j}^{l}})_{k}\right\|_{F}^{2}\geq\epsilon_{\tau}^{l}\left\|\mathit{\hat{C}_{1,j}^{l}}\right\|_{F}^{2}.$

After calculating all classes for task 1, we construct the bases $S_{1}^{l}$.

##### When learning Task $\tau$ ¿ 1:

We train the network using the loss function defined in Eq. 1 as usual.
However, before taking gradient updates from the backpropagation, we put
constraints on the gradient updates. We modify the origin gradients using:

(6)
$\textit{Proj}_{\nabla_{\mathbb{W}}\mathcal{L}}=({\nabla_{\mathbb{W}}\mathcal{L}})S^{l}({S^{l}})^{T}.$

Hence, the final gradient can be expressed as:

(7)
$\nabla_{\mathbb{W}}\mathcal{L}={\nabla_{\mathbb{W}}\mathcal{L}}-\textit{Proj}_{\nabla_{\mathbb{W}}\mathcal{L}}.$

After the network training converged, we construct the bases for task $\tau$.
We use the same process as the calculation in task $\tau=1$, shown in Eq. 4
and Eq. 5. After the construction, we concatenate the bases for task $\tau$,
i.e. $S_{\tau}^{l}$ and previous bases $S^{l}$ together in a new $S^{l}$ which
used for future learning. We repeat bases construction and gradient projection
until all tasks are learned.

###  4.2. CGP with Base Refining (BR)

There are many classes in a task $\tau$, as illustrated in Sec. 1. Some
classes are different from others. However, there may be some classes similar
to others too. For example, the man is different from the sea and similar to
the boy. So we introduce the base refining to combine similar classes to
construct the bases of gradient subspace. We estimate the class similarity
using the prototype of class. Concretely, after the task $\tau$ is learned, we
collect the samples $\mathit{X}_{c_{\tau}}$ of all classes. Then the class
prototype is calculated by the normalized mean embedding:

(8)
$\mathit{z}^{l}_{c_{\tau}}=\frac{1}{\left|\mathit{X}_{c_{\tau}}\right|}\sum_{i\in\mathit{X}_{c_{\tau}}}x_{i}^{l},\quad\mathit{P}^{l}_{c_{\tau}}=\frac{\mathit{z}^{l}_{c_{\tau}}}{\left\|\mathit{z}^{l}_{c_{\tau}}\right\|_{2}},$

where $x_{i}^{l}$ is the representation of sample at layer $l$. The similarity
between classes is estimated by the calculated prototype using cosine
distance:

(9)
$sim(\mathit{P}^{l}_{u},\mathit{P}^{l}_{v})=\frac{{\mathit{P}^{l}_{u}}^{T}\mathit{P}^{l}_{v}}{\left\|\mathit{P}^{l}_{u}\right\|\left\|\mathit{P}^{l}_{v}\right\|}.$

If the $sim(\mathit{P}^{l}_{u},\mathit{P}^{l}_{v})$ is greater than threshold
$\eta$, we construct the bases for both class $u$ and $v$. In contrast, if the
similarity is smaller than $\eta$, we construct the bases for class $u$ and
$v$ as before. We introduce how to combine the base refining with our base
construction in the next paragraph.

When constructing bases in $\tau=1$, as we can get the representations of
classes, the representations of class $j$ and $k$ are concatenated together,
$C_{1,(j,k)}^{l}=concat(C_{1,j}^{l},C_{1,k}^{l})$. Then, we perform the SVD on
$C_{1,(j,k)}^{l}$. Following the criteria in Eq. 3, we construct the bases
$S_{1,(j,k)}^{l}$ for both class $j$ and class $k$. Finally, we replace the
origin bases $S_{1,j}^{l}$ and $S_{1,k}^{l}$ with $S_{1,(j,k)}^{l}$. In the
construction for task $\tau>1$, we construct the common bases for class $j$
and $k$ as in (Lin et al., 2022). Firstly, we calculate the square of the
singular value of $C_{\tau,j}^{l}$ with respect to $S^{l}$ which are
constructed by previous class by:

(10) $\delta_{\tau,j}^{l}=S^{l}C_{\tau,j}^{l}(C_{\tau,j}^{l})^{T}(S^{l})^{T}.$

Then, the SVD is applied to the result of performing Eq. 4 on
$C_{\tau,j}^{l}$,
$\hat{C}_{\tau,j}^{l}=\mathit{\hat{U}}_{\tau,j}^{l}\hat{\Sigma}_{\tau,j}^{l}(\mathit{\hat{V}}_{\tau,j}^{l})^{T}$,
where $\hat{\Sigma}_{\tau,j}^{l}$ contains singular values
$\\{\hat{\delta}_{\tau,j}^{l}\\}$. Next, we concatenate the
$\\{\delta_{\tau,j}^{l}\\}$ and $\\{(\hat{\delta}_{\tau,j}^{l})^{2}\\}$
together in a vector $\delta$. By performing k-rank ( Eq. 5 ) on
$C_{\tau,j}^{l}$, we choose the corresponding bases of first $k$ elements in
$\delta$ to be the bases for class $j$ and $k$.

Furthermore, in practice, one does not need all $\mathit{X}_{c_{\tau}}$ for
calculation. Another alternative is to select the samples
$\mathit{X}_{right_{c_{\tau}}}$ which are predicting right to the ground truth
label. This alternative is referred to as BR-GTL (Ground Truth Label).
Furthermore, BR-GTL reduces the storage size and calculate consumption. We
empirically observe that BR-GTL slightly outperforms the BR-STD (Standard)
which uses samples no matter the prediction result. The BR-GTL is used in all
of our following experiments.

###  4.3. CGP with Contrastive Learning (Con)

Although class gradient projection can reserve the knowledge well, it reduces
the optimization space for learning fresh knowledge. To deal with this
problem, we introduce the contrastive learning to explore the optimization
space for new tasks. Specifically, given the $N_{\tau}$ samples from task
$\tau$, we apply augmentation to each sample and obtain 2$N_{\tau}$ inputs
$\\{x_{\tau,i}\\}_{i=1}^{2N_{\tau}}$. The augmentation consists of color and
brightness changes with details given in Sec. 5. We collect the normalized
embeddings $\\{\mathit{x_{\tau,i}^{L+1}}\\}_{i=1}^{2N_{\tau}}$ before the
classifier. The contrastive learning loss is defined as:

(11)
$\mathcal{L}_{con}=\sum_{i=1}^{N_{\tau}}-log\frac{exp(x_{i}\cdot{}x_{j(i)}/\mu)}{{\textstyle\sum_{k=1}^{2N_{\tau}}}\mathbb{1}_{i\neq
k}exp(x_{i}\cdot{}x_{k}/\mu)},$

where $x_{j(i)}$ is the augmentation input from the same source image $x_{i}$,
and $\mu$ is the scalar temperature parameter. Through this contrastive
learning, we can pull the embeddings between the pair of positive samples
$x_{i}$ and $x_{j(i)}$ closer, while pushing the embeddings with the
$2(N_{\tau}-1)$ pairs of negative samples apart. This contrastive learning
encourages the network to learn a discriminative representation that is robust
to low-level image corruption.

Algorithm 1 Algorithm for Class Gradient Projection.

1:Model $\Phi=\\{\mathbb{W},\varphi\\}$; Task sequence $T$; Class base memory
$S$; Class prototype memory $P$; Learning rate $\alpha$; Similarity threshold
$\eta$; Threshold $\epsilon$; Sample size for Base Refining $n_{r}$; Train
epoch $E$.

2:Model $\Phi=\\{\mathbb{W},\varphi\\}$; Class base memory $S$; Class
prototype memory $P$.

3:Initialize Model $\Phi$: $\Phi\leftarrow\Phi_{0}$.

4:for $\tau=0,\ldots,\left|T\right|$ do

5: for $e_{\tau}=0,\ldots,E$ do $\triangleright$ train and project

6: $B_{n}\sim D_{\tau}$ $\triangleright$ sample a mini-batch of size n from
task $\tau$.

7: $B_{2n}\sim AUGMENT(B_{n})$ $\triangleright$ augment samples

8: $\nabla_{\mathbb{W}}\mathcal{L}\leftarrow OPTIMIZER(B_{2n},\Phi)$

9: $\nabla_{\mathbb{W}}\mathcal{L}\leftarrow
PROJECT(\nabla_{\mathbb{W}}\mathcal{L},S)$ $\triangleright$ Eq. (6,7)

10: $\Phi\leftarrow\Phi-\alpha\nabla_{\mathbb{W}}\mathcal{L}$

11: end for

12: for $c=0,\ldots,c_{\tau}$ do

13: $B_{c}\sim D_{\tau}$ $\triangleright$ sample prediction right of size
$n_{r}$ for class $c$.

14: ${C_{c}}\leftarrow FORWARD(B_{c},\Phi)$ $\triangleright$ construct
representation

15: $\mathit{P}_{c}\leftarrow CALCULATE(C_{c})$ $\triangleright$ Eq. 8

16: $simi(\mathit{P}_{c},P)\leftarrow CALCULATE(\mathit{P}_{c},P)$
$\triangleright$ Eq. 9

17: $\hat{C_{c}}\leftarrow PROJECT(C_{c},S)$ $\triangleright$ Eq. 4

18: $U_{c},\Sigma_{c}\mathit{V}_{c}^{T}\leftarrow SVD(\hat{C_{c}})$
$\triangleright$ Eq. 2

19: if $Simi(\mathit{P}_{c},P)<\eta$ then

20: $k\leftarrow CRITERIA(\hat{C_{c}},C_{c},\epsilon_{\tau})$ $\triangleright$
Eq. (3,5)

21: $S\leftarrow[S,U_{c}[0:k]]$

22: else

23: $\delta_{c}\leftarrow CALCULAT(C_{c},S)$ $\triangleright$ Eq. 10

24: $k\leftarrow CRITERIA(C_{c},\epsilon_{\tau})$ $\triangleright$ Eq. 3

25: $\delta\leftarrow SORT([\delta_{c},\Sigma_{c}^{2}])$

26: $S\leftarrow[S,[U_{c},S](\delta[0:k])]$ $\triangleright$ select first k
bases corresponding to the first k elements in $\delta$

27: end if

28: $P\leftarrow[P,\mathit{P}_{c}]$

29: end for

30:end for

31:return $\Phi,S,P$

Algorithm 1 describes the details of our CGP combining with the base refining
and contrastive learning.

##  5\. Experiments

To evaluate the effectiveness of our proposed method, we first compare it with
state-of-the-art CL methods. Then, we conduct ablation studies to empirically
analyze the main components.

###  5.1. Experimental setting

Let us first describe the experimental settings on datasets, comparison
methods, evaluation metrics and implementation details.

####  5.1.1. Datasets

Following (Saha et al., 2021), we evaluate on several continual learning
benchmarks, including 10-Split CIFAR-100 (Krizhevsky et al., 2009), 20-Split
CIFAR-100 (Krizhevsky et al., 2009), 5-Split CIFAR-100 (Krizhevsky et al.,
2009), CIFAR-100 Superclass, and sequence of 5-Datasets (Ebrahimi et al.,
2020). The 10-Split CIFAR-100 is constructed by randomly splitting 100 classes
of CIFAR-100 into 10 tasks with 10 classes per task. The 20-Split CIFAR-100
consists of splitting CIFAR-100 into 20 tasks, each one with 5 classes. The
5-Split CIFAR-100 is constructed by splitting the CIFAR-100 into 5 tasks, each
task consisting of 20 classes. The 10-Split CIFAR-100 is constructed by
randomly splitting the 100 classes of CIFAR-100 into 10 tasks with 10 classes
per task. CIFAR-100 Superclass (Yoon et al., 2020) split the 100 classes of
CIFAR-100 into 20 tasks where each task has 5 different but semantically
related classes. Moreover, we also use the 5-Datasets which consists of
CIFAR-10, MNIST, SVHN (Netzer et al., 2011), notMNIST (Bulatov, 2011) and
Fashion MNIST (Xiao et al., 2017). Each dataset in 5-Datasets is regarded as a
learning task.

####  5.1.2. Comparison methods

We compare our method with various continual learning methods including
memory-based approaches and regularization-based approaches. Concretely, the
memory-based approaches include reservoir sampling (ER_Res) (Chaudhry et al.,
2019), Averaged GEM (A-GEM) (Chaudhry et al., 2018), and Orthogonal Weight
Modulation (OWM) (Zeng et al., 2019). For regularization-based methods, we use
HAT (Serra et al., 2018) and Elastic Weight Consolidation (EWC) (Kirkpatrick
et al., 2017). Besides, the state-of-the-art gradient projection method
Gradient Projection Memory (GPM) (Saha et al., 2021) is also adopted for
comparison. Additionally, we add the “Multitask” baseline where all tasks are
learned jointly using the entire dataset at once in a single network.
Multitask serves as the upper bound on average accuracy on all tasks.

####  5.1.3. Evaluation metrics

Following (Saha et al., 2021), we evaluate the performance on the following
metrics: Average Accuracy (ACC) and Backward Transfer (BWT). ACC is the
average test classification accuracy of all tasks. BWT measures the model’s
capability of retaining previous knowledge after learning a new task.
Formally, ACC and BWT are defined as:

(12) $ACC=\frac{1}{T}\sum_{i=1}^{T}A_{T,i},\quad
BWT=\frac{1}{T-1}\sum_{i=1}^{T-1}A_{T,i}-A_{i,i}$

where T is the total number of sequential tasks, $A_{T,i}$ is the evaluated
accuracy of the model on $\tau=i$ task after learning the $\tau=T$ task
sequentially.

####  5.1.4. Implementation details

Following the general experiment setting of CL (Saha et al., 2021; Yoon et
al., 2020), in our experiments, we use a 5-layer AlexNet for the 5-Split,
10-Split and 20-Split CIFAR-100 dataset. For CIFAR-100 Superclass, we use the
LeNet-5 architecture. As for the 5-datasets, similar to (Chaudhry et al.,
2019), we use a reduced ResNet18 architecture. In the -Split CIFAR-100 and
5-Datasets experiments, we train each task for a maximum of 200 and 100 epochs
respectively. The early termination strategy is also adopted as in (Serra et
al., 2018). For all the datasets, we set the batch size 64. We set the values
of $\lambda$ and $\eta$ to 0.1 and 0.7, respectively. In the network training
stage, all tasks share the same backbone network but each task has its own
classifier. The classifier is fixed after the model is trained on the
corresponding task. At inference, the task identifier can not be accessed. We
use the threshold $\epsilon$ in (Saha et al., 2021) for SVD k-rank
approximation. For the augmentation scheme in contrastive learning, we use
AugMix (Hendrycks et al., 2020) as the augmentation.

###  5.2. Experimental Results

Here, we present the quantitative evaluation results on various benchmark
datasets and network structures to investigate our method.

Table 1. Performance comparison of CL methods in terms of ACC (%) and BWT (%) on CIFAR-100 and 5-Datasets. Methods | CIFAR-100 | 5-Datasets
---|---|---
ACC | BWT | ACC | BWT
OWM(Zeng et al., 2019) | 50.94 | -0.30 | - | -
HAT(Serra et al., 2018) | 72.06 | -0.00 | 91.32 | -0.01
A-GEM(Chaudhry et al., 2018) | 63.98 | -0.15 | 84.04 | -0.12
ER_Res(Chaudhry et al., 2019) | 71.73 | -0.06 | 88.31 | -0.04
EWC(Kirkpatrick et al., 2017) | 68.80 | -0.02 | 88.64 | -0.04
GPM(Saha et al., 2021) | 72.25 | 0.17 | 90.44 | -1.41
CGP (ours) | 74.26 | -0.37 | 90.94 | -1.48
Multitask | 79.58 | - | 91.54 | -

####  5.2.1. Comparison Results on CIFAR-100 and 5-Datasets

Quantitative comparisons with state-of-the-art methods on CFAR-100 and
5-Datasets are shown in Tab. 1. As can be observed from the table, on the
CIFAR-100 dataset, our CGP consistently outperforms all baselines. It is worth
noting that our method produces around 2.0% gain in terms of ACC compared to
the outstanding competitor GPM. As discussed in Sec. 4, the main difference
between CGP and GPM is the way to calculate the bases. The comparison results
demonstrate that there is severe class deviation in tasks, in addition, our
method can solve this problem effectively. Furthermore, the results achieved
by CGP corroborates with our motivation that it is more beneficial to consider
the class deviation in the task. We also observe that even on the more
challenging 5-Datasets, our method achieves the accuracy of 90.94%, which is
0.5% higher than that of the strong baseline GPM.

Table 2. Experiments result on CIFAR-100 Superclass dataset. $(\dagger)$ denotes the result reported from APD (Yoon et al., 2020).  Methods | ACC (%) | Capacity (%)
---|---|---
PGN$(\dagger)$ | 50.76 | 271
DEN$(\dagger)$ | 51.10 | 191
RCL$(\dagger)$ | 51.99 | 184
APD$(\dagger)$ | 56.81 | 130
CGP(ours) | 57.53 | 100
Multitask$(\dagger)$ | 61.00 | 100

####  5.2.2. Comparison Results on CIFAR-100 Superclass

To compare our method with the state-of-the-art expansion based methods, we
conduct experiment on CIFAR-100 Superclass dataset. In this dataset, we split
the CIFAR-100 to make each task contains 5 different but semantically related
classes. Comparison results are shown in Tab. 2, where “Capacity” denotes the
percentage of network capacity used with respect to the original network. As
shown, our CGP outperforms other methods with a fixed capacity network, which
suggests that gradient projection with class bases is indeed helpful for
improving both plasticity and stability. Specifically, our method achieves
57.53% and -0.26% in terms of ACC and BWT with the smallest network,
respectively. For instance, CGP outperforms APD with around 0.72% gain in
terms of ACC by 30% fewer network parameters, showing the good
implementability of our method. To sum up, our approach successfully preserves
the knowledge learned before and learns the useful representations for future
learning, and thus it significantly mitigates catastrophic forgetting.

###  5.3. Ablation Studies

To analyze our method in more depth, we study the impact of different ablation
variants of CGP on performance.

####  5.3.1. Components Analysis.

Table 3. Ablation study on the designed components. “BR” and “Con” represents the base refining and contrastive learning, respectively. CGP | BR | Con | ACC(%) | BWT(%)
---|---|---|---|---
|  |  | 58.32 | -22.96
✓ |  |  | 69.19 | -0.08
✓ | ✓ |  | 72.61 | -1.64
✓ | ✓ | ✓ | 74.26 | -0.37

Figure 3. Performance comparison between GPM and CGP in terms of ACC (%) on
CIFAR-100 dataset with 5-Split (left), 10-Split (center) and 20-Split (right).
We report the test results after these methods learn one task completely.

Framework

Figure 4. Performance comparison in terms of ACC (%) on 10-Split CIFAR-100
dataset with different variants of our method.

Framework

We perform an ablation experiment on the 10-Split CIFAR-100 dataset to
scrutinize the effectiveness of different versions of CGP. The ablative
results are described in Tab. 3. Note that the first row in the table
represents the result of finetuning, which trains a single model to solve all
the tasks without adding any component. It can be observed from the results
that finetuning suffers from catastrophic forgetting (drops to -22.96% in
terms of BWT). Oppositely, we can observe that Class Gradient Projection
brings a significant performance gain (row 2), producing around 10.87% and
22.86% increase in terms of ACC and BWT, respectively. In particular, the our
CGP achieves the best results on BWT (-0.08%), suggesting its capability of
preserving the knowledge of the learned tasks. When CGP equipped Base Refining
(row 3), we can observe our method achieves a better result of 72.61% in terms
of ACC. This comparison indicates that estimating the similarity between
classes has an positive impact on the class gradient projection and
maintaining a Base Refining structure of bases calculation is of particular
importance. In addition, when we combine contrastive learning with our CGP,
the approach achieves the best performance, demonstrating the ability of our
approach to learn new knowledge and preserve old knowledge. Furthermore, Fig.
4 indicates a more detailed results in terms of the average accuracy of the
components after each task is completed. We plot the learning curve of GPM to
compare with our methods. As shown in the figure, without contrastive
learning, our CGP is still better than GPM. Furthermore, when CGP is equipped
with contrastive learning, it outperforms the GPM by a sizable margin. This
comparison indicates that our method can ensure less forgetting of old
knowledge when learning knowledge from new tasks.

Table 4. Ablation study on the similarity threshold. We conduct experiments on the CIFAR-100 dataset. Threshold | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 1.0
---|---|---|---|---|---|---
ACC(%) | 74.12 | 73.98 | 74.26 | 73.43 | 70.76 | 69.19
BWT(%) | -1.78 | -1.36 | -0.37 | -1.22 | -2.30 | -0.08

####  5.3.2. Analysis on Stability and Plasticity.

To study the balance of stability and plasticity, which is controlled by
similarity threshold $\eta$, we compare the performance of our CGP by varying
$\eta=$ 0.6, 0.7, 0.8, 0.9, 1.0 on 10-Split CIFAR-100 dataset. As shown in
Tab. 4, when the $\eta=$ 1.0, the backward transfer (BWT) becomes the best,
just -0.08%, which means that the network hardly forgets knowledge. But the
average accuracy comes to 69.19%, this result clearly indicates the method
just preserves the knowledge learned before, refusing to accept new knowledge.
When the $\eta$ decreases to 0.9, although the ACC is better, the BWT is
worse. Maybe the relaxation of constraints increases the optimization space,
disturbing the learning of new knowledge and the preserving of old knowledge.
With the decrease of $\eta$, the balance is going to be better. So the
performance is going to be better no matter ACC and BWT. When the $\eta$
equals 0.7, the performance comes to the best with 74.26% in ACC and -0.37% in
BWT. Since ACC is affected by both stability and plasticity, showing that the
network can learn more new knowledge and preserve the learned knowledge at the
same time. In addition, as the $\eta$ decreases, the BWT is going to be worse,
which indicates that although the ability of the network to preserve knowledge
is becoming weak, the ability to learn new knowledge is becoming stronger.

####  5.3.3. Analysis on Different Sequences.

In order to further evaluate the ability to prevent catastrophic forgetting on
the different sequences of the proposed CGP, we conduct the experiments under
the 20-task, 5-task settings on the CIFAR-100 dataset. The 20-Split CIFAR-100
evaluates the ability of the network to prevent catastrophic forgetting on the
longer sequence. In addition, the 5-Split CIFAR-100 evaluates the power of the
network to classify more classes maintained in each task. With the number of
classes increasing, the class deviation behaves severely. The experimental
results are shown in Fig. 3. It is clear that the proposed approach is
significantly better than GPM in each split setting. In particular, when we
conduct the experiments on 5-Split CIFAR-100, our approach outperforms the GPM
by a sizable margin at each task. After the training, our CGP produces around
1.0% gain in terms of the average accuracy. The comparison results confirm
that our approach has the ability to classify more different classes
maintained in each task than GPM. On 10-Split, as the experiments in the main
experiment, our approach significantly outperforms the GPM. The curve of
average accuracy indicates that our method can consistently outperform GPM.
Furthermore, although GPM can achieve a comparable result to our approach on
20-Split, our method still is in general better than it.

Table 5. The performance of the CGP method with a various number of samples to calculate representations experimented on CIFAR-100. Setting | 20 | 125 | 200
---|---|---|---
ACC | BWT | ACC | BWT | ACC | BWT
BR-STD | 75.03 | -1.09 | 73.43 | -2.04 | 72.81 | -2.06
BR-GTL | 75.03 | -1.10 | 74.26 | -0.37 | 72.72 | -1.74

####  5.3.4. Analysis on Number of Representation Samples.

As described in Sec. 4.2, we perform experiments on similarity calculation
with two settings. Our CGP selects the representation of the samples using a
certain number for similarity estimation and representation matrix
construction. We conduct experiments for BR-GTL and BR-STD including the
number of 20, 125, and 200. Tab. 5 summarizes the results of this experiment.
In general, we observe that the BR-GTL has a similar performance to BR-STD in
most cases, with a bit of an edge over BR-STD. Particularly, when we just
choose 20 for similarity estimation and representation matrix construction,
the results are not much different in terms of average accuracy and backward
transfer. Furthermore, a small number of samples are not enough to represent
the class, which lead to inaccurately calculated bases and lead catastrophic
forgetting. When the number comes to 125, disparities are starting to appear.
The ACC and BWT of BR-STD come to 73.43% and -2.04%. Compared to the results
of 20, there is a 1.6% and 0.95% dropping, respectively. It is worth noting
that the ACC of BR-GTL is decreasing to 74.26%. But there is a 0.73% relative
gain in terms of BWT, indicating that the network can preserve more knowledge.
Finally, when the number comes to 200, the performance of BR-STD and BR-GTL
decreases both. The comparison results show that directly increasing the
number does not bring better performance.

####  5.3.5. Analysis on $\lambda$.

Table 6. Comparison of average accuracy and backward transfer on 10-Split CIFAR-100 with the $\lambda$ from 0.0 to 1.0. When $\lambda$ is 0, it represents just using classification loss. $\lambda$ | 0.0 | 0.1 | 0.3 | 0.5 | 0.7 | 1.0
---|---|---|---|---|---|---
ACC(%) | 72.61 | 74.26 | 72.85 | 72.07 | 73.02 | 71.22
BWT(%) | -1.64 | -0.37 | -1.98 | -3.16 | -2.09 | -3.91

To evaluate the influence of different $\lambda$ values on the final
performance, we perform an ablation study with $\lambda$ from 0.0 to 1.0 under
the similarity threshold of 0.7. Table 5 shows the results of the comparison
on the 10-Split CIFAR-100 dataset. When the $\lambda$ is 0, it means that the
method is just the CGP with similarity calculation, resulting in 72.61% in
terms of ACC and -1.64% in terms of BWT. When equipped with contrastive
learning whose $\lambda$ equaling to 0.1, the ACC increases to 74.26% and the
BWT decreases to -0.37%. This comparison result illustrates that our CGP when
equipped with contrastive learning has the stronger power to learn new
knowledge and preserve the old learned knowledge. We also note that the
performance of BWT becomes worse with the increase of the $\lambda$,
suggesting that as the network learns more knowledge, it gradually loses the
ability to preserve the learned knowledge.

##  6\. Conclusion

In this paper, we propose class gradient projection (CGP) for addressing the
plasticity-stability dilemma for continual learning. CGP calculates the
gradient subspace from individual classes rather than tasks Based on the CGP
framework, we introduce a Base Refining (BR) algorithm as well as a
contrastive loss to further alleviate catastrophic forgetting. The two
components can complement each other to enhance the plasticity and stability
of the model simultaneously. The contrastive learning augments the samples to
pull positive samples closer and push negative samples apart, which encourages
the network to learn discriminative and robust representation. We conduct
extensive experiments on several benchmark datasets and various network
architectures. The achieved results demonstrate the effectiveness of our
method.

###### Acknowledgements.

This study was supported by grants from Chinese National Science & Technology
Pillar Program (No. 2022YFC2009900/2022YFC200990
3), the National Natural Science Foundation of China (Grant No. 62122018, No.
62020106008, No. 61772116, No. 61872064).

## References

  * (1)
  * Abati et al. (2020) Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. 2020.  Conditional channel gated networks for task-aware continual learning. In _Proceedings of the Conference on Computer Vision and Pattern Recognition_. 3931–3940.
  * Aljundi et al. (2018) Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018\.  Memory aware synapses: Learning what (not) to forget. In _Proceedings of the European Conference on Computer Vision_. 139–154.
  * Ayub and Wagner (2021) Ali Ayub and Alan R. Wagner. 2021.  EEC: Learning to Encode and Regenerate Images for Continual Learning. In _Proceedings of the International Conference on Learning Representations_.
  * Bulatov (2011) Yaroslav Bulatov. 2011\.  Notmnist dataset.  _Google (Books/OCR), Tech. Rep.[Online]. Available: http://yaroslavvb. blogspot. it/2011/09/notmnist-dataset. html_ 2 (2011).
  * Cai et al. (2020) Yuanqiang Cai, Dawei Du, Libo Zhang, Longyin Wen, Weiqiang Wang, Yanjun Wu, and Siwei Lyu. 2020.  Guided Attention Network for Object Detection and Counting on Drones. In _MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020_. 709–717.
  * Cha et al. (2021) Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. 2021.  Co2L: Contrastive Continual Learning. In _Proceedings of the International Conference on Computer Vision_. 9516–9525.
  * Chaudhry et al. (2018) Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2018.  Efficient lifelong learning with a-gem.  _arXiv preprint arXiv:1812.00420_ (2018).
  * Chaudhry et al. (2019) Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip H. S. Torr, and Marc’Aurelio Ranzato. 2019.  Continual Learning with Tiny Episodic Memories.  _CoRR_ abs/1902.10486 (2019).
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _Proceedings of the International conference on machine learning_. 1597–1607.
  * Chen and He (2021) Xinlei Chen and Kaiming He. 2021.  Exploring simple siamese representation learning. In _Proceedings of the Conference on Computer Vision and Pattern Recognition_. 15750–15758.
  * Delange et al. (2021) Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. 2021\.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2021).
  * Deng and Liu (2020) Ruoxi Deng and Shengjun Liu. 2020.  Deep Structural Contour Detection. In _MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020_. 304–312.
  * Ebrahimi et al. (2020) Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach. 2020.  Adversarial continual learning. In _Proceedings of the European Conference on Computer Vision_. 386–402.
  * Farajtabar et al. (2020) Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. 2020\.  Orthogonal gradient descent for continual learning. In _Proceedings of the International Conference on Artificial Intelligence and Statistics_. 3762–3773.
  * Gutmann and Hyvärinen (2010) Michael Gutmann and Aapo Hyvärinen. 2010.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In _Proceedings of the thirteenth international conference on artificial intelligence and statistics_. 297–304.
  * Hendrycks et al. (2020) Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. 2020.  AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. In _Proceedings of the International Conference on Learning Representations_.
  * Hung et al. (2019) Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. 2019\.  Compacting, picking and growing for unforgetting continual learning.  _Advances in Neural Information Processing Systems_ 32 (2019).
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised contrastive learning.  _Advances in Neural Information Processing Systems_ 33 (2020), 18661–18673.
  * Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017\.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ 114 (2017), 3521–3526.
  * Krizhevsky et al. (2009) Alex Krizhevsky, Geoffrey Hinton, et al. 2009\.  Learning multiple layers of features from tiny images.  (2009).
  * Lee et al. (2018) Seung Hyun Lee, Dae Ha Kim, and Byung Cheol Song. 2018\.  Self-supervised knowledge distillation using singular value decomposition. In _Proceedings of the European Conference on Computer Vision_. 335–350.
  * Lee et al. (2017) Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. 2017.  Overcoming catastrophic forgetting by incremental moment matching.  _Advances in neural information processing systems_ 30 (2017).
  * Li et al. (2020b) Junnan Li, Pan Zhou, Caiming Xiong, and Steven CH Hoi. 2020b.  Prototypical contrastive learning of unsupervised representations.  _arXiv preprint arXiv:2005.04966_ (2020).
  * Li et al. (2020a) Xinke Li, Chongshou Li, Zekun Tong, Andrew Lim, Junsong Yuan, Yuwei Wu, Jing Tang, and Raymond Huang. 2020a.  Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene. In _MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020_. 238–246.
  * Li et al. (2019) Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. 2019.  Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In _Proceedings of the International Conference on Machine Learning_. 3925–3934.
  * Lin et al. (2022) Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. 2022\.  TRGP: Trust Region Gradient Projection for Continual Learning.  _CoRR_ abs/2202.02931 (2022).
  * Lopez-Paz and Ranzato (2017) David Lopez-Paz and Marc’Aurelio Ranzato. 2017.  Gradient episodic memory for continual learning.  _Advances in neural information processing systems_ 30 (2017).
  * McCloskey and Cohen (1989) Michael McCloskey and Neal J Cohen. 1989.  Catastrophic interference in connectionist networks: The sequential learning problem.  In _Psychology of learning and motivation_. Vol. 24. 109–165.
  * Netzer et al. (2011) Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. 2011\.  Reading Digits in Natural Images with Unsupervised Feature Learning. In _NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011_.
  * Nguyen et al. (2017) Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. 2017.  Variational continual learning.  _arXiv preprint arXiv:1710.10628_ (2017).
  * Rajasegaran et al. (2019) Jathushan Rajasegaran, Munawar Hayat, Salman H Khan, Fahad Shahbaz Khan, and Ling Shao. 2019\.  Random path selection for continual learning.  _Advances in Neural Information Processing Systems_ 32 (2019).
  * Ratcliff (1990) Roger Ratcliff. 1990\.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.  _Psychological review_ 97 (1990), 285.
  * Rebuffi et al. (2017) Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017.  icarl: Incremental classifier and representation learning. In _Proceedings of the Conference on Computer Vision and Pattern Recognition_. 2001–2010.
  * Rusu et al. (2016) Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016.  Progressive Neural Networks.  _CoRR_ abs/1606.04671 (2016).
  * Saha et al. (2021) Gobinda Saha, Isha Garg, and Kaushik Roy. 2021.  Gradient Projection Memory for Continual Learning. In _Proceedings of the International Conference on Learning Representations_.
  * Serra et al. (2018) Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018.  Overcoming catastrophic forgetting with hard attention to the task. In _Proceedings of the International Conference on Machine Learning_. 4548–4557.
  * Sohn et al. (2020) Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. 2020.  FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence.  _CoRR_ abs/2001.07685 (2020).
  * Sprechmann et al. (2018) Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel, Adrià Puigdomènech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, and Charles Blundell. 2018.  Memory-based Parameter Adaptation. In _Proceedings of the International Conference on Learning Representations_.
  * van den Oord et al. (2018) Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018\.  Representation Learning with Contrastive Predictive Coding.  _CoRR_ abs/1807.03748 (2018).
  * Veniat et al. (2021) Tom Veniat, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2021\.  Efficient Continual Learning with Modular Networks and Task-Driven Priors. In _Proceedings of the International Conference on Learning Representations_.
  * Wang et al. (2020) Xin Wang, Wei Huang, Qi Liu, Yu Yin, Zhenya Huang, Le Wu, Jianhui Ma, and Xue Wang. 2020.  Fine-Grained Similarity Measurement between Educational Videos and Exercises. In _MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020_. 331–339.
  * Xiao et al. (2017) Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  _arXiv preprint arXiv:1708.07747_ (2017).
  * Yoon et al. (2020) Jaehong Yoon, Saehoon Kim, Eunho Yang, and Sung Ju Hwang. 2020\.  Scalable and Order-robust Continual Learning with Additive Parameter Decomposition. In _Proceedings of the International Conference on Learning Representations_.
  * Zeng et al. (2019) Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. 2019\.  Continual learning of context-dependent processing in neural networks.  _Nature Machine Intelligence_ 1 (2019), 364–372.
  * Zenke et al. (2017) Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017\.  Continual learning through synaptic intelligence. In _Proceedings of the International Conference on Machine Learning_. 3987–3995.
  * Zhang et al. (2022) Ji Zhang, Jingkuan Song, Lianli Gao, Ye Liu, and Heng Tao Shen. 2022.  Progressive Meta-learning with Curriculum.  _IEEE Transactions on Circuits and Systems for Video Technology_ (2022).
  * Zhang et al. (2021) Ji Zhang, Jingkuan Song, Yazhou Yao, and Lianli Gao. 2021\.  Curriculum-Based Meta-learning. In _MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 - 24, 2021_. 1838–1846.