# CBCL-PR: A Cognitively Inspired Model for Class-Incremental Learning in
Robotics

Ali Ayub1,2 and Alan R. Wagner3 1Department of Electrical and Computer
Engineering, University of Waterloo, Waterloo, ON N2L3G1, Canada2Concordia
Institute for Information Systems Engineering, Concordia University, Montreal,
QC H3G1M8, Canada<EMAIL_ADDRESS>of Aerospace Engineering,
The Pennsylvania State University, State College, PA 16802, USA
<EMAIL_ADDRESS>
###### Abstract

For most real-world applications, robots need to adapt and learn continually
with limited data in their environments. In this paper, we consider the
problem of Few-Shot class Incremental Learning (FSIL), in which an AI agent is
required to learn incrementally from a few data samples without forgetting the
data it has previously learned. To solve this problem, we present a novel
framework inspired by theories of concept learning in the hippocampus and the
neocortex. Our framework represents object classes in the form of sets of
clusters and stores them in memory. The framework replays data generated by
the clusters of the old classes, to avoid forgetting when learning new
classes. Our approach is evaluated on two object classification datasets
resulting in state-of-the-art (SOTA) performance for class-incremental
learning and FSIL. We also evaluate our framework for FSIL on a robot
demonstrating that the robot can continually learn to classify a large set of
household objects with limited human assistance.

###### Index Terms:

Continual Learning, Catastrophic Forgetting, Few-Shot Learning, Cognitively-
Inspired Architectures, HRI.

##  I Introduction

One of the hallmarks of human intelligence is the ability to learn
continuously over time. The human brain is flexible enough to learn new tasks
quickly (from a few examples), yet stable enough to maintain and recall old
tasks over its lifetime. In neuroscience, this is known as the stability-
plasticity dilemma [1, 2]. Robots operating in dynamic ever-changing real-
world environments face the same dilemma i.e. they must be able to learn about
new tasks quickly from a small number of examples without forgetting the
previously learned tasks.

For example, consider a household robot [3, 4] tasked with locating and
organizing various household objects. The robot must be trained on what items
it should organize in a context by its non-expert owner and recognize that the
items to be organized might change over time. Further, the robot must learn
these items from a few examples because of the limited supervision available
from its non-expert owner. In this paper, we use concept learning ideas and
theories from cognitive science to develop a framework that can allow robots
to learn novel concepts continually in real-world environments.

Although deep learning has achieved remarkable success in object recognition
tasks [5], deep models suffer from catastrophic forgetting when learning
continually [6]. Catastrophic forgetting occurs when continually training a
system to recognize new object classes, the system drastically forgets the
previously learned classes and the overall classification accuracy decreases
significantly. Several deep learning techniques have been proposed in recent
years to tackle the catastrophic forgetting problem [7, 8, 9, 10, 11]. Most
continual learning (also termed as incremental learning) approaches [7, 8],
however, require data of the old classes to be stored which makes them
impractical on robots which usually have a limited amount of on-board memory
available. Further, many continual learning approaches [12, 7, 8] require a
large number of training examples per class. Robots, on the other hand,
typically only have access to limited amount of supervised training data in
real-world environments. Thus, it is imperative for robots to learn
continually from a few examples per class. We term this problem as Few-Shot
Incremental Learning (FSIL).

In this paper, we tackle FSIL by developing a novel framework inspired by the
ideas of concept learning and replay in the hippocampus and the neocortex [13,
14, 15, 16]. Our framework, named Centroid Based Concept Learning with Pseudo-
rehearsal (CBCL-PR), uses a cognitively-inspired clustering technique to
generate multiple clusters of each new class in an increment and store them in
the robot’s memory. For the classification of objects, CBCL-PR recalls old
object classes from its memory and trains a neural network on generated data
of the old classes and real data of a new class in an increment. Thus, CBCL-PR
avoids catastrophic forgetting without storing the real data of the object
classes in its memory.

We perform extensive evaluations on two benchmark continual learning datasets
and on a real robot tasked with learning household objects continually through
human assistance. Our results show that CBCL-PR does not suffer from
catastrophic forgetting when learning continually, and it significantly
outperforms state-of-the-art (SOTA) approaches for continual learning and
FSIL. An ablation study of CBCL-PR further demonstrates the contribution of
our proposed cognitively-inspired components toward the overall performance of
the architecture. Our results suggest that our proposed architecture may allow
robots to operate and adapt in real-world environments over the long term.

##  II Related Work

###  II-A Continual Learning

Most continual learning techniques proposed in recent years rely on storing a
portion of the data from prior classes when learning new classes [7, 8, 9].
iCaRl [7] is one of the earliest approaches for class-incremental learning
(CiL) that stores data of the old classes. iCaRL uses a regularization term
called distillation loss [17], which forces the model to keep the labels of
the training images of the previous classes to remain the same when learning
from new data. iCaRL also uses Nearest Class Mean (NCM) classifier [18] for
classification, instead of the softmax output to mitigate catastrophic
forgetting. The NCM classifier uses the distance of the feature vectors of
test images from the centroids of each class to predict the labels for the
test images. EEIL [8] and BiC [9] use similar ideas but improve over iCaRL
using end-to-end learning. One of the main limitations of these approaches is
that they require storing high-dimensional images of the old classes, which is
neither biologically inspired nor practical in situations when the system has
limited memory.

To avoid storing real images, various regularization-based techniques have
been proposed [19, 20, 21]. These techniques use a regularization loss term to
prevent the weights of the model from changing drastically when learning new
classes. Another set of approaches uses generative memory to regenerate images
of the old classes using generative models [22, 23, 24], and avoid storing the
real images of the old classes. Both regularization-based and generative
memory-based approaches are more prone to catastrophic forgetting compared to
the approaches that store real images of the old classes. One major issue with
all CiL approaches is that they require a large amount of training data for
each class. Thus, these approaches are not suitable for FSIL.

As our approach follows the constraints of a class-incremental setup, we
mainly review class-incremental learning techniques in this paper. A more
natural setting of continual learning is open-ended learning in which
instances of possible classes can be presented in any arbitrary sequence, e.g.
[25, 26, 27]. We hope to adapt our approach in the future for open-ended
learning.

###  II-B Few-Shot Learning

Few-shot learning (FSL) has been extensively studied in machine learning in
recent years to create models that learn from a few examples per class [28,
29, 30]. Following the general setup of FSL, these approaches first learn a
large set of base classes with a large number of training examples to develop
a general representation. Then, they learn a small number of classes ($k$)
with a few training examples ($n$) and are tested on the test set of the newly
learned classes, termed as $n$-shot $k$-way tasks. Some approaches also try to
remember the base classes when learning the new classes and are tested on the
combined test set of the base and the new classes [29, 30]. However, these
approaches do not learn continually for a large number of increments and
suffer from catastrophic forgetting.

###  II-C Few-Shot Incremental Learning

For the few-shot incremental learning problem (FSIL) considered in this paper,
a model is trained on $n$ examples for $k$ classes in each increment. The
model does not have access to the data of the $l$ previously learned classes.
After the training process in an increment, the model is evaluated on the test
set of all the classes learned ($k+l$). Thus, the model is evaluated on
$n$-shot ($k+l$) way tasks in each increment, where $k+l$ keeps increasing in
each new increment making the task continually harder. In the past couple of
years, there are some approaches that have been proposed for FSIL [31, 32].
Tao et al. [31] use a neural gas representation to preserve the knowledge of
the previous classes. Zhang et al. [32] use a graph model to preserve past
knowledge and use it to adapt new classifiers. However, both of these
approaches start with a large number of base classes in the first increment.

###  II-D Robotics Applications of Continual Learning

Continual learning has also been applied to real robots in recent years for
object learning tasks [33, 34]. Turkoglu et al. present a deep learning
approach for learning objects continually using mobile robots [35]. They also
use prior class data when learning new classes which limit the usefulness of
this approach for realistic applications. Similar to Rebuffi et al. [7],
Deghhan et al. [36] present a continual object learning approach that uses the
nearest class mean (NCM) classifier [18] and apply it on a robot manipulator.
One important issue with these prior works is that most approaches [35, 36]
continually learn a very small number of object classes (usually 10 or fewer),
which is unrealistic. It is also not suitable for testing continual learning
approaches, as catastrophic forgetting occurs when learning a large number of
classes over a large number of increments (as shown in Section V). Further,
most prior work [37, 36] trains and tests the robot on the same objects, which
positively skews the results and limits real-world applicability, because in
the real-world test data can differ from the training data. In this paper, we
present an approach for FSIL that avoids these limitations.

###  II-E Models of Concept Learning in Cognitive Science

Decades of neuroscience research has shown that the neural structures
associated with concept learning are the hippocampus and the neocortex [13,
14, 15, 16]. Based on experimental support from neuroscience, the hippocampus,
and the neocortex use several mechanisms to learn and store new memories [16,
13, 15, 38]. The theoretical models proposed in this research suggest that
whenever a new episode (stimulus/event) is encountered by the hippocampus it
first extracts feature information about the episode. Then the difference
between the feature map of the episode to previously learned concepts is
calculated. This difference is known as memory-based prediction error. When
the memory-based prediction error is large, the hippocampus performs pattern
separation by creating a new, distinct concept for the episode. When the
memory-based prediction error is small, memory integration is performed by
updating an existing concept to incorporate the new episode.

Evidence from neuroscience also suggests that long-term memories are recalled
by the neocortex for memory replay during REM sleep [39, 40]. Using a
combination of these processes, the mammalian brain continues to develop and
store memories over time without catastrophic forgetting. CBCL-PR is inspired
by these mechanisms of concept learning and memory replay in the mammalian
brain for class-incremental learning in robots.

Figure 1: Complete architecture of CBCL-PR. The model extracts features for
the training images of a class and generates clusters using Agg-Var
clustering. Pseudo-exemplars of the old classes are then generated which are
used with the feature vectors of the new class to train a shallow classifier.

##  III Centroid-Based Concept Learning with Pseudo-rehearsal

CBCL-PR consists of two major phases, 1) concept learning phase and 2) pseudo-
rehearsal phase, for continual learning of object classes. We use a pre-
trained convolutional neural network (CNN) on image data of object classes to
generate feature vectors which are then fed to CBCL-PR. Figure 1 shows our
complete CBCL-PR architecture.

###  III-A Agg-Var Clustering

The learning phase of CBCL-PR consists of a clustering approach (termed as
Agg-Var clustering) inspired by the concept learning models in the hippocampus
and the neocortex. For the continual learning setting, when CBCL-PR receives
the training data of a new image class $1\leq y\leq N$ in a new increment,
Agg-Var clustering is applied on the feature vectors
$\\{x_{1}^{y},x_{2}^{y},...,x_{N_{y}}^{y}\\}$ of the class. The process begins
by initializing a new cluster for the class $y$, with the centroid $c_{1}^{y}$
equated to the first feature vector $x_{1}^{y}$ of the class. Next, each
feature vector $x_{i}^{y}$ is compared to all the clusters of class $y$ by
computing the Euclidean distance between the feature vector $x_{i}^{y}$ and
the centroids of the clusters of class $y$. If the distance
$dist(x_{i}^{y},c_{min}^{y})$ between the feature vector $x_{i}^{y}$ and the
closest centroid $c_{min}^{y}$ is lower than the distance threshold $D$ (a
hyperparameter), $x_{i}^{y}$ is integrated into the corresponding cluster
(similar to memory integration), and the closest centroid $c_{min}^{y}$ is
updated by taking a weighted mean of the centroid and the feature vector:

$c_{min}^{y}=\frac{w_{c}\times c_{min}^{y}+x_{i}^{y}}{w_{c}+1}$ (1)

where, $w_{c}$ is the number of feature vectors already clustered in the
cluster represented by the centroid $c_{min}^{y}$. If, on the other hand,
distance $dist(x_{i}^{y},c_{min}^{y})$ is greater than the distance threshold
$D$, a new cluster is initialized for class $y$, with the centroid of the
cluster equated to the feature vector $x_{i}^{y}$ (similar to pattern
separation).

This process is applied on the all feature vectors of the new class $y$ which
results in a set of clusters $N^{*}_{y}$ for the class111$N^{*}_{y}$ can be
different for different classes and is unknown beforehand. The total number of
clusters for each class is dependant on the similarity among the images
(intra-class variance) of the class, thus it is determined by the distance
threshold $D$.. After finding the clusters for all the feature vectors, Agg-
Var clustering also computes the covariance matrices for the clusters of class
$y$ using the corresponding feature vectors. Thus, the clusters for class $y$
are represented by a set of centroids
$C^{y}=\\{c_{1}^{y},...,c_{N^{*}_{y}}^{y}\\}$ and covariance matrices
$\sum^{y}=\\{\sigma_{1}^{y},...,\sigma_{N^{*}_{y}}^{y}\\}$, both of which are
stored in the system’s memory. Note that after the centroids and covariance
matrices for the new class are computed and stored in memory, the original
feature vectors are discarded. An intuitive explanation of Agg-Var clustering
is illustrated in Figure 2, and a formal description of the complete process
of Agg-Var clustering in a single increment is described in Algorithm 1.

Algorithm 1 CBCL-PR: Agg-Var Clustering

Input: $X=\\{X^{1},...,X^{t}\\}$ $\triangleright$ feature vector sets of $t$
classes

require: $D$ $\triangleright$ distance threshold

Output: $C=\\{C^{1},...,C^{t}\\}$ $\triangleright$ Centroid sets for $t$
classes

Output: $\sum=\\{\sum^{1},...,\sum^{t}\\}$ $\triangleright$ Covariance
matrices sets for $t$ classes

1:for $y=1$; $j\leq t$ do

2: $C^{y}\leftarrow\\{x_{1}^{y}\\}$$\triangleright$ initialize centroids for
each class

3:for $y=1$; $y\leq t$ do

4: for $i=2$; $i\leq N_{y}$ do

5:
$d_{min}\leftarrow\min_{l=1,..,N^{*}_{y}}dist(c^{y}_{l},x^{y}_{i})$$\triangleright$
distance to closest centroid

6:
$i_{min}\leftarrow\operatorname*{argmin}_{l=1,..,N^{*}_{y}}dist(c^{y}_{l},x^{y}_{i})$$\triangleright$
index of the closest centroid

7: Set $w^{y}_{i_{min}}$ to be the number of images clustered

8: in the $i_{min}$th cluster of class $y$

9: Assign $i_{min}$ as the cluster label to $x_{i}^{y}$

10: if $d_{min}<D$ then

11: $c^{y}_{i_{min}}\leftarrow\frac{w^{y}_{i_{min}}\times
c^{y}_{i_{min}}+x^{y}_{i}}{w^{y}_{i_{min}}+1}$$\triangleright$ see equation
(1)

12: else

13: $C^{y}.append(x^{y}_{i})$$\triangleright$ new centroid for class $y$

14: for $i=1$;$i\leq N^{*}_{y}$ do

15: $X^{y*}_{i}=\\{x_{k}^{y}|x_{k}^{y}\in c_{i}^{y}\\}\forall
k\in\\{1,...,N_{y}\\}$$\triangleright$ get all feature vectors belonging to
$i$th cluster of class $y$

16: $\sum^{y}.append(cov(X^{y*}_{i}))$$\triangleright$ add covariance matrix
for the feature vectors in cluster $i$ to the set $\sum^{y}$

Figure 2: The complete process of Agg-Var clustering. (a) New feature vector
closest to one of the previous centroids of the same class, is integrated in
the cluster and a new centroid is generated. (b) In the next increment, a new
feature vector is far from the two centroids, so a new cluster is generated
with new feature vector as the centroid. (c) Another new feature vector for
the class is integrated in the previous cluster and a new centroid is
generated.

###  III-B Pseudo-rehearsal

Inspired by memory replay during the sleep phase of the mammalian brain, CBCL-
PR generates pseudo-exemplars using the clusters of the old classes to train a
classifier. The process is known as pseudo-rehearsal (also called intrinsic
replay) [40].

Let $M^{y}=\\{m_{1}^{y},...m_{i}^{y},...,m_{N^{*}_{y}}^{y}\\}$ represent the
number of images clustered in all the $N^{*}_{y}$ clusters of class $y$.
During pseudo-rehearsal, each centroid $c_{i}^{y}$ and covariance matrix
$\sigma_{i}^{y}$ pair of class $y$ is used to initialize a multivariate
Gaussian distribution. Each of the Gaussian distributions
$\mathcal{N}(c_{i}^{y},\sigma_{i}^{y})$ are sampled to generate $m_{i}^{y}$
number of pseudo-exemplars for class $y$. For a balanced training set, the
model generates a total of $N_{y}$ pseudo-exemplars after sampling all the
Gaussian distributions, where $N_{y}$ is the original number of feature
vectors of class $y$.

In a new increment, the pseudo-exemplars of the previous classes are mixed
with the real feature vectors of the new classes in the increment. The mixture
is used to train a linear classifier using back-propagation. During the test
phase, the model simply passes an unlabeled feature vector $x$ through the
classifier, to predict the label $y^{*}$ as:

$y^{*}=\operatorname*{argmax}_{y}(w_{y}^{T}x)$ (2)

where, $w_{y}^{T}$ is the weight vector of the linear classifier that
transforms the feature vector $x$ into probability values. Algorithm 2
formally describes the complete process of pseudo-rehearsal.

To show an illustration of pseudo-rehearsal, we performed a small experiment
on 2 classes in the MNIST dataset [41]. Figure 3 shows the original feature
vectors and pseudo-exemplars for the 2 classes in the MNIST dataset. The 2D
space covered by the pseudo-exemplars looks almost the same as the original
feature vectors. This shows that CBCL-PR can learn the complex distributions
of the image data using clusters, and it can use the clusters to regenerate
pseudo-exemplars that are significantly similar to the original feature
vectors. Finally, note that an object class can contain multiple clusters
representing clustered groups of different feature vectors of the class.

Figure 3: Comparison of the original feature space (11379 feature vectors)
with the pseudo-exemplars generated by 1000 clusters for 2 classes in the
MNIST dataset. Algorithm 2 CBCL-PR: Pseudo-rehearsal

require: $X^{t}=\\{x_{1}^{t},x_{2}^{t},...,x_{N_{t}}^{t}\\}$ $\triangleright$
training samples of new class $t$

require: $C=\\{C^{1},...,C^{t-1}\\}$ $\triangleright$ centroid sets for old
classes

require: $\sum=\\{\sum^{1},...,\sum^{t-1}\\}$ $\triangleright$ covariance
matrices for old classes

require: $M=\\{M^{1},...,M^{t-1}\\}$ $\triangleright$ Number of samples
clustered in clusters of old classes

require: $D$ $\triangleright$ Linear classifier

1:for $y=1$;$y<t$ do

2: $X^{*y}=\\{\\}$$\triangleright$ empty variable to temporarily store pseudo-
exemplars of class $y$

3: for $i=1$;$i\leq N^{*}_{y}$ do

4:
$X^{*y}.append(multivariate\\_normal(c_{i}^{y},\sigma_{i}^{y},m_{i}^{y}))$$\triangleright$
pseudo-exemplars using each centroid and covariance matrix pair of class $y$

5:$X^{*}=\\{X^{*1},X^{*2},...,X^{*t-1},X^{t}\\}$$\triangleright$ Mixture of
pseudo-exemplars and new feature vectors

6:Train parameters $\theta$ of $D$ with $X^{*}$ using back-propagation with
cross-entropy loss:

7:$l(\theta)=-\Sigma_{y=1}^{t}[\Sigma_{i=1}^{N_{y}}\delta_{y=y_{i}^{*}}{\rm{log}}(p(x_{i}^{y}))+\delta_{y\neq
y_{i}^{*}}{\rm{log}}(1-p(x_{i}^{y}))]$

###  III-C CBCL-PR for FSIL

The sections above describe our method for a general continual learning
setting. For few-shot incremental learning (FSIL), the Agg-Var clustering part
of CBCL-PR remains unchanged and, for each set of new classes, a set of
centroids and covariance matrices are generated and stored in memory. The
pseudo-rehearsal phase changes slightly for FSIL. Since each class only has a
small set of training examples (usually 5 or 10), we do not use the real
feature vectors of a new class when training the linear classifier. Instead we
generate a larger number of pseudo-exemplars (usually 40) for all the classes
(old and new) in each increment and train the linear classifier using the
generated pseudo-exemplars.

##  IV Prediction Time Analysis of CBCL

Our previously proposed vanilla CBCL method [42] uses a weighted voting scheme
for classification of unlabeled images. However, prediction time for the
weighted voting scheme increases as the number of clusters increases. The
prediction time complexity of CBCL’s weighted voting scheme for classification
(see [42] for details) is determined with respect to the number of clusters
(centroids) learned so far ($N_{C}$). Thus:

  * •

The time required to find the distance of the test feature vector from the
complete set of centroids learned so far is $O(N_{C})$;

  * •

Plus, the time required to find top $n$ closest centroids to the test feature
vector, $O(N_{C}{\rm{log}}(N_{C}))$;

  * •

Plus, the time required to update the prediction weights of all $t\leq N_{C}$
classes learned so far using the distances of the test feature vector from the
$n$ closest centroids, $O(t)$;

  * •

Plus, the time required to find the class with the highest prediction weight,
$O(t)$.

Thus, the time complexity ($T(N_{C})$) as a function of the number of
centroids learned so far is:

$\begin{split}T(N_{C})&=O(N_{C})+O(N_{C}{\rm{log}}(N_{C}))+O(t)+O(t)\\\
&=O(N_{C}{\rm{log}}(N_{C}))\end{split}$ (3)

Equation 3 shows that time complexity of the weighted voting scheme for
classification is directly proportional to the number of clusters $N_{C}$.
Thus, as the model learns more classes continually, the total number of
clusters increases which in turn increases the prediction time of the model.
Section V shows a comparison of the prediction times of CBCL and CBCL-PR on
the Caltech-101 dataset when incrementally learning 10 classes per increment
which confirms our analysis.

##  V Experiments

In this section, we first describe the datasets and implementation details for
the experiments. We then compare CBCL-PR to state-of-the-art (SOTA) approaches
and the batch learning baseline for continual learning, and 5-shot and 10-shot
incremental learning. Finally, we evaluate CBCL-PR for few-shot incremental
learning of household objects on the Baxter robot. Our code is available at
https://github.com/aliayub7/CBCL-PR.

TABLE I: Total Number of classes, number of training and test images per class and number of classes per increment for the CIFAR-100 and Caltech-101 datasets. Dataset |  CIFAR-100 |  Caltech-101
---|---|---
# classes |  100 |  100
# training images |  500 |  80% of data
# testing images |  100 |  20% of data
# classes/batch |  2, 5, 10, 20 |  10

###  V-A Datasets

We evaluated CBCL-PR on two object classification datasets that have been used
previously to test class-incremental learning techniques: CIFAR-100 [43] and
Caltech-101 [44] . CIFAR-100 contains 60,000 images of resolution $32\times
32$ belonging to 100 object classes. Each image class is composed of 500
training and 100 test images. Caltech-101 consists of 101 object classes with
8,677 total images and 40 to 800 images per class. The statistical details for
both of the datasets are described in Table I. These settings are in
accordance with the previous works [21, 42, 7] for a fair comparison. For FSIL
experiments, the training images for each class in Table I were changed to 5
and 10 for 5-shot and 10-shot incremental learning experiments, respectively,
while all the other statistics were kept the same. We report top-1 accuracy in
each increment and the average of the accuracies achieved over all increments
(termed as average incremental accuracy).

We performed all of our experiments 10 times with different random seeds to
randomize the order of the classes in different experiments. Average and
standard deviation of the accuracy achieved over the 10 runs are reported.

Figure 4: Accuracy curves on the CIFAR-100 dataset for all the models trained
with (a) 2, (b) 5, (c) 10, (d) 20 classes per increment. Mean and standard
deviation of the accuracies over 10 runs are reported for our method. Average
incremental accuracy is shown in parenthesis for each method. CBCL-PR_D shows
our method with the diagonal entries for the covariance matrices. (Best viewed
in color)

###  V-B Implementation Details

We used the Pytorch framework [45] to implement and train the neural network
models. We used the ResNet-18 and ResNet-34 [5] models pre-trained on the
ImageNet dataset [46] as feature extractors for the Caltech-101 and CIFAR-100
datasets, respectively. These model choices are consistent with [21, 42, 7]
for a fair comparison. For continual learning, when our method uses all the
training examples for each class, storing the covariance matrices can require
a large amount of memory. Hence, we also show results using our method when it
only stores the diagonal entries of the covariance matrices which reduces the
memory requirement by $\sim$99%, although the decrease in accuracy is only
$\sim$5%. For FSIL, as there are only a small number of clusters per class, we
stored the complete covariance matrices per class without requiring a huge
memory budget.

As mentioned in Section I, since none of the prior incremental learning
approaches are applicable on FSIL, we compare our method to CBCL and a few-
shot batch learning baseline (FLB) [28]. FLB also uses a pre-trained feature
extractor (ResNet) to get the feature vectors for the training images. The
feature vectors are then used to train a shallow linear classifier [28]. FLB
is not designed for continual learning and will suffer from catastrophic
forgetting if trained continually. Therefore, we train FLB using the training
data of all the old and new classes in each increment. In other words, FLB is
the batch learning baseline and, as such, should have a considerable advantage
in terms of classification accuracy over continual learning approaches.

For CBCL-PR, the distance threshold ($D$) for Agg-Var clustering was set to
20.0 in all the experiments. For the linear classifier used for
classification, we use a linear layer of the same size as the feature vectors
generated by the ResNet. Both FLB and the linear classifier in CBCL-PR were
trained for 25 epochs using the cross-entropy loss optimized with stochastic
gradient descent (with 0.9 as momentum) in the first increment and we kept on
increasing the number of epochs by 2 for each new increment. A fixed learning
rate of 0.001 with minibatches of size 8 were used for FLB and a fixed
learning rate of 0.01 and minibatches of size 64 were used for the linear
classifier in CBCL-PR.

###  V-C Results on the CIFAR-100 Dataset

We compare CBCL-PR to seven state-of-the-art (SOTA) methods on CIFAR-100
dataset: Finetuning (FT), LWM [21], LWF-MC [7], iCaRL [7], EEIL [8], BiC [9]
and CBCL [42]. All of these methods have been introduced in Section II. We
also compare our classification accuracy to a batch learning upperbound with a
ResNet-34 trained on the complete CIFAR-100 dataset in a single batch.

Figure 4 shows a comparison of CBCL-PR and CBCL-PR_D (CBCL-PR with only
diagonal entries of the covariance matrices) to other methods for class-
incremental learning experiments with 2, 5, 10 and 20 classes per increment.
CBCL-PR outperforms all seven approaches (including CBCL) on all increment
settings by significant margins. Note that iCaRL, EEIL and BiC store and use
the real images of the old classes, requiring more memory than CBCL-PR, but
they perform significantly lower accuracy than CBCL-PR. Even CBCL-PR_D
outperforms all seven approaches while using significantly lesser memory. The
accuracy of the other methods deteriorates quickly compared to CBCL-PR, which
shows the ability of our model to retain most of the past knowledge when
learning incrementally. While other approaches produce lower final accuracy
when learning with a smaller number of classes per increment (Figure 4 (a)),
CBCL-PR and CBCL-PR_D produce the same final accuracy for all increment
settings. It should also be noted that when using the complete covariance
matrices our method achieves an accuracy that is only $\sim$1% lower than the
batch learning upperbound. These results show that, unlike batch learning
systems, our approach has the advantage of learning continually without
storing all the original data, while also achieving performance similar to the
batch learning systems.

For this experiment, the maximum number of clusters that were allowed to be
stored by our model was 1600, which requires a total memory of 1.63 MB.
However, other approaches [7, 8, 9] usually store $\sim$2000 images for the
previously learned classes, which requires 17.6 MB. Thus, our approach is
computationally more efficient which allows for application on real-world
systems with limited computational resources.

TABLE II: Results on FSIL settings in terms of average incremental accuracy (%) on the CIFAR-100 dataset. | Classes per increment
---|---
Methods |  k-Shot |  2 |  5 |  10 |  20
FLB |  5 |  41.1 |  39.9 |  41.3 |  44.4
CBCL |  5 |  56.9 |  55.6 |  54.7 |  53.8
CBCL-PR |  5 |  57.9 |  57.2 |  57.0 |  53.9
FLB |  10 |  53.5 |  52.4 |  55.1 |  55.6
CBCL |  10 |  61.9 |  61.4 |  61.3 |  60.7
CBCL-PR |  10 |  64.5 |  63.8 |  62.5 |  60.7

####  V-C1 FSIL Results

Table II shows a comparison of CBCL-PR with CBCL and FLB for the few-shot
incremental learning experiments in terms of average incremental accuracy. For
both 5-shot and 10-shot incremental learning and all four increment settings,
CBCL-PR outperforms FLB by $\sim$11-17%. Note that FLB is the batch learning
baseline and has the advantage of training on the data of all the classes,
still CBCL-PR outperforms FLB significantly. Similar to continual learning
results, CBCL-PR produces higher accuracy than CBCL for all FSIL settings.
Also, note that CBCL-PR outperforms FLB with the largest margin for 5-shot
incremental learning and using 2 classes per increment. These results show
that our method is best suitable for real-world situations with limited data
in a single increment.

We also compare CBCL-PR with other SOTA approaches for FSIL on the CIFAR-100
dataset. All of these approaches have been introduced in Section II. For a
fair comparison with these approaches, we use the training and test settings
proposed in TOPIC [31] and CEC [32], where a large number of CIFAR-100 classes
are learned in the first increment and the rest of the classes are learned in
10 increments. Table III compares the average incremental accuracy of all the
approaches with CBCL-PR for FSIL. NCM, iCaRL and EEIL produce significantly
lower accuracy compared to CBCL-PR, as these methods are not specifically
designed to solve FSIL. Compared to the other two methods (TOPIC and CEC) that
are specifically designed to solve FSIL, CBCL-PR still outperforms them by
margins of 18.9% and 2.6%, respectively.

TABLE III: Comparison of CBCL-PR with SOTA methods for FSIL on the CIFAR-100 dataset. Methods |  Accuracy (%)
---|---
NCM [18] |  36.9
iCaRL [7] |  35.5
EEIL [8] |  37.5
TOPIC [31] |  43.1
CEC [32] |  59.4
CBCL-PR |  62.0

The time required to generate a feature vector for an image is about 0.6
milliseconds, the time required to generate clusters for all of the images in
an increment is $\sim$1 milliseconds, while pseudo-rehearsal takes an
approximate maximum of 14 seconds during the last increment when the system
must generate and train on pseudo-exemplars for all of the prior classes.
During testing, feature extraction and prediction of an image takes about
$\sim$1.2 milliseconds. These results show that our approach has fast training
and test times, making it suitable for domestic robot applications when the
robot is required to learn and make predictions in near real time.

Figure 5: (Left) Results on the Caltech-101 dataset for continual learning
with 10 classes per increment. Mean and standard deviation of the
classification accuracy (%) in each increment over the 10 runs is reported.
(Right) Comparison of prediction time (in milliseconds) of CBCL and CBCL-PR
corresponding to the total number of centroids/clusters learned in 10
increments on the Caltech-101 dataset.

###  V-D Results on the Caltech-101 Dataset

Figure 5 (left) shows a comparison of CBCL-PR and CBCL-PR_D to FT, LWM, and
CBCL on the Caltech dataset with 10 classes per increment. CBCL-PR and CBCL-
PR_D both outperform LWM by $\sim$44% and $\sim$43%, respectively, after
learning all 100 classes. Our methods also outperform CBCL by a margin of
$\sim$6%. Similar to the results on CIFAR-100, FT suffers from catastrophic
forgetting and its final classification accuracy is 69.52% lower than the base
accuracy (in the first increment). LWM produces better performance than FT but
its final accuracy also decreases by 49.36% with respect to the base accuracy.
In contrast, for CBCL-PR and CBCL-PR_D the decrease in accuracy is only 5.8%
and 6.84%, which indicate that our method does not suffer from catastrophic
forgetting.

####  V-D1 FSIL Results

For 5-shot and 10-shot incremental learning, CBCL-PR produces 88.67% and
91.54% average incremental accuracy, respectively. FT and LWM, however,
produce only 43.64% and 62.67% average incremental accuracy even when they use
the complete training set for each class. Thus, CBCL-PR for 5-shot incremental
learning produces 26% higher accuracy than LWM, which uses all the training
examples for each class. CBCL produces 87.70% and 89.92% average incremental
accuracy for 5-shot and 10-shot incremental learning, respectively, which is
$\sim$1% lower than CBCL-PR. These results are in accordance with the FSIL
results on CIFAR-100.

We also tested FLB on 5-shot and 10-shot incremental learning and compared it
with CBCL-PR. For 5-shot and 10-shot incremental learning, FLB produces 72.48%
and 83.81% average incremental accuracy, respectively. CBCL-PR outperforms FLB
with a significant margin (16.19% and 7.71% lower) for both 5-shot and 10-shot
incremental learning. These results reinforce the CIFAR-100 results,
demonstrating the value of our method for FSIL.

####  V-D2 Prediction Time Analysis

Figure 5 (right) compares the prediction times of CBCL and CBCL-PR
corresponding to the total number of centroids/clusters learned in 10
increments on the Caltech-101 dataset. The results clearly show that the
prediction time of CBCL-PR is significantly lower than CBCL. Further, the
prediction time of CBCL-PR remains constant irrespective of the total number
of centroids learned. In contrast, CBCL’s prediction time increases
drastically with the increase in the number of centroids learned.

###  V-E Ablation Study

We performed an ablation study to better understand CBCL-PR’s performance and
to quantify the contribution of different stages of processing towards the
overall performance. This experiment was performed on the CIFAR-100 dataset
with increments of 10 classes and memory budget of $K=$1600 clusters using all
the training data per class. Only the diagonal entries of the covariance
matrices are stored for this study. We report average incremental accuracy.

We created hybrid versions of CBCL-PR to ablate the clustering and pseudo-
rehearsal components. Trad-Agg-PR uses traditional agglomerative clustering,
k-means-PR uses $K$-means clustering to generate centroids and covariance
matrices for all the image classes. CBCL does not use pseudo-rehearsal.
Instead, it directly uses a $k$-Nearest Neighbors–based weighted voting scheme
on the centroids (generated by Agg-Var clustering) for image classification
[42]. Trad-Agg uses traditional agglomerative clustering with a weighted
voting scheme for classification (no pseudorehearal). k-means uses $K$-means
clustering with the weighted voting scheme for classification. Lastly, NCM
uses an NCM classifier with a pre-trained feature extractor. For each of the
hybrids, all other components remained the same as in CBCL-PR.

TABLE IV: Effect on average incremental accuracy by switching off each component separately in CBCL-PR. Methods |  Accuracy (%)
---|---
Trad-Agg-PR |  64.7
k-means-PR |  65.0
CBCL |  69.0
Trad-Agg |  59.2
k-means |  60.0
NCM |  58.2
CBCL-PR |  73.6

Table IV shows the results of the ablation study. All of the hybrid methods
are less accurate than the complete CBCL-PR algorithm. The most drastic
decrease in accuracy is seen for Trad-Agg, k-means, and NCM hybrids. Among
these three hybrids, NCM shows the lowest accuracy because it only uses a
single centroid to represent each class and simply uses the closest centroid
for the classification of an unlabeled image. Both Trad-Agg and k-means are
significantly inferior to CBCL-PR ($\sim$14.0%) because they use different
clustering techniques and do not use pseudo-rehearsal. Trad-Agg-PR and
k-means-PR achieve higher accuracy than simple Trad-Agg and k-means showing
the significance of pseudo-rehearsal towards the overall accuracy.
Particularly, it shows that pseudo-rehearsal technique is effective at
regenerating feature vectors that can cover the original feature space which
helps improve the accuracy of the classifier. In contrast, for the weighted
voting scheme, the centroids simply represent the mean of the feature vectors
of the classes and not the entire feature space, which results in the loss of
some information and produces inferior accuracy. CBCL achieves the highest
accuracy among all the hybrids since it uses the Agg-Var clustering to
generate centroids. Still, its accuracy is lower than CBCL-PR because it does
not use pseudo-rehearsal. Note that even CBCL beats the state-of-the-art
approaches, which shows that Agg-Var clustering is the most important
component of our approach. Pseudo-rehearsal’s contribution is also
significant, increasing the accuracy of all approaches by a margin of
$\sim$4.0-5.0%.

###  V-F FSIL Experiments on a Robot

This experiment evaluates a robot’s capability to learn to classify new
objects continually using CBCL-PR from only a small number of labeled examples
(5 or 10) provided by a person. Note, that the only purpose of the robot in
this experiment was to use its camera to learn household objects and then
classify the test objects. The robot also moved its arm to point to the
classified object. Overall, this setup is the same as a traditional FSIL setup
applied to real household objects captured from the robot’s camera.

Figure 6: Some example images of object classes lipstick and glass learned by
the robot using the camera in the robot’s hand.

We used 22 object classes for the experiment. Each object class was composed
of 15 objects. We took pictures of 2 poses of each object, generating 30
images per class. For the 5-shot and 10-shot incremental learning experiments,
we used 5 and 10 training images per class. The rest of the images for each
class were used for testing. We trained the robot incrementally with 2 classes
per increment. Example images of the objects used in this experiment are shown
in Figure 6. Unlike the object images in the benchmark datasets, these images
are not ideal. The background of the objects is not perfect and some objects
are small in size compared to other objects.

CBCL-PR was compared to Fine-Tuning (FT), Few-shot Learning Baseline (FLB) and
CBCL. We conducted this experiment 10 times with randomized order of classes
presented to the robot in different increments. Average accuracies over all
ten rounds are presented as results.

Table V compares CBCL-PR against the three methods (FT, FLB, and CBCL) for
5-shot and 10-shot incremental learning. Similar to the results on benchmark
datasets, FT suffers from catastrophic forgetting. CBCL produces good results
on both 5-shot and 10-shot learning, achieving accuracy slightly lower than
FLB. CBCL-PR, however, produces the best results and even outperforms FLB
which does not learn continually. These results depict the effectiveness of
our approach for FSIL in real-world conditions using a physical robot.

During the teaching phase, the time required to collect a single object image
by the robot takes only $\sim$2 seconds and the time required to extract
features for the image takes about 0.6 milliseconds. In the learning phase,
the time required to learn clusters for a new batch of classes takes
approximately 1.0 milliseconds and pseudo-rehearsal takes approximately 3
seconds. During the prediction phase, the time required to extract the
features and to make a prediction about a test image takes $\sim$0.6
milliseconds.

TABLE V: Comparison of CBCL-PR to FT, FLB and CBCL for FSIL in terms of average incremental accuracy (%). k_shot |  FT |  FLB |  CBCL |  CBCL-PR
---|---|---|---|---
5 |  27.16 |  93.58 |  92.97 |  94.85
10 |  27.36 |  96.76 |  95.67 |  97.50

##  VI Conclusion

In this paper, we have presented a novel cognitively-inspired framework (CBCL-
PR) for a challenging but practical problem: few-shot incremental learning
(FSIL). Our approach uses ideas of concept learning to develop computational
techniques, such as Agg-Var clustering and pseudo-rehearsal to mitigate
catastrophic forgetting while learning incrementally from limited data.
Extensive experiments on two object classification datasets have shown that
CBCL-PR outperforms SOTA methods by significant margins for class-incremental
learning and FSIL. We also performed FSIL experiments on a robot for learning
household objects from a few examples provided by a human. Our results show
that CBCL-PR may have applications for FSIL on real robots.

One limitation of our approach is that it was not tested with real human
participants for learning household objects. In the FSIL experiment with a
real robot, household objects were taught by the experimenter. In our future
work, we plan to run user studies with human participants to continually teach
household objects to a robot. Further, the training time required by CBCL-PR
is similar to that of a batch learner as both techniques retrain on a similar
amount of data. We hope to improve the training time of our approach in the
future.

We believe that our approach can be applied in the real world for learning
objects continually from a few shots from likely impatient human users that
require quick results. We hope that this work will evolve into novel robotics
applications, such as domestic cleaning robots and industrial packaging robots
that can continually adapt for long-term operation in real-world environments.

## Acknowledgments

Preliminary version of this work was presented in [42, 47]. This work
significantly extends these works in several ways. First we have updated Agg-
Var clustering and introduced a memory consolidation (pseudo-rehearsal) phase
to improve on our previous approach (termed CBCL). Second, we have added a
prediction time analysis of CBCL to show its limitations. We have also
performed experiments on the Caltech-101 dataset to show the improvement in
prediction time by CBCL-PR. Third, we have performed new experiments using
CBCL-PR on two datasets and on a real robot. We have also added comparison of
CBCL-PR with most recent FSIL methods in our experiments. Finally, the paper
contains a more complete background and related work, and provides a more
detailed explanation of the entire architecture through various intuitive
examples.

This work was partially supported by the Air Force Office of Sponsored Re-
search contract FA9550-17-1-0017 and National Science Foundation grant
CNS-1830390. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily
reflect the views of the National Science Foundation.

## References

  * [1] W. C. Abraham and A. Robins, “Memory retention – the synaptic stability versus plasticity dilemma,” _Trends in Neurosciences_ , vol. 28, no. 2, pp. 73–78, 2005.
  * [2] M. Mermillod, A. Bugaiska, and P. BONIN, “The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects,” _Frontiers in Psychology_ , vol. 4, p. 504, 2013.
  * [3] M. J. Matarić, “Socially assistive robotics: Human augmentation versus automation,” _Science Robotics_ , vol. 2, no. 4, 2017.
  * [4] I. D. Inc., “Aido Robot | Ingen Dynamics Aido Robot | Family Robot,” Aug. 2019. [Online]. Available: http://www.aidorobot.com/
  * [5] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _CVPR_ , June 2016.
  * [6] R. M. French, “Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference,” _CogSci_ , pp. 335–340, 2019.
  * [7] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “iCaRL: Incremental classifier and representation learning,” in _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , July 2017.
  * [8] F. M. Castro, M. J. Marin-Jimenez, N. Guil, C. Schmid, and K. Alahari, “End-to-end incremental learning,” in _The European Conference on Computer Vision (ECCV)_ , September 2018.
  * [9] Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu, “Large scale incremental learning,” in _CVPR_ , June 2019.
  * [10] A. Ayub and A. R. Wagner, “Tell me what this is: Few-shot incremental object learning by a robot,” _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2020.
  * [11] A. Ayub and C. Fendley, “Few-shot continual active learning by a robot,” in _Advances in Neural Information Processing Systems_ , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 30 612–30 624.
  * [12] R. Kemker and C. Kanan, “Fearnet: Brain-inspired model for incremental learning,” in _ICLR_ , 2018.
  * [13] M. J. Sekeres, G. Winocur, and M. Moscovitch, “The hippocampus and related neocortical structures in memory transformation,” _Neuroscience Letters_ , vol. 680, pp. 39–53, 2018.
  * [14] C. R. Bowman and D. Zeithamova, “Abstract memory representations in the ventromedial prefrontal cortex and hippocampus support concept generalization,” _Journal of Neuroscience_ , vol. 38, no. 10, pp. 2605–2614, 2018.
  * [15] Z. M. Reagh and C. Ranganath, “What does the functional organization of cortico-hippocampal networks tell us about the functional organization of memory?” _Neuroscience Letters_ , vol. 680, pp. 69–76, 2018.
  * [16] M. L. Mack, B. C. Love, and A. R. Preston, “Building concepts one episode at a time: The hippocampus and concept formation,” _Neuroscience Letters_ , vol. 680, pp. 31–38, 2018.
  * [17] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in _NIPS Deep Learning and Representation Learning Workshop_ , 2015\.
  * [18] T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka, “Distance-based image classification: Generalizing to new classes at near-zero cost,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, no. 11, pp. 2624–2637, Nov 2013.
  * [19] J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, “Overcoming catastrophic forgetting in neural networks,” _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 114, no. 13, pp. 3521–3526, 2017.
  * [20] Z. Li and D. Hoiem, “Learning without forgetting,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 12, pp. 2935–2947, Dec 2018.
  * [21] P. Dhar, R. V. Singh, K.-C. Peng, Z. Wu, and R. Chellappa, “Learning without memorizing,” in _CVPR_ , June 2019.
  * [22] A. Ayub and A. R. Wagner, “Storing encoded episodes as concepts for continual learning,” _arXiv:2007.06637_ , 2020.
  * [23] O. Ostapenko, M. Puscas, T. Klein, P. Jahnichen, and M. Nabi, “Learning to remember: A synaptic plasticity driven framework for continual learning,” in _CVPR_ , June 2019, pp. 11 321–11 329.
  * [24] A. Ayub and A. Wagner, “EEC: Learning to encode and regenerate images for continual learning,” in _International Conference on Learning Representations (ICLR)_ , 2021.
  * [25] L. S. Lopes and A. Chauhan, “Open-ended category learning for language acquisition,” _Connection Science_ , vol. 20, no. 4, pp. 277–297, 2008.
  * [26] A. Chauhan and L. S. Lopes, “An experimental protocol for the evaluation of open-ended category learning algorithms,” in _2015 IEEE International Conference on Evolving and Adaptive Intelligent Systems (EAIS)_. IEEE, 2015, pp. 1–8.
  * [27] S. H. Kasaei, L. S. Lopes, and A. M. Tomé, “Local-lda: Open-ended learning of latent topics for 3d object recognition,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 42, no. 10, pp. 2567–2580, 2019\.
  * [28] W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang, “A closer look at few-shot classification,” in _International Conference on Learning Representations_ , 2019.
  * [29] S. Gidaris and N. Komodakis, “Dynamic few-shot visual learning without forgetting,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * [30] M. Ren, R. Liao, E. Fetaya, and R. S. Zemel, “Incremental few-shot learning with attention attractor networks,” in _Advances in Neural Information Processing Systems (NeurIPS)_ , 2019.
  * [31] X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y. Gong, “Few-shot class-incremental learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [32] C. Zhang, N. Song, G. Lin, Y. Zheng, P. Pan, and Y. Xu, “Few-shot incremental learning with continually evolved classifiers,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021, pp. 12 455–12 464.
  * [33] A. Ayub and A. R. Wagner, “F-siol-310: A robotic dataset and benchmark for few-shot incremental object learning,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , 2021, pp. 13 496–13 502.
  * [34] ——, “What am i allowed to do here?: Online learning of context-specific norms by pepper,” in _Social Robotics_ , A. R. Wagner, D. Feil-Seifer, K. S. Haring, S. Rossi, T. Williams, H. He, and S. Sam Ge, Eds. Cham: Springer International Publishing, 2020, pp. 220–231.
  * [35] M. O. Turkoglu, F. B. T. Haar, and N. van der Stap, “Incremental learning-based adaptive object recognition for mobile robots,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , Oct 2018, pp. 6263–6268.
  * [36] M. Dehghan, Z. Zhang, M. Siam, J. Jin, L. Petrich, and M. Jagersand, “Online object and task learning via human robot interaction,” in _2019 International Conference on Robotics and Automation (ICRA)_ , May 2019, pp. 2132–2138.
  * [37] S. Valipour, C. P. Quintero, and M. Jägersand, “Incremental learning for robot perception through hri,” _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pp. 2772–2777, 2017.
  * [38] L. Genzel, J. I. Rossato, J. Jacobse, R. M. Grieves, P. A. Spooner, F. P. Battaglia, G. Fernández, and R. G. M. Morris, “The yin and yang of memory consolidation: Hippocampal and neocortical,” _PLOS Biology_ , vol. 15, no. 1, 2017.
  * [39] S. Gais, G. Albouy, M. Boly, T. T. Dang-Vu, A. Darsaud, M. Desseilles, G. Rauchs, M. Schabus, V. Sterpenich, G. Vandewalle, P. Maquet, and P. Peigneux, “Sleep transforms the cerebral trace of declarative memories,” _Proceedings of the National Academy of Sciences_ , vol. 104, no. 47, pp. 18 778–18 783, 2007.
  * [40] A. Robins, “Catastrophic forgetting, rehearsal and pseudorehearsal,” _Connection Science_ , vol. 7, no. 2, pp. 123–146, 1995.
  * [41] Y. LeChun, “The mnist database of handwritten digits,” 1998. [Online]. Available: http://yann.lecun.com/exdb/mnist/
  * [42] A. Ayub and A. R. Wagner, “Cognitively-inspired model for incremental learning using a few examples,” in _CVPR Workshops_ , June 2020.
  * [43] A. Krizhevsky, “Learning multiple layers of features from tiny images,” 2009, technical report, University of Toronto.
  * [44] L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 28, no. 4, pp. 594–611, April 2006.
  * [45] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_. Curran Associates, Inc., 2019, pp. 8024–8035.
  * [46] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “Imagenet large scale visual recognition challenge,” _Int. J. Comput. Vision_ , vol. 115, no. 3, pp. 211–252, Dec. 2015.
  * [47] A. Ayub and A. R. Wagner, “Centroid based concept learning for rgb-d indoor scene classification,” in _British Machine Vision Conference (BMVC)_ , 2020\.
  * [48] A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: a review,” _ACM computing surveys (CSUR)_ , vol. 31, no. 3, pp. 264–323, 1999.

##  Appendix A Cluster Reduction

Implementations on real hardware only have access to limited memory (i.e. disk
space). It is thus important to consider the memory footprint required for a
continual learning algorithm [7]. We therefore propose a novel method that
restricts the number of clusters while attempting to maintain classification
accuracy.

Assume that a system can store a maximum of $K$ clusters (centroids and
covariance matrices) and that it currently has stored $K_{t}$ clusters for $t$
classes. For the next batch of classes the system needs to store $K_{new}$
more clusters but the total number of clusters is $K_{t}+K_{new}>K$. Hence,
the system needs to reduce the total stored clusters to
$K_{r}=K_{t}+K_{new}-K$. Rather than reducing the number of clusters for each
class equally, CBCL-PR reduces the clusters for each class proportional to the
number of clusters for the class before reduction. The reduction in the number
of clusters $N_{y}^{{}^{\prime}}$ for each class $y$ is calculated as (whole
number):

$N^{{}^{\prime}}_{y}(new)=N_{y}^{{}^{\prime}}(1-\frac{K_{r}}{K_{t}})$ (4)

where $N_{y}^{{}^{\prime}}(new)$ is the number of clusters for class $y$ after
reduction. Rather than simply removing the extra clusters from each class, we
cluster the closest clusters in each class to get new clusters, keeping as
much information as possible about the previous classes. This process is
accomplished by applying k-means clustering [48] on the cluster set of each
class $y$ to reduce them to a total of $N_{y}^{{}^{\prime}}(new)$ clusters.

##  Appendix B Analysis of Different Memory Budgets on CIFAR-100

We perform a set of experiments on the CIFAR-100 dataset to analyze the effect
of different memory budgets on the performance of CBCL-PR. We only use the
diagonal entries of covariance matrices for this set of experiments. To show
the significance of our cluster reduction technique, we also show results when
clusters are simply removed when the memory limit is reached. Figure 7
compares the average incremental accuracy of CBCL-PR with our proposed cluster
reduction technique and with simple removal of clusters for different memory
budgets. As expected, both approaches achieve higher accuracy when provided
with higher memory budgets. However, CBCL-PR using the cluster reduction
technique constantly outperforms CBCL-PR with simple removal of clusters for
all memory budgets (except for $K=1600$ when there is no need for any
reduction), and the performance gap increases for smaller memory budgets. This
clearly shows the effectiveness of our proposed cluster reduction technique
over simple removal of clusters. Furthermore, it should be noted that even for
only $K=1000$ clusters, CBCL-PR’s average incremental accuracy (65.4%) is
higher than that of the state-of-the-art methods.

Figure 7: Average incremental accuracy of CBCL-PR with the centroid reduction
technique and centroid removal for different memory budgets (K: total number
of clusters). Figure 8: Comparison of the original feature space (10 feature
vectors per class) with 80 pseudo-exemplars generated by 6 clusters (3 per
class) for 2 classes in the MNIST dataset.

##  Appendix C Illustration of Pseudorehearsal for FSIL

An argument can be made that applying pseudorehearsal to Gaussian
distributions learned through FSIL could be vulnerable. It could lead to
unexpected features and, thus, negative classification performance. To test
this, we showed on CIFAR-100 dataset (Table II in the main paper) that
pseudorehearsal actually improves the overall accuracy instead of effecting it
negatively. To further illustrate this, we performed a similar experiment as
in Section III-B with 2 classes of the MNIST dataset. However, instead of
using all the feature vectors, we used only 10 feature vectors per class
(Figure 8 (left)). The 2D space covered by the 40 pseudo-exemplars per class
looks similar to the original feature space, although it is more dense than
the original (Figure 8 (right)). This shows that CBCL-PR is able to generate
more data covering the original feature space to help improve the
classification performance.