#  $\text{CL}^{2}\text{R}$: Compatible Lifelong Learning Representations

Niccoló Biondi  https://orcid.org/0000-0003-1153-1651 University of
FlorenceV.le Morgagni, 65FlorenceItaly50134<EMAIL_ADDRESS>,
Federico Pernici  0000-0001-7036-6655 University of FlorenceV.le Morgagni,
65FlorenceItaly50134<EMAIL_ADDRESS>,  Matteo Bruni
0000-0003-2017-1061 University of FlorenceV.le Morgagni, 65FlorenceItaly50134
<EMAIL_ADDRESS>,  Daniele Mugnai  1234-5678-9012 University of
FlorenceV.le Morgagni, 65FlorenceItaly50134<EMAIL_ADDRESS>and
Alberto Del Bimbo  0000-0002-1052-8322 University of FlorenceV.le Morgagni,
65FlorenceItaly50134<EMAIL_ADDRESS>
###### Abstract.

In this paper, we propose a method to partially mimic natural intelligence for
the problem of lifelong learning representations that are compatible. We take
the perspective of a learning agent that is interested in recognizing object
instances in an open dynamic universe in a way in which any update to its
internal feature representation does not render the features in the gallery
unusable for visual search. We refer to this learning problem as Compatible
Lifelong Learning Representations ($\text{CL}^{2}\text{R}$) as it considers
compatible representation learning within the lifelong learning paradigm. We
identify stationarity as the property that the feature representation is
required to hold to achieve compatibility and propose a novel training
procedure that encourages local and global stationarity on the learned
representation. Due to stationarity, the statistical properties of the learned
features do not change over time, making them interoperable with previously
learned features. Extensive experiments on standard benchmark datasets show
that our $\mathrm{CL^{2}R}$ training procedure outperforms alternative
baselines and state-of-the-art methods. We also provide novel metrics to
specifically evaluate compatible representation learning under catastrophic
forgetting in various sequential learning tasks. Code at
https://github.com/NiccoBiondi/CompatibleLifelongRepresentation.

Deep Learning, Compatible Learning, Lifelong Learning, Representation
Learning, Fixed Classifier

††ccs: Computing methodologies Visual content-based indexing and retrieval

##  1\. Introduction

The universe is dynamic and the emergence of novel data and new knowledge is
unavoidable. The unique ability of natural intelligence to lifelong learning
is highly dependent on memory and knowledge representation (Ericsson and
Kintsch, 1995). Through memory and knowledge representation, natural
intelligent systems continually search, recognize, and learn new objects in an
open universe after exposure to one or a few samples. Memory is substantially
a cognitive function that encodes, stores, and retrieves knowledge. Artificial
representations learned by Deep Convolutional Neural Network (DCNN) models
(Bengio et al., 2013; Taigman et al., 2014; Sun et al., 2014; Sharif Razavian
et al., 2014; Yosinski et al., 2014) stored in a memory bank (i.e., the
gallery-set) have been shown to be very effective in searching and recognizing
objects in an open-set/open-world learning context. Successful examples are
face recognition (Chopra et al., 2005; Schroff et al., 2015; Deng et al.,
2019), person re-identification (Zheng et al., 2015, 2016; Zhou et al., 2019)
and image retrieval (Yandex and Lempitsky, 2015; Gordo et al., 2016; Tolias et
al., 2016).

Figure 1.  Overview of the Compatible Lifelong Learning Representations
($\text{CL}^{2}\text{R}$) problem and proposed training procedure. The
learning agent searches object instances from query images $I_{\mathcal{Q}}$
without re-indexing the gallery-set. Any update to the internal feature
representation $\phi$ does not render the features in the gallery-set unusable
(i.e., no images are stored). Compatible feature representation under
catastrophic forgetting is learned imposing stationarity to features learned
from the the Class-incremental Learning surrogate task. Training is based on
rehearsal with the episodic memory $\mathcal{M}_{t}$.

These approaches rely on learning feature representations from static datasets
in which all images are accessible at training time. On the other hand,
dynamic assimilation of new data for lifelong learning suffers from
catastrophic forgetting: the tendency of neural networks to abruptly forget
previously learned information (McCloskey and Cohen, 1989; Ratcliff, 1990).

In the case of visual search, even avoiding catastrophic forgetting by
repeatedly training DCNN models on both old and new data, the feature
representation still irreversibly changes (Li et al., 2015). Thus, in order to
benefit from the newly learned model, features stored in the gallery must be
reprocessed and the “old” features replaced with the “new” ones. Reprocessing
not only requires the storage of the original images (a noticeable leap from
natural intelligence), but also their authorization to access them (Van
Noorden, 2020). More importantly, extracting new features at each update of
the model is computationally expensive or infeasible in the case of large
gallery-sets. The speed at which the representation is updated to benefit from
the newly learned data may impose time constraints on the re-indexing process.
This may occur from timescales on the order of weeks/months as in retrieval
systems or social networks (Shen et al., 2020), to within seconds as in
autonomous robotics or real-time surveillance (Pernici et al., 2018; Pernici
et al., 2020a). Recently in (Shen et al., 2020), a novel training procedure
has been proposed to avoid re-indexing the gallery-set. The representation
obtained in this manner is said to be compatible, as the features before and
after the learning upgrade can be directly compared. Training takes advantage
of all the data from previous tasks (i.e., no lifelong learning), guaranteeing
the absence of catastrophic forgetting. The advantage of considering
compatible representation learning within the lifelong learning paradigm, as
in this paper, is that compatible representation allows visual search systems
not only to distribute the computation over time, but also to avoid or
possibly limit the storage of images on private servers for gallery data. This
can have important implications for the societal debate related to privacy,
ethical and sustainable issues (e.g., carbon footprint) of modern AI systems
(Price and Cohen, 2019; Van Noorden, 2020; Schwartz et al., 2020; Cossu et
al., 2021).

We identify stationarity as the key requirement for feature representation to
be compatible during lifelong learning. Stationary features have been shown to
be biologically plausible in many studies of working memory in the prefrontal
cortex of macaques (Murray et al., 2017; Meyers, 2018; Libby and Buschman,
2021). The works (Murray et al., 2017; Meyers, 2018) decoded the information
from the neural activity of the working memory using a classifier with a
single fixed set of weights. They noted that a non-stationary feature
representation seems to be biologically problematic since it would imply that
the synaptic weights would have to change continuously for the information to
be continuously available in memory.

Inspired by this, in this paper, we formalize the problem of Compatible
Lifelong Learning Representations ($\text{CL}^{2}\text{R}$) in relation to the
relevant areas of compatible learning and lifelong (continual) learning. We
call any training procedure that aims to obtain compatible features and
minimize catastrophic forgetting as $\text{CL}^{2}\text{R}$ training, and we
propose (1) a novel set of metrics to properly evaluate
$\text{CL}^{2}\text{R}$ training procedures (2) a training procedure based on
rehearsal (Ratcliff, 1990; Robins, 1993) and feature stationarity (Pernici et
al., 2021a, b) to jointly address catastrophic forgetting and feature
compatibility. Fig. 1 provides an overview of the problem and the training
procedure. Specifically, our $\text{CL}^{2}\text{R}$ training procedure is
achieved by encouraging global and local stationarity to the learned features.

The paper is organized as follows. In Sec. 2, we discuss related work and in
Sec. 3, we highlight our contributions. Sec. 4 presents the formulation of
$\text{CL}^{2}\text{R}$, Sec. 5 proposes new metrics to evaluate
compatibility, and Sec. 6 describes a new training procedure. In Sec. 7, we
compare our results with adapted state-of-the-art methods. Sec. 8 presents the
ablation study.

##  2\. Related Work

#### Compatible Learning.

The work proposed in (Shen et al., 2020), called Backward-Compatible Training
(BCT), first formalizes the problem of learning compatible representation to
avoid re-indexing. The method takes advantage of an influence loss that
encourages the feature representation towards one that can be used by the old
classifier. The old classifier is fixed while learning with the novel data
(i.e., its parameters are no longer updated by back-propagation) and
cooperates with the new representation model. Cooperation is achieved by
aligning the prototypes of the new classifier with the prototypes of the old
fixed one. The underlying assumption is that the upgraded feature
representation follows the representation learned by the old classifier. BCT
has been evaluated in scenarios without the effects of catastrophic forgetting
by repeatedly training DCNN models on both old and new images (i.e., jointly
re-training from scratch at each upgrade). To compare with this learning
strategy in a lifelong learning scenario instead of starting from scratch
every time, we have added to BCT the capability of learning by fine-tuning the
previously learned model according to a memory based rehearsal strategy
(Ratcliff, 1990; Robins, 1993).

Compatibility under catastrophic forgetting has been implicitly studied in
(Iscen et al., 2020) (FAN), in which authors presented a method for storing
features instead of images in Class-incremental Learning (CiL). They introduce
a feature adaptation function to update the preserved features as the network
learns novel classes. We compared to this method by storing the updated-
preserved features obtained at each task. Although designed to improve
classification accuracy, the work can be considered close to a lifelong
learning approach with compatible representation in which the feature
adaptation function they defined, addresses implicitly the problem of feature
compatibility as in (Chen et al., 2019), (Hu et al., 2019), (Wang et al.,
2020), (Meng et al., 2021). Differently from BCT, these methods do not
completely prevents the cost of re-indexing since the learned mappings require
to be evaluated every time the dataset is upgraded and are therefore not
suited to lifelong learning and/or large gallery-set. For example, the
ResNet-101 architecture is one order slower than the mapping proposed in (Chen
et al., 2019), therefore, when the size of the gallery increases by an order
of magnitude, it is equivalent to re-index the images. The method described in
(Ramanujan et al., 2022), in addition to the current feature model, trains
from the same data an auxiliary model in a different way (i.e., using self-
supervised learning). The auxiliary feature model will then be used with
future learned models to learn a mapping model to obtain compatible
representations as in (Wang et al., 2020; Iscen et al., 2020; Meng et al.,
2021). The underlying assumption is that as the auxiliary feature model is
trained with a different strategy, it encodes different knowledge which may
facilitate learning the mapping between the representation spaces.

Compatibility of the representation in a more general sense has been
considered in (Li et al., 2015; Wang et al., 2018) where similarity between
features extracted from identical architectures and trained from different
initialization has been extensively evaluated. The work in (Budnik and
Avrithis, 2021) avoids re-indexing the gallery although the new model used for
queries is not trained on more data. Their work is motivated by the scenario
where the gallery is indexed by a large model and the queries are captured
from mobile devices in which the use of small models is the only viable
solution.

#### Lifelong Learning.

Lifelong Learning or Continual Learning (CL) studies the problem of learning
from a non i.i.d. stream of data with the goal of assimilating new knowledge
preventing catastrophic forgetting (Chen and Liu, 2018; McCloskey and Cohen,
1989). Methods for preventing catastrophic forgetting have been explored
primarily in the classification task, where catastrophic forgetting often
manifests itself as a significant drop in classification accuracy (Vijayan and
Sridhar, 2021; Delange et al., 2021; Masana et al., 2020; Parisi et al., 2019;
Belouadah et al., 2021). The key aspects that distinguish lifelong feature
learning for visual search from classification are: (i) categorical data often
have coarser granularity than visual search data; (ii) evaluation metrics do
not involve classification accuracy; and (iii) class labels are not required
to be explicitly learned. These differences may suggest that these two
catastrophic forgetting occurrences are of different origins. In this context,
recent works have discussed the importance of the specific task in assessing
catastrophic forgetting of learned representations (Davari and Belilovsky,
2021; Chen et al., 2020; Pu et al., 2021; Chen et al., 2021; Barletti et al.,
2022; Pernici et al., 2020b). Among others, empirical evidence presented in
(Davari and Belilovsky, 2021) suggests that feature forgetting is not as
catastrophic as classification forgetting and that many approaches that
address the problem of catastrophic forgetting do not improve feature
forgetting in terms of the usefulness of the representation. We argue that
such evidence is relevant in visual search and that it can be exploited with
techniques that further encourage learning compatible feature representation.
According to this, we consider CiL as the basic building blocks for the
general purpose of learning feature representation incrementally.

In this paper, the focus is on CiL methods based on Knowledge Distillation
(KD) (Hinton et al., 2015) and rehearsal (Robins, 1995) which are known to be
versatile, effective and widely applicable to reduce catastrophic forgetting.
We leverage the classification task in CiL as a surrogate task to learn
feature representation as typically performed in face/body-identification and
retrieval (Deng et al., 2019; Zheng et al., 2016; Tolias et al., 2016). The
work in (Li and Hoiem, 2017) first introduces KD in lifelong learning as an
effective way to preserve the knowledge previously acquired from old tasks. In
iCaRL (Rebuffi et al., 2017), KD is combined with rehearsal to reserve samples
of exemplars stored in an episodic memory for classes already seen. The BiC
work, proposed in (Wu et al., 2019), extends (Rebuffi et al., 2017) developing
a bias correction layer to recalibrate the output probabilities learning an
additional linear layer on a small set of data. Along a similar vein, in (Zhao
et al., 2020), the bias correction is performed by aligning the norms of the
weight vectors of the classifier for new classes to those for old classes
without using additional model parameters or reserved data. The work in
(Romero et al., 2015) introduces Feature Distillation (FD), a distillation
loss evaluated on the feature vectors instead on the classifier outputs. FD
has recently been successfully applied in (Hou et al., 2019) (LUCIR) and
(Douillard et al., 2020) to reduce catastrophic forgetting. Differently from
LUCIR, PODNet uses a spatial-based distillation loss to constrain the
statistics of intermediate features after each residual block. Similar to
LUCIR, PODNet and many others paper on Continual/Lifelong learning in
literature, our problem formulation takes advantage of the general concept of
KD. Differently from these works, our approach is novel in that it considers
feature distillation for the dual purpose of learning feature compatibility
and mitigating feature forgetting. The work in (Iscen et al., 2020) (FAN),
also discussed in the previous paragraph, combines strategies from (Hou et
al., 2019), (Li and Hoiem, 2017), (Rebuffi et al., 2017) to learn and preserve
previous features. Although the work does not consider the compatibility
problem, it is the closest work to our approach. Recently, (Yan et al., 2021)
(DER) has shown an interesting performance improvement in Class-incremental
Learning by freezing the previously learned representation and expanding its
dimension from a new learnable feature extractor. Despite the clear
improvements in classification performance, this has no trivial exploitation
in compatible training as the varying dimension across tasks do not allow
direct application of nearest neighbor search between models. Features with
different dimensions typically require to be projected into a common single
space to allow nearest-neighbor to be applied. The FOSTER method (Wang et al.,
2022) improves upon DER by addressing this specific problem by transforming
the growing dimension of the feature representation with a trainable linear
layer that maps the growing feature vector into a fixed dimension. More in
general, CiL methods addressing catastrophic forgetting are in a certain sense
related to compatible representation, since forgetting is the change in the
feature representation of classifiers that will be learned in the future. We
evaluate these methods as baselines to quantify the level of lifelong-
compatible representation they intrinsically may have.

##  3\. Main Contributions

  1. (1)

We consider compatible representation learning within the lifelong learning
paradigm. We refer to this general learning problem as Compatible Lifelong
Learning Representations ($\text{CL}^{2}\text{R}$).

  2. (2)

We define a novel set of metrics to properly evaluate $\text{CL}^{2}\text{R}$
training procedures.

  3. (3)

We propose a $\text{CL}^{2}\text{R}$ training procedure that imposes global
and local stationarity on the learned features to achieve compatibility
between representations under catastrophic forgetting. Global and local
interactions show a significant performance improvement when local
stationarity is promoted only from already observed samples in the episodic
memory.

  4. (4)

We empirically assess the effectiveness of our approach in several benchmarks
showing improvements over baselines and adapted state-of-the-art methods.

##  4\. $\text{CL}^{2}\text{R}$ Problem Formulation

In a $\text{CL}^{2}\text{R}$ setting, a sequence of representation models,
$\\{\phi_{t}\\}_{t=1}^{T}$, is learned incrementally with a sequence of $T$
tasks, $\\{(\mathcal{D}_{t},K_{t})\\}_{t=1}^{T}$, where $\mathcal{D}_{t}$ are
the images of the $t$-th task represented by $K_{t}$ different classes.
Specifically, each task is disjoint from the others: $K_{k}\cap
K_{t}=\emptyset$ with $t\neq k$. The learned representation model $\phi_{t}$
is used to transform the query images into feature vectors that are used to
retrieve the images most similar to a set of given gallery images transformed
with a previous model $\phi_{k}$. Specifically, we indicate with the couple
$\mathcal{G}=(I_{\mathcal{G}},F_{\mathcal{G}})$ the gallery-set, where
${I}_{\mathcal{G}}=\\{\mathbf{x}_{i}\\}_{i=1}^{N}$ is the image collection
from which the features $F_{\mathcal{G}}=\\{\mathbf{f}_{i}\\}_{i=1}^{N}$ are
extracted, and $N$ is the number of elements of the two sets. Without loss of
generality, we assume that the features in $F_{\mathcal{G}}$ are extracted
using the representation model
$\phi_{k}:{\mathbb{R}}^{D}\rightarrow{\mathbb{R}}^{d}$ that transforms an
image $\mathbf{x}\in{\mathbb{R}}^{D}$ into a feature vector
$\mathbf{f}\in{\mathbb{R}}^{d}$, where $d$ and $D$ are the dimensionality of
the feature and the image space, respectively. Analogously, we will refer to
$\mathcal{Q}=(I_{\mathcal{Q}},F_{\mathcal{Q}})$ as the query-set, where
$I_{\mathcal{Q}}$ and $F_{\mathcal{Q}}$ are the corresponding image-set and
the feature-set, respectively. As the $t$-th task becomes available, the model
$\phi_{t}$ is incrementally learned from the previous one along with the new
task data $\mathcal{D}_{t}$. Our goal is to design a training procedure to
learn the model $\phi_{t}$ so that any query image transformed with it can be
used to perform visual search through some distance ${\rm
dist}:{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}\rightarrow\mathbb{R}_{+}$ to
identify the closest features ${F}_{\mathcal{G}}$ to the query features
${F}_{\mathcal{Q}}$ without forgetting the previous representation and without
computing
$F_{\mathcal{G}}=\\{\mathbf{f}\in\mathbb{R}^{d}\,|\,\mathbf{f}=\phi_{t}(\mathbf{x})\,\forall\mathbf{x}\in
I_{\mathcal{G}}\\}$ (i.e., re-indexing).  If this holds, then the resulting
representation $\phi_{t}$ is said to be lifelong-compatible with $\phi_{k}$.

The main challenge of the $\text{CL}^{2}\text{R}$ problem is to jointly
alleviate catastrophic forgetting and learn a compatible representation
between the previously learned models. In Fig. 1 we illustrate the complete
$\text{CL}^{2}\text{R}$ training example using rehearsal to alleviate the
effects of catastrophic forgetting.

##  5\. Compatibility Evaluation

A representation model $\phi_{\rm new}$ upgraded with new data is said to be
compatible with an old representation model $\phi_{\rm old}$ when it holds
(Shen et al., 2020):

(1) $M\big{(}\phi_{\rm new}^{\mathcal{Q}},\phi_{\rm
old}^{\mathcal{G}}\big{)}>{M}\big{(}\phi_{\rm old}^{\mathcal{Q}},\phi_{\rm
old}^{\mathcal{G}}\big{)}.$

Eq. 1 represents the Empirical Compatibility Criterion (ECC), where ${M}$ is
an evaluation metric specific to the given visual search problem. Notable
examples of the metric $M$ can be found in face verification accuracy (Huang
et al., 2007; Learned-Miller, 2014), face verification/identification accuracy
in terms of true acceptance and false acceptance rate (TAR$@$FAR) (Klare et
al., 2015), person re-identification mean average precision (mAP) (Ye et al.,
2021). The intuition of these metrics is based on the observation that they
can be instantiated with two different representation models $\phi_{\rm new}$
and $\phi_{\rm old}$ when considering the query-gallery pair. The specific
notation ${M}\big{(}\phi_{\rm new}^{\mathcal{Q}},\phi_{\rm
old}^{\mathcal{G}}\big{)}$ defines the _cross-test_ between the new and the
old model and it represents the case in which $\phi_{\rm new}$ is used to
extract the features of the query-set, $F_{\mathcal{Q}}$, while $\phi_{\rm
old}$ is used to extract the gallery-set ones, $F_{\mathcal{G}}$.
${M}\big{(}\phi_{\rm old}^{\mathcal{Q}},\phi_{\rm old}^{\mathcal{G}}\big{)}$
is the self-test and it represents the case in which both query and gallery
features are extracted with $\phi_{\rm old}$. When the model is trained
incrementally on $T$ tasks, Eq. 1 is evaluated according to the Multi-model
Empirical Compatibility Criterion introduced in (Biondi et al., 2021):

(2) $\displaystyle
M\big{(}\phi_{t}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}>M\big{(}\phi_{k}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}{\rm\quad
with\>}t>k,$

where $t,k\in\\{1,2,\ldots,T\\}$ refer to two different tasks such that task
$k$ is processed by the model before task $t$. The model $\phi_{t}$ is
compatible with the model $\phi_{k}$, when the cross-test
$M\big{(}\phi_{t}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}$ between
$\phi_{t}$ and $\phi_{k}$ is greater than the self-test
$M\big{(}\phi_{k}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}$ of the model
$\phi_{k}$. The underlying intuition is that if the performance of matching
the gallery feature vectors extracted with the old model with the query
feature vectors extracted with the new model (i.e., cross-test) is better than
the performance of matching the gallery feature vectors with the query feature
vectors both extracted with the old model (i.e., self-test), then the system
is learning compatible representations. That is, learning from the new task
data improves the representation without breaking the compatibility with the
previously learned model. Based on Eq. 2, the Compatibility Matrix $C$ is
defined as:

(3)
$C_{t,k}=\begin{cases*}M\big{(}\phi_{t}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}&if
$t>k$\\\ M\big{(}\phi_{k}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}&if
$t=k$\\\ \qquad 0&if $t<k$\end{cases*},$

where the element in the row $t$ and the column $k$ of the compatibility
matrix denotes the evaluation metric $M$ of the model $t$ to the model $k$.
This definition combines the basic intuition of the classification accuracy
matrix $R$ defined in (Lopez-Paz and Ranzato, 2017; Díaz-Rodríguez et al.,
2018), used to evaluate the Class-incremental Learning problem (CiL), with the
two specific aspects that distinguish the $\text{CL}^{2}\text{R}$ learning
setting from the CiL one. Namely: (a) in CiL at each task, the train and test
data are sampled from the same distribution, while in $\text{CL}^{2}\text{R}$
the test-set classes are sampled from an unknown distribution (i.e.,
$\text{CL}^{2}\text{R}$ addresses the open-set recognition problem); (b) in
CiL the test-set is dynamic (i.e., it grows including images from the task
distributions) while in $\text{CL}^{2}\text{R}$ it is assumed static for the
purpose of a reliable evaluation (Shen et al., 2020). In the
$\text{CL}^{2}\text{R}$ setting, a dynamic test-set, as used in CiL, is of
difficult definition, as there are infinite ways to make the gallery dynamic
and each of them may change unexpectedly the performance of the evaluation. We
follow (Shen et al., 2020) and perform the evaluation assuming a static test-
set (i.e., a static query-gallery pair). According to this, we set the
elements of the matrix $C$ with $t<k$ to zero to indicate the impossibility of
a reliable evaluation of a growing test-set that should be sampled from an
unknown changing distribution. For the remaining elements, the cross-test
values are the elements of the matrix with $t>k$, while the self-test are the
values of the main diagonal (i.e., when $t=k$). Given a compatibility matrix
$C$, the Average Compatibility ($AC$) is defined as follows:

(4) $AC=\frac{2}{T(T-1)}\sum\limits_{1\leq k<t\leq
T}\mathds{1}{\Big{(}M\big{(}\phi_{t}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}>M\big{(}\phi_{k}^{\mathcal{Q}},\phi_{k}^{\mathcal{G}}\big{)}}\Big{)},$

where $\mathds{1}(\cdot)$ denotes the indicator function. The Average
Compatibility ($AC$) summarizes the Compatibility Matrix values in a single
number that quantifies the number of times that compatibility is verified
against all possible $\frac{T(T-1)}{2}$ occurrences.

###  5.1. Proposed $\text{CL}^{2}\text{R}$ Metrics

The work in (Lopez-Paz and Ranzato, 2017; Díaz-Rodríguez et al., 2018)
proposes a set of metrics to assess the ability of the learner to transfer
knowledge based on a matrix that reports the test classification accuracy of
the model on task $j$ after learning task $i$. Along a similar vein, we
present a set of metrics to evaluate the compatibility between representation
models in a compatible lifelong learning setting.

Let $C\in\mathbb{R}^{T\times T}$ be the compatibility matrix of the Eq. 3 for
$T$ tasks, the proposed criteria are the following:

  1. (1)

_Backward Compatibility_ ($BC$) measures the gap in compatibility performance
between the representation learned at task $T$ with respect to the
representation learned at task $k$ with $k\in\\{1,\ldots,T-1\\}$. When $BC$
$<0$ the learning procedure is also influenced by catastrophic forgetting
because the performance degrades with newer learned tasks. $BC$ is defined as
follows:

(5)
$\mbox{\normalsize$BC$}=\frac{1}{T-1}\sum_{k=1}^{T-1}\big{(}C_{T,k}-C_{k,k}\big{)}$

  2. (2)

_Forward Compatibility_ ($FC$) estimates the influence that learning a
representation on a task $k-1$ has on the compatibility performance of the
representation learned on a future task $k$ by comparing the cross-test
(between models at task $k$ and $k-1$) with respect to the self-test at task
$k$. $FC$ $\geq 0$ denotes that, on average, the cross-test values are greater
than the self-test evaluated on the subsequent tasks, therefore, re-indexing
does not necessarily provide improved results. $FC$ is defined as follows:

(6)
$\mbox{\normalsize$FC$}=\frac{1}{T-1}\sum_{k=2}^{T}\big{(}C_{k,k-1}-C_{k,k}\big{)}.$

The intuition behind the definition of this metric comes from noticing that as
the number of tasks increases, the cross-test may result better than the self-
test. As this is not typically observed when there is no catastrophic
forgetting (i.e., when repeatedly training with new and old data), we argue
this is due to the joint interaction between the compatibility constraint and
catastrophic forgetting. This observation led us to define something
“positive” when the compatible representation with the previously learned
model is higher than the self-test of the current model. This metric is
designed to yield high values when a $\text{CL}^{2}\text{R}$ training
procedure is able to positively exploit the joint interaction between feature
forgetting and compatible representation.

From Eqs. 5 and 6, it can be deduced that $BC$ and $FC$ $\in[-1,1]$. Backward
compatibility for the first task and forward compatibility for the last task
are not defined. The larger these metrics, the better the model. When $AC$
values are comparable, both $BC$ and $FC$ represent two metrics that quantify
the positive interaction between search accuracy under catastrophic forgetting
and compatibility. This allows evaluating how catastrophic forgetting affects
the representation and its compatibility.

As $BC$ evaluates the relationship between the representations learned at the
final task $T$ and the previous ones, it is possible to follow their evolution
during $\text{CL}^{2}\text{R}$ training. According to this, we define the
Backward Compatibility at task $t$ as
$BC{(t)}=\frac{1}{t-1}\sum_{\begin{subarray}{c}k=1\end{subarray}}^{t-1}\big{(}C_{t,k}-C_{k,k}\big{)},\;{\rm
with}\;t>1$ where $t\in\\{1,2,\ldots,T\\}$. This represents the average of the
element-wise difference between the $t$-th row and the first $t$ elements of
the main diagonal on the compatibility matrix.

##  6\. Proposed $\text{CL}^{2}\text{R}$ Training

To achieve compatibility, we encourage global and local stationarity to the
feature representation. Global stationarity is encouraged according to the
approach described in (Pernici et al., 2021a), in which features are learned
to follow a set of special fixed classifier prototypes. The work (Pernici et
al., 2021a) imposes global stationarity using a classifier in which prototypes
cannot be trained (i.e., fixed) and are set before training. Under this
condition, only the direction of the features aligns toward the fixed
directions of the classifier prototypes and not the opposite. This constraint
imposes learned features to follow their corresponding fixed prototypes,
therefore, encouraging representation stationarity. The lack of trainable
classifier functionality is basically replaced by previous layers. Fixed
prototypes are set according to the coordinate vertices of a $d$-Simplex
regular polytope that, in addition to stationarity, allows maximally separated
features to be learned (Pernici et al., 2019a, b).

We take advantage of this result and perform Class-incremental Learning (CiL)
as surrogate task to learn stationary features’ representation to achieve
compatibility. More formally, let $\mathbf{W}\;\forall t\in\\{1,2,...,T\\}$ be
the $d$-Simplex fixed classifier, we instantiate the CiL problem as
$\sigma(\phi_{t}\circ\mathbf{W})$, where $\sigma$ indicates the softmax
function, and perform learning according to incremental fine-tuning. The
evolving training-set
$\mathcal{T}_{t}\leftarrow\mathcal{M}_{t}\cup\mathcal{D}_{t}$ is computed
according to a rehearsal base strategy using the episodic memory,
$\mathcal{M}_{t}$ which contains an updating set of samples from
$\\{\mathcal{D}_{1},...,\mathcal{D}_{t-1}\\}$. The memory is updated as
$\mathcal{M}_{t+1}\leftarrow\mathcal{M}_{t}\cup\textsc{Sampling}({\rm}D_{t})$.
The loss optimized in (Pernici et al., 2021a) is adapted to
$\text{CL}^{2}\text{R}$ training as follows:

(7)
$\displaystyle\mathcal{L}_{t}=-\dfrac{1}{|\mathcal{T}_{t}|}\sum\limits_{\mathbf{x}\in\mathcal{T}_{t}}\log\\!\Bigg{(}\dfrac{\exp{\big{(}{\mathbf{w}}_{y_{i}}^{\top}\cdot{\mathbf{\phi(\mathbf{x})}}}\big{)}}{\sum\limits_{\scriptscriptstyle
j\in
K_{s}}\exp\big{(}{{\mathbf{w}}_{j}^{\top}\cdot{\mathbf{\phi(\mathbf{x})}}}\big{)}+\sum\limits_{\scriptscriptstyle
j\in
K_{u}}\exp{\big{(}{\mathbf{w}}_{j}^{\top}\cdot{\mathbf{\phi(\mathbf{x})}}\big{)}}}\Bigg{)},$

where $K_{s}$ is the set of classes learned up to time $t$,
$|\mathcal{T}_{t}|$ is the number of elements in the training-set, $K_{u}$ is
the set of the outputs of the classifier that have not yet been assigned to
classes at time $t$ (i.e., future unseen classes (Pernici et al., 2021b)),
$\mathbf{w}^{\top}_{\leavevmode\resizebox{4.69496pt}{}{$(\cdot)$}}$ is a class
prototype of the fixed classifier $\mathbf{W}$, and $y_{i}$ is the supervising
label. In particular, $\mathbf{W}$ is the weight matrix of the fixed
classifier, which does not undergo learning during model training. In (Pernici
et al., 2021a) the $d$-Simplex prototypes are defined as
$\mathbf{W}=\\{e_{1},e_{2},\dots,e_{d-1},\alpha\sum_{i=1}^{d-1}e_{i}\\},$
where $d$ the feature dimensionality of the $d$-Simplex,
$\alpha=\frac{1-\sqrt{d+1}}{d}$, and $e_{i}$ denotes the standard basis in
$\mathbb{R}^{d-1}$, with $i\in\\{1,2,\dots,d-1\\}$.

The loss of Eq. 7 imposes global stationarity and does not require any
knowledge to be extracted from the previously learned models. However,
catastrophic forgetting causes misalignment between features and fixed
classifier prototypes. Therefore, we further impose additional stationarity
constraints in a local neighborhood of a feature by encouraging the current
model to mimic the feature representation of the model previously learned.
This allows the overall stationarity to also be determined by a local learning
mechanism interacting with the global one provided by the $d$-Simplex
classifier of Eq. 7. The global-to-local interaction is achieved through the
Feature Distillation loss (FD) (Romero et al., 2015). Differently from the
more common practice of FD in CiL (Jung et al., 2018; Hou et al., 2019;
Douillard et al., 2020) in which each mini-batch is sampled from both the
episodic memory and the current task (i.e.,
$\mathcal{T}_{t}\leftarrow\mathcal{M}_{t}\cup\mathcal{D}_{t}$), we evaluate
the FD loss, at each task $t$, only on the samples stored in episodic memory
$\mathcal{M}_{t}$ observed from previous tasks:

(8)
$\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}=\frac{1}{|\mathcal{M}_{t}|}\sum_{\mathbf{x}_{i}\in\mathcal{M}_{t}}\bBigg@{2.5}(1-\frac{\phi_{t}(\mathbf{x}_{i})\cdot\phi_{t-1}(\mathbf{x}_{i})}{\left\lVert\phi_{t}(\mathbf{x}_{i})\right\rVert\left\lVert\phi_{t-1}(\mathbf{x}_{i})\right\rVert}\bBigg@{2.5}),$

where $\phi_{t-1}$ is the model learned from the previous task. This
encourages local stationarity and stability from only the previous classes in
the episodic memory and the assimilation of new knowledge (plasticity) from
only the classes of the current task. As confirmed by ablation in Sec. 8, this
learning strategy leads to a significant performance improvement. The final
optimized loss function is the sum of Eq. 8 and Eq. 7

(9)
$\mathcal{L}=\mathcal{L}_{t}+\lambda\;\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}},$

where $\lambda$ balances the contribution of global and local alignment
provided by the two losses. The pseudo-code in Algorithm 1 and in Algorithm 2
detail our training procedure and its application in visual search,
respectively.

Algorithm 1 $\text{CL}^{2}\text{R}$ Training Procedure

$\\{(\mathcal{D}_{t},K_{t})\\}_{t=1}^{T}$

Model $\phi_{t}$ with parameters $\Theta$ $\triangleright$ Current model
parameters

$\mathcal{M}_{1}\leftarrow\emptyset$

Initialize the $d$-Simplex fixed classifier $\mathbf{W}$

Set $\mathbf{W}{.\mathtt{requires\\_grad=False}}$ $\triangleright$ Exclude
from training

for each task $t=\\{1,2,...,T\\}$ do

$\mathcal{T}_{t}\leftarrow\mathcal{M}_{t}\cup\mathcal{D}_{t}$ $\triangleright$
Current training-set of task $t$

Update the model $\sigma(\phi_{t}\circ\mathbf{W})$ on the training-set
$\mathcal{T}_{t}$

$\mathcal{L}_{t}(\Theta)\big{|}_{\mathcal{T}_{t}}+\lambda\cdot\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}(\Theta)\big{|}_{\mathcal{M}_{t}}\leftarrow$
SGD

$\mathcal{M}_{t+1}\leftarrow\mathcal{M}_{t}\cup{\rm\textsc{Sampling}}(D_{t})$
$\triangleright$ Update memory with sampled data from $\mathcal{D}_{t}$

end for

Algorithm 2 Compatible $\text{CL}^{2}\text{R}$ Search

$\phi_{t}$ $\triangleright$ Model under learning by Alg. 1

$F_{\mathcal{G}}=\\{\mathbf{f}_{i}\\}_{i=1}^{N_{\mathcal{G}}}$
$\triangleright$ Gallery-set features

$I_{\mathcal{Q}}=\\{\mathbf{x}_{i}\\}_{i=1}^{N_{\mathcal{Q}}}$
$\triangleright$ Query images

$F_{\mathcal{Q}}$$\leftarrow\phi_{t}(I_{\mathcal{Q}})$ $\triangleright$
Extract query-set feature vectors

Search$(F_{\mathcal{Q}},F_{\mathcal{G}})$ $\triangleright$ Compatible Search

##  7\. Experimental Results

###  7.1. Datasets and Verification Protocol

We compare our proposed $\text{CL}^{2}\text{R}$ training procedure and the
baseline methods on several benchmarks: CIFAR10 (Krizhevsky et al., 2009),
ImageNet20111 To meet the open-set protocol, we generated a training set from
ImageNet (Russakovsky et al., 2015) by randomly sampling 20 classes that are
not included in the Tiny-ImageNet200 dataset. The indices of the ImageNet
classes we use are the following: {n02276258, n01728572, n03814906, n02817516,
n03769881, n03220513, n04442312, n04252225, n13037406, n04266014, n03929855,
n02804414, n01873310, n03532672, n01818515, n03916031, n03345487, n02114855,
n04589890, n03776460}., ImageNet100 (Rebuffi et al., 2017; Hou et al., 2019;
Wu et al., 2019), Labeled Face in the Wild (LFW) (Huang et al., 2007), and
IJB-C (Maze et al., 2018). Evaluation is performed in the open-set 1:1 search
problem, with verification accuracy as the performance metric $M$ in Eq. 1 and
Eq. 2 for all datasets except IJB-C in which the true acceptance rate at false
acceptance rate (TAR@FAR) is used. They are defined as
$\text{TAR}={\text{TP}}/{(\text{TP}+\text{FN})}$,
$\text{FAR}={\text{FP}}/{(\text{FP}+\text{TN})}$ and
$\text{ACC}={(\text{TP}+\text{TN})}/{(\text{TP}+\text{TN}+\text{FP}+\text{FN})}$,
where TP, TN, FP, FN indicate true positives, true negatives, false positives,
and false negatives, respectively (Klare et al., 2015; Salehi et al., 2021).
Following the verification protocol defined in (Huang et al., 2007), we
generate a set of pairs of images that do or do not belong to the same class.
A pair is verified on the basis of the distance between feature vectors of its
samples. During the evaluation of task $t$, $\phi_{t}$ is used to extract the
feature representation for the first image of each pair (i.e., the query-set)
and $\phi_{k}$, with $k\in\\{1,\ldots,t\\}$, is used to extract the feature
representation for the second image (i.e., the gallery-set). When $k=t$, the
compatibility test is the self-test, otherwise it is the cross-test between
the two representations learned from the tasks at time $t$ and $k$. For the
LFW and IJB-C evaluation, we use the original pairs provided by the respective
datasets; for the CIFAR10, ImageNet20, and ImageNet100 evaluation, the
verification pairs are randomly generated. As the open-set evaluation requires
no overlap between classes of the training-set and test-set, we use CIFAR100
to perform Class-incremental Learning (i.e., classification is the surrogate
task from which the feature representation is learned) and the CIFAR10 pairs
are used as verification test-set. Similarly, Tiny-ImageNet200 (Le and Yang,
2015) is used as training-set to evaluate the ImageNet20 pairs; LFW and IJB-C
pairs are evaluated with models trained on CASIA-WebFace (Yi et al., 2014).
Finally, for ImageNet100, we train the models with images not included in
ImageNet100, i.e., the subset of the images of the remaining 900 classes that
we named ImageNet900. These datasets are divided into tasks as described in
Sec. 7.2.

###  7.2. Implementation Details

Our $\text{CL}^{2}\text{R}$ training procedure is implemented in PyTorch
(Paszke et al., 2019) and uses the publicly available library Continuum
(Douillard and Lesort, 2021). We used 4 NVIDIA Tesla A100 to train the
representation models, the neural network architectures are based on the
PODNet implementation222
https://github.com/arthurdouillard/incremental_learning.pytorch. The
evaluation is carried out on several ResNet (He et al., 2016) architectures.
Specifically, a 32, 18, and 50 layers ResNet is used for: CIFAR10; ImageNet20
and ImageNet100; and for LFW and IJB-C, respectively. As typically used in CiL
(Hou et al., 2019; Wu et al., 2019), the episodic memory $\mathcal{M}$
contains 20 samples for each class. The value of $\lambda$ in Eq. 9 is set
$\lambda=\lambda_{\rm base}\sqrt{{k_{n}}/{k_{0}}}$ (Hou et al., 2019), in
which $\lambda_{\rm base}$ is a scalar, $k_{n}$ is the number of classes of
the current task and $k_{0}$ is the number of old classes in the episodic
memory. The training details for each dataset are listed below.

CIFAR100 and CIFAR10. We train the model for 70 epochs for each task with
batch size 128, optimization is performed with SGD with an initial learning
rate of 0.1 and weight decay of $2\cdot 10^{-4}$. The learning rate is divided
by 10 at epochs 50 and 64. The input images are RGB, $32\times 32$.
$\lambda_{\rm base}$ is set to 5.

Tiny-ImageNet200 and ImageNet20. We train the model for 90 epochs at each task
with batch size 256, optimization is performed with SGD with an initial
learning rate of 0.1 and a weight decay of $2\cdot 10^{-4}$. The learning rate
is divided by 10 at epochs 30 and 60. To properly evaluate the models in this
learning setting, input images and the ImageNet test images are resized to
match the Tiny-ImageNet200 input size (RGB $64\times 64$). $\lambda_{\rm
base}$ is set to 5.

ImageNet900 and ImageNet100. We train the model for 90 epochs in each task
with batch size 256, optimization is performed with SGD with an initial
learning rate of 0.1 and weight decay of $2\cdot 10^{-4}$. The learning rate
is divided by 10 at epochs 30 and 60. The input images are RGB, $224\times
224$. $\lambda_{\rm base}$ is set to 10.

CASIA-WebFace and LFW/IJB-C. For each task, we train the model for 120 epochs
with batch size 1024. Optimization is carried out with SGD with an initial
learning rate of 0.1 and a weight decay of $2\cdot 10^{-4}$. The learning rate
is divided by 10 at epoch 30, 60 and 90. The input size are RGB images
$112\times 112$. $\lambda_{\rm base}$ is set to 10.

In Tab. 1, we summarize the datasets and the training details of our
experiments.

Table 1.  Datasets used in $\text{CL}^{2}\text{R}$ training procedures. Training-set and test-set of the same configuration have non overlapping classes to properly evaluate different approaches in a open-set setup.  |  |  training-set |  test-set
---|---|---|---
network |  input size |  dataset |  # classes |  dataset |  # pairs
ResNet-32 | $32\times 32$ | CIFAR100 | 100 | CIFAR10 | 6k
ResNet-18 | $64\times 64$ | Tiny-ImageNet200 | 200 | ImageNet20 | 6k
ResNet-18 | $224\times 224$ | ImageNet900 | 900 | ImageNet100 | 6k
ResNet-50 | $112\times 112$ | CASIA-WebFace | 10575 | LFW | 6k
ResNet-50 | $112\times 112$ | CASIA-WebFace | 10575 | IJB-C | 15M

###  7.3. Baselines and Compared Methods

We compare our training procedure with both the CiL methods and the recently
proposed methods for compatible learning. Our baselines include LwF (Li and
Hoiem, 2017), LUCIR (Hou et al., 2019), BiC (Wu et al., 2019), PODNet
(Douillard et al., 2020), FOSTER (Wang et al., 2022), FAN (Iscen et al.,
2020), and BCT (Shen et al., 2020). In particular, FAN and BCT are the only
approaches with an explicit mechanism to address feature compatibility. We
adapted FAN so that the learned adaptation functions are used to transform the
features into compatible features. Since in BCT the model is trained from
scratch at each task using all available data, for a fair comparison, we also
re-implemented it with an episodic memory and refer to it as lifelong-BCT
($\ell$-BCT). At each task, the model is initialized with the parameters of
the model of the previous task and the data of the previous tasks can be
accessed only through the episodic memory. For LwF, BiC, and PODNet we use
their publicly available implementationsfootnote 2, while for LUCIR and FOSTER
we adopted their official implementations333
https://github.com/hshustc/CVPR19_Incremental_Learning and
https://github.com/G-U-N/ECCV22-FOSTER. Finally, we also include a traditional
experience replay-based baseline, denoted as ER, where the model is
continuously fine-tuned as new tasks become available. Finally, in order to
evaluate our training procedure without considering the catastrophic
forgetting phenomenon, we define as upper bound (UB) our training procedure
re-trained from scratch at each task using an episodic memory with infinite
size.

###  7.4. Evaluation on CIFAR10

Table 2.  CIFAR10 evaluation. Two-task $\text{CL}^{2}\text{R}$ setting with
models trained on CIFAR100. Initial Task (i.e., the previous task) shows the
verification accuracy on the first 50 classes, the other rows represent the
performance obtained after two tasks.  Table 3.  Evaluation of CIFAR10. Three,
five, and ten-task $\text{CL}^{2}\text{R}$ setting with models trained on
CIFAR100. We report $AC$ (Eq. 4), $BC$ (Eq. 5), and $FC$ (Eq. 6) for the
methods we evaluated.

In this section, we report the experiments in two, three, five, and ten tasks
$\text{CL}^{2}\text{R}$ settings with models trained on CIFAR100 (i.e., using
50, 33, 20, 10 classes per task) where compatibility is evaluated on the
CIFAR10 generated pairs.

In Tab. 2, we summarize the performance of our $\text{CL}^{2}\text{R}$
training procedure with respect to the other baselines in the two-task
scenario. We evaluate the compatibility of the updated model according to the
ECC (Eq. 1), $BC$ (Eq. 5), and $FC$ (Eq. 6). The first row of Tab. 2 reports
the verification accuracy of the model trained on the first 50 classes of
CIFAR100. Experiments show that, among the methods compared, LUCIR and PODNet
may have an inherent, although limited, level of compatible representations.
This substantially confirms the importance of having some form of mechanism to
preserve the local geometry of the learned features. Our training procedure
achieves the highest cross test, $BC$, and $FC$, thus resulting to be the most
suited training procedure to avoid re-indexing.

In the last rows of the table, we report the performance of the BCT and our
upper bound (UB) that are not affected by catastrophic forgetting. The effect
of catastrophic forgetting and its implications on the reduction of
performance in compatibility can be observed in the self-test, as these values
are significantly higher than the values reported by the methods learned using
CiL.

In Tab. 3, results for the scenario of three, five, and ten-task
$\text{CL}^{2}\text{R}$ are presented. For each experiment, we report $AC$
(Eq. 4), $BC$ (Eq. 5), and $FC$ (Eq. 6). As can be noticed, our method always
achieves the highest $AC$, thus obtaining the largest number of compatible
representations between models, and always achieves the highest $BC$ between
methods that are subject to catastrophic forgetting. FAN achieves almost the
same performance as our procedure in the three-task scenario, while, when the
number of tasks increases, it has a significant decrease in performance,
especially in the ten-task setting. This may be due to increasing number of
adaptation functions between different feature spaces that FAN uses to adapt
old features with respect to the new ones. As can be noticed from the two
tables FOSTER does not learn compatible features. This may be due to the fact
that feature space compression forces the representation to change abruptly
reducing the overall compatibility with previous models. BCT reports higher
values since its representation is learned from scratch for each new task.
Compared to the upper bound (UB), our training procedure achieves lower $AC$
and $BC$, this is due to the influence of catastrophic forgetting. From the
table it can also be noticed that BiC, LUCIR, and PODNet do not satisfy
compatibility when catastrophic forgetting is more severe, as, for example, in
the case of ten-tasks. Overall, these results suggest that the interaction
between local and global stationarity promoted by our training procedure shows
a significant improvement in performance that feature distillation alone
cannot provide.

Table 4.  ImageNet20 evaluation. Two-task $\text{CL}^{2}\text{R}$ setting with
models trained on Tiny-ImageNet200. The Initial Task (i.e., the previous task)
shows verification accuracy on the first 100 classes, the other rows represent
the performance obtained after two tasks.

###  7.5. Evaluation on ImageNet

In this section, we conducted the experiments with models trained on Tiny-
ImageNet200 in $\text{CL}^{2}\text{R}$ setting with two (Tab. 4), three, five
and ten (Tab. 5) tasks.

Tab. 4 follows the same structure as Tab. 2 showing the ECC (Eq. 1), $BC$ (Eq.
5), and $FC$ (Eq. 6) values. For all the compared methods, the initial model
(i.e, the previous model) is trained on the first 100 classes of Tiny-
ImageNet200. As can be seen in the table, our method achieves the best
performance. Although with low values, other methods such as FAN and LUCIR
have a certain level of compatibility, which confirms again that distillation,
with which they are equipped, is a useful tool to support learning compatible
features. As also observed in the CIFAR results, methods not subject to
catastrophic forgetting, (i.e., BCT and ours upper bound), achieves higher
$BC$ and lower $FC$.

Table 5.  ImageNet20 evaluation. Three, five, and ten-task
$\text{CL}^{2}\text{R}$ setting with models trained on Tiny-ImageNet200. We
summarize $AC$ (Eq. 4), $BC$ (Eq. 5), and $FC$ (Eq. 6) for the methods we
evaluated.

Tab. 5 shows the three, five, and ten-task $\text{CL}^{2}\text{R}$ settings
for Tiny-ImageNet200. In these learning scenarios, each task is made up of 66,
40 and 20 classes, respectively. In this table, we discuss the results by
analyzing the values of $AC$ (Eq. 4), $BC$ (Eq. 5), and $FC$ (Eq. 6). Our
approach always achieves the highest value of $AC$. In particular, ER, LwF,
BiC, FAN, and $\ell$-BCT do not achieve lifelong-compatible representation in
the three-task setting as a result of $AC$ = 0. In the ten-task
$\text{CL}^{2}\text{R}$ setting, it is more evident that as the number of
tasks increases, methods without any specific mechanism to preserve the
representation typically cannot learn compatible representations. LUCIR, BiC,
and $\ell$-BCT obtain significantly lower values than our method.
Specifically, the $AC$ performance is more than twice that of BCT, which means
that our $\text{CL}^{2}\text{R}$ procedure obtains twice the number of
compatible representations than that of BCT. This may be caused by the fact
that the constraints imposed by these techniques on the learned representation
seem to have very little effect on its stationarity and consequently in its
compatibility. The results on the ten task are also important as they suggest
that catastrophic forgetting is not an intrinsic impediment to learning
compatible representations. The performance difference of 0.11 in $AC$ with
respect to the upper bound can be considered clear evidence of this effect.
Finally, the table shows how our training procedure provides the highest $FC$
and is the only case where $FC$ is always positive. As a result, our training
procedure achieves, on average, cross-tests higher than self-tests indicating
that the system performs better even without re-indexing the gallery.

Table 6.  ImageNet100 evaluation. Two and three-task $\text{CL}^{2}\text{R}$
setting with models trained on ImageNet900. We compare our training procedure
and $\ell$-BCT reporting $AC$, $BC$, and $FC$.

Tab. 6 reports ImageNet100 results when models are trained on ImageNet900 with
two and three tasks. We compare our approach with the $\ell$-BCT method as
having reasonable performance and with an explicit mechanism to learn
compatible features under catastrophic forgetting. As can be noticed from the
table, our $\text{CL}^{2}\text{R}$ training clearly outperforms $\ell$ -BCT.
Our method achieves good scores for the AC in both scenarios. As remarked in
the Sec.7.5 of the novel revised manuscript, the reduced performance of
$\ell$-BCT appears to be connected to the fact that the training procedure is
only based on pairwise model training (i.e., compatibility is only learned
from the previous model). In contrast, our method is not based only on
pairwise learning and does not use previous classifiers, which may be
incorrectly learned.

###  7.6. Face Verification

In this section, we report the experimental results on the LFW and IJB-C
benchmarks in two, three, five and ten $\text{CL}^{2}\text{R}$ settings. We
incrementally train the representation models with CASIA-WebFace resulting in
tasks composed of 5287, 3525, 2115, and 1057 classes, respectively.

The results are summarized in Tab. 7 and Tab. 8 for LFW and IJB-C,
respectively. In particular, for IJB-C, we report accuracy in terms of $AC$,
$BC$ and $FC$ at different false acceptance rates (FAR): $10^{-1}$, $10^{-2}$,
and $10^{-4}$. In this evaluation, we do not report LUCIR when training on the
CASIA-WebFace due to the excessive memory requirements of the original
implementationfootnote 3. Although in the two-task scenario comparable results
are observed to those of $\ell$-BCT, in the settings of three and five tasks,
our training procedure achieves complete compatibility resulting in $AC$ = 1
and $BC$ always positive. In ten-task compatibility, the difference in
performance increases more significantly, confirming a clear overall positive
performance. Generally, the reported performances are higher on face datasets
than on CIFAR10, ImageNet20, and ImageNet100. Possible reasons may be found in
the fact that in face recognition, the domain shift between classes is lower
than that for CIFAR or ImageNet. Finally, this experiment shows that the
proposed method is effective not only with a larger number of model updates,
but also with larger datasets.

Table 7.  Face verification on LFW dataset. Two, three, five, and ten-task
$\text{CL}^{2}\text{R}$ setting with models trained on CASIA-WebFace. We
compare our training procedure and $\ell$-BCT reporting $BC$ (Eq. 5), $FC$
(Eq. 6), and $AC$ (Eq. 4), which corresponds to the ECC (Eq. 1) when evaluated
in two tasks.  Table 8.  Face verification on IJB-C dataset. Two, three, five,
and ten-task $\text{CL}^{2}\text{R}$ setting with models trained on CASIA-
WebFace. We report $\text{($AC$, $BC$, $FC$)@FAR}=\text{10}^{\text{-1}},\text{
10}^{\text{-2}}\text{, and }{\text{10}^{\text{-4}}}$ and compare our training
procedure with $\ell$-BCT.

(a) Five-task $\text{CL}^{2}\text{R}$ setting CIFAR10.

(b) Ten-task $\text{CL}^{2}\text{R}$ setting CIFAR10.

(c) Five-task $\text{CL}^{2}\text{R}$ setting ImageNet20.

(d) Ten-task $\text{CL}^{2}\text{R}$ setting ImageNet20.

Figure 2.  Backward Compatibility evolution across tasks $t$ (i.e., $BC(t)$).
Comparison between our $\text{CL}^{2}\text{R}$ training and other methods in
five and ten learning setup. (a) and (b) CIFAR10 results, (c) and (d)
ImageNet20.

###  7.7. Compatibility & Catastrophic Forgetting

In this section, we study how compatibility is related to the problem of
catastrophic forgetting. In Fig. 2, we show the evolution of $BC$ in a five
and ten-task $\text{CL}^{2}\text{R}$ scenario. In particular, Figs. 2(a) to
2(b) and in Figs. 2(c) to 2(d) show the evaluations on the CIFAR10 and
ImageNet20 datasets, respectively. We compared our approach with ER, LwF, BiC,
LUCIR, FAN and $\ell$-BCT. As can be observed, our training procedure achieves
the highest performance. As the $BC$ metric is, on average, the closest to
zero than the other evaluated methods, the representation learned by our
training procedure can be considered to be the most compatible and, from the
perspective of visual search, equivalent to the representation models learned
from previous tasks. More practically, this allows for the reduction of the
computational cost of re-indexing.

In contrast, FAN achieves a negative value of $BC$ in all four settings,
confirming that the composition of an increasing number of feature adaption
functions between sequentially learned representations causes a decrease in
compatibility. Despite the absence of considerable performance loss, as in the
case of FAN, negative $BC$ values indicate a constant deterioration in
performance as the number of tasks increases. In general, except for our
method, the figure shows that all the other methods follow a common trend with
lower performance.

##  8\. Ablation Studies

We analyze by ablation the main components of our training procedure. The
ablation is performed on the CIFAR100 dataset as described in Sec 7.4 and
considers the ten-task $\text{CL}^{2}\text{R}$ setting which can be regarded
as a worst-case scenario for this dataset. We analyze the impact of: (a) the
specific classifier: Trainable vs. Fixed $d$-Simplex with or without the
feature distillation component, (b) how the FD loss is evaluated, and (c) the
sensitivity of the number of samples reserved per class in the episodic
memory.

Table 9. Ablation of the different main components of our
$\text{CL}^{2}\text{R}$ training procedure. The evaluation is performed on
CIFAR10 and training is based on CIFAR100 with ten tasks, where: Trainable
indicates the traditional Experience Replay (ER) baseline, Fixed indicates ER
with stationary features learned from Eq. 7 according to the Fixed $d$-Simplex
classifier, $\mathcal{L}_{\scriptscriptstyle\textrm{FD}}$ is the traditional
Feature Distillation, and
$\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}$
is the Feature Distillation evaluated on the only samples stored in episodic
memory as defined in Eq. 8.

#### Impact of the $d$-Simplex fixed classifier and Feature Distillation

As it can be noticed from Tab. 9, the Trainable classifier is not able to
learn compatible representations. When combined with feature distillation, the
performance improves only marginally and not sufficiently to be compared with
the CiL approaches shown in Tab. 3. Feature Distillation evaluated on the only
samples stored in episodic memory as defined in
$\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}$
(Eq. 8), improves the values of the reported metrics showing a better
supervision signal for the updated model. The $d$-Simplex alone improves on
the previous components obtaining a values of $AC$ =$0.27$ and $FC$ =$0.003$,
which are higher than the Trainable classifier with
$\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}$.
This remarks on the importance of preserving the global geometry of the
learned features according to the $d$-Simplex fixed classifier.

#### Impact of Memory Samples on Feature Distillation

Tab. 9 shows that when the distillation loss is evaluated on the only samples
stored in episodic memory
$\mathcal{L}_{\scriptscriptstyle\textrm{FD}}^{\scriptscriptstyle\mathcal{M}}$
(Eq. 8), our approach achieves the better overall results. We argue this
positive effect is mostly due to the interaction between the global feature
stationarity learned using the fixed classifier and the local one promoted
through feature distillation from the only observed samples in the episodic
memory. The interaction is most likely related to the fact that the fixed
$d$-Simplex classifier in general does not allow novel classes to interfere in
the feature space of the already learned one. This in turn provides favorable
working conditions (i.e., a kind of coarse pre-alignment) for achieving
feature alignment with respect to the previous model by the distillation loss.
As expected, the impact intensifies when evaluated only on already known
classes, as alignment is less prone to unexpected noisy features which may
reduce the degree of the alignment. This confirms the effectiveness of
restricting FD only on memory samples in contrast to the traditional feature
distillation commonly used in CiL.

Figure 3. The effect of the number of reserved samples per class in the
episodic memory.

#### Impact of the Episodic Memory Size.

Fig. 3 shows the effect of different numbers of reserved samples per class for
both our learning procedure and other baselines. As expected, the more samples
per class are reserved in the episodic memory, the better the performance. Our
approach, with 20 samples per class, achieves results similar to those
obtained by the other methods with more examples per class. Although ER, LUCIR
and FAN have a better relative improvement with 50 samples per class, overall
our approach results in the highest performance in learning compatible
features.

We also evaluated the methods in the challenging memory-free training setting,
(i.e., without the episodic memory). Our training procedure achieves the
highest results also in this condition remarking on the fact that CiL methods
typically do not have an inherent mechanism to learn compatible features.

##  9\. Conclusions

In this paper, we have introduced the problem of Compatible Lifelong Learning
Representations ($\text{CL}^{2}\text{R}$) that considers the compatibility
learning problem within the lifelong learning paradigm. We introduced a novel
set of metrics to properly evaluate this problem and proposed a novel
$\text{CL}^{2}\text{R}$ training procedure that imposes global and local
stationarity on the learned features to achieve compatibility between
representations under catastrophic forgetting. Global and local stationarity
is imposed according to the $d$-Simplex fixed classifier and the feature
distillation loss, respectively. Empirical evaluation of the learned lifelong-
compatible representation shows the effectiveness of our method with respect
to baselines and state-of-the-art methods.

####  Acknowledgments.

This work was supported by the European Commission under European Horizon 2020
Programme, grant number 951911 - AI4Media. The authors also acknowledge the
CINECA award under the ISCRA initiative (ISCRA-C - “ILCoRe”, ID: HP10CRMI87),
for the availability of high performance computing resources.

## References

  * (1)
  * Barletti et al. (2022) Tommaso Barletti, Niccoló Biondi, Federico Pernici, Matteo Bruni, and Alberto Del Bimbo. 2022\.  Contrastive Supervised Distillation for Continual Representation Learning. In _International Conference on Image Analysis and Processing_. Springer, 597–609.
  * Belouadah et al. (2021) Eden Belouadah, Adrian Popescu, and Ioannis Kanellos. 2021\.  A comprehensive study of class incremental learning algorithms for visual tasks.  _Neural Networks_ 135 (2021), 38–54.
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ 35, 8 (2013), 1798–1828.
  * Biondi et al. (2021) Niccolo Biondi, Federico Pernici, Matteo Bruni, and Alberto Del Bimbo. 2021.  CoReS: Compatible Representations via Stationarity.  _arXiv preprint arXiv:2111.07632_ (2021).
  * Budnik and Avrithis (2021) Mateusz Budnik and Yannis Avrithis. 2021.  Asymmetric Metric Learning for Knowledge Transfer. In _CVPR_. Computer Vision Foundation / IEEE, 8228–8238.
  * Chen et al. (2019) Ken Chen, Yichao Wu, Haoyu Qin, Ding Liang, Xuebo Liu, and Junjie Yan. 2019\.  R3 Adversarial Network for Cross Model Face Recognition. In _CVPR_. Computer Vision Foundation / IEEE, 9868–9876.
  * Chen et al. (2021) Wei Chen, Yu Liu, Nan Pu, Weiping Wang, Li Liu, and Michael S Lew. 2021\.  Feature Estimations based Correlation Distillation for Incremental Image Retrieval.  _IEEE Transactions on Multimedia_ (2021), 1–1.  https://doi.org/10.1109/TMM.2021.3073279
  * Chen et al. (2020) Wei Chen, Yu Liu, Weiping Wang, Tinne Tuytelaars, Erwin M. Bakker, and Michael S. Lew. 2020\.  On the Exploration of Incremental Learning for Fine-grained Image Retrieval. In _BMVC_. BMVA Press.
  * Chen and Liu (2018) Zhiyuan Chen and Bing Liu. 2018.  Lifelong machine learning.  _Synthesis Lectures on Artificial Intelligence and Machine Learning_ 12, 3 (2018), 1–207.
  * Chopra et al. (2005) Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005\.  Learning a similarity metric discriminatively, with application to face verification. In _2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)_ , Vol. 1. IEEE, 539–546.
  * Cossu et al. (2021) Andrea Cossu, Marta Ziosi, and Vincenzo Lomonaco. 2021\.  Sustainable Artificial Intelligence through Continual Learning.  _arXiv preprint arXiv:2111.09437_ (2021).
  * Davari and Belilovsky (2021) MohammadReza Davari and Eugene Belilovsky. 2021.  Probing Representation Forgetting in Continual Learning. In _NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications_.
  * Delange et al. (2021) Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. 2021\.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2021).
  * Deng et al. (2019) Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019\.  Arcface: Additive angular margin loss for deep face recognition. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4690–4699.
  * Díaz-Rodríguez et al. (2018) Natalia Díaz-Rodríguez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. 2018.  Don’t forget, there is more than forgetting: new metrics for Continual Learning.  _arXiv preprint arXiv:1810.13166_ (2018).
  * Douillard et al. (2020) Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. 2020.  Podnet: Pooled outputs distillation for small-tasks incremental learning. In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16_. Springer, 86–102.
  * Douillard and Lesort (2021) Arthur Douillard and Timothée Lesort. 2021.  Continuum: Simple Management of Complex Continual Learning Scenarios.
  * Ericsson and Kintsch (1995) K Anders Ericsson and Walter Kintsch. 1995.  Long-term working memory.  _Psychological review_ 102, 2 (1995), 211.
  * Gordo et al. (2016) Albert Gordo, Jon Almazán, Jerome Revaud, and Diane Larlus. 2016.  Deep Image Retrieval: Learning Global Representations for Image Search. In _Computer Vision – ECCV 2016_ , Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 241–257.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016\.  Deep residual learning for image recognition. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 770–778.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015\.  Distilling the Knowledge in a Neural Network. In _NIPS Deep Learning and Representation Learning Workshop_.  http://arxiv.org/abs/1503.02531
  * Hou et al. (2019) Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. 2019.  Learning a unified classifier incrementally via rebalancing. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 831–839.
  * Hu et al. (2019) Jie Hu, Rongrong Ji, Hong Liu, Shengchuan Zhang, Cheng Deng, and Qi Tian. 2019\.  Towards visual feature translation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 3004–3013.
  * Huang et al. (2007) Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007.  _Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments_.  Technical Report 07-49. University of Massachusetts, Amherst.
  * Iscen et al. (2020) Ahmet Iscen, Jeffrey Zhang, Svetlana Lazebnik, and Cordelia Schmid. 2020.  Memory-efficient incremental learning through feature adaptation. In _European Conference on Computer Vision_. Springer, 699–715.
  * Jung et al. (2018) Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. 2018\.  Less-forgetful learning for domain expansion in deep neural networks. In _Thirty-Second AAAI Conference on Artificial Intelligence_.
  * Klare et al. (2015) Brendan F Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K Jain. 2015.  Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1931–1939.
  * Krizhevsky et al. (2009) Alex Krizhevsky, Geoffrey Hinton, et al. 2009\.  Learning multiple layers of features from tiny images.  _Citeseer_ (2009).
  * Le and Yang (2015) Ya Le and Xuan Yang. 2015\.  Tiny imagenet visual recognition challenge.  _CS 231N_ (2015), 3\.
  * Learned-Miller (2014) Gary B. Huang Erik Learned-Miller. 2014\.  _Labeled Faces in the Wild: Updates and New Reporting Procedures_.  Technical Report UM-CS-2014-003. University of Massachusetts, Amherst.
  * Li et al. (2015) Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John Hopcroft. 2015.  Convergent Learning: Do different neural networks learn the same representations?. In _Feature Extraction: Modern Questions and Challenges_. PMLR, 196–212.
  * Li and Hoiem (2017) Zhizhong Li and Derek Hoiem. 2017.  Learning without forgetting.  _IEEE transactions on pattern analysis and machine intelligence_ 40, 12 (2017), 2935–2947.
  * Libby and Buschman (2021) Alexandra Libby and Timothy J Buschman. 2021.  Rotational dynamics reduce interference between sensory and memory representations.  _Nature Neuroscience_ (2021), 1–12.
  * Lopez-Paz and Ranzato (2017) David Lopez-Paz and Marc’Aurelio Ranzato. 2017.  Gradient episodic memory for continual learning.  _Advances in neural information processing systems_ 30 (2017).
  * Masana et al. (2020) Marc Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer. 2020.  Class-incremental learning: survey and performance evaluation.  _arXiv preprint arXiv:2010.15277_ (2020).
  * Maze et al. (2018) Brianna Maze, Jocelyn Adams, James A Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K Jain, W Tyler Niggel, Janet Anderson, Jordan Cheney, et al. 2018\.  Iarpa janus benchmark-c: Face dataset and protocol. In _2018 international conference on biometrics (ICB)_. IEEE, 158–165.
  * McCloskey and Cohen (1989) Michael McCloskey and Neal J Cohen. 1989.  Catastrophic interference in connectionist networks: The sequential learning problem.  In _Psychology of learning and motivation_. Vol. 24. Elsevier, 109–165.
  * Meng et al. (2021) Qiang Meng, Chixiang Zhang, Xiaoqiang Xu, and Feng Zhou. 2021\.  Learning Compatible Embeddings. In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_. 9939–9948.
  * Meyers (2018) Ethan M Meyers. 2018\.  Dynamic population coding and its relationship to working memory.  _Journal of Neurophysiology_ 120, 5 (2018), 2260–2268.
  * Murray et al. (2017) John D Murray, Alberto Bernacchia, Nicholas A Roy, Christos Constantinidis, Ranulfo Romo, and Xiao-Jing Wang. 2017.  Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex.  _Proceedings of the National Academy of Sciences_ 114, 2 (2017), 394–399.
  * Parisi et al. (2019) German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019.  Continual lifelong learning with neural networks: A review.  _Neural Networks_ 113 (2019), 54–71.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019\.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ 32 (2019), 8026–8037.
  * Pernici et al. (2018) Federico Pernici, Federico Bartoli, Matteo Bruni, and Alberto Del Bimbo. 2018.  Memory Based Online Learning of Deep Representations From Video Streams. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Pernici et al. (2019a) Federico Pernici, Matteo Bruni, Claudio Baecchi, and Alberto Del Bimbo. 2019a.  Fix your features: Stationary and maximally discriminative embeddings using regular polytope (fixed classifier) networks.  _arXiv preprint arXiv:1902.10441_ (2019).
  * Pernici et al. (2019b) Federico Pernici, Matteo Bruni, Claudio Baecchi, and Alberto Del Bimbo. 2019b.  Maximally Compact and Separated Features with Regular Polytope Networks. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_.
  * Pernici et al. (2021a) Federico Pernici, Matteo Bruni, Claudio Baecchi, and Alberto Del Bimbo. 2021a.  Regular Polytope Networks.  _IEEE Transactions on Neural Networks and Learning Systems_ (2021).
  * Pernici et al. (2020b) Federico Pernici, Matteo Bruni, Claudio Baecchi, Francesco Turchini, and Alberto Del Bimbo. 2020b.  Class-incremental Learning with Pre-allocated Fixed Classifiers. In _25th International Conference on Pattern Recognition, ICPR 2020, Milan, Italy, January 10-15, 2021_. IEEE Computer Society.
  * Pernici et al. (2021b) Federico Pernici, Matteo Bruni, Claudio Baecchi, Francesco Turchini, and Alberto Del Bimbo. 2021b.  Class-incremental learning with pre-allocated fixed classifiers. In _2020 25th International Conference on Pattern Recognition (ICPR)_. IEEE, 6259–6266.
  * Pernici et al. (2020a) Federico Pernici, Matteo Bruni, and Alberto Del Bimbo. 2020a.  Self-supervised on-line cumulative learning from video streams.  _Computer Vision and Image Understanding_ 197 (2020), 102983.
  * Price and Cohen (2019) W Nicholson Price and I Glenn Cohen. 2019.  Privacy in the age of medical big data.  _Nature medicine_ 25, 1 (2019), 37–43.
  * Pu et al. (2021) Nan Pu, Wei Chen, Yu Liu, Erwin M Bakker, and Michael S Lew. 2021.  Lifelong Person Re-Identification via Adaptive Knowledge Accumulation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 7901–7910.
  * Ramanujan et al. (2022) Vivek Ramanujan, Pavan Kumar Anasosalu Vasu, Ali Farhadi, Oncel Tuzel, and Hadi Pouransari. 2022\.  Forward Compatible Training for Representation Learning.  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2022).
  * Ratcliff (1990) Roger Ratcliff. 1990\.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.  _Psychological review_ 97, 2 (1990), 285.
  * Rebuffi et al. (2017) Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017.  icarl: Incremental classifier and representation learning. In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_. 2001–2010.
  * Robins (1993) Anthony Robins. 1993\.  Catastrophic forgetting in neural networks: the role of rehearsal mechanisms. In _Proceedings 1993 The First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems_. IEEE, 65–68.
  * Robins (1995) Anthony Robins. 1995\.  Catastrophic forgetting, rehearsal and pseudorehearsal.  _Connection Science_ 7, 2 (1995), 123–146.
  * Romero et al. (2015) Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015.  FitNets: Hints for Thin Deep Nets. In _3rd International Conference on Learning Representations, ICLR 2015_.
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015\.  Imagenet large scale visual recognition challenge.  _International journal of computer vision_ 115, 3 (2015), 211–252.
  * Salehi et al. (2021) Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. 2021.  A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges.  _arXiv preprint arXiv:2110.14051_ (2021).
  * Schroff et al. (2015) Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015\.  Facenet: A unified embedding for face recognition and clustering. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 815–823.
  * Schwartz et al. (2020) Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020.  Green ai.  _Commun. ACM_ 63, 12 (2020), 54–63.
  * Sharif Razavian et al. (2014) Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014.  CNN features off-the-shelf: an astounding baseline for recognition. In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_. 806–813.
  * Shen et al. (2020) Yantao Shen, Yuanjun Xiong, Wei Xia, and Stefano Soatto. 2020\.  Towards backward-compatible representation learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 6368–6377.
  * Sun et al. (2014) Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. 2014\.  Deep Learning Face Representation by Joint Identification-Verification. In _Advances in Neural Information Processing Systems_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc.
  * Taigman et al. (2014) Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. 2014\.  Deepface: Closing the gap to human-level performance in face verification. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1701–1708.
  * Tolias et al. (2016) Giorgos Tolias, Ronan Sicre, and Hervé Jégou. 2016\.  Particular Object Retrieval With Integral Max-Pooling of CNN Activations. In _ICLR 2016 - International Conference on Learning Representations_ _(International Conference on Learning Representations)_.
  * Van Noorden (2020) Richard Van Noorden. 2020\.  The ethical questions that haunt facial-recognition research.  _Nature_ 587, 7834 (2020), 354–358.
  * Vijayan and Sridhar (2021) Mochitha Vijayan and SS Sridhar. 2021.  Continual Learning for Classification Problems: A Survey. In _International Conference on Computational Intelligence in Data Science_. Springer, 156–166.
  * Wang et al. (2020) Chien-Yi Wang, Ya-Liang Chang, Shang-Ta Yang, Dong Chen, and Shang-Hong Lai. 2020\.  Unified Representation Learning for Cross Model Compatibility. In _31st British Machine Vision Conference 2020, BMVC 2020_. BMVA Press.
  * Wang et al. (2022) Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. 2022\.  FOSTER: Feature Boosting and Compression for Class-Incremental Learning.  _arXiv preprint arXiv:2204.04662_ (2022).
  * Wang et al. (2018) Liwei Wang, Lunjia Hu, Jiayuan Gu, Zhiqiang Hu, Yue Wu, Kun He, and John Hopcroft. 2018.  Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation. In _Advances in Neural Information Processing Systems_ , Vol. 31.
  * Wu et al. (2019) Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. 2019.  Large scale incremental learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 374–382.
  * Yan et al. (2021) Shipeng Yan, Jiangwei Xie, and Xuming He. 2021.  Der: Dynamically expandable representation for class incremental learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 3014–3023.
  * Yandex and Lempitsky (2015) Artem Babenko Yandex and Victor Lempitsky. 2015.  Aggregating Local Deep Features for Image Retrieval. In _2015 IEEE International Conference on Computer Vision (ICCV)_. 1269–1277.  https://doi.org/10.1109/ICCV.2015.150
  * Ye et al. (2021) Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. 2021\.  Deep learning for person re-identification: A survey and outlook.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2021).
  * Yi et al. (2014) Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014\.  Learning face representation from scratch.  _arXiv preprint arXiv:1411.7923_ (2014).
  * Yosinski et al. (2014) Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014\.  How transferable are features in deep neural networks?. In _Advances in Neural Information Processing Systems_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc.
  * Zhao et al. (2020) Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. 2020.  Maintaining discrimination and fairness in class incremental learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13208–13217.
  * Zheng et al. (2015) Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015\.  Scalable person re-identification: A benchmark. In _Proceedings of the IEEE international conference on computer vision_. 1116–1124.
  * Zheng et al. (2016) Liang Zheng, Yi Yang, and Alexander G Hauptmann. 2016\.  Person re-identification: Past, present and future.  _arXiv preprint arXiv:1610.02984_ (2016).
  * Zhou et al. (2019) Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. 2019.  Omni-scale feature learning for person re-identification. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 3702–3712.