# Mitigating Interference in the Knowledge Continuum through Attention-Guided
Incremental Learning

Prashant Bhat ,1,2, Bharath Renjith*,2, Elahe Arani†,1,3, Bahram Zonooz†,1
1Eindhoven University of Technology (TU/e) 2TomTom 3Wayve
{p.s.bhat, e.arani<EMAIL_ADDRESS><EMAIL_ADDRESS>Equal
contribution. † Shared last author.

###### Abstract

Continual learning (CL) remains a significant challenge for deep neural
networks, as it is prone to forgetting previously acquired knowledge. Several
approaches have been proposed in the literature, such as experience rehearsal,
regularization, and parameter isolation, to address this problem. Although
almost zero forgetting can be achieved in task-incremental learning, class-
incremental learning remains highly challenging due to the problem of inter-
task class separation. Limited access to previous task data makes it difficult
to discriminate between classes of current and previous tasks. To address this
issue, we propose ‘Attention-Guided Incremental Learning’ (AGILE), a novel
rehearsal-based CL approach that incorporates compact task attention to
effectively reduce interference between tasks. AGILE utilizes lightweight,
learnable task projection vectors to transform the latent representations of a
shared task attention module toward task distribution. Through extensive
empirical evaluation, we show that AGILE significantly improves generalization
performance by mitigating task interference and outperforming rehearsal-based
approaches in several CL scenarios. Furthermore, AGILE can scale well to a
large number of tasks with minimal overhead while remaining well-calibrated
with reduced task-recency bias.111 Code: https://github.com/NeurAI-Lab/AGILE.

##  1 Introduction

In recent years, deep neural networks (DNNs) have been shown to perform better
than humans on certain specific tasks, such as Atari games (Silver et al.,
2018) and classification (He et al., 2015). Although impressive, these models
are trained on static data and are unable to adapt their behavior to novel
tasks while maintaining performance on previous tasks when the data evolve
over time (Fedus et al., 2020). Continual learning (CL) refers to a training
paradigm in which DNNs are exposed to a sequence of tasks and are expected to
learn potentially incrementally or online (Parisi et al., 2019). CL has
remained one of the most daunting tasks for DNNs, as acquiring new information
significantly deteriorates the performance of previously learned tasks, a
phenomenon termed “catastrophic forgetting” (French, 1999; McCloskey & Cohen,
1989). Catastrophic forgetting arises due to the stability-plasticity dilemma
(Mermillod et al., 2013), the degree to which the system must be stable to
retain consolidated knowledge while also being plastic to assimilate new
information. Catastrophic forgetting often results in a significant decrease
in performance, and in some cases, previously learned information is
completely erased by new information (Parisi et al., 2019).

Several approaches have been proposed in the literature to address the problem
of catastrophic forgetting in CL. Rehearsal-based approaches (Ratcliff, 1990;
Sarfraz et al., 2023; Vijayan et al., 2023) explicitly store a subset of
samples from previous tasks in the memory buffer and replay them alongside
current task samples to combat forgetting. In scenarios where buffer size is
limited due to memory constraints (e.g., edge devices), these approaches are
prone to overfitting on the buffered data (Bhat et al., 2022). On the other
hand, regularization-based approaches (Kirkpatrick et al., 2017; Vijayan et
al., 2023) introduce a regularization term in the optimization objective and
impose a penalty for changes in parameters important for previous tasks.
Although regularization greatly improves stability, these approaches often
cannot discriminate classes from different tasks, thus failing miserably in
scenarios such as Class-Incremental Learning (Class-IL) (Lesort et al., 2019).
Parameter isolation approaches limit interference between tasks by allocating
a different set of parameters for each task, either within a fixed model
capacity (Gurbuz & Dovrolis, 2022; Vijayan et al., 2023) or by expanding the
model size (Rusu et al., 2016; Bhat et al., 2023). However, these approaches
mostly suffer from several shortcomings, including capacity saturation and
scalability issues in longer task sequences. With an increasing number of
tasks, selecting the right expert in the absence of task identity is
nontrivial (Aljundi et al., 2017), and therefore limits its application
largely to Task-Incremental Learning (Task-IL).

Figure 1: Attention-Guided Incremental Learning (AGILE) consists of a shared
task-attention module and a set of task-specific projection vectors, one for
each task. Each sample is passed through the task-attention module once for
each projection vector, and the outputs are fed into task-specific
classifiers. AGILE effectively reduces task interference and facilitates
accurate task-id prediction (TP) and within-task prediction (WP).

The problem of inter-task class separation in Class-IL remains a significant
challenge due to the difficulty in establishing clear boundaries between
classes of current and previous tasks (Lesort et al., 2019). When a limited
number of samples from previous tasks are available in the buffer in
experience rehearsal, the CL model tends to overfit on the buffered samples
and incorrectly approximates the class boundaries. Kim et al. (2022)
decomposes the Class-IL problem into two sub-problems: task-id prediction (TP)
and within-task prediction (WP). TP involves identifying the task of a given
sample, whereas WP refers to making predictions for a sample within the
classes of the task identified by TP. Therefore, the Class-IL problem can be
seen as a combination of the Task-IL problem (WP) and the task discovery (TP).
Regardless of whether the CL algorithm defines it explicitly or implicitly,
good TP and good WP are necessary and sufficient to ensure good Class-IL
performance (Kim et al., 2022; Bhat et al., 2023). As task interference
adversely affects both WP and TP, we hypothesize that focusing on the
information relevant to the current task can facilitate more accurate TP and
WP by filtering out extraneous or interfering information.

To this end, we propose ‘Attention-Guided Incremental Learning’ (AGILE), a
rehearsal-based novel CL approach that encompasses compact task-attention to
effectively mitigate interference between tasks and facilitate a good WP and
TP in Class-IL. To further augment rehearsal-based learning in Class-IL, AGILE
leverages parameter isolation to bring in task specificity with little
computational or memory overhead. Specifically, AGILE entails a shared feature
encoder and task-attention module, and as many task projection vectors as the
number of tasks. Each task projection vector is a lightweight learnable vector
associated with a particular task, specialized in transforming the latent
representations of the shared task-attention module towards the task
distribution. With dynamic expansion of task projection vectors, AGILE scales
well to a large number of tasks while leaving a negligible memory footprint.
Across CL scenarios, AGILE greatly reduces task interference and outperforms
rehearsal-based approaches while being scalable and well-calibrated with less
task-recency bias.

##  2 Related Works

##### Rehearsal-based Approaches:

Earlier work sought to combat catastrophic forgetting in CL by explicitly
storing and replaying previous task samples through Experience-Rehearsal (ER)
(Ratcliff, 1990). Several works build on top of ER: Since soft targets carry
more information and capture complex similarity patterns in the data compared
to hard targets (Hinton et al., 2015), DER++ (Buzzega et al., 2020) enforces
consistency in predictions through regularization of the function space. To
further improve knowledge distillation through consistency regularization,
CLS-ER (Arani et al., 2022) employs multiple semantic memories that better
handle the stability-plasticity trade-off. More recent works focus on reducing
representation drift right after task switching to mitigate forgetting: ER-ACE
(Caccia et al., 2022) through asymmetric update rules shields learned
representations from drastic adaptations while accommodating new information.
Co2L (Cha et al., 2021) employs contrastive representation learning to learn
robust features that are less susceptible to catastrophic forgetting. However,
under low-buffer regimes, these approaches are prone to overfitting. Under
low-buffer regimes, the quality of the buffered samples plays a significant
role in defining the ability of the CL model to approximate past behavior. GCR
(Tiwari et al., 2022) proposed a core set selection mechanism that
approximates the gradients of the data seen so far to select and update the
memory buffer. In contrast, DRI (Wang et al., 2022a) employs a generative
replay to augment the memory buffer under low buffer regimes. Although
reasonably successful in many CL scenarios, rehearsal-based approaches lack
task-specific parameters and run the risk of shared parameters being
overwritten by later tasks.

##### Task Attention:

As the weights in DNNs hold knowledge of previous tasks, intelligent
segregation of weights per task is an attractive alternative to rehearsal to
reduce catastrophic forgetting in CL. Dynamic sparse parameter isolation
approaches (e.g., NISPA (Gurbuz & Dovrolis, 2022), CLNP (Golkar et al., 2019),
PackNet (Mallya & Lazebnik, 2018)) leverage overparameterization of DNNs and
learn sparse architecture for each task within a fixed model capacity.
However, these approaches suffer from capacity saturation and fail miserably
in longer task sequences. By contrast, some parameter-isolation approaches
grow in size, either naively or intelligently, to accommodate new tasks with
the least forgetting. PNN (Rusu et al., 2016) was one of the first works to
propose a growing architecture with lateral connections to previously learned
features to reduce forgetting and enable forward transfer simultaneously.
Since PNN instantiates a new sub-network for each task, it quickly runs into
scalability issues. Approaches such as CPG (Hung et al., 2019a) and PAE (Hung
et al., 2019b) grow drastically slower than PNN, but require task identity at
inference. HAT (Serra et al., 2018) employed a task-based layer-wise hard
attention mechanism in fully connected or convolutional networks to reduce
interference between tasks. However, layer-wise attention is quite cumbersome
as many low-level features can be shared across tasks. Due to the limitations
mentioned above, task-specific learning approaches have been largely limited
to the Task-IL setting.

Although almost zero forgetting can be achieved in Task-IL (Serra et al.,
2018), the Class-IL scenario still remains highly challenging due to the
problem of inter-task class separation. Therefore, we propose AGILE, a
rehearsal-based CL method that encompasses task attention to facilitate a good
WP and TP by reducing interference between tasks.

##  3 Proposed Method

###  3.1 Motivation

Task interference arises when multiple tasks share a common observation space
but have different learning goals. In the presence of task interference, both
WP and TP struggle to find the right class or task, resulting in reduced
performance and higher cross-entropy loss. Continual learning in the brain is
governed by the conscious processing of multiple knowledge bases anchored by a
rich set of neurophysiological processes (Goyal & Bengio, 2020). Global
Workspace Theory (GWT) (Baars, 1994; 2005; Baars et al., 2021) provides a
formal account of cognitive information access and posits that one such
knowledge base is a common representation space of fixed capacity from which
information is selected, maintained, and shared with the rest of the brain
(Juliani et al., 2022). During information access, the attention mechanism
creates a communication bottleneck between the representation space and the
global workspace, and only behaviorally relevant information is admitted into
the global workspace. Such conscious processing could help the brain achieve
systematic generalization (Bengio, 2017) and deal with problems that could
only be solved by multiple specialized modules (VanRullen & Kanai, 2021).

In functional terms, GWT as a model of cognitive access has several benefits
for CL. (i) The common representation space is largely a shared function,
resulting in maximum re-usability across tasks; (ii) The attention mechanism
can be interpreted as a task-specific policy for admitting task-relevant
information, thereby reducing interference between tasks; And (iii) multiple
specialized attention modules enable solving more complex tasks that cannot be
solved by a single specialized function. Combining intuitions from both
biological and theoretical findings (Appendix B), we hypothesize that focusing
on the information relevant to the current task can facilitate good TP and WP,
and consequently systemic generalization by filtering out extraneous or
interfering information. In the following section, we describe in detail how
we mitigate interference between tasks through task attention.

###  3.2 Preliminary

Continual learning typically involves sequential tasks $t\in\\{1,2,..,T\\}$
and classes $j\in\\{1,2,...,J\\}$ per task, with data appearing over time.
Each task is associated with a task-specific data distribution
$(\mathbf{X}_{t,j},\mathbf{Y}_{t,j})\in\mathcal{D}_{t}$. We consider two
popular CL scenarios, Class-IL and Task-IL, defined in Definitions 1 and 2,
respectively. Our CL model
$\Phi_{\theta}=\\{f_{\theta},\tau_{\theta},\delta_{\theta},g_{\theta}\\}$
consists of a backbone network (e.g. ResNet-18) $f_{\theta}$, a shared
attention module $\tau_{\theta}$, a single expanding head
$g_{\theta}=\\{g^{i}_{\theta}\mid i\leq t\\}$ representing all classes for all
tasks, and a set of task projection vectors up to the current task
$\delta_{\theta}=\\{\delta_{i}\mid i\leq t\\}$.

Training DNNs sequentially has remained a daunting task since acquiring new
information significantly deteriorates the performance of previously learned
tasks. Therefore, to better preserve the information from previous tasks, we
seek to maintain a memory buffer $\mathcal{D}_{m}$ that represents all tasks
seen previously. We employ reservoir sampling (Algorithm 3) (Vitter, 1985) to
update $\mathcal{D}_{m}$ throughout CL training. At each iteration, we sample
a mini-batch from both $\mathcal{D}_{t}$ and $\mathcal{D}_{m}$, and update the
CL model $\Phi_{\theta}$ using experience-rehearsal as follows:

$\displaystyle\mathcal{L}_{er}=\mathop{\mathbb{E}}_{(x_{i},y_{i})\sim\mathcal{D}_{t}}\left[\mathcal{L}_{ce}(\sigma(\Phi_{\theta}(x_{i})),y_{i})\right]+\alpha\mathop{\mathbb{E}}_{(x_{k},y_{k})\sim\mathcal{D}_{m}}\left[\mathcal{L}_{ce}(\sigma(\Phi_{\theta}(x_{k})),y_{k})\right]$
(1)

where $\sigma(.)$ is a softmax function and $\mathcal{L}_{ce}$ is a cross-
entropy loss. The learning objective for ER in Equation 1 promotes plasticity
through the supervisory signal from $\mathcal{D}_{t}$ and improves stability
through $\mathcal{D}_{m}$. Thus, buffer size $|D_{m}|$ is critical to
maintaining the right balance between stability and plasticity. In scenarios
where the buffer size is limited ($|D_{t}|\gg|D_{m}|$) due to memory
constraints and/or privacy reasons, repeatedly learning from the restricted
buffer leads to overfitting on the buffered samples. Following Arani et al.
(2022), we employ an EMA of the weights ($\theta_{\scaleto{EMA}{3pt}}$) of the
CL model to enforce consistency in the predictions through $\mathcal{L}_{cr}$
to enable better generalization (see Appendix E.4). By implementing the
complementary system theory for memory, this approach significantly enhances
the model’s capability to effectively retain and utilize learned information.

###  3.3 Shared task-attention module

We seek to facilitate good WP and TP by reducing task interference through
task attention. Unlike multi-head self-attention in vision transformers, we
propose using a shared, compact task-attention module to attend to features
important for the current task. The attention module
$\tau_{\theta}=\\{\tau^{e},\tau^{s},\tau^{tp}\\}$ consists of a feature
encoder $\tau^{e}$, a feature selector $\tau^{s}$, and a task classifier
$\tau^{tp}$. Specifically, $\tau_{\theta}$ is a bottleneck architecture with
$\tau^{e}$ represented by a linear layer followed by Sigmoid activation, while
$\tau^{s}$ is represented by another linear layer with Sigmoid activation. To
orient attention to the current task, we employ a linear classifier
$\tau^{tp}$ that predicts the corresponding task for a given sample.

We denote the output activation of the encoder $f_{\theta}$ as
$z_{f}\in\mathbb{R}^{b\times N_{f}}$, $\tau^{e}$ as
$z_{e}\in\mathbb{R}^{b\times N_{e}}$, $\tau^{s}$ as
$z_{s}\in\mathbb{R}^{b\times N_{s}}$ and that of $\tau^{tp}$ as
$z_{tp}\in\mathbb{R}^{b\times N_{tp}}$, where $N_{f}$, $N_{e}$, $N_{s}$, and
$N_{tp}$ are the dimensions of the output Euclidean spaces, and $b$ is the
batch size. To exploit task-specific features and reduce interference between
tasks, we equip the attention module with a learnable task projection vector
$\delta_{i}$ associated with each task. Each $\delta_{i}\in\mathbb{R}^{1\times
N_{e}}$ is a lightweight $N_{e}$-dimensional randomly initialized vector,
learnable during the corresponding task training and then fixed for the rest
of the CL training. During CL training, for any sample
$x\in\mathcal{D}_{t}\cup\mathcal{D}_{m}$, the incoming features $z_{f}$ and
the corresponding task projection vector $\delta_{t}$ are processed by the
attention module as follows:

$\displaystyle z_{e}=\tau^{e}(z_{f});\quad
z_{s}=\tau^{s}(z_{e}\otimes\delta_{t});\quad
z_{tp}=\tau^{tp}(z_{e}\otimes\delta_{t}).$ (2)

The attention module first projects the features onto a common latent space,
which is then transformed using a corresponding task projection vector. As
each task is associated with a task-specific projection vector, we expect
these projection vectors to capture task-specific transformation coefficients.
To further encourage task-specificity in task-projection vectors, AGILE
entails an auxiliary task classification:

$\mathcal{L}_{tp}=\displaystyle\mathop{\mathbb{E}}_{(x,y)\sim\mathcal{D}_{t}}\left[\mathcal{L}_{ce}(\sigma(z_{tp}),y^{t})\right]$
(3)

where $y^{t}$ is the ground truth of the task label.

Algorithm 1 Proposed Method: AGILE

1:Input: Data streams $\mathcal{D}_{t}$, Model
$\Phi_{\theta}=\\{f_{\theta},\tau_{\theta},\delta_{\theta},g_{\theta}\\}$,
Hyperparameters $\alpha$, $\beta$, $\gamma$, $\lambda$, Memory buffer
$\mathcal{D}_{m}\leftarrow\\{\\}$

2:for all tasks $t\in\\{1,2,..,T\\}$ do

3: for all epochs $e\in\\{1,2,..,E\\}$ do

4: Sample a minibatch $\\{x_{j},y_{j}\\}_{j=1}^{N}\in\mathcal{D}_{t}$

5: $\hat{y}_{j},z_{sj},z_{tpj}=$ TaskAttention$(x_{j})$

6: $\mathcal{L}=\gamma\mathcal{L}_{tp}+\lambda\mathcal{L}_{pd}$

7: if $\mathcal{D}_{m}\neq\emptyset$ then

8: Sample a minibatch $\\{x_{k},y_{k}\\}_{k=1}^{N}\in\mathcal{D}_{m}$

9: $\hat{y}_{k},z_{sk},z_{tpk}=$ TaskAttention$(x_{k})$

10: $\mathcal{L}$ += $\mathcal{L}_{er}+\beta\mathcal{L}_{cr}$

11: Update $\Phi_{\theta}$ and $\mathcal{D}_{m}$

12: Update $\theta_{\scaleto{EMA}{3pt}}$

13:Return: model $\Phi_{\theta}$

Algorithm 2 Task-Attention

function TaskAttention($x$):

$z_{f}=f_{\theta}(x)$

for all $i\leq t$ do

$z^{i}_{e}=\tau^{e}(z_{f})$

$z^{i}_{s}=\tau^{s}(z^{i}_{e}\otimes\delta_{i})$

$z_{tp}=\tau^{tp}(z^{i}_{e}\otimes\delta_{i})$

$\hat{y}^{i}=g^{i}(z^{i}_{s}\otimes z_{f})$

$\hat{y}_{j}=concat(\hat{y}^{i}_{j};\>\>\forall i\leq t)$

return $\hat{y},z_{s},z_{tp}$

###  3.4 Network expansion

As detailed above, the shared attention module has two inputs: the encoder
output $z_{f}$ and the corresponding task projection vector $\delta_{i}$. As
the number of tasks evolves during CL training, we propose to expand our
parameter space by adding new task projection vectors commensurately. These
projection vectors are sampled from a truncated normal distribution with
values outside $[-2,2]$ and redrawn until they are within the bounds. Thus, in
task $t$ there are $\\{\delta_{i}\in{1,2,..,t}\\}$ projection vectors. For
each sample, AGILE performs as many forward passes through the attention
module as the number of seen tasks and generates as many feature importances
($\in\mathbb{R}^{b\times t\times N_{s}}$) (see Figure 1). To encourage the
diversity among these feature importances, we employ a pairwise discrepancy
loss as follows:

$\mathcal{L}_{pd}=-\sum_{i=1}^{t-1}\displaystyle\mathop{\mathbb{E}}_{(x,y)\sim
D_{t}}\lVert\sigma(z^{t}_{s})-stopgrad(\sigma(z^{i}_{s}))\rVert_{1}$ (4)

where $z^{i}_{s}$ is a feature importance generated with the help of the task
projection vector $\delta_{i}$. Since there are multiple feature importances,
selecting the right feature importance is non-trivial for longer task
sequences. Therefore, we propose to expand
$g_{\theta}=\\{g_{\theta}^{i}\\}\>\forall\>i\leq t$ with task-specific
classifiers. Each $g_{\theta}^{i}$ takes corresponding feature importance
$z^{i}_{s}$ and the encoder output $z_{f}$ as input and returns predictions
for classes belonging to the corresponding task. We concatenate all the
outputs from task-specific classifiers and compute the final learning
objective as follows:

$\mathcal{L}=\mathcal{L}_{er}+\beta\mathcal{L}_{cr}+\gamma\mathcal{L}_{tp}+\lambda\mathcal{L}_{pd}$
(5)

where $\beta$, $\gamma$, and $\lambda$ are all hyperparameters. At the end of
each task, we freeze the learned task projection vector and its corresponding
classifier. Figure 1 depicts our proposed approach, which is detailed in
Algorithms 1 and 2.

###  3.5 Implementation Details

We evaluate the effectiveness of our technique in two distinct CL situations:
Class Incremental Learning (Class-IL) and Task Incremental Learning (Task-IL).
In both Task-IL and Class-IL, each task includes a specific number of new
classes that the CL model must learn. A CL model learns multiple tasks, one
after the other, while being able to distinguish all the classes it has
encountered so far. Task-IL is quite similar to Class-IL, with the only
difference being that task labels are also provided during the inference
stage, making it the simplest scenario. We obtained the popular CL datasets,
Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImageNet, by dividing the original
datasets CIFAR10, CIFAR100, and TinyImageNet into number tasks for the Class-
IL and Task-IL scenarios: CIFAR10 into 5 tasks of 2 classes each, CIFAR100
into 5 tasks of 20 classes each, and TinyImageNet into 10 tasks of 20 classes
each. In Figure 2, we divide CIFAR100 into 20 tasks of 5 classes each to
compare with the parameter isolation methods. To allow for a comprehensive
evaluation of different CL methods, we consider two low-buffer regimes 200 and
500, and report average accuracy on all tasks at the end of CL training.

Table 1: Comparison of SOTA methods across various CL scenarios. We provide
the average top-1 ($\%$) accuracy of all tasks after training. † Results of
the single EMA model.

Buffer | Methods | Seq-CIFAR10 | Seq-CIFAR100 | Seq-TinyImageNet
---|---|---|---|---
Class-IL | Task-IL | Class-IL | Task-IL | Class-IL | Task-IL
- | SGD |  19.62$\pm$0.05 |  61.02$\pm$3.33 |  17.49$\pm$0.28 |  40.46$\pm$0.99 |  07.92$\pm$0.26 |  18.31$\pm$0.68
- | Joint |  92.20$\pm$0.15 |  98.31$\pm$0.12 |  70.56$\pm$0.28 |  86.19$\pm$0.43 |  59.99$\pm$0.19 |  82.04$\pm$0.10
- | PNNs | - |  95.13$\pm$0.72 | - |  74.01$\pm$1.11 | - |  67.84$\pm$0.29
200 | ER |  44.79$\pm$1.86 |  91.19$\pm$0.94 |  21.40$\pm$0.22 |  61.36$\pm$0.35 |  8.57$\pm$0.04 |  38.17$\pm$2.00
DER++ |  64.88$\pm$1.17 |  91.92$\pm$0.60 |  29.60$\pm$1.14 |  62.49$\pm$1.02 |  10.96$\pm$1.17 |  40.87$\pm$1.16
CLS-ER† |  61.88$\pm$2.43 |  93.59$\pm$0.87 |  43.38$\pm$1.06 |  72.01$\pm$0.97 |  17.68$\pm$1.65 |  52.60$\pm$1.56
ER-ACE |  62.08$\pm$1.44 |  92.20$\pm$0.57 |  35.17$\pm$1.17 |  63.09$\pm$1.23 |  11.25$\pm$ 0.54 |  44.17$\pm$1.02
Co2L |  65.57$\pm$1.37 |  93.43$\pm$0.78 |  31.90$\pm$0.38 |  55.02$\pm$0.36 |  13.88$\pm$0.40 |  42.37$\pm$0.74
GCR |  64.84$\pm$1.63 |  90.8$\pm$1.05 |  33.69$\pm$1.40 |  64.24$\pm$0.83 |  13.05$\pm$0.91 |  42.11$\pm$1.01
DRI |  65.16$\pm$1.13 |  92.87$\pm$0.71 | - | - |  17.58$\pm$1.24 |  44.28$\pm$1.37
AGILE |  69.37$\pm$0.40 |  94.25$\pm$0.42 |  45.73$\pm$0.15 |  74.37$\pm$0.34 |  20.19$\pm$1.65 |  53.47$\pm$1.60
500 | ER |  57.74$\pm$0.27 |  93.61$\pm$0.27 |  28.02$\pm$0.31 |  68.23$\pm$0.17 |  9.99$\pm$0.29 |  48.64$\pm$0.46
DER++ |  72.70$\pm$1.36 |  93.88$\pm$0.50 |  41.40$\pm$0.96 |  70.61$\pm$0.08 |  19.38$\pm$1.41 |  51.91$\pm$0.68
CLS-ER† |  70.40$\pm$1.21 |  94.35$\pm$0.38 |  49.97$\pm$0.78 |  76.37$\pm$0.12 |  24.97$\pm$0.80 |  61.57$\pm$0.63
ER-ACE |  68.45$\pm$1.78 |  93.47$\pm$1.00 |  40.67$\pm$0.06 |  66.45$\pm$0.71 |  17.73$\pm$ 0.56 |  49.99$\pm$1.51
Co2L |  74.26$\pm$0.77 |  95.90$\pm$0.26 |  39.21$\pm$0.39 |  62.98$\pm$0.58 |  20.12$\pm$0.42 |  53.04$\pm$0.69
GCR |  74.69$\pm$0.85 |  94.44$\pm$0.32 |  45.91$\pm$1.30 |  71.64$\pm$2.10 |  19.66$\pm$0.68 |  52.99$\pm$0.89
DRI |  72.78$\pm$1.44 |  93.85$\pm$0.46 | - | - |  22.63$\pm$0.81 |  52.89$\pm$0.60
AGILE |  75.69$\pm$0.62 |  95.51$\pm$0.32 |  52.65$\pm$0.93 |  78.21$\pm$0.15 |  29.30$\pm$0.53 |  64.74$\pm$0.56

##  4 Results

For all of our experiments, we employ ResNet-18 without pretraining as a
backbone. Task projection vectors in AGILE are implemented as learnable
parameters, while the shared task-attention module is an undercomplete
autoencoder-like structure with an additional task-prediction classifier. We
emphasize that we use a single expanding head so as not to be confused with a
multiple-head setting. It is to remedy the problem of selecting the right task
projection vector during inference. We trained all our models on NVIDIA
GeForce RTX 2080 Ti (11GB). On average, it took around 2 hours to train AGILE
on Seq-CIFAR10 and Seq-CIFAR100, and approximately 8 hours to train on Seq-
TinyImageNet.

###  4.1 Experimental results

Table 1 compares AGILE with recent rehearsal-based approaches in Class-IL and
Task-IL scenarios. The associated forgetting analysis is available in Appendix
5. Several observations can be made from these results: (1) AGILE outperforms
rehearsal-based approaches by a large margin across almost all datasets and
buffer sizes, highlighting the importance of task attention in CL. (2)
Approaches using consistency regularization (e.g., DER++ and CLS-ER) perform
considerably better than others. However, AGILE demonstrates that
regularization alone is insufficient to discriminate classes from different
tasks. (3) While approaches addressing representation drift (e.g., Co2L and
ER-ACE) work well in simpler datasets, they struggle in challenging ones. For
instance, in Seq-TinyImageNet with a small buffer-to-class ratio, their
performance lags behind AGILE. The reliance of shared task attention on task
projection vectors in AGILE helps limit representation drift by fixing these
vectors after corresponding task training. (4) Approaches enhancing the
quality or quantity of buffered samples (e.g., GCR and DRI) improve over
vanilla ER. However, the additional computational overhead in selecting or
generating buffered samples can be problematic on resource-constrained
devices. In contrast, AGILE, with compact task attention using task projection
vectors, outperforms rehearsal-based approaches significantly with minimal
memory and computational overhead.

Task-specific learning approaches, whether within a fixed model capacity or
through growth, involve isolating parameters to minimize task interference in
CL. AGILE, similarly, uses task projection vectors to reduce interference
between tasks. Figure 2 compares AGILE with fixed capacity models (NISPA,
CLNP) and growing architectures (PNN, PAE, PackNet, CPG) trained on Seq-
CIFAR100 with 20 tasks (buffer size 500 for AGILE). At the end of CL training
across 20 tasks, AGILE significantly outperforms baselines with an average of
83.94%. Regarding parameter growth, PNN grows excessively, CPG grows by 1.5x,
and PAE by 2x. In contrast, AGILE grows marginally by 1.01x, even for 20
tasks, without compromising performance in longer task sequences (Table 3).

Figure 2: Comparison of AGILE with task-specific learning approaches in Task-
IL setting. We report the accuracy on all tasks at the end of CL training with
an average across all tasks in the legend. AGILE outperforms other baselines
with little memory overhead. Figure 3: Latent features and task projection
vectors after training on Seq-CIFAR100 with 5 tasks. (Left) t-SNE
visualization of the latent features of the shared task attention module in
the absence of task projection vectors; (Middle) Task projection vectors along
leading principle components. (Right) t-SNE visualization of latent features
of the shared task attention module in the presence of task projection
vectors. Task projection vectors specialize in transforming the latent
representations of shared task-attention module towards the task distribution,
thereby reducing interference.

###  4.2 How AGILE facilitates a good WP and TP?

Figure 3 (left) shows the visualization of t-distributed stochastic neighbor
embedding (t-SNE) of latent features in the absence of task projection
vectors. As can be seen, samples belonging to different tasks are distributed
across the representation space. On the other hand, Figure 3 (right) shows a
t-SNE visualization of well-clustered latent features in the presence of task
projection vectors. For each sample, we visualize its latent features in task
attention after transforming it with the corresponding task projection vector.
We also show how task projection vectors are distributed along the principal
components using PCA in Figure 3 (middle). AGILE entails a shared task-
attention module and as many lightweight, learnable task projection vectors as
the number of tasks. As each task projection vector learns the task-specific
transformation, they project samples belonging to the corresponding task
differently, resulting in less interference and improved WP and TP in CL.

###  4.3 Ablation study

We aim to determine the impact of each component of AGILE. As mentioned above,
AGILE utilizes consistency regularization through the use of EMA and a shared
task-attention mechanism with a single expanding head. Each of these
components brings unique benefits to AGILE: consistency regularization aids in
consolidating previous task information in scenarios with low buffer sizes,
while EMA functions as an ensemble of task-specific models. Furthermore, the
EMA provides better stability and acts as an inference model in our method.
AGILE employs shared task-attention using task-specific projection vectors,
one for each task. As the number of tasks increases, selecting the appropriate
task (projection vector) without task identity becomes increasingly difficult
(Aljundi et al., 2017). To address this issue, we implement a single expanding
head instead of a single head, where each projection vector is responsible for
classes of the corresponding task. Table 2 presents the evaluation of
different components in Seq-TinyImageNet (buffer size 500). As shown, AGILE
takes advantage of each of these components and improves performance in both
Class-IL and Task-IL settings.

Table 2: Comparison of the contributions of each of the components in AGILE.
Consistency regularization in the absence of EMA implies consistency
regularization by storing past logits.

Consistency regularization | EMA | Single-expanding head | Task-attention | Class-IL | Task-IL
---|---|---|---|---|---
✓ | ✓ | ✓ | ✓ |  29.30$\pm$0.53 |  64.74$\pm$0.56
✓ | ✓ | ✓ | ✗ |  25.43$\pm$1.07 |  58.89$\pm$0.84
✓ | ✓ | ✗ | ✗ |  24.97$\pm$0.80 |  61.57$\pm$0.63
✓ | ✗ | ✗ | ✗ |  19.38$\pm$1.41 |  51.91$\pm$0.68
✗ | ✗ | ✗ | ✗ |  9.99$\pm$0.29 |  48.64$\pm$0.46

###  4.4 Parameter growth

AGILE entails as many task projection vectors as the number of tasks.
Therefore, the CL model grows in size as and when it encounters a new task. To
this end, we compare the parameter growth in AGILE with respect to the fixed
capacity model and the PNNs in Table 3. AGILE encompasses as many lightweight,
learnable task projection vectors as the number of tasks, specialized in
transforming the latent representations of the shared task-attention module
towards the task distribution with negligible memory and computational
overhead. Compared to fixed capacity models, which suffer from capacity
saturation, AGILE grows marginally in size and facilitates a good within-task
and task-id prediction, thereby resulting in superior performance even under
longer task sequences. On the other hand, PNNs grow enormously in size,
quickly rendering them unscalable in longer task sequences.

Table 3: Growth in the number of parameters (millions) for different number of task sequences in Seq-CIFAR100. Method | 5 tasks | 10 tasks | 20 tasks
---|---|---|---
Fixed capacity model (with EMA) | 22.461 | 22.461 | 22.461
AGILE | 23.074 | 23.079 | 23.089
PNNs | 297.212 | 874.015 | 2645.054

##  5 Model characteristics

A broader overview of the characteristics of the model is a necessary
precursor for the deployment of CL in the real world. To provide a qualitative
analysis, we evaluate the recency bias and model calibration for AGILE and
other CL methods trained on Seq-CIFAR100 with a buffer size of 500 in Class-IL
scenario.

Figure 4: (Left) Confusion matrix for various CL models. ER and DER++ show
high recency biases, while AGILE makes evenly distributed predictions. (Right)
Reliability diagram with ECE indicating AGILE’s well-calibrated performance
and lowest ECE value. – denotes the perfect calibration. All models are
trained on Seq-CIFAR100, 5 tasks.

##### Model Calibration:

CL systems are considered well calibrated when prediction probabilities
accurately reflect correctness likelihood. Despite recent high DNN accuracy,
overconfident predictions (Guo et al., 2017) reduce reliability in safety-
critical applications. Expected Calibration Error (ECE) estimates model
reliability by gauging the difference between confidence and accuracy
expectations. In Figure 4(right), different CL methods, using a calibration
framework (Küppers et al., 2020), are compared. AGILE achieves the lowest ECE,
showing significant calibration. By minimizing task interference, AGILE
supports informed decision-making, reducing CL overconfidence.

##### Task Recency Bias:

When a CL model learns a new task sequentially, it encounters a few samples of
previous tasks while aplenty of the current task, thus skewing the learning
towards the recent task (Hou et al., 2019). Ideally, the CL model is expected
to have the least recent bias with predictions spread across all tasks evenly.
To analyze task-recency bias, we compute the confusion matrix for different CL
models .For any test sample, if the model predicts any of the classes within
the sample’s true task label, it is considered to have predicted the task
label accurately. Figure 4 (left) shows that ER and DER++ tend to predict most
samples as classes in the most recent task. On the other hand, the predictions
of AGILE are evenly distributed on the diagonal. Essentially, AGILE captures
task-specific information through separate task projection vectors and reduces
interference between tasks, resulting in the least recency bias.

##### Forgetting Analysis:

Learning continuously on a sequence of novel tasks often results in the new
information interfering with the consolidated knowledge in the model, causing
catastrophic forgetting. Chaudhry et al. (2018) introduced the forgetting
measure ($F_{T}$) to quantify the extent to which previously learned
information is retained in the CL model. Let $a_{ij}$ be the test accuracy of
the model for task $j$ after learning task $i$. Then, after training a CL
model for $T$ tasks, the forgetting measure $F_{T}$ for the model is defined
as,

$F_{T}=\frac{1}{T-1}\sum_{t=0}^{T-1}a_{t}^{*}-a_{tT}$ (6)

where $a_{t}^{*}$ denotes the best test accuracy for the task $t$. After
training for $T$ tasks, $a_{t}^{*}$ is typically computed at the task
boundaries as,

$a_{t}^{*}=\max_{l\in\\{t,t+1,..,T-1\\}}a_{lt},\forall{t<T}$ (7)

AGILE’s inference model, the stochastically updated EMA, can achieve maximum
accuracy for a previous task at any point during training. To compute the
forgetting measure $F_{T}$, we assess AGILE on previous tasks after each
epoch, comparing forgetting measures with different CL methods across datasets
and buffer sizes in the Class-IL setting (see Table 4). AGILE consistently
exhibits significantly lower forgetting compared to other baselines due to its
task attention module and task projection vectors, which reduce interference
between tasks.

Table 4: Comparison of the forgetting measure for different CL methods for all scenarios reported in Table 1. Compared to other baselines AGILE suffers the least forgetting. | Buffer size
---
Methods | Seq-CIFAR10 | Seq-CIFAR100 | Seq-TinyImageNet
200 | ER |  61.24$\pm$2.62 |  75.54$\pm$0.45 |  76.37$\pm$0.53
DER++ |  32.59$\pm$2.32 |  68.77$\pm$1.72 |  72.74$\pm$0.56
AGILE |  25.40$\pm$0.15 |  22.74$\pm$2.52 |  36.95$\pm$0.51
500 | ER |  45.35$\pm$0.07 |  67.74$\pm$1.29 |  75.27$\pm$0.17
DER++ |  22.38$\pm$4.41 |  50.99$\pm$2.52 |  64.58$\pm$2.01
AGILE |  17.57$\pm$1.45 |  22.71$\pm$0.07 |  23.97$\pm$0.73

##  6 Conclusion

We proposed AGILE, a novel rehearsal-based CL learning approach that employs a
compact, shared task-attention module with task-specific projection vectors to
effectively reduce task interference in CL. AGILE encompasses as many
lightweight, learnable task projection vectors as the number of tasks,
specialized in transforming the latent representations of shared task-
attention module towards the task distribution with negligible memory and
computational overhead. By reducing interference between tasks, AGILE
facilitates good within-task and task-id prediction, resulting in superior
performance across CL scenarios. With extensive empirical evaluation, we
demonstrate that AGILE outperforms the rehearsal-based and parameter-isolation
approaches by a large margin, signifying the efficacy of task attention in CL.
Extending AGILE to rehearsal-free CL, and exploring different forms of shared
task-attention are some of the useful research directions for this work.

## References

  * Aljundi et al. (2017) Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars.  Expert gate: Lifelong learning with a network of experts.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 3366–3375, 2017.
  * Arani et al. (2022) Elahe Arani, Fahad Sarfraz, and Bahram Zonooz.  Learning fast, learning slow: A general continual learning method based on complementary learning system.  In _International Conference on Learning Representations_ , 2022.
  * Baars (1994) Bernard J Baars.  A global workspace theory of conscious experience.  _Consciousness in philosophy and cognitive neuroscience_ , pp. 149–171, 1994.
  * Baars (2005) Bernard J Baars.  Global workspace theory of consciousness: toward a cognitive neuroscience of human experience.  _Progress in brain research_ , 150:45–53, 2005.
  * Baars et al. (2021) Bernard J Baars, Natalie Geld, and Robert Kozma.  Global workspace theory (gwt) and prefrontal cortex: Recent developments.  _Frontiers in Psychology_ , pp. 5163, 2021.
  * Bang et al. (2021) Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi.  Rainbow memory: Continual learning with a memory of diverse samples.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 8218–8227, 2021.
  * Bengio (2017) Yoshua Bengio.  The consciousness prior.  _arXiv preprint arXiv:1709.08568_ , 2017.
  * Bhat et al. (2022) Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani.  Consistency is the key to further mitigating catastrophic forgetting in continual learning.  In _Conference on Lifelong Learning Agents_ , pp. 1195–1212. PMLR, 2022.
  * Bhat et al. (2023) Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani.  Task-aware information routing from common representation space in lifelong learning.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=-M0TNnyWFT5.
  * Bonicelli et al. (2022) Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, and Simone Calderara.  On the effectiveness of lipschitz-driven rehearsal in continual learning.  _Advances in Neural Information Processing Systems_ , 35:31886–31901, 2022.
  * Boschini et al. (2022) Matteo Boschini, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara.  Continual semi-supervised learning through contrastive interpolation consistency.  _Pattern Recognition Letters_ , 162:9–14, 2022.
  * Buzzega et al. (2020) Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara.  Dark experience for general continual learning: a strong, simple baseline.  _Advances in neural information processing systems_ , 33:15920–15930, 2020.
  * Caccia et al. (2022) Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky.  New insights on reducing abrupt representation change in online continual learning.  In _International Conference on Learning Representations_ , 2022.
  * Cha et al. (2021) Hyuntak Cha, Jaeho Lee, and Jinwoo Shin.  Co2l: Contrastive continual learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 9516–9525, 2021.
  * Chaudhry et al. (2018) Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr.  Riemannian walk for incremental learning: Understanding forgetting and intransigence.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 532–547, 2018.
  * Choi & Choi (2022) Hong-Jun Choi and Dong-Wan Choi.  Attractive and repulsive training to address inter-task forgetting issues in continual learning.  _Neurocomputing_ , 500:486–498, 2022.
  * Douillard et al. (2022) Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord.  Dytox: Transformers for continual learning with dynamic token expansion.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 9285–9295, 2022.
  * Fedus et al. (2020) William Fedus, Dibya Ghosh, John D Martin, Marc G Bellemare, Yoshua Bengio, and Hugo Larochelle.  On catastrophic interference in atari 2600 games.  _arXiv preprint arXiv:2002.12499_ , 2020.
  * French (1999) Robert M French.  Catastrophic forgetting in connectionist networks.  _Trends in cognitive sciences_ , 3(4):128–135, 1999.
  * Golkar et al. (2019) Siavash Golkar, Micheal Kagan, and Kyunghyun Cho.  Continual learning via neural pruning.  In _Real Neurons & Hidden Units: Future directions at the intersection of neuroscience and artificial intelligence @ NeurIPS 2019_, 2019.
  * Goyal & Bengio (2020) Anirudh Goyal and Yoshua Bengio.  Inductive biases for deep learning of higher-level cognition.  _arXiv preprint arXiv:2011.15091_ , 2020.
  * Guo et al. (2017) Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  In _International conference on machine learning_ , pp. 1321–1330. PMLR, 2017.
  * Gurbuz & Dovrolis (2022) Mustafa B Gurbuz and Constantine Dovrolis.  Nispa: Neuro-inspired stability-plasticity adaptation for continual learning in sparse networks.  In _International Conference on Machine Learning_ , pp. 8157–8174. PMLR, 2022.
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 1026–1034, 2015.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  _stat_ , 1050:9, 2015.
  * Hou et al. (2019) Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin.  Learning a unified classifier incrementally via rebalancing.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 831–839, 2019.
  * Hung et al. (2019a) Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen.  Compacting, picking and growing for unforgetting continual learning.  _Advances in Neural Information Processing Systems_ , 32, 2019a.
  * Hung et al. (2019b) Steven CY Hung, Jia-Hong Lee, Timmy ST Wan, Chein-Hung Chen, Yi-Ming Chan, and Chu-Song Chen.  Increasingly packing multiple facial-informatics modules in a unified deep-learning model via lifelong learning.  In _Proceedings of the 2019 on International Conference on Multimedia Retrieval_ , pp. 339–343, 2019b.
  * Jin et al. (2021) Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren.  Gradient-based editing of memory examples for online task-free continual learning.  _Advances in Neural Information Processing Systems_ , 34:29193–29205, 2021.
  * Juliani et al. (2022) Arthur Juliani, Kai Arulkumaran, Shuntaro Sasai, and Ryota Kanai.  On the link between conscious function and general intelligence in humans and machines.  _Transactions on Machine Learning Research_ , 2022.  ISSN 2835-8856.  Survey Certification.
  * Kim et al. (2022) Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu.  A theoretical study on solving continual learning.  _Advances in Neural Information Processing Systems_ , 35:5065–5079, 2022.
  * Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ , 114(13):3521–3526, 2017.
  * Küppers et al. (2020) Fabian Küppers, Jan Kronenberger, Amirhossein Shantia, and Anselm Haselhoff.  Multivariate confidence calibration for object detection.  In _The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_ , June 2020.
  * Lesort et al. (2019) Timothée Lesort, Andrei Stoian, and David Filliat.  Regularization shortcomings for continual learning.  _arXiv preprint arXiv:1912.03049_ , 2019.
  * Mallya & Lazebnik (2018) Arun Mallya and Svetlana Lazebnik.  Packnet: Adding multiple tasks to a single network by iterative pruning.  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , pp. 7765–7773, 2018.
  * McCloskey & Cohen (1989) Michael McCloskey and Neal J Cohen.  Catastrophic interference in connectionist networks: The sequential learning problem.  In _Psychology of learning and motivation_ , volume 24, pp. 109–165. Elsevier, 1989.
  * Mermillod et al. (2013) Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin.  The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013.
  * Parisi et al. (2019) German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter.  Continual lifelong learning with neural networks: A review.  _Neural Networks_ , 113:54–71, 2019.
  * Ratcliff (1990) Roger Ratcliff.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.  _Psychological review_ , 97(2):285, 1990.
  * Rusu et al. (2016) Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell.  Progressive neural networks.  _arXiv preprint arXiv:1606.04671_ , 2016.
  * Sarfraz et al. (2022) Fahad Sarfraz, Elahe Arani, and Bahram Zonooz.  Synergy between synaptic consolidation and experience replay for general continual learning.  In Sarath Chandar, Razvan Pascanu, and Doina Precup (eds.), _Proceedings of The 1st Conference on Lifelong Learning Agents_ , volume 199 of _Proceedings of Machine Learning Research_ , pp. 920–936. PMLR, 22–24 Aug 2022.
  * Sarfraz et al. (2023) Fahad Sarfraz, Elahe Arani, and Bahram Zonooz.  Error sensitivity modulation based experience replay: Mitigating abrupt representation drift in continual learning.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=zlbci7019Z3.
  * Serra et al. (2018) Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou.  Overcoming catastrophic forgetting with hard attention to the task.  In _International Conference on Machine Learning_ , pp. 4548–4557. PMLR, 2018.
  * Silver et al. (2018) David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al.  A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.  _Science_ , 362(6419):1140–1144, 2018.
  * Tiwari et al. (2022) Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy.  Gcr: Gradient coreset based replay buffer selection for continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 99–108, 2022.
  * VanRullen & Kanai (2021) Rufin VanRullen and Ryota Kanai.  Deep learning and the global workspace theory.  _Trends in Neurosciences_ , 44(9):692–704, 2021.
  * Verwimp et al. (2021) Eli Verwimp, Matthias De Lange, and Tinne Tuytelaars.  Rehearsal revealed: The limits and merits of revisiting samples in continual learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 9385–9394, 2021.
  * Vijayan et al. (2023) Preetha Vijayan, Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani.  Trire: A multi-mechanism learning paradigm for continual knowledge retention and promotion.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Vitter (1985) Jeffrey S Vitter.  Random sampling with a reservoir.  _ACM Transactions on Mathematical Software (TOMS)_ , 11(1):37–57, 1985.
  * Wang et al. (2022a) Zhen Wang, Liu Liu, Yiqun Duan, and Dacheng Tao.  Continual learning through retrieval and imagination.  In _AAAI Conference on Artificial Intelligence_ , volume 8, 2022a.
  * Wang et al. (2022b) Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tiehang Duan, and Mingchen Gao.  Improving task-free continual learning by distributionally robust memory evolution.  In _International Conference on Machine Learning_ , pp. 22985–22998. PMLR, 2022b.
  * Wang et al. (2022c) Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister.  Learning to prompt for continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 139–149, 2022c.

##  Appendix A Limitations and Future Work

We proposed AGILE to mitigate task interference and, in turn, facilitate good
WP and TP through task attention. AGILE entails shared task attention and as
many task projection vectors as the number of tasks. Task projection vectors
capture task-specific information and are frozen after corresponding task
training. Selection of the right projection vector during inference is
nontrivial in longer-task sequences. To address this lacuna, we employ a
single expanding head with task-specific classifiers. However, a better
alternative can be developed to fully exploit task specificity in the task
projection vectors. Second, AGILE strongly assumes no overlap between classes
of two tasks in Class-IL / Task-IL settings. As each task-projection vector
captures different information when there is non-overlap between classes, an
overlap might create more confusion among projection vectors, resulting in
higher forgetting. Furthermore, the shared task-attention module is still
prone to forgetting due to the sequential nature of CL training.

We concentrated on integrating our approach with the ResNet18 architecture
within the context of convolutional neural networks, a common practice in
continual learning literature. Our method, AGILE, demonstrates superior
performance compared to existing methods under uniform experimental
conditions, attributed to our innovative task attention mechanism. Notably,
AGILE’s design, utilizing undercomplete auto-encoders and tensors without
relying on architecture-specific layers like convolutions, suggests its
potential applicability beyond vision tasks, including adaptation to
transformer models. However, adapting AGILE to alternative architectures such
as transformers involves considerable experimentation and hyperparameter
adjustments, which exceeds the scope of the current discussion. We acknowledge
the value of such extensions to enhance AGILE’s generalizability and consider
it an exciting direction for future research. Additionally, improving task-
projection vector selection criterion, extending AGILE to other more complex
Class-IL / Task-IL scenarios, and reducing forgetting in shared task-attention
module through parameter isolation are some of the useful research directions
for this work.

##  Appendix B Theoretical Insight

We consider a widely adopted Class-IL setting within which classes and their
domains appear at most in one task, i.e., there is no overlap of classes
between tasks.

######  Definition 1.

(Class-IL): The CL model encounters $t\in\\{1,2....,T\\}$ tasks with
$j\in\\{1,2....,J\\}$ classes per task sequentially such that the classes
belonging to different tasks are disjoint i.e. for task-specific data
$(\mathbf{X}_{t,j},\mathbf{Y}_{t,j})\in\mathcal{D}_{t}$,
$\mathbf{Y}_{t,j}\cap\mathbf{Y}_{t^{\prime},j^{\prime}}=\emptyset,\>\>\forall
j\neq j^{\prime},\>\>\forall t\neq t^{\prime}$. Given such a setting, the
primary goal of the CL model is to learn
$\mathbf{P}\left(y\in\mathbf{Y}_{t,j}\mid\mathcal{D}\right)$.

For any ground event $\mathcal{D}$, Kim et al. (2022) partitioned this
probability into two sub-problems, namely within-task prediction (WP)
probability: $\mathbf{P}\left(y\in\mathbf{Y}_{t,j}\mid
y\in\mathbf{Y}_{t},D\right)$ and task-id prediction (TP) probability:
$\mathbf{P}\left(y\in\mathbf{Y}_{t}\mid D\right)$ as follows:

$\displaystyle\mathbf{P}\left(y\in\mathbf{Y}_{t_{0},j_{0}}\mid D\right)$
$\displaystyle=\sum_{t=1,\ldots,n}\mathbf{P}\left(y\in\mathbf{Y}_{t,j_{0}}\mid
y\in\mathbf{Y}_{t},D\right)\mathbf{P}\left(y\in\mathbf{Y}_{t}\mid D\right)$
(8) $\displaystyle=\mathbf{P}\left(y\in\mathbf{Y}_{t_{0},j_{0}}\mid
y\in\mathbf{Y}_{t_{0}},D\right)\mathbf{P}\left(y\in\mathbf{Y}_{t_{0}}\mid
D\right)$

where $t_{0}$ and $j_{0}$ represent a particular task and one of its classes,
respectively. TP indicates the task-id of the sample, and WP means that the
prediction for a test instance is only done within the classes of the task to
which the test instance belongs, which is basically the Task-IL problem as
follows:

######  Definition 2.

(Task-IL): Given the same setting as in Definition 1, the goal of the CL model
is to learn the mapping function
$f:\mathbf{X}\times\mathbf{T}\rightarrow\mathbf{Y}$ i.e. predict the class
label $y_{j_{0}}\in\mathbf{Y}_{t_{0}}$ for a sample $x$ from task
$t_{0}\in\mathbf{T}$.

In fact, it is possible to achieve almost zero forgetting in Task-IL (e.g. see
Serra et al. (2018)). However, Class-IL remains challenging due to the
difficulty in establishing class boundaries between classes of current and
previous tasks. We seek to uncover how Class-IL performance can be further
improved. To this end, Let $\mathcal{H}(p,q)=-\sum_{i}p_{i}$ log $q_{i}$ be
the cross-entropy of two probability distributions p and q. We use cross-
entropy as a performance measure to assess the relation between WP, TP, and
Class-IL. We define the cross-entropy of WP, TP and Class-IL as
$\mathcal{H}_{WP}(x)=\mathcal{H}\left(\tilde{y},\left\\{\mathbf{P}\left(x\in\mathbf{X}_{t_{0},j}\mid
x\in\mathbf{X}_{t_{0}},D\right)\right\\}_{j}\right)$, $\mathcal{H}_{TP}(x)=$
$\mathcal{H}\left(\bar{y},\left\\{\mathbf{P}\left(x\in\mathbf{X}_{t}\mid
D\right)\right\\}_{t}\right)$ and $\mathcal{H}_{Class-IL}(x)$ =
$\mathcal{H}\left(y,\left\\{\mathbf{P}\left(x\in\mathbf{X}_{t,j}\mid
D\right)\right\\}_{t,j}\right)$ respectively, where $\tilde{y}$, $\bar{y}$ and
$y$ are ground-truth values $\in\\{0,1\\}$. We now describe how WP, TP, and
Class-IL are related to each other, and how interference affects their
performance.

######  Theorem 3.

If $\mathcal{H}_{WP}(x)\leq\epsilon$ and $\mathcal{H}_{TP}(x)\leq\xi$, then
$\mathcal{H}_{Class-IL}(x)\leq\epsilon+\xi$ (Kim et al., 2022).

For any $\epsilon>0$ and $\xi>0$, Theorem 3 establishes a functional
relationship between WP, TP, and Class-IL. The theorem states that if
$\mathcal{H}_{WP}$ and $\mathcal{H}_{TP}$ are bounded by $\epsilon$ and $\xi$
respectively, then the Class-IL cross-entropy loss $\mathcal{H}_{Class-IL}$ is
bounded by the sum of them. Therefore, having good TP and WP, lowers the upper
bound of the Class-IL loss.

Task interference arises when multiple tasks share a common observation space
but have different learning goals. In the presence of task interference, both
WP and TP struggle to find the right class or task, resulting in reduced
performance and higher cross-entropy loss. Specifically, in the presence of
task interference, the upper bounds of WP and TP increase, indicating the
corresponding decrease in performance, i.e.
$\mathcal{H}_{WP}(x)\leq\epsilon+\hat{\epsilon}$ and
$\mathcal{H}_{TP}(x)\leq\xi+\hat{\xi}$. According to Theorem 3, the upper
bound of $\mathcal{H}_{Class-IL}$ will also increase proportionately. Assuming
$\hat{\epsilon},\hat{\xi}\gg 0$, task interference can have a substantial
effect on overall Class-IL performance.

Therefore, it is quintessential to reduce task interference in CL to ensure
optimum performance. Combining intuitions from both biological and theoretical
findings, we hypothesize that focusing on the information relevant to the
current task can facilitate good TP and WP, and consequently systemic
generalization by filtering out extraneous or interfering information. In the
following section, we describe in detail how we mitigate interference between
tasks through task attention.

##  Appendix C Broader Related Works

In Section 2, we compared and contrasted several methods that are closely
related to AGILE. We will now explore the broader related works whose problem
statement overlaps with that of AGILE.

Continually learning on a sequence of tasks blurs the decision boundaries
between the classes of current task and previous tasks (Lesort et al., 2019).
Some approaches address inter-task forgetting indirectly by mitigating the
effect of class imbalance in rehearsal-based learning. Attractive and
repulsive training (ART) (Choi & Choi, 2022), aims to reduce the correlation
between new and old classes through a training strategy that attracts samples
from the same class while repelling other similar samples. LUCIR (Hou et al.,
2019) proposes a new framework to learn a unified classifier using a
combination of cosine normalization, less-forget constraint, and inter-class
separation. Although these approaches aim to reduce catastrophic forgetting in
CL, they address the fine-grained problem of inter-task class separation
without explicitly encouraging a common representation and task-specific
learning. In vision transformers, DyToX (Douillard et al., 2022) modified the
final multi-head self-attention layer to act as a task attention block using
task tokens. However, Dytox is an adhoc approach for transformer architectures
and cannot be extended to convolutional architectures.

Although rehearsal-based approaches have been highly efficient in CL, repeated
learning on a small subset of previous task data results in overfitting, thus
inhibiting generalization (Verwimp et al., 2021). Several methods employ
augmentation techniques, either by combining multiple data points into one
(Boschini et al., 2022) or by producing multiple versions of the same buffer
sample (Bang et al., 2021). Gradient-based Memory EDiting (GMED) (Jin et al.,
2021) proposes editing individual examples stored in the buffer to create more
challenging data to alleviate catastrophic forgetting. Distributionally Robust
Optimization (DRO) (Wang et al., 2022b) proposes a principled memory evolution
framework to evolve buffer data distribution focusing on population-level and
distribution-level evolution. Contrary to the methods that modify memory
buffer, Lipschitz Driven Rehearsal (LiDER) (Bonicelli et al., 2022) proposes a
regularization objective that induces decision boundary smoothness by
enforcing Lipschitz continuity of the model with respect to replay samples.
Since these approaches focus on the orthogonal problem of mitigating
overfitting in buffer data, they can be integrated into popular rehearsal-
based approaches to further improve generalization in CL.

Learning to prompt (L2P) (Wang et al., 2022c) learns to dynamically prompt a
pre-trained model to learn tasks sequentially under different task
transitions. Prompts are small, learnable parameters, which are maintained in
a memory space. The objective is to optimize prompts to instruct the model
prediction and explicitly manage task-invariant and task-specific knowledge
while maintaining model plasticity. AGILE differs from L2P in several ways:
Firstly, the backbone network is learnable in AGILE, while it is fixed in L2P.
As the ability to capture task-agnostic information relies heavily on the
representations captured in the backbone, a fixed backbone serves the purpose
only when it is pre-trained on a huge dataset. Therefore, we let the backbone
learn throughout the training. Second, AGILE entails a shared feature encoder
and task-attention module, and as many task projection vectors as the number
of tasks. Each task projection vector is a lightweight learnable vector
associated with a particular task, specialized in transforming the latent
representations of the shared task-attention module towards the task
distribution. On the other hand, L2P does not entail specialized task-
attention modules; instead, it steers the pre-trained model toward task
distribution using learnable task-specific prompts. Although similar in their
dynamics, L2P aims to re-direct a pre-trained model towards a task
distribution, while AGILE is a fully learnable, rehearsal-based CL approach
that transforms the latent representations of the shared task-attention module
towards the task distribution.

##  Appendix D Computational Cost

The task attention module is a lightweight undercomplete autoencoder with two
MLP layers followed by ReLu activation, and the task projection vectors are
implemented as learnable parameters. Thus, AGILE does not introduce any
complex modules and adds negligible computational overhead. Table 5 provides
the relative time taken to train on Seq-CIFAR100 for 5 tasks with a buffer
size of 200. As can be seen, even with as many forward passes through the
attention module as the number of tasks, the additional computational overhead
incurred in AGILE is very minimal.

Table 5: Relative training time comparison of several CL methods trained on Seq-CIFAR100 with buffer size 200.  Method | ER | DER++ | CLS-ER | AGILE
---|---|---|---|---
Relative time taken | 1x | 1.5x | 1.6x | 1.7x

##  Appendix E Characterization of AGILE

###  E.1 Task-wise performance

As CL model learns new tasks in succession, it is exposed to a limited number
of examples of earlier tasks, while receiving many more from the task
currently being learned. This can cause the model to place more emphasis on
recent tasks and less on earlier ones, leading to a bias towards the most
recent tasks. Ideally, the CL model is expected to have the least recent bias
with predictions spread across all tasks evenly. Figure 5 provides task-wise
performance of CL models trained on Seq-CIFAR100 with buffer size 500. As can
be seen, the performances of ER and DER++ emanate mostly from the final task,
while that of AGILE is much more distributed across tasks.

Figure 5: Task-wise performance of CL models trained on Seq-CIFAR100 with
buffer size 500. The performances of ER and DER++ mainly emanate from the most
recent task, while that of AGILE comes more evenly from all the tasks.

###  E.2 Stability-Plasticity Dilemma

Stability of a CL model refers to its ability to retain previously learned
knowledge, whereas plasticity refers to its ability to adapt to novel
information. Every CL model is faced with the dilemma of finding the optimal
balance between being stable and being plastic. Consequently, measuring this
stability-plasticity trade-off plays a crucial role in analyzing CL models.
Sarfraz et al. (2022) proposed a Trade-off measure that provides a formal way
to estimate the balance between stability and plasticity of the model. If a CL
model is trained for $T$ tasks, then the stability ($\mathcal{S}$) of the
model is defined as the average performance of all previous tasks, that is,

$\mathcal{S}=\sum_{t=0}^{T-1}a_{Tt}$ (9)

The plasticity ($\mathcal{P}$) of the model is calculated as the average
performance of learning every task for the first time, that is,

$\mathcal{P}=\sum_{t=0}^{T}a_{tt}$ (10)

The stability-plasticity trade-off is then measured as the harmonic mean of
$\mathcal{S}$ and $\mathcal{P}$.

$\textit{Trade-
off}=\frac{2\times\mathcal{S}\times\mathcal{P}}{\mathcal{S}+\mathcal{P}}$ (11)

Figure 6 compares the stability, plasticity, and trade-off of different CL
methods trained on Seq-CIFAR100 with 5 tasks for a buffer size of 500. While
ER and DER++ quickly adapt to novel tasks, the new information interferes with
the previously learned information leading to low stability. On the other
hand, AGILE maintains a better balance between stability and plasticity and
achieves a much higher trade-off compared to other baselines.

Figure 6: Stability-Plasticity Trade-off for CL models trained on Seq-CIFAR100
with 5 tasks. AGILE maintains a better balance between stability and
plasticity and achieves the highest trade-off compared to other baselines.

###  E.3 Reservoir sampling

We maintain a fixed size buffer $\mathcal{B}$ following the reservoir sampling
strategy (Vitter, 1985). Reservoir sampling samples from a data stream of
unknown length by assigning equal probability to each sample to be represented
in the memory buffer. Replacements are performed at random once the buffer is
full. Algorithm 3 provides the steps to maintain the buffer.

Algorithm 3 Reservoir sampling

1:Input: Memory buffer $\mathcal{D}_{m}$, maximum buffer size $\mathcal{B}$,
number of seen samples $N$, current sample $x$, current label $y$

2:if $\mathcal{B}>N$ then

3: $\mathcal{D}_{m}[N]\leftarrow(x,y)$

4:else

5: $k=randomInteger(min=0,max=N)$

6: if $k<\mathcal{B}$ then

7: $\mathcal{D}_{m}[k]\leftarrow(x,y)$

8:return $\mathcal{D}_{m}$

###  E.4 Consistency regularization using EMA

The CL model’s predictions (soft-targets) capture the complex patterns and
rich similarity structures in the data. As CL training progresses, soft
targets (model predictions) carry more information per training sample than
hard targets (ground truths) (Hinton et al., 2015). Therefore, in addition to
ground truth labels, soft targets can be leveraged to better preserve the
knowledge of the previous tasks. Consistency regularization has traditionally
been used to enforce consistency in the predictions either by storing the past
predictions in the buffer or by employing an exponential moving average (EMA)
of the weights of the CL model. Following Arani et al. (2022), we employ an
EMA of the weights of the CL model to enforce consistency in the predictions
as follows:

$\mathcal{L}_{cr}\triangleq\displaystyle\mathop{\mathbb{E}}_{(x_{k},y_{k})\sim
D_{m}}\lVert\Phi_{\theta_{\scaleto{EMA}{3pt}}}(x_{k})-\Phi_{\theta}(x_{k})\rVert^{2}_{F}$
(12)

where $\|\cdot\|_{F}$ is the Frobenius norm,
$\Phi_{\theta_{\scaleto{EMA}{3pt}}}$ is the EMA of model $\Phi_{\theta}$. We
update the EMA model as follows:

$\theta_{\scaleto{EMA}{3pt}}=\begin{cases}\eta\;\theta_{\scaleto{EMA}{3pt}}+\left(1-\eta\right)\theta,&\text{if
}\>\zeta\geq\mathcal{U}(0,1)\\\
\theta_{\scaleto{EMA}{3pt}},&\text{otherwise}\end{cases}$ (13)

where $\eta$ is a decay parameter, $\zeta$ is an update rate, and $\theta$ and
$\theta_{\scaleto{EMA}{3pt}}$ are the weights of the CL model and its EMA. As
the knowledge of the previous tasks is encoded in the weights of the CL model,
we employ EMA for inference instead of the CL model as it serves as a proxy
for the self-ensemble of models specialized in different tasks.

###  E.5 Hyperparameters

Table 6: Selected hyperparameters for AGILE for all the scenarios reported in
Table 1

Dataset |  Buffer size | EMA params | Loss balancing params | Learning rate | $\delta_{i}$ dimension
---|---|---|---|---|---
$\zeta$ | $\eta$ | $\alpha$ | $\beta$ | $\gamma$ | $\lambda$
Seq-CIFAR10 | 200 | 0.2 | 0.999 | 1 | 0.15 | 1 | 0.1 | 0.07 | 256
500 | 0.2 | 0.999 | 1 | 0.10 | 1 | 0.1 | 0.05 | 256
Seq-CIFAR100 | 200 | 0.05 | 0.999 | 1 | 0.10 | 1 | 0.1 | 0.03 | 256
500 | 0.08 | 0.999 | 1 | 0.15 | 1 | 0.1 | 0.07 | 256
Seq-TinyImageNet | 200 | 0.05 | 0.999 | 1 | 0.10 | 1 | 0.1 | 0.05 | 256
500 | 0.05 | 0.999 | 1 | 0.10 | 1 | 0.5 | 0.05 | 256

We report the AGILE hyperparameters to reproduce the results reported in Table
1. These hyperparameters were found after tuning with multiple random
initializations. In addition to these hyperparameters, we use a standard batch
size of 32 and 50 epochs of training for all of our experiments. We use the
SGD optimizer and other tools available in PyTorch to build AGILE.

####  E.5.1 Hyperparameter Tuning

Table 7: Hyperparamter tuning for AGILE on Seq-CIFAR100 with buffer size 500. As can be seen, AGILE is fairly robust to choice of hyperparameters.  Varying $\beta$, for $\gamma=1.0$, $\lambda=0.1$ | Varying $\gamma$, for $\beta=0.15$, $\lambda=0.15$ | Varying $\lambda$, for $\beta=0.15$, $\gamma=1.0$
---|---|---
$\beta$ | Top-1 Acc % | $\gamma$ | Top-1 Acc % | $\lambda$ | Top-1 Acc %
0.1 | 52.27 | 0.1 | 51.78 | $\mathbf{0.1}$ | $\mathbf{52.65}$
$\mathbf{0.15}$ | $\mathbf{52.65}$ | 0.5 | 52.46 | 0.2 | 52.33
0.2 | 51.98 | $\mathbf{1.0}$ | $\mathbf{52.65}$ | 0.5 | 52.31
0.5 | 49.56 |  |  |  |