Continual Learning in Linear Classification on Separable Data

Itay Evrontech

Edward Moroshkotech

Gon Buzaglotech

Maroun Khrieshtech

Badea Marjiehtech

Nathan Srebrotyt

Daniel Soudrytech

techDepartment of Electrical and Computer Engineering, Technion, Haifa, Israel
tytToyota Technological Institute at Chicago, Chicago IL, USA

Itay<EMAIL_ADDRESS>
Continual learning, Linear classification, Catastrophic forgetting

We analyze continual learning on a sequence of separable linear classification tasks with binary labels.
We show theoretically that learning with weak regularization reduces to solving a sequential max-margin problem, corresponding to a special case of the Projection Onto Convex Sets (POCS) framework.
We then develop upper bounds on the forgetting
and other quantities of interest under various settings with recurring tasks, including cyclic and random orderings of tasks.
We discuss several practical implications to popular training practices like regularization scheduling and weighting.
We point out several theoretical differences between our continual classification setting and a recently studied continual regression setting.

Illustrating our setting from sec:setting and
the Sequential Max-Margin dynamics from proc:adaptive in sec:algorithmic_bias.

A $2$-dimensional task with two samples (one positive and one negative).

Each sample $(\x,y)$ induces a constraint to a halfspace $y\w^\top \x\ge1$.

The task's feasible set $\feasible_1$ is defined as the intersection of these two halfspaces,
and is thus an affine polyhedral cone.

Two tasks, the first having two samples and the second having a single sample.

The intersection of $\feasible_1$ and $\feasible_2$ defines the offline feasible set $\teachers$,
in which all samples of all tasks are correctly classified with a margin of at least $1$.

Notice the min-norm offline solution $\teacher\in\teachers$.

Learning the 1st task projects ${\w_0\!=\!\0_{\dim}}$
onto $\feasible_1$.

The obtained iterate
${\w_1=\mP_1(\w_0)}$ is the max-margin solution of the 1st task.

Then, learning
the 2nd task, projects $\w_1$ onto $\feasible_2$
to obtain (in this case)
an offline solution
\in
\teachers\subset\feasible_2}$.

$\w_2$ is not the min-norm  $\teacher$

§ INTRODUCTION

Continual learning deals with learning settings where distributions, or tasks, change over time,
breaking traditional i.i.d. assumptions.

While models trained sequentially are expected to accumulate knowledge and improve over time, practically they suffer from
catastrophic forgetting [McCloskey & Cohen, 1989, Goodfellow et al., 2013],
their performance on previously seen tasks deteriorates over time.

Much research in continual learning has focused on heuristic approaches to remedying forgetting.
Recent approaches achieve impressive empirical performance, but often require storing examples from previous tasks
(Robins, 1995, Rolnick et al., 2019), iteratively expanding the learned models
(Yoon et al., 2018),
or lessening the plasticity of these models and harming their performance on new tasks
(Kirkpatrick et al., 2017).

We theoretically study the continual learning of a linear classification model on separable data with binary classes.

Even though this is a fundamental setup to consider, there are still very few analytic results on it,
since most of the continual learning theory thus far has focused on regression settings
(Bennani et al., 2020, Doan et al., 2021, Asanuma et al., 2021, Lee et al., 2021, Evron et al., 2022, Goldfarb & Hand, 2023, Li et al., 2023).

Even in the broader deep learning scope, theoreticians often start from very simple models and use them to gain insight into phenomena arising in more practical models
(Belkin et al., 2018, Woodworth et al., 2020).

Our paper reveals a surprising algorithmic bias of linear classifiers trained continually to minimize the exponential loss on separable data with weak regularization.
Specifically, we prove that the weights converge in the same direction as the iterates of a Sequential Max-Margin scheme.

This creates a bridge between the popular regularization methods
for continual learning
[Kirkpatrick et al., 2017, Zenke et al., 2017]
and the well-studied POCS framework – Projections Onto Convex Sets (also known as the convex feasibility problem or successive projections).

Our results complement those of a recent paper
[Evron et al., 2022]
that analyzed the worst-case performance of continual linear regression.
That paper showed that continually learning linear regression tasks with vanilla SGD,
implicitly performs sequential projections onto closed subspaces,
and connected that regime to the area of Alternating Projections [Von Neumann, 1949, Halperin, 1962].

In our paper, we draw comparisons between our continual classification setting and their continual regression setting
(summarized in App. <ref>).
We point out inherent differences between these two settings, emphasizing the need for a proper and thorough analytical understanding dedicated to continual classification settings.

Our Contributions
Our analysis reveals the following:

* Explicit regularization methods for continual learning of linear classification models are linked to the framework of Projections Onto Convex Sets (POCS).

* Each learned task
brings the learner closer to an “offline” feasible solution solving
all tasks.

However, there exist task sequences for which the learner stays arbitrarily far from offline feasibility, even after infinitely many tasks.

* When tasks recur cyclically or randomly, the learner converges to an offline solution with linear rates.

* If we converge to an offline solution, it may not be the minimum-norm solution (in contrast to continual regression), but it still needs to be $2$-optimal (minimal).

* Scheduling the regularization strength endangers convergence to an offline solution and optimality guarantees.

* Using popular regularization weighting schemes based on Fisher-information matrices,
does not necessarily prevent forgetting
(in contrast to continual regression).

* Early stopping (without regularization) does not yield the same solutions as weak regularization (unlike in stationary settings with a single task).

§ SETTING

We consider $T\ge 2$ binary classification tasks.
Each task $m=1,\dots,T$
is defined by
a dataset $\dataset_m$ consisting of
of $\dim$-dimensional samples and their binary labels,
each tuple is
$(\x, y)
\!\in\!
{\,\reals^{\dim} \!\times\! \left\{-1,+1\right\}\,}$,
for a finite $\dim$.

Throughout the paper, we denote the (isotropic) Euclidean norm of vectors by $\norm{\vect{v}}$,
and the weighted norm by
\triangleq
\vv^\top \B \vv$,

for some $\B\succ\0_{\dim\times\dim}$.

We denote the set of natural numbers starting from $1$ by ${\naturals\triangleq \mathbb{N}\setminus\{0\}}$
and the natural numbers from $1$ to $n$ by $\cnt{n}$.

We define the distance of a vector $\w\in\reals^{\dim}$ from a closed set $\mathcal{C}\subseteq\reals^{\dim}$ as
\min_{\vv\in\mathcal{C}}\norm{\w-\vv}}$.

Finally, we denote the maximal norm of any data point by ${R\triangleq \max_{m\in\cnt{T}}\max_{(\x,y)\in\dataset_m }\norm{\x}}$.

Our main assumption in this paper is that the tasks are jointly-separable,
they can be perfectly fitted simultaneously, as in “offline” non-continual settings.
This can be formally stated as follows.

Each task $m\in\cnt{T}$ is separable,
it has a non-empty feasible set
defined as

\feasible_m
\,\!\triangleq\!\,
\left\{
\w\!\in\!\reals^{\dim}
\mid
y \w^\top \x\ge 1,
\forall (\x, y)\in \dataset_m \right\}\,.

Moreover, the $T$ tasks are jointly-separable —
there exists a non-empty offline
feasible set:

\,\!\triangleq\!\,
\feasible_1
\cap \dots \cap
\feasible_T
\neq\emptyset
\,.$$

A similar assumption was made in the continual regression setting [Evron et al., 2022].
It is a reasonable assumption in overparameterized regimes,
where feasible offline solutions often do exist.
Practically, this is commonly the case in modern deep networks.
Theoretically, with sufficient overparameterization [Du et al., 2019] or high enough margin [Ji & Telgarsky, 2020],
it is often easy to converge to a zero-loss solution.

To facilitate our results and discussions,
we specifically define the minimum-norm offline solution.
This solution is traditionally linked to good generalization.

We denote the offline solution with the minimal norm by

\begin{align*}
\begin{aligned}
\teacher
\triangleq
\norm{\w}\,.
\end{aligned}
\end{align*}

Figures <ref> and <ref> illustrate our definitions.

Notice how both the feasible set $\feasible_m$ of each task and the offline feasible set $\teachers$ are closed, convex, and affine polyhedral cones.

§ ALGORITHMIC BIAS IN REGULARIZATION METHODS

Regularization methods are highly influential in continual learning
[Kirkpatrick et al., 2017, Zenke et al., 2017, Aljundi et al., 2018].

In this section, we propose a novel analysis for such methods on separable datasets in the spirit of theoretical work on algorithmic biases outside the scope of continual learning.

Concretely, we discover that
weakly-regularized models, trained sequentially to minimize the exponential loss,
It should be possible to extend our results to other losses with exponential tails, cross-entropy,
as has been done
in previous theoretical works on stationary settings
[Soudry et al., 2018].
converge in direction to the iterates of a sequential projection scheme.

Specifically, we study proc:regularized, in which the learner sequentially sees one task
(out of $T$)
at a time,
for $k$ iterations
($k\!>\!T$ implies repetitions).

Starting from $\w_0^{(\lambda)}\!\!=\!\!\0_{\dim}$,
at each iteration $t\!\in\!\cnt{k}$,
the learner minimizes
the exponential loss of the current task's dataset $\dataset_{t}$,
while biasing towards the previous task's solution $\w_{t-1}^{(\lambda)}$
using the Euclidean norm.

The regularization strength is determined by a sequence of (possibly constant) positive scalars $\lambda_1,\smalldots,\lambda_{k}\!>\!0$.

The norms are possibly weighted by a sequence of positive-definite matrices $\B_1,\smalldots,\B_{k}\succ\0_{\dim\times\dim}$.

Regularized Continual Learning

$\w_{0}^{(\lambda)} = \0_{\dim}$
Iterative update for each task $t\in[k] %\ge1

\begin{align}
\algmargin
\label{eq:weakly_regularized_problem}
\w_{t}^{(\lambda)}\!
\argmin_{\w \in \reals^{\dim}} \!\!
\sum_{(\x,y)\in \dataset_{t} }
\!\!\!
e^{-y \w^\top \x }
\frac{\lambda_t}{2}
\norm{\w\!-\!\w_{t-1}^{(\lambda)}}^2_{\B_{t}}
\end{align}

To clarify, each of the $k$ iterations
corresponds to learning a whole task (to convergence), and not to performing a single gradient step.

As a start, we first focus on
constant strengths $\lambda\!>\!0$
and on
“vanilla” L$2$ regularization, $\B_1 = \smalldots = \B_{k} = \I$.

Vanilla L$2$ regularization has recently been shown to be competitive
with more popular norm-weighting schemes like EWC
[Lubana et al., 2022, Smith et al., 2023].

We address the more complicated setups in sec:extensions.

For an arbitrary $\lambda$,
the objective in (<ref>)
is hard to analyze,
even for the first task, where the regularization term reduces to the traditional unbiased term $\norm{\w}^2$.

Previous works were still able to perform non-trivial analysis
by examining the weakly-regularized case,
in the limit of $\lambda\to 0$
(for linear models [Rosset et al., 2004]
or homogeneous neural networks [Wei et al., 2019]).

Under our continual setting,
we also take this approach and analyze
the weakly-regularized model.

This enables us to gain analytical insights into regularization methods for continual learning.
We establish an equivalence between the weakly-regularized proc:regularized and the following Sequential Max-Margin proc:adaptive,
and propose novel perspectives and techniques for analyzing regularization methods.

Sequential Max-Margin

$\w_0 = \0_{\dim}$
Iterative update for each task $t\in[k]

\begin{align}
\algmargin
\label{eq:adaptive_dynamics}
\w_{t}
\mP_{t}\prn{\w_{t-1}}
\triangleq
\argmin_{\w \in \reals^{\dim}}
\enskip &
\!
\norm{\w-{\w}_{t-1}}^2
\\
\suchthat
\enskip &
\,
y\w^\top \x \!\ge\! 1,
\,
\forall (\x,y)\!\in\! \dataset_{t}
\nonumber
\end{align}

We are now ready to state our fundamental result,
showing that
the regularized continual iterates $\tprn{\w_t^{(\lambda)}}$  of proc:regularized
converge in direction to the Sequential Max-Margin
$\prn{\w_t}=\tprn{\mP_{t}\prn{\w_{t-1}}}$ of proc:adaptive,
obtained by successive projections onto closed convex sets, as depicted in fig:feasibility_iterates.

Let $\lambda_t\!=\!\lambda\!>\!0$,
$\B_{t}\!=\!\I$, $\forall t\!\in\!\cnt{k}$.

for almost all separable datasets,
This holds w.p. $1$ for
separable datasets (Assumption <ref>) sampled from any absolutely continuous distribution.
It also holds even when the datasets are separable
but not jointly-separable.

in the limit of $\lambda\!\to\!0$,
it holds that
\to
with a residual of

\!-
As a result,
at any iteration
$t=o\prn{\frac{\ln\prn{\nicefrac{1}{\lambda}}}{\ln \ln\prn{\nicefrac{1}{\lambda}}}}$,
we get

\lim_{\lambda\to0}
\frac{\w_t^{(\lambda)}}{\tnorm{\w_t^{(\lambda)}}}
\frac{\w_t}{\tnorm{\w_t}}
\,.

In app:algorithmic_bias, we prove this theorem.
There, we also discuss the limitations of our analysis (sec:limitations) and identify aspects where it can be improved.

Existing works (Rosset et al., 2004) analyzed weakly-regularized models with an unbiased regularizer $\norm{\w}_{p}^{p}$.

This allowed them to concentrate on the limit margin which implies convergence in direction.

However, we show that to analyze the continual regularizer in (<ref>),
one must also take into account the scale of the solutions $(\w_{t}^{(\lambda)})$.

Therefore, we analyze both the scale and the direction of weakly-regularized solutions,
requiring more refined techniques.

§ SEQUENTIAL MAX-MARGIN PROJECTIONS

Now, we turn to exploit the connection we have established between the weakly-regularized continual
Scheme <ref>
and the Sequential Max-Margin Scheme <ref>.

Using tools from existing literature on Projections Onto Convex Sets (POCS), we gain valuable insights into
the dynamics of training
models continually.

Specifically, we derive optimality guarantees and convergence bounds in several interesting settings.

§.§ Quantities of Interest

We have three quantities of interest.

While the first two are widely used in the POCS literature,
the latter is specific to our continual classification setting.

Let $\w_{t}\in\reals^{\dim}$ be the $t$iterate,
obtained while continually learning $T$ jointly-separable tasks.

The following quantities are of interest:

* Distance to the offline feasible set:
$\dist\!\prn{\w_{t}, \teachers}$

* Maximum dist. to any feasible set:
\m@th\displaystyle
\max_{m\in\cnt{T}}
\!%\!\:
\dist\!\prn{\w_{t},\feasible_m}

* Forgetting:
We define the forgetting of a previously-seen task ${m\in\cnt{t}}$ as the maximal squared hinge loss on any sample $(\x,y)$ of the task.
More formally,

\triangleq
\!
\left(
\max\left\{0,\,1-y\w_{t}^\top \x\right\}
\right)^2\,.

Throughout our paper,
analyze both the maximal and the average forgetting,

Explaining our forgetting.

Previous works on continual linear regression defined forgetting using the MSE on previously-seen tasks,
\forall m<t\!:~
\tfrac{1}{\abs{\dataset_{m}}}
\sum_{(\x,y)\in\dataset_{m}}
\prn{\w_{t}^{\top}\x-y}^2
(Doan et al., 2021, Evron et al., 2022).
Then, lower forgetting implies better training loss on previous tasks.

In continual linear classification,
hinge losses capture similar properties.

First, per our definitions,
immediately after learning the $m$task,
the forgetting on it is $0$,
since $\forall \vv\!\in\!\reals^{\dim}\!:
\mP_m(\vv)\!\in\!\feasible_m$

a lower forgetting implies
better training margins and generalization performance on previous tasks.

Since the squared hinge loss is a surrogate for the 0-1 loss, $F_m \!\prn{\w_{t}}\!<\!1$
implies no prediction errors on the $m$task.

Forgetting is different from the regret used to analyze online learning algorithms [Crammer et al., 2006, Shalev-Shwartz et al., 2012, Hoi et al., 2021].

Forgetting quantifies the degradation on previous tasks in hindsight,
while regret cumulatively captures the ability to predict future datapoints (or tasks).

A favorable property of the quantities we defined, is that they bound each other.

Recall our definition of
\max_{m\in\cnt{T}}
\max_{(\x,y)\in\dataset_m }
\!\!\norm{\x}}$.

The quantities of def:quantities are related
as follows,
\,m\!\in\!\cnt{T}$:

\begin{align*}
\le
\dist^2(\w, \feasible_m)
\!\!
\max_{(\x,y)\in\dataset_m}
\!\!\!\!
\tnorm{\x}^2
\le
\dist^2(\w, \teachers) R^2.
\end{align*}

Moreover, for the Sequential Max-Margin iterates
$(\w_t)$ of Scheme <ref>,
all quantities are upper bounded by the
“problem complexity”,
$\dist^2(\w_t, \teachers) R^2\le\norm{\teacher}^2\!R^2$.

The proof is given in app:successive-projections.

Many of our bounds use $\norm{\teacher}^2\!R^2$,
which can be seen as the problem complexity due to its links to sample complexities [Novikoff, 1962].

Assumption <ref>,
implies that

The following is a known useful result from the POCS literature
(Lemma 3 in Gubin et al., 1967).

Distances from the Sequential Max-Margin iterates
of proc:adaptive
to the offline feasible set
are non-increasing,

\dist(\w_{t}, \teachers) \le \dist(\w_{t-1}, \teachers),
~\,\forall t\in\cnt{k}%\naturals
\,.

An immediate question arises from lem:feasible_distance_monotonicity:
when learning infinite jointly-separable tasks
must we converge to the offline feasible set $\teachers$?

Next, we answer this question in the negative.

§.§ Adversarial Construction:
Maximal Forgetting

[Adversarial construction]

We present a construction of task sequences that seemingly exhibit arbitrarily bad continual performance.
Even after seeing $k=T\to\infty$ jointly-separable tasks, the learner stays afar from the offline feasible set $\teachers$, and the forgetting of previously-seen tasks is maximal.
The learner fails to successfully accumulate experience.
See further details in

Our construction with $T\!=\!20$ tasks in $\dim\!=\!3$.

Datapoints (each defining a task) have a norm of $R\!=\!1$
and are spread uniformly on a plane, slightly elevated above the $xy$-plane.
It holds that $\norm{\teacher}\!=\!10$.
As $T\!\to\!\infty$,
angles between consecutive tasks
and applied projections, get smaller.

The average and maximum forgetting for the adversarial construction
for an increasing number of tasks $T$.

Recall that the maximum forgetting lower bounds the distance to the offline feasible set (lem:euclidean_to_hinge).

Notably, after learning $T\to \infty$ jointly separable tasks,
the quantities do not decay but seemingly become arbitrarily bad (close to $\norm{\teacher}^2\!R^2$) at some point of learning.

Illustrations of our adversarial construction.

Our paper analyzes the continual learning of $T$ tasks for $k$ iterations,
possibly taking ${k\!\to\!\infty}$
(by repeating tasks).

We take $\lambda\!\to\!0$ after fixing the number of iterations $k$.

As a result, limit iterates hold

\frac{\wlim}{\tnorm{\wlim}}
\triangleq
\lim_{k\to \infty} \frac{\w_k}{\tnorm{\w_k}}
\lim_{k\to \infty} \lim_{\lambda\to 0}
\frac{\w_k^{(\lambda)}}{\tnorm{\w_k^{(\lambda)}}}
\,.

§.§ Convergence to the Minimum-Norm Solution

When solving feasibility problems for classification
(in hard-margin SVM), the minimum-norm solution is also the max-margin solution.
In turn, max-margin solutions are theoretically linked to better generalization performance.

In realizable continual linear regression
(or generally, in alternating projections onto closed subspaces),
it is known that if iterates converge to an offline (or globally feasible) solution,
then that solution must
be the closest to $\w_0$,
have a minimum norm
[Evron et al., 2022, Halperin, 1962].

In contrast, in separable continual linear classification settings like ours
(or generally, in projections onto closed convex sets),
there is no such guarantee
and we can converge to a suboptimal offline solution,
as depicted in fig:feasibility_iterates.

Nevertheless, the following optimality guarantee does hold.

Any iterate $\w_{t}$ obtained by proc:adaptive
$\norm{\w_{t}}\le 2\norm{\teacher}$,
where $\teacher$ is the minimum-norm offline solution

If additionally, $\w_{t}$ is “offline”-feasible,

$\w_{t} \in \teachers$,

\norm{\teacher} \le \norm{\w_{t}} \le 2\norm{\teacher}\,.

The proof is given in app:min_norm.

Further comparisons to continual regression are drawn in app:comparison.

§.§ Recurring Tasks

Naturally, in many practical continual problems,
certain concepts and experiences recur at different tasks
(environments of an autonomous vehicle, levels of a computer game, trends of a search engine, etc.).

Several recent papers have observed empirically that  task repetitions mitigate catastrophic forgetting in continual learning
[Stojanov et al., 2019, Cossu et al., 2022],
even when training is performed with vanilla SGD, without any forgetting-preventing method
[Lesort et al., 2022].

In this section, we analytically study the influence of repetitions.
To accomplish this, we leverage the connection that we have established between continual learning and successive projection algorithms for convex feasibility problems.

Importantly, we do not propose repetitions as a training method but rather aim to understand their effects on continual learning from a projection perspective.

The results of this section are summarized in Table <ref>.

Summary of our upper bounds for recurring orderings (sec:repetitions).

Upper bounds of random orderings apply to the expectations.

Iterate type

\tfrac{1}{\tnorm{\teacher}^2 R^{2}}
F_m (\w_k)

\tfrac{1}{\tnorm{\teacher}^2}{\max}_{m\in\cnt{T}} \dist^2\prn{\w_k, \feasible_m}$

\tfrac{1}{\tnorm{\teacher}^2} \dist^2\prn{\w_k, \teachers}$

Last ($T=2$)

\min\bigg\{
\frac{1}{k+1},\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,
\exp
\!\Bigprn{-\frac{k}{4\tnorm{\teacher}^2 R^2}}
\Big\}

\exp
\!\Bigprn{-\frac{k}{4\tnorm{\teacher}^2 R^2}}

Last ($T\ge3$)

\min\bigg\{\,
\frac{2T^2}{\sqrt{k}},
\,\,\,4
\exp
\!\Bigprn{-\frac{k}{16T^2\tnorm{\teacher}^2 R^2}}
\Big\}

\exp
\!\Bigprn{-\frac{k}{16T^2\tnorm{\teacher}^2 R^2}}

Average ($T\ge2$)

$\hfrac{T^2}{k} $

\exp
\!\Bigprn{-\frac{k}{4T\tnorm{\teacher}^2 R^2}}

\exp
\!\Bigprn{-\frac{k}{4T\tnorm{\teacher}^2 R^2}}

$\hfrac{T}{k} $

Recurring tasks vs. Batch learning
The recurring tasks setting should not be confused with standard batch training
(by regarding each task as a batch).
While in batch training, a single gradient-descent step is made for each batch,
in our continual learning setting
each task is solved completely (to separation).

The following is a key lemma in our paper.

Much of the research on POCS has focused on defining and applying linear regularity conditions, under which iterates $(\w_t)$ converge linearly
(like $c^{t}$ for some $c\!\in\!\left[0,1\right)$)
to the feasible sets' intersection $\teachers$
(Bauschke & Borwein, 1993).

In realizable continual linear regression, Evron et al., 2022 showed that no such regularity holds, and while their forgetting,

${F_m(\w_t)\triangleq \dist^2 (\w_t, \feasible_m),~\forall m\!\in\!\cnt{T}}$,
is upper bounded universally,
no universal bounds can be derived for
$\dist^2 (\w_t, \teachers)$
even when $\norm{\teacher}$ and $R$ are bounded.

In contrast, we prove that in separable continual linear classification,
linear regularity does hold
and nontrivial bounds can be derived for $\dist^2 (\w_t, \teachers)$.

At the $t$iteration, the distance to the offline feasible set is tied to the distance to the farthest feasible set of any specific task.
Specifically, it holds that $\forall t\!\in\!\naturals$,

\begin{align*}
\dist^2\!\prn{\w_{t}, \teachers}
\le
4\,\tnorm{\teacher}^2 R^2
\max_{m\in\cnt{T}} \dist^2
\bigprn{\w_{t}, \feasible_m}
\,
\end{align*}

The proofs for this section are given in app:repetitions_proofs.

So far we assumed that at iteration $t$,
the learner solves a task, a dataset $\dataset_{t}$ (out of $T$ possible datasets),
corresponding to a projection $\mP_{t}$.

To facilitate our next results for cases where tasks recur, we now define task ordering functions.

A task ordering is a function

that maps an iteration to a learning task.

§.§.§ Cyclic Ordering

when analyzing successive projections (like in our proc:adaptive),
it is common to first study a cyclic setting, where the projections form a clearer (cyclic) operator.

This setting has been at the center of focus in many theoretical papers (Agmon, 1954, Halperin, 1962, Deutsch & Hundal, 2006, Borwein et al., 2014, Evron et al., 2022).

Practically, in continual learning, cyclic orderings are indeed less flexible.

However, we believe that they do emerge naturally in real-world scenarios.

For instance, virtual assistants support the recurring daily routines of their customers
and should be able to continue learning from new experiences.

A cyclic ordering over $T$ tasks is defined as

\triangleq
1+\prn{\prn{t-1}\bmod T},
\enskip~
\forall t\in\naturals
\,,

and induces a cyclic operator,
in the sense that

\prn{\mP_{\tau(nT)}\circ\smallcdots\circ\mP_{\tau(1)}}
\!(\w_0)

We illustrate such orderings in the following fig:cyclic.

A cyclic setting with two tasks.

Each time we solve a task, we project the previous iterate onto its corresponding convex set.

Using the linear regularity from lem:regularity,
we prove linear convergence results for cyclic orderings.
Our proofs use tools and results from existing POCS literature.

Under a cyclic ordering
(and the separability assumption <ref>),
the iterates converge to a 2-optimal
That is,

\lim_{k\to \infty}
\dist(\w_k, \teachers)=0,
\norm{\teacher} \le \norm{\w_\infty} \le 2\norm{\teacher}
\,.

The proofs for this section are given in app:cyclic_proofs.

For ${T\ge2}$
jointly-separable tasks
learned cyclically,

${k\!=\!nT}$ iterations ($n$ cycles),
our quantities of interest
converge linearly as

\begin{align*}
\underbrace{
\max_{m\in\cnt{T}}
F_m (\w_k)}_{
\substack{
\text{Maximum forgetting}
\underbrace{
\max_{m\in\cnt{T}}
\dist^{2\!}\prn{\w_k, \feasible_{m}}
\substack{
\text{Max.~dist.~to any feasible set}
\le
\\
\underbrace{
\dist^{2\!}\prn{\w_k, \teachers}
\substack{
\text{Dist.~to}
\\
\text{offline feasible set}
\le
\norm{\teacher}^2 \!R^2
\,,
\end{align*}

\triangleq
\begin{cases}
\enskip\,
\exp
\!\prn{-\tfrac{k}{4\tnorm{\teacher}^2 R^2}} & T=2
\\
\exp\!\prn{-\tfrac{k}{16T^2\tnorm{\teacher}^2 R^2}}
& T\ge 3
\end{cases}

$\norm{\teacher}^2 \!R^2\!\ge\! 1$ is the problem complexity (Rem. <ref>).

Detour: Universal rates for general cyclic Projections onto Convex Sets (POCS) settings.

We take a brief detour to derive universal bounds for general POCS settings, where problems do not necessarily hold regularity conditions like the ones in our lem:regularity.

While cyclic POCS settings have been studied for decades
[Agmon, 1954, Deutsch & Hundal, 2006, Deutsch & Hundal, 2006, Deutsch & Hundal, 2008],
most previous works either focused on settings with regularity assumptions or yielded problem-dependent rates that can be arbitrarily bad.

Our result below extends the universal results of a recent work that focused on closed subspaces only
[Evron et al., 2022]; and is also related to a recent work that considered closed convex sets  as well,
but dismissed the number of sets $T$ as $\bigO\prn{1}$
[Reich & Zalas, 2023].

Let $\feasible_1, \dots, \feasible_{T}$ be closed convex sets with nonempty intersection $\teachers$.
be the iterate after $k\!=\!nT$ iterations ($n$ cycles)
of cyclic projections onto these convex sets.
Then, the maximal distance to any (specific) convex set,
is upper bounded universally as,

\begin{align*}
\text{For $T=2$: }~~
\max_{m\in\cnt{T}}
\dist^{2\!}\prn{\w_k, \feasible_{m}}
\le
\frac{1}{k+1}\,
\dist^{2\!}\prn{\w_{0}, \teachers}
\,,
\\
\text{For $T\ge3$: }~~
\max_{m\in\cnt{T}}
\dist^{2\!}\prn{\w_k, \feasible_{m}}
\le
\frac{2T^2}{\sqrt{k}}\,
\dist^{2\!}\prn{\w_{0}, \teachers}
\,.
\end{align*}

Clearly, the above yields
universal rates for the forgetting as well
(see lem:euclidean_to_hinge),
but these are essentially worse than the linear rates we got (in the presence of regularity).

§.§.§ Random Ordering

In this section, we consider a uniform i.i.d. task ordering
over a set of $T$ tasks.

Random orderings are considered more realistic than cyclic ones
(driverless taxis will likely encounter recurring routes/environments randomly, rather than in a certain cycle).

Mathematically, they also require other analytical tools and often offer different convergence guarantees.

Much like cyclic orderings, random orderings have been studied in many related areas
(Nedić, 2010, Needell & Tropp, 2014, Evron et al., 2022).

A random (uniform) ordering over $T$ tasks is defined as

\!=\!
\Pr\prn{\tau(t')\!=\!m'}\!,
\enskip
\forall t,t'\!\in\!\naturals\!,\,m,m'\!\in\!\cnt{T}
\!.

For $T$ jointly-separable tasks
learned in a random ordering,
our quantities of interest
converge linearly as

F_m (_k)_

Maximum forgetting

^2_k, _m

Max. dist. to any feasible set

Dist. to

offline feasible set

-k4T^2 R^2

^2 R^2

where $\norm{\teacher}^2 \!R^2\!\ge\! 1$ is the problem complexity (Rem. <ref>).

To show this theorem, we prove a property of our convex sets
(polyhedral cones; see fig:one_task_feasibility)
and apply
a result from the (random) POCS literature [Nedić, 2010].

The proofs for this section are given in app:random_proofs.

Under a random ordering
(and assumption <ref>),
the iterates converge almost surely to ${\teachers}$,
such that

\lim_{k\to \infty}
\dist(\w_k, \teachers)=0,
\norm{\teacher} \le \norm{\w_\infty} \le 2\norm{\teacher}
\,.

The result from Nedić, 2010   that we used to prove thm:random_rates,
is applicable to any  distribution
with a nonzero
\triangleq
\min_{m\in\cnt{T}}
\Pr\prn{\tau\prn{{\cdot}}\!=\!m}>0$.
In such cases, the rate from thm:random_rates changes to

\prn{
\!
-\tfrac{p_{\min} k}{4\tnorm{\teacher}^2 R^2}
\!

§ RELATED WORK

Many papers have investigated various aspects of catastrophic forgetting, including when it occurs [Evron et al., 2022], strategies to avoid it [Peng et al., 2023], the influence of task similarity [Lee et al., 2021], its impact on transferability [Chen et al., 2023], and other related factors.

A sound understanding of forgetting can potentially  advance the continual learning field significantly.

Throughout our paper, we discussed many connections to other works from many fields.

Interestingly, our Sequential Max-Margin (SMM) scheme can be seen as a hard variant of the Adaptive SVM algorithm [Yang et al., 2007],
which was previously used to practically tackle domain adaptation and transfer learning.
(in Li, 2007, Pentina et al., 2015, Tang et al., 2022).

To the best of our knowledge, our paper is the first to highlight the connection between regularization methods for continual learning and Adaptive SVM.

For the special case where each task has one datapoint, the SMM scheme corresponds to the online Passive-Aggressive algorithm [Crammer et al., 2006].

If additionally, we consider only two tasks learned in a cyclic ordering, the regret after training task $t$ is related (but not identical) to the forgetting
(see Remark <ref>).

Consequently, our forgetting bound in prop:cyclic_universal matches the regret bound in their Theorem 2.

§ DISCUSSION AND FUTURE WORK

§.§ Early Stopping

When training a single task, it is well known that explicit regularization is related to early stopping.
In linear classification with an exponential loss, Rosset et al., 2004 showed that the regularization path converges
(in the $\lambda\rightarrow 0$ limit)
to the hard-margin SVM, which is also the limit of the optimization path [Soudry et al., 2018].
In regression, Ali et al., 2019 connected early stopping to ridge regression.

Contrastingly, when learning a sequence of tasks, explicit regularization and early stopping can behave differently.
We first discuss a case where both methods lead to SMM.
More generally, we demonstrate that solutions may differ.

Consider a task sequence with
$1$ sample $(\x_t, y_t)$ per task.[Equivalently, $2$ samples $(\x_t, y_t)$ and $(-\x_t, -y_t)$.

We minimize the exponential loss of
each task $t$ with gradient flow (GF) until
$L_t(\w)\!\triangleq\!\exp(-y_t\x_t^{\top}\w)\!=\!\epsilon$ for
fixed $\epsilon\!<\!1$,
yielding a predictor $\w_t^{(\epsilon)}$.

Denote the
“normalized” predictor by ${\hat{\w}_t^{(\epsilon)}\!=\!{\w_t^{(\epsilon)}}/
\ln(\hfrac{1}{\epsilon})}$.

Since GF stays in the data span, we have
${\w_t^{(\epsilon)}\!=\!\w_{t-1}^{(\epsilon)}\!+\!\alpha_t y_t\x_t}$
for some $\alpha_t>0$.

The early stopping implies
and thus
holds the KKT conditions of the SMM,
\argmin_{\w}\!\bignorm{\w\!-\!\hat{\w}_{t-1}^{(\epsilon)}}^2 \,\suchthat\,\,y_t\x_t^{\top}\w\geq1}$
(where ${\hat{\alpha}_t \triangleq \prn{\alpha_t \,/ \ln(\hfrac{1}{\epsilon})} > 0}$ is the dual variable).[
If iteration $t$ begins and
we already have
both ES and SMM will not change their solutions.

In the
general case
(more than one sample per task),
early stopping might not agree with SMM, as depicted below.

Comparing the weakly-regularized (WR) and early stopping (ES) schemes.
We train $2$ tasks:
(where all points are labeled $+1$).
We plot the angle of different predictors ($\angle(\w,\left[\substack{1\\0}]\right]) $) while training on the 2task.
For instance, the Sequential Max-Margin (SMM) solution for that task is $163.1^{\circ}$.

For ES, we train both tasks with normalized GD (NGD) until their loss is ${\epsilon=e^{-200}}$.
Importantly, ES stops when the loss of the 2task, shown in the secondary y-axis,
is $e^{-200}$ and the angle is $166.8^{\circ}$.

For WR, we set ${\lambda\!=\!e^{-200}}$
and solve
(<ref>) by running NGD to convergence.

For the 2task, we initialize the NGD by $\w_1^{(\lambda)}$.
Unlike ES, the WR solution does converge to $163.1^{\circ}$ like SMM, as guaranteed by thm:weakly_regularized.
Further details in App. <ref>.

§.§ Future Work

There are several interesting avenues for future work, extending our results
to non-separable data (perhaps in the spirit of Yang et al., 2007),
multiclass classification with cross-entropy loss
(similarly to Appendix 4.1 in Soudry et al., 2018),
or non-linear models.

One can also try to derive forgetting bounds for weighted regularization schemes and look for optimal weighting matrices.

Another challenging but rewarding avenue is to extend our analysis to finite regularization strengths $(\lambda_t)$.

Finally, it is interesting to understand the exact algorithmic bias for early stopping in continual learning and its relation to explicit regularization.

§ ACKNOWLEDGEMENTS

We thank Lior Alon (MIT) for the fruitful discussions.

The research of DS was Funded by the European Union (ERC, A-B-C-Deep, 101039436). Views and opinions expressed are however those of the author only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency (ERCEA). Neither the European Union nor the granting authority can be held responsible for them. DS also acknowledges the support of Schmidt Career Advancement Chair in AI.

NS was partially supported by the Simons Foundation and NSF-IIS/CCF awards.

[Agmon, 1954]
Agmon, S.
The relaxation method for linear inequalities.
Canadian Journal of Mathematics, 6:0 382–392, 1954.

[Ali et al., 2019]
Ali, A., Kolter, J. Z., and Tibshirani, R. J.
A continuous-time view of early stopping for least squares
In AISTATS, volume 89 of Proceedings of Machine
Learning Research, pp.  1370–1378. PMLR, 2019.

[Aljundi et al., 2018]
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., and Tuytelaars, T.
Memory aware synapses: Learning what (not) to forget.
In Proceedings of the European Conference on Computer Vision
(ECCV), pp.  139–154, 2018.

[Asanuma et al., 2021]
Asanuma, H., Takagi, S., Nagano, Y., Yoshida, Y., Igarashi, Y., and Okada, M.
Statistical mechanical analysis of catastrophic forgetting in
continual learning with teacher and student networks.
Journal of the Physical Society of Japan, 900
(10):0 104001, Oct 2021.

[Bauschke, 2001]
Bauschke, H. H.
Projection algorithms: results and open problems.
In Studies in Computational Mathematics, volume 8, pp.  11–22. Elsevier, 2001.

[Bauschke & Borwein, 1993]
Bauschke, H. H. and Borwein, J. M.
On the convergence of von neumann's alternating projection algorithm
for two sets.
Set-Valued Analysis, 10 (2):0 185–212, 1993.

[Bauschke et al., 2011]
Bauschke, H. H., Combettes, P. L., et al.
Convex analysis and monotone operator theory in Hilbert
spaces, volume 408.
Springer, 2011.

[Belkin et al., 2018]
Belkin, M., Ma, S., and Mandal, S.
To understand deep learning we need to understand kernel learning.
In International Conference on Machine Learning, pp.  541–549. PMLR, 2018.

[Bennani et al., 2020]
Bennani, M. A., Doan, T., and Sugiyama, M.
Generalisation guarantees for continual learning with orthogonal
gradient descent.
arXiv preprint arXiv:2006.11942, 2020.

[Benzing, 2022]
Benzing, F.
Unifying regularisation methods for continual learning.
In International Conference on Artificial Intelligence and
Statistics. PMLR, 2022.

[Borwein et al., 2014]
Borwein, J. M., Li, G., and Yao, L.
Analysis of the convergence rate for the cyclic projection algorithm
applied to basic semialgebraic convex sets.
SIAM Journal on Optimization, 240 (1):0
498–527, 2014.

[Boyd & Vandenberghe, 2004]
Boyd, S. and Vandenberghe, L.
Convex optimization.
Cambridge university press, 2004.

[Chen et al., 2023]
Chen, J., Nguyen, T., Gorur, D., and Chaudhry, A.
Is forgetting less a good inductive bias for forward transfer?
In The Eleventh International Conference on Learning
Representations, 2023.

[Cossu et al., 2022]
Cossu, A., Graffieti, G., Pellegrini, L., Maltoni, D., Bacciu, D., Carta, A.,
and Lomonaco, V.
Is class-incremental enough for continual learning?
Frontiers in Artificial Intelligence, 5, 2022.

[Crammer et al., 2006]
Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., and Singer, Y.
Online passive-aggressive algorithms.
Journal of Machine Learning Research, 70
(19):0 551–585, 2006.

[Deutsch & Hundal, 2006]
Deutsch, F. and Hundal, H.
The rate of convergence for the cyclic projections algorithm i:
Angles between convex sets.
Journal of Approximation Theory, 1420 (1):0
36–55, 2006a.

[Deutsch & Hundal, 2006]
Deutsch, F. and Hundal, H.
The rate of convergence for the cyclic projections algorithm ii:
norms of nonlinear operators.
Journal of Approximation Theory, 1420 (1):0
56–82, 2006b.

[Deutsch & Hundal, 2008]
Deutsch, F. and Hundal, H.
The rate of convergence for the cyclic projections algorithm iii:
Regularity of convex sets.
Journal of Approximation Theory, 1550 (2):0
155–184, 2008.

[Doan et al., 2021]
Doan, T., Abbana Bennani, M., Mazoure, B., Rabusseau, G., and Alquier, P.
A theoretical analysis of catastrophic forgetting through the ntk
overlap matrix.
In Proceedings of The 24th International Conference on
Artificial Intelligence and Statistics, pp.  1072–1080, 2021.

[Du et al., 2019]
Du, S., Lee, J., Li, H., Wang, L., and Zhai, X.
Gradient descent finds global minima of deep neural networks.
In International conference on machine learning, pp.  1675–1685. PMLR, 2019.

[Evron et al., 2022]
Evron, I., Moroshko, E., Ward, R., Srebro, N., and Soudry, D.
How catastrophic can catastrophic forgetting be in linear regression?
In Conference on Learning Theory (COLT), pp.  4028–4079.
PMLR, 2022.

[Goldfarb & Hand, 2023]
Goldfarb, D. and Hand, P.
Analysis of catastrophic forgetting for random orthogonal
transformation tasks in the overparameterized regime.
In International Conference on Artificial Intelligence and
Statistics, pp.  2975–2993. PMLR, 2023.

[Goodfellow et al., 2013]
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., and Bengio, Y.
An empirical investigation of catastrophic forgetting in
gradient-based neural networks.
arXiv preprint arXiv:1312.6211, 2013.

[Gubin et al., 1967]
Gubin, L., Polyak, B. T., and Raik, E.
The method of projections for finding the common point of convex
USSR Computational Mathematics and Mathematical Physics,
70 (6):0 1–24, 1967.

[Halperin, 1962]
Halperin, I.
The product of projection operators.
Acta Sci. Math.(Szeged), 230 (1):0 96–99,

[Hoi et al., 2021]
Hoi, S. C., Sahoo, D., Lu, J., and Zhao, P.
Online learning: A comprehensive survey.
Neurocomputing, 459:0 249–289, 2021.
ISSN 0925-2312.

[Ji & Telgarsky, 2020]
Ji, Z. and Telgarsky, M.
Polylogarithmic width suffices for gradient descent to achieve
arbitrarily small test error with shallow relu networks.
In International Conference on Learning Representations, 2020.

[Kirkpatrick et al., 2017]
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu,
A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.
Overcoming catastrophic forgetting in neural networks.
Proceedings of the national academy of sciences, 1140
(13):0 3521–3526, 2017.

[Lee et al., 2021]
Lee, S., Goldt, S., and Saxe, A.
Continual learning in the teacher-student setup: Impact of task
In International Conference on Machine Learning, pp.  6109–6119. PMLR, 2021.

[Lesort et al., 2022]
Lesort, T., Ostapenko, O., Misra, D., Arefin, M. R., Rodríguez, P.,
Charlin, L., and Rish, I.
Scaling the number of tasks in continual learning.
arXiv preprint arXiv:2207.04543, 2022.

[Lewkowycz & Gur-Ari, 2020]
Lewkowycz, A. and Gur-Ari, G.
On the training dynamics of deep networks with $ l\_2 $
Advances in Neural Information Processing Systems,
33:0 4790–4799, 2020.

[Li et al., 2023]
Li, H., Wu, J., and Braverman, V.
Fixed design analysis of regularization-based continual learning.
arXiv preprint arXiv:2303.10263, 2023.

[Li, 2007]
Li, X.
Regularized adaptation: Theory, algorithms and applications.
Citeseer, 2007.

[Lubana et al., 2022]
Lubana, E. S., Trivedi, P., Koutra, D., and Dick, R.
How do quadratic regularizers prevent catastrophic forgetting: The
role of interpolation.
In Conference on Lifelong Learning Agents, pp.  819–837.
PMLR, 2022.

[McCloskey & Cohen, 1989]
McCloskey, M. and Cohen, N. J.
Catastrophic interference in connectionist networks: The sequential
learning problem.
In Psychology of learning and motivation, volume 24, pp.  109–165. Elsevier, 1989.

[Mirzadeh et al., 2020]
Mirzadeh, S. I., Farajtabar, M., Pascanu, R., and Ghasemzadeh, H.
Understanding the role of training regimes in continual learning.
In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin,
H. (eds.), Advances in Neural Information Processing Systems,
volume 33, pp.  7308–7320. Curran Associates, Inc., 2020.

[Morshed et al., 2022]
Morshed, M. S., Islam, M. S., and Noor-E-Alam, M.
Sampling kaczmarz-motzkin method for linear feasibility problems:
generalization and acceleration.
Mathematical Programming, 1940 (1-2):0
719–779, 2022.

[Nedić, 2010]
Nedić, A.
Random projection algorithms for convex set intersection problems.
In 49th IEEE Conference on Decision and Control (CDC), pp.  7655–7660. IEEE, 2010.

[Needell & Tropp, 2014]
Needell, D. and Tropp, J. A.
Paved with good intentions: analysis of a randomized block kaczmarz
Linear Algebra and its Applications, 441:0 199–221,

[Novikoff, 1962]
Novikoff, A. B.
On convergence proofs on perceptrons.
In Proceedings of the Symposium on the Mathematical Theory of
Automata, volume 12, pp.  615–622, New York, NY, USA, 1962. Polytechnic
Institute of Brooklyn.

[Peng et al., 2023]
Peng, L., Giampouras, P., and Vidal, R.
The ideal continual learner: An agent that never forgets.
In International Conference on Machine Learning, 2023.

[Pentina et al., 2015]
Pentina, A., Sharmanska, V., and Lampert, C. H.
Curriculum learning of multiple tasks.
In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp.  5492–5500, 2015.

[Reich & Zalas, 2023]
Reich, S. and Zalas, R.
Polynomial estimates for the method of cyclic projections in hilbert
Numerical Algorithms, pp.  1–26, 2023.

[Robins, 1995]
Robins, A.
Catastrophic forgetting, rehearsal and pseudorehearsal.
Connection Science, 70 (2):0 123–146, 1995.

[Rolnick et al., 2019]
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., and Wayne, G.
Experience replay for continual learning.
Advances in Neural Information Processing Systems, 32, 2019.

[Rosset et al., 2004]
Rosset, S., Zhu, J., and Hastie, T. J.
Margin maximizing loss functions.
In Advances in neural information processing systems, pp.  1237–1244, 2004.

[Shalev-Shwartz et al., 2012]
Shalev-Shwartz, S. et al.
Online learning and online convex optimization.
Foundations and Trends® in Machine Learning,
40 (2):0 107–194, 2012.

[Shamir & Zhang, 2013]
Shamir, O. and Zhang, T.
Stochastic gradient descent for non-smooth optimization: Convergence
results and optimal averaging schemes.
In International conference on machine learning, pp.  71–79.
PMLR, 2013.

[Sidford, 2020]
Sidford, A.
Ms&e 213/cs 269o: Chapter 3-convexity, 2020.

[Smith et al., 2023]
Smith, J. S., Tian, J., Hsu, Y.-C., and Kira, Z.
A closer look at rehearsal-free continual learning.
In CVPR Workshop on Continual Learning in Computer Vision,

[Soudry et al., 2018]
Soudry, D., Hoffer, E., Shpigel Nacson, M., Gunasekar, S., and Srebro, N.
The implicit bias of gradient descent on separable data.
JMLR, 2018.

[Stojanov et al., 2019]
Stojanov, S., Mishra, S., Thai, N. A., Dhanda, N., Humayun, A., Yu, C., Smith,
L. B., and Rehg, J. M.
Incremental object learning from contiguous views.
In Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition, pp.  8777–8786, 2019.

[Tang et al., 2022]
Tang, J., Lin, K.-Y., and Li, L.
Using domain adaptation for incremental svm classification of drift
Mathematics, 100 (19):0 3579, 2022.

[Von Neumann, 1949]
Von Neumann, J.
On rings of operators. reduction theory.
Annals of Mathematics, pp.  401–485, 1949.

[Wei et al., 2019]
Wei, C., Lee, J. D., Liu, Q., and Ma, T.
Regularization matters: Generalization and optimization of neural
nets vs their induced kernel.
Advances in Neural Information Processing Systems, 32, 2019.

[Woodworth et al., 2020]
Woodworth, B., Gunasekar, S., Lee, J. D., Moroshko, E., Savarese, P., Golan,
I., Soudry, D., and Srebro, N.
Kernel and rich regimes in overparametrized models.
In Conference on Learning Theory, pp.  3635–3673. PMLR,

[Yang et al., 2007]
Yang, J., Yan, R., and Hauptmann, A. G.
Adapting svm classifiers to data with shifted distributions.
In Seventh IEEE international conference on data mining
workshops (ICDMW 2007), pp.  69–76. IEEE, 2007.

[Yoon et al., 2018]
Yoon, J., Yang, E., Lee, J., and Hwang, S. J.
Lifelong learning with dynamically expandable networks.
In International Conference on Learning Representations, 2018.

[Zenke et al., 2017]
Zenke, F., Poole, B., and Ganguli, S.
Continual learning through synaptic intelligence.
In International Conference on Machine Learning, pp.  3987–3995. PMLR, 2017.