# A Theory for Knowledge Transfer in Continual Learning

Diana Benavides-Prado
School of Computer Science
The University of Auckland
New Zealand
<EMAIL_ADDRESS>
&Patricia Riddle
School of Computer Science
The University of Auckland
New Zealand
<EMAIL_ADDRESS>

###### Abstract

Continual learning of a stream of tasks is an active area in deep neural
networks. The main challenge investigated has been the phenomenon of
catastrophic forgetting or interference of newly acquired knowledge with
knowledge from previous tasks. Recent work has investigated forward knowledge
transfer to new tasks. Backward transfer for improving knowledge gained during
previous tasks has received much less attention. There is in general limited
understanding of how knowledge transfer could aid tasks learned continually.
We present a theory for knowledge transfer in continual supervised learning,
which considers both forward and backward transfer. We aim at understanding
their impact for increasingly knowledgeable learners. We derive error bounds
for each of these transfer mechanisms. These bounds are agnostic to specific
implementations (e.g. deep neural networks). We demonstrate that, for a
continual learner that observes related tasks, both forward and backward
transfer can contribute to an increasing performance as more tasks are
observed.

##  1 Introduction

Learning a continual stream of tasks has been a long-standing challenge in
machine learning (Ring, 1997; Chen & Liu, 2018). Continual learning with deep
neural networks has been an active area of research over the past few years
(Delange et al., 2021), and it has multiple applications in a range of problem
domains (Lesort et al., 2020; Lee & Lee, 2020; Maschler et al., 2021).
Catastrophic forgetting of existing knowledge for tasks learned sequentially
has been the main challenge (Delange et al., 2021). A variety of methods for
this problem in supervised continual learning have been proposed, including
approaches for replaying examples (Lopez-Paz & Ranzato, 2017), regularisation-
based methods (Kirkpatrick et al., 2017) and network expansion methods
(Ostapenko et al., 2019).

Knowledge transfer has recently been explored as an alternative for improving
the performance of continual learning systems. Transferring knowledge in the
forward direction has demonstrated some gains (Ke et al., 2021). Backward
transfer on the other hand has been paid much less attention in continual
learning with deep neural networks (Riemer et al., 2018; Ke et al., 2020;
Vogelstein et al., 2020; New et al., 2022). However, backward transfer has
succeeded in other lifelong learning studies that use techniques such as
Support Vector Machines (SVMs) (Benavides-Prado et al., 2020), and continues
to be a desired property of continual learning systems (Rish, 2022).

We develop a theory for knowledge transfer in continual learning. We first
derive error bounds for individual tasks, when these are subject to forward
transfer when learned for the first time, or to backward transfer from future
tasks when these are learned. We then consider the order of arrival of tasks,
since this influences the the amount of transfer that task is subject to.
Based on the bounds derived for individual tasks, we calculate error bounds
for a continual learner that learns related tasks sequentially using forward
and backward transfer.

Our framework relies on three core assumptions. First, the continual learner
is embedded into an environment of related tasks. This allows us to treat the
problem of learning a sequence of tasks as the problem of learning a bias for
the whole environment incrementally. Learning this bias is helpful since the
continual learning will perform better at any task in that environment. Our
second assumption is that relatedness between these tasks relies on the
similarity between their example generating distributions. This assumption
allows us to use a set of transformation functions as a tool for constraining
the hypothesis family for learning a particular task, based on its similarity
to other tasks in the environment (from which forward or backward transfer are
to be performed). This tool has been used in other studies in multitask
learning (Ben-David & Borbely, 2008). Our final assumption is that each task
has a sufficient number of examples from which to learn. This assumption
distinguishes our framework from approaches in zero-shot or few-shot learning.
However, in Section 4 we demonstrate that the number of examples required to
learn decreases with the number of tasks.

Our proposed framework is generic and does not rely on any practical
assumptions about the implementation of the continual learner (e.g. in terms
of the learning technique, the implementation architecture, the mechanisms
used for transfer or the continual learning scenario - task-incremental,
domain-incremental or class-incremental (Van de Ven & Tolias, 2019)). We aim
to provide a rigorous theoretical analysis to show the potential of knowledge
transfer while learning sequentially, and to encourage more research in this
direction.

This paper is organised as follows. Section 2 describes previous research in
knowledge transfer for continual learning. Section 3 provides preliminaries
and notation. Section 4 describes error bounds derived for tasks learned
continually using forward knowledge transfer. Section 5 describes error bounds
for tasks learned continually using backward transfer. Section 6 describes
error bounds for a continual learner that uses both forward and backward
transfer. Finally, Section 8 provides some discussion and final remarks.

##  2 Previous Research

Catastrophic forgetting or interference of new tasks with previously acquired
knowledge has been studied extensively in supervised continual learning with
deep neural networks (Delange et al., 2021). Several methods to avoid
catastrophic forgetting have been proposed, ranging from example replay
(Lopez-Paz & Ranzato, 2017; van de Ven et al., 2020) to regularisation-based
(Kirkpatrick et al., 2017; Zeng et al., 2019) to dynamic networks (Yoon et
al., 2017; Hung et al., 2019). Beyond catastrophic forgetting, the classic aim
of continual learning systems has been to achieve increasingly knowledgeable
systems (Ring, 1997; Chen & Liu, 2018). Knowledge transfer has been proposed
as a mechanism to achieve this (Ke et al., 2020; Rostami et al., 2020;
Benavides-Prado, 2020). Forward transfer with continual deep neural networks
has been studied recently (Ke et al., 2021). Backward transfer, in contrast,
has received much less attention (Riemer et al., 2018; Ke et al., 2020;
Vogelstein et al., 2020), although it was explored with alternative techniques
such as SVM (Benavides-Prado et al., 2020).

Ben-David and Borbely (2008) and Baxter (2000) studied the effects of learning
multiple related tasks jointly with multitask learning. Baxter (2000) derived
the expected average error for a group of tasks learned jointly. Ben-David and
Borbely (2008) derived similar bounds for a single task learned under the same
framework. More recently, Benavides-Prado, Koh and Riddle (2020) derived error
bounds of knowledge transfer across SVM models in supervised continual
learning. This research showed that given a set of related tasks, backward
transfer with SVM can be used to achieve systems that improve their
performance with each incoming task (Benavides-Prado et al., 2020).
Furthermore, forward transfer can also be used to aid learning of new tasks.
Although novel, these bounds were specific to the implementation using SVM.
Here we extend this work by deriving error bounds that are agnostic to the
implementation, for both forward transfer and backward transfer. We also
derive error bounds for a continual learner that uses knowledge transfer
whilst learning related tasks sequentially.

Other theoretical frameworks in transfer learning have studied how the degree
of relatedness among tasks helps transfer (Lampinen & Ganguli, 2018), and how
transfer helps curriculum learning (Weinshall et al., 2018). Theoretical
studies in continual learning have studied the effects of task similarity in
catastrophic forgetting (Lee et al., 2021), and discovered that optimal
continual learning is NP-hard and requires perfect memory (Knoblauch et al.,
2020). However, to the best of our knowledge there is no prior study that
evaluates the effects of forward and backward knowledge transfer in learning a
continual stream of supervised tasks.

##  3 Preliminaries and Definitions

Supervised continual learning is about learning a stream of tasks
$\mathbf{T}=\\{T_{1},\ldots,T_{n}\\}$. A given task $T$ in the sequence has an
underlying probability distribution $P(X,Y)$ (or simply $P$, which we use
later indistinctly). For that task, the aim is to learn a function
$f:X\rightarrow Y$, that maps the input space $X$ to the output space $Y$.
Learning works by exploring a hypothesis space $H$ on that task, and finding
the hypothesis $h\in H$ such that:

$\begin{gathered}Er^{P}(h)=min_{h\in H}\mathcal{L}(h(x),y)\end{gathered}$ (1)

where $\mathcal{L}$ is a loss function. Naturally, estimating the error of $h$
over the actual distribution $P$ is difficult since $P$ can not be observed
directly. Instead, a sample $S$ of $m$ examples extracted repeatedly from $P$
is used such that:

$\begin{gathered}S=\\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots,(x_{m},y_{m})\\}\end{gathered}$
(2)

And the empirical error of $h\in H$ over $S$ is such that:

$\begin{gathered}\hat{Er}^{S}(h)=min_{h\in
H}\dfrac{1}{m}\sum_{i=1}^{m}\mathcal{L}(h(x_{i}),y_{i})\end{gathered}$ (3)

To find the best $h$ that satisfies Eq. 3, the learner aims to find the
hypothesis that best fits this sample better, such that:

$\begin{gathered}h^{*}=\inf_{h\in H}\mathcal{L}(h(x),y)\end{gathered}$ (4)

Knowledge transfer for continual learning aims to share knowledge across tasks
observed sequentially. In our framework, we distinguish two types of transfer:
1) forward transfer, which aims to learn new or target tasks better or faster
by transferring knowledge gained during tasks learned earlier, and 2) backward
transfer, which aims to improve future performance over previous or source
tasks by using knowledge collected while learning new tasks. We assume that
tasks observed by the continual learner are related. Therefore, these tasks
are assumed to belong to the same environment, and the continual learner can
become better at learning in this environment as more tasks are observed.

Formally, we define the environment of the continual learner as follows:

######  Definition 1.

An environment $(\mathcal{P},\mathcal{Q})$ of related tasks, corresponds to
the set of all probability distributions on $\mathcal{X}\times\mathcal{Y}$,
denoted $\mathcal{P}$ , and a distribution on $\mathcal{P}$, denoted
$\mathcal{Q}$. Instead of exploring a single hypothesis space, the continual
learner has access to a family of hypothesis spaces
$\mathbb{H}=\\{H_{1},H_{2},\ldots,H_{n}\\}$, one for each task. In practice,
the learner has access to multiple samples to learn from, one sample for each
task, such that $\mathbf{S}=\\{S_{1},\ldots,S_{n}\\}$ are drawn at random from
underlying probability distributions $\mathbf{P}=\\{P_{1},\ldots,P_{n}\\}$.

Access to a family of hypothesis spaces rather than a single hypothesis space,
as in single-task learning, gives the continual learner the potential to learn
a good bias that can generalise well to novel tasks from the same environment.
Rather than producing a hypothesis that with high probability will perform
well on future examples of a particular task, by learning related tasks
continually the learner will produce a hypothesis space that with high
probability will perform well on future tasks within the same environment.
This main result has been demonstrated in the context of multitask learning
(Baxter et al., 2000), and is the main result we demonstrate in Sections 4-6
for a stream of tasks learned continually using knowledge transfer.

The notion of relatedness for tasks in the environment of the continual
learner relies on the similarity of their example generating distributions
(Ben-David & Borbely, 2008). Formally, given a set of transformation functions
$\mathpzc{f}\in\mathcal{F}$ such that
$\mathpzc{f}:\mathcal{X}\rightarrow\mathcal{X}$, tasks in the environment are
$\mathcal{F}$-related if, for some fixed probability distribution over
$\mathcal{X}\times Y$, if the examples in each of these tasks can be generated
by applying some $\mathpzc{f}\in\mathcal{F}$ to that distribution. Therefore,
we can define the equivalence relation (Raczkowski & Sadowski, 1990)
$\sim_{\mathcal{F}}$ on $\mathbb{H}$, where $\mathbb{H}$ is a family of
hypothesis spaces for all tasks in the environment, as follows:

######  Definition 2.

Let $\\{P_{1},\ldots,P_{n}\\}$ be the underlying probability distributions of
a set of $n$ tasks over a domain $\mathcal{X}\times Y$. Let $\mathcal{F}$ be a
set of transformations $\mathpzc{f}:\mathcal{X}\rightarrow\mathcal{X}$. Let
$P_{1}$ and $P_{2}$ be related if one can be generated from the other by
applying some $\mathpzc{f}\in\mathcal{F}$, such that
$P_{1}=\mathpzc{f}[P_{2}]$ (and therefore $P_{2}=\mathpzc{f}^{-1}[P_{1}]$) or
$P_{2}=\mathpzc{f}[P_{1}]$ (and therefore $P_{1}=\mathpzc{f}^{-1}[P_{2}]$).
The samples $\\{S_{1},\ldots,S_{n}\\}$ to be used during learning tasks
$\\{T_{1},\ldots,T_{n}\\}$ are said to be $\mathcal{F}$-related if these
samples come from $\mathcal{F}$-related probability distributions.

Let $\mathbb{H}$ be a family of hypothesis spaces over the domain
$\mathcal{X}\times Y$, and $\mathbb{H}$ be closed under the action of
$\mathcal{F}$. Let $\mathcal{H}$ be a family of hypothesis spaces that consist
of sets of hypotheses $[\mathpzc{h}]\in\mathbb{H}$ which are equivalent up to
transformations in $\mathcal{F}$. If $\mathcal{F}$ acts as a group over
$\mathbb{H}$ because:

  * •

For every $\mathpzc{f}\in\mathcal{F}$ and every $\mathpzc{h}\in\mathbb{H}$,
$\mathpzc{h}\circ\mathpzc{f}\in\mathbb{H}$, and

  * •

$\mathcal{F}$ is closed under transformation composition and inverses, i.e.
for every $\mathpzc{f},\mathpzc{g}\in\mathcal{F}$, the inverse transformation,
$\mathpzc{f}^{-1}$, and the composition, $\mathpzc{f}\circ\mathpzc{g}$ are
also members of $\mathpzc{F}$

Then the equivalence relation $\sim_{\mathcal{F}}$ on $\mathbb{H}$ is defined
by: $\mathpzc{h_{1}}\sim_{\mathcal{F}}\mathpzc{h_{2}}\iff$ there exists
$\mathpzc{f}\in\mathcal{F}$ such that
$\mathpzc{h_{2}}=\mathpzc{h_{1}}\circ\mathpzc{f}$.

Therefore this framework considers the family of hypothesis spaces
$\mathcal{H}=\\{[\mathpzc{h}]:[\mathpzc{h}]\in\mathbb{H}\\}$, which is the
family of all equivalence classes of $\mathbb{H}$ under $\sim_{\mathcal{F}}$.

The original setting of this framework is in multitask learning (Ben-David &
Borbely, 2008), where the equivalence class $[\mathpzc{h}]$ for a target task
is first found using samples from all tasks. This requires to first identify
aspects of all tasks that are invariant under $\mathcal{F}$. A second step
restricts the learning of a particular task to selecting a hypothesis
$\mathpzc{h^{\prime}}\in[\mathpzc{h}]$ as the hypothesis for that task.
Therefore, the target task can benefit from transfer during this second step
by exploring the hypothesis space to be explored for the target task
$[\mathpzc{h}]$ that contains these invariances.

In continual learning we are faced with a similar problem, but rather than
learning tasks jointly these are observed sequentially. However, provided
these tasks are $\mathcal{F}$-related, we can adopt a similar framework to
derive error bounds of a target task that is learned with forward transfer
from a set of source tasks, and of source tasks for which knowledge is updated
with backward transfer from a recently learned target tas. In the following
sections we develop a theory of knowledge transfer across continual tasks that
use these two transfer mechanisms.

##  4 Forward Knowledge Transfer across Related Tasks

In this and following sections, we will use (t) to refer to a target task, or
target probability distribution or target sample, and (s) to denote a source
task, or source probability distribution or source sample. In forward
transfer, the aim is to learn a target task $T^{(t)}$ helped by knowledge
obtained during previous $n$ source tasks
$\\{T^{(s)}_{1},\ldots,T^{(s)}_{n}\\}$, with probability distributions
$P^{(t)}$ and $\\{P^{(s)}_{1},\ldots,P^{(s)}_{n}\\}$ and their corresponding
observed samples $S^{(t)}$ and $\\{S^{(s)}_{1},\ldots,S^{(s)}_{n}\\}$. Forward
transfer for a continual learner which observes $\mathcal{F}$-related tasks is
defined as follows:

######  Definition 3.

Given classes $\mathcal{F}$ and $\mathcal{H}$, and a set of labeled samples
$\\{S^{(s)}_{1},\ldots,S^{(s)}_{n}\\}$ for a set of $n$ source tasks and a
labeled sample $S^{(t)}$ for a target task, in forward knowledge transfer
while learning task $T^{(t)}$, the continual learner:

  1. 1.

Has access to $[\mathpzc{h}^{*}]\in\mathcal{H}$, obtained as a result of
minimising
$\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h}]}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})$
over all $\mathpzc{[h]}\in\mathcal{H}$.

  2. 2.

Selects $\mathpzc{h^{\Diamond}}\in\mathpzc{[h^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h^{*}]}$, and outputs
$\mathpzc{h^{\Diamond}}$ as the hypothesis for $T^{(t)}$.

In practice, having access to $[\mathpzc{h^{*}}]$ during a target task
$T^{(t)}$ implies that the continual learner can access to some representation
of the knowledge obtained during previous tasks (e.g. access to a neural
network representing that knowledge). We derive error bounds for learning a
target task $T^{(t)}$ helped by knowledge transfer from $\mathcal{F}$-related
source tasks as follows:

######  Theorem 1.

Let $\\{P^{(s)}_{1}$, …, $P^{(s)}_{n}\\}$ and $P^{(t)}$ be a set of
$\mathcal{F}$-related probability distributions, and $\\{S^{(s)}_{1}$, …,
$S^{(s)}_{n}\\}$ and $S^{(t)}$ random samples representing these
distributions. Let $\mathcal{F}$ and $\mathcal{H}$ be defined as in Definition
2. Let $d_{max}=sup\\{VC\mbox{-}dim(H):H\in\mathcal{H}\\}$. Let
$d_{\mathcal{H}}(n)=max_{[\mathpzc{h}]\in\mathcal{H}}VC\mbox{-}dim([\mathpzc{h}])$.
Let $\mathpzc{h^{\Diamond}}$ be selected according to Definition 3. Then, for
every constant $\epsilon_{1}$, $\epsilon_{2}$, $\delta>0$, with $|S^{(t)}|$
and $|S^{(s)}_{i}|$ defined similarly to Theorem $3$ in Ben-David and Borbely
(2008):

$\begin{gathered}|S^{(t)}|\geq\dfrac{64}{\epsilon_{1}^{2}}\bigg{[}2d_{max}log\dfrac{12}{\epsilon_{1}}+log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(5)

and, for all i $\leq$ n:

$\begin{gathered}|S^{(s)}_{i}|\geq\dfrac{88}{\epsilon_{2}^{2}}\bigg{[}2d_{\mathcal{H}}(n)log{\dfrac{22}{\epsilon_{2}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(6)

then with probability greater than $(1-\delta)$:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}(\mathpzc{h^{\Diamond}})\leq\inf_{\mathpzc{h}\in\mathbb{H}}\mathbb{E}r^{P^{(t)}}(\mathpzc{h})+2(\epsilon_{1}+\epsilon_{2})\end{gathered}$
(7)

###### Proof.

Let $\mathpzc{h^{\\#}}$ be the best $P^{(t)}$ label predictor in $\mathbb{H}$,
i.e.
$\mathpzc{h}^{\\#}=\operatorname*{arg\,min}_{\mathpzc{h}\in\mathbb{H}}\mathbb{E}r^{P^{(t)}}(\mathpzc{h})$.
Let $[\mathpzc{h}^{*}]$ be the equivalence class picked according to
Definition 3. By the choice of $\mathpzc{h}^{*}$:

$\begin{gathered}\inf_{\mathpzc{h_{1}},\ldots,\mathpzc{h_{n}}\in\mathpzc{[h^{*}]}}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\leq\inf_{\mathpzc{h_{1}},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h^{\\#}]}}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\end{gathered}$
(8)

By Theorem 2 in Ben-David and Borbely (2008), with probability greater than
$(1-\delta/2)$:

$\begin{gathered}\inf_{\mathpzc{h_{1}},\ldots,\mathpzc{h_{n}}\in\mathpzc{[h^{\\#}]}}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\leq\mathbb{E}r^{P^{(t)}}([\mathpzc{h}^{\\#}])+\epsilon_{1}\end{gathered}$
(9)

and:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}([\mathpzc{h^{*}}])\leq\inf_{\mathpzc{h_{1},\ldots,\mathpzc{h_{n}}\in\mathpzc{[h^{*}]}}}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})+\epsilon_{1}\end{gathered}$
(10)

Then, combining the inequalities above, with probability greater than
$(1-\delta/2)$:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}([\mathpzc{h^{*}}])\leq\mathbb{E}r^{P^{(t)}}([\mathpzc{h}^{\\#}])+2\epsilon_{1}\end{gathered}$
(11)

Since $\mathpzc{h^{\Diamond}}\in[\mathpzc{h^{*}}]$, with probability greater
than $(1-\delta/2)$, $\mathpzc{h^{\Diamond}}$ will have an error for $P^{(t)}$
which is within $2\epsilon_{2}$ of the best hypothesis there, i.e.
$\mathbb{E}r^{P^{(t)}}([\mathpzc{h^{*}}])$. Therefore:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}(\mathpzc{h^{\diamond}})\leq\mathbb{E}r^{P^{(t)}}(\mathpzc{h}^{\\#})+2(\epsilon_{1}+\epsilon_{2})\end{gathered}$
(12)

∎

Theorem 7 implies that, for a sufficiently large number of examples for the
sources and the target tasks, forward transfer is expected to benefit learning
of a target task. This result is achieved by choosing a hypothesis space for
$T^{(t)}$ which is biased towards the hypothesis space learned for previous
$\mathcal{F}$-related tasks from the same environment. The extent of this
benefit depends on the number of examples per task (see Eq. 5 and Eq. 6).
Baxter (2000) demonstrated that the number of examples required per task
decreases along with an increasing number of tasks, in particular:

$\begin{gathered}|S|=\mathcal{O}\Big{(}\dfrac{1}{n}log\mathcal{C}(\epsilon,\mathcal{H}_{l}^{n})\Big{)}\end{gathered}$
(13)

where $\mathcal{C}(\epsilon,\mathcal{H}_{l}^{n})$ is the capacity of the
learner given an error $\epsilon$ and a set of $n$ sets of loss functions
$\\{\mathbf{h}_{l}^{1},\ldots,\mathbf{h}_{l}^{n}\\}\in\mathcal{H}_{l}^{n}$ for
the family of hypothesis spaces $\mathcal{H}$. Provided that this capacity
increases sublinearly with $n$, the number of examples required per task will
decrease with an increasing number of tasks.

The amount of transfer to a target task and therefore the extent to which the
bound in Theorem 7 is satisfied depends on how many source tasks are used for
transfer. Intuitively, the larger this number, the smaller the bound, since
the target task will have a better bias of its environment with more related
tasks having been observed, which would lead to a better hypothesis space to
be selected for that task. Therefore, the later a target task is observed, the
greater the opportunity for it to benefit from forward transfer. This is in
accordance with previous research that demonstrated that a larger number of
tasks learned continually benefits transfer (Benavides-Prado et al., 2017;
2020). Next we analyse the effect of the task order in forward transfer, and
the error bounds of a target task depending on that order. Next we derive
bounds for forward transfer that account for the order of the task being
observed in the sequence.

######  Definition 4.

Given classes $\mathcal{F}$ and $\mathcal{H}$, a set of labeled samples
{$S^{(s)}_{1}$, …, $S^{(s)}_{n}\\}$ for a set of source tasks and a labeled
sample $S^{(t)}$ for a target task. Let:

  * •

$[\mathpzc{h}_{n}^{*}]\in\mathcal{H}$ be the result of minimising
$\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h_{n}^{*}}]}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h_{i}})$
over all $\mathpzc{[h]}\in\mathcal{H}$, at time $n$.

  * •

$[\mathpzc{h}_{n+z}^{*}]\in\mathcal{H}$ be the result of minimising
$\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n+z}}\in[\mathpzc{h_{n+z}^{*}}]}\dfrac{1}{n+z}\sum_{i=1}^{n+z}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h_{i}})$
over all $\mathpzc{[h]}\in\mathcal{H}$, at time $n+z$.

  * •

$\mathpzc{h_{n}^{\Diamond}}\in\mathpzc{[h_{n}^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h_{n}^{*}]}$, and outputs
$\mathpzc{h_{n}^{\Diamond}}$ as the hypothesis for task $T^{(t)}$ at time $n$.

  * •

$\mathpzc{h_{n+z}^{\Diamond}}\in\mathpzc{[h_{n+z}^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h_{n+z}^{*}]}$, and outputs
$\mathpzc{h_{n+z}^{\Diamond}}$ as the hypothesis for task $T^{(t)}$ at time
$n+z$.

######  Corollary 1.

Let $\\{P^{(s)}_{1}$, …, $P^{(s)}_{n}\\}$, $\\{S^{(s)}_{1}$, …,
$S^{(s)}_{n}\\}$ and $S^{(t)}$, $\mathcal{F}$, $\mathcal{H}$, $d_{max}$,
$d_{\mathcal{H}}(n)$ be defined as in Theorem 7, at time $n$. Similarly, let
$\\{P^{(s)}_{1}$, …, $P^{(s)}_{n+z}\\}$ and $P^{(t)}$ be a set of
$\mathcal{F}$-related probability distributions, $\\{S^{(s)}_{1}$, …,
$S^{(s)}_{n+z}\\}$ and $S^{(t)}$ random samples representing these
distributions, at time $n+z$. Let $\mathpzc{h^{\Diamond}}_{n}$ and
$\mathpzc{h^{\Diamond}}_{n+z}$ be selected according to Definition 4, at time
$n$ and $n+z$, respectively. Then, for every $\epsilon_{1}$, $\epsilon_{2}$,
$\delta>0$, if:

$\begin{gathered}|S^{(t)}|\geq\dfrac{64}{\epsilon_{1}^{2}}\bigg{[}2d_{max}log\dfrac{12}{\epsilon_{1}}+log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(14)

and, at time $n$, for all i $\leq$ n:

$\begin{gathered}|S^{(s)}_{i}|\geq\dfrac{88}{\epsilon_{n}^{2}}\bigg{[}2d_{\mathcal{H}}(n)log{\dfrac{22}{\epsilon_{n}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(15)

while, at time $n+z$, for all i $\leq$ (n+z):

$\begin{gathered}|S^{(s)}_{i}|\geq\dfrac{88}{\epsilon_{n+z}^{2}}\bigg{[}2d_{\mathcal{H}}(n+z)log{\dfrac{22}{\epsilon_{n+z}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(16)

then with probability greater than $(1-\delta)$:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}_{n+z}(\mathpzc{h_{n+z}^{\Diamond}})\leq\mathbb{E}r^{P^{(t)}}_{n}(\mathpzc{h_{n}^{\Diamond}})+\epsilon_{n}+\epsilon_{n+z}\end{gathered}$
(17)

See Appendix A for the proof of this corollary. The main part of the proof in
Appendix A lies in Eq. 46. Since the best hypothesis space for a larger number
of tasks is better than the best hypothesis space for a smaller number of
tasks in the same environment, i.e. the bias over the environment gets refined
over time, tasks observed later in the sequence will benefit more from
transfer.

Bounds in Theorem 7 and Corollary 17 depend on the difference between
$d_{max}$ and $d_{\mathcal{H}}(n)$, and $VC\mbox{-}dim(\mathbb{H})$, with
$d_{max}=sup\\{VC\mbox{-}dim(H):H\in\mathcal{H}\\}$ and
$d_{\mathcal{H}}(n)=max_{[\mathpzc{h}]\in\mathcal{H}}VC\mbox{-}dim([\mathpzc{h}])$,
and $d_{max}\leq d_{\mathcal{H}}(n)\leq VC\mbox{-}dim(\mathbb{H})$. Ben-David
and Borbely (2008) showed that, for a sufficiently large number or tasks $n$,
$d_{max}=max_{[\mathpzc{h}]\in\mathcal{H}}VC\mbox{-}dim([\mathpzc{h}])=d_{\mathcal{H}}(n)$.
We refer readers to Section 6 of Ben-David and Borbely (2008) for details.

##  5 Backward Knowledge Transfer across Related Tasks

Backward transfer works by updating a source task $T^{(s)}$ using knowledge
gained during the most recent target task $T^{(t)}$. Transfer occurs from the
space of a target probability distribution $P^{(t)}$, represented by a sample
$S^{(t)}$, to the space of a probability distribution $P^{(s)}$ that uses a
sample $S^{(s)}$ for learning that source task. In continual learning, the aim
is to use $P^{(t)}$, and its corresponding sample $S^{(t)}$, to bias the
update of a refined version of $P^{(s)}$ towards aspects that are invariant
with $P^{(t)}$, provided these are related. Benavides-Prado, Koh and Riddle
(2020), analysed the special case of two tasks, one source $T^{(s)}$ and one
target $T^{(t)}$, for a specific implementation of a continual learner based
on SVM. Here, we present bounds for an agnostic continual learner, as follows:

######  Definition 5.

Given classes $\mathcal{F}$ and $\mathcal{H}$, and a pair of labeled samples
$S^{(s)}$, $S^{(t)}$ for tasks $T^{(s)}$, $T^{(t)}$, during backward transfer
the continual learner:

  1. 1.

Selects $[\mathpzc{h}^{*}]\in\mathcal{H}$ that minimises
$\inf_{\mathpzc{h}^{(s)},\mathpzc{h^{(t)}}\in[\mathpzc{h}]}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)})\big{)}$
over all $\mathpzc{[h]}\in\mathcal{H}$.

  2. 2.

Selects $\mathpzc{h^{\Diamond}}\in\mathpzc{[h^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h^{*}]}$, and outputs
$\mathpzc{h^{\Diamond}}$ as the hypothesis for task $T^{(s)}$.

In practice, the two steps in Definition 5 could be performed sequentially or
jointly. For example, selecting $[\mathpzc{h}^{*}]$ in the first step could be
performed by jointly training an auxiliary learner with examples from both
$T^{(s)}$ and $T^{(t)}$, and then transferring back this information to
$T^{(s)}$ during the second step. Alternatively, both $[\mathpzc{h^{*}}]$
could be selected jointly while training for $T^{(s)}$ aided by $T^{(t)}$.

Based on Definition 5, in the special case of two tasks $T^{(s)}$ and
$T^{(t)}$:

######  Theorem 2.

Let $P^{(s)}$ and $P^{(t)}$ be a set of $\mathcal{F}$-related probability
distributions,and $S^{(s)}$ and $S^{(t)}$ random samples representing these
distributions on tasks $T^{(s)}$ and $T^{(t)}$ respectively. Let $\mathcal{F}$
and $\mathcal{H}$ be defined as in Definition 2. Let $d_{max}$ and
$d_{\mathcal{H}}(n)$ be defined as in Theorem 7. Let $\mathpzc{h^{\Diamond}}$
be selected according to Definition 5. Then, for every $\epsilon_{1}$,
$\epsilon_{2}$, $\delta>0$, if:

$\begin{gathered}|S^{(s)}|\geq\dfrac{64}{\epsilon_{1}^{2}}\bigg{[}2d_{max}log\dfrac{12}{\epsilon_{1}}+log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(18)

and:

$\begin{gathered}|S^{(t)}|\geq\dfrac{88}{\epsilon_{2}^{2}}\bigg{[}2d_{\mathcal{H}}(2)log{\dfrac{22}{\epsilon_{2}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(19)

then with probability greater than $(1-\delta)$:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}(\mathpzc{h^{\Diamond}})\leq\inf_{\mathpzc{h}\in\mathbb{H}}\mathbb{E}r^{P^{(s)}}(\mathpzc{h})+2(\epsilon_{1}+\epsilon_{2})\end{gathered}$
(20)

###### Proof.

Let $\mathpzc{h^{\\#}}$ be the best $P^{(s)}$ label predictor in $\mathbb{H}$,
i.e.
$\mathpzc{h}^{\\#}=\operatorname*{arg\,min}_{\mathpzc{h}\in\mathbb{H}}\mathbb{E}r^{P^{(s)}}(h)$.
Let $[\mathpzc{h}^{*}]$ be the equivalence class picked according to
Definition 5. By the choice of $\mathpzc{h}^{*}$:

$\begin{gathered}\inf_{\mathpzc{h^{(s)}},\mathpzc{h^{(t)}}\in\mathpzc{[h^{*}]}}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)})\big{)}\leq\inf_{\mathpzc{h^{(s)}},\mathpzc{h^{(t)}}\in[\mathpzc{h^{\\#}]}}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)})\big{)}\end{gathered}$
(21)

By Theorem 2 in Ben-David and Borbely (2008), in the case of two tasks:

$\begin{gathered}\bigg{|}\mathbb{E}r^{P^{(s)}}([\mathpzc{h}])-\inf_{h^{(s)},h^{(t)}\in[\mathpzc{h}]}\dfrac{1}{2}(\hat{\mathbb{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\hat{\mathbb{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)}))\bigg{|}\leq\epsilon_{1}\end{gathered}$
(22)

then with probability greater than $(1-\delta/2)$:

$\begin{gathered}\inf_{\mathpzc{h^{(s)}},\mathpzc{h^{(t)}}\in\mathpzc{[h^{\\#}]}}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)})\big{)}\leq\mathbb{E}r^{P^{(s)}}([\mathpzc{h}^{\\#}])+\epsilon_{1}\end{gathered}$
(23)

and:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}([\mathpzc{h^{*}}])\leq\inf_{\mathpzc{h^{(s)},\mathpzc{h^{(t)}}\in\mathpzc{[h^{*}]}}}(\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}}(\mathpzc{h}^{(t)}))+\epsilon_{1}\end{gathered}$
(24)

Then, combining the inequalities above, with probability greater than
$(1-\delta/2)$:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}([\mathpzc{h^{*}}])\leq\mathbb{E}r^{P^{(s)}}([\mathpzc{h}^{\\#}])+2\epsilon_{1}\end{gathered}$
(25)

Since $\mathpzc{h^{\Diamond}}\in[\mathpzc{h^{*}}]$, with probability greater
than $(1-\delta/2)$, $\mathpzc{h^{\Diamond}}$ will have an error for $P^{(s)}$
which is within $2\epsilon_{2}$ of the best hypothesis there, i.e.
$\mathbb{E}r^{P_{s}}([\mathpzc{h^{*}}])$. Therefore:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}(\mathpzc{h^{\Diamond}})\leq\inf_{\mathpzc{h}\in\mathbb{H}}\mathbb{E}r^{P^{(s)}}(\mathpzc{h})+2(\epsilon_{1}+\epsilon_{2})\end{gathered}$
(26)

∎

Similar to forward transfer, these bounds depend on the difference between
$d_{max}$, $d_{\mathcal{H}}(n)$, and $VC\mbox{-}dim(\mathbb{H})$. Section 4
provides details on the meaning of these parameters and their relation to each
other.

The main result from Theorem 20 and its corresponding proof is that an
existing source task can also benefit from knowledge acquired during a related
target task. This benefit is expected to be smaller than that of transferring
forward, since forward transfer benefits from multiple sources (see Eq. $10$)
while backward transfer benefits from a single target task (see Eq. $26$). We
show that doing backward transfer helps to select a better hypothesis space
and therefore provides a better bound on the performance of that task (see Eq.
$30$). Therefore, a natural next question is whether backward transfer from a
sequence of target tasks, learned one at a time, can help improve these
bounds. we prove that doing backward transfer multiple times sequentially
helps to decrease the error on a source task $T^{(s)}$ sequentially as well.

######  Definition 6.

Given classes $\mathcal{F}$ and $\mathcal{H}$, a set of labeled samples
$S^{(s)}$ for a source task, and labeled samples $S^{(t)}_{n}$,
$S^{(t)}_{n+1}$ for target tasks at times $n$ and $n+1$. Let:

  * •

$[\mathpzc{h}_{n}^{*}]\in\mathcal{H}$ be the result of minimising
$\inf_{\mathpzc{h}^{(s)},\mathpzc{h^{(t)}}\in[\mathpzc{h_{n}^{*}}]}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})$
\+ $\mathbb{\hat{E}}r^{S^{(t)}_{n}}(\mathpzc{h}^{(t)})\big{)}$ over all
$\mathpzc{[h]}\in\mathcal{H}$, at time $n$.

  * •

$[\mathpzc{h}_{n+1}^{*}]\in\mathcal{H}$ be the result of minimising
$\inf_{\mathpzc{h}^{(s)},\mathpzc{h^{(t)}_{n}},\mathpzc{h^{(t)}_{n+1}}\in[\mathpzc{h_{n}^{*}}]}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})$
\+ $\mathbb{\hat{E}}r^{S^{(t)}_{n}}(\mathpzc{h}^{(t)}_{n})$
+$\mathbb{\hat{E}}r^{S^{(t)}_{n+1}}(\mathpzc{h}^{(t)}_{n+1})\big{)}$ over all
$\mathpzc{[h]}\in\mathcal{H}$, at time $n+1$.

  * •

$\mathpzc{h_{n}^{\Diamond}}\in\mathpzc{[h_{n}^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h_{n}^{*}]}$, and outputs
$\mathpzc{h_{n}^{\Diamond}}$ as the hypothesis for task $T^{(s)}$ at time $n$.

  * •

$\mathpzc{h_{n+1}^{\Diamond}}\in\mathpzc{[h_{n+1}^{*}]}$ that minimises
$\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h^{\prime}})$ over all
$\mathpzc{h^{\prime}}\in\mathpzc{[h_{n+1}^{*}]}$, and outputs
$\mathpzc{h_{n+1}^{\Diamond}}$ as the hypothesis for task $T^{(s)}$ at time
$n+1$.

######  Corollary 2.

Let $P^{(s)}$, $P^{(t)}_{n}$ and $P^{(t)}_{n+1}$ be a set of
$\mathcal{F}$-related probability distributions, $S^{(s)}$, $S^{(t)}_{n}$ and
$S^{(t)}_{n+1}$ random samples representing these distributions. Let
$\mathcal{F}$ and $\mathcal{H}$ be defined as in Definition 2. Let $d_{max}$
and $d_{\mathcal{H}}(n)$ be defined as in Theorem 7. Let
$\mathpzc{h_{n}^{\Diamond}}$ and $\mathpzc{h_{n+1}^{\Diamond}}$ be selected
according to Definition 6. Then, for every $\epsilon_{1}$, $\epsilon_{n}$,
$\epsilon_{n+1}$, $\delta>0$, if:

$\begin{gathered}|S^{(s)}|\geq\dfrac{64}{\epsilon_{1}^{2}}\bigg{[}2d_{max}log\dfrac{12}{\epsilon_{1}}+log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(27)

and, at time $n$:

$\begin{gathered}|S^{(t)}_{n}|\geq\dfrac{8}{\epsilon_{n}^{2}}\bigg{[}2d_{\mathcal{H}}(2)log{\dfrac{22}{\epsilon_{n}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(28)

while, at time $n+1$:

$\begin{gathered}|S^{(t)}_{n+1}|\geq\dfrac{88}{\epsilon_{n+1}^{2}}\bigg{[}2d_{\mathcal{H}}(2)log{\dfrac{22}{\epsilon_{n+1}}}+\dfrac{1}{2}log\dfrac{8}{\delta}\bigg{]}\end{gathered}$
(29)

then with probability greater than $(1-\delta)$:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}_{n+1}(\mathpzc{h^{\Diamond}_{n+1}})\leq\mathbb{E}r^{P^{(s)}}_{n}(\mathpzc{h^{\Diamond}_{n}})+\epsilon_{n}+\epsilon_{n+1}\end{gathered}$
(30)

See Appendix B for the proof of this corollary. These results imply that doing
backward transfer sequentially whilst learning target tasks will lead to more
refined hypothesis spaces in a source task, beyond the hypothesis space
learned initially (with or without forward transfer). Furthermore, this
suggests that continually learning $\mathcal{F}$-related tasks while doing
both forward and backward transfer can lead to a better bias over the learning
environment of these tasks, i.e. the result demonstrated by Baxter (2000) for
multitask learning, which we demonstrate in the next section.

##  6 Continual Learning of Related Tasks using Knowledge Transfer

Based on the bounds derived in Section 4 and Section 5, now we are ready to
derive bounds of a continual learner that observes supervised related tasks
sequentially while doing knowledge transfer. First, lets recall from
Definition 1 that the continual learner is embedded in an environment of
related tasks, $(\mathcal{P},\mathcal{Q})$, where $\mathcal{P}$ is the set of
all probability distributions on $\mathcal{X}\times Y$ and $\mathcal{Q}$ is a
distribution on $\mathcal{P}$. The error of a selected hypothesis space
$[\mathpzc{h}^{*}]\in\mathcal{H}$ for all tasks in such environment is defined
as:

$\begin{gathered}\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{*}])=\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h^{*}}]}\sum_{i=1}^{n}\mathbb{E}r^{P_{i}}([\mathpzc{h_{i}}])\end{gathered}$
(31)

for any $P$ drawn at random from $\mathcal{P}$ according to $\mathcal{Q}$.
Let’s define $\epsilon_{f}$ as the average $\epsilon$ when performing forward
transfer to a new task, i.e. $\epsilon_{f}$ corresponds to
$2(\epsilon_{1}+\epsilon_{2})$ in Theorem 7, averaged across all tasks.
Similarly, let’s define $\epsilon_{b}$ as the average $\epsilon$ when
performing backward transfer to a new task, i.e. $\epsilon_{b}$ corresponds to
$2(\epsilon_{1}+\epsilon_{2})$ in Theorem 20 averaged across all tasks.
Although the definitions of $\epsilon_{f}$ and $\epsilon_{b}$ oversimplify the
continual learner to the case of all tasks achieving roughly the same error
bounds by means of transfer, this will serve to demonstrate how forward and
backward transfer help to improve the bounds for the continual learner as a
whole. For a task $i$, the error bound of applying forward and backward
transfer and selecting $[\mathpzc{h}^{*}]$ instead of $[\mathpzc{h}^{\\#}]$ as
the hypothesis space for that task is:

$\begin{gathered}\mathbb{E}r^{P_{i}}([\mathpzc{h}^{*}])\leq\mathbb{E}r^{P_{i}}([\mathpzc{h}^{\\#}])+(i-1)\epsilon_{f}+(n-i)\epsilon_{b}\end{gathered}$
(32)

As demonstrated in Corollary 17 and 30, the extent to which transfer helps to
improve the error bounds of a particular task $i$ depends on the order of that
task in the sequence, which in Eq. 32 impacts the total amount of transfer
through $(i-1)$ for forward transfer and $(n-i)$ for backward transfer. Given
Eq. 32, for a sequence of tasks $n$, we can define the error bounds on the
environment that learns those tasks by means of transfer as follows.

######  Theorem 3.

Let $\\{P_{1},\ldots,P_{n}\\}$ be a set of distributions, one for each task,
drawn at random from $\mathcal{P}$, the set of all probability distributions
on $\mathcal{X}\times Y$, according to $\mathcal{Q}$, a distribution on
$\mathcal{P}$. Let $\mathcal{H}$ be the family of hypothesis spaces for $n$
tasks to be learned in the environment $\mathcal{Q}$, according to Definition
2, with $[\mathpzc{h}^{*}]\in\mathcal{H}$ selected according to Theorem 7 and
Theorem 20. Let $\mathbb{H}$ be the family of hypothesis spaces for $n$ tasks
with no transfer, and let $[\mathpzc{h}^{\\#}]\in\mathbb{H}$ be selected as
the hypothesis space for the $n$ tasks with no transfer. If the number of
tasks $n$ satisfies:

$\begin{gathered}n\geq\max\Big{\\{}\dfrac{256}{\epsilon^{2}}log\dfrac{8\mathcal{C}\Big{(}\dfrac{\epsilon}{32},\mathcal{H}^{*}\Big{)}}{\delta},\dfrac{64}{\epsilon^{2}}\Big{\\}}\end{gathered}$
(33)

with $\mathcal{H^{*}}$ = $\\{[\mathpzc{h}^{*}]:\mathpzc{h}\in\mathcal{H}\\}$,
i.e. the set of all hypothesis spaces in the hypothesis space family
$\mathcal{H}$ such that each $[\mathpzc{h}^{*}]$ is defined by:

$\begin{gathered}(P)=\inf_{\mathpzc{h}\in\mathcal{H}}\mathbb{E}r^{P}(\mathpzc{h})\end{gathered}$
(34)

and, for all $1\leq i\leq n$, the number of examples per task, $|S_{i}|$
satisfies:

$\begin{gathered}|S_{i}|\geq\max\Big{\\{}\dfrac{256}{n\epsilon^{2}}log\dfrac{8\mathcal{C}\Big{(}\dfrac{\epsilon}{32},\mathcal{H}_{l}^{n}\Big{)}}{\delta},\dfrac{64}{\epsilon^{2}}\Big{\\}}\end{gathered}$
(35)

where $\mathcal{H}_{l}^{n}$ =
$\cup_{[\mathpzc{h}^{*}]\in\mathcal{H}}[\mathpzc{h}^{*}]_{l}^{n}$ (i.e.
$\mathcal{H}_{l}^{n}$ is the union of all sequences of hypothesis
$[\mathpzc{h}^{*}]\in\mathcal{H}$, each of size $n$, subject to loss function
$l$), and with:

$\begin{gathered}\epsilon=\sum_{i=1}^{n}(i-1)\epsilon_{f}+\sum_{i=1}^{n}(n-1)\epsilon_{b}\end{gathered}$
(36)

then, with probability at least $(1-\delta)$, $[h^{*}]\in\mathcal{H}$ will
satisfy:

$\begin{gathered}\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{*}])\leq\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{\\#}])+\epsilon\end{gathered}$
(37)

###### Proof.

According to Eq. 32, for all $1\leq i\leq n$:

$\begin{gathered}\mathbb{E}r^{P_{i}}([\mathpzc{h}^{*}])\leq\mathbb{E}r^{P_{i}}([\mathpzc{h}^{\\#}])+(i-1)\epsilon_{f}+(n-i)\epsilon_{b}\end{gathered}$
(38)

which leads to:

$\begin{gathered}\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h^{*}}]}\sum_{i=1}^{n}\mathbb{E}r^{P_{i}}([\mathpzc{h_{i}^{*}}])\leq\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h}]}\sum_{i=1}^{n}\Big{(}\mathbb{E}r^{P_{i}}([\mathpzc{h_{i}^{\\#}}]))+(i-1)\epsilon_{f}+(n-1)\epsilon_{b}\Big{)}\end{gathered}$
(39)

with:

$\begin{gathered}\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{*}])=\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h^{*}}]}\sum_{i=1}^{n}\mathbb{E}r^{P_{i}}([\mathpzc{h^{*}_{i}}])\end{gathered}$
(40)

and:

$\begin{gathered}\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{\\#}])=\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h}]}\sum_{i=1}^{n}\mathbb{E}r^{P_{i}}([\mathpzc{h}_{i}^{\\#}])\end{gathered}$
(41)

then:

$\begin{gathered}\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{*}])\leq\mathbb{E}r^{\mathcal{Q}}([\mathpzc{h}^{\\#}])+\epsilon\end{gathered}$
(42)

∎

Theorem 37 and its corresponding proof provide the most relevant result of our
framework: learning a set of tasks continually with forward and backward
transfer will lead to incrementally learning a better bias over the
environment itself. Furthermore, the larger the number of tasks $n$, the
better the bounds. This can be inferred from Eq. 39, since for a particular
$\mathbb{E}r^{P_{i}}([\mathpzc{h_{i}^{\\#}}]))$ on the right-hand side the
larger the number of tasks the larger $i$ and $n$ are, the larger the
difference, or gap, with the left-hand side. This implies that the hypothesis
space for a particular task which is selected by considering other tasks, i.e.
via transfer, is a better hypothesis space than would be selected by learning
that task in isolation. Since this occurs for all tasks in the environment,
the better the bias learned over that environment will be and therefore the
better future tasks will be learned, leading to an increasingly knowledgeable
system.

As a final remark, note that in practice the bounds in Theorem 37 depend on
the samples $S_{i}$, $1\leq i\leq n$, drawn from the corresponding $P_{i}$
probability distributions, since this is the data that can be observed while
learning. For these bounds to apply, for all $1\leq i\leq n$, the bounds
between $P_{i}$ and $S_{i}$ must satisfy (Baxter et al., 2000):

$\begin{gathered}\mathbb{E}r^{P_{i}}(h)\leq\mathbb{\hat{E}}r^{S_{i}}(h)+\Big{[}\dfrac{32}{m}\Big{(}dlog\dfrac{2\epsilon
m}{d}+log\dfrac{2}{\delta}\Big{)}\Big{]}^{1/2}\end{gathered}$ (43)

with $d$ the VC-dimension of $\mathcal{H}$ and $m$ the number of examples.
Then, with probability at least $(1-\delta)$ all $h\in\mathcal{H}$ will
satisfy Eq. 43.

##  7 Experiments

We experiment with an example inspired by multitask learning (Zhang & Yeung,
2014). A continual learner observes a set of regression tasks to learn four
functions, three of which are linear and related while one is unrelated (see
Figure 1). We report six different scenarios to prove bounds presented in
previous sections: 1) forward transfer from $f_{1}$ to $f_{2}$ (bound in
Theorem 7), 2) backward transfer from $f_{2}$ to $f_{1}$ (bound in Theorem
20), 3) forward transfer from $f_{1}$ and $f_{2}$ to $f_{3}$ (bound in
Corollary 17), 4) backward transfer from $f_{2}$ to $f_{1}$ and then from
$f_{3}$ to $f_{1}$ (bound in Corollary 30), 5) forward transfer from $f_{1}$
to $f_{4}$ (bound in Theorem 7 for an unrelated task), and 6) backward
transfer from $f_{4}$ to $f_{1}$ (bound in Theorem 20 for an unrelated task).
Each of these six scenarios is trained and tested independently from the other
scenarios. We measure the final $R^{2}$ of the three tasks learned by that
continual learner. We use a neural network with $1$ hidden layer of $10$ units
to learn in each of these scenarios. For each scenario, the task from which
transfer occurs is trained only partially (i.e. before full convergence),
while the task which is subject to transfer is trained until convergence. We
generate $30$ random examples for each task, for values of $x$ between $0$ and
$10$. We add Gaussian noise with mean $1$ and standard deviation $2$. We split
data from each task into training ($75\%$) and test ($25\%$) sets. We repeat
sampling, splitting, training and testing $10$ times. Table 1 shows that
transfer across related tasks (scenarios $1$ to $4$) benefits $R^{2}$
performance. Our main finding is that, by training source tasks only
partially, we are able to keep the hypothesis space large enough for the
backward/forward transfer to have an effect. This also allows exploring a
larger set of $\mathcal{F}$ transformation functions between hypothesis
spaces, which appears to be critical for transfer. The practical implication
of this is that we will need to store partially converged versions of each
task’s model for future transfer.

Figure 1: A set of problems or tasks. Three of these are linear functions and are related, while one is an unrelated task. Scenario | $R^{2}f_{1}$ | $R^{2}f_{2}$ | $R^{2}f_{3}$ | $R^{2}f_{4}$ | $R^{2}f_{1},f_{2},f_{3}$
---|---|---|---|---|---
Isolated learning | $0.9997\pm 0.0002$ | $0.999\pm 0.001$ | $0.999\pm 0.001$ | $0.9842\pm 0.0382$ | $0.9955\pm 0.0101$
Isolated learning (noise) | $0.8463\pm 0.033$ | $0.8737\pm 0.0127$ | $0.896\pm 0.0133$ | $0.7825\pm 0.0754$ | $0.8496\pm 0.0034$
Scenario 1 (noise) | $--$ | $0.8919\pm 0.0174$ | $--$ | $--$ | $--$
Scenario 2 (noise) | $0.9613\pm 0.0181$ | $--$ | $--$ | $--$ | $--$
Scenario 3 (noise) | $--$ | $--$ | $0.9137\pm 0.0135$ | $--$ | $--$
Scenario 4 (noise) | $0.9636\pm 0.0172$ | $--$ | $--$ | $--$ | $--$
Scenario 5 (noise) | $--$ | $--$ | $--$ | $0.4322\pm 0.0933$ | $--$
Scenario 6 (noise) | $0.7035\pm 0.0364$ | $--$ | $--$ | $--$ | $--$
Table 1: Mean $R^{2}$, and their standard deviations, of six transfer
scenarios on a toy example of four functions: $y_{1}=-3x+10$, $y_{2}=-3x-5$,
$y_{3}=-6x-12$ and $y_{4}=x^{2}$ ($--$ denotes not applicable).

##  8 Discussion and Conclusion

We proposed a theory for knowledge transfer in supervised continual learning.
We aim to encourage further research in knowledge transfer for achieving
increasingly knowledgeable continual learning systems. Our proposed framework
relies on the assumption of relatedness among tasks in a specific environment.
This assumption may be applicable to a variety of domains that learn different
but related tasks, e.g. tasks in a clinical domain, tasks in manufacturing,
etc. Our error bounds are agnostic to the implementation of the continual
learner. One would naturally wonder how these bounds apply to implementations
with deep neural networks. Since those bounds depend on the number of examples
per task, which itself depends on the number of tasks (see Eq. 13), it is
possible that some data from previous tasks will be needed. Strategies such as
memories per task or generative models, which have been used in several
studies, could be helpful. We also believe that modular or semi-modular
networks, with specialized components for each task, will potentially be
necessary for effective knowledge transfer. Furthermore, having specialized
modules for representing the set of transformations between hypothesis spaces
for each task could be helpful. We also hypothesise that scenarios such as
class-incremental learning of related classes could benefit more from transfer
than scenarios such as task or domain-incremental learning, although more
research is required in this direction.

Recent work suggests that the framework of the VC-dimension is not appropriate
for deep neural networks. Other frameworks based on infinite-width networks
(Golikov, 2020) and robustness-based networks (Bubeck et al., 2021) have been
widely studied for deep neural networks that observe examples of all tasks at
once. Although most work in supervised continual learning has used deep neural
networks, the infinite-width and robustness-based frameworks have not been
analysed with the lens of learning incrementally. Intuitively,
overparameterised networks would imply a bigger challenge for continual
learning, as this overparameterisation would lead to excellent performance on
a particular task, making the network harder to adapt to subsequent ones.
Extending studies on infinite-width or robustness-based networks to the
challenges of continual learning would be an interesting avenue of research.

## References

  * Baxter et al. (2000) Jonathan Baxter et al.  A model of inductive bias learning.  _Journal of Artificial Intelligence Research_ , 12(149-198):3, 2000.
  * Ben-David & Borbely (2008) Shai Ben-David and Reba Schuller Borbely.  A notion of task relatedness yielding provable multiple-task learning guarantees.  _Machine learning_ , 73(3):273–287, 2008.
  * Benavides-Prado (2020) Diana Benavides-Prado.  Beyond catastrophic forgetting in continual learning: An attempt with svm.  In _ICML_. The International Conference on Machine Learning (ICML), 2020.
  * Benavides-Prado et al. (2017) Diana Benavides-Prado, Yun Sing Koh, and Patricia Riddle.  Accgensvm: Selectively transferring from previous hypotheses.  In _Proc. Intern. Joint Conf. Artificial Intel_ , pp. 1440–1446, 2017.
  * Benavides-Prado et al. (2020) Diana Benavides-Prado, Yun Sing Koh, and Patricia Riddle.  Towards knowledgeable supervised lifelong learning systems.  _Journal of Artificial Intelligence Research_ , 68:159–224, 2020.
  * Bubeck et al. (2021) Sébastien Bubeck, Yuanzhi Li, and Dheeraj M Nagaraj.  A law of robustness for two-layers neural networks.  In _Conference on Learning Theory_ , pp. 804–820. PMLR, 2021.
  * Chen & Liu (2018) Zhiyuan Chen and Bing Liu.  Lifelong machine learning.  _Synthesis Lectures on Artificial Intelligence and Machine Learning_ , 12(3):1–207, 2018.
  * Delange et al. (2021) Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2021.
  * Golikov (2020) Eugene Golikov.  Towards a general theory of infinite-width limits of neural classifiers.  In _International Conference on Machine Learning_ , pp. 3617–3626. PMLR, 2020.
  * Hung et al. (2019) Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen.  Compacting, picking and growing for unforgetting continual learning.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Ke et al. (2020) Zixuan Ke, Bing Liu, and Xingchang Huang.  Continual learning of a mixed sequence of similar and dissimilar tasks.  _Advances in Neural Information Processing Systems_ , 33:18493–18504, 2020.
  * Ke et al. (2021) Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu.  Achieving forgetting prevention and knowledge transfer in continual learning.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ , pp. 201611835, 2017.
  * Knoblauch et al. (2020) Jeremias Knoblauch, Hisham Husain, and Tom Diethe.  Optimal continual learning has perfect memory and is np-hard.  In _International Conference on Machine Learning_ , pp. 5327–5337. PMLR, 2020.
  * Lampinen & Ganguli (2018) Andrew K Lampinen and Surya Ganguli.  An analytic theory of generalization dynamics and transfer learning in deep linear networks.  _arXiv preprint arXiv:1809.10374_ , 2018.
  * Lee & Lee (2020) Cecilia S Lee and Aaron Y Lee.  Clinical applications of continual learning machine learning.  _The Lancet Digital Health_ , 2(6):e279–e281, 2020.
  * Lee et al. (2021) Sebastian Lee, Sebastian Goldt, and Andrew Saxe.  Continual learning in the teacher-student setup: Impact of task similarity.  In _International Conference on Machine Learning_ , pp. 6109–6119. PMLR, 2021.
  * Lesort et al. (2020) Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia Díaz-Rodríguez.  Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges.  _Information fusion_ , 58:52–68, 2020.
  * Lopez-Paz & Ranzato (2017) David Lopez-Paz and Marc’Aurelio Ranzato.  Gradient episodic memory for continual learning.  In _Advances in Neural Information Processing Systems_ , pp. 6467–6476, 2017.
  * Maschler et al. (2021) Benjamin Maschler, Thi Thu Huong Pham, and Michael Weyrich.  Regularization-based continual learning for anomaly detection in discrete manufacturing.  _Procedia CIRP_ , 104:452–457, 2021.
  * New et al. (2022) Alexander New, Megan Baker, Eric Nguyen, and Gautam Vallabha.  Lifelong learning metrics.  _arXiv preprint arXiv:2201.08278_ , 2022.
  * Ostapenko et al. (2019) Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi.  Learning to remember: A synaptic plasticity driven framework for continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 11321–11329, 2019.
  * Raczkowski & Sadowski (1990) Konrad Raczkowski and Paweł Sadowski.  Equivalence relations and classes of abstraction.  _Formalized Mathematics_ , 1(3):441–444, 1990\.
  * Riemer et al. (2018) Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro.  Learning to learn without forgetting by maximizing transfer and minimizing interference.  _arXiv preprint arXiv:1810.11910_ , 2018.
  * Ring (1997) Mark B Ring.  Child: A first step towards continual learning.  _Machine Learning_ , 28(1):77–104, 1997.
  * Rish (2022) Irina Rish.  Towards general and robust ai at scale, 2022.  URL https://www.youtube.com/watch?v=OL3hUZh6lTc.
  * Rostami et al. (2020) Mohammad Rostami, David Isele, and Eric Eaton.  Using task descriptions in lifelong machine learning for improved performance and zero-shot transfer.  _Journal of Artificial Intelligence Research_ , 67:673–704, 2020.
  * Van de Ven & Tolias (2019) Gido M Van de Ven and Andreas S Tolias.  Three scenarios for continual learning.  _arXiv preprint arXiv:1904.07734_ , 2019.
  * van de Ven et al. (2020) Gido M van de Ven, Hava T Siegelmann, and Andreas S Tolias.  Brain-inspired replay for continual learning with artificial neural networks.  _Nature communications_ , 11(1):1–14, 2020.
  * Vogelstein et al. (2020) Joshua T Vogelstein, Jayanta Dey, Hayden S Helm, Will LeVine, Ronak D Mehta, Ali Geisa, Haoyin Xu, Gido M van de Ven, Emily Chang, Chenyu Gao, et al.  Representation ensembling for synergistic lifelong learning with quasilinear complexity.  _arXiv preprint arXiv:2004.12908_ , 2020.
  * Weinshall et al. (2018) Daphna Weinshall, Gad Cohen, and Dan Amir.  Curriculum learning by transfer learning: Theory and experiments with deep networks.  In _International Conference on Machine Learning_ , pp. 5238–5246. PMLR, 2018.
  * Yoon et al. (2017) Jaehong Yoon, Eunho Yang, et al.  Lifelong Learning with Dynamically Expandable Networks.  _arXiv:1708.01547_ , 2017.
  * Zeng et al. (2019) Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu.  Continual learning of context-dependent processing in neural networks.  _Nature Machine Intelligence_ , 1(8):364–372, 2019.
  * Zhang & Yeung (2014) Yu Zhang and Dit-Yan Yeung.  A regularization approach to learning task relationships in multitask learning.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ , 8(3):1–31, 2014.

## Appendix A

Proof for Corollary 17:

###### Proof.

By Theorem 2 in Ben-David and Borbely, at time $n+z$:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}_{n+z}([\mathpzc{h}^{*}_{n+z}])\leq\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n+z}}\in[\mathpzc{h_{n+z}^{*}}]}\dfrac{1}{n+z}\sum_{i=1}^{n+z}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h_{i}})+\epsilon_{n+z}\end{gathered}$
(44)

while, at time $n$:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}_{n}([\mathpzc{h}^{*}_{n}])\leq\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h_{n}^{*}}]}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h_{i}})+\epsilon_{n}\end{gathered}$
(45)

And by Ben-David and Borbely (2008), and also Baxter (2000):

$\begin{gathered}\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n+z}}\in[\mathpzc{h_{n+z}^{*}}]}\dfrac{1}{n+z}\sum_{i=1}^{n+z}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\leq\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h_{n}^{*}}]}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h_{i}})\end{gathered}$
(46)

Therefore:

$\begin{gathered}\mathbb{E}r^{P^{(t)}}_{n+z}([\mathpzc{h}^{*}_{n+z}])\leq\mathbb{E}r^{P^{(t)}}_{n}([\mathpzc{h}^{*}_{n}])+\epsilon_{n}+\epsilon_{n+z}\end{gathered}$
(47)

With $\mathbb{E}r^{P^{(t)}}_{n+z}(\mathpzc{h}^{\diamond}_{n+z})$ the best
hypothesis in $\mathbb{E}r^{P^{(t)}}_{n+z}([\mathpzc{h}^{*}_{n+z}])$ and
$\mathbb{E}r^{P^{(t)}}_{n}(\mathpzc{h}^{\diamond}_{n})$ the best hypothesis in
$\mathbb{E}r^{P^{(t)}}_{n}([\mathpzc{h}^{*}_{n}])$, the theorem is proved. ∎

## Appendix B

Proof for Corollary 30:

###### Proof.

By Theorem 2 in Ben-David and Borbely, at time $n+1$:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}_{n+1}([\mathpzc{h}^{*}_{n+1}])\leq\inf_{\mathpzc{h}^{(s)},\mathpzc{h}^{(t)}_{n},\mathpzc{h}^{(t)}_{n+1}\in[\mathpzc{h_{n+1}^{*}}]}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}_{n}}(\mathpzc{h}^{(t)}_{n})+\mathbb{\hat{E}}r^{S^{(t)}_{n+1}}(\mathpzc{h}^{(t)}_{n+1})\big{)}+\epsilon_{n+1}\end{gathered}$
(48)

while, at time $n$:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}_{n}([\mathpzc{h}^{*}_{n}])\leq\inf_{\mathpzc{h}^{(s)},\mathpzc{h}^{(t)}_{n}\in[\mathpzc{h_{n}^{*}}]}\big{(}\mathbb{\hat{E}}r^{S^{(s)}}(\mathpzc{h}^{(s)})+\mathbb{\hat{E}}r^{S^{(t)}_{n}}(\mathpzc{h}^{(t)}_{n})\big{)}+\epsilon_{n}\end{gathered}$
(49)

And by Ben-David and Borbely (2008), and also Baxter (2000):

$\begin{gathered}\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n+1}}\in[\mathpzc{h_{n+1}^{*}}]}\dfrac{1}{n+1}\sum_{i=1}^{n+1}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\leq\inf_{\mathpzc{h}_{1},\ldots,\mathpzc{h_{n}}\in[\mathpzc{h_{n}^{*}}]}\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{\hat{E}}r^{S^{(s)}_{i}}(\mathpzc{h}_{i})\end{gathered}$
(50)

Therefore:

$\begin{gathered}\mathbb{E}r^{P^{(s)}}_{n+1}([\mathpzc{h}^{*}_{n+1}])\leq\mathbb{E}r^{P^{(s)}}_{n}([\mathpzc{h}^{*}_{n}])+\epsilon_{n}+\epsilon_{n+1}\end{gathered}$
(51)

With $\mathbb{E}r^{P^{(s)}}_{n+1}(\mathpzc{h}^{\diamond}_{n+1})$ the best
hypothesis in $\mathbb{E}r^{P^{(s)}}_{n+1}([\mathpzc{h}^{*}_{n+1}])$ and
$\mathbb{E}r^{P^{(s)}}_{n}(\mathpzc{h}^{\diamond}_{n})$ the best hypothesis in
$\mathbb{E}r^{P^{(s)}}_{n}([\mathpzc{h}^{*}_{n}])$, the theorem is proved. ∎