# Learning to Transfer: A Foliated Theory

Janith Petangoda<EMAIL_ADDRESS>
Department of Computing
Imperial College London
London, United Kingdom Marc Peter Deisenroth<EMAIL_ADDRESS>
Centre for Artificial Intelligence
University College London
London, United Kingdom Nicholas A. M. Monk<EMAIL_ADDRESS>
School of Mathematics and Statisitics
University of Sheffield
Sheffield, United Kingdom

###### Abstract

Learning to transfer considers learning solutions to tasks in a such way that
relevant knowledge can be transferred from known task solutions to new,
_related_ tasks. This is important for general learning, as well as for
improving the efficiency of the learning process. While techniques for
learning to transfer have been studied experimentally, we still lack a
foundational description of the problem that exposes what related tasks are,
and how relationships between tasks can be exploited constructively. In this
work, we introduce a framework using the differential geometric theory of
foliations that provides such a foundation.

Keywords: Differential Geometry, Foliations, Transfer Learning, Multitask
Learning, Meta Learning

##  1 Introduction

In human learning, transfer is key to the efficiency in learning that we
exhibit (Ellis, 1965; Schunk, 2012). Transfer allows us to apply knowledge
from past experiences when learning solutions to new problems, and thereby
improve the efficiency of this learning; this applies to both physical and
intellectual activities. We know, for example, that knowing how to walk
_informs_ us about how to run, and that abstract mathematical thinking can
apply to many, varied fields. Learning to transfer in machine learning (ML) is
the application of such transfer in the types of learning problems we apply ML
methods to.

Learning to transfer in ML is not a new concept; initial discussions have
appeared perhaps since the 1980s and 1990s in work such as Mitchell (1980);
Schmidhuber (1995). The inspiration for learning using transfer almost
certainly stemmed from noticing transfer in human learning (Ellis, 1965;
Schunk, 2012). Transfer can be thought of as a form of generalisation;
Mitchell (1980) describes the need for biases in generalisation in single task
problems. The type of transfer that we consider is generalisation between
tasks. Work in the field progressed through suggestions for lifelong learning
(Thrun and Mitchell, 1994, 1995), multi-task learning (Caruana, 1997; Silver
and Mercer, 1996), and learning internal representations of biases, and
inductive bias transfer (Baxter, 1995, 2000). An idea of meta-learning, or
learning to learn, was written by Schmidhuber (1995); Schmidhuber et al.
(1996). More modern approaches that have built on these works include advances
in deep learning based meta-learning (Vilalta and Drissi, 2002; Hospedales et
al., 2020), few-shot learning (Wang et al., 2020), deep multitask learning
(Ruder, 2017) and neural architecture search (Elsken et al., 2019). In these,
transfer allows us to learn using less data, computational resources, and time
by leveraging relevant information. In the case of multitask learning, it can
also lead to better generalisation (Caruana, 1997).

An exemplary ML problem where transfer can be applied to is shown in Figure 1.
Suppose we have learned how to swing a simple pendulum $p_{1}$ from its
bottom-most position to its top-most position using clamped torques applied at
the pivot; see Figure 1. This control strategy is denoted $\pi_{1}$. The
defining characteristics of this particular pendulum are:

  * •

it is a pendulum,

  * •

it has a rod of length $l_{1}$, and

  * •

it has a point mass of $m_{1}$ at the end of this rod.

Further suppose that we are given a new pendulum $p_{2}$ with a rod of length
$l_{2}\neq l_{1}$, and a point mass of $m_{2}\neq m_{1}$. That is, when
compared to the first pendulum, $p_{2}$ has the same characteristics listed
previously, and is _only_ different in length and mass. Thus, if we wanted to
learn a control policy $\pi_{2}$ that can swing up $p_{2}$, it is intuitive to
assume that there is _something_ about the learned policy of the first,
related pendulum $p_{1}$ that can be used to make finding such a policy easier
than learning from scratch.

Figure 1: An example of a transfer problem. Having trained a policy $\pi_{1}$,
which can swing up a pendulum with mass $m_{1}$ and length $l_{1}$ (left
pane), we want to learn a policy $\pi_{2}$ for a new pendulum with properties
$m_{2}$ and $l_{2}$ (right pane).

This is _transfer_ , and _learning to transfer_ is to learn $\pi_{1}$ in such
a way that learning $\pi_{2}$ can exploit the _relatedness_ of $p_{1}$ and
$p_{2}$. To reiterate, we are given tasks that are, in some sense of the word,
_related_ to each other. Thus, we expect that the solutions to such tasks are
also _related_ to each other in a similar sense. We would then like to exploit
this relationship when learning a solution to one problem once a solution to
the other, related problem is known. We expect that this exploitation will
make the learning of the second solution easier than if we were to attempt
learning without it. Learning to transfer, therefore, is the process of
identifying such a relationship between tasks, and applying it to learning
related solutions.

There are many methods in the literature that can solve such problems (Finn et
al., 2017; Sæmundsson et al., 2018; Petangoda et al., 2019). Specific works,
e.g., by Finn et al. (2017) and Sæmundsson et al. (2018), have been shown to
be very powerful. However, most modern works are focused on obtaining working
models and algorithms, rather than developing an understanding of the
necessary structural and elementary components of learning to transfer. While
the works mentioned above have shown wonderful and useful properties, it is
not clear _why_ they work the way they do, nor even a good way of quantifying
in _what ways_ they do work well. Having a concrete description of how
transfer can be achieved allows us to measure how well it can be done; for
example, how can we measure how easily transfer is possible? The practical
consequences of lacking such an understanding is that we cannot _deliberately_
adjust our techniques and approaches for application in novel and/or more
complex problems.

This is where this article comes into play. This article provides a unifying
framework for approaching problems of learning to transfer. In particular, we
pose the transfer problem in terms of a mathematical framework that describes
the structural building blocks of learning to transfer.

As discussed, transfer involves leveraging a _known relationship_ between
tasks to find solutions that are similarly related. We will isolate the
relevant properties of such a relationship, and characterise them
mathematically. In particular, we will see that this naturally leads us to the
differential geometric notion of a foliation. Following this, we will show
that some well known classes of models for transfer learning implicitly make
use of this structure.

The key goal of this article is to introduce this foliated theory of learning
to transfer as a useful point of view of the problem. The framework aims to
provide a conceptual guideline for thinking about problems of transfer, and to
allow researchers and thinkers in the field to answer questions that include:

  1. 1.

What does it mean for tasks or problems to be _related_?

  2. 2.

What does it mean to transfer between related tasks?

  3. 3.

How can the relationship between tasks be represented?

  4. 4.

Can we learn this relationship? How similar are they?

We will begin by describing how the familiar learning of a single task can be
stated within the language of our framework. In Sections 3 and 4, we introduce
our framework, and detail its components respectively. We will then exposit an
in-depth discussion that will elaborate the breadth and impact of our
framework in the final sections.

##  2 Learning a Single Task

At the lowest level of a learning to transfer problem, there is the problem of
learning a single task. We begin our explanation of the framework by
describing this familiar learning paradigm.

###  2.1 Learning Task

A learning task is meant to encompass what we generally think of as a single
learning problem in machine learning. This can include supervised learning,
unsupervised learning, reinforcement learning (RL), among others. While
traditional definitions describe a learning task in terms of generative
distributions, we choose an equivalent definition that focusses on structure:

######  Definition 1 (Learning Task)

A learning task is a single machine learning problem. The problem will consist
of some structure $\mathfrak{S}$, which can be used by a generative process
$\mathfrak{P}:\mathfrak{S}\rightarrow\mathcal{F}_{\mathfrak{D}}$ to generate a
dataset $D\in\mathfrak{D}$ via $f\in\mathfrak{P}(\mathfrak{S})$.
$(f\in\mathcal{F}_{\mathfrak{D}}):\Omega\times\mathcal{Z}^{+}\rightarrow\mathfrak{D}$
is a map that takes a positive integer $n$ and some other inputs $\Omega$ to
return the dataset of size $n$. The learning task is a subset of
$\mathfrak{S}$ that we would like to learn. We will denote a learning task by
$t$.

In this definition, the generative process is the _mechanism_ by which data
can be generated; it defines the class of problems we are dealing with. For
example, in supervised learning, this process is the application of a
generative function; in RL the process involves rolling out the full Markov
Decision Process (MDP) that samples a trajectory given the system’s transition
dynamics, reward function and policy.

The structure $\mathfrak{S}$ then contains everything else that is required to
specify the particular instance of the relevent generative process from a set
of all possible such processes. As we will describe in Section 8.1,
$\mathfrak{S}$ is often a set of maps that satisfy particular properties; the
learning task then, is the subset of such maps and their properties that we
want to learn, while the rest can be thought of as an inductive bias that is
assumed. The structure of a problem can be described in different ways; a
useful description of structure is in terms of a hierarchy of biases that, at
each level, creates smaller sets from the set of _all_ things we want to
consider, until we are able to uniquely identify the task.

To see this, consider carrying out supervised learning on data generated from
a sinusoid $y(x)=\sin(x)$. In writing the previous statement, we have already
described the structure of the problem; it is a function
$y:\mathbb{R}\rightarrow\mathbb{R}$ that can be written as $y=\sin(x)$.
Alternatively, we could also describe $y=\sin(x)$ as a continuous, odd, scalar
valued, circular function with domain $\mathbb{R}$ that is periodic with
period $2\pi$, and has a constant amplitude of $1$. We can rewrite this as a
hierarchy of subsets, starting from the largest, set of maps $\rightarrow$
maps on $\mathbb{R}$ $\rightarrow$ scalar valued maps $\rightarrow$ continous
functions $\rightarrow$ odd functions $\rightarrow$ circular functions
$\rightarrow$ constant period of $2\pi$ $\rightarrow$ constant amplitude of
$1$. Note that each subsequent level inherits the properties of the previous
level. We will see in Section 4.2 that writing structure in this way is
crucial to transfer.

###  2.2 A Space of Models

Machine learning often involves finding a representation of an aspect of the
generative process described above (see Section 8.2). Thus, in order to carry
out learning, we need to define a model space from which to choose a suitable
representation. The model space can be thought of as a way of representing the
task space, and the structure that it carries. In the present work, we will
consider parametric model spaces:

######  Definition 2 (Parametric Model Space)

Given an input space $\mathcal{X}$, an output space $\mathcal{Y}$ and a
parameter space $\Theta$, a model architecture is given by
$\phi:\mathcal{X}\times\Theta\rightarrow\mathcal{Y}$. A parametric model space
$\mathcal{P}_{\mathcal{X},\mathcal{Y},\Theta,\phi}=\\{\phi(\theta,\cdot)\\}$,
$\forall\theta\in\Theta$.

The parameteric model space
$\mathcal{P}_{\mathcal{X},\mathcal{Y},\Theta,\phi}$ is therefore a collection
of functions that can be indexed by the parameter space, under a given model
architecture. Note that the model architecture is a structure that applies to
the entire model space; in choosing an appropriate architecture, we have
implicitly assumed that the learning task $t$ at hand also carries such a
structure.

The parametric model space $\mathcal{P}_{\mathcal{X},\mathcal{Y},\Theta,\phi}$
is isomorphic to its parameter space $\Theta$, which is assumed to be
$\mathbb{R}^{d}$. We will assume that $\Theta$ is a smooth manifold. It should
be noted that there is an identifiability issue here; there can be several
elements of $\mathcal{P}_{\mathcal{X},\mathcal{Y},\Theta,\phi}$ that are
functionally equivalent, but with distinct parameters. For example, suppose
that $\phi(\theta,x)=\sin(x+\theta)$; due to the periodicity of the sinusoid,
we know that $\phi(\theta,x)=\phi(\theta+2\pi n)$, $\forall x\in\mathbb{R}$
and $n\in\mathbb{Z}$, the set of integers. We will therefore define a model
space as follows:

######  Definition 3 (Model Space)

Given a parametric model space
$\mathcal{P}_{\mathcal{X},\mathcal{Y},\Theta,\phi}$, the model space
$\mathcal{M}$ is the quotient space defined as $\mathcal{M}=\Theta/\sim$,
where $\sim$ defines an equivalence relation given by:

$\theta_{1}\sim\theta_{2}\iff\phi(x,\theta_{1})=\phi(y,\theta_{2}),\>\forall
x\in\mathcal{X}\>\mathrm{and}\>\theta_{1},\theta_{2}\in\Theta$ (1)

We will denote the model space as $\mathcal{M}$.

In general, we can consider other equivalences too. For example, two
parameters could be equivelent if the loss value they produce is the same;
this means that our learning algorithm cannot differentiate between these.
Thus, we can define a general model space to be the quotient space w.r.t an
equivelence between the learning algorithm. The construction of such a space
is beyond the scope of the present work. In most practical cases, and for the
purposes of the present paper, it suffices to look at local regions of the
space; that is open neighbourhoods taken from the topology on the manifold,
and therefore, local structures, provided that $\mathcal{M}$ can be described
as a smooth manifold (which we assume111proving that the quotient space
described here is a manifold is beyond the scope of the present work, and will
be left for future work.).

###  2.3 Learning Algorithm

Given a task, and a model space, we can define a learning algorithm as:

######  Definition 4 (Learning Algorithm)

Given a model space $\mathcal{M}$, a task space $\mathcal{T}$, and a set of
_relevant algorithmic properties_ $\Omega$, the learning algorithm
$\mathfrak{L}$ is a map
$\mathfrak{L}:\\{t\\}\times\Omega\rightarrow\mathcal{M}$, such that
$\mathfrak{L}(t,\omega)=m_{t}^{*}$ for some set of algorithm parameters
$\omega\in\Omega$, is the _most suitable_ model in $\mathcal{M}$ for the
learning task $t\in\mathcal{T}$.

The learning algorithm $\mathfrak{L}$ is the mechanism by which we find a
suitable representation of the learning task. Often in machine learning, we
measure the suitability of a model for a particular task using a loss
function, defined as:

######  Definition 5 (Loss Function)

Given a learning task $t$, and model space $\mathcal{M}$, a loss function
$\mathcal{L}_{t}$ is a map $\mathcal{L}_{t}:\mathcal{M}\rightarrow[0,\infty]$.

Note here, that the loss function is defined for a particular learning task
$t$. It is often assumed that $\mathcal{M}$ contains at least a single model
$m_{t}^{*}$ such that $\mathcal{L}_{t}(m_{t}^{*})<\epsilon\in\mathbb{R}^{+}$,
for a desired accuracy $\epsilon$. The learning algorithm is a map from a
learning task to the most suitable model, in this sense. The set of
algorithmic parameters are hyperparameters that define the learning algorithm.
The loss function can be included in this set, as well as the optimiser that
is used, to name another example.

###  2.4 The Problem of Learning a Single Task

Putting these altogether, we make the following definition:

######  Definition 6 (The Problem of Solving a Single Learning Task)

Suppose we are given a learning task $t$. The problem of solving a single
learning task is to

  1. 1.

construct the model space $\mathcal{M}$ that contains a sufficiently suitable
description of the structure defining $t$.

  2. 2.

construct a loss function $\mathcal{L}_{t}:\mathcal{M}\rightarrow[0,\infty]$
that appropriately measures the suitability of $m\in\mathcal{M}$ as a
description of $t$,

  3. 3.

and devise a learning algorithm $\mathfrak{L}$ that maps $t$ to
$m_{t}^{*}=\operatorname*{arg\,min}_{m\in\mathcal{M}}\mathcal{L}_{t}(m)$ is
the most suitable model in $\mathcal{M}$, according to $\mathcal{L}$.

A graphical representation of this problem is shown in Figure 2.

Figure 2: Learning a single task. Here, $m^{*}_{t}$ represents the optimal
solution to the learning task.

##  3 Learning to Transfer

The formal definition of learning to transfer, as defined within our framework
is as follows; we will elaborate on its many components subsequently:

######  Definition 7 (The Problem of Learning to Transfer)

Suppose there is a space of learning tasks $\mathcal{T}$, represented a smooth
manifold. From a subset of tasks $U_{\mathcal{T}}$, we are given a finite
number of learning tasks. Using these, the problem of learning to transfer is
to

  1. 1.

construct the model space $\mathcal{M}$ that contains sufficiently suitable
descriptions of $U_{\mathcal{T}}$,

  2. 2.

choose a notion of (or a class of) $\Pi_{\mathcal{T}}$-relatedness on
$\mathcal{T}$ and the corresponding, homomorphic $\Pi_{\mathcal{M}}$ on
$\mathcal{M}$,

  3. 3.

and devise a learning to transfer algorithm $\mathfrak{LT}$ that produces a
learning algorithm $\mathfrak{L}$ that is equivariant w.r.t. the chosen, or
_in some sense_ optimal $\Pi_{\mathcal{T}}$,

such that they allow for biased learning of subsequent tasks
$t_{\scriptstyle{new}}\in U_{\mathcal{T}}$.

This definition is quite general. It has several components that allow for
choice by the designer of a learning to transfer system. In particular, it
allows us to encompass the many different ways in which the literature has
approached transfer. See Section 5 for more details. A graphical depiction of
this is shown in Figure 3.

Figure 3: Learning to transfer. Here, we have chosen a notion of relatedness,
as made precise in Section 4.2. Here, we assume that we are initially given
some tasks, denoted by the black dots; we choose a notion of relatedness that
applies to both $\mathcal{T}$ and $\mathcal{M}$. The learning to transfer
algorithm $\mathfrak{LT}$ then generates a learning algorithm $\mathfrak{L}$
that is equivariant w.r.t. the chosen notion of relatedness.

##  4 The Structure of Learning to Transfer

We can now describe our framework for learning to transfer.

###  4.1 The Space of Learning Tasks

If we compare Definition 6 of the problem of learning a single task, to that
of learning to transfer, as given by Definition 7, an important difference is
that in the latter, we look at spaces of tasks, rather than just a single
task.

######  Definition 8 (Task Space)

A task space is a collection of unique learning tasks.

A task space simply contains learning tasks. In particular, this space is
simply a set that does not have any other useful structure that is attached to
it, other than the tasks that it contains. In this subsection, we argue that
it is useful to think about the task space as a smooth manifold. Formal
definitions of what this is will be given in the Appendices.

A topological manifold (Lee, 2001) is a generalisation of a topological
Euclidean space; the global topology of a manifold can be different from a
Euclidean space, but locally it would look like a subset of one. An example of
this is the circle manifold $\mathbb{S}^{1}$, which is a 1-dimensional
topological manifold.

Figure 4: A topological manifold $M$ with examples of its coordinate charts.
We see here that despite being a complicated topological entity (see the hold
in the middle), locally, it is equivalent to $\mathbb{R}^{2}$. Further, note
that the chart transition map
$\phi\circ\psi^{-1}:V_{\mathbb{R}^{2}}\rightarrow U_{\mathbb{R}^{2}}$ exists
because a homeomorphism is invertible and continuous, since the composition of
continuous maps remains so.

An important aspect of a topological manifold are its local coordinates. These
are homeomorphisms from local, open neighbourhoods of the manifold on to a
fixed dimensional Euclidean space; these maps are called charts, and the
collection of charts covering the manifold is called an atlas. Any particular
neighbourhood can be given several different, yet valid charts. In fact, there
can be regions that intersect with different charts; on a manifold, the map
between the charts over the intersected region is also homeomorphism (see
Figure 4).

These charts can be thought of as a representation scheme (see Section 8.2 for
more details) of the tasks. The dimensionality of the manifold describes the
number of degrees of freedom we have to move in to describe tasks in the
constrained set of tasks $\mathcal{T}$. This is opposed to a universal set of
tasks that contains _all possible_ learning tasks; we believe that this is too
general to consider222There does exist an obvious counter example to this
however: a reasonable set of tasks that one could consider in a regression
task is the set of all continuous functions. This is technically _infinite_
dimensional. While we believe that it is possible that our theory could be
extended to this case, we will restrict ourselves to the finite dimensional
scenario for the time being.. The charts then describe local coordinate
systems that _quantify_ these degrees of freedom. However, the particular
chosen charts themselves do not matter. Theories on manifolds are coordinate
independent.

This is useful for thinking about task spaces and learning tasks, since we do
not want to have to worry about how to represent a task. We want to consider
tasks as abstract objects, that exist with particular structure. This also
extends to the full space of tasks. The manifold structure allows us to think
about this.

The finiteness of the task space could be thought of as pathological; for
example, it is reasonable to consider the space of all continuous functions as
a task space. However, we propose that given some useful structure, and
therefore bias, we will often find subspaces of this full set which are indeed
finite dimensional, and form a topological manifold. Certainly, it is possible
to find such subspaces, and to therefore restrict our attention to solving
learning tasks from such spaces. We mentioned in Section 2.1 that the elements
of a task space themselves provide the most fundamental level of structure,
since they are chosen, from the set of all possible tasks to belong to the
given task space $\mathcal{T}$. For example, if we define the task space to be
all regression tasks generated from scalar valued truncated Fourier series on
$\mathbb{R}$, we immediately note there is a constraint (and therefore
structure) that defines the task space.

Finally, a topological manifold, as the name would suggest, must contain a
topology on the set of things on the manifold. A topology gives us a concrete
notion of neighbourhoods of elements; elements that are close to each
other333It should be noted that while we used the term _close_ here, a
topology does not require a metric, or a notion of distance, although a metric
topology can be defined from one. A topology merely tells us about which
points are near each other.. A topology is a very basic structure that is
placed on a set of things, in this case tasks. It allows us to start
organising our space of tasks.

To be a topological manifold, $\mathcal{T}$ requires there be a topology on
the task space. In our framework, we would assume a reasonable topology on the
task space, where neighbourhoods are formed by tasks we expect to be close to
similar to each other. However, in practice, specifying a loss function and
model space can generate a topology on the task space. For example, we could
choose any topology for which the loss function is continuous w.r.t. the task
space. An example of such a topology is the following:

######  Theorem 9 (Induced Topology on Task Space)

We are given a task space $\mathcal{T}$, a model space $\mathcal{M}$, a loss
function $L:\mathcal{T}\times\mathcal{M}\rightarrow[0,\infty]$ and a learning
algorithm $\mathfrak{L}_{L}:\mathcal{T}\rightarrow\mathcal{M}$. Assume that
each $L_{t\in\mathcal{T}}:\mathcal{M}\rightarrow[0,\infty]$ is continuous.
Then, a loss ball centered at $t\in\mathcal{T}$ is given by

$\mathcal{B}_{t}(\epsilon)=\\{s\in\mathcal{T}:\>\big{|}\>L(s,\mathfrak{L}(t))-L(t,\mathfrak{L}(t))\>\big{|}\><\epsilon\\}$
(2)

for $\epsilon>0$. A subset $U_{\mathcal{T}}\subseteq\mathcal{T}$ is then an
element of the induced topology $\mathcal{O}_{\mathcal{T}}$ iff $\forall t\in
U_{\mathcal{T}},\>\exists\>\epsilon>0$ s.t.
$\mathcal{B}_{t}(\epsilon)\subseteq U_{\mathcal{T}}$.

The proof that this is a valid topology can be found in Appendix B.1. Note
that the definition of a loss function has been extended compared to
Definition 5, and as far as the rest of the present work is concerned,
redefined, as the following:

######  Definition 10 (Loss Function)

Given a learning task $t$ which comes from a task space $\mathcal{T}$ and
model space $\mathcal{M}$, a loss function $\mathcal{L}$ is a map
$\mathcal{L}:\mathcal{M}\times\mathcal{T}\rightarrow[0,\infty]$.

Theorem 9 defines a topology similar to a metric topology, but defined w.r.t.
a loss function. Here, the open sets of the topology consist of tasks that are
close to each other given the loss function.

A topological manifold endowed with a smooth structure is called a smooth
manifold. This is a requirement that all chart transition maps be smooth. Such
a structure allows us to talk about notions of calculus on a manifold. These
include gradients, vector fields, flows, integration, and many others. As
such, we can think of a task space as a smooth manifold, since notions of
differentiation are useful in ML. For example, to think about gradient descent
is to think about flows along a manifold.

###  4.2 Relatedness

In Definition 1, we made specific mention of structure. The elements that
constitute the task space themselves inform us of a fundamental structure that
identifies this particular set of tasks from the set of all possible tasks.
Knowledge of such constraints would immediately allow us to meaningfully
transfer between elements of this task space. In practice however, we lack
access to such knowledge, and learning to transfer amounts to learning the
structure that defines a subset of a large space of tasks. That is, given a
rather large space of tasks, the structure of which we know of, we want to
find a hierarchy of this space that creates subsets of it, each of which
carries some inductive bias, or structure that defines it, within the original
space of tasks.

Each level of structure describes some inductive bias which identifies a
smaller subset of tasks. We posit that transfer can be then written as
identifying the bias that describes a subset of tasks, and exploiting this
bias to making learning within this subset more efficient. The efficiency
gains can immediately be attributed to the smaller size of each subset. There
is not a unique hierarchy of biases that can be used to describe a learning
task. The choice of such a hierarchy depends on the needs of the learned
system (we will discuss this more in Section 8.4).

Learning to transfer at a chosen level will then involve exploiting this
hierarchy at that level. For this, we are required to define a scheme that can
be used to describe such a hierarchy. We propose two methods that will lead to
a precise notion of relatedness. Consider a space of learning tasks
$\mathcal{T}$. A hierarchy with a single level can be described as a partition
of $\mathcal{T}$. The hierarchy then consists of the identities of each
partition at the highest level, then the learning tasks that belong to each
partition form the lowest level. Then,

  1. a)

we can tessellate $\mathcal{T}$, where we create subsets (or submanifolds)
that are of the same dimension as $\mathcal{T}$.

  2. b)

we can create parallel submanifolds that are of dimensions lower than
$\mathcal{T}$.

Note that the dimensions of the submanifolds need not be equal to each other.
Figure 5 graphically shows examples of these. It is possible to consider
hierarchies that consist of combinations of these partitions.

Figure 5: Comparison of tessellation and parallel spaces. The shaded regions
denote the subsets we are considering in each case. A tessellation creates
disjoint, smaller subsets that are of the same dimension as the original set.
Parallel spaces on the other hand creates disjoint subsets that are of a lower
dimension than the original set.

A key aspect of either partition is that each subset defines an equivalence
relation that uniquely identifies the membership of a particular task in its
partition. That is, given a partition of $\mathcal{T}$,
$\\{U_{i}\\}_{i\in\mathcal{I}}$, then for $t_{1},t_{2}\in\mathcal{T}$,
$t_{1}\sim t_{2}\iff t_{1},t_{2}\in U_{i}$. Thus, we can define a notion of
sets of _related_ tasks as,

######  Definition 11 (Sets of Related Tasks)

Given a set of tasks $\mathcal{T}$, and partition
$\mathcal{P}_{\mathcal{T}}=\\{U_{i}\\}_{i\in\mathcal{I}}$, each
$U_{i}\subseteq\mathcal{T}$ is defined as a set of related tasks.

Then,

######  Definition 12 (A Notion of Transfer)

Given a set of tasks $\mathcal{T}$, a partition
$\mathcal{P}_{\mathcal{T}}=\\{U_{i}\\}_{i\in\mathcal{I}}$ on $\mathcal{T}$
will be called a _notion of transfer_.

Such a notion of transfer allows us to define transfer as a transformation.
Each subset $U_{i}\in\mathcal{P}_{\mathcal{T}}$ is naturally equipped with a
permutation group of transformations that allows us to transform one task to
another within a set of transferable tasks. In the case when $\mathcal{T}$ is
discrete and finite, the permutation group is also discrete and finite; in
other cases, such as when $\mathcal{T}$ can be written as a smooth manifold,
this group can be quite unwieldy. In general, it might be sufficient to
consider smaller set of pseudogroups that reproduce
$\mathcal{P}_{\mathcal{T}}$. Such a set of pseudogroups will be called a
notion of _relatedness_.

######  Definition 13 (A Notion of Relatedness)

Given a set of learning tasks $\mathcal{T}$, a set of groups or pseudogroups
$\Pi_{\mathcal{T}}=\\{\mathcal{G}_{i}\\}_{i\in\mathcal{I}}$ of transformations
which act on subsets of $\mathcal{T}$ is called a _notion of relatedness_ if:

  1. 1.

$U_{i},U_{j}$ are the domains of
$\mathcal{G}_{i},\mathcal{G}_{j}\in\Pi_{\mathcal{T}}$, then $U_{i}\cap
U_{j}=\emptyset$,

  2. 2.

$\bigcup_{i\in\mathcal{I}}{U_{i}}=\mathcal{T}$,

  3. 3.

$\bigcup_{i\in\mathcal{I}}\mathcal{O}_{i}$, where $\mathcal{O}_{i}$ is the set
of orbit $\mathcal{G}_{i}$, is a partition of $\mathcal{T}$. An element of
such a partition is then _a set of related tasks_.

Definitions of groups and pseudogroups are given in Appendix A.3 and Appendix
A.4, respectively. Then, we can define transfer as being:

######  Definition 14 (Transfer)

Given a notion of relatedness $\Pi_{\mathcal{T}}$, _transfer_ from $t_{1}\in
U_{i}\subseteq\mathcal{T}$ to $t_{2}\in U_{i}\subseteq\mathcal{T}$ is the
action of a transformation from $\mathcal{G}_{i}\in\Pi_{\mathcal{T}}$.

Thus, transfer can be precisely thought of as transformations between elements
in a set of related tasks. That is, two tasks are related if they exists in a
set of related tasks. One can induce the sets of related tasks by first
defining a notion of relatedness, or conversely, induce relatedness via a
defined partition of $\mathcal{T}$. $\Pi_{\mathcal{T}}$ was chosen to contain
sets of groups and/or pseudogroups for reasons of consistency; in particular,
such restrictions allow transformations to be agnostic of the order of
transformations taking place. They also ensure that inverse and identity
transformations exist, making our set of transformations intellectually
pleasing and interpretable.

Our notion of relatedness is similar to a definition given in (Ben-David and
Schuller, 2003; Ben-David and Borbely, 2008), where a single group of
transformations act on $\mathcal{T}$; our definition generalises this notion
to include different groups of transformations that can act on subset of
$\mathcal{T}$, but produce the partition as needed.

It also agrees with what we intuitively expect relatedness to mean. In our
introductory example, we wanted to transfer from one pendulum to another; the
two pendulums are _related_ by the fact that they are both pendulums. We could
select a mass and length from $\mathbb{R}^{+}$, and still obtain what we would
classically think of as a pendulum, by plugging them into the differential
equations of a pendulum. With relatedness, we want to capture the notion that
there is a set of properties that distinguish a set of things, and that there
is set of transformations that preserves these properties; one can think of
these as factors of variation that preserve the identify of a set of related
things. Such a set of properties are the inductive biases we had mentioned
previously. Here, these properties are the information encoded in the
differential equations of a pendulum. In Section 7.2, using a particular
notion of relatedness, we can show that there are invariant quantities that
can be thought of as identifying structure of a set of related things.

In particular, knowledge of the invariant properties of a set of related tasks
allows us to identify the tasks in such a set using less information than if
we were to identify them from $\mathcal{T}$; we can assume the knowledge that
is given by the membership of a task in a particular set of related tasks. In
this way, the key benefit of transfer can be seen. A notion of relatedness
provides us with a smaller set of factors of variation to consider when
describing tasks. Another way to see this is to consider that it is possible
to write a single set of transformations that acts transitively on
$\mathcal{T}$. Here, all tasks are related to each other. If we trivially
consider this set to be the group of permutations of $\mathcal{T}$, then, we
can note that such a group would be larger than the group of permutations of a
subset of $\mathcal{T}$.

###  4.3 Representing Relatedness Using Foliations

A foliation is a geometric structure that can be placed on a smooth
manifold444Smoothness is not a necessary condition to introduce a foliation on
a manifold, but we consider it for our present purposes.. In particular, a
foliation can allow us to mathematically write partitions such as those we saw
in Figure 5 on a smooth manifold.

####  4.3.1 Foliations

A foliation is a restriction on the atlas that is given to a smooth manifold.
There are two types of foliations.

######  Definition 15 (Regular Foliation)

If $M$ is a smooth manifold of dimension $d$ with an atlas
$\mathcal{A}_{\mathcal{M}}$, a $n$-dimensional foliation $\mathcal{F}$ of $M$
is a subset of its atlas, that satisfies:

  1. a)

if $(U,\phi)\in\mathcal{F}$, then
$\phi:U\rightarrow(U_{1}\subseteq\mathbb{R}^{d-n})\times(U_{2}\subseteq\mathbb{R}^{n})$,
and

  2. b)

if $(U,\phi),(V,\psi)\in\mathcal{F}$, where $U\cap V\neq\emptyset$, the chart
transition functions, such as $\psi\circ\phi^{-1}:\phi(U\cap
V)\rightarrow\psi(U\cap V)$ has the form of
$\psi\circ\phi^{-1}(x,y)=(h_{1}(x),h_{2}(x,y))$, where $x\in U_{1}$ and $y\in
U_{2}$.

Figure 6: A 1-dimensional regular foliation on $\mathbb{R}^{2}$.

Intuitively speaking, this definition describes the decomposition of a
manifold into non-overlapping connected submanifolds that are of dimension $n$
(Camacho and Neto, 2013). Each such submanifold is called a _leaf_.

A key feature of a regular foliation is that the dimension of the leaves is
constant. In this case, the leaves create a partition of the manifold that
looks similar to how we described parallel spaces. However, a generalisation
of this exists, called a singular foliation (Sussmann, 1973; Stefan, 1974),
which allows us to write tesselations, and combinations of the two styles of
partitioning in the same language.

######  Definition 16 (Singular Foliation (Stefan, 1974))

If $M$ is a smooth manifold of dimension $d$ with an atlas
$\mathcal{A}_{\mathcal{M}}$, a singular foliation is a partition of $M$ into
immersed, connected submanifolds of non-constant dimensions. In particular,
there exists an atlas of _distinguished_ charts, where for each $x\in M$ there
exists a chart $\phi$ such that:

  1. a)

if $x\in U$, then
$\phi:U\rightarrow(U_{1}\subseteq\mathbb{R}^{d-n})\times(U_{2}\subseteq\mathbb{R}^{n})$,
where $n$ is the dimension of the leaf containing $x$, and $U_{1}$ and $U_{2}$
contain $0$,

  2. b)

$\phi(x)=(0,0)$, and

  3. c)

if $L$ is a leaf, then $L\cap\phi^{-1}(U_{1}\times U_{2})=\phi^{-1}(l\times
U_{2})$, where $l=\\{w\in U_{1}:\phi^{-1}(w,0)\in L\\}$.

It should be noted that foliations, in general, can have complicated
topologies and properties on their leaves, and space of leaves (Rosenberg and
Roussarie, 1970; Camacho and Neto, 2013; Lawson Jr, 1974).

####  4.3.2 Relevance to transfer

Suppose we are given a task space $\mathcal{T}$, which we write as a finite
dimensional smooth manifold. We can then define a notion of transfer on this
by defining a foliated structure $\mathcal{F}$, regular or otherwise, on
$\mathcal{T}$. The following theorem shows that, given any foliation, we can
construct a set of pseudogroups of transformations on $\mathcal{T}$ that can
behave as a notion of relatedness.

######  Theorem 17 (Relatedness from $\mathcal{F}$)

Given a leaf $L$ of a foliation $\mathcal{F}$ on a manifold $\mathcal{T}$,
there exists a (pseudo)group of transformations that act transitively on $L$.
Then the collection of such (pseudo)groups over all leaves of $\mathcal{F}$ is
a notion of relatedness.

A proof of this is given in Appendix B.2.

The converse is also possible, where we define a notion of relatedness, and
derive a foliation from it. Foliations, particular singular foliations can be
generated in many ways. For example, this is of great importance to the field
of geometric control theory, where singular foliations are used to describe
solvable families of control strategies (Jurdjevic, 1997). Several results in
the study of singular foliations (Lavau, 2018) were derived for this; in
particular, it has been shown in an Orbit Theorem (Jurdjevic, 1997), that
orbits of families of vector fields, under certain conditions generate
singular foliations. Since we can use such families of vector fields to
generate (local) diffeomorphisms, they can give us a notion of relatedness.
More simply however, it is known that the locally free action of a Lie group
produces a regular foliation (Camacho and Neto, 2013; Lawson Jr, 1971).

Thus, using foliations, we can go either way. If a notion of relatedness that
is thought to be relevant to a particular problem of learning to transfer is
known, we can apply it to generate a structure on a task space that reflects
the induced notion of transfer. Alternatively, perhaps, a foliation can be
learned, or simply defined, from which a notion of relatedness can be derived,
for purposes of interpretability and understanding.

###  4.4 Equivariant Learning Algorithms

So far, we have defined a notion of relatedness on $\mathcal{T}$. Since we
want the model space to reflect our assumptions about $\mathcal{T}$ (see
Section 8.2), we would want there to be a notion of relatedness that is
_similar_. Here, we define this similarity to be group, or pseudogroup
homomorphisms, given the notion of relatedness. That is, given
$\Pi_{\mathcal{T}}$, we would want $\Pi_{\mathcal{M}}$ to contain elements
that are homomorphic to the elements of $\Pi_{\mathcal{T}}$; the simplest way
to achieve this is to assume that they are the same, and that $\mathcal{T}$
and $\mathcal{M}$ are also the same.

Following this, it is natural to expect that an appropriate learning
algorithm, which is able to transfer, ensures that sets of related tasks are
mapped to sets of related models (for this, simply replace $\mathcal{T}$ with
$\mathcal{M}$ in Definition 11). This brings us to the final part of
Definition 7 — a learning to transfer algorithm $\mathfrak{LT}$ produces an
equivariant learning algorithm $\mathfrak{L}$. $\mathfrak{L}$ here is as
defined in Section 2.3, but extended to work on $\mathcal{T}$, rather than a
single task.

######  Definition 18 (Equivariance)

Given a group $\mathcal{G}_{M}$ and an action $\theta_{M}$ on a space $M$, a
group $\mathcal{G}_{N}$ and an action $\theta_{N}$ on a space $N$, and a group
homomorphism $\rho:\mathcal{G}_{M}\rightarrow\mathcal{G}_{N}$, we say that a
map $f:M\rightarrow N$ is equivariant if
$f(\theta_{M}(g_{m},m))=\rho(g_{m})(f(m))$, for any $m\in M$ and
$g_{m}\in\mathcal{G}_{M}$.

Equivariance (Olver, 1995) is a property which, not unlike invariance, can be
placed on maps, and transformations that act on the domain and codomain. Here,
instead of the output staying constant when the domain is transformed, we
expect the output to also change in a similar manner. Similar here implies
either that the transformation set is the same, or that there is a structure
preserving map to a transformation set that acts on the codomain. This notion
has been growing in interest in the ML community; for example, it is known
that CNNs are translation equivariant, as visual problems follow similar
properties (Cohen et al., 2018, 2019).

A learning algorithm $\mathfrak{L}$ is equivariant, if given a transformation
$\pi\in\Pi_{\mathcal{T}}$ which acts on $t\in\mathcal{T}$,it it satisfies
$\mathfrak{L}(\pi(t1))=\rho(\pi)\mathfrak{L}(t_{1})$, where
$\rho(\pi)\in\Pi_{\mathcal{M}}$. Thus, the goal of a learning to transfer
algorithm is to produce a learning algorithm that satisfies this property.

In the most general case, choosing a notion of relatedness _a priori_ is
infeasible, and we would want to learn the most suitable notion. In this case,
we can define a class of notions of relatedness (for example, by
parameterising foliations), and a learning to transfer algorithm that
optimises equivariant learning algorithms. Section 8.4 discusses some
preliminary ways by which to measure suitability of a notion of relatedness.

##  5 Relation to Existing Work

In this section, we will show that a lot of work in the literature can come
under the umbrella of our framework; in the following section, we will show
this fit more precisely for a few archetypal examples. We will see that
several problem statements that come with different names describe the same
underlying problem. This problem is what we have described previously, the one
of trying to find structure within a set of tasks. We will see that the
solution that are sought in fields such as meta learning, transfer learning,
continual learning, and the like, can be cast as specific instances under
different assumptions of the key variables of the framework. We will begin
with Meta-Learning.

In Definition 7, we stated that learning to transfer is to equivariantly learn
a foliated structure on the model space; this foliated structure will be
chosen to be suitable for some set of tasks that are trained on. The foliated
structure corresponded to having learnt an inductive bias that informs us
about a set of related tasks that our tasks belong to. Knowledge about the set
of related tasks to which our tasks belong to then allows us to _assume_ this
bias in subsequent tasks that are taken from the same set of related tasks;
this makes learning faster, or easier.

There are several scenarios that we can imagine that this general setup can
generate. In the first instance, let us look at when the set of tasks that are
used at training are available _simultaneously_. This corresponds to the type
of problem that is tackled by multitask learning (Caruana, 1997), transfer
learning (Pan and Yang, 2009; Baxter, 2000), meta-learning (Hospedales et al.,
2020; Finn et al., 2017) and few-show learning (Snell et al., 2017; Sung et
al., 2018). The key differences between these boils down to their use case and
learning algorithms. The types of models that each of these techiques use can
be very similar. We can also further differentiate these methods in terms of
whether they use use a notion of similarity or relatedness to make learning
faster.

(a) Hard parameter sharing in multitask learning using a neural network. The
black nodes represent nodes that are shared between all tasks, whereas the
gray nodes are task specific.

(b) Weight sharing in inductive transfer learning. The black nodes represent
nodes that are fixed during learning the new task, whereas the gray nodes are
updated during this time.

Figure 7: Example network architectures for multitask learning and inductive
transfer learning.

Multitask learning uses the relatedness structure between a set of tasks to
learn _better_ than if we learned on just a single task. We can imagine this
as reusing data; if given 2 tasks that are related to each other, we can
assume that the data that they generate would also be related. That is, the
data from $t_{1}$ can be _useful_ to solving $t_{2}$, but maybe not as much as
data from $t_{2}$ itself555Super transfer (Hanneke and Kpotufe, 2019) is an
interesting phenomenon, where data from $t_{1}$ is more useful for $t_{2}$
than data from $t_{2}$ itself.. Thus, we could use data from $t_{1}$ to
augment the learning of $t_{2}$, and vice versa. This is usually done via some
form of parameter sharing, either hard (shown in Figure 7(a) or soft; these
models in fact correspond to the local representation of a foliation. In
multitask learning, we do not assume that we see a subsequent task. The
correspondence with our framework should be clear here.

Transfer learning, particularly that done via inductive transfer (Pan and
Yang, 2009; Baxter, 2000) does. Typically, we initially train a neural
network666We describe transfer learning in neural networks as they are the
most clear, pedagogically. However, transfer learning is not restricted to
neural networks (Skolidis, 2012). on a vast dataset that contains information
about several different problems; these problems could be defined as being
separate tasks. We would then be given a new dataset from a particular task,
and be expected to adapt the learned model to this new task. This is often
carried out by fixing several layers of the network, and retraining the
remaining. This is depicted in Figure 7(b). The weights of the layers that are
learnt initially then correspond to the bias; it contains information that is
relevant to the current task, and the updated parameters contain the task
specific information, and must be learned again.

According to Pan and Yang (2009), transfer learning can also be transductive.
In the previous case, the domain and codamains of the tasks remained constant,
whereas the maps between them changed. They called this a change in the task,
but our definition of the task encompassed the domain and codomain too.
Transductive transfer could involve, for example, a change in the domain; this
is called domain adaptation. Typical solutions in domain adptation involve
learning intermediate representation maps that can be used on subsequent
tasks, simiilar to inductive transfer learning (Wang and Deng, 2018); the key
differences lie in the losses used, and perhaps other intuitive constraints,
such as geometric ones (Chopra et al., 2013).

Meta learning, or learning to learn is a recently revitalised field that is
popular. Historically, it was initially discussed in papers such as Thrun and
Mitchell (1994), Schmidhuber et al. (1996) and Thrun and Pratt (2012).
Schmidhuber (1995) and Schmidhuber et al. (1996) introduced a lot of the
intuition that we have tried to expound in the present work, but focussed on a
machine perspective of the system. Schmidhuber really identified how most of
what we think about when trying to learn across several tasks boiled down to
learning biases over these tasks. More modern methods in this field, such as
MAML (Finn et al., 2017), Siamese networks (Bertinetto et al., 2016) and CAVIA
(Zintgraf et al., 2019) are deep learning based methods that follow the
principles of learning that are familiar today.

Meta learning (Hospedales et al., 2020) under the deep learning paradigm can
be described as a multitask learning paradigm that learns over two
optimisation levels; the higher level which learns something that is useful
across all tasks, and the lower level that optimises task specific parameters.
During the inference phase, where a new task is provided, assumed from the
same set of related tasks, only the lower level optimisation is carried out.
This should sound familiar at this point. The learning to learn paradigm
involves learning what is useful for the given set of tasks, as is done by the
higher level optimiser; in practice, this could be, for example, choosing
hyperparameters, or a network architecture (Neural Architecture Search).
Often, few shot learning is conflated with meta learning.

The second class of methods are applied to tasks that are obtained
sequentially. Continual learning (Lesort, 2020; Lesort et al., 2020; Delange
et al., 2021), also called lifelong learning (Thrun and Mitchell, 1995),
incremental learning, sequential learning or online learning (Hoi et al.,
2018), assumes that data from different tasks arrive in a sequential fashion.
Here, on an episode, the learner is given data from a particular task, and is
asked to adapt to this task, without forgetting what has been learned in
previous episodes; additional constraints could include a lack of access to
past data. Forgetting what has been learned previously is, quite
bombastically, called catastrophic forgetting777We can try to understand
catastrophic in terms of similarity (see Section 7.3); the further you move
away from a particular solution, the less similar you become. At some point,
resemblence to the previous task has sufficiently vanished, that we can
classify the learner to have forgetten this..

These tasks are of course assumed to be related; if not, then catastrophic
forgetting is not an issue. The assumption that past tasks are useful to
present tasks implies that these tasks are related, that there is something to
transfer. As such, we can pose the continual learning problem attempting to
learn the bias of a set of related tasks, such that subsequent tasks can
exploit this knowledge without forgetting. This is exactly what we had
described previously, except for the fact that the tasks appear sequentially,
rather than at the same time.

Curriculum learning (Bengio et al., 2009) is an interesting iteration on this
sequential learning problem. Inspired by human learning, a learner in this
case is given a specifically chosen sequence of tasks, where learning occurs
sequentially on each episode as before. The choice of the learning tasks is
made in such a way that the learner is able to learn a _concept_ easier than
if they were given the hardest task initially. Here, rather than attempting to
learn a useful bias, we assume that knowledge of such a bias, and generate a
sequence of task that respect this bias. We will not discuss this technique
further.

##  6 Existing Methods Implicitly use Foliated Structures

In this section, we will provide examples of well established models that
perform transfer, and how they can be derived from our framework. There are
some points that are applicable to all the present cases.

Firstly, all these models assume that given a set of tasks, we implicitly
assume that all the tasks lie in the same set of related tasks. That is, we
assume that they all exist on the same leaf of a foliation on the task space.
In that sense, when trying to learn on the model space, the problem becomes
one of finding the appropriate leaf given a chosen foliation, and the points
along each leaf that can represent the given tasks. Thus the equivariance
requirement of the learning algorithm is implicitly satisfied by the
construction of the model space and the implicitly chosen foliation.

Secondly, in these models, we are only interested in a local neighbourhood of
the model space; it is likely that these regions also contain non-identified
models, for example if a $\mathrm{ReLu}$ non-linearity is used. In fact,
unless we are interested in global properties, it is sufficient to look at a
local neighbourhood.

The general strategy that we use here is to specify how each of these models
create a split of the total degrees of freedom that the model space has into
global and local parameters; the foliated structure is really just a way to
generalise this idea.

###  6.1 Transfer Learning with Global and Local Parameters

These types of models are commonly seen in the transfer learning setting (Pan
and Yang, 2009), and in some meta learning models, such as (Sæmundsson et al.,
2018; Petangoda et al., 2019; Hausman et al., 2018; Kaddour et al., 2020).
These models simply take the direct definition of the local coordinates of a
foliation, and use that as the definition of the model. Figure 7(b) shows an
example network topology of such a model.

In inductive transfer learning, a common approach is to pretrain a model on a
large dataset, to obtain optimal parameters. Suppose these are $\theta$. Then,
given a new dataset, the model is rectified into
$(\theta_{fixed},\theta_{retrain})$; the $\theta_{retrain}$ components are
retrained while the $\theta_{fixed}$ components are kept fixed. We see
immediately that this is simply describing the local coordinates of a
foliation.

That is, in a local open set $U\subset\mathcal{M}$, we have a chart $\phi$
such that $\phi(m)=(\theta_{fixed},\theta_{retrain})$, where
$\theta_{fixed}\in\mathbb{R}^{a}$, $\theta_{retrain}\in\mathbb{R}^{b}$ and
$a+b=n$, the total number of parameters. Each $\theta_{fixed}$ defines a local
plaque of dimension $b$. During transfer, since $\theta_{fixed}$ remains
constant, the retraining occurs on this plaque.

Something similar occurs in (Sæmundsson et al., 2018). Here, the split between
the fixed and retrained parameters is done at the same time. In this case,
they are called the global and local parameters respectively. They are called
as such because the global parameters apply to all tasks, whereas the local
parameters are task specific. Furthermore, the global parameters are the data
points themselves (since the model they use is a nonparametric Gaussian
Process). These values are fixed, and the optimisation that is carried out is
over the hyperparameters of the GP model, and a set of latent variables. These
latent variables are written for each task. The hyperparameters could be
thought of as a part of the global parameters, and the latent variables are
the task specific parameters. Thus the learning objective finds a portion of
the global parameters, as well as suitable values of the latent variables.
During inference time, the hyperparameters are fixed, and a new latent
variable is learned.

###  6.2 Prototypical Networks: Simple Meta-Learning

Prototypical Networks are used for few-shot classification (Snell et al.,
2017). It learns an emdedding function which can be used to compute a
distribution over the different classes of the problem. The embedding function
is used to compute a prototype vector in the embedding space for each class. A
distribution over the classes for a new query point is then computed by
running a softmax over the distance from the prototypes to the embedding of
the new data point. This algorithm then carries out few-shot learning by
generating the prototypes by applying the embedding function on the dataset
given for the new task, and immediately computing the predictions on queries.
During training, each iteration involves choosing a subset of the classes and
the given data for each, and optimising the embedding function so as to
minimise the negative log likelihood of the true predictions.

Figure 8: A foliation in the space $(\theta,c_{1},...,c_{k},...,c_{K})$ gives
us what we expect from a prototypical network. For a given dataset
$\mathfrak{D}$, we find $\theta^{\mathfrak{D}}$ during the initial training
phase. For a given task, which has classes from
$\\{2,4,5\\}\subset\\{1,...,K\\}$, the values $(c_{2},c_{4},c_{5})$ define the
distribution over the class probabilities of a particular input $x$. These
values are calculated from a small task specific dataset
$\mathfrak{D}_{t}=\\{(x_{1},2),(x_{2},2),(x_{3},4),(x_{4},4),(x_{5},5),(x_{6},5)\\}$,
in this example.

To be more precise, suppose we are given a data
$\mathfrak{D}=\\{(x_{i},y_{i})\\}_{i\in\mathcal{I}}$, where $x_{i}\in X$ and
$y_{i}\in\\{1,...,K\\}$ is a class identity. A classification task $t$ is
given by choosing $n_{t}\leq K$ classes from $\\{1,...,K\\}$, denoted as
$Y_{t}$ and $S\leq\>\big{|}\>\mathcal{I}\>\big{|}\>$ to generate a dataset
$\mathfrak{D}_{t}=\bigcup_{k\in
Y_{t}}\\{(x_{i},y_{i}):(x_{i},y_{i})\in\mathfrak{D},y_{i}\in Y_{t}\\}$. The
embedding function is a map
$f_{\theta^{\mathfrak{D}}}:X\rightarrow\mathbb{R}^{m}$, where
${\theta^{\mathfrak{D}}}\in\theta$ are the parameters of this function,
expressed as an element from a parameter space $\theta$. For the $k$-th class
in $Y_{t}$, the prototype is given by,

$c_{k}=\frac{1}{\>\big{|}\>S_{k}\>\big{|}\>}\sum_{(x_{i},y_{i})\in
S_{k}}f_{\theta^{\mathfrak{D}}}(x)(x_{i}).$ (3)

$S_{k}$ is the number of data points given for the class $k\in Y_{t}$. The
probability of a particular class, given a distance metric $\rho$ on the
embedding space, is then:

$p(y=k\>\big{|}\>x)=\frac{\exp(-\rho(f_{\theta^{\mathfrak{D}}}(x),c_{k}))}{\sum_{k^{\prime}\in\mathcal{Y}_{t}}\exp(-\rho(f_{\theta^{\mathfrak{D}}}(x),c_{k^{\prime}}))}.$
(4)

In this model, the full parameters that specify a prediction are
$(\theta,c_{1},...,c_{k},...,c_{K})$. $\theta^{\mathfrak{D}}$ corresponds to
the global bias that specifies how each input should be transformed; this is
independent of the task that is currently being solved. The coordinates given
by the remaining $(c_{1},...,c_{k},...,c_{K})$ specify a distribution over the
particular classes; they are the task specific parameters. Each $c_{k}$
therefore corresponds to a particular class. Since for a given task,
$n_{t}\leq K$, we would be ignoring the $c_{k}$ coordinates that correspond to
classes in the set $\\{1,...,K\\}\backslash Y_{t}$. It should be noted that
here, the adaption to the new task is deterministic and immediate; there is no
_training_ (optimisation) required. This approach is illustrated in Figure 8.

###  6.3 Model Agnostic Meta Learning (MAML)

MAML (Finn et al., 2017) is a meta-learning technique that has been very
popular since it was derived a few years ago; it is a good example of a method
that makes use of 2 levels of optimisation. For MAML, we are given a set of
tasks; its goal is to find a common initialisation point, from where following
gradient descent (or another gradient based optimisation method) for a fixed
number of iterations will yield a suitable model for each task. MAML does this
by optimising the initial point in a higher level loop, and following gradient
descent for a fixed number of steps starting from this point; see Figure 9.

Figure 9: MAML finds an optimal initial model $m_{0}$ such that solutions to
other tasks $m^{*}_{1}$, $m^{*}_{2}$ and $m^{*}_{3}$ are $k$ gradient descent
steps away.

Showing that this type of model uses the structures we have discussed in
general is difficult. This is particularly because we need to study the
dynamics of the inner optimisation algorithm. This is made further difficult
by the fact that in practice, MAML uses stochastic optimisers, and also only
needs to reach a _sufficiently_ accurate model; while the latter point is also
true in the previous cases, here, it can form a key part of the description of
the model. We will make some simplifying assumptions regarding these; the
dynamics of the descent algorithms follow the gradient flow of the vector
field generated by the gradient of the loss function w.r.t the parameters of
the model space. The task, per such descent, is fixed.

If MAML is to carry out learning to transfer, then it must find some invariant
structure that can be reused, without learning, when solving subsequent tasks.
One can analyse MAML in several ways. One such method is given in Grant et al.
(2018), where it is posed as a hierarchical bayes model. Another method could
be to look at it from a foliated point of view. We make the following
proposition:

Figure 10: Showing the leaf generated by MAML for a loss function that is
quadratic w.r.t the task space and the model space. Centered at each
$m_{t_{i}}^{*}$, we have drawn some of the level sets of the loss function.
This pictorially shows the claim that MAML finds points that are on a circle,
when the coordinate system is centered at the starting point $m_{0}$. The
arrows show the flows, starting at $m_{0}$ along the positive $t$ direction
for each task.

######  Proposition 19 (Local foliation induced by MAML)

We are given a task space $\mathcal{T}$, a model space $\mathcal{M}$ (a
$d$-dimensional smooth manifold) and an _appropriate_ loss function
$\mathcal{L}:\mathcal{T}\times\mathcal{M}\rightarrow[0,\infty]$. Assume that
$L$, restricted to some neighbourhood $U_{\mathcal{T}}$ of $\mathcal{T}$ and
$U_{\mathcal{M}}$ of $\mathcal{M}$, is smooth and continuous w.r.t both
variables. These neighbourhoods are chosen such that $\forall t\in
U_{\mathcal{T}}$, there exists a unique $m^{*}_{t}\in U_{\mathcal{M}}$, where
$m^{*}_{t}=\operatorname*{arg\,min}_{m\in U_{\mathcal{M}}}L(t,m)$.

Under certain conditions regarding how the gradient field of $\mathcal{L}$
changes as the task is changed, and given a set of related tasks
$R|_{U_{\mathcal{T}}}=R\cap U_{\mathcal{T}}$ which is restricted to
$U_{\mathcal{T}}$, we propose that MAML finds a point $m_{0}\in
U_{\mathcal{M}}$ such that for any task $t\in R|_{U_{\mathcal{T}}}$, we can
obtain a solution $m_{t}$ where
$\>\big{|}\>\mathcal{L}(t,m_{t})-\mathcal{L}(t,m_{t}^{*})\>\big{|}\>=\epsilon$,
for $\epsilon$ that is fixed for all tasks in $U_{\mathcal{T}}$, by following
the flow $\theta_{m_{0}}:\mathbb{R}\rightarrow\mathcal{M}$ of the gradient
field of the negative loss for $k$ time steps, starting at $m_{0}$. That is
$\theta_{m_{0}}(k)$ for any task in $R|_{U_{\mathcal{T}}}$ is a solution that
is of the prescribed accuracy $\epsilon$.

Then, the set $\\{m\in U_{\mathcal{M}}\\}$ such that the above condition is
true for all for a fixed $k$ and $\epsilon$ forms a leaf of a folation; other
leaves can be reached by changing $k$. Thus MAML describes a $d-1$ foliation
(locally).

The proof of this very general statement is beyond the scope of the present
work. It requires us to closely study how a given loss function changes as
tasks are transformed; this needs careful specification of task spaces and its
notion of relatedness. There are some points to note here. We have made the
accuracy fixed; in practice, we are okay with models that are _at worst_ of a
chosen accuracy; in the view of the foliations, we are saying that several,
neighbouring leaves are good enough for our solution. The same can be said
about stating that we want to reach a sufficiently suitable solution in at
most $k$ steps. It is possible that this requirement ends up creating a
partition of a tesselation; that is we have found a region of similarity. All
this will be explored in future work.

In order to present an intuition as to why the above would be correct, we
present a very simplified example that illustrates Proposition 19. Let us
assume that the model space is locally 2-dimensional; that is
$\phi(U_{\mathcal{M}})\subset\mathbb{R}^{2}$, for some chart on the
$\mathcal{M}$. Write the same for $U_{\mathcal{T}}$. Further, make the very
restrictive assumption that the loss function
$\mathcal{L}:U_{\mathcal{T}}\times U_{\mathcal{M}}\rightarrow[0,\infty]$ is
homogeneously quadratic in both variables. That is, we can write the loss as

$\mathcal{L}((t_{1},t_{2}),(m_{1},m_{2}))=\sum_{i=1}^{2}(m_{i}-t_{i})^{2}$ (5)

Here, we see that changing the task merely changes the position of
$m_{t}^{*}$, but keeps the shape of loss curve over $\mathcal{M}$ fixed. The
level sets of this system create circles centered at $t$; this is shown in
Figure 10. The loss value at the optimal model for a given model is $0$. For
such as system, we can show the following theorem:

###### Theorem 20

For a given $m_{0}$, and a coordinate system centered at $m_{0}$, the subsets
of $\mathcal{T}$ for which the accuracy of the model is a fixed $\epsilon$, is
given by

$\sum_{i=1}^{2}t_{i}^{2}=\frac{\epsilon}{e^{-4k}}.$ (6)

Here, $(t_{1},t_{2})$, are the coordinates of each of the dimensions of a
point $t\in\mathcal{T}$. $k$ denotes a variable that controls the radius; this
$k$ corresponds to time that one must follow the flow of a task starting at
$m_{0}$ to end up at the model that is of $\epsilon$ accuracy. The following
corollary can then be shown:

###### Corollary 21

Given a task $t=(t_{1},t_{2})\in\mathcal{T}$, a chosen accuracy
$\epsilon\in[0,\infty]$, and an initial model $m_{0}$, the time $k$ needed to
get to a model $m_{t}$ for which the accuracy
$\>\big{|}\>\mathcal{L}(t,m_{t})-L(t,m_{t}^{*})\>\big{|}\>=\epsilon$ is given
by:

$k_{\epsilon}=4\ln\bigg{(}\frac{\epsilon}{t_{1}^{2}+t_{2}^{2}}\bigg{)},$ (7)

and the model coordinates $m_{t}=(m_{1},m_{2})$ are given by:

$m_{i}=-t_{i}\bigg{(}\frac{\epsilon}{t_{1}^{2}+t_{2}^{2}}\bigg{)}+t_{i}.$ (8)

The proofs for Theorem 20 and Corollary 21 are given in Appendix B.3. These
equations show that the subsets that satisfy the criterion for a set of
related tasks under the MAML scheme are _circles_ in $\mathbb{R}^{2}$, for the
given problem specification; this is a regular foliation on
$\mathbb{R}^{2}/{0}$. These results can be extended to higher dimensions, and
to loss functions that aren’t symmetric; for example, if the level sets look
like ellipsoids, then we can expect that leaves would be ellipsoids too.

In this example, the behaviour of the loss function was specified from the
outset. Typically, under the targeted equivariance of the learning algorithm,
this behaviour would be derived from the defined charts of the task space, the
assumed notions of relatedness on the task space, and the chosen foliation
(and therefore charts) on the model space. In Equation 5 (and the subsequent
proofs) for example, we have assumed that the task space has trivial
coordinates in $\mathbb{R}^{2}$. It therefore stands to reason for the
existence of a class of loss functions, for which the conditions of the MAML
specification are satisfied. That is:

######  Proposition 22 (Loss functions for MAML)

We are given a task space $\mathcal{T}$ and a model space $\mathcal{M}$ which
are described as smooth manifolds with foliations. $U_{\mathcal{T}}$ and
$U_{\mathcal{M}}$ are open subsets of $\mathcal{T}$ and $\mathcal{M}$
respectively. Given a notion of relatedness, there is a local foliation on
$U_{\mathcal{T}}$, and an associated local foliation on $U_{\mathcal{M}}$ such
that a leaf on $U_{\mathcal{T}}$, denoted ${L}_{\mathcal{T}}$ has a
corresponding leaf on $U_{\mathcal{M}}$, denoted as
${L}_{\mathcal{M},\mathcal{T}}$, which contains _the best_ or _sufficiently
good_ descriptions of the elements of the former. For a given $t\in
U_{\mathcal{T}}$, $m_{t}\in U_{\mathcal{M}}$ is this descriptor.

Any loss function $\mathcal{L}_{t}$ on $U_{\mathcal{M}}$, restricted to $t\in
U_{\mathcal{T}}$ will produce a flow on $U_{\mathcal{T}}$. Suppose we choose
an $m_{0}\in U_{\mathcal{M}}$ and a $k\in[0,\infty]$. We propose that there
exists a set of loss functions $\mathcal{L}_{\mathrm{MAML}}$ such that for a
task $t\in U_{\mathcal{T}}$, $\theta(k,m_{0})_{{L}_{t}}=m_{t}$, and that if
$t\in{L}_{\mathcal{T}}$ for some leaf, then for all such $t$,
$m_{t}\in{L}_{\mathcal{M},\mathcal{T}}$.

This proposition can be rephrased in terms of the transformations that form
the leaves on the task and model spaces. That is, if $\pi_{\mathcal{T}}$ and
$\pi_{\mathcal{M}}$ are such transformations, then for
$s=\pi_{\mathcal{T},ts}(t)$, and $m_{s}=\pi_{\mathcal{M},ts}(m_{t})$,
$\theta(k,m_{0})_{L_{s}}=\pi_{\mathcal{M},ts}(\theta(k,m_{0})_{L_{t}})$.

Proposition 22 tells us that there are several components that can be
controlled to gain the learning system that we want. In this case, we have
chosen $k$, $m_{0}$ and the notions of relatedness, and are trying to
construct a loss function appropriately. In the original MAML problem, we are
trying to find the $m_{0}$, given the rest of the components. Proving this
proposition is beyond the scope of the present paper.

It must be noted that the condition where
$\>\big{|}\>\mathcal{L}(t,m_{t})-\mathcal{L}(t,m_{t}^{*})\>\big{|}\>=\epsilon$,
stated in Proposition 19 might make it difficult to prove the existence of the
proposed foliation. In Theorem 20, we showed that it can exist in a simple
case; in particular this depended heavily on how the gradient fields of the
loss functions changed as we changed that tasks. As such, an alternative
method of looking at MAML is to consider when
$\>\big{|}\>\mathcal{L}(t,m_{t})-\mathcal{L}(t,m_{t}^{*})\>\big{|}\><\epsilon$
is allowed. In this case, it is possible that MAML describes an open set
around $m_{0}$ which corresponds to a set of related tasks. It doesn’t define
a global foliation, or any global structure at all. In both cases, MAML
parameterises a single set of related tasks, and moves this set until the
observed tasks fall within it. Further analysis of such a picture of MAML is
left for future work.

##  7 Discussion

In this section, we will provide a discussion of some key aspects of the
framework we have presented.

###  7.1 Examples of Relatedness

In Section 4.2, we provided a definition of a notion of relatedness in terms
of groups and/or pseudogroups. In this section, we will provide some examples
of this, to help illustrate the concept.

Figure 11: A simple task space $\mathcal{T}$, where its structure is defined
by functions $f_{a,b}(x)=ax+b$, for $a,b,x\in\mathbb{R}$.

Let us look at a simple problem. Consider a set of functions that can be
written as
$\mathcal{T}=\\{f_{a,b}:\mathbb{R}\rightarrow\mathbb{R}\>\big{|}\>a,b\in\mathbb{R}\\}$,
where $f_{a,b}(x)=af(x)+b$ for $x\in\mathbb{R}$, and $f(x)$ is some smooth and
continuous function $f:\mathbb{R}\rightarrow\mathbb{R}$. Figure 11 shows an
example of this, where $f(x)=x$. We are interested in understanding a notion
of relatedness between tasks in the task space. We can see that $\mathcal{T}$
is isomorphic to $\mathbb{R}^{2}$, where each point represents a particular
function, with coordinates given by $(a,b)$.

Imagine that from this set, we are given some learning tasks to solve that
satisfy the constraint that $b=1$. For example, this set of learning tasks
contain $t_{1},t_{2}\in\mathcal{T}_{b=1}=\\{f_{a,b=1}\\}$, with
$t_{1}=f_{a=1,b=1}$ and $t_{2}=f_{a=2,b=1}$. The set $\mathcal{T}_{b=1}$ is
simply a straight line, parallel to the horizontal axis in $\mathbb{R}^{2}$
space, and _any element_ of this subset can be accessed by choosing the
appropriate value of $a$. The difference between the two tasks here can be
represented in terms of the difference between their values of $a$. That is,
we can transform from $t_{1}$ to $t_{2}$ by _adding_ $1$ to the first
coordinate. The additive group $\mathbb{R}$ that acts on the whole of
$\mathcal{T}$ is the notion of relatedness here; the horizontal parallel lines
in $\mathbb{R}^{2}$ are the sets of related tasks.

A notion of relatedness provides us with standard by which to measure
relatedness against. For example, suppose in a different problem, we were
given tasks from a set
$\mathcal{T}_{circ}=\\{f_{a,b}|a,b\in\mathbb{R},a^{2}+b^{2}=1\\}$. In this
case, we can see that $\mathcal{T}_{circ}$ can be represented by a unit circle
in $\mathbb{R}^{2}$. The notion of relatedness then is the circle group
$\mathbb{S}^{1}$.

Figure 12: Images for image classification that contain some basic shapes.
These shapes can be generated as the interior points of a set bounded by
constant $L_{p}$ norms about origins given by $(x,1)$; $p\in[0,\infty]$. The
reference coordinate system has $(0,0)$ at the bottom left corner of the image
boundary. This set consists of all images obtained by horizontally and
continuously translating the “object” in the image between the left-most and
right-most columns.

As a more complicated example, consider the diagram given in Figure 12. We are
introducing a set of images that can be used for a classification problem; let
us denote this set by $I$. There are several classification problems we can
consider here. Let’s consider the task of multi-class classification, where we
must identify whether the object on the image is a square, circle or a
diamond, and the objects are centered at $x_{1}$; thus we are looking at the
first column of Figure 12. Imagine then that our dataset is translated
rightwards, by moving the objects by some amount to the right; the classes
still stay the same, and the task is to learn a classifier that will adapt to
the transformed domain. Here, the transformations can again be thought of in
terms of position along a circle $\mathbb{S}$; like in a game of Snake, once
the object hits the right edge, it resets to the left most edge888If there was
vertical movement, this group would be the torus
$\mathbb{S}\times\mathbb{S}$..

Note that this is qualitatively different from the regression task from the
previous example. That is, instead of changing the qualitative nature of the
structure, we have changed the domain. That is, we are still trying to map
images of diamonds, circles and squares to their labels; instead the images
are slightly altered. This is called domain adaptation in the literature (Pan
et al., 2010; Daume III and Marcu, 2006; Kouw and Loog, 2018; Ben-David et
al., 2007). As far as our definition of a learning task is concerned, there is
no real difference, abstractly speaking. The map from input to output still
has to change.

Another problem setting we can consider from this domain is that of binary
classifiers that make true or false statements about the identity of the
image. That is, $f_{p,x}:I\rightarrow\\{0,1\\}$, where $f_{p,x}(i)=1$ iff the
image $i$ consists of the shape given by the interior of the boundary made of
points $t$ in $\mathbb{R}^{2}$ such that $||t||_{p}=1$, and is centered at
$(x,1)$ in the reference coordinate system. In this case, we can see that an
extra dimension has been added to the problems; this full space consists of
the direct product $[0,\infty]\times\mathbb{S}^{1}$. We can then consider
smaller subsets of related tasks to consider for learning to transfer; for
example, each of the rows in Figure 12 are circles of fixed $p$ in
$[0,\infty]\times\mathbb{S}^{1}$.

In terms of the pendulum example, we can note that the mass and length values
can be transformed as the direct product $[0,\infty]\times[0,\infty]$. The
full task space that contains the pendulums can be the set of classical
dynamical systems that vary with respect to a notion of a single length and
mass. This space could include cartpoles, double pendulums where only a single
pendulum changes between tasks, etc.

###  7.2 Invariances

In the first example in Section 7.1, we saw that we could create a set of
related tasks by keeping $b$ fixed at $1$. Similarly, in $f_{circ}$, we had
the radius of the circle at $1$, and in $I$, we could fix $p$. These show
another, and possibly complementary characteristic of a set of related tasks;
in addition to containing a notion of relatedness, there is also an _invariant
quantity_ that stays constant within this set. The invariant quantity is
derived from the transformation set.

An invariant quantity is defined in the following way.

######  Definition 23 (Invariant quantity)

Given a space $X$, a quantity on $X$ is a map $f:X\rightarrow\mathbb{R}$. This
quantity is said to be invariant with respect to a transformation $\pi$ on
$X$, where $\pi:X\rightarrow X$, iff $\forall x\in X$, $f(x)=f(\pi(x))$.

This definition implies that irrespective of a particular transformation
applied on the input space, the quantity $f(x)$ remains unchanged for any
value of $x$. $g$ is the transformation with respect to which the quantity is
invariant; it is a key component. In the definition, $g$ is a single function.
It is often useful to define invariant quantities are described with respect
to a set of functions $\mathcal{G}$, where for any $g\in\mathcal{G}$, the
above condition holds. If the set of functions respects the definition of a
group, then $\mathcal{G}$ is said to form a symmetry group of $X$.

Such a group falls within our intuitive notion of symmetries. We know that a
circle in $\mathbb{R}^{2}$ is symmetric under rotations; an invariant quantity
here is the radius of the circle. That is, if we were to express the circle in
Cartesian coordinates, then the radius $r(x,y)=\sqrt{x^{2}+y^{2}}$ does not
change if were to rotate about the centre by some angle. Now the
transformations do change the superficial “quantifiable value” of a point. It
does not however change the “nature” of it; the nature of the object then, is
given by its invariant quantities. This is equivalent to saying that the
nature of the object is defined by being symmetric with respect to the given
group. That is, considering a set of “all possible things”, an invariance can
be used to collect a subset of this set such that they have a consistent and
constant property.

The notion of symmetries, and invariances can therefore be used to identify
system. In physics for example, particular invariances are a characteristic
feature of a physical system (Wigner, 1949), in that there are some canonical
groups which our laws of physics are symmetric to. In classical systems, we
expect the behaviour of systems to be invariant to translations in space and
time. For example, the way that a pendulum swings (assuming a closed system
with a uniform gravitational field) should not depend on _where_ in the
laboratory the experiment is carried out. It also should not depend on whether
the experiment is started at midnight, or at noon. Thus, from all possible
ways that a physical object should behave, the assumed invariances focus our
search to a smaller set of behaviours.

There are of course other invariances that are needed to uniquely define the
behaviour of a system. In the pendulum, these include the fact that the motion
of a pendulum occurs around a circle, since the length of the pole is fixed.
In a double pendulum, another such constraint999Note that we have not
distinguished between constraint or invariance. A constraint is often
described as a function $f_{con}(x)=0,\>\forall x\in X$. This looks remarkably
similar to the definition of an invariance; a constraint implicitly defines an
invariance. is that the 2 poles are connected. A set of invariances then allow
us to identify the particular system we want from the set of all systems, in
this case classical systems. When we apply Newton’s Second Law to a physical
problem, we explicitly write down these invariances. Furthermore, classical
systems themselves obey Hamilton’s principle where the path taken over time
through its generalised coordinates is an extremal of its action functional
(Arnol’d, 2013); this distinguishes this set from the set of all possible
physical systems.

The pendulum is also commonly used in RL problems, as we described in Figure
1. Changing the mass and length of the pendulum will also change the policies
in a similar way. The invariant quantity in the set of RL tasks must also
contain details about the control problem at hand; that is the fact that we
always start at the bottom, and want to swing up and balance at the top, in
the fastest strategy possible. It will also contain any other control
constraints that are enforced in the problem. We could consider the case when
the goals of the problem change, as was done in (Petangoda et al., 2019) in
the case of the cart-pole dynamical system.

Figure 13: Rigid body transformations in $\mathbb{R}^{2}$ on a squares and
triangles. These transformations include rotations, reflections and
translations.

An invariant quantity, along with its corresponding transformation, can be
used to identify a class of objects from within a larger set. Consider the
example in Figure 13. Here, we have 2 objects, a square and a triangle. The
transformation group we are considering is the union of reflections, rotations
and translations in $\mathbb{R}^{2}$; together, these form the rigid body
transformations. We will denote this set as $\mathcal{G}_{RB}$.

Suppose we considered the quantity which is defined as the area enclosed by
each object. The area is a map $A:ST\rightarrow[0,\infty]$, where $ST$ is the
set of all possible triangles and squares, of different sizes and
orientations. Another quantity that can be consider is the number of vertices;
$N:ST\rightarrow\mathbb{N}$. It is clear that under a transformation from
$\mathcal{G}_{RB}$, the area of a particular shape does not change. This is
true whether we are talking about a square or triangle. This is true for any
polygon or closed curve in $\mathbb{R}^{2}$. We can therefore say that, a
defining characteristic of closed curves in $\mathbb{R}^{2}$ is that their
areas are invariant under rotations. The defining characteristic of $ST$ is
that it _also_ only consists of polygons with 3 or 4 vertices; of course,
$N_{ST}$ is also an invariant quantity here.

In our framework, where notions of relatedness are represented in the learning
problem as a foliation, there is a natural, local set of invariant quantities
that arise. This is most clear on a regular foliation, where we know that a
subset of the local coordinates of a chart remain constant.

###  7.3 Similarity

A related concept that is often used to compare tasks is _similarity_.
Similarity is often used interchangeably with relatedness; here we will
present a distinction.

######  Definition 24 (A Set of Similar Tasks)

A metric $\rho$ on $\mathcal{T}$ is called a notion of similarity.
$t_{1},t_{2}\in\mathcal{T}$ are said to be similar if
$\rho(t_{1},t_{2})<\epsilon$. Then, given $t_{1}\in\mathcal{T}$, the set
$\\{t\in\mathcal{T}|\rho(t,t_{1})<\epsilon\in\mathbb{R}^{+}\\}$ is called a
set of tasks similar to $t_{1}$.

If we choose a countable number of reference tasks on $\mathcal{T}$, then a
notion of similarity can induce a partition on $\mathcal{T}$ in the way of a
Voronoi diagram (Reddy and Jana, 2012). That is, we can write an equivalence
relation, given a set of reference tasks
$R_{\mathcal{T}}={r_{i}\in\mathcal{T}}_{i\in\mathcal{I}}$, $t_{1}\sim
t_{2}\iff\operatorname*{arg\,min}_{r_{i}\in
R_{\mathcal{T}}}\rho(r_{i},t_{1})=\operatorname*{arg\,min}_{r_{i}\in
R_{\mathcal{T}}}\rho(r_{i},t_{1})$. Such a partition would look like the
tesselation in Figure 5. Since this is a partition, there is a notion of
relatedness that can be associated with it (for example, by trivially
considering the permutation groups on each element of the partition).

It is possible to consider transfer in terms of similarity, by transferring
within a set of similar tasks, as defined above, given a task
$t\in\mathcal{T}$. This is often carried out in Transfer Learning (Pan and
Yang, 2009), or Continual Learning (Lesort, 2020), where pre-trained models
are _slightly_ updated to a new dataset. Our notion of similarity can describe
why catastrophic forgetting occurs (Goodfellow et al., 2013) in such methods.
The further away from the original pre-trained model we move, the less similar
the new models are; thus, gradually, we lose any resemblance to it. In the
case of transfer using relatedness, such an issue does not occur as long as we
fix and stay on the parallel space we operate on.

Figure 14: The distinction between similarity and relatedness. Similarity is
defined in terms of an $\epsilon$-ball around a particular element. Here,
points $t_{2}$ and $t_{3}$ are similar to $t_{1}$. On the other hand,
relatedness is defined in terms of a transformation set; this relationship is
shown as a line joining them. Thus, tasks $t_{1},t_{2}$ and $t_{3}$ are
related to each other. However, $t_{4}$ is not related to $t_{2}$ and $t_{1}$
though they are similar; $t_{3}$ is not similar to $t_{2}$ and $t_{1}$, though
they are related.

There are some notable practical differences between relatedness and
similarity. In the former the consistency of parallel spaces depends on the
set of transformation groups or pseudogroups that are chosen. That is, if we
were to pick out two tasks that are related to each other, then a third task,
which is related to the second would also be related to the first, all with
respect to the transformation set. In particular, since a notion of
relatedness can give rise to a partition of $\mathcal{T}$, it provides us with
some global structure that can be used for transfer.

On the other hand, a set of similar tasks is only useful locally (unless it is
used to generate a tesselation, in which case there is an induced notion of
relatedness), since relative to a given task, we can talk about a single set
of similar tasks. In particular, the transitivity we mentioned previously,
does not hold, unless the reference task is kept constant. Thus if $t_{1}$
being similar to $t_{2}$, which is similar to $t_{3}$ does not necessarily
imply that $t_{1}$ is similar to $t_{3}$.

##  8 Some Philosophical Considerations

In this section, we will describe some philosophical considerations of machine
learning in general, and how our framework for learning to transfer carries
naturally from them.

###  8.1 The Structure of Machine Learning

In Definition 1, we defined a learning task in terms of some structure that it
contains, and uniquely identifies it. We hinted that this structure is often
written as maps that carry certain properties. In this section we will
describe such structure in more detail as applied to different problems in ML.
The clearest example is supervised learning (Bishop, 2006).

Supervised learning considers labelled data, which are generated from a joint
distribution $p(X,Y)$ over input and output random variables $X$ and $Y$
respectively. We could consider that $p(X)$ is known, and since
$p(X,Y)=p(Y)p(X|Y)$, we are really interested in learning the conditional
distribution over $Y$ given $X$. An equivalent view point is to assume that
there is a generative map $f:X\rightarrow Y$, which has some noise added to
it; that is, $y=f(x)+\epsilon$, where $\epsilon\sim\mathcal{N}(0,1)$. The map
$f$, and its defining properties is the structure of the problem, and learning
task is this entire structure.

The learning task in RL often contains the policy. A RL problem is usually
defined by a Markov Decision Process (MDP) $(S,A,T,r,\gamma)$; that is, given
a system’s transition dynamics $T=p(S^{\prime}|S,A)$ which dictate the
evolution of the system’s state $S$, we want to find a policy $\pi=p(A|S)$
that maximises the long term return $R=\sum_{t=0}^{T}\gamma^{t+1}r(s_{t})$
over the path taken by the state. $r$ is a reward function and $\gamma$ is a
discount factor that describes how myopic the system is. The tuple
$(S,A,T,r,\gamma)$ is the structure of an RL problem. Depending on the class
of RL, the learning task is the subset of the structure that corresponds to
the policy (model free), or $T$ and $r$ too (model based).

Unsupervised learning can be slightly more complicated. Let us consider the
clustering problem (Madhulatha, 2012) as an example. Here, we are given a set
of unlabelled data from some domain $\mathcal{X}$, and we would like to
identify clusters to which each data point belongs to. Suppose we assume that
there exist an indexed set of $n$ clusters $\\{c_{i}\\}_{i=1}^{n}$ where each
$c_{i}\in\mathcal{X}$, in unsupervised learning, we want to learn a map
$f:\mathcal{X}\rightarrow\\{c_{i}\\}$. In k-means (Bishop, 2006), this map
outputs the cluster which is the shortest distance (based on some metric on
$\mathcal{X}$ such as the Euclidean distance) to the new point. In kernel
k-means (Dhillon et al., 2004), the point is first mapped onto a different
space, before the same algorithm as k-means is run; this allows us to consider
a different distance metric. The structure here are the clusters, and their
members.

More modern problems such as semi-supervised learning (Zhu and Goldberg, 2009)
can also become complicated when trying to define the precise structure we
want to find; however, such a structure does exist. In semi-supervised
classification for example, a dataset that consists of a mix of labelled and
unlabelled data is given. The ratio is often skewed heavily towards the
unlabelled data. The goal is to find a classifier using the total dataset that
is _better_ than a classifier found using just the labelled data. This could,
for example, be done by assuming that the data that map to a particular class
clusters around some center (as in k-means). The classification aspect of this
is supervised learning. The assumed structure is found in the assumption that
the unlabelled data follow some regularity with the labelled data.

###  8.2 Machine Learning as Learning Structure and Representations

Given a learning task and a model space, the problem of learning is to find an
element of $\mathcal{M}$ that best _represents_ the learning task. This was
described in Definition 6. The choice of the word _represent_ here was no
accident. This view of the learning problem was inspired by Marr’s visionary
book _Vision_ (Marr, 1982). A key component of this is the computational
theory.

The computational theory of a system embodies _what_ the system must do. These
are written in terms of transformations that must be carried out on the
information that is received (if any), and the information that must be
output. The simple example Marr presents is that of a cash register. The
inputs it receives are the prices of items in a shopping cart; the output it
must provide is the sum total of these prices. The action the cash register
must do to accomplish this is addition. As simple as this sounds, it is
important to understand that addition here is an abstract notion that carries
with it some necessary rules.

The addition that the cash register must carry out is its _process_ ; it is
what it must do to its inputs to obtain its output. Now one can think of such
a system that can carry out the process of addition abstractly, not just as a
cash register. A generative process as we described in Definition 1 is such a
process. In order to make the abstract real and useful, we need a
_representation_ of the objects that are its inputs, outputs and its process.
In (Marr, 1982), a representation is a formal _scheme_ by which an object can
be described; the result of the application of such a scheme to a particular
object is its _description_. Thus, the goal in ML is to find a description of
the generative process w.r.t. a chosen representation. Often, some aspects of
the process are known, or assumed; the remaining unknown structure is what we
called the learning task. By defining model space, we have written a
representation, and the learning problem is to find the best description of
the learning task.

As an example of a representation, we can look at number systems. The Arabic
numeral scheme expresses integers as $(x\in\mathbb{N}):=\sum_{i}a_{i}10^{i}$,
where $a_{i}\in\\{0,1,2,...,9\\}$. Therefore, by writing the abstract number
forty-two as “42”, we are setting $a_{0}=2$, $a_{1}=4$ and the rest to zero.
The same object $42$ can be similarly be described in the binary
representation scheme as $101010$. The representation of addition operation
then depends on the representation chosen of its inputs and outputs. Addition
in the Arabic numeral system and the binary system are conceptually similar,
since we add each unit, and carry over if the sum becomes larger than $9$ or
$1$ respectively. If however, we had chosen a Roman numeral system, where $42$
is written as XLII, then addition will be carried out using different, more
complicated rules, even though it is abstractly doing the same thing.
Representations satisfy some properties. A representation is not unique; there
can be many ways to describe a single system. The important aspects of a
representation that must be preserved between different representations are
the behavioural properties that are important to the task of the system. In
the case of a cash register, any representation of prices must be able to
carry the additive group structure. It is then possible to consider mapping
between suitable representations. Perhaps this is more conceivable if one
considers vector spaces and linear maps between them. Suppose $f:X\rightarrow
Y$ is a linear map that is written as a matrix $\mathbf{F}$ under the bases
$e_{X}$ and $e_{Y}$ for $X$ and $Y$ respectively. If there is a basis change
by transformations $\mathbf{B_{X}}$ and $\mathbf{B_{Y}}$, then under the new
bases, the linear map will be described by
$\bar{\mathbf{F}}=\mathbf{B_{Y}FB_{X}^{-1}}$. $\bar{\mathbf{F}}$ and
$\mathbf{F}$ are still the same linear map; it was merely their descriptions
that changed when we changed the representation scheme; see Figure 15 for the
commutation graph for verification. Thus, all representations are equivalent
up to isomorphisms.

Figure 15: Commutation of linear maps under basis changes.

Each representation can be thought to be composed of some elementary
_primitives_. These primitives are basic units that are combined, in some way,
to describe all things a representation must describe. In the case of the
Arabic numerals, these are the powers of 10, in binary, the powers of 2, and
in vector spaces, the basis vectors. The representation also depends on _how_
these primitives are combined. In particular, we see that in the Arabic
numerals, the powers of 10 are combined by a summing their multiples with one
of $\\{0,1,...,9\\}$. In binary, this set is $\\{0,1\\}$, and in vector
spaces, it is $\mathbb{R}$. As a more complicated example, consider an image,
often represented as grid of pixels. In a grayscale image, each pixel can take
a value from the closed set $[0,1]$, and the image cosists of the
concatenation of several such numbers from $[0,1]$ in a 2D grid.

The example of an image brings forward another important aspect of a
representation; it is possible for a representation to only _approximate_ the
objects describes. An image (as we consider here) itself is a representation
of some macroscopic scene. Depending on the camera that is used, such an image
can be of different resolutions, where each pixel now describes some average
of the region of the scene that it covers. However, the approximate
representation must still satisfy whatever properties the exact representation
does. Take the example of the cash register. A suitable approximating
representation for prices are the positive integers, where each price is
approximated by rounding up or down. Such a system still satisfies the
requirement for the additive structure. That is, suppose the prices of 2 items
are £5.4 and £7.10. In this approximate scheme, these prices are £$5\pm 0.5$
and £$7\pm 0.5$. The sum of the original prices is £12.50, which is £13 when
approximated; the latter is within the accuracy of $(5\pm 0.5)+(7\pm
0.5)=(12\pm 1.0)$.

Finally, there is a notion by which some representation schemes are better
than others; in the example of addition, as a process that humans can carry
out mentally, the Arabic numeral scheme makes the arithmetic easier than if we
were to use the binary or Roman numeral schemes. For use in computers however,
the binary scheme is better.

A model space we had defined previously can be used as a representation of
$\mathfrak{P}(\mathfrak{S})$. A model in $\mathcal{M}$ is assumed to be able
to act as a suitable description for the learning task at hand. Of course, as
with Marr’s representations, this element doesn’t necessarily have to be
_exact_ , just _suitable_. Furthermore, there can be many model spaces that
can similarly describe the task at hand, and as is fairly obvious with the
most famous example of a parametric model, neural networks, these spaces can
be built from elementary primitives. While each of these is a non-unique,
suitable representation, they do however, contain other salient differences;
these are their inductive biases.

###  8.3 Inductive Bias and Learning to Transfer

Each model and task space contains some inductive bias. Inductive biases are
important for learning. Albeit not in familiar language, Mitchell (1980)
describes this eloquently. In a world where learning is described as finding a
generalisation that is _consistent_ with a set of instances, a bias is
anything that will make a particular generalisation, contained in the set of
all possible and correct generalisations, more likely to be chosen. An
unbiased generalizer assigns equal probability to each suitable
generalisation. A bias introduces additional information or assumptions that
have been made of the problem at hand, which make some equally suitable
solutions (according to a purely data-fitting point of view) more relevant.

The No Free Lunch Theorem (Wolpert and Mcreader, 1997; Shalev-Shwartz and Ben-
David, 2014; Ho and Pepyne, 2002) is a more formal description of this idea
that appeared a few years later. It states that no general purpose learning
algorithm that can be usefully applied to _any_ problem is possible; there
will always be a problem that another, more specialised or biased algorithm
will outperform it in.

This can also be illustrated in the bias-variance trade-off in classical
learning theory (Shalev-Shwartz and Ben-David, 2014). The error associated
with a model of a hypothesis class can be decomposed into the sum of
approximation and estimation errors. The approximation error is the lowest
possible error in the hypothesis class; this of course independent of any data
that is gathered. The estimation is the difference between this and the full
error; this is therefore the error associated with choosing a hypothesis that
is not the best in class, and depends on the data and the learning algorithm
chosen. The approximation error can be reduced by increasing the complexity or
the _size_ of the hypothesis class; this is reducing bias. This will
invariably increase the estimation error since it is harder to search for the
best element in a larger set; this is the variance.

More bias increases the approximation error, but reduces the estimation error.
Classical theory, such as the VC dimension (Vapnik and Chervonenkis, 2015) was
devised to determine how to increase the bias such that the approximation
error doesn’t grow significantly faster than the estimation error. This will
then reduce the overall error.

Bias represents external information that is implicitly assumed to true about
the learning task, and in particular, a set of related tasks. Thus bias
creates subsets of _all possible tasks_. Bias contains the subset of the
structure of a learning problem that is not _learned_ by the learning
algorithm, but is simply assumed to be true. That is, it contains
$\mathfrak{S}-t\subseteq\mathrm{bias}$, where $t$ is a learning task as
defined in Definition 1. When solving a single learning task, biases can
define a representation scheme (or model space) from which to choose a
description (or model) for the learning task. In particular, biases allow us
to make a choice between competing, equally competent descriptions (from
different model spaces).

Humans for example, are biased towards simpler and more general models of
things; see Newton’s Laws for example. Simplicity has a certain _intellectual
romance_ to it; Occam’s razor could, for example, be posited as an imperative
for elegance. Occam’s razor requires us to chose the _simplest_ theory or
model from a set of theories and models that can represent or explain the data
observed equally well. The elegance arises from the lack of redundancies in
the structure of a simpler theory. It is also the case that such a rule often
works well in the kinds of problems that humans encounter.

Such regularisation of model complexity can be found in ML methods. For
example, choosing hyperparameter values of a Gaussian Process by maximising
the marginal likelihood (Williams and Rasmussen, 2006) explicitly penalises
_complex_ models; such complexity is represented by one of the terms of the
log marginal likelihood. Complexity here can be seen as the smoothness of the
corresponding input-output maps of, say the posterior mean function; think the
differences between the graph of a polynomial with low and high order.

All this was to say that bias is important, and choosing the _correct bias_ is
paramount to good generalisation. In learning to transfer, the notion of
relatedness is a form of choosing a bias that is assumed to be applicable to
each set of related tasks. That is, choosing a set of relatedness biases the
model space in which we assume a suitable solutions to set of related tasks
lie.

###  8.4 Choosing a Notion of Relatedness

As final thought, we bring attention to the problem of choosing the
appropriate notion of relatedness. As mentioned, choosing a notion of
relatedness is choosing a bias; the previous section argued that this is
important. Such bias can also be introduced via domain knowledge from experts;
past examples in similar fields or applications (as the expert determines) can
guide the choice of how we write models. If it is known that the set of
related tasks that are likely to be observed follow a particular notion of
relatedness, then one can imbue their learning algorithm with the appropriate
foliation.

The more interesting case is when we can attempt to learn this structure. If
our (task or model) space is given a foliated structure, we know that each
leaf will typically be of a dimension less than that of the ambient space. A
useful way of thinking about these dimensions is as factors of variations that
allow us to reach all tasks in a set of related tasks. The search for a notion
of relatedness would be the search for the sets of related tasks, and
additionally the factors of variation that are useful. A useful formulation of
this search then is in terms of finding the most useful factors of variation.

Given a set of tasks to learn to transfer over, the search for factors of
variation can be carried out in many ways. These include statistical factors
of variation, group theoretic disentanglement (Pfau et al., 2020; Higgins et
al., 2018), or information theoretic (Botvinick et al., 2015). In the
information theoretic case for example, the appropriate choice would be made
in terms of the efficiency of coding. That is, there can be an assumption
about the distribution from which tasks are sampled from, and the best notion
of relatedness would be the most efficient way of encoding these tasks.

##  9 Conclusion

In the present article, we introduced a framework that poses problems of
learning to transfer from first principles. In particular, our framework
explicitly exposes what it means for tasks to be related. We argued that
relatedness requires a _standard_ against which to measure it, and learning to
transfer amounted to finding a representation of the structure that defined
such a standard. This structure can be written in terms of foliations. We then
showed that modern problems, such as multitask learning, meta learning and
transfer learning, can all be described within a coherent learning-to-transfer
paradigm, and we specifically demonstrated how some example models implicitly
make use of foliations.

We believe that our framework will be crucial in studying and understanding
problems of transfer using precise and interpretable techniques. In future, we
will be investigating how this framework can applied to answering foundational
questions regarding the transferrability between two arbitrary learning tasks,
the complexity of learning to transfer, or the development of novel methods of
transfer.

Acknowledgments

We would like to thank the School of Mathematics and Statisitics at the
University of Sheffield for funding this project. We also thank So Takao,
Laura Krasowska, Kate Highnam and Daniel Lengyal for the many discussions we
had when developing and writing up this work.

## References

  * Arnol’d (2013) Vladimir Igorevich Arnol’d.  _Mathematical Methods of Classical Mechanics_ , volume 60.  Springer Science & Business Media, 2013.
  * Baxter (1995) Jonathan Baxter.  Learning internal representations.  In _Proceedings of the Conference on Computational Learning Theory_ , pages 311–320, 1995.
  * Baxter (2000) Jonathan Baxter.  A model of inductive bias learning.  _Journal of Artificial Intelligence Research_ , 12:149–198, 2000.
  * Ben-David and Borbely (2008) Shai Ben-David and Reba Schuller Borbely.  A notion of task relatedness yielding provable multiple-task learning guarantees.  _Machine Learning_ , 73(3):273–287, 2008.
  * Ben-David and Schuller (2003) Shai Ben-David and Reba Schuller.  Exploiting task relatedness for multiple task learning.  In _Learning Theory and Kernel Machines_ , pages 567–580. Springer, 2003.
  * Ben-David et al. (2007) Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira.  Analysis of representations for domain adaptation.  In _Advances in Neural Information Processing Systems_ , pages 137–144, 2007.
  * Bengio et al. (2009) Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston.  Curriculum learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 41–48, 2009.
  * Bertinetto et al. (2016) Luca Bertinetto, Jack Valmadre, Joao F. Henriques, Andrea Vedaldi, and Philip H. S. Torr.  Fully-convolutional siamese networks for object tracking.  In _European Conference on Computer Vision_ , pages 850–865. Springer, 2016.
  * Bishop (2006) Christopher M. Bishop.  _Pattern Recognition and Machine Learning_.  Springer, 2006.
  * Botvinick et al. (2015) Matthew Botvinick, Ari Weinstein, Alec Solway, and Andrew Barto.  Reinforcement learning, efficient coding, and the statistics of natural tasks.  _Current Opinion in Behavioral Sciences_ , 5:71–77, 2015\.
  * Camacho and Neto (2013) César Camacho and Alcides Lins Neto.  _Geometric Theory of Foliations_.  Springer Science & Business Media, 2013.
  * Caruana (1997) Rich Caruana.  Multitask learning.  _Machine Learning_ , 28(1):41–75, 1997.
  * Cederquist and Negri (1995) Jan Cederquist and Sara Negri.  A constructive proof of the heine-borel covering theorem for formal reals.  In _International Workshop on Types for Proofs and Programs_ , pages 62–75, 1995.
  * Chopra et al. (2013) Sumit Chopra, Suhrid Balakrishnan, and Raghuraman Gopalan.  Dlid: Deep learning for domain adaptation by interpolating between domains.  In _ICML Workshop on Challenges in Representation Learning_ , 2013\.
  * Cohen et al. (2019) Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling.  Gauge equivariant convolutional networks and the icosahedral CNN.  In _Proceedings of the International Conference on Machine Learning_ , pages 1321–1330, 2019.
  * Cohen et al. (2018) Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling.  Spherical CNNs.  _arXiv preprint arXiv:1801.10130_ , 2018.
  * Daume III and Marcu (2006) Hal Daume III and Daniel Marcu.  Domain adaptation for statistical classifiers.  _Journal of Artificial Intelligence Research_ , 26:101–126, 2006.
  * Delange et al. (2021) Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2021.
  * Dhillon et al. (2004) Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis.  Kernel k-means: spectral clustering and normalized cuts.  In _Proceedings of the International Conference on Knowledge Discovery and Data Mining_ , pages 551–556, 2004.
  * Ellis (1965) Henry C. Ellis.  _The Transfer of Learning._ Macmillan, 1965.
  * Elsken et al. (2019) Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter.  Neural architecture search: A survey.  _Journal of Machine Learning Research_ , 20(55):1–21, 2019.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _Proceedings of the International Conference on Machine Learning_ , 2017.
  * Goodfellow et al. (2013) Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio.  An empirical investigation of catastrophic forgetting in gradient-based neural networks.  _arXiv preprint arXiv:1312.6211_ , 2013.
  * Grant et al. (2018) Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths.  Recasting gradient-based meta-learning as hierarchical Bayes.  In _Proceedings of the International Conference on Representation Learning_ , 2018.
  * Hall (2015) Brian Hall.  _Lie Groups, Lie Algebras, and Representations: An Elementary Introduction_ , volume 222.  Springer, 2015.
  * Hanneke and Kpotufe (2019) Steve Hanneke and Samory Kpotufe.  On the value of target data in transfer learning.  In _Advances in Neural Information Processing Systems_ , pages 9871–9881, 2019.
  * Hausman et al. (2018) Karol Hausman, Jost Tobias Springenberg, Ziyu Wang, Nicolas Heess, and Martin Riedmiller.  Learning an embedding space for transferable robot skills.  In _Proceedings of the International Conference on Learning Representations_ , 2018.
  * Higgins et al. (2018) Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner.  Towards a definition of disentangled representations.  _arXiv preprint arXiv:1812.02230_ , 2018.
  * Ho and Pepyne (2002) Yu-Chi Ho and David L. Pepyne.  Simple explanation of the no-free-lunch theorem and its implications.  _Journal of Optimization Theory and Applications_ , 115(3):549–570, 2002.
  * Hoi et al. (2018) Steven C. H. Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao.  Online learning: A comprehensive survey.  _arXiv preprint arXiv:1802.02871_ , 2018.
  * Hospedales et al. (2020) Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey.  Meta-learning in neural networks: a survey.  _arXiv preprint arXiv:2004.05439_ , 2020.
  * Jurdjevic (1997) Velimir Jurdjevic.  _Geometric Control Theory_.  Cambridge University Press, 1997.
  * Kaddour et al. (2020) Jean Kaddour, Steindór Sæmundsson, and Marc P. Deisenroth.  Probabilistic active meta-learning.  In _Advances in Neural Information Processing Systems_ , 2020.
  * Kouw and Loog (2018) Wouter M. Kouw and Marco Loog.  An introduction to domain adaptation and transfer learning.  _arXiv preprint arXiv:1812.11806_ , 2018.
  * Lavau (2018) Sylvain Lavau.  A short guide through integration theorems of generalized distributions.  _Differential Geometry and its Applications_ , 61:42–58, 2018.
  * Lawson (1998) Mark V. Lawson.  _Inverse Semigroups, the Theory of Partial Symmetries_.  World Scientific, 1998.
  * Lawson Jr (1971) H. Blaine Lawson Jr.  Codimension-one foliations of spheres.  _Annals of Mathematics_ , pages 494–503, 1971.
  * Lawson Jr (1974) H. Blaine Lawson Jr.  Foliations.  _Bulletin of the American Mathematical Society_ , 80(3):369–418, 1974.
  * Lee (2001) John M. Lee.  _Introduction to Smooth Manifolds_.  Springer, 2001.
  * Lesort (2020) Timoth’ee Lesort.  Continual learning: tackling catastrophic forgetting in deep neural networks with replay processes.  _arXiv preprint arXiv:2007.00487_ , 2020.
  * Lesort et al. (2020) Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia Díaz-Rodríguez.  Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges.  _Information Fusion_ , 58:52–68, 2020.
  * Lisle and Reid (1998) Ian G. Lisle and Gregory J. Reid.  Geometry and structure of lie pseudogroups from infinitesimal defining systems.  _Journal of Symbolic Computation_ , 26(3):355–379, 1998.
  * Madhulatha (2012) T. Soni Madhulatha.  An overview on clustering methods.  _arXiv preprint arXiv:1205.1117_ , 2012.
  * Marr (1982) David Marr.  _Vision: A Computational Investigation into the Human Representation and Processing of Visual Information_.  Henry Holt and Co., Inc., 1982.  ISBN 0716715678.
  * Mendelson (1990) Bert Mendelson.  _Introduction to Topology_.  Courier Corporation, 1990.
  * Mitchell (1980) Tom M. Mitchell.  _The Need for Biases in Learning Generalizations_.  Department of Computer Science, Laboratory for Computer Science Research, 1980.
  * Nakahara (2003) Mikio Nakahara.  _Geometry, Topology and Physics_.  CRC Press, 2003.
  * Olver (1995) Peter J. Olver.  _Equivalence, Invariants and Symmetry_.  Cambridge University Press, 1995.
  * Pan and Yang (2009) Sinno Jialin Pan and Qiang Yang.  A survey on transfer learning.  _IEEE Transactions on knowledge and data engineering_ , 22(10):1345–1359, 2009.
  * Pan et al. (2010) Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok, and Qiang Yang.  Domain adaptation via transfer component analysis.  _IEEE Transactions on Neural Networks_ , 22(2):199–210, 2010.
  * Petangoda et al. (2019) Janith C. Petangoda, Sergio Pascual-Diaz, Vincent Adam, Peter Vrancx, and Jordi Grau-Moya.  Disentangled skill embeddings for reinforcement learning.  _arXiv preprint arXiv:1906.09223_ , 2019.
  * Pfau et al. (2020) David Pfau, Irina Higgins, Aleksandar Botev, and Sébastien Racanière.  Disentangling by subspace diffusion.  _arXiv preprint arXiv:2006.12982_ , 2020.
  * Reddy and Jana (2012) Damodar Reddy and Prasanta K. Jana.  Initialization for k-means clustering using voronoi diagram.  _Procedia Technology_ , 4:395–400, 2012.
  * Rosenberg and Roussarie (1970) H. Rosenberg and R. Roussarie.  Reeb foliations.  _Annals of Mathematics_ , pages 1–24, 1970.
  * Ruder (2017) Sebastian Ruder.  An overview of multi-task learning in deep neural networks.  _arXiv preprint arXiv:1706.05098_ , 2017.
  * Sæmundsson et al. (2018) Steindór Sæmundsson, Katja Hofmann, and Marc P. Deisenroth.  Meta reinforcement learning with latent variable Gaussian processes.  In _Proceedings of the Conference on Uncertainty in Artificial Intelligence_ , 2018.
  * Schmidhuber et al. (1996) Juergen Schmidhuber, Jieyu Zhao, and Marco. A. Wiering.  Simple principles of metalearning.  _Technical Report IDSIA_ , 69:1–23, 1996.
  * Schmidhuber (1995) Jürgen Schmidhuber.  On learning how to learn learning strategies.  Technical report, 1995.
  * Schunk (2012) Dale H. Schunk.  _Learning Theories, an Educational Perspective_.  Pearson, 2012.
  * Shalev-Shwartz and Ben-David (2014) Shai Shalev-Shwartz and Shai Ben-David.  _Understanding Machine Learning: From Theory to Algorithms_.  Cambridge University Press, 2014.
  * Silver and Mercer (1996) Daniel L. Silver and Robert E. Mercer.  The parallel transfer of task knowledge using dynamic learning rates based on a measure of relatedness.  In _Learning to learn_ , pages 213–233. Springer, 1996.
  * Skolidis (2012) Grigorios Skolidis.  _Transfer Learning in Gaussian Processes_.  School of Informatics, University of Edinburgh, 2012.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In _Advances in Neural Information Processing Systems_ , pages 4077–4087, 2017.
  * Stefan (1974) Peter Stefan.  Accessible sets, orbits, and foliations with singularities.  _Proceedings of the London Mathematical Society_ , 3(4):699–713, 1974.
  * Sung et al. (2018) Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales.  Learning to compare: relation network for few-shot learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1199–1208, 2018.
  * Sussmann (1973) Héctor J. Sussmann.  Orbits of families of vector fields and integrability of distributions.  _Transactions of the American Mathematical Society_ , 180:171–188, 1973.
  * Thrun and Mitchell (1994) Sebastian Thrun and Tom M Mitchell.  Learning one more thing.  Technical report, Carenegie Mellon University, 1994.
  * Thrun and Mitchell (1995) Sebastian Thrun and Tom M. Mitchell.  Lifelong robot learning.  _Robotics and Autonomous Systems_ , 15(1-2):25–46, 1995.
  * Thrun and Pratt (2012) Sebastian Thrun and Lorien Pratt.  _Learning to Learn_.  Springer Science & Business Media, 2012.
  * Vapnik and Chervonenkis (2015) Vladimir N. Vapnik and Alexey Ya. Chervonenkis.  On the uniform convergence of relative frequencies of events to their probabilities.  In _Measures of Complexity_ , pages 11–30. Springer, 2015.
  * Vilalta and Drissi (2002) Ricardo Vilalta and Youssef Drissi.  A perspective view and survey of meta-learning.  _Artificial Intelligence Review_ , 18(2):77–95, 2002.
  * Wang and Deng (2018) Mei Wang and Weihong Deng.  Deep visual domain adaptation: A survey.  _Neurocomputing_ , 312:135–153, 2018.
  * Wang et al. (2020) Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni.  Generalizing from a few examples: A survey on few-shot learning.  _ACM Computing Surveys_ , 53(3):1–34, 2020.
  * Weyl (2015) Hermann Weyl.  _Symmetry_ , volume 104.  Princeton University Press, 2015.
  * Wigner (1949) Eugene P. Wigner.  Invariance in physical theory.  In _Proceedings of the American Philosophical Society_ , volume 93, pages 521–526, 1949.
  * Williams and Rasmussen (2006) Christopher K. I. Williams and Carl Edward Rasmussen.  _Gaussian Processes for Machine Learning_.  MIT Press, 2006.
  * Wolpert and Mcreader (1997) Daniel H. Wolpert and William G. Mcreader.  No free lunch theorems for optimization.  _IEEE Transactions on Evolutionary Computation_ , 1(1):67–82, 1997.
  * Zhu and Goldberg (2009) Xiaojin Zhu and Andrew B. Goldberg.  Introduction to semi-supervised learning.  _Synthesis Lectures on Artificial Intelligence and Machine Learning_ , 3(1):1–130, 2009.
  * Zintgraf et al. (2019) Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson.  Fast context adaptation via meta-learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 7693–7702. PMLR, 2019.

##  Appendix A Mathematical Preliminaries

###  A.1 Topology

We begin by describing what a topology is. In Section 4.1 we argued that
thinking about the topology of our set of related tasks is important. To
understand the reasons for this, a rudimentary understanding of concepts in
topology will be useful.

A topology is perhaps the simplest structure one can place on a set
$\mathcal{M}$.

######  Definition 25 (Topology)

Given a set $X$, a topology $\mathcal{O}_{X}$ on $X$ is a set of subsets of
$X$ that satisfy the following conditions:

  1. 1.

$X\in\mathcal{O}_{X}$ and $\emptyset\in\mathcal{O}_{X}$,

  2. 2.

$U_{1},U_{2}\in\mathcal{O}_{X}\implies(U_{1}\cup U_{2})\in\mathcal{O}_{X}$,

  3. 3.

$U_{1},U_{2},...,U_{n}\in\mathcal{O}_{X}\implies\bigcap_{i=1}^{n}{U_{i}}\in\mathcal{O}_{X}$.

Simply put, a topology $\mathcal{O}_{\mathcal{M}}$ is a set of subsets of a
set which is closed under intersections and unions, and also contains the null
set $\emptyset$, and the set $\mathcal{M}$ itself. An element of a topology
$U\in\mathcal{O}_{M}$, which is a subset of $M$ is called an _open set_ , the
difference of which with $M$ is called a _closed set_. Despite these names, it
is incorrect to think of an open set as an open interval; particularly, some
sets can be both open and closed (for example, the full set $\mathcal{M}$).
The tuple $(\mathcal{M},\mathcal{O}_{\mathcal{M}})$ is called a topological
space.

An intuitively familiar example of a topology is the called the standard
topology $\mathcal{O}^{S}_{\mathbb{R}^{d}}$ on $\mathbb{R}^{d}$. To define the
standard topology, one defines its elements, the open sets implicitly as
follows. An open set $U\in\mathcal{O}^{S}_{\mathbb{R}^{d}}$ iff $\forall p\in
U,\>\exists r\in[0,\infty]:\mathcal{B}^{r}(p)\subseteq U$, where
$\mathcal{B}^{r}(p):=\big{\\{}x\>\big{|}\>\sqrt{\sum_{i=0}^{d}(x_{i}-p_{i})^{2}}\big{\\}}$
are open balls centred at $p$. This topology in $\mathbb{R}^{2}$ is depicted
in Figure 16.

Figure 16: An open set in the standard topology on $\mathbb{R}^{2}$. Note that
the dashed lines imply that the boundary is not included. Furthermore, the
open set $U$ is populated with other open balls at all other points $p\in U$.

An important idea a topology affords us is a neighbourhood of a point.
Different authors define this slightly differently, but these differences are
inconsequential to our discourse. (Lee, 2001) defines a neighbourhood of a
point $p\in\mathcal{M}$ as an open set $U_{p}\in\mathcal{O}_{\mathcal{M}}$
that contains $p$; on the other hand, (Mendelson, 1990) define
$A_{p}\subseteq\mathcal{M}$ to be a neighbourhood of $p$ if it completely
contains an open set $U_{p}\in\mathcal{O}_{\mathcal{M}}$ that contains $p$.
Presently, we will stick to the former definition by (Lee, 2001), since it is
simpler to consider.

A neighbourhood of a point $p$ allows us to talk about which points are
_close_ to $p$. That is, we can consider that all points in a neighbourhood
$U_{p}$ are close to $p$ in this sense. Furthermore, consider 3 points
$p,q,r\in\mathcal{M}$. Suppose that almost any neighbourhood $U_{pq}$ that
contains $p$ and $q$ also contains $r$, but one can find many neighbourhoods
$U_{pr}$ that contain $p$ and $r$, but do not contain $q$. We can consider
this in terms of the size of $U_{pq}$ and $U_{pr}$ (using a measure), or say
in terms of the radii of open balls that contain them. Then in this sense, we
can say that $p$ and $r$ are qualitatively closer to each other than they are
to $q$.

As an example, consider the circle space $\mathbb{S}^{1}$. Such an object can
be defined by the space of equivalent classes of numbers on the real line,
where the equivalence relation is given by $p\sim q$ if $p=q+2\pi$. This is
similar to taking the interval $[0,2\pi]$ and gluing the 2 ends together.
Topologically this means that $0.00001$ is _close_ to $2\pi-0.00001$; this is
of course different to real line, where these points are _further away_. Thus,
$\mathbb{S}^{1}$ is different to $\mathbb{R}$, and even the closed interval
$[0,2\pi]$.

At this point, it is worth noting the relationship between topological spaces
and metric spaces. Loosely, a metric space is a set equipped with a metric,
which is a numerical measure of distance between points; this is defined more
formally in Definition 26.

######  Definition 26 (Metric)

Given a set $X$, a metric is a map $\rho:X\times X\rightarrow[0,\infty]$ that
satisfies:

  1. 1.

$\rho(x,y)=0\iff x=y$,

  2. 2.

$\rho(x,y)=\rho(y,x)$ (symmetry),

  3. 3.

$\rho(x,y)\leq\rho(x,z)+\rho(z,y)$ (triangle inequality),

for $x,y,z\in X$.

Every metric space defines a topological space through the metric topology;
this is a topology similar to the standard topology, where the open balls are
defined with respect to the metric on the metric space. In fact, the standard
topology is the metric topology on $\mathbb{R}^{d}$ defined with respect to
the Euclidean metric. There are, however topological spaces which do not have
a corresponding metric; those that do, are called _metrizable_ (Mendelson,
1990). In this way, a topological space is more general. That is, as in the
case of the circle example above, we are allowed to consider $closeness$ in
terms of a metric, but it is not the only allowed way. Choosing a metric fixes
the _shape_ of a space; for example, topologically, a geometric circle and a
geometric ellipse are topologically equivalent to the circle space
$\mathbb{S}^{1}$.

Now, we can imagine transforming a circle into an ellipse by simply stretching
or squashing actions (imagine a spherical balloon being squashed between 2
poles). Importantly, we do not tear (or create holes) the space. Such a
transformation, that maps between 2 topological spaces is called a continuous
map. More specifically, a continuous map is the topology preserving map; a map
$f:X\rightarrow Y$ between 2 topological spaces $(X,\mathcal{O}_{X})$ and
$(Y,\mathcal{O}_{Y})$ is _continuous_ if the pre-image of open sets in $Y$ is
open in $X$. If such a map is invertible, $f$ is said to be a _homeomorphism_
, which is the special name given to topological isomorphisms. Thus, the map
between a circle and an ellipse is a homeomorphism.

We note that in this way of defining a continuous map, the only necessary
information about $X$ and $Y$ are their topologies. A typical way of defining
a continuous map is in terms of the $\epsilon-\delta$ notation. Such a
definition is only possible on a metric space, and we state that the
definition given above is therefore more general. Of course, for a metrizable
topological space, the $\epsilon-\delta$ definition follows once a suitable
metric is found.

A topology allows us to then define a host of useful and important other
notions that are preserved under continuous mappings. One such is the notion
of compactness. Intuitively, one can think of a compact subset as a subset
that is closed and bounded; for $\mathbb{R}^{d}$, the Heine-Borel Theorem
(Cederquist and Negri, 1995) says that this is exactly true. A compact set
remains compact under a continuous map. Another such property is that of
(path)-connectedness. A subset $A$ is (path)-connected if one can draw a
continuous line $\gamma_{pq}:[0,1]\rightarrow A$ between any two points
$p,q\in A$, where $\gamma(0)=p$ and $\gamma(1)=q$. Generally, path-
connectedness and connectedness are distinct properties, but this distinction
is not necessary in the present work, and we mean path-connectedness if we
state that a set is connected.

###  A.2 Manifolds

A relatively simple, yet powerful object that we can start with is a
topological manifold:

######  Definition 27 (Topological manifold)

A tuple $(\mathcal{M},\mathcal{O}_{\mathcal{M}},\mathcal{A}_{\mathcal{M}})$,
is called a $d$-dimensional topological manifold if:

  1. 1.

$\mathcal{M}$ is a set,

  2. 2.

$\mathcal{O}_{\mathcal{M}}$ is a topology on $\mathcal{M}$, where
$(U\in\mathcal{O}_{\mathcal{M}})\subseteq\mathcal{M}$ is called an open set.
The topology must be Hausdorff and paracompact.

  3. 3.

$\mathcal{A}_{\mathcal{M}}$ is a collection of charts, where a chart
$(U,\phi)\in\mathcal{A}_{\mathcal{M}}$ consists of an open set and a
homeomorphism $\phi:U\rightarrow(U_{\mathbb{R}^{d}}\subseteq\mathbb{R}^{d})$.

Here, $\mathbb{R}^{d}$ is the usual real space endowed with
$\mathcal{O}^{S}_{\mathbb{R}^{d}}$, the standard topology.

A topological manifold is created by endowing a topological space with a set
of charts, called an atlas. A chart, from Definition 27 consists of a tuple of
an open set $U\in\mathcal{O}_{\mathcal{M}}$ and a map $\phi$ that is a
homeomorphism between $U$ and a subset $U_{\mathbb{R}^{d}}$ of