# Leveraging convergence behavior to balance conflicting tasks in multi-task
learning

Angelica Tiemi Mizuno Nakamura<EMAIL_ADDRESS>Valdir Grassi Jr
<EMAIL_ADDRESS>Denis Fernando Wolf<EMAIL_ADDRESS>Mobile Robotics
Laboratory, Institute of Mathematics and Computer Sciences, University of São
Paulo, São Carlos, Brazil.  Department of Electrical and Computer Engineering,
São Carlos School of Engineering, University of São Paulo, São Carlos, Brazil.

###### Abstract

Multi-Task Learning is a learning paradigm that uses correlated tasks to
improve performance generalization. A common way to learn multiple tasks is
through the hard parameter sharing approach, in which a single architecture is
used to share the same subset of parameters, creating an inductive bias
between them during the training process. Due to its simplicity, potential to
improve generalization, and reduce computational cost, it has gained the
attention of the scientific and industrial communities. However, tasks often
conflict with each other, which makes it challenging to define how the
gradients of multiple tasks should be combined to allow simultaneous learning.
To address this problem, we use the idea of multi-objective optimization to
propose a method that takes into account temporal behaviour of the gradients
to create a dynamic bias that adjust the importance of each task during the
backpropagation. The result of this method is to give more attention to the
tasks that are diverging or that are not being benefited during the last
iterations, allowing to ensure that the simultaneous learning is heading to
the performance maximization of all tasks. As a result, we empirically show
that the proposed method outperforms the state-of-art approaches on learning
conflicting tasks. Unlike the adopted baselines, our method ensures that all
tasks reach good generalization performances.

###### keywords:

Multi-task learning , Neural networks , Multi-objective optimization

##  1 Introduction

Machine learning can be used on several types of applications, and many of
them may require many abilities. For example, self-driving cars must be able
to detect obstacles for collision avoidance, to detect navigable areas and
horizontal lane markings for lane keeping, detect and recognise traffic signs
to respect traffic rules, and so on. Each of these abilities can be
interpreted as a task to be performed at the same time. One common way of
resolving multiple tasks is by implementing one solution for each task and
running all them in parallel. However, this approach can be computationally
very costly. Alternatively, there is a machine learning concept that allows
simultaneous learning of multiple tasks, i.e., Multi-Task Learning (MTL). This
approach has the potential of reducing the inference latency and computational
storage when using a single architecture by sharing common knowledge among
related tasks. Moreover, MTL can also be used to explore the idea of creating
an inductive bias between tasks [1]. Due to its potential to improve
generalization performance, it has gained attention in several areas of
scientific and industrial communities, such as computer vision [2, 3, 4, 5]
and natural language processing [6, 7, 8].

In deep neural networks, MTL can be categorized according to how parameters
are shared between tasks, i.e., soft and hard parameter sharing approaches. In
soft parameter sharing, each task has its own model with its own parameters,
and the inductive bias occurs only by a regularization function between
models. On the other hand, the hard parameter sharing creates the inductive
bias through multiple tasks by sharing the same subset of parameters. This
work focus on the second approach.

The main advantage of hard parameter sharing is related to the fact that
multiple tasks are learned using a single network. It allows to implement
simpler solutions, to reduce computational cost and inference latency at the
same time it potentially improves generalization by implicit bias induction.
In this approach, the simultaneous learning of multiple tasks is usually
performed by a linear combination of each task’s gradients over the shared
neurons [9, 3]. Therefore, the tasks will share and compete for the same
resource (e.g., neurons or convolutional kernels), making it necessary to
handle conflicting tasks. We define two tasks as conflicting when their
gradients have different directions. Thus, when the descent direction of a
task is not representative for the other, optimizing the network parameters in
relation to a bad direction can harm the performance of some tasks. Therefore,
it is necessary to find a trade-off between tasks, which is beyond what a
linear combination can achieve.

As an alternative, [2] proposed to model MTL as a multi-objective optimization
problem to find a solution that is not dominated by any other. That is, to
find a trade-off in which it is not possible to improve one task without
degrading another. The problem is that, in multi-objective optimization, there
is no single solution that is optimal for all tasks, but several solutions
with different trade-offs between them [10, 11]. Furthermore, considering
problems of different natures, variations in the task’s gradient scales can
influence the definition of the solution. Thus, when a solution is said to be
optimal, it does not mean that all tasks will be performing well, but only
that the solution has reached a point where no further improvement can be made
to all tasks simultaneously. This idea will be better explained in Section 3.

Based on the above considerations, this work proposes a method that guarantees
better performances to all tasks, without giving up the optimality condition
of multi-objective optimization. The difference between the existing methods
and the proposed one is the way how gradients are combined.

The existing methods only assume the gradients computed in the current
iteration to estimate an appropriate combination of them in order to obtain a
common descent direction between tasks. However, since these methods get only
a snapshot of the current gradients, they need to create a static bias when
combining the gradients, such as getting the direction with the minimum norm
[2] or the central direction between gradients [12]. Because of this, they
cannot ensure that all tasks are converging to their optimal performances,
although they are meeting the optimality conditions. On the other hand, if the
temporal factor of the gradients is considered to compute their combination on
the current iteration, it would be possible to analyze whether the
performances of all tasks are converging or not and create a dynamic bias. For
this, the proposed method creates the idea of tensors that strengthen when
their respective tasks lose performance and weaken when they increase
performance. This is done at the same time the optimality conditions are met.
By doing so, the combination of gradients is modulated to give more preference
to those that are diverging from its optimal performance, ensuring that all
tasks will converge together.

Therefore, the main contributions proposed in this paper are as follows: 1) A
method to create a dynamic learning bias by leveraging the convergence
behavior of tasks. 2) An approach that is independent of a specific
application to combine multiple gradient vectors and find a common feasible
descent direction, aiming to maximize the performance of all tasks. To confirm
the performance of the proposed method in learning conflicting tasks, we
performed ablation studies and a series of experiments on a public handwritten
digit classification dataset.

The remainder of this paper is structured as follows. Section 2 provides a
literature review. In Section 3, we present our proposed method to combine
gradients of multiple tasks by leveraging their convergence behavior over the
past iterations. In Section 4, we perform ablation studies and compare our
proposed method with other weighting methods, followed by a discussion of the
results. Finally, we conclude in Section 5.

##  2 Related Work

Introduced by [13], the Multi-Task Learning (MTL) is a machine learning
paradigm in which multiple related tasks are learned simultaneously, such that
each task can contribute to a shared knowledge that enhances the
generalization performance of every other tasks at hand. This idea of learning
multiple tasks may cause confusion among researchers. For example, one could
say that MTL applied to neural networks is simply the fact of creating an
architecture with multiple outputs, which has already been extensively
explored. However, this kind of solution usually encodes a single task [14, 1,
15]. On the other hand, the MTL aims to explore the idea that related tasks
can create an inductive bias, and based on that bias, a certain task will
prefer a hypothesis $x$ out of the $n$ possible hypothesis. In deep neural
networks, multi-task learning can be categorized according to how parameters
are shared between tasks, i.e., soft or hard parameter sharing.

In soft parameter sharing, each task has its own model with its own
parameters, and the bias induction is accomplished through functions that
reduce the distance between the models’ parameters [16, 17, 18, 19]. However,
this approach results in very complex architectures with higher computational
cost, which makes it unfeasible for many applications. On the other hand, the
hard parameter sharing approach allows a set of parameters to be shared among
all tasks, reducing inference latency and storage costs [13]. In computer
vision, the architecture of a hard parameter sharing models is usually
composed of a shared encoder and multiple task-specific decoders, in which the
models are trained by optimizing the gradient combination of all tasks. This
is commonly done by a linear combination of the negative gradients of each
task [9, 20, 21, 22], or some variation with adaptive weights [3, 4, 23, 24,
25].

The main challenge of this approach is to define how the gradients of
different tasks should be combined. Since the tasks share the same parameters
and compete for the same resources (e.g. convolutional kernels from encoder),
this combination must be done by finding a trade-off between them so that
shared layers can learn relevant features to all tasks [3, 5]. However, the
linear combination does not allow finding this balance between tasks [2, 26].
As an alternative, [2] proposed to model multitasking learning as a multi-
objective optimization problem to find a solution that is not dominated by any
other (Pareto optimal solutions), achieving better results than previous
methods of linear combination and its variations. However, in problems with
variations in the gradients’ scale, these variations can significantly impact
the final solution. In [2], his solution may end up prioritizing only the
tasks that have a lower gradient norm and not learning the other tasks well.
To find a solution that considers the tasks equally important, [27, 12] remove
the scale influence by normalizing the gradients for each task. In [6], it is
proposed to use the Tchebycheff metric to handle non-convex problems. Instead
of finding a single optimal solution, [26, 28] propose to find a set of Pareto
optimal solutions with different trade-offs between tasks.

However, these methods consider the current tasks’ gradients to find a common
descent direction, not analyzing the previously defined descent directions or
any indication if they are being representative to learn all tasks. It creates
a tendency for the solution to prefer a specific direction. Therefore, this
work proposes to analyze the convergence of tasks during the training process
to dynamically correct the common descent direction and find a trade-off
between conflicting tasks that maximizes simultaneous learning. Thus, we
define a new descent direction that prevents the model from learning more
about a specific task by pulling tasks that are diverging or that are not
being benefited during training. Through experimental results, we show that
the proposed method can handle conflicting tasks, finding a good balance
between them during the training process.

##  3 Approach

((a)) Feedforward

((b)) Backpropagation

Figure 1: Multi-task learning architecture. The output from the shared layers
is split into a set of $m$ task-specific layers. During backpropagation, the
parameters from task-specific layers are optimized according to the gradients
of each task, while in the shared layers, the optimization occurs by combining
the gradients of all tasks. This combination is usually performed by a
weighted sum, in which each task is associated with a weighting coefficient,
$k_{m}$.

The idea of Multi-task Learning (MTL) is that related tasks can create an
inductive bias, and based on that bias, a certain task will prefer some
hypothesis over the $n$ possible hypotheses [13]. To perform simultaneous
learning of multiple tasks, we will consider a hard parameter sharing
approach, in which a subset of hidden layers parameters with low-level
features are shared between all tasks, while the rest of the high-level
parameters are kept as task-specific (Figure 1). Hence, considering a multi-
task learning problem over a space $X$ and a set of tasks
$\\{Y_{m}\\}_{m\in[M]}$, the decision function of each task $m$ can be defined
as $f_{m}(x;\boldsymbol{\theta}_{sh};\boldsymbol{\theta}_{m}):X\rightarrow Y$.
Where the $\boldsymbol{\theta}_{sh}$ parameters are shared among tasks and
$\boldsymbol{\theta}_{m}$ are task-specific parameters.

Given a set of objective functions
$\mathscr{L}(f_{m}(x;\boldsymbol{\theta}_{sh},\boldsymbol{\theta}_{m}))$ that
defines the empirical risk of the task $m$, the MTL problem is commonly
formulated as the minimization of the linear combination of objective
functions:

$\min_{\scriptstyle\boldsymbol{\theta}_{sh},\atop\scriptstyle\boldsymbol{\theta}_{1},\ldots,\boldsymbol{\theta}_{M}}\sum_{m=1}^{M}\frac{1}{N}\sum_{i=1}^{N}k_{m}\mathscr{L}(f_{m}(x_{i};\boldsymbol{\theta}_{sh},\boldsymbol{\theta}_{m}),y_{i,m}),$

where $y_{i,m}$ is the label of the $m^{th}$ task of the $i^{th}$ data, $N$ is
the number of data and $k_{m}>0$ are the predefined or dynamically calculated
weighting coefficients associated with each objective function.

Although the linear combination is a simple method to solve multi-task
learning problem, this approach cannot adequately model competing tasks [2,
28]. On the other hand, multi-objective optimization is an optimization
problem that deals with multiple conflicting objective functions, which makes
it an intuitive way to deal with problems where there is no single optimal
solution for all objective functions [10, 11].

###  3.1 Feasible descent direction

According to [29], a direction $\vec{v}$ can simultaneously improve all tasks
if:

$-\nabla_{sh,m}\cdot\vec{v}\geq 0,\forall m\in{1,\dots,M},$ (1)

where $\nabla_{sh,m}$ represents the gradient vector of the $m^{th}$ task
w.r.t to the shared parameters, $sh$. Thus, given the negative gradient
vectors of three tasks, there will be several feasible descent directions. The
set of all feasible descent directions are shown in gray in Figure 2.

Figure 2: Feasible set and Pareto descent directions considering the negative
gradient vectors of $\nabla_{sh,1}$, $\nabla_{sh,2}$ and $\nabla_{sh,3}$. The
blue vectors are examples of feasible descent directions given the set of
negative gradients of the tasks. Best visualized with color.

However, not all feasible descent directions will be representative for all
objective functions. For example, considering the feasible descent directions
$\vec{d_{1}}$, $\vec{d_{2}}$, $\vec{d_{3}}$ and $\vec{d_{4}}$, we can see that
$\vec{d_{2}}$ decreases more each objective function than $\vec{d_{1}}$:

$\displaystyle\vec{d_{1}}\cdot-\nabla_{sh,1}(\boldsymbol{\theta})$
$\displaystyle<\vec{d_{2}}\cdot-\nabla_{sh,1}(\boldsymbol{\theta}),$
$\displaystyle\vec{d_{1}}\cdot-\nabla_{sh,2}(\boldsymbol{\theta})$
$\displaystyle<\vec{d_{2}}\cdot-\nabla_{sh,2}(\boldsymbol{\theta}),$
$\displaystyle\vec{d_{1}}\cdot-\nabla_{sh,3}(\boldsymbol{\theta})$
$\displaystyle<\vec{d_{2}}\cdot-\nabla_{sh,3}(\boldsymbol{\theta}).$

Similarly, the same holds for $\vec{d_{3}}$ and $\vec{d_{4}}$. On the other
hand, $\vec{d_{2}}$ and $\vec{d_{3}}$ descent directions cannot improve an
objective function without degrading the improvement of another:

$\displaystyle\vec{d_{2}}\cdot-\nabla_{sh,1}(\boldsymbol{\theta})$
$\displaystyle>\vec{d_{3}}\cdot-\nabla_{sh,1}(\boldsymbol{\theta}),$
$\displaystyle\vec{d_{2}}\cdot-\nabla_{sh,2}(\boldsymbol{\theta})$
$\displaystyle>\vec{d_{3}}\cdot-\nabla_{sh,2}(\boldsymbol{\theta}),$
$\displaystyle\vec{d_{2}}\cdot-\nabla_{sh,3}(\boldsymbol{\theta})$
$\displaystyle<\vec{d_{3}}\cdot-\nabla_{sh,3}(\boldsymbol{\theta}),$

being necessary to find a trade-off between the objectives. These solutions
are known as Pareto optimal solutions (dark gray region in Figure 2).

###  3.2 Pareto descent directions

######  Definition 1 (Pareto-optimality).

A decision vector $\mathbf{x^{*}}\in S$ is Pareto optimal if there does not
exist another decision vector $\mathbf{x}\in S$ such that:

  1. 1.

$f_{i}(\textbf{x})\leq f_{i}(\textbf{x}^{*}),\forall i\in\\{1,\dots,M\\}$;

  2. 2.

$f_{j}(\textbf{x})<f_{j}(\textbf{x}^{*}),\text{ for some
}j\in\\{1,\dots,M\\}$,

A Pareto optimal solution can be expressed as a convex combination of the
negative gradients [27, 10]. In [30], it was introduced the notion of Pareto-
stationarity as a necessary condition for Pareto-optimality:

######  Definition 2 (Pareto-stationarity).

Given M differentiable objective functions
$\mathscr{L}_{m}(\boldsymbol{\theta}),\\\
m=1,\cdots,M,\;\boldsymbol{\theta}\in{\rm I\\!R^{d}}$, a point
$\boldsymbol{\hat{\theta}}$ is considered a Pareto-stationary point, if and
only if there exists a convex combination of the gradients that is equal to
zero, i.e.:

$\sum_{i=1}^{M}k_{m}\nabla_{sh,m}(\boldsymbol{\hat{\theta}})=0$, where
$\sum_{i=1}^{M}k_{m}=1,k_{m}\geq 0\quad\forall m$.

Therefore, if a given point $\boldsymbol{\hat{\theta}}$ is not Pareto-
stationary, it will be a descent direction for all objective functions [31,
30].

Based on Definition 2, a common descent direction can be defined by searching
for weighting coefficients that minimize the norm of the convex combination
[2, 30]:

$\operatorname*{argmin}_{k_{1},\cdots,k_{M}}\bigg{\\{}\Big{\|}\sum_{m=1}^{M}k_{m}\nabla_{sh,m}\Big{\|}_{2}^{2}\bigg{|}\sum_{m=1}^{M}k_{m}=1,\,k_{m}\geq
0\quad\forall m\bigg{\\}}.$ (2)

This solution will result in the minimal-norm element in the convex hull of
the negative gradients. However, when we deal with problems of different
natures in multi-task learning, we may also have to deal with variations in
the order of gradients’ magnitude. Thus, when a task is given more importance
then another during training, the final model’s performance can be harmed.
Although the descent direction found by Equation 2 is based on an optimality
condition (Definition 1), this solution may not be representative for all
tasks when the tasks’ losses are not balanced. In this case, the solution will
tend to give more preference to the direction with least norm (red vector in
Figure 3).

Figure 3: Common descent directions considering the negative gradient vectors
of $\nabla_{sh,1}$ and $\nabla_{sh,2}$. Best visualized with color.

To avoid prioritizing only the direction of a specific task, we remove the
scale influence on the problem by considering the unit vector of each task’s
gradient, $\bar{\nabla}_{sh,m}=\nabla_{sh,m}/\|\nabla_{sh,m}\|$. Therefore,
all tasks will be equally important during the descent direction computation.
The Equation 2 can be rewritten as:

$\operatorname*{argmin}_{k_{1},\cdots,k_{M}}\bigg{\\{}\Big{\|}\sum_{m=1}^{M}k_{m}\bar{\nabla}_{sh,m}\Big{\|}_{2}^{2}\bigg{|}\sum_{m=1}^{M}k_{m}=1,\,k_{m}\geq
0\quad\forall m\bigg{\\}}.$ (3)

Solving optimization problem (3), the common descent direction can be obtained
by a convex combination of each task’s gradients with their respective
weighting coefficients, $\sum_{m=1}^{M}k_{m}\bar{\nabla}_{sh,m}$. This
direction will be close to the central region of the cone formed by the
negative gradient vectors of different tasks (orange vector in Figure 3),
which will result in the same relative decrease among them. Since this
direction was calculated considering unit vectors, we must recover the scale
factor so that the resulting vector lies in the convex hull formed by the
negative gradient vectors of the tasks. This can be done using a scale factor
$\gamma$ as proposed in [12]. Therefore, the resulting common descent
direction, $\vec{v}$, that lies in the central region between tasks can be
defined as:

$\vec{v}=\gamma\sum_{m=1}^{M}k_{m}\bar{\nabla}_{sh,m},$ (4)

where
$\gamma=\Big{(}\sum_{m=1}^{M}\frac{k_{m}}{\left\|\nabla_{sh,m}\right\|}\Big{)}^{-1}$.

However, note that this direction and all others studied so far create an
assumption that the direction found is representative considering only the
gradient analysis at a current time, without considering the history or any
indication that the directions defined during the training process were being
representative for all tasks. This means that, regardless of the method
adopted to compute a common descent direction, the direction found will tend
to prefer a specific descent direction (i.e., direction that results in lesser
or greater magnitude, central direction, etc.). Thus, to enable the correction
of the common descent direction during training, we propose to analyse the
convergence of the models by tracking the gradients’ norm of each task during
a period of $T$ iterations and adopt the idea of tensioners that pull the
common descent direction for the task that is diverging.

((a))

((b))

Figure 4: Method for changing the common descent direction between tasks. (a)
Tensioners that will pull the common descent direction for the diverging task.
(b) New defined descent direction.

###  3.3 Task tensioner

Given the central direction, $\vec{v}$, obtained through the Equation 4, the
tensions of each task can be computed according to the following equation:

$u_{m}=c_{m}\Bigg{(}\frac{\nabla_{sh,m}-\vec{v}}{\|\nabla_{sh,m}-\vec{v}\|}\Bigg{)},\quad\forall
m=1,\dots,M,$ (5)

where $\|\cdot\|$ is l2-norm, and the $c_{m}$ factor measure how much each
task must be pulled to correct the common descent direction considering a
relative variation of gradients’ norm during a predefined number of iterations
(Figure 4). Therefore, given the gradient vector of each task, the average of
gradients’ magnitude accumulated during a period of T iterations is calculated
by:

$\zeta_{m}(t)=\frac{1}{T}\sum_{i=0}^{T-1}\left\|\nabla_{sh,m}(t-i)\right\|.$

Then, the relative variation of the gradient’s norm can be obtained by the
ratio between current and previous accumulated gradient:

$\delta_{m}=\frac{\left\|\zeta_{m}(t)\right\|}{\left\|\zeta_{m}(t-1)\right\|}+\log_{10}(\mathscr{L}_{m}).$
(6)

The term $\log_{10}(\mathscr{L}_{m})$ aims to penalize tasks that have higher
loss, increasing the tension factor of the respective tasks. Therefore, given
the value of $\delta_{m}$, we can get the tension factor for each task $m$:

$c_{m}=\frac{\alpha}{1+e^{(-\delta_{m}\cdot e+e)}}+1-\alpha.$ (7)

As we want to change the descent direction only if some task is diverging, the
$c_{m}$ values are calculated so that the direction does not change too much
from the convergence region of all tasks. Thus, a constant variable,
$\alpha\in[0,1]$, is used to regulate the tension factor’s sensitivity.
Therefore, if $\alpha=0$, the resulting vector will be exactly the vector
$\vec{v}$. And, the higher the alpha value, the more the common descent
direction will be pulled towards the task that is under-performing or
diverging.

The new common descent direction among tasks can be defined by the sum of the
current common descent direction, $\vec{v}$, with the linear combination of
each task’s tension (Equation 5):

$\vec{v}_{n}=\vec{v}+\sum_{m=1}^{M}c_{m}\Bigg{(}\frac{\nabla_{sh,m}-\vec{v}}{\|\nabla_{sh,m}-\vec{v}\|}\Bigg{)},$
(8)

subject to $\nabla_{sh,m}\cdot\vec{v}_{n}\geq 0,\forall m=1,\dots,M$.

Algorithm 1 shows the proposed method that takes into account gradient history
during training to pull the common descent direction for the diverging task.

Algorithm 1 Gradient accumulation

1:for $i={0,...,I-1}$ do

2: for $m={1,\cdots},M$ do

3:
$\mathscr{L}_{m}\leftarrow\mathscr{L}(f_{m}(x_{i};\theta_{sh},\theta_{m}),y_{i,m})$
$\triangleright$ Compute task-specific loss

4: $\nabla_{m}\leftarrow\frac{\partial\mathscr{L}_{m}}{\partial\theta_{m}}$
$\triangleright$ Gradient descent on task-specific parameters

5:
$\nabla_{sh,m}\leftarrow\frac{\partial\mathscr{L}_{m}}{\partial\theta_{sh}}$
$\triangleright$ Gradient descent on shared parameters

6: end for

7: Solve (3) to obtain $k_{m}$ and find common descent direction $\sum
k_{m}\bar{\nabla}_{sh,m}$

8: if $i\geq T$ then

9: Compute new common descent direction using Algorithm 2.

10: else

11: $\vec{v}_{n}\leftarrow\vec{v}$

12: end if

13: Update $\theta_{sh}$ w.r.t $\vec{v}_{n}$ with chosen optimizer;

14: Update $\theta_{m},\;\forall m$ with chosen optimizer.

15:end for

Algorithm 2 Compute task tension

1:$i\geq T$

2:for $m={1,\cdots},M$ do

3: Compute relative change, $\delta_{m}$ (6)

4: Compute tension factor, $c_{m}$ (7)

5: Compute task tension, $u_{m}$ (5)

6:end for

7:$\vec{v}_{n}\leftarrow\vec{v}+\sum_{m=1}^{M}u_{m}$ $\triangleright$ Update
common descent direction (8)

8:return $\vec{v}_{n}$

##  4 Experiments

To analyze the behavior of different descent direction’s methods, and ensure
that it is not influenced by the network architecture or other variables,
ablation studies are performed considering a Multi-layer Perceptron (MLP) to
learn problems of multiple logical operators. Afterward, the evaluation is
performed in an image domain application considering the classification and
reconstruction problems in the MNIST dataset adapted to perform multi-task
learning.

###  4.1 Experimental setup and metrics

The experiments were conducted using a machine with Intel Core i7-10750H, 16GB
RAM, and RTX 2060 6GB. For all experiments described in this section, it was
considered the values of $\alpha=0.3$ (Equation 7) and period ${T}={10}$
(Equation 6).

For quantitative evaluations, we used the mean squared error (MSE) and
classification accuracy:

$\displaystyle MSE$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\big{(}y_{i}-\hat{y}_{i}\big{)}^{2},$
$\displaystyle Accuracy$ $\displaystyle=\frac{TP+TN}{TP+FP+FN+TN},$

where $\hat{y}$ is the estimated value, $y$ is the ground truth, $n$ is the
total number of pixels in the evaluated images. TP, TN, FP, and FN are the
numbers of true positive, true negative, false positive, and false negative
classifications, respectively.

###  4.2 Ablation Studies

In these ablation studies, the simultaneous learning of the logical operators
XOR and AND was evaluated. We adopted this study because we know exactly how
many neurons are needed to generate the hyperplanes that separate the sets of
tasks’ data in the feature space. Thus, we eliminate external influences that
generate uncertainty if it was not possible to obtain a better result due to
the architecture or problem modeling, allowing to evaluate only the descent
direction methods. Furthermore, the problems are both complementary and
conflicting. Since in shared neurons, the AND task will try to separate the
feature space into two regions, while the XOR task will need to separate the
feature space into three regions. Figure 5(a) shows a MLP with two shared
neurons and two task-specific output neurons. The hyper-planes necessary to
separate the data of both tasks are shown in Figure 5(b).

((a))

((b))

Figure 5: (a) MLP for AND and XOR tasks. (b) Feature space and hyper-planes
necessary to separate the input data. Best visualized with color.

The experiments were performed considering the common descent directions
computed according to the methods of uniform weighting, minimum-norm [2],
central direction [12], and gradient modification with tasks’ tensors
(proposed). To analyse the descent methods considering tasks with different
orders of magnitude, the annotations of XOR task were multiplied by a constant
$c\geq 1$ to change the problem’s scale.

Each model was trained using mean squared error (MSE) loss function and stop
condition with the limit of 3,000 epochs or an error smaller than
$1\mathrm{e}{-3}$. Since training results can vary from run to run, each
method was trained 50 times with a different seed value to get deterministic
random data. Furthermore, the models were trained considering a set
$lr=\\{5\mathrm{e}{-5},1\mathrm{e}{-4},5\mathrm{e}{-3},1\mathrm{e}{-3},5\mathrm{e}{-2},1\mathrm{e}{-2}\\}$
of learning rates and the Stochastic Gradient Descent (SGD) optimization
algorithm with momentum. Thus, the reported results are the models that best
separated the input data.

Similar gradients’ magnitude: Table 1 presents the average performance of each
method considering $c=1$, i.e., without much difference in the gradients’
magnitude. The first two rows correspond to the models separately trained for
each task, which we define as our baseline.

As can be seen, all methods were able to create hyperplanes capable of
separating the data for both tasks and without significant difference in the
convergence time. With the exception of the method proposed by [2], which has
a slightly slower convergence compared to other methods because it prioritizes
the task direction with a lower gradient magnitude.

Regarding the use of the tensors proposed in the present work to change the
common descent direction according to each task’s learning progress, the
performance must be similar or better than the unaltered directions. As can be
seen, the proposed method resulted in a faster convergence. In the case of the
central direction, the use of tensors reduced the number of epochs needed from
574.44 to 443.02 epochs, resulting in the method that obtained the fastest
convergence. And when we apply the tensors in the Sener method, which was the
method that had the slowest convergence, it was possible to reduce from 758.56
to 493.16 epochs.

| Descent direction
---
method
| Convergence
---
rate (%)
| Training epochs
---
average
Single AND | 100.00 | $351.70\pm 2.62$
Single XOR | 100.00 | $871.20\pm 52.95$
Uniform weighting | 100.00 | $484.86\pm 18.12$
Min-norm | 100.00 | $758.56\pm 137.15$
Central dir. | 100.00 | $574.44\pm 43.37$
Min-norm + tensor | 100.00 | $493.16\pm 27.94$
Central dir. + tensor | 100.00 | $\mathbf{443.02\pm 21.14}$
Table 1: Average performance of evaluated methods without changing the
problems’ scale.

Different gradient’s magnitude: To analyze the behavior of descent direction
methods considering problems closer to which the proposed method seeks to
solve, i.e., when we have tasks with variation in the gradients’ magnitude due
to the tasks having different types of nature or even loss functions, the
annotations of the XOR task were multiplied by the constant $c=10$ to simulate
this variation.

((a)) Uniform

((b)) Minimum-norm

((c)) Central direction

((d)) Minimum-norm + tensor

((e)) Central direction + tensor

Figure 6: Learning curves of different descent direction methods. Pink and
light blue curves represent the angles of AND and XOR tasks in relation to the
common descent direction, respectively. Red and dark blue curves represents
the gradients’ norm of AND and XOR tasks, respectively. Finally, orange curves
represents the gradient’s norm of the common descent direction. Best
visualized with color.

The behaviour of each descent direction method can be observed in the learning
curves of Figure 6. When uniform weighting is used, we can see that the common
direction gives preference to the task with greater magnitude, in which the
angle between the common descent direction and the XOR task is close to zero
(light blue curve in the Figure 6(a)). This can make the simultaneous learning
to be substantially accomplished in relation to a single task, and the common
descent direction does not be feasible for the other tasks. As in the case of
the AND task, where in some iterations the angle between its direction and the
defined common descent direction is greater than 90 degrees (Definition 1). On
the other hand, when we have tasks with difference in the gradient scale, the
method proposed by [2] gives preference to the gradient direction with the
lowest norm, making the model converge faster in relation to this task, which
can hinder the learning of the other tasks or slow down the learning process
(Figure 6(c)).

The central direction is the direction close to the central region of the cone
formed by the negative gradients, resulting in the same angle between tasks
(light blue and pink curves are overlapping in Figure 6(b)). Therefore, this
direction tends to give equal importance to all tasks, regardless of the
gradients norm. Figures 6(e) and 6(d) show the minimum-norm and central
directions changed with the use of tensors, respectively. As it can be seen,
in the first iterations the central direction method assigns the same
importance to both tasks, resulting in the common descent direction having the
same angle between the tasks. However, we can observe that, in addition to the
XOR task norm being higher, it increases significantly compared to another
task (dark blue curve in the Figure 6(e)). From $T=10$ iterations, the use of
tensors starts to dynamically change the common descent direction in relation
to the task that is diverging. Thus, as soon as the norm of both tasks starts
to reduce simultaneously, the common descent direction starts to give
practically the same importance to the tasks (after approximately 230
iterations), with a small variation due to the penalty factor in Equation 6.
The same behavior can be observed when we apply the use of tensors in the
direction of the minimum-norm method, proving that the proposed method can
adapt to different descent directions.

| Descent direction
---
method
| Convergence
---
rate (%)
| Training epochs
---
average
Single-task AND | 100.00 | $351.70\pm 2.62$
Single-task XOR | 100.00 | $2{,}979.00\pm 0.80$
Uniform weighting | 46.00 | $1{,}831.60\pm 810.81$
Min-norm | 100.00 | $1{,}998.44\pm 316.38$
Central dir. | 100.00 | $328.50\pm 25.45$
Min-norm + tensor | 100.00 | $210.38\pm 14.96$
Central dir. + tensor | 100.00 | $\mathbf{188.20\pm 24.29}$
Table 2: Average performance of evaluated methods considering $c=10$ for XOR
task.

The average of the $50$ trainings are shown in the Table 2. Since only the
scale of the XOR problem was changed, the performance of the model trained
separately for AND task remains the same. In this experiment, the impact of
using different descent directions’ methods becomes more visible. For the
single-task XOR, the model needs more iterations to converge when the scale’s
problem changes. When performed the simultaneous learning of both tasks, the
uniform weighting resulted in the worst performance, being able to separate
the feature space only in $46\%$ of the $50$ trainings, and, in the best case,
needing more than $1,000$ epochs for convergence. Regarding the methods of
central direction and minimum-norm [2, 12], both converged in all trainings.
However, as can be seen in the fourth row of Table 2, the method proposed by
[2] needs more iterations. Finally, when we use the tensors to change the
central and the minimum-norm direction, we again improve the convergence of
the models, significantly reducing the number of epochs compared to the
minimum norm method and obtaining the best performance among the evaluated
methods.

###  4.3 Handwritten Digit Problem

Figure 7: Images examples from the MNIST dataset modified for multi-task
learning. Images have a size of $28\times 28$ pixels.

In order to validate that the proposed method is independent of a specific
application, we performed the experiment in the image domain considering the
dataset of handwritten digits [32]. To turn it into a multi-task learning
problem, we use the idea of adapting the MNIST dataset [33, 2], in which each
image is composed by the overlapping of two digit images (Figure 7). For this
purpose, two randomly selected 28x28 images were placed in the upper-left and
lower-right corners of a 32x32 image. Then, the resulting image was resized to
28x28. The training and validation images were generated splitting the train
set images, and the test images using the test set from MNIST dataset. A total
of 50,000 training images, 10,000 validation images and 10,000 test images
were generated.

Figure 8: Multi-task learning architecture for upper-left digit classification
and lower-right digit reconstruction. Best visualized with color.

In the original proposal, the problem is to classify the digits in the upper
left and lower right corners. However, since all digits are from the same
dataset, note that both tasks have the same nature, changing only the position
where they are located in the image. In order to change the scale and nature
of the problem, we defined the first task as classification of the upper-left
digit and the second as an autoencoder to reconstruct the lower-right digit.
The intuition behind the problem is to keep tasks with complementary
information, but make them conflicting during training. Therefore, while the
autoencoder will try to compress the input data into a latent space and reduce
the reconstruction error, the other task will try to extract representative
features to correctly classify the upper-left digit.

We use the modification of the LeNet architecture shown in Figure 8 [32, 2].
The models were trained considering the Stochastic Gradient Descent (SGD)
optimization algorithm with momentum and a set $lr=$ {$1\mathrm{e}{-3}$,
$5\mathrm{e}{-3}$, $1\mathrm{e}{-2}$, $5\mathrm{e}{-2}$, $1\mathrm{e}{-1}$} of
learning rates. Thus, we selected the model that obtained the best validation
performance to report the results.

The Negative Log-likelihood (NLL) loss was used for the classification task,
and the MSE loss for the autoencoder. Each model was trained ten times with a
different seed value to get deterministic random data. Furthermore, the
training process was interrupted only when there was no improvement in
validation metrics during 10 epochs.

| Descent direction
---
method
| Upper-left digit
---
Classification accuracy $\uparrow$
| Lower-right digit
---
Reconstruction error $\downarrow$
Single-task | $0.9639\pm 0.0024$ | $0.0181\pm 0.0017$
Uniform weighting | $\mathbf{0.9714\pm 0.0018}$ | $0.0418\pm 0.0021$
Min-norm | $0.7393\pm 0.0213$ | $0.0552\pm 0.0042$
Central dir. | $0.8473\pm 0.0107$ | $0.0350\pm 0.0010$
Min-norm + tensor | $0.9627\pm 0.0015$ | $\mathbf{0.0181\pm 0.0005}$
Central dir. + tensor | $0.9649\pm 0.0015$ | $0.0185\pm 0.0003$
Table 3: Average results of the digit classification and reconstruction tasks
in the test set. We report the accuracy for classification (higher is better)
and MSE for reconstruction task (lower is better).

Table 3 shows the average performance of the models evaluated in the test set.
The first row represents the accuracy and reconstruction error of the models
separately trained for each task. Therefore, we use them as a baseline to
evaluate the multi-task models. As presented in the ablation studies, the
common descent direction obtained by the uniform weighting tends to give
greater importance to the task with the highest gradient norm. In this
experiment, uniform weighting was the method that resulted in the best
accuracy for the classification problem (second row). However, in addition to
extracting relevant features to classification, the shared encoder must be
able to compress the input data (${\rm I\\!R}^{784}$) into a latent space
representation (${\rm I\\!R}^{50}$), so that the task-specific decoder can
then reconstruct this representation. In this case, since the shared encoder
was mainly optimized in relation to the classification problem, the uniform
weighting reconstruction error was greater than the individual task.

The central direction method (fourth row) obtained the result a little better
than the minimum-norm method (third row). However, both also had a lower
performance than separately training each model.

Figure 9: Plot of all trained models with the average performance of single
tasks as baseline. The proposed method resulted in a good balance between
tasks, resulting in a performance similar or even better than single-task
models. Upper-right is better. Best visualized with color.

Finally, the last two rows present the use of tensors to modify the minimum-
norm and central directions. Although the proposed method did not achieve the
best performance separately for each task, it resulted in the best balance
among the evaluated methods. This highlights that there are many trade-offs
between conflicting tasks, and the proposed method was able to find a balance
that maximizes the performance of all of them. Thus, while we maintain a
similar performance to those models trained dedicatedly for each task, we also
have the advantage of reducing the use of computational resources and
inference latency, since most of the model parameters are shared between
tasks. This can be better observed in Figure 9, which shows the performance of
the ten trained models of each method.

To simplify the visualization, we inverted the $y$-axis of the plot. Thus, the
methods with the best results are those closest to the upper-right corner.

##  5 Conclusion

In this work, we address the simultaneous learning of conflicting tasks that
have differences in the magnitude scale. Our method is motivated by the
difficulty in performing simultaneous learning of multiple tasks in a hard
parameter sharing approach without harming the performance of tasks. To this
end, we propose to dynamically change the common descent direction between
multiple tasks considering their convergence behavior, ensuring that the new
direction will be feasible for all of them.

Experimental results showed that the proposed method outperforms the state-of-
the-art methods in simultaneous learning of conflicting tasks. Thanks to the
use of temporal information to properly adjust the importance of each task, it
was possible to maximize performance without harming any task. Furthermore,
the proposed method provided consistent performance with low variance in the
tested problems.

Since our method is independent of a specific task, it is possible to extend
our approach to other applications, such as robotics or intelligent vehicles,
where several related tasks must be estimated simultaneously with reduced
inference time and computational resource usage. In addition to multi-task
learning, we believe that it is also possible to use the proposed method to
explore other areas where it is necessary to find a trade-off between learning
multiple tasks, such as to deal with the problem of catastrophic forgetting in
continuous learning, restricting the learning of a new task to be according to
a descent direction that is representative for the previously learned tasks.

Acknowledgement: The authors would like to express their gratitude to Luiz
Ricardo Takeshi Horita for insightful discussions and feedback.

Funding: This work was supported by the São Paulo Research Foundation - FAPESP
[grant number 2019/03366-2] and the Coordination of Improvement of Higher
Education Personnel - Brazil - CAPES [Finance Code 001]. This work was also
supported in part by the Brazilian National Council for Scientific and
Technological Development - CNPq [grant number 465755/2014- 3], by CAPES
[grant number 88887.136349/2017-00], and the FAPESP [grant number
2014/50851-0].

## References

  * [1] R. Caruana, Multitask Learning, Ph.D. thesis, Carnegie Mellon University (1997).
  * [2] O. Sener, V. Koltun, Multi-task learning as multi-objective optimization, in: Advances in Neural Information Processing Systems, Vol. 2018-Decem, 2018, pp. 527–538.
  * [3] R. Cipolla, Y. Gal, A. Kendall, Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vol. abs/1705.0, IEEE, 2018, pp. 7482–7491.
  * [4] Z. Chen, V. Badrinarayanan, C. Y. Lee, A. Rabinovich, GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks, 35th International Conference on Machine Learning, ICML 2018 (2018) 1240–1251.
  * [5] A. T. M. Nakamura, V. Grassi, D. F. Wolf, An effective combination of loss gradients for multi-task learning applied on instance segmentation and depth estimation, Engineering Applications of Artificial Intelligence 100 (2021) 104205\.  doi:10.1016/j.engappai.2021.104205.
  * [6] Y. Mao, S. Yun, W. Liu, B. Du, Tchebycheff Procedure for Multi-task Text Classification, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Stroudsburg, PA, USA, 2020, pp. 4217–4226.  doi:10.18653/v1/2020.acl-main.388.
  * [7] P. Liu, X. Qiu, H. Xuanjing, Recurrent neural network for text classification with multi-task learning, in: IJCAI International Joint Conference on Artificial Intelligence, Vol. 2016-Janua, 2016, pp. 2873–2879.  arXiv:1605.05101.
  * [8] L. Xiao, H. Zhang, W. Chen, Gated multi-task network for text classification, in: NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, Vol. 2, 2018, pp. 726–731.  doi:10.18653/v1/n18-2114.
  * [9] J. Uhrig, M. Cordts, U. Franke, T. Brox, Pixel-level encoding and depth layering for instance-level semantic labeling, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2016) 14–25.
  * [10] K. Miettinen, Nonlinear Multiobjective Optimization, Springer US, 1998.
  * [11] M. Ehrgott, Multicriteria optimization, 2nd Edition, Springer-Verlag, Berlin/Heidelberg, 2005.  doi:10.1007/3-540-27659-9.
  * [12] A. Katrutsa, D. Merkulov, N. Tursynbek, I. Oseledets, Follow the bisector: a simple method for multi-objective optimization (2020).  arXiv:2007.06937.
  * [13] R. A. Caruana, Multitask Learning: A Knowledge-Based Source of Inductive Bias, in: Machine Learning Proceedings 1993, 1993, pp. 41–48.
  * [14] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel, Backpropagation Applied to Handwritten Zip Code Recognition, Neural Computation 1 (4) (1989) 541–551.  doi:10.1162/neco.1989.1.4.541.
  * [15] Y. Zhang, Q. Yang, A Survey on Multi-Task Learning, IEEE Transactions on Knowledge and Data Engineering (2021) 1–1arXiv:1707.08114, doi:10.1109/TKDE.2021.3070203.
  * [16] S. Vandenhende, S. Georgoulis, B. De Brabandere, L. Van Gool, Branched Multi-Task Networks: Deciding What Layers To Share (apr 2019).  arXiv:1904.02920.
  * [17] I. Misra, A. Shrivastava, A. Gupta, M. Hebert, Cross-Stitch Networks for Multi-task Learning, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2016-Decem, 2016, pp. 3994–4003.
  * [18] S. Ruder, J. Bingel, I. Augenstein, A. Søgaard, Latent Multi-Task Architecture Learning, Proceedings of the AAAI Conference on Artificial Intelligence 33 (2019) 4822–4829.  arXiv:1705.08142, doi:10.1609/aaai.v33i01.33014822.
  * [19] P. Liu, X. Qiu, X. Huang, Adversarial multi-task learning for text classification, in: ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), Vol. 1, Association for Computational Linguistics, 2017, pp. 1–10.
  * [20] S. Chennupati, G. Sistu, S. Yogamani, S. Rawashdeh, AuxNet: Auxiliary Tasks Enhanced Semantic Segmentation for Automated Driving, in: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Vol. 5, 2019, pp. 645–652.
  * [21] M. Teichmann, M. Weber, M. Zöllner, R. Cipolla, R. Urtasun, MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving, in: IEEE Intelligent Vehicles Symposium, Proceedings, 2018, pp. 1013–1020.
  * [22] D. Sanchez, M. Oliu, M. Madadi, X. Baro, S. Escalera, Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation, in: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), IEEE, 2019, pp. 1–8.
  * [23] S. Liu, E. Johns, A. J. Davison, End-To-End Multi-Task Learning With Attention, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 1871–1880.
  * [24] C. Li, J. Yan, F. Wei, W. Dong, Q. Liu, H. Zha, Self-Paced Multi-Task Learning, 31st AAAI Conference on Artificial Intelligence, AAAI 2017 (2016) 2175–2181arXiv:1604.01474.
  * [25] M. Guo, A. Haque, D. A. Huang, S. Yeung, L. Fei-Fei, Dynamic Task Prioritization for Multitask Learning, in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2018, pp. 282–299.
  * [26] X. Lin, H.-L. Zhen, Z. Li, Q.-F. Zhang, S. Kwong, Pareto multi-task learning, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems, Vol. 32, Curran Associates, Inc., 2019.
  * [27] K. Harada, J. Sakuma, S. Kobayashi, Local search for multiobjective function optimization: Pareto descent method, in: GECCO 2006 - Genetic and Evolutionary Computation Conference, Vol. 1, 2006, pp. 659–666.
  * [28] D. Mahapatra, V. Rajan, Multi-Task learning with user preferences: Gradient descent with controlled ascent in pareto optimization, in: 37th International Conference on Machine Learning, ICML 2020, Vol. PartF16814, 2020, pp. 6553–6563.
  * [29] J. Fliege, B. F. Svaiter, Steepest descent methods for multicriteria optimization, Mathematical Methods of Operations Research 51 (3) (2000) 479–494.  doi:10.1007/s001860000043.
  * [30] J. A. Désidéri, Multiple-gradient descent algorithm (MGDA) for multiobjective optimization, Comptes Rendus Mathematique 350 (5-6) (2012) 313–318.  doi:10.1016/j.crma.2012.03.014.
  * [31] S. Schäffler, R. Schultz, K. Weinzierl, Stochastic method for the solution of unconstrained vector optimization problems, Journal of Optimization Theory and Applications 114 (1) (2002) 209–222.  doi:10.1023/A:1015472306888.
  * [32] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2323.  doi:10.1109/5.726791.
  * [33] S. Sabour, N. Frosst, G. E. Hinton, Dynamic routing between capsules, in: Advances in Neural Information Processing Systems, Vol. 2017-Decem, 2017, pp. 3857–3867.  arXiv:1710.09829.