# Sequential IoT Data Augmentation using Generative Adversarial Networks

###### Abstract

Sequential data in industrial applications can be used to train and evaluate
machine learning models (e.g. classifiers). Since gathering representative
amounts of data is difficult and time consuming, there is an incentive to
generate it from a small ground truth. Data augmentation is a common method to
generate more data through a priori knowledge with one specific method, so
called generative adversarial networks (GANs), enabling data generation from
noise. This paper investigates the possibility of using GANs in order to
augment sequential Internet of Things (IoT) data, with an example
implementation that generates household energy consumption data with and
without swimming pools. The results of the example implementation seem
subjectively similar to the original data. Additionally to this subjective
evaluation, the paper also introduces a quantitative evaluation technique for
GANs if labels are provided. The positive results from the evaluation support
the initial assumption that generating sequential data from a small ground
truth is possible. This means that tedious data acquisition of sequential data
can be shortened. In the future, the results of this paper may be included as
a tool in machine learning, tackling the small data challenge.

Index Terms—  Data Augmentation, Sequential Data, GAN, Small Data, IoT

##  1 Motivation

In the Internet of Things (IoT), where devices are connected and share
information, sequential data, in the form of time-based sensor readings, are
used in applications such as predictive maintenance [1] (e.g. in the form of
malfunction detection), performance estimation or home automation. Being able
to predict malfunctions can decrease maintenance costs and reduce upkeep times
[2], while home automation increases the control inhabitants have about their
IoT devices [3]. Performance estimation, on the other hand, can be used to
predict outcomes, depending on changes of specific inputs, comparable to
regression. Since ML and especially deep learning (DL) need substantial
amounts of labelled, representative data to train its models, a tedious data
acquisition phase is typically needed in order to utilize these applications
[4, p. 163]. Especially when dealing with private or even sensitive data (e.g.
home automation), as well as machines which have a rare occasion of
malfunctions (predictive maintenance), collecting a significant amount of data
conventionally is difficult. In some of these cases, data augmentation can be
applied through the use of priori knowledge to effectively increase the
starting data amount, minimizing acquisition delays. Looking at image data,
basic approaches to data augmentation include patch extraction, or translation
and rotation [5].

Apart from classical data augmentation, deep neural networks can be used to
generate data by training convolutional neural networks (CNNs), as proposed by
Lecun et al. [6], to generate data from noise. These networks are trained by
updating transposed convolutional kernels to generate realistic images from
some input like noise, which are called ”generators”. To actually train the
network in a meaningful way, another network, a so called ”discriminator” or
”critic” is constructed which tries to classify if an image is real or
generated. The composition of these networks, so called generative adversarial
networks (GANs), proposed by Goodfellow et al. [7], have had widespread
success in the generation of image data but are rarely applied to sequential
1D data. This paper is going to investigate the possibility of generating 1D
sequential data by first of all converting it into 2D, with the goal of
exploiting locality (local features) in the original signal and a following
GAN implementation.

### Related Work

Conventional data augmentation typically enriches the dataset through a priori
knowledge. Since deep (representation) neural networks are able to learn
distributed representations by themselves [4, p. 4], they can be used to find
representations in data and, in the context of GANs, apply this knowledge to
construct new data from the previously learned representations. Therefore, a
prior knowledge is no longer needed since GANs learn to apply it automatically
[7]. Similarly, autoencoders, an unsupervised representation learning model,
learn representations from the original data $x$ to a reconstruction $P(Q(x))$
through an internal code $Q(x)$. Their target is to get $P(Q(x))$ to be
similar to $x$ with the intermediate step $Q(x)$ having less neurons than $x$
and $P(Q(x))$ have features. Variational autoencoders (VAEs) add noise ($z$)
to the internal code $Q(x|z)$, which, if reconstructed into some $P(Q(x|z))$,
results in an augmented form of the original $x$ [8]. Although VAEs are
typically applied to image data, they can also be applied to sequential data
if this data is converted using the process introduced in section 2.

Another way of generating sequential data is to apply recurrent neural
networks (RNNs) or long-short term memory networks (LSTMs). These networks
take a sequence of a specific size as an input and generate data following the
given sequence. More formally, a RNN, trained on a series of inputs
$(x_{1},x_{2},...,x_{T})$, uses the outputs of the applied test data
$(o_{1},o_{2},...,o_{t})$ in order to generate the next datapoint of the
sequence ($softmax(o_{t})$). This newly generated data then gets added as the
latest output $o_{t+1}$ [9]. Although RNNs are designed mainly for sequential
data, CNN based networks can utilize an advantage if the target sequential
data is periodic and local 2D features are important to the dataset as in a
specific energy consumption profile throughout multiple sample periods [6].

### Contribution

After introducing the chosen time based test dataset, the paper shows the
conversion of the sequential dataset to 2D and why this conversion intuitively
makes sense as well as how much neurons can be economized by this conversion.
Afterwards, the dataset is augmented using GANs and evaluated using a process,
combining the augmented dataset with a CNN and labelled data. The main
contribution of this paper is to check the feasibility of augmenting 1D
sequential data from the IoT sector using 2D GANs.

##  2 Dataset and Preprocessing

Household energy consumption data is commonly obtained by smart meters to
analyse and control smart grid loads. In order to enable non-intrusive load
monitoring (NILM), energy consumption data is commonly sampled in a short
period, from seconds to several minutes, in combination with energy
consumption profiles of household appliances. In addition to enabling the
Smart Grid, Smart Home and IoT, data like this, in combination with NILM, can
also be used for criminal intents to e.g. analyse if a certain household is
currently inhabited or if the inhabitants are on vacation [10]. Studies based
on the dataset used in this paper have aimed to investigate the possibilities
of finding security and privacy issues in this kind of smart meter data [11].

The dataset consists of 869 Austrian households with their smart meter
readings recorded in a 15-min interval. This leads to $37920$ datapoints per
household per year. For each household, an equipment list denoting the
presence of some consumption-heavy devices (e.g. swimming pool pump, sauna,
home cinema) is also available.

As shown by Ferner et al. [11], this dataset can be used to analyse if a house
has a swimming pool or not, which was chosen as a demo implementation and
comparison for this paper. The main difficulty with this dataset is that the
data labelled as pool is vastly under-represented, making the training of a
classifier CNN complicated, which can be seen in the reported precision value
of $0.719$ [11]. Since this power consumption data is a time based 1D signal
showing how much power was utilized each $15$ minutes, a transformation into
2D can be applied. Using the daily periodicity of the signal, it was
transformed by taking each period, in this case each day and putting it into a
column. Repeating this process for each of the $395$ days converts the 1D
signal into a 2D signal that can be interpreted as a heatmap [11].

The transformation into this 2D signal is important since it allows the
exploitation of locality through convolutions. The paper aims to investigate
if a GAN working with 2D convolutions is capable of generating 1D sequential
data, enabling use of sophisticated methods of CNNs like parameter sharing,
pooling and striding. Interpreting this 2D signal as pixels in an image, it
can intuitively be shown by Fig. 1 that a 1D convolution applied to a 1D
signal needs to look at more pixels than a 2D convolution on a 2D signal,
further reducing model complexity through convolutional parameter sharing.
This outcome results from the way 2D convolutions make use of locality. In
more detail, relationships between periods can be exploited along two axis
instead of one, resulting in a reduction of needed neurons, in the context of
CNNs. If some object to determine via a convolution is placed in the middle of
a 2D representation and the distance from the object to the border of the
image is called $d$, this reduction is $2d$. If one wants to use these
advantages when working with sequential data, a transformation into 2D should
be performed.

Fig. 1: Reduction of the convolutional kernel size through a 1D to 2D
conversion

##  3 Sequential Data Generation

There are multiple methods of generating sequential data like conventional
data augmentation through a priori knowledge or automated methods like VAEs,
RNNs or GANs. GANs have already been used with success in a wide range of
image applications as shown by Denton et al. [12] and Radford et al. [13]. A
GAN is an adversarial network consisting of a generator which tries to
generate fake data that is similar to real data (like counterfeit money) as
well as a discriminator, which aims to classify real data from fake (like
police, checking the real or fake money). This adversarial structure enables
training on images, by exploiting established CNN techniques like convolutions
in the discriminator and transposed convolutions in the generator [7].

Since the chosen energy consumption data has a very high variability with a
heavily under-represented class, applying a Deep Convolutional GAN
(DCGAN)111adopted from Naoki Shibuya deep-learning github did not supply any
meaningful results. The main problem during training the DCGAN, vanishing
gradients, was successfully countered by switching from the Kullback Leibler
Divergence (KLD) to the Earth Mover’s Distance (EMD). Additionally, the
transfer function of the classifying discriminator neuron is removed and a
gradient penalty is introduced, resulting in a WGAN [14]. The used WGAN
generator consists of $128$ neurons in the input layer (times the input
dimensions) and $3$ transposed convolutional layers with $64$, $32$ and $1$
$5\times 5$ kernel respectively. Batch norm, leaky ReLUs as well as padding is
applied in the generator. The critic consists of $3$ convolutional layers with
$32$, $64$ and $128$ convolutions each with $5\times 5$ kernels. $2\times 2$
Striding as well as padding is applied with leaky ReLUs as transfer functions.
After these convolutional layers, the network flattens, followed by $2$ fully
connected layers with the first consisting of $1024$ neurons and the second,
being the classifier layer, with only $1$ neuron. This layer outputs a score
on how good of a fake the generated image is. Adam is chosen as an optimizer
with a starting learning rate of $0.0001$ and a learning rate decay of $0.5$.
The chosen batch size is $4$ and each GAN was trained for $220$ epochs with a
gradient penalty of $10$. After training each GAN, it is used to generate
$5000$ images labelled as either pool or non-pool. As Fig. 3 and Fig. 3 show,
this WGAN, adopted from the Keras WGAN implementation with updated input
dimensions, was subjectively able to learn how to generate household energy
consumption heatmaps. Since two WGANs were trained, one with only swimming
pool data and one without swimming pool data, two generators resulted from
this experiment.

(a) Non-Pool 1

(b) Non-Pool 2

(c) Pool 1

(d) Pool 2

Fig. 2: Real Non-Pool/Pool Heatmaps

(e) Non-Pool 1

(f) Non-Pool 2

(g) Pool 1

(h) Pool 2

Fig. 3: Generated Non-Pool/Pool Heatmaps

In order to quantitatively evaluate these generators additionally to a
subjective analysis, an evaluation workflow was designed. The target was to
train a generator with only a subset of all data and then use only the output
of this generator to train the classifier network, from [11]. Therefore, the
classifier CNN is trained only with fake data, and evaluated against real
data. The assumption is that, if this is successful (successful as in having
an f-score $>0.5$) the generator learned to generate some important
representations automatically, resulting in a useful training process for
classifier networks like CNNs.

##  4 Results and Discussion

Fig. 3 shows selected outputs from one of the in Section 3 introduced
generators. Disregarding the greyish background, the generated household
energy consumption heatmaps are very similar to the real data. The generated
pool data shows that, about half the time, the usual daily information (noise)
is lost but other than that it successfully generates noisy, pool like
structures (filled rectangles) in the beginning half of the data (swimming
seasons). This seems to suggest that, even with the low amount of training
data ($58$ pools), the generator was able to learn the representation of
multiple, rectangular shapes. Qualitatively, this suggests that generating
household energy consumption data with these kinds of noisy data and simple
rectangular features is possible.

Fig. 4: Augmented (GAN) classification result

This statement is reinforced by the classification results which can be seen
in Fig. 4. These classification results stem from the in section 3 described
evaluation process. In detail it is the precision, recall and f-score of a CNN
trained by GAN generated data, evaluated against real data. The boxplot shows
the macro average over these performance measures of class 0 (without pools)
and class 1 (with pools). Since the pool class is heavily under-represented,
accuracy should not be used as an evaluation metric. The evaluation shows
almost perfect classification for the non-pool class ($0.95$ \- $1$ f-score)
and about $0.31$ f-score for class $1$. It should be noted that a perfect
f-score for class $1$ is impossible since the pool pump feature classes are
faulty in some cases and do not appear in every pool dataset. The macro
average over class $0$ and $1$ results in a F-score of about $0.6$, precision
of $0.62$ and recall of about $0.57$. These values are obtained from 64
independent trials and show that the CNN was able to learn some
representations from only generated data. However, the variation of
classification scores is substantial.

Macro Average
---
F1 | Prec | Rec
0.88 | 0.84 | 0.93
0.83 | 0.87 | 0.79
0.82 | 0.91 | 0.77
0.82 | 0.91 | 0.77
0.81 | 0.78 | 0.84
Table 1: Top five CNN results

Table 1 shows the top 5 CNN classification scores. These generators seem to
have learned their respective representations very well. However, due to the
high variance in the results, it is still inconclusive if these results stem
from the random training through back propagation or possible sampling.

Using GANs for sequential data generation seems unintuitive since RNNs,
especially, LSTMs are designed with sequential data in mind [9]. However, as
shown in this paper, in some cases, sequential data can be transformed into 2D
by exploiting periodic behaviour. In this 2D space, GANs can be used to
exploit locality through convolutions and apply further CNN methods like
pooling and striding. In these cases, GANs can be used to generate data from
noise with a fixed output size, trained from a dataset with distinct
representations.

##  5 Conclusion

This paper investigates the possibility of augmenting 1D sequential data using
GANs, which is especially interesting on data with a long gathering time
(through rare anomaly occasions or a long sequence duration) or on sensitive
data with privacy concerns. In order to do so, the paper discusses standard
data augmentation methods as well as a set of related work in RNNs and VAEs.
After that, the demo dataset from the IoT sector as well as GANs are
introduced and data augmentation using GANs is performed. The evaluation of
this process shows success in a qualitative as well as quantitative way.
However, there is still a huge variation in performance measures, probably due
to the small experiment size of $64$ independent trials, that will be
increased in future experiments using openly accessible data.

One further step is to evaluate the introduced system against a generative
LSTM implementation by comparing their classification scores based on the same
input data. Another further step would be to convert the sequential 1D data
into 2D, followed by obtaining the local information and converting it back
into a 1D space. One example on how to reconvert the data to 1D could be by
applying pseudo-hilbert curves [15] or line scans of the whole image,
separated into local fields. This would theoretically enable a combination of
the advantages of CNNs (locality) and RNNs (sequential relations) by applying
this preprocessing to a generative LSTM.

In conclusion, the results of this paper suggest that GANs can be used as a
Data Augmentation tool in the ML workflow even if working with sequential
data. This leads to shorter data acquisition phases, in e.g. sensor based ML
projects for the IoT, and as a possible help to approach the small data
challenge. A further investigation of the introduced methods and improvements
for the IoT sector is advised.

## References

  * [1] Kanawaday A. and Sane A.,  “Machine learning for predictive maintenance of industrial machines using iot sensor data,”  in 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), 2017, pp. 87–90.
  * [2] J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami,  “Internet of things (iot): A vision, architectural elements, and future directions,”  Future generation computer systems, vol. 29, no. 7, pp. 1645–1660, 2013.
  * [3] Pavithra D. and Balakrishnan R.,  “Iot based monitoring and control system for home automation,”  in 2015 Global Conference on Communication Technologies (GCCT), 2015, pp. 169–173.
  * [4] I. Goodfellow, Y. Bengio, and A. Courville,  Deep learning,  MIT press, 2016.
  * [5] J. Wang and L. Perez,  “The effectiveness of data augmentation in image classification using deep learning,”  Convolutional Neural Networks Vis. Recognit, 2017.
  * [6] Y. LeCun, Y. Bengio, et al.,  “Convolutional networks for images, speech, and time series,”  The handbook of brain theory and neural networks, vol. 3361, no. 10, pp. 1995, 1995.
  * [7] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. l. Ozair, A. Courville, and Y. Bengio,  “Generative adversarial nets,”  in Advances in neural information processing systems, 2014, pp. 2672–2680.
  * [8] Gregor K., Danihelka I., Graves A., Rezende D., and Wierstra D.,  “Draw: A recurrent neural network for image generation,”  in Proceedings of the 32nd International Conference on Machine Learning, 2015, vol. 37 of Proceedings of Machine Learning Research, pp. 1462–1471.
  * [9] I. Sutskever, J. Martens, and G. E. Hinton,  “Generating text with recurrent neural networks,”  in Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 1017–1024.
  * [10] Stephen McLaughlin, Patrick McDaniel, and William Aiello,  “Protecting consumer privacy from electric load monitoring,”  in Proceedings of the 18th ACM conference on Computer and communications security. ACM, 2011, pp. 87–98.
  * [11] C. Ferner, G. Eibl, A. Unterweger, S. Burkhart, and S. Wegenkittl,  “Pool detection from smart metering data with convolutional neural networks,”  in Energy Informatics 2019, 2019.
  * [12] E. L. Denton, S. Chintala, A. Szlam, and R. Fergus,  “Deep generative image models using a laplacian pyramid of adversarial networks,”  in Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds., pp. 1486–1494. Curran Associates, Inc., 2015.
  * [13] Radford A., Metz L., and Chintala S.,  “Unsupervised representation learning with deep convolutional generative adversarial networks,”  in 4th International Conference on Learning Representations, 2016\.
  * [14] M. Arjovsky, S. Chintala, and L. Bottou,  “Wasserstein gan,”  stat, vol. 1050, pp. 9, 2017.
  * [15] N. Sgouros, I. Kontaxakis, and M. Sangriotis,  “Effect of different traversal schemes in integral image coding,”  Applied optics, vol. 47, no. 19, pp. D28–D37, 2008.


# Simultaneous Perturbation Method for Multi-Task Weight Optimization in One-
Shot Meta-Learning

Andrei Boiarov
SIT Rolos
Schaffhausen, Switzerland
<EMAIL_ADDRESS>
&Kostiantyn Khabarlak
Dnipro University of Technology
Dnipro, Ukraine
<EMAIL_ADDRESS>
&Igor Yastrebov
OTP Bank
Budapest, Hungary

###### Abstract

Meta-learning methods aim to build learning algorithms capable of quickly
adapting to new tasks in low-data regime. One of the most difficult benchmarks
of such algorithms is a one-shot learning problem. In this setting many
algorithms face uncertainties associated with limited amount of training
samples, which may result in overfitting. This problem can be resolved by
providing additional information to the model. One of the most efficient ways
to do this is multi-task learning. In this paper we investigate the
modification of a standard meta-learning pipeline. The proposed method
simultaneously utilizes information from several meta-training tasks in a
common loss function. The impact of these tasks in the loss function is
controlled by a per task weight. Proper optimization of the weights can have
big influence on training and the final quality of the model. We propose and
investigate the use of methods from the family of Simultaneous Perturbation
Stochastic Approximation (SPSA) for optimization of meta-train tasks weights.
We also demonstrate superiority of stochastic approximation in comparison to
gradient-based method. The proposed Multi-Task Modification can be applied to
almost all meta-learning methods. We study applications of this modification
on Model-Agnostic Meta-Learning and Prototypical Network algorithms on CIFAR-
FS, FC100, miniImageNet and tieredImageNet one-shot learning benchmarks.
During these experiments Multi-Task Modification has demonstrated improvement
over original methods. SPSA-Tracking algorithm first adapted in this paper for
multi-task weight optimization shows the largest accuracy boost that is
competitive to the state-of-the-art meta-learning methods. Our code is
available online111https://github.com/andrewbo29/mtm-meta-learning-sa.

_Keywords_ Meta-Learning $\cdot$ Multi-Task Learning $\cdot$ One-Shot Learning
$\cdot$ Stochastic Approximation $\cdot$ Deep Learning

##  1 Introduction

One of the most important hallmarks of human intelligence is the ability to
learn new concepts from few examples. However, despite significant progress in
deep learning in a variety of fields in recent years limitations of deep
learning approaches still exist in many practical applications where labeled
data is intrinsically rare or expensive. For machine learning systems it is
difficult to adapt to new concepts fast with very little supervision. That
raises attention to the challenging few-shot learning problem and its most
difficult setting: one-shot learning. One of the promising areas of research
in recent years was to tackle this problem using the approach of meta-learning
or “learning to learn” [1, 2, 3].

Meta-learning process is performed on a family of tasks set on disjoint meta-
training and meta-testing sets. Each task includes only a limited amount of
training data which requires fast adaptability of the meta-learner. The
performance of the meta-learner is evaluated on meta-testing tasks. In this
paper we have modified Multi-Task Modification (MTM) approach for meta-
learning from [4] where this algorithm was investigated on Omniglot dataset,
which is a less challenging setting than those discussed in this paper. Multi-
task weight optimization plays the key role in MTM. Therefore, we consider
members of the family of Simultaneous Perturbation Stochastic Approximation
(SPSA) methods as optimizers of multi-task weights and compare them with a
gradient-based approach. To the best of our knowledge, we have adapted SPSA
for Tracking method for multi-task weight optimization in the one-shot meta-
learning setting for the first time. To summarize, our main contributions are
as follows:

  * •

We have successfully applied our MTM to the important optimization-based
method MAML [1] and the important metric-based method Prototypical Networks
[5]. For both of these methods, the proposed approach achieved an improvement
over the original method on the main one-shot learning benchmarks.

  * •

We have formulated novel SPSA for Tracking method (SPSA-Track) as a multi-task
weights optimizer in MTM for meta-learning. This method demonstrated a
significant performance boosting on average.

  * •

We have shown robustness of SPSA-based approaches on multiple benchmarks.
Also, they appear to outperform gradient-based methods as a multi-task weights
optimizer.

  * •

We have demonstrated that combining MTM SPSA-Track with modern backbone model
achieves a significant boost in accuracy. Such a performance improvement puts
our result among the best in the field.

##  2 Related Works

Few-Shot Learning. Meta-learning attempts to acquire general knowledge of a
target domain by learning many tasks that lie within it [1]. Few-shot learning
is widely used as one of the main benchmarks for meta-learning approaches [6,
1, 7]. In one-shot learning setting, training set consists of only one sample
per task. It is expected that by training on similar yet different tasks the
model will learn common features that will still be relevant to unseen tasks
and, thus, acquire general understanding of the field. Few-shot learning
models are typically divided into 2 broad categories based on how the problem
is modelled: optimization-based and metric-based.

The class of optimization-based few-shot learning algorithms uses explicit
optimization for fast adaptation to new tasks. Model-Agnostic Meta-Learning
(MAML) [1] attempted to find network weights that are able to quickly adapt to
new tasks through an optimization procedure. Many refinements to the MAML
algorithm have been proposed since its inception, for instance, Latent
Embedding Optimization (LEO) [3] tried to improve generalization by learning
in a lower-dimensional latent embedding space. R2D2 [8] and MetaOptNet [2]
improved accuracy by using Ridge Regression and SVM as classifiers.

Metric-based approaches are a class of methods for few-shot learning problems
that aim to learn a discriminative embedding transferable to a target task.
Metric learning has a long history of research and various applications [9,
10]. Siamese Convolutional Neural Networks [11] was the first metric-based
method for one-shot image classification that learned task-agnostic feature
embeddings. Matching Networks [6] enhanced this approach by using two
different networks and episodic training. Prototypical Networks (ProtoNet) [5]
introduced the idea of learning class representation by using mean features
embeddings. Two other important approaches were suggested in Relational
Networks [12] that introduced an architecture to model the similarity of
embeddings and Task Dependent Adaptive Metric (TADAM) [13] that switched to
learning task-specific features. Recently, Deep Subspace Networks (DSN) [14]
have been used to predict class labels by calculating the distance between a
query point and its projections onto discriminative subspaces formed by the
support sets for each class.

Transductive inference as an approach to the few-shot learning problem was the
subject of research in several recent papers [15]. In this setting a
classifier model performs class predictions jointly for all the unlabeled
query examples of a single few-shot learning task, instead of making
predictions for one sample at a time as in inductive methods. As an
alternative to the meta-learning framework, several recent studies
investigated performance of standard end-to-end pre-trained classifiers on
few-shot tasks [16, 7, 15]. However, in this work we focus on improving the
meta-training phase and consider only the inductive methods that use meta-
learning pipeline without pre-training. For this reason, we selected MAML and
ProtoNet because papers describing these methods are among the most cited in
the field. Thus, we expect that results of applying multi-task meta-learning
modification should generalize when used for the modification of other one-
shot learning algorithms following these restrictions.

Multi-Task Learning. Multi-task learning is an approach where a model is
trained to give predictions for multiple tasks jointly. Multi-tasking can be
considered as an implicit data augmentation and regularization technique
assisting the network to focus on more relevant features in the input [17].
However, proper task weighting might be crucial to improving model
performance. Kendall et al. [18] have introduced a multi-task loss function
that relies on maximizing the Gaussian likelihood with task-dependent
uncertainty. The proposed single model has outperformed separate models for
each task. In [4], multi-tasking approach has been applied for a few-shot
character recognition problem, which resulted in an improvement over the
baseline model. A close connection in terms of optimization task formulation
between multi-task learning and gradient-based meta-learning was established
in [19].

Stochastic Approximation. The stochastic approximation algorithm was developed
for solving the optimization problem by Kiefer and Wolfowitz [20].
Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm [21] uses
only two observations at each iteration which recursively generates estimates
along random directions. In high-dimensional setting SPSA has the same order
of convergence rate as Kiefer-Wolfowitz approach while requiring significantly
fewer measurements of a function. When an unknown but bounded disturbance
corrupts the observed data, the quality of methods based on stochastic
gradient decreases. However, SPSA-like algorithms demonstrate a high level of
resistance to such disturbances [22]. Stochastic approximation algorithms are
successfully used in various machine learning problems [22].

##  3 Multi-Task Meta-Learning Modification

###  3.1 One-Shot Learning Problem Definition

We consider one-shot learning problem as a special case of few-shot learning.
According to the few-shot learning problem formulation, we need to train a
classifier that can quickly adapt to new unseen classes using only few labeled
examples of classes. To cast this problem as meta-learning problem, Vinyals et
al. [6] proposed the pipeline where elements of each class were randomly
divided into support set and query set. A set of classes in training phase
does not overlap with a set of classes in testing phase. In the meta-learning
pipeline, training and testing processes consist of a series of episodes where
each episode $\xi_{t}$ includes one or more tasks and each task $t_{i}$
consists of support and query sets for several classes.

Consider labeled dataset
$\left\\{(\mathbf{x}_{1},y_{1}),\ldots,(\mathbf{x}_{CN},y_{CN})\right\\}$ with
$C$ classes and $N$ examples per each class, where
$\mathbf{x}_{i}\in\mathbb{R}^{d}$ is the input data vector and
$y_{i}\in\left\\{1,\ldots,C\right\\}$ is the class label. Let $N_{S}$ be the
number of examples in the support set for each class, $N_{Q}$ be the number of
examples in the query set, $N_{S}~{}+~{}N_{Q}~{}=~{}N$, $N_{C}\leq C$ be the
number of classes in a task that are randomly selected from all set of $C$
classes. Corresponding few-shot learning classification procedure is called
$N_{S}$-shot $N_{C}$-way classification. In one-shot setting, $N_{S}=1$.

Let episode $\xi_{t}:(t_{1},\ldots,t_{M})$ consist of $M$ tasks. Each task
$t_{i}$ involves per task support set
$S_{t_{i}}=\left\\{(\mathbf{x}_{i},y_{i})\right\\}_{i=1}^{N_{S}N_{C}}$ and
query set
$Q_{t_{i}}=\left\\{(\mathbf{x}_{i},y_{i})\right\\}_{i=1}^{N_{Q}N_{C}}$
containing elements of all $N_{C}$ classes included in the task $t_{i}$ of per
class support and query sets, respectively. Parameter $M\in\mathbb{N}$ can
vary in different algorithms. For each task $t_{i}$ in an episode,
$\mathcal{L}_{\theta,t_{i}}(D)$ denotes the value of loss function on $D$,
where $\theta$ is the vector of learnable parameters of model $\phi_{\theta}$
(convolutional neural network in this paper) and $D$ can be support, query, or
another dataset.

Model-Agnostic Meta-Learning. MAML uses loss function which is a two-step
training procedure:
1) adaptation step (inner-loop) where network parameters are adapted to the
specific task $t_{i}$:

$\theta^{\prime}_{i}=\theta-\alpha\nabla_{\theta}\mathcal{L}_{\theta,t_{i}}(S_{t_{i}});$
(1)

2) meta-gradient update (outer-loop) step where parameters $\theta$ are
updated by backpropagating through the adaptation procedure:

$\theta\leftarrow\theta-\beta\nabla_{\theta}\sum_{i=1}^{M}{\mathcal{L}_{\theta^{\prime}_{i},t_{i}}(Q_{t_{i}})},$
(2)

where $\alpha$, $\beta$ are adaptation and meta-learning rates
correspondingly.

Prototypical Networks. In ProtoNet algorithm model $\phi_{\theta}$ computes
embedding of input data. Each class $k$ is represented by prototype vector
$\mathbf{c}^{k}_{t_{i}}$ which is calculated as a mean vector of the
corresponding support set. The Prototypical Networks model $\phi_{\theta}$ is
trained via stochastic gradient descent (SGD) by minimizing loss function for
train task $t_{i}$, where $d(\cdot,\cdot)$ is Euclidean distance function:

$\mathcal{L}_{\theta,t_{i}}(Q_{t_{i}})=\frac{1}{N_{C}}\sum_{k=1}^{N_{C}}\frac{1}{N_{Q}}\sum_{\mathbf{x}_{j}\in
Q_{t_{i}}^{k}}-\log\frac{\exp(-d(\phi_{\theta}(\mathbf{x}),\mathbf{c}^{k}_{t_{i}}))}{\sum_{k^{\prime}}\exp(-d(\phi_{\theta}(\mathbf{x}),\mathbf{c}^{k^{\prime}}_{t_{i}}))}.$
(3)

Figure 1: Multi-task meta-learning modification pipeline for a training
episode $\xi_{t}$ described in Algorithm 1.

###  3.2 Multi-Task Meta-Learning Loss Function

Many few-shot learning methods [6, 5, 2] work with $M=1$ (i.e. with one task
in a training episode $\xi_{t}$). Other approaches [1, 23] utilize a batch of
tasks with $M>1$ per episode but consider the contribution of each task
equally. In this paper we develop the idea of adaptive use of information from
multiple tasks simultaneously via the multi-task approach for meta-learning
proposed in [4].

Multi-task learning methods for deep neural networks can be divided into two
main areas: soft and hard parameter sharing of hidden layers of neural network
[17]. In accordance with [4], we use hard parameter sharing for all hidden
layers of a convolutional network. Thus, one neural network is used for all
tasks, and the presence of several tasks is reflected only in the loss
function. For this purpose [4] proposed an adapted approach discussed in [18]
which used task-depended (homoscedastic) uncertainty as a basis for weighting
losses in a multi-task learning problem. Corresponding multi-task meta-
learning loss function takes the following form:

$\mathcal{L}^{MT}_{\xi_{t}}(\boldsymbol{\omega}_{t},\left\\{Q_{t_{i}}\right\\}_{i=1}^{M})=\sum_{i=1}^{M}\frac{1}{(\omega_{t}^{(i)})^{2}}\mathcal{L}_{\theta,t_{i}}(Q_{t_{i}})+\sum_{i=1}^{M}\log(\omega_{t}^{(i)})^{2},$
(4)

where weights
$\boldsymbol{\omega}_{t}=(\omega_{t}^{(1)},\ldots,\omega_{t}^{(M)})$ are
hyperparameters.

###  3.3 Multi-Task Weights Optimization

As shown in [18], model performance is extremely sensitive to multi-task
weights selection and tuning of $\boldsymbol{\omega}_{t}$ is critical for the
success of a multi-task learning. On the other hand, searching for these
optimal weights is expensive and increasingly difficult for a large model with
numerous tasks.

In this work we focus on developing and investigating performance of
optimization methods for hyperparameters $\boldsymbol{\omega}_{t}$ in the loss
function (4). Nowadays gradient optimization approach is a natural choice in
many deep learning algorithms [24, 18]. Therefore, we consider an approach
with embedding weights optimization in the backpropagation procedure similar
to the one described in [18]. This procedure is described in Algorithm 1. Let
$\widehat{\boldsymbol{\omega}}_{t}$ and $\widehat{\theta}_{t}$ denote
estimates at iteration $t$ of multi-task weights and model parameters,
respectively. This multi-task meta-learning modification pipeline for a
training episode $\xi_{t}$ is visualized in Figure 1.

0: $N_{S}$, $N_{Q}$, $N_{C}$, $\widehat{\theta}_{t-1}$,
$\widehat{\boldsymbol{\omega}}_{t-1}$

0: Updated parameters estimates $\widehat{\theta}_{t}$,
$\widehat{\boldsymbol{\omega}}_{t}$

1: for $i$ in $\left\\{1,\ldots,M\right\\}$ do

2: Sample $N_{C}$ random classes

3: Sample random elements in $S_{t_{i}}$ and $Q_{t_{i}}$

4: Compute task loss function $\mathcal{L}_{\theta,t_{i}}(Q_{t_{i}})$

5: end for

6: Compute
$\mathcal{L}^{MT}_{\xi_{t}}(\widehat{\boldsymbol{\omega}}_{t-1},\left\\{Q_{t_{i}}\right\\}_{i=1}^{M})$
via (4)

7: Use multi-task weights optimizer to update
$\widehat{\boldsymbol{\omega}}_{t}$

8: Update parameters $\widehat{\theta}_{t}$ via SGD by
$\mathcal{L}^{MT}_{\xi_{t}}$

Algorithm 1 Training for episode $\xi_{t}:(t_{1},\ldots,t_{M})$

In this paper gradient-based and stochastic approximation methods are
considered and compared as the multi-task weights optimizer.

When solving a meta-learning problem, algorithms face model uncertainties
associated with a critical issue of limited amount of training samples per
task, which may result in overfitting [25]. This phenomenon is especially
pronounced in the extreme case of one-shot learning. Therefore, meta-learning
algorithms must be able to effectively adapt to these uncertainties [26, 27].
We approach this issue from optimization point and propose to use methods from
the family of Simultaneous Perturbation Stochastic Approximation (SPSA)
algorithms as a multi-task weights optimizer due to their robustness and
successful application in various machine learning and control problems with
uncertainties [28, 22, 29].

Gradient-Based Methods. As stated earlier, one of the proposed gradient-based
approaches is to incorporate multi-task weights in a computational graph of
meta-learning algorithm (MAML and ProtoNet in this work). In that case
backpropagation procedure is used to compute weights updates and a gradient-
based optimizer is shared with the whole model. This multi-task modification
of meta-learning is denoted by MTM Backprop. We also explored first-order
optimization approaches as a separate multi-task weights optimizer described
in Algorithm 1. Adam optimization method [30] was selected based on the
results of experiments and due to multiple mentions in the deep learning
literature. This approach is named MTM Inner First-Order by analogy with inner
loop optimization step in MAML (1).

###  3.4 SPSA for Tracking method

When solving a meta-learning problem, algorithms face model uncertainties
associated with a critical issue of limited amount of training samples per
task, which may result in overfitting [25]. This phenomenon is especially
pronounced in the extreme case of one-shot learning. Therefore, meta-learning
algorithms must be able to effectively adapt to these uncertainties [26, 27].
We approach this issue from optimization point and propose to use methods from
the family of SPSA algorithms as a multi-task weights optimizer due to their
robustness and successful application in various machine learning and control
problems with uncertainties [22]. In this subsection, we propose the SPSA for
Tracking algorithm firstly adapted for multi-task weights optimization.

In order to cast the SPSA-based approach as a multi-task weights optimizer, we
reformulate the problem of finding estimates of hyperparameters
$\boldsymbol{\omega}_{t}$ in the multi-task loss function (4) as a non-
stationary optimization problem suitable for a stochastic approximation [31,
4]. In order to do this, we introduce an observation model for the training
episode $\xi_{t}$ that takes into account uncertainties arising in meta-
learning:

$L_{t}(\boldsymbol{\omega}_{t})=\mathcal{L}^{MT}_{\xi_{t}}(\boldsymbol{\omega}_{t},\left\\{Q_{t_{i}}\right\\}_{i=1}^{M})+\nu_{t},$
(5)

where $\nu_{t}$ is an additive external noise caused by uncertainties arising
from the limited amount of training samples per task. Therefore, it is
necessary to find an algorithm that produces an estimate
$\widehat{\boldsymbol{\omega}}_{t}$ of an unknown vector
$\boldsymbol{\omega}_{t}$ that minimizes mean-risk functional of objective
functions (5) based on observations $L_{1},L_{2},\ldots,L_{t}$ from training
episodes $\xi_{1},\xi_{2},\ldots,\xi_{t}$.

To the best of our knowledge, we propose to use SPSA for Tracking approach
[31] as a multi-task weights optimizer for the first time. This algorithm
simultaneously uses observation from the current and previous iterations which
allows it to be more stable under parameters drift conditions. Let
$\boldsymbol{\Delta}_{t}\in{\mathbb{R}}^{d}$ be a vector consisting of
independent random variables with Bernoulli distribution,
$\widehat{\boldsymbol{\omega}}_{0}$ a vector with the initial values, and
$\\{\alpha_{t}\\}$ and $\\{\beta_{t}\\}$ sequences of positive numbers. Then
SPSA for tracking multi-task weights optimizer (MTM SPSA-Track) constructs the
following estimates of multi-task weights:

$\displaystyle\begin{cases}L_{2t}=L_{2t}(\widehat{\boldsymbol{\omega}}_{2t-2}+\beta_{n}\boldsymbol{\Delta}_{t}),\;\\\
L_{2t-1}=L_{2t-1}(\widehat{\boldsymbol{\omega}}_{2t-2}-\beta_{t}\boldsymbol{\Delta}_{t})\\\
\\\
\widehat{\boldsymbol{\omega}}_{2t-1}=\widehat{\boldsymbol{\omega}}_{2t-2}\\\
\widehat{\boldsymbol{\omega}}_{2t}=\widehat{\boldsymbol{\omega}}_{2t-1}-\alpha_{t}\boldsymbol{\Delta}_{t}\frac{L_{2t}-L_{2t-1}}{2\beta_{t}}.\end{cases}$
(6)

Algorithm (6) has theoretical convergence guarantees expressed in terms of an
upper bound of residuals between the estimates and the theoretical optimal
solution [31].

###  3.5 Multi-Task Modification for Class-Based Weights

In Algorithm 1, $N_{C}$ classes in task $t_{i}$ is sampled randomly. However,
if additional meta-information about classes is available, the data can be
sampled from some coarse classes. For this, we define the modified multi-task
loss function (4) applicable to coarse classes as well as to usual finer-
grained classes. A per-task loss function $\mathcal{L}_{\theta,t_{i}}$ can be
rewritten as a sum of terms corresponding to classes:

$\mathcal{L}_{\theta,t_{i}}(Q_{t_{i}})=\sum_{k=1}^{C}\mathcal{L}_{\theta,t_{i},k}(Q_{t_{i}}),$

where $k$ is a class, $k\in\left\\{1,\ldots,K\right\\}$, $K$ is the number of
coarse classes (or $K=C$), $\mathcal{L}_{\theta,t_{i},k}(Q_{t_{i}})$ is the
algorithm-specific partial loss computed for the corresponding class which is
equal to zero if the class $k$ is not present in the task $t_{i}$. Then we
modify multi-task loss function (4) by introducing weights $\omega_{C}^{(k)}$
for each class and combining like terms from $M$ training tasks:

$\mathcal{L}^{MT}_{\xi_{t}}(\boldsymbol{\omega}_{t},\left\\{Q_{t_{i}}\right\\}_{i=1}^{M})=\sum_{k=1}^{C}\left(\left(\frac{1}{(\omega_{C}^{(k)})^{2}}\sum_{i=1}^{M}\mathcal{L}_{\theta,t_{i},k}(Q_{t_{i}})\right)+\log(\omega_{C}^{(k)})^{2}\right).$
(7)

SPSA-based multi-task weights optimizer with loss function (7) is denoted as
MTM SPSA-Coarse.

##  4 Experiments

We conducted experiments on four datasets – CIFAR-FS [8], FC100 [13],
miniImageNet [6] and tieredImageNet [32]. These datasets were selected as they
became standard benchmarks over the last couple of years [15, 7]. Experiments
were performed on Rolos platform111https://rolos.com/ with a A100 vGPU, 48
vCPUs at 2.30GHz, 128GiB RAM. Tables 1, 2 and 3 summarize results of these
experiments.

The CIFAR-FS and FC100 datasets are derived from CIFAR-100 dataset having 100
classes with each class consisting of 600 images of size 32 × 32. In the
CIFAR-FS dataset classes are randomly divided into groups of 64, 16 and 20 for
training, validation, and testing, respectively, while in the FC100 datasets
20 superclasses of CIFAR-100 are split into groups of 12, 4 and 4. The
miniImageNet and tieredImageNet datasets are derived from ILSVRC-2012 dataset
having 1000 categories. The miniImageNet dataset consists of 100 classes
randomly chosen from ILSVRC-2012 split into groups of 64, 16 and 20 as
proposed in [33] which is a standard convention. The tieredImageNet dataset
consists of 608 categories from ILSVRC-2012 grouped into 34 supercategories
that are then split into groups of 20, 6 and 8. In both cases 600 images of 84
× 84 pixels in size are sampled for each class.

Table 1: Multi-Task Modification results on CIFAR-FS, FC100, miniImageNet and tieredImageNet (1-shot setting). Improvements are shown in bold. | CIFAR-FS | FC100 | miniImageNet | tieredImageNet
---|---|---|---|---
Configuration | 2-way | 5-way | 2-way | 5-way | 2-way | 5-way | 2-way | 5-way
MAML
Reproduced |  74.8 % |  54.5 % |  66.0 % |  36.4 % |  73.2 % |  45.9 % |  73.3 % |  47.9 %
MTM SPSA | 76.4 % | 54.7 % | 67.0 % | 36.4 % | 74.9 % | 48.0 % | 74.9 % | 47.8 %
MTM SPSA-Track | 76.8 % | 54.6 % | 66.8 % | 38.3 % | 75.8 % | 46.5 % | 73.8 % | 48.3 %
ProtoNet
Reproduced |  77.8 % |  58.9 % |  65.0 % |  35.7 % |  74.2 % |  50.0 % |  72.9 % |  49.4 %
MTM SPSA | 79.1 % | 59.7 % | 65.1 % | 36.0 % | 74.7 % | 50.2 % | 73.6 % | 49.5 %
MTM SPSA-Track | 78.2 % | 59.8 % | 65.3 % | 36.1 % | 74.8 % | 50.8 % | 73.1 % | 50.0 %

As mentioned in Section 2, the experiments with Multi-Task Modification were
performed using MAML and Prototypical Networks. For MAML algorithm we used the
neural network $\phi_{\theta}$ defined in the original paper [1] with
32-32-32-32 configuration where a-b-c-d denotes a 4-layer convolutional neural
network with a, b, c, d filters in convolutional layers. Adam was used as the
meta-optimizer with learning rate $\beta=10^{-3}$ and adaptation step size
$\alpha=0.01$ as in (2). We used 5 adaptation steps during training and 10
during testing for all datasets. We selected meta-batch size of $M=4$ and
trained the model for 300 epochs, each containing 100 tasks, unless specified
otherwise. Following [33] and the original paper, 15 samples per class were
taken for evaluation. For the MTM the baseline model was trained for extra 40
epochs.

In the case of ProtoNet we used 64-64-64-64 feature extraction backbone
$\phi_{\theta}$ as suggested in the original paper [5]. We followed the
standard practice [34] for the meta-learning setup using SGD with Nesterov
momentum of 0.9 and weight decay of 0.0005 as an optimizer. The learning rate
was initially set to 0.1 and then decreased according to the strategy from
[34]. During meta-training, we used the first 20 epochs for pre-training the
model by using the original ProtoNet method and implementing the Multi-Task
Modifications only for the last 40 epochs.

We designed several experiment settings to research the relative advantage of
using the multi-task loss function (4) and SPSA-based optimization over
original methods (Table 1) and over gradient-based method (Table 2) where
multi-task weights in the loss function are optimized jointly with the network
parameters $\theta$. In experiments we mainly used $M=4$ tasks per training
episode $\xi_{t}$ (other values of $M$ are indicated explicitly) and the best
model for testing was selected on the validation set. For SPSA and SPSA-Track
we set $\alpha_{n}=0.25/n^{\frac{1}{6}},\beta_{n}~{}=~{}15/n^{\frac{1}{24}}$
in (6) as per the theoretical result from [22]. During the experiments we
found that $L^{2}$-normalization of multi-task weights in MTM SPSA and MTM
SPSA-Track improves the stability of training. Results of all experiments are
formulated in terms of average few-shot classification accuracy after 1000
testing iterations.

Table 2: Comparison of gradient-based (Backprop), SPSA and SPSA for Tracking multi-task weights optimizers on 1-shot, 5-way experiments. Algorithm |  CIFAR-FS |  FC100 |  miniImageNet |  tieredImageNet
---|---|---|---|---
MAML
Backprop | 53.1 % | 37.6 % | 47.4 % | 46.7 %
SPSA | 54.7 % | 36.4 % | 48.0 % | 47.8 %
SPSA-Track | 54.6 % | 38.3 % | 46.5 % | 48.3 %
ProtoNet
Backprop | 59.4 % | 35.5 % | 50.4 % | 49.2 %
SPSA | 59.7 % | 36.0 % | 50.2 % | 49.5 %
SPSA-Track | 59.8 % | 36.1 % | 50.8 % | 50.0 %

We used the miniImageNet dataset to compare our method with the prior work.
Since the majority of recent approaches use more advanced convolutional neural
networks with higher embedding dimension such as residual networks (ResNet)
[24] as feature extraction backbones, we implemented the original ProtoNet
with ResNet-12 backbone provided in [2] to compare against the results of
other methods with backbones from the ResNet family. We did not include
approaches that were developed for semi-supervised and transductive learning
settings in this comparison since such approaches use the statistics of query
examples or statistics across the one-shot tasks. We also excluded methods
that use non-episodic pre-training as mentioned in Section 2. Table 3 shows
that, when used with a comparable backbone, our multi-task meta-learning
modification of ProtoNet with stochastic approximation increases one-shot
classification accuracy to the level of significantly more advanced methods
and is competitive against state-of-the-art meta-learning approaches. These
results are presented with 95 % confidence intervals.

Table 3: Comparison to prior work on miniImageNet meta-test split. Bold values are the accuracy no less than 1 % compared with the highest one. Algorithm | Backbone | 1-shot 5-way
---|---|---
MAML [1, 16] |  ResNet-18 | 49.61 $\pm$ 0.92 %
Chen et al. [16] | ResNet-18 | 51.87 $\pm$ 0.77 %
Relation Networks [12, 16] | ResNet-18 | 52.48 $\pm$ 0.86 %
Matching Networks [6, 16] | ResNet-18 | 52.91 $\pm$ 0.88 %
RAP-ProtoNet [35] | ResNet-10 | 53.64 $\pm$ 0.60 %
ProtoNet (reproduced) [5] | ResNet-12 | 56.52 $\pm$ 0.45 %
Gidaris et al. [34] | ResNet-15 | 55.45 $\pm$ 0.89 %
SNAIL [23] | ResNet-15 | 55.71 $\pm$ 0.99 %
Bauer et al. [36] | ResNet-34 | 56.30 $\pm$ 0.40 %
AdaResNet [37] | ResNet-12 | 56.88 $\pm$ 0.62 %
TADAM [13] | ResNet-12 | 58.50 $\pm$ 0.30 %
Shot-Free [38] | ResNet-12 | 59.04 $\pm$ n/a %
CAML [39] | ResNet-12 | 59.23 $\pm$ 0.99 %
Wang et al. [19] | ResNet-12 | 59.84 $\pm$ 0.22 %
MTL [40] | ResNet-12 | 61.20 $\pm$ 1.80 %
vFSL [41] | ResNet-12 | 61.23 $\pm$ 0.26 %
MetaOptNet [2] | ResNet-12 | 62.64 $\pm$ 0.61 %
DSN [14] | ResNet-12 | 62.64 $\pm$ 0.66 %
ProtoNet MTM SPSA (Ours) | ResNet-12 | 60.24 $\pm$ 0.71 %
ProtoNet MTM SPSA-Track (Ours) | ResNet-12 | 61.33 $\pm$ 0.74 %
ProtoNet MTM SPSA-Track ($M=2$) (Ours) | ResNet-12 |  61.94 $\pm$ 0.73 %

##  5 Ablation Study

###  5.1 Improvements of MTM with SPSA and SPSA-Track.

We have conducted experiments for four datasets most widely used in the field
of one-shot learning as shown in Table 1. On CIFAR-FS, we have improved on
original methods up to ${\bf 2.0}$%, with the largest improvement by 2-way
MAML MTM SPSA-Track. On FC100 benchmark, the largest improvement of
$\bf{1.9}$% has been achieved with the novel MTM SPSA-Track method on MAML in
5-way scenario. On tieredImageNet, we got improvements up to $\bf{1.6}$% for
MAML in 2-way scenario with SPSA. The top performing methods include MTM SPSA
and MTM SPSA-Track for both MAML and ProtoNet. MTM SPSA-Track gives the
greatest boost in 5-way settings. The last dataset we considered was
miniImageNet, which is the most widely used benchmark for few-shot learning.
Here we have achieved significant improvements up to $\bf{2.6}$% with MAML MTM
SPSA-Track leading and MAML MTM SPSA following. Similar results are observed
for ProtoNet. The results presented in Table 1 show that proposed MTM with
SPSA and SPSA-Track methods outperform the original approaches on all four
benchmarks. Novel SPSA-Track demonstrates the largest improvement over the
baseline in most cases.

The experiment results shown in Table 3 suggest that applying our method to
ProtoNet with a more modern ResNet-12 backbone gives performance improvement
of $\bf{3.72}$% for MTM SPSA, $\bf{4.81}$% for MTM SPSA-Track, making MTM
SPSA-Track better performing in this scenario as well. Such a performance
improvement puts our result among the best in the field. Applying MTM SPSA-
Track to ProtoNet with number of tasks $M=2$ gives a performance improvement
of $\bf{5.42}$% that is competitive against state-of-the-art methods. The fact
that this improvement has been achieved by modifying the loss function only
and the fact that MTM SPSA and MTM SPSA-Track can be applied to almost any of
the meta-learning methods in the table makes our result even more significant.
It is worth noting that in the case of meta-learning methods like MAML that
originally use meta-batch with several tasks, our Multi-Task Modification
requires practically no additional computational costs.

###  5.2 Gradient-Based vs SPSA-Based Optimization.

We explored the comparison between gradient-based and SPSA-based approaches of
multi-task weight optimization. As can be seen from Table 2, SPSA-based
approaches give superior results in all experiments, so we can conclude that
zero-order optimizers are better suited for the proposed multi-task
modification in one-shot setting.

###  5.3 SPSA Multi-Task Weights Behavior

In MTM SPSA and MTM SPSA-Track experiments multi-task weights are renormalized
during the optimization procedure after each epoch. The corresponding dynamics
of SPSA-Track multi-task weights during training in miniImageNet 1-shot 2-way
experiment is depicted in Figure 2 (a) for MAML and (b) for ProtoNet. Weights
in ProtoNet experiment demonstrate larger fluctuation, yet in both cases there
is a visible trend of multi-task weights splitting despite the difference in
meta-learning methods.

Figure 2: MTM SPSA-Track multi-task weights dynamic during training with
weights normalization on
miniImageNet 1-shot 2-way: (a) MAML, (b) ProtoNet.

###  5.4 Task Data Sampling

We used data sampling provided by Torchmeta library [42] to supply a unified
approach to the problem. It follows the usual procedure where classes are
sampled from the set of candidates and then an appropriate number of examples
are selected per class. This procedure is randomized for every task and is
independent from the choice of algorithm or its multi-task modification.

In MTM SPSA-Coarse experiment setting the tasks are sampled in the same way,
however, information about coarse classes is passed to the MTM SPSA algorithm.
We have modified MTM SPSA, so that instead of having $M=4$ multi-task weights
(one per task), we use 20 (one per CIFAR-100 coarse class). Consequently,
instead of weighting tasks, we now weight instances of the corresponding
coarse classes. A tie between a particular coarse class and the corresponding
weight is established via modified multi-task loss function (7). At the end of
a training episode the weights are updated only for the coarse classes that
are present in the episode’s meta-batch.

As can be seen from Figure 3 (a), in case of MAML the coarse classes weights
$\omega_{C}^{(k)}$ dynamic during training varies significantly between
different classes, which contrasts with ProtoNet, where weights follow the
general dynamics established in [4] as shown in Figure 3 (b).

Figure 3: MTM SPSA-Coarse class weights dynamic during training on CIFAR-FS
1-shot 5-way:
(a) MAML, (b) ProtoNet.

##  6 Conclusion

In this paper we have developed a new multi-task meta-learning modification
with stochastic approximation for one-shot learning. The application of this
approach to optimization-based method MAML and metric-based method ProtoNet
was investigated on four most widely used one-shot learning benchmarks. In all
experiments our algorithm showed significant improvements over baseline
methods. In addition, in most cases SPSA-based approach was better than
gradient method. We presented novel SPSA for Tracking algorithm as a multi-
task weights optimizer which has demonstrated the largest performance boost on
average. For future work, we aim to apply the described approach to state-of-
the-art few-shot learning algorithms and to reinforcement learning.

## References

  * Finn et al. [2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 1126–1135. PMLR, 2017.
  * Lee et al. [2019] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto.  Meta-learning with differentiable convex optimization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * Rusu et al. [2019] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell.  Meta-learning with latent embedding optimization.  In _International Conference on Learning Representations_ , 2019.
  * Boiarov et al. [2020] Andrei Boiarov, Oleg Granichin, and Olga Granichina.  Simultaneous perturbation stochastic approximation for few-shot learning.  In _2020 European Control Conference (ECC)_ , pages 350–355, 2020\.
  * Snell et al. [2017] Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc., 2017.
  * Vinyals et al. [2016] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra.  Matching networks for one shot learning.  In _Advances in Neural Information Processing Systems_ , volume 29. Curran Associates, Inc., 2016.
  * Tian et al. [2020] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, and Phillip Isola.  Rethinking few-shot image classification: A good embedding is all you need?  In _Computer Vision – ECCV 2020_ , pages 266–282. Springer International Publishing, 2020.
  * Bertinetto et al. [2019] Luca Bertinetto, Joao F. Henriques, Philip Torr, and Andrea Vedaldi.  Meta-learning with differentiable closed-form solvers.  In _International Conference on Learning Representations_ , 2019.
  * Boiarov and Tyantov [2019] Andrei Boiarov and Eduard Tyantov.  Large scale landmark recognition via deep metric learning.  In _CIKM ’19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , pages 169–178, 2019.
  * Musgrave et al. [2020] Kevin Musgrave, Serge Belongie, and Ser-Nam Lim.  A metric learning reality check.  In _Computer Vision – ECCV 2020_ , pages 681–699. Springer International Publishing, 2020.
  * Koch et al. [2015] Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov.  Siamese neural networks for one-shot image recognition.  In _ICML Deep Learning Workshop_ , 2015.
  * Sung et al. [2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales.  Learning to compare: Relation network for few-shot learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * Oreshkin et al. [2018] Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste.  TADAM: Task dependent adaptive metric for improved few-shot learning.  In _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc., 2018.
  * Simon et al. [2020] Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi.  Adaptive subspaces for few-shot learning.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * Dhillon et al. [2020] Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto.  A baseline for few-shot image classification.  In _International Conference on Learning Representations_ , 2020.
  * Chen et al. [2019] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang.  A closer look at few-shot classification.  In _International Conference on Learning Representations_ , 2019.
  * Ruder [2017] Sebastian Ruder.  An overview of multi-task learning in deep neural networks.  _arXiv preprint arXiv:1706.05098_ , 2017.
  * Kendall et al. [2018] Alex Kendall, Yarin Gal, and Roberto Cipolla.  Multi-task learning using uncertainty to weigh losses for scene geometry and semantics.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * Wang et al. [2021] Haoxiang Wang, Han Zhao, and Bo Li.  Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation.  In _Proceedings of the 38th International Conference on Machine Learning_ , volume 139 of _Proceedings of Machine Learning Research_ , pages 10991–11002. PMLR, 2021.
  * Kiefer and Wolfowitz [1952] J. Kiefer and J. Wolfowitz.  Stochastic estimation of the maximum of a regression function.  _The Annals of Mathematical Statistics_ , 23(3):462–466, 1952.
  * Spall [1992] James C. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation.  _IEEE Transactions on Automatic Control_ , 37(3):332–341, 1992.
  * Granichin et al. [2015] Oleg Granichin, Zeev Volkovich, and Dvora Toledano-Kitai.  _Randomized Algorithms in Automatic Control and Data Mining_ , volume 67 of _Intelligent Systems Reference Library_.  Springer Berlin Heidelberg, 2015.  ISBN 978-3-642-54785-0.
  * Mishra et al. [2018] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel.  A simple neural attentive meta-learner.  In _International Conference on Learning Representations_ , 2018.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2016.
  * Nguyen et al. [2020] Cuong Nguyen, Thanh-Toan Do, and Gustavo Carneiro.  Uncertainty in model-agnostic meta-learning using variational inference.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , March 2020.
  * Finn et al. [2018] Chelsea Finn, Kelvin Xu, and Sergey Levine.  Probabilistic model-agnostic meta-learning.  In _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc., 2018.
  * Abdar et al. [2021] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi.  A review of uncertainty quantification in deep learning: Techniques, applications and challenges.  _Information Fusion_ , 76:243–297, 2021.
  * Yue and Burges [2007] Yisong Yue and Christopher J. C. Burges.  On using simultaneous perturbation stochastic approximation for learning to rank, and the empirical optimality of LambdaRank.  Technical Report MST-TR-2007-115, Microsoft Research, 2007.
  * Boiarov and Granichin [2019] A. A. Boiarov and O. N. Granichin.  Stochastic approximation algorithm with randomization at the input for unsupervised parameters estimation of gaussian mixture model with sparse parameters.  _Autom. and Remote Cont._ , 80(8):1403–1418, 2019\.
  * Kingma and Ba [2015] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations_ , 2015.
  * Granichin and Amelina [2015] Oleg Granichin and Natalia Amelina.  Simultaneous perturbation stochastic approximation for tracking under unknown but bounded disturbances.  _IEEE Transactions on Automatic Control_ , 60(6):1653–1658, 2015.
  * Ren et al. [2018] Mengye Ren, Sachin Ravi, Eleni Triantafillou, Jake Snell, Kevin Swersky, Josh B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel.  Meta-learning for semi-supervised few-shot classification.  In _International Conference on Learning Representations_ , 2018.
  * Ravi and Larochelle [2017] Sachin Ravi and Hugo Larochelle.  Optimization as a model for few-shot learning.  In _International Conference on Learning Representations_ , 2017.
  * Gidaris and Komodakis [2018] Spyros Gidaris and Nikos Komodakis.  Dynamic few-shot visual learning without forgetting.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * Hong et al. [2021] Jie Hong, Pengfei Fang, Weihao Li, Tong Zhang, Christian Simon, Mehrtash Harandi, and Lars Petersson.  Reinforced attention for few-shot learning and beyond.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 913–923, June 2021.
  * Bauer et al. [2017] Matthias Bauer, Mateo Rojas-Carulla, Jakub Bartłomiej Świątkowski, Bernhard Schölkopf, and Richard E Turner.  Discriminative k-shot learning using probabilistic models.  _arXiv preprint arXiv:1706.00326_ , 2017.
  * Munkhdalai et al. [2018] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler.  Rapid adaptation with conditionally shifted neurons.  In _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pages 3664–3673. PMLR, 2018.
  * Ravichandran et al. [2019] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto.  Few-shot learning with embedded class models and shot-free meta training.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2019.
  * Jiang et al. [2019] Xiang Jiang, Mohammad Havaei, Farshid Varno, Gabriel Chartrand, Nicolas Chapados, and Stan Matwin.  Learning to learn with conditional class dependencies.  In _International Conference on Learning Representations_ , 2019.
  * Sun et al. [2019] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele.  Meta-transfer learning for few-shot learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * Zhang et al. [2019] Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang.  Variational few-shot learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2019.
  * Deleu et al. [2019] Tristan Deleu, Tobias Würfl, Mandana Samiei, Joseph Paul Cohen, and Yoshua Bengio.  Torchmeta: A meta-learning library for pytorch.  _arXiv preprint arXiv:1909.06576_ , 2019.