WGMwgm QEqe EPep PMSpms BECbec DEde

[1]

[cor]C. author<EMAIL_ADDRESS><EMAIL_ADDRESS>
#  SuSana Distancia is all you need:
Enforcing class separability in metric learning via two novel
distance-based loss functions for few-shot image classification

Mauricio Mendez-Ruiz  Jorge Gonzalez-Zapata  Ivan Reyes-Amezcua  Daniel
Flores-Araiza  Francisco Lopez-Tiro  Tecnologico de Monterrey, School of
Engineering and Sciences, Jalisco, 45138, Mexico  Andres Mendez-Vazquez
Centro de Investigación y de Estudios Avanzados, Computer Sciences Department,
Guadalajara, Mexico  Gilberto Ochoa-Ruiz

###### Abstract

Few-shot learning is a challenging area of research that aims to learn new
concepts with only a few labeled samples of data. Recent works based on
metric-learning approaches leverage the meta-learning approach, which is
encompassed by episodic tasks that make use a support (training) and query set
(test) with the objective of learning a similarity comparison metric between
those sets. Due to the lack of data, the learning process of the embedding
network becomes an important part of the few-shot task. In this work, we
propose two different loss functions which consider the importance of the
embedding vectors by looking at the intra-class and inter-class distance
between the few data. The first loss function is the Proto-Triplet Loss, which
is based on the original triplet loss with the modifications needed to better
work on few-shot scenarios. The second loss function, which we dub ICNN loss
is based on an inter and intra class nearest neighbors score, which help us to
assess the quality of embeddings obtained from the trained network. Our
results, obtained from a extensive experimental setup show a significant
improvement in accuracy in the miniImagenNet benchmark compared to other
metric-based few-shot learning methods by a margin of $2\%$, demonstrating the
capability of these loss functions to allow the network to generalize better
to previously unseen classes. In our experiments, we demonstrate competitive
generalization capabilities to other domains, such as the Caltech CUB, Dogs
and Cars datasets compared with the state of the art.

###### keywords:

Few-Shot Learning Computer Vision Deep Learning Characterization Image
Generation

##  1 Introduction

Despite recent advances in deep learning research in various computer vision
and NLP tasks [40], it remains a challenge for the standard supervised
learning setting to achieve satisfactory results when learning from just a
small amount of labeled data [37]. Current deep learning algorithms tend to
overfit when they are given a small dataset for training, limiting their
generalization capabilities [18]. Moreover, there are many problem domains
where obtaining labeled data can be difficult to obtain or it entails a lot of
manual work to get the data with the corresponding annotations or ground
truth. This represents a major problem in many real-world applications (i.e.,
medicine) as some of the instances we are interested in are very rare [31].

Therefore in recent years the Few-shot learning (FSL) paradigm has been
proposed ([19, 36, 41, 38, 11, 29]) as a way to deal with this data scarcity
problem. For classification tasks, the main goal of such methods is to
categorize previously unseen data into a set of new classes, given only a
small amount of labeled instances per class. The main challenge for FSL is to
apply a fine-tuning process to an existing embedding network to adapt it to
new classes; as a matter of fact, the main issue is that such process can
easily lead to overfitting, due to the few labeled samples available for each
class.

Figure 1: In our work we propose to use two loss functions that work well for
metric meta-learning approaches for few-shot classification. We optimize an
embedding network based on the error calculated by the ICNN Loss or the proto-
triplet loss. Both losses aim to increase the inter-class distance and
decrease the intra-class distance between samples of different classes.

(a) The ICNN Loss is based on a score given by the intra and inter class
distance from the batch points. For each data point, we measure its Inter and
Intra Class Nearest Neighbors score and optimize the embedding network based
on the quality of the generated features.

(b) The Proto-Triplet loss takes as anchor a query point, the prototype from
the same class as positive point and the nearest prototype of different class
as the negative point.

Figure 2:  Comparison between the two distances.

Currently, two main approaches to FSL exist: the first one is based on meta-
learning methods ([11, 1, 23, 9]), where the basic idea is to learn from
diverse tasks and datasets and adapt the learned model to new datasets. A
second approach is based on metric-learning methods ([44, 19]), where the
objective is to learn a pairwise similarity metric such that a score (given by
some distance) is high for similar samples, whilst dissimilar samples get a
low score. Subsequently, these metric learning methods undertook a hybrid
approach, as they started to adopt the meta-learning policy to learn across
tasks ([36, 41, 38]). The main objective of these methods is to learn an
effective embedding network in order to extract useful features from a given
task and discriminate on the classes which we are trying to predict. From this
basic learning setting, many extensions have been introduced to improve the
performance of metric learning methods. Some of these works focus on pre-
training the embedding network ([6]), others introduce task attention modules
([7, 22, 45]), whereas others seek to optimize the embeddings ([21]) and yet
others employ a variety of loss functions ([45]). However, only a few methods
have explored mechanisms for enforcing class separability via a custom loss
function during training.

Our approach introduced herein can be seen as a hybrid between some of the
above-mentioned approaches, as we attempt to improve jointly the embeddings
and we also investigate the impact of two loss functions for FSL-based
classification tasks.

More specifically, we explore the use of two different loss functions based on
the concepts of inter-class and intra-class distance. The first one is the
well-known proto-triplet loss, which to the best of our knowledge has not been
explored in the context of few shot learning; the second, which we dubbed ICNN
loss (based on the Inter and Intra Class Nearest Neighbor score first
introduced in [13]) represents a novel loss function that evaluates the
quality of the features obtained based on the inter/intra class, the variance
and the class ratio from nearest neighbors.

The basic working principle of our proposed approach is shown in Figure 1. As
it can be observed in the figure, the used loss functions allow us to jointly
optimize the embedding network and learn more discriminative features across
tasks. In our experiments, we obtained an accuracy performance of 61.32% and
79.93% in the 5-way 1-shot and 5-way 5-shot settings respectively using the
proto-triplet loss, and 60.79% and 80.41% using our novel ICNN Loss on the
MiniImagenet dataset which represents an improvement of about 2% compared to
the state of the art. We further test our methods on the CUB, Caltech,
Stanford Dogs and Stanford Cars datasets to assess the generalization
capabilities of the proposed framework, obtaining satisfactory results.

The ICNN Loss and Proto-Triplet Loss are two methods used for optimizing an
embedding network based on the quality of the generated features. As shown in
Figure 2, the ICNN Loss measures the intra and inter class distances of batch
points and assigns each data point an Inter and Intra Class Nearest Neighbors
score. The Proto-Triplet Loss, on the other hand, is based on triplet loss and
calculates the similarity between points using prototypes of different classes
as anchor, positive, and negative points.

The rest of the paper is organized as follows: In Section 2, we describe the
related work in the few-shot learning problem. In Section 3 we describe the
proposed loss functions. We first explain the Proto-triplet loss function, and
then we explain the rationale for the ICNN Loss function and its derivation.
Then we discuss some of the design choices we needed to take for the ICNN
loss. In Section 4 we detail the experimental setup used for the
implementation of the models. In Section 5 we show the results obtained and
discuss about its performance. Finally, in Section 6 we present our
conclusions and discuss future work.

##  2 Related Work and Motivation

###  2.1 Deep Metric Learning

The goal of metric learning is to learn a similarity function from the data.
More specifically, it aims to learn feature embeddings in a way that reduces
the distance between embeddings corresponding to instances of the same class
(intra-class distance) and increases the distance between embeddings
corresponding to instances of different classes (inter-class distance). The
vast majority of deep metric learning (DML) approaches make use of an
embedding network to learn the discriminating features that will be exploited
for computing the similarity metric. Below, we review some of the more
relevant deep metric learning methods in the recent literature.

One of the fundamental methods in DML is the Siamese Networks ([19]), which is
a symmetric neural network architecture that consists of two sub-networks,
both sharing the same parameters. These networks learn their parameters by
calculating a distance metric between the feature embeddings of each sub-
network, which are fed a different input. The loss function used in Siamese
Networks is the contrastive loss or pairwise ranking loss, which seeks for the
distance of samples from the same class to be small and from different classes
to be large.

The second important metric learning method is the Triplet Network ([35]),
whose underlying architecture can also be described by a symmetric neural
network. However, this method makes use of three identical sub-networks
sharing the same parameters. The input of the three sub-networks is
encompassed by three different images: The first one is the anchor (the
baseline image), the second is the positive sample (an instance that belongs
to the same class as the anchor), and the third is the negative sample (an
instance that belongs to a different class than the anchor). This network uses
the triplet loss to learn discriminative feature embeddings, and it works by
ensuring that the anchor image is close to the positive images and far away
from the negative images.

These metric learning methods have been widely used for different purposes,
such as image retrieval ([42]), face recognition ([35, 39, 15]), person re-
identification ([43]), video surveillance ([16]), three-dimensional modelling
([10]), signature verification ([5]), medical image analysis ([2]), text
understanding ([28, 3]), among other problems.

###  2.2 Meta-learning for Few-shot learning

As deep learning began to yield superior results in many machine learning
problems, some authors proposed to use meta-learning policies in order to
optimize deep models [1, 23, 9]. The meta-learning policy refers to an
approach geared towards learning across tasks, followed by a fine-tuning
process to adapt the model to new tasks, instead of a learning setting based
on samples. The meta-learning objective aims to learn the parameters $\theta$
that minimize the loss across all given tasks.

FSL represents the perfect setting for testing meta-learning algorithms
because of the few-labeled data given to each task. [4]. The meta-learning
strategy for tackling FSL problems has been traditionally divided into two
stages: a meta-train and a meta-test phase. The meta-learning setup consists
of episodic tasks, which can be seen as batches in traditional deep learning.
A few-shot $K$-way $C$-shot image classification task is given $K$ classes and
$C$ images per class. The task-specific dataset can be formulated as
$D=\\{D_{train},D_{test}\\}$, where
$D_{train}=\\{(X_{i},y_{i})\\}_{i=1}^{N_{train}}$ denotes the classes reserved
for the training phase and $D_{test}=\\{(X_{i},y_{i})\\}_{i=1}^{N_{test}}$
denotes the classes reserved for testing.

For each meta-train task $T$, $K$ class labels are randomly chosen from
$D_{train}$ to form a support set and a query set. The support set, denoted by
$S$, contains $K\times C$ samples ($K$-way $C$-shot) and the query set,
denoted by $Q$, contains $n$ number of randomly chosen samples from the $K$
classes. The training phase uses an episodic mechanism, in which each episode
$E$ is loaded with a new random task taken from the training data. For the
meta-test phase, the model is tested with a new task $T$ constructed with
classes that were not seen during the meta-training stage.

Few-shot learning methods can be categorized based on what a given model seeks
to meta-learn. Some approaches consist on having a base-learner and a meta-
learner, where the meta-learner parameters are optimized by gradually learning
across tasks to facilitate the fast learning of the base-learner for each
specific task. Model-Agnostic Meta-Learning (MAML) ([11]) belongs to this
class and its main idea is to search for a good parameter initialization such
that the base learner can rapidly generalize with this initialization. Then,
REPTILE ([29]) incorporates an $L_{2}$ loss to simplify the computation of
MAML. Further on, LEO ([34]) was proposed as a network to learn low
dimensional latent embeddings of the model.

On the other hand, Meta-SGD ([26]) also learns the base learner update
direction and learning rate on the meta-learning process. Meta-Learner LSTM
([32]) proposes to fine-tune the base learner by a LSTM-based meta-learner,
which takes as input the loss and gradient of the base learner with respect to
each support sample. Other approaches seek to learn the similarity metric that
is expected to be transferable across different tasks.

###  2.3 Metric meta-learning for few-shot learning

There exists an entire branch of meta-learning approaches that aim to solve
the few-shot learning problem by leveraging some of the core ideas of metric
learning. These approaches adopt the meta-learning setup to learn the
similarity metric expected to generalize across different tasks. Certain
baseline methods have achieved important milestones for few-shot learning,
such as Prototypical Networks ([36]), Matching Networks ([41]) and Relation
Networks ([38]). Prototypical Networks is the model we will be using as a
basis in this work, as it also functions by taking the center of support
samples embeddings from each class to create class prototypes.

These models use a distance metric (typically the euclidean distance) to
predict the probability of belonging to a class for each query sample. The
Matching Networks predict the probability of a query sample by measuring the
cosine similarity between the query embedding and each support sample
embedding. On the other hand, the Relation Networks adopt a learnable CNN as
the pairwise similarity metric, which takes the concatenation of feature maps
of the support sample and the query sample as input and outputs the relation
score. These three methods can be considered the base metric learning
approaches for few-shot learning.

Further on, some recent works have focused on introducing task attention
modules ([7, 22, 45]), whereas others try to optimize the embeddings ([21])
and others add a second term to the loss function ([45]). Up until now, there
has been a lack of research for loss functions that work for the problem of
few-shot learning tackled from a metric-learning perspective. Although these
methods utilize deep networks to extract discriminant deep features, they do
not take full advantage of the relationship among the input samples. Hence, we
are motivated to explore strategies to improve the feature embedding in terms
of their efficiency to be transferable to handle unseen class samples and
their generality for few-shot classification.

Our work, which is based on prototypical networks, seeks to contribute
alleviate this problem, by introducing the concept of distance in the loss
function. Our proposed model belongs to these meta-learning approaches based
on metric learning, by adopting the idea of the intra-class and inter-class
variance into account for the construction of two different loss functions,
which will help us to better optimize an embedding network to obtain more
discriminant feature vectors. We aim to meta-learn a feature embedding that
performs well, not only in the training classes but more importantly, in the
novel classes. Specifically, the feature embedding should map similar samples
close to one another and dissimilar ones far apart. This is well-aligned with
the philosophy of triplet-like learning. However, the general triplet network
only interacts with a single negative sample per update, while few-shot
classification requires a comparison with multiple query samples, typically of
different classes.

For this, we make use of two novel loss functions, briefly introduced above: a
modified proto-triplet loss and inter-intra class nearest neighbors score loss
for training a Convolutional Neural Network (CNN) as an embedding function.
These loss functions optimize the embedding space across the meta-learning
tasks. Our experimental results show the proposed losses yield an increase in
accuracy in the miniImagenNet benchmark compared to other metric-based few-
shot learning methods by a margin of $2\%$. We also demonstrate the capability
of these loss functions to produce models able to generalize better to
previously unseen classes, as demonstrated by experiments in the CUB-200,
Caltech-101 and the Stanford Dogs and Cars datasets.

The main contributions of this paper are two loss functions based on
contrastive learning approaches. The first loss function is inspired by the
well-known triplet loss with adjustments for the few-shot learning setting.
The second loss function is a novel method that calculates the error based on
the quality of feature embeddings obtained.

##  3 DML-based loss functions for FSL

###  3.1 Proto-Triplet Loss

The traditional Triplet Loss ([35]) is widely used for metric learning, where
the objective is to train a learner by using a similarity comparison metric
between the sampled data. The loss function aims at pulling similar samples
close to each other while pushing away the samples of different classes far
away. This is the classic paradigm of clusters for good classification. The
objective of the triplets is to create a manifold, where classes live on.
Thus, the triplets are used to learn why classes are different and why are
similar on the manifold.

Our proposed Proto-Triplet Loss is based on the original Triplet Loss with the
difference that we are using the data points obtained as prototypes for
calculating the loss, making it more suitable for the few-shot learning
setting. The formula for the proto-triplet loss is similar to the original
triplet loss:

$L(X_{a},X_{p},X_{n})=\left[\left\|f_{a}-f_{p}\right\|^{2}-\left\|f_{a}-f_{n}\right\|^{2}+\alpha\right]_{+},$
(1)

where $\left[\right]_{+}$ is the max between 0 and the result, also known as
hinge loss. $f_{a}$ is the embedding from the query, $f_{p}$ is the prototype
from the same class as the query and $f_{n}$ is the nearest prototype of a
different class as the query. $\alpha$ is a margin hyperparameter to be
enforced between the negative and positive pairs. The goal of this function is
to keep the distance between the query and the prototype of the same class
smaller than the distance between the query and the prototype from a different
class.

Furthermore, inspired in ([25]), we propose to use the Proto-Triplet Loss with
$K$ reference points to use more than one negative sample. In the meta-
learning setting, we compare an instance with multiple classes, so our loss
function should be more effective by performing a comparison with more
prototypes of different classes.

For the negative samples, we take the $K$ nearest prototypes of different
classes to form the triplets. The modified function is formulated as:

$L(X_{a},X_{p},X_{n})=\frac{1}{K}\sum_{i\in
U}\left[\left\|f_{a}-f_{p}\right\|^{2}-\left\|f_{a}-f_{n}\right\|^{2}+\alpha\right]_{+},$
(2)

where $U$ is the set of triplets made by the $K$ negative prototypes. The rest
of the terms in the formula are the same as stated in the previous equation.

###  3.2 ICNN Loss

Given the basic idea of class separation and high density per class, the Inter
and Intra Class Nearest Neighbors Score (ICNN Score) ([13]) is proposed as a
measure to remove noisy features and improve the performance of manifold-based
algorithms for supervised dimensionality reduction, aiding the feature
selection by a subset evaluation. The ICNN score is the result of searching
for better methods of feature selection than frameworks based on things as the
Maximal Information Coefficient [12]. For this, the ICNN is based on two main
concepts: the inter-class and the intra-class distances. The former describes
the distance between data points of different classes, while the latter refers
to the distance between data points of the same class.

The ICNN assigns a score by measuring the distance and variance of the inter
and intra $k$-nearest neighbors of each instance in the data. The equation to
measure the data points and assign them a score is represented by three
different terms: $\lambda$, $\omega$ and $\gamma$. The ICNN formula is given
as follows:

$ICNN(X)=\frac{1}{\left|X\right|}\sum_{x_{i}\in
X}\lambda(x_{i})^{\frac{1}{p}}\omega(x_{i})^{\frac{1}{q}}\gamma(x_{i})^{\frac{1}{r}},$
(3)

where $p$, $q$ and $r$ are control constants.

##  4 Designing the ICNNScore

In order to build and understand the Inter/Intra Class Nearest Neighbors
Distance Score (ICNNScore), it is necessary to define the concepts of inter-
class distance and intra-class distance [17]. The main idea for inter-class
distance is that different classes should be distant or optimal with respect
to a distance $d$ (Eq. 4),

$D^{*}\left(C_{i},C_{j}\right)=\sup_{d}\frac{1}{n_{i}n_{j}}\sum_{x_{j},x_{k}\in
C_{i}}d\left(x_{j},x_{k}\right),$ (4)

which basically represents an average of the maximal distance matrix between
elements of the two classes $i$ and $j$. In a similar way, the optimal intra-
class distance is defined as

$I^{*}\left(l_{i}\right)=\inf_{d}\frac{1}{n_{i}}\sum_{k_{1}=1}^{n_{i}}\sum_{k_{2}=k_{1}+1}^{n_{i}}d\left(x_{i,k_{1}},x_{j,k_{2}}\right).$
(5)

These quantities represent the distance between different classes and the
density of those classes, respectively. Although these equations represent the
basic ideas of inter- and intra-class distances, they are useful to develop a
score that can be transformed into a loss function. For this, we propose a
function that minimizes penalization to the neighborhood class elements to a
particular sample $x_{i}\in C_{j}$. In addition, we also define a function
that maximizes the penalization of elements $x_{k}\notin C_{j}$ to such sample
$x_{i}$ based on their distance proximity. In the ideal case with respect to
$x_{i}$ at its neighborhood, the distances of elements in the same class are
close to zero. In addition, elements in the same neighborhood, but in
different classes than $x_{i}$, their distances to it are as large as possible
(Figure 3). The former bound is easy to control and has tendency to zero.
However, the latter bound can be problematic given the different distances
that can be obtained. Thus, instead of having any large random possible
distances, we force the bound of these distances to a known constant.

Figure 3: The ideal case for the ICNNScore

For this purpose, we define the following neighborhoods: $N_{x_{i}}$ the
$k$-nearest neighbors of $x_{i}$ at the same class, and $N_{\tilde{x}_{i}}$
the $k$-nearest neighbors of $x_{i}$ at different classes. Using these sets,
it is possible to define the following function based on a differentiable
distance $d\left(x_{i},x_{j}\right)$,

$\displaystyle\alpha\left(x_{i}\right)=\max\left\\{d\left(x_{i},p\right)|p\in
N_{x_{i}}\cup N_{\tilde{x}_{i}}\right\\}$ (6)

We also have the distance function,

$\theta\left(x_{i}\right)=\min\left\\{d\left(x_{i},p\right)|p\in N_{x_{i}}\cup
N_{\tilde{x}_{i}}\right\\}$ (7)

Using these functions, we define a normalization of the distances for all the
elements in the neighborhood of $x_{i}$:

$h\left(x_{i},p\right)=\frac{d\left(x_{i},p\right)-\theta\left(x_{i}\right)}{\alpha\left(x_{i}\right)-\theta\left(x_{i}\right)},$
(8)

which is a min-max normalization that enforces the desired bounds in the
interval $[0,1]$. Now, we define a penalization $\lambda$ function for
defining the concept of ”far” or ”near” to a specific $x_{i}$:

$\displaystyle\lambda_{\tilde{x}}\left(x_{i}\right)$ $\displaystyle=\sum_{p\in
N_{\tilde{x}_{i}}}h\left(x_{i},p\right),$ (9)
$\displaystyle\lambda_{x}\left(x_{i}\right)$ $\displaystyle=\sum_{p\in
N_{x_{i}}}\left[1-h\left(x_{i},p\right)\right]$ (10)

The first equation (Eq. 9) represents the normalized penalization of the
elements in a certain neighborhood of elements in different classes than the
class of $x_{i}$, and the second equation represents a similarity to elements
at $N_{x_{i}}$. Therefore, we have

$\displaystyle\lambda\left(x_{i}\right)=\frac{\lambda_{\tilde{x}}\left(x_{i}\right)+\lambda_{x}\left(x_{i}\right)}{\left|N_{\tilde{x}_{i}}\cup
N_{x_{i}}\right|}$ (11)

Thus, if we want to maximize this function, the average
$\lambda_{\tilde{x}}/N$, with $N=\left|N_{\tilde{x}_{i}}\cup
N_{x_{i}}\right|$, needs to be maximized. Thus, the distances of the elements
not belonging to the class of $x_{i}$ are maximized to it. In the case of
$\lambda_{x}/N$, the similarities are maximized toward one, thus forcing the
elements belonging to the class of $x_{i}$ to get near to it or to zero.
However, when going through the battery of experiments a change is proposed on
the original $\lambda$ function (Eq. 11) to improve results for the few-shot
learning setup,

$\lambda\left(x_{i}\right)=\frac{\lambda_{\tilde{x}}\left(x_{i}\right)}{\left|N_{\tilde{x}_{i}}\right|}+\frac{\lambda_{x}\left(x_{i}\right)}{\left|N_{x_{i}}\right|}.$
(12)

Therefore, we have that each of $\lambda^{\prime}s$ $x$ functions finish as

$\displaystyle\frac{\lambda_{\tilde{x}}\left(x_{i}\right)}{\left|N_{\tilde{x}_{i}}\right|}$
$\displaystyle\thickapprox E_{h}\left[h\left(x_{i},p\right)|p\in
N_{\tilde{x}_{i}}\right],$
$\displaystyle\frac{\lambda_{x}\left(x_{i}\right)}{\left|N_{x_{i}}\right|}$
$\displaystyle\thickapprox 1-E_{h}\left[h\left(x_{i},p\right)|p\in
N_{x_{i}}\right].$

These expected values depend on the original distribution of the
$d\left(x_{i},p\right)$ given that we have a uniform a priori, and the
likelihood of the defined distance. Now, if the $\lambda$ function is
maximized with respect to the representation of $x_{i}$ and $p$, two phenomena
occur: First, the inter-distance on $N_{x_{i}}$ is minimized, and second, the
intra-distance on $N_{\tilde{x}_{i}}$ is maximized. This basically corresponds
to highly dense and compact class clusters with large distances between them,
our ideal case for classification. Therefore, integrating this $\lambda$
function to a cost function will enforce these properties in the generated
embedding during the learning process of a neural network.

Figure 4: Lambda ($\lambda$) is the function that penalizes the neighbors of
$X_{i}$ with the same class based on how distant they are, and the neighbors
of different classes based on how close they are.

######  Proposition 1.

Given the $\lambda$ function, and a smooth embedding representation for the
samples $x_{i}$ based on weights $\mathcal{F}$. If $\lambda$ is maximized by
using a gradient method on those weights, the embedding will change to
increase the inter-class distance and minimize the intra-class distance at the
classes.

###### Proof.

Appendix A.1 ∎

This proposition allows to say that a cost function based on the ICNN helps to
separate the class clusters when training on data. Next, given that we have
$h\left(x_{i}\right)\leq 1$, if we add overall terms $i$, we finish with

$\displaystyle\sum_{p\in N_{\tilde{x}_{i}}}h\left(x_{i},p\right)$
$\displaystyle\leq\left|N_{\tilde{x_{i}}}\right|\longrightarrow\frac{\lambda_{\tilde{x}}\left(x_{i}\right)}{\left|N_{\tilde{x_{i}}}\right|}\leq
1$ (13)

In a similar way,
$0\leq\frac{\lambda_{x}\left(x_{i}\right)}{\left|N_{x_{i}}\right|}\leq 1$
given that also $h\left(x_{i},p\right)\leq 1$ for $p\in N_{\tilde{x}}\cup
N_{x}$. In this way, $0\leq\lambda\left(x_{i}\right)\leq 2$.

Now, we would like to avoid large changes while learning the distances between
elements in the neighborhood of $x_{i}$. For this, we introduce a penalization
function over the terms $h$ that minimizes the possible variance (Eq. 14).

$\omega\left(x_{i}\right)=\alpha-\left[Var\left(\lambda_{\tilde{x}}\left(x_{i}\right)\right)+Var\left(\lambda_{x}\left(x_{i}\right)\right)\right]$
(14)

The variance of this function depends on the original distribution of values
$d\left(x_{i},p\right)$.

######  Proposition 2.

Given the $\omega$ function, and the smooth embedding representation based on
weights $\mathcal{F}$. If the loss function is minimized, the $\omega$ is
maximized by using the gradient method on those weights, then the inter and
intra-class variances can be minimized by selecting an adequate $alpha$.

###### Proof.

Appendix A.2 ∎

Thus, optimizing a loss function based on the ICNN helps to minimize the
variance of the learned manifold or in other words it favors dense clusters.
In order to obtain the bounds for $\omega$, we can use a classic definition of
variance,

$\displaystyle Var\left(\lambda_{\tilde{x}}\left(x\right)\right)$
$\displaystyle=E\left(\lambda_{\tilde{x}}^{2}\left(x\right)\right)-E^{2}\left[\lambda_{\tilde{x}}\left(x\right)\right]$

Thus,

$\displaystyle E\left[\lambda_{\tilde{x}}\left(x\right)\right]$
$\displaystyle\thickapprox\frac{1}{\left|X\right|}\sum_{x\in
X}\lambda_{\tilde{x}}\left(x\right)$
$\displaystyle\thickapprox\frac{1}{\left|X\right|}\sum_{x\in X}\sum_{p\in
N_{\tilde{x}}}h\left(x,p\right)$

In this way, we have that the variance is bounded as

$\displaystyle Var\left(\lambda_{\tilde{x}}\left(x\right)\right)\leq$
$\displaystyle\frac{1}{\left|X\right|}\sum_{x\in X}\left(\sum_{p\in
N_{\tilde{x}}}h\left(x,p\right)\right)^{2}-...$
$\displaystyle\left(\max_{x_{i}}\left|N_{\tilde{x}_{i}}\right|\right)^{2}$
$\displaystyle\leq$ $\displaystyle\frac{1}{\left|X\right|}\sum_{x\in
X}\left(\sum_{p\in N_{\tilde{x}}}h\left(x,p\right)\right)^{2}$
$\displaystyle-\left(\max_{x_{i}}\left|N_{\tilde{x}}\right|\right)^{2}\leq
k_{1}^{2},$

where $k_{1}=\max_{x_{i}}\left|N_{\tilde{x}}\right|$ This $k_{1}$ happens when
all the elements at $N_{\tilde{x}}$ have a distance of one. In a similar way,
we have $Var\left(\lambda_{x}\left(x\right)\right)\leq k_{2}^{2}$ which is
also $k_{2}=\max_{x_{i}}\left|N_{x}\right|$ In the reverse case, we have that
the lower bound for both variances is 0. In this way, we have that we can use
$\omega=k_{1}^{2}+k_{2}^{2}$ to go force the best case where $x$ has the
maximal separation from elements not in the class and minimal separation from
elements in the same class.

Figure 5: Omega ($\omega$) penalizes the distance variance of neighbors. High
variance is not desirable as it increases the chance of class overlaps.

Finally, we have the last penalization for the ICNNScore which represents how
many elements in the neighborhood $N_{\tilde{x}_{i}}\cup N_{x_{i}}$ belong to
the same class as $x_{i}$,

$\gamma\left(x_{i}\right)=\frac{\left|N_{x_{i}}\right|}{\left|N_{x_{i}}\right|+\left|N_{\tilde{x_{i}}}\right|}$
(15)

Therefore, using these three functions, it is possible to build the ICCNScore,

$ICNN\left(X\right)=\frac{1}{\left|X\right|}\sum_{x_{i}\in
X}\lambda\left(x_{i}\right)^{\frac{1}{p}}\omega\left(x_{i}\right)^{\frac{1}{q}}\gamma\left(x_{i}\right)^{\frac{1}{r}}$
(16)

In order to use this score as a loss function we simply use the negative value
of the logarithm function,

$ICNNLoss\left(X\right)=-log\left(ICNN\left(X\right)\right)$ (17)

Thus minimization of the ICNNLoss becomes the maximization of each of the
elements of the ICNN score.

##  5 Experimental Design

The experiments made on this work were designed to answer the following
questions: First, is the Proto-Triplet Loss competitive with other metric-
learning state-of-the-art models? Second, is the ICNN Loss competitive with
other metric-learning state-of-the-art models? Third, how do our loss
functions optimize the feature space to make the features more discriminating?

The results obtained in this study indicate that both the Proto-Triplet loss
and the ICNN loss are competitive with current state-of-the-art approaches.
Furthermore, when these loss functions are combined, they achieve a high level
of performance that surpasses other models. Finally, we can see in Figure 6
that our loss functions optimize the feature space by providing a clearer
intra-class and inter-class separation of the feature embeddings.

Having defined the equation for the ICNN Loss function, we test different
scenarios by varying the data points given to the algorithm and combining it
with cross entropy loss and Proto-Triplet loss. For training the network, we
know to which class the support data and the query data belong, therefore, we
can use both sets to assign a score to the current batch. The different
options were tested by assigning a different score to the support data, the
query data and both the support and query data. For the support data,
calculating the ICNN Score is straightforward, as each data point has an
assigned score based on its nearest neighbors and is averaged across the set.
For the query data, each query is given a score based on the nearest neighbors
of the support data points. When using both the support and query data, the
data points are combined and considered for calculating the score for each
instance. The options for the algorithm are the following:

  * •

Score only on support data

  * •

Score on support data and score in query data

  * •

Score of support and query data together

We can also opt to make a different choice of data given to the ICNN algorithm
when working with the query set. As our method aims at classifying the queries
based on the prototypes, we added the option to give a score to the query set
by using the prototypes as neighbors. Each query instance is given a score by
using the $k$ nearest prototypes.

##  6 Materials, Metrics, and Methods

###  6.1 Datasets

We evaluate our experiments using the MiniImageNet dataset ([41]), which is a
version of the ImageNet Large Scale Visual Recognition Competition 2012
([33]). Following the split proposed by [32], this version of ImageNet is
divided into 64 classes for training, 16 classes for validation and 20 classes
for testing, making a total of 100 classes for the meta-learning. Each class
contains 600 images to have a total of 60,000 images. This dataset is used as
a benchmark to evaluate most of the state-of-the-art few-shot learning
methods.

We further test the proposed framework with other well-known datasets for
assessing the generalization capabilities of te final model trained in
MiniImagnet, namely the Caltech-101, Stanford Dogs, Stanford Cars and CUB
datasets. Caltech [14] is a dataset consisting of 101 widely varied
categories, as well as a background category. For each class, there are around
40 and 800 images, while most classes have about 50 images. These classes are
randomly split to use 20 classes as the testing set. Stanford Cars [20]
contains fine-grained images from 196 classes of cars.

###  6.2 Evaluation Metrics

We follow the same method as other metric learning methods for evaluating our
results ([36, 41, 38]), reporting the mean accuracy (%) of 1,000 randomly
constructed tasks taken from the testing set along with the 95% of confidence
interval. Each task of the testing phase contains 15 query images per class.

###  6.3 Implementation Details

We follow the same setting as other few-shot learning models ([41, 36, 38]),
making the experiments under the setting of 5-way 1-shot and 5-way 5-shot, and
using 15 query images for each class in the task. The input images are resized
to 84 $\times$ 84, and then normalized. We construct 100 random tasks for the
training phase over 200 epochs. We validated each epoch with another 500
randomly constructed tasks using images from the validation set. Finally, in
the testing phase we construct 1000 tasks with images from the testing set. We
use the same random seed over all the experiments.

Based on the current state-of-the-art typical setting, we tested two different
networks for the feature extractor: a ConvNet and a ResNet-12. Using larger
networks makes the model more prone to overfitting, since there is low
availability of data. The ConvNet follows the same architecture setting as
previous works ([41], [36]). This network is composed of 4 layers of
convolutional blocks, with each block having a 3 $\times$ 3 convolution with
64 filters, followed by a batch normalization and a ReLU layer. For this
network, we used the Adam Optimizer, as in previous works, with an initial
learning rate of $1\times 10^{-3}$ and a step size of 20. The ResNet follows
the same architecture as other recent works ([22], [21]). This network is pre-
trained with images from the training set, using the Stochastic Gradient
Descent (SGD) optimizer with a momentum of 0.9 and a learning rate of 0.1 over
100 epochs with a batch size of 128. After the pre-training, we meta-train the
network using the SGD optimizer with a learning rate of $1\times 10^{-4}$,
momentum of 0.9 and a step size of 20.

####  6.3.1 Ablation Studies

The ablation studies were split into three separate experimental setups:
experiments with different combinations of the proto-triplets and the ICNN
lossess and various tests with combinations of the network used as feature
extractor (ConvNet and ResNet-12). For all the experiments, we repeated the
design choices using both networks to compare against baselines and recent
methods in few-shot image classification. The models using a ResNet-12 greatly
improve the accuracy performance compared to those using a ConvNet.

The initial step in our methodology is to conduct experimentation in order to
assess the algorithm’s efficacy when applied to the individual components in a
few-shot task.

  * •

(i) ICNN using only the support data. This represents the most basic use of
our loss function, since it is used only on a subset of the data available on
each training task.

  * •

(ii) ICNN using the support data and the query data separately with the aim of
determining the degree of improvement achieved when utilizing both sets of
data, in comparison to utilizing only the support data. The loss for each set
is calculated and subsequently combined to determine the overall loss.

The next step of our experimentation incorporates the cross-entropy loss
function into the pre-existing settings in order to evaluate how the model
improves by its inclusion.

  * •

(iii) Addition of the cross-entropy loss and ICNN using only the support data.
This experiment was executed to compare how the model performance improves
under the most basic setting.

  * •

(iv) Addition of the cross-entropy loss and ICNN in support and query data
separately. Similarly to the previous settings, we added the cross-entropy
loss to evaluate the impact on the performance of the model.

In the next phase of experimentation, we introduce prototypes into the task
data. The prototypes are computed using both the support and query sets, and
subsequently used for the calculations of the ICNN algorithm.

  * •

(v) ICNN with the prototypes from the support and query data. The rationale
behind this approach is that the prototypes serve as the central
representation of each class embedding, therefore the ICNN algorithm should be
able to provide accurate scores with these mean representations.

  * •

(vi) Add the cross entropy and the ICNN with the prototypes from the support
and query data. Similarly to previous settings, this experiment aims to
determine how the model performance is enhanced by adding the cross-entropy
loss to the prototypes setting.

In the final step of our experimentation, we evaluate the algorithm’s
performance by calculating the ICNN loss using the complete task data, which
is obtained by combining the support and query sets.

  * •

(vii) ICNN using the full data from each task. Through this approach, our goal
is to evaluate the algorithm’s performance using the entirety of the data
available for each task.

  * •

(viii) Adding the cross-entropy loss and the ICNN using the full data from
each task. Same as with previous settings, we aim to evaluate how the cross-
entropy improves the current ICNN setting.

##  7 Results and Discussion

###  7.1 Standard few-shot learning evaluation

Model | Feature Extractor | 1-shot | 5-shot
---|---|---|---
Matching Networks [41] | ConvNet | 43.56 $\pm$ 0.84% | 55.31 $\pm$ 0.73%
Prototypical Networks [36] | ConvNet | 49.42 $\pm$ 0.78% | 68.20 $\pm$ 0.66%
Relation Networks [38] | ConvNet | 50.44 $\pm$ 0.82% | 65.32 $\pm$ 0.70%
Baseline* [8] | ConvNet | 41.08 $\pm$ 0.70% | 54.50 $\pm$ 0.66%
MAML [11] | ConvNet | 48.70 $\pm$ 1.84% | 63.11 $\pm$ 0.92%
Reptile [29] | ConvNet | 49.97 $\pm$ 0.32% | 65.99 $\pm$ 0.58%
Ours Proto-Triplet | ConvNet | 49.82 $\pm$ 0.73% | 68.76 $\pm$ 0.46%
Ours ICNN | ConvNet | 49.71 $\pm$ 0.78% | 68.66 $\pm$ 0.58%
SNAIL [27] | ResNet-12 | 55.71 $\pm$ 0.99% | 68.88 $\pm$ 0.92%
DN4 [24] | ResNet-12 | 54.37 $\pm$ 0.36% | 74.44 $\pm$ 0.29%
TADAM [30] | ResNet-12 | 58.50% | 76.70%
K-tuplet Net [25] | ResNet-12 | 58.30 $\pm$ 0.84% | 72.37 $\pm$ 0.63%
ProtoNets + CTM [22] | ResNet-12 | 59.34 $\pm$ 0.55% | 77.95 $\pm$ 0.06%
Principal Characteristic Net [45] | ResNet-12 | 63.29 $\pm$ 0.76% | 77.08 $\pm$ 0.68%
Ours Proto-Triplet | ResNet-12 | 61.32 $\pm$ 0.52% | 79.93 $\pm$ 0.76%
Ours ICNN | ResNet-12 | 60.79 $\pm$ 0.62% | 80.41 $\pm$ 0.47%
|  |  |

Table 1: Test accuracies on miniImagenet in the 5-way setting for both 1-shot
and 5-shot.

For the MiniImageNet dataset, we evaluate our method under the two most common
few-shot learning settings: 5-way 1-shot and 5-way 5-shot. The results are
compared against base metric learning methods for few-shot classification
([36, 41, 38]), and against recently proposed methods ([25, 22, 45]).

As detailed in Table 7.1, our method outperforms most of the baselines in the
5-way 1-shot setting, getting only lower accuracy than Reptile ([29]) and
Relation Networks [38]) when using a ConvNet as Feature extractor, and lower
accuracy than Principal Characteristics Net ([45]) when using a ResNet-12. For
the 5-way 5-shot setting, our method outperforms all the baseline methods and
the recent ones using ConvNet and ResNet-12 as feature extractors.

When using a ConvNet as the feature extractor, our method achieves an
improvement of 8.8% over the baseline ([8]), 5.3% over the Matching Networks
([41]) and 0.4% over the base prototypical Networks for the 5-way 1-shot
setting. For the setting of 5-way 5-shot, we achieve an improvement of 13.4%
over the Matching Networks, 3.4% over the Relation Networks and 0.5% over the
prototypical networks.

When using a ResNet-12 as the feature extractor, our method achieves an
improvement of 5.6% over SNAIL ([27]), 3% over the K-tuplet Net ([25]) and 2%
over the Category Traversal Module ([22]) for the few-shot setting of 5-way
1-shot. On the setting of 5-way 5-shot, we obtained an improvement of 11.6%
over SNAIL, 8.1% over the K-tuplets Net and 3.4% over the Principal
Characteristics Net ([45]).

###  7.2 Results of the ablation studies

Model | Backbone | 1-shot | 5-shot
---|---|---|---
$(i)$ ICNN in Support | ConvNet | 41.82 |  55.52
$(ii)$ ICNN in Support + Query | ConvNet | 42.81 | 55.93
$(iii)$ CrossEntropy + ICNN in Support | ConvNet | 49.28 | 68.33
$(iv)$ CrossEntropy + ICNN in Support & Query | ConvNet | 49.71 | 68.66
$(v)$ ICNN in Support & Query(Prototypes) | ConvNet | 41.81 | 55.93
$(vi)$ CrossEntropy + ICNN in Support&Query(Protos) | ConvNet | 48.86 | 67.74
$(vii)$ Full ICNN | ConvNet | 45.42 | 65.86
$(viii)$ Cross Entropy + Full ICNN | ConvNet | 48.62 | 68.12
$(i)$ ICNN in Support | ResNet-12 | 56.07 | 77.44
$(ii)$ ICNN in Support & Query | ResNet-12 | 58.93 | 78.42
$(iii)$ CrossEntropy + ICNN in Support | ResNet-12 | 60.55 | 79.11
$(iv)$ CrossEntropy + ICNN in Support & Query | ResNet-12 | 60.50 | 79.58
$(v)$ ICNN in Support + Query(Prototypes) | ResNet-12 | 59.07 | 79.18
$(vi)$ CrossEntropy + ICNN in Support&Query(Protos) | ResNet-12 | 60.79 | 80.41
$(vii)$ Full ICNN | ResNet-12 | 60.37 | 78.79
$(viii)$ Cross Entropy + Full ICNN | ResNet-12 | 60.32 | 79.30
|  |  |

Table 2: Design choices for the ICNN Loss function using a ConvNet and
ResNet-12 as feature extractors and 5-way tasks on both 1-shot and 5-shot
settings.

Model | Backbone | 1-shot | 5-shot
---|---|---|---
Proto-Triplet | ConvNet | 48.85 | 67.79
Cross Entropy + Proto-Triplet | ConvNet | 41.66 | 66.09
Proto-Triplet + Full ICNN | ConvNet | 46.00 | 62.72
Cross-Entropy + Proto-Triplet + Full ICNN | ConvNet | 49.82 | 68.76
Proto-Triplet | ResNet-12 | 60.87 |  78.78
Cross Entropy + Proto-Triplet | ResNet-12 | 60.09 | 79.33
Proto-Triplet + Full ICNN | ResNet-12 | 59.58 | 78.53
Cross-Entropy + Proto-Triplet + Full ICNN | ResNet-12 | 61.32 | 79.93
|  |  |

Table 3: Design choices for the Proto-Triplet Loss function using a ConvNet
and ResNet-12 as feature extractors and 5-way tasks on both 1-shot and 5-shot
settings.

(a) MiniImagenet Dataset

(b) Caltech Dataset

(c) CUB Dataset

(d) Dogs Dataset

(e) Cars Dataset

Figure 6: Umap visualization of the improved feature embeddings obtained from
the test set for each dataset. The feature embeddings were computed in a 5-way
5-shot setting by constructing 1,000 tasks from the test data. When creating
the tasks for the feature visualization, we kept fixed 5 classes and randomly
draw samples to construct the support and query sets

Dataset | ProtoNet | K-tuplets | Ours
---|---|---|---
2*CUB-200 | 5way-1shot | 39.39 ± 0.68 | 40.16 ± 0.68 | 67.56 ± 0.65
| 5way-5shot | 56.06 ± 0.66 | 56.96 ± 0.65 | 83.62 ± 0.73
2*Caltech-101 | 5way-1shot | 53.28 ± 0.78 | 61.00 ± 0.81 | 69.79 ± 0.57
| 5way-5shot | 72.96 ± 0.67 | 75.60 ± 0.66 | 87.85 ± 0.96
2*Stanford Dogs | 5way-1shot | 33.11 ± 0.64 | 37.33 ± 0.65 | 58.29 ± 0.28
| 5way-5shot | 45.94 ± 0.65 | 49.97 ± 0.66 | 78.08 ± 0.21
2*Stanford Cars | 5way-1shot | 29.10 ± 0.75 | 31.20 ± 0.58 | 72.76 ± 0.86
| 5way-5shot | 38.12 ± 0.60 | 47.10 ± 0.62 | 87.67 ± 0.69

Table 4: Average few-shot classification accuracies (%) on other datasets. All
the experiments are conducted with the same network for fair comparison.

(a) Caltech Dataset

(b) CUB Dataset

(c) Dogs Dataset

(d) Cars Dataset

Figure 7: Examples of tasks on the testing phase for different datasets. For
each dataset, we report 5 tasks where the model classified correctly (green
side) and 5 tasks where it failed to correctly classify and where it was very
far from the correct result.

The results for these different ablation studies for the ICNN loss are
reported in Table 7.2. Our baseline is the simplest form of using the ICNN
loss, by applying the score only to the support data. This gives us the worst
results from the performed experiments, and are not competitive with state-of-
the-art models. Then, we test our loss function by applying the score to
different sets of data. For $(ii)$ and $(iv)$, we carried out tests using ICNN
on support data and query data separately, with the difference that $(iv)$
uses the additional loss of the cross-entropy. When using a ConvNet, we
obtained the best results by using the cross entropy and ICNN in support and
query (iv), improving the baseline performance by around 8%. For $(v)$ and
$(vi)$ we modify the ICNN score on the query by comparing distances from each
query to the prototypes instead of the comparing the distances from each query
to the support data. When using the ResNet-12 as the feature extractor, this
combination $(vi)$ gave us the best results, improving the baseline by around
4% for the 1-shot setting and 3% for the 5-shot setting. The last experiments,
$(vii)$ and $(viii)$, apply the ICNN score to the combined set of support data
and query data. These last experiments didn’t lead us to the best results, but
they are also quite competitive.

The results for the ablation studies for the proto-triplet loss are reported
in Table 7.2. We evaluate the combination of three loss functions: the cross-
entropy, proto-triplet and the full version of the ICNN loss. First we test
the proto-triplet loss alone to optimize the network, and obtain already a
good accuracy performance compared with state-of-the-art results. Then, we
continue by mixing the loss functions and we can observe that combining cross
entropy with proto-triplet and ICNN loss with proto-triplet we get a worse
performance than the obtained using only the proto-triplet loss. The last
design choice is the combination of cross entropy, ICNN loss and proto-triplet
loss, which is the one with the best accuracy performance by improving the
result by 1% using both feature extractors.

###  7.3 Results of generalization on other datasets

In order to measure the capabilities of generalization of the approach
presented in this work, we evaluated the proposed losses and models on
different datasets. Each dataset has distinct characteristics such as the
number of classes, and the number of samples per class. The selected datasets
are the following: CUB-200, Caltech-101 [14], Stanford dogs, and Stanford cars
[20].

In order to compare our results with the state-of-the-art, we evaluated two
backbones: ProtoNet and K-tuplets. For each backbone, two FSL configurations
(5way-1shot, and 5way-5shot) were evaluated for each of the datasets (see
Table 7.2).

For visualization purposes and a qualitative evaluation of the results, we
compared our model with those of ProtoNet using UMAP (Fig. 6). In order to
reinforce the quantitative (Fig. 6) and qualitative (Table 7.2) results, we
show in Fig. 2 examples of tasks on the testing phase for different datasets.
For each dataset, we report 5 tasks where the model classified correctly
(green side) and 5 tasks where it failed to correctly classify and where it
was very far from the correct result.

CUB dataset: Our results, evaluated in CUB dataset have yielded an accuracy of
83.62% for 5way-5shot congiguration (up to 26% higher than K-tuplets and
ProtoNet). Similarly, in 5way-1shot configuration, our approach surpasses the
results of both models up to 27% accuracy. In order to illustrate
qualitatively the results in Fig. 6c, we can see the effect of our model with
respect to ProtoNet, which produces a sparse and overlapping point cloud. On
the other hand, in our model, it can be observed that the groups of points for
each class are well separated, although the points are slightly more
dispersed. The CUB dataset is the most difficult group of images to classify
in our tests. This may be due to high intra-class similarities in the samples
(birds with similar characteristics, Fig. 7c). However, although it is
difficult to classify this type of images, our approach has successfully
reduced the intra-class distance and maximize the inter-class distances.

Caltech dataset: For the Caltech dataset, the results obtained for ProtoNet
and K-tuplets using a 5way-1shot configuration have lower performances than
our model. Although in a 5way-5shot configuration, the results for ProtoNet
and K-tuplets are relatokively good (achieving performances close to 70%).
However, our model obtains up to 87.85%, surpassing the two compared models by
up to 12%. In the UMAP visualization, the results for our model are
outstanding (Fig. 6b): the point clouds are well separated by maximizing the
inter-class distance, and it compacts points of the same classes. This allows
us to observe in Fig. 6b compact and separated groups of points, which is
desirable in classification problems. In Fig. 7a, we can observe that during
the Caltech test our model is efficient when there is a strong similarity
between the true class of the support set and the query set (green side).
However, the model tends to reduce its classification capability when natural
images are presented, i.e., images with non-uniform background or rotation
effects (red side).

Dogs dataset: In the Dogs dataset the 5way-1shot configuration (58.29%) shows
an improvement over ProtoNet and K-tuplets, while it is only surpassed by the
5way-5shot configuration (78.08%). In the UMAP visualization (Fig. 6d) we can
see how the points are better grouped when compared to the results produced by
ProtoNet, which exhibit more dispersion. Although ProtoNet shows a good
separation between classes, our model improves the inter-class distance. For
the Dogs dataset test (Fig. 7c), we can observe that even though this group of
images contains non-homogeneous backgrounds, the model is able to discriminate
between the different classes (green side). However, it can be observed that
it is sensitive to objects that do not represent the class (red side), such as
the presence of people or other objects in the test image.

Cars dataset: The Cars dataset has a larger scatter in the UMAP feature space
for ProtoNet. As can be seen in Fig. 6e (ProtoNet), the intra-class distance
is relatively large, whereas inter-class distances are very small. The results
achieved with ProtoNet go as high as 29.10% and 38.14% using 5-way 1-shot, and
5-way 5-shot, respectively. On the other hand, the results of our model are
promising. Quantitatively, results of 72.76% and 87.67% are achieved for 5-way
1-shot, and 5-way 5-shot, respectively. Qualitatively, the feature space is
considerably improved. In Fig. 6e, tight clusters are observed for all
individual classes, while the extra-class distance increases considerably.
Finally in the Cars dataset (Fig. 7d)we can observe a similar issue as with
the CUB dataset. Despite having classes with very similar characteristics,
this problem can be solved with models such as the one we propose in this
paper. Maximizing the distance that exists between classes and reducing the
intra-class distance, which allows us to discriminate even though the test
images have highly similar distributions.

Overall, for each of the datasets , we have observed that our proposal
outperforms both ProtoNet and K-tuplets using the two tested backbones, both
quantitatively and qualitatively. The models trained using 5way-5shot present
the best results for each of the datasets. This is because our models have
more samples to learn during the FSL training process, yielding better
representations. However, the 5way-1shot results are not entirely negligible.
On the contrary, they extract features relevant for classification and are
close to those obtained in 5way-5shot without the need to use a larger shot.
In applications with few samples, where the number of images is limited, the
use of techniques such as 5way-1shot for FSL could provide excellent results
close to 5way-5shot.

##  8 Conclusion

In this paper, we proposed two different loss functions to train an embedding
network for few-shot image classification. These loss functions are based on
the concepts of intra-class and inter-class distance, and have as the main
objective to pull together instances of the same class and push further away
the instances of different classes. The proposed model improves the accuracy
performance when compared with baseline models which make use of metric
learning approaches to solve the few-shot classification problem on the public
benchmark dataset. The proposed model also obtains competitive results when
compared with more recent methods which make use of a more robust embedding
network, as we improve the accuracy for the 5-way 5-shot setting. Our current
framework can be extended in several ways. For instance, we could make the
hyper-parameters of the ICNN algorithm to be learnable instead of making them
of a fixed value. Another direction is to test these loss functions with
different metric meta-learning methods to see if they are allowing the network
to learn better feature representations for few-shot tasks.

## Acknowledgments

The authors wish to thank the AI Hub and the Centro de Innovación de Internet
de las Cosas at Tecnológico de Monterrey for their support for carrying the
experiments in this paper in their NVIDIA’s DGX computer.

This work has been supported by Azure Sponsorship credits granted by
Microsoft’s AI for Good Research Lab through the AI for Health program.

## References

  * Andrychowicz et al. [2016] Andrychowicz, M., Denil, M., Colmenarejo, S.G., Hoffman, M.W., Pfau, D., Schaul, T., de Freitas, N., 2016.  Learning to learn by gradient descent by gradient descent.  CoRR abs/1606.04474.  URL: http://arxiv.org/abs/1606.04474, arXiv:1606.04474.
  * Annarumma and Montana [2017] Annarumma, M., Montana, G., 2017\.  Deep metric learning for multi-labelled radiographs.  arXiv:1712.07682.
  * Benajiba et al. [2018] Benajiba, Y., Sun, J., Zhang, Y., Jiang, L., Weng, Z., Biran, O., 2018\.  Siamese networks for semantic pattern similarity.  CoRR abs/1812.06604.  URL: http://arxiv.org/abs/1812.06604, arXiv:1812.06604.
  * Bendre et al. [2020] Bendre, N., Terashima-Marín, H., Najafirad, P., 2020.  Learning from few samples: A survey.  CoRR abs/2007.15484.  URL: https://arxiv.org/abs/2007.15484, arXiv:2007.15484.
  * Bromley et al. [1993] Bromley, J., Bentz, J., Bottou, L., Guyon, I., Lecun, Y., Moore, C., Sackinger, E., Shah, R., 1993\.  Signature verification using a "siamese" time delay neural network.  International Journal of Pattern Recognition and Artificial Intelligence 7, 25.  doi:10.1142/S0218001493000339.
  * Chen et al. [2019a] Chen, D., Chen, Y., Li, Y., Mao, F., He, Y., Xue, H., 2019a.  Self-supervised learning for few-shot image classification.  CoRR abs/1911.06045.  URL: http://arxiv.org/abs/1911.06045, arXiv:1911.06045.
  * Chen et al. [2020] Chen, H., Li, H., Li, Y., Chen, C., 2020.  Multi-scale adaptive task attention network for few-shot learning.  arXiv:2011.14479.
  * Chen et al. [2019b] Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C.F., Huang, J.B., 2019b.  A closer look at few-shot classification.  arXiv:1904.04232.
  * Chen et al. [2017] Chen, Y., Hoffman, M.W., Colmenarejo, S.G., Denil, M., Lillicrap, T.P., Botvinick, M., Freitas, N., 2017.  Learning to learn without gradient descent by gradient descent, in: International Conference on Machine Learning, PMLR. pp. 748--756.
  * Dai et al. [2017] Dai, G., Xie, J., Zhu, F., Fang, Y., 2017.  Deep correlated metric learning for sketch-based 3d shape retrieval, in: Thirty-First AAAI Conference on Artificial Intelligence.
  * Finn et al. [2017] Finn, C., Abbeel, P., Levine, S., 2017.  Model-agnostic meta-learning for fast adaptation of deep networks.  CoRR abs/1703.03400.  URL: http://arxiv.org/abs/1703.03400, arXiv:1703.03400.
  * Garcia-Ramirez et al. [2022] Garcia-Ramirez, I.A., Calderon-Mora, A., Mendez-Vazquez, A., Ortega-Cisneros, S., Reyes-Amezcua, I., 2022.  A novel framework for fast feature selection based on multi-stage correlation measures.  Machine Learning and Knowledge Extraction 4, 131--149.  URL: https://www.mdpi.com/2504-4990/4/1/7, doi:10.3390/make4010007.
  * García Ramírez [2021] García Ramírez, I., 2021.  Supervised feature space reduction for multi-label classification.  PhD dissertation. CINVESTAV Unidad Guadalajara.
  * Griffin et al. [2007] Griffin, G., Holub, A., Perona, P., 2007.  Caltech-256 object category dataset .
  * Hu et al. [2014] Hu, J., Lu, J., Tan, Y.P., 2014.  Discriminative deep metric learning for face verification in the wild, in: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1875--1882.  doi:10.1109/CVPR.2014.242.
  * Huang et al. [2018] Huang, W., Ding, H., Chen, G., 2018.  A novel deep multi-channel residual networks-based metric learning method for moving human localization in video surveillance.  Signal Processing 142, 104--113.  URL: https://www.sciencedirect.com/science/article/pii/S016516841730261X, doi:https://doi.org/10.1016/j.sigpro.2017.07.015.
  * James et al. [2013] James, G., Witten, D., Hastie, T., Tibshirani, R., 2013\.  An introduction to statistical learning. volume 112\.  Springer.
  * Kawaguchi et al. [2017] Kawaguchi, K., Kaelbling, L.P., Bengio, Y., 2017.  Generalization in deep learning.  arXiv preprint arXiv:1710.05468 .
  * Koch et al. [2015] Koch, G., Zemel, R., Salakhutdinov, R., 2015.  Siamese neural networks for one-shot image recognition.
  * Krause et al. [2013] Krause, J., Stark, M., Deng, J., Fei-Fei, L., 2013\.  3d object representations for fine-grained categorization, in: 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia.
  * Lee et al. [2019] Lee, K., Maji, S., Ravichandran, A., Soatto, S., 2019\.  Meta-learning with differentiable convex optimization.  CoRR abs/1904.03758.  URL: http://arxiv.org/abs/1904.03758, arXiv:1904.03758.
  * Li et al. [2019a] Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X., 2019a.  Finding task-relevant features for few-shot learning by category traversal.  CoRR abs/1905.11116.  URL: http://arxiv.org/abs/1905.11116, arXiv:1905.11116.
  * Li and Malik [2016] Li, K., Malik, J., 2016.  Learning to optimize.  CoRR abs/1606.01885.  URL: http://arxiv.org/abs/1606.01885, arXiv:1606.01885.
  * Li et al. [2019b] Li, W., Wang, L., Xu, J., Huo, J., Gao, Y., Luo, J., 2019b.  Revisiting local descriptor based image-to-class measure for few-shot learning.  arXiv:1903.12290.
  * Li et al. [2019c] Li, X., Yu, L., Fu, C., Fang, M., Heng, P., 2019c.  Revisiting metric learning for few-shot image classification.  CoRR abs/1907.03123.  URL: http://arxiv.org/abs/1907.03123, arXiv:1907.03123.
  * Li et al. [2017] Li, Z., Zhou, F., Chen, F., Li, H., 2017.  Meta-sgd: Learning to learn quickly for few shot learning.  CoRR abs/1707.09835.  URL: http://arxiv.org/abs/1707.09835, arXiv:1707.09835.
  * Mishra et al. [2018] Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P., 2018\.  A simple neural attentive meta-learner.  arXiv:1707.03141.
  * Mueller and Thyagarajan [2016] Mueller, J., Thyagarajan, A., 2016\.  Siamese recurrent architectures for learning sentence similarity, in: AAAI.
  * Nichol et al. [2018] Nichol, A., Achiam, J., Schulman, J., 2018.  On first-order meta-learning algorithms.  arXiv:1803.02999.
  * Oreshkin et al. [2019] Oreshkin, B.N., Rodriguez, P., Lacoste, A., 2019.  Tadam: Task dependent adaptive metric for improved few-shot learning.  arXiv:1805.10123.
  * Piccialli et al. [2021] Piccialli, F., Di Somma, V., Giampaolo, F., Cuomo, S., Fortino, G., 2021.  A survey on deep learning in medicine: Why, how and when?  Information Fusion 66, 111--137.
  * Ravi and Larochelle [2017] Ravi, S., Larochelle, H., 2017\.  Optimization as a model for few-shot learning, in: ICLR.
  * Russakovsky et al. [2014] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Li, F., 2014.  Imagenet large scale visual recognition challenge.  CoRR abs/1409.0575.  URL: http://arxiv.org/abs/1409.0575, arXiv:1409.0575.
  * Rusu et al. [2018] Rusu, A.A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., Hadsell, R., 2018.  Meta-learning with latent embedding optimization.  CoRR abs/1807.05960.  URL: http://arxiv.org/abs/1807.05960, arXiv:1807.05960.
  * Schroff et al. [2015] Schroff, F., Kalenichenko, D., Philbin, J., 2015.  Facenet: A unified embedding for face recognition and clustering, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815--823.  doi:10.1109/CVPR.2015.7298682.
  * Snell et al. [2017] Snell, J., Swersky, K., Zemel, R., 2017.  Prototypical networks for few-shot learning, in: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (Eds.), Advances in Neural Information Processing Systems 30. Curran Associates, Inc., pp. 4077--4087.  URL: http://papers.nips.cc/paper/6996-prototypical-networks-for-few-shot-learning.pdf.
  * Song et al. [2022] Song, Y., Wang, T., Mondal, S.K., Sahoo, J.P., 2022\.  A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities.  arXiv preprint arXiv:2205.06743 .
  * Sung et al. [2017] Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M., 2017.  Learning to compare: Relation network for few-shot learning.  CoRR abs/1711.06025.  URL: http://arxiv.org/abs/1711.06025, arXiv:1711.06025.
  * Taigman et al. [2014] Taigman, Y., Yang, M., Ranzato, M., Wolf, L., 2014\.  Deepface: Closing the gap to human-level performance in face verification, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1701--1708.
  * Torfi et al. [2020] Torfi, A., Shirvani, R.A., Keneshloo, Y., Tavaf, N., Fox, E.A., 2020.  Natural language processing advancements by deep learning: A survey.  arXiv preprint arXiv:2003.01200 .
  * Vinyals et al. [2016] Vinyals, O., Blundell, C., Lillicrap, T., kavukcuoglu, k., Wierstra, D., 2016.  Matching networks for one shot learning, in: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (Eds.), Advances in Neural Information Processing Systems 29. Curran Associates, Inc., pp. 3630--3638.  URL: http://papers.nips.cc/paper/6385-matching-networks-for-one-shot-learning.pdf.
  * Wang et al. [2014] Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., Wu, Y., 2014.  Learning fine-grained image similarity with deep ranking, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1386--1393.
  * Xiao et al. [2017] Xiao, T., Li, S., Wang, B., Lin, L., Wang, X., 2017\.  Joint detection and identification feature learning for person search, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3415--3424.
  * Xing et al. [2003] Xing, E., Jordan, M., Russell, S.J., Ng, A., 2003\.  Distance metric learning with application to clustering with side-information, in: Becker, S., Thrun, S., Obermayer, K. (Eds.), Advances in Neural Information Processing Systems, MIT Press.  URL: https://proceedings.neurips.cc/paper/2002/file/c3e4035af2a1cde9f21e1ae1951ac80b-Paper.pdf.
  * Zheng et al. [2019] Zheng, Y., Wang, R., Yang, J., Xue, L., Hu, M., 2019\.  Principal characteristic networks for few-shot learning.  Journal of Visual Communication and Image Representation 59, 563 -- 573.  URL: http://www.sciencedirect.com/science/article/pii/S1047320319300574, doi:https://doi.org/10.1016/j.jvcir.2019.02.006.

##  Appendix A Appendix

###  A.1 Proof Proposition 1

In order to prove the previous hypothesis (Propositions 1), the loss function
ICNN (17) for $X$111In the case of $\left|X\right|=2$, we have that the second
term becomes the soft approximation of the rectifier relu function.:

$ICNNLoss\left(X\right)=-\log\left[\frac{1}{\left|X\right|}\sum_{x_{i}\in
X}\lambda\left(x_{i}\right)^{\frac{1}{p}}\omega\left(x_{i}\right)^{\frac{1}{q}}\gamma\left(x_{i}\right)^{\frac{1}{r}}\right]$

Thus, without losing generality at each step of the gradient descent, there
are lower bounds for $\omega$ and $\gamma$,
$\sigma_{\omega}\leq\omega\left(x_{i}\right)^{\frac{1}{q}}$ and
$\sigma_{\gamma}\leq\gamma\left(x_{i}\right)^{\frac{1}{r}}$. Thus, we can
obtain the following inequality,

$\displaystyle-ICNNLoss\left(X\right)\geq$
$\displaystyle\log\left[\tau\sum_{x_{i}\in
X}\lambda\left(x_{i}\right)^{\frac{1}{p}}\right]$

with

$\tau=\frac{\sigma_{\omega}\times\sigma_{\gamma}}{\left|X\right|},$

we have that

$ICNNLoss\left(X\right)\leq-\log\left[\sum_{x_{i}\in
X}\tau\lambda\left(x_{i}\right)^{\frac{1}{p}}\right]$

Thus, the minimization of $-\log\left[\sum_{x_{i}\in
X}\lambda\left(x_{i}\right)^{\frac{1}{p}}\right]$ implies the maximization of
$\sum_{x_{i}\in X}\lambda\left(x_{i}\right)^{\frac{1}{p}}$. Now, it is known
that under the gradient descent, and sample $\theta_{t+1}$ generated from
$\theta_{t}$, the loss function $L$:

$L\left(\theta_{t+1}\right)\leq L\left(\theta_{t}\right).$

Then, at each time $t$, $-\log\left[\sum_{x_{i}\in
X}\lambda\left(x_{i}|\theta_{t}\right)^{\frac{1}{p}}\right]$ is smaller than
the previous iteration. Therefore, intra-class is minimized and inter-class is
maximized QED.

###  A.2 Proof of Proposition 2

(Propositions 2) Given the fact when $ICNNLoss\left(X\right)$ is minimized,
all building elements are maximized. In particular, in the case of

$\omega\left(x_{i}\right)=\alpha-\left[Var\left(\lambda_{\tilde{x}}\left(x_{i}\right)\right)+Var\left(\lambda_{x}\left(x_{i}\right)\right)\right],$

the variances $Var\left(\lambda_{\tilde{x}}\left(x_{i}\right)\right)$ and
$Var\left(\lambda_{x}\left(x_{i}\right)\right)$ are bounded by the
$\alpha=k_{1}^{2}+k_{2}^{2}$ term. Thus, using a similar setup than the
previous proof (Proposition A.1) we can say that at each step $t$ of the
gradient descent the
$Var\left(\lambda_{\tilde{x}}\left(x_{i}\right)\right)+Var\left(\lambda_{x}\left(x_{i}\right)\right)$
is minimized thus regularizing the intra-class is minimized and inter-class
distances QED.