section [1.5cm] 2.5em

# Select-ProtoNet: Learning to Select
for Few-Shot Disease Subtype Prediction

Ziyi Yang
Macau University of Science and Technology
<EMAIL_ADDRESS>
&Jun Shu
Xi’an Jiaotong University
<EMAIL_ADDRESS>
&Yong Liang
Macau University of Science and Technology
<EMAIL_ADDRESS>
&Deyu Meng
Xi’an Jiaotong University
<EMAIL_ADDRESS>
&Zongben Xu
Xi’an Jiaotong University
<EMAIL_ADDRESS>

###### Abstract

Current machine learning has made great progress on computer vision and many
other fields attributed to the large amount of high-quality training samples,
while it does not work very well on genomic data analysis, since they are
notoriously known as small data. In our work, we focus on few-shot disease
subtype prediction problem, identifying subgroups of similar patients that can
guide treatment decisions for a specific individual through training on small
data. In fact, doctors and clinicians always address this problem by studying
several interrelated clinical variables simultaneously. We attempt to simulate
such clinical perspective, and introduce meta learning techniques to develop a
new model, which can extract the common experience or knowledge from
interrelated clinical tasks and transfer it to help address new tasks. Our new
model is built upon a carefully designed meta-learner, called Prototypical
Network [35], that is a simple yet effective meta learning machine for few-
shot image classification. Observing that gene expression data have
specifically high dimensionality and high noise properties compared with image
data, we proposed a new extension of it by appending two modules to address
these issues. Concretely, we append a feature selection layer to automatically
filter out the disease-irrelated genes and incorporate a sample reweighting
strategy to adaptively remove noisy data, and meanwhile the extended model is
capable of learning from a limited number of training examples and generalize
well. Simulations and real gene expression data experiments substantiate the
superiority of the proposed method for predicting the subtypes of disease and
identifying potential disease-related genes.

##  1 Introduction

Disease subtype prediction is to identify subgroups of similar patients that
can guide treatment decisions for a specific individual [29, 40]. For
instance, in the past 15 years, five subtypes of breast cancer have been
identified and intensively studied [36]. At the level of molecular biology,
the use of gene expression data to predict disease subtypes is of great
significance for improving the accuracy of disease diagnosis and identifying
potential disease-related genes. However, one of the challenging problem is
that the gene expression data are notoriously known as small data [33], i.e.,
we only have a relatively small number of samples for each disease subtype.
The small data learning or few-shot learning has recently attracted many
researches in machine learning community [33, 35]. This inspires us to bring
the state-of-the-art algorithms of few-shot learning to genomic data analysis.

In our paper, we attempt to deal with few-shot disease subtype prediction
problem. The most used method in genomic data is to augment sample size by
aggregating data from multiple studies under comparable conditions or
treatments [17, 10, 46]. This strategy commonly encounters bottlenecks due to
the complex properties of gene expression data, in which the aggregation of
data from different platforms or experiments inevitably suffers from batch
effects, heterogeneity, and other sources of bias [14]. Besides, we observe
that the current methods usually consider only the subtype prediction task for
a particular disease, without taking into account several clinical variables
simultaneously that are often of concern to both doctors and clinicians. To
simulate the process of doctors and clinicians studying disease subtype
prediction, we introduce the meta learning techniques to develop a new data
efficient model, which can extract the shared experience or knowledge from a
series of related tasks, and rapidly transfer it to the new tasks [5, 34, 9].
Therefore, the basic idea of the proposed new model is to learn from
interrelated clinical tasks through meta learning techniques to extract the
valuable information to help model generalize to the disease subtype
prediction task well.

Prototypical Network (ProtoNet) [35] is a carefully designed meta-learner that
is a simple and yet effective meta learning machine for few-shot image
classification. It tries to learn a metric space in which classification can
be performed by computing distances to prototype representations of each
class. However, unlike images, gene expression data are much harder to analyze
due to their high-dimensional and high-noise properties. The curse of
dimensionality problems tends to make predictions become more challenging
since a large number of redundant features involving in decision. Besides,
existing evidence suggests that a high level of technical or biological noise
inevitably exists in gene expression data [22, 31], which tends to easily
occur overfitting issue and lead to poor generalization performance. To
address these issues, we propose Select-ProtoNet, a new extension of ProtoNet
by appending two modules, feature selection layer and sample selection net,
making it filter out the disease-irrelated genes and remove noisy data towards
few-shot disease subtype prediction.

Our proposed method is built on top of ProtoNet, and we additionally append a
feature selection layer to automatically cherry-pick the disease-related
genes, and incorporate a sample reweighting strategy to adaptively suppress
the negative influence of noisy data. With these two modules, our method can
perform robust to high-dimensional and high-noise gene expression data, and
allows for better generalization on small data benefited from ProtoNet.

Figure 1: _(Left)_ (a) Diagram of the Select-ProtoNet structure. The weighted
prototype $\mathbf{c}_{n}$ is calculated by weighted representation of all the
embedded support samples for class $n$. The blue boxes represent the newly
appending modules to ProtoNet. _(Right)_ Qualitative example of how Select-
ProtoNet works. Assume a query sample $x$ belongs to class $i$. (b) The
prototypes $\mathbf{c}_{n}$ are calculated by the mean of embedded support
samples for each class. The closet prototype to the query sample $x$ is
$\mathbf{c}_{j}$. (c) The prototypes $\mathbf{c}_{n}$ are calculated by
weighted representation of all the embedded support samples for each class.
The closet prototype to the query sample is now the one of the class $i$.

The key contributions of this paper can be summarized as follows:

  * •

We propose a new extensions of ProtoNet, called Select-ProtoNet, for few-shot
disease subtype prediction, as shown in Fig. 1(a). To our best knowledge, this
should be the first work to use meta-learning techniques for Bioinformatics.
By additionally appending two modules, feature selection layer and sample
selection net, our model can adaptively select important features and clean
samples, as well as learn from small data and generalize well.

  * •

The additionally appending modules and the vanilla ProtoNet are updated in a
unified framework, which can be easily implemented on the basis of the
ProtoNet.

  * •

To address the absence of benchmark datasets for testing few-shot
classification in the field of genomics, we devise a new dataset, _mini_ TCGA
Meta-Dataset, which is derived from TCGA Meta-Dataset [28]. This dataset can
be used as a benchmark for meta learning methods in genomics applications
under few-shot learning setting.

  * •

We experimentally show the superiority of the proposed method for predicting
the subtypes of disease and identifying potential disease-related genes.

The remaining sections are organized as follows. Section 2 reviews the related
work. Section 3 detailedly introduces the proposed method. Section 4
demonstrates simulation and real data experimental results and the conclusion
is finally made.

##  2 Related Work

Data Integration. Genomics data integration is usually used method to deal
with small sample problem [13]. It can be roughly performed through two
different strategies: (1) ‘meta-analysis’ [27, 26, 42] analyzes each dataset
independently and finally combines the statistic results to identify disease-
related genes; (2) ‘integrative analysis via data merging’ [32, 16, 10] first
aggregates samples from different datasets into a unified large dataset and
then analyzes the new integrated dataset. However, integrating data from
different experiments or platforms for integrative analysis easily suffers
from batch effects and heterogeneity, which is still very challenging in
computational biology.

Few-shot Learning. Meta-learning has a prominent history in machine learning
[1, 30, 37], and many meta learning methods have been applied to address the
small data learning or few-shot learning [34]. They can be roughly divided
into two main types: (1) gradient-based optimization methods [5, 6, 23, 15]
learn a meta-learner in the outer loop to initialize a base-learner for the
inner loop that is then trained on a novel few-shot task; (2) metric-based
methods [39, 35, 24, 41, 43] learn a metric space of sample features in which
classification can be efficient with few samples. Most current methods rely
solely on image classification tasks. There is a lack of systematic researches
about using meta-learning for few-shot disease subtype prediction problems. In
this work, we extend a well-known meta-learner, Prototypical Networks [35], to
address biological problems. Our model appends two modules that performs
robust to high-dimensional and high-noise gene expression data.

Learning with High-Dimensional Features. Feature extraction and feature
selection are two commonly used manners to address the curse of dimensionality
problems. Feature extraction transforms the original features into a set of
new features through subspace learning [21]. However, a reasonable biological
interpretation is difficult to obtain from the learned feature subspace.
Feature selection removes irrelevant, redundant data, and selects a set of
important features that are significantly related to the objective [14].
Regularization technique is a typical feature selection method [38, 18, 44,
49]. However, they usually have assumptions about the prior distribution of
the data [45] and inevitably involve hyper-parameters (i.e., regularization
parameters) required to be tuned by cross-validation.

Sample Reweighting for Noisy Samples. The main idea of sample reweighting
strategy is to impose weights on samples based on their reliability for
training. Typically methods include self-paced learning (SPL) [12] and its
variations [11, 47], iterative reweighting [3, 48], FWL [4], L2RW [25], and
Meta-Weight-Net (MW-Net) [33]. They incline to suppress the effects of samples
with extremely large loss values, possibly with high noisy samples.

##  3 Proposed Select-ProtoNet for Few-Shot Disease Subtype Prediction.

In this section, we first present preliminaries in the meta learning setting
with independent episodic tasks and the prototypical networks (ProtoNet)
method for few-shot learning, then give an overview of our approach, finally
elaborate the details of our method.

###  3.1 Meta-Learning for Few-Shot Learning

Few-shot learning (FSL) involves building a model using available training
data of seen classes that can classify unseen novel classes using only few
examples. Under the FSL setting, we have a large sample set $\mathcal{D}_{s}$
from a set of source classes $\mathcal{C}_{s}$, while a few-shot sample set
$\mathcal{D}_{t}$ from a set of target classes $\mathcal{C}_{t}$, and a test
set $\mathcal{T}$ from $\mathcal{C}_{t}$, where
$\mathcal{C}_{s}\cap\mathcal{C}_{t}=\emptyset$. The goal of the FSL is to
training a classification model with $\mathcal{C}_{s}$ that can generalize
well to $\mathcal{T}$.

We then define the episodic training strategy widely used by existing meta-
learning based FSL models [39, 35]. Concretely, we can define a series of
$n$-way $k$-shot tasks randomly sampled from $\mathcal{D}_{s}$, and each
$n$-way $k$-shot task is defined as an episode $D=(S,Q)$. Generally, we call
$S$ the support set containing $n$ classes and $k$ samples per class, and $Q$
the query set with the same $n$ classes. In our paper, we try to construct a
new meta dataset of FSL for disease subtype prediction in biological field.
The episode $D$ can be constructed by the following process: we first select a
small set of source class containing $n$ classes
$\mathcal{C}=\\{\mathcal{C}_{i}|i=1,2,\cdots,n_{c}\\}$ from $\mathcal{C}_{s}$,
and then generate $S$ and $Q$ by randomly sampling $k$ support samples and $q$
query samples from each class in $C$, respectively. Therefore, we have
$S=\\{(x_{i},y_{i})|y_{i}\in C,i=1,2,\cdots,m_{s}\\}$, and
$Q=\\{(x_{i},y_{i})|y_{i}\in C,i=1,2,\cdots,m_{q}\\}$, where $m_{s}=n\times
k$, $m_{q}$ is the number of samples for query set in each episodic, and
$S\cap Q=\emptyset$.

We denote the classifier we want to learn as $p_{\phi}(y|x)$, with parameter
$\phi$, which outputs a probability of a data point belonging to the class $y$
given the data sample $x$. The classifier is trained between its predicted
labels and the ground truth labels over the query set $Q$ with different
episodes such that it can generalize to other datasets, the objective function
is written as follows:

$\phi=\arg\min_{\phi}\mathbb{E}_{\mathcal{D}\subset\mathcal{D}_{s}}\sum_{(x,y)\in
Q}-\log p_{\phi}(y|x,S),$ (1)

where different forms of $p_{\phi}(y|x,S)$ define different kinds of meta
learning algorithm.

###  3.2 Review of Prototypical Networks

Here, we will give a introduction to the prototypical networks (ProtoNet),
which is a well-known meta-learning algorithm, and our model is built on top
of it. ProtoNet learns a prototype of each class in the support set $S$ and
classifies each sample in the query set $Q$ based on its distances to
different prototypes. Thus $p_{\phi}(y|x,S)$ in ProtoNet is defined as
follows:

$\displaystyle
p_{\phi}(y|x,S)=\frac{\exp\left(-d\left(g_{\phi}(x),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(x),\mathbf{c}_{n^{\prime}}\right)\right)},$
(2)

where $d$ represents the Euclidean distance metric in the feature space, and
$\mathbf{c}_{n}$ is prototype for every episode. Eq.2 means that ProtoNet
produces the class distributions of a query sample $x$ based on the softmax
output w.r.t. the distance between $g_{\phi}(x)$ and the class prototype
$\mathbf{c}_{n}$, where $g_{\phi}(x)$ is the embedding function which maps a
sample $x$ to the feature vector $g_{\phi}(x)$. Each prototype
$\mathbf{c}_{n}$ is calculated by averaging the vectors of all the embedded
support samples $\mathcal{S}_{n}$ belonging to the class $n$, given by the
following form:

$\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}g_{\phi}\left(x_{i}\right),$
(3)

where $\mathcal{S}_{n}\in S$ denotes the set of support samples that belong to
the class $n$.

###  3.3 Learning to Select Important Features and Clean Data for Disease
Subtype Prediction

We attempt to introduce ProtoNet to disease subtype prediction problem to
enhance the capability of few-shot recognition. However, the gene expression
data is harder to deal with compared with image dataset, since for its high
dimensionality and high noise properties. To further overcome these issues, we
proposed to append two modules (as shown in Fig.1 (a)), i.e., a feature
selection layer (FS-Layer) to automatically filtering out the disease-
irrelated genes and incorporate an adaptive sample reweighting strategy (MW-
NetV2) against data noise, building up on the ProtoNet.

####  3.3.1 Feature Selection Layer

For each sample $x\in\mathbb{R}^{p}$ of gene expression data, the feature
dimension $p$ is high. Generally, feature selection methods try to find a
selection vector $\beta=(\beta_{1},\beta_{2},\cdots,\beta_{p})$, which is
element-wise multiplication with data $x$, to filter out the useless feature
and obtain a new representation of data $x_{new}$ to help the following tasks
perform well, i.e.,

$\displaystyle x_{new}=\beta\odot x,\beta_{j}\in[0,1].$ (4)

The regularization technique, the effective method on this problem, often
manually set a specific form of regularization under a certain assumption on
training data, which is infeasible when we know little knowledge underlying
gene expression data. To overcome this, we model the selection vector as a
Softmax network layer as follows, which can learn an adaptive feature
weighting vector from data,

$\displaystyle x_{new}=\beta(\theta)\odot x\triangleq
f_{\theta}(x),\beta_{i}(\theta)=\frac{\exp(\theta)_{i}}{\sum_{j}\exp(\theta)_{j}},\sum_{i}\beta_{i}(\theta)=1,$
(5)

where $\theta\in\mathbb{R}^{p}$ is the parameter vector of Softmax layer, and
$\exp$ is the element-wise exponential operator. It can be be easily embedded
into Eq.(2),

$\displaystyle
p_{\phi,\theta}(y|x,S)=\frac{\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n^{\prime}}\right)\right)},$
(6)

where $\mathbf{c}_{n}$ can be rewritten as:

$\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}g_{\phi}\left(f_{\theta}(x_{i})\right),$
(7)

This formulation needs no expert-level knowledge to character $\beta$, and
$\theta$ can efficiently learned by automatic differentiation techniques as
$\phi$ does, which makes it easy to scale to other problems.

Algorithm 1 Training episode loss computation for Select-ProtoNet. $N$ is the
number of examples in the training set, $K$ is the number of classes in the
training set, $n_{c}\leq K$ is the number of classes per episode, $k$ is the
number of support examples per class, $q$ is the number of query examples per
class. RandomSample($S$,$n$) denotes a set of $n$ elements chosen uniformly at
random from set $S$, without replacement. _This algorithm is built upon
ProtoNet[35], and the revised parts are shown in blue._

0: Training set $\mathcal{D}=\\{(x_{1},y_{1}),\ldots,(x_{N},y_{N})\\}$, where
each $y_{i}\in\\{1,\ldots,K\\}$. $\mathcal{D}_{k}$ denotes the subset of
$\mathcal{D}$ containing all elements $(x_{i},y_{i})$ such that $y_{i}=n$.

0: The loss $\mathcal{J}(\phi,\theta,\Theta)$ for a randomly generated
training episode.

1: $V\leftarrow\textsc{RandomSample}(\\{1,\dots,K\\},n_{c})$ $\triangleright$
Select class indices for episode

2: for $n$ in {$1,\dots,n_{c}$} do

3: $\mathcal{S}_{n}\leftarrow\textsc{RandomSample}(\mathcal{D}_{V_{n}},k)$
$\triangleright$ Select support examples

4:
$\mathcal{Q}_{n}\leftarrow\textsc{RandomSample}(\mathcal{D}_{V_{n}}\setminus\mathcal{S}_{n},q)$
$\triangleright$ Select query examples

5:
$\mathbf{c}_{n}\leftarrow\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot
g_{\phi}\left(f_{\theta}(x_{i})\right)$ $\triangleright$ Compute the weighted
prototype

6: end for

7: $\mathcal{J}(\phi,\theta,\Theta)\leftarrow 0$ $\triangleright$ Initialize
loss

8: for $n$ in {$1,\dots,n_{c}$} do

9: for $(x,y)$ in $\mathcal{Q}_{n}$ do

10:
$\mathcal{J}(\phi,\theta,\Theta)\leftarrow\mathcal{J}(\phi,\theta,\Theta)+\frac{1}{n_{c}\cdot
q}\left[d\left(g_{\phi}\left(f_{\theta}(x)\right),\mathbf{c}_{n}\right)+\log\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}\left(f_{\theta}(x)\right),\mathbf{c}_{n^{\prime}}\right)\right)\right]$
$\triangleright$ Update loss

11: end for

12: end for

####  3.3.2 Sample Selection Net

There exist high noise in gene expression data, which can easily lead to poor
performance in generalization. To overcome this, instead of the simple average
strategy in ProtoNet, we attempt to assign weights to support samples to
character the clean confidence of data, expecting to suppress the effects of
samples with extremely noise. Specifically, the final prototype
$\mathbf{c}_{n}$ should be calculated by weighted representation of all the
embedded support samples for class $n$, i.e.,

$\displaystyle\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}v_{i}\cdot
g_{\phi}\left(f_{\theta}(x_{i})\right),v_{i}\in[0,1],$ (8)

where $v_{i}$ reflect the confidence of the support sample $x_{i}$ belonging
to clean data. In general, large weights $v$ are more likely to be high-
confident ones with clean data.

To determine the $v$, inspired by current adaptive sample weighting strategy
Meta-Weight-Net [33], we attempt to learn a weighting function to distinguish
clean and noisy data. Specifically, MW-Net models the sample weights $v$ as an
MLP network $\mathcal{V}(\ell;\Theta)$ with only one hidden layer, which is a
universal approximator for almost any continuous function and thus can fit a
wide range of weighting functions. Its input is the loss function of the
sample, and output is the weight to this sample. Since ProtoNet does not
compute the losses of the support samples, we use the embedding representation
$g_{\phi}\left(f_{\theta}(x)\right)$ as input of MW-Net. Therefore, the Eq.(8)
function can be furtherly rewritten as:

$\displaystyle\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot
g_{\phi}\left(f_{\theta}(x_{i})\right),$ (9)

where $\Theta$ represents the parameters of MW-NetV2 (to distinction with the
original MW-Net), and the architecture of MW-NetV2 is same with MW-Net, except
for the input. The input of MW-NetV2 is the embedding representation for each
sample instead of the loss in MW-Net.

####  3.3.3 Learning Algorithm

By additionally appending feature selection layer (FS-Net) and sample
selection net (MW-NetV2) to vanilla ProtoNet, the proposed method, Select-
ProtoNet, can simultaneously cherry-pick the disease-related genes helping
classification and suppress the negative influence of noisy data. The learning
procedure is similar to vanilla ProtoNet, and the parameters of FS-Net and MW-
NetV2 are simultaneously updated with the parameters of ProtoNet. The final
loss function is:

$\displaystyle\phi,\theta,\Theta=\arg\min_{\phi,\theta,\Theta}\mathbb{E}_{\mathcal{D}\subset\mathcal{D}_{s}}$
$\displaystyle\sum_{(x,y)\in Q}-\log p_{\phi,\theta,\Theta}(y|x,S),$ (10)
$\displaystyle
p_{\phi,\theta,\Theta}(y|x,S)=\frac{\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n}\right)\right)}{\sum_{n^{\prime}}\exp\left(-d\left(g_{\phi}(f_{\theta}(x)),\mathbf{c}_{n^{\prime}}\right)\right)},$
$\displaystyle\mathbf{c}_{n}=\frac{1}{\left|\mathcal{S}_{n}\right|}\sum_{\left(x_{i},y_{i}\right)\in\mathcal{S}_{n}}\mathcal{V}(g_{\phi}(f_{\theta}(x_{i}));\Theta)\cdot
g_{\phi}\left(f_{\theta}(x_{i})\right).$ (11)

Fig. 1(right) illustrates an example on how the proposed method works. The
pseudocode for calculating the episode training loss
$\mathcal{J}(\phi,\theta,\Theta)$ is provided in Algorithm 1.

##  4 Experiments

To evaluate the capability of the proposed method, Select-ProtoNet, we conduct
experiments on simulated data sets and real gene expression datasets. We show
that Select-ProtoNet outperforms its backbone methods and several conventional
methods.

###  4.1 Simulation Experiments

Simulated Datasets. We construct two sets of data $\mathcal{D}_{train}$ and
$\mathcal{D}_{test}$ with distinct sets of classes under the FSL setting. The
generation of simulated data is refer to the work in [20]. For more detailed
information on generating simulated datasets, see Appendix A.

Noise and Feature Dimension Settings. We consider four-level settings of
corrupted class labels on the support set. The class label of each support
sample is independently changed to a random class with probability $p$, where
$p=0\%,10\%,30\%,50\%$. Moreover, we consider four-level settings of
irrelevant feature with $d$ dimension, where $d=100,500,1000,2000$.

Baselines. We compare against two family of methods. The first is metric-based
meta learning methods: ProtoNet and ProtoNet with either of the two appended
modules. ‘SelectF-ProtoNet’ and ‘SelectS-ProtoNet’ are models with feature
selection layer and sample selection net, respectively. The second is the
conventional methods, usually used to analyze gene expression data to predict
disease subtypes. The conventional comparison methods include: Support Vector
Machines (SVM), Naive Bayes, Logistic Regression, Logistic regression with
Lasso penalty (Logistic_lasso), Random Forest, NeuralNet [20], and AffinityNet
[20]. Details of baseline implementations, training and test procedures can be
found in Appendix B.

Results. Table 1 shows the accuracy of Select-ProtoNet and its backbones on
simulated datasets under different experiment settings with $30$ random runs.
It can be seen that the appended two modules both make contributions to
improve the performance, and the proposed method achieves the best results.
With the increase of noise rate, the advantages of Select-ProtoNet over
ProtoNet become more obvious. As shown in Fig. 4, the curves of training loss
and accuracy comparison between ProtoNet and Select-ProtoNet. We can observe
that ProtoNet takes almost twice the time of our model to achieve the best
classification accuracy. To illustrate the effect of two appended modules of
our model, we plot the weight distribution of clean and noisy training support
samples in Fig. 4. It can be seen that almost all lager weights belong to
clean samples, and the weight value of noisy samples is less than that of
clean samples, which implies that MW-NetV2 can distinguish clean and noisy
samples. Fig. 5(c) shows the feature weights learned by Select-ProtoNet. It
can be observed that the weights of important features are higher than those
of irrelevant features, and almost all the weights of irrelevant features are
less than a certain threshold value, indicating that the feature selection
layer can select important features.

We also compare Select-ProtoNet with state of the art conventional methods.
Fig. 4 shows the test accuracy of all competing methods on the few selected
training samples (only 1% of datasets). It can be seen that our model is
significantly superior to all other competing methods. There are big
performance gaps between our model and conventional methods (i.e., Random
Forest, NeuralNet, SVM, and Logistic Regression) when training data is small.
Particularly, NeuralNet, is the worst performer because the training pool is
quite small that the power of deep learning can only be manifested when a
large amount of data is available. In addition, we plot the weights of
features learned by NeuralNet and AffinityNet, are shown in Fig 5. We can
observe that feature selection performance of our model outperforms competing
methods. We further increase the feature dimension to evaluate the
classification accuracy of all comparison methods. The results are shown in
Table 2 with $30$ random runs. As can be seen, Select-ProtoNet can improve the
accuracy compared with ProtoNet, and outperforms all other conventional
comparison methods. With the increase of the feature dimension, the
performance gaps between the proposed method and all competing methods
increase gradually.

Table 1: Few-shot classification accuracy (%) comparison on simulated data with varying experiment settings. The best results are highlighted in bold. Noise rate | Model | 5 way 5 shot | 5 way 10 shot | 10 way 5 shot | 10 way 10 shot
---|---|---|---|---|---
0% | ProtoNet |  91.93 $\pm$ 3.08 |  92.30 $\pm$ 2.61 |  93.01 $\pm$ 1.81 |  93.40 $\pm$ 1.30
SelectS-ProtoNet |  92.74 $\pm$ 1.42 |  93.29 $\pm$ 1.39 |  93.70 $\pm$ 1.21 |  94.30 $\pm$ 1.40
SelectF-ProtoNet |  96.55 $\pm$ 1.75 |  96.78 $\pm$ 1.02 |  97.42 $\pm$ 0.48 |  97.21 $\pm$ 0.51
Select-ProtoNet | 96.69 $\pm$ 1.08 | 96.94 $\pm$ 0.98 | 97.44 $\pm$ 0.41 | 97.62 $\pm$ 0.47
10% | ProtoNet |  83.23 $\pm$ 4.90 |  82.25 $\pm$ 4.10 |  85.27 $\pm$ 3.41 |  86.38 $\pm$ 4.00
SelectS-ProtoNet |  84.30 $\pm$ 4.44 |  84.49 $\pm$ 3.73 |  88.26 $\pm$ 2.81 |  89.89 $\pm$ 2.59
SelectF-ProtoNet |  91.17 $\pm$ 3.59 |  92.29 $\pm$ 2.05 |  92.62 $\pm$ 2.24 |  92.96 $\pm$ 2.00
Select-ProtoNet | 93.56 $\pm$ 2.78 | 94.40 $\pm$ 1.34 | 94.67 $\pm$ 1.55 | 94.31 $\pm$ 1.73
30% | ProtoNet |  72.33 $\pm$ 7.42 |  73.98 $\pm$ 6.59 |  74.02 $\pm$ 6.43 |  74.52 $\pm$ 6.13
SelectS-ProtoNet |  75.74 $\pm$ 6.21 |  76.34 $\pm$ 4.92 |  77.64 $\pm$ 7.53 |  77.81 $\pm$ 6.00
SelectF-ProtoNet |  86.56 $\pm$ 4.48 |  87.07 $\pm$ 5.49 |  87.72 $\pm$ 4.53 |  86.29 $\pm$ 6.22
Select-ProtoNet | 88.95 $\pm$ 3.74 | 89.66 $\pm$ 4.02 | 89.87 $\pm$ 3.85 | 89.60 $\pm$ 4.38
50% | ProtoNet |  64.35 $\pm$ 7.99 |  65.95 $\pm$ 8.01 |  66.93 $\pm$ 7.60 |  66.84 $\pm$ 6.86
SelectS-ProtoNet |  68.69 $\pm$ 6.42 |  68.79 $\pm$ 7.99 |  70.22 $\pm$ 7.75 |  70.90 $\pm$ 6.30
SelectF-ProtoNet |  79.04 $\pm$ 9.33 |  80.85 $\pm$ 7.44 |  81.30 $\pm$ 7.08 |  81.47 $\pm$ 8.95
Select-ProtoNet | 83.42 $\pm$ 6.06 | 83.21 $\pm$ 5.59 | 84.49 $\pm$ 5.33 | 84.43 $\pm$ 5.54

Figure 2: Training loss and accuracy (%) vs. number of iterations under
unbiased data.

Figure 3: Sample weight distribution on training data under 30% noise rate.

Figure 4: Box-plot diagram of test accuracy (%) of all competing methods over
$30$ repetitions.

(a) NeuralNet

(b) AffinityNet

(c) Select-ProtoNet

Figure 5: Plots of feature weights learned by the competing methods on unbiased simulated data. The red bar represents the true features and the blue bar represents the irrelevant features. Table 2: Test accuracy (%) comparison on simulated data with varying feature dimension settings. The best results are highlighted in bold. Methods | 100 | 500 | 1000 | 2000
---|---|---|---|---
SVM |  40.79 $\pm$ 1.47 |  30.84 $\pm$ 0.82 |  29.34 $\pm$ 1.08 |  27.95 $\pm$ 0.73
Naive Bayes |  76.91 $\pm$ 5.03 |  54.19 $\pm$ 5.34 |  48.17 $\pm$ 3.07 |  40.47 $\pm$ 3.10
Random Forest |  52.23 $\pm$ 9.47 |  26.43 $\pm$ 2.48 |  26.21 $\pm$ 2.71 |  25.73 $\pm$ 1.63
Logistic Regression |  41.59 $\pm$ 1.86 |  30.83 $\pm$ 0.92 |  29.36 $\pm$ 1.07 |  27.97 $\pm$ 0.56
Logistic_lasso |  72.96 $\pm$ 4.72 |  59.24 $\pm$ 4.78 |  49.65 $\pm$ 5.94 |  42.31 $\pm$ 5.60
NeuralNet |  32.42 $\pm$ 1.50 |  27.61 $\pm$ 0.75 |  26.29 $\pm$ 0.85 |  25.92 $\pm$ 0.66
AffinityNet |  66.06 $\pm$ 25.13 |  39.17 $\pm$ 13.30 |  27.55 $\pm$ 6.98 |  34.18 $\pm$ 10.66
ProtoNet |  80.48 $\pm$ 4.68 |  71.90 $\pm$ 2.64 |  54.09 $\pm$ 10.95 |  48.66 $\pm$ 6.29
Select-ProtoNet | 96.08 $\pm$ 0.35 | 93.13 $\pm$ 2.44 | 85.40 $\pm$ 5.12 | 81.49 $\pm$ 6.10

###  4.2 Experiments on _mini_ TCGA Meta-Dataset

Real Datasets. TCGA Meta-Dataset [28] is a publicly available benchmark
dataset in the field of gene expression analysis, containing 174 clinical
tasks derived from The Cancer Genome Atlas (TCGA) [19], which can be used in a
multi-task learning framework. However, it cannot be directly used as a few-
shot learning benchmark dataset because some tasks have extremely imbalance
classes. Thus, after carefully selecting samples, we devise a new dataset,
_mini_ -TCGA Meta-Dataset, consisting of 68 TCGA benchmark clinical tasks,
where each task has two classes and each class has at least 60 samples. More
details about datasets can be found in Appendix D.

Baselines. We compare against the following state-of-art methods: ProtoNet,
the majority class prediction (Majority), Logistic Regression, and Neural
Network. The implementations of three conventional methods are the same as the
work in [28]. For details on implementations, training and test procedures,
see Appendix E.

Results. The classification accuracy of _mini_ TCGA Meta-Dataset with varying
noise rate settings are reported in Table 3. As can be seen, our model
improves on its backbone method with a large margin for all noise rate
settings, and outperforms the three conventional supervised methods.

Figure 6: Survival curves of the 20 top-ranked genes selected by Select-
ProtoNet.

Table 4 shows the lung cancer subtype prediction accuracy of all competing
methods on the TCGA Meta-Dataset with task id (‘Expression_Subtype’, ‘LUNG’).
Especially, ProtoNet and Select-ProtoNet consider the interrelationship of
clinical tasks, leveraging samples from all clinical tasks of _mini_ TCGA
Meta-Dataset to build shared experience or knowledge and transfer it to help
predict the lung cancer subtype. The conventional methods, however, only
consider the lung cancer subtype task. As shown in Table 4, we can observe
that our model achieves the best results, with an accuracy gain of more than
20% compared with its backbone, and over 29% against the best result of the
conventional methods. It implies that our model significantly improves the
prediction performance of disease subtypes by incorporating various
interrelated clinical tasks. Fig. 6 shows the Kaplan-Meier (KM) survival
curves of the 20 top-ranked significant genes selected by Select-ProtoNet on
the lung cancer subtype task of TCGA Meta-Dataset. By using bBioPortal [2, 7],
the survival analysis of significant genes is done based on the Pan-Cancer
Atlas dataset [8]. As shown in Fig. 6, the two curves without intersect
(logrank test $p$-value $=5.801e-4$). The patients without alterations in the
selected genes (blue line) have long survival times.

Table 3: Test accuracy (%) comparison on _mini_ TCGA Meta-Dataset with varying
noise rate settings. The mean accuracy ($\pm$ std) over 30 repetitions are
reported. The best results are highlighted in bold.

Noise Rate | Conventional supervised methods | Select-ProtoNet and its backbone
---|---|---
Majority | Logistic Regression | Neural Network | ProtoNet | Select-ProtoNet
0% |  64.10 $\pm$ 8.83 |  67.96 $\pm$ 12.80 |  68.30 $\pm$ 11.63 |  72.01 $\pm$ 19.91 | 84.71 $\pm$ 4.49
10% |  63.63 $\pm$ 9.38 |  65.08 $\pm$ 10.49 |  64.85 $\pm$ 9.98 |  66.69 $\pm$ 19.54 | 81.33 $\pm$ 2.32
30% |  61.98 $\pm$ 9.94 |  60.89 $\pm$ 8.00 |  61.50 $\pm$ 8.89 |  64.18 $\pm$ 19.48 | 79.89 $\pm$ 2.40
50% |  59.73 $\pm$ 9.56 |  57.06 $\pm$ 5.72 |  58.73 $\pm$ 7.59 |  60.72 $\pm$ 17.31 | 78.15 $\pm$ 2.30

Table 4: Accuracy (%) comparison on the lung cancer subtype task of TCGA Meta-
Dataset.

Method | Majority | Logistic Regression | Neural Network | ProtoNet | Select-ProtoNet
---|---|---|---|---|---
Accuracy | 38.00 $\pm$ 0.00 | 60.64 $\pm$ 8.23 | 68.20 $\pm$ 1.60 | 77.78 $\pm$ 27.22 | 97.78 $\pm$ 3.44

##  5 Conclusion

In this paper, we propose a novel meta learning method for few-shot disease
subtype prediction problem. The proposed method, carefully designs to append
two modules to the vanilla Prototypical Network, making it able to address the
high dimensionality and high noise issues in gene expression data compared
with image data. Meanwhile, it possesses the ability of Prototypical Network
to extract the shared experience or knowledge from interrelated clinical
tasks, capable of learning from a limited number of training examples and
generalize well. Synthetic and a new released benchmark dataset for disease
subtype prediction, _mini_ -TCGA Meta-Dataset experiments substantiate the
superiority of the proposed method for predicting the subtypes of disease and
identifying potential disease-related genes. In future work, we need to
further consider the interaction between genes and integrate gene regulatory
networks into meta-learning techniques and further promote the application of
meta-learning techniques in the field of few-shot biological genomics, thus
bringing us closer to the clinic of the future.

## References

  * [1] Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei.  On the optimization of a synaptic learning rule.  In Preprints Conf. Optimality in Artificial and Biological Neural Networks, volume 2. Univ. of Texas, 1992.
  * [2] Ethan Cerami, Jianjiong Gao, Ugur Dogrusoz, Benjamin E Gross, Selcuk Onur Sumer, Bülent Arman Aksoy, Anders Jacobsen, Caitlin J Byrne, Michael L Heuer, Erik Larsson, et al.  The cbio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data, 2012.
  * [3] Fernando De La Torre and Michael J Black.  A framework for robust subspace learning.  International Journal of Computer Vision, 54(1-3):117–142, 2003\.
  * [4] Mostafa Dehghani, Arash Mehrjou, Stephan Gouws, Jaap Kamps, and Bernhard Schölkopf.  Fidelity-weighted learning.  arXiv preprint arXiv:1711.02799, 2017.
  * [5] Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1126–1135. JMLR. org, 2017.
  * [6] Chelsea Finn, Kelvin Xu, and Sergey Levine.  Probabilistic model-agnostic meta-learning.  In Advances in Neural Information Processing Systems, pages 9516–9527, 2018.
  * [7] Jianjiong Gao, Bülent Arman Aksoy, Ugur Dogrusoz, Gideon Dresdner, Benjamin Gross, S Onur Sumer, Yichao Sun, Anders Jacobsen, Rileen Sinha, Erik Larsson, et al.  Integrative analysis of complex cancer genomics and clinical profiles using the cbioportal.  Sci. Signal., 6(269):pl1–pl1, 2013.
  * [8] Katherine A Hoadley, Christina Yau, Toshinori Hinoue, Denise M Wolf, Alexander J Lazar, Esther Drill, Ronglai Shen, Alison M Taylor, Andrew D Cherniack, Vésteinn Thorsson, et al.  Cell-of-origin patterns dominate the molecular classification of 10,000 tumors from 33 types of cancer.  Cell, 173(2):291–304, 2018.
  * [9] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey.  Meta-learning in neural networks: A survey.  arXiv preprint arXiv:2004.05439, 2020.
  * [10] Jacob J Hughey and Atul J Butte.  Robust meta-analysis of gene expression using the elastic net.  Nucleic acids research, 43(12):e79–e79, 2015.
  * [11] Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann.  Easy samples first: Self-paced reranking for zero-example multimedia search.  In Proceedings of the 22nd ACM international conference on Multimedia, pages 547–556, 2014.
  * [12] M Pawan Kumar, Benjamin Packer, and Daphne Koller.  Self-paced learning for latent variable models.  In Advances in Neural Information Processing Systems, pages 1189–1197, 2010.
  * [13] Cosmin Lazar, Stijn Meganck, Jonatan Taminau, David Steenhoff, Alain Coletta, Colin Molter, David Y Weiss-Solís, Robin Duque, Hugues Bersini, and Ann Nowé.  Batch effect removal methods for microarray gene expression data integration: a survey.  Briefings in bioinformatics, 14(4):469–490, 2013.
  * [14] Cosmin Lazar, Jonatan Taminau, Stijn Meganck, David Steenhoff, Alain Coletta, Colin Molter, Virginie de Schaetzen, Robin Duque, Hugues Bersini, and Ann Nowe.  A survey on filter techniques for feature selection in gene expression microarray analysis.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 9(4):1106–1119, 2012.
  * [15] Yoonho Lee and Seungjin Choi.  Gradient-based meta-learning with learned layerwise metric and subspace.  arXiv preprint arXiv:1801.05558, 2018.
  * [16] Jeffrey T Leek, Robert B Scharpf, Héctor Corrada Bravo, David Simcha, Benjamin Langmead, W Evan Johnson, Donald Geman, Keith Baggerly, and Rafael A Irizarry.  Tackling the widespread and critical impact of batch effects in high-throughput data.  Nature Reviews Genetics, 11(10):733–739, 2010.
  * [17] Quefeng Li, Sijian Wang, Chiang-Ching Huang, Menggang Yu, and Jun Shao.  Meta-analysis based variable selection for gene expression data.  Biometrics, 70(4):872–880, 2014.
  * [18] Yong Liang, Cheng Liu, Xin-Ze Luan, Kwong-Sak Leung, Tak-Ming Chan, Zong-Ben Xu, and Hai Zhang.  Sparse logistic regression with a l 1/2 penalty for gene selection in cancer classification.  BMC bioinformatics, 14(1):198, 2013.
  * [19] Jianfang Liu, Tara Lichtenberg, Katherine A Hoadley, Laila M Poisson, Alexander J Lazar, Andrew D Cherniack, Albert J Kovatich, Christopher C Benz, Douglas A Levine, Adrian V Lee, et al.  An integrated tcga pan-cancer clinical data resource to drive high-quality survival outcome analytics.  Cell, 173(2):400–416, 2018.
  * [20] Tianle Ma and Aidong Zhang.  Affinitynet: semi-supervised few-shot learning for disease type prediction.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1069–1076, 2019.
  * [21] Fotios D Mandanas and Constantine L Kotropoulos.  Subspace learning and feature selection via orthogonal mapping.  IEEE Transactions on Signal Processing, 2020.
  * [22] Jonathan M Raser and Erin K O’Shea.  Noise in gene expression: origins, consequences, and control.  Science, 309(5743):2010–2013, 2005.
  * [23] Sachin Ravi and Hugo Larochelle.  Optimization as a model for few-shot learning.  2016\.
  * [24] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel.  Meta-learning for semi-supervised few-shot classification.  arXiv preprint arXiv:1803.00676, 2018.
  * [25] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun.  Learning to reweight examples for robust deep learning.  arXiv preprint arXiv:1803.09050, 2018.
  * [26] Daniel R Rhodes and Arul M Chinnaiyan.  Integrative analysis of the cancer transcriptome.  Nature genetics, 37(6s):S31, 2005.
  * [27] Daniel R Rhodes, Jianjun Yu, K Shanker, Nandan Deshpande, Radhika Varambally, Debashis Ghosh, Terrence Barrette, Akhilesh Pandey, and Arul M Chinnaiyan.  Large-scale meta-analysis of cancer microarray data identifies common transcriptional profiles of neoplastic transformation and progression.  Proceedings of the National Academy of Sciences, 101(25):9309–9314, 2004.
  * [28] Mandana Samiei, Tobias Würfl, Tristan Deleu, Martin Weiss, Francis Dutil, Thomas Fevens, Geneviève Boucher, Sebastien Lemieux, and Joseph Paul Cohen.  The tcga meta-dataset clinical benchmark.  arXiv preprint arXiv:1910.08636, 2019.
  * [29] Suchi Saria and Anna Goldenberg.  Subtyping: What it is and its role in precision medicine.  IEEE Intelligent Systems, 30(4):70–75, 2015.
  * [30] Jürgen Schmidhuber.  Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-… hook.  PhD thesis, Technische Universität München, 1987.
  * [31] Jörn M Schmiedel, Lucas B Carey, and Ben Lehner.  Empirical mean-noise fitness landscapes reveal the fitness impact of gene expression noise.  Nature communications, 10(1):1–12, 2019.
  * [32] Andrey A Shabalin, Håkon Tjelmeland, Cheng Fan, Charles M Perou, and Andrew B Nobel.  Merging two gene-expression studies via cross-platform normalization.  Bioinformatics, 24(9):1154–1160, 2008.
  * [33] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng.  Meta-weight-net: Learning an explicit mapping for sample weighting.  arXiv preprint arXiv:1902.07379, 2019.
  * [34] Jun Shu, Zongben Xu, and Deyu Meng.  Small sample learning in big data era.  arXiv preprint arXiv:1808.04572, 2018.
  * [35] Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In Advances in Neural Information Processing Systems, pages 4077–4087, 2017.
  * [36] Bo Hwa Sohn, Jun-Eul Hwang, Hee-Jin Jang, Hyun-Sung Lee, Sang Cheul Oh, Jae-Jun Shim, Keun-Wook Lee, Eui Hyun Kim, Sun Young Yim, Sang Ho Lee, et al.  Clinical significance of four molecular subtypes of gastric cancer identified by the cancer genome atlas project.  Clinical Cancer Research, 23(15):4441–4449, 2017.
  * [37] Sebastian Thrun.  Lifelong learning algorithms.  In Learning to learn, pages 181–209. Springer, 1998.
  * [38] Robert Tibshirani.  Regression shrinkage and selection via the lasso.  Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
  * [39] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al.  Matching networks for one shot learning.  In Advances in neural information processing systems, pages 3630–3638, 2016.
  * [40] Bo Wang, Aziz M Mezlini, Feyyaz Demir, Marc Fiume, Zhuowen Tu, Michael Brudno, Benjamin Haibe-Kains, and Anna Goldenberg.  Similarity network fusion for aggregating data types on a genomic scale.  Nature methods, 11(3):333, 2014.
  * [41] Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan.  Low-shot learning from imaginary data.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7278–7286, 2018.
  * [42] Pratyaksha Wirapati, Christos Sotiriou, Susanne Kunkel, Pierre Farmer, Sylvain Pradervand, Benjamin Haibe-Kains, Christine Desmedt, Michail Ignatiadis, Thierry Sengstag, Frédéric Schütz, et al.  Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures.  Breast Cancer Research, 10(4):R65, 2008.
  * [43] Chen Xing, Negar Rostamzadeh, Boris Oreshkin, and Pedro OO Pinheiro.  Adaptive cross-modal few-shot learning.  In Advances in Neural Information Processing Systems, pages 4848–4858, 2019.
  * [44] Zongben Xu, Hai Zhang, Yao Wang, XiangYu Chang, and Yong Liang.  L 1/2 regularization.  Science China Information Sciences, 53(6):1159–1169, 2010.
  * [45] Bing Xue, Mengjie Zhang, Will N Browne, and Xin Yao.  A survey on evolutionary computation approaches to feature selection.  IEEE Transactions on Evolutionary Computation, 20(4):606–626, 2015\.
  * [46] Zi-Yi Yang, Xiao-Ying Liu, Jun Shu, Hui Zhang, Yan-Qiong Ren, Zong-Ben Xu, and Yong Liang.  Multi-view based integrative analysis of gene expression data for identifying biomarkers.  Scientific reports, 9(1):1–15, 2019.
  * [47] Zi-Yi Yang, Liang-Yong Xia, Hui Zhang, and Yong Liang.  Mspl: Multimodal self-paced learning for multi-omics feature selection and data integration.  IEEE Access, 7:170513–170524, 2019.
  * [48] Zhilu Zhang and Mert Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  In Advances in neural information processing systems, pages 8778–8788, 2018.
  * [49] Hui Zou and Trevor Hastie.  Regularization and variable selection via the elastic net.  Journal of the royal statistical society: series B (statistical methodology), 67(2):301–320, 2005.