# Cross-Domain Few-Shot Graph Classification

Kaveh Hassani

###### Abstract

We study the problem of few-shot graph classification across domains with
nonequivalent feature spaces by introducing three new cross-domain benchmarks
constructed from publicly available datasets. We also propose an attention-
based graph encoder that uses three congruent views of graphs, one contextual
and two topological views, to learn representations of task-specific
information for fast adaptation, and task-agnostic information for knowledge
transfer. We run exhaustive experiments to evaluate the performance of
contrastive and meta-learning strategies. We show that when coupled with
metric-based meta-learning frameworks, the proposed encoder achieves the best
average meta-test classification accuracy across all benchmarks.

## Introduction

In Few-shot learning a model learns to adapt to novel categories from a few
labeled samples. Common practices such as using augmentation, regularization,
and pretraining may help in such a data-scarce regime, but cannot circumvent
the problem. Inspired by human learning (Lake, Salakhutdinov, and Tenenbaum
2015), meta-learning (Hospedales et al. 2020) leverages a distribution of
similar tasks (Satorras and Estrach 2018) to accumulate transferable knowledge
from prior experience which then can serve as a strong inductive bias for fast
adaptation to down-stream tasks (Sung et al. 2018). In meta-learning, rapid
learning occurs within a task whereas the knowledge about changes in task
structure is gradually learned across tasks (Huang and Zitnik 2020). Examples
of such learned knowledge are embedding functions (Vinyals et al. 2016; Snell,
Swersky, and Zemel 2017; Satorras and Estrach 2018; Sung et al. 2018), initial
parameters (Finn, Abbeel, and Levine 2017; Raghu et al. 2020), optimization
strategies (Li et al. 2017), or models that can directly map training samples
to network weights (Garnelo et al. 2018; Mishra et al. 2018).

A fundamental assumption in meta-learning is that tasks in meta-training and
meta-testing phases are sampled from the same distribution, i.e., tasks are
i.i.d. However, in many real-world applications, collecting tasks from the
same distribution is infeasible. Instead, there are datasets available from
the same modality but different domains. In transfer learning, this is
referred as heterogeneous transfer learning where the feature/label spaces
between the source and target domains are nonequivalent and are generally non-
overlapping (Day and Khoshgoftaar 2017). It is observed that when there is a
large shift between source and target domains, meta-learning algorithms are
outperformed by pre-training/fine-tuning methods (Chen et al. 2019b).

A few work in computer vision addresses cross-domain few-shot learning by
meta-learning the statistics of normalization layers (Tseng et al. 2020; Du et
al. 2021). These methods are limited to natural images that still contain a
high degree of visual similarity (Guo et al. 2020). Cross-domain learning is
more crucial on variable-size order-invariant graph-structured data. Labeling
graphs is more challenging compared to other common modalities because they
usually represent concepts in specialized domains such as biology where
labeling through wet-lab experiments is resource-intensive (Hu et al. 2020b)
and labeling them procedurally using domain knowledge is costly (Sun et al.
2020). Furthermore, nonequivalent and non-overlapping feature spaces is common
across graph datasets in addition to shifts on marginal/conditional
probability distributions. As an example, one may have access to small
molecule datasets where each dataset uses a different set of features to
represent the molecules (Day and Khoshgoftaar 2017).

To the best of our knowledge, this is the first work pertaining cross-domain
few-shot learning on graphs. To address this problem, we design a task-
conditioned encoder that learns to attend to different representations of a
task. Our contributions are as follows:

  * •

We introduce three benchmarks for cross-domain few-shot graph classification
and perform exhaustive experiments to evaluate the performance of supervised,
contrastive, and meta-learning strategies.

  * •

We propose a graph encoder that learns to attend to three congruent views of
graphs, one contextual and two topological views, to learn representations of
task-specific information for fast adaptation, and task-agnostic information
for knowledge transfer.

  * •

We show that when coupled with metric-based meta-learning frameworks, the
proposed encoder achieves the best average meta-testing classification
accuracy across all three benchmarks.

## Related Work

Graph Neural Networks (GNNs) are a class of deep models that combine the
expressive power of graphs in modeling interactions with the unparalleled
capacity of deep learning in learning representations. GNNs learn node
representations over order-invariant and variable-size data, structured as
graphs, through an iterative process of transferring, transforming, and
aggregating the representations from topological neighbors. The learned
representations are then summarized into a graph-level representation (Li et
al. 2015a; Gilmer et al. 2017; Kipf and Welling 2017; Veličković et al. 2018;
Xu et al. 2019; Khasahmadi et al. 2020; Hassani and Khasahmadi 2020). GNNs are
applied to non-Euclidean data such as point clouds (Hassani and Haley 2019),
robot designs (Wang et al. 2019), physical processes (Sanchez-Gonzalez et al.
2020), molecules (Duvenaud et al. 2015), social networks (Kipf and Welling
2017), and knowledge graphs (Vivona and Hassani 2019). For an overview on GNNs
see (Zhang, Cui, and Zhu 2020).

Meta-Learning on Graphs is an under-addressed problem. A few works including
Meta-GNN (Zhou et al. 2019), G-Meta (Huang and Zitnik 2020), and GFL (Yao et
al. 2020) focus on few-shot node classification via meta gradients, whereas
other work such as MetaR (Chen et al. 2019a), GMatching (Xiong et al. 2018),
and Meta-Graph (Bose et al. 2019) focus on few-shot link prediction and
generalization over relations in knowledge graphs. More relevant to this
study, MetaSpecGraph (Chauhan, Nathani, and Kaul 2020) uses a super-class
prototypical network for few-shot graph classification where super-classes are
constructed by clustering graphs using spectrum of normalized Laplacian. All
these works assume i.i.d distribution across the tasks which makes them
infeasible for real-world applications. On the other hand, we for the first
time address the problem of few-shot graph classification within the cross-
domain setting.

Domain Adaptation is a type of transductive transfer learning in which the
source and target classes are equivalent but the domains are different, and
the goal is to reduce the distribution shift between the two domains (Zhuang
et al. 2020; Wilson and Cook 2020). Most work addresses it by reducing the
shift in input, feature, or output spaces using adversarial training (Tzeng et
al. 2017; Chen et al. 2018; Hoffman et al. 2018). A major drawback of these
methods is that they require access to unlabeled samples from the target
domain during the training which makes them less practical (Tseng et al.
2020). For a review see (Wilson and Cook 2020).

Domain Generalization aims to generalize from a set of seen domains to unseen
domains without knowledge about the target distribution during training (Dou
et al. 2019; Tseng et al. 2020). Different strategies such as adversarial data
augmentation (Volpi et al. 2018; Shankar et al. 2018), extracting task-
specific domain-invariant features (Li et al. 2018b; Akuzawa, Iwasawa, and
Matsuo 2019), or learning-to-learn strategies to simulate the generalization
process (Dou et al. 2019; Li et al. 2018a, 2019) are used to tackle this
problem. Domain generalization is more challenging in few-shot setting. On
proper _cross-domain few-shot_ benchmarks, meta-learning methods are
outperformed by simple fine-tuning and even in some cases by networks with
random weights (Guo et al. 2020). Only a few methods are introduced for cross-
domain few-shot learning. Feature-wise transform (FWT) (Tseng et al. 2020)
uses feature-wise transform layers to encourage learning representations with
an improved ability to generalize whereas MetaNorm (Du et al. 2021) learns to
infer adaptive statistics for batch normalization. We use a combination of
learning task-specific domain-invariant features and inductive normalization
layers to achieve domain generalization in cross-domain few-shot graph
classification setting.

## Problem Formulation

A _domain_
$\mathcal{D}=\\{\mathcal{X},\mathcal{Y},P_{\mathcal{X},\mathcal{Y}}\\}$ is
defined as a joint distribution $P_{\mathcal{X},\mathcal{Y}}$ over the feature
space $\mathcal{X}$ and label space $\mathcal{Y}$. We denote the marginal
distribution over feature space as $P_{\mathcal{X}}$ and a parametric model
over joint distribution as $f_{\theta}:\mathcal{X}\longmapsto\mathcal{Y}$
where $f_{\theta}(x)=\\{P(y_{k}|x,\theta)\ |\ y_{k}\in\mathcal{Y}\\}$. The
model parameters are learned by minimizing the expected error over loss
function $\mathcal{L}$: $\mathbb{E}_{(x,y)\sim
P_{\mathcal{X},\mathcal{Y}}}\left[\mathcal{L}(f_{\theta}(x),y)\right]$. In
cross-domain few-shot learning, it is assumed that two domains exist: source
domain $\mathcal{D}_{S}$ and target domain $\mathcal{D}_{T}$ such that their
marginal distributions are different $P_{\mathcal{X}_{S}}\neq
P_{\mathcal{X}_{T}}$, and $\mathcal{Y}_{S}$ and $\mathcal{Y}_{T}$ are
disjoint. The source domain is available during meta-training phase whereas
the target domain is only seen in meta-testing phase. During meta-training,
tasks $\\{\mathcal{T}_{i}|i=1...N\\}$ are drawn from a distribution of tasks
defined over $\mathcal{D}_{S}$, i.e., $\mathcal{T}_{i}\sim
P_{S}(\mathcal{T})$, where each task consists of two non-overlapping small
datasets:$D_{i}^{support}=\\{(x_{j},y_{j})\\}_{j=1}^{k\times
n},D_{i}^{query}=\\{(x_{j},y_{j})\\}_{j=1}^{k\times m}$. $k$ denotes the
number of sampled classes and $n$ and $m$ are number of examples per category,
i.e., _k-way n-shot learning_.

In the meta-training phase, the model’s error on the support set provides
_task-level_ update signals, while error on the query set after the model
adapts to the support set, provides _meta-level_ update signals. During the
meta-testing stage, the model is expected to quickly adapt to task
$\mathcal{T}_{j}\sim P_{T}(\mathcal{T})$ by only accessing the support set for
that task. The tasks in the meta-testing phase are sampled from
$\mathcal{D}_{T}$ and $P_{S}(\mathcal{T})\neq P_{T}(\mathcal{T})$. It is
noteworthy that learning in cross-domain few-shot setting is more difficult
than learning in the transductive setting of traditional meta-learning where
$\mathcal{D}_{S}=\mathcal{D}_{T}$ and as a result
$P_{\mathcal{X}_{S}}=P_{\mathcal{X}_{T}}$ and
$P_{S}(\mathcal{T})=P_{T}(\mathcal{T})$.

In cross-domain few-shot learning, it is assumed that: (1) there is a domain
shift between the source and target domains ($P_{\mathcal{X}_{S}}\neq
P_{\mathcal{X}_{T}}$), and (2) the feature spaces are equivalent between
domains ($\mathcal{X}_{S}=\mathcal{X}_{T}$). This makes sense in computer
vision where various image acquisition methods such as satellite images,
dermatology images, and radiology images share a similar feature space with
natural images (Guo et al. 2020). However, in domains where data is
represented as graphs, this assumption does not hold. For example, two
molecular property prediction datasets may have different node/edge feature
spaces due to the method used to generate the datasets (e.g., one may contain
additional atom features such as formal charge and whether the atom is in the
ring) (Hu et al. 2020a).

As such, we go beyond cross-domain few-shot learning and investigate its
heterogeneous variant for graph classification. In this setting, we assume
that: (1) Each task is possibly sampled from a dedicated domain different from
all other tasks, either in meta-training or meta-testing phase, i.e. if there
are $N$ tasks in meta-training and $M$ tasks in meta-testing phases, there
exist $|\mathcal{D}|\leq N+M$ domains. (2) Tasks are heterogeneous, which
means they may have nonequivalent non-overlapping feature spaces, i.e.,
different dimensions in addition to distribution differences and disjoint
label spaces: $\forall i\neq
j:\mathcal{X}_{\mathcal{T}_{i}}\neq\mathcal{X}_{\mathcal{T}_{j}},P_{\mathcal{X}_{\mathcal{T}_{i}}}\neq
P_{\mathcal{X}_{\mathcal{T}_{j}}},\mathcal{Y}_{\mathcal{T}_{i}}\neq\mathcal{Y}_{\mathcal{T}_{j}}$.
(3) Tasks can be grouped based on their _meta-domains_ which essentially
defines the conceptual domain of a task. For example, all datasets that
represent social networks can be grouped under a _social network_ meta-domain
despite the fact that they may have different feature and label spaces. We
investigate whether there exists underlying knowledge that can be transferred
across these meta-domains.

## Method

Figure 1: The proposed model for cross-domain few-shot graph classification.
Graphs from sampled tasks are augmented to one contextual view and two
geometric views and fed to three dedicated encoders resulting in three
representations of the same graph. An attention mechanism is then used to
aggregate the representations into a single graph representation. The
parameters of the encoders and the attention mechanism are learned end-to-end
using an arbitrary metric-based meta-learning approach.

Graph-structured data can be analyzed from two congruent views: a contextual
view and a topological view. The contextual view is based on initial node or
edge features (for simplicity and without loss of generality, we only consider
node features) and carries task-specific information. The topological view, on
the other hand, represents topological properties of a graph which are task-
agnostic and hence can be used as an anchor to align graphs from various
domains in the feature space. We exploit this dual representation and
explicitly disentangle them by designing dedicated encoders for each view
which in return imposes the needed inductive bias to learn task-specific
domain-invariant features. In a heterogeneous few-shot setting, the
topological features can help with knowledge transfer across tasks whereas the
contextual features can help with fast adaptation. We also use an attention
mechanism that is implicitly conditioned on the tasks and learns to aggregate
the learned features from the two views. We use a meta-learning strategy that
simulates the generalization process by jointly learning the parameters of the
encoders and the attention mechanism. As shown in Figure 1, our method
consists of the following components:

  * •

An augmentation mechanism that transforms a sampled graph into one contextual
view and two topological views. The augmentations are applied to the initial
node feature and graph structure.

  * •

An encoder consisting of two dedicated GNNs, i.e., graph encoders, and an MLP
for the contextual and topological views, respectively, and an attention
mechanism to aggregate the learned features.

  * •

A meta-learning mechanism to jointly learn the parameters of the dedicated
encoders and attention model based on error signals from the query set.

### Augmentations

Recent works on self-supervised learning on graphs suggest that contrasting
graph augmentations allows encoders to learn rich node/graph representations
(Hassani and Khasahmadi 2020). In this work, we are specifically interested in
task-specific and domain-agnostic views of graphs to help the meta-learner to
gradually accumulate domain-agnostic knowledge while utilizing task-specific
information for fast adaptation. We use both feature-space and structure-space
augmentations as follows. For the contextual view, we considered three
feature-space augmentations on the initial node features including: (1)
Heterogeneous feature augmentation (Duan, Xu, and Tsang 2012) where the
initial feature and its projection by a linear layer are concatenated and
padded to a predefined dimension, (2) Deep set (Zaheer et al. 2017) approach
in which we considered the initial node feature space as a set, projecting
each dimension independently to a new space using a linear layer, and
aggregating them by a permutation-invariant function. This augmentation can
capture the shared information across tasks with overlapping features when the
alignment among the features is not available. (3) Simple padding of the
features to a predetermined dimension. Surprisingly, we observed that the
simplest augmentation achieves better results. We speculate this is because
the tasks are not sharing overlapping features.

For the topological view, we apply one feature-space and one structure-space
augmentation. In the feature-space augmentation, we replace the task-dependent
node features with sinusoidal node degree encodings which allow the model to
extrapolate to node degrees greater than the ones encountered during meta-
training stage (Vaswani et al. 2017). Because node degrees are universal
properties of graph nodes, encoding a graph with such initial features will
capture task-agnostic geometric structure of the graph. We also use graph sub-
sampling to keep the degree distribution in a similar order of magnitude
across domains. For the structure-space augmentation, we compute graph
diffusion to provide a global view of the graph’s structure. We used
Personalized PageRank (PPR) (Page et al. 1999), a specific instantiation of
the generalized graph diffusion. We compute the eigenvalues of the diffusion
matrix, sort them in a descending order, and select the top-k eigenvalues as
the structural representation. We also experimented heat kernel diffusion,
eigen values of normalized graph Laplacian, and shortest path matrix, and
found that diffusion produced better results.

### Encoders

Assume a support set $D_{i}^{sup}=[g_{1},g_{2},...,g_{N}]$ of $N$ graphs
belonging to a randomly sampled task $i$. Augmenting each graph $g$ produces
three views: a contextual view represented as graph
$g_{c}=(\textbf{A},\textbf{X})$ where $\textbf{A}\in\\{0,1\\}^{n\times n}$ and
$\textbf{X}\in\mathbb{R}^{n\times d_{x}}$ denote the adjacency matrix and the
task-specific node features, a topological view represented as graph
$g_{g}=(\textbf{A},\textbf{U})$ where $\textbf{U}\in\mathbb{R}^{n\times
d_{u}}$ denotes sinusoidal node degree encodings, and another topological view
represented as vector $\textbf{z}\in\mathbb{R}^{d_{z}}$ denoting the sorted
eigenvalues of the corresponding diffusion matrix
$\textbf{S}\in\mathbb{R}^{n\times n}$. Our framework allows various choices of
network architecture without any constraints. For encoding graph-structured
views, we opted for expressive power and adopted graph isomorphism network
(GIN) (Xu et al. 2019). The $k^{th}$ layer of our graph encoder consists of a
GIN layer followed by a feature-wise transformation layer (FWT) (Tseng et al.
2020) and a swish activation. FWT layer simulates various feature
distributions under different domains: $h_{v}^{(k)}=\gamma^{(k)}\times
h_{v}^{(k)}+\beta^{(k)}$ where
$\gamma\sim\mathcal{N}\left(1,\text{softplus}(\theta_{\gamma})\right)$ and
$\beta\sim\mathcal{N}\left(0,\text{softplus}(\theta_{\beta})\right)$.
$\theta_{\gamma},\theta_{\beta}$ are the standard deviations of the Gaussian
distributions for sampling the affine transformation parameters.

We use a dedicated graph encoder for each view:
$g_{\theta}(.):\mathbb{R}^{n\times d_{x}}\times\mathbb{R}^{n\times
n}\longmapsto\mathbb{R}^{n\times d_{h}}$ and $g_{\phi}(.):\mathbb{R}^{n\times
d_{u}}\times\mathbb{R}^{n\times n}\longmapsto\mathbb{R}^{n\times d_{h}}$
resulting in two sets of node representations $\mathbf{H}^{x}$,
$\mathbf{H}^{u}\in\mathbb{R}^{n\times d_{h}}$ corresponding to the contextual
and the topological views of the sampled graph. For each view, we aggregate
the node representations into a graph representation using a pooling (readout)
function$\mathcal{R}(.):\mathbb{R}^{n\times
d_{h}}\longmapsto\mathbb{R}^{d_{h}}$. We experimented with global soft
attention pooling (Li et al. 2015b), jumping knowledge network (Xu et al.
2018), and summation and mean pooling layers, and found that they produce
similar results. Therefore, we opted for simplicity and used a simple mean
pooling layer. This results in two graph representations:
$\textbf{h}^{x},\textbf{h}^{u}\in\mathbb{R}^{d_{h}}$. We also feed the
topological view from the eigenvalues of the graph diffusion into a projection
head $f_{\psi}(.):\mathbb{R}^{d_{z}}\longmapsto\mathbb{R}^{d_{h}}$, modeled as
an MLP resulting in the third representation:
$\textbf{h}^{z}\in\mathbb{R}^{d_{h}}$.

To aggregate the learned representations, we feed the concatenation of the
learned representations into an attention module
$f_{\omega}(.):\mathbb{R}^{3\times d_{h}}\longmapsto\mathbb{R}^{3}$ that
generates attention scores for each representation. The attention module is
modeled as a single-layer MLP followed by a softmax function:

$\alpha=\text{Softmax}\bigg{(}\text{ReLU}\bigg{(}\bigg{[}\textbf{h}^{x}\parallel\textbf{h}^{u}\parallel\textbf{h}^{z}\bigg{]}\mathbf{W_{1}}\bigg{)}\mathbf{W_{2}}\bigg{)}$
(1)

where $\mathbf{W_{1}}\in\mathbb{R}^{(3\times h_{d})\times h_{d}}$ and
$\mathbf{W_{2}}\in\mathbb{R}^{h_{d}\times 3}$ are network parameters. The
attention scores are then used to aggregate the learned features into a final
graph representation.

The attention mechanism gates the representations and decides if the model
should rely more on contextual or topological representations. If the samples
are from a task that is similar to seen tasks, the model will pay more
attention to contextual representation whereas if there is a drastic shift in
feature space, the model will rely more on geometric representations. We
assume that the target domain is not available during training and hence there
is no information in advance about whether there are shared features among
tasks. If there is, the attention module will pass the shared contextual
information through, otherwise it will not attend to the contextual features
and will let the learner to learn them from scratch during the meta-test
adaptation phase. Hence, rather than naively throwing the information away and
learning from scratch, we let the model decide if it can use the information.
It is noteworthy that we are not introducing a new meta-learning framework.
Instead we are introducing an encoder with attention module that can
seamlessly be integrated into any meta-learning framework. As an example, we
show the training procedure of the encoder within a mini-batch of tasks using
prototypical approach (Snell, Swersky, and Zemel 2017) in Algorithm 1.
Depending on the meta-learner, the aggregated representation can be then fed
into a linear classifier or a non-parametric classifier such as a prototypical
classifier.

Input: Concatenation operator $\parallel$, readout function $\mathcal{R}$,
eigenvalue function $\mathcal{E}$, prototype function $\mathcal{P}$,
prototypical loss $\mathcal{L}$, encoders $g_{\theta}$, $g_{\phi}$,
$f_{\psi}$, attention module $f_{\omega}$, and meta-training task batch
$\\{\mathcal{T}_{j}|\mathcal{T}_{j}=D^{s}_{\mathcal{T}_{j}}\cup
D^{q}_{\mathcal{T}_{j}}\\}_{j=1}^{N}$

for  _$\mathcal{T}$ in a task batch $\\{\mathcal{T}_{j}\\}_{j=1}^{N}$_ do

$[\textbf{H}^{s},\textbf{H}^{q},\mathcal{Y}^{s},\mathcal{Y}^{q}]\leftarrow\emptyset$

for  _$(g,y)$ in $D^{s}_{\mathcal{T}}\cup D^{q}_{\mathcal{T}}$_ do

$\textbf{h}_{x}\leftarrow\mathcal{R}\left(g_{\theta}\left(\textbf{A}_{g},\textbf{X}_{g}\right)\right)$

$\textbf{h}_{u}\leftarrow\mathcal{R}\left(g_{\phi}\left(\textbf{A}_{g},\textbf{U}_{g}\right)\right)$

$\textbf{h}_{z}\leftarrow
f_{\psi}\left(\mathcal{E}\left(\textbf{S}_{g}\right)\right)$

$\alpha\leftarrow
f_{\omega}\left(\left[\textbf{h}_{x}\parallel\textbf{h}_{u}\parallel\textbf{h}_{z}\right]\right)$

$\textbf{h}\leftarrow\alpha_{0}\textbf{h}_{x}+\alpha_{1}\textbf{h}_{u}+\alpha_{2}\textbf{h}_{z}$

if  _$g\in D^{s}_{\mathcal{T}}$_ then

$\textbf{H}^{s}\leftarrow\textbf{H}^{s}\parallel\textbf{h}$ ,
$\mathcal{Y}^{s}\leftarrow\mathcal{Y}^{s}\parallel y$

else

$\textbf{H}^{q}\leftarrow\textbf{H}^{q}\parallel\textbf{h}$ ,
$\mathcal{Y}^{q}\leftarrow\mathcal{Y}^{q}\parallel y$

end if

end for

$\textbf{C}_{\mathcal{T}}\leftarrow\mathcal{P}\left(\mathbf{H}^{s},\mathcal{Y}^{s}\right)$
$\textbf{L}_{\mathcal{T}}\leftarrow\textbf{L}_{\mathcal{T}}+\mathcal{L}\left(\textbf{C}_{\mathcal{T}},\mathbf{H}^{q},\mathcal{Y}^{q}\right)$

end for

$\left[\theta,\omega,\phi,\psi\right]\leftarrow\left[\theta,\omega,\phi,\psi\right]-\gamma\nabla_{\left[\theta,\omega,\phi,\psi\right]}\frac{1}{N\times|D^{q}|}\sum\limits_{j=1}^{N}{\left[\textbf{L}_{\mathcal{T}_{j}}\right]}$

Algorithm 1 Training the proposed encoder with prototypical approach for one mini-batch of tasks. $\textbf{A}_{g},\textbf{S}_{g},\textbf{X}_{g},\textbf{U}_{g}$ denote adjacency, diffusion, node features, and node degree encodings of graph $g$. Table 1: Statistics of the proposed benchmarks for heterogeneous cross-domain few-shot graph classification. Meta-Domain |  $|$Task$|$ | Avg. on target |  $|$shot$|$ |  $|$class$|$ |  $|$query$|$
---|---|---|---|---|---
Source | Target | Train | Dev | Test | Node | Edge | Feature |  |  |
Molecules | Molecules | 169 | 5 | 18 |  26.6 $\pm$ 15.7 |  28.6$\pm$16.6 |  18.1 $\pm$ 18.7 | 1,5,10,20 | 2 | 50
Molecules | Bioinformatics | 187 | 5 | 24 |  79.2 $\pm$ 58.5 |  406.6$\pm$300.3 |  19.8 $\pm$ 15.1 | 1,5,10,20 | 2 | 50
Molecules | Social Networks | 187 | 5 | 12 |  54.1 $\pm$ 58.8 |  98.1$\pm$117.9 | 0 | 1,5,10,20 | 2 | 50
Table 2: Mean and Standard Deviation of meta-test accuracy on Bioinformatics benchmark after ten runs. method | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---
| Empirical Upper Bound | 66.78 $\pm$ 10.30
sup | GCN |  54.88 $\pm$ 7.55 |  55.05 $\pm$ 8.85 |  55.03 $\pm$ 8.91 |  54.99 $\pm$ 8.82
GAT |  54.75 $\pm$ 8.85 |  54.69 $\pm$ 8.90 |  54.76 $\pm$ 8.97 |  54.63 $\pm$ 8.94
GIN |  55.37 $\pm$ 9.83 |  55.52 $\pm$ 9.79 |  55.47 $\pm$ 9.89 |  55.52 $\pm$ 9.65
uns | InfoGraph |  54.00 $\pm$ 6.65 |  53.67 $\pm$ 7.35 |  54.42 $\pm$ 6.41 |  54.96 $\pm$ 7.63
MVGRL |  57.12 $\pm$ 7.75 |  57.25 $\pm$ 9.04 |  57.17 $\pm$ 8.01 |  57.54 $\pm$ 8.06
GSFE |  52.84 $\pm$ 6.71 |  52.96 $\pm$ 7.82 |  53.06 $\pm$ 7.64 |  53.16 $\pm$ 7.87
GCC |  53.11 $\pm$ 6.51 |  53.17 $\pm$ 6.43 |  53.18 $\pm$ 7.18 |  53.35 $\pm$ 7.64
meta | MatchNet |  54.83 $\pm$ 7.66 |  55.62 $\pm$ 7.60 |  55.92 $\pm$ 6.67 |  56.04 $\pm$ 7.78
ProtoNet |  54.71 $\pm$ 8.86 |  55.75 $\pm$ 7.84 |  55.96 $\pm$ 6.73 |  55.50 $\pm$ 9.65
RelationNet |  54.93 $\pm$ 8.55 |  55.92 $\pm$ 8.69 |  56.02 $\pm$ 7.69 |  56.15 $\pm$ 7.81
MAML |  53.83 $\pm$ 9.62 |  54.46 $\pm$ 6.77 |  54.50 $\pm$ 8.77 |  54.79 $\pm$ 8.90
MetaSGD |  53.83 $\pm$ 8.79 |  54.21 $\pm$ 7.70 |  54.67 $\pm$ 9.72 |  54.71 $\pm$ 7.90
MetaSpecGraph | $-$ |  55.47 $\pm$ 7.79 |  55.82 $\pm$ 8.91 |  55.97 $\pm$ 8.89
MatchNet + Our Encoder | 59.14 $\pm$ 7.00 | 59.19 $\pm$ 9.77 |  59.22 $\pm$ 8.72 |  59.56 $\pm$ 6.97
ProtoNet + Our Encoder |  57.17 $\pm$ 7.76 |  57.58 $\pm$ 8.89 |  57.79 $\pm$ 8.76 |  58.17 $\pm$ 7.88
RelationNet + Our Encoder |  58.83 $\pm$ 8.03 |  58.83 $\pm$ 9.68 | 59.29 $\pm$ 7.87 | 59.82 $\pm$ 7.93
MAML + Our Encoder |  56.00 $\pm$ 8.74 |  56.21 $\pm$ 7.76 |  56.37 $\pm$ 8.81 |  57.04 $\pm$ 7.85
MetaSGD + Our Encoder |  55.08 $\pm$ 8.67 |  56.12 $\pm$ 8.23 |  57.08 $\pm$ 8.86 |  57.33$\pm$8.86
Table 3: Mean and Standard Deviation of meta-test accuracy on Social Networks benchmark after ten runs. method | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---
| Empirical Upper Bound | 72.35 $\pm$ 12.38
sup | GCN |  60.51 $\pm$ 10.54 |  60.54 $\pm$ 10.18 |  60.33 $\pm$ 10.19 |  60.35 $\pm$ 10.52
GAT |  61.32 $\pm$ 10.31 |  61.37 $\pm$ 10.20 |  61.17 $\pm$ 10.17 |  61.51 $\pm$ 10.13
GIN |  62.11 $\pm$ 10.11 |  62.98 $\pm$ 10.04 |  63.27 $\pm$ 10.05 |  63.24 $\pm$ 9.28
uns | InfoGraph |  61.92 $\pm$ 9.84 |  62.25 $\pm$ 7.12 |  62.58 $\pm$ 8.57 |  62.58 $\pm$ 7.32
MVGRL |  63.00 $\pm$ 10.70 |  63.75 $\pm$ 11.17 |  63.25 $\pm$ 11.69 |  63.75 $\pm$ 11.99
GSFE |  60.38 $\pm$ 9.74 |  60.45 $\pm$ 9.62 |  60.46 $\pm$ 9.95 |  60.55 $\pm$ 9.11
GCC |  60.61 $\pm$ 9.55 |  60.73 $\pm$ 9.74 |  60.81 $\pm$ 9.61 |  60.98 $\pm$ 9.97
meta | MatchNet |  62.25 $\pm$ 9.14 |  62.42 $\pm$ 9.98 |  62.92 $\pm$ 9.22 |  63.33 $\pm$ 9.92
ProtoNet |  60.50 $\pm$ 9.05 |  61.25 $\pm$ 10.01 |  61.75 $\pm$ 9.06 |  63.50 $\pm$ 9.97
RelationNet |  61.25 $\pm$ 10.04 |  61.08 $\pm$ 10.25 |  61.83 $\pm$ 10.17 |  62.00 $\pm$ 10.17
MAML |  58.33 $\pm$ 9.05 |  58.75 $\pm$ 10.84 |  59.00 $\pm$ 9.87 |  60.17 $\pm$ 10.99
MetaSGD |  59.25 $\pm$ 10.15 |  59.33 $\pm$ 9.50 |  59.83 $\pm$ 9.89 |  60.25 $\pm$ 9.88
MetaSpecGraph | $-$ |  62.55 $\pm$ 9.79 |  62.74 $\pm$ 9.91 |  63.73 $\pm$ 9.89
MatchNet + Our Encoder | 67.40 $\pm$ 10.37 | 67.71 $\pm$ 9.24 | 68.34 $\pm$ 9.09 |  68.90 $\pm$ 9.18
ProtoNet + Our Encoder |  66.42 $\pm$ 10.21 |  66.92 $\pm$ 10.55 |  67.00 $\pm$ 9.13 |  67.50 $\pm$ 10.46
RelationNet + Our Encoder |  66.75 $\pm$ 9.89 |  67.08 $\pm$ 10.80 |  67.33 $\pm$ 10.33 | 69.67 $\pm$ 10.05
MAML + Our Encoder |  62.25 $\pm$ 10.12 |  64.75 $\pm$ 10.79 |  66.58 $\pm$ 10.93 |  66.83 $\pm$ 10.21
MetaSGD + Our Encoder |  61.92$\pm$ 9.56 |  63.17 $\pm$ 10.51 |  64.42 $\pm$ 10.16 |  64.83 $\pm$ 10.06

## Benchmarks

We also introduce three new few-shot graph classification benchmarks with
fixed meta– train/val/test splits constructed from publicly available graph
datasets. In all benchmarks, the source meta-domain consists of molecule
classification tasks. We made this decision because most of the available
graph classification datasets are molecule datasets and hence using them as
the source meta-domain can provide sufficient tasks during meta-training. The
target meta-domains are molecule, bioinformatics, and social networks. Note
that although both source and target meta-domains in the first benchmark
pertain to molecule, the tasks differ in both feature and class spaces. The
process of creating these benchmark was as follows.

We collected all the datasets from TUDataset (Morris et al. 2020) and OGB (Hu
et al. 2020a). We kept graphs with maximum node degree of 50, maximum node
feature dimension of 100, and graphs with a minimum of two nodes and one edge
and ignored the rest. We also filtered out graphs with disconnected
components. For graphs with more than 500 nodes, we sorted the nodes by their
harmonic centrality (Boldi and Vigna 2014) in a descending order and used the
sub-graphs containing the top 500 nodes. For multi-task datatsets, we drew
samples from each task without replacement and split them into several single-
task datasets without sharing any data samples. Because the majority of
datasets in TUDataset and OGB are binary classification tasks, we opted for a
“k-shot 2-way” setting and split the few remaining multi-class datasets into
binary datasets by sampling without replacement. We then randomly selected 20
and 50 samples per class as support and query sets, respectively.

For the second and third benchmarks, we used the processed datasets from
bioinformatics and social network categories as the meta-testing tasks. For
the first benchmark, we split the tasks such that if a task is originated from
a multi-task or a multi-class dataset, it is allocated to a split with all of
the tasks that also originated from the same dataset. The statistics of the
proposed benchmarks are shown in Table 1. We believe these benchmarks will
help the community to drive further advances in heterogeneous meta-learning.

## Experimental Results

We exhaustively perform empirical evaluations to answer the following
questions: (1) what is the the empirical upper bound of the classification
accuracy on the meta-test set for the benchmarks? (2) does any knowledge
transfer occur across meta-domains? If not, does negative transfer occur? (3)
How well does pre-training based on contrastive methods perform? (4) How do
metric-based meta-learning methods perform compared to optimization-based
methods? (5) what is the effect of using the proposed encoder?

To estimate the empirical upper bound of the classification accuracy, we
approximated how fully-supervised models with access to all data would
perform. To accomplish this, we used all the data available within the
datasets from which the few-shot tasks in the meta-testing phase are sampled
and trained one classifier per meta-testing task in a fully supervised
fashion. We evaluated the classifiers on the same query sets used in the few-
shot setting, i.e., query samples are fixed, but the support set is replaced
with all data available in the original dataset. To investigate knowledge
transfer, we trained independent classifiers on the support set of the meta-
testing tasks and evaluated them on their corresponding query set. Negative
transfer occurs when the performance on meta-testing tasks is negatively
affected by knowledge transferred from meta-training tasks (Pan and Yang
2009). Therefore, if the independently trained models outperform models that
utilize knowledge transfer, it implies negative transfer. We also investigated
two strategies to utilize knowledge transfer: pre-training using contrastive
methods and learning-to-learn strategies. For contrastive methods, we
aggregated support and query samples from the meta-training tasks into a
unified dataset, pre-trained a GNN, and then fine-tuned it to the meta-test
tasks. Finally, we investigated the effectiveness of the proposed encoder by
comparing the performance of the meta-learning models with and without using
it.

We report the mean classification accuracy with standard deviation over query
samples of the meta-testing tasks after ten runs. For estimating empirical
upper bound accuracy, we used a GIN (Xu et al. 2019) classifier. For
independent classifiers, we used GCN (Kipf and Welling 2017), GAT (Veličković
et al. 2018), and GIN layers. For contrastive methods, we used GCC (Qiu et al.
2020), GSFE (Hu et al. 2019), InfoGraph (Sun et al. 2020), and MVGRL (Hassani
and Khasahmadi 2020). During the meta-testing phase, we first fine-tuned the
encoder on the support set using the same contrastive approach, and then
trained an MLP on the graph representations. We found that concatenating the
graph representation produced by the contrastive method with a mean pooling of
initial node features improves the performance. For meta-learning methods, we
used three metric-based techniques including matching network (Vinyals et al.
2016), prototypical network (Snell, Swersky, and Zemel 2017), and relation
network (Sung et al. 2018), and two optimization-based methods including MAML
(Finn, Abbeel, and Levine 2017) and MetaSGD (Li et al. 2017). For
optimization-based methods, we used a linear classifier in both the meta-
training and meta-testing phases whereas for metric-based methods, we found
that using a cosine distance classifier (Chen et al. 2019b) for adaptation
achieves better performance. We trained the meta-learning models with and
without our proposed encoder. We also trained a specialized meta-learning
approach for graph classification that uses spectral information (Chauhan,
Nathani, and Kaul 2020). The results are shown in Tables 2 and 3 (for Molecule
benchmark results, hyper-parameters and other details refer to Appendix).

Results suggest that: (1) On all three benchmarks, there exists underlying
knowledge that can be transferred. This is validated by the observation that
both meta-learning and contrastive approaches outperform naive classifiers.
(2) Contrastive approaches are competitive with meta-learning methods without
the proposed encoder. As an example, on 20-shot bioinformatics benchmark,
MVGRL outperforms the best performing meta-learning method by 1.57% absolute
accuracy. (3) Coupling metric-based meta-learning methods with our proposed
encoder significantly enhances performance. As an instance, on 1-shot setting,
the best meta-learning methods coupled with our encoder outperform the best
results achieved by regular meta-learning methods by 3.28%, 4.29%, and 5.17%
absolute accuracy on molecules, bioinformatics, and social network benchmarks,
respectively. (4) RelationNet coupled with our encoder and only trained with
20 examples is only 4.46%, 6.96%, and 2.68% less accurate than fully-
supervised models trained on all available data of molecules, bioinformatics,
and social network benchmarks, respectively. Note that some of these datasets
have tens of thousands training samples. (5) We get the most improvement when
we transfer knowledge from molecular meta-training to social network meta-
testing. This is because social network tasks do not contain any initial node
features, and hence classifying them completely depends on task-agnostic
geometric features. This suggests that our encoder is able to learn expressive
geometric representations on one domain and generalize to another domain.

### Ablation Study

#### Effect of Views.

To investigate the effect of views and the proposed encoder, we run the
experiments with: (1) Only node features X which is equivalent to traditional
meta-learning, (2) a combination of node features X and node degree encodings
U, (3) a combination of node features X and eigenvalues of diffusion matrix Z,
and (4) combination of all the features. The result (see Tables 6-8 in
Appendix) suggest that: (1) both topological views contribute complementary
enhancements to the performance, and (2) the proposed encoder has a better
performance when coupled with metric-based methods compared to optimization-
based methods. Also we observe that the diffusion based view has a slightly
larger effect on performance. We speculate this is because of the inductive
bias that diffusion provides (it directly encodes graph encoding from global
information).

#### Why topological properties are task-agnostic?

Node degree distributions are different across domains (e.g., social networks
and molecules). We are addressing this by a few tricks: (1) we use sinusoidal
node degree encodings which allow the model to extrapolate to unseen node
degrees, (2) we use graph sub-sampling to keep the degree distribution in a
similar order of magnitude, and most importantly, (3) we are interleaving FWT
and GNN layers to address the distribution shift. This allows us to first
relax the heterogeneity by relying on topological signals and then use the
mentioned tricks to control the distribution shifts that may occur. It is
noteworthy that we found the encodings from diffusion to be less sensitive to
the shifts in graph size and node degrees distributions.

#### Effect of Meta-Learning Approach.

We considered optimization-based and metric-based approaches. Results suggest
that without using our encoder, metric-based approaches are competitive with
contrastive methods and also show some degree of knowledge transfer, whereas
optimization-based approaches are outperformed by independent classifiers.
That suggests that optimization-based approaches lead to negative knowledge
transfer, confirming the observation in (Guo et al. 2020). When coupled with
our encoder, optimization-based approaches become competitive with contrastive
methods and show some knowledge transfer but are still outperformed by metric-
based approaches (see Appendix). As suggested in (Huang and Zitnik 2020), this
is likely because metric-based approaches leverage the inductive bias between
representation and labels to effectively propagate scarce label information.

#### Effect of Task Encoding.

We attempted to explicitly condition the attention module on the task
encodings as follows. We pretrained an encoder using MVGRL and encode the
support set of each task. We then aggregated the representation into a task
encoding and stored meta-training task encodings within a memory-bank. We
trained the attention module by feeding it with the graph views, the task
encodings, and the encodings of the top-3 similar tasks. During meta-testing,
we retrieve the top-3 similar tasks from the memory bank. Surprisingly, we
found that using task encodings produces nearly identical results to not
conditioning the attention module on task encodings.

## Conclusion

We investigated the problem of cross-domain few-shot graph classification by
introducing three new benchmarks. We performed exhaustive experiments using
independent classifiers, contrastive methods, and meta-learning strategies. We
also introduced a simple yet powerful multi-view graph encoder with an
attention-based aggregation mechanism for better knowledge transfer and
adaptation. We showed that metric-based meta-learning approaches coupled with
the proposed encoder achieve the best performance across all benchmarks. We
also showed that optimization-based meta learning methods struggle in cross-
domain setting. In future work, we plan to investigate black-box and hybrid
adaptations of meta-learning strategies.

## References

  * Akuzawa, Iwasawa, and Matsuo (2019) Akuzawa, K.; Iwasawa, Y.; and Matsuo, Y. 2019.  Domain Generalization via Invariant Representation under Domain-Class Dependency.
  * Boldi and Vigna (2014) Boldi, P.; and Vigna, S. 2014.  Axioms for centrality.  _Internet Mathematics_ , 10(3-4): 222–262.
  * Bose et al. (2019) Bose, A. J.; Jain, A.; Molino, P.; and Hamilton, W. L. 2019.  Meta-Graph: Few Shot Link Prediction via Meta Learning.  _arXiv preprint arXiv:1912.09867_.
  * Chauhan, Nathani, and Kaul (2020) Chauhan, J.; Nathani, D.; and Kaul, M. 2020.  Few-Shot Learning on Graphs via Super-Classes Based on Graph Spectral Measures.  In _International Conference on Learning Representations_.
  * Chen et al. (2019a) Chen, M.; Zhang, W.; Zhang, W.; Chen, Q.; and Chen, H. 2019a.  Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs.  In _Conference on Empirical Methods in Natural Language Processing_ , 4208–4217.
  * Chen et al. (2019b) Chen, W.-Y.; Liu, Y.-C.; Kira, Z.; Wang, Y.-C. F.; and Huang, J.-B. 2019b.  A Closer Look at Few-shot Classification.  In _International Conference on Learning Representations_.
  * Chen et al. (2018) Chen, Y.; Li, W.; Sakaridis, C.; Dai, D.; and Van Gool, L. 2018.  Domain adaptive faster r-cnn for object detection in the wild.  In _IEEE Conference on computer vision and pattern recognition_ , 3339–3348.
  * Day and Khoshgoftaar (2017) Day, O.; and Khoshgoftaar, T. M. 2017.  A survey on heterogeneous transfer learning.  _Journal of Big Data_ , 4(1): 29.
  * Dou et al. (2019) Dou, Q.; de Castro, D. C.; Kamnitsas, K.; and Glocker, B. 2019.  Domain generalization via model-agnostic learning of semantic features.  In _Advances in Neural Information Processing Systems_ , 6450–6461.
  * Du et al. (2021) Du, Y.; Zhen, X.; Shao, L.; and Snoek, C. G. M. 2021.  MetaNorm: Learning to Normalize Few-Shot Batches Across Domains.  In _International Conference on Learning Representations_.
  * Duan, Xu, and Tsang (2012) Duan, L.; Xu, D.; and Tsang, I. W. 2012.  Learning with augmented features for heterogeneous domain adaptation.  In _International Conference on Machine Learning_ , 667–674.
  * Duvenaud et al. (2015) Duvenaud, D. K.; Maclaurin, D.; Iparraguirre, J.; Bombarell, R.; Hirzel, T.; Aspuru-Guzik, A.; and Adams, R. P. 2015.  Convolutional networks on graphs for learning molecular fingerprints.  In _Advances in Neural Information Processing Systems_ , 2224–2232.
  * Finn, Abbeel, and Levine (2017) Finn, C.; Abbeel, P.; and Levine, S. 2017.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks.  In _International Conference on Machine Learning_ , 1126–1135.
  * Garnelo et al. (2018) Garnelo, M.; Rosenbaum, D.; Maddison, C.; Ramalho, T.; Saxton, D.; Shanahan, M.; Teh, Y. W.; Rezende, D.; and Eslami, S. A. 2018.  Conditional Neural Processes.  In _International Conference on Machine Learning_ , 1704–1713.
  * Gilmer et al. (2017) Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; and Dahl, G. E. 2017.  Neural message passing for quantum chemistry.  In _International Conference on Machine Learning_ , 1263–1272.
  * Guo et al. (2020) Guo, Y.; Codella, N. C.; Karlinsky, L.; Codella, J. V.; Smith, J. R.; Saenko, K.; Rosing, T.; and Feris, R. 2020.  A broader study of cross-domain few-shot learning.  In _European Conference on Computer Vision_ , 124–141. Springer.
  * Hassani and Haley (2019) Hassani, K.; and Haley, M. 2019.  Unsupervised Multi-Task Feature Learning on Point Clouds.  In _International Conference on Computer Vision_ , 8160–8171.
  * Hassani and Khasahmadi (2020) Hassani, K.; and Khasahmadi, A. H. 2020.  Contrastive Multi-View Representation Learning on Graphs.  In _Proceedings of the 37th International Conference on Machine Learning_ , 4116–4126.
  * Hoffman et al. (2018) Hoffman, J.; Tzeng, E.; Park, T.; Zhu, J.-Y.; Isola, P.; Saenko, K.; Efros, A.; and Darrell, T. 2018.  Cycada: Cycle-consistent adversarial domain adaptation.  In _International conference on machine learning_ , 1989–1998.
  * Hospedales et al. (2020) Hospedales, T.; Antoniou, A.; Micaelli, P.; and Storkey, A. 2020.  Meta-learning in neural networks: A survey.  _arXiv preprint arXiv:2004.05439_.
  * Hu et al. (2020a) Hu, W.; Fey, M.; Zitnik, M.; Dong, Y.; Ren, H.; Liu, B.; Catasta, M.; and Leskovec, J. 2020a.  Open graph benchmark: Datasets for machine learning on graphs.  _arXiv preprint arXiv:2005.00687_.
  * Hu et al. (2020b) Hu, W.; Liu, B.; Gomes, J.; Zitnik, M.; Liang, P.; Pande, V.; and Leskovec, J. 2020b.  Strategies for Pre-training Graph Neural Networks.  In _International Conference on Learning Representations_.
  * Hu et al. (2019) Hu, Z.; Fan, C.; Chen, T.; Chang, K.-W.; and Sun, Y. 2019.  Pre-training graph neural networks for generic structural feature extraction.  _arXiv preprint arXiv:1905.13728_.
  * Huang and Zitnik (2020) Huang, K.; and Zitnik, M. 2020.  Graph meta learning via local subgraphs.  _Advances in Neural Information Processing Systems_.
  * Khasahmadi et al. (2020) Khasahmadi, A. H.; Hassani, K.; Moradi, P.; Lee, L.; and Morris, Q. 2020.  Memory-Based Graph Networks.  In _International Conference on Learning Representations_.
  * Kipf and Welling (2017) Kipf, T. N.; and Welling, M. 2017.  Semi-supervised classification with graph convolutional networks.  In _International Conference on Learning Representations_.
  * Lake, Salakhutdinov, and Tenenbaum (2015) Lake, B. M.; Salakhutdinov, R.; and Tenenbaum, J. B. 2015.  Human-level concept learning through probabilistic program induction.  _Science_ , 350(6266): 1332–1338.
  * Li et al. (2018a) Li, D.; Yang, Y.; Song, Y.-Z.; and Hospedales, T. 2018a.  Learning to generalize: Meta-learning for domain generalization.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32.
  * Li et al. (2015a) Li, Y.; Tarlow, D.; Brockschmidt, M.; and Zemel, R. 2015a.  Gated graph sequence neural networks.  In _International Conference on Learning Representations_.
  * Li et al. (2015b) Li, Y.; Tarlow, D.; Brockschmidt, M.; and Zemel, R. 2015b.  Gated graph sequence neural networks.  _arXiv preprint arXiv:1511.05493_.
  * Li et al. (2018b) Li, Y.; Tian, X.; Gong, M.; Liu, Y.; Liu, T.; Zhang, K.; and Tao, D. 2018b.  Deep domain generalization via conditional invariant adversarial networks.  In _Proceedings of the European Conference on Computer Vision_ , 624–639.
  * Li et al. (2019) Li, Y.; Yang, Y.; Zhou, W.; and Hospedales, T. 2019.  Feature-Critic Networks for Heterogeneous Domain Generalization.  In _International Conference on Machine Learning_ , 3915–3924.
  * Li et al. (2017) Li, Z.; Zhou, F.; Chen, F.; and Li, H. 2017.  Meta-sgd: Learning to learn quickly for few-shot learning.  _arXiv preprint arXiv:1707.09835_.
  * Mishra et al. (2018) Mishra, N.; Rohaninejad, M.; Chen, X.; and Abbeel, P. 2018.  A Simple Neural Attentive Meta-Learner.  In _International Conference on Learning Representations_.
  * Morris et al. (2020) Morris, C.; Kriege, N. M.; Bause, F.; Kersting, K.; Mutzel, P.; and Neumann, M. 2020\.  TUDataset: A collection of benchmark datasets for learning with graphs.  In _ICML 2020 Workshop on Graph Representation Learning and Beyond_.
  * Page et al. (1999) Page, L.; Brin, S.; Motwani, R.; and Winograd, T. 1999.  The pagerank citation ranking: Bringing order to the web.  Technical report, Stanford InfoLab.
  * Pan and Yang (2009) Pan, S. J.; and Yang, Q. 2009.  A survey on transfer learning.  _IEEE Transactions on knowledge and data engineering_ , 22(10): 1345–1359.
  * Qiu et al. (2020) Qiu, J.; Chen, Q.; Dong, Y.; Zhang, J.; Yang, H.; Ding, M.; Wang, K.; and Tang, J. 2020.  _GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training_ , 1150–1160.
  * Raghu et al. (2020) Raghu, A.; Raghu, M.; Bengio, S.; and Vinyals, O. 2020.  Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML.  In _International Conference on Learning Representations_.
  * Sanchez-Gonzalez et al. (2020) Sanchez-Gonzalez, A.; Godwin, J.; Pfaff, T.; Ying, R.; Leskovec, J.; and Battaglia, P. 2020.  Learning to Simulate Complex Physics with Graph Networks.  In _Proceedings of the 37th International Conference on Machine Learning_ , 8459–8468.
  * Satorras and Estrach (2018) Satorras, V. G.; and Estrach, J. B. 2018.  Few-Shot Learning with Graph Neural Networks.  In _International Conference on Learning Representations_.
  * Shankar et al. (2018) Shankar, S.; Piratla, V.; Chakrabarti, S.; Chaudhuri, S.; Jyothi, P.; and Sarawagi, S. 2018.  Generalizing Across Domains via Cross-Gradient Training.  In _International Conference on Learning Representations_.
  * Snell, Swersky, and Zemel (2017) Snell, J.; Swersky, K.; and Zemel, R. 2017.  Prototypical networks for few-shot learning.  In _Advances in neural information processing systems_ , 4077–4087.
  * Sun et al. (2020) Sun, F.-Y.; Hoffman, J.; Verma, V.; and Tang, J. 2020.  InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization.  In _International Conference on Learning Representations_.
  * Sung et al. (2018) Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Torr, P. H.; and Hospedales, T. M. 2018\.  Learning to compare: Relation network for few-shot learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_.
  * Tseng et al. (2020) Tseng, H.-Y.; Lee, H.-Y.; Huang, J.-B.; and Yang, M.-H. 2020.  Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation.  In _International Conference on Learning Representations_.
  * Tzeng et al. (2017) Tzeng, E.; Hoffman, J.; Saenko, K.; and Darrell, T. 2017.  Adversarial discriminative domain adaptation.  In _IEEE conference on computer vision and pattern recognition_ , 7167–7176.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , 5998–6008.
  * Veličković et al. (2018) Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; and Bengio, Y. 2018.  Graph Attention Networks.  In _International Conference on Learning Representations_.
  * Vinyals et al. (2016) Vinyals, O.; Blundell, C.; Lillicrap, T.; Wierstra, D.; et al. 2016.  Matching networks for one shot learning.  _Advances in neural information processing systems_ , 29: 3630–3638.
  * Vivona and Hassani (2019) Vivona, S.; and Hassani, K. 2019.  Relational Graph Representation Learning for Open-Domain Question Answering.  _Advances in Neural Information Processing Systems, Graph Representation Learning Workshop_.
  * Volpi et al. (2018) Volpi, R.; Namkoong, H.; Sener, O.; Duchi, J. C.; Murino, V.; and Savarese, S. 2018\.  Generalizing to unseen domains via adversarial data augmentation.  In _Advances in Neural Information Processing Systems_ , 5334–5344.
  * Wang et al. (2019) Wang, T.; Zhou, Y.; Fidler, S.; and Ba, J. 2019.  Neural Graph Evolution: Automatic Robot Design.  In _International Conference on Learning Representations_.
  * Wilson and Cook (2020) Wilson, G.; and Cook, D. J. 2020.  A survey of unsupervised deep domain adaptation.  _ACM Transactions on Intelligent Systems and Technology_ , 11(5): 1–46.
  * Xiong et al. (2018) Xiong, W.; Yu, M.; Chang, S.; Guo, X.; and Wang, W. Y. 2018.  One-Shot Relational Learning for Knowledge Graphs.  In _Conference on Empirical Methods in Natural Language Processing_ , 1980–1990.
  * Xu et al. (2019) Xu, K.; Hu, W.; Leskovec, J.; and Jegelka, S. 2019.  How Powerful are Graph Neural Networks?  In _International Conference on Learning Representations_.
  * Xu et al. (2018) Xu, K.; Li, C.; Tian, Y.; Sonobe, T.; Kawarabayashi, K.-i.; and Jegelka, S. 2018\.  Representation Learning on Graphs with Jumping Knowledge Networks.  In _International Conference on Machine Learning_ , 5453–5462.
  * Yao et al. (2020) Yao, H.; Zhang, C.; Wei, Y.; Jiang, M.; Wang, S.; Huang, J.; Chawla, N.; and Li, Z. 2020.  Graph few-shot learning via knowledge transfer.  In _AAAI Conference on Artificial Intelligence_ , 6656–6663.
  * Zaheer et al. (2017) Zaheer, M.; Kottur, S.; Ravanbakhsh, S.; Poczos, B.; Salakhutdinov, R. R.; and Smola, A. J. 2017.  Deep sets.  In _Advances in neural information processing systems_.
  * Zhang, Cui, and Zhu (2020) Zhang, Z.; Cui, P.; and Zhu, W. 2020.  Deep learning on graphs: A survey.  _IEEE Transactions on Knowledge and Data Engineering_.
  * Zhou et al. (2019) Zhou, F.; Cao, C.; Zhang, K.; Trajcevski, G.; Zhong, T.; and Geng, J. 2019.  Meta-GNN: On Few-shot Node Classification in Graph Meta-learning.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , 2357–2360.
  * Zhuang et al. (2020) Zhuang, F.; Qi, Z.; Duan, K.; Xi, D.; Zhu, Y.; Zhu, H.; Xiong, H.; and He, Q. 2020\.  A comprehensive survey on transfer learning.  _Proceedings of the IEEE_ , 109(1): 43–76.

## Appendix

### Datasets, Pre-processing, & Implementation

The statistics of the used datasets is reported in Tables 1-3. Mutli-task
column indicates whether the original dataset is a multi-task dataset whereas
task column denotes the number of tasks extracted from the dataset for few-
shot setting. Class column indicates the number of classification categories
in the original dataset. As shown, most of the datasets are for binary
classification problems and also molecule datasets are the most frequent ones.
This is why we choose to set the molecules meta-domain as the meta-training
domain and set the problem to “k-way 2-shot”.

We addressed the datasets without initial node features as follows. If a
dataset does not carry initial node features but contains node labels, we use
the one-hot encoding of the node labels as the initial features. Otherwise, we
initialize the node features with a 16-dimensional vector of ones. We also
normalize the initial node features row-wise.

We implemented the experiments using PyTorch, and used Pytorch Geometric and
Learn2Learn libraries to implement graph encoders and optimization-based meta-
learning algorithms. The experiments are run on four RTX 6000 GPUs where on
average one epoch of training optimization-based algorithms takes 45 seconds
and training one epoch of metric-based methods takes about 6 seconds.

### Hyper-Parameters

We initialize the parameters using Xavier initialization and train all the
model using Adam optimizer. For all the experiments, we fix the number of
epochs to 1000, the size of hidden dimension to 256, size of sinusoidal node
degree embeddings to 32, and the number of eigenvalues of the diffusion matrix
to 128. During the meta-training, we use a cosine scheduler to schedule the
learning rate, and also use early stopping with patience of 30. We also use a
meta-batch of 16 tasks. We choose the number of task steps, number of
adaptation steps, task learning rate, and adaptation learning rate from [10,
25, 50], [10, 25, 50], [0.001, 0.01], and [0.01, 0.1], respectively. For the
encoders we choose the number of GNN and MLP layers from [1, 2, 3]. To prevent
the models from over-fitting, we use dropout with probability of 0.6, and
feature-wise transform after the linear layers. We found that sampling the
hyper-parameters of the feature-wise transform layers produces good results in
our heterogeneous setting. We set the teleport probability for computing the
diffusion to 0.2. For MAML and MetaSGD, we also experimented with first and
second order optimization settings. Finally, for matching network, we used a
bidirectional LSTM to generate fully conditional embeddings of the support set
and an attentional LSTM to generate fully conditional embeddings of the query
set. The selected hyper-parameters for 20-shot setting are shown in Table 4.

### Effect of Views & Encoder

To investigate the effect of views and the proposed encoder, we run the
experiments with: (1) Only initial node features X which is equivalent to
traditional meta-learning, (2) initial node features X and node degree
features U, (3) initial node features X and eigenvalues of diffusion matrix Z,
and (4) all the mentioned features. Setting 2-4 are essentially adding the
proposed encoder to the meta-learning algorithms. The result shown in Tables
6-8 suggest that: (1) both geometric views contribute complementary
enhancements to the performance, and (2) the proposed encoder has a better
performance when coupled with metric-based methods compared to optimization-
based methods.

## subsection

Diffusion is formulated as Eq. (2) where $\textbf{T}\in\mathbb{R}^{n\times n}$
is the generalized transition matrix and $\Theta$ is the weighting coefficient
which determines the ratio of global-local information. Imposing
$\sum_{k=0}^{\infty}\theta_{k}=1$, $\theta_{k}\in[0,1]$, and
$\lambda_{i}\in[0,1]$ where $\lambda_{i}$ are eigenvalues of T, guarantees
convergence. Diffusion is computed once using fast approximation and
sparsification methods.

$\textbf{S}=\sum_{k=0}^{\infty}{\Theta_{k}\textbf{T}^{k}}\in\mathbb{R}^{n\times
n}$ (2)

Given an adjacency matrix $\textbf{A}\in\mathbb{R}^{n\times n}$ and a diagonal
degree matrix $\textbf{D}\in\mathbb{R}^{n\times n}$, Personalized PageRank
(PPR) and heat kernel , i.e., two instantiations of the generalized graph
diffusion, are defined by setting $\mathbf{T}=\textbf{AD}^{-1}$, and
$\theta_{k}=\alpha(1-\alpha)^{k}$ and $\theta_{k}=e^{-t}t^{k}/{k!}$,
respectively, where $\alpha$ denotes teleport probability in a random walk and
$t$ is diffusion time.

Table 4: Statistics of the datasets used in molecules meta-domain. Dataset | Source | Multi-Task |  $|$class$|$ |  $|$graph$|$ | Avg. Node | Avg. Edge |  $|$task$|$ | split
---|---|---|---|---|---|---|---|---
AIDS | TU | N | 2 | 2000 | 15.69 | 16.20 | 1 | meta-train
BZR | TU | N | 2 | 405 | 35.75 | 38.36 | 1 | meta-train
COX2 | TU | N | 2 | 467 | 41.22 | 43.45 | 1 | meta-train
DHFR | TU | N | 2 | 467 | 42.43 | 44.54 | 1 | meta-train
MCF-7 | TU | N | 2 | 27770 | 26.39 | 28.52 | 1 | meta-train
MCF-7H | TU | N | 2 | 27770 | 47.30 | 49.43 | 1 | meta-train
MOLT-4 | TU | N | 2 | 39765 | 26.09 | 28.13 | 1 | meta-train
MOLT-4H | TU | N | 2 | 39765 | 46.70 | 48.73 | 1 | meta-train
MUTAGENICITY | TU | N | 2 | 4337 | 30.32 | 30.77 | 1 | meta-train
NCI1 | TU | N | 2 | 4110 | 9.87 | 32.30 | 1 | meta-test
NCI109 | TU | N | 2 | 4127 | 29.68 | 32.13 | 1 | meta-test
P388 | TU | N | 2 | 41472 | 22.11 | 23.55 | 1 | meta-train
P388H | TU | N | 2 | 41472 | 40.44 | 41.88 | 1 | meta-train
PC-3 | TU | N | 2 | 27509 | 26.35 | 28.49 | 1 | meta-train
PC-3H | TU | N | 2 | 27509 | 47.19 | 49.32 | 1 | meta-train
PTC-FM | TU | N | 2 | 349 | 14.11 | 14.48 | 1 | meta-test
PTC-FR | TU | N | 2 | 351 | 14.56 | 15.00 | 1 | meta-test
PTC-MM | TU | N | 2 | 336 | 13.97 | 14.32 | 1 | meta-test
PTC-MR | TU | N | 2 | 344 | 14.29 | 14.69 | 1 | meta-test
SF-295 | TU | N | 2 | 40271 | 26.06 | 28.08 | 1 | meta-train
SF-295H | TU | N | 2 | 40271 | 46.65 | 48.68 | 1 | meta-train
SN12C | TU | N | 2 | 40004 | 26.08 | 28.11 | 1 | meta-train
SN12CH | TU | N | 2 | 40004 | 46.69 | 48.71 | 1 | meta-train
SW-620 | TU | N | 2 | 40532 | 26.05 | 28.08 | 1 | meta-val
SW-620H | TU | N | 2 | 40532 | 46.62 | 48.65 | 1 | meta-val
Tox21-AhR | TU | N | 2 | 8169 | 18.09 | 18.50 | 1 | meta-test
Tox21-AR | TU | N | 2 | 9362 | 18.39 | 18.84 | 1 | meta-test
Tox21-AR-LBD | TU | N | 2 | 8599 | 17.77 | 18.16 | 1 | meta-test
Tox21-ARE | TU | N | 2 | 7167 | 16.28 | 16.52 | 1 | meta-test
Tox21-aromatase | TU | N | 2 | 7226 | 17.50 | 17.79 | 1 | meta-test
Tox21-ATAD5 | TU | N | 2 | 9091 | 17.89 | 18.30 | 1 | meta-test
Tox21-ER | TU | N | 2 | 7697 | 17.58 | 17.94 | 1 | meta-test
Tox21-ER-LBD | TU | N | 2 | 8753 | 18.06 | 18.47 | 1 | meta-test
Tox21-HSE | TU | N | 2 | 8150 | 16.72 | 17.04 | 1 | meta-test
Tox21-MMP | TU | N | 2 | 7320 | 17.49 | 17.83 | 1 | meta-test
Tox21-p53 | TU | N | 2 | 8634 | 17.79 | 18.19 | 1 | meta-test
Tox21-PPAR-GAMMA | TU | N | 2 | 8184 | 17.23 | 17.55 | 1 | meta-test
UACC257 | TU | N | 2 | 39988 | 26.09 | 28.12 | 1 | meta-train
UACC257H | TU | N | 2 | 39988 | 46.68 | 48.71 | 1 | meta-train
YEAST | TU | N | 2 | 79601 | 21.54 | 22.84 | 1 | meta-val
YEASTH | TU | N | 2 | 79601 | 39.44 | 40.74 | 1 | meta-val
MOLHIV | OGB | N | 2 | 41127 | 25.51 | 25.51 | 1 | meta-train
MOLBACE | OGB | N | 2 | 1513 | 34.09 | 73.72 | 1 | meta-train
MOLBBBP | OGB | N | 2 | 2039 | 24.06 | 51.91 | 1 | meta-train
MOLPCBA | OGB | Y | 2 | 437929 | 25.97 | 56.22 | 121 | meta-train
MOLCLINTOX | OGB | Y | 2 | 1477 | 26.16 | 55.77 | 1 | meta-val
MOLSIDER | OGB | Y | 2 | 1427 | 33.64 | 70.72 | 8 | meta-train
MOLTOXCAST | OGB | Y | 2 | 8576 | 18.78 | 38.52 | 17 | meta-train
Table 5: Statistics of the datasets used in bioinformatics meta-domain. Dataset | Source | Multi-Task |  $|$class$|$ |  $|$graph$|$ | Avg. Node | Avg. Edge |  $|$task$|$ | Split
---|---|---|---|---|---|---|---|---
DD | TU | N | 2 | 1178 | 284.32 | 715.66 | 1 | meta-test
ENZYMES | TU | N | 6 | 600 | 32.63 | 62.14 | 3 | meta-test
PROTEINS | TU | N | 2 | 1113 | 39.06 | 72.82 | 1 | meta-test
PROTEINS-full | TU | N | 2 | 1113 | 39.06 | 72.82 | 1 | meta-test
PPA | OGB | N | 37 | 158100 | 243.42 | 4532.19 | 18 | meta-test
Table 6: Statistics of the datasets used in social networks meta-domain. Dataset | Source | Multi-Task |  $|$class$|$ |  $|$graph$|$ | Avg. Node | Avg. Edge |  $|$task$|$ | Split
---|---|---|---|---|---|---|---|---
COLLAB | TU | N | 3 | 5000 | 74.49 | 2457.78 | 1 | meta-test
DEEZER-EGO-NETS | TU | N | 2 | 9629 | 23.49 | 65.25 | 1 | meta-test
GITHUB-STARGAZERS | TU | N | 2 | 12725 | 113.79 | 234.64 | 1 | meta-test
IMDB-BINARY | TU | N | 2 | 1000 | 19.77 | 96.53 | 1 | meta-test
IMDB-MULTI | TU | N | 3 | 1500 | 13.00 | 65.94 | 1 | meta-test
REDDIT-BINARY | TU | N | 2 | 2000 | 429.63 | 497.75 | 1 | meta-test
REDDIT-MULTI-5K | TU | N | 5 | 4999 | 508.52 | 594.87 | 1 | meta-test
REDDIT-MULTI-12K | TU | N | 11 | 11929 | 391.41 | 456.89 | 3 | meta-test
REDDIT-THREADS | TU | N | 2 | 203088 | 23.93 | 24.99 | 1 | meta-test
TWITCH-EGOS | TU | N | 2 | 127094 | 29.67 | 86.59 | 1 | meta-test
Table 7: Selected hyper-parameters for metric-based meta-learning algorithms in 20-shot setting. method | Task step | Adapt step | Task LR | Adapt LR |  $|$Layers$|$
---|---|---|---|---|---
Mol | MatchNet | 25 | 10 | 0.01 | 0.1 | 2
ProtoNet | 50 | 50 | 0.01 | 0.01 | 3
RelationNet | 50 | 50 | 0.001 | 0.01 | 2
Bio | MatchNet | 50 | 50 | 0.001 | 0.01 | 2
ProtoNet | 50 | 10 | 0.001 | 0.1 | 2
RelationNet | 25 | 25 | 0.01 | 0.1 | 3
Social | MatchNet | 50 | 25 | 0.01 | 0.1 | 3
ProtoNet | 25 | 10 | 0.01 | 0.1 | 3
RelationNet | 50 | 10 | 0.01 | 0.01 | 3
Table 8: Mean and Standard Deviation of meta-test accuracy on Molecules benchmark after ten runs. method | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---
| Empirical Upper Bound | 67.62 $\pm$ 7.67
Sup | GCN |  56.68 $\pm$ 6.16 |  56.83 $\pm$ 6.03 |  56.59 $\pm$ 6.28 |  56.69 $\pm$ 6.06
GAT |  56.48 $\pm$ 5.99 |  56.80 $\pm$ 5.87 |  56.66 $\pm$ 5.89 |  56.76 $\pm$ 6.02
GIN |  57.12 $\pm$ 6.73 |  57.59 $\pm$ 6.61 |  57.77 $\pm$ 6.72 |  58.96 $\pm$ 4.96
Uns | InfoGraph |  57.11 $\pm$ 6.89 |  57.44 $\pm$ 6.61 |  57.28 $\pm$ 7.16 |  57.06 $\pm$ 6.66
MVGRL |  58.89 $\pm$ 6.68 |  58.78 $\pm$ 5.33 |  58.83 $\pm$ 6.06 |  58.39 $\pm$ 6.33
GSFE |  56.74 $\pm$ 5.93 |  56.93 $\pm$ 6.24 |  56.98 $\pm$ 6.85 |  57.21 $\pm$ 6.43
GCC |  56.61 $\pm$ 5.88 |  56.84 $\pm$ 5.97 |  57.05 $\pm$ 6.32 |  57.36 $\pm$ 6.14
Meta | MatchNet |  58.89 $\pm$ 6.31 |  59.33 $\pm$ 5.71 |  59.50 $\pm$ 6.83 |  60.06 $\pm$ 6.87
ProtoNet |  58.72 $\pm$ 6.75 |  59.11 $\pm$ 6.80 |  59.28 $\pm$ 4.89 |  59.44 $\pm$ 6.13
RelationNet |  58.78 $\pm$ 7.73 |  59.06 $\pm$ 6.96 |  59.17 $\pm$ 6.96 |  60.11 $\pm$ 5.02
MAML |  56.00 $\pm$ 6.71 |  56.78 $\pm$ 6.76 |  57.28 $\pm$ 5.95 |  57.39 $\pm$ 6.76
MetaSGD |  56.89 $\pm$ 5.79 |  57.11 $\pm$ 6.68 |  57.39 $\pm$ 6.91 |  58.33 $\pm$ 4.89
MetaSpecGraph | $-$ |  58.64 $\pm$ 6.11 |  58.83 $\pm$ 5.48 |  60.59 $\pm$ 5.89
MatchNet + Our Encoder | 62.11 $\pm$ 6.14 |  62.28 $\pm$ 6.06 |  62.52 $\pm$ 6.03 |  62.87 $\pm$ 5.98
ProtoNet + Our Encoder |  61.33 $\pm$ 6.91 |  61.67 $\pm$ 5.99 |  61.83 $\pm$ 5.14 |  62.44 $\pm$ 5.97
RelationNet + Our Encoder | 62.11 $\pm$ 5.05 | 62.50 $\pm$ 4.13 | 62.83 $\pm$ 6.82 | 63.16 $\pm$ 5.16
| MAML + Our Encoder |  58.39 $\pm$ 0.68 |  59.00 $\pm$ 1.16 |  59.56 $\pm$ 1.40 |  60.22 $\pm$ 1.08
| MetaSGD + Our Encoder |  58.67 $\pm$ 1.42 |  59.28 $\pm$ 1.27 |  59.39 $\pm$ 0.86 |  60.56 $\pm$ 1.20
Table 9: Mean and Standard Deviation of meta-test accuracy on Molecules benchmark after ten runs. method | view | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---|---
MatchNet | X |  58.89 $\pm$ 6.31 |  59.33 $\pm$ 5.71 |  59.50 $\pm$ 6.83 |  60.06 $\pm$ 6.87
ProtoNet | X |  58.72 $\pm$ 6.75 |  59.11 $\pm$ 6.80 |  59.28 $\pm$ 4.89 |  59.44 $\pm$ 6.13
RelationNet | X |  58.78 $\pm$ 7.73 |  59.06 $\pm$ 6.96 |  59.17 $\pm$ 6.96 |  60.11 $\pm$ 5.02
MAML | X |  56.00 $\pm$ 6.71 |  56.78 $\pm$ 6.76 |  57.28 $\pm$ 5.95 |  57.39 $\pm$ 6.76
MetaSGD | X |  56.89 $\pm$ 5.79 |  57.11 $\pm$ 6.68 |  57.39 $\pm$ 6.91 |  58.33 $\pm$ 4.89
MatchNet | X, U |  60.15 $\pm$ 6.35 |  60.37 $\pm$ 6.70 |  60.73 $\pm$ 5.62 |  61.42 $\pm$ 6.25
ProtoNet | X, U |  59.89 $\pm$ 6.46 |  60.14 $\pm$ 5.85 |  60.28 $\pm$ 5.55 |  60.63 $\pm$ 5.41
RelationNet | X, U |  60.93 $\pm$ 6.67 |  61.06 $\pm$ 5.23 |  61.19 $\pm$ 4.84 |  61.41 $\pm$ 6.07
MAML | X, U |  57.12 $\pm$ 5.89 |  57.48 $\pm$ 6.43 |  57.98 $\pm$ 5.89 |  58.12 $\pm$ 6.34
MetaSGD | X, U |  57.64 $\pm$ 6.56 |  57.72 $\pm$ 6.47 |  58.26 $\pm$ 5.97 |  58.65 $\pm$ 6.67
MatchNet | X, Z |  61.47 $\pm$ 7.82 |  60.68 $\pm$ 6.02 |  61.11 $\pm$ 5.81 |  61.76 $\pm$ 6.12
ProtoNet | X, Z |  60.72 $\pm$ 6.65 |  60.94 $\pm$ 7.35 |  61.28 $\pm$ 5.78 |  61.64 $\pm$ 4.42
RelationNet | X, Z |  61.88 $\pm$ 5.74 |  61.97 $\pm$ 7.75 |  52.02 $\pm$ 7.82 |  62.11 $\pm$ 5.21
MAML | X, Z |  57.85 $\pm$ 6.31 |  58.13 $\pm$ 6.48 |  58.24 $\pm$ 5.91 |  59.09 $\pm$ 5.62
MetaSGD | X, Z |  57.91 $\pm$ 7.11 |  58.11 $\pm$ 7.66 |  58.35 $\pm$ 6.88 |  58.92 $\pm$ 5.48
MatchNet | X, U, Z | 62.17 $\pm$ 6.14 |  61.50 $\pm$ 6.06 |  61.94 $\pm$ 6.03 |  62.33 $\pm$ 5.98
ProtoNet | X, U, Z |  61.33 $\pm$ 6.91 |  61.67 $\pm$ 5.99 |  61.83 $\pm$ 5.14 |  62.44 $\pm$ 5.97
RelationNet | X, U, Z |  62.11 $\pm$ 5.05 | 62.50 $\pm$ 4.13 | 62.83 $\pm$ 6.82 | 63.16 $\pm$ 5.16
MAML | X, U, Z |  58.39 $\pm$ 6.68 |  59.00 $\pm$ 6.16 |  59.56 $\pm$ 6.40 |  60.22 $\pm$ 6.08
MetaSGD | X, U, Z |  58.67 $\pm$ 5.42 |  59.28 $\pm$ 6.27 |  59.39 $\pm$ 6.86 |  60.56 $\pm$ 6.20
Table 10: Mean and Standard Deviation of meta-test accuracy on Bioinformatics benchmark after ten runs. method | view | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---|---
MatchNet | X |  54.83 $\pm$ 7.66 |  55.62 $\pm$ 7.60 |  55.92 $\pm$ 6.67 |  56.04 $\pm$ 7.78
ProtoNet | X |  54.71 $\pm$ 8.86 |  55.75 $\pm$ 7.84 |  55.96 $\pm$ 6.73 |  55.50 $\pm$ 9.65
RelationNet | X |  54.93 $\pm$ 8.55 |  55.92 $\pm$ 8.69 |  56.02 $\pm$ 7.69 |  56.15 $\pm$ 7.81
MAML | X |  53.83 $\pm$ 9.62 |  54.46 $\pm$ 6.77 |  54.50 $\pm$ 8.77 |  54.79 $\pm$ 8.90
MetaSGD | X |  53.83 $\pm$ 8.79 |  54.21 $\pm$ 7.70 |  54.67 $\pm$ 9.72 |  54.71 $\pm$ 7.90
MatchNet | X, U |  56.72 $\pm$ 8.11 |  56.44 $\pm$ 8.39 |  56.89 $\pm$ 6.39 |  57.06 $\pm$ 8.50
ProtoNet | X, U |  56.39 $\pm$ 8.61 |  56.06 $\pm$ 7.50 |  56.50 $\pm$ 7.83 |  56.33 $\pm$ 6.33
RelationNet | X, U |  56.83 $\pm$ 8.11 |  56.94 $\pm$ 9.83 |  56.50 $\pm$ 7.56 |  56.94 $\pm$ 8.33
MAML | X, U |  55.62 $\pm$ 8.08 |  55.17 $\pm$ 8.50 |  55.87 $\pm$ 9.83 |  55.92 $\pm$ 7.83
MetaSGD | X, U |  54.75 $\pm$ 7.92 |  55.50 $\pm$ 7.83 |  55.21 $\pm$ 7.29 |  55.58 $\pm$ 7.62
MatchNet | X, Z |  57.46 $\pm$ 6.29 |  57.08 $\pm$ 7.60 |  56.62 $\pm$ 7.67 |  57.14 $\pm$ 7.87
ProtoNet | X, Z |  56.68 $\pm$ 7.11 |  56.87 $\pm$ 6.56 |  57.35 $\pm$ 7.73 |  57.47 $\pm$ 8.46
RelationNet | X, Z |  56.98 $\pm$ 8.34 |  57.77 $\pm$ 8.34 |  57.13 $\pm$ 8.76 |  58.25 $\pm$ 7.61
MAML | X, Z |  55.47 $\pm$ 7.67 |  55.79 $\pm$ 7.67 |  55.40 $\pm$ 8.71 |  56.72 $\pm$ 7.87
MetaSGD | X, Z |  54.63 $\pm$ 9.21 |  55.46 $\pm$ 8.50 |  56.47 $\pm$ 8.79 |  56.77 $\pm$ 8.96
MatchNet | X, U, Z | 59.12 $\pm$ 7.00 |  58.67 $\pm$ 9.77 |  57.71 $\pm$ 8.72 |  57.62 $\pm$ 6.97
ProtoNet | X, U, Z |  57.17 $\pm$ 7.76 |  57.58 $\pm$ 8.89 |  57.79 $\pm$ 8.76 |  58.17 $\pm$ 7.88
RelationNet | X, U, Z |  58.83 $\pm$ 8.03 | 58.83 $\pm$ 9.68 | 59.29 $\pm$ 7.87 | 59.82 $\pm$ 7.93
MAML | X, U, Z |  56.00 $\pm$ 8.74 |  56.21 $\pm$ 7.76 |  56.37 $\pm$ 8.81 |  57.04 $\pm$ 7.85
MetaSGD | X, U, Z |  55.08 $\pm$ 8.67 |  56.12 $\pm$ 8.23 |  57.08 $\pm$ 8.86 |  57.33 $\pm$ 8.86
Table 11: Mean and Standard Deviation of meta-test accuracy on Social Networks benchmark after ten runs. method | view | 1-shot | 5-shot | 10-shot | 20-shot
---|---|---|---|---|---
MatchNet | X |  62.25 $\pm$ 9.14 |  62.42 $\pm$ 9.98 |  62.92 $\pm$ 9.22 |  63.33 $\pm$ 9.92
ProtoNet | X |  60.50 $\pm$ 9.05 |  61.25 $\pm$ 10.01 |  61.75 $\pm$ 9.06 |  63.50 $\pm$ 9.97
RelationNet | X |  61.25 $\pm$ 10.04 |  61.08 $\pm$ 10.25 |  61.83 $\pm$ 10.17 |  62.00 $\pm$ 10.17
MAML | X |  58.33 $\pm$ 9.05 |  58.75 $\pm$ 10.84 |  59.00 $\pm$ 9.87 |  60.17 $\pm$ 10.99
MetaSGD | X |  59.25 $\pm$ 10.15 |  59.33 $\pm$ 9.50 |  59.83 $\pm$ 9.89 |  60.25 $\pm$ 9.88
MatchNet | X, U |  65.31 $\pm$ 10.11 |  65.42 $\pm$ 10.82 |  65.92 $\pm$ 10.13 |  66.43 $\pm$ 9.89
ProtoNet | X, U |  65.14 $\pm$ 10.21 |  65.37 $\pm$ 9.12 |  65.88 $\pm$ 9.14 |  66.11 $\pm$ 10.95
RelationNet | X, U |  65.44 $\pm$ 9.26 |  65.74 $\pm$ 9.18 |  65.91 $\pm$ 9.18 |  66.66 $\pm$ 10.83
MAML | X, U |  61.14 $\pm$ 9.17 |  61.68 $\pm$ 10.74 |  62.41 $\pm$ 10.74 |  63.68 $\pm$ 10.41
MetaSGD | X, U |  61.21 $\pm$ 10.08 |  61.92 $\pm$ 10.41 |  62.58 $\pm$ 9.71 |  63.11 $\pm$ 10.81
MatchNet | X, Z |  66.44 $\pm$ 10.34 |  66.81 $\pm$ 10.85 |  67.12 $\pm$ 10.33 |  67.39 $\pm$ 9.88
ProtoNet | X, Z |  65.83 $\pm$ 9.53 |  65.98 $\pm$ 10.07 |  66.05 $\pm$ 9.13 |  66.48 $\pm$ 10.99
RelationNet | X, Z |  66.02 $\pm$ 10.31 |  66.29 $\pm$ 9.31 |  66.81 $\pm$ 10.21 |  67.33 $\pm$ 9.12
MAML | X, Z |  61.52 $\pm$ 9.21 |  62.98 $\pm$ 10.46 |  64.04 $\pm$ 10.94 |  64.89 $\pm$ 9.91
MetaSGD | X, Z |  61.36 $\pm$ 9.13 |  62.84 $\pm$ 10.63 |  63.77 $\pm$ 10.83 |  63.96 $\pm$ 9.89
MatchNet | X, U, Z | 67.42 $\pm$ 10.37 | 67.50 $\pm$ 9.24 | 68.08 $\pm$ 9.09 |  68.25 $\pm$ 9.18
ProtoNet | X, U, Z |  66.42 $\pm$ 10.21 |  66.92 $\pm$ 10.55 |  67.00 $\pm$ 9.13 |  67.50 $\pm$ 10.46
RelationNet | X, U, Z |  66.75 $\pm$ 9.89 |  67.08 $\pm$ 10.80 |  67.33 $\pm$ 10.33 | 69.67 $\pm$ 10.05
MAML | X, U, Z |  62.25 $\pm$ 10.12 |  64.75 $\pm$ 10.79 |  66.58 $\pm$ 10.93 |  66.83 $\pm$ 10.21
MetaSGD | X, U, Z |  61.92$\pm$ 9.56 |  63.17 $\pm$ 10.51 |  64.42 $\pm$ 10.16 |  64.83 $\pm$ 10.06