11institutetext:  ♯Dept. of Computer Science & Engineering, IIT Kanpur, India
<EMAIL_ADDRESS>

# A Simple Exponential Family Framework
for Zero-Shot Learning

Vinay Kumar Verma♯ Piyush Rai♯

###### Abstract

We present a simple generative framework for learning to predict previously
unseen classes, based on estimating _class-attribute-gated_ class-conditional
distributions. We model each class-conditional distribution as an exponential
family distribution and the parameters of the distribution of each seen/unseen
class are defined as functions of the respective observed class attributes.
These functions can be learned using only the seen class data and can be used
to predict the parameters of the class-conditional distribution of each unseen
class. Unlike most existing methods for zero-shot learning that represent
classes as fixed embeddings in some vector space, our generative model
naturally represents each class as a probability distribution. It is simple to
implement and also allows leveraging additional unlabeled data from unseen
classes to improve the estimates of their class-conditional distributions
using transductive/semi-supervised learning. Moreover, it extends seamlessly
to few-shot learning by easily updating these distributions when provided with
a small number of additional labelled examples from unseen classes. Through a
comprehensive set of experiments on several benchmark data sets, we
demonstrate the efficacy of our framework111Code & data are available:
https://github.com/vkverma01/Zero-Shot/.

##  1 Introduction

The problem of learning to predict unseen classes, also popularly known as
Zero-Shot Learning (ZSL), is an important learning paradigm which refers to
the problem of recognizing objects from classes that were not seen at training
time [13, 26]. ZSL is especially relevant for learning “in-the-wild”
scenarios, where new concepts need to be discovered on-the-fly, without having
access to labelled data from the novel classes/concepts. This has led to a
tremendous amount of interest in developing ZSL methods that can learn in a
robust and scalable manner, even when the amount of supervision for the
classes of interest is relatively scarce.

A large body of existing prior work for ZSL is based on embedding the data
into a semantic vector space, where distance based methods can be applied to
find the most likely class which itself is represented as a point in the same
semantic space [26, 20, 33]. However, a limitation of these methods is that
each class is represented as a fixed point in the embedding space which does
not adequately account for intra-class variability [2, 18]. We provide a more
detailed overview of existing work on ZSL in the Related Work section.

Another key limitation of most of the existing methods is that they usually
lack a proper generative model of the data. Having a generative model has
several advantages [19]. For example, (1) data of different types can be
modeled in a principled way using appropriately chosen class-conditional
distributions; (2) unlabeled data can be seamlessly integrated (for both seen
as well as unseen classes) during parameter estimation, leading to a
transductive/semi-supervised estimation procedure, which may be useful when
the amount of labeled data for the seen classes is small, or if the
distributions of seen and unseen classes are different from each other [11];
and (3) a rich body of work, both frequentist and Bayesian, on learning
generative models [19] can be brought to bear during the ZSL parameter
estimation process.

Motivated by these desiderata, we present a generative framework for zero-shot
learning. Our framework is based on modelling the class-conditional
distributions of seen as well as unseen classes using exponential family
distributions [3], and further conditioning the parameters of these
distributions on the respective class-attribute vectors via a linear/nonlinear
regression model of one’s choice. The regression model allows us to predict
the parameters of the class-conditional distributions of _unseen_ classes
using only their class attributes, enabling us to perform zero-shot learning.

In addition to the generality and modelling flexibility of our framework,
another of its appealing aspects is its simplicity. In contrast with various
other state-of-the-art methods, our framework is very simple to implement and
easy to extend. In particular, as we will show, parameter estimation in our
framework simply reduces to solving a linear/nonlinear regression problem, for
which a closed-form solution exists. Moreover, extending our framework to
incorporate unlabeled data from the unseen classes, or a small number of
labelled examples from the unseen classes, i.e., performing few-shot learning
[23, 17] is also remarkably easy under our framework which models class-
conditional distributions using exponential family distributions with
conjugate priors.

##  2 A Generative Framework For ZSL

In zero-shot learning (ZSL) we assume there is a total of $S$ seen classes and
$U$ unseen classes. Labelled training examples are only available for the seen
classes. The test data is usually assumed to come only from the unseen
classes, although in our experiments, we will also evaluate our model for the
setting where the test data could come from both seen and unseen classes, a
setting known as generalised zero-shot learning [6].

We take a generative modeling approach to the ZSL problem and model the class-
conditional distribution for an observation $\boldsymbol{x}$ from a
seen/unseen class $c$ ($c=1,\ldots,S+U$) using an exponential family
distribution [3] with natural parameters $\boldsymbol{\theta}_{c}$

$p(\boldsymbol{x}|\boldsymbol{\theta}_{c})=h(\boldsymbol{x})\exp\left(\boldsymbol{\theta}_{c}^{\top}\phi(\boldsymbol{x})-A(\boldsymbol{\theta}_{c})\right)\vspace{0mm}$
(1)

where $\phi(\boldsymbol{x})$ denotes the sufficient statistics and
$A(\boldsymbol{\theta}_{c})$ denotes the log-partition function. We also
assume that the distribution parameters $\boldsymbol{\theta}_{c}$ are given
conjugate priors

$p(\boldsymbol{\theta}_{c}|\boldsymbol{\tau}_{0},\boldsymbol{\nu}_{0})\propto\exp(\boldsymbol{\theta}_{c}^{\top}\boldsymbol{\tau}_{0}-\boldsymbol{\nu}_{0}A(\boldsymbol{\theta}_{c}))\vspace{0mm}$
(2)

Given a test example $\boldsymbol{x}_{*}$, its class $y_{*}$ can be predicted
by finding the class under which $\boldsymbol{x}_{*}$ is most likely (i.e.,
$y_{*}=\arg\max_{c}p(\boldsymbol{x}_{*}|\theta_{c})$), or finding the class
that has the largest posterior probability given $\boldsymbol{x}_{*}$ (i.e.,
$y_{*}=\arg\max_{c}p(\theta_{c}|\boldsymbol{x}_{*})$). However, doing this
requires first estimating the parameters
$\\{\boldsymbol{\theta}_{c}\\}_{c=S+1}^{S+U}$ of all the unseen classes.

Given labelled training data from any class modelled as an exponential family
distribution, it is straightforward to estimate the model parameters
$\boldsymbol{\theta}_{c}$ using maximum likelihood estimation (MLE), maximum-
a-posteriori (MAP) estimation, or using fully Bayesian inference [19].
However, since there are no labelled training examples from the unseen
classes, we cannot estimate the parameters
$\\{\boldsymbol{\theta}_{c}\\}_{c=S+1}^{S+U}$ of the class-conditional
distributions of the unseen classes.

To address this issue, we learn a model that allows us to predict the
parameters $\boldsymbol{\theta}_{c}$ for any class $c$ using the _attribute
vector_ of that class via a _gating_ scheme, which is basically defined as a
linear/nonlinear regression model from the attribute vector to the parameters.
As is the common practice in ZSL, the attribute vector of each class may be
derived from a human-provide description of the class or may be obtained from
an external source such as Wikipedia in form of word-embedding of each class.
We assume that the class-attribute of each class is a vector of size $K$. The
class-attribute of all the classes are denoted as
$\\{{\boldsymbol{a}}_{c}\\}_{c=1}^{S+U}$,
${\boldsymbol{a}}_{c}\in\mathbb{R}^{K}$.

###  2.1 Gating via Class-Attributes

We assume a regression model from the class-attribute vector
${\boldsymbol{a}}_{c}$ to the parameters $\boldsymbol{\theta}_{c}$ of each
class $c$. In particular, we assume that the class-attribute vector
${\boldsymbol{a}}_{c}$ is mapped via a function $f$ to generate the parameters
$\boldsymbol{\theta}_{c}$ of the class-conditional distribution of class $c$,
as follows

$\boldsymbol{\theta}_{c}=f_{\theta}({\boldsymbol{a}}_{c})\vspace{0mm}$ (3)

Note that the function $f_{\theta}$ itself could consist of multiple functions
if $\boldsymbol{\theta}_{c}$ consists of multiple parameters. For
concereteness, and also to simplify the rest of the exposition, we will focus
on the case when the class-conditional distribution is a $D$ dimensional
Gaussian, for which $\boldsymbol{\theta}_{c}$ is defined by the mean vector
$\boldsymbol{\mu}_{c}\in\mathbb{R}^{D}$ and a p.s.d. covariance matrix
$\boldsymbol{\Sigma}_{c}\in\mathcal{S}_{+}^{D\times D}$. Further, we will
assume $\boldsymbol{\Sigma}_{c}$ to be a diagonal matrix defined as
$\boldsymbol{\Sigma}_{c}=\text{diag}(\boldsymbol{\sigma}_{c}^{2})$ where
$\boldsymbol{\sigma}_{c}^{2}=[\sigma_{c1}^{2},\ldots,\sigma_{cD}^{2}]$. Note
that one can also assume a full covariance matrix but it will significantly
increase the number of parameters to be estimated. We model
$\boldsymbol{\mu}_{c}$ and $\boldsymbol{\sigma}_{c}^{2}$ as functions of the
attribute vector ${\boldsymbol{a}}_{c}$

$\displaystyle\boldsymbol{\mu}_{c}$ $\displaystyle=$ $\displaystyle
f_{\boldsymbol{\mu}}({\boldsymbol{a}}_{c})$ (4)
$\displaystyle\boldsymbol{\sigma}_{c}^{2}$ $\displaystyle=$ $\displaystyle
f_{\boldsymbol{\sigma}^{2}}({\boldsymbol{a}}_{c})\vspace{0mm}$ (5)

Note that the above equations define two regression models. The first
regression model defined by the function $f_{\boldsymbol{\mu}}$ has
${\boldsymbol{a}}_{c}$ as the input and $\boldsymbol{\mu}_{c}$ as the output.
The second regression model defined by $f_{\boldsymbol{\sigma}^{2}}$ has
${\boldsymbol{a}}_{c}$ as the input and $\boldsymbol{\sigma}^{2}$ as the
output. The goal is to learn the functions $f_{\boldsymbol{\mu}}$ and
$f_{\boldsymbol{\sigma}^{2}}$ from the available training data. Note that the
form of these functions is a modelling choice and can be chosen appropriately.
We will consider both linear as well as nonlinear functions.

###  2.2 Learning The Regression Functions

Using the available training data from all the seen classes $c=1,\ldots,S$, we
can form empirical estimates of the parameters
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=1}^{S}$
of respective class-conditional distributions using MLE/MAP estimation. Note
that, since our framework is generative, both labeled as well as unlabeled
data from the seen classes can be used to form the empirical estimates
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=1}^{S}$.
This makes our estimates of
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=1}^{S}$
reliable even if each seen class has very small number of labeled examples.
Given these estimates for the seen classes

$\displaystyle\hat{\boldsymbol{\mu}}_{c}$ $\displaystyle=$ $\displaystyle
f_{\boldsymbol{\mu}}({\boldsymbol{a}}_{c})\quad\quad\ c=1,\ldots,S$ (6)
$\displaystyle\hat{\boldsymbol{\sigma}}_{c}^{2}$ $\displaystyle=$
$\displaystyle f_{\boldsymbol{\sigma}^{2}}({\boldsymbol{a}}_{c})\quad\quad
c=1,\ldots,S\vspace{0mm}$ (7)

We can now learn $f_{\boldsymbol{\mu}}$ using “training” data
$\\{{\boldsymbol{a}}_{c},\hat{\boldsymbol{\mu}}_{c}\\}_{c=1}^{S}$ and learn
$f_{\boldsymbol{\sigma^{2}}}$ using training data
$\\{{\boldsymbol{a}}_{c},\hat{\boldsymbol{\sigma}^{2}}_{c}\\}_{c=1}^{S}$. We
consider both linear and nonlinear regression models for learning these.

####  2.2.1 The Linear Model

For the linear model, we assume $\hat{\boldsymbol{\mu}}_{c}$ and
$\hat{\boldsymbol{\sigma}}_{c}^{2}$ to be linear functions of the class-
attribute vector ${\boldsymbol{a}}_{c}$, defined as

$\displaystyle\hat{\boldsymbol{\mu}}_{c}$ $\displaystyle=$ $\displaystyle{{\bf
W}}_{\mu}{\boldsymbol{a}}_{c}\quad\quad\ c=1,\ldots,S$ (8)
$\displaystyle\hat{\boldsymbol{\rho}}_{c}=\log\hat{\boldsymbol{\sigma}}_{c}^{2}$
$\displaystyle=$ $\displaystyle{{\bf
W}}_{\sigma^{2}}{\boldsymbol{a}}_{c}\quad\quad c=1,\ldots,S\vspace{0mm}$ (9)

where the regression weights ${{\bf W}}_{\mu}\in\mathbb{R}^{D\times K}$,
${{\bf W}}_{\sigma^{2}}\in\mathbb{R}^{D\times K}$, and we have re-
parameterized $\hat{\boldsymbol{\sigma}}_{c}^{2}\in\mathbb{R}_{+}^{D}$ to
$\hat{\boldsymbol{\rho}}_{c}\in\mathbb{R}^{D}$ as
$\hat{\boldsymbol{\rho}}_{c}=\log\hat{\boldsymbol{\sigma}}_{c}^{2}$.

We use this re-parameterization to map the output space of the second
regression model $f_{\boldsymbol{\sigma}^{2}}$ (defined by ${{\bf
W}}_{\sigma^{2}}$) to real-valued vectors, so that a standard regression model
can be applied (note that $\hat{\boldsymbol{\sigma}}_{c}^{2}$ is positive-
valued vector).

Estimating Regression Weights of Linear Model: We will denote ${\bf
M}=[\hat{\boldsymbol{\mu}}_{1},\ldots,\hat{\boldsymbol{\mu}}_{S}]\in\mathbb{R}^{D\times
S}$, ${{\bf
R}}=[\hat{\boldsymbol{\rho}}_{1},\ldots,\hat{\boldsymbol{\rho}}_{S}]\in\mathbb{R}^{D\times
S}$, and ${{\bf
A}}=[{\boldsymbol{a}}_{1},\ldots,{\boldsymbol{a}}_{S}]\in\mathbb{R}^{K\times
S}$. We can then write the estimation of the regression weights ${{\bf
W}}_{\mu}$ as the following problem

$\hat{{{\bf W}}}_{\mu}=\arg\min_{{{\bf W}}_{\mu}}||{\bf M}-{{\bf
W}}_{\mu}{{\bf A}}||_{2}^{2}+\lambda_{\mu}||{{\bf
W}}_{\mu}||_{2}^{2}\vspace{0mm}$ (10)

This is essentially a multi-output regression [7] problem ${{\bf
W}}_{\mu}:{\boldsymbol{a}}_{s}\mapsto\hat{\boldsymbol{\mu}}_{s}$ with least
squares loss and an $\ell_{2}$ regularizer. The solution to this problem is
given by

$\hat{{{\bf W}}}_{\mu}={\bf M}{{\bf A}}^{\top}({{\bf A}}{{\bf
A}}^{\top}+\lambda_{\mu}{\bf I}_{K})^{-1}\vspace{0mm}$ (11)

Likewise, we can then write the estimation of the regression weights ${{\bf
W}}_{\sigma^{2}}$ as the following problem

$\hat{{{\bf W}}}_{\sigma^{2}}=\arg\min_{{{\bf W}}_{\sigma^{2}}}||{{\bf
R}}-{{\bf W}}_{\sigma^{2}}{{\bf A}}||_{2}^{2}+\lambda_{\sigma^{2}}||{{\bf
W}}_{\sigma^{2}}||_{2}^{2}\vspace{0mm}$ (12)

The solution of the above problem is given by

$\hat{{{\bf W}}}_{\sigma^{2}}={{\bf R}}{{\bf A}}^{\top}({{\bf A}}{{\bf
A}}^{\top}+\lambda_{\sigma^{2}}{\bf I}_{K})^{-1}\vspace{0mm}$ (13)

Given $\hat{{{\bf W}}}_{\mu}$ and $\hat{{{\bf W}}}_{\sigma^{2}}$, parameters
of the class-conditional distribution of each unseen class $c=S+1,\ldots,S+U$
can be easily computed as follows

$\displaystyle\hat{\boldsymbol{\mu}}_{c}$ $\displaystyle=$
$\displaystyle\hat{{{\bf W}}}_{\mu}{\boldsymbol{a}}_{c}$ (14)
$\displaystyle\hat{\boldsymbol{\sigma}}_{c}^{2}$ $\displaystyle=$
$\displaystyle\exp(\hat{\boldsymbol{\rho}}_{c})=\exp(\hat{{{\bf
W}}}_{\sigma^{2}}{\boldsymbol{a}}_{c})\vspace{-0.5em}\vspace{0mm}$ (15)

####  2.2.2 The Nonlinear Model

For the nonlinear case, we assume that the inputs
$\\{{\boldsymbol{a}}_{c}\\}_{c=1}^{S}$ are mapped to a kernel induced space
via a kernel function $k$ with an associated nonlinear mapping $\phi$. In this
case, using the representer theorem [24], the solution for the two regression
models $f_{\boldsymbol{\mu}}$ and $f_{\boldsymbol{\sigma}^{2}}$ can be written
as the spans of the inputs $\\{\phi({\boldsymbol{a}}_{c})\\}_{c=1}^{S}$. Note
that mappings $\phi({\boldsymbol{a}}_{c})$ do not have to be computed
explicitly since learning the nonlinear regression model only requires dot
products
$\phi({\boldsymbol{a}}_{c})^{\top}\phi({\boldsymbol{a}}_{c^{\prime}})=k({\boldsymbol{a}}_{c},{\boldsymbol{a}}_{c^{\prime}})$
between the nonlinear mappings of two classes $c$ and $c^{\prime}$.

Estimating Regression Weights of Nonlinear Model: Denoting ${{\bf K}}$ to be
the $S\times S$ kernel matrix of the pairwise similarities of the attributes
of the seen classes, the nonlinear model $f_{\boldsymbol{\mu}}$ is obtained by

$\hat{\boldsymbol{\alpha}}_{\mu}=\arg\min_{\boldsymbol{\alpha}_{\mu}}||{\bf
M}-\boldsymbol{\alpha}_{\mu}{{\bf
K}}||_{2}^{2}+\lambda_{\mu}||\boldsymbol{\alpha}_{\mu}||_{2}^{2}\vspace{0mm}$
(16)

where $\hat{\boldsymbol{\alpha}}_{\mu}$ is a $D\times S$ matrix consists of
the coefficients of the span of $\\{\phi({\boldsymbol{a}}_{c})\\}_{c=1}^{S}$
defining the nonlinear function $f_{\boldsymbol{\mu}}$.

Note that the problem in Equation 16 is essentially a multi-output _kernel_
ridge regression [7] problem, which has a closed form solution. The solution
for $\hat{\boldsymbol{\alpha}}_{\mu}$ is given by

$\hat{\boldsymbol{\alpha}}_{\mu}={\bf M}({{\bf K}}+\lambda_{\mu}{\bf
I}_{S})^{-1}\vspace{0mm}$ (17)

Likewise, the nonlinear model $f_{\boldsymbol{\sigma}^{2}}$ is obtained by
solving

$\hat{\boldsymbol{\alpha}}_{\sigma^{2}}=\arg\min_{\boldsymbol{\alpha}_{\sigma^{2}}}||{\bf
M}-\boldsymbol{\alpha}_{\sigma^{2}}{{\bf
K}}||_{2}^{2}+\lambda_{\sigma^{2}}||\boldsymbol{\alpha}_{\sigma^{2}}||_{2}^{2}\vspace{0mm}$
(18)

where $\hat{\boldsymbol{\alpha}}_{\sigma^{2}}$ is a $D\times S$ matrix
consists of the coefficients of the span of
$\\{\phi({\boldsymbol{a}}_{c})\\}_{c=1}^{S}$ defining the nonlinear function
$f_{\boldsymbol{\sigma^{2}}}$. The solution for
$\hat{\boldsymbol{\alpha}}_{\sigma^{2}}$ is given by

$\hat{\boldsymbol{\alpha}}_{\sigma^{2}}={{\bf R}}({{\bf K}}+\lambda_{\mu}{\bf
I}_{S})^{-1}\vspace{0mm}$ (19)

Given $\hat{\boldsymbol{\alpha}}_{\mu}$,
$\hat{\boldsymbol{\alpha}}_{\sigma^{2}}$, parameters of class-conditional
distribution of each unseen class $c=S+1,\ldots,S+U$ will be

$\displaystyle\hat{\boldsymbol{\mu}}_{c}$ $\displaystyle=$
$\displaystyle\hat{\boldsymbol{\alpha}}_{\mu}{\boldsymbol{k}}_{c}$ (20)
$\displaystyle\hat{\boldsymbol{\sigma}}_{c}^{2}$ $\displaystyle=$
$\displaystyle\exp(\hat{\boldsymbol{\rho}}_{c})=\exp(\hat{\boldsymbol{\alpha}}_{\sigma^{2}}{\boldsymbol{k}}_{c})\vspace{0mm}$
(21)

where
${\boldsymbol{k}}_{c}=[k({\boldsymbol{a}}_{c},{\boldsymbol{a}}_{1}),\ldots,k({\boldsymbol{a}}_{c},{\boldsymbol{a}}_{S})]^{\top}$
denotes an $S\times 1$ vector of kernel-based similarities of the class-
attribute of unseen class $c$ with the class-attributes of all the seen
classes.

####  2.2.3 Other Exponential Family Distributions

Although we illustrated our framework taking the example of Gaussian class-
conditional distributions, our framework readily generalizes to the case when
these distributions are modelled using any exponential family distribution.
The estimation problems can be solved in a similar way as the Gaussian case
with the basic recipe remaining the same: Form empirical estimates of the
parameters $\boldsymbol{\Theta}=\\{\hat{\boldsymbol{\theta}}_{c}\\}_{c=1}^{S}$
for the seen classes using all the available seen class data and then learn a
linear/nonlinear regression model from the class-attributes ${{\bf A}}$ (or
their kernel representation ${{\bf K}}$ in the nonlinear case) to
$\boldsymbol{\Theta}$.

In additional to its modeling flexibility, an especially remarkable aspect of
our generative framework is that it is very easy to implement, since both the
linear model as well as the nonlinear model have closed-form solutions given
by Eq. 11 and Eq. 13, and Eq. 17 and Eq. 19, respectively (the solutions will
be available in similar closed-forms in the case of other exponential family
distributions). A block-diagram describing our framework is shown in Figure 1.
Note that another appealing aspect of our framework is its modular
architecture where each of the blocks in Figure 1 can make use of a suitable
method of one’s choice.

Figure 1: Block-diagram of our framework. $\mathcal{D}_{s}$ denotes the seen
class data (can be labeled (and optionally also unlabeled); ${{\bf A}}_{s}$
denotes seen class attributes; ${{\bf A}}_{u}$ denotes unseen class
attributes; $\boldsymbol{\hat{\Theta}}_{s}$ denotes the estimated seen class
parameters; $\boldsymbol{\hat{\Theta}}_{u}$ denotes the estimated unseen class
parameters. The last stage - transductive/few-shot refinement - is optional
(Section 2.3 and 4.2)

###  2.3 Transductive/Semi-Supervised Setting

The procedure described in Section 2.2 relies only on the seen class data
(labeled and, optionally, also unlabeled). As we saw for the Gaussian case,
the seen class data is used to form empirical estimates of the parameters
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=1}^{S}$
of the class-conditional distributions of seen classes, and then these
estimates are used to learn the linear/nonlinear regression functions
$f_{\boldsymbol{\mu}}$ and $f_{\boldsymbol{\sigma}^{2}}$. These functions are
finally used to compute the parameters
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
of class-conditionals of unseen classes. We call this setting the _inductive_
setting. Note that this procedure does not make use of any data from the
unseen classes. Sometimes, we may have access to unlabeled data from the
unseen classes.

Our generative framework makes it easy to leverage such _unlabeled_ data from
the _unseen_ classes to further improve upon the estimates
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
of their class-conditional distributions. In our framework, this can be done
in two settings, _transductive_ and _semi-supervised_ , both of which leverage
unlabeled data from unseen classes, but in slightly different ways. If the
unlabeled data is the unseen class test data itself, we call it the
_transductive_ setting. If this unlabeled data from the unseen classes is
different from the actual unseen class test data, we call it the _semi-
supervised_ setting.

In either setting, we can use an Expectation-Maximization (EM) based procedure
that alternates between inferring the labels of unlabeled examples of unseen
classes and using the inferred labels to update the estimates of the
parameters
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
of the distributions of unseen classes.

For the case when each class-conditional distribution is a Gaussian, this
procedure is equivalent to estimating a Gaussian Mixture Model (GMM) using the
unlabeled data $\\{\boldsymbol{x}_{n}\\}_{n=1}^{N_{u}}$ from the unseen
classes. The GMM is initialized using the estimates
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
obtained from the inductive procedure of Section 2.2. Note that each of the
$U$ mixture components of this GMM corresonds to an unseen class.
The EM algorithm for the Gaussian case is summarized next

  1. 1.

Initialize mixing proportions $\boldsymbol{\pi}=[\pi_{1},\ldots,\pi_{U}]$
uniformly set mixture parameters as
$\boldsymbol{\Theta}=\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$

  2. 2.

E Step: Infer the probabilities for each $\boldsymbol{x}_{n}$ belonging to
each of the unseen classes $c=S+1,\ldots,S+U$ as

$p(y_{n}=c|\boldsymbol{x}_{n},\pi,\boldsymbol{\Theta})=\frac{\pi_{c}\mathcal{N}(\boldsymbol{x}_{n}|\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2})}{\sum_{c}\pi_{c}\mathcal{N}(\boldsymbol{x}_{n}|\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2})}\vspace{-0.5em}$

  3. 3.

M Step: Use to inferred class labels to re-estimate $\boldsymbol{\pi}$ and
$\boldsymbol{\Theta}=\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$.

  4. 4.

Go to step 2 if not converged.

Note that the same procedure can be applied even when each class-conditional
distribution is some exponential family distribution other than Gaussian. The
E and M steps in the resulting mixture model are straightforward in that case
as well. The E step will simply require the Gausian likelihood to be replaced
by the corresponding exponential family distribution’s likelihood. The M step
will require doing MLE of the exponential family distribution’s parameters,
which has closed-form solutions.

###  2.4 Extension for Few-Shot Learning

In few-shot learning, we assume that a very small number of labeled examples
may also be available for the unseen classes [23, 17]. The generative aspect
of our framework, along with the fact the the data distribution is an
exponential family distribution with a conjugate prior on its parameters,
makes it very convenient for our model to be extended to this setting. The
outputs
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
of our generative zero-shot learning model can naturally serve as the hyper-
parameters of a conjugate prior on parameters of class-conditional
distributions of unseen classes, which can then be updated given a small
number of labeled examples from the unseen classes. For example, in the
Gaussian case, due to conjugacy, we are able to update the estimates
$\\{\hat{\boldsymbol{\mu}}_{c},\hat{\boldsymbol{\sigma}}_{c}^{2}\\}_{c=S+1}^{S+U}$
in a straightforward manner when provided with such labeled data. In
particular, given a small number of labeled examples
$\\{\boldsymbol{x}_{n}\\}_{n=1}^{N_{c}}$ from an unseen class $c$,
$\hat{\boldsymbol{\mu}}_{c}$ and $\hat{\boldsymbol{\sigma}}_{c}^{2}$ can be
easily updated as

$\displaystyle\boldsymbol{\mu}_{c}^{(FS)}$ $\displaystyle=$
$\displaystyle\frac{\hat{\boldsymbol{\mu}}_{c}+\sum_{n=1}^{N_{c}}\boldsymbol{x}_{n}}{1+N_{c}}$
(22) $\displaystyle{\boldsymbol{\sigma}_{c}^{2}}^{(FS)}$ $\displaystyle=$
$\displaystyle\left(\frac{1}{\hat{\boldsymbol{\sigma}}_{c}^{2}}+\frac{N_{c}}{\sigma^{2}}\right)^{-1}\vspace{0mm}$
(23)

where
$\sigma^{2}=\frac{1}{N_{c}}\sum_{n=1}^{N_{c}}(\boldsymbol{x}_{n}-\hat{\boldsymbol{\mu}}_{c})^{2}$
denotes the empirical variance of the $N_{c}$ observations from the unseen
class $c$.

A particularly appealing aspect of our few-shot learning model outlined above
is that it can also be updated in an online manner as more and more labelled
examples become available from the unseen classes, without having to re-train
the model from scratch using all the data.

##  3 Related Work

Some of the earliest works on ZSL are based on predicting attributes for each
example [13]. This was followed by a related line of work based on models that
assume that the data from each class can be mapped to the class-attribute
space (a shared semantic space) in which each seen/unseen class is also
represented as a point [26, 1, 33]. The mapping can be learned using various
ways, such as linear models or feed forward neural networks or convolutional
neural networks. Predicting the label for a novel unseen class example then
involves mapping it to this space and finding the “closest” unseen class. Some
of the work on ZSL is aimed at improving the semantic embeddings of
concepts/classes. For example, [29] proposed a ZSL model to incorporate
relational information about concepts. In another recent work, [4] proposed a
model to improve the semantic embeddings using a metric learning formulation.
A complementary line of work to the semantic embedding methods is based on a
“reverse” mapping, i.e., mapping the class-attribute to the observed feature
space [32, 37].

In contrast to such semantic embedding methods that assume that the classes
are collapsed onto a single point, our framework offers considerably more
flexibility by modelling each class using its own distribution. This makes our
model more suitable for capturing the intra-class variability, which the
simple point-based embedding models are incapable of handling.

Another popular approach for ZSL is based on modelling each unseen class as a
linear/convex combination of seen classes [20] or of a set of “abstract” or
“basis” classes [22, 5]. The latter class of methods, in particular, can be
seen as a special case of our framework since, for our linear model, we can
view the columns of the $D\times K$ regression weights as representing a set
of $K$ basis classes. Note however that our model has such regression weights
for each parameter of the class-conditional distribution, allowing it to be
considerably more flexible. Moreover, our framework is also significantly
different in other ways due to its fully generative framework, due to its
ability to incorporate unlabeled data, performing few-shot learning, and its
ability to model different types of data using an appropriate exponential
family distribution.

A very important issue in ZSL is the _domain shift_ problem which may arise if
the seen and unseen class come from very different domains. In these
situations, standard ZSL models tend to perform badly. This can be somewhat
alleviated using some additional unlabeled data from the unseen classes. To
this end, [11] provide a dictionary learning based approach for learning
unseen class classifiers in which the dictionary is adapted to the unseen
class domain. The dictionary adaptation is facilitated using unlabeled data
from the unseen classes. In another related work, [8] leverage unlabeled data
in a transductive ZSL framework to handle the domain shift problem. Note that
our framework is robust to the domain shift problem due to its ability to
incorporate unlabeled data from the unseen classes (the transductive setting).
Our experimental results corroborate this.

Semi-supervised learning for ZSL can also be used to improve the semantic
embedding based methods. [16] provide a semi-supervised method that leverages
prior knowledge for improving the learned embeddings. In another recent work,
[37] present a model to incorporate unlabeled unseen class data in a setting
where each unseen class is represented as a linear combination of seen
classes. [34] provide another approach, motivated by applications in computer
vision, that jointly facilitates the domain adaptation of attribute space and
the visual space. Another semi-supervised approach presented in [15] combines
a semisupervised classification model over the observed classes with an
unsupervised clustering model over unseen classes together to address the
zero-shot multi-class classification.

In contrast to these models for which the mechanism for incorporating
unlabeled data is model-specific, our framework offers a general approach for
doing this, while also being simple to implement. Moreover, for large-scale
problems, it can also leverage more efficient solvers (e.g., gradient methods)
for estimating the regression coefficients associated with class-conditional
distributions.

##  4 Experiments

We evaluate our generative framework for zero-shot learning (hereafter
referred to as GFZSL) on several benchmark data sets and compare it with a
number of state-of-the-art baselines. We conduct our experiments on various
problem settings, including standard _inductive_ zero-shot learning (only
using seen class labeled examples), _transductive_ zero-shot learning (using
seen class labeled examples and unseen class unlabeled examples), and few-shot
learning (using seen class labeled examples and a very small number of unseen
class labeled examples). We report our experimental results on the following
benchmark data sets:

  * –

Animal with Attribute(AwA): The AwA data set contains 30475 images with 40
seen classes (training set) and 10 unseen classes (test set). Each class has a
human-provided binary/continuous 85-dimensional class-attribute vector [12].
We use continuous class-attributes since prior works have found these to have
more discriminative power.

  * –

Caltech-UCSD Birds-200-2011 (CUB-200): The CUB-200 data set contains 11788
images with 150 seen classes (training set) and 50 unseen class (test set).
Each image has a binary 312-dimensional class-attribute vector, specifying the
presence or absence of various attribute of that image [28]. The attribute
vectors for all images in a class are averaged to construct its continuous
class-attribute vector [2]. We use the same train/test split for this data set
as used in [2].

  * –

SUN attribute (SUN): The SUN data set contains 14340 images with 707 seen
classes (training set) and 10 unseen classes (test set). Each image is
described by a 102-dimensional binary class-attribute vector. Just like the
CUB-200 data set, we average the attribute vectors of all images in each class
to get its continuous attribute vector [10]. We use the same train/test split
for this data set as used in [10].

For image features, we considered both GoogleNet features [27] and
VGG-19(4096) fc7 features [25] and found that our approach works better with
VGG-19. All of the state-of-the-art baselines we compare with in our
experiments use VGG-19 fc7 features or GoogleNet features [27]. For the
nonlinear (kernel) variant of our model, we use a quadratic kernel. Our set of
experiments include:

  * –

Zero-Shot Learning: We consider both inductive ZSL as well as transductive
ZSL.

    * –

Inductive ZSL: This is the standard ZSL setting where the unseen class
parameters are learned using only seen class data.

    * –

Transductive ZSL: In this setting [34], we also use the unlabeled test data
while learning the unseen class parameters. Note that this setting has access
to more information about the unseen class; however, it is only through
unlabeled data.

  * –

Few-Shot Learning: In this setting [23, 17], we also use a small number of
labelled examples from each unseen class.

  * –

Generalized ZSL: Whereas standard ZSL (as well as few-shot learning) assumes
that the test data can only be from the unseen classes, generalized ZSL
assumes that the test data can be from unseen as well as seen classes. This is
usually a more challenging setting [6] and most of the existing methods are
known to be biased towards predicting the seen classes.

We use the standard train/test split as given in the data description section.
For selecting the hyperparameters, we further divide the train set further
into train and validation set. In our model, we have two hyper-parameter
$\lambda_{\mu}$ and $\lambda_{\sigma^{2}}$, which we tune using the validation
dataset. For AwA, from the 40 seen classes, a random selection of 30 classes
are used for the training set and 10 classes are used for the validation set.
For CUB-200, from the 150 seen classes, 100 are used for the training set and
rest 50 are used for the validation set. Similarly, for the SUN dataset from
the 707 seen classes, 697 are used for the training set and rest 10 is used
for the validation set. We use cross-validation on the validation set to
choose the best hyperparameter $[\lambda_{\mu},\lambda_{\sigma^{2}}]$ for the
each data set and use these for testing on the unseen classes.

###  4.1 Zero-Shot Learning

In our first set of experiments, we evaluate our model for zero-shot learning
and compare with a number of state-of-the-art methods, for the inductive
setting (which uses only the seen class labelled data) as well as the
transductive setting (which uses the seen class data and the unseen class
unlabeled data).

####  4.1.1 Inductive ZSL

Table-1 shows our results for the inductive ZSL setting. The results of the
various baselines are taken from the corresponding papers. As shown in the
Table-1, on CUB-200 and SUN, both of our models (linear and nonlinear) perform
better than all of the other state-of-the-art methods. On AwA, our model has
only a marginally lower test accuracy as compared to the best performing
baseline [34]. However, we also have an average improvement 5.67% on all the 3
data sets as compared to the overall best baseline [34]. Among baselines using
VGG-19 features (bottom half of Table-1), our model achieves a 21.05% relative
improvement over the best baseline on the CUB-200 data, which is considered to
be a difficult data set with many fine-grained classes.

Method |  AwA |  CUB-200 |  SUN |  Average
---|---|---|---|---
Akata et al. [2] |  66.70 |  50.1 |  – |  –
Qiao et al. [21] |  66.46$\pm$0.42 |  29$\pm$0.28 |  – |  –
Xian et al. [31] |  71.9 |  45.5 |  – |  –
Changpimyo et al.[5] |  72.9 |  54.7 |  62.7 |  63.43
Wang et al.[29] |  75.99 |  33.48 |  – |  –
Lampert et al.[14] |  57.23 |  – |  72.00 |  –
Romera and Torr[22] |  75.32$\pm$2.28 |  – |  82.10$\pm$ 0.32 |  –
Bucher et al.[4] |  77.32$\pm$1.03 |  43.29$\pm$0.38 |  84.41$\pm$0.71 |  68.34
Z. Zhang et al.[35] |  79.12$\pm$0.53 |  41.78$\pm$0.52 |  83.83$\pm$.29 |  68.24
Wang et al.[30] |  79.2$\pm$0.0 |  46.7$\pm$0.0 |  – |  –
Z. Zhang et al.[34] |  81.03$\pm$0.88 |  46.48$\pm$1.67 |  84.10$\pm$1.51 |  70.53
GFZSL: Linear |  79.90 |  52.09 |  86.50 |  72.23
GFZSL: Nonlinear |  80.83 |  56.53 |  86.50 |  74.59
Table 1: Accuracy(%) of different type of images features. Top: Deep features
like AlexNet, GoogleNet, etc. Bottom: Deep VGG-19 features. The ’-’ indicates
that this result was not reported.

In contrast to other models that embed the test examples in the semantic space
and then find the most similar class by doing a Euclidean distance based
nearest neighbor search, or models that are based on computing the similarity
scores between seen and unseen classes [33], for our models, finding the “most
probable class” corresponds to computing the distance of each test example
from a _distribution_. This naturally takes into account the shape and spread
of the class-conditional distribution. This explains the favourable
performance of our model as compared to the other methods.

####  4.1.2 Transductive Setting

For transductive ZSL setting [9, 35, 36], we follow the procedure described in
Section 2.3 to estimate parameters of the class-conditional distribution of
each unseen class. After learning the parameters, we find the most probable
class for each test example by evaluating its probability under each unseen
class distribution and assign it to the class under which it has the largest
probability. Table-2 and 3 compare our results from the transductive setting
with other state-of-the-art baselines designed for the transductive setting.
In addition to accuracy, we also report precision and recall results of our
model and the other baselines (wherever available). As we can see from Table-2
and 3, both of our models (linear and kernel) outperform the other baselines
on all the 3 data sets. Also comparing with the inductive setting results
presented in Table-1, we observe that our generative framework is able to very
effectively leverage unlabeled data and significantly improve upon the results
of a purely inductive setting

Method | AwA | CUB-200 | SUN | Average
---|---|---|---|---
Guo et al.[9] | 78.47 | – | 82.00 | –
Romera et al.[22]\+ Zhang et al. [36] | 84.30 | – | 37.50 | –
Zhang et al.[35]+Zhang et al. [36] |  92.08$\pm$0.14 |  55.34$\pm$0.77 |  86.12$\pm$0.99 | 77.85
Zhang et al.[34]+Zhang et al. [36] |  88.04$\pm$0.69 |  55.81$\pm$1.37 |  85.35$\pm$1.56 | 76.40
GFZSL: Linear | 94.20 | 57.14 | 87.00 | 79.45
GFZSL: Kernel | 94.25 | 63.66 | 87.00 | 80.63
Table 2: ZSL accuracy(%) obtained in the transductive setting: results reported using the VGG-19 feature. Average Precision and recall for the all dataset with its standard daviation over the 100 iteration. The ’-’ indicates that this result was not reported in the original paper. | Average Precision | Average Recall
---|---|---
Method | AwA | CUB-200 | SUN | AwA | CUB-200 | SUN
Zhang et al.[35]+Zhang et al. [36] |  91.37$\pm$14.75 |  57.09$\pm$27.91 |  85.96$\pm$10.15 |  90.28$\pm$8.08 |  55.73$\pm$31.80 |  86.00$\pm$13.19
Zhang et al.[34]+Zhang et al. [36] |  89.19$\pm$15.09 |  57.20$\pm$25.96 |  86.06$\pm$ 12.36 |  86.04$\pm$9.82 |  55.77$\pm$26.54 |  85.50$\pm$13.68
GFZSL: Linear | 93.70 | 57.90 | 87.40 | 92.20 | 57.40 | 87.00
GFZSL: Kernel | 93.80 | 64.09 | 87.40 | 92.30 | 63.96 | 87.00
Table 3: ZSL precision and recall scores obtained in the transductive setting: results reported using the VGG-19 features. Average Precision and recall for the all dataset with its standard daviation over the 100 iteration. Note: Precision and recall scores not available for Guo et al.[9] and Romera et al.[22]\+ Zhang et al. [36] Dataset | Method | 2 | 5 | 10 | 15 | 20
---|---|---|---|---|---|---
AwA | GFZSL | 87.96$\pm$1.47 | 91.64 $\pm$0.81 | 93.31 $\pm$0.50 | 94.01 $\pm$.36 | 94.30 $\pm$0.33
SVM | 74.81 | 83.19 | 90.44 | 91.22 | 92.04
CUB-200 | GFZSL |  60.84 $\pm$1.39 |  64.81$\pm$ 1.14 |  68.44$\pm$ 1.21 |  70.11$\pm$ 0.93 |  71.23$\pm$ 0.87
SVM | 46.19 | 59.33 | 68.75 | 73.87 | 75.42
SUN | GFZSL |  75.57$\pm$ 4.79 |  83.05$\pm$ 3.60 |  82.09 $\pm$3.30 | – | –
SVM | 56.00 | 77.00 | 78.00 | – | –
Table 4: Accuracy(%) in the few-shot learning setting: For each data set, the
accuracies are reported using $2,5,10,15,20$ labeled examples for each unseen
class

###  4.2 Few-shot Learning (FSL)

We next perform an experiment with the few-shot learning setting [23, 17]
where we provide each model with a small number of labelled examples from each
of the unseen classes. For this experiment, we follow the procedure described
in Section 4.2 to learn the parameters of the class-conditional distributions
of the unseen classes. In particular, we train the inductive ZSL model (using
only the seen class training data) and the refine the learned model further
using a very small number of labelled examples from the unseen classes (i.e.,
the few-shot learning setting).

To see the effect of knowledge transfer from the seen classes, we use a
multiclass SVM as a baseline that is provided with the same number of labelled
examples from each unseen class. In this experiment, we vary the number of
labelled examples of unseen classes from 2 to 20 (for SUN we only use 2, 5,
and 10 due to the small number of labelled examples). In Figure-2, we also
compare with standard (inductive) ZSL which does not have access to the
labelled examples from the unseen classes. Our results are shown Table-4 and
Figure-2.

As shown in Table-4 (all data sets) and Figure-2, the classification accuracy
on the unseen classes shows a significant improvement over the standard
inductive ZSL, even with as few as 2 or 5 additional labelled examples per
class. We also observe that the few-shot learning method outperform multiclass
SVM which only relies on the labelled data from the unseen classes. This
demonstrates the advantage of the knowledge transfer from the seen class data.

Figure 2: (On AwA data): A comparison on classification accuracies of the few-
shot learning variant of our model with multi-class SVM (training on labeled
examples from seen classes) and the inductive ZSL

###  4.3 Generalized Few-Shot Learning (GFSL)

We finally perform an experiment on the more challenging generalized few-shot
learning setting [6]. This setting assumes that test examples can come from
seen as well as unseen classes. This setting is known to be notoriously hard
[6]. In this setting, although the ZSL models tend to do well on predicting
test examples from seen classes, the performance on correctly predicting the
unseen class example is poor [6] since the trained models are heavily biased
towards predicting the seen classes.

One way to mitigate this issue could be to use some labelled examples from the
unseen classes (akin to what is done in few-shot learning). We, therefore,
perform a similar experiment as in Section 4.2. In Table-5, we show the
results of our model on classifying the unseen class test examples in this
setting.

As shown in Table-5, our model’s accuracies on the generalized FSL task
improve as it gets to see labelled examples from unseen classes. However, it
is still outperformed by a standard multiclass SVM. The better performance of
SVM can be attributed to the fact that it is not biased towards the seen
classes since the classifier for each class (seen/unseen) is learned
independently.

Our findings are also corroborated by other recent work on generalized FSL [6]
and suggest the need of finding more robust ways to handle this setting. We
leave this direction of investigation as a possible future work.

Dataset | Method | 2 | 5 | 10 | 15 | 20
---|---|---|---|---|---|---
AwA | GFZSL |  25.32 $\pm$ 2.43 |  37.42 $\pm$ 1.60 |  43.20 $\pm$ 1.39 |  45.09 $\pm$ 1.17 |  45.96 $\pm$ 1.09
SVM | 40.84 | 60.81 | 75.36 | 77.00 | 77.10
CUB-200 | GFZSL |  6.64 $\pm$ 0.87 |  15.12 $\pm$ 1.17 |  22.02 $\pm$ 0.76 |  25.03 $\pm$ 0.71 |  26.47 $\pm$ 0.83
SVM | 25.97 | 37.98 | 47.10 | 53.87 | 54.42
SUN | GFZSL |  1.17 $\pm$ 1.16 |  4.20 $\pm$ 1.77 |  9.48 $\pm$ 2.22 | – | –
SVM | 9.94 | 20.00 | 27.00 | – | –
Table 5: Accuracies (%) in the generalized few-shot learning setting.

##  5 Conclusion

We have presented a flexible generative framework for zero-shot learning,
which is based on modelling each seen/unseen class using an exponential family
class-conditional distribution. In contrast to the semantic embedding based
methods for zero-shot learning which model each class as a point in a latent
space, our approach models each class as a distribution, where the parameters
of each class-conditional distribution are functions of the respective class-
attribute vectors. Our generative framework allows learning these functions
easily using seen class training data (and optionally leveraging additional
unlabeled data from seen/unseen classes).

An especially appealing aspect of our framework is its simplicity and modular
architecture (cf., Figure 1) which allows using a variety of algorithms for
each of its building blocks. As we showed, our generative framework admits
natural extensions to other related problems, such as transductive zero-shot
learning and few-shot learning. It is particularly easy to implement and scale
to a large number of classes, using advances in large-scale regression. Our
generative framework can also be extended to jointly learn the class
attributes from an external source of data (e.g., by learning an additional
embedding model with our original model). This can be an interesting direction
of future work. Finally, although we considered a point estimation of the
parameters of class-conditional distributions, it is also possible to take a
fully Bayesian approach for learning these distributions. We leave this
possibility as a direction for future work.

Acknowledgements: This work is supported by a grant from Tower Research CSR,
Dr. Deep Singh and Daljeet Kaur Fellowship, and Research-I Foundation, IIT
Kanpur. Vinay Verma acknowledges support from Visvesvaraya Ph.D. fellowship.

## References

  * [1] Akata, Z., Perronnin, F., Harchaoui, Z., and Schmid, C. Label-embedding for attribute-based classification.  In CVPR (2013).
  * [2] Akata, Z., Reed, S., Walter, D., Lee, H., and Schiele, B. Evaluation of output embeddings for fine-grained image classification.  In CVPR (2015).
  * [3] Brown, L. D. Fundamentals of statistical exponential families.  Institute of Mathematical Statistics (1986).
  * [4] Bucher, M., Herbin, S., and Jurie, F. Improving semantic embedding consistency by metric learning for zero-shot classification.  arXiv preprint arXiv:1607.08085 (2016).
  * [5] Changpinyo, S., Chao, W.-L., Gong, B., and Sha, F. Synthesized classifiers for zero-shot learning.  In CVPR (2016).
  * [6] Chao, W.-L., Changpinyo, S., Gong, B., and Sha, F. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild.  In ECCV (2016).
  * [7] Friedman, J., Hastie, T., and Tibshirani, R. The elements of statistical learning, vol. 1.  Springer series in statistics Springer, Berlin, 2001.
  * [8] Fu, Y., Hospedales, T. M., Xiang, T., and Gong, S. Transductive multi-view zero-shot learning.  PAMI (2015).
  * [9] Guo, Y., Ding, G., Jin, X., and Wang, J. Transductive zero-shot recognition via shared model space learning.  In AAAI (2016).
  * [10] Jayaraman, D., and Grauman, K. Zero-shot recognition with unreliable attributes.  In NIPS (2014).
  * [11] Kodirov, E., Xiang, T., Fu, Z., and Gong, S. Unsupervised domain adaptation for zero-shot learning.  In ICCV (2015).
  * [12] Krizhevsky, A., and Hinton, G. Learning multiple layers of features from tiny images.
  * [13] Lampert, C. H., Nickisch, H., and Harmeling, S. Learning to detect unseen object classes by between-class attribute transfer.  In CVPR (2009).
  * [14] Lampert, C. H., Nickisch, H., and Harmeling, S. Attribute-based classification for zero-shot visual object categorization.  PAMI (2014).
  * [15] Li, X., and Guo, Y. Max-margin zero-shot learning for multi-class classification.  In AISTATS (2015).
  * [16] Li, X., Guo, Y., and Schuurmans, D. Semi-supervised zero-shot classification with label representation learning.  In CVPR (2015).
  * [17] Mensink, T., Gavves, E., and Snoek, C. G. Costa: Co-occurrence statistics for zero-shot classification.  In CVPR (2014).
  * [18] Mukherjee, T., and Hospedales, T. Gaussian visual-linguistic embedding for zero-shot recognition.  In EMNLP (2016).
  * [19] Murphy, K. P. Machine learning: a probabilistic perspective.  MIT press, 2012.
  * [20] Norouzi, M., Mikolov, T., Bengio, S., Singer, Y., Shlens, J., Frome, A., Corrado, G. S., and Dean, J. Zero-shot learning by convex combination of semantic embeddings.  ICLR (2014).
  * [21] Qiao, R., Liu, L., Shen, C., and van den Hengel, A. Less is more: zero-shot learning from online textual documents with noise suppression.  In CVPR (2016).
  * [22] Romera-Paredes, B. An embarrassingly simple approach to zero-shot learning.  In ICML (2015).
  * [23] Salakhutdinov, R., Tenenbaum, J. B., and Torralba, A. Learning with hierarchical-deep models.  PAMI (2013).
  * [24] Schölkopf, B., and Smola, A. J. Learning with kernels: support vector machines, regularization, optimization, and beyond.  MIT press, 2001.
  * [25] Simonyan, K., and Zisserman, A. Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556 (2014).
  * [26] Socher, R., Ganjoo, M., Manning, C. D., and Ng, A. Zero-shot learning through cross-modal transfer.  In NIPS (2013).
  * [27] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions.  In CVPR (2015).
  * [28] Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset.
  * [29] Wang, D., Li, Y., Lin, Y., and Zhuang, Y. Relational knowledge transfer for zero-shot learning.  In AAAI (2016).
  * [30] Wang, Q., and Chen, K. Zero-shot visual recognition via bidirectional latent embedding.  arXiv preprint arXiv:1607.02104 (2016).
  * [31] Xian, Y., Akata, Z., Sharma, G., Nguyen, Q., Hein, M., and Schiele, B. Latent embeddings for zero-shot classification.  In CVPR (2016).
  * [32] Zhang, L., Xiang, T., and Gong, S. Learning a deep embedding model for zero-shot learning.  arXiv preprint arXiv:1611.05088 (2016).
  * [33] Zhang, Z., and Saligrama, V. Zero-shot learning via semantic similarity embedding.  In ICCV (2015).
  * [34] Zhang, Z., and Saligrama, V. Learning joint feature adaptation for zero-shot recognition.  arXiv preprint arXiv:1611.07593 (2016).
  * [35] Zhang, Z., and Saligrama, V. Zero-shot learning via joint latent similarity embedding.  In CVPR (2016).
  * [36] Zhang, Z., and Saligrama, V. Zero-shot recognition via structured prediction.  In ECCV (2016).
  * [37] Zhao, B., Wu, B., Wu, T., and Wang, Y. Zero-shot learning via revealing data distribution.  arXiv preprint arXiv:1612.00560 (2016).