# Deep Episodic Memory: Encoding, Recalling, and Predicting
Episodic Experiences for Robot Action Execution

Jonas Rothfuss${}^{\ast}{{}^{\dagger}}$, Fabio
Ferreira${}^{\ast}{{}^{\dagger}}$, Eren Erdal Aksoy ‡, You Zhou† and Tamim
Asfour† The research leading to these results has received funding from the
European Union’s Horizon 2020 research and innovation programme under grant
agreement No 641100 (TimeStorm) and from the German Research Foundation (DFG:
Deutsche Forschungsgemeinschaft) under Priority Program on Autonomous Learning
(SPP 1527).†The authors are with the Institute for Anthropomatics and
Robotics, Karlsruhe Institute of Technology, Karlsruhe, Germany.‡Eren E. Aksoy
is with with the School of Information Technology, Halmstad University,
Sweden∗The first two authors contributed equally to this work.

###### Abstract

We present a novel deep neural network architecture for representing robot
experiences in an episodic-like memory which facilitates encoding, recalling,
and predicting action experiences. Our proposed unsupervised deep episodic
memory model 1) encodes observed actions in a latent vector space and, based
on this latent encoding, 2) infers most similar episodes previously
experienced, 3) reconstructs original episodes, and 4) predicts future frames
in an end-to-end fashion. Results show that conceptually similar actions are
mapped into the same region of the latent vector space. Based on these
results, we introduce an action matching and retrieval mechanism, benchmark
its performance on two large-scale action datasets, 20BN-something-something
and ActivityNet and evaluate its generalization capability in a real-world
scenario on a humanoid robot.

##  I Introduction

Humans are ingenious: We have unique abilities to predict the consequences of
observed actions, remember the most relevant experiences from the past, and
transfer knowledge from previous observations in order to adapt to novel
situations. The episodic memory which encodes contextual, spatial and temporal
experiences during development plays a vital role to introduce such cognitive
abilities in humans.

A core challenge in cognitive robotics is compact and generalizable mechanism
which allow for encoding, storing and retrieving spatio-temporal patterns of
visual observations. Such mechanisms would enable robots to build a memory
system, allowing them to efficiently store gained knowledge from past
experiences and both recalling and applying such knowledge in new situations.

Inspired by infants that learn by observing and memorizing what adults do in
the same visual setting, we investigate in this paper how to extend cognitive
abilities of robots to autonomously infer the most probable behavior and
ultimately adapt it to the current scene. Considering the situation of the
humanoid robot ARMAR-IIIa standing in front of a table with a juice carton
(see Fig. 1) one can ask what the most suitable action is and how it would
best be performed.

To achieve this goal, we introduce a novel deep neural network architecture
for encoding, storing, and recalling past action experiences in an episodic
memory-like manner. The proposed deep network encodes observed action episodes
in a lower-dimensional latent space. Such a formulation in the latent space
allows robots to store visual experiences, compare them based on their
conceptual similarity and retrieve the most similar episodes to the query
scene or action. Further, the same network leads to predict and generate the
next possible frames of a currently observed action.

To the best of our knowledge, this is the first study introducing that vision-
based cognitive abilities concerning action representing, storing, memorizing,
and predicting can be achieved in a single coherent framework.

Figure 1: The ARMAR-IIIa humanoid robot, see [1], recalling previous visual
episodes in a kitchen scene.

We hypothesize that latent subsymbolic encodings that our network generates
from visual observations are rich and descriptive enough to be compared with
those collected from previously experienced episodes. In this way, ARMAR-IIIa
can trace all previous observations and select the most similar episode (e. g.
“pushing the juice" or “grasping the juice") in the latent space. The robot
can further generate similar behavior by adapting to new situations based on
memorized action representations.

Contribution: (1) We implement a new deep network to encode action frames into
a low-dimensional latent vector space. (2) Such a vector representation is
used to reconstruct the action frames in an auto-encoder manner. (3) We show
that the same latent vectors can also be employed to predict future action
frames. (4) We introduce a mechanism for matching and retrieving visual
episodes and provide an evaluation of the proposed method on two action
datasets. (5) Finally, we demonstrate how this meachanism can facilitate case-
based reasoning for robotic object manipulation in an unstructured real-world
scenario.

##  II Related Work

We discuss related work from two relevant perspectives: the role of episodic
memories in cognitive architectures and action understanding based on deep
learning approaches.

###  II-A Episodic Memory and Cognitive Architectures

In contrast to working memory where the information is temporarily stored for
a finite length of time, the long-term memory holds the innate knowledge that
enables operation of the system and facilitates learning. The episodic memory,
considered as a part of the long-term memory, persists instances of past
experiences which can be retrieved to support planning and inference [2].
Hereby, the persisted experiences can be represented in manifold ways.

Reinforcement learning based architectures implement memories to store and
retrieve action episodes[3, 4]. A different approach is to persist instances
of the working memory that were involved in solving a specific problem and
subsequently retrieve previous solutions from the episodic memory. Thereby,
planning can be enhanced and even facilitate one-shot learning capabilities
[5, 6, 7, 8, 9]. Predominantly, instances stored in the episodic memory are
symbolic high-level representations [10, 7]. When restricted to a specific
context, symbolic representations and pre-specified perceptual instances
stored in an episodic memory can indeed be a powerful approach for enhancing
the reasoning capabilities of a cognitive system, as shown in Soar [5].
However, most of the described approaches are customized to a specific problem
domain and rely on pre-defined, problem specific representations [10, 6]. In
complex real world scenarios transferring and generalizing knowledge persisted
in the episodic memory is very limited when pre-defined symbolic
representations are used. Accounting for nuances and fuzziness may require
interpolation between concepts, demanding more flexibility than traditional
declarative memory concepts. Our proposed episodic memory, on the other hand,
derives subsymbolic representation of actions in a data driven manner and,
hence, requires no pre-defined information.

An approach towards an episodic-like memory of video scenes, based on
subsymbolic representations, uses Fisher Vectors of convolutional neural
network (CNN) percepts to generate encodings of temporal video segments [11].
Although the approach is able to match conceptually related video segments, it
is not possible to reconstruct perceptual information from the Fisher Vector
representations.

###  II-B Action Understanding with Deep Neural Nets

Many deep neural network based approaches to understand human action videos
combine CNNs and recurrent neural networks (RNNs) [12, 13, 14]. CNNs capture
spatial information in each video frame and aggregate it into a higher-level
representation, which is then fed through a Long Short- Term Memory (LSTM)
that captures temporal information throughout a sequence of frames. Instead of
stacking an LSTM on top of a CNN, Shi et al. [15] combine the ideas of spatial
weight sharing through convolution and temporal weight sharing through
recurrence into a new model called convolutional LSTMs (convLSTM).

Overall, there are three main approaches with regard to deep learning based
action understanding: 1) Supervised learning on activity recognition corpora
[16, 17, 18], 2) Unsupervised video frame prediction [19, 20, 21, 22] and 3)
Unsupervised prediction of visual representations [23].

Aside from human action recognition, the lack of comprehensively labeled video
datasets makes supervised training challenging. Another approach towards
learning to understand videos is the future frame prediction. Given a sequence
of video frames, a deep neural network is trained to predict the next frame(s)
in a video. To successfully predict future video frames, the network is forced
to generate latent representations of the inherent structure and dynamics of
videos.

Srivastava et al. [14] present a composite model consisting of three LSTM
networks, conceptually combining a sequence-to-sequence autoencoder with
future frame prediction. They show that the composite architecture outperforms
both a pure autoencoder and future frame prediction model. However, since
input and output space are CNN features instead of raw video frames, their
model is not able to recover the video frames from the latent representation.
Our approach is inspired by the composite encoder-decoder architecture but
overcomes the described drawback by being able to encode raw frames in an end-
to-end fashion and also reconstruct the raw frame sequence from the latent
representation.

Subsequent work aims to predict the pixel changes between the current and the
next frame [20, 21, 22] instead of regressing directly into the RGB-pixel
space. While such models are shown to be useful for semantic segmentation
[21], planning robot motion [24] or generating videos [25], they do not create
a representation of an episode that can later be reconstructed.

##  III Method

###  III-A The Neural Network Model

In this section, we describe our neural network model and the methods applied
for comparing and matching visual experiences in the latent space. The network
architecture is illustrated in Fig. 2.

Figure 2: Structure of the proposed composite encoder-decoder network. It
represents the shape of an unrolled network over multiple time steps. The
encoder $\mathit{E}$ receives multiple video frames as input and maps them
into a latent vector space. The resulting vector representation $\mathit{V}$
(highlighted in red) is forwarded to the two decoder networks. The first
decoder ($\mathit{D}_{r}$) is trained to reconstruct the video frames that
were provided to the encoder while the second decoder ($\mathit{D}_{p}$)
attempts to predict the future frames. The dashed box on the left depicts the
layers of the encoder network. The label inside each layer denotes the kernel
size of the convolutional layer or the number of hidden units of the fully
connected (fc) layer, respectively.

We were inspired by the composite encoder-decoder architecture introduced in
[14]. Our proposed model conceptually combines an autoencoder with future
frame prediction and consists of one encoder and two decoders (see Fig. 2).
Different from the model proposed by [14], we utilize convolutional LSTM cells
to capture both spatial and temporal information. In our model, a visual
experience is represented as a sequence of consecutive video frames
$X=X_{r}\|X_{p}=x_{1},..,x_{k}\|x_{k+1},...,x_{n}$, wherein $X_{r}$ is the
first part of the frame sequence until frame $x_{k}$ and $X_{p}$ represents
the remaining $n-k$ frames.

The encoder network $E$ processes the sequence $X_{r}=x_{1},...,x_{k}$ and
projects it into a latent vector space, yielding a representation of the given
frame sequence as a single latent vector $V$ as:

$V=E(X_{r})\quad.$ (1)

Subsequently, the vector $V$, indicated in red in Fig. 2, is forwarded to both
decoders independently which receive the latent vector representation as input
and construct a sequence of video frames in return. The first decoder, i. e.
the reconstruction-decoder $D_{r}$, attempts to recover the frames
$X_{r}=x_{1},..,x_{k}$ that were initially provided to the encoder. Therefore,
$D_{r}$ is trained to output a frame sequence $Y_{r}=y_{1},...,y_{k}$ that
matches $X_{r}$, such that

$D_{r}(V)=Y_{r}=y_{1},...,y_{k}\quad.$ (2)

The second decoder, the so-called prediction-decoder $D_{p}$, attempts to
predict the future frames $Y_{p}=y_{k+1},...,y_{n}$ as,

$D_{p}(V)=Y_{p}=y_{k+1},...,y_{n}\quad.$ (3)

During training, $X_{p}$ is employed as ground truth for assessing how good
the predictions $Y_{p}$ are and for also computing the error. It is important
to note that for determining the reconstruction and prediction error during
training, both image sequences $X_{r}$ and $X_{p}$ are used. However, during
test time, only $X_{r}$ is fed into the encoder network.

The core idea of the proposed network structure rests upon the latent space
vector $V$ being the only linkage between $E$ and both $D_{r}$ and $D_{p}$.
The two decoder networks solely rely on $V$ as their only source of
information to reconstruct a given scene and predict future frames. To obtain
robust reconstructions and future frame predictions, the encoder is forced to
compress the entire video frame sequence $X_{r}$ into a comparably low-
dimensional latent representation $V$ and, at the same time, to preserve as
much relevant information as possible. $E$ and $D_{r}$ together constitute an
autoencoder architecture, requiring that relevant information is preserved
throughout the network. However, this only involves remembering the frame
sequence $X_{r}$ but not necessarily requires to capture abstract concepts
such as temporal dynamics of objects or actors. By adding the frame predictor
which has to extrapolate motions into the future, the encoder must capture
higher-level concepts like the scene dynamics in $X_{r}$ and embed abstract
concepts such as trajectories in $V$ so that $D_{p}$ can properly infer
possible future frames.

The input and output frames $x_{i}$ and $y_{i}$ used in this work have a
resolution of $128\times 128$ pixels and 3 color channels.

Since the main task of the network is to capture spatio-temporal concepts, we
make use of convolutional LSTM cells [15]. The encoder network $E$ is
comprised of a stack of convolution LSTM and normal convolution layers
(henceforth referred to as _convLSTM_ and _conv_ layers) in alternating order
(see Fig. 2). While the conv layers are operated with a stride of 2 in order
to reduce the spatial size of the feature maps, the convLSTM layers preserve
the spatial size and forward information to the next time step through their
hidden state and cell state. After the alternating series of conv and convLSTM
layers in the encoder, we add a fully connected layer, followed by a fully
connected LSTM layer (fc LSTM) to the stack.

With the fc LSTM cell being the top layer of $E$ and as LSTM cell connected
over time, its cell state $c_{i}$ and hidden state $h_{i}$ represent the video
frame sequence until the current time step $i$. Once the entire frame sequence
$X_{r}=x_{1},...,x_{k}$ is processed by the encoder, the hidden state $h_{k}$
and cell state $c_{k}$ of the fc LSTM cell at time step $k$ are extracted and
concatenated, yielding the latent vector $V=h_{k}\|c_{k}$.

Both decoders have the inverted structure of the encoder, meaning that
transposed convolution layers are used to increase the spatial size of the
feature maps throughout the decoding layers until the full video frame
resolution of $128\times 128$ is recovered.

To compute the error $\mathit{L}$ during the network training, we use a linear
combination of image reconstruction loss ($\mathit{L}_{mse}$) and gradient
difference loss ($\mathit{L}_{gd}$) [19] functions as follows

$\mathit{L}=(1-\eta)~{}\mathit{L}_{mse}+\eta~{}\mathit{L}_{gd}\quad,$ (4)

where we set $\eta=0.4$ to trade off between the two loss functions

$\mathit{L}_{mse}=\frac{1}{n}\sum_{i=1}^{n}\lVert{y_{i}-x_{i}}\lVert^{2}_{2}\quad,~{}~{}\text{and}$
(5)
$\small\begin{split}\mathit{L}_{gd}=\frac{1}{n}\sum_{i=1}^{n}\sum_{u,v}\lVert\lvert
x_{i_{u,v}}-x_{i_{u-1,v}}|-|y_{i_{u,v}}-y_{i_{u-1,v}}\rvert\rVert^{2}_{2}+\\\
\lVert\lvert x_{i_{u,v-1}}-x_{i_{u,v}}\rvert-\lvert
y_{i_{u,v-1}}-y_{i_{u,v}}\rvert\rVert^{2}_{2}\quad.\end{split}$ (6)

The loss is computed over all ground truth frames $x_{i}$ in $X=X_{r}\|X_{p}$
and the output frames $y_{i}$ in $Y=Y_{r}\|Y_{p}$ which are produced by
$D_{r}$ and $D_{p}$. The reconstruction loss $\mathit{L}_{mse}$ compares the
generated images $y_{i}$ and ground truth images $x_{i}$ in a pixel wise
manner. When solely trained with $\mathit{L}_{mse}$ loss, neural network
models that regress on images are prone to linear blurring and unstable to
small image deformations [12]. In contrast, the $\mathit{L}_{gd}$ loss
compares the horizontal and vertical image gradients of $x_{i}$ and $y_{i}$,
thereby penalizing blurriness and enforcing sharper edges [19].

The described neural network is trained with mini-batch gradient decent using
the adaptive learning rate method ADAM [26] in conjunction with an
exponentially decaying learning rate schedule. After each network layer except
the last encoder and decoder layer (_since these are the output layers_), we
use layer normalization[27] and dropout with a dropout rate between $10\%$ and
$20\%$. In order to force the encoder to use the entire latent vector space
and produce distinct representations $V$, we add Gaussian noise $N(0,\sigma)$
with $\sigma=0.1$ to the latent vector $V$ during the training, before
forwarding $V$ to the decoder networks. In all of our experiments, the vector
$V$ has a dimension of $2000$. The specifications of the neural network model
have been determined with heuristic hyper-parameter search using train/test-
splits of the original training data of the two datasets. The source code and
experimental data are publicly available on the supplementary web
page111h2t-projects.webarchiv.kit.edu/projects/episodicmemory .

###  III-B Matching Visual Experiences in the Latent Space

One of the central contributions of our work is to compare visual experiences
based on their conceptual similarities encoded in the latent space. Given a
new visual experience, we can retrieve the most similar episodes from the
episodic memory that holds the hidden representations $V_{i}$ of episodes
experienced in the past. We use the cosine similarity to measure the
similarity of latent vectors. To find the best matches in the latent space, we
compute the cosine similarity between the query representation $V_{q}$ and
each of the $V_{i}$ in the memory. Finally, the $n$ memory instances
corresponding to the $V_{i}$ with the highest cosine similarity are retrieved
from the memory.

##  IV Experimental Evaluation

We evaluate the hypothesis that the encoder-decoder network creates latent
representations that embed the inherent dynamics and concepts of a provided
visual episode.

For this purpose, we train the neural network in an unsupervised fashion on
two video datasets and analyze the similarity structure within the latent
space. We assess the model’s abilities to reconstruct the past episode from
the latent representation and predict future frames. Subsequently, we
benchmark the proposed matching / retrieval mechanism against other state-of-
the art approaches and test its robustness in a robotic application.

###  IV-A Datasets

For the evaluation of our methods we use the large-scale labeled video
datasets ActivityNet [17] and 20BN-something-something (from now on referred
to as 20BN) [28]. We favored these two datasets over other popular datasets
like UCF-101 [16] and HMDB-51 [29] since our emphasis is reasoning, planning
and executing of robotic tasks in indoor household environments of which the
latter two datasets contain fewer relevant scenes.

The ActivityNet dataset [17], a benchmarking corpus for human activity
understanding, consists of $10,024$ training and $4926$ validation video
snippets collected from YouTube. It is organized in 93 higher level categories
that comprise $203$ different activity classes involving activities such as
household work, sports and personal care.

While ActivityNet targets higher-level concepts like “vacuuming the floor" and
“shoveling snow" that embed semantic meaning, the 20BN dataset focuses on
detailed physical properties of actions and scenes. It contains $174$ classes
such as “Pushing something from left to right" and “Putting something into
something". The core challenge of this novel dataset is that the type of
involved objects as well as the background setting of a given scene only play
a neglectable role. Rather than recognizing familiar items and scene
backgrounds, the neural network needs to understand the physical composition
and motion within the video clips. The dataset consists of $86,017$ training
and $11,522$ validation videos in total.

###  IV-B Training

We train the neural network on the respective training split of the
ActivityNet (actNet) and 20BN dataset. Both models are trained with $n=10$
frames per video (selected equally spaced). The first $k=5$ frames are fed
into the encoder $\mathit{E}$ to be reconstructed by $\mathit{D}_{r}$ while
the last five frames are employed as ground truth for the future frame
predictor $\mathit{D}_{p}$.

The choice to only use 10 frames, sampled equally spaced, during training
results from hardware constraints (i.e. GPU RAM) while under these constraints
attempting to maximize the video length that can be captured.

###  IV-C Conceptual Similarity and Proximity in the Latent Space

To examine our hypothesis that conceptually similar videos are mapped into the
same region of the latent space, we compute the pairwise cosine similarities
(see section III-B) of latent vectors. To measure the conceptual similarity,
we use the class labels provided in the datasets as the proxy value and assume
that videos belonging to the same class are conceptually similar. We generate
the latent representation for each video in the validation split of both
ActivityNet and 20BN datasets and subsequently compute all pairwise cosine
similarities between the latent vectors.

In Fig. 3, the results are visualized as similarity matrices where each row
and column of a matrix corresponds to a class label and each entry represents
the mean pairwise cosine similarity between the latent vectors belonging to
the respective classes. The class labels are arranged horizontally and
vertically in the same order, ensuring that the diagonal elements of the
matrix depict intra-class similarities and off-diagonals represent inter-class
similarities. Fig. 3 shows that in each matrix the intra-class similarity
(diagonal elements) is considerably higher than the inter-class similarity.
This is a clear indication that conceptual similarity of videos is reflected
by the proximity of their vector representations in the latent space.
Consequently, our proposed model captures high-level action concepts within
two different datasets, although the model is trained in an _unsupervised_
fashion and thus has never seen any class labels.

Since the latent representation must embed all information for the decoders
necessary to reconstruct and predict frames, it may also encode details (e. g.
colors and shapes) in the background that are irrelevant for describing
actions. To compile the information embedded in the latent representation to a
subset that is more relevant for optimally separating the different classes
within the latent space, we apply principal component analysis (PCA) on the
mean latent vectors of each class. We assume that less important features are
identically distributed in all the classes and thus share approximately the
same mean value when averaged over the class. Hence, transforming the latent
space towards the principal components computed on the covariance matrix of
the mean vectors emphasizes relevant features and neutralizes less important
features. Fig. 3(b) and 3(d) show that transforming the latent representations
with PCA leads to a better distribution of latent vectors, pushing
conceptually similar representations closer together while keeping
representations of different classes farther apart.

(a) latent vector $V$

(b) 200 PCA components

(c) latent vector $V$

(d) 200 PCA components

Figure 3: Cosine similarity matrices for ActivityNet (top) ($93\times 93$) and
20BN (bottom) ($174\times 174$). PCA (right column) yields a more favorable
distribution in the latent space. The figures are available in full resolution
and with class labels on our website${}^{\ref{website}}$. Figure 4: Frame
reconstruction and future frame prediction of our model compared to the
original frame sequence. Top: Validation sample from ActivityNet. Bottom:
Validation sample (pulling sth. from right to left) from 20BN.

###  IV-D Frame Reconstruction and Future Frame Prediction

By reconstructing video frames and predicting upcoming frames, the network
resembles episodic memory-like capabilities. Fig. 4 depicts generated video
frame sequences from both ActivityNet and 20BN for a qualitative assessment.
To evaluate the quality of the reconstructed and predicted frames, we compute
the Peak-Signal-to-Noise-Ratio (PSNR) between the original frames $X$ and
generated frames $Y$ as proposed in [19]. Fig. 5 depicts the PSNR for each of
the 5 reconstructed and predicted frames, averaged frame-wise over the entire
respective validation samples of both datasets. We additionally compare our
results to a simple baseline by computing the PSNR between the ground truth
frames and the mean of the input frames. Results indicate that the
reconstruction quality is significantly higher than the quality of the
predicted frames. Also, the PSNR is roughly constant throughout the
reconstructed frames whereas for the predicted future frames it decreases over
time. The expected decline in prediction quality is very much due to the
increase in the uncertainty about the future over successive time steps.
Moreover, the frame prediction results in Fig. 4 indicate that the decline in
PSNR is mainly caused by the increase in blurriness while e.g. the object
motions are still extrapolated correctly into the future. Hence, this
successive increase in blurriness predominantly constitutes a general
limitation of regression into the pixel space and gives little to no
indication about the semantic richness of the latent representations.

Figure 5: Peak-Signal-to-Noise-Ratio (PSNR) for each of the 5 reconstructed
(blue) and predicted (green) frames generated by the 20BN and actNet model
averaged over the entire validation dataset. The shaded regions depict the
corresponding standard deviation. Additionally using the channel-wise mean of
input frames as reconstruction/prediction is reported as simple baseline
(red/orange).

###  IV-E Matching and Retrieving Visual Episodes

To investigate the matching and retrieval of visual episodes introduced in
section III-B, we compare our approach to standard baselines as well as state-
of-the-art action descriptors. The benchmarking comprises Fisher vector
encodings of CNN, SIFT and STIP features as well as LSTM encodings to
represent and match visual experiences.

As proposed in [11], we compute Fisher Vectors based on GMMs in order to
create a visual vocabulary from CNN features of the video frames. In
particular, we select 5 equidistant frames from the videos and compute VGG-fc1
[30] as well as ResNet-50 [31] features. Furthermore, we compare our approach
against the composite LSTM network introduced in [14]. We train their LSTM
network using VGG-FC1 and ResNet-50 features. In all our experiments we used
the default parameters coming with the publicly available source codes.

To quantitatively benchmark our approach against the baselines, we phrase the
matching and retrieval of memorized episodes as a document retrieval problem.
Thereby, we assume that a retrieved episode is only relevant if it originates
from the same action category as the query episode. For evaluating the
performance of retrieving relevant visual episodes, we report the precision of
the first match and the mean average precision (mAP). Since the setting is
purely unsupervised, the precisions reported in this context are not to be
confused with precisions for a classification task. Overall, we use the
ActivityNet and the 20BN dataset to train and evaluate the different methods.
For computing the performance metrics, we split the validation set of the
respective dataset into 5 shuffled folds. In each fold, 80% of the videos are
used as memory whereas the remaining 20% are used to query the memory. The
results illustrated in Table I are averaged over the 5 folds.

The evaluations were conducted with various numbers of GMM components and two
different distance metrics (i.e. Euclidean and cosine) for matching the video
encodings. We want to emphasize that we always report the best results out of
these experimental evaluations.

As the results in Table I indicate, representations of conventional
descriptors such as SIFT and STIP seemingly lack the representational capacity
to capture abstract spatio-temporal concepts in videos such as action. In
contrast, using CNN features yields significantly higher average matching
precision. Our proposed method has the highest precision values and notably
outperforms state of the art approaches with substantially deeper networks
such as ResNet-50. For completeness, Table I also reports the matching
precisions with 200 PCA components. Similar to the observations discussed in
section IV-C, PCA further improves the matching. The precisions on ActivityNet
are substantially higher since it has a) fewer categories than the 20BN
dataset and b) richer features among the categories.

In addition to the quantitative benchmark, we qualitatively examined the
matching and retrieval results. The matching appears to be conceptually
consistent, predominantly yielding visual episodes with closely related action
types and settings. Fig 6 (a) shows sample matching results of our approach on
the 20BN dataset. In most cases of categorical mismatches, the retrieved
episode is closely related to the action type of the query episode, e.g.
"pushing sth off of sth" and "moving sth away of sth" or "lifting sth up" and
"moving sth away from sth". However, we also observe that the background color
biases the matching results, meaning that videos with dark background tend to
be matched to videos in memory of the same kind. The same applies to bright
videos respectively.

All in all, our proposed model achieves the best matching results among the
compared approaches. The qualitative study shows that, beyond the strict
categorical setting imposed in the baseline comparison, our mechanism
predominantly produces consistent matches. It is also important to note that
our approach is the only one that can reconstruct the visual episode given the
encoding.

TABLE I: Benchmark results showing the matching and retrieving performance of our approach against the baselines. We report the precision for the first match and the mean average precision (mAP) for retrieving the 3 closest matches. Fisher Vectors are abbreviated to FV. | ActivityNet (in %) | 20BN-sth.-sth. (in %)
---|---|---
Model and Features | Precision | mAP | Precision | mAP
SIFT FV | 5.27 | 3.76 | 1.05 | 0.64
STIP FV [32] | 5.47 | 3.71 | 3.37 | 2.67
ResNet-50 FV [11] | 32.31 | 23.23 | 6.08 | 3.98
VGG-16 FV [11] | 24.30 | 18.19 | 5.56 | 3.62
ResNet-50 LSTM [14] | 36.29 | 26.32 | 10.08 | 7.20
VGG-16 LSTM [14] | 17.18 | 12.16 | 4.51 | 2.82
ours (no PCA) | 44.31 | 26.93 | 11.63 | 8.12
ours with PCA (200) | 45.55 | 28.18 | 11.81 | 8.32
Figure 6: The proposed matching and retrieval mechanism evaluated on different
visual episodes of object manipulations. The figure depicts three exemplary
query episodes and the corresponding 3 closest matches in the latent space of
our proposed network. The latent representations are generated by the 20BN
model. While (a) shows a query on the 20BN dataset, (b) and (c) comprise human
demonstrations for the ARMAR-IIIa robot (see section IV-F ).

###  IV-F Robot Manipulation Learned From Episodic Memory

Results in the previous section indicate that given a query action, our
proposed network provides a robot with the ability to remember similar
scenarios from a large video corpora. In the following, we show how our
episodic memory approach can facilitate case-based reasoning for robot
manipulation in unstructured real world environments.

For this purpose, we record 120 visual episodes in which a human subject is
demonstrating to our humanoid robot ARMAR-IIIa [1] how to perform 10 different
manipulation actions such as “pushing two objects closer to each other" and
“putting something behind something" (see Fig. 6 (b-c)). The reason of
introducing this new dataset is twofold: First, we attempt to evaluate the
scalability of our approach to new datasets that have much less training data.
Second, for the purpose of action execution we require the depth cue which is
missing in the ActivityNet and 20BN datasets. We store 100 of our new visual
episodes to form the memory, whereas the remaining 20 episodes are introduced
as queries to test the matching and retrieval mechanism with ARMAR-IIIa. The
visual episodes are fed through the encoder of the trained 20BN model, thereby
receiving its respective latent representations. The cosine similarity in the
latent space is then computed based on the first 50 principal components of
the latent representations (see section III-B). Fig. 6 (b-c) illustrate two
exemplary query episodes and the corresponding 3 closest matches in the latent
space from our recordings.

So far, the matching and retrieval mechanism is evaluated on manipulation
action videos where spatio-temporal cues are implicitly embedded. We further
investigate whether static scene frames can trigger recalling of past
episodes, which is an even harder problem since effectively only one frame is
given as a cue. This gives a high chance to robots to autonomously predict and
even execute an action that can possibly be performed in the observed scene.
Matching results for static scene queries can be viewed on our supplementary
website${}^{\ref{website}}$ We conduct a pilot study to explore the use of our
method.

Figure 7:  Robot execution of a matched pushing action.

Fig. 7 illustrates a scenario where the robot ARMAR-IIIa is observing a scene
with a sponge on the table. Acquired images of this static scene are sent to
our matching and retrieval mechanism which returns the matched episode where a
subject is demonstrating “pushing a green cup". Next, we apply a real-time
object detector [33] to detect and track all objects in the recalled episode.
This process yields the extracted pushing trajectory that the subject is
following. The tracked motion is then learned by dynamic movement primitives
[34] to be further processed by the robot in order to execute the same pushing
motion on the perceived sponge. Fig. 7 depicts sample frames from the best
matched episode and detected objects together with the computed motion profile
and snapshots from the robot execution of the recalled pushing action. See the
supplementary movie showing the entire robot execution.

This experiment clearly supports our hypothesis that the proposed network
model can help robots autonomously trace previous observations and select the
one that matches best to the currently observed scene even without necessarily
requiring any temporal cue. The robot can transfer relevant knowledge, e. g.
the motion profile, from previous experiences to further apply the remembered
action to novel objects in the scene. Learning comparable vision triggered
behaviors through reinforcement learning would require thousands of trials,
while such a case-based reasoning approach only requires one memory-instance
of a similar action. Hence, these findings play a vital role in cognitive
robotics to infer possible actions, reason about the action consequences, and
even generate actions by transferring knowledge from the past experiences in a
data efficient manner.

##  V Conclusion

We proposed a deep neural network implementing an episodic memory. Given a set
of training data, the proposed network first generates subsymbolic
representation of action episodes. Such a latent encoding can be used to
distinguish actions, reconstruct memorized episodes, and predict future frames
based on the spatio-temporal features extracted by the deep architecture. We
show that conceptual similarity of videos is reflected by the proximity of
their vector representation in the latent space. Using this property of the
latent space, we introduce a matching and retrieval mechanism, which enables
the recollection of previously experienced visual episodes. Benchmarking our
proposed mechanism against a variety of action descriptors, we show that our
model outperforms other state-of-the-art approaches in terms of matching
precision. We conduct various experiments showing that the proposed framework
can help extending the cognitive abilities of a humanoid robot such as action
encoding, storing, memorizing, and predicting.

Since the memory instances in our approach are purely visual, the adaptation
of stored actions on a robot to new scenario is not straightforward. For
instance, our robot manipulation pilot study requires an auxiliary object
detector for trajectory extraction. In future work it would be desirable to
embed relevant action information such as trajectories into the latent
representation and directly use them for action adaptation and execution.
Another limitation of our approach is the observed background bias. To make
the model less sensitive to different backgrounds it might be promising to
avoid pixel-level predictions and instead employ segmentation masks as
reconstruction targets.

To the best of our knowledge, this is the first comprehensive study that
attempts to encode visual experiences not only for matching and retrieving
purposes but also for prediction and reconstruction in a longer time scale.
Comparable work such as [11, 14] can only achieve unsupervised action matching
without reconstructing the memorized video episodes. Video prediction models,
introduced in [24, 21], can only predict a single future frame at a time.
Thus, the mentioned approaches lack key features to resemble an episodic
memory. Our model overcomes the architectural drawback of [14] and comprises
the full episodic-memory capabilities described above. Also, we are not aware
of any previous work that extensively applies an episodic memory-like
framework to such large and complex datasets.

## References

  * [1] T. Asfour _et al._ , “Armar-iii: An integrated humanoid platform for sensory-motor control,” in _Humanoids_ , 2006, pp. 169–175.
  * [2] I. Kotseruba, O. J. A. Gonzalez, and J. K. Tsotsos, “A Review of 40 Years of Cognitive Architecture Research: Focus on Perception, Attention, Learning and Applications,” 2016.
  * [3] R. Sun, E. Merrill, and T. Peterson, “From implicit skills to explicit knowledge: A bottom-up model of skill learning,” in _Cognitive Science_ , vol. 25, 2001, pp. 203–244.
  * [4] B. Rohrer, M. Bernard, D. J. Morrow, F. Rothganger, and P. Xavier, “Model-free learning and control in a mobile robot,” in _ICNC_ , 2009\.
  * [5] A. M. Nuxoll and J. E. Laird, “Extending Cognitive Architecture with Episodic Memory,” in _AAAI_ , 2007, pp. 1560–1565.
  * [6] N. S. Kuppuswamy, S. H. Cho, and J. H. Kim, “A cognitive control architecture for an artificial creature using episodic memory,” in _SICE-ICASE_ , 2006, pp. 3104–3110.
  * [7] D. Stachowicz and G. J. M. Kruijff, “Episodic-like memory for cognitive robots,” _TAMD_ , vol. 4, no. 1, pp. 1–16, 2012.
  * [8] D. Vernon, C. von Hofsten, and L. Fadiga, “The iCub Cognitive Architecture,” in _A Roadmap for Cognitive Development in Humanoid Robots_ , 2010, pp. 121–153.
  * [9] G. M. Park _et al._ , “Deep art neural model for biologically inspired episodic memory and its application to task performance of robots,” _IEEE Transactions on Cybernetics_ , no. 99, pp. 1–14, 2018.
  * [10] D. R. Kuokka, “MAX: A Meta-Reasoning Architecture for "X",” _SIGART Bulletin_ , vol. 2, no. 4, pp. 93–97, 1991.
  * [11] J. Doshi, Z. Kira, and A. Wagner, “From deep learning to episodic memories: Creating categories of visual experiences,” in _ACS_ , 2015.
  * [12] M. A. Ranzato _et al._ , “Video (language) modeling: a baseline for generative models of natural videos,” _CoRR_ , vol. abs/1412.6604, 2014.
  * [13] J. Donahue _et al._ , “Long-term recurrent convolutional networks for visual recognition and description,” in _CVPR_ , 2015.
  * [14] N. Srivastava, E. Mansimov, and R. Salakhudinov, “Unsupervised learning of video representations using lstms,” in _ICML_ , 2015.
  * [15] X. Shi _et al._ , “Convolutional LSTM network: A machine learning approach for precipitation nowcasting,” in _NIPS_ , 2015.
  * [16] K. Soomro, A. R. Zamir, and M. Shah, “UCF101: A dataset of 101 human actions classes from videos in the wild,” _CoRR_ , 2012.
  * [17] F. C. Heilbron _et al._ , “ActivityNet: A large-scale video benchmark for human activity understanding,” in _CVPR_ , 2015.
  * [18] H. Koppula and A. Saxena, “Learning spatio-temporal structure from rgb-d videos for human activity detection and anticipation,” in _ICML_ , 2013, pp. 792–800.
  * [19] M. Mathieu, C. Couprie, and Y. LeCun, “Deep multi-scale video prediction beyond mean square error,” in _ICLR_ , 2015, pp. 1–14.
  * [20] W. Lotter, G. Kreiman, and D. Cox, “Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning,” 2016.
  * [21] V. Patraucean, A. Handa, and R. Cipolla, “Spatio-temporal video autoencoder with differentiable memory,” in _ICLR_ , 2016, pp. 1–10.
  * [22] C. Finn, I. Goodfellow, and S. Levine, “Unsupervised Learning for Physical Interaction through Video Prediction,” in _NIPS_ , 2016.
  * [23] C. Vondrick, H. Pirsiavash, and A. Torralba, “Anticipating Visual Representations from Unlabeled Video,” in _CVPR_ , 2016.
  * [24] C. Finn and S. Levine, “Deep visual foresight for planning robot motion,” in _ICRA_ , 2017, pp. 2786–2793.
  * [25] C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynamics,” in _NIPS_ , 2016.
  * [26] D. P. Kingma and J. L. Ba, “Adam: a Method for Stochastic Optimization,” in _ICLR_ , 2015, pp. 1–15.
  * [27] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer Normalization,” 2016.
  * [28] G. et al., “The "something something" video database for learning and evaluating visual common sense,” 2017.
  * [29] H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “HMDB: A large video database for human motion recognition,” in _ICCV_ , 2011.
  * [30] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _CoRR_ , vol. abs/1409.1556, 2014.
  * [31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” _CoRR_ , vol. abs/1512.03385, 2015.
  * [32] I. Laptev and T. Lindeberg, “Space-time interest points,” in _ICCV_ , 2003, pp. 432–439.
  * [33] J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” _CoRR_ , 2015.
  * [34] J. A. Ijspeert, J. Nakanishi, and S. Schaal, “Movement imitation with nonlinear dynamical systems in humanoid robots,” in _ICRA_ , 2002.


# Age Progression/Regression by Conditional Adversarial Autoencoder

Zhifei Zhang , Yang Song∗, Hairong Qi
The University of Tennessee, Knoxville, TN, USA
{zzhang61, ysong18<EMAIL_ADDRESS>with equal contribution.

###### Abstract

“If I provide you a face image of mine (without telling you the actual age
when I took the picture) and a large amount of face images that I crawled
(containing labeled faces of different ages but not necessarily paired), can
you show me what I would look like when I am 80 or what I was like when I was
5?” The answer is probably a “No.” Most existing face aging works attempt to
learn the transformation between age groups and thus would require the paired
samples as well as the labeled query image. In this paper, we look at the
problem from a generative modeling perspective such that no paired samples is
required. In addition, given an unlabeled image, the generative model can
directly produce the image with desired age attribute. We propose a
conditional adversarial autoencoder (CAAE) that learns a face manifold,
traversing on which smooth age progression and regression can be realized
simultaneously. In CAAE, the face is first mapped to a latent vector through a
convolutional encoder, and then the vector is projected to the face manifold
conditional on age through a deconvolutional generator. The latent vector
preserves personalized face features (i.e., personality) and the age condition
controls progression vs. regression. Two adversarial networks are imposed on
the encoder and generator, respectively, forcing to generate more photo-
realistic faces. Experimental results demonstrate the appealing performance
and flexibility of the proposed framework by comparing with the state-of-the-
art and ground truth.

Figure 1: We assume the face images lie on a manifold ($\mathcal{M}$) , and
images are clustered according to their ages and personality by a different
direction. Given a query image, it will first projected to the manifold, and
then after the smooth transformation on the manifold, the corresponding images
will be projected back with aging patterns.

##  1 Introduction

Face age progression (i.e., prediction of future looks) and regression (i.e.,
estimation of previous looks), also referred to as face aging and
rejuvenation, aims to render face images with or without the “aging” effect
but still preserve personalized features of the face (i.e., personality). It
has tremendous impact to a wide-range of applications, _e.g_., face prediction
of wanted/missing person, age-invariant verification, entertainment, etc. The
area has been attracting a lot of research interests despite the extreme
challenge in the problem itself. Most of the challenges come from the rigid
requirement to the training and testing datasets, as well as the large
variation presented in the face image in terms of expression, pose,
resolution, illumination, and occlusion. The rigid requirement on the dataset
refers to the fact that most existing works require the availability of
“paired” samples, i.e., face images of the same person at different ages, and
some even require paired samples over a long range of age span, which is very
difficult to collect. For example, the largest aging dataset “Morph” [11] only
captured images with an average time span of 164 days for each individual. In
addition, existing works also require the query image to be labeled with the
true age, which can be inconvenient from time to time. Given the training
data, existing works normally divide them into different age groups and learn
a transformation between the groups, therefore, the query image has to be
labeled in order to correctly position the image.

Although age progression and regression are equally important, most existing
works focus on age progression. Very few works can achieve good performance of
face rejuvenating, especially for rendering baby face from an adult because
they are mainly surface-based modeling which simply remove the texture from a
given image [18, 14, 7]. On the other hand, researchers have made great
progress on age progression. For example, the physical model-based methods
[27, 26, 14, 22] parametrically model biological facial change with age,
_e.g_., muscle, wrinkle, skin, etc. However, they suffer from complex
modeling, the requirement of sufficient dataset to cover long time span, and
are computationally expensive; the prototype-based methods [28, 11, 24, 29]
tend to divide training data into different age groups and learn a
transformation between groups. However, some can preserve personality but
induce severe ghosting artifacts, others smooth out the ghosting effect but
lose personality, while most relaxed the requirement of paired images over
long time span, and the aging pattern can be learned between two adjacent age
groups. Nonetheless, they still need paired samples over short time span.

In this paper, we investigate the age progression/regression problem from the
perspective of generative modeling. The rapid development of generative
adversarial networks (GANs) has shown impressive results in face image
generation [19, 31, 21, 17]. In this paper, we assume that the face images lie
on a high-dimensional manifold as shown in Fig.1. Given a query face, we could
find the corresponding point (face) on the manifold. Stepping along the
direction of age changing, we will obtain the face images of different ages
while preserving personality. We propose a conditional adversarial autoencoder
(CAAE)111Bitbucket: https://bitbucket.org/aicip/face-aging-caae
Github: https://zzutk.github.io/Face-Aging-CAAE network to learn the face
manifold. By controlling the age attribute, it will be flexible to achieve age
progression and regression at the same time. Because it is difficult to
directly manipulate on the high-dimensional manifold, the face is first mapped
to a latent vector through a convolutional encoder, and then the vector is
projected to the face manifold conditional on age through a deconvolutional
generator. Two adversarial networks are imposed on the encoder and generator,
respectively, forcing to generate more photo-realistic faces.

The benefit of the proposed CAAE can be summarized from four aspects. First,
the novel network architecture achieves both age progression and regression
while generating photo-realistic face images. Second, we deviate from the
popular group-based learning, thus not requiring paired samples in the
training data or labeled face in the test data, making the proposed framework
much more flexible and general. Third, the disentanglement of age and
personality in the latent vector space helps preserving personality while
avoiding the ghosting artifacts. Finally, CAAE is robust against variations in
pose, expression, and occlusion.

##  2 Related Work

###  2.1 Age Progression and Regression

In recent years, the study on face age progression has been very popular, with
approaches mainly falling into two categories, physical model-based and
prototype-based. Physical model-based methods model the biological pattern and
physical mechanisms of aging, _e.g_., the muscles [25], wrinkle [23, 26],
facial structure [22, 14] etc. through either parametric or non-parametric
learning. However, in order to better model the subtle aging mechanism, it
will require a large face dataset with long age span (_e.g_., from 0 to 80
years old) of each individual, which is very difficult to collect. In
addition, physical modeling-based approaches are computationally expensive.

On the other hand, prototype-based approaches [1, 11] often divide faces into
groups by age, e.g., the average face of each group, as its prototype. Then,
the difference between prototypes from two age groups is considered the aging
pattern. However, the aged face generated from averaged prototype may lose the
personality (_e.g_., wrinkles). To preserve the personality, [24] proposed a
dictionary learning based method — age pattern of each age group is learned
into the corresponding sub-dictionary. A given face will be decomposed into
two parts: age pattern and personal pattern. The age pattern was transited to
the target age pattern through the sub-dictionaries, and then the aged face is
generated by synthesizing the personal pattern and target age pattern.
However, this approach presents serious ghosting artifacts. The deep learning-
based method [29] represents the state-of-the-art, where RNN is applied on the
coefficients of eigenfaces for age pattern transition. All prototype-based
approaches perform the group-based learning which requires the true age of
testing faces to localize the transition state which might not be convenient.
In addition, these approaches only provide age progression from younger face
to older ones. To achieve flexible bidirectional age changes, it may need to
retrain the model inversely.

Face age regression, which predicts the rejuvenating results, is comparatively
more challenging. Most age regression works so far [18, 7] are physical model-
based, where the textures are simply removed based on the learned
transformation over facial surfaces. Therefore, they cannot achieve photo-
realistic results for baby face predictions.

###  2.2 Generative Adversarial Network

Generating realistically appealing images is still challenging and has not
achieved much success until the rapid advancement of the generative
adversarial network (GAN). The original GAN work [8] introduced a novel
framework for training generative models. It simultaneously trains two models:
1) the generative model $G$ captures the distribution of training samples and
learns to generate new samples imitating the training, and 2) the
discriminative model $D$ discriminates the generated samples from the
training. $G$ and $D$ compete with each other using a min-max game as Eq. 1,
where $z$ denotes a vector randomly sampled from certain distribution
$p(\mathbf{z})$ (_e.g_., Gaussian or uniform), and the data distribution is
$p_{data}(\mathbf{x})$, _i.e_., the training data $x\sim
p_{data}(\mathbf{x})$.

$\centering\begin{split}\underset{G}{\min}\;\underset{D}{\max}\;&\mathbb{E}_{x\sim
p_{data}(\mathbf{x})}[\log D(x)]+\\\ &\mathbb{E}_{z\sim
p(\mathbf{z})}[\log(1-D(G(z)))]\end{split}\@add@centering$ (1)

The two parts, $G$ and $D$, are trained alternatively.

One of the biggest issues of GAN is that the training process is unstable, and
the generated images are often noisy and incomprehensible. During the last two
years, several approaches [21, 20, 9, 3, 4, 10, 19] have been proposed to
improve the original GAN from different perspectives. For example, DCGAN [21]
adopted deconvolutional and convolutional neural networks to implement $G$ and
$D$, respectively. It also provided empirical instruction on how to build a
stable GAN, _e.g_., replacing the pooling by strides convolution and using
batch normalization. CGAN [20] modified GAN from unsupervised learning into
semi-supervised learning by feeding the conditional variable (_e.g_., the
class label) into the data. The low resolution of the generated image is
another common drawback of GAN. [4, 10] extended GAN into sequential or
pyramid GANs to handle this problem, where the image is generated step by
step, and each step utilizes the information from the previous step to further
improve the image quality. Some GAN-related works have shown visually
impressive results of randomly drawing face images [30, 19, 31, 21, 17].
However, GAN generates images from random noise, thus the output image cannot
be controlled. This is undesirable in age progression and regression, where we
have to ensure the output face looks like the same person as queried.

##  3 Traversing on the Manifold

We assume the face images lie on a high-dimensional manifold, on which
traversing along certain direction could achieve age progression/regression
while preserving the personality. This assumption will be demonstrated
experimentally in Sec. 4.2. However, modeling the high-dimensional manifold is
complicated, and it is difficult to directly manipulate (traversing) on the
manifold. Therefore, we will learn a mapping between the manifold and a lower-
dimensional space, referred to as the latent space, which is easier to
manipulate. As illustrated in Fig. 2, faces $x_{1}$ and $x_{2}$ are mapped to
the latent space by $E$ (_i.e_., an encoder), which extracts the personal
features $z_{1}$ and $z_{2}$, respectively. Concatenating with the age labels
$l_{1}$ and $l_{2}$, two points are generated in the latent space, namely
$[z_{1},l_{1}]$ and $[z_{2},l_{2}]$. Note that the personality $z$ and age $l$
are disentangled in the latent space, thus we could simply modify age while
preserving the personality. Starting from the red rectangular point
$[z_{2},l_{2}]$ (corresponding to $x_{2}$) and evenly stepping bidirectionally
along the age axis (as shown by the solid red arrows), we could obtain a
series of new points (red circle points). Through another mapping $G$ (_i.e_.
a generator), those points are mapped to the manifold $\mathcal{M}$ –
generating a series of face images, which will present the age
progression/regression of $x_{2}$. By the same token, the green points and
arrows demonstrate the age progressing/regression of $x_{1}$ based on the
learned manifold and the mappings. If we move the point along the dotted arrow
in the latent space, both personality and age will be changed as reflected on
$\mathcal{M}$. We will learn the mappings $E$ and $G$ to ensure the generated
faces lie on the manifold, which indicates that the generated faces are
realistic and plausible for a given age.

Figure 2: Illustration of traversing on the face manifold $\mathcal{M}$. The
input faces $x_{1}$ and $x_{2}$ are encoded to $z_{1}$ and $z_{2}$ by an
encoder $E$, which represents the personality. Concatenated by random age
labels $l_{1}$ and $l_{2}$, the latent vectors $[z_{1},l_{1}]$ and
$[z_{2},l_{2}]$ are constructed as denoted by the rectangular points. The
colors indicate correspondence of personality. Arrows and circle points denote
the traversing direction and steps, respectively. Solid arrows direct
traversing along the age axis while preserving the personality. The dotted
arrow performs a traversing across both the age and personality axes. The
traversing in the latent space is mapped to the face manifold $\mathcal{M}$ by
a generator $G$, as illustrated by the points and arrows with corresponding
markers and colors. Each point on $\mathcal{M}$ is a face image, thus
achieving age progression and regression. Figure 3: Structure of the proposed
CAAE network for age progression/regression. The encoder $E$ maps the input
face to a vector $z$ (personality). Concatenating the label $l$ (age) to $z$,
the new latent vector $[z,l]$ is fed to the generator $G$. Both the encoder
and the generator are updated based on the $L_{2}$ loss between the input and
output faces. The discriminator $D_{z}$ imposes the uniform distribution on
$z$, and the discriminator $D_{img}$ forces the output face to be photo-
realistic and plausible for a given age label.

##  4 Approach

In this section, we first present the pipeline of the proposed conditional
adversarial autoencoder (CAAE) network (Sec. 4.1) that learns the face
manifold (Sec. 4.2). The CAAE incorporates two discriminator networks, which
are the key to generating more realistic faces. Sections 4.3 and 4.4
demonstrate their effectiveness, respectively. Finally, Section 4.5 discusses
the difference of the proposed CAAE from other generative models.

###  4.1 Conditional Adversarial Autoencoder

The detailed structure of the propose CAAE network is shown in Fig. 3. The
input and output face images are $128\times 128$ RGB images
$x\in\mathbb{R}^{128\times 128\times 3}$. A convolutional neural network is
adopted as the encoder. The convolution of stride 2 is employed instead of
pooling (_e.g_., max pooling) because strided convolution is fully
differentiable and allows the network to learn its own spacial downsampling
[21]. The output of encoder $E(x)=z$ preserves the high-level personal feature
of the input face $x$. The output face conditioned on certain age can be
expressed by $G(z,l)=\hat{x}$, where $l$ denotes the one-hot age label. Unlike
existing GAN-related works, we incorporate an encoder to avoid random sampling
of $z$ because we need to generate a face with specific personality which is
incorporated in $z$. In addition, two discriminator networks are imposed on
$E$ and $G$, respectively. The $D_{z}$ regularizes $z$ to be uniform
distributed, smoothing the age transformation. The $D_{img}$ forces $G$ to
generate photo-realistic and plausible faces for arbitrary $z$ and $l$. The
effectiveness of the two discriminators will be further discussed in Secs. 4.3
and 4.4, respectively.

###  4.2 Objective Function

The real face images are supposed to lie on the face manifold $\mathcal{M}$,
so the input face image $x\in\mathcal{M}$. The encoder $E$ maps the input face
$x$ to a feature vector, _i.e_., $E(x)=z\in\mathbb{R}^{n}$, where $n$ is the
dimension of the face feature. Given $z$ and conditioned on certain age label
$l$, the generator $G$ generates the output face $\hat{x}=G(z,l)=G(E(x),l)$.
Our goal is to ensure the output face $\hat{x}$ lies on the manifold while
sharing the personality and age with the input face $x$ (during training).
Therefore, the input and output faces are expected to be similar as expressed
in Eq. 2, where $\mathcal{L}(\cdot,\cdot)$ denotes L2 norm.

$\centering\underset{E,G}{\min}\;\mathcal{L}\left(x,G(E(x),l)\right)\@add@centering$
(2)

Simultaneously, the uniform distribution is imposed on $z$ through $D_{z}$ –
the discriminator on $z$. We denote the distribution of the training data as
$p_{data}(\mathbf{x})$, then the distribution of $z$ is
$q(\mathbf{z}|\mathbf{x})$. Assuming $p(\mathbf{z})$ is a prior distribution,
and $z^{*}\sim p(\mathbf{z})$ denotes the random sampling process from
$p(\mathbf{z})$. A min-max objective function can be used to train $E$ and
$D_{z}$,

$\centering\begin{split}\underset{E}{\min}\;\underset{D_{z}}{\max}\;&\mathbb{E}_{z^{*}\sim
p(\mathbf{z})}\left[\log D_{z}(z^{*})\right]+\\\ ~{}&\mathbb{E}_{x\sim
p_{data}(\mathbf{x})}\left[\log(1-D_{z}(E(x)))\right]\end{split}\@add@centering$
(3)

By the same token, the discriminator on face image, $D_{img}$, and $G$ with
condition $l$ can be trained by

$\centering\begin{split}\underset{G}{\min}&\;\underset{D_{img}}{\max}\;\mathbb{E}_{x,l\sim
p_{data}(\mathbf{x,l})}\left[\log D_{img}(x,l)\right]+\\\
~{}&\mathbb{E}_{x,l\sim
p_{data}(\mathbf{x,\l})}\left[\log(1-D_{img}(G(E(x),l)))\right]\end{split}\@add@centering$
(4)

Finally the objective function becomes

$\centering\begin{split}\underset{E,G}{\min}\;&\underset{D_{z},D_{img}}{\max}\lambda\mathcal{L}\left(x,G(E(x),l)\right)+\gamma
TV(G(E(x),l))\\\ +&\mathbb{E}_{z^{*}\sim p(\mathbf{z})}\left[\log
D_{z}(z^{*})\right]\\\ +&\mathbb{E}_{x\sim
p_{data}(\mathbf{x})}\left[\log(1-D_{z}(E(x)))\right]\\\ +&\mathbb{E}_{x,l\sim
p_{data}(\mathbf{x,l})}\left[\log D_{img}(x,l)\right]\\\ +&\mathbb{E}_{x,l\sim
p_{data}(\mathbf{x,\l})}\left[\log(1-D_{img}(G(E(x),l)))\right],\end{split}\@add@centering$
(5)

where $TV(\cdot)$ denotes the total variation which is effective in removing
the ghosting artifacts. The coefficients $\lambda$ and $\gamma$ balance the
smoothness and high resolution.

Note that the age label is resized and concatenated to the first convolutional
layer of $D_{img}$ to make it discriminative on both age and human face.
Sequentially updating the network by Eqs. 2, 3, and 4, we could finally learn
the manifold $\mathcal{M}$ as illustrated in Fig. 4.

Figure 4: Illustration of the learned face manifold $\mathcal{M}$. The
horizontal axis indicates the traversing of age, and the vertical axis
indicates different personality.

###  4.3 Discriminator on $\boldsymbol{z}$

The discriminator on $z$, denoted by $D_{z}$, imposes a prior distribution
(_e.g_., uniform distribution) on $z$. Specifically, $D_{z}$ aims to
discriminate the $z$ generated by encoder $E$. Simultaneously, $E$ will be
trained to generate $z$ that could fool $D_{z}$. Such adversarial process
forces the distribution of the generated $z$ to gradually approach the prior.
We use uniform distribution as the prior, forcing $z$ to evenly populate the
latent space with no apparent “holes”. As shown in Fig. 5, the generated $z$’s
(depicted by blue dots in a 2-D space) present uniform distribution under the
regularization of $D_{z}$, while the distribution of $z$ exhibits a “hole”
without the application of $D_{z}$.

Figure 5: Effect of $D_{z}$, which forces $z$ to a uniform distribution. For
simplicity, $z$ is illustrated in a 2-D space. Blue dots indicate $z$’s mapped
from training faces through the encoder. With $D_{z}$, the distribution of $z$
will approach uniform. Otherwise, $z$ may present “holes”. The rectangular
points denote the corresponding $z$ mapped from the input faces $x_{1}$ and
$x_{2}$, and the dotted arrow indicates the traversing from $z_{1}$ to
$z_{2}$. The intermediate points along the traversing are supposed to generate
a series of plausible morphing faces from $x_{1}$ to $x_{2}$. Without $D_{z}$,
the learned $z$ presents a sparse distribution along the path of traversing,
causing the generated face to look unreal. The series of figures at the bottom
shows the traversing with and without $D_{z}$.

Exhibition of the “hole” indicates that face images generated by interpolating
between arbitrary $z$’s may not lie on the face manifold – generating
unrealistic faces. For example, given two faces $x_{1}$ and $x_{2}$ as shown
in Fig. 5, we obtain the corresponding $z_{1}$ and $z_{2}$ by $E$ under the
conditions with and without $D_{z}$, respectively. Interpolating between
$z_{1}$ and $z_{2}$ (dotted arrows in Fig. 5), the generated faces are
expected to show realistic and smooth morphing from $x_{1}$ to $x_{2}$ (bottom
of Fig. 5). However, the morphing without $D_{z}$ actually presents distorted
(unrealistic) faces in the middle (indicated by dashed box), which corresponds
to the interpolated $z$’s passing through the “hole”.

###  4.4 Discriminator on Face Images

Inheriting the similar principle of GAN, the discriminator $D_{img}$ on face
images forces the generator to yield more realistic faces. In addition, the
age label is imposed on $D_{img}$ to make it discriminative against unnatural
faces conditional on age. Although minimizing the distance between the input
and output images as expressed in Eq. 2 forces the output face to be close to
the real ones, Eq. 2 does not ensure the framework to generate plausible faces
from those unsampled faces. For example, given a face that is unseen during
training and a random age label, the pixel-wise loss could only make the
framework generate a face close to the trained ones in a manner of
interpolation, causing the generated face to be very blurred. The $D_{img}$
will discriminate the generated faces from real ones in aspects of reality,
age, resolution, etc. Fig. 6 demonstrates the effect of $D_{img}$.

Figure 6: Effect of $D_{img}$, which forces the generated faces to be more
realistic in aspects of age and resolution. The first column shows the
original faces, and their true ages are marked on the top. The right four
columns are generated faces through the proposed framework, without (the upper
row) or with (the lower row) $D_{img}$. The generated faces fall in four age
groups as indicated at the top of each column.

Comparing the generated faces with and without $D_{img}$, it is obvious that
$D_{img}$ assists the framework to generate more realistic faces. The outputs
without $D_{img}$ could also present aging but the effect is not as obvious as
that with $D_{img}$ because $D_{img}$ enhances the texture especially for
older faces.

Figure 7: Comparison to prior works of face aging. The first column shows
input faces, and second column are the best aged faces cited from prior works.
The rest columns are our results from both age progression and regression. The
red boxes indicate the comparable results to the prior works.

###  4.5 Differences from Other Generative Networks

In this section, we comment on the similarity and difference of the proposed
CAAE with other generative networks, including GAN [8], variational
autoencoder (VAE) [13], and adversarial autoencoder (AAE) [19].

VAE vs. GAN: VAE uses a recognition network to predict the posterior
distribution over the latent variables, while GAN uses an adversarial training
procedure to directly shape the output distribution of the network via back-
propagation [19]. Because VAE follows an encoding-decoding scheme, we can
directly compare the generated images to the inputs, which is not possible
when using a GAN. A downside of VAE is that it uses mean squared error instead
of an adversarial network in image generation, so it tends to produce more
blurry images [15].

AAE vs. GAN and VAE: AAE can be treated as the combination of GAN and VAE,
which maintains the autoencoder network like VAE but replaces the KL-
divergence loss with an adversarial network like in GAN. Instead of generating
images from random noise as in GAN, AAE utilizes the encoder part to learn the
latent variables approximated on certain prior, making the style of generated
images controllable. In addition, AAE better captures the data manifold
compared to VAE.

CAAE vs. AAE: The proposed CAAE is more similar to AAE. The main difference
from AAE is that the proposed CAAE imposes discriminators on the encoder and
generator, respectively. The discriminator on encoder guarantees smooth
transition in the latent space, and the discriminator on generator assists to
generate photo-realistic face images. Therefore, CAAE would generate higher
quality images than AAE as discussed in Sec. 4.4.

##  5 Experimental Evaluation

In the section, we will first clarify the process of data collection (Sec.
5.1) and implementation of the proposed CAAE (Sec. 5.2). Then, both
qualitative and quantitative comparisons with prior works and ground truth are
performed in Sec. 5.3. Finally, the tolerance to occlusion and variation in
pose and expression is illustrated in Sec. 5.4 .

###  5.1 Data Collection

We first collect face images from the Morph dataset [11] and the CACD [2]
dataset. The Morph dataset [11] is the largest with multiple ages of each
individual, including 55,000 images of 13,000 subjects from 16 to 77 years
old. The CACD [2] dataset contains 13,446 images of 2,000 subjects. Because
both datasets have limited images from newborn or very old faces, we crawl
images from Bing and Google search engines based on the keywords, _e.g_.,
baby, boy, teenager, 15 years old, etc. Because the proposed approach does not
require multiple faces from the same subject, we simply randomly choose around
3,000 images from the Morph and CACD dataset and crawl 7,670 images from the
website. The age and gender of the crawled faces are estimated based on the
image caption or the result from age estimator [16]. We divide the age into
ten categories, _i.e_., 0–5, 6–10, 11–15, 16–20, 21–30, 31–40, 41–50, 51–60,
61–70, and 71–80. Therefore, we can use a one-hot vector of ten elements to
indicate the age of each face during training. The final dataset consists of
10,670 face images with a uniform distribution on gender and age. We use the
face detection algorithm with 68 landmarks [5] to crop out and align the
faces, making the training more attainable.

###  5.2 Implementation of CAAE

We construct the network according to Fig. 3 with kernel size of $5\times 5$.
The pixel values of the input images are normalized to $[-1,1]$, and the
output of $E$ (_i.e_., $z$) is also restricted to $[-1,1]$ by the hyperbolic
tangent activation function. Then, the desired age label, the one-hot vector,
is concatenated to $z$, constructing the input of $G$. To make fair
concatenation, the elements of label is also confined to $[-1,1]$, where -1
corresponds to 0. Finally, the output is also in range $[-1,1]$ through the
hyperbolic tangent function. Normalizing the input may make the training
process converge faster. Note that we will not use the batch normalization for
$E$ and $G$ because it blurs personal features and makes output faces drift
far away from inputs in testing. However, the batch normalization will make
the framework more stable if it is applied on $D_{img}$. All intermediate
layers of each block (_i.e_., $E$, $G$, $D_{z}$, and $D_{img}$) use the ReLU
activation function.

In training, $\lambda=100$, $\gamma=10$, and the four blocks are updated
alternatively with a mini-batch size of 100 through the stochastic gradient
descent solver, ADAM [12] ($\alpha=0.0002$, $\beta_{1}=0.5$). Face and age
pairs are fed to the network. After about 50 epochs, plausible generated faces
can be obtained. During testing, only $E$ and $G$ are active. Given an input
face without true age label, $E$ maps the image to $z$. Concatenating an
arbitrary age label to $z$, $G$ will generate a photo-realistic face
corresponding to the age and personality.

###  5.3 Qualitative and Quantitative Comparison

To evaluate that the proposed CAAE can generate more photo-realistic results,
we compare ours with the ground truth and the best results from prior works
[29, 11, 24, 26], respectively. We choose FGNET [14] as the testing dataset,
which has 1002 images of 82 subjects aging from 0 to 69.

Comparison with ground truth: In order to verify whether the personality has
been preserved by the proposed CAAE, we qualitatively and quantitatively
compare the generated faces with the ground truth. The qualitative comparison
is shown in Fig. 8, which shows appealing similarity. To quantitatively
evaluate the performance, we pair the generated faces with the ground truth
whose age gap is larger than 20 years. There are 856 pairs in total. We design
a survey to compare the similarity where 63 volunteers participate. Each
volunteer is presented with three images, an original image X, a generated
image A, and the corresponding ground truth image B under the same group. They
are asked whether the generated image A looks similar to the ground truth B;
or not sure. We ask the volunteers to randomly choose 45 questions and leave
the rest blank. We receive 3208 votes in total, with 48.38% indicating that
the generated image A is the same person as the ground truth, 29.58%
indicating they are not, and 22.04% not sure. The voting results demonstrate
that we can effectively generate photo-realistic faces under different ages
while preserving their personality.

Figure 8: Comparison to the ground truth.

Comparison with prior work: We compare the performance of our method with some
prior works [29, 11, 24, 26], for face age progression and Face Transformer
(FT) [6] for face age regression. To demonstrate the advantages of CAAE, we
use the same input images collected from those prior works and perform long
age span progression. To compare with prior works, we cite their results as
shown in Fig. 7. We also compare with age regression works using the FT demo
[6] as shown in Fig. 9. Our results obviously show higher fidelity,
demonstrating the capability of CAAE in achieving smooth face aging and
rejuvenation. CAAE better preserves the personality even with a long age span.
In addition, our results provide richer texture (_e.g_., wrinkle for old
faces), making old faces look more realistic. Another survey is conducted to
statistically evaluate the performance as compared with prior works, where for
each testing image, the volunteer is to select the better result from CAAE or
prior works, or hard to tell. We collect 235 paired images of 79 subjects from
previous works [29, 11, 24, 26]. We receive 47 responses and 1508 votes in
total with 52.77% indicating CAAE is better, 28.99% indicating the prior work
is better, and 18.24% indicating they are equal. This result further verifies
the superior performance of the proposed CAAE.

Figure 9: Comparison to prior work in rejuvenation. The first row shows the
input faces, the middle row shows the baby faces generated by FT [6] and the
last row shows our results.

###  5.4 Tolerance to Pose, Expression, and Occlusion

As mentioned above, the input images have large variation in pose, expression,
and occlusion. To demonstrate the robustness of CAAE, we choose the faces with
expression variation, non-frontal pose, and occlusion, respectively, as shown
in Fig. 10. It is worth noting that the previous works [29, 11] often apply
face normalization to alleviate from the variation of pose and expression but
they may still suffer from the occlusion issue. In contrast, the proposed CAAE
obtains the generated faces without the need to remove these variations,
paving the way to robust performance in real applications.

Figure 10: Tolerance to occlusion and variation in pose and expression. The
very left column shows the input faces, and the right columns are generated
faces by CAAE from younger to older ages. The first input face presents
relatively more dramatic expression, the second input shows only the face
profile, and the last one is partially occluded by facial marks.

##  6 Discussion and Future Works

In this paper, we proposed a novel conditional adversarial autoencoder (CAAE),
which first achieves face age progression and regression in a holistic
framework. We deviated from the conventional routine of group-based training
by learning a manifold, making the aging progression/regression more flexible
and manipulatable — from an arbitrary query face without knowing its true age,
we can freely produce faces at different ages, while at the same time
preserving the personality. We demonstrated that with two discriminators
imposed on the generator and encoder, respectively, the framework generates
more photo-realistic faces. Flexibility, effectiveness, and robustness of CAAE
have been demonstrated through extensive evaluation.

The proposed framework has great potential to serve as a general framework for
face-age related tasks. More specifically, we trained four sub-networks,
_i.e_., $E$, $G$, $D_{z}$, and $D_{img}$, but only $E$ and $G$ are utilized in
the testing stage. The $D_{img}$ is trained conditional on age. Therefore, it
is able to tell whether the given face corresponds to a certain age, which is
exactly the task of age estimation. For the encoder $E$, it maps faces to a
latent vector (face feature), which preserves the personality regardless of
age. Therefore, $E$ could be considered a candidate for cross-age recognition.
The proposed framework could be easily applied to other image generation
tasks, where the characteristics of the generated image can be controlled by
the conditional label. In the future, we would extend current work to be a
general framework, simultaneously achieving age progressing ($E$ and $G$),
cross-age recognition ($E$), face morphing ($G$), and age estimation
($D_{img}$).

## References

  * [1] D. M. Burt and D. I. Perrett.  Perception of age in adult caucasian male faces: Computer graphic manipulation of shape and colour information.  Proceedings of the Royal Society of London B: Biological Sciences, 259(1355):137–143, 1995.
  * [2] B.-C. Chen, C.-S. Chen, and W. H. Hsu.  Cross-age reference coding for age-invariant face recognition and retrieval.  In Proceedings of the European Conference on Computer Vision, 2014\.
  * [3] X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel.  InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets.  In Advances in Neural Information Processing Systems, 2016.
  * [4] E. L. Denton, S. Chintala, R. Fergus, et al.  Deep generative image models using a￼ laplacian pyramid of adversarial networks.  In Advances in Neural Information Processing Systems, pages 1486–1494, 2015.
  * [5] Dlib C++ Library.  http://dlib.net/.  [Online].
  * [6] Face Transformer (FT) demo.  http://cherry.dcs.aber.ac.uk/transformer/.  [Online].
  * [7] Y. Fu and N. Zheng.  M-face: An appearance-based photorealistic model for multiple facial attributes rendering.  IEEE Transactions on Circuits and Systems for Video Technology, 16(7):830–842, 2006.
  * [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.  Generative adversarial nets.  In Advances in Neural Information Processing Systems, pages 2672–2680, 2014.
  * [9] K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra.  Draw: A recurrent neural network for image generation.  arXiv preprint arXiv:1502.04623, 2015.
  * [10] D. J. Im, C. D. Kim, H. Jiang, and R. Memisevic.  Generating images with recurrent adversarial networks.  arXiv preprint arXiv:1602.05110, 2016.
  * [11] I. Kemelmacher-Shlizerman, S. Suwajanakorn, and S. M. Seitz.  Illumination-aware age progression.  In IEEE Conference on Computer Vision and Pattern Recognition, pages 3334–3341. IEEE, 2014.
  * [12] D. Kingma and J. Ba.  ADAM: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [13] D. P. Kingma and M. Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * [14] A. Lanitis, C. J. Taylor, and T. F. Cootes.  Toward automatic simulation of aging effects on face images.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4):442–455, 2002.
  * [15] A. B. L. Larsen, S. K. Sønderby, and O. Winther.  Autoencoding beyond pixels using a learned similarity metric.  arXiv preprint arXiv:1512.09300, 2015.
  * [16] G. Levi and T. Hassner.  Age and gender classification using convolutional neural networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 34–42, 2015.
  * [17] M. Y. Liu and O. Tuzel.  Coupled generative adversarial networks.  In Advances in neural information processing systems, 2016.
  * [18] Z. Liu, Z. Zhang, and Y. Shan.  Image-based surface detail transfer.  IEEE Computer Graphics and Applications, 24(3):30–35, 2004.
  * [19] A. Makhzani, J. Shlens, N. Jaitly, and I. Goodfellow.  Adversarial autoencoders.  In International Conference on Learning Representations, 2016.
  * [20] M. Mirza and S. Osindero.  Conditional generative adversarial nets.  arXiv preprint arXiv:1411.1784, 2014.
  * [21] A. Radford, L. Metz, and S. Chintala.  Unsupervised representation learning with deep convolutional generative adversarial networks.  In International Conference on Learning Representations, 2016.
  * [22] N. Ramanathan and R. Chellappa.  Modeling age progression in young faces.  In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1, pages 387–394. IEEE, 2006.
  * [23] N. Ramanathan and R. Chellappa.  Modeling shape and textural variations in aging faces.  In IEEE International Conference on Automatic Face & Gesture Recognition, pages 1–8. IEEE, 2008.
  * [24] X. Shu, J. Tang, H. Lai, L. Liu, and S. Yan.  Personalized age progression with aging dictionary.  In Proceedings of the IEEE International Conference on Computer Vision, pages 3970–3978, 2015.
  * [25] J. Suo, X. Chen, S. Shan, W. Gao, and Q. Dai.  A concatenational graph evolution aging model.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2083–2096, 2012.
  * [26] J. Suo, S.-C. Zhu, S. Shan, and X. Chen.  A compositional and dynamic model for face aging.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3):385–401, 2010.
  * [27] Y. Tazoe, H. Gohara, A. Maejima, and S. Morishima.  Facial aging simulator considering geometry and patch-tiled texture.  In ACM SIGGRAPH 2012 Posters, page 90. ACM, 2012.
  * [28] B. Tiddeman, M. Burt, and D. Perrett.  Prototyping and transforming facial textures for perception research.  IEEE Computer Graphics and Applications, 21(5):42–50, 2001.
  * [29] W. Wang, Z. Cui, Y. Yan, J. Feng, S. Yan, X. Shu, and N. Sebe.  Recurrent face aging.  In IEEE Conference on Computer Vision and Pattern Recognition, pages 2378–2386. IEEE, 2016.
  * [30] X. Yu and F. Porikli.  Ultra-resolving face images by discriminative generative networks.  In European Conference on Computer Vision, pages 318–333. Springer, 2016.
  * [31] J. Zhao, M. Mathieu, and Y. LeCun.  Energy-based generative adversarial network.  arXiv preprint arXiv:1609.03126, 2016.