# OR-Net: Pointwise Relational Inference for Data Completion under Partial
Observation

Qianyu Feng  University of Technology Sydney ,  Linchao Zhu  University of
Technology Sydney ,  Bang Zhang  Alibaba DAMO Academy ,  Pan Pan  Alibaba DAMO
Academy and  Yi Yang  University of Technology Sydney

(2021)

###### Abstract.

Contemporary data-driven methods are typically fed with full supervision on
large-scale datasets which limits their applicability. However, in the actual
systems with limitations such as measurement error and data acquisition
problems, people usually obtain incomplete data. Although data completion has
attracted wide attention, the underlying data pattern and relativity are still
under-developed. Currently, the family of latent variable models allows
learning deep latent variables over observed variables by fitting the marginal
distribution. As far as we know, current methods fail to perceive the data
relativity under partial observation. Aiming at modeling incomplete data, this
work uses relational inference to fill in the incomplete data. Specifically,
we expect to approximate the real joint distribution over the partial
observation and latent variables, thus infer the unseen targets respectively.
To this end, we propose Omni-Relational Network (OR-Net) to model the
pointwise relativity in two aspects: (i) On one hand, the inner relationship
is built among the context points in the partial observation; (ii) On the
other hand, the unseen targets are inferred by learning the cross-relationship
with the observed data points. It is further discovered that the proposed
method can be generalized to different scenarios regardless of whether the
physical structure can be observed or not. It is demonstrated that the
proposed OR-Net can be well generalized for data completion tasks of various
modalities, including function regression, image completion on MNIST and
CelebA datasets, and also sequential motion generation conditioned on the
observed poses.

Conditional Generation, Data Completion, Variational Auto-encoders

††copyright: acmcopyright††journalyear: 2021††doi:
10.1145/1122445.1122456††conference: ; ; ††submissionid: 1975††ccs: Theory of
computation Semi-supervised learning††ccs: Theory of computation Models of
learning††ccs: Theory of computation Structured prediction Figure 1.
Different application scenarios for data completion with conditional
generation under partial observation. Take image completion for example, it is
widely applied when images are damaged with defect points. Besides, it is
applicable for modeling the function in systems where only few variables can
be measured. We further investigate the completion of motion sequences with
several observed poses as the context.

##  1\. Introduction

Deep learning has evolved the data-driven methods that rely on rich
supervision in large-scale datasets (Krizhevsky et al., 2012; Lin et al.,
2014; Krizhevsky et al., 2009). However, these methods suffer from over-
fitting the training data and thus being vulnerable to attacks, which impedes
the generalization ability. Stochastic processes (Insua et al., 2012; Lei et
al., 2016; Ye and Xie, 2015; Easley and Kiefer, 1988) provide alternative
options to cover the distribution of data samples. With the estimated
distribution conditioned on the context samples, the unseen targets can be
inferred from the partial observation. A collection of models are devised to
tackle these issues via learning a distribution over predictors, which allows
incorporating data from observation and related tasks. However, traditional
approaches are computationally expensive and intractable to be applied to
large datasets. The family of neural processes (Garnelo et al., 2018a, b; Kim
et al., 2019) are proposed to leverage the efficiency and scalability of
neural network to parameterize the data distribution with a set of input-
output pairs as exemplars. Nonetheless, they merely focus on learning the
appearance features while overlooking the relativity among data samples, thus
tend to fail to generalize to different modalities under various scenarios.

Differently, we aim to take a step further to learn the joint distribution
over both observed and latent variables. It is challenging to learn the
underlying joint distribution which assists model to approximate prior and
posterior distributions over latent variables. Learning these distributions
can be beneficial for various purposes, for example, inferring the latent
distribution where data is originated. The critical limitation in the existing
architectures (Ye and Xie, 2015; Garnelo et al., 2018b; Kim et al., 2019) is
that they fail to build induction across a broader context due to the
unobserved heterogeneity. It is widely adopted that stacks of
feature‐extracting layers compose the model, with the first layer processing
raw inputs and the subsequent layers gradually learning more complex
representation given a set of input/output pairs. (Kim et al., 2019) tried to
apply the attention mechanism into the neural processes to capture the linear
mapping of data points. However, the linear attentive layers fail to satisfy
diverse cases thus lead to blurry results for data completion. Therefore, we
consider modeling the relationship in data points with functions incorporating
prior knowledge, i.e., position information, geometric structure, extracting
from the neighboring context. Unlike convolutional neural network (CNN) (LeCun
et al., 1989; Krizhevsky et al., 2012) and recurrent neural network (RNN)
(Hochreiter and Schmidhuber, 1997; Elman, 1990) exploiting simple redundancies
and invariances in data, GNN functions by directly encoding the structural
information presented in data. Long-range dependencies in the nonstructural
data can be better captured via graph structures, allowing the information to
propagate for multiple-hops hierarchically. An implication of this is that,
among data points sampled from the distribution, they merely use the distance
information to re-weight the importance of messages passed from the neighbors.
This frequently leads to introduce blurring artifacts in incomplete or missing
contexts.

Inspired by these works, our main idea is to leverage the structural
relationship in data points for learning the target samples w.r.t. other
observed samples in the context set. To learn such embedding, we first connect
each data point to its context points in the neighborhood, then learn the
relative information for each pair. Taking in both the appearance embedding
and geometric embedding, an attentive aggregation scheme further weighs the
neighbors to perform the message passing. Such aggregations can be naturally
chained and combined into multiple layers to enhance the expressiveness of
representation. In the reconstruction of unseen target samples, the geometric
embedding is inductive across different node orderings. When other attributes
are available, the node embedding is further enriched by aggregating messages
from its neighborhood.

Therefore, the proposed OR-Net is equipped with the following desirable
traits: (i) Structure-aware: We devise a new realization of neural processes
to incorporate the geometric information which is more expressive by capturing
the relativity in the data points; (ii) Reliability: We propose to learn a
compact representation via reducing the redundant information for learning an
optimal estimation of data distribution; (iii) Better Generalization: OR-Net
is directly applicable to data completion under different scenarios, e.g., 1D
function, imperfect images and human motion sequences, where it can better
estimate the unseen samples. Furthermore, we show that OR-Net is expressive
and achieves competitive performances on multifold tasks, including function
regression, image completion and motion generation.

##  2\. Related Work

Implicit Learning. Albeit the remarkable successes reached by deep learning
over these years, the efficiency of learning (Lake et al., 2015; Santoro et
al., 2016) on large datasets still remains a long-standing problem. One
possible solution is to model function $f$ on the data distribution using only
an arbitrary number of samples by exploiting the domain-wide statistics.
Gaussian Processes (Williams and Rasmussen, 1996) capture the prior knowledge
in the distributional assumptions with Bayesian inference. Conditioned on the
observed samples, GP learns a parametric function $g$ approximating $f$ in the
functional space where $g$ is initialized as a random function distributed
according to the predictive posterior distribution. However, GP tends to fail
to face the big data or large dimensionality (Csató and Opper, 2002; Snelson
and Ghahramani, 2006) due to the high intensive computation depending on the
choice of kernels. Another prototype of the model is proposed by (Garnelo et
al., 2018b), named as Neural Processes. NP can be trained in an end-to-end
style which combines neural networks with learning the approximating functions
in a similar way as GP. The difference lies in that NP learns parameterized
functions in a supervised way via gradient descent optimization which computes
more efficiently by learning the uncertainties directly from the data. The
regression on a toy 1D domain is discussed in (Bachman et al., 2018) with a
similar setting but optimizing an approximation to the entropy of the modeled
function. However, the aforementioned models are unconscious about the
relationship among data samples. Without learning the relation in the data
structure, models tend to embed similar samples at different positions into
the same embedding. However, learning is still done in a GP framework by
maximizing the marginal likelihood. The linear attention layers limit the
expressiveness of the representation with the overlook in the inherited
relationship in the set of data samples.

Causal Inference. Learning the distribution conditioned on a randomly sampled
subset, the NP-related task can also be regarded as few-shot learning. Most
techniques cannot rapidly generalize from a few amount of examples relying on
learning from large-scale data. However, the cost of computation will become a
side-effect. Meta-learning with neural networks is one example of this
approach. Given input-output pairs drawn from a new function at test time, one
can reason about this function by looking at the predictive distribution
conditioning on these input-output pairs. There exist substantial works in
few-shot classification (Kipf and Welling, 2016; Vinyals et al., 2016; Snell
et al., 2017; Santoro et al., 2016) dealing with locating relevant observed
image/prototype given a query image. (Edwards and Storkey, 2016; Hewitt et
al., 2018) have a similar permutation invariant encoder that predicts
summaries of a data set using local latent features on top of global features.
(Sung et al., 2018) learns to explore the relationship between samples a deep
distance metric to compare a small number of images. (Garcia and Bruna, 2017)
studies the problem of few-shot learning with the prism of inference on a
partially observed graphical model. Many approaches either assume full
knowledge of the intervention, make strong assumptions about the model class,
or have scalability limitations.

Deep Variable Models. To some extent, Neural Processes tackle the problem from
a representation learning (Bengio et al., 2013) perspective. It is ubiquitous
that a stack of feature‐extracting layers compose the model, with the first
layer processing raw inputs and the subsequent layers gradually learning more
complex representation given a set of input/output pairs. Unlike convolutional
neural network (CNN) (LeCun et al., 1989; Krizhevsky et al., 2012)and
recurrent neural network (RNN) (Hochreiter and Schmidhuber, 1997; Elman, 1990)
exploiting simple redundancies and invariances in grid‐structured data, GNN
models are more data‐efficient by directly encoding the structural inductive
biases present in data. Existing GNN models belong to a family of graph
architectures that adopt different aggregation schemes for learning the
representation of the nodes. Current models focus on learning representations
that capture local network structure around a given node. Without relying on
the node feature information, above models will always embed nodes at
symmetric positions into same embedding vectors, which means that such nodes
are indistinguishable from the GNN’s point of view. Returning to our
motivation for implanting position embedding in NPs, a clear parallel exists
between position embedding learning and self-attention. Both of them targeting
at digging into the relationship among data samples for generating valuable
representations. Current models focus too much on learning node embeddings
which tend to become assimilated. So as to relieve this kind of issue, (Kipf
and Welling, 2016; Hamilton et al., 2017) assign an unique node identifier
with pre-trained transductive node features for training. However, such models
are limited which cannot be generalized to unseen targets where the ordering
or structure is unbeknownst. In contrast, PGNN (You et al., 2019) propose to
add up the positional information to the node embeddings in the graph. Another
option to assimilate the positional information is adopting a graph kernel
that relies on the node positional information. Graph kernels implicitly or
explicitly map graphs to a Hilbert space. Weisfeiler-Lehman and Subgraph
kernels have been incorporated into deep graph kernels (Yanardag and
Vishwanathan, 2015) to capture structural properties of neighborhoods.
(Kashima et al., 2003; Gärtner et al., 2003) also propose to exploit graph
kernels based on random walks, which count the number of walks two graphs have
in common.

##  3\. Omni-Relational Network

Aiming at measuring the underlying distribution, variational models are put
forward to reconstruct the data space conditionally. However, without
exploiting the relation among data points, it is intractable to learn reliable
and generalized representations invariant to the given context points. To
address the above issue, we propose Omni-Relational Network (OR-Net) to model
the intrinsic data relationship. First, our network is built upon variational
auto-encoders to estimate the joint distribution over the latent and observed
variables. For better learning the data relativity, we not only build
connections inner the context observed points, but also cross the given
context and unseen targets. With the rich features and relativity learned, we
further add constraint in the learning objectives with information bottleneck
to guarantee the compactness of representations.

Figure 2.  The network structure of Neural Processes is built on the encoder-
decoder structure, where exist a deterministic path and latent path in the
encoder for modeling the latent variables. ${\bm{z}}$ is re-parameterized from
$s_{c}$.  Figure 3.  Illustration of the message passing in OR-Net. Given a
point set, each sample is a node connected to other nodes within its
neighborhood. Then, the embedding of the node feature together with the
learned geometric embedding are messages to be attentively aggregated into the
next layer. The right part shows the details about the geometric attentive
message aggregation and updating.

###  3.1. Preliminaries

Stochastic processes aim to empirically fit the data distribution over the
observed inputs to generate the values for unobserved samples. Given the
context samples $\\{{\bm{x}}_{j},{\bm{y}}_{j}\\}_{j\in C}$, the goal is to
learn a function to map an input ${\bm{x}}_{i}\in\mathbb{R}^{d}$ to an output
${\bm{y}}_{i}\in\mathbb{R}^{d}$ in the target space, $i\in T$. The data
distribution can be modeled with parameterized functions learned on a set of
observed samples which is invariant to the permutations of the elements. Here,
we denote $\tau$ as a permutation for a set of $n$ samples, then:

(1) $\displaystyle p({\bm{y}}_{1:n}|{\bm{x}}_{1:n})$ $\displaystyle\coloneqq
p({\bm{y}}_{1},...,{\bm{y}}_{n}|{\bm{x}}_{1},...,{\bm{x}}_{n})$
$\displaystyle=p({\bm{y}}_{\tau(1)},...,{\bm{y}}_{\tau(n)}|{\bm{x}}_{\tau(1)},...,{\bm{x}}_{\tau(n)})$
$\displaystyle\coloneqq p(\tau({\bm{y}}_{1:n})|\tau({\bm{x}}_{1:n})).$

Additionally, conditioned on the context set $C$, representations can be
learned with a finite dimensionality which should also be consistent across
context sets with different realizations of the data generation. Modeled as
regression functions, the aim is to estimate and optimise on a set of target
samples $\\{{\bm{x}}_{i},{\bm{y}}_{i}\\}_{i\in T}$:

(2)
$p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{x}}_{C},{\bm{y}}_{C})=p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{r}}_{C}),$

where ${\bm{r}}_{C}$ is the latent representations that encode information
from input pairs as prior knowledge.
$p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{r}}_{C})$ is a likelihood that can be
modelled by a Gaussian distribution factorised on target pairs
$({\bm{x}}_{i},{\bm{y}}_{i})_{i\in T}$ with the same distribution learning on
${\bm{x}}_{i}$ and ${\bm{r}}_{C}$. Beside the above deterministic path,
another latent path with an encoder that extracts a latent representation
${\bm{z}}$ from a learned distribution ${\bm{s}}_{C}\coloneqq
s(x_{j},y_{j})_{j\in C}$ to measure the uncertainty of output
${\bm{y}}_{i},i\in T$. Then, we have

(3) $p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{x}}_{C},{\bm{y}}_{C})\coloneqq\int
p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{r}}_{C},{\bm{z}})q({\bm{z}}|{\bm{s}}_{C})d{\bm{z}},$

where ${\bm{z}}$ represents different realizations of the stochastic process.
The parameters in the deterministic path and latent path are trained on
optimizing the following variational lower bound, which is also called
evidence lower bound (ELBO):

(4) $\displaystyle\mathcal{L}_{D}$
$\displaystyle({\bm{x}}_{T},{\bm{x}}_{C},{\bm{y}}_{C})=\log
p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{x}}_{C},{\bm{y}}_{C})\geq$
$\displaystyle\mathbb{E}_{q}[\log
p({\bm{y}}_{i}|{\bm{x}}_{i},{\bm{r}}_{C},{\bm{z}})]-D_{\mathrm{KL}}(q({\bm{z}}|{\bm{s}}_{T})\|q({\bm{z}}|{\bm{s}}_{C})).$

$D_{KL}$ measures the distance between the distribution learned on the target
set and the context set. Practically, $C$ is a subset of $T$ to maintain the
consistency of the reconstruction. The desired model is required to meet the
following principles: (i) Flexibility: define a rich family of distributions,
where an arbitrary number of targets can be predicted conditioning on an
arbitrary number of context samples. (ii) Scalability: computation cost scales
linearly for points sampled during training and inference. (iii) Consistency:
the predictions of the targets are invariant in the order of context samples.
However, it comes out that the existing methods are trapped with the problem
that taken randomly sampled context points as individual elements, the
dependency between the samples is disassembled and intractable. Simply relying
on the previous linear mapping mechanism is still far from fitting the target
data distribution.

###  3.2. Variational Auto-encoder

The primary purpose is to learn generative function
$p_{\theta}({\bm{x}},{\bm{z}})=p_{\theta}({\bm{x}}|{\bm{z}})p_{\theta}({\bm{z}})$
and an inference function $q_{\phi}({\bm{z}}|{\bm{x}})$ that approximates its
posterior $p_{\theta}({\bm{z}}|{\bm{x}})$. The problem is that there is
generally no guarantees about what these learned distributions actually are.
We target at learning models satisfying the following implication which holds
for all $({\bm{x}},{\bm{z}})$:

(5)
$\displaystyle\forall(\theta,\theta^{\prime}):\;\;p_{\theta}({\bm{x}})=p_{\theta^{\prime}}({\bm{x}})\;\;\implies\;\;\theta=\theta^{\prime}.$

In other words, if any two different groups of parameter $\theta$ and
$\theta^{\prime}$ lead to the equal marginal density $p_{\theta}({\bm{x}})$,
this could imply that they have matching joint distributions
$p_{\theta}({\bm{x}},{\bm{z}})$. This means that if we learn a model with
parameter $\theta$ that fits the data perfectly:
$p_{\theta}({\bm{x}})=p_{\theta^{*}}({\bm{x}})$ , then its joint density also
matches properly:
$p_{\theta}({\bm{x}},{\bm{z}})=p_{\theta^{*}}({\bm{x}},{\bm{z}})$, which also
means that we found the prior $p({\bm{z}})=p_{\theta^{*}}({\bm{z}})$ and
posteriors $p({\bm{z}}|{\bm{x}})=p_{\theta^{*}}({\bm{z}}|{\bm{x}})$ correctly.
On the other side, the inference model $q({\bm{z}}|{\bm{x}})$ can also be used
to efficiently perform inference over the sources from which the data
originates.

However, a general problem for current methods is that they only consider
mapping the global distribution while neglecting the importance of data
relativity which ensures the model resistant to noise factors and observation
bias. In this case, we can always find transformations of ${\bm{z}}$ that
change the values without interfering with the distribution. Take a spherical
Gaussian distribution $p({\bm{z}})$ for example, applying rotation keeps its
distribution the same. We can then incorporate this transformation in
$p({\bm{x}}|{\bm{z}})$. This will not change $p({\bm{x}})$, but it will affect
$p({\bm{z}}|{\bm{x}})$, since now the values of ${\bm{x}}$ come from different
values of ${\bm{z}}$. The backbone structure of the proposed method is
illustrated in Figure 2, which simultaneously learns a deep latent generative
model and a variational approximation $q({\bm{z}}|{\bm{x}},{\bm{u}})$ of its
posterior $p_{\theta}({\bm{z}}|{\bm{x}},{\bm{u}})$, the latter being often
intractable.

The conditional marginal distribution of the observations is denoted by
$p({\bm{x}}|{\bm{u}})=\int p({\bm{x}},{\bm{z}},|{\bm{u}})d{\bm{z}}$, and with
$q_{\mathcal{D}}({\bm{x}},{\bm{u}})$ we denote the empirical data distribution
given by dataset $\mathcal{D}$. The goal of VAEs is learn the vector of
parameters $(\theta,\phi)$ by maximizing $\mathcal{L}(\theta,\phi)$, a lower
bound on the data log-likelihood defined by:

(6) $\displaystyle\mathbb{E}_{q_{\mathcal{D}}}$ $\displaystyle\left[\log
p({\bm{x}}|{\bm{u}})\right]\geq\mathcal{L}(\theta,\phi):=$
$\displaystyle\mathbb{E}_{q_{\mathcal{D}}}\left[\mathbb{E}_{q({\bm{z}}|{\bm{x}},{\bm{u}})}\left[\log
p({\bm{x}},{\bm{z}}|{\bm{u}})-\log
q({\bm{z}}|{\bm{x}},{\bm{u}})\right]\right].$

Then we apply the reparameterization trick (Kingma et al., 2015) to sample
from $q({\bm{z}}|{\bm{x}},{\bm{u}})$. This trick provides a low-variance
stochastic estimator for gradients of the lower bound with respect to $\phi$.
Estimates of the latent variables can be obtained by sampling from the
variational posterior. The training algorithm is the same as in a regular VAE.

###  3.3. Pointwise Relational Inference

To exploit the data relativity within sample points, we propose to build
connections inner the context points as well as cross the context and target
points for relational inference. Specifically, our method targets at learning
representations that capture both the appearance features and data
relationship which relies on position-aware embedding. Instead of learning the
point-to-point mapping with linear layers, we propose to formulate the data
points into a graph structure to better learn the relativity among the data
points. The pipeline and the details of OR-Net are illustrated in Figure 3.
OR-Net is elaborated to learn the representation invariant to the order of
samples while maintains the structural features which conform with the
principles of exchangeability and consistency.

Omni-Relational Layer. The Omni-Relational Layer is designed to learn the data
relativity among the nodes in the input graph, and can be transformed into
Inner-Relational Layer and Cross-Relational Layer accordingly. The embedding
of the relative geometric information enables each node to better capture its
relationship to its neighbors. A node embedding is position-aware if it fits
in a function conditioned on its n-hop neighborhood. At first, given a point
set $S$, each sample $({\bm{x}}_{i},{\bm{y}}_{i})_{i\in S}$ represents one
node in the graph. $S$ is a subset of the context/target set. To embed the
location information, we set a fixed value of radius as a distance threshold
$\gamma$ to define the neighborhood for each node. The relative position-aware
embedding is learned with MLP layers. The representation of each node is a
combination of the node embedding with the geometric embedding.

The key insight of OR-Net is that the node location can be captured by
quantifying the relative distances between a point and a set of context
points. To compute the geometric embedding, we first calculate the relative
location between each sample with its neighbors, then learn from the distances
$\\{d^{m}\\}_{m=0}^{M}$ along the $M$ dimensions of the position between node
$v_{i}$ and $v_{j}$. $\bar{d}$ denotes the distance between two points. $M=2$
when points are sampled from a 2D coordinate. Note that the values of location
are normalized before calculation. The relative geometric embedding
$\mathbf{r}_{ij}$ can be learned for each neighbor $j$ of node $i$ as
following,

(7)
$\mathbf{r}_{ij}=W_{d}\textsc{concat}[\\{d^{m}_{ij}\\}_{m=0}^{M},\bar{d}],$

where $W_{d}$ is a learnable weight matrix that transforms the distances into
a vector. Combining node features and geometric embedding can then be
concatenated. Geometric information is critical to reveal the relationship
between each node with its neighbors, while the node features provide side
information that is useful for the prediction task.

Inner-Relational Learning in the Context. Given the context points from the
partial observation, we first build a context graph $G_{c}$ to learn the inner
relationship. We assume the message computation function
$\mathcal{F}(i,j,\mathbf{h}_{j},\mathbf{r}_{ij})$ accounts for both the node
features as well as geometric embeddings. The following function $\mathcal{F}$
performs well for learning the node embedding parameterized with $W_{h}$:

(8)
$\mathcal{F}(i,j,\mathbf{h}_{j},\mathbf{r}_{ij})=W_{h}\textsc{Concat}(\mathbf{h}_{j},\mathbf{r}_{ij}).$

The node feature is learned by concatenating the features of its neighbors,
similar to (Hamilton et al., 2017), or by passing in the information from the
neighboring nodes. The former can be realized by learning attention computed
by a learnable linear transformation. Inspired by (Veličković et al., 2017),
we use the attention mechanism to retrieve useful messages from the neighbors
for each node. As an initial step, a shared linear transformation is applied
to every node. Then, self-attention is performed on the nodes, a shared
attentional mechanism computes attention coefficients $a_{ij}$ which indicates
the importance of the embedding of node $j$ to node $i$. In its most general
formulation, the model allows every node to attend to every other node,
dropping all structural information.

In this way, we inject the graph structure into the mechanism by calculating
attention as $a_{ij}$ for nodes $j\in\mathcal{N}_{i}$, where $\mathcal{N}_{i}$
is the neighborhood of node $i$. The use of dot-product attention allows the
query values to be computed with two matrix multiplications and a softmax,
allowing for use of highly optimised matrix multiplication code. Thereafter,
the normalized attention coefficients are used to compute a linear combination
of the features corresponding to them, to serve as the final output features
for every node:

(9) $\hat{h}_{i}=\sum_{j\in\mathcal{N}_{i}}a_{ij}W_{a}h_{j},$

where $h_{j}$ is learned embedding from the message computation function
$\mathcal{F}$ in Eq. 8. Once the attention weights are learned, we normalize
them across all neighbor nodes using the softmax function to make coefficients
easily comparable across different nodes.

Cross-Relational Inference for the Target. We further investigate building the
relationship cross the context points and the target data points. It is noted
that we include the context points into the target points in this step. So, a
full graph $G_{t}$ is built upon both the context points and target points.
Similarly, we first embed both the node features and geometric features with
$\mathcal{F}$ to compute the messages. Then, the attention coefficients
$a_{ij}$ are calculated and normalized accordingly. Each node embedding is
updated by aggregating the messages from its neighbors. By passing messages
across Omni-Relational Layers, each node retrieves messages from its multi-hop
neighbors in a large receptive field. After the message aggregation, the
updated node features are adopted to learn the deterministic feature
${\bm{r}}$ and the latent feature ${\bm{z}}$.

###  3.4. Objectives

The previous methods only focus on the maximization of the mutual information
$I$ between the context points with the data distribution. However, the
learned distribution is variant to the bias brought by the limited observation
and thus causes unstable and unreliable estimation. It is widely adopted as a
critical principle for representation learning that an optimal representation
should be composed of minimal sufficient information. It encourages the
representation to be informative and compact about the target to make the
prediction satisfied. On the other hand, the information bottleneck (IB) also
discourages the representation from acquiring additional information from the
data that is irrelevant for predicting the target (minimal). Motivated by this
paradigm, the learned model tends to avoid over-fitting and becomes more
robust and reliable to the uncertainty of estimation by minimizing the
function:

(10)
$\mathcal{L}_{\mathrm{IB}}({\bm{x}},{\bm{z}};\beta)=-I({\bm{z}};{\bm{y}})+\beta
I({\bm{x}};{\bm{z}}),$

where $\beta$ is a balancing parameter for the loss terms. Inherited from the
IB principle, each node embedding is optimized by minimizing the information
from the conditional context and maximizing the information from its original
distribution. In order to better incorporate IB with the graph-based
structure, here we follow the local dependence assumption to constrain the
optimizing space: Messages aggregated on each node from its neighbors within
n-hops. Hence, the objective of OR-Net could be reformulated as:

(11)
$\mathcal{L}=\mathcal{L}_{D}({\bm{x}}_{T},{\bm{x}}_{C},{\bm{y}}_{C})+\mathcal{L}_{IB}({\bm{x}}_{T},{\bm{z}};\beta).$

The proposed OR-Net transforms the randomly sampled point set into structural
data with the constrain of the neighborhood of each point. Beyond that, the
embedded geometric information assists in building the relationship between
each target point with the context point set through the neighbors. With the
geometric attentive message passing and aggregation, both of the appearance
embedding and geometric embedding are passed into the next layer. It is
further proved empirically that representation learned by OR-Net is more
expressive and reliable.

##  4\. Experiment

###  4.1. Training Details

Network structure. OR-Net is built upon encoder-decoder structure.
Specifically, the encoder is composed of a deterministic path to learn the
representation and a latent path to estimate the data distribution. It is
worth noted that the same weights are shared in the latent path for learning
both the prior distribution on the observed data points and the posterior
distribution. In the Omni-Relational Layer, attentive pooling functions as the
message passing function. Then, a three-layer MLP functions as the decoder to
predict the concrete values for the target points. Optimization is performed
with Adam (Kingma and Ba, 2014). We set batch size for training to be 32 for
function regression and image completion, 128 for motion generation. The
learning rate is initialized to be 0.001. The balancing parameter $\beta$ is
set to 0.05 empirically in our experiment.

Comparison methods. For a comprehensive comparison, we conduct experiments for
conditional generation of different modalities against Gaussian Processes
(Williams and Rasmussen, 1996), Conditional NP (Garnelo et al., 2018a),
Attentive NP (Kim et al., 2019). Additionally, we also implement a graph-based
method PGNN (You et al., 2019). During training, our model is built upon
graphs with a fixed node ordering and re-trained invariant to the ordering of
the nodes. We report the test set performance when the best performance on the
validation set is achieved, and results are reported over 5 runs with random
seeds.

###  4.2. 1D Function Regression

We start with conducting experiments for 1D function regression task. As a
stochastic process, OR-Net is trained by sampling a batch of realizations from
the data generating where random points are selected to be the targets and a
subset to be the context set to approach the target points. We first explore
data generated from a Gaussian Process with a squared-exponential kernel and
small likelihood noise. The number of contexts (n) and number of targets (m)
are chosen randomly at each iteration, (n $\sim$ U[3, 20], m $\sim$ n + U[0,
20 $-$ n]). Each data point is drawn uniformly with random choice for
${\bm{x}}$ in (-2, 2). Each sequence contains 100 points.

We show the performance for function regression in Figure 4. The estimated
regression results are shown in blue lines conditioned on the context points
drawn with dots. It is observed on the right that OR-Net performs better
compared to Neural Processes. Furthermore, with the learned data structure,
OR-Net can better adapt to the observed data points with higher accuracy. When
supplied with less context points, NPs have incorrectly collapsed the
distribution over functions to a set of almost horizontal lines. OR-Net, on
the other hand, is able to produce a wide range of possible trajectories. Even
when a large number of points have been supplied, the NP posterior does not
converge on a good fit, while OR-Net correctly captures the latent variables.

Table 1. We evaluate methods on MNIST with context points number in (50, 100, 200, 400). OR-Net$\dagger$ indicates OR-Net is trained with IB loss. | 50 | 100 | 200 | 400
---|---|---|---|---
CNP (Garnelo et al., 2018a) | $0.073\small{\pm.013}$ | $0.036\small{\pm.006}$ | $0.022\small{\pm.005}$ | $0.024\small{\pm.006}$
PGNN (You et al., 2019) | $0.060\small{\pm.010}$ | $0.032\small{\pm.006}$ | $0.025\small{\pm.007}$ | $0.022\small{\pm.003}$
ANP (Kim et al., 2019) | $0.056\small{\pm.010}$ | $0.029\small{\pm.004}$ | $0.015\small{\pm.003}$ | 0.009$\small{\pm.003}$
OR-Net | $0.049\small{\pm.005}$ | $0.025\small{\pm.004}$ | $0.012\small{\pm.004}$ | $0.010\small{\pm.003}$
OR-Net$\dagger$ | 0.045$\small{\pm.006}$ | 0.022$\small{\pm.003}$ | 0.010$\small{\pm.004}$ | 0.009$\small{\pm.002}$
Figure 4.  We present the training and inference results for 1D function
regression. Specifically, we show the predicted results with blue lines in (a)
for function regression where the context points are drawn with dots and the
ground truth are grey dotted lines, and also the evaluated accuracy with
varying context numbers in (b).

###  4.3. Image Completion

Images are composed of arranged points that can be interpreted as being
generated from a stochastic process. The performance of the proposed OR-Net is
compared quantitatively with the former approaches, kNN, GP (Williams and
Rasmussen, 1996), NP (Garnelo et al., 2018a), ANP (Kim et al., 2019) and also
our realization of PGNN (You et al., 2019). Each image is considered as one
realization sampled on a fixed 2-dimensional grid. All experiments are trained
up to 200 context/target points at training with a random number at each
iteration. Neural processes aim to predict the value of target points which
can be dealt with as a regression problem. Given the context points as pairs
of $\\{{\bm{x}}_{j},{\bm{y}}_{j}\\}_{j\in C}$, the objective is to map the
location of a target point ${\bm{x}}_{i}$ to its pixel value
${\bm{y}}_{i}\in\mathbb{R}^{1}$ as greyscale, ${\bm{y}}_{i}\in\mathbb{R}^{3}$
as RGB.

We conduct experiments on MNIST (LeCun et al., 1998) and CelebA (Liu et al.,
2015) with the default train/test splits. We resize the MNIST images as
$28\times 28$. For CelebA, we crop the images as $128\times 128$ and then
resize images as $32\times 32$. Empirically, the number of graph layers is set
as 2 in our experiments. We first compare models during inference with
different numbers of context points on MNIST dataset. Models are uniformly
trained with less than 200 context points. It is also observed that OR-Net
performs better with fewer context points during the inference with results in
Table 1. Both insights show the fact that with the geometric graph, OR-Net
takes the advantage of the position-aware representation to better fit the
data distribution. The results on the CelebA dataset are summarized in Table 2
with an increasing number of context points in (10, 100, 1000) sampled either
at random or ordered from the top-left corner.

Table 2. Mean square error measured pixel-wise for image completion task on CelebA. Performance of experiments with random and ordered (top-left) context points of (10, 100, 1000) are compared.  | Random | Ordered
---|---|---
Model | 10 | 100 | 1000 | 10 | 100 | 1000
kNN | $0.215$ | $0.052$ | $0.007$ | $0.370$ | $0.273$ | $0.007$
GP (Williams and Rasmussen, 1996) | $0.247$ | $0.137$ | 0.001 | $0.257$ | $0.220$ | 0.002
PGNN (You et al., 2019) | $0.057$ | $0.026$ | $0.015$ | $0.122$ | $0.049$ | $0.017$
CNP (Garnelo et al., 2018a) | $0.039$ | $0.016$ | $0.009$ | $0.057$ | $0.047$ | $0.021$
ANP (Kim et al., 2019) | $0.032$ | $0.013$ | $0.006$ | $0.044$ | $0.031$ | $0.012$
OR-Net | $0.024$ | $0.011$ | $0.008$ | $0.039$ | $0.027$ | $0.014$
OR-Net$\dagger$ | 0.021 | 0.008 | 0.006 | 0.036 | 0.024 | 0.011
Figure 5.  We report our results with different numbers of Omni-Relational
Layer in OR-Net and the hyper-parameter neighborhood radius $\gamma$ in the
graph structure in the left. The right part compares the performance of OR-Net
with and without IB loss.

We further executed grid search with different values of neighborhood radius
$\gamma$ in range of [1, 2, 3, 5, 8, 12, 16, 32] and graph layer number in [1,
2, 3] to further explore the position embedding with results shown in Figure
5. It is discovered that the performance reaches the best when $\gamma$ is set
to 5 with 2 geometric graph layers. The performance drops when the value of
$\gamma$ is too small or too large. We also applied different values of
neighborhood radius as the percentage against the image size to further
explore the position embedding. The performance reaches the best at 0.5. Thus,
with a modest receptive field, OR-Net can better capture the data structure in
a hierarchical way. It is also shown that OR-Net performs consistently better
with the constrain of information bottleneck.

In Figure 6, we show predictions of the full image pixels as target points at
different training stages chronologically for randomly selected images. 100
context points are sampled for the inference. It is observed that OR-Net gives
reasonable predictions with better reconstructions in the fine-grained
details, e.g., the edge of face, the shape of nose and eyes. The embedding of
position information also helps achieve more smooth and natural image
completion results compared to the other methods. It is also proved that OR-
Net can better model the global structure of the image, with one sample
corresponding to one realization of the data generating stochastic process. We
also visualized the reconstruction of digits with increasing context number in
the left of Figure 7 and compared the face reconstruction for images in CelebA
against other methods. When sampling from this model with a small number of
observed pixels, we get coherent samples and we see that the variability of
the datasets is captured. As the model is conditioned on more and more
observations, the variability of the samples drops and they eventually
converge to a single possibility. It is demonstrated that by exploiting the
potential pointwise relationship in modalities without observed structure, OR-
Net can better model the latent variables and underlying distribution.

Figure 6.  Reconstruction results by OR-Net on the test set of CelebA with
context points with different percentages.  Figure 7.  Reconstruction results
on digit dataset: MNIST and SVHN with increasing context points on the left,
and reconstruction on CelebA by Conditional NP, Attentive NP and OR-Net (with
only 10% points are sampled as context).  Figure 8.  Reconstruction sequences
for action “walking” and “sitting” in HumanAct12 without observed poses on the
left. On the right, we show the reconstructed results given 3 poses randomly
sampled as the context set.

###  4.4. Sequential Motion Generation

Motion capture equipment is widely adopted to generate realistic and smooth
motion. However, the motion capture process is expensive and time-consuming.
These drawbacks require the original data to be well labeled and further
processed. To address these issues, we extend conditional generation into the
motion completion based on HumanAct12 (Guo et al., 2020), an in-house dataset
which is adopted from an existing dataset PHSPD (Zou et al., 2020), consisting
of 1,191 motion clips and 90,099 frames in total, with hierarchical action
type annotations. HumanAct12 has more organized action annotation, with a
balanced number of motions per action compared to other datasets. A body pose
consists of 24 joints (23 bones).

We show in Figure 8 that given 3 randomly sampled poses from a sequence, our
method is able to generate natural and meaningful motion sequences. We use
conditional GRU as our deterministic baseline. Vanilla RNN model takes the
condition vector and pose vector together as input at current step and
predicts the pose vector for the next step. For conditional GRU, generated
poses all collapse to a set of spatial points near the root joint, which shows
the inefficacy of simple RNN models toward a non-deterministic generative
task. It is worth noting that motion generation should not be a one-to-one
mapping process. Instead, the generated motions are expected to be close to
the real motions in terms of their respective distributions.

The most notable difference from the aforementioned tasks is that human motion
is generated upon visible skeleton structure. Additionally, we use LSTM in
preserving the motion variance in the time dimension. We investigate how well
OR-Net can capture the latent patterns in different motions, using several
types of motions with different properties. We first train OR-Net on them
separately then jointly. We observe that OR-Net can learn the transition
stochasticity well when trained on a single type of motions. The diversity can
be found in short-term and long-term transitions, which are two levels of
multi-modality captured well by OR-Net. The action-level transition has also
been captured and generated by conditioning on the given action label. The
reconstruction sequences for action “walking” and “sitting” are presented in
Figure 8. We can observe that with the given context, the generated motion
sequences can be more diverse with the required poses. Similar observations
are also found in other motions.

Table 3. The ablation of different modules in Omni-Relational Network reported with MSE on MNIST given 10$\%$ context points by ablating (1) graph structure (2) attentive pooling (3) positional embedding (P.E.) (4) information bottleneck constrain (I.B.). Graph | Attention | P.E. | I.B. | MSE
---|---|---|---|---
|  |  |  | $0.073$
$\checkmark$ |  |  |  | $0.066$
$\checkmark$ |  |  | $\checkmark$ | $0.058$
$\checkmark$ |  | $\checkmark$ | $\checkmark$ | $0.053$
$\checkmark$ | $\checkmark$ |  | $\checkmark$ | $0.049$
$\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $0.045$
Figure 9.  The visualized trajectories for joint 18 in the left arm and joint
19 in the right arm for different action classes. It can be observed that the
correlation is high in “sit” and “walk” but not for the other two actions.

Furthermore, in table 3, performances are reported with the ablation of the
sub-modules in OR-Net: (1) graph structure (2) attentive pooling (3)
positional embedding (P.E.) (4) information bottleneck constrain (I.B.). It is
observed that with the relational inference in the graph structure, OR-Net
reaches considerable results. Additionally, the information bottleneck
constrain further boosts the performance steadily.

##  5\. Conclusion

In this paper, we explore a new scheme, Omni-Relational Network, to derive a
more robust and general representation of the data. In this regard, OR-Net
exploits the data relativity within not only the context samples but also
cross the context and the target points by fusing the positional embedding.
Simultaneously, it retains the benefits of information bottleneck to enhance
the compactness of representation. By transforming the randomly sampled data
into a position-based structure, we offer a promising direction to dig into
the potential relevance under the partial observation. Furthermore, this work
can be seen as a step towards learning high-level abstractions of the data
distribution with fewer samples or incomplete data. In future work, we will
explore how far OR-Net can help to tackle learning problems hinge on the
inherited relationship in data of different modalities, such as meta-learning
and data efficiency.

## References

  * (1)
  * Bachman et al. (2018) Philip Bachman, Riashat Islam, Alessandro Sordoni, and Zafarali Ahmed. 2018.  Vfunc: a deep generative model for functions.  _arXiv preprint arXiv:1807.04106_ (2018).
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ 35, 8 (2013), 1798–1828.
  * Csató and Opper (2002) Lehel Csató and Manfred Opper. 2002.  Sparse on-line Gaussian processes.  _Neural computation_ 14, 3 (2002), 641–668.
  * Easley and Kiefer (1988) David Easley and Nicholas M. Kiefer. 1988.  Controlling a Stochastic Process with Unknown Parameters.  _Econometrica_ 56, 5 (1988), 1045–1064.
  * Edwards and Storkey (2016) Harrison Edwards and Amos Storkey. 2016.  Towards a neural statistician.  _arXiv preprint arXiv:1606.02185_ (2016).
  * Elman (1990) Jeffrey L Elman. 1990\.  Finding structure in time.  _Cognitive science_ 14, 2 (1990), 179–211.
  * Garcia and Bruna (2017) Victor Garcia and Joan Bruna. 2017.  Few-shot learning with graph neural networks.  _arXiv preprint arXiv:1711.04043_ (2017).
  * Garnelo et al. (2018a) Marta Garnelo, Dan Rosenbaum, Chris J Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo J Rezende, and SM Eslami. 2018a.  Conditional neural processes.  _arXiv preprint arXiv:1807.01613_ (2018).
  * Garnelo et al. (2018b) Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. 2018b.  Neural processes.  _arXiv preprint arXiv:1807.01622_ (2018).
  * Gärtner et al. (2003) Thomas Gärtner, Peter Flach, and Stefan Wrobel. 2003\.  On graph kernels: Hardness results and efficient alternatives.  In _Learning theory and kernel machines_. Springer, 129–143.
  * Guo et al. (2020) Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. 2020\.  Action2motion: Conditioned generation of 3d human motions. In _Proceedings of the 28th ACM International Conference on Multimedia_. 2021–2029.
  * Hamilton et al. (2017) Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017\.  Inductive representation learning on large graphs. In _Advances in neural information processing systems_. 1024–1034.
  * Hewitt et al. (2018) Luke B Hewitt, Maxwell I Nye, Andreea Gane, Tommi Jaakkola, and Joshua B Tenenbaum. 2018.  The variational homoencoder: Learning to learn high capacity generative models from few examples.  _arXiv preprint arXiv:1807.08919_ (2018).
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9, 8 (1997), 1735–1780.
  * Insua et al. (2012) David Insua, Fabrizio Ruggeri, and Mike Wiper. 2012\.  _Bayesian analysis of stochastic process models_. Vol. 978.  John Wiley & Sons.
  * Kashima et al. (2003) Hisashi Kashima, Koji Tsuda, and Akihiro Inokuchi. 2003\.  Marginalized kernels between labeled graphs. In _Proceedings of the 20th international conference on machine learning (ICML-03)_. 321–328.
  * Kim et al. (2019) Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. 2019\.  Attentive neural processes.  _arXiv preprint arXiv:1901.05761_ (2019).
  * Kingma and Ba (2014) Diederik P. Kingma and Jimmy Ba. 2014.  Adam: A Method for Stochastic Optimization. In _Proceedings of the 3rd International Conference for Learning Representations_.
  * Kingma et al. (2015) Diederik P Kingma, Tim Salimans, and Max Welling. 2015\.  Variational dropout and the local reparameterization trick.  _arXiv preprint arXiv:1506.02557_ (2015).
  * Kipf and Welling (2016) Thomas N Kipf and Max Welling. 2016.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ (2016).
  * Krizhevsky et al. (2009) Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. 2009\.  CIFAR-10 (Canadian Institute for Advanced Research).  (2009).
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012\.  Imagenet classification with deep convolutional neural networks. In _Advances in neural information processing systems_. 1097–1105.
  * Lake et al. (2015) Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. 2015\.  Human-level concept learning through probabilistic program induction.  _Science_ 350, 6266 (2015), 1332–1338.
  * LeCun et al. (1989) Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. 1989.  Backpropagation applied to handwritten zip code recognition.  _Neural computation_ 1, 4 (1989), 541–551.
  * LeCun et al. (1998) Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. 1998\.  Gradient-based learning applied to document recognition.  _Proc. IEEE_ 86, 11 (1998), 2278–2324.
  * Lei et al. (2016) Yaguo Lei, Naipeng Li, and Jing Lin. 2016.  A new method based on stochastic process models for machine remaining useful life prediction.  _IEEE Transactions on Instrumentation and Measurement_ 65, 12 (2016), 2671–2684.
  * Lin et al. (2014) Tsung-Yi Lin, Maire Michael, Belongie Serge, Hays James, Perona Pietro, Ramanan Deva, Dollár Piotr, and Zitnick C. Lawrence. 2014.  Microsoft coco: Common objects in context. In _European conference on computer vision_.
  * Liu et al. (2015) Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015\.  Deep Learning Face Attributes in the Wild. In _Proceedings of International Conference on Computer Vision (ICCV)_.
  * Santoro et al. (2016) Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016\.  One-shot learning with memory-augmented neural networks.  _arXiv preprint arXiv:1605.06065_ (2016).
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel. 2017\.  Prototypical networks for few-shot learning. In _Advances in neural information processing systems_. 4077–4087.
  * Snelson and Ghahramani (2006) Edward Snelson and Zoubin Ghahramani. 2006.  Sparse Gaussian processes using pseudo-inputs. In _Advances in neural information processing systems_. 1257–1264.
  * Sung et al. (2018) Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018.  Learning to compare: Relation network for few-shot learning. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 1199–1208.
  * Veličković et al. (2017) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017.  Graph attention networks.  _arXiv preprint arXiv:1710.10903_ (2017).
  * Vinyals et al. (2016) Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. 2016\.  Matching networks for one shot learning. In _Advances in neural information processing systems_. 3630–3638.
  * Williams and Rasmussen (1996) Christopher KI Williams and Carl Edward Rasmussen. 1996.  Gaussian processes for regression. In _Advances in neural information processing systems_. 514–520.
  * Yanardag and Vishwanathan (2015) Pinar Yanardag and SVN Vishwanathan. 2015.  Deep graph kernels. In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 1365–1374.
  * Ye and Xie (2015) Zhi-Sheng Ye and Min Xie. 2015.  Stochastic modelling and analysis of degradation for highly reliable products.  _Applied Stochastic Models in Business and Industry_ 31, 1 (2015), 16–32.
  * You et al. (2019) Jiaxuan You, Rex Ying, and Jure Leskovec. 2019.  Position-aware graph neural networks.  _arXiv preprint arXiv:1906.04817_ (2019).
  * Zou et al. (2020) Shihao Zou, Xinxin Zuo, Yiming Qian, Sen Wang, Chi Xu, Minglun Gong, and Li Cheng. 2020.  Polarization Human Shape and Pose Dataset.  _arXiv preprint arXiv:2004.14899_ (2020).