# Human Pose Estimation from Sparse Inertial Measurements
through Recurrent Graph Convolution

Patrik Puchert
Institute of Mediainformatics, Ulm University
89081 Ulm, Germany
<EMAIL_ADDRESS>Timo Ropinski
Institute of Mediainformatics, Ulm University
89081 Ulm, Germany

###### Abstract

Conventional methods for human pose estimation either require a high degree of
instrumentation, by relying on many inertial measurement units (IMUs), or
constraint the recording space, by relying on extrinsic cameras. These
deficits are tackled through the approach of human pose estimation from sparse
IMU data. We define adjacency adaptive graph convolutional long-short term
memory networks (AAGC-LSTM), to tackle human pose estimation based on six
IMUs, while incorporating the human body graph structure directly into the
network. The AAGC-LSTM combines both spatial and temporal dependency in a
single network operation, more memory efficiently than previous approaches.
This is made possible by equipping graph convolutions with adjacency
adaptivity, which eliminates the problem of information loss in deep or
recurrent graph networks, while it also allows for learning unknown
dependencies between the human body joints. To further boost accuracy, we
propose longitudinal loss weighting to consider natural movement patterns.
With our presented approach, we are able to utilize the inherent graph nature
of the human body, and thus can outperform the state of the art (SOTA) for
human pose estimation from sparse IMU data.

##  1 Introduction

A correct estimation of human poses is important in many applications. These
range from various applications in virtual and augmented reality [21, 26] to
medical applications, such as gait analysis [29], patient monitoring [2] or
human activity recognition [3]. Unfortunately, today’s SOTA methods for human
pose estimation either only work in constrained environments, or are very
intrusive [7, 45, 47]. These constraints make them impractical for outdoor
applications, indoor scenarios spanning multiple rooms or suffering from
occlusions [8]. These deficits can be solved by measuring the human body pose
with body-worn IMUs [16]. To aid user acceptance and usability, the number of
body mounted sensors must be minimal, resulting in sparse inertial
measurements.

In this paper, we propose a novel approach for human pose estimation based on
a set of 6 IMUs. While this scenario has been tackled by others before [38,
12, 44, 43], we are the first to enhance pose estimation accuracy by
incorporating the structure of the human body through deep graph learning,
instead of predicting the pose from a flat array of input data. While using
graph structures in deep neural networks is a well studied field [40], the
usage of standard graph convolutions in recurrent architectures poses the same
over-smoothing problem as in other deep graph architectures [20]. To address
this challenge, we propose adjacency adaptive graph convolutions (AAGCs),
which we realize in a novel recurrent long-short term memory (LSTM)
architecture. To do so, we fuse the AAGC with the LSTM by using it as the
neural building block inside the LSTM. This connects the spatial and temporal
computations in AAGC-LSTM, whereby the adjacency adaptive component avoids
over-smoothing, which would result in a constant average pose. With this new
approach we do not only outperform the SOTA but also require significantly
less memory during training, as compared to other approaches combining graph
learning with recurrent cells in different fields [19].

To further improve pose estimation, we show how AAGC-LSTMs benefit from
respecting the nature of human movements as well as bilateral body symmetry,
for which we propose longitudinal loss weighting and utilize contralateral
data augmentation. The former solves the problem of underrepresented body
regions in recorded movements and a more difficult estimation of the legs
based on sparse IMU data [12]. The latter simply augments the available
training data by respecting the bilateral symmetry of the human body, by
mirroring all movements.

Thus, within this paper we propose the first graph convolution approach to
solve sparse IMU-based pose estimation, and we make the following technical
contributions in this context:

  * •

We introduce the AAGC-LSTM cell as a memory efficient recurrent graph LSTM
formulation incorporating learnable adjacency matrices, to address the over-
smoothing problem of recurrent graph convolutional networks.

  * •

We propose longitudinal loss weighting, to adapt for underrepresentation of
specific joints in the training data based on movement statistics.

  * •

Using our proposed AAGC-LSTM model with longitudinal loss weighting we
outperform the SOTA for sparse IMU based human pose estimation.

While we make these technical contributions specifically for IMU-based human
pose estimation, we would like to emphasize that they are not restricted to
this task and could likely be applied to other pose estimation scenarios
without IMUs, as well as for other body types.

##  2 Related Work

IMU-based pose estimation. Traditional motion Capture technologies constrain
the recording volume by relying on extrinsic sensors like cameras [5, 49, 17,
27, 32, 41]. This problem is lifted by completely body-worn systems. While
other systems have been proposed [13], IMU based systems have the benefit of
being light weight, small and widely available. Today, such systems are
commercially available, whereby IMU measurements from different units are
fused to reconstruct a subject’s pose [28]. While they yield good results,
their large amount of required IMUs, _e.g_., 17 IMUs in Xsens’ current MTw
Awinda setup for full pose estimation111https://www.xsens.com/products/mtw-
awinda, can be considered intrusive, require long setup times, and provoke
sensor placement errors. To tackle these shortcomings, several methods have
been proposed to reduce the amount of necessary sensors. Many such approaches
are limited in application by matching query samples to prerecorded databases
[35, 36] or training a model for each activity of interest [31]. Von Marcard
et al. instead use a generative approach to estimate 3D poses from only 6 IMUs
with their sparse inertial poser (SIP) [38]. To do so, they equip the Skinned
Multi-Person Linear (SMPL) body model [22] with synthetic IMUs, and solve for
the SMPL pose that best matches the measured IMU data. As this has to be done
at query time, it is a computaionally expensive procedure. With the deep
inertial poser (DIP) Huang et al. have enhanced the accuracy of pose
estimation based on 6 IMUs using a deep learning approach [12]. They built a
bidirectional LSTM network [30] to map the flattened input of 5 IMUs,
normalized by a 6th IMU on the pelvis, to the target pose in the SMPL body
model. In contrast to the deep learning approaches the use of shallow fully
connected networks has been proposed [39]. Yi et al. proposed an improvement
for the deep architectures by predicting intermediate representations, along
with a network branch for global position estimation [44]. With their
Transpose network they effectively apply the network of DIP on three
subsequent steps, where they first predict the position of leaf joints from
the IMU data, then the position of all joints from the former joints as well
as the IMU data, and finally the target pose in the SMPL model using the IMU
data and the position of all joints as input. Recently they further improved
the accuracy by introducing a physics based module to postprocess the model
predictions [43].

Graph learning. Graph learning has been used for human body related tasks, by
mapping a 2D pose, predicted from static RGB images, to a 3D pose [45, 50],
eliminating the need to cope with time dependencies. Several approaches have
been proposed which work with spatio-temporal data by separating the spatial
and temporal learning steps. These separations are either a consecutive
application of graph convolutions (GCN) and recurrent layers [46, 25], or of
GCNs in space and convolutions in time domain consecutively [42, 33]. Some of
these methods modulate the fixed adjacency of the underlying graph by
incorporating learnable adjacency components. The combination of spatial and
temporal dependencies has been proposed for different tasks. Bai et al. build
the adjacency matrix out of a learnable embedding of all graph nodes and use
them in gated recurrent units (GRUs) [4] for traffic forecasting. While such
an embedding has benefits for large graphs, it is not beneficial for the small
graph of our problem. Li et al. have proposed graph-based GRU cells (G-GRU)
for human motion prediction, in which they also employ an adaptive adjacency
matrix [19]. While they keep a fixed adjacency matrix, they modulate it with a
multiplicative and an additive weight matrix before using it in a GCN to
modify the cells hidden input state. While we consider it counter-intuitive to
just add a graph computation to the network while keeping the linear
computation, it also requires a significantly larger memory footprint than our
approach, as we will detail below. LSTMs differ from GRUs by having a better
deep context understanding [10], which makes them better suited to tasks such
as human pose estimation, where input sequences can consist of data recorded
with 60Hz, while the context of movements can span from only a few frames to
many seconds. Si et al. use a combination of LSTMs and GCNs with fixed
adjacency together with an attention mechanism for human activity recognition
(AGC-LSTM). While they obtain good results, their model does not scale well to
our problem with indefinitely long sequences and sparse inputs. While the
over-smoothing problem of deep GCN networks [20] has been tackled in the
spatial domain [6] by introducing initial residual connections and identity
mapping to the GCN, this is not applicable to recurrent architectures.

##  3 The Method

Figure 1: We learn towards the SMPL skeleton (_light gray_), whereby we limit
the joint connectivity (_dark gray_) to match influence area of the 5
normalized _IMUs_.

In this section, we detail the technical concepts behind our proposed graph
convolution approach. The input of our model is the data of 5 IMUs placed on
the SMPL skeletal graph, whereby their input is transformed into a body-
centric system as per Huang et al. by a 6th IMU on the pelvis [12]. We also
train towards the SMPL body model [22] as target, in order to obtain realistic
poses, and allow for comparison to the SOTA. To incorporate body topology, we
transform the IMU input to a graph structure by automatically placing the
sensors on a human skeletal graph at the corresponding nodes. For this graph
we chose the SMPL skeletal model, whereby we focus on only 15 core joints out
of the 24 joints of the model [22], i.e., without the outer extremities of
hands, feet and the root joint (see Fig. 1). We chose this focus due to the
fact, that the information sparse IMUs can give for the former is naturally
limited, while the root joint at the Pelvis is fixed by definition through the
data transformation with respect to it. The nodes containing no IMU
measurements are initialized with zeros in the input graph. Thus, we operate
on an input graph of dimensionality $N\times F_{in}$, where $N=15$ and
$F_{in}=12$ are the input features given by the elements of a $3x3$ rotation
matrix and a three dimensional acceleration vector. The training target is the
same graph with the SMPL pose parameters $\theta$ on the corresponding nodes.

Adjacency adaptive graph convolution. The core of our model is the combination
of LSTM cells and GCNs in a bidirectional recurrent layer. As the inclusion of
standard GCNs in recurrent applications suffers from the over-smoothing
problems as described by Li et al. [20], we define the adjacency adaptive
graph convolution (AAGC). The definition of AAGC follows the notation of the
commonly used approximation of the graph convolution as proposed by Kipf and
Welling [15], with the propagation rule:

$\mathbf{Z}=\tilde{\mathbf{A}}\mathbf{X}\mathbf{W}+\mathbf{b},$ (1)

where $\mathbf{X}$ is the input and $\mathbf{W}$ and $\mathbf{b}$ are the
trainable weights and biases. In standard GCNs
$\tilde{\mathbf{A}}=\mathbf{D}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I_{N}})\mathbf{D}^{\frac{1}{2}}$
is the constant symmetric normalization of the adjacency matrix $\mathbf{A}$
with added self-connections $\mathbf{I_{N}}$ using the diagonal node degree
matrix $\mathbf{D}$ of $\mathbf{A}$. To now employ adjacency adaptivity, we
make $\tilde{\mathbf{A}}$ a learnable matrix, initialized by the normalized
complemented distance on the graph:

$\tilde{\mathbf{A}}_{ij}^{init}=1-\frac{d(n_{i},n_{j})}{\sum_{j}d(n_{i},n_{j})},$
(2)

where $d(n_{i},n_{j})$ is the euclidean distance between node $i$ and node $j$
on the graph. In addition to the necessity of adjacency adaptivity in our
application we assume another benefit of the learnable adjacency, which is the
hardly factorizable dependency of all joints to each other. The problem in
factorization of all joint dependencies comes from the fact, that biological
joints are not only actuators driven by many muscles, but even in a free state
underlie many factors of dampening [24, 18]. Thus the free joint, _i.e_. one
not actively driven by muscular movement, is affected by all other joints both
through the connection along a line of damped oscillators, as well as by
inertia, as the skeleton is neither completely stiff. While it is true that
all effects will always also effect the neighbouring joints, it can be better
for the model to learn dependencies to the other joints as well, as this will
give the context to small changes in acceleration or rotation. Using this
definition of the AAGC operation inside LSTM cells (AAGC-LSTM) thus lifts the
over-smoothing problem. The definition of the AAGC-LSTM cell however follows
on a coarse level the definition of the conventional LSTM cell [11, 9],
whereby in the AAGC-LSTM cell we simply replace every learnable with AAGCs:

$\displaystyle\mathbf{X}_{i}$
$\displaystyle=\sigma(\tilde{\mathbf{A}}_{i}\mathbf{X}\mathbf{W}_{i}+\mathbf{b}_{i})$
(3) $\displaystyle\mathbf{X}_{f}$
$\displaystyle=\sigma(\tilde{\mathbf{A}}_{f}\mathbf{X}\mathbf{W}_{f}+\mathbf{b}_{f})$
$\displaystyle\mathbf{X}_{c}$
$\displaystyle=tanh(\tilde{\mathbf{A}}_{c}\mathbf{X}\mathbf{W}_{c}+\mathbf{b}_{c})$
$\displaystyle\mathbf{X}_{o}$
$\displaystyle=\sigma(\tilde{\mathbf{A}}_{o}\mathbf{X}\mathbf{W}_{o}+\mathbf{b}_{o}),$

where $\mathbf{X}$ is the concatenation of the current input and the last
hidden state $\mathbf{H}_{t-1}$, to both of which dropout is applied, and the
index $t$ denotes the timestep. $\mathbf{\tilde{A}}$, $\mathbf{W}$ and
$\mathbf{b}$ are the adjacency and weight matrices and biases of the AAGC, and
$\sigma$ is the sigmoid activation function. The gates $X_{\\{i,f,c,o\\}}$ are
then processed with the common LSTM scheme [11]:

$\displaystyle\mathbf{C}_{t}$
$\displaystyle=\mathbf{C}_{t-1}\odot\mathbf{X}_{i}+\mathbf{X}_{f}\odot\mathbf{X}_{c}$
(4) $\displaystyle\mathbf{H}_{t}$
$\displaystyle=\sigma_{out}(\mathbf{C}_{t})\odot\mathbf{X}_{o}$
$\displaystyle\mathbf{O}_{t}$ $\displaystyle=\sigma_{out}(\mathbf{H}_{t}),$

where $\mathbf{O}$ is the cells output and $\mathbf{C}$ is the cells carry
respectively. $\sigma_{out}$ is the output activation function, which is
realized as $tanh$ function for the hidden AAGC-LSTM layer. Thus, we are able
to combine the spatial and temporal learning part in a single recurrent cell,
and are able to learn all dependencies of a current state in a single
computation. This is desirable as the spatial context needs to be accessible
for every joint to define the pose. Furthermore, since the single pose is only
a discrete timestep in a continuous movement, the temporal dependency must be
accessible to consider coherence and consider the pose in the context of an
animation. As such, the pose of a raised arm could inherit from a movement of
a raising arm or of relaxing an arm from a raised pose, in which the sensor
rotation values will be identical while the temporal information puts the
sparse acceleration data in a context better suited for robust estimation.

Number of Parameters Compared. The formulation of AAGCs has a great benefit
compared to the similar graph-based gated recurrent unit (G-GRU) formulation
proposed by Li et al. [19]. Applied to the data matrices of size $N\cdot
F_{i}$, where $N$ is the number of nodes in the graph and $F_{i}$ the layers
input feature size, we have $(F_{i}+F_{h})\cdot F_{o}$ parameters in each of
the four linear operations of the standard LSTM cell, with the layers hidden
state and output feature size $F_{h}$ and $F_{o}$. By replacing the linear
operations with our AAGCs, we increase the number of parameters to
$N^{2}+(F_{i}+F_{h})\cdot F_{o}$, thus increasing the parameter count of one
cell by $4\cdot N^{2}$. The approach of G-GRU applied to an LSTM cell would
instead keep the linear operations and add one adjacency adaptive graph
convolution on the hidden state, resulting in a total increase of
$N^{2}+F_{h}\cdot F_{o}$. We can simplify this to $N^{2}+F_{h}^{2}$ by knowing
$F_{i}=F_{o}=F_{h}$ for the LSTM layers in our network. With our hidden
feature size of $512$ we result in a total difference in the number of
parameters $\mathcal{N}$ per recurrent layer of
$\mathcal{N}_{\textit{G-GRU}}-\mathcal{N}_{\textit{AAGC-LSTM}}=261,469$.
Applied to our complete network setup, this difference results in $9,460,047$
parameters for the G-GRUs and $8,412,123$ parameters for the AAGC-LSTMs, a
reduction by 11%.

Model Architecture and Training Objective. We define the general layout of our
model in Fig. 2. The graph data is first fed to an AAGC layer, then to two
bidirectional AAGC-LSTM layers and finally to another AAGC layer.

Figure 2:  Layout of our proposed AAGC-LSTM network. The adjacency adaptive
graph convolutions form the input and output layer and are used as learnable
operation inside the recurrent AAGC-LSTM layers, which we define as a memory
efficient formulation to combine spatiel and temporal dependencies in one
cell. The feature sizes $h=512$ and $\theta=9$ are the feature size per node
in the graph, $[ORI,ACC]$ are the concatenated orientation and acceleration
values from the IMUs.

In addition to this standalone realization, our approach can also be utilized
inside higher level training schemes, such as the Transpose pipeline [44].
While the pathway of Transpose for pose estimation is essentially a
combination of three bidirectional LSTM networks, akin to the one proposed
with DIP [12], we can replace all these networks with our bidirectional AAGC-
LSTM network. In such a setup, first one network predicts the position of the
leaf joints, i.e., hands, feet and head from the IMU input. The second network
then takes the concatenation of these leaf positions and the IMU input to
predict the position of all joints, which are then again fed to the third
network together with the IMU input to predict the pose. The networks used in
the original Transpose model operate on flattened data, similar to DIP, which
requires us though to make a few adjustments for such a setup. To allow for
the same graph in each network, while also keeping the same graph structure
our standalone model employs, we do not directly feed the root joint data as
input to the model and predict the position of the outermost joints in the
first task, i.e., elbows, knees and head, as the leaf joints (hands and feet)
are not part of our graph. Similarly, the prediction of the position of all
joints with the second network is reduced to only the 15 joints which are part
of our graph. Furthermore, we also differ from Transpose by not training
towards the reduced 6-dimensional representation of rotation matrices [48],
but towards the conventional 9D representation instead, as we empirically
found better results for this representation. A short elaboration on these
results can be found in the supplemental material. Apart from the split of the
learning task in three steps, our two models are trained in the same fashion,
with the only difference being the scale normalization of the input data. For
our Transpose style model we followed Transpose and rescaled the acceleration
data by a constant factor of $30$, while we did not adapt any scale
normalization for the input data as we found this to achieve better results.
In both setups, standalone and Transpose, the models are trained towards
global rotations, using an MSE loss function utilizing longitudinal loss
weighting:

$L_{\theta}=\frac{1}{T}\sum_{t=1}^{T}\sum_{n=1}^{N}\lambda_{n}\sum_{f=1}^{F=9}(y_{t,n,f}^{true}-y_{t,n,f}^{pred})^{2}.$
(5)

where $L_{\theta}$ is the target loss of SMPL model parameter predictions. The
sum over $T$ defines the sequence mean, $N$ denotes the number of joints, $F$
the number of features and $\lambda$ denotes a joint specific weight.

Longitudinal loss weighting. When analyzing human movement data, a greater
variance and angular range of arm movements as compared to leg movements can
be observed. While the training data, especially the large synthetic corpus,
is assumed to be a good representation of common movements, the statistics of
different body parts show significant differences. As such, we have measured
the standard deviation of the transformed and to unit range normalized
rotation angles of the joints corresponding to arms, legs and torso in the
target data of the training corpus with $\sigma_{legs}=0.397$,
$\sigma_{arms}=0.473$ and $\sigma_{torso}=0.24$. Thus, we can see a
significant difference in the different body-regions. The arms show a
significant prevalence with respect to the legs and the torso. While the torso
shows the most significant difference to the other body regions, it should be
noted that the flexibility of the torso with respect to the root pelvis joint
is highly limited. In contrast to that, the legs and arms have a comparable
flexibility, while the difference in the underlying variance can be explained
by the simple fact that a big part of the data consists of upright movements,
which constrain the pose of the legs to such allowing for the subject to
stand. As a larger variance in specific joints also leads to a higher loss
contribution by these joints, the network will tend to learn better towards
the regions with higher variance. To make up for this fact, we exploit
longitudinal loss weighting (LLW), as a simple node-based weighting scheme,
which scales up the loss contribution of the lower body and scales down the
loss contribution of the upper body, such that the legs are trained with the
same accuracy as the arms. To do so, we set the values for $\lambda$
empirically as $\lambda_{n}=0.25$ if $n\in N_{torso}$, $\lambda_{n}=0.5$ if
$n\in N_{arms}$ and $\lambda_{n}=1$ if $n\in N_{legs}$, with the joint sets
defined as $N_{legs}=\\{0,1,3,4\\}$, $N_{arms}=\\{8,9,11,12,13,14\\}$ and
$N_{torso}=\\{2,5,6,7,10\\}$. For details on the analysis leading to this
selection, please refer to the supplemental material.

Contralateral data augmentation. Similar to Huang et al. [12], we train our
model on synthetic IMU data, generated from the AMASS motion capture dataset
[23] by utilizing the SMPL body model. Specifically, we use the preselected
and preprocessed data, provided by Huang et al.222
https://dip.is.tuebingen.mpg.de/ With the contralateral data augmentation
(CDA) we essentially mirror every sequence along the body’s main axis. Thus,
we utilize the joint based bilateral symmetry of the human body, in order to
enhance the set of movements both for training and validation, as well as
reducing bias in the data contributing to any left- or right-handedness in the
movements.

##  4 Experiments

We evaluate our method and compare against DIP and Transpose which define the
SOTA for sparse IMU-driven pose estimation. For this, we compare the methods
quantitatively and qualitatively on the DIP-IMU dataset [12] and the Total
Capture dataset [37]. For the implementation of our model we use TensorFlow
version 2.3 [1], whereby optimization is conducted with the Adam optimizer
[14] with an initial learning rate of $0.001$ and an exponential decay with a
rate of $0.96$ and a decay step of $2,000$. With the dropout on the input we
employ a rate of $0.2$ and on the cells hidden state we employ a dropout rate
of $0.3$. The gradients are further clipped to a norm of $1$. While Yi et al.
[44] train every network of their Transpose model separately, we note that we
trained our Transpose-like model combined, as we found no benefits of separate
training while it takes longer. Given this training procedure, we analyze the
results of our best performing models, before analyzing the influence of our
LLW and CDA.

| Total Capture |  | DIP-IMU
---|---|---|---
|  DIP Err $[deg]$ |  Ang Err $[deg]$ |  Pos Err $[cm]$ |  Jerk Err $[\frac{km}{s^{3}}]$ |  |  DIP Err $[deg]$ |  Ang Err $[deg]$ |  Pos Err $[cm]$ |  Jerk Err $[\frac{km}{s^{3}}]$
Trained on synthetic data
DIP [12] | $30.00(\pm 19.09)$ | $24.24(\pm 15.18)$ | $14.87(\pm 9.33)$ | $4.92(\pm 6.85)$ |  | $33.17(\pm 19.16)$ | $24.33(\pm 15.17)$ | $13.74(\pm 8.29)$ | $3.60(\pm 5.66)$
Transpose [44] | $18.49(\pm 15.68)$ | $13.78(\pm 9.40)$ | $8.22(\pm 6.82)$ | $\mathbf{0.64(\pm 1.97)}$ |  | $29.92(\pm 16.78)$ | $12.46(\pm 7.32)$ | $8.10(\pm 6.08$ | $\mathbf{1.00(\pm 3.50)}$
G-GRU [19] | $16.77(\pm 13.97)$ | $13.38(\pm 9.06)$ | $7.55(\pm 6.46)$ | $0.81(\pm 2.11)$ |  | $\mathbf{27.06(\pm 15.22)}$ | $11.86(\pm 6.80)$ | $7.58(\pm 5.66$ | $1.05(\pm 3.55)$
AGC-LSTM [34] | $18.35(\pm 15.08)$ | $13.89(\pm 9.81)$ | $8.30(\pm 7.56)$ | $0.80(\pm 2.22)$ |  | $29.42(\pm 16.53)$ | $13.38(\pm 7.99)$ | $8.41(\pm 6.14$ | $1.03(\pm 3.60)$
Ours | $\mathbf{15.81(\pm 12.38)}$ | $12.53(\pm 8.41)$ | $7.27(\pm 5.32)$ | $1.16(\pm 2.61)$ |  | $\mathbf{28.12(\pm 14.28)}$ | $\mathbf{11.35(\pm 6.28)}$ | $\mathbf{7.73(\pm 5.61)}$ | $1.12(\pm 3.55)$
Ours (TP) | $\mathbf{15.77(\pm 13.22)}$ | $\mathbf{12.28(\pm 8.38)}$ | $\mathbf{7.02(\pm 5.58)}$ | $0.89(\pm 2.32)$ |  | $29.04(\pm 14.42)$ | $11.59(\pm 6.22)$ | $8.10(\pm 5.73)$ | $1.07(\pm 3.56)$
Fine-tuned on real-world data
DIP [12] | $17.45(\pm 15.59)$ | $14.40(\pm 10.94)$ | $8.26(\pm 7.26)$ | $2.40(\pm 3.51)$ |  | $17.75(\pm 11.77)$ | $15.68(\pm 11.13)$ | $7.71(\pm 5.43)$ | $2.04(\pm 3.92)$
Transpose [44] | $17.03(\pm 14.74)$ | $11.72(\pm 8.29)$ | $7.43(\pm 5.95)$ | $\mathbf{0.63(\pm 1.96)}$ |  | $18.52(\pm 13.50)$ | $9.57(\pm 6.45)$ | $6.71(\pm 4.95)$ | $\mathbf{1.00(\pm 3.5)}$
G-GRU [19] | $14.61(\pm 12.80)$ | $10.95(\pm 7.81)$ | $6.70(\pm 5.61)$ | $0.89(\pm 2.12)$ |  | $15.77(\pm 10.69)$ | $9.02(\pm 5.71)$ | $6.09(\pm 4.06$ | $1.07(\pm 3.56)$
AGC-LSTM [34] | $17.39(\pm 15.39)$ | $12.58(\pm 9.49)$ | $8.08(\pm 7.42)$ | $0.86(\pm 2.30)$ |  | $19.03(\pm 14.12)$ | $10.76(\pm 7.40)$ | $6.88(\pm 5.14$ | $1.05(\pm 3.70)$
Ours | $\mathbf{13.12(\pm 10.99)}$ | $\mathbf{10.12(\pm 7.03)}$ | $\mathbf{6.00(\pm 4.64)}$ | $1.08(\pm 2.46)$ |  | $\mathbf{15.18(\pm 9.83)}$ | $\mathbf{8.13(\pm 5.23)}$ | $\mathbf{5.65(\pm 3.73)}$ | $1.13(\pm 3.54)$
Ours (TP) | $15.09(\pm 14.39)$ | $10.67(\pm 7.83)$ | $6.77(\pm 5.76)$ | $0.88(\pm 2.30)$ |  | $15.55(\pm 11.20)$ | $8.18(\pm 5.41)$ | $5.84(\pm 4.21)$ | $1.08(\pm 3.56)$

Table 1:  Evaluation of a single AAGC-LSTM network (Ours) and a AAGC-LSTM
network in Transpose setup (Ours (TP)) compared to the state of the art and
comparable approaches on DIP-IMU [12] and Total Capture [37]. We report the
mean global angular error over the shoulder and hip joints (DIP Err), as well
as the mean global angular error, the mean position error and the mean jerk
error averaged over all 15 joints.

###  4.1 Data Preparation

As mentioned above, the synthetic data used for training the models is
provided by Huang et al.[12]. The positions necessary for Transpose and our
model with Transpose setup were obtained by posing the SMPL model with the
true pose parameters. The DIP-IMU dataset comes with preprocessed and raw data
for each sequence. We used the raw data and filled any _nan_ values via
quadratic interpolation of acceleration and rotation in axis-angle
representation respectively, while the positions are computed similar to the
synthetic data. As the raw data is missing for one of the preprocessed
sequences we also have one less sequence in our processed DIP-IMU dataset. For
Total Capture [37] we used the same data preparation as Huang et al. to obtain
calibrated orientation and acceleration values, while we took the ground truth
from the AMASS dataset which provides the SMPL parameters for the Total
Capture dataset among others [23], with positions again computed as before.
For every model we report values of a model trained only on synthetic data as
well as one fine-tuned on real data. The fine-tuning is conducted on subsets
of the respective real datasets, while the test sets on which values are
reported are disjunct from this. The corresponding data split for DIP-IMU is
already provided with the dataset. For Total Capture we first split the data
by subjects, where the first three subjects are used for fine-tuning and the
last two are used as the test set on which the metrics are reported. The
former is again split randomly into training and validation sets for the fine-
tuning, with a ration of 70% to 30%. The split by subjects for the test set
allows for a complete test of the generalization capabilities of the model, as
it otherwise could overfit to specific characteristics of single subjects.

###  4.2 Quantitative Evaluation

We evaluate the methods on four metrics. These are the mean joint angle error
with respect to the joints selected for analysis by Huang et al. [12] building
on the work of SIP [38], as well as the mean joint error for angle, position
and jerk with respect to all 15 joints. The first metric only evaluates the
error on the shoulders and hip joints (0, 1, 11 and 12 according to the
numbering introduced in Fig. 1), which is a meaningful decision to analyze the
general capability of the model to generalize to the full pose from the IMU
measurements, as there is no IMU data directly available for these joints, but
they still are part of the extremities and thus a good indicator for the
correctness of the pose. As this gives no direct information of the total
accuracy for the pose estimation, we employ the other three metrics on the
complete skeleton. Angle and position are used together to give a valid
expression of the correctness of the pose, the error on the jerk as the third
derivative of position measures the temporal stability of the prediction by
quantifying effects such as trembling of joints or body parts in the
prediction, which are still in the ground truth and vice versa. To get the
values for position and jerk, we compute the respective joint positions using
the SMPL body model given the true and predicted $\theta$ values. The jerk $j$
is obtained from the respective true or predicted positions as a discrete
value:

$j_{t}=\frac{p_{t}-3p_{t-1}+3p_{t-2}-p_{t-3}}{\Delta t^{3}},$ (6)

where $t$ counts the frames, $p$ is the position and $\Delta t$ is the time
per frame. We report the comparison of our models with the SOTA in Table 1,
whereby we compare models trained solely on synthetic data as well as such
fine-tuned on a disjunct subset of the DIP-IMU and Total Capture data, which
are not part of the respective evaluation sets. We also compare to the methods
of Li et al. (G-GRU) [19] and Si et al. (AGC-LSTM) [34]. As they are build for
different tasks, we test them by replacing aour proposed AAGC-LSTMS with their
respective formulation for recurrent cells. We observe that the inclusion of
adjacency adaptive GCNs leads to great improvements on every metric but the
jerk error. While the latter is always best for Transpose, we note that only
the difference on the Total Capture dataset is of any significance, while at
the same time Transpose scores significantly worse in every other metric. We
further observe that the pairing of standard GCNs with an attention mechanism
in the recurrent cell can not compete with our approach, while it is only
slightly worse than Transpose. While the splitting of the pose estimation task
as proposed by Transpose leads to great improvement in almost all metrics
compared to DIP, we observe that this does not hold for its application on our
proposed model, except for the evaluation on Total Capture with a
synthetically trained model, where the results are still close. At this point
we highlight that the Transpose style model is by nature three times as large
as the standalone model and thus more compute intensive. Comparing the scores
of all models we see that our approach outperforms both DIP and Transpose by a
large margin in every metric but the jerk error. While we still outperform DIP
in the latter with a rise in accuracy of about $50\%$ in the worst case and
about $87\%$ in the best case, we do not outperform Transpose on this metric.
Nevertheless, our jerk error is still comparably good for the DIP-IMU dataset
and only beaten by Transpose on the Total Capture dataset, while our methods
in turn show significantly better results on all other metrics. At the same
time, the comparable techniques proposed in different fields can not compete
with our achieved accuracy, while G-GRU still outperforms the current SOTA on
our task. Thus, we conclude that our standalone model achieves the overall
best results, both for a model trained only on synthetic data, as well as for
one fine-tuned on real data.

###  4.3 Qualitative Evaluation

For the qualitative evaluation we visualized the poses using the SMPL model.
As the models predict only the parameters for the $15$ core joints, the
remaining $8$ joints for hands and feet are set to identity, while the pelvis
joint is fixed. Thus, in all visualizations there is no further rotation
applied to these regions.

Ground Truth DIP Transpose Ours Ours (TP) Figure 3:  Representative frames of
DIP-IMU. From left to right we show the ground truth, DIP, Transpose, our
approach and our approach with Transpose setup for different movements. On top
we show two examples among the best scoring poses relative to Transpose and on
the bottom two examples of the worst scoring poses with respect to our
standalone model.

In Fig. 3 we show some examples of our proposed method compared to the ground
truth and the SOTA for the DIP-IMU dataset. The top two rows show $2$
representative frames out of those with the best score of our standalone model
relative to Transpose. To select these we sort all frames from best to worst
with respect to the relative score between our standalone model and Transpose,
from which we selected the $10$ leading examples with the additional
constraint that at least $300$ frames are between samples to assert a
variation in the shown examples. The bottom two rows show $2$ representative
examples of the $10$ worst scoring frames with respect to our standalone
model, selected similarly. We observe that our model is better in
reconstructing poses with bent legs as can be seen in the first example of a
sitting pose. While there are still aspects to this pose which the model could
not fully reproduce, as for instance the arm and leg positions are still quite
off, we note that the reconstruction is visually closer to the ground truth
than that of DIP and Transpose. In the second presented example we see an
upright pose of a gesturing human. While every model can get this correctly in
the coarse structure, our standalone model is much better in reconstructing
the fine details of arm and leg positions, such as one straight and one
slightly bent leg as well as the visually perfect arm poses. With the examples
of the worst scoring poses we can observe the two prime factors leading to
problems. These are on one hand poses with high acceleration as can be seen
with the jumping jack pose (_third row_), as well as such with local rotations
of the torso with respect to the root (_fourth row_), as these are rare and
short-timed in the training data. In the former case no method can predict the
correct extent of the arms and rotations of the lower legs, with visually
equivalent poses given by our models and Transpose. While our model can
predict the poses of arms and legs to some degree in the latter case, it fails
to predict the torso rotation to any degree, while both competing methods have
the same problems. In addition to this short discussion we encourage the
reader to watch the supplemental video, in which we show more sequences in
animation to get a better understanding of the quality of the predictions, as
these are difficult to capture in static images.

###  4.4 Ablation Study

In this section we analyze the effects of the proposed longitudinal loss
weighting and the contralateral data augmentation. We list the results of this
ablation study in Table 2 for our standalone network. Refer to the
supplemental material for an additional study on our model in Transpose setup.

|  DIP Err $[deg]$ |  Ang Err $[deg]$ |  Pos Err $[cm]$ |  Jerk Err $[\frac{km}{s^{3}}]$
---|---|---|---|---
Ours | $\mathbf{15.18(\pm 9.83)}$ | $\mathbf{8.13(\pm 5.23)}$ | $\mathbf{5.65(\pm 3.73)}$ | $1.13(\pm 3.54)$
no LLW | $16.74(\pm 12.06)$ | $8.68(\pm 5.90)$ | $6.26(\pm 4.55)$ | $\mathbf{1.10(\pm 3.59)}$
no CDA | $20.14(\pm 14.58)$ | $9.41(\pm 6.40)$ | $6.54(\pm 5.07)$ | $1.13(\pm 3.65)$
Table 2: Ablation study on our standalone network. The rows indicate our best
performing model, the same trained without longitudinal loss weighting (LLW)
and the same trained without contralateral data augmentation (CDA).

We observe that for our model neither LLW nor CDA had any significant effect
on the jerk error. For the other metrics however wee see a great increase in
accuracy when incorporating both together. The application of LLW alone
(ablation of CDA) results in a significant increase of the angular and
position error, which is even surpassed by the application of CDA alone
(ablation of LLW), while the best results are only obtained by using both LLW
and CDA. While the jerk error shows the best result for the ablation of LLW,
we note that all models are very close on this metric.

##  5 Conclusion and Future Work

In this paper we have shown that the utilization of the human body graph
structure leads to better generalization towards pose estimation of unobserved
movements. We base this on the following observations. (i) the estimation of
the complete skeletal pose showed significant increases for both, a model
trained purely on synthetic data, and one fine-tuned on real-world data with
an angular error of only $8.13$ degrees on DIP-IMU, around 15% less than the
prior SOTA Transpose and $10.12$ degrees on Total Capture, around 14% less
than the prior SOTA. This was achieved by (ii) combining the spatio-temporal
processing of the sequential movement data in a single step, for which we
proposed the AAGC-LSTM cell, which processes both spatial and temporal
dependencies of the data in one recurrent cell with a lower amount of
parameters needed as existing methods. In addition to this we (iii) report a
boost in accuracy when using longitudinal, node-based loss weighting on the
graph and by (iv) utilizing the bilateral symmetry of the human body through
contralateral data augmentation. All code necessary to reproduce our results,
including the data preparation is publicly available on GitHub.333Link will be
provided upon acceptance of the paper.

Future work could consider a deeper look into the longitudinal loss weights,
and possibly make them incorporate case specific variability to better
generalize on rare poses. As the model is trained on synthetic data, the
differences of the synthetic IMUs to real IMUs should be explored to generate
even more realistic data. Furthermore, the large difference between the scores
on DIP-IMU and Total Capture for synthetically trained models suggest a great
need for more real IMU datasets. The synthetic training of the models also
allows for a exploration of optimal sensor positions. While all our
contributions have been developed for human pose estimation from sparse IMU
data, we would further like to investigate how they perform in other spatio-
temporal learning scenarios.

## References

  * [1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.  TensorFlow: Large-scale machine learning on heterogeneous systems, 2015\.  Software available from tensorflow.org.
  * [2] Felix Achilles, Alexandru-Eugen Ichim, Huseyin Coskun, Federico Tombari, Soheyl Noachtar, and Nassir Navab.  Patient mocap: Human pose estimation under blanket occlusion for hospital monitoring applications.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 491–499. Springer, 2016.
  * [3] Saeid Agahian, Farhood Negin, and Cemal Köse.  An efficient human action recognition framework with pose-based spatiotemporal features.  Engineering Science and Technology, an International Journal, 23(1):196–203, 2020.
  * [4] LEI BAI, Lina Yao, Can Li, Xianzhi Wang, and Can Wang.  Adaptive graph convolutional recurrent network for traffic forecasting.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 17804–17815. Curran Associates, Inc., 2020.
  * [5] Cristian Canton-Ferrer, Josep R Casas, and Montse Pardas.  Marker-based human motion capture in multiview sequences.  EURASIP Journal on Advances in Signal Processing, 2010:1–11, 2010\.
  * [6] Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li.  Simple and deep graph convolutional networks.  In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1725–1735. PMLR, 13–18 Jul 2020\.
  * [7] Yucheng Chen, Yingli Tian, and Mingyi He.  Monocular human pose estimation: A survey of deep learning-based methods.  CoRR, abs/2006.01423, 2020.
  * [8] Yu Cheng, Bo Yang, Bo Wang, Wending Yan, and Robby T Tan.  Occlusion-aware networks for 3d human pose estimation in video.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 723–732, 2019.
  * [9] Felix A Gers, Jürgen Schmidhuber, and Fred Cummins.  Learning to forget: Continual prediction with lstm.  1999\.
  * [10] Nicole Gruber and Alfred Jockisch.  Are gru cells more specific and lstm cells more sensitive in motive classification of text?  Frontiers in Artificial Intelligence, 3(40):1–6, 2020.
  * [11] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [12] Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll.  Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time.  ACM Transactions on Graphics (TOG), 37(6):1–15, 2018.
  * [13] Manuel Kaufmann, Yi Zhao, Chengcheng Tang, Lingling Tao, Christopher Twigg, Jie Song, Robert Wang, and Otmar Hilliges.  Em-pose: 3d human pose estimation from sparse electromagnetic trackers.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11510–11520, October 2021.
  * [14] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [15] Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  arXiv preprint arXiv:1609.02907, 2016.
  * [16] Jochen Klenk, Sebastian Wekenmann, Lars Schwickert, Ulrich Lindemann, Clemens Becker, and Kilian Rapp.  Change of objectively-measured physical activity during geriatric rehabilitation.  Sensors, 19(24):5451, 2019.
  * [17] Dennis Laurijssen, Steven Truijen, Wim Saeys, Walter Daems, and Jan Steckel.  An ultrasonic six degrees-of-freedom pose estimation sensor.  IEEE Sensors Journal, 17(1):151–159, 2016.
  * [18] Alberto Leardini, John J O’Connor, and Sandro Giannini.  Biomechanics of the natural, arthritic, and replaced human ankle joint.  Journal of foot and ankle research, 7(1):1–16, 2014.
  * [19] Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, and Qi Tian.  Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [20] Qimai Li, Zhichao Han, and Xiao-Ming Wu.  Deeper insights into graph convolutional networks for semi-supervised learning.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [21] Huei-Yung Lin and Ting-Wen Chen.  Augmented reality with human body interaction based on monocular 3d pose estimation.  In Jacques Blanc-Talon, Don Bone, Wilfried Philips, Dan Popescu, and Paul Scheunders, editors, Advanced Concepts for Intelligent Vision Systems, pages 321–331, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg.
  * [22] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black.  Smpl: A skinned multi-person linear model.  ACM transactions on graphics (TOG), 34(6):1–16, 2015.
  * [23] Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black.  AMASS: Archive of motion capture as surface shapes.  In International Conference on Computer Vision, pages 5442–5451, Oct. 2019.
  * [24] Alberto E Minetti, Alex P Moorhead, and Gaspare Pavei.  Frictional internal work of damped limbs oscillation in human locomotion.  Proceedings of the Royal Society B, 287(1931):20201410, 2020.
  * [25] Andrei Nicolicioiu, Iulia Duta, and Marius Leordeanu.  Recurrent space-time graph neural networks.  Advances in neural information processing systems, 32, 2019.
  * [26] Stěpán Obdržálek, Gregorij Kurillo, Jay Han, Richard Abresch, and Ruzena Bajcsy.  Real-time human pose detection and tracking for tele-rehabilitation in virtual reality.  Studies in health technology and informatics, 173:320–4, 01 2012\.
  * [27] Helge Rhodin, Jörg Spörri, Isinsu Katircioglu, Victor Constantin, Frédéric Meyer, Erich Müller, Mathieu Salzmann, and Pascal Fua.  Learning monocular 3d human pose estimation from multi-view images.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [28] Daniel Roetenberg, Henk Luinge, and Per Slycke.  Xsens mvn: Full 6dof human motion tracking using miniature inertial sensors.  Xsens Motion Technologies BV, Tech. Rep, 1, 2009.
  * [29] Ali Rohan, Mohammed Rabah, Tarek Hosny, and Sung-Ho Kim.  Human pose estimation-based real-time gait analysis using convolutional neural network.  IEEE Access, 8:191542–191550, 2020.
  * [30] Mike Schuster and Kuldip K Paliwal.  Bidirectional recurrent neural networks.  IEEE transactions on Signal Processing, 45(11):2673–2681, 1997\.
  * [31] Loren Arthur Schwarz, Diana Mateus, and Nassir Navab.  Discriminative human full-body pose estimation from wearable inertial sensor data.  In 3D physiological human workshop, pages 159–172. Springer, 2009\.
  * [32] Saurabh Sharma, Pavan Teja Varigonda, Prashast Bindal, Abhishek Sharma, and Arjun Jain.  Monocular 3d human pose estimation by generation and ordinal ranking.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
  * [33] Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu.  Two-stream adaptive graph convolutional networks for skeleton-based action recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12026–12035, 2019.
  * [34] Chenyang Si, Wentao Chen, Wei Wang, Liang Wang, and Tieniu Tan.  An attention enhanced graph convolutional lstm network for skeleton-based action recognition.  In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1227–1236, 2019.
  * [35] Ronit Slyper and Jessica K Hodgins.  Action capture with accelerometers.  In Symposium on Computer Animation, pages 193–199. Citeseer, 2008\.
  * [36] Jochen Tautges, Arno Zinke, Björn Krüger, Jan Baumann, Andreas Weber, Thomas Helten, Meinard Müller, Hans-Peter Seidel, and Bernd Eberhardt.  Motion reconstruction using sparse accelerometer data.  ACM Transactions on Graphics (ToG), 30(3):1–12, 2011.
  * [37] Matt Trumble, Andrew Gilbert, Charles Malleson, Adrian Hilton, and John Collomosse.  Total capture: 3d human pose estimation fusing video and inertial sensors.  In 2017 British Machine Vision Conference (BMVC), 2017.
  * [38] Timo Von Marcard, Bodo Rosenhahn, Michael J Black, and Gerard Pons-Moll.  Sparse inertial poser: Automatic 3d human pose estimation from sparse imus.  In Computer Graphics Forum, volume 36, pages 349–360. Wiley Online Library, 2017.
  * [39] Frank J. Wouda, Matteo Giuberti, Nina Rudigkeit, Bert-Jan F. van Beijnum, Mannes Poel, and Peter H. Veltink.  Time coherent full-body poses estimated using only five inertial sensors: Deep versus shallow learning.  Sensors, 19(17), 2019.
  * [40] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip.  A comprehensive survey on graph neural networks.  IEEE transactions on neural networks and learning systems, 2020\.
  * [41] Jingwei Xu, Zhenbo Yu, Bingbing Ni, Jiancheng Yang, Xiaokang Yang, and Wenjun Zhang.  Deep kinematics analysis for monocular 3d human pose estimation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [42] Sijie Yan, Yuanjun Xiong, and Dahua Lin.  Spatial temporal graph convolutional networks for skeleton-based action recognition.  In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  * [43] Xinyu Yi, Yuxiao Zhou, Marc Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu.  Physical inertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
  * [44] Xinyu Yi, Yuxiao Zhou, and Feng Xu.  Transpose: Real-time 3d human translation and pose estimation with six inertial sensors.  ACM Transactions on Graphics, 40(4), 08 2021.
  * [45] Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, and Dimitris N Metaxas.  Semantic graph convolutional networks for 3d human pose regression.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3425–3435, 2019.
  * [46] Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li.  T-gcn: A temporal graph convolutional network for traffic prediction.  IEEE Transactions on Intelligent Transportation Systems, 21(9):3848–3858, 2019.
  * [47] Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding.  3d human pose estimation with spatial and temporal transformers.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11656–11665, 2021.
  * [48] Yi Zhou, Connelly Barnes, Lu Jingwan, Yang Jimei, and Li Hao.  On the continuity of rotation representations in neural networks.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [49] Christian Zimmermann, Tim Welschehold, Christian Dornhege, Wolfram Burgard, and Thomas Brox.  3d human pose estimation in rgbd images for robotic task learning.  In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1986–1992. IEEE, 2018.
  * [50] Zhiming Zou and Wei Tang.  Modulated graph convolutional network for 3d human pose estimation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11477–11487, 2021.