Case I: The General Non-Abelian Case

# Warped Angle-deficit of a 5 Dimensional Cosmic String.

R J Slagter and D Masselink  Institute of Physics, University of Amsterdam
and
ASFYON, Astronomisch Fysisch Onderzoek Nederland, Bussum, The Netherlands
<EMAIL_ADDRESS>
###### Abstract

We present a cosmic string on a warped five dimensional space time in
Einstein-Yang-Mills theory. Four-dimensional cosmic strings show some serious
problems concerning the mechanism of string smoothing related to the string
mass per unit length, $G\mu\approx 10^{-6}$. A warped cosmic string could
overcome this problem and also the superstring requirement that $G\mu$ must be
of order 1, which is far above observational bounds. Also the absence of
observational evidence of axially symmetric lensing effect caused by cosmic
strings could be explained by the warped cosmic string model we present: the
angle deficit of the string is warped down to unobservable value in the brane,
compared to its value in the bulk. It turns out that only for negative
cosmological constant, a consistent numerical solution of the model is
possible.

##  1 Introduction

Recently, there is growing interest in the Randall-Sundrum(RS) warped 5D
geometry[1, 2]. One of the interesting outcomes of this idea is the solution
of the large hierarchy problem between the weak scale and the fundamental
scale of gravity. The predicted Kaluza-Klein particles in the model could be
detected with the LHC at CERN. In the original RS scenario, it was proposed
that our universe is five dimensional, described by the metric

$ds^{2}=e^{-2\mid y\mid ky_{c}}g_{\mu\nu}dx^{\mu}dx^{\nu}+y_{c}dy^{2}.$ (1)

The extra dimension y makes a finite contribution to the 5D volume because of
the exponential warp factor, where $y_{c}$ is the size of the extra dimension.
At low energies, gravity is localized at the brane and general relativity is
recovered. At high energy gravity ”leaks” into the bulk. The 4D Planck scale
will be an effective scale which can become much larger than the fundamental
Planck scale $M_{P}$ if the extra dimension is much larger than $M_{P}^{-1}$.
Further, the self-gravity of the brane must be incorporated. This will protect
the 3 dimensional space from the large extra dimensions by curvature rather
than straightforward compactification. Also matter fields in the bulk can be
incorporated. This will lead to a kind of ”holographic” principle, i.e., the
5D dynamics may be determined from knowledge of the fields on the 4D boundary.
For an overview, see [3]. We will consider here the 5D model with a general
Yang-Mills field, dependent of $r,y$ and $t$. In a following article we
investigate the interplay of the 4D and 5D coupled equations with the junction
conditions.

##  2 The model

We will consider here the RS2 model with two branes at $y=0$, the weak visible
brane and at $y=y_{c}$, the gravity brane (in the RS1 model one let
$y_{c}\rightarrow\infty$). The action of the model under consideration is [4]

$\displaystyle{\cal S}=\frac{1}{16\pi}\int
d^{5}x\sqrt{-^{(5)}g}\Bigl{[}\frac{1}{G_{5}}(^{(5)}R-\Lambda_{5})+\kappa\Bigl{(}^{(5)}R_{\mu\nu\alpha\beta}^{(5)}R^{\mu\nu\alpha\beta}-4^{(5)}R_{\alpha\beta}^{(5)}R^{\alpha\beta}$
(2) $\displaystyle+^{(5)}R^{2}\Bigr{)}-\frac{1}{g^{2}}Tr{\bf
F^{2}}\Bigr{]}+\int
d^{4}x\sqrt{-^{(4)}g}\Bigl{[}\frac{1}{G_{5}}\Lambda_{4}+S_{4}\Bigr{]}$ (3)

with $G_{5}$ the gravitational constant, $\Lambda_{5}$ the cosmological
constant, $\kappa$ the Gauss-Bonnet coupling, $g$ the gauge coupling,
$\Lambda_{4}$ the brane tension and $S_{4}$ the effective 4D Lagrangian, which
is given by a generic functional of the brane metric and matter fields on the
brane and will also contain the extrinsic curvature corrections due to the
projection of the 5D curvature. For the moment we will consider here only the
5D equation in a general setting and with a Yang-Mills matter field. The 4D
induced equations together with the junction conditions will be presented in
part 2 of a next article.

The coupled set of equations of the EYM-GB system will then become( from now
on all the indices run from 0..4)

$\displaystyle\Lambda_{5}g_{\mu\nu}+G_{\mu\nu}-\kappa GB_{\mu\nu}=8\pi
G_{5}T_{\mu\nu},$ (4) $\displaystyle{\cal D}_{\mu}F^{\mu\nu a}=0,$ (5)

with the Einstein tensor

$\displaystyle G_{\mu\nu}=R_{\mu\nu}-\frac{1}{2}g_{\mu\nu}R,$ (6)

and Gauss-Bonnet tensor

$\displaystyle
GB_{\mu\nu}=\frac{1}{2}g_{\mu\nu}\Bigl{(}R_{\gamma\delta\lambda\sigma}R^{\gamma\delta\lambda\sigma}-4R_{\gamma\delta}R^{\gamma\delta}+R^{2}\Bigr{)}-2RR_{\mu\nu}+4R_{\mu\gamma}{R^{\gamma}}_{\nu}$
(7)
$\displaystyle+4R_{\gamma\delta}{{{R^{\gamma}}_{\mu}}^{\delta}}_{\nu}-2R_{\mu\gamma\delta\lambda}{R_{\nu}}^{\gamma\delta\lambda}.$
(8)

Further, with $R_{\mu\nu}$ the Ricci tensor and $T_{\mu\nu}$ the energy-
momentum tensor

$\displaystyle T_{\mu\nu}={\bf
Tr}F_{\mu\lambda}F_{\nu}^{\lambda}-\frac{1}{2}g_{\mu\nu}{\bf
Tr}F_{\alpha\beta}F^{\alpha\beta},$ (9)

and with
$F_{\mu\nu}^{a}=\partial_{\mu}A_{\nu}^{a}-\partial_{\nu}A_{\mu}^{a}+g\epsilon^{abc}A_{\mu}^{b}A_{\nu}^{c}$,
and ${\cal
D}_{\alpha}F_{\mu\nu}^{a}=\nabla_{\alpha}F_{\mu\nu}^{a}+g\epsilon^{abc}A_{\alpha}^{b}F_{\mu\nu}^{c}$
where $A_{\mu}^{a}$ represents the YM potential.

We will consider the warped axially symmetric space time

$ds^{2}=-F(t,r,y)[dt^{2}-dz^{2}-dr^{2}-A(t,r,y)d\varphi^{2}]+dy^{2},$ (10)

with y the bulk dimension and the YM parameterization

$\displaystyle A_{t}^{(a)}=\Bigl{(}0,0,\Phi(t,r,y)\Bigr{)},\quad
A_{r}^{(a)}=A_{z}^{(a)}=A_{y}^{(a)}=0,$ (11) $\displaystyle
A_{\phi}^{(a)}=\Bigl{(}0,0,W(t,r,y)\Bigr{)}.$ (12)

So the metric and YM components depend t and the two space dimensions r and y.

The set of PDE’s become, for $\kappa=0$ for the time being,

$F_{tt}=F_{rr}+\frac{1}{2}FF_{yy}+\frac{3}{4F}(F_{t}^{2}-F_{r}^{2})+\frac{1}{2}\Lambda
F^{2}-\frac{4\pi G}{A}\Bigl{[}W_{r}^{2}-W_{t}^{2}+FW_{y}^{2}\Bigr{]},$ (13)
$\displaystyle
A_{tt}=A_{rr}+FA_{yy}-\frac{1}{2A}(A_{r}^{2}+A_{y}^{2}-A_{t}^{2})+\frac{1}{F}(F_{r}A_{r}-F_{t}A_{t}+2FF_{y}A_{y})$
(14) $\displaystyle-\frac{16\pi
G}{F}\Bigl{[}W_{t}^{2}-W_{r}^{2}-FW_{y}^{2}\Bigr{]},$ (15)
$W_{tt}=W_{rr}+FW_{yy}+\frac{1}{2A}W_{t}A_{t}+W_{y}(F_{y}-\frac{F}{2A}A_{y})-\frac{1}{2A}W_{r}A_{r}.$
(16)

##  3  The Static case

In the static case the resulting PDE’s become

$A_{rr}+FA_{yy}+2F_{y}A_{y}+\frac{F_{r}A_{r}}{F}-\frac{FA_{y}^{2}+A_{r}^{2}}{2A}+\frac{16\pi
G}{F}\Bigl{(}W_{r}^{2}+FW_{y}^{2}\Bigr{)}=0,$ (17)
$F_{rr}+FF_{yy}+F_{y}^{2}+\frac{F_{r}A_{r}+FF_{y}A_{y}}{2A}+\frac{2}{3}\Lambda
F^{2}-\frac{16\pi G}{3A}\Bigl{(}W_{r}^{2}+FW_{y}^{2}\Bigr{)}=0,$ (18)
$W_{rr}+FW_{yy}-\frac{W_{r}A_{r}}{2A}+W_{y}(F_{y}-\frac{FA_{y}}{2A})=0,$ (19)
$\Phi_{rr}+F\Phi_{yy}+\frac{\Phi_{r}A_{r}}{2A}+\Phi_{y}(F_{y}+\frac{FA_{y}}{2A})=0.$
(20)

We also have the two constraints

$F\Phi_{y}^{2}+\Phi_{r}^{2}=0,\quad W_{r}\Phi_{r}+FW_{y}\Phi_{y}=0.$ (21)

When we substitute the equations for $\Phi$ and W into the conservation
equation $\nabla_{\mu}T^{\nu\mu}=0$, we obtain identically zero, as it should
be. When we introduce the quantities $\theta_{i}$ defined by

$\displaystyle\theta_{1}\equiv\frac{F}{\sqrt{A}}A_{r},\quad\theta_{2}\equiv\sqrt{A}F_{r},\quad\theta_{3}\equiv\frac{F^{2}}{\sqrt{A}}A_{y},\quad\theta_{4}\equiv
F\sqrt{A}F_{y},$ (22)

then the equations can be written as

$\frac{\partial}{\partial r}\theta_{1}+\frac{\partial}{\partial
y}\theta_{3}=-\frac{16\pi G}{\sqrt{A}}(W_{r}^{2}+FW_{y}^{2}),$ (23)
$\frac{\partial}{\partial r}\theta_{2}+\frac{\partial}{\partial
y}\theta_{4}=\frac{16\pi
G}{3\sqrt{A}}(W_{r}^{2}+FW_{y}^{2})-\frac{2}{3}\Lambda F^{2}\sqrt{A},$ (24)
$\Bigl{[}\frac{W_{r}}{\sqrt{A}}\Bigr{]}_{r}+\Bigl{[}\frac{FW_{y}}{\sqrt{A}}\Bigr{]}_{y}=0,$
(25)
$\Bigl{[}\sqrt{A}\Phi_{r}\Bigr{]}_{r}+\Bigl{[}F\sqrt{A}\Phi_{y}\Bigr{]}_{y}=0.$
(26)

The Ricci scalar ${}^{(5)}R$ becomes:

${}^{(5)}R=\frac{8\pi
G}{3AF^{2}}\bigl{(}FW_{y}^{2}+W_{r}^{2}\Bigr{)}+\frac{5}{3}\Lambda_{5}.$ (27)

We now investigate the properties of the static solution for large values of
$r$ and $y$. We will assume that

$\int_{0}^{\infty}\sqrt{A}\sigma dr$ (28)

converges, where $\sigma$ is the energy density $T_{0}^{0}=-4\pi
G\frac{W_{r}^{2}+FW_{y}^{2}}{2AF^{2}}$. Further,

$\lim_{r\rightarrow\infty}\sqrt{A}\sigma=0.$ (29)

##  4  Analysis of the Angle Deficit

The angle deficit can be calculated for a class of static translational
symmetric space times which are asymptotically Minkowski minus a wedge. If we
denote with $l$ the length of an orbit of
$\Bigl{(}\frac{\partial}{\partial\varphi}\Bigr{)}^{a}$ in the brane, then the
angle deficit is given by[5, 6, 7]

$(2\pi-\Delta\varphi)=\lim_{r\rightarrow\infty}\frac{dl}{dr},$ (30)

with

$l=\int_{0}^{2\pi}\sqrt{g_{ab}\Bigl{(}\frac{\partial}{\partial\varphi}\Bigr{)}^{a}\Bigl{(}\frac{\partial}{\partial\varphi}\Bigr{)}^{b}}d\varphi.$
(31)

One better can use the Gauss-Bonnet theorem to obtain the angle deficit by
calculating the integral of the Gaussian curvature over the surface of
$S(t,z)$ = const. If one transports a vector around a closed curve, then the
angle rotation $\alpha$ will be given[7] by the area integral over of the
subsurface of S

$\alpha=\int d^{3}x\sqrt{{}^{(3)}g}^{(3)}K,$ (32)

with

${}^{(3)}K=\frac{1}{2}{{}^{(3)}g}^{ik}{{}^{(3)}g}^{jl}{{}^{(5)}R}_{ijkl}.$
(33)

For our case, we obtain

$\displaystyle\sqrt{{}^{(3)}g}{{}^{(3)}K}=-\frac{1}{2}\Bigl{(}\frac{F_{r}\sqrt{A}}{F}\Bigr{)}_{r}-\frac{1}{2}\Bigl{(}\frac{A_{r}}{\sqrt{A}}\Bigr{)}_{r}-\frac{1}{2}\Bigl{(}\frac{FA_{y}}{\sqrt{A}}\Bigr{)}_{y}-(\sqrt{A}F_{y})_{y}$
(34)
$\displaystyle+\frac{1}{4}\sqrt{A}F_{y}\Bigl{(}\frac{F_{y}}{F}+\frac{A_{y}}{A}\Bigr{)}$
(35)
$\displaystyle=-\frac{1}{4}\Bigl{[}\frac{F_{y}^{2}}{F^{2}}+\frac{F_{r}^{2}}{F^{3}}-\frac{2}{3}\Lambda-\frac{80\pi
G}{3F^{2}A}(W_{r}^{2}+FW_{y}^{2})\Bigr{]}.$ (36)

Then Eq.(28) becomes

$\displaystyle\alpha=-\pi\Bigl{\\{}\Big{[}\Bigl{(}\frac{\sqrt{A}F_{r}}{F}\Bigr{)}+\Bigl{(}\frac{A_{r}}{\sqrt{A}}\Bigr{)}\Bigr{]}^{\infty}_{r=0}+\Bigl{[}\frac{FA_{y}}{\sqrt{A}}+2\sqrt{A}F_{y}\Bigr{]}^{\infty}_{y=0}$
(37)
$\displaystyle-\frac{1}{2}\int_{0}^{\infty}\int_{0}^{\infty}\sqrt{A}F_{y}(\frac{F_{y}}{F}+\frac{A_{y}}{A})drdy\Bigr{\\}}.$
(38)

Or, using the second expression in Eq.(30)

$\displaystyle\alpha=-\frac{1}{2}\pi\int_{0}^{\infty}\int_{0}^{\infty}F\sqrt{A}\Bigl{(}\frac{20}{3}\sigma-\frac{2}{3}\Lambda\Bigr{)}drdy$
(39)
$\displaystyle-\frac{1}{2}\pi\int_{0}^{\infty}\int_{0}^{\infty}F\sqrt{A}\Bigl{[}\frac{F_{y}^{2}}{F^{2}}+\frac{F_{r}^{2}}{F^{3}}\Bigr{]}drdy.$
(40)

If one assumes that in the 4 dimensional case, for
$r\rightarrow\infty:F\rightarrow 1$ and $\sqrt{A}\rightarrow br$ and for
$r\rightarrow 0:F\rightarrow 1$ and $\sqrt{A}\rightarrow r$, than the first
term of Eq.(31) represents the well-known result [6] that $\alpha=2\pi(1-b)$
in de brane, so S is asymptotically a conical surface. In the 5 dimensional
case the results depend on the boundary values of our warp factor F, i.e., the
last two terms in Eq. (31). From Eq.(32) one observes that the first term
represents the proper mass per unit length of the string plus a contribution
from the cosmological constant. The second term is the correction term.

Now we try to obtain for the asymptotic warped metric

$ds^{2}=F_{c}e^{k_{2}y+a_{2}}\Bigl{[}-dt^{2}+dz^{2}+dr^{2}+(k_{1}r+a_{1})^{2}d\varphi^{2}\Bigr{]}+dy^{2}.$
(41)

For $y=0$ we recover de 4D result of a flat space time minus a wedge by the
transformation[6]

$\displaystyle
r^{\prime}=r+\frac{a_{1}}{k_{1}},\qquad\varphi^{\prime}=k_{1}\varphi\quad(0\leq\varphi\leq
2\pi),$ (42)

i.e.,

$ds^{2}=-dt^{2}+dz^{2}+d(r^{\prime})^{2}+(r^{\prime})^{2}d(\varphi^{\prime})^{2}+dy^{2},$
(43)

where now $\varphi^{\prime}$ has a different range then $\varphi$. For $y\neq
0$ we have the warped metric

$ds^{2}=F_{c}e^{k_{2}y+a_{2}}\bigl{[}-dt^{2}+dz^{2}+d(r^{\prime})^{2}+(r^{\prime})^{2}d(\varphi^{\prime})^{2}\Bigl{]}+dy^{2}.$
(44)

The angle deficit is determined by $k_{1}F_{c}e^{k_{2}y+a_{2}}$.

Let us consider now

$\displaystyle\frac{\partial}{\partial
r}(\theta_{1}+\theta_{2})+\frac{\partial}{\partial
y}(\theta_{3}+\theta_{4})=-\frac{32\pi
G}{3\sqrt{A}}\Bigl{(}W_{r}^{2}+FW_{y}^{2}\Bigr{)}-\frac{2}{3}\Lambda\sqrt{A}F^{2}$
(45) $\displaystyle=-\frac{32\pi G}{3}\Bigl{[}\frac{\partial}{\partial
r}\Bigl{(}\frac{WW_{r}}{\sqrt{A}}\Bigr{)}+\frac{\partial}{\partial
y}\Bigl{(}\frac{FWW_{y}}{\sqrt{A}}\Bigr{)}\Bigr{]}-\frac{2}{3}\Lambda\sqrt{A}F^{2},$
(46)

where we used the Eq.’s (19)-(22). After rearranging we then obtain

$\frac{\partial}{\partial r}\Bigl{(}\theta_{1}+\theta_{2}+\frac{32}{3}\pi
G\frac{WW_{r}}{\sqrt{A}}\Bigr{)}=-\frac{\partial}{\partial
y}\Bigl{(}\theta_{3}+\theta_{4}+\frac{32}{3}\pi
G\frac{FWW_{y}}{\sqrt{A}}\Bigr{)}-\frac{2}{3}\Lambda\sqrt{A}F^{2}.$ (47)

So we notice that the $\Phi$-field disappears from the equation. It will have
only a contribution on the brane. It is quite easy to obtain a particular
solution of this equation, Eq.(38). For

$\displaystyle F=F_{c}e^{\pm\sqrt{-\Lambda}y+a_{2}}\qquad
A=A_{c}(k_{1}r+a_{1})^{2},$ (48)

we obtain for $W$ a solution of the form $W(r,y)=W_{1}(r)W_{2}(y)$, where
$W_{1}$ and $W_{2}$ are given by Bessel functions. This oscillatory behavior
of $W$ is not uncommon for gravitating YM vortices.

So it seems to be possible to find the desired asymptotic warped form for the
conical space time, i.e., Eq.(33).

The next task is to obtain from the junction condition and the brane-bulk
splitting, relations between the several constants in the model.

##  5 Numerical solutions

For a given set of initial conditions, these PDE’s determine the behavior of
F, A, W and $\Phi$. We will impose particular asymptotic conditions, in order
to obtain acceptable solutions of the cosmic string. First, we have
$\lim_{r\rightarrow\infty}\Phi(r,y)=1,\lim_{r\rightarrow\infty}W(r,y)=0,F(0,y)=1,A(0,y)=0,\frac{\partial}{\partial
r}A(0,y)=1$. Further,
$\lim_{r\rightarrow\infty}\frac{g_{\varphi\varphi}}{r^{2}}=1$.

We solve the system using the numerical code CADSOL-FIDISOL. The asymptotic
form of the metric component $g_{\varphi\varphi}$ behaves as expected.

Figure 1: Typical solution of F, A, W and $\Phi$ for negative $\Lambda$ with
initial values: $F=e^{(-r+y)},A=r^{2},W=e^{(-r^{2}-y^{2})},\Phi=1-e^{(-r-y)}$
and Dirichlet boundary conditions on the outer boundaries. We also plotted
$g_{\varphi\varphi}$

##  6 Conclusions

In earlier attempts[8, 9, 10], we tried to build a 5-dimensional cosmic string
without a warp factor and investigated the causal structure. Here we
considered a different approach. It seems possible that the absence of cosmic
strings in observational data could be explained by our model, where the
effective angle-deficit resides in the bulk and not in the brane. In this part
we considered the 5D equations in general form, without the splitting of the
energy-momentum tensor in a bulk and brane part. We find a consistent set of
equations in the bulk. The asymptotic behavior of the metric outside the core
of the string seems to have the desired form. This solution must be consistent
with the system of equations obtained by the bulk-brane splitting. It is also
interesting to investigate holographic ideas in our model. These subjects are
under study by the authors and will be published in a followup article.

Figure 2: The 5-D cosmic string

## References

## References

  * [1] Randall L and Sundrum R 1999 Phys. Rev. Lett. 83 3370, 4690
  * [2] Randall L and Sundrum R ,hep-th/9905221, hep-th/9906064
  * [3] Maartens R, gr-qc/0312059
  * [4] Okuyama N and Maeda K 2008 arXiv: gr-qc/0212022v2
  * [5] Vilenkin A and Shellard E P S 1994 Cosmic Strings and Other Topological Defects Cambridge Monographs
  * [6] Garfinkle D Phys. Rev. D 32,6 1323
  * [7] Ford L H and Vilenkin A 1981 J. Phys. A: Math. Gen. 14 2353
  * [8] Slagter R J 2006 in the proceedings of the conferenceMG11 Berlin page 1434
  * [9] Slagter R J 2008 Int.J.Mod.Phys.D 18 613
  * [10] Slagter R J 2009 in the procedings of the conference The Invisible Universe Paris


# Learning Local-Global Contextual Adaptation for Multi-Person Pose Estimation

Nan Xue 1 Tianfu Wu 2 Gui-Song Xia 1 Liangpei Zhang 3
1 School of Computer Science, Wuhan University
2 Department of ECE, NC State University
3 LIESMARS, Wuhan University

###### Abstract

This paper studies the problem of multi-person pose estimation in a bottom-up
fashion. With a new and strong observation that the localization issue of the
center-offset formulation can be remedied in a local-window search scheme in
an ideal situation, we propose a multi-person pose estimation approach, dubbed
as LOGO-CAP, by learning the LOcal-GlObal Contextual Adaptation for human
Pose. Specifically, our approach learns the keypoint attraction maps (KAMs)
from the local keypoints expansion maps (KEMs) in small local windows in the
first step, which are subsequently treated as dynamic convolutional kernels on
the keypoints-focused global heatmaps for contextual adaptation, achieving
accurate multi-person pose estimation. Our method is end-to-end trainable with
near real-time inference speed in a single forward pass, obtaining state-of-
the-art performance on the COCO keypoint benchmark for bottom-up human pose
estimation. With the COCO trained model, our method also outperforms prior
arts by a large margin on the challenging OCHuman dataset.

##  1 Introduction

2D human pose estimation is a classical computer vision problem that aims to
parsing articulated structures of human parts from natural images. With rich
and longstanding studies, we have witnessed great successes on single-person
pose estimation [22, 32] by convolutional neural networks. Therefore, it is of
great interest in pushing the pose estimation from the single-person to multi-
person configuration.

Figure 1: An illustrative example of multi-person pose estimation by the
proposed LOGO-CAP with the HRNet-W32 backbone. For each initial pose obtained
by the center-offset regression, LOGO-CAP learns $11\times 11$ local filters
for each joint, and then refines the initial keypoint by convolution with the
learned kernels: The local filters are learned to refocusing those initially-
less-accurate pose keypoints towards better placement. In more detailed, we
show an example of the initial center-offset pose that only obtains the OKS of
0.826 due to the misplacement between the keypoints of the right elbow and the
right wrist. We mark the residual vectors between the predictions and the
groundtruth with yellow and red dash lines respectively. The LOGO-CAP improves
the OKS by 10.4%. Please see text for details.

The problem of multi-person pose estimation from images has been extensively
studied in top-down paradigms [32, 37, 22] that formulate the problem as
single-person pose estimation with an off-the-shelf person detector with an
impressively high AP (e.g, 56% on the COCO-2017 validation dataset [37]).
Although the top-down paradigm has been dramatically pushed into a high-
performing stages, it remains several problems in both aspects of efficiency
and accuracy due to the dependency of detecting person bounding boxes.
Motivated by this, we are interested in studying the problem of multi-person
pose estimation without incurring extra priors of bounding boxes, which is
conventionally termed as _bottom-up pose estimation_.

50100150200$\displaystyle\geq$ 500Inference time (in ms)60.065.070.088.9COCO
Keypoints APAn Empirical Upper BoundW32-384W32-512W48-512W48-640LOGO-CAP
(Ours)DEKR [11]PifPaf [15]CenterNet [42]AE+HrHRNet-W32 [5]Baseline-W32

Figure 2: Our LOGO-CAP is motivated by a strong empirical observation that a
vanilla center-offset baseline with an AP of 60.1 (marked by green dashed
line) can be improved by leveraging a searching scheme in $11\times 11$ local
windows to an AP of 88.9 (marked by red dashed line). In the meanwhile, we
illustrate the speed-accuracy comparisons between our LOGO-CAP and prior arts
on the COCO val-2017 dataset. W$x$-$Y$ (e.g. W32-384) means that a model uses
the backbone HRNet-W$x$ [32] and is tested with the image resolution $Y$ in
the short side.

Bottom-up pose estimation approaches pay much attention on estimating pose
parameters from any given multi-person image instead of using the cropped
single-person ones, which poses several challenges on accurately identifying
the learned bottom-level keypoints as person parts and grouping them into
different individuals by learning part affinity fields [3, 2], part
association fields [15], associative embedding [21]. Those grouping approaches
are accurate but sophisticated due to the necessity of ad-hoc
grouping/decoding schemes. Recently, many researchers attempted to learning
center-offsets [42, 6, 31, 11] for pose estimation as its intrinsic simplicity
and efficiency. However, most of the center-offset approaches are suffering
from the main challenge of localization inaccuracy due to the large structural
variations of human pose, thus leading to inferior performance than the
grouping ones.

In this paper, we are interested in studying the bottom-up pose estimation
using the center-offset formulation for its simplicity and efficiency. We
directly address the aforementioned main challenges for center-offset based
multi-person pose estimation. Our proposed approach is motivated by a
surprisingly strong empirical observation via analyzing what the fundamental
issues are with the vanilla center-offset pose estimation network.

A Surprisingly Strong Observation. Our analyses are based on results in the
fully-annotated subset of the COCO val-2017 dataset111Note that the COCO-
val-2017 dataset contain many partially-annotated images (with only ground-
truth bounding boxes), we use 2346 testing samples that are fully annotated
with keypoint annotations.. The vanilla center-offset regression method
utilizes the HRNet-W32 [32] as the feature backbone to directly predict
keypoints center heatmap and the offset vectors. As shown in Fig. 2, it
obtains $60.1$ average precision (AP), which is not great, but reasonably
good. It clearly shows that the pose keypoints center and the offset vectors
can be learned reasonably well. We ask: How exactly bad are the center-offset
estimation? We want to know (i) whether some of keypoints are truly bad being
far away from the underlying ground-truth (GT) locations, or (ii) whether most
of them are already in the close proximity of the GT ones? We observe that the
latter is true. To quantitatively characterize the close proximity, instead of
directly utilizing the learned offset vectors for human pose estimation, we
treat them as human pose keypoint initialization and do a local window search
to compute the empirical upper-bound of performance. More specifically, based
on the initially predicted human poses, by introducing a local window (e.g.,
$11\times 11$) centered at each detected key point and by computing the single
keypoint similarity with the ground-truth keypoint, an empirical upper-bound
of $88.9$ AP is obtained, which is significantly higher than the state of the
art and shows the potential of improving the vanilla center-offset regression
paradigm.

A Direct Solution Leveraging the Observation. The implication of the
observation is significant: It reveals that the fundamental challenge of
improving the center-offset approach for pose estimation is to resolve the
local misplacement. To that end, a straightforward way is just to learn a
local heatmap (e.g., $11\times 11$) for each human pose keypoint based on the
learned center and offset vectors, and then to compute the refined keypoints
by taking $\arg\max$ within the local heatmap. Although appealing, this does
not work as observed during our development of the LOGO-CAP. The underlying
reason is also straightforward: if this can work, the original offset vector
regression should work at the first place since no additional information is
introduced through learning the local heatmap.

We hypothesize that on the one hand, in addition to the local heatmap, the
structural relationship between different pose keypoints needs to be taken
into account, and on the other hand, the intrinsic uncertainty of the local
information in a local heatmap needs to be resolved. The former is the key
challenge of structured output prediction problems. Many message passing
algorithms have been developed in the literature. The latter can not be
addressed by simply increasing the local window size. It entails learning
stronger local-global information interaction and adaptation.

To verify the two hypotheses, the proposed LOGO-CAP lifts the initial
keypoints via the center-offset prediction to keypoint expansion maps (KEMs)
to counter their lack of localization accuracy in two modules (Section 3). The
KEMs extend the star-structured representation of the center-offset
formulation to the pictorial structure representation [10, 9]. As shown in
Fig. 1, in our LOGO-CAP, one module computes local KEMs and learns to account
for the structured output prediction nature of the human pose estimation
problem, resulting in the keypoint attraction maps (KAMs), i.e., the local
filters in Fig. 1. Another module computes global KEMs and learns to refine
the global KEMs by integrating the KAMs.

Our LOGO-CAP is a fully end-to-end bottom-up human pose estimation method with
near real-time inference speed. It obtains $70.0$ AP in the fully-annotated
subset of the COCO val-2017 dataset, which is an absolute increase of $9.9$ AP
compared to the vanilla center-offset method, making a significant step
forward. Fig. 2 shows the advantage of the proposed LOGO-CAP in terms of
overall speed-accuracy comparisons between our LOGO-CAP and prior arts.
Meanwhile, we should notice that there is still a significant gap towards the
empirical upper bound in Fig. 2, which encourages more work to be
investigated.

##  2 Related Works and Our Contributions

There is a vast body of literature for human pose estimation. Many elegant
representation schema have been developed for modeling articulated human pose
in the traditional approaches such as the well-known pictorial structure model
[10, 9] and its many variants [27, 1, 26, 38, 29]. With the resurgence of DNNs
and the end-to-end learning, the performance of single-person pose estimation
has been largely improved. We briefly review the recent deep learning based
approaches for multi-person pose estimation.

Top-Down Pose Estimation. It essentially exploits single-person pose
estimation approaches on cropped single human image patches [22, 32, 37, 4, 7,
25, 17], where human detection is often done by an off-the-shelf object
detector (_e.g_., Faster-RCNN [28]). Although excellent performance has been
achieved in such a pipeline, they are suffering from efficiency issue due to
the dependency of object detectors. Accordingly, there are work focusing on
developing efficient backbone networks (_e.g_., Lite-HRNet [39, 30, 20]) to
reduce the inference latency on single-person pose estimation, often at the
expense of degenerating the accuracy of pose estimation by large margins.
Mask-RCNN [12] addresses this problem by learning heatmaps from features
extracted by ROIAlign in object proposals, whose performance on pose
estimation lags behind. To clarify the use of terminology, top-down approaches
referred in this paper are those using the precomputed bounding boxes as done
in the SimpleBaseline method [37]. Compared with the top-down approaches, our
LOGO-CAP directly addressed the problem of multi-person pose estimation in a
bottom-up way without incurring the regional image/feature context over the
person boxes. Our proposed LOGO-CAP obtains competitive performance in terms
of accuracy while achieving nearly real-time inference speed.

Limb-based Grouping Approaches. Motivated by the naturalness of modeling limbs
based on keypoints, the problem of multi-person pose estimation has been
extensively studied by grouping persons from the learned limb associations.
Given a predefined limb configuration (_e.g_., the COCO person skeleton
template consisting of 19 limbs based on 17 keypoints), the grouping can be
addressed by Part affinity field (PAF) [3, 2], Associative Embedding (AE)
[21], mid-range offset fields in PersonLab [24] and the fields of Part
Intensity and Association [15]. Typically, sophisticated designs are entailed
to achieve good performance. For example, a bipartite graph matching is used
in OpenPose [2]. In addition to be computationally expensive, another drawback
of these methods is not fully end-to-end trainable. More recently, the
differentiability issue was studied by the Hierarchical Graph Clustering (HGG)
method [13], which utilizes graph convolution networks to repeatedly delineate
pose parameters of multiple persons from a keypoint graph. HGG improves the
performance compared to its baseline, the Associative Embedding method [21] at
the expense of significantly increased computational cost. In contrast to
those approaches, our proposed LOGO-CAP is fully end-to-end trainable and
achieves near real-time inference speed.

Figure 3: Schematic illustration of the proposed LOGO-CAP for bottom-up human
pose estimation. See text for detail.

Direct Regression based Approaches. This formulation has attracted much
attention due to their conceptual simplicity [6, 33, 31, 11, 35], inspired by
the recent remarkable success of direct bounding box regression in object
detection such as the FCOS method [34] and CenterNets [42, 6]. As
aforementioned, one main challenge is the difficulty of accurately regressing
the offset vectors, especially for the long-range keypoints with respect to
the center. Sophisticated post-processing schema are often entailed to improve
the performance. For example, a method of matching the directly regressed
poses to the nearest keypoints that are extracted from the global keypiont
heatmaps is used in [42]. Although being simple, the performance of this line
of work is usually inferior to the limb-based approaches. The mixture
regression network [35] alleviated the issue of regression quality to some
extent, but still remained an indispensable performance gap comparing with the
grouping-based approaches. Most recently, Geng _et al._ presented the first
competitive direct method, DEKR [11] with a novel pose-specific neural
architecture for disentangled keypoint regression. To improve the performance,
the DEKR method utilizes a lightweight rescoring network to recalibrate the
pose scores that are computed based on the keypoint heatmaps , and thus is not
fully end-to-end. The proposed LOGO-CAP retains the simplicity of the vanilla
center-offset formulation and enjoys fully end-to-end training and fast
inference speed.

Our Contributions. The proposed LOGO-CAP makes three main contributions to the
field of bottom-up human pose estimation: (i) We address the drawback of the
vanilla center-offset formulation while retaining its efficiency. It proposes
the key idea of lifting a keypoint to a keypoint expansion map to counter the
lack of localization accuracy. (ii) We present a novel local-global contextual
adaptation formulation that accounts for the nature of structured output
prediction in human pose estimation and harnesses local-global structural
information integration. (iii) Our proposed LOGO-CAP obtains state-of-the-art
performance on the COCO val-2017 and test-2017 datasets. It also shows the
strong generalization ability with state-of-the-art performance on the OCHuman
dataset.

##  3 The Proposed LOGO-CAP

Fig. 3 illustrates the proposed LOGO-CAP, which consists of three main
components: a feature backbone, the initial center-offset human pose
estimation, and the proposed local-global contextual adaptation component for
the final human pose estimation.

###  3.1 Learning Local and Global Context

Backbone Network and Pose Initialization. Given an input image $I$, the output
of the feature backbone is a $C$-dim feature map, denoted by $\mathbf{F}\in
R^{C\times h\times w}$, where $C$ is the feature dimension of the last
convolutional layer in the feature backbone, and the spatial size $h\times w$
depends on the total stride in the feature backbone. The feature map $F$ is
used as the shared features by two different head branches for the prediction
of center heatmaps $\mathbf{C}$ and offsets fields $\mathbf{O}$. Then, the
initial pose parameters are extracted using the top-$N$ local maximal
locations and the corresponding offset vectors, denoted by
$\mathbf{p}_{i}\in\mathbb{R}^{17\times 2}$ for the $i$-th person.

Local Context. We first introduce the local keypoint expansion maps (KEMs) for
the initial pose parameters. Specifically, for the $j$-type keypoint (_e.g_.,
nose of a person), we follow the strong observation in Sec. 1 to compute the
local KEMs in $11\times 11$ windows as $\mathcal{M}_{N\times 17\times 11\times
11\times 2}$, as shown in Algorithm 1.

1

$\sigma$= coco_sigmas #$17\times 1$, keypoint sigmas provided in the COCO
dataset.

2

3def  KptsExpansionCoco($\mathcal{P}$,ks=11)

#Initial poses $\mathcal{P}$: Nx17x2

4 r = ks // 2

5 dy, dx = meshgrid(arange(-r,r),arange(-r,r))

#dx, dy: ks$\times$ks

6

7 dy = dy.reshape(1, 1, ks, ks)

8 dx = dx.reshape(1, 1, ks, ks)

scale = $\sigma$.reshape(1, 17, 1, 1)/$\sigma$.min() #keypoint type specific
expansion rate

9

dy = dy $*$ scale #1x17x11x11

10

dx = dx $*$ scale #1x17x11x11

11

dxy = stack((dx,dy),dim=-1) #1x17x11x11x2

12

$\mathcal{M}$ = $\mathcal{P}$.reshape(N, 17, 1, 1, 2) + dxy #Nx17x11x11x2

13

14 return $\mathcal{M}$

15

Algorithm 1 Computing KEMs in a PyTorch-like style

Then, we encode the geometric mesh of KEMs $\mathcal{M}\in\mathbb{R}^{N\times
17\times 11\times 11\times 2}$ in a $d$-dim latent space (_e.g_., $d=64$ in
our experiments), computed based on the feature backbone output. A pose
instance is represented by concatenating all the 17 keypoints. All initial
keypoint-based poses are geometrically “expanded / lifted” and feature-
activated, resulting the initial local context,
$\mathcal{K}\in\mathbb{R}^{N\times(17\times d)\times 11\times 11}$.

Local Context Convolutional Message Passing. To facilitate the structural
information flow between different latent codes of the keypoints of a pose
instance, we propose a simple convolutional message passing (CMP) module with
three layers of Conv+Norm+ReLU operations with the Attentive Norm [18] used in
the second layer. The transformed latent code $\mathcal{K}^{\prime}$ is
decoded by a $1\times 1$ Conv. layer as the local keypoints attraction maps
(KAMs), $\mathbf{K}\in\mathbb{R}^{N\times 17\times 11\times 11}$ to measure
the uncertainty of the initial poses.

Local-Global Contextual Adaptation. Through the CMP, we obtain the dynamic
(a.k.a., data-driven) kernels for the 17 keypoints in a pose instance-
sensitive way, which are used to refine the global heatmaps $\mathcal{H}$ for
the 17 keypoints. In detail, we first compute another geometric mesh with
window $a\times a$ (e.g., $a=97$) for each keypoint of the $N$ pose instances,
and the entire mesh is denoted by $\mathcal{M}_{G}\in\mathbb{R}^{N\times
17\times a\times a\times 2}$. The mesh can be interpreted as the global KEM.
It is then instantiated with appearance features extracted from the global
heatmaps, and we have,

$\displaystyle\mathcal{H}^{\uparrow}_{(1:17)}\xRightarrow[\text{bi-
linear}]{\mathcal{M}_{G}}\mathbb{H}\xRightarrow[\text{reweighing}]{\mathcal{G}_{a\times
a}(0,\sigma)}\bar{\mathbb{H}}\in\mathbb{R}^{N\times 17\times a\times a},$ (1)

where to encode the Gaussian prior of keypoint heatmaps, the resulting pose-
guided heatmaps $\mathbb{H}$ is reweighed by a Gaussian kernel
$\mathcal{G}_{a\times a}(0,\sigma=\frac{a-1}{2\times 3})$ (e.g., $\sigma=16$
when $a=97$) in an element-wise way. By doing so, it means that the enlarged
mesh follows the $3\sigma$ principle.

Then, we apply the learned keypoint $11\times 11$ kernels $K_{n,i}$’s to
convolve the reweighed $a\times a$ heatmap $\bar{\mathbb{H}}_{n,i}$ in a pose
instance-sensitive and keypoint-specific way, leading to LOcal-GlObal
Contextual Adaptation,

$\bar{\mathbb{H}}\xRightarrow[\text{LOGO-CA}]{K_{N\times 17\times 11\times
11}}\tilde{\mathbb{H}}\in\mathbb{R}^{N\times 17\times a\times a},$ (2)

which represents the refined heatmaps for the $17$ human pose keypoints.

The Pose Estimation Output. With the local-global contextually adpated
heatmaps $\tilde{\mathbb{H}}_{N\times 17\times a\times a}$, we maintain the
top-2 locations for each keypoint within the $a\times a$ heatmap, and then
utilize a convex average of the top-2 locations as the final predicted offset
vectors (i.e. $(\Delta x^{\prime}_{i},\Delta y^{\prime}_{i})$’s in Fig. 3),
and of their confidence scores as the prediction score, with a predefined
weight $\lambda$ for the top-1 location ($0.75$ in our experiments). Together
with the predicted keypoints centers $\mathcal{C}_{N\times 3}$, the final
prediction score for each keypoint is the product between the convex average
confidence score and the center confidence score. We keep the keypoints whose
final scores are greater than $0$. We have,

$\\{\mathcal{C}_{N\times 3},\tilde{\mathbb{H}}\\}\xRightarrow[\text{Score
thresholding}]{\text{Output}}\\{\hat{L}^{n}_{I};n=1,\cdots N^{\prime}\\},$ (3)

where $N^{\prime}$ is the number of the final predicted pose instances in an
image $I$.

###  3.2 Loss Functions in Training

In the fully end-to-end training, we need to define loss functions for the
global heatmap $\mathcal{H}$, the refined local heatmap $\tilde{\mathbb{H}}$,
the offset field $\mathcal{O}$, and the keypoint kernels.

The Heatmap Loss. The widely adopted mean squared error (MSE) loss is used.
Denoted by $\mathcal{H}^{GT}_{18\times h\times w}$ the ground truth heatmaps
in which each keypoint (including the center) is modeled by a 2-D Gaussian
with dataset-provided mean and variance. Let $\mathbf{p}=(i,\mathbf{x})$ be
the index of the domain $D$ of dimensions ${18\times h\times w}$. For the
predicted heatmaps $\mathcal{H}_{18\times h\times w}$, the MSE loss is defined
by,

$\mathcal{L}_{\mathcal{H}}=1/|D|\cdot\sum_{\mathbf{p}\in
D}\|w(\mathbf{x})(\mathcal{H}(\mathbf{p})-\hat{\mathcal{H}}(\mathbf{p}))\|_{2}^{2},$
(4)

where $w(\mathbf{x})$ represents the weight for the foreground and the
background pixels. The foreground mask is provided by the dataset annotation.
In our experiment, we set $w(\mathbf{x})=1/0.1$ for a foreground / background
pixel respectively.

In defining the loss function $\mathcal{L}_{\tilde{\mathbb{H}}}$ for the
refined local heatmap $\tilde{\mathbb{H}}$ (Eqn. 2), the ground-truth heatmap
$\tilde{\mathbb{H}}^{GT}$ is generated on-the-fly based on the mesh
$\mathcal{M}^{G}_{N\times 17\times a\times a}$ (Eqn. 1) and the ground-truth
keypoints using a Gaussian model with mean being the displacement between the
current predicted keypoints and the ground-truth ones, and variance $\sigma$
(i.e., the standard deviation of the reweighing Gaussion prior model in Eqn.
1).

The Offset Field Loss. The widely adopted SmoothL1 loss [28] is used. Let
$\mathcal{O}^{GT}\in\mathbb{R}^{34\times h\times w}$ be the ground-truth
offset field, and $\mathcal{C}^{GT}$ be the non-empty set of ground-truth
keypoints centers. For the predicted offset field $\mathcal{O}$, we have,

$\mathcal{L}_{\mathcal{O}}(\mathbf{p})=\mathcal{A}(\mathbf{p})\ell_{1}^{\text{smooth}}\left(\mathcal{O}(\cdot,\mathbf{p}),\mathcal{O}^{GT}(\cdot,\mathbf{p});\beta\right),$
(5)

for each foreground pixel $\mathbf{p}\in\mathcal{C}^{GT}$, and

$\mathcal{L}_{\mathcal{O}}=\frac{1}{|\mathcal{C}^{GT}|}\cdot\sum_{\mathbf{p}\in\mathcal{C}^{GT}}\mathcal{L}_{\mathcal{O}}(\mathbf{p})$
(6)

where $\mathcal{A}(\mathbf{p})$ is the area of the person centered at the
pixel $\mathbf{p}$, and $\beta$ the cutting-off threshold (e.g., $\frac{1}{9}$
in our experiments).

The OKS Loss for the Keyoint Kernels. Consider a single predicted pose
instance, learning the keypoint kernels, $K_{17\times 11\times 11}$ is the key
to facilitate the local-global contextual adaptation. To that end, the figure
of merits of the KEMs, $\mathcal{M}_{17\times 11\times 11\times 2}$ needs to
directly reflect the task loss, i.e., the OKS loss. With respect to the
$N^{GT}$ ground-truth pose instances in an image, we can compute the
similarity score per keypoint candidate in the KEMs, and obtain the score
tensor $S_{17\times 11\times 11\times N^{GT}}$. The score tensor is further
clamped with a threshold $0.5$, i.e., $S_{17\times 11\times 11\times
N^{GT}}=\max(S_{17\times 11\times 11\times N^{GT}},0.5)$. A mean reduction is
applied to the first three dimensions of the clamped score tensor to compute
the matching score for each of the $N^{GT}$ pose instance. Then, the best
ground-truth pose instance indexed by $n^{*}$ is selected in terms of the
matching score, and its matching score is denoted by $s_{n^{*}}$. Based on the
selected ground-truth pose instance, we compute the per-keypoint similarity
score for the predicted pose instance at hand, denoted by $s_{k}$
($k\in[1,17]$). Then, the loss function fo the keypoint kernels are defined
by,

$\mathcal{L}_{K}=s_{n^{*}}\cdot\sum_{k,i,j}s_{k}\cdot|K_{k,i,j}-{S}_{k,i,j,n^{*}}|^{2}.$
(7)

The Total Loss is then defined by
$\mathcal{L}=\mathcal{L}_{\mathcal{H}}+\mathcal{L}_{\tilde{\mathbb{H}}}+\lambda\cdot(\mathcal{L}_{\mathcal{O}}+\mathcal{L}_{K})$,
where the trade-off parameter $\lambda$ is used to balance the different loss
items ($\lambda=0.01$ in our experiments).

##  4 Experiments

Table 1: Evaluation results on the COCO-val-2017 and COCO-testdev-2017
dataset. For HGG [13] and SimplePose [16], the multi-scale inference† is
applied on the testdev-2017 dataset. For DEKR [11] that uses an rescoring
network to get the final predictions, we report both the performance with and
without rescoring (which is the fair baseline for our LOGO-CAP). The numbers
of SPM [23] and HGG [13] are extracted from their papers.

|  |  | COCO-val-2017 | COCO-testdev-2017
---|---|---|---|---
| Method | Backbone | AP [%] | AP50 [%] | AP75 [%] | APM [%] | APL [%] | AP [%] | AP50 [%] | AP75 [%] | APM [%] | APL [%]
Top-Down | HRNet [32] | HRNet-W48 | 76.3 | 90.8 | 82.9 | 72.3 | 83.4 | 75.5 | 92.5 | 83.3 | 71.9 | 81.5
Lite-HRNet [39] | Lite-HRNet-30 | 70.4 | 88.7 | 77.7 | 76.2 | 92.8 | 69.7 | 90.7 | 77.5 | 66.9 | 75.0
Mask-RCNN [12] | ResNet-50-FPN | 64.2 | 86.6 | 69.7 | 58.7 | 73.0 | 63.1 | 87.3 | 68.7 | 57.8 | 71.4
Grouping | OpenPose [42] | VGG-19 | 61.0 | 84.9 | 67.5 | 56.3 | 69.3 | 61.8 | 84.9 | 67.5 | 57.1 | 68.2
PifPaf [15] | ResNet-152 | 67.4 | 86.9 | 73.8 | 63.1 | 74.1 | 66.7 | 87.8 | 73.6 | 62.4 | 72.9
PersonLab [24] | ResNet-152 | 66.5 | 86.2 | 71.9 | 62.3 | 73.2 | 66.5 | 88.0 | 72.6 | 62.4 | 72.3
AE [21, 5] | HrHRNet-W32 | 67.1 | 86.2 | 73.0 | 61.5 | 76.1 | 66.4 | 87.5 | 72.8 | 61.2 | 74.2
HrHRNet-W48 | 69.9 | 87.2 | 76.1 | 65.4 | 76.4 | 68.4 | 88.2 | 75.1 | 64.4 | 74.2
HGG [13] | Hourglass | 60.4 | 83.0 | 66.2 | $-$ | $-$ | 67.6† | 85.1† | 73.7† | 62.7† | 74.6†
SimplePose [16] | IMHN | 66.1 | 85.9 | 71.6 | 59.8 | 76.2 | 68.5† | 86.7† | 74.9† | 66.4† | 71.9†
Direct | SPM [23] | Hourglass | $-$ | $-$ | $-$ | $-$ | $-$ | 66.9 | 88.5 | 72.9 | 62.6 | 0.731
CenterNet [42] | Hourglass | 64.0 | 85.6 | 70.2 | 59.4 | 72.1 | 63.0 | 86.8 | 69.6 | 58.9 | 70.4
DEKR [11] (w. Rescoring) | HRNet-W32 | 68.0 | 86.7 | 74.5 | 62.1 | 77.7 | 67.3 | 87.9 | 74.1 | 61.5 | 76.1
HRNet-W48 | 71.0 | 88.3 | 77.4 | 66.7 | 78.5 | 70.0 | 89.4 | 77.3 | 65.7 | 76.9
DEKR [11] (w.o. Rescoring) | HRNet-W32 | 67.2 | 86.3 | 73.8 | 61.7 | 77.1 | 66.6 | 87.6 | 73.5 | 61.2 | 75.6
HRNet-W48 | 70.3 | 87.9 | 76.8 | 66.3 | 78.0 | 69.3 | 89.1 | 76.7 | 65.3 | 76.4
LOGO-CAP (Ours) | HRNet-W32 | 69.6 | 87.5 | 75.9 | 64.1 | 78.0 | 68.2 | 88.7 | 74.9 | 62.8 | 76.0
HRNet-W48 | 72.2 | 88.9 | 78.9 | 68.1 | 78.9 | 70.8 | 89.7 | 77.8 | 66.7 | 77.0

In this section, we present detailed experimental results and analyses of the
proposed LOGO-CAP. Our PyTorch source code will be released for
reproducibility.

Training and Testing Settings. We train two LOGO-CAP networks with the
ImageNet pretrained HRNet-W32 and HRNet-W48 [32] as the feature backbone
respectively on the COCO-train-2017 dataset [19]. Common training and testing
specifications are used in experiments, which are provided in the appendix.

###  4.1 Datasets and Evaluation Metrics

Two datasets are used: The COCO dataset [19] is the most popular testbed for
human pose estimation. It consists of 65k, 5k and 20k images with human pose
well-annotated in the training, validation and testing datasets respectively.
In all experiments, the proposed LOGO-CAP is trained using the 65k training
images. The OCHuman dataset [41] is one popular testing-only dataset for
evaluating human pose estimation under the occlusion scenarios. It consists of
a total number of 4713 images with 8110 detailed annotated human pose
instances using the COCO keypoint configuration. All the annotated 8110 human
pose instances have occlusions with the maxIOU$\geq 0.5$. Furthermore, $32\%$
instances are more challenging with the maxIOU$\geq 0.75$.

Figure 4: Qualitative results of our LOGO-CAP (HRNet-W32). All images were
picked thematically without considering our algorithms performance. The first
two rows display our approach on the COCO-val-2017 dataset and the last two
ones show our results on the OCHuman test dataset.

###  4.2 Results on the COCO Dataset

The proposed LOGO-CAP is compared with prior arts including the bottom-up
approaches of OpenPose [2], PifPaf [15], PersonLab [24], AE [21] and DEKR
[11], as well as the top-down approaches of Mask-RCNN [12], HRNet [32], and
the light-weight LiteHRNet [39]. As reported in Table 1, the proposed LOGO-CAP
outperforms all the bottom-up approaches and the efficiency-toward top-down
approaches (_i.e_., Lite-HRNet [39] and Mask-RCNN [12]) on both validation and
test-dev datasets.

Fig. 4 shows some qualitative examples of human pose estimation by the
proposed LOGO-CAP.

In comparisons to the best-performing grouping approach, AE [21] with a larger
backbone HrHRNet-W48 [5], our LOGO-CAP obtains competitive performance with a
smaller HRNet-32 backbone, and improves the AP score with HRNet-W48 backbone
on the validation and test-dev datasets by 2.3 and 2.5 points, respectively.
For the fully differentiable grouping approach HGG [13], our LOGO-CAP achieves
better performance by a significantly large margin, more than 9.2 points on
the validation set under the single-scale testing. Although the performance of
HGG is improved by the multi-scale testing on the test-dev set, the
performance of our LOGO-CAP is still significantly better without using the
multi-scale testing scheme.

In comparisons to the direct regression based approaches, our LOGO-CAP obtains
the _best results_ without incurring either the matching scheme used in
CenterNet [42] or the additional rescoring network used in DEKR [11]. When we
disable the rescoring network for DEKR [11] for fair comparisons, our LOGO-CAP
significantly improves the AP on the validation and testdev datasets by 2.4
points and 1.6 points respectively when HRNet-W32 is used as backbone. The
larger backbone is beneficial for both DEKR and our method, which further
improves the AP score of our LOGO-CAP to 72.2 and 70.8 on the validation and
test-dev dataset respectively, outperforming DEKR by 1.9 and 1.5 respectively.

In comparisons to the top-down approaches, our LOGO-CAP outperforms the end-
to-end Mask-RCNN [12] by 7.7 AP on the test-dev-2017 dataset. For the
efficiency-toward Lite-HRNet, we improves the AP on the test-dev split from
69.7 to 70.8. Although the performance of heavy top-down HRNet [32] is better
than our LOGO-CAP, it should be noted that the headnet of our method is only
an one-layer convolution network on the heatmap space, instead of leveraging a
very deep model in the feature space of the cropped large-size images.

###  4.3 Results on the OCHuman Dataset

Table 2 shows that our LOGO-CAP achieves the best AP performance on both the
validation and testing datasets by significant margins of 2.4 and 2.2 points
in comparing with the bottom-up approaches. For the top-down approaches,
although they obtain strong AP scores on the validation split, there exists a
large performance gap between the validation and testing sets. In comparisons
to DEKR [11] (with the rescoring network), our LOGO-CAP improves the
performance from 37.9 to 39.0 and from 36.5 to 38.1 on the validation and
testing splits with the same backbone HRNet-W32, respectively. The similar
improvement is observed when the HRNet-W48 backbone is used, outperforming
both bottom-up and top-down approaches.

Table 2: Results on the OCHuman datasets [41].

| Methods | Backbone |  Val. AP [%] |  Test AP [%]
---|---|---|---|---
Top-down | RMPE [8] | Hourglass | 38.8 | 30.7
SBL [37] | ResNet-50 | 37.8 | 30.4
SBL [37] | ResNet-152 | 41.0 | 33.3
Bottom-up | AE [21] | Hourglass | 32.1 | 29.5
HGG [21] | Hourglass | 35.6 | 34.8
DEKR [11] | HRNet-W32 | 37.9 | 36.5
HRNet-W48 | 38.8 | 38.2
LOGO-CAP (Ours) | HRNet-W32 | 39.0 | 38.1
HRNet-W48 | 41.2 | 40.4

###  4.4 Inference Speed

In comparing the inference speed, we test all the models on a single TITAN RTX
GPU for its popularity in practice. The average inference speed, FPS (frames
per second), over the 5000 images in COCO-val-2017 is used for the comparison.
For DEKR [11], we re-implement their inference code with better speed obtained
for fair comparisons at the algorithm level. For methods that have post-
processing schema on CPU, only one thread is used. As shown in Table 3, our
LOGO-CAP runs significantly faster than PifPaf [15] and AE [21]. The CenterNet
[42] runs slower than DEKR and our LOCO-CAP as it requires a post-processing
scheme to match the predicted offsets to the keypoints obtained from heatmaps.
Comparing with DEKR, the speed improvement of our LOGO-CAP is from the
lightweight design of head modules since the same backbones are used. For the
comparisons in Table 2, we run the models with different resolutions of
testing images.

Table 3: The single image inference speed comparison for bottom-up human pose
estimation approaches.

Method | AP [%] | Backbone |  Time $\downarrow$ [ms] | FPS $\uparrow$
---|---|---|---|---
PifPaf [15] | 67.4 | ResNet-152 | 213 | 4.68
AE [21, 5] | 67.1 | HrHRNet-W32 | 560 | 1.78
CenterNet [42] | 64.0 | Hourglass | 147 | 6.80
DEKR [11] | 68.0 | HRNet-W32 | 63 | 15.8
DEKR [11] | 71.0 | HRNet-W48 | 139 | 7.21
LOGO-CAP | 69.6 | HRNet-W32 | 48 | 20.7
LOGO-CAP | 72.2 | HRNet-W48 | 112 | 8.95

Furthermore, we analyze the inference time for different number of persons in
the input image. As shown in Table 4, the main bottleneck of the inference
time is the backbone. For the Local-Global Contextual Adaptation module, it
only takes about 10 ms on average.

Table 4: The breakdown of inference time of the proposed LOGO-CAP method. For
each model, we seperately report the averaged inference time across 5000
images, the averaged inference time in the images that detect only one person,
the avaraged inference time in the images that have 30 persons.

LOGO-CAP |  $\\#$Persons | Backbone | Local KEMs | Local KAMs | Global KAMs
---|---|---|---|---|---
W32 | - | 38.6 ms | 3.05 ms | 2.49 ms | 2.85 ms
1 | 2.39 ms | 1.14 ms | 1.12 ms
30 | 3.69 ms | 3.49 ms | 5.87 ms
W48 | - | 99.9 ms | 4.18 ms | 3.00 ms | 3.34 ms
1 | 3.19 ms | 1.10 ms | 1.07 ms
30 | 2.97 ms | 3.59 ms | 5.97 ms

###  4.5 Ablation Studies on COCO Validation

In this section, we run a series of experiments to verify the effectiveness of
the design for our proposed Local-Global Contextual Adaptation module. We use
HRNet-W32 as our backbone for all ablation studies.

Table 5: Ablation studies on the three components of the LOGO-CAP: the OKS
loss, the Gaussian reweighing method for heatmaps and the Attentive
Normalization.

| OKS Loss | Reweigh | AttNorm | AP | AP50 | AP75 | APM | APL
---|---|---|---|---|---|---|---|---
baseline | - | - | - | 60.0 | 84.4 | 66.4 | 54.0 | 71.1
(a) | ✓ | - | - | 66.1 | 86.7 | 72.7 | 60.0 | 75.6
(b) | - | ✓ | - | 67.6 | 87.0 | 74.3 | 62.1 | 76.7
(c) | ✓ | ✓ | - | 69.0 | 87.0 | 75.2 | 63.4 | 77.5
(d) | ✓ | - | ✓ | 65.8 | 86.8 | 72.3 | 59.3 | 75.4
(e) | - | ✓ | ✓ | 67.5 | 86.6 | 74.1 | 62.2 | 76.7
(f) | ✓ | ✓ | ✓ | 69.6 | 87.5 | 75.9 | 64.1 | 78.0

Designs for Contextual Adaptation. We train 6 models to study the
effectiveness in Table 5 for the used components: (1) OKS Loss for learning
local KAMs, (2) the Gaussian Reweighing scheme and (3) the Attentive Norm for
convolutional message passing. Compared with the center-offset baseline, the
model (a) trained with OKS loss for the local KAMs estimation obtains a large
improvement of AP by 6.1 points, justifying that the potential of using local
KAMs for better pose estimation. For (b), we set the factor of OKS Loss to $0$
and train the model by using the end-to-end reweighing scheme. In this
setting, the local KAMs are learned only under the supervision of the
contextual adaptation. Similar to model (a), a large improvement is also
obtained compared against the baseline. Then, we train the model (c) that uses
both OKS loss and the reweighing scheme while replacing the Attentive Norm
[18] to BatchNorm in the convolutional message passing module. It is shown
that the OKS loss and the reweighing scheme collaborate very well with further
improvement obtained. For the effectiveness of Attentive Norm, it is shown in
Table 5 (c-f) that its feature recalibration mechanism requires different
information sources in human pose estimation. By enabling all the components,
our LOGO-CAP-W32 finally obtains an AP of 69.6 on the COCO-val-2017 dataset.

Table 6: Ablation study of the different size of the local KEMs.

size of local KEM |  Message Passing | AP | AP50 | AP75 | APM | APL | FPS
---|---|---|---|---|---|---|---
$7\times 7$ | 3+1 | 68.4 | 86.6 | 74.9 | 63.4 | 76.6 | 21.8
$11\times 11$ | 1 | 68.8 | 86.9 | 74.9 | 63.1 | 77.5 | 22.5
$11\times 11$ | 3+1 | 69.6 | 87.5 | 75.9 | 64.1 | 78.0 | 20.7
$15\times 15$ | 3+1 | 69.3 | 87.1 | 75.2 | 63.2 | 78.3 | 16.5
$19\times 19$ | 3+1 | 69.0 | 87.1 | 75.2 | 62.8 | 78.2 | 13.2

Size of the Local KEMs and the Design of Convolutional Message Passing. We
perform an ablation study with the results shown in Table 6, which confirms
that the kernel size of $11\times 11$ obtains the best performance. One
possible explanation is that smaller kernel sizes fail to compensate the
uncertainty of the initial keypoint estimation results, while larger kernel
sizes may introduce more nuisances factors that affect the performance, such
as the “collision” between different local KEMs of different keypoints from
either the same person or adjacent different persons. Third, for different
designs of the CMP module, we perform the ablation study for the different
implementations. In detail, we replace the original architecture that is
consist of 3 Conv+Norm+ReLU layers and 1 output Conv layer (denoted by 3+1 in
Table 6) to a simpler architecture that only use 1 output Conv layer (denoted
by 1) for dimension reduction. It is shown that the 3+1 architecture performs
better than the one only using the output Conv layer.

Table 7: Ablation study of using different priors in LOGO-CAP.

Type of the Prior | AP | AP50 | AP75 | APM | APL
---|---|---|---|---|---
KEMs only | 59.4 | 80.8 | 62.8 | 50.9 | 71.6
KAMs only | 65.7 | 86.0 | 72.3 | 60.6 | 74.0
LOGO-CAP (KEMs + KAMs) | 69.6 | 87.5 | 75.9 | 64.1 | 78.0

Different Type of the Priors for Contextual Adaptation. As our contextual
adaptation takes both the global KEMs and KAMs as priors for final pose
predictions, we quantitatively compare the possible designs for the contextual
adaptation. In Table 7, we compare the performance on the COCO-val-2017
dataset by using either global KEMs or the learned global KAMs to the one that
use two sources for prediction. On one hand, since the global KEMs are
actually the standard Gaussian around each initial keypoint, it cannot provide
more information for refinement. On another hand, when we enforce the use of
local KAMs for adaptation with only global KEMs, the uncertainty from local
KEMs will affect the results. That is the reason why only using global KEMs is
worse than using global KAMs. When using both global KEMs and KAMs, our
approach obtains the best performance.

###  4.6 Potentials and Limitations

Consider the generic applicability of the center-offset formulation to many
computer vision tasks as demonstrated in [42], we hypothesize that the
proposed LOGO-CAP has a great potential to remedy the lack of sufficient
accuracy using the vanilla center-offset method in those tasks. We also notice
that the minimally-simple design in learning the contextual adaptation for
refinement can be relaxed for different accuracy-speed trade-offs in practice.
For example, for the convolutional message passing module, an alternative
method could be the Transformer model [36], which potentially will further
improve the performance at the expense of inference speed. We leave these for
future work.

##  5 Conclusion

This paper presents a method of learning LOcal-GlObal Contextual Adaptation
for Pose estimation in a bottom-up fashion, dubbed as LOGO-CAP. The proposed
LOGO-CAP is built on the conceptually simple center-offset paradigm. The key
idea of our LOG-CAP is to lift the center-offset predicted keypoints to
keypoint expansion maps (KEMs) to address the inaccuracy of the initial
keypoints. Two types of KEMs are introduced: Local KEMs are used to learn
keypoint attraction maps (KAMs) via a convolutional message passing module
that accounts for the structural information of human pose. Global KEMs are
used to learn local-global contextual adaptation which convolves global KEMs
using the KAMs as kernels. The refined global KEMs are used in computing the
final human pose estimation. The proposed LOGO-CAP obtains state-of-the-art
performance in COCO val-2017 and test-dev 2017 datasets for bottom-up human
pose estimation. It also achieves state-of-the-art transferability performance
in the OCHuman dataset with the COCO trained models.

##  Appendix A Experimental Settings

Our PyTorch source code will be released. We briefly present the details of
training and testing as follows.

Training. We train two LOGO-CAP networks with the ImageNet pretrained
HRNet-W32 and HRNet-W48 [32] as the feature backbone respectively on the COCO-
train-2017 dataset [19]. Common training specifications are used for
simplicity in experiments. The Adam optimizer [14] is used with default
coefficients $\beta_{1}=0.9$ and $\beta_{2}=0.999$. For both the backbones,
the total number of epochs is set to $140$ and the batch size is set to $12$
images per GPU card. The same learning rate schedule is used for both models.
The learning rate is initially set to $0.001$ and then decayed to $10^{-4}$
and $10^{-5}$ at the 90-th and 120-th epoch respectively. We use $4$ and $8$
V100 GPUs to accelerate the training for the two LOGO-CAP models with
HRNet-W32 and HRNet-W48 respectively. The resolution of training images is
$512\times 512$ and $640\times 640$ for the two models respectively. Following
the widely adopted experimental settings in [40], the data augmentations in
training include (1) random rotation with the rotation degree from
$-30^{\circ}$ to $30^{\circ}$, (2) random scaling with the factor in the range
of $[0.75,1.5]$, (3) random translation in the range
$[-40\text{pix},40\text{pix}]$ along both $x$ and $y$ directions, and (4)
random horizontal flipping with the probability of $0.5$.

Similarly, for different hyperparamters such as the trade-off parameter
$\lambda$ in the total loss, we did not run computationally expensive
hyperparameter optimization for simplicity.

Testing. We focus on the single-scale testing protocol in the COCO keypoint
benchmark for the sake of efficient human pose estimation. In the testing
phase, the short side of input images is resized to a specific length (_e.g_.,
384, 512, or 640 pixels) and keep unchanged the aspect ratio between the
height and the width. As commonly adopted in many bottom-up pose estimation
approaches (_e.g_., AE [21], HrHRNet [5], DEKR [11]), the flip testing is used
as our default setting for the fair comparison. In the implementation, we feed
the stacked tensor with an input image and a horizontally-flipped one together
to get the global heatmaps and the offset fields. The flipped outputs are then
averaged (according to the flip index) to get the final global heatmaps and
the offset fields. For the computation of local heatmaps and the local-global
adaptation, only the non-flipped outputs are used for the final predictions.

##  Appendix B The Empirical Upper Bound

We elaborate on the details of computing the empirical upper bound of
performance for a vanilla center-offset pose estimation method. The detailed
results are reported in Tab. 8.

Table 8: The performance of a vanilla center-offset regression approach, its empirical upper bound, and the performance of our proposed LOGO-CAP using HRNet-W32 [32] as the feature backbone. See text for detail. | Baseline | Emp. Bound | LOGO-CAP
---|---|---|---
AP | 60.1 | 88.9 | 70.0
AP50 | 85.2 | 93.1 | 88.2
AP75 | 66.7 | 90.6 | 76.4
APM | 53.7 | 87.7 | 64.4
APL | 71.5 | 90.2 | 78.4

Network Architecture. The vanilla center-offset regression baseline uses the
ImageNet pretrained HRNet-W32 [32] as the backbone, and the same modules as in
our LOGO-CAP+HRNet-W32 for the center heatmap regression and the offset vector
regression. We present the details of computing keypoint expansion maps (KEMs)
that are used in calculating the empirical uppper bound as follows.

Computation of Keypoint Expansion Maps. Denoted by
$\mathcal{P}\in\mathbb{R}^{N\times 17\times 2}$ the initial pose parameters
(i.e., the 2-D locations for the $17$ keypoints of the $N$ pose instances)
estimated by the vanilla center-offset method, we expand each of the estimated
keypoints with a local $11\times 11$ mesh grid, that is to lift a keypoint to
a 2-D mesh to counter the estimation uncertainty. We use the COCO benchmark
provided keypoint sigmas to scale the unit length of the meshgrid for
different types (_e.g_., nose, eyes, hips) of keypoints. After getting the
expanded keypoint meshes $\mathcal{M}$ of the initial poses, we compute their
keypoint similarities $\mathcal{S}\in\mathbb{R}^{N\times 11\times 11\times
K\times 17}$ between the groundtruth keypoints
$\mathcal{G}\in\mathbb{R}^{K\times 17\times 3}$ and the keypoint expansion
maps. By applying the sum reduction on the similarity tensor $\mathcal{S}$
along the $2$-nd, $3$-rd and the last axes, we have known the optimal
correspondence (including the low-quality matches) for each center anchor,
denoted by $\mathcal{S}_{N\times 11\times 11\times 17}$. Then, the pose with
the maximal similarity in the $11\times 11$ local window for each center
anchor are used as the best one to compute the empirical upper bound on the
fully-annotated COCO-val-2017 dataset.

##  Appendix C More Qualitative Results

Results on the COCO-val-2017 and the OCHuman Datasets. Fig. 5 shows examples
of pose estimation in the two datasets by the proposed LOGO-CAP with the
HRNet-W32 backbone. Our proposed LOCO-CAP is able to handle large structural
and appearance variations in human pose estimation.

Figure 5: Qualitative results of our LOGO-CAP (HRNet-W32). All images were
picked thematically without considering our algorithms performance. The first
two rows display our approach on the COCO-val-2017 dataset and the last two
ones show our results on the OCHuman test dataset.

Fast pose estimation for video frames. To justify the potential of our
proposed approach in practical applications, we run our LOGO-CAP (W32 model)
on two videos that have the resolution of $1280\times 720$ from YouTube. We
follow our testing protocol to resize the short side of the video frames to
$512$ pixels and keep their original aspect ratios for inference. Without
using any pose tracking techniques, our LOGO-CAP achieves fast and accurate
human pose estimation. Please click the following anonymous links for the demo
videos (with background musics):

  * -

https://bit.ly/3cFcJ75 (video credit: https://youtu.be/2DiQUX11YaY)

  * -

https://bit.ly/30RkyEg (video credit: https://youtu.be/kTvzU1sGSyA)

In these two demo videos, the instantaneous FPS for each video frame is marked
in the left corner of the video.

## References

  * [1] Mykhaylo Andriluka, Stefan Roth, and Bernt Schiele.  Pictorial structures revisited: People detection and articulated pose estimation.  In 2009 IEEE conference on computer vision and pattern recognition, pages 1014–1021. IEEE, 2009.
  * [2] Zhe Cao, Gines Hidalgo Martinez, Tomas Simon, Shih-En Wei, and Yaser A. Sheikh.  Openpose: Realtime multi-person 2d pose estimation using part affinity fields.  IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2019.
  * [3] Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh.  Realtime multi-person 2d pose estimation using part affinity fields.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1302–1310, 2017.
  * [4] Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun.  Cascaded pyramid network for multi-person pose estimation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7103–7112, 2018.
  * [5] Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S. Huang, and Lei Zhang.  Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5385–5394. IEEE, 2020.
  * [6] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian.  Centernet: Keypoint triplets for object detection.  In IEEE/CVF International Conference on Computer Vision (ICCV), pages 6568–6577, 2019.
  * [7] Haoshu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu.  RMPE: regional multi-person pose estimation.  In IEEE International Conference on Computer Vision (ICCV), pages 2353–2362, 2017.
  * [8] Haoshu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu.  RMPE: regional multi-person pose estimation.  In IEEE International Conference on Computer Vision (ICCV), pages 2353–2362, 2017.
  * [9] Pedro F Felzenszwalb and Daniel P Huttenlocher.  Pictorial structures for object recognition.  International journal of computer vision, 61(1):55–79, 2005.
  * [10] Martin A Fischler and Robert A Elschlager.  The representation and matching of pictorial structures.  IEEE Transactions on computers, 100(1):67–92, 1973.
  * [11] Zigang Geng, Ke Sun, Bin Xiao, Zhaoxiang Zhang, and Jingdong Wang.  Bottom-up human pose estimation via disentangled keypoint regression.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  * [12] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick.  Mask R-CNN.  In IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017.
  * [13] Sheng Jin, Wentao Liu, Enze Xie, Wenhai Wang, Chen Qian, Wanli Ouyang, and Ping Luo.  Differentiable hierarchical graph grouping for multi-person pose estimation.  In European Conference on Computer Vision (ECCV), volume 12352, pages 718–734, 2020.
  * [14] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In Yoshua Bengio and Yann LeCun, editors, International Conference on Learning Representations (ICLR), 2015.
  * [15] Sven Kreiss, Lorenzo Bertoni, and Alexandre Alahi.  Pifpaf: Composite fields for human pose estimation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11977–11986, 2019.
  * [16] Jia Li, Wen Su, and Zengfu Wang.  Simple pose: Rethinking and improving a bottom-up approach for multi-person pose estimation.  In AAAI Conference on Artificial Intelligence (AAAI), pages 11354–11361, 2020.
  * [17] Ke Li, Shijie Wang, Xiang Zhang, Yifan Xu, Weijian Xu, and Zhuowen Tu.  Pose recognition with cascade transformers.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1944–1953, 2021.
  * [18] Xilai Li, Wei Sun, and Tianfu Wu.  Attentive normalization.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, European Conference on Computer Vision (ECCV), volume 12362, pages 70–87, 2020.
  * [19] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft COCO: common objects in context.  In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, European Conference on Computer Vision (ECCV), volume 8693, pages 740–755, 2014.
  * [20] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.  Shufflenet V2: practical guidelines for efficient CNN architecture design.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIV, volume 11218 of Lecture Notes in Computer Science, pages 122–138. Springer, 2018.
  * [21] Alejandro Newell, Zhiao Huang, and Jia Deng.  Associative embedding: End-to-end learning for joint detection and grouping.  In Advances in Neural Information Processing Systems 30 (NeurIPS), pages 2277–2287, 2017.
  * [22] Alejandro Newell, Kaiyu Yang, and Jia Deng.  Stacked hourglass networks for human pose estimation.  In European Conference on Computer Vision (ECCV), pages 483–499, 2016.
  * [23] Xuecheng Nie, Jiashi Feng, Jianfeng Zhang, and Shuicheng Yan.  Single-stage multi-person pose machines.  In IEEE/CVF International Conference on Computer Vision (ICCV), pages 6950–6959, 2019.
  * [24] George Papandreou, Tyler Zhu, Liang-Chieh Chen, Spyros Gidaris, Jonathan Tompson, and Kevin Murphy.  Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model.  In European Conference on Computer Vision (ECCV), pages 282–299, 2018.
  * [25] George Papandreou, Tyler Zhu, Nori Kanazawa, Alexander Toshev, Jonathan Tompson, Chris Bregler, and Kevin Murphy.  Towards accurate multi-person pose estimation in the wild.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3711–3719, 2017.
  * [26] Leonid Pishchulin, Mykhaylo Andriluka, Peter Gehler, and Bernt Schiele.  Poselet conditioned pictorial structures.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 588–595, 2013.
  * [27] Deva Ramanan, David A Forsyth, and Andrew Zisserman.  Strike a pose: Tracking people by finding stylized poses.  In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 1, pages 271–278. IEEE, 2005.
  * [28] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun.  Faster R-CNN: towards real-time object detection with region proposal networks.  IEEE Trans. Pattern Anal. Mach. Intell., 39(6):1137–1149, 2017\.
  * [29] Brandon Rothrock, Seyoung Park, and Song-Chun Zhu.  Integrating grammar and segmentation for human pose estimation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3214–3221, 2013.
  * [30] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510–4520, 2018.
  * [31] Ke Sun, Zigang Geng, Depu Meng, Bin Xiao, Dong Liu, Zhaoxiang Zhang, and Jingdong Wang.  Bottom-up human pose estimation by ranking heatmap-guided adaptive keypoint estimates.  CoRR, abs/2006.15480, 2020.
  * [32] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang.  Deep high-resolution representation learning for human pose estimation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5693–5703, 2019.
  * [33] Zhi Tian, Hao Chen, and Chunhua Shen.  Directpose: Direct end-to-end multi-person pose estimation.  CoRR, abs/1911.07451, 2019.
  * [34] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  FCOS: fully convolutional one-stage object detection.  In IEEE/CVF International Conference on Computer Vision (ICCV), pages 9626–9635, 2019.
  * [35] Ali Varamesh and Tinne Tuytelaars.  Mixture dense regression for object detection and human pose estimation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13083–13092, 2020.
  * [36] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  arXiv preprint arXiv:1706.03762, 2017.
  * [37] Bin Xiao, Haiping Wu, and Yichen Wei.  Simple Baselines for Human Pose Estimation and Tracking.  Computer Vision and Pattern Recognition, 2018.
  * [38] Yi Yang and Deva Ramanan.  Articulated human detection with flexible mixtures of parts.  IEEE transactions on pattern analysis and machine intelligence, 35(12):2878–2890, 2012.
  * [39] Changqian Yu, Bin Xiao, Changxin Gao, Lu Yuan, Lei Zhang, Nong Sang, and Jingdong Wang.  Lite-hrnet: A lightweight high-resolution network.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 10440–10450, 2021.
  * [40] Feng Zhang, Xiatian Zhu, Hanbin Dai, Mao Ye, and Ce Zhu.  Distribution-aware coordinate representation for human pose estimation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7091–7100, 2020.
  * [41] Song-Hai Zhang, Ruilong Li, Xin Dong, Paul L. Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, and Shi-Min Hu.  Pose2seg: Detection free human instance segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 889–898, 2019.
  * [42] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl.  Objects as points.  CoRR, abs/1904.07850, 2019.