# On profinite polyadic groups

M. Shahryari M. Shahryari
Department of Mathematics, College of Sciences, Sultan Qaboos University,
Muscat, Oman  and  M. Rostami M. Rostami
Department of Pure Mathematics, Faculty of Mathematical Sciences, University
of Tabriz, Tabriz, Iran<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

We study the structure of profinite polyadic groups and we prove that a
polyadic topological group $(G,f)$ is profinite, if and only if, it is
compact, Hausdorff, totally disconnected. More generally, for a pseudo-variety
(or a formation) of finite groups $\mathfrak{X}$, we define the class of
$\mathfrak{X}$-polyadic groups, and we show that a polyadic group $(G,f)$ is
pro-$\mathfrak{X}$, if and only if, it is compact, Hausdorff, totally
disconnected and for every open congruence $R$, the quotient $(G/R,f_{R})$ is
$\mathfrak{X}$-polyadic.

MSC(2010): 20N15
Keywords: Polyadic groups; $n$-ary groups; Profinite groups and polyadic
groups; Post’s cover and retract of a polyadic group

##  1\. Introduction

In this article, we study the structure of profinite polyadic groups: polyadic
groups which are the inverse limit of a system of finite polyadic groups. A
polyadic group is a natural generalization of the concept of group to the case
where the binary operation of group replaced with an $n$-ary associative
operation, one variable linear equations in which have unique solutions. So,
in this article, polyadic group means an $n$-ary group for a fixed natural
number $n\geq 2$. These interesting algebraic objects are introduced by Kasner
and Dörnte ([7] and [2]) and studied extensively by Emil Post during the first
decades of the last century, [12]. During decades, many articles have been
published on the structure of polyadic groups. Already, homomorphisms and
automorphisms of polyadic groups are studied in [9]. A characterization of the
simple polyadic groups is obtained by them in [10]. Also, the representation
theory of polyadic groups is studied in [5] and the complex characters of
finite polyadic groups are also investigated in [13]. The structure of free
polyadic groups is determined in [1], [11], and [8].

It is easy to define topological polyadic groups, and so, one can ask which
topological polyadic groups are profinite. In this paper, we study this
problem and as the main result, we prove that a polyadic topological group
$(G,f)$ is profinite, if and only if, it is compact, Hausdorff, totally
disconnected. More generally, for a pseudo-variety (formation) of finite
groups $\mathfrak{X}$, we define the class of $\mathfrak{X}$-polyadic groups,
and we show that a polyadic group $(G,f)$ is pro-$\mathfrak{X}$, if and only
if, it is compact, Hausdorff, totally disconnected and for every open
congruence $R$, the quotient $(G/R,f_{R})$ is $\mathfrak{X}$-polyadic.

##  2\. Polyadic groups

A polyadic group is a pair $(G,f)$ where $G$ is a non-empty set and
$f:G^{n}\to G$ is an $n$-ary operation, such that

i- the operation is associative, i.e.

$f(x_{1}^{i-1},f(x_{i}^{n+i-1}),x_{n+i}^{2n-1})=f(x_{1}^{j-1},f(x_{j}^{n+j-1}),x_{n+j}^{2n-1})$

for any $1\leq i<j\leq n$ and for all $x_{1},\ldots,x_{2n-1}\in G$, and

ii- for all $a_{1},\ldots,a_{n},b\in G$ and $1\leq i\leq n$, there exists a
unique element $x\in G$ such that

$f(a_{1}^{i-1},x,a_{i+1}^{n})=b.$

Note that, here we use the compact notation $x_{i}^{j}$ for every sequence

$x_{i},x_{i+1},\ldots,x_{j}$

of elements in $G$, and in the special case when all terms of this sequence
are equal to a fixed $x$, we denote it by $\stackrel{{\scriptstyle(t)}}{{x}}$,
where $t$ is the number of terms.

Clearly, the case $n=2$ is exactly the definition of ordinary groups. During
this article, we assume that $n$ is fixed. Note that an $n$-ary system $(G,f)$
of the form $f(x_{1}^{n})=x_{1}x_{2}\ldots x_{n}b$, where $(G,\cdot)$ is a
group and $b$ a fixed element belonging to the center of $(G,\cdot)$, is a
polyadic group, which is called $b$-derived from the group $(G,\cdot)$ and it
is denoted by $\mathrm{der}_{b}^{n}(G,\cdot)$. In the case when $b$ is the
identity of $(G,\cdot)$, we say that such a polyadic group is reduced to the
group $(G,\cdot)$ or derived from $(G,\cdot)$ and we use the notation
$\mathrm{der}^{n}(G,\cdot)$ for it. For every $n>2$, there are $n$-ary groups
which are not derived from any group. A polyadic group $(G,f)$ is derived from
some group if and only if, it contains an element $a$ (called an $n$-ary
identity) such that

$f(\stackrel{{\scriptstyle(i-1)}}{{a}},x,\stackrel{{\scriptstyle(n-i)}}{{a}})=x$

holds for all $x\in G$ and for all $i=1,\ldots,n$, see [3].

From the definition of an $n$-ary group $(G,f)$, we can directly see that for
every $x\in G$, there exists only one $y\in G$, satisfying the equation

$f(\stackrel{{\scriptstyle(n-1)}}{{x}},y)=x.$

This element is called skew to $x$ and it is denoted by $\overline{x}$. As
Dörnte [2] proved, the following identities hold for all $\,x,y\in G$, $2\leq
i\leq n$,

$f(\stackrel{{\scriptstyle(i-2)}}{{x}},\overline{x},\stackrel{{\scriptstyle(n-i)}}{{x}},y)=f(y,\stackrel{{\scriptstyle(n-i)}}{{x}},\overline{x},\stackrel{{\scriptstyle(i-2)}}{{x}})=y.$

These identities together with the associativity identities, axiomatize the
variety of polyadic groups in the algebraic language $(f,^{-})$.

Suppose $(G,f)$ is a polyadic group and $a\in G$ is a fixed element. Define a
binary operation

$x\bullet y=f(x,\stackrel{{\scriptstyle(n-2)}}{{a}},y).$

Then $(G,\bullet)$ is an ordinary group, called the retract of $(G,f)$ over
$a$. Such a retract will be denoted by $\mathrm{ret}_{a}(G,f)$. All retracts
of a polyadic group are isomorphic. The identity of the group $(G,\bullet)$ is
$\overline{a}$. One can verify that the inverse element to $x$ has the form

$y=f(\overline{a},\stackrel{{\scriptstyle(n-3)}}{{x}},\overline{x},\overline{a}).$

One of the most fundamental theorems of polyadic group is the following, now
known as Hosszú -Gloskin’s theorem. We will use it frequently in this article
and the reader can use [4], [6] and [14] for detailed discussions.

######  Theorem 2.1.

Let $(G,f)$ be a polyadic group. Then there exists an ordinary group
$(G,\bullet)$, an automorphism $\theta$ of $(G,\bullet)$ and an element $b\in
G$ such that

1\. $\theta(b)=b$,

2\. $\theta^{n-1}(x)=bxb^{-1}$, for every $x\in G$,

3\.
$f(x_{1}^{n})=x_{1}\theta(x_{2})\theta^{2}(x_{3})\cdots\theta^{n-1}(x_{n})b$,
for all $x_{1},\ldots,x_{n}\in G$.

According to this theorem, we use the notation
$\mathrm{der}_{\theta,b}(G,\bullet)$ for $(G,f)$ and we say that $(G,f)$ is
$(\theta,b)$-derived from the group $(G,\bullet)$.

There is one more important ordinary group associated to a polyadic group
which we call it the Post’s cover. This is the first fundamental theorem
concerning polyadic groups. The proof can be find in [12].

######  Theorem 2.2.

Let $(G,f)$ be a polyadic group. Then, there exists a unique group
$(G^{\ast},\circ)$ such that

1- $G$ is contained in $G^{\ast}$ as a coset of some normal subgroup $K$.

2- $K$ is isomorphic to a retract of $(G,f)$.

3- We have $G^{\ast}/K\cong\mathbb{Z}_{n-1}$.

4- Inside $G^{\ast}$, for all $x_{1},\ldots,x_{n}\in G$, we have
$f(x_{1}^{n})=x_{1}\circ x_{2}\circ\cdots\circ x_{n}$.

5- $G^{\ast}$ is generated by $G$.

The group $G^{\ast}$ is also universal in the class of all groups having
properties 1, 4. More precisely, if $\beta:(G,f)\to\mathrm{der}^{n}(H,\ast)$
is a polyadic homomorphism, then there exists a unique ordinary homomorphism
$h:G^{\ast}\to H$, such that $h_{|_{G}}=\beta$. This universal property
characterizes $G^{\ast}$ uniquely. The explicit construction of the Post’s
cover can be find in [13].

Finally, we have to mention that the structure of polyadic homomorphisms will
be needed in what follows. The reader can see [9] for details.

######  Theorem 2.3.

Suppose $(G,f)=\mathrm{der}_{\theta,b}(G,\cdot)$ and
$(H,h)=\mathrm{der}_{\eta,c}(H,\ast)$ are two polyadic groups. Let
$\psi:(G,f)\to(H,h)$ be a homomorphism. Then there exists $a\in H$ and an
ordinary homomorphism $\phi:(G,\cdot)\to(H,\ast)$, such that $\psi=R(a)\phi$,
where $R(a)$ denotes the map $x\mapsto x\ast a$. Further $a$ and $\phi$
satisfy the following conditions;

$h(\stackrel{{\scriptstyle(n)}}{{a}})=\phi(b)\ast a\ \ \ and\ \ \
\phi\theta=I_{a}\eta\phi,$

where, $I_{a}$ denotes the inner automorphism $x\mapsto a\ast x\ast a^{-1}$.
Conversely, if $a$ and $\phi$ satisfy the above two conditions, then
$\psi=R_{a}\phi$ is a homomorphism $(G,f)\to(H,h)$.

##  3\. Profinite polyadic groups

A profinite polyadic group is the inverse limit of an inverse system of finite
polyadic groups. More precisely, let $(I,\leq)$ be a directed set and suppose
$\\{(G_{i},f_{i}),\varphi_{ij},I\\}$ is an inverse system of finite polyadic
groups. This means that for every pair $(i,j)$ of elements of $I$ with $j\leq
i$, we are given a polyadic homomorphism

$\varphi_{ij}:(G_{i},f_{i})\to(G_{j},f_{j})$

such that the equality $\varphi_{jk}\varphi_{ij}=\varphi_{ik}$ holds for all
$k\leq j\leq i$. Now, assume that

$(G,f)=\varprojlim_{i}(G_{i},f_{i}).$

Then $(G,f)$ is called a profinite polyadic group. From now on, we consider
the pair $(G,f)$ which is the above mentioned inverse limit. A realization of
this pair can be given as follows: Let $\prod_{i}(G_{i},f_{i})$ be the direct
product of the family $\\{(G_{i},f)_{i}\\}_{i\in I}$. This is a polyadic group
with the $n$-ary operation

$(\prod
f_{i})((x_{i1}),(x_{i2}),\ldots,(x_{in}))=(f_{i}(x_{i1},x_{i2},\ldots,x_{in}))_{i\in
I}.$

Here of course, we denoted an arbitrary element of the direct product as
sequence $(a_{i})_{i\in I}$ or simply $(a_{i})$. Now, we have

$G=\\{(x_{i})_{i\in I}:\ \forall j\leq i\ \varphi_{ij}(x_{i})=x_{j}\\},$

and hence

$f((x_{i1}),(x_{i2}),\ldots,(x_{in}))=(f_{i}(x_{i1},x_{i2},\ldots,x_{in}))_{i\in
I}.$

Note that, as each $G_{i}$ is finite, being a closed subspace of the direct
product of a family of finite sets, $(G,f)$ is compact, Hausdorff, and totally
disconnected topological polyadic group, of course, if it has been shown that
$G$ is non-empty. Indeed, using standard topological arguments, we can prove
that $G\neq\emptyset$ as every $G_{i}$ is compact.

Recall that, according to Hosszú -Gloskin’s theorem, we have
$(G_{i},f_{i})=\mathrm{der}_{\theta_{i},b_{i}}(G_{i},\bullet_{i})$, for some
ordinary group $(G_{i},\bullet_{i})$, an element $b_{i}\in G_{i}$, and an
automorphism $\theta_{i}$, satisfying the conclusions of Theorem 2.1. We will
prove that in some sense, there exists a binary operation $\bullet$ on $G$
such that

$(G,\bullet)=\varprojlim_{i}(G_{i},\bullet_{i}),$

and hence $(G,\bullet)$ will be proved to be profinite. Consider the polyadic
homomorphism $\varphi_{ij}$. According to Theorem 2.3, there exist an element
$a_{ij}\in G_{j}$, and a group homomorphism
$\psi_{ij}:(G_{i},\bullet_{i})\to(G_{j},\bullet_{j})$, such that

$\varphi_{ij}=R(a_{ij})\psi_{ij}.$

Further, we have the following equalities:

$1.\ f_{j}(a_{ij},a_{ij},\ldots,a_{ij})=\psi_{ij}(b_{i})\bullet_{j}a_{ij}$,

$2.\ \psi_{ij}\theta_{i}=I(a_{ij}^{-1})\theta_{j}\psi_{ij}$.

For any triple of indices $k\leq j\leq i$, we have

$\varphi_{ij}=R(a_{ij})\psi_{ij},\ \varphi_{ik}=R(a_{ik})\psi_{ik},\
\varphi_{jk}=R(a_{jk})\psi_{jk},$

therefore

$a_{ij}=\varphi_{ij}(1),\ a_{ik}=\varphi_{ik}(1),\ a_{jk}=\varphi_{jk}(1).$

Note that in each equality, $1$ is the identity element of the corresponding
group. Since $\varphi_{ik}=\varphi_{jk}\varphi_{ij}$, so we have

$a_{ik}=\varphi_{jk}(a_{ij}).$

Now, let $Y_{i}$ be the set of all sequences $(x_{j})$ (in the direct product)
such that for any $j$ and $k$ $\leq i$, we have $\varphi_{jk}(x_{j})=x_{k}$.
This set is non-empty, because we can consider a sequence where
$x_{j}=a_{ij}$, for $j\leq i$, and for all other $j$, $x_{j}$ is arbitrary.
This sequence will be an element of $Y_{i}$. The set $Y_{i}$ is closed and if
$i\leq s$, then $Y_{s}\subseteq Y_{i}$. As the direct product is compact, and
the family $\\{Y_{i}\\}$ has finite intersection property, we have

$\bigcap_{i}Y_{i}\neq\emptyset,$

showing that $G$ is not empty.

Our first result, shows that the property of being profinite is inherited by
the retract and Post’s cover:

######  Proposition 3.1.

Let $(G,f)=\mathrm{der}_{\theta,b}(G,\bullet)$ be a profinite polyadic group.
Then the retract group $(G,\bullet)$ and the Post’s cover are also profinite.

###### Proof.

We know that $(G,\bullet)=\mathrm{ret}_{a}(G,f)$ for some $a$. As we have

$x\bullet y=f(x,\stackrel{{\scriptstyle(n-2)}}{{a}},y),$

and

$x^{-1}=f(\overline{a},\stackrel{{\scriptstyle(n-3)}}{{x}},\overline{x},\overline{a}),$

so, the group $(G,\bullet)$ is a topological group which is compact,
Hausdorff, and totally disconnected. This shows that the retract is profinite.
We know that there exists a normal subgroup $K$ of the Post’s cover which has
the index $n-1$ and $K$ is isomorphic to the retract $(G,\bullet)$. Hence, $K$
is profinite. Now, being a finite extension of a profinite group, $G^{\ast}$
is also profinite. ∎

The next result shows that, in some sense, the retract is the inverse limit of
the retracts of the polyadic groups $(G_{i},f_{i})$.

######  Proposition 3.2.

Let $(G,f)=\mathrm{der}_{\theta,b}(G,\bullet)$ be a polyadic group and
$(G,f)=\varprojlim_{i}(G_{i},f_{i})$. Then there exist elements $v_{i}\in
G_{i}$, such that

$(G,\bullet)=\varprojlim_{i}\mathrm{ret}_{v_{i}}(G_{i},f_{i}).$

###### Proof.

As $G\neq\emptyset$, we choose an arbitrary element $(v_{i})\in G$. We know
that all retracts of a polyadic group are isomorphic to each other. So, we
consider the retract

$(G_{i},\bullet_{i})=\mathrm{ret}_{v_{i}}(G_{i},f_{i}).$

By the construction of Sokolov (see [14]), we have

$\theta_{i}(x)=f_{i}(\overline{v}_{i},x,\stackrel{{\scriptstyle(n-2)}}{{v_{i}}}),$

for any $x$. Also we have

$b_{i}=f_{i}(\overline{v}_{i},\ldots,\overline{v}_{i}).$

Using this special form of the retract, we see that the maps $\varphi_{ij}$
are group homomorphisms as well, because

$\displaystyle\varphi_{ij}(x\bullet_{i}y)$ $\displaystyle=$
$\displaystyle\varphi_{ij}(f_{i}(x,\stackrel{{\scriptstyle(n-2)}}{{v_{i}}},y))$
$\displaystyle=$ $\displaystyle
f_{j}(\varphi_{ij}(x),\stackrel{{\scriptstyle(n-2)}}{{\varphi_{ij}(v_{i})}},\varphi_{ij}(y))$
$\displaystyle=$ $\displaystyle
f_{j}(\varphi_{ij}(x),\stackrel{{\scriptstyle(n-2)}}{{v_{j}}},\varphi_{ij}(y))$
$\displaystyle=$ $\displaystyle\varphi_{ij}(x)\bullet_{j}\varphi_{ij}(y).$

Note that, here we use the fact $\varphi_{ij}(v_{i})=v_{j}$ as we assumed that
$(v_{i})\in G$. This shows that the maps $\varphi_{ij}:G_{i}\to G_{j}$ are in
the same time, group homomorphisms and
$\\{(G_{i},\bullet_{i}),\varphi_{ij},I\\}$ is an inverse system of finite
groups. Obviously, $(G,\bullet)$ is the inverse limit of this system. ∎

Note that in some sense, the inverse of the above theorem is also true: if we
consider a profinite group $(G,\bullet)$ together with a continuous
automorphism $\theta$ and an element $b$ satisfying the requirement of 2.1,
then the polyadic group $\mathrm{der}_{\theta,b}(G,\bullet)$ will be
profinite. We will see the proof soon. One may ask also about the automorphism
$\theta$ in the above proof. The above construction shows that, for any
$(x_{i})\in G$, we have

$\theta((x_{i})_{i\in I})=(\theta_{i}(x_{i}))_{i\in I}.$

As a result, we see that the inverse limit commutes with $\mathrm{der}$:

######  Corollary 3.3.

Let $\\{(G_{i},f_{i}),\varphi_{ij},I\\}$ be an inverse system of finite
polyadic groups and for any $i$, suppose
$(G_{i},f_{i})=\mathrm{der}_{\theta_{i},b_{i}}(G_{i},\bullet_{i})$. Then

$\varprojlim_{i}\mathrm{der}_{\theta_{i},b_{i}}(G_{i},\bullet_{i})=\mathrm{der}_{\hat{\theta},\hat{b}}\varprojlim_{i}(G_{i},\bullet_{i}),$

where

$\hat{b}=(b_{i})_{i\in I},$

and

$\hat{\theta}((x_{i})_{i\in I})=(\theta_{i}(x_{i}))_{i\in I}.$

We are ready now, to give a characterization of the profinite polyadic groups.

######  Theorem 3.4.

A topological polyadic group $(G,f)$ is profinite, if and only if, it is
compact, Hausdorff, and totally disconnected.

###### Proof.

We already have seen that a profinite polyadic group is compact, Hausdorff and
totally disconnected. Now, assume that
$(G,f)=\mathrm{der}_{\theta,b}(G,\bullet)$ is a topological polyadic group
which is compact, Hausdorff, and totally disconnected. As we saw before, the
retract $(G,\bullet)$ is also a topological group with the same properties, so
it is a profinite group. This means that

$(G,\bullet)=\varprojlim\\{\frac{G}{K}:\ K\unlhd G,\ K=open\\}.$

Let

$I=\\{K\unlhd G:\ K=open,\ \theta(K)\subseteq K\\},\ J=\\{K\unlhd G:\
K=open\\}.$

We show that $I$ is a cofinal in the directed set $J$. In other words, we show
that for any $L\in J$, there exists a $K\in I$ such that $K\subseteq L$. So,
let $L\in J$ and consider

$K=\bigcap_{i=0}^{n-1}\theta^{i}(L).$

As $G$ is compact and $\theta$ is continuous, $K$ is an open normal subgroup
of $(G,\bullet)$. It is also $\theta$-invariant, since if $u\in K$, then

$\forall i\exists x_{i}\in L:\ u=\theta^{i}(x_{i}).$

Hence

$\theta(u)=\theta^{i+1}(x_{i})\in\theta^{i+1}(L),$

and since $\theta^{n-1}(L)\subseteq L$, so $\theta(u)\in K$. This proves that

$(G,\bullet)=\varprojlim_{K\in I}\frac{G}{K}.$

Now, for each $K\in I$, we can define an automorphism

$\theta_{K}:\frac{G}{K}\to\frac{G}{K},\ \theta_{K}(xK)=\theta(x)K.$

Note that we also have

$\theta_{K}(bK)=bK,\ \theta_{K}^{n-1}(xK)=(bK)(xK)(bK)^{-1},$

therefore, we can consider the finite polyadic group

$(\frac{G}{K},f_{K})=\mathrm{der}_{\theta_{K},bK}(\frac{G}{K},\bullet).$

Now, as we have

$(G,\bullet)=\varprojlim_{K\in I}\frac{G}{K},$

the mapping $x\mapsto(xK)_{K\in I}$ is an isomorphism, so every element in the
right hand side can be represented as a sequence $(xK)_{K\in I}$, for a unique
$x\in G$. This means that the automorphism

$\hat{\theta}((xK)_{K})=(\theta(x)K)_{K},$

is the same as $\theta$. Similarly, we have $\hat{b}=b$. Therefore, by the
previous corollary, we have

$\displaystyle(G,f)$ $\displaystyle=$
$\displaystyle\mathrm{der}_{\theta,b}(G,\bullet)$ $\displaystyle=$
$\displaystyle\mathrm{der}_{\theta,b}(\varprojlim_{K\in
I}(\frac{G}{K},\bullet))$ $\displaystyle=$ $\displaystyle\varprojlim_{K\in
I}\mathrm{der}_{\theta_{K},bK}(\frac{G}{K},\bullet),$

and this shows that $(G,f)$ is profinite. ∎

As a result, we have

######  Corollary 3.5.

A polyadic group $(G,f)=\mathrm{der}_{\theta,b}(G,\bullet)$ is profinite, if
and only if, $(G,\bullet)$ is profinite and $\theta$ is continuous.

Now, we consider a more general case, where a class $\mathfrak{Y}$ of finite
polyadic groups is given and discuss pro-$\mathfrak{Y}$ polyadic groups:
polyadic groups which are the inverse limits of inverse systems of elements of
$\mathfrak{Y}$. Let $\mathfrak{X}$ be an arbitrary class of groups. Define a
new class

$\mathrm{Pol}_{n}(\mathfrak{X})=\\{(G,f)=n-ary:\
\mathrm{ret}(G,f)\in\mathfrak{X}\\}.$

######  Proposition 3.6.

If the class $\mathfrak{X}$ is closed under each of the closure operators:
subgroup, direct product, quotient, or subdirect product, then the class
$\mathrm{Pol}_{n}(\mathfrak{X})$ is also closed under the similar operation.

###### Proof.

Let $\mathfrak{X}$ be closed under subgroup and
$(G,f)\in\mathrm{Pol}_{n}(\mathfrak{X})$. Let $(H,f)\leq(G,f)$. Then for any
$a\in H$, we have

$\mathrm{ret}_{a}(H,f)\leq\mathrm{ret}_{a}(G,f)\in\mathfrak{X},$

and so $\mathrm{ret}_{a}(H,f)\in\mathfrak{X}$, which shows that
$(H,f)\in\mathrm{Pol}_{n}(\mathfrak{X})$.

Now, assume that $\mathfrak{X}$ is closed under direct product and
$(G_{i},f_{i})\in\mathrm{Pol}_{n}(\mathfrak{X})$ be a family of polyadic
groups, where $i\in I$. For any arbitrary sequence
$(a_{i})_{i}\in\prod_{i}(G_{i},f_{i})$ we have

$\mathrm{ret}_{(a_{i})}(\prod_{i}(G_{i},f_{i}))=\prod_{i}\mathrm{ret}_{a_{i}}(G_{i},f_{i})\in\mathfrak{X},$

and this shows that $\prod_{i}(G_{i},f_{i})\in\mathrm{Pol}_{n}(\mathfrak{X})$.

Let $\mathfrak{X}$ be closed under quotient,
$(G,f)\in\mathrm{Pol}_{n}(\mathfrak{X})$, and $R$ be a congruence of $(G,f)$.
In the quotient polyadic group $(G/R,f_{R})$, we have

$f_{R}([x_{1}]_{R},\ldots,[x_{n}]_{R})=[f(x_{1},\ldots,x_{n})]_{R},$

where $[\ ]_{R}$ denotes the congruent class. Let $a\in G$ and define a map

$\lambda:\mathrm{ret}_{a}(G,f)\to\mathrm{ret}_{[a]}(\frac{G}{R},f_{R})$

by $\lambda(x)=[x]_{R}$. We have

$\displaystyle\lambda(x\bullet y)$ $\displaystyle=$ $\displaystyle[x\bullet
y]_{R}$ $\displaystyle=$
$\displaystyle[f(x,\stackrel{{\scriptstyle(n-2)}}{{a}},y)]_{R}$
$\displaystyle=$ $\displaystyle
f_{R}([x],\stackrel{{\scriptstyle(n-2)}}{{[a]}},[y])$ $\displaystyle=$
$\displaystyle[x]_{R}\bullet[y]_{R}.$

This shows that $\lambda$ is a group epimorphism and hence

$\mathrm{ret}_{[a]}(\frac{G}{R},f_{R})\cong\frac{\mathrm{ret}_{a}(G,f)}{\ker\lambda}\in\mathfrak{X}.$

As a result, $(G/R,f_{R})\in\mathrm{Pol}_{n}(\mathfrak{X})$. The case of
subdirect product can be proved by a similar argument. ∎

Therefore $\mathrm{Pol}_{n}(\mathfrak{X})$ will be a variety, if we begin with
a variety of groups $\mathfrak{X}$. It could be a good question if one ask
about the set of identities of this variety. We are not interested in such
problems in this work. Recall that a pseudo-variety of finite groups, is a
class of finite groups which is closed under subgroup, quotient and finite
direct product. Similarly, a formation of finite groups, is a class of finite
groups which is closed under quotient and finite subdirect products. Now, we
are ready to prove the next result. In what follows, a polyadic
$\mathfrak{X}$-group means a polyadic $\mathrm{Pol}_{n}(\mathfrak{X})$-group.
So, the name $\mathfrak{X}$ will be used both for the class of groups and its
corresponding class of polyadic groups.

######  Theorem 3.7.

Let $\mathfrak{X}$ be a pseudo-variety (formation) of finite groups. Let
$(G,f)$ be a pro-$\mathfrak{X}$ polyadic group and $R$ be an open congruence
of it. Then

$(\frac{G}{R},f_{R})\in\mathrm{Pol}_{n}(\mathfrak{X}).$

###### Proof.

Here we only consider the case where $\mathfrak{X}$ is a pseudo-variety.
Assume that $(G,f)=\mathrm{der}_{\theta,b}(G,\bullet)$. Suppose $R\subseteq
G\times G$ is an open congruence of the polyadic group $(G,f)$. In [10], it is
proved that in this case, the equivalence relation $R$ is a subgroup of the
ordinary group $G\times G$ (the letter $G$ here stands for the group
$(G,\bullet)$). So, we define a map

$\psi:\frac{G}{R}\to\frac{G\times G}{R}$

by $\psi([x]_{R})=(x,1)R$. This map is well-defined as if we suppose
$[x]_{R}=[y]_{R}$, then $(x,y)\in R$ and so $(x,y)R=R$. This means that

$(x,1)R=(1,y^{-1})R,$

and as in the quotient we have $(1,y^{-1})R=(y,1)R$, so the map is well-
defined. Also, it is injective, since if $(x,1)R=(y,1)R$, then $(x^{-1}y,1)\in
R$, so, we have also

$(y,x)=(x,x)(x^{-1}y,1)\in R.$

Note that

$\displaystyle\psi(f_{R}([x_{1}],\ldots,[x_{n}]))$ $\displaystyle=$
$\displaystyle\psi([f(x_{1},\ldots,x_{n})]_{R})$ $\displaystyle=$
$\displaystyle(f(x_{1},\ldots,x_{n}),1)R$ $\displaystyle=$
$\displaystyle(x_{1},1)(\theta(x_{2}),1)\cdots(\theta^{n-1}(x_{n}),1)(b,1)R$
$\displaystyle=$ $\displaystyle f_{\frac{G\times
G}{R}}((x_{1},1)R,\ldots,(x_{n},1)R).$

This shows that we indeed have a polyadic embedding

$\psi:(\frac{G}{R},f_{R})\to\mathrm{der}_{\bar{\theta},\bar{b}}(\frac{G\times
G}{R},\bullet).$

As $(G,f)$ is assumed to be a pro-$\mathfrak{X}$ polyadic group, $(G,\bullet)$
is a pro-$\mathfrak{X}$ group, and hence $G\times G$ is so. Now, $R\unlhd
G\times G$ is an open subgroup, so the quotient $G\times G/R$ belongs to
$\mathfrak{X}$. This means that

$\mathrm{der}_{\bar{\theta},\bar{b}}(\frac{G\times
G}{R},\bullet)\in\mathrm{Pol}_{n}(\mathfrak{X}),$

and hence $(G/R,f_{R})\in\mathrm{Pol}_{n}(\mathfrak{X})$. ∎

The converse of the above theorem is also true. For a proof, we need to
proceed as in the theorem 3.4.

######  Theorem 3.8.

Let $\mathfrak{X}$ be a pseudo-variety (formation) of finite groups. Let
$(G,f)$ be a topological polyadic group which is compact, Hausdorff, and
totally disconnected. Assume that for any open congruence $R$, the polyadic
group $(G/R,f_{R})$ belongs to $\mathrm{Pol}_{n}(\mathfrak{X})$. Then $(G,f)$
is pro-$\mathfrak{X}$.

## References

  * [1] Artamonov V., Free $n$-groups, Matematicheskie Zametki, 1970, 8, pp. 499-507.
  * [2] Dörnte W., Unterschungen über einen verallgemeinerten Gruppenbegriff, Math. Z., 1929, 29, pp. 1-19.
  * [3] Dudek W., Remarks on $n$-groups, Demonstratio Math., 1980, 13, pp. 65-181.
  * [4] Dudek W., Glazek K., Around the Hosszú-Gluskin Theorem for $n$-ary groups, Discrete Math., 2008, 308, pp. 4861-4876.
  * [5] Dudek W., Shahryari M., Representation theory of polyadic groups, Algebras and Representation Theory, 2012, 15, pp. 29-51.
  * [6] Hosszú M., On the explicit form of $n$-groups, Publ. Math., 1963, 10, pp. 88-92.
  * [7] Kasner E., An extension of the group concept, Bull. Amer. Math. Soc., 1904, 10, pp. 290-291.
  * [8] Khodabandeh H., On free polyadic groups, 2019, arXiv:1905-1182.
  * [9] Khodabandeh H., Shahryari M.,  On the representations and automorphisms of polyadic groups,  Communications in Algebra, 2012, 40, pp. 2199-2212.
  * [10] Khodabandeh H., Shahryari M.,  Simple polyadic groups,  Siberian Math. Journal, 2014, 55, pp. 734-744.
  * [11] Khodabandeh H., Shahryari M.,  Equations over polyadic groups,  Communication in Algebra, 2017, 45, pp. 1227-1238.
  * [12] Post E., Polyadic groups, Trans. Amer. Math. Soc., 1940, 48, pp. 208-350.
  * [13] Shahryari M., Representations of finite polyadic groups, Communications in Algebra, 2012, 40, pp. 1625-1631.
  * [14] Sokolov E., On the Gluskin-Hosszú theorem for Dornte $n$-groups, Mat. Issled., 1976, 39, pp. 187-189.
  * [15] Ribes L., Zalesskii P., Profinite groups, second edition, Springer, 2010.


# NCHO: Unsupervised Learning for
Neural 3D Composition of Humans and Objects

Taeksoo Kim1 Shunsuke Saito2 Hanbyul Joo1
1Seoul National University 2Meta Reality Labs
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
https://taeksuu.github.io/ncho

###### Abstract

Deep generative models have been recently extended to synthesizing 3D digital
humans. However, previous approaches treat clothed humans as a single chunk of
geometry without considering the compositionality of clothing and accessories.
As a result, individual items cannot be naturally composed into novel
identities, leading to limited expressiveness and controllability of
generative 3D avatars. While several methods attempt to address this by
leveraging synthetic data, the interaction between humans and objects is not
authentic due to the domain gap, and manual asset creation is difficult to
scale for a wide variety of objects. In this work, we present a novel
framework for learning a compositional generative model of humans and objects
(backpacks, coats, scarves, and more) from real-world 3D scans. Our
compositional model is interaction-aware, meaning the spatial relationship
between humans and objects, and the mutual shape change by physical contact is
fully incorporated. The key challenge is that, since humans and objects are in
contact, their 3D scans are merged into a single piece. To decompose them
without manual annotations, we propose to leverage two sets of 3D scans of a
single person with and without objects. Our approach learns to decompose
objects and naturally compose them back into a generative human model in an
unsupervised manner. Despite our simple setup requiring only the capture of a
single subject with objects, our experiments demonstrate the strong
generalization of our model by enabling the natural composition of objects to
diverse identities in various poses and the composition of multiple objects,
which is unseen in training data.

Figure 1: From 3D scans of the “source human” in casual clothing (left top)
and with additional outwear or objects (left bottom), our method automatically
decomposes objects from the source human and builds a compositional generative
model that enables 3D avatar creations of novel human identities with variety
of outwear and objects (right) in an unsupervised manner.

##  1 Introduction

Generative modeling of 3D humans from real-world data has shown promise to
represent and synthesize diverse human shapes, poses, and motions. Especially,
the ability to create realistic humans in diverse clothing and accessories
(e.g. backpacks, scarves, and hats) is indispensable for a myriad of
applications including VR/AR, entertainment, and virtual try-on. The early
work [4, 50, 36, 28, 71] has demonstrated success in modeling undressed human
bodies from real-world scans. Recently, the research community has been
focused on the generative modeling of clothed humans [38, 13, 16], to better
represent humans in everyday life.

Recent advancements in shape representations such as Neural Fields [69]
mitigate the need for pre-defining topology or template of clothing, enabling
to build animatable clothed humans from raw 3D scans [56, 14]. Along with its
advantage in strong expressive power for avatar modeling, this approach also
allows the models to learn faithful interactions between objects and humans.
However, since raw 3D scans do not provide a clear separation of different
components, existing approaches typically treat humans, clothing, and
accessories as an entangled block of geometry [13]. In this paper, we argue
that this leads to suboptimal expressiveness and composability of the
generative avatars. Many applications require more intuitive control to add,
replace, or modify objects while maintaining human identity. To make avatars
explicitly compositable with objects, some approaches propose to leverage
synthetic data [6, 27, 16]. However, the manual creation of 3D assets remains
a challenge and is extremely difficult to scale. Moreover, the physical
interaction of bodies, clothing, and accessories in synthetic data tends to be
less faithful due to the domain gap.

In contrast to prior methods, our goal is to build a compositional generative
model of objects and humans from real-world observations. The core challenge
lies in the difficulty of learning the composition and decomposition of
objects in contact from raw 3D scans. Capturing objects in isolation does not
lead to faithful composition due to the lack of realistic deformations induced
by physical contact. Thus, while it is essential to collect 3D scan data on
objects and humans in contact, the joint scanning of humans with objects only
provides an entangled block of 3D geometry as mentioned, and accurately
segmenting different components requires non-trivial 3D annotation efforts.

Upon these challenges, our contributions are: scalable data capture protocol,
unsupervised decomposition of objects and humans, and generalizable neural
object composition.

Scalable Data Capture. Capturing multiple identities with various poses and
objects requires prohibitively large time and storage. To overcome this issue,
we propose to collect human-object interactions with diverse poses only from a
single subject, referred to as the “source human”. To enable the decomposition
of objects, we also capture the same person without any objects, where the
deviation between two sets defines “objects” in our setup. Examples are shown
in Fig. 2. This capture protocol offers sufficient diversity in poses and
object types within a reasonable capture time.

Unsupervised Decomposition of Objects. To separate objects from the source
human, we leverage the expressiveness of the generative human model based on
implicit surface representation [13]. We train a human module without objects,
and then jointly optimize the latent codes of the avatar and a generative
model for objects to best explain the 3D scans of the person with objects.
While the human module accounts for state differences in pose and clothing,
the object-only module learns to synthesize the residual geometry as an object
layer in an unsupervised manner. Notably, objects in our work are defined as
residual geometry that cannot be explained by the trained human-only module.

Neural Object Composition. While the unsupervised decomposition successfully
separates objects from the source human, we observe that naively composing it
to novel identities from other datasets [52, 75] leads to undesired artifacts
and misalignment in the contact regions. To address this, we propose a neural
composition method by introducing another composition MLP that takes latent
features from both human and object modules to make a final shape prediction.
Due to the local nature of MLPs, our approach plausibly composes objects to
novel identities without retraining as in Fig. 1.

Our experiments show that our compositional generative model is superior to
existing approaches without explicit disentanglement of objects and humans
[13]. In addition, we show that our model can be used for fine-grained
controls including object removal from 3D scans and multiple object
compositions on a human, demonstrating the utility and expressiveness of our
approach beyond our training data.

##  2 Related Work

3D Human Models. Representing plausible 3D human bodies while handling diverse
variations in shapes and poses is a long-standing problem. Due to the
challenge in modeling diverse shape variation, the early work [4, 28, 36, 50,
71] mainly focuses on the undressed 3D human body by learning mesh-based
statistical models deformed from a template mesh. To model dressed 3D humans,
the follow-up work [2, 1, 37] adds 3D offsets on top of the parametric
undressed human body models to represent clothing. Yet, the topological
constraints and the resolution of the template model restrict these methods
from modeling arbitrary shapes of clothing with high-frequency details.
Recently emerging deep implicit shape representation [15, 39, 49, 42] provides
a breakthrough in expressing 3D humans by leveraging neural networks for
representing continuous 3D shape space, where its efficacy is demonstrated in
reconstructing clothed humans with high-fidelity from images [54, 55, 70].
There also has been an actively growing field to represent animatable 3D human
avatars using 3D scans [17, 14, 56, 62, 24, 40, 41, 13]. However, prior 3D
human models have paid little attention to the joint modeling of humans and
objects in close contact.

2D/3D Generative Models. Generative models intend to express the plausible
variations over the latent space, which can be used to create diverse
realistic samples. There have been extensive studies in 2D generative modeling
to create realistic photos [29, 30, 31] via generative adversarial networks
(GANs) [20, 21], variational autoencoders (VAEs) [33], and more recently,
diffusion models [60, 18, 23, 53]. Generative 3D modeling has also been
actively explored. By leveraging the availability of a large-scale 3D object
scans [12], many approaches present generative models for 3D objects [15, 39,
44, 45, 49, 58, 11]. Relatively few approaches have been presented for
generative 3D human modeling, due to the lack of available 3D datasets for
humans [71, 3, 16, 37, 13]. We show that our scalable data capture protocol
and compositional generative model enable the synthesis of 3D humans with
diverse objects in novel poses.

Compositional Models. Compositional generative models via neural networks have
been explored to represent different components as independent models,
representing a whole scene by compositing them together. These approaches
pursue controlling or sampling one component without affecting the rest. The
early approaches focus on building such models in 2D for creating realistic 2D
images via generative models [77, 63, 35, 5]. More recent approaches explore
the compositional reasoning for 3D [45, 74, 67, 73, 47, 9, 66, 34]. Most
approaches in this direction aim at synthesizing realistic novel views by
compositing NeRFs [42] for 3D objects and scenes [45, 67, 73] and for human
faces [45, 72, 9]. However, these approaches do not consider mutual shape
deformations between objects. Human bodies are also treated as a composition
of multiple body parts. These approaches attain final composition output by
either max-pooling the outputs of individual components [17, 40] or by using
another neural network [3, 7, 61, 46]. While a recent work shows interaction-
aware 3D composition reasoning is possible for faces and eyeglasses with
extensive annotations and data preprocessing [34], our approach supports
diverse object categories without requiring any manual annotations.

Garment Modeling. Due to the deformable nature of garments, capturing and
modeling 3D clothing is challenging. Only a few 3D garment datasets have been
presented [6, 76], where laborious segmentation and post-processing are
required to separate the garments from dummies or human bodies. While most
methods reconstruct a clothed 3D human as a single chunk of geometry [54, 55,
70], there exist methods reconstructing the 3D clothing as a separate layer on
top of parametric mesh model (e.g., SMPL) using segmentation [19] or synthetic
3D assets [27, 16]. Virtual try-on has also been actively explored in graphics
via physics simulation [68] and or synthetic data [64]. In contrast, our
approach learns a generative clothing and accessory model from real-world
observations in an unsupervised fashion.

##  3 Preliminaries

Figure 2: Examples of Our Datasets. Top row: sample scans of
$\mathcal{S}_{sh}$ containing the source human without objects. Bottom row:
sample scans of $\mathcal{S}_{sh+o}$ containing the source human with objects.

#### Data Acquisition.

To model humans and objects in contact, we capture two sets of datasets,
$\mathbf{S}_{sh}$ and $\mathbf{S}_{sh+o}$. $\mathbf{S}_{sh}$ consists of 3D
scans of a single identity, denoted as “source human” with various poses.
$\mathbf{S}_{sh+o}$ consists of 3D scans of the source human with a variety of
objects or additional outwear as shown in Fig. 2. In this work, we choose
coats, vests, backpacks, scarves, and hats to demonstrate the generality of
our approach for outwear and everyday accessories. To support the generative
modeling of objects, we capture multiple objects in each category. In addition
to $\mathbf{S}_{sh}$ and $\mathbf{S}_{sh+o}$, we also use other 3D human
dataset [75] to train another target generative human model for composition,
denoted $\mathbf{S}_{th}$.

We collect 3D scans with a system with synchronized and calibrated 8 Azure
Kinects (see supp. mat. for details). We apply KinectFusion [43] to fuse the
depth maps, and then reconstruct watertight meshes with screened-poisson
surface reconstruction [32]. We also detect 2D keypoints using OpenPose [10]
and apply the multi-view extension of SMPLify [8] to obtain SMPL parameters
[36] for each scan.

#### Generative Articulated Models.

We adopt the generative human model [13] which extends forward skinning with
root finding [14] for cross-identity modeling. We briefly discuss the
framework and highlight our key modifications. The key idea in gDNA [13] is to
represent occupancy fields conditioned by identity-specific latent codes
$\mathbf{z}$ in a canonical space, and transform them into a posed space using
forward linear blend skinning (LBS). The occupancy field defined for the
location $\mathbf{x}^{c}$ of a person in the canonical space can be
represented as follows:

$\displaystyle o(\mathbf{x}^{c})=O(\mathbf{x}^{c},G(\mathbf{z})),$ (1)

where $G(\cdot)$ is a spatially varying feature generator taking the latent
code. While the original work [13] uses 3D feature voxels for the output of
$G$, we use a tri-plane feature representation [11], which achieves better
performance with higher memory efficiency. The generated feature map is
conditioned on the latent code $\mathbf{z}$ via adaptive instance
normalization [25].

To query the occupancy fields in a posed space point $\mathbf{x}^{d}$, we
transform the canonical coordinate $\mathbf{x}^{c}$ as follows:

$\displaystyle\mathbf{x}^{d}=\sum_{i=1}^{n_{b}}W_{i}(N(\mathbf{x}^{c},\beta),\mathbf{z})\cdot\mathbf{B}_{i}(\beta,\theta)\cdot\mathbf{x}^{c},$
(2)

where $W_{i}$ is the identity conditioned skinning network, which outputs LBS
skinning weights for the $i$-th bone, and $N$ is the warping network given
SMPL shape parameters $\beta\in\mathbb{R}^{10}$.
$\mathbf{B}_{i}(\beta,\theta)$ is the transformation of the $i$-th bone in
SMPL model given SMPL pose parameters $\theta\in\mathbb{R}^{24\times 3}$ and
$\beta$. To jointly learn the occupancy and deformation networks, we solve for
$\mathbf{x}^{c}$ in Eq. 2 given $\mathbf{x}^{d}$ using iterative root finding
[14]. We discard the surface normal prediction networks used in [13] in both
canonical space and screen space. Instead of hallucinating details with fake
normals, we propose to model detailed geometry by jointly representing shapes
as SDF together with the occupancy fields. As we can directly supervise SDF on
surface normals [22], we model detailed geometry as true surface. However, we
empirically find that directly replacing the occupancy with SDF leads to
unstable training. To mitigate instability, we propose a hybrid modeling of
occupancy and SDF. We disable the backpropagation of gradients from SDF to the
deformation networks so that it is only supervised by the occupancy head. See
supp. mat. for details.

Figure 3: Overview. From captured scans of the source human with and without
objects, our method succesfully decomposes objects from humans without any
supervision, allowing a generative model to learn the shapes of various
objects. These objects are then added to novel identities via neural
composition, resulting in the creation of diverse human avatars with
controllable objects.

##  4 Method

Our goal is to build a compositional generative model that composes generative
objects on target humans from raw 3D scans. To this end, we introduce a
generative human module and a generative object module, followed by a
composition module. Fig. 3 shows an overview of our pipeline.

#### Human Module.

The human module $\mathcal{M}_{h}=(G_{h},O_{h},F_{h},D_{h})$ represents the
geometry of the human part and it is composed of a feature generator $G_{h}$,
a decoder $O_{h}$, and deformation networks $D_{h}=(W_{h},N_{h})$, where
$W_{h}$ and $N_{h}$ are a skinning weight network and a warping network as in
Eq. 2. As an output, $\mathcal{M}_{h}$ produces an occupancy value $o_{h}$, a
feature vector $\mathbf{f}_{h}$, and a signed distance $d_{h}$ in the
canonical space:

$\displaystyle(o_{h},\mathbf{f}_{h})=O_{h}(\mathbf{x}^{c},G_{h}(\mathbf{z}_{h})),$
(3) $\displaystyle d_{h}=F_{h}(\mathbf{x}^{c},G_{h}(\mathbf{z}_{h})).$ (4)

$\mathbf{f}_{h}$ is the intermediate latent feature before the last layer, and
$\mathbf{z}_{h}$ is a learnable latent code to vary the geometry of the human
part. Note that the hybrid modeling of occupancy and SDF is applied only to
the human module as our losses for unsupervised object decomposition require
occupancy.

#### Object Module.

The object module $\mathcal{M}_{o}=(G_{o},O_{o})$ is responsible for modeling
the geometry of the object part. Since the object module and the human module
share the same canonical space, the object module does not require separate
deformation networks. $\mathcal{M}_{o}$ returns an occupancy value $o_{o}$,
and a feature vector $\mathbf{f}_{o}$, which is the intermediate latent
feature before the last layer, in the canonical space:

$\displaystyle(o_{o},\mathbf{f}_{o})=O_{o}(\mathbf{x}^{c},G_{o}(\mathbf{z}_{o})),$
(5)

where $\mathbf{z}_{o}$ is a learnable latent code to vary the geometry of the
object part.

###  4.1 Neural Object Composition

Since the outputs of the human module and the object module share the same
canonical space and deformation networks, compositing the occupancy of the
human and object modules in a closed-form [17, 40] is possible. However, we
observe that this leads to misalignment in the contact regions and floating
artifacts. To address these issues, we introduce a neural composition module
parameterized by MLPs.

The composition module $\mathcal{M}_{comp}=(O_{comp},D_{comp})$ is used to
integrate humans and objects in the canonical space. We directly feed the
feature vectors $\mathbf{f}_{h}$ and $\mathbf{f}_{o}$ from the human module
and object module respectively as inputs. $\mathcal{M}_{comp}$ outputs the
final occupancy value $o_{comp}$, after composition in the canonical space:

$\displaystyle
o_{comp}=O_{comp}(\mathbf{x}^{c},\mathbf{f}_{h},\mathbf{f}_{o})$ (6)

Similar to the human module, the deformation networks
$D_{comp}=(W_{comp},N_{comp})$ provide the mapping from the canonical space to
the posed space. The entire model is illustrated in Fig. 4.

###  4.2 Unsupervised Object Decomposition

To decompose object layers from raw 3D scans in an unsupervised manner, our
key idea is to represent objects as the residual of human geometry. To this
end, we first train the human module $\mathcal{M}_{h}$ using
$\mathbf{S}_{sh}$, the dataset of source human without objects, along with the
learnable shape code $\mathbf{z}_{sh}$ for each scan. This allows the human
module to account for slight shape variations of the source human by changing
$\mathbf{z}_{sh}$. In the next step, using $\mathbf{S}_{sh+o}$, the dataset of
source human with objects, we jointly train all modules together. In
particular, we freeze the human module $\mathcal{M}_{h}$ while optimizing
$\mathbf{z}_{sh}$, $\mathcal{M}_{o}$, $\mathbf{z}_{o}$, and
$\mathcal{M}_{comp}$. Intuitively, the pretrained human module tries to handle
the geometry of the human part via optimization of $\mathbf{z}_{sh}$, while
the object parts, which cannot be expressed by $\mathcal{M}_{h}$, are handled
by $\mathcal{M}_{o}$ and $\mathbf{z}_{o}$. Given the composed occupancy
$o_{comp}$ in Eq. 6 and the predicted occupancy of the human module $o_{h}$,
the target occupancy of the object module can be computed as $(1-o_{h})\cdot
o_{comp}$. We jointly optimize the neural composition module and the object
module $\mathcal{M}_{o}$ in an end-to-end manner using the loss functions
discussed in Sec. 4.3.

Figure 4: Model. Given latent code $\mathbf{z}_{h}$, $\mathcal{M}_{h}$
predicts the occupancy fields and SDFs for humans in canonical space.
Similarly, with latent code $\mathbf{z}_{o}$, $\mathcal{M}_{o}$ predicts the
occupancy fields for objects. The features $\mathbf{f}_{h}$ and
$\mathbf{f}_{o}$ from each network are passed to $\mathcal{M}_{comp}$ to
predict the occupancy fields for final compositional outputs of humans and
objects in the same canonical space.

###  4.3 Training

Our system is trained using the datasets $\mathbf{S}_{th}$, $\mathbf{S}_{sh}$
and $\mathbf{S}_{sh+o}$ with their SMPL shape and pose parameters. Following
the auto-decoding framework of [49], we jointly optimize the latent code
$\mathbf{z}$ assigned for each scan along with the network weights during
training. Every scan in each dataset is assigned its own latent code, denoted
$\mathbf{z}_{th}\in\mathbb{R}^{L_{th}}$ for scans in $\mathbf{S}_{th}$,
$\mathbf{z}_{sh}\in\mathbb{R}^{L_{sh}}$ for scans in $\mathbf{S}_{sh}$ and
$\mathbf{z}_{o}\in\mathbb{R}^{L_{o}}$ for scans in $\mathbf{S}_{sh+o}$. For
$\mathbf{z}_{o}$, we use one-hot encoding for each object category using the
first 5 bits to enable random sampling from a specific category. Note that all
latent codes are initialized with zero.

To allow the unsupervised decomposition of objects from the source human as
discussed in Sec. 4.2, and to enable the creation of novel human identities
with objects, we train two separate human modules $\mathcal{M}_{sh}$ and
$\mathcal{M}_{th}$. $\mathcal{M}_{sh}$ is the instance of the human module for
modeling shapes of the source human, and $\mathcal{M}_{th}$ is another
instance of the human module for generating novel target human shapes.

Training consists of three stages: We first train $\mathcal{M}_{th}$ and
$\mathbf{z}_{th}$ with $\mathbf{S}_{th}$, to leverage the wide variation of
shapes and poses of samples in $\mathbf{S}_{th}$ for the multi-subject forward
skinning module, $D_{th}$. For later stages, $D_{th}$ is used to initialize
other deformation networks with its warping network $N_{th}$ frozen, to let
all samples share the same canonical space. Next, we train $\mathcal{M}_{sh}$
and $\mathbf{z}_{sh}$ with $\mathbf{S}_{sh}$. For the last stage, using all
the samples, we train $\mathcal{M}_{o}$, $\mathcal{M}_{comp}$,
$\mathbf{z}_{sh}$, and $\mathbf{z}_{o}$ with the pre-trained
$\mathcal{M}_{th}$, $\mathcal{M}_{sh}$ and $\mathbf{z}_{th}$ frozen. Note that
$\mathbf{z}_{sh}$ for the last stage are re-initialized as the mean of
$\mathbf{z}_{sh}$ after the second stage, denoted
$\mathbf{\overline{z}}_{sh}$. $\mathcal{M}_{comp}$ models all training samples
using the feature vector from either $\mathcal{M}_{th}$ or $\mathcal{M}_{sh}$
for the human part, and from $\mathcal{M}_{o}$ for the object part. In the
case of $\mathbf{S}_{th}$ and $\mathbf{S}_{sh}$ where scans are with no
objects, we introduce a new latent code $\mathbf{z}_{emp}$ as an alternative
input to $\mathcal{M}_{o}$ for no objects.

Losses: For the first stage, we use losses following [13]. We use the binary
cross entropy loss $\mathcal{L}_{th}$ between the predicted occupancy of
$\mathcal{M}_{th}$ and the ground truth occupancy. Note that $O^{d}(\cdot)$
and $F^{d}(\cdot)$ denote the occupancy field and SDF in posed space,
respectively. We also use guidance losses $\mathcal{L}_{bone}$,
$\mathcal{L}_{joint}$ and $\mathcal{L}_{warp}$ to aid training.
$\mathcal{L}_{bone}$ encourages the occupancy of $\mathbf{x}_{bone}$ to be
one, where $\mathbf{x}_{bone}$ are randomly selected points along the SMPL
bones in canonical space. $\mathcal{L}_{joint}$ encourages the skinning
weights of SMPL joints to be 0.5 for connected two bones and 0 for all other
bones. $\mathcal{L}_{warp}$ encourages deformation network $N$ to change body
size consistently, by enforcing vertices of a fitted SMPL to warp to vertices
of the mean SMPL shape, achieved by having shape parameter $\beta$ as zero.
Lastly, we use $\mathcal{L}_{reg\\_th}$ to regularize the latent code
$\mathbf{z}_{th}$ to be close to zero.

$\displaystyle\mathcal{L}_{th}=BCE((O_{th}^{d}(\mathbf{x}^{c},G_{th}(\mathbf{z}_{th})),o_{gt})$
(7)
$\displaystyle\mathcal{L}_{bone}=BCE((O_{th}(\mathbf{x}_{bone},G_{th}(\mathbf{z}_{th})),1)$
(8) $\displaystyle\mathcal{L}_{joint}=\lVert
W(\mathbf{x}_{joint},\mathbf{z}_{th})-\mathbf{w}_{gt}\rVert$ (9)
$\displaystyle\mathcal{L}_{warp}=\lVert
N(\mathbf{v}(\beta),\beta)-\mathbf{v}(\beta_{0})\rVert$ (10)
$\displaystyle\mathcal{L}_{reg\\_th}=\lVert\mathbf{z}_{th}\rVert$ (11)

For training the SDF network, we use L1 loss $\mathcal{L}_{sdf}$ between the
predicted and the ground truth signed distance and L2 loss $\mathcal{L}_{nml}$
between the gradients of SDF and the ground truth normals of points on the
surface. We additionally use $\mathcal{L}_{igr}$ for SDF to satisfy the
Eikonal equation [22] and $\mathcal{L}_{bbox}$ to prevent SDF values of off-
surface points from being the zero-level surface as in [59].

$\displaystyle\mathcal{L}_{sdf}=\lvert
F_{th}^{d}(\mathbf{x^{c}},G_{th}(\mathbf{z}_{th}))-d_{gt}\rvert$ (12)
$\displaystyle\mathcal{L}_{nml}=\lVert\nabla
F_{th}^{d}(\mathbf{x^{c}},G_{th}(\mathbf{z}_{th}))-n_{gt}\rVert$ (13)
$\displaystyle\mathcal{L}_{igr}=(\lVert\nabla
F_{th}(\mathbf{x^{c}},G_{th}(\mathbf{z}_{th}))\rVert-1)^{2}$ (14)
$\displaystyle\mathcal{L}_{bbox}=exp(-\alpha\cdot\lvert
F_{th}(\mathbf{x^{c}},G_{th}(\mathbf{z}_{th}))\rvert),\alpha\gg 1$ (15)

For the second stage, we use the binary cross entropy loss $\mathcal{L}_{sh}$
between the predicted occupancy of $\mathcal{M}_{sh}$ and the ground truth
occupancy, and $\mathcal{L}_{reg\\_sh}$ to regularize the latent code
$\mathbf{z}_{sh}$ to be close to zero. Since we initialize $D_{sh}$ with pre-
trained $D_{th}$, additional guidance losses are not required.

$\displaystyle\mathcal{L}_{sh}=BCE((O_{sh}^{d}(\mathbf{x}^{c},G_{sh}(\mathbf{z}_{sh})),o_{gt})$
(16) $\displaystyle\mathcal{L}_{reg\\_sh}=\lVert\mathbf{z}_{sh}\rVert$ (17)

For the last stage, we use the binary cross entropy loss $\mathcal{L}_{comp}$
between the predicted occupancy of $\mathcal{M}_{comp}$ and the ground truth
occupancy. We also use $\mathcal{L}_{o}$ between the predicted occupancy of
$\mathcal{M}_{o}$ and the residual part of $\mathbf{S}_{sh+o}$ where
$\mathcal{M}_{h}$ cannot explain. Moreover, we optimize $\mathbf{z}_{sh}$ by
using the binary cross entropy loss $\mathcal{L}_{fit}$ between the output of
$\mathcal{M}_{sh}$ and the ground truth occupancy. Finally, we regularize
$\mathbf{z}_{sh}$ to be close to $\mathbf{\overline{z}}_{sh}$ and
$\mathbf{z}_{o}$ to be close to zero.

$\displaystyle\mathcal{L}_{comp}=BCE((O_{comp}^{d}(\mathbf{x}^{c},\mathbf{f}_{h},\mathbf{f}_{o}),o_{gt})$
(18)
$\displaystyle\mathcal{L}_{o}=BCE((O_{o}(\mathbf{x}^{c},G_{o}(\mathbf{z}_{o})),(1-o_{h})\cdot
o_{comp})$ (19)
$\displaystyle\mathcal{L}_{fit}=BCE((O_{sh}^{d}(\mathbf{x}^{c},G_{sh}(\mathbf{z}_{sh})),o_{gt})$
(20)
$\displaystyle\mathcal{L}_{reg\\_sh}=\lVert\mathbf{z}_{sh}-\mathbf{\overline{z}}_{sh}\rVert$
(21) $\displaystyle\mathcal{L}_{reg\\_o}=\lVert\mathbf{z}_{o}\rVert$ (22)

##  5 Experiments

We evaluate our generative composition model across various scenarios. We
first demonstrate the quality of the random 3D avatar creations from our model
and the disentangled natures of human and object controls. Quantitative and
qualitative comparisons against the previous SOTA [13] are performed,
incorporating a user study via CloudResearch Connect. We also conduct ablation
studies to validate our design choices.

###  5.1 Dataset

Our 3D Scans: As described in Sec. 3, we use our multi-Kinect system to
capture the source human with and without objects, $\mathbf{S}_{sh}$ (180
samples) and $\mathbf{S}_{sh+o}$ (342 samples). For $\mathbf{S}_{sh+o}$, we
consider 4 categories of objects: 5 backpacks (77 samples in total), 6 outwear
(94 samples), 8 scarves (89 samples), and 6 hats (82 samples).

We run quantitative evaluation by focusing on backpacks as other objects such
as outwear are already incorporated in $\mathbf{S}_{th}$. We use another set
with 300 samples of the source human with backpacks only, denoted as
$\mathbf{S}_{sh+bp}$. To build a testing set for FID computation in this
quantitative evaluation, we further capture 343 samples of 3 different unseen
identities who wear unseen backpacks. We denote this test dataset,
$\mathbf{S}_{unseen+bp}$.

THuman2.0 [75]: THuman2.0111The THuman2.0 dataset was downloaded, accessed,
and used in this research exclusively at SNU. provides high-quality 3D dataset
for dressed humans. We use 526 samples for $\mathbf{S}_{th}$.

###  5.2 Qualitative Evaluation

We demonstrate the expressive power and controllability of our composition
model via inferences in various scenarios by controlling latent codes for
humans $\mathbf{z}_{h}$ and object $\mathbf{z}_{o}$.

Figure 5: Random Generation. Top row: randomly sampled outputs of the human
module before composition. Bottom row: composition outputs of target humans on
top with specific objects.

Random Generation. The 3D avatars created by attaching specific object latent
codes $\mathbf{z}_{o}$ to random sampled human codes $\mathbf{z}_{h}$ are
shown in Fig. 5 (bottom). The outputs of the human module $\mathcal{M}_{h}$
are also shown on the top of Fig. 5 for reference. Our model enables the
creation of diverse 3D avatars with controllable objects.

Figure 6: Disentangled Human and Object. Top row: composition outputs of the
same object (a scarf), added to different human identities. Bottom row:
composition outputs of different objects added to the single human identity
shown in the leftmost column.

Disentangled Controls over Human and Objects. To further test the disentangled
nature of our composition model, we create 3D humans with objects by changing
either human latent code or object latent code, as shown in Fig. 6. The
examples on the top vary the human part by keeping the same object code that
represents a scarf. On the bottom examples, we vary object codes for a fixed
identity shown on the leftmost side. These results show the core advantage of
our composition model in individual controls.

Figure 7: Interpolation. Top row: human module interpolation. Bottom row:
object module interpolation. Notice that interpolating one module doesn’t
deteriorate the geometry of the other.

Interpolation. Fig. 7 demonstrates smooth interpolation of each module without
deteriorating the other module.

Figure 8: Composition of Multiple Objects. Two or more objects are added to
the leftmost human. Note that our train data contain no scans of the source
human with multiple objects.

Composition of Multiple Objects. Fig. 8 shows that our system allows the
composition of multiple objects. To add multiple objects, we use the latent
code of each object and get the occupancy and the feature vector of objects.
Using the normalized occupancy of multiple objects as weights, we calculate
the weighted sum of feature vectors. The aggregated feature is then fed to the
composition module along with the human feature to get the final composition
output. Note that our dataset has no such sample with multiple objects.

Method | FID | User Preference
---|---|---
gDNA (w/ object) | 41.71 | 43.6%
Arith. gDNA (w/ object) | 73.81 | 13.6%
Ours (Naive composition) | 55.29 | 22.4%
Ours | 51.03 | 100% - (above)
Table 1: Quantitative evaluation of the importance of compositional modeling.
User preference score reflects the frequency with which participants of our
perceptual study favored each method over ours. Figure 9: Qualitative
Comparison on Compositional Modeling. Compared to our method, baselines suffer
from generating outputs of diverse humans with complete objects.

###  5.3 Comparison with SOTA

Since our method is the first generative model for compositing humans and
objects, there is no direct competitor, and comparison with the previous non-
compositional model such as gDNA is non-trivial. To make the assessment
possible at our best, we consider a specific scenario where a user wants to
create samples with a specific object category, being the backpack here. To
provide such controllability on gDNA, we first extend the gDNA model with our
dataset. Note that, in this evaluation, we use the same dataset
$\mathbf{S}_{th}$, $\mathbf{S}_{sh}$, $\mathbf{S}_{sh+bp}$ for training both
our model and gDNA.

Extending gDNA for Composition. We train gDNA model using the public code with
our datasets. Both human-only outputs and the ones with a backpack can be
sampled from the trained model. To intentionally generate outputs with a
backpack, we search the latent codes associated with the training samples with
backpacks and fit a gaussian from which we can perform a sampling. We denote
this baseline method as _‘gDNA (w/ object)’_.

The second possible extension of gDNA is based on the arithmetic operation
among gDNA’s latent codes, which is widely used for GAN-based image
manipulation [51]. We found that gDNA’s original framework allows some level
of composition by adding or subtracting the latent codes. Specifically, we
choose a latent code $\mathbf{z}_{sh}^{*}$ for the source human without a
backpack and another latent code $\mathbf{z}_{sh+bp}^{*}$ for the source human
with the backpack. We simply take their subtraction
$\mathbf{z}_{bp}=\mathbf{z}_{sh+bp}^{*}-\mathbf{z}_{sh}^{*}$, which can be
considered as a residual for the backpack. We found that composition can be
performed by adding this residual to another human’s latent code, that is
$\mathbf{z}_{bp}+\mathbf{z}_{th}$. We denote this baseline method as _‘Arith.
gDNA (w/ object)’_.

Qualitative Comparison with User Study. The visual comparison between ours and
the extended gDNAs is shown in Fig. 9. In the first row, we show random
samples generated from _‘gDNA (w/ object)’_. Since the human scans with the
backpack are only of the source human’s (other samples from $\mathbf{S}_{th}$
do not have any backpack), the generated outputs lack shape variety for the
human part, producing always the source human’s identity. In the second row of
Fig. 9, backpacks are added to novel identities; however, the method suffers
from lack of details on both humans and objects. In contrast, the outputs of
our method shown in the last row show strong generalization by creating
diverse human identities with naturally attached detailed objects.

To further validate this comparison, we perform a user test (A/B test) on
CloudResearch Connect. We render samples from three viewpoints (same views for
all) and show ours with each baseline (A/B examples) in a random order to each
subject. Each subject answers 5 questions per baseline by choosing more
authentic 3D human samples. The data was collected from 50 subjects. The
results are shown in the “User Preference” column in Tab. 1. As shown, our
methods are preferred over extended gDNA baselines. Moreover, to confirm the
diversity of identities in our method and _‘gDNA (w/ object)’_ , 50 subjects
were shown the rendering of the source human and were asked to choose samples
that don’t resemble the source human. Samples of our method were chosen by
92.4%, indicating that _‘gDNA (w/ object)’_ suffers to generate novel
identities with a backpack.

Quantiative Evaluation via FID. To evaluate the generation quality of our
method, we compare Fréchet Inception Distance (FID) between the 2D normal
renderings of the test dataset $\mathbf{S}_{unseen+bp}$ and the generated
outputs, following [13]. The result is shown in Tab. 1. _‘gDNA (w/ object)’_
has a relatively better score than ours, due to the fact that it only samples
3D humans around $\mathbf{S}_{sh+bp}$, which are always close to the GT
samples. A more fair comparison is between ours and _‘Arith. gDNA (w/
object)’_ , where both approaches try to attach the backpack to novel
identities. Our method significantly outperforms this baseline.

Method |  Pred-to-Scan${\downarrow}$ |  Scan-to-Pred${\downarrow}$
---|---|---
gDNA | 0.0162 | 0.0190
gDNA(w/ object) | 0.0218 | 0.0112
Ours | 0.0116 | 0.0099
Table 2: Fitting accuracy comparison with the SOTA method [13]. Figure 10:
Fitting and Object Removal. Compared to baselines, our method successfully
explains both human shapes and object shapes, enabling the natural removal of
objects after fitting.

Performance on Fitting. We evaluate the expressiveness of our model by fitting
it to unseen scans with objects. As a baseline, we consider gDNA [13] as it
demonstrates better fitting results on 3D clothed human scans over other SOTA
methods [48, 16]. Besides the original gDNA trained with $\mathbf{S}_{th}$, we
also consider gDNA trained with $\mathbf{S}_{th}$, $\mathbf{S}_{sh}$ and
$\mathbf{S}_{sh+bp}$ (_‘gDNA (w/ object)’_) to enable fitting of the object
part. We use scans with backpacks from Renderpeople222RenderPeople was
downloaded, accessed, and used in this research exclusively at SNU. [52] and
captured dataset $\mathbf{S}_{unseen+bp}$ for fitting comparison.

As shown in Tab. 2, our method reports better fitting accuracy than the
baselines. Our method effectively fits the geometry of both humans and objects
while baselines only reconstruct either the human part or the object part as
shown in Fig. 10. Moreover, since our method separately models humans and
objects, it enables the high-quality removal of objects after fitting.

###  5.4 Ablation Study

Figure 11: Composition Comparison. While naive composition suffers from severe
artifacts, neural composition reduces these artifacts and produces high-
quality outputs.

Neural Composition. Our system provides two ways of extracting the final
composition output. One is by using $o_{comp}$: neural composition, and the
other is by using the maximum value between $o_{h}$ and $o_{o}$ of queried
points: naive composition. We verify the necessity of using neural composition
in order to generate high-quality outputs of humans with objects. Compared to
naive composition, neural composition remarkably reduces the artifacts induced
by the imperfect fitting of the source human, resulting in lower FID values
(Tab. 1). Qualitative comparison is presented in Fig. 11.

##  6 Discussion

We present a novel framework for learning a compositional generative model of
humans and objects. Our compositional generative model provides separate
control over the human part and the object part. To train our compositional
model without manual annotation for the object geometries, we propose to
leverage 3D scans of a single person with and without objects. Our results
show that the learned generative model for the object part can be
authentically transferred to novel human identities.

Limitations and Future Work. While our approach is general and supports
diverse objects, decomposing thin layers of clothing in an unsupervised manner
remains a challenge due to the limited precision of 3D scans. Extending our
approach to modeling from RGB images is also an exciting research direction
for future work.

Acknowledgements: This work of H. Joo and T. Kim was supported by SNU
Creative-Pioneering Researchers Program and IITP grant funded by the Korean
government (MSIT) [NO.2021-0-01343 and No.2022-0-00156]

## References

  * [1] Thiemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian Theobalt, and Gerard Pons-Moll.  Learning to reconstruct people in clothing from a single rgb camera.  In CVPR, 2019.
  * [2] Thiemo Alldieck, Marcus Magnor, Weipeng Xu, Christian Theobalt, and Gerard Pons-Moll.  Video based reconstruction of 3d people models.  In CVPR, 2018.
  * [3] Thiemo Alldieck, Hongyi Xu, and Cristian Sminchisescu.  imghum: Implicit generative models of 3d human shape and articulated pose.  In ICCV, 2021.
  * [4] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.  Scape: shape completion and animation of people.  In TOG. 2005.
  * [5] Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, and Trevor Darrell.  Compositional gan: Learning image-conditional binary composition.  IJCV, 2020.
  * [6] Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll.  Multi-garment net: Learning to dress 3d people from images.  In ICCV, 2019.
  * [7] Sourav Biswas, Kangxue Yin, Maria Shugrina, Sanja Fidler, and Sameh Khamis.  Hierarchical neural implicit pose network for animation and motion retargeting.  arXiv preprint arXiv:2112.00958, 2021.
  * [8] Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black.  Keep it smpl: Automatic estimation of 3d human pose and shape from a single image.  In ECCV, 2016.
  * [9] Mallikarjun BR, Ayush Tewari, Xingang Pan, Mohamed Elgharib, and Christian Theobalt.  gcorf: Generative compositional radiance fields.  3DV, 2022.
  * [10] Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh.  Realtime multi-person 2d pose estimation using part affinity fields.  In CVPR, 2017.
  * [11] Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al.  Efficient geometry-aware 3d generative adversarial networks.  In CVPR, 2022.
  * [12] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al.  Shapenet: An information-rich 3d model repository.  arXiv preprint arXiv:1512.03012, 2015.
  * [13] Xu Chen, Tianjian Jiang, Jie Song, Jinlong Yang, Michael J Black, Andreas Geiger, and Otmar Hilliges.  gdna: Towards generative detailed neural avatars.  In CVPR, 2022.
  * [14] Xu Chen, Yufeng Zheng, Michael J Black, Otmar Hilliges, and Andreas Geiger.  Snarf: Differentiable forward skinning for animating non-rigid neural implicit shapes.  In ICCV, 2021.
  * [15] Zhiqin Chen and Hao Zhang.  Learning implicit fields for generative shape modeling.  In CVPR, 2019.
  * [16] Enric Corona, Albert Pumarola, Guillem Alenya, Gerard Pons-Moll, and Francesc Moreno-Noguer.  Smplicit: Topology-aware generative model for clothed people.  In CVPR, 2021.
  * [17] Boyang Deng, John P Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi.  Nasa neural articulated shape approximation.  In ECCV, 2020.
  * [18] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  In NeurIPS, 2021.
  * [19] Yao Feng, Jinlong Yang, Marc Pollefeys, Michael J. Black, and Timo Bolkart.  Capturing and animation of body and clothing from monocular video.  In SIGGRAPH Asia, 2022.
  * [20] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In NeurIPS, 2014.
  * [21] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial networks.  Communications of the ACM, 2020.
  * [22] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman.  Implicit geometric regularization for learning shapes.  In ICML. 2020.
  * [23] Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans.  Cascaded diffusion models for high fidelity image generation.  JMLR, 2022.
  * [24] Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, and Ziwei Liu.  Avatarclip: Zero-shot text-driven generation and animation of 3d avatars.  TOG, 2022.
  * [25] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In ICCV, 2017.
  * [26] Boyi Jiang, Yang Hong, Hujun Bao, and Juyong Zhang.  Selfrecon: Self reconstruction your digital avatar from monocular video.  In CVPR, 2022.
  * [27] Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, and Hujun Bao.  Bcnet: Learning body and cloth shape from a single image.  In ECCV, 2020.
  * [28] Hanbyul Joo, Tomas Simon, and Yaser Sheikh.  Total capture: A 3d deformation model for tracking faces, hands, and bodies.  In CVPR, 2018.
  * [29] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of gans for improved quality, stability, and variation.  ICLR, 2017.
  * [30] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In CVPR, 2019.
  * [31] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In CVPR, 2020.
  * [32] Michael Kazhdan and Hugues Hoppe.  Screened poisson surface reconstruction.  TOG, 2013.
  * [33] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  ICLR, 2013.
  * [34] Junxuan Li, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Hongdong Li, and Jason Saragih.  Megane: Morphable eyeglass and avatar network.  In CVPR, 2023.
  * [35] Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, and Simon Lucey.  St-gan: Spatial transformer generative adversarial networks for image compositing.  In CVPR, 2018.
  * [36] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black.  SMPL: A skinned multi-person linear model.  SIGGRAPH Asia, 2015.
  * [37] Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J Black.  Learning to dress 3d people in generative clothing.  In CVPR, 2020.
  * [38] Qianli Ma, Jinlong Yang, Siyu Tang, and Michael J. Black.  The power of points for modeling humans in clothing.  In ICCV, 2021.
  * [39] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger.  Occupancy networks: Learning 3d reconstruction in function space.  In CVPR, 2019.
  * [40] Marko Mihajlovic, Shunsuke Saito, Aayush Bansal, Michael Zollhoefer, and Siyu Tang.  Coap: Compositional articulated occupancy of people.  In CVPR, 2022.
  * [41] Marko Mihajlovic, Yan Zhang, Michael J Black, and Siyu Tang.  Leap: Learning articulated occupancy of people.  In CVPR, 2021.
  * [42] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  In ECCV, 2020.
  * [43] Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon.  Kinectfusion: Real-time dense surface mapping and tracking.  In ISMAR, 2011.
  * [44] Thu Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt, and Yong-Liang Yang.  Hologan: Unsupervised learning of 3d representations from natural images.  In ICCV, 2019.
  * [45] Michael Niemeyer and Andreas Geiger.  Giraffe: Representing scenes as compositional generative neural feature fields.  In CVPR, 2021.
  * [46] Atsuhiro Noguchi, Xiao Sun, Stephen Lin, and Tatsuya Harada.  Neural articulated radiance field.  In ICCV, 2021.
  * [47] Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide.  Neural scene graphs for dynamic scenes.  In CVPR, 2021.
  * [48] Pablo Palafox, Aljaž Božič, Justus Thies, Matthias Nießner, and Angela Dai.  Npms: Neural parametric models for 3d deformable shapes.  In ICCV, 2021.
  * [49] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation.  In CVPR, 2019.
  * [50] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black.  Expressive body capture: 3D hands, face, and body from a single image.  In CVPR, 2019.
  * [51] Alec Radford, Luke Metz, and Soumith Chintala.  Unsupervised representation learning with deep convolutional generative adversarial networks.  ICLR, 2016.
  * [52] Renderpeople, 2018.  https://renderpeople.com/3d-people.
  * [53] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In CVPR, 2022.
  * [54] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li.  Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization.  In ICCV, 2019.
  * [55] Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo.  Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization.  In CVPR, 2020.
  * [56] Shunsuke Saito, Jinlong Yang, Qianli Ma, and Michael J Black.  Scanimate: Weakly supervised learning of skinned clothed avatar networks.  In CVPR, 2021.
  * [57] Johannes L Schonberger and Jan-Michael Frahm.  Structure-from-motion revisited.  In CVPR, 2016.
  * [58] Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger.  Graf: Generative radiance fields for 3d-aware image synthesis.  NeurIPS, 2020.
  * [59] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein.  Implicit neural representations with periodic activation functions.  NeurIPS, 2020.
  * [60] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In ICML, 2015.
  * [61] Shih-Yang Su, Frank Yu, Michael Zollhöfer, and Helge Rhodin.  A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose.  NeurIPS, 2021.
  * [62] Garvita Tiwari, Nikolaos Sarafianos, Tony Tung, and Gerard Pons-Moll.  Neural-gif: Neural generalized implicit functions for animating people in clothing.  In ICCV, 2021.
  * [63] Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, and Ming-Hsuan Yang.  Deep image harmonization.  In CVPR, 2017.
  * [64] Raquel Vidaurre, Igor Santesteban, Elena Garces, and Dan Casas.  Fully convolutional graph neural networks for parametric virtual try-on.  In CGF, 2020.
  * [65] Shaofei Wang, Katja Schwarz, Andreas Geiger, and Siyu Tang.  Arah: Animatable volume rendering of articulated human sdfs.  In ECCV, 2022.
  * [66] Ziyan Wang, Timur Bagautdinov, Stephen Lombardi, Tomas Simon, Jason Saragih, Jessica Hodgins, and Michael Zollhofer.  Learning compositional radiance fields of dynamic human heads.  In CVPR, 2021.
  * [67] Qianyi Wu, Xian Liu, Yuedong Chen, Kejie Li, Chuanxia Zheng, Jianfei Cai, and Jianmin Zheng.  Object-compositional neural implicit surfaces.  In ECCV, 2022.
  * [68] Donglai Xiang, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, et al.  Dressing avatars: Deep photorealistic appearance for physically simulated clothing.  TOG, 2022.
  * [69] Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar.  Neural fields in visual computing and beyond.  In CGF, 2022.
  * [70] Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J Black.  Icon: implicit clothed humans obtained from normals.  In CVPR, 2022.
  * [71] Hongyi Xu, Eduard Gabriel Bazavan, Andrei Zanfir, William T Freeman, Rahul Sukthankar, and Cristian Sminchisescu.  Ghum & ghuml: Generative 3d human shape and articulated pose models.  In CVPR, 2020.
  * [72] Yang Xue, Yuheng Li, Krishna Kumar Singh, and Yong Jae Lee.  Giraffe hd: A high-resolution 3d-aware generative model.  In CVPR, 2022.
  * [73] Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, and Zhaopeng Cui.  Learning object-compositional neural radiance field for editable scene rendering.  In ICCV, 2021.
  * [74] Hong-Xing Yu, Leonidas J. Guibas, and Jiajun Wu.  Unsupervised discovery of object radiance fields.  In ICLR, 2022.
  * [75] Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu.  Function4d: Real-time human volumetric capture from very sparse consumer rgbd sensors.  In CVPR, 2021.
  * [76] Heming Zhu, Yu Cao, Hang Jin, Weikai Chen, Dong Du, Zhangye Wang, Shuguang Cui, and Xiaoguang Han.  Deep fashion3d: A dataset and benchmark for 3d garment reconstruction from single images.  In ECCV, 2020.
  * [77] Jun-Yan Zhu, Philipp Krahenbuhl, Eli Shechtman, and Alexei A Efros.  Learning a discriminative model for the perception of realism in composite images.  In ICCV, 2015.

##  Appendix A Implementation details

###  A.1 Network Architectures

Latent codes assigned to each scan, $\mathbf{z}_{th}$, $\mathbf{z}_{sh}$, and
$\mathbf{z}_{o}$ are 64-dimensional. For $\mathbf{z}_{o}$, we use its first 5
bits to encode the object category via one-hot encoding and optimize only the
last 59 bits during training. The generator $G$ of the human module and the
object module generates the $256\times 256\times 64$ feature image from a
constant vector of size $256\times 16\times 16$ via 4 layers of (bilinear
upsampler with a scale factor of 2, 2D convolution of kernel size 3 and stride
1, adaIN for conditioning the generator with the latent code $\mathbf{z}$, and
leaky ReLU activations). The $256\times 256\times 64$ output feature image is
split into one $256\times 256\times 32$ and two $256\times 128\times 32$ to
form a tri-plane feature map. Note that the feature map is 128-dimensional
along z-axis and 256-dimensional along other axes. The decoder for predicting
the occupancy of the human module and the object module is a multi-layer
perceptron having the intermediate neuron size of (256, 256, 256, 229, 1) with
skip connection from the input features to the 4th layer and nonlinear
activations of softplus with $\beta=100$ except for the last layer that uses
sigmoid. As an input, it takes the Cartesian coordinates in canonical space
which are encoded using a positional encoding with 4 frequency components, and
the 32-dimensional feature queried from the generated tri-plane. The decoder
for predicting SDF of the human module has the same architecture as the
decoder for predicting the occupancy, except that it has no activations for
the last layer. The decoder for predicting the occupancy of the composition
module has the same architecture as the decoders for predicting the occupancy
of other modules. However, instead of taking in the feature from the generated
tri-plane as an input, it takes in the intermediate latent feature vectors
before the last layer of the decoders for predicting the occupancy of the
human module and object module, which are 229-dimensional each.

Our deformation networks $D=(W,N)$ follow the architecture of the deformer of
gDNA [13]. The skinning network $W$ is a multi-layer perceptron having the
intermediate neuron size of (128, 128, 128, 128, 24) with nonlinear
activations of softplus with $\beta=100$, except for the last layer that uses
softmax in order to get normalized skinning weights. As an input, it takes the
Cartesian coordinates in canonical space and the latent code
$\mathbf{z}\in\mathbb{R}^{64}$ of the training sample. The warping network $N$
is also a multi-layer perceptron having the intermediate neuron size of (128,
128, 128, 128, 3) with nonlinear activations of softplus. As an input, it
takes the Cartesian coordinates in canonical space and the SMPL shape
parameter $\beta\in\mathbb{R}^{10}$ of the training sample. The input
Cartesian coordinates are passed to the last layer for the network to learn
residual displacements.

###  A.2 Training Procedure

Our training consists of three stages. First, we train $\mathcal{M}_{th}$ and
$\mathbf{z}_{th}$ with $\mathbf{S}_{th}$ with losses following [14, 13] and
additional losses to train the SDF network. The total loss
$\mathcal{L}_{M_{th}}$ is as follows:

$\displaystyle\mathcal{L}_{M_{th}}=\mathcal{L}_{th}+\lambda_{bone}\mathcal{L}_{bone}+\lambda_{joint}\mathcal{L}_{joint}+\lambda_{warp}\mathcal{L}_{warp}$
(23)
$\displaystyle+\lambda_{reg\\_th}\mathcal{L}_{reg\\_th}+\mathcal{L}_{sdf}+\mathcal{L}_{nml}+\mathcal{L}_{igr}+\mathcal{L}_{bbox},$

where $\lambda_{warp}=10$ and $\lambda_{reg\\_th}=10^{-3}$. We set
$\lambda_{bone}=1$ and $\lambda_{joint}=10$ only for the first epoch and 0
afterwards.

For the second stage, we train $\mathcal{M}_{sh}$ and $\mathbf{z}_{sh}$ with
$\mathbf{S}_{sh}$ with the total loss $\mathcal{L}_{M_{th}}$ being,

$\displaystyle\mathcal{L}_{M_{sh}}=\mathcal{L}_{sh}+\lambda_{reg\\_sh}\mathcal{L}_{reg\\_sh},$
(24)

where $\lambda_{reg\\_sh}=10^{-3}$. As described in the main paper, since we
initialize $D_{sh}$ with the pre-trained $D_{th}$, additional guidance losses
as in the first stage are not required. Note that since it is not our primary
objective to model the detailed surface of the source human, we don’t utilize
the hybrid modeling of occupancy and SDF for $\mathcal{M}_{sh}$.

For the last stage, we train $\mathcal{M}_{o}$, $\mathcal{M}_{comp}$,
$\mathbf{z}_{sh}$, and $\mathbf{z}_{o}$ with the pre-trained
$\mathcal{M}_{th}$, $\mathcal{M}_{sh}$ and $\mathbf{z}_{th}$ frozen. As
described in the main paper, $\mathbf{z}_{sh}$ for the last stage are re-
initialized as the mean of $\mathbf{z}_{sh}$ after the second stage. The total
loss $\mathcal{L}$ is as follows:

$\displaystyle\mathcal{L}=\mathcal{L}_{comp}+\mathcal{L}_{o}+\lambda_{fit}\mathcal{L}_{fit}$
(25)
$\displaystyle+\lambda_{reg\\_sh}\mathcal{L}_{reg\\_sh}+\lambda_{reg\\_o}\mathcal{L}_{reg\\_o},$

where $\lambda_{fit}=0.2$, $\lambda_{reg\\_sh}=50$, and
$\lambda_{reg\\_sh}=10^{-3}$.

We train each stage with the Adam optimizer with a learning rate of 0.001
without decay. All stages are trained for 300 epochs.

###  A.3 Inference

We generate the composited canonical shapes of general people with objects by
random sampling $\mathbf{z}_{th}$ and $\mathbf{z}_{o}$ from the Gaussian
distribution fitted to each set of latent codes. We then extract meshes using
$o_{comp}$ with a resolution of $256^{3}$. We finally repose the output mesh
using the SMPL pose parameter with the learned skinning fields.

##  Appendix B Data

###  B.1 Acquisition

We collect 3D scans of the source human with and without objects using a
system with synchronized and calibrated 8 Azure Kinects. We capture data 5FPS
with the resolution of $2048\times 1536$ for the RGB cameras, and $1024\times
1024$ for the depth cameras. We perform image-based calibration using COLMAP
[57] and adjust the optimized camera extrinsics to real-world scale based on
the corresponding depth maps. We apply KinectFusion [43] with the code from
the repository 333https://github.com/andyzeng/tsdf-fusion-python to fuse the
captured depth maps with the voxel resolution of 1.5mm. We reconstruct
watertight meshes from the fused output using screened-poisson surface
reconstruction [32] of depth 9. In order to obtain SMPL parameters for each
captured scan, we use the multi-view extension of SMPLify [8] with the code
from the repository 444https://github.com/ZhengZerong/MultiviewSMPLifyX. For
each scan, we render images from 18 viewpoints and detect 2D keypoints using
OpenPose [10], and apply the multi-view extension of SMPLify to estimate SMPL
parameters for each scan.

###  B.2 Data Statistics

We use 180 samples for $\mathbf{S}_{sh}$ and 342 samples for
$\mathbf{S}_{sh+o}$. For $\mathbf{S}_{sh+o}$, we consider 4 categories of
objects: 5 backpacks (77 samples in total), 6 outwear (94 samples), 8 scarves
(89 samples), and 6 hats (82 samples). For running the quantitative evaluation
focused on backpacks, we use another set with 300 samples of the source human
with 5 backpacks, denoted as $\mathbf{S}_{sh+bp}$. To build a testing set for
FID computation, we further capture 343 samples of 3 different unseen
identities who wear unseen backpacks, denoted as $\mathbf{S}_{unseen+bp}$. We
also use 526 samples of THuman2.0 [75] for $\mathbf{S}_{th}$.

##  Appendix C Discussion

Method | Chamfer${\downarrow}$ | P2S${\downarrow}$ | Normal${\downarrow}$
---|---|---|---
Occ | 0.0140 | 0.0169 | 0.0092
Occ & SDF | 0.0098 | 0.0128 | 0.0074
Table 3: Quantitative evaluation of the significance of using the hybrid
modeling of occupancy and SDF is presented. Figure 12: Qualitative Comparison
on Introducing SDF Network in the Human Module. Top row: Generated outputs
when trained with occupancy only. Bottom row: Generated outputs when trained
with the hybrid modeling of occupancy and SDF. Additionally predicting the SDF
improves the details of generated outputs.

###  C.1 Geometry Modeling with SDF

As mentioned in the main paper, we model detailed geometry by jointly
predicting SDF together with the occupancy fields. We find that directly
replacing the occupancy with the SDF leads to failures in canonicalization.
Among the set of correspondences resulting from multiple initials for the root
finding algorithm, previous work that uses occupancy representation [14, 13]
determines the final correspondence by choosing the point with the highest
estimated occupancy. However, in the case of the SDF representation, we
empirically find out that choosing the point by only utilizing the estimated
SDF leads to poor canonicalization. Moreover, using a single initial by
linearly combining the skinning weights of the nearest neighbor on the fitted
SMPL mesh and the inverse bone transformations as in [65, 26] also leads to
incorrect canonicalization. Hence, we utilize a hybrid modeling of occupancy
and SDF by leveraging the advantage of each representation. While directly
supervising SDF on the surface normals, we select final correspondences and
train the deformation networks using occupancy. For stable training, it is
crucial to disable the backpropagation of gradients from the SDF head to the
deformation networks and let only the occupancy head supervise them.

We verify the significance of predicting both occupancy and SDF over
predicting only occupancy to generate outputs with higher frequency details.
For each method, we reconstruct the ground truth data used for training with
assigned latent codes. We compute the Chamfer distance and point-to-surface
distance (P2S) between the ground truth and the reconstruction output. We also
render 2D normal maps from fixed views and compute the L2 error (Normal). As
demonstrated in Tab. 3, reconstruction outputs are improved when both
occupancy and SDF are predicted. In Fig. 12 we show the qualitative comparison
between samples generated via each method.

##  Appendix D Quantitative Evaluation Details

###  D.1 FID Computation

We compute FID score using the code from the
repository555https://github.com/mseitzer/pytorch-fid. For the test set, we
render 2D normal maps in resolution $256^{2}$ of 343 samples in
$\mathbf{S}_{unseen+bp}$ from 18 viewpoints, resulting in 6174 images. For
each method, we generate 200 samples in random body sizes and poses of the
$\mathbf{S}_{unseen+bp}$ and similarly render 2D normal maps in resolution
$256^{2}$ from 18 viewpoints, resulting in 3600 images.

Figure 13: Example Images of the First User Study. Subjects are asked to
choose the sample with a more authentic shape between top and bottom. Figure
14: Example Images of the Second User Study. Subjects are asked to choose the
sample that does not resemble the shape of the source human shown on the left,
between top and bottom.

###  D.2 User Preference Study

We perform two user preference studies (A/B test) via CloudResearch Connect.
The first study aims to validate the generation quality of our method over all
baselines, and the second study aims to validate the generation diversity of
our method over _‘gDNA (w/ object)’_.

For the first user study, we show a sample generated with our method along
with another sample generated with one of the baseline methods in random
order. For each sample, we render 2D normal maps in resolution $256^{2}$ from
3 viewpoints. We ask 50 subjects to answer 5 A/B pairs per baseline by
choosing the preferred sample with a more authentic shape. An example of a
question is presented in Fig. 13

For the second user study, we only compare our method with the baseline,
_‘gDNA (w/ object)’_ , with a different protocol. In this study, we similarly
render the normal maps from 3 viewpoints from each method and additionally
show an image of the source human along with the A/B pairs. Then, we request
the observers to choose the sample that looks more different from the source
human. The test is intended to see whether the methods can produce diverse
human identities with objects, sufficiently different from the source human’s
appearance. An example of a question is presented in Fig. 14. Similar to the
first study, we ask 50 subjects to answer 5 A/B pairs by choosing the sample
that better satisfies the question.

###  D.3 Fitting Comparison

For fitting our model to unseen scans with objects, we follow the fitting
process of gDNA [13]. During fitting, we optimize the latent code for the
human part, $\mathbf{z}_{h}$, the latent code for the object part,
$\mathbf{z}_{o}$, and the SMPL shape parameter $\beta$ with other network
frozen. We use $\mathcal{M}_{th}$ for the human module. We initialize
$\mathbf{z}_{h}$ and $\mathbf{z}_{o}$ each with 8 randomly sampled codes from
the Gaussian distribution fitted to each set of latent codes. $\beta$ is
initialized with the obtained SMPL shape parameter during our data acquisition
process. The loss $\mathcal{L}_{fitting}$ used for fitting raw scans is as
follows:

$\displaystyle\mathcal{L}_{fitting}=\mathcal{L}_{comp}+\lambda_{reg\\_h}\mathcal{L}_{reg\\_h}+\lambda_{reg\\_o}\mathcal{L}_{reg\\_o}$
(26) $\displaystyle\mathcal{L}_{comp}=BCE(o_{comp},o_{unseen})$ (27)
$\displaystyle\mathcal{L}_{reg\\_h}=\lVert\mathbf{z}_{h}\rVert$ (28)
$\displaystyle\mathcal{L}_{reg\\_o}=\lVert\mathbf{z}_{o}\rVert,$ (29)

where $\lambda_{reg\\_h}=50$ and $\lambda_{reg\\_o}=50$. We optimize for 500
iterations using the Adam optimizer with a learning rate of 0.01 without any
weight decay or learning rate decay. Of 8 fitted outputs, the one with the
minimum bi-directional Chamfer distance to the target scan is chosen as the
final output.

##  Appendix E Addtional Qualitative Results

Please refer to the supplementary video for additional qualitative results on
individual control of the human and object modules, latent code interpolation,
and composition of multiple objects.