(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version
11institutetext: Meta Reality Labs
nsarafianos.github.io/garment3dgen

# Garment3DGen: 3D Garment Stylization and Texture Generation

Nikolaos Sarafianos  Tuur Stuyck  Xiaoyu Xiang  Yilei Li
Jovan Popovic  Rakesh Ranjan

###### Abstract

We introduce Garment3DGen a new method to synthesize 3D garment assets from a
base mesh given a single input image as guidance. Our proposed approach allows
users to generate 3D textured clothes based on both real and synthetic images,
such as those generated by text prompts. The generated assets can be directly
draped and simulated on human bodies. First, we leverage the recent progress
of image-to-3D diffusion methods to generate 3D garment geometries. However,
since these geometries cannot be utilized directly for downstream tasks, we
propose to use them as pseudo ground-truth and set up a mesh deformation
optimization procedure that deforms a base template mesh to match the
generated 3D target. Second, we introduce carefully designed losses that allow
the input base mesh to freely deform towards the desired target, yet preserve
mesh quality and topology such that they can be simulated. Finally, a texture
estimation module generates high-fidelity texture maps that are globally and
locally consistent and faithfully capture the input guidance, allowing us to
render the generated 3D assets. With Garment3DGen users can generate the
textured 3D garment of their choice without the need of artist intervention.
One can provide a textual prompt describing the garment they desire to
generate a simulation-ready 3D asset. We present a plethora of quantitative
and qualitative comparisons on various assets both real and generated and
provide use-cases of how one can generate simulation-ready 3D garments.

Figure 1: We present Garment3DGen, a fully-automated method to transform a
base garment mesh to simulation-ready asset directly from images or text
prompts. Our method enables rapid asset generation in a frictionless manner,
commoditizing content creation which would otherwise require specialized
software and expertise. We demonstrate applications such as physics-based
simulation and hand-cloth interaction in VR.

##  1 Introduction

3D asset creation is the process of designing and generating geometries and
materials for 3D experiences. It has direct applications across several
industries such as gaming, movies, fashion as well as VR applications.
Traditionally, simulation-ready garments are hard to obtain and are created
through a laborious time-consuming process requiring specialized software [11,
7, 64] relying on experienced artists. Currently, creating virtual clothing
for simulation is a challenging task. Garments need to be manually designed
and draped onto an underlying body. Additionally, the topology of the garment
needs to take simulation into consideration in order to enable pleasing
results. Low-friction asset creation will be the key enabler in the future to
unlock virtual applications at scale. Generative AI will be a cornerstone
technology that will allow anyone, ranging from novice users to experts, to
create customized avatars and to contribute to building personalized virtual
experiences. In addition, it will assist in the design process to facilitate
faster exploration and creation of new designs.

To tackle this task, we set out to develop a method termed _Garment3DGen_ that
creates 3D garments directly from image inputs. Given a base geometry mesh and
a single image, Garment3DGen performs topology-preserving mesh-based
deformations to match the image guidance and synthesizes new 3D assets on the
fly. Our generated garments comprise of posed geometries that stylistically
match the input image, and high-resolution texture maps. The provided image
guidance can be either from the real world or synthetically generated [56, 12]
which enables us to create both real and fantastical 3D garments.

While there are several methods in the literature that have focused on this
problem, they all suffer from some key limitations. One way to tackle this
problem would be to utilize recent image-to-3D techniques [37]. Given a single
image as input, such methods synthesize a specific number of views captured
from pre-set viewpoints and then employ multi-view reconstruction techniques
to obtain the 3D asset. However, the output geometries tend to be coarse and
lack fine-level details due to the use of Marching Cubes to extract the output
3D geometry. Another drawback is that the output garments are watertight, have
arbitrary scale making it challenging to drape them om human bodies and
simulate them directly as this would require manual intervention to post-
process the geometry (_e.g_., manually create arm/neck/waist holes for a
t-shirt geometry as well as re-meshing that would alter the mesh topology).
Alternatively, one could follow a NeRF-based approach where a handful of views
of the base mesh are utilized to train a NeRF which can be stylized in an
iterative manner [24]. Such an approach however does not guarantee multi-view
consistency of the newly stylized garment as the NeRF training and stylization
are happening in an iterative manner. In addition, it is a time-consuming
process and the final geometry is not simulation-ready. Recent works have
focused on 3D Gaussian Splatting [29] to generate 3D assets from image inputs.
While such methods are fast and of fairly high reconstruction quality, their
output splats are hard to be used for any downstream task besides rendering.
Another direction of research predicts 2D garment patterns [30, 33, 5, 44]
which can be optimized using differentiable simulation. Such approaches
generate simulation-ready garments but cannot generalize to fantastical AI-
generated garments and cannot benefit from the recent progress of text and
image-based diffusion models.

To this end, we carefully designed Garment3DGen to tackle each of the
aforementioned challenges: i) reconstruction-based approaches output
geometries that are watertight, coarse and the garments cannot be draped on
human bodies ii) deformation-based approaches are under-constrained when given
a single image/text prompt and their outputs do not faithfully match the
provided guidance and iii) simulation-based approaches fail to generalize to
new garment types. Garment3DGen is capable of producing high-quality
simulation-ready stylized garment assets complete with associated textures. We
approach this task from a mesh deformation-based perspective as we believe
that it provides better properties and more fine-grained control for the
output geometries compared to alternative NeRF-based or reconstruction-based
approaches. Mesh-based deformations can preserve the mesh topology which in
turn can allow for UV texture transfer, they can preserve the arm/body/head
holes of the garment geometry instead of outputting watertight meshes, and can
provide output meshes the triangles of which are not distorted and can be
draped on human bodies and simulated. Our method takes as inputs a single
image and a base template mesh and outputs a deformed mesh that faithfully
follows the image guidance while preserving the structure and topology of the
base mesh. Our first contribution stems from supervising the mesh deformation
process directly in the 3D space which directly allows physics-inspired losses
that ensure simulation-readiness instead of solely relying on image-based or
embedding-based supervisions [74, 46, 48]. In the absence of 3D ground-truth,
we build upon the progress of diffusion-based multi-view consistent image
generation to obtain a coarse 3D geometry that can serve as pseudo ground-
truth. However, strictly enforcing 3D supervisions using a coarse mesh would
result in deformed meshes that lack fine-level details and are not simulation-
ready. Thus, we utilize a pre-trained MetaCLIP model [74] since it provides a
balanced subset over the metadata distribution and finetune it to garment data
while introducing additional supervisions in the image space using
differentiable rendering and in the embedding space. Finally, we propose a
carefully designed texture estimation module to predict the texture maps
required to create the final 3D garment.

We conducted a plethora of experiments that demonstrate that our method can
generate 3D garment assets i) directly from images allowing a frictionless
experience where users indicate a requested garment by providing a reference
image and quickly obtain a corresponding high-quality 3D asset without manual
intervention and ii) from textual inputs describing both real and fantastical
garments, and iii) even simple garment sketches that one can quickly draw.
Moreover, we have developed a body-garment co-optimization framework that
enables us to scale and fit the garment to a parametric body model. This
allows us to animate the body model and perform physics-based cloth
simulation, resulting in a more accurate representation of the garment’s
behavior in various novel scenarios. In summary, our contributions are as
follows:

  * •

We propose a new approach for 3D geometry and texture generation for garments
given a base mesh and a single image guidance as inputs. We believe that this
is the first work aiming to generate textured garment assets that can be
useful for downstream simulation tasks.

  * •

We introduce geometry supervisions directly on the 3D space by generating
coarse-guidance meshes from the image inputs and use them as soft constraints
during the optimization. In addition, we provide valuable insights on the
impact of different losses that ensure that the output geometries are suitable
for downstream tasks such as cloth-simulation or hand-garment interaction.

  * •

We introduce a texture enhancement module that generates high-fidelity UV
textures from a single image allowing us to render the output geometries.

  * •

We propose a body-cloth optimization framework that fits the generated 3D
garment on a parametric body which unlocks applications such as dressing
avatars and the accurate simulation of clothes without artist intervention.

##  2 Related work

Garment Modeling: An important line of work is focused on designing[14, 6],
capturing [77], registering, reconstructing [54, 78], and representing [66,
39] clothes and their texture [8] from image or video inputs. Cloth
registration is an important task as it allows us to fit parametric garment
templates to in-the-wild scans or in controlled environments [23] and then use
the registered clothes for downstream tasks. [21] learns a shape diffusion-
based prior from captured 4D data in order to enable registration of texture-
less cloth, whereas [34] aligns the garment geometry to real world captures
using a coarse-to-fine method that leverages intrinsic manifold properties
with neural deformation fields. Aiming to accurately model clothes and
discover new compact ways to represent them, CaPhy [65] recovers a dynamic
neural model of clothing in a similar fashion to SNUG [58] by leveraging 3D
supervised training in combination with physics-based losses. [73] introduced
a physically-inspired appearance representation by learning view-dependent and
dynamic shadowing effects. Finally, recent methods have explored modeling
clothes using graph neural networks [19, 22, 50]. An alternative way to
represent clothes is via sewing patterns as this ensures an efficient
representation of developable [62, 57] and manufacturable garment items which
can be easily modified. A plethora of works [52, 30, 2, 4] have followed this
approach for garment reconstruction [36, 69, 68, 49], generation [59] and
draping [32].

Garment Deformation and Stylization: The recent progress in large language and
image-to-3D models has unlocked new ways of representing and reconstructing
dressed avatars [27, 70] from a single text or image prompt. These methods
typically generate a handful of multi-view consistent views [38, 72, 37, 53,
40] given a single image or text input or directly optimize a 3D scene [51]
using a 3D scene parameterization, similar to Neural Radiance Fields [47].
However such methods generate coarse, watertight meshes that in the context of
garments do not have the required topology and structure to be draped on
humans and simulated [63]. Editing and stylizing 3D surfaces has been explored
in the context of optimizing directly on the 3D space [61, 35, 28] and more
recently using triplanes [16] and text-to-mesh formulations [10, 48]. For
example, [46] performs mesh stylization by predicting color and local
geometric details that follow a text prompt. Deformation-based approaches [67,
26, 3, 71, 80, 17, 20, 76] can leverage these foundation models to enforce
supervision signals for text and image-based stylization [13] and manipulation
[18] of 3D meshes. Recently, a line of work [55, 9, 79, 75] applied text-to-
image generation models to create textures based on the mesh and given
text/image. Extending such techniques to clothes is a complex task as the
supervision signals of a single image or text-prompt are insufficient to
ensure that the deformed clothes will be simulation-ready.

Figure 2: Overview: Given an input 3D base mesh and a target garment image we
first generate 3D pseudo ground-truth using a diffusion-based method and
utilize the output geometry as a soft supervision signal during the
deformation process. Our 3D generated geometry preserves the topology and
structure of the base mesh as depicted by the colors of the sleeves/collar
while accurately capturing the geometry of the input image. The texture-
estimation module then outputs the corresponding UV texture that along with
the geometry comprise our final generated 3D garment.

##  3 Method

Our approach takes as input a single image $I$ and a base garment template
mesh $M_{\text{in}}$ and performs a topology-preserving deformation of the
input geometry given the image guidance to obtain the target deformed mesh
$M_{\text{def}}=D(I,M_{\text{in}})$ where $D$ is a function represented by a
neural network that optimizes over the input mesh. An overview of our method
is depicted in Fig. 2.

###  3.1 Target Geometry Generation

We propose to leverage the recent progress of single-image-to-3D methods to
obtain a coarse geometry of $I$ denoted by $M_{\text{guide}}(I)$ and use it as
much stronger supervision both directly in the 3D space as well as in
projected 2D space through differentiable rendering. A cross-domain diffusion
model [40] is employed which synthesizes RGB and normal images from six views
given the input image $I$ captured from the same predefined viewpoint. A
multi-view 3D reconstruction algorithm [83] is then utilized that, given the
generated views, it outputs a watertight, relatively coarse geometry
$M_{\text{guide}}(I)$ along with vertex colors of the garment in the input
image. While this mesh cannot serve as the final simulation-ready result due
to its poor mesh quality which is due to Marching Cubes [42] or potential
inaccuracies of the multi-view generation. Additionally, the fact that it is
watertight prevents us from draping the garment on a body (_e.g_. due to
missing armholes). Nonetheless, it provides useful information to serve as a
pseudo ground-truth that serves as a reference to deform $M_{\text{in}}$
towards. Hence, we update the optimization function as follows
$M_{\text{def}}=D(I,M_{\text{in}},M_{\text{guide}}(I))$. The alternative
approach would be to rely on the input image $I$ as the sole supervision for
mesh deformation which would result in a severely under-constrained
optimization with low-quality output meshes that are uncanny, over-deformed,
and they fail to capture the subtle details of the image guidance. For
example, starting from a template T-shirt geometry with guidance of the image
of an armor as input, one could extract CLIP embeddings for both $I$ and the
renders of $M_{\text{def}}$ following a similar approach to TextDeformer [18].
By enforcing supervisions on the embeddings, the goal is obtain an output mesh
that would resemble the requested armor. In practice, the supervision from the
embedding of a single exemplar image is not strong enough to produce high
quality output.

###  3.2 Mesh Deformer: Topology-Preserving Deformations

Aiming to preserve the structure and topology of the input base mesh while
still allowing for image-base stylizations, we propose an approach which
deforms $M_{\text{in}}$ instead of generating a new geometry that would be
hard to directly use in downstream tasks like reconstruction-based methods
would do. Hence, inspired by Neural Jacobian Fields [1], we parameterize
$M_{\text{in}}$ using a set of per-triangle Jacobians which define a
deformation. Following the same formulation, for simplicity we represent per-
triangle Jacobians as matrices $J_{i}\in R^{3x3}$ and solve a Poisson
optimization problem to obtain the deformation map $\Phi^{*}$ as the mapping
with Jacobian matrices for each triangle that are closest to $J_{i}$. More
formally this is represented as:

$\Phi^{*}=\min_{\Phi}\sum|t_{i}|\lVert\Phi\nabla_{i}^{T}-J_{i}\rVert^{2},$ (1)

where $\nabla(\Phi)$ denotes the Jacobian of $\Phi$ at triangle $t_{i}$, with
$|t_{i}|$ being the area of the triangle. We optimize the deformation mapping
$\Phi$ indirectly by optimizing the matrices $J_{i}$ which define $\Phi^{*}$.
These Jacobians are initialized to identity matrices. With the Jacobian
representation at hand, we optimize over the triangles of $M_{\text{in}}$ by
introducing a several losses each one of which addressing a specific issue.

3D Supervisions: We employ the one-directional Chamfer Distance (CD) loss to
evaluate the similarity between sets of points $p_{\text{def}}\in
S_{\text{def}}$ and $p_{\text{I}}\in S_{\text{I}}$ sampled randomly in each
iteration from $M_{\text{def}}$ and $M_{\text{guide}}(I)$. This is defined as:

$L_{\text{CD}}=\frac{1}{|S_{\text{def}}|}\sum_{p_{\text{def}}\in
S_{\text{def}}}\min_{p_{\text{I}}\in S_{\text{I}}}\lVert
p_{\text{def}}-p_{\text{I}}\rVert_{2}^{2}.$ (2)

Regularizations: We introduce several regularizations on the deformed 3D mesh
to ensure that it maintains key properties. First, we introduce Laplacian
smoothing [15] denoted by $L_{\text{Lap}}$ to redistribute vertex positions
based on the average positions of neighboring vertices. This smoothing process
helps to reduce irregularities and improves the overall shape of the mesh. To
produce simulation-ready meshes we penalize very small surface area for
triangles denoted by $L_{\text{triag}}$ as that would result in meshes that
are difficult to simulate. We do this by regularizing the edge length and by
minimizing the inverse of the squared sum of the triangle areas. Note that
there is a trade-off between how much a mesh can freely deform, (_e.g_. a
shirt becoming a medieval armor with spikes), and how much regularization it
requires such that the new garment can be placed on a parametric body and
simulated.

2D Supervisions: We utilize a rasterization-based differentiable renderer [31]
denoted by $R$ and pass both the deformed mesh $M_{\text{def}}$ in each
iteration and target pseudo ground-truth mesh $M_{\text{guide}}(I)$ to obtain
$K$ image renders:

${I_{\text{def}}}_{i}=R(M_{\text{def}},C_{i}),\;\;\;i=1\dots K,$ (3)

from randomly sampled camera views $C_{i}$. With ${I_{\text{I}}}_{i}$ computed
in a similar fashion for $M_{\text{guide}}(I)$ we employ the L1 loss between
the deformed and target renders:

$L_{\text{2D}}=\frac{1}{K}\sum_{i=1}^{K}\lVert{I_{\text{def}}}_{i}-{I_{\text{I}}}_{i}\rvert.$
(4)

This supervision in the 2D space captures well the silhouette of the garment
from multiple views as well as its fine-level details thereby enforcing the
deformed mesh to not deviate far from the target along each step of the
optimization.

Embedding Supervisions: We observe that passing garment images through a pre-
trained MetaCLIP model [74] results in deformed output garments that are
overly distorted, uncanny and fail to capture the fine-level details provided
in the input images. This is due to the weak supervision signal contained in
these embeddings. Likewise, this holds true for other mesh classes that are
not well represented in the data on which MetaCLIP was trained on, such as
humans. This is because MetaCLIP fails to capture the subtle differences
between different garments, their properties, and materials. To overcome this
limitation, we propose to use garment-specific embeddings obtained fine-tuned
on fashion data termed as FashionCLIP. The latent space for this model is
better tuned for fashion concepts and as a result, the embeddings provide a
stronger guidance for the deformations. We represent this loss as:

$L_{\text{E}}=\frac{1}{K}\sum_{i=1}^{K}\text{CosSim}\left(\text{FashionCLIP}\big{(}{I_{\text{def}}}_{i}\big{)},\text{FashionCLIP}\big{(}{I_{\text{I}}}_{i}\big{)}\right),$
(5)

where CosSim is the cosine similarity. This embedding loss acts as a soft
supervision signal between the embeddings of the deformed mesh
$M_{\text{def}}$ and those of the pseudo-ground-truth $M_{\text{guide}}(I)$.
This behavior is desired since we aim to benefit from the embedding
representations of the MetaCLIP model and sufficiently deform the base mesh
towards the target without capturing all of its shortcomings. For example we
need to preserve the arm/head holes of the base geometry while the target mesh
is watertight, or capture the fine-level details depicted in the input image
that $M_{\text{guide}}(I)$ might have failed to represent well. In summary,
the total loss $L_{\text{T}}$ is defined as follows where $w_{*}$ is the
corresponding weight for each loss:

$L_{\text{T}}=w_{\text{CD}}L_{\text{CD}}+w_{\text{Lap}}L_{\text{Lap}}+w_{\text{triag}}L_{\text{triag}}+w_{\text{2D}}L_{\text{2D}}+w_{\text{E}}L_{\text{E}},$
(6)

where the corresponding weights aim to strike a balance between utilizing the
3D pseudo ground-truth with the CD loss while capturing the finer image
details through the embedding and 2D losses. We provide additional details in
the supplementary material.

###  3.3 Texture Estimation

Given the untextured deformed 3D geometry $M_{\text{def}}$, we aim to generate
high-fidelity textures that match the input image. We leverage a 2D text-to-
image generation model to create high-quality, high-resolution textures with
vivid details given a text prompt. For garment generation starting from
images, a text prompt can automatically be obtained using image caption models
or by specifying them manually. Our pipeline consists of the following steps:
generating images of the given fashion asset from multiple views and
backprojecting these images onto the mesh surface to create UV texture
$T\in\mathbb{R}^{H\times W\times C}$. There are two challenges in adapting a
2D generation model to 3D objects: establishing geometry-texture
correspondence and ensuring multi-view consistency.

Shape-Aware Generation: To ensure the texture faithfully reflects the
underlying shapes, we propose using a depth-aware text-to-image generation
model. Given a set of camera poses $\mathcal{C}=\\{C_{i}\\}_{i=1}^{n}$, we
render the depth $D_{i}$ of each view from $M_{\text{def}}$, and sample the
appearance image $I_{i}$ of view $C_{i}$ conditioned on $D_{i}$ and the text
prompt using the text-to-image model.

Multi-view Consistency: Directly conducting view-by-view image synthesis
cannot guarantee that the generated views are consistent with each other. To
solve this problem, we first synthesize the front and back views
simultaneously to implicitly enforce global consistency. In addition, we
leverage the fact that the geometry of fashion assets is mostly flat, and
conduct depth-aware inpainting for the remaining views to ensure the newly
generated textures are locally consistent. To handle occluded areas, we design
an automatic view selection algorithm to inpaint the textures in a coarse-to-
fine manner: from the remaining unpainted area, we select the view with the
most unfilled pixels to generate the textures iteratively from large regions
to small pieces.

Texture Enhancement: The above texture generation pipeline can also be applied
to texture refinement: given a low-quality, low-resolution initial texture
$T^{LQ}$, we leverage the 2D appearance priors to further enhance the details.
We adopt SDEdit [45], which perturbs the above sampling procedure with
Gaussian noise and progressively denoises by simulating the reverse stochastic
differential equations. As a result, the low-quality $I_{i}^{LQ}$ is projected
onto the manifold of realistic images, yielding $I_{i}^{HQ}$. By
backprojecting these images, we acquire a high-quality texture $T^{HQ}$.

###  3.4 Fitting the Generated Garments to Parametric Bodies

While the aforementioned supervisions and regularizations aim to ensure that
the quality of the generated garment will be satisfactory enough to be
simulated, there are no guarantees that the garment shape, scale, pose and
orientation will be those required to drape it on a parametric body [41] and
simulate it. To accomplish this task, we run an optimization procedure during
which the generated garment remains fixed in the generated pose and the pose
and shape of the parametric body are transformed such that the garment can
accurately fit the body. This optimization process shown in Fig. 3, starts
with a rigid transformation and scaling of the body and continues with an
optimization of the body pose and shape using the Chamfer distance loss.
Finally, once the optimization has converged, we run an additional
optimization step penalizing body-cloth collisions, ensuring that the fit is
as accurate and realistic as possible.

Figure 3: 3D Garment Generation (left): Given an image or a text prompt as
guidance and a base geometry mesh (bottom left inset) that can be far from the
target we generate high-quality textured 3D geometries of both real as well as
fantastical garments. Fitting (right): We start with the generated textured 3D
garment (in this case a medieval armor) and a parametric body in its canonical
pose (left) and run the body-garment optimization process to optimize for the
body pose and shape parameters such that the generated garment can fit in the
body accurately without penetrations.

##  4 Experiments

Data: We rely on a handful of artist-created garment templates covering basic
cloth categories (_e.g_. T-shirt, shirt, tank-top, dress, etc). These
untextured 3D meshes are in a canonical pose and will be publicly released in
order to facilitate future 3D garment research. When it comes to images that
serve as a guidance, we collect a variety of real images with garments in
different poses, different textures, garments that do not exist in our mesh
library and even AI generated garments both real and fantastical generated
from a textual prompt.

Metrics: Quantitatively evaluating our results is a challenging task in the
absence of 2D or 3D ground-truth 2D for what are we trying to accomplish.
However we can evaluate how consistent the geometries are to the input image
and hence we render the untextured outputs of all methods from 36 views and
compute their perceptual scores using the LPIPS metric [82] as well as their
image-based CLIP similarity score using the cosine distance between their
embeddings.

Baselines: We evaluate our approach against: i) TextDeformer [18] which
deforms input meshes based on text prompts, ii) ImageDeformer - a variant of
TextDeformer we set up - where the input text is replaced with an image, iii)
Wonder3D [40] and iv) Zero123++[60] both of which generate 3D geometries given
a single image using 2D diffusion models, v) the newly introduced ZeroShape
[25] which performs zero-shot reconstruction and vi) a recent triplane-based
3D Gaussian Splatting approach [84]. The first two works also take as input a
base mesh whereas the latter four reconstruct the final result given a single
image as input.

Table 1: Quantitative Comparisons: Our approach outperforms deformation-based
(rows 1,2) and reconstruction-based (rows 3-6) methods across both metrics
while generating textured geometries that can be used for downstream tasks
which is not the case for any of the prior methods.

Method | CLIP-Sim$\uparrow$ | LPIPS $\downarrow$ | Faithful to Image | Colored Output | Head/Arm Holes
---|---|---|---|---|---
TextDeformer [18] | 0.51 | 0.42 |  |  |
ImageDeformer [18] | 0.54 | 0.41 |  |  |
Wonder3D [40] | 0.56 | 0.41 |  |  |
Zero123++ [60] | 0.52 | 0.42 |  |  |
ZeroShape [25] | 0.48 | 0.46 |  |  |
T-3DGS [84] | 0.57 | 0.41 |  |  |
Garment3DGen | 0.59 | 0.39 |  |  |

###  4.1 Image-to-3D Garments

Given an image prompt and a base mesh, Garment3DGen generates textured 3D
garments matching the input image guidance. We present a plethora of image-
to-3D results in Fig. 1, Fig. 3 and Fig. 6 and showcase that the generated 3D
assets can be of various topologies, respect the pose, shape and texture of
the input image, garments can be both real and fantastical, contain high-
quality texture maps, contain fine-level details such as cloth wrinkles and
folds and preserve the topology and structure of the input (_e.g_., head,
bottom and armholes are respected such that the garments can be draped on a
body). In addition we quantitatively evaluate Garment3DGen against several
baseline methods and report our results in Table 1. Our approach outperforms
all methods in terms of both embedding as well as perceptual similarity with
the input image guidance while it is the only approach that generates textured
geometries that can be used for downstream tasks. Deformation-based approaches
(rows 1-2) preserve the topology and the holes of the garment but lack strong
supervision signals to generate outputs that match the input image.
Reconstruction-based approaches (rows 3-6) are good at preserving and
reconstructing what is visible in the input image but generate unusable
geometries (or splats) and their color estimates for the non-visible regions
are usually blurry or single colored (as shown in Fig. 6).

###  4.2 Text-to-3D Garments

Our work can be extended to generating textured 3D garments given textual
prompt inputs. Unlike TextDeformer that utilizes a text-prompt we opt for a
text-to-image diffusion model as an initial step because image-based
supervisions can provide a stronger guidance for our mesh deformation process.
Such an approach allows the user to iterate with several text-prompts to
generate the image of their desired real or fantastical garment in contrast to
providing a text-prompt and waiting for the 3D asset creation process to
finish to see if the result matches their intent. In Fig. 3 (row 2) and Fig. 6
(rows 3-6) we present a variety of generated garments using only the provided
textual prompt as guidance.

Figure 4: Ablation Study: Starting from a base input mesh we showcase that our
key contributions result in deformed geometries that capture the input image
guidance, comprise fine-level garment details and are suitable for our
downstream tasks.

###  4.3 Ablation Studies

We conduct an ablation study to assess the impact of the key components of our
proposed approach. We start from a basic baseline approach and add components
one at a time and showcase our findings in Fig. 4. We start with the off-the-
shelf TextDeformer [18] where given a text prompt and a base-mesh we deform
the input geometry to match the target text using their framework and losses.
Text prompts are not ideal to capture the fine-level details of a garment as
there can be many “medieval armors” and in addition a pretrained MetaCLIP [74]
model is not capable of capturing the subtle differences between a “jacket”
and a “puffer jacket”. To overcome this limitation we adapt TextDeformer to
take image inputs as guidance (ImageDeformer) and observe that the deformed
geometries are one step closer to what we are after but still they fail to
capture the details of the image. By swapping out the original MetaCLIP model
and introducing a model fine-tuned on fashion data we observe that subtle
details are better preserved across garments. An image-based reconstruction
approach faithfully represents the geometry but generates coarse and
watertight meshes that are unusable for downstream tasks. We use these meshes
as pseudo ground-truth for our approach. Our Garment3DGen results in garments
that faithfully follow the image guidance while containing the wrinkles and
fine details. However the quality of the output geometry is not always ideal
for physics-based downstream tasks (_e.g_., cloth simulation) which is why we
introduced additional 3D supervisions that preserve a better mesh quality, see
Fig. 5. All prior works either do not follow the input image guidance or
generate low-quality geometries that cannot be used for downstream tasks.

Figure 5: Mesh Quality (Top) and Geometry Comparisons (Bottom): We showcase
the wireframes of all approaches. Our method stands out as the only one that
produces geometries that adhere to the input image while maintaining good mesh
quality and incorporating necessary holes for physics-based simulation tasks.
At the bottom we showcase the output geometry of various techniques to
highlight that our approach captures fine geometric details without geometric
artifacts (Wonder3D).

###  4.4 Applications & Discussion

Figure 6: Qualitative Comparisons: We demonstrate several mesh generation
methods starting from an input image shown on the left. We show front and back
views of each reconstruction. The 3D Gaussian Splatting [84] method generates
distorted frontal colors and dark or blurry back colors while its geometry is
not suitable for downstream tasks such as simulation. The second
reconstruction approach [40, 83] generates watertight meshes with very coarse
geometric details and blurred out colors. Our proposed approach outputs 3D
geometries that are geometrically correct with fine-level texture details that
prior works fail to generate.

Physics-based Cloth Simulation Automatically generated garments by
Garment3DGen are simulation-ready and can be used directly in downstream
applications, see Fig. 1, Fig. 7 and the supplemental material. We consider
the generated 3D shape as rest shape for our simulations and incorporate a
zero rest-angle dihedral energy to model the out-of-plane banding of the
fabric. Fantastical garments such as armors are not well modeled using a zero
rest-angle as they would lose their distinct shape during simulation and
wrinkle unnaturally. To obtain visually pleasing results, we take the rest
angle to be the 3D generated mesh one which allows it to maintain its shape
throughout the simulation.

Hand-Cloth Interaction in VR Our garments are suitable for real-time
simulations with hand-interaction using modern VR headsets. The rightmost part
of Fig. 1 as well as the last row of Fig. 7 show a user interacting with the
garments through real-time simulation with integrated hand-tracked
interactions.

Garment Resizing Generated garments can further be manipulated to achieve
distinct looks through garment resizing. Fig.1 shows both the generated dress
on the left as well as a larger version of it on the right, draped on the same
body.

Sketch to Garment Fig. 7 shows an application to generate 3D simulation-ready
garments starting from a rough sketch shown in the top left. Given a sketch,
we generate a realistic image using ControlNet [81] which then serves as the
input to our method to generate the corresponding 3D asset. We show simulation
results of this automatically generated dress.

Runtime: Our approach takes $\scriptstyle\mathtt{\sim}$24 mins on a single
TITAN RTX to generate the final 3D garment where $60\%$ of this time is
dedicated to the MeshDeformer (Sec.3.1), $20\%$ to the target geometry
generation (Sec.3.2), $18\%$ to the cloth-body fitting (Sec.3.4) and $2\%$ to
the texture estimation (Sec.3.3). Optimizing for speed was not among our
objectives but several low-hanging fruits (_e.g_., early stopping of the
deformation) exist and could be tackled to improve performance.

Limitations: Garment3DGen handles a variety of garment types both real and
fantastical. Due to the requirement of a template mesh, there is a limitation
to what garments can be generated whilst still providing good distortion-free
meshes. This can be mitigated by providing a more diverse template library.
Our estimated textures, while faithful to the image, sometimes do not fully
preserve fine-level details. We plan to address this by tuning the texture
enhancement module to be conditioned on the reference image across all views
while maintaining its multi-view color consistency properties.

Figure 7: Applications: Garment3DGen can generate textured 3D garments from
images, from text prompts, from simple sketches, that can be fitted to human
bodies and drive them with physics-based cloth simulation or even enable
interaction between hands and garments in a VR environment.

##  5 Conclusion

We proposed Garment3DGen a new approach to generate high-quality, physically
plausible garment assets that can be directly used for downstream
applications. We introduced a deformation-based approach that takes a base
mesh as input along with an image guidance and outputs a textured geometry
that faithfully matches the input image while preserving the structure and
topology of the input mesh. Our key contributions stem from utilizing novel
diffusion-based generative models to synthesize 3D pseudo ground-truth that
can be used as a soft supervision signal along with additional
regularizations, a texture enhancement module that generates high-fidelity
texture maps and a body-cloth optimization framework that fits the generated
3D garments to parameteric bodies. Our approach clearly outperforms prior work
across all metrics while producing physically plausible and high-quality
garment assets. Finally, we showcased several applications where the output
geometries of Garment3DGen are used for physics-based cloth simulation, hand-
garment interaction in a VR environment as well as going directly from a
simple sketch image to a drivable 3D garment.

## References

  * [1] Aigerman, N., Gupta, K., Kim, V.G., Chaudhuri, S., Saito, J., Groueix, T.: Neural jacobian fields: Learning intrinsic mappings of arbitrary meshes. ACM Trans. Graph. 41(4) (jul 2022)
  * [2] Bang, S., Korosteleva, M., Lee, S.H.: Estimating garment patterns from static scan data. In: Computer Graphics Forum. vol. 40, pp. 273–287. Wiley Online Library (2021)
  * [3] Baran, I., Vlasic, D., Grinspun, E., Popović, J.: Semantic deformation transfer. In: ACM SIGGRAPH 2009 papers. pp. 1–6 (2009)
  * [4] Bartle, A., Sheffer, A., Kim, V.G., Kaufman, D.M., Vining, N., Berthouzoz, F.: Physics-driven pattern adjustment for direct 3d garment editing. ACM Trans. Graph. 35(4), 50–1 (2016)
  * [5] Berthouzoz, F., Garg, A., Kaufman, D.M., Grinspun, E., Agrawala, M.: Parsing sewing patterns into 3d garments. Acm Transactions on Graphics (TOG) 32(4), 1–12 (2013)
  * [6] Brouet, R., Sheffer, A., Boissieux, L., Cani, M.P.: Design preserving garment transfer. ACM Transactions on Graphics 31(4), Article–No (2012)
  * [7] Browzwear: v-stitcher. https://browzwear.com/products/v-stitcher (2024), accessed on January 2024
  * [8] Chaudhuri, B., Sarafianos, N., Shapiro, L., Tung, T.: Semi-supervised synthesis of high-resolution editable textures for 3d humans. In: CVPR (2021)
  * [9] Chen, D.Z., Siddiqui, Y., Lee, H.Y., Tulyakov, S., Nießner, M.: Text2tex: Text-driven texture synthesis via diffusion models. arXiv preprint arXiv:2303.11396 (2023)
  * [10] Chen, K., Choy, C.B., Savva, M., Chang, A.X., Funkhouser, T., Savarese, S.: Text2shape: Generating shapes from natural language by learning joint embeddings. In: Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14. pp. 100–116. Springer (2019)
  * [11] CLO3D: Marvelous Designer. https://www.marvelousdesigner.com/ (2024), accessed on January 2024
  * [12] Dai, X., Hou, J., Ma, C.Y., Tsai, S., Wang, J., Wang, R., Zhang, P., Vandenhende, S., Wang, X., Dubey, A., et al.: Emu: Enhancing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807 (2023)
  * [13] Decatur, D., Lang, I., Aberman, K., Hanocka, R.: 3d paintbrush: Local stylization of 3d shapes with cascaded score distillation. arXiv preprint arXiv:2311.09571 (2023)
  * [14] Decaudin, P., Julius, D., Wither, J., Boissieux, L., Sheffer, A., Cani, M.P.: Virtual garments: A fully geometric approach for clothing design. In: Computer Graphics Forum. vol. 25, pp. 625–634. Wiley Online Library (2006)
  * [15] Field, D.A.: Laplacian smoothing and delaunay triangulations. Communications in applied numerical methods 4(6), 709–712 (1988)
  * [16] Frühstück, A., Sarafianos, N., Xu, Y., Wonka, P., Tung, T.: VIVE3D: Viewpoint-independent video editing using 3D-aware GANs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
  * [17] Gao, L., Yang, J., Qiao, Y.L., Lai, Y.K., Rosin, P.L., Xu, W., Xia, S.: Automatic unpaired shape deformation transfer. ACM Transactions on Graphics (ToG) 37(6), 1–15 (2018)
  * [18] Gao, W., Aigerman, N., Groueix, T., Kim, V., Hanocka, R.: Textdeformer: Geometry manipulation using text guidance. In: ACM SIGGRAPH 2023 Conference Proceedings. pp. 1–11 (2023)
  * [19] Grigorev, A., Black, M.J., Hilliges, O.: Hood: Hierarchical graphs for generalized modelling of clothing dynamics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16965–16974 (2023)
  * [20] Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: Unsupervised cycle-consistent deformation for shape matching. In: Computer Graphics Forum. vol. 38, pp. 123–133. Wiley Online Library (2019)
  * [21] Guo, J., Prada, F., Xiang, D., Romero, J., Wu, C., Park, H.S., Shiratori, T., Saito, S.: Diffusion shape prior for wrinkle-accurate cloth registration. In: 3DV (2023)
  * [22] Halimi, O., Larionov, E., Barzelay, Z., Herholz, P., Stuyck, T.: Physgraph: Physics-based integration using graph neural networks. arXiv preprint arXiv:2301.11841 (2023)
  * [23] Halimi, O., Stuyck, T., Xiang, D., Bagautdinov, T., Wen, H., Kimmel, R., Shiratori, T., Wu, C., Sheikh, Y., Prada, F.: Pattern-based cloth registration and sparse-view animation. ACM Transactions on Graphics (TOG) 41(6), 1–17 (2022)
  * [24] Haque, A., Tancik, M., Efros, A.A., Holynski, A., Kanazawa, A.: Instruct-nerf2nerf: Editing 3d scenes with instructions. arXiv preprint arXiv:2303.12789 (2023)
  * [25] Huang, Z., Stojanov, S., Thai, A., Jampani, V., Rehg, J.M.: Zeroshape: Regression-based zero-shot shape reconstruction. arXiv preprint arXiv:2312.14198 (2023)
  * [26] Jacobson, A., Baran, I., Popovic, J., Sorkine, O.: Bounded biharmonic weights for real-time deformation. ACM Trans. Graph. 30(4), 78 (2011)
  * [27] Jiang, Y., Yang, S., Qiu, H., Wu, W., Loy, C.C., Liu, Z.: Text2human: Text-driven controllable human image generation. ACM Transactions on Graphics (TOG) 41(4), 1–11 (2022)
  * [28] Jung, H., Nam, S., Sarafianos, N., Yoo, S., Sorkine-Hornung, A., Ranjan, R.: Geometry transfer for stylizing radiance fields. In: CVPR (2024)
  * [29] Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42(4) (2023)
  * [30] Korosteleva, M., Lee, S.H.: Neuraltailor: Reconstructing sewing pattern structures from 3d point clouds of garments. ACM Trans. Graph. 41(4) (2022)
  * [31] Laine, S., Hellsten, J., Karras, T., Seol, Y., Lehtinen, J., Aila, T.: Modular primitives for high-performance differentiable rendering. ACM Transactions on Graphics (TOG) 39(6), 1–14 (2020)
  * [32] Li, R., Guillard, B., Fua, P.: Isp: Multi-layered garment draping with implicit sewing patterns. In: NeurIPS (2023)
  * [33] Li, Y., yu Chen, H., Larionov, E., Sarafianos, N., Matusik, W., Stuyck, T.: Diffavatar: Simulation-ready garment optimization with differentiable simulation. In: CVPR (2024)
  * [34] Lin, S., Zhou, B., Zheng, Z., Zhang, H., Liu, Y.: Leveraging intrinsic properties for non-rigid garment alignment. In: ICCV (2023)
  * [35] Liu, H.T.D., Tao, M., Jacobson, A.: Paparazzi: surface editing by way of multi-view image processing. ACM Trans. Graph. 37(6), 221–1 (2018)
  * [36] Liu, L., Xu, X., Lin, Z., Liang, J., Yan, S.: Towards garment sewing pattern reconstruction from a single image. ACM Transactions on Graphics (TOG) 42(6), 1–15 (2023)
  * [37] Liu, M., Xu, C., Jin, H., Chen, L., Xu, Z., Su, H., et al.: One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization. arXiv preprint arXiv:2306.16928 (2023)
  * [38] Liu, Y., Lin, C., Zeng, Z., Long, X., Liu, L., Komura, T., Wang, W.: Syncdreamer: Generating multiview-consistent images from a single-view image. arXiv preprint arXiv:2309.03453 (2023)
  * [39] Liu, Z., Feng, Y., Xiu, Y., Liu, W., Paull, L., Black, M.J., Schölkopf, B.: Ghost on the shell: An expressive representation of general 3d shapes. arXiv preprint arXiv:2310.15168 (2023)
  * [40] Long, X., Guo, Y.C., Lin, C., Liu, Y., Dou, Z., Liu, L., Ma, Y., Zhang, S.H., Habermann, M., Theobalt, C., et al.: Wonder3d: Single image to 3d using cross-domain diffusion. arXiv preprint arXiv:2310.15008 (2023)
  * [41] Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 34(6), 248:1–248:16 (Oct 2015)
  * [42] Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. In: Seminal graphics: pioneering efforts that shaped the field. pp. 347–353 (1998)
  * [43] Macklin, M., Müller, M., Chentanez, N.: Xpbd: position-based simulation of compliant constrained dynamics. In: Proceedings of the 9th International Conference on Motion in Games. pp. 49–54 (2016)
  * [44] de Malefette, C., Qi, A., Parakkat, A.D., Cani, M.P., Igarashi, T.: Perfectdart: Automatic dart design for garment fitting. In: SIGGRAPH Asia 2023 Technical Communications. pp. 1–4 (2023)
  * [45] Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.Y., Ermon, S.: Sdedit: Guided image synthesis and editing with stochastic differential equations. In: International Conference on Learning Representations (2022)
  * [46] Michel, O., Bar-On, R., Liu, R., Benaim, S., Hanocka, R.: Text2mesh: Text-driven neural stylization for meshes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13492–13502 (2022)
  * [47] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)
  * [48] Mohammad Khalid, N., Xie, T., Belilovsky, E., Popa, T.: Clip-mesh: Generating textured meshes from text using pretrained image-text models. In: SIGGRAPH Asia 2022 conference papers. pp. 1–8 (2022)
  * [49] Pesavento, M., Xu, Y., Sarafianos, N., Maier, R., Wang, Z., Yao, C.H., Volino, M., Boyer, E., Hilton, A., Tung, T.: Anim: Accurate neural implicit model for human reconstruction from a single rgb-d image. In: CVPR (2024)
  * [50] Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., Battaglia, P.: Learning mesh-based simulation with graph networks. In: International Conference on Learning Representations (2021)
  * [51] Poole, B., Jain, A., Barron, J.T., Mildenhall, B.: Dreamfusion: Text-to-3d using 2d diffusion. arXiv (2022)
  * [52] Qi, A., Nag, S., Zhu, X., Shamir, A.: Personaltailor: Personalizing 2d pattern design from 3d garment point clouds. arXiv preprint arXiv:2303.09695 (2023)
  * [53] Qian, G., Mai, J., Hamdi, A., Ren, J., Siarohin, A., Li, B., Lee, H.Y., Skorokhodov, I., Wonka, P., Tulyakov, S., et al.: Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors. arXiv preprint arXiv:2306.17843 (2023)
  * [54] Qiu, L., Chen, G., Zhou, J., Xu, M., Wang, J., Han, X.: Rec-mv: Reconstructing 3d dynamic cloth from monocular videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4637–4646 (2023)
  * [55] Richardson, E., Metzer, G., Alaluf, Y., Giryes, R., Cohen-Or, D.: Texture: Text-guided texturing of 3d shapes. arXiv preprint arXiv:2302.01721 (2023)
  * [56] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)
  * [57] Rose, K., Sheffer, A., Wither, J., Cani, M.P., Thibert, B.: Developable surfaces from arbitrary sketched boundaries. In: SGP’07-5th Eurographics Symposium on Geometry Processing. pp. 163–172. Eurographics Association (2007)
  * [58] Santesteban, I., Otaduy, M.A., Casas, D.: Snug: Self-supervised neural dynamic garments. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8140–8150 (2022)
  * [59] Shen, Y., Liang, J., Lin, M.C.: Gan-based garment generation using sewing pattern images. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16. pp. 225–247. Springer (2020)
  * [60] Shi, R., Chen, H., Zhang, Z., Liu, M., Xu, C., Wei, X., Chen, L., Zeng, C., Su, H.: Zero123++: a single image to consistent multi-view diffusion base model (2023)
  * [61] Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., Seidel, H.P.: Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing. pp. 175–184 (2004)
  * [62] Stein, O., Grinspun, E., Crane, K.: Developability of triangle meshes. ACM Transactions on Graphics (TOG) 37(4), 1–14 (2018)
  * [63] Stuyck, T.: Cloth simulation for computer graphics. Springer Nature (2022)
  * [64] Style3D: Style3D. https://www.linctex.com/ (2024), accessed on January 2024
  * [65] Su, Z., Hu, L., Lin, S., Zhang, H., Zhang, S., Thies, J., Liu, Y.: Caphy: Capturing physical properties for animatable human avatars. In: ICCV (2023)
  * [66] Su, Z., Yu, T., Wang, Y., Liu, Y.: Deepcloth: Neural garment representation for shape and style editing. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(2), 1581–1593 (2023)
  * [67] Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Transactions on graphics (TOG) 23(3), 399–405 (2004)
  * [68] Tiwari, G., Antic, D., Lenssen, J.E., Sarafianos, N., Tung, T., Pons-Moll, G.: Pose-ndf: Modeling human pose manifolds with neural distance fields. In: European Conference on Computer Vision (ECCV) (October 2022)
  * [69] Wang, A., Xu, Y., Sarafianos, N., Maier, R., Boyer, E., Yuille, A., Tung, T.: HISR: Hybrid implicit surface representation for photorealistic 3d human reconstruction. In: AAAI (2024)
  * [70] Wang, J., Liu, Y., Dou, Z., Yu, Z., Liang, Y., Li, X., Wang, W., Xie, R., Song, L.: Disentangled clothed avatar generation from text descriptions. arXiv preprint arXiv:2312.05295 (2023)
  * [71] Wang, Y., Jacobson, A., Barbič, J., Kavan, L.: Linear subspace design for real-time shape deformation. ACM Transactions on Graphics (TOG) 34(4), 1–11 (2015)
  * [72] Weng, H., Yang, T., Wang, J., Li, Y., Zhang, T., Chen, C., Zhang, L.: Consistent123: Improve consistency for one image to 3d object synthesis. arXiv preprint arXiv:2310.08092 (2023)
  * [73] Xiang, D., Bagautdinov, T., Stuyck, T., Prada, F., Romero, J., Xu, W., Saito, S., Guo, J., Smith, B., Shiratori, T., et al.: Dressing avatars: Deep photorealistic appearance for physically simulated clothing. ACM Transactions on Graphics (TOG) 41(6), 1–15 (2022)
  * [74] Xu, H., Xie, S., Tan, X.E., Huang, P.Y., Howes, R., Sharma, V., Li, S.W., Ghosh, G., Zettlemoyer, L., Feichtenhofer, C.: Demystifying clip data. In: ICLR (2024)
  * [75] Yeh, Y.Y., Huang, J.B., Kim, C., Xiao, L., Nguyen-Phuoc, T., Khan, N., Zhang, C., Chandraker, M., Marshall, C.S., Dong, Z., et al.: Texturedreamer: Image-guided texture synthesis through geometry-aware diffusion. arXiv preprint arXiv:2401.09416 (2024)
  * [76] Yifan, W., Aigerman, N., Kim, V.G., Chaudhuri, S., Sorkine-Hornung, O.: Neural cages for detail-preserving 3d deformations. In: CVPR (2020)
  * [77] Yu, T., Zheng, Z., Zhong, Y., Zhao, J., Dai, Q., Pons-Moll, G., Liu, Y.: Simulcap: Single-view human performance capture with cloth simulation. In: CVPR (2019)
  * [78] Yu, Z., Dou, Z., Long, X., Lin, C., Li, Z., Liu, Y., Müller, N., Komura, T., Habermann, M., Theobalt, C., et al.: Surf-d: High-quality surface generation for arbitrary topologies using diffusion models. arXiv preprint arXiv:2311.17050 (2023)
  * [79] Zeng, X.: Paint3d: Paint anything 3d with lighting-less texture diffusion models. arXiv preprint arXiv:2312.13913 (2023)
  * [80] Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., Van Kaick, O., Tagliasacchi, A.: Deformation-driven shape correspondence. In: Computer Graphics Forum. vol. 27, pp. 1431–1439. Wiley Online Library (2008)
  * [81] Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3836–3847 (2023)
  * [82] Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018)
  * [83] Zhao, F., Jiang, Y., Yao, K., Zhang, J., Wang, L., Dai, H., Zhong, Y., Zhang, Y., Wu, M., Xu, L., Yu, J.: Human performance modeling and rendering via neural animated mesh. ACM Trans. Graph. 41(6) (nov 2022)
  * [84] Zou, Z.X., Yu, Z., Guo, Y.C., Li, Y., Liang, D., Cao, Y.P., Zhang, S.H.: Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. arXiv preprint arXiv:2312.09147 (2023)