#  Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text

Junshu Tang1222Work done when interning at Shanghai AI Lab. Yanhong Zeng2 Ke
Fan1 Xuheng Wang3
Bo Dai2 Kai Chen2333Corresponding authors. Lizhuang Ma1333Corresponding
authors.
1 Shanghai Jiao Tong University 2 Shanghai AI Lab 3 Tsinghua University
https://make-it-vivid.github.io/

###### Abstract

Creating and animating 3D biped cartoon characters is crucial and valuable in
various applications. Compared with geometry, the diverse texture design plays
an important role in making 3D biped cartoon characters vivid and charming.
Therefore, we focus on automatic texture design for cartoon characters based
on input instructions. This is challenging for domain-specific requirements
and a lack of high-quality data. To address this challenge, we propose Make-
It-Vivid, the first attempt to enable high-quality texture generation from
text in UV space. We prepare a detailed text-texture paired data for 3D
characters by using vision-question-answering agents. Then we customize a
pretrained text-to-image model to generate texture map with template structure
while preserving the natural 2D image knowledge. Furthermore, to enhance fine-
grained details, we propose a novel adversarial learning scheme to shorten the
domain gap between original dataset and realistic texture domain. Extensive
experiments show that our approach outperforms current texture generation
methods, resulting in efficient character texturing and faithful generation
with prompts. Besides, we showcase various applications such as out of domain
generation and texture stylization. We also provide an efficient generation
system for automatic text-guided textured character generation and animation.

##  1 Introduction

3D biped cartoon characters [29] breathe life into fictional characters,
conveying actions, and storytelling elements engagingly. These characters find
applications in various domains, including video games (_e.g_., Animal
Crossing [1]), movies (_e.g_., Zootopia [59]), and the upcoming metaverse.
Yet, the creation and animation of these characters heavily rely on skilled
artists utilizing specialized software, making it a labor-intensive and time-
consuming process.

Figure 1: We present Make-it-Vivid, the first attempt that can create
plausible and consistent texture in UV space for 3D biped cartoon characters
from text input within few seconds. Make-it-vivid enables texture generation
with fine-grained details in multiple styles (see above), and also supports
efficient text-guided animatable textured character production (see bottom).

Compared to the shape of 3D biped cartoon characters, their textures exhibit a
significantly higher level of diversity, playing a crucial role in creating
vivid and charming characters. This work focuses on the automatic design of
high-quality textured characters by generating textures based on text
descriptions, which presents two main challenges. (1) Demanding domain-
specific requirements. Simply dressing 3D biped cartoon characters with
appropriate textures is insufficient to make them attractive. These characters
require textures that possess unique traits, including semantic harmony,
consistent global configuration, and rich local high-frequency details.
Consequently, conventional shape texturing methods are inadequate for cartoon
characters, often resulting in textures with smooth and blurry details, as
well as noticeable seam artifacts [31, 8, 30, 22, 42, 7, 6]. (2) Limited
availability of high-quality data. The scarcity of high-quality data that
meets the demanding requirements further complicates the task. The creation of
high-quality cartoon characters involves a costly and skill-intensive process,
resulting in limited availability of such data. Additionally, due to
intellectual property (IP) concerns, these data are often kept private, making
it impractical to gather them from publicly accessible sources on the
Internet. Existing datasets [29] that include character texture data also
suffer from significant limitations in terms of high-frequency details, inter-
instance variations, and paired text descriptions.

In this work, we introduce Make-It-Vivid, a novel texture generation framework
specifically designed for 3D biped cartoon characters. Our framework enables
the generation of diverse, high-fidelity, and visually compelling textures in
a single forward pass, given text input. To address the challenge of limited
high-quality data, we propose marrying a knowledgeable pre-trained text-to-
image (T2I) diffusion model with a topology-aware representation of the UV
space, making Make-It-Vivid the first framework to leverage diffusion priors
in the UV space for 3D biped cartoon characters. We start by developing a
specialized multi-agent-based captioning system tailored for 3D biped
characters. By utilizing vision-question-answering agents, we can easily
generate high-quality descriptions of color, clothing, and character types
based on rendered frontal views for the UV maps. This process results in a
dataset of high-quality text-UVMap pairs. Once the dataset is prepared, we
customize the pre-trained T2I model to generate high-quality UV maps. This
customization involves introducing learnable parameters and fine-tuning them
on the paired text-UVMap data while keeping the T2I model fixed to retain its
open-domain knowledge. This design allows our framework to seamlessly
integrate with various customized T2I style models, such as Shuimo style [34]
and American comics, for creative texturing.

While the customized diffusion model generates various plausible textures
faithful to text prompts, the texture quality often suffers from over-
smoothing, making it challenging to meet demanding domain-specific
requirements. This limitation can be attributed to the lack of high-frequency
details in the training data. Therefore, we create high-quality images using a
T2I model as a proxy for high-frequency details and innovatively introduce
adversarial training [14] into the diffusion training process, leading to
enhanced texture details. We extensively evaluate the performance of Make-It-
Vivid, demonstrating its superiority in texturing 3D biped characters. Our
main contributions are as follows:

  * •

We present Make-It-Vivid, which empowers non-expert users to effortlessly
customize vivid 3D textured characters with desired identities, styles, and
attributes.

  * •

To overcome the limitation of training data, we are the first to introduce
adversarial training into the diffusion training process, achieving improved
image fidelity.

  * •

We showcase the versatility of our approach by exploring captivating
applications in stylized generation and multi-modality textured character
animation.

##  2 Related Work

3D Generation under Text Guidance. Recent advancements in image generation
[41, 40, 18, 54, 43, 38, 44, 9, 10] have greatly boosted the research progress
in 3D assets generation [17, 39, 50, 26, 8, 51] under text guidance. A set of
works [17, 39, 51, 31, 46] propose to generate 3D shapes by optimizing a NeRF
representation [2, 35], either through CLIP guidance [17] or Score
Distillation Sampling [39] and Variational Score Distillation [51] via 2D
diffusion models. Though effective, implicit representations such as NeRF can
be infeasible to be deployed for most practical applications [48, 23].
Subsequent methods [26, 48, 33] tackle the above problem by directly
generating highly realistic 3D meshes from textual prompts. Specifically,
Magic3D [26] presents a two-stage optimization framework to address the
efficiency and resolution problems observed in NeRF-based models, while
TextMesh [48] employs an SDF backbone to extract realistic-looking 3D meshes.
There are also a number of works trying to address the different aspects of 3D
content generation. For example, Fantasia3D [8] utilizes a hybrid
representation of 3D shape, namely DMTet [45], and decouples the problem to
geometry and appearance modeling. Point·E [36] presents an alternative
approach to 3D content generation by utilizing a point cloud diffusion model.
Latent-NeRF [31] optimizes SDS loss in Stable Diffusion’s latent space to
allow increased control over the generation process. Besides, [31] also
presents Latent-Paint, which optimizes neural texture maps based on the input
mesh.

3D Mesh Texturing under Text Guidance. In addition to generate fully textured
shapes, texture generation based on the given 3D geometry has recently gained
significant popularity. A number of works [8, 32, 30, 22, 39, 31] approach
this task by optimizing an implicit representation of both 3D geometry and
texture. For example, Text2Mesh [32], Tango [22] and XMesh [30] innovate 3D
mesh texturing by optimizing the color and displacement for each vertex on the
base mesh based on corresponding text prompt using CLIP loss [40]. While other
methods [39, 49, 26, 31, 8, 51] leverage SDS loss which helps create high-
fidelity and realistic texture. Another set of works introduce an iterative
painting scheme to paint a given 3D model from different view points [42, 7,
6]. These methods synthesize multi-view textures based on observations under
different viewpoints, and use depth-aware texture generation and inpainting to
refine the new unpainted areas while preserving consistent texture from the
partially painted area. Albeit improving results, these works still suffer
from severe inconsistencies across multiple views and seam artifacts due to
their inpainting nature. Furthermore, some of generative models focus on
generating high-fidelity UV textures [12, 13, 21, 19, 20, 55, 53] directly,
which shows impressive quality in 3D face reconstruction and generation. In
this paper, we explore UV texture generation on a more challenging but crucial
scenario on vivid texture generation.

Figure 2: Multi-rounds of dialogue for captioning 3D characters. For each
rendered image, we hard code three questions and use ChatGPT for asking
follow-up three questions, then summarize.

##  3 Preliminary

Parametric models of shapes, like 3DMM [4], FLAME [25] and SMPL [27], are
widely used in computer graphics, computer vision, and other related fields.
These models are designed to represent the 3D shape and appearance of complex
objects, such as human bodies and faces, in a compact and expressive manner.

In order to represent 3D cartoon biped characters with a consistent topology,
we adopt a parametric model Rabit [29] which contains a linear blend model for
shapes. Specifically, the 3D biped character is parameterized as
$M=F(B,\Theta,Z)$, where $B$ denotes the identity-related body parameter,
$\Theta$ represents the non-rigid pose-related parameter, and $Z$ represents
the latent embedding of texture.

In detail, the generated character shape is defined as
$M_{S}=\bar{M}_{S}+\sum_{i}^{|B|}\beta_{i}s_{i}$, where the mean shape is
denoted as $\bar{M}_{S}$ and $|B|$ denotes the dimension of shape
coefficients. $s_{i}\in\mathbb{R}^{3*N}$ denotes the orthogonal principal
components of vertex displacements of the geometry shape. The coefficients
models the variants of shapes under different identities. Besides, the shape
of eyeballs can be calculated based on the predefined landmarks.

The pose parameters
$\Theta=[\theta_{1},\theta_{2},...,\theta_{K}]\in\mathbb{R}^{69}$ denotes the
axis-angle of the relative rotation of joint $k$ with respect to its parents.
$K=23$ denotes the number of the joints. Each parameter $\theta_{k}$ can be
converted to the rotation matrix using Rodrigues’ formular:

$\displaystyle v_{i}^{\prime}$
$\displaystyle=\sum^{K}_{k=1}w_{k,i}G_{k}^{\prime}(\theta,J)v_{i},$ (1)
$\displaystyle G_{k}^{\prime}(\Theta,J)$
$\displaystyle=G_{k}(\Theta,J)G_{k}(\Theta^{\prime},J)^{-1},$ $\displaystyle
G_{k}(\Theta,J)$ $\displaystyle=\prod_{j\in
A(k)}[\begin{array}[]{cc}R(\theta_{j})&J_{j}\\\ 0&1\end{array}],$

where $w_{k,j}$ denotes the skinning weight for the $i$-th vertex.
$G_{k}(\Theta,J)$ is the global transformation of joint $k$. $J_{j}$ denotes
the location of the $j$-th joint. We use this representation to animate the
mesh using specific pose parameters $\Theta$.

As for parametric texture embedding, Rabit uses a StyleGAN2-based [18]
generator for embedding texture map to latent codes. Specifically, the texture
image $T\in\mathbb{R}^{H\times W\times 3}$ is generated by a latent code
$Z\in\mathbb{R}^{d}$ where the resolution is $1024$ and the dimension $d=512$.
However, it can only generate textures unconditionally with low quality. In
this paper, we propose a new text-driven vivid texture generator which is
editable with multiple concepts, such as color, clothes, style and so on.

Figure 3: Overall framework for training texture generator. Our method takes a
pair of data as input including a texture map $T$, corresponding text
description $P$ and mesh model $\mathcal{M}$. We finetune the low-rank adaptor
$\Delta\theta$ for pretrained text-to-image diffusion model to generate high
quality UV texture. In order to improve the quality and perceptual fidelity of
synthetic textures, we introduce adversarial training to enhance the texture
details. We leverage synthetic plausible images $I_{v}$ conditioned by the
rendered depth $I^{d}_{v}$ generated by ControlNet $\mathcal{C}$ as a proxy to
guide this adversarial training.

##  4 Text-guided UV Texture Generation

Texturing on the non-rigid cartoon character under simple instructions is
crucial yet inherently challenging. We therefore propose the first attempt to
prioritize the generation of texture maps using UV unwrapping, a consistent
and essential representation for mesh textures in the traditional computer
graphics pipeline. Drawing inspiration from the achievements in image
synthesis through text-conditional diffusion models, we have adopted a
diffusion model for generating textures from random noise conditioned by text.

To this end, given a user prompt $P$, our method is able to generate a vivid
and consistent texture map aware of the definition of the correspondence
between UV space and the geometry. Inspired by the appealing results from the
pretrained latent diffusion models, which is trained on large and diverse
text-image pairs, we leverage these knowledge as semantic priors for our
texture generation. To enforce the texture specifications and meanwhile
preserve the generating ability, we train our texture generator by fine-tuning
on the pretrained LDM.

We train our model on the 3DBiCar dataset for its topology consistent mesh
model and diversity of cartoon identities. Our training samples consist of a
3D mesh, a geometry related UV texture map and a corresponding description.
Since the detail description is not supported, we first propose a multi-agent
character captioning pipeline for generating detail caption of each 3D
textured model.

###  4.1 Multi-agent Character Captioning

In this section, we focus on building a text-texture paired data for training
the texture generator. In order to obtain the detailed description of the 3D
model, inspired with [58], we propose a 3D caption pipeline which focuses on
identity and texture information. The illustration of the caption pipeline is
shown on Fig 2. Specifically, for each model, we firstly render multi-view
images for each textured model and use Visual Question Answering(VQA) model,
BLIP2 [24], to obtain detailed corresponding descriptions.

Similar with [57], we hard-code the first question as: 1) “Describe the image
in detail” to let BLIP2 generated a brief initial description of the image.
Then, in order to obtain more information about the detail attribute of the
color and cloth types, we start with another two specific questions to
encourage the attention: 2) “What is the color of this object?”. 3) “What is
the object wearing?”.

Furthermore, in order to enrich image captions and generate more informative
descriptions, we integrate strong vision-language model, ChatGPT [37], for
asking relevant questions according to the previous knowledge and
progressively obtain more information. ChatGPT is prompted to ask follow-up
questions to investigate more information about the image. Besides, in order
to avoid the caption model for generating pose ore action-related information,
we deign a head instruction of BLIP2 including: “Answer given questions. Don’t
answer any contents about the pose or action of the object.”

At last, we use ChatGPT to summarize the descriptions across multi-views and
result in the final caption. ChatGPT is able to merge the similar detail
information in multi-views and remove the unlikely ones. We use the final
caption as the detailed prompt of the 3D model and 3D texture for subsequent
training.

###  4.2 Enhanced UV Texture Generation from Text

Now we use the prepared data for vivid and high-quality UV texture generation.
Each pair of data includes a mesh model $\mathcal{M}$, a texture map
$T\in\mathbb{R}^{H\times W\times 3}$ and a corresponding caption $P$. To
ensure the generation of similar patterns or templates of the UV map within
the dataset while preserving the generating capabilities, we customize
specific parameters of pretrained text-to-image diffusion model. This
customization allows us to leverage image knowledge as semantic priors in our
texture generation process. We first start with a simple baseline that a
parameter-efficient finetuning, Low-Rank Adaptation (LoRA), on the U-Net of
the pretrained latent diffusion model (LDM) [43]. We encode the input texture
$T$ into latent $x_{0}$ and achieve diffusion process. The objective of the
training is:

$\displaystyle\mathcal{L}_{diff}=\mathbb{E}_{\epsilon,x_{0},t}[||\epsilon-\epsilon_{\theta+\Delta\theta}(\sqrt{\alpha}_{t}x_{0}+\sqrt{1-\alpha_{t}}\epsilon)||^{2}_{2}].$
(2)

$\Delta\theta$ denotes the tuned parameters, $\epsilon$ denotes the random
noise map, $\epsilon_{\theta+\Delta\theta}(\cdot)$ is the predicted noise
generated by denoiser integrated with LoRA adapter, $\alpha_{t}$ is the
parameter of noise scheduler at timestamp $t$.

After fine-tuning, we can infer the LDM and generate plausible texture map
results related with the text instructions. However, simple fine-tuning on the
collected dataset can only achieve UV-related texture map in the same domain
with the dataset, result in limit concept and style variations. We show some
synthetic results in Fig 4 related to the text “overall”. We first show some
selected texture maps from the dataset in (a), the selected results generated
by the text generator of [29] in (b) and the synthetic results generated by
the simple LDM in in (c). It is obvious that a simple finetuning of LDM tend
to synthesis the structure of the overall without fine-grained details such as
cloth wrinkles.

Figure 4: We showcase texture samples related with prompt “wearing overall”
(a) from 3DBiCar [29] dataset; (b) generated by the texture generator from
Rabit [29]; (c) generated by the simple finetuned LDM; (d) generated by the
enhanced fintuned LDM.

Therefore, bring the original texture domain to the realistic domain is
necessary as it ensures the perceptual realism of the textured 3D model. To
fix the domain issue and boost the quality of texture with fine-grained local
details, we need a set of realistic texture image to represent the realistic
distribution. However, it is hard to build or collect UV texture data that
meets the requirements and related with the character geometry. Hence, we turn
to create vivid synthetic images that looks like object renderings from
different viewpoints.

Specifically, we apply a depth-guided image generator of ControlNet [56] to
produce multi-view images guided by the rendered depth. Then we propose to
impose an adversarial loss simultaneously when fine-tuning the parameters of
the adapter. At each iteration, we randomly sample a camera view $v$ from the
pre-defined view set $\mathcal{V}$ and render the input mesh $\mathcal{M}$ to
multi-view depth images. ControlNet receives the depth image $I^{d}_{v}$ the
text prompt $P$ corresponding with the object, and, in response, synthesis
high-quality images: $I_{v}=\mathcal{C}(I^{d}_{v},y)$, where $y$ denotes the
text embedding of $P$. In our case, we set number of renderings for each 3D
characters $|\mathcal{V}|=8$. As for generated sample, we randomly sample a
timestamp $t\in(0,1000)$ and achieve diffusion process. Then we use the
pretrained decoder to decode the denoised latent
$\hat{x}_{0}^{t}=(x_{t}-\sqrt{1-\alpha_{t}}\epsilon_{\theta+\Delta\theta}(x_{t},y,t))/\sqrt{\alpha_{t}}$
to the image $\hat{T}_{0}^{t}$. Then we use a differentiable mesh renderer
$\mathcal{R}$ to render the textured mesh with texture $\hat{T}_{0}^{t}$ at
view $v$. The render output is denoted as $\mathcal{R}(\hat{T}_{0}^{t},v)$.
And then we adopt adversary loss to make the rendered image
$\mathcal{R}(\hat{T}_{0}^{t},v)$ has the similar local structure and
perceptual realism with the generated 2D images $I_{v}$ at the same view. The
objective of the adversarial training can be formulated as:

$\displaystyle\mathcal{L}_{adv}=\mathbb{E}_{t,x_{0},\epsilon}[logD(\mathcal{R}(\hat{T}_{0}^{t},v)]+\mathbb{E}_{x_{0}}[log(1-D(I_{v}))],$
(3)

where the rendered image $\mathcal{R}(\hat{T}_{0}^{t},v)$ is considered as a
fake image while the output of the ControlNet is considered as the real
sample. $D$ is the adversarial discriminator that tries to maximize
$\mathcal{L}_{adv}$. The boost texture results is shown on Fig 4(d). We can
see that after the adversarial training, the network is able to generate more
realistic texture details.

Texture seam fixing. When texturing the 3D model with synthetic UV texture
image, we find that using image generative model would inevitably ignore the
consistency at the seam of the 3D model and results in black seam artifacts.
This might due to the reason that each processed texture data is agnostic to
the whole perspective 3D knowledge. To help fix this issue , we first apply a
Gaussian filter around the boundary part of the texture image, which will
remove the ”black seam” on the back of the model. However, this cannot solve
the misalignment at the boundary. Therefore, we also conduct a simple image
restoration technique on the back view of the model to mitigate the problem.

Specifically, for the generated texture map $x_{0}$, we render the textured
mesh using the renderer $\mathcal{R}$ to obtain the rendered image $T_{v}$ as
seen from the back view. Then we apply a state-of-the-art image restoration
method [52] to help make the rendered view perceptualy realistic without seam
artifacts. Then we back project the $I_{v}$ to the updated texture.

##  5 Experiments

Figure 5: Qualitative comparison on the test prompt set with state-of-the-art
shape texturing approaches, TEXTure [42], Text2Tex [7], LatentPaint [31],
Fantasia3D [8] and Rabit [29], We show our results with high-quality and
consistent texture faithful to the input prompt.

###  5.1 Dataset

Our model is trained on 3DBiCar [29] dataset. 3DBiCar spans a wide range of 3D
biped cartoon characters, containing 1,500 high-quality 3D models. The 3D
cartoon characters have diverse identities and shape resulting in 15 character
species, including Human, Bear, Mouse, Cat, Tiger, Dog, Rabbit, Monkey,
Elephant, Fox, Pig, Deer, Hippo, Cattle and Sheep. All the 3D models are
rigged and skinned by the predefined skeleton and skinning weight matrix,
which supports further animation. Note that eyeball meshes and textures are
extra modeled to support the facial expression in the future better. In our
experiments, we use the default texture for eyeballs.

###  5.2 Implementation Details

Our texture generator is fine-tuned based on the cutting edge open source
model Stable Diffusion [43] version 1.5. We inject LoRA into the projection
matrices of query, key and value in all of the attention modules. We set the
rank of the LoRA to 8. Then the modified forward pass of input $x$ is:
$h=W_{0}x+B_{uv}A_{uv}x$, where $B_{uv}A_{uv}$ denote the parameters of
adapter. We fine-tune the adapter using the AdamW [28] with a learning rate
$1e-4$. For inference, we use classifier-free guidance with a guidance weight
$\omega:\hat{\epsilon}_{\phi}(x_{t};y,t)=(1+\omega)\epsilon_{\phi}(x_{t};y,t)-\omega\epsilon_{\phi}(x_{t};t)$.
In our experiments, we set $\omega=7.5$. All the training and inference are
performed on a single NVIDIA A100 GPU.

|  CLIP$\uparrow$ |  Time(min)$\downarrow$ |  GPU(GB)$\downarrow$
---|---|---|---
LatentPaint [31] | 27.15 | 13.95 | 11.46
Fantasia3D [8] | 29.20 | 21.55 | 12.42
Text2Tex [7] | 28.81 | 14.35 | 20.31
TEXTure [42] | 29.25 | 2.38 | 12.05
Rabit [29] | - | 0.01 | 2.39
Ours | 29.86 | 0.03 | 6.20
Table 1: Quantitative comparison on the test prompt set with other state-of-the-art texturing method. We also report the inference time for a single prompt and GPU memory to show our efficiency. |  FID$\downarrow$ |  KID($\times 10^{-3}$)$\downarrow$
---|---|---
Rabit [29] | 42.55 | 6.37
Ours | 35.25 | 5.25
Table 2: Quantitative comparison on the 3DBiCar dataset. Since results of
other approaches have a large domain gap with the original texture dataset, so
we only compare with Rabit [29] trained on the same dataset.

###  5.3 Comparison with State-of-the-art Approach

To the best of our knowledge, we are the first method focusing on texture
creation in UV space of 3D biped cartoon model under text guidance. For far
comparison, we build a test benchmark consisting of 300 test prompts,
comprising all the 15 different species in dataset. For each species, we
design 20 types of attributes about different combinations of cloth type and
color. All the prompts follow the template: A cartoon [Species Name] wearing
[Cloth Type]. For example: “A cartoon rabbit wearing blue shirt and white
pants.” Then we select 15 mesh models for all species from the dataset as the
base mesh and texture each 3D model using corresponding text prompts. For
example, we apply the texture generated from “A cartoon bear wearing suits.”
on the bear mesh. For all baselines, we render 8 views of each textured object
with white background using the same renderer setting from Blender [5] under
resolution 1024*1024.

Figure 6: Ablation studies of different hyper-parameters and technical
components. We visualize the synthetic results of models trained with
different settings. (d) denotes our current setting.

Baselines. We compare our method against two types of shape texturing
approach: one is based on multi-view texture optimization, the other paints
shape in a progressive manner. We first compare our approach with LatenPaint
[31] and Fantasia3D [8], which optimize a 3D implicit scene based on the
explicit mesh guided by text under multi-view SDS loss. For Fantasia3D, we
only initialize the DMTet [45] based on the conditioned mesh model, and
optimize the texture appearance under the correspondent text prompts. For
painting methods, we compare with TEXTure [42] and Text2tex [7], which
progressively generates partial textures across viewpoints and back-projects
them to the texture space. Besides, we also compare the results from our
texture generator with the StyleGAN2-based texture generator proposed by Rabit
[29].

Qualitative comparisons. We compare the rendering results across several
geometries textured from our approach against other baselines, as shown in Fig
5. We can see that our method is able to generated consistent texture to align
faithfully with the conditioned text prompt. In contrast, painting-based
methods like TEXTure and Text2Tex have noticeable seam artifacts when viewing
the side and back sides of outputs. Optimizing-based methods can generate
multi-view consistent texture. However, LatentPaint can hardly generate high-
quality and text-related neural texture. While Fantasia3D demonstrates
improved rendering results, there are still noticeable non-smooth artifacts
present on the surfaces. We provide additional visualization of results using
the texture generator from Rabit [29]. Since Rabit can only generate texture
image unconditionally, so we randomly generate 100 texture maps and select
relative results for visually comparison. The results show that the synthetic
texture exhibits low-quality with indistinct structure. We conduct the user
study to obtain the user’s subjective evaluation of the fidelity and
plausibility of the texture results. The detail can be found in Supplementary
Material.

Quantitative comparisons. We evaluate the text-driven synthetic textures using
average CLIP score to measure the alignment between texture image with the
conditioned text prompts. The results is shown in Table 1. From the result we
can see that our model achieves the best CLIP scores, indicating better text-
texture alignment. We also report the run time for generating texture under a
specific text guidance using the default hyper-parameters of each method on a
single GPU. Notably, our method and Rabit are significantly faster than the
optimization-based methods which indicates our efficiency. Besides, we also
use the image quality and diversity metric Frechet Inception Distance (FID)
[16] and Kernel Inception Distance (KID) [3] in Table 2. In our experiments,
on 3DBiCar dataset, the real distribution comprises renders of the geometries
with the same settings using their artist designed textures. Results show that
our method achieves better score than the texture generator of Rabit in terms
of both FID and KID.

###  5.4 Ablation Studies

We perform extensive ablation studies on different choices of hyper-parameters
and the importance of the proposed adversarial learning scheme to investigate
their effects on the final results. Specifically, we vary the rank of the LoRA
adapter, exploring settings of 64, 32, and 8 training without adversarial
loss. Then we investigate the effect of adversarial training for texture
enhancement. The visualization results are presented in Figure 6, where the
qualitative analyses unveil the influence of different settings on texture
quality and diversity. According to the visualizations, it is evident that
finetuning with a large rank introduces noticeable sawtooth artifacts. While
reducing the rank mitigates this issue, it concurrently leads to textures with
a low-poly and excessively smooth appearance. Lower ranks, such as 8, tend to
yield more plausible semantic details. Adding adversarial training will help
to enhance the fine-grained patterns in the texture output. Similar
visualization results are also shown on the last two rows on the Fig 4.

##  6 Applications

Out of domain texture generation. Our method could enable realistic UV texture
generation that highly faithful with the text instruction, and even support
out of domain generation such as fashion icons or unreal humanoid characters
from famous fiction or movies while retaining high recognition. We show some
of results in Fig 7.

Prompt-based local editing. We also explore the controlbility of our model as
a prompt-based editing method in Fig 7. Simply using prompt-based editing can
help to modify the texture according to the text while retaining other
concepts. Such an editing capability makes the 3D texture creation with our
model more controllable.

Figure 7: Make-It-Vivid enables out of domain generation about famous fictions
and prompt-based local editing. Figure 8: Make-It-Vivid enables stylized
texturing. We show some synthetic results in shuimo style generated from our
method injected with adapter MoXin 1.0 [34].

Stylized texture generation. Besides, we can achieve stylization for generated
texture by injecting additional parameters from the other pretrained adapter
$\mathcal{S}$ training on the styled image set. Then the modified forward pass
of an input x is: $h=W_{0}x+B_{uv}A_{uv}x+wB_{s}A_{s}x$, where $B_{s}A_{s}$
denotes the parameters of $\mathcal{S}$. We set the balance weight $w=0.5$. We
show some samples generated by our model and a pretrained adapter MoXin1.0
[34] which is trained in a ink and wash painting dataset. We can see that
after the stylization, the model is encouraged to generate plausible and
stylized cloth types which takes large gap with original domain while preserve
the original structure.

Figure 9: Make-It-Vivid supports efficient characters production and animation
under text or video input.

Textured characters production and animation. Our method aims to help users to
create and customize vivid and plausible cartoon character efficiently.
Therefore, we show the progressive generation system capable of creating
textured animatable characters, driven by either text or video in Figure 9.
Specifically, given a text prompt, we first employ the Large Language Model
(LLM) [47] to process the text and extract three information including
subject, texture and motion. For subject, we leverage a CLIP-based retrieval
method to retrieve the shape with the nearest semantic in the dataset as the
base geometry. Then we leverage our proposed texture model to design its
appearance. To generated related motion according to the text, we directly
apply a state-of-the-art text to motion model [15] to process the text and
generate body rotation parameters. We then derive animated characters by
applying the generated rotation parameters to the pre-defined joint points.
Besides, we can also use video or other human interactions to drive or animate
the created cartoon character.

##  7 Conclusion

We propose a novel text-guided texture generation in UV space for 3D biped
cartoon characters, which enables to generate high-quality and semantic
plausible UV textures. To accomplish the lack of high-fidelity data, we
leverage priors from pretrained text-to-image model, which helps to generate
texture map with template structure while preserving the natural knowledge.
Furthermore, we propose an adversarial loss to shorten the domain gap between
original dataset and realistic texture domain while training. Experiments show
that our model can achieve efficient texture creation faithful with text
input, supporting multiple stylization and local editing. Our approach can be
easily applied to 3D character production and animation system, advance the 3D
content creation.

## Appendix

In the following, we provide additional implementation details. Later, we show
the detailed quantitative ablation study and user study. Then, we discuss the
limitation of our method and our future work. At last, we provide more
synthetic results by Make-It-Vivid and rendered images of textured cartoon
model.

##  Appendix A Additional Implementation Details

Rendering. During training, we implement a differentiable mesh renderer using
Kaolin [11] for rendering textured model. The viewpoint is randomly sampled
from $\mathcal{V}:(\phi\in\\{0,45,90,135,180,225,270,315\\},r=1.5,\theta=80)$,
where $r$ is the radius of the camera, $\phi$ is the azimuth camera angle, and
$\theta$ is the camera elevation. We composite the final rendering using
generated textures with Blender [5] rendering pipeline “CYCLES”.

Texture Seam Fixing. We conduct a simple image restoration and inpainting
technique on the back view of the model to mitigate the “black seam”. We show
the illustration of the inpainting process on Figure 10. Specifically, we
apply a state-of-the-art image restoration method [52] to help make the
rendered view perceptually realistic without seam artifacts. We begin by
applying a line mask to the center of the image, employing an inpainting
technique to seamlessly enhance the smoothness of the seam.

##  Appendix B Additional Experiments

###  B.1 Quantitative Ablation Study

We provide additional results of quantitative ablation study on different
choices of hyper-parameters and the importance of the proposed adversarial
learning scheme to investigate their effects on the final results in Table 3.
We can see that decrease the rank leads to better FID and KID scores, which
indicates that keep the diversity. On the other hand, with reconstruction loss
on the predicted noises only, the synthetic images contains fewer high-
frequency details. In contrast, the adversarial losses can signiﬁcantly
improve the image quality and perceptual realism.

###  B.2 User Study

We conduct a user study to obtain the user’s subject evaluation of different
approaches. We present all the results produced by each comparing approaches
to more than 50 participants and ask them to rank these methods on two
perspectives independently: the texture quality and the fidelity with
instruction. All participants are asked to rank different methods with 15 pair
of comparisons in each study. 5 is for the best, and 1 for the worst. The
average score is shown in Table 4. Our method earns user preferences the best
in both aspects.

Figure 10: Illustration of texture seam fixing. |  FID$\downarrow$ |  KID($\times 10^{-3}$)$\downarrow$
---|---|---
(a) Rank 64 | 78.52 | 21.38
(b) Rank 32 | 60.54 | 14.72
(c) Rank 8 | 40.27 | 6.42
(d) Rank 8 + Adv | 35.25 | 5.25
Table 3: Effect of different hyper-parameters and components during training. Method |  Quality$\uparrow$ |  Fidelity$\uparrow$
---|---|---
LatentPaint [31] | 1.92 | 1.45
Fantasia3D [8] | 2.83 | 3.05
Text2Tex [7] | 2.72 | 2.44
TEXTure [42] | 2.67 | 2.16
Ours | 4.53 | 4.21
Table 4: Average ranking score of user study. For each object, users are asked
to give a rank score where 5 for the best, and 1 for the worst. User prefer
ours the best in both aspects.

##  Appendix C Limitations and Future Work

While Make-It-Vivid shows promising results on 3D biped cartoon dataset, it
cannot be applied for arbitrary 3D models. That is because the collection of
high-quality texture maps for all model is non-trival. Automatically generated
mesh topology and UV maps are uninterpretable for 3D artists. For future work,
we will put more effort on automated meshing and texturing on universial 3D
data.

##  Appendix D Additional Results

In this section, we provide additional results of synthetic texture map from
different text instructions using our method. The results are shown on Figure
11 and Figure 12. Results show that our method has a strong ability on
creating high-quality, diversity, and multi-view consistency texture map.

Figure 11: Additional results by Make-It-Vivid part 1. We show the conditioned
text prompt, synthetic texture map and rendered images of textured model.
Figure 12: Additional results by Make-It-Vivid part 2. We show the conditioned
text prompt, synthetic texture map and rendered images of textured model.

## References

  * [1] Animal Crossing.  https://animalcrossing.nintendo.com/.
  * Barron et al. [2021] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan.  Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 5855–5864, 2021.
  * Bińkowski et al. [2018] Mikołaj Bińkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton.  Demystifying mmd gans.  _arXiv preprint arXiv:1801.01401_ , 2018.
  * Blanz and Vetter [2023] Volker Blanz and Thomas Vetter.  A morphable model for the synthesis of 3d faces.  In _Seminal Graphics Papers: Pushing the Boundaries, Volume 2_ , pages 157–164. 2023.
  * [5] Blender.  https://www.blender.org/.
  * Cao et al. [2023] Tianshi Cao, Karsten Kreis, Sanja Fidler, Nicholas Sharp, and Kangxue Yin.  Texfusion: Synthesizing 3d textures with text-guided image diffusion models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4169–4181, 2023.
  * Chen et al. [2023a] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner.  Text2tex: Text-driven texture synthesis via diffusion models.  _arXiv preprint arXiv:2303.11396_ , 2023a.
  * Chen et al. [2023b] Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia.  Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation.  _arXiv preprint arXiv:2303.13873_ , 2023b.
  * [9] DeepFloyd IF.  https://www.deepfloyd.ai/deepfloyd-if.
  * Esser et al. [2021] Patrick Esser, Robin Rombach, and Bjorn Ommer.  Taming transformers for high-resolution image synthesis.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 12873–12883, 2021.
  * Fuji Tsang et al. [2022] Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jianing Li, Michael Li, and Rev Lebaredian.  Kaolin: A pytorch library for accelerating 3d deep learning research.  https://github.com/NVIDIAGameWorks/kaolin, 2022.
  * Fukamizu et al. [2019] Kentaro Fukamizu, Masaaki Kondo, and Ryuichi Sakamoto.  Generation high resolution 3d model from natural language by generative adversarial network.  _arXiv preprint arXiv:1901.07165_ , 2019.
  * Gecer et al. [2019] Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou.  Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 1155–1164, 2019.
  * Goodfellow et al. [2020] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial networks.  _Communications of the ACM_ , 63(11):139–144, 2020.
  * Guo et al. [2022] Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng.  Generating diverse and natural 3d human motions from text.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5152–5161, 2022.
  * Heusel et al. [2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  _Advances in neural information processing systems_ , 30, 2017.
  * Jain et al. [2022] Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole.  Zero-shot text-guided object generation with dream fields.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 867–876, 2022.
  * Karras et al. [2020] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 8110–8119, 2020.
  * Lattas et al. [2020] Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou.  Avatarme: Realistically renderable 3d facial reconstruction” in-the-wild”.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 760–769, 2020.
  * Lattas et al. [2023] Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Jiankang Deng, and Stefanos Zafeiriou.  Fitme: Deep photorealistic 3d morphable model avatars.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 8629–8640, 2023.
  * Lee et al. [2020] Myunggi Lee, Wonwoong Cho, Moonheum Kim, David Inouye, and Nojun Kwak.  Styleuv: Diverse and high-fidelity uv map generative model.  _arXiv preprint arXiv:2011.12893_ , 2020.
  * Lei et al. [2022] Jiabao Lei, Yabin Zhang, Kui Jia, et al.  Tango: Text-driven photorealistic and robust 3d stylization via lighting decomposition.  _Advances in Neural Information Processing Systems_ , 35:30923–30936, 2022.
  * Li et al. [2023a] Chenghao Li, Chaoning Zhang, Atish Waghwase, Lik-Hang Lee, Francois Rameau, Yang Yang, Sung-Ho Bae, and Choong Seon Hong.  Generative ai meets 3d: A survey on text-to-3d in aigc era.  _arXiv preprint arXiv:2305.06131_ , 2023a.
  * Li et al. [2023b] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_ , 2023b.
  * Li et al. [2017] Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero.  Learning a model of facial shape and expression from 4D scans.  _ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)_ , 36(6):194:1–194:17, 2017.
  * Lin et al. [2023] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin.  Magic3d: High-resolution text-to-3d content creation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 300–309, 2023.
  * Loper et al. [2023] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black.  Smpl: A skinned multi-person linear model.  In _Seminal Graphics Papers: Pushing the Boundaries, Volume 2_ , pages 851–866. 2023.
  * Loshchilov and Hutter [2017] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * Luo et al. [2023] Zhongjin Luo, Shengcai Cai, Jinguo Dong, Ruibo Ming, Liangdong Qiu, Xiaohang Zhan, and Xiaoguang Han.  Rabit: Parametric modeling of 3d biped cartoon characters with a topological-consistent dataset.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12825–12835, 2023.
  * Ma et al. [2023] Yiwei Ma, Xiaoqing Zhang, Xiaoshuai Sun, Jiayi Ji, Haowei Wang, Guannan Jiang, Weilin Zhuang, and Rongrong Ji.  X-mesh: Towards fast and accurate text-driven 3d stylization via dynamic textual guidance.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 2749–2760, 2023.
  * Metzer et al. [2023] Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or.  Latent-nerf for shape-guided generation of 3d shapes and textures.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12663–12673, 2023.
  * Michel et al. [2022] Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka.  Text2mesh: Text-driven neural stylization for meshes.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 13492–13502, 2022.
  * Mohammad Khalid et al. [2022] Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, and Tiberiu Popa.  Clip-mesh: Generating textured meshes from text using pretrained image-text models.  In _SIGGRAPH Asia 2022 conference papers_ , pages 1–8, 2022.
  * [34] MoXin.  https://civitai.com/models/12597/moxin.
  * Müller et al. [2022] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller.  Instant neural graphics primitives with a multiresolution hash encoding.  _ACM Transactions on Graphics (ToG)_ , 41(4):1–15, 2022.
  * Nichol et al. [2022] Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen.  Point-e: A system for generating 3d point clouds from complex prompts.  _arXiv preprint arXiv:2212.08751_ , 2022.
  * [37] OpenAI.  https://openai.com/blog/chatgpt.
  * Podell et al. [2023] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.  Sdxl: Improving latent diffusion models for high-resolution image synthesis.  _arXiv preprint arXiv:2307.01952_ , 2023.
  * Poole et al. [2022] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall.  Dreamfusion: Text-to-3d using 2d diffusion.  _arXiv preprint arXiv:2209.14988_ , 2022.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Ramesh et al. [2021] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.  Zero-shot text-to-image generation.  In _International Conference on Machine Learning_ , pages 8821–8831. PMLR, 2021.
  * Richardson et al. [2023] Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or.  Texture: Text-guided texturing of 3d shapes.  _arXiv preprint arXiv:2302.01721_ , 2023.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10684–10695, 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _Advances in Neural Information Processing Systems_ , 35:36479–36494, 2022.
  * Shen et al. [2021] Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler.  Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis.  _Advances in Neural Information Processing Systems_ , 34:6087–6101, 2021.
  * Tang et al. [2023] Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen.  Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior.  _arXiv preprint arXiv:2303.14184_ , 2023.
  * Touvron et al. [2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Tsalicoglou et al. [2023] Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari.  Textmesh: Generation of realistic 3d meshes from text prompts.  _arXiv preprint arXiv:2304.12439_ , 2023.
  * Wang et al. [2023a] Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich.  Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12619–12629, 2023a.
  * Wang et al. [2023b] Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al.  Rodin: A generative model for sculpting 3d digital avatars using diffusion.  _CVPR_ , 2023b.
  * Wang et al. [2023c] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu.  Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.  _arXiv preprint arXiv:2305.16213_ , 2023c.
  * Xia et al. [2023] Bin Xia, Yulun Zhang, Shiyin Wang, Yitong Wang, Xinglong Wu, Yapeng Tian, Wenming Yang, and Luc Van Gool.  Diffir: Efficient diffusion model for image restoration.  _arXiv preprint arXiv:2303.09472_ , 2023.
  * Yu et al. [2023] Xin Yu, Peng Dai, Wenbo Li, Lan Ma, Zhengzhe Liu, and Xiaojuan Qi.  Texture generation on 3d meshes with point-uv diffusion.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4206–4216, 2023.
  * Zhang et al. [2022] Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo.  Styleswin: Transformer-based gan for high-resolution image generation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11304–11314, 2022.
  * Zhang et al. [2023a] Longwen Zhang, Qiwei Qiu, Hongyang Lin, Qixuan Zhang, Cheng Shi, Wei Yang, Ye Shi, Sibei Yang, Lan Xu, and Jingyi Yu.  Dreamface: Progressive generation of animatable 3d faces under text guidance.  _arXiv preprint arXiv:2304.03117_ , 2023a.
  * Zhang et al. [2023b] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3836–3847, 2023b.
  * Zhu et al. [2023a] Deyao Zhu, Jun Chen, Kilichbek Haydarov, Xiaoqian Shen, Wenxuan Zhang, and Mohamed Elhoseiny.  Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions.  _arXiv preprint arXiv:2303.06594_ , 2023a.
  * Zhu et al. [2023b] Deyao Zhu, Jun Chen, Kilichbek Haydarov, Xiaoqian Shen, Wenxuan Zhang, and Mohamed Elhoseiny.  Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions.  _arXiv preprint arXiv:2303.06594_ , 2023b.
  * [59] Zootopia.  https://en.wikipedia.org/wiki/Zootopia.