11institutetext: Bio and Brain Engineering, KAIST1 Graduate School of AI,
KAIST2
11email<EMAIL_ADDRESS>
# One-Shot Adaptation of GAN in Just One CLIP

Gihyun Kwon 11 Jong Chul Ye 1122

###### Abstract

There are many recent research efforts to fine-tune a pre-trained generator
with a few target images to generate images of a novel domain. Unfortunately,
these methods often suffer from overfitting or under-fitting when fine-tuned
with a single target image. To address this, here we present a novel single-
shot GAN adaptation method through unified CLIP space manipulations.
Specifically, our model employs a two-step training strategy: reference image
search in the source generator using a CLIP-guided latent optimization,
followed by generator fine-tuning with a novel loss function that imposes CLIP
space consistency between the source and adapted generators. To further
improve the adapted model to produce spatially consistent samples with respect
to the source generator, we also propose contrastive regularization for
patchwise relationships in the CLIP space. Experimental results show that our
model generates diverse outputs with the target texture and outperforms the
baseline models both qualitatively and quantitatively. Furthermore, we show
that our CLIP space manipulation strategy allows more effective attribute
editing.

###### Keywords:

StyleGAN, Domain adaptation, One-shot learning, CLIP, Consistency, Generative
adversarial networks

##  1 Introduction

Recently, several studies have tried to fine-tune a pre-trained generator
model with limited number of target images so that the fine-tuned model can
generate images of a novel domain. Early methods [45, 29, 30] showed the
results using fine-tuned the models with about 100 training images. Subsequent
studies [31, 25] have shown that GAN domain adaptation is possible even in
extreme situations using fewer than 10 training images. A recent method [50]
attempted to transform a pre-trained source generator to generate novel domain
images by fine-tuning with only one target image.

Although above methods show good performance in limited-shot situations, when
there are smaller number of available training images, the generation
performance drops significantly due to the severe overfitting. The problem is
more pronounced when only one target image is available, which is the
situation we are trying to solve. Although a recently proposed model [50] has
mitigated the overfitting problem, the model often suffers from underfitting
so that the model cannot fully reflect the domain of the target image.

Figure 1: Various domain adaptation results from our model. Our model
successfully fine-tuned the models pre-trained on large data into a target
domain with only a single shot target image. The adapted model can generate
various images corresponding to the target domain while preserving the content
attribute of the source domain. More generated samples can be found in the
experiment section.

To address this, we present here a novel single-shot fine-tuning approach of
the pre-trained generator by using a unified CLIP space manipulation, which
shows better perceptual quality compared to other state-of-the-art few-shot
adaptation models. For example, Fig. 1 shows that our model successfully
generates new images with only a single target by fine-tuning the source model
trained on large datasets (e.g. FFHQ, LSUN church, LSUN cars, and AFHQ dog).

Specifically, our method consists of two steps: finding the reference image
and fine-tuning the model, all in the CLIP space. More specifically, inspired
by the idea of the recent single-shot adaptation method called Mind the GAP
[50], the first step is to find the image from the pre-trained source
generator that most closely resembles a given target image $I_{trg}$ and uses
it as a reference image $I_{ref}$. Unlike existing GAN inversion methods and
Mind the GAP [50], we propose a new strategy that precisely aligns the
semantic attributes between $I_{ref}$ and $I_{trg}$ in the CLIP space. In the
second step, we fine-tune the model using the patch-wise and global
discriminators. In this stage, $I_{ref}$ obtained in the previous step plays
an important role as a reference point that guides the adapted generator to
maintain the principal attribute information (e.g. content shape) of the
source generator. To further match the source-target domain consistency, we
propose a new consistency loss that maintains the CLIP space relation between
the generated samples. Also, in order to maintain detailed spatial information
between source and target models, we propose to maintain patch-wise semantic
relations with contrastive learning in the CLIP space. To further verify the
performance of our model, we show that our CLIP space manipulation strategy
allows flexible and effective attribute editing.

##  2 Related Work

####  2.0.1 Generative Adversarial Networks.

Recent advances in generative adversarial networks (GAN) has shown impressive
performance in generating realistic images. Starting from the seminal work of
GAN [13], Progressive GAN [16] and BigGAN [7] models showed high generation
performance for human face dataset and natural images. StyleGAN [18] showed
generation quality that is almost indistinguishable from the real data.

One of the great advantages of StyleGAN results from the model flexibility,
which has inspired numerous subsequent models through modifications of
StyleGAN. For example, StyleGAN with a modified structure for light weight
model [5], a disentangled model with additional latent space by modifying the
architecture of StyleGAN [4, 24], a model with additional encoder network [20,
35], etc. Recently proposed StyleGAN2 [19] showed further improved generation
performance through more efficient model structure and training strategies.
StyleGAN-ADA [17], also another version of StyleGAN2, showed better
generational performance through a novel training strategy using adaptive
augmentation in discriminator training. Besides, various methods have been
proposed to perform diverse tasks by leveraging the characteristics of
StyleGAN, such as semantic segmentation [48, 2] for dataset generation, 3D
image rendering [43, 41], inverse problems [28, 46], etc.

####  2.0.2 Image manipulation.

Manipulating specific attribute or domain of a given image is an important
topics in computer vision. CycleGAN [49] successfully converted the image to
other domains with novel cycle consistency, which was followed by several
improved image translation methods [11, 6]. The recent contrastive unpaired
translation model (CUT) [33] approach achieved state-of-the-art performance by
preserving the spatial attributes between the input and the generated output
by applying contrastive learning to the patchwise feature embedding space.

In addition, multi-domain models that enable arbitrary image transformation
between various domains have also been developed. StarGAN [8] is the first
work for multi-domain translation through a single common generator, and
StarGANv2 [9] extended the multi-domain single-output translation capability
of StarGAN into multi-domain diverse image translation.

In addition, various attribute manipulation methods leveraging the
disentangled characteristic of StyleGAN have been proposed lately. These
include a method to find specific direction for independent attribute by
separating the principal component of the latent space [14], or to find a
latent space that can edit a meaningful area of generated images by
additionally using label information [40] or semantic map [15]. Since these
StyleGAN-based processing methods inevitably require a mapping of a given
image to StyleGAN’s latent space, various StyleGAN inversion methods have also
been proposed. These include methods which train additional encoder model
suitable for StyleGAN [44, 38], or latent optimization methods for a given
image [1, 51], etc.

Recently, with the introduction of the CLIP model [37] by OpenAI, image
manipulation methods through text conditions, such as text-to-image generation
[27] and text-guided style transfer [23], have received a lot of attention. In
addition, by exploiting the disentangled characteristic of StyleGAN, StyleCLIP
[34] proposed a method for editing the latent attribute of StyleGAN through
text, and StyleGAN-NADA [12] proposed a method of adapting the model to a
novel domain through texts. Especially StyleGAN-NADA further showed the
versatility of model adaptation for a given target image.

####  2.0.3 Few-shot domain adaptation.

The purpose of adapting a generative model using few-shot images is to guide
the model to a target domain while inheriting the diversity of source model
trained on large-scale data. One of the first works of GAN adaptation is
TransferGAN [45], which provided a novel insight of few-shot generation as a
kind of transfer learning. Various subsequent few-shot GAN adaptation models
focused on preventing the generator from overfitting by introducing
regularization components during the fine-tuning process. FreezeD [29]
performed GAN adaptation by freezing some layers of the discriminator in order
to maintain the prior features of generator model. Others proposed to mix the
weights of fine-tuned model and source model [36], or maintain singular
vectors of the pre-trained weights [39]. In addition, EWC [25] used Fisher
information to identify important layers of generator features and freeze them
to prevent overfitting.

Recently, Ojha et al [31] prosed an improved GAN adaptation method even in
few-shot setting of less than 10 training samples by exploiting the cross-
domain consistency and patch-based adversarial training. Furthermore, Mind the
GAP [50] attempted to fine-tune the generator model using only one target
image. Specifically, by aligning the vector direction within CLIP-space,
stable domain adaptation was possible [50].

Although recently proposed models addressed the overfitting problems of GAN
adaptation, most of previous methods still suffer from overfitting when the
available samples are extremely limited. Mind the GAP could solve the
overfitting with single-shot domain adaptation, but we empirically discovered
that this model suffer from underfitting problems when the target image
textures are complex or the appearance is far from source domain.

##  3 Methods

By extending the existing approaches, our goal is to find improved
regularization and training strategies, which can adapt the model robustly
without over- and under- fitting problems regardless of the target image types
or pre-trained source domains. One of the most important contributions of this
work is the discovery of the importance of the unified CLIP space
manipulation.

Specifically, our method consists of two steps. First, similar to the previous
work [50], we search a latent code $w_{ref}$ in the latent space of the source
domain generator $G_{s}$ so that it can generate reference image
$I_{ref}=G(w_{ref})$ that is most similar to a given single-shot target image
$I_{trg}$. In contrast to Mind the GAP [50] which uses pixel-wise similarity,
we found that the CLIP space similarity significantly improves the search. The
resulting image $I_{ref}$ is then used as a reference point to be aligned with
$I_{trg}$ in the next step of model fine-tuning. In the second step, we fine-
tune the pre-trained generator to create a target generator $G_{t}$ which
follow the domain information of $I_{trg}$. At this time, our goal is to guide
$G_{t}$ to maintain the diverse content attribute of $G_{s}$ by aligning the
target image $I_{trg}$ and the reference image $I_{ref}$ obtained in the
previous step. Again we reinforce this by using regularizations in the CLIP
space to enforce semantic consistency between source and target generators.
More detailed descriptions are as follows.

###  3.1 Step 1: Clip-guided Latent Search

In order to find the image most consistent with the target image, the previous
adaptation model Mind the Gap [50] leveraged the existing StyleGAN inversion
method II2S [51]. However, we found that when the domain of the $I_{trg}$
(e.g. abstract sketch) is far from the source domain (e.g. FFHQ), the existing
inversion model failed to reconstruct the attribute of $I_{trg}$.

Figure 2: (Left) Overview of our proposed CLIP-guided latent optimization to
find the reference image $I_{ref}$. (Right) Comparison between various
baselines. Our results contain the desired attribute of the target image and
outperforms the baseline results.

We conjecture that this performance degradation is caused by the existing
inversion model focusing only on pixel-wise similarity. To correct this, we
propose to guide the generated $I_{ref}$ to follow the semantic attributes of
$I_{trg}$ using the pre-trained CLIP model. More specifically, as shown in
Figure 2(left), we include additional loss that reduces the cosine distance
between the CLIP space embeddings of the two images while using the pixel-wise
loss between the $I_{ref}$ and $I_{trg}$. In addition, we employ the
augmentation for $I_{ref}$ to avoid artifacts. The resulting optimization
problem is formulated as:

$\displaystyle\arg\min_{w\in
W}D_{CLIP}(I_{ref},I_{trg})+||I_{ref}-I_{trg}||_{2}+lpips(I_{ref},I_{trg})+\lambda_{reg}||w-\bar{w}||_{2},$

where $I_{ref}=G_{s}(w)$, $D_{CLIP}(\cdot,\cdot)$ is cosine distance between
CLIP embeddings, $lpips$ is perceptual loss, and $\bar{w}$ is the averaged
style code of the source generator $G_{s}$. Here, we additionally use $l_{2}$
regularization between $w$ and $\bar{w}$ to avoid reconstructing unrealistic
images. Furthermore, to improve the efficiency of the search, rather than
using random starting point for $w$, we set $\bar{w}$ as a starting point of
$w$. After optimization step, we use the final latent codes $w$ as the
reference latent $w_{ref}$.

The results of the proposed CLIP-guided optimization are shown in Fig.
2(right). When compared to the existing inversion methods II2S [51] and e4e
[44], we can observe that the images from our method reflect the attributes
(e.g. gender, component shape) of the target image while the baselines fail.
Since our purpose is to obtain $I_{ref}$ representing the source domain while
accurately having the attributes of the target image, the proposed method is
more suitable in generating a reference.

###  3.2 Step 2: Generative Model Fine-tuning

####  3.2.1 Cross-domain semantic consistency.

In the next step, we create a target generator $G_{t}$ by fine-tuning the
weights of the pre-trained generator. Recall that Ojha et al [31] used a patch
discriminator $D_{patch}$ so that the images generated by $G_{t}$ have the
texture of the target image $I_{trg}$. However, the use of the discriminator
alone easily leads to overfitting, so it is necessary to use additional
regularization so that $G_{t}$ can inherit the generation diversity of
$G_{s}$. To address this, the previous works attempted to match the
distributions of the features between the source and target generators.
Unfortunately, under the single-shot condition, such regularization based on
the distribution of generator features does not work and still causes the
overfitting.

Similar to Step 1, we found that CLIP space regularization solves the issues.
Specifically, we found that maintaining the similarity distribution in the
CLIP space as in Fig. 3(a) is more effective in preventing overfitting because
it considers the semantic information of the generated images. Specifically,
when arbitrary latent variables $[w_{i}]^{N}_{0}$ are sampled, we first
calculate sample-wise similarity scores for $G_{s}$ and $G_{t}$ defined as:

$\displaystyle c^{s}_{n}=D_{CLIP}(G_{s}(w_{i}),G_{s}(w_{j})),\quad
c^{t}_{n}=D_{CLIP}(G_{t}(w_{i})),G_{t}(w_{j})),$

where $n$ is the reordered index of $(i,j),i\neq j$ and $D_{CLIP}$ denotes the
cosine similarity in the pre-trained CLIP embedding space. With the calculated
similarity scores, the similarity loss between $G_{s}$ and $G_{t}$ can be
computed. Different from the existing work [31] which use softmax with KL
divergence for distribution matching, we simply used $l_{2}$ distance between
the similarity scores, since we observed that the use of KL divergence in the
CLIP space reduces the training stability. Accordingly, our loss function for
semantic consistency between $G_{s}$ and $G_{t}$ is formulated as:

$\displaystyle L_{con}=\mathop{\mathbb{E}}_{w\sim
p_{w}(w)}\sum_{n}||c_{n}^{s}-c_{n}^{t}||_{2}$

Figure 3: Overview of our proposed model fine-tuning step. (a) Cross-domain
semantic consistency in the CLIP space. We generate image from the source
model $G_{s}$ and the target model $G_{t}$ with the same latents $w$. Then we
obtain the embedded feature vector using a pre-trained CLIP model, and the
cosine similarity scores between the embedded vectors in the CLIP space are
calculated. The similarity scores between both domains ($c^{t},c^{s}$) are
aggregated through the $l_{2}$ regression. To guide the texture of the $G_{t}$
to match the target $I_{trg}$, we additionally use the patch discriminator.
(b) Patch-wise semantic consistency in the CLIP space. We crop the patches
from generated images and embed the cropped patches using a pre-trained CLIP
model. Then we apply contrastive learning on those embedded vectors. (c)
Reference target alignment. In the case of using reference latent $w_{ref}$,
the generated image should match $I_{trg}$.

####  3.2.2 Patch-wise semantic consistency.

Although we can avoid the overfitting problem using the novel loss $L_{con}$,
the loss is a regularization for the sample-wise semantics of the generated
images so that their local features are ignored. Therefore, for better
consistency of fine details between source and target generators, we propose a
new patch-wise consistency loss.

To preserve the local attribute between two domains, we start from the idea of
patch-wise contrastive loss (PatchNCE) in CUT [33]. Recall that CUT employs
the contrastive learning to the embedded features of the generator. Instead of
using generator features directly, additional MLP header network is used in
CUT to embed the features into another space. However when we applied this
directly to our framework, we observed that the training fails due to the
imbalance between the header and the pre-trained generator. For this problem,
we also found that CLIP is essential so that we propose to use the pre-trained
CLIP model as a patch-wise embedding network.

Specifically, after cropping patches at random locations from the images
generated by $G_{s}$ and $G_{t}$, we embed the image patches with the CLIP
encoder as shown in Fig. 3(b). Then, we reduce the distance between the
positive patches cropped at the same location, and push away the negative
patches cropped from other locations. More specifically, if we set arbitrary
location $s_{0}$, the cropped patch of the outputs from the generators $G_{s}$
and $G_{t}$ are denoted as $[G_{s}(w)]_{s_{0}}$ and $[G_{t}(w)]_{s_{0}}$,
respectively. Then if we set other $N$ locations, we can obtain $N$ patches
from source domain as $[G_{s}(w)]_{s_{i}}$ where $i\in\\{1,...,N\\}$. With the
cropped patches, we can calculate the patch-wise loss such as:

$\displaystyle L_{patch}=-\text{log}\left[\frac{\text{exp}(v\cdot
v^{+})}{\text{exp}(v\cdot v^{+})+\sum^{N}_{i=1}\text{exp}(v\cdot
v_{i}^{-})}\right],$ (1)

where $v=E([G_{t}(w)]_{s_{0}})$, $v^{+}=E([G_{s}(w)]_{s_{0}})$ are positive
vectors, and $v^{-}=E([G_{s}(w)]_{s_{i}})$ are negative vectors. Here, $E$
refers to the pre-trained CLIP encoder for the image patch, and the dot mark
$\cdot$ represents cosine similarity.

####  3.2.3 Reference Target Alignment.

The losses suggested in the previous part serves to prevent overfitting when
we sample arbitrary latents. This should be alternated with the additional
loss for generating images through reference latent $w_{ref}$ found in Step 1.
Since $I_{ref}$ obtained in Step 1 has the attribute most similar to $I_{trg}$
and is an image representing the source domain, the domain-adapted output
$G_{t}(w_{ref})$ should be matched to $I_{trg}$. Therefore, as shown in
Fig.3(c), we match the images in both of pixel and perceptual perspective as
proposed in [50]. Also, we further guide the $G_{t}(w_{ref})$ with global
discriminator $D_{glob}$ to make the image much closer to $I_{trg}$.

####  3.2.4 Overall Training.

Overall, our network is trained by alternately minimizing the two losses.
First, when we use reference latent $w_{ref}$, our loss is defined as:

$\displaystyle
L_{ref}=||G_{t}(w_{ref})-I_{trg}||_{2}+lpips(G_{t}(w_{ref}),I_{trg})+L^{g}_{adv}(G_{t},D_{glob}),$

where StyleGAN2 adversarial loss is defined as
$L^{g}_{adv}(G,D_{glob})=D_{glob}(G(w_{ref}))-D_{glob}(I_{trg})$. For a global
discriminator, we fine-tuned the pre-trained StyleGAN2 discriminator. Second,
when arbitrary latents $w$ are sampled, our loss is:

$\displaystyle
L_{rand}=\lambda_{con}L_{con}+\lambda_{patch}L_{patch}+L^{p}_{adv}(G_{t},D_{patch}).$

In this case, $L^{p}_{adv}$ is defined as
$L^{p}_{adv}(G,D)=D_{patch}(G(w))-D_{patch}(I_{trg})$. For $D_{patch}$, we
used the proposed network in Ojha et al [31], in which $D_{patch}$ is a subset
of $D_{glob}$. More specifically, we extract the intermediate feature from
$D_{glob}$, then obtain the logit of $D_{patch}$ with mapping the feature
through several conv layers. Since simultaneously training using two losses is
observed to reduce training stability and consume excessive memory, we
therefore alternately train the model 3 iterations only using $L_{ref}$
(Fig.3(a,b)), then 1 iteration only using $L_{rand}$(Fig.3(c)).

##  4 Experiments

####  4.0.1 Experimental details.

Our training starts with the pre-trained StyleGAN2 [19]. To verify the
versatility of our model, we conducted three different experiments using pre-
trained generators trained on three datasets (FFHQ, LSUN church, LSUN cars).
The models trained on FFHQ, LSUN church, and LSUN cars generate 1024x1024,
256x256, and 512x512 resolution images, respectively. We used the Adam
optimizer [21] for all cases, and set the learning rate to 0.02. We used batch
size 2 for FFHQ and LSUN cars, and batch size 4 for LSUN church. The training
time took about 30 minutes with 2,000 iterations using a single RTX 2080 GPU.
This is the time including 3 minutes of CLIP-guided optimization in step 1.
For $L_{patch}$ calculation, we used patch size of 128x128, 64x64, and 32x32
in adapting the pre-trained models on FFHQ , LSUN church, and LSUN cars,
respectively. For hyperparameters, we set $\lambda_{reg}$ in step 1 as 0.01,
and set $\lambda_{con}$, $\lambda_{patch}$ as 10, 1, respectively. Similar to
previous work [50], we freezed ‘toRGB’ layer and mapping network in the
refining stage.

For inference , in the case of adapting FFHQ-pretrained model, we inverted the
CelebA-HQ [16] images into latent codes $w_{rec}$ using e4e [44] and generated
images $G_{t}(w_{rec})$. In the case of LSUN church and LSUN cars models, we
do not have proper validation image sets. Therefore we showed the results with
same sampled latent $w^{\prime}$, in which the source image is
$G_{s}(w^{\prime})$ and adapted outputs are $G_{t}(w^{\prime})$

Figure 4: Comparison results with baseline models. The images in the uppermost
row are the source content images. The results in the following rows are
generated by the models adapted with target images. Our results contain the
texture of the target images while preserving the content attributes of the
source images.

As baseline models, we adopted three state-of-the-art GAN domain adaptation
methods: Few-shot GAN adaptation [31], StyleGAN-NADA [12], and Mind the Gap
[50]. For Few-shot GAN adaptation and StyleGAN-NADA, we used the official
source codes111https://github.com/utkarshojha/few-shot-gan-
adaptation222https://github.com/rinongal/StyleGAN-nada to obtain the results.
For the baseline of Mind the Gap, we implemented the model by following the
descriptions in the original paper.

With the given latent code $w$, we could show the adapted output with
$G_{t}(w)$. However, we observed that the generated images often have minor
artifacts in background. To alleviate them, we used style mixing trick in
which we replace the fine layer style code to average style vector $\bar{w}$
of $G_{t}$. In case of FFHQ model adaptation, instead of using $w$ which is
composed of 18 vectors, we use mixed style code $w_{mix}$ in which the last 7
vectors are substituted to $\bar{w}$. Detailed settings for other models are
in our Supplementary Materials.

Figure 5: Comparison results using other datasets. The images in the uppermost
row are the source content images. (Left) Generated outputs using the adapted
model pre-trained with LSUN church dataset.(Right) Generated outputs using the
adapted model pre-trained with LSUN cars dataset. Our results have texture of
the target images while preserving the content attributes of the source
images.

####  4.0.2 Qualitative Results.

Fig. 4 shows the image result generated by the target generator adapted from
the source generator pre-trained with FFHQ. For comparison, we also show the
images generated from the baseline models. In all cases, the results show that
the baseline models of few-shot GAN adaptation and StyleGAN-NADA suffer from
overfitting problems in which facial identities are almost identical.
Especially in the case of Few-shot GAN adaptation, we could observe that the
generated images failed to reflect the attributes of the source domain due to
the severe overfitting, and in some cases, model training for domain
adaptation failed. In the case of StyleGAN-NADA, the generated images
partially contains the content of source domain images, but the model could
not accurately reflect the texture of the target image.

In the case of Mind the Gap, unlike previous baselines, the model generates
diverse images while preserving the content of the source domain images.
However, the generated images did not properly reflect the domain texture of
the target image in most cases, and only relatively easy characteristics such
as global color or background were mainly changed. In particular, when the
target image shows a large difference from the source domain (e.g. rough
sketch in 2nd row of left side), the above issue is more pronounced. On the
other hand, our model generates images that accurately reflect the complex
texture (e.g. brushstroke, sketch line) of the target image while preserving
the contents of the source domain images.

####  4.0.3 Additional Qualitative Results.

In order to verify the versatility of the proposed model, we conducted
additional experiments with domain adapted models using source generator pre-
trained with LSUN cars and LSUN church datasets. We also show the results from
baselines models for comparison in Figure 5.

Again, the baseline models such as Few shot adaptation and StyleGAN-NADA
suffer from overfitting problems. Although they generate slightly different
images by reflecting the geometry information of the source domain to some
extent compared to the results for the human face, the generated images still
show the same type of cars or uniform shape of buildings. In the case of Mind
the GAP, the result images have the content information of the source images
but the model weakly reflect the style of the target domain. On the other
hand, we observed that our model sufficiently follows the domain information
of the target image while maintaining the content attribute of the source
domain. For further evaluation, we will show additional results with generator
pre-trained on animal faces in our Supplementary materials.

Figure 6: Latent space editing resulting. We manipulated the attribute of the
faces using a tool in the StyleCLIP. The results show that adapted model
$G_{t}$ still maintains the latent space characteristic of source domain
generator.

####  4.0.4 Latent Space Editing.

Recall that our purpose is to train a fine-tuned model $G_{t}$ to generate
images with the target texture while preserving the latent space
characteristics of the source domain. To verify this, we tested whether the
latent space of $G_{t}$ sufficiently includes the disentangled characteristic
of the source domain through attribute editing.

In this part, we adopted the latent space editing technique in StyleCLIP [34],
which is a state-of-the-art attribute editing method. With StyleCLIP, we can
edit the face attributes with various text conditions. Among the three editing
approaches proposed in StyleCLIP, we used the editing through global direction
which can be universally applied. When there is a latent code $w$
corresponding to the source image, we can manipulate it with StyleCLIP to get
a edited latent $\hat{w}$, then we put the code as an input to the fine-tuned
generator $G_{t}$ to get the attribute edited output in the target domain.

We show experimental results of attribute editing in Fig. 6. When editing
through various text conditions, we show that domain-adapted generator $G_{t}$
can also generate edited outputs without entanglement. This shows that our
adapted generator maintains the content characteristics of the source domain.

####  4.0.5 Ablation study.

In order to check how the various components proposed in our method operate,
we conducted ablation studies by removing each component. In Fig. 7, when our
proposed semantic consistency loss $L_{con}$ is removed, we can see that the
facial attributes of the generated faces have identical shape due to the
overfitting of adapted model. When removing the patch-wise consistency loss
$L_{patch}$, we can see the generated faces have different face identities
from the source domains images. Although using only the suggested $L_{con}$
can prevent overfitting, we found that when $L_{patch}$ is not used, the
generated images are excessively deformed due to lack of consistency between
the local regions of source and output images.

In addition, to evaluate the superiority of our proposed CLIP-guided
optimization, we trained the model using baseline II2S [51] instead of our the
proposed CLIP space optimization. At this time, the model generates decent
result images, but we can see that there are misalignments between the
attributes (e.g eyes shape, beard) of the generated and source images. We
conjecture that the problem occurs because the reference image generated with
II2S does not properly cover the attribute of the target image. For more
detailed ablation study results, please refer to our Supplementary Materials.

Figure 7: Ablations study results. The absence of $L_{con}$ leads to
overfitting, generating the same face identity regardless of the input images.
Without $L_{patch}$, detailed component shapes are misaligned with the source
image. When II2S is used for reference research instead of our proposed CLIP
guided optimization (CLIPopt), several facial attributes are not correctly
reflected.

####  4.0.6 User study.

Methods Scores $\uparrow$ Few-shot adapt[31] 1.96 StyleGAN-NADA[12] 1.83 Mind
the Gap[50] 3.37 Ours 3.92

Table 1: User study results on various single-shot domain adaptation models.
Our model outperforms baseline methods in user preference scores. The results
show that our model have better quality in overall perceptual quality.

For further evaluation of our proposed model, we additionally conducted a user
study. To quantitatively measure the detailed preference from users, we used a
custom-made opinion scoring system. More specifically, we provided the
generated images to users, and asked them to score the images according to
three evaluation criteria: 1) Do the generated images properly reflect the
domain style of the target image? 2) Does the content of the generated images
properly match with the source domain images? 3) Do the generated images have
sufficient diversity?

For each question, users can choose the scores among 5 options: 1-very bad,
2-bad, 3-neutral, 4-good, 5-very good. We randomly recruited 30 users using
Google Form, who come from the age group between 20s and 40s. As baselines, we
again adopted the models of Few-shot adaptation, StyleGAN-NADA, and Mind the
Gap. We provided users with 100 generated images per model (total 400 images).

Table 1 shows the user study results. We can see that our model outperforms
all baseline models in terms of perceptual preference scores. In the cases of
Few-shot adaptation and StyleGAN-NADA, the models show lower scores compared
to other models. For Mind the GAP, the users gave relatively high score
compared to previous models. However, our model showed the highest scores
among all selected models.

Figure 8: Text-guided adaptation results. The results from our model follow
the overall style of text condition with better content preservation compared
to StyleGAN-NADA.

####  4.0.7 Text-guided image manipulation.

In order to further evaluate our method, we additionally show text-guided
model adaptation results. For this, we replaced our adversarial loss part with
the directional CLIP loss proposed by StyleGAN-NADA[12]. Detailed
implementation process is elaborated in our Supplementary Materials.

In Fig. 8, we show comparison results of model adaptation with single text
condition. Although baseline StyleGAN-NADA can modulate the overall texture of
outputs, the generated faces have almost same identity regardless of the input
images, suggesting that the model suffers from overfitting. In contrast, our
model can synthesize reasonable outputs that reflect both text condition and
source content. The results confirmed that our CLIP space regularization can
avoid overfitting, further confirming that our proposed model have a strong
advantage in versatility.

##  5 Conclusions

In this article, we proposed a new framework that can transform a pre-trained
StyleGAN to generate target domain images by fine tuning with a single target
image. The key idea is the CLIP space manipulation through a two step
approach. Specifically, we proposed an optimization method in CLIP space to
find the reference image which have the most similar attribute to the target
image in the source domain. The reference image was then used as an anchor
point to maintain the content attribute of target generator. In addition, to
prevent model overfitting, we proposed two different regularization losses:
semantic consistency loss and patch-wise consistency both in CLIP space. Our
experimental results showed that the proposed method leads to better
perceptual quality and diversity than the existing methods. The limitations
and future work can be found in Supplementary Material.

##  Appendix 0.A More Experimental Details

For the patch discriminator $D_{patch}$, we used a subset of $D_{glob}$ as
proposed by Ojha et al.[31]. More specifically, using a pre-trained
discriminator of StyleGAN2, we first extract the intermediate features using
the first $l$ residual conv layers of $D_{glob}$, and then map the s to the
final logit through an additional conv layer
$F^{l}$(CONV3$\times$3-LeakyReLU). In order to deal with various patch window
sizes, multiple $l$ and $F^{l}$ were used. Specifically, when we adapt the
model pre-trained with FFHQ, we used $l=[5,6]$; for the pretrained model with
LSUN church [47], we used $l=[3,4]$; for the pretrained models with LSUN cars
[22] and AFHQ dog [9], $l=[4,5]$ was used.

For the latent optimization, in contrast to the baseline methods such as II2S
[51] and e4e [44], which use extended StyleGAN latent space
$W^{+}\in\mathbb{R}^{18\times 512}$, the optimization was performed in the
single latent space in $W\in\mathbb{R}^{512}$ since it produced better
perceptual quality and preserved $I_{trg}$ attribute.

As for the style mixing trick described in the main article, in the LSUN car
model fitting experiment, the last 7 vectors in the sampled code $w$, which
consisted of 16 vectors, were replaced; and in adapting the LSUN church model,
the last 6 vectors in the sampled code $w$, which consisted of 14 vectors,
were replaced. For adapting the model with AFHQ dog, the last 9 vectors in the
sampled code $w$, which consisted of 16 vectors, were replaced.

For training iterations, our default setting of training iteration is 2,000.
However, if the content shape of target image is not much far from the source
domain (e.g. realistic portrait), we trained the model with only 1,000
iterations. We selected perceptually better image between the outputs from
models trained on 1,000 and 2,000 iterations. In case of adapting LSUN church
models, we used 1,500 iterations since the model converged faster than the
others.

For target images, we selected random image from the portrait face dataset [3]
in FFHQ model adaptation. For others, we randomly collected publicly available
images from the internet.

For more details, please refer to our source code in
GitHub333https://github.com/submission6378/OneshotCLIP

##  Appendix 0.B Text-guided Image Manipulation

For text-guided image manipulation, we replaced the adversarial loss of our
model with the directional CLIP loss. More specifically, in the first step we
find the reference image $I_{ref}$ that most closely matches the target text
constraint $t_{trg}$. Specifically, our goal is to find latent code $w_{ref}$
for the source domain generator $G_{s}$ as follows:

$\displaystyle w_{ref}:=\arg\min_{w\in
W}D_{CLIP}(I_{ref},t_{trg})+\lambda_{reg}||w-\bar{w}||_{2},$

where $I_{ref}=G_{s}(w)$, $D_{CLIP}(\cdot,\cdot)$ is the cosine distance
between CLIP embeddings, and $\bar{w}$ is averaged style code of the source
generator. After the optimization step, we use the final latent code $w$ as
the reference latent $w_{ref}$.

Next, we fine-tune the generator model using the directional CLIP loss
proposed in StyleGAN-NADA [12]:

$\displaystyle\Delta T=E_{T}(t_{trg})-E_{T}(t_{src}),$ $\displaystyle\Delta
I=E_{I}(G_{t}(w))-E_{I}(G_{s}(w)),$ (2) $\displaystyle L_{dir}=1-\frac{\Delta
I\cdot\Delta T}{|\Delta I||\Delta T|},$ (3)

where $E_{T}$ is text encoder of CLIP, and $E_{I}$ is image encoder of CLIP,
$w$ is randomly sampled latent, and $t_{source}$ is the source text condition
representing the text description of the domain of $G_{s}$. In our case, we
used the text “Photo” as $t_{src}$.

Figure 9: Additional results from our text-guided model adaptation. The images
in the uppermost row are the source content images. The results in the
following rows are generated by the models adapted given the target texts at
the leftmost column.

In order to prevent overfitting problems, again we use our proposed
consistency losses $L_{con}$ and $L_{patch}$. For $L_{con}$, we first
calculate

$\displaystyle c^{s}_{n}=D_{CLIP}(G_{s}(w_{i}),G_{s}(w_{j})),\quad
c^{t}_{n}=D_{CLIP}(G_{t}(w_{i})),G_{t}(w_{j})),$

where $[w_{i}]^{N}_{0}$ is arbitrary latent variables, $n$ is the reordered
index of $(i,j),i\neq j$ and $D_{CLIP}$ denotes the cosine similarity in the
pre-trained CLIP embedding space. Then we obtain $L_{con}$ as follows:

$\displaystyle L_{con}=\mathop{\mathbb{E}}_{w\sim
p_{w}(w)}\sum_{n}||c_{n}^{s}-c_{n}^{t}||_{2}.$ (4)

For $L_{patch}$, if we set arbitrary location $s_{0}$, the cropped patch of
the outputs from the generators $G_{s}$ and $G_{t}$ are denoted as
$[G_{s}(w)]_{s_{0}}$ and $[G_{t}(w)]_{s_{0}}$, respectively. Then if we set
other $N$ locations, we can obtain $N$ patches from source domain as
$[G_{s}(w)]_{s_{i}}$ where $i\in\\{1,...,N\\}$. With the cropped patches, we
calculate the patch-wise loss such as:

$\displaystyle L_{patch}=-\text{log}\left[\frac{\text{exp}(v\cdot
v^{+})}{\text{exp}(v\cdot v^{+})+\sum^{N}_{i=1}\text{exp}(v\cdot
v_{i}^{-})}\right],$ (5)

where $v=E([G_{t}(w)]_{s_{0}})$, $v^{+}=E([G_{s}(w)]_{s_{0}})$ are positive
vectors, and $v^{-}=E([G_{s}(w)]_{s_{i}})$ are negative vectors. Here, $E$
refers to the pre-trained CLIP encoder for the image patch, and the dot mark
$\cdot$ represents cosine similarity.

Our full loss function for $G_{t}$ fine-tuning is then defined as:

$\displaystyle L_{text}=L_{con}+L_{patch}+L_{dir}+L_{dir}^{ref},$

where $L_{dir}^{ref}$ is the directional clip loss with using reference latent
$w_{ref}$ instead of sampled $w$. Due to the lack of target images, we do not
use discriminator network. We trained the model for 1,000 iterations.

Similar to Mind the GAP [50], instead of showing direct output $G_{t}(w)$, we
also use $G_{t}(w_{mix})$, where the mixed style code $w_{mix}$ was
constructed by replacing the last 11 vectors of sampled latent $w$ with
$w_{ref}$.

In Fig. 9, we show the results using our text-guided image manipulation. Our
model can adapt the target model $G_{t}$ with the given text conditions while
preserving the content diversity of the source generator model.

Figure 10: Ablation study results. We show the results with various different
settings. (a) Generated images from $G_{t}$ adapted using $L_{con}$ with
Softmax and KL divergence. (b) Generated images from $G_{t}$ adapted using
consistency loss on generator features [31] instead of our proposed CLIP space
consistency $L_{con}$. (c) Results from $G_{t}$ adapted using contrastive
regularization proposed in CUT [33] instead of our $L_{patch}$. In (a) and (b)
, we can see the model suffer from overfitting. For the results in (c), local
attributes between source and output images are not properly matched.

##  Appendix 0.C Additional Experiments

####  0.C.0.1 Ablation studies.

Ablation study results are shown in Fig. 10, which are conducted under various
settings. First, we trained the model with using $L_{con}$ with different
settings. (a) When we use $L_{con}$ composed of Softmax & KL Divergence
instead of our proposed $L_{2}$ regression, we observed that the results
suffer from severe overfitting as the loss does not work properly. (b) Next,
when we use the consistency loss on generator features proposed by Ojha et al
[31] instead of our CLIP-space consistency loss $L_{con}$, the model suffers
from severe overfitting due to training failure. (c) We show the results using
contrastive regularization on generator features with additional header
network $F$ similar to CUT[33], instead of using our $L_{patch}$. In this
case, we observed that overfitting problem is resolved as $L_{con}$ correctly
operates, but the local features of output images are not properly matched to
the source images, due to training imbalance between generator and header
networks.

Figure 11: Additional comparison results with state-of-the-art image style
transfer models. The result from our model reflects accurate semantic
information of the target image while baseline models fail.

####  0.C.0.2 Comparison with Other Style Transfer Models.

To further verify the performance of our proposed model, in Fig. 11 we show
the comparison results with the existing state-of-the-art image style transfer
models, such as AdaAttn [26], SANet [32], and CST[42].

In the case of the existing style transfer models, the model failed to reflect
the semantic information of target images, as they excessively concentrate on
applying the overall color and texture of the target images. We can see that
there are several artifacts such as the background color of the target image
applied to the face, or the background color applied to entire image areas as
shown in Fig. 11. In the case of our model, we can see that the generated
images accurately reflect the characteristics of target domain by considering
both the texture and semantic information of the targets.

####  0.C.0.3 Comparison with Face Domain Adaptation Model.

Additionally, in Fig. 12, we further compare our results with another one-shot
face domain adaptation model JojoGAN[10]. When compared to our model, most of
the images generated from JojoGAN have highly similar contents (e.g. same
beard, same eyes) by excessively reflecting the content attributes of the
target image, which shows that JojoGAN suffers from overfitting problems. In
addition, we observed that JojoGAN could not properly reflect the detailed
texture of the target image with generating blurry images. In contrast, our
model accurately reflects the style and texture of the target image while
properly maintaining the content attribute of the source image.

####  0.C.0.4 Additional Results.

We also show additional results with adapting various pre-trained models. In
Fig. 13, we show more results on adapting FFHQ pre-trained models. In Fig. 14,
we show the results on adapting the models pre-trained on LSUN Car, LSUN
church, and AFHQ dog datasets. The generated images have semantic style of the
target images while preserving the content information of source images.

Figure 12: Comparison results with recent one-shot face domain adaptation
model JojoGAN [10]. The results from baseline JojoGAN suffer from overfitting
problems.

##  Appendix 0.D Limitation and Future work

Our model showed superior performance compared to the baseline models in one-
shot domain adaptation experiments, and the results on various adapted models
further confirmed that our model has enough versatility. However, since our
method still relies heavily on the StyleGAN model pre-trained for specific
domain data, it is not clear whether our method will work with the GAN model
that generates more diverse natural images such as BigGAN [7]. Therefore, as a
further extension, we are planning to try domain adaptation on generative
models for natural images. In addition, we plan to explore whether our
proposed regularization can be applied to other generative framework such as
diffusion models.

Figure 13: Additional results with adapting the models pre-trained on FFHQ
dataset. The images in the uppermost row are the source content images. The
results in the following rows are generated by the models adapted with the
target images. Figure 14: Additional results with adapting the models pre-
trained on LSUN Cars (Up), LSUN church (Middle), and AFHQ Dog (Bottom)
datasets. The images in the uppermost row are the source content images. The
results in the following rows are generated by the models adapted with the
target images.

## References

  * [1] Abdal, R., Qin, Y., Wonka, P.: Image2stylegan: How to embed images into the stylegan latent space? In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4432–4441 (2019)
  * [2] Abdal, R., Zhu, P., Mitra, N.J., Wonka, P.: Labels4free: Unsupervised segmentation using stylegan. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 13970–13979 (2021)
  * [3] Afifi, M., Brubaker, M.A., Brown, M.S.: Histogan: Controlling colors of gan-generated and real images via color histograms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021)
  * [4] Alharbi, Y., Wonka, P.: Disentangled image generation through structured noise injection. pp. 5134–5142 (2020)
  * [5] Belousov, S.: Mobilestylegan: A lightweight convolutional neural network for high-fidelity image synthesis. arXiv preprint arXiv:2104.04767 (2021)
  * [6] Benaim, S., Wolf, L.: One-sided unsupervised domain mapping. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017), https://proceedings.neurips.cc/paper/2017/file/59b90e1005a220e2ebc542eb9d950b1e-Paper.pdf
  * [7] Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)
  * [8] Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8789–8797 (2018)
  * [9] Choi, Y., Uh, Y., Yoo, J., Ha, J.W.: StarGAN v2: Diverse image synthesis for multiple domains. pp. 8188–8197 (2020)
  * [10] Chong, M.J., Forsyth, D.: Jojogan: One shot face stylization. arXiv preprint arXiv:2112.11641 (2021)
  * [11] Fu, H., Gong, M., Wang, C., Batmanghelich, K., Zhang, K., Tao, D.: Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
  * [12] Gal, R., Patashnik, O., Maron, H., Chechik, G., Cohen-Or, D.: Stylegan-nada: Clip-guided domain adaptation of image generators. arXiv preprint arXiv:2108.00946 (2021)
  * [13] Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. p. 2672â€“2680. NIPS’14, MIT Press, Cambridge, MA, USA (2014)
  * [14] Härkönen, E., Hertzmann, A., Lehtinen, J., Paris, S.: Ganspace: Discovering interpretable gan controls. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 9841–9850. Curran Associates, Inc. (2020), https://proceedings.neurips.cc/paper/2020/file/6fe43269967adbb64ec6149852b5cc3e-Paper.pdf
  * [15] Kafri, O., Patashnik, O., Alaluf, Y., Cohen-Or, D.: Stylefusion: A generative model for disentangling spatial segments. arXiv preprint arXiv:2107.07437 (2021)
  * [16] Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation (2018)
  * [17] Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., Aila, T.: Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems 33, 12104–12114 (2020)
  * [18] Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. pp. 4401–4410 (2019)
  * [19] Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8110–8119 (2020)
  * [20] Kim, H., Choi, Y., Kim, J., Yoo, S., Uh, Y.: Exploiting spatial dimensions of latent in gan for real-time image editing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 852–861 (2021)
  * [21] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [22] Kramberger, T., Potočnik, B.: Lsun-stanford car dataset: Enhancing large-scale car image datasets using deep learning for usage in gan training. Applied Sciences 10(14) (jul 2020). https://doi.org/10.3390/app10144913, https://www.mdpi.com/2076-3417/10/14/4913
  * [23] Kwon, G., Ye, J.C.: Clipstyler: Image style transfer with a single text condition. arXiv preprint arXiv:2112.00374 (2021)
  * [24] Kwon, G., Ye, J.C.: Diagonal attention and style-based gan for content-style disentanglement in image generation and translation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 13980–13989 (2021)
  * [25] Li, Y., Zhang, R., Lu, J., Shechtman, E.: Few-shot image generation with elastic weight consolidation. arXiv preprint arXiv:2012.02780 (2020)
  * [26] Liu, S., Lin, T., He, D., Li, F., Wang, M., Li, X., Sun, Z., Li, Q., Ding, E.: Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6649–6658 (2021)
  * [27] Liu, X., Gong, C., Wu, L., Zhang, S., Su, H., Liu, Q.: Fusedream: Training-free text-to-image generation with improved clip+ gan space optimization. arXiv preprint arXiv:2112.01573 (2021)
  * [28] Menon, S., Damian, A., Hu, S., Ravi, N., Rudin, C.: Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In: Proceedings of the ieee/cvf conference on computer vision and pattern recognition. pp. 2437–2445 (2020)
  * [29] Mo, S., Cho, M., Shin, J.: Freeze the discriminator: a simple baseline for fine-tuning gans. arXiv preprint arXiv:2002.10964 (2020)
  * [30] Noguchi, A., Harada, T.: Image generation from small datasets via batch statistics adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2750–2758 (2019)
  * [31] Ojha, U., Li, Y., Lu, J., Efros, A.A., Lee, Y.J., Shechtman, E., Zhang, R.: Few-shot image generation via cross-domain correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10743–10752 (2021)
  * [32] Park, D.Y., Lee, K.H.: Arbitrary style transfer with style-attentional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5880–5888 (2019)
  * [33] Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision – ECCV 2020. pp. 319–345. Springer International Publishing, Cham (2020)
  * [34] Patashnik, O., Wu, Z., Shechtman, E., Cohen-Or, D., Lischinski, D.: Styleclip: Text-driven manipulation of stylegan imagery. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2085–2094 (2021)
  * [35] Pidhorskyi, S., Adjeroh, D.A., Doretto, G.: Adversarial latent autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14104–14113 (2020)
  * [36] Pinkney, J.N., Adler, D.: Resolution dependent gan interpolation for controllable image synthesis between domains. arXiv preprint arXiv:2010.05334 (2020)
  * [37] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021)
  * [38] Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., Cohen-Or, D.: Encoding in style: A stylegan encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2287–2296 (June 2021)
  * [39] Robb, E., Chu, W.S., Kumar, A., Huang, J.B.: Few-shot adaptation of generative adversarial networks. arXiv preprint arXiv:2010.11943 (2020)
  * [40] Shen, Y., Gu, J., Tang, X., Zhou, B.: Interpreting the latent space of gans for semantic face editing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9240–9249. IEEE Computer Society, Los Alamitos, CA, USA (jun 2020). https://doi.org/10.1109/CVPR42600.2020.00926, https://doi.ieeecomputersociety.org/10.1109/CVPR42600.2020.00926
  * [41] Shi, Y., Aggarwal, D., Jain, A.K.: Lifting 2d stylegan for 3d-aware face generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6258–6266 (June 2021)
  * [42] Svoboda, J., Anoosheh, A., Osendorfer, C., Masci, J.: Two-stage peer-regularized feature recombination for arbitrary image style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13816–13825 (2020)
  * [43] Tewari, A., Elgharib, M., Bharaj, G., Bernard, F., Seidel, H.P., Pérez, P., Zollhofer, M., Theobalt, C.: Stylerig: Rigging stylegan for 3d control over portrait images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6142–6151 (2020)
  * [44] Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., Cohen-Or, D.: Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG) 40(4), 1–14 (2021)
  * [45] Wang, Y., Wu, C., Herranz, L., van de Weijer, J., Gonzalez-Garcia, A., Raducanu, B.: Transferring gans: generating images from limited data. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 218–234 (2018)
  * [46] Yang, T., Ren, P., Xie, X., Zhang, L.: Gan prior embedded network for blind face restoration in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 672–681 (2021)
  * [47] Yu, F., Zhang, Y., Song, S., Seff, A., Xiao, J.: Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 (2015)
  * [48] Zhang, Y., Ling, H., Gao, J., Yin, K., Lafleche, J.F., Barriuso, A., Torralba, A., Fidler, S.: Datasetgan: Efficient labeled data factory with minimal human effort. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10145–10155 (2021)
  * [49] Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (Oct 2017)
  * [50] Zhu, P., Abdal, R., Femiani, J., Wonka, P.: Mind the gap: Domain gap control for single shot domain adaptation for generative adversarial networks. arXiv preprint arXiv:2110.08398 (2021)
  * [51] Zhu, P., Abdal, R., Qin, Y., Femiani, J., Wonka, P.: Improved stylegan embedding: Where are the good latents? arXiv preprint arXiv:2012.09036 (2020)