# Peer is Your Pillar: A Data-unbalanced Conditional GANs for Few-shot Image
Generation

$\text{Ziqiang Li}^{1}\quad\text{Chaoyue Wang}^{2}\quad\text{Xue
Rui}^{1}\quad\text{Chao Xue}^{3}\quad\text{Jiaxu Leng}^{4}\quad\text{Bin
Li}^{1,5}$
${}^{1}\text{Big Data and Decision Lab, University of Science and Technology
of China.}$
${}^{2}\text{The University of Sydney}$ ${}^{3}\text{JD Explore Academic}$
${}^{4}\text{School of computer science, Chongqing University of Posts and
Telecommunications}$
${}^{5}\text{CAS Key Laboratory of Technology in Geo-spatial Information
Processing and Application System.}$
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Corresponding
author: Chaoyue Wang and Bin Li.

###### Abstract

Few-shot image generation aims to train generative models using a small number
of training images. When there are few images available for training (e.g. 10
images), Learning From Scratch (LFS) methods often generate images that
closely resemble the training data while Transfer Learning (TL) methods try to
improve performance by leveraging prior knowledge from GANs pre-trained on
large-scale datasets. However, current TL methods may not allow for sufficient
control over the degree of knowledge preservation from the source model,
making them unsuitable for setups where the source and target domains are not
closely related. To address this, we propose a novel pipeline called Peer is
your Pillar (PIP), which combines a target few-shot dataset with a peer
dataset to create a data-unbalanced conditional generation. Our approach
includes a class embedding method that separates the class space from the
latent space, and we use a direction loss based on pre-trained CLIP to improve
image diversity. Experiments on various few-shot datasets demonstrate the
advancement of the proposed PIP, especially reduces the training requirements
of few-shot image generation.

###### Index Terms:

Few-shot image generation, Generative adversarial networks, Image synthesis.

##  I Introduction

Generative Adversarial Networks (GANs) [7] have achieved remarkable
performance in generating high-fidelity and diverse synthetic images [57, 46],
but rely heavily on large-scale, single-class datasets. For instance,
StyleGAN2 [15] is trained on a dataset of 70K high-resolution facial images.
However, when the available training data is limited [58, 23, 5], GANs tend to
suffer from issues such as discriminator overfitting [49, 12] and latent
discontinuity [26, 18], which can significantly damage the quality and
diversity of the generated images. For few-shot image generation (FSIG), we
expect three desired properties: (i) High fidelity. The generated images
should be photo-realistic. (ii) Enough diversity. The generator should not
simply replicate the training data. (iii) Target-domain consistency. Synthetic
images should have coherent content and style with the few-shot training data.

To address these challenges, previous studies have explored various data
augmentation [53, 56, 12, 41, 10, 28] and regularization [49, 26, 42, 25, 29,
4] techniques for training GANs from scratch, known as the Learning From
Scratch (LFS) approach. Although LFS can produce images with high fidelity and
target-domain consistency, it often lacks enough diversity in the generated
images, resulting in the replication of the training data. To address this
issue, some previous studies [21, 33, 47, 35, 37, 48, 55, 51, 50, 22, 52] have
explored Transfer Learning (TL), which leverages the prior knowledge of GANs
pre-trained on large-scale datasets and adapts it to a small target domain
dataset. While TL can achieve high fidelity, enough diversity, and target-
domain consistency, it requires a semantically related large source domain
dataset to pre-train the model, which can be challenging to obtain in some
domains. Additionally, TL’s creativity is limited without access to the source
data, and it can only accomplish few-shot image generation through extensive
inheritance from the source dataset. For instance, the popular TL method, CDC
[35], cannot adapt the AFHQ-CAT source generator to the 10-shot panda dataset,
as shown in Fig. 1. Overall, TL struggles to transfer domain-independent
knowledge to the target dataset, which is critical for few-shot image
generation.

Figure 1: The Middle part shows the reference image from the 10-shot training
dataset with red box. The Top part displays the generated images and
corresponding metrics (FID [9] and Intra-LPIPS [35]) of the TL baseline CDC
[35] and LFS baseline StyleGAN2-ADA [12]. The large images are generated
images that are Down similar to the reference image on training images, and
the small images are generated images that are Top similar to the reference
image on training images. To generate these images, we first generated 4,000
images and assigned each of them to one of the 10 target samples, forming 10
clusters. We then selected the cluster with the reference image and computed
the LPIPS distance between the images within the reference cluster and the
reference image. The Top similar and Down similar images are the images with
the lowest and highest LPIPS distance, respectively. Note that the Panda
dataset only contains 100 images, so the large synthesis diversity will cause
a significant increase in the FID. Therefore, replicating the training images
on StyleGAN2-ADA produces the best FID results, but the diversity measured by
Intras-LPIPS is poor. The Bottom part shows the generated images of our
method, which exhibit various fidelity levels different from reference images
and achieve competitive FID and Intra-LPIPS metrics.

We propose a new pipeline called Peer is your Pillar (PIP) for few-shot image
generation that eases the requirements of the TL method for semantic relevance
between source and target datasets and a large amount of source dataset. PIP
combines the target few-shot dataset with a peer dataset to create a data-
unbalanced conditional generation task, allowing domain-independent knowledge
to be shared across the two datasets through collaborative training. This
approach offers several advantages: (i) Limited data dependence. Unlike TL
[21, 33, 47], which requires pre-training models on the source domain with
tens of thousands of images, PIP can work with peer datasets that typically
contain 1K or even 0.5K images. (ii) Relaxed semantic similarity. PIP has a
more relaxed semantic similarity requirement compared to TL, making it more
practical. As shown in Fig. 1, PIP is the only method that can achieve good
performance in all three properties on the 10-shot Panda dataset.

The straightforward way to exploit conditional GANs in PIP will introduce
overly strong category conditions, which results in limited knowledge gained
from large-scale peer datasets. To address it, we propose a novel class
embedding method that separates the class space from the latent space. By
reducing the intensity of class injection, we can leverage more knowledge from
large-scale peer datasets, resulting in improved diversity for target few-shot
generation. Additionally, to ensure sufficient diversity, we use the direction
loss [6] realized by pre-trained Contrastive-Language-Image-Pretraining (CLIP)
[36], which guarantees that the image-pair direction is equal to the CLIP-
space text direction between the peer and target domains. Our proposed method
has been validated in terms of fidelity, diversity, and target-domain
consistency across multiple 10-shot generation datasets.

In summary, our contributions are as follows:

  * •

We propose a novel pipeline, PIP, that learns knowledge from peer datasets for
few-shot image generation.

  * •

We design a class embedding method that provides soft condition injection and
use the direction loss realized by CLIP to improve the diversity of generated
images significantly.

  * •

We validate the performance of our method across multiple few-shot generation
datasets, and our approach is the first to achieve diversity generation on the
10-shot Panda.

##  II Related Work

###  II-A Generative Adversarial Networks

Generative Adversarial Networks (GANs) [7] aim to learn the underlying real
distribution by training a generator and a discriminator through a min-max
game. However, GANs are notorious for their instability during training, and
several approaches have been proposed to mitigate this issue, including the
use of efficient architectures [11, 2, 15], loss functions [8, 43],
optimization methods [9, 2], regularization [49, 30, 24], and normalization
[31]. Recently, Style-based generators [14, 15, 13, 38] have shown promising
results, even on high-resolution and diverse datasets such as ImageNet [38].
Another approach is to introduce class information into GANs through class
embedding, which enables conditional generation (cGANs) [2, 15, 38]. However,
the layer-wise structure used by both BigGAN [2] and StyleGANs [15, 38]
involves too much conditional injection, which limits domain-independent
knowledge sharing.

###  II-B Data-efficient GANs

While Generative Adversarial Networks (GANs) have achieved remarkable
performance in image generation with sufficient training data, in some
practical applications, the available training data is limited. To address
this, recent research has focused on developing Data-efficient GANs (DE-GANs)
[27]. DE-GANs aim to fit the entire data distribution using limited training
samples [49, 42, 12, 26, 35]. Based on the scale of the training dataset, DE-
GANs can be divided into two tasks: Limited-data Image Generation and Few-shot
Image Generation. While both tasks have similar settings and motivations, the
main challenges and employed techniques differ.

####  II-B1 Limited-data Image Generation

Limited-data image generation refers to GANs that are specifically designed to
generate high-quality images with limited amounts of training data, typically
1K or 5K samples. One of the major challenges in this area is discriminator
overfitting, which can lead to unstable training and unreasonable decision
boundaries. To address this issue, various techniques have been developed,
such as data augmentation and regularization. Data augmentation is a popular
approach to mitigate overfitting, and several methods [53, 12, 41, 10, 56]
have been proposed for GANs training, including Differentiable Augmentation
(DA) [53], Adaptive Discriminator Augmentation (ADA) [12], Adaptive Pseudo
Augmentation (APA) [10]. Among these, ADA [12] has gained impressive
performance and is widely used in most DE-GANs due to its adaptive strategy
for controlling the strength of augmentations. Regularization [49, 26, 42] is
another effective solution to mitigate discriminator overfitting by
introducing prior and additional tasks. Among them, InsGen [49] and FakeCLR
[26] add contrastive terms to Limited-data generation, stabilizing the
training and achieving impressive results.

####  II-B2 Few-shot Image Generation (FSIG)

Compared to limited-data image generation, few-shot image generation (FSIG)
employs significantly less training data, sometimes as little as one-shot,
10-shot, or 100-shot. While learning from scratch (LFS) methods proposed in
limited-data GANs can mitigate discriminator overfitting and stabilize GAN
training in FSIG, generator collapse tends to replicate training samples,
presenting a major challenge for few-shot image generation. To alleviate
generator collapse, Kong et al. [18] introduce some regularization
technologies for few-shot GANs, which partially relieve the overfitting of LFS
methods, but the generated images’ quality, diversity, and consistency with
the target domain remain limited. Furthermore, most studies [32, 35, 37, 48,
55, 50, 22, 54, 54] rely on transfer learning (TL) to leverage knowledge from
large-scale source datasets. Some of these studies ensure cross-domain
consistency between the source and target domains through a consistency loss
based on Kullback-Leibler (KL) divergence [35, 48] or contrastive term [55] to
improve few-shot generation diversity. Additionally, some studies [6, 51, 59]
leverage the semantic power of large contrastive-language-image-pretraining
(CLIP) [36] models to define the domain-gap direction in CLIP embedding space,
which guides generator optimization. Finally, to leverage prior knowledge from
the source model, some studies have frozen lower layers [32], important
parameters [37, 22, 54], or even the entire set of parameters of the source
model [50, 1, 16]. Despite impressive synthesis quality, TL methods are often
limited in practice, as they require large-scale source domain datasets that
are semantically related to the target domain. Therefore, they perform poorly
on setups where the source and target domains are not closely related.

Figure 2: The architecture of our method. Our architecture differs from the
baseline StyleGAN2 [12] architecture in that we incorporate an additional
lightweight class mapping network to build the class latent space
($\mathcal{C}$), instead of blending the latent code ($z$) and class code
($c$) before a common mapping network. Additionally, we adopt another affine
module to acquire the class vector, which adjusts the synthesis network
through class modulation (c-Mod). To improve the diversity of few-shot
generation and share knowledge between different classes, we use pre-trained
class embeddings and the Key module to control the strength of the class
modulation.

##  III Methodology

The objective of this paper is to generate images with high fidelity, adequate
diversity, and target-domain consistency using only a small amount of training
data. Previous pipelines, such as TL, require large source datasets that are
semantically related to complete the generation task, making it unsuitable for
tasks where the source and target domains are not closely related. To overcome
this limitation, we propose a new pipeline called Peer is Your Pillar (PIP) in
Sec. III-A. To further enhance the realism and diversity of the generated
images, we introduce class embedding and modulation in Sec. III-B and
regularization in Sec. III-C. Our baseline model is StyleGAN2-ADA [12].

###  III-A The Pipeline of Peer is your Pillar (PIP)

Training Generative Adversarial Networks (GANs) can be challenging, especially
when training data is limited. Discriminator overfitting and generator
collapse are common issues in this scenario, particularly for conditional
generation [12, 39]. One solution for few-shot image generation is to acquire
and reuse knowledge from large-scale datasets. However, Transfer Learning (TL)
has limited creativity since it only offers a pre-trained model without access
to the source data. TL’s performance can suffer when working on datasets
without semantically related large source datasets since too much target
domain knowledge needs to be created beyond the source domain. To address this
issue, we propose Peer is your Pillar (PIP), which uses co-training to gain
knowledge from peer datasets to improve performance on target datasets. PIP
trains conditional GANs on a data-unbalanced dataset containing a large-scale
peer dataset and few-shot target datasets to cover the few-shot image
generation task. Our approach shares more knowledge from the peer dataset to
the target dataset, leading to the first diversity generation on tasks where
the peer and target domains are not closely related. Notably, we found that
conditional GANs on data-unbalanced datasets have not met model collapse,
which was observed in data-balanced conditional generation tasks [39].
However, directly adopting the conditional StyleGAN2-ADA [12] has limited
diversity and reality. Therefore, we introduce gradual changes to the baseline
to improve the performance of PIP.

###  III-B Architecture

PIP aims to improve the diversity of few-shot image generation by sharing
knowledge from the peer dataset through data-unbalanced conditional GANs. The
form of class embedding plays a crucial role in controlling the shared
knowledge. In the original StyleGAN2 [15], the latent code ($z$) and class
code ($c$) are blended before a common mapping network, which provides too
much class information and limits the information sharing from the peer
dataset. To address this issue, we introduce two modifications in this
section, namely, Pretrained Class Embedding and Separable Class Modulation, as
illustrated in Fig. 2.

Pretrained Class Embedding. The original conditional GANs use the one-hot
class label to control the sample class and improve overall performance.
However, this approach creates hard labels, and the distance between different
classes is the same. To soften the label, we adopt a pre-trained VGG-19 [40]
network (Pretraining Embedding in Fig. 2), denoted as $E_{p}(\cdot)$, to
extract a 512-dimensional feature vector and calculate the mean as the pre-
trained class embedding of each class ($c_{m}=E_{p}(c)$). This strategy allows
for more knowledge sharing from the peer dataset and improves the diversity of
few-shot generation. This method has also been investigated recently by [3,
38], but we adopt it specifically to share more knowledge from the peer
dataset for few-shot image generation.

Separable Class Modulation. Original conditional GANs [2, 15] inject
conditional information at each resolution of the generated model to ensure
the strength of the class conditions, which is unsuitable for PIP. We aim to
strike a balance between class strength and information sharing, and therefore
propose a separable class modulation method. Specifically, we use two separate
mapping networks, $f_{w}(\cdot)$ and $f_{c}(\cdot)$, to map $z$ and $c$ to
$\mathcal{W}$ and $\mathcal{C}$ spaces, respectively. The class mapping
network $f_{c}$ has fewer layers than the $f_{w}$ network due to the limited
amount of class information. This process can be formalized as:

$\displaystyle
w=f_{w}(z)\in\mathcal{W},\quad\hat{c}_{m}=f_{c}(c_{m})\in\mathcal{C}.$ (1)

TABLE I: FID metric (lower is better) and intra-LPIPS metric (higher is better) over Sunglasses, Sketches, Obama datasets. Apart from MixDL (LFS), StyleGAN2-ADA (LFS), and ours (PIP), other methods (TL) all pretrain on FFHQ and finetune to target datasets. Results with ∗ are obtained when we run the official code of reference. Following [35], intra-LPIPS is computed with 1K generated images.  | Methods | FFHQ$\rightarrow$Sunglasses | FFHQ$\rightarrow$Sketches | FFHQ$\rightarrow$Obama
---|---|---|---|---
| FID ($\downarrow$) | intra-LPIPS ($\uparrow$) | FID ($\downarrow$) | intra-LPIPS ($\uparrow$) | FID ($\downarrow$) | intra-LPIPS ($\uparrow$)
TL | TGAN [45] | $55.61$ | - | $53.42$ | 0.394 | - | -
TGAN+ADA [12] | $53.64$ | - | $66.99$ | 0.414 | - | -
BSA [34] | $76.12$ | - | $69.32$ | - | - | -
FreezeD [32] | $51.29$ | $0.337^{*}$ | $46.54$ | 0.353 | $62.4^{*}$ | $0.19^{*}$
MineGAN [44] | $68.91$ | $0.402^{*}$ | $64.34$ | 0.405 | $75.6^{*}$ | $0.23^{*}$
EWC [22] | $59.73$ | $0.411^{*}$ | $71.25$ | 0.421 | $71.8^{*}$ | $0.24^{*}$
CDC [35] | $42.13$ | $0.562^{*}$ | $45.67$ | 0.453 | $67.2^{*}$ | $0.35^{*}$
DCL [55] | ${38.01}$ | - | ${37.90}$ | 0.486 | - | -
RSSA [48] | $77.77^{*}$ | $0.563^{*}$ | $70.41^{*}$ | $0.534^{*}$ | $88.86^{*}$ | $0.36^{*}$
| ADAM [54] | $32.17^{*}$ | $0.592^{*}$ | $55.74$ | $0.494^{*}$ | $76.99^{*}$ | $0.56^{*}$
LFS | StyleGAN2-ADA [12] | $71.43^{*}$ | $0.245^{*}$ | $48.20^{*}$ | $0.220^{*}$ | $57.39^{*}$ | $0.26^{*}$
MixDL [18] | $61.97^{*}$ | $0.414^{*}$ | $55.40^{*}$ | $0.400^{*}$ | $68.8^{*}$ | $0.53^{*}$
PIP | Ours | 29.28 | 0.668 | 37.40 | 0.440 | 55.52 | 0.59

Accordingly, we transform $w$ and $\hat{c}_{m}$ into style code and class code
using learned affine transformations $A_{s}(\cdot)$ and $A_{c}(\cdot)$,
respectively. The style code is then inserted into the synthesis network
$g(\cdot)$ through the style modulation component (s-Mod) at each convolution
layer, while the class code is controlled by a Key function $K(\cdot)$. Only a
part of the class code is inserted into the synthesis network $g(\cdot)$
through the class modulation component (c-Mod), which improves knowledge
sharing from the peer dataset effectively. This can be formalized as follows:

style code $\displaystyle:=A_{s}(w),\quad\text{class
code}:=K(A_{c}(\hat{c}_{m})),$ (2) s-Mod $\displaystyle:=s_{i}\cdot
w_{i,j,k},\quad\text{c-Mod}:=c_{i}\cdot w_{i,j,k},$

where $w_{i,j,k}$ is original weight of the Synthesis network. $s_{i}$ and
$c_{i}$ are the style scale and class scale corresponding to the $i$-th input
feature map, respectively, and $j$ and $k$ enumerate the output feature maps
and spatial footprint of the convolution, respectively. $K(\cdot)$ is a Key
function that only allows part of the class code to pass and participate in
the c-Mod. Accordingly, the whole weight demodulation of StyleGAN2 has been
replaced in this paper as:

$\displaystyle w_{ijk}^{\prime\prime}$
$\displaystyle=w_{ijk}^{\prime}\bigg{/}\sqrt{\sum_{i,k}w_{ijk}^{\prime}{}^{2}+\epsilon},\
w_{ijk}^{\prime}$ $\displaystyle=s_{i}\cdot c_{i}\cdot w_{i,j,k}.$ (3)

Here,$\epsilon$ is a small constant to avoid numerical issues.
$w_{ijk}^{\prime}$ and $w_{ijk}^{\prime\prime}$ are the modulated and
demodulated weights, respectively. Separating style and class modulations was
also recently investigated by [20]. In contrast to our method, [20] adopts a
hyper-modulation to transfer a pre-trained unconditional GAN to a conditional
GAN and does not have the Key component to reduce the class injection.

###  III-C Adapting Regularization

Regularization is crucial for the training of GANs [42, 49, 26]. In this
section, we introduce a direction regularization to improve the diversity in
few-shot image generation.

Direction Regularization. To further enhance the diversity of few-shot
generation, we adopt direction regularization using a pre-trained CLIP [36]
model. Recent works [51, 6, 19, 59] demonstrate that CLIP-based methods can
provide accurate domain characteristics in TL. In PIP, given the same latent
code $z$, peer label $c_{1}$, and target label $c_{2}$, we aim for the sample-
shift direction $\Delta d_{\text{sample}}$ to be parallel to the domain
direction $\Delta d_{\text{domain}}$. The direction loss is given by:

$\displaystyle\Delta d_{\text{sample}}$
$\displaystyle=E_{I}\left(G(z,c_{1}))-E_{I}(G(z,c_{2})\right),$ (4)
$\displaystyle\Delta d_{\text{domain}}$
$\displaystyle=E_{T}(t_{\text{peer}})-E_{T}(t_{\text{target}}),$
$\displaystyle\mathcal{L}_{\text{direction}}$ $\displaystyle=1-\frac{\Delta
d_{\text{sample}}\cdot\Delta d_{\text{domain}}}{|\Delta
d_{\text{sample}}||\Delta d_{\text{domain}}|},$

where $E_{I}$ and $E_{T}$ refer to pre-trained CLIP image and text encoders,
respectively, while $t_{\text{peer}}$ and $t_{\text{target}}$ represent the
text labels of the peer dataset and target dataset. Our direction loss aligns
the target-generated space with the diverse peer-generated space, which
greatly improves the diversity of the few-shot image generation. Unlike TL,
which fine-tunes the pre-trained generator, PIP trains the generator from
scratch. To prevent excessive direction loss from leading to cGAN training
divergence and degradation, we employ lazy regularization, where direction
loss is performed only once every 16 mini-batches.

In addition, we adopt regularization removal in our model. Path length
regularization and style mixing are regularization techniques used in
unconditional StyleGAN2 on single-class datasets like FFHQ to improve the
continuity and editability of the latent space. However, these regularization
techniques may be unnecessary and even detrimental on multi-modal datasets
[13, 38]. In our PIP setting, we have found that these regularization
technologies are not suitable, and we have therefore removed path length
regularization and style mixing from the original StyleGAN2.

Figure 3: Top-Left:The 10-shot Panda dataset used as the target dataset. We
compare the results of three pipelines and our method in terms of qualitative
(Top-Middle) and quantitative evaluation (Top-Right). Bottom: To evaluate the
visual similarity of the generated images, we generated a large number of
images (size=4,000) and assigned them to one of the 10 target samples (with
the lowest standard LPIPS), resulting in 10 clusters. We then chose the
cluster containing the reference image and computed the LPIPS distance between
the images within that cluster and the reference image. The Top-5 similar and
Bottom-5 similar images are those with the five lowest and highest LPIPS
distances, respectively. The images generated by LFS closely replicated the
reference image, while the images generated by CDC (TL) differed significantly
from the reference image. This indicates that CDC does not perform well when
the source dataset has a large semantic gap with the few-shot dataset. Figure
4: Left Part showcases the latent interpolation of the class space (between
the peer class (z1, c1) and target class (z1, c2)) and the latent space
(between the target class (z1, c2) and (z2, c2)). The amount of knowledge
shared between peer and target classes depends on their domain gap. For
instance, when the domain gap is narrow (e.g., FFHQ $\rightarrow$ Sunglasses),
a significant amount of information, including pose, hairstyle, and facial
features, is shared. When the domain gap is moderate (AFHQ-Cat $\rightarrow$
Panda), global knowledge such as pose is shared. However, when the domain gap
is large, the interpolation exhibits discontinuity, and only limited
information is shared. Furthermore, the generated samples in the target
maintain continuity in their latent space across all settings. Right part
showcases the Intra-LPIPS with training iteration on AFHQ-Cat$\rightarrow$
Panda dataset. TL methods exhibit high diversity in the early stages of the
training process. However, as the training progresses, a gradual decline in
diversity is observed.

##  IV Experiments

In this section, we present the results of our experiments, which involve a
comparison on datasets owning semantically related large source dataset
(discussed in Sec. IV-A) and fixing the target dataset (10-shot Panda) and
vary the peer datasets from a close to large domain gap (from AFHQ-Cat,
Sunglasses, to FFHQ) (discussed in Sec. IV-B), as well as ablation studies
(discussed in Sec. IV-D).

Implementation. We adopt the StyleGAN2 [15] as the backbone for our
experiments, as it is commonly used in few-shot image generation tasks.
Additionally, we use the adaptive data augmentation in StyleGAN2-ADA [12] to
alleviate the discriminator overfitting. We use the official implementation of
StyleGAN2-ADA [12] for our experiments, with the other training parameters and
settings set to their default values.

Datasets. In this section, we present the datasets used for evaluation. In
Sec. IV-A, we evaluate our method on Sketch, FFHQ-Sunglasses, and Obama
datasets, which consist of approximately 300, 2500, and 100 images,
respectively. To simulate few-shot generation setting, we choose 10-shot
images as the training dataset and evaluate the performance of fitting the
true distribution with the entire dataset. For previous transfer learning (TL)
methods, we follow their settings and adopt the model pre-trained on the FFHQ
dataset (which contains 70K images) as the source model and 10-shot images as
the target dataset. In our PIP method, we randomly sample 1000 or 500 images
from the FFHQ dataset [14] as the peer dataset, and 10-shot images as the
target dataset. In Sec. IV-B, we fix the target dataset to 10-shot Panda and
vary the peer datasets from close to large domain gaps (from AFHQ-Cat,
Sunglasses, to FFHQ). Similar to Sec. IV-A, the source datasets (AFHQ-Cat,
Sunglasses, and FFHQ) in TL consist of approximately 5K, 2.5K, and 70K images,
respectively. In PIP, we choose 1K images from the source dataset to conduct
the peer dataset. In the Sec. IV-C, we further evaluate our method on the
Grumpy Cat dataset (using AFHQ-Cat as the peer dataset) and the Anime-face
dataset (using FFHQ as the peer dataset). Both two datasets contain 100
images, and we choose 10-shot images as the training dataset to simulate few-
shot generation setting and evaluate the performance of fitting the true
distribution with the entire dataset. All images in our paper are resized to
256×256.

Evaluation Metrics. We adopt Fréchet Inception Distance (FID) [9] and Intra-
Cluster LPIPS (intra-LPIPS) [35, 55] as the primary evaluation metrics. FID is
the most widely used metric for image generation, and we calculate it between
5,000 generated images and the entire target dataset. Additionally, we use
intra-LPIPS to measure the synthesis diversity. Specifically, we synthesize
1,000 images and assign each synthesized image to the $k$ training images with
the lowest LPIPS distance, forming $k$ clusters. We then calculate the average
LPIPS distance within each cluster and average over all the clusters.

TABLE II: FID and intra-LPIPS metrics over experiments with different/large domain gap. Baseline result are obtained when we run the official code of reference. Our results demonstrate that PIP has greater robustness to increasing domain gaps, producing better results in terms of both reality and diversity compared to current state-of-the-art methods.  | Dataset (LPIPS-EMD) | FFHQ$\rightarrow$ Sunglass (0.68) | Cat$\rightarrow$ Panda (0.71) | Sunglass$\rightarrow$ Panda (0.75) | FFHQ$\rightarrow$ Panda (0.77)
---|---|---|---|---|---
| Methods | FID ($\downarrow$) | i-LPIPS ($\uparrow$) | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS
TL | MineGAN [44] | 68.91 | 0.40 | 67.29 | 0.22 | 70.53 | 0.24 | 69.81 | 0.23
EWC [22] | 59.73 | 0.41 | 53.71 | 0.23 | 66.68 | 0.23 | 63.27 | 0.25
CDC [35] | 42.13 | 0.56 | 32.48 | 0.44 | 35.56 | 0.40 | 31.40 | 0.50
RSSA [48] | 77.77 | 0.56 | 135.0 | 0.52 | 142.9 | 0.42 | 128.8 | 0.49
ADAM [54] | 32.17 | 0.59 | 239.6 | 0.52 | 205.7 | 0.49 | 215.9 | 0.54
PIP | Ours | 27.83 | 0.67 | 15.67 | 0.52 | 21.77 | 0.50 | 22.13 | 0.54
TABLE III: Ablation study on different components of our method.  Dataset | Sunglasses | Sketches | Obama
---|---|---|---
Configuration | FID $\downarrow$ | intra-LPIPS $\uparrow$ | FID | intra-LPIPS | FID | intra-LPIPS
$\mathbf{A}$ StyleGAN2-ADA (U) | 71.43 | 0.25 | 48.20 | 0.22 | 57.39 | 0.26
$\mathbf{B}$ StyleGAN2-ADA (C) | 44.12 | 0.56 | 39.84 | 0.37 | 73.77 | 0.50
$\mathbf{C}$ $+$ Separable class modulation | 34.51 | 0.62 | 48.17 | 0.39 | 68.69 | 0.52
$\mathbf{D}$ $+$ Pretrained class embedding | 32.53 | 0.62 | 46.12 | 0.41 | 58.38 | 0.53
$\mathbf{E}$ $+$ Regularization removing | 31.83 | 0.64 | 41.28 | 0.40 | 60.19 | 0.54
$\mathbf{F}$ $+$ Direction regularization | 29.28 | 0.67 | 37.40 | 0.44 | 55.52 | 0.59
Figure 5: Left: Target samples on 10-shot Sketch and Sunglasses datasets. Mid:
We showcase the generated results of three pipelines. StyleGAN 2-ADA (LFS)
tends to replicate the training data, while MixDL (LFS) can be used as a
regularization method to mitigate this issue slightly. CDC (TL) is pretrained
on a large-scale source dataset, which provides enough diversity and realism.
Our method (PIP) combines 1k images from a peer dataset and the 10-shot target
dataset, effectively acquiring shared knowledge from the peer dataset. This
approach meets all three desired properties of few-shot image generation
Right: Metrics (FID and Intra-LPIPS) of different methods. Figure 6: Generated
images on Obama and Grumpy Cat datasets. Middle line is the reference image on
10-shot training dataset. The other lines show the generated images and
corresponding metrics (FID and Intra-LPIPS) of both baselines and our method.
The big images are generated images Down similar to reference image on
training images and the small images are generated images Top similar to
reference image on training images. Specifically, we generated a large number
of images (size=4,000) and assigned them to one of the 10 target samples (with
the lowest standard LPIPS), resulting in 10 clusters. We then chose the
cluster containing the reference image and computed the LPIPS distance between
the images within that cluster and the reference image. The Top similar and
Bottom similar images are those with the lowest and highest LPIPS distances,
respectively.

###  IV-A Results on Datasets Owning Semantically Related Large Source Dataset

In experiments on datasets with semantically related large source dataset, we
select three common datasets, Sunglasses, Sketches, and Obama. Table I
presents a comparison of FID and intra-LPIPS with previous studies. We
compared our results with the most recent TL and LFS methods. LFS methods,
such as [12, 18], showed poor performance on all datasets, as measured by both
FID and intra-LPIPS. On the other hand, TL methods showed significant
improvement on these datasets, especially for intra-LPIPS on the Sketches
dataset, with the use of semantically related large source datasets. Notably,
our proposed method, which only uses 1K peer data, achieves remarkable results
on most datasets. Furthermore, we present a qualitative comparison of our PIP
method with a popular baseline on three tasks: FFHQ $\rightarrow$ Sketch (top
part of Fig. 5), FFHQ $\rightarrow$ Sunglasses (bottom part of Fig. 5), and
FFHQ $\rightarrow$ Obama (top part of Fig. 6 ). Our PIP method has
demonstrated significant improvement in terms of both realism and diversity.

###  IV-B Experiments with different/large domain gap

TABLE IV: Ablation study on size of peer dataset with different domain gap.  Dataset (LPIPS-EMD) | FFHQ $\rightarrow$ Sunglass (0.68) | Cat $\rightarrow$ Panda (0.71) | Sunglass $\rightarrow$ Panda(0.75) | FFHQ $\rightarrow$ Panda (0.77)
---|---|---|---|---
Size of peer dataset | FID ($\downarrow$) | i-LPIPS ($\uparrow$) | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS
200 | 50.49 | 0.59 | 26.16 | 0.48 | 35.36 | 0.45 | 110.29 | 0.31
500 | 41.99 | 0.60 | 21.23 | 0.51 | 24.42 | 0.48 | 27.28 | 0.50
1000 | 29.28 | 0.67 | 15.67 | 0.52 | 21.77 | 0.50 | 22.13 | 0.54
2000 | 27.83 | 0.67 | 88.19 | 0.49 | 92.34 | 0.45 | 104.31 | 0.40
TABLE V: Ablation study on the key value of class modulation. LPIPS-EMD indicates the domain gap between peer and target domains. The size of peer dataset is set to 1000 in this experiment. Dataset (LPIPS-EMD) |  FFHQ$\rightarrow$ Obama (0.68) |  FFHQ$\rightarrow$ Sunglass (0.68) |  Cat$\rightarrow$ Panda (0.71) |  Sunglass$\rightarrow$ Panda (0.75) |  FFHQ $\rightarrow$ Panda (0.77)
---|---|---|---|---|---
Key value |  FID ($\downarrow$) |  i-LPIPS ($\uparrow$) | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS
$\mathbf{A}:$ 4 | 73.28 | 0.53 | 29.28 | 0.67 | 27.19 | 0.55 | 33.61 | 0.51 | 34.58 | 0.56
$\mathbf{B}:$ 8 | 76.89 | 0.55 | 30.57 | 0.62 | 31.57 | 0.56 | 40.46 | 0.52 | 40.18 | 0.56
$\mathbf{D}:$ 4+8 | 65.93 | 0.48 | 32.60 | 0.61 | 22.90 | 0.54 | 28.90 | 0.51 | 30.52 | 0.54
$\mathbf{E}:$ 4+16 | 71.77 | 0.57 | 34.10 | 0.61 | 21.80 | 0.54 | 32.21 | 0.51 | 33.82 | 0.54
$\mathbf{F}:$ 4+8+16 | 70.31 | 0.53 | 31.45 | 0.62 | 19.89 | 0.53 | 26.19 | 0.51 | 27.44 | 0.54
$\mathbf{G}:$ 4+8+16+64+128+256 | 55.52 | 0.59 | 45.22 | 0.60 | 15.67 | 0.52 | 21.77 | 0.50 | 22.13 | 0.54

In this section, we investigate the effectiveness of our proposed PIP method
on the 10-shot Panda dataset with peer datasets that vary from close to large
domain gaps (AFHQ-Cat, Sunglasses, to FFHQ). We use LPIPS-based Earth Movers
Distance (EMD) as a basic measurement of the domain gap. EMD is a common
distribution distance that has been used in many fields [17]. To calculate the
EMD, we randomly sample $m$ (1000) source images and $n$ (100) target images
as the source domain set $I_{S}$ and the target domain set $I_{T}$,
respectively. We extract the intermediate features of $I_{S}$ and $I_{T}$ from
the image encoder $\mathcal{E}$ used in LPIPS. The LPIPS-based EMD is the mean
LPIPS distance for each source feature from $F_{S}$ with its closest target
feature from $F_{T}$, where $F_{S}={F^{1}_{S},\cdots,F^{m}_{S}}$ and
$F_{T}={F^{1}_{T},\cdots,F^{n}_{T}}$ are the extracted intermediate features
through $\mathcal{E}$. The LPIPS-EMD is defined as:

$\text{LPIPS-
EMD}=\max\left(\frac{1}{n}\sum_{i}\min_{j}\bm{C}_{i,j},\frac{1}{m}\sum_{j}\min_{i}\bm{C}_{i,j}\right),$
(5)

where $\bm{C}$ is the LPIPS matrix to measure the sample-wise distances from
$F_{S}$ to $F_{T}$, and each element is computed as:
$\bm{C}_{i,j}=\text{LPIPS}(F^{i}_{S},F^{j}_{T})$. Table II shows that PIP
outperforms state-of-the-art TL methods consistently, although its performance
decreases as the domain gap increases. We further provide qualitative
comparisons and Intra-LPIPS with training iterations on the AFHQ-
Cat$\rightarrow$Panda dataset in Figs. 3 and 4, respectively, which
demonstrate the effectiveness of our PIP pipeline and our method. Finally, we
show the latent interpolations in class and latent spaces on datasets with
different domain gaps in Fig. 4. Our results indicate that the extent of
knowledge sharing between peer and target classes is adaptively contingent
upon their domain gap in our PIP, which is the main factor behind the
advancement of our method.

TABLE VI: FID and intra-LPIPS metrics over Grumpy Cat and Anime-face datasets. Baseline result are obtained when we run the official code of reference. Note that the Grumpy Cat and Anime-face datasets only contains 100 images, the large synthesis diversity will cause a great increase in the FID. Therefore, replicating the training images on StyleGAN2-ADA has the competitive FID results. However, the diversity measured by intras-LPIPS is poor for StyleGAN2-ADA on all datasets.  | Methods | Grumpy Cat | Anime-face
---|---|---|---
|  FID ($\downarrow$) |  intra-LPIPS ($\downarrow$) | FID | intra-LPIPS
LFS |  StyleGAN2-ADA [12] | 94.89 | 0.18 | 79.72 | 0.22
MixDL [18] | 37.54 | 0.48 | 82.34 | 0.46
TL |  CDC [35] | 39.52 | 0.51 | 95.24 | 0.48
RSSA [48] | 49.72 | 0.53 | 112.5 | 0.49
PIP | ours | 33.90 | 0.58 | 78.88 | 0.52

###  IV-C Experiments on Grumpy Cat and Anime-face Datasets

In this section, we illustrate the quantitative (Tab VI) and qualitative
comparison on Grumpt Cat (bottom part of Fig. 6) and Anime-face (Fig. 7)
datasets. In this case, other methods have limited diversity and reality, and
our method acquires significant improvement.

Figure 7:  Top-Left:The 10-shot Anime-face dataset used as the target dataset.
We compare the results of three pipelines and our method in terms of
qualitative (Top-Middle) and quantitative evaluation (Top-Right). Bottom: To
evaluate the visual similarity of the generated images, we generated a large
number of images (size=4,000) and assigned them to one of the 10 target
samples (with the lowest standard LPIPS), resulting in 10 clusters. We then
chose the cluster containing the reference image and computed the LPIPS
distance between the images within that cluster and the reference image. The
Top-5 similar and Bottom-5 similar images are those with the five lowest and
highest LPIPS distances, respectively. The images generated by LFS closely
replicated the reference image, while the images generated by CDC (TL)
differed significantly from the reference image. This indicates that CDC does
not perform well when the source dataset has a large semantic gap with the
few-shot dataset.

###  IV-D Ablation Study

Effect of Different Components in Our Method. Ablation studies conducted on
the various components of our proposed method are presented in Table III. The
PIP pipeline (Config-B) was found to improve diversity but decrease reality
when compared to the LFS pipeline (Config-A). However, the four technologies
proposed in Config-C, D, E, and F significantly improved both reality and
diversity in the generated images. Our method (Config-F) achieved an
impressive FID improvement of $59\%$, $22.4\%$, and $3.3\%$ on the Sunglasses,
Sketches, and Obama datasets, respectively, over StyleGAN2-ADA (U).
Importantly, our method (Config-F) also achieved significant intra-LPIPS
improvements on all datasets compared to the LFS method StyleGAN2-ADA (U).

Effect of the Size of Peer Dataset. When the peer dataset is excessively
large, the PIP model faces an imbalanced conditional generation task between
the peer and few-shot target datasets. This causes the model to focus
excessively on generating the peer dataset, thereby compromising its ability
to generate the few-shot target dataset. Consequently, a large peer dataset is
unnecessary for our PIP method, which is a significant advantage over TL
methods that require a substantial source dataset to pre-train the source
generator, making it challenging to implement in real-world scenarios. In our
PIP method, determining the appropriate size of the peer dataset is
straightforward. As demonstrated in Table IV, excellent results can be
achieved across all datasets by limiting the size of the peer dataset to 1K.

Effect of the Key Value of Class Modulation. The function K($\cdot$) serves as
a gate function that determines whether class modulation should be
implemented. Key value equals ’4’ means that class injection is implemented
only at the $4\times 4$ feature layer of the generator. When the domain gap is
large, more class injection is needed to improve performance. As shown in
Table V, a key value of ’4+8+16+64+128+256’ is used for Cat/Sunglasses/FFHQ
$\rightarrow$ Panda datasets. Conversely, when the domain gap is narrow, a key
value of ’4’ is sufficient, and using too much injection can reduce diversity
and performance. Notably, transferring multiple identities to a specific
person is challenging for both TL-based methods and our PIP, resulting in
FFHQ$\rightarrow$Obama being an outlier in the monotonic trend. In summary,
for settings with a narrow domain gap (FFHQ$\rightarrow$Sunglasses) or easy to
complete (FFHQ$\rightarrow$Sketch), the key value is set to ’4’. For settings
with a moderate domain gap (FFHQ $\rightarrow$Anime-face and AFHQ
Cat$\rightarrow$Grumpy Cat), the key value is set to ’4+16’. For settings with
a large domain gap (AFHQ-Cat/Sunglasses /FFHQ $\rightarrow$ Panda) or
challenging to complete (FFHQ $\rightarrow$Obama), the key value is set to
’4+8+16+64+128+256’.

TABLE VII: Ablation on Pre-trained technologies. $*$ indicates that we set the class injection with resolution 4+8+16+64+128+256 instand of 4+16 on Panda generation task. The other settings are the same as original paper (Size of peer dataset is 1000). Source dataset | FFHQ | AFHQ-Cat
---|---|---
Target datset | Babies | Sunglasses | Sketches | Obama | Anime-face | Grumpy Cat | $\text{Panda}^{*}$
Configuration |  FID($\downarrow$) |  i-LPIPS($\uparrow$) | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS | FID | i-LPIPS
CDC | 74.39 | 0.57 | 42.13 | 0.56 | 45.67 | 0.45 | 67.2 | 0.35 | 95.24 | 0.48 | 39.52 | 0.51 | 32.48 | 0.44
RSSA | 63.44 | 0.58 | 77.77 | 0.56 | 70.41 | 0.53 | 88.86 | 0.36 | 112.5 | 0.49 | 49.72 | 0.53 | 135.0 | 0.52
w/o VGG+w/o CLIP | 72.75 | 0.62 | 34.19 | 0.63 | 43.32 | 0.42 | 64.67 | 0.56 | 84.29 | 0.49 | 35.02 | 0.55 | 28.32 | 0.48
w/ VGG+w/o CLIP | 66,32 | 0.60 | 31.83 | 0.64 | 41.28 | 0.40 | 60.19 | 0.54 | 82.31 | 0.51 | 34.27 | 0.54 | 20.38 | 0.49
w/ VGG+w/ CLIP | 61.84 | 0.66 | 29.28 | 0.67 | 37.40 | 0.44 | 55.52 | 0.59 | 78.88 | 0.52 | 33.90 | 0.58 | 15.67 | 0.52
TABLE VIII: Time and space cost of introducing the pre-trained model during training and inference.  Backbones | Training | Inference
---|---|---
| w/ CLIP&VGG | w/o CLIP&VGG | w/ CLIP&VGG | w/o CLIP&VGG
Time cost (Second/kimg) | 11.05 | 10.53 | 6.55 | 6.54
Space cost (G) | 32 | 25 | 17 | 16

Effect of the Pre-trained class embedding and CLIP. Our proposed Progressive
Image Processing (PIP) approach necessitates the utilization of pre-trained
class embedding and Contrastive Language–Image Pretraining (CLIP) models. This
characteristic sets it apart from conventional train-from-scratch
methodologies. Moreover, the prerequisite for pre-trained models within our
PIP framework is less intricate to satisfy compared to traditional Transfer
Learning (TL) techniques. Initially, the pre-trained CLIP technique finds
widespread employment in TL methodologies, as evidenced by its application in
recent works such as [51, 52]. Furthermore, the VGG model adopted in our study
has undergone pre-training on the ImageNet dataset, while CLIP has been pre-
trained on a collection of image-text pairs. This pre-training renders both
models versatile and amenable to deployment across diverse datasets. Notably,
both the CLIP and VGG models are conveniently accessible online and obviate
the need for supplementary training efforts.

In order to comprehensively assess the impact of pre-trained VGG and CLIP
models on our PIP approach, we conducted an ablation study involving various
pre-training techniques, the results of which are detailed in Table VII. It is
imperative to acknowledge that although the integration of pre-trained CLIP
and VGG models demonstrably enhances the performance of our approach, the
methodology lacking pre-trained class embedding and CLIP regularization still
yields noteworthy improvements over conventional TL methodologies.

Moreover, we emphasize that the incorporation of pre-trained VGG and CLIP
encoders within our approach yields compelling cost-effectiveness. The CLIP
embedding is exclusively indispensable during the training phase and exerts
negligible adverse ramifications on time and memory consumption during
inference. Simultaneously, the VGG network’s lightweight nature enables
seamless forward propagation, incurring minimal temporal costs. To
substantiate our assertions, we provide a comprehensive breakdown of the
temporal and spatial costs incurred by the integration of pre-trained models
during both the training and inference phases, as documented in Table VIII.
All experimental trials were executed on a hardware setup comprising 2 A100
GPUs.

##  V Limitation and Conclusion

Limitation. Although our proposed method has achieved remarkable generation
results on datasets without semantically related large source dataset, it has
not outperformed state-of-the-art methods with TL pipeline in terms of reality
and diversity on some datasets with semantically related large source dataset.
For example, as shown in Table I, our method shows poorer diversity compared
to state-of-the-art methods on the Sketch dataset. Another limitation of our
method is that it relies on layer-wise information injection in the generator,
where the latent code is fed into all convolution layers. Fortunately, this
layer-wise structure is commonly used in state-of-the-art GANs such as [2, 14,
15, 38]. Additionally, our Key value is manually set through experience and
may not be suitable for all datasets. An adaptive method may further improve
the performance.

Conclusion. Our paper proposes a novel few-shot image generation pipeline
called Peer is your Pillar (PIP), which combines a target few-shot dataset
with a peer dataset to create a data-unbalanced conditional generation. To
enhance the diversity of the generated images, we also present several new
techniques including Pre-trained Class Embedding, Separable Class Modulation,
and Direction Regularization. Our approach is uniquely capable of adapting to
different knowledge preservation levels from the source model, making it ideal
for scenarios where the source and target domains have varying degrees of
relatedness. Through qualitative and quantitative evaluations, we demonstrate
that our approach significantly outperforms previous methods and achieves all
three desired properties in few-shot image generation.

## Acknowledgments

The work was supported in part by the National Natural Science Foundation of
China under Grands U19B2044 and 61836011

## References

  * [1] Aibek Alanov, Vadim Titov, and Dmitry P Vetrov.  Hyperdomainnet: Universal domain adaptation for generative adversarial networks.  Advances in Neural Information Processing Systems, 35:29414–29426, 2022.
  * [2] Andrew Brock, Jeff Donahue, and Karen Simonyan.  Large scale gan training for high fidelity natural image synthesis.  arXiv preprint arXiv:1809.11096, 2018.
  * [3] Arantxa Casanova, Marlene Careil, Jakob Verbeek, Michal Drozdzal, and Adriana Romero Soriano.  Instance-conditioned gan.  Advances in Neural Information Processing Systems, 34:27517–27529, 2021.
  * [4] Yuanqi Chen, Cece Jin, Ge Li, Thomas H Li, and Wei Gao.  Mitigating label noise in gans via enhanced spectral normalization.  IEEE Transactions on Circuits and Systems for Video Technology, 2023\.
  * [5] Zhuohang Dang, Minnan Luo, Chengyou Jia, Caixia Yan, Xiaojun Chang, and Qinghua Zheng.  Counterfactual generation framework for few-shot learning.  IEEE Transactions on Circuits and Systems for Video Technology, 2023\.
  * [6] Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, and Daniel Cohen-Or.  Stylegan-nada: Clip-guided domain adaptation of image generators.  arXiv preprint arXiv:2108.00946, 2021.
  * [7] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Advances in Neural Information Processing Systems, pages 2672–2680, 2014.
  * [8] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.  Improved training of wasserstein gans.  In Advances in Neural Information Processing Systems, pages 5767–5777, 2017.
  * [9] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  arXiv preprint arXiv:1706.08500, 2017.
  * [10] Liming Jiang, Bo Dai, Wayne Wu, and Chen Change Loy.  Deceive d: Adaptive pseudo augmentation for gan training with limited data.  In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
  * [11] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of gans for improved quality, stability, and variation.  arXiv preprint arXiv:1710.10196, 2017.
  * [12] Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila.  Training generative adversarial networks with limited data.  Advances in Neural Information Processing Systems, 33, 2020.
  * [13] Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Alias-free generative adversarial networks.  Advances in Neural Information Processing Systems, 34:852–863, 2021\.
  * [14] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
  * [15] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8110–8119, 2020.
  * [16] Seongtae Kim, Kyoungkook Kang, Geonung Kim, Seung-Hwan Baek, and Sunghyun Cho.  Dynagan: Dynamic few-shot adaptation of gans to multiple domains.  In SIGGRAPH Asia 2022 Conference Papers, pages 1–8, 2022.
  * [17] Nicholas Kolkin, Jason Salavon, and Gregory Shakhnarovich.  Style transfer by relaxed optimal transport and self-similarity.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10051–10060, 2019.
  * [18] Chaerin Kong, Jeesoo Kim, Donghoon Han, and Nojun Kwak.  Smoothing the generative latent space with mixup-based distance learning.  arXiv preprint arXiv:2111.11672, 2021.
  * [19] Gihyun Kwon and Jong Chul Ye.  One-shot adaptation of gan in just one clip.  arXiv preprint arXiv:2203.09301, 2022.
  * [20] Héctor Laria, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu.  Transferring unconditional to conditional gans with hyper-modulation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3840–3849, 2022.
  * [21] Bonan Li, Zicheng Zhang, Xuecheng Nie, Congying Han, Yinhan Hu, and Tiande Guo.  Styo: Stylize your face in only one-shot.  arXiv preprint arXiv:2303.03231, 2023.
  * [22] Yijun Li, Richard Zhang, Jingwan (Cynthia) Lu, and Eli Shechtman.  Few-shot image generation with elastic weight consolidation.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 15885–15896. Curran Associates, Inc., 2020.
  * [23] Zechao Li, Hao Tang, Zhimao Peng, Guo-Jun Qi, and Jinhui Tang.  Knowledge-guided semantic transfer network for few-shot image recognition.  IEEE Transactions on Neural Networks and Learning Systems, 2023\.
  * [24] Ziqiang Li, Muhammad Usman, Rentuo Tao, Pengfei Xia, Chaoyue Wang, Huanhuan Chen, and Bin Li.  A systematic survey of regularization and normalization in gans.  ACM Comput. Surv., nov 2022.  Just Accepted.
  * [25] Ziqiang Li, Muhammad Usman, Rentuo Tao, Pengfei Xia, Chaoyue Wang, Huanhuan Chen, and Bin Li.  A systematic survey of regularization and normalization in gans.  ACM Computing Surveys, 55(11):1–37, 2023.
  * [26] Ziqiang Li, Chaoyue Wang, Heliang Zheng, Jing Zhang, and Bin Li.  Fakeclr: Exploring contrastive learning for solving latent discontinuity in data-efficient gans.  In European Conference on Computer Vision, pages 598–615. Springer, 2022.
  * [27] Ziqiang Li, Xintian Wu, Beihao Xia, Jing Zhang, Chaoyue Wang, and Bin Li.  A comprehensive survey on data-efficient gans in image generation.  arXiv preprint arXiv:2204.08329, 2022.
  * [28] Ziqiang Li, Pengfei Xia, Xue Rui, and Bin Li.  Exploring the effect of high-frequency components in gans training.  ACM Transactions on Multimedia Computing, Communications and Applications, 19(5):1–22, 2023.
  * [29] Ziqiang Li, Pengfei Xia, Rentuo Tao, Hongjing Niu, and Bin Li.  A new perspective on stabilizing gans training: Direct adversarial training.  IEEE Transactions on Emerging Topics in Computational Intelligence, 7(1):178–189, 2023.
  * [30] Lars Mescheder, Sebastian Nowozin, and Andreas Geiger.  The numerics of gans.  In Advances in Neural Information Processing Systems, pages 1825–1835, 2017.
  * [31] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida.  Spectral normalization for generative adversarial networks.  arXiv preprint arXiv:1802.05957, 2018.
  * [32] Sangwoo Mo, Minsu Cho, and Jinwoo Shin.  Freeze the discriminator: a simple baseline for fine-tuning gans.  arXiv preprint arXiv:2002.10964, 2020.
  * [33] Yotam Nitzan, Michaël Gharbi, Richard Zhang, Taesung Park, Jun-Yan Zhu, Daniel Cohen-Or, and Eli Shechtman.  Domain expansion of image generators.  arXiv preprint arXiv:2301.05225, 2023.
  * [34] Atsuhiro Noguchi and Tatsuya Harada.  Image generation from small datasets via batch statistics adaptation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2750–2758, 2019.
  * [35] Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A Efros, Yong Jae Lee, Eli Shechtman, and Richard Zhang.  Few-shot image generation via cross-domain correspondence.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10743–10752, 2021.
  * [36] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In International Conference on Machine Learning, pages 8748–8763. PMLR, 2021.
  * [37] Esther Robb, Wen-Sheng Chu, Abhishek Kumar, and Jia-Bin Huang.  Few-shot adaptation of generative adversarial networks.  arXiv preprint arXiv:2010.11943, 2020.
  * [38] Axel Sauer, Katja Schwarz, and Andreas Geiger.  Stylegan-xl: Scaling stylegan to large diverse datasets.  In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022.
  * [39] Mohamad Shahbazi, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool.  Collapse by conditioning: Training class-conditional gans with limited data.  arXiv preprint arXiv:2201.06578, 2022.
  * [40] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [41] Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Trung-Kien Nguyen, and Ngai-Man Cheung.  Towards good practices for data augmentation in gan training.  arXiv preprint arXiv:2006.05338, 2020.
  * [42] Hung-Yu Tseng, Lu Jiang, Ce Liu, Ming-Hsuan Yang, and Weilong Yang.  Regularizing generative adversarial networks under limited data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7921–7931, 2021.
  * [43] Chaoyue Wang, Chang Xu, Xin Yao, and Dacheng Tao.  Evolutionary generative adversarial networks.  IEEE Transactions on Evolutionary Computation, 23(6):921–934, 2019\.
  * [44] Yaxing Wang, Abel Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer.  Minegan: Effective knowledge transfer from gans to target domains with few images.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [45] Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia, and Bogdan Raducanu.  Transferring gans: generating images from limited data.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 218–234, 2018.
  * [46] Yixuan Wang, Wengang Zhou, Jianmin Bao, Weilun Wang, Li Li, and Houqiang Li.  Clip2gan: Towards bridging text with the latent space of gans.  IEEE Transactions on Circuits and Systems for Video Technology, 2023\.
  * [47] Yi Wu, Ziqiang Li, Chaoyue Wang, Heliang Zheng, Shanshan Zhao, Bin Li, and Dacheng Ta.  Domain re-modulation for few-shot generative domain adaptation.  arXiv preprint arXiv:2302.02550, 2023.
  * [48] Jiayu Xiao, Liang Li, Chaofei Wang, Zheng-Jun Zha, and Qingming Huang.  Few shot generative model adaption via relaxed spatial structural alignment.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11204–11213, 2022.
  * [49] Ceyuan Yang, Yujun Shen, Yinghao Xu, and Bolei Zhou.  Data-efficient instance generation from instance discrimination.  arXiv preprint arXiv:2106.04566, 2021.
  * [50] Ceyuan Yang, Yujun Shen, Zhiyi Zhang, Yinghao Xu, Jiapeng Zhu, Zhirong Wu, and Bolei Zhou.  One-shot generative domain adaptation.  arXiv preprint arXiv:2111.09876, 2021.
  * [51] Yabo Zhang, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Wangmeng Zuo, et al.  Towards diverse and faithful one-shot adaption of generative adversarial networks.  In Advances in Neural Information Processing Systems, 2022.
  * [52] Zicheng Zhang, Yinglu Liu, Congying Han, Tiande Guo, Ting Yao, and Tao Mei.  Generalized one-shot domain adaption of generative adversarial networks.  arXiv preprint arXiv:2209.03665, 2022.
  * [53] Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han.  Differentiable augmentation for data-efficient gan training.  Advances in Neural Information Processing Systems, 33, 2020.
  * [54] Yunqing Zhao, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Man Cheung.  Few-shot image generation via adaptation-aware kernel modulation.  Advances in Neural Information Processing Systems, 35:19427–19440, 2022.
  * [55] Yunqing Zhao, Henghui Ding, Houjing Huang, and Ngai-Man Cheung.  A closer look at few-shot image generation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9140–9150, 2022.
  * [56] Zhengli Zhao, Zizhao Zhang, Ting Chen, Sameer Singh, and Han Zhang.  Image augmentations for gan training.  arXiv preprint arXiv:2006.02595, 2020.
  * [57] Dongliang Zhou, Haijun Zhang, Jianghong Ma, and Jianyang Shi.  Bc-gan: A generative adversarial network for synthesizing a batch of collocated clothing.  IEEE Transactions on Circuits and Systems for Video Technology, 2023\.
  * [58] Fei Zhou, Lei Zhang, and Wei Wei.  Meta-generating deep attentive metric for few-shot classification.  IEEE Transactions on Circuits and Systems for Video Technology, 32(10):6863–6873, 2022.
  * [59] Peihao Zhu, Rameen Abdal, John Femiani, and Peter Wonka.  Mind the gap: Domain gap control for single shot domain adaptation for generative adversarial networks.  arXiv preprint arXiv:2110.08398, 2021.