# Generative Active Learning for Image Synthesis Personalization

Xulu Zhang12, Wengyu Zhang1, Xiao-Yong Wei31, Jinlin Wu25, Zhaoxiang Zhang245,
Zhen Lei245, Qing Li1,

1Department of Computing, the Hong Kong Polytechnic University, Hong Kong
2Center for Artificial Intelligence and Robotics, HKISI, CAS, Hong Kong
3College of Computer Science, Sichuan University, Chengdu, China  4School of
Artificial Intelligence, UCAS, Beijing, China  5State Key Laboratory of
Multimodal Artificial Intelligence Systems, CASIA, Beijing, China

###### Abstract

This paper presents a pilot study that explores the application of active
learning, traditionally studied in the context of discriminative models, to
generative models. We specifically focus on image synthesis personalization
tasks. The primary challenge in conducting active learning on generative
models lies in the open-ended nature of querying, which differs from the
closed form of querying in discriminative models that typically target a
single concept. We introduce the concept of anchor directions to transform the
querying process into a semi-open problem. We propose a direction-based
uncertainty sampling strategy to enable generative active learning and tackle
the exploitation-exploration dilemma. Extensive experiments are conducted to
validate the effectiveness of our approach, demonstrating that an open-source
model can achieve superior performance compared to closed-source models
developed by large companies, such as Google’s StyleDrop. The source code is
available at https://github.com/zhangxulu1996/GAL4Personalization.

##  I Introduction

Recently, generative models, such as large language models (e.g., ChatGPT [1],
Llama [2]) and image generation models DALL•E [3], Stable Diffusion [4]), have
demonstrated impressive capabilities in producing compelling and diverse
results. The key to the success lies in the availability of high-quality
training samples on an incredibly large scale. In addition to the real-world
datasets that are expensive to collect, numerous studies [5, 6] have
demonstrated that incorporating synthetic samples can effectively improve the
capability and generalization of models. However, as the number of generated
samples can be extensive and of varying quality, a crucial question arises:
how can we select the most informative samples with minimal cost for training?
This issue has been extensively discussed in the field of active learning,
which attempts to maximize a model’s performance while annotating the fewest
samples [7]. However, traditional active learning approaches primarily focus
on improving discriminative models. The application of active learning in
generative models, particularly in utilizing synthetic samples to enhance
model performance, remains an open and challenging research area.

In this paper, we present a pilot study on the application of active learning
in generative models, specifically focusing on the image synthesis
personalization (ISP) [8]. ISP is a representative family of generative tasks
that requires the cost-effective selection of synthetic data for training. The
learning objective of ISP is to model the user’s “subject of interest” (SoI)
based on a limited number of reference images and generate new images that
feature the SoI. For instance, in the case of learning from a few images of
the user’s pet cat, the trained model should be capable of generating diverse
scenes with the cat, such as the cat being situated under the Eiffel Tower or
flying alongside Trump, depending on the given prompt [9]. Similarly, when the
SoI revolves around a specific style, like Van Gogh paintings or the user’s
own artwork, the ISP model should be able to adopt that style and generate new
images with the same artistic characteristics [10]. Given the highly specific
nature of personal interests, the availability of reference images is often
limited. Therefore, selecting good samples from the newly generated images to
augment the reference set has proven to be a more practical approach [10].
This can be done in an iterative manner, which aligns well with the framework
of active learning.

While the idea of bringing active learning from discriminative models to
generative models holds promise, it also presents several challenges. One key
challenge is the causal loop in the querying strategy design. In
discriminative active learning (DAL), informative samples are selected and
queried from a closed set of unlabeled data, typically for tasks like
recognizing predefined simple concepts (e.g., dog). This closed set nature
makes it feasible to design strategies that compare the information carried by
different unlabeled samples (e.g., entropy in uncertainty sampling) so as to
prioritize directions in the feature space for querying. In contrast,
generative active learning (GAL) faces a scenario where the querying is open
to all directions, because the user may combine the SoI with all possible
prompts, which can carry much more complex and undetermined semantics. This
openness makes the sample-evaluation-based DAL querying strategies infeasible
in GAL. It is due to the fact that generated samples are not readily available
unless prompts are given, and it is not easy to design prompts before
determining the directions to query. This creates a typical causal loop,
making it challenging to establish a clear sequence of actions between
determining what to generate and knowing which directions to query.

In this paper, we tackle this challenge by transforming the open querying
problem into a semi-open one. Our approach involves collecting prompts from
existing benchmarks to create a pool of querying intentions. The embeddings of
these prompts serve as anchors in the target space, indicating the candidate
directions to query and explore. During each iteration of the GAL process, we
generate samples using these prompts for evaluation. This semi-open scheme
strikes a balance by constraining the candidate directions for querying while
allowing enough freedom to explore the target space through the generation of
samples. Although this approach provides access to generated samples, the
sample-based evaluation commonly used in DAL cannot be directly applied to GAL
due to the fundamental differences between discriminative and generative
models. Discriminative models learn a single distribution to distinguish
simple semantics (e.g., dog), resulting in semantically consistent information
carried by positive (negative) samples [11]. On the other hand, generative
models focus on generalizing to various mixed semantics (e.g., to generate
images not only of the user’s pet dog flying alongside Trump but also of the
pet dog chasing butterflies on the beach) [3]. Consequently, generative models
need to handle multiple sub-distributions, each modeling a specific
combination of semantics. The information carried by samples from different
sub-distributions are not consistent, rendering sample-based evaluation
infeasible. To address this issue, we propose a distribution-based querying
strategy that adapts the classical Uncertainty Sampling [7, 12] to the new
generative scenario. This strategy takes into account the distributional
aspects of generative models and provides a more suitable framework for
querying and evaluating samples in GAL.

Another challenge is the exploitation-exploration dilemma [13, 14]. In DAL,
the collected samples from different iterations are accumulated for training,
and the learned distribution or decision boundary may gradually shift from the
samples collected in the early iterations. This is generally not a problem as
long as it benefits the classifier’s performance. In contrast, in GAL, the
fidelity to the reference images is of great importance which pushes the
generated samples towards the references. Additionally, samples generated in
the early iterations have been shown to have a higher likelihood of fulfilling
the fidelity criteria compared to later iterations, and thus should be
exploited as new references with greater attention. However, the generated
samples cannot be too close to the references, otherwise, this causes over-
fitting. Meanwhile, the generated samples need to be generalized to a certain
target direction indicated by corresponding prompt, which attracts them to
move toward the target direction against the references. The GAL process needs
to learn how to navigate this balance between adhering to the references and
exploring new directions. We propose a balancing scheme that evaluates the
importance of references, thereby allowing us to weigh the contributions of
different iterations.

The contribution of this paper can be summarized as: 1) A pilot work to
discuss the application of active learning in generative models; 2) A
distribution-based querying strategy for personalized image synthesis; and 3)
A strategy to balance the exploitation and exploration in GAL.

##  II Related Work

###  II-A Active Learning

Active learning is a subfield of machine learning, which aims to find an
optimal querying strategy to maximize model performance with the fewest
labeling cost. The most common strategies include uncertainty sampling [7,
12], query by committee [15], and representation-based sampling [16, 17], etc.
The rationale behind is to provide the most valuable samples to learn a better
decision boundary. However, acquiring real-world datasets still poses
challenges in certain scenarios, such as few-shot learning. To address this
issue, some researchers have explored the use of generative networks for data
augmentation. For example, GAAL [18] first introduced GAN [19] to generate
training samples. But this random generation does not guarantee more
informative samples compared to the original dataset. In contrast, BGADL [20]
jointly trained a generative network and a classifier so as to generate
samples in disagreement regions [7]. Subsequent approaches, such as VAAL [21]
and TA-VAAL [22] employed adversarial training for data augmentation to
improve the feature representation. It is important to note that while these
works have explored the use of generative models, their primary focus is on
improving the discriminative model’s ability.

###  II-B Personalized Content Generation

Text-to-image synthesis has earned significant attention for its potential
applications in content creation, virtual reality, and computer graphics.
Impressive works such as DALL•E [3], Stable Diffusion [4], Imagen [23], have
shown immense potential to generate compelling and diverse images. As an
application of image generation, personalized image synthesis offers user an
opportunity to create customized object or style that is difficult to generate
using pre-trained models. To accomplish content personalization, some studies
[24, 25, 26] have concentrated on training a unified model capable of
personalizing any input image. However, these approaches often struggle to
achieve satisfactory fidelity with the references. In contrast, other research
studies [9, 27] adopt a fine-tuning approach on pre-trained models for each
reference group. These methods first predefine a pseudo word to represent the
target visual concepts and then fine-tune models using provided references,
aiming to establish a mapping between the predefined words and the
corresponding visual elements. In particular, DreamBooth [9] retrains the
entire diffusion model and incorporates a prepared regularization dataset to
alleviate the overfitting problem. Following this regularization approach,
other works focus on enhancing different aspects of personalized image
synthesis, like training acceleration [27] and multiple concepts composition
[28, 29, 30]. SVDiff [31], on the other hand, applies image stitching
techniques to expand the training samples, but does not explore the
utilization of generated samples. Different from DreamBooth, Textual Inversion
[32] aims to find an optimal token embedding to reconstruct the training
images without additional regularization samples. Although additional training
samples are adopted in the training process, no generated samples are involved
in these studies.

###  II-C Personalized Style Generation

Style generation is one of the notable advancements in the field of image
synthesis. Style transfer [33, 34, 35] aims to transform the visual style of a
given image to another input image while preserving its contents. However,
these methods do not offer the chance to generate images based on text
prompts. Meanwhile, another line of research focuses on personalized style
generation, which aims to reverse visual styles on textual descriptions. A
recent study, StyleDrop [10], introduces a parameter-efficient fine-tuning
method and an iterative training framework with feedback to facilitate style
recreation. Specifically, preset prompts are used to generate images and these
images are then subject to user filtering, where users will identify high-
quality images that can be used for further training. While this approach
leverages human feedback to enhance model performance, the need for human
inspection and the equal weighting of selected samples pose limitations. In
this paper, we propose methods that effectively alleviate the burden on human
resources through active learning and reduce selection bias by balancing the
importance of synthetic and real samples.

Algorithm 1 Generative Active Learning

Input: anchor embedding sets $\mathcal{A}$, reference images $\mathbf{x}$,
subject of interest $\mathbf{e}^{*}$, non-SoI $\bar{\mathbf{e}}^{*}$, pre-
trained model $f_{\mathbf{\theta}}$

Initialize training set
$\mathbf{T}=\\{(\mathbf{x},\mathbf{e}^{*}\oplus\bar{\mathbf{e}}^{*})\\}$.

repeat

Fine-tune $f_{\mathbf{\theta}}$ on $\mathbf{T}$

for $\mathbf{a}_{i}$ in $\mathcal{A}$ do

for $j=1$ to $m$ do

Generate image $\mathbf{I}_{ij}$ on $\mathbf{a}_{i}$ by $f_{\mathbf{\theta}}$

Verify whether $\mathbf{I}_{ij}$ is overfitted by 10

end for

Calculate $\Omega(\mathbf{a}_{i})$ according to 6

end for

Update $\mathbf{T}$ with top-$k$ anchor embeddings

Update openness score according to 11

until Stopping criterion is met according to 12

##  III Method

In this section, we introduce our implementation of generative active learning
for image synthesis personalization. The algorithm, along with its
corresponding pseudo-code, is depicted in 1.

###  III-A Preliminaries for Image Synthesis Personalization

The current state-of-the-art methods for Image Synthesis Personalization (ISP)
are all based on diffusion models [36, 37]. What sets diffusion models apart
is their “generate-by-denoise” approach. During training, a text-image pair is
used, and the process begins by iteratively adding noise to the image
$\mathbf{x}$ according to the Markov chain, resulting in a noisy image
${\mathbf{x}_{t}}$. The noisy image is then combined with the text embedding
$\mathbf{e}$ to create a new noisy image embedded with the text semantics,
denoted as $\mathbf{x}_{t}\circ\mathbf{e}$. Learning then proceeds to denoise
this image and reconstruct the original image $\mathbf{x}$, which is
represented as

$\hat{\mathbf{x}}=f_{\mathbf{\theta}}(\mathbf{x}_{t}\circ\mathbf{e})$ (1)

The objective is to minimize the reconstruction loss

$L_{rec}=\mathbb{E}\left[w_{t}\|\hat{\mathbf{x}}-\mathbf{x}\|_{2}^{2}\right]$
(2)

where $w_{t}$ represents a time-step dependent weight. In the generation
phase, the embedding of a user prompt $\tilde{\mathbf{e}}$ is then fused with
a random noise $\mathbf{\epsilon}$ to generate the image
$\tilde{\mathbf{x}}=f_{\mathbf{\theta}}(\mathbf{\epsilon}\circ\tilde{\mathbf{e}})$
that align with the semantics of interest.

To perform an ISP process, a pre-trained model $f_{\mathbf{\theta}}$ is
typically fine-tuned using reference images that contain the Subject of
Interest (SoI). A pseudo text word $S^{*}$ is utilized to represent the SoI
and is incorporated into simple sentences, such as “a photo of $S^{*}$,” as a
reference prompt. The training process involves updating the parameters of the
model $f_{\mathbf{\theta}}$ to establish the association between the visual
appearance of the SoI (indicated by given reference images $\mathbf{I}_{r}$)
and its corresponding semantic embedding $\mathbf{e}^{*}$. After the fine-
tuning, new images of SoI can be generated with prompts like “$S^{*}$ running
on the street with a dog” or “$S^{*}$ ridding a house on the Golden Bridge” if
the SoI is an object. In case the SoI is a specific style, new images can be
generated using prompts like “a drawing of New York City with style $S^{*}$”
or “a teddy bear of style $S^{*}$”.

Figure 1: Examples of overfitted and well-aligned generations. The model has
to exclude the non-SoI for successful generations.

###  III-B Direction-based Uncertainty Sampling

It is evident that a limited number of reference images for the SoI is
insufficient to ensure the fine-tuned model’s generalizability to a broader
range of semantics. We need to generate new samples to augment the references,
which requires prompts to determine the direction to query. However, the
querying remains open to all directions since users may combine the pseudo
text word $S^{*}$ with various unseen concepts in future prompts. To address
this, we transform the problem into a semi-open one by incorporating the SoI
with a set of predefined concepts (e.g., cat and table) that can be gathered
from existing benchmarks. These concepts serve as anchors in the target space,
with each anchor representing a specific direction for querying when combined
with the SoI to form anchor prompts (e.g., “$S^{*}$ with a dog”). The model’s
ability to generate high-quality samples for these anchor prompts determines
its level of generalization. While the anchor directions are predetermined,
the querying process remains open due to the introduction of random noise
$\mathbf{\epsilon}$, which leads to variations in the generated images for the
same prompt. To ease the discussion, let us denote the anchor embedding set as
$\mathcal{A}=\\{\mathbf{a}_{i}\\},i\in\mathbb{N}$ and the set of embeddings of
anchor prompts or directions to query can be denoted as

$\mathbf{e}^{*}\oplus\mathcal{A}=\\{\mathbf{e}^{*}\oplus\mathbf{a}_{i}\\},i\in\mathbb{N}.$
(3)

where $\oplus$ is a model-dependent operator. In the majority of models that
employ CLIP [38] as the text encoder, the operation $\oplus$ is typically
implemented by directly inputting the embeddings as a sequence.

In each iteration of the generative active learning (GAL), we generate $m$
samples for each anchor prompt. We need to initiate the next round of GAL by
selecting informative ones from the generated samples as new references.
However, as discussed, conventional sample-based querying is infeasible in
GAL, because evaluating performance on individual samples lacks of global
perspective to measure the model’s generalizability. Additionally, relying
solely on generalizability to build a metric is challenging because higher
generalizability may indicate well-explored directions, where samples would
not provide novel information for improving the model. This is similar to the
situation in Discriminative Active Learning (DAL), where including samples
from well-classified locations does not contribute to performance improvement
and instead hinders exploration. A popular solution is Uncertainty Sampling
[7], which selects samples from areas where the model exhibits uncertainty. In
the context of GAL, we can adapt this idea to identify directions where the
quality of model-generated samples lies between well-generalized and
overfitted. Let $\mathbf{I}_{ij},j\in[1,m]$ denote an image generated for the
$i^{th}$ anchor direction $\mathbf{a}_{i}$ as

$\mathbf{I}_{ij}=f_{\mathbf{\theta}}\left(\mathbf{\epsilon}\circ(\mathbf{e}^{*}\oplus\mathbf{a}_{i})\right)$
(4)

and there is an oracle function to verify whether $\mathbf{I}_{ij}$ is
overfitted as

$\Phi(\mathbf{I}_{ij})\in\\{0,1\\},$ (5)

we can implement a direction-based uncertainty sampling for GAL by measuring
the entropy on the portions of overfitted (non-overfitted) samples as

$\displaystyle\Omega(\mathbf{a}_{i})$
$\displaystyle=-\left[(1-\beta_{i})\log(1-\beta_{i})+\beta_{i}\log\beta_{i}\right]$
(6) $\displaystyle\beta_{i}$
$\displaystyle=\frac{\sum_{j=1}^{m}\Phi(\mathbf{I}_{ij})}{m}.$ (7)

In DAL, the learning employs human annotators as oracles. However, due to the
computational expense of current diffusion models, it becomes impractical for
human annotators to wait for the results of each iteration, resulting in
significant delays. Hiring human annotators as oracles can be extremely
costly, which might be one of the reasons why successful ISP models using
generated results as argumentation are predominantly developed by large
companies like Google [10, 39], who can afford such expenses. In our study, we
found the oracle function $\Phi(\mathbf{I}_{ij})$ can be estimated by
evaluating the generated image $\mathbf{I}_{ij}$’s fidelity to both the anchor
direction and irrelevant semantics in the reference prompt. This observation
stems from the fact that the reference prompt consists of two components: the
SoI and non-SoI semantics. Most previous studies focus on the fidelity to the
SoI semantics, while the non-SoI semantics are not fully leveraged. These non-
SoI semantics can be considered distractor semantics that the generated images
should avoid, similar to negative labels in discriminative models. One such
example can be found from 1, in which the SoI is the drawing style while the
non-SoI is the concept cat. The overfitted samples are those failed to
disentangle the cat from the generation. Therefore, we propose a
straightforward metric to simulate the oracle function. Let
$\bar{\mathbf{e}}^{*}$ denote the non-SoI embedding, the function is written
as

$\displaystyle\Phi(\mathbf{I}_{ij})=$
$\displaystyle\left\\{\begin{array}[]{cc}1,&sim(\mathbf{I}_{ij},\mathbf{a}_{i})\leq
sim(\mathbf{I}_{ij},\bar{\mathbf{e}}^{*})\\\
0,&sim(\mathbf{I}_{ij},\mathbf{a}_{i})>sim(\mathbf{I}_{ij},\bar{\mathbf{e}}^{*})\\\
\end{array}\right.$ (10)

where the $sim()$ is a fidelity metric of an image to a semantics. In this
study, we simply adopt the CLIP similarity [38].

With all necessary components built, the querying can then be conducted by
evaluating all the anchor directions and select the ones with top-$k$
uncertainty scores (using 6). For each direction, we choose $n$ generated
images with the highest $sim(\mathbf{I}_{ij},\mathbf{a}_{i})$ scores
(indication of the faithfulness to the directions) as new reference images.

###  III-C Balancing the Exploitation and Exploration

As aforementioned, in the progression of GAL iterations, we need to keep the
knowledge learned at past rounds while encouraging the model to explore. This
introduces a exploitation-exploration dilemma [13]. To address this challenge,
we propose evaluating the openness of the model at each round, using it as an
indicator of the expected contribution of the novel information introduced in
that round. Given that the novel information is encapsulated within the newly
included reference images, we can utilize this indicator as a weight to
regulate their impact on the learning process in the subsequent round. This
encourages the exploration when the expected contribution is high, otherwise
encourages the exploitation.

To assess the openness of a round, we can utilize the uncertainty score
previously computed using 6. Our rationale is that as the model explores more
directions, its level of openness increases. Hence, the openness score for the
round can be estimated by calculating

$\Delta(f_{\mathbf{\theta}})=\frac{\lambda}{|\mathcal{A}|}\sum_{i=1}^{|\mathcal{A}|}\Omega(\mathbf{a}_{i})$
(11)

where $\lambda$ is a learning rate. This can be used to weight the newly
include reference images to control their degrees of influence to the loss
$L_{rec}$ (2).

An additional outcome of 11 is its potential to establish an adaptive stopping
criterion for GAL learning, in contrast to the fixed number of iterations
often set in DAL. The concept behind this approach is to halt the learning
process when there are fewer directions left to explore than anticipated. The
stopping criteria is then simply written as

$\Big{|}\\{\Omega(\mathbf{a}_{i})\hskip 2.84544pt|\hskip
2.84544pt\Omega(\mathbf{a}_{i})>0,\mathbf{a}_{i}\in\mathcal{A}\\}\Big{|}<k.$
(12)

TABLE I: The performance of different strategies. |  | Object |  |  | Style |
---|---|---|---|---|---|---
MODELS |  TXT-ALN $\uparrow$ |  IMG-ALN $\uparrow$ |  OVF $\downarrow$ |  TXT-ALN $\uparrow$ |  IMG-ALN $\uparrow$ |  OVF $\downarrow$
Baseline (DreamBooth) | 0.298 | 0.796 | 0.363 | 0.318 | 0.694 | 0.171
Random | 0.285 | 0.714 | 0.391 | 0.247 | 0.620 | 0.327
Human | 0.297 | 0.721 | 0.331 | 0.272 | 0.622 | 0.212
Uncertainty (ours) | 0.305 | 0.755 | 0.268 | 0.286 | 0.628 | 0.110
Uncertainty $+$ Balance (ours) | 0.311 | 0.762 | 0.235 | 0.339 | 0.664 | 0.049
Human $+$ Balance (ORACLE+ours) | 0.307 | 0.772 | 0.254 | 0.342 | 0.650 | 0.023

##  IV Experiments

Datasets. To evaluate the performance of active learning in ISP, we conduct
experiments on two most representative tasks, style- and object-driven
personalization.

In style-driven personalization, we adopt the evaluation dataset used in the
StyleDrop [10]. This dataset comprises various styles, such as watercolor
painting, oil painting, 3D rendering, and cartoon illustration. 190 basic text
prompts sourced from the Parti prompts dataset [40] are used to generate
images, yielding a total of 36,480 images.

In object-driven personalization, we adopt almost all concepts that have been
previously used in related studies [27, 32], comprising a total of 10
categories including animals, furniture, containers, houses, plants, and toys.
We use the 20 prompts in [27], which cover a wide range of test scenarios. In
total, this process generates 6,400 images for a complete training cycle.

Figure 2: Examples of images generated by anchor prompts with higher priority
(left) and lower priority (right). Their features are highlighted in the
feature space (middle). Poor-quality images that exhibit non-SoI are
distributed near the reference, while high-quality images are located far from
the reference. These images are from the anchor prompts in round 2.

Evaluation Metrics. We utilize three metrics: 1) Text-alignment (TXT-ALN)
assesses how well the generated images align with the intended textual
descriptions This can be implemented by calculating the similarity between the
CLIP image feature and the text feature. 2) Image-alignment (IMG-ALN) measures
the extent to which the generated images capture the content or style present
in the reference images. This can be implemented by the CLIP feature
similarity between reference images and generated images. 3) Overfit (OVF)
evaluates the proportion of overfitting in the test samples. Lower scores
indicate better performance in terms of generalization and avoiding
overfitting.

Base Model. In contrast to many closed-source ISP models developed by large
companies, DreamBooth [9] is one of a few most widely used open-source methods
with promising generation results. Therefore, we also employ DreamBooth as our
base model. For our proposed method, we set the values of $m$, $k$, and
$\lambda$ to 18, 3, and 0.0035, respectively.

###  IV-A Does generative active learning work in ISP?

To evaluate the performance of different strategies on ISP, we compare our
method with two commonly adopted querying strategies, including Random
Sampling and Human Sampling. To be fair and efficient, we set a maximum number
of rounds to 4 in all experiments. The initial round is based on original
references without any synthetic data. The results are shown in I. It is
evident from the results that both Random and Human strategies do not
necessarily enhance the baseline performance. Instead, these strategies show a
degradation on style-personalization of 22.3% (14.5%), 10.7%(10.4%), and
91.2%(24.0%) on TXT-ALN, IMG-ALN, and OVF, respectively. The unexpected
degradation observed in the Human strategy, which is often considered an
oracle in DAL, confirms the fundamental distinction between discriminative and
generative tasks: while human annotators can easily differentiate between
positive and negative samples, evaluating generalizability is a more
challenging aspect. Therefore, we integrate human annotators with our
balancing scheme to create a run that combines their selections and fairly
weighs them for improved learning. The results are shown as the last row in I
which demonstrates an approaching optimal performance and thus can be used as
an oracle. Spuriously, our proposed method (Uncertainty+Balance) achieves a
comparable performance with the oracle run. This has validated its
effectiveness.

Figure 3: Results of GAL over iterations. The images shown in the $1^{st}$ and
$2^{nd}$ groups are for style- and object-driven ISP, respectively. The non-
SoI and SoI are gradually disentangled and dragons or glasses are generated.
Additional examples are available in the Appendix.

Figure 4: The curves shown in the figure resemble clock arms extending from
the baseline performance points. As these arms move in an anti-clockwise
direction towards the top-right corners, better performance is observed.

###  IV-B How does the uncertainty sampling work?

To gain deeper insights, we conduct a case study to observe the rationale
behind the uncertainty sampling. 2 shows the distribution of images generated
by the anchor prompts. Within the feature space, multiple sub-distributions
can be observed. One particular distribution is centered around the reference,
consisting of poor-quality images that exhibit non-SoI of the reference, such
as the failure cases illustrated in 2. In contrast, images that align well are
located far from the reference, forming smaller distributions that exclude
non-SoI, like the successful samples of generated billboard, key, and candy
images. Additionally, the distributions of good and bad samples across these
three directions demonstrate significant diversity, suggesting a limited
ability to generalize along these directions. As a result, these directions
are given higher priority for querying based on our uncertainty metric. On the
other hand, distributions at the directions of table and banana are
homogeneous. Consequently, these directions exhibit lower entropy and lower
querying priority. This observation aligns with the rationale we presented
earlier.

###  IV-C How does GAL progress over iterations?

To examine the progress of GAL over iterations, we present the performance in
each round, as shown in 4, and visualize the evolution through the cases in 3.
One notable observation is the dramatic and consistent decrease in performance
of the Random strategy due to the inferior samples by random selection. After
adopting a better querying strategy, the rate of decrease becomes much slower,
and Uncertainty sampling begins to outperform the baseline on TXT-ALN for
object-driven personalization, which suggests the effectiveness of valuable
samples in enhancing generative models. The best overall progress is achieved
through the combined strategies of Uncertainty sampling and the balancing
scheme. We can find that TXT-ALN consistently improves and reaches its highest
alignment in round 4, while IMG-ALN remains within a reasonable range. This
trend is evident in 3, where the non-SoI semantics gradually disappear, and
the number of successful generations of glasses placed on $S^{*}$ or dragon in
style $S^{*}$ increases. Meanwhile, the SoI is maintained throughout the
iteration rounds. These results indicate a progressive improvement by GAL as
the iterations proceed. Additional comprehensive examples are available in the
Appendix.

TABLE II: Comparison with SOTA methods for object-driven ISP. MODELS | TXT-ALN | IMG-ALN | OVF
---|---|---|---
IP-Adapter | 0.270 | 0.858 | 0.734
Textual Inversion | 0.277 | 0.778 | 0.441
Custom Diffusion | 0.301 | 0.776 | 0.287
DreamBooth | 0.298 | 0.796 | 0.363
\+ UNC + BAL (R2) | 0.306 | 0.777 | 0.270
\+ UNC + BAL (R3) | 0.305 | 0.782 | 0.293
\+ UNC + BAL (R4) | 0.311 | 0.762 | 0.235
\+ ORACLE + BAL | 0.307 | 0.772 | 0.254
TABLE III: Comparison with SOTA methods for style-driven ISP. MODELS | TXT-ALN | IMG-ALN | OVF
---|---|---|---
Imagen | 0.337 | 0.569 | -
DB on Imagen | 0.335 | 0.644 | -
Muse | 0.323 | 0.556 | -
StyleDrop | 0.313 | 0.705 | -
StyleDrop-Random | 0.316 | 0.678 | -
StyleDrop-CF | 0.329 | 0.673 | -
StyleDrop-HF | 0.322 | 0.694 | -
DreamBooth | 0.318 | 0.694 | 0.171
\+ UNC + BAL (R2) | 0.328 | 0.679 | 0.096
\+ UNC + BAL (R3) | 0.337 | 0.666 | 0.056
\+ UNC + BAL (R4) | 0.339 | 0.664 | 0.049
\+ ORACLE + BAL | 0.342 | 0.650 | 0.023

Figure 5: Qualitative comparison between our method and SOTA methods for
personalized content generation. Our method produces text-aligned images
compared with other methods. Besides, our method shows better preservation of
details, including the color of cat statue and the spout of teapot. Additional
comprehensive examples are available in the Appendix.

###  IV-D Comparison with SOTA methods

For object driven-personalization, we compare 4 popular state-of-the-art
(SOTA) methods including Textual Inversion [32], Custom Diffusion [27],
DreamBooth [9], IP-Adapter [25]. The results are shown in II. Compared to IP-
Adapter, Textual Inversion, and Custom Diffusion, our method demonstrates
significant improvements on TXT-ALN and OVF throughout all rounds, achieving
15.2%(68.0%), 12.3%(46.7%), and 3.3%(18.1%) on TXT-ALN (OVF) in terms of round
4. Since the non-SoI semantics dominate the outputs of the other approaches,
our method exhibits a slight decrease on IMG-ALN. 5 provides visual evidence
of our method’s superior text and object fidelity. The success in higher text
fidelity can be observed in the accurate placement of the cat statue in the
Grand Canyon and the realistic interaction between the marigold flowers and
the teapot. Furthermore, our method enhances object fidelity by accurately
reconstructing only one spout and better preserving the color of the cat
statue.

For style-driven personalization, we conduct a comparison between four
variations of StyleDrop [10]: base model, random feedback (StyleDrop-Random),
clip-based feedback (StyleDrop-CF), and human feedback (StyleDrop-HF).
Additionally, we include the results of DreamBooth on Imagen [23] as well as
other pre-trained models like Imagen and Muse [41], as reported by [10]. It is
clear that our method significantly outperforms the pre-trained models and
achieves superior performance in terms of 5.3% and 3.0% on TXT-ALN compared to
the dedicated human feedback and clip-based feedback of StyleDrop. It is worth
noting that the closed-source StyleDrop is built on a more powerful backbone,
Muse, compared to Stable Diffusion. This indicates that the open-sourced ISP
models are able to achieve better performance with GAL.

##  V Conclusion

This paper presents a pilot study that investigates the application of active
learning to generative models, specifically focusing on image synthesis
personalization tasks. To solve the open-ended nature of querying in
generative active learning, this paper introduces anchor directions,
transforming the querying process into a semi-open problem. An uncertainty
sampling strategy is introduced to select informative directions, and a
balance scheme is proposed to solve the exploitation-exploration dilemma.
Through extensive experiments, the effectiveness of the approach is validated,
indicating new possibilities for leveraging active learning techniques in the
context of generative models.

## References

  * [1] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray _et al._ , “Training language models to follow instructions with human feedback,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 27 730–27 744, 2022.
  * [2] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar _et al._ , “Llama: Open and efficient foundation language models,” _arXiv preprint arXiv:2302.13971_ , 2023.
  * [3] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8821–8831.
  * [4] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10 684–10 695.
  * [5] S. I. Nikolenko, _Synthetic data for deep learning_. Springer, 2021, vol. 174.
  * [6] S. Azizi, S. Kornblith, C. Saharia, M. Norouzi, and D. J. Fleet, “Synthetic data from diffusion models improves imagenet classification,” _arXiv preprint arXiv:2304.08466_ , 2023.
  * [7] A. Tharwat and W. Schenck, “A survey on active learning: State-of-the-art, practical challenges and research directions,” _Mathematics_ , vol. 11, no. 4, p. 820, 2023.
  * [8] R. Po, W. Yifan, V. Golyanik, K. Aberman, J. T. Barron, A. H. Bermano, E. R. Chan, T. Dekel, A. Holynski, A. Kanazawa _et al._ , “State of the art on diffusion models for visual computing,” _arXiv preprint arXiv:2310.07204_ , 2023.
  * [9] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 22 500–22 510.
  * [10] K. Sohn, N. Ruiz, K. Lee, D. C. Chin, I. Blok, H. Chang, J. Barber, L. Jiang, G. Entis, Y. Li _et al._ , “Styledrop: Text-to-image generation in any style,” _arXiv preprint arXiv:2306.00983_ , 2023.
  * [11] I. Rish _et al._ , “An empirical study of the naive bayes classifier,” in _IJCAI 2001 workshop on empirical methods in artificial intelligence_ , vol. 3, no. 22, 2001, pp. 41–46.
  * [12] D. Wang and Y. Shang, “A new active labeling method for deep learning,” in _2014 International joint conference on neural networks (IJCNN)_. IEEE, 2014, pp. 112–119.
  * [13] X.-Y. Wei and Z.-Q. Yang, “Coached active learning for interactive video search,” in _Proceedings of the 19th ACM international conference on Multimedia_ , 2011, pp. 443–452.
  * [14] ——, “Coaching the exploration and exploitation in active learning for interactive video retrieval,” _IEEE Transactions on Image Processing_ , vol. 22, no. 3, pp. 955–968, 2012.
  * [15] H. S. Seung, M. Opper, and H. Sompolinsky, “Query by committee,” in _Proceedings of the fifth annual workshop on Computational learning theory_ , 1992, pp. 287–294.
  * [16] O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” _arXiv preprint arXiv:1708.00489_ , 2017.
  * [17] Y. Geifman and R. El-Yaniv, “Deep active learning over the long tail,” _arXiv preprint arXiv:1711.00941_ , 2017.
  * [18] J.-J. Zhu and J. Bento, “Generative adversarial active learning,” _arXiv preprint arXiv:1702.07956_ , 2017.
  * [19] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _Advances in Neural Information Processing Systems_ , vol. 27, 2014.
  * [20] T. Tran, T.-T. Do, I. Reid, and G. Carneiro, “Bayesian generative active deep learning,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 6295–6304.
  * [21] S. Sinha, S. Ebrahimi, and T. Darrell, “Variational adversarial active learning,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 5972–5981.
  * [22] K. Kim, D. Park, K. I. Kim, and S. Y. Chun, “Task-aware variational adversarial active learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 8166–8175.
  * [23] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans _et al._ , “Photorealistic text-to-image diffusion models with deep language understanding,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 36 479–36 494, 2022.
  * [24] Y. Wei, Y. Zhang, Z. Ji, J. Bai, L. Zhang, and W. Zuo, “Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation,” _arXiv preprint arXiv:2302.13848_ , 2023.
  * [25] H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,” _arXiv preprint arXiv:2308.06721_ , 2023.
  * [26] D. Li, J. Li, and S. C. Hoi, “Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing,” _arXiv preprint arXiv:2305.14720_ , 2023.
  * [27] N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y. Zhu, “Multi-concept customization of text-to-image diffusion,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 1931–1941.
  * [28] Y. Tewel, R. Gal, G. Chechik, and Y. Atzmon, “Key-locked rank one editing for text-to-image personalization,” in _ACM SIGGRAPH 2023 Conference Proceedings_ , 2023, pp. 1–11.
  * [29] Z. Liu, R. Feng, K. Zhu, Y. Zhang, K. Zheng, Y. Liu, D. Zhao, J. Zhou, and Y. Cao, “Cones: Concept neurons in diffusion models for customized generation,” _arXiv preprint arXiv:2303.05125_ , 2023.
  * [30] X.-L. Zhang, X.-Y. Wei, J.-L. Wu, T.-Y. Zhang, Z.-X. Zhang, Z. Lei, and Q. Li, “Compositional inversion for stable diffusion models,” _arXiv preprint arXiv:2312.08048_ , 2023.
  * [31] L. Han, Y. Li, H. Zhang, P. Milanfar, D. Metaxas, and F. Yang, “Svdiff: Compact parameter space for diffusion fine-tuning,” _arXiv preprint arXiv:2303.11305_ , 2023.
  * [32] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-or, “An image is worth one word: Personalizing text-to-image generation using textual inversion,” in _International Conference on Learning Representations_ , 2022.
  * [33] L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 2414–2423.
  * [34] Y. Zhang, N. Huang, F. Tang, H. Huang, C. Ma, W. Dong, and C. Xu, “Inversion-based style transfer with diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 10 146–10 156.
  * [35] Z. Wang, L. Zhao, and W. Xing, “Stylediffusion: Controllable disentangled style transfer via diffusion models,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2023, pp. 7677–7689.
  * [36] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 6840–6851, 2020.
  * [37] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in _International Conference on Learning Representations_ , 2020.
  * [38] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark _et al._ , “Learning transferable visual models from natural language supervision,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8748–8763.
  * [39] W. Chen, H. Hu, Y. Li, N. Rui, X. Jia, M.-W. Chang, and W. W. Cohen, “Subject-driven text-to-image generation via apprenticeship learning,” _arXiv preprint arXiv:2304.00186_ , 2023.
  * [40] J. Yu, Y. Xu, J. Y. Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan _et al._ , “Scaling autoregressive models for content-rich text-to-image generation,” _arXiv preprint arXiv:2206.10789_ , vol. 2, no. 3, p. 5, 2022.
  * [41] H. Chang, H. Zhang, J. Barber, A. Maschinot, J. Lezama, L. Jiang, M.-H. Yang, K. Murphy, W. T. Freeman, M. Rubinstein _et al._ , “Muse: Text-to-image generation via masked generative transformers,” _arXiv preprint arXiv:2301.00704_ , 2023.

##  VI Appendix

To provide a more comprehensive understanding of the method, we have included
additional details in the following sections. The source code can be accessed
at https://github.com/zhangxulu1996/GAL4Personalization

###  VI-A Additional Implementation Details

For a fair comparison, we utilize the third-party implementation of
HuggingFace for all SOTA methods in our experiments. We use Stable Diffusion
1.4 version as the pre-trained model. Besides, we apply 50 steps of DDPM
(Denoising Diffusion Probabilistic Model) sampling with a scale of 7.5 in all
experiments. All experiments are conducted using A-100 GPUs.

In each round, we adopt DreamBooth to retrain the diffusion model.
Specifically, the parameters of the text Transformer are frozen and the U-Net
diffusion model is fine-tuned with learning rate $5\times 10^{-6}$ and the
batch size is 1. The training step for content learning is 800 steps (wooden
pot for 600 steps to avoid overfitting) and style is 500 steps.

###  VI-B Text Prompts for Additional Training Samples Generation

Here we provide the prompts used to synthesize the images for iterative
training in IV and V. In each round, 3 prompts paired images will be selected
as additional training samples.

TABLE IV: The anchor prompts of object personalization to generate synthetic samples. “$S^{*}$ in park” |  “$S^{*}$ near a pool” |  “$S^{*}$ on street”
---|---|---
“$S^{*}$ in a forest” |  “$S^{*}$ in a kitchen” |  “$S^{*}$ on the beach”
“$S^{*}$ in a restaurant” |  “$S^{*}$ on snowy ground” |  “$S^{*}$ on the moon”
“$S^{*}$ in desert” |  “$S^{*}$ in library” |  “$S^{*}$ under the night sky”
“$S^{*}$ in front of the castle” |  “$S^{*}$ on the bridge” |  “$S^{*}$ next to waterfall”
“$S^{*}$ in the cave” |  “$S^{*}$ at a concert venue” |  “$S^{*}$ on a balcony”
TABLE V: The anchor prompts of style personalization to generate synthetic samples. “A tie in style $S^{*}$” |  “A table in style $S^{*}$” |  “A broccoli in style $S^{*}$”
---|---|---
“A box in style $S^{*}$” |  “candy in style $S^{*}$” |  “A billboard in style $S^{*}$”
“A drum in style $S^{*}$” |  “A cell phone in style $S^{*}$” |  “An empty bottle in style $S^{*}$”
“A glass in style $S^{*}$” |  “A golden key in style $S^{*}$” |  “A koala in style $S^{*}$”
“A toy in style $S^{*}$” |  “A skateboard in style $S^{*}$” |  “An apple on the table in style $S^{*}$”
“A bridge in style $S^{*}$” |  “Beach scene in style $S^{*}$” |  “A banana on the table in style $S^{*}$”

###  VI-C Stopping Criterion

6 illustrates a case study to show the effectiveness of the early stopping
setting. We can find that the personalized model performs well toward text
fidelity and image fidelity in the termination round. However, beyond this
point, introducing non-informative samples results in a decline in
performance. We can find the prominent edge features and unnatural
brushstrokes in the generated outputs. These deteriorated results suggest that
the model struggles to maintain its initial level of quality when confronted
with less informative prompts. The stopping criterion plays an important role
in preventing the inclusion of samples that could negatively impact its output
quality when there are few anchor directions to explore.

Figure 6: Comparison between the termination round and extended round in the
iterative training process. The details are better preserved in the
termination round.

###  VI-D Additional Results

This section provides samples of generated images. 7 and 8 show the generated
images in each round based on object and style references. 9 gives a more
visualized comparison with SOTA methods. 10, 11, and 12 show the
generalization of our method on a wide range of prompts.

Figure 7: Examples of images generated in each round. The model in round 1
fails to produce the desired subjects, including food, sword, butterfly, and
car. These subjects are successfully generated after using generative active
learning.

Figure 8: Examples of images generated in each round. The model in round 1
fails to produce the desired subjects, including chocolate cake, marigold
flowers, crochet, and orange sofa. These subjects are successfully generated
after using generative active learning.

Figure 9: Comparison with other SOTA methods. Our method achieves better text
and object fidelity.

Figure 10: Generated images using style references. Our method generalizes
well on a diverse range of prompts.

Figure 11: Generated images using object references. Our method can be used
for a variety of personalization purposes, such as background replacement,
artistic rendering, and attribute editing.

Figure 12: Generated images using object references. Our method can be used
for a variety of personalization purposes, such as background replacement,
artistic rendering, and attribute editing.