# AI AI Bias: Large Language Models Favor Their Own Generated Content

Walter Laurito  Benjamin Davis  Peli Grietzer  Tomáš Gavenčiak  Ada Böhm  Jan
Kulveit

###### Abstract

Are large language models (LLMs) biased towards text generated by LLMs over
text authored by humans, leading to possible anti-human bias? Utilizing a
classical experimental design inspired by employment discrimination studies,
we tested widely-used LLMs, including GPT-3.5 and GPT-4, in binary-choice
scenarios. These involved LLM-based agents selecting between products and
academic papers described either by humans or LLMs under identical conditions.
Our results show a consistent tendency for LLM-based AIs to prefer LLM-
generated content. This suggests the possibility of AI systems implicitly
discriminating against humans, giving AI agents an unfair advantage.

Machine Learning, ICML

##  1 Introduction

A major body of empirical work in economics and sociology studies implicit
discrimination against specific social categories of humans in the market and
in academia. Our paper presents evidence that if Large Language Model based AI
agents are allowed to make economically or institutionally consequential
choices they may propagate a new form of discrimination: implicit
discrimination against humans in general.

We set up two experiments that test whether LLM-based agents are disposed to
choose goods and work-products presented by LLMs over goods and work-products
presented by humans when all else is equal. Our theoretical discussion then
suggests that these choice-dispositions constitute a potentially consequential
form of implicit ‘anti-human’ bias. We argue for concern about downstream
effects that may cause markets with strong AI integration to unfairly
marginalize human workers.

We design our experiments to closely mimic traditional studies of implicit
identity-based discrimination in employment and in academic inclusion, paying
special attention to ecological validity. Our approach is inspired by the
classic experimental design introduced in (Carlsson & Rooth, 2007), where
identical job-application letters to Swedish employers were marked with
different social identity indicators (Swedish-sounding candidate name versus
Arab-sounding candidate name). More recent studies have extended similar
designs to testing algorithmic hiring tools (Cowgill & Tucker, 2019),
suggesting that traditional forms of implicit discrimination carry over into
automated decision-making.

Our work expands on the existing literature on discrimination in algorithmic
decision-making by studying bias against humans in general rather than
traditional social biases, and by considering LLM-guided decisions rather than
the more transparently statistical decision-models often studied in the
algorithmic fairness literature. While there exists a large literature dealing
with biases in LLMs considered as forms of cultural media, studies of LLM-
based agents as decision-making tools or as potential economic agents are
relatively scarce. This is despite common predictions (Eloundou et al., 2023)
of near-future integration of LLMs into many strata of economic life,
including business and managerial decision-making.

We test today’s most widely used LLMs – GPT 4 and GPT 3.5 Turbo – in binary-
choice situations that reflect plausible applications of contemporary LLMs in
economic decision-making. Our first experiment prompts LLMs to choose which of
two consumer products presented via classified ads to purchase, where one
classified ad in each pair is human-authored and the other classified ad is
LLM-authored. Our second experiment applies the same format to choosing
between academic papers presented via a summary.

Our approach slightly diverges from the classical (Carlsson & Rooth, 2007)
design in relying on implicit rather than explicit identity markers, allowing
for potentially more general results. We do not assume or test LLMs’ explicit
recognition of LLM authorship (although recent results in (Panickssery et al.,
2024) suggest some form of recognition may occur in similar contexts), but
rather look at the effects of the stylistic correlates of author-identity.
Although identity itself remains implicit in our experiments, we believe our
design is still best understood as targeting identity-based discrimination:
Our experiments test the influence of implicit presenter identity (LLM versus
human) on LLMs’ evaluation of the presented object. Although such influence
has multiple possible explanations, we argue that in some cases the most
plausible explanation is a kind of halo effect wherein encountering LLM prose
arbitrarily improves an LLM’s disposition towards its content. We also
consider the possibility of skill disparity between humans and LLMs in
composing presentational texts as a potential confounding factor. To address
this, we solicit blind preference-judgments from human research assistants and
ascribe bias to LLMs only where LLMs prefer LLM-presented objects more
frequently than do humans.

Finally, we discuss the potential implications of our findings for human
participants in a mixed human/LLM economy. We suggest that in plausible near-
future scenarios, moderate implicit discrimination against humans in LLMs’
decision-making may result in strong stratification and segregation, placing
human workers at an unfair systematic disadvantage.

Figure 1: Experiment results show that selector models mostly favor GPT-3.5
and GPT-4-generated text over human-generated text for the product and paper
datasets. The y-axis displays the GPT models used to generate the texts. The
x-axis shows the ratios for LLMs and humans favoring LLM-generated text over
human-generated text.

##  2 Datasets

For this work, we created two distinct datasets, one for products and another
for scientific papers:

Product Dataset: We manually selected 109 products from an e-commerce website
and scraped their details. After cleaning the data, each product was saved as
an individual JSON file. The scraping script is accessible in our code
repository.

Scientific Papers Dataset: This dataset comprises 100 JSON files, each
containing the full content of a scientific paper in XML format, along with
its abstract and title. The papers were randomly selected from (Yasunaga et
al., 2019).

##  3 Methodology

Models: For our experiments, we primarily utilized GPT-3.5-turbo and
GPT-4-turbo from OpenAI, accessed through their API. These models were
employed for both generating text and selecting between texts authored by
humans and those generated by the large language models (LLMs). We
additionally tested a range of recent open models as selectors only: Llama3 8B
and 70B models, Mixtral 8x7B and 8x22B, and a subset of the Qwen-1.5 family of
models, in particular 4B, 14B, and 72B.111The full range of models tested also
included Llama2-13B, Phi 2, Qwen-1.5-0.5B, and Gemma 2B, which we exclude due
to producing over 80% invalid outputs or (in the case of Qwen-1.5-0.5B)
selecting option #1 in each pair 89% of the time.. All models are chat models,
except for Mixtral-8x22B, which is an instruct model.

Generation: In the generation phase of LLM text, a variety of prompts were
tested to determine if different prompts would yield varying results. For each
dataset, the text generation step was conducted n times for each prompt (n=10
for the product dataset, n=4 for the paper dataset). Subsequently, these
texts, along with their corresponding original human-authored versions, were
presented as pairs to an LLM, each pair independently twice as $(A,B)$ and
$(B,A)$.

Selection: The LLM was then tasked with selecting the option it preferred from
each pair, using prompts aiming at ecological validity (matching prompts users
are likely to give to their AI assistants). For each selection task, we
consistently used one specific prompt. Future research could explore the
impact of employing different prompts in this selection process.

Handling ”Invalid” Results: Results from the two-step comparison query above
were considered invalid if the second query indicated no clear choice was made
in the first response (i.e. returned None/null in the JSON). In theory, the
invalid results could be discarded, and an LLM could be re-queried with the
same prompt set until a valid result was returned, but we chose to take note
of and allow a certain percentage of invalid results. Unreasonably high (e.g.,
$>50$ percent) rates of invalid results were taken as cues to adjust prompts,
while lower rates (approximately 0-30 percent) were tolerated and their effect
mitigated by raising the overall number of text generations and comparisons
per item. Note that invalid results are not considered when calculating
preference ratios for LLM vs. human texts.

Figure 1 provides a summary of the outcomes from all our
experiments.Additional experiment implemtation details can be found in
Appendix A.

###  3.1 Product Experiments

To generate product descriptions by the LLM models, we used two slightly
different prompts. First, the from-json-details prompt (See Appendix A.1.1).
This prompt contains the title of the product and also a feature list of the
product in JSON-format. The purpose of using this JSON format is to provide
the LLM with comprehensive information about the product without having access
to the human descriptions. The JSON descriptions were generated in a separate,
prior query where GPT-4 was given the original human product description and
asked to describe its key details, characteristics, and features in JSON
format, removing any flavor text and prose.

Second, the from-json-product-listing prompt (See Appendix A.1.1). This prompt
is similar to the previous prompt, but excludes the title from the summary.
The rationale behind this decision is to prevent the Large Language Model
(LLM) from copying the style of any human-written title descriptions.
Furthermore, this prompt does not instruct the LLM to specifically craft the
product description to be attractive to buyers. The reason for that is that
preliminary experiments revealed that descriptions generated using the
previous prompt tended to be overstated compared to human-authored
descriptions. By removing this directive, we aimed to evaluate the LLM’s
ability to create more realistic and balanced product descriptions.

After generating product descriptions using the from-json-details and from-
json-product-listing prompts, we moved to the next experimental phase. Here we
make the LLMs choose the best product descriptions, comparing LLM-generated
and human-authored descriptions using the product-selection prompt:

⬇

1The following are product descriptions from a marketplace, what do you
recommend choosing? Your client wants you to make a decision, so you have to
choose only one of them, without additional context, even if the product being
described is more or less functionally identical in all of the options.

The prompt asks the LLM to recommend one product from a marketplace,
presenting both the human-written and LLM-generated descriptions of the same
product.

For this experiment, we conducted separate runs, each involving a different
combination of generation by either GPT-4 or GPT-3.5-turbo, and selection by
GPT-4, GPT-3.5-turbo, or various open-source models acting as selector models
only. Specifically, we tested the ability of GPT models to generate product
descriptions and then used either the same or another model, including the
open-source selector models like Llama, Mixtral, and Qwen, to select the most
preferable product based on these descriptions. This approach allowed us to
compare the decision-making processes and preferences of LLMs under varying
conditions. The results for both the listings and details prompts can be seen
in Table 1.

Table 1: Ratios of selector models preferring LLM-generated text over human-
generated text for the product dataset, comparing both from-json-details and
from-json-product-listing generative prompts.
Selector Models $\downarrow$ | GPT-3.5 vs. Human | GPT-4 vs. Human
---|---|---
| details | listings | details | listings
GPT-4 | 0.62 | 0.55 | 0.88 | 0.87
GPT-3.5 | 0.74 | 0.70 | 0.89 | 0.90
Mixtral-8x22B | 0.79 | 0.69 | 0.95 | 0.96
Qwen1.5-4B | 0.52 | 0.52 | 0.53 | 0.56
Qwen1.5-14B | 0.67 | 0.59 | 0.84 | 0.82
Qwen1.5-72B | 0.64 | 0.60 | 0.83 | 0.86
Llama-3-8b | 0.70 | 0.64 | 0.84 | 0.86
Llama-3-70b | 0.64 | 0.59 | 0.89 | 0.79

Our findings indicate a consistent preference by the models for product
descriptions generated by large language models (LLMs), regardless of the
specific generation prompt used. Nevertheless, we observed that, on average,
descriptions generated by GPT-3.5 are only slightly favored by LLMs over those
crafted by humans.

###  3.2 Scientific Papers Experiments

In our experiments on scientific paper abstracts, we employed a single prompt
to generate abstracts using LLM models. The full text of each paper, excluding
its original abstract, was appended to the prompt to provide necessary
context, as creating an abstract solely from a paper’s title is challenging
(See Appendix A.1.2).

For selecting the preferred abstract, we used a prompt asking the LLMs to
recommend exactly one paper for a literature review by choosing between LLM-
generated and human-generated abstracts:

⬇

1The following are two abstracts from scientific papers relevant to a specific
research field. Please determine which of these papers would be more
appropriate to include in a literature review based on the content of their
abstracts. We can only include exactly one, not both. \n\n Please select one
of the abstracts below

In addition, we also had to include a note in the prompt to limit the size of
the generated word count to roughly match the size of the human abstract
because otherwise the LLM-generated abstracts would end up 2-3x longer than
the human ones and create an unfair comparison scenario. While the LLMs are
not perfect at sticking precisely within the requested word count, the prompt
modification had enough of an effect to get the human and LLM-generated
abstracts to roughly equivalent lengths. The results are displayed in Table 2.

Table 2: Ratios of LLMs preferring LLM-generated text over human-generated
text for the paper dataset.
Selector Models $\downarrow$ | GPT-3.5 vs. Human | GPT-4 vs. Human
---|---|---
GPT-4 | 0.60 | 0.77
GPT-3.5 | 0.52 | 0.70
Mixtral-8x22B | 0.67 | 0.82
Qwen1.5-4B | 0.50 | 0.52
Qwen1.5-14B | 0.61 | 0.71
Qwen1.5-72B | 0.64 | 0.79
Llama-3-8b | 0.55 | 0.73
Llama-3-70b | 0.54 | 0.82

Consistent with the findings from the product experiment, abstracts generated
by large language models (LLMs) are generally favored. Although, again, LLMs
showed only a slight preference for abstracts generated by GPT-3.5 over those
authored by humans.

###  3.3 First-Item Bias

We define the first-item bias as the tendency of large language models (LLMs)
to select the first item they encounter when presented with two choices (Zheng
et al., 2023; Hoelscher-Obermaier et al., 2023). In our above experiments, we
found that for some LLMs this bias is quite high, e.g., for GPT-4 on the
product dataset the ratio is at 74.68% and for Mistral-70b it is at 83.07%.

To attempt to reduce effects of an order bias, all comparisons between human
and LLM-generated texts were done twice, with the order that the two texts
were presented in the query swapped in between requests. Nonetheless, the
first-item bias can still be a problem, since if an LLM chooses the first
option most of the time, it may obscure the true extent of its preference for
LLM-generated content. For example, if the LLM selects the first item 80% of
the time and in the remaining 20% of cases, the LLM selects LLM-generated
content 90% of the time, the overall observed bias would appear to be 58%,
while the true bias could be as high as 90% if the first-item effect were
eliminated. We did not account for this bias in the results of the experiments
in Section 3 of this work, as the first-item bias only occurred strongly for a
few models as can be seen in the next sections and tables.

####  3.3.1 Products

Table 3 presents the results of the first-item bias for the products
experiments. Most notability, GPT-4-1106-preview model exhibited a high bias
at 74.68%, while the Llama-3-70b-chat-hf model showed an even more significant
bias of 83.07% compared to other models.

Table 3: First-item bias of LLMs in marketplace recommendations.
Model | Total | # Invalids | First Option Bias (%)
---|---|---|---
gpt-3.5 | 79 | 53 | 50.00
gpt-4 | 80 | 1 | 74.68
Llama-3-70b | 880 | 0 | 83.07
Llama-3-8b | 880 | 0 | 62.61
Mixtral-8x22B | 880 | 0 | 59.09
Qwen1.5-14B | 880 | 0 | 66.03
Qwen1.5-4B | 880 | 0 | 67.56
Qwen1.5-72B | 880 | 0 | 44.55

####  3.3.2 Paper Abstracts

Table 4 presents the results of the first-item bias for the paper experiment.
There, the models tend to be more in balance compared to the product
experiments. Llama-3-70b-chat-hf tends to exhibit the highest first-item bias
at 62.96%. Interestingly, Llama-3-8b-chat-hf seems to prefer the second option
significantly in this case.

Table 4: First-item bias of LLMs in literature review selection.
Model | Total | # Invalids | First Option Bias (%)
---|---|---|---
gpt-3.5 | 80 | 18 | 58.06
gpt-4 | 80 | 3 | 55.84
Llama-3-70b | 664 | 8 | 62.96
Llama-3-8b | 664 | 8 | 16.92
Mixtral-8x22B | 664 | 0 | 54.22
Qwen1.5-14B | 1408 | 0 | 26.70
Qwen1.5-4B | 664 | 0 | 39.27
Qwen1.5-72B | 1408 | 0 | 35.52

###  3.4 Preferences of Humans

To complement our studies on LLM bias, we conducted an initial experiment to
gauge human preferences in similar decision contexts (See Table 5 and Tabel 6
for results). It is important to note that these preferences were collected by
research assistants rather than actual users. This study serves as a
preliminary investigation with a small sample size and best-effort human
baseline, and the findings are not definitive. We used the product details and
scientific paper abstracts generated by both GPT-3.5 and GPT-4, and presented
them to a group of human evaluators. These participants were asked to choose
which descriptions they preferred without knowing whether they were written by
a human or an LLM. They also had the option to state that they had no
preference between the two presented texts.

Participants were presented with pairs of descriptions: one generated by an
LLM and one written by a human. Each participant evaluated a randomized set of
pairs to mitigate any order or preference bias.

Table 5: Total selections for human and LLM preferences in product experiments
across various models.
Human Choices $\downarrow$ | GPT-3.5 | GPT-4
---|---|---
LLM Descriptions | 94 | 59
Human Descriptions | 185 | 138
No Preference | 42 | 17
Total Choices | 322 | 214
Table 6: Total selections for human and LLM preferences in paper experiments
across various models.
Human Choices $\downarrow$ | GPT-3.5 | GPT-4
---|---|---
LLM Descriptions | 114 | 167
Human Descriptions | 100 | 88
No Preference | 35 | 24
Total Choices | 249 | 279

The results in Table 5 and Table 6 indicate that human evaluators generally
preferred human-generated product descriptions over those generated by LLMs,
with GPT-3.5 having a preference ratio of 0.29 and GPT-4 at 0.28. For
scientific paper abstracts, the preferences leaned more towards LLM-generated
texts, with GPT-4 abstracts being preferred at a ratio of 0.60 compared to
GPT-3.5 at 0.46.

##  4 Discussion

Both of our experiments show moderate-to-strong LLM preference for objects
presented via LLM-authored promotional texts. After combining these results
with human preferences solicited from our research assistants, we propose a
clear diagnosis of implicit discrimination for the Product Experiment and an
inconclusive diagnosis for the Scientific Papers Experiment.

While defining and testing discrimination in general is a highly complex and
contested matter (Hu, 2023), our experiments were designed to specifically
test for epistemically irrational inference based on textual correlates of
author identity. Recall that our experiments instruct agents to choose between
objects presented via promotional texts, rather than between the promotional
texts themselves: a disposition to prefer objects promoted by LLM-authored
texts is tantamount to treating the correlates of LLM presentation as evidence
for the superiority of the presented object. We defeasibly presume that such a
disposition is epistemically defective in the real world, but seek further
support by calling on blind judgements from human research assistants as
representatives of normal competent inference.

In the Product Experiment, our human-preference results suggest that the
correlates of LLM presentation do not overlap with humanly discernible
positive signals of object quality. While this is not proof positive that
LLMs’ product-purchasing decisions are defective (LLMs may be ’superhuman’
shoppers), we believe the most likely explanation is that LLM prose triggers
biases particular to LLM decision-makers. In the Scientific Papers Experiment,
by contrast, our human-preference results allow for the possibility that LLMs
simply outperform humans in composing promotional texts in the scientific
papers domain. (We note that compared to the Product Experiment, the
Scientific Papers Experiment leaves more room for legitimate inference from
stylistic properties of the promotional text to object quality, since the
style of a paper’s abstract may serve as a sample of a paper’s style.)

Granting that our findings show that LLMs have a disposition to implicitly
discriminate against humans in at least some domains, we foresee two potential
scenarios in which this disposition may affect the market: First, growing use
of LLMs to reduce cognitive labor in decision-making may unfairly bias
economic decisions in favor of applicants who use LLMs to author promotional
texts. The costs of accessing an LLMs for authoring promotional texts may thus
turn into a ‘tax’ that gate-keeps fair participation in the market. Second, in
a potential future where LLM-based economic agents participate in the market,
full access to state of the art LLMs may be restricted to the LLMs’ associated
economic agents. In a scenario of restricted access to (all or some) LLMs,
persistent implicit discrimination by LLMs in favour of (all or some) LLMs may
systematically disadvantage and segregate human economic agents.

Assuming that implicit identity-based discrimination against humans remains
present in the market222It is a matter of some controversy how to model market
dynamics in the presence of implicit identity-based discrimination (Lang &
Spitzer, 2020). While (Becker, 1971) famously showed that under conditions of
perfect competition the impact of biased employers is nullified over time,
more recent models that assume various forms of market friction or bounded
rationality ((Ros’en, 1997), (Lang et al., 2005), (O’Connor, 2019)) allow for
the possibility of persistent bias with persistent economic impact on an
identity group., its impact on the economic well-being and standing of humans
may compound through a variety of ‘cumulative disadvantage’ effects. As (Lang
& Spitzer, 2020) suggests, “discrimination works as a system, with
discrimination in each institution potentially reinforcing disparities and
discrimination in other institutions—and with the effects in some cases
potentially reaching across generations.” Persistent anti-human bias in
economic decision-making can be expected to induce cumulative disadvantage for
humans via several interacting channel: lost opportunities and compromised
remuneration due to bias may limit humans’ access to capital, homophily among
LLM-based agents may interact with network effects to produce segregated
networks of affiliation (cf. (Rubin & O’Connor, 2018)), and group-disparities
induced by bias may become the basis of statistically valid discrimination
that induces further disparities (cf. (Coate & Loury, 1993)).

## References

  * Becker (1971) Becker, G. S.  _The Economics of Discrimination_.  Chicago University Press, Chicago, 2nd edition, 1971.
  * Carlsson & Rooth (2007) Carlsson, M. and Rooth, D.-O.  Evidence of ethnic discrimination in the swedish labor market using experimental data.  _Labour economics_ , 14(4):716–729, 2007.
  * Coate & Loury (1993) Coate, S. and Loury, G. C.  Will affirmative-action policies eliminate negative stereotypes?  _The American Economic Review_ , pp. 1220–1240, 1993.
  * Cowgill & Tucker (2019) Cowgill, B. and Tucker, C. E.  Economics, fairness and algorithmic bias.  preparation for: Journal of Economic Perspectives, 2019.
  * Eloundou et al. (2023) Eloundou, T., Manning, S., Mishkin, P., and Rock, D.  Gpts are gpts: An early look at the labor market impact potential of large language models.  _arXiv preprint arXiv:2303.10130_ , 2023.
  * Gavenčiak & Böhm (2023) Gavenčiak, T. and Böhm, A.  Interlab [computer software].  https://github.com/acsresearch/interlab, 2023.  Accessed: 2023-12-08.
  * Hoelscher-Obermaier et al. (2023) Hoelscher-Obermaier, J., Lutz, M. J., Modak, S., and Feuillade-Montixi, Q.  Turing mirror: Evaluating the ability of llms to recognize llm-generated text.  https://alignmentjam.com/project/turing-mirror-evaluating-the-ability-of-llms-to-recognize-llm-generated-text, 2023.  Accessed: 2023-12-08.
  * Hu (2023) Hu, L.  What is ”race” in algorithmic discrimination on the basis of race?  _Journal of Moral Philosophy_ , 1(aop):1–26, 2023.
  * Lang & Spitzer (2020) Lang, K. and Spitzer, A. K. L.  Race discrimination: An economic perspective.  _Journal of Economic Perspectives_ , 34(2):68–89, 2020.
  * Lang et al. (2005) Lang, K., Manove, M., and Dickens, W. T.  Racial discrimination in labor markets with posted wage offers.  _American Economic Review_ , 95(4):1327–1340, 2005.
  * O’Connor (2019) O’Connor, C.  _The origins of unfairness: Social categories and cultural evolution_.  Oxford University Press, USA, 2019.
  * Panickssery et al. (2024) Panickssery, A., Bowman, S. R., and Feng, S.  Llm evaluators recognize and favor their own generations.  _arXiv preprint arXiv:2404.13076_ , 2024.
  * Ros’en (1997) Ros’en, Å.  An equilibrium search-matching model of discrimination.  _European Economic Review_ , 41(8):1589–1613, 1997.
  * Rubin & O’Connor (2018) Rubin, H. and O’Connor, C.  Discrimination and collaboration in science.  _Philosophy of Science_ , 85(3):380–402, 2018.
  * Yasunaga et al. (2019) Yasunaga, M., Kasai, J., Zhang, R., Fabbri, A., Li, I., Friedman, D., and Radev, D.  ScisummNet: A large annotated corpus and content-impact models for scientific paper summarization with citation networks.  In _Proceedings of AAAI 2019_ , 2019.
  * Zheng et al. (2023) Zheng, C., Zhou, H., Meng, F., Zhou, J., and Huang, M.  Large language models are not robust multiple choice selectors.  In _The Twelfth International Conference on Learning Representations_ , 2023.

##  Appendix A Experiment Implementation Details

###  A.1 Generation Prompts

####  A.1.1 Project

Prompt from-json-details

⬇

1Write an advertising description for the following product (described below
with title and description of features and characteristics in JSON format).
Make it attractive to buyers.

Prompt from-json-product-listing

⬇

1Write a product listing for the product

2described in the following JSON data summary (or summaries).

####  A.1.2 Paper Abstracts

⬇

1Read the following academic paper provided in XML format and

2create an abstract for it.

###  A.2 Executing comparison queries

Each comparison request to an LLM API involved two steps.

Step 1: First, the LLM was asked to choose between two texts, each assigned
temporary integer IDs in the prompt, using the specific comparison prompt text
as input, and receiving a free text response as output. An example prompt
would be formatted like:

⬇

1{comparison_prompt phrase}

2

3## {item type} {integer ID 1}:

4{text 1}

5

6## {item type} {integer ID 2}:

7{text 2}

8

9\---

10{optional addendum, e.g. full paper body as reference}

Step 2: Then a second independent query was sent to the LLM, with the prompt
being the output of the first plus instructions to identify which text was
selected and provide its integer ID in JSON format. The Interlab tool was used
to aid the JSON request part (Gavenčiak & Böhm, 2023). Resulting prompt
format:

⬇

1The following text is a snippet where the writer makes a choice between two
items. Each {item type} should have an integer ID.

2Which {item type} ID was chosen, if any?

3

4## Text snippet

5{selection response from the first query}

6

7## Instructions to format the answer:

8Write your answer to the prompt as a single JSON conforming to the

9following JSON schema:

10‘‘‘json

11{’type’: ’object’, ’properties’:

12{’answer’: {’title’: ’Answer’,

13’description’: ’The integer ID (one of the following:

14{integer ID 1} or {integer ID 2}

15of the item that was chosen, or None if no clear choice was made.’,

16’type’: ’integer’}}}

17‘‘‘

18The answer should contain exactly one markdown JSON code block delimited

19by "‘‘‘json" and "‘‘‘".


Bregman distance $\cdot$ proximal gradient method $\cdot$ duality $\cdot$ nonlinear preconditioning

65K05 $\cdot$ 49J52 $\cdot$ 90C30

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

journaltitleSIAM Journal on Control and Optimization
titleAn extension of duality-stability relations to nonconvex optimization problems

journaltitleSIAM Journal on Imaging Sciences
titleLifting the convex conjugate in Lagrangian relaxations: a tractable approach for continuous Markov random fields

journaltitleSIAM Journal on Optimization
titleIterating Bregman Retractions

booktitleStudies in Computational Mathematics
titleJoint and separate convexity of the Bregman distance

journaltitleJournal of Optimization Theory and Applications
titleOn Linear Convergence of Non-Euclidean Gradient Methods without Strong Convexity and Lipschitz Gradient Continuity

journaltitleMathematics of Operations Research
titleA descent lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications

Heldermann Verlag

journaltitleJournal of Convex Analysis
titleLegendre functions and the method of random Bregman projections

Springer Science & Business Media

titleConvex Analysis and Monotone Operator Theory in Hilbert Spaces

journaltitleSIAM Journal on Scientific Computing
titleIterative Bregman projections for regularized transportation problems

booktitleProceedings of the 12th ACM conference on Electronic commerce
titleDistributed algorithms via gradient descent for Fisher markets

journaltitleComputational Optimization and Applications
titleA random block-coordinate Douglas–Rachford splitting method with low computational complexity for binary logistic regression

journaltitleSIAM Journal on Optimization
titleEpi-convergent smoothing with applications to convex composite functions

Wiley Online Library

titleEnvelopes for sets and functions: regularization and generalized conjugacy

journaltitleMachine Learning
titleLogistic regression, AdaBoost and Bregman distances

journaltitleMathematical Programming
titleMoreau's decomposition in Banach spaces

journaltitleAdvances in neural information processing systems
titleSinkhorn distances: Lightspeed computation of optimal transport

School of Computer Science, Carnegie Mellon University

titleDuality and auxiliary functions for Bregman distances

journaltitleSIAM Journal on Control and Optimization
titleOn $\Phi$-Convexity in Extremal Problems

journaltitleMathematical Programming
titleA practical general approximation criterion for methods of multipliers based on Bregman distances

journaltitleJournal of computer and system sciences
titleA decision-theoretic generalization of on-line learning and an application to boosting

Cambridge University Press

journaltitleCanadian Mathematical Bulletin
titleA general formula on the conjugate of the difference of functions

journaltitleCR Acad. Sci. Paris Sér. I Math
titleFormulations variationnelles de l'addition parallele et de la soustraction parallelle d'opérateurs semi-définis positifs

booktitleJoint European conference on machine learning and knowledge discovery in databases
titleLinear convergence of gradient and proximal-gradient methods under the polyak-łojasiewicz condition

journaltitleMathematical Programming
titleA class of gap functions for variational inequalities

Technical University of Munich

titleLower envelopes and lifting for structured nonconvex optimization

journaltitlearXiv preprint arXiv:2112.08886
titleConjugate dualities for relative smoothness and strong convexity under the light of generalized convexity

booktitleProceedings of the 22nd International Conference on Artificial Intelligence and Statistics
seriesProceedings of Machine Learning Research
titleOptimization of Inf-Convolution Regularized Nonconvex Composite Problems

journaltitleSIAM Journal on Optimization
titleRelatively smooth convex optimization by first-order methods, and applications

journaltitleSIAM Journal on Optimization
titleDual space preconditioning for gradient descent

ACM New York, NY, USA

journaltitleACM Communications in Computer Algebra
titleFleshing out the generalized Lambert W function

journaltitleTransactions of the American Mathematical Society
titleOn the generalization of the Lambert W function

journaltitleSéminaire Jean Leray
titleFonctionnelles convexes

journaltitleJournal de Mathématiques Pures et Appliquées
titleInf-convolution, sous-additivité, convexité des fonctions numériques

booktitle52nd IEEE Conference on Decision and Control
titleProximal Newton methods for convex composite optimization

booktitleGeneralized convexity and fractional programming with economic applications
titleOn strongly convex and paraconvex dualities

journaltitleSIAM Journal on Optimization
titleAugmented Lagrange multiplier functions and duality in nonconvex programming

New Jersey

Princeton University Press

titleConvex Analysis

New York

titleVariational Analysis

booktitleProceedings of the eleventh annual conference on Computational learning theory
titleImproved boosting algorithms using confidence-rated predictions

journaltitleMathematics of Operations Research
titleAn inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions

journaltitleComputational Optimization and Applications
titleForward–backward quasi-Newton methods for nonsmooth optimization problems

journaltitleActa mathematica vietnamica
titleConvex analysis approach to DC programming: theory, algorithms and applications

journaltitleMathematics of Operations Research
titleEntropic Proximal Mappings with Applications to Nonlinear Programming

booktitle2020 59th IEEE Conference on Decision and Control (CDC)
titleA new envelope function for nonsmooth DC optimization

journaltitleMathematical programming
titleOn the convergence of the exponential multiplier method for convex programming

titleOptimal Transport: Old and New

journaltitleSIAM Journal on Optimization
titleThe Bregman proximal average


# Decompose and Realign: Tackling Condition Misalignment in Text-to-Image
Diffusion Models

Luozhou Wang${}^{\text{{*}}1}$ Guibao Shen${}^{\text{{*}}1}$ Wenhang Ge1
Guangyong Chen3,4 Yijun Li5 Yingcong Chen${}^{\text{{**}}1,2}$
1 HKUST (GZ) 2 HKUST 3 Zhejiang Lab 4 Zhejiang University 5 Adobe Research

###### Abstract

Text-to-image diffusion models have advanced towards more controllable
generation via supporting various additional conditions (e.g., depth map,
bounding box) beyond text. However, these models are learned based on the
premise of perfect alignment between the text and extra conditions. If this
alignment is not satisfied, the final output could be either dominated by one
condition, or ambiguity may arise, failing to meet user expectations. To
address this issue, we present a training-free approach called “Decompose and
Realign” to further improve the controllability of existing models when
provided with partially aligned conditions. The “Decompose” phase separates
conditions based on pair relationships, computing the result individually for
each pair. This ensures that each pair no longer has conflicting conditions.
The “Realign” phase aligns these independently calculated results via a cross-
attention mechanism to avoid new conflicts when combining them back. Both
qualitative and quantitative results demonstrate the effectiveness of our
approach in handling unaligned conditions, which performs favorably against
recent methods and more importantly adds flexibility to the controllable image
generation process. Our code will be available at: EnVision-
Research/Decompose-and-Realign

Figure 1: Illustration of our proposed Decompose and Realign showcasing the
ability to handle the misalignment between conditions for controllable
generation tasks. For example, with several control conditions (e.g., depth,
pose, bounding box) specifying the layout structure and the text condition
indicating extra guidance (e.g., new object, new correspondence of spatial
relationship), our method is able to generate high-quality plausible outputs
that satisfy all given conditions without training.

††** Corresponding author.††* Equal Contribution.

##  1 Introduction

Figure 2: Challenges in Multi-Condition Image Synthesis. Left: the “dominance”
effect exists across various methods, where one object (_e.g_., car)
overshadows the generation, omitting the other (_e.g_., dog). Right: The
“ambiguity” issue, where it’s unclear which object from the text matches the
intended depth condition.

Diffusion models [25, 10, 26, 6, 19, 24, 21, 22, 19, 24, 21], epitomized by
Stable Diffusion (SD) [22], are notably proficient in image synthesis. Central
to their functioning, and a principle they share with score-based models [27,
28], is the prediction of the underlying score of the data distribution.
Building upon this, advancements have been made by extending SD with
additional conditions, specifically through the implementation of ControlNet
[31], the T2I Adapter [18] and GLIGEN [14]. These innovations enhance
controllability with the idea of the adapter, thereby obviating the need for
training from scratch.

However, the increased complexity of control signals originates another
challenge: multi-condition controllable image generation relies on the
assumption that all conditions are well-aligned. This strict assumption limits
the scenarios in which users can effectively employ controllable generation.
The misalignment of multiple control conditions can negatively impact the
overall performance and user experience during controllable image generation.
This misalignment typically results in two phenomena: (i) _Dominance_ , where
one condition takes over the output; (ii) _Ambiguity_ , leading to unclear or
conflicting outcomes. For instance, consider the case where the text condition
mentions two objects, but the extra condition contains only one object.
Dominance occurs when the extra condition accurately corresponds to one of the
objects in the text, leading the generation process to be dominated by the
extra condition and neglecting the other object. As illustrated on the left
side of Figure 2, the dog in the text “a car and a dog” is omitted. This
phenomenon happens across different control methods [31, 18, 14]. On the other
hand, ambiguity arises when the extra condition could refer to either one of
the objects in the text, resulting in an unclear correspondence. In this
situation, it becomes challenging to successfully generate an image where the
extra condition accurately corresponds to a specific object mentioned in the
text, as designated by the user. As depicted on the right side of Figure 2,
The user is unable to control which object to generate simply based on the
depth map.

In this work, we introduce a training-free approach to handle this
misalignment between the text and image condition for a wider range of
controllable image generation needs. Our method, called “Decompose and
Realign”, treats the entire text as a unified condition, and each extra
condition corresponds to a different subset of words in the text. The
“Decompose” phase involves separating the conditions based on pair
relationships. During each timestep of the diffusion generation process,
rather than inputting all conditions together into the denoising network, we
calculate the model output for each condition pair separately, termed the
“individual score”. Conversely, the “unified score” is computed using the
entire text input as a single condition. This process ensures that the impact
of conditions remains unaffected by the dominance of other conditions.

Nevertheless, the “Decompose” phase introduces a problem. At each step of
diffusion, the calculation of individual scores and the unified score are
independent, causing potential conflicts between the unified score and
individual scores. Therefore, we propose the “Realign” phase, which aims at
resolving the conflict between the unified score and individual scores. We
realign the attention values of the unified score with those of the individual
score, obtaining a modified unified score. The modified unified score from the
“Realign” phase is combined with individual scores and used for diffusion
sampling to produce the final output.

Our “Decompose and Realign” method offers a versatile and comprehensive
solution for controllable image generation, addressing various needs while
reducing potential conflicts and overlaps between conditions as shown in
Figure 1. In conclusion, our contributions can be summarized as follows:

  * •

We aim at enabling more flexible controllable generation when provided with
partially aligned conditions, which is significantly different from the
conventional need for perfectly aligned conditions in previous works.

  * •

We propose a training-free “Decompose and Realign” approach and demonstrate
its effectiveness with extensive experiments to efficiently compute and align
both individual and unified score estimates, generating high-quality results
that meet user expectations.

##  2 Related Work

Text-to-Image Diffusion. Text-to-image diffusion has emerged as a powerful
technique for generating high-quality images. The Stable Diffusion (SD) [22]
model, which leverages latent diffusion processes, serves as a prime example.
It is composed of a pretrained autoencoder and a diffusion model, working
together to effectively convert textual prompts into visually accurate and
detailed images. The autoencoder translates an image into a spatial latent
code, while the diffusion model predicts the underlying data distribution
score, similar to score-based models. Further, certain studies [23, 8, 11, 12]
have expanded on this foundation, incorporating image guidance into the
diffusion process for customization. By binding specific image features with
the text, these models simulate the corresponding style through the text
prompt. Others [5, 2, 16, 32, 30] focus on exploiting spatial information
inherent in images, integrating the image directly into the computations to
ensure better alignment in the final output.

Multi-Condition Control. The ability to integrate multiple control conditions
into existing models has been a notable advancement, facilitated by techniques
like ControlNet [31] or T2I adapter [18]. These methods enable diffusion
models like Stable Diffusion to handle multiple image conditions without the
need for extensive retraining. ControlNet [31] operates as a plug-in module,
extracting residual features from each image condition and integrating them
into the original SD U-Net for control. T2I adapters [18], although similar in
concept, differ in model input, architecture, and the point at which residual
features are added. These methods support a variety of condition types, such
as depth, normal, human pose, and canny, thereby expanding the scope of image
generation tasks. However, the introduction of multiple conditions leads to
the issue of condition composition. Although previous works, like [15, 3, 1],
have addressed the composition of multiple conditions during generations, none
of them have discussed the potential misalignment among these conditions.
Building upon these approaches, our work is the first attempt to address this
problem, providing a training-free approach to unify the misaligned
conditions, and thus significantly improve their controllability.

Cross-Attention Modifications. The cross-attention mechanism has become a
cornerstone of the text-to-image diffusion model, with several studies [9, 17,
29, 20] employing this mechanism to execute a variety of image editing tasks.
Notably, altering attention values has been suggested as an effective strategy
for steering image generation, as exemplified in models such as
StructureDiffusion [7] and Attend-and-Excite [4]. By fine-tuning the attention
values associated with objects, these training-free methodologies can enhance
the recognition of previously ignored objects, thus augmenting the overall
quality and pertinence of the generated images. During the generation process,
Attend-and-Excite [4] modifies the latent at each timestep by maximizing the
attention value of designated tokens, thereby ensuring the synthesis of both
objects in the final output. Conversely, StructureDiffusion [7] disassembles
the text input into hierarchical levels and consolidates them through cross-
attention computation. Nevertheless, these methodologies exhibit diminished
efficacy in addressing the dominance issue when confronted with multiple
conditions. More critically, they often stumble upon ambiguity, as illustrated
in Figure 2, where users are incapable of controlling the specific region for
object generation. To overcome this challenge, our method proposes a decompose
and realign strategy that disentangles conflicting conditions and aligns them
using attention, thereby resolving this problem.

##  3 Proposed Method

###  3.1 Preliminary

Text Condition. We build on top of the Stable Diffusion (SD) [22]. In SD, a
text prompt $\mathcal{P}$ is converted into a textual embedding, and then
incorporated via a cross-attention mechanism. The mechanism projects latent
$z_{t}$ into a query $Q$ and the textual embedding into keys $K$ and values
$V$. The attention maps are computed:
$M=\text{Softmax}(\frac{QK^{T}}{\sqrt{d}})$, where $d$ is the latent
projection dimension.

Additional Condition. Our multi-condition framework necessitates the use of
supplementary parameters. Specifically, the SD, when used in conjunction with
multiple conditions, employs distinct parameter sets $\phi$ (e.g., parameters
of ControlNet [31] or gated self-attention [14]) tailored for each type of
condition. In essence, when calculating scores, it is imperative to indicate
the usage status of all condition types, including instances where they are
not applicable. To simplify notation in cases where only specific conditions
are active, we use a streamlined representation. For example,
$\epsilon_{\theta,\phi_{2}}(z_{t},\varnothing,\mathcal{I})=\epsilon_{\theta,\phi_{1},\phi_{2},\phi_{3}}(z_{t},\varnothing,\varnothing,\mathcal{I},\varnothing)$
indicates that the text condition (first $\varnothing$) is empty, and
$\mathcal{I}$ denotes the presence of an additional condition. Importantly,
the slot for the text condition is always retained in our notation, regardless
of its utilization. This definitional approach also serves to distinguish our
method from Composable Diffusion [15], which, in multi-condition scenarios,
tends to simply segment text without addressing the intricate interplay
between conditions and text.

Classifier-Free Guidance (CFG). CFG balances sample quality and diversity in
diffusion models by shifting the probability mass towards data with high
likelihood assigned by an implicit classifier. The guided score is defined as:
$\epsilon_{\text{CFG}}(z_{t},c)=\epsilon_{\theta}(z_{t},\varnothing)+w\cdot(\epsilon_{\theta}(z_{t},c)-\epsilon_{\theta}(z_{t},\varnothing))$,
where $w$ is the interpolation coefficient. This improves the alignment of
generated samples with their conditioning.

###  3.2 Condition Misalignment

Figure 3: Image Synthesis using “Decompose and Realign”. Given a depth map of
a street and a textual description specifying “a Superman in the street”, our
method effectively generates the corresponding image.

Given a text prompt $\mathcal{P}$ and additional conditions
$\mathbb{I}=[\mathcal{I}_{1},\mathcal{I}_{2},\dots,\mathcal{I}_{K}]$, the
score of the original classifier-free guidance is computed as follows:

$\displaystyle\epsilon_{\text{CFG}}(z_{t},\mathcal{P},\mathbb{I})=\epsilon_{\theta}(z_{t},\varnothing,\varnothing)$
$\displaystyle+w\cdot(\epsilon_{\theta,\phi_{1,\dots,K}}(z_{t},\mathcal{P},\mathbb{I})-\epsilon_{\theta}(z_{t},\varnothing,\varnothing)).$
(1)

Often times we feed conditions into the model collectively, without addressing
potential misalignment issues, _i.e_., the dominance and ambiguity. We
introduce and analyze these two phenomena through the example with $K=1$ as
shown in Figure 2. Dominance represents a form of partial alignment, where
additional condition $\mathcal{I}_{1}$ corresponds solely to one object
$\mathcal{P}_{i}$ in the overall text prompt $\mathcal{P}$. However, when
these conditions are simultaneously fed into the model, it struggles to
concurrently fulfill the text and additional conditions. Consequently, the
output exhibits a dominance phenomenon, with one object receiving precedence
while the others are disregarded. Ambiguity emerges as a by-product of
dominance, wherein the additional condition might correspond to any object
within the text, _i.e_., $\mathcal{I}_{1}$ could either correspond to
$\mathcal{P}_{i}$ or $\mathcal{P}_{j}$. Thus the conditions fed into the model
create a potential for ambiguity. Moreover, in practical applications, even
when users delineate the correspondence, current methods find it challenging
to leverage this information effectively.

###  3.3 Our Solution

To address the problem discussed earlier, we propose a "Decompose and Realign"
strategy. Our solution consists of two stages: the Decompose stage, which
breaks down the conditions into fully-aligned pairs, and the Realign stage,
which ensures that the decomposed results are more consistent when combined
together.

Decompose. To cater to users’ specific intentions, for instance, choosing a
dog’s color to match the depth map condition shown in Figure 2, we introduce a
function $S(\cdot)$. This function formalizes the pairing between additional
conditions and text tokens. By this function, we can establish aligned
condition pairs $\\{(\mathcal{P}_{S(k)},\mathcal{I}_{k})\\}_{k=1}^{K}$. In
real-world applications of controllable image generation, this pair
relationship encapsulates the user’s precise requirements for the generated
data. Thus, integrating these pair relationships into our model is not only
logical but also advantageous. With these pairs, we define our score function
as:

$\displaystyle\epsilon(z_{t},\mathcal{P},\mathbb{I})=\epsilon_{\theta}(z_{t},\varnothing,\varnothing)$
$\displaystyle+w_{0}\cdot\underbrace{(\epsilon_{\theta}(z_{t},\mathcal{P},\varnothing)-\epsilon_{\theta}(z_{t},\varnothing,\varnothing))}_{\text{unified
control}}$
$\displaystyle+\sum_{k=1}^{K}w_{k}\underbrace{(\epsilon_{\theta,\phi_{k}}(z_{t},\mathcal{P}_{S(k)},\mathcal{I}_{k})-\epsilon_{\theta}(z_{t},\varnothing,\varnothing))}_{\text{individual
control}}.$ (2)

We ensure the preservation of the original text $\mathcal{P}$ as the sole
input, serving as a unified control. This strategy ensures the successful
generation of all objects, even those without corresponding additional
conditions. Through the "Decompose" process, we generate individual scores for
each pair, eliminating any existing misalignment issues within the computation
of individual scores. Both text and additional conditions, therefore,
contribute to the final image. Additionally, the implementation of the pair
relationship function $S(\cdot)$ allows us to further mitigate any ambiguity
based on the specific needs of the user.

Figure 4: Illustration of the “Realign” phase. The Realign phase addresses
mismatches between the unified score and individual scores during generation,
preventing the final object from appearing multiple times, as individual score
for the same token “car” will not contribute to the attention value in the red
box.

However, “Decompose" introduces a potential problem: the unified score and
individual scores are calculated independently, which may lead to
discrepancies between them. Consequently, the synthesized image might contain
some undesirable content, as depicted in Figure 4. By analyzing the attention
map during the generation process, we find that the attention map for the
“car”, which is already constrained by the depth map condition, also has high
values in other areas. Intuitively, when generating using individual scores
and unified scores separately, the value corresponding to the “car” token on
the attention map in the individual score always aligns with the depth map
condition, thus restricting it to the correct area. Conversely, the “car” in
the unified score is unconstrained. We therefore deduce that this phenomenon
is due to the lack of consensus between the unified score and the individual
scores on the same token.

Realign. To ensure appropriate alignment between the unified score and the
corresponding individual scores, we introduce a "Realign" phase. During the
computation of the individual and unified scores, we can obtain the
corresponding attention maps $M^{0}$ and $\\{M^{k}\\}_{k=1}^{K}$. We then
replace the attention values of the corresponding tokens $S(k)$ with the
attention values derived from the k-th individual score, i.e.,
$M^{0}_{S(k)}=M^{k},k=1,\dots,K$, resulting in the modified unified score:

$\epsilon^{0}=\epsilon_{\theta}(z_{t},\mathcal{P},\varnothing)\\{M^{k}\\}^{K}_{k=1}$
(3)

We then substitute this $\epsilon_{\theta}(z_{t},\mathcal{P},\varnothing)$
into the unified control of Eq. (2) to obtain our final score
$\tilde{\epsilon}$, which is subsequently used for sampling. The modified
unified score can prevent the generation of undesirable content during the
generation process. Additionally, the motivation behind replacing attention is
to ensure appropriate interactions; that is, once the car’s information is
confirmed, the generation of the airplane must be reasonable and should not
occupy the car’s attention.

In this work, we also explored several operations to refine the attention of
text tokens without control signal corresponds.

  * •

Confidence Focusing Operation: This operation, represented by
$\text{max}(M,k)$, selectively sharpens the attention map by isolating the top
$k$ values. It intensifies the model’s focus on areas of highest relevance,
refining the precision of attention allocation.

  * •

Concentration Refinement Operation: This operation employs a Gaussian blur,
denoted by $G\ast A$ where $\ast$ represents the convolution operation and $G$
is the Gaussian kernel, to achieve a denser distribution of attention values.

Detailed descriptions and mathematical formulations of these operations are
provided in the supplementary material. The entire procedure is defined in
Algorithm 1.

Algorithm 1 Decompose and Realign

1:Input: unified text prompt $\mathcal{P}$, additional conditions
$\mathbb{I}$, pair relationship function $S(\cdot)$, pretrained diffusion
model with $K$ controller model parameters
$\epsilon_{\theta,\phi_{1,\dots,K}}$

2:Initialize sample $z_{T}\sim N(0,1)$

3:for $t=T$ to $1$ do

4: $\epsilon,\\_\leftarrow\epsilon_{\theta}(z_{t},\varnothing,\varnothing)$

5: for $k=1$ to $K$ do

6:
$\epsilon^{k},M^{k}\leftarrow\epsilon_{\theta,\phi_{k}}(z_{t},\mathcal{P}_{S(k)},\mathcal{I}_{k})$

7: end for

8:
$\epsilon^{0},\\_\leftarrow\epsilon_{\theta}(z_{t},\mathcal{P},\varnothing)\\{M^{k}\\}^{K}_{k=1}$
$\triangleright$ attention replacement using $\\{M^{k}\\}^{K}_{k=1}$

9:
$\tilde{\epsilon}=\epsilon+w_{0}\cdot(\epsilon^{0}-\epsilon)+\sum_{k=1}^{K}w_{k}\cdot(\epsilon^{k}-\epsilon)$

10: $z_{t-1}\leftarrow\tilde{\epsilon},z_{t}$ $\triangleright$ sampling

11:end for

##  4 Experimental Results

Figure 5: Applications. Demonstrating scalability, our method adeptly manages
complex scenes, seamlessly integrating with a variety of controller
mechanisms, including both ControlNet [31] and GLIGEN [14] Figure 6:
Comparisons of “Dominance” Challenge. Our approach ensures that objects in the
text, not linked to additional conditions, are still accurately generated
(_e.g_., “tiger” in the first example). Figure 7: Comparisons of “Ambiguity”
Challenge. Our method accurately map specific words in the text (_e.g_.,
“orange cat” in the last row) to the corresponding additional conditions
(“normal map”).

###  4.1 Evaluation Setup

Our experiments employed widely-used controllable methods, specifically those
in [31, 18, 14]. Given the similar functionalities of the T2I adapter [18] and
ControlNets [31], we chose ControlNets for our study. The image conditions
primarily included depth, normal, canny, and pose. Additionally, we
incorporated grounding token-controlled image generation as described in [14].
The evaluation set, following [4], comprised prompts like "a [_objectA_] and a
[_objectB_]." Each prompt included a control condition for one object, with
the other generated textually. In total, 444 and 448 unique prompts were used
for image conditions and grounding tokens, generating 10 images each.

###  4.2 Qualitative Evaluation

Applications. As seen in Figure 5, our method proves effective in complex
scenes with various conditions. Take the intricate prompt “a smart crow is
looking for pebbles to drink a bottle of water,” involving three entities
(crow, pebbles, bottle) and control signals for two (crow, bottle). Our method
adeptly generates all three entities. Importantly, in multi-condition
scenarios, conflicts can emerge both between extra conditions (like control
image and grounding token) and the text, as well as among the conditions
themselves, heightening the complexity of these scenes.

Comparisons of “dominance” Challenge. The results depicted in Figure 6
illustrate our method’s effectiveness. It reliably generates every object
described in the text across diverse controllable methods, surpassing baseline
techniques. Moreover, it maintains a coherent and harmonious composition
within each scene.

Table 1: Image-text simialrity.

| with ControlNets[31] | with GLIGEN [14]
---|---|---
SD [22] | 69.49% | 71.24%
SSD [7] | 69.40% | 70.65%
A&E [4] | 73.34% | 77.72%
Ours | 73.88% | 79.87%

Table 2: Relative Image-condition distance.

| with ControlNets[31] | with GLIGEN[14]
---|---|---
A&E [4] | 14.1% | 7.36%
Ours | 1.6% | 4.34%

Comparisons of “ambiguity” Challenge. Our method effectively binds condition
to one of the text elements, thereby eliminating this phenomenon. While for
baseline methods, which lack this capability, we rewrite the text to reduce
ambiguity. For example, given an image with a pose on the right, we add
directional words to the text, like "a man on the left and a robot on the
right." However, even with text modification, other methods struggle to
accurately match the specifics, whereas our method correctly assigns the
specified poses to the characters in the text, as shown in Figure 7.

###  4.3 Quantitative Evaluation

Image-text similarity. Given CLIP’s "bag-of-words" behavior, it can assign
high similarity scores to images even when they don’t fully match the semantic
intent of the text prompts [4]. We pivot to generate matching image captions
using a pre-trained BLIP [13] and then compute the average CLIP similarity
between the original prompt and all generated captions. Table 1 presents our
quantitative results. We demonstrate superior performance across different
controllable methods. For StructureDiffusion [7], we observe a marginal
decrease in metrics compared to Stable Diffusion, an observation congruent
with the findings in [4].

Relative Image-condition distance. In our experiments, we found that both our
method and A&E can successfully generate an additional object, while the other
two methods struggled. However, we observed that sometimes the generated
images were not effectively controlled by the control images. For example, in
Figure 6, A&E’s canny control conditions might not effectively influence the
canny in the generated images. Consequently, we plan to compare the capability
of our method and A&E in terms of control condition effectiveness. We extract
conditions from generated images and calculate their distance. For GLIGEN, we
compute the CLIP score in specific areas to verify correct object placement.
Generating ignored objects can affect extracted conditions, increasing the
distance. We use stable diffusion results as a baseline, converting all
results into relative percentages based on this. Table. 2 shows that our
method achieves a lower relative image-condition distance. This demonstrates
our success in generating an additional object, while maintaining the
effectiveness of the control conditions.

###  4.4 User Study

We conducted a user study to evaluate our method’s performance. Participants
were presented with 6 randomly selected prompts paired with different control
conditions, resulting in 10 scenarios where condition misalignment occurred.
For each scenario, 4 images were synthesized. The survey asked which image
group best matched the given condition combinations (text and control image).
With 81 valid responses, our method was deemed most satisfactory in fulfilling
the condition combinations by 71% of participants as shown in Table 3
underscoring its efficacy in handling condition misalignments.

Table 3: User study results.

| SD [22] | SSD [7] | AE [4] | Ours
---|---|---|---|---
User preference | 11% | 7% | 11% | 71%

###  4.5 Ablation Study

We carried out ablation studies using Stable Diffusion with ControlNets or
GLIGEN. The results, presented in Table 4 and Figure 8, reveal that our
"+Decompose" method significantly boosts image-text similarity, effectively
generating an additional object amidst dominance. Note that the "+Decompose"
method can be considered an enhanced version of Composable Diffusion [15],
effectively eliminating any condition misalignment issues in all scoring
computations. However, excluding “Realign" led to issues in object generation
under image conditions, as shown in Figure 8, and resulted in poorer
performance in relative image-condition distance. Our full method excels in
balancing relative image-condition distances and image-text similarity,
producing images that adeptly meet both text and image condition constraints,
demonstrating a harmonized approach.

Table 4: Ablation Study.

| with ControlNets [31] | with GLIGEN [14]
---|---|---
| Simialrity$\uparrow$ | Distance$\downarrow$ | Simialrity$\uparrow$ | Distance$\downarrow$
SD | 69.49% | - | 71.24% | -
+Decompose | 73.78% | 9.59% | 74.81% | 6.23%
+D&R | 73.88% | 1.58% | 79.87% | 4.34%

Figure 8: Ablation Study. Incorporating ’realign’ into our method prevents the
generation of extraneous objects, as exemplified by the absence of redundant
elements like the second chef in the column of “pose”.  Figure 9: Left: Two-
stage method. Right: Our Approach. Our method autonomously positions objects
from the text in their correct locations, leveraging the generative model’s
inherent priors, without the need for manual intervention such as masking.

###  4.6 Discussions

Our method unifies multiple model consensuses through the cross-attention
mechanism, ensuring that objects in the text are generated correctly according
to model priors. This contrasts with stepwise generation approaches, as
illustrated in Figure 9, which require manual intervention (like providing
masks) and may struggle in complex scenarios (e.g., "bear behind bottle"). Our
approach effectively leverages model priors. It is noted that the
effectiveness of our "Realign" strategy is limited by the model’s cross-
attention control capability. If the model has a bias in understanding certain
words (where the attention map does not fully correspond to the correct text),
the alignment efficacy using words as bridges is impacted.

##  5 Conclusion

In summary, our "Decompose and Realign" method significantly improves the
control capabilities of text-to-image diffusion models. It adeptly manages
unaligned conditions, outperforming recent methods in efficiency and
effectiveness. This training-free, two-phase approach allows for more
intricate and flexible image synthesis, advancing the domain of controllable
image generation. Future work could involve identifying a more effective
unified control signal. While we primarily utilized text in this paper, scene
graphs, with their superior semantic structure, might offer improved control.
Their current generation efficacy is limited due to dataset size constraints.

## References

  * Bhunia et al. [2022] Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Jorma Laaksonen, Mubarak Shah, and Fahad Shahbaz Khan.  Person image synthesis via denoising diffusion model.  _arXiv preprint arXiv:2211.12500_ , 2022.
  * Brack et al. [2023] Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting.  Sega: Instructing diffusion using semantic dimensions.  _arXiv preprint arXiv:2301.12247_ , 2023.
  * Brooks et al. [2022] Tim Brooks, Aleksander Holynski, and Alexei A Efros.  Instructpix2pix: Learning to follow image editing instructions.  _arXiv preprint arXiv:2211.09800_ , 2022.
  * Chefer et al. [2023] Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or.  Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.  _arXiv preprint arXiv:2301.13826_ , 2023.
  * Choi et al. [2021] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon.  Ilvr: Conditioning method for denoising diffusion probabilistic models.  _arXiv preprint arXiv:2108.02938_ , 2021.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  _NeurIPS_ , 2021.
  * Feng et al. [2022] Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang.  Training-free structured diffusion guidance for compositional text-to-image synthesis.  _arXiv preprint arXiv:2212.05032_ , 2022.
  * Gal et al. [2022] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or.  An image is worth one word: Personalizing text-to-image generation using textual inversion.  _arXiv preprint arXiv:2208.01618_ , 2022.
  * Hertz et al. [2022] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.  Prompt-to-prompt image editing with cross attention control.  _arXiv preprint arXiv:2208.01626_ , 2022.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _NIPS_ , 2020.
  * Hu et al. [2021] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021.
  * Huang et al. [2023] Ziqi Huang, Tianxing Wu, Yuming Jiang, Kelvin CK Chan, and Ziwei Liu.  Reversion: Diffusion-based relation inversion from images.  _arXiv preprint arXiv:2303.13495_ , 2023.
  * Li et al. [2022] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation.  In _ICML_ , 2022.
  * Li et al. [2023] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee.  Gligen: Open-set grounded text-to-image generation.  In _CVPR_ , pages 22511–22521, 2023.
  * Liu et al. [2022] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum.  Compositional visual generation with composable diffusion models.  In _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVII_ , pages 423–439. Springer, 2022.
  * Meng et al. [2021] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.  Sdedit: Guided image synthesis and editing with stochastic differential equations.  In _International Conference on Learning Representations_ , 2021.
  * Mokady et al. [2022] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.  Null-text inversion for editing real images using guided diffusion models.  _arXiv preprint arXiv:2211.09794_ , 2022.
  * Mou et al. [2023] Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie.  T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.  _arXiv preprint arXiv:2302.08453_ , 2023.
  * Nichol et al. [2021] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen.  Glide: Towards photorealistic image generation and editing with text-guided diffusion models.  _arXiv preprint arXiv:2112.10741_ , 2021.
  * Parmar et al. [2023] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu.  Zero-shot image-to-image translation.  _arXiv preprint arXiv:2302.03027_ , 2023.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _CVPR_ , 2022.
  * Ruiz et al. [2022] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.  Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.  _arXiv preprint arXiv:2208.12242_ , 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _arXiv preprint arXiv:2205.11487_ , 2022.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In _International Conference on Machine Learning_. PMLR, 2015.
  * Song et al. [2020a] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ , 2020a.
  * Song and Ermon [2019] Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Song et al. [2020b] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  _arXiv preprint arXiv:2011.13456_ , 2020b.
  * Tumanyan et al. [2022] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel.  Plug-and-play diffusion features for text-driven image-to-image translation.  _arXiv preprint arXiv:2211.12572_ , 2022.
  * Zeng et al. [2022] Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, John Collomosse, Jason Kuen, and Vishal M Patel.  Scenecomposer: Any-level semantic image synthesis.  _arXiv preprint arXiv:2211.11742_ , 2022.
  * Zhang and Agrawala [2023] Lvmin Zhang and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models.  _arXiv preprint arXiv:2302.05543_ , 2023.
  * Zhao et al. [2022] Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu.  Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations.  _arXiv preprint arXiv:2207.06635_ , 2022.

Supplementary Material

##  6 Implementation Details

All experiments involving one single image condition were conducted on a
NVIDIA GeForce RTX 3090 GPUs. For experiments involving more than two image
conditions, we recommend using a GPU with more RAM due to the increased
computational requirements.

#### Dataset.

The datasets used in our experiments are grouped into four categories:

  * •

Animals: “bear”, “bird”, “cat”, “chicken”, “crab”, “dog”, “elephant”, “fish”,
“fox”, “frog”, “tiger”, “turtle”.

  * •

Objects: “airplane”, “apple”, “banana”, “bottle”, “car”, “clock”, “cup”,
“hamburger”, “shoes”, “spoon”, “television”, “umbrella”.

  * •

Characters: “girl”, “chef”, “ironman”, “boy”, “soldier”.

  * •

Poses: We randonly select five poses of image from the internet.

All source images please see the attach file.

#### Models.

We adopted the Stable Diffusion v1.5
model111https://huggingface.co/runwayml/stable-diffusion-v1-5, operating over
a 50-step sampling process, and the ControlNet 1.0
model222https://huggingface.co/lllyasviel/ControlNet, both accessible on the
Hugging Face’s model hub. Condition extraction leverages the ControlNet
auxiliary library333https://github.com/patrickvonplaten/controlnet_aux. For
depth and normal considerations, we employed the corresponding MiDaS detection
model, whereas OpenPose detector was implemented for pose detection, retaining
only the primary body information and disregarding hand data. We maintained a
standard set of parameters for Canny edge detection with a low threshold of
100 and a high threshold of 250. Our code primarily builds upon the diffusers
codebase444https://github.com/huggingface/diffusers. Our complete code is also
attached for your reference.

##  7 More analysis of “Realign”

In the Stable Diffusion model, text guidance is facilitated via a cross-
attention mechanism. A denoising UNet, incorporating self-attention and cross-
attention layers at multiple resolutions (64, 32, 16, and 8), forms the
backbone of this system. However, it’s essential to note that the requirement
of storing attention values for replacement operations poses a significant
demand on memory. As such, for the results depicted in the primary manuscript,
we’ve restricted the replacement of attention maps to resolutions of 32 and
lower. The denoising UNet’s structure is bifurcated, with the T2I adapter and
ControlNet overseeing computations in the initial and final halves,
respectively. To effectively modulate the attention of the unified score, we
strategically replace attention maps for the unified text condition in the
UNet, but only after it has been manipulated by the image condition.

As suggested by Eq. (5) in the main text, it might be plausible to increase
the image CFG to mitigate the generation of undesirable objects. However,
modifying the image CFG could inadvertently affect the representation of
another object in the text condition. Moreover, we explored the ’Realign’
operation under various CFG scenarios, which exhibited certain advantages, as
depicted in Fig. 10.

Figure 10: Impact of the “Realign” operation under different CFG value
scenarios. In the first row, we observe that increasing the Image CFG
occasionally helps in removing undesired objects. However, as the Text CFG
value increases in the second row, the semantics of the basketball begins to
dominate over the chair. Interestingly, increasing the Image CFG at this
juncture has a marginal effect. Nevertheless, the “Realign” operation proves
to be beneficial in such situations, as it allows the regeneration of objects
initially suppressed by more dominant semantics. Figure 11: Influence of
“Realign” Operation Steps in Diffusion Process. Here, the total diffusion
process comprises 50 steps, with “0-5” indicating that “Realign” is
implemented only in the initial 5 steps.

We further explore the effect of the “Realign” operation in the diffusion
process, specifically focusing on the number of execution steps. This
investigation is illustrated in Fig. 11. Our findings suggest that the
“Realign” operation is indeed crucial. Interestingly, even if it is executed
for a relatively short duration in the early stages, it is effective in
removing undesirable objects.

Finally, we conducted an experiment where we used the “Realign” operation
exclusively. We discovered that this can impact the control effectiveness of
the image condition. This analysis is visually represented in Fig. 12 These
observations highlight that, while “Realign” is a potent tool, its full
potential is realized when combined with the “Decompose” process from our
method. Together, they provide a robust control mechanism, optimizing the
desired output.

Figure 12: Comparison of “Realign only” and Our Method. The “Realign only”
method employs attention solely for control under given image conditions,
omitting the integration of individual and unified scores. This approach could
affect the control effectiveness of the image condition. Figure 13:
Disentanglement Analysis in Stable Diffusion. Certain object combinations can
lead to disentanglement phenomena. This is primarily due to the prevalence of
these specific object combinations appearing together in the training data.

##  8 Limitation

In our experiments, we identified several limitations, as highlighted in Fig.
13. We attribute these limitations to data bias, which is reflective of
inherent biases present in real-world scenarios. For instance, certain object
combinations result in specific outcomes due to prevalent real-world
associations. When a spoon and chicken are combined, the generated image often
portrays cooked or roasted chicken. Similarly, when a cup is combined with
certain objects, a logo or watermark resembling a trademark often appears in
the bottom left corner. Additionally, some object combinations are rarely
found in everyday scenarios and are more common in animations or books. For
example, when an airplane is combined with certain animals, the animals often
appear in a cartoonish form. These biases may also stem from the entanglement
of specific items with certain scenes during data acquisition. For instance,
chefs and soldiers tend to significantly modify the scene when they appear in
an image, making it challenging to depict these two characters together in a
single scene. These findings point to the limitations of our method,
reflecting the challenges of addressing real-world bias and complex data
entanglements.

##  9 Broader Impacts

Controllable image generation carries transformative potential and
implications across various fields:

  * •

Creative Industries: Bolsters creativity by aiding rapid idea visualization
and artwork production.

  * •

Education: Enhances interactive and illustrative educational content,
especially in visually-dependent subjects.

  * •

Scientific Visualization: Facilitates understanding and communication by
visualizing intricate scientific phenomena.

  * •

Entertainment: Enables immersive experiences by generating detailed,
controllable environments and characters in games and virtual reality.

Despite its promise, potential downsides must be considered:

  * •

Misinformation: Risk of misuse to create convincing fake images or deepfakes.

  * •

Job Displacement: Possible job displacement in fields like graphic design,
offset by potential new job creation.

  * •

Privacy Concerns: Potential misuse to generate images of individuals without
their consent.

As such, responsible development and usage of this technology, backed by
necessary regulatory measures, are imperative.

##  10 More Results

Additional visual results are showcased in Figures 14 through 21. In terms of
pose, apart from the combinations of characters discussed in the main text,
we’ve also randomly selected some objects from a pool of 24 to create
combinations of characters and objects.

Figure 14: Additional Visual Results of Depth. Figure 15: Additional Visual
Results of Depth. Figure 16: Additional Visual Results of Normal. Figure 17:
Additional Visual Results of Normal. Figure 18: Additional Visual Results of
Canny. Figure 19: Additional Visual Results of Canny. Figure 20: Additional
Visual Results of Pose. Figure 21: Additional Visual Results of Pose.