# AI-generated faces free from racial and gender stereotypes

Nouar AlDahoul  New York University Abu Dhabi, UAE.  Talal Rahwan  New York
University Abu Dhabi, UAE.  Yasir Zaki

###### Abstract

Text-to-image generative AI models such as Stable Diffusion are used daily by
millions worldwide. However, many have raised concerns regarding how these
models amplify racial and gender stereotypes. To study this phenomenon, we
develop a classifier to predict the race, gender, and age group of any given
face image, and show that it achieves state-of-the-art performance. Using this
classifier, we quantify biases in Stable Diffusion across six races, two
genders, five age groups, 32 professions, and eight attributes. We then
propose novel debiasing solutions that outperform state-of-the-art
alternatives. Additionally, we examine the degree to which Stable Diffusion
depicts individuals of the same race as being similar to one another. This
analysis reveals a high degree of stereotyping, e.g., depicting most middle
eastern males as being dark-skinned, bearded, and wearing a traditional
headdress. We address these limitations by proposing yet another novel
solution that increases facial diversity across genders and racial groups. Our
solutions are open-sourced and made publicly available.

## Introduction

Artificial intelligence (AI) biases refer to the systematic and unfair
preferences or prejudices embedded in AI systems, often reflecting the biases
present in the data used to train these systems [1]. Such biases can
perpetuate and even exacerbate societal inequalities, as the AI algorithms may
inadvertently discriminate against certain groups. One glaring example of AI
bias is the COMPAS (Correctional Offender Management Profiling for Alternative
Sanctions) algorithm, used by judges and parole officers in the U.S. to assess
the likelihood of recidivism. This algorithm, which has been used on over a
million offenders to date, has been shown to produce racially biased
predictions. In particular, among defendants who did not recidivate, Black
ones were incorrectly predicted to do so at a rate twice as high as their
White counterparts [2, 3]. Another notable example of AI bias is found in face
recognition algorithms, which have been shown to discriminate based on race
and gender. Specifically, an examination of commercial face classifiers
revealed that they had significantly higher error rates given darker-skinned
females compared to their lighter-skinned male counterparts [4]. Amazon’s
endeavor to use an algorithm to evaluate job candidates based on their resumes
is yet another example of AI bias. In this case, the bias stemmed from the
fact that the algorithm was trained on historical data, consisting mostly of
male applicants’ resumes. Consequently, the algorithm reinforced gender
preconceptions by penalizing resumes that included phrases associated with
women, leveraging subtle linguistic cues that were indicative of the
candidate’s gender [5]. In the context of natural language processing, notable
biases have been reported in popular word embedding models such as BERT and
GPT-2. In particular, such models often associate certain occupations or
stereotypes more strongly with one gender or racial group than another,
thereby reinforcing societal prejudices [6].

In this study, we focus on racial and gender stereotypes in Stable Diffusion
[7], one of the most popular text-to-image generative models, used daily by
millions worldwide [8]. Multiple, very recent studies have demonstrated that
Stable Diffusion underrepresents certain races or genders [9, 10, 11, 12, 13];
see the Discussion section for a technical summary of these studies. Some of
them quantified certain types of biases but did not propose any debiasing
solutions. One such study focused on ten professions and three races—Asian,
Black, and White [10]. However, they did not consider certain racial groups,
such as Indian, Latinx, and Middle Eastern. Another study focused on biases
pertaining to three genders and 27 nationalities [12]. However, they did not
consider professions (e.g., Doctor, Nurse, etc.) nor attributes (e.g.,
Beautiful, Intelligent, etc.) in their analysis. Moreover, in the context of
race, the authors manually classified skin-color rather than proposing a
classifier, and only provided qualitative rather than quantitative
conclusions. Wang et al. [11] studied the association of pleasant and
unpleasant attributes with certain concepts such as: European American and
African American names, Light Skin and Dark Skin, Straight and Gay. In
addition, they also studied gender stereotypes across eight professions.
However, they did not consider racial groups apart from European American
(White) and African American (Black).

In principle, biases in Stable Diffusion can be addressed by curating the
dataset on which the model is trained. However, this approach not only
requires a massive dataset (possibly consisting of billions of samples) but
also requires retraining Stable Diffusion from scratch, which is
computationally very expensive. One alternative that has been proposed in the
literature is to follow a “user in control” approach, allowing the user to
steer Stable Diffusion towards fairer outcomes [13]. Following this approach,
the authors examined professional stereotypes across genders, but did not
quantify stereotypes across racial backgrounds. Another study focused on
mitigating unsafe content generation (disturbing, hateful, sexually explicit,
violent, and political) [14]. The authors addressed this issue by relying on a
large language model, namely ChatGPT [15], to automatically rephrase the
prompts before feeding them to Stable Diffusion. Nevertheless, such a “hard
prompt” approach has not been examined in the context of racial and gender
biases. Furthermore, hard prompt searching often yields sub-optimal outcomes
due to linguistic ambiguity [9] in addition to being opaque and laborious
[16]. Instead of using hard prompts, a more practical alternative was recently
proposed [9], relying on Stable Diffusion’s pretrained vision-language model
(namely CLIP [17]) to obtain the embeddings of the reference images and
learnable prompts. The authors show that, by aligning the directions of the
prompt and image features in the joint embedding space, the model becomes able
to represent different categories (e.g., male and female) equally. However, as
we will show later on in our analysis, their solution is unable to generate
images that adequately represent certain prompts.

Against this background, we develop a classifier to predict the race, gender,
and age group of any given face image, and show that it achieves state-of-the-
art performance on all three fronts. Using this classifier, we quantify biases
in Stable Diffusion-generated images, focusing on six races, two genders, five
age groups, 32 professions, and eight attributes. To address these biases, we
propose a novel solution, and demonstrate its ability to outperform state-of-
the-art alternatives across various benchmarks. Finally, we examine the degree
to which Stable Diffusion depicts individuals of the same race as being
similar to one another. This analysis revealed, for the first time in the
literature, a high degree of stereotyping, e.g., depicting most middle eastern
males as being dark-skinned, bearded, and wearing a traditional headdress. We
address this issue by proposing yet another novel solution, which successfully
increases facial diversity across genders and racial groups.

## Results

### Evaluating our classifiers

To examine the biases in Stable Diffusion, we start off by developing a novel
classifier that predicts the race, gender, and age group of any given face
image. The classifier is composed of three stages: face detection, face
embedding generation, and the classification stage; see Material and Methods
for technical details. To train and validate our classifier, we utilize
FairFace [18]—one of the largest publicly-available datasets of face images.
FairFace specifies the race (Black, East Asian, Indian, Latinx, Middle
Eastern, Southeast Asian, and White), the gender (Female, Male), and the age
group (0-2, 3-9, 10-19, 20-29, 30-39, 40-49, 50-59, 60-69, $\geq$70) of each
image. We simplified FairFace’s categorization by combining the East Asian and
Southeast Asian categories into a single one (Asian), and merging their age
groups into five simpler categories; see the Materials and Methods section for
more details.

We trained our classifier on FairFace’s training set, and evaluated it using
FairFace’s validation set. Additionally, we benchmarked our classifier against
several alternatives from the literature, namely: CLIP’s zero-shot [17],
Google’s FaceNet [19] \+ SVM [20], FairFace’s ResNet-34 [18], EfficientNet-B7
(tuning all layers) [21], Large Vision Transformer VIT (tuning all layers)
[22], and Amazon Rekognition AgeRange [23]. Tables 1, 2, and 3 summarize the
results for race, gender, and age, respectively. As can be seen, across all
three categories, our classifier consistently achieves state-of-the-art
performance in terms of accuracy, precision, recall, and F1 score.

Figure 1a depicts the confusion matrices corresponding to race and gender,
along with sample images representing each cell. As can be seen, Asian is the
easiest race to predict, followed by Black. In contrast, the hardest races to
predict are Latinx and Middle Eastern. On the other hand, there is no
difference in performance when it comes to predicting male vs. female images.

We reproduced the same confusion matrices, but using images generated by
Stable Diffusion XL (SDXL) which has been shown to outperform previous
versions of Stable Diffusion [7]. More specifically, we generated 10,000
images per race using the following prompt: “a photo of $X$ person, looking at
the camera, closeup headshot facing forward, ultra quality, sharp focus”,
where $X\in$ {a Black, a White, an Asian, an Indian, a Latinx or Hispanic, a
Middle Eastern}. As for gender, we generated 10,000 images per gender using
the same prompt as before, but with $X\in$ {male, female}. The results of this
evaluation are depicted in the first two rows of Table 4, showing that our
classifier works better with images from SDXL than those from FairFace,
regardless of the performance measure. This indicates that our classifier is
particularly suited to examine the racial-, and gender-composition of images
generated by SDXL.

The confusion matrices depicted in Figure 1b reveal that all races are
significantly easier to predict in the SDXL dataset compared to its FairFace
counterpart, apart from Latinx which is significantly harder (see how the
numbers along the diagonal are greater in Figure 1b than in Figure 1a, except
for Latinx). One possible explanation could be that the dataset used to train
SDXL contains several images that are labelled as Latinx but are highly
similar to Indian and Middle Eastern individuals. In terms of gender, our
classifier achieves a perfect score for both male and female.

Table 1: A comparison between our race classifier and other alternatives from the literature, using FairFace’s validation set and FairFace’s seven-race classification. Bold font highlights the highest score(s). | Accuracy | Precision | Recall | F1 score
---|---|---|---|---
Our classifier | 73% | 72% | 72% | 72%
CLIP’s zero-shot classifier | 64% | 67% | 65% | 65%
Google’s FaceNet + SVM | 69% | 69% | 68% | 68%
FairFace’s (ResNet34) classifier | 72% | 72% | 71% | 72%
EfficientNet-B7 (tuning all layers) | 70% | 70% | 70% | 70%
Large Vision Transformer VIT (tuning all layers) | 71% | 72% | 70% | 71%
Table 2: A comparison between our gender classifier and other alternatives from the literature, using FairFace’s validation set. Bold font highlights the highest score(s). | Accuracy | Precision | Recall | F1 score
---|---|---|---|---
Our classifier | 94% | 94% | 94% | 94%
FairFace’s (ResNet34) classifier | 94% | 94% | 94% | 94%
CLIP’s zero-shot classifier | 94% | 94% | 94% | 94%
Table 3: A comparison between our age classifier and other alternatives from the literature, using FairFace’s validation set. Bold font highlights the highest score. | Accuracy | Precision | Recall | F1 score
---|---|---|---|---
Our classifier | 79% | 75% | 68% | 71%
FairFace’s (ResNet34) classifier | 67% | 65% | 62% | 63%
Amazon Rekognition AgeRange | 71.7% | 68% | 61% | 63%
CLIP’s zero-shot classifier | 38% | 43% | 46% | 36%
Table 4: Evaluating our classifier’s ability to predict race, gender, and age using SDXL images and our six racial groups. Bold font highlights the highest score. | Accuracy | Percision | Recall | F1 score
---|---|---|---|---
FairFace | SDXL | FairFace | SDXL | FairFace | SDXL | FairFace | SDXL
Racial | 78% | 88% | 76% | 90% | 75% | 88% | 76% | 87%
Gender | 94% | 100% | 94% | 100% | 94% | 100% | 94% | 100%
Age | 79% | 97% | 75% | 97% | 68% | 97% | 71% | 97%

a

b

c

Figure 1:  Evaluating our classifier. a, Confusion matrices of race and gender
prediction based on the FairFace validation set, along with sample images
representing different cells. b, Similar to (a), but based on images generated
by Stable Diffusion XL (SDXL). c, Confusion matrices of age prediction based
on the FairFace validation set (left) and based on images generated by SDXL
(right).

Finally, we evaluated our classifier’s ability to predict the age group of a
given image. Again, we used the FairFace dataset as well as the SDXL dataset
mentioned earlier; the prompt used to generate the latter dataset was: “a
photo of $X$, looking at the camera, closeup headshot facing forward, ultra
quality, sharp focus.”, where $X\in$ {a child, a teenager, a young adult
person, a middle-aged person, an elderly person}. The results of this
evaluation are summarized in the third row of Table 4, showing that our
classifier is highly accurate on both the FairFace dataset as well as the SDXL
dataset. Figure 1c depicts the confusion matrices corresponding to FairFace
(left) and SDXL (right). Starting with the FairFace-based matrix, it reveals
that the easiest age groups to predict are 20-39 and 0-9, while, the hardest
one to predict by far is 10-19. All age groups become significantly easier to
predict when replacing the FairFace dataset with the SDXL dataset, with the
most notable difference being the 10-19 age group.

### Examining underrepresentation in Stable Diffusion

Armed with our state-of-the-art classifier, we are now ready to examine the
degree to which different races, genders, and age groups are represented in
SDXL-generated images. To this end, we generated 10,000 images using the
following racial- and gender-neutral prompt: “a photo of a person”. The
distribution of the resulting images is summarized by the dashed bars in
Figure 2. As can be seen, White is the most represented race (47% of images),
followed by Black (33%) The remaining races are all extremely
underrepresented, e.g., only 3% are Asian, and only 5% are Indian. As for
gender, males are overrepresented, constituting 65% of all images. These
findings mirror what has been reported by Ghosh and Caliskan [12], showing
that most images generated by Stable Diffusion (v2.1) representing the prompt
“a front-facing photo of a person” depict light-skinned Western men.

One possible explanation behind these biases could be that SDXL is merely
reflecting the biases already present in the dataset on which it was trained,
namely LAION-5B [24]. To examine this possibility, we used a subset of
LAION-5B [25] consisting of 88,714 images; see Material and Methods for more
details. The distribution of those images is summarized by the plain bars in
Figure 2. As can be seen, White is even more represented in LAION-5B than SDXL
(63% vs. 47%), while all remaining races are underrepresented. As for gender,
both male and female individuals are equally represented with a 50-50 split.
This indicates that SDXL contains its own biases beyond just the data it was
trained on, e.g., raising the percentage of Black individuals from 11% to 33%,
and raising the percentage of males from 50% to 65%.

One way to address the aforementioned biases is to develop a model that
generates images in which all races are equally represented. To this end, we
introduce a fine-tuned version of SDXL, which we call: “SDXL-Inc” (where Inc
is a shorthand for Inclusion). This model was created as follows. First, we
identified 32 professions, and split them into 21 that are used for fine-
tuning, and 11 that are used for testing (these professions are listed in
Supplementary Table 1). Then, for every combination $(X,Y)$ such that $X\in$
{Black, White, Asian, Indian, Latinx or Hispanic, Middle Eastern} and $Y\in$
{male, female}, we generated a separate dataset consisting of images depicting
the 21 professions. The generation of these 12 datasets (6 races × 2 genders)
was done using the following prompt: “a photo of $X$ $Y$ $Z$ looking at the
camera, closeup headshot facing forward, ultra quality, sharp focus”, where
$Z$ denotes one of the 21 professions. After that, we fine-tuned SDXL with
each dataset, yielding 12 different sets of weights. The fine-tuning was done
using LORA (Low Rank Adaptation) [26], which reduces the number of trainable
parameters for the downstream task; see Materials and Methods for more
details. The final model (SDXL-Inc) randomly selects one of the 12 sets of
weights in response to any given prompt.

To evaluate SDXL-Inc, we used it to generate 10,000 images, relying on the
same prompt used earlier, i.e., “a photo of a person”. The distribution of the
resulting images is summarized by the starred bars in Figure 2. As shown in
this figure, all races are almost equally represented, and the differences
between them are markedly smaller than the differences present in both
LAION-5B and SDXL. As for gender, SDXL-Inc is able to represent males and
females equally, unlike SDXL.

Figure 2:  Examining biases in LAION-5B, Stable Diffusion XL (SDXL), and our
SDXL-Inc. Comparing the representation of different races and genders in a
random sample of 88,714 images from the LAION-5B dataset, and in a sample of
10,000 images generated by Stable Diffusion XL using the prompt: “a photo of a
person”. The same prompt is then used to generate 10,000 images by our SDXL-
Inc.

### Examining professional stereotypes in Stable Diffusion

Let us now shift our attention to professional stereotypes in SDXL. To this
end, we used SDXL to generate 320,000 images depicting 32 different
professions (10,000 per profession). More specifically, for each profession,
$Z$, we used the following prompt: “a photo of a $Z$, looking at the camera,
closeup headshot facing forward, ultra quality, sharp focus”. Moreover, to
specify the types of images that we want to avoid, we used the following
negative prompt: “cartoon, anime, 3d, painting, b&w, low quality”. For each
profession, we used our classifier to examine the racial-, gender-, and age-
composition of the resulting images.

a

b

Figure 3:  Professional stereotypes in Stable Diffusion XL (SDXL). Given 25
professions, SDXL was used to generate 10,000 images per profession. a, Racial
distribution per profession. b, Gender and age distribution per profession.

a

b

c

Figure 4:  Results of SDXL-Inc. Given eight attributes and eight professions,
both SDXL and SDXL-Inc were used to generate 10,000 images per profession and
per attribute. a, Race distribution per attribute, with the upper row
corresponding to SDXL, and the lower row corresponding to SDXL-Inc. b, The
same as (a) but for professions instead of attributes. c, Gender distribution
per profession and per attribute for SDXL and SDXL-Inc. The standard
deviation(s) corresponding to each subplot is denoted by $\sigma$ followed by
a subscript indicating the model.

We started off by examining racial stereotypes in each profession. Figure 3a
depicts the results for the 25 professions, while Supplementary Figure 2
depicts the remaining seven professions. Numeric values below 15% are omitted
to improve the visualization; see Supplementary Table 2 for all values. As can
be seen, in 18 out of the 25 professions, the percentage of White faces
surpasses that of all other races combined, even exceeding 80% in some cases.
White also happens to be the most commonly represented race in 21 out of the
25 professions. As for the remaining four, two of them are typically low-
income jobs, namely Cleaner and Security Guard, and both are mostly
represented by images depicting Black individuals. Similarly, when considering
the other seven professions (Supplementary Figure 2), we find that TV
presenter is mostly White, while Janitor and Garbage Collector are mostly
Black and Middle Eastern, respectively. These findings confirm the findings of
Bianchi et al. [10], showing that prestigious, high-paying professions are
often represented as White.

Having examined the racial distribution, we now shift our attention to the
gender distribution. The results are depicted in the upper row of Figure 3b;
and the exact percentages of males and females are listed in Supplementary
Table 2. As can be seen, males represent 90% of the images in 16 different
professions, including Doctor, Manager, and Professor. Moreover, the
percentage of males exceeds that of females in 20 out of the 25 professions.
As for the remaining five, they include Nurse and Secretary—two common
stereotypes of women. Similar findings were reported in the literature,
showing that AI-generated images tend to associate certain jobs with men and
certain other jobs with women [13, 11].

Finally, we consider the age distribution of SDXL-generated images. Results
are depicted in the bottom row of Figure 3b, where values smaller than 1% are
omitted to improve the visualization; see Supplementary Table 3 for all
values. As can be seen, 20-39 is the most represented age group in all
professions apart from Secretary and Pharmacist which are mostly represented
by people aged 40-59. This age group (40-59) also happens to be the second
most represented in 17 other professions, although its representation is an
order of magnitude smaller than that of 20-39 in many cases. As for the
remaining age groups, they are represented in less than 1% of the images, with
the exception of Secretary (1.9% older than 60), Manager (1.2% older than 60),
and Programmer (1.1% aged between 10y and 19y). This finding mirrors what has
been reported by Zhang et al. [9], showing that people whose age is $<$10 and
$>$50 are extremely underrepresented by Stable Diffusion v1-4.

### Debiasing Stable Diffusion across professions and attributes

Having examined stereotypes in terms of profession, we now turn to stereotypes
in terms of the following attributes: Winner, Beautiful, Intelligent, Parent,
Sibling, Terrorist, Poor, and Criminal. The results of this analysis are
depicted in the upper row of Figure 4a, and the corresponding numerical values
are listed in Supplementary Table 5. We find that White is extremely
overrepresented in the three attributes that tend to be associated with
success and attractiveness, i.e., Winner, Beautiful, and Intelligent. White is
also extremely overrepresented in the two family-related attributes, i.e.,
Parent and Sibling. In contrast, when it comes to Terrorism, Middle Eastern
are the most represented race, and none of the images depict a White
individual, reinforcing existing stereotypes connecting Middle Easterners to
terrorism [27]. Similarly, when it comes to crime and poverty, the majority of
images depict Black individuals, reinforcing common stereotypes in this regard
[28]. Similar findings were reported by Bianchi et al. [10] when analyzing
Stable Diffusion (v1-4), showing that attractiveness is associated with White
people, thugs are associated with Black people, and terrorists are associated
with Brown men with beards.

Having established that SDXL exhibits biases in terms of attributes, we now
evaluate SDXL-Inc’s ability to overcome these biases. To this end, we repeated
the same procedure used earlier with SDXL, but using our SDXL-Inc model
instead. The results for professions are depicted in the bottom row of Figure
4a. As can be seen, the underrepresentation of different races is markedly
reduced compared to SDXL, as evidenced by the substantial reduction in
standard deviation ($\sigma$). Importantly, none of the eight attributes
considered in this analysis were used during the fine-tuning phase, and yet
SDXL-Inc was able to significantly reduce the racial biases related to them.
This indicates that our solution can indeed be generalized beyond the features
it was fine-tuned for.

Let us now reexamine professional stereotypes to further evaluate SDXL-Inc’s
debiasing capability. To this end, we selected four of the professions that
were used during the fine-tuning phase (Dietitian, Manager, Pharmacist, and
Pilot), as well as four professions that were not used during that phase
(Accountant, Journalist, Musician, and Firefighter) to further assess the
generalizability of SDXL-Inc. For each profession, Figure 4b shows the racial
distribution of images using SDXL (upper row) and SDXL-Inc (lower row). As can
be seen, regardless of whether the profession is Black-dominated (Musician and
Firefighter) or White-dominated, our solution is able to significantly reduce
racial biases. Again, the reduction in standard deviation is substantial,
reaching multiple folds across all professions. Similar trends are observed
when examining the remaining 24 professions considered in our analysis; see
Supplementary Figure 3a.

Finally, we evaluate SDXL-Inc’s ability to address gender biases across the
aforementioned attributes and professions. As can be seen in Figure 4c, SDXL
(solid bars) exhibits substantial biases, e.g., depicting the vast majority of
beautiful individuals as female while depicting the vast majority of
intelligent individuals as male. In contrast, our solution consistently
produces an equal, or near-equal, split between female and male (dashed bars),
regardless of the attribute or profession. These performance improvements are
reflected by the vast reduction in standard deviation, from 40.3 to just 2.7.
In fact, for the remaining 24 professions, the standard deviation
corresponding to our solution drops even further, reaching 1.0; see
Supplementary Figure 3b.

### Benchmarking our debiasing solution

Next, we compare our solution (SDXL-Inc) to another recently proposed
alternative, namely ITI-GEN [9]; see the Discussion section for a brief
description of this model, and see the Materials and Methods for details on
how we retrained it. In their official GitHub repository [29], the authors
imply that their solution may struggle with certain generalizations. More
specifically, they state that the training prompt (e.g., a headshot of a
person) and the inference prompt (e.g., a headshot of a doctor) “should not
differ a lot”. To compare the degree to which both models (ITI-GEN and SDXL-
Inc) can be generalized, we used each of them to generate 1,200 images for
each of the following five prompts: 1) a headshot of a person with the Eiffel
Tower in the background; 2) a headshot of a mechanic with a guitar; 3) a
headshot of a skillful trainer with a pet tiger; 4) a headshot of a happy and
healthy family; and 5) a headshot of a person with green hair and eyeglasses.

Figure 5 depicts the race and gender distribution of SDXL, SDXL-Inc, and ITI-
GEN for each of the five prompts. As can be seen in the left column, SDXL is
extremely biased, depicting the vast majority of images as White in all
prompts. As for ITI-GEN (middle column, dashed bars), it manages to reduce a
substantial amount of the bias compared to SDXL. Nevertheless, the percentage
of images depicting White individuals remains $\geq$40% in four out of the
five prompts. Our SDXL-Inc (middle column, starred bars) yields a more even
distribution of images across races compared to ITI-GEN; see how the standard
deviations for our solution are smaller than those of ITI-GEN in all five
prompts. As for gender (right column), we can see that SDXL is the most
biased, ITI-GEN reduces a considerable amount of bias, and SDXL-Inc further
reduces bias compared to ITI-GEN across all prompts; see the corresponding
standard deviations.

One reason behind these observed differences in performance could be the fact
that the models occasionally intend to generate images of a certain race, but
accidentally end up generating images of a different race (note that both
SDXL-Inc and ITI-GEN contain steps designed to generate an image of certain
races). If these steps work perfectly well, we would expect the models to
achieve a perfect score in terms of accuracy, recall, precision, and F1 score.
To test this possibility, we used out classifier to label the 1,200 images
generated by each solution (200 per race) for each of the five prompts. The
result of this analysis is summarized in Table 5. As can be seen, SDXL-Inc
outperforms ITI-GEN across all metrics in all five prompts. This suggests that
the observed difference in performance between the two solutions could be (at
least partially) explained by the fact that the error rate in generating the
intended race(s) is higher in ITI-GEN than in SDXL-Inc.

Figure 5:  SDXL vs. ITI-GEN vs. SDXL-Inc Comparing the distribution of race
and gender across the three models, given five prompts. Each row corresponds
to a different prompt. The standard deviation(s) corresponding to each subplot
is denoted by $\sigma$ followed by a subscript indicating the model. Gender
results are omitted from the bottom row since the generated images depict
families rather than individuals.  Table 5: Comparing SDXL-Inc to ITI-GEN in
terms of accuracy, recall, precision, and F1 score. For each performance
measure, the highest score is highlighted in bold.

| Accuracy | Recall | Percision | F1 score
---|---|---|---|---
Prompt |  | SDXL-
---
Inc
| ITI-
---
GEN
| SDXL-
---
Inc
| ITI-
---
GEN
| SDXL-
---
Inc
| ITI-
---
GEN
| SDXL-
---
Inc
| ITI-
---
GEN
| a person with Eiffel
---
tower in the background
87 | 70 | 87 | 70 | 89 | 71 | 87 | 67
| a mechanic with
---
a guitar
86 | 39 | 86 | 39 | 88 | 48 | 86 | 37
| a skillful trainer
---
with a pet tiger
77 | 37 | 77 | 37 | 83 | 47 | 77 | 32
| a happy and healthy
---
family
70 | 66 | 70 | 66 | 75 | 68 | 70 | 64
| a person with green
---
hair and eyeglasses
60 | 40 | 60 | 40 | 72 | 55 | 60 | 38
Average | 76 | 50.4 | 76 | 50.4 | 81.4 | 57.8 | 76 | 47.6

a

b

c

Figure 6:  Results of SDXL-Div. a, For each ethnicity, the distribution of
cosine similarity of images generated using SDXL (plain) and SDXL-Div
(dashed). b and c, Sample images of Middle Eastern individuals generated using
SDXL and SDXL-Div, respectively.

### Introducing diverse facial features

Finally, we aimed to develop a version of SDXL capable of generating images
with greater facial diversity per race compared to the original version of
SDXL. To this end, we downloaded the images hosted on the Flickr-Faces-HQ
dataset, which hosts high-resolution (1024×1024) images of human faces with
“considerable variation in terms of age, race, and image background” [30].
Since the dataset is unlabeled, we used our classifier to infer the race of
each image; see Supplementary Table 4 for the number of images per race. The
resulting labelled dataset was then used to fine-tune SDXL. Again, the fine-
tuning was done using LORA [26] to reduce the number of trainable parameters
for the downstream task. We call the resulting model “SDXL-Div” (where Div is
a shorthand for Diversity); see Materials and Methods for technical details.

To evaluate SDXL-Div’s ability to increase the facial diversity of SDXL, each
of the two models (SDXL and SDXL-Div) was used to generate $\sim$10,000 images
per race. Then, our racial classifier was used to obtain an embedding for each
image. Finally, we computed the cosine similarity between every pair of images
that have the same race and are generated by the same model, resulting in
$\sim$50 million cosine similarity values per race per model. The results of
this analysis are depicted in Figure 6a. As can be seen, SDXL-Div is indeed
able to increase the facial diversity of SDXL, regardless of the race. The
greatest difference is observed in the case of Middle Eastern individuals
(dropping the mean cosine similarity from 0.61 to 0.41) and Latinx individuals
(from 0.55 to 0.39). To help the reader appreciate the difference in diversity
between the two models, Figures 6b and 6c provide sample images of Middle
Eastern individuals generated using SDXL and SDXL-Div, respectively. As can be
seen, images generated using the latter model have markedly greater facial
diversity compared to those generated using the former model.

## Discussion

We set out to examine the stereotypes and biases in Stable Diffusion XL
(SDXL), a text-to-image generator used daily by millions worldwide [8]. To the
end, we developed a classifier to predict the race, gender, and age group of
any given face image, and demonstrated that it achieves state-of-the-art
performance. Using this classifier, we showed that the vast majority of faces
generated by SDXL are White males (as previously shown by Ghosh and Caliskan
[12]), while certain racial groups are extremely underrepresented, e.g., only
3% are Asian and only 5% are Indian. Biases were also found when considering
various attributes, e.g., associating beauty with femininity and intelligence
with masculinity. Some of these biases are less severe in the dataset on which
Stable Diffusion was trained, suggesting that certain biases are further
exacerbated by the model itself.

Biased text-to-image models may contribute to the normalization of gender
stereotypes, potentially shaping societal attitudes towards the roles and
capabilities of women in various professions. For instance, as we have
demonstrated, Stable Diffusion mostly depicts secretaries and nurses as women
while depicting managers, doctors, and professors as men; similar gender
stereotypes were reported in the literature [13, 11]. Bearing in mind that
millions of people worldwide are already using such models daily, the
consequences of gender stereotyping can be profound, as they reinforce norms
that hinder progress towards gender equality.

Biased representations of gender and race may contribute to the creation of
content that not only misrepresents certain groups but may also perpetuate
discriminatory practices. This could be detrimental in the context of
advertising, marketing, and media campaigns, where visuals hold substantial
influence. In such cases, biased content may perpetuate economic disparities,
reinforcing existing social inequalities and further marginalizing already
underrepresented groups. For example, as our analysis has shown, Stable
Diffusion associates low-income jobs such as Cleaner, Janitor, and Security
Guard with Black people, while associating higher-prestige jobs such as
Doctor, Lawyer, and Professor with White people. These findings reflect what
has been reported by Bianchi et al. [10], showing that several of the most
prestigious, high-paying professions are represented by Stable Diffusion
(v1-4) as White. Another example of bias is the association between crime and
Black people, as well as the assassination between terrorism and Middle
Easterners, which may reinforce existing biases against these racial groups
[27, 28].

Stable Diffusion’s portrayal of people from any given race as resembling one
another may reinforce existing racial stereotypes. For instance, as we have
demonstrated, Stable Diffusion mostly depicts middle eastern males as being
dark-skinned, bearded, and wearing a traditional headdress, and mostly depicts
middle eastern females as being dark-skinned, wearing a headscarf. Such
oversimplified and generalized depictions of a particular racial group can be
culturally insensitive, and may misrepresent the true diversity within that
group. They may also lead to feelings of alienation, low self-esteem, and a
sense of being misunderstood.

We proposed a debiasing solution called SDXL-Inc, which substantially reduces
the race and gender biases exhibited by SDXL. We also compared it to a
recently-proposed alternative, ITI-GEN [9], and showed that our solution is
more capable of handling complex prompts. We also proposed a novel solution
called SDXL-Div to address a limitation previously neglected in the
literature—the fact that generated faces from certain races exhibit a high
degree of similarity to one another. We showed that our solution can indeed
diversify the facial features of people belonging to the same race, thereby
overcoming this important challenge. In addition to our debiasing solution
(SDXL-Inc), we experimented with an alternative debiasing solution that
follows a fundamentally different approach, which is to use a Large Language
Model, namely GPT-4 [15], “in-the-loop”; see Materials and Methods for
technical details. The results of this solution are depicted in Supplementary
Figure 4, and the corresponding numerical values are listed in Supplementary
Table 6. As can be seen, this solution is also capable of drastically reducing
the race and gender biases exhibited by SDXL.

### Related Work

Bianchi et al. [10] used a previous version of Stable Diffusion (v1-4) to
examine biases across ten professions and three races, but did not propose a
debiasing solution. They classified the race and gender of any generated image
as follows. First, for each of the five demographic categories considered
(i.e., Male, Female, White, Black, Asian), they took the images that represent
this category in the Chicago Face Dataset [31], and fed those images to CLIP
(the core representational component of Stable Diffusion) to generate vector
representations. These were then averaged to obtain a single archetypal vector
representation of the category (e.g., a single vector for Black). Any image
can then be classified as $X$ if its vector representation is most closely
aligned (in cosine distance) to the archetypal vector representation of $X$
(e.g., Black) compared to the alternative categories (e.g., Asian or White).
The authors used this classifier to analyze 100 images per occupation, and
found professional stereotypes that reinforce racial and gender disparities.

Ghosh and Caliskan [12] used a previous version of Stable Diffusion (v2.1) to
analyze biases across three genders and 27 nationalities, but did not consider
professions nor proposed debiasing solutions. Using CLIP’s cosine similarity,
they compared images generated using the prompt: ‘a front-facing photo of a
person’ against those generated using the prompt: ‘a front-facing photo of
$X$’ where $X$ is either a gender (i.e., a man, a woman, a nonbinary gender)
or a nationality (e.g., a person from Brazil). The authors then manually
classified the images based on skin tone (light vs. dark) and found that most
generated images were light-skinned.

Friedrich et al. [13] proposed a “user in control” solution, called Fair
Diffusion, which utilizes a textual interface allowing users to instruct
generative image models on fairness. Intuitively, this is made possible by
extending classifier-free guidance [32] with an additional fair guidance term,
which depends on additional fairness instructions provided by the user. This
approach requires no data filtering nor additional training. The authors used
their approach to analyze images generated by a previous version of Stable
Diffusion (1.5), focusing on professional stereotypes across genders, although
they did not quantify stereotypes across races.

Zhang et al. [9] proposed a solution called ITI-GEN (inclusive Text-to-Image
Generation), designed to generate images that are uniformly distributed across
attributes of interest. One such attribute could be, e.g., “gender”, in which
case the images generated by ITI-GEN would be equally split between the
various attribute categories, e.g., “male” and “female”. More specifically,
given an input prompt and some desired attribute(s), the model learns
discriminative token embeddings representing each category of each attribute.
It then injects these learned tokens after the original prompt, thereby
synthesizing a set of prompts, each representing a unique combination of
categories belonging to different attributes. Finally, this set is used to
generate an equal number of images for any category combination. The authors
used their model to analyze a previous version of Stable Diffusion (v1-4),
focusing on 40 physical attributes (each consisting of two categories, one
positive and one negative), along with gender, skin tone, and age.

In the context of racial stereotype, Zhang et al. focused on skin tone rather
than race. Thus, unlike our analysis, theirs does not distinguish between,
say, Asian and White individuals who happen to be equally light-skinned, or
between Indian and Latinx individuals who happen to have similar skin tones.
Moreover, the authors focus on physical attributes (e.g., Black hair,
Mustache, etc.) as opposed to those describing various characteristics like
the ones used in our analysis (e.g., Criminal, Intelligent, Parent, Poor,
etc.). Additionally, their analysis of professional stereotypes focuses on
four professions (Professor, Doctor, Worker, Firefighter) and 200 images per
profession, while we focus on 32 professions and 10,000 images per profession.
Finally, as we have demonstrated in Figure 5, their solution is unable to
debias complex prompts, such as “a person with green hair and eyeglasses”, or
“a person with the Eiffel tower”. This difference in performance could (at
least partially) be explained by the fact that their solution has a higher
error rate when it comes to generating images of a certain race (see Table 5).

## Materials and Methods

### Data Overview

This section describes the datasets used in our study.

I) The LAION-5B dataset: This is the dataset that was originally used to train
Stable Diffusion [24]. We utilized a subset of this dataset, consisting of
high-resolution images [25], to determine whether the biases in Stable
Diffusion XL can be entirely attributed to the data on which it was trained.
We randomly selected 172,923 images from this subset, and kept those having
one or more of the following keywords: face, person, child, woman, or man. We
then cropped the images to retain only the face(s) appearing therein, and
discarded any resulting face images that are smaller than $100\times 100$
pixels. This filtering process left us with a final set of 88,714 face images.

II) The FairFace dataset: This is one of the largest publicly-available
datasets of face images. For each image, the dataset specifies the race
(Black, East Asian, Indian, Latinx, Middle Eastern, Southeast Asian, and
White), the gender (female, male), and the age group (0-2, 3-9, 10-19, 20-29,
30-39, 40-49, 50-59, 60-69, $\geq$70) [18]. Moreover, the dataset is readily
divided into two sets: 86,744 images for training and 10,954 for validation.
We combined East Asian and Southeast Asian into a single category: Asian. As
for age groups, we combined (0-2) and (3-9) into (0-9); kept (10-19)
unchanged; combined (20-29) and (30-39) into (20-39); combined (40-49) and
(50-59) into (40-59); and combined (60-69) and ($\geq$70) into ($\geq$60). The
resulting dataset was then used to train and validate our race, gender, and
age classifiers. For race, the number of images used for validation was: 1556
for Black; 2965 for Asian; 1516 for Indian; 1623 for Latinx or Hispanic; 1209
for Middle Eastern; and 2085 for White. As for gender, the number of images
used for validation was: 5162 for female; and 5792 for male. Finally, for age
groups, the number of images used for validation was: 4855 for (0-9); 1181 for
(10-19), 2330 for (20-39); 2149 for (40-59); and 439 for ($\geq$60).

III) The Flickr-Faces-HQ dataset: This dataset consists of 70,000 high-
resolution ($1024\times 1024$) images of human faces crawled from Flickr with
considerable variation in terms of age, race, and image background [30]. It is
unlabeled, and was originally created as a benchmark for generative
adversarial networks (GAN). We utilized this dataset to fine-tune our SDXL-Div
model in order to generate face images with varying races and facial features.
This was done to overcome the fact that, for certain races, the images
generated by Stable Diffusion XL seem too similar to one another.

IV) The Stable Diffusion validation dataset: This dataset consists of images
that we have generated using Stable Diffusion XL. In particular, the dataset
consists of 10,000 images per race, 10,000 images per gender, and 10,000
images per age group. For race, the prompt used to generate the images was: “a
photo of a $X$”, where $X\in$ {Asian, Black, Indian, Latino or Hispanic,
Middle Eastern, White}. As for gender, the prompt used was: “a photo of a
$X$”, where $X\in$ {female, male}. Finally, for age groups, the prompt used
was: “a photo of a $X$”, where $X\in$ child, teenager, young adult person,
middle age person, elderly person}. This dataset was used to validate our
race, gender, and age classifiers.

V) The SDXL-Inc fine-tuning dataset: This dataset consists of Stable
Diffusion-generated images with varying race, gender, and profession. The
images were generated using the prompt: “a photo of a $X$ $Y$ $Z$, looking at
the camera, closeup headshot facing forward, ultra quality, sharp focus”,
where $X\in$ {Asian, Black, Indian, Latino or Hispanic, Middle Eastern,
White}; $Y\in$ {female, male}; and $Z$ is one of the 21 professions listed in
Supplementary Table 1 under the “fine-tuning” category. For any of the twelve
combinations of race and gender ($X$, $Y$), we compiled the corresponding
images from all 21 professions into a single dataset, which was used to fine-
tune a version of SDXL tailored specifically for race $X$ and gender $Y$. This
process yielded 12 fine-tuned models (one per race-gender combination); these
are the 12 components constituting our SDXL-Inc solution.

VI) The profession dataset: This dataset consists of Stable Diffusion-
generated images depicting 32 professions. More specifically, we generated
10,000 images per profession using the prompt: “a photo of $Z$, looking at the
camera, closeup headshot facing forward, ultra quality, sharp focus”, where
$Z$ is one of the 32 professions listed in Supplementary Table 1. In this
table, the 21 professions listed under the “fine-tuning” category were used to
fine-tune SDXL, while the remaining 11 professions (i.e., those listed under
“generalization testing”) were used to evaluate the generalization capability
of SDXL-Inc.

VII) The attribute dataset: This dataset consists of Stable Diffusion-
generated images depicting eight attributes. In particular, we generated
10,000 images per attribute using the prompt: “a photo of a $X$, looking at
the camera, closeup headshot facing forward, ultra quality, sharp focus”,
where $X\in$ {Poor, Winner, Beautiful, Intelligent, Parent, Sibling,
Terrorist, Criminal}. This dataset was used to evaluate the generalization
capability of SDXL-Inc.

### Methods

This section presents the tools and techniques used in this work, including
Stable Diffusion, face detection, face embedding generation, classification
models, as well as the use of GPT-in-the-loop.

#### Stable Diffusion

Stable Diffusion is a text-to-image generative model used to generate images
based on prompts fed as input. A recent state-of-the-art version, called
Stable Diffusion XL (SDXL), has a second text encoder and 3x larger UNet
backbone, which increases the number of parameters [7]. Additionally, SDXL has
multiple novel conditioning schemes, as well as a refinement model that
improves the visual fidelity of generated images. In our work, we utilized
SDXL due to its significantly improved ability to generate human faces
compared to its predecessors. We used the Hugging Face repository
“stabilityai” and the model “stable-diffusion-xl-base-1.0“ [33] to generate
images to analyze racial and gender stereotypes in the context of various
professions and attributes.

We fine-tuned SDXL using LORA (Low Rank Adaptation) [26] in the process of
creating our debiasing solution. This process, which was carried out using a
V100 GPU, yielded our SDXL-Inc model—a debiasing solution to overcome bias in
race and gender representation. To fine-tune SDXL, we created a dataset
consisting of prompt-image pairs. Each such pair consisted of an image taken
from the “SDXL-Inc fine-tuning dataset” (described above) along with the
prompt: “a photo of $Z$, looking at the camera, closeup headshot facing
forward, ultra quality, sharp focus”, where $Z$ is the profession depicted in
the image.

Similarly, we fine-tuned SDXL using LORA in the process of creating SDXL-
Div—our solution for mitigating the lack of diverse facial features in images
generated by SDXL. To this end, we created a dataset consisting of prompt-
image pairs. Each such pair consisted of an image taken from the “Flickr-
Faces-HQ dataset” (described above) along with the prompt: “a photo of $X$
person, looking at the camera, closeup headshot facing forward, ultra quality,
sharp focus”, where $X$ is the race depicted in the image. This process
yielded the SDXL-Div model.

#### Hyper-parameters

Whenever an image is generated in our analysis, the following hyper-parameters
were used: Image resolution = 1024; number of inference steps = 40; and
guidance scale = 5. On the other hand, when fine-tuning SDXL, the following
hyper-parameters were used: Image resolution = 1024; training batch size = 1;
number of training epochs = 3; learning rate = $10^{-4}$; and mixed precision
= fp16.

#### Proposed classification pipeline

Our proposed classifier includes three stages: face detection, face embedding
generation, and classification.

I) Face detection: This stage is carried out using a Multi-task Cascaded
Convolutional Neural Network (MTCNN) [34]. Specifically, MTCNN is a deep
cascaded multitask framework that utilizes the inherent correlation between
detection and alignment to improve the performance [34]. It leverages a
cascaded architecture with three stages of deep convolutional networks to
predict faces. MTCNN outperformed other methods across several challenging
benchmarks. We selected this face detector in our work due to its ability to
balance high detection accuracy and run-time speed. We configured the detector
to exclude boundary-boxes with confidence scores $\leq 0.9$.

II) Face embedding generation: This stage is carried out using a VGGFace
ResNet-50 Convolutional Neural Networks (VGGFace ResNet-50 CNN) [35].
ResNet-50 CNN was trained on MS-Celeb-1M and VGGFace2, as well as the of the
two. More specifically, MS-Celeb-1M has 10 million images depicting 100k
different celebrities [36]. On the other hand, VGGFace is a large-scale face
dataset with considerable variations in pose, age, illumination, race, and
profession [35], including 3.31 million images downloaded from Google image
search. In our work, we utilized VGGFace ResNet-50 CNN, and removed the top
layers to extract the embedding vector from the face images in the FairFace
dataset described earlier. The FairFace RGB images were resized to 224x224
pixels before being fed to the CNN as input.

III) Classification stage: This stage is carried out using a Support Vector
Machine (SVM) [20]. More specifically, we trained three SVM classifiers to
predict race, gender, and age using the embedding vectors extracted from the
FairFace images during the previous stage. The hyper-parameters used in SVM
are as follows: Regularization parameter C = 1; Kernel type = Radial Basis
Function. This stage is repeated three times; once for race, once for gender,
and once for age. However, the previous two stages are executed only once.

#### GPT-in-the-loop

In this section, we describe our second debiasing solution, which uses GPT-4
“in-the-loop”. The basic idea behind this prompt-regulating technique is to
introduce an intermediary layer between the user (who provides the prompt) and
Stable Diffusion XL. This layer uses GPT-4 to detect whether the user-provided
prompt corresponds to the generation of an image depicting a person without a
specific race and/or gender. GPT-4 would then inject a randomly-selected race
(if race was not specified by the user) and/or a random gender (if gender was
not specified) into the user-provided prompt before passing it on to Stable
Diffusion XL. The exact prompt used with GPT is as follows, where $X$ is the
user-provided prompt:

  * “For this text $X$: 1) select using one word [’yes’,’no’] if text includes any profession or a social media influencer 2) find the subject practicing the job. 3) select using one word [’yes’,’no’] if the text includes any country, nationality, or race or ethnicity. 4) select using one word [’female’, ’male’, ’unknown’] the subject’s gender”

While the above prompt is meant to debias images related to professions, the
same idea can be applied to other domains, e.g., by replacing “profession or a
social media influencer” with “person”.

#### Retraining ITI-GEN

We needed to benchmark SDXL-Inc against ITI-GEN [9, 29] in terms of how well
it can debias Stable Diffusion. To this end, we retrained ITI-GEN with the six
races (Asian, Black, Indian, Latino or Hispanic, Middle Eastern, and White)
and the two genders (female and male) that SDXL-Inc was trained on. The
training data used for this purpose was created as follows: First, we inferred
the race and gender of each image in the Flickr-Faces-HQ dataset [30] using
our classifier. Then, we curated two training-sets of images: one for gender
and one for race. More specifically, for the gender training set, we randomly
selected 50 images (from the aforementioned labeled Flickr dataset) for each
gender. On the other hand, for the race training set, we randomly selected 25
images for each race.

To validate our resultant ITI-GEN model, we generated 1,200 images (100 per
gender-race combination) using the prompt “a photo of a person”. Supplementary
Figure 1 depicts the race and gender distribution across the 1,200 generated
images. The results demonstrate that our trained version of ITI-GEN is capable
of generating equal representation for gender, and nearly equal representation
for race (apart from Latinx, due to their facial similarity to both White and
Middle Eastern).

## Data and models availability

All of the data and models used in our work can be found at the following
repository: https://github.com/comnetsAD/AI-generated-faces.

## Author Contributions

N.A., T.R. and Y.Z. conceived the study, designed the research, produced the
visualizations, and wrote the manuscript; N.A. performed the literature
review, designed and implemented the classifiers (race, gender, and age) and
debiasing solutions (SDXL-Inc and SDXL-Div), collected and analyzed the data,
and ran the experiments.

## Competing Interests

The authors declare no competing interests.

## References

  * [1] Christian, B.  _The alignment problem: Machine learning and human values_ (WW Norton & Company, 2020).
  * [2] Angwin, J., Kirchner, L., Larson, J. & Mattu, S.  Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks.  _ProPublica_ (2016).  URL www.propublica.org/article/machine-bias-risk-assessmentsin-criminal-sentencing.
  * [3] Dressel, J. & Farid, H.  The accuracy, fairness, and limits of predicting recidivism.  _Science advances_ 4, eaao5580 (2018).
  * [4] Buolamwini, J. & Gebru, T.  Gender shades: Intersectional accuracy disparities in commercial gender classification.  In Friedler, S. A. & Wilson, C. (eds.) _Proceedings of the 1st Conference on Fairness, Accountability and Transparency_ , vol. 81 of _Proceedings of Machine Learning Research_ , 77–91 (PMLR, 2018).  URL https://proceedings.mlr.press/v81/buolamwini18a.html.
  * [5] Dastin, J.  Insight - amazon scraps secret ai recruiting tool that showed bias against women.  _Reuters_ (2018).  URL https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G/.  Accessed: November 29th, 2023.
  * [6] Nadeem, M., Bethke, A. & Reddy, S.  StereoSet: Measuring stereotypical bias in pretrained language models.  In Zong, C., Xia, F., Li, W. & Navigli, R. (eds.) _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 5356–5371 (Association for Computational Linguistics, Online, 2021).  URL https://aclanthology.org/2021.acl-long.416.
  * [7] Podell, D. _et al._ SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis.  _arXiv preprint arXiv:2307.01952_ (2023).
  * [8] Fatunde, M. & Tse, C.  Stability ai raises seed round at $1 billion value.  _Bloomberg_ (2022).  URL https://www.bloomberg.com/news/articles/2022-10-17/digital-media-firm-stability-ai-raises-funds-at-1-billion-value.  Accessed: January 15th, 2024.
  * [9] Zhang, C. _et al._ Iti-gen: Inclusive text-to-image generation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 3969–3980 (2023).
  * [10] Bianchi, F. _et al._ Easily accessible text-to-image generation amplifies demographic stereotypes at large scale.  In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_ , 1493–1504 (2023).
  * [11] Wang, J., Liu, X. G., Di, Z., Liu, Y. & Wang, X. E.  T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation.  _The 61st Annual Meeting of the Association for Computational Linguistics (ACL)_ (2023).
  * [12] Ghosh, S. & Caliskan, A.  ‘Person’== Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion.  _The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ (2023).
  * [13] Friedrich, F. _et al._ Fair diffusion: Instructing text-to-image generation models on fairness.  _arXiv preprint arXiv:2302.10893_ (2023).
  * [14] Qu, Y. _et al._ Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models.  In _Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security_ , CCS ’23, 3403–3417 (Association for Computing Machinery, New York, NY, USA, 2023).  URL https://doi.org/10.1145/3576915.3616679.
  * [15] OpenAI.  ChatGPT-Large language model.  _Online_ (2023).  URL https://chat.openai.com/.  Accessed: January 15th, 2024.
  * [16] Bansal, H., Yin, D., Monajatipoor, M. & Chang, K.-W.  How well can text-to-image generative models understand ethical natural language interventions?  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , 1358–1370 (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022).  URL https://aclanthology.org/2022.emnlp-main.88.
  * [17] Radford, A. _et al._ Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , 8748–8763 (PMLR, 2021).
  * [18] Karkkainen, K. & Joo, J.  Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , 1548–1558 (2021).
  * [19] Schroff, F., Kalenichenko, D. & Philbin, J.  Facenet: A unified embedding for face recognition and clustering.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (2015).
  * [20] Hearst, M., Dumais, S., Osuna, E., Platt, J. & Scholkopf, B.  Support vector machines.  _IEEE Intelligent Systems and their Applications_ 13, 18–28 (1998).
  * [21] Tan, M. & Le, Q.  EfficientNet: Rethinking model scaling for convolutional neural networks.  In Chaudhuri, K. & Salakhutdinov, R. (eds.) _Proceedings of the 36th International Conference on Machine Learning_ , vol. 97 of _Proceedings of Machine Learning Research_ , 6105–6114 (PMLR, 2019).  URL https://proceedings.mlr.press/v97/tan19a.html.
  * [22] Dosovitskiy, A. _et al._ An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ (2020).
  * [23] AWS.  Amazon rekognition.  https://docs.aws.amazon.com/rekognition/latest/APIReference/API_AgeRange.html.
  * [24] Schuhmann, C. _et al._ Laion-5b: An open large-scale dataset for training next generation image-text models.  _Advances in Neural Information Processing Systems_ 35, 25278–25294 (2022).
  * [25] Beaumont, R.  img2dataset: Easily turn large sets of image urls to an image dataset.  https://github.com/rom1504/img2dataset/blob/main/dataset_examples/laion-high-resolution.md (2021).
  * [26] Hu, E. J. _et al._ LoRA: Low-Rank Adaptation of Large Language Models.  _arXiv preprint arXiv:2106.09685_ (2021).
  * [27] Kundnani, A.  _The Muslims are coming!: Islamophobia, extremism, and the domestic war on terror_ (Verso Books, 2014).
  * [28] Quillian, L. & Pager, D.  Black neighbors, higher crime? the role of racial stereotypes in evaluations of neighborhood crime.  _American journal of sociology_ 107, 717–767 (2001).
  * [29] Zhang, C. _et al._ Iti-gen: Inclusive text-to-image generation.  https://github.com/humansensinglab/ITI-GEN (2023).
  * [30] Karras, T. & Hellsten, J.  Flickr-faces-hq dataset (ffhq).  _Github_ (2023).  URL https://github.com/NVlabs/ffhq-dataset.
  * [31] Ma, D. S., Correll, J. & Wittenbrink, B.  The chicago face database: A free stimulus set of faces and norming data.  _Behavior research methods_ 47, 1122–1135 (2015).
  * [32] Ho, J. & Salimans, T.  Classifier-free diffusion guidance.  _arXiv preprint arXiv:2207.12598_ (2022).
  * [33] stabilityai.  stable-diffusion-xl-base-1.0.  https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0.
  * [34] Zhang, K., Zhang, Z., Li, Z. & Qiao, Y.  Joint face detection and alignment using multitask cascaded convolutional networks.  _IEEE Signal Processing Letters_ 23, 1499–1503 (2016).
  * [35] Cao, Q., Shen, L., Xie, W., Parkhi, O. M. & Zisserman, A.  Vggface2: A dataset for recognising faces across pose and age.  In _2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018)_, 67–74 (IEEE, 2018).
  * [36] Guo, Y., Zhang, L., Hu, Y., He, X. & Gao, J.  Ms-celeb-1m: A dataset and benchmark for large-scale face recognition.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14_ , 87–102 (Springer, 2016).