Sruthi<EMAIL_ADDRESS>Viraj<EMAIL_ADDRESS>Arvindkumar
<EMAIL_ADDRESS>Judy<EMAIL_ADDRESS>Georgia
Institute of Technology
Atlanta, GA Mitigating Bias in Visual Transformers

# Mitigating Bias in Visual Transformers via Targeted Alignment

###### Abstract

As transformer architectures become increasingly prevalent in computer vision,
it is critical to understand their fairness implications. We perform the first
study of the fairness of transformers applied to computer vision and benchmark
several bias mitigation approaches from prior work. We visualize the feature
space of the transformer self-attention modules and discover that a
significant portion of the bias is encoded in the query matrix. With this
knowledge, we propose TADeT, a targeted alignment strategy for debiasing
transformers that aims to discover and remove bias primarily from query matrix
features. We measure performance using Balanced Accuracy and Standard
Accuracy, and fairness using Equalized Odds and Balanced Accuracy Difference.
TADeT consistently leads to improved fairness over prior work on multiple
attribute prediction tasks on the CelebA dataset, without compromising
performance.

##  1 Introduction

Over the past decade, deep learning-based computer vision has been advancing
at a rapid pace and is being deployed in high-stakes applications ranging from
candidate job hiring to facial recognition systems. Several recent studies
have shown that deep models take advantage of spurious correlations between
attributes in the data when learning to make predictions. Such unintended
correlations can cause the model to act in a biased way, such as having lower
accuracy on certain sub-populations of the data [Buolamwini and Gebru(2018),
Singh et al.(2020)Singh, Mahajan, Grauman, Lee, Feiszli, and Ghadiyaram,
Wilson et al.(2019)Wilson, Hoffman, and Morgenstern, Kay et al.(2015)Kay,
Matuszek, and Munson, du2020fairness]. These biases can be harmful and cause
discrimination towards these populations once deployed.

While the fairness of computer vision tasks learnt using convolutional neural
network (CNN) models is well explored, the fairness implications of _Visual
Transformers_ are yet to be studied. Transformers were first introduced in NLP
[Vaswani et al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez,
Kaiser, and Polosukhin], and later found to be effective on computer vision
tasks [Dosovitskiy et al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn,
Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, Uszkoreit, and Houlsby]
due to their ability to learn long-range relationships, unlike CNN’s which
have have a limited field of view at any given layer. There has been a rise in
the use of transformers in vision tasks [khan2021transformers]. In this paper,
we perform the first study of visual transformers with respect to fairness.
Transformers have a naturally separated and spatially cohesive feature space
[Vaswani et al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez,
Kaiser, and Polosukhin], and we investigate how this unique architecture may
be leveraged to ensure fairness.

There are several ways to measure the fairness of a model [Dwork et
al.(2011)Dwork, Hardt, Pitassi, Reingold, and Zemel, Hardt et al.(2016)Hardt,
Price, and Srebro, Zhao et al.(2017)Zhao, Wang, Yatskar, Ordonez, and Chang,
Dieterich et al.(2016)Dieterich, Mendoza, and Brennan]. In this work, we focus
on an image classification task for which all predictive outputs are equally
desirable, and hence measure performance using _balanced_ (multiclass)
accuracy [Park et al.(2020)Park, Kim, Hwang, and Byun]. Further, we focus on
two fairness metrics: Equalized Odds [Dwork et al.(2011)Dwork, Hardt, Pitassi,
Reingold, and Zemel], as it focuses on equal true positive and false positive
rates across the protected attribute, and a new auxiliary measure we
introduce, Balanced Accuracy Difference ($\Delta$ BA) which measures the
difference in Balanced Accuracy across the two groups. This metric, which is
similar to accuracy equity [Dieterich et al.(2016)Dieterich, Mendoza, and
Brennan], gives practitioners an overall idea of the _difference_ in model
performance across a protected attribute.

Several debiasing algorithms have been proposed and studied for CNNs,
including re-weighting instances [Wang et al.(2019)Wang, Zhao, Yatskar, Chang,
and Ordonez], MMD alignment [Long et al.(2015)Long, Cao, Wang, and Jordan],
domain independent training [Dwork et al.(2011)Dwork, Hardt, Pitassi,
Reingold, and Zemel, Wang et al.(2020b)Wang, Qinami, Karakozis, Genova, Nair,
Hata, and Russakovsky], and adversarial training [Zhang et al.(2018)Zhang,
Lemoine, and Mitchell]. However, many of these techniques suffer from a trade-
off where increased bias mitigation results in reduced predictive performance
[Wang et al.(2020b)Wang, Qinami, Karakozis, Genova, Nair, Hata, and
Russakovsky]. In addition, these works either require a large number of the
protected class data or are designed for learning with a CNN model, making it
unclear how well these methods will translate to newer backbone architectures
like visual transformers.

Figure 1: We discover that bias in a visual transformer model across a
particular attribute label (e.g. A = high cheekbones / not high cheekbones)
can be attributed to a variance in the query matrix activations for a fixed
task label (Y = smiling / not smiling), even with similar key activations.
This observation motivates TADeT (example shown on CelebA [Liu et
al.(2015)Liu, Luo, Wang, and Tang]).

We perform the first fairness assessment of visual transformer models and
benchmark existing bias mitigation algorithms. We then introduce TADeT,
Targeted Alignment for Debiasing Transformers, a novel algorithm that applies
class-specific Maximum Mean Discrepancy (MMD) [Gretton et al.(2008)Gretton,
Borgwardt, Rasch, Scholkopf, and Smola] alignment and class-specific
adversarial [Zhang et al.(2018)Zhang, Lemoine, and Mitchell] debiasing, to a
_targeted_ part of the visual transformer feature space. We discover that when
bias exists for a particular attribute, the core piece of the model which
differs between two subgroups (one with and one without said attribute) can be
observed through the query activations. For example, in Figure 1 we consider a
Transformer trained to predict the presence of the “Smiling” attribute, and
visualize its _average_ query and key matrix activations over the dataset. We
find that for the subset of images where smiling is present, a large
discrepancy exists in average query activations across the protected attribute
of “High cheekbones” (box 1, column 1), whereas the key activations are much
more similar (box 1, column 2). This result is also observed in the case of
_not_ smiling images (box 2). Thus, by imposing alignment [Ganin and
Lempitsky(2015)] directly on the query activations we may be able to
effectively reduce model bias while limiting the impact on predictive
performance.

We evaluate our method on the task of single-label attribute prediction on the
CelebA dataset [Liu et al.(2015)Liu, Luo, Wang, and Tang], while requiring
invariance to a defined protected attribute. We measure the performance of our
method using Balanced Accuracy and Standard Accuracy, and the fairness of our
method using Equalized Odds [Hardt et al.(2016)Hardt, Price, and Srebro] and
Balanced Accuracy Difference. We make the following contributions:

  1. 1.

We perform the first assessment of the fairness of representations learned by
visual transformers, and benchmark several existing debiasing methods from the
literature.

  2. 2.

We visualize the transformer feature space and discover that a significant
portion of learned bias is encoded in the activations of the transformer’s
query matrices.

  3. 3.

Motivated by this, we propose TADeT, a debiasing strategy for visual
transformers using _targeted_ , _class-specific_ , MMD alignment and
adversarial debiasing to improve upon existing fairness algorithms without
compromising accuracy.

##  2 Related Work

Transformers in computer vision. Transformers were first introduced in NLP by
Vaswani  _et al._ [Vaswani et al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit,
Jones, Gomez, Kaiser, and Polosukhin], which used self-attention to develop
better context and allow models to attend to relevant words across the entire
corpus of text. Recently, ViT (Visual Transformer) [Dosovitskiy et
al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner,
Dehghani, Minderer, Heigold, Gelly, Uszkoreit, and Houlsby] has emerged as a
new architecture for vision capable of achieving state-of-the-art performance
on standard recognition tasks. Given the strong performance of this
architectural paradigm, many future works are bound to leverage the visual
transformer model, necessitating a study into the fairness implications and
mitigation approaches designed for transformers.

Measuring Biases. Several metrics exist for measuring bias in machine learning
models. Demographic parity [Dwork et al.(2011)Dwork, Hardt, Pitassi, Reingold,
and Zemel] measures whether the rates of positive outcomes across all
protected groups are equal, making it susceptible to the self-fulfilling
prophecy and subset targeting. Bias Amplification [Zhao et al.(2017)Zhao,
Wang, Yatskar, Ordonez, and Chang] measures how a model _amplifies_ biases,
though it does not provide feedback about the inherent biases common in visual
recognition datasets [Tommasi et al.(2015)Tommasi, Patricia, Caputo, and
Tuytelaars, Bhargava and Forsyth(2019), Wang et al.(2019)Wang, Zhao, Yatskar,
Chang, and Ordonez, Denton et al.(2020)Denton, Hutchinson, Mitchell, Gebru,
and Zaldivar]. Equalized Odds [Hardt et al.(2016)Hardt, Price, and Srebro],
one of the metrics we report, offers a stricter definition of Equality of
Opportunity, by adding an extra constraint on the Demographic Parity measure
where each group must receive the positive and negative outcomes at equal
rates if the datapoints in this group qualify for that outcome. Finally
accuracy equity [Dieterich et al.(2016)Dieterich, Mendoza, and Brennan]
compares accuracy across subpopulations. However, when a dataset is class
imbalanced, this measurement will have an implicit bias against the minority
class. Therefore, we introduce Balanced Accuracy Difference as a way to
measure the difference in a model’s performance across a protected attribute
while accounting for class imbalance.

Mitigation Methods. Some of the most straightforward mitigation methods focus
on re-sampling or re-weighting techniques [Elkan(2001), Bickel et
al.(2009)Bickel, Brückner, and Scheffer], but these methods may have limited
information gain through resampling and in some cases suffer from overfitting
[Weiss et al.(2007)Weiss, McCarthy, and Zabar]. A different style of approach
seeks to directly impose alignment of feature spaces by optimizing an MMD loss
between features from different subgroups [Long et al.(2015)Long, Cao, Wang,
and Jordan], though mitigation benefits are variable [Madras et
al.(2018)Madras, Creager, Pitassi, and Zemel]. A domain discriminative [Dwork
et al.(2011)Dwork, Hardt, Pitassi, Reingold, and Zemel] approach trains a
classifier for each protected attribute and task combination and hence
attempts to induce ’fairness through awareness’, though recent works [Wang et
al.(2020b)Wang, Qinami, Karakozis, Genova, Nair, Hata, and Russakovsky]
demonstrate limitations including a lack of leveraging between-class
information and an increased number of parameters. A domain independent
approach [Wang et al.(2019)Wang, Zhao, Yatskar, Chang, and Ordonez] trains an
ensemble of classifiers with a shared feature representation to ensure that
rare attribute values are adequately represented. While this approach can lead
to large bias reductions, it is limited in that it assumes adequate labeled
data for each setting of the protected attribute in order to train a
representative model.

Many recent approaches have leveraged the success of adversarial learning
techniques for domain alignment [Ganin and Lempitsky(2015)]. Adversarial
Debiasing [Zhang et al.(2018)Zhang, Lemoine, and Mitchell] alternates between
training a classifier to distinguish the protected attribute and updating the
representation to be invariant to the protected attribute. LAFTR [Madras et
al.(2018)Madras, Creager, Pitassi, and Zemel] is an adversarial method with
the ability to modify the training objective based on the desired fairness
measure (equalized odds, equality of opportunity, and demographic parity).
However, in our experiments using this method with the equalized odds
objective, we find that it is unable to consistently achieve fairness
improvements while maintaining high performance.

We benchmark MMD, Domain Independent, Adversarial Debiasing, and LAFTR methods
on transformers. We then propose TADeT, a targeted alignment approach to
debias transformers (leveraging the query matrix features) that achieves
improved fairness with minimal reduction in predictive performance.

##  3 Approach

Recently, visual transformers [Dosovitskiy et al.(2020)Dosovitskiy, Beyer,
Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold,
Gelly, Uszkoreit, and Houlsby] have emerged as a popular architecture for
learning visual recognition models. While the fairness of CNN models has been
documented [Buolamwini and Gebru(2018)] and mitigation strategies have been
proposed [Zhang et al.(2018)Zhang, Lemoine, and Mitchell, Madras et
al.(2018)Madras, Creager, Pitassi, and Zemel], we study the bias which is
present within a visual transformer-based model and propose TADeT, a new bias
mitigation algorithm designed for debiasing visual transformers. TADeT focuses
alignment on query activations within the transformer feature space to remove
bias against a protected attribute.

###  3.1 Notation and Background

Let ${\bm{x}}$ denote an input image and ${y}$ the corresponding task label.
Our goal is to learn a classifier ${h}$, on top of deep encoder, ${f}$, such
that ${h}({f}({\bm{x}}))\xrightarrow{}{y}$ achieves high heldout task accuracy
while being fair with respect to a protected attribute, ${a}$. Fairness has
many definitions; in this work we focus on designing a model which is
invariant with respect to the protected attribute (see Sec 2).

The model ${f}$ employs the visual transformer architecture introduced in
Dosovitskiy  _et al._ [Dosovitskiy et al.(2020)Dosovitskiy, Beyer, Kolesnikov,
Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, Uszkoreit,
and Houlsby]. The input image ${\bm{x}}\in\mathbb{R}^{H\times W\times C}$ is
reshaped into $N$ patches $\\{{x}_{1},..,{x}_{N}\\}$, where each patch $x_{i}$
is of size $\frac{HW}{N}\times\frac{HW}{N}\times C$. Each patch is then
flattened and mapped to ${D}$ dimensions with a trainable linear projection. A
final learnable class token ${x}_{\text{class}}$ of the same dimension, ${D}$,
is appended to the sequence of patches. This class token is randomly
initialized and updated throughout training. These $N+1$ patch embeddings are
appended with a positional embedding to retain positional information, and
then fed through ${L}$ layers of the transformer encoder. Each layer of the
encoder is identical and includes a multi-headed self-attention module
followed by alternating MLP blocks and layernorms.

The multi-headed self-attention module includes ${M}$ attention heads. For
each attention head, ${m}$, the concatenated $N+1$ patch embeddings are split
into 3 branches and each fed to a linear layer. The 3 branches result in
learning Query (${Q}$), Key (${K}$), and Value (${V}$) activations
respectively, each of dimension ${M}\times(N+1)\times{D}$, which attempt to
encode specific pieces of information. The value matrix is tasked with
learning an encoding of the actual attributes of the data (_eg._ for a face
image, information such as hair color, eye color, etc.). The key matrix
encodes the location of _where_ specific information is stored in the value
matrix (_eg._ where the eye color is located). Finally, the query matrix is
trained to “ask” the key matrix for the information it requires for the task.
Activations produced from these 3 matrices are then combined to produce the
self-attention defined in Vaswani  _et al._[Vaswani et al.(2017)Vaswani,
Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin] as:

$\text{Attention}({Q},{K},{V})=\text{softmax}\left(\frac{{Q}{K}^{T}}{\sqrt{{D}}}\right){V}$
(1)

The activations from the self-attention module are fed into alternating MLP
and layernorms in each layer of the encoder. Finally, the features for the
class token, ${x}_{\text{class}}$, are extracted from the output of the final
layer transformer encoder and used as the feature representation of input
${\bm{x}}$ to predict output ${y}$.

Figure 2: Overview of TADeT. First, a preprocessed input image and an
additional class token are passed through ${L}$ transformer encoder layers,
and features extracted from the class token are used for task classification.
Second, query activations of the last layer of the encoder are used for
Targeted Debiasing in two ways: i) query activations corresponding to the
class token are used for class-specific adversarial alignment (Sec. 3.2.1),
and ii) for each class, the $L_{2}$ distance between average query activations
for each setting of the protected attribute is minimized (Sec. 3.2.2).

###  3.2 TADeT: Targeted Alignment for Debiasing Transformers

Unlike CNNs, transformers possess a naturally interpretable feature space due
to separation into queries and keys. On studying the average query and key
activations of the last layer of a visual transformer encoder before making a
prediction, we find a measurable difference across average query activations
across the protected attribute values, but only minimal differences between
average key activations across the protected attribute values (see Figure 1).
These differences generalize across the D dimensions and M attention heads of
the query matrix. This observation of a large difference in query activations
motivates our focus on alignment of query activations.

We introduce TADeT, Targeted Alignment for Debiasing Transformers, a bias
mitigation algorithm that focuses on aligning _query activations_ (see Figure
2). TADeT considers a visual transformer model trained using a standard task
loss and performs debiasing through the introduction of two class-specific
alignment losses, 1) an adversarial loss which aligns the class token of the
query activations, $L_{\text{adv}}$, (Sec 3.2.1) and, 2) an $L_{2}$ loss which
penalizes large deviations between the average query activations for values of
the protected attribute, $L_{q}$, (Sec 3.2.2). Our overall model considers the
joint optimization between these two objectives together with a task
supervised cross-entropy loss, $L_{\text{CE}}$.

####  3.2.1 Class-specific Adversarial Alignment

We begin alignment across populations with different attributes by leveraging
adversarial mitigation [Zhang et al.(2018)Zhang, Lemoine, and Mitchell]. In
adversarial training for feature alignment, an adversary ${g}$ is applied to a
feature space to determine if two groups are distinguishable. Optimization
alternates between training the discriminator to distinguish the groups, ${a}$
in our case, and updating the feature encoder, ${f}$, to confuse the
discriminator. Whereas prior work aligns the penultimate features for an
instance, ${\bm{x}}$, we seek to align the class token of the query matrix,
${Q}_{\bm{x}}[:,N+1,:]$. This results in a standard min/max optimization over
this targeted feature space:

$\mathcal{L}_{\text{adv}}({\bm{x}};{f},{g})=\max_{{f}}\min_{{g}}\mathbb{E}_{{\bm{x}},{y},{a}}[\mathcal{L_{\text{CE}}}({g}({Q}_{\bm{x}}[:,N+1,:]),{a})]$
(2)

Furthermore, prior works propose aligning the features across the protected
attribute by aligning $p(\hat{{y}}=y|{a}=0)$ and $p(\hat{{y}}=y|{a}=1)$.
However, this does not take into account the ground truth task label ${y}$,
allowing two datapoints with different task labels to be aligned, which will
lead to incorrect classification. Therefore, we propose using _class-specific_
alignment where we align the features based on both the protected attribute
value _and_ the task label value (_i.e._ we only align
$p(\hat{{y}}=y|{a}=0,{y}=1)$ with $p(\hat{{y}}=y|{a}=1,{y}=1)$, and only align
$p(\hat{{y}}=y|{a}=0,{y}=0)$ with $p(\hat{{y}}=y|{a}=1,{y}=0)$).

####  3.2.2 Direct Alignment of Class-Specific Query Activations

Our hypothesis is that by reducing the differences between the average query
activations, we can encourage invariance to the protected attribute and
improve fairness. Following this we add a Maximum Mean Discrepancy (MMD) loss
[Gretton et al.(2008)Gretton, Borgwardt, Rasch, Scholkopf, and Smola] to our
objective function that minimizes the L2 distance between the average query
activations for instances from each attribute. We exclude the class token of
the query activation, ${Q}_{\bm{x}}[:,N+1,:]$, as it does not encode spatial
information. We first compute the average query activation in each minibatch,
B, for each task label (y) and protected attribute (a) combination, where
${Q}_{{\bm{x}}_{i}}$ indicates the query activation for image ${\bm{x}}_{i}$:

$\hat{{Q}}_{{y},{a}}=\frac{1}{|{B}|}\sum_{i\in{B},\;\text{s.t.}\;{y}_{i}={y},{a}_{i}={a}}{Q}_{{\bm{x}}_{i}}[:,1:N,:]$
(3)

Next, we compute the average of $L_{2}$ differences across the protected
attribute for each element in $\hat{{Q}}_{{y},{a}}$ independently, and then
average these differences across the ${D}$ channels and ${M}$ attention heads
of the matrix. Due to the reasoning provided in Sec.3.2.1, we utilize class-
specific alignment of the query loss to ensure that the appropriate features
are aligned. For binary attributes this yields the following alignment loss:

$\mathcal{L}_{q}({\bm{x}};{f})=\frac{1}{2\cdot{M}{D}}\sum_{{m}\in{M}}\sum_{{d}\in{D}}\sum_{{y}\in\\{0,1\\}}||\hat{{Q}}_{{y},0}[{m},:,{d}]-\hat{{Q}}_{{y},1}[{m},:,{d}]||_{2}$
(4)

Altogether, our model results from the following joint minimization:

$\mathcal{L}({\bm{x}},{y};{f},{h},{g})=\mathcal{L}_{\text{CE}}({h}({f}({\bm{x}})),{y})+\alpha\mathcal{L}_{\text{adv}}({\bm{x}};{g},{f})+\beta\mathcal{L}_{q}({\bm{x}};{f})$
(5)

where hyper-parameters $\alpha$ and $\beta$ are tuned and optimized for
fairness.

##  4 Experiments

###  4.1 Setup & Implementation Details

We evaluate TADeT on CelebA [Liu et al.(2015)Liu, Luo, Wang, and Tang], a
dataset containing 200k celebrity face images with annotations for 40 binary
attributes. We present results on three settings, each with a corresponding
binary task (${y}$) that the model is trained to predict, and a binary
protected attribute (${a}$) over which we wish the model to be unbiased. The
three settings described as a tuple (${y},{a}$) are as follows: i) (Smiling,
High Cheekbones), ii) (Wavy Hair, Male), and iii) (Wavy Hair, Wearing
Lipstick). We take careful steps to choose these settings: First we train a
baseline Transformer model on different task attributes (including Smiling and
Wavy Hair). Then we measure the widely-used Equalized Odds fairness metric to
estimate how much bias is correlated with each protected attribute (results
for Smiling reported in Figure 3). Finally, we choose the protected attribute
with the highest Equalized Odds and thus the most bias (more details in the
supplement).

Figure 3: We choose our evaluation settings by measuring the fairness of a
Transformer model trained to predict a given task attribute across different
protected attributes. This figure above shows the fairness (measured by
Equalized Odds) across each protected attribute for a Transformer trained to
predict Smiling. We select High Cheekbones (in green) as it is the attribute
with the greatest bias. A similar process is followed for the (Wavy Hair,
Male) and (Wavy Hair, Wearing Lipstick) settings.

Implementation Details. We first train a transformer from scratch on each of
these 3 settings. The transformer consists of 196 patches (each representing a
16x16 area of the image), 1 class token patch, 12 transformer encoder layers,
and 8 attention heads. We flatten and project each patch into a 64-dimensional
vector and add positional embeddings [Wang et al.(2020b)Wang, Qinami,
Karakozis, Genova, Nair, Hata, and Russakovsky]. The embedded patches are fed
into the transformer encoder. We use parameter sharing across key and value
weights [Wang et al.(2020a)Wang, Li, Khabsa, Fang, and Ma]. After the
transformer encoder processes the patch embeddings, the class token patch is
fed into 2 fully-connected layers and a sigmoid layer to produce a single
normalized output score (since we deal with binary classification). We use the
Adam optimizer [kingma2017adam] with a learning rate of 1$\times 10^{-4}$,
train for 50 epochs, and following Wang  _et al._[Wang et al.(2020b)Wang,
Qinami, Karakozis, Genova, Nair, Hata, and Russakovsky], select the model with
the best weighted average precision (AP) on the validation set.

We implement TADeT by applying a class-specific adversarial and class-specific
query loss on top of our pretrained transformer model to perform targeted
debiasing. We align the query class token activations by applying one
adversary head per setting of the task label. The adversary takes in the
512-dimensional vector query class token of the final (12${}^{\text{th}}$)
layer of the transformer encoder. The class token is processed through 2
fully-connected layers and a sigmoid layer to produce a single normalized
output score of the binary protected attribute ${a}$. We perform a sweep
across multiple loss weights for the adversarial ($\alpha$) and query
($\beta$) losses, and select the one yielding the best fairness metrics.

Metrics. As performance metrics, we report Standard Accuracy and Balanced
Accuracy [Park et al.(2020)Park, Kim, Hwang, and Byun],
$\dfrac{1}{4}[TPR_{{a}=0}+TNR_{{a}=0}+TPR_{{a}=1}+TNR_{{a}=1}]$, which is
needed when task data is imbalanced. As fairness metrics, we report Equalized
Odds [Hardt et al.(2016)Hardt, Price, and Srebro] and introduce Balanced
Accuracy Difference. Equalized Odds is used to understand the discrepancy in
true positive rates and false positive rates across the protected attribute.
Balanced Accuracy Difference looks at the difference in Balanced Accuracy
between the binary values of the protected group. By computing the difference
in Balanced Accuracy, as opposed to difference in Standard Accuracy proposed
by Dieterich  _et al._[Dieterich et al.(2016)Dieterich, Mendoza, and Brennan],
we are able to account for the class imbalance in the original dataset. A
detailed use case is described in the supplement.

$\text{Balanced\ Acc.\
Difference}(\Delta\text{BA})=\dfrac{1}{2}[TPR_{{a}=0}+TNR_{{a}=0}]-\dfrac{1}{2}[TPR_{{a}=1}+TNR_{{a}=1}]$
(6)

###  4.2 Baselines

We evaluate the following debiasing algorithms with the pretrained transformer
as baselines.

i) Maximum Mean Discrepancy (MMD) [Long et al.(2015)Long, Cao, Wang, and
Jordan]:  MMD computes a mean of penultimate layer feature activations for
each setting of the protected attribute, and then minimizes their L2 distance.
We select a loss weight that yields the highest validation results.

ii) Domain Adversarial Neural Network (DANN) [Ganin and Lempitsky(2015)]:
Following successful adversarial debiasing [Hardt et al.(2016)Hardt, Price,
and Srebro], we build on domain adversarial alignment [Ganin and
Lempitsky(2015)] using an attribute adversary learned on top of the
penultimate layer activations. The adversarial head consists of 2 linear
layers that take in a 512-dimensional vector class token, followed by a
sigmoid, and select the adversarial loss weight that yields the highest
validation results.

iii) LAFTR [Madras et al.(2018)Madras, Creager, Pitassi, and Zemel]: We train
a model with a modified adversarial objective which attempts to satisfy the
Equalized Odds fairness measure. This objective is implemented by minimizing
the average absolute difference on each task attribute-protected attribute
(${y},{a}$) combination. This allows the model to directly optimize for the
equalized odds measure.

iv) Domain Independent Training [Wang et al.(2020b)Wang, Qinami, Karakozis,
Genova, Nair, Hata, and Russakovsky]:  We learn a shared feature
representation with an ensemble of classifiers (two), trained with the same
baseline hyperparameters. Given the predictions, we perform inference by
averaging the class decision boundaries.

###  4.3 Results

Method |  Y: Smiling A: High Cheekbones |  Y: Wavy Hair A: Male |  Y: Wavy Hair A: Wearing Lipstick
---|---|---|---
EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$ | EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$ | EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$
Transformer | 15.89 | 9.18 | 87.00 | 92.21 | 19.76 | 12.10 | 72.79 | 78.18 | 19.72 | 9.86 | 74.33 | 78.81
MMD [Long et al.(2015)Long, Cao, Wang, and Jordan] | 14.83 | 3.26 | 87.65 | 92.61 | 20.86 | 12.26 | 72.90 | 78.51 | 17.88 | 7.82 | 76.00 | 79.98
DANN [Ganin and Lempitsky(2015)] | 15.13 | 3.58 | 87.67 | 92.72 | 19.74 | 9.36 | 74.70 | 79.65 | 18.63 | 7.67 | 75.21 | 79.31
LAFTR [Madras et al.(2018)Madras, Creager, Pitassi, and Zemel] | 16.13 | 4.95 | 87.42 | 92.78 | 19.03 | 9.24 | 74.75 | 79.55 | 18.17 | 6.50 | 76.11 | 79.55
Domain Ind. [Wang et al.(2020b)Wang, Qinami, Karakozis, Genova, Nair, Hata, and Russakovsky] | 13.12 | 8.59 | 86.61 | 90.89 | 14.99 | 11.01 | 69.71 | 74.09 | 13.29 | 9.03 | 70.61 | 73.80
TADeT (Ours) | 14.77 | 2.42 | 87.73 | 92.68 | 18.58 | 8.63 | 75.10 | 79.73 | 17.31 | 7.08 | 76.11 | 79.91

Table 1: Debiasing results. Y=Task Attribute. A=Protected Attribute.
EO=Equalized Odds. $\Delta$BA=Balanced Accuracy Difference. BA=Balanced
Accuracy. Best and second best performing method bolded and underlined
respectively.

We report results in Table 1. We find that TADeT consistently improves both
Equalized Odds and Balanced Accuracy Difference while maintaining Balanced
Accuracy and Standard Accuracy compared to the original transformer model
(best or second best in 11/12 settings). Within a test setting there may be
other debiasing methods which produce stronger debiasing results according to
a single fairness metric, such as LAFTR on the (Wavy Hair,Wearing Lipstick)
setting, but we note that across settings and considering all metrics our
method consistently performs comparably or better than prior work. We further
note that while Domain Independent training provides the strongest Equalized
Odds performance, this approach does not produce strong Balanced Accuracy
Difference (fairness) or Balanced Accuracy and Standard Accuracy
(performance), likely due to imbalanced task data for the underrepresented
protected attribute. For example, in the (Wavy Hair, Male) setting, though we
see a 3.59% drop in Equalized Odds compared to TADeT, this is accompanied by
worse performance on all other metrics including: a 2.38% increase in Balanced
Accuracy Difference, a 5.39% drop in Balanced Accuracy, and a 5.64% drop in
Standard Accuracy.

###  4.4 Ablating TADeT

| Adversarial Loss | Query Loss | Fairness Metrics $\downarrow$ | Performance Metrics $\uparrow$ |
---|---|---|---|---|---
# | activations | on top of | alignment | alignment |  |  |  |  |
| penultimate | query | full | class token | full | class spe | full | class spe | Eq Odds |  $\Delta$ Bal Acc | Bal Acc | Std Acc |
1* [Dosovitskiy et al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, Uszkoreit, and Houlsby] | (Original Transformer Model) | 15.89 | 9.18 | 87.00 | 92.21 |
2* [Ganin and Lempitsky(2015)] | ✓ |  | ✓ |  | ✓ |  |  |  | 15.13 | 3.58 | 87.67 | 92.72 |
3 | ✓ |  | ✓ |  |  | ✓ |  |  | 14.64 | 3.46 | 88.01 | 92.90 |
4 |  | ✓ | ✓ |  | ✓ |  |  |  | 15.35 | 3.54 | 87.71 | 92.84 |
5 |  | ✓ | ✓ |  |  | ✓ |  |  | 14.93 | 3.89 | 87.59 | 92.57 |
6 |  | ✓ |  | ✓ | ✓ |  |  |  | 16.17 | 2.24 | 87.24 | 92.67 |
7 |  | ✓ |  | ✓ |  | ✓ |  |  | 14.34 | 2.98 | 87.90 | 92.69 |
8 |  | ✓ |  | ✓ |  | ✓ | ✓ |  | 15.10 | 2.40 | 87.48 | 92.55 |
9 (TADeT) |  | ✓ |  | ✓ |  | ✓ |  | ✓ | 14.77 | 2.42 | 87.73 | 92.68 |

Table 2: Ablating TADeT. This analysis portrays the importance of applying a
class specific adversary to align the class token activations across the
protected attribute, and performing MMD alignment on the query matrix
activations across the protected attribute. * indicates a baseline method we
are comparing against.

In Table 2 we ablate TADeT and observe that each component of our method
contributes to improved fairness measures.

  * $\Rightarrow$

Class specific adversary improves fairness (Row 3): We first experiment with
applying a class-specific adversarial loss to align distributions within a
setting of the task attribute. This leads to improvements in both fairness
metrics and both accuracy metrics.

  * $\Rightarrow$

Positioning adversary head on full query activations is ineffective (Row 4 &
5): Next we move the adversary head on top of the full output of the final
query matrix. We experiment with both a standard (Row 4) and class-specific
(Row 5) adversarial loss. While this does not do better than DANN, we
hypothesize that this is because the input to the adversary is the entire
query matrix rather than just the class token, which is what is actually used
for task classification.

  * $\Rightarrow$

Only operating on the query class token greatly improves fairness(Row 6 & 7):
To test our prior hypothesis, we allow the adversary to only operate on the
class token of the query matrix. We experiment with both a standard (Row 6)
and class specific (Row 7) adversarial loss and notice that the latter results
in strong Equalized Odds measures.

  * $\Rightarrow$

Adding query loss improves Balanced Accuracy Difference but sacrifices
Equalized Odds (Row 8): Next, following our intuition of equalizing the query
activations, we add a query loss. We notice an improvement in Balanced
Accuracy Difference but no significant improvement in Equalized Odds. We again
hypothesize that class specificity is required to notice improvements.

  * $\Rightarrow$

Adding class specificity to query loss improves Equalized Odds while
maintaining Balanced Accuracy Difference (Row 9): Finally, a class-specific
query loss combined with a class-specific adversarial loss on the query class
token achieves the strongest and most consistent performance across all
metrics, implying that each component of TADeT is helpful in reducing bias
while maintaining strong predictive performance.

###  4.5 Analysis

Figure 4: We show the average query and key activations for a fixed task label
(Y=smiling or not smiling) and a fixed attribute label (A = High Cheekbones or
not High Cheekbones) on the CelebA dataset after debiasing. The difference in
query activations across attribute labels is reduced compared to the results
observed before debiasing (Figure 1).

As seen in Figure 1 (_left_), before applying TADeT, the average query
activations across both settings of the protected attribute are very different
(large L2 distance of 5.82 and 2.4 for smiling and not smiling respectively),
while the average activations from the key matrix are much closer (L2 distance
of 3.07 and 0.49). These qualitative visualization results and quantitative L2
results are consistent across different attention heads and channels of the
query and key activations (additional results provided in the supplement).

After performing debiasing using TADeT, we visualize the new values of the
average query and key activations and calculate the same L2 distances. As seen
in Figure 4, the query activations look very similar and have a small L2
difference (0.01 and 0.0) across settings of the protected attribute. These
results, combined with our improved fairness results with TADeT (Table 1),
validates our hypothesis that targeted debiasing focused on the query
activations is an effective debiasing strategy for visual transformers.

Comparing CNN’s and Transformers. We train a CNN on the same 3 settings
described previously, and evaluate accuracy and fairness. As shown in Tab. 3,
the baseline Transformer consistently performs significantly worse on both
fairness metrics (Equalized Odds and Balanced Accuracy Difference). However,
we refrain from concluding that Transformers are more biased than CNN’s,
because we see that Transformers also perform slightly worse in terms of
accuracy measures (Balanced Accuracy and Standard Accuracy), which makes this
an unfair comparison. As techniques develop to improve the accuracy of Vision
Transformers to be commensurate with CNNs, we hope to address this question
more conclusively.

Method |  Y: Smiling A: High Cheekbones |  Y: Wavy Hair A: Male |  Y: Wavy Hair A: Wearing Lipstick
---|---|---|---
EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$ | EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$ | EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$
CNN | 14.66 | 2.69 | 88.15 | 93.06 | 16.71 | 8.08 | 77.99 | 82.20 | 15.64 | 6.41 | 78.75 | 82.20
Transformer | 15.89 | 9.18 | 87.00 | 92.21 | 19.76 | 12.10 | 72.79 | 78.18 | 19.72 | 9.86 | 74.33 | 78.81

Table 3: Fairness and Accuracy metrics compared for a CNN and Transformer
trained on the same task attribute-protected attribute combination. Y=Task.
A=Protected Attribute. EO=Equalized Odds. $\Delta$BA=Balanced Accuracy
Difference. BA=Balanced Accuracy.

##  5 Conclusion

In this work, we perform the first benchmarking of several existing debiasing
algorithms on visual transformers. We visualize the feature space learned by
the transformer self-attention modules and find a significant portion of the
bias is encoded in the query matrices. Using this information, we introduce
TADeT, a targeted debiasing algorithm for transformers that aims to remove
bias encoded within the query matrix.

Our method suffers from some limitations. First, we note that it is important
for users to consider the application of their model before using our
findings. In this work, we measure fairness using Equalized Odds and Balanced
Accuracy Difference, but if one’s intended use case does not align with these
definitions of fairness, the results presented may not be applicable.
Furthermore, all previous methods benchmarked in this paper, along with TADeT,
require annotations for the protected attribute, which may not always be
feasible.

Acknowledgments: This work was funded in part by Cisco Inc.

## References

  * [Bhargava and Forsyth(2019)] Shruti Bhargava and David Forsyth.  Exposing and correcting the gender bias in image captioning datasets and models, 2019.
  * [Bickel et al.(2009)Bickel, Brückner, and Scheffer] Steffen Bickel, Michael Brückner, and Tobias Scheffer.  Discriminative learning under covariate shift., 2009.
  * [Buolamwini and Gebru(2018)] Joy Buolamwini and Timnit Gebru.  Gender shades: Intersectional accuracy disparities in commercial gender classification, 2018.
  * [Denton et al.(2020)Denton, Hutchinson, Mitchell, Gebru, and Zaldivar] Emily Denton, Ben Hutchinson, Margaret Mitchell, Timnit Gebru, and Andrew Zaldivar.  Image counterfactual sensitivity analysis for detecting unintended bias, 2020.
  * [Dieterich et al.(2016)Dieterich, Mendoza, and Brennan] William Dieterich, Christina Mendoza, and Tim Brennan.  Compas risk scales: Demonstrating accuracy equity and predictive parity, 2016.
  * [Dosovitskiy et al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, Uszkoreit, and Houlsby] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale, 2020.
  * [Dwork et al.(2011)Dwork, Hardt, Pitassi, Reingold, and Zemel] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Rich Zemel.  Fairness through awareness, 2011.
  * [Elkan(2001)] Charles Elkan.  The foundations of cost-sensitive learning, 2001.
  * [Ganin and Lempitsky(2015)] Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation, 2015.
  * [Gretton et al.(2008)Gretton, Borgwardt, Rasch, Scholkopf, and Smola] Arthur Gretton, Karsten Borgwardt, Malte J. Rasch, Bernhard Scholkopf, and Alexander J. Smola.  A kernel method for the two-sample problem, 2008.
  * [Hardt et al.(2016)Hardt, Price, and Srebro] Moritz Hardt, Eric Price, and Nathan Srebro.  Equality of opportunity in supervised learning, 2016.
  * [Kay et al.(2015)Kay, Matuszek, and Munson] Matthew Kay, Cynthia Matuszek, and Sean A Munson.  Unequal representation and gender stereotypes in image search results for occupations, 2015.
  * [Liu et al.(2015)Liu, Luo, Wang, and Tang] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild, 2015.
  * [Long et al.(2015)Long, Cao, Wang, and Jordan] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan.  Learning transferable features with deep adaptation networks, 2015.
  * [Madras et al.(2018)Madras, Creager, Pitassi, and Zemel] David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel.  Learning adversarially fair and transferable representations, 2018.
  * [Park et al.(2020)Park, Kim, Hwang, and Byun] Sungho Park, Dohyung Kim, Sunhee Hwang, and Hyeran Byun.  Readme: Representation learning by fairness-aware disentangling method, 2020.
  * [Singh et al.(2020)Singh, Mahajan, Grauman, Lee, Feiszli, and Ghadiyaram] Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadiyaram.  Don’t judge an object by its context: Learning to overcome contextual bias, 2020.
  * [Tommasi et al.(2015)Tommasi, Patricia, Caputo, and Tuytelaars] Tatiana Tommasi, Novi Patricia, Barbara Caputo, and Tinne Tuytelaars.  A deeper look at dataset bias, 2015.
  * [Vaswani et al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2017.
  * [Wang et al.(2020a)Wang, Li, Khabsa, Fang, and Ma] Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma.  Linformer: Self-attention with linear complexity, 2020a.
  * [Wang et al.(2019)Wang, Zhao, Yatskar, Chang, and Ordonez] Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez.  Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations, 2019.
  * [Wang et al.(2020b)Wang, Qinami, Karakozis, Genova, Nair, Hata, and Russakovsky] Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky.  Towards fairness in visual recognition: Effective strategies for bias mitigation, 2020b.
  * [Weiss et al.(2007)Weiss, McCarthy, and Zabar] G. Weiss, Kate McCarthy, and Bibi Zabar.  Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs?, 2007.
  * [Wilson et al.(2019)Wilson, Hoffman, and Morgenstern] Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern.  Predictive inequity in object detection, 2019.  URL http://arxiv.org/abs/1902.11097.
  * [Zhang et al.(2018)Zhang, Lemoine, and Mitchell] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell.  Mitigating unwanted biases with adversarial learning, 2018.
  * [Zhao et al.(2017)Zhao, Wang, Yatskar, Ordonez, and Chang] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang.  Men also like shopping: Reducing gender bias amplification using corpus-level constraints, 2017.

##  6 Supplementary Work

###  6.1 Evaluation Setting Details

In Sec 4.1 we mention that we choose three settings to benchmark prior work
and evaluate TADeT. These three settings, described as a tuple (${y},{a}$),
are as follows: i) (Smiling, High Cheekbones), ii) (Wavy Hair, Male), and iii)
(Wavy Hair, Wearing Lipstick). Here we provide details on the choice of these
three settings.

####  6.1.1 Smiling with High Cheekbones

Figure 5: Full dataset distribution of task, protected attribute tuple
(Smiling, High Cheekbones).

The first setting we evaluate on is (Smiling, High Cheekbones) (See Fig 5). We
notice that there is a significant skew in the distribution of the CelebA
dataset, where most “Smiling” faces are correlated with “High Cheekbones” and
most “Not Smiling” faces are correlated with “Not High Cheekbones”. However,
we understand that not all such correlations will necessarily translate to a
_model_ bias. Therefore, we analyze the true positive rate and false positive
rate of a transformer trained on this dataset to predict the “Smiling”
attribute, for each setting of the protected attribute “High Cheekbones”. We
obtain the following results: $\text{TPR}_{{a}=1}=92.83\%$,
$\text{TPR}_{{a}=0}=67.76\%$, $\text{FPR}_{{a}=1}=9.65\%$, and
$\text{FPR}_{{a}=0}=2.94\%$. Clearly, the model performs significantly worse
on correctly predicting “Smiling” when the individual does _not_ have “High
Cheekbones” (${a}=0)$. Furthermore, the model has a $6.71\%$ larger FPR for
the “High Cheekbones” group (${a}=1)$. This confirms that the model indeed has
a strong bias of (spuriously) correlating the presence of “High Cheekbones”
with whether they are “Smiling”. Due to this clear bias, we choose (Smiling,
High Cheekbones) as our first setting for evaluation.

####  6.1.2 Wavy Hair with Male

Figure 6: Full dataset distribution of task, protected attribute tuple (Wavy
Hair, Male).

Next, we evaluate on (Wavy Hair, Male) (See Fig 6). First, we notice that the
dataset contains less “Male” individuals (28.5%) than “Not Male” individuals
(71.5%). Due to the fact that “Male” is underrepresented in the data, we
suspect that the model will not be able to learn as strong of a representation
of the images with the “Male” attribute as it will for the “Not Male”
attribute, which could lead to some type of bias. Next, we notice that most
“Male” individuals are labeled as “Not Wavy Hair”, which could cause a
spurious correlation between these two attributes. When analyzing a
Transformer trained on the “Wavy Hair” prediction task, we notice exactly
that: a high difference in TPR where $\text{TPR}_{{a}=1}=36.06\%$ and
$\text{TPR}_{{a}=0}=67.92\%$. This indicates that a (spurious) dataset
correlation of “Male” with “Not Wavy Hair” is being learned by the model,
leading to a large bias wherein the model is overpredicting “Male” to have
“Not Wavy Hair”.

####  6.1.3 Wavy Hair with Wearing Lipstick

Figure 7: Full dataset distribution of task, protected attribute tuple (Wavy
Hair, Wearing Lipstick).

Finally, we evaluate on (Wavy Hair, Wearing Lipstick) (See Fig 7). There is a
natural correlation in the dataset where most “Male” individuals are “Not
Wearing Lipstick”, and vice versa. Since “Wearing Lipstick” and “Male” are
correlated, and “Male” and “Wavy Hair” are correlated (as shown in 6.1.2), we
want to see if the bias that is present when the protected attribute is “Male”
will persist when we set the protected attribute as “Wearing Lipstick”.
Looking at the dataset distributions themselves, we notice that the number of
people “Wearing Lipstick” and “Not Wearing Lipstick” is closer to 50/50 than
in the previous setting (protected attribute=“Male”). Next, we notice that
there is a correlation in the dataset of people with “Wavy Hair” and people
“Wearing Lipstick”. After training a transformer, we notice that the Equalized
Odds and Balanced Accuracy Difference are almost as high as the (Wavy Hair,
Male) setting. Therefore, we conclude that this setting will be a good test-
bed, as the distribution is not as clearly skewed as (Wavy Hair, Male), but
there is poor performance on both fairness metrics Equalized Odds and Balanced
Accuracy Difference.

###  6.2 Balanced Accuracy Difference Use Cases

In Sec. 4.1 of the main paper, we introduce Balanced Accuracy Difference, a
fairness metric that shares some of the motivation behind accuracy equity
[Dieterich et al.(2016)Dieterich, Mendoza, and Brennan], but with an important
implementation difference to account for real-world data distributions. While
accuracy equity suggests taking the difference in Standard Accuracy across the
protected attribute, we take the difference in Balanced Accuracy (our
performance metric) across the protected attribute. By doing so, we can
account for class imbalance in the dataset, as we saw in Sec. 6.1. Furthermore
Balanced Accuracy Difference is important because it provides a more holistic
understanding of the Equalized Odds metric, as we now elaborate.

Consider a situation wherein after debiasing a model, the true positive rate
(TPR) difference across a protected attribute _increases_ slightly while false
positive rate (FPR) difference decreases substantially. Since Equalized Odds
is an average of TPR difference and FPR difference ($\text{{Equalized
Odds}}=\dfrac{1}{2}[TPR_{{a}=1}-TPR_{{a}=0}]+\dfrac{1}{2}[FPR_{{a}=1}-FPR_{{a}=0}]$),
the resulting Equalized Odds measure will reduce, indicating that the model is
fairer than the original model. However due to the _increased_ TPR difference,
predicting the positive outcome for the protected attribute is actually _more_
unfair than before debiasing! This means that for the positive outcome, a
large bias across the protected attribute still exists, which the Equalized
Odds metric does not adequately capture.

However, in this situation, the Balanced Accuracy Difference will be high as
it looks at the _differences_ in Balanced Accuracy within each subgroup. More
specifically, Balanced Accuracy Difference can be rewritten as:

$\text{Balanced\ Acc.\
Difference}(\Delta\text{BA})=\dfrac{1}{2}[TPR_{{a}=0}+TNR_{{a}=0}]-\dfrac{1}{2}[TPR_{{a}=1}+TNR_{{a}=1}]$
(7)
$=\dfrac{1}{2}[TPR_{{a}=1}+TNR_{{a}=1}]-\dfrac{1}{2}[TPR_{{a}=0}+TNR_{{a}=0}]$
(8)
$=\dfrac{1}{2}[TPR_{{a}=1}-TPR_{{a}=0}]+\dfrac{1}{2}[(TNR_{{a}=1}-TNR_{{a}=0}]$
(9)
$=\dfrac{1}{2}[TPR_{{a}=1}-TPR_{{a}=0}]+\dfrac{1}{2}[(1-FPR_{{a}=1})-(1-FPR_{{a}=0})]$
(10)
$=\dfrac{1}{2}[TPR_{{a}=1}-TPR_{{a}=0}]+\dfrac{1}{2}[(FPR_{{a}=0}-FPR_{{a}=1})]$
(11)
$=\dfrac{1}{2}[TPR_{{a}=1}-TPR_{{a}=0}]-\dfrac{1}{2}[(FPR_{{a}=1}-FPR_{{a}=0})]$
(12)

By Eq. 12, it is clear that given our situation where TPR difference is
slightly higher than before debiasing, and FPR difference is lower than before
debiasing, Balanced Accuracy Difference would _increase_ , indicating that the
model is behaving in a biased manner, even though the Equalized Odds measure
decreases. Therefore, a user will realize that the drop in Equalized Odds does
not tell the full story, as Balanced Accuracy Difference will indicate that
their model still encodes a bias, especially towards predicting the positive
value for one setting of the protected attribute. Hence, we advocate for using
Balanced Accuracy Difference as an _additional_ fairness metric, along with
Equalized Odds.

###  6.3 Transformer Feature Visualizations

Recall that in Figure 1 of the paper, we presented a visualization of the
average Query and Key activations for each (y,a) tuple combination, for a
specific attention head and channel of a transformer trained for the “Smiling”
prediction task. In Figure 8, we provide visualizations that demonstrate that
the differences we notice in the query activations, and the similarity noticed
in the key activations, generalizes across different attention heads and
channels of the query and key matrices. Further, this also generalizes across
different tasks.

Figure 8: We show that the variance in the query matrix activations for a
particular task label, y, across a protected attribute a, generalizes across
attention heads, channels, and data settings. The Transformer Query/Key Matrix
has 64 channels and 8 heads, and we have chosen 3 random channel/head
combinations from all 3 tasks to depict the generalization of differences in
activations.

###  6.4 Analyzing Class-Specific Alignment for CNNs

In TADeT, we propose using class-specific alignment wherein we align the
distribution of the protected attribute within the task attribute by utilizing
an adversary head _per-attribute_ during adversarial learning. We have shown
the benefits of this method for debiasing visual transformers. In Table 4, we
show that such class-specific alignment for adversarial training improves upon
previous debiasing algorithms in most settings for CNN’s as well.

Method |  Y: Wavy Hair A: Male |  Y: Smiling A: High Cheekbones
---|---|---
EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$ | EO $\downarrow$ |  $\Delta$ BA ($\%)\downarrow$ | BA (%)$\uparrow$ | Acc (%)$\uparrow$
Original CNN | 16.71 | 8.08 | 77.99 | 82.20 | 14.66 | 2.69 | 88.15 | 93.06
DANN [Ganin and Lempitsky(2015)] | 14.75 | 7.39 | 77.36 | 81.12 | 15.04 | 1.85 | 87.97 | 93.03
DANN Class Specific | 14.57 | 7.06 | 77.21 | 80.89 | 14.51 | 3.37 | 88.40 | 93.24

Table 4: CNN Debiasing results. Y=Task. A=Protected Attribute. EO=Equalized
Odds. $\Delta$BA=Balanced Accuracy Difference. BA=Balanced Accuracy.


[cft] [cft] [cft] [cft] cft]Center For Theoretical Physics, Aleja Lotników
32/46, 02-668 Warsaw, Poland Deep-learning photometric redshifts for KiDS DR4
bright galaxies

# Deep learning based photometric redshifts for the Kilo-Degree Survey bright
galaxy sample

Anjitha John William  Priyanka Jalan  Maciej Bilicki  Wojciech A. Hellwing  [

###### Abstract

In cosmological analyses, precise redshift determination remains pivotal for
understanding cosmic evolution. However, with only a fraction of galaxies
having spectroscopic redshifts (spec-$z$s), the challenge lies in estimating
redshifts for a larger number. To address this, photometry-based redshift
(photo-$z$) estimation, employing machine learning algorithms, is a viable
solution. Identifying the limitations of previous methods, this study focuses
on implementing deep learning (DL) techniques within the Kilo-Degree Survey
(KiDS) Bright Sample for more accurate photo-$z$ estimations. Comparing our
new DL-based model against prior ‘shallow’ neural networks, we showcase
improvements in redshift accuracy. Our model gives mean photo-$z$ bias
$\langle\Delta z\rangle=10^{-3}$ and scatter $\mathrm{SMAD}(\Delta z)=0.016$,
where $\Delta z=(z_{\mathrm{phot}}-z_{\mathrm{spec}})/(1+z_{\mathrm{spec}})$.
This research highlights the promising role of DL in revolutionizing photo-$z$
estimation.

##  1 Introduction

Redshift is a basic quantity in cosmological analyses, serving as one of the
indicators of galaxy distances. Its precise determination enables the mapping
of the three-dimensional large-scale structure of the Universe and facilitates
our comprehension of cosmic evolution. Achieving sub-percent accuracy in
redshift measurements is exclusively feasible through spectroscopy. In
obtaining spectroscopic redshifts (spec-zs), we initially capture the spectrum
of a celestial source and subsequently identify the shift in spectral lines
relative to their rest frame.

Due to expensive telescope time needed, spec-$z$s are only attainable for a
fraction of galaxies. However, a viable alternative is the estimation of
redshifts through photometric measurements. Such photometric redshifts
(photo-$z$s) may not provide the same level of redshift accuracy and precision
as spec-$z$s; however, they have proven indispensable in the era of massive
imaging surveys. Such redshift estimates rely on the correlation between
observed galaxy photometry and the true redshift (e.g. Baum, 1957). This
proceeding focuses on estimating this correlation using machine learning (ML)
algorithms.

Machine learning-based techniques for photo-z estimation include supervised
learning (Wadadekar, 2005; Collister & Lahav, 2004; Oyaizu et al., 2008),
unsupervised learning (Way & Klose, 2012), k-nearest neighbors (Graham et al.,
2018), ensemble learning and Gaussian processes (Way & Srivastava, 2006;
Bonfield et al., 2010), mixed density networks (Ansari et al., 2021) and
finally, deep neural networks (Hoyle et al., 2015; D’Isanto & Polsterer,
2018). In supervised ML techniques, the algorithm establishes an empirical
relationship between observed quantities and corresponding labels through
training on appropriately labeled data. The main challenge and limitation of
these methods lie in extrapolating results beyond the representative training
set.

Deep learning (DL) based photo-$z$ derivation is a promising technique among
other supervised ML techniques (e.g. Menou, 2019; Dey et al., 2022; Li et al.,
2022; Treyer et al., 2024). In such frameworks, the term ‘deep’ signifies that
the models typically employ intricate, multi-layer architectures. Deep
learning methods, specifically convolutional neural networks (CNNs), are
selected for predicting photo-zs due to their exceptional ability to discern
patterns in data, particularly images.

In this work, we present a deep learning model for photometric redshift
estimation of bright galaxies in the Kilo-Degree Survey (KiDS, de Jong et al.,
2013), a multiband imaging survey covering about 1350 deg2 of the sky.
Following up on earlier KiDS work by Li et al. (2022), we applied the DL
models for photo-z estimation within the flux-limited ‘KiDS Bright Sample’,
encompassing all galaxies from KiDS with $r<20$ mag (Bilicki et al., 2021). We
compare our derivations with the previously released KiDS bright sample photo-
zs obtained with ‘shallow’ neural networks from the public package ANNz2
(Sadeh et al., 2016). Our new model improves over those previous results and
gives great perspectives for the final KiDS data release.

##  2 Input Data

This section details the datasets used in this study. The training sample
required for the neural network is KiDS-DR4 four-band (u, g, r, and i) optical
galaxy images and their 9-band magnitudes, with their corresponding labels of
true (spectroscopic) redshifts from the Galaxy And Mass Assembly (GAMA, Driver
et al., 2011) survey.

###  2.1 Images and Photometry

KiDS is an extensive optical survey conducted by the European Southern
Observatory (ESO) at the VLT Survey Telescope (VST, Capaccioli & Schipani,
2011), which utilizes the OmegaCAM CCD mosaic camera, boasting an impressive
268 million-pixel focal plane. It is positioned at the ESO Paranal Observatory
in Chile. The VST is an alt-azimuth mounted telescope based on a modified
Ritchey-Chrétien design. KiDS captured images in four distinct broad bands
($ugri$) and encompasses an area of 1350 square degrees within the
extragalactic sky.

We have used KiDS Data Release 4 (DR4, Kuijken et al.,
2019)111https://kids.strw.leidenuniv.nl/DR4 co-added images that had been
calibrated for both astrometry and photometry, ensuring a consistent pixel
scale of 0.2 arcseconds. We have also incorporated the nine-band magnitudes of
galaxies. These latter join original KiDS $ugri$ measurements with those from
the VIKING (Edge et al., 2013) survey, encompassing five near-IR bands
($ZYJHK_{s}$).

###  2.2 GAMA Spectroscopic Data

The true (spectroscopic) redshifts used for training and testing are obtained
from the GAMA dataset. This survey, conducted across five fields, of which
four are fully within the KiDS footprint, covers a total area of approx. 286
square degrees. The data were obtained using the AAOmega fiber-fed
spectrograph facility mounted on the 3.9-meter Anglo-Australian Telescope
(Driver et al., 2011). GAMA gives spectra, redshifts, their quality, and other
ancillary information. KiDS-DR4 fully overlaps with the G09, G12, G15 and G23
fields. In this work, we have used the GAMA-II spectroscopic redshift catalog
of galaxies from the GAMA DR4 (Driver et al., 2022)222http://www.gama-
survey.org/dr4.

We have identified galaxies within KiDS tiles that overlap between the GAMA
and KiDS-bright sample based on their sky coordinates. Subsequently, we
created cutouts for these galaxies, positioning each at the center of the
cutout with dimensions of $7.2^{\prime\prime}\times 7.2^{\prime\prime}$. The
final cutout catalog comprises $\sim 136$k galaxies in the equatorial field,
with $\sim 95$k utilized for model training. Model testing was conducted on
$\sim 20$k galaxies present in both the equatorial field and KiDS bright
sample.

###  2.3 KiDS-DR4 Bright Galaxy Sample

The KiDS-bright galaxy sample (Bilicki et al., 2021) is a collection of KiDS
galaxies selected based on their flux and free from artifacts etc.
Specifically, it includes galaxies with a magnitude limit of
$r_{\mathrm{auto}}<20$ mag, where ‘auto’ stands for SExtractor-derived
estimate of the total flux via automatic aperture photometry.

##  3 Methodology

Convolutional Neural Networks (CNNs), a subtype of Artificial Neural Networks
(ANNs), excel in addressing computer vision challenges like image detection
and classification. The operational principles of CNNs draw inspiration from
the human neural system, mirroring the functionality of neurons (McCulloch,
1943). Similar to biological neurons that receive input and transmit
electrochemical signals, CNNs employ artificial neurons to process
information. The importance of CNNs becomes apparent through their widespread
application in image recognition tasks, as highlighted by e.g. LeCun et al.
(1998). The adaptability and efficacy of CNNs make them a key technology in
the field of computer vision, contributing significantly to advancements in
image-based tasks. Our model includes a special CNN architecture called
Inception (Szegedy et al., 2015). Our approach employs CNNs to forecast the
photo-zs for the KiDS-bright sample, treating it as a regression problem. To
label the training set, we used the spectroscopic GAMA data detailed earlier.

Our ML model for photo-zs is a combined, multi-input one. Inspired by Henghes
et al. (2022); Li et al. (2022), we use both 4-band KiDS images and the
corresponding 9-band KiDS+VIKING magnitudes. We combined two different ML
schemes: ordinary neural networks (ONN) and CNN (Inception). The architecture
of our model is shown in Fig. 3.

When tuning the model and assessing its performance, we primarily employed
three metrics: Mean Squared Error (MSE), Mean Absolute Error (MAE), and
R-squared error ($R^{2}$):

$R^{2}=1-\frac{\sum_{i=1}^{n}(z_{i}-\hat{z}_{i})^{2}}{\sum_{i=1}^{n}(z_{i}-\bar{z})^{2}}\;,$
(1)

where $n$ is the number of samples used, $z_{i}$ is the predicted value,
$\hat{z}_{i}$ is the true value, and $\bar{z}$ is the mean of true redshift
values. $R^{2}$ is a statistical measure indicating how well a model predicts
outcomes in a regression analysis. It represents the proportion of variance
explained by the model relative to the total variance. In an ideal scenario,
its value is unity, and our combined model has demonstrated the capability to
achieve an $R^{2}$ value exceeding $0.92$.

MSE is effective for learning outliers, while MAE is advantageous for
disregarding them. In our approach, we employed the Huber (1964) loss function
that combines the characteristics of both MSE and MAE, providing a balanced
approach to handling outliers during the training of the model:

$H(z,\hat{z})=\begin{cases}\frac{1}{2}(z-\hat{z})^{2},&\text{if
}|z-\hat{z}|\leq\delta,\\\
\delta(|z-\hat{z}|-\frac{\delta}{2}),&\text{otherwise}.\end{cases}$ (2)

The value of $\delta$ we use is 0.0001 by trial and error.

(a) Architecture of the Combined model.

(b) Comparison of spectroscopic and the predicted photometric redshifts for
the test sample. The thick red solid line represents the running median, while
the thinner red lines enclose the scatter, quantified by the SMAD.

(c) Normalised bias as a function of photo-$z$, with red lines encoding the
running median and SMAD as in the upper panel.

##  4 Results

The photo-z performance is evaluated using the following statistics:

  * •

bias:

$\delta z=z_{\mathrm{phot}}-z_{\mathrm{spec}}\,,$ (3)

  * •

normalized bias:

$\Delta z=\frac{\delta z}{1+z_{\mathrm{spec}}}\,,$ (4)

  * •

the standard deviation of $\Delta z$,

  * •

scaled median absolute deviation (SMAD) of $\Delta z$, where

$\text{SMAD}(x)=1.4826\times\text{median}\left(\lvert
x-\text{median}(x)\rvert\right)\,.$ (5)

These statistics for the overall test sample (based on GAMA equatorial data)
are provided in Table 1, where they are directly compared with previous
results from Bilicki et al. (2021), derived with the ANNz2 software employing
9-band KiDS+VIKING magnitudes. There is visible improvement in terms of
scatter, both in the standard deviation and SMAD, from the earlier ANNz2
results to our new DL model. This shows that our combined model gives promise
for improved photo-$z$s for the KiDS Bright Galaxy Sample. We additionally
visualize our DL photo-$z$ performance in Figs. 3 & 3, which show generally
unbiased and stable photo-$z$s as a function of the photometric redshift
itself.

Table 1: Statistics of photometric redshift performance obtained for KiDS-GAMA equatorial spectroscopic sample.  Method | $\langle\delta z\rangle$ | $\langle\Delta z\rangle$ | $\sigma(\Delta z)$ | SMAD$(\Delta z)$
---|---|---|---|---
ANNz2 (Bilicki et al., 2021) | $0.0005$ | $0.0009$ | $0.024$ | $0.018$
Combined model (this work) | $0.001$ | $0.001$ | $0.021$ | $0.016$

##  5 Conclusion and Future Prospects

In these proceedings, we described a new deep-learning model for photometric
redshift derivation in the KiDS bright sample that joins Inception-based CNN
using 4-band KiDS images, with an ANN employing 9-band magnitudes. We see
improvement over previous work by Bilicki et al. (2021), where ANNs with
magnitudes only were used. In the forthcoming paper, we will present more
details and apply our model to the entire KiDS DR4 Bright Galaxy Sample, which
will give new, improved photo-$z$s for that dataset.

In the near future, we plan to build a deep-learning model which will use the
9-band imaging from both KiDS and VIKING (unlike KiDS $ugri$ only as
presently). That model will be subsequently used to derive photo-$z$s for the
Bright Galaxy Sample in the final KiDS Data Release 5 (Wright et al., in
press; Wright, 2023).

##  6 Acknowledgement

This work is funded via the research project ‘Precision and Accuracy for
Cosmological Imaging Surveys (PACIS)’ from the Polish National Science Centre
through a Sonata-Bis grant no. 2020/38/E/ST9/00395. We thank Rui Li and Nicola
Napolitano for their assistance and feedback.

## References

  * Ansari et al. (2021) Ansari, Z., Agnello, A., Gall, C., _A &A_ 650, A90 (2021)
  * Baum (1957) Baum, W. A., _AJ_ 62, 6 (1957)
  * Bilicki et al. (2021) Bilicki, M., et al., _A &A_ 653, A82 (2021)
  * Bonfield et al. (2010) Bonfield, D. G., et al., _MNRAS_ 405, 2, 987 (2010)
  * Capaccioli & Schipani (2011) Capaccioli, M., Schipani, P., _The Messenger_ 146, 2 (2011)
  * Collister & Lahav (2004) Collister, A. A., Lahav, O., _PASP_ 116, 818, 345 (2004)
  * de Jong et al. (2013) de Jong, J. T. A., Verdoes Kleijn, G. A., Kuijken, K. H., Valentijn, E. A., _Experimental Astronomy_ 35, 1-2, 25 (2013)
  * Dey et al. (2022) Dey, B., et al., _MNRAS_ 515, 4, 5285 (2022)
  * D’Isanto & Polsterer (2018) D’Isanto, A., Polsterer, K. L., _A &A_ 609, A111 (2018)
  * Driver et al. (2011) Driver, S. P., et al., _MNRAS_ 413, 2, 971 (2011)
  * Driver et al. (2022) Driver, S. P., et al., _MNRAS_ 513, 1, 439 (2022)
  * Edge et al. (2013) Edge, A., et al., _The Messenger_ 154, 32 (2013)
  * Graham et al. (2018) Graham, M. L., et al., _AJ_ 155, 1, 1 (2018)
  * Henghes et al. (2022) Henghes, B., et al., _MNRAS_ 512, 2, 1696 (2022)
  * Hoyle et al. (2015) Hoyle, B., et al., _MNRAS_ 449, 2, 1275 (2015)
  * Huber (1964) Huber, P. J., _Annals of Mathematical Statistics_ 35, 492 (1964), URL https://api.semanticscholar.org/CorpusID:121252793
  * Kuijken et al. (2019) Kuijken, K., et al., _A &A_ 625, A2 (2019)
  * LeCun et al. (1998) LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., _Proc. IEEE_ 86, 2278 (1998), URL https://api.semanticscholar.org/CorpusID:14542261
  * Li et al. (2022) Li, R., et al., _A &A_ 666, A85 (2022)
  * McCulloch (1943) McCulloch, _The bulletin of mathematical biophysics_ (1943), URL https://doi.org/10.1007/BF02478259
  * Menou (2019) Menou, K., _MNRAS_ 489, 4, 4802 (2019)
  * Oyaizu et al. (2008) Oyaizu, H., et al., _ApJ_ 689, 2, 709 (2008)
  * Sadeh et al. (2016) Sadeh, I., Abdalla, F. B., Lahav, O., _PASP_ 128, 968, 104502 (2016)
  * Szegedy et al. (2015) Szegedy, C., et al., in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–9 (2015)
  * Treyer et al. (2024) Treyer, M., et al., _MNRAS_ 527, 1, 651 (2024)
  * Wadadekar (2005) Wadadekar, Y., _PASP_ 117, 827, 79 (2005)
  * Way & Klose (2012) Way, M. J., Klose, C. D., _PASP_ 124, 913, 274 (2012)
  * Way & Srivastava (2006) Way, M. J., Srivastava, A. N., _ApJ_ 647, 1, 102 (2006)
  * Wright (2023) Wright, A., in A Decade of ESO Wide-field Imaging Surveys (surveys2023, 5 (2023)


SF2A 2022

# Gender and precarity in Astronomy

N. A. Webb  IRAP, Université de Toulouse, CNRS, CNES, Toulouse, France  C. Bot
CDS, Université de Strasbourg, CNRS, Observatoire astronomique de Strasbourg,
UMR 7550, 67000 Strasbourg, France  S. Charpinet1 T. Contini1 L. Jouve1 H.
Meheut3 Université Côte d’Azur, Observatoire de la Côte d’Azur, CNRS,
Laboratoire Lagrange, Laboratoire Artḿis, France  S. Mei  Université de Paris,
CNRS, Astroparticule et Cosmologie, F-75013 Paris, France  B. Mosser  LESIA,
Observatoire de Paris, PSL Research University, CNRS, Université Pierre et
Marie Curie, Université Paris Diderot, 92195 Meudon, France  G. Soucail1

###### Abstract

Following the survey Well-being in astrophysics that was sent out in March
2021, to establish how astrophysics researchers, primarily in France,
experience their career, some of the results were published in Webb et al.
(2021). Here we further analyse the data to determine if gender can cause
different experiences in astrophysics. We also study the impact on the well-
being of temporary staff (primarily PhD students and postdocs), compared to
permanent staff. Whilst more temporary staff stated that they felt permanently
overwhelmed than permanent staff, the experiences in astrophysics for the
different genders were in general very similar, except in one area. More than
three times more females than males experienced harassment or discrimination,
rising sharply for gender discrimination and sexual harassment, where all of
those having experienced sexual harassment and who had provided their gender
in the survey, were female. Further, as previously reported (Webb et al.
2021), 20% of the respondents had suffered mental health issues before
starting their career in astrophysics. We found that whilst this group was
split approximately equally with regards to males and females, the number rose
sharply to almost 45% of astronomers experiencing mental health issues since
starting in astrophysics. Of this population, there were 50% more females than
males. This excess of females was almost entirely made up of the population of
women that had been harassed or discriminated against.

###### keywords:

careers, well-being, gender issues

##  1 Introduction

In order to establish how astrophysics researchers, primarily in France,
experience their career, we sent out a survey in March 2021, to understand the
impact that their job was having on their well-being. 276 people responded to
the survey and some initial results were published towards the end of the year
(Webb et al. 2021). Amongst other things in Webb et al. (2021), we showed that
around half of the respondents expressed pleasure derived from their career,
but it was also clear that many (early career) researchers were suffering due
to overwork, with more than a quarter saying that they work in excess of 50
hours per week and 2% in excess of 90 h per week. Almost 30% professed to
having suffered harrassment or discrimination in the course of their work.
Further, whilst only 20% had suffered mental health issues before starting
their career in astrophysics, $\sim$45% said that they suffered with mental
health problems since starting in astrophysics.

In order to understand the origin of the mental health problems and explore in
greater detail whether there was any difference depending on gender or between
temporary and permanent staff, we examined the data further and the results on
these aspects are presented in the following.

##  2 Comparison of temporary and permanent staff

The majority of respondents to the survey were temporary staff (undergraduate
or masters students, PhD students or post-doctoral scholars), as the survey
was aimed mostly at this population, as many studies (e.g. Auerbach et al.
2018; Woolston 2019) had already shown that this population was experiencing
the most difficulties. This group amounted to 215 people and 61 permanent
staff responded to the survey. Correcting for the different population sizes
as we do for gender, see Section 3, it was clear that the majority of people
responding to the survey that said that they had experienced
harassment/discrimination was in large majority the post-docs. Concerning
colleagues feeling overwhelmed at work, see Fig. 1, those that felt
overwhelmed all of the time were all temporary staff.

##  3 Comparison of the genders

For those people that provided their gender, 57.8% identified as male, 40.6%
as female and 1.6% identified as other. As few people identified as other, it
is difficult to obtain reliable conclusions for this group and therefore we
focus mainly on the male and female groups in this work. The data presented in
the following is corrected for the differences in the number of women and men
that answered the survey, so the percentages provided correspond to a sample
of 50% male and 50% female. In the majority of the areas we addressed, namely
people’s favourite aspects of astrophysics research, the number of hours
worked, perceived external constraints, job satisfaction and future plans for
remaining in the field, the results were very similar for both males and
females.

There were slight differences in living conditions for males and females. 17%
of respondents said that they have children to care for and of these, 63% were
male. Conversely, 7% stated that they have other family members to care for,
and here 61% were female. 25% replied saying that they live apart from their
partner and/or children and slightly more, 61% were female. 12% of respondents
admitted to having debt problems, 66% being female.

However, there were a couple of areas where the differences between male and
female respondants were highly different and that was in
harassment/discrimination and health. Overall, 3.3 times more women
experienced harassment/discrimination, than men. Figure 2 in Webb et al.
(2021) showed the number of people that had experienced different forms of
harassment or discrimination. In Table 1 we provide the type of
harassment/discrimination when at least five people indicated that they had
been subjected to these problems. We also provide the total number of people
having been subjected to this type of harassment/discrimination, and the ratio
of women to men that were the subject of these issues.

Table 1: Types of harassment/discrimination experienced by at least five respondents, number of people subjected to the issue and the ratio of women to men subjected to the problem, for which the gender was known. Harassment/discrimination | Number of people subjected to issue | Ratio of women to men
---|---|---
Sexual harassment | 15 | $\infty$
Age discrimination | 21 | 4.5
Racial discrimination/harassment | 6 | 7
Gender discrimination | 39 | 22
Shaming by a superior or a colleague | 33 | 3.1
Harassment by supervisor/superior | 29 | 3.2

With regards to health issues, only a third of all respondents said that they
were happy with their health, with significantly less than half of these being
female. Figure 1 shows how often respondents felt overwhelmed. The histogram
shows the number of respondents for the five possibilities, never, rarely,
sometimes, often or all the time. For the group that felt overwhelmed either
often or all of the time, there were 50% more females than males. 44% of all
respondents felt that the expectations were too high in astrophysics and of
these, there were again 50% more females than males. Only 37% of those that
felt that senior colleagues were there to provide support when they needed it,
were female.

Figure 1: Number of respondents that felt overwhelmed never, rarely,
sometimes, often or all the time.

As stated above, 20% of the respondents had suffered mental health issues
before starting their career in astrophysics and these were approximately
equal in proportion between males and females. Since starting in astrophysics,
$\sim$45% suffered with mental health problems. Of this population, there were
50% more females than males. The number of people suffering with bipolar
disorder, eating/body dismorphic disorder, or obsessive-compulsive disorder
remained the same, but for depression, anxiety and panic attacks, the number
of people more than doubled in each case. Whilst our survey took place since
the beginning of the COVID-19 pandemic and some of the cases could be
attributed to the restrictions introduced to stem infections spreading, the
majority of people that started suffering with mental health issues since
starting astrophysics, is not only women, but are those that have been
harassed or discriminated against.

##  4 Discussion

It is clear from the survey that colleagues on temporary contracts are
frequently overwhelmed. This may be due to the workload, down to high
expectations or due to the short contracts, where a great deal is expected
from a PhD student or a post-doc in a short period of time. This can be
exacerbated by those working in projects funded through the Agence Nationale
de la Recherche (the French National Research Agency, ANR) or the European
Research Council (ERC), for example, or in big collaborations, as deadlines to
deliver are often frequent and difficult to achieve. However, aspects external
to the astronomy part of the work can also contribute to temporary contract
colleagues feeling overwhelmed, including: living away from family, be it a
partner, children or parents, which can also remove a support network;
cultural and administrative problems i.e. with paperwork for the job or a
visa, a possible language barrier in a new country, insufficient information
on institute practices, etc; or financial difficulties due to months between
contracts or due to moving to a new institute/country. This may also be due to
harassment or bullying and this is discussed further below.

More than three times as many women than men responding to the survey have
experienced harassment or discrimination in astrophysics in France. Women and
men experienced all types of harassment and discrimination. However, women
were the large majority in gender discrimination, where of the 39 people that
responded to say that they had experienced this kind of discrimination, 32
were female, 2 were male and 0 were ’other’. Five people did not give their
gender. Worse still was sexual harassment, where all of those that experienced
these issues and who provided their gender were female. It is clear that we
are still a long way from gender equality in the world of astrophysics.

The negative outcome of this harassment is that it has lead to mental health
issues, including depression, anxiety and panic attacks, making people’s daily
lives miserable. This is not the first survey to reveal such facts. A recent
survey***https://ras.ac.uk/news-and-press/news/survey-finds-bullying-and-
harassment-systemic-astronomy-and-geophysics of astronomers and geophysicists
commissioned by the RAS Committee on Diversity in Astronomy and Geophysics
and that ran in the UK a year prior to our survey, but for which the results
are only starting to appear, found that 44% of respondents had suffered
bullying and harassment in the workplace within the preceding 12 months. They
also stated that “Women and non-binary people in the field are 50% more likely
than men to be bullied and harassed.”, exactly in line with our findings.
Thanks to more than twice the number of respondents in the UK survey compared
to the French survey (650 respondents from the UK astronomy and geophysics
communities), they were also able to draw conclusions on other minorities.
They found that 50% of lesbian, gay, bisexual, and queer astronomers and
geophysicists were bullied in the 12 months prior to the survey, and that 12%
of bisexual astronomers reported being bullied at least once a week!

The American Institute of Physics (AIP) carries out the Longitudinal Survey of
Astronomy Graduate Students†††https://www.newswise.com/articles/aip-report-
harassment-discrimination-in-astronomy-takes-many-forms. It was initiated by
the American Astronomical Society (AAS) in 2006 to gain an understanding of
the forms and long-term impacts of harassment in astronomy. This study was
carried out by polling astronomy graduates from 2006-07 and followed them in
both 2012-13 and in 2015-16 after they entered the workplace. This study
showed that 33% of the respondents had experienced harassment and
discrimination whilst studying or working. The AIP broke the
harassment/discrimination that was reported into four general groups. These
were: biased assumptions (regarding status, career, or personal life); verbal
put-downs (jokes, criticisms, or undermining comments); demographic-based
inequitable treatment, limiting social support or professional development;
and unwanted sexual attention, ranging from inappropriate comments to more
serious behaviors such as threats, stalking, and assault. The AAS has tried to
improve things by introducing a Code of Ethics and an Anti-Harassment Policy
for AAS-sponsored meetings, publications, etc and supported by a complaint
process and an Ethics Committee. They plan to implement a new survey to
determine whether these policies have helped improve the situation.

Pending the results of the new AAS survey and thus understanding the
effectiveness of the measures implemented by the AAS and whether they should
be implemented more broadly, more can still be done. This could be through
education, as some perpetrators of harassment or discrimination may not
necessarily be aware that their actions are having a negative impact. For
example, long working hours have long been endemic in astronomy and some
researchers may think that this is the only way astronomy can work, and insist
that colleagues work very long hours, as they do not know otherwise. Also, in
the past, astronomy has lacked diversity and some colleagues may lack
experience in interacting with people from other backgrounds. Whilst it is
clear we should all be respectful of each other, it may happen that colleagues
inadvertently say something inappropriate. Becoming familiar with the range of
diversity, be it gender, race, neurodiversity or other, can help to avoid
inappropriate behaviour. As outlined in Webb et al. (2021), everyone working
in astrophysics should be trained in well-being, harassment, discrimination
etc in order to be aware of diversity and be familiar with the language
associated with these issues. We recommend using online courses with tests,
which need to be passed every few years or through information provided in
each institute.

Many people in academia feel a sense of shame when they recognise that they
have been the target of negative behaviour (Keashly 2019). This can extend to
feelings of personal failure, or put them at risk for more targeting. As many
as 20-40% of academic personnel have been harassed, and 50% have witnessed it
according to Keashly (2019). It is therefore essential that anyone is able to
report cases of harassment or discrimination. Action has been taken to ensure
that every astrophysics laboratory in France has a person to whom such
problems can be reported‡‡‡https://www.dgdr.cnrs.fr/bo/2021/BO-
avril-2021-compressed.pdf. However, care must be taken as our survey showed
that in many cases, people did not report the harassment. Only 26% of
respondents that experienced such issues said that they had reported them. Of
those that did not report the harassment, 40% did not report it as they felt
that they wouldn’t be listened to. A further 25% did not report the
harassment, as they felt that other people have worse problems and that it was
not worth reporting. 18% were embarassed and 12% did not even know where to go
to report the problem. Finally 5% did not report the problem due to not
speaking the local language. This is a major hindrance to helping with the
problem, but it appears that this culture of silence is changing, though this
change is still slow (Keashly 2019). To ensure that all problems are reported,
so that action can be taken, it is important that everyone feels that they are
listened to, no matter how big or small the problem and have people that are
able to speak both the local language and English, which is widely spoken by
researchers in astronomy. Having an email address where people can send a
short report of a seemingly minor incident and the name of the perpertrator,
could help to highlight a perpertrator that carries out micro-agressions that
may go unnoticed as isolated incidents, but that could be frequently carried
out on a number of different people. However, the modality for putting this
into place should be discussed to avoid misuse. Finally, it is essential that
if colleagues witness harassment or bullying that they report the incident or
intervene, if they feel that they are able to.

Other areas that can be explored to improve the situation are introducing
Codes of Conduct in different organisations, in a similar way to the work done
by the AAS, specifying clearly what behaviour is acceptable. Codes of Conduct
are becoming more common, so it should be possible to see an improvement in
the coming years. Positive conflict management practices should be introduced.
They should include clear procedures and dedicated committees and
ombudspersons. Through listening and searching for common ground,
collaborative solutions should be found. Some collaborations and organisations
have now trained teams (allies) to observe interactions and help identify
improvements which should also help improve situations. Finally training and
dedicated talks to raise awareness should be held. These should provide the
background needed for everyone to participate and create a positive
environment.

In France improvements are being introduced. In addition to the action taken
to ensure that every astrophysics laboratory has a person where harassment and
discrimination can be reported, a group associated with the National society
for astronomy and astrophysics (SF2A) has been put into place and national
workshops to discuss the problems have been held. A well-being webpage on the
SF2A website is being put into place to centralise information that is already
available, but maybe difficult to find. It includes a description of different
issues and examples of problems to make it clear to everyone, as well as where
to find help. Meetings with gouverning bodies for astrophysics have been held
to try to ensure that paperwork is always provided in the local language and
in English in every institute and in institutes where it is not yet done, up
to date, introductory information about the institue should be provided to new
arrivals either through welcome guides or through bi-annual meetings (or
both). It is also recommended that more (non-)scientific events are organised
to encourage a team spirit and more open discussion on well-being, working
hours, racism, sexism, etc should be organised. The possibility of harmonising
post-doc salaries on a national level and provide an evolution in the
renumeration is being investigated. Many other actions are taking place on a
local level in different French institutes.

##  5 Lessons learnt

This survey was a first attempt to gain insight into the current state of
well-being of primarily French astronomers. It has been highly instructive in
showing that a large proportion of our colleagues are suffering due to the
environment in which they work. It has also provided suggestions as to how to
make improvements. A second survey is planned within a year to extend the
participation to include all colleagues (young and old) across physics. We are
also investigating the possibilty of extending the survey to other countries
and to make the survey more inclusive to include engineers working in the
domain who are also affected by similar problems. This will increase the
number of participants and allow us to improve statistics to better understand
the well-being of minority groups and make a fairer comparison of colleagues
in different age ranges and positions. Allowing two years between the surveys
will allow us to ascertain if any of the actions we have started to put into
place are indeed helping to improve well-being.

##  6 Conclusions

It is clear that we are still a very long way from having equality in
astrophysics. Whilst with the data available in our survey we could only
investigate the prevelance and impact of harassment and discrimination on two
main groups, male and female, other studies have shown that minorities other
than women are also disproportionately impacted by harassment and
discrimination and that this is having a major impact on mental health. It is
clear that this has to stop and through education and speaking out about the
problem, the situation can be improved. We invite you all to be a part of
making astronomy a safer and a happier environment to work in, which will also
help to make it a more productive and prolific one.

## References

  * Auerbach et al. (2018) Auerbach, R. P., Mortier, P., Bruffaerts, R., et al. 2018, Journal of Abnormal Psychology, 127, 623
  * Keashly (2019) Keashly, L. 2019, Workplace Bullying, Mobbing and Harassment in Academe: Faculty Experience, ed. P. D’Cruz, E. Noronha, L. Keashly, & S. Tye-Williams (Singapore: Springer Singapore), 1–77
  * Webb et al. (2021) Webb, N. A., Bot, C., Charpinet, S., et al. 2021, in SF2A-2021: Proceedings of the Annual meeting of the French Society of Astronomy and Astrophysics, ed. A. Siebert, K. Baillié, E. Lagadec, N. Lagarde, J. Malzac, J. B. Marquette, M. N’Diaye, J. Richard, & O. Venot, 35–40
  * Woolston (2019) Woolston, C. 2019, Nature, 575, 403