# Intersectional Fairness: A Fractal Approach

Giulio Filippi∗, Sara Zannone∗, Adriano Koshiyama
Holistic AI
London
UK

###### Abstract

The issue of fairness in AI has received an increasing amount of attention in
recent years. The problem can be approached by looking at different protected
attributes (e.g., ethnicity, gender, etc) independently, but fairness for
individual protected attributes does not imply intersectional fairness. In
this work, we frame the problem of intersectional fairness within a
geometrical setting. We project our data onto a hypercube, and split the
analysis of fairness by levels, where each level encodes the number of
protected attributes we are intersecting over. We prove mathematically that,
while fairness does not propagate "down" the levels, it does propagate "up"
the levels. This means that ensuring fairness for all subgroups at the lowest
intersectional level (e.g., black women, white women, black men and white
men), will necessarily result in fairness for all the above levels, including
each of the protected attributes (e.g., ethnicity and gender) taken
independently. We also derive a formula describing the variance of the set of
estimated success rates on each level, under the assumption of perfect
fairness. Using this theoretical finding as a benchmark, we define a family of
metrics which capture overall intersectional bias. Finally, we propose that
fairness can be metaphorically thought of as a “fractal” problem. In fractals,
patterns at the smallest scale repeat at a larger scale. We see from this
example that tackling the problem at the lowest possible level, in a bottom-up
manner, leads to the natural emergence of fair AI. We suggest that
trustworthiness is necessarily an emergent, fractal and relational property of
the AI system.

**footnotetext: These authors contributed equally to this work. contact:
<EMAIL_ADDRESS>
_K_ eywords Intersectional Fairness $\cdot$ Dynamic Programming $\cdot$
Statistical Parity $\cdot$ Geometry

##  1 Introduction

The issue of fairness in AI has received an increasing amount of attention in
recent years. A number of AI systems involved in sensitive applications, like
recruitment or credit scoring, were found to be biased against minority groups
[Dastin(2018), Telford(2019)]. For these reasons, the machine learning
research community has focused on finding solutions to reduce the bias found
in these models [Agarwal et al.(2018), Agarwal et al.(2019), Li and
Vasconcelos(2019), Mehrabi et al.(2021)]. Many of these solutions focused on
comparing either the outputs of the model (Equality of Outcome [Feldman et
al.(2015)]) or the error made by the algorithm (Equality of Opportunity [Hardt
et al.(2016)]) for different groups. These groups are classically identified
by splitting the population using individual protected attributes, for
instance according to gender, ethnicity or age.

However, what happens when these attributes intersect? The concept of
intersectional fairness, initially introduced by Black feminist scholars
[Crenshaw(2013a), Crenshaw(2013b)], has only recently been introduced in the
context of AI. In their seminal work, Buolamwini and Gebru [Buolamwini and
Gebru(2018)] performed a thorough analysis of three of the most popular facial
recognition algorithms on the market (Microsoft, IBM, and Face++). They found
that all algorithms were better at recognizing men and people with lighter
skin, but the performance drastically decreased for darker-skinned females.
The fact that the discrimination faced by Black women was “greater than the
sum” of the discrimination experienced by Black men and white women is a well-
known concept in the feminist literature [Crenshaw(2013a)], also known as
fairness gerrymandering in the AI literature [Kearns et al.(2018)].

These results have sparked an interest and motivated the need for studying the
intersectional fairness of AI algorithms. Practically, this has meant
extending the study of bias from individual protected attributes to their
intersectional groups. Most of the works on intersectional fairness thus far
has been concerned with measuring bias when intersecting all the protected
attributes [Foulds et al.(2020b), Foulds et al.(2020a), Jin et al.(2020),
Morina et al.(2019)]. Since every dataset may include an arbitrary number of
protected attributes, which can be combined in any number of ways, it is not
clear how one should pick the right level of granularity for the analysis
[Kong(2022)].

In our paper, we tackle the problem from a slightly different perspective.
Instead of fixing the level of analysis a priori, we develop a framework that
allows us to examine all possible intersectional groups, at all levels of
granularity, and their relationship to each other. We frame the problem of
intersectional fairness using a geometrical structure, specifically a
hypercube, where each protected attribute can be seen as a different dimension
of the hypercube. We then project our data on the hypercube and use it to
analyse how bias-related metrics like success rate or accuracy vary for
different subgroups. The geometrical and mechanistic nature of our framework
also allows us to compute the fairness of all possible subgroups in a
computationally efficient way.

This allows us to see how the fairness properties vary for different levels.
From our framework it is easy to see that fairness indeed does not propagate
downwards (i.e., gerrymandering). However, we prove that it does propagate
upwards. This means that ensuring fairness at the most granular level, the
intersection of all protected attributes in the dataset, implies fairness at
any possible intersection. We will show that these results hold for both
Equality of Outcome and Equality of Opportunity frameworks. This mirrors and
extends the results of [Foulds et al.(2020b), Morina et al.(2019), Yang et
al.(2020)]. Finally, we find that, under perfect fairness, the variance of the
set of estimated success rates on a given level decreases exponentially with
increasing levels. We use this formula as a benchmark, and propose a family of
metrics which capture overall intersectional bias.

We argue that the novelty of our work lies especially in the fact that our
framework provides a tool to observe fairness for all the possible
intersectional subgroups at once, and how their fairness properties interlink.
Not only this could be great for visualisation, but it could be useful for
modelling, possibly improving a mechanistic understanding of intersectional
fairness.

##  2 Problem Setting

(a)

(b)

(c)

(d)

Figure 1: Hypercubes. (a) 1D hypercube. Assuming we have only one protected
attribute (gender) with subgroups male/female. (b) 2D hypercube. Assuming
there are two protected attributes (gender and ethnicity), with subgroups
male/female and white/black respectively. (c) 3D hypercube. Assuming there are
three protected attributes (gender, ethnicity and age), with subgroups
male/female, white/black and <40/40+ respectively. (d) 4D hypercube.
Projection of 4D hypercube.

We are given a binary classification dataset $\mathcal{D}$ with $N$ instances
along with $M$ protected attributes (named $p_{1},...,p_{M}$). We assume all
$M$ protected attributes are binary with groups labelled as 0 and 1. Please
note that our analysis can be easily extended beyond the binary case, however,
we choose to have binary protected attributes so that the results have a
simple geometric interpretation. The first insight of our framework is that we
can see our data as living on a $M$ dimensional hypercube.

A hypercube is a geometrical figure that can be extended to an arbitrary
number of dimensions. A 1-dimensional hypercube is a line (Fig. 1(a)), a
2-dimensional hypercube is a square (Fig. 1(b)), and a 3-dimensional hypercube
is a cube (Fig. 1(c)). Each hypercube is made of two copies of the previous
one linked together. Higher dimensional hypercubes are difficult to visualise,
but share the same property (see Fig. 1(d) for an example of a projection of a
4-dimensional hypercube).

First, we would like to explain the analogy between intersectional subgroups
and hypercubes. If we consider a dataset with only one protected attribute,
our dataset can be seen as belonging to a line, with each vertex being one of
the subgroups (Fig. 1(a)). For example, if we consider gender as our protected
attribute, then one vertex will be “male” and the other vertex will be
“female”. If instead we consider two protected attributes, such as gender and
ethnicity, our data will lie on a square (Fig. 1(b)). In this case, each
vertex will be the intersection of gender and ethnicity (e.g., white male,
black male, white female and black female), and each line will be an
individual attribute. Adding a third protected attribute, such as age, will
increase the dimensionality again, and make our geometrical structure into a
cube (Fig. 1(c)). In this case a single protected attribute will corresponds
to the face of the cube, while the intersection of two attributes will
correspond to an edge. The vertices of the cube will be the intersection of
all three protected attributes.

The idea is that each protected attribute creates a division in the data,
which can be seen as adding a dimension to the hypercube. The number of
protected attributes therefore defines the dimensionality of the main-
hypercube, which contains the whole dataset. The vertices of the hypercube,
instead, comprise the intersection of all the protected attributes (e.g.,
$p_{1}=0,...,p_{M}=0$). On the other hand, each edge contains the intersection
of all protected attributes but one. We denote the unspecified protected
attribute with the star symbol (*). More generally, if $K$ of the $M$
protected attributes are unspecified (denoted by *), the associated hypercube
will be a $K$-dimensional hypercube embedded in the main-hypercube. Each
hypercube will then have an associated vector $\mathbf{x}\in\\{0,1,*\\}^{M}$,
which we refer to as it’s vectorial specification.

We will often speak of the hypercubes as separated by levels. The level refers
to the number of stars in the vectorial specification, and encodes the
dimensionality of the embedded hypercube. There are $M+1$ possible levels
$(0,...,M)$. Level 0 (the lowest level) corresponds to the vertices, which are
the deepest intersectional groups and have dimensionality 0. Level $M$ (the
highest level) consists only of the main-hypercube, which contains the whole
dataset and has dimensionality $M$. The total number of hypercubes
(intersectional groups), $H_{tot}$, can be computed by summing the number of
hypercubes $H_{K}$ at each level $K$, with $K\in\\{0,\ldots,M\\}$:

$H_{tot}=\sum_{K=0}^{M}H_{K}=\sum_{K=0}^{M}\binom{M}{K}2^{M-K}=3^{M}$ (1)

For each vertex $\mathbf{v}$ of the hypercube, we can compute the number of
data points belonging to that specific subgroup. We denote this number by
$N(\mathbf{v})$. We can similarly compute the number of data points with
positive label that fall in the same intersection (vertex),
$N^{(1)}(\mathbf{v})$. From these we can compute the success rate for any
vertex as

$SR(\mathbf{v})=\frac{N^{(1)}(\mathbf{v})}{N(\mathbf{v})}$ (2)

Note that it only takes one run through the dataset to compute all success
rates at vertex level. We do so by updating a dictionary with keys being all
possible vertices. For each of the $N$ instances, we are comparing arrays of
length $M$, so the total time complexity is $O(N\times M)$. In the next
section, we explain how this can be used to compute the success rates for all
hypercubes by propagating the computation upwards.

##  3 Propagation Algorithm

We extend the definitions of $N$ and $N^{(1)}$ to work on any hypercube
$\mathbf{x}\in\\{0,1,*\\}^{M}$. The definition is the same as the one for
vertices, $N(\mathbf{x})$ and $N^{(1)}(\mathbf{x})$ are respectively the
number of datapoints and successful datapoints on a given structure. From
these we can compute the success rate of a hypercube as

$SR(\mathbf{x})=\frac{N^{(1)}(\mathbf{x})}{N(\mathbf{x})}$ (3)

If $\mathbf{x}$ is fully unspecified, the associated structure is the main-
hypercube, in this case we use special notation $SR(\mathbf{x})=SR_{tot}$. Let
$\mathbf{x}$ be a hypercube which is not a vertex. Then by definition
$\mathbf{x}$ contains at least one star. Pick the first star in $\mathbf{x}$
and consider the two sub-hypercubes contained by setting that star to 0 or 1,
which we denote as $\mathbf{x}^{(0)}$ and $\mathbf{x}^{(1)}$ respectively.
$\mathbf{x}^{(0)}$ and $\mathbf{x}^{(1)}$ are a partition of $\mathbf{x}$,
meaning that they are disjoint and their union is $\mathbf{x}$. So the
following formulas immediately follow

$N(\mathbf{x})=N(\mathbf{x}^{(0)})+N(\mathbf{x}^{(1)})$ (4)

and

$N^{(1)}(\mathbf{x})=N^{(1)}(\mathbf{x}^{(0)})+N^{(1)}(\mathbf{x}^{(1)})$ (5)

Using these, we can also compute the success rate for hypercube $\mathbf{x}$,
as $SR(\mathbf{x})=N^{(1)}(\mathbf{x})/N(\mathbf{x})$. With equations 4 and 5,
we are in a position to devise a dynamic programming algorithm for computing
all the success rates.

To help in visualizing the propagation process, we think of each possible
split as a branching on a network graph (See Fig. 2). We refer to the
resulting graph $G$ as the hypercube graph. It’s nodes consist of all possible
hypercubes, and the edges encode how these hypercubes split into lower
dimensional ones. It is useful to separate the nodes of the graph by levels
when visualising it. If we have $M$ protected attributes, the resulting graph
$G$ has $3^{M}$ nodes in total, with $H_{K}=\binom{M}{K}2^{M-K}$ of them at
level $K$.

Figure 2: Hypercube graph. Example of a hypercube graph $G$ with $M=2$
protected attributes. The nodes of the graph are all hypercubes. The edges
represent the possible splits.

Algorithm 1. We start by computing $N(\mathbf{v})$ and $N^{(1)}(\mathbf{v})$
for all vertices $\mathbf{v}$. If there is no data on a given vertex, we store
a value of 0 for both of these. Then, while there exists a hypercube with
uncomputed values. Since this hypercube is not a vertex, it’s vector must
contain a star. Pick the first star within the vector and let
$\mathbf{x}^{(0)}$ and $\mathbf{x}^{(1)}$ be the hypercubes obtained by
setting the star to 0, 1 respectively. We recursively compute the result of
equations 4 and 5. Use these to store the success rate if needed. Finish. See
Algorithm 1 for pseudo-code.

Algorithm 1 $SR$ propagation

1:Initialize $to\\_compute$ as a set containing all $3^{M}$ hypercube vectors.

2:Initialize $N,N^{(1)}$ as dictionaries with keys being all elements of
$to\\_compute$.

3:Loop once through dataset updating $N,N^{(1)}$ at vertex level.

4:Remove all vertex vectors from $to\\_compute$.

5:while $to\\_compute$ non empty do

6: Pick first element $\mathbf{x}$ of $to\\_compute$.

7: Find first star in $\mathbf{x}$ and decompose into $\mathbf{x}^{(0)}$ and
$\mathbf{x}^{(1)}$.

8: Recursively compute values of $N,N^{(1)}$ using equations 4 and 5.

9: Remove $\mathbf{x}$ from $to\\_compute$.

10:end while

In Appendix A, we include a study of the complexity of the propagation
algorithm, as opposed to the "brute force" approach. The results of the
analysis are summarised in Table 1. The main takeaway is that for large $N$
and $3^{M}$, the difference between the multiplicative ("brute force") and
additive ("propagation") complexities becomes considerable.

Algorithm | Complexity
---|---
Brute Force Approach | $O(N\times 3^{M})$
$SR$ Propagation | $O(N+3^{M})$
Table 1: Complexity Table. Brute force approach consists of filtering the
dataset for each intersectional subgroup, and computing success rate on
filtered data. $SR$ Propagation first computes success rates at vertex level,
and then propagates them upwards (Algorithm 1).

##  4 Fairness Propagation

###  4.1 Equality of Outcome

We are given a dataset $\mathcal{D}$ of size $N$ with $M$ binary protected
attributes $p_{1},...,p_{M}$ and one binary label $\mathbf{y}$. Define the
minimum and maximum success at level $K$ as

$SR_{min}(K)=\min_{\mathbf{x}\in\textrm{Level }K}SR(\mathbf{x})$ (6)
$SR_{max}(K)=\max_{\mathbf{x}\in\textrm{Level }K}SR(\mathbf{x})$ (7)

In Theorem 1, we will show that the minimum and maximum success rates must
narrow down as we go up the levels. This result is similar to (Theorem IV.1.
[Foulds et al.(2020b)]), although derived in a different framework.

######  Theorem 1.

$SR_{min}(K)$ is an increasing function of $K$ and $SR_{max}(K)$ is a
decreasing function of $K$. In language, we refer to this result as: "success
rates narrow as we go up the levels".

###### Proof.

Let $\mathbf{x}$ be a non-vertex hypercube (of dimension $K$) and let
$\mathbf{x}_{i}^{(0)}$ and $\mathbf{x}_{i}^{(1)}$ be a partition of it using
the $i^{th}$ star. Recall that we can compute the success rate as

$SR(\mathbf{x})=\frac{N^{(1)}(\mathbf{x})}{N(\mathbf{x})}$ (8)

Using equation 5, this is equivalent to

$SR(\mathbf{x})=\frac{N^{(1)}(\mathbf{x}_{i}^{(0)})+N^{(1)}(\mathbf{x}_{i}^{(1)})}{N(\mathbf{x})}$
(9)

Which can be rewritten as

$SR(\mathbf{x})=\frac{N(\mathbf{x}_{i}^{(0)})}{N(\mathbf{x})}SR(\mathbf{x}_{i}^{(0)})+\frac{N(\mathbf{x}_{i}^{(1)})}{N(\mathbf{x})}SR(\mathbf{x}_{i}^{(1)})$
(10)

Note that equation 10 is a weighed average of the success rates of the two
sub-hypercubes. The weights are given by the respective proportions of
datapoints coming from each of the two sub-hypercubes. A weighed average lies
between the minimum and maximum values we are averaging over. So by looking at
the $i^{th}$ split, we obtain the following bound on the success rate
$SR(\mathbf{x})$.

$\min_{j}SR(\mathbf{x}_{i}^{(j)})\leq
SR(\mathbf{x})\leq\max_{j}SR(\mathbf{x}_{i}^{(j)})$ (11)

If we instead choose to look at all $K$ ways of splitting the hypercube
$\mathbf{x}$, we obtain a better bound.

$\max_{i}\min_{j}SR(\mathbf{x}_{i}^{(j)})\leq
SR(\mathbf{x})\leq\min_{i}\max_{j}SR(\mathbf{x}_{i}^{(j)})$ (12)

Using equation 12, we conclude the proof of the theorem.

$SR_{min}(K)=\min_{\mathbf{x}\in\textrm{Level
}K}SR(\mathbf{x})\geq\min_{\mathbf{x}\in\textrm{Level
}K}\max_{i}\min_{j}SR(\mathbf{x}_{i}^{(j)})\geq\min_{\mathbf{x}\in\textrm{Level
}K-1}SR(\mathbf{x})=SR_{min}(K-1)$ (13)
$SR_{max}(K)=\max_{\mathbf{x}\in\textrm{Level
}K}SR(\mathbf{x})\leq\max_{\mathbf{x}\in\textrm{Level
}K}\min_{i}\max_{j}SR(\mathbf{x}_{i}^{(j)})\leq\max_{\mathbf{x}\in\textrm{Level
}K-1}SR(\mathbf{x})=SR_{max}(K-1)$ (14)

∎

Please note that we didn’t need to use all $K$ ways of splitting (equation 12)
to obtain the above result, picking any one of them (equation 11) would have
sufficed. We choose to include all ways of splitting within the formula,
because it highlights an important property of how the success rates relate
across levels. That is that each hypercube at level $K$ is constrained in $K$
different ways.

If we further define the worst case disparate impact [Feldman et al.(2014)]
and the worst case statistical parity [Feldman et al.(2014)] at level $K$ as

$DI(K)=\frac{SR_{min}(K)}{SR_{max}(K)}$ (15)

and

$SP(K)=SR_{max}(K)-SR_{min}(K)$ (16)

We can prove the following corollary:

######  Corollary 1.1.

$DI(K)$ increases with increasing $K$, and $SP(K)$ decreases with increasing
$K$. In language, we refer to these results as: "Equality of Outcome fairness
propagates up the levels".

###### Proof.

As follows from Theorem 1:

$DI(K)=\frac{SR_{min}(K)}{SR_{max}(K)}\geq\frac{SR_{min}(K-1)}{SR_{max}(K-1)}=DI(K-1)$
(17)

and

$SP(K)=SR_{max}(K)-SR_{min}(K)\leq SR_{max}(K-1)-SR_{min}(K-1)=SP(K-1)$ (18)

∎

###  4.2 Equality of Opportunity

In Appendix B we extend the above analysis to the Equality of Opportunity
setting. We do so by adapting Theorem 1, to show that other fairness measures
also narrow as we progress up the levels such as: accuracy, precision, true
positive rates, false positive rates, true negative rates, false negative
rates [Hardt et al.(2016)]. In language, these results can be summarized as
follows: "Equality of Opportunity fairness also propagates up the levels".

##  5 Intersectional Statistical Parity

In the previous sections, we used a minimum and maximum approach, to get worst
case scenario proofs about the evolution of fairness metrics as we progress up
the levels. In this section, we use a statistical interpretation of our
framework. First, it will be useful to introduce some notation from
probability theory. We let $A_{1},\ldots,A_{M}$ denote the random variables
associated with our $M$ protected attributes and $Y$ be the random variable
associated to the label. Intersectional Statistical Parity (ISP) holds when
there is independence between the protected attributes (including
intersections of an arbitrary number of them) and the output label ([Dwork et
al.(2012), Agarwal et al.(2018), Kearns et al.(2018), Foulds et al.(2020b)]).
In equations,

$P(Y|A_{i_{1}},\ldots,A_{i_{p}})=P(Y)$ (19)

In the presence of ISP, every single hypercube has precisely the same
probability of success $P(Y=1)=p_{tot}$. For each hypercube $\mathbf{x}$, the
success rate $SR(\mathbf{x})$ can therefore be seen as the sample mean of
$N(\mathbf{x})$ draws of a $Bernoulli(p_{tot})$ random variable. Hence, the
mean and variance are given by:

$\mathbb{E}[SR(\mathbf{x})]=p_{tot}$ (20)

and

$Var[SR(\mathbf{x})]=\frac{p_{tot}(1-p_{tot})}{N(\mathbf{x})}$ (21)

We first consider the simplified case where each subgroup on a given level has
an equal number of instances. In this case, the number of datapoints for each
hypercube at level $K$ is exactly $N_{K}=N2^{K-M}$ (Appendix C). In this
scenario, all success rates $SR(\mathbf{x})$ have the same distribution, with
mean and variance given by equations 20 and 21 respectively and
$N(\mathbf{x})=N_{K}$. The theoretical value for the variance (at level $K$)
is then:

$Var_{ISP}(K)=\frac{p_{tot}(1-p_{tot})}{N_{K}}=\frac{2^{M}p_{tot}(1-p_{tot})}{N}\times
2^{-K}$ (22)

Note that the variance decreases exponentially with increasing levels. By
setting $\alpha=\frac{2^{M}p_{tot}(1-p_{tot})}{N}$ and transforming to the log
domain, we get a simpler formulation

$\log Var_{ISP}(K)=\log\alpha-K\log 2$ (23)

This means that, if ISP holds, the log of the variance at level $K$ decreases
linearly with increasing $K$. The slope of the linear curve is $-\log 2$ and
the intercept is $\log\alpha$.

Supposing we are now in possession of data, obtained from an ISP classifier.
We can use the empirical success rates at each level to compute an estimate of
the variance as:

$\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}(SR(\mathbf{x})-\overline{SR}(K))^{2}$ (24)

The above formula is known to be an estimator of the theoretical variance,
under the assumption of independent samples. However, on any level above Level
0, the samples are technically not independent. That is because different
hypercubes share vertices and we find that especially at very high levels, the
approximation of assuming independence breaks down. We mitigate this issue by
using sub-sampling. By sub-sampling our dataset several times, we solve two
problems at once. The first is that we ensure the dataset is balanced, which
is a crucial assumption in the derivations. The second is that we produce more
samples, that are less correlated overall. We call the empirical estimate of
the variance obtained using sub-sampling $Var(K)$. If we subsample our data
$n_{sub}$ times, $Var(K)$ can be defined mathematically as

$Var(K)=\frac{1}{n_{sub}\times
H_{K}}\sum_{i=1}^{n_{sub}}\sum_{\mathbf{x}\in\textrm{Level
}K}(SR_{i}(\mathbf{x})-\overline{SR}(K))^{2}$ (25)

What happens if ISP does not hold? In this case, the set of success rates
$SR(\mathbf{x})$ at each level is made up of copies of random variables with
different means. Since the means are different, we expect the variance to be
greater than the theoretically computed minimum $Var_{ISP}(K)$. We can then
use this theoretical value for the variance as a benchmark, to define a
measure of intersectional bias at each level. We call this family of metrics
$VarRatio(K)$, and define it mathematically as

$VarRatio(K)=\frac{Var(K)}{Var_{ISP}(K)}$ (26)

In this paper, we restrict our focus to the metric obtained at vertex level,
for $K=0$. Once again using $\alpha=\frac{2^{M}p_{tot}(1-p_{tot})}{n_{sub}}$
to simplify the expression, the metric can be written as:

$VarRatio(0)=\frac{Var(0)}{\alpha}$ (27)

There are three cases that can arise:

  1. 1.

$VarRatio(0)\approx 1$ indicating the classifier satisfies (or is close to
satisfying) ISP.

  2. 2.

$VarRatio(0)>1$ indicating the classifier contains some intersectional bias.

  3. 3.

$VarRatio(0)<1$ hinting that the classifier was trained or post-processed with
some bias mitigation procedures.

Please note that if the data is unbalanced and we do not use a sub-sampling
approach, $Var_{ISP}(K)$ still serves as an approximate lower bound for the
expected variance at Level $K$ (Appendix D).

In conclusion, we showed that, under perfect fairness, the variance of the
empirical success rates decreases exponentially across levels. This means that
the log-variance against level curve is linear, and this can be used as a test
for the ISP criterion. These theoretical findings helped us define a family of
metrics $VarRatio(K)$ which capture intersectional bias, and because of how
they are constructed, we expect them to be less affected by small sample
variance than other approaches within the literature.

##  6 Experiments

###  6.1 Synthetic Data

Figure 3: Success Rate Distributions. Estimated empirical distributions of the
success rates over all the hypercubes at levels 1, 4 and 7 for Experiment 1.
The distributions get narrower around $SR_{tot}=0.5$ as we travel up the
levels.

In order to verify our theoretical results, we run experiments with synthetic
data. We create a function to generate random datasets, with $M=10$ protected
attributes, and one label. Each of the $2^{10}=1024$ smallest intersectional
groups (vertices), $\mathbf{v}$, contains $N(\mathbf{v})=200R$ instances,
where $R$ is a randomly selected integer between 1 and 10. Each vertex also
has an associated probability of success $p(\mathbf{v})$, which we use to
sample its binary label data from a $Bernoulli(p(\mathbf{v}))$ distribution.
We sub-sample the dataset $n_{repeats}=20$ times so as to have all vertices
contain precisely $n_{sub}=100$ datapoints. The sub-sampling is crucial for
two reasons. The first is that it allows us to generate a larger number of
samples, with minimal amounts of correlation. The second is that it ensures
all hypercubes have equal number of datapoints. Both of these conditions were
assumptions in our theoretical derivations.

In this first example, we will consider a fair dataset, where
$p(\mathbf{v})=0.5$ for all vertices. Since the output label is here
independent of all the protected attributes, this example satisfies
Intersectional Statistical Parity. We run the Propagation Algorithm 1 to
compute the empirical success rates over all the possible intersectional
groups, that is, all the hypercubes in our geometrical model. Fig. 3 shows the
distribution of the empirical success rates at different levels. As predicted
by our theoretical results, the distribution gets narrower around
$SR_{tot}=0.5$ as we go up the levels. Fig. 4(a) shows how the minimum and
maximum values evolve as we progress up the levels. The maximum value
decreases and minimum value increases, as shown in Theorem 1. Fig. 4(b) and
Fig. 4(c) show plots of $Var(K)$ and $\log Var(K)$ as a function of $K$. Both
curves match the theoretical results precisely, except for small numerical
approximations.

(a) minimum and maximum

(b) variance

(c) log variance

Figure 4: Experiment 1. (a) Evolution of the minimum and maximum empirical
success rates across levels. As expected from the theoretical results, these
values tend to 0.5. (b) Evolution of the Variance across levels. The variance
decreases exponentially, as predicted by our theoretical results. (c)
Evolution of the log-Variance across levels. The log-variance linearly, as
predicted by our theoretical results.

In our second experiment, we want to analyse how these results change in the
presence of bias. We model bias by allowing each vertex $\mathbf{v}$ to have a
different probability of success $p(\mathbf{v})$. Specifically, we select a
set of $100$ vertices to have probability of success
$p(\textbf{v})=0.5-\delta$, and a separate set of $100$ vertices to have
probability of success $p(\textbf{v})=0.5+\delta$. We set our parameter
$\delta$ to vary in the interval $\\{0,0.1,0.2,0.3,0.4\\}$. When $\delta=0$,
we retrieve the case of perfect fairness from the previous example. Otherwise,
ISP will not be satisfied, and indeed our dataset will get less and less fair
as $\delta$ increases. The results show that, as expected, the minimum and
maximum success rates narrow down as fairness increases, that is, for lower
values of $\delta$ (Fig. 5(a)). We also show that, just as our theoretical
results had predicted, the variance of the empirical success rates increases
with higher levels of bias (Fig. 5(b)). Accordingly, the evolution of the log-
variance across levels is not linear but concave for higher levels of
$\delta$, another indicator that ISP is not satisfied. Finally, we calculate
our metric $VarRatio(0)$ (Table 2), and find that it increases for increasing
values of $\delta$, which indicate larger bias.

(a) minimum and maximum

(b) variance

(c) log variance

Figure 5: Experiment 2. (a) Evolution of the minimum and maximum empirical success rates across levels. As expected from the theoretical results, these values tend to 0.5. These values narrow down for smaller $\delta$, as bias decreases. (b) Evolution of the Variance across levels. The variance increases in the presence of bias. (c) Evolution of the log-Variance across levels. The log-variance curve departs from the theoretical value as we increase $\delta$. | $\delta=0$ | $\delta=0.1$ | $\delta=0.2$ | $\delta=0.3$ | $\delta=0.4$
---|---|---|---|---|---
Level 0 | $0.98$ | $1.73$ | $4.19$ | $7.96$ | $13.34$
Table 2: Metrics Table. $VarRatio(0)$ calculated for experiment 2.

###  6.2 Real Data Experiment

For the real data test, we chose to use the Adult dataset. The Adult dataset
is one of the most commonly used datasets in fair-AI research. That is because
it has many attributes that can be considered sensitive (sex, race, age, etc)
and the task is that of binary classification. The dataset consists of
$N=48842$ instances with 14 features and one label feature. The usual task
given on this dataset is the prediction of the binary attribute of salary
being above 50K from the other features.

For the purposes of our experiments, we performed some simple prepossessing on
the data. The first step is we keep only 5 of the 15 provided columns. That is
$M=4$ protected attributes (sex, race, age, marital-status) and the binary
label (class). We also performed some grouping on the attributes age, race and
marital-status. For age, we binarize the data so that instances with age $\geq
40$ are set to 1 and all others are set to 0. For race we grouped Black,
Asian-Pac-Islander and Amer-Indian-Eskimo with the Other group. For marital-
status we grouped the 7 provided statuses (Married-civ-spouse, Never-married,
Divorced, Separated, Widowed, Married-spouse-absent, Married-AF-spouse) into
Married and Not-Married. The reason we had to perform grouping on some
attributes is to match the setting of the theoretical work (all protected
attributes binary). Furthermore the grouping ensures all intersectional
subgroups contain sufficient data. For the average and minimum number of
datapoints over all hypercubes per level, see Table 3.

| Level 0 | Level 1 | Level 2 | Level 3 | Level 4
---|---|---|---|---|---
Average Number | 3052 | 6105 | 12210 | 24421 | 48842
Minimum Number | 228 | 521 | 2511 | 7080 | 48842
Table 3: Adult Dataset. Minimum number and average number of datapoints at
each level $K=0,\ldots,4$.

(a) minimum and maximum

(b) variance

(c) log variance

Figure 6: Adult Dataset Experiment. (a) Evolution of the minimum and maximum
empirical success rates across levels. (b) Evolution of the Variance across
levels. (c) Evolution of the log Variance across levels.

The following results are obtained by sub-sampling the dataset
$n_{repeats}=20$ times so that each vertex contains exactly $n_{sub}=100$
samples. In Fig. 6(a) the minimum and maximum success rates are displayed as a
function of level. As we have shown, the success rates must narrow with
increasing levels, although they narrow more slowly for the Adult dataset.
Fig. 6(b) and Fig. 6(c) show the evolution of the variance and log-variance
across levels. Observe the big difference in variance between the
theoretically computed values (obtained under ISP condition) and Adult dataset
values. All results indicate the presence of intersectional bias within the
data.

##  7 Fractal Fairness

Intersectional fairness in the context of AI typically focuses on splitting
groups of people into smaller identity categories by intersecting multiple
protected attributes. Although it is commendable that the issue of
intersectionality is being tackled by the community, we feel that the current
approaches have some limitations. For example, it is not clear how many
protected attributes one should consider, which protected attributes one
should consider and how to select the level of granularity for the analysis
[Kong(2022)].

On the other hand, we sought a framework that would allow us to think of
intersectional fairness as a whole: one complex system involving all possible
intersectional groups at all levels of description. We turned to geometry, and
particularly fractals, for inspiration. Fractals are geometrical objects that,
informally, can be broken down into smaller objects that resemble the original
shape [Mandelbrot and Mandelbrot(1982)]. Although our framework is not
mathematically a fractal, we identified a geometrical structure where each
hypercube separates into further instances of hypercubes. This provided us
with a mathematical model to study a nested and recursive notion of fairness.
In fractals, patterns at a small scale repeat at a larger scale. Similarly,
our framework asks that our data is fair for any possible subgroup of the
population, no matter which or how many attributes are intersected.

Furthermore, the geometrical nature of our setting allows us to examine how
fairness propagates and scales. We found that fairness necessarily propagates
up the levels, while bias propagates down. This suggests that we should, first
and foremost, think of fairness in a bottom-up and local way. It also
indicates that we cannot think of individual parts of a system in isolation
from the whole. We propose that the trustworthiness of an AI system cannot
thus be enforced in a top-down way, or by separating individual components
from the larger system. Instead, we argue for a bottom-up emergent and
holistic approach to trustworthy AI.

##  8 Discussion

In this paper, we introduced a geometrical setting for studying the fairness
properties of all possible intersectional subgroups together, in a unified
framework. There are multiple advantages to this setting. Firstly, it allows
for a quicker computation of all the success rates (or other metrics if
needed), using a dynamic programming approach. Secondly, it reveals the inter-
connectivity between the fairness properties of groups at different levels. In
particular, we prove that success rates must narrow as we go up the levels. We
use that to prove worst case disparate impact increases with increasing
levels, and worst case statistical parity decreases with increasing levels. We
summarise these results as "Equality of Outcome fairness propagates up the
levels". We then extend our results to the Equality of Opportunity setting. In
particular, we show that accuracy, precision, true positive rates, false
positive rates, true negative rates and false negative rates all narrow down
as we progress up the levels. We summarise these results as "Equality of
Opportunity fairness also propagates up the levels". In the future, it might
be interesting to derive a mathematical description of all metrics for which
this method of proof applies.

Furthermore, we suggest that the variance of the empirically computed success
rates on a given level can be used as a fairness measure. We prove that under
perfect fairness (Intersectional Statistical Parity), the variance follows a
simple exponential scaling law. Using this theoretical value as benchmark, we
define a family of metrics which capture the intersectional bias on each
level. Future work will definitely focus on examining these metrics further
and studying the behaviour of the variance and log-variance curves more
closely in different settings. One of the hopes is that, the shape of the log-
variance curve can help in detecting but also in locating the source of bias.
For instance if bias was injected into the system using combinations of two
protected attributes (Level $M-2$), we might see a bend in the log-variance
curve at around those values of $K$. This could help answer questions related
to the sources of bias and identifying gerrymandering. Alternatively, the
variance scaling law could be used to devise similarly structured statistical
tests.

One of the advantages of our framework is that it allows us to analyse the
whole system at once. Bias is not exclusively measured for given groups at a
prescribed intersectional level, but instead its evolution is examined across
all levels. However, most of our analyses aggregate over groups. This issue
could be mitigated, for example, by using our framework in conjunction with
other bias measures. In the future, it would be interesting to employ our
propagation algorithm for more detailed modelling. For example, we could
analyse how the success rates for two specific attributes interact and spread
across levels. Otherwise, we could study the evolution of the fairness metrics
for one specific attribute (e.g. gender) across levels or even specific paths
within the hypercube. This would provide us with further insight into how bias
affects that specific attribute. More broadly, our way of framing the problem
of intersectionality can reveal the interconnections among intersectional
subgroups, which could be key to developing appropriate metrics and mitigation
techniques.

## References

  * [1]
  * [Agarwal et al.(2018)] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. 2018\.  A reductions approach to fair classification. In _International Conference on Machine Learning_. PMLR, 60–69.
  * [Agarwal et al.(2019)] Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. 2019\.  Fair Regression: Quantitative Definitions and Reduction-based Algorithms.  _36th International Conference on Machine Learning, ICML 2019_ 2019-June (5 2019), 166–183.  https://doi.org/10.48550/arxiv.1905.12843
  * [Buolamwini and Gebru(2018)] Joy Buolamwini and Timnit Gebru. 2018.  Gender shades: Intersectional accuracy disparities in commercial gender classification. In _Conference on fairness, accountability and transparency_. PMLR, 77–91.
  * [Crenshaw(2013a)] Kimberlé Crenshaw. 2013a.  Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics.  In _Feminist legal theories_. Routledge, 23–51.
  * [Crenshaw(2013b)] Kimberlé Williams Crenshaw. 2013b.  Mapping the margins: Intersectionality, identity politics, and violence against women of color.  In _The public nature of private violence_. Routledge, 93–118.
  * [Dastin(2018)] Jeffrey Dastin. 2018\.  Amazon scraps secret AI recruiting tool that showed bias against women.  In _Ethics of data and analytics_. Auerbach Publications, 296–299.
  * [Dwork et al.(2012)] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012.  Fairness through awareness. In _Proceedings of the 3rd innovations in theoretical computer science conference_. 214–226.
  * [Feldman et al.(2014)] Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2014.  Certifying and removing disparate impact.  _Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ 2015-August (12 2014), 259–268.  https://doi.org/10.48550/arxiv.1412.3756
  * [Feldman et al.(2015)] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015\.  Certifying and removing disparate impact. In _proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining_. 259–268.
  * [Foulds et al.(2020a)] James R Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. 2020a.  Bayesian Modeling of Intersectional Fairness: The Variance of Bias. In _Proceedings of the 2020 SIAM International Conference on Data Mining_. SIAM, 424–432.
  * [Foulds et al.(2020b)] James R. Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. 2020b.  An intersectional definition of fairness.  _Proceedings - International Conference on Data Engineering_ 2020-April (4 2020), 1918–1921.  https://doi.org/10.1109/ICDE48307.2020.00203
  * [Hardt et al.(2016)] Moritz Hardt, Eric Price, and Nati Srebro. 2016.  Equality of opportunity in supervised learning.  _Advances in neural information processing systems_ 29 (2016).
  * [Jin et al.(2020)] Zhongjun Jin, Mengjing Xu, Chenkai Sun, Abolfazl Asudeh, and HV Jagadish. 2020.  Mithracoverage: a system for investigating population bias for intersectional fairness. In _Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data_. 2721–2724.
  * [Kearns et al.(2018)] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018.  Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness.  (2018).
  * [Kong(2022)] Youjin Kong. 2022\.  Are "Intersectionally Fair" AI Algorithms Really Fair to Women of Color? A Philosophical Analysis.  _ACM International Conference Proceeding Series_ (6 2022), 485–494.  https://doi.org/10.1145/3531146.3533114
  * [Li and Vasconcelos(2019)] Yi Li and Nuno Vasconcelos. 2019.  Repair: Removing representation bias by dataset resampling. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 9572–9581.
  * [Mandelbrot and Mandelbrot(1982)] Benoit B Mandelbrot and Benoit B Mandelbrot. 1982.  _The fractal geometry of nature_. Vol. 1.  WH freeman New York.
  * [Mehrabi et al.(2021)] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021\.  A survey on bias and fairness in machine learning.  _ACM Computing Surveys (CSUR)_ 54, 6 (2021), 1–35.
  * [Morina et al.(2019)] Giulio Morina, Viktoriia Oliinyk, Julian Waton, Ines Marusic, and Konstantinos Georgatzis. 2019\.  Auditing and Achieving Intersectional Fairness in Classification Problems.  (11 2019).  https://doi.org/10.48550/arxiv.1911.01468
  * [Telford(2019)] Taylor Telford. 2019\.  Apple Card algorithm sparks gender bias allegations against Goldman Sachs.  _Washington Post_ 11 (2019).
  * [Yang et al.(2020)] Forest Yang, Moustapha Cisse, Google Research, and Accra Sanmi Koyejo. 2020.  Fairness with Overlapping Groups.  (2020).

##  Appendix A Complexity of Success Rate Propagation Algorithm

We first study the complexity of the "brute force" approach to computing all
success rates. Suppose we filter the dataset by the appropriate intersectional
group and then compute the success rates on the filtered data for each of the
$3^{M}$ intersections. For each of the $3^{M}$ intersections, we are looping
over $N$ instances and for each instance we must compare arrays of length $M$.
So this method would have time complexity $O(N\times M\times 3^{M})$.

As we have seen, we can compute the quantities at vertex-level in $O(N\times
M)$ time. What is the time complexity of the propagation algorithm? To compute
the complexity of the propagation algorithm, we consider the hypercube graph
$G$ (see Fig. 2). Each hypercube at level $K$ has $K$ stars, so it is
connected with $2K$ hypercubes one level below. Recall that level $K$ consists
of $H_{K}=\binom{M}{K}2^{M-K}$ hypercubes. So the total number of edges $E(G)$
in $G$ can be written as a sum over levels $K\in\\{1,\ldots,M\\}$.

$E(G)=\sum_{K=1}^{M}2K\binom{M}{K}2^{M-K}\leq\sum_{K=0}^{M}2M\binom{M}{K}2^{M-K}=2M\times
3^{M}$ (28)

Since our algorithm functions with recursion on the graph $G$, by reusing
values that have already been computed, it would never cross the same edge
twice. So the number of edges in $G$ is a strict upper bound on the time taken
by the algorithm. So the time complexity of the dynamic programming approach
is upper bounded by $O(N\times M+M\times 3^{M})$. Assuming $M$ is small
compared to $N$ and $3^{M}$ (and therefore modelled as a constant), the
complexities of both approaches are summarized in Table 1.

##  Appendix B Extension to Equality of Opportunity

###  B.1 Accuracy

Until now we have only been working in an Equality of Outcome fairness
framework. Meaning that we are seeking to equalize the outcomes of our model
for different subgroups of the population. There are also Equality of
Opportunity fairness notions, that seek to equalize the performance of our
model on different subgroups. There is a quick way to translate our findings
from an Equality of Outcome framework to an Equality of Opportunity framework.
Suppose that we are given the true labels as well as the predicted labels.
Name these vectors $\mathbf{y}_{true}$ and $\mathbf{y}_{pred}$ respectively.
Consider the new vector

$\mathbf{y}_{correct}=\mathbf{y}_{true}\land\mathbf{y}_{pred}$ (29)

Suppose we use Theorem 1 with $\mathbf{y}_{correct}$ instead of
$\mathbf{y}_{pred}$. We can show that the success rate of
$\mathbf{y}_{correct}$ narrows as we go up the levels. However, the success
rate of $\mathbf{y}_{correct}$ is precisely the accuracy of
$\mathbf{y}_{true}$, $\mathbf{y}_{pred}$. So we have shown that accuracy also
narrows as we go up the levels. If we define the worst case accuracy ratio and
worst case accuracy difference at level $K$ as

$Acc_{ratio}(K)=\frac{Acc_{min}(K)}{Acc_{max}(K)}$ (30)
$Acc_{diff}(K)=Acc_{max}(K)-Acc_{min}(K)$ (31)

By using Corollary 2, we can show that $Acc_{ratio}(K)$ increases and
$Acc_{diff}(K)$ decreases with increasing $K$.

###  B.2 Other Metrics

In the previous section, we showed how we can extend the original result about
success rates to the analogous result about accuracy. The method we used to do
so can be framed more generally. In particular, we can show analogous results
for any metric that can be written as a success rate of a binary vector
$\mathbf{y}_{new}$ obtained from $\mathbf{y}_{true}$, $\mathbf{y}_{pred}$. We
also allow for filtering the dataset if convenient in defining the metric (as
shown in the next example).

Suppose we wish to prove that the worst case true positive rates ($TPR$)
narrow as we go up the levels. We first filter our dataset $\mathcal{D}$ to
contain only rows where $\mathbf{y}_{true}=1$, we temporarily discard other
rows. Then create a new binary vector $\mathbf{y}_{truepos}$ which is an
indicator on true positives (1 if instance is a true positive, 0 otherwise).
Observe that the success rate of this new vector is precisely the true
positive rate of the original predictions and true labels. In equations

$TPR(\mathbf{x})=\frac{\textrm{\\# $true\ positives$}(\mathbf{x})}{\textrm{\\#
$true$}(\mathbf{x})}=\frac{N^{(1)}(\mathbf{x})}{N(\mathbf{x})}=SR(\mathbf{x})$
(32)

By using Theorem 1 with this new dataset and new binary vector, we show true
positive rates also narrow as we go up the levels. Using the same approach, we
can also show that false positive rates, true negative rates, false negative
rates and precision all narrow as we go up the levels.

##  Appendix C Average number of datapoints on hypercube at Level $K$

We claim the following formula for the average number of datapoints on a
hypercube at level $K$:

$N_{K}=\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}N(\mathbf{x})=N2^{M-K}$ (33)

###### Proof.

For $K=0$, the formula holds

$N_{0}=\frac{1}{2^{M}}\sum_{\mathbf{x}\in\textrm{Level
}0}N(\mathbf{x})=\frac{N}{2^{M}}$ (34)

For any level $K>0$,

$N_{K}=\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}N(\mathbf{x})=\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}\frac{1}{K}\sum_{i}\sum_{j}N(\mathbf{x}_{i}^{(j)})$ (35)

Each hypercube at level $K-1$ appears precisely $M-K+1$ times in the triple
sum, so we can rewrite it as

$N_{K}=\frac{1}{H_{K}}\frac{M-K+1}{K}\sum_{\mathbf{x}\in\textrm{Level
}K-1}N(\mathbf{x})$ (36)

and since, $H_{K}=\binom{M}{K}2^{M-K}$, we have

$\frac{1}{H_{K}}\frac{M-K+1}{K}=\frac{K!(M-K)!}{M!}2^{K-M}\frac{M-K+1}{K}=2\frac{(K-1)!(M-K+1)!}{M!}2^{K-1-M}=2\frac{1}{H_{K-1}}$
(37)

Therefore

$N_{K}=2\frac{1}{H_{K-1}}\sum_{\mathbf{x}\in\textrm{Level
}K-1}N(\mathbf{x})=2N_{K-1}$ (38)

Which concludes the proof by induction. ∎

##  Appendix D Variance of empirical success rates at Level $K$ under ISP
assumption

We claimed that under ISP (and uncorrelated samples), the expected empirical
variance can be lower bounded by
$\frac{H_{K}-1}{H_{K}}\frac{p_{tot}(1-p_{tot})}{N_{K}}$. The proof is as
follows

###### Proof.

We make 2 assumptions (A1, A2)

  1. 1.

The classifier generating the label data is ISP.

  2. 2.

The samples $SR(\mathbf{x}),\mathbf{x}\in\textrm{Level }K$ are uncorrelated.

$\mathbb{E}[Var(K)]=\mathbb{E}\left[\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}SR(\mathbf{x})^{2}-\left(\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}SR(\mathbf{x})\right)^{2}\right]$ (39)

Expand the sum

$=\mathbb{E}\left[\frac{1}{H_{K}}\sum_{\mathbf{x}}SR(\mathbf{x})^{2}-\frac{1}{H_{K}^{2}}\sum_{\mathbf{x},\mathbf{y}}SR(\mathbf{x})SR(\mathbf{y})\right]$
(40)

Remove the cross terms that amount to squares, and aggregate them in the first
sum

$=\mathbb{E}\left[\left(\frac{1}{H_{K}}-\frac{1}{H_{K}^{2}}\right)\sum_{\mathbf{x}}SR(\mathbf{x})^{2}-\frac{1}{H_{K}^{2}}\sum_{\mathbf{x}\neq\mathbf{y}}SR(\mathbf{x})SR(\mathbf{y})\right]$
(41)

Now we use linearity of expectation and both assumptions A1, and A2. A1 is
used to replace expectations with their theoretical value $p_{tot}$. A2 is
used to transform expectations of products into products of expectations.

$=\left(\frac{1}{H_{K}}-\frac{1}{H_{K}^{2}}\right)\sum_{\mathbf{x}}\mathbb{E}[SR(\mathbf{x})^{2}]-\frac{1}{H_{K}^{2}}\sum_{\mathbf{x}\neq\mathbf{y}}p_{tot}^{2}$
(42)
$=\frac{H_{K}-1}{H_{K}^{2}}\sum_{\mathbf{x}}\mathbb{E}[SR(\mathbf{x})^{2}]-\frac{H_{K}-1}{H_{K}}p_{tot}^{2}$
(43)
$=\frac{H_{K}-1}{H_{K}^{2}}\sum_{\mathbf{x}}\mathbb{E}[SR(\mathbf{x})^{2}]-\mathbb{E}[SR(\mathbf{x})]^{2}$
(44) $=\frac{H_{K}-1}{H_{K}^{2}}\sum_{\mathbf{x}}Var[SR(\mathbf{x})]$ (45)
$=\frac{H_{K}-1}{H_{K}^{2}}\sum_{\mathbf{x}}\frac{p_{tot}(1-p_{tot})}{N(\mathbf{x})}$
(46)

Using Jensen’s inequality applied to concave function $1/X$,

$\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}\frac{1}{N(\mathbf{x})}\geq\frac{1}{\frac{1}{H_{K}}\sum_{\mathbf{x}\in\textrm{Level
}K}N(\mathbf{x})}=\frac{1}{N_{K}}$ (47)

So we obtain

$\mathbb{E}[Var(K)]\geq\frac{H_{K}-1}{H_{K}}\frac{p_{tot}(1-p_{tot})}{N_{K}}$
(48)

And the bound is tight iff all sample sizes are the same on the given level. ∎

##  Appendix E Metrics for Experiment 2 measured across all levels

| $\delta=0$ | $\delta=0.1$ | $\delta=0.2$ | $\delta=0.3$ | $\delta=0.4$
---|---|---|---|---|---
Level 0 | 0.989688 | 1.735759 | 4.191394 | 7.966448 | 13.347224
Level 1 | 0.993014 | 2.219624 | 6.220185 | 12.409443 | 21.233369
Level 2 | 0.995939 | 3.001205 | 9.507072 | 19.598287 | 34.003209
Level 3 | 1.000630 | 4.251804 | 14.776882 | 31.110746 | 54.461175
Level 4 | 1.009613 | 6.221990 | 23.086325 | 49.241288 | 86.686218
Level 5 | 1.024945 | 9.240775 | 35.823894 | 77.001449 | 136.032915
Level 6 | 1.046565 | 13.636296 | 54.377402 | 117.403449 | 207.857395
Level 7 | 1.069158 | 19.405291 | 78.747043 | 170.448156 | 302.164360
Level 8 | 1.074935 | 25.157049 | 103.123825 | 223.490712 | 396.491092
Level 9 | 1.013403 | 25.073643 | 103.158698 | 223.463451 | 396.570403
Table 4: Experiment 2 Metrics Table. We show the values of the metric
$VarRatio(K)$ for $K=0,\ldots,9$ and parameter $\delta=0,\ldots,0.4$


# Combining Humor and Sarcasm for Improving Political Parody Detection

Xiao Aoα Danae Sánchez Villegasα Daniel Preoţiuc-Pietroβ Nikolaos Aletrasα
α Computer Science Department, University of Sheffield, UK
β Bloomberg
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Parody is a figurative device used for mimicking entities for comedic or
critical purposes. Parody is intentionally humorous and often involves
sarcasm. This paper explores jointly modelling these figurative tropes with
the goal of improving performance of political parody detection in tweets. To
this end, we present a _multi-encoder_ model that combines three parallel
encoders to enrich parody-specific representations with humor and sarcasm
information. Experiments on a publicly available data set of political parody
tweets demonstrate that our approach outperforms previous state-of-the-art
methods.111Code is available here
https://github.com/iamoscar1/Multi_Encoder_Model_for_Political_Parody_Prediction

##  1 Introduction

Parody is a figurative device which imitates entities such as politicians and
celebrities by copying their particular style or a situation where the entity
was involved (Rose, 1993). It is an intrinsic part of social media as a
relatively new comedic form Vis (2013). A very popular type of parody is
political parody, which is used to express political opposition and civic
engagement (Davis et al., 2018).

One of the hallmarks of parody expression is the deployment of other
figurative devices, such as humor and sarcasm, as emphasized on studies of
parody in linguistics (Haiman et al., 1998; Highfield, 2016). For example, in
Table 4 the text expresses sarcasm about Myspace222https://myspace.com being a
‘winning technology’, while mocking the fact that three more popular social
media sites were unavailable. This example also highlights the similarities
between parody and real tweets, which may pose issues to misinformation
classification systems Mu and Aletras (2020).

Twitter Handle |  @Queen_UK
---|---
Parody tweet |  Boris Johnson on the phone. Very smug that #myspace hasn’t gone down. Says he’s always backed winning technologies #whatsappdown #instagramdown #FacebookIsDown
Table 1: Example of a _parody_
tweet444https://twitter.com/Queen_UK/status/1445103605355323393?t=FGMNsMVFF_G2tABYxFmkFw&s=07
by the Twitter handle @Queen_UK. Humor and sarcasm are expressed
simultaneously.

These figurative devices have so far been studied in isolation to parody.
Previous work on modeling humor in computational linguistics has focused on
identifying jokes, i.e., short comedic passages that end with a hilarious line
(Hetzron, 1991), based on linguistic features (Taylor and Mazlack, 2004;
Purandare and Litman, 2006; Kiddon and Brun, 2011) and deep learning
techniques (Chen and Soo, 2018; Weller and Seppi, 2019; Annamoradnejad and
Zoghi, 2020). Similarly, computational approaches for modeling sarcasm (i.e.,
a form of verbal irony used to mock or convey content) in texts have been
explored (Davidov et al., 2010; González-Ibáñez et al., 2011; Liebrecht et
al., 2013; Rajadesingan et al., 2015; Ghosh et al., 2020, 2021), including
multi-modal utterances, i.e. texts, images, and videos (Cai et al., 2019;
Castro et al., 2019; Oprea and Magdy, 2020). Recently, parody has been studied
with natural language processing (NLP) methods by Maronikolakis et al. (2020)
who introduced a data set of political parody accounts. Their method for
automatic recognition of posts shared by political parody accounts on Twitter
is solely based on vanilla transformer models.

In this paper, we hypothesize that humor and sarcasm information could guide
parody specific text encoders towards detecting nuances of figurative
language. For this purpose, we propose a _multi-encoder_ model (§2) consisting
of three parallel encoders that are subsequently fused for parody
classification. The first encoder learns parody specific information
subsequently enhanced using the representations learned by a humor and sarcasm
encoder respectively.

Our contributions are: (1) new state-of-the-art results on political parody
detection in Twitter, consistently improving predictive performance over
previous work by Maronikolakis et al. (2020); and (2) insights on the
limitations of neural models in capturing various linguistic characteristics
of parody from extensive qualitative and quantitative analyses.

##  2 Multi-Encoder Model for Political Parody Prediction

Maronikolakis et al. (2020) define political parody prediction as a binary
classification task where a social media post $T$, consisting of a sequence of
tokens $T=\\{t_{1},...,t_{n}\\}$, is classified as real or parody. Real posts
have been authored by actual politicians (e.g., realDonaldTrump) while parody
posts come from their corresponding parody accounts (e.g., realDonaldTrFan).

Parody tends to express complex tangled semantics of both humor and sarcasm
simultaneously (Haiman et al., 1998; Highfield, 2016). To better exploit this
characteristic of parody, we propose a _multi-encoder_ model that consists of
three parallel encoders, a feature-fusion layer and a parody classification
layer depicted in Fig.1.555Early experiments with multi-task learning did not
result in improved performance. The results of these experiments can be found
in Appendix A..

Figure 1: The structure of our _multi-encoder_ model for combining humor and
sarcasm information for political parody prediction.

###  2.1 Text Encoders

#### Parody

As a task-specific parody encoder, we use the vanilla pretrained BERTweet
(Nguyen et al., 2020), a BERT Devlin et al. (2019) based model pre-trained on
a corpus of English Tweets and fine-tuned on the parody data set (§3.1).

#### Humor

To capture humor specific characteristics in social media text, we use the
data set introduced by Annamoradnejad and Zoghi (2020) which contains humorous
and non-humorous short texts collected from Reddit and Huffington Post. First,
we adapt BERTweet using domain-adaptive pre-training (Sun et al., 2020a;
Gururangan et al., 2020) on 10,000 randomly selected humor-only short texts
with masked language modeling. Subsequently, we use a continual learning
strategy (Li and Hoiem, 2018; Sun et al., 2020b) to gradually learn humor-
specific properties by further fine-tuning BERTweet on a humor classification
task (i.e., predicting whether a text is humorous or not) by using 40,000
randomly selected humorous and non-humorous short texts from the humor corpus
described above (see Figure 2).

Figure 2: Humor Encoder. Figure 3: Sarcasm Encoder.

#### Sarcasm

Similar to humor, we extract sarcasm-related semantic information from a post
$T$ by using sarcasm annotated data sets from Oprea and Magdy (2020) and
Rajadesingan et al. (2015). The first data set consists of 777 and 3,707
sarcasm and non-sarcasm posts from Twitter and the second data set consists of
9,104 sarcasm and more than 90,000 non-sarcasm posts from Twitter. We first
perform domain-adaptive pre-training of BERTweet on all sarcastic posts with
masked language modeling. Then, we fine-tune the model on a sarcasm
classification task, similar to the humor encoder (see Figure 3). For the
fine-tuning step, we use the 9,881 sarcastic tweets and 10,000 randomly
sampled non-sarcasm tweets from the two data sets (i.e., 3,707 from the first
and 6,293 from the second).

We compute parody $f^{t}$, humor $f^{h}$, and sarcasm $f^{s}$ representations
by extracting the ‘classification’ [CLS] token from each encoder respectively,
where $f\in\mathbf{R}^{768}$.

###  2.2 Combining Encoders

We explore three approaches to combine $f^{t}$, $f^{h}$, and $f^{s}$
representations.

#### Concatenation

First, the three text representations are simply concatenated to form a
combined representation $f\in\mathbf{R}^{768\times 3}$.

#### Self-Attention

We also use a 4-head self-attention666Early experimentation with larger
attention heads did not improve results in the dev set. mechanism Vaswani et
al. (2017) on $f^{t},f^{h},f^{s}$. The goal is to find correlations between
representations and learn the contribution of each encoder in the final
representation.

#### Max-Pooling

Finally, we perform a max-pooling operation on each dimension of $f^{t}$,
$f^{h}$, $f^{s}$ to obtain a representation $f\in\mathbf{R}^{768}$. The aim is
to use the most dominant features learned by each encoder.

###  2.3 Classification

Finally, we pass the combined representation $f$ to a classification layer
with a sigmoid activation function for predicting whether a post is a parody
or not. Three encoders are fine-tuned simultaneously on the parody data set
(§3.1).777Early experimentation with humor and sarcasm encoders frozen during
the fine-tuning process did not show any performance improvement.

##  3 Experimental Setup

###  3.1 Data

We use the data set introduced by Maronikolakis et al. (2020) which contains
131,666 tweets written in English, with 65,956 tweets from political parody
accounts and 65,710 tweets posted by real politician accounts. The data set is
publicly available888https://archive.org/details/parody_data_acl20 and allows
us to compare our results to state-of-the-art parody detection methods.

We use the three data splits provided: (i) _Person Split_ , each split (train,
dev, test) contains tweets from different real – _parody_ account pairs; (ii)
_Gender Split_ , two different splits based on the gender of the politicians
(i.e., female accounts in train/dev and male in test, and male accounts in
train/dev and female in test); _Location Split_ , data is split according to
the location of the politicians in three groups (US, UK, Rest of the World or
RoW). Each group is assigned to the test set and the other two groups to the
train and dev sets.

###  3.2 Baselines

We compare our _multi-encoder_ models with transformers for parody detection
(Maronikolakis et al., 2020): BERT (Devlin et al., 2019) and RoBERTa (Liu et
al., 2019). Also, we compare our models to BERTweet (Nguyen et al., 2020).

###  3.3 Implementation details

#### Humor Encoder

For adaptive pre-training, the batch-size is set to $16$ and the number of
training epochs is set to $3$ with a learning rate of $2e^{-5}$. For humor
classification, we use batch size of $128$ and the number of epochs is set to
$2$ with a learning rate of $3e^{-5}$.

#### Sarcasm Encoder

We pretrain using a batch-size of $16$ over $5$ epochs with a learning rate of
$2e^{-5}$. For fine-tuning on a sarcasm classification task, we use the
$9,881$ sarcasm tweets and $10,000$ randomly sampled non-sarcasm tweets from
the two data sets (i.e., $3,707$ from the first and $6,293$ from the second)
using the same hyperparameters to the humor-specific encoder.

#### Multi-encoder

For the complete _multi-encoder_ model, we use a batch size of $128$ and the
learning rate is set to $2e^{-5}$. The entire model is fine-tuned for $2$
epochs.

###  3.4 Evaluation

We evaluate the performance of all models using F1 score as Maronikolakis et
al. (2020). Results are obtained over $3$ runs using different random seeds
reporting average and standard deviation.

##  4 Results

###  4.1 Predictive Performance

Table 2 shows the results for parody detection on the _Person Split_. We
observe that BERTweet has the best performance (F1: $90.72$) among
transformer-based models (BERT, RoBERTa, BERTweet), outperforming previous
state-of-the-art by Maronikolakis et al. (2020). This is due to the fact that
BERTweet has been specifically pre-trained on Twitter text. Similar behavior
is observed on the _Gender_ and _Location_ splits (see Table 3 and 4
respectively).

Person
---
Model | F1
Single-Encoder
BERT∗∗ | $87.65\pm 0.18$
RoBERTa∗∗ | $89.66\pm 0.33$
BERTweet | $90.72\pm 0.31$
Multi-encoder (Ours)
Concatenation | $88.99\pm 0.17$
Self-Attention | $\textbf{91.19}\pm\textbf{0.31}$
Max-Pooling | $91.05\pm 0.30$
Table 2: F1-scores for parody detection on the _Person Split_. ∗∗ Results from
Maronikolakis et al. (2020). Best results are in bold. Gender
---
Model | M$\to$F | F$\to$M
Single-Encoder
BERT∗∗ | $85.85\pm 0.28$ | $84.40\pm 0.35$
RoBERTa∗∗ | $87.11\pm 0.31$ | $84.87\pm 0.38$
BERTweet | $88.01\pm 0.29$ | $85.57\pm 0.27$
Multi-encoder (Ours)
Concatenation | $86.84\pm 0.15$ | $84.21\pm 0.22$
Self-Attention | $\textbf{89.97}\pm\textbf{0.34}$ | $\textbf{88.56}\pm\textbf{0.39}$
Max-Pooling | $88.39\pm 0.27$ | $86.89\pm 0.56$
Table 3: F1-scores on the _Gender Split_. ∗∗ Results from Maronikolakis et al.
(2020). Best results are in bold.

Our proposed _multi-encoder_ achieves the best performance when using Self-
Attention to combine the three parallel encoders (F1: $91.19$; $89.97$,
$88.56$; $88.37$, $87.91$, $87.16$; for _Person_ , _Gender_ , and _Location_
splits respectively). Moreover, it outperforms the best single-encoder model
BERTweet in the majority of cases which corroborates that parody detection
benefits from combining general contextual representations with humor and
sarcasm specific information, as humor and sarcasm are important
characteristics of parody (Haiman et al., 1998; Highfield, 2016). On the other
hand, simply concatenating the three parallel encoders degrades the
performance across different splits (_Person_ : $88.99$; _Gender_ : $86.84$,
$84.21$ _Location_ : $85.41$, $84.74$, $83.62$). This happens because the
concatenation operation treats the three encoders as equally important. While
humor and sarcasm are related to parody, they may not necessarily have the
same relevance as indicators of parody.

Our best performing model (Self-Attention) outperforms the vanilla BERTweet by
$3$ F1 points when trained on female accounts and by almost $2$ F1 points when
trained on male accounts. We speculate that the additional linguistic
information from the two encoders (i.e., sarcasm and humor) is more beneficial
in low data settings. The number of female politicians is considerably smaller
than males in the data set (see Maronikolakis et al. (2020) for more details).

Location
---
Model |  | UK+US
---
$\to$ RoW
| RoW+US
---
$\to$ UK
| RoW+UK
---
$\to$ US
Single-Encoder
BERT∗∗ | $86.69\pm 0.45$ | $83.78\pm 0.19$ | $83.12\pm 0.60$
RoBERTa∗∗ | $87.70\pm 0.45$ | $85.10\pm 0.27$ | $85.99\pm 0.61$
BERTweet | $88.21\pm 0.26$ | $87.85\pm 0.24$ | $\textbf{87.18}\pm\textbf{0.41}$
Multi-encoder (Ours)
Concatenation | $85.41\pm 0.26$ | $84.74\pm 0.20$ | $83.62\pm 0.35$
Self-Attention | $\textbf{88.37}\pm\textbf{0.28}$ | $\textbf{87.91}\pm\textbf{0.19}$ | $87.16\pm 0.37$
Max-Pooling | $88.25\pm 0.39$ | $86.49\pm 0.33$ | $86.54\pm 0.41$

Table 4: F1-scores on the _Location Split_. ∗∗ Results from Maronikolakis et
al. (2020). Best results are in bold.

###  4.2 Ablation Study

We also examine the effect of combining parody-specific representations with
humor and sarcasm information by running an ablation study. We compare
performance of four models: using parody representations only (P), and
combining parody representations with humor (P+H), or sarcasm (P+S)
information, as well as with both (P+S+H). The results of this analysis are
depicted in Tables 5, 6 and 7. We observe that both sarcasm and humor
contribute to the performance gain, but using both is more beneficial.
Modelling sarcasm leads to more gains than humor and this could be attributed
to the characteristics of the parody corpus, namely that it focuses primarily
on the political domain, which have a high sarcastic component (Anderson and
Huntington, 2017).

Person
---
Model | F1
Single-Encoder
BERTweet (P) | $90.72\pm 0.31$
Multi-encoder (Ours)
Concatenation (P+S+H) | $88.99\pm 0.17$
Concatenation (P+S) | $90.51\pm 0.26$
Concatenation (P+H) | $89.98\pm 0.23$
Self-Attention (P+S+H) | $\textbf{91.19}\pm\textbf{0.31}$
Self-Attention (P+S) | $91.14\pm 0.40$
Self-Attention (P+H) | $90.98\pm 0.36$
Max-Pooling (P+S+H) | $91.05\pm 0.30$
Max-Pooling (P+S) | $91.06\pm 0.39$
Max-Pooling (P+H) | $90.78\pm 0.42$
Table 5: F1-scores for parody detection on the _Person Split_ with various
settings: parody (P) representations only, and combining parody
representations with humor (P+H), or sarcasm (P+S) information, as well as
with both (P+S+H). Best results are in bold.  Gender
---
Model | M$\to$F | F$\to$M
Single-Encoder
BERTweet (P) | $88.01\pm 0.29$ | $85.57\pm 0.27$
Multi-encoder (Ours)
Concatenation (P+S+H) | $86.84\pm 0.15$ | $84.21\pm 0.22$
Concatenation (P+S) | $86.93\pm 0.40$ | $83.70\pm 0.41$
Concatenation (P+H) | $86.58\pm 0.31$ | $83.34\pm 0.38$
Self-Attention (P+S+H) | $\textbf{89.97}\pm\textbf{0.34}$ | $\textbf{88.56}\pm\textbf{0.39}$
Self-Attention (P+S) | $89.49\pm 0.37$ | $88.23\pm 0.44$
Self-Attention (P+H) | $88.71\pm 0.42$ | $87.62\pm 0.50$
Max-Pooling (P+S+H) | $88.39\pm 0.27$ | $86.89\pm 0.56$
Max-Pooling (P+S) | $88.36\pm 0.46$ | $86.55\pm 0.49$
Max-Pooling (P+H) | $88.14\pm 0.52$ | $86.53\pm 0.53$
Table 6:  F1-scores for parody detection on the _Gender Split_ with various
settings: parody (P) representations only, and combining parody
representations with humor (P+H), or sarcasm (P+S) information, as well as
with both (P+S+H). Best results are in bold.

Location
---
Model |  | UK+US
---
$\to$ RoW
| RoW+US
---
$\to$ UK
| RoW+UK
---
$\to$ US
Single-Encoder
BERTweet (P) | $88.21\pm 0.26$ | $87.85\pm 0.24$ | $\textbf{87.18}\pm\textbf{0.41}$
Multi-encoder (Ours)
Concatenation (P+S+H) | $85.41\pm 0.26$ | $84.74\pm 0.20$ | $83.62\pm 0.35$
Concatenation (P+S) | $85.92\pm 0.24$ | $85.67\pm 0.18$ | $84.09\pm 0.39$
Concatenation (P+H) | $85.39\pm 0.29$ | $85.33\pm 0.26$ | $83.75\pm 0.44$
Self-Attention (P+S+H) | $\textbf{88.37}\pm\textbf{0.28}$ | $\textbf{87.91}\pm\textbf{0.19}$ | $87.16\pm 0.37$
Self-Attention (P+S) | $88.24\pm 0.33$ | $87.88\pm 0.23$ | $86.47\pm 0.32$
Self-Attention (P+H) | $88.13\pm 0.35$ | $87.05\pm 0.28$ | $85.36\pm 0.40$
Max-Pooling (P+S+H) | $88.25\pm 0.39$ | $86.49\pm 0.33$ | $86.54\pm 0.41$
Max-Pooling (P+S) | $88.28\pm 0.42$ | $87.83\pm 0.39$ | $86.56\pm 0.36$
Max-Pooling (P+H) | $88.22\pm 0.52$ | $86.44\pm 0.42$ | $85.96\pm 0.45$

Table 7:  F1-scores for parody detection on the _Location Split_ with various
settings: parody (P) representations only, and combining parody
representations with humor (P+H), or sarcasm (P+S) information, as well as
with both (P+S+H). Best results are in bold.

##  5 Error Analysis

Finally, we perform an error analysis to examine the behavior and limitations
of our best-performing model (_multi-encoder_ with Self-Attention).

The next two examples correspond to real tweets that were misclassified as
parody:

  * (1)

Congratulations, <mention>! <url>.

  * (2)

It’s a shame that Boris isn’t here answering questions from the public this
evening.

We speculate that the model misclassified these tweets as parody because they
contain terms that are related to sarcastic short texts such as user mentions,
punctuation marks (!), and negation (isn’t) González-Ibáñez et al. (2011);
Highfield (2016).

The following two examples correspond to parody tweets that were misclassified
as real:

  * (3)

Hey America, it’s time to use your safe word.

  * (4)

I fully support the Digital Singles Market.

Example (3) is a call-to-action message, while Example (4) is a statement
expressing support for a particular subject. These statements are written in a
style that is similar to political slogans or campaign speeches (Fowler et
al., 2021) that the model fails to recognise. As a result, in addition to
humor and sarcasm semantics, the model might be improved by integrating
knowledge from the political domain such as from political speeches.

##  6 Conclusion

In this paper, we studied the impact of jointly modelling figurative devices
to improve predictive performance of political parody detection in tweets. Our
motivation was based on studies in linguistics which emphasize the humorous
and sarcastic components of parody (Haiman et al., 1998; Highfield, 2016). We
presented a method that combines parallel encoders to capture parody, humor,
and sarcasm specific representations from input sequences, which outperforms
previous state-of-the-art proposed by Maronikolakis et al. (2020).

In the future, we plan to combine information from other modalities (e.g.,
images) for improving parody detection (Sánchez Villegas and Aletras, 2021;
Sánchez Villegas et al., 2021).

## Acknowledgements

We would like to thank all reviewers for their valuable feedback. DSV is
supported by the Centre for Doctoral Training in Speech and Language
Technologies (SLT) and their Applications funded by the UK Research and
Innovation grant EP/S023062/1.

## References

  * Anderson and Huntington (2017) Ashley A Anderson and Heidi E Huntington. 2017.  Social media, science, and attack discourse: How twitter discussions of climate change use sarcasm and incivility.  _Science Communication_ , 39(5):598–620.
  * Annamoradnejad and Zoghi (2020) Issa Annamoradnejad and Gohar Zoghi. 2020.  Colbert: Using bert sentence embedding for humor detection.  _arXiv preprint arXiv:2004.12765_.
  * Cai et al. (2019) Yitao Cai, Huiyu Cai, and Xiaojun Wan. 2019.  Multi-modal sarcasm detection in Twitter with hierarchical fusion model.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2506–2515, Florence, Italy. Association for Computational Linguistics.
  * Caruana (1993) Richard Caruana. 1993.  Multitask learning: A knowledge-based source of inductive bias.  pages 41–48.
  * Castro et al. (2019) Santiago Castro, Devamanyu Hazarika, Verónica Pérez-Rosas, Roger Zimmermann, Rada Mihalcea, and Soujanya Poria. 2019.  Towards multimodal sarcasm detection (an _Obviously_ perfect paper).  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4619–4629, Florence, Italy. Association for Computational Linguistics.
  * Chen and Soo (2018) Peng-Yu Chen and Von-Wun Soo. 2018.  Humor recognition using deep learning.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 113–117, New Orleans, Louisiana. Association for Computational Linguistics.
  * Davidov et al. (2010) Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010.  Semi-supervised recognition of sarcasm in Twitter and Amazon.  In _Proceedings of the Fourteenth Conference on Computational Natural Language Learning_ , pages 107–116, Uppsala, Sweden. Association for Computational Linguistics.
  * Davis et al. (2018) Jenny L Davis, Tony P Love, and Gemma Killen. 2018.  Seriously funny: The political work of humor on social media.  _New Media & Society_, 20(10):3898–3916.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Fowler et al. (2021) Erika Franklin Fowler, Michael M Franz, Gregory J Martin, Zachary Peskowitz, and Travis N Ridout. 2021.  Political advertising online and offline.  _American Political Science Review_ , 115(1):130–149.
  * Ghosh et al. (2020) Debanjan Ghosh, Elena Musi, and Smaranda Muresan. 2020.  Interpreting verbal irony: Linguistic strategies and the connection to theType of semantic incongruity.  In _Proceedings of the Society for Computation in Linguistics 2020_ , pages 82–93, New York, New York. Association for Computational Linguistics.
  * Ghosh et al. (2021) Debanjan Ghosh, Ritvik Shrivastava, and Smaranda Muresan. 2021.  “laughing at you or with you”: The role of sarcasm in shaping the disagreement space.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1998–2010, Online. Association for Computational Linguistics.
  * González-Ibáñez et al. (2011) Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011\.  Identifying sarcasm in Twitter: A closer look.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 581–586, Portland, Oregon, USA. Association for Computational Linguistics.
  * Gururangan et al. (2020) Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020.  Don’t stop pretraining: Adapt language models to domains and tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8342–8360, Online. Association for Computational Linguistics.
  * Haiman et al. (1998) John Haiman et al. 1998.  _Talk is cheap: Sarcasm, alienation, and the evolution of language_.  Oxford University Press on Demand.
  * Hetzron (1991) Robert Hetzron. 1991.  On the structure of punchlines.
  * Highfield (2016) Tim Highfield. 2016.  News via voldemort: Parody accounts in topical discussions on twitter.  _New Media Soc._ , 18(9):2028–2045.
  * Kiddon and Brun (2011) Chloé Kiddon and Yuriy Brun. 2011.  That’s what she said: Double entendre identification.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 89–94, Portland, Oregon, USA. Association for Computational Linguistics.
  * Li and Hoiem (2018) Zhizhong Li and Derek Hoiem. 2018.  Learning without forgetting.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 40(12):2935–2947.
  * Liebrecht et al. (2013) Christine Liebrecht, Florian Kunneman, and Antal van den Bosch. 2013.  The perfect solution for detecting sarcasm in tweets #not.  In _Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_ , pages 29–37, Atlanta, Georgia. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Maronikolakis et al. (2020) Antonis Maronikolakis, Danae Sánchez Villegas, Daniel Preotiuc-Pietro, and Nikolaos Aletras. 2020.  Analyzing political parody in social media.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4373–4384, Online. Association for Computational Linguistics.
  * Mu and Aletras (2020) Yida Mu and Nikolaos Aletras. 2020.  Identifying twitter users who repost unreliable news sources with linguistic information.  _PeerJ Computer Science_ , 6:e325.
  * Nguyen et al. (2020) Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020.  BERTweet: A pre-trained language model for English tweets.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 9–14, Online. Association for Computational Linguistics.
  * Oprea and Magdy (2020) Silviu Oprea and Walid Magdy. 2020.  iSarcasm: A dataset of intended sarcasm.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1279–1289, Online. Association for Computational Linguistics.
  * Purandare and Litman (2006) Amruta Purandare and Diane Litman. 2006.  Humor: Prosody analysis and automatic recognition for F*R*I*E*N*D*S*.  In _Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing_ , pages 208–215, Sydney, Australia. Association for Computational Linguistics.
  * Rajadesingan et al. (2015) Ashwin Rajadesingan, Reza Zafarani, and Huan Liu. 2015.  Sarcasm detection on twitter: A behavioral modeling approach.  _Proceedings of the Eighth ACM International Conference on Web Search and Data Mining_.
  * Rose (1993) Margaret A Rose. 1993.  _Parody: ancient, modern and post-modern_.  Cambridge University Press.
  * Sánchez Villegas and Aletras (2021) Danae Sánchez Villegas and Nikolaos Aletras. 2021.  Point-of-interest type prediction using text and images.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7785–7797, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Sánchez Villegas et al. (2021) Danae Sánchez Villegas, Saeid Mokaram, and Nikolaos Aletras. 2021.  Analyzing online political advertisements.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3669–3680, Online. Association for Computational Linguistics.
  * Sun et al. (2020a) Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2020a.  How to fine-tune bert for text classification?  _arXiv:1905.05583_.
  * Sun et al. (2020b) Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020b.  Ernie 2.0: A continual pre-training framework for language understanding.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8968–8975.
  * Taylor and Mazlack (2004) J.M. Taylor and L.J. Mazlack. 2004.  Humorous wordplay recognition.  4:3306–3311 vol.4.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Vis (2013) Farida Vis. 2013.  Twitter as a reporting tool for breaking news: Journalists tweeting the 2011 uk riots.  _Digit. J._ , 1(1):27–47.
  * Weller and Seppi (2019) Orion Weller and Kevin Seppi. 2019.  Humor detection: A transformer gets the last laugh.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3621–3625, Hong Kong, China. Association for Computational Linguistics.

##  Appendix A Multitask-Learning

We also tested applying _multi-task learning_ approaches (Caruana, 1993) to
use either sarcasm prediction (P+S), humor prediction (P+H) or both (P+S+H) as
auxiliary tasks for parody detection. We utilize BERTweet as the share encoder
and independent classification layers for parody and humor or sarcasm. Three
sets of weights are applied to losses from each independent classification
layer and the three layers are stacked. The best results are chosen and
depicted in Table 8, Table 9 and Table 10.

Person
---
Model | F1
Single-Encoder
BERTweet (P) | $\textbf{90.72}\pm\textbf{0.31}$
Multi-Task
P+S+H | $87.46\pm 0.18$
P+S | $89.41\pm 0.31$
P+H | $87.41\pm 0.38$
Table 8: F1-scores for parody detection on the _Person Split_ using Multi-task
Learning models (P: Parody, S: Sarcasm, H: Humor). Best results are in bold.
Gender
---
Model | M$\to$F | F$\to$M
Single-Encoder
BERTweet (P) | $88.01\pm 0.29$ | $85.57\pm 0.27$
Multi-Task
P+S+H | $85.28\pm 0.29$ | $84.10\pm 0.37$
P+S | $\textbf{88.13}\pm\textbf{0.21}$ | $\textbf{86.07}\pm\textbf{0.44}$
P+H | $84.53\pm 0.31$ | $86.07\pm 0.47$
Table 9: F1-scores on the _Gender Split_ using Multi-task Learning models (P:
Parody, S: Sarcasm, H: Humor). Best results are in bold.

Location
---
Model |  | UK+US
---
$\to$ RoW
| RoW+US
---
$\to$ UK
| RoW+UK
---
$\to$ US
Single-Encoder
BERTweet (P) | $\textbf{88.21}\pm\textbf{0.26}$ | $\textbf{87.85}\pm\textbf{0.24}$ | $\textbf{87.18}\pm\textbf{0.41}$
Multi-Task
P+S+H | $86.41\pm 0.17$ | $86.23\pm 0.20$ | $85.13\pm 0.29$
P+S | $87.74\pm 0.36$ | $87.26\pm 0.34$ | $86.67\pm 0.43$
P+H | $85.54\pm 0.38$ | $84.78\pm 0.47$ | $84.15\pm 0.56$

Table 10: F1-scores on the _Location Split_ using Multi-task Learning models
(P: Parody, S: Sarcasm, H: Humor). Best results are in bold.