# Can light-rings self-gravitate?

Francesco Di Filippo  Institute of Theoretical Physics, Faculty of Mathematics
and Physics, Charles University, V Holešovičkách 2, 180 00 Prague 8, Czech
Republic  Luciano Rezzolla  Institut für Theoretische Physik, Max-von-Laue-
Strasse 1, 60438 Frankfurt, Germany  Frankfurt Institute for Advanced Studies,
Ruth-Moufang-Strasse 1, 60438 Frankfurt, Germany  School of Mathematics,
Trinity College, Dublin 2, Ireland

###### Abstract

In a spherically symmetric and static spacetime of a compact object, such as
that of a Schwarzschild black hole, the light-ring is a 2-sphere where photons
experience the only possible circular orbits. As a “Gedankenexperiment”, we
imagine an advanced civilisation able to populate the light-ring of a
nonrotating black hole of mass $M$ with photons having a fine-tuned impact
parameter that allows their orbits to be exactly circular with radius $r=3M$.
As the number of photons in the light-ring increases in time, its mass will no
longer be negligible and hence it will impact on the background spacetime,
that is, it will “self-gravitate”. We here consider two different routes to
assign a nonzero mass to the light-ring that are either based on a discrete
concentration of photons on a specific radial location or on a suitable
distribution of photons in a given region. In both cases, and using the
Einstein equations, we find that the inclusion of the energy from the
accumulated photons leads to the generation of new light-rings. Such new
light-rings can either appear at well-defined but discrete locations, or be
fused in a well-defined region. In either case, we show that such light-ring
configurations are dynamically unstable and a small perturbation, either via
the inclusion of an additional photon onto the light-ring or via the
absorption of a photon by the black hole, leads to a catastrophic destruction
of the light-ring structures.

##  I Introduction

Light-rings, namely, the location of unstable circular orbits, play a crucial
role in the study of astrophysical compact objects as shown by a number of a
theoretical investigations exploring the spacetime on those scales (see, e.g.,
Chirenti and Rezzolla (2007); Vincent  _et al._ (2016); Glampedakis  _et al._
(2017); Cardoso  _et al._ (2017); Jai-akson  _et al._ (2017); Glampedakis and
Pappas (2018); Cardoso  _et al._ (2016); Carballo-Rubio  _et al._ (2018);
Cardoso and Pani (2019); Völkel  _et al._ (2021); Kocherlakota  _et al._
(2021); Völkel  _et al._ (2022); Eichhorn  _et al._ (2023); Moreira  _et al._
(2024); Pedrotti and Vagnozzi (2024); Genzel  _et al._ (2024)), but also by
the observations of supermassive black holes Akiyama  _et al._ (2019, 2022a,
2022b); Kocherlakota and Rezzolla (2022) or through the ringdown associated
with the gravitational-wave signal from merging black-hole binaries Abbott and
et al (2016); Abbott  _et al._ (2019, 2021a, 2021b). Indeed, it is generally
accepted that the study of light-rings, either via black-hole imaging or via
black-hole quasi-normal modes, represents one of the most promising routes to
test theories of gravity beyond general relativity.

Interestingly, while light-rings in general relativity are well understood,
both for nonrotating and rotating black holes, but also for other compact
objects not possessing an event horizon, and have been the subject of
uncountable works, there is an question that so far has escaped the attention
of the literature, namely, “can light-rings self-gravitate?”

This work is dedicated to addressing this question by carrying out a
“Gedankenexperiment” to explore the behaviour of the light-ring of a
Schwarzschild black hole. More specifically, we postulate the existence of an
advanced civilisation that is able to approach a (nonrotating) black hole and
has developed a technology that allows it to shoot large numbers of photons
towards the black hole with the exact impact parameter corresponding to the
unstable circular orbit placed at $r=3\,M$, where $M$ is the mass of the black
hole. In this way, the civilisation is able to carefully populate the light-
ring with an arbitrarily large number of photons, all having the necessary
fine-tuned impact parameter $b=\sqrt{27}M$.

However, as the number of photons in the light-ring increases with time, its
mass will no longer be negligible and hence it will start to “self-gravitate”,
that is, it will have a sizable impact on the background spacetime. Under
these conditions, the dynamics of the photons can no longer be considered as
that in the original black-hole spacetime in vacuum, but it will be regulated
by the geometry of the spacetime resulting from the proper solution of the
Einstein equations for a black hole surrounded by the energy contributions
coming from the thin null shells of photons on (unstable) circular orbits.

To address this scenario, we consider two different routes. In the first one,
which we refer to as the “discrete sequence”, we show that the solution of the
Einstein equations leads to the formation of a new light-ring located outside
the first one and at a position that depends on the mass of the light-ring. In
full analogy with the first light-ring, also the second light-ring can be
populated with photons and acquire a mass, thus leading to the appearance of a
third light-ring placed outside the second light-ring at a radial location
that will again depend on the total energy of the photons. Clearly, the
advanced civilisation can iterate this procedure for an arbitrarily number of
times and hence construct a whole hierarchy of discrete, nested, and self-
gravitating light-rings that are increasingly large in size. However, adding a
single photon on any of the light-rings would change the geometry of the
constructed spacetime and hence destabilise all the photons on the various
discrete light-rings, leading to a catastrophic destruction of the hierarchy.
In the second scenario, we do not consider a series of discrete shells of
light-rings but a suitably constructed continuum distribution of energy in
which the shells are infinitesimally separated and thus lead to a region of
densely packed light-rings, namely, a “light-region”. Also in this case,
however, the fine-tuned series of light rings is unstable once any additional
photon is added.

The structure of the paper is as follows. Section II briefly reviews the
properties of light-rings around Schwarzschild black holes when these are seen
as populated by test photons and illustrates the solution of the Einstein
equations to construct a hierarchy of discrete, thin, self-gravitating light-
rings, pointing out its intrinsic instability. Similarly, Sec. III discusses
how to solve the Einstein equations to construct a continuous self-gravitating
light-region and it shows that it is equally unstable. Finally, Sec. IV
summarises our results and the implications for realistic black holes.

##  II Discrete sequence of light-rings

We start by recalling that for an axisymmetric and stationary spacetime
described by spherical polar coordinates $(t,r,\theta,\phi)$, $\partial_{t}$
and $\partial_{\phi}$ are Killing vectors. Light-rings in such axisymmetric
and stationary spacetimes containing a black hole are then defined as regions
of the spacetime in which null geodesics only have momenta along such Killing
vectors, namely, along the $t$ and $\phi$ directions. If we further restrict
to a spherically symmetric configuration, which is the interest of this paper,
we can write the metric as

$ds^{2}=g_{tt}dt^{2}+g_{rr}dr^{2}+r^{2}d\Omega^{2}\,,$ (1)

and the locations of the light-rings can be obtained by studying the
Hamiltonian for a massless particle with four-momentum $p_{\mu}$

$\mathcal{H}:=g^{\mu\nu}p_{\mu}p_{\nu}=g^{rr}p_{r}^{2}+g^{\theta\theta}p_{\theta}^{2}+V(r,\theta)=0\,.$
(2)

More specifically, given the potential $V(r,\theta)$

$V(r,\theta):=\frac{E^{2}}{g_{tt}}+\frac{\Phi^{2}}{r^{2}\sin^{2}\theta}\,,$
(3)

where $E:=-p_{t}$ and $\Phi:=p_{\phi}$ are constants of motion, the definition
(2) and Hamilton’s equations

$\dot{p}_{\mu}=-\partial_{\mu}\mathcal{H}=-\left(\partial_{\mu}\left(g^{rr}p_{r}^{2}\right)+\partial_{\mu}\left(g^{\theta\theta}p_{\theta}^{2}\right)+\partial_{\mu}V(r,\theta)\right)\,,$
(4)

where the dot refers to a derivative with respect to the affine parameter,
imply that the conditions for the occurrence of light-rings, i.e.,
$p_{r}=p_{\theta}=\dot{p}_{\mu}=0$, are equivalent to $V(r,\theta)=0$ and
$\partial_{\mu}V(r,\theta)=0$. Furthermore, motion on the light-ring will be
stable if its location corresponds to either a minimum of the potential and
unstable if it corresponds to a maximum or a saddle point.

Rather than considering the potential, we can determine the position of the
light-rings by restricting to the equatorial plane and studying the stationary
points of the effective potential (see e.g., Cunha  _et al._ (2016, 2017); Di
Filippo (2024)) $H(r):={g_{tt}}/{g_{\phi\phi}}$. This approarch reveals, for
example, that a single light-ring exists in the case of a Schwarzschild black
hole and it is located at $r=3M$, thus marking the position of the only
circular but unstable photon orbit.

Our next step consists of adding a discrete set of shells on a Schwarzschild
background. In particular, in our “Gedankenexperiment” we imagine that the
advanced civilisation mentioned above is able to populate the light-ring of a
Schwarzschild black hole of initial mass $M_{0}$ with photons having a fine-
tuned impact parameter that allows their orbits to be circular at the light-
ring. We assume that the angular components of the momentum of each photon are
randomly but uniformly distributed, so that the corresponding light-ring will
be be a 2-sphere, thus preserving the spherical symmetry of the spacetime. As
the number of photons in the light-ring increases, the energy of the shell of
photons will no longer be negligible, impacting the background spacetime and
hence leading to a new spacetime metric that is no longer that of the
Schwarzschild black hole with mass $M_{0}$. In order to obtain the resulting
metric, we assume that the new metric is actually composed of a black hole and
a discrete series of $N$ nested null shells of photons on (unstable) circular
orbits. This allows us to consider a fully generic approach but it is trivial
to restrict the relevant results to the case of a single shell, i.e., for
$N=1$.

More specifically, because the spacetime in between the shells is in vacuum,
we can invoke Birkhoff’s theorem Birkhoff and Langer (1923) to deduce that the
solution of the Einstein equations in this case yields a series of $N+1$
spacetime regions and for each of them the line element, $ds^{2}_{n}$, is
given by (see also Berry  _et al._ (2022))

$ds^{2}_{n}=-\xi^{2}_{n}\left(1-\frac{2M_{n}}{r}\right)dt^{2}+\left(1-\frac{2M_{n}}{r}\right)^{-1}dr^{2}+r^{2}d\Omega^{2}\,,$
(5)

where $n\in\\{0,N\\}$ and $\xi^{2}_{n}$ is a set of coefficients needed to
guarantee the continuity between the different portions of the spacetime and
where $\xi^{2}_{0}=1$. Clearly, the region $n=0$ corresponds to the section
between the black hole and the first shell, while the $n$-th region
corresponds to the section between the $(n-1)$-th and the $n$-th shell.
Obviously, the region outside the $n$-th shell is described by a Schwarzschild
spacetime with mass $M_{n}$.

The mass parameter $M_{n}$ in Eq. (5) represents the total mass enclosed
within the corresponding region and is thus given by

$M_{n}:=M_{0}+\sum_{i=1}^{n}\delta m_{i}=M_{0}+\delta M_{n}\,,$ (6)

where $\delta m_{i}$ indicates the mass of a single shell and $\delta M_{n}$
denotes the sum of the masses of the first $n$ shells, i.e.,

$\delta M_{n}:=\sum_{i=1}^{n}\delta m_{i}\,.$ (7)

Figure 1: Shown with a black solid line is the effective potential in the case
of a discrete hierarchy of light-rings (left panel, where $M_{0}=1$, $\delta
m_{1}=\delta m_{2}=0.01$ and the light rings are at $r_{0}=3\,M_{0}$,
$r_{1}=3\left(M_{0}+\delta m_{1}\right)$ and $r_{2}=3\left(M_{0}+\delta
m_{1}+\delta m_{2}\right)$) or a continuous sequence of light-rings (right
panel, where $M_{0}=1$, $R_{\rm in}=3.0\,M_{0}$ and $R_{\rm out}=3.1\,M_{0}$).
The location of the light rings is shown with vertical dashed lines in the
left panel and with a shaded area in the right panel. In both cases, the red
dashed line shows the effective potential for a Schwarzschild black hole.

As mentioned above, $\xi^{2}_{n}$ are constants and can be fixed by simply
redefining the time coordinate in each spacetime region. While this is in
principle possible, it would complicate the subsequent analysis and hence we
determine these constants by requiring the requiring the $\mathcal{C}_{0}$
continuity of the induced metric on the shells.

$\xi^{2}_{n}=\left(\frac{1-{2M_{n-1}}/{R_{n}}}{1-{2M_{n}}/{R_{n}}}\right)\xi^{2}_{n-1}\,,$
(8)

where $R_{n}$ is the radius of the $n$-th shell. Given the metric (5), the
corresponding effective potential for massless particles is

$H_{n}(r)=\frac{1}{r^{2}}\xi^{2}_{n}\left(1-\frac{2M_{n}}{r}\right)\,,$ (9)

whose extremal points marking the position of the unstable photon circular
orbits can be easily computed after imposing $\partial_{r}H_{n}(r)=0$ and are
located at the $n$ different shells with coordinate radii $r_{n}=3M_{n}$. Note
that the location of the $n$-th shell coincides with the location of the
light-ring of the $n-1$ spacetime region, i.e., $R_{n}=r_{n-1}$, and this
allows us to determine the conditions under which the hierarchy of light-rings
actually forms a new black hole. In practice, this would happen for
$M_{n}/R_{n}\geq 1/2$, that is, for

$\frac{M_{n}}{R_{n}}=\frac{M_{n}}{3M_{n-1}}=\frac{M_{n-1}+\delta
m_{n}}{3M_{n-1}}\geq\frac{1}{2}\,.$ (10)

Stated differently, no new black hole is formed as long as the additional mass
outside the initial black hole is

$\frac{\delta
m_{n}}{M_{0}}\leq\frac{M_{n-1}}{2M_{0}}=\frac{1}{2}\left(1+\frac{\delta
M_{n-1}}{M_{0}}\right)\,,$ (11)

which is easily true since we are exploring a scenario with $\delta
m_{n}/M_{0}\ll 1$.

Let us now restrict our attention to the simplest case of a single self-
gravitating shell of photons on unstable circular orbits. In this case, $N=1$
and

$\displaystyle\xi^{2}_{1}=\frac{R_{1}-{2M_{0}}}{R_{1}-{2M_{1}}}\,,$ (12)
$\displaystyle r_{1}=3M_{1}=3\left(M_{0}+\delta M_{1}\right)\,,$ (13)

from which

$H_{1}(r)=\frac{1}{r^{2}}\left(\frac{R_{1}-{2M_{0}}}{R_{1}-{2M_{1}}}\right)\left(1-\frac{2M_{1}}{r}\right)\,.$
(14)

Clearly, it is now possible to populate also the new light-ring with a second
shell of carefully chosen photons and the resulting spacetime would correspond
to the construction discussed above but in the case in which, $N=2$. The
corresponding effective potential is shown in the left panel of Fig. 1, where
we report with a solid black line the effective potential for a black hole
with $M_{0}=1$ and the two shells with mass $\delta m_{1}=\delta m_{2}=0.01$.
Note the presence of a first light-ring at $r_{0}=3M_{0}$ and of a second one
at $r_{1}=3(M_{0}+\delta m_{1})$ and a third one at $r_{2}=3(M_{0}+\delta
m_{1}+\delta m_{2})$, where the effective potential, which is everywhere
continuous but with discontinuous first derivatives at the light-rings, has
extrema. In this case, only two light-rings are populated and have a nonzero
mass, while the most external one has an unstable circular orbit but a zero
mass. Clearly, as long as one is able to populate these light-rings with fine-
tuned photons, the construction illustrated here can be extended simply for
any integer $N$, thus creating a hierarchy of $N+1$ light-rings.

Is this configuration stable? In principle, to address this question we would
need to perform a linear perturbation analysis taking Eq. (5) as background
spacetime and assess the properties of the imaginary eigenfrequencies (see,
e.g., Nagar and Rezzolla (2005)). In practice, we take a shortcut and note
that it is not difficult to realise that such a spacetime is dynamically
unstable. Consider, for instance, the addition of a single photon on any of
the shells of the hierarchy. This will be sufficient to change the mass of the
corresponding shell and hence place on unstable photon orbits all the photons
on the light-rings that are at larger radii than the one on which the photon
is deposited. Similarly, if a single photon is absorbed by the black hole,
this would destabilise all of the light-rings, dispersing or absorbing all the
photons that have been accumulated. learly, while we believe this conclusion
is correct, a proper perturbative analysis is needed to confirm it.

##  III Continuous sequence of light-rings

As mentioned in the introduction, the “Gedankenexperiment” can be carried out
also in a different way. More specifically, rather than accumulating photons
on the first light-ring till they produce a discrete contribution to the
spacetime and generate a new light-ring, we can imagine a situation in which
the advanced civilization concentrates on adding photons in the right amount,
both in terms on numbers and impact parameters, so that the spacetime
properties are varied smoothly. This would correspond to a situation in which
both the mass of each individual shell and the distance among shells goes to
zero while the total mass of the shells remains finite. This configuration
would then describe a continuous energy distribution within a finite region of
light-rings, i.e., a “light-region” with radius $r\in\left[R_{\rm in},R_{\rm
out}\right]$, with $R_{\rm in}=3\,M_{0}$.

Hence, we start by replacing the discrete nature of the mass in the light-ring
given by Eq. (6) with the continuous function

$M(r)=M_{0}+\delta M(r)\,,$ (15)

where $\delta M(r)$ denotes the full energy within a radius $r$, so that
$\delta M(R_{\rm in})=0$. Furthermore, as $M(r)$ denotes the total energy
within a radius $r$, it must be a non-decreasing function, i.e.,
$\partial_{r}\delta M(r)\geq 0$. Also in this case we can use the coefficients
$\xi_{n}$ and calculate their values in the continuous limit; it is however
more straightforward to use a perturbative approach and expanding in terms of
the ratio $\delta M/M\ll 1$. More specifically, we can rewrite Eq. (8) as

$\begin{array}[]{rl}\xi^{2}_{n}=&\displaystyle\left(\frac{R_{n}-{2M_{n-1}}}{R_{n}-{2M_{n}}}\right)\xi^{2}_{n-1}=\prod_{i=1}^{n}\left(1+\frac{2\delta
m_{i}}{R_{i}-{2M_{i}}}\right)\\\ =&\displaystyle 1+\sum_{i=1}^{n}\frac{2\delta
m_{i}}{R_{i}-{2M_{i}}}+\\\
&\displaystyle\frac{1}{2}\sum_{i,j=1}^{n}\left(\frac{2\delta
m_{i}}{R_{i}-{2M_{i}}}\right)\left(\frac{2\delta
m_{j}}{R_{j}-{2M_{j}}}\right)+\mathcal{O}\left(\left(\frac{\delta
M}{M}\right)^{3}\right)\,,\end{array}$

so that

$\xi^{2}_{n}\approx 1+\sum_{i=1}^{n}\frac{2\delta
m_{i}}{R_{i}-{2M_{i}}}+\frac{1}{2}\left(\sum_{i=1}^{n}\frac{2\delta
m_{i}}{R_{i}-{2M_{i}}}\right)^{2}\,,$ (16)

when neglecting terms $\mathcal{O}\left(\left({\delta
M}/{M}\right)^{3}\right)$ and where quadratic term is necessary since the
effective potential does not vary at $\mathcal{O}\left(\left({\delta
M}/{M}\right)\right)$. The continuous limit of expression (16) can be obtained
by replacing the sum with an integral, namely

$\xi^{2}(r)=\displaystyle
1+\int_{3M_{0}}^{r}\frac{2M^{\prime}(\bar{r})}{\bar{r}-2M(\bar{r})}{\rm
d}\bar{r}+\frac{1}{2}\left(\int_{3M_{0}}^{r}\frac{2M^{\prime}(\bar{r})}{\bar{r}-2M(\bar{r})}{\rm
d}\bar{r}\right)^{2}\,,$ (17)

whose analytic solution requires now a reasonable assumption. In particular,
after defining $\delta r:=r-3M_{0}$, we can rewrite the linear term in Eq.
(17) as

$\displaystyle\int_{3M_{0}}^{r}\frac{2M^{\prime}(\bar{r})}{\bar{r}-2M(\bar{r})}{\rm
d}\bar{r}=$
$\displaystyle\int_{3M_{0}}^{r}\frac{2M^{\prime}(\bar{r})}{3M_{0}+\delta
r-2M(\bar{r})}{\rm d}\bar{r}$ $\displaystyle=$
$\displaystyle\int_{M_{0}}^{M_{0}+\delta M}\frac{2{\rm d}M}{3M_{0}+\delta
r(\delta M)-2M}\,,$ (18)

where we have indicated with $\delta r(\delta M)$ the inverse of the function
$\delta M(r)$. Making the assumption (whose correctness we will validate
a-posteriori) that $\delta M$ and $\delta r$ are linearly, related, i.e., that
$\delta r(\delta M)=\alpha\delta M$, with $\alpha$ a positive constant, we can
solve the two integrals in (17) and obtain

$\xi^{2}(r)=1+\frac{2\delta M(r)}{M_{0}}+\frac{\left(4-\alpha\right)\delta
M^{2}(r)}{\left(M_{0}\right)^{2}}+\mathcal{O}\left(\left(\frac{\delta
M}{M}\right)^{3}\right)\,.$ (19)

As a result, at the quadratic order in $\delta M/M$ the effective potential
reads

$\displaystyle H(r)$
$\displaystyle=\frac{1}{r^{2}}\left(1-\frac{2M_{0}}{r}\right)+\frac{2(r-3M_{0})}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)$
$\displaystyle\phantom{=}+\frac{2M_{0}(\alpha-6)+r(4-\alpha)}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)^{2}\,,$ (20)

where it should be noted that also the second term in (III) is quadratic in
$\delta M/M$ being the product of a term $\delta r/M_{0}$ and of a term
$\delta M/M$. Hence, as anticipated, the first modification of the effective
potential arises at the quadratic order in $\delta M/M$ Requiring now that for
the light-ring $H(r)={\rm const.}=H(R_{\rm in})$, yields

$\delta M=\displaystyle\frac{\sqrt{9-3\alpha}+3}{3\alpha}\delta
r+\mathcal{O}\left(\delta r^{2}\right)\,.$ (21)

This result is compatible with our linearity assumption $\delta r(\delta
M)=\alpha\delta M$ in Eq. (21) provided that the value of the constant is
$\alpha=3$. In summary, we have shown that it is possible to create a self-
gravitating “light-region” that extends from the position of the original
light-ring $R_{\rm in}$ up to $R_{\rm out}$ and whose effective potentials
reads

$\displaystyle H(r)=$
$\displaystyle\frac{1}{r^{2}}\left(1-\frac{2M_{0}}{r}\right)+\frac{2(r-3M_{0})}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)$ $\displaystyle-\frac{6M_{0}-r}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)^{2}\,.$ (22)

The corresponding potential is shown with a black solid line in the right
panel of Fig. 1 and it is then easy to appreciate how we have essentially
created a smooth region of light-rings where the potential is flat at the
order considered in the region $R_{\rm in}\leq r\leq R_{\rm out}$, hence
leading to an infinite sequence of neutral circular photon orbits in such a
region, i.e., with zero first and second derivative of the effective
potential.

We can now ask the same question: is this configuration stable? Obviously, a
photon on a circular orbit well inside the light-region, i.e., at $R_{\rm
in}\lesssim r\lesssim R_{\rm out}$, and that is perturbed, would still
encounter in its neighbourhood other circular orbits and hence remain in the
newly and perturbed location. However, it is also easy to conclude that the
whole light-region is dynamically unstable after a small change in the mass of
the black hole. Consider in fact that the mass of the spacetime is changed by
a small but finite amount $\delta M_{0}$ without changing the mass in the
light-rings, namely, consider the new mass profile $M(r)=M_{0}+\delta
M_{0}+\delta M(r)$ with both $M_{0}$ and $\delta M_{0}$ constants. The new
effective potential will be

$\displaystyle H(r)$
$\displaystyle=\frac{1}{r^{2}}\left(1-\frac{2M_{0}}{r}\right)+\frac{2(r-3M_{0})}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)$
$\displaystyle\phantom{=}-\frac{6M_{0}-r}{r^{3}}\left(\frac{\delta
M}{M_{0}}\right)^{2}-\frac{2\delta M_{0}}{r^{3}}\,,$ (23)

where the first three terms, which are the same appearing in Eq. (III), have a
combined zero radial derivative, so that, in practice:
$\partial_{r}H(r)={6\delta M_{0}}/{r^{4}}>0$. Stated differently, a small
perturbation in the central black hole would completely destabilise the
circular orbits in the light-region, dispersing all the photons carefully
accumulated there.

##  IV Conclusion

Light-rings represent one of the most interesting aspects of the spacetime
around compact objects in general and of black holes in particular. Being
characterised by circular but unstable orbits, they represent a powerful tool
to probe the spacetime near these objects and test different theories of
gravity, both in terms of electromagnetic and gravitational radiation. So far,
however, the self-gravity of light-rings has been neglected and this is
because one expects that, on average, the number of photons entering the
light-ring equals that of photons leaving the light-ring, and that such a
number is not sufficient to modify the spacetime appreciably. We have here
explored what would happen when this assumption is not true and for this we
considered ways in which the light-ring in a Schwarzschild spacetime can be
filled with carefully injected photons so as to make the light-ring self-
gravitate.

In particular, we have considered two different routes to assigning a nonzero
mass to the light-ring and that are either based on a discrete concentration
of photons on a specific radial location or on suitable distribution of
photons in a given region. Notwithstanding a number of differences between the
two scenarios, we have found that, in both cases, the inclusion of the energy
contribution from the accumulated photons leads to the generation of new
light-rings. Such new light-rings can either appear at well-defined but
discrete locations, or be fused with zero separation in a well-defined region.
Interestingly, and somewhat disappointingly, all light-ring configurations
considered here are unstable and a small perturbation, either via the
inclusion on an additional photon onto the light-ring or via the absorption of
a photon by the black hole, leads to a catastrophic destruction of the light-
ring structures and to the dispersion of the photons to spatial infinity or
inside the black hole. Hence, the answer to the question in the title is:
“yes, light-rings can self-gravitate, but they are dynamically unstable”.

This work could be extended in a number of different ways. First, it would be
useful to perform a rigorous perturbative analysis that would provide, among
other things, the typical timescale of the instability. Second, although these
spacetimes are unstable, the fact that they can be described analytically
makes them useful tools to study the response of non-trivial black-hole
configurations to various perturbations and explore, for instance, the
properties of quasi-normal modes or the emergence of echos as different light-
rings are involved in the perturbation. Third, while we do not expect that the
outcome would change in the presence of rotation, the work presented here
could be extended by considering Kerr spacetimes and the corresponding light-
rings there. Finally, the exploration of self-gravity started here does not
require the presence of an horizon and can be extended to horizonless
ultracompact objects as long as they possess an unstable light-ring.
Interestingly, these compact objects also possess stable light-rings Mazur and
Mottola (2001); Cunha  _et al._ (2017); Jampolski and Rezzolla (2024); Di
Filippo (2024). While the analysis for the unstable light-ring would not
deviate from the one studied here, novel and potentially intriguing results
could emerge regarding self-gravitating stable light-rings. As a final remark,
we note that while the scenarios explored in this paper are so idealised to
require the intervention of an advanced civilisation, they actually provide
important information on the fundamental properties of realistic light-rings
in general relativity.

## Acknowledgements

It is a pleasure to thank R. Carballo-Rubio, P. Cunha, C. Ecker, C. Herdeiro,
P. Kocherlakota, R. Kumar Walia, D. Jampolski, S. Liberati, and P. Schupp for
interesting discussions and suggestions. FDF acknowledges financial support
from the PRIMUS/23/SCI/005 and UNCE24/SCI/016 grants by Charles University,
and the GAR 23-07457S grant from the Czech Science Foundation. LR acknowledges
financial support from the State of Hesse within the Research Cluster ELEMENTS
(Project ID 500/10.006), from the ERC Advanced Grant “JETSET: Launching,
propagation and emission of relativistic jets from binary mergers and across
mass scales” (Grant No. 884631), and from the Walter Greiner Gesellschaft zur
Förderung der physikalischen Grundlagenforschung e.V. through the Carl W.
Fueck Laureatus Chair.

## References

  * Chirenti and Rezzolla (2007) C. B. M. H. Chirenti and L. Rezzolla, Class. Quant. Grav. 24, 4191 (2007), arXiv:0706.1513 [gr-qc] .
  * Vincent  _et al._ (2016) F. H. Vincent, Z. Meliani, P. Grandclement, E. Gourgoulhon, and O. Straub, Class. Quant. Grav. 33, 105015 (2016), arXiv:1510.04170 [gr-qc] .
  * Glampedakis  _et al._ (2017) K. Glampedakis, G. Pappas, H. O. Silva, and E. Berti, Phys. Rev. D 96, 064054 (2017), arXiv:1706.07658 [gr-qc] .
  * Cardoso  _et al._ (2017) V. Cardoso, E. Franzin, A. Maselli, P. Pani, and G. Raposo, Phys. Rev. D 95, 084014 (2017), [Addendum: Phys.Rev.D 95, 089901 (2017)], arXiv:1701.01116 [gr-qc] .
  * Jai-akson  _et al._ (2017) P. Jai-akson, A. Chatrabhuti, O. Evnin, and L. Lehner, Phys. Rev. D 96, 044031 (2017), arXiv:1706.06519 [gr-qc] .
  * Glampedakis and Pappas (2018) K. Glampedakis and G. Pappas, Phys. Rev. D 97, 041502 (2018), arXiv:1710.02136 [gr-qc] .
  * Cardoso  _et al._ (2016) V. Cardoso, E. Franzin, and P. Pani, Phys. Rev. Lett. 116, 171101 (2016), [Erratum: Phys.Rev.Lett. 117, 089902 (2016)], arXiv:1602.07309 [gr-qc] .
  * Carballo-Rubio  _et al._ (2018) R. Carballo-Rubio, F. Di Filippo, S. Liberati, and M. Visser, Phys. Rev. D 98, 124009 (2018), arXiv:1809.08238 [gr-qc] .
  * Cardoso and Pani (2019) V. Cardoso and P. Pani, Living Rev. Rel. 22, 4 (2019), arXiv:1904.05363 [gr-qc] .
  * Völkel  _et al._ (2021) S. H. Völkel, E. Barausse, N. Franchini, and A. E. Broderick, Class. Quant. Grav. 38, 21LT01 (2021), arXiv:2011.06812 [gr-qc] .
  * Kocherlakota  _et al._ (2021) P. Kocherlakota _et al._ (Event Horizon Telescope), Phys. Rev. D 103, 104047 (2021), arXiv:2105.09343 [gr-qc] .
  * Völkel  _et al._ (2022) S. H. Völkel, N. Franchini, E. Barausse, and E. Berti, Phys. Rev. D 106, 124036 (2022), arXiv:2209.10564 [gr-qc] .
  * Eichhorn  _et al._ (2023) A. Eichhorn, A. Held, and P.-V. Johannsen, JCAP 01, 043 (2023), arXiv:2204.02429 [gr-qc] .
  * Moreira  _et al._ (2024) Z. S. Moreira, C. A. R. Herdeiro, and L. C. B. Crispino, Phys. Rev. D 109, 104020 (2024), arXiv:2401.05658 [gr-qc] .
  * Pedrotti and Vagnozzi (2024) D. Pedrotti and S. Vagnozzi, (2024), arXiv:2404.07589 [gr-qc] .
  * Genzel  _et al._ (2024) R. Genzel, F. Eisenhauer, and S. Gillessen, Astron. Astrophys. Rev. 32, 3 (2024), arXiv:2404.03522 [astro-ph.GA] .
  * Akiyama  _et al._ (2019) K. Akiyama _et al._ (Event Horizon Telescope), Astrophys. J. Lett. 875, L1 (2019), arXiv:1906.11238 [astro-ph.GA] .
  * Akiyama  _et al._ (2022a) K. Akiyama _et al._ (Event Horizon Telescope), The Astrophysical Journal Letters 930 (2022a).
  * Akiyama  _et al._ (2022b) K. Akiyama _et al._ (Event Horizon Telescope), Astrophys. J. Lett. 930, L17 (2022b), arXiv:2311.09484 [astro-ph.HE] .
  * Kocherlakota and Rezzolla (2022) P. Kocherlakota and L. Rezzolla, Mon. Not. Roy. Astron. Soc. 513, 1229 (2022), arXiv:2201.05641 [gr-qc] .
  * Abbott and et al (2016) B. P. Abbott and et al (LIGO Scientific Collaboration and Virgo Collaboration), Phys. Rev. Lett. 116, 061102 (2016).
  * Abbott  _et al._ (2019) B. P. Abbott _et al._ (LIGO Scientific, Virgo), Phys. Rev. D 100, 104036 (2019), arXiv:1903.04467 [gr-qc] .
  * Abbott  _et al._ (2021a) R. Abbott _et al._ (LIGO Scientific, Virgo), Phys. Rev. D 103, 122002 (2021a), arXiv:2010.14529 [gr-qc] .
  * Abbott  _et al._ (2021b) R. Abbott _et al._ (LIGO Scientific, VIRGO, KAGRA), (2021b), arXiv:2112.06861 [gr-qc] .
  * Cunha  _et al._ (2016) P. V. P. Cunha, J. Grover, C. Herdeiro, E. Radu, H. Runarsson, and A. Wittig, Phys. Rev. D 94, 104023 (2016), arXiv:1609.01340 [gr-qc] .
  * Cunha  _et al._ (2017) P. V. P. Cunha, E. Berti, and C. A. R. Herdeiro, Phys. Rev. Lett. 119, 251102 (2017), arXiv:1708.04211 [gr-qc] .
  * Di Filippo (2024) F. Di Filippo, (2024), arXiv:2404.07357 [gr-qc] .
  * Birkhoff and Langer (1923) G. D. Birkhoff and R. E. Langer,  _Relativity and modern physics_ (1923).
  * Berry  _et al._ (2022) T. Berry, A. Simpson, and M. Visser, Phys. Rev. D 106, 084001 (2022), arXiv:2207.02465 [gr-qc] .
  * Nagar and Rezzolla (2005) A. Nagar and L. Rezzolla, Class. Quantum Grav. 22, R167 (2005), gr-qc/0502064 .
  * Mazur and Mottola (2001) P. O. Mazur and E. Mottola, arXiv e-prints , gr-qc/0109035 (2001), arXiv:gr-qc/0109035 [gr-qc] .
  * Jampolski and Rezzolla (2024) D. Jampolski and L. Rezzolla, Classical and Quantum Gravity 41, 065014 (2024), arXiv:2310.13946 [gr-qc] .


# A Step Toward More Inclusive People Annotations for Fairness

Candice Schumann  GoogleUnited States<EMAIL_ADDRESS>,  Susanna Ricco
GoogleUnited States<EMAIL_ADDRESS>,  Utsav Prabhu  GoogleUnited States
<EMAIL_ADDRESS>,  Vittorio Ferrari  GoogleSwitzerland
<EMAIL_ADDRESS>and  Caroline Pantofaru  GoogleUnited States
<EMAIL_ADDRESS>
(2021)

###### Abstract.

The Open Images Dataset (Kuznetsova et al., 2020) contains approximately 9
million images and is a widely accepted dataset for computer vision research.
As is common practice for large datasets, the annotations are not exhaustive,
with bounding boxes and attribute labels for only a subset of the classes in
each image. In this paper, we present a new set of annotations on a subset of
the Open Images dataset called the _MIAP (More Inclusive Annotations for
People)_ subset, containing bounding boxes and attributes for all of the
people visible in those images. The attributes and labeling methodology for
the _MIAP_ subset were designed to enable research into model fairness. In
addition, we analyze the original annotation methodology for the _person_
class and its subclasses, discussing the resulting patterns in order to inform
future annotation efforts. By considering both the original and exhaustive
annotation sets, researchers can also now study how systematic patterns in
training annotations affect modeling.

datasets, fairness, computer vision

††journalyear: 2021††copyright: rightsretained††conference: Proceedings of the
2021 AAAI/ACM Conference on AI, Ethics, and Society; May 19–21, 2021; Virtual
Event, USA††booktitle: Proceedings of the 2021 AAAI/ACM Conference on AI,
Ethics, and Society (AIES ’21), May 19–21, 2021, Virtual Event, USA††doi:
10.1145/3461702.3462594††isbn: 978-1-4503-8473-5/21/05††ccs: Computing
methodologies Computer vision††ccs: Computing methodologies Machine
learning††ccs: Social and professional topics User characteristics

##  1\. Introduction

The Open Images Dataset (Kuznetsova et al., 2020) contains approximately 9
million images with annotations supporting research into image classification,
object detection and instance segmentation, visual relationship detection, and
spatially localized voice and text descriptions of the image contents. It is
widely accepted as a key dataset to both measure and advance the state of the
art in image and scene understanding.

In this paper, we introduce a new set of annotations, released as the _MIAP
(More Inclusive Annotations for People)_ collection in Open Images Extended,
which improve the bounding box annotations for the five original classes in
the _person_ subtree. The annotations use a new labeling protocol designed to
produce annotations that enable fairness analysis. With the increasing focus
on reducing unfair bias as part of responsible AI research, we hope these
annotations will encourage researchers already leveraging Open Images to
incorporate fairness analysis in their research.

Open Images contains image-level labels for more than 19,000 classes, and
bounding box annotations for a subset of 600 of those classes. Even for those
600 classes, it is infeasible to exhaustively label each of them as present
(positive) or absent (negative) in every image. Hence, Open Images annotations
are not exhaustive, as is common practice for large scale computer vision
datasets, e.g. (Lin et al., 2014; Deng et al., 2009). Each image is associated
with a small set of positive labels and a small set of negative labels. These
labels are proposed by an ensemble of image classifiers (e.g., labels produced
with confidence above a threshold by some algorithm) and then verified as
either positive or negative by a human annotator. The presence/absence of any
label not proposed by the initial image classifier is simply unknown.

After this stage of producing image-level labels, annotators then draw
bounding boxes for the classes at the most specific level of the 600-class
hierarchy (leaves) that are included in the image-level positive label set.

Of the 600 classes with bounding-boxes, five classes are part of the _person_
hierarchy: _person_ , _man_ , _woman_ , _boy_ , and _girl_.

Here, Open Images departs from other large-scale object detection datasets
(Lin et al., 2014; Russakovsky et al., 2015) by including these subclasses.
Although most applications require annotations for only the encompassing
_person_ class, the availability of gender- and age range-specific labels
would in theory allow for use of these labels for fairness analysis and bias
mitigation. However, we found the specific labeling procedure used impacted
the accuracy of these subgroup labels and resulted in non-exhaustive bounding
box annotations for the _person_ superclass for some images.

Our new _MIAP_ annotations collapse the gender- and age range-specific
subclasses into a single _person_ superclass at annotation time. We ask
annotators to draw bounding boxes around _all_ people appearing in an image
with any positive image-level label of either _person_ , _man_ , _woman_ ,
_boy_ , or _girl_. _Person_ boxes are then annotated with additional
attributes corresponding to the perceived age range and perceived gender
presentation of the individual. This procedure adds a significant number of
boxes that were previously missing due to complex interactions between the
scene content, the characteristics of the individual as interpreted by the
initial image classifier, and the perception of two sets of human annotators.
Our boxes give more complete ground truth for training a person detector as
well as more accurate subgroup labels for fairness analysis and bias
mitigation.

###  1.1. Summary of Contributions

The main contributions of this paper are:

  * •

An analysis of the original Open Images annotation pipeline’s impact on the
completeness of person-related box annotations.

  * •

An updated annotation procedure to produce more complete annotations for the
_person_ class.

  * •

The resulting annotations on a subset of images and a comparative analysis of
the differences between the two annotation types. We publicly release our new
annotations.

###  1.2. Intended Use

We include annotations for perceived age range and gender presentation for
_person_ bounding boxes because we believe these annotations are necessary to
advance our field’s ability to better understand and work to mitigate and
eliminate unfair bias or disparate performance across protected subgroups. We
note that the labels capture the gender and age range presentation as assessed
by a third party based on visual cues alone, rather than an individual’s self-
identified gender or actual age. We do not support or condone building or
deploying gender and/or age presentation classifiers trained from these
annotations as we believe the risks associated with the use of these
technologies outside fairness research outweigh any potential benefits. We
refer readers to the excellent work detailing these potential risks (_e.g._ ,
(Hamidi et al., 2018; Keyes, 2018)) for more information.

##  2\. Related Work

Research into techniques to improve fairness in computer vision algorithms is
of increasing importance, as these systems become widely used. Existing
approaches span the model development cycle, including (a) careful formulation
of the task, (b) identifying and reducing potential biases in training
datasets, (c) inventing techniques for training fairer models in spite of
biased datasets, (d) defining and evaluating whether the resulting system
achieves some desired fairness property. This work is mainly focused on (b)
and (d).

###  2.1. Identifying and reducing bias in datasets

Bias can manifest in a dataset as unintended consequence of design choices
made during dataset construction. Datasets containing people are particularly
sensitive to these biases, and can have harmful downstream implications - such
as resulting in widely-used systems that have unequal performance across
population demographics. For example, datasets that sample images from
internet sources may over-represent some geographical regions compared to
others. Both Open Images and its predecessor ImageNet have been shown to
contain a disproportionate number of images from North America and Europe
(Shankar et al., 2017; de Vries et al., 2019). Classifiers trained on this
distribution were more likely to misclassify images from the developing world
for both person-related classes (e.g., _bridegroom_) (Shankar et al., 2017)
and object classes (e.g., _spices_) (de Vries et al., 2019).

To partially address this limitation and encourage further research (Atwood et
al., 2020), Open Images now includes the Crowdsourced
Extension111https://storage.googleapis.com/openimages/web/extended.html,
containing new images from countries underrepresented in the original set.
This Open Images extension is focused on image-level labels for non-human
classes. The faces of people appearing in this set are blurred, making it less
suitable for analysis of person detectors.

Of course, the potential for bias in large web-scale datasets goes far beyond
skewed geographic distributions. The exceptional work updating the person
subtree of ImageNet (Yang et al., 2020) provides detailed analysis of sources
of bias, from the definitions of class labels to the distribution of people
representing given classes. Their work focuses on correcting image-level
labels while we chose to focus on the bounding box annotations for people in
Open Images. With our updated annotations, we provide the attribute labels
similar to those annotated at the image level in ImageNet. We hope future
research can leverage both datasets to build on this work, improving these and
future large-scale computer vision datasets.

###  2.2. Identifying and reducing bias in trained models

Creating better balanced and less biased datasets are steps toward the
eventual goal of ML models that work well for everyone, whose predictions do
not unfairly disadvantage any segment of the population. Most definitions of
model fairness (_e.g._ , equality of opportunity (Hardt et al., 2016)) require
partitioning an evaluation set by the attributes to which model performance
should be agnostic. Proposed standards for model transparency reporting
(Mitchell et al., 2019), which complement proposed transparency reports for
datasets (Gebru et al., 2018), advocate for always reporting performance
metrics broken down by relevant population groups, independent of specific
fairness goals. Finally, many bias mitigation strategies require attribute
labels on at least a portion of the model’s training set (Beutel et al., 2019;
Madras et al., 2018).

There is a rich history of datasets containing images of people that include
attribute annotations such as perceived gender and/or age. For some, these
annotations were added for the explicit purpose of furthering ML fairness
research. For example, the Pilot Parliaments Benchmark from Gender Shades
(Buolamwini and Gebru, 2018) used gender and skin type annotations to
highlight disparate performance of available gender classifiers across
intersectional groups. Similarly, datasets for training or evaluating face
recognition algorithms may include annotations for fairness analysis (Watson,
2016; Grother et al., 2018). In other cases, these annotations were initially
curated as training data for attribute classifiers (e.g., Celeb-A (Liu et al.,
2015), FairFace (Karkkainen and Joo, 2021), UTKFace (Zhang et al., 2017)) but
may also be useful for studying bias (Ryu et al., 2017; Alvi et al., 2018).

The attribute labels we provide differ from previous datasets in two key
areas. First, our annotations do not rely on machine learned models for
annotating either perceived gender or age range. Other large-scale annotation
efforts exploring bias in ImageNet representation (Dulhanty and Wong, 2019) or
attempting to quantify the diversity in appearance across the population
(Merler et al., 2019) resort to attribute labels produced by trained attribute
classifiers. We prefer to reduce the influence of potentially biased attribute
classifiers, especially when using these annotations for fairness
applications. Second, and importantly, most other datasets with similar
annotations for fairness applications are targeted toward either face
attribute estimation or face recognition as their primary applications. As a
result, they contain a single face of interest per image. In contrast, we
augment a portion of the existing Open Images dataset, thus adding fairness
annotations to images with multiple people per image, including for people
whose faces are not clearly visible.

To our knowledge, the only other dataset containing multiple people in complex
scenes that has been augmented with attribute annotations for the purpose of
fairness research is BDD100k (Wilson et al., 2019). This work annotates
Fitzpatrick skin type for localized pedestrians of sufficient size in the
images. Skin tone is an important attribute to consider for fairness, and one
that our work does not address. While both datasets can be used for evaluating
whether a person detector performs consistently across population slices,
BDD100k is limited to self-driving applications. In contrast, our annotations
also provide data for fairness analysis of a broad set of scene understanding
tasks.

Figure 1. The Open Images annotation pipeline from raw images to labeled
object bounding boxes.

##  3\. Analysis of the Original Open Images

In this section we take a deep dive into the effects of the partially
annotated ground truth label generation process on the annotation of people
boxes. Prior to this deep dive we do want to mention that the original Open
Images dataset functions correctly as a standard partial ground truth dataset.
However, due to the non-exhaustive nature of the annotations, fairness
evaluations or bias mitigation using this dataset could provide potentially
misleading results.

###  3.1. The Annotation Pipeline

Open Images employs a two-stage process, first annotating image-level labels
and then object bounding-boxes based on them (Figure 1). In total, while Open
Images considers a very large set of 19,794 possible classes at the image-
level, object bounding-boxes are annotated for 600 of them. We focus on these
from now on.

Given an image, an image classifier is used to generate potential positive and
negative labels. These potential labels are then sent to annotators to verify.
Each potential label is either marked as positive (i.e. the named object class
is present in the image) or negative (i.e. the named object class does not
appear in the image). The presence or absence of any label that is not
proposed by the image classifier is considered simply unknown (i.e. it is
neither positive nor negative).

Annotators are then asked to draw bounding-boxes around objects for the
positive image labels, focusing on the most specific labels in the hierarchy
(Figure 2). For example, if an image has positive labels of “car”,
“limousine”, and “screwdriver”, the annotators box all limousines and
screwdrivers (since limousine is more specific than car in the hierarchy).
This means that if the image contains a car that is not a limousine, there
will not be a box drawn around it. Moreover, there will also be no boxes for
classes that are unknown to be present or absent in the image (i.e. not
proposed by the initial classifier, see above).

Note that this partial annotation protocol is working as intended – it would
be a huge undertaking to box everything in every image and partial labeling is
standard practice (Kuznetsova et al., 2020; Deng et al., 2009; Lin et al.,
2014). The Open Images protocol enables users to reconstruct the list of
classes for which boxes might be missing, and take this into account when
training or evaluating detectors (i.e. all unknown labels for an image, plus
all positive labels which are not the most specific for that image). That
being said, partial annotation can have unintended consequences, especially
when annotating people.

Figure 2. Visualization of the Open Images hierarchy of classes that are boxed
(Kuznetsova et al., 2020) to demonstrate its scope and complexity.

###  3.2. Annotation Pipeline Mixing with Societal Norms

Many standard academic datasets for object detection (such as MSCOCO (Lin et
al., 2014) and ImageNet (Russakovsky et al., 2015)) contain a single _person_
class. Instead, the Open Images hierarchy contains five classes that
correspond to localized people: _person_ , _man_ , _woman_ , _boy_ , and
_girl_. While these gender- and age-specific labels are relevant and useful
for some applications (such as fairness evaluations or bias mitigation), for
others, the additional specificity is unnecessary and even undesirable.

The most obvious approach for training a detector with a single gender- and
age-agnostic _person_ class would be to simply drop _man_ , _woman_ , _boy_ ,
_girl_ from the label set. However, because the hierarchical labeling process
preferred labeling for a subclass when both superclass and subclass appeared
in the positive image-level labels, dropping annotations for these four
subclasses will result in dropping many ground truth boxes. For example, a
training image containing a single man, with positive image-level labels for
both _person_ and _man_ will contain a box labeled as _man_ but no
corresponding box labeled as _person_ will exist.

Another option is to collapse the hierarchy to a single class, treating boxes
labeled as _man_ , _woman_ , _boy_ , _girl_ as _person_ boxes. This remapping
could happen as a preprocessing step before training or as postprocessing at
inference time. For the moment, we focus on remapping prior to training and
defer a discussion of postprocessing to Section 3.3.

The previous example is resolved correctly with label preprocessing, with the
_man_ box remapped to _person_ providing correct ground truth for that image.
Unfortunately, this procedure does not always result in a complete set of
bounding boxes at training time, which requires localizing every person in the
image. For a person appearing in an image to have a corresponding bounding box
after label remapping given the original annotation process, one of two things
had to be true:

  * •

The image classifier proposed _person_ as a potential image-level label for
the image without proposing any gender- and age-specific subclass that the
annotators marked as positive, _or_

  * •

The image classifier must have proposed the subclass that both the image-level
annotator and box-level annotators agreed described the specific person.

We show a few examples below where these conditions are not satisfied.

Figure 3. Image with true image-level labels _person_ , _woman_ , _suit_.
Figure 4. Image with true image-level labels _person_ , _man_ , _suit_ ,
_clothing_.

#### Gender Presentation

After-the-fact label manipulation can cause individuals of different genders
to be treated differently in the resulting annotations.

Consider the case of the image in Figure 3. The following image-level labels
were proposed and marked positive: _person_ , _woman_ , and _suit_. Using the
hierarchical labeling strategy, annotators were asked to draw boxes around
objects of the classes _woman_ and _suit_ , but not _person_. As a result, the
annotations for this image are missing the bounding boxes around the men
wearing the suits.

Note that this problem can also happen in the other direction. See Figure 4,
where the image-level label set includes _man_ and _person_ , but not _woman_.

The cause of the missing boxes in these examples is obvious: the imbalance of
the specificity of the image-level labels proposed for inclusion in the
positive set, coupled with the decision to prefer subclasses for box
annotations. But subtle omissions can occur even in cases where the positive
image-level labels include all five _person_ -related classes.

Consider another image containing five people: one man, one woman, one girl,
one boy, and one person with unidentifiable or nonconforming gender
presentation. Assume (unlike the previous example), that the image classifier
successfully proposed all relevant labels: _person_ , _man_ , _woman_ , _boy_
, and _girl_. Bounding-box annotators would be instructed to draw boxes around
individuals corresponding to the four subclasses. Depending on the specific
decisions made by the annotators for this image, this process could
potentially miss annotations for people that do not fit into a gender-
stereotyped role.

Figure 5. Image with a negative label for _girl_ and no _woman_ replacement

#### Age and language

Similar problems arise in instances where the image classifier and annotators
perceive the age of the image subjects differently. In Figure 5, we show an
example where an age-dependent image-level label (_girl_) is labeled as
negative with no replacement of the age-appropriate label (_woman_). The
result is similar to Figure 4, with a bounding box annotation for only one of
the two people in the image.

In Figure 6, the _girl_ image-level label is considered positive, but an
annotator only perceived that label as applying to two people in the image. As
a result, several people of similar ages do not have boxes drawn around them.

We observed that phenomena of this type were more likely to cause missed
bounding boxes for images with boxes for _girl_ or _woman_ than for images
with _man_ or _boy_. This may be the result of the difference in how these
terms are used and understood by society. The word _girl_ can be used to
describe a female of any age while _boy_ typically describes only a child or
adolescent.222For example, Wikipedia defines _girl_ as “A girl is a young
female human, usually a child or an adolescent. When she becomes an adult, she
is described as a woman. The term girl may also be used to mean a young woman,
and is sometimes used as a synonym for daughter. Girl may also be a term of
endearment used by an adult, usually a woman, to designate adult female
friends.” (Wikipedia contributors, 2021) while _boy_ is defined as “A boy is a
young male human. The term is usually used for a child or an adolescent. When
a male human reaches adulthood, he is described as a man.” (Wikipedia
contributors, 2020)

Figure 6. Several people of similar age, only two have bounding boxes drawn
around them.

(a) Weddings

(b) Military

Figure 7. Image context and gender norms sometimes dictate which image-level
labels are flagged.

#### Potential correlation with image context

Because the image content influences which labels are proposed in the
classification step of the process (step 2 in Figure 1), it affects which
boxes are included in the annotations. Partial annotations may be more
prevalent, and also potentially more concerning, in scenes that correspond to
existing societal stereotypes. Take, for example, the three images in Figure
7. In traditional heterosexual European and US-style weddings the focus of the
day is generally placed on the bride. This emphasis is reinforced in the
annotations, with a bounding box for the woman in Figure 7(a) but not for the
man. As another example, in the US the majority of military personnel are male
(DoD, 2018); a prior that is reinforced in the missing bounding box annotation
for the woman in uniform in Figure 7(b).

###  3.3. Potential risks of direct use of non-exhaustive annotations

A person detector trained on these partial annotations may learn to imitate
the non-exhaustive patterns in the training data. As a result, a trained
detector may be less likely to correctly detect individuals from certain
subpopulations in certain contexts. This is also true for detectors that rely
on postprocessing to collapse predictions of _man_ , _woman_ , _boy_ , and
_girl_ to _person_ , where correct output would rest on the ability of a
specific classifier (e.g., _woman_) to generalize to scene contexts that may
not have been sufficiently represented in the non-exhaustive training data.

The severity of such errors depends on the use case in question, ranging from
annoyances like a less efficient experience searching through personal image
collections to highly consequential failures in life safety systems (e.g.,
pedestrian detection in assistive driving features).

The degree to which trained models are resilient to noisy annotations remains
an active area of research. In particular, we do not yet understand the
effects that _structured patterns_ of the type we observe here have on model
performance; typically studies of robustness in the presence of noise make
simplifying assumptions and experiment with annotations that are modified or
missing uniformly at random (Wu et al., 2018). Especially concerning here is
that the ground truth in the validation and test sets are also non-exhaustive,
so it may be difficult to detect disparate performance of a model inheriting
bias from these annotations. A detailed study of these important questions
warrants significant future work that our _MIAP_ annotations are designed to
support.

##  4\. More Inclusive Annotations for People

To generate a subset with complete ground truth for the _person_ class and
enable related fairness research, we relabel 100K randomly sampled images
(70,000 from training and 30,000 from validation/test) that contained at least
one of the five _person_ classes (_man_ , _woman_ , _boy_ , _girl_ , or
_person_) in the positive image-level set. Annotators were asked to draw boxes
around all people in these images. They were then asked to provide two
different attributes for each person.

#### Perceived Gender Presentation

Annotators selected either _predominantly feminine_ , _predominantly
masculine_ , or _unknown_ to describe the human-perceived gender presentation
of an individual based on the visual cues in the image.

We recognize that gender is not binary and that an individual’s gender
identity may not match their perceived or intended gender presentation. We
chose to use _predominantly feminine_ and _predominantly masculine_ rather
than the historical labels _female_ and _male_ to acknowledge that an
individual’s gender presentation is complex while maintaining the ability to
analyze these labels in the context of the existing image-level and box labels
of _man_ , _woman_ , _boy_ , and _girl_. When used in aggregate, these
annotations are useful to assess how model performance may differ for people
who present gender differently.

We did not include _non-binary_ as a class label as it is not possible to
label gender identity from images. Gender identity should only be used in
situations where participants are able to self-report gender (Scheuerman et
al., 2020).

In an effort to mitigate the effects of unconscious bias on the annotations,
we reminded annotators that norms around gender expression vary across
cultures and have changed over time. Annotation guidelines included statements
like:

> No single aspect of a person’s appearance “defines” their gender expression.
> For example, a person may still present as predominantly masculine while
> wearing jewelry. Another may present as predominantly feminine while having
> short hair. Consider the entirety of a person’s appearance in the image and
> avoid rigid definitions based on cultural stereotypes.

In addition to the written instructions, we provided annotators with visual
examples that included people across a variety of ages and from different
cultures. These sets included visual examples to counteract common stereotypes
(e.g., including images of women with short hair).

Finally, annotators were instructed to select the _unknown_ label in
situations where there was not enough information in the image to make a
determination. Note that annotators could decide to select either
_predominantly feminine_ or _predominantly masculine_ even if an individual’s
face was not visible.

#### Perceived Age Range

Annotators selected either _young_ , _middle_ , _older_ , or _unknown_ to
describe the perceived age range of an individual based on their appearance in
the image. Annotators were instructed to prefer the older of two categories in
situations where there was enough information to form an impression but were
unsure of a boundary case. Someone who appears old enough to possibly belong
to _middle_ should be assigned that attribute label. Someone who appears old
enough to possibly belong to _older_ should be assigned that attribute label.

Annotators could select _unknown_ for perceived age range while selecting one
of the other three labels for perceived gender presentation. Similarly they
could select _young_ , _middle_ , or _older_ even if they annotated the same
bounding box as _unknown_ for perceived gender presentation.

Datasets | Train | Val | Test | Total
---|---|---|---|---
images sampled | 70,000 | 7,410 | 22,590 | 100,000
images with missing boxes | 23,241 | 1,836 | 5,397 | 30,474
Table 1. Distribution of the images selected for relabeling. The second row counts the number of images where the _MIAP_ annotations add boxes that do not exist in the original annotations. Datasets | Train | Validation | Test | Total
---|---|---|---|---
| original | _MIAP_ | original | _MIAP_ | original | _MIAP_ | original | _MIAP_
boxes | 294,695 | 363,165 | 15,398 | 22,060 | 47,777 | 69,106 | 357,870 | 454,331
_predominantly_ _feminine_ | 67,979 | 83,004 | 2,039 | 4,453 | 6,265 | 13,215 | 76,283 | 100,672
_predominantly_ _masculine_ | 127,468 | 141,300 | 3,779 | 7,898 | 12,073 | 24,849 | 143,320 | 174,047
_unknown_ gender presentation | 99,248 | 138,861 | 9,580 | 9,709 | 29,439 | 31,042 | 138,267 | 179,612
_young_ | 16,634 | 22,621 | 1,205 | 1,508 | 3,709 | 4,677 | 21,548 | 28,806
_middle_ | 178,813 | 190,179 | 4,613 | 10,591 | 14,629 | 32,904 | 198,055 | 233,674
_older_ | - | 7,176 | - | 471 | - | 1,376 | - | 9,023
_unknown_ age range | 99,248 | 143,189 | 9580 | 9490 | 29,439 | 30,149 | 138,267 | 182,828
Table 2. Comparison counts of boxes across the 100,000 samples from the
original Open Images dataset and the _MIAP_ dataset. Counts are split between
the training set, the validation set, and the testing set, with the totals
appearing in the last two columns. Gender presentation and age range
presentation for the original set was assumed via the class label (for
example, the class _woman_ would have gender presentation _predominantly
feminine_ and age range presentation _middle_). There was no class for _older_
in the original Open Images dataset.

##  5\. Analysis

It is important to note the differences between the original Open Images
dataset and the new _MIAP_ dataset to understand what has improved and what
still needs work.

##### Raw counts

In Table 1, we see that approximately 30% of the 100,000 images had additional
boxes added during the relabeling effort. This is quite a large percentage of
the images, underscoring how often the phenomena described in Section 3.2 did,
in fact, occur.

Table 2 compares the total number of boxes in the original annotations to the
_MIAP_ set, both in aggregate and broken down by attribute label. This
analysis assigns a bounding box in the original annotation the two attribute
labels according to the class labels (e.g., _woman_ maps to attribute labels
_predominantly feminine_ and _middle_).

There is a large increase in _unknown_ labels (for both gender presentation
and age range) in the training set but only a small increase in the validation
and test sets. This is explained by the difference in annotation procedure
between training and validation/test in the original protocol. The training
set used the process described in Section 3.1. For the validation and test
sets, annotators drew boxes for all positive image-level labels. This means
that if the label _person_ appeared in the positive image-level labels for a
validation or testing image, the annotator would be asked to draw generic
_person_ boxes, regardless of the presence of subclass labels. Boxes for which
annotators were not able to assign attribute labels are likely to have been
drawn in response to a prompt for _person_ boxes but not drawn when only a
more specific class was requested.

We see a significant increase for both _predominantly feminine_ and
_predominantly masculine_ labels across the training, validation, and test
splits. Although this labeling effort has decreased the gap between
_predominantly masculine_ and _predominantly feminine_ slightly, the dataset
is still unbalanced with the majority of gender presentation labels, after
_unknown_ , belonging to _predominantly masculine_. _Middle_ is by far the
most common of the age range attribute labels.

##### Statistical analysis

In order to measure the association of attribute labels with missing boxes we
use normalized pointwise mutual information (_nPMI_) (Bouma, 2009). Pointwise
mutual information is defined as

$\textit{PMI}(x,y)=\ln\frac{p(x,y)}{p(x)p(y)}$

where $x$ and $y$ are discrete random variables. Normalized pointwise mutual
information is therefore defined as

$\textit{nPMI}(x,y)=\frac{\textit{PMI}(x,y)}{-\ln p(x,y)}.$

We use nPMI to calculate the co-occurrences of images with missing boxes and a
given label (for example the co-occurrence of images with missing boxes and
gender presentation _predominantly feminine_), denoted as
$\textit{nPMI}(\text{Missing},\text{label})$, as well as the co-occurrence of
images without missing boxes and a given label (for example the co-occurrence
of images without missing boxes and gender presentation _predominantly
feminine_), denoted as $\textit{nPMI}(\text{Not Missing},\text{label})$. Using
these two co-occurrences we can take the difference

$\textit{nPMI}(\text{Missing},\text{label})-\textit{nPMI}(\text{Not
Missing},\text{label})$

to find labels that are more likely to co-occur with images that are missing
people boxes than images that are not missing people boxes in the original
annotations.

Table 3 shows these results in order of significance. Unsurprisingly _unknown_
labels for both age range and gender presentation are much more likely to be
found in images with missing boxes. This is mostly likely due to the training
set labeling process only annotating the most specific _person_ classes.
Additionally, the label _predominantly feminine_ is much more likely to co-
occur with missing boxes than the label _predominantly masculine_ , possibly
due to the syntactic confusion between _woman_ and _girl_. The age range
labels exhibit the least amount of skew between images with missing boxes and
those without.

Label | nPMI difference
---|---
gender presentation _unknown_ | 0.458
age range _unknown_ | 0.453
_predominantly feminine_ | 0.365
_predominantly masculine_ | 0.251
_young_ | 0.236
_middle_ | 0.156
_older_ | 0.155
Table 3. nPMI differences between images with missing boxes and images without
missing boxes
($\textit{nPMI}(\text{Missing},\text{label})-\textit{nPMI}(\text{Not
Missing},\text{label})$) in order of significance. A high nPMI difference
indicates that images with missing boxes were more likely to have new boxes
with the given labels.

##### Empirical visualization

In addition to exploring raw counts and statistical analysis, we visualized
the differences between the original Open Images and the _MIAP_ annotations.
Figure 8 shows four examples. In each image, the original Open Images had at
least one person not annotated. The new _MIAP_ dataset has bounding box
annotations for all people.

(a)

(b)

(c)

(d)

Figure 8. Examples of new boxes in _MIAP_. In each subfigure the magenta boxes
are from the original Open Images dataset, while the yellow boxes are new
boxes from the _MIAP_ dataset.

##  6\. Discussion

Exhaustively annotated, large-scale visual data is particularly valuable,
since it often enables new types of analyses that are ineffectively
addressable by partial annotations. With the exhaustive box annotations for
the _person_ class, we hope to close this gap for fairness-related analysis of
people-centric ML models. We appreciate and encourage its use for tasks such
as disaggregating/stratifying metrics along age range and gender presentation
slices, in the hope that such analyses will eventually lead to less biased
models, classifiers, and workflows that work equitably for all.

It is important to note that the gender presentation and age range
presentation labels, while useful for evaluation purposes, do not represent
gender identity or actual age of the people. This data should not be used to
create gender and/or age classifiers. As a further precaution, a gender
presentation of _unknown_ has been set for any box labeled as _young_ in the
released dataset333Analysis included in this paper includes gender
presentation annotations for people marked as _young_..

###  6.1. Limitations

The continually evolving and expanding nature of Internet imagery implies that
the distribution of Flickr images included within the original Open Images
dataset (and consequently, the subset included in this relabeling effort) may
not accurately represent the uses of models trained or evaluated using this
data. Additionally, recent works have demonstrated that the geographical
sampling of Flickr images as well as the use of English as the primary
language for dataset construction and taxonomy definition result in inherent
cultural bias within the datasets (de Vries et al., 2019).

While this effort makes initial strides towards relabeling the dataset, there
is much work yet to be done. Several parts of the original annotation pipeline
for the Open Images dataset were not addressed - including image selection,
object class hierarchy, determination of classes that are boxed, and
annotation of relationship triplets. Our image selection task rejected images
which did not contain at least one _person_ -subclass label, which may have
propagated/amplified any biases that may have existed in the labeling workflow
for the original dataset. While this effort has slightly aided in balancing
the bounding box annotations, there remains an imbalance towards a gender
presentation of _predominantly masculine_. This relabeling effort did not
improve age range imbalance. Owing to the large amount of manual effort
required to exhaustively label each image, this task was also restricted only
to certain _person_ labels (_man_ /_woman_ /_girl_ /_boy_ /_person_), but not
all _person_ -related labels that exist in the Open Images taxonomy (e.g.
parts of the human body, activities, clothing, etc). Other, non-_person_
-related classes in the Open Images taxonomy may also benefit from similar
analysis and exhaustive relabeling, but were not addressed in this paper.
Relationship triplets were not re-annotated, so they only exist for the subset
of boxes that were included in the original dataset.

We acknowledge that there are trade-offs between providing the capability to
do fairness analysis based on societal concepts of perceived gender, and
perpetuating binary definitions of gender. Future work could include analysis
of objective attributes that can confound computer vision models, such as
makeup or facial hair.

##  7\. Conclusion

Dataset construction, annotation procedures, fairness analysis and techniques
for modeling under noisy ground truth conditions are all continually evolving
areas of research.

In this paper, we have presented a new annotation procedure and the resulting
annotations for a subset of the Open Images dataset, titled _MIAP (More
Inclusive Annotations for People)_. The new annotation procedure results in
more annotations for the _person_ class, as well as annotations for gender
presentation and age range presentation. The annotation procedure was
additionally designed to enable fairness analysis.

Our analyses of the two annotation pipelines and resulting annotation patterns
demonstrate the importance of understanding annotation design decisions and
their consequences. We have discussed how annotation procedures can be seen to
interact with societal norms, language, and image context, as well as other
data patterns, here resulting in differences for the _person_ class and
subclass labels. The original annotations and new annotations for Open Images
are both useful in different situations since they vary in scope and coverage,
or they can be used as complementary information. We hope that the findings in
this paper can inform usage of the current dataset as well as future dataset
annotation efforts.

We encourage researchers to use our new _MIAP_ annotations to incorporate
fairness analysis into their research. Given the exhaustive nature of our
updated annotation procedure, researchers can expect more robust fairness
evaluations.

## Image citations

All images used in this paper are licensed under Creative Commons By 2.0 from
Open Images. References for each image are as follows:

  * •

Figure 3: thalisantunes,
https://www.flickr.com//photos//thalisantunes//21322592792//

  * •

Figure 4: Gobierno de Chile,
https://www.flickr.com/photos/gobiernodechile/16822006848/

  * •

Figure 5: Waifer X, https://www.flickr.com/photos/waiferx/85365663/

  * •

Figure 6: Alex Erde, https://www.flickr.com/photos/alexerde/9658256603/

  * •

Figure 7(a): sapex, https://www.flickr.com/photos/sapexoxo/4884905970/

  * •

Figure 7(b): 29th Combat Aviation Brigade Public Affairs,
https://www.flickr.com/photos/29thcab/6155989139/

  * •

Figure 8(a): Grigory Kravchenko,
https://www.flickr.com/photos/grisha_21/3464190158/

  * •

Figure 8(b): Quincena Musical,
https://www.flickr.com/photos/quincenamusical/15026832341

  * •

Figure 8(c): Simon Fraser University - Communications & Marketing,
https://www.flickr.com/photos/sfupamr/6238487408/

  * •

Figure 8(d): Emma Racila,
https://www.flickr.com/photos/94605075@N05/8617525388/

## References

  * (1)
  * Alvi et al. (2018) M. Alvi, A. Zisserman, and C. Nellaker. 2018.  Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings. In _ECCV Workshop on Bias Estimation in Face Analytics_.
  * Atwood et al. (2020) James Atwood, Yoni Halpern, Pallavi Baljekar, Eric Breck, D. Sculley, Pavel Ostyakov, Sergey I. Nikolenko, Igor Ivanov, Roman Solovyev, Weimin Wang, and Miha Skalic. 2020.  The Inclusive Images Competition. In _The NeurIPS ’18 Competition_ , Sergio Escalera and Ralf Herbrich (Eds.). Springer International Publishing, Cham, 155–186.
  * Beutel et al. (2019) Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Allison Woodruff, Christine Luu, Pierre Kreitmann, Jonathan Bischof, and Ed H Chi. 2019.  Putting fairness principles into practice: Challenges, metrics, and improvements. In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_. 453–459.
  * Bouma (2009) Gerlof Bouma. 2009\.  Normalized (pointwise) mutual information in collocation extraction.  _Proceedings of GSCL_ (2009), 31–40.
  * Buolamwini and Gebru (2018) Joy Buolamwini and Timnit Gebru. 2018.  Gender shades: Intersectional accuracy disparities in commercial gender classification. In _Fairness, Accountability, and Transparency (FAT*)_. PMLR, 77–91.
  * de Vries et al. (2019) Terrance de Vries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019.  Does Object Recognition Work for Everyone?. In _CVPR Workshop on Fairness Accountability Transparency and Ethics in Computer Vision (FATE CV)_.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009\.  Imagenet: A large-scale hierarchical image database. In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. Ieee, 248–255.
  * DoD (2018) DoD. 2018.  _2018 Demographics Report: Profile of the Military Community_.  Technical Report. Department of Defence, Office of the Deputy Assistant Secretary of Defense for Military Community and Family Policy (ODASD (MC&FP)).
  * Dulhanty and Wong (2019) Chris Dulhanty and Alexander Wong. 2019.  Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasets. In _CVPR Workshop on Fairness Accountability Transparency and Ethics in Computer Vision (FATE CV)_.
  * Gebru et al. (2018) Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2018\.  Datasheets for datasets.  _arXiv preprint arXiv:1803.09010_ (2018).
  * Grother et al. (2018) Patrick Grother, Mei Ngan, and Kayee Hanaoka. 2018\.  Ongoing Face Recognition Vendor Test (FRVT) Part 1: Verification.  _National Institute of Standards and Technology_ (2018).
  * Hamidi et al. (2018) Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M. Branham. 2018\.  Gender Recognition or Gender Reductionism? The Social Implications of Embedded Gender Recognition Systems. In _CHI Conference on Human Factors in Computing Systems_ (Montreal QC, Canada). 1–13.  https://doi.org/10.1145/3173574.3173582
  * Hardt et al. (2016) Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. 2016\.  Equality of Opportunity in Supervised Learning. In _Advances in Neural Information Processing Systems_ , D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc., 3315–3323.  https://proceedings.neurips.cc/paper/2016/file/9d2682367c3935defcb1f9e247a97c0d-Paper.pdf
  * Karkkainen and Joo (2021) Kimmo Karkkainen and Jungseock Joo. 2021.  FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation. In _IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_. 1548–1558.
  * Keyes (2018) Os Keyes. 2018.  The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition.  _Proceedings of the ACM on Human-Computer Interaction_ , Article 88 (Nov. 2018), 22 pages.  https://doi.org/10.1145/3274357
  * Kuznetsova et al. (2020) Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. 2020\.  The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale.  _International Journal of Computer Vision (IJCV)_ (2020), 1–26.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context. In _European Conference on Computer Vision (ECCV)_. Springer, 740–755.
  * Liu et al. (2015) Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015\.  Deep Learning Face Attributes in the Wild. In _International Conference on Computer Vision (ICCV)_.
  * Madras et al. (2018) David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018.  Learning adversarially fair and transferable representations. In _International Conference on Machine Learning_. PMLR, 3384–3393.
  * Merler et al. (2019) Michele Merler, Nalini Ratha, Rogerio S Feris, and John R Smith. 2019.  Diversity in faces.  _arXiv preprint arXiv:1901.10436_ (2019).
  * Mitchell et al. (2019) Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019.  Model Cards for Model Reporting. In _Fairness, Accountability, and Transparency (FAT*)_. 220–229.  https://doi.org/10.1145/3287560.3287596
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015\.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision (IJCV)_ 115, 3 (2015), 211–252.  https://doi.org/10.1007/s11263-015-0816-y
  * Ryu et al. (2017) Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. 2017\.  Inclusivefacenet: Improving face attribute detection with race and gender diversity.  _arXiv preprint arXiv:1712.00193_ (2017).
  * Scheuerman et al. (2020) Morgan Klaus Scheuerman, Katta Speil, Oliver L. Haimson, Foad Hamidi, and Stacy M. Branham. 2020\.  HCI Guidelines for Gender Equity and Inclusivity.  http://morgan-klaus.com/gender-guidelines.html.  Version 1.1.
  * Shankar et al. (2017) Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. 2017\.  No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World. In _NeurIPS Workshop: Machine Learning for the Developing World_.
  * Watson (2016) Craig I Watson. 2016\.  NIST Special Database 18. NIST Mugshot Identification Database (MID).
  * Wikipedia contributors (2020) Wikipedia contributors. 2020\.  Boy — Wikipedia, The Free Encyclopedia.  https://en.wikipedia.org/w/index.php?title=Boy&oldid=992668770.  [Online; accessed 29-January-2021].
  * Wikipedia contributors (2021) Wikipedia contributors. 2021\.  Girl — Wikipedia, The Free Encyclopedia.  https://en.wikipedia.org/w/index.php?title=Girl&oldid=1000129400.  [Online; accessed 29-January-2021].
  * Wilson et al. (2019) Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019\.  Predictive Inequity in Object Detection. In _CVPR Workshop on Fairness Accountability Transparency and Ethics in Computer Vision (FATE CV)_.
  * Wu et al. (2018) Zhe Wu, Navaneeth Bodla, Bharat Singh, Mahyar Najibi, Rama Chellappa, and Larry S Davis. 2018\.  Soft sampling for robust object detection.  _arXiv preprint arXiv:1806.06986_ (2018).
  * Yang et al. (2020) Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. 2020.  Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy. In _Fairness, Accountability, and Transparency (FAT*)_.
  * Zhang et al. (2017) Zhifei Zhang, Yang Song, and Hairong Qi. 2017.  Age Progression/Regression by Conditional Adversarial Autoencoder. In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.