# Quasi-local masses and cosmological coupling of black holes and mimickers

Mariano Cadoni<EMAIL_ADDRESS>Dipartimento di Fisica, Università
di Cagliari, Cittadella Universitaria, 09042 Monserrato, Italy  INFN, Sezione
di Cagliari, Cittadella Universitaria, 09042 Monserrato, Italy  Riccardo
Murgia<EMAIL_ADDRESS>Gran Sasso Science Institute (GSSI), Viale F.
Crispi 7, L’Aquila (AQ), I-67100, Italy  INFN - Laboratori Nazionali del Gran
Sasso (LNGS), L’Aquila (AQ), I-67100, Italy  Mirko Pitzalis
<EMAIL_ADDRESS>Dipartimento di Fisica, Università di Cagliari,
Cittadella Universitaria, 09042 Monserrato, Italy  INFN, Sezione di Cagliari,
Cittadella Universitaria, 09042 Monserrato, Italy  Andrea P. Sanna
<EMAIL_ADDRESS>Dipartimento di Fisica, Università di Cagliari, Cittadella
Universitaria, 09042 Monserrato, Italy  INFN, Sezione di Cagliari, Cittadella
Universitaria, 09042 Monserrato, Italy

###### Abstract

Motivated by the recent heated debate on whether the masses of local objects,
such as compact stars or black holes (BHs), may be affected by the large-
scale, cosmological dynamics, we analyze the conditions under which, in a
general relativity framework, such a coupling small/large scales is allowed.
We shed light on some controversial arguments, which have been used to rule
out the latter possibility. We argue that the actual observational quantity at
play is the quasi-local Misner-Sharp mass (MS), and we find that the
cosmological coupling occurs whenever the energy of the central object is
quantified by it. Conversely, the decoupling occurs whenever the MS mass is
fully equivalent to the (nonlocal) Arnowitt-Deser-Misner (ADM) mass.
Consequently, for singular BHs embedded in cosmological backgrounds, like the
Schwarzschild-de Sitter or McVittie solutions, we show that there is no
cosmological coupling, confirming previous results in the literature.
Furthermore, we show that nonsingular compact objects couple to the
cosmological background, as quantified by their MS mass. We conclude that
observational evidence of cosmological coupling of astrophysical BHs would be
the smoking gun of their nonsingular nature.

##  I Introduction

Recently, there has been renewed interest in an old question of general
relativity (GR): are small-scale, local systems, like planets, stars or
compact objects/black holes (BHs), affected by the large-scale dynamics of the
cosmological background they are embedded in?
The first known attempt to consistently answer this question dates back to
McVittie [1], who found a solution of Einstein’s field equations describing a
point-like object embedded in a spatially-flat Friedmann-Lemaître-Robertson-
Walker (FLRW) spacetime. However, the issue was far from being settled, as the
extremely nontrivial physics involved in this embedding entails conceptual and
interpretative problems (see, $e.g.$, Refs. [2, 3, 4, 5, 6]).

As a result, a considerable body of work has ensued over the years, with,
however, contradictory results (for an incomplete list, see, $e.g.$, Refs. [7,
8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
47, 48, 49, 50, 51] and references therein). The main conceptual obstacle is
caused by the huge separation of scales between local inhomogeneities, whose
characteristic scale is their virial radius $\sim GM$, and the large-scale
cosmological dynamics, occurring instead at the Hubble radius $H$. Although
there seems to be general agreement on the negligible impact of the
cosmological expansion on small-scale111However, this is not necessarily the
case with large-scale structures [12, 52, 53, 54, 36, 55, 56, 57, 58, 59, 60,
61, 62, 63, 64, 65, 66, 67]. Newtonian systems [68, 69, 37, 35], the issue is
still opened for local, relativistic bodies, like BHs.

Notice that the coupling could be a rather natural feature of highly-compact
gravitational systems. Naïvely, the mass/radius relation of a BH would suggest
that, if lengths are affected by the cosmological expansion, so should do
masses.

In the latest years, the debate has rekindled due to recent developments based
on the theoretical work of Croker and collaborators [70, 71, 72]. Through a
perturbative approach and an averaging procedure, they derived the Friedmann’s
equations from varying the gravitational action. What they show, in this way,
is that the pressure in the interior of BHs/compact objects contribute
actively to the energy density sourcing the cosmological equations. The
conservation of the stress-energy tensor, then, implies the presence of a
coupling of these objects with the cosmological expansion, which should
manifest as a significant shift in their masses. Their model also allows to
predict that the masses of local objects should vary with the scale factor $a$
according to the power law $M(a)\propto a^{k}$ [70]. This formula was then
tested in Ref. [73] against an observational sample of supermassive BHs at the
centre of elliptical galaxies at different redshift. Such objects are
quiescent, $i.e.$, they are concerned by negligible processes of accretion or
mergers, so that the data-set is not sensibly affected by other growth
channels other than the supposed coupling mechanism with the expanding
background. This set of data showed a preference for $k\sim 3$ [73, 74]. The
conclusion of the authors of Ref. [73] is that BHs may be the source of dark
energy.

However, this claim and the underlying theoretical framework have faced
significant criticism. Even if the underlying framework could be flawed from
the beginning [75], most criticism has been directed at the concept of
coupling itself. On the one hand, the substantial separation in scales between
local and cosmological systems makes such coupling implausible [76, 77]. On
the other hand, the equation of state of matter inside a BH, which is
typically taken to be dust, is unable to mimic dark energy [78, 79]. Moreover,
current observational constraints on the slope parameter $k$ capturing the
mass-redshift dependence, are highly controversial, as they heavily depend on
the astrophysical probes employed in the analysis [80, 81, 82, 83].

These critiques cast again doubts on the feasibility and validity of the
proposed coupling between cosmological dynamics and the masses of BHs/compact
objects. However, in Ref. [74], we and collaborators built a solid general
relativistic framework that enabled us to describe the coupling of local
inhomogeneities with the cosmological background in full generality, as well
as to recover the expression of the mass-shift from Refs. [70, 73].

Independently of the intricate situation on the observational side, the
theoretical question that we want to address in the present paper is: in which
conditions does the cosmological expansion affect dynamical quantities, such
as the BH masses?

We give a precise answer to this question by working on a solid theoretical
ground. Previously, the debate was biased by the use of the nonlocal Arnowitt-
Deser-Misner (ADM) mass to quantify the energy pertaining to local objects.
The starting point of this work, instead, is the identification of the quasi-
local Misner-Sharp (MS) mass as the most appropriate quantity to determine the
energy of local compact objects, and to investigate their cosmological
coupling.

The MS mass is covariantly defined, and it reduces to the ADM mass at
asymptotically-flat infinity. Therefore, it can be identified as the actual
astrophysical observable. We then use the MS mass to compute the energy of
various cosmologically-embedded solutions: Schwazschild-de Sitter, McVittie,
Sultana-Dyer (SD), and nonsingular BHs sourced by anisotropic fluids.

We find that when the energy of the central object is described using the MS
mass, the cosmological coupling becomes manifest, while the decoupling occurs
whenever the energy of the central object is equivalent to its ADM mass in an
asymptotically-flat spacetime, as in singular BH models. We explicitly show
that when the energy of the nonembedded object can be quantified everywhere by
the MS mass, but not by the ADM one, the presence of cosmological coupling is
inevitable. That is the case of nonsingular BHs and other compact objects.

The paper is structured as follows.

In Section II, we discuss the role played by the different definitions of mass
for BHs/compact objects in cosmology, and provide a brief overview of the key
properties of the MS mass.

In Section III, we revisit the singular Schwarzschild-de Sitter and McVittie
solutions, demonstrating that the mass of the central local object does not
couple to the cosmological dynamics.

In Section IV, we examine the cosmological embedding of nonsingular
BHs/compact objects: we firstly revisit the SD solution, then discuss the most
general case of compact objects with anisotropic sources, and finally the
isotropic case. We explicitly show that the cosmological coupling is
quantified by the MS mass, and we conclude that any observational evidence of
cosmological coupling could be a smoking gun for the nonsingular nature of
astrophysical BHs.

We present our conclusions in Section V.

##  II Definitions of mass for compact objects and cosmological coupling

Answering the question posed in Section I, about the mass growth of compact
objects due to cosmological expansion, is complicated by the fact that there
are two relevant formal definitions for the mass of a static, spherically-
symmetric compact object: $(i)$ the MS mass, a quasi-local quantity
characterizing the energy inside a sphere of a given radius; $(ii)$ the ADM
mass, a nonlocal quantity defined in terms of a surface integral at spatial
infinity.

Static eternal BHs embedded in a cosmological background are usually described
by neglecting the cosmological asymptotics and resorting to asymptotic
flatness instead, where all observables can be precisely identified and
quantified in terms of surface integrals at spatial infinity [84]. In this
case, one can safely use the ADM mass. For spacetimes with different
asymptotics, like the FLRW ones, the identification and interpretation of such
nonlocal observables become much more involved.

The key issue in this type of problem is not purely kinematic, like, $e.g.$,
that concerning the cosmological redshift of distances in cosmology, but fully
dynamic. As such, it implies some explicit or implicit assumption about how
the small-scale, inhomogeneous dynamics of the compact object is related to
the large-scale, homogeneous and isotropic cosmological background dynamics.
The usual assumption is that there exists a scale of _decoupling_ , which is
essentially justified by the huge separation of scales between the heaviest
known galactic BH ($\sim 10^{-3}$ pc) and the Hubble radius ($\sim 10^{10}$
pc) [77].

The use of the ADM mass to characterize the energy of cosmologically-embedded
BHs is fully justified only if one accepts the assumption of the decoupling of
scales. This is the only case where a cosmologically-embedded BH can be safely
treated as an eternal, asymptotically-flat object.

The decoupling of scales assumption is physically justified only for the
Schwarzschild-de Sitter solution, where one has a globally-defined, static,
radial coordinate to safely define the $r\to 0$ and $r\to\infty$ limits. In
other cases, such as, $e.g.$, the McVittie solution, the $r\to 0$ and
$r\to\infty$ limits use different radial coordinates, related by a time-
dependent coordinate transformation.

Finally, another strong limitation of the ADM mass is that it correctly
quantifies the energy of the compact objects only in the case of astrophysical
bodies in which the stress-energy tensor is zero outside, such as singular
BHs. This is not the case for nonsingular BHs [85].

We thus argue that, in the most general case, the actual physical observable
is the quasi-local MS mass. In the following, we will briefly review its basic
properties.

###  II.1 Basic features of the Misner-Sharp mass

Depending on the asymptotics of a given spacetime, there are several ways to
quantify the energy of a gravitational system. As already stated, in
asymptotically-flat spacetimes (or, more in general, for spacetimes with a
timelike asymptotic boundary, like, $e.g.$, anti de Sitter), the key
observables can be unambiguously quantified, through the ADM decomposition, as
_nonlocal_ quantities defined at the boundary of the spacetime, the so-called
“hair” of classical, singular Kerr-Newman solutions [86].

For BHs not in vacuum, like nonsingular BHs, this identification is less
straightforward, even if the manifold is asymptotically flat (see, $e.g.$,
Refs. [87, 85] and references therein). In these cases, there is a different
definition which better encapsulates the _local_ properties that the energy of
a gravitational system should satisfy: the Hawking-Hayward quasi-local mass
[88, 89], which, for spherically-symmetric spacetimes, reduces to the MS mass
[90]. In a generic asymptotically-flat spacetime, the ADM and MS masses
coincide only at spatial infinity, namely
$M_{\text{ADM}}=\lim_{r\to\infty}M_{\text{MS}}$. They are fully equivalent
outside the compact object only if the stress-energy tensor vanishes outside
of the object.

On the contrary, the MS mass can be defined _covariantly_ also for non-
asymptotically flat and non-stationary spacetimes. Moreover, as it encodes the
_local_ properties of the energy of a given gravitational system, it can be
considered as the physical mass tested with astrophysical observations. For
this reason, it represents the most appropriate tool to investigate the
possible cosmological coupling of local objects embedded in cosmological
backgrounds.

For a spherically-symmetric spacetime with a metric of the general
form222Throughout the paper, we shall use natural units in which
$c=\hbar=k_{\text{B}}=1$. Latin indices $a,b=1,2$ denote the time and radial
coordinates. We use Greek indices to denote four-dimensional spacetime
coordinates. $\text{d}\Omega$ is the line element of the two-sphere.

$\text{d}s^{2}=h_{ab}(x)\text{d}x^{a}\text{d}x^{b}+r(x)^{2}\text{d}\Omega^{2}\,,$
(1)

the MS mass is defined as

$M_{\text{MS}}=\frac{r(x)}{2G}\left[1-h^{ab}(x)\nabla_{a}r(x)\nabla_{b}r(x)\right]\,.$
(2)

Given that the MS mass is defined in a covariant way, all the physical results
based on its use are coordinate-independent, while its explicit form rests, of
course, on the particular gauge chosen. In the following, we consider the
systems of coordinates that are mostly adopted when discussing the embedding
of spherical objects in cosmological backgrounds. One system is given by
Lemaître coordinates $\left(t,\,r,\,\theta,\,\phi\right)$

$\text{d}s^{2}=-e^{\alpha(t,r)}\text{d}t^{2}+e^{\beta(t,r)}\text{d}r^{2}+R(t,r)^{2}\text{d}\Omega^{2}\,,$
(3)

where $\alpha$, $\beta$ and $R$ all depend on the radial and time coordinates.
Note also that this metric generalizes the ones written in isotropic
coordinates

$\text{d}s^{2}=-e^{\alpha(t,r)}\text{d}t^{2}+e^{\tilde{\beta}(t,r)}\left(\text{d}r^{2}+r^{2}\text{d}\Omega^{2}\right)\,,$
(4)

that have also been frequently adopted to discuss the cosmological embedding
of compact objects.

However, in order to discuss the MS mass, it is more convenient to use
$R(t,\,r)$ as the radial coordinate. Through a straightforward change of
coordinates (see Ref. [91]), one can recast the metric (3) into the form

$\text{d}s^{2}=-A(T,R)\text{d}T^{2}+B(T,R)\text{d}R^{2}+R^{2}\text{d}\Omega^{2}\,,$
(5)

with the relations

$\displaystyle A=\left(e^{\alpha}-e^{\beta}\frac{\dot{R}^{2}}{R^{\prime
2}}\right)F^{2}\,;$ (6a) $\displaystyle B=\frac{e^{\alpha+\beta}}{R^{\prime
2}\left(e^{\alpha}-e^{\beta}\frac{\dot{R}^{2}}{R^{\prime 2}}\right)}\,,$ (6b)

where the dot and prime stand for derivation with respect to $t$ and $r$,
respectively. $F$ is an integration function entering the time-coordinate
transformation, it is required to guarantee that $\text{d}T$ is an exact
differential [91]. Using Eq. 5, the MS mass reads as

$\displaystyle M_{\text{MS}}$
$\displaystyle=\frac{R}{2G}\left(1-g^{\mu\nu}\nabla_{\mu}R\nabla_{\nu}R\right)$
(7)
$\displaystyle=\frac{R}{2G}\left(1-g^{RR}\right)=\frac{R}{2}\left(1-B^{-1}\right)\,.$
(8)

Using Eq. 6b, in the gauge (3), it becomes

$M_{\text{MS}}=\frac{R}{2G}\left(1+\dot{R}^{2}e^{-\alpha}-R^{\prime
2}e^{-\beta}\right)\,.$ (9)

In the following sections we will use this formula to compute the mass of
cosmologically-embedded compact objects.

##  III Embedding point-like objects and perfect fluid stars in cosmological
backgrounds

In this Section we will use the MS mass to reproduce already-known results
regarding the nonexistence of a cosmological coupling of standard singular
BHs. By doing so, we will clarify the physical reasons behind the absence of
the small-/large- scale coupling in two well-known solutions that were
recently reconsidered to advocate against the ubiquity of the cosmological
coupling [77]. The gist of their argument is that there is a complete
separation between the scales pertaining to local objects (like BHs) and the
dynamics of the cosmological background, such that the mass of the central BH
can be approximately identified with its ADM mass. First of all, let us note
that the ADM mass cannot be properly defined in non-asymptotically flat
spacetimes, as those corresponding to cosmologically embedded objects.
Secondly, as previously stressed, the ADM mass is a _nonlocal_ quantity,
thereby unable to quantify local effects, such as the coupling. Finally, the
separation of scales presented in [77] involves rather questionable limits on
small and large scales, due to the use of time-dependent radial coordinates.
The two limits truly represent separated scales only if we consider the small-
scale local dynamics at times for which the scale factor does not change
significantly. If this not the case, the notion of small- and large-scale
limits would instead depend on time. The only case in which one can truly show
that the two limits give neatly separates scales at all times is the
Schwarzschild-de Sitter solution, since one can define coordinates in which
the spacetime is static.

We shall show that, for point-like objects, the separation of scales and the
resulting decoupling emerges naturally when considering the MS mass of the
solutions mentioned above.

###  III.1 Schwarzschild-de Sitter solution

The simplest known example of an embedding of a compact object (mass-particle)
in a cosmological background is the Schwarzschild-de Sitter metric, which is a
vacuum solution of Einstein’s equations with a positive cosmological constant.
A peculiarity of this metric is that it can be written in the static patch

$\begin{split}\text{d}s^{2}=&-\left(1-\frac{2Gm}{r}-H^{2}r^{2}\right)\text{d}t^{2}+\frac{\text{d}r^{2}}{1-\frac{2Gm}{r}-H^{2}r^{2}}\\\
&+r^{2}\text{d}\Omega^{2}\,.\end{split}$ (10)

As already noted, here we have a clear separation between the small scales,
where the solution reduces to the Schwarzschild one with an ADM mass $m$, and
the large scales, where we instead have the de Sitter asymptotics. The
decoupling can be readily seen using the MS mass given by Eq. 9, instead of a
more intricate change of coordinates as done in Ref. [77]. A straightforward
calculation yields indeed

$M_{\text{MS}}=m+\frac{H^{2}}{2G}r^{3}\,.$ (11)

The first term is the ADM mass of the Schwarzschild BH, while the second term
is simply the mass contribution due to the constant cosmological density over
a volume $r^{3}$. There is no trace of the growth of $m$ due to the expanding
cosmological background given by the Hubble parameter $H=\dot{a}/a$, namely
there is no cosmological coupling.

###  III.2 McVittie solution

The McVittie spacetime [1] represents a generalization of the Schwarzschild-de
Sitter spacetime to a generic FLRW model. It was the first exact solution of
GR which allowed for the embedding of spherically-symmetric objects in a
generic cosmological background. It is based on some assumptions, the most
important ones being a perfect, isotropic and spherically-symmetric fluid as a
source, and the absence of fluxes of matter/energy into/away from the central
object. Moreover, the metric is required to reduce to the Schwarzschild one,
written in isotropic coordinates, when expressed in terms of radial coordinate
of the observer, $\hat{r}=ar$. It thus has the same singularity at the origin.

In the coordinates used in Eq. 3, it reads as

$\begin{split}\text{d}s^{2}=&-\frac{\left(1-\frac{Gm(t)}{2r}\right)^{2}}{\left(1+\frac{Gm(t)}{2r}\right)^{2}}\text{d}t^{2}\\\
&+a^{2}\left(1+\frac{Gm(t)}{2r}\right)^{4}\left(\text{d}r^{2}+r^{2}\text{d}\Omega^{2}\right)\,,\end{split}$
(12)

where $a$ is the scale factor and $m(t)=m_{0}/a(t)$, from Einstein’s equations
and from the requirement of absence of radial fluxes.

We identify the areal radius as

$R(t,r)\equiv a(t)\,r\left(1+\frac{Gm_{0}}{2ra(t)}\right)^{2}\,.$ (13)

By writing this solution in the gauge (5), Eq. 9 yields,

$\begin{split}\text{d}s^{2}=&-\left(1-\frac{2Gm_{0}}{R}-H^{2}R^{2}\right)F^{2}\,\text{d}T^{2}\\\
&+\frac{\text{d}R^{2}}{1-\frac{2Gm_{0}}{R}-H^{2}R^{2}}+R^{2}\,\text{d}\Omega^{2}\,,\end{split}$
(14)

which is very similar to the Schwarzschild-de Sitter solution, the only
differences being the factor $F$ in the $g_{TT}$ component, and the fact that
$H$ is not restricted to describe a de Sitter cosmological background. This
similarity translates also in the behavior of the MS mass, where again we have
a complete separation of scales

$M_{\text{MS}}=\frac{R}{2G}\left(\frac{2Gm_{0}}{R}+H^{2}R^{2}\right)=m_{0}+\frac{H^{2}}{2G}R^{3}\,.$
(15)

We again identify two independent terms: a contribution of the ADM mass of the
BH, and a purely cosmological term. No coupling is present, as expected, since
the density sourcing the McVittie solution is purely cosmological, $i.e.$,
$\rho=3H^{2}/8\pi G$, while the contribution of the ADM mass of the central
object can only be accounted for by inserting by hand the usual Dirac delta
distribution.

##  IV Coupling of compact objects: local anisotropic and isotropic sources

In Section III we showed that, for solutions of Einstein’s equations
describing the cosmological embedding of the Schwarzschild black hole, there
is no cosmological coupling.

On the contrary, when the impact of small-scale anisotropies is taken into
account, it is possible to have nontrivial solutions describing compact
objects/BHs [92, 93, 94] circumventing the Penrose theorem [95], and enabling
the construction of nonsingular solutions [96, 97, 98, 99, 100, 101, 102, 103,
104, 105, 106, 107, 108, 85, 109]. In this Section, we will consider several
scenarios of cosmological embedding of compact objects: the SD solution,
solutions sourced by anisotropic fluids, solutions sourced by isotropic
fluids, and charged singular solutions.

###  IV.1 The Sultana-Dyer solution

The SD solution [30] is another exact solution describing a BH embedded in a
spatially flat FLRW. It was found by conformally transforming the
Schwarzschild metric with the goal of changing the Schwarzschild global
timelike Killing vector into a conformal Killing one. The conformal
transformation also allows the spacetime to be nonsingular at $r=0$.

Despite it being problematic due to the fluid becoming tachyonic at late times
near the horizon [30, 35], it is still interesting for our purposes.

The metric is essentially the McVittie one (3), with an important difference:
the mass $m$ appearing in the metric is now a constant

$\text{d}s^{2}=-\frac{\left(1-\frac{Gm_{0}}{2r}\right)^{2}}{\left(1+\frac{Gm_{0}}{2r}\right)^{2}}\text{d}t^{2}+a^{2}\left(1+\frac{Gm_{0}}{2r}\right)^{4}\left(\text{d}r^{2}+r^{2}\text{d}\Omega^{2}\right)\,.$
(16)

The fact that the mass does not depend on $a$ naturally introduces fluxes,
making the source anisotropic. In fact, the source of the SD solution is a
combination of two noninteracting perfect fluids, one in the form of an
ordinary massive dust and the other of a null dust [30, 35]. It is well-known
that such a combination can be recast as a single anisotropic fluid [110]. As
we shall see, this is the origin of the coupling with the cosmological
background.

We now compute the MS mass (9) of the SD solution. We first identify

$R\equiv ar\left(1+\frac{Gm_{0}}{2r}\right)^{2}\,,$ (17)

with which, using Eq. 6b and reading $e^{\alpha}$ and $e^{\beta}$ from Eq. 16,
we get

$B=\frac{1-\frac{2Gam_{0}}{R}}{\left(1-\frac{2Gam_{0}}{R}\right)^{2}-H^{2}R^{2}}\,.$
(18)

Therefore, the MS mass (9) reads as

$M_{\text{MS}}=a\,m_{0}+\frac{H^{2}R^{3}}{2G\left(1-\frac{2Gam_{0}}{R}\right)}\,.$
(19)

The first term represents the coupling of the mass of the solution with the
cosmological background, and it is consistent with the linearly-scaling
universal coupling term derived for the first time in Ref. [74] for generic
anisotropic fluids. The second term cannot be interpreted as a pure
cosmological contribution, due to the presence, in the denominator, of a term
depending on $a\,m_{0}$. The latter encodes the interaction between the small
and large scales, as a physical consequence of the accretion flow of cosmic
fluid onto the central object, due to the presence of nonzero fluxes in the
source. Note that similar results hold for the class of exact models analyzed
in Ref. [35], devised to correct the problems of the SD metric, as well as for
the solutions considered in Ref. [111].

###  IV.2 Compact objects sourced by anisotropic fluids

In Ref. [74] it was shown that the metric parametrization ($\eta$ is the
conformal time)

$\text{d}s^{2}=a^{2}(\eta)\left[-e^{\alpha(\eta,r)}\text{d}\eta^{2}+e^{\beta(\eta,r)}\text{d}r^{2}+r^{2}\text{d}\Omega^{2}\right]\,,$
(20)

representing the cosmological embedding of a generic compact object sourced by
an anisotropic fluid, allows to describe the coupling of GR BHs/horizonless
configurations to the cosmological background. The stress-energy tensor
pertaining to the source has the form
$T^{\mu}_{\nu}=\text{diag}\left(-\rho,\,p_{{\mkern 3.0mu\vphantom{\perp}\vrule
depth=0.0pt\mkern 2.0mu\vrule depth=0.0pt\mkern
3.0mu}},\,p_{\perp},\,p_{\perp}\right)$.

Einstein’s equations and stress-energy tensor conservation give:

$\displaystyle e^{-\beta(r,\eta)}=g(r)a^{r\alpha^{\prime}}\,;$ (21a)
$\displaystyle\frac{\dot{a}^{2}}{a^{2}}\left(3-r\alpha^{\prime}\right)e^{-\alpha}+\frac{1-e^{-\beta}+r\beta^{\prime}e^{-\beta}}{r^{2}}=8\pi
Ga^{2}\rho\,;$ (21b)
$\displaystyle\frac{e^{-\beta}+re^{-\beta}\alpha^{\prime}-1}{r^{2}}+e^{-\alpha}\left(-2\frac{\ddot{a}}{a}+\frac{\dot{a}^{2}}{a^{2}}\right)=8\pi
Ga^{2}p_{\parallel}\,;$ (21c)
$\displaystyle\dot{\rho}+\frac{\dot{a}}{a}\left(3\rho+3p_{{\mkern
3.0mu\vphantom{\perp}\vrule depth=0.0pt\mkern 2.0mu\vrule depth=0.0pt\mkern
3.0mu}}+rp^{\prime}_{{\mkern 3.0mu\vphantom{\perp}\vrule depth=0.0pt\mkern
2.0mu\vrule depth=0.0pt\mkern 3.0mu}}\right)=0\,,$ (21d)

where a dot now means derivation with respect to $\eta$. The remaining
equation, stemming from the conservation of the stress-energy tensor, is used
to compute $p_{\perp}$.

The field equations allow for a regime in which $\dot{\alpha}=0$, which is the
only one in which compact objects can be consistently embedded in a
cosmological background (see Refs. [112, 113, 74]). It describes the _absence_
of fluxes, unlike the case analyzed in Section IV.1.

This set-up is suitable to describe both the large- (cosmological) and the
small- (inhomogeneity) scale dynamics, and it allows for a non-zero
interaction term between these two scales333It is worth noting that the
cosmological embedding realized using the metric (20) is more general than the
one used by McVittie in Eq. 12. In our case the metric is required to reduce
to the static metric of the local compact object at any fixed instant of time.
Conversely, in the McVittie parametrization, such reduction must happen by
passing to the observer radial coordinate $\hat{r}=ar$..

The cosmological mass coupling is immediately manifest when computing the
density through Einstein’s equations and integrating it over a reference
cosmological volume (see Ref. [74] for further details)

$\begin{split}M(\eta)&=4\pi
a^{3}(\eta)\int_{0}^{L}\text{d}r\,r^{2}\,\rho(r,\eta)\\\
&=\frac{4\pi}{3}\rho_{1}a^{3}L^{3}\,e^{-\alpha(L)}+M(a_{i})\frac{a}{a_{i}}\left[1-e^{-\beta_{0}(L)}a^{k_{L}}\right]\,,\end{split}$
(22)

where $L$ is the scale of a particular compact objects, while

$k_{L}\equiv k(L)=r\alpha^{\prime}(r)\biggr{|}_{r=L}\,.$ (23)

Here we defined $M(a_{i})\equiv a_{i}L/2G$, which is the mass of the object
computed at the coupling epoch. This expression defines the proper
Schwarzschild radius $a_{i}L=2GM(a_{i})$ at this reference time.

The first term in Eq. 22 corresponds to a purely cosmological contribution,
which depends on the cosmological background energy density $\rho_{1}$. It is
therefore expected to be relevant only beyond the transition scale to
homogeneity and isotropy. It does not play a role at the typical scales $L$ of
the compact object. As here we are not interested in whether and how the
small-scale dynamics affects the large-scale cosmological one, another long-
standing problem commonly known as ”cosmological backreaction” [55, 56, 57,
58, 59], we can safely neglect this term in the following discussion.

The second term in Eq. 22 represents a “universal cosmological Schwarzschild
mass”. The cosmological coupling emerges as a linear dependence between the
mass of the object and the scale factor $a$. Note that this is the same
universal coupling term found for the SD solution (see Eq. 19). It has here a
geometric origin in terms of the local curvature generated by the compact
object [74]. Finally, the last term encodes model-dependent corrections to the
universal term.

Note that, for standard Schwarzschild BHs, the sum of the second and third
terms in Eq. 22 is identically zero, and we are left with the purely
cosmological contribution.

Eq. (22) can also be derived from the general definition of the MS mass. Using
$R=ar$, from Eq. 9 one gets

$\begin{split}M_{\text{MS}}&=\frac{ar}{2G}\left[1+\frac{\dot{a}^{2}}{a^{2}}r^{2}e^{-\alpha}-\frac{a^{2}}{a^{2}}e^{-\beta}\right]\\\
&=\frac{4\pi}{3}\rho_{1}a^{3}r^{3}e^{-\alpha}+\frac{ar}{2G}\left[1-e^{-\beta_{0}(r)}a^{k(r)}\right]\,,\end{split}$
(24)

where, in the last step, we made use of the Friedmann equation
$3\dot{a}^{2}/a^{2}=8\pi Ga^{2}\rho_{1}$, with $\rho_{1}=\rho_{1}(\eta)$ being
the density of the cosmological fluid sourcing the background. To obtain the
last element we also used Eq. 21a. Eq. 24 is equivalent to Eq. 22 when
evaluated at the radius of the compact object $r=L$.

Notice that the cosmological coupling is present independently of the
underlying cosmological background (for instance, it is present also in a de
Sitter background). That is because $\rho_{1}$ can be freely specified, and it
determines the scale factor _regardless_ of the dynamics of the small-scale
inhomogeneities444This is strictly true only if one neglects the backreaction
(see the discussion above). If we impose the solution at constant time to be
Schwarzschild-de Sitter, we have no coupling, $i.e.$, we have a complete
separation between scales. Let us finally recall that, differently from the SD
and other solutions, here the coupling is not due to some accretion onto
compact objects, since we imposed the absence of radial fluxes.

###  IV.3 Compact objects sourced by isotropic fluids

The case of nonsingular compact objects sourced by isotropic fluids can be
considered as a particular case of the previously discussed anisotropic fluid,
where $p_{\perp}=p_{\mkern 3.0mu\vphantom{\perp}\vrule depth=0.0pt\mkern
2.0mu\vrule depth=0.0pt\mkern 3.0mu}=p$. The only difference is that, now, the
conservation of the stress-energy tensor gives the additional equation
$p^{\prime}+\alpha^{\prime}\left(\rho+p\right)/2=0$. This makes the system
(21) more constrained, so that, as shown by McVittie [1], it does not allow
for smooth solutions describing the cosmological embedding of compact objects.
Only the standard FLRW cosmological solutions are allowed. On the contrary, in
the derivation of the main results of Section IV.2, _i.e._ , Eqs. 22 and 24,
we did not exploit the conservation equations for the stress-energy tensor.
Therefore they still hold true also in the case of isotropic compact objects,
provided that the cosmologically embedded solutions exist. This could be, for
instance, the case of a non-smooth cosmological embedding of a local compact
object. Thus, we expect also nonsingular compact objects sourced by isotropic
fluids to couple to the cosmological evolution in the same way as their
anisotropic counterparts.

###  IV.4 Charged singular solution embedded in a FLRW background

The cosmological embedding of singular solutions can also be extended to the
charged Reissner-Nordström (RN) solution of GR. There has been some work
devoted to the charged generalization of the McVittie spacetime, the so-called
Shah-Vaidya solution (see Refs. [11, 27, 33, 41, 47] and references therein).
Even in this case, the stress-energy tensor is anisotropic, but any flux
onto/away from the central object is absent. One might attempt to naïvely
apply the general results of this section to the cosmological embedding of the
RN BH, given that the stress-energy tensor is nonzero outside of the horizon,
implying that the MS and ADM mass are not identical in the BH exterior.
However, there is a problem in the definition of the quasi-local mass, due to
the divergence associated to the electric field at $r=0$.

In the presence of an electric field, the gravitational potential scales as
$1/r^{2}$, due to the density of the electric field scaling as $r^{-4}$). As a
consequence, there is an extra factor – scaling as $1/r$ in the quasi-local
energy – which is ill-defined at $r=0$. This feature is interpreted as a
repulsive effect due to the intensity of the electric field in the vicinity of
the singularity (for further details, see, $e.g.$, Section VIII in Ref. [90],
or Section III in Ref. [114]).

Alternatively, one can quantify the energy of a charged solution with the ADM
mass, which is anyhow undefined in non-asymptotically flat spacetimes, and
moreover does not capture the coupling to the cosmological background, as
discussed in detail throughout this paper. We thus conclude that a clear
description of the coupling between a charged singular spacetime embedded in a
cosmological background is quite problematic. Moreover, from a purely
phenomenological point of view, the presence of an electromagnetic charge is
astrophysically irrelevant.

##  V Conclusions

In this paper we have shown that the longstanding debate on the theoretical
status of the coupling of BHs/compact astrophysical objects to the
cosmological background can be finally put on solid ground, if the physically
observable mass of the compact object is identified as the MS mass. By doing
so, we have explicitly demonstrated that singular BHs cannot couple to the
large-scale cosmological dynamics.

We have also shown that the cosmological coupling is not only allowed, but
quite natural, for generic compact objects sourced both by isotropic fluids
and local anisotropies, like, $e.g.$, nonsingular BHs. The energy of these
systems has a quasi-local nature, so that it can be correctly quantified by
the MS mass (instead of the nonlocal ADM mass). In this case, we have found
that the mass of the object is intrinsically linked to the scale factor $a$ by
a universal linearly-scaling leading term, with a geometric origin in terms of
the local curvature of spacetime.

From a purely theoretical point of view, there are two main issues that still
need further investigation: $(i)$ fully understand the cosmological coupling
of singular objects sourced by local anisotropies, but characterized by an
ill-defined MS mass (which prevents a direct application of the results of
Section IV). This is the case of the RN BH briefly discussed in this paper;
$(ii)$ generalize the framework in order to encompass rotating compact objects
(see, _e.g._ , Refs. [13, 14, 15] for earlier works on the embedding of the
Kerr metric in a FLRW cosmology). Note, however that including rotation is
expected to have a non-neglibile impact only on the subleading non-universal
term in Eq. 22, without affecting at all our conclusions about the leading
universal linear coupling term and the decoupling of singular BHs.

As described in Section I, the situation still remains rather complicated from
the point of view of observations, mainly due to the lack of clear
observational results. New sets of data are needed in order to validate the
theoretical predictions for the exponent $k$. Our theoretical analysis shows
quite clearly that eternal BHs, $i.e.$, objects with event horizons, are
characterized either by $k=0$ if they are singular objects, or by $k=1$ if
they are regular. It seems that GR can not allow for other possibilities.
Observational evidence of a nonzero cosmological coupling would thus be the
smoking gun of the nonsingular nature of the actual astrophysical black holes.
Conversely, a clear detection of $k=0$ would imply that nonsingular GR BHs are
hardly compatible with observations.

##  VI Acknowledgements

We thank Valerio Faraoni and Francesca Lepori for very interesting and
fruitful discussions.

## References

  * McVittie [1933] G. C. McVittie, MNRAS 93, 325 (1933).
  * Sussman [1985] R. A. Sussman, Gen. Rel. Grav. 17, 251 (1985).
  * Ferraris  _et al._ [1996] M. Ferraris, M. Francaviglia, and A. Spallicci, Nuovo Cim. B 111, 1031 (1996).
  * Nolan [1998] B. C. Nolan, Phys. Rev. D 58, 064006 (1998), arXiv:gr-qc/9805041 .
  * Nolan [1999] B. C. Nolan, Class. Quant. Grav. 16, 1227 (1999).
  * Faraoni  _et al._ [2012] V. Faraoni, A. F. Zambrano Moreno, and R. Nandra, Phys. Rev. D 85, 083526 (2012), arXiv:1202.0719 [gr-qc] .
  * Einstein and Straus [1945] A. Einstein and E. G. Straus, Rev. Mod. Phys. 17, 120 (1945).
  * Einstein and Straus [1946] A. Einstein and E. G. Straus, Rev. Mod. Phys. 18, 148 (1946).
  * Pachner [1963] J. Pachner, Phys. Rev. 132, 1837 (1963).
  * Dicke and Peebles [1964] R. H. Dicke and P. J. E. Peebles, Phys. Rev. Lett. 12, 435 (1964).
  * Vaidya and Yashodhara [1968] P. C. Vaidya and P. S. Yashodhara, Tensor (Japan) 19, 191 (1968).
  * Noerdlinger and Petrosian [1971] P. D. Noerdlinger and V. Petrosian, Astrophys. J. 168, 1 (1971).
  * Guha Thakurta [1981] S. N. Guha Thakurta, Indian Journal of Physics 55B, 304 (1981).
  * Patel and Trivedi [1982] L. K. Patel and H. B. Trivedi, Journal of Astrophysics and Astronomy 3, 63 (1982).
  * Vaidya [1977] P. C. Vaidya, Pramana 8, 151 (1977).
  * D’Eath [1975] P. D. D’Eath, Phys. Rev. D 11, 1387 (1975).
  * Gautreau [1984] R. Gautreau, Phys. Rev. D 29, 198 (1984).
  * Cooperstock  _et al._ [1998] F. I. Cooperstock, V. Faraoni, and D. N. Vollick, Astrophys. J. 503, 61 (1998), arXiv:astro-ph/9803097 .
  * Hogan [1990] P. A. Hogan, Astrophys. J. 360, 315 (1990).
  * Buchert and Ehlers [1997] T. Buchert and J. Ehlers, Astron. Astrophys. 320, 1 (1997), arXiv:astro-ph/9510056 .
  * Buchert  _et al._ [2000] T. Buchert, M. Kerscher, and C. Sicka, Phys. Rev. D 62, 043525 (2000), arXiv:astro-ph/9912347 .
  * Nayak  _et al._ [2001] K. R. Nayak, M. A. H. MacCallum, and C. V. Vishveshwara, Phys. Rev. D 63, 024020 (2001), arXiv:gr-qc/0006040 .
  * Baker [2000] G. A. Baker, (2000), arXiv:astro-ph/0003152 .
  * Bolen  _et al._ [2001] B. Bolen, L. Bombelli, and R. Puzio, Class. Quant. Grav. 18, 1173 (2001), arXiv:gr-qc/0009018 .
  * Dominguez and Gaite [2001] A. Dominguez and J. C. Gaite, EPL 55, 458 (2001), arXiv:astro-ph/0106199 .
  * Ellis [2002] G. F. R. Ellis, Int. J. Mod. Phys. A 17, 2667 (2002), arXiv:gr-qc/0102017 .
  * Gao and Zhang [2004] C. J. Gao and S. N. Zhang, Phys. Lett. B 595, 28 (2004), arXiv:gr-qc/0407045 .
  * Sheehan and Kriss [2004] D. P. Sheehan and V. G. Kriss, (2004), arXiv:astro-ph/0411299 .
  * Nesseris and Perivolaropoulos [2004] S. Nesseris and L. Perivolaropoulos, Phys. Rev. D 70, 123529 (2004), arXiv:astro-ph/0410309 .
  * Sultana and Dyer [2005] J. Sultana and C. C. Dyer, Gen. Rel. Grav. 37, 1347 (2005).
  * Li and Wang [2007] Z.-H. Li and A. Wang, Mod. Phys. Lett. A 22, 1663 (2007), arXiv:astro-ph/0607554 .
  * Adkins  _et al._ [2007] G. S. Adkins, J. McDonnell, and R. N. Fell, Phys. Rev. D 75, 064011 (2007), arXiv:gr-qc/0612146 .
  * McClure and Dyer [2006] M. L. McClure and C. C. Dyer, Class. Quant. Grav. 23, 1971 (2006).
  * Sereno and Jetzer [2007] M. Sereno and P. Jetzer, Phys. Rev. D 75, 064031 (2007), arXiv:astro-ph/0703121 .
  * Faraoni and Jacques [2007] V. Faraoni and A. Jacques, Phys. Rev. D 76, 063510 (2007), arXiv:0707.1350 [gr-qc] .
  * Balaguera-Antolinez and Nowakowski [2007] A. Balaguera-Antolinez and M. Nowakowski, Class. Quant. Grav. 24, 2677 (2007), arXiv:0704.1871 [gr-qc] .
  * Mashhoon  _et al._ [2007] B. Mashhoon, N. Mobed, and D. Singh, Class. Quant. Grav. 24, 5031 (2007), arXiv:0705.1312 [gr-qc] .
  * Carrera and Giulini [2010] M. Carrera and D. Giulini, Rev. Mod. Phys. 82, 169 (2010), arXiv:0810.2712 [gr-qc] .
  * Gao  _et al._ [2011] C. Gao, X. Chen, Y.-G. Shen, and V. Faraoni, Phys. Rev. D 84, 104047 (2011), arXiv:1110.6708 [gr-qc] .
  * Giulini [2014] D. Giulini, Stud. Hist. Phil. Sci. B 46, 24 (2014), arXiv:1306.0374 [gr-qc] .
  * Faraoni  _et al._ [2014] V. Faraoni, A. F. Zambrano Moreno, and A. Prain, Phys. Rev. D 89, 103514 (2014), arXiv:1404.3929 [gr-qc] .
  * Kopeikin [2015] S. M. Kopeikin, European Physical Journal Plus 130, 11 (2015).
  * Faraoni  _et al._ [2015] V. Faraoni, M. Lapierre-Léonard, and A. Prain, JCAP 10, 013 (2015), arXiv:1508.01725 [gr-qc] .
  * Mello  _et al._ [2017] M. M. C. Mello, A. Maciel, and V. T. Zanchin, Phys. Rev. D 95, 084031 (2017), arXiv:1611.05077 [gr-qc] .
  * Faraoni [2018] V. Faraoni, Universe 4, 109 (2018), arXiv:1810.04667 [gr-qc] .
  * Macpherson  _et al._ [2019] H. J. Macpherson, D. J. Price, and P. D. Lasky, Phys. Rev. D 99, 063522 (2019), arXiv:1807.01711 [astro-ph.CO] .
  * Guariento  _et al._ [2019] D. C. Guariento, A. Maciel, M. M. C. Mello, and V. T. Zanchin, Phys. Rev. D 100, 104050 (2019), arXiv:1908.04961 [gr-qc] .
  * Neishtadt and Bisnovatyi-Kogan [2020] A. I. Neishtadt and G. S. Bisnovatyi-Kogan, Astron. Rep. 64, 731 (2020).
  * Spengler  _et al._ [2022] F. Spengler, A. Belenchia, D. Rätzel, and D. Braun, Class. Quant. Grav. 39, 055005 (2022), arXiv:2109.03280 [gr-qc] .
  * Agatsuma [2022] K. Agatsuma, Phys. Dark Univ. 38, 101134 (2022), arXiv:2211.15668 [gr-qc] .
  * Bisnovatyi-Kogan and Merafina [2023] G. S. Bisnovatyi-Kogan and M. Merafina, Int. J. Mod. Phys. D 32, 2350030 (2023).
  * Sato and Maeda [1983] H. Sato and K.-i. Maeda, Prog. Theor. Phys. 70, 119 (1983).
  * Axenides  _et al._ [2000] M. Axenides, E. G. Floratos, and L. Perivolaropoulos, Mod. Phys. Lett. A 15, 1541 (2000), arXiv:astro-ph/0004080 .
  * Busha  _et al._ [2003] M. T. Busha, F. C. Adams, R. H. Wechsler, and A. E. Evrard, Astrophys. J. 596 (2003), 2 596, 713 (2003).
  * Larena  _et al._ [2009] J. Larena, J.-M. Alimi, T. Buchert, M. Kunz, and P.-S. Corasaniti, Phys. Rev. D 79, 083011 (2009), arXiv:0808.1161 [astro-ph] .
  * Baumann  _et al._ [2012] D. Baumann, A. Nicolis, L. Senatore, and M. Zaldarriaga, JCAP 07, 051 (2012), arXiv:1004.2488 [astro-ph.CO] .
  * Wiegand and Buchert [2010] A. Wiegand and T. Buchert, Phys. Rev. D 82, 023523 (2010), arXiv:1002.3912 [astro-ph.CO] .
  * Buchert  _et al._ [2015] T. Buchert _et al._ , Class. Quant. Grav. 32, 215021 (2015), arXiv:1505.07800 [gr-qc] .
  * Montanari and Rasanen [2017] F. Montanari and S. Rasanen, JCAP 12, 008 (2017), arXiv:1710.02451 [astro-ph.CO] .
  * Adamek  _et al._ [2019] J. Adamek, C. Clarkson, D. Daverio, R. Durrer, and M. Kunz, Class. Quant. Grav. 36, 014001 (2019), arXiv:1706.09309 [astro-ph.CO] .
  * Nesseris  _et al._ [2022] S. Nesseris _et al._ (Euclid), Astron. Astrophys. 660, A67 (2022), arXiv:2110.11421 [astro-ph.CO] .
  * Schander and Thiemann [2021] S. Schander and T. Thiemann, Front. Astron. Space Sci. 0, 113 (2021), arXiv:2106.06043 [gr-qc] .
  * Koksbang [2021] S. M. Koksbang, Phys. Rev. Lett. 126, 231101 (2021), arXiv:2105.11880 [astro-ph.CO] .
  * Vigneron [2022] Q. Vigneron, Phys. Rev. D 105, 043524 (2022), arXiv:2109.10336 [gr-qc] .
  * Di Valentino  _et al._ [2021] E. Di Valentino, O. Mena, S. Pan, L. Visinelli, W. Yang, A. Melchiorri, D. F. Mota, A. G. Riess, and J. Silk, Class. Quant. Grav. 38, 153001 (2021), arXiv:2103.01183 [astro-ph.CO] .
  * Abdalla  _et al._ [2022] E. Abdalla _et al._ , JHEAp 34, 49 (2022), arXiv:2203.06142 [astro-ph.CO] .
  * Zhang and Zhang [2023] R. Zhang and Z. Zhang, JCAP 06, 031 (2023), arXiv:2303.14047 [physics.gen-ph] .
  * Iorio [2006] L. Iorio, Int. J. Mod. Phys. D 15, 473 (2006), arXiv:gr-qc/0511137 .
  * Kagramanova  _et al._ [2006] V. Kagramanova, J. Kunz, and C. Lammerzahl, Phys. Lett. B 634, 465 (2006), arXiv:gr-qc/0602002 .
  * Croker and Weiner [2019] K. S. Croker and J. L. Weiner, Astrophys. J. 882, 19 (2019), arXiv:2107.06643 [gr-qc] .
  * Croker  _et al._ [2020a] K. S. Croker, K. A. Nishimura, and D. Farrah, Astrophys. J. 889 (2020) no.2, 115 889, 115 (2020a).
  * Croker  _et al._ [2020b] K. S. Croker, J. Runburg, and D. Farrah, Astrophys. J. 900, 57 (2020b).
  * Farrah  _et al._ [2023] D. Farrah _et al._ , Astrophys. J. Lett. 944, L31 (2023), arXiv:2302.07878 [astro-ph.CO] .
  * Cadoni  _et al._ [2023] M. Cadoni, A. P. Sanna, M. Pitzalis, B. Banerjee, R. Murgia, N. Hazra, and M. Branchesi, (2023), arXiv:2306.11588 [gr-qc] .
  * Mistele [2023] T. Mistele, Res. Notes AAS 7, 101 (2023), arXiv:2304.09817 [gr-qc] .
  * Wang and Wang [2023] Y. Wang and Z. Wang, (2023), arXiv:2304.01059 [gr-qc] .
  * Gaur and Visser [2023] R. Gaur and M. Visser, (2023), arXiv:2308.07374 [gr-qc] .
  * Parnovsky [2023] S. L. Parnovsky, (2023), arXiv:2302.13333 [gr-qc] .
  * Avelino [2023] P. P. Avelino, JCAP 08, 005 (2023), arXiv:2303.06630 [gr-qc] .
  * Rodriguez [2023] C. L. Rodriguez, Astrophys. J. Lett. 947, L12 (2023), arXiv:2302.12386 [astro-ph.CO] .
  * Andrae and El-Badry [2023] R. Andrae and K. El-Badry, Astron. Astrophys. 673, L10 (2023), arXiv:2305.01307 [astro-ph.CO] .
  * Lei  _et al._ [2023] L. Lei _et al._ , (2023), arXiv:2305.03408 [astro-ph.CO] .
  * Amendola  _et al._ [2023] L. Amendola, D. C. Rodrigues, S. Kumar, and M. Quartin, (2023), arXiv:2307.02474 [astro-ph.CO] .
  * Arnowitt  _et al._ [2008] R. L. Arnowitt, S. Deser, and C. W. Misner, Gen. Rel. Grav. 40, 1997 (2008), arXiv:gr-qc/0405109 .
  * Cadoni  _et al._ [2022] M. Cadoni, M. Oi, and A. P. Sanna, Phys. Rev. D 106, 024030 (2022), arXiv:2204.09444 [gr-qc] .
  * Bardeen  _et al._ [1973] J. M. Bardeen, B. Carter, and S. W. Hawking, Commun. Math. Phys. 31, 161 (1973).
  * Ma and Zhao [2014] M.-S. Ma and R. Zhao, Class. Quant. Grav. 31, 245014 (2014), arXiv:1411.0833 [gr-qc] .
  * Hawking [1968] S. Hawking, J. Math. Phys. 9, 598 (1968).
  * Hayward [1994] S. A. Hayward, Phys. Rev. D 49, 831 (1994), arXiv:gr-qc/9303030 .
  * Hayward [1996] S. A. Hayward, Phys. Rev. D 53, 1938 (1996), arXiv:gr-qc/9408002 .
  * Faraoni [2015] V. Faraoni, Gen. Rel. Grav. 47, 84 (2015), arXiv:1506.06358 [gr-qc] .
  * [92] M. Cosenza, L. Herrera, M. Esculpi, and L. Witten, J. Math. Phys. 22, 118 (1981) .
  * Herrera and Santos [1997] L. Herrera and N. O. Santos, Phys. Rept. 286, 53 (1997).
  * Raposo  _et al._ [2019] G. Raposo, P. Pani, M. Bezares, C. Palenzuela, and V. Cardoso, Phys. Rev. D 99, 104072 (2019), arXiv:1811.07917 [gr-qc] .
  * Penrose [1965] R. Penrose, Phys. Rev. Lett. 14, 57 (1965).
  * Bardeen [1968] J. M. Bardeen, (1968).
  * Dymnikova [1992] I. Dymnikova, Gen. Rel. Grav. 24, 235 (1992).
  * Bonanno and Reuter [2000] A. Bonanno and M. Reuter, Phys. Rev. D 62, 043008 (2000), arXiv:hep-th/0002196 .
  * Modesto [2004] L. Modesto, Phys. Rev. D 70, 124009 (2004), arXiv:gr-qc/0407097 .
  * Hayward [2006] S. A. Hayward, Phys. Rev. Lett. 96, 031103 (2006), arXiv:gr-qc/0506126 .
  * Ansoldi [2008] S. Ansoldi, in  _Conference on Black Holes and Naked Singularities_ (2008) arXiv:0802.0330 [gr-qc] .
  * Modesto [2010] L. Modesto, Int. J. Theor. Phys. 49, 1649 (2010), arXiv:0811.2196 [gr-qc] .
  * Nicolini [2009] P. Nicolini, Int. J. Mod. Phys. A 24, 1229 (2009), arXiv:0807.1939 [hep-th] .
  * Spallucci and Ansoldi [2011] E. Spallucci and S. Ansoldi, Phys. Lett. B 701, 471 (2011), arXiv:1101.2760 [hep-th] .
  * Frolov [2016] V. P. Frolov, Phys. Rev. D 94, 104056 (2016), arXiv:1609.01758 [gr-qc] .
  * Fan and Wang [2016] Z.-Y. Fan and X. Wang, Phys. Rev. D 94, 124027 (2016), arXiv:1610.02636 [gr-qc] .
  * Simpson and Visser [2019] A. Simpson and M. Visser, Universe 6, 8 (2019), arXiv:1911.01020 [gr-qc] .
  * Lobo  _et al._ [2021] F. S. N. Lobo, M. E. Rodrigues, M. V. de Sousa Silva, A. Simpson, and M. Visser, Phys. Rev. D 103, 084052 (2021), arXiv:2009.12057 [gr-qc] .
  * Akil  _et al._ [2023] A. Akil, M. Cadoni, L. Modesto, M. Oi, and A. P. Sanna, Phys. Rev. D 108, 044051 (2023), arXiv:2211.01657 [gr-qc] .
  * Bayin [1986] S. S. Bayin, Astrophys. J. 303, 101 (1986).
  * Culetu [2013] H. Culetu, J. Phys. Conf. Ser. 437, 012005 (2013), arXiv:1201.3769 [gr-qc] .
  * Cadoni and Sanna [2021a] M. Cadoni and A. P. Sanna, Int. J. Mod. Phys. A 36, 2150156 (2021a), arXiv:2012.08335 [gr-qc] .
  * Cadoni and Sanna [2021b] M. Cadoni and A. P. Sanna, Class. Quant. Grav. 38, 135004 (2021b), arXiv:2101.07642 [gr-qc] .
  * Chellathurai and Dadhich [1990] V. Chellathurai and N. Dadhich, Classical and Quantum Gravity 7, 361 (1990).


# Using a GAN to Generate Adversarial Examples to Facial Image Recognition

Andrew Merrigan 1 and Alan F. Smeaton 1,2
1School of Computing and 2Insight Centre for Data Analytics
Dublin City University, Glasnevin, Dublin, 9, Ireland.
Email<EMAIL_ADDRESS>
###### Abstract

Images posted online present a privacy concern in that they may be used as
reference examples for a facial recognition system. Such abuse of images is in
violation of privacy rights but is difficult to counter. It is well
established that adversarial example images can be created for recognition
systems which are based on deep neural networks. These adversarial examples
can be used to disrupt the utility of the images as reference examples or
training data. In this work we use a Generative Adversarial Network (GAN) to
create adversarial examples to deceive facial recognition and we achieve an
acceptable success rate in fooling the face recognition. Our results reduce
the training time for the GAN by removing the discriminator component.
Furthermore, our results show knowledge distillation can be employed to
drastically reduce the size of the resulting model without impacting
performance indicating that our contribution could run comfortably on a
smartphone.111(c) Copyright of this article remains with the Society for
Imaging Science and Technology (IS&T), 2022

## Introduction and Problem Statement

Deep neural networks (DNNs) have superior performance to more conventional
approaches when applied to a variety of computer vision tasks including object
recognition, activity detection and facial recognition. DNN’s are however
susceptible to adversarial examples which are images crafted so that the
machine’s interpretation differs greatly from that of a person looking at the
same image [3]. Several recent works have shown that such adversarial examples
are actually effective against a variety of trained models, not just those
they have been generated to mislead [3].

The use of facial recognition systems (FRS) coupled with unauthorised use of
images taken from social media is a threat to individuals’ privacy. The use of
facial recognition systems is increasing worldwide. The development of open-
source facial recognition models based on DNNs has made it possible for anyone
with access to moderate computing power to create an FRS. More troubling is
corporate use of images from social media such as Clearview.ai who have
scraped approximately 3 billion images for an FSR without individuals’ consent
[23].

The work reported in this paper examines the feasibility of employing the
AdvGAN++ architecture to generate adversarial examples targeted against facial
recognition. This design was proposed and tested in the generation of
adversarial examples using the MNIST and CIFAR-10 datasets. As it has yet to
be examined in the context of facial recognition its performance
characteristics in this task are unknown.

Furthermore, the actual utility of discriminators in the generation of
adversarial image examples via a GAN will be examined by determining the
discriminator’s function within the network, ultimately with the possibility
of its removal. Removing the discriminator from the network would provide a
significant reduction in the computational cost of training the network. It
would also reduce the complexity of the model making the training of the model
simpler by removing many of the hyper-parameters.

In summary, the motivation behind this work is to create a small effective
model, which can be deployed on user devices to enable them to convert images
into adversarial examples, prior to putting them online and the novelty and
contribution of the paper is the reduction and simplification of the process
of generating adversarial faces. The rest of this paper is organised as
follows. In the next section, we describe some related work, followed by a
section on the datasets we used. The subsequent section describes our
experiments, covering model design, performance, the removal of the
discriminator from the GAN and the distillation of the generator’s knowledge.

## Related Work

There are many approaches to facial recognition however the general pipeline
consists of four stages namely detecting, alignment, representation and
verification [17]. Through the use of deep convolutional neural networks
(DCNNs), facial recognition systems have reached a level of human-like
performance including on the set of “Labeled Faces in the Wild” (LFW) dataset.
However, problems still exist because of the high degree of variability among
features such as head position, lighting, facial expression and ageing. These
challenges cumulatively have been termed the problem of PIE (pose-
illumination-expression) in face recognition. The detection stage is no longer
considered a challenging task for frontal faces.

Research into adversarial examples in facial recognition typically targets the
representation stage which is responsible for creating a feature vector to
represent a given face in an image. There are a number of methods for
generating adversarial examples which either take an optimisation approach or
use a neural network to generate a perturbation. In optimisation based methods
the parameters of the model are held constant while an algorithm attempts to
find changes (or perturbations) that can be made to the input image to achieve
the desired output. This is done either within the constraints that the
perturbation is less than some maximum or within some bounded area of the
input. Many different approaches have been suggested to solve the optimisation
problem such as gradient descent [4], FGSM [14], L-BFGS [19] and Projected
Gradient Descent [13].

Originally the emergence of adversarial examples was suggested to originate in
the non-linearity of deep neural networks or in insufficient regularisation
being applied to a model. However, it can be shown that adversarial examples
arise from a model being too linear. It has also been demonstrated that
training using adversarial examples with correct labels termed adversarial
training, can be used to provide regularisation benefits [5]. Several papers
have shown that these adversarial examples are transferable. This means that
adversarial examples are not limited to specific DCNN models or the models
they have been trained against. This makes it unnecessary to know exactly what
models the adversarial example will be used against. The transferability will
be maximised between models with similar architecture, number of parameters
and with high test accuracy [11].

Broadly, the research into adversarial examples can be placed into two
different categories. One category calculates a minimal set of pixel value
changes based on a specific input image which is the approach we take here.
The benefit of this approach is that the original and modified images are
usually indistinguishable from each other. In some literature this approach of
bespoke modification of an input image is termed “cloaking”. The effectiveness
of a cloaking method is not just measured by the proportion of test samples
which are incorrectly identified but also by how visually similar the cloaked
image is to the original image. The primary disadvantage of this approach is
the computational cost of cloaking an input image. To cloak a single image
takes approximately 1 minute on a modern CPU powered computer [18].

There are several methods which can be used to assess the similarity of two
images from simple approaches that only consider the mean squared error of the
averaged pixel intensities, to methods which consider the capabilities of
human vision [24]. The approach found in the literature is a typically
structural similarity index measure (SSIM) or a derived value such as DSSIM,
which is a measure of the structural dissimilarity [18]. Since SSIM is a full
reference metric both the original and distorted images are required to
compute the value. SSIM compares local patterns within the image after pixel
values have been normalised for luminance and contrast [24].

Generating adversarial examples using a neural network takes considerably less
time when compared with optimisation approaches. [1] found that When using a
neural network to generate adversarial examples the resulting training process
is similar to that of a Generative Adversarial Network (GAN) where the goal is
to find a weakness in another model. The authors in [2] propose Adversarial
Transformation Networks (ATNs). ATNs come in two distinct types, one model
type which is based on a CNN termed a Perturbation ATN (P-ATN) and another
based on an autoencoder called an Adversarial Autoencoding (AAE). AAE may have
some issues in terms of detectability in that they remove the high frequency
data from the input image due to limitations in the architecture [2].

Some researchers have used modified GANs to create adversarial examples and
examples include AdvGAN [21], SLP-GAN [10], and Attack-Inspired GAN (AI-GAN)
[1].

AdvGAN has a generator which attempts to create a perturbation for an input
image to fool two separate adversaries. One of the adversaries attempts to
find the modified image when given both the original and perturbed image. The
other adversary is the network that is being targeted [21]. The performance
and training time of AdvGAN was later improved by using a feature space
representation of the input image to train the generator rather than the whole
input image [9].

An adaption of the original AdvGAN termed PcAdvGAN is similar to the improved
AdvGAN++ in that it first encodes the input image prior to the generator
network. The encoding used however was based on principle component analysis
of different segments of the input image. Unlike the other GAN designs this
model was tested on facial recognition systems not classification problems
[22].

AI-GAN is a similar design to that of the first implementation of AdvGAN
however it incorporates the desired target class for the input image into the
generator, as well as incorporating the target into the loss function. This
contrasts with the AdvGAN in which the adversarial example will have a non-
specific class selected by the generator [1].

## Datasets

In this work the CelebFaces Attributes Dataset  (CelebA) dataset was used as a
training set with the Labeled Faces in the Wild (LFW) dataset split into a
validation and test set [12, 8]. CelebA is a large-scale face attribute
dataset containing 202,506 images of 10,177 individuals [12]. The LFW dataset
contains 13,233 images of 5,749 individuals, it is a publicly available
benchmark focused on face verification of subjects in unconstrained
environments [8].

CelebA was used for training because whilst the images are typically much more
constrained when compared to the LFW, CelebA is much larger. Therefore, the
LFW dataset is valuable for testing as it contains images of individuals in
more natural unconstrained settings. Using the LFW dataset for testing also
allows for comparisons with work by others.

The images in both datasets are of varying sizes but are all in JPEG format
with each accompanied by a unique name for the primary subject of the image,
who is featured prominently in the centre with some images also including
multiple other individual faces [8]. A sample of faces from CelebA is shown in
Figure 1.

Figure 1: Some original aligned images in the CelebA dataset

The LFW dataset was split into a validation set of 2,677 images and a test of
10,552 images. Four images of Hollywood actor Nicolas Cage were reserved from
the dataset for use as a target image. This ensured a number of images of each
individual were present, which maximises the distance that an image’s
representation must be moved to be incorrectly classified.

The images in the LFW dataset do not contain close-cropped images of the
subjects’ faces that are typically used as an input in face verification
tasks. To transform the images into this form a pre-trained CNN model was used
to place bounding boxes over each of the faces in an image. The largest of
these was then used to crop the primary subject’s face from the original
image. Initial attempts to use a HOG model resulted in several images where
faces were obscured, being excluded or poorly cropped. This cropping also
removes interference from other individuals whose faces may be present in the
same image. It also reduces the variance between images as variations in pose
and the image background are largely removed [8]. After cropping, all of the
images were scaled using bilinear interpolation to 160x160 pixels in size.

A concern with the dataset is the lack of variance from lighting, pose, and
perspective transformation but also the range of characteristics of subjects
in the images. The dataset has a disproportionately large number of males
compared to females. Many races and ethnicities are not represented highly in
the dataset or even at all. The age range of subjects is also restricted with
few young or old individuals [8]. However, this dataset is used as a benchmark
for several facial recognition models which will allow the success of our own
created adversarial models to be measured in terms of decreases in accuracy
between the original LFW dataset and an adversarial version of the LFW
dataset.

## Experiments

### Model Design

Initial work was performed on an AdvGAN++ architecture modified to operate
against facial recognition models. The original AdvGAN++ architecture is shown
in Figure 2.

Figure 2: Original AdvGAN++ Architecture taken from [9]

One of the main differences between the targeted and untargeted tasks is the
output of the model, which for the target facial recognition model used in our
experiments was a 128-digit encoding. This required that the loss function had
to be adjusted to accommodate this change and the loss function for the
generator is formulated as:

$L(G)=\alpha L_{GAN}+\beta L_{adv}+\gamma L_{pert}$

where $L_{GAN}$ is the binary cross-entropy loss of the discriminator,
$L_{adv}$ is a measure of the error introduced into the output of the target
model by the changes made by the generator and $L_{pert}$ is a measure of how
much the input image was modified. The terms $\alpha$, $\beta$ and $\gamma$
are hyperparameters used to adjust the relative importance of each term. The
tuning of these three components was difficult and resulted in large stability
problems.

The $L_{adv}$ of the original network was binary cross-entropy which was
changed to mean squared error after experimentation with mean squared error,
2-norm and cosine distance. This was used for both the targeted and non-
targeted versions of the network. However, in the case of the non-target model
of operation the loss function was instead maximised.

The $L_{pert}$ term is very important in the functioning of the network. If
$L_{pert}$ is more permissive the model changes the structure of the face
acting like a style transfer model in the case of a targeted attack or more
generally as a form of anonymisation in the case of a non-targeted attack. A
restrictive $L_{pert}$ instead results in adversarial noise which is not
readily apparent to a human observer. Experiments were conducted on several
different functions for $L_{pert}$. These sought to maintain the total
modification of each pixel below a specified threshold. An alternate
experiment instead sought to minimise the SSIM between the original and
modified image.

The structure of the generator is shown in Figure 3 as a fully convolutional
network comprised of up-sampling, convolutional and batch normalisation
layers. It is designed to convert the features of some of the last layers of
the target model back into the same dimensions as the original input image.
However the pixel values in the outputted perturbation range in value from -1
to +1 so that the model can learn to both increase and decrease each pixel of
the input image. The perturbation is then summed with the input image and the
result clipped to the range 0 to 1. Originally the increases in the dimensions
were achieved using transposed convolutional layers however this led to a
tiling effect on the output image. To address this the transposed
convolutional layers were replaced with up-sampling.

Figure 3: Architecture of generator

The target model used both in white box experiments and as the feature
extraction layers, is an implementation of Facenet which was pre-trained with
an accuracy of 0.9963 on the LFW dataset [16]. Using the feature extraction
layers of the target model acts as a type of transfer learning, though
existing work does not describe it as such. On the basis that this is a form
of transfer learning, experiments added an additional fine-tuning stage to the
training. Consequently some of the higher feature extraction layers of the
target model were made trainable and the model was allowed to train again with
a lower learning rate.

### Performance

Basic results for the created models are presented in Table 1 which includes
the introduction of image blurring which is a very basic defence against
adversarial examples [6]. The model in several different versions achieves a
higher success rate when compared with Fawkes [18], a previously proposed
iterative cloaking method. However, the successful models introduce
considerably more graphical changes to the input images than the iterative
cloaking method. In some of the generated images the generator acts more as a
form of anonymisation than cloaking. Anonymisation whilst destructive is more
robust to defensive techniques. These higher-performing models however also
tend to introduce more spurious artefacts, examples of which, both generated
by the same model are presented in Figure 4.

Table 1: Performance Characteristics Of Various Models Model |  Success rate against white box recogniser |  Success rate against black box recogniser |  Success rate against white box after blurring |  $L_{pert}$ Threshold used in loss function |  Training Time (mins per Epoch) |  Time to generate example image (secs)
---|---|---|---|---|---|---
SSIM |  0.84 |  0.81 |  0.80 |  n/a |  20 |  0.024
High Threshold |  0.97 |  0.94 |  0.96 |  0.2 |  20 |  0.026
Medium Threshold |  0.98 |  0.99 |  0.87 |  0.1 |  20 |  0.025
Low Threshold |  0.65 |  0.55 |  0.60 |  0.001 |  20 |  0.025
Medium Threshold No Target |  0.96 |  0.88 |  0.95 |  0.05 |  20 |  0.026
Medium Threshold No Discriminator |  0.98 |  0.97 |  0.59 |  0.1 |  15 |  0.024
Fawkes System (Shan et al., 2020) |  0.74 |  0.97 |  0.54 |  n/a |  0 |  8.001

Figure 4: Visual artefacts, original (T) and modified (B) images

All models were tested on a black box, a pre-trained implementation of
VGGFace222https://github.com/rcmalli/keras-vggface of which they had no
existing knowledge or access. The performance of many of the models are
maintained when deployed against a blackbox system. However this is contrasted
by the performance of the the iterative approach which actually performs
better against the blackbox system.

As a GAN based approach uses a fully trained model as opposed to an iterative
approach like Fawkes which only makes use of feature extractors from a trained
model, the time to generate an adversarial example is far lower. However, the
iterative approach does not require any explicit training as opposed to the
GAN which requires 20 minutes per epoch.

### Removal of the Discriminator

The GAN used throughout the experiments was retrained without the
discriminator both to measure the model’s performance and the computation time
required for training. The success rate of that model is 0.98 against a white
box system. This is comparable to that of a model with a medium perturbation
threshold trained with a discriminator which had a success rate of 0.96
against a white box system. Though the success rate has only been reduced by
0.02 against the white box system, the training time per epoch reduced from 20
to 15 minutes. This experiment is listed as “Medium Threshold no
Discriminator“ in Table 1. Without a discriminator as part of the
architecture, instabilities during training are reduced allowing for higher
learning rates. The initial experiments all focused on the use of an Adam
optimiser with a learning rate of 1e-4 with beta1 set to 0.5, however without
a discriminator the network can be trained with a learning rate of 1e-3 with
beta1 at its default of 0.9. Whilst the performance is comparable with that of
the original GAN as the threshold for pixel value modification increases, the
results rapidly begin to suffer as shown in Figure 5.

Figure 5: Results generated without a discriminator in the GAN, medium
threshold (L) and high threshold (R)

### Role of the Discriminator

Adversarial examples can be deployed as part of a poisoning attack used to
pollute a pool of reference examples used by an FRS to identify an individual.
By inserting adversarial images into this pool genuine images taken from an
uncontrolled source such as a surveillance camera will not match reference
images. As the goal in the creation of adversarial examples is to have them
inserted into the reference pool it is important that they are not easily
detectable as an adversarial example.

As illustrated previously, the same success rate in the creation of
adversarial examples may be obtained in a model trained without a
discriminator when the level of pixel-level modification is been tightly
controlled. We may also find that the discriminator provides an additional
benefit such as preventing the adversarial examples from being easily
identifiable. If the discriminator was preventing detection then it should be
able to distinguish adversarial from unmodified images but the probability of
all images tested was $\sim 0.5$ regardless of whether they were modified or
not. As the discriminator is not learning to detect adversarial images, it is
inferred that the discriminator is not teaching the generator to produce
adversarial which are more difficult to detect.

### Model size reduction

The created adversarial model contains 426 layers making the values with a
total of 22,137,171 parameters with an H5 model size of 85.5MB. To
successfully deploy this model in a real-world setting it would ideally have
fewer parameters . We applied knowledge distillation to transfer the knowledge
gained from an adversarial model which includes the feature extraction layers
taken from an FRS model to a much smaller model. Knowledge distillation is a
method of model compression that attempts to transfer knowledge from a teacher
model to student models while maintaining accuracy [7].

For this experiment, the knowledge learned by the middle threshold model,
trained without a discriminator as well as feature extraction layers was
transferred to a smaller U-Net architecture [15]. The resultant distilled
model had a total of 2,058,979 parameters with an H5 model size of 8.04MB. The
success rate of the distilled model is 0.98 compared to the teacher model
which has a success rate of 0.98. From this, it is evident that in this case
the overall size of the model can be reduced whilst maintaining the
performance.

### Visualising the Generator Operation

One approach taken to visualise the effect of the model was to use
t-distributed stochastic neighbour embedding (t-SNE) [20] to reduce the
dimensionality of a facial recognition’s embedding of modified and original
images such that they can be visualised. The results of this are shown in
Figure 6.

Figure 6: Plot of dimensionally-reduced embeddings using t-SNE for a model
with a target Figure 7: Plot of dimensionally-reduced embeddings using t-SNE
for a model without a target

In Figure 6 we see that as this model is targeted all of the images have been
shifted from their original embedding to a position closer to the target
image. This confirms that the original images have been altered, causing their
embeddings to form a cluster around the target, far from their original
position. This is in contrast to the t-SNE of the untargeted model in which
the modifications result in movements in non-specific directions as shown in
Figure 7.

## Conclusions

The work reported here shows that the AdvGAN++ architecture previously applied
to classification on the MNIST and CIFAR-10 datasets can be applied to create
adversarial examples for facial recognition. The created models have been
revealed to have comparable performance to the more traditional iterative
approach with a substantially reduced time to generate examples. It has been
demonstrated that so long as the objective of the network is to produce
imperceptible adversarial noise then the use of a discriminator is an
unnecessary computational burden. Our experiments also show that the model can
be reduced in size substantially through knowledge distillation without an
appreciable loss in performance.

We acknowledge that if adversarial image generation for face recognition such
as we propose here were to become popular, then face recognition algorithms
themselves could pivot and be trained on adversarial images rather than on
unaltered images, and so the “cold war” between face recognition and deception
or polluting the models, would continue. Also, as part of our future work it
would be interesting to add multiple white-box and black-box detection models
and to perform diversity experiments on several different datasets of face
recognition.

### Acknowledgements

This work was part-funded by Science Foundation Ireland through grant number
(SFI/12/RC/2289_P2).

## References

  * [1] Tao Bai, Jun Zhao, Jinlin Zhu, Shoudong Han, Jiefeng Chen, Bo Li, and Alex Kot.  AI-GAN: Attack-Inspired Generation of Adversarial Examples.  In 2021 IEEE International Conference on Image Processing (ICIP), pages 2543–2547. IEEE, 2021.
  * [2] Shumeet Baluja and Ian Fischer.  Adversarial transformation networks: Learning to generate adversarial examples.  arXiv preprint arXiv:1703.09387, 2017.
  * [3] Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer.  Adversarial patch.  arXiv preprint arXiv:1712.09665, 2018.
  * [4] Neil Fendley, Max Lennon, I-Jeng Wang, Philippe Burlina, and Nathan Drenkow.  Jacks of All Trades, Masters of None: Addressing Distributional Shift and Obtrusiveness via Transparent Patch Attacks.  In Adrien Bartoli and Andrea Fusiello, editors, Computer Vision – ECCV 2020 Workshops, pages 105–119, Cham, 2020. Springer International Publishing.
  * [5] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [6] Shixiang Gu and Luca Rigazio.  Towards deep neural network architectures robust to adversarial examples.  arXiv preprint arXiv:1412.5068, 2015.
  * [7] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  2015\.
  * [8] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller.  Labeled faces in the wild: A database for studying face recognition in unconstrained environments.  Technical Report 07-49, University of Massachusetts, Amherst, October 2007\.
  * [9] Surgan Jandial, Puneet Mangla, Sakshi Varshney, and Vineeth Balasubramanian.  Advgan++: Harnessing latent layers for adversary generation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019.
  * [10] Yongsu Kim, Hyoeun Kang, Afifatul Mukaroh, Naufal Suryanto, Harashta Tatimma Larasati, and Howon Kim.  Spatially localized perturbation GAN (SLP-GAN) for generating invisible adversarial patches.  In Ilsun You, editor, Information Security Applications, pages 3–15, Cham, 2020. Springer International Publishing.
  * [11] Alexey Kurakin, Ian Goodfellow, and Samy Bengio.  Adversarial examples in the physical world.  2017\.
  * [12] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [13] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  In International Conference on Learning Representations, 2018.
  * [14] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami.  The limitations of deep learning in adversarial settings.  In 2016 IEEE European symposium on security and privacy (EuroS&P), pages 372–387. IEEE, 2016.
  * [15] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional Networks for Biomedical Image Segmentation.  In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  * [16] Florian Schroff, Dmitry Kalenichenko, and James Philbin.  Facenet: A unified embedding for face recognition and clustering.  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  * [17] S. I. Serengil and A. Ozpinar.  Lightface: A hybrid deep face recognition framework.  In 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), pages 1–5, 2020.
  * [18] Shawn Shan, Emily Wenger, Jiayun Zhang, Huiying Li, Haitao Zheng, and Ben Y Zhao.  Fawkes: Protecting privacy against unauthorized deep learning models.  In 29th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 20), pages 1589–1604, 2020.
  * [19] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
  * [20] Laurens Van der Maaten and Geoffrey Hinton.  Visualizing data using t-SNE.  Journal of Machine Learning Research, 9(11), 2008.
  * [21] Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song.  Generating adversarial examples with adversarial networks.  In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 3905–3911, 2018.
  * [22] J. Yang, J. Liu, and J. Wu.  Facial image privacy protection based on principal components of adversarial segmented image blocks.  IEEE Access, 8:103385–103394, 2020.
  * [23] J. Zhang and C. Li.  Adversarial examples: Opportunities and challenges.  IEEE Transactions on Neural Networks and Learning Systems, 31(7):2578–2593, 2020.
  * [24] Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.

## Author Biography

Andrew Merrigan obtained an M.Sc. in Data Analytics from Dublin City
University in 2021 and is a senior software engineer working for Blockdaemon
Limited in the development of a multi-chain API for accessing blockchain data
with a specialism in blockchain technology.

Alan Smeaton is Professor of Computing at Dublin City University. He received
his PhD from University College Dublin and is an elected member of the Royal
Irish Academy and a winner of the Academy’s Gold Medal in Engineering
Sciences, an award given to individuals who have made an outstanding scholarly
contribution in their fields. He is a Fellow of IEEE and a Principal Fellow of
AdvanceHE.