# Casimir Effect and Gravitational Balance:
a Search for Stable Configurations

Leonardo Bellinato Giacomelli  Institut für Theoretische Physik, TU Wien,
Wiedner Hauptstrasse 8-10, 1040 Vienna, Austria  Universität Innsbruck,
Fakultät für Mathematik, Informatik und Physik, Institut für
Experimentalphysik, 6020 Innsbruck, Austria  Benjamin Koch E-mail:
<EMAIL_ADDRESS>Institut für Theoretische Physik, TU Wien, Wiedner
Hauptstrasse 8-10, 1040 Vienna, Austria  Instituto de Física, Pontificia
Católica Universidad de Chile, Av. Vicuña Mackenna 4860, Santiago, Chile  Iva
Lovrekovic  Institut für Theoretische Physik, TU Wien, Wiedner Hauptstrasse
8-10, 1040 Vienna, Austria  Angel Rincon E-mail<EMAIL_ADDRESS>Departamento de Física Aplicada, Universidad de Alicante, Campus de San
Vicente del Raspeig, E-03690 Alicante, Spain.

(July 2023)

###### Abstract

In this study, we examine the role of the repulsive Casimir force in
counteracting the gravitational contraction of a thin spherically symmetric
shell. Our main focus is to explore the possibility of achieving a stable
balanced configuration within the theoretically reliable weak field limit. To
this end, we consider different types of Casimir forces, including those
generated by massless scalar fields, massive scalar fields, electromagnetic
fields, and temperature-dependent fields.

###### Contents

  1. 1 Introduction
    1. 1.1 Gravitational balance
    2. 1.2 Casimir effect
    3. 1.3 Thin shells as toy model for gravitational collapse
    4. 1.4 The research question
    5. 1.5 Structure of the paper
  2. 2 Results
    1. 2.1 The setup
    2. 2.2 Massless scalar field
    3. 2.3 Massive scalar field
    4. 2.4 Temperature dependence
      1. 2.4.1 High temperature for scalar field
      2. 2.4.2 Low temperature for scalar field
      3. 2.4.3 Electromagnetic Casimir at low temperature and large $R\gg r_{S}$
  3. 3 Conclusion
  4. 4 Acknowledgements

##  1 Introduction

###  1.1 Gravitational balance

Gravitational balance is a fascinating area of research within astrophysics
and general relativity that explores the processes by which massive objects,
such as stars, balance out with their gravitational force or collapse under it
[1, 2, 3, 4, 5, 6, 7]. Research in this area continues to push the boundaries
of our understanding of the universe, offering tantalizing prospects for
breakthroughs in both the theoretical and observational realms. It pushes our
understanding of the laws of nature to its boundaries.

Among the possible forces that might stabilize a gravitational system, one has
considered electromagnetic forces, strong forces, the Pauli exclusion
principle, and quantum vacuum energy. The former forces are directly
associated with the coupling strength of the standard model of particle
physics. The latter forces are of totally different nature, because their
existence is directly linked to genuine quantum effects.

Among these candidate forces, the quantum vacuum energy is certainly the least
explored and least understood. One reason for our limited knowledge of vacuum
energy is certainly the difficulty in measuring it [8]. Another reason is the
fact that the role of quantum vacuum energy is largely unclear when it is
coupled to gravity. This lies at the core of the infamous cosmological
constant problem. Which is the large discrepancy between the theoretical
predictions of vacuum energy density from quantum field theory and the
observed value of this density from astronomical observations at cosmological
scales, which differ by approximately 120 orders of magnitude [9].
Understanding the interplay between vacuum energy density and gravity might
also play a non-trivial role in the evolution process of compact gravitational
objects [10, 11]. This is the region of high curvature that we do not explore
here.

It is thus an interesting research objective to learn more about vacuum energy
in general (see subsection 1.2) and in particular in the context of a
gravitational collapse for the thin shells similar as in Israel (see 1.3 and
the following). Due to the specific nature of the theory, we expect our
observations to appear at extreme scales.

###  1.2 Casimir effect

Studies of the Casimir effect span multiple disciplines. It is important in
quantum field theory, atomic and molecular physics, condensed matter physics,
gravity and cosmology, mathematical physics, and others [12]. For instance, in
Quantum Field Theory, bag models of hadrons, the Casimir modes of quarks, and
gluons significantly contribute to the total nucleon energy. In Kaluza-Klein
theory, the Casimir effect provides an effective mechanism for the spontaneous
compactification of extra dimensions. In Condensed Matter Physics, the Casimir
effect can be either attractive or repulsive between closely aligned material
boundaries, strongly depending on the geometry of the boundaries, temperature,
and the mechanical and electrical properties of the boundary surface [13].

In the bag model of Quantum Chromodynamics (QCD), the boundary conditions on
the spinor and gluon fields are related through equations of motion.
Essentially, there is a Lagrangian model for the bag, along with a
phenomenological Lagrangian description for the quantum behavior of the fields
within the bag. Although the MIT bag model, since its discovery long ago, has
been largely replaced by newer methods in QCD to study hadrons, it was a
successful phenomenological description in its time [14, 15]. The accuracy of
this phenomenological effective description led to numerous discoveries [16,
17].

Boyer demonstrated the importance of geometry in calculating the Casimir force
[18]. He investigated the quantum electromagnetic zero-point energy (known as
the Casimir energy) of a conducting spherical shell. By analogy with the
parallel plate configuration, he expected the negative contribution from zero-
point energy to induce a collapse of the sphere, balanced by the electrostatic
energy of the configuration. To his surprise, the Casimir energy contribution
was positive. He considered quantum electromagnetic modes within the sphere.
Further studies yielded interesting results on the dependence on geometry
[12]. Specifically, the energy contribution for a rectangular cavity can vary
between positive and negative values, depending on the dimensions of the
cavity. In one case, it may be positive, while in another it may be negative.

There have been numerous studies on the conducting sphere initiated by Boyer
[18] and followed by analyses using Green?s function methods [19, 20]. These
include the calculation of the vacuum energy for (i) spinors in the bag model
[21, 22], (ii) massive scalar enclosed in the system [23], (iii) and for
fermions [24]. For the massless scalar field inside a sphere, the Casimir
energy is given by

$\displaystyle E_{Cas}=\frac{\hbar
c}{R}\left(0.0044+\frac{1}{630\pi}\left[\frac{1}{s}+\left(\frac{\mu
cR}{\hbar}\right)\right]\right),$ (1.1)

where sphere is of a radius R in D=3 dimensions with Dirichlet boundary
conditions [13]. Also experimental studies of this effect are intensified
recently [25].

In the gravitational case, the analysis of quantum modes within an analogous
conducting sphere is more complex and has not been extensively considered.
However, studies have been conducted on gravitons in the Casimir box, showing
a linear dependence compared to the analysis for scalars. In the scenario with
two infinite parallel planes separated by distance $d$, linearized gravity,
similar to electromagnetism, can be represented as two free massless scalar
fields?one with Neumann boundary conditions and one with Dirichlet boundary
conditions. The calculated partition function can then be obtained also by
studying a single massless scalar field on an interval of length $2d$ with
periodic boundary conditions [26].

###  1.3 Thin shells as toy model for gravitational collapse

In this paper, on phenomenological grounds, we consider a scenario in which a
thin spherical shell with non-vanishing mass counteracts the repulsive Casimir
force due to its gravity. The Casimir energy arises from the fluctuations of
the dynamical field both inside and outside the shell.

If we adopt the line of thinking used in the MIT bag model to a scenario
involving gravity, we can attempt to model the interplay between zero-point
energy and gravitational effects. To utilize the result for the Casimir energy
of dynamical field inside a perfectly conducting sphere, we can imagine a
model similar to a bag, but with quantum fluctuations occurring on both sides
of the boundary. The sphere is massive and is described in flat space,
experiencing gravitational effects due to its mass. In realistic physical
scenarios, this description would be too simplistic. It would be necessary to
carefully determine the energy of particles fluctuating within the system and
the scale at which these fluctuations would be sufficient to counterbalance
the mass.

A very instructive toy model in this direction was introduced by Brevik [27],
where he included the contribution of Casimir energy to the gravitational
system of a shell by hand. In his model, the shell contracts to a minimum
radius, at which point it encounters a barrier at the speed of light. Without
the Casimir force, this setup would lead to the collapse of the shell.

###  1.4 The research question

As a toy model, we use a thin spherical shell that experiences both
gravitational and vacuum effects simultaneously. A description of this system
was already implemented by Brevik, as mentioned in the previous section [27].
In his work, the focus is on a regime of strong curvature and large
velocities. However, in this regime, the Newtonian concept of simply summing
forces to obtain a total force is likely not valid or is at least obscured by
significant theoretical uncertainties.

Therefore, in our work, we will focus on a regime of low curvature and small
velocities, where the Newtonian approach of adding the Casimir force to the
gravitational equations of motion is certainly valid. Consequently, we
concentrate on the following research question:

“Under which circumstances can Casimir forces counteract gravitational
attraction and lead to stable configurations, even in the non-relativistic,
low-curvature limit?”

In this regime of non-relativistic dynamics, we consider three distinct
scenarios:

  * $\alpha$)

The shell collapses, with gravitational forces overwhelming the Casimir force.

  * $\beta$)

The shell expands from its initial radius due to a stronger Casimir force.

  * $\gamma$)

The shell oscillates around a stable radius.

Our primary focus is to explore whether the dynamics of the Casimir shell is
always destructive (either $\alpha$ or $\beta$), or if there exists a region
in the parameter space and initial conditions where the shell is stable
($\gamma$). We will classify different regimes and types of Casimir forces
according to the scenarios ($\alpha,\beta,\gamma$) they produce for the
spherical massive shell.

###  1.5 Structure of the paper

In the following section we will define the setup of a thin self-gravitating
shell 2.1. Then, we will explore this system for different types of Casimir
forces and categorize it into the scenarios $\alpha$, $\beta$, $\gamma$. In
particular, we will consider a massless scalar field, a massive scalar field,
and temperature dependence for massless scalar and electromagnetic fields.

##  2 Results

In this section, we explore various avenues to determine whether they have the
potential to provide an oscillating scenario $\gamma)$.

###  2.1 The setup

If we have a vacuum region surrounding any spherically symmetric matter
distribution, and we describe this vacuum region with a metric, there exist
coordinates in which the metric takes the standard Schwarzschild form, with a
parameter $M_{0}$ equal to the enclosed interior mass. This holds true even
when the vacuum region itself is embedded in a spherically symmetric
distribution of matter, as verified in [28]. It is important to note that when
considering specific scenarios, we must be careful in interpreting Birkhoff’s
theorem (see Appendix A for a discussion). For a metric with the diagonal line
element:

$\displaystyle ds^{2}=-A(r)dt^{2}+B(r)dr^{2}+r^{2}d\Omega^{2}$ (2.1)

with a priori arbitrary functions $A(r)$ and $B(r)$, we can consider a thin
spherical shell by first considering a shell with mass density $\rho(r)$ and
finite thickness, which supports tangential pressure $p(r)$. However, there is
no radial pressure component. If the shell consists of this specific ”fluid”
with zero radial stress and non-zero compressive stress, it is possible to
have a static system. If the radial stress did not vanish, it would be
required that at the boundaries with interior and exterior vacua, the stress
matches to zero. In this model, the components of the energy-momentum tensor
within the shell are $T_{\mu\nu}=\text{diag}(\rho
A,0,pr^{2},pr^{2}\sin^{2}\theta)$. In this scenario, in the interior region,
one obtains [28]

$\displaystyle
ds^{2}=-\frac{r_{S}-2(m_{i}+m_{S})G_{N}}{r_{S}-2m_{i}G_{N}}\left(1-\frac{2m_{i}G_{N}}{r}\right)dt^{2}+\frac{dr^{2}}{1-\frac{2m_{i}G_{N}}{r}}+r^{2}d\Omega^{2}.$
(2.2)

The metric in the exterior region is

$\displaystyle
ds^{2}=-\left(1-\frac{2(m_{i}+m_{S})G_{N}}{r}\right)dt^{2}+\frac{dr^{2}}{1-\frac{2(m_{i}+m_{S})G_{N}}{r}}+r^{2}d\Omega^{2}.$
(2.3)

Here, $d\Omega^{2}=d\theta^{2}+\sin^{2}\theta d\phi^{2}$, $m_{i}$ is the
enclosed mass, $m_{S}$ is the mass of the shell, and $r_{S}$ is the radial
position of the shell. This is the metric we will consider in our
calculations, while setting the mass inside the shell $m_{i}$ to zero
($m_{\text{i}}=0$). When $m_{i}=0$, the spacetime inside the cavity is flat
rather than Schwarzschild. When $m_{i}\neq 0$, one could perform a coordinate
rescaling of ”t” component (or equivalently set
$\frac{r_{S}-2(m_{i}+m_{S})G_{N}}{r_{S}-2m_{i}G_{N}}$ to 1) to recover the
Schwarzschild line element. However, this would be incorrect while leaving the
line element outside the shell unchanged, as it would lead to an unphysical
discontinuity in the time coordinate and in $A(r)$ across the shell. Now, the
line elements read

$\displaystyle ds^{2}$
$\displaystyle=-\frac{r_{S}-2m_{S}G_{N}}{r_{S}}dt^{2}+dr^{2}+r^{2}d\Omega^{2},$
in the interior and, (2.4) $\displaystyle ds^{2}$
$\displaystyle=-\left(1-\frac{2m_{S}G_{N}}{r}\right)dt^{2}+\frac{dr^{2}}{1-\frac{2m_{S}G_{N}}{r}}+r^{2}d\Omega^{2},$
in the exterior. (2.5)

Next, we outline the equations of motion (EOM) for the shell presented in [29]
with this metric and explore its implications on the existence of a minimal
radius within which the shell is not able to collapse further. The metric
enters the derivation of the EOM through the Christoffel symbols. The relevant
Christoffel symbols are:
$\Gamma_{tt}^{r}=\frac{m_{S}G_{N}}{r^{2}},\Gamma_{rr}^{r}=0$. Following the
prescription of Israel, we can obtain the normal interior and exterior
acceleration. Similarly to the Israel situation, we have the Schwarschild
manifold in the exterior, however in the interior, the time-time component of
the metric has the prefactor $h(t)=\frac{r_{S}-2m_{S}G_{N}}{r_{S}}$. To
consider a variable radius of the shell we denote

$\displaystyle r_{S}=R(s).$ (2.6)

Here, $s$ is proper time measured along the streamlines [30]. If we denote the
$\xi$ as intrinsic coordinates of the smooth hypersurface $\Sigma$ in
Riemannian manifold $V$, and $ds$ an infinitesimal displacement in $\Sigma$,
we can define $u^{\alpha}=\frac{d\xi^{\alpha}}{ds}$ as streamilnes of the dust
particles, following the notation of Israel [30].

We will denote the derivatives with respect to $s$ with a dot. Using the
velocities of the shell and normal vectors from the exterior region
$u^{a}_{+}=\frac{dx^{a}_{+}}{ds}$, $n_{a}^{+}$, and from the interior
$u^{a}_{-}=\frac{dx^{a}_{-}}{ds}$, $n_{a}^{-}$, we can obtain

the exterior: $\displaystyle\boldsymbol{a}_{+}=n_{\alpha}\frac{\delta
u^{\alpha}}{\delta s}|^{+}=n_{a}\frac{du_{+}^{a}}{ds}\boldsymbol{n},$ (2.7)
and the interior: $\displaystyle\boldsymbol{a}_{-}=n_{\alpha}\frac{\delta
u^{\alpha}}{\delta s}|^{-}=n_{a}\frac{du_{-}^{a}}{ds}\boldsymbol{n}$ (2.8)

normal acceleration. This leads to the exterior normal acceleration equivalent
to the standard Israel result.

$a_{+}=\frac{\ddot{R}+\frac{m_{S}G_{N}}{R^{2}}}{(1+\dot{R}^{2}-\frac{2m_{S}G_{N}}{R})^{1/2}},$
(2.9)

while for the interior acceleration we obtain

$\displaystyle
a_{-}=\frac{1}{\sqrt{1-\frac{2m_{S}G_{N}}{R}}}\left(\frac{\ddot{R}}{\sqrt{\dot{R}^{2}+1}}-\frac{m_{S}G_{N}\sqrt{1+\dot{R}^{2}}}{R(2m_{S}G_{N}-R)}\right).$
(2.10)

If we now consider the radial Israel equation [30]

$\displaystyle n_{\alpha}\frac{\delta u^{\alpha}}{\delta
s}|^{+}+n_{\alpha}\frac{\delta u^{\alpha}}{\delta s}|^{-}=0$ (2.11)

we can insert the values for the $\boldsymbol{a}_{-}$ and $\boldsymbol{a}_{+}$
to obtain the EOM for the spherical shell of dust which combines exterior and
interior, $\boldsymbol{a}_{+}+\boldsymbol{a}_{-}=0$. As we commented above,
this configuration can only be stabilized if the stress energy tensor provides
some tangential pressure. In our considerations, the role of stabilization
will be taken by the Casimir force in the radial direction, i.e. the vacuum
fluctuations.

###  2.2 Massless scalar field

We wish to incorporate the effect of quantum vacuum fluctuations on the motion
of the shell. In the low curvature non-relativistic regime this can be
achieved by adding the corresponding Casimir force to the Israel’s radial
equation of motion of a shell. Casimir energy in three dimensions, since it is
odd number of dimensions will be finite, [13].

First, let’s define the Casimir pressure $P$ as $F$ divided by the surface
area $A=4\pi R^{2}$

$P\equiv\frac{F}{A}.$ (2.12)

The Force, in turn, is given from the radial change in the Casimir energy $E$

$\displaystyle P=-\frac{1}{4\pi R^{2}}\frac{\partial E}{\partial R}.$ (2.13)

The Casimir energy $E$ is the sum of Casimir energies $E_{\text{in}}$ and
$E_{\text{out}}$ for the inside and the outside of the shell. We are going to
denote the contribution of Casimir pressure with

$P=\frac{cC\hbar}{8\pi R^{4}},$ (2.14)

where $C$ is a numerical constant. From here on, we are going to set
$\hbar=1=c$. To implement Casimir energy, we assume that:
(i) Backreactions are sub-leading contributions. This means that on the one
hand, the space-time curvature is dominated by the mass of the shell,
recognizing that Casimir energies typically much smaller than the energy
corresponding to the rest mass. Thus, the local Casimir energy does not
contribute to the total energy-momentum tensor appearing in Einstein’s field
equations. On the other hand one also treats the background metric that is
used to calculate the Casimir energy as approximately flat. In any realistic
scenario, both approximations are perfectly reasonable.
(ii) In the regime of the above assumptions, the Casimir force is introduced
as an additional term in the equations of motion, rendering the treatment non-
relativistic.
Under these assumptions, the gravitational collapse of a shell in the presence
of the Casimir force is described by [31]

$\displaystyle M\Bigg{[}\frac{1}{2}\Bigl{(}a_{+}+a_{-}\Bigl{)}\Bigg{]}=(4\pi
R^{2})P,$ (2.15)

where $m_{S}=M$ is the total proper mass of the shell, the term in square
brackets represents the mean acceleration and the right-hand side of the last
equation represents the force. Note also that the last equation is reduced to
the simplest case by neglecting the Casimir force. This approach, for the
scenario when the metric in the interior is $diag(-1,1,1,1)$, can be found in
[27]. Now, combining (2.14), (2.15) and $M$, we can write the EOM as

$\displaystyle\frac{1}{2}\sigma\left(a_{+}+a_{-}\right)=\frac{C}{8\pi R^{4}},$
(2.16)

where $\sigma$ is the mass density per unit area (i.e., $\sigma\equiv M/A$).
Multiplying by $2/\sigma$ and writing the accelerations explicitly one gets

$\displaystyle\frac{m_{S}G_{N}\sqrt{1-\frac{2m_{S}G_{N}}{R}}\sqrt{\dot{R}^{2}+1}}{(R-2m_{S}G_{N})^{2}}+\frac{\ddot{R}}{\sqrt{1-\frac{2m_{S}G_{N}}{R}}\sqrt{\dot{R}^{2}+1}}+\frac{\frac{m_{S}G_{N}}{R^{2}}+\ddot{R}}{\sqrt{-\frac{2m_{S}G_{N}}{R}+\dot{R}^{2}+1}}=\frac{C}{m_{S}R^{2}}.$
(2.17)

The Casimir force ordinarily includes its two-loop radiative corrections.
However, these corrections do not account for significant differences in the
dynamics of the system and will thus be neglected in further analysis.

We wish to investigate the existence of a minimal radius resulting from the
interplay between the inward-pointing gravitational force and the outward-
pointing Casimir force. To do so, we make the simplifying assumption that the
initial velocity of the shell is $\dot{R}=0$, an

$\displaystyle\left[\frac{m_{S}G_{N}}{R^{2}}\left(1+\frac{1}{\Bigl{(}1-\frac{2m_{S}G_{N}}{R}\Bigl{)}}\right)+2\ddot{R}\right]\frac{1}{\sqrt{1-\frac{2m_{S}G_{N}}{R}}}$
$\displaystyle=\frac{C}{m_{S}R^{2}}.$ (2.18)

Further, we are only interested in dynamics well outside the Schwarzschild
radius $R>2m_{S}G_{N}$, where both the Newtonian and non-relativistic
approximations can be trusted, and neglecting backreactions is well justified.
In this limit, we can expand the left-hand side of (2.18) in inverse powers of
$R$. This yields the equation for slow motions in the far region:

$\ddot{R}=\frac{3G_{N}m_{S}(C+2G_{N}m_{S}^{2})-2(C-2G_{N}m_{S}^{2})R}{2m_{S}R^{4}}.$
(2.19)

The “far region” is defined by

$R\ll 2m_{S}G_{N}.$ (2.20)

Now, we seek for a stability condition. One necessary condition for a stable
configuration $\gamma)$ is the existence of a “rest” radius $R_{r}$ which has
vanishing acceleration

$\ddot{R}|_{R=R_{r}}=0.$ (2.21)

For the case of (2.19), this condition is solved by

$\displaystyle R_{r}=\frac{G_{N}m_{S}(C+2G_{N}m_{S}^{2})}{C-2G_{N}m_{S}^{2}}.$
(2.22)

The second condition for stability, is that the dynamics is attractive towards
the rest radius, in some finite vicinity of $R_{r}$. This condition implies

$\left.\frac{d(\ddot{R})}{dR}\right|_{R_{r}}<0.$ (2.23)

Using (2.22) to evaluate the left hand side of (2.23) we find

$\left.\frac{d(\ddot{R})}{dR}\right|_{R_{r}}=\frac{(C-2G_{N}m_{S}^{2})^{4}}{2G_{N}^{3}m_{S}^{4}(C+2G_{N}m_{S}^{2})^{3}}>0.$
(2.24)

We remember that $C>0$. Thus, (2.24) is in contradiction with (2.23), which
means that a stable balance between the Casimir force and the gravitational
dynamics is not possible in the Newtonian regime of a massless scalar particle
(2.17). A numerical analysis shows that the same is true if one explores slow
motion for all radii $R>2m_{S}G_{N}$. This can nicely be exemplified with
figure 1.

Figure 1: $\ddot{R}$ as a function of $R$ for (2.17). The parameters are
chosen to be $G_{N}=m_{S}=1$. The blue, orange, green curves are for values of
$C={10,5,1}$.

Since in the true Casimir effect $C<0.1$, all scenarios where the shell is
heavier than the Planck mass are represented by the green curve. Light shells
instead $m_{S}^{2}<C/(2G_{N})$, would be represented by the blue or orange
cuver in figure 1. In any case, the stability condition (2.23) is never
fulfilled in the massless case. This means that in this case it is not
possible to realize scenario $\gamma)$ (only $\alpha)$ or $\beta)$).

###  2.3 Massive scalar field

In the previous section, we explored the influence of the Casimir force of a
massless scalar field on the dynamics of the gravitational collapse of a
shell. The analysis shows that there exists no stable minimal radius. We now
expand the EOM to include the Casimir force for a massive scalar field. The
force is pointed outward and has magnitude [23, 32]

$F_{\text{massive}}=\sum_{j=1}^{\infty}\frac{(-1)^{j+1}\left(\boldsymbol{H}_{2}(2\pi
jm_{\phi}R)+\frac{4jm_{\phi}R}{3}-I_{2}(2\pi jm_{\phi}R)\right)}{j}$ (2.25)

where $m_{\phi}$ is the mass of the scalar field. The outcome of the
regularization of the sum (2.25) can be evaluated numerically. This result is
well fitted by the phenomenological function of the dimensionless product of
radius $R$ and energy as a function of the dimensionless product of mass
$m_{\phi}$ times radius $R$

$RE_{m}=\frac{a+b\,(Rm_{\phi})}{1+c\,(Rm_{\phi})+d(Rm_{\phi})^{3}}.$ (2.26)

Where the resulting dimensionless fitting parameters are
$a=0.0029,\,b=0.0674,\,c=16.28,\,d=57.55$. This fit is shown in figure 2.

Figure 2:  Casimir force in a spherical shell produced by a massive scalar
field. The points are the numerical results given in [12]. The continuous
curve is the fit given in equation (2.26).

Equipped with this relation, we can proceed to calculate the Casimir force via

$F_{m_{\phi}}=-\frac{dE_{m}}{dR}.$ (2.27)

This force is then divided by the mass of the shell $m_{S}$ and inserted on
the RHS of the relations (2.17 and 2.18). This gives the equation of motion
for the shell with non-relativistic velocities

$\displaystyle\ddot{R}$ $\displaystyle=$
$\displaystyle\frac{a+m_{\phi}R(2ac+m_{\phi}R(bc+dm_{\phi}R(4a+3bm_{\phi}R)))}{2m_{S}R^{2}(1+cm_{\phi}R+dm_{\phi}^{3}R^{3})^{3/2}}\sqrt{1-\frac{2m_{S}G_{N}}{R}}$
$\displaystyle-\frac{G_{N}m_{S}}{2R^{2}}-\frac{G_{N}m_{S}}{2R^{2}(1-\frac{2G_{N}m_{S}}{R})}.$

The radii with vanishing acceleration are found by imposing (2.21). This can
be solved analytically in the far region, yielding

$r_{S}=\frac{\sqrt{6bdG_{N}m_{S}^{2}m_{\phi}^{2}-7d^{2}G_{N}^{4}m_{S}^{6}m_{\phi}^{4}}-dG_{N}^{2}m_{S}^{3}m_{\phi}^{3}}{2dG_{N}m_{S}^{2}m_{\phi}^{2}}.$
(2.29)

For consistency, we have to demand that this radius is both real and much
larger than the Schwarzschild radius of the mass $m_{S}$. Interestingly, this
implies an upper limit for the mass of the shell as a function of $m_{\phi}$

$m_{S}^{4}<\frac{3b}{16dG_{N}^{3}m_{\phi}^{2}}.$ (2.30)

Next we have to check whether the point of vanishing acceleration is actually
stable. This condition can be implemented in terms of the relation (2.23). In
this region we find

$\left.\frac{d\ddot{R}}{dR}\right|_{r_{S}}=16d^{4}G_{N}^{5}m_{S}^{9}m_{\phi}^{8}\frac{G_{N}m_{S}\sqrt{dG_{N}m_{S}^{2}m_{\phi}^{2}(6b-7dG_{N}^{3}m_{S}^{4}m_{\phi}^{2})}+7dG_{N}^{3}m_{S}^{4}m_{\phi}^{2}-6b}{(\sqrt{dG_{N}m_{S}^{2}m_{\phi}^{2}(6b-7dG_{N}^{3}m_{S}^{4}m_{\phi}^{2})}-dG_{N}^{2}m_{S}^{3}m_{\phi}^{2})^{5}}.$
(2.31)

The right hand side of this relation is negative if

$m_{S}^{4}<\frac{6b}{7dG_{N}^{3}m_{\phi}^{2}},$ (2.32)

which is already assured by imposing (2.30). Thus, from the analytic
approximations (2.29-2.31) we conclude that the Casimir force for massive
fields could produce a meta-stable configuration (in the sense of dependence
on the initial conditions) in the interplay with the gravitational attraction,
if the condition (2.30) is satisfied.

Next, we proceed to check this analytical intuition numerically by plotting
(2.3) as a function of $R$. The result is shown in Figure 3.

Figure 3:  Acceleration (2.3) as a function of $R$ in Planck units $G_{N}=1$
and fixing $m_{S}^{2}m_{\phi}=0.007$. The blue curve is for $m_{S}=0.01$. The
orange curve is for $m_{S}=0.015$. The green curve is for $m_{S}=0.25$. The
zeros of these curves are the solutions of $\ddot{R}=0$. Only the blue curve
has a meta-stable configuration.

To get an idea of which type of shells could be stabilized, let’s assume that
the stabilizing scalar field is the Higgs field with $m_{\phi}=125~{}$GeV. The
Casimir force for this field could stabilize, under the right initial
conditions, a spherical shell with a mass smaller than $m_{S}<10^{25}~{}$GeV
$\approx 0.2~{}$kg. However, the stable radius for this configuration would be
eight orders of magnitude smaller than the proton radius. This is extremely
small, even though it is still three orders of magnitude larger than the
Schwarzschild radius of this configuration.

This result can be seen from two perspectives. On the one hand, it is
encouraging because it could be interpreted as a hint of a mechanism
stabilizing the gravity-matter interplay at very high densities. On the other
hand, it applies classical mechanics at distance scales that are deeply
quantum. Thus, even though we found a scenario that realizes the stable
scenario $\gamma)$, it has to be taken with a big grain of salt.

Note that another candidate for scalar fields could be cosmological fields
such as the inflaton. These could provide very different masses and radii.
However, since we do not know how to implement boundary conditions for this
field, we do not explore this possibility further at this point.

###  2.4 Temperature dependence

In this section, we consider if there is a balance of the thin spherical shell
and the repulsive Casimir force due to the fluctuation of massless scalar
fields in the sphere at non-vanishing temperature. To model this behavior, we
are going to use the acceleration from the interior and the exterior of the
shell as above, while on the right-hand side, we implement the contribution
from the Casimir force at finite temperature. The Casimir energy of the thin
spherical shell in the limit of low and high temperature has been studied, and
a review can be found in [13].

####  2.4.1 High temperature for scalar field

In the high temperature limit, the EOM for the shell is given by

$\displaystyle\frac{\ddot{R}+\frac{m_{S}G_{N}}{R^{2}}}{(1+\dot{R}^{2}-\frac{2m_{S}G_{N}}{R})^{1/2}}+\frac{1}{\sqrt{1-\frac{2m_{S}G_{N}}{R}}}\left(\frac{\ddot{R}}{\sqrt{\dot{R}^{2}+1}}-\frac{m_{S}G_{N}\sqrt{1+\dot{R}^{2}}}{R(2m_{S}G_{N}-R)}\right)=-\frac{1}{m_{S}}\frac{T}{24R}$
(2.33)

Like before, we are interested in the slow motion regime, for which we impose
$\dot{R}=0$. Furthermore, we remember that the entire classical approach
without backreaction is only reliable in the Newtonian regime $R\gg
m_{S}G_{N}$. In this expansion, we get from (2.33)

$\ddot{R}=-\frac{1}{m_{S}}\frac{T}{24R}-G_{N}\frac{m_{S}}{R^{2}}.$ (2.34)

From this expansion, it becomes clear that in the high-temperature limit, both
the gravitational and the Casimir interactions contribute to an inwards-
directed pull. Thus, in this regime, no stable configuration is possible;
instead, only scenario $\alpha$) is realized.

####  2.4.2 Low temperature for scalar field

The Casimir energy for a scalar field in the low temperature is [13],

$\Delta_{T}\mathcal{F}^{D}=\frac{\pi^{3}R}{6}(k_{B}T)^{2}-\zeta_{R}(3)R^{2}(k_{B}T)^{3}.$

Here, the index $D$ denotes Dirichlet boundary conditions imposed on the
sphere. The Boltzman constant is denoted with $k_{B}$ and $\zeta_{R}(3)$ is
the Riemman zeta function. Combining the energy at zero temperature and the
thermal correction, we obtain the total contribution from Casimir energy at
low temperature. That gives the corresponding radial force acting on the
spherical shell, which we insert on the right-hand side of the equation. This
leads to

$\displaystyle\frac{\ddot{R}+\frac{m_{S}G_{N}}{R^{2}}}{(1+\dot{R}^{2}-\frac{2m_{S}G_{N}}{R})^{1/2}}+$
$\displaystyle\frac{1}{\sqrt{1-\frac{2m_{S}G_{N}}{R}}}\left(\frac{\ddot{R}}{\sqrt{\dot{R}^{2}+1}}-\frac{m_{S}G_{N}\sqrt{1+\dot{R}^{2}}}{R(2m_{S}G_{N}-R)}\right)=$
$\displaystyle=\frac{C}{m_{S}R^{2}}-\frac{\frac{1}{6}\pi^{3}T^{2}-2RT^{3}\zeta(3)}{m_{S}}$
(2.35)

In the limit $T\rightarrow 0$, the EOM reduces to the case of a temperature
independent massless scalar field.

As in the previous cases we investigate the conditions for which
$\ddot{R}=\dot{R}=0$. The stable $R>0$ solutions for which this conditions are
satisfied would be the sought after minimal radii $R_{\text{min}}$. Again, we
first impose $\dot{R}=0$ and solve for $\ddot{R}$

$\displaystyle\ddot{R}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sqrt{1-\frac{2m_{S}G_{N}}{R}}\left[\frac{\frac{C}{R^{2}}-\left(\frac{1}{6}\pi^{3}T^{2}-2RT^{3}\zeta(3)\right)}{m_{S}}-\frac{m_{S}G_{N}}{R\sqrt{1-\frac{2m_{S}G_{N}}{R}}}\left(\frac{1}{R}-\frac{1}{2m_{S}G_{N}-R}\right)\right]$
(2.36) $\displaystyle\approx$
$\displaystyle\frac{T^{2}}{m_{S}G_{N}R^{2}}\left[\frac{C-G_{N}m_{S}^{2}}{T^{2}}-\frac{\pi^{3}R^{2}}{12}+\frac{R^{3}T\zeta(3)}{1}\right]\,.$

Before solving the right hand side for $\ddot{R}=0$, let’s discuss the
apparent properties of these three roots. Since the leading term in the
bracket $\sim R^{3}$ is positive, the largest positive root will always be
unstable. If the $\sim cte.$ term is also positive, there can be two positive
roots, from which the smaller one would be stable. If the $\sim cte.$ term is
negative, there could be one or three real roots. In the former case, the root
is unstable. The latter case does not happen because if at some radius the
$\sim R^{2}$ term dominates, it does so for positive and negative $R$, in
particular since there is no $\sim R$ term. The analytic expressions for the
roots of the above equation (2.36) we present in (B Analytic expressions for
temerperature dependence), in the Appendix. The number of real roots can be
shown to only depend on the parameter $m_{S}$. In Figure 4 we plot ($T\cdot$(B
Analytic expressions for temerperature dependence)) as a function of the
dimensionless ratio $m_{z}/\sqrt{G_{N}}$.

Figure 4:  Value of the roots $TR_{i}$ as a function of $m_{S}/m_{p}$, where
$m_{p}=1/\sqrt{G_{N}}$. The black line is the solution $R_{1}$, the blue line
is the stable solution $R_{3}$. The solution $R_{2}$ is only plotted as dotted
line, because for $m_{S}^{2}<G_{N}$ it is negative and for $m_{S}^{2}>G_{N}$
it is imaginary. The green lines are $1/(m_{S}200),\,1/(m_{S}20),1/(m_{S}2)$.
They represent the underlying criterium of large radius expansion, namely that
we only should trust scenarios below the green line which means temperatures
below the Planck energy.

Figure 4 shows nicely that a stable solution is only possible for light
shells, as long as $C>G_{N}m_{S}^{2}$. To illustrate this effect, we plot
$\ddot{R}$ as a function of $R$, for different temperatures. In the right
panel, we plotted the case where condition $C>G_{N}m_{S}^{2}$ does hold,
whereas in the left panel we plot the case where this condition does not hold.

Figure 5:  First panel: Radial acceleration $\ddot{R}$ versus $R$, assuming
four (small) different temperature values. Second panel: Radial acceleration
$\ddot{R}$ versus $R$ at four different (moderate) temperature values. We also
plotted the zero-temperature case for comparison. For illustrative purposes,
we have used the mass $m_{S}=0.1$, $G_{N}=1$, and $C=1$ and further
$k_{B}=\hbar=1$.

Looking at the first panel of Fig.(5), we see the effect of including
temperature corrections on the evolution of the acceleration. For the large
$m_{S}$, the behavior is similar to the zero temperature case, and the main
difference appears for a small shift on the peak of the acceleration. From the
second panel of Fig.(5) we confirm that a stable configuration can be found
for small $m_{S}$, even when no large radius expansion is imposed. The
stability can be seen from the fact that the radial acceleration is negative
for $R>R_{\text{min}}$ and positive for $R<R_{\text{min}}$. Therefore, the
interplay between the temperature dependent Casimir force of a massless scalar
field with the gravitational attraction caused by the massive shell, can
stabilize the system, which corresponds to scenario $\gamma)$.

####  2.4.3 Electromagnetic Casimir at low temperature and large $R\gg r_{S}$

Since the scalar Casimir effect at low temperatures yielded the desirable
scenario $\gamma$), it is reasonable to hope for a similar result when
electromagnetic interactions are considered. The low-temperature corrections
to the electromagnetic Casimir effect are, however, simply $\sim T^{4}$ [13].
Thus, the motion of the shell is described by

$\ddot{R}=\frac{C}{m_{S}R^{2}}-G_{N}\frac{m_{S}}{R^{2}}+\frac{1}{5}\pi^{3}R^{2}(k_{B}T)^{4}.$
(2.37)

The radius with zero acceleration is

$R_{0}^{2}=\frac{\sqrt{5}\sqrt{G_{N}m_{S}^{2}-{\color[rgb]{.75,0,.25}C}}}{k_{B}^{2}\sqrt{m_{S}}\pi^{3/2}T^{2}}.$
(2.38)

The stability criterium at this radius is

$\left.\frac{d\ddot{R}}{dR}\right|_{R_{0}}=4k_{B}^{3}\frac{(G_{N}m_{S}-C/m_{S})^{1/4}\pi^{9/4}T^{3}}{5^{3/4}}>0.$
(2.39)

Thus, if $m_{S}^{2}<C/G_{N}$, the configuration always explodes, which
corresponds to scenario $\beta)$. Instead, if $m_{S}^{2}>C/G_{N}$, the
configuration is unstable in the sense that it depends on the initial
condition, whether the outcome is the scenario $\alpha)$ or $\beta)$. A stable
scenario $\gamma)$ is not possible here.

##  3 Conclusion

In the present work, we have investigated the dynamics of the gravitational
collapse of a shell in the presence of the Casimir force in four dimensions
considering different types of vacuum fluctuations:

  * i)

massless scalar fields,

  * ii)

massive scalar fields,

  * iii)

massless scalar fields at high temperature limit,

  * iv)

massless scalar fields at low temperature limit,

  * v)

electromagnetic fields at low temperature limit.

Specifically, we have investigated, among other things, the existence of a
minimal radius as a result of the interplay between the inward-pointing
gravitational force and the outward-pointing Casimir force. We defined the
minimal radius as the point at which the acceleration and velocity of the
collapsing shell can reach $\dot{R}=0=\ddot{R}$. To evaluate the stability of
the minimal radius $R_{\text{min}}$, we explored the derivatives of the
acceleration $\ddot{R}$ with respect to the radius $R$ at this point of
interest. Finally, we have illustrated our results with some figures.

In two out of the five scenarios that we analyzed we obtain the stable region,
while in the other three scenarios no stability region is found: (i) when the
Casimir force is produced by the massless scalar particles at the zero
temperature, there is no radius at which the thin spherical shell would
oscillate around a minimum. (ii) When the Casimir force is caused by massive
scalar particles, the stable region is obtained when the mass shell is lighter
than the fluctuating massive particle. (iii) For the thin spherical shell of
the mass $m_{S}$ with the massless scalars at the low non-zero temperature, we
obtain a stable region. The region satisfies the conditions that the radius of
the shell is bigger than the two masses of the shell, $R>2m_{S}$, and that
low-temperature behavior for the thermal corrections can be used $k_{B}TR\ll
1$ [13]. (iv) In the limit when the Casimir energy is expanded around a high-
temperature limit, we do not obtain a stable region. The same negative result
is true for the electromagnetic Casimir effect in the low temperature limit
(v).

Note that due to the large theoretical uncertainties on the interplay between
quantum effects and general relativity we chose to study this simple model in
the regime where we can expect that all assumptions still hold, namely in the
regime of weak gravity and non-relativistic motion. In this regime, we do not
find realistic scenarios on our model, where the Casimir force could stabilize
a shell against the gravitational pull.

##  4 Acknowledgements

The work of IL was supported by the Hertha Firnberg grant T1269-N and Elise
Richter grant V 1052-N by the Austrian Science Fund, FWF. A. R. acknowledges
financial support from the Generalitat Valenciana through PROMETEO PROJECT
CIPROM/2022/13. A. R. is funded by the María Zambrano contract ZAMBRANO 21-25
(Spain) (with funding from NextGenerationEU).

## Appendix

### A Remark on Birkhoff’s theorem

According to Birkhoff’s theorem [33], in Einstein’s theory of relativity, when
we have a symmetric mass distribution that is not rotating in empty space we
can describe the gravitational field outside this distribution using the
Schwarzschild metric. In essence, this metric enables us to simplify the
understanding and calculation of effects in several situations and has
important implications. Birkhoff’s theorem can be seen as a characterization
of the structure of vacuum spacetimes that exhibit local spherical symmetry.
Its significance lies in the fact that it serves not only as a classical
cornerstone of general relativity but also as a vital tool in the fields of
gravitational physics and cosmology. In particular, Birkhoff’s theorem is
conventionally employed for the following purposes: i) to establish that the
empty spacetime within a spherically symmetric mass shell corresponds to
Minkowski spacetime, ii) to demonstrate that spherical gravitational collapse
does not emit gravitational waves, iii) to finalize the proof of the classical
uniqueness theorem for static black holes [34]. Given its significance, it
becomes evident that Birkhoff’s theorem plays a crucial role in the context of
spherically symmetric spacetimes within the framework of general relativity.

There is some discussion in the literature concerning the applicability an
validity of Birkhoffs’ theorem. In particular, Israel [29] and Brevik [27]
used the Birkhoff theorem to derive the corresponding equations of motion
(EOMs) describing the gravitational collapse of a thin shell. However, Zhang
and Yi [35] pointed out that there is a misunderstanding in the interpretation
of the Birkhoff theorem and that the original result of [29] is questionable.
They argue that the correct metric, which is continuous with the location of
an external observer, is determined by both the enclosed mass and the mass
distribution outside. The statements in the work by Zhang and Yi were
elaborated in [28] where it was demonstrated that the Birkhoff’s theorem holds
using a traditional metric-based approach. They also show that it holds at the
level of tetrad components. What they consider does not depend on the
distribution of matter in the interior or exterior regions, or state of motion
of the system, under the assumption that spherical symmetry holds. It would be
unphysical though to rescale the interior vacuum region and keep the line-
element outside of the spherical shell unchanged because it would led to
discontinuity in the time coordinate and corresponding term in the metric
element. An additional example of the gravitational field at a certain radius
in a spherically-symmetric matter distribution which is outside of the radius
has also been analyzed in [36].

Even though, this discussion on the different versions of Birkhoff’s theorem
is interesting from a theoretical perspective, it is irrelevant for our
purposes. The reason for this irrelevance is that in the non-relativistic low
curvature limit both versions of the metrics give same result for the
stability of the shell in our construction. We decided to follow the
formulation used by ([29, 27]), but in our case, ([28]) would have given the
same results.

### B Analytic expressions for temerperature dependence

The analytic expressions for the roots of the EOM (2.36) are

$\displaystyle R_{1}$ $\displaystyle=$
$\displaystyle\frac{1}{36T\zeta(3)}\left(\pi^{3}+\frac{\pi^{6}}{(\pi^{9}+216\zeta(3)(108(G_{N}-m_{S}^{2})\zeta(3)+\sqrt{(C-G_{N}m_{S}^{2})(11664(C-G_{N}m_{S}^{2})\zeta(3)-\pi^{9})}))^{1/3}}+\right.$
$\displaystyle\left.(\pi^{9}+216\zeta(3)(108(G_{N}-m_{S}^{2})\zeta(3)+\sqrt{(C-G_{N}m_{S}^{2})(11664(C-G_{N}m_{S}^{2})\zeta(3)-\pi^{9})}))^{1/3}\right),$
$\displaystyle R_{2,3}$ $\displaystyle=$
$\displaystyle\frac{1}{72T\zeta(3)}\left(2\pi^{3}+\frac{\mp\pi^{6}(1+i\sqrt{3})}{(\pi^{9}+216\zeta(3)(108(G_{N}-m_{S}^{2})\zeta(3)+\sqrt{(C-G_{N}m_{S}^{2})(11664(C-G_{N}m_{S}^{2})\zeta(3)-\pi^{9})}))^{1/3}}+\right.$
$\displaystyle\left.\pm(1+i\sqrt{3})(\pi^{9}+216\zeta(3)(108(G_{N}-m_{S}^{2})\zeta(3)+\sqrt{(C-G_{N}m_{S}^{2})(11664(C-G_{N}m_{S}^{2})\zeta(3)-\pi^{9})}))^{1/3}\right).$

Interestingly, the temperature factors out of the solutions (B Analytic
expressions for temerperature dependence), thus the number of real roots only
depends on the values of $(C,\,G_{N},\,$ and $m_{S})$. Further, $C$ has a
definite value $C=0.0461766$ [13]. Thus, we can multiply both sides of (B
Analytic expressions for temerperature dependence) with $T$ to make it a
dimensionless identity. Now we can express all dimensionful quantities in
Planck units, which leaves $m_{S}$ as the only parameter which determines the
number of real roots.

## References

  * [1] Richard Tolman and Paul Ehrenfest.  Temperature Equilibrium in a Static Gravitational Field.  Phys. Rev., 36(12):1791–1798, 1930.
  * [2] Jacob D. Bekenstein.  Hydrostatic Equilibrium and Gravitational Collapse of Relativistic Charged Fluid Balls.  Phys. Rev. D, 4:2185–2190, 1971.
  * [3] Marek Nowakowski, Juan-Carlos Sanabria, and Alejandro Garcia.  Gravitational equilibrium in the presence of a positive cosmological constant.  Phys. Rev. D, 66:023003, 2002.
  * [4] Pankaj S. Joshi, Daniele Malafarina, and Ramesh Narayan.  Equilibrium configurations from gravitational collapse.  Class. Quant. Grav., 28:235018, 2011.
  * [5] Riccardo Belvedere, Daniela Pugliese, Jorge A. Rueda, Remo Ruffini, and She-Sheng Xue.  Neutron star equilibrium configurations within a fully relativistic theory with strong, weak, electromagnetic, and gravitational interactions.  Nucl. Phys. A, 883:1–24, 2012.
  * [6] Juan M. Z. Pretel.  Equilibrium, radial stability and non-adiabatic gravitational collapse of anisotropic neutron stars.  Eur. Phys. J. C, 80(8):726, 2020.
  * [7] Charles W. Misner, K. S. Thorne, and J. A. Wheeler.  Gravitation.  W. H. Freeman, San Francisco, 1973.
  * [8] S. K. Lamoreaux.  The Casimir force: Background, experiments, and applications.  Rept. Prog. Phys., 68:201–236, 2005.
  * [9] Steven Weinberg.  The Cosmological Constant Problem.  Rev. Mod. Phys., 61:1–23, 1989.
  * [10] Ignacio A. Reyes and Giovanni Maria Tomaselli.  Quantum field theory on compact stars near the Buchdahl limit.  Phys. Rev. D, 108(6):065006, 2023.
  * [11] Ignacio A. Reyes.  Trace anomaly and compact stars.  8 2023.
  * [12] Michael Bordag, U. Mohideen, and V. M. Mostepanenko.  New developments in the Casimir effect.  Phys. Rept., 353:1–205, 2001.
  * [13] M. Bordag, G. L. Klimchitskaya, U. Mohideen, and V. M. Mostepanenko.  Advances in the Casimir effect, volume 145.  Oxford University Press, 2009.
  * [14] A. Chodos, R. L. Jaffe, K. Johnson, Charles B. Thorn, and V. F. Weisskopf.  A New Extended Model of Hadrons.  Phys. Rev. D, 9:3471–3495, 1974.
  * [15] A. Chodos, R. L. Jaffe, K. Johnson, and Charles B. Thorn.  Baryon Structure in the Bag Theory.  Phys. Rev. D, 10:2599, 1974.
  * [16] Ted Barnes, F. E. Close, and F. de Viron.  Q anti-Q G Hermaphrodite Mesons in the MIT Bag Model.  Nucl. Phys. B, 224:241, 1983.
  * [17] Eduardo S. Fraga and Leticia F. Palhares.  Deconfinement in the presence of a strong magnetic background: an exercise within the MIT bag model.  Phys. Rev. D, 86:016008, 2012.
  * [18] Timothy H. Boyer.  Quantum electromagnetic zero point energy of a conducting spherical shell and the Casimir model for a charged particle.  Phys. Rev., 174:1764–1774, 1968.
  * [19] Roger Balian and Bertrand Duplantier.  Electromagnetic Waves Near Perfect Conductors. 2. Casimir Effect.  Annals Phys., 112:165, 1978.
  * [20] Kimball A. Milton, Lester L. DeRaad, Jr., and Julian S. Schwinger.  Casimir Selfstress on a Perfectly Conducting Spherical Shell.  Annals Phys., 115:388, 1978.
  * [21] Kimball A. Milton.  Zero Point Energy in Bag Models.  Phys. Rev. D, 22:1441, 1980.
  * [22] Kimball A. Milton.  FERMIONIC CASIMIR STRESS ON A SPHERICAL BAG.  Annals Phys., 150:432, 1983.
  * [23] Michael Bordag, E. Elizalde, K. Kirsten, and S. Leseduarte.  Casimir energies for massive fields in the bag.  Phys. Rev. D, 56:4896–4904, 1997.
  * [24] E. Elizalde, Michael Bordag, and K. Kirsten.  Casimir energy for a massive fermionic quantum field with a spherical boundary.  J. Phys. A, 31:1743–1759, 1998.
  * [25] Hamid Haghmoradi, Hauke Fischer, Alessandro Bertolini, Ivica Galić, Francesco Intravaia, Mario Pitschmann, Raphael Schimpl, and René I. P. Sedmik.  Force metrology with plane parallel plates: Final design review and outlook.  MDPI Physics, 6(2), 2024.
  * [26] Francesco Alessio, Glenn Barnich, and Martin Bonte.  Gravitons in a Casimir box.  JHEP, 02:216, 2021.  [Erratum: JHEP 03, 228 (2021)].
  * [27] Iver Brevik and Kolbenstvedt Høye.  Casimir effect and gravitational collapse.  Il Nuovo Cimento B (1971-1996), 82:71?84, 1984.
  * [28] Do Young Kim, Anthony N. Lasenby, and Michael P. Hobson.  Spherically-symmetric solutions in general relativity using a tetrad-based approach.  Gen. Rel. Grav., 50(3):29, 2018.
  * [29] W. Israel.  Gravitational collapse and causality.  Phys. Rev., 153:1388–1393, Jan 1967.
  * [30] W. Israel.  Singular hypersurfaces and thin shells in general relativity.  Nuovo Cim. B, 44S10:1, 1966.  [Erratum: Nuovo Cim.B 48, 463 (1967)].
  * [31] D. Vick.  Gravitational collapse of a shell subject to the Casimir force.  Nuovo Cimento B Serie, 94B(1):54–62, January 1986.
  * [32] M.A. Valuyan.  Casimir energy calculation for massive scalar field on spherical surfaces: an alternative approach.  Canadian Journal of Physics, 96(9):1004–1009, 2018.
  * [33] George David Birkhoff and Rudolph Ernest Langer.  Relativity and modern physics.  1923\.
  * [34] Werner Israel.  Event horizons in static vacuum space-times.  Phys. Rev., 164:1776–1779, 1967.
  * [35] SHUANG-NAN ZHANG and SHUXU YI.  On a common misunderstanding of the birkhoff theorem and light deflection calculation: Generalized shapiro delay and its possible laboratory test.  International Journal of Modern Physics: Conference Series, 12:419–430, 2012.
  * [36] Roshina Nandra, Anthony Lasenby, and Michael Hobson.  Dynamics of a spherical object of uniform density in an expanding universe.  Phys. Rev. D, 88(4):044041, 2013.


# Bi-level Alignment for Cross-Domain Crowd Counting

Shenjian Gong1, Shanshan Zhang1,*, Jian Yang1, Dengxin Dai2, and Bernt
Schiele2
1PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional
Information
of Ministry of Education, and Jiangsu Key Lab of Image and Video Understanding
for Social Security,
School of Computer Science and Engineering, Nanjing University of Science and
Technology
2MPI Informatics
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Recently, crowd density estimation has received increasing attention. The main
challenge for this task is to achieve high-quality manual annotations on a
large amount of training data. To avoid reliance on such annotations, previous
works apply unsupervised domain adaptation (UDA) techniques by transferring
knowledge learned from easily accessible synthetic data to real-world
datasets. However, current state-of-the-art methods either rely on external
data for training an auxiliary task or apply an expensive coarse-to-fine
estimation. In this work, we aim to develop a new adversarial learning based
method, which is simple and efficient to apply. To reduce the domain gap
between the synthetic and real data, we design a bi-level alignment framework
(BLA) consisting of (1) task-driven data alignment and (2) fine-grained
feature alignment. In contrast to previous domain augmentation methods, we
introduce AutoML to search for an optimal transform on source, which well
serves for the downstream task. On the other hand, we do fine-grained
alignment for foreground and background separately to alleviate the alignment
difficulty. We evaluate our approach on five real-world crowd counting
benchmarks, where we outperform existing approaches by a large margin. Also,
our approach is simple, easy to implement and efficient to apply. The code is
publicly available at https://github.com/Yankeegsj/BLA.

††*Corresponding author

(a) Style Transfer

(b) Domain Randomization and Task-Driven Data Alignment

Figure 1: Comparison of three different ways for source domain augmentation.
(a). Style transfer translates images from source to a target-like domain
based on target style priors, but the translation is usually limited to color
changes, and blind to the task objective. (b). Domain randomization augments
the source domain randomly in a more diverse manner (colors, scales, etc.) but
without any priors from target; our proposed task-driven data alignment is
more similar to domain randomization; but instead of random selection, we pick
the most suitable augmentation based on the task objective, which enables a
more dynamic and robust model to the target domain.

##  1 Introduction

Crowd counting aims to estimate the number of persons in crowded scenes. This
task has gained a lot of attention [11, 19] as it is useful for video
surveillance, traffic control and human behavior analysis. Especially under
pandemics such as COVID-19, it can be used to monitor and regulate the flow of
people for safety reasons.

In recent years, many methods have been proposed for crowd counting. Most
state-of-the-art approaches rely on ground truth density maps for a large
number of training images, with each human head marked. However, it is
extremely expensive to annotate so many human heads and for high-density
regions such manual labels can be noisy. To reduce annotations costs, a large-
scale synthetic dataset GTA5 Crowd Counting (GCC) [26] was created, serving as
a well-established training set with automatic annotations. However, models
trained on the synthetic dataset perform poorly due to the large domain gap
between synthetic and real-world images. Thus, it is necessary to investigate
how to adapt the models trained on the synthetic domain to the real domain,
without requiring annotations on the latter, i.e. via unsupervised domain
adaptation (UDA).

There are a few UDA methods proposed for crowd counting. For instance, SE
Cycle-GAN [26] translates synthetic images to the real domain with improved
Cycle-GAN and then trains purely on the translated images; Gaussian Process-
based iterative learning (GP) [24] generates pseudo-labels on the target
images via a Gaussian process to allow for supervised training on the target
domain. More recently, better performance has been achieved by employing an
adversarial framework to align features from both source and target domains
[8, 6]. However, FSC [8] introduces an auxiliary task of semantic
segmentation, relying on external labeled human body segmentation datasets for
pre-training; FADA [6] performs a coarse-to-fine estimation, making the
inference less efficient.

In this paper, we aim to develop a new adversarial learning based method,
which is more effective and flexible. We investigate the key components to
boost performance. Previous methods employed either domain randomization or
style transfer for source domain augmentation, using no priors or the target
style priors only. In contrast, our task-driven data alignment is able to
control the domain augmentation based on both the target style priors and the
task objective such that it is optimized for our crowd counting task on the
given target domain. We show a comparison of three different source domain
augmentation methods in Fig. 1(b). On the other hand, since the foreground and
background regions differ significantly in semantics, we propose a fine-
grained feature alignment to handle them separately.

To summarize, the contributions of our work are as follows: (1) For more
effective and efficient synthetic to real adaptive crowd counting, we propose
a novel adversarial learning based method, consisting of bi-level alignments:
task-driven data alignment and fine-grained feature alignment. (2) To the best
of our knowledge, it is the first UDA approach to search for the optimal
source data transform based on the downstream task performance on the target
domain. (3) Experimental results on various real datasets show that our method
achieves state-of-the-art results for synthetic-to-real domain adaptation;
also, our method is simple and efficient to apply.

##  2 Related Works

Since we solve the problem of domain adaptive crowd counting, we first review
recent works; our major contribution is a novel domain augmentation method via
AutoML, so we also discuss related works in the above two areas.

###  2.1 Domain Adaptive Crowd Counting

There are two groups of domain adaptation works in crowd counting: real-to-
real and synthetic-to-real. Real-to-real adaptation aims to generalize models
across real scenarios [9, 16], but since one real-world dataset is taken as
the source domain, manual annotations are still needed. In contrast,
synthetic-to-real adaptation fully avoids the requirement for manual
annotations and thus is more interesting. In this work, we focus on synthetic-
to-real adaptation. One direct way is to translate the labeled synthetic
images to the style of the real images and then train on the translated images
[26, 4], but it is limited by the performance of the translation method.
Another intuitive way is to generate pseudo-labels on the target real images
to allow for supervised learning on the target domain [24, 4]. More recently,
the adversarial framework has been leveraged to achieve better performance via
feature alignment between source and target domains [8, 6]. However, previous
works are not efficient, requiring external training data or additional
inference time. For instance, FSC [8] introduces an auxiliary task of semantic
segmentation, relying on external labeled human body segmentation datasets for
pre-training; FADA [6] performs a coarse-to-fine estimation, making the
inference less efficient. In this work, we leverage adversarial training but
aim to develop a simple yet effective method.

###  2.2 Domain Augmentation

Previously, there are two ways to augment the existing source domain: one is
domain randomization, randomly changing the style of source images; the other
is style transfer, translating the source images to the target style. Both of
them preserve the contents of source images to allow for supervised learning.

Domain randomization is mostly used for domain generalization, which handles
unknown target domains. The generated new samples with random styles are
helpful to enhance the generalization ability of the trained model. One direct
way is to generate various styles of images by style transfer CNN [31] or data
augmentation [25]. On the other hand, AdaIN [13] has demonstrated that the
mean and variance of convolutional feature maps can be used to represent the
image style, making the domain randomization easier [21]. Some recent works
modify styles in the frequency domain [28, 12] by randomizing the domain-
variant components, which represent styles.

In contrast, style transfer is used for domain adaptation, where the target
domain images are given to provide style guidance. The most common way is to
generate source-to-target images through Cycle-GAN [10, 26, 4]. Similar to
domain randomization, some works also do it in the frequency domain by
replacing the style-related components with target ones, e.g. the amplitude of
Fourier transform [30].

In this work, we propose a novel domain augmentation method named task-driven
data alignment which is superior than domain randomization and style transfer.
The major difference is that the augmentation is controlled by both the target
style priors and by verifying the counting performance on a target-like
domain. In this way, we are able to optimize augmentation for our crowd
counting task on the given target domain.

Figure 2: Overview of our proposed bi-level alignment framework (BLA), which
mainly consists of four components: feature extractor $(\mathcal{F})$, density
estimator $(\mathcal{E})$, task-driven data alignment and local fine-grained
discriminator $(\mathcal{D})$. At training time, the source dataset $S$ is
transformed to $S^{+}$ with the optimal transform searched via task-driven
data alignment (Alg. 1), during which the validation feature generator
provides target-like features for candidate transform validation. Then the
entire network is optimized based on $S^{+}$ and $T$ using the training
objective in Eq. 5. At test time, we simply feed a target image $x_{T}^{i}$ to
$\mathcal{F}$ and $\mathcal{E}$ to obtain the predicted density map
$\widetilde{y}_{T}$.

###  2.3 Auto Machine Learning

Auto machine learning (AutoML) aims to free human practitioners and
researchers from selecting the optimal values for each hyperparameter, such as
learning rate, weight decay, and dropout [2], or designing well-performing
network architectures [1]. Pioneers in this field develop optimization methods
to guide the search process based on reinforcement learning (RL) [7],
evolutionary algorithm (EA) [29] and Bayesian optimization [18]. These works
are often impractical because of the required computational overhead. In
contrast, a differentiable controller [20] converts the selection into a
continuous hidden space optimization problem, allowing for an efficient search
process performed by a gradient-based optimizer.

We apply a differentiable controller to search for several hyperparameters
which represent styles. The transformed source images are then used for
training, where the feature alignment becomes easier as the domain gap is
reduced. In order to verify the searched hyperparameters, we construct a
target-like domain in the feature space via AdaIN.

##  3 Bi-Level Alignment Counting (BLA)

In this section, we will introduce our bi-level alignment method for cross-
domain crowd counting. Our core idea is to perform alignment between the
source and target domains at both data-level and feature-level via two
components namely task-driven data alignment and fine-grained feature
alignment. The overall pipeline is depicted in Fig. 2 and detailed
descriptions are provided in the following.

###  3.1 Problem Formulation

For UDA crowd counting, we have an annotated synthetic dataset
$S=(x_{S}^{i},y_{S}^{i})^{N_{S}}_{i=1}$ as source and an unlabeled real-world
dataset $T={(x_{T}^{i})}^{N_{T}}_{i=1}$ as target, where
$x_{S}^{i},x_{T}^{i}\in\mathbb{R}^{3\times H\times W}$ denote an arbitrary
image from the source and target domain, and $y_{S}^{i}\in\mathbb{R}^{H\times
W}$ represents the ground truth density map in source. Our goal is to obtain a
model that performs well on the target domain via reducing the large domain
gap between the source and target.

###  3.2 Overview

As shown in Fig. 2, we propose a new UDA method for crowd counting based on
adversarial learning. It consists of feature extractor $(\mathcal{F})$,
density estimator $(\mathcal{E})$, task-driven data alignment and local fine-
grained discriminator $(\mathcal{D})$.

At training time, the source dataset $S$ is first transformed to $S^{+}$, with
the same labels; a pair of images $(x_{S^{+}},x_{T})$ from the augmented
source and target domains are fed into $\mathcal{F}$, obtaining corresponding
feature maps $(F_{S},F_{T})$, $F_{S}$ and $F_{T}\in\mathbb{R}^{C\times h\times
w}$; $\mathcal{D}$ performs feature alignment by passing reversed gradients to
$\mathcal{F}$; in the end, $\mathcal{E}$ predicts density maps
$\widetilde{y}_{S}$ based on $F_{S}$, supervised by $y_{S}$. At test time, the
inference is rather simple: each target image $x_{T}^{i}$ only goes through
$\mathcal{F}$ and $\mathcal{E}$ to obtain the predicted density map
$\widetilde{y}_{T}$.

Following previous works, we employ VGG16 [22] as our feature extractor
$\mathcal{F}$. For $\mathcal{E}$, we stack a series of convolution and
deconvolution layers, inspired by [4].

The density estimation loss $\bm{\mathcal{L}}_{E}$ on a labeled source image
can be defined as follows:

$\bm{\mathcal{L}}_{E}=\sum\left\|y_{S}-\widetilde{y}_{S}\right\|^{2}.$ (1)

###  3.3 Task-Driven Data Alignment

Typically, domain augmentation enhances the performance on the target domain
using generated new samples via domain randomization or style transfer. In
contrast to previous domain augmentation methods that are blind to the
downstream task, our method searches for the most suitable augmentation based
on both the target styles and the task performance on target via AutoML. In
such a task-driven way, our method is expected to find a transform that better
serves for the downstream task on the given target domain.

It has been shown in [19] that for the crowd counting task, images mainly
differ in color, scale and perspective. Accordingly, in this paper, we define
three basic transform units: RGB2Gray, scaling and perspective transform. Each
transform is a combination of the above three transform units. As shown in
Tab. 1, one transform is defined by five parameters, among which only three
are searched for simplicity. It is notable the transform is not limited to the
above three units and can be easily extended to different and also more types,
with proper manual definitions.

Transform unit | Parameters
---|---
Split ratio | Attribute
RGB2Gray |  p${}_{\text{G}}$* | -
Scaling |  p${}_{\text{S}}$ | Scale factor*
Perspective transform |  p${}_{\text{PT}}$ | Angle*
Table 1: Each transform consists of three different units, each represented by
two parameters: one for split ratio and another for attribute. * marks those
parameters we search while others are fixed. A full transform set is generated
by iterating each parameter.

Given a transform, we split the whole source set into several subsets via a
transform tree, as shown in Fig. 3. At the 1st level, the whole source dataset
is split into two subsets with a ratio of p${}_{\text{G}}$, i.e. some images
are converted to gray scale images (along path Y), while others are kept the
same (along path N). At the following levels, each subset generated from the
previous level goes through one split with a given split ratio and an
attribute parameter. Given three transform units, we finally generate 8
($2^{3}$) subsets.

Figure 3: We split the whole source set into several subsets via a transform
tree according to p${}_{\text{G}}$, p${}_{\text{S}}$ and p${}_{\text{PT}}$.
“Y: p${}_{\text{G}}$” refers to performing the transformation G with a ratio
of p${}_{\text{G}}$; “N: 1 - p${}_{\text{G}}$” means that no transformation is
performed with a ratio of 1 - p${}_{\text{G}}$. Finally, we obtain the
corresponding subsets as shown at the bottom row, which constitute $S^{+}$.

In this paper, we use a differential controller to guide the direction of our
search process. The search process is iterated via multiple rounds, each of
which is described in Alg. 1. At each round, we first transform the source
data given some transform candidates, and then obtain the reward of each
transform via validation on a generated target-like set; after that, we learn
the mapping function from transforms to corresponding rewards via training a
differentiable controller; finally, we update the transform candidates based
on the controller and goes to the next search round. In the following, we
explain the above process in more detail.

Source Data Transformation Based on Candidate Transforms. At this stage, we
first randomly initialize a transform set
$\mathbb{D}={(d_{k})}^{N_{\mathbb{D}}}_{k=1}$. Given an arbitrary transform
$d_{k}$, we split the whole source set into several subsets via a transform
tree, as illustrated in Fig. 3. In this way, we apply one transform on the
source data and obtain a new mixed source dataset
$S^{+}_{k}=(x_{S^{+}_{k}}^{i},y_{S^{+}_{k}}^{i})^{N_{S}}_{i=1}$. Due to the
large number of source images, we do not do standard data augmentation, but
apply the transform on the original data to keep the same size of $S^{+}_{k}$
and $S$.

Candidate Transform Validation. Based on each new source dataset $S^{+}_{k}$,
we train the whole network as shown in Fig. 2 with the learning objective from
Eq. 5. Now we need to evaluate the alignment quality of each $S^{+}_{k}$.
Ideally, this should be done by measuring the counting performance on $T$.
Unfortunately, we do not have labels for $T$. To address this problem, we
propose a validation feature generator, which takes the features of a pair of
source and target image features $(F_{S},F_{T})$ from the feature extractor as
input and generate a new feature $F_{V}$ via AdaIN [13], which is a mixture of
source contents and target style, namely a target-like image feature.

Specifically, we first compute the source style representation with channel-
wise mean and standard deviation $\mu(F_{S})$,
$\sigma(F_{S})\in\mathbb{R}^{C}$ and the target style representation
$\mu(F_{T})$, $\sigma(F_{T})\in\mathbb{R}^{C}$. Then we replace the style of
$F_{S}$ with that of $F_{T}$ and obtain $F_{V}$:

$F^{c}_{V}=\mu(F^{c}_{T})+\sigma(F^{c}_{T})\cdot(\frac{F^{c}_{S}-\mu(F^{c}_{S})}{\sigma(F^{c}_{S})}),$
(2)

where $c\in\\{1,2,3,\cdot\cdot\cdot C\\}$ is the channel index. After that, we
feed $F_{V}$ to the density estimator and get $\widetilde{y}_{V}$. Because
$F_{S}$ and $F_{V}$ share the same contents, we evaluate $\widetilde{y}_{V}$
based on $y_{S}$. In this way, we obtain the evaluated validation performance
$p_{k}$ as reward for transform $d_{k}$.

Candidate Transform Update. After obtaining the reward for each transform in
$\mathbb{D}$, we then train a differentiable controller and let it learn the
mapping function from a transform to its corresponding reward. The controller
is of encoder-decoder structure. The encoder takes a transform as input, maps
it to a hidden state, and predicts its performance as $\widetilde{p}_{k}$. The
decoder reconstructs the transform $d_{k}$ as $\widetilde{d}_{k}$ from the
hidden state. The loss function of our controller is defined as:

$\bm{\mathcal{L}}_{C}=\left\|d_{k}-\widetilde{d}_{k}\right\|^{2}+\left\|p_{k}-\widetilde{p}_{k}\right\|^{2}.$
(3)

Same with NAO [20], we then update the hidden state towards the gradient
direction of improved performance and obtain a new transform set
$\mathbb{D^{\prime}}$, for better alignment. After several rounds, we choose
the optimal transform from all validated transforms based on their rewards.
Please refer to [20] for more details regarding the update procedure.

Algorithm 1 Pseudo code of one-round search procedure of data-level alignment

Input: Source and target domain training set $S$, $T$; the pre-trained source
only network
$\mathcal{\widehat{G}}=\\{\widehat{\theta}_{\mathcal{F}},\widehat{\theta}_{\mathcal{E}}\\}$;
the candidate transform set $\mathbb{D}$ and the controller $\mathcal{C}$

1: for $d_{k}\in\mathbb{D}$ do

2: initialize
$\mathcal{G}=\\{\theta_{\mathcal{F}},\theta_{\mathcal{E}},\theta_{\mathcal{D}}\\}$
with $\mathcal{\widehat{G}}$

3: $S^{+}_{k}=$ transform ($S$,$d_{k}$){As Fig. 3}

4: $\mathcal{G}=$ train ($\mathcal{G},S^{+}_{k},T$)

5: $p_{k}=$ validate ($\mathcal{G},S,T$) {Val performance as reward}

6: end for

7: $\mathbb{P}=\\{p_{1},p_{2}...\\}$

8: $\mathcal{C}=$ train controller($\mathcal{C},\mathbb{D},\mathbb{P}$) {As
Eq. 3}

9: $\mathbb{D}=$ update ($\mathcal{C},\mathbb{D}$) {Same with NAO [20]}

10: return $\mathbb{D},\mathbb{P}$

###  3.4 Fine-Grained Feature Alignment

To perform feature alignment, we employ adversarial learning via a
discriminator and a gradient reverse layer. Inspired by the success of using
segmentation as an auxiliary task for crowd counting [23], we propose a fine-
grained discriminator, with two separated classification heads for foreground
and background regions. To handle the unbalanced numbers of foreground and
background pixels, the discriminator is applied on local patches instead of
pixels.

Given the grid size $G=(g_{h},g_{w})$, we feed a pair of feature maps
$(F_{S},F_{T})$ to $\mathcal{D}$ and obtain two pairs of patch-level
discrimination maps: $(O_{FS},O_{BS})$ for source and $(O_{FT},O_{BT})$ for
target, separating foreground and background. Each map
$O_{FS},O_{FT},O_{BS},O_{BT}\in\mathbb{R}^{(H/g_{h})\times(W/g_{w})}$.

The segmentation masks ($M_{S},\widetilde{M}_{T}$) are obtained by
thresholding the ground truth density maps. Please note that for the target we
use pseudo density maps instead. Specifically, we apply a threshold of $th$ on
each patch, to threshold the sum of all its pixel values.

As shown in Fig. 2, we define local fine-grained discrimination losses of
background and foreground $\bm{\mathcal{L}}_{B}$, $\bm{\mathcal{L}}_{F}$, as
follows:

$\begin{split}\bm{\mathcal{L}}_{D}&=\bm{\mathcal{L}}_{B}+\bm{\mathcal{L}}_{F}\\\
\bm{\mathcal{L}}_{B}&=\bm{\mathcal{L}}_{BS}+\bm{\mathcal{L}}_{BT}\\\
&=\sum-(1-M_{S})\cdot log(1-O_{BS})\\\ &+\sum-(1-\widetilde{M}_{T})\cdot
log(O_{BT}),\\\
\bm{\mathcal{L}}_{F}&=\bm{\mathcal{L}}_{FS}+\bm{\mathcal{L}}_{FT}\\\ &=\sum-
M_{S}\cdot log(1-O_{FS})\\\ &+\sum-\widetilde{M}_{T}\cdot
log(O_{FT}).\end{split}$ (4)

We use the same back-propagation optimizing scheme with the gradient reverse
layer [3] for adversarial learning.

###  3.5 Optimization

The optimization objective of the whole method is:

$\bm{\mathcal{L}}=\bm{\mathcal{L}}_{E}+\lambda\bm{\mathcal{L}}_{D},$ (5)

where $\lambda$ is a weight factor to balance the task loss
$\bm{\mathcal{L}}_{E}$ and the domain adaptation loss $\bm{\mathcal{L}}_{D}$.

The whole network is optimized via two steps. For data alignment, we first
optimize all the parameters in the network including the feature alignment
component for each transform to obtain the corresponding reward, during the
search process in Alg. 1. After selecting the best transform, we retrain the
whole network including the feature alignment component with the transformed
$S^{+}$.

##  4 Experiments

We first introduce the datasets, evaluation metrics and implementation
details; then we provide comparisons with state-of-the-art methods, followed
by analysis on data alignment; finally, we perform some ablation studies.

###  4.1 Datasets and Evaluation Metrics

To evaluate the proposed method, the experiments are conducted under
adaptation scenarios from GCC [26] to five large-scale real-world datasets,
i.e. ShanghaiTech Part A/B (SHA/SHB) [33], QNRF [15], UCF-CC-50 [14] and
WorldExpo’10 [32] respectively. Statistics are listed in Tab. 2.

Dataset | Attribute | # Images |  | Cnt
---
(Mean $\pm$ Std)
GCC | Syn | 15,211 | 501$\pm$718
SHA | Real | 482 | 501$\pm$456
SHB | Real | 716 | 123$\pm$94
QNRF | Real | 1,535 | 815$\pm$1176
UCF-CC-50 | Real | 50 | 1,279$\pm$950
WorldExpo’10 | Real | 3,980 | 50$\pm$41
Table 2: Statistics of five real-world (Real) datasets and one synthetic (Syn)
dataset GCC used for experiments.

Following previous works, we adopt Mean Absolute Error (MAE) and Mean Squared
Error (MSE) as evaluation metrics. They are formulated as:
$MAE=\frac{1}{N}\sum_{i=1}^{N}\left|\sum y_{i}-\sum\widetilde{y}_{i}\right|,$
and $MSE=\sqrt{\frac{1}{N}\sum_{i=1}^{N}{\left|\sum
y_{i}-\sum\widetilde{y}_{i}\right|}^{2}},$ where $N$ is the number of test
images; $\sum y_{i}$, $\sum\widetilde{y}_{i}$ represent the ground truth and
predicted number on the $i$-th image respectively.

Method | SHA | SHB | QNRF | UCF-CC-50 | WorldExpo’10
---|---|---|---|---|---
MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | Avg. MAE
NoAdpt [34] | 160.0 | 216.5 | 22.8 | 30.6 | 275.5 | 458.5 | 487.2 | 689.0 | 42.8
Cycle-GAN [34] | 143.3 | 204.3 | 25.4 | 39.7 | 257.3 | 400.6 | 404.6 | 548.2 | 26.3
SE Cycle-GAN [26] | 123.4 | 193.4 | 19.9 | 28.3 | 230.4 | 384.5 | 373.4 | 528.8 | 26.3
SE Cycle-GAN(JT) [27] | 119.6 | 189.1 | 16.4 | 25.8 | 225.9 | 385.7 | 370.2 | 512.0 | 24.4
FSC [8] | 129.3 | 187.6 | 16.9 | 24.7 | 221.2 | 390.2 | - | - | -
FADA [6] | - | - | 16.0 | 24.7 | - | - | - | - | 21.6
GP [24] | 121.0 | 181.0 | 12.8 | 19.2 | 210.0 | 351.0 | 355.0 | 505.0 | 20.4
DACC [4] | 112.4 | 176.9 | 13.1 | 19.4 | 211.7 | 357.9 | - | - | 17.4
BLA (ours) | 99.3 | 145.0 | 11.9 | 18.9 | 198.9 | 316.1 | 346.8 | 480.0 | 17.9
Table 3: Comparison of our method with previous methods for synthetic-to-real
adaptation. All methods employ VGG16 [22] as backbone.

###  4.2 Implementation Details

The architectures of feature extractor ($\mathcal{F}$), density estimator
($\mathcal{E}$), fine-grained discriminator ($\mathcal{D}$) and controller
($\mathcal{C}$) are listed in supplementary. We input 4 pairs of source and
target images with a uniform size of $576\times 768$ at each iteration.
Following the previous work [4], we generate the ground truth density map
using Gaussian kernel with a kernel size of 15$\times$ 15 and a fixed standard
deviation of 4. We set $th$, $G$ in Eq. 4 and $\lambda$ in Eq. 5 to 0.005,
(16,16) and 1.0 respectively for all our experiments; both p${}_{\text{S}}$
and p${}_{\text{PT}}$ are set to 0.5. And the gradient factor of the gradient
reverse layer is set to 0.01. We also adopt a scene regularization strategy
proposed by [26] to avoid negative knowledge transfer. We train the
adversarial framework and controller with Adam optimizer with default
parameters, and their learning rates are initialized as $10^{-5}$ and
$10^{-1}$ respectively. All experiments are conducted on a single NVIDIA RTX
2080TI GPU with 11GB of VRAM and our code is implemented with Pytorch.

###  4.3 Comparisons with State-of-the-Art

We compare our method BLA with previous published unsupervised domain adaptive
crowd counting methods under the adaptation scenarios from synthetic GCC
dataset to five different real-world datasets. All methods employ VGG16 [22]
as backbone.

Figure 4: Qualitative adaptation results of images with different levels of
density on SHA.

From the results in Tab. 3, we have the following observations: (1) Our
proposed method outperforms all existing domain adaptation methods by a large
margin across different datasets and on WorldExpo’10 we achieve comparable
results with DACC. In particular, on SHA our proposed method achieves 99.3 MAE
and 145.0 MSE, outperforming previous best results by 13.1 pp w.r.t. MAE and
31.9 pp w.r.t. MSE. (2) Our method is robust across various real target
datasets, showing high adaptability. As shown in Tab. 2, although the density
of these real-world datasets varies a lot, we perform the best on all target
domains.

###  4.4 Analysis on Task-Driven Data Alignment

In order to understand how the data alignment behaves, we show the searched
transforms for different target datasets in Tab. 4. It shows that our task-
driven data alignment is quite interpretable, representing various domain gaps
between source and different target domains. For instance, GCC contains
highly-saturated color images while UCF-CC-50 and WorldExpo’10 contain lots of
images with low saturations, so the RGB2Gray ratios on them are rather high
(0.85 and 0.98) as gray scale images are of 0 saturation; in contrast, SHB is
closer to GCC in terms of saturation, so the RGB2Gray ratio on SHB is rather
low (0.16). Similarly, since UCF-CC-50 is denser than other datasets, its
scaling factor is particularly smaller, such that denser regions with small-
scale heads will be generated.

Dataset | Searched transform $d$
---|---
p${}_{\text{G}}$ | Scale factor | Angle
SHA | 0.78 | 0.77 | $12^{\circ}$
SHB | 0.16 | 0.42 | $3^{\circ}$
QNRF | 0.67 | 0.41 | $25^{\circ}$
UCF-CC-50 | 0.85 | 0.23 | $4^{\circ}$
WorldExpo’10 | 0.98 | 0.57 | $17^{\circ}$
Table 4: Searched transforms vary across different target datasets. Dataset | Searched $d$ for SHA | Searched $d$ for SHB
---|---|---
[0.78, 0.77, $12^{\circ}$] | [0.16, 0.42, $3^{\circ}$]
| MAE | MSE | MAE | MSE
SHA | 99.3 | 145.0 | 104.3 | 153.7
SHB | 14.7 | 26.4 | 11.9 | 18.9
Table 5: Different datasets require different transformations for optimal
results.

Moreover, we perform cross dataset validation by applying the searched
transform on one dataset to another. As shown in Tab. 5, when testing on SHB,
the transform searched on SHA underperforms that searched on SHB; and vice
versa. These results indicate that each dataset requires its personalized
transform to achieve the optimal performance. It is necessary to search for
the most suitable transform on each dataset in an automatic way so as to avoid
tedious manual designs.

Additionally, Fig. 4 shows some qualitative results on the SHA dataset. From
Column 3, we can see that without adaptation, the model either fails to detect
the presence of people in some areas (top row), or fails to get a correct
estimate of the local density (middle and bottom rows). Our task-driven data
alignment helps to reduce the errors largely, indicating the domain gaps are
significantly narrowed. After further adding our fine-grained feature
alignment, our BLA method provides more accurate final counts.

###  4.5 Ablation Studies

In this subsection, we conduct some ablation studies to analyze different
components of our proposed BLA. All experiments are conducted under the GCC
$\xrightarrow{}$ SHA adaptation due to its large variation in crowd density.

Effects of Two Levels of Alignment. We first analyze the effects of data
alignment and feature alignment. As shown in Tab. 6, the performance is
significantly improved by $\sim$25 pp w.r.t MAE (from 134.7 to 109.1) when
task-driven data alignment is employed. On the other hand, we also observe a
large improvement of $\sim$14 pp w.r.t MAE (from 134.7 to 121.1) by replacing
global feature alignment with fine-grained feature alignment. Moreover, we
obtain a total gain of $\sim$35 pp w.r.t MAE by adding both alignments. These
results indicate the effects of two levels of alignment and the
complementarity between them.

| Task-Driven
---
Data Alignment
| Fine-Grained
---
Feature Alignment
MAE | MSE
$\times$ | $\times$ | 134.7 | 210.9
✓ | $\times$ | 109.1 | 153.8
$\times$ | ✓ | 121.1 | 200.8
✓ | ✓ | 99.3 | 145.0
Table 6: Effects of two levels of alignment.

Effect of Task-Driven Data Alignment To evaluate the effectiveness of task-
driven data alignment, we replace it with domain randomization and style
transfer. From Tab. 7, we can see that our task-driven data alignment
outperforms previous two data augmentation methods by a large margin.

Method | MAE | MSE
---|---|---
Style Transfer | 119.4 | 194.6
Domain Randomization | 110.0 | 164.2
Task-Driven Data Alignment | 99.3 | 145.0
Table 7: Effect of Task-Driven Data Alignment.

Effect of Validation Feature Generator. To evaluate the alignment quality of
each transform, we generate a target-like feature set for validation via
AdaIN. In Fig. 5, we compare the counting performance on our generated
validation set and the real target training set w.r.t. MAE, where the index
indicates different combinations of transform parameters. We can see that the
two curves go in a similar trend, i.e. the worst performance happens at index
0, the best performance happens at index 7, and there is fluctuation in
between. This comparison demonstrates that our generated validation set is of
high similarity to the real target set, allowing us to do effective validation
without relying on target annotations. On the other hand, we observe the
performance varies a lot along the choice of transform parameters, showing
that different transforms highly affect the performance. Thus it is of great
importance to search for an optimal transform for a given target set.

Figure 5: Comparison of validation performance on the generated set (left) and
real target set (right) across different transforms w.r.t. MAE. The similar
trend verifies the effect of our validation feature generator.

Impact of Grid Size in Fine-Grained Feature Alignment. Our fine-grained
feature alignment strategy is conducted in a patch-wise style. We analyze how
the grid size $G$ affects the performance. As shown in Tab. 8, if the size $G$
is too small or too large, there will be a data imbalance between the numbers
of background and foreground patches, which results in poor feature alignment.
Since the patch size of 16 performs the best, we use 16 as the default size in
all experiments.

Grid size $G$ | MAE | MSE
---|---|---
(2,2) | 138.1 | 222.9
(4,4) | 130.5 | 205.4
(8,8) | 129.0 | 206.3
(16,16) | 121.1 | 200.8
(32,32) | 141.4 | 223.6
Table 8: Impact of local grid size $G$ used in fine-grained feature alignment.

In the supplementary material, we provide more ablation studies on the impact
of segmentation threshold, effect of additional style transfer from $S^{+}$ to
$T$, effect of using more transformations, and a comparison to grid search.

##  5 Conclusion

In this work, we propose a bi-level alignment framework for synthetic-to-real
UDA crowd counting. On one hand, we propose task-driven data alignment to
search for a specific transform given the target set, which is applied on the
source data to narrow down the domain gap at the data level. On the other
hand, to alleviate the alignment difficulty on the entire image, we propose to
perform fine-grained feature alignment on foreground and background patches
separately. Extensive experiments on five real-world crowd counting benchmarks
have demonstrated the effectiveness of our contributions.

## Acknowledgements

This work was supported in part by the “111” Program B13022, Fundamental
Research Funds for the Central Universities (No. 30920032201) and the National
Natural Science Foundation of China (Grant No. 62172225).

## References

  * [1] Minghao Chen, Jianlong Fu, and Haibin Ling.  One-shot neural ensemble architecture search by diversity-guided search space shrinking.  In CVPR, pages 16530–16539, 2021.
  * [2] Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, and Quoc V Le.  Autohas: Differentiable hyper-parameter and architecture search.  arXiv.
  * [3] Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation.  pages 1180–1189, 2015.
  * [4] Junyu Gao, Tao Han, Qi Wang, and Yuan Yuan.  Domain-adaptive crowd counting via inter-domain features segregation and gaussian-prior reconstruction.  arXiv.
  * [5] Junyu Gao, Qi Wang, and Xuelong Li.  Pcc net: Perspective crowd counting via spatial convolutional network.  IEEE TCSVT, 30(10):3486–3498, 2019.
  * [6] Junyu Gao, Yuan Yuan, and Qi Wang.  Feature-aware adaptation and density alignment for crowd counting in video surveillance.  arXiv.
  * [7] Xinyu Gong, Shiyu Chang, Yifan Jiang, and Zhangyang Wang.  Autogan: Neural architecture search for generative adversarial networks.  In ICCV, pages 3224–3234, 2019.
  * [8] Tao Han, Junyu Gao, Yuan Yuan, and Qi Wang.  Focus on semantic consistency for cross-domain crowd understanding.  In ICASSP, pages 1848–1852, 2020.
  * [9] Yuhang He, Zhiheng Ma, Xing Wei, Xiaopeng Hong, Wei Ke, and Yihong Gong.  Error-aware density isomorphism reconstruction for unsupervised cross-domain crowd counting.  In AAAI, number 2, pages 1540–1548, 2021.
  * [10] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell.  Cycada: Cycle-consistent adversarial domain adaptation.  pages 1989–1998, 2018.
  * [11] Yutao Hu, Xiaolong Jiang, Xuhui Liu, Baochang Zhang, Jungong Han, Xianbin Cao, and David Doermann.  Nas-count: Counting-by-density with neural architecture search.  In ECCV, pages 747–766, 2020.
  * [12] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu.  Fsdr: Frequency space domain randomization for domain generalization.  In CVPR, pages 6891–6902, 2021.
  * [13] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In ICCV, pages 1501–1510, 2017.
  * [14] Haroon Idrees, Imran Saleemi, Cody Seibert, and Mubarak Shah.  Multi-source multi-scale counting in extremely dense crowd images.  In CVPR, pages 2547–2554, 2013.
  * [15] Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Al-Maadeed, Nasir Rajpoot, and Mubarak Shah.  Composition loss for counting, density map estimation and localization in dense crowds.  In ECCV, pages 532–546, 2018.
  * [16] Wang Li, Li Yongbo, and Xue Xiangyang.  Coda: Counting objects via scale-aware adversarial density adaption.  In ICME, pages 193–198, 2019.
  * [17] Yuhong Li, Xiaofan Zhang, and Deming Chen.  Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes.  In CVPR, pages 1091–1100, 2018.
  * [18] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy.  Progressive neural architecture search.  In ECCV, pages 19–34, 2018.
  * [19] Yongtuo Liu, Qiang Wen, Haoxin Chen, Wenxi Liu, Jing Qin, Guoqiang Han, and Shengfeng He.  Crowd counting via cross-stage refinement networks.  IEEE TIP, 29:6800–6812, 2020.
  * [20] Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu.  Neural architecture optimization.  In NeurIPS, pages 7827–7838, 2018.
  * [21] Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo.  Reducing domain gap by reducing style bias.  In CVPR, pages 8690–8699, 2021.
  * [22] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv, 2014.
  * [23] Vishwanath A Sindagi and Vishal M Patel.  Inverse attention guided deep crowd counting network.  pages 1–8, 2019.
  * [24] Vishwanath A Sindagi, Rajeev Yasarla, Deepak Sam Babu, R Venkatesh Babu, and Vishal M Patel.  Learning to count in the crowd from limited labeled data.  In ECCV, pages 212–229, 2020.
  * [25] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel.  Domain randomization for transferring deep neural networks from simulation to the real world.  pages 23–30, 2017.
  * [26] Qi Wang, Junyu Gao, Wei Lin, and Yuan Yuan.  Learning from synthetic data for crowd counting in the wild.  In CVPR, pages 8198–8207, 2019.
  * [27] Qi Wang, Junyu Gao, Wei Lin, and Yuan Yuan.  Pixel-wise crowd understanding via synthetic data.  IJCV, 129(1):225–245, 2021.
  * [28] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian.  A fourier-based framework for domain generalization.  In CVPR, pages 14383–14392, 2021.
  * [29] Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, and Huchuan Lu.  Lighttrack: Finding lightweight neural networks for object tracking via one-shot architecture search.  In CVPR, pages 15180–15189, 2021.
  * [30] Yanchao Yang and Stefano Soatto.  Fda: Fourier domain adaptation for semantic segmentation.  In CVPR, pages 4085–4095, 2020.
  * [31] Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincentelli, Kurt Keutzer, and Boqing Gong.  Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data.  In ICCV, pages 2100–2110, 2019.
  * [32] Cong Zhang, Kai Kang, Hongsheng Li, Xiaogang Wang, Rong Xie, and Xiaokang Yang.  Data-driven crowd understanding: A baseline for a large-scale crowd dataset.  IEEE TMM, 18(6):1048–1061, 2016.
  * [33] Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma.  Single-image crowd counting via multi-column convolutional neural network.  In CVPR, pages 589–597, 2016.
  * [34] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In ICCV, pages 2223–2232, 2017.