# GRB 211211A: prolonged central engine under strong magnetic field
environment

He Gao Department of Astronomy , Beijing Normal University, Beijing, People’s
Republic of China;  Wei-Hua Lei  Department of Astronomy , School of Physics,
Huazhong University of Science and Technology, Wuhan, Hubei 430074, China  Zi-
Pei Zhu  Department of Astronomy , School of Physics, Huazhong University of
Science and Technology, Wuhan, Hubei 430074, China  Key Laboratory of Space
Astronomy and Technology, National Astronomical Observatories, Chinese Academy
of Sciences, Beijing 100012, China

###### Abstract

Recently, a Kilonova-associated gamma-ray burst (GRB 211211A) has attracted
great attentions, whose lightcurve consists a precursor ($\sim 0.2$ s), a hard
spiky emission ($\sim 10$ s), and a soft long extended emission ($\sim 40$ s).
Kilonova association could prove its merger origin, while the detection of the
precursor infers at least one highly magnetized NS being involved in the
merger. In this case, a strong magnetic flux $\Phi$ is expected to surround
the central engine of GRB 211211A. Here we suggest that when $\Phi$ is large
enough, the accretion flow could be halted far from the innermost stable
radius, which will significantly prolong the lifetime of the accretion
process, so as the GRB duration. For example, we show that as long as the
central BH is surrounded by a strong magnetic flux $\Phi\sim 10^{29}\rm
cm^{2}G$, an accretion flow with $\dot{M}_{\rm ini}\simeq 0.1M_{\odot}s^{-1}$
could be halted at 40 times gravitational radius and slowly transfer into the
black hole in order of $\sim$10 s, which naturally explains the duration of
hard spiky emission. After most of the disk mass has been accreted onto the
BH, the inflow rate will be reduced, so a long and soft extended emission is
expected when a new balance between the magnetic field and the accretion
current is reconstructed at a further radius. Our results further support that
the special behavior of GRB 211211A is mainly due to the strong magnetic field
of its progenitor stars. Multi-messenger detections of GRB 211211A-like events
(sometimes may disguise as a typical LGRBs without extended emission) could
help to diagnose their progenitor system and to better study the events of
compact binary mergers involving high magnetic field NSs.

Gamma-ray burst: general

##  1 Introduction

Phenomenologically, Gamma-ray bursts (GRBs) were classified into two
categories: long-duration, soft-spectrum class (LGRBs) and the short-duration,
hard-spectrum class (SGRBs), based on the bimodal distribution of GRBs in the
duration-hardness diagram (Kouveliotou et al., 1993). The boundary between the
two classes is vague and instrument dependent (Qin et al., 2013).
Traditionally, an observer-frame duration $T_{90}$ $\sim$ 2 s is taken as the
separation line: bursts with $T_{90}$ $>$ 2s are “long” and bursts with
$T_{90}$ $<$ 2 s are “short.”

Theoretically, different type of progenitor were invoked for these two
different classes, i.e., core collapse from Wolf–Rayet star for LGRBs
(Woosley, 1993; Paczyński, 1998; MacFadyen & Woosley, 1999; Woosley & Bloom,
2006) and mergers of two compact stellar objects (NS–NS and NS–BH systems) for
SGRBs (Paczynski, 1986; Eichler et al., 1989; Paczynski, 1991; Narayan et al.,
1992). Many important observations seem to support such interpretations for
the progenitor, for instance, most host galaxies of LGRBs are irregular, star-
forming galaxies, with a few being spiral galaxies with active star formation
(Fruchter et al., 2006), while the majority of host galaxies of short GRBs are
elliptical or early type (Gehrels et al., 2005). On the other hand, the offset
of the SGRB location with respect to their center of the host galaxy are
systemically larger than LGRBs (Berger, 2014). For the collapsar scenario, the
most direct evidence is that a handful of LGRBs are firmly associated with
Type Ib/c supernovae (Woosley & Bloom, 2006). For the merger scenario, the
smoking gun evidence was established by the association between the GW-
detected binary NS (BNS) merger, GW170817 and a weak short duration GRB
170817A (Abbott et al., 2017).

Most recently, a peculiar LGRB 211211A, detected by Fermi/GBM (Mangan et al.,
2021), Swift/BAT (D’Ai et al., 2021) and Insight-HXMT/HE (Zhang et al., 2021),
severely challenges this clean dichotomy of the two populations. The total
duration of this burst is $51.37\pm 0.80$s in BAT ($\sim$34.3 s in GBM), whose
light curve contains three emission episodes (Xiao et al., 2022): a precursor
with duration of $\sim 0.2$ s, a $\sim 10$ s spiky hard main emission (ME),
and a soft long extended emission (EE) up to $>\sim$50 s (Mangan et al.,
2021). This source has attracted great attentions, because it
phenomenologically definitely belongs to the long-duration category (even
without counting the EE part), but many obvious evidence links it to a compact
object merger (Rastinejad et al., 2022; Yang et al., 2022; Xiao et al., 2022):
1) despite its promising proximity, surprisingly no SN was observed to
accompany the GRB down to very deep detection limits, yet an associated
kilonova was discovered based on the detailed analysis of observation data
from multiple optical-NIR telescopes; 2) the physical offset between the burst
and the nucleus of the host galaxy is more consistent with the known offsets
of SGRBs; 3) spectral lag of the ME is consistent with the known values of
SGRB, but obviously deviate from LGRBs in the spectral Lag - $L_{\rm iso}$
diagram. The biggest challenge this burst poses to theorists is: In the
context of merger scenario, how to make the hard spiky emission last for $\sim
10$ s?

Based on the special properties of the precursor (e.g. the waiting time
between it and the spiky main emission is $\sim$ 1 s, which is about the same
as the time interval between GW 170817 and GRB 170817A), especially its
$\sim$22 Hz quasi-periodic oscillations, Xiao et al. (2022) proposes that the
progenitor system of GRB 211211A very likely contains a magnetar. In this
case, the strong seed magnetic field of $\sim 10^{14-15}G$ at the surface of
the magnetar could leave behind a strong magnetic flux $\Phi$ surrounding the
central engine of GRBs (Kiuchi et al., 2014). It has long been proposed that
due to the magnetic barrier effect, radial angular momentum transfer may
significantly prolong the lifetime of the accretion process, so as the GRB
duration (Proga & Zhang, 2006; Liu et al., 2012). In this work, we further
refine the magnetic barrier model (see Figure 1 for a cartoon picture) and
search for a reasonable parameter space to interpret the observations of GRB
211211A.

Figure 1: Illustration of our model. With magnetic barrier effect, the
accretion of the major inflow corresponds to the hard spiky ME of GRB 211211A,
and accretion of the residual inflow corresponds to the soft EE of GRB
211211A. The GRB jet is powered by the Blandford–Znajek mechanism, which
extracts the rotational energy of the Kerr BH through the large-scale magnetic
field. Light curve picture of GRB 211211A is adopted from Xiao et al. (2022).

##  2 Magnetic Barrier Model

Considering that a GRB’s central engine is a spinning black with mass
$M_{\bullet}$ and angular momentum $J_{\bullet}$, surrounded by a strong
magnetic flux $\Phi$. Before encountering the magnetic barrier, the initial
mass flow rate is $\dot{M}_{\rm ini}$. The existence of strong magnetic field
could have a significant impact on the accretion process. Firstly, the
accretion flow will be halted by magnetic barrier at some radius $R_{\rm
halt}$. This occurs when the radial magnetic force can support against the
gravitational force (Proga & Zhang, 2006; Liu et al., 2012; Zhang et al.,
2018),

$\displaystyle\frac{2B_{\rm R}B_{\rm
Z}}{4\pi}\simeq\frac{GM_{\bullet}\Sigma}{R_{\rm halt}^{2}},$ (1)

where $\Sigma$ is the surface density at $R_{\rm halt}$, which can be given by

$\Sigma=\frac{\dot{M}_{\rm ini}}{2\pi R_{\rm halt}\epsilon_{\rm i}v_{\rm
ff}}.$ (2)

Here the flow radial velocity, when the magnetic field is just in contact with
the accretion flow, is assumed to be a fraction $\epsilon_{i}$ of the free-
fall velocity $v_{\rm ff}=\sqrt{GM_{\bullet}/r}$. It is reasonable to adopt
$\epsilon_{i}=10^{-3}-10^{-2}$ (Proga & Zhang, 2006; Liu et al., 2012; Zhang
et al., 2018).

We thus have

$\displaystyle r_{\rm halt}=\frac{R_{\rm halt}}{R_{\rm
g}}=91.0\epsilon_{i,-2}^{2/3}m_{3}^{-4/3}\dot{m}_{\rm
ini,-1}^{-2/3}\Phi_{30}^{4/3},$ (3)

where $R_{\rm g}=GM_{\bullet}/c^{2}$, $\Phi_{30}=\Phi/(10^{30}\rm cm^{2}G)$,
$m_{3}=M_{\bullet}/3M_{\odot}$, $\dot{m}_{\rm
ini,-1}=\dot{M}/0.1M_{\odot}s^{-1}$. Here we assume $B_{\rm R}\simeq B_{\rm
Z}=B$ and the magnetic flux is connected with the magnetic field as
$\Phi\simeq\pi R^{2}B$.

As mass accumulated at $R_{\rm halt}$, the accretion will restart with a much
lower radial velocity $\epsilon_{\rm m}v_{\rm ff}\ll\epsilon_{\rm i}v_{\rm
ff}$ due to the magnetic tension. The accretion time can be estimated with

$\displaystyle t_{\rm acc}=\frac{R_{\rm halt}}{\epsilon_{\rm m}v_{\rm
ff}}\simeq 13{\rm
s}\frac{\epsilon_{i,-2}}{\epsilon_{m,-3}}m_{3}^{-1}\dot{m}_{\rm
ini,-1}^{-1}\Phi_{30}^{2}.$ (4)

Therefore, a larger magnetic flux leads to a larger halting radius $R_{\rm
halt}$ and a longer accretion duration $t_{\rm acc}$.

The initial magnetic filed threading BH horizon can be estimated by (assuming
the magnetic field as a uniform field)

$\displaystyle B_{\rm ini}\sim 1.9\times 10^{14}{\rm G}\ \
m_{3}^{2/3}\dot{m}_{\rm ini,-1}^{4/3}\epsilon_{\rm
i,-2}^{-4/3}\Phi_{30}^{-5/3}.$ (5)

The B field will increase as the flow push the magnetic flux to the radius of
the marginally stable orbit $R_{\rm ms}$. The B field at $R_{\rm ms}$ is

$\displaystyle B_{\rm ms}=4.5\times 10^{16}{\rm G}\ \
m_{3}^{-2}\Phi_{30}(r_{\rm ms}/6)^{-2},$ (6)

where the radius of the marginally stable orbit is expressed as (Bardeen et
al., 1972)

$r_{\rm ms}=3+Z_{2}-sgn(a_{\bullet})[(3-Z_{1})(3+Z_{1}+2Z_{2})]^{1/2},$ (7)

for $0\leq a_{\bullet}\leq 1$, where $a_{\bullet}$ is the BH spin parameter
defined as $a_{\bullet}=J_{\bullet}c/GM_{\bullet}^{2}$, $Z_{1}\equiv
1+(1-a_{\bullet}^{2})^{1/3}[(1+a_{\bullet})^{1/3}+(1-a_{\bullet})^{1/3}]$ and
$Z_{2}\equiv(3a_{\bullet}^{2}+Z_{1}^{2})^{1/2}$.

In this model, the GRB prompt emission can be powered by the Blandford-Znajek
(Blandford & Znajek (1977), hereafter BZ) mechanism, in which the spin energy
of the BH is extracted via the open field lines penetrating the event horizon.
The BZ jet power could be estimated as (Li, 2000; Lee et al., 2000; Wang et
al., 2002; McKinney, 2005; Lei et al., 2005; Lei & Zhang, 2011; Lei et al.,
2013; Liu et al., 2015)

$\displaystyle\dot{E}_{\rm B}=1.7\times
10^{50}a_{\bullet}^{2}m_{\bullet}^{2}B_{\bullet,15}^{2}F(a_{\bullet})\ {\rm
erg\ s^{-1}}$ (8) $\displaystyle\simeq 1.1\times
10^{50}a_{\bullet}^{2}m_{\bullet}^{2}B_{\bullet,15}^{2}\ {\rm erg\ s^{-1}},$

where $B_{\bullet,15}=B_{\bullet}/10^{15}{\rm G}$ and
$F(a_{\bullet})=[(1+q^{2})/q^{2}][(q+1/q)\arctan q-1]$. Here
$q=a_{\bullet}/(1+\sqrt{1-a^{2}_{\bullet}})$, and $2/3\leq
F(a_{\bullet})\leq\pi-2$ for $0\leq a_{\bullet}\leq 1$.

In this model, the prompt emission would be carried out in two parts. The
first part is within the timescale of $t_{\rm acc}$. In this part, most of the
accretion flow will fall into the BH. The accretion rate would normally be
higher than the igniting accretion rates $\dot{m}_{\rm ign}$ for neutrino
emitting reactions, especially considering that strong magnetic field could
effectively reduce $\dot{m}_{\rm ign}$ (Lei et al., 2009), so the hyper-
accreting disk would be neutrino-dominated. In this case, the baryon loading
rate for the BZ driven jet could be estimated as (Lei et al., 2013),

$\displaystyle\dot{M}_{\rm j,BZ}$ $\displaystyle\simeq$ $\displaystyle
3.5\times 10^{-7}A^{0.58}B^{-0.83}f_{\rm p,-1}^{-0.5}\theta_{\rm
j,-1}\theta_{\rm B,-2}^{-1}$ (9)
$\displaystyle\alpha_{-1}^{0.38}\epsilon_{-1}^{0.83}\dot{m}_{-1}^{0.83}m_{3}^{-0.55}r_{z,11}^{0.5}\
M_{\sun}{\rm s}^{-1},$

where $A=1-2r^{-1}+a_{\bullet}^{2}r^{-2}$ and
$B=1-3r^{-1}+2a_{\bullet}r^{-3/2}$ are the general relativistic correction
factors for a thin accretion disk around a Kerr BH (Riffert & Herold, 1995),
$f_{\rm p}$ is the fraction of protons, $\theta_{\rm j}$ is the jet opening
angle, $\alpha$ is the dimensionless viscosity parameter (Shakura & Sunyaev,
1973), $r_{\rm z,11}$ is the distance from the BH in the jet direction
normalized to $10^{11}$cm. Protons with an ejected direction larger than
$\theta_{\rm B}$ with respect to the field lines would be blocked due to the
existence of a strong magnetic field. Apparently, baryons from the disk will
be suppressed by the strong magnetic field lines. The maximum available energy
per baryon in the BZ jet can be denoted by parameter $\mu_{0}$ as

$\mu_{0}\equiv\frac{\dot{E}}{\dot{M}_{\rm j,BZ}c^{2}}=\frac{\dot{E}_{\rm
m}+\dot{E}_{\rm B}}{\dot{M}_{\rm j,BZ}c^{2}}=\eta(1+\sigma_{0}),$ (10)

where $\dot{E}_{\rm m}=\dot{E}_{\nu\bar{\nu}}+\dot{M}_{\rm j,BZ}c^{2}$, and
$\sigma_{0}=\dot{E}_{\rm B}/\dot{E}_{\rm m}$,
$\eta=\dot{E}_{\nu\bar{\nu}}/(\dot{M}_{\rm j,BZ}c^{2})$, where
$\dot{E}_{\nu\bar{\nu}}$ is the neutrino annihilation power as of (Lei et al.,
2017; Liu et al., 2015, 2017)

$\displaystyle\dot{E}_{\nu\bar{\nu}}\simeq\dot{E}_{\nu\bar{\nu},\rm
ign}\left(\frac{\dot{m}}{\dot{m}_{\rm ign}}\right)^{2.23},$ (11)

for $\dot{m}>\dot{m}_{\rm ign}$, where

$\displaystyle\dot{E}_{\nu\bar{\nu},\rm
ign}=10^{(48.0+0.15a_{\bullet})}m_{3}^{\log(\dot{m}/\dot{m}_{\rm
ign})-3.3}{\rm erg\ s^{-1}}.$ (12)

The magnetic dissipation and acceleration dynamics of the jet are quite
uncertain. we take $\Gamma_{\rm min}=\max(\mu_{0}^{1/3},\eta)$ and
$\Gamma_{\rm max}=\mu_{0}$, which correspond to the beginning and the end of
the slow acceleration phase in a hybrid outflow (Gao & Zhang, 2015). The jet
will reach a terminating Lorentz factor $\Gamma_{\rm min}<\Gamma<\Gamma_{\rm
max}$. For typical parameters, relatively large values for $\Gamma$ and
$\sigma$ are expected. So the first part of emission would be hard spiky
emission with duration $t_{\rm acc}$.

Considering both the accretion and the BZ processes, the evolution equations
of BH are given by

$\frac{dM_{\bullet}c^{2}}{dt}=\dot{M}c^{2}E_{\rm ms}-\dot{E}_{\rm B},$ (13)

$\frac{dJ_{\bullet}}{dt}=\dot{M}L_{\rm ms}-T_{\rm B}$ (14)

where $E_{\rm ms}$ and $L_{\rm ms}$ are the specific energy and the specific
momentum corresponding to the inner most radius $r_{\rm ms}$ of the disk,
which are defined in Novikov & Thorne (1973) as $E_{\rm ms}=(4\sqrt{r_{\rm
ms}}-3a_{\bullet})/(\sqrt{3}r_{\rm ms})$, $L_{\rm
ms}=(GM_{\bullet}/c)(2(3\sqrt{r_{\rm ms}}-2a_{\bullet}))/(\sqrt{3}\sqrt{r_{\rm
ms}})$. $T_{\rm B}=\dot{E}_{\rm B}/(0.5\Omega_{\bullet})$ is the BZ torque
applied on BH, and $\Omega_{\bullet}=qc/(2R_{\rm g})$ is the angular velocity
of BH horizon.

After the first part, since most of the disk mass has been accreted onto the
BH, the inflow rate of the remaining mass should be quite low. A majority of
the magnetic flux would quickly diffuse out (Liu et al., 2012; Zhang et al.,
2018). The balance between the magnetic field and the accretion current will
be reconstructed at a new and further radius. Nevertheless, because the
magnetic field and accretion rate decrease at the same time, it is difficult
to reach the ignition threshold of neutrino emitting reactions. The disk will
thus be advection-dominated, which has a strong disk wind driven by a positive
Bernoulli constant (Narayan & Yi, 1994; Song et al., 2018). For typical
parameters, relatively small values for $\Gamma$ and $\sigma$ are expected. So
the second part of emission would be longer and softer than the first part.
Depending on the detector’s sensitivity, the second emission episodes may not
be detected or may be detected as an extended emission phase (see detailed
modeling for this part in Liu et al. (2012)).

##  3 Application to GRB 211211A

The prompt emission of GRB 211211A could be divided into three episodes: a
precursor with duration of $\sim 0.2$ s, a $\sim 10$ s spiky hard ME, and a
soft long EE up to $>\sim$50 s. Here we show that the ME and EE parts could be
well interpreted within the magnetic barrier model, as long as the central
engine of GRB 211211A contained a newly formed BH with mass $M_{\bullet}=\sim
2.5M_{\odot}$, initial magnetic flux of $\Phi_{30}\simeq 0.2$ and a relatively
low initial BH spin $a_{\bullet}=0.1$ (low spin is due to the possible
rotation braking caused by strong magnetic field). The initial accretion flow
rate is taken as $\dot{m}_{\rm ini}\simeq 0.1$, which is a typical value for
regular SGRBs.

Firstly, from Eq.(3), the accretion flow will be halted at radius $r\simeq 40$
due to the magnetic barrier. The accretion timescale is $t_{\rm acc}\simeq
10s$ for $\epsilon_{i}=0.05$ and $\epsilon_{m}=3\times 10^{-4}$, which
explains the duration of ME.

Secondly, during the first 10 s, the average magnetic filed threading BH would
be $\sim 10^{16}$ G. From Eq.(8), the luminosity of BZ jet would be $\sim
10^{51}{\rm erg\ s^{-1}}$, which is well consistent with the jet-corrected
energies $E_{\gamma}+E_{K}$ for ME (Xiao et al., 2022).

Thirdly, for typical values of $A=0.8$, $B=0.7$, $\alpha=0.01$, $f_{\rm
p}=0.1$, $\theta_{\rm B}=0.01$ and $r_{\rm z,11}=1$ (Lei et al., 2017, for
details), from Eq.(10), the maximum available energy per baryon in the BZ jet
would be $\sim 3000$. In this case, the Lorentz factor of the jet will be much
larger than 400, which could be the reason why ME consists of many spiky
pulses exhibiting little spectral evolution.

Finally, after the first 10 s, the majority of the remanet disk mass (e.g.,
$\sim 0.2M_{\odot}$) is accreted. The inflow rate of the remaining mass is
largely reduced, so as the magnetic flux maintaining by the inflow. Here we
take $\dot{m}_{\rm flow}\simeq 0.001$ and $\Phi_{30}\simeq 0.07$. In this
case, the accretion flow will be halted at radius $r\simeq 200$ due to the
magnetic barrier. The accretion timescale is $t_{acc}\simeq 40s$ for
$\epsilon_{i}=0.05$ and $\epsilon_{m}=8\times 10^{-4}$, which explains the
duration of EE. Due to the accretion, the BH spin has increased to
$a_{\bullet}=0.3$. At this stage, the magnetic filed threading BH would be up
to $\sim 10^{15}$ G, so the luminosity of BZ jet would be $\sim 10^{50}{\rm
erg\ s^{-1}}$, which is also consistent with the jet-corrected energies
$E_{\gamma}$ for EE (Xiao et al., 2022).

##  4 Conclusion and Discussion

Compared with other EE-SGRBs, GRB 211211A has three particularities: 1) an
associated KN has been found; 2) a peculiar precursor has been found; 3) the
ME duration prior to EE is in order of 10 s. KN association could prove its
merger origin, while the detection of the precursor infers at least one highly
magnetized NS being involved in the merger. Numerical relativity simulations
have already shown that for the merger of binary highly magnetized NSs,
central engine of GRBs would be modeled by a highly magnetized accretion torus
(Kiuchi et al., 2014). Here we show that such a scenario could naturally
explain the long ME duration by invoking magnetic barrier effect. We find that
as long as the central BH is surrounded by a strong magnetic flux $\Phi\sim
10^{29}\rm cm^{2}G$, an accretion flow with $\dot{m}_{\rm ini}\simeq 0.1$
could be halted at $40R_{\rm g}$ and slowly transfer into the black hole in
order of $\sim$10 s.

Our results further support that the special behavior of GRB 211211A is mainly
due to the strong magnetic field of its progenitor stars. For the merger
events involving NSs with ordinary magnetic field, the magnetic flux around
the central BH may not be strong enough to block the accretion flow resistance
beyond the radius of the innermost stable orbit. Therefore, a short duration
ME will be generated. For such events (including GRB 211211A-like events),
whether the ME would be followed by an EE depends on the nature of late
accretion (Liu et al., 2012).

Numerical simulations show that the gravitational waveforms of magnetized and
unmagnetized NS binaries could be well distinguished as long as the NS
magnetic field is strong enough (Giacomazzo et al., 2009). In the future,
multi-messenger detections of GRB 211211A-like events could help to diagnose
their progenitor system and thus justify our model. Nevertheless, it is worth
to note that GRB 211211A-like events could disguise as a typical LGRBs, once
their EE are too weak to be recognized. In the future, LGRBs with
$T_{90}<\sim$10 s, especially those with clear signatures of merger origin
(e.g., small spectral lag, large host galaxy offset, KN association etc.),
should also be carefully studied, which might be help to the estimation of the
event rate of merging events involving high magnetic field NSs.

## Acknowledgments

This work is supported by the National Natural Science Foundation of China
(Projects:12021003, U2038107, U1931203). We acknowledge the science research
grants from the China Manned Space Project with NO. CMS-CSST-2021-A13 and CMS-
CSST-2021-B11.

## References

  * Abbott et al. (2017) Abbott, B. P., Abbott, R., Abbott, T. D., et al. 2017, ApJ, 848, L13. doi:10.3847/2041-8213/aa920c
  * Bardeen et al. (1972) Bardeen, J. M., Press, W. H., & Teukolsky, S. A. 1972, ApJ, 178, 347. doi:10.1086/151796
  * Berger (2014) Berger, E. 2014, ARA&A, 52, 43. doi:10.1146/annurev-astro-081913-035926
  * Blandford & Znajek (1977) Blandford, R. D. & Znajek, R. L. 1977, MNRAS, 179, 433. doi:10.1093/mnras/179.3.433
  * D’Ai et al. (2021) D’Ai, A., Ambrosi, E., D’Elia, V., et al. 2021, GRB Coordinates Network, Circular Service, No. 31202, 31202
  * Eichler et al. (1989) Eichler, D., Livio, M., Piran, T., et al. 1989, Nature, 340, 126. doi:10.1038/340126a0
  * Fruchter et al. (2006) Fruchter, A. S., Levan, A. J., Strolger, L., et al. 2006, Nature, 441, 463. doi:10.1038/nature04787
  * Gao & Zhang (2015) Gao, H. & Zhang, B. 2015, ApJ, 801, 103. doi:10.1088/0004-637X/801/2/103
  * Gehrels et al. (2005) Gehrels, N., Sarazin, C. L., O’Brien, P. T., et al. 2005, Nature, 437, 851. doi:10.1038/nature04142
  * Giacomazzo et al. (2009) Giacomazzo, B., Rezzolla, L., & Baiotti, L. 2009, MNRAS, 399, L164. doi:10.1111/j.1745-3933.2009.00745.x
  * Kiuchi et al. (2014) Kiuchi, K., Kyutoku, K., Sekiguchi, Y., et al. 2014, Phys. Rev. D, 90, 041502. doi:10.1103/PhysRevD.90.041502
  * Kouveliotou et al. (1993) Kouveliotou, C., Meegan, C. A., Fishman, G. J., et al. 1993, ApJ, 413, L101. doi:10.1086/186969
  * Lee et al. (2000) Lee, H. K., Wijers, R. A. M. J., & Brown, G. E. 2000, Phys. Rep., 325, 83. doi:10.1016/S0370-1573(99)00084-8
  * Lei et al. (2005) Lei, W. H., Wang, D. X., & Ma, R. Y. 2005, ApJ, 619, 420
  * Lei et al. (2009) Lei, W. H., Wang, D. X., Zhang, L., et al. 2009, ApJ, 700, 1970. doi:10.1088/0004-637X/700/2/1970
  * Lei & Zhang (2011) Lei, W.-H. & Zhang, B. 2011, ApJ, 740, L27. doi:10.1088/2041-8205/740/1/L27
  * Lei et al. (2013) Lei, W.-H., Zhang, B., & Liang, E.-W. 2013, ApJ, 765, 125. doi:10.1088/0004-637X/765/2/125
  * Lei et al. (2017) Lei, W.-H., Zhang, B., Wu, X.-F., et al. 2017, ApJ, 849, 47. doi:10.3847/1538-4357/aa9074
  * Li (2000) Li, L.-X. 2000, Phys. Rev. D, 61, 084016. doi:10.1103/PhysRevD.61.084016
  * Liu et al. (2012) Liu, T., Liang, E.-W., Gu, W.-M., et al. 2012, ApJ, 760, 63. doi:10.1088/0004-637X/760/1/63
  * Liu et al. (2015) Liu, T., Hou, S.-J., Xue, L., et al. 2015, ApJS, 218, 12. doi:10.1088/0067-0049/218/1/12
  * Liu et al. (2017) Liu, T., Gu, W.-M., & Zhang, B. 2017, New A Rev., 79, 1. doi:10.1016/j.newar.2017.07.001
  * MacFadyen & Woosley (1999) MacFadyen, A. I. & Woosley, S. E. 1999, ApJ, 524, 262. doi:10.1086/307790
  * Mangan et al. (2021) Mangan, J., Dunwoody, R., Meegan, C., et al. 2021, GRB Coordinates Network, Circular Service, No. 31210, 31210
  * McKinney (2005) McKinney, J. C. 2005, ApJ, 630, L5. doi:10.1086/468184
  * Narayan et al. (1992) Narayan, R., Paczynski, B., & Piran, T. 1992, ApJ, 395, L83. doi:10.1086/186493
  * Narayan & Yi (1994) Narayan, R. & Yi, I. 1994, ApJ, 428, L13. doi:10.1086/187381
  * Novikov & Thorne (1973) Novikov, I. D. & Thorne, K. S. 1973, Black Holes (Les Astres Occlus), 343
  * Paczynski (1986) Paczynski, B. 1986, ApJ, 308, L43. doi:10.1086/184740
  * Paczynski (1991) Paczynski, B. 1991, Acta Astron., 41, 257
  * Paczyński (1998) Paczyński, B. 1998, ApJ, 494, L45. doi:10.1086/311148
  * Proga & Zhang (2006) Proga, D. & Zhang, B. 2006, MNRAS, 370, L61. doi:10.1111/j.1745-3933.2006.00189.x
  * Qin et al. (2013) Qin, Y., Liang, E.-W., Liang, Y.-F., et al. 2013, ApJ, 763, 15. doi:10.1088/0004-637X/763/1/15
  * Rastinejad et al. (2022) Rastinejad, J. C., Gompertz, B. P., Levan, A. J., et al. 2022, arXiv:2204.10864
  * Riffert & Herold (1995) Riffert, H. & Herold, H. 1995, ApJ, 450, 508. doi:10.1086/176161
  * Shakura & Sunyaev (1973) Shakura, N. I. & Sunyaev, R. A. 1973, A&A, 24, 337
  * Song et al. (2018) Song, C.-Y., Liu, T., & Li, A. 2018, MNRAS, 477, 2173. doi:10.1093/mnras/sty783
  * Wang et al. (2002) Wang, D. X., Xiao, K., & Lei, W. H. 2002, MNRAS, 335, 655. doi:10.1046/j.1365-8711.2002.05652.x
  * Woosley (1993) Woosley, S. E. 1993, ApJ, 405, 273. doi:10.1086/172359
  * Woosley & Bloom (2006) Woosley, S. E. & Bloom, J. S. 2006, ARA&A, 44, 507. doi:10.1146/annurev.astro.43.072103.150558
  * Xiao et al. (2022) Xiao, S., Zhang, Y.-Q., Zhu, Z.-P., et al. 2022, arXiv:2205.02186
  * Yang et al. (2022) Yang, J., Zhang, B.-B., Ai, S., et al. 2022, arXiv:2204.12771
  * Zhang et al. (2021) Zhang, Y. Q., Xiong, S. L., Li, X. B., et al. 2021, GRB Coordinates Network, Circular Service, No. 31236, 31236
  * Zhang et al. (2018) Zhang, Q., Lei, W. H., Zhang, B. B., et al. 2018, MNRAS, 475, 266. doi:10.1093/mnras/stx3229


# Gauge invariance of the local phase in the Aharonov-Bohm interference:
quantum electrodynamic approach

Kicheon Kang<EMAIL_ADDRESS>Department of Physics, Chonnam National
University, Gwangju 61186, Republic of Korea

###### Abstract

In the Aharonov-Bohm (AB) effect, interference fringes are observed for a
charged particle in the absence of the local overlap with the external
electromagnetic field. This notion of the apparent “nonlocality” of the
interaction or the significant role of the potential has recently been
challenged and are under debate. The quantum electrodynamic approach provides
a microscopic picture of the characteristics of the interaction between a
charge and an external field. We explicitly show the gauge invariance of the
local phase shift in the magnetic AB effect, which is in contrast to the
results obtained using the usual semiclassical vector potential. Our study can
resolve the issue of the locality in the magnetic AB effect. However, the
problem is not solved in the same way in the electric counterpart wherein
virtual scalar photons play an essential role.

Introduction-. A charged particle moving under the influence of an external
electromagnetic field exhibits a topological quantum interference, known as
the Aharonov-Bohm (AB) effect Ehrenberg and Siday (1949); Aharonov and Bohm
(1959). An intriguing aspect of the AB effect is that interference occurs even
when the particle does not locally overlap the field. Thus, the AB
interference is widely regarded as a pure topological effect which cannot be
represented by the local action of gauge-invariant quantities (i.e.,
electromagnetic field). In our previous studies, we demonstrated that the
notion of “nonlocality” contradicts the prediction of local phase measurement
in certain experimental arrangements Kang (2017); Kim and Kang (2018). The
discrepancy could be resolved using a semiclassical approach based on the
local field interaction Kang (2013, 2015), in which the external
electromagnetic field locally interacts with the field produced by a charged
particle.

In recent years, it has been proposed that the local phase in the AB effect
can be described by the quantum electrodynamic (QED) approach Marletto and
Vedral (2020); Saldanha (2021). In the QED picture, the interaction between a
charge and magnetic flux is mediated by the exchange of virtual photons,
represented by the gauge field $A^{\mu}$. It has been claimed that the gauge-
invariant local phase is derived from this approach. Although the QED picture
of the interaction between the two objects should ultimately be right, the
gauge invariance of the local phase remains unclear. This is because the local
phase has been derived by adopting specific choices of gauge in $A^{\mu}$
(e.g., Coulomb Marletto and Vedral (2020) and Lorenz Saldanha (2021) gauges,
respectively). The general gauge invariance should be proved to address this
issue.

In this Letter, we demonstrate the gauge invariance of the local phase in the
magnetic AB effect using the quantum electrodynamic approach. Moreover, the
result is equivalent to that obtained using the semiclassical local field
interaction approach Kang (2017). The QED approach yields an unambiguous
prediction of the local phase measurement in a certain type of superconducting
Andreev interferometer. Our result is remarkable in that the semiclassical
vector potential does not produce a well-defined gauge-invariant local phase.

Quantum electrodynamic Hamiltonian of a charge and a fluxon-. The system
considered comprises a charge $e$ and a ”fluxon” $\Phi$ in two spatial
dimension (Fig. 1). This simplified configuration is sufficient to derive the
essential physics of the AB effect. The interaction between the two entities
is indirect, i.e., it is mediated by virtual photons (gauge field). The
Hamiltonian of the system can be written as

$H=\frac{1}{2m}(\mathbf{p}-\frac{e}{c}\mathbf{A})^{2}+\frac{\mathbf{P}^{2}}{2M}-\frac{\Phi}{4\pi}\hat{z}\cdot\mathbf{B}+\sum_{\mathbf{k},\lambda}\hbar\omega
a_{\mathbf{k}\lambda}^{\dagger}a_{\mathbf{k}\lambda}\,,$ (1)

where the motion of the particles is confined in the $x$-$y$ plane. The charge
($e$) with mass $m$ located at $\mathbf{x}_{a}$ interacts with the quantized
radiation through the vector potential $\mathbf{A}$. In contrast, the fluxon
($\Phi$) with mass $M$ at $\mathbf{x}_{b}$ interacts with the magnetic field
$\mathbf{B}$ of the radiation. The operator $a_{\mathbf{k}\lambda}^{\dagger}$
($a_{\mathbf{k}\lambda}$) represents the creation (annihilation) of a photon
with wave number $\mathbf{k}$ and polarization $\lambda$. Among the four
possible modes of the polarization, only the two transverse modes are real
excitation of radiation (last term of Eq. (1)).

The Hamiltonian can be rewritten as

$H=H_{0}+V_{a}+V_{b},$ (2a) where
$H_{0}=\frac{\mathbf{p}^{2}}{2m}+\frac{\mathbf{P}^{2}}{2M}+\sum_{\mathbf{k},\lambda}\hbar\omega
a_{\mathbf{k}\lambda}^{\dagger}a_{\mathbf{k}\lambda}$ (2b) is the
noninteracting part, and $V_{a}=-\frac{e}{mc}\mathbf{A}\cdot\mathbf{p}$ (2c)
and $V_{b}=-\frac{\Phi}{4\pi}\hat{z}\cdot\mathbf{B}$ (2d)

represent the charge–vacuum and fluxon–vacuum interactions, respectively. In
the charge-potential interaction ($V_{a}$), we have omitted the term
$(i\hbar/2mc)\nabla\cdot\mathbf{A}+(e^{2}/2mc^{2})\mathbf{A}^{2}$ which is
independent of the charge variable and thus irrelevant to the present study.

We can observe the asymmetry between $V_{a}$ and $V_{b}$ in the role of the
gauge field $\mathbf{A}$. The charge interacts with the radiation
$\mathbf{A}$, which has some freedom of gauge selection. However, the
interaction of the fluxon with the radiation is described by the gauge-
independent $\mathbf{B}$. The gauge independence of $V_{b}$ can be confirmed
by considering the fluxon as a current loop (with current $I$) interacting
with $\mathbf{A}$ as

$V_{b}=-\frac{I}{c}\oint\mathbf{A}\cdot d\mathbf{l}\,,$ (3)

where the integration is made along the loop. Applying the Stokes’ theorem, we
recover Eq.(2d), which is independent of gauge choice in $\mathbf{A}$.

The vector potential $\mathbf{A}$ corresponds to the spatial part of the four-
potential $A^{\mu}$. The case of arbitrary gauge will be treated later. First,
we begin with the Coulomb gauge ($\nabla\cdot\mathbf{A}=0$), where the two
transverse modes (represented by $\lambda$) of $A^{\mu}$ are present:

$\mathbf{A}(\mathbf{x},t)=\sum_{\mathbf{k},\lambda}\alpha_{\mathbf{k}}\left[u_{\mathbf{k}}(\mathbf{x})a_{\mathbf{k}\lambda}e^{-i\omega
t}+u_{\mathbf{k}}^{*}(\mathbf{x})a_{\mathbf{k}\lambda}^{\dagger}e^{i\omega
t}\right]\hat{e}_{\lambda}\,,$ (4)

expanded by the plane wave modes
$u_{\mathbf{k}}(\mathbf{x})=e^{i\mathbf{k}\cdot\mathbf{x}}/\sqrt{V}$ with
normalization coefficient $\alpha_{\mathbf{k}}=\sqrt{2\pi\hbar c^{2}/\omega}$
and the corresponding angular velocity $\omega=ck$. The polarization
$\hat{e}_{\lambda}$ is limited to the transverse modes by the constraint
$\mathbf{k}\cdot\hat{e}_{\lambda}=0$ in the Coulomb gauge. The magnetic field
of the radiation, given by $\mathbf{B}=\nabla\times\mathbf{A}$, is gauge-
independent.

$\mathbf{B}(\mathbf{x},t)=i\sum_{\mathbf{k},\lambda}k\alpha_{\mathbf{k}}\left[u_{\mathbf{k}}(\mathbf{x})a_{\mathbf{k}\lambda}e^{-i\omega
t}-u_{\mathbf{k}}^{*}(\mathbf{x})a_{\mathbf{k}\lambda}^{\dagger}e^{i\omega
t}\right]\hat{n}_{\lambda}\,,$ (5)

where $\hat{n}_{\lambda}=\hat{\mathbf{k}}\times\hat{e}_{\lambda}$.

Canonical transformation and effective interaction-.
We adopt the canonical transformation technique to derive the charge–fluxon
interaction mediated by the exchange of virtual photons. The Hamiltonian in
(2) is transformed to $\tilde{H}=e^{-S}He^{S}$, where the first-order
interaction part is eliminated by an appropriate choice of $S$. We obtain

$\displaystyle\tilde{H}$ $\displaystyle=$ $\displaystyle H_{0}+H_{2}\,,$ (6a)
$\displaystyle H_{2}$ $\displaystyle=$
$\displaystyle\sum_{\gamma}|0\rangle\frac{\langle
0|V_{a}+V_{b}|\gamma\rangle\langle\gamma|V_{a}+V_{b}|0\rangle}{-\hbar\omega}\langle
0|\,,$ (6b) where $|0\rangle$ denotes the radiation vacuum and
$|\gamma\rangle=a_{\mathbf{k}\lambda}^{\dagger}|0\rangle$. The self-
interaction terms containing $\langle
0|V_{a}|\gamma\rangle\langle\gamma|V_{a}|0\rangle$ or $\langle
0|V_{b}|\gamma\rangle\langle\gamma|V_{b}|0\rangle$ are irrelevant and thus
discarded. The second-order interaction between the two particles is given by
$\tilde{H}_{2}=\sum_{\gamma}|0\rangle\frac{\langle
0|V_{a}|\gamma\rangle\langle\gamma|V_{b}|0\rangle+\mathrm{h.c.}}{-\hbar\omega}\langle
0|\,.$ (6c)

The evaluation of the matrix elements in Eq. (6c) yields

$\displaystyle\langle 0|V_{a}|\gamma\rangle$ $\displaystyle=$
$\displaystyle-\frac{e}{mc}\alpha_{\mathbf{k}}u_{\mathbf{k}}(\mathbf{x}_{a})(\hat{e}_{\lambda}\cdot\mathbf{p})e^{-i\omega
t}\,,$ (7a) $\displaystyle\langle\gamma|V_{b}|0\rangle$ $\displaystyle=$
$\displaystyle
i\frac{\Phi}{4\pi}k\alpha_{\mathbf{k}}u_{\mathbf{k}}^{*}(\mathbf{x}_{b})(\hat{z}\cdot\hat{n}_{\lambda})e^{i\omega
t}\,,$ (7b)

and we obtain

$\tilde{H}_{2}=\frac{ie\Phi}{2mc}F(\mathbf{x}_{a}-\mathbf{x}_{b})+\mathrm{h.c.}\,,$
(8a) where $F$ is a function of the relative position of the two objects given
by $F(\mathbf{x})=\frac{1}{4\pi^{2}}\int
d^{2}\mathbf{k}\frac{1}{k}e^{i\mathbf{k}\cdot\mathbf{x}}\sum_{\lambda}(\hat{e}_{\lambda}\cdot\mathbf{p})(\hat{z}\cdot\hat{n}_{\lambda})\,.$
(8b)

Applying the condition $\hat{e}_{\lambda}\cdot\mathbf{k}=0$ in the Coulomb
gauge, we find

$(\hat{e}_{\lambda}\cdot\mathbf{p})(\hat{z}\cdot\hat{n}_{\lambda})=\hat{\phi}\cdot\mathbf{p}\,,$

where $\hat{\phi}$ is the angular unit vector in the space of $\mathbf{k}$.
Finally we can derive $\tilde{H}_{2}$ as

$\tilde{H}_{2}=-\frac{e}{mc}\mathbf{p}\cdot\mathbf{a}\,,$ (9a) where the
“effective vector potential” $\mathbf{a}$ is given by
$\mathbf{a}(\mathbf{x})=\frac{\Phi}{2\pi|\mathbf{x}|}\hat{\theta}$ (9b)

($\hat{\theta}$ denotes the azimuthal unit vector of
$\mathbf{x}\equiv\mathbf{x}_{a}-\mathbf{x}_{b}$). This yields the expression
of the effective Hamiltonian of Eq. (6)

$\tilde{H}=\frac{1}{2m}\left(\mathbf{p}-\frac{e}{c}\mathbf{a}\right)^{2}\,,$
(10)

where terms independent of the interaction between the the charge and the
fluxon are omitted.

The expression of the “effective vector potential” in Eq. (9b) was previously
obtained by adopting the QED approach Santos and Gonzalo (1999); Lee and Choi
(2004), and it has recently been addressed in the context of the locality of
the interaction Marletto and Vedral (2020); Saldanha (2021). Marletto and
Vedral Marletto and Vedral (2020) and Saldanha Saldanha (2021) claimed that
they had shown the locality of the interaction with particular selection of
gauges (Coulomb and the Lorenz gauges, respectively). However, this claim is
incomplete without an explicit derivation of the gauge invariance. This is
clear if we compare it to the semiclassical approach where the vector
potential $\mathbf{A}_{\Phi}$ produced by a magnetic flux possesses some
degree of choosing the gauge with the constraint $\oint\mathbf{A}_{\Phi}\cdot
d\mathbf{x}=\Phi$. The gauge invariance of $\mathbf{a}$ leads to a notable
consequence in real experiments. Without the gauge invariant $\mathbf{a}$ we
cannot predict the value of the local phase shift in a certain experimental
arrangement Kang (2017), namely, the AB effect without an AB loop.

Gauge invariance of the interaction Hamiltonian and the local phase
measurement-. Consider a gauge transformation of the four-potential
$A^{\mu}\rightarrow A^{\prime\mu}=A^{\mu}+\partial^{\mu}\Lambda$ with an
arbitrary single-valued scalar function $\Lambda=\Lambda(\mathbf{x},t)$. The
vector potential $\mathbf{A}$ in Eq. (4) is then transformed to

$\mathbf{A}^{\prime}=\mathbf{A}+\nabla\Lambda\,,$ (11a) where $\Lambda$ can be
expanded as
$\Lambda(\mathbf{x},t)=\sum_{\mathbf{k},\lambda}\left[\gamma_{\mathbf{k}}u_{\mathbf{k}}(\mathbf{x})a_{\mathbf{k}\lambda}e^{-i\omega
t}+\gamma_{\mathbf{k}}^{*}u_{\mathbf{k}}^{*}(\mathbf{x})a_{\mathbf{k}\lambda}^{\dagger}e^{i\omega
t}\right]\,,$ (11b)

with an arbitrary $\mathbf{k}$-dependent coefficient $\gamma_{\mathbf{k}}$.

The additional contribution to $\tilde{H}_{2}$ of Eq. (6c) may be produced by
$\partial^{\mu}\Lambda$ in the gauge transform. First, scalar potential
$A^{0}$ is nonzero unlike in the Coulomb gauge. However, it does not couple to
the motion of charge $\mathbf{p}$ and is irrelevant to the interaction between
$q$ and $\Phi$. Second, the spatial part $\nabla\Lambda$ generates an
additional term in $\tilde{H}_{2}$, given by

$\delta\tilde{H}_{2}=\sum_{\gamma}|0\rangle\frac{\langle 0|\delta
V_{a}|\gamma\rangle\langle\gamma|V_{b}|0\rangle+\mathrm{h.c.}}{E_{0}-E_{\gamma}}\langle
0|\,,$ (12a) where $\delta V_{a}=-\frac{e}{mc}\nabla\Lambda\cdot\mathbf{p}\,.$
(12b) $\nabla\Lambda$ consists of the longitudinal modes (parallel to
$\mathbf{k}$), and we obtain $\langle 0|\delta
V_{a}|\gamma\rangle=-\frac{ie}{mc}k\gamma_{\mathbf{k}}u_{\mathbf{k}}(\mathbf{x}_{a})(\hat{\mathbf{k}}\cdot\mathbf{p})e^{-i\omega
t}.$ (12c)

However, this term does not couple to $\langle\gamma|V_{b}|0\rangle$, because
the latter contains only transverse components (see Eq. (7b)). Therefore we
conclude that $\delta\tilde{H}_{2}=0$. The second order interaction
$\tilde{H}_{2}$ is gauge invariant.

This result of deriving gauge invariance in $\tilde{H}_{2}$ is remarkable.
First, the gauge-invariant effective vector potential ($\mathbf{a}$ in Eq.
(9b)) is in sharp contrast with the semiclassical counterpart. In the
semiclassical approach, a charged particle under an external magnetic field is
described by the Hamiltonian

$H=\frac{1}{2m}\left(\mathbf{p}-\frac{e}{c}\mathbf{A}_{\Phi}\right)^{2}\,,$
(13)

where the vector potential $\mathbf{A}_{\Phi}$, generated by $\Phi$, can be
transformed to another function
$\mathbf{A}_{\Phi}\rightarrow\mathbf{A}_{\Phi}+\nabla\chi(\mathbf{x})$. The
constraint in $\mathbf{A}_{\Phi}$ is not local: $\oint\mathbf{A}_{\Phi}\cdot
d\mathbf{r}=\Phi$.

Second, the gauge-invariant $\tilde{H}$ provides the unambiguous prediction of
the local phase that can be measured in a certain type of experimental
arrangement Kang (2017). The Hamiltonian (13) with semiclassical
$\mathbf{A}_{\Phi}$ fails to make this prediction. Here, we briefly review the
essential feature of the local phase measurement experiment (For the details,
see Ref. Kang, 2017). Its schematic setup consists of two independent
superconducting leads ($S_{1}$ and $S_{2}$), which are tunnel-coupled to a
common normal electrode ($N$) (Fig. 2). The two superconducting electrodes are
biased with identical voltages below the superconducting gap. The electrical
current flows to the normal metallic output via Andreev reflection (AR)
Andreev (1964), in which a Cooper pair is converted to two normal electrons in
$N$. Interference is caused by the indistinguishability of the two different
AR processes ($S_{1}$ to $N$ or $S_{2}$ to $N$), and the phase shift produced
by the external flux is

$\phi_{\mathrm{loc}}=\frac{e^{*}}{\hbar c}\int_{C}\mathbf{a}\cdot
d\mathbf{x}=\frac{e^{*}\Phi}{2\pi\hbar c}\Delta\theta\,,$ (14)

where the integration is considered along path $C$, as shown in Fig. 2. The
effective charge $e^{*}=2e$ corresponds to the charge of a Cooper pair, and
$\Delta\theta$ is the angle formed in the geometry of the system. The
superconducting state is associated with gauge symmetry breaking, and the
Cooper pairs satisfy the bosonic statistics. Therefore, the charge
conservation and the Fermionic superselection rule do not prevent the
interference in this setup (contrary to the argument in Ref. Marletto and
Vedral, 2020).

Third, this value of the local phase (Eq. (14)) is equivalent to the result
predicted by the semiclassical local field interaction (LFI) approach Kang
(2017). As mentioned above, this cannot be achieved based on the semiclassical
vector potential (Eq. (13)). With $\mathbf{A}_{\Phi}$, the local phase is
given by $\phi_{\mathrm{loc}}=(e^{*}/\hbar c)\int_{C}\mathbf{A}_{\Phi}\cdot
d\mathbf{x}$, and it is not invariant under the gauge transformation
$\mathbf{A}_{\Phi}\rightarrow\mathbf{A}_{\Phi}+\nabla\chi$. This implies that
the potential-based semiclassical approach fails to predict the value of
$\phi_{\mathrm{loc}}$.

We need to address about how the ambiguity of the charge-flux interaction
could be eliminated in the QED approach (represented by the effective vector
potential $\mathbf{a}$ in Eq. (9b)). The key point is that the semiclassical
Hamiltonian of Eq. (13) does not include the information regarding the local
configuration of the system. Owing to the freedom of selecting a gauge in
$\mathbf{A}_{\Phi}$, the same configuration of the flux can be described by a
different function for $\mathbf{A}_{\Phi}$; furthermore, different locations
of the flux may be expressed by the same $\mathbf{A}_{\Phi}$. In contrast,
transformation of the gauge field (Eq. (11)) in the QED approach does not
change the interaction Hamiltonian (Eq. (9a)). Its difference from the
semiclassical vector potential is twofold. First, the local configuration of
the charge and flux is well specified in the QED approach. Second,
$\nabla\Lambda$ in the transformation of $\mathbf{A}$ (Eq. (11)) contains only
the longitudinal modes in the gauge field. The longitudinal modes do not
contribute to the interaction because the magnetic flux is coupled to the
transverse modes.

It was recently proposed that the local phase $\phi_{\mathrm{loc}}$ can be
predicted in the QED approach based on the particular choices of gauge
Marletto and Vedral (2020); Saldanha (2021). As described above, this claim
should be supported by an explicit demonstration of the gauge invariance of
$\mathbf{a}$. This is evident if we compare it to the semiclassical vector
potential $\mathbf{A}_{\Phi}$ in Eq. (13), where such a gauge invariance is
not satisfied. We have explicitly shown the gauge invariance of the effective
vector potential and confirmed that the QED approach eliminates the ambiguity
in the interaction strength of the charge and flux.

Scalar photons and the scalar Aharonov–Bohm effect-. The gauge invariance of
the effective vector potential $\mathbf{a}$ raises a question whether this
invariance is universal for any type of the electromagnetic interaction
mediated by virtual photons. In the following, we show that this is not the
case if the scalar photons come into play. Consider a stationary charge ($e$)
under external charge distribution ($\rho$) in the ideal force-free condition,
as shown in Fig. 3. This condition can be achieved in real experiments
Aharonov and Bohm (1959); Kim and Kang (2018), though it has never been
realized. The generation of external potential with a vanishing electric field
requires an appropriate distribution of charge density
$\rho(\mathbf{x}^{\prime})$. In practice, this condition is achieved in the
Faraday cage. The interaction between $e$ and $\rho$ is mediated by virtual
photons, and the system is described by the Hamiltonian

$H=H_{0}+eV(\mathbf{x},t)+\int\rho(\mathbf{x}^{\prime})V(\mathbf{x}^{\prime},t)\,,$
(15a) where $H_{0}$ represents the noninteracting part of $e$, $\rho$, and the
electromagnetic vacuum, respectively. The charges interact with the scalar
potential $V$ (time component of $A_{\mu}$) of the radiation
$V(\mathbf{x},t)=A_{0}(\mathbf{x},t)=\sum_{\mathbf{k}}\alpha_{\mathbf{k}}\left[u_{\mathbf{k}}(\mathbf{x})a_{\mathbf{k}0}e^{-i\omega
t}+u_{\mathbf{k}}^{*}(\mathbf{x})a_{\mathbf{k}0}^{\dagger}e^{i\omega
t}\right]\,,$ (15b)

where $a_{\mathbf{k}0}$ ($a_{\mathbf{k}0}^{\dagger}$) annihilates (creates) a
photon in the scalar mode. Adopting the same canonical transformation
technique used above (in obtaining Eq. (6)), we can derive the Coulomb
interaction mediated by the virtual scalar photons (see e.g., Ref. Zee, 2010),
as

$\tilde{H}=H_{0}+e\int\frac{\rho(\mathbf{x}^{\prime})}{|\mathbf{x}-\mathbf{x}^{\prime}|}d^{3}\mathbf{x}^{\prime}\,.$
(16)

This expression of the Coulomb interaction is derived from the exchange of
virtual photons in the scalar mode. It is obtained by imposing the Lorenz
gauge condition $\partial_{\mu}A^{\mu}=0$ and is not fully gauge-invariant.
For instance, if we choose the Coulomb gauge, the scalar and longitudinal
modes are absent. In this case, $V=0$ in Eq. (15b) and the Coulomb interaction
of Eq. (16) cannot be derived.

This implies that the question regarding the reality of $A^{\mu}$ is more
subtle in the quantum electrodynamics involving scalar modes. In any case,
there are two notable points: The scalar photons (i) can never be observed but
(ii) are indispensable as the mediator of the Coulomb interaction between two
separate charges.

Conclusion-. In the quantum electrodynamic approach to the AB effect, the
interaction between a charge and a magnetic flux is mediated by the exchange
of virtual photons. We have shown the gauge invariance of the charge-flux
interaction, which leads to an unambiguous prediction of the local phase
shift. We have shown that the problem is more subtle in the electric AB
effect. In contrast to the case of the magnetic AB, the scalar component of
the gauge field plays an essential role and cannot be gauged away in the
electric AB effect. This may indicate a significant role of the gauge field
beyond mathematical construction.

## References

  * Ehrenberg and Siday (1949) W. Ehrenberg and R. Siday, Proceedings of the Physical Society. Section B 62, 8 (1949).
  * Aharonov and Bohm (1959) Y. Aharonov and D. Bohm, Phys. Rev. 115, 485 (1959).
  * Kang (2017) K. Kang, J. Kor. Phys. Soc. 71, 565 (2017).
  * Kim and Kang (2018) Y.-W. Kim and K. Kang, New J. Phys. 20, 103046 (2018).
  * Kang (2013) K. Kang, arXiv:1308.2093 (2013).
  * Kang (2015) K. Kang, Phys. Rev. A 91, 052116 (2015).
  * Marletto and Vedral (2020) C. Marletto and V. Vedral, Phys. Rev. Lett. 125, 040401 (2020).
  * Saldanha (2021) P. L. Saldanha, Foundations of Physics 51, 1 (2021).
  * Santos and Gonzalo (1999) E. Santos and I. Gonzalo, Europhysics Letters 45, 418 (1999).
  * Lee and Choi (2004) M. Lee and M. Y. Choi, Journal of Physics A: Mathematical and General 37, 973 (2004); M. Y. Choi and M. Lee, Current Applied Physics 4, 267 (2004).
  * Andreev (1964) A. Andreev, Sov. Phys. JETP 19, 1228 (1964).
  * Zee (2010) A. Zee, _Quantum field theory in a nutshell_ (Princeton university press, 2010).

Figure 1: Charge ($e$) and fluxon ($\Phi$) in the vacuum. The mutual
interaction between the two objects is mediated by the vacuum radiation field
(represented by $\gamma$). Figure 2: Schematic setup of the Andreev
interferometer comprising two superconducting electrodes ($S_{1}$ and $S_{2}$)
in contact with a normal metal ($N$).  Figure 3: A charge ($e$) and the
external charge distribution (represented by the charge density $\rho$). The
scalar component of gauge field mediates the Coulomb interaction between $e$
and $\rho$.


# Consistency-Aware Anchor Pyramid Network for Crowd Localization

Xinyan Liu1 Guorong Li1 Yuankai Qi2
Zhenjun Han1 Qingming Huang1,3,4 Ming-Hsuan Yang5 Nicu Sebe6
1 University of Chinese Academy of Science Corresponding author. Beijing
China
2Australian Institute for Machine Learning  The University of Adelaide
3Key Lab of Intell. Info. Process  Inst. of Comput. Tech  CAS  Beijing  China
4Peng Cheng Laboratory  Shenzhen  China  5University of California  Merced
6University of Trento  Italy
<EMAIL_ADDRESS>{liguorong, hanzhj<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Crowd localization aims to predict the spatial position of humans in a crowd
scenario. We observe that the performance of existing methods is challenged
from two aspects: (i) ranking inconsistency between test and training phases;
and (ii) fixed anchor resolution may underfit or overfit crowd densities of
local regions. To address these problems, we design a supervision target
reassignment strategy for training to reduce ranking inconsistency and propose
an anchor pyramid scheme to adaptively determine the anchor density in each
image region. Extensive experimental results on three widely adopted datasets
(ShanghaiTech A&B, JHU-CROWD++, UCF-QNRF) demonstrate the favorable
performance against several state-of-the-art methods.

##  1 Introduction

Figure 1: Existing methods use a fixed number of evenly distributed anchors on
all images of a dataset, which leads to redundant or insufficient target
sparse or dense regions. We propose a pyramid anchor strategy to alleviate
this issue.

Crowd counting and localization aim to learn to count and localize crowds with
only point annotations. It has received increasing attention due to its broad
applications, such as traffic flow analysis and crowd abnormality detection.
Despite recent advances [38, 27, 5, 35, 14, 9, 37, 15, 6, 10, 36, 18] in this
field, it remains challenging due to the significant density variations in
crowded scenes.

Crowd localization is usually formulated as a regression problem. Existing
methods fall into two main groups according to their regression target:
density map regression and coordinate regression. The former predicts a
density map from an input image and then compares against ground truth density
maps generated by smoothing Gaussian Kernels [18, 30, 31, 7, 10]. The local
maximum pixels larger than a pre-defined threshold are considered as predicted
locations. Since the predicted density map is mainly optimized for counting,
it is usually too smooth to generate accurate locations.

Following the paradigm of anchor-based object detection [20, 21, 22, 11, 19,
13], coordinate regression methods utilize dense anchors to predict the
coordinates of objects or humans directly. Each anchor predicts a target
coordinate offset and a probability of being a target.

Although significant progress has been made, existing coordinate regression
methods suffer two limitations.

First, there is a ranking discrepancy during training and test. During
training, only top-K (K is the number of targets in a training image) anchors
are selected for gradient back-propagation, and the selection criteria are
based on both localization distance and classification confidence. However,
the top-K anchors during the test are selected merely based on classification,
as shown in Figure. 1(a). Such selection discrepancy may lead to some anchors
that should but are not used for network optimization. Second, as shown in
Figure. 1(b), existing methods adopt a fixed number of evenly distributed
anchors on all images in a dataset, which can be redundant in sparse target
regions and insufficient in dense target regions.

To address these problems, we propose a consistency-aware anchor pyramid
network for crowd localization. It consists of two main components:
Consistency-aware Target Rearrangement (CTR) and Adaptive Anchor Pyramid
Selection (AAPS). Specifically, the CTR module first selects two groups of
top-K anchors according to separate criteria: the first group is chosen
according to both conventional localization distance and classification
confidence as existing methods, and each of them is assigned a ground truth
annotation; the second group is chosen according to just classification
confidence to keep consistency with the test phase. Then we reassign the first
group’s targets to the second group’s anchors according to location distance,
and the union of these two groups of anchors is used for gradient back-
propagation. In such a way, the ranking discrepancy problem is alleviated.

Regarding the AAPS, we divide the input image into several grids. Multiple
anchor densities are used in each grid to predict target locations. At the
same time, we use another branch to predict the target number in each grid.
Then, in each grid, only the anchors belonging to a density that is most close
to the predicted target number are retained. Next, all the retained anchors
are used as candidates for computing the best matching with ground truth
annotations. In this way, we not only fulfill using different anchor densities
at different image regions but also vastly reduce the computation cost
compared to directly using all anchors for best-matching computation. The main
contributions of the paper are:

  * •

We propose a consistency-aware target rearrangement (CTR) strategy for anchor-
based crowd-counting methods, alleviating the ranking discrepancy problem
during training and testing.

  * •

We propose an adaptive anchor pyramid selection (AAPS) strategy to adaptively
determine the anchor density in each image region with only a slight
computation increase.

  * •

Extensive experiments on the ShanghaiTech, JHU-CROWD++, and UCF-QNRF benchmark
datasets demonstrate the effectiveness of the proposed method compared against
several state-of-the-art approaches.

##  2 Related Work

There are two main types of crowd localization methods: density map regression
and coordinate regression.

Density map regression. A density map is generated by applying smoothing
kernels on the point annotations. Existing density map regression methods
mainly focus on better extracting coordinates methods and generating clear
density maps. Rebera _et al_. [23] propose a hand-craft method to extract
coordinates from density maps. It involves selecting pixels whose confidence
of being a target point is higher than around pixels (peak density point),
filtering out low confidence points by a pre-defined threshold, and lastly,
using clustering methods, such as K-means, on the filtered points to generate
cluster center points. Those clustered center points are considered as
locations of objects. Haroon [5] _et al_. use cascade adaptive Gaussian
kernels to refine the density map to localization map. To extract coordinates
from the localization map, they apply a Dirac Delta function to pixels with
positive confidence. However, those post-refine methods assume that predicted
density maps have clear peak points and no noise, which is hard to guarantee
in practice. The method based on Bayesian loss [18] learns from the point
annotation by considering the maximum posterior probability. However, it still
requires a Gaussian kernel function to calculate the conditional probability
of different distances. The estimation of Gaussian kernel size depends on the
object’s density, and its accuracy will drop at the sparse region. DM-Count
[31] and D2CNet [1] decouple the counting task into two targets, total count
learning, and normalized probability map learning. DM-Count uses the optimal
transport method to measure the difference between the predicted density map
and the point annotation map. The D2CNet estimates possible object boundaries
to generate a probability map. Although it has better peak points for locating
individual objects, the probability maps need to be normalized to 0-1, thus
will lose the absolute density information.

Coordinate regression. Coordinate regression methods can be divided into two-
stage methods [24, 30] and end-to-end methods [3, 27, 23, 8]. Two-stage
methods usually predict more objects and then refine them by a dataset prior
information such as a pseudo bounding box.

LSC-CNN [24] first estimates pseudo-bounding boxes from the training sets
using minimal neighbor distance. The pseudo-bounding boxes are used to perform
NMS to filter out redundant predictions.

Generalized loss [30] applies radius rather than a pseudo-bounding box, which
is easier to estimate and more suitable for the localization of human heads.
However, the bounding box size or radius length is hard to estimate but
sensitive to the final performance in this method.

The end-to-end method can predict coordinates without needing other processes.
Javier Ribera [23] applies weighted Hausdorff distance as set distance metrics
and extends this distance into an average form to make it robust to outlier
points. The P2PNet [27] uses Hungarian matching to obtain the optimal one-to-
one match between the target and the predicted point and only learns the
matched GT points. Both of those methods face a problem: determining the
threshold for positive, as it is evident that a fixed threshold is poor for
crowd scenes. IIM [3] and LSC-FCN [8] use a binarize module to learn a
threshold map, but a time-consuming post-process is needed to transfer the
medium map to coordinates.

##  3 Method

Figure 2: The architecture of the proposed method, which contains two main
components: an adaptive anchor pyramid selection strategy (AAPS, Sec 3.1) and
a consistency-aware target rearrangement (CTR, Sec 3.2). AAPS consists of a
Locating Branch (LB) and an Anchor Mask generator (AMG). The input image is
evenly divided into grids and LB predicts target locations using different
anchor densities in each grid. Then, AMG predicts a target density for each
grid, which is used to guide the selection of anchor density. Anchors at
selected density are retained (others are discarded). All retained anchors
serve as target candidates. The proposed consistency-aware target reassignment
(CTR) module then finds the best matching between retained anchors and ground
truth annotations. Losses of matched anchors are used for gradient back-
propagation.

As shown in Figure. 2, besides the backbone, our model consists of two main
components: Adaptive Anchor Pyramid Selection(AAPS) and Consistency-aware
Target Rearrangement (CTR). Taking image features as input, AAPS first adopts
a counting head (CH) to obtain density maps, which are used to generate anchor
masks (AM). Combining the anchor distribution prior learned from the training
set and anchor mask, unevenly distributed anchors are generated to handle
various crowd densities efficiently. Then, the anchors, as well as anchor
masks and FPN features, are fed into locating heads used to regress points,
coordinate offsets, and probability. Finally, consistency-aware target
rearrangement (CTR) is used to evaluate the loss between predicted candidates
and ground truth points. As mentioned in [25], long-range feature interaction
is essential, so we adopt ConvNext-S [17] as the backbone.

###  3.1 Adaptive Anchor Pyramid Selection

Our AAPS consists of an anchor mask generator (AMG) and locating branch (LB).
AMG predicts density maps using FPN features and generates anchor masks ($AM$)
according to each feature grid’s predicted density values $d$. The anchor
masks are used to select anchor priors $A_{i}$ to determine the anchor density
in each image region adaptively. Using the generated anchors, anchor masks,
and features from FPN as input, LB predicts the coordinates and probability
for objects. We denote the grid size by $h*w$, and for a $H*W$ image, there
are $N_{g}$ grids, where $N_{g}=(H*W)/(n*w)$.

Unlike previous methods [27, 32, 28] that apply fixed numbers of anchors (or
proposals) in each grid, AAPS dynamically generates anchors based on the
density map to fit the crowd rate variety. Inspired by FPN, we design anchor
sets of different densities and select an adaptive anchor set for each grid
according to its density information.

Specifically, there are $K$ anchor prior sets represented as
$\\{A_{1},A_{2},\cdots,A_{K}\\}$, and anchor set $A_{i}$ contains $s_{i}$
anchors. If the predicted density value in grid $(u,v)$ is in
$[s_{i},s_{i+1})$, we select anchor sets $A_{i}$ for this grid. As a result,
we generate $K$ anchor masks $\\{AM_{1},AM_{2},\cdots,AM_{K}\\}$, the shape of
which is $(H*W)/(n*w)$, for coordinates prediction:

$AM_{i}(u,v)=\left\\{\begin{array}[]{cc}1&s_{i}<\mathbf{D}(u,v)\leq s_{i+1}\\\
0&\mbox{others}\end{array}\right.$ (1)

where $\mathbf{D}(u,v)$ denotes the predicted number of objects among grid
$(u,v)$.

The relative positions of anchors within a grid are learned by performing
K-means on ground truth point annotations in the training dataset to achieve
better priors for the network. Since we want to learn anchor priors for the
grid, we crop training images into the grid size (16x16) and run K-means on
the ground truth points in grids with Euclidean distance of points as the
distance metric. To discover priors for anchors of different densities, we
conduct K-means $K$ times by setting cluster number
$\in\\{s_{1},s_{2},\cdots,s_{K}\\}$. The cluster centers denoted as
$\\{ac_{i}^{1},ac_{i}^{2},...ac_{i}^{s_{i}}\\},i\in\\{1,\cdots,K\\}$ are used
as positions of anchors in anchor sets $AC_{i}$.

$AC_{i}(u,v)=\left\\{\begin{array}[]{cc}\\{ac_{i}^{1},ac_{i}^{2},...ac_{i}^{s_{i}}\\}&AM_{i}(u,v)==1\\\
&\mbox{others}\end{array}\right.$ (2)

To predict points based on anchor sets of different densities, we design $K$
heads in our locating branch. The $i^{th}$ head ($\mathbf{LH}_{i}(\cdot)$)
takes the $i^{th}$ pyramid level features from FPN, $AM_{i}$ and $A_{i}$ as
input, and output the probability for objects ($\mathcal{P}_{i}$) and its
relative coordinates offset ($\mathcal{O}_{i}$) to the corresponding anchor
points,

$\mathcal{P}_{i},\mathcal{O}_{i}=\mathbf{LH}_{i}(\mathbf{FPN}_{i}*AM_{i},A_{i})$
(3)

where $\mathbf{FPN}_{i}$ denotes the $i^{th}$ level FPN features.
$\mathcal{O}_{i}$ includes two channels, which are the relative offset to the
anchor points $AC_{i}$. The absolute coordinates $(\mathbf{x},\mathbf{y})$
prediction of anchor $A_{i}^{j}$ at grid $(u,v)$ in the image is calculated
by:

$\mathbf{x}_{u,v,i,j}=u+ac_{i}^{j}+\textbf{Sigmoid}(\mathcal{O}_{i}(u,v,0))H$
(4)
$\mathbf{y}_{u,v,i,j}=v+ac_{i}^{j}+\textbf{Sigmoid}(\mathcal{O}_{i}(u,v,1))W$
(5)

During the inference time, in grid $(u,v)$, the points with top
$\mathbf{D}(u,v)$ classification probability is denoted as the positive
prediction.

As the generation of anchor masks depends on the density maps, unlike popular
density map based crowd counting methods applying smoothing density map, which
is more accessible for learning but outputs decimal values, our counting head
is designed to predict integer numbers. To this end, we select point maps
rather than density maps as targets, which are too sharp to learn directly by
convolutions [38]. We observe that the popular loss for density maps are
pixel-wised, such as Mean-Squared Error (MSE) or Mean-Absolute Error (MAE),
which is too strict for tiny localization shift error. Thus when using point
maps as learning targets, it can only determine what is better prediction if
they are strictly matched to the point map. As there is no precisely defined
center on various appearances of human heads, the points can be annotated in a
region nearby heads. So it is hard for the network to distinguish one pixel as
positive and simultaneously predict its neighbor points as negative. Motivated
by this, we propose to use a CasCade Region Loss (CCRL) rather than pixel-wise
loss to learn a point map, which can be defined by,

$\mathcal{L}_{cnt}^{cas}=\sum_{r=0}^{t}\sum_{R\in
T_{i}}\underbrace{\dfrac{2^{r+1}}{H*W}}_{\text{size
reduction}}\underbrace{\dfrac{\mathbf{e}^{\mathcal{L}_{1}^{r-1}(R^{\prime})}}{\mathbf{e}^{\sum_{R^{\prime}\in
T_{i-1}}\hat{\mathcal{L}}_{1}^{r-1}(R^{\prime})}}}_{reweight}\hat{\mathcal{L}}_{1}^{r}(R)$
(6)

where $\hat{\mathcal{L}}_{1}^{r}(R)$ denotes the counting loss in region $R$
with size $2^{r}$, and can be calculated by

$\hat{\mathcal{L}}_{1}^{r}(R)=\dfrac{1}{B}\sum_{i=1}^{B}|\mathcal{D}^{i}_{R}-\hat{\mathbf{D}^{i}}_{R}|$
(7)

where $B$ is the batch size, and the ground truth of object numbers in the
region $R$ in cascade level $i$ are denoted as $\hat{\mathbf{D}^{i}}_{R}$. The
loss defined by Eq. (6) models the counting error from low resolution to high
resolution and thus reduces the impact of localization shift errors. In Eq.
(6), $\dfrac{2^{r+1}}{H*W}$ is used to make loss reduction by the number of
regions. If the loss calculated on the low-resolution region is small, then
the error in sub-regions within this region is more likely to be a
localization shift error. Therefore, we apply
$\dfrac{\mathbf{e}^{\mathcal{L}_{1}^{r-1}}}{\mathbf{e}^{\sum_{R\in
T_{i-1}}\hat{\mathcal{L}}_{1}^{r-1}}}$ to decrease the weight of
$\hat{\mathcal{L}}_{1}^{r}(R)$.

As the decimal number in $\mathbf{D}$ will be quantified to a round number, to
reduce the impact of rounding operation on the accuracy, we add a regular term
to encourage the output to be close to a round number by:

$\mathcal{L}_{cnt}^{reg}=\dfrac{1}{B}\sum|\left[\mathbf{D}(u,v)\right]-\mathcal{D}(u,v))|$
(8)

where $\left[\mathbf{D}(u,v))\right]$ denotes the round number of the
predicted density value in grid $(u,v)$. The cascade region loss is defined
as:

$\mathcal{L}_{cnt}=\mathcal{L}_{cnt}^{cas}+\mathcal{L}_{cnt}^{reg}.$ (9)

###  3.2 Consistency-aware Target Rearrangement

Figure 3: Traditional One-to-One match strategy (Left) and the proposed CTR
match strategy (right). Except for candidates selected by
$\mathcal{L}_{loc}$(dashed line), we add more candidates (denoted as blue
triangles) with high classification probability and associate them with GT
annotations using $\mathcal{L}_{dist}$ as metric (solid line).

Combing all the candidates predicted by our $K$ locating heads, we obtain the
candidates points $S=\cup_{i=1}^{K}\\{\mathcal{P}_{i},\mathcal{O}_{i}\\}$. For
simplicity, we denote
$S=\\{t^{i}=(x^{i},y^{i},p^{i})|t_{1},t_{2},\cdots,t_{N_{s}}\\}$, where
$p^{i}$ is the predicted probability. The number of these candidates $N_{S}$
is:

$N_{S}=\sum_{u=0}^{H/h}\sum_{v=0}^{W/w}\sum_{i=1}^{K}AM_{i}(u,v)s_{i}$ (10)

During training, in order to calculate training loss, the predicted candidate
points need to be assigned to ground truth coordinates and classification
scores. The training loss for locating heads consists of classification and
distance losses:

$\mathcal{L}_{loc}(S,G,\Omega)=\sum_{t^{i}\in
S}\mathcal{L}_{cls}(p^{i},\hat{p}^{i})+\mathcal{L}_{dist}(x^{i},y^{i},\hat{x}^{i},\hat{y}^{i})$
(11)

where $G$ denotes the sets of ground truth points, $\Omega$ denotes a match
between ground truth points and predicts candidates. $t^{i}$ is matched to
ground truth point $\hat{t}^{i}=\Omega(G,t^{i})$ under $\Omega$, and
$\hat{p}^{i}=1$ denotes that $t^{i}$ is assigned to a ground truth point, the
position of which is $\hat{x}^{i},\hat{y}^{i}$. We apply focal loss [11] as
the classification loss to balance the weight of foreground and background and
set parameter $\lambda=2,\alpha=0.25$:

$\mathcal{L}_{cls}(t^{i},\hat{t}^{i})=-\alpha
p^{i}(1-\hat{p}^{i})\lambda\log(\hat{p}^{i})+\hat{p}^{i}\lambda(1-p^{i})\log(1-\hat{p}^{i})$
(12)

and apply L2 loss as the distance loss:
$\mathcal{L}_{dist}(t^{i},\hat{t}^{i})=\sqrt{(x^{i}-\hat{x}^{i})^{2}+(y^{i}-\hat{y}^{i})^{2}}$.
First, like the assignment rule mentioned in [27, 29, 32], we apply training
loss as the cost function to find the optimal match using Hungarian Match. We
define $\Omega_{1}$ as the optimal match between $S$ and $G$:

$\Omega_{1}=\mathop{\arg\min}\limits_{\Omega}\mathcal{L}_{loc}(S,G,\Omega)$
(13)

We refereed the matched candidate points determined by $\Omega_{1}$ as dual
matched candidates $S_{1}$, because the cost metric considers both distance
loss and classification loss. Formally, $S_{1}$ is defined as,

$t^{i}\in S_{1}\leftarrow\hat{p}^{i}=1$ (14)

Applying Eq. (12) as the cost metric for the optimal match, we select a set of
candidates $S_{2}$ from $S$. Both of the size of $S_{1}$ and $S_{2}$ are
$\sum\hat{\mathbf{D}}$. We introduce $S^{\prime}\leftarrow S_{2}-(S_{2}\cap
S_{1})$. The candidates in $S^{\prime}$ have high classification probability
value and high $\mathcal{L}_{loc}$. These candidates will be selected during
the inference time, although they are not assigned to ground truth points. In
other words, candidates in $S^{\prime}$ are not optimized during training.

Procedure 1 Matching strategy for proxy candidates

$S,G$

$\Omega_{2}$

$\Omega_{1}\leftarrow\mathop{\arg\min}\limits_{\Omega}\mathcal{L}_{loc}(S,G,\Omega)$

$S_{1},S_{2},S^{\prime},G^{\prime}\leftarrow\\{\\},\\{\\},\\{\\},\\{\\}$

$p_{high}\leftarrow Topk(\mathcal{P},\sum\hat{\mathbf{D}})$

for $t\in S$ do

$x,y,p\leftarrow t$

$\hat{t}\leftarrow\Omega_{1}(G,t)$

$\hat{x},\hat{y},\hat{p}\leftarrow\hat{t}$

if $\hat{p}==1$ then

$Add(S1,t)$

end if

if $p>p_{high}$ then

$Add(S_{2},t)$

end if

end for

$S^{\prime}=S_{2}-(S_{1}\cap S_{2})$

$p_{low}\leftarrow Topk(\mathcal{P},S^{\prime}_{size})$

for $t\in S_{1}$ do

if $\hat{p}==1$ then

$Add(G^{\prime},\hat{t})$

end if

end for

$\Omega_{2}\leftarrow\text{argmin}_{\Omega}\mathcal{L}_{dist}(S^{\prime},G^{\prime},\Omega)$

As shown in Figure. 3, we design a reassignment rule to reduce the
inconsistency between training and inference. Under match $\Omega$, some
ground truth points are assigned to prediction in $S_{1}$ with a low
classification score. We denote those ground truth points $G^{\prime}$.

Those predictions are easy to be distinguished as negative predictions in
inference time. The size of $G^{\prime}$ is limited to the same as
$S^{\prime}$ by ranking the classification score to its corresponding
prediction in $S_{1}$. We assign $S^{\prime}$ as proxy candidates for
$G^{\prime}$. The metric apply to match $S^{\prime}$ and $G^{\prime}$ distance
cost $\mathcal{L}_{dist}$. Those proxy candidates are also optimized with
$\mathcal{L}_{dist}$ to encourage them to predict $G^{\prime}$. It should be
noted that those candidates are still negative samples to satisfy the one-to-
one label assignment strategy. The details of matching strategy $\Omega_{2}$
for $S^{\prime}$ and $G$ are demonstrated in Procedure 1. Then the loss
function can be defined by:

$\displaystyle eee$ $\displaystyle\mathcal{L}_{loc}(S,G)=\sum_{t^{i}\in
S}\mathcal{L}_{cls}(p^{i},\hat{p}^{i})+$ (15) $\displaystyle\sum_{t^{i}\in
S_{1}}\mathcal{L}_{dist}(x^{i},y^{i},\hat{x}^{i},\hat{y}^{i})+\sum_{t^{i}\in
S^{\prime}}\mathcal{L}_{dist}(x^{i},y^{i},\hat{x}^{i},\hat{y}^{i})$

##  4 Experimental Results

We evaluate our approach on three datasets: ShanghaiTech, JHU-CROWD++, and
UCF-QNRF. In this section, we first evaluate and compare our method with the
previous state-of-the-art approaches to these datasets. Then we present
ablation study results on JHU-CROWD++.

Methods | Backbone(flops) | STA | STB | UCF-QNRF | JHU-CROWD++
---|---|---|---|---|---
F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R
TinyFaces[4] | ResNet-101 | 57.3 | 43.1 | 85.5 | 71.1 | 64.7 | 79.0 | 49.4 | 36.3 | 77.3 | - | - | -
RAZLoc[12] | VGG-16 | 69.2 | 61.3 | 79.5 | 68.0 | 60.0 | 78.3 | 53.3 | 59.4 | 48.3 | - | - | -
LSC-CNN[24] | VGG-16 | 68.0 | 69.6 | 66.5 | 71.2 | 71.7 | 70.6 | 74.0 | 74.6 | 73.5 | 51.9 | 51.2 | 52.7
IIM[3] | HRNet-48 | 73.9 | 79.8 | 68.7 | 86.2 | 90.7 | 82.1 | 72.0 | 79.3 | 65.9 | 62.5 | 74.0 | 54.2
FIDT[10] | HRNet-48 | 77.6 | 77.0 | 78.1 | 83.5 | 83.2 | 83.9 | 78.9 | 82.2 | 75.9 | 74.7 | 74.7 | 74.7
DCST[2] | Swin | 74.5 | 77.2 | 72.1 | 86.0 | 88.8 | 83.3 | 72.4 | 77.1 | 68.2 | - | - | -
P2PNet[27] | VGG-16 | 74.7 | 73.0 | 76.5 | 82.7 | 81.1 | 84.4 | 74.6 | 72.6 | 76.7 | - | - | -
Proposed Methods | VGG-16 | 77.1 | 76.5 | 77.8 | 84.9 | 84.2 | 85.6 | 76.1 | 75.1 | 77.2 | 78.8 | 80.1 | 77.5
Proposed Methods | HRNet-48 | 80.3 | 79.8 | 80.7 | 87.0 | 85.6 | 88.4 | 79.3 | 77.3 | 81.3 | 80.7 | 82.0 | 79.4
Proposed Methods | ConvNext-S | 82.8 | 83.5 | 82.1 | 89.0 | 87.2 | 91.1 | 79.2 | 77.5 | 81.0 | 82.5 | 84.7 | 80.5
Table 1: Comparison to state-of-the-art methods on the four datasets, JHU-
CROWD++, UCF-QNRF, and ShanghaiTech Part A&B (STA, STB). From left to right
are F1-measure (F1), Precision (P), and Recall(R).

###  4.1 Experimental Setups

Implementation Details. During the training, the data augmentations we apply
include Color Jitter, Rand Scaling( from 0.5 to 1.5), Random crop( to
256x256), Random Rotate (from 0 to 45), and Horizontal Flip (the probability
of which is 0.5). During the inference time, the image size is padded to an
integer multiple of 64. For the large picture datasets, we limited the longest
edge within 1920 as P2PNet[27] and kept the original aspect ratio.

The output stride is set to 16 for prediction. We set $K=3$, and for all
mentioned datasets except QNRF, $s_{1},s_{2}$ and $s_{3}$ are set to 1, 4, and
8. For the UCF-QNRF dataset, $s_{1},s_{2}$ and $s_{3}$ are set to 1, 8, and
16. We apply AdamW as the optimizer, and the learning rate is set to 1e-4 with
a cosine scheduler. All the backbone is pretrained on ImageNet. The
experiments are conducted on eight NVIDIA RTX 3090 ( 24 GB GPU Memory).

Figure 4: Sample results of our methods on the JHU-CROWD++ test dataset (id
1795, 3178, 3560). Left to right: original images, predicted density maps, and
final outputs. In the density map, the color is deeper as the density is more
significant. In the third column, the color of the circles denotes the level
of the corresponding anchors at three levels, blue, yellow, and red, from the
crowd to sparse. The circle sizes represent the estimated scale estimated by
the method proposed in FIDT[10].

.

Datasets and Evaluation Metrics ShanghaiTech Part A consists of web images
that are very congested. Pictures in ShanghaiTech Part B are collected from
streets and are relatively sparse. Pictures in the UCF-QNRF dataset are more
challenging for their high resolution and much more absolute human counts
(1525 images with human counts from 49 to 12865). The JHU-CROWD++ dataset is a
more comprehensive dataset that covers multiple crowd rates and scenes (4250
images with human counts from 0 to 7286). Successful application of our
methods on those datasets demonstrates its capacity to handle multiple crowd
cases.

To evaluate the localization performance, we use Precision, Recall, and
F1-measure (P, R, F1 for short) for evaluation where the positive rates are
defined by $\sigma$ for the scale of objects [34]. We set $\sigma_{i}$ for the
JHU-CROWD++ dataset by $\sigma_{i}=\sqrt{h_{i}^{2}+w_{i}^{2}}$ based on the
height $h_{i}$ and width $w_{i}$ of the bound box annotation of points $i$.
For other datasets that have no bounding boxes annotations, we apply
$\sigma=8$ for ShanghaiTech A&B and an average metric for $\sigma$ from 1 to
100 for the UCF-QNRF dataset as FIDT[10].

To evaluate the counting performance, we apply Mean Absolute Error (MAE for
short) and Mean Squared Error (MSE for short) as metrics.

###  4.2 Evaluation Results

In this section, we demonstrate the performance of our methods on previous on
the four datasets. The main localization results compared with previous SOTAs
(including density map based methods (IIM[3], FIDT[10], DCST[2] ) and
regression based methods (Tiny Faces[4], RAZLoc[12], LSC-CNN[24], P2PNet[27])
are shown in Table 1.

It should be noted that the backbone of previous methods is not unified. Some
density map based methods (DCST[2], IIM[3], FIDT[10]) adopted strong backbones
(Swin or HRNet-48) for high-resolution predictions. For a fair comparison, we
test our method with different backbones, i.e., VGG-16, HRNet-48, and
ConvNext-S. It can be seen that under the same backbone, our methods achieve
the highest F1-measure on all three datasets. Using VGG-16 as the backbone,
our method increases the F1 measure by at least 2.1 (on the UCF-QNRF) compared
with previous SOTA methods. Although TinyFaces[4] achieved the highest recall
on STA, its precision is much lower than other methods. Using HRNet-48 as the
backbone, the F1-measure on the largest dataset (JHU-CROWD++) is improved by 6
(from 74.7 to 84.7). Actually, the flops of HRNet-48[33] and Swin-B[16] are
17.36G and 15.14G, respectively, whereas the flops of ConvNext-S[17] is 8.69G
(224x224). Our method with ConvNext-S as backbone achieves higher F1-measure
on ShanghaiTech Part (A&B) and JHU-CROWD++. Because the resolution of images
in UCF-QNRF is very high, HRNet-48 is more suitable for extracting image
features. Our method with HRNet-48 obtains the best F1.

Adopting VGG-16 as the backbone, we also compare the counting performance of
our method with SOTA crowd counting methods on those three datasets in Table
4. It can be seen that, among all crowd localization methods, we achieved the
top 2 counting performances comparable with crowd counting methods. It should
also be noted that the thresholds for the best counting performance of
FIDT[11] are not equal to that for the best localization performance. As a
result, for FIDT, the number of predicted points is not equal to the predicted
crowd number, while our method can ensure that the number of predicted points
is consistent with the predicted crowd count.

In Figure. 4, we show the visualization result on the JHU-CROWD++ test
datasets. In the density map, the deeper the color, the larger the density. In
the third column, the color of the circles in an image denotes the density
level of their corresponding anchors. There are three levels of anchors
denoted by blue, yellow, and red, from the crowd to sparse. It can be seen
that our methods can apply a suitable adaptive base guided by the predicted
density map.

###  4.3 Ablation Studies

AAPS | CTR | F1 | P | R | MAE | MSE
---|---|---|---|---|---|---
✓ | ✓ | 82.5(+6.3) | 84.7 | 80.5 | 64.2 | 258.7
✓ |  | 81.5(+5.3) | 85.2 | 78.2 | 68.2 | 297.4
| ✓ | 78.7(+2.5) | 81.1 | 76.4 | 62.7 | 255.0
|  | 76.2 | 78.6 | 73.9 | 66.2 | 299.6
Table 2: Ablation study of the main components. Levels | $s_{i}$ | F1 | P | R | MAE | MSE
---|---|---|---|---|---|---
| 1 | 72.0 | 85.9 | 62.0 | 64.7 | 271.92
| 4 | 78.7 | 81.1 | 76.4 | 62.7 | 255.0
1 | 8 | 76.0 | 65.1 | 91.2 | 65.6 | 274.2
| 1, 4, 8 | 82.5 | 84.7 | 80.5 | 64.2 | 258.7
3 | 1, 2, 4 | 76.2 | 77.5 | 74.6 | 63.2 | 222.5
| 1,4,8,10 | 82.1 | 84.3 | 80.0 | 63.8 | 235.7
| 1,4,8,12 | 81.8 | 83.7 | 80.0 | 62.8 | 240.3
| 1,4,8,14 | 81.1 | 83.7 | 78.7 | 64.1 | 254.4
4 | 1,4,8,16 | 81.2 | 84.5 | 78.9 | 65.0 | 224.0
Table 3: Ablation study of using different anchor pyramid levels. Methods | STA | STB | JHU-CROWD++ | UCF-QNRF
---|---|---|---|---
MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE
SAANet[25] | 51.7 | 80.1 | 6.1 | 9.4 | - | - | 79.3 | 137.3
ChfL[26] | 57.5 | 94.3 | 6.9 | 11.0 | 57.0 | 235.7 | 80.3 | 137.6
P2PNet[27] | 52.7 | 85.06 | 6.25 | 9.9 | - | - | 85.32 | 154.5
LSC-CNN[24] | 66.4 | 117.0 | 8.1 | 12.7 | - | - | 120.5 | 218.3
FIDT[10] | 57.0 | 103.4 | 6.9 | 11.8 | 66.6 | 253.6 | 89.0 | 153.5
Ours | 57.4 | 101.2 | 6.6 | 10.7 | 64.2 | 258.7 | 83.2 | 144.1
Table 4: Counting results on ShanghaiTech Part A and B, JHU-CROWD++ and UCF-QNRF. The last four methods output coordinates. K-means | count loss | F1 | P | R | MAE | MSE
---|---|---|---|---|---|---
✓ |  | 80.8 | 82.0 | 79.7 | 63.4 | 269.5
| ✓ | 82.1 | 83.8 | 80.5 | 63.4 | 260.6
✓ | ✓ | 82.5 | 84.7 | 80.5 | 64.2 | 258.7
Table 5: Ablations using different components of AAPS. Figure 5: Consistency
of the selected top K samples between training metric and counting metric.

Effectiveness of AAPS. In Table 2, we show the performance of with and without
AAPS. For the experiment without AAPS we apply the 4 anchors in a $16\times
16$ grid. Compared with using only one density of anchor, the anchor pyramid
design achieves 5.6 F1-measure improvements and can get 3.7 F1-measure
improvements with CTR.

Different Anchor Pyramid Level. In our methods, we learn anchor priors of
multiple density levels. Each density level is referred to as the anchor
pyramid level (denoted as $K$ in our method). We conduct experiments with
different $K$ to show the effect of anchor pyramid levels and report results
in Table 3. As shown in Table 3, when using only one pyramid level, when
setting the anchor number to 4, the performance is the best. Compared to the
first group and the second (or third) group, we can see that applying multiple
anchor levels achieves better locating performance than using only one level.
To verify whether more levels of pyramids are helpful, we design experiments
by adding extra convolution on the FPN to get a stride 64 outputs and set its
anchor numbers from 10 to 16. It can be seen that more levels of anchor do not
helpful. It may be because there are too few ground truth points assigned to
those anchors, making this branch not well trained.

Effectiveness of Our Counting Loss. To encourage round output of density map,
we design region cascade loss defined by Eq (6). To test its effectiveness, we
replace our counting loss in Eq (6) with conventional MSE loss with $\sigma$
adaptively learned using a method proposed by [38]. The result reported in
Table 5 shows that although using MSE loss can bring 0.8 MAE improvements
(63.4 v.s. 64.2), the localization performance of MSE is worse than ours. The
F1-measure decreases 1.7, demonstrating that the proposed cascade region loss
in Eq (6) can improve the localization performance for AAPS with a tiny
counting accuracy cost.

K-means V.S. Uniform Distribution. Besides anchor pyramids, AAPS also includes
K-means to learn prior to datasets. Instead of K-means, we use anchors evenly
distributed in the grid as [27]. The result is shown in Table 5. The anchor
prior obtained by K-means brings 0.4 F1-measure (82.1 v.s. 82.5) improvements.

Effectiveness of CTR. We replace our CTR with the matching strategy used in
P2PNet[27], and report the results in Table 2. It can be seen that the
proposed CTR reassignment rules improve the F1-measure by 2.5 points, and with
the help of AAPS, the performance can keep improving by 1 point. To give an
insight of the inconsistency problem in this task, we calculate the IOU of the
set $S_{1}$ and set $S_{2}$ mentioned in Algorithm 1. The result is shown in
Figure. 5. It can be seen that, as the number of epochs increases, the
training loss decreases, indicating that the model is convergent, but the IOU
of the set $S_{1}$ and set $S_{2}$ is always less is far less than one. This
means that there is a large inconsistency between the two sets. When applying
the proposed CTR, the consistency between these two sets are higher with about
0.1 IOU improvements.

##  5 Conclusions

In this paper, we proposed an anchor pyramid network for crowd localization to
predict the precise numbers of human coordinates. The main differences from
existing methods are the Consistency-Aware Target Rearrangement (CTR) and
Adaptive Anchor Generation(AAPS). CTR can reduce the ranking inconsistency
between matched candidate points in the training and inference stage. The key
component of CTR is to reassign ground truth points to the predicted
candidates with high classification scores but not selected by the traditional
label assignment rule. AAPS can adaptively determine the anchor density in
each image region and provide anchors with higher predicting ability. The
proposed simple framework achieves state-of-the-art localization performance
and satisfactory counting accuracy.

## References

  * [1] Jian Cheng, Haipeng Xiong, Zhiguo Cao, and Hao Lu.  Decoupled two-stage crowd counting and beyond.  IEEE TIP, 30:2862–2875, 2021.
  * [2] Junyu Gao, Maoguo Gong, and Xuelong Li.  Congested crowd instance localization with dilated convolutional swin transformer.  CoRR, abs/2108.00584, 2021.
  * [3] Junyu Gao, Tao Han, Yuan Yuan, and Qi Wang.  Learning independent instance maps for crowd localization.  arXiv preprint arXiv:2012.04164, 2020.
  * [4] Peiyun Hu and Deva Ramanan.  Finding tiny faces.  In CVPR, 2017.
  * [5] Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Al-Maadeed, Nasir Rajpoot, and Mubarak Shah.  Composition loss for counting, density map estimation and localization in dense crowds.  In ECCV, pages 532–546, 2018.
  * [6] Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Al-Maadeed, Nasir Rajpoot, and Mubarak Shah.  Composition loss for counting, density map estimation and localization in dense crowds.  In ECCV, pages 532–546, 2018.
  * [7] Kang Kim and Hee Seok Lee.  Probabilistic anchor assignment with iou prediction for object detection.  In ECCV, pages 355–371, 2020.
  * [8] Issam H Laradji, Negar Rostamzadeh, Pedro O Pinheiro, David Vazquez, and Mark Schmidt.  Where are the blobs: Counting by localization with point supervision.  In ECCV, pages 547–562, 2018.
  * [9] Yuhong Li, Xiaofan Zhang, and Deming Chen.  Csrnet: Dilated convolutional neural networks for understanding the highly congested scenes.  In CVPR, pages 1091–1100, 2018.
  * [10] Dingkang Liang, Wei Xu, Yingying Zhu, and Yu Zhou.  Focal inverse distance transform maps for crowd localization and counting in dense crowd.  arXiv preprint arXiv:2102.07925, 2021.
  * [11] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In ICCV, pages 2980–2988, 2017.
  * [12] Chenchen Liu, Xinyu Weng, and Yadong Mu.  Recurrent attentive zooming for joint crowd counting and precise localization.  In CVPR, pages 1217–1226, 2019.
  * [13] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg.  Ssd: Single shot multibox detector.  In ECCV, pages 21–37. Springer, 2016.
  * [14] Weizhe Liu, Mathieu Salzmann, and Pascal Fua.  Context-aware crowd counting.  In CVPR, pages 5099–5108, 2019.
  * [15] Weizhe Liu, Mathieu Salzmann, and Pascal Fua.  Counting people by estimating people flows.  IEEE TPAMI, 44(11):8151–8166, 2021.
  * [16] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In ICCV, 2021.
  * [17] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In CVPR, pages 11976–11986, 2022.
  * [18] Zhiheng Ma, Xing Wei, Xiaopeng Hong, and Yihong Gong.  Bayesian loss for crowd count estimation with point supervision.  In ICCV, pages 6142–6151, 2019.
  * [19] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In CVPR, pages 779–788, 2016.
  * [20] Joseph Redmon and Ali Farhadi.  YOLO9000: better, faster, stronger.  In CVPR, pages 6517–6525, 2017.
  * [21] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  arXiv preprint arXiv:1804.02767, 2018.
  * [22] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In NeurIPS, 2015.
  * [23] Javier Ribera, David Güera, Yuhao Chen, and Edward J. Delp.  Locating objects without bounding boxes.  In CVPR, pages 6472–6482, 2019.
  * [24] Deepak Babu Sam, Skand Vishwanath Peri, Mukuntha Narayanan Sundararaman, Amogh Kamath, and R. Venkatesh Babu.  Locate, size, and count: Accurately resolving people in dense crowds via detection.  IEEE TPAMI, 43(8):2739–2751, 2021.
  * [25] Siddharth Singh Savner and Vivek Kanhangad.  Crowdformer: Weakly-supervised crowd counting with improved generalizability.  arXiv preprint arXiv:2203.03768, 2022.
  * [26] Weibo Shu, Jia Wan, Kay Chen Tan, Sam Kwong, and Antoni B. Chan.  Crowd Counting in the Frequency Domain.  In CVPR, pages 19618–19627, 2022.
  * [27] Qingyu Song, Changan Wang, Zhengkai Jiang, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yang Wu.  Rethinking counting and localization in crowds: A purely point-based framework.  In ICCV, pages 3365–3374, 2021.
  * [28] Peize Sun, Yi Jiang, Enze Xie, Wenqi Shao, Zehuan Yuan, Changhu Wang, and Ping Luo.  What makes for end-to-end object detection?  In ICML, pages 9934–9944, 2021.
  * [29] Peize Sun, Yi Jiang, Enze Xie, Wenqi Shao, Zehuan Yuan, Changhu Wang, and Ping Luo.  What makes for end-to-end object detection?  In ICML, pages 9934–9944, 2021.
  * [30] Jia Wan, Ziquan Liu, and Antoni B Chan.  A generalized loss function for crowd counting and localization.  In CVPR, pages 1974–1983, 2021.
  * [31] Boyu Wang, Huidong Liu, Dimitris Samaras, and Minh Hoai Nguyen.  Distribution matching for crowd counting.  In NeurIPS, pages 1595–1607, 2020.
  * [32] Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, and Nanning Zheng.  End-to-end object detection with fully convolutional network.  In CVPR, pages 15849–15858, 2021.
  * [33] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al.  Deep high-resolution representation learning for visual recognition.  IEEE TPAMI, 43(10):3349–3364, 2020.
  * [34] Qi Wang, Junyu Gao, Wei Lin, and Xuelong Li.  Nwpu-crowd: A large-scale benchmark for crowd counting and localization.  IEEE TPAMI, 43(6):2141–2149, 2020.
  * [35] Yi Wang, Junhui Hou, Xinyu Hou, and Lap-Pui Chau.  A self-training approach for point-supervised object detection and counting in crowds.  IEEE TIP, 30:2876–2887, 2021.
  * [36] Yi Wang, Xinyu Hou, and Lap-Pui Chau.  Dense point prediction: A simple baseline for crowd counting and localization.  In ICME Workshops, 2021.
  * [37] Cong Zhang, Hongsheng Li, Xiaogang Wang, and Xiaokang Yang.  Cross-scene crowd counting via deep convolutional neural networks.  In CVPR, pages 833–841, 2015.
  * [38] Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma.  Single-image crowd counting via multi-column convolutional neural network.  In CVPR, pages 589–597, 2016.