# Cold Gas in Massive Galaxies as A Critical Test of Black Hole Feedback
Models

Jingjing Shi Kavli Institute for Astronomy and Astrophysics, Peking
University, Beijing 100871, China  Kavli Institute for the Physics and
Mathematics of the Universe (WPI), The University of Tokyo Institutes for
Advanced Study (UTIAS), The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa-
shi, Chiba, 277-8583, Japan  Yingjie Peng  Kavli Institute for Astronomy and
Astrophysics, Peking University, Beijing 100871, China  Benedikt Diemer
Department of Astronomy, University of Maryland, College Park, MD 20742, USA
Adam R. H. Stevens  International Centre for Radio Astronomy Research, The
University of Western Australia, Crawley, WA 6009, Australia  Australian
Research Council Centre of Excellence for All Sky Astrophysics in 3 Dimensions
(ASTRO 3D)  Annalisa Pillepich  Max-Planck-Institut für Astronomie, Königstuhl
17, 69117 Heidelberg, Germany  Alvio Renzini  INAF - Osservatorio Astronomico
di Padova, Vicolo dell’Osservatorio 5, I-35122 Padova, Italy  Jing Dou
Shanghai Astronomical Observatory, Chinese Academy of Sciences, 80 Nandan
Road, Shanghai 200030, China  University of Chinese Academy of Sciences, 19A
Yuquan Road, Beijing 100049, People’s Republic of China  Yu Gao  Department of
Astronomy, Xiamen University, Xiamen, Fujian 361005, China  Purple Mountain
Observatory & Key Laboratory for Radio Astronomy, Chinese Academy of Sciences,
10 Yuanhua Road, Nanjing 210033, PR China  Qiusheng Gu  School of Astronomy
and Space Science, Nanjing University, Nanjing 210093, China  Key Laboratory
of Modern Astronomy and Astrophysics (Nanjing University), Ministry of
Education, Nanjing 210093, China  Luis C. Ho  Kavli Institute for Astronomy
and Astrophysics, Peking University, Beijing 100871, China  Department of
Astronomy, School of Physics, Peking University, 5 Yiheyuan Road, Beijing
100871, P. R. China  Xu Kong  CAS Key Laboratory for Research in Galaxies and
Cosmology, Department of Astronomy, University of Science and Technology of
China, Hefei 230026, China  School of Astronomy and Space Sciences, University
of Science and Technology of China, Hefei 230026, China  Claudia del P. Lagos
International Centre for Radio Astronomy Research, The University of Western
Australia, Crawley, WA 6009, Australia  Australian Research Council Centre of
Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)  Di Li  CAS Key
Laboratory of FAST, National Astronomical Observatories, Chinese Academy of
Sciences, Beijing 100012, P. R. China  School of Astronomy and Space Science,
University of Chinese Academy of Sciences, Beijing 100049, P. R. China
Jiaxuan Li Kavli Institute for Astronomy and Astrophysics, Peking University,
Beijing 100871, China  Department of Astrophysical Sciences, 4 Ivy Lane,
Princeton University, Princeton, NJ 08544, USA  Roberto Maiolino  Cavendish
Laboratory, University of Cambridge, 19 J. J. Thomson Avenue, Cambridge CB3
0HE, UK  Kavli Institute for Cosmology, University of Cambridge, Madingley
Road, Cambridge CB3 0HA, UK  Department of Physics and Astronomy, University
College London, Gower Street, London WC1E 6BT, UK  Filippo Mannucci  Istituto
Nazionale di Astrofisica, Osservatorio Astrofisico di Arcetri, Largo Enrico
Fermi 5, I-50125 Firenze, Italy  Lizhi Xie  Tianjin Normal University,
Binshuixidao 393, Tianjin, China  Chengpeng Zhang  Kavli Institute for
Astronomy and Astrophysics, Peking University, Beijing 100871, China Jingjing
Shi & Yingjie Peng<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Black hole feedback has been widely implemented as the key recipe to quench
star formation in massive galaxies in modern semi-analytic models and
hydrodynamical simulations. As the theoretical details surrounding the
accretion and feedback of black holes continue to be refined, various feedback
models have been implemented across simulations, with notable differences in
their outcomes. Yet, most of these simulations have successfully reproduced
some observations, such as stellar mass function and star formation rate
density in the local Universe. We use the recent observation on the change of
neutral hydrogen gas mass (including both $\rm H_{2}$ and $\rm
H\,{\textsc{i}}$) with star formation rate of massive central disc galaxies as
a critical constraint of black hole feedback models across several
simulations. We find that the predictions of IllustrisTNG agree with the
observations much better than the other models tested in this work. This
favors IllustrisTNG’s treatment of active galactic nuclei - where kinetic
winds are driven by black holes at low accretion rates - as more plausible
amongst those we test. In turn, this also indirectly supports the idea that
the massive central disc galaxy population in the local Universe was likely
quenched by AGN feedback.

methods: numerical - galaxies: evolution - galaxies: star formation -
galaxies: ISM

specialdate--

##  1 Introduction

Baryons cool and form stars within the potential well of dark matter halos
(Rees & Ostriker, 1977; White & Rees, 1978; White & Frenk, 1991). However,
only $5\%$–$25\%$ of baryons within dark matter halos are converted into stars
by $z=0$ efficiently in most galaxies (Wechsler & Tinker, 2018; Dutton et al.,
2010; Zheng, Coil & Zehavi, 2007; Moster, Naab & White, 2018; Wang & Jing,
2010; Yang, Mo & van den Bosch, 2009; Yang et al., 2012; Kravtsov, Vikhlinin &
Meshcheryakov, 2018; Behroozi et al., 2019). Stellar feedback, such as
supernova (SN) explosions and stellar winds, can heat up and even eject the
gas in low-mass systems, reducing their star formation (SF) efficiency
(Larson, 1974; White & Rees, 1978; Silk, 2003; Springel & Hernquist, 2003).
However, to suppress the SF efficiency in massive systems where the potential
well is deeper, SN feedback alone is not enough (Benson et al., 2003). Active
galactic nuclear (AGN) feedback is invoked as a necessary mechanism to reduce
further SF activity in massive systems in galaxy formation and evolution
models (Silk & Rees 1998; Croton et al. 2006; Bower et al. 2006; Henriques et
al. 2015; Yuan & Narayan 2014; Yuan et al. 2015, 2018, see Somerville & Davé
2015 and Naab & Ostriker 2017 for a review on the current status of galaxy
formation and evolution models).

AGN feedback is launched in quite different ways in different theoretical
models of cosmological simulation and semi-analytic models (SAMs). In general,
the way that feedback is launched depends on the BH accretion rate: the high
accretion rate mode and low accretion rate mode. In SAMs (Croton et al., 2006;
Bower et al., 2006; Guo et al., 2011; Henriques et al., 2015), BHs continue to
accrete gas from the circumgalactic medium, the induced feedback injects
thermal energy into the gas in DM halo to effectively reduce the hot-gas
cooling rate. In Illustris (Vogelsberger et al., 2013; Torrey et al., 2014)
and IllustrisTNG (Weinberger et al., 2017; Pillepich et al., 2018b), when the
BH accretion rate is higher than a certain threshold, the thermal energy is
inserted into the surrounding gas within galaxies; when the BH accretion rate
is lower, a hot bubble is injected (in Illustris) or a certain amount of
kinetic energy is added to the gas surrounding the BH (so called ”kinetic
feedback”, in IllustrisTNG). In EAGLE, a single heating mode from BH is
launched, which nevertheless mimics the relatively quiescent ‘radio mode’ and
vigorous ‘quasar mode’ when the BH accretion rate is much smaller than or
similar to the Eddington rate (Schaye et al., 2015; Crain et al., 2015).
Despite the non-negligible differences among these implementations, they are
all adjusted to reproduce the $z=0$ stellar mass function and the stellar
mass–BH mass scaling relation to various levels of agreement. More
observations, especially those that are not implemented for model calibration,
are needed to constrain the AGN feedback models in cosmological simulations
and SAMs.

One promising constraint comes from investigating the gas content in galaxies.
AGN feedback can act either negatively or positively (Cresci & Maiolino,
2018). Negative AGN feedback is believed to suppress the SF in massive
galaxies in two ways: through prevention (Cresci et al., 2015) and ejection.
Preventative feedback is when the hot gas can not cool efficiently due to the
heating from AGN, while ejective feedback is when cold gas is pushed away from
the galaxy due to the wind produced by the accreting BH (Zinger et al., 2020).
The role of AGN feedback being preventive or ejective is not necessarily
directly tied to how the feedback is implemented in practice. For example, the
kinetic mode in IllustrisTNG can be also preventive, as kinetic energy can
also transform into thermal energy via shocks (Weinberger et al., 2017);
likewise, the thermal injection in EAGLE can produce gas ouflows (i.e. can be
ejective too) due to the induced pressure gradients. By contrast, and of
lesser importance, positive AGN feedback is when additional star formation is
induced by the compression of molecular gas in a galaxy’s disc (Silk, 2013) or
directly in the outflowing cold gas (Ishibashi & Fabian, 2012; Zubovas et al.,
2013; Maiolino et al., 2017; Gallagher et al., 2019). Understanding how AGN
feedback impacts galaxies’ gas and star formation properties can help us to
distinguish/constrain various implementations in these models (Terrazas et
al., 2020).

On average, passive galaxies unsurprisingly have less cold gas than star-
forming ones (Saintonge et al., 2016; Tacconi et al., 2018; Catinella et al.,
2018). However, when massive central disc galaxies ($10^{10.6}<M_{\star}/{\rm
M}_{\odot}<10^{11}$) are selected, i.e. focusing on internal quenching
mechanisms and excluding external environmental ones, Zhang et al. (2019) (Z19
hereafter) found that, as SFR decreases, their $\rm H\,{\textsc{i}}$ gas mass
remains surprisingly constant, but both $\rm H_{2}$ gas mass and $\rm H_{2}$
star formation efficiency decrease. Zhang et al. (2021) ((Z21 hereafter))
further show the change of gas content is also accompanied by the rapid
increase of stellar concentration index, bulge-to-total mass ratio, central
velocity dispersion and AGN frequency (which in Z19 are mostly LINERs). These
altogether suggest more massive black holes, and possibly stronger AGN
feedback, in quenching or quenched galaxies. These results are robust against
different SFR estimators, including the ${\rm H}{\alpha}$/$D_{4000}$-based SFR
(Brinchmann et al., 2004) with aperture correction by performing spectral
energy distribution (SED) fitting to the photometry outside the fiber, and
SFRs obtained from SED fitting of UV, optical, and mid-IR bands (Salim et al.,
2016; Salim, Boquien & Lee, 2018), see more detailed discussion in Z21. Since
cold gas is the fuel of star formation and can provide direct evidence of how
quenching may happen in galaxies, this observational result of massive central
disc galaxies can be used as a new constraint for current galaxy formation and
evolution simulations/models, especially for AGN feedback. In particular,
since $\rm H_{2}$ is mainly distributed in the inner stellar disc and $\rm
H\,{\textsc{i}}$ is usually located at larger radii, the observed $\rm
H\,{\textsc{i}}$ and $\rm H_{2}$ versus SFR relations as in Z19 potentially
put a strong observational constraint on the strength of AGN feedback and on
its “inside-out” nature, as it needs to clear out most of the $\rm H_{2}$ gas
in the inner disc while retaining much of the $\rm H\,{\textsc{i}}$ gas in the
outer disc.

In this work, we focus mainly on the prediction for the neutral hydrogen
gas–SFR relation of massive central discs produced by IllustrisTNG, EAGLE,
L-Galaxies, and Illustris. We show that IllustrisTNG agrees with the observed
gas–SFR relation better than the others. We then study the gas distribution
and quenching mechanism of the central discs in IllustrisTNG in more detail,
as it may give useful clues on the quenching process in the real Universe.

##  2 Data and methods

###  2.1 The IllustrisTNG simulation

The IllustrisTNG project (Springel et al., 2018; Marinacci et al., 2018;
Naiman et al., 2018; Pillepich et al., 2018a; Nelson et al., 2018) is a suite
of cosmological hydrodynamical simulations in a $\Lambda$CDM Universe run with
the moving-mesh code arepo (Springel, 2010). It contains simulations of three
different volumes, TNG50, TNG100, and TNG300 with respective box lengths of
$35\,h^{-1}\,\rm Mpc$, $75\,h^{-1}\,\rm Mpc$, and $205\,h^{-1}\,\rm Mpc$. In
this work, we use the publicly available TNG100 data as a good balance between
resolution and volume (Nelson et al., 2019b)111https://www.tng-project.org.
The numerical resolution of TNG100 is also closest to the test runs on which
the free parameters of the subgrid physics were calibrated, which results in
the best match with observational constraints (e.g. stellar mass function
etc.). The IllustrisTNG galaxy formation and evolution model includes gas
cooling and heating, star formation, stellar evolution and chemical
enrichment, SN feedback, BH growth, AGN feedback, and cosmic magnetic field
(see Weinberger et al. 2017; Pillepich et al. 2018b for more detailed
information on the models).

Galaxies in TNG100 are identified using the Friends-of-Friends (FOF) and
SUBFIND algorithm (Davis et al., 1985; Springel et al., 2001). The neutral gas
fraction of non-star-forming gas cells is calculated self-consistently in the
simulation by computing the cooling rate and photo-ionization rate due to the
UV background, while star-forming gas cells are modelled as a two-phase medium
following Springel & Hernquist (2003), where the hot phase gas is assumed
fully ionized and the cold phase gas fully neutral (see appendix A1 of Stevens
et al. 2019b for more information). In each gas cell, the molecular hydrogen
fraction can be obtained by post-processing using empirical, simulation-based,
or theoretical prescriptions. In this work, we use the molecular hydrogen gas
fraction calculated using five different models (Diemer et al., 2018): the
empirical model where the molecular gas fraction is found to be correlated
with the mid-plane pressure (Leroy et al. 2008, L08 hereafter), the high-
resolution chemical-network-inclusive simulations that produced calibrated
relations between molecular gas fraction and surface density, metallicity, and
UV field (Gnedin & Kravtsov 2011, GK11 hereafter; Gnedin & Draine 2014, GD14
hereafter), and analytical equilibrium models of molecular clouds with
detailed calculations of molecular hydrogen creation and destruction (Krumholz
2013, K13 hereafter; Sternberg et al. 2014, S14 hereafter). Galaxies are
rotated into a face-on position using the angular momentum of gas and stars,
and the $\rm H\,{\textsc{i}}$ and ${\rm H_{2}}$ fractions are computed using
the projected quantities (such as surface density, SFR surface density etc.)
That is, we use the ‘map’ methods of Diemer et al. (2018). We refer the
readers to Diemer et al. (2018, 2019); Stevens et al. (2019b, a, 2021) for
more details on the decomposition methods and comparison with observation
(also see Davé et al. 2020 for an additional comparison with other
hydrodynamic simulations).

In this work, $M_{\star}$ for each galaxy is calculated within the radius of
$2R_{\star}$, where $R_{\star}$ is the stellar half-mass radius. To
approximate the scale of the Arecibo beam width ($3.4^{\prime}$ at $21$ cm,
e.g. Jones et al. 2018) for the relevant galaxies in the ALFALFA survey, we
only count the $\rm H\,{\textsc{i}}$ gas within a fixed radius of $70$ kpc
(i.e. the corresponding physical scale of the beam witdth at $z=0.037$) as in
Diemer et al. (2019) and Bahé et al. (2016); but see Stevens et al. (2019b)
for a more precise method. We also test our results with fixed radii of $50$
kpc and $80$ kpc and see no significant variations of our results. We also
count the $\rm H_{2}$ gas within the radius of $70$ kpc and test the results
using the radius of $20$ kpc (which roughly corresponds to the IRAM aperture
for xCOLD GASS survey). Our results stay unchanged since $\rm H_{2}$ is mainly
located in the central region of galaxies. To make a direct comparison with
the observed trend of cold gas content versus SFR as in Z19 and Z21, we apply
the same selection criteria of massive central disc galaxies with
$10^{10.6}<M_{\star}/{\rm M}_{\odot}<10^{11}$. We use the $\kappa_{\rm rot}$
parameter to distinguish disc galaxies from elliptical galaxies, defined as
the ratio of the ordered rotation versus the total kinetic energy using
stellar particles (Sales et al., 2012; Rodriguez-Gomez et al., 2015). If
$\kappa_{\rm rot}>0.5$, the galaxy is rotation-dominated, disc-like; if
$\kappa_{\rm rot}<0.5$, the galaxy is dispersion-dominated, elliptical-like.
Figure A1 in Diemer et al. (2019) compared the elliptical galaxy fraction in
TNG100 based on $\kappa_{\rm rot}$ with the observed fraction given by Calette
et al. (2018), finding a good match between simulations and observations in
the stellar mass range studied in this work. In Section 3 and Appendix A we
give more discussion on the morphology selection. With the above selection,
our final sample in TNG100 includes $674$ centrals, of which $340$ are disc-
like. The SFRs are obtained by averaging the star formation over the past
$200\,{\rm Myr}$ within two different apertures: everything that are
gravitation-ally self-bounded within the SUBFIND identified subhalo, SFR(all
grav.), and that within $2R_{\star}$, SFR($<2R_{\star}$). We take SFR(all
grav.) as our default choice. In Appendix B, we also test our results with SFR
calculated using various apertures and time-scales, and find that the general
trends remain similar.

Due to the finite mass resolution in the simulation, galaxies’ SFRs suffer a
resolution limit (see Donnari et al. 2019 for a detailed discussion). For SFRs
averaged in the past $200\,{\rm Myr}$, this SFR resolution limit is
$10^{-2.46}\,{\rm M}_{\odot}\,{\rm yr^{-1}}$. For galaxies with SFRs below
this limit, we reassign each of them a random SFR between
$10^{-2.8}$–$10^{-2.5}\,{\rm M}_{\odot}\,{\rm yr^{-1}}$ so that they show up
in the log-scale scatter plots.

###  2.2 L-Galaxies, EAGLE, and Illustris-1

L-Galaxies 2020 is the newest version of the Munich semi-analytic galaxy
formation model (Henriques et al.,
2020)222https://lgalaxiespublicrelease.github.io/. This model is built upon
the subhalo merger trees from the Millennium and Millennium-II simulations
scaled to first-year Planck cosmology. The evolution of baryonic components
are traced in the model, including hot gas atmosphere, cold interstellar gas,
a gas reservoir ejected by winds, stars in the bulge, disc, and intracluster
light, and central supermassive BHs. Elemental abundances (including H) are
traced in a galactic chemical enrichment scheme introduced by Yates et al.
(2013). In this work, we select the central galaxies with
$10^{10.6}<M_{\star}/{\rm M}_{\odot}<10^{11}$ and disc-to-total stellar mass
ratios (D/T) $>0.5$ from the publicly available online
database333http://www.mpa-garching.mpg.de/millennium. The amount of hydrogen
in cold gas component gives us the neutral hydrogen mass.

The EAGLE simulation (Schaye et al., 2015; Crain et al., 2015; McAlpine et
al., 2016)444http://eagle.strw.leidenuniv.nl is a cosmological hydrodynamical
simulation run with a smoothed particle hydrodynamics’ (SPH) solver gadget3
with Planck cosmology. We use the largest-volume run, Ref-L100N1504, with box
length of $100$ Mpc. Halos and galaxies are identified using the FOF and
SUBFIND algorithms. Stochastic, isotropic heating of gas particles is
implemented for SF feedback ($\Delta T_{\rm SF}=10^{7.5}$K) and BH feedback
($\Delta T_{\rm BH}=10^{8.5}$K), where $\Delta T_{\rm SF}$ and $\Delta T_{\rm
BH}$ are the temperature jump of gas particles receiving feedback energy. The
neutral hydrogen fractions of gas particles are estimated in post-processing
(Lagos et al., 2015; Bahé et al., 2016; Crain et al., 2017), using the fitting
function of Rahmati et al. (2013), which is calibrated using TRAPHIC radiative
transfer simulations (Pawlik & Schaye, 2008). Davé et al. (2020) compared the
atomic and molecular hydrogen content of galaxies from IllustrisTNG, EAGLE,
and SIMBA (Davé et al., 2019), we refer the readers to there for more
information. Similar to TNG100, the massive central discs are selected using
$\kappa_{\rm rot}>0.5$. The neutral hydrogen mass are calculated within a
radius of $60$ kpc, however, the gas content stays roughly unchanged even with
a different radius choice of $40$ kpc.

The Illustris simulation is the precursor simulation of IllustrisTNG
(Vogelsberger et al., 2014; Sijacki et al., 2015; Genel et al., 2014; Nelson
et al., 2015)555https://www.illustris-project.org. In this work, we use
Illustris-1, which has the same initial condition, volume, and resolution as
TNG100. The main difference of Illustris, compared to IllustrisTNG, is the
treatment of BH accretion and feedback (as discussed in Section 1) and of
galactic winds. Illustris also lacks magnetohydrodynamics. The central discs
in Illustris-1 are selected in the same way as TNG100. The neutral hydrogen
gas mass of a galaxy is also summed up over the gas cells within the radius of
$70$ kpc.

An non-trivial caveat in the above models is that none of them attempts to
actually resolve the neutral phases of the ISM. Thus the predicted neutral
hydrogen fraction is not a real prediction based on the ISM chemistry, but a
modeling related to the effective equation of state function, where star
formation is initiated once the gas density $n>0.13$cm-3.

##  3 Results

###  3.1 Total neutral hydrogen in simulations and real galaxies

Figure 1:  Neutral hydrogen ($\rm H\,{\textsc{i}}+\rm H_{2}$) mass versus SFR
for central disc galaxies ($10^{10.6}<M_{\star}/{\rm M}_{\odot}<10^{11}$) in
TNG100 ($\kappa_{\rm rot}>0.5$), Illustris ($\kappa_{\rm rot}>0.5$),
L-Galaxies (D/T$>0.5$), and EAGLE ($\kappa_{\rm rot}>0.5$). The black and gray
lines show the observational results from Z19 for SFRs calculated using ${\rm
H\alpha}$ emission and $D_{4000}$ (corrected for the limited SDSS fiber
aperture) and UV+IR SEDs, respectively. The solid lines are the median
relations, with the 16th–84th percentile ranges shown with the shaded regions.
We also overplot lines of fixed star formation efficiency $\epsilon={\rm
SFR}/M_{\rm gas}$ for $1\,{\rm Gyr^{-1}}$, $0.1\,{\rm Gyr^{-1}}$, and
$0.01\,{\rm Gyr^{-1}}$.

Figure 1 shows the neutral hydrogen (${\rm H\,{\textsc{i}}+\rm H_{2}}$) gas
mass versus SFR for massive central disc galaxies ($10^{10.6}<M_{\star}/{\rm
M}_{\odot}<10^{11}$) in TNG100, EAGLE, Illustris, L-Galaxies, and observations
(Z19). In those four models, notably different black hole feedback models were
implemented.

First, all simulations except for EAGLE reproduce well the observed neutral
hydrogen gas content for the galaxies with largest SFRs, while EAGLE predicts
about $0.3$ dex less gas than observed. With SFR decreasing, only TNG100
roughly reproduces the Z19 observational trend (which is also shown by the
results in Figure 2), although the scatter around the median relation is not
very small. By contrast, galaxies in EAGLE, L-Galaxies, and Illustris are too
gas-poor at fixed SFR. When $10^{-0.5}<{\rm SFR}/({\rm M}_{\odot}{\rm
yr^{-1}})<10^{0.5}$, the neutral hydrogen gas decreases too fast in EAGLE,
L-Galaxies, and Illustris, compared to observations and TNG100. For ${\rm
SFR<10^{-0.5}\,{\rm M}_{\odot}\,{\rm yr^{-1}}}$, the neutral hydrogen gas
keeps decreasing rapidly in EAGLE, Illustris, and L-Galaxies, and becomes
significantly below the observed values (with either the H$\alpha$/$D_{4000}$
or UV+IR SFR estimator). We also explored an additional semi-analytic model,
Shark (Lagos et al., 2018), and found it behaves similarly to Illustris and
EAGLE (not shown here to avoid overcrowding of lines). Since the change in gas
as SFR decreases in simulations is mainly driven by the implemented AGN
feedback model for galaxies in this stellar mass range, the results in Figure
1 suggest that the AGN feedback during quenching might be too efficient in
heating/expelling gas from the galaxies in EAGLE, L-Galaxies, and Illustris.

There is a concern on the Z19 observation result that if different SFR
estimator (UV+IR instead of ${\rm H}_{\alpha}/D_{\rm 4000}$) is used, the
lowest SFRs of central disks are around log SFR=$-0.5$ whereas they go down to
$\sim$ $-1$ in the former case. So, these galaxies appear having more or less
progressed towards full quenching, depending on the adopted SFR diagnostics
(see Figure 1 in Cortese et al. 2020, hereafter C20). The effect of using
alternative SFR estimator has been discussed in detail in Z21. The general
trends of gas content and other galaxy properties with respect to SFR hold for
both SFR estimators. It should also be noted that “disc” galaxies are defined
in a very different way in Z19&Z21 (visual classification by Galaxy Zoo) and
C20 (a threshold in B/T). These two selections result in very different
populations. As shown in Z21, the visually defined disc galaxies with low SFR
often have massive bulges, and many of them will be classified as ellipticals
using B/T; while most S0s have been classified as ellipticals (or uncertains)
in Galaxy Zoo, but will be classified as discs by B/T. This different disc
selection leads to the fact that the UV+IR derived SFRs in Z19 do not extend
to the SFRs as low as in Cortese et al. (2020).

One issue in observation is that the SFRs derived from the SED fitting of UV,
optical and mid-IR bands (Salim et al., 2016; Salim, Boquien & Lee, 2018) used
in Z19 and Z21, and the UV+IR SFRs used in Cortese et al. (2020) are not
corrected for AGN contamination or by hot old stars. Z21 shows almost 100% of
the massive discs with the lowest SFRs (by either SFR estimator) host LINERs.
Thus the UV+IR SFRs are likely to be the upper limits and true SFRs could
reach lower SFRs. One could subtract off an AGN component by doing the SED
decomposition, but this is a very uncertain procedure. For these LINERs, Z19
and Z21 used the SFRs derived from $D_{\rm 4000}$, while such derivation is
calibrated with non-AGN galaxies (Brinchmann et al., 2004). Apparently, SFRs
from both estimators contain uncertainties at the low SFR end (dominated by
LINERs). Therefore, in our comparison analysis, we use the results derived
from both SFR estimators. As shown in Figure 1, the approximate agreement
between TNG100 and either SFR estimator holds.

###  3.2 $\rm H\,{\textsc{i}}$ and $\rm H_{2}$ contents (in TNG100)

####  3.2.1 $M_{\rm H\,{\textsc{i}}}$ and $M_{\rm H_{2}}$ versus SFR

Figure 2:  Atomic ($\rm H\,{\textsc{i}}$, blue) and molecular ($\rm H_{2}$,
red) hydrogen mass versus SFR in massive central disc galaxies in TNG100. The
squares show the median relation for a 200-Myr historical SFR that is summed
up over all the star particles gravitationally bound to a galaxy, i.e. to the
subhalo; the crosses are the median relation based on particles within
$2R_{\star}$. The shaded areas indicate the 16th–84th percentile ranges. The
first five panels show the results from five $\rm H\,{\textsc{i}}$/${\rm
H_{2}}$ decomposition models as discussed in Section 2. The small dots in the
first panel show the distribution of individual galaxies in this massive
central disc sample, and are not shown in other panels for clarity. The lime-
green ($\rm H\,{\textsc{i}}$) and gold ($\rm H_{2}$) lines are the sliding
medians from the observational results of Z19, where SFR is calculated using
aperture corrected ${\rm H\alpha}$/$D_{4000}$ emission from the SDSS fibre
spectra. The observational results using SFRs calculated from SED fitting of
UV+IR bands (i.e. Figure 6 of Z19) are overplotted. The gray vertical dashed
lines are the SFR resolution limits. We also overplot lines of fixed star
formation efficiency $\epsilon={\rm SFR}/M_{\rm gas}$ for $1\,{\rm Gyr^{-1}}$,
$0.1\,{\rm Gyr^{-1}}$, and $0.01\,{\rm Gyr^{-1}}$. In the last panel, the
light blue and red shaded areas show the 16th–84th percentile range of the
$\rm H\,{\textsc{i}}$ and $\rm H_{2}$ gas components from all decomposition
models for SFR(all grav.), and the darker areas indicate the range of the
median relations of the five models.

Given that TNG100 agrees the best with the observed neutral hydrogen mass–SFR
relation, in Figure 2 we show the $\rm H\,{\textsc{i}}$ and $\rm H_{2}$ gas
mass as a function of SFR for these massive central disc galaxies in TNG100,
for the five different $\rm H\,{\textsc{i}}$/${\rm H_{2}}$ decomposition
models as described in Section 2. In each panel, we show the results for SFRs
derived within the entire subhalo (square) and $2R_{\star}$ (cross) to check
the possible influence of varying apertures. It is evident from Figure 2 that
above the SFR resolution limits (the vertical dashed line in each panel), all
five of these $\rm H\,{\textsc{i}}$/${\rm H_{2}}$ decomposition models show
qualitatively similar trends. That is, during the quenching of these massive
central disc galaxies, as their SFR decreases, their ${\rm H_{2}}$ gas mass
and star formation efficiency drop but their $\rm H\,{\textsc{i}}$ gas mass
remains about constant. These trends are in general agreement with that
reported by Z19. The results from S14 and L08 models match the observations
remarkably well. Nevertheless, given the large systematic uncertainties in the
post-processing (Diemer et al., 2018), we avoid stressing the performance of
any individual model, but show the variation of all five models in the last
panel in Figure 2. It is important to notice that the $\rm H_{2}$ mass–SFR
relation is more affected by the choices of aperture size. If the
SFR($<2R_{\star}$) is used, in some $\rm H\,{\textsc{i}}/\rm H_{2}$
decomposition models, the $\rm H_{2}$ gas stays roughly unchanged with
decreasing SFR. This indicates the existence of a population of galaxies with
a good amount of SF outside $2R_{\star}$ in IllustrisTNG. The observed
aperture corrected $H_{\alpha}/D_{\rm 4000}$ SFR and UV+IR SFR represent more
the SFR over the whole galaxies including the outer region, thus we take
SFR(all grav.) as our default SFR choice. The relations of $\rm
H\,{\textsc{i}}$ and $\rm H_{2}$ mass with SFR agree with the observations
within model uncertainties. We further test our results with SFRs calculated
using varying apertures and time-scales, which is shown in Appendix B.

We note that for galaxies with SFR below the resolution limit, their median
$\rm H\,{\textsc{i}}$ gas mass is still large, with median $M_{\rm
H\,{\textsc{i}}}$ larger than $10^{8.6}\,{\rm M}_{\odot}$ when
SFR($<2R_{\star}$) is used and larger than $10^{9.6}\,{\rm M}_{\odot}$ when
SFR(all grav.) is used, but in general is lower than those with SFR above the
limit in all five models. As shown in the figure, observations do not extend
to such low SFR. Hence the existence of these fully quenched massive central
disc galaxies with a good amount but lower $\rm H\,{\textsc{i}}$ gas mass
becomes a prediction that can be observationally tested by future deeper $\rm
H\,{\textsc{i}}$ surveys.

Although morphology (i.e. disk galaxies) is measured in a different way in the
simulations compared to observations. In Appendix A, we show that the cold
gas–SFR relation in TNG100 and EAGLE is almost independent of morphology,
implying that the morphology parameter is irrelevant in interpreting the
trends in Figures 1 and 2 when compared with observations. However, the
independence (or weak dependence) of the $\rm H\,{\textsc{i}}$ mass on
morphology clearly contradicts observations. Using the same sample selection,
Z19 shows that massive central disc galaxies have an average $\rm
H\,{\textsc{i}}$ detection fraction of $>90\%$, while the $\rm
H\,{\textsc{i}}$ detection fraction of massive central elliptical galaxies is
only $<20\%$. AGN feedback by itself does not necessarily directly change the
morphology. Therefore, the disagreement between observation and simulation may
be due to the gas having not been properly processed during morphological
transformation in the simulations, such as when mergers and interactions
occur.

Star formation efficiency $\epsilon\equiv{\rm SFR}/M_{\rm gas}$ is a widely
used observable quantity to explore star formation and quenching in galaxies.
$\epsilon$ (of the total neutral hydrogen gas) in EAGLE and Illustris is
approximately constant at $0.5\,{\rm Gyr}^{-1}$, meaning quenching in massive
central galaxies in EAGLE and Illustris is primarily driven by the decrease of
cold gas. Similarly, in L-Galaxies, $\epsilon$ is approximately constant for
galaxies with SFR$>10^{-0.5\,}{\rm M}_{\odot}\,{\rm yr^{-1}}$ and then drops
rapidly for the most quenched galaxies. Hence quenching in L-Galaxies is first
driven by the decrease of the cold gas (with a constant $\epsilon$), and then
followed by the decrease of $\epsilon$. In TNG100, as SFR drops by $\sim 3$
dex, total neutral hydrogen gas mass drops by $\sim 0.5$ dex and hence
$\epsilon$ drops by $\sim 2.5$ dex. Therefore, quenching in TNG100 is mainly
driven by a decrease in the SF efficiency of the total neutral hydrogen gas.
It should be noted that this conclusion relates to the total neutral hydrogen
gas, and hence does not contradict the result regarding ${\rm H_{2}}$ gas
shown in Figure 2 (and also in Dou et al. 2021b, a) that quenching is driven
by the decrease of both ${\rm H_{2}}$ gas mass and ${\rm H_{2}}$ star
formation efficiency $\epsilon_{\rm H_{2}}$. It is straightforward to show
that

$\epsilon_{\rm H\,{\textsc{i}}+\rm H_{2}}=\epsilon_{\rm H_{2}}/(1+M_{\rm
H\,{\textsc{i}}}/M_{\rm H_{2}}).$ (1)

The decrease of $\epsilon_{\rm H_{2}}$ is smaller than that of $\epsilon_{\rm
H\,{\textsc{i}}+\rm H_{2}}$, and the exact amount depends on $M_{\rm
H\,{\textsc{i}}}/M_{\rm H_{2}}$. Meanwhile, we stress that in TNG100 the
instantaneous SFR in the modeling is calculated based on the local gas
density, not based on the ${\rm H_{2}}$ gas density (which is calculated in
post-processing). The dramatic decrease of the $\epsilon_{\rm
H\,{\textsc{i}}+\rm H_{2}}$ of $\sim 2.5$ dex during quenching in TNG100 is
caused by the decrease of the gas density likely as a consequence of black
hole feedback. Note that in TNG100, when the gas density falls below the
minimum threshold (i.e. $0.13\ {\rm cm^{-3}}$), star formation halts, by
construction in the simulation.

We suspect that the large differences in the cold gas content and $\epsilon$
during quenching are mainly caused by the different black hole feedback models
implemented in IllustrisTNG (kinetic winds), EAGLE (thermal feedback),
Illustris (thermal bubble model) and L-Galaxies (preventive feedback). In many
semi-analytic models, such as L-Galaxies, when the average heating rate from
BH feedback exceeds the gas cooling rate in the halo, cold gas accretion stops
and star formation is quenched. The decrease of the cold gas amount during
quenching in central discs in L-Galaxies is caused by the heating from the BH,
which reduces the cooling rate of the gas. In EAGLE, the BH feedback heats up
the ambient gas by a temperature increment of $10^{8.5}$ K; this thermal
injection creates a pressure gradient that can produce an outflow. However,
this feedback mode might be overly aggressive in heat up cold gas (Davé et
al., 2020; Mitchell et al., 2020), resulting in a deficiency of the neutral
hydrogen gas. The overall energy injected by BH feedback is similar in
IllustrisTNG and Illustris, but the implementation is very different
(Pillepich et al., 2018b). The thermal bubble model (Sijacki et al., 2007) in
Illustris expels very large amounts of gas from the halo in massive galaxies.
The improved kinetic winds (Weinberger et al., 2017) in IllustrisTNG still
remove large quantities of gas from galaxies during quenching (see related
discussions in Stevens et al., 2019b, 2021; Terrazas et al., 2020), but it
retains more gas on average, as shown in Figure 1 and Figure 3. We will
discuss the effect of BH feedback in IllustrisTNG on gas properties more
later.

####  3.2.2 Gas distribution profiles

Figure 3:  The surface density (face-on) and cumulative radial mass
distribution of neutral hydrogen (dashed lines), $\rm H\,{\textsc{i}}$ (shaded
area), and $\rm H_{2}$ (netted shaded region) for higher SFR (blue) and lower
SFR (red) central disc galaxies in TNG100, as indicated by the label. The
shaded regions indicate the range of median profiles of five $\rm
H\,{\textsc{i}}$/${\rm H_{2}}$ decomposition models. The vertical gray dashed
lines correspond to radii of $10$ kpc and $70$ kpc, and are plotted to guide
the eye.

Given the fact that TNG100 approximately reproduces the average cold gas
mass–SFR relation as shown in Figures 1 and 2, we further explore in TNG100
the cold gas distributions in galaxies with higher SFR(all
grav.)$>10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$ and lower SFR${}_{\rm
limit}<$SFR(all grav.)$<10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$, to understand
the cause of these relations. In the analysis here we exclude the galaxies
with the SFR below the resolution limit. Current observations in cold gas do
not extend to such low SFRs, hence the gas properties in this regime remain a
prediction from IllustrisTNG to be tested by future deeper surveys.

Figure 3 shows the median surface density profile and the median cumulative
radial distributions of the total neutral hydrogen gas (${\rm
H\,{\textsc{i}}+\rm H_{2}}$), ${\rm H\,{\textsc{i}}}$, and ${\rm H_{2}}$ for
galaxies with higher and lower SFRs in TNG100. The two vertical dashed lines
mark radii of $10$ kpc and $70$ kpc to guide the eye. Comparing the total
neutral hydrogen gas distribution of galaxies with higher and lower SFR, the
difference between the blue and red dashed line is $\sim 0.9$ dex ($\sim 0.7$
dex) in the inner regions at $\sim 10$ kpc, and is $0.2$ dex ($\sim 0.5$ dex)
in the outer regions at $\sim 70$ kpc for the surface density and cumulative
mass. The $\rm H_{2}$ surface density profile and cumulative mass profiles of
galaxies with SFR(all grav.)$>10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$ is
similar to that in $\rm H\,{\textsc{i}}$, in particular in the inner regions
within $30$ kpc. By contrast, the ${\rm H_{2}}$ gas mass of galaxies with
SFR(all grav.)$>10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$ is significantly lower
than their $\rm H\,{\textsc{i}}$ at all radii. The average $\rm
H\,{\textsc{i}}$ gas within $70$ kpc is $\sim 10^{10}{\rm M}_{\odot}$ for
lower SFR galaxies and $\sim 10^{9.65}\,{\rm M}_{\odot}$ for higher FSR
galaxies. This difference ($\sim 0.35$ dex) is smaller than the difference in
the total neutral hydrogen gas, which is $\sim 0.5$ dex at this radius. This
makes sense in light of the flat $M_{\rm H\,{\textsc{i}}}$–SFR relation shown
in Figure 2.

The significant decrease of total neutral hydrogen in the central regions of
lower SFR galaxies compared to that of the higher SFR galaxies is likely due
to the BH-driven kinetic feedback in IllustrisTNG (Terrazas et al., 2020). The
kinetic winds in IllustrisTNG are implemented by giving the gas immediately
surrounding the black hole a momentum kick in a random direction away from the
black hole (Weinberger et al., 2017). The kinetic wind this generates is
isotropic at the injection scales; it impacts the gas not only in the bi-
conical regions, but also in the disc. This kinetic wind quenches galaxy SF by
both pushing gas out of galaxies (i.e. ejective) and also heating up the gas
(i.e. preventive), more so in the central regions (Zinger et al., 2020;
Terrazas et al., 2020; Truong et al., 2020). Since $\rm H_{2}$ is mainly
located in galaxies’ centers, it is the most affected gas phase. While central
$\rm H\,{\textsc{i}}$ is also decreased, the majority of $\rm
H\,{\textsc{i}}$—which lies in the outskirts of galaxies—is less affected. The
deficiency of cold gas in the central regions of the lower SFR galaxies is in
agreement with Z19’s observations that the central regions of the disc
galaxies with lower SFR have ${\rm H\alpha/H\beta}$ close to the intrinsic
value of $3.1$, which indicates that there is little dust and gas there. Hence
Figure 3 is in fact an important prediction from TNG100, which can be
observationally tested with spatially-resolved $\rm H\,{\textsc{i}}$ and $\rm
H_{2}$ observations for both star-forming galaxies and those in the process of
being quenched.

Interestingly, at any given radius, the $\rm H_{2}$ gas mass difference
between lower SFR and higher SFR galaxies is significantly larger than their
$\rm H\,{\textsc{i}}$ gas mass difference, indicating that the quenching
mechanism, i.e. BH feedback, affects $\rm H_{2}$ more (at the same radius).
One important result here is that even in the inner regions of most lowest SFR
galaxies, there is a lower but still significant amount of cold gas.
Therefore, the quenching (i.e. decreasing of SFR) is mainly caused by the fact
that most of this neutral hydrogen gas is in the form of non-star-forming $\rm
H\,{\textsc{i}}$, not ${\rm H_{2}}$, which leads to the very low $\epsilon$ of
the total neutral hydrogen gas as discussed above. As shown in Diemer et al.
(2018), for all of the five $\rm H\,{\textsc{i}}/\rm H_{2}$ decomposition
models, the fraction of neutral hydrogen that is ${\rm H_{2}}$ depends steeply
on the gas density in a certain range (see also Morselli et al. 2020). The
removal of the high-density gas due to feedback can have a stronger effect on
$\rm H_{2}$ than $\rm H\,{\textsc{i}}$, leading to the low $\rm H_{2}$-to-$\rm
H\,{\textsc{i}}$ ratio, as shown here in Figure 3 and also in Figure 2. In
Appendix C, we show the stellar light and total gas distribution for two
typical central disc galaxies in TNG100, one with SFR(all
grav.)$>10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$ and one with SFR(all
grav.)$<10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$, to give a more direct view on
the above discussion.

##  4 Discussion and Summary

Recent observations indicate the ubiquitous existence of large regularly-
rotating $\rm H\,{\textsc{i}}$ discs in massive central disc galaxies, both in
those that are star-forming and in those in the process of being quenched
(Z19). This result has yet been used to calibrate the recipes in the
hydrodynamical simulations and semi-analytic models, and can therefore serve
as a new observational test of these simulations, in particular of their AGN
feedback models, as the existence of an $\rm H\,{\textsc{i}}$ disc at high
stellar masses requires the strength and modes of AGN feedback to be just
right. If feedback is too violent, $\rm H\,{\textsc{i}}$ discs will not
survive during quenching; if it is too weak, it will not be able to deplete
the $\rm H_{2}$ gas in the galaxy and to quench the star formation.

Among the simulations we tested, only TNG100 of the IllustrisTNG project
appears to approximately reproduce the trend in cold gas vs SFR as reported in
Z19 and some $\rm H_{2}/\rm H\,{\textsc{i}}$ decomposition method may even
reproduce the observed $\rm H_{2}$ and $\rm H\,{\textsc{i}}$ trends,
separately. This lends some support to the AGN feedback implementation in
TNG100, though other factors may also be at play. Given the coarse resolution
of cosmological hdyro-simulation relative to the high-resolution simulation of
individual galaxy that can resolve Bondi radius (Yuan et al., 2015, 2018) and
the sub-grid nature of the the AGN feedback models implemented, it is
difficult to tell too much details of the AGN feeback physics. However,
reproducing the observed gas content of galaxies of various properties will be
critical for future development in cosmological simulations. Also, the fact
that, none of the models tested in this work reproduce the morphology–HI gas
mass relation challenges the current models.

In IllustrisTNG, the kinetic winds drive outflows that push gas out of
galaxies (Nelson et al., 2019a) and increase the gas entropy (Zinger et al.,
2020), more so in the central regions, while the $\rm H\,{\textsc{i}}$ gas at
larger radii ($\gtrsim 30$kpc) is less affected. This explains the relatively
flat $M_{\rm H\,{\textsc{i}}}$–SFR relation shown in Figure 2. Meanwhile, the
kinetic feedback also leads to a sharp decrease in gas density, which then
leads to a sharp decrease in the fraction of molecular hydrogen computed in
the decomposition models (cf. Stevens et al., 2021). This explains the rapid
decrease of ${\rm H_{2}}$ gas mass and SFR during quenching. The simultaneous
role of being ejective and preventive for AGN feedback in IllustrisTNG also
implies that cold gas accretion during quenching is strangulated. These
results are also in qualitative agreement with those derived from the observed
SFR/$M_{\star}$–$M_{\rm H_{2}}/M_{\star}$–SFR/$M_{\rm H_{2}}$ scaling
relations (Dou et al., 2021b), in that to quench massive galaxies,
strangulation plus additional $\rm H_{2}$ gas removal (with a mass-loading
factor of about unity) are required (Dou et al., 2021a).

It should be noted that the existence of the observed regularly rotating $\rm
H\,{\textsc{i}}$ disk around the massive central disk galaxies that are
undergoing quenching process as in Z19 & Z21, can be explained by AGN feedback
as above. Alternatively, it can be caused by angular momentum quenching as
proposed in Peng & Renzini (2020) and Renzini (2020) that the inflowing gas
with excess angular momentum can settle on a stable outer neutral hydrogen
disk for a long timescale in the absence of perturbation. This is supported by
recent studies in simulations which found that a sufficiently high CGM angular
momentum is an important factor in keeping a galaxy quenched (Lu et al., 2021;
Wang et al., 2022). Although AGN feedback must be important to quench massive
galaxies in simulations (e.g. Su et al. 2019) and can well reproduce the
observed cold gas content during quenching as shown in this paper, it may not
be the case in the real Universe. We will further investigate both AGN and
non-AGN solutions to the quenching of massive galaxies in our future work.

##  5 Acknowledgments

We gratefully acknowledge the anonymous referee for comments and criticisms
that have improved the paper.

J.S., Y.P. and L.C.H. acknowledge National Science Foundation of China (NSFC)
Grant No. 12125301, 11773001, 12192222, 11721303, 11991052, and the National
Key R&D Program of China grant 2016YFA0400702.

J.S. acknowledges Martina Donnari for providing the SFRs of galaxies in
TNG100.

A.R.H.S. acknowledges receipt of the Jim Buckee Fellowship at ICRAR-UWA.

A.R. acknowledges support from the INAF/PRIN-SKA 2017 ‘ESKAPEHI’ grant.

We acknowledge the science research grants from the China Manned Space Project
with NO. CMS-CSST-2021-A04 and NO. CMS-CSST-2021-A07.

Y.G.’s work is partially supported by National Key Basic Research and
Development Program of China (grant No. 2017YFA0402704), National Natural
Science Foundation of China (NSFC, Nos. 12033004, and 11861131007), and
Chinese Academy of Sciences Key Research Program of Frontier Sciences (grant
No. QYZDJ-SSW-SLH008).

We acknowledge the Virgo Consortium for making their simulation data
available. The eagle simulations were performed using the DiRAC-2 facility at
Durham, managed by the ICC, and the PRACE facility Curie based in France at
TGCC, CEA, Bruyéres-le-Châtel.

## References

  * Bahé et al. (2016) Bahé Y. M. et al., 2016, MNRAS, 456, 1115
  * Behroozi et al. (2019) Behroozi P., Wechsler R. H., Hearin A. P., Conroy C., 2019, MNRAS, 488, 3143
  * Benson et al. (2003) Benson A. J., Bower R. G., Frenk C. S., Lacey C. G., Baugh C. M., Cole S., 2003, ApJ, 599, 38
  * Bower et al. (2006) Bower R. G., Benson A. J., Malbon R., Helly J. C., Frenk C. S., Baugh C. M., Cole S., Lacey C. G., 2006, MNRAS, 370, 645
  * Brinchmann et al. (2004) Brinchmann J., Charlot S., White S. D. M., Tremonti C., Kauffmann G., Heckman T., Brinkmann J., 2004, MNRAS, 351, 1151
  * Calette et al. (2018) Calette A. R., Avila-Reese V., Rodríguez-Puebla A., Hernández-Toledo H., Papastergis E., 2018, Rev. Mexicana Astron. Astrofis., 54, 443
  * Catinella et al. (2018) Catinella B. et al., 2018, MNRAS, 476, 875
  * Cortese et al. (2020) Cortese L., Catinella B., Cook R. H. W., Janowiecki S., 2020, MNRAS, 494, L42
  * Crain et al. (2017) Crain R. A. et al., 2017, MNRAS, 464, 4204
  * Crain et al. (2015) —, 2015, MNRAS, 450, 1937
  * Cresci et al. (2015) Cresci G. et al., 2015, ApJ, 799, 82
  * Cresci & Maiolino (2018) Cresci G., Maiolino R., 2018, Nature Astronomy, 2, 179
  * Croton et al. (2006) Croton D. J. et al., 2006, MNRAS, 365, 11
  * Davé et al. (2019) Davé R., Anglés-Alcázar D., Narayanan D., Li Q., Rafieferantsoa M. H., Appleby S., 2019, MNRAS, 486, 2827
  * Davé et al. (2020) Davé R., Crain R. A., Stevens A. R. H., Narayanan D., Saintonge A., Catinella B., Cortese L., 2020, MNRAS, 497, 146
  * Davis et al. (1985) Davis M., Efstathiou G., Frenk C. S., White S. D. M., 1985, ApJ, 292, 371
  * Diemer et al. (2018) Diemer B. et al., 2018, ApJS, 238, 33
  * Diemer et al. (2019) —, 2019, MNRAS, 487, 1529
  * Donnari et al. (2019) Donnari M. et al., 2019, MNRAS, 485, 4817
  * Dou et al. (2021a) Dou J. et al., 2021a, arXiv e-prints, arXiv:2104.12794
  * Dou et al. (2021b) —, 2021b, ApJ, 907, 114
  * Dutton et al. (2010) Dutton A. A., Conroy C., van den Bosch F. C., Prada F., More S., 2010, MNRAS, 407, 2
  * Gallagher et al. (2019) Gallagher R., Maiolino R., Belfiore F., Drory N., Riffel R., Riffel R. A., 2019, MNRAS, 485, 3409
  * Genel et al. (2014) Genel S. et al., 2014, MNRAS, 445, 175
  * Gnedin & Draine (2014) Gnedin N. Y., Draine B. T., 2014, ApJ, 795, 37
  * Gnedin & Kravtsov (2011) Gnedin N. Y., Kravtsov A. V., 2011, ApJ, 728, 88
  * Guo et al. (2011) Guo Q. et al., 2011, MNRAS, 413, 101
  * Henriques et al. (2015) Henriques B. M. B., White S. D. M., Thomas P. A., Angulo R., Guo Q., Lemson G., Springel V., Overzier R., 2015, MNRAS, 451, 2663
  * Henriques et al. (2020) Henriques B. M. B., Yates R. M., Fu J., Guo Q., Kauffmann G., Srisawat C., Thomas P. A., White S. D. M., 2020, MNRAS, 491, 5795
  * Ishibashi & Fabian (2012) Ishibashi W., Fabian A. C., 2012, MNRAS, 427, 2998
  * Jones et al. (2018) Jones M. G., Haynes M. P., Giovanelli R., Moorman C., 2018, MNRAS, 477, 2
  * Kravtsov, Vikhlinin & Meshcheryakov (2018) Kravtsov A. V., Vikhlinin A. A., Meshcheryakov A. V., 2018, Astronomy Letters, 44, 8
  * Krumholz (2013) Krumholz M. R., 2013, MNRAS, 436, 2747
  * Lagos et al. (2015) Lagos C. d. P. et al., 2015, MNRAS, 452, 3815
  * Lagos et al. (2018) Lagos C. d. P., Tobar R. J., Robotham A. S. G., Obreschkow D., Mitchell P. D., Power C., Elahi P. J., 2018, MNRAS, 481, 3573
  * Larson (1974) Larson R. B., 1974, MNRAS, 169, 229
  * Leroy et al. (2008) Leroy A. K., Walter F., Brinks E., Bigiel F., de Blok W. J. G., Madore B., Thornley M. D., 2008, AJ, 136, 2782
  * Lu et al. (2021) Lu S. et al., 2021, MNRAS
  * Maiolino et al. (2017) Maiolino R. et al., 2017, Nature, 544, 202
  * Marinacci et al. (2018) Marinacci F. et al., 2018, MNRAS, 480, 5113
  * McAlpine et al. (2016) McAlpine S. et al., 2016, Astronomy and Computing, 15, 72
  * Mitchell et al. (2020) Mitchell P. D., Schaye J., Bower R. G., Crain R. A., 2020, MNRAS, 494, 3971
  * Morselli et al. (2020) Morselli L. et al., 2020, MNRAS, 496, 4606
  * Moster, Naab & White (2018) Moster B. P., Naab T., White S. D. M., 2018, MNRAS, 477, 1822
  * Naab & Ostriker (2017) Naab T., Ostriker J. P., 2017, ARA&A, 55, 59
  * Naiman et al. (2018) Naiman J. P. et al., 2018, MNRAS, 477, 1206
  * Nelson et al. (2015) Nelson D. et al., 2015, Astronomy and Computing, 13, 12
  * Nelson et al. (2019a) —, 2019a, MNRAS, 490, 3234
  * Nelson et al. (2018) —, 2018, MNRAS, 475, 624
  * Nelson et al. (2019b) —, 2019b, Computational Astrophysics and Cosmology, 6, 2
  * Pawlik & Schaye (2008) Pawlik A. H., Schaye J., 2008, MNRAS, 389, 651
  * Peng & Renzini (2020) Peng Y.-j., Renzini A., 2020, MNRAS, 491, L51
  * Pillepich et al. (2018a) Pillepich A. et al., 2018a, MNRAS, 475, 648
  * Pillepich et al. (2018b) —, 2018b, MNRAS, 473, 4077
  * Rahmati et al. (2013) Rahmati A., Pawlik A. H., Raičević M., Schaye J., 2013, MNRAS, 430, 2427
  * Rees & Ostriker (1977) Rees M. J., Ostriker J. P., 1977, MNRAS, 179, 541
  * Renzini (2020) Renzini A., 2020, MNRAS, 495, L42
  * Rodriguez-Gomez et al. (2015) Rodriguez-Gomez V. et al., 2015, MNRAS, 449, 49
  * Saintonge et al. (2016) Saintonge A. et al., 2016, MNRAS, 462, 1749
  * Sales et al. (2012) Sales L. V., Navarro J. F., Theuns T., Schaye J., White S. D. M., Frenk C. S., Crain R. A., Dalla Vecchia C., 2012, MNRAS, 423, 1544
  * Salim, Boquien & Lee (2018) Salim S., Boquien M., Lee J. C., 2018, ApJ, 859, 11
  * Salim et al. (2016) Salim S. et al., 2016, ApJS, 227, 2
  * Schaye et al. (2015) Schaye J. et al., 2015, MNRAS, 446, 521
  * Sijacki et al. (2007) Sijacki D., Springel V., Di Matteo T., Hernquist L., 2007, MNRAS, 380, 877
  * Sijacki et al. (2015) Sijacki D., Vogelsberger M., Genel S., Springel V., Torrey P., Snyder G. F., Nelson D., Hernquist L., 2015, MNRAS, 452, 575
  * Silk (2003) Silk J., 2003, MNRAS, 343, 249
  * Silk (2013) —, 2013, ApJ, 772, 112
  * Silk & Rees (1998) Silk J., Rees M. J., 1998, A&A, 331, L1
  * Somerville & Davé (2015) Somerville R. S., Davé R., 2015, ARA&A, 53, 51
  * Speagle et al. (2014) Speagle J. S., Steinhardt C. L., Capak P. L., Silverman J. D., 2014, ApJS, 214, 15
  * Springel (2010) Springel V., 2010, MNRAS, 401, 791
  * Springel & Hernquist (2003) Springel V., Hernquist L., 2003, MNRAS, 339, 289
  * Springel et al. (2018) Springel V. et al., 2018, MNRAS, 475, 676
  * Springel et al. (2001) Springel V., White S. D. M., Tormen G., Kauffmann G., 2001, MNRAS, 328, 726
  * Sternberg et al. (2014) Sternberg A., Le Petit F., Roueff E., Le Bourlot J., 2014, ApJ, 790, 10
  * Stevens et al. (2019a) Stevens A. R. H., Diemer B., Lagos C. d. P., Nelson D., Obreschkow D., Wang J., Marinacci F., 2019a, MNRAS, 490, 96
  * Stevens et al. (2019b) Stevens A. R. H. et al., 2019b, MNRAS, 483, 5334
  * Stevens et al. (2021) —, 2021, MNRAS, 502, 3158
  * Su et al. (2019) Su K.-Y. et al., 2019, MNRAS, 487, 4393
  * Tacconi et al. (2018) Tacconi L. J. et al., 2018, ApJ, 853, 179
  * Terrazas et al. (2020) Terrazas B. A. et al., 2020, MNRAS, 493, 1888
  * Torrey et al. (2014) Torrey P., Vogelsberger M., Genel S., Sijacki D., Springel V., Hernquist L., 2014, MNRAS, 438, 1985
  * Truong et al. (2020) Truong N. et al., 2020, MNRAS, 494, 549
  * Vogelsberger et al. (2013) Vogelsberger M., Genel S., Sijacki D., Torrey P., Springel V., Hernquist L., 2013, MNRAS, 436, 3031
  * Vogelsberger et al. (2014) Vogelsberger M. et al., 2014, Nature, 509, 177
  * Wang & Jing (2010) Wang L., Jing Y. P., 2010, MNRAS, 402, 1796
  * Wang et al. (2022) Wang S., Xu D., Lu S., Cai Z., Xiang M., Mao S., Springel V., Hernquist L., 2022, MNRAS, 509, 3148
  * Wechsler & Tinker (2018) Wechsler R. H., Tinker J. L., 2018, ARA&A, 56, 435
  * Weinberger et al. (2017) Weinberger R. et al., 2017, MNRAS, 465, 3291
  * White & Frenk (1991) White S. D. M., Frenk C. S., 1991, ApJ, 379, 52
  * White & Rees (1978) White S. D. M., Rees M. J., 1978, MNRAS, 183, 341
  * Yang, Mo & van den Bosch (2009) Yang X., Mo H. J., van den Bosch F. C., 2009, ApJ, 695, 900
  * Yang et al. (2012) Yang X., Mo H. J., van den Bosch F. C., Zhang Y., Han J., 2012, ApJ, 752, 41
  * Yates et al. (2013) Yates R. M., Henriques B., Thomas P. A., Kauffmann G., Johansson J., White S. D. M., 2013, MNRAS, 435, 3500
  * Yuan et al. (2015) Yuan F., Gan Z., Narayan R., Sadowski A., Bu D., Bai X.-N., 2015, ApJ, 804, 101
  * Yuan & Narayan (2014) Yuan F., Narayan R., 2014, ARA&A, 52, 529
  * Yuan et al. (2018) Yuan F., Yoon D., Li Y.-P., Gan Z.-M., Ho L. C., Guo F., 2018, ApJ, 857, 121
  * Zhang et al. (2019) Zhang C. et al., 2019, ApJ, 884, L52
  * Zhang et al. (2021) —, 2021, ApJ, 911, 57
  * Zheng, Coil & Zehavi (2007) Zheng Z., Coil A. L., Zehavi I., 2007, ApJ, 667, 760
  * Zinger et al. (2020) Zinger E. et al., 2020, MNRAS, 499, 768
  * Zubovas et al. (2013) Zubovas K., Nayakshin S., King A., Wilkinson M., 2013, MNRAS, 433, 3079

##  Appendix A Cold Gas–SFR Relation for Centrals in TNG100 and EAGLE

A major concern for the study of the neutral hydrogen gas–SFR relation is the
definition of the galaxy morphology; how we choose what constitutes a disc can
affect our results. There are various morphology metrics used for both
observations and simulations that allow one to distinguish between discs and
ellipticals. However, exploring which one is better is not the task of this
work. Here we briefly assess the $\rm H\,{\textsc{i}}$ gas–SFR relation for
ellipticals in TNG100 and EAGLE. As shown in Figure A.1, the $\rm
H\,{\textsc{i}}$ mass for ellipticals is slightly lower but not significantly
different from that of discs in TNG100. This is mainly caused by the higher-
than-observed $\rm H\,{\textsc{i}}$ content of TNG100 ellipticals, as pointed
out by Diemer et al. (2019). The independence of the cold gas content at given
SFR with morphology is also shown in EAGLE, as seen in Figure A.2. These
results indicate that the neutral hydrogen gas mass–SFR relation shown in the
main text is not significantly affected even if the morphological selection of
galaxies in the simulations is not perfect.

Figure A.1:  Similar to Figure 2, but for ellipticals in TNG100. Note that
here the observations are for central discs. Figure A.2:  Neutral hydrogen gas
mass versus SFR for central discs and ellipticals in EAGLE. The left- and
right-hand panels respectively show the morphology divisions using disc-to-
total mass ratio (D/T) and rotation support parameter ($\kappa_{\rm rot}$).
The blue and oranges lines are median relations for discs and ellipticals
separately, together with the 16th–84th percentile ranges shown by the shaded
areas.

##  Appendix B SFR over different time-scales and apertures

In observations, different SFR indicators, such as UV continuum, ${\rm
H\alpha}$ emission/$D_{4000}$, IR continuum and $1.4$GHz emission, are
sensitive to the SFR over different timescales, varying from less than $\sim
10$ Myr for ${\rm H\alpha}$ to $\sim 100$ Myr for UV+IR (Speagle et al.,
2014). In simulations, SFRs can also be calculated using different time-scales
and apertures. To test the impact of time-scale and aperture choices, in
Figure B.1, we plot the gas–SFR relation of the L08 model as in Figure 2, but
using SFRs derived within different timescales (from instantaneous to $200$
Myr) and apertures ($<2R_{\star}$, $<30$ kpc, all grav.) (Donnari et al.,
2019). There are differences and variations of the results in different
panels, but the general trends for both $\rm H\,{\textsc{i}}$ and ${\rm
H_{2}}$ remain similar, and are in general agreement with observations. We
also tested the results derived with other $\rm H\,{\textsc{i}}/\rm H_{2}$
decomposition models, and find that the general trends remain unchanged
against SFRs calculated using different apertures and time-scales. We want to
point out that while the time-scale and aperture choices for the simulation
are not particularly important, the SFR indicators used for the observations
is still very important, as the modeling uncertainties in observations are
more complicated than just time-scales and apertures (for example, see the
difference between $H_{\alpha}$/$D_{4000}$ and UV+IR SFRs as highlighted in
Z19).

Figure B.1:  $\rm H\,{\textsc{i}}$ and ${\rm H_{2}}$ gas mass versus SFRs
calculated over different time-scales and different apertures. TNG100 $\rm
H\,{\textsc{i}}$ and ${\rm H_{2}}$ masses here use the L08 decomposition
model. The limegreen and gold lines with error bars are the observational
results given in Z19 for $\rm H\,{\textsc{i}}$ and $\rm H_{2}$ respectively.

##  Appendix C Gas density in Example SF and quenching galaxies

In Figure C.1, we show the stellar light and total gas distribution for a
typical higher SFR central disc galaxy and a lower SFR central disc in TNG100.
The lower SFR galaxy also shows a regular rotating gas disc, similar to the
higher SFR one. This is also in good agreement with the results in Z19, where
they find that most of the quenching central disc galaxies show
characteristically symmetric double-horn $\rm H\,{\textsc{i}}$ profiles,
indicating regularly rotating $\rm H\,{\textsc{i}}$ discs with little
significant kinematic perturbations, similar to the star-forming ones. The
difference between the two galaxies is more directly shown in the right panel.
Comparing to the one with higher SFR, in the lower SFR galaxy, there are fewer
regions with high gas density (hence less gas is in the form of ${\rm
H_{2}}$). The dashed line corresponds to density threshold of $0.13\,{\rm
cm^{-3}}$ (i.e. $n_{\rm H}\sim 0.1\,{\rm cm^{-3}}$) set up for star formation.
Note the gas cell volumes in quenching galaxy is much larger than the SF one,
as a result, the surface density profile of these two galaxies are not
significantly different.

Figure C.1:  Left: The composite stellar light of JWST wide bands of
F200W+F115W+F070W (upper) and gas (lower) distribution of a representative
higher SFR(all grav.)$>10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$ and a lower
SFR${}_{\rm limit}<$SFR(all grav.)$<10^{-0.5}{\rm M}_{\odot}{\rm yr}^{-1}$
disc galaxy at $z=0$. The images size is $60$ kpc $\times$ $60$ kpc for
stellar light and $140$ kpc $\times$ $140$ kpc for the gas. Galaxies are
rotated to be face-on. The circles indicate the $1.0R_{\star}$ and
$2.0R_{\star}$ radius separately. Please note that the scale of the color
coding for gas density is the same for both galaxies. Masses and SFRs are in
units of ${\rm M}_{\odot}$ and ${\rm M}_{\odot}\,{\rm yr}^{-1}$, respectively.
Right: The 3D gas density distribution in the example galaxies in TNG100. The
gray vertical line indicates the density threshold for star formation adopted
in IllustrisTNG model.


# SSPNet: Scale Selection Pyramid Network for Tiny Person Detection from UAV
Images

Mingbo Hong, Shuiwang Li, Yuchao Yang, Feiyu Zhu, Qijun Zhao, Li Lu Mingbo
Hong, Shuiwang Li,Yuchao Yang, Feiyu Zhu, Qijun Zhao and Li Lu are with
College of Computer Science, Sichuan University, China, 30332 USA e-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

With the increasing demand for search and rescue, it is highly demanded to
detect objects of interest in large-scale images captured by Unmanned Aerial
Vehicles (UAVs), which is quite challenging due to extremely small scales of
objects. Most existing methods employed Feature Pyramid Network (FPN) to
enrich shallow layers’ features by combing deep layers’ contextual features.
However, under the limitation of the inconsistency in gradient computation
across different layers, the shallow layers in FPN are not fully exploited to
detect tiny objects. In this paper, we propose a Scale Selection Pyramid
network (SSPNet) for tiny person detection, which consists of three
components: Context Attention Module (CAM), Scale Enhancement Module (SEM),
and Scale Selection Module (SSM). CAM takes account of context information to
produce hierarchical attention heatmaps. SEM highlights features of specific
scales at different layers, leading the detector to focus on objects of
specific scales instead of vast backgrounds. SSM exploits adjacent layers’
relationships to fulfill suitable feature sharing between deep layers and
shallow layers, thereby avoiding the inconsistency in gradient computation
across different layers. Besides, we propose a Weighted Negative Sampling
(WNS) strategy to guide the detector to select more representative samples.
Experiments on the TinyPerson benchmark show that our method outperforms other
state-of-the-art (SOTA) detectors.

###### Index Terms:

Tiny object detection, Feature pyramid network, Unmanned aerial vehicle, Scale
selection, Feature fusion.

##  I Introduction

As a high-efficiency image acquisition system, UAVs have the advantages of
high intelligence, high mobility and large field-of-view, and have thus been
widely used in the emerging field for searching persons in a large area and at
a very long distance. However, in such a scenario, finding persons is
challenging since most persons in the obtained images are of tiny scale with
low signal-noise ratio and easily contaminated by backgrounds[1].

Figure 1: Illustrations of FPN (a) and our SSPNet (b). FPN employs the
element-wise addition operation to integrate the adjacent features directly.
Our SSPNet can learn adjacent layers’ relationships to fulfill suitable
feature sharing between deep layers and shallow layers, thereby avoiding the
inconsistency in gradient computation across different layers.

A couple of methods have recently been proposed for tiny object detection.
Some of them employ multi-training stages to enrich features to improve
performance[2, 3], and others use a general-purpose framework with data
augmentation to improve the detection of tiny objects[4, 1, 5]. Since deeper
layers may furnish more semantic features, most existing methods[6, 7]
employed Feature Pyramid Network (FPN) to enrich shallow layers’ features for
boosting performance by integrating deep layers’ features. Gong et al.[8]
propose a feature-level statistic-based approach to avoid propagating too much
noise from deep layers to shallow layers in FPN.

Despite the impressive results obtained by the FPN-based methods, they still
suffer from the inconsistency in gradient computation[6], thereby downgrading
the effectiveness of FPN. Since the anchors of deep layers can not match tiny
objects, the corresponding positions on most deep layers are assigned as the
negative sample, and only a few shallow layers assign them as the positive
sample. Those corresponding features optimized toward the negative sample are
directly delivered from deep layers to shallow layers by the element-wise
addition operation (as shown in Fig. 1 (a)) to integrate with the features
that are optimized toward the positive sample, which causes the inconsistency
in gradient computation across different layers since the addition operation
can not adaptively adjust the adjacent data flows. In other words, the
gradient computation inconsistency downgrades deep layers’ representation
ability. Thereby, deep layers may not be able to guide the training of shallow
layers but instead increase the burden on the shallow layers.

In this paper, we propose a Scale Selection Pyramid Network, namely SSPNet, to
improve tiny person detection performance by mitigating the inconsistency in
gradient computation across different layers. We notice that the inconsistency
in gradient computation does not occur in adjacent layers if an object is
assigned as positive samples in adjacent layers, where the adjacent features
corresponding to the object can be treated as suitable features of the
adjacent layers since they are both optimized toward the positive sample. This
motivates us to design a Context Attention Module (CAM) to generate
hierarchical attention heatmaps that point out objects of which scale can be
assigned as positive samples in each layer of SSPNet. If those objects in
adjacent attention heatmaps have intersections, the gradients in these
intersected regions are consistent. Thus, we propose a Scale Selection Module
(SSM) to deliver suitable features from deep layers to shallow layers with the
guidance of heatmap intersections to resolve the inconsistency in gradient
computation across different layers as shown in Fig. 1 (b). Besides, with the
guidance of attention heatmaps, we also design a Scale Enhancement Module
(SEM) to focus the detector on those objects of specific scales (assigned as
positive samples) in each layer rather than vast and cluttered backgrounds.
Meanwhile, to further reduce false alarms, we employ a Weighted Negative
Sampling (WNS) strategy to guide the detector to look more at representative
samples to avoid missing representative samples in thousands of easy samples.

Our main contributions are summarized as follows:

  * •

A novel SSPNet is proposed to suppress the inconsistency of gradient
computation in FPN by controlling adjacent layers’ data flow.

  * •

A WNS strategy is proposed to decrease false alarms by giving priority to
those representative samples.

  * •

A mathematical explanation is given to explain why our SSPNet can relieve the
inconsistency in gradient computation.

  * •

The proposed SSPNet significantly improves the detector’s performance and
outperforms SOTA detectors on the TinyPerson benchmark.

Figure 2: (a) The overview of our proposed SSPNet. $P_{k}$ refers to the
SSPNet’s output at the $k^{th}$ layer. (b) Visualization of the attention
heatmaps. The top row indicates the detection results, and the bottom row
indicates the visualization of the attention heatmaps and the corresponding
supervised attention heatmaps at different layers. The red dashed boxes
contain large objects that are not objects of specific scales in the $1^{st}$
layer since they can not be matched by anchors of the $1^{st}$ layer, and the
green boxes indicates most tiny objects that can not be matched by anchors of
the $5^{th}$ layer.

##  II Methodology

Our SSPNet, based on the framework of Faster R-CNN[9], includes CAM, SEM, SSM,
as illustrated in Fig. 2 (a). In the following, we describe each module in
detail.

###  II-A Context Attention Module

To produce hierarchical attention heatmaps, we design CAM to generate
attention heatmaps of different layers. As discussed in prior work[10, 11],
the context information can boost the performance of finding small objects.
Thus, we first upsample all features produced by the backbone at different
stages to the same shape as the bottom one and integrate them by
concatenation. Then, atrous spatial pyramid pooling[12] (ASPP), with filters
at multiple sampling rates and effective fields-of-views, is employed to find
the object cues by considering multi-scale features. The context-aware
features produced by ASPP are delivered to an activation gate that consists of
multiple 3x3 convolutions with different strides and the sigmoid activation
function to generate hierarchical attention heatmaps $A_{k}$:

$A_{k}=\sigma(\phi_{k}(F_{c},w,s)),$ (1)

where $\sigma$ is the sigmoid activation function, $\phi_{k}$ denotes a 3x3
convolution at the $k^{th}$ layer, $w\in\mathbb{R}^{C_{F}\times 1\times
3\times 3}$ represents the convolutional parameters, $F_{c}$ indicates the
context-aware features produced by ASPP, and $s=2^{k-2}$ denotes the stride of
the convolution.

To point out which scale objects can be assigned as positive samples in each
layer of SSPNet, we employ a supervised attention mechanism[13] to highlight
objects of specific scales in each layer of SSPNet and avoid being overwhelmed
by vast backgrounds. Specifically, the supervised attention heatmaps are
associated with the objects matched by the anchors at different layers. As
shown in Fig. 2 (b), the supervised attention heatmap show different specific
scale ranges at different layers, among which the red and green dashed boxes
show that those objects that the corresponding layers’ anchors do not match
will be regarded as the background. Moreover, the corresponding attention
heatmaps are shown in Fig. 2 (b), and our CAM is able to yield the attention
heatmaps with specific scale ranges.

###  II-B Scale Enhancement Module

SEM is implemented to enhance the cues of objects of specific scales. Since
the attention heatmaps at different layers have different scale preferences,
allowing SEM to generate scale-aware features:

$F^{o}_{k}=(1+A_{k})\odot F^{i}_{k},$ (2)

where $F^{i}_{k}$ and $F^{o}_{k}$, respectively, are the input feature maps
and output scale-aware features, $A_{k}$ is the attention heatmap at the
$k^{th}$ layer, and $\odot$ is element-wise multiplication. Note that residual
connection is used to avoid degrading the features around the objects since
context information may facilitate detection.

###  II-C Scale Selection Module

To select suitable features from deep layers for shallow layers, we propose
the SSM to guide deep layers to deliver suitable features to shallow layers,
where the suitable features do not cause inconsistency in gradient computation
since they are optimized toward the same class. On the other hand, if the
objects can be all detected in the adjacent layers, the deep layers will
provide more semantic features and optimize with the next layer
simultaneously[7]. Our SSM can be formulated as follows:

$P^{\prime}_{k-1}=(A_{k-1}\odot f_{nu}(A_{k}))\odot
f_{nu}(P^{\prime}_{k})+C_{k-1},$ (3)

where the intersection of $A_{k-1}$ and $A_{k}$ is obtained by $\odot$,
$f_{nu}$ denotes the nearest upsampling operation, $P^{\prime}_{k}$ is the
merged map at the ${k}^{th}$ layer, $C_{k-1}$ is the ${(k-1)}^{th}$ residual
block’s output.

Specifically, SSM plays the role of scale selector. Those features
corresponding to objects within the scale range of the next layer will be
treated as suitable features to flow into the next layer, while other features
will be weakened to suppress the inconsistency in gradient computation.

###  II-D Weighted Negative Sampling

In large fields-of-view images captured by UAVs, complicated background
typically introduces more noise than natural scene images. Besides, the
partial occlusion in those images causes some objects to be annotated with
only visible parts, resulting in the detector treats a person’s parts as
complete individuals, especially when the dataset is not large. Motivated by
those considerations, we propose WNS to enhance the detector’s generalization
ability by looking more at representative samples.

Firstly, hard negative samples are usually regarded as positive ones with high
confidence by the detector. Thus, confidence is the most intuitive factor that
needs to be considered. Then, to quantify the degree of incomplete objects, we
adopt the intersection over foreground (IoF) [1] criterion. Next, we construct
a score-fusion function to consider the two factors of IoF and confidence as
follows:

$s_{i}=\frac{e^{\lambda C_{i}+(1-\lambda)I_{i}}}{\sum_{i=1}^{N}e^{\lambda
C_{i}+(1-\lambda)I_{i}}},$ (4)

where $C_{i}$ and $I_{i}$, respectively, denote the $i^{th}$ detection
result’s confidence and the corresponding maximum IoF, and $\lambda$ indicates
the balanced coefficient utilized to adjust the weights of IoF and confidence.
Then, we can adjust the probability of selection for each sample based on
$s_{i}$.

###  II-E Loss Function

Our SSPNet can be optimized by a joint loss function, which is formulated as:

$L=L_{RPN}+L_{Head}+L_{A},$ (5)
$L_{RPN}=\frac{1}{N_{bce}}\sum_{i=1}L_{bce}(rc_{i},rc^{*}_{i})+\frac{\mu_{1}}{N_{reg}}\sum_{i=1}L_{reg}(rt_{i},rt^{*}_{i}),$
(6)
$L_{Head}=\frac{1}{N_{ce}}\sum_{i=1}L_{ce}(hc_{i},hc^{*}_{i})+\frac{\mu_{2}}{N_{reg}}\sum_{i=1}L_{reg}(ht_{i},ht^{*}_{i}),$
(7)

where both $L_{RPN}$ and $L_{Head}$ employ a smooth L1 loss for bounding-box
regression, but for classification, the former employs the binary cross-
entropy (BCE) loss, while the latter employs cross-entropy loss. For
$L_{RPN}$, $i$ is the index of bounding box in minibatch. $rc_{i}$ and
$rc^{*}_{i}$, respectively, denote the probability distributions of the
predicted classes and the ground-truth. $rt_{i}$ and $rt^{*}_{i}$,
respectively, denote the predicted bounding box and ground-truth box. The
classification and regression losses are normalized by $N_{cls}$ (minibatch
size) and $N_{reg}$ (number of boxes locations) and weighted by a balanced
parameter $\mu_{1}$. By default we set $\mu_{1}$ and $\mu_{2}$ as 1.
$L_{Head}$ is defined in a similar way.

$L_{A}$ indicates the attention loss to guide CAM to generate hierarchical
attention heatmaps. In particular, the attention loss can be formulated as:

$L_{A}=\alpha L^{b}_{A}+\beta L^{d}_{A},$ (8)

where $\alpha$ and $\beta$, respectively, indicate the hyper-parameters of the
dice loss[14] $L^{d}_{A}$ and the BCE loss $L^{b}_{A}$. In detail, to avoid
being overwhelmed by the vast backgrounds, we employ the dice loss to
prioritize the foreground since it is only relevant to the intersection
between the attention heatmap and the supervised attention heatmap. Secondly,
to remedy the gradient vanishing when the attention heatmap and the supervised
attention heatmap have no intersection, we utilize BCE loss to deal with this
extreme case and provide the valid gradient for optimizing. Moreover, we
employ OHEM[15] to guarantee the detector focuses mainly on the non-object
areas that are easily regarded as foreground, and we set the ratio of
positives and negatives as 1:3 instead of considering all negatives.
Specifically, the BCE loss is employed to learn poorly classified negatives,
and the dice loss is employed to learn the class distribution to alleviate the
imbalanced data.

TABLE I: APs of different methods on TinyPerson.The best result in each MR is highlighted in bold, and the second best is highlighted in underline. ∗: our implemented baseline. Detector | $AP^{tiny}_{50}\uparrow$ | $AP^{tiny1}_{50}\uparrow$ | $AP^{tiny2}_{50}\uparrow$ | $AP^{tiny3}_{50}\uparrow$ | $AP^{small}_{50}\uparrow$ | $AP^{tiny}_{25}\uparrow$ | $AP^{tiny}_{75}\uparrow$ | $years$
---|---|---|---|---|---|---|---|---
RetinaNet[16] | 33.53 | 12.24 | 38.79 | 47.38 | 48.26 | 61.51 | 2.28 | 2017
Faster RCNN-FPN[17] | 47.35 | 30.25 | 51.58 | 58.95 | 63.18 | 63.18 | 5.83 | 2017
Faster RCNN-PANet∗(PAFPN)[22] | 57.17 | 44.53 | 59.85 | 65.52 | 70.32 | 77.30 | 8.10 | 2018
Grid RCNN[18] | 47.14 | 30.65 | 52.21 | 57.21 | 62.48 | 68.89 | 6.38 | 2018
Libra RCNN(Balanced FPN)[21] | 44.68 | 27.08 | 49.27 | 55.21 | 62.65 | 64.77 | 6.26 | 2019
FCOS[19] | 17.90 | 2.88 | 12.95 | 31.15 | 40.54 | 41.95 | 1.50 | 2019
NAS-FPN[20] | 37.75 | 26.71 | 40.69 | 45.33 | 52.63 | 66.24 | 3.10 | 2019
Faster RCNN-FPN-MSM[1] | 50.89 | 33.79 | 55.55 | 61.29 | 65.76 | 71.28 | 6.66 | 2020
Faster RCNN-FPN-MSM+[23] | 52.61 | 34.20 | 57.60 | 63.61 | 67.37 | 72.54 | 6.72 | 2021
RetinaNet+SM with S-$\alpha$[8] | 52.56 | 33.90 | 58.00 | 63.72 | 65.69 | 73.09 | 6.64 | 2021
Swin-T[24] | 40.52 | 31.92 | 41.67 | 47.06 | 52.53 | 59.42 | 4.24 | 2021
RetinaNet∗ | 52.86 | 42.22 | 58.07 | 59.04 | 66.40 | 76.79 | 6.5 | 2017
RetinaNet-SPPNet | 54.66 ($\uparrow 1.8$) | 42.72 | 60.16 | 61.52 | 65.24 | 77.03 | 6.31 | 2021
Cascade RCNN-FPN∗ | 57.19 | 45.21 | 60.06 | 65.06 | 70.71 | 76.99 | 8.56 | 2018
Cascade RCNN-SSPNet |  58.59 ($\uparrow 1.4$) | 45.75 | 62.03 | 65.83 | 71.80 | 78.72 | 8.24 | 2021
Faster RCNN-FPN∗ | 57.05 | 43.82 | 60.41 | 65.06 | 70.15 | 76.39 | 7.90 | 2017
Faster RCNN-SPPNet |  59.13 ($\uparrow$2.08) | 47.56 | 62.36 | 66.15 | 71.17 | 79.47 | 8.62 | 2021
TABLE II: MRs of different methods on TinyPerson.The best result in each MR is highlighted in bold, and the second best is highlighted in underline. ∗: our implemented baseline. Detector | $MR^{tiny}_{50}\downarrow$ | $MR^{tiny1}_{50}\downarrow$ | $MR^{tiny2}_{50}\downarrow$ | $MR^{tiny3}_{50}\downarrow$ | $MR^{small}_{50}\downarrow$ | $MR^{tiny}_{25}\downarrow$ | $MR^{tiny}_{75}\downarrow$ | $years$
---|---|---|---|---|---|---|---|---
RetinaNet[16] | 88.31 | 89.65 | 81.03 | 81.08 | 74.05 | 76.33 | 98.76 | 2017
Faster RCNN-FPN[17] | 87.57 | 87.86 | 82.02 | 78.78 | 72.56 | 76.59 | 98.39 | 2017
Faster RCNN-PANet∗(PAFPN)[22] | 85.18 | 83.24 | 77.39 | 75.77 | 65.38 | 72.25 | 98.32 | 2018
Grid RCNN[18] | 87.96 | 88.31 | 82.79 | 79.55 | 73.16 | 78.27 | 98.21 | 2018
Libra RCNN (Balanced FPN)[21] | 89.22 | 90.93 | 84.64 | 81.62 | 74.86 | 82.44 | 98.39 | 2019
FCOS[19] | 96.28 | 99.23 | 96.56 | 91.67 | 84.16 | 90.34 | 99.56 | 2019
NAS-FPN[20] | 92.41 | 90.37 | 87.41 | 87.50 | 81.78 | 77.79 | 99.29 | 2019
Faster RCNN-FPN-MSM[1] | 85.86 | 86.54 | 79.20 | 76.86 | 68.76 | 74.33 | 98.23 | 2020
RetinaNet+SM with S-$\alpha$[8] | 87.00 | 87.62 | 79.47 | 77.39 | 69.25 | 74.85 | 98.57 | 2021
Swin-T[24] | 89.91 | 87.20 | 85.44 | 85.31 | 80.28 | 82.36 | 98.89 | 2021
RetinaNet∗ | 86.48 | 82.40 | 78.80 | 78.18 | 72.82 | 70.52 | 98.57 | 2017
RetinaNet-SPPNet∗ | 85.30 ($\downarrow 1.18$) | 82.87 | 76.73 | 77.20 | 72.37 | 69.25 | 98.63 | 2021
Cascade RCNN-FPN∗ | 84.66 | 82.32 | 76.84 | 75.03 | 64.77 | 73.40 | 98.18 | 2018
Cascade RCNN-SSPNet |  83.47 ($\downarrow 1.19$) | 82.80 | 75.02 | 73.52 | 62.06 | 68.93 | 98.27 | 2021
Faster RCNN-FPN∗ | 84.12 | 83.98 | 76.10 | 74.58 | 64.03 | 73.82 | 98.19 | 2017
Faster RCNN-SPPNet |  82.79 ($\downarrow$1.33) | 81.88 | 73.93 | 72.43 | 61.26 | 66.80 | 98.06 | 2021

##  III Experiments

###  III-A Experimental Setup and Evaluation Metric

We employ the TinyPerson benchmark[1] to evaluate our method’s effectiveness,
and TinyPerson contains 794 and 816 images for training and inference,
respectively. We crop each image into patches of $640\times 512$ pixels with
$30$ pixels overlap in the training phase to avoid the GPU being out of
memory. Our codes are based on the Mmdetection toolkit[25]. We choose
ResNet-50 as the backbone. Motivated by YOLOv2[26], we also utilize k-means
clustering to find better prior anchors. By default, our SSPNet is trained in
$10$ epochs, and the stochastic gradient descent is utilized as the optimizer
with learning rate initialized as $0.002$ and decreased by a factor of $0.1$
after $8$ epochs. Besides, we set the proposal number of the RPN to $2000$ in
the training phase and $1000$ in the testing phase, the loss weight $\alpha$
and $\beta$ are empirically set to $0.01$ and $1$, and the hyper-parameter
$\lambda$ is set to $0.6$.

Keeping consistent with the TinyPerson benchmark[1], we also adopt AP (average
precision) as evaluation metrics. The TinyPerson benchmark[1] further divides
$tiny[2,20]$ into 3 sub-intervals: $tiny1[2,8]$, $tiny2[8,12]$, and
$tiny3[12,20]$.

###  III-B Comparison to State-of-the-arts

We compare our SSPNet with some SOTA methods on TinyPerson. As a standard
criterion in the detection tasks, the higher the AP value, the better the
detector’s performance. However, contrary to AP, as a criterion to reflect the
percentage of objects being missed by the detector, the lower the MR value,
the better the detector’s performance. Tables I and II show the performance of
different methods in terms of MR and APmetrics. We found that most of the
general-purpose detectors failed to achieve promising performance in such a
scenario since there is a cross-domain gap between the UAV images and natural
scene images. Note that we do not directly adopt the performance of Yu et
al.[1] as the baseline, but implement stronger baselines with data
augmentation (such as ShiftScaleRotate, MotionBlur, etc.) and multi-scale
training, which achieves better results than the SOTA detectors. Note that all
these detectors, i.e., RetinaNet, FasterRCNN, and Cascade RCNN, gain further
improvements with our proposed SSPNet.

###  III-C Ablation Study

TABLE III: Effect of each component of Faster RCNN-SSPNet SEM | SSM | WNS | Attention Loss | $AP^{tiny}_{50}$
---|---|---|---|---
|  |  |  | 57.05
$\checkmark$ |  |  | $\checkmark$ | 58.12 ($\uparrow 1.07$)
| $\checkmark$ |  | $\checkmark$ | 58.25 ($\uparrow 1.20$)
|  | $\checkmark$ |  | 57.40 ($\uparrow 0.35$)
$\checkmark$ | $\checkmark$ |  | $\checkmark$ | 58.43 ($\uparrow 1.38$)
$\checkmark$ | $\checkmark$ | $\checkmark$ |  | 57.86 ($\uparrow 0.81$)
$\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | 59.13 ($\uparrow 2.08$)
Figure 3: Visualization of detection results on TinyPerson’s test set, where
those areas labeled as $uncertain$ in test set images are erased.

In this part, we analyze each module’s effect in our proposed method by
applying the modules gradually. Results are reported in Table III.

  * •

SEM: SEM is proposed to facilitate the detector to focus on objects of
specific scales instead of the vast background. Results in Table III prove its
effectiveness with $1.07\%$ improvement in $AP^{tiny}_{50}$ over the baseline.

  * •

SSM: SSM delivers those suitable features from deep layers to shallow layers
and significantly improves $AP^{tiny}_{50}$ of the baseline by $1.2\%$.

  * •

WNS: WNS guides the detector to look more at those human parts and backgrounds
that are easily considered as individuals. Without increasing more training
parameters, WNS improves $AP^{tiny}_{50}$ of the baseline by $0.35\%$.

  * •

Attention loss: In the last two rows of Table III, we compare the attention
loss with a single BCE loss, and the experimental results show that the
attention loss can improve the performance by compensating for a single BCE
loss’s shortcoming that only focuses on classification error at pixel-level.

###  III-D Influence of Balanced Coefficient

Figure 4: The impact of the balanced coefficient

To choose an appropriate balanced coefficient, we investigate the impact of
the balanced coefficient $\lambda$. We set $\lambda$ from 0.0 to 1.0, as shown
in Fig. 4. When $\lambda$ is 0, it means that $s_{i}$ depends entirely on the
value of confidence and vice versa. Besides, we can observe from Fig. 4, the
best performance is achieved when the coefficient is 0.6 while relying on only
one of the two factors fails to achieve better performance. This might be
because most UAV images are accompanied by cluttered backgrounds or partial
occlusion in the TinyPerson dataset. Therefore, we need to focus on those
negative samples with both high confidence and high IoF.

###  III-E Effectiveness of Attention loss

Figure 5: Illustration of the qualitative comparison on the attention loss.

In this part, we present a qualitative evaluation to analyze why we employ the
attention loss. As shown in Fig. 5, comparing the attention heatmap supervised
by the attention loss with that supervised by a single BCE loss, the former
attention map appears visually more apparent. On the contrary, the tiny
objects’ boundary in the attention map supervised by a single BCE loss is very
blurred. This phenomenon can be attributed to that the dice loss is able to
extract the foreground better. Besides, as mentioned in Section II-E, since we
only need to focus on suppressing those non-object areas that are easily
regarded as foregrounds, the other non-objects areas will be retained
adaptively to keep context information to facilitate detection. In other
words, the attention heatmaps will not be entirely close to binary maps.

###  III-F Visual Analysis

To further demonstrate the effectiveness of the proposed SSPNet, we show some
visual detection results in Fig. 3, where the first two rows indicate the
scenes of the persons in the sea, and the last row reflects the scenes of the
persons on the land. It can be observed from the first two rows that the
postures of the persons on the surfboards have large variations, even some
persons only reveal heads in backlight condition, but our SSPNet still can
accurately recognize and locate them. Besides, the last row contains scale
variations, crowd scenarios, and cluttered backgrounds. Even so, most of the
persons can be recognized and located. Although the detector may miss a few
persons due to occlusion, we believe that these problems can be significantly
improved if the dataset can be more extensive and contains more of such
crowded scenarios rather than being limited to a small number of training
images.

###  III-G Mathematical Explanation

In this part, we will analyze why our SSM can address the inconsistency in
gradient computation from the perspective of gradient propagation. Without
loss of generality, we discuss the gradient of a specific location $(i,j)$ in
the $5^{th}$ layer. Suppose it is assigned as the center of a positive sample
in the $2^{nd}$ layer and regarded as background in the other layers. The
gradient $\frac{\partial L}{\partial P^{\prime}_{5,(i,j)}}$ can be represented
as follows:

$\displaystyle\frac{\partial L}{\partial
P^{\prime}_{5,(i,j)}}=\sum_{k=2}^{5}\frac{\partial L_{P_{k}}}{\partial
P_{k,(i,j)}}*\frac{\partial P_{k,(i,j)}}{\partial
P^{\prime}_{k,(i,j)}}*\frac{\partial P^{\prime}_{k,(i,j)}}{\partial
P^{\prime}_{5,(i,j)}},$ (9)

where $L_{P_{k}}$ indicates the loss of the $k^{th}$ layer in FPN. Since
$P^{\prime}_{k,(i,j)}$ is associated with $P^{\prime}_{5,(i,j)}$ by an
interpolation operation, $\frac{\partial P^{\prime}_{k,(i,j)}}{\partial
P^{\prime}_{5,(i,j)}}$ is usually a fixed value. In this case, $\frac{\partial
L_{P^{2}}}{\partial P^{\prime}_{2,(i,j)}}$will be optimized towards the
positive sample, but $\frac{\partial L_{P_{3}}}{\partial
P^{\prime}_{3,(i,j)}}$, $\frac{\partial L_{P_{4}}}{\partial
P^{\prime}_{4,(i,j)}}$, and $\frac{\partial L_{P_{5}}}{\partial
P^{\prime}_{5,(i,j)}}$ will be optimized towards the negative sample,
resulting in the inconsistency in gradient computation at the position $(i,j)$
in the $5^{th}$ layer.

For our SSM, we can express $P^{\prime}_{k,(i,j)}$ as follows:

$\displaystyle
P^{\prime}_{k,(i,j)}=\underbrace{A_{5,(i,j)}*A_{k,(i,j)}*\prod_{n=k+1}^{4}A^{2}_{n,(i,j)}}_{\psi_{k,(i,j)}}*P^{\prime}_{5\to
k,(i,j)},$ (10)

where $P^{\prime}_{m\to l,(i,j)}$ indicates the merged map in SSPNet resized
from layer $m$ to layer $l$ at the position $(i,j)$.

Next, we can adjust the inconsistency in gradient computation by the attention
coefficient $\psi_{k,(i,j)}$ as follows:

$\displaystyle\frac{\partial P_{k,(i,j)}}{\partial
P^{\prime}_{5,(i,j)}}={\psi_{k,(i,j)}}*\frac{\partial P_{k,(i,j)}}{\partial
P^{\prime}_{k,(i,j)}}*\frac{\partial P^{\prime}_{5\to k,(i,j)}}{\partial
P^{\prime}_{5,(i,j)}},$ (11)

where the interpolation gradient $\frac{\partial P^{\prime}_{m\to
l,(i,j)}}{\partial P^{\prime}_{m,(i,j)}}$ is usually a fixed value. Let us
assume $\frac{\partial P^{\prime}_{m\to l,(i,j)}}{\partial
P^{\prime}_{m,(i,j)}}\approx 1$. Then, we can simplify Equation (9) as:

$\displaystyle\frac{\partial L}{\partial
P^{\prime}_{5,(i,j)}}\approx\sum_{k=2}^{5}\frac{\partial L_{P_{k}}}{\partial
P^{\prime}_{k,(i,j)}}*\psi_{k,(i,j)}$ (12)

##  IV Conclusion

This paper discusses the inconsistency of gradient computation encountered in
FPN-based methods for tiny person detection, which weakens the representation
ability of shallow layers in FPN. We propose a novel model called SSPNet to
overcome those challenges. Specifically, under the supervision of attention
loss, CAM is able to yield the attention heatmaps with specific scale ranges
at each layer in SSPNet. With the guidance of heatmaps, SEM strengthens those
cues of objects of specific scales; SSM controls the data flow with
intersection heatmaps to fulfill suitable feature sharing between deep layers
and shallow layers. Besides, to fully exploit small datasets to train better
detectors, we propose a WNS to select representative samples to enable
efficient training. In future work, we will extend our method for a
bidirectional feature pyramid network.

## References

  * [1] X. Yu, Y. Gong, N. Jiang, Q. Ye, and Z. Han, “Scale match for tiny person detection,” in _WACV_ , 2020, pp. 1257–1265.
  * [2] Y. Bai, Y. Zhang, M. Ding, and B. Ghanem, “Finding tiny faces in the wild with generative adversarial network,” in _CVPR_ , 2018, pp. 21–30.
  * [3] J. Noh, W. Bae, W. Lee, J. Seo, and G. Kim, “Better to follow, follow to be better: Towards precise supervision of feature super-resolution for small object detection,” in _CVPR_ , 2019, pp. 9725–9734.
  * [4] M. Kisantal, Z. Wojna, J. Murawski, J. Naruniec, and K. Cho, “Augmentation for small object detection,” _arXiv preprint arXiv:1902.07296_ , 2019.
  * [5] X. Yu, Z. Han, Y. Gong, N. Jan, J. Zhao, Q. Ye, J. Chen, Y. Feng, B. Zhang, X. Wang _et al._ , “The 1st tiny object detection challenge: Methods and results,” _arXiv preprint arXiv:2009.07506_ , 2020.
  * [6] S. Liu, D. Huang, and Y. Wang, “Learning spatial fusion for single-shot object detection,” _arXiv preprint arXiv:1911.09516_ , 2019.
  * [7] T. Kong, F. Sun, H. Liu, Y. Jiang, L. Li, and J. Shi, “Foveabox: Beyound anchor-based object detection,” _TIP_ , vol. 29, pp. 7389–7398, 2020.
  * [8] Y. Gong, X. Yu, Y. Ding, X. Peng, J. Zhao, and Z. Han, “Effective fusion factor in fpn for tiny object detection,” in _WACV_ , January 2021, pp. 1160–1168.
  * [9] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” _TPAMI_ , vol. 39, no. 6, pp. 1137–1149, 2016.
  * [10] F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” _arXiv preprint arXiv:1511.07122_ , 2015.
  * [11] P. Hu and D. Ramanan, “Finding tiny faces,” in _TPAMI_ , 2017, pp. 951–959.
  * [12] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” _TPAMI_ , vol. 40, no. 4, pp. 834–848, 2017.
  * [13] J. Wang, Y. Yuan, and G. Yu, “Face attention network: An effective face detector for the occluded faces,” _arXiv preprint arXiv:1711.07246_ , 2017\.
  * [14] F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in _3DV_. IEEE, 2016, pp. 565–571.
  * [15] A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in _CVPR_ , 2016, pp. 761–769.
  * [16] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in _ICCV_ , 2017, pp. 2980–2988.
  * [17] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in _CVPR_ , 2017, pp. 2117–2125.
  * [18] X. Lu, B. Li, Y. Yue, Q. Li, and J. Yan, “Grid r-cnn,” in _CVPR_ , 2019, pp. 7363–7372.
  * [19] Z. Tian, C. Shen, H. Chen, and T. He, “Fcos: Fully convolutional one-stage object detection,” in _ICCV_ , 2019, pp. 9627–9636.
  * [20] G. Ghiasi, T.-Y. Lin, and Q. V. Le, “Nas-fpn: Learning scalable feature pyramid architecture for object detection,” in _CVPR_ , 2019, pp. 7036–7045.
  * [21] J. Pang, K. Chen, J. Shi, H. Feng, W. Ouyang, and D. Lin, “Libra r-cnn: Towards balanced learning for object detection,” in _CVPR_ , 2019, pp. 821–830.
  * [22] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, “Path aggregation network for instance segmentation,” in _CVPR_ , 2018, pp. 8759–8768.
  * [23] N. Jiang, X. Yu, X. Peng, Y. Gong, and Z. Han, “Sm+: Refined scale match for tiny person detection,” in _ICASSP_. IEEE, 2021, pp. 1815–1819.
  * [24] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” _arXiv preprint arXiv:2103.14030_ , 2021.
  * [25] K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu _et al._ , “Mmdetection: Open mmlab detection toolbox and benchmark,” _arXiv preprint arXiv:1906.07155_ , 2019.
  * [26] J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in _CVPR_ , July 2017.