# One-Stage Cascade Refinement Networks for Infrared Small Target Detection

Yimian Dai, Xiang Li, Fei Zhou, Yulei Qian, Yaohong Chen, Jian Yang  This work
was supported by the fellowship of China Postdoctoral Science Foundation (No.
2021M701727), the National Key R&D Plan of the Ministry of Science and
Technology (Project No.2020AAA0104400), the Young Scientists Fund of the
National Natural Science Foundation of China (Grant No.62206134), and the
National Science Fund of China (Grant No.U1713208). _(Corresponding author:
Xiang Li and Jian Yang.)_ Yimian Dai and Jian Yang are with PCA Lab, Key Lab
of Intelligent Perception and Systems for High-Dimensional Information of
Ministry of Education, School of Computer Science and Engineering, Nanjing
University of Science and Technology, Nanjing, China. (e-mail:
<EMAIL_ADDRESS>csjyang@mail.njust.edu.cn). Xiang Li is with IMPlus@PCA
Lab, College of Computer Science, Nankai University.
(e-mail:xiang.li.implus@nankai.edu.cn). Fei Zhou is with College of
Information Science and Engineering, Henan University of Technology,
Zhengzhou, China. (e-mail: hellozf1990@163.com). Yulei Qian is with Nanjing
Marine Radar Institute, Nanjing, China (e-mail: yuleifly@126.com). Yaohong
Chen is with Xi’an Institute of Optics and Precision Mechanics, Chinese
Academy of Sciences, Xi’an, China. (e-mail: chenyaohong@opt.ac.cn).

###### Abstract

Single-frame InfraRed Small Target (SIRST) detection has been a challenging
task due to a lack of inherent characteristics, imprecise bounding box
regression, a scarcity of real-world datasets, and sensitive localization
evaluation. In this paper, we propose a comprehensive solution to these
challenges. First, we find that the existing anchor-free label assignment
method is prone to mislabeling small targets as background, leading to their
omission by detectors. To overcome this issue, we propose an all-scale pseudo-
box-based label assignment scheme that relaxes the constraints on scale and
decouples the spatial assignment from the size of the ground-truth target.
Second, motivated by the structured prior of feature pyramids, we introduce
the one-stage cascade refinement network (OSCAR), which uses the high-level
head as soft proposals for the low-level refinement head. This allows OSCAR to
process the same target in a cascade coarse-to-fine manner. Finally, we
present a new research benchmark for infrared small target detection,
consisting of the SIRST-V2 dataset of real-world, high-resolution single-frame
targets, the normalized contrast evaluation metric, and the DeepInfrared
toolkit for detection. We conduct extensive ablation studies to evaluate the
components of OSCAR and compare its performance to state-of-the-art model-
driven and data-driven methods on the SIRST-V2 benchmark. Our results
demonstrate that a top-down cascade refinement framework can improve the
accuracy of infrared small target detection without sacrificing efficiency.
The DeepInfrared toolkit, dataset, and trained models are available at
https://github.com/YimianDai/open-deepinfrared.

###### Index Terms:

Infrared small target, one-stage cascade refinement, label assignment,
SIRST-V2 dataset, normalized contrast

##  I Introduction

Infrared imaging systems have become critical optoelectronic components in
surveillance systems due to their ability to detect objects at any time of
day, their long range, and their ability to capture heat sources [1]. Long-
distance targets of interest often appear as small targets in the field of
vision because of the requirements of early warning detection systems and the
resolution of infrared imaging systems. As a result, infrared small target
detection is essential for many applications and has been a focus of research
in recent years [2].

Since the seminal work of Miss Detection vs. False Alarm (MDvsFA) [3] and
Asymmetric Contextual Modulation Networks (ACMNet) [4], many Convolutional
Neural Network (CNN) based methods have been proposed [5] and model the
infrared small target detection as a semantic segmentation problem [6].
Despite their success, it has been difficult to further improve the detection
performance even with deeper backbones [7] and more powerful attention modules
[8].

We argue that the performance bottleneck in infrared small target detection
originates in the way the task is modeled as semantic segmentation[9], which
is unreasonable for infrared small targets in three ways. First, for real-
world applications, semantic segmentation is only an intermediary
representation; the ultimate objective is to locate the centroids of infrared
small targets so they can be used as input for the tracking task. Therefore,
segmentation integrity is only a rough approximation of detection accuracy.
Second, due to the dispersion effect of long-distance imaging, it is
impossible for humans to label infrared small targets with pixel-level
accuracy, and ground-truth annotations include inherent ambiguities. The
impact of these uncertain pixels is significant because they make up the
majority of positive samples and their loss dominates the training process.
Third, in order to segment infrared small targets accurately, inference must
be performed on high-resolution feature maps, making it computationally
intensive. Therefore, considering both detection accuracy and computational
overhead, it is better to model infrared small target detection as an object
detection problem rather than a semantic segmentation problem. However, the
performance of infrared small target detection still lags behind that of
normal-scale object detection by a significant margin.

The performance gap in infrared small target detection is mainly caused by
three factors. First, current label assignment paradigms, whether anchor-based
or anchor-free, tend to mistakenly label tiny groundtruth targets as
background, leading detectors to pay less attention to small targets. Second,
for infrared small targets, a small perturbation in the bounding box can cause
a significant disturbance in the Intersection over Union (IoU) metric, making
both bounding box regression and localization evaluation more challenging than
for large objects. Third, collecting real-world infrared small target images
is more challenging than collecting generic images, resulting in a lack of
non-synthetic data to train machine learning models.

(a) Faster RCNN (b) Cascade RCNN (c) RefineDet (d) OSCAR

Figure 1:  Comparisons of architecture frameworks for cascade detection
between existing works and ours. Existing works focus on cascade detection the
object on the same level of feature map, including (a): Faster RCNN [10],
which detects objects in two stages on C2 layer, (b) Cascade RCNN [11], which
detects objects in multiple stages on C2 layer, (c): RefineDet [12], which
parallelly detects the object on each feature maps of different scales in a
cascaded manner. In contrast, our OSCAR approach utilizes the highly
structured feature pyramids to perform cascaded detection in a top-down,
coarse-to-fine manner. Here, "Pool" denotes region-wise feature extraction,
"Cls" indicates the classification, "Reg" stands for the bounding box
regression. The backbone feature levels are denoted as "C2", "C3", and "C4",
while the levels of the feature pyramids are denoted as "P2", "P3", and "P4".

In this paper, we propose a comprehensive solution to address the challenges
faced in infrared small target detection. We first introduce a new research
benchmark, consisting of (1) SIRST-V2, a dataset of real-world, high-
resolution single-frame infrared small targets, (2) normalized contrast, a
reliable measure of localization quality that accounts for the inherent label
ambiguity in long-range imaging, and (3) DeepInfrared, an open source toolkit
specifically designed for infrared small target detection. We hope that our
benchmark and toolkit will facilitate the development of novel algorithms for
this task.

To address the performance gap in infrared small target detection, we also
propose a new label assignment scheme called All-Scale Pseudo-Box (ASPB). This
scheme decouples the ground-truth target size from the spatial assignment by
using scale-adaptive pseudo-boxes and also relaxes the scale constraints by
treating all target boxes at all scales as positive samples. This new scheme
is motivated by the observation that infrared small targets, such as those
with bounding boxes of $3\times 3\sim 5\times 5$, may not contain any feature
points when using a common feature map stride, such as $8$. This is a notable
difference from generic object detection. However, infrared small target
detection is actually a sparse target detection task with an occlusion-free
prior and a known scale. In the highly structured feature pyramids [13], the
multi-scale predictions can be viewed as multiple approximations of the target
in a coarse-to-fine manner, which allows for more accurate localization.

Finally, motivated by this cross-scale spatial correspondence, we propose the
One-Stage Cascade Refinement (OSCAR) network, a new data-driven framework for
single-frame infrared small target detection that aims to perform cascaded
bounding box regression while still maintaining the efficiency of one-stage
methods. In contrast to generic object detection, which adopts a divide-and-
conquer strategy to represent different scales of objectives with different
layers, OSCAR reformulates a feature pyramid’s multi-level predictions as
approximations of cascaded soft region proposals in a coarse-to-fine order
towards the same ground-truth bounding box. To address the class imbalance
issue and down-weigh well-classified negative samples, a top-down cross-head
modulation mechanism is used as a soft region proposal module. We also
incorporate a normalized contrast branch in the refining head, which
significantly improves performance by suppressing low-quality predicted
bounding boxes caused by pseudo-boxes.

Our main contributions in this work are:

  1. 1.

We propose a real-world high-resolution dataset for infrared small target
detection, called _SIRST-V2_.

  2. 2.

We design _normalized contrast_ , a novel evaluation metric for infrared small
targets, which takes into account centroid localization and robustness under
the ambiguities of long-range imaging.

  3. 3.

We provide an open-source toolkit, called _DeepInfrared_ , to support
researchers in developing algorithms for the SIRST-V2 benchmark and normalized
contrast metric.

  4. 4.

We propose _OSCAR_ , a new framework for single-frame infrared small target
detection that aims to perform cascaded bounding box regression in a top-down
cascade manner. By utilizing the conclusion-free prior in highly structured
feature pyramids, our method achieves state-of-the-art detection performance
while maintaining the efficiency of one-stage methods.

##  II Related Work

###  II-A Infrared Small Target Detection

####  II-A1 Traditional Methods

Infrared small target detection is typically formulated as either a target
enhancement problem [14, 15] or a background suppression problem [16]. These
methods can be further classified based on the modeling assumptions they use,
such as local contrast approaches [17] and sparse plus low-rank decomposition
methods [18, 19]. Local contrast methods model infrared small targets as
outliers in a slowly-changing background of highly correlated pixels [17].
Different variants of local contrast measures have been proposed to enhance
small targets and suppress pixel pulse noise and edges simultaneously [20].
Sparse plus low-rank decomposition methods exploit the non-local self-
correlation property of background patches by assuming that all background
patches come from a single subspace or a mixture of low-rank subspace clusters
[18, 21]. This allows target-background separation to be achieved by
decomposing the original data into a sparse target image and a low-rank
background image [22].

However, in real-world scenarios, there are many distractors that stand out as
outliers in the background alongside true targets [23]. Traditional approaches
lack the ability to differentiate between true targets and similar distractors
at a high semantic level, because they only use grayscale values or low-level
handcrafted features such as central differences and entropy [5]. When given
an infrared image without any targets, traditional approaches often yield
false positive detections because the contrasts they predict are not
semantically normalized. Another problem is that the contrast values of
different targets in multi-target settings cannot be normalized to the same
scale, which makes it more likely to miss targets with low contrast values and
results in a high number of missed detections. These modeling issues can cause
traditional methods to perform poorly when dealing with complicated scenarios.

####  II-A2 Deep Learning Methods

Infrared small target detection can now be modeled as a supervised machine-
learning problem due to the availability of publicly accessible datasets like
SIRST-V1 [4]. Currently, most studies focus on designing multi-scale feature
fusion models that can produce more refined target representations because of
the problem of insufficient intrinsic features [13]. To exchange high-level
semantics and low-level details, Dai et al. proposed an asymmetric contextual
modulation module that incorporates a bottom-up modulation pathway based on
point-wise channel attention and a top-down global contextual feedback [4]. To
enhance features that were matched with prior knowledge, Dai et al. proposed a
specialized nonlinear feature transformation layer, which they embedded into a
deep network [5]. Meanwhile, Li et al. designed an interactive module that is
densely nested, allowing for gradual interaction between high-level and low-
level elements [24]. Another significant challenge in infrared small target
detection is achieving a balance between miss detections and false alarms. To
address this, Wang et al. proposed splitting the task into two smaller tasks,
each of which would be handled by an adversarially trained model that focuses
on reducing either false alarms or miss detections [3].

Although these methods significantly outperform traditional methods,
formulating the detection problem as a semantic segmentation or image
translation [25] requires computationally expensive high-resolution feature
maps to recover pixel-level details. In contrast to the pixel-wise
segmentation methods mentioned above, we formulate the infrared small target
detection as a bounding box regression task, which is less affected by
labeling ambiguity. Furthermore, in the proposed OSCAR, the top-down
modulation is applied to the detection head on the label space rather than the
feature space in the neck, specifically the feature pyramid [13].

###  II-B Generic Object Detection

####  II-B1 Two-Stage Methods

The two-stage detector, such as Faster R-CNN [10], is a paradigm of object
detection algorithm that first generates a set of region proposals and then
classifies the objects within those proposals (see fig. 1 (a)). These
algorithms can be conceptualized as cascades, where the first stage removes a
significant number of background samples and the second stage classifies the
remaining regions. In addition, Cascade R-CNN [11] extends the two-stage
paradigm by using multiple sub-networks to progressively improve the quality
of the region proposals (see fig. 1 (b)). Our OSCAR follows the idea of a
cascade refinement, but it differs in two important aspects: (1) Instead of
using a sparse set of candidate object boxes for the next stage, OSCAR
performs soft region proposal on dense predictions by down-weighting easy
background samples; (2) Although it performs progressive refinement on the
predicted results, OSCAR remains a one-stage detector, not a two-stage method,
avoiding the additional computational overhead of general cascading pipelines.

####  II-B2 Single-Stage Methods

One-stage approaches, such as DenseBox [26] and CenterNet [27], bypass the
region proposal step and directly predict classification scores and bounding
box regression offsets, aiming for real-time performance while maintaining
high accuracy. There are some works that try to transplant the merits of two-
stage methods to one-stage approaches. For example, RefineDet [12] uses a
refinement module to mimic the second regression of two-stage approaches (see
fig. 1 (c)), and Consistent Optimization [28] reduces the gap between the
training and testing phases by tying consecutive classification targets to the
regressed anchors.

To some extent, our proposed OSCAR can be viewed as a continuation of this
concept, but it differs in two ways: (1) OSCAR abandons the commonly used
divide-and-conquer strategy, which assigns high-level feature maps to large
objects and low-level feature maps to small objects, and instead treats multi-
scale predictions as progressive approximations of the target. (2) to the best
of our knowledge, this is the first time that top-down cascade bounding box
refinement has been applied.

##  III DeepInfrared Eco-System

###  III-A SIRST-V2: Pay More Attention to Urban Scenarios

Figure 2: The representative infrared images from the SIRST-V2 dataset with
various backgrounds, excluding many simple cases. The  red boxes indicate
real targets, while the  green boxes indicate  background distractors that are
similar to the targets. Compared to SIRSTv1, SIRSTv2 includes more urban
scenes with many cranes, streetlights, and other non-target background
interference, which require a high-level semantic understanding to
distinguish.

First, we introduce our new SIRST-V2 dataset, which is the largest highest-
resolution dataset for single-frame infrared small target detection that we
are aware of. It is a milestone update of our previous SIRST-V1 dataset [4]
and contains 1024 representative images, mostly at a resolution of $1280\times
1024$. These images have been extracted from real-world videos of various
scenarios and provide a challenging testbed for infrared small target
detection in complex real-world scenarios.

fig. 2 illustrates some representative images from our new SIRST-V2 dataset.
In these images, it is evident that most infrared small targets are dim and
difficult to distinguish from their backgrounds due to the presence of
clutter. Additionally, the newly added urban scenes in SIRST-V2 contain many
background interference elements such as cranes and street lights that are
similar in appearance to the targets. This makes it challenging for
traditional approaches that rely on target saliency or low-rank and sparse
decomposition to effectively identify the targets in these complex scenarios.
A more effective approach would require the use of a network with a high-level
semantic understanding of the entire image to accurately distinguish targets
from non-target interferences.

###  III-B Normalized Contrast: A Measure for Locating Accuracy

(a) Sensitivity analysis of IoU (b) Dispersion effect and labeling ambiguity

Figure 3:  Drawbacks of IoU metric for infrared small targets. (a) The
sensitivity analysis of IoU. Each grid denotes a pixel, box $A$ indicates the
groundtruth box, box $B$ and $C$ stand for the predicted box with 1 pixel and
3 pixels diagonal deviation, respectively. (b) Illustration of the dispersion
effect and labeling ambiguity. Because the boundaries of infrared small
targets cannot be precisely defined due to long-range imaging, the groundtruth
labels (blue masks) are ambiguous and can be annotated in a variety of ways,
as illustrated in the figure (\raisebox{-0.7pt}{1}⃝, \raisebox{-0.7pt}{2}⃝,
\raisebox{-0.7pt}{3}⃝ three cases). Box $D$, $E$ denote the possible
groundtruth boxes, box $F$ indicates the predicted box. It can be seen that a
minor location deviation in either predicted box or groundtruth box will lead
to notable IoU drop, resulting in inaccurate localization evaluation. Despite
their different IoU values, box $E$ and $F$ share the same target centroid.

The IoU metric is highly skewed to large objects, very sensitive for infrared
small targets due to their tiny size (see fig. 3 (a)). A small perturbation in
either predicted box or groundtruth box can cause a significant change in the
IoU value, which leads to unreliable results. However, in real-world
applications, the main objective is to accurately locate the centroids of the
infrared small targets so that they can be used for tracking. Therefore, IoU
is only an intermediate representation and does not accurately reflect the
overall detection performance. As such, it is necessary to design a new
evaluation metric specifically for this task.

Figure 4:  The generation of normalized contrast (NoCo) map. First, the linear
local contrast is calculated to provide a distribution that matches the
appearance of the target. Then, a Gaussian is applied to give preference to
the geometric center of the target. Finally, a coarse label is used to
semantically normalize the labeled target region and some background pixels,
and the rest of the values are set to 0. This process results in a NoCo map
that is not sensitive to disturbances in the bounding box, making it a
reliable representation of the target.

As far as we are concerned, a good localization quality measure for infrared
small targets should have two key properties. First, it should be robust to
label ambiguity, meaning that it can accurately reflect the location of the
target even when the labels are not perfectly aligned with the target. Second,
it should be able to directly reflect the accuracy of the target centroid,
meaning that it should provide a clear indication of the precision of the
target’s location.

In our opinion, if the issues of computational efficiency and semantic
normalization are addressed, the local contrast map has the potential to be a
good quality measure for infrared small targets, as it offers a soft
distribution that is adaptable to the target’s shape and appearance. Despite
the numerous definitions of local contrast, their computational efficiency is
very low, as can be seen in table IX for local contrast measure (LCM) [17],
multi-scale patch-based contrast measure (MPCM) [29] and weighted local
difference measure (WLDM) [30] in terms of frame per second (FPS). This is
because these methods do not have groundtruth information, and therefore
require complex nonlinear filtering operations to suppress background noise.
In fact, due to the availability of bounding box information, using only the
linear normalization of the target bounding box region can also generate high-
quality local contrast maps. As a result, we propose to construct a new
representation, called _normalized contrast_ , by normalizing the local
contrast map according to the bounding box annotation, as shown in fig. 4.
Since the groundtruth box is known, we only need to calculate the normalized
contrast for the bounding box area and its surroundings, and set the rest of
the values to 0. Given a groundtruth box $T=(c_{x},c_{y},h_{t},w_{t})$, we can
obtain a larger region $R=(c_{x},c_{y},h_{t}+2h_{b},w_{t}+2w_{b})$ by
extending a specific number of pixels in all directions ($h_{b}$ in Y-axis,
$w_{b}$ in X-axis). $\mathcal{C}_{i}=R_{i}-\min(R)$ is the unnormalized local
contrast. $R_{i}$ denotes the grayscale of the $i$-th pixel in the region $R$
and $\min(R)$ indicates the minimum grayscale in region $R$. Here we set
$h_{t}=\gamma h_{b},w_{t}=\gamma w_{b}$, where $\gamma\in(0,1]$ is a constant.
The normalized contrast is defined as

$n_{i}=\frac{C_{i}\odot G_{\sigma}-\min(C_{i}\odot
G_{\sigma})}{\max(C_{i}\odot G_{\sigma})-\min(C_{i}\odot G_{\sigma})}$ (1)

The Gaussian distribution, denoted by $G_{\sigma}$, is used to impose a
central preference on the map, which helps to suppress the background clutters
in the surrounding region. The normalized contrast map depicts the normalized
distance from a given location to the center of the infrared small target,
providing a clear indication of the precision of the target’s location. In
other words, the groundtruth normalized contrast map is actually a look-up
table. Given a center point of a predicted bounding box, we can obtain a
normalized contrast value for evaluating the localization accuracy of the
predicted bounding box.

Note that our normalized contrast measure differs from the center-ness measure
used in the FCOS in several key ways.

  1. 1.

First, the centroid of infrared small targets is typically annotated with the
pixel of the highest signal-to-clutter ratio, rather than the geometric center
of bounding boxes. The requirement for centroid locating accuracy is often at
the sub-pixel level (smaller than 1 pixel), and using the center of the
bounding box as the centroid may result in an error of several pixels. To
reflect this domain knowledge, the normalized contrast is designed to directly
reflect the signal-to-clutter ratio of each pixel, which aligns with the
definition of the centroid.

  2. 2.

Second, the center-ness is purely a measure of coordinate distances, and a
small perturbation of the bounding box can cause a significant disturbance. In
contrast, our normalized contrast is a soft distribution adaptive to the
appearance of infrared small targets. By replacing the delta distribution
(step function) of the bounding box with a slowly-changing soft distribution,
we are able to provide a more stable measure of target localization.

The mean Normalized Contrast Average Precision (mNoCoAP) is a metric that we
propose to fairly and precisely assess the performance of algorithms for
detecting infrared small targets. It is similar to the mean Average Precision
(mAP) metric, which is widely used in generic object detection. The mNoCoAP is
calculated based on the Average Precision, which is the area under the
Precision-Recall curve. The key difference between the mAP and our mNoCoAP
lies in the definition of the True Positive criterion. In the mNoCoAP, we
replace the IoU with the NoCo value of the predicted bounding box. This means
that a prediction will be classified as a True Positive if the predicted NoCo
value is above a certain threshold. Following the common practice in the COCO
dataset, the mNoCoAP is an average of nine levels of $\delta$, which controls
the desired centroid locating precision, ranging from 0.1 to 0.9 with a step
size of 0.1

We have chosen the mNoCoAP as the main evaluation metric for our DeepInfrared
benchmark because it offers several advantages over existing metrics.

  1. 1.

First, the mNoCoAP is a uniform evaluation metric that supports fair and
preference-free evaluation of the performance of different detection
paradigms, such as bounding box regression, semantic segmentation, and
background suppression.

  2. 2.

Second, the mNoCoAP is different from the receiver operating characteristic
(ROC) curve in that it can not only be used to evaluate the performance of
producing as few false positives or false negatives as possible, but it can
also be used to evaluate the centroid locating accuracy of the tested method.
This is an important consideration when dealing with infrared small targets,
as the precision of the target’s location is often critical for successful
detection.

  3. 3.

Third, the mNoCoAP is more resistant to label ambiguity than other metrics
because it softens the harsh truncated boundary of an ambiguous ground-truth
bounding box. This is especially important in the case of long-range imaging,
where label ambiguity can be a significant issue.

###  III-C DeepInfrared Toolkit: An Open-Source Benchmark

In the field of generic computer vision, there are many open-source object
detection toolboxes, such as MMDetection and GluonCV, that implement and
integrate state-of-the-art detection algorithms. These toolboxes are widely
used by researchers to reproduce experiments and improve algorithms. However,
in the case of infrared small targets, there is currently no open-source
toolbox available. This lack of a common platform makes it difficult for
researchers to reproduce experiments and compare different algorithms, which
can hinder progress in this field.

To this end, we have developed DeepInfrared, a deep learning toolkit specially
developed for the infrared small target detection task based on PyTorch.
Compared with its counterparts in generic computer vision, our DeepInfrared
toolkit differs in the following two points:

  1. 1.

DeepInfrared provides well-trained models, training scripts, and logs of
state-of-the-art infrared small target detection algorithms. This allows
researchers to easily reproduce experiments and compare different algorithms.

  2. 2.

DeepInfrared offers more flexible choices for backbones and necks, and has
implemented various dataset loaders, attention blocks, and data augmentation
pipelines. It also includes evaluation metrics specifically designed for
infrared small target detection, and supports multiple annotation formats,
such as semantic segmentation, bounding boxes, and normalized contrast.

##  IV One-Stage Cascade Refinement Networks

###  IV-A All Scale Label Assignment with Pseudo-Boxes

Let $F_{i}\in\mathbb{R}^{H\times W\times C}$ be the feature maps at layer $i$
of a feature pyramid and $s_{i}$ be the total down-sampling stride of this
layer. A ground-truth bounding box is defined as
$B_{\mathrm{g}}=\left(c_{x},c_{y},h,w\right)$, where
$\left(c_{x},c_{y}\right)$ denotes the center of the bounding box, and $h,w$
are its height and width. To avoid using hyper-parameters related to anchor
boxes, we choose a one-stage anchor-free method as the baseline, which
generally regards the center (e.g., the center point or part) of the object as
foreground to define positives [31]. However, such center-based label
assignment has an implicit assumption that the positive area inside the
bounding box of size $h\times w$ must be larger than the square of the stride
$s_{i}$, namely, $h\times w\geq s_{i}^{2}$, so that at least one feature point
would be labeled as positive.

This is a valid assumption for generic object detection, but not for infrared
small targets, whose sizes generally vary from $2\times 2$ to $12\times 12$.
For example, in FCOS, the stride $s$ of the finest layer is $8$. If $hw<s^{2}$
for a given target, the bounding box of that target may not cover any feature
points, leading to it being mislabeled and resulting in low detection
performance, as shown in fig. 5. This is because a large portion of infrared
small objects in a certain scale are not being trained.

Figure 5:  Illustration of pseudo-box based label assignment. left: The
ground-truth box based label assignment defines positive samples for each
output spacial position by checking whether it falls into the box. middle: In
the pseudo-box based label assignment, each real target is assigned to at
least one feature point by using a pseudo-box that is larger than the ground-
truth box. right: The class imbalance problem between positive and negative
samples will be resolved by increasing the pseudo-box size, but at the expense
of increasing the number of low-quality predicted boxes.

This motivates us to _re-examine the label assignment for infrared small
targets_. In fact, label assignment can be further divided into scale
assignment and spatial assignment. In generic object detection, the scale
assignment assigns targets of different sizes to the appropriate feature
pyramid level for detection, and the spatial assignment then selects positive
sample locations at the selected level (e.g., center sampling in FCOS [32]).
However, infrared small targets do not exhibit large scale variation. Instead,
many targets are missed if the stride of the prediction layer is larger than
the target.

Motivated by these observations, we propose an All-Scale Pseudo-Box (ASPB)
based scheme to customize the label assignment for anchor-free infrared small
target detection to improve detection performance. The main idea is to
decouple the ground-truth target size from the label assignment. Specifically,
for scale assignment, we no longer consider the small targets predicted by
each scale as competitors, but rather as coarse-to-fine approximations under
different computational budgets. To this end, we relax the scale constraints
on the label assignment and treat all target boxes at all scales as positive
samples. For the spatial assignment, we introduce a pseudo-box with the same
center and size of $p\times p$ ($p\geq s$) to replace the ground-truth
bounding boxes of the tiny targets whose sizes are smaller than the feature
map stride, i.e., $h\times w<s^{2}$. This allows us to assign positive labels
to more feature points, improving detection performance for infrared small
targets.

Given a ground-truth bounding box $(c_{x},c_{y},h,w)$, the pseudo-box
$B_{\mathrm{p}}$ is defined as

$B_{\mathrm{p}}=\left\\{\begin{array}[]{lc}(c_{x},c_{y},h,w),&\text{ if
}hw>p^{2}\\\ (c_{x},c_{y},p,p),&\text{ otherwise }\end{array}\right.$ (2)

As illustrated in fig. 5, when $p=s$, at least one feature point will be
labeled as positive and the target mislabeling issue is resolved. As $p$
increases, i.e., $p>s$, more neighboring feature points will be labeled as
positive samples, which further relieves the sample imbalance issue but at the
cost of introducing more low-quality detected bounding boxes.

###  IV-B Top-Down Cascade Refinement Head

The performance of a one-stage detector is also affected by the severe sample
imbalance and inaccuracy of bounding box regression. The tricky problem is
that the solutions to these two issues have opposite requirements in terms of
scale. With our ASPB label assignment scheme, making predictions on the high-
level feature maps of the feature pyramid is a straightforward way to solve
the class imbalance problem because the number of negative samples will drop
exponentially as the down-sampling stride increases. However, this solution is
not ideal for improving bounding box regression accuracy because high-level
feature maps have low spatial resolution and are not suitable for precise
localization. Instead, because it is simpler to regress a local box than a
remote one, the bounding box regression task favors low-level fine feature
maps over high-level coarse feature maps. Simply dividing the classification
and localization tasks into different scales, namely classifying the feature
points at the high level and regressing the bounding box at the low level, is
not a workable solution because the one-to-many mapping relationship between
multi-scale feature points can lead to high rankings for low-quality bounding
boxes, degrading the final detection performance.

Figure 6:  The network architecture of OSCAR. In OSCAR, a feature pyramid’s
multi-level predictions are reformulated as approximations of cascaded soft
region proposals in a coarse-to-fine order towards the same infrared small
target. In addition, a normalized contrast branch is augmented to suppress
low-quality predicted bounding boxes caused by pseudo-boxes. $H\times W$ is
the height and width of feature maps. "/s" ($s=4,8,16,32$) is the down-
sampling ratio of the feature maps at the level to the input image. All the
numbers, for instance, are computed using an input of $800\times 1024$.

The proposed OSCAR Head is a solution to the issue of combining the efficiency
of a one-stage detector with the merits of region proposal and cascaded
bounding box refinement from two-stage detectors. The OSCAR Head consists of a
high-level soft proposal head and a low-level refinement head, as shown in
fig. 6. The soft proposal head predicts a classification score
$\bm{p}^{h}{x,y}$ and a coarse bounding box $\bm{b}^{h}{x,y}$ for each feature
point $(x,y)$. These coarse boxes are then passed as initial guesses with top-
down modulation scores to the refinement head, where they are further refined
by predicting a category score $\bm{p}^{l}{x,y}$ and a refined bounding box
$\bm{b}^{l}{x,y}$.

####  IV-B1 Top-Down Score Modulation

To address the extreme imbalance between small targets and background during
training, we propose a top-down score modulation mechanism as a soft region
proposal module. This module is designed to improve the accuracy of the
detector by using the more accurate score of the coarse layer to modulate the
less accurate score of the fine layer. Compared to the serialized region
proposal operation in two-stage detectors, which involves cropping feature
maps and performing ROI Pooling, our soft region proposal can be implemented
in a single forward pass and is therefore more efficient. This allows for
parallel processing, which can improve the overall performance of the
detector.

Given a coarse feature map $X^{h}$ of size $(H^{h},W^{h})$ and a fine feature
map $X^{l}$ of size $(h^{l},w^{l})$, both are the last feature maps before
prediction in corresponding layers. The top-down score modulation weight is
defined as follows:

$T(X^{h})=\sigma(\mathrm{Conv}_{2}(\delta(\mathrm{Conv}_{1}(\mathrm{Up}(X^{h})))))$
(3)

where $\mathrm{Conv}_{1}$ and $\mathrm{Conv}_{2}$ are convolutional layers
whose kernel sizes are $\frac{C}{4}\times C\times 3\times 3$ and
$C\times\frac{C}{4}\times 3\times 3$, respectively. The $\delta$ denotes the
Rectified Linear Unit (ReLU) and $\sigma$ is a Sigmoid function.
$\mathrm{Up}(\cdot)$ denotes the up-sampling operation to interpolate the
high-level feature maps to the same size of low-level feature maps. Then the
modulated low-level classification scores can be obtained via

$S^{l}=\sigma(\mathrm{Conv}(T(X^{h})\odot X^{l}))$ (4)

where $\odot$ is the element-wise multiplication. By modulating the scores of
the fine layer with the more accurate scores from the high-level features, the
top-down modulation approach effectively eliminates false negatives and
reduces the impact of class imbalance on the detector’s performance.

####  IV-B2 Top-Down Cascaded Regression

One-stage methods typically rely on a single step of regression using
different feature layers to predict the locations and sizes of objects with
varying scales. However, this approach can be ineffective in certain
challenging scenarios, particularly for infrared small targets. To address
this limitation, we propose a cascaded regression strategy for more accurate
prediction of target locations and sizes. To improve the regression accuracy
in the fine-level feature maps, we use the high-level layer to first predict
coarse locations and sizes of underlying infrared small targets. This initial
coarse prediction serves as a learnable anchor for the fine-level layer to
further refine the predictions.

The decoded coarse bounding box is denoted as
$B^{\mathrm{h}}=\left(x_{0}^{h},y_{0}^{h},x_{1}^{h},y_{1}^{h}\right)$. To
align with the next-level refined feature maps, we first need to up-sample the
coarse bounding boxes. Here, we denote the interpolated bounding box as
$\tilde{B}^{\mathrm{h}}=\left(\tilde{x}_{0}^{h},\tilde{y}_{0}^{h},\tilde{x}_{1}^{h},\tilde{y}_{1}^{h}\right)$.
To further produce precise object locations and sizes, we feed both the coarse
features and the coarse boxes to the refinement head. Formally, if a feature
point is associated to a groundtruth box
$B_{\mathrm{g}}=\left(x_{0},y_{0},x_{1},y_{1}\right)$, the regression targets
for this feature point can be formulated as

$\displaystyle\Delta x_{0}$
$\displaystyle=(x_{0}-\tilde{x}_{0}^{h})/s,\quad\Delta
x_{1}=(x_{1}-\tilde{x}_{1}^{h})/s$ (5) $\displaystyle\Delta y_{0}$
$\displaystyle=(y_{0}-\tilde{y}_{0}^{h})/s,\quad\Delta
y_{1}=(y_{1}-\tilde{y}_{1}^{h})/s.$

where $s$ is the stride of the low-level feature map. During inference, the
regressed target $\hat{B}^{l}$ is predicted via concatenating the up-sampled
feature maps, denoted as

$\hat{B}^{l}=\mathrm{Conv}(\mathrm{Concat}(\mathrm{Up}(X^{h}),X^{l}))$ (6)

where $\mathrm{Concat}$ is the concatenating operation and $\mathrm{Up}$
denotes the feature map up-sampling. To some extent, the coarse boxes can be
viewed as dynamic anchors.

###  IV-C Normalized Contrast Prediction

The proposed ASPB label assignment method significantly improves the class
imbalance issue [33]. However, it also introduces many low-quality predicted
bounding boxes by assigning positive labels to feature points that are far
away from the center of an infrared small target. To suppress these low-
quality bounding boxes, we propose to add a new branch to the refinement head
that predicts the localization quality score of a detected bounding box.

It is important to note that the calculation of center-ness requires that the
feature point be within the ground-truth bounding box. However, for infrared
small targets, this requirement may not always be satisfied. As shown in fig.
5, when the target size is smaller than the feature map stride, the center-
ness cannot be calculated. In contrast, our normalized contrast is calculated
using the original input image, which can be thought of as a softened semantic
segmentation mask. No matter how the feature map is down-sampled, the
normalized contrast can always be calculated.

Loss Function. The overall loss is defined as follows:

$\displaystyle L$ $\displaystyle=\frac{1}{N^{h}_{\text{pos
}}}\sum_{x,y}L_{\mathrm{cls}}\left(\bm{p}^{h}_{x,y},c_{x,y}^{h,*}\right)+\mathbbm{1}_{\left\\{c_{x,y}^{h,*}>0\right\\}}L_{\mathrm{reg}}\left(\bm{t}^{h}_{x,y},\bm{t}_{x,y}^{h,*}\right)$
(7) $\displaystyle+\frac{1}{N^{l}_{\text{pos
}}}\sum_{x,y}L_{\mathrm{cls}}\left(\bm{p}^{l}_{x,y},c_{x,y}^{l,*}\right)+\mathbbm{1}_{\left\\{c_{x,y}^{l,*}>0\right\\}}L_{\mathrm{reg}}\left(\bm{t}^{l}_{x,y},\bm{t}_{x,y}^{l,*}\right)$
$\displaystyle+\lambda
L_{\mathrm{noco}}\left(n^{l}_{x,y},n_{x,y}^{l,*}\right)$

where $L_{\mathrm{cls}}$ is focal loss [33], $L_{\mathrm{reg}}$ the IOU loss,
and $L_{\mathrm{noco}}$ is the quality focal loss as in [31].
$N_{\mathrm{pos}}$ denotes the number of positive samples and $\lambda$ being
1 in this paper is the balance weight for the regression loss.
$\mathbbm{1}_{\left\\{c_{i}^{*}>0\right\\}}$ is the indicator function, being
1 if the $c_{i}^{*}>0$ and 0 otherwise.

Inference. During testing, the predicted normalized contrast is combined with
the two-level classification scores to calculate the final score for each
detected bounding box. As a result, bounding boxes that are far from the
center of an infrared small target may be given lower scores due to the effect
of the normalized contrast. Therefore, the non-maximum suppression is likely
to filter out these low-quality bounding boxes, which can greatly improve the
detection performance.

##  V Experiments

###  V-A Experimental Settings

We have compared the OSCAR network to several model-driven methods for
infrared small target detection. These methods include the method based on
facet kernel and random walker (FKRW) [34], LCM [17], MPCM [29], WLDM [30],
infrared patch-image model (IPI) [18], non-negative IPI model via partial sum
minimization of singular values (NIPPS) [35], reweighted infrared patch-tensor
model (RIPT) [21], and stable multi-subspace learning (SMSL) [36]. The
detailed hyper-parameter settings for these methods are listed in table I,
which shows the parameters that were determined by an exhaustive search on the
trainval set of the SIRSTv2 dataset.

TABLE I: Detailed hyper-parameter settings of model-driven methods for comparison. Methods | Hyper-parameter settings
---|---
FKRW [34] |  $K=5$, $p=6$, $\beta=200$, window size: $11\times 11$
LCM [17] |  Cell Size: $3\times 3$, threshold factor: $k=3$
MPCM [29] |  $N=1,3,...,9$, threshold factor: $k=13$
WLDM [30] |  $L=4,m=2,n=2$, threshold factor: $k=10$
IPI [18] |  Patch size: 50$\times$50, stride: 20, $\lambda{\rm{=}}L{\rm{/min(m,n}}{{\rm{)}}^{1/2}}$,$L=2.5$, $\varepsilon{\rm{=1}}{{\rm{0}}^{{\rm{-7}}}}$
NIPPS [35] |  Patch size: 50$\times$50, stride: 20, $\lambda=L/\sqrt{\min{(m,n)}}$, $L=1.2$, energy constraint ratio: $r=0.005$
RIPT [21] |  Patch size: 50$\times$50, stride: 10, $\lambda=L/\sqrt{\min{(I,J,P)}}$, $L=0.001$, $h=10$, $\epsilon$=0.01, $\varepsilon=10^{-7}$, threshold factor:$k=10$
SMSL [36] |  Patch size: 30$\times$30, $\lambda=2\times L/\sqrt{\min{(m,n)}}$, $L=5.0$, threshold factor: $k=1$

We also have conducted a thorough comparison of the OSCAR network with several
state-of-the-art methods for infrared small target detection. These methods
include the Infrared Small-Target Detection U-Net (ISTDU) [37], ACMNet [4],
Attentional Local Contrast Networks (ALCNet) [5], FCOS [32], Faster R-CNN
[10], QueryDet [38], and Gaussian Receptive Field based Label Assignment
(RFLA) [39]. To ensure fair comparisons, we have used our DeepInfrared toolkit
to produce all of the results. We have adopted the default hyper-parameters in
all cases, and have used the standard 1x learning schedule (12 epochs) for the
ablation study. The training and testing details for each method are based on
the descriptions in the original papers. All of the trained models, codes, and
training logs, are available in our DeepInfrared toolkit. This allows
researchers to easily reproduce our experiments and compare the performance of
different algorithms.

###  V-B Ablation Study

####  V-B1 Impact of Pseudo-Box Based Spatial Assignment

One of the key performance bottlenecks of current anchor-free detectors for
infrared small targets is the issue of target mislabeling caused by existing
center-based label assignment methods [32]. In this part, we show that this
problem can be largely resolved by replacing the ground-truth target size with
a pseudo-box in spatial label assignment. To do this, we will investigate the
following two sub-questions: (1) To what extent does the target mislabeling
issue hurt the performance of a detector? (2) What is the best pseudo-box size
for infrared small target detection?

We used a simple anchor-free detector as the baseline for this ablation study,
which can be seen as a simplified version of FCOS without the centerness
branch and only one layer of prediction, as shown in fig. 7 (a). The results
of our experiments are reported in table II, where "None" denotes the use of
the original center-based label assignment scheme, and $p=s$ means that at
least one nearest neighbor feature point labeled as a positive sample. As
shown in the table, using a pseudo-box in spatial label assignment can improve
the AP from 71.9% to 75.6%. This gap indicates the performance degradation
brought by the label noise caused by the center-based label assignment, which
mislabels a portion of real targets as background samples.

(a) Anchor-free baseline

(b) Decoupled head

Figure 7: The illustration of network architectures used for ablation study.
(a) The anchor-free baseline detector, (b) Decoupled head that decouples the
classification and regression tasks into separate levels. The high-level
branch classifies feature points on coarse feature maps, while the low-level
branch regresses bounding boxes on refined feature maps.

To determine the optimal pseudo-box size, we varied the value of $p$ in our
experiments and measured the resulting detection performance. As shown in
table II, the detection performance increases as we increase the value of $p$
from $p=s$. This is because as more feature points are set as positive
samples, the imbalance problem of positive and negative samples is relieved.
However, when $p$ is too large, those positive feature points that are far
from the target produce many low-quality detections, thus reducing the final
detection performance. From the results, we consider $p=1.5s$ as a suitable
size and use it in the following experiments.

TABLE II: Impact of pseudo-box sizes on our ASPB scheme. Pseudo-Box Size | AP | $\mathrm{AP}_{10}$ | $\mathrm{AP}_{30}$ | $\mathrm{AP}_{50}$ | $\mathrm{AP}_{70}$ | $\mathrm{AP}_{90}$
---|---|---|---|---|---|---
None | 71.9 | 88.9 | 86.8 | 77.7 | 66.2 | 32.9
$p=s$ | 75.6 | 89.2 | 86.3 | 80.3 | 74.9 | 37.7
$p=1.5s$ | 77.6 | 93.6 | 92.6 | 83.7 | 73.9 | 35.5
$p=2s$ | 71.2 | 90.7 | 83.2 | 77.2 | 67.5 | 27.0

####  V-B2 Reasonableness of Top-Down Refinement

We further investigate the reasons for the performance gains of top-down
modulation in our OSCAR head. Specifically, we examine two questions: (1) Is
it regression or classification that requires higher resolution feature maps
more? (2) How do feature map resolutions affect classification and regression
separately?

(a) The architecture of SimpleGrid (b) SimpleGrid Label Assignment (c) Oracle
Localization vs BBox Regression

Figure 8:  Illustration for the ablation study of the reasonableness of our
top-down refinement. In (a), we show the network architecture of the
SimpleGrid, which is equipped with oracle localization to individually inspect
the classification performance over various feature map sizes. In (b), we show
the label assignment for the SimpleGrid, where a patch is considered a
positive sample if the target centroid lies within it. In (c), we compare the
performance of SimpleGrid and the baseline, demonstrating that for infrared
small target detection, a larger feature map improves the accuracy of bounding
box regression rather than the accuracy of classification. In contrast, the
latter faces a more serious class imbalance problem as the feature map becomes
larger.

To study how feature map resolution affects the classification sub-task, we
employ a simple network called SimpleGrid, whose architecture is shown in fig.
8(a). We endow it with the ability of oracle localization so that the
performance can still be expressed in terms of mNoCoAP. Specifically, we
extract the groundtruth coordinates and use them to identify the target
centroid as long as the feature points are accurately classified. The label
assignment scheme for the SimpleGrid is shown in fig. 8(b). We divide an input
image into a regular grid of equally sized patches. A patch is considered a
positive sample if the target centroid lies within it. We keep the input image
size constant, but use different down-sampling rates for the network to obtain
the prediction results with different feature map sizes, as shown in table
III.

TABLE III: Backbone down-sampling schemes on feature map resolutions. Down-sampling | 4 | 8 | 16 | 32 | 64
---|---|---|---|---|---
Stem | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$
Max-Pooling | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$
Stage-1 |  |  |  |  | $\downarrow 2$
Stage-2 |  | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$
Stage-3 |  |  | $\downarrow 2$ | $\downarrow 2$ | $\downarrow 2$
Stage-4 |  |  |  | $\downarrow 2$ | $\downarrow 2$

The results are shown in fig. 8(c). As can be seen, with perfect oracle
localization, the performance of SimpleGrid, which only needs to classify
feature points, increases as the feature map resolution decreases. In
contrast, the performance of the baseline, which performs both bounding box
regression and classification, decreases as the down-sampling rate increases.
This answers our previous question: it is bounding box regression, not
classification, that requires high-resolution feature maps. In fact, the
classification accuracy drops as the feature map size increases due to severe
class imbalance.

Based on these observations, we can draw the following conclusions. First,
low-resolution feature maps are sufficient for preserving infrared small
targets. Therefore, if the goal is to determine the presence or absence of a
target in a certain region rather than its precise location, low-resolution
feature maps are more appropriate. Second, since the high-level layers can
classify feature points more accurately, they can be used to modulate the
features of the lower layers, which is the source of the performance gains
from our top-down modulation.

####  V-B3 Necessity of Joint Classification and Regression

In our previous ablation study, we adopted a top-down modulation approach to
jointly predict the target category and location. This raises the question of
whether the classification and bounding box regression sub-tasks can be
directly conducted on different layers, using coarse high-level feature maps
for classification and refined low-level feature maps for regression. To
investigate this, we implemented a decoupled head, as shown in fig. 7 (b), and
compared it to the anchor-free baseline. This allowed us to evaluate the
necessity of joint classification and regression.

TABLE IV: Ablation study for joint classification and regression. | AP | $\mathrm{AP}_{10}$ | $\mathrm{AP}_{30}$ | $\mathrm{AP}_{50}$ | $\mathrm{AP}_{70}$ | $\mathrm{AP}_{90}$
---|---|---|---|---|---|---
Baseline | 75.6 | 89.2 | 86.3 | 80.3 | 74.9 | 37.7
DecoupledHead | 55.7 | 81.9 | 66.0 | 59.9 | 48.7 | 14.5

In table IV, we can see that decoupling the classification and localization
sub-tasks into different layers leads to a significant drop in the final
detection performance, from 75.6% to 55.7%. We believe that this decline is
due to the one-to-many relationship between high and low-level feature points,
which allows low-quality bounding boxes predicted by redundant feature points
to rank highly, thus significantly reducing the final average precision. This
sharp decline highlights the importance of jointly predicting the target
category and location.

####  V-B4 Impact of Top-Down Cascade Refinement

The proposed OSCAR model has two major differences compared to other cascaded
detectors: top-down score modulation (TDSM) and top-down cascaded regression
(TDCR). TDSM views the high-level predictions as coarse estimates of the
infrared small targets in the low-level refinement head, while TDCR uses a
multi-stage bounding box regression to predict the final bounding box more
accurately than the single-stage method.

In this part, we investigate the impact of TDSM and TDCR on the detection
performance of OSCAR. The results are shown in table V. As can be seen from
the table, both TDCR and TDSM alone improve the detection performance of the
baseline. Combining the two gives the best performance, improving the baseline
from 75.6% to 79.2%. We attribute the performance gain from TDSM to the fact
that more accurate classification information at higher levels suppresses the
scores of target-like background interferences. The performance gain from TDCR
comes from its ability to predict the final bounding box more accurately using
a multi-stage approach.

TABLE V: Ablation study for Impact of Top-Down Cascade Refinement. TDSM | TDCR | AP | $\mathrm{AP}_{10}$ | $\mathrm{AP}_{30}$ | $\mathrm{AP}_{50}$ | $\mathrm{AP}_{70}$ | $\mathrm{AP}_{90}$
---|---|---|---|---|---|---|---
✕ | ✕ | 75.6 | 89.2 | 86.3 | 80.3 | 74.9 | 37.7
$\checkmark$ | ✕ | 78.5 | 88.9 | 86.7 | 84.2 | 71.9 | 54.0
✕ | $\checkmark$ | 78.2 | 93.1 | 89.4 | 84.7 | 78.4 | 36.2
$\checkmark$ | $\checkmark$ | 79.2 | 94.8 | 89.3 | 86.3 | 74.4 | 43.2

####  V-B5 With or Without Normalized Contrast

In this part, we investigate the impact of the normalized contrast branch. As
mentioned previously, this branch is designed to suppress the low-quality
detected bounding boxes produced by feature points that are far from the
infrared small targets. These false alarms are introduced by the pseudo-boxes
in our ASPB label assignment scheme. The results of this investigation are
shown in table VI. In this table, "None" indicates that no normalized contrast
is used, "NoCo" denotes that the normalized contrast is used, "NoCo†" stands
for that the normalized contrast is multiplied with the interpolated
classification scores from the soft proposal head as the final score, and
"Centerness†" means that the centerness is used instead of the normalized
contrast.

As can be seen from the table, the augmented branch effectively suppresses the
false alarms introduced by the pseudo-boxes, which improves the performance
from 79.2% to 80.3%. Additionally, the normalized contrast branch achieves a
higher score than the centerness, which is due to the fact that in our mNoCoAP
metric, the centroid of an infrared small target is annotated with the pixel
of the highest signal-to-clutter ratio, while the centerness is merely a sub-
optimal proxy since the center pixel does not necessarily have the highest
local contrast.

TABLE VI: Ablation study for the proposed normalized contrast branch. | AP | $\mathrm{AP}_{10}$ | $\mathrm{AP}_{30}$ | $\mathrm{AP}_{50}$ | $\mathrm{AP}_{70}$ | $\mathrm{AP}_{90}$
---|---|---|---|---|---|---
None | 79.2 | 94.8 | 89.3 | 86.3 | 74.4 | 43.2
NoCo | 79.9 | 90.8 | 89.2 | 84.6 | 77.3 | 49.1
Centerness† | 79.7 | 92.9 | 90.1 | 81.9 | 75.5 | 50.9
NoCo† | 80.3 | 91.7 | 89.5 | 85.9 | 77.4 | 47.9

####  V-B6 Choice of backbone depth and prediction level

In generic computer vision, the detection performance is usually improved
gradually as the depth of the network increases. However, this is not the case
for infrared small targets. The features that highlight small objects tend to
be extinguished because of the down-sampling operations in the backbone, which
can result in the contamination of the features of infrared small targets by
noise in the background in deep networks. In addition, different prediction
levels result in very different computational effort. For example, the
computational load of our OSCAR head on the P2 layer is 300% more than
predicting on the P3 layer. Therefore, it is necessary to explore the optimal
network depth and prediction level to balance the performance and efficiency.

In table VII, we provide the mNoCoAP values and GFLOPS of our OSCAR network on
different backbones and prediction levels. Here, $C$ is the output channel
number of the feature pyramid and $s$ is the feature map stride of the
refinement layer. As shown in the table, the detection performance of the
network with the same parameter settings decreases as the network depth
increases. Therefore, the ResNet-18 is already sufficient for the infrared
small target detection task, and it is more important to focus on how to more
accurately regress targets than to use a more powerful backbone to improve
detection performance. When we predict the target at a shallower layer, the
detection performance is better, but at the cost of a 2-3 times higher
computational effort. Finally, we can significantly reduce the computation on
top of the P2 layer by reducing the number of channels in the head. As shown
in the table, by halving the number of channels, we can achieve better
performance (83.52% vs 82.77%) while maintaining less computation than the P3
layer prediction (25.36 vs 25.55 GFLOPS).

TABLE VII: Ablation study on OSCAR’s choice of backbone depth and prediction level Backbone | ResNet-18 | ResNet-34 | ResNet-50 | ResNet-101
---|---|---|---|---
Refining Layer | mNoCoAP | GFLOPS | mNoCoAP | GFLOPS | mNoCoAP | GFLOPS | mNoCoAP | GFLOPS
P4 $(C=256,s=16)$ | 74.38 | 14.85 | 72.66 | 24.54 | 71.36 | 28.37 | 70.82 | 47.80
P3 $(C=256,s=8)$ | 82.77 | 25.55 | 82.04 | 35.23 | 80.60 | 39.06 | 80.24 | 58.54
P2 $(C=256,s=4)$ | 85.02 | 68.32 | 82.98 | 78.00 | 82.33 | 81.84 | 82.12 | 101.31
P2 $(C=128,s=4)$ | 83.52 | 25.36 | 82.15 | 35.04 | 80.95 | 38.12 | 80.75 | 57.59

####  V-B7 Choice of Hyper-paramter $\lambda$

In this part, we study the choice of the hyper-parameter $\lambda$, which is
the weight of the loss of the normalized contrast branch in the OSCAR network.
As shown in table VIII, the tested OSCAR networks (with a ResNet-50 backbone
and predicting on the P2 head) achieve the highest mNoCoAP value when
$\lambda=10^{3}$. As a result, we choose this value as the default parameter.

TABLE VIII: Ablation study for Impact of Top-Down Cascade Refinement. $\lambda$ | AP | $\mathrm{AP}_{10}$ | $\mathrm{AP}_{30}$ | $\mathrm{AP}_{50}$ | $\mathrm{AP}_{70}$ | $\mathrm{AP}_{90}$
---|---|---|---|---|---|---
$10^{2}$ | 80.2 | 92.4 | 88.5 | 85.5 | 78.3 | 49.9
$10^{3}$ | 80.6 | 93.4 | 91.6 | 81.9 | 77.3 | 51.6
$10^{4}$ | 78.5 | 94.5 | 88.6 | 85.7 | 77.3 | 43.9
$10^{5}$ | 77.2 | 92.9 | 89.1 | 82.1 | 74.0 | 42.6

###  V-C Comparison with State-of-the-Arts

In this subsection, we compare our OSCAR model with several model-driven and
data-driven methods, including some state-of-the-art deep networks. The
mNoCoAP values and inference times are listed in table IX. The inference speed
comparison is provided in terms of FPS. Traditional methods are evaluated on
the original size of the images, while deep learning methods are evaluated on
resized images with a specific width and height, e.g., $(1000,600)$.

####  V-C1 Model-driven versus Data-driven

It can be seen that data-driven methods outperform in terms of both detection
performance and detection speed. This large performance gap shows that the
global sparsity and contrast priors no longer hold for model-driven methods
when dealing with complicated images, yet data-driven methods can still
produce satisfying detection results. We attribute this performance gain to
two main factors. First, data-driven methods use deep networks to extract
high-level semantics from the images, which allows them to better distinguish
real targets from false alarms that look similar to infrared small targets.
Second, data-driven methods have fewer hyper-parameters than model-driven
methods, making them more robust to variations in the scene.

In addition, traditional methods, particularly low-rank approaches, require
dozens of iterations to make predictions, whereas deep networks can make
predictions in a single forward pass. This makes data-driven methods faster
and more efficient. Furthermore, the computational complexity of traditional
methods grows exponentially with the size of the input image, while data-
driven methods maintain a linear relationship with the image size. As infrared
imaging technology advances, infrared images of high resolutions, even higher
than $1280\times 1024$ in SIRST-V2, will become more common in the future.
Therefore, data-driven methods should receive more attention from researchers,
both in terms of detection performance and detection speed.

TABLE IX: Comparison with other state-of-the-art methods on mNoCoAP. Methods | Model-Driven | Data-Driven |
---|---|---|---
Local Contrast | Low-Rank + Sparse | Semantic Seg. | BBox Regression
FKRW | LCM | MPCM | WLDM | IPI | NIPPS | RIPT | SMSL | ISTDU | ACMNet | ALCNet | FCOS | Faster RCNN | RFLA | QueryDet | OSCAR
AP | 27.8 | 20.7 | 32.2 | 11.2 | 37.7 | 33.5 | 29.2 | 20.6 | 65.1 | 61.4 | 66.4 | 73.1 | 73.4 | 78.8 | 80.1 | 85.2
$\mathrm{AP}_{10}$ | 31.9 | 31.1 | 34.4 | 13.4 | 39.6 | 35.5 | 34.6 | 23.1 | 83.3 | 85.5 | 81.1 | 87.7 | 88.7 | 92.5 | 93.0 | 93.8
$\mathrm{AP}_{20}$ | 31.6 | 30.3 | 33.8 | 13.1 | 39.6 | 35.4 | 33.8 | 22.7 | 82.6 | 82.5 | 79.5 | 87.0 | 88.7 | 91.5 | 93.0 | 93.4
$\mathrm{AP}_{30}$ | 30.5 | 28.6 | 33.5 | 13.1 | 39.0 | 34.8 | 33.3 | 22.2 | 81.3 | 79.5 | 77.5 | 84.7 | 86.7 | 90.2 | 91.5 | 93.3
$\mathrm{AP}_{40}$ | 29.8 | 27.4 | 33.3 | 13.1 | 38.3 | 34.2 | 33.3 | 22.2 | 79.2 | 79.5 | 76.1 | 81.0 | 84.4 | 88.1 | 89.1 | 90.4
$\mathrm{AP}_{50}$ | 28.5 | 23.3 | 33.3 | 12.5 | 38.3 | 34.2 | 32.1 | 21.2 | 76.2 | 70.6 | 72.1 | 77.5 | 80.7 | 84.0 | 86.5 | 88.8
$\mathrm{AP}_{60}$ | 28.5 | 19.9 | 33.3 | 11.3 | 38.3 | 33.7 | 28.8 | 20.5 | 69.4 | 59.8 | 68.4 | 71.4 | 74.8 | 79.9 | 83.0 | 86.2
$\mathrm{AP}_{70}$ | 27.2 | 13.2 | 32.2 | 10.1 | 38.3 | 33.0 | 26.1 | 19.5 | 57.4 | 48.4 | 57.1 | 69.6 | 66.8 | 72.8 | 76.6 | 82.3
$\mathrm{AP}_{80}$ | 26.6 | 7.8 | 31.2 | 8.1 | 36.2 | 31.3 | 23.5 | 19.0 | 39.9 | 31.1 | 50.4 | 59.8 | 54.1 | 63.7 | 63.6 | 76.9
$\mathrm{AP}_{90}$ | 15.2 | 4.8 | 24.9 | 6.0 | 32.0 | 29.5 | 17.2 | 17.3 | 16.3 | 16.0 | 35.2 | 39.4 | 35.4 | 46.6 | 45.2 | 60.2
FPS | 0.52 | 7.93 | 1.11 | 0.12 | 0.05 | 0.05 | 0.11 | 0.37 | 19.23 | 29.41 | 2.62 | 28.57 | 4.25 | 4.21 | 11.90 | 34.6

####  V-C2 Among Data-Driven Methods

Then we compare the proposed OSCAR method with other deep learning-based
approaches. Our results show that: (1) Semantic segmentation approaches are
outperformed by all bounding box regression methods. We believe the reason for
this lies in the impact of uncertainty in the labels of pixels caused by the
dispersion effect of long-distance imaging, which makes up the vast majority
of positive samples and predominates the training loss. Since the objective of
this task is to locate the centroids of infrared small targets rather than
their entire contours, bounding box regression is a more reasonable detection
pipeline for infrared small targets. (2) Our proposed OSCAR method outperforms
FCOS by 7.2%. This performance gain demonstrates the effectiveness of our
approach in achieving the primary goal of this paper, which is to improve
detection results by regressing the target bounding box coordinates in a two-
step cascade. (3) Compared to the two-stage Faster R-CNN, which also regresses
the final coordinates based on initial guesses, our proposed OSCAR method
achieves a much better result. We believe this is because OSCAR avoids using
anchor-related hyper-parameters, which are known to be sensitive to the final
detection performance. Additionally, Faster R-CNN assigns multiple samples per
feature point, whereas OSCAR only assigns one, which makes the class imbalance
more severe for Faster R-CNN and leads to inferior classification accuracy.

In addition to outperforming FCOS, our proposed OSCAR method also outperforms
both RFLA and QueryDet. It is important to note that, although both RFLA and
QueryDet are designed for small objects, the small objects they assume still
have a scale gap with the infrared small targets used in this study. RFLA is
generally designed for objects with sizes ranging from $16\times 16$ to
$32\times 32$, while OSCAR is typically used for objects smaller than
$12\times 12$, mostly in the range of $3\times 3$ to $5\times 5$. Therefore,
the poorer performance of RFLA is predictable because it predicts small
targets in the P3 head at a stride of $8$, which is too large for detecting
infrared small targets. As for QueryDet, our OSCAR method differs in the way
it utilizes more information from the high-level head. We believe this is the
reason why OSCAR achieves better performance than QueryDet. In QueryDet, only
the existence of small objects is passed down from the P3 head to the P2 head,
whereas OSCAR fully uses the predictions (both classification and regressed
bounding boxes) produced by the coarse head.

Figure 9:  Visualization of predicted results by tested methods on four real
images containing infrared small targets. In the input images,  real targets
are indicated by the  red boxes, whereas  background distractors (target-like
background components) are indicated by the  green boxes. In the rest
detection results,  real circles denote the  True Positive (TP) detections;
green circles denote the  False Positive (FP) detections;  yellow circles
denote the  False Negative (FN)  detections. The better the detection
performance is indicated by the greater number of red circles and the fewer
green and yellow circles. In general, low-rank methods outperform local
contrast methods in terms of missed detections, but are still outperformed by
deep learning approaches in suppressing false alarms. It is worth noting that
FCOS tends to miss small targets, while the proposed OSCAR is able to detect
them well. We believe this improvement in performance is due to our pseudo-box
based label assignment, which addresses the issue of mislabeling targets. For
best viewing, the image should be zoomed in on a computer screen.  Figure 10:
Visualization of predicted results by tested methods on four background images
without any real target. The goal of this experiment set is to evaluate the
ability of the methods to suppress false alarms. In the input images,
background distractors (target-like background components) are indicated by
the  green boxes. In the rest detection results,  green circles denote the
False Positive (FP) detections. As can be seen, traditional methods produce a
very high number of false alarms, while deep learning methods are able to
extract the semantics of the images and generate far fewer false alarms. Among
the deep learning methods, the OSCAR method proposed in this paper performs
the best, not generating any false alarms for these four images. These results
demonstrate the effectiveness of OSCAR in suppressing false alarms and
accurately detecting infrared small targets.

###  V-D Visual Analysis

To showcase the benefits of the proposed OSCAR, we provide visualizations of
the experimental results in two groups, fig. 9 and fig. 10, depending on
whether the test images contain real targets. We use  real circles to denote
True Positive (TP) detections,  green circles for  False Positive (FP)
detections, and  yellow circles for  False Negative (FN)  detections. The
better the performance, the more red circles there are and the fewer green and
yellow circles there are.

From fig. 9, it can be seen that the traditional methods, such as FKRW, IPI,
NIPPS, and SMSL, are able to detect the targets in the images, but may
generate false alarms. In particular, the IPI method performed better because
it did not generate false alarms, which is consistent with its good
performance among traditional methods in the quantitative metrics, as shown in
table IX. It can be concluded that low-rank methods typically outperform local
contrast methods in terms of detection accuracy. Among the deep learning
approaches, OSCAR is able to detect real targets while suppressing false
alarms. It should be noted that FCOS often misses the tiny real targets in the
chosen test images. This is likely because FCOS’s label assignment ignores
real targets whose bounding boxes do not include any feature points,
preventing the smallest scale targets from being trained. In contrast, our
proposed OSCAR solves this problem by using the pseudo-box for label
assignment and performs better in detecting very small scale targets. Overall,
this visualization of the results highlights the importance of semantic
understanding and careful label assignment for accurate infrared small target
detection.

As shown in fig. 10, the second group of input images primarily consist of
urban scenes, which are newly added in the SIRST-V2 dataset and contain many
cranes, street lights, and other non-target background distractions that are
similar in scale to the infrared small targets. This set of experiments
focuses on the ability of different methods to suppress false alarms using
semantic understanding of the images. The traditional methods shown in fig. 10
all produce a high number of false alarms, because in their models the
detection of infrared small targets is simplified to the detection of salient
regions of the image that match the scale of the targets. However, in complex
backgrounds not everything that fits the model’s appearance prior is a real
target, and it is likely to be a background distractor. In contrast, deep
learning methods can extract the semantics of the images to produce
significantly fewer false alarms. The performance gap between traditional and
deep learning methods is largely due to their ability to suppress false
alarms. Additionally, OSCAR outperforms FCOS in terms of not generating any
false alarms for these four images. We believe this is due to the usage of
normalized contrast branch, which helps to suppress false alarms.

In conclusion, our experiments demonstrate that the OSCAR network is a
promising approach for detecting infrared small targets. It achieves superior
performance on the SIRST-V2 dataset and is efficient in terms of both
detection performance and inference speed. We hope that the proposed OSCAR
network will serve as a reliable baseline for future research in this area.

##  VI Conclusion

We have proposed a comprehensive solution to the challenges of infrared small
target detection, including a new research benchmark consisting of the
SIRST-V2 dataset, the normalized contrast evaluation metric, and the
DeepInfrared toolkit. Furthermore, our proposed one-stage cascade refinement
network outperforms existing methods through three improvements, namely, an
all-scale pseudo-box-based label assignment scheme, the use of the high-level
head as soft proposals for the low-level refinement head, and a normalized
contrast branch for better localization quality. Extensive experiments and
analyses on the SIRST-V2 dataset further validate the effectiveness and
efficiency of OSCAR. We hope it can serve as a simple yet effective baseline
for the community.

## References

  * [1] M. Zhao, W. Li, L. Li, J. Hu, P. Ma, and R. Tao, “Single-frame infrared small-target detection: A survey,” _IEEE Geoscience and Remote Sensing Magazine_ , vol. 10, no. 2, pp. 87–119, 2022.
  * [2] L. Deng, J. Zhang, G. Xu, and H. Zhu, “Infrared small target detection via adaptive m-estimator ring top-hat transformation,” _Pattern Recognition_ , vol. 112, p. 107729, 2021.
  * [3] H. Wang, L. Zhou, and L. Wang, “Miss Detection vs. False Alarm: Adversarial learning for small object segmentation in infrared images,” in _IEEE International Conference on Computer Vision (ICCV)_ , 2019, pp. 8508–8517.
  * [4] Y. Dai, Y. Wu, F. Zhou, and K. Barnard, “Asymmetric contextual modulation for infrared small target detection,” in _IEEE Winter Conference on Applications of Computer Vision (WACV)_ , 2021, pp. 949–958.
  * [5] ——, “Attentional local contrast networks for infrared small target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 59, no. 11, pp. 9813–9824, 2021.
  * [6] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2015, pp. 3431–3440.
  * [7] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 770–778.
  * [8] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 3146–3154.
  * [9] H. Zhu, J. Zhang, G. Xu, and L. Deng, “Tensor field graph-cut for image segmentation: A non-convex perspective,” _IEEE Trans. Circuits Syst. Video Technol._ , vol. 31, no. 3, pp. 1103–1113, 2021.
  * [10] S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” in _Annual Conference on Neural Information Processing Systems (NeurIPS)_ , 2015, pp. 91–99.
  * [11] Z. Cai and N. Vasconcelos, “Cascade R-CNN: delving into high quality object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 6154–6162.
  * [12] S. Zhang, L. Wen, X. Bian, Z. Lei, and S. Z. Li, “Single-shot refinement neural network for object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 4203–4212.
  * [13] T. Lin, P. Dollár, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie, “Feature pyramid networks for object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 936–944.
  * [14] H. Zhu, S. Liu, L. Deng, Y. Li, and F. Xiao, “Infrared small target detection via low-rank tensor completion with top-hat regularization,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 58, no. 2, pp. 1004–1016, 2020.
  * [15] H. Zhu, J. Zhang, G. Xu, and L. Deng, “Balanced ring top-hat transformation for infrared small-target detection with guided filter kernel,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 56, no. 5, pp. 3892–3903, 2020.
  * [16] J. Han, S. Liu, G. Qin, Q. Zhao, H. Zhang, and N. Li, “A local contrast method combined with adaptive background estimation for infrared small target detection,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 16, no. 9, pp. 1442–1446, 2019.
  * [17] C. L. P. Chen, H. Li, Y. Wei, T. Xia, and Y. Y. Tang, “A local contrast method for small infrared target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 52, no. 1, pp. 574–581, 2014.
  * [18] C. Gao, D. Meng, Y. Yang, Y. Wang, X. Zhou, and A. G. Hauptmann, “Infrared patch-image model for small target detection in a single image,” _IEEE Transactions on Image Processing_ , vol. 22, no. 12, pp. 4996–5009, 2013.
  * [19] M. Zhao, W. Li, L. Li, P. Ma, Z. Cai, and R. Tao, “Three-order tensor creation and tucker decomposition for infrared small-target detection,” _IEEE Trans. Geosci. Remote. Sens._ , vol. 60, pp. 1–16, 2022.
  * [20] H. Cui, L. Li, X. Liu, X. Su, and F. Chen, “Infrared small target detection based on weighted three-layer window local contrast,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2022.
  * [21] Y. Dai and Y. Wu, “Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 10, no. 8, pp. 3752–3767, 2017.
  * [22] H. Zhu, H. Ni, S. Liu, G. Xu, and L. Deng, “TNLRS: target-aware non-local low-rank modeling with saliency filtering regularization for infrared small target detection,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 9546–9558, 2020.
  * [23] Y. Dai, Y. Wu, and Y. Song, “Infrared Small Target and Background Separation via Column-Wise Weighted Robust Principal Component Analysis,” _Infrared Physics & Technology_, vol. 77, pp. 421–430, Jul 2016.
  * [24] B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, and Y. Guo, “Dense nested attention network for infrared small target detection,” _IEEE Transactions on Image Processing_ , pp. 1–12, 2022.
  * [25] B. Zhao, C. Wang, Q. Fu, and Z. Han, “A novel pattern for infrared small target detection with generative adversarial network,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 59, no. 5, pp. 4481–4492, 2021.
  * [26] L. Huang, Y. Yang, Y. Deng, and Y. Yu, “Densebox: Unifying landmark localization with end to end object detection,” _CoRR_ , vol. abs/1509.04874, 2015.
  * [27] X. Zhou, D. Wang, and P. Krähenbühl, “Objects as points,” _CoRR_ , vol. abs/1904.07850, 2019.
  * [28] T. Kong, F. Sun, H. Liu, Y. Jiang, and J. Shi, “Consistent optimization for single-shot object detection,” _CoRR_ , vol. abs/1901.06563, 2019.
  * [29] Y. Wei, X. You, and H. Li, “Multiscale patch-based contrast measure for small infrared target detection,” _Pattern Recognition_ , vol. 58, pp. 216–226, 2016.
  * [30] H. Deng, X. Sun, M. Liu, C. Ye, and X. Zhou, “Small infrared target detection based on weighted local difference measure,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 54, no. 7, pp. 4204–4214, 2016.
  * [31] X. Li, C. Lv, W. Wang, G. Li, L. Yang, and J. Yang, “Generalized focal loss: Towards efficient representation learning for dense object detection,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pp. 1–14, 2022.
  * [32] Z. Tian, C. Shen, H. Chen, and T. He, “FCOS: fully convolutional one-stage object detection,” in _IEEE International Conference on Computer Vision (ICCV)_ , 2019, pp. 9626–9635.
  * [33] T. Lin, P. Goyal, R. B. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in _IEEE International Conference on Computer Vision (ICCV)_ , 2017, pp. 2999–3007.
  * [34] Y. Qin, L. Bruzzone, C. Gao, and B. Li, “Infrared small target detection based on facet kernel and random walker,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 57, no. 9, pp. 7104–7118, 2019.
  * [35] Y. Dai, Y. Wu, Y. Song, and J. Guo, “Non-negative infrared patch-image model: Robust target-background separation via partial sum minimization of singular values,” _Infrared Physics & Technology_, vol. 81, pp. 182–194, 2017.
  * [36] X. Wang, Z. Peng, D. Kong, and Y. He, “Infrared dim and small target detection based on stable multisubspace learning in heterogeneous scene,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 10, pp. 5481–5493, 2017.
  * [37] Q. Hou, L. Zhang, F. Tan, Y. Xi, H. Zheng, and N. Li, “ISTDU-Net: Infrared small-target detection U-Net,” _IEEE Geosci. Remote. Sens. Lett._ , vol. 19, pp. 1–5, 2022.
  * [38] C. Yang, Z. Huang, and N. Wang, “QueryDet: Cascaded sparse query for accelerating high-resolution small object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022, pp. 13 658–13 667.
  * [39] C. Xu, J. Wang, W. Yang, H. Yu, L. Yu, and G. Xia, “RFLA: gaussian receptive field based label assignment for tiny object detection,” in _European Conference on Computer Vision (ECCV)_ , vol. 13669, 2022, pp. 526–543.