# Triple-domain Feature Learning with Frequency-aware Memory Enhancement for
Moving Infrared Small Target Detection

Weiwei Duan, Luping Ji∗, , Shengjia Chen, Sicheng Zhu, Mao Ye ∗Corresponding
author: Luping Ji.This work is supported by the Aeronautical Science
Foundation of China (ASFC) under Grant No. 2022Z071080006 and the National
Natural Science Foundation of China (NSFC) under Grant No. 61972072.Weiwei
Duan, Luping Ji, Shenjia Chen, Sicheng Zhu and Mao Ye are with the School of
Computer Science and Engineering, University of Electronic Science and
Technology of China, Chengdu 611731, China (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Moving infrared small target detection presents significant challenges due to
tiny target sizes and low contrast against backgrounds. Currently-existing
methods primarily focus on extracting target features only from the spatial-
temporal domain. For further enhancing feature representation, more
information domains such as frequency are believed to be potentially valuable.
To extend target feature learning, we propose a new _Triple-domain Strategy_
(Tridos) with the frequency-aware memory enhancement on the spatial-temporal
domain. In our scheme, it effectively detaches and enhances frequency features
by a local-global frequency-aware module with Fourier transform. Inspired by
the human visual system, our memory enhancement unit aims to capture the
target spatial relations between video frames. Furthermore, it encodes
temporal dynamics motion features via differential learning and residual
enhancing. Additionally, we further design a residual compensation unit to
reconcile possible cross-domain feature mismatches. To our best knowledge, our
Tridos is the first work to explore target feature learning comprehensively in
spatial-temporal-frequency domains. The extensive experiments on three
datasets (DAUB, ITSDT-15K, and IRDST) validate that our triple-domain learning
scheme could be obviously superior to state-of-the-art ones. Source codes are
available at https://github.com/UESTC-nnLab/Tridos.

###### Index Terms:

Moving Infrared Small Target Detection, Feature Learning, Fourier Transform,
Frequency Aware, Memory Enhancement

##  I Introduction

Infrared small target detection (ISTD) has the advantages of being less
negatively affected by the environment and highly resistant to external
electromagnetic interference [48]. It holds significant application value in
some military fields, such as maritime early warning and infrared guidance,
and some civil fields, such as maritime rescue [47]. Due to the tiny size of
targets, they could often lack obvious visual features and have blurred
boundaries, characterized by low contrast and low signal-to-noise ratio (SNR).
Consequently, accurately locating and tracking moving small targets in
infrared images and videos has been a crucial and full of challenging research
area in computer vision.

Over the past decades, many approaches have been proposed. From the aspect of
research strategy, they could usually be categorized into model-driven methods
and data-driven methods [16]. Model-driven methods deeply analyze image
characteristics to distinguish targets from backgrounds, often based on
contrast and texture differences [1, 3]. Although these methods have achieved
impressive results, they heavily depend on prior knowledge and hand-crafted
features, lacking learning ability [24]. In contrast, data-driven methods have
become a type of mainstream schemes in recent years [9]. They can effectively
learn target detection tasks from labeled datasets [14, 15], Therefore, our
work mainly focus on the challenges faced by data-driven methods.

Figure 1:  The comparison between the existing spatial-temporal scheme and our
scheme of triple-domain feature learning. Our scheme extract features in
spatial-temporal-frequency domains.

Furthermore, according to the different ways of using images, existing
detection methods could be generally further divided into single-frame and
multi-frame schemes [29]. Generally, single-frame ISTD (SISTD) does not need
to consider the relations between multiple consecutive frames [12], as in [43]
and [19]. These methods extract target features only from the spatial domain
of images [10], owning some obvious advantages of low complexity and high
detection speed. Multi-frame ISTD (MISTD) could further improve detection
performance by extracting target features from both spatial and temporal
domains [29], as shown in Fig. 1(a). For example, STDMANet [30] designs a
feature extractor to obtain spatial-temporal features from multi-frame
consecutive images, and SSTNet [32] employs a cross-slice ConvLSTM to leverage
spatial-temporal contexts also on multiple frames.

Currently, almost all MISTD methods are based on the spatial-temporal domain.
They precisely capture the spatial-temporal contexts to reduce interference or
enhance target features [49, 52]. Nevertheless, while these approaches are
promising, they often fail to capture the intricate characteristics of small
moving targets entirely. Besides, targets in intricate backgrounds could not
exhibit a high correlation in consecutive frames In these challenging
situations, obtaining targets’ motion details and contours is extremely
difficult, decreasing the detection performance. The information from
different domains is complementary, which helps to characterize the target
features more comprehensively. Therefore, it is compelling to introduce more
information domains in feature learning.

Consequently, a vital issue for MISTD is capturing fine-grained target
features without being affected by complex scenarios. Many image processing
studies [51, 50] have shown that frequency domain features provide information
at different frequencies. Frequency information is potentially valuable
because, in the frequency domain, noise is usually distributed at higher
frequencies, while targets are distributed at lower frequencies. Frequency
domain processing could reduce noise disturbance, effectively improving the
accuracy of ISTD. Inspired by these findings, we develop a new triple-domain
feature learning scheme to integrate the frequency domain into the existing
spatial-temporal domain, as shown in Fig. 1(b). However, further exploration
is needed to effectively acquire and integrate features from both spatial-
temporal and frequency domains simultaneously.

To effectively tackle the challenges above, we propose a new _Triple-domain
Strategy_ (Tridos) of frequency-aware memory enhancement for further enhancing
target feature learning. It is dedicated to exploiting the potential of
frequency information, enhancing target representation comprehensively. Our
scheme concentrates on depicting targets in spatial-temporal-frequency
domains, exploiting their dynamic variations in the frequency domain for
promoting detection performance. It is the first time to model the task of
MISTD from a frequency-aware spatial-temporal perspective. In summary, the
main contributions are summarized as follows:

(I) We explore and propose Tridos, a pioneering triple-domain scheme to extend
the feature learning perspective for MISTD. Besides traditional spatial-
temporal domains, it can capture target features from the frequency domain and
then realize the fusion and enhancement of spatial-temporal-frequency
features.

(II) Based on the Fourier transform, a local-global frequency-aware module
(LGFM) is developed to extract comprehensive frequency features from local and
global perceptual patterns. Moreover, inspired by the human visual system, a
memory-enhanced spatial relationship module (MSRM) is designed to model the
inter-frame correlations of small targets.

(III) A residual compensation unit (RCU) is constructed to eliminate the
possible feature mismatches between cross-different domains, then auxiliarily
fuse & enhance spatial-temporal-frequency features of small targets.

(IV) By improving popular IoU loss, a new dual-view regression loss function,
especially for infrared small target detection, is defined to optimize model
training.

##  II Related Work

###  II-A Single-frame Infrared Small Target Detection

Single-frame infrared small target detection deals with stationary targets in
a single image, which can be divided into model-driven and data-driven
methods.

Model-driven methods extract tiny targets by exploiting the characteristics of
the image for target enhancement or background suppression, followed by
threshold segmentation [53]. They can be further categorized into filter-
based, human visual system (HVS)-based, and data structure-based methods.
Filter-based methods utilize the difference between target and background
pixels to highlight targets and remove background noise interference, such as
Top-hat [1] and Maxmean [2]. HVS-based methods extract salient regions by
measuring the maximum contrast between the center pixel and neighboring
regions, such as local contrast measure (LCM) [3] and its modified version:
RLCM [4], HBMLCM [5], and WSLCM [6]. Data structure-based methods employ the
structure of infrared images to separate the background and the target, such
as IPI [7] and SMSL [8]. Although these model-driven approaches have achieved
outstanding performance, they cannot adapt to intricate environments.

Data-driven methods primarily exploit deep neural networks that can adaptively
learn target features by training on many training samples. They become
dominant with the availability of small infrared target detection datasets [9,
12, 10]. For example, ACMNet [9] integrates low-level details and high-level
semantics through asymmetric contextual modulation, and they further introduce
dilated local contrast measurements in ALCNet [13]. DNANet [12] designs a
dense nested interactive network to enhance the features of small targets.
Moreover, AGPCNet [14] develops an attention-guided pyramid context network to
obtain a global association of semantics. RDIAN [15] uses multi-scale
convolutions to capture diverse target features and extend the receptive
field. UIUNet [17] integrates two U-nets to learn the multi-sale and multi-
level representations. ISNet [10] creatively emphasizes the importance of
target shape. Furthermore, EFLNet [18] constructs a feature-enhancing learning
network. RPCANet [19] proposes an interpretable deep neural network with
theory-guided to replace matrix computation.

Single-frame infrared small target detection has been widely studied and has
performed well. However, this pipeline ignores the motion features of targets.
It is susceptible to noise and background interference, especially in
challenging scenarios.

###  II-B Multi-frame Infrared Small Target Detection

In contrast, multi-frame schemes process multiple consecutive frames
simultaneously. Therefore, extracting the spatial-temporal features of targets
has received extensive attention.

In terms of the traditional schemes, optical flow-based methods [21, 20]
utilize brightness variations in different frames to describe target motion
features. STCP [21] employs the optical flow algorithm to compute the dense
trajectory of targets and then creates a binary image to extract salient
contours as candidate target regions. Some methods construct spatial-temporal
tensors to distinguish targets from backgrounds, such as 4D STT [22] and NPSTT
[24]. To solve the issue of inaccurate background estimation, NTLA [25]
proposes a nonconvex tensor low-order approximation strategy. FST-FLNN [23]
improves tensor nuclear norm using log operation for target enhancement and
background suppression. Moreover, energy accumulation-based methods [26, 27]
can effectively improve the SNR and enhance the energy of small targets. The
dynamic programming-based methods [28] search for the optimal motion
trajectory of targets through the dynamic programming algorithm. Although
these traditional methods have made some progress, they rely heavily on prior
knowledge.

To overcome the problems of the traditional methods above, some approaches
apply deep learning to MISTD. This pipeline can exploit the temporal
association between sequential frames to improve detection performance.
However, it has yet to be entirely explored due to the lack of adequate MISTD
datasets with high-quality annotations. The most representative methods are as
follows. For instance, the interframe energy accumulation enhancement method
is proposed in [29] to enhance small targets. STDMANet [30] introduces a
spatial-temporal differential multiscale attention network. It is the first
study to introduce a temporal attention mechanism for MISTD. DTUM [31] is
proposed to encode motion direction and thus extract motion information.
Moreover, SSTNet [32] designs a sliced spatial-temporal network to employ
cross-slice motion context. Recently, ST-Trans [33] introduces a spatial-
temporal transformer to learn the associations between consecutive frames to
handle complex scenarios. However, these methods only consider effectively
extracting and exploiting target features from the spatial-temporal domain and
scarcely consider the utilization of frequency information implied in
consecutive frames.

Therefore, in this work, we focus on applying the frequency-domain information
of sequential frames to traditional spatial-temporal networks, enhancing
target feature learning. Furthermore, referencing available repositories, we
construct a new MISTD dataset (ITSDT-15K) with accurate bounding box
annotations to additionally validate our proposed method on MISTD. Their
details are in section IV.

##  III METHODOLOGY

###  III-A Overall Architecture

We aim to integrate frequency information into spatial-temporal networks to
effectively promote detection performance for MISTD. The overall workflow of
our Tridos is presented in Fig. 2.

Figure 2: Overview of the proposed framework Tridos. $\textbf{I}_{\textbf{c}}$
is a collection of consecutive frames randomly sampled from infrared video.
Tridos contains a backbone of shared weights to extract target features from
consecutive frames, followed by the Memory-enhanced Spatial Relationship
Module (MSRM), the Temporal Dynamic Encoding Module (TDEM), and the Local-
Global Frequency-aware Module (LGFM). Then, the features of the three branches
are compensated by the Residual Compensate Units (RCU). Finally, the
compensated features pass through the detection head to obtain the final
results.

In our pipeline, we randomly sample a collection of consecutive frames
$\boldsymbol{I_{c}}=\\{\boldsymbol{I_{1}},\boldsymbol{I_{2}},\cdots,\boldsymbol{I_{t}}\\}$
from infrared video with a time window size of $T=t$ as input.
$\boldsymbol{I_{t}}$ is the keyframe that needs to be detected, and the other
ones are reference frames to provide context information. Then, we feed each
frame into feature extractors with shared weights to obtain multi-frame
features
$\boldsymbol{F_{c}}=\\{\boldsymbol{F_{1}},\boldsymbol{F_{2}},\cdots,\boldsymbol{F_{t}}\\}\in\mathbb{R}^{T\times
C\times H\times W}$, where $C$, $H$ and $W$ indicate the channel, height and
width of feature matrix, respectively. Specifically, we adopt a CSPDarknet
[35] pre-trained on the COCO [36] dataset as the backbone. We design three
branches to realize the strategy of triple-domain feature learning with
frequency-aware memory enhancement through memory-enhanced spatial
relationship module (MSRM), temporal dynamic encoding module (TDEM), and
local-global frequency-aware module (LGFM). The residual compensation unit
(RCU) aims to eliminate the possible feature mismatches across different
domains while fusing and enhancing different domain features. Then, the
adequately interacted spatio-temporal-frequency features
$\boldsymbol{F_{stf}}$ are delivered to the detection head to obtain the final
results.

The MSRM receives multi-frame features $\boldsymbol{F_{c}}$ as input and
generates the memory-enhanced spatial features $\boldsymbol{F_{S}}$.
Similarly, we introduce TDEM to extract target motion features
$\boldsymbol{F_{T}}$ by employing differential learning and residual enhancing
between neighboring frames. For using both the location and motion information
of targets, we concatenate the spatial features $\boldsymbol{F_{S}}$ and
temporal features $\boldsymbol{F_{T}}$ to capture the spatial-temporal
features $\boldsymbol{F_{st}}$, as follows:

$\boldsymbol{F_{st}}=Conv(Concat\left[\boldsymbol{F_{S}},\boldsymbol{F_{T}}\right])$
(1)

where $Conv$ denotes a $3\times 3$ convolutional layer, batch normalization
and SiLU activation function. Inspired by the Fourier Transform, we transform
the collection of multi-frame features $\boldsymbol{F_{c}}$ into frequency
domain. The LGFM is proposed for modeling the local-global variation of
targets in the frequency domain, enhancing target representation with
different views.

$\boldsymbol{F_{lf},F_{gf}}=LGFM(\boldsymbol{F_{c}})$ (2)

where $\boldsymbol{F_{lf}}$ and $\boldsymbol{F_{gf}}$ are local and global
features in frequency domain, respectively. Then, $\boldsymbol{F_{lf}}$,
$\boldsymbol{F_{gf}}$ and $\boldsymbol{F_{st}}$ are fed into the carefully
designed RCUs to realize feature interaction and alleviate the mismatches in
different domain features. The calculation process is as follows:

$\left\\{\begin{split}\boldsymbol{F_{stf_{1}}}&=RCU_{1}(\boldsymbol{F_{lf}},\boldsymbol{F_{st}})\\\
\boldsymbol{F_{stf_{2}}}&=RCU_{2}(\boldsymbol{F_{gf}},\boldsymbol{F_{st}})\\\
\boldsymbol{F_{stf}}&=RCU_{3}(\boldsymbol{F_{stf_{1}}},\boldsymbol{F_{stf_{2}}})\end{split}\right.$
(3)

After that, we obtain the compensated enhanced spatial-temporal-frequency
features $\boldsymbol{F_{stf}}$. Finally, the final detection results are
acquired by the decoupled detection head of YOLOX [35] and non-maximal
suppression.

###  III-B Memory-enhanced Spatial Relationship Module

Spatial relationship modeling is essential for accurate target location
prediction and tracking. Previous methods directly fuse multi-frame spatial
features via concatenation [29] or multi-layer convolution [30]. They seldom
consider the keyframe’s global context information and the spatial
dependencies between neighboring frames. To solve this, we propose the MSRM,
as shown in Fig. 2. In this module, the nonlocal attention block (NAB) is
introduced to obtain the long-distance dependencies between pixels in
different time steps. When tracking moving targets, humans can recognize
targets more accurately and extract the target’s spatial relationship by
comparing and inferring the features between reference frames and the
keyframe. Therefore, inspired by the human visual system, we devise the memory
enhancement unit (MEU) to expand target features. It could fully employ multi-
frame features by creating key-value pairs to better capture spatial
relationships between targets.

Specifically, in MSRM, we first perform a preliminary merging of the reference
frames and keyframe features through the following computational process. We
assign different weights to different pixels of the reference frame.

$\displaystyle\begin{split}\boldsymbol{\hat{F_{t}}}&=\sigma(Conv(Concat[\boldsymbol{F_{1}},\cdots,\boldsymbol{F_{t-1}}]))\otimes\boldsymbol{F_{t}}\\\
\boldsymbol{F_{l}}&=Conv(Concat[\boldsymbol{\hat{F_{t}}},\boldsymbol{F_{t}}])\end{split}$
(4)

where $\sigma$ denotes Sigmoid function, $\otimes$ is matrix multiplication,
$\boldsymbol{\hat{F_{t}}}$ is the updated keyframe features, and
$\boldsymbol{F_{l}}$ indicates local inter-frame spatial relations.

Then, we send $\boldsymbol{F_{l}}$ to NAB to explore the correlation of
different pixels between frames for overcoming the convolutional layer’s local
property, that is

$\begin{split}\boldsymbol{Q},\boldsymbol{K},\boldsymbol{V}&=Projection(\boldsymbol{F_{l}})\\\
\boldsymbol{F_{g}}&=\gamma\cdot
Softmax(\frac{\boldsymbol{Q}\boldsymbol{K^{T}}}{\sqrt{d}})V+\boldsymbol{F_{l}}\end{split}$
(5)

where $Projection(\cdot)$ is a linear transform implemented by $1\times 1$
convolution, $\gamma$ is a hyperparameter, $\sqrt{d}$ is the scale factor for
normalization and $\boldsymbol{F_{g}}$ denotes inter-frame local-global
spatial relations. In this way, any two pixels can interact and capture long-
distance dependencies, not just limited to the local neighboring ones.

After that, we put the local-global spatial features $\boldsymbol{F_{g}}$ in
memory and the keyframe features $\boldsymbol{F_{t}}$ as query, similar to the
tracking mechanism of human brain. In details, we employ four distinct
embedding layers to generate the corresponding key-value pairs.

$\left\\{\begin{split}\boldsymbol{K_{Q}},\boldsymbol{V_{Q}}&=f_{Q}(\boldsymbol{F_{t}})\\\
\boldsymbol{K_{M}},\boldsymbol{V_{M}}&=f_{M}(\boldsymbol{F_{g}})\end{split}\right.$
(6)

where $f_{Q}(\cdot)$ and $f_{M}(\cdot)$ indicate different embedding layers.
Key is utilized to align the location information of targets, and value holds
the high-level semantic features. In MEU, we first calculate the similarity of
$\boldsymbol{K_{M}}$ and $\boldsymbol{K_{Q}}$ and normalize it through the
$Softmax$ function. Subsequently, the regions related to the target features
are queried from $\boldsymbol{V_{M}}$ and concatenated with
$\boldsymbol{V_{Q}}$. This establishes the connection between the keyframe and
reference frames and improves the target features with visual information from
memory. The MEU is computed mathematically as follows:

$\begin{split}\boldsymbol{M_{s}}&=Softmax(\boldsymbol{K_{M}}\otimes\boldsymbol{K_{Q}})\\\
\boldsymbol{F_{S}}&=Mathing(Concat[\boldsymbol{M_{s}}\otimes\boldsymbol{V_{M}},\boldsymbol{V_{Q}}])\end{split}$
(7)

where $\boldsymbol{M_{s}}$ is the similarity matrix between
$\boldsymbol{K_{M}}$ and $\boldsymbol{K_{Q}}$, $Matching$ represents $1\times
1$ convolution and $\boldsymbol{F_{S}}$ denotes the local-global spatial
features by memory enhancement.

###  III-C Temporal Dynamics Encoding Module

How to extract the motion paradigm of targets is a challenging problem in
MISTD. Previous methods use optical flow [20] or recurrent neural networks
[32]. However, since the target is too small, the motion information between
neighboring frames is not prominent. Therefore, we design TDEM, as shown in
Fig. 3.

It stacks all adjacent frames’ differential information within the time window
to address a single difference’s sparsity problem. We further introduce
residual blocks for differential information enhancement to obtain more
representative motion features.

Figure 3: The details of our proposed Temporal Dynamic Encoding Module (TDEM),
with a time window $T=5$ as an example. The $\text{ResB}_{1}$ and
$\text{ResB}_{2}$ are residual blocks for enhancing differential information.

In details, TDEM dynamically encodes difference maps and propagates the local
motions into the keyframe features. As shown in Fig. 3, we set the size of
time window $T=5$. Initially, we acquire continuous local motions
$\boldsymbol{D}=[d_{1},d_{2},d_{3},d_{4}]$ in sequential images by subtracting
adjacent frames. Then, we use a $3\times 3$ convolution and downsample through
a average pooling with a stride of $2$ to extract the motion dependencies of
targets $\boldsymbol{F_{D}}$, which can be expressed as follows:

$\boldsymbol{F_{D}}=Avg(Conv(Stack(d_{1},d_{2},d_{3},d_{4})))$ (8)

where $Avg$ means average pooling and $Stack$ is the splice operation on
dimension $1$. We perform downsampling due to two main considerations. One is
that differential information is sparse, and the motions are more prominent in
low-resolution space. Another is that the pooling operation can reduce the
computational cost of temporal dynamic encoding.

Moreover, we integrate motions and keyframe features to cope with the dynamic
changes of targets in the temporal dimension. The motion dependencies
$\boldsymbol{F_{D}}$ restores the original size and then added to the keyframe
features. Furthermore, we employ two residual blocks to boost local motions
and deeply incorporate motion dependencies and target features, which can be
calculated as follows:

$\begin{split}\boldsymbol{F_{u}}&=\boldsymbol{F_{t}}+f_{u}(Conv(\boldsymbol{F_{D}}))\\\
\boldsymbol{F_{T}}&=ResB_{2}(\boldsymbol{F_{u}})+f_{u}(ResB_{1}(Conv(\boldsymbol{F_{D}})))\end{split}$
(9)

where $f_{u}$ is the upsample operation, $\boldsymbol{F_{T}}$ denotes the
motion features through temporal dynamic encoding, and $ResB$ represents the
residual block which contains two $3\times 3$ convolutions and a $PReLU$
activation function.

###  III-D Local-global Frequency-aware Block

Frequency domain features can provide detailed information about targets at
different frequencies and suppress image noise and interference. We aim to
thoroughly explore the potential of frequency to extend target feature
learning.

As shown in Fig. 2, our LGFM has two branches that can model and extract the
frequency features from multiple views and levels. One is the convolution
branch, which captures the local correlation and spatial structure from
frequency information. The other is the swin transformer [38] branch, which
models and encodes frequency features globally through a sliding window
mechanism, improve the receptive field, and extract global context
information.

Concretely, the Fourier transform [37] $f_{\mathcal{F}}(\cdot)$ converts each
frame’s features to the frequency domain space. Simply, the process can be
formulated as follows for input features $x$:

$f_{\mathcal{F}}(x)(u,v)=\sum_{h=0}^{H-1}\sum_{w=0}^{W-1}x(h,w)e^{-j2\pi\left(\frac{h}{H}u+\frac{w}{W}v\right)}$
(10)

where $u$ and $v$ represent the horizontal and vertical components of
frequency. H and W are the height and width of $x$. The frequency domain
information $f_{\mathcal{F}}(x)$ can be further decomposed to the amplitude
$f_{\mathcal{A}}(x)$ and the phase $f_{\mathcal{P}}(x)$, that is

$\left\\{\begin{split}f_{\mathcal{A}}(x)(u,v)&=\begin{bmatrix}\mathcal{R}^{2}(x)(u,v)+\mathcal{I}^{2}(x)(u,v)\end{bmatrix}^{1/2}\\\
f_{\mathcal{P}}(x)(u,v)&=\arctan\left[\frac{\mathcal{I}(x)(u,v)}{\mathcal{R}(x)(u,v)}\right]\end{split}\right.$
(11)

where $\mathcal{R}(x)$ and $\mathcal{I}(x)$ denote the real and imaginary part
of $f_{\mathcal{F}}(x)$, respectively.

The motivation for decomposing the frequency features into amplitude and phase
is to employ the information from these two aspects. Amplitude reflects the
features’ energy distribution at different frequencies, and phase comprises
the position and relative relationship in the frequency domain. Initially, we
process the amplitude and utilize the amplitude residuals to further extract
and tune the phase. We use a $3\times 3$ depthwise convolution $DW(\cdot)$ to
independently learn spatial context information on each channel, followed by
cross-channel blending with $1\times 1$ pointwise convolution $PW(\cdot)$
aggregated pixels, as follows:

$f^{\prime}_{\mathcal{A}}(x)=ReLU(PW(ReLU(DW(f_{\mathcal{A}}(x)))))$ (12)

To enhance the energy of the moving targets, we apply max pooling $Max(\cdot)$
and average pooling $Avg(\cdot)$ in the channel dimension and concatenate
them. Then, the same transform as in Eq. (12) to get the amplitude attention
map $\boldsymbol{M_{a}}$. We multiply $\boldsymbol{M_{a}}$ with the amplitude
features $f^{\prime}_{\mathcal{A}}(x)$ to get the final refined amplitude
features $f^{{}^{\prime\prime}}_{\mathcal{A}}(x)$, which can be formulated as
follows:

$\left\\{\begin{split}\boldsymbol{Q_{1}}&=Concat[Avg(f^{\prime}_{\mathcal{A}}(x)),Max(f^{\prime}_{\mathcal{A}}(x))]\\\
\boldsymbol{M_{a}}&=\sigma(PW(ReLU(DW(\boldsymbol{Q_{1}}))))\\\
f^{{}^{\prime\prime}}_{\mathcal{A}}(x)&=f^{\prime}_{\mathcal{A}}(x)\otimes\boldsymbol{M_{a}}\end{split}\right.$
(13)

where $\boldsymbol{Q_{1}}\in\mathbb{R}^{2\times H\times W}$ and $\sigma$ is
$Sigmoid$ function.

After that, we employ amplitude residuals
$f_{\mathcal{R}}(x)=f^{{}^{\prime\prime}}_{\mathcal{A}}(x)-f_{\mathcal{A}}(x)$,
which contain subtle amplitude variations, to modify the input and guide the
phase features extraction. As for phase, we use a similar procedure as for
amplitude, that is

$\left\\{\begin{split}f^{\prime}_{\mathcal{P}}(x)&=\sigma(PW(ReLU(DW(Concat[f_{\mathcal{P}}(x),f_{\mathcal{R}}(x)]))))\\\
\boldsymbol{Q_{2}}&=Concat[Avg(f^{\prime}_{\mathcal{P}}(x)),Max(f^{\prime}_{\mathcal{P}}(x))]\\\
\boldsymbol{M_{p}}&=\sigma(PW(ReLU(DW(\boldsymbol{Q_{2}}))))\\\
f^{{}^{\prime\prime}}_{\mathcal{P}}(x)&=f^{\prime}_{\mathcal{P}}(x)\otimes\boldsymbol{M_{p}}\end{split}\right.$
(14)

Moreover, we use post-processing to limit the phase values to $[-\pi,\pi]$,
which can maintain the continuity and stability of the phase. For combining
amplitude and phase, we employ the $Cos(\cdot)$ and $Sin(\cdot)$ functions to
compute the real and imaginary components, as follows:

$\left\\{\begin{split}f^{{}^{\prime\prime\prime}}_{\mathcal{P}}(x)&=2\pi\cdot
f^{{}^{\prime\prime}}_{\mathcal{P}}(x)-\pi\\\
\mathcal{R}^{\prime}(x)(u,v)&=f^{{}^{\prime\prime}}_{\mathcal{A}}(x)(u,v)\cdot
Cos(f^{{}^{\prime\prime\prime}}_{\mathcal{P}}(x)(u,v))\\\
\mathcal{I}^{\prime}(x)(u,v)&=f^{{}^{\prime\prime}}_{\mathcal{A}}(x)(u,v)\cdot
Sin(f^{{}^{\prime\prime\prime}}_{\mathcal{P}}(x)(u,v))\end{split}\right.$ (15)

where
$x^{\prime}_{\mathcal{F}}=\mathcal{R}^{\prime}(x)+\mathcal{I}^{\prime}(x)j$ is
the integrated frequency features. Then, the inverse Fourier transformation
$f^{-1}_{\mathcal{F}}(\cdot)$ is used to transform $x^{\prime}_{\mathcal{F}}$
to the original domain. Suppose the input feature is $x$, the final output
$x^{\prime}$ is obtained by adding the input $x$ and the transformed features
$\hat{x}$ through the residuals, that is

$\begin{split}\hat{x}&=f^{-1}_{\mathcal{F}}(x^{\prime}_{\mathcal{F}})\\\
x^{\prime}&=x+\hat{x}\\\ \end{split}$ (16)

For the input multi-frame features collection $\boldsymbol{F_{c}}$, we process
each frame separately to get outputs
$\boldsymbol{F^{\prime}_{c}}=\\{\boldsymbol{F^{\prime}_{1}},\boldsymbol{F^{\prime}_{2}},\cdots,\boldsymbol{F^{\prime}_{t}}\\}\in\mathbb{R}^{T\times
C\times H\times W}$. Finally, we splice them to feed into two separate local-
global branches, that is

$\begin{array}[]{c}\boldsymbol{F_{lf}}=ConvB(Concat[\boldsymbol{F^{\prime}_{1}},\boldsymbol{F^{\prime}_{2}},\cdots,\boldsymbol{F^{\prime}_{t}}])\\\
\boldsymbol{F_{gf}}=SwinB(Concat[\boldsymbol{F^{\prime}_{1}},\boldsymbol{F^{\prime}_{2}},\cdots,\boldsymbol{F^{\prime}_{t}}])\end{array}$
(17)

where $\boldsymbol{F_{lf}}$ is the local frequency features from the
convolution branch $ConvB(\cdot)$. This branch contains two $3\times 3$
convolutions with a stride of 1, followed by batch normalization and SiLU
activation function. $\boldsymbol{F_{gf}}$ is the global frequency features
from the swin transformer branch $SwinB(\cdot)$.

###  III-E Residual Compensation Unit

Our scheme thoroughly explores the target’s spatial, temporal, and frequency
domain features. However, mismatches could happen due to the differences in
the learning ways of different domain features. Therefore, we need to
compensate features further to retain valuable information in different
domains. With the help of RCU, the differences and commonalities between the
features of different domains can be fused and enhanced, which helps to
enhance the network’s understanding of data and reduce the sensitivity to
interference.

As shown in Fig. 2, our RCU comprises several channel spatial attention blocks
(CSAB). For example, for input local frequency features $\boldsymbol{F_{lf}}$
and spatial-temporal features $\boldsymbol{F_{st}}$, we first splice them in
channel dimension and use a $3\times 3$ convolution followed by batch
normalization and ReLU (denoted as $Conv$) for further enhance target
features. Then, we introduce a channel attention block (CAB) and a spatial
attention block (SAB) to weight features from different channels and spatial
locations to adaptively focus on target significant information. Besides,
feature compensation is performed by the residual branch. The residual
compensation between $\boldsymbol{F_{lf}}$ and $\boldsymbol{F_{st}}$ can be
expressed as follows:

$\left\\{\begin{split}\boldsymbol{F_{r}}&=Concat[\boldsymbol{F_{lf}},\boldsymbol{F_{st}}]\\\
CSAB&=f_{SAB}(f_{CAB}(Conv(\boldsymbol{F_{r}})))+\boldsymbol{F_{r}}\\\
\boldsymbol{F_{stf_{1}}}&=CASB_{m}(CASB_{m-1}(\cdots
CSAB_{1}(\boldsymbol{F_{r}})))\end{split}\right.$ (18)

where $m$ is the number of CSAB. Similarly, we perform the same operations
between $\boldsymbol{F_{lf}}$ and $\boldsymbol{F_{st}}$,
$\boldsymbol{F_{stf_{1}}}$ and $\boldsymbol{F_{stf_{2}}}$ for the final
refinement.

###  III-F Dual View Regression Loss

Following the general paradigm of detection, the loss function can be
expressed as

$\mathcal{L}=\lambda_{reg}\mathcal{L}_{reg}+\lambda_{cls}\mathcal{L}_{cls}+\lambda_{obj}\mathcal{L}_{obj}$
(19)

where $\mathcal{L}_{reg}$ is a bounding box regression loss,
$\mathcal{L}_{cls}$ is a classification loss, and $\mathcal{L}_{obj}$ is a
target probability loss. $\lambda_{reg}$, $\lambda_{cls}$, and $\lambda_{obj}$
are three hyper-parameters to balance portion.

For $\mathcal{L}_{cls}$ and $\mathcal{L}_{obj}$, we employ sigmoid focal loss
[39] to solve the problem of positive and negative sampling imbalance. In
terms of $\mathcal{L}_{reg}$, due to the small size of the targets, using
popular IoU loss could be insufficient to capture detailed information about
the targets’ distribution. The normalized Gaussian wasserstein distance (NWD)
loss [40] can minimize the difference between target boxes by introducing a
Gaussian distribution. It could efficiently model target boxes of different
sizes and positions.

Therefore, we design a dual view regression loss $\mathcal{L}_{dvd}$ to
incorporate the advantages of IoU loss and NWD loss. The IoU loss assists the
network in accurately localizing small targets, while the NWD loss learns the
distributional properties, that is

$\left\\{\begin{array}[]{l}\mathcal{L}_{reg}=\mathcal{L}_{dvr}=\alpha\mathcal{L}_{iou}+\beta\mathcal{L}_{nwd}\\\
\mathcal{L}_{iou}=1-IoU(B_{P},B_{G})\\\
\mathcal{L}_{nwd}=1-NWD(N_{P},N_{G})\\\
NWD(N_{p},N_{g})=exp(-\frac{\sqrt{\|N_{P},N_{G}\|^{2}_{2}}}{C})\end{array}\right.$
(20)

where $B_{P}$ is predicted bounding box, $B_{G}$ is ground truth, $N_{P}$ and
$N_{G}$ are the Gaussian distributions for predicted boxes and ground truth,
respectively. $C$ is a constant related to datasets, and $\alpha$ and $\beta$
are two hyper-parameters.

TABLE I: Quantitative comparison results of different state-of-the-art methods
on three datasets. The best and second best results are highlighted in red and
blue, respectively.

Scheme | Methods | Publication | DAUB | ITSDT-15K | IRDST
---|---|---|---|---|---
$\textbf{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\textbf{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\textbf{mAP}_{\textbf{50}}$ | Precision | Recall | F1
Model-driven | MaxMean [2] | SPIE 1999 | 10.71 | 20.38 | 53.87 | 29.57 | 0.87 | 10.85 | 8.74 | 9.68 | 0.01 | 0.28 | 1.48 | 0.47
TopHat [1] | IPT 2006 | 16.99 | 21.69 | 79.83 | 34.11 | 11.61 | 27.21 | 43.07 | 33.35 | 1.81 | 18.22 | 10.60 | 13.40
RLCM [4] | IEEE TGRS 2013 | 0.02 | 0.27 | 5.21 | 0.51 | 4.62 | 15.38 | 30.76 | 20.50 | 1.58 | 16.28 | 9.70 | 12.16
HBMLCM [5] | IEEE GRSL 2019 | 3.90 | 23.96 | 16.52 | 19.56 | 0.72 | 7.97 | 9.37 | 8.61 | 1.16 | 29.14 | 4.66 | 8.03
PSTNN [24] | RS 2019 | 17.31 | 25.56 | 68.86 | 37.28 | 7.99 | 22.98 | 35.21 | 27.81 | 1.45 | 16.28 | 9.70 | 12.16
WSLCM [6] | SP 2020 | 1.37 | 11.88 | 11.57 | 11.73 | 2.36 | 16.78 | 14.53 | 15.58 | 1.69 | 20.87 | 8.70 | 12.28
Data-driven | ACM [9] | WACV 2021 | 64.02 | 70.96 | 91.3 | 79.86 | 55.38 | 78.37 | 71.69 | 74.88 | 52.40 | 76.33 | 69.32 | 72.66
RISTD [43] | IEEE GRSL 2022 | 81.05 | 83.46 | 98.27 | 90.26 | 60.47 | 85.49 | 71.60 | 77.93 | 66.57 | 84.7 | 79.63 | 82.08
ISNet [10] | CVPR 2022 | 83.43 | 89.36 | 94.99 | 92.09 | 62.29 | 83.46 | 75.32 | 79.18 | 59.78 | 80.24 | 75.08 | 77.58
UIUNet [17] | IEEE TIP 2022 | 86.41 | 94.46 | 92.03 | 93.23 | 65.15 | 84.07 | 78.39 | 81.13 | 56.38 | 80.95 | 70.29 | 75.25
SANet [44] | ICASSP 2023 | 87.12 | 93.44 | 94.93 | 94.18 | 62.17 | 87.78 | 71.23 | 78.64 | 64.54 | 84.29 | 77.02 | 80.49
AGPCNet [14] | IEEE TAES 2023 | 76.72 | 82.29 | 94.43 | 87.95 | 67.27 | 91.19 | 74.77 | 82.16 | 59.21 | 79.47 | 75.51 | 77.44
RDIAN [15] | IEEE TGRS 2023 | 84.92 | 88.2 | 97.27 | 92.51 | 68.49 | 90.56 | 76.06 | 82.68 | 59.08 | 77.99 | 76.35 | 77.16
DNANet [12] | IEEE TIP 2023 | 89.93 | 92.49 | 98.27 | 95.29 | 70.46 | 88.55 | 80.73 | 84.46 | 63.61 | 82.92 | 77.48 | 80.11
DTUM [31] | IEEE TNNLS 2023 | 85.86 | 87.54 | 99.79 | 93.26 | 67.97 | 77.95 | 88.28 | 82.79 | 71.48 | 82.87 | 87.79 | 85.26
SIRST5K [45] | IEEE TGRS 2024 | 93.31 | 97.78 | 96.93 | 97.35 | 61.52 | 86.95 | 71.32 | 78.36 | 52.28 | 76.12 | 69.07 | 72.42
MSHNet [46] | CVPR 2024 | 85.97 | 93.13 | 93.12 | 93.13 | 60.82 | 89.69 | 68.44 | 77.64 | 63.21 | 82.31 | 77.64 | 79.91
RPCANet [19] | WACV 2024 | 85.98 | 89.38 | 97.56 | 93.29 | 62.28 | 81.46 | 77.10 | 79.22 | 56.50 | 77.77 | 73.80 | 75.73
SSTNet [32] | IEEE TGRS 2024 | 95.59 | 98.08 | 98.10 | 98.09 | 76.96 | 91.05 | 85.29 | 88.07 | 71.55 | 88.56 | 81.92 | 85.11
Tridos (Ours) | - | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65 | 73.72 | 84.49 | 89.35 | 86.85

##  IV EXPERIMENTS

###  IV-A Datasets and Evaluation Metrics

The datasets’ quality, sample quantity, and scenario diversity significantly
impact data-driven detection methods’ performance. Currently, there are few
datasets [15, 42] for MISTD. These ones are intended for aerial-based
applications and aim at airborne, not ground-based vehicle targets. Thus, we
reorganize an additional MISTD dataset, ITSDT-15K, to improve the quality,
quantity, and diversity of infrared multi-frame datasets. The raw infrared
images are from this public repository [34]. The targets in these images are
moving vehicles captured by UAV infrared cameras, which are small, dim and
poorly defined. We remove some wrong image sequences and revise the bounding
boxes. Finally, our ITSDT-15K dataset contains 15K images, including 10K
training images of 40 videos and 5K test images of 20 videos, with an image
resolution of $640\times 480$.

The ITSDT-15K dataset contains a variety of complex scenarios, including
grass, forests, obstacles, and ground clutter. Overall, our ITSDT-15K is
adequate for evaluating the performance of different methods for MISTD.
Accordingly, we employ the ITSDT-15K and two public datasets, DAUB [42] and
IRDST [15], for evaluation in our experiments. For DAUB and IRDST, we follow
[32] to divide the training and test sets.

In terms of evaluation metrics, following the common practice of the target
detection paradigm, we apply _Precision_ (%), _Recall_ (%), F1 score (%), and
_Average Precision_ (%) (e.g., $\text{mAP}_{50}$, the average Precision with
an IOU threshold 0.5), which can be formulated as

$\displaystyle\begin{split}\text{Precision}&=\frac{\text{TP}}{\text{TP}+\text{FP}}\\\
\text{Recall}&=\frac{\text{TP}}{\text{TP}+\text{FN}}\\\
\text{F1}&=\frac{2\times\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}\end{split}$
(21)

where $TP$, $FP$, and $FN$ denotes the number of correct detection (True
positive), false alarms (False positive), and missed detection (False
negative), respectively. F1 score combines _Precision_ and _Recall_ to assess
the detector.

###  IV-B Implementation Details

In implementation, the time window size $T$ is set to $5$. For all methods, we
reshape the resolution of infrared frames to $512\times 512$, as fair as
possible. We train our Tridos for 100 epochs with batch size 4. The initial
learning rate is 0.001 and adjusts adaptively with the training epoch. Adam is
employed as an optimizer with the CosineAnnealingLR scheduler and momentum
0.937. We initialize the model weights with the normal distribution. For
hyper-parameters, we set $\lambda_{reg}$, $\lambda_{cls}$, and $\lambda_{obj}$
in Eq. (19) to 5, 1, and 1, respectively. The $\alpha$ and $\beta$ in Eq. (20)
are both 0.5. During model testing, the IoU threshold for non-maximum
suppression is 0.65, and only predicted boxes with a confidence greater than
0.001 are retained. Regarding hardware, we conduct experiments on two Nvidia
GeForce 3090 GPUs.

###  IV-C Comparison With Other Methods

We choose several representative methods for comparison, including model-
driven and data-driven ones. Since most of the methods are based on pixel-
level segmentation, we follow the paradigm of combined detector [32] for a
fair comparison.

####  IV-C1 Quantitative Comparison

The quantitative results of different detection methods on three datasets are
shown in Table I. From this table, we could have two obvious findings.

One is that our Tridos achieves the best performance on most evaluation
metrics over three datasets, especially on $\text{mAP}_{50}$ and F1 score. For
example, on DAUB dataset, Tridos could obtain the highest $\text{mAP}_{50}$
97.80% and the highest F1 99.43%. In terms of Recall, the 99.67% by Tridos is
only slightly lower than the SOTA 99.79% by SSTNet. Furthermore, on ITSDT-15K,
our Tridos still obtain the highest $\text{mAP}_{50}$ 80.41% and F1 90.65%,
far superior to the old SOTA $\text{mAP}_{50}$ 76.96%, and F1 score 88.07% by
SSTNet. Additionally, we also outperform the comparison methods on almost all
metrics on IRDST.

Figure 4: The visualization comparisons of different methods on DAUB. GT is
ground truth. Red, blue and green boxes represent correctly detected targets,
amplified detection regions and false alarms. Figure 5: The visualization
comparisons of different methods on ITSDT-15K. Red, blue, and green boxes
represent correctly detected targets, amplified detection regions, and false
alarms.

The other is that our Tridos has strong robustness than other ones. Almost all
methods perform better on DAUB than on ITSDT-15K and IRDST datasets. For
example, on DAUB the SSTNet achieves $\text{mAP}_{50}$ 95.59% and F1 98.09%,
but just $\text{mAP}_{50}$ 76.96%, F1 88.07% on ITSDT-15K, and
$\text{mAP}_{50}$ 71.55%, F1 85.11% on IRDST. That could be because the
ITSDT-15K and IRDST datasets contain more complex scenes with the effects of
noise and occlusion. Nevertheless, Tridos improves $\text{mAP}_{50}$ by 2.21%
and F1 by 1.34% on DAUB. On ITSDT-15K and IRDST, $\text{mAP}_{50}$ improves by
3.45% and 2.17%, respectively, and F1 improves by 2.58% and 1.74%,
respectively. This shows that our method is tolerant to complex scenarios.
Besides, model-driven methods usually perform less well than data-driven
methods. One possible reason is that they rely on hand-crafted features and do
not have the ability to adaptively learn target features.

####  IV-C2 Visual Comparison

We present the visual comparisons of different methods in Fig. 4 \- Fig. 6.

Figure 6: The visualization comparisons of different methods on IRDST. Red,
blue and green boxes represent correctly detected targets, amplified detection
regions and false alarms.

It is evident that our proposed method could usually accurately detect small
moving targets. In contrast, other ones often lead to miss detection or false
detection.

For example, in Fig. 4, on DAUB, our Tridos precisely detects the target
occluded by grass. However, ACM and RISTD cause miss detection. AGPC
incorrectly treats a bright spot as a target, producing false detection.
Moreover, in Fig. 5, on ITSDT-15K, ACM, RISTD, AGPC, DNANet fail to detect all
targets. MSHNet even detects five targets, causing a false detection.
Meanwhile, in Fig. 6, on IRDST, ACM, RISTD, AGPC, and DNANet occur miss
detection. ISNet appears false detection and it seems to have a bigger
bounding box than ground truth. Besides, our Tridos produces bounding boxes
with the largest similarity to ground truth. In summary, we can see that the
qualitative comparison results of these visualizations correspond to the
quantitative results in Table I, indicating the ascendency of our method.

Figure 7: PR curves of fourteen representative methods on DAUB, ITSDT-15K and
IRDST.

####  IV-C3 PR Curve Comparison

To evaluate the comprehensive performance of different methods, we draw three
groups of _Precision_ -_Recall_ (PR) cures on DAUB, ITSDT-15K, and IRDST, as
shown in Fig. 7. By comparison, we could easily observe that our curves almost
always above other ones on three datasets. The larger the area under the
curve, the better the method. Therefore, the comparisons of these three groups
of PR curves indicate that our Tridos has the best comprehensive performance
with an optimal balance between _Precision_ and _Recall_.

TABLE II:  Comparative results on the model complexity of some representative
methods on ITSDT-15K. The best results are marked in bold.

Methods | Frames |  $\text{mAP}_{\textbf{50}}$ $\uparrow$ |  F1 $\uparrow$ |  Params $\downarrow$ |  GFlops $\downarrow$ |  FPS $\uparrow$
---|---|---|---|---|---|---
ACM [9] | 1 | 55.38 | 74.88 | 3.04M | 24.73 | 29.11
RISTD [43] | 1 | 60.47 | 77.93 | 3.28M | 76.28 | 10.21
ISNet [10] | 1 | 62.29 | 79.18 | 3.49M | 265.73 | 11.20
UIUNet [17] | 1 | 65.15 | 81.13 | 53.06M | 456.70 | 3.63
SANet [44] | 1 | 62.17 | 78.64 | 12.40M | 42.04 | 10.55
AGPCNet [14] | 1 | 67.27 | 82.16 | 14.88M | 366.15 | 4.79
RDIAN [15] | 1 | 68.49 | 82.68 | 2.74M | 50.44 | 20.52
DNANet [12] | 1 | 70.46 | 84.46 | 7.22M | 135.24 | 4.82
SIRST5K [45] | 1 | 61.52 | 78.36 | 11.48M | 182.61 | 7.37
MSHNet [46] | 1 | 60.82 | 77.64 | 6.59M | 69.59 | 18.55
RPCANet [19] | 1 | 62.28 | 79.22 | 3.21M | 382.69 | 15.89
DTUM [31] | 5 | 67.97 | 82.79 | 9.64M | 128.16 | 14.28
SSTNet [32] | 5 | 76.96 | 88.07 | 11.95M | 123.60 | 7.37
Ours | 5 | 80.41 | 90.65 | 14.13M | 130.72 | 13.71

####  IV-C4 Model Complexity Comparison

We compare the model complexity of our Tridos with thirteen representative
methods in Table II, mainly in _Params_ , _GFlops_ , and _FPS_. In table, we
could obviously have two discoveries.

TABLE III: Ablation study on different components on two datasets, including
MSRM, TDEM, LGFM and RCU. RCU(A) only use $\text{RCU}_{1}$, RCU(B) uses
$\text{RCU}_{1}$ and $\text{RCU}_{2}$, and RCU(C) uses all RCUs.

Settings | MSRM | TDEM | LGFM | RCU (A) | RCU (B) | RCU (C) | DAUB | ITSDT-15K
---|---|---|---|---|---|---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
w/o All | - | - | - | - | - | - | 84.43 | 89.73 | 95.45 | 92.50 | 71.95 | 83.43 | 87.35 | 85.34
w MSRM | ✓ | - | - | - | - | - | 92.17 | 94.92 | 93.86 | 94.39 | 75.36 | 83.57 | 91.19 | 87.21
w MSRM & TDEM | ✓ | ✓ | - | - | - | - | 93.73 | 94.51 | 96.24 | 95.37 | 76.77 | 87.03 | 89.72 | 88.35
w/o RCU | ✓ | ✓ | ✓ | - | - | - | 95.61 | 96.81 | 97.37 | 97.09 | 78.24 | 85.83 | 92.06 | 88.84
w RCU(A) | ✓ | ✓ | ✓ | ✓ | - | - | 95.74 | 97.61 | 97.12 | 97.36 | 79.08 | 89.66 | 89.50 | 89.58
w RCU(B) | ✓ | ✓ | ✓ | - | ✓ | - | 96.25 | 98.00 | 99.33 | 98.66 | 79.68 | 89.36 | 90.10 | 89.73
w RCU(C) | ✓ | ✓ | ✓ | - | - | ✓ | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

One is that our Tridos has a slight increase in _Params_ and _GFlops_. For
example, the smallest _Params_ in multi-frame methods is 9.64M by DTUM. the
best _GFlops_ in multi-frame methods is 123.60 by SSTNet. Our Tridos has a
medium _Params_ 14.13M and _GFlops_ 130.72. One probable reason is that SSTNet
and DTUM are based on spatial-temporal domain, but our method introduces the
frequency domain and needs to deal with triple domains. Overall, compared to
the performance gain implied in Table I, these costs are acceptable and
worthwhile.

The other is that the increase in computational costs results in a middle
inference speed with a _FPS_ of 13.71. Besides, since multi-frame methods
require processing five frames of images during inference, their FPS is lower
than some single-frame methods, like ACM, RDIAN, MSHNet, and RPCANet.
Nevertheless, our FPS is still seems higher than RISTD, ISNet, UIUNet,
AGPCNet, DNANet, SIRST5K, and SSTNet.

###  IV-D Ablation Study

####  IV-D1 Effects of Different Components

We conduct four groups of ablation studies on DAUB and ITSDT-15K to
investigate the role of each component in our method, as shown in Table III \-
VI.

From Table III, we could have two findings. One is that each component
improves network performance to some extent. For example, on DAUB, the
baseline without any components just acquires 84.43% and 92.50% on
$\text{mAP}_{50}$ and F1, respectively. MSRM could increase the
$\text{mAP}_{50}$ from 84.43% to 92.17%, the F1 from 92.50% to 94.39%. With
the assembly of TDEM, the $\text{mAP}_{50}$ and F1 could be further improved
to 93.73% and 95.37%, respectively. Furthermore, adding frequency domain
processing (w/o RCU) could provide a more distinct gain, with
$\text{mAP}_{50}$ and F1 rising to 95.61% and 97.09%. Besides, the network
performs variably with the help of different kinds of RCUs (A, B, and C).
Another is that the best performance could be obtained by applying these
components together, proving the complementary of each component. For example,
when assembling all components, on DAUB, $\text{mAP}_{50}$ and F1 are
increased to 97.80% and 99.43%. On ITSDT-15K, $\text{mAP}_{50}$ and F1 are
refreshed to 80.41% and 90.65%, respectively.

To explore the potential contribution of NAB and MEU in MSRM more deeply, we
conduct a group of ablation studies, as shown in Table IV. In it, MEU plays an
essential role in modeling inter-frame spatial relationships. For example,
with the help of MEU, the $\text{mAP}_{50}$ and F1 on DAUB could be increased
from 94.92% to 97.80%, and 98.19% to 99.43%, respectively. It implies the
effectiveness of our memory mechanism. Similarly, on ITSDT-15K, NAB could
improve $\text{mAP}_{50}$ from 78.49% to 80.41%, and F1 from 89.08% to 90.65%.
It could be attributed to the ability of NAB to capture long-distance
dependencies.

TABLE IV: Ablation study on nonlocal attention block (NAB) and memory
enhancement unit (MEU) of MSRM.

Settings | DAUB | ITSDT-15K
---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
MSRM w/o NAB | 96.53 | 98.24 | 99.52 | 98.88 | 78.49 | 88.03 | 90.16 | 89.08
MSRM w/o MEU | 94.92 | 97.57 | 98.81 | 98.19 | 74.05 | 84.91 | 88.74 | 86.78
MSRM w/o All | 93.47 | 95.33 | 99.04 | 97.15 | 73.16 | 85.67 | 86.29 | 85.97
MSRM | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

From Table V, it is obvious that using two residual blocks together will get
the best performance. For example, on ITSDT-15K, TDEM without residual blocks
(TDEM w/o All) only obtain 74.05% and 86.78% on $\text{mAP}_{50}$ and F1. With
ResB1 (TDEM w ResB1), $\text{mAP}_{50}$ and F1 will raise to 76.25% and
87.69%. Moreover, with all residual blocks (TDEM), $\text{mAP}_{50}$ and F1
will further improve 4.13% and 2.27%. This could be because ResB1 and ResB2
enhance the differential information from different perspectives, formulating
more representative motion features.

TABLE V: Ablation study on two residual blocks of TDEM.

Settings | DAUB | ITSDT-15K
---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
TDEM w/o ResB1 | 95.49 | 98.08 | 98.87 | 98.48 | 77.21 | 86.58 | 90.25 | 88.38
TDEM w/o ResB2 | 94.45 | 96.24 | 99.42 | 97.80 | 76.25 | 84.57 | 91.05 | 87.69
TDEM w/o All | 93.89 | 96.37 | 98.87 | 97.60 | 74.05 | 84.91 | 88.74 | 86.78
TDEM | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

Additionally, We investigate the effects of _Fourier Transform_ (FT), _Conv
Branch_ (ConvB), and _Swin Transformer Branch_ (SwinB) in Table VI.

From it, we could have two apparent findings. First, the global context
acquired by the SwinB is as valuable as the local features obtained by the
ConvB. For example, on DAUB, without ConvB (LGFM w/o ConvB), obtain 95.95% and
98.72%, and without SwinB achieve 95.76% and 98.39% on $\text{mAP}_{50}$ and
F1, with almost the same decrease. Second, frequency transformation is a
crucial strategy for detecting small targets. For example, on ITSDT-15K, “LGFM
w/o FT” only acquires 73.54% and 85.47% on $\text{mAP}_{50}$ and F1, even
assemble with two branches. It proves that our LGFM is a successful strategy
for perceiving frequency information.

TABLE VI: Ablation study of Fourier transform (FT), local branch (ConvB) and
global branch (SwinB) of LGFM.

Settings | DAUB | ITSDT-15K
---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
LGFM w/o ConvB | 95.95 | 98.61 | 98.94 | 98.72 | 76.73 | 86.04 | 90.35 | 88.14
LGFM w/o SwinB | 95.76 | 97.52 | 99.27 | 98.39 | 74.77 | 82.01 | 92.37 | 86.88
LGFM w/o FT | 94.27 | 97.46 | 97.60 | 97.53 | 73.54 | 86.38 | 84.59 | 85.47
LGFM w/o All | 92.93 | 97.14 | 96.45 | 96.80 | 72.90 | 80.64 | 89.61 | 84.89
LGFM w All | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

####  IV-D2 Effects of the Time Window Size $T$

We conduct a group of experiments with different time window sizes to
investigate the impact of the time window $T$ on the detection performance of
our Tridos, as shown in Fig. 8.

Figure 8: The effects of the time window $T$ on our Tridos.

The time window $T$ could provide context information for the keyframe. In
Figure, we could have two findings. First, our method obtains the peak
detection performance on two datasets when the time window size $T=5$. Second,
the proper time window size is crucial for infrared small target detection.
For example, when $T\leq 3$, increasing the size of $T$ results in less gain,
but when $T>3$, the detection performance improves more evidently.
Furthermore, a further increase in $T$ when $T>5$ will lead to fluctuation or
even a decrease in metrics. This could be attributed to the fact that
capturing the temporal context of consecutive frames would be insufficient if
the time window size is too small. Conversely, if the size is too large, there
would be a risk of introducing excessive redundant messages and causing
interference. Therefore, we set the time window size $T=5$ in our final
experiments.

####  IV-D3 Effects of Different Detection Losses

To validate the impacts of dual view regression loss, we further design a
group of experiments, as shown in Table VII.

By comparison, our dual view regression loss gains the peak performance. For
example, on DAUB, only use IoU loss realizes 96.15% and 98.56% on
$\text{mAP}_{50}$ and F1. Nonetheless, only employing NWD loss gets 93.66% and
97.48% on $\text{mAP}_{50}$ and F1. One possible reason is that NWD loss
improves detection accuracy by minimizing the difference in distribution
between bounding boxes. However, it is sensitive to noise, which could result
in inaccurate target localization. In summary, our dual view regression loss
incorporates the advantages of balancing small target localization and the
distribution of bounding boxes.

TABLE VII: The performance comparisons of different losses.

Settings | DAUB | ITSDT-15K
---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
$\mathcal{L}_{iou}$ | 96.15 | 98.09 | 99.04 | 98.56 | 78.20 | 86.71 | 89.85 | 88.25
$\mathcal{L}_{nwd}$ | 93.66 | 96.45 | 98.54 | 97.48 | 73.87 | 82.17 | 84.45 | 83.29
$\mathcal{L}_{dvr}$ | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

####  IV-D4 Effects of Different Hyper-parameters

As shown in Fig. 9, we design two groups of experiments to evaluate the
effects of hyper-parameters $w$ (the window size of the swin transformer) and
$c$ (the hidden channel numbers) on Tridos. It indicates that F1 reaches a
peak on DAUB and ITSDT-15K when the window size of the swin transformer is
$8$. Similar to this, the highest values of F1 are obtained when the hidden
channel numbers $c=128$. Besides, we could find that Tridos’s detection
performance is more sensitive to $w$ than $c$. Overall, the optimal window
size of the swin transformer and hidden channel numbers are 8 and 128,
respectively.

Figure 9: The effects of hyper-parameters $w$ and $c$ on our Tridos.

####  IV-D5 Comparison with Swin Transformer and YOLOX

To further illustrate the effectiveness of our method, we compare it with a
conventional swin transformer and YOLOX, as shown in Table VIII.

TABLE VIII: The performance comparisons with swin transformer (SwinT) and
YOLOX.

Methods | DAUB | ITSDT-15K
---|---|---
$\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1 | $\text{mAP}_{\textbf{50}}$ | Precision | Recall | F1
SwinT | 83.40 | 90.46 | 93.45 | 91.93 | 49.93 | 78.56 | 64.30 | 70.72
YOLOX | 85.62 | 90.73 | 94.75 | 92.70 | 72.15 | 84.43 | 86.85 | 85.62
Ours | 97.80 | 99.20 | 99.67 | 99.43 | 80.41 | 90.71 | 90.60 | 90.65

We could find that our Tridos with frequency-aware and memory enhancement
achieves evidently higher detection metrics on two datasets. For example, on
ITSDT-15K, Tridos could acquire an $\text{mAP}_{50}$ 80.41%, _Precision_
90.71%, _Recall_ 90.60%, and F1 90.65%, much better than swin transformer and
YOLOX. This group of ablation studies further proves that our method’s
superior performance is due to the strategy of triple-domain feature learning,
which eliminates the effects of the swin transformer and YOLOX architecture.

##  V Conclusion

For extending feature learning perspective, this paper proposes Tridos to
capture the saptial-temporal-frequency features of moving infrared small
targets. It enhances spatial-temporal feature representation through frequency
domain and models inter-frame spatial relationships by a memory enhancement
mechanism. To comprehensively evaluate different detection methods, we collect
an additional MISTD dataset ITSDT-15K. The comparison experiments on DAUB,
IRDST, and our own one prove the superiority of proposed Tridos to existing
SOTA detection methods. Moreover, ablation studies further verify the
effectiveness and merits of all elaborately-designed components in our Tridos.
Although achieving expected detection performance, the utilization of
frequency domain will inevitably increase network parameters and computation
costs. More effective and lightweight triple-domain detection schemes could be
worth further investigation in future work.

## References

  * [1] X. Bai and F. Zhou, “Analysis of new top-hat transformation and the application for infrared dim small target detection,” _Pattern Recognition_ , vol. 43, no. 6, pp. 2145–2156, 2010.
  * [2] S. D. Deshpande, M. H. Er, R. Venkateswarlu, and P. Chan, “Max-mean and max-median filters for detection of small targets,” in _Signal and Data Processing of Small Targets 1999_ , vol. 3809. SPIE, 1999, pp. 74–83.
  * [3] C. P. Chen, H. Li, Y. Wei, T. Xia, and Y. Y. Tang, “A local contrast method for small infrared target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 52, no. 1, pp. 574–581, 2013.
  * [4] J. Han, K. Liang, B. Zhou, X. Zhu, J. Zhao, and L. Zhao, “Infrared small target detection utilizing the multiscale relative local contrast measure,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 15, no. 4, pp. 612–616, 2018.
  * [5] J. Han, S. Moradi, I. Faramarzi, C. Liu, H. Zhang, and Q. Zhao, “A local contrast method for infrared small-target detection utilizing a tri-layer window,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 17, no. 10, pp. 1822–1826, 2019.
  * [6] J. Han, S. Moradi, I. Faramarzi, H. Zhang, Q. Zhao, X. Zhang, and N. Li, “Infrared small target detection based on the weighted strengthened local contrast measure,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 18, no. 9, pp. 1670–1674, 2020.
  * [7] C. Gao, D. Meng, Y. Yang, Y. Wang, X. Zhou, and A. G. Hauptmann, “Infrared patch-image model for small target detection in a single image,” _IEEE Transactions on Image Processing_ , vol. 22, no. 12, pp. 4996–5009, 2013.
  * [8] X. Wang, Z. Peng, D. Kong, and Y. He, “Infrared dim and small target detection based on stable multisubspace learning in heterogeneous scene,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 10, pp. 5481–5493, 2017.
  * [9] Y. Dai, Y. Wu, F. Zhou, and K. Barnard, “Asymmetric Contextual Modulation for Infrared Small Target Detection,” in _2021 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , 2021, pp. 949–958.
  * [10] M. Zhang, R. Zhang, Y. Yang, H. Bai, J. Zhang, and J. Guo, “ISNet: Shape matters for infrared small target detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 877–886.
  * [11] H. Wang, L. Zhou, and L. Wang, “Miss detection vs. false alarm: Adversarial learning for small object segmentation in infrared images,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 8509–8518.
  * [12] B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, and Y. Guo, “Dense nested attention network for infrared small target detection,” _IEEE Transactions on Image Processing_ , vol. 32, pp. 1745–1758, 2022.
  * [13] Y. Dai, Y. Wu, F. Zhou, and K. Barnard, “Attentional local contrast networks for infrared small target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 59, no. 11, pp. 9813–9824, 2021.
  * [14] T. Zhang, L. Li, S. Cao, T. Pu, and Z. Peng, “Attention-Guided Pyramid Context Networks for Detecting Infrared Small Target Under Complex Background,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 59, no. 4, pp. 4250–4261, 2023.
  * [15] H. Sun, J. Bai, F. Yang, and X. Bai, “Receptive-Field and Direction Induced Attention Network for Infrared Dim Small Target Detection With a Large-Scale Dataset IRDST,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–13, 2023.
  * [16] J. Lin, S. Li, L. Zhang, X. Yang, B. Yan, and Z. Meng, “IR-TransDet: Infrared Dim and Small Target Detection With IR-Transformer,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–13, 2023.
  * [17] X. Wu, D. Hong, and J. Chanussot, “UIU-Net: U-Net in U-Net for infrared small object detection,” _IEEE Transactions on Image Processing_ , vol. 32, pp. 364–376, 2022.
  * [18] B. Yang, X. Zhang, J. Zhang, J. Luo, M. Zhou, and Y. Pi, “EFLNet: Enhancing Feature Learning Network for Infrared Small Target Detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 62, pp. 1–11, 2024.
  * [19] F. Wu, T. Zhang, L. Li, Y. Huang, and Z. Peng, “RPCANet: Deep Unfolding RPCA Based Infrared Small Target Detection,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2024, pp. 4809–4818.
  * [20] C. Kwan and B. Budavari, “Enhancing small moving target detection performance in low-quality and long-range infrared videos using optical flow techniques,” _Remote Sensing_ , vol. 12, no. 24, p. 4024, 2020.
  * [21] F. Zhao, T. Wang, S. Shao, E. Zhang, and G. Lin, “Infrared moving small-target detection via spatiotemporal consistency of trajectory points,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 17, no. 1, pp. 122–126, 2020.
  * [22] F. Wu, H. Yu, A. Liu, J. Luo, and Z. Peng, “Infrared small target detection using spatio-temporal 4d tensor train and ring unfolding,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2023.
  * [23] Y. Luo, X. Li, and S. Chen, “Feedback spatial-temporal infrared small target detection based on orthogonal subspace projection,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2024.
  * [24] G. Wang, B. Tao, X. Kong, and Z. Peng, “Infrared small target detection using nonoverlapping patch spatial–temporal tensor factorization with capped nuclear norm regularization,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–17, 2021.
  * [25] T. Liu, J. Yang, B. Li, C. Xiao, Y. Sun, Y. Wang, and W. An, “Nonconvex tensor low-rank approximation for infrared small target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–18, 2021.
  * [26] F. Zhang, C. Li, and L. Shi, “Detecting and tracking dim moving point target in ir image sequence,” _Infrared Physics & Technology_, vol. 46, no. 4, pp. 323–328, 2005.
  * [27] X. Ren, J. Wang, T. Ma, K. Bai, M. Ge, and Y. Wang, “Infrared dim and small target detection based on three-dimensional collaborative filtering and spatial inversion modeling,” _Infrared physics & technology_, vol. 101, pp. 13–24, 2019.
  * [28] X. Sun, X. Liu, Z. Tang, G. Long, and Q. Yu, “Real-time visual enhancement for infrared small dim targets in video,” _Infrared physics & technology_, vol. 83, pp. 217–226, 2017.
  * [29] J. Du, H. Lu, L. Zhang, M. Hu, S. Chen, Y. Deng, X. Shen, and Y. Zhang, “A spatial-temporal feature-based detection framework for infrared dim small target,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–12, 2021.
  * [30] P. Yan, R. Hou, X. Duan, C. Yue, X. Wang, and X. Cao, “STDMANet: Spatio-temporal differential multiscale attention network for small moving infrared target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–16, 2023.
  * [31] R. Li, W. An, C. Xiao, B. Li, Y. Wang, M. Li, and Y. Guo, “Direction-coded temporal u-shape module for multiframe infrared small target detection,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2023.
  * [32] S. Chen, L. Ji, J. Zhu, M. Ye, and X. Yao, “SSTNet: Sliced Spatio-Temporal Network With Cross-Slice ConvLSTM for Moving Infrared Dim-Small Target Detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 62, pp. 1–12, 2024.
  * [33] X. Tong, Z. Zuo, S. Su, J. Wei, X. Sun, P. Wu, and Z. Zhao, “ST-Trans: Spatial-Temporal Transformer for Infrared Small Target Detection in Sequential Images,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2024.
  * [34] R. Fu, H. Fan, Y. Zhu, B. Hui, Z. Zhang, P. Zhong, D. Li, S. Zhang, G. Chen, and L. Wang, “A dataset for infrared time-sensitive target detection and tracking for air-ground application,” May 2022.
  * [35] Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “YOLOX: Exceeding yolo series in 2021,” _arXiv preprint arXiv:2107.08430_ , 2021.
  * [36] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_. Springer, 2014, pp. 740–755.
  * [37] J. W. Cooley, P. A. Lewis, and P. D. Welch, “The fast fourier transform and its applications,” _IEEE Transactions on Education_ , vol. 12, no. 1, pp. 27–34, 1969.
  * [38] Z. Liu, J. Ning, Y. Cao, Y. Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 3202–3211.
  * [39] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 2980–2988.
  * [40] C. Xu, J. Wang, W. Yang, H. Yu, L. Yu, and G.-S. Xia, “Detecting tiny objects in aerial images: A normalized wasserstein distance and a new benchmark,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 190, pp. 79–93, 2022.
  * [41] Z. Zheng, P. Wang, D. Ren, W. Liu, R. Ye, Q. Hu, and W. Zuo, “Enhancing geometric factors in model learning and inference for object detection and instance segmentation,” _IEEE Transactions on Cybernetics_ , vol. 52, no. 8, pp. 8574–8586, 2021.
  * [42] B. Hui, Z. Song, H. Fan, P. Zhong, W. Hu, X. Zhang, J. Lin, H. Su, W. Jin, Y. Zhang, and Y. Bai, “A dataset for infrared image dim-small aircraft target detection and tracking under ground / air background,” Oct. 2019.
  * [43] Q. Hou, Z. Wang, F. Tan, Y. Zhao, H. Zheng, and W. Zhang, “RISTDnet: Robust infrared small target detection network,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2021.
  * [44] J. Zhu, S. Chen, L. Li, and L. Ji, “SANet: Spatial attention network with global average contrast learning for infrared small target detection,” in _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2023, pp. 1–5.
  * [45] Y. Lu, Y. Lin, H. Wu, X. Xian, Y. Shi, and L. Lin, “SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2024.
  * [46] Q. Liu, R. Liu, B. Zheng, H. Wang, and Y. Fu, “Infrared small target detection with scale and location sensitivity,” in _Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition_ , 2024.
  * [47] R. Kou, C. Wang, Z. Peng, Z. Zhao, Y. Chen, J. Han, F. Huang, Y. Yu, and Q. Fu, “Infrared small target segmentation networks: A survey,” _Pattern Recognition_ , vol. 143, p. 109788, 2023.
  * [48] M. Zhao, W. Li, L. Li, J. Hu, P. Ma, and R. Tao, “Single-frame infrared small-target detection: A survey,” _IEEE Geoscience and Remote Sensing Magazine_ , vol. 10, no. 2, pp. 87–119, 2022.
  * [49] X. Bai and Y. Bi, “Derivative entropy-based contrast measure for infrared small-target detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 56, no. 4, pp. 2452–2466, 2018.
  * [50] K. Xu, M. Qin, F. Sun, Y. Wang, Y.-K. Chen, and F. Ren, “Learning in the frequency domain,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 1740–1749.
  * [51] J. Li, L.-Y. Duan, X. Chen, T. Huang, and Y. Tian, “Finding the secret of image saliency in the frequency domain,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 37, no. 12, pp. 2428–2440, 2015.
  * [52] S. Chen, L. Ji, S. Zhu, and M. Ye, “MICPL: Motion-Inspired Cross-Pattern Learning for Small-Object Detection in Satellite Videos,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2024.
  * [53] S. Chen, J. Zhu, L. Ji, H. Pan, and Y. Xu, “Augtarget data augmentation for infrared small target detection,” in _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2023, pp. 1–5.