#  FlightScope: A Deep Comprehensive Assessment of Aircraft Detection
Algorithms in Satellite Imagery

Safouane EL GHAZOUALI*
Machine Learning Research
& Development, TOELT LLC AI lab
Winterthur, Swintzerland
<EMAIL_ADDRESS>
&Arnaud GUCCIARDI
Machine Learning Research
& Development, TOELT LLC AI lab
Winterthur, Swintzerland
<EMAIL_ADDRESS>
&Francesca VENTURINI
Machine Learning Research
& Development, TOELT LLC AI lab
Winterthur, Switzerland
Institute of Applied Mathematics
& Physics, ZHAW - Zurich
University of Applied Sciences,
Winterthur, Swintzerland
<EMAIL_ADDRESS>/
<EMAIL_ADDRESS>
&Nicola VENTURI
Competence Center for AI
and Simulation, armasuisse S+T,
Thun, Swintzerland
<EMAIL_ADDRESS>
&Michael RUEEGSEGGER
Competence Center for AI
and Simulation, armasuisse S+T,
Thun, Swintzerland
<EMAIL_ADDRESS>
&Umberto MICHELUCCI
Machine Learning Research
& Development, TOELT LLC AI lab
Winterthur, Switzerland
Computer Science Department,
Lucerne University of Applied
Science and Arts,
Luzern, Swintzerland
<EMAIL_ADDRESS>/
<EMAIL_ADDRESS>

###### Abstract

Object detection in remotely sensed satellite pictures is fundamental in many
fields such as biophysical, and environmental monitoring. While deep learning
algorithms are constantly evolving, they have been mostly implemented and
tested on popular ground-based taken photos. This paper critically evaluates
and compares a suite of advanced object detection algorithms customized for
the task of identifying aircraft within satellite imagery. Using the large
HRPlanesV2 dataset, together with a rigorous validation with the GDIT dataset,
this research encompasses an array of methodologies including YOLO versions 5
and 8, Faster RCNN, CenterNet, RetinaNet, RTMDet, and DETR, all trained from
scratch. This exhaustive training and validation study reveal YOLOv5 as the
preeminent model for the specific case of identifying airplanes from remote
sensing data, showcasing high precision and adaptability across diverse
imaging conditions. This research highlight the nuanced performance landscapes
of these algorithms, with YOLOv5 emerging as a robust solution for aerial
object detection, underlining its importance through superior mean average
precision, Recall, and Intersection over Union scores. The findings described
here underscore the fundamental role of algorithm selection aligned with the
specific demands of satellite imagery analysis and extend a comprehensive
framework to evaluate model efficacy. The benchmark toolkit and codes,
available via GitHub, aims to further exploration and innovation in the realm
of remote sensing object detection, paving the way for improved analytical
methodologies in satellite imagery applications.

_K_ eywords Object detection $\cdot$ survey $\cdot$ remote sensing $\cdot$
satellite image $\cdot$ aircraft localization

##  1 Introduction

Remote sensing plays a fundamental role in acquiring information about the
Earth’s surface using various types of vision sensors such as Thermal Infra-
Red (TIR) and RGB cameras [1]. This field encompasses a wide range of
technologies and methodologies aimed at capturing, analyzing, and interpreting
data from various sources. Within the realm of remote sensing, one of the most
significant applications is the detection and localization of small objects
[2]. Object detection from satellite imagery holds great importance in various
domains, such as defence and military applications [3], urban studies [4],
airport surveillance, vessel traffic monitoring [5] and transportation
infrastructure determination [6, 7]. Unlike photographic pictures, remote
sensing images obtained from satellite sensors are more difficult to interpret
due to factors such as atmospheric interference, viewpoint variation,
background clutter, and illumination differences [8, 9]. Additionally,
satellite images cover larger areas, typically surface like 10 $\times$ 10 km2
per frame, representing the intricate landscape of the Earth’s surface with
two-dimensional images that possess less spatial detail compared to digital
photographs from cameras.

Traditional approaches to aircraft detection relied on manual feature
engineering and machine learning techniques. However, these methods often
struggles to handle the complexities of satellite imagery and achieve high
accuracy. The advent of deep learning, particularly Convolutional Neural
Networks (CNNs), has revolutionized object detection tasks by enabling the
automatic extraction of intricate visual representations. One notable deep
learning method for object detection is the You Only Look Once (YOLO) [10].
YOLO divides the input image into a grid and predicts bounding boxes and class
probabilities directly from the grid cells. Its architecture allows for real-
time detection and can handle multiple object classes simultaneously. However,
YOLO may encounter challenges in accurately localizing small objects due to
the grid cell structure and limited receptive fields. Another popular deep
learning technique is the Single Shot MultiBox Detection (SSD) approach [11],
which utilizes a series of convolutional layers to generate a diverse set of
default bounding boxes at different scales and aspect ratios. By applying a
set of predefined anchor boxes to the feature maps, SSD performs multi-scale
object detection efficiently. However, it may face difficulties in detecting
small objects and suffers from a large number of default boxes, leading to
computational overhead. Region-based CNNs (RCNN) [12] and its variants, such
as Fast RCNN [13], Faster RCNN [14] and Mask RCNN [15], have also been widely
used for object detection tasks. These methods employ a two-stage approach, in
which potential object regions are first proposed and then classified and
refined. By utilizing region proposals, these methods achieve accurate
localization and exhibit strong performance. However, they are computationally
expensive and slower than one-stage detectors such as YOLO and SSD. In
addition to YOLO, SSD, and RCNN variants, there are several other deep
learning methods that have been explored for object detection, such as
EfficientDet [16], RetinaNet [17], and CenterNet [18]. More recently, the
success of attention mechanism in the field of language processing has
received a lot of consideration and has been brought into the field of
computer vision as well [19]. This rise has led to an interesting improvement
of performance in many computer vision subfields, including image
classification [20] and object detection [21]. Each of these methods has its
own unique architecture and advantages, aiming to improve both accuracy and
efficiency in object detection tasks.

Table 1: Overview of the different neural network architectures implemented, trained and validated in this paper. Models Tested |  Short Description
---|---
YOLO v5 [22] |  Real-time object detection model that processes images in one pass through a CNN, widely used for various applications including autonomous vehicles, surveillance systems, and robotics.
YOLO v8 [10] |  Updated version of the YOLO object detection system, incorporating advancements in network architecture and training techniques to achieve better efficiency in real-time object detection tasks.
RTMDet [23] |  An improved CNN network for higher accuracy while maintaining the same real-time performance of the YOLO model series.
DETR [24] |  New object detection architecture based on transformers and attention mechanisms. It has proved its efficiency on the COCO dataset.
Faster RCNN [14] |  Region-Based CNN is an architecture for object detection that has proven its efficiency in accurate bounding box extraction. Faster RCNN is an algorithm that improves the detection speed of RCNNs.
SSD [11] |  Another real-time object detection model that predicts bounding boxes from features map at multiple scales.
CenterNet [18] |  This model, initially based on the CornerNet [25] architecture, has been able to achieve state-of-the-art performance on COCO dataset for object localization in term of real-time and accuracy.
RetinaNet [26] |  Real-time object detection model that addresses the imbalance between foreground and background examples during training using a novel focal loss function.

The objective of this study is to benchmark and compare multiple state-of-the-
art object detection methods prepared and trained specifically for the use
case of aircraft detection in satellite images. The focus is on their
application in satellite surveillance, and air traffic management. This paper
significantly contributes to the research on satellite imagery analysis by
implementing, training and validating one variant of the eight leading object
detection neural network architectures listed in Table 1, alongside other
cutting-edge deep learning architectures. The authors have directly
implemented and tested each network to control the testing conditions and
prevent bias or spurious outcomes that might arise. In addition to a detailed
overview of object detection models that is fundamental for both researchers
and practitioners, this work provides also a thorough examination of their
performance, precision, and computational complexity. The findings provide
critical information that improves the selection process for the most
efficient aircraft detection methods in satellite imagery. This is supported
by comprehensive training and initial validation on the HRPlanesv2 and further
validation on GDIT datasets (for details on the datasets see Section 3),
marking an important advancement in the study of the precision and efficiency
of remote sensing technologies and their application in real-world scenarios,
thus underlining its substantial relevance and potential impact on future
research in the field.

Additionally, to make a more useful contribution to the computer vision
community, all the code used for the benchmarks in this paper is available,
and could be reproduced from the GitHub repository at
https://github.com/toelt-llc/FlightScope_Bench.

Following the introduction, the rest of the paper is structured as follow:
similar comparative studies are discussed and reviewed in Section 2. An
overview of existing datasets including airplanes in satellite images is
discussed in Section 3, while in Section 4 a more detailed state-of-the-art
description of the object detection architectures (listed in Table 1) is
given. Section 5 present the setup of the benchmark, along with the results
and discussion. Finally, Section 6 concludes and summarizes the findings of
this comparative study.

##  2 Related work

Many remote sensing methods and models have been studied and proposed during
the past decade in various fields such as environmental monitoring [27],
object and image Geo-localization [28, 29], urban planning [30], and
agriculture [29]. In the context of object detection, the methods that are
usually proposed are trained, validated and tested on images gathered from
ground and usual vision sensors. On those types of images, good performances
are observed, therefore, the need for performance evaluation of remote sensing
types of images. Remote sensing imagery typically encompasses vast areas with
varying resolutions, making the detection of small objects, such as vehicles
or infrastructure, particularly challenging [31]. Additionally, factors such
as varying illumination conditions, occlusions due to weather, and object
scale variations further complicate the task.

A similar study to this work, conducted by Alganci, et. al. [9] delves into
the detection of small objects from satellite imagery. This research focusses
on evaluating the performance of three state-of-the-art convolutional neural
network (CNN)-based object detection models specifically tailored for
identifying airplanes in very high-resolution (VHR) satellite images. The
authors underscore the importance of accurate and efficient detection methods
in satellite imagery due to its large data size and expansive aerial coverage.
For their study, the authors have used the DOTA dataset [32], a multiple
classes open-source repository explicitly created for object detection in
remote sensing images. This dataset encompasses satellite image patches
sourced from platforms such as Google Earth, Jilin 1 (JL-1), and Gaofen 2
(GF-2) satellites, featuring 15 object categories, among which is the airplane
class. The comparative evaluation conducted in [9] assesses three object
detection models: Faster R-CNN [14], SSD [11], and YOLO-v3 [33], using the
DOTA dataset for both training and testing. Performance metrics including COCO
metrics, F1 scores, and processing time are employed for evaluation. The
summary of this work reveal that Faster R-CNN exhibits superior detection
accuracy, with YOLO-v3 showcasing faster convergence capabilities. SSD,
although proficient in object localization, faces challenges with training
convergence.

Additionally, another study presented in [34] focusses on the development and
evaluation of the first version of the HRPlanes benchmark dataset for deep
learning-based airplane detection using satellite imagery from Google Earth.
The authors describe the HRPlanes dataset and some of the images captured by
different satellites to represent diverse landscapes, seasonal variations, and
satellite geometry conditions. The dataset is then selected for training and
validation of two widely used object detection methods, YOLOv4 [35] and Faster
R-CNN [14]. The comparative study between YOLOv4 and Faster R-CNN in the
context of airplane detection from satellite imagery reveals interesting
findings. The study highlights that the boundaries of bounding boxes for
YOLOv4 are better at certain scales compared to Faster R-CNN. For instance, in
some cases, YOLOv4 performs better in detecting small airplanes, while Faster
R-CNN excels in detecting larger ones. The results also indicate that YOLOv4
is more effective in creating accurate bounding boxes for commercial planes in
large-scale imagery, possibly due to the presence of boarding bridges near the
planes. Additionally, both deep learning models demonstrate the ability to
detect moving planes, even in scenarios with motion blur effects in the
images.

##  3 Aircraft Datasets

In the field of aircraft detection and remote sensing, access to high-quality
and diverse datasets is important for the development and evaluation of
computer vision algorithms. In some comparative studies, DOTA dataset [32] is
selected, this latter encompasses objects other than aircrafts such as
airports, bridges and containers. This section reviews a collection of
datasets specifically and exclusively designed for aircraft detection
research, each offering unique features and advantages. An overview of these
datasets is given in Table 2.

Table 2: Overview of the available datasets exclusively created for aircraft detection from aerial imagery. Dataset |  Short Description |  Number of Images
---|---|---
Airbus aircraft [36] |  The Airbus Aircraft Dataset is extracted from a larger deep learning dataset, created with the use of Airbus satellite imagery. The dataset draws its primary imagery from the Pleiades twin satellites operated by Airbus. Images have a resolution of approximately 50 cm per pixel, stored as JPEG files. These images have a resolution of 2560 x 2560 pixels, representing an on-ground area of 1280 metres. |  109
HRPlanesV2 [34] |  The HRPlanesv2 dataset is comprised of 2,120 ultra-high-resolution images from Google Earth, featuring a total of 14,335 labelled aircrafts. Each image is preserved in JPEG format, measuring 4800 x 2703 pixels, and the labels for each aircraft are documented in the YOLO text format. |  2120
RarePlanes [37] |  This dataset incorporates real and synthetically generated satellite images. The ‘real’ portion of the dataset consists of 253 Maxar WorldView-3 satellite scenes, including 112 locations and 2142 km2 with 14700 hand-annotated aircrafts. The ‘synthetic’ portion features 50000 synthetic satellite images with roughly 63000 aircraft annotations. Only the ‘real’ part was used for this paper. |  253
GDIT [38] |  The GDIT Aerial Airport dataset is composed of aerial photographs that show parked airplanes. All varieties of plane are classified under a single category called ‘airplane.’ |  338
Planesnet [39] |  Planesnet is a collection of images extracted from the Planet satellite imagery. The main purpose of the dataset is the classification and localization of airplanes in medium-resolution images. The dataset includes 32000 very small images (20x20 pixels) labelled with either a "plane" or "no-plane" class. |  32000
Flying Airpl. [40] |  Flying Airplanes is a massive dataset that contains satellite images of flying airplanes that surround 30 different European airports. Images are from the Sentinel-2 satellite. |  Not available.
OPT-Aircraft [41] |  This dataset is a public remote sensing dataset with images stored in .png format that consists of 3594 data files with an approximate size of 69.3 MB. This dataset allows the identification of aircraft and classifies them according to their type and shape. |  3595

###  3.1 Airbus Aircraft Dataset

The Airbus Aircraft dataset [36] consists of 109 high-resolution images that
capture airplanes at various airports around the world. The images are taken
at airport gates or tarmacs and are categorized into two folders: ‘images’ and
‘extras’. The ‘images’ folder contains 103 pictures extracted from Pleiades
imagery, offering a resolution of approximately 50 cm. Each image is stored as
a JPEG file with dimensions of $2560\times 2560$ pixels, corresponding to a
ground area of 1280 m2. In particular, the dataset includes snapshots of
certain airports taken on different dates, allowing researchers to explore
temporal variations. Some images in the dataset also exhibit challenging
weather conditions such as fog or clouds. Additionally, the ‘extras’ folder
provides a separate set of images that can be used for testing purposes,
ensuring the evaluation of algorithms on completely unseen data.

###  3.2 HRPlanesv2 Dataset

The Google Earth HRPlanesv2 [34] dataset is a comprehensive collection of
high-resolution aerial images for aircraft detection research. It comprises
2120 images sourced from Google Earth, showcasing airports from diverse
regions and serving different purposes, including civil, military, and joint
airports. These images offer a rich variety of aircraft instances, providing
an extensive dataset for training and evaluation. Each image is stored as a
JPEG file with dimensions of $4800\times 2703$ pixels, ensuring detailed
representations of the airport scenes. To facilitate object detection tasks,
the dataset includes precise labels for 14,335 aircraft instances, provided in
the YOLO annotation format. Moreover, the dataset is divided into three
subsets: 70% for training, 20% for validation, and 10% for testing, enabling
researchers to assess and compare the performance of their algorithms
accurately.

###  3.3 RarePlanes Dataset

The RarePlanes dataset [37] comprises both real and synthetic satellite
imagery. Developed by CosmiQ Works and AI.Reverie, this dataset aims to
evaluate the efficacy of AI.Reverie’s synthetic data in enhancing computer
vision algorithms for aircraft detection in satellite imagery. The number of
the real images in the dataset comprises 253 Maxar WorldView-3 satellite
scenes, taken at 112 distinct locations and spanning an impressive 2142 km².
These scenes contain hand-annotated aircraft instances, totaling 14,700
annotations. In addition, the dataset includes 50,000 synthetic satellite
images generated using AI.Reverie’s advanced simulation platform. These
synthetic images feature approximately 630,000 aircraft annotations, providing
a valuable resource to explore the benefits of synthetic data in overhead
aircraft detection.

###  3.4 GDIT Dataset

The GDIT Aerial Airport dataset [38] is a specialized collection of aerial
images that focuses on parked airplanes at airports. It presents an
opportunity for researchers to explore aircraft detection algorithms in the
context of airport environments. The dataset consists of 338 high-quality
images with a resolution of 600 $\times$ 600 pixels, which are further
categorized into training, validation, and testing subsets with 236, 68, and
34 images, respectively. Notably, some of the training images exhibit
variations such as different filters, zoom levels, or rotations, resulting in
an expanded dataset of 810 images. The dataset offers a unified classification
label for all types of airplanes, simplifying the detection task.

###  3.5 Planesnet Dataset

The Planesnet dataset [39] provides an extensive collection of satellite
imagery extracted from Planet satellites, focusing on multiple airports in
California. This dataset comprises 32 000 RGB images, each measuring $20\times
20$ pixels. The images are meticulously labeled as either ’plane’ or ’no-
plane’ enabling researchers to train and evaluate aircraft detection
algorithms. Derived from PlanetScope full-frame visual scene products, the
dataset ensures an orthorectified 3 m pixel size, capturing fine-grained
details. The Planesnet dataset is available in two formats: a zipped directory
containing the PNG images and a JSON file containing corresponding metadata.
Each image is accompanied by a filename that includes the label, scene ID, and
metadata such longitude, and latitude coordinates.

###  3.6 Flying Airplanes Dataset

The dataset of Flying airplanes on satellite images [40] offers valuable
resources for research related to the detection of aircraft in satellite
imagery. It includes 180 satellite images covering areas of interest
surrounding 30 European airports. The dataset incorporates ground-truth
annotations of flying airplanes, which can be used to support various research
investigations. These annotations serve as a reference for developing and
evaluating algorithms for flying airplane detection. The dataset comprises
modified Sentinel-2 data processed by Euro Data Cube, providing high-quality
satellite imagery suitable for multiple applications.

###  3.7 OPT-Aircraft Dataset

The OPT-Aircraft V1.0 [41] dataset focuses on the identification of aircraft
groups in remote sensing images. It includes 3594 airplane images obtained
from various public datasets, such as DIOR, UCA AOD, NWPU VHR-10, DOTA, and
Google Earth. The dataset encompasses seven aircraft groups categorized based
on wings and propellers, further divided into fourteen sub-groups considering
aircraft color and engine position. The dataset, stored in PNG format,
consists of 3594 files with a compressed size of 69.3MB.

##  4 Literature Review: Object Detection

Many object detection models have been proposed in the past decades, the
majority are based on CNNs. Researchers have so far classified those models
into two primary categories: one-stage and two-stage architectures [42, 43].
However, the rise of self-attention mechanism [44] recently has led to a new
category of network architectures that are based on transformers. Fig. 1)
group some of the most used and known models along with two types of
classifications of deep learning models: (1) either categorized based on their
network architecture (one-stage, two-stages, or transformer), or (2) based on
their performances in real-time and detection accuracy highlighted in
respectively blue and yellow colors. The overlapping area drawn in green
feature the object detection models with a balanced performances in both real-
time detection and accuracy.

One-stage models [45] have been mainly known for their real-time deployment
performances, because they are usually not computationally extensive and a
single pass through the network is sufficient to produce estimations of object
bounding boxes. However, the main limitation in this category is the detection
accuracy, which might not be enough for some applications requiring very high
detection confidence. In the case of a two-stage object detection models, an
additional stage is introduced to generate generic object proposals which make
the model more efficient in its detection operation [46]. The purpose of this
stage is to produce candidate bounding boxes, which may not be highly
accurate, and effectively exclude background areas from further processing.
Subsequently, the next stage of the model undertakes the more computationally
intensive tasks of classifying objects and refining the bounding boxes
generated by the previous stage. The third category is transformer-based
architecture [47], which seems to be a good balance between accuracy on real-
time detection as it has been tested and compared on common datasets. This
category have been tested on common large datasets and it leverage the use of
the self-attention mechanism to produce reliable and accurate bounding box
estimations while being able to perform real-time detection.

One-stageDetection AccuracyReal-time inferenceTransformerTwo-stages
Overlapping Deep learning based Object Detection CenterNet [18]CornerNet
[25]YOLO [10]RetinaNet [26]RTMDet [18]SSD [11]RefineDet [48]Casc.-RCNN
[49]R-FCN [50]RCNN [51]Fast-RCNN [13]Faster-RCNN [14]Mask-RCNN [15]DETR
[52]RT-DETR [24]ViTDet [53] Figure 1: Classification of object detection
methods based on: (1) their architecture (One-stage, two-stages and
transformer network); (2) on detection accuracy (in orange) and real-time
detection (blue). The red dot highlights the models that are implemented,
trained and validated in this work. The green outline (indicated as
‘Overlapping’ in the image) groups the models that usually perform well in
both accuracy and inference time response.

The choice between object detection architecture depends on the application
and the nature of images to process which involves a trade-off between speed
and accuracy [54]. One-stage models tend to offer faster processing speeds but
may exhibit lower accuracy compared to their two-stage counterparts. The
advantages of one-stage algorithms could be summarized as follow [55, 56, 57]:

  * •

Simplicity and Efficiency: One-stage detectors have a simpler architecture
compared to two-stage algorithms. They directly predict object locations and
class probabilities without the need for an intermediate proposal generation
step. This simplicity leads to computational efficiency, as one-stage
detectors can process images faster than two-stage detectors.

  * •

Real-time Performance: One-stage detectors are designed to achieve real-time
or near real-time performance, making them suitable for applications where
fast inference is crucial. These algorithms are commonly used in scenarios
that require quick responses, such as autonomous driving, video analysis, and
robotics [58].

  * •

Higher Recall: One-stage detectors tend to have a higher recall rate compared
to two-stage detectors [59, 11]. They are capable of detecting a larger number
of objects in an image, including small or occluded objects, due to their
dense and dense-like prediction strategies. This higher recall can be
advantageous in applications where comprehensive object detection is more
important than achieving extremely precise bounding box localization.

  * •

Training Simplicity: One-stage detectors have a simpler training pipeline
compared to two-stage detectors. They typically use a single-shot training
strategy, where object locations and class predictions are directly regressed
from the network output. This simplifies the training process, reduces the
number of hyperparameters to tune, and requires fewer computational resources
for training. Consequently, one-stage detectors are easier to implement and
experiment with, especially for researchers or practitioners new to object
detection algorithms.

On the other hand, two-stage models generally achieve higher accuracy but
sacrifice some speed due to the additional processing stage. The advantages of
two-stage algorithms over the others could be summarized into the following
points:

  * •

Sampling efficiency: Two-stage detectors employ a sampling mechanism to select
a sparse set of region proposals, effectively eliminating a significant
portion of the negative proposals. Conversely, one-stage detectors take a
different approach by directly considering all regions in the image which
sometimes introduce class imbalance [60].

  * •

Feature extraction: Two-stage detectors can allocate a larger head network for
proposal classification and regression as they only process a small number of
proposals. This allows for the extraction of richer features, contributing to
improved performance.

  * •

Recall: Two-stage detectors leverage the RoIAlign [15, 61] operation to
extract high-quality features from each proposal, ensuring location
consistency. Conversely, in one-stage detectors, different region proposals
may share the same feature, leading to coarse and spatially implicit
representations that can cause feature misalignment.

  * •

Accuracy: Two-stage detectors refine the object locations twice, once in each
stage. Consequently, the bounding boxes generated by these models exhibit
better accuracy compared to one-stage methods but at the expense of real time
performance.

Consequently, the trade-off between one-stage, two-stage and transformer-based
architectures necessitates careful consideration based on the specific
requirements of the application at hand as well as the nature of images. In
this context, special attention is directed towards the detection of airplanes
within remote sensing images. The choice of algorithms is guided by their
widespread usage, popularity in performances when tested on other types of
images, and availability as open-source implementations. This will include
YOLO, CenterNet, RTMDet, SSD, RetinaNet, Faster-RCNN and DETR.

###  4.1 You Only Look Once

The You Only Look Once (YOLO) framework [59] has emerged as a popular deep
learning-based object detection algorithm that revolutionized real-time object
detection tasks. It presents a unified approach to object detection by
formulating it as a regression problem, enabling the model to predict bounding
boxes and class probabilities of multiple objects in a single pass through the
network.

Figure 2: Basic YOLO architecture. Reproduced from [62].

In the early stages, YOLOv2 [63] introduced batch normalization and high-
resolution classifiers to improve detection performance. YOLOv3 [33] further
refined the architecture by incorporating skip connections and multi-scale
prediction, enhancing both accuracy and localization capabilities. The
introduction of CSPDarknet, SPP, PAN, and Mish activation in YOLOv4 [35] led
to significant improvements in the network architecture. YOLOv5 [64], building
upon the scaled-YOLOv4 [65], introduced anchor-free object detection and a new
architecture. This version expanded the options available by providing models
of varying sizes, allowing users to balance speed and accuracy according to
their specific requirements (Fig. 2).

The latest known versions, YOLOv8 [22] and YOLO-NAS [66], brought forth the
concept of neural architecture search (NAS) [67] to automatically design
network architectures and achieve state-of-the-art performance in object
detection tasks. The continuous evolution of the YOLO framework highlights the
trade-offs between speed and accuracy, necessitating consideration of the
specific application requirements when selecting an appropriate YOLO model.

###  4.2 Single Shot Detection

The Single Shot Detector (SSD) [68] is an efficient and accurate object
detection algorithm that introduces a unified framework for single-pass
detection. In theory, SSD addresses the challenge of detecting objects at
various scales and aspect ratios by utilizing predefined anchor boxes [69].
The SSD architecture consists of three main components: a base network,
convolutional feature maps, and convolutional predictors (Fig. 3). The base
network, usually a pre-trained CNN [70], acts as a feature extractor,
generating high-level feature maps with different spatial resolutions. These
feature maps are then processed by convolutional predictors, which predict
object presence and locations for the anchor boxes. Each predictor corresponds
to a specific feature map and produces class scores and bounding box offsets.
The anchor boxes, distributed densely across the feature map, serve as
reference boxes for object detection. To capture objects at multiple scales,
SSD employs feature maps from different stages of the base network. Higher-
resolution feature maps are effective at detecting small objects, while lower-
resolution ones are suitable for larger objects. The predictions from each
feature map are combined to generate final class predictions and refined
bounding box coordinates. During training, SSD utilizes a multi-task loss
function [71] that optimizes the model by considering both localization loss
(Smooth L1 loss) and classification loss (softmax loss) [72]. The localization
loss penalizes the discrepancy between predicted and ground truth bounding box
coordinates, while the classification loss encourages accurate class
predictions. The loss is computed for positive and negative samples, including
hard negatives based on confidence scores.

Figure 3: SSD architecture diagram. Reproduced from [68].

While SSD offers a good balance between accuracy and efficiency with its
varying feature map resolutions and fixed anchor boxes, it does have
limitations [73]. One limitation is the dependence on predefined anchor boxes,
which may not adequately cover objects with extreme aspect ratios or
unconventional shapes. Scaling the anchor boxes can help address this, but it
introduces additional computational overhead. Another limitation is the
challenge of handling tiny objects [74]. Since SSD relies on a limited number
of feature maps, it may struggle to accurately detect small objects due to the
loss of fine-grained details at higher resolutions. This can result in reduced
localization accuracy and increased false negatives for small or densely
packed objects. Furthermore, SSD’s fixed anchor boxes limit its ability to
handle objects at arbitrary scales and aspect ratios.

###  4.3 Region-based CNNs

The introduction of the Region-based Convolutional Neural Network (R-CNN) [75]
marked a significant milestone in the development of object detection
techniques, showcasing the substantial improvements that convolutional neural
networks (CNNs) can bring to detection performance. R-CNN introduced the
concept of utilizing CNNs [76] in combination with a class-agnostic region
proposal [77] module to transform object detection into a classification and
localization problem. The detection process starts with a mean-subtracted
input image, which is fed through the region proposal (RPN) module. This
module employs techniques such as Selective Search [78] to identify regions
within the image that have a higher likelihood of containing objects.
Approximately 2000 object candidates are generated based on this region
proposal step. These candidates are then warped and passed through a CNN
network, such as the widely used ImageNet [79], to extract a 4096-dimensional
feature vector for each proposal. The feature vectors obtained from the CNN
are then inputted into class-specific Support Vector Machines (SVMs) [80],
which have been trained beforehand. The SVMs generate confidence scores for
each candidate region, aiding in the classification process. To refine the
results, non-maximum suppression (NMS) is applied based on the Intersection
over Union (IoU) and class information. Once the class has been identified, a
trained bounding-box regressor is employed to predict the precise bounding box
coordinates, including the center coordinates, width, and height of the object
(Fig. 4).

Figure 4: Faster RCNN architecture. Reproduced from [14].

However, despite its groundbreaking contributions, R-CNN has several
limitations [81, 82]. The training process of R-CNN is complex and multistage.
It involves pretraining the CNN with a large classification dataset, followed
by fine-tuning on domain-specific images that undergo mean subtraction and
warping to align with the proposals. The CNN classification layer is replaced
with a randomly initialized N+1-way classifier, where N represents the number
of classes, and stochastic gradient descent (SGD) is utilized for
optimization. Additionally, separate SVMs and bounding box regressors need to
be trained for each class, adding to the computational complexity.

Although R-CNN is capable of performing highly accurate object detection
research, it suffered from slow inference times, taking approximately 47
seconds per image, and was resource intensive in terms of both time and space
[75]. Training R-CNN models on small datasets took days to complete, even with
shared computations. These limitations sparked the need for further
advancements in object detection algorithms that could address these
challenges and improve overall efficiency.

###  4.4 RetinaNet Framework

RetinaNet is a one-stage object detection model that was introduced by Tsung-
Yi Lin et. al [26] designed to address the extreme foreground-background class
imbalance encountered during training of dense detectors. In object detection,
the goal is to detect objects of interest in an image and localize them by
drawing bounding boxes around them. However, the vast majority of the image is
typically background, and there are usually many more negative examples
(background) than positive examples (objects of interest). This class
imbalance can make it difficult for the detector to learn to distinguish
between objects and background, and can lead to poor performance. RetinaNet is
a single, unified network that is composed of a backbone network and two task-
specific subnetworks (Fig. 5). The backbone is responsible for computing a
convolutional feature map over an entire input image and is an off-the-shelf
convolutional network. The first subnetwork is a dense prediction subnet that
produces a fixed number of object detections of different scales and aspect
ratios at each spatial location in a feature map. The second subnetwork is a
set of class-specific subnets that further refine the predictions of the first
subnet.

Figure 5: One stage RetinaNet architecture. Reproduced from [17]. (a) ResNet
[83] backbone. (b) generation of multiscale convolutional pyramid. This is
attached to two subnetworks: (c) anchor box classification and (d) anchor box
regression to ground truth bounding box.

This model uses a novel focal loss function that down-weights the contribution
of easy examples during training to focus on hard examples and prevent the
vast number of easy background examples from overwhelming the detector. The
focal loss is a dynamically scaled cross-entropy loss, where the scaling
factor decays to zero as confidence in the correct class increases. This loss
function is designed to address the mechanisms used by other detectors to
address class imbalance, such as biased minibatch sampling and object proposal
mechanisms, in a one-stage detection system directly via the loss function.

The obtained results showed a good performance compared to previous one-stage
and two-stage detectors, including the best reported Faster R-CNN system, on
the COCO dataset. It achieves state-of-the-art performance on both the COCO
detection tasks, with a better COCO test-dev average precision while running
at 5 fps. This is a significant improvement over previous state-of-the-art
techniques for training one-stage detectors, such as training with the
sampling heuristics or hard example mining. RetinaNet is also designed to be
efficient and scalable. It uses an efficient in-network feature pyramid that
allows it to detect objects at multiple scales and resolutions, and it uses
anchor boxes to improve localization accuracy. The anchor boxes are pre-
defined boxes of different scales and aspect ratios that are placed at each
spatial location in the feature map. The dense prediction subnet predicts the
offsets and scales of the anchor boxes to generate object detections.

RetinaNet has been widely adopted in industry and academia and selected for a
variety of applications, including object detection in autonomous driving,
face detection, and medical image analysis. It has also inspired further
research in the field of object detection, including the development of other
novel loss functions and architectures.

###  4.5 CenterNet Framework

CenterNet [18] is another real-time object detection algorithm designed to
operate in real-time, with an average inference time of 270 ms using a
52-layer hourglass backbone and 340 ms using a 104-layer hourglass backbone
per image according to the author (Fig. 6). CenterNet is inspired from the
architecture of CornerNet [25] which is based on a one-stage keypoint-based
detector, while introducing several novel components and strategies to enhance
its effectiveness.

Figure 6: One stage CenterNet architecture. Reproduced from [18].

Unlike CornerNet, which detects object bounding boxes using pairs of
keypoints, CenterNet introduces the concept of detecting each object as a
triplet of keypoints. This innovation allows CenterNet to capture more
comprehensive information about the objects, leading to improved detection
performance while being fast for live inferencing. CenterNet incorporates two
customized modules named cascade corner pooling and center pooling. These
modules play crucial roles in enriching information collected by both top-left
and bottom-right corners and providing more recognizable information at the
central regions, respectively. These modules contribute to the improved
performance of CenterNet by enhancing the detection of object keypoints and
bounding boxes. In their benchmark study [18], authors stated that this
proposed architecture achieves significant improvements over existing one-
stage detectors typically an average precision of 47.0% on the MS-COCO
dataset, outperforming the previously proposed one-stage detectors by at least
4.9%. Additionally, it demonstrates comparable performance to the two-stage
detectors while maintaining a faster inference speed thanks to the effective
reduction of incorrect bounding boxes, particularly for small objects. It
achieves notable improvements in the detection of small objects, with an
average precision (AP) improvements of 5.5% to 8.1% for different backbone
configurations. This reduction in incorrect bounding boxes is attributed to
the effectiveness of CenterNet in modeling center information using center
keypoints.

###  4.6 End-to-End Transformer

The advent of transformer architectures has revolutionized the field of
artificial intelligence by introducing a novel approach to processing
sequential data. Unlike older neural network architectures, transformers rely
on self-attention mechanisms [44] to capture dependencies between input
tokens, enabling them to effectively model long-range dependencies and capture
complex patterns in the data. This ability to process sequences in parallel,
rather than sequentially, has significantly improved the efficiency and
effectiveness specially of natural language processing (NLP) tasks [84]. The
success of transformers can be attributed to their ability to capture global
context and relationships within the input data, making them particularly
well-suited for tasks that require understanding of complex interdependencies.

Later-on, researchers have translated the use of transformers in NLP to
computer vision including object detection operation. One pioneering algorithm
is DETR (DEtection TRansformer) [52] belonging to the category of end-to-end
object detection systems based on transformers and bipartite matching loss for
direct set prediction. Unlike traditional object detection methods, DETR does
not fall into the conventional one-stage or two-stages detector categories.
Instead, it introduces a novel approach by predicting all objects at once
using a bipartite matching loss function, which uniquely assigns a prediction
to a ground truth object and is invariant to a permutation of predicted
objects. This unique approach simplifies the detection pipeline by eliminating
the need for hand-designed components such as spatial anchors or non-maximal
suppression [85], making it optimal for both accuracy and real-time
processing. DETR achieves this by leveraging transformers with parallel
decoding, as opposed to autoregressive decoding with recurrent neural
networks, which was the focus of previous work.

Figure 7: DETR architecture. Reproduced from [52].

The architecture of DETR is simple yet highly effective. It comprises three
main components: a CNN backbone for feature extraction [86], an encoder-
decoder transformer [87] for modeling relationships between feature
representations of different detections, and a simple feed-forward network for
making the final detection predictions (Fig. 7). The process begins with the
extraction of a lower-resolution activation map from the input image using a
conventional CNN backbone. This activation map is then passed through the
transformer encoder, along with spatial positional encodings that are added to
queries and keys at every multi-head self-attention layer. The decoder
receives queries, output positional encodings (object queries), and encoder
memory, and produces the final set of predicted class labels and bounding
boxes through multiple multi-head self-attention and decoder-encoder attention
(FFNs). The simplicity and modularity of the DETR architecture make it easily
implementable in any deep learning framework that provides a common CNN
backbone and a transformer architecture implementation.

In comparison to existing models, DETR has demonstrated remarkable performance
on the challenging COCO object detection dataset. It achieves competitive
results with the same number of parameters as Faster R-CNN, a widely used
object detection model, achieving 42 AP on the COCO validation subset.
Notably, DETR outperforms Faster R-CNN in terms of AP improvement,
particularly in the context of direct set prediction. However, it lags behind
in terms of small object AP. Additionally, DETR with a ResNet-101 [88]
backbone shows comparable results to Faster R-CNN. The success of DETR can be
attributed to its unique combination of bipartite matching loss [89] and
transformers with parallel decoding, which enables it to effectively model
relations between feature representations of different detections and achieve
competitive performance in object detection tasks.

###  4.7 RTMDet

RTMDet [23] is a groundbreaking real-time object detection model designed to
achieve optimal efficiency without compromising accuracy. It belongs to the
family of fully convolutional single-stage detectors [90] such as YOLO series.
RTMDet [23] operates as a one-stage detector, enabling it to swiftly recognize
and localize objects in real-world scenarios such as autonomous driving,
robotics, and drones. The model is engineered to push the boundaries of the
YOLO series by introducing a new family of Real-Time Models for object
Detection. Notably, RTMDet is capable of performing instance segmentation and
rotated object detection, features that were previously unexplored.

Figure 8: One stage RTMDet architecture. Reproduced from [23].

RTMDet operates on a model architecture that emphasizes efficiency and
compatibility in both the backbone and neck. The basic building block of the
model consists of large-kernel depth-wise convolutions [91], which contribute
to its ability to capture global context effectively. This architectural
choice enhances the model’s capacity to recognize and localize objects with
high precision. Furthermore, RTMDet incorporates soft labels during the
calculation of matching costs in the dynamic label assignment process [92],
leading to an improved accuracy. The combination of these architectural
features and training techniques culminates in an object detector that
achieves exceptional performance. RTMDet’s macro architecture follows the one-
stage object detector paradigm, and it balances model depth, width, and
resolution to optimize efficiency. Additionally, the model is designed to be
versatile, allowing for easy extension to instance segmentation and rotated
object detection tasks with minimal modifications.

In comparison to the state-of-the-art industrial detectors, the authors [23]
demonstrate remarkable performance of RTMDet in terms of both speed and
accuracy. The model achieves an impressive 52.8% AP on the COCO dataset while
operating at over 300 frames per second (FPS) on an NVIDIA 3090 GPU. This
outperforms the current mainstream industrial detectors, showcasing the
superior parameter-accuracy trade-off of RTMDet. The model’s versatility is
evident in its ability to deliver optimal performance across various
application scenarios, offering different model sizes for different object
recognition tasks.

##  5 Method: Aircraft Detection

Typically, novel methods are assessed, validated, and contrasted with other
algorithms to demonstrate their effectiveness. However, these methods are
primarily evaluated on the COCO dataset, which is a comprehensive yet highly
effective performance evaluator due to the number of images with multiple
classes and resolutions. In this research, the main focus is evaluating these
algorithms for aircraft detection from remote sensing, where the challenges
are significantly different, and other types of noise and object sizes must be
dealt with.

For the comparative study, the google earth HRPlanesV2 dataset was selected as
a training dataset because it contains the highest number of high resolution
images (2120) among the other datasets (Table 2) while also providing
different positions orientations and ground cases to the model for its
generalization. As for the validation and test, unseen image from the
HRPlanesV2 as well as the the GDIT Aerial airports dataset was included to
evaluate with more accuracy the pre-trained models (Graphical summary of the
work in the Fig. 9). The frameworks used in this comparative study are the one
implemented by Ultralytics [10], Detectron2 [93] and MMLab detection toolbox
[90].

Figure 9: Flowchart of the FlightScope comparative study: The training is
performed on HRPlanesV2 dataset and the Validation and Test conducted on
HRPlanesV2 and GDIT Aerial airport datasets.

###  5.1 Setup and Data Preparation

Google earth HRPlanesV2 is already annotated and splited into three subsets:
70% for training, 20% for validation, and 10% for testing but initially
available in YOLO format while MMLab requires the dataset annotation format to
be in COCO. Fig. 10 provide an sample from HRPlanesV2 dataset with the
corresponding annotation bounding boxes.

Figure 10: Sample preview from HRPlanesV2 dataset: (a) blue: training subset,
(b) green: test subset, (c) red: validation subset

An overview of the setup configuration of the used training package is
presented in Table 3. For the training and the evaluation of the model 3
NVIDIA RTX A6000 each with 48G of memory have been used. These processors were
made available by the TOELT LLC AI Lab. The maximum number of epochs have been
fixed to 500 and batch sizes to 32 and for some neural network architectures
to 64 because the GPU memory allowed it.

Table 3: System Configuration Setup Software Setup
---
Name | Version
Ubuntu | 20.04.1 LTS
Python | 3.8
PyTorch | 2.0.1
CUDA | 12.2
Hardware Setup
GPU | NVIDIA RTX A6000 48 GB $\times$ 3
CPU | Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz
Memory | 128 GB

###  5.2 Evaluation Metrics

Before delving into the specific evaluation metrics employed, it’s essential
to establish a comprehensive understanding of each metric’s role in assessing
the performance of object detection models, particularly concerning bounding
box estimation accuracy. Within this study, PASCAL VOC metrics have been
selected: AP, Recall and bounding boxes IoU [94].

Average Precision (AP) stands as a fundamental metric in object detection
evaluation, providing a comprehensive measure of the model’s ability to
precisely identify objects of interest within an image. The AP metric,
formulated in Eq. 1, is computed by integrating the precision-recall curve
$p(r)$, which represents the trade-off between true positive detections and
false positives across various confidence thresholds. This integration yields
a scalar value reflecting the model’s overall detection accuracy, with higher
AP scores indicating superior performance.

$\text{AP}=\int_{0}^{1}p(r)\,dr$ (1)

Recall, another metric in object detection assessment, quantifies the model’s
ability to correctly identify all relevant instances of objects present within
an image. It signifies the sensitivity of the model in capturing true
positives (TP) while minimizing false negatives (FN). Mathematically, Recall
is defined as the ratio of true positive detections to the total number of
ground truth objects (Eq. 2).

$\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (2)

Lastly, Intersection over Union (IoU) serves as a metric for evaluating the
spatial alignment between predicted bounding boxes and ground truth
annotations. IoU quantifies the extent of overlap between these bounding
boxes, providing insight into the localization accuracy of detected objects.
IoU, expressed by Eq. 3, is computed as the ratio of the intersection area
between the predicted and ground-truth bounding boxes to their union.

$\text{IoU}=\frac{\text{Area of Intersection}}{\text{Area of Union}}$ (3)

Employing these metrics in the evaluation process enables an accurate
assessment of object detection model performance, particularly concerning
bounding-box estimation accuracy.

###  5.3 Results

This subsection presents a detailed result showcase of the object detection
algorithms training during the 500 epochs. Fig. 11 display the mAP (overall
mean avec precision accros different confidence thresholds) and the mAP50 (for
object detected by an Intersection over Union (IoU) threshold of 50% and up)
curves collectively visualizing the performance of the object detection
algorithms in the task of aircraft detection from remote sensing imagery.
Notably, the performances of the algorithms are quite comparable with an
overall mAP varying between 0.86 and 0.99. Among the models, YOLOv5 emerges as
a standout performer, achieving the highest mAP value of 0.99471 at step 150,
showcasing its great precision and robustness. YOLOv8 closely follows,
reaching a peak mAP value of 0.99236 at step 395, emphasizing the efficacy of
the YOLO architecture in aerial object detection. However, SSD lags behind
with a comparatively modest mAP value of 0.86 at step 74 which stabilizes in
the rest of the training process.

Figure 11: Comparison of bounding box mean average precision (mAP) curves for
trained object detection algorithms. To the left raw figures of the curves,
the right figures are magnifications from epoch 450 to 500. (a) Represents the
mAP. (b) Illustrates the mAP50

The results in Fig. 11-b confirm the previous discussion, as YOLOv5 continues
to outperform, achieving the highest mAP50 value of 0.84454 at step 493.
RTMDet and YOLOv8 remains strong with mAP50 values of 0.838 at step 340 and
0.8372 at step 492 successively. CenterNet consistently performs well,
achieving an mAP50 value of 0.826 at step 439. Faster-RCNN maintains a
balanced mAP50 value of 0.775 at step 402, showcasing reliability in
detection. DETR contributes robustly with an mAP50 value of 0.774 at step 472,
while RetinaNet exhibits stability with an mAP50 value of 0.765 at step 250.
Finally, SSD, with a noticeable gap between mAP and mAP50 values, suggests
potential challenges in localization precision, emphasizing the need for
refinement in specific scenarios.

(a) Bounding box loss

(b) Total loss

Figure 12: Comparison of loss curves for the 8 trained object detection
models: (a) Bounding box loss curves (b) Total loss curves.

In addition, Fig. 12(a) and 12(b) provide successively a visualization of the
bounding box and total loss curves for the algorithms, where it is noticeable
that the majority of the algorithm converge quickly around 100 epochs (apart
from RTMDet which has a jump value around the epoch 280) and reach a seemingly
horizontal asymptote line within 300 epochs. While extending the training
duration beyond 500 epochs might promote further convergence, there is a
potential risk of overfitting the models.

###  5.4 Evaluation on GDIT Dataset

To evaluate the efficacy of object detection models, originally trained on the
HRPlanesv2 dataset, a thorough assessment was conducted using the GDIT
dataset. The evaluation encompassed all subsets of images: train, test, and
validation, each representing diverse scenarios. This comprehensive evaluation
aimed to gauge the adaptability and robustness of the algorithms under varied
conditions encountered in different subsets. A sample of unseen images from
both datasets highlighting the bounding boxes of estimated aircrafts is
presented in Fig. 13 and Fig. 14 where the ND, FP and IE stands respectively
for ’Not Detected’, ’False Positives’ and ’Inaccurate Estimation’. The figures
shows the struggle of CenterNet, Faster RCNN and SSD in the detection of small
object is observable while both YOLO versions and RTMDet are able to detect
over 80% of the aircraft in the image with a minimum confidence of 32.6% with
small amount of FP and/or ND.

Figure 13: Inference examples of YOLOv8, DETR, SSD and CenterNet on unseen
images from Google Earth HRPlanesv2 dataset and Airbus GDIT. ‘FP’ stands for
False Positives, ‘ND’ for No Detection and ‘IE’ for Inaccurate Estimation.
Figure 14: Inference examples of YOLOv5, RTMDet, RetinaNet and Faster-RCNN on
other set of unseen images from Google Earth HRPlanesv2 dataset and Airbus
GDIT. ‘FP’ stands for False Positives, ‘ND’ for No Detection.

The results of the metrics IoU, Recall and AP are presented in the histograms
(Fig. 15). On the train subset, YOLOv8 demonstrated notable performance with
an AP of 91.9%, Recall of 68.6%, and IoU of 71.1%. YOLOv5 exhibited excellence
with an AP of 96.8%, Recall of 66.1%, and IoU of 74.0%. Conversely, SSD
displayed a comparatively lower AP of 59.4%, Recall of 38.6%, and IoU of
49.2%. Performance metrics such as AP, Recall, and IoU showed variations for
Faster RCNN and CenterNet.

The evaluation on the Test subset provided further insights into the
generalization capabilities of the pre-trained algorithms. YOLOv8 maintained
high performance with an AP of 90.3%, Recall of 70.9%, and IoU of 70.4%.
Similarly, YOLOv5 exhibited commendable performance with an AP of 95.6%,
Recall of 70.5%, and IoU of 75.2%. However, SSD, Faster RCNN, and CenterNet
displayed variations in performance metrics across the Test subset.

Finally, in the Validation subset, distinct challenges were encountered,
revealing the algorithms’ robustness in diverse scenarios. YOLOv8 achieved an
AP of 90.0%, Recall of 78.1%, and IoU of 69.6%. YOLOv5 maintained high
standards with an AP of 94.2%, Recall of 77.0%, and IoU of 74.2%. Performance
nuances were observed for SSD, Faster RCNN, and CenterNet, underscoring their
adaptability to distinct datasets.

Figure 15: Estimated evaluation metrics when inferencing the 8 models (initially trained on HRPlanesV2) on all images from unseen subsets ’Train’ (a), ’Test’ (b) and ’Validation’ (c) from GDIT aircraft dataset. Table 4: Object Detection Performance on Remote Sensing Images. Tr. subset refers to all images from the ’train’, Te. subset refers to all images from the ’test’, Val. subset refers to all images from the ’validation’ and All dataset overall images in the dataset. Architecture | Metric | Tr. subset | Te. subset | Val. subset | All dataset
---|---|---|---|---|---
YOLOv8 | AP | 0.919 | 0.903 | 0.900 | 0.907
| Recall | 0.686 | 0.710 | 0.781 | 0.726
| IoU | 0.711 | 0.704 | 0.696 | 0.704
YOLOv5 | AP | 0.968 | 0.956 | 0.942 | 0.955
| Recall | 0.661 | 0.705 | 0.770 | 0.712
| IoU | 0.740 | 0.752 | 0.742 | 0.745
SSD | AP | 0.594 | 0.653 | 0.638 | 0.628
| Recall | 0.386 | 0.432 | 0.496 | 0.438
| IoU | 0.492 | 0.517 | 0.516 | 0.508
Faster RCNN | AP | 0.573 | 0.714 | 0.726 | 0.671
| Recall | 0.422 | 0.413 | 0.491 | 0.442
| IoU | 0.473 | 0.564 | 0.564 | 0.533
RetinaNet | AP | 0.819 | 0.830 | 0.815 | 0.821
| Recall | 0.566 | 0.620 | 0.644 | 0.610
| IoU | 0.647 | 0.652 | 0.641 | 0.647
CenterNet | AP | 0.566 | 0.742 | 0.758 | 0.689
| Recall | 0.434 | 0.441 | 0.536 | 0.470
| IoU | 0.473 | 0.593 | 0.603 | 0.556
DETR | AP | 0.705 | 0.725 | 0.718 | 0.716
| Recall | 0.610 | 0.636 | 0.680 | 0.642
| IoU | 0.553 | 0.576 | 0.558 | 0.563
RTMDet | AP | 0.863 | 0.850 | 0.836 | 0.850
| Recall | 0.670 | 0.685 | 0.718 | 0.691
| IoU | 0.672 | 0.676 | 0.664 | 0.670

Table 4 summarizes the object detection performance metrics of various models
on remote sensing images across different subsets—Train, Test, Validation and
all the dataset. The overall performances showed the YOLOv5 emerged as the
top-performing algorithm across all subsets, with the highest evaluation
metrics AP and IoU. Notably, YOLOv5 demonstrated commendable recall rates and
IoU scores, positioning it as the leading algorithm for aircraft detection in
this study. Conversely, SSD consistently exhibited comparatively lower
performance metrics, indicating challenges in accurately detecting aircraft
instances across subsets. While other algorithms, including YOLOv8, Faster
RCNN, and CenterNet, displayed varying degrees of success, YOLOv5 consistently
outperformed them in terms of AP, Recall, and IoU.

##  6 Conclusion

This study presents a comprehensive evaluation of aircraft detection
algorithms in satellite imagery, namely YOLO (v5 and v8), Faster RCNN,
CenterNet, RetinaNet, RTMDet and DETR, specifically focusing on the HRPlanesV2
dataset and extending the assessment to subsets of the GDIT dataset: train,
test, and validation. The used training setup consists of training the
aformentioned object detection algorithms on the dataset for 500 epochs on a 3
NVIDIA RTX A6000 GPUs configuration, each GPU has 48 GB of memory. The results
of the evaluation demonstrate the adaptability and robustness of the trained
object detection algorithms. Among these algorithms, YOLOv5 emerges as the
standout performer, achieving the highest mean average precision (mAP) of
0.99, highlighting its precision and robustness. YOLOv8 closely follows,
further emphasizing the effectiveness of the YOLO architecture in aerial
object detection. On the other side, the SSD displayed the lowest performances
in both the training and evaluation. Furthermore, the evaluation extends to
the GDIT dataset, providing a more comprehensive assessment by deploying the
trained network on other scenarios, including different satellite imagery
sources. By employing evaluation metrics such as Average Precision, Recall,
and Intersection over Union, YOLOv5 still consistently outperforms the other
algorithms, demonstrating superior performance across all subsets. This
solidifies YOLOv5’s position as the top-performing algorithm for aircraft
detection, characterized by commendable AP, Recall, and IoU scores.

The topic of this research offers a extensive overview and comparative study,
providing an in-depth analysis of the performance, accuracy, and computational
demands of object detection method. The insights gained from this study
significantly enhance the decision-making process for selecting the most
effective aircraft localization techniques in satellite imagery, supported by
detailed training and validation processes using the HRPlanesv2 dataset and
additional validation with the GDIT dataset.

## References

  * [1] Akash Ashapure, Jinha Jung, Anjin Chang, Sungchan Oh, Murilo Maeda, and Juan Landivar.  A comparative study of rgb and multispectral sensor-based cotton canopy cover modelling using multi-temporal uas data.  Remote Sensing, 11(23):2757, 2019.
  * [2] Shenshen Fu, Yifan He, Xiaofeng Du, and Yi Zhu.  Anchor-free object detection in remote sensing images using a variable receptive field network.  EURASIP Journal on Advances in Signal Processing, 2023(1):53, 2023.
  * [3] Sushobhan Majumdar.  The Role of Remote Sensing and GIS in Military Strategy to Prevent Terror Attacks, pages 79–94.  2021\.
  * [4] Alberta Bianchin and Laura Bravin.  Remote sensing and urban analysis.  In Osvaldo Gervasi, Beniamino Murgante, Antonio Laganà, David Taniar, Youngsong Mun, and Marina L. Gavrilova, editors, Computational Science and Its Applications – ICCSA 2008, pages 300–315, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg.
  * [5] Harm Greidanus.  Satellite imaging for maritime surveillance of the european seas.  In Remote Sensing of the European Seas, pages 343–358. Springer, 2008.
  * [6] X. Liu, Y. Zhang, and Z. Wang.  Aircraft detection in high-resolution satellite images using deep learning.  International Journal of Remote Sensing, 43(10):4349–4367, 2022.
  * [7] Z. Pan, X. Wu, J. Ma, B. Hu, Y. Wang, and L. Zhao.  Sar-dad: A novel small target detection algorithm for spaceborne sar imagery.  Complexity, 2022:2262549, 2022.
  * [8] Ivan Prudyus, Victor Tkachenko, Petro Kondratov, Serhiy Fabirovskyy, Leonid Lazko, and Andriy Hryvachevskyi.  Factors affecting the quality of formation and resolution of images in remote sensing systems.  JCPEE, 5(1):41–46, 2015.
  * [9] Ugur Alganci, Mehmet Soydas, and Elif Sertel.  Comparative research on deep learning approaches for airplane detection from very high-resolution satellite images.  Remote Sensing, 12(3), 2020.
  * [10] Glenn Jocher, Ayush Chaurasia, and Jing Qiu.  Ultralytics yolov8, 2023.
  * [11] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg.  SSD: Single Shot MultiBox Detector.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, Lecture Notes in Computer Science, pages 21–37, Cham, 2016. Springer International Publishing.
  * [12] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1):142–158, January 2016.  Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [13] Ross Girshick.  Fast R-CNN.  In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1440–1448, December 2015.  ISSN: 2380-7504.
  * [14] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks, 2016.
  * [15] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask R-CNN.  In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, October 2017.  ISSN: 2380-7504.
  * [16] Mingxing Tan, Ruoming Pang, and Quoc V. Le.  EfficientDet: Scalable and Efficient Object Detection.  pages 10778–10787, 2020.
  * [17] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal Loss for Dense Object Detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327, February 2020.  Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [18] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian.  CenterNet: Keypoint Triplets for Object Detection, April 2019.  arXiv:1904.08189 [cs].
  * [19] Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R Martin, Ming-Ming Cheng, and Shi-Min Hu.  Attention mechanisms in computer vision: A survey.  Computational visual media, 8(3):331–368, 2022.
  * [20] Xiaoxia Meng, Xiaowei Wang, Shoulin Yin, and Hang Li.  Few-shot image classification algorithm based on attention mechanism and weight fusion.  Journal of Engineering and Applied Science, 70(1):14, 2023.
  * [21] Wei Li, Kai Liu, Lizhe Zhang, and Fei Cheng.  Object detection based on an adaptive attention mechanism.  Scientific Reports, 10(1):11307, 2020.
  * [22] Glenn Jocher, Ayush Chaurasia, and Jing Qiu.  YOLOv8 by Ultralytics.  Software.
  * [23] Chengqi Lyu, Wenwei Zhang, Haian Huang, Yue Zhou, Yudong Wang, Yanyi Liu, Shilong Zhang, and Kai Chen.  Rtmdet: An empirical study of designing real-time object detectors, 2022.
  * [24] Wenyu Lv, Shangliang Xu, Yian Zhao, Guanzhong Wang, Jinman Wei, Cheng Cui, Yuning Du, Qingqing Dang, and Yi Liu.  Detrs beat yolos on real-time object detection.  arXiv preprint arXiv:2304.08069, 2023.
  * [25] Hei Law and Jia Deng.  Cornernet: Detecting objects as paired keypoints.  CoRR, abs/1808.01244, 2018.
  * [26] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327, 2020.
  * [27] Oran R. Young and Masami Onoda.  Satellite Earth Observations in Environmental Problem-Solving, pages 3–27.  Springer Singapore, Singapore, 2017.
  * [28] Daniel Wilson, Xiaohan Zhang, Waqas Sultani, and Safwan Wshah.  Image and object geo-localization.  International Journal of Computer Vision, pages 1–43, 2023.
  * [29] Omer Wosner, Guy Farjon, and Aharon Bar-Hillel.  Object detection in agricultural contexts: A multiple resolution benchmark and comparison to human.  Computers and Electronics in Agriculture, 189:106404, 2021.
  * [30] Hao Zhang and Jiayi Ma.  Stp-som: Scale-transfer learning for pansharpening via estimating spectral observation model.  International Journal of Computer Vision, 131(12):3226–3251, 2023.
  * [31] CS Ayush Kumar, Advaith Das Maharana, Srinath Murali Krishnan, Sannidhi Sri Sai Hanuma, V Sowmya, and Vinayakumar Ravi.  Vehicle detection from aerial imagery using principal component analysis and deep learning.  In International Conference on Innovations in Bio-Inspired Computing and Applications, pages 129–140. Springer, 2022.
  * [32] Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang.  Dota: A large-scale dataset for object detection in aerial images.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3974–3983, 2018.
  * [33] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement, 2018.  cite arxiv:1804.02767Comment: Tech Report.
  * [34] Tolga BAKIRMAN and Elif SERTEL.  A benchmark dataset for deep learning-based airplane detection: Hrplanes.  International Journal of Engineering and Geosciences, 8(3):212–223, 2023.
  * [35] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.  Yolov4: Optimal speed and accuracy of object detection.  arXiv preprint arXiv:2004.10934, 2020.
  * [36] Class.  Airbus aircraft detection dataset.  https://universe.roboflow.com/class-dvpyb/airbus-aircraft-detection, jan 2023.  visited on 2024-01-21.
  * [37] Jacob Shermeyer, Thomas Hossler, Adam Van Etten, Daniel Hogan, Ryan Lewis, and Daeil Kim.  Rareplanes: Synthetic data takes flight.  CoRR, abs/2006.02963, 2020.
  * [38] GDIT.  Aerial airport dataset.  https://universe.roboflow.com/gdit/aerial-airport, sep 2023.  visited on 2024-01-21.
  * [39] Robert Hammell.  Planes in satellite imagery, 2023.
  * [40] Mauricio Pamplona Segundo, Allan Pinto, Rodrigo Minetto, Ricardo da Silva Torres, and Sudeep Sarkar.  A dataset for detecting flying airplanes on satellite images, 2021.
  * [41] J Chen, H Li, G Zhang, et al.  Dataset of aircraft classification in remote sensing images.  J. Glob. Change Data Discov, 2:183–190, 2020.
  * [42] C. Bhagya and A. Shyna.  An overview of deep learning based object detection techniques.  In 2019 1st International Conference on Innovations in Information and Communication Technology (ICIICT), pages 1–6, 2019.
  * [43] Payal Mittal, Raman Singh, and Akashdeep Sharma.  Deep learning-based object detection in low-altitude uav datasets: A survey.  Image and Vision Computing, 104:104046, 2020.
  * [44] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [45] Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen.  Deep learning for generic object detection: A survey.  International journal of computer vision, 128:261–318, 2020.
  * [46] Xingxing Xie, Gong Cheng, Jiabao Wang, Ke Li, Xiwen Yao, and Junwei Han.  Oriented r-cnn and beyond.  International Journal of Computer Vision, pages 1–23, 2024.
  * [47] Hritik Vaidwan, Nikhil Seth, Anil Singh Parihar, and Kavinder Singh.  A study on transformer-based object detection.  In 2021 International Conference on Intelligent Technologies (CONIT), pages 1–6, 2021.
  * [48] Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, and Stan Z Li.  Single-shot refinement neural network for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4203–4212, 2018.
  * [49] Zhaowei Cai and Nuno Vasconcelos.  Cascade r-cnn: Delving into high quality object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6154–6162, 2018.
  * [50] Jifeng Dai, Yi Li, Kaiming He, and Jian Sun.  R-fcn: Object detection via region-based fully convolutional networks.  Advances in neural information processing systems, 29, 2016.
  * [51] Chenyi Chen, Ming-Yu Liu, Oncel Tuzel, and Jianxiong Xiao.  R-cnn for small object detection.  In Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part V 13, pages 214–230. Springer, 2017.
  * [52] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers, 2020.
  * [53] Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He.  Exploring plain vision transformer backbones for object detection.  In European Conference on Computer Vision, pages 280–296. Springer, 2022.
  * [54] Xin Lu, Quanquan Li, Buyu Li, and Junjie Yan.  MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection.  pages 541–557. November 2020.
  * [55] Jeremy Jordan.  An overview of object detection: One-stage methods.  https://www.jeremyjordan.me/object-detection-one-stage/, 2021.  Accessed on 8 June 2023.
  * [56] Hang Zhang and Rayan S Cloutier.  Review on one-stage object detection based on deep learning.  EAI Endorsed Transactions on e-Learning, 7(23), 6 2022.
  * [57] Licheng Jiao, Fan Zhang, Fang Liu, Shuyuan Yang, Lingling Li, Zhixi Feng, and Rong Qu.  A survey of deep learning-based object detection.  IEEE Access, 7:128837–128868, 2019.
  * [58] Aditya Lohia, Kalyani Kadam, Rahul Joshi, and Arunkumar Bongale.  Bibliometric analysis of one-stage and two-stage object detection.  02 2021.
  * [59] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You Only Look Once: Unified, Real-Time Object Detection.  pages 779–788, June 2016.  ISSN: 1063-6919.
  * [60] Xin Lu, Quanquan Li, Buyu Li, and Junjie Yan.  Mimicdet: Bridging the gap between one-stage and two-stage object detection.  CoRR, abs/2009.11528, 2020.
  * [61] Tao Gong, Kai Chen, Xinjiang Wang, Qi Chu, Feng Zhu, Dahua Lin, Nenghai Yu, and Huamin Feng.  Temporal roi align for video object recognition.  CoRR, abs/2109.03495, 2021.
  * [62] Tausif Diwan, G Anirudh, and Jitendra V Tembhurne.  Object detection using yolo: Challenges, architectural successors, datasets and applications.  multimedia Tools and Applications, 82(6):9243–9275, 2023.
  * [63] Joseph Redmon and Ali Farhadi.  Yolo9000: Better, faster, stronger.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6517–6525, 2017.
  * [64] Glenn Jocher, Ayush Chaurasia, Alex Stoken, Jirka Borovec, NanoCode012, Yonghye Kwon, Kalen Michael, TaoXie, Jiacong Fang, Imyhxy, ”Zeng Yifu” Lorna and, Colin Wong, Abhiram V, Diego Montes, Zhiqiang Wang, Cristi Fati, Jebastin Nadar, Laughing, UnglvKitDe, Victor Sonck, Tkianai, YxNONG, Piotr Skalski, Adam Hogan, Dhruv Nair, Max Strobel, and Mrinal Jain.  ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation.  Zenodo, November 2022.
  * [65] Chien-Yao Wang, Alexey Bochkovskiy, and Hong-yuan Liao.  Scaled-yolov4: Scaling cross stage partial network.  pages 13024–13033, 06 2021.
  * [66] Shay Aharon, Louis-Dupont, Ofri Masad, Kate Yurkova, Lotem Fridman, Lkdci, Eugene Khvedchenya, Ran Rubin, Natan Bagrov, Borys Tymchenko, Tomer Keren, Alexander Zhilko, and Eran-Deci.  Super-gradients, 2021.
  * [67] Krishna Teja Chitty-Venkata, Murali Emani, Venkatram Vishwanath, and Arun Somani.  Neural architecture search benchmarks: Insights and survey.  IEEE Access, PP:1–1, 01 2023.
  * [68] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg.  Ssd: Single shot multibox detector.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 21–37, Cham, 2016. Springer International Publishing.
  * [69] Sandro Augusto Magalhães, Luís Castro, Germano Moreira, Filipe Neves dos Santos, Mário Cunha, Jorge Dias, and António Paulo Moreira.  Evaluating the single-shot multibox detector and yolo deep learning models for the detection of tomatoes in a greenhouse.  Sensors, 21(10), 2021.
  * [70] Soulef Bouaafia, Seifeddine Messaoud, Amna Maraoui, Ahmed Chiheb Ammari, Lazhar Khriji, and Mohsen Machhout.  Deep pre-trained models for computer vision applications: Traffic sign recognition.  In 2021 18th International Multi-Conference on Systems, Signals & Devices (SSD), pages 23–28, 2021.
  * [71] Alex Kendall, Yarin Gal, and Roberto Cipolla.  Multi-task learning using uncertainty to weigh losses for scene geometry and semantics, 2018.
  * [72] Juan Terven, Diana M. Cordova-Esparza, Alfonso Ramirez-Pedraza, and Edgar A. Chavez-Urbiola.  Loss functions and metrics in deep learning, 2023.
  * [73] Jisoo Jeong, Hyojin Park, and Nojun Kwak.  Enhancement of SSD by concatenating feature maps for object detection.  CoRR, abs/1705.09587, 2017.
  * [74] Ashwani Kumar, Zuopeng Justin Zhang, and Hongbo Lyu.  Object detection in real time based on improved single shot multi-box detector algorithm.  EURASIP Journal on Wireless Communications and Networking, 2020(1):204, October 2020.
  * [75] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 11 2013.
  * [76] Rikiya Yamashita, Mizuho Nishio, Richard Do, and Kaori Togashi.  Convolutional neural networks: an overview and application in radiology.  Insights into Imaging, 9, 06 2018.
  * [77] Ayush Jaiswal, Yue Wu, Pradeep Natarajan, and P. Natarajan.  Class-agnostic object detection.  2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 918–927, 2020.
  * [78] Jasper Uijlings, K. Sande, T. Gevers, and A.W.M. Smeulders.  Selective search for object recognition.  International Journal of Computer Vision, 104:154–171, 09 2013.
  * [79] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  25, 2012.
  * [80] M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf.  Support vector machines.  IEEE Intelligent Systems and their Applications, 13(4):18–28, 1998.
  * [81] Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye.  Object detection in 20 years: A survey, 2023.
  * [82] Shahid Karim, Ye Zhang, Shoulin Yin, Irfana Bibi, and Ali Brohi.  A brief review and challenges of object detection in optical remote sensing imagery.  Multiagent and Grid Systems, 16:227–243, 10 2020.
  * [83] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [84] Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu.  A survey of transformers, 2021.
  * [85] Zhong-Qiu Zhao, Peng Zheng, Shou tao Xu, and Xindong Wu.  Object detection with deep learning: A review, 2019.
  * [86] Omar Elharrouss, Younes Akbari, Noor Almaadeed, and Somaya Al-Maadeed.  Backbones-review: Feature extraction networks for deep learning and deep reinforcement learning approaches, 2022.
  * [87] Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu.  A survey of transformers.  AI Open, 3:111–132, 2022.
  * [88] Syifa Auliyah Hasanah, Anindya Apriliyanti Pravitasari, Atje Setiawan Abdullah, Intan Nurma Yulita, and Mohammad Hamid Asnawi.  A deep learning review of resnet architecture for lung disease identification in cxr image.  Applied Sciences, 13(24), 2023.
  * [89] Wenchao Gu, Shuang Bai, and Lingxing Kong.  A review on 2d instance segmentation based on deep neural networks.  Image and Vision Computing, 120:104401, 2022.
  * [90] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin.  Mmdetection: Open mmlab detection toolbox and benchmark, 2019.
  * [91] Xiaohan Ding, Xiangyu Zhang, Yizhuang Zhou, Jungong Han, Guiguang Ding, and Jian Sun.  Scaling up your kernels to 31x31: Revisiting large kernel design in cnns, 2022.
  * [92] Tianxiao Zhang, Bo Luo, Ajay Sharda, and Guanghui Wang.  Dynamic label assignment for object detection by combining predicted ious and anchor ious.  Journal of Imaging, 8(7):193, July 2022.
  * [93] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick.  Detectron2.  https://github.com/facebookresearch/detectron2, 2019.
  * [94] Rafael Padilla, Wesley L Passos, Thadeu LB Dias, Sergio L Netto, and Eduardo AB Da Silva.  A comparative analysis of object detection metrics with a companion open-source toolkit.  Electronics, 10(3):279, 2021.