# Rethinking Intersection Over Union for Small Object Detection in
Few-Shot Regime

Pierre Le Jeune
L2TI, Université Sorbonne Paris Nord
COSE
<EMAIL_ADDRESS>Anissa Mokraoui
L2TI, Université Sorbonne Paris Nord
First line of institution2 address
<EMAIL_ADDRESS>
###### Abstract

In Few-Shot Object Detection (FSOD), detecting small objects is extremely
difficult. The limited supervision cripples the localization capabilities of
the models and a few pixels shift can dramatically reduce the Intersection
over Union (IoU) between the ground truth and predicted boxes for small
objects. To this end, we propose Scale-adaptive Intersection over Union
(SIoU), a novel box similarity measure. SIoU changes with the objects’ size,
it is more lenient with small object shifts. We conducted a user study and
SIoU better aligns than IoU with human judgment. Employing SIoU as an
evaluation criterion helps to build more user-oriented models. SIoU can also
be used as a loss function to prioritize small objects during training,
outperforming existing loss functions. SIoU improves small object detection in
the non-few-shot regime, but this setting is unrealistic in the industry as
annotated detection datasets are often too expensive to acquire. Hence, our
experiments mainly focus on the few-shot regime to demonstrate the superiority
and versatility of SIoU loss. SIoU improves significantly FSOD performance on
small objects in both natural (Pascal VOC and COCO datasets) and aerial images
(DOTA and DIOR). In aerial imagery, small objects are critical and SIoU loss
achieves new state-of-the-art FSOD on DOTA and DIOR.

##  1 Introduction

Object detection is a fundamental task in industry and has applications in
many domains such as medical imaging, agriculture, and autonomous driving.
However, it is often impracticable or too expensive to build sufficiently
large annotated datasets to train detection models. It is therefore crucial to
improve data-efficient approaches and particularly Few-Shot Object Detection
(FSOD) methods. However, the limited number of examples provides poor
supervision and prevents the model to learn accurate localization, which is
especially problematic for small objects. Besides, the difficulty of detecting
small objects was already reported in many object detectors [27, 26, 23, 34,
8, 18, 4]. Numerous attempts partially solved this issue by proposing various
improvements such as pyramidal features [23, 21, 46] or multiscale training
[30, 31]. However, this difficulty greatly intensifies in the few-shot regime
as shown by [24]. One of the reasons for the poor FSOD performance on small
objects is the extensive use of Intersection over Union (IoU). Most detection
(and so FSOD) pipelines employ IoU as a regression loss [43, 34]; for example
selection [27, 26, 23]; or an evaluation criterion, but IoU is not an optimal
choice when dealing with small objects.

Figure 1: (Left) Evolution of IoU, NWD [40], the proposed SIoU and
$\alpha$-IoU [12] when a box is shifted from the ground truth box by
$\varepsilon_{\text{loc}}$ pixels for various box sizes
$\omega\in\\{4,16,64,128\\}$. (Right) Ratio between pixel localization error
$\varepsilon_{\text{loc}}$ and object size $\omega$ for a trained detection
model on DOTA dataset. Each point represents the localization error of one
object in DOTA test set.

IoU has a remarkable property: scale invariance. It means that scaling all
coordinates of two bounding boxes by the same amount will not change their
IoU. At first glance, this seems a desirable property, all objects will be
treated identically no matter their size. In practice, it has a fundamental
drawback: small boxes are prone to large IoU changes from only small position
or size modifications. To clarify, let us consider a simple example. Two
square boxes of width $\omega$ are shifted diagonally by
$\varepsilon_{\text{loc}}$ pixels. In this setup, a 1-pixel shift leads to a
larger decrease in IoU when boxes are small. This comes from the scale
invariance property, IoU stays constant as the ratio
$\frac{\varepsilon_{\text{loc}}}{\omega}$ remains fixed. However, this ratio
is not constant for trained detection models, it increases as objects get
smaller (see Fig. 1 right), leading to lower IoU values for smaller objects.
Hence, small objects are much more likely to fall under the IoU thresholds
which decide if a box is a true or false detection, even though being
satisfactory from a human perspective (see the user study in Sec. 4.3). Secs.
4.1 and 4.2 explore the resilience of various criteria to localization
inaccuracies and confirm that IoU is not an optimal box similarity measure.

Only a handful of works question the adequation of IoU for object detection.
Among those, [28] proposed a generalization of IoU when boxes do not overlap,
[40] introduced a novel loss function to target small objects and [32] showed
that human perception and IoU are not fully aligned. This lack of interest in
new criterion design is explained by the great detection performance in the
regular setting (_i.e_. natural images with sufficient annotations). In the
few-shot regime, and when targets are small, the flaws of IoU become critical.
Therefore, we revisit IoU to improve FSOD methods and focus on aerial images
which mostly contain small objects. We propose Scale-adaptive Intersection
over Union (SIoU), a novel criterion that can replace IoU for training and
evaluating detection models. To demonstrate the superiority of the proposed
SIoU, Sec. 4 compares it with various existing criteria. This section analyzes
the criteria’s distributions when exposed to randomly shifted boxes. To our
knowledge, this is the first attempt to study empirically and theoretically
the distributions of these criteria. The conclusions of this analysis are then
compared with human perception through a user study which shows that SIoU
aligns better with human appraisal than IoU (see Sec. 4.3). The comparison of
the criteria also highlights that SIoU as a loss function can guide training
towards small objects better than other criteria and in a more controlled
fashion. SIoU loss can be tuned to improve the detection of small objects just
as it can be tuned to align with human perception. Finally, these analyses are
confirmed by extensive experiments on both aerial images (DOTA [38] and DIOR
[19] datasets) and natural images (Pascal VOC [9] and COCO [22] datasets).

The main contributions of this paper are as follows:

  * -

A novel scale-adaptive criterion called SIoU that can be tuned to detect
objects of various sizes.

  * -

An empirical and theoretical analysis of existing criteria that help to
understand the required properties for designing regression loss functions and
evaluation criteria.

  * -

A user study that demonstrates the misalignment between IoU and human
perception for the detection task.

  * -

Extensive experiments to support the superiority of SIoU for detecting small
objects in the few-shot regime.

##  2 Related Works

###  2.1 Intersection over Union and its Variants

To begin, let us review the definition of existing criteria for set
similarity. First, the IoU is defined as the intersection area of two sets
divided by the area of their union:

$\text{IoU}(A,B)=\frac{|A\cap B|}{|A\cup B|},$ (1)

where $A$ and $B$ are two sets. When $A$ and $B$ are rectangular boxes, IoU
can be computed easily with simple operations on box coordinates. This
explains why IoU is such a widespread criterion for object detection. It is
used as a loss function ($\mathcal{L}_{\text{reg}}=1-\text{IoU}$) by several
well established detection frameworks (_e.g_. [43, 34]). IoU is also involved
in the process of example selection during training of most detection methods,
_i.e_. all the ones inspired either by Faster R-CNN [27] or YOLO [26]. In
these frameworks, regression loss is computed from the coordinates of proposed
boxes and ground truth. Not all pairs of proposals and ground truth are kept
for the computation. Only proposals with a sufficient IoU with a ground truth
box are selected. Finally, IoU is also used at the heart of the evaluation
process. A proposed box is considered a positive detection if it meets two
conditions: 1) an IoU greater than a given threshold with a ground truth box,
and 2) the same label as this ground truth.

Several attempts were made to improve IoU but existing works mostly focus on
the regression loss part, disregarding the other IoU uses in the detection
task. First, [28] proposed a generalized version of IoU which yields negative
values when boxes do not overlap:

$\text{GIoU}(A,B)=\text{IoU}(A,B)-\frac{|C\backslash(A\cup B)|}{|C|},$ (2)

where $C$ is the convex hull around $A$ and $B$. This criterion is employed as
a loss function by several detection frameworks [34, 3, 45]. It is sometimes
also combined with other regression loss as in [20, 4], which both combine it
with an L1 regression on box coordinates. Combining IoU loss with other
regression terms was also proposed by [47]. They introduce two losses
Distance-IoU (DIoU) and Complete-IoU which respectively add an L2 regression
term and an aspect ratio penalty to the IoU loss. Recently, $\alpha$-IoU [12]
extends DIoU [47] by proposing a family of losses following the same structure
as DIoU with the IoU term raised to the power $\alpha$. Alternatively, Bounded
IoU [35] computes an IoU upper bound between a proposal and a ground truth.

All previous IoU improvements were made to tackle the regression part of the
models. However, IoU is involved in other parts of the framework including
example selection, Non-Maximal Suppression, and evaluation. A recent user
study [32] indicates that IoU does not completely align with human perception.
Humans have strong positional and size preferences based on conceptual
information contained in the boxes. It suggests that IoU is not an optimal
choice either for example selection or for evaluation as it will lead to
detections that do not satisfy human users.

###  2.2 Object Detection

Object Detection is a problem that has been studied for decades. It witnessed
rapid progress with the rise of deep learning methods [27, 26, 23]. Recent
methods achieve very satisfactory results when provided with sufficient data.
However, there remain some challenges to mastering object detection. Most
object detectors still struggle with small objects, and when data is scarce.

Small Object Detection is a challenging task. There has been plenty of
attempts to improve it based on pyramidal features [23, 21, 46], multiscale
training [30, 31], data-augmentation [17] or super-resolution [25, 1, 29, 7,
10]. But only a few works tackle this problem by changing the loss function.
Normalized Wasserstein Distance [40] (NWD) proposes an alternative to IoU loss
specifically designed for detecting small objects. It consists in computing
the Wasserstein distance between two Gaussian distributions fitted on the two
compared bounding boxes. Moreover, NWD is also used as an example selection
criterion.

Few-Shot Object Detection (FSOD) is the task of detecting objects only
provided with a handful of examples per class. Many approaches were proposed
in the literature to address this problem: metric learning [16, 42, 33, 13],
simple fine-tuning [5, 36, 37, 41] and attention-based methods [15, 39, 11,
44, 6]. The similarity between all these methods is that they learn generic
knowledge from a set of base classes with plenty of annotations and adapt to
novel classes from the few available examples. Recently, it has been shown
[24] that FSOD is even more sensitive to small objects than regular object
detection. Extracting information from small objects is hard and produces
spurious features that do not condition well the detection. Some solutions are
proposed to overcome this issue with augmentation and careful example cropping
[24] or with dedicated attention mechanisms [14]. Nevertheless, this is not
enough to solve the issue of small objects in FSOD.

##  3 Novel Scale-Adaptive Intersection over Union

Before introducing the proposed criterion, let us define two bounding boxes
$b_{1}=[x_{1},y_{1},w_{1},h_{1}]^{T}$ and
$b_{2}=[x_{2},y_{2},w_{2},h_{2}]^{T}$ (the prediction box and ground truth
respectively), where $x_{i}$ and $y_{i}$ are the center coordinates of the box
$b_{i}$, while $w_{i}$ and $h_{i}$ denote its width and height respectively.
In the following section, the adjectives small, medium, and large will be used
extensively. They have a precise meaning for object detection, defined in COCO
dataset [22]. The box $b_{i}$ is small if $\sqrt{w_{i}h_{i}}\leq 32$ pixels,
medium if $32<\sqrt{w_{i}h_{i}}\leq 96$, and large if $\sqrt{w_{i}h_{i}}>96$.

IoU is scale-invariant, hence if $\text{IoU}(b_{1},b_{2})=u$, scaling all
coordinates of both boxes by the same factor $k$ will produce the same IoU:
$\text{IoU}(b_{1},b_{2})=\text{IoU}(kb_{1},kb_{2})=u$. However, detection
models are not scale-invariant, they do not localize equally well small and
large objects. Fig. 1 (right) clearly shows that the ratio between the
localization error ($\varepsilon_{\text{loc}}=\|b_{1}-b_{2}\|_{1}$) and the
object size ($\omega=\sqrt{w_{2}h_{2}}$) increases as the objects become
smaller. This figure is made with a model trained on DOTA with all
annotations. Each point represents the ratio
$\frac{\varepsilon_{\text{loc}}}{\omega}$ for one object in the test set.
Hence, because of the scale-invariance property, IoU scores are lower for
small objects. A way to alleviate this issue is by relaxing the invariance
property of the IoU so it favors more small objects without penalizing large
ones. To this end, we propose a novel criterion called Scale-adaptive
Intersection over Union (SIoU):

$\displaystyle\text{SIoU}(b_{1},b_{2})$
$\displaystyle=\text{IoU}(b_{1},b_{2})^{p}$ (3)
$\displaystyle\text{with}\quad\quad p$
$\displaystyle=1-\gamma\exp\left({-\frac{\sqrt{w_{1}h_{1}+w_{2}h_{2}}}{\sqrt{2}\kappa}}\right),$

$p$ is a function of the object sizes, thus, the scores are rescaled according
to the objects’ size. $\gamma\in]-\infty,1]$ and $\kappa>0$ are two parameters
that control how the rescaling occurs (hence, $p\geq 0$). $\gamma$ governs the
scaling for small objects while $\kappa$ controls how fast the behavior of
regular IoU is recovered for large objects. Fig. 4 (left) in App. A shows the
evolution of $p$ with object size for various $\gamma$ and $\kappa$.

This new criterion follows the same structure as $\alpha$-IoU [12], but
differs greatly as it sets different powers for different object sizes. SIoU
provides a solution for small object detection while $\alpha$-IoU only aims to
improve general detection. However, SIoU inherits a few properties from
$\alpha$-IoU.

######  Property 1 (SIoU Relaxation).

Let $b_{1}$ and $b_{2}$ be two bounding boxes and introduce
$\tau=\frac{w_{1}h_{1}+w_{2}h_{2}}{2}$ their average area. SIoU preserves the
behavior of IoU in certain cases such as:

  * -

$\textup{IoU}(b_{1},b_{2})=0\Rightarrow\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})=0$

  * -

$\textup{IoU}(b_{1},b_{2})=1\Rightarrow\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})=1$

  * -

$\lim\limits_{\tau\to+\infty}\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})$

  * -

$\lim\limits_{\kappa\to
0}\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})$

Property 1 shows that SIoU is sound: it equals IoU when boxes have no
intersection and when they perfectly overlap. Therefore, the associated loss
function (see Property 2) will take maximal values for boxes that do not
overlap and minimum ones for identical boxes. In addition, SIoU behaves
similarly to IoU when dealing with large objects (_i.e_. when
$\tau\to\infty$). When boxes are large, the power $p$ that rescales the IoU is
close to 1. Hence, this change of criterion only impacts small objects.
However, when discussing the properties of SIoU, the limit between
small/medium/large objects is relative to the choice of $\kappa$. If
$\kappa\gg\sqrt{wh}$, even large objects will be rescaled. On the contrary,
when $\kappa\xrightarrow{}0$, all objects are treated as large and are not
rescaled. In practice, $\kappa$ and $\gamma$ are chosen empirically, but Sec.
4 provides useful insights for the choice of these parameters.

######  Property 2 (Loss and gradients reweighting).

Let $\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})=1-\textup{IoU}(b_{1},b_{2})$ and
$\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})=1-\textup{SIoU}(b_{1},b_{2})$ be the
loss functions associated respectively with IoU and SIoU. Let denote the ratio
between SIoU and IoU losses by
$\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})=\frac{\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})}{\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})}$.
Similarly,
$\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})=\frac{|\nabla\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})|}{|\nabla\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})|}$
denotes the ratio of gradients generated from SIoU and IoU losses:

$\displaystyle\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})$
$\displaystyle=\frac{1-\textup{IoU}(b_{1},b_{2})^{p}}{1-\textup{IoU}(b_{1},b_{2})},$
(4) $\displaystyle\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})$
$\displaystyle=p\textup{IoU}(b_{1},b_{2})^{p-1},$ (5)

$\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\mathcal{\nabla}}$ are
increasing (resp. decreasing) functions of IoU when $p\geq 1$ (resp. $p<1$)
which is satisfied when $\gamma\leq 0$ (resp. $\gamma>0$). As the IoU goes to
1, $\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\mathcal{\nabla}}$ approaches
$p$:

$\displaystyle\lim\limits_{\textup{IoU}(b_{1},b_{2})\to
1}\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})$ $\displaystyle=p,$ (6)
$\displaystyle\lim\limits_{\textup{IoU}(b_{1},b_{2})\to
1}\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})$ $\displaystyle=p.$ (7)

We employ the same tools as in [12] to analyze how SIoU affects the losses and
associated gradients. We show in property 2 that their results hold for a non-
constant power $p$ as well. From this, it can be observed that when IoU is
close to 1, losses and gradients are both rescaled by $p$. Hence, the
gradients coming from objects of different sizes will be rescaled differently.
The setting of $\gamma$ and $\kappa$ allows focusing the training on specific
object sizes. Experimental results are provided in Sec. 5 to support these
findings. Proofs for properties 1 and 2 are available in App. B

However, order preservingness is not satisfied by using power value changing
with the size of the objects. This property ensures that the order given by
the IoU is preserved with the novel criterion, _e.g_.
$\text{IoU}(b_{1},b_{2})<\text{IoU}(b_{1},b_{3})\Rightarrow\alpha\text{-IoU}(b_{1},b_{2})<\alpha\text{-IoU}(b_{1},b_{3})$.
$\alpha$-IoU preserves the order of IoU, but SIoU does not. We show in App. B
that even though this property is not always satisfied, a large proportion of
boxes meet the conditions for the order to hold.

Extensions and generalization
Finally, SIoU can very well be extended as IoU was with GIoU or DIoU. We
provide here an extension following GIoU as it appears especially well-
designed for small object detection. When detecting small targets, it is
easier for a model to completely miss the object, producing an IoU of 0 no
matter how far the predicted box is. On the contrary, GIoU yields negative
values for non-intersecting boxes. This produces more relevant guidance during
the early phase of training when the model outputs poorly located boxes.
Therefore, we extend SIoU by raising GIoU to the same power $p$ as in Eq. 3:

$\text{GSIoU}(b_{1},b_{2})=\begin{cases}\text{g}(b_{1},b_{2})^{p}&\text{if
}\text{g}(b_{1},b_{2})\geq 0\\\ -|\text{g}(b_{1},b_{2})|^{p}&\text{if
}\text{g}(b_{1},b_{2})<0\end{cases},$ (8)

where $g(b_{1},b_{2})=\text{GIoU}(b_{1},b_{2})$.

##  4 Scale-Adaptive Criteria Analysis

Figure 2: Analysis of the distribution of IoU, SIoU, GIoU, GSIoU and
$\alpha$-IoU when computed on inaccurately positioned boxes. This is done by
observing the probability distribution functions for various $\omega$ values
(left), the expectation (middle) and standard deviation (right) for all
criteria. For SIoU and GSIoU, we fixed $\gamma=0.5$ and $\kappa=64$, for
$\alpha$-IoU, $\alpha=3$ (as recommended in the original paper [12]). The
inaccuracy of the detector is set to $\sigma=16$. Note that the empirical pdfs
were smoothed using a Kernel Density Estimator method. This affects
particularly IoU, SIoU , and $\alpha$-IoU for the actual pdf is defined only
on $[0,1]$. For the sake of visualization, GIoU and GSIoU were rescaled
between 0 and 1 for the expectation and standard deviation plots.

This section analyzes both empirically and theoretically the behaviors of IoU,
GIoU [28], $\alpha$-IoU [12], NWD [40], SIoU and GSIoU. We investigate the
desirable properties of such criteria for model training and performance
evaluation.

###  4.1 Response Analysis to Box Shifting

As mentioned in Sec. 3, IoU drops dramatically when the localization error
increases for small objects. Shifting a box a few pixels off the ground truth
can result in a large decrease in IoU, without diminishing the quality of the
detection from a human perspective. This is depicted in Fig. 1 (left), plain
lines represent the evolution of IoU for various object sizes. These curves
are generated by diagonally shifting a box away from the ground truth. Boxes
are squares, but similar curves would be observed otherwise. In this plot,
boxes have the same size, therefore, when there is no shift in between
($\varepsilon_{\text{loc}}=0$), IoU equals 1. However, if the sizes of the
boxes differ by a ratio $r$, IoU would peak at $1/r^{2}$. Other line types
represent other criteria. SIoU decreases slower than IoU when
$\varepsilon_{\text{loc}}$ increases, this is especially true when boxes are
small. This holds because $\gamma=0.5$, if it was negative, SIoU would adopt
the opposite behavior. In addition, the gap between IoU and SIoU is even
larger when objects are small. Only NWD shares this property, but it only
appears when boxes have different sizes (all lines coincide for NWD). Hence,
SIoU is the only criterion that allows controlling its decreasing rate, _i.e_.
how much SIoU is lost for a 1-pixel shift. As GIoU and GSIoU values range in
$[-1,1]$, they were not included in Fig. 1, but the same analysis holds for
them as well (see App. C).

###  4.2 Resilience Analysis to Detector Inaccuracy

Knowing how a criterion responds to shifts and size variations is important to
understand what makes a sensible box similarity measure. Pushing beyond the
shift analysis, we study empirically and theoretically the criteria’s
distributions when exposed to detector inaccuracies, _i.e_. randomly shifted
boxes. This setting mimics the inaccuracy of the model either during training
or at test time.

####  4.2.1 Empirical Protocol

To simplify, let us suppose that all boxes are squares of the same size
$\omega$ and can be shifted only horizontally. Similar results are observed by
relaxing these constraints, see App. C. A box is then entirely defined by its
position $x$ and its width $\omega$. If a detector is not perfect, it will
produce bounding boxes slightly shifted horizontally from the ground truth. To
model the detector’s inaccuracy, we suppose that the box position is randomly
sampled from a centered Gaussian distribution:
$X\sim\mathcal{N}(0,\sigma^{2})$ where $\sigma$ controls how inaccurate the
model is. We are interested in the distribution of
$\mathcal{C}\in\\{\text{IoU},\text{GIoU},\text{SIoU},\text{GSIoU},\alpha\text{-IoU},\text{NWD}\\}$
and how it changes with $\omega$. To this end, let $Z=\mathcal{C}(X)$. More
precisely, we are interested in the probability density function (pdf) of $Z$
and its two first moments (which exist because $\mathcal{C}$ is continuous and
bounded).

Fig. 2 gathers the results of this analysis. It shows the pdf of each
criterion for various box sizes (left) along with the evolution of the
expectation and standard deviation of $Z$ against $\omega$ (middle and right).
From this, it can be noted that the size of the boxes has a large influence on
the distributions of all criteria. The expected values of all criteria are
monotonically increasing with object size. In particular, small objects have
lower expected IoU values than larger ones. This is consistent with the
initial assessment from Fig. 1 (right) and it validates the choice of $\sigma$
constant for this study (although App. C discusses this assumption).

When building detection models, we hope to detect equally well objects of all
sizes, this means having a constant expected IoU, no matter the objects’ size.
This would require the localization error to be an affine function of
$\omega$. Of course, the localization error of the detector is likely to
depend on $\omega$. However, it cannot be an affine function, otherwise, small
objects would be perfectly detected, which is not observed (see Fig. 1,
right). As SIoU has larger expected values than IoU for small objects, it can
compensate for their larger localization errors. The setting of $\gamma$ and
$\kappa$ allows controlling how much small objects are favored (see Fig. 4 in
App. A). NWD is not included in these plots as its expected value and variance
are constant when dealing with same-size boxes.

Influence Analysis on the Performance Evaluation
If the expected value of a criterion is too small, it is likely that the boxes
will be considered negative detections during evaluation and therefore reduce
the performance. Therefore, having a criterion with larger expected values for
small objects would better reflect the true performance of a detector. One
might think that it would be equivalent to scale-adaptive IoU thresholds
during the evaluation, but this is not completely true as the variances of the
criteria also differ.

Having an accurate criterion (_i.e_. with low variance) is crucial for
evaluation. Let us take a detector that produces well-localized boxes on
average, _i.e_. on average the criterion computed between the boxes and their
corresponding ground truths is above a certain threshold. As the detector is
not perfect, it will randomly produce boxes slightly better or slightly worse
than the average. If the criterion has a high variance, it will be more likely
that poor boxes get scores below the criterion threshold and therefore will be
considered negative detections. This will reduce the performance of the
detector even though on average, it meets the localization requirements. In
addition, a criterion with a higher variance will be less reliable and would
produce more inconsistent evaluations of a model. The fact that the IoU
variance is high for small objects partly explains why detectors have much
lower performance on these objects. Hence, SIoU seems more adapted for
evaluation. Of course, using this criterion for evaluation will attribute
higher scores for less precise localization of small objects. However, this
aligns better with human perception as demonstrated in Sec. 4.3. Employing
SIoU as a metric also allows tweaking that metric for the needs of a specific
application.

Influence Analysis on Training
All criteria discussed above are employed as regression losses in the
literature. The loss associated with each criterion $\mathcal{C}$ is
$\mathcal{L}_{\mathcal{C}}(b_{1},b_{2})=1-\mathcal{C}(b_{1},b_{2})$.
Therefore, the expected value of the criterion determines the expected value
of the loss and thus the magnitude of the gradients. Large values of the
criterion give low values of the loss. Now, as the expected values of the
criteria change with the objects size, the expected values of the losses also
change. Small objects generate greater loss values than larger ones on
average. However, this is not enough as performance on small objects is poor
when training with IoU. To achieve better detection, training must focus even
more on small objects. One way to do this is to set larger loss values for
small objects. That way, the equilibrium is shifted toward smaller objects and
gradients will point to regions where the loss of small objects is lower. As
shown in Fig. 5 (App. A), with the right parameters, SIoU can do that. It
attributes lower values for small objects while keeping similar values for
large ones. The contrast between small and large objects is accentuated and
optimization naturally focuses on smaller objects. SIoU’s parameters control
which object size gets more emphasis. This is closely linked to Property 2
which states that employing SIoU (compared to IoU) reweights the loss and the
gradient by $p$. If $\gamma<0$, $p$ decreases with the size of the objects and
thus the optimization focuses on small objects. This also explains why
generalizations of existing criteria (_i.e_. with negative values for non-
overlapping boxes) often outperform their vanilla version. Taking IoU and GIoU
as an example, the gap between their expected values for small and large
objects is greater with GIoU. It nudges the optimization towards small
objects.

####  4.2.2 Theoretical study of GIoU

Criteria pdfs and first moments can also be derived theoretically. We provide
such results for GIoU in Proposition 1.

######  Proposition 1 (GIoU’s distribution).

Let $b_{1}=(0,y_{1},w_{1},h_{1})$ be a bounding box horizontally centered and
$b_{2}=(X,y_{2},w_{2},h_{2})$ another bounding box randomly positioned, with
$X\sim\mathcal{N}(0,\sigma^{2})$ and $\sigma\in\mathbb{R}^{*}_{+}$. Let’s
suppose that the boxes are identical squares, shifted only horizontally
(_i.e_. $w_{1}=w_{2}=h_{1}=h_{2}$ and $y_{1}=y_{2}$).
Let $Z=\mathcal{C}(X)$, where $\mathcal{C}$ is the generalized intersection
over union. The probability density function of $Z$ is given by:

$\displaystyle d_{Z}(z)$
$\displaystyle=\frac{2\omega}{(1+z)^{2}\sqrt{2\pi}\sigma}\exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^{2}\right).$
(9)

The two first moments of $Z$ exist and are given by:

$\displaystyle\mathbb{E}[Z]$
$\displaystyle=\frac{2}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}0&\frac{1}{2}&\frac{1}{2}\\\
\frac{1}{2}&0\end{matrix}\right),$ (10) $\displaystyle\mathbb{E}[Z^{2}]$
$\displaystyle=1-\frac{8a}{\sqrt{2\pi}}+\frac{16a^{2}}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}-1&\frac{1}{2}&-\frac{1}{2}\\\
\frac{1}{2} &0\end{matrix}\right),$ (11)

where $G$ is the Meijer G-function [2] and $a=\frac{\sigma}{\omega}$.

The proof of this proposition and derivations for other criteria are available
in App. C. The theoretical expressions completely agree with empirical
results, which confirms the soundness of our simulations.

###  4.3 SIoU Alignment with Human Perception

As discussed in Sec. 4.2.1, having an accurate criterion _i.e_. one with low
variance, is crucial for evaluation. However, such a criterion must also align
with human perception. Most image processing models are destined to assist
human users. Therefore, to maximize the usefulness of such models, the
evaluation process should align as closely as possible with human perception.
To assess the agreement between the criteria and human perception, we
conducted a user study in which participants had to rate on a 1 to 5 scale
(_i.e_. from very poor to very good) how a bounding box detects an object.
Specifically, an object is designated by a green ground truth box and a red
box is randomly sampled around the object (_i.e_. with random IoU with the
ground truth). Then, the participants rate how well the red box localizes the
object within the green one. The study gathered 75 different participants and
more than 3000 individual answers. We present here the main conclusion of this
study. Detailed results and protocol are available in App. D.

Human perception does not fully align with IoU. People tend to be more lenient
than IoU towards small objects. Specifically, comparing a small and a large
box with the same IoU with respect to their own ground truth, people will rate
the small one better. This suggests that IoU is too strict for small objects
in comparison with human perception. From a human perspective, precise
localization seems less important for small objects. Fig. 3 represents the
relative gap of IoU (left) and SIoU (right) values for each object size and
rating. The relative differences $c_{s,r}$ are computed against the average
IoU (or SIoU) value per rating:

$c_{s,r}=\frac{\mathcal{C}_{s,r}-\sum\limits_{s}\mathcal{C}_{s,r}}{\sum\limits_{s}\mathcal{C}_{s,r}},$
(12)

where $\mathcal{C}_{s,r}$ is the average criterion value
($\mathcal{C}\in\\{\text{IoU},\text{SIoU}\\}$) for objects of size $s$ and
rating $r$. IoU values for small objects (in orange) are lower than for large
objects (in red) for all rating $r$. For a human to give a rating $r$ to a
box, it requires that a box overlaps less with the ground truth (according to
IoU) if the boxes are small. SIoU compensates for this trend (see Fig. 3
(right)): SIoU differences between objects of different sizes but the same
rating are smaller than for IoU. This means that SIoU process objects
independently of their size. Similar charts with absolute IoU and SIoU values
can be found in App. D (Fig. 10), it also includes charts for $\alpha$-IoU,
NWD and other SIoU parameters. Here $\gamma=0.2$ and $\kappa=64$ for SIoU.
Choosing a higher $\gamma$ value would reverse the trend and produce a
criterion even more lenient than humans for small objects. It will also
decrease further SIoU’s variance. However, this setting has been chosen to
maximize the alignment with human perception. SIoU correlates better with
human rating compared with other criteria. As the rating is an ordered
categorical variable, we choose Kendall rank correlation to make the
comparison. The correlation between the human rating $r$ and each criterion
can be found in Tab. 1 (correlations between criteria are available in Tab. 9
in App. D). SIoU with $\gamma=0.2$ and $\kappa=64$ aligns best with human
perception and has a low variance. This showcases the superiority of SIoU over
existing criteria. It should be preferred over IoU to assess the performance
of models on all visual tasks that commonly employ IoU within their evaluation
process. It supports recent findings that show misalignment between IoU and
human preference [32].

| IoU | SIoU |  $\bm{\alpha}$-IoU | NWD
---|---|---|---|---
$\bm{r}$ | 0.674 | 0.701 | 0.674 | 0.550

Table 1: Kendall’s $\tau$ correlation between various criteria and human
rating $r$. For SIoU, $\gamma=0.2$ and $\kappa=64$, for $\alpha$-IoU,
$\alpha=3$. Figure 3: Average IoU (left) and SIoU (right) scores for different
object sizes and human ratings $r\in\\{1,2,3,4,5\\}$. Values are reported as
the relative gap with the average value per rating.

##  5 Experimental Results

To support our analysis from Sec. 4, we conduct various experiments, mainly on
aerial images with DOTA [38] and DIOR [19] datasets. To showcase the
versatility of SIoU, we also experiment with natural images on Pascal VOC [9]
and COCO [22]. Detecting small objects in the few-shot regime is extremely
challenging and could have much more applications than regular detection
pipelines. Therefore, most of our experiments focus on the few-shot setting.
However, we also report results in regular object detection to display the
potential of SIoU. For the few-shot experiments, we choose a recently proposed
FSOD method: Cross-Scale Query Support Alignment (XQSA) [14]. A comparison
with other methods is available in Sec. E.2. Since the FSOD training is
relatively complex, we defer the implementation details in Sec. E.1. In few-
shot literature, it is common to evaluate models separately on base and novel
classes, however novel classes performance is what matters the most as it
assesses the generalization capabilities of the models. Performance is
computed using mean Average Performance (mAP) with a 0.5 IoU threshold.

Comparison with Existing Criteria
To begin, we compare the few-shot performance on DOTA with various loss
functions designed with the criteria discussed in Sec. 4. The result of these
experiments is available in Tab. 2. The criteria are divided into two groups,
generalized (which includes NWD) and vanilla criteria. As discussed in Sec.
4.2.1, the generalized versions of the criteria outperform their original
counterparts and therefore should be compared separately. Scale-adaptive
criteria (SIoU and GSIoU) largely outperforms other criteria on novel classes
and especially on small objects. For SIoU and GSIoU, we choose $\gamma=-3$ and
$\kappa=16$ according to a series of experiments conducted on DOTA to
determine their optimal values (see App. A). It is important to point out the
relatively good performance of NWD despite not checking all the desirable
properties highlighted in Sec. 4.

| Base classes |  | Novel Classes
---|---|---|---
Loss | All | S | M | L |  | All | S | M | L
IoU | 50.67 | 25.83 | 57.49 | 68.24 |  | 32.41 | 10.06 | 47.87 | 67.09
$\alpha$-IoU | 46.72 | 13.24 | 55.21 | 69.94 |  | 33.95 | 12.58 | 46.58 | 74.50
SIoU | 53.62 | 24.07 | 61.91 | 67.34 |  | 39.05 | 16.59 | 54.42 | 74.49
NWD | 50.79 | 19.19 | 58.90 | 67.90 |  | 41.65 | 28.26 | 50.16 | 65.06
GIoU | 52.41 | 26.94 | 61.17 | 63.00 |  | 41.03 | 24.01 | 52.13 | 69.78
GSIoU | 52.91 | 22.14 | 61.19 | 66.02 |  | 45.88 | 34.83 | 51.26 | 70.78

Table 2: Few-shot performance comparison between several criteria: IoU,
$\alpha$-IoU, SIoU, NWD, GIoU, and GSIoU trained on DOTA. mAP is reported with
a 0.5 IoU threshold for small (S), medium (M), large (L), and all objects.

FSOD on Aerial and Natural Images
As the previous set of experiments was only carried out on DOTA, we showcase
the versatility of GSIoU on three other datasets: DIOR, Pascal VOC and COCO.
As it is clear that generalized criteria outperform other methods, the
comparison here is only done between GIoU and GSIoU. For DOTA and DIOR,
current state-of-the-art is achieved by XQSA [14], which employs GIoU as
regression loss. Therefore, we replace it with GSIoU and achieve significantly
better performance on these two datasets (see Tab. 3).

For Pascal VOC and COCO, similar gains are observed, but it requires a
different tuning of SIoU. $\gamma=-3$ and $\kappa=16$ produce mitigated
results these datasets, and $\gamma=-1$ and $\kappa=64$ is a more sensible
choice. This was predictable as the objects in Pascal VOC and COCO are
substantially larger than in DOTA and DIOR. This can also explain the slightly
smaller gains on DIOR compared to DOTA. Finding optimal values of $\gamma$ and
$\kappa$ could yield slightly better performance on DIOR. The tuning of SIoU
is quite straightforward, as lower values of $\gamma$ and $\kappa$ skew the
training towards smaller objects. The right balance depends on the proportion
of small, medium and large objects in the datasets. With natural images which
contain fewer small objects, the training balance does not need to be shifted
as much as for aerial images. In addition to these results, we also conducted
several experiments with various FSOD methods to demonstrate the plug-and-play
nature of GSIoU. These results are available in App. E and show consistent
improvements when replacing GIoU with GSIoU.

|  | Base classes |  | Novel Classes
---|---|---|---|---
| XQSA | All | S | M | L |  | All | S | M | L
DOTA | w/ GIoU | 52.41 | 26.94 | 61.17 | 63.00 |  | 41.03 | 24.01 | 52.13 | 69.78
w/ GSIoU | 52.91 | 22.14 | 61.19 | 66.02 |  | 45.88 | 34.83 | 51.26 | 70.78
DIOR | w/ GIoU | 58.90 | 10.38 | 40.76 | 80.44 |  | 47.93 | 9.85 | 47.61 | 68.40
w/ GSIoU | 60.29 | 11.28 | 43.24 | 81.63 |  | 52.85 | 13.78 | 53.73 | 71.22
Pascal | w/ GIoU | 51.09 | 13.93 | 40.26 | 62.01 |  | 48.42 | 18.44 | 36.06 | 59.99
w/ GSIoU | 54.47 | 13.88 | 40.13 | 66.82 |  | 55.16 | 22.94 | 36.24 | 67.40
COCO | w/ GIoU | 19.15 | 8.72 | 22.50 | 30.59 |  | 26.25 | 11.96 | 23.95 | 38.60
w/ GSIoU | 19.57 | 8.41 | 23.02 | 31.07 |  | 27.11 | 12.81 | 26.02 | 39.20

Table 3: Few-shot performance on three datasets: DOTA, DIOR, Pascal VOC and
COCO. GIoU and GSIoU losses are compared. mAP is reported with a 0.5 IoU
threshold and for all object sizes.

Regular Object Detection on DOTA and DIOR
Of course, GSIoU is not only beneficial for FSOD, but it also improves the
performance of regular object detection methods. Tab. 4 compares the
performance of FCOS [34] trained on DOTA and DIOR with GIoU and GSIoU. The
same pattern is visible, we get better performance with GSIoU. However, the
gain for small objects is not as large as for FSOD. Nevertheless, it suggests
that other tasks relying on IoU could also benefit from GSIoU.

| DOTA |  | DIOR
---|---|---|---
FCOS | All | S | M | L |  | All | S | M | L
w/ GIoU | 34.9 | 17.4 | 36.6 | 43.3 |  | 48.1 | 10.1 | 40.3 | 63.2
w/ GSIoU | 36.8 | 17.5 | 40.4 | 45.2 |  | 49.2 | 11.0 | 41.2 | 66.1

Table 4: Regular Object Detection performance on DOTA and DIOR datasets with
GIoU and GSIoU ($\gamma=-3$ and $\kappa=16$) losses. mAP is computed with
several IoU thresholds (0.5 to 0.95) as it is commonly done in regular
detection.

Discussions and Limitations
As mentioned in Sec. 4.2.1 SIoU is a better choice for performance analysis.
However, as IoU is almost the only choice in literature for evaluation, we
must use it as well for a fair comparison with existing works. Nonetheless, we
provide results from previous tables using SIoU as the evaluation criterion in
App. F. They agree with the IoU evaluation and strengthen the conclusions of
our experiments. While these results are promising, we must emphasize a few
limitations of SIoU. First, SIoU requires a slight tuning to get the best
performance, but that tuning is quite straightforward and mostly depends on
the size distribution in the target images. SIoU allows being more lenient
with small objects for evaluation ($\gamma\geq 0$), and stricter for training
($\gamma\leq 0$) to prioritize the detection of small targets. Although they
are not always part of the detection pipeline, it would be relevant to
investigate the replacement of IoU by SIoU for example selection and Non-
Maximal Suppression (in our case, the example selection of FCOS does not rely
on IoU). Finally, even though SIoU aligns better than IoU with human
perception, it does not match completely with it. IoU and SIoU do not account
for object content whereas humans heavily do, as highlighted by [32].

##  6 Conclusion

SIoU is a more suitable alternative than IoU both for object detection model
evaluation and training, especially in the few-shot regime. It better aligns
with human perception and our theoretical analysis confirms sounder properties
for evaluation. To our knowledge, this is the first statistical analysis of
bounding boxes similarity measure and hopefully, this will lead to more
reflections on object detection criteria. As a loss function, SIoU can incline
the training toward small objects and therefore greatly improve FSOD
performance. Its flexibility allows to easily focus the detection on specific
target sizes and adapt to various tasks. Extensive experiments on aerial and
natural images demonstrate the superiority of SIoU on small objects, without
performance loss on medium and large objects. On aerial images, which contain
a lot of small objects, SIoU even achieves state-of-the-art FSOD performance.

## References

  * [1] Yancheng Bai, Yongqiang Zhang, Mingli Ding, and Bernard Ghanem.  Sod-mtgan: Small object detection via multi-task generative adversarial network.  In ECCV, 2018.
  * [2] Richard Beals and Jacek Szmigielski.  Meijer g–functions: A gentle introduction, 08 2013.
  * [3] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.  Yolov4: Optimal speed and accuracy of object detection.  arXiv preprint arXiv:2004.10934, 2020.
  * [4] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In European Conference on Computer Vision, pages 213–229. Springer, 2020.
  * [5] Hao Chen, Yali Wang, Guoyou Wang, and Yu Qiao.  Lstd: A low-shot transfer detector for object detection.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [6] Tung-I Chen, Yueh-Cheng Liu, Hung-Ting Su, Yu-Cheng Chang, Yu-Hsiang Lin, Jia-Fong Yeh, and Winston H Hsu.  Should i look at the head or the tail? dual-awareness attention for few-shot object detection.  arXiv preprint arXiv:2102.12152, 2021.
  * [7] Luc Courtrai, Minh-Tan Pham, and Sébastien Lefèvre.  Small object detection in remote sensing images based on super-resolution with auxiliary generative adversarial networks.  Remote. Sens., 12:3152, 2020.
  * [8] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian.  Centernet: Keypoint triplets for object detection.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6569–6578, 2019.
  * [9] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman.  The pascal visual object classes (voc) challenge.  International journal of computer vision, 88(2):303–338, 2010.
  * [10] Syeda Nyma Ferdous, Moktari Mostofa, and Nasser M. Nasrabadi.  Super resolution-assisted deep aerial vehicle detection.  In Defense + Commercial Sensing, 2019.
  * [11] Guangxing Han, Shiyuan Huang, Jiawei Ma, Yicheng He, and Shih-Fu Chang.  Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment.  arXiv preprint arXiv:2104.07719, 2021.
  * [12] Jiabo He, Sarah Erfani, Xingjun Ma, James Bailey, Ying Chi, and Xian-Sheng Hua.  Alpha-iou: A family of power intersection over union losses for bounding box regression.  Advances in Neural Information Processing Systems, 34:20230–20242, 2021.
  * [13] Pierre Le Jeune, Mustapha Lebbah, Anissa Mokraoui, and Hanene Azzag.  Experience feedback using representation learning for few-shot object detection on aerial images.  In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 662–667, 2021.
  * [14] Pierre Le Jeune and Anissa Mokraoui.  A comparative attention framework for better few-shot object detection on aerial images, 2022.
  * [15] Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, and Trevor Darrell.  Few-shot object detection via feature reweighting.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8420–8429, 2019.
  * [16] Leonid Karlinsky, Joseph Shtok, Sivan Harary, Eli Schwartz, Amit Aides, Rogerio Feris, Raja Giryes, and Alex M Bronstein.  Repmet: Representative-based metric learning for classification and few-shot object detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5197–5206, 2019.
  * [17] Mate Kisantal, Zbigniew Wojna, Jakub Murawski, Jacek Naruniec, and Kyunghyun Cho.  Augmentation for small object detection.  arXiv preprint arXiv:1902.07296, 2019.
  * [18] Hei Law and Jia Deng.  Cornernet: Detecting objects as paired keypoints.  In Proceedings of the European conference on computer vision (ECCV), pages 734–750, 2018.
  * [19] Ke Li, Gang Wan, Gong Cheng, Liqiu Meng, and Junwei Han.  Object detection in optical remote sensing images: A survey and a new benchmark.  ISPRS Journal of Photogrammetry and Remote Sensing, 159:296–307, 2020.
  * [20] Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang.  Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection.  Advances in Neural Information Processing Systems, 33:21002–21012, 2020.
  * [21] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.
  * [22] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In European conference on computer vision, pages 740–755. Springer, 2014.
  * [23] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg.  Ssd: Single shot multibox detector.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 21–37, Cham, 2016. Springer International Publishing.
  * [24] Anissa Mokraoui Pierre Le Jeune.  Improving few-shot object detection through a performance analysis on aerial and natural images.  In Proceedings of the 30th European Signal Processing Conference (EUSIPCO), 2022.
  * [25] Jakaria Rabbi, Nilanjan Ray, Matthias Schubert, Subir Chowdhury, and Dennis Chao.  Small-object detection in remote sensing images with end-to-end edge-enhanced gan and object detector network.  Remote Sensing, 12(9):1432, 2020.
  * [26] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  * [27] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  Advances in neural information processing systems, 28:91–99, 2015\.
  * [28] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese.  Generalized intersection over union: A metric and a loss for bounding box regression.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658–666, 2019.
  * [29] Jacob Shermeyer and Adam Van Etten.  The effects of super-resolution on object detection performance in satellite imagery.  2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1432–1441, 2019.
  * [30] Bharat Singh, Mahyar Najibi, and Larry S Davis.  Sniper: Efficient multi-scale training.  Advances in neural information processing systems, 31, 2018.
  * [31] Bharat Singh, Mahyar Najibi, Abhishek Sharma, and Larry S Davis.  Scale normalized image pyramids with autofocus for object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3749–3766, 2021.
  * [32] Ombretta Strafforello, Vanathi Rajasekart, Osman S Kayhan, Oana Inel, and Jan van Gemert.  Humans disagree with the IoU for measuring object detector localization error.  In 2022 IEEE International Conference on Image Processing (ICIP), pages 1261–1265. IEEE, 2022.
  * [33] Bo Sun, Banghuai Li, Shengcai Cai, Ye Yuan, and Chi Zhang.  Fsce: Few-shot object detection via contrastive proposal encoding.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7352–7362, 2021.
  * [34] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  Fcos: Fully convolutional one-stage object detection.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9627–9636, 2019.
  * [35] Lachlan Tychsen-Smith and Lars Petersson.  Improving object localization with fitness nms and bounded iou loss.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6877–6885, 2018.
  * [36] Xin Wang, Thomas E. Huang, Trevor Darrell, Joseph E Gonzalez, and Fisher Yu.  Frustratingly simple few-shot object detection.  In International Conference on Machine Learning (ICML), July 2020\.
  * [37] Stefan Wolf, Jonas Meier, Lars Sommer, and Jürgen Beyerer.  Double head predictor based few-shot object detection for aerial imagery.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 721–731, 2021.
  * [38] Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang.  Dota: A large-scale dataset for object detection in aerial images.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3974–3983, 2018.
  * [39] Zixuan Xiao, Jiahao Qi, Wei Xue, and P. Zhong.  Few-shot object detection with self-adaptive attention network for remote sensing images.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:4854–4865, 2021.
  * [40] Chang Xu, Jinwang Wang, , Wen Yang, Huai Yu, Lei Yu, and Gui-Song Xia.  Detecting tiny objects in aerial images: A normalized wasserstein distance and a new benchmark.  ISPRS Journal of Photogrammetry and Remote Sensing (ISPRS J P & RS), 2022.
  * [41] Yang Xu, Bohao Huang, Xiong Luo, Kyle Bradbury, and Jordan M. Malof.  Simpl: Generating synthetic overhead imagery to address zero-shot and few-shot detection problems, 2021.
  * [42] Yukuan Yang, Fangyun Wei, Miaojing Shi, and Guoqi Li.  Restoring negative information in few-shot object detection.  Advances in neural information processing systems, 33:3521–3532, 2020.
  * [43] Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang.  UnitBox.  In Proceedings of the 24th ACM international conference on Multimedia. ACM, oct 2016.
  * [44] Gongjie Zhang, Zhipeng Luo, Kaiwen Cui, and Shijian Lu.  Meta-detr: Few-shot object detection via unified image-level meta-learning.  arXiv preprint arXiv:2103.11731, 2021.
  * [45] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li.  Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9759–9768, 2020.
  * [46] Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, and Stan Z Li.  Single-shot refinement neural network for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4203–4212, 2018.
  * [47] Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren.  Distance-iou loss: Faster and better learning for bounding box regression.  In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 12993–13000, 2020.

##  Appendix A Influence of $\gamma$ and $\kappa$ on SIoU and GSIoU

In this appendix, we analyze the influence of the parameters $\gamma$ and
$\kappa$ on the SIoU and GSIoU.

$\gamma$ and $\kappa$ influence on SIoU
First, following the analysis from Sec. 4.2.1, Fig. 4 investigates the
influence of $\gamma$ and $\kappa$ on SIoU behavior. Fig. 4(a) shows the value
of $p$, the expectation and variance of SIoU against object size for
$\gamma\in\\{1.0,0.75,0.5,0.1,0.0,-0.25,-2,-4\\}$. $p$ is a function of the
average area of the boxes ($\tau=\frac{w_{1}h_{1}+w_{2}h_{2}}{2})$, for
simplicity we suppose here that the boxes are squares of the same width
$\omega$, hence $\sqrt{\tau}=\omega$. Then $p$ can be viewed as a function of
$\omega$: $p(\omega)=1-\gamma\exp(-\omega/\kappa)$. For negative values of
$\gamma$, $p$ decreases from $p(0)=1-\gamma$ to 1, small objects get higher
exponents in comparison with larger objects. On the contrary, when $\gamma>0$,
$p$ increases from $p(0)=1-\gamma$ to 1. Changing $\gamma$ also influences the
distribution of SIoU. As $\gamma$ increases, the expected value for small
objects increases as well, while the variance decreases.

Fig. 4(b) shows the same curves for $\kappa\in\\{8,16,32,64,256\\}$. $\kappa$
controls how fast $p$ approaches 1 and therefore, changing $\kappa$ simply
shifts the curves of expectation and variance accordingly. As $\kappa$
increases, IoU’s behavior is retrieved for larger objects reducing the
expected value of SIoU. The variance is not changed much by $\kappa$, it
slightly shifts the maximum of the curve, _i.e_. the object size for which
SIoU’s variance is maximum.

Fig. 5 also provides pdfs plots for various object sizes for SIoU and GSIoU,
in addition to expectation and variance comparison between existing criteria.
For this figure, $\gamma=-3$ and $\kappa=16$. This figure echoes Fig. 2 which
plots the same curves but with $\gamma=0.5$ and $\kappa=64$.

(a) $\gamma$’s influence, with $\kappa=64$

(b) $\kappa$’s influence, with $\gamma=-3$

Figure 4: Influence of $\gamma$ and $\kappa$ on the expected value and
standard deviation of SIoU. Figure 5: Analysis of the distribution of IoU,
SIoU, GIoU, GSIoU and $\alpha$-IoU when computed on inaccurately positioned
boxes. This is done by observing the probability distribution functions (pdfs)
for various $\omega$ values (left), the expectation (middle) and standard
deviation (right) for all criteria. For SIoU and GSIoU, we fixed $\gamma=-4$
and $\kappa=16$, for $\alpha$-IoU, $\alpha=3$ (as recommended in the original
paper [12]). The inaccuracy of the detector is set to $\sigma=16$. Note that
the empirical pdfs were smoothed using a Kernel Density Estimator method. This
affects particularly IoU, SIoU and $\alpha$-IoU for the actual pdf is defined
only on $[0,1]$. For the sake of visualization, GIoU and GSIoU were rescaled
between 0 and 1 for the expectation and standard deviation plots.

$\gamma$ and $\kappa$ influence on FSOD Performance
The setting of $\gamma$ and $\kappa$ are crucial for training. Therefore, we
conducted various experiments on dataset DOTA to find the best parameters for
GSIoU loss. The results can be found in Tabs. 5 and 6. This shows that the
optimal values for DOTA dataset are $\gamma=-3$ and $\kappa=8$. However,
$\kappa=16$ is also a good choice and seems more consistent across datasets.
Thus, we choose to keep $\gamma=-3$ and $\kappa=16$ for other experiments. An
exhaustive grid search should be done to find even better settings. Our search
was sparse and a better combination of $\gamma$ and $\kappa$ probably exists.

| Base classes | Novel Classes
---|---|---
$\bm{\gamma}$ | All | S | M | L | All | S | M | L
0.5 | 47.09 | 21.29 | 54.67 | 65.48 | 30.50 | 8.83 | 44.97 | 65.89
0.25 | 45.94 | 21.60 | 54.39 | 63.40 | 30.96 | 12.53 | 42.37 | 64.14
0 | 52.41 | 26.94 | 61.17 | 63.00 | 41.03 | 24.01 | 52.13 | 69.78
-0.5 | 52.80 | 27.16 | 61.19 | 64.61 | 41.06 | 25.20 | 50.18 | 72.04
-1 | 53.03 | 23.20 | 61.53 | 66.68 | 42.77 | 27.55 | 52.01 | 70.76
-2 | 54.06 | 23.68 | 62.69 | 66.62 | 43.67 | 30.04 | 51.69 | 69.66
-3 | 52.91 | 22.14 | 61.19 | 66.02 | 45.88 | 34.83 | 51.26 | 70.78
-4 | 53.59 | 22.50 | 62.48 | 66.18 | 42.43 | 27.56 | 51.79 | 68.70
-9 | 53.11 | 20.98 | 62.13 | 67.00 | 42.63 | 30.53 | 48.89 | 68.62

Table 5: Evolution of the few-shot performance (XQSA with GSIoU loss) on DOTA
for various values of $\gamma$ ($\kappa=16$ is fixed).

| Base classes | Novel Classes
---|---|---
$\bm{\kappa}$ | All | S | M | L | All | S | M | L
4 | 51.65 | 21.50 | 59.76 | 65.85 | 42.98 | 30.33 | 48.57 | 73.41
8 | 52.70 | 21.96 | 61.49 | 66.43 | 44.16 | 31.35 | 50.70 | 71.99
16 | 54.06 | 23.68 | 62.69 | 66.62 | 43.67 | 30.04 | 51.69 | 69.66
32 | 53.88 | 22.33 | 63.00 | 67.35 | 37.36 | 23.65 | 44.60 | 66.29
64 | 52.82 | 21.79 | 61.46 | 66.77 | 43.68 | 29.43 | 52.47 | 69.46
128 | 53.42 | 21.73 | 62.90 | 66.75 | 41.32 | 26.85 | 49.40 | 70.38

Table 6: Evolution of the few-shot performance (XQSA with GSIoU loss) for
various values of $\kappa$ ($\gamma=-2$ is fixed).

Influence of $\bm{\gamma}$ and $\bm{\kappa}$ on FSOD performance on Pascal VOC
dataset
The search conducted above was carried out on DOTA dataset. It transposes
nicely on DIOR dataset as well. Yet, these two datasets are similar. They both
contain aerial images with some classes in common, but most importantly, the
size of their objects are highly similar. When applied to different datasets,
these results may not hold. For instance, Pascal VOC requires other
combinations of $\gamma$ and $\kappa$ to outperform the training with GIoU.
This is shown in Tab. 7.

| Base Classes | Novel Classes
---|---|---
Loss | All | S | M | L | All | S | M | L
GIoU | 51.09 | 13.93 | 40.26 | 62.01 | 48.42 | 18.44 | 36.06 | 59.99
GSIoU $\bm{\gamma}=-3$, $\bm{\kappa}=16$ | 45.22 | 10.06 | 34.85 | 57.10 | 43.16 | 14.89 | 33.92 | 54.16
GSIoU $\bm{\gamma}=-1$, $\bm{\kappa}=64$ | 54.47 | 13.88 | 40.13 | 66.82 | 55.16 | 22.94 | 36.24 | 67.40
GSIoU $\bm{\gamma}=0.5$, $\bm{\kappa}=64$ | 56.97 | 13.88 | 40.75 | 70.31 | 55.36 | 20.25 | 36.85 | 68.05

Table 7: Few-shot performance on Pascal VOC dataset with different values of
$\gamma$ and $\kappa$.

##  Appendix B Proofs of Properties

In this appendix, we provide the proofs for Properties 1 and 2, and discuss
the order-preservigness of SIoU.

######  Property 1 (SIoU Relaxation).

Let $b_{1}$ and $b_{2}$ be two bounding boxes and introduce
$\tau=\frac{w_{1}h_{1}+w_{2}h_{2}}{2}$ their average area. SIoU preserves the
behavior of IoU in certain cases such as:

  * -

$\textup{IoU}(b_{1},b_{2})=0\Rightarrow\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})=0$

  * -

$\textup{IoU}(b_{1},b_{2})=1\Rightarrow\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})=1$

  * -

$\lim\limits_{\tau\to+\infty}\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})$

  * -

$\lim\limits_{\kappa\to
0}\textup{SIoU}(b_{1},b_{2})=\textup{IoU}(b_{1},b_{2})$

######  Proof.

First let recall the expression of SIoU,
$\text{SIoU}(b_{1},b_{2})=\text{IoU}(b_{1},b_{2})^{p}$ with
$p=1-\gamma\exp\left({-\frac{\sqrt{\tau}}{\kappa}}\right)$. $\tau>0$ because
boxes cannot be empty and as $\gamma\in]-\infty,1]$ and
$\kappa\in\mathbb{R}_{+}^{*}$, we have $p>0$.

From this, the first two items of Property 1 follow clearly.

The two other points hold because the function $f\colon
x\mapsto\text{IoU}(b_{1},b_{2})^{x}$ is continuous on $\mathbb{R}$ for any
couple of boxes $b_{1}$ and $b_{2}$ ($\text{IoU}(b_{1},b_{2})\in[0,1]$) and
$\lim\limits_{\tau\to\infty}p=\lim\limits_{\kappa\to 0}p=1$.

######  Property 2 (Loss and gradients reweighting).

Let $\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})=1-\textup{IoU}(b_{1},b_{2})$ and
$\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})=1-\textup{SIoU}(b_{1},b_{2})$ be the
loss functions associated respectively with IoU and SIoU. Let denote the ratio
between SIoU and IoU losses by
$\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})=\frac{\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})}{\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})}$.
Similarly,
$\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})=\frac{|\nabla\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})|}{|\nabla\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})|}$
denotes the ratio of gradients generated from SIoU and IoU losses:

$\displaystyle\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})$
$\displaystyle=\frac{1-\textup{IoU}(b_{1},b_{2})^{p}}{1-\textup{IoU}(b_{1},b_{2})},$
(4) $\displaystyle\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})$
$\displaystyle=p\textup{IoU}(b_{1},b_{2})^{p-1},$ (5)

$\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\mathcal{\nabla}}$ are
increasing (resp. decreasing) functions of IoU when $p\geq 1$ (resp. $p<1$)
which is satisfied when $\gamma\leq 0$ (resp. $\gamma>0$). As the IoU goes to
1, $\mathcal{W}_{\mathcal{L}}$ and $\mathcal{W}_{\mathcal{\nabla}}$ approaches
$p$:

$\displaystyle\lim\limits_{\textup{IoU}(b_{1},b_{2})\to
1}\mathcal{W}_{\mathcal{L}}(b_{1},b_{2})$ $\displaystyle=p,$ (6)
$\displaystyle\lim\limits_{\textup{IoU}(b_{1},b_{2})\to
1}\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})$ $\displaystyle=p.$ (7)

######  Proof.

Let denote the $\text{IoU}(b_{1},b_{2})$ by $\mu$, and define two functions
$f\colon\mu\mapsto 1-\mu=\mathcal{L}_{\text{IoU}}(b_{1},b_{2})$ and
$g\colon\mu\mapsto 1-\mu^{p}=\mathcal{L}_{\text{SIoU}}(b_{1},b_{2})$.

$f$ and $g$ are differentiable on $[0,1]$ and $\lim\limits_{\mu\to
1}f(\mu)=\lim\limits_{\mu\to 1}g(\mu)=0$. This holds because $p$ is
independent of the IoU (_i.e_. $\mu$).

Therefore L’Hôpital’s rule can be applied:
$\lim\limits_{\mu\to 1}\mathcal{W}_{\mathcal{L}}=\lim\limits_{\mu\to
1}\frac{g(\mu)}{f(\mu)}=\lim\limits_{\mu\to
1}\frac{g^{\prime}(\mu)}{f^{\prime}(\mu)}=\lim\limits_{\mu\to 1}p\mu^{p-1}=p$.

The expression of the second ratio
$\mathcal{W}_{\mathcal{\nabla}}(b_{1},b_{2})$ follows directly as
$|\nabla\mathcal{L}_{\textup{SIoU}}(b_{1},b_{2})|=g^{\prime}(\mu)$ and
$|\nabla\mathcal{L}_{\textup{IoU}}(b_{1},b_{2})|=f^{\prime}(\mu)$, hence
$\lim\limits_{\mu\to 1}\mathcal{W}_{\nabla}=\lim\limits_{\mu\to
1}p\mu^{p-1}=p$.

Order-preservigness
Let us take three boxes $b_{1}$, $b_{2}$, and $b_{3}$. Order-preservigness
does not hold for SIoU. Therefore
$\text{IoU}(b_{1},b_{2})\leq\text{IoU}(b_{1},b_{3})$ does not imply
$\text{SIoU}(b_{1},b_{2})\leq\text{SIoU}(b_{1},b_{3})$. However, this property
is often true in practice. Denoting by $\tau_{i,j}$ the average area between
boxes $i$ and $j$, we can study the conditions for the order to hold. We will
also note
$p_{i,j}=1-\gamma\exp\left({-\frac{\sqrt{\tau_{i,j}}}{\kappa}}\right)$

Let’s suppose, without loss of generality, that $\tau_{1,2}\leq\tau_{1,3}$.
Otherwise, cases 1 and 2 would be swapped.

Case 1:  $\gamma\leq 0$
We have $p_{1,2}>p_{1,3}$ as $\tau_{1,2}\leq\tau_{1,3}$ and $\gamma\leq 0$.
Therefore, $p_{1,2}=p_{1,3}+\varepsilon$, with $\varepsilon>0$.

Then,

$\displaystyle\text{IoU}(b_{1},b_{2})^{p_{1,2}}$
$\displaystyle=\text{IoU}(b_{1},b_{2})^{p_{1,3}+\varepsilon}$
$\displaystyle=\text{IoU}(b_{1},b_{2})^{p_{1,3}}\text{IoU}(b_{1},b_{2})^{\varepsilon}$
$\displaystyle\leq\text{IoU}(b_{1},b_{2})^{p_{1,3}}$
$\displaystyle\leq\text{IoU}(b_{1},b_{3})^{p_{1,3}}$

Line 3 holds because $0<\text{IoU}(b_{1},b_{2})^{\varepsilon}\leq 1$. Line 4
is true because $\text{IoU}(b_{1},b_{2})\leq\text{IoU}(b_{1},b_{3})$ and the
function $h\colon x\mapsto x^{p_{1,3}}$ is monotonically increasing. Hence,
when $\gamma\leq 0$ the order is preserved.

Case 2:  $\gamma>0$
We have $p_{1,3}>p_{1,2}$ as $\tau_{1,3}\leq\tau_{1,2}$ and $\gamma>0$.
Therefore, $p_{1,3}=p_{1,2}+\varepsilon$, with $\varepsilon>0$.

In this case, the order does not always hold, counter-examples can be found.
However, it is useful to study the conditions for it to hold:

$\text{IoU}(b_{1},b_{2})^{p_{1,2}}\leq\text{IoU}(b_{1},b_{3})^{p_{1,3}}\Leftrightarrow\frac{\ln\big{(}\text{IoU}(b_{1},b_{2})\big{)}}{\ln\big{(}\text{IoU}(b_{1},b_{3})\big{)}}\geq\frac{p_{1,3}}{p_{1,2}}$

In practice, the right condition is often true as $p_{1,2}$ and $p_{1,3}$ are
close due to scale matching, a trick present in most detection frameworks to
prevent comparison of proposals and ground truth of very different sizes. In
addition, the ratio of log values gets large rapidly, even if the gap between
$\text{IoU}(b_{1},b_{2})$ and $\text{IoU}(b_{1},b_{3})$ is small, the ratio
$\frac{\ln\big{(}\text{IoU}(b_{1},b_{2})\big{)}}{\ln\big{(}\text{IoU}(b_{1},b_{3})\big{)}}$
can be large.

The order-preservigness property holds in many cases. This is sensible as IoU
is still a reliable metric that has been used extensively for the training and
evaluation of detection models. However, in the rare cases where this order is
broken, the $\text{IoU}(b_{1},b_{2})$ and $\text{IoU}(b_{1},b_{3})$ are close,
so IoU does not discriminate much between the boxes. On the contrary, SIoU
prefers the smallest one (or largest one, according to the choice of
$\gamma$). This stronger discrimination is probably beneficial for training
and evaluation.

##  Appendix C Inaccuracy Analysis

###  C.1 Shift plots for GIoU and GSIoU

Figure 6: Evolution of various criteria (IoU, GIoU, and GSIoU) when a box is
shifted from the ground truth box by $\rho$ pixels for various box sizes
$\omega\in\\{4,16,64,128\\}$. With boxes of the same size (left) and different
sizes (right).

###  C.2 Inaccuracy Tolerance Assumptions

Several assumptions were made in Sec. 4.2 to analyze the criteria for box
similarity:

  1. 1.

Boxes are shifted only horizontally.

  2. 2.

Boxes have the same size.

  3. 3.

The detector’s inaccuracy is fixed and does not depend on the object size.

The first two assumptions are relatively harmless. Allowing diagonal shifts
simply accelerates the IoU decreasing rate. A 1-pixel diagonal shift is
equivalent to a vertical and a horizontal shift. Intuitively, this is similar
to a 2-pixels horizontal shift. This is not true because, with a 1-pixel
diagonal shift, the area of intersection decreases slower than with a 2-pixels
horizontal shift. Following the notations from Sec. 4.2, the intersection
between two boxes of width $\omega$ diagonally shifted by $\rho$ pixels is
$(\omega-\rho)^{2}=\omega^{2}-2\omega\rho+\rho^{2}$ while the intersection
between same boxes horizontally shifted by $2\rho$ pixels is
$\omega(\omega-2\rho)=\omega^{2}-2\omega\rho$. To ensure that this does not
question the conclusions of Sec. 4.2.1, Fig. 7(a) compares the expected values
and variances of IoU and GIoU with horizontal and diagonal shifts. Similar
behaviors are observed with or without diagonal shifting. The only difference
is that the expected values of the criteria for diagonally shifted boxes are
lower as the shifts get larger. It also increases the variances as the
distributions are more spread. Relaxing the second constraint also results in
slightly different distributions, but with similar behavior. Having different
size boxes only changes the maximum value of the criteria. If boxes have
different sizes, the maximum value must be smaller than 1\. Therefore, the
expected values approach smaller values than 1 as objects get larger. And the
variance is reduced as the range of criteria values is smaller. This is
visible in Fig. 7(b).

(a) IoU and GIoU expected values and standard deviation with horizontally and
diagonally shifted boxes.

(b) IoU and GIoU expected values and standard deviation with and without boxes
of the same size.

(c) IoU, SIoU, GIoU, and GSIoU expected values and standard deviation with the
detector’s inaccuracy modeled as an affine function,
$\sigma(\omega)=\sigma_{0}+\lambda\omega$ ($\sigma_{0}=16$,
$\lambda=\frac{1}{4}$).

Figure 7: Relaxing the constraints for criteria’ distribution analysis.

It is less straightforward that the conclusions hold without the last
assumption. In the analysis from Sec. 4.2.1, we assume that the inaccuracy of
the detector is fixed. This means that the detector generates randomly shifted
boxes by the same number of pixels on average no matter the size of the
object. This is certainly false, in practice, in terms of absolute distance,
detectors are better with smaller objects. However, the inaccuracy cannot
simply be proportional to object sizes because small objects would then be
perfectly detected. Thus, we tried to change the inaccuracy of the detector as
an affine function of the box width:
$\sigma(\omega)=\sigma_{0}+\lambda\omega$. We choose to set $\sigma_{0}$ to
the fixed value of $\sigma$ used in Sec. 4.2.1 and $\lambda=1/4$. This setting
better reflects the inaccuracy of a true detector. The expected values and
standard deviations of IoU, SIoU, GIoU, and GSIoU with this inaccuracy setting
are plotted in Fig. 7(c). The main difference with fixed inaccuracy is that
expected values do not approach 1 as object size gets larger, instead they
tend towards lower values. It also leads to non-zero variance for large
objects. For small objects, however, the curves of the different criteria are
mostly unchanged, and the conclusions formulated in Sec. 4.2.1 are still
valid.

###  C.3 Theoretical GIoU’s distribution

In this appendix, we give a proof of the formulas of the probability density
function, expected value, and variance of GIoU. We also provide some non-
closed-form expressions for other criteria.

######  Proposition 1 (GIoU’s distribution).

Let $b_{1}=(0,y_{1},w_{1},h_{1})$ be a bounding box horizontally centered and
$b_{2}=(X,y_{2},w_{2},h_{2})$ another bounding box randomly positioned, with
$X\sim\mathcal{N}(0,\sigma^{2})$ and $\sigma\in\mathbb{R}^{*}_{+}$. Let’s
suppose that the boxes are identical squares, shifted only horizontally
(_i.e_. $w_{1}=w_{2}=h_{1}=h_{2}$ and $y_{1}=y_{2}$).
Let $Z=\mathcal{C}(X)$, where $\mathcal{C}$ is the generalized intersection
over union. The probability density function of $Z$ is given by:

$\displaystyle d_{Z}(z)$
$\displaystyle=\frac{4\omega}{(1+z)^{2}\sqrt{2\pi}\sigma}\exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^{2}\right).$
(8)

The two first moments of $Z$ exist and are given by:

$\displaystyle\mathbb{E}[Z]$
$\displaystyle=\frac{2}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}0&\frac{1}{2}&\frac{1}{2}\\\
\frac{1}{2}&0\end{matrix}\right),$ (9) $\displaystyle\mathbb{E}[Z^{2}]$
$\displaystyle=1-\frac{8a}{\sqrt{2\pi}}+\frac{16a^{2}}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}-1&\frac{1}{2}&-\frac{1}{2}\\\
\frac{1}{2} &0\end{matrix}\right),$ (10)

where $G$ is the Meijer G-function [2] and $a=\frac{\sigma}{\omega}$.

| Criterion Expression | Probability Density Function | $\bm{\mathbb{E}[Z]}$ | $\bm{\mathbb{E}[Z^{2}]}$ |
---|---|---|---|---|---
IoU |  $\displaystyle\mathcal{C}\colon\mathbb{R}$ $\displaystyle\to[0,1]$ $\displaystyle x$ $\displaystyle\mapsto\max\left(0,\frac{\omega-|x|}{\omega+|x|}\right)$ |  $\displaystyle d_{Z}(z)=2\bigg{[}($ $\displaystyle 1-F_{X}(\omega\frac{1-z}{1+z})\delta_{0}(z)$ $\displaystyle+\mathds{1}_{\mathbb{R}_{+}}(z)\frac{4\omega}{(1+z)^{2}}d_{X}(\omega\frac{1-z}{1+z}))\bigg{]}$ |  $\displaystyle\mathbb{E}[Z]=\frac{4}{\sqrt{2\pi}a}\int_{0}^{1}\frac{1}{1+u}e^{-\frac{u^{2}}{2a^{2}}}\,du-\text{erf}(\frac{1}{\sqrt{2}a})$ |  $\displaystyle\mathbb{E}[Z^{2}]=\text{erf}(\frac{1}{\sqrt{2}a})-\frac{8a}{\sqrt{2\pi}a}\bigg{[}$ $\displaystyle 1-\frac{1}{2}e^{-\frac{1}{2a^{2}}}$ $\displaystyle-2\int_{0}^{1}\frac{1}{(1+u)^{3}}e^{-\frac{u^{2}}{2a^{2}}}\,du\bigg{]}$ |
GIoU |  $\displaystyle\mathcal{C}\colon\mathbb{R}$ $\displaystyle\to[-1,1]$ $\displaystyle x$ $\displaystyle\mapsto\frac{\omega-|x|}{\omega+|x|}$ |  $\displaystyle d_{Z}(z)$ $\displaystyle=\frac{4\omega}{(1+z)^{2}\sqrt{2\pi}\sigma}\exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^{2}\right)$ |  $\displaystyle\mathbb{E}[Z]$ $\displaystyle=\frac{2}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}0&\frac{1}{2}&\frac{1}{2}\\\ \frac{1}{2}&0\end{matrix}\right)$ |  $\displaystyle\mathbb{E}[Z^{2}]$ $\displaystyle=1-\frac{8a}{\sqrt{2\pi}}+\frac{16a^{2}}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}-1&\frac{1}{2}&-\frac{1}{2}\\\ \frac{1}{2} &0\end{matrix}\right)$ |
SIoU |  $\displaystyle\mathcal{C}\colon\mathbb{R}$ $\displaystyle\to[0,1]$ $\displaystyle x$ $\displaystyle\mapsto\begin{cases}\left(\frac{\omega-|x|}{\omega+|x|}\right)^{p}\,&\text{if }\omega-|x|\geq 0\\\ 0\,&\text{otherwise}\end{cases}$ |  $\displaystyle d_{Z}(z)=2\bigg{[}($ $\displaystyle 1-F_{X}(\omega\frac{1-z^{1/p}}{1+z^{1/p}})\delta_{0}(z)$ $\displaystyle+\mathds{1}_{\mathbb{R}_{+}}(z)\frac{4\omega z^{1/p-1}}{(1+z^{1/p})^{2}}d_{X}(\omega\frac{1-z^{1/p}}{1+z^{1/p}}))\bigg{]}$ |  $\displaystyle\mathbb{E}[Z]$ $\displaystyle=2\omega\int_{0}^{1}\big{(}\frac{1-u}{1+u}\big{)}^{p}e^{-\frac{u^{2}}{2a^{2}}}\,du$ |  $\displaystyle\mathbb{E}[Z^{2}]$ $\displaystyle=2\omega\int_{0}^{1}\big{(}\frac{1-u}{1+u}\big{)}^{2p}e^{-\frac{u^{2}}{2a^{2}}}\,du$ |
GSIoU |  $\displaystyle\mathcal{C}\colon\mathbb{R}$ $\displaystyle\to[-1,1]$ $\displaystyle x$ $\displaystyle\mapsto\begin{cases}\left(\frac{\omega-|x|}{\omega+|x|}\right)^{p}\,&\text{if }\omega-|x|\geq 0\\\ -\left(\frac{|x|-\omega}{\omega+|x|}\right)^{p}\,&\text{otherwise}\end{cases}$ |  $\displaystyle d_{Z}(z)=2\bigg{[}($ $\displaystyle 1-F_{X}(\omega\frac{1-z^{1/p}}{1+z^{1/p}})\delta_{0}(z)$ $\displaystyle+\mathds{1}_{\mathbb{R}_{+}}(z)\frac{4\omega z^{1/p-1}}{(1+z^{1/p})^{2}}d_{X}(\omega\frac{1-z^{1/p}}{1+z^{1/p}}))$ $\displaystyle-(1-F_{X}(\omega\frac{1+|z|^{1/p}}{1-|z|^{1/p}})\delta_{0}(z))$ $\displaystyle+\mathds{1}_{\mathbb{R}_{-}}(z)\frac{4\omega|z|^{1/p-1}}{(1-|z|^{1/p})^{2}}d_{X}(\omega\frac{1+|z|^{1/p}}{1-|z|^{1/p}}))\bigg{]}$ |  $\displaystyle\mathbb{E}[Z]=2\omega\bigg{[}$ $\displaystyle\int_{0}^{1}\big{(}\frac{1-u}{1+u}\big{)}^{p}e^{-\frac{u^{2}}{2a^{2}}}\,du$ $\displaystyle-$ $\displaystyle\int_{1}^{+\infty}\big{(}\frac{1-u}{1+u}\big{)}^{p}e^{-\frac{u^{2}}{2a^{2}}}\,du\bigg{]}$ |  $\displaystyle\mathbb{E}[Z^{2}]=2\omega\bigg{[}$ $\displaystyle\int_{0}^{1}\big{(}\frac{1-u}{1+u}\big{)}^{2p}e^{-\frac{u^{2}}{2a^{2}}}\,du$ $\displaystyle-\int_{1}^{+\infty}\big{(}\frac{1-u}{1+u}\big{)}^{2p}e^{-\frac{u^{2}}{2a^{2}}}\,du\bigg{]}$ |

Table 8: Criteria expression, probability distribution and first two moments
for IoU, GIoU, SIoU, and GSIoU. These are valid for the comparison of same
size square boxes of width $\omega$ randomly shifted horizontally. Random
shifts are sampled from a centered Gaussian distribution of variance
$\sigma^{2}$ and $a=\sigma/\omega$.

######  Proof.

First, in the setup described in Proposition 1, GIoU can be expressed in terms
of the width of the boxes $\omega$ and the shift in between $x$. Let’s call
this function $\mathcal{C}$:

$\displaystyle\mathcal{C}\colon\mathbb{R}$ $\displaystyle\to[-1,1]$
$\displaystyle x$ $\displaystyle\mapsto\frac{\omega-|x|}{\omega+|x|}$

The shifts are sampled from a Gaussian distribution:
$X\sim\mathcal{N}(0,\sigma^{2})$, therefore we are interested in the
distribution of the variable $Z=\mathcal{C}(X)$.

The cumulative density function of $Z$ is given by:

$\displaystyle F_{Z}(z)=P(Z\leq z)$
$\displaystyle=P(\frac{\omega-|X|}{\omega+|X|}\leq y)$
$\displaystyle=P(\omega\frac{1-z}{1+z}\leq|X|)$
$\displaystyle=2P(\omega\frac{1-z}{1+z}\leq X)$
$\displaystyle=2(1-F_{X}(\omega\frac{1-z}{1+z}))$
$\displaystyle=2(1-F_{X}(g(z)))$

With $g(z)=\omega(\frac{1-z}{1+z})$.

Hence, the density function of $Z$ can be derived by taking the derivative of
$F_{Z}$:

$\displaystyle d_{Z}(z)$ $\displaystyle=\frac{d}{dz}F_{Z}(z)$
$\displaystyle=-2g^{\prime}(z)F_{X}^{\prime}(g(y))$
$\displaystyle=\frac{4\omega}{(1+z)^{2}\sqrt{2\pi}\sigma}\exp\left(-\frac{1}{2}\left[\frac{\omega(1-z)}{\sigma(1+z)}\right]^{2}\right)$

To determine the first and second moments of $Z$, we make use of the change of
variable formula:

$\displaystyle\mathbb{E}[Z]=\mathbb{E}[\mathcal{C}(X)]=\int_{-\infty}^{+\infty}\mathcal{C}(x)d_{X}(x)\,dx$
$\displaystyle\mathbb{E}[Z^{2}]=\mathbb{E}[\mathcal{C}(X)^{2}]=\int_{-\infty}^{+\infty}\mathcal{C}(x)^{2}d_{X}(x)\,dx$

Let’s start with $\mathbb{E}[Z]$:

$\displaystyle\mathbb{E}[Z]$
$\displaystyle=\int_{-\infty}^{+\infty}\mathcal{C}(x)d_{X}(x)\,dx$
$\displaystyle=\int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}\sigma}\frac{\omega-|x|}{\omega+|x|}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=\frac{2}{\sqrt{2\pi}\sigma}\int_{0}^{+\infty}\frac{\omega-x}{\omega+x}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=\frac{2}{\sqrt{2\pi}\sigma}\int_{0}^{+\infty}\frac{2\omega-(\omega+x)}{\omega+x}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=\frac{2}{\sqrt{2\pi}\sigma}\left[2\int_{0}^{+\infty}\frac{\omega}{\omega+x}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx-\int_{0}^{+\infty}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx\right]$
$\displaystyle=\frac{4}{\sqrt{2\pi}\sigma}\int_{0}^{+\infty}\frac{\omega}{\omega+x}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx-1$
$\displaystyle=\frac{4}{\sqrt{2\pi}a}\int_{0}^{+\infty}\frac{1}{1+u}e^{-\frac{u^{2}}{2a^{2}}}\,du-1$
$\displaystyle=\frac{4}{\sqrt{2\pi}a}\frac{\sqrt{2}a}{2\pi}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}0&\frac{1}{2}&\frac{1}{2}\\\
\frac{1}{2}&0\end{matrix}\right)-1$
$\displaystyle=\frac{2}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}0&\frac{1}{2}&\frac{1}{2}\\\
\frac{1}{2}&0\end{matrix}\right)-1$

From line 2 to 3, we used the parity of function $\mathcal{C}$, between 6 and
7, a change of variable $u=x/\omega$ is done and $a$ is set to
$\sigma/\omega$. Finally, in the second-to-last line, we identify a Meijer-G
function [2] evaluated at $2a^{2}$. Unfortunately, there exist no closed-form
of the integral
$\int_{0}^{+\infty}\frac{1}{1+u}e^{-\frac{u^{2}}{2a^{2}}}\,dx$. In this case,
a Mellin transform of this integral allows recognizing a Meijer-G function.
For other criteria, their first two moments cannot be expressed in a similar
closed form. That is why we only provide the theoretical expressions of the
expectation and variance of GIoU.

A similar derivation leads to the expression of the second moment of $Z$:

$\displaystyle\mathbb{E}[Z^{2}]$
$\displaystyle=\int_{-\infty}^{+\infty}\mathcal{C}(x)^{2}d_{X}(x)\,dx$
$\displaystyle=\int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}\sigma}\big{(}\frac{\omega-|x|}{\omega+|x|}\big{)}^{2}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=\frac{2}{\sqrt{2\pi}\sigma}\int_{0}^{+\infty}\big{(}\frac{\omega-x}{\omega+x}\big{)}^{2}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=1-\frac{8\omega}{\sqrt{2\pi}\sigma}\int_{0}^{+\infty}\frac{x}{(\omega+x)^{2}}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx$
$\displaystyle=1-\frac{8}{\sqrt{2\pi}a}\left[a^{2}-2\sigma^{2}\int_{0}^{+\infty}\frac{1}{(\omega+x)^{3}}e^{-\frac{x^{2}}{2\sigma^{2}}}\,dx\right]$
$\displaystyle=1-\frac{8}{\sqrt{2\pi}a}\left[a^{2}-2a^{2}\int_{0}^{+\infty}\frac{1}{(1+u)^{3}}e^{-\frac{u^{2}}{2a^{2}}}\,du\right]$
$\displaystyle=1-\frac{8a}{\sqrt{2\pi}}+\frac{16a^{2}}{\pi^{3/2}}G^{2,3}_{3,2}\left(2a^{2}\bigg{\rvert}\begin{matrix}-1&\frac{1}{2}&-\frac{1}{2}\\\
\frac{1}{2} &0\end{matrix}\right)$

From line 2 to 3, we again use the parity of $\mathcal{C}$, from 4 to 5, an
integration by parts is done, and finally, from 5 to 6, we apply the change of
variable $u=x/\omega$. Once again, we get an integral that does not have any
closed form but can be expressed by another Meijer-G function.

For completeness, we recall here the definition of the Meijer-G function:

$\displaystyle
G^{m,n}_{p,q}\left(z\bigg{\rvert}\begin{matrix}a_{1}&...&a_{p}\\\
b_{1}&...&b_{q}\end{matrix}\right)$ (11) $\displaystyle=\frac{1}{2\pi
i}\int_{L}\frac{\prod\limits_{j=1}^{m}\Gamma(b_{j}-s)\prod\limits_{j=1}^{n}\Gamma(1-a_{j}+s)}{\prod\limits_{j=m+1}^{q}\Gamma(1-b_{j}+s)\prod\limits_{j=n+1}^{p}\Gamma(a_{j}-s)}z^{s}\,ds,$

where $L$ is the integration path and $\Gamma$ is the gamma function. $m$,
$n$, $p$ and $q$ are integers while $a_{j}$ and $b_{j}$ are real or complex
numbers. There are some constraints on these parameters, but we do not detail
them here, they can be found in [2].

###  C.4 Non-Closed-Form Expressions for moments of IoU, SIoU, and GSIoU

Other criteria do not have closed forms for their first and second moments.
Nonetheless, we provide in this appendix their expressions keeping the
integrals as simple as possible, which allows relatively easy evaluation. In
addition, we provide the expression of the pdf for each criterion. The setup
remains identical as in Proposition 1, the boxes, of the same width $\omega$,
are only horizontally shifted. For clarity, we also give simple expressions of
each criterion in such a setup (see Tab. 8)

##  Appendix D User study

Experimental Protocol
To carry out the user study about detection preferences, we developed a Web
App to gather participants’ answers. Each participant had to sign in with a
brief form. They are asked about their age and if they are familiar with image
analysis. After completing the form, participants are brought to the
experiment page. On this page, one image is visible with bounding boxes drawn
on it. A green one, which represents the ground truth annotation of an object,
and a red one randomly shifted and deformed. Each participant must rate how
well the red box is detecting the object inside the green box. The rating is
done on a 5-levels scale, going from very poor to very good. A set of 50
different images is shown to each participant. After 25 images, the experiment
changes slightly: the background image is replaced by a completely black
image. This would remove any contextual bias coming from the variety of
objects inside the green box. We refer to the two phases of the experiment as
the phases with or without context respectively. The red boxes are sampled
around the green box, but to enforce a uniform distribution of the IoU with
the green box, a random IoU value $\psi$ is first uniformly sampled between 0
and 1. Then, we randomly generate a red box that has an IoU $\psi$ with the
green box (direct box sampling does not produce uniformly distributed IoU
values). Participants are instructed to answer quickly and are provided with
examples for each possible rating (see Fig. 8). The images and the annotations
are randomly picked from the dataset DOTA.

Figure 8: Examples given to participants of the user study. The IoU between
the green and red boxes are 0.1, 0.25, 0.5, 0.75, and 0.9 for the ratings from
very poor to very good respectively.

General Statistics about Participants
The study gathered 75 participants and a total of 3136 individual answers
(because some participants did not complete the entire experiment). The age of
the participants ranges from 21 to 64 years old with an average of 31.
Approximately half (37) of the participants are knowledgeable in computer
vision or image analysis, we will refer to this group as the expert group. On
average, the response time is 10.3s per evaluated image during the first phase
of the experiment (when a background image is visible). It drops to 7.2s when
the image is replaced by a uniform background during the second phase. This
time difference suggests that humans do take into account the contextual
information of the image inside their decision-making process.

User Study Insights
We are interested here in several variables: the object size, the presence of
contextual information, the expertise, and the age of the participants. The
object size (small, medium, or large) is at the heart of our analysis, and we
want to study its influence on human rating. The other three variables are
studied mostly to explore possible biases in the experiment. The main result
coming from this study is that for the same IoU value, humans will attribute a
higher rating to smaller objects. This suggests that IoU is a too strict
criterion for small objects and that a better criterion should not be scale-
independent. This is reported in Tab. 9, which gathers the average rating $r$
under different groupings (by object size, presence of contextual information,
the expertise of the participants, and age of the participants). In addition,
the average value for each criterion is given for each group. IoU value is
close to 0.5 for every group as expected (boxes were chosen to have a
uniformly distributed IoU). However, values of other criteria vary from one
group to another. This is especially true for scale-dependent criteria (SIoU
and NWD) on different object size groups. To check whether the different
groups are statistically different, we conducted one-way ANOVA tests on the
four variables from Tab. 9. The results confirm that the mean ratings for
various object sizes are statistically different ($p<8.4\times 10^{-26}$
111$p$ stands for the $p$-value of the statistical test here, not the exponent
from SIoU’s definition.). However, the tests find no statistical difference
for other variables: participant expertise ($p<0.47$), participant age
($p<0.02$), and presence of contextual information ($p<0.28$).

To investigate the alignment of the various criteria with the human
perception, Fig. 9 plots the rating values against the criteria value. For
visualization purposes, random vertical shifts were added to rating values to
distinguish between the values of each variable and between data points. From
this figure, it is clear that the IoU is not a perfect criterion as a wide
range of IoU values is attributed to the same rating value. It seems also
clear that contextual information, participant expertise, and age do not
introduce much change in the human rating. However, the object size has a
consistent influence on the rating: the average IoU value for a rating value
decreases with the object size (these are visible as the black vertical lines
in Fig. 3). This completely agrees with the statistical test results. SIoU can
compensate for this trend and produces more aligned averages for the different
object sizes. NWD has the same effect, but largely reverses the trend in the
other direction. Finally, the exact values of the averages for each object
size and rating value are available in Tab. 10. These are also presented in
the form of bar charts in Figs. 3 and 10. Fig. 10 presents similar bar charts
for all criteria and even for SIoU with various values of $\gamma$.

Finally, we also investigate the correlation between the various criteria and
human rating. The results of this analysis are available in Tab. 11. We choose
Kendall’s $\tau$ to deal with human rating correctly as it is an ordered
categorical variable. SIoU correlates best with human rating. Of course, the
correlation between the criteria is high as they are mostly derivated from
IoU, except NWD which shows a lower correlation both with other criteria and
human rating.

Figure 9: Rating against IoU, SIoU ($\gamma=0.2$, $\kappa=64$), NWD and
$\alpha$-IoU ($\alpha=3$) values, overall and for different groupings of the
variables of interest (object size, presence of contextual information,
expertise and age of the participants). Colors represent different values for
each variable, a legend for each row is included in the right-most column of
the figure. For the Age variable, the participants have been separated into
three groups of the same size.  Figure 10: Criteria’ scores for different
object sizes and human ratings $r\in\\{1,2,3,4,5\\}$. Multiple values of
$\gamma$ are employed for SIoU to highlight its influence on the criterion
values.

|  | $\bm{r}$ | IoU | SIoU | NWD |  $\bm{\alpha}$-IoU
---|---|---|---|---|---|---
Object size | Small | 3.406 | 0.507 | 0.550 | 0.610 | 0.203
Medium | 3.158 | 0.502 | 0.532 | 0.424 | 0.199
Large | 2.824 | 0.491 | 0.500 | 0.151 | 0.189
Context | w/o context | 3.144 | 0.504 | 0.531 | 0.397 | 0.197
w/ context | 3.112 | 0.496 | 0.523 | 0.390 | 0.197
Expertise | Inexperienced | 3.104 | 0.493 | 0.520 | 0.392 | 0.194
Expert | 3.152 | 0.507 | 0.535 | 0.395 | 0.200
Age | (10, 25] | 3.215 | 0.504 | 0.531 | 0.397 | 0.196
(25, 40] | 3.078 | 0.496 | 0.524 | 0.390 | 0.198
(40, 65] | 3.085 | 0.501 | 0.529 | 0.394 | 0.197

Table 9: Average rating and criteria values for different groupings of the
variables of interest (object size, presence of contextual information,
expertise and age of the participants).

| $\bm{r}$ | IoU | SIoU | NWD |  $\bm{\alpha}$-IoU
---|---|---|---|---|---
Small
Objects | 1 | 0.214 | 0.262 | 0.346 | 0.013
2 | 0.279 | 0.330 | 0.415 | 0.035
3 | 0.449 | 0.498 | 0.551 | 0.108
4 | 0.584 | 0.627 | 0.683 | 0.234
5 | 0.746 | 0.776 | 0.822 | 0.435
Medium
Objects | 1 | 0.223 | 0.257 | 0.160 | 0.016
2 | 0.299 | 0.335 | 0.230 | 0.039
3 | 0.474 | 0.506 | 0.361 | 0.127
4 | 0.651 | 0.677 | 0.575 | 0.306
5 | 0.771 | 0.791 | 0.716 | 0.470
Large
Objects | 1 | 0.245 | 0.258 | 0.015 | 0.025
2 | 0.322 | 0.334 | 0.036 | 0.051
3 | 0.517 | 0.527 | 0.121 | 0.168
4 | 0.713 | 0.720 | 0.318 | 0.383
5 | 0.766 | 0.772 | 0.422 | 0.480

Table 10: Average criteria (IoU, SIoU, NWD and $\alpha$-IoU) values for
different object sizes and human rating values.

| $\bm{r}$ | IoU | SIoU |  $\bm{\alpha}$-IoU | NWD
---|---|---|---|---|---
$\bm{r}$ | 1.000 | 0.674 | 0.701 | 0.674 | 0.550
IoU | 0.674 | 1.000 | 0.892 | 0.997 | 0.474
SIoU | 0.701 | 0.892 | 1.000 | 0.892 | 0.576
$\bm{\alpha}$-IoU | 0.674 | 0.997 | 0.892 | 1.000 | 0.472
NWD | 0.550 | 0.474 | 0.576 | 0.472 | 1.000

Table 11: Kendall’s $\tau$ correlation between various criteria and human
rating $r$. For SIoU, $\gamma=0.2$ and $\kappa=64$, for $\alpha$-IoU,
$\alpha=3$.

##  Appendix E Few-Shot Experiments

In this appendix, we provide the implementation details for our FSOD
experiments. Some additional results with different FSOD methods are also
included in Tab. 12.

###  E.1 Implementation details

Our few-shot experiments are based on Cross-scale Query Support Alignment
(XQSA) [14], a recently proposed method for few-shot object detection. Except
for experiments from Tab. 12, all the few-shot experiments were done with
XQSA. This method is based on an attention mechanism and combines features
from support and query images to condition the detection of the selected
classes. This method leverages a ResNet-50 as a backbone with a 3-layers
Feature Pyramid Network on top. The detection head is based on FCOS and
detects independently objects from different classes. The training of all FSOD
methods employed in this paper is done episodically. For each episode, a
subset $\mathcal{C}_{ep}\subset\mathcal{C}$ of the classes is randomly sampled
($\lvert\mathcal{C}_{ep}\rvert=3$). Only the annotations of the episode
classes are used for training the model. A support set is sampled at each
episode containing $K$ examples for each episode class. A query set is also
sampled for each episode, containing 100 images per class. The query set is
used to compute the losses and update model’s weights.

The training is divided into base training and fine-tuning. During base
training, only base classes can be sampled
($\mathcal{C}_{ep}\subset\mathcal{C}_{base}$) and one example per class is
drawn for the support set ($K=1$). The optimization is done with SGD and a
learning rate of $1\text{\times}{10}^{-3}$ for 1000 episodes. During fine-
tuning, the backbone is frozen, the learning rate is divided by 10 and the
episode classes can be sampled from
$\mathcal{C}_{base}\cup\mathcal{C}_{novel}$, with two novel classes per
episode. Examples from novel classes are selected among the $K=10$ examples
sampled once before fine-tuning. The base/novel splits are selected randomly
before training. 3 classes are selected as novel classes for DOTA, 5 for DIOR
and Pascal VOC, and 20 for COCO. Note that for COCO, the split is not chosen
randomly, but instead the 20 novel classes are the ones that overlap with
Pascal VOC.

The evaluation is also conducted in an episodic manner. The performance is
averaged over multiple episodes each containing 500 examples for each class
and this is repeated 10 times with randomly sampled support sets. The query
and support examples are drawn from test set, thus the support examples are
different from the ones used during fine-tuning. This prevents overestimations
of the performance due to overfitting on the support examples.

###  E.2 Few-Shot Methods Comparison

To support the versatility of GSIoU, we also experiment with several few-shot
approaches. We select three existing FSOD techniques Feature Reweighting [15]
(FRW), Dual-Awarness Atention [6] (DANA) and Cross Scale Query Support
Alignment [14] (XQSA). We compare GIoU and GSIoU as regression losses. Tab. 12
regroups the results and shows consistent improvements with GSIoU over GIoU.
Except for DANA, GSIoU improves detection performance, especially for small
objects.

|  | Base classes |  | Novel Classes
---|---|---|---|---
| XQSA | All | S | M | L |  | All | S | M | L
FRW | w/ GIoU | 34.60 | 16.15 | 48.61 | 59.00 |  | 32.00 | 15.29 | 44.50 | 54.77
w/ GSIoU | 30.36 | 11.94 | 44.30 | 54.87 |  | 32.94 | 16.69 | 42.87 | 62.64
DANA | w/ GIoU | 48.09 | 27.34 | 66.06 | 68.00 |  | 44.49 | 30.10 | 52.24 | 74.40
w/ GSIoU | 50.10 | 32.19 | 65.46 | 67.77 |  | 41.40 | 21.07 | 54.80 | 75.23
XQSA | w/ GIoU | 45.30 | 26.94 | 61.17 | 63.00 |  | 41.03 | 24.01 | 52.13 | 69.78
w/ GSIoU | 43.42 | 22.14 | 61.19 | 66.02 |  | 45.88 | 34.83 | 51.26 | 70.78

Table 12: Performance comparison with three different FSOD methods: Feature
Reweighting [15] (FRW), Dual Awarness Attention [6] (DANA) and Cross-scale
Query Support Alignment [14] (XQSA), trained with GIoU and GSIoU. mAP is
reported with a 0.5 IoU threshold for small (S), medium (M), large (L) and all
objects.

##  Appendix F Evaluation with SIoU

In this appendix, we present the results from Sec. 5 but using SIoU as the
evaluation criterion. Specifically, instead of choosing an IoU threshold to
decide if a box is a positive or negative detection, an SIoU threshold is
employed. For the sake of comparison, we kept the same thresholds as in Tabs.
2, 3 and 4, _i.e_. 0.5 for Few-Shot methods and 0.5:0.95 for regular object
detection. The results are available in Tabs. 13, 15 and 14. The conclusions
from Sec. 5 still hold, and the superiority of GSIoU over other criteria is
clear. However, a few changes are noticeable. First, SIoU loss seems to
perform better than IoU. This is expected, the model is directly optimized to
satisfy this criterion. Then, when evaluated with SIoU, models trained with
NWD perform well. Indeed, NWD puts a lot of emphasis on size matching during
training, and less on position. Therefore, it is logical to observe better
performance compared to other losses when using SIoU as the evaluation
criterion.

One crucial point is that SIoU evaluation mostly changes the score for small
objects. SIoU behaves like IoU for large objects, therefore relatively small
changes are visible for medium and large objects. Overall, the scores are
higher than with IoU as the expected value of SIoU is higher than IoU. More
boxes are considered positive detection. But what is important to note is that
the gap between small and large objects performance is reduced and therefore
aligns better with human perception.

| Base classes |  | Novel Classes
---|---|---|---
Loss | All | S | M | L |  | All | S | M | L
IoU | 55.81 | 35.03 | 62.57 | 70.05 |  | 39.10 | 18.58 | 53.93 | 68.83
$\bm{\alpha}$-IoU | 53.05 | 20.60 | 61.05 | 72.41 |  | 41.93 | 20.99 | 55.74 | 76.79
SIoU | 59.77 | 36.38 | 67.29 | 70.06 |  | 49.51 | 31.06 | 62.53 | 77.24
NWD | 58.80 | 34.16 | 66.81 | 70.05 |  | 53.66 | 42.02 | 62.53 | 68.92
GIoU | 59.27 | 44.07 | 66.91 | 65.46 |  | 49.02 | 35.10 | 57.58 | 74.30
GSIoU | 59.32 | 35.32 | 66.29 | 69.03 |  | 57.70 | 46.77 | 65.56 | 73.67

Table 13: Few-shot performance comparison between several criteria: IoU,
$\alpha$-IoU, SIoU, NWD, GIoU and GSIoU trained on DOTA. mAP is reported with
a 0.5 SIoU threshold for small (S), medium (M), large (L), and all objects.

| DOTA | DIOR |
---|---|---|---
FCOS | All | S | M | L |  | All | S | M | L
w/ GIoU | 43.9 | 27.4 | 46.5 | 47.2 |  | 54.5 | 17.6 | 49.8 | 66.4
w/ GSIoU | 45.4 | 27.7 | 50.2 | 49.2 |  | 55.4 | 18.0 | 50.1 | 69.2

Table 14: Regular Object Detection performance on DOTA and DIOR datasets with
GIoU and GSIoU ($\gamma=-3$ and $\kappa=16$) losses. mAP is computed with
several SIoU thresholds (0.5 to 0.95) as it is commonly done in regular
detection.

|  | Base classes |  | Novel Classes
---|---|---|---|---
| XQSA | All | S | M | L |  | All | S | M | L
DOTA | w/ GIoU | 59.27 | 44.07 | 66.91 | 65.46 |  | 49.02 | 35.10 | 57.58 | 74.30
w/ GSIoU | 59.32 | 35.32 | 66.29 | 69.03 |  | 57.70 | 46.77 | 65.56 | 73.67
DIOR | w/ GIoU | 62.06 | 17.49 | 45.55 | 82.22 |  | 53.81 | 23.79 | 53.46 | 71.63
w/ GSIoU | 63.81 | 17.77 | 49.62 | 82.53 |  | 58.79 | 25.60 | 59.28 | 73.78
Pascal | w/ GIoU | 55.51 | 26.10 | 46.82 | 64.31 |  | 52.43 | 28.97 | 40.73 | 62.58
w/ GSIoU | 58.74 | 27.47 | 46.56 | 68.93 |  | 58.92 | 31.36 | 41.65 | 69.71
COCO | w/ GIoU | 21.46 | 12.77 | 24.79 | 31.86 |  | 29.21 | 17.36 | 27.62 | 40.05
w/ GSIoU | 21.97 | 12.80 | 25.72 | 32.35 |  | 29.94 | 18.87 | 29.93 | 40.47

Table 15: Few-shot performance on three datasets: DOTA, DIOR, Pascal VOC and
COCO. GIoU and GSIoU losses are compared. mAP is reported with a 0.5 SIoU
threshold and for various object sizes.