# Progressive Self-Distillation for Ground-to-Aerial Perception Knowledge
Transfer

Junjie Hu, Chenyou Fan, Mete Ozay, Hua Feng, Yuan Gao, and Tin Lun Lam, J.Hu,
H.Feng, Y.Gao and T.L.Lam are with the Shenzhen Institute of Artificial
Intelligence and Robotics for Society (AIRs), Shenzhen, China. E-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>C.Fan
is with the School of Artificial Intelligence, South China Normal University,
China. E-mail<EMAIL_ADDRESS>M.Ozay is with the Samsung Research,
UK. E-mail<EMAIL_ADDRESS>T.L.Lam is also with the School of Science and
Engineering, the Chinese University of Hong Kong, Shenzhen, China.
<EMAIL_ADDRESS>T.L.Lam is the corresponding author.

###### Abstract

We study a practical yet hasn’t been explored problem: how a drone can
perceive in an environment from different flight heights. Unlike autonomous
driving, where the perception is always conducted from a ground viewpoint, a
flying drone may flexibly change its flight height due to specific tasks,
requiring the capability for viewpoint invariant perception. Tackling the such
problem with supervised learning would entail tremendous costs for data
annotation of different flying heights. On the other hand, current semi-
supervised learning methods are not effective under viewpoint differences. In
this paper, we introduce the ground-to-aerial perception knowledge transfer
and propose a progressive semi-supervised learning framework that enables
drone perception using only labeled data of ground viewpoint and unlabeled
data of flying viewpoints. Our framework has four core components: i) a dense
viewpoint sampling strategy that splits the range of vertical flight height
into a set of small pieces with evenly-distributed intervals, ii) nearest
neighbor pseudo-labeling that infers labels of the nearest neighbor viewpoint
with a model learned on the preceding viewpoint, iii) MixView that generates
augmented images among different viewpoints to alleviate viewpoint
differences, and iv) a progressive distillation strategy to gradually learn
until reaching the maximum flying height. We collect a synthesized and a real-
world dataset, and we perform extensive experimental analyses to show that our
method yields $25.7\%$ and $16.9\%$ relative accuracy improvement for the
synthesized dataset and the real world. Code and datasets are available on
https://github.com/FreeformRobotics/Progressive-Self-Distillation-for-Ground-
to-Aerial-Perception-Knowledge-Transfer.

##  I Introduction

ROBOT perception plays a critical role in understanding interactive
environments and providing substantial instruments for subsequent task
execution. To date, most autonomous perception systems have focused on
unmanned ground vehicles (UGVs) [27, 48, 10], e.g. autonomous driving cars.
However, only a small portion of works studied drone perception due to
hardness of the task and data collection.

Modern paradigm of machine learning introduces a data-driven deep learning
based solution to address perception problems which has been validated its
effectiveness on various tasks, e.g., semantic segmentation [29, 24], object
detection [34, 4], depth regression [15, 14, 38], classification [13, 16], and
therefore provides a potential solution to autonomous driving. Recent studies
on drone perception [47, 45, 2] have attempted to transfer this paradigm from
cars to drones, i.e., first to prepare a labeled training set of drones and
then to learn a deep network on a specific task.

On the other hand, prior works scarcely considered the essential difference of
the perception between drones and cars. In many works [9, 12], the perception
is assumed to be conducted at only a certain flying height. However, in
practical scenarios, as shown in Fig. 1, a drone may appear at an arbitrary
flight height due to its specifically allocated task. Given the maximum flight
height $h$, we argue that an essential requirement of drone perception is the
capability to precisely perceive from different flight heights in the range of
$[0,h]$ meters.

Figure 1: An example of a trajectory comparison between a drone and a car. One
essential difference is that a drone is capable of optionally changing its
flight height at time instance $\mathbf{t}_{j},j=1,2,3,4$.

It is, however, hard to tackle the above problem in a fully supervised
learning manner. We may need to prepare even dozens of times more training
data to obtain performance comparable to that of an autonomous driving car. In
reality, data acquisition of ground truths is both costly and time-consuming.
For instance, robot vision tasks, including object detection, semantic
segmentation, and classification, require annotation of RGB images to be
manual labeling, which is an unavoidable obstacle for developing data-driven
approaches.

Figure 2: Examples of RGB images captured in a simulated environment at
different heights.

In this paper, we introduce the concept of ground-to-aerial (GoA) perception
knowledge transfer that transfers the perception knowledge from a UGV for
ground viewpoint perception to our drone perception tasks, without additional
data annotation at different flying heights other than ground viewpoint. To
this end, we consider a semi-supervised learning (SSL) approach to enable
drone perception where only data collected from the ground viewpoint are
labeled, and all images captured from flying viewpoints are unlabeled. The
fundamental challenge is the inaccuracy caused by viewpoint differences among
different flight heights. Fig. 2 shows several images captured at three
vertically different viewpoints, and they demonstrate clear viewpoint
differences.

To overcome the above difficulty, we first propose a dense viewpoint sampling
strategy that splits the vertical flying range to heights with evenly-
distributed intervals, and at each height, we sample data from that viewpoint.
Even though the performance gap of large viewpoint differences is significant,
we observe that the nearest viewpoint yields similar performance. Hence, we
propose the nearest neighbor pseudo-labeling that predicts data of the
viewpoint $h_{i}$ with a network learned on its nearest preceding viewpoint
$h_{i-1}$. In addition, we propose MixView which mixes data from different
viewpoints to generate augmented images, which is beneficial for relieving the
issue of viewpoint change among data samples. We further propose a progressive
SSL framework to gradually learn a model from the ground viewpoint until
reaching the maximum flight height.

To facilitate this work, we create both a synthesized dataset collected from
AirSim [37] and a real-world dataset captured from a city street. Both
datasets include images of different flying heights. Note that, in this paper,
we technically consider the vertical viewpoint difference which is exclusive
for drones, rather than the horizontal viewpoint difference. We study drone
perception on semantic segmentation which is a fundamental task of robot
perception. We evaluate our method on those two datasets and provide both
quantitative and qualitative analyses.

In summary, our contributions include:

  * •

To the best of our knowledge, the present study is the first attempt to tackle
drone perception under large viewpoint change caused by different flight
heights. We aim at enabling drone perception to be precise from different
flight heights.

  * •

A novel semi-supervised learning framework that solves ground-to-aerial
perception knowledge distillation with labeled images captured from the ground
viewpoint with unlabeled images captured from flight viewpoints.

  * •

Two proposed methods, i) the nearest neighbor pseudo-labeling and ii) MixView,
to overcome the significant viewpoint change among different flying heights.

  * •

Two elaborately crafted datasets for drone perception on semantic
segmentation, including i) a synthesized one collected from AirSim with fixed
lighting conditions and objects, and ii) a real-world one taken from a city
street with changing lighting conditions and dynamic objects.

The remainder of this paper is organized as follows. In Sec. II, we discuss
the necessary background and related studies. We describe our progressive
semi-supervised learning framework in Sec. III. We then present two self-
collected datasets used to verify our method in Sec. IV. Finally, we provide
extensive numerical evaluations in Sec. V.

##  II Related Work

###  II-A Drone Perception

Consumer drones are usually equipped with only optical cameras, and thus
enabling successful robotic perception from RGB images captured by these
cameras will greatly boost the further development of autonomous drones.
Similar to modern studies on self-driving cars, prior works tackle it as a
classic data-driven learning problem, i.e., collecting a set of training data
and training a deep neural network for perception. Early works addressed
indoor gate detection for drone navigation with a convolutional neural network
[17], or aimed at directly learning navigation in GPS-denied indoor corridor
arenas [32]. The same tendency is also observed in object detection [39, 28,
46], and semantic segmentation [26, 11].

One fundamental difference of drone perception from autonomous driving is that
a drone may fly at an arbitrary height, while self-driving cars only perform
perception from a ground viewpoint. Drone perception is, therefore, more
challenging, as it has to be accurate from different flying heights. A
straightforward solution is to label all data from different viewpoints, and
solve it in a purely supervised learning setup. However, it is not practical
as all efforts will be put into data labeling. Therefore, in this paper, we
propose a semi-supervised approach that enables drone perception under large
viewpoint change.

Figure 3: The diagram of our progressive semi-supervised learning framework.
We propose to transfer the knowledge from a UGV to enable drone perception
while assuming only data from the ground viewpoint is labeled and all data
from other viewpoints are not labeled. Here, we demonstrate an application
scenario of three viewpoints, i.e., one ground viewpoint $h_{1}$ and two
flying viewpoints $h_{2}$ and $h_{3}$, and $h_{3}>h_{2}>h_{1}$ in flight
height. $x_{1},x_{2},x_{3}$ are images taken from each viewpoint,
respectively. $y_{1}$ is label of $x_{1}$, and $(x_{1},y_{1})$ are employed to
train the UGV. To enable drone perception at $h_{2}$, we propose the nearest
neighbor pseudo-labeling to infer labels of $x_{2}$ with the UGV. Moreover, we
propose MixView to mix data of different viewpoints to generate augmented
samples. The labeled samples, as well as the augmented samples, are then used
to learn at $h_{2}$. The same learning strategy is then utilized to learn at
$h_{3}$.

###  II-B Multi-Robot Perception

Several prior works studied homogeneous multi-robot collaborative semantic
segmentation. In [22, 23], online collaboration methods were proposed to boost
the performance of a single robot when its captured images suffer from
deterioration. Recently, a few-shot learning based approach [8] was proposed
to solve multi-robot perception in data-scarce scenarios. However, the
viewpoint difference between homogeneous robots is assumed to be small in
these methods.

In this work, we introduce the ground-to-aerial perception distillation. To
the best of our knowledge, we are the first to demonstrate the possibility of
drone perception by purely learning from a UGV. Unlike [23, 22, 8], our method
can be applied to a heterogeneous multi-robot system consisting of a UGV and a
UAV for perception in fully 3D spaces. Besides, unlike prior works on multi-
robot systems [41, 20] that assume multiple sensors are available, we target
more hard cases where only a visual camera is mounted on a UAV.

###  II-C Semi-supervised Learning

Semi-supervised learning (SSL) is an active research field and has been
extensively studied in machine learning. It seeks to learn a highly
discriminative model from both labeled and unlabeled samples [42, 40]. Prior
studies have shown promising results using deep learning. Here, we mainly
introduce three types of basic approaches. (i) A generative learning approach
proposed to generate more data from real data distribution with generative
adversarial networks [30, 3] or variational auto-encoders [18]. (ii) Pseudo-
Labeling based methods firstly train a deep model on labeled data in a
supervised manner and use it to predict pseudo-labels from unlabeled data [35,
25, 21, 5]. (iii) Furthermore, data augmentation methods such as MixUp [44]
and MixMatch [1], propose to blend two images and their respective labels at
pixel-level, demonstrating superior performance. Instead of directly mixing
pixels, CutMix [43] and ClassMix [31] mix object-label information, making
them applicable to semantic segmentation.

Despite all the efforts made by prior works, it is, however, difficult to
apply SSL for our drone perception task. Since ground viewpoint and drone
flying viewpoints have significant differences, pseudo-labeling methods tend
to generate wrong pseudo-labels. Similarly, data augmentation methods cannot
generate effective training pairs of images and semantic maps. Besides, due to
the lack of labeled data of flying viewpoints, generative approaches cannot
produce realistic flying data. In this paper, we propose the nearest-neighbor
pseudo-labeling and MixView to deal with the viewpoint difference.

##  III Technical Approach

In this section, we present our progressive self-distillation framework for
SSL-based drone perception. Our goal is to train a viewpoint invariant model
that enables a drone to perceive precisely from any flying height in low
altitude111In high altitudes, objects eventually tend to be extremely small
and deformed in a camera coordinate, and viewpoint difference hardly further
degrades the perception performance. In this case, supervised learning is
considered the only effective method.. Instead of assuming a simple case where
labeled data is evenly distributed in each viewpoint, we consider a
challenging scenario where only data collected from the ground viewpoint are
labeled and all data captured from other viewpoints are unlabeled. We pose
this problem as a heterogeneous multi-robot knowledge distillation problem and
perform semi-supervised ground-to-aerial knowledge transfer by distilling the
perception ability from a UGV to a drone.

Fig. 3 demonstrates the overall information flow in our progressive SSL
framework. The labeled samples $(x_{1},y_{1})$ of ground viewpoint are
utilized to train a network for UGV perception. The network is denoted by
$N_{1}$ and will be a baseline used for comparison throughout the paper. We
then predict pseudo-labels of data from the nearest neighbor viewpoint $h_{2}$
with $N_{1}$. The resulting pseudo-labels sever as supervision for training at
$h_{2}$. In addition, we propose MixView that mixes data from $h_{1}$ and
$h_{2}$ to generate augmented training samples. The same learning strategy is
then applied to $h_{3}$. After $n$ progressive applications on heights $h_{1}$
to $h_{n}$, we will obtain the final model $N_{n}$ that is able to handle any
image taken from $h_{1}$ to $h_{n}$.

In the rest of this section, we provide the details of each important
component in our framework. Specifically, we discuss viewpoint sampling when
applying our method in Sec. III-A. In Sec. III-B, we describe how to take
advantage of the nearest neighbor viewpoint and apply pseudo-labeling to it.
In Sec. III-C, we further propose MixView to better overcome viewpoint
differences. In Sec. III-D, we present the progressive self-distillation to
obtain the final model.

###  III-A Dense Sampling of Viewpoint

One essential key to achieving a superior performance of our method is the
strategy of dense sampling of viewpoints. As we handle significant viewpoint
differences by gradually leveraging intermediate viewpoints, the intermediate
viewpoints play a role in viewpoint assistance. Hence, the more assistant
viewpoints we use, the better accuracy we will have.

Formally, given the maximum flight height $h$, we propose to sample flight
data with a uniform interval $h/n$ by

$h_{i}=h/n\times i$ (1)

where $i\in[1,n]$. Eq. 1 divides the flight range $[0,h]$ into
$[h_{1},h_{2},\ldots,h_{n}]$, where $h_{1}<h_{2}<\ldots<h_{n}$.

We assume only images of the ground viewpoint are labeled, while those of the
other viewpoints are unlabeled. For simplicity, $\mathcal{X}_{1}$ denotes a
labeled set of the ground viewpoint, while $\mathcal{U}_{i}$ and
$\mathcal{X}_{i}$ denote the unlabeled set and its pseudo-labeled version of
the flying viewpoint $h_{i}$, respectively, throughout the paper.

###  III-B Nearest Neighbor Pseudo-Labeling

Fig. 4 shows the accuracy of $N_{1}$ for various flight heights. It is obvious
that the viewpoint difference deteriorates UGV’s performance. On the other
hand, close observation shows a small performance gap between two neighbor
viewpoints, e.g., flight height of 2 meters and 3 meters. It is, therefore,
reasonable to exploit SSL between a viewpoint and its nearest neighbor. We
propose the nearest neighbor pseudo-labeling that applies pseudo-labeling to
the nearest viewpoint.

Let $(x_{1},y_{1})$ be a pair at $h_{1}$, and $x_{2}$ be an image at $h_{2}$,
where $(x_{1},y_{1})\in\mathcal{X}_{1}$ and $x_{2}\in\mathcal{U}_{2}$. A
straightforward way to infer a label at $h_{2}$ is utilizing $N_{1}$ learned
at $h_{1}$ by

$\hat{y}_{2}=N_{1}(x_{2})$ (2)

where $\hat{y}_{i}$ is the one-hot label of $x_{i},\forall i,$ called “pseudo-
label” as it provides an approximation to its correct semantic map.

Figure 4: The mean IoU accuracy of a network model trained on data collected
from the ground viewpoint for different flight heights.

Eq. (2) allows an implicit entropy minimization to improve SSL and is
effective due to small viewpoint differences. Then, we can learn a better
model at $h_{2}$ by mixing data from the nearest neighbors of $h_{1}$ and
$h_{2}$. The model $N_{2}$ is learned through minimizing the objective by

$\underset{N_{2}}{\min}\
\frac{1}{|\mathcal{X}_{1}|}\\!\sum_{(x_{1},y_{1})\\!\in\mathcal{X}_{1}}\\!\mathcal{H}(y_{1},N_{2}(x_{1}))\\!+\frac{1}{|\mathcal{U}_{2}|}\sum_{x_{2}\in\mathcal{U}_{2}}\mathcal{H}(\hat{y}_{2},N_{2}(x_{2}))$
(3)

where $\mathcal{H}$ is the cross entropy function. The pseudo-labels are
obtained at $h_{i}$ by

$\hat{y}_{i}=N_{i-1}(x_{i})$ (4)

where $N_{i-1}$ denotes a model trained at $h_{i-1}$. To avoid confusion, we
simplify our notation by stating that the ground truths $y_{1}$ at the ground
viewpoint are equivalent to $\hat{y}_{1}$ for the rest of the paper. Then, the
optimization objective for learning $N_{i}$ can be written as

$\displaystyle\underset{N_{i}}{\min}\
\frac{1}{|\Psi_{i}|}\sum_{(x,\hat{y})\in\Psi_{i}}\mathcal{H}(\hat{y},N_{i}(x))$
(5) $\displaystyle\text{s.t.}\ \ \ \mathcal{X}_{i}=N_{i-1}(\mathcal{U}_{i})$
$\displaystyle\qquad\Psi_{i}=\bigcup_{k=1}^{i}\mathcal{X}_{k}$

###  III-C MixView

Ideally, an object, e.g., a car, should be correctly recognized from different
flight heights despite its appearance/shape being different among viewpoints.
This can be readily implemented if we have ground truths. In our setting, we
propose MixView to ensure this viewpoint invariance. We argue that an object
should also be correctly recognized if we manually move it to another
viewpoint. With this intuition, we apply MixView to mix data samples among
different viewpoints at the object level to generate augmented data.

MixView can be seen as a variant of ClassMix [31] to handle SSL under
viewpoint difference. Instead of augmenting only the unlabeled set, we make an
improvement of the original ClassMix where we blend the labeled set and
unlabeled set to generate viewpoint robust images. We apply MixView when
training $N_{i}$ in an online fashion, and the model gradually learns to
recognize objects under viewpoint difference. We formulate the MixView by

$\begin{gathered}\vspace{1mm}(x^{\prime}_{i},y^{\prime}_{i})=MixView\left((x,\hat{y}),(x_{i},\tilde{y}_{i}))\right)\\\
x^{\prime}_{i}=m\odot x_{i}+(1-m)\odot x\\\
y^{\prime}_{i}=m\odot\tilde{y}_{i}+(1-m)\odot\hat{y}\end{gathered}$ (6)

where $(x,\hat{y})\in\Psi_{i}$, $\tilde{y}_{i}=N_{i}(x_{i})$, $m$ is a binary
mask that randomly exchanges half of the classes between $x_{i}$ and $x$ to
generate a new image $x^{\prime}_{i}$. $\odot$ denotes element-wise
multiplication.

The “manifold” assumption of SSL assumes that the data space is composed of
multiple lower-dimensional manifolds, and data points lying on the same
manifold should have the same label [40]. Viewpoint discrepancy causes
misclassification in the data space. To solve this problem, MixView imposes a
regularization to viewpoint differences and helps to move misclassified data
points to the correct manifold.

###  III-D Progressive Self-Distillation

We formally describe our progressive learning strategy that integrates the
nearest neighbor pseudo-labeling and MixView. Specifically, given a labeled
set $\mathcal{X}_{1}$ and trained model $N_{1}$ at $h_{1}$, as well as a set
of unlabeled samples $\mathcal{U}_{2},\mathcal{U}_{3},...,\mathcal{U}_{n}$
taken from $h_{2},h_{3},...,h_{n}$, respectively, we will progressively
transfer the perception ability from $N_{1}$ to various flying heights.

We train the model $N_{i}$ with the labeled set
${\Psi_{i}=\bigcup_{k=1}^{i}\mathcal{X}_{k}}$ and the augmented data of
$\mathcal{U}_{i}$. We train the final model $N_{n}$ looping over all
viewpoints by

$\displaystyle\underset{N_{n}}{\min}\ \mathcal{L}$ (7)
$\displaystyle\text{s.t.}\ \ \ \mathcal{X}_{i}=N_{i-1}(\mathcal{U}_{i})$
$\displaystyle\qquad\Psi_{i}=\bigcup_{k=1}^{i}\mathcal{X}_{k}$
$\displaystyle\qquad(x^{\prime}_{i},y^{\prime}_{i})=MixView\left((x,\hat{y}),(x_{i},\tilde{y}_{i}))\right),$

where

$\mathcal{L}\\!=\\!\sum_{i=2}^{n}\\!\Big{(}\\!\frac{1}{|\Psi_{i}|}\\!\sum_{(x,\hat{y})\\!\in\Psi_{i}}\\!\mathcal{H}(\hat{y},N_{i}(x))\\!+\lambda\
\frac{1}{|\mathcal{U}_{i}|}\\!\sum_{x^{\prime}\\!\in\mathcal{U}_{i}}\mathcal{H}(y^{\prime},N_{i}(x^{\prime}))\Big{)}$
(8)

and $\lambda\in[0,1]$ is a hyperparameter varying from 0 to 1 during the
optimization.

Algorithm 1 Algorithm of our progressive SSL framework.

1:$\mathcal{X}_{1}$: A labeled set of image pairs $(x_{1},y_{1})$ at the
ground viewpoint $h_{1}$; $\mathcal{U}_{i}$: An unlabeled set from the
viewpoint $h_{i}$, $\forall i=2,3,\ldots,n$; $N_{1}$: The baseline model for
ground viewpoint perception.

2:Initial learning rate: $0.01$, weight decay: $1e^{-4}$, number of training
steps: iterations.

3:$N_{n}$: The final model trained on images captured from viewpoints $h_{1}$
to $h_{n}$.

4:Freeze $N_{1}$;

5:$\triangleright$ COMMENT % Progressive Learning %

6:for $i=2$ to $n$ do

7: $\triangleright$ COMMENT % Applying the nearest neighbor pseudo-labeling %

8: $\mathcal{X}_{i}=N_{i-1}(\mathcal{U}_{i})$

9: $\Psi_{i}=\bigcup_{k=1}^{i}\mathcal{X}_{k}$

10: Initialize $N_{i}$;

11: for $j$ = 1 to $iterations$ do

12: Set gradients of $N_{i}$ to 0;

13: Select $(x,\hat{y})$ from $\Psi_{i}$ and $x_{i}$ from $\mathcal{U}_{i}$;

14: $\tilde{y}_{i}=N_{i}(x_{i})$;

15: $\triangleright$ COMMENT % Applying MixView %

16: $(x^{\prime}_{i},y^{\prime}_{i})$ =
MixView$\left((x,\hat{y}),(x_{i},\tilde{y}_{i}))\right)$;

17: Calculate the loss $\mathcal{L}$ with Eq. (8);

18: Backpropagate $\mathcal{L}$;

19: Update $N_{i}$;

20: end for

21: $\triangleright$ COMMENT % Updating $\mathcal{X}_{i}$ %

22: $\mathcal{X}_{i}=N_{i}(\mathcal{U}_{i})$

23:end for

Figure 5: Examples of selected images from our datasets. (a) shows nine images
of the flight heights from 1 meter to 9 meters in AirSim-Drone, and similarly,
(b) shows nine images of the flight heights from 1 meter to 9 meters in AIRs-
Street.

##  IV Datasets for Drone Perception

In this section, we introduce our datasets for the evaluation of the proposed
method. As there is no prior dataset that can be applied to our problem, to
better verify the performance of the proposed method and inspire more future
explorations, we create both a synthesized AirSim-Drone and a real-world AIRs-
Street, respectively. The former is collected from the Microsoft AirSim and is
used for evaluating our method in an ideal case, and the latter is captured in
a city street scenario and is used to quantify our method in the real world.
We will present the details of each dataset for the rest of this section.

###  IV-A AirSim-Drone

We synthesize the AirSim-Drone from the AirSim Neighborhood environment where
we capture the data at the ground viewpoint with a car, and flight data with a
drone. The flight route of the drone follows a predefined map while objects
and lighting conditions such as illumination and weather, are all fixed. We
collect the flight data while varying flight height to 2, 3, 4, 5, 6, 7, 8, 9,
and 10 meters, respectively. Since we can stably and accurately control the
drone in this simulated environment, the viewpoint discrepancy is only derived
from the flying height. In addition, we collect a test sequence from a
different area to the above sequences. We fly the drone at random flight
heights. The test sequence is called “uav_random” and is used to evaluate the
generalization performance.

There are 9 semantic attributes in total, including “Plant”, “Building”,
“Road”, “Sky”, “Car”, “Ground”, “Fence”, “Pole”, and “Others”. Each image has
a resolution of $768\times 432$. Detailed information of the dataset,
including the flying height, number of images, and labeled semantic maps are
given in Table I. Note that the start and end time of video recording have to
be manually determined, and thus the number of captured images in each
sequence is slightly different. Since semantic maps can be readily obtained
from the simulator, we have ground truth semantic annotations for all RGB
images. Only ground truths of the ground viewpoint are used for training, and
other viewpoints are used for validation.

###  IV-B AIRs-Street

AIRs-Street is a real-world dataset in which images are captured in a street
scenario. To precisely measure the distance from the ground, we use an
advanced DJI Phantom Pro 4 which is equipped with a 3D TOF for data
acquisition. The drone has to be manually controlled to maintain its flying
height following a predefined route. Thus, there are some unavoidable
deviations in terms of flying height. We carefully fly the drone with a fairly
slow forward speed, such that the deviation of the flight height is guaranteed
within $\pm 0.5$ meters. Similar to the AirSim-Drone, we take images of
different flight heights from 1 meter to 9 meters with an interval of 1 meter,
and we have 9 sequences in total. The resolution of the captured images is
$1920\times 1080$.

Due to the difference in flying speed, the number of captured images is also
different among viewpoints. Despite that, all sequences have the same start
point and endpoint. For the sequence of the ground viewpoint, we uniformly
select 117 frames to annotate and use for training. For each of the flight
sequences, we uniformly select 25 frames over the video length to annotate and
use them for evaluation 222Owing to lack of additional labeled images, we
directly use these labeled flight data for evaluation.. We manually annotate
the pixel-wise attribute labels. In our dataset, there are 13 semantic
classes, including “Plant”, “Building”, “Road”, “Sky”, “Car”, “Sidewalk”,
“Pedestrian”, “Motorcycle”, “Wall”, “Fence”, “Traffic Sign”, “Traffic Light”,
and “Others”. More detailed information is given in Table II.

TABLE I: The detailed information of our AirSim-Drone dataset. Sequences |  Flight height |  All samples |  Labeled samples
---|---|---|---
car01 |  1 |  1947 |  1947
uav02 |  2 |  1057 |  1057
uav03 |  3 |  1057 |  1057
uav04 |  4 |  1043 |  1043
uav05 |  5 |  1046 |  1046
uav06 |  6 |  1047 |  1047
uav07 |  7 |  1048 |  1048
uav08 |  8 |  1048 |  1048
uav09 |  9 |  1037 |  1037
uav10 |  10 |  1037 |  1037
uav_random |  2 to 10 |  1143 |  1143
TABLE II: The detailed information of our AIRs-Street dataset. Sequences |  Flight height |  All samples |  Labeled samples
---|---|---|---
car01 |  1 |  770 |  117
uav02 |  2 |  750 |  25
uav03 |  3 |  717 |  25
uav04 |  4 |  548 |  25
uav05 |  5 |  591 |  25
uav06 |  6 |  475 |  25
uav07 |  7 |  428 |  25
uav08 |  8 |  657 |  25
uav09 |  9 |  623 |  25
TABLE III: The mean IoU accuracy of our approach and other methods on the AirSim-Drone dataset. Methods |  uav02 |  uav03 |  uav04 |  uav05 |  uav06 |  uav07 |  uav08 |  uav09 |  uav10 |  mean, |  std
---|---|---|---|---|---|---|---|---|---|---|---
Ground-only |  0.672 |  0.616 |  0.561 |  0.519 |  0.479 |  0.446 |  0.418 |  0.388 |  0.365 |  0.496 |  0.105
Pseudo-Labeling [19] |  0.639 |  0.578 |  0.530 |  0.485 |  0.444 |  0.404 |  0.369 |  0.337 |  0.323 |  0.457 |  0.011
ClassMix [31] |  0.209 |  0.197 |  0.190 |  0.183 |  0.180 |  0.178 |  0.176 |  0.174 |  0.173 |  0.184 |  0.011
Ours |  0.680 |  0.650 |  0.625 |  0.609 |  0.594 |  0.578 |  0.564 |  0.551 |  0.539 |  0.599 |  0.047

##  V Experimental Evaluation

10m |  |  |  |  |  |
---|---|---|---|---|---|---
9m |  |  |  |  |  |
8m |  |  |  |  |  |
7m |  |  |  |  |  |
6m |  |  |  |  |  |
5m |  |  |  |  |  |
4m |  |  |  |  |  |
3m |  |  |  |  |  |
2m |  |  |  |  |  |
|  (a) Images |  (b) Ground Truths |  (c) Ground-only |  (d) Pseudo-Labeling [19] |  (e) ClassMix [31] |  (f) Ours
Figure 6: Qualitative comparisons between our method and other approaches on
the AirSim-Drone dataset. We show the results for different flight heights
from 2 meters to 10 meters. We draw red boxes on the Ground truth and results
of Ground-only and Ours at 10m for better visualization.

###  V-A Implementation Details

##### Implementation of Our Method

Due to the limitation of GPU memory, we change the original image resolution
from $768\times 432$ to $384\times 216$ for AirSim-Drone, and from $1920\times
1080$ to $480\times 270$ for AIRs-Street, respectively. For the network of
semantic segmentation, we use the pretrained DeepLabV3+ [6] on the CityScape
dataset [7]. We modify the output layers to fit the semantic categories of our
datasets. We train $N$ for 500 epochs for each viewpoint according to
Algorithm 1 with an NVIDIA GeForce RTX 2080 Ti. We use the SGD optimizer [36]
with a learning rate of 0.01 and apply polynomial learning rate decay during
training. We conduct all the experiments using PyTorch [33].

##### Baseline Methods

Since the perception capability of the drone is distilled from the UGV, we
consider the trained model at the ground viewpoint as a baseline and name it
as Ground-only. Moreover, we additionally evaluate two previous methods of SSL
on the under-explored problem for more fair comparisons. The first is Pseudo-
Labeling [19] which is a classical approach for SSL, and the second is
ClassMix [31], the current state-of-the-art for SSL-based semantic
segmentation. Entropy minimization is employed to identify pseudo-labels from
unlabeled data. Using these classical two methods as representatives, we will
show through experiments that existing semi-supervised learning methods cannot
cope with large viewpoint differences.

###  V-B Evaluation Metrics

For a fair comparison of semantic segmentation, we use the metrics that were
frequently employed in previous studies. We calculate the mean intersection-
over-union (IoU) value. The mean IoU metric, also referred to as the Jaccard
index, is essentially a method to quantify the percentage of overlap between
the target mask and the prediction output. It is calculated by

$IoU(y,\hat{y})=\sum_{k=1}^{c}\frac{{|M(\hat{y}_{k})\cap
M(y_{k})|}}{{|M(\hat{y}_{k})\cup M(y_{k})|}}$ (9)

where $c$ is the number of categories, $M(y_{k})$ and $M(\hat{y}_{k})$ denotes
a set of elements of the binary masks of $y$ and $\hat{y}$ on class $k$,
respectively.

In addition to the mean IoU accuracy, we also quantify statistics, i.e., the
mean and standard deviation (std) of the results. Ideally, we expect to obtain
a high mean value and a low std, i.e., the model can achieve equally accurate
performance for different flight heights. We consider a method to be a failure
if it underperforms the trained model at the ground viewpoint, i.e, Ground-
only.

###  V-C Quantitative Evaluation

####  V-C1 Results on AirSim-Drone

The results of different approaches are given in Table III. A close look at
Ground-only shows a tendency of performance degradation from uav02 to uav10,
and we observe a 45.7% relative accuracy drop caused by the significant
viewpoint difference.

Moreover, both Pseudo-Labeling and ClassMix failed on the task, as they are
even worse than Ground-only. Pseudo-Labeling demonstrates a lower accuracy
with a mean of 0.457. Although ClassMix is specially tailored to boost
semantic segmentation, it significantly deteriorated due to the viewpoint
difference. We are not surprised that the previous methods of SSL failed.
Intuitively, since clear viewpoint differences exist in data samples, they
could not correctly predict labels from unlabeled data.

On the other hand, our method yields promising results for each sequence.
Although we can also observe a tendency for performance degradation, the
accuracy drop is only 20.7% from uav02 to uav10. Compared with the original
drop of 45.7%, the improvement brought by our method is significant. We
obtained the mean of 0.599 and the std of 0.047, which shows 20.8% and 55.2%
improvement from Ground-only of 0.496 and 0.105, respectively.

Fig. 6 shows the estimated semantic maps provided by different methods at
different flying heights. Clearly, ClassMix yielded inaccurate predictions,
and it failed on all sequences. It tends to ignore small objects, and the
estimated semantic maps have only three classes, i.e., Plant, Road, and Sky.
When observing the results of Ground-only and Pseudo-Labeling, we can see how
viewpoint differences mislead the semantic segmentation, e.g., in the left
bottom areas of images, false positives gradually increase from 2 meters to 10
meters, while our method shows consistently correct results.

Table IV shows the generalization performance on the test sequence. Although
the accuracy is slightly lower than the results given in Table V, i.e., 0.599,
our method achieved a 25.7% performance boost from Ground-only, and other
methods yielded worse results.

####  V-C2 Results on AIRs-Street

The real-world AIRs-Street dataset is more challenging than the synthesized
AirSim-Drone dataset, as it has more semantic attributes, few labeled images
at the ground viewpoint, and dynamic objects, e.g., moving cars, and non-rigid
objects, e.g., pedestrians, as well as different lighting conditions due to
the change of the sunlight intensity. Thus, the performance of drone
perception will also be deteriorated by the above difficulties other than the
viewpoint difference.

Table V shows the numerical results for different methods. The results show
that there is a 39.2% accuracy drop of Ground-only from uav02 to uav09. Also,
ClassMix yielded inferior results for each sequence than Ground-only, and
Pseudo-labeling obtained the same mean accuracy as Ground-only. Our method
outperformed Ground-only from uav02 to uav09. Similar to the results on the
AirSim-Drone, the accuracy improvement tends to increase from lower flying
height to higher flying height, e.g., there is only a 6.0% boost for uav03 and
a 32.2% boost for uav09. Overall, our method obtained a mean of 0.561 and a
std of 0.003, which shows 16.9% and 66.7% performance improvement from Ground-
only of 0.480 and 0.009, respectively.

Fig. 7 shows the qualitative comparisons between our method and other
approaches. The qualitative results agree well with those on AirSim-Drone.
ClassMix demonstrates inaccurate predictions for all flying viewpoints.
Ground-only and Pseudo-Labeling gradually deteriorated from a flying height of
2 meters to 9 meters. As seen, false positives tend to increase for the two
methods, and our method can correctly predict pixel attributes better, such as
the cars.

TABLE IV: Generalization performance on the AirSim-Drone test set. Methods | Accuracy
---|---
Ground-only | 0.417
Pseudo-Labeling [19] | 0.400 (– 4.1%)
ClassMix [31] | 0.179 (– 57.1%)
Ours |  0.524 (+25.7%)
TABLE V: The mean IoU accuracy of our approach and other methods on the AIRs-Street dataset. Methods |  uav02 |  uav03 |  uav04 |  uav05 |  uav06 |  uav07 |  uav08 |  uav09 |  mean |  std
---|---|---|---|---|---|---|---|---|---|---
Ground-only |  0.607 |  0.618 |  0.563 |  0.458 |  0.413 |  0.408 |  0.402 |  0.369 |  0.480 |  0.009
Pseudo-Labeling [19] |  0.611 |  0.623 |  0.546 |  0.467 |  0.406 |  0.421 |  0.386 |  0.381 |  0.480 |  0.009
ClassMix [31] |  0.282 |  0.260 |  0.244 |  0.238 |  0.233 |  0.233 |  0.229 |  0.230 |  0.244 |  0.000
Ours |  0.610 |  0.655 |  0.624 |  0.539 |  0.549 |  0.528 |  0.492 |  0.488 |  0.561 |  0.003
9m |  |  |  |  |  |
---|---|---|---|---|---|---
8m |  |  |  |  |  |
7m |  |  |  |  |  |
6m |  |  |  |  |  |
5m |  |  |  |  |  |
4m |  |  |  |  |  |
3m |  |  |  |  |  |
2m |  |  |  |  |  |
|  (a) Images |  (b) Ground Truths |  (c) Ground-only |  (d) Pseudo-Labeling [19] |  (e) ClassMix [31] |  (f) Ours
Figure 7: Qualitative comparisons between our method and other approaches on
the AIRs-Street dataset. We show the results for different flight heights from
2 meters to 9 meters. We draw red boxes on the Ground truth and results of
Ground-only and Ours at 8m and 9m for better visualization.

####  V-C3 Adaptability to Various Flight Heights

We set different height ranges starting from $h_{2}$ to maximum flight height
$h_{i}$, in which $h_{i}$ ranges from $2,3,\ldots,10$ for AirSim-Drone and
$2,3,\ldots,9$ for AIRs-Street, respectively. Then we evaluate our methods and
baselines at these ranges.

Fig. 8 shows the results on the two datasets. As seen, ClassMix demonstrated
unstable behavior on AirSim-Drone and still had the worst performance on the
two datasets. Our method outperformed the other two approaches in all
different settings, except for the maximum flight height of 2 meters on the
AIRs-Street.

We also visualize the accuracy change in our progressive learning framework
for each flight height in Fig. 9, where $N_{i}$ denotes the trained model
applied on the maximum flight height $h_{i}$, e.g., $N_{5}$ is learned from
$h_{1}$ to $h_{5}$. As seen in Fig. 9 (a), our method demonstrates significant
improvement for large viewpoint differences, e.g., uav10, while maintaining
the accuracy for small viewpoint differences, e.g., uav02 on AirSim-Drone. The
same tendency can also be observed in Fig. 9 (b).

Also, owing to the lack of supervision for flying viewpoints, we can observe
that the increase in accuracy gradually slows down during the application of
our self-distillation. Since the viewpoint discrepancy increases gradually, we
need additional supervision to handle some hard cases. We leave it for our
future work.

(a) Results on the AirSim-Drone. (b) Results on the AIRs-Street.

Figure 8: Results for different methods while varying the maximum flying
heights. The x-axis denotes the flight heights and the y-axis shows the mean
accuracy on the corresponding range.

(a) Results on the AirSim-Drone. (b) Results on the AIRs-Street.

Figure 9: Visualization of the application of our self-distillation method on each flight height. $N_{i}$ denotes the trained model for the maximum flight height $h_{i}$. TABLE VI: Split of AirSim-Drone with different sampling intervals. Intervals |  Labeled view |  Unlabeled view
---|---|---
1 meter |  $h1$ |  $h2,h3,h4,h5,h6,h7,h8,h9,h10$
2 meters |  $h1$ |  $h3,h5,h7,h9$
3 meters |  $h1$ |  $h4,h7,h10$

(a) Results for different sampling intervals. (b) Results with and without
using the MixView. (c) Results with and without using the nearest neighbor
pseudo-labeling.

Figure 10: Results of three ablation studies. We show the effect of different
sampling intervals of viewpoints in (a), and performance with and without
using MixView in (b), results with and without using the nearest neighbor
pseudo-labeling in (c), respectively.

####  V-C4 Summary

Considering a model trained at the ground viewpoint as a baseline (i.e., a
Ground-only model), we show through experiments on both a synthesized dataset
and a real-world dataset that:

  * •

The performance of a Ground-only model gradually deteriorates from lower
flying height to higher flying height as viewpoint discrepancy gradually
increases.

  * •

The viewpoint difference leads the previous SSL-based methods to malfunction.
In our experiments, both Pseudo-Labeling and ClassMix failed on the task, as
their performance is even worse than that of the Ground-only model.

  * •

Our method shows a substantial performance boost compared to the Ground-only
model. The mean relative accuracy improvement is 25.7% and 16.9% for the fixed
AirSim-Drone and challenging AIRs-Street, respectively.

  * •

Most importantly, we find that the performance improvement is significant for
large viewpoint differences. We obtained 47.7% and 32.2% relative accuracy
improvement for the flight height of 10 meters, i.e., uav10 of AirSim-Drone,
and the flight height of 9 meters, i.e., uav09 of AIRs-Street, respectively.

  * •

Our method demonstrates good adaptability to various height ranges. It
outperformed the other two SSL-based approaches in all different height
ranges, except for the maximum flight height of 2 meters on the AIRs-Street.

###  V-D Ablation Studies

We perform several ablation studies to analyze and understand better each
component of our progressive SSL framework. To be specific, we conduct several
experiments to investigate the 1) performance of different sampling intervals
of viewpoints, 2) results with and without using the MixView, and 3) results
with and without using the nearest neighbor pseudo-labeling. All ablation
studies are performed on AirSim-Drone. The details are given as follows:

##### Analyses with different sampling intervals

We evaluate the three settings of the sampling interval, i.e., 1 meter, 2
meters, and 3 meters. Detailed information on data sampling is given in Table
VI. The experimental results are shown in Fig. 10. (a), where the blue,
yellow, and red color denotes results for a sampling interval of 1, 2, and 3
meters, respectively. The results show that dense sampling of viewpoints
contributes to better model performance, and we obtained $13.8\%$, $18.3\%$,
and $23.8\%$ mean accuracy boost from the Ground-only model, respectively.

##### Analyses with and without using the MixView

In our progressive learning framework, we propose to mix images of different
viewpoints to generate augmented training samples. We provide experimental
results to show that MixView can contribute to alleviating viewpoint
differences. As seen in Fig. 10. (b), where red, yellow, and blue color
denotes the results of the Ground-only model, our method $w/o$ and $w$ using
the MixView, respectively. Even without utilizing MixView, we observe that the
progressive distillation integrated with the nearest neighbor pseudo-labeling
already outperformed Ground-only by $16.5\%$, and MixView can further improve
the performance by $7.3\%$.

##### Analyses with and without using the nearest neighbor pseudo-labeling

In our original setting, for training a model at $h_{i}$, we first use the
trained model at $h_{i-1}$ to predict pseudo-labels from data captured at
$h_{i}$. $N_{i}$ is then trained with pseudo-labeled samples of
$h_{1},\ldots,h_{i}$ and unlabeled samples of $h_{i}$. We skip the above step
to remove the effect of the nearest viewpoint pseudo-labeling, then the model
$N_{i}$ is trained with pseudo-labeled samples of $h_{1},\ldots,h_{i-1}$, and
unlabeled samples of $h_{i}$. The results are shown in Fig. 10. (c), where
red, yellow, and blue color denotes the results of the Ground-only model, our
method $w/o$ and $w$ using the nearest neighbor pseudo-labeling, respectively.
As a result, the mean accuracy degradation from the Ground-only model up to
$47.8\%$ without utilizing the nearest neighbor pseudo-labeling.

##  VI Conclusion

In this paper, we have explored ground-to-aerial knowledge distillation to
enable drone perception by learning from a UGV without increasing any flight
data labeling cost. We argued that the essential requirement of drone
perception is the capability to perceive from different flight heights, and
the fundamental challenge is the significant viewpoint difference among flying
heights. Given only labeled images from the ground viewpoint and a set of
unlabeled images from various flying viewpoints, we formulated it as a semi-
supervised learning problem. We proposed a progressive learning framework that
gradually learns from the ground viewpoint to the maximum flying height. We
proposed two methods to overcome the viewpoint difference. The first is the
nearest neighbor pseudo-labeling that infers pseudo-labels of the nearest
neighbor viewpoint with the model learned at the preceding viewpoint, and the
second is the MixView which mixes data obtained from different viewpoints to
generate viewpoint invariant data samples.

To quantitatively and qualitatively verify the proposed method and inspire
more future explorations, we created both a synthesized dataset and a real-
world dataset. The experimental results show that the proposed method can
yield promising results for different flight heights. In practice, our
approach can relax the requirement of data annotation for drone perception and
expand a model for ground viewpoint perception to perform perception in fully
3D spaces. In the future, we will work on improving the training efficiency.
As a preliminary exploration on the task, we hope our work can inspire more
future explorations in the community.

## References

  * [1] D. Berthelot, N. Carlini, I. J. Goodfellow, N. Papernot, A. Oliver, and C. Raffel, “Mixmatch: A holistic approach to semi-supervised learning,” in _Advances in Neural Information Processing Systems_ , 2019, pp. 5050–5060.
  * [2] Y. Cao, Z. He, L. Wang, W. Wang, Y. Yuan, D. Zhang, J. Zhang, P. Zhu, L. Van Gool, J. Han _et al._ , “Visdrone-det2021: The vision meets drone object detection challenge results,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2021, pp. 2847–2854.
  * [3] Q. H. Cap, H. Uga, S. Kagiwada, and H. Iyatomi, “Leafgan: An effective data augmentation method for practical plant disease diagnosis,” _IEEE Transactions on Automation Science and Engineering_ , vol. 19, no. 2, pp. 1258–1267, 2020.
  * [4] F. Ceola, E. Maiettini, G. Pasquale, G. Meanti, L. Rosasco, and L. Natale, “Learn fast, segment well: Fast object segmentation learning on the icub robot,” _IEEE Transactions on Robotics_ , vol. 38, no. 5, pp. 3154–3172, 2022.
  * [5] C.Fan, J.Hu, and J.Huang, “Private semi-supervised federated learning,” in _International Joint Conference on Artificial Intelligence_ , 2022, pp. 2009–2015.
  * [6] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in _Proceedings of the European conference on computer vision_ , 2018, pp. 801–818.
  * [7] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in _Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 3213–3223.
  * [8] C. Fan, J. Hu, and J. Huang, “Few-shot multi-agent perception,” _Proceedings of the 29th ACM International Conference on Multimedia_ , pp. 1712–1720, 2021.
  * [9] A. Gawel, C. D. Don, R. Y. Siegwart, J. I. Nieto, and C. Cadena, “X-view: Graph-based semantic multiview localization,” _IEEE Robotics and Automation Letters_ , vol. 3, pp. 1687–1694, 2018.
  * [10] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition_ , 2012, pp. 3354–3361.
  * [11] S. Girisha, M. M. M. Pai, U. Verma, and R. M. Pai, “Semantic segmentation of uav aerial videos using convolutional neural networks,” _International Conference on Artificial Intelligence and Knowledge Engineering_ , pp. 21–27, 2019\.
  * [12] X. Guo, J. Hu, J. Chen, F. Deng, and T. L. Lam, “Semantic histogram based graph matching for real-time multi-robot global localization in large scale environment,” _IEEE Robotics and Automation Letters_ , vol. 6, pp. 8349–8356, 2021.
  * [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [14] J. Hu, C. Fan, H. Jiang, X. Guo, X. Lu, and T. L. Lam, “Boosting light-weight depth estimation via knowledge distillation,” _arXiv preprint arXiv:2105.06143_ , 2021.
  * [15] J. Hu, M. Ozay, Y. Zhang, and T. Okatani, “Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries,” in _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , 2019, pp. 1043–1051.
  * [16] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.
  * [17] S. Jung, S. Hwang, H. Shin, and D. H. Shim, “Perception, guidance, and navigation for indoor autonomous drone racing using deep learning,” _IEEE Robotics and Automation Letters_ , vol. 3, pp. 2539–2544, 2018.
  * [18] D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling, “Semi-supervised learning with deep generative models,” in _Advances in neural information processing systems_ , 2014, pp. 3581–3589.
  * [19] D.-H. Lee _et al._ , “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _Workshop on challenges in representation learning, ICML_ , vol. 3, no. 2, 2013, p. 896.
  * [20] J. Li, T. Sun, X. Huang, L. Ma, Q. Lin, J. Chen, and V. C. Leung, “A memetic path planning algorithm for unmanned air/ground vehicle cooperative detection systems,” _IEEE Transactions on Automation Science and Engineering_ , vol. 19, no. 4, pp. 2724–2737, 2021.
  * [21] S. Liao, X. Jiang, and Z. Ge, “Weakly supervised multilayer perceptron for industrial fault classification with inaccurate and incomplete labels,” _IEEE Transactions on Automation Science and Engineering_ , vol. 19, no. 2, pp. 1192–1201, 2020.
  * [22] Y.-C. Liu, J. Tian, N. Glaser, and Z. Kira, “When2com: Multi-agent perception via communication graph grouping,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2020, pp. 4105–4114.
  * [23] Y.-C. Liu, J. Tian, C.-Y. Ma, N. Glaser, C.-W. Kuo, and Z. Kira, “Who2com: Collaborative perception via learnable handshake communication,” in _IEEE International Conference on Robotics and Automation_ , 2020, pp. 6876–6883.
  * [24] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 3431–3440.
  * [25] S. Lu and Y. Wen, “Semi-supervised condition monitoring and visualization of fused magnesium furnace,” _IEEE Transactions on Automation Science and Engineering_ , vol. 19, no. 4, pp. 3471–3482, 2021.
  * [26] Y. Lyu, G. Vosselman, G.-S. Xia, A. Yilmaz, and M. Y. Yang, “Uavid: A semantic segmentation dataset for uav imagery,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 165, pp. 108–119, 2020.
  * [27] W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 Year, 1000km: The Oxford RobotCar Dataset,” _The International Journal of Robotics Research_ , vol. 36, no. 1, pp. 3–15, 2017.
  * [28] P. Mittal, R. Singh, and A. Sharma, “Deep learning-based object detection in low-altitude uav datasets: A survey,” _Image and Vision Computing_ , vol. 104, p. 104046, 2020.
  * [29] H. Noh, S. Hong, and B. Han, “Learning deconvolution network for semantic segmentation,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 1520–1528.
  * [30] A. Odena, “Semi-supervised learning with generative adversarial networks,” _arXiv preprint arXiv:1606.01583_ , 2016.
  * [31] V. Olsson, W. Tranheden, J. Pinto, and L. Svensson, “Classmix: Segmentation-based data augmentation for semi-supervised learning,” in _IEEE Winter Conference on Applications of Computer Vision_ , 2021, pp. 1368–1377.
  * [32] R. P. Padhy, S. Verma, S. Ahmad, S. K. Choudhury, and P. K. Sa, “Deep neural network for autonomous uav navigation in indoor corridor environments,” _Procedia Computer Science_ , vol. 133, pp. 643–650, 2018.
  * [33] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., 2019, pp. 8024–8035.
  * [34] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 779–788.
  * [35] M. N. Rizve, K. Duarte, Y. S. Rawat, and M. Shah, “In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning,” in _International Conference on Learning Representations_ , 2021.
  * [36] S. Ruder, “An overview of gradient descent optimization algorithms,” _arXiv preprint arXiv:1609.04747_ , 2016.
  * [37] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in _Field and Service Robotics_ , vol. 5, 2017, pp. 621–635.
  * [38] L. Sun, W. Yin, E. Xie, Z. Li, C. Sun, and C. Shen, “Improving monocular visual odometry using learned depth,” _IEEE Transactions on Robotics_ , vol. 38, no. 5, pp. 3173–3186, 2022.
  * [39] S. Vaddi, C. Kumar, and A. Jannesari, “Efficient object detection model for real-time uav applications,” _arXiv preprint arXiv:1906.00786_ , 2019.
  * [40] J. E. van Engelen and H. H. Hoos, “A survey on semi-supervised learning,” _Machine Learning_ , vol. 109, pp. 373–440, 2019.
  * [41] J. Xin, G. Xie, B. Yan, M. Shan, P. Li, and K. Gao, “Multimobile robot cooperative localization using ultrawideband sensor and gpu acceleration,” _IEEE Transactions on Automation Science and Engineering_ , vol. 19, no. 4, pp. 2699–2710, 2021.
  * [42] X. Yang, Z. Song, I. King, and Z. Xu, “A survey on deep semi-supervised learning,” _arXiv preprint arXiv:2103.00550_ , 2021.
  * [43] S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. J. Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable features,” in _Proceedings of the IEEE conference on Computer Vision_ , 2019, pp. 6022–6031.
  * [44] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” in _International Conference on Learning Representations_ , 2018.
  * [45] P. Zhu, D. Du, L. Wen, X. Bian, H. Ling, Q. Hu, T. Peng, J. Zheng, X. Wang, Y. Zhang _et al._ , “Visdrone-vid2019: The vision meets drone object detection in video challenge results,” in _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , 2019, pp. 227–235.
  * [46] P. Zhu, L. Wen, D. Du, X. Bian, H. Fan, Q. Hu, and H. Ling, “Detection and tracking meet drones challenge,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 44, no. 11, pp. 7380–7399, 2021.
  * [47] P. Zhu, L. Wen, D. Du, X. Bian, H. Ling, Q. Hu, Q. Nie, H. Cheng, C. Liu, X. Liu _et al._ , “Visdrone-det2018: The vision meets drone object detection in image challenge results,” in _Proceedings of the European Conference on Computer Vision Workshops_ , vol. 11133, 2018, pp. 437–468.
  * [48] J. Zurn, W. Burgard, and A. Valada, “Self-supervised visual terrain classification from unsupervised acoustic feature learning,” _IEEE Transactions on Robotics_ , vol. 37, pp. 466–481, 2021.

|  Junjie Hu (Member, IEEE) received the M.S. and Ph.D. degrees from the
Graduate School of Information Science, Tohoku University, Sendai, Japan, in
2017 and 2020, respectively. He is currently a Research Scientist with the
Shenzhen Institute of Artificial Intelligence and Robotics for Society. His
research interests include artificial intelligence and robotics.
---|---
|  Chenyou Fan is an Associate Professor with the School of Artificial
Intelligence, South China Normal University, China. He received the B.S.
degree in computer science from the Nanjing University, China, in 2011, and
the M.S. and Ph.D. degrees from Indiana University, USA, in 2014 and 2019,
respectively. He was a Research Scientist with the Shenzhen Institute of
Artificial Intelligence and Robotics for Society. His research interests
include machine learning and computer vision.
---|---
|  Mete Ozay (M’09) received the B.Sc., M.Sc., Ph.D. degrees in mathematical
physics, information systems, and computer engineering & science from METU,
Turkey. He has been a visiting Ph.D. and fellow in the Princeton University,
USA, a research fellow in the University of Birmingham, UK, and an Assistant
Professor in the Tohoku University, Japan. His current research interests
include pure and applied mathematics, theoretical computer science &
neuroscience.
---|---
|  Hua Feng received the B.S. degree from the Dongguan University of
Technology, Dongguan, China, in 2017, and master degree from the Wuyi
University, Jiangmen, China, in 2022. He is currently a research intern with
the Shenzhen Institute of Artificial Intelligence and Robotics for Society.
His research interests include machine learning and robotics.
---|---
|  Yuan Gao received M.S. from the University of Helsinki in 2016 and Ph.D.
degrees from the Department of Computer Science, Uppsala University, Uppsala,
Sweden, in 2020, under the supervision of Prof. Ginevra Castellano and Prof.
Danica Kragic. He is currently a Research Scientist with the Shenzhen
Institute of Artificial Intelligence and Robotics for Society. His research
interests include machine learning and robotics.
---|---
|  Tin Lun Lam (Senior Member, IEEE) received the Ph.D. degrees from the
Chinese University of Hong Kong, Hong Kong, in 2010. He is an Assistant
Professor with the Chinese University of Hong Kong, Shenzhen, China, and the
Director of Center for the Intelligent Robots, Shenzhen Institute of
Artificial Intelligence and Robotics for Society. He has published two
monographs and more than 50 research papers in top-tier international journals
and conference proceedings in robotics. His research interests include multi-
robot systems, field robotics, and collaborative robotics. Dr. Lam received an
IEEE/ASME T-MECH Best Paper Award in 2011 and the IEEE/RSJ IROS Best Paper
Award on Robot Mechanisms and Design in 2020.
---|---

##### Definition of relative accuracy improvement

Let Ground-only model be a baseline, we define the relative accuracy
improvement as:

$RAI=\frac{{Acc1-Acc2}}{{Acc2}}$ (10)

Where Acc1 and Acc2 are IoU accuracy of our method and the ground-only model,
respectively. The accuracy improvements reported in experiments are calculated
by Eq.(10).

##### Difference between MixUp, ClassMix, and MixView

MixUp and ClassMix are SSL techniques that utilize pixel-wise and object-wise
mixing operations. Despite their effectiveness in certain scenarios, these
methods fail to perform well under large viewpoint differences. To address
this limitation, we propose MixView, an enhanced version of ClassMix
specifically designed for handling viewpoint differences in SSL. This
improvement is made possible by employing a dense sampling strategy to capture
various flying heights and utilizing nearest neighbor pseudo-labeling to infer
more accurate pseudo-labels.

TABLE VII: Comparison between MixUp, ClassMix, and MixView. Method | Mixing operation | Accurate pseudo-labels | Viewpoint robust
---|---|---|---
MixUp | pixel-wise | ✗ | ✗
ClassMix | object-wise | ✗ | ✗
MixView | object-wise | ✓ | ✓

##### Accuracy for each category

To conduct more comprehensive evaluations, we calculate the IoU accuracy for
each category of the two datasets. Table VIII shows the results on the Airsim-
Drone dataset, indicating that our method demonstrates comparable performance
on the “Plant”, “Sky”, and “Ground” categories, and superior results on the
remaining six categories compared to the Ground-only method. Furthermore, on
the AIRs-street dataset, our approach achieves marginally higher accuracy on
the “Plant”, “Road”, ’Building’ and “Sky” categories, while exhibiting
significant improvements on the remaining nine categories, as shown in Table
IX.

Upon examining these results and the proportion of pixels in each category, we
can conclude that our method is particularly effective for large viewpoint
differences. Specifically, objects such as “Plant”, “Sky”, and “Road” that
remain stationary and occupy a significant percentage of pixels in both
datasets are more robust to viewpoint changes. These observations suggest that
we can pay more attention to those small and dynamic objects for performance
improvement in future exploration.

TABLE VIII: IoU accuracy for each category on the AirSim-Drone test set. Categories | Percentage | Ground-only | Ours
---|---|---|---
Plant | 0.375 | 0.809 | 0.805
Sky | 0.210 | 0.885 | 0.879
Road | 0.186 | 0.630 | 0.727
Building | 0.105 | 0.563 | 0.780
Car | 0.010 | 0.331 | 0.695
Ground | 0.089 | 0.119 | 0.117
Fence | 0.011 | 0.186 | 0.333
Pole | 0.002 | 0.081 | 0.126
Others | 0.012 | 0.150 | 0.258
TABLE IX: IoU accuracy for each category on the AIRs-Street (UAV02 to UAV09). Categories | Percentage | Ground-only | Ours
---|---|---|---
Plant | 0.326 | 0.907 | 0.936
Road | 0.303 | 0.902 | 0.931
Building | 0.164 | 0.915 | 0.933
Sky | 0.072 | 0.942 | 0.947
Fence | 0.045 | 0.465 | 0.641
Wall | 0.034 | 0.553 | 0.684
Car | 0.023 | 0.655 | 0.766
Sidewalk | 0.015 | 0.021 | 0.029
Traffic Light | 0.009 | 0.176 | 0.396
Person | 0.006 | 0.202 | 0.214
Others | 0.002 | 0.062 | 0.100
Motorcycle | 0.001 | 0.126 | 0.265
Traffic Sign | 0.001 | 0.299 | 0.408


#  $\bf{A+2n}$ compound nuclei and the unitary limit in nuclear physics

P E Georgoudis1,2 1Grand Accelerateur National d’Ions Lourds, CEA/DRF-
CNRS/IN2P3, BP 55027, F-14076 Caen Cedex 5, France. 2 Department of Physics,
National and Kapodistrian University of Athens, Zografou Campus, GR-15784
Athens, Greece<EMAIL_ADDRESS>
###### Abstract

This contribution discusses a new perception of the structure of compound
nuclei by introducing intermediate states of the Feshbach formalism of nuclear
reactions in the Interacting Boson Model of nuclear structure. The stake is to
explore the manifestation of the unitary limit in heavy, even-even nuclei.
Interactions that govern Feshbach resonances of cold and dilute atomic gases
suggest the formulation of an IBM-compound Hamiltonian for scattering two
neutrons ($2n$) from a heavy, even-even target ($A$). The solutions of the
corresponding coupled channel equations host a $2n$-IBM state resonance. It
turns out that the unitary limit is measurable in a heavy $A+2n$ compound
nucleus at low temperatures. That measurement is feasible through the
fluctuations of the cross-sections that tune the $2n$-$A$ scattering length.

##  I Introduction

Heavy, even-even nuclei serve as an inclusive testing ground for a remarkable
pluralism of physical phenomena that range from Bardeen Cooper Schriefer
behavior of nucleon pairs at valence shells to boson condensates for the
emergence of the nuclear shapes as well as to quantum critical points in
Shape/Phase transitions CJC . The Interacting Boson Model provides a group
theoretical framework that synthesizes a simple and integrated approach out of
such a diversity of physical aspects in nuclear physics [2]. On the other
hand, cold and dilute atomic gases manifest the so-called unitary limit in the
vicinity of Feshbach resonances Randeria . The unitary limit serves as a
theoretical benchmark that, like IBM, accompanies a similar but enriched
diversity of physical aspects ranging from the BCS-Bose Einstein Condensation
crossover to a quantum critical point. In parallel, the unitary limit refers
to a strong coupling problem that is describable in terms of a Conformal Field
Theory CFT . The theoretical application of the unitary limit in nuclear
physics was first realized in light nuclei light . It emerged parallel with
the development of Effective Field Theories EFTs . This contribution briefly
reviews the first exploration of the unitary limit in heavy even-even nuclei
Georgoudis . Apart from the phenomenological insights that this contribution
focuses on, that exploration initiated the examination of conformal symmetry
within the group theoretical framework of IBM. Conformal invariance arises at
the critical point of a second-order phase transition Fisher  as well as in
Quantum Chromo Dynamics at the limit of a large number of gluons Gross . In
that perspective, introducing the unitary limit in heavy even-even nuclei
commences the investigation of algebraic relations between the symmetries of
the IBM with the conformal symmetry of second-order critical points. In
parallel, it explores algebraic relations with the limit of conformal
invariance in a strong coupling problem (unitary limit) that is amenable to be
incorporated afterward with QCD.

##  II Lessons from the unitary limit in systems of cold atoms

Consider a scattering problem with a wavefunction that depends on the radial
distance $r$ between the scattering particles and the phase shifts
$\delta(k)$. In what follows, the discussion is restricted to $s$-wave
scattering. One introduces the quantity of the scattering length $a$ through
the behavior of the phase shifts at very low energies, that is $\delta(k)\sim-
ka$ for $k\rightarrow 0$. For example, a scattering wavefunction of the form
$\sin(kr+\delta(k))$ looks like $k(r-a)$ at very low energies. The scattering
length $a$ shows the intercept of the wavefunction with respect to the
horizontal axis, that is the radial distance. Apropos unitarity, one now
focuses on the exhaustion of the unitarity bound in the cross-section. The
parameter that controls the deviation from the unitarity bound is the
generalized scattering length $a(k)$ as defined by Bethe Bethe  through the
effective range ($r^{*}$) expansion

$-k\cot\delta(k)=\frac{1}{a(k)}=\frac{1}{a}-\frac{1}{2}k^{2}r^{*}+\cdots.$ (1)

The condition $1/a(k)=0$ on the generalized scattering length defines a
resonance at the scattering amplitude Bethe . At very low kinetic energies,
$kr^{*}<<1$, that resonance condition on Eq (1) implies an infinite value for
the scattering length, i.e. $a\rightarrow\infty$. The latter amounts to
maximize the interaction strength between two particles, $g=4\pi a\hbar^{2}/m$
with $m$ the mass of each particle, i.e., it reflects a strong coupling limit.
The unitary limit refers to the infinite value of the scattering length $a$ at
low kinetic energy. A paradigm of its experimental observation has been
achieved in the cold and dilute atomic gases Randeria .

The open-closed channel crossing during an atom-atom collision is the
underlying mechanism that realizes the unitary limit. The open channel
reflects a scattering state of two cold atoms, while the closed channel is the
bound state of a diatomic molecule formed by these same cold atoms. Channels’
crossing means the coincidence in the energy of two different channels. In the
Feshbach formalism for reactions, the open-closed channel crossing is achieved
through the resonating energy of the open channel with the energy of an
intermediate state of the closed channel. That resonating energy gives rise to
a resonance that manifests the intermediate state of the closed channel. In
atomic and molecular physics, these resonances are the celebrated Feshbach
resonances Timmermans . However, they were initially introduced in compound
nuclei Feshbach .

The intermediate state of the Feshbach formalism affects the wavefunction’s
scattering length $a$. In general, for low $k$, the element of the scattering
matrix is expressed through the phase shifts as
$S_{0}=e^{2i\delta(k)}=e^{-2ika}$. In the presence of an intermediate state of
energy $E_{m}$ and width $\Gamma_{m}$, the quantity $S_{0}$ takes the form

$S^{\prime}_{0}=e^{2ika}\left(1-i\frac{\Gamma_{m}}{E-E_{m}+i\Gamma_{m}/2}\right).$
(2)

From the perspective of nuclear physics Feshbach , Eq. (2) generates the
scattering matrix element $S^{\prime}_{0}=S_{0}S_{R}$, with
$S_{R}=1-i\Gamma_{m}/\left(E-E_{m}+i\Gamma_{m}/2\right)$ a fluctuating part
that fluctuates rather rapidly with the energy by the resonating energies
$E_{m}$ and widths $\Gamma_{m}$. Now, like in atomic and molecular physics
Timmermans , one identifies the effect of that fluctuating part in the
emergence of an effective scattering length $a_{eff}=a+a^{\prime}$, with

$e^{2ika^{\prime}}=1-i\frac{\Gamma_{m}}{E-E_{m}+i\frac{\Gamma_{m}}{2}},\
a_{eff}=a+\frac{1}{2k}\tan^{-1}\left(\frac{\Gamma_{m}(E-E_{m})}{(E-E_{m})^{2}+\frac{\Gamma_{m}^{2}}{4}}\right).$
(3)

The effective scattering length goes to infinity at the resonating energies
$E=E_{m}$ of the open channel with the intermediate states. Therefore, a
Feshbach resonance maximizes the scattering length. Experimentally in
ultracold atoms, an external magnetic field tunes the energy of the channels
to achieve their crossing that generates the Feshbach resonance.

That said, one devises now an approach to introduce the unitary limit in
heavy, even-even nuclei. Their low-lying collective states are formed by
nucleon pairs that behave as bosons and generate coherent states to produce
the nuclear shapes IBM . Therefore, one examines the formation of an IBM boson
in analogy with a Feshbach resonance during the formation of a diatomic
molecule out of two cold atoms. Such a resonance is introduced in Georgoudis
for scattering two slow neutrons ($2n$) with a heavy even-even nucleus of mass
number $A$. In that case, the Feshbach formalism’s intermediate states affect
the scattering matrix’s element by a fluctuating term like in Eq. (2). Such
fluctuations are experimentally testable in nuclear physics and represent the
formation of the compound system. It is outlined below how the observation of
a particular fluctuation centered around the two-neutron separation energy
($S_{2n}$) maximizes the neutron pair $(2n)$-IBM state scattering length.

##  III Algebraic correspondences between the IBM and systems of cold atoms

Werner and Castin Castin  introduced mappings between zero-energy states and
trapped states widely used in cold atoms. A trapped state is merely a
quantized state of a harmonic oscillator. One opens the walls of the trap by
reaching the zero-frequency limit $\omega=0$ in the harmonic oscillator and
obtains the zero-energy state. Werner-Castin mappings were introduced in the
solutions of the Schrodinger equation for N trapped cold atoms or particles in
general. They preserve the unitary limit and are realized through the
generators of the $SO(2,1)$ group in an isomorphic realization to the
generators of the conformal group in one dimension - time. An algebraic
correspondence for those mappings is established with the simplest form of the
$O(6)$ limit of the IBM Georgoudis . In other words, the Werner-Castin
mappings Castin  correspond to certain relations in the $O(6)$ limit of the
IBM under the appropriate algebraic replacements. By this process, one
introduces the one-dimensional conformal group in the IBM and writes down the
wavefunction for the corresponding zero-energy state. This state contains a
scaling exponent that arises out of the invariant quantity of dilatation
(scale) transformations of the boson number radius $\rho$. That invariant
quantity is the $O(6)$ quantum number $\sigma$. Table 1 summarizes the main
replacements/relations of this correspondence.

The practical result for nuclear physics emerges by noticing that Werner-
Castin mappings apply to Feshbach resonances by generating the intermediate
(trapped) state of the closed channel out of the channels’ crossing (zero-
energy state) and vice-versa. By applying those mappings to the intermediate
states of the $A+2n$ compound nucleus, one obtains the corresponding zero-
energy states that reveal the open-closed channel crossings. For instance, if
the ground state of the target $A$ contains $N_{b}$ bosons at equilibrium, one
applies the mapping to the intermediate state of the $A+2n$ compound nucleus
with the target having $N_{b}+1$ bosons. The result is to remove one boson and
take the zero-energy state in the form $\rho^{N_{b}+1}$. That state is the
crossing of the open channel - consisting of the neutron pair $(2n)$ and the
target nucleus of $N_{b}$ bosons - with the closed channel consisting of the
$N_{b}+1$ bound state of the IBM formed by adding that $(2n)$ as one boson.

##  IV A+2n compound nucleus at low temperatures like a cold and dilute atomic
gas

Table 1: Algebraic correspondences between the Schrodinger equation of $N=2$ trapped atoms with the IBM O(6) limit. $r_{1,2}$ is each atom’s radial distance from the trap’s center, and $l_{1,2}$ are their angular momenta. $L_{\pm}$, $L_{0}$ are the $SO(2,1)$ generators Georgoudis . |  $N=2$ hyperspherical |  $O(6)$ IBM
---|---|---
radial variable | $R=\sqrt{r^{2}_{1}+r^{2}_{2}}$ | boson number radius $\rho$
dilatation eigenvalue | $\lambda=l_{1}+l_{2}$ | quantum number $\sigma$
energy | $\left(\lambda+2q+6/2\right)\hbar\omega$ |  $\left(N_{b}+6/2\right)\hbar\omega$, $N_{b}=\sigma+2J$
zero energy state | $\psi^{0}_{\lambda}=R^{\lambda+2q}$ | $\psi^{0}_{\sigma}=\rho^{N_{b}}$
Werner-Castin mapping | $|F^{q}_{\lambda}\rangle=L^{q}_{+}e^{-R^{2}/2a^{2}_{ho}}|\psi^{0}_{\lambda}\rangle$ | $|\Phi^{J}_{\sigma}\rangle=L^{J}_{+}e^{-\rho^{2}/2a^{2}_{ho}}|\psi^{0}_{\sigma}\rangle$

The target nucleus’ two neutron separation energy $S_{2n}$ determines the
length scale. The boson number radius $\rho$ is measured in units of the
harmonic oscillator length $a_{ho}=\hbar/\sqrt{MS_{2n}}$ where $M$ is the
neutron mass. In that case, one writes down the radial distance between the
$2n$ and the IBM state in the form of $R-\rho\equiv r$ and the corresponding
wavenumber as $k_{r}$.

Channel states are restricted to scalar angular momentum couplings of the form
$\Psi(r,\rho)=\sum_{n}\Psi_{n}(r)\Phi_{n}(\rho)$ where $\Psi_{n}(r)$ is the
$2n$ wavefunction, and $\Phi_{n}(\rho)$ is the IBM wavefunction. The
scattering occurs at the cold limit, i.e., at a much lower kinetic energy of
the $2n$ concerning the target’s $S_{2n}$. The open channel is the $n=0$,
where the target contains $N_{b}$ bosons, while channel states with the target
in higher boson numbers $N_{b}+1,2\dots$ are those for $n>0$. These channel
states form the Hilbert space of the IBM-compound Hamiltonian

$H_{c}=H(r)+H(\rho)+H(\rho,r).$ (4)

$H(r)$ is the Hamiltonian for the relative kinetic energy of the $2n$ with
respect to the target $A$ and the interaction between the two neutrons.
$H(\rho)$ is the IBM Hamiltonian for the target, and $H(\rho,r)$ is the
$2n$-IBM state interaction term. To investigate the unitary limit of $H_{c}$,
the interaction terms are specified in analogy with the interactions and the
external magnetic field governing the vicinity of Feshbach resonances in cold
and dilute atomic gases. There, when the formation of diatomic molecules
reaches the unitary limit, the atom-atom unitary interaction induces a
molecule-molecule unitary interaction Petrov . In $H_{c}$, that means one
introduces a unitary interaction for the incident neutrons themselves (atom-
atom) plus a unitary interaction for the $2n$-IBM state coupling (molecule-
molecule). The dilute character of the target’s valence space concerning the
short range of the strong interaction and the cold neutron pair rationalizes
the analogy. Accordingly, a $2n$-IBM state scattering length $a_{r}$ is
introduced through the corresponding effective range expansion as shown in
Table 2.

One has the neutron-neutron scattering length $a$ and the pair-IBM state
scattering length $a_{r}$ and the corresponding unitary interactions are

$\begin{split}&\frac{4\pi
a\hbar^{2}}{M}\delta(r_{1}-r_{2})\rightarrow\lim_{r_{1}\rightarrow
r_{2}}\Psi_{0}(r)=\frac{C}{r_{1}-r_{2}}-\frac{1}{a},\\\
&\frac{4\pi^{3}a_{r}\hbar^{2}}{M}\delta(r)\rightarrow\lim_{r\rightarrow
0}\Psi_{0}(r,\rho)=\Phi_{0}(\rho)\left(\frac{C}{r^{4}}-\frac{1}{a^{4}_{r}}\right).\end{split}$
(5)

These boundary conditions apply to the $2n$ scattering wavefunction
$\Psi_{0}(r)$, and to the channel wavefunction $\Psi_{0}(r,\rho)$. They
replace the two unitary interactions, respectively. The full IBM-compound
Hamiltonian now reads

$H_{c}=-\frac{\hbar^{2}}{2M}\left(\frac{1}{r^{5}}\frac{\partial}{\partial
r}r^{5}\frac{\partial}{\partial
r}-\frac{\lambda(\lambda+4)}{r^{2}}\right)+H(\rho)+s^{\dagger}+s.$ (6)

The $s^{\dagger}+s$ term changes target states by one $s$ boson. It is the
analog of the magnetic field tuning to achieve the open-closed channel
crossing. The effective range of the $2n$-IBM state interaction is the $r^{*}$
as seen in Table 2. That range is detemrined experimentally by the width of
the resonance that corresponds to the channels’ crossing through the relation
$\Gamma_{m}=\hbar^{2}k_{r}/Mr^{*}$.

Intermediate states of the Feshbach formalism are stationary states of the
$A+2n$ compound nucleus formed by the target plus two neutrons. They serve as
resonance states of energy $\epsilon_{n}$ with respect to the total energy $E$
of the open channel. The coupled channels equations read

$\begin{split}(E-H_{PP})P|\Psi\rangle=H_{PQ}Q|\Psi\rangle,\\\
(E-H_{QQ})Q|\Psi\rangle=H_{QP}P|\Psi\rangle.\end{split}$ (7)

The projection operators are the open channel
$P=|\Phi_{0}(\rho)\rangle\langle\Phi_{0}(\rho)|$ and the set of closed
channels $Q=\sum_{n>0}|\Phi_{n}(\rho)\rangle\langle\Phi_{n}(\rho)|$. Open-open
$H_{PP}$ and closed-closed $H_{QQ}$ channel couplings are the unitary
interactions and are included in the boundary conditions (5). We examine the
coupling of the open channel ($n=0$) of $N_{b}$ bosons with the first closed
channel ($n=1$) of $N_{b}+1$ bosons. The coupling $H_{PQ}$ now is
$H_{10}=\langle N_{b}+1|s^{\dagger}+s|N_{b}\rangle=\sqrt{N_{b}+1}$ and the
reverse $H_{QP}$ is the same $H_{01}=\langle
N_{b}|s+s^{\dagger}|N_{b}+1\rangle=\sqrt{N_{b}+1}$.

##  V Results

The energy scale is normalized to the energy of the target’s ground state,
i.e., to $N_{b}$ bosons. The $2n$ $s$-wave ($\lambda=0$) open channel
solutions are presented in sufficient detail in Georgoudis . Table 2
summarizes the results for the $2n$ scattering compared to $1n$ scattering.
The main difference is the cubic power in the solid angle factor of the cross-
section and on the content of the wave numbers.

In absence of a resonance with the intermediate state, the $s$-wave phase
shifts of the $2n$-IBM state scattering give the scattering matrix element
$S_{0}=e^{2i\delta(k_{r})}=e^{-2ik_{r}a_{r}}$. Now, let us focus on the first
closed channel where the energy of the target (IBM state) is denoted by the
capital $E_{1}=(N_{b}+1+6/2)\hbar\omega$ and differs by $S_{2n}$ from the
energy of the target in the open channel of $N_{b}$ bosons. The intermediate
state $\Psi_{1}(r)$ of the neutron pair on that closed channel has, in
general, an unknown energy denoted by $\epsilon_{1}$. Its Schrodinger equation
is obtained by the second equation of (7) by setting $H_{10}=0$. Then, the
total energy $E$ is restricted to the energy $\epsilon_{1}$ of the
intermediate state, and its equation reads

$\begin{split}\left(T_{r}+E_{1}\right)\Psi_{1}(r)=\epsilon_{1}\Psi_{1}(r).\end{split}$
(8)

This equation supports a zero-energy state under the condition
$E_{1}=\epsilon_{1}$. That condition is satisfied when the energy of the
intermediate state of the $2n$ in the compound $A+2n$ nucleus coincides with
the bound IBM state of $N_{b}+1$ bosons. Unitarity manifests itself for that
state by turning to the effect on the scattering matrix element. The new
element of the scattering matrix reads $S^{\prime}_{0}=S_{0}S_{R}$, with the
fluctuating term $S_{R}=1-i\Gamma_{1}/\left(E-E_{1}+i\Gamma_{1}/2\right)$.
Like in Eq. (3), that fluctuation generates an effective $2n$-IBM state
scattering length $a_{reff}$ in the same sense with the emergence of the
$a_{eff}$ in cold atoms. Therefore, at resonance, the condition
$1/a_{r}(k_{r})=0$ applies and the $a_{reff}$ affects the $1/a_{r}$ part of
the effective range expansion. That resonance is measurable through the
fluctuation $S_{R}$, which generates the compound-elastic cross-section

$\sigma_{ce}=\frac{(4\pi)^{3}}{k^{2}_{r}}\frac{\Gamma^{2}_{1}}{(E-E_{1})^{2}+(\Gamma_{1})^{2}/4}.$
(9)

The exhaustion of the unitarity bound occurs when the resonance’s energy is
the two neutron separation energy $E=E_{1}=S_{2n}$. The vicinity of the
unitary limit is quantified by the width
$\Gamma_{1}=b^{2}_{1}\left(\frac{4M}{\hbar^{2}}\right)k_{r}$, with
$b^{2}_{1}=(N_{b}+1)\mathinner{\\!\left\lvert\int
dr\Psi_{1}(r)\Psi_{0}(r)\right\rvert}^{2}$ Georgoudis . The latter depends on
the neutron mass, the kinetic energy, and the boson number of the closed
channel times a spectroscopic factor for the intermediate state.

##  VI Conclusions

This work did not examine the conditions under which the incident $2n$ are
captured as one boson by the target nucleus to form the $A+2n$ compound
nucleus. Instead, this is the result of the investigation of unitarity in the
$A+2n$ compound nucleus, i.e., that the capture of the slowly incident $2n$ as
one boson maximizes the $2n$-IBM state scattering length $a_{r}$. The
accompanied phenomenological insight is sufficiently important. Namely, the
energies and the widths of the fluctuations of the cross-sections of $A+2n$
compound nuclei propose an experimental case study to examine the unitary
limit in nuclear physics. The proposed IBM-compound Hamiltonian guarantees the
$2n$ capture as one boson by the coupling term $s^{\dagger}+s$. How is the
neutron-neutron scattering length related to the $a_{r}$? In cold atoms the
atom-atom scattering length $a_{t}$ is related to the molecule-molecule
scattering length $a_{m}$ by the empirical relation $a_{m}=0.6a_{t}$ Petrov .
The maximization of the one induces the maximization of the other at
resonance. In the same sense, one argues that the maximization of the $2n$-IBM
state scattering length $a_{r}$ induces the maximization of the neutron-
neutron scattering length.

Table 2: The solutions of the open channel of the $2n$-$A$ scattering compared to those of $1n$-$A$ scattering. $r_{1,2}$ is the radial distance of each neutron with respect to the heavy $A$ core. | One neutron | Two neutrons
---|---|---
radial variable | $r_{1}$ |  $r=R-\rho$, $R^{2}=r^{2}_{1}+r^{2}_{2}$
wavenumber | $k_{1}$ | $k_{r}$
Scattering wavefunction | $\Psi_{0}(r_{1})=\frac{e^{-ik_{1}r_{1}}}{r_{1}}-S_{0}\frac{e^{ik_{1}r_{1}}}{r_{1}}$ | $\Psi_{0}(r)=\frac{e^{-ik_{r}r}}{r^{5/2}}-S_{0}\frac{e^{ik_{r}r}}{r^{5/2}}$
Effective Range Expansion | $\frac{1}{a_{1}(k_{1})}=\frac{1}{a_{1}}-\frac{1}{2}k^{2}_{1}r^{*}_{1}+\cdots$ | $\frac{1}{a_{r}(k_{r})}=\frac{1}{a_{r}}-\frac{1}{2}k^{2}_{r}r^{*}+\cdots$
cross - section | $\sigma=\frac{4\pi}{k^{2}+1/a_{1}^{2}(k)}$ | $\sigma=\frac{(4\pi)^{3}}{k_{r}^{2}+1/a_{r}^{2}(k)}$

##  VII Acknowledgments

This research has received funding by the European Union’s H2020 program,
Marie Sklodowska Curie Actions - Individual Fellowships, Grant Agreement No
793900-GENESE 17.

## References

## References

  * (1) Cejnar P, Jolie J and Casten R F 2010 Rev. of Mod. Phys 82 2155
  * (2) Iachello F and Arima A 1987 The Interacting Boson Model (Cambridge: University Press)
  * (3) Randeria M, Zwerger W and Swierlein M 2012 BCS to BEC Crossover and the Unitary Fermi gas (Berlin: Springer-Verlag)
  * (4) Son D T 2008 Phys. Rev. D 78 046003
  * (5) Stetcu I, Rotureau J, Barrett B R and Kolck U van 2010 Ann. of Phys. 325 1644; Konig S, Griesshammer H W, Hammer H W and Kolck U van 2017 Phys. Rev. Lett. 118 202501
  * (6) Hammer H -W, Konig S and Kolck U van 2020 Rev. Mod. Phys. 92 025004
  * (7) Georgoudis P E 2021 Nucl. Phys. A 1015 122297
  * (8) Fisher M E 1967 Physics Physique Fizika 3 255
  * (9) Gross D J 2004 Nucl. Phys. B Proc. Suppl. 135 193
  * (10) Bethe H A and Morrison P 2006 Elementary Nuclear Theory (New York: Dover)
  * (11) Timmermans E, Tommasini P, Husein M and Kerman A 1999 Phys. Rep. 315 199
  * (12) Feshbach H 1958 Ann. of Phys. 5 357; Feshbach H, Kerman A K and Lemmer R H 1967 Ann of Phys. 41 230
  * (13) Werner F and Castin Y 2006 Phys. Rev. A 74 053604
  * (14) Petrov D S, Salomon C and Shlyapnikov G V 2004 Phys. Rev. Lett. 93 090404\.