#  NEAT Beating DRL in Managing NOMA-mmWave-UAV Resources

Ahmed Benfaid, Nadia Adem, and Abdurrahman Elmaghbub  Ahmed Benfaid∗, Nadia
Adem∗, and Abdurrahman Elmaghbub∗∗

∗University of Tripoli, Tripoli, Libya, E-mail<EMAIL_ADDRESS>∗∗Oregon State University, OR, USA, Email<EMAIL_ADDRESS>
#  How Efficiently Can NEAT Manage NOMA-MmWave-UAV Resources?

Ahmed Benfaid, Nadia Adem, and Abdurrahman Elmaghbub  Ahmed Benfaid∗, Nadia
Adem∗, and Abdurrahman Elmaghbub∗∗

∗University of Tripoli, Tripoli, Libya, E-mail<EMAIL_ADDRESS>∗∗Oregon State University, OR, USA, Email<EMAIL_ADDRESS>
#  Can NEAT Manage NOMA-MmWave-UAV Resources Efficiently?

Ahmed Benfaid, Nadia Adem, and Abdurrahman Elmaghbub  Ahmed Benfaid∗, Nadia
Adem∗, and Abdurrahman Elmaghbub∗∗

∗University of Tripoli, Tripoli, Libya, E-mail<EMAIL_ADDRESS>∗∗Oregon State University, OR, USA, Email<EMAIL_ADDRESS>
#  How Efficient Can NEAT Be in Managing NOMA-MmWave-UAV Resources?

Ahmed Benfaid, Nadia Adem, and Abdurrahman Elmaghbub  Ahmed Benfaid∗, Nadia
Adem∗, and Abdurrahman Elmaghbub∗∗

∗University of Tripoli, Tripoli, Libya, E-mail<EMAIL_ADDRESS>∗∗Oregon State University, OR, USA, Email<EMAIL_ADDRESS>
#  ProSky: NEAT Meets NOMA-mmWave
in the Sky of 6G

Ahmed Benfaid, Nadia Adem, and Abdurrahman Elmaghbub  Ahmed Benfaid∗, Nadia
Adem∗, and Abdurrahman Elmaghbub∗∗

∗University of Tripoli, Tripoli, Libya, E-mail<EMAIL_ADDRESS>∗∗Oregon State University, OR, USA, Email<EMAIL_ADDRESS>
###### Abstract

Rendering to their abilities to provide ubiquitous connectivity, flexibly and
cost effectively, unmanned aerial vehicles (UAVs) have been getting more and
more research attention. To take the UAVs' performance to the next level,
however, they need to be merged with some other technologies like non-
orthogonal multiple access (NOMA) and millimeter wave (mmWave), which both
promise high spectral efficiency (SE). As managing UAVs efficiently may not be
possible using model-based techniques, another key innovative technology that
UAVs will inevitably need to leverage is artificial intelligence (AI).
Designing an AI-based technique that adaptively allocates radio resources and
places UAVs in 3D space to meet certain communication objectives, however, is
a tough row to hoe. In this paper, we propose a neuroevolution of augmenting
topologies NEAT framework, referred to as $\tt{ProSky}$, to manage NOMA-
mmWave-UAV networks. $\tt{ProSky}$ exhibits a remarkable performance
improvement over a model-based method. Moreover, $\tt{ProSky}$ learns $5.3$
times faster than and outperforms, in both SE and energy efficiency EE while
being reasonably fair, a deep reinforcement learning DRL based scheme. The
$\tt{ProSky}$ source code is accessible to use here:
https://github.com/Fouzibenfaid/ProSky

###### Index Terms:

Deep reinforcement learning (DRL), millimeter wave (mmWave), neuroevolution of
augmenting topologies (NEAT), non-orthogonal multiple access (NOMA), unmanned
aerial vehicle (UAV).

††publicationid: pubid:  978-1-6654-5975- 4/22 © 2022 IEEE

##  I Introduction

Despite the unprecedented advancements in telecommunication technologies in
recent years, around half of the world's population, mostly living in rural
and developing areas, still has limited or no access to cellular communication
services[1]. Providing connectivity to those underprivileged areas could
unequivocally enhance the quality of their lives. One of the envisioned, must-
be-met, requirements of 6G is enabling ubiquitous geographical coverage
anywhere, anytime. Due to the lack of essential cellular infrastructures in
rural areas and the high cost of establishing them, along with the
incapability of terrestrial base stations (BSs) to cover hotspot areas during
special events or disaster scenarios, unmanned aerial vehicles (UAVs), thanks
to their flexible 3D mobility and ease of deployment, are envisioned to be a
major part of the 6G wireless networks, acting as flying BSs [2]. Allowing
sharing same spectrum resources among multiple wireless nodes simultaneously,
non-orthogonal multiple access (NOMA) is emerging as another 6G enabler
offering massive device connectivity without exhausting spectrum resources
[2]. The emergence of NOMA-aided UAV networks necessitates a careful
investigation of optimal UAV 3D deployment, and power allocation (PA)
management [2]. Due to the high complexity of such an optimization problem,
authors in [3], for example, approached the UAV placement and PA problems
disjointly, whereas the number of served users was limited to two in [4, 5],
and the UAV mobility was restricted to a 2D plane in [4, 3, 5]. There have
been some attempts to address the UAV 3D placement problem, but these have
primarily been accomplished by disjoining the UAV placement and PA problems,
as done in [6].

Furthermore, due to the high probability of line-of-sight (LoS) links UAVs
offer, the vast millimeter wave (mmWave) and terahertz spectrum can be
utilized to satisfy the major 6G data rate enhancement requirement [2].
Nevertheless, managing NOMA-mmWave-UAV network resources to satisfy dynamic,
heterogeneous, and massive needs adaptively yet fairly and efficiently is a
complicated challenge, which conventional and even machine learning methods
fail to handle. The good news, however, is that deep learning and, more
generally, artificial intelligence (AI) technologies have the strong potential
to handle multi-state network statuses and demands. After proving their
effectiveness in solving problems with a large degree of freedom in various
fields, AI techniques are proposed to be a key enabler for self-organizing,
self-optimized networks in the 6G era [2].

Inspired by the remarkable successes of incorporating deep reinforcement
learning (DRL) into different fields, a DRL model has been used to solve the
placement problem of UAVs that fly at a fixed height [7]. More interestingly,
our previous work, AdaptSky [8], unlike any other work, jointly solved the
non-convex optimization problem of the 3D deployment and the PA of a NOMA-
equipped UAV BS in the mmWave spectrum. Another AI tool that has recently
shown outstanding performance in a variety of applications, including robotics
and gaming [9], is neuroevolution of augmenting topologies (NEAT) [10].
Furthermore, although fairly limited, NEAT has been recently used in the area
of communications, for example, in [11] to improve beam management in vehicle-
to-vehicle communications. To the best of our knowledge, no work has studied
the use of NEAT to manage UAV-based communications. In this work, we propose
$\tt{ProSky}$, a novel NEAT-based framework that jointly optimizes 3D
deployment and PA for NOMA-aided UAV BSs operating in the mmWave spectrum. The
main contributions of this work, and advancements over [8], are summarized as
follows:

  1. (i)

$\tt{ProSky}$ incorporates and demonstrates the effectiveness of NEAT, for the
first time, to manage NOMA-mmWave-UAV networks. We set the NEAT environment
that leads to the optimal UAV placement and NOMA PA such that, without
sacrificing fairness, the total network data rate is maximized.

  2. (ii)

Although DRL based algorithms show tremendous improvements over a state-of-art
in managing 3D networks while exhibiting high generalization capabilities [8],
they have two drawbacks: i) specifying their neural network (NN) structure
beforehand and tuning it using trials and errors does not only lead to
degradation in efficiency but also some times in performance, ii) they train,
relatively, slow and run into local minima issues. As NEAT optimally
determines the NN structure, $\tt{ProSky}$, however, demonstrates, while being
reasonably fair, $5.8\%$ improvements in SE and $21.9\%$ in EE over a DRL
based benchmark. More interestingly, as NEAT simultaneously trains multiple
NNs to evolve based on a genetic algorithm (GA), $\tt{ProSky}$ exhibits more
than $400\%$ improvement in learning rate.

##  II System model

###  II-A Network Model

Embracing the 3D coverage capability of UAVs, and similar to our work in [8],
we consider a 3D downlink cellular network that covers an area $\mathcal{A}$
of $L\times L$ units in which a UAV serves a total of $2{N}$, for some integer
number ${N}$, uniformly distributed ground users, $\tt{N_{UE}}$. The UAV and
ground users are assumed to be equipped with $\mathcal{N}_{UAV}$ and
$\mathcal{N}_{\tt{UE}}$ antennas, respectively. Throughout the paper, user $i$
is denoted by ${\tt{UE}}_{i}$ where $i\in\\{1,2,..,\tt{N_{UE}}\\}$. We assume
that the users are grouped into clusters, as depicted in Fig. 1, in such a way
that users ${\tt{UE}}_{i}$ and ${\tt{UE}}_{i+1}$ for
$i\in\\{1,3,..,\tt{N_{UE}}-1\\}$ are associated with the same cluster and
$\tt{UE}_{i}$ has a stronger channel gain than ${\tt{UE}}_{i+1}$, following
the distance-based pairing strategy discussed in [12]. The assumption of
having $2{N}$ users is set only for convenience and should not affect the
model's generality. In case there is an odd number of users, a cluster will
encompass a single user and every thing else is still valid. The UAV serves
each cluster over an orthogonal power resource with a total power $P_{T}$,
distributed between the two corresponding users based on their channel
conditions. The received power at the ${\tt{UE}}_{i}$ at a given time step
$\tau$ can be expressed as

$\displaystyle\hat{P}_{i,\tau}$ $\displaystyle=$ $\displaystyle
P_{T}{g}_{i,\tau}^{\tt{MIMO}}(d_{i,\tau}){\alpha}_{i,\tau},$ (1)

where ${g}^{\tt{MIMO}}_{i,\tau}(d_{i,\tau})$ is the gain between the UAV and
${\tt{UE}}_{i}$ separated by a 3D distance of $d_{i,\tau}$, and
${\alpha}_{i,\tau}$ is the percentage of the $P_{T}$ assigned to
${\tt{UE}}_{i}$. If we consider only a large scale fading and assume a slight
difference between antenna pairs, the gain can be approximated as
${g}^{\tt{MIMO}}_{i,\tau}(d_{i,\tau})=G{g}_{i,\tau}(d_{i,\tau})$, where $G$,
equals $\mathcal{N}_{UAV}\times$ $\mathcal{N}_{\tt{UE}}$, is the gain
resulting from applying multiple-input multiple-output (MIMO) antenna
configurations at the UAV and ${\tt{UE}}_{i}$. ${g}_{i,\tau}(d_{i,\tau})$ is
the channel gain.

Figure 1: System model.

Based on the principle of NOMA, the superposition coding (SC) is used at the
UAV to transmit signals to users located in the same cluster. SC encodes
different signals into a single signal while assigning them different power
values. The successive interference cancellation (SIC) is used at the receiver
side for signal detection. The received signal to interference plus noise
ratio at $\tau$ for ${\tt{UE}}_{i}$, SINRi,τ, is expressed as

$SINR_{i,\tau}=\frac{P_{T}G{g}_{i,\tau}(d_{i,\tau})\alpha_{i,\tau}}{P_{T}G{g}_{i,\tau}(d_{i,\tau})\beta_{i,\tau}+\sigma^{2}}$
(2)

where $\beta_{i,\tau}=\alpha_{i-1,\tau}$ if $i$ is even and zero otherwise.
$\sigma^{2}$ is the noise power. The first term in the denominator of (2)
represents the interference from the user with the stronger channel gain on
the other user in the same cluster. Using the SIC technique, however, the
interference from the user with the stronger channel gain gets removed. The
data rate of ${\tt{UE}}_{i}$ at $\tau$ is given by

$R_{i,\tau}=W\log_{2}(1+SINR_{i,\tau}),$ (3)

where $W$ is the communication bandwidth. The channel gain,
${g}_{i,\tau}(d_{i,\tau})$, between the UAV and $\tt{UE}_{i}$ at the mmWave
spectrum, in the presence of the LoS link [13], is expressed as [14]

${g}_{i,\tau}(d_{i,{\tau}})=Cd_{i,\tau}^{-a},$ (4)

where $a$ and $C$ are the path loss exponent and intercept, respectively. We
assume that the UAV collects the channel state information at the beginning of
each time step [15], in that a user can send a pilot signal prior to
transmission to allow the UAV to estimate $g_{i,\tau}$.

###  II-B UAV Mobility Model

The UAV is assumed to be located at $(x_{UAV,\tau},y_{UAV,\tau},h_{UAV,\tau})$
at time step $\tau$, and it is able to move, in the next time step, to
$(x_{UAV,\tau}+d_{x}\delta_{x},y_{UAV,\tau}+d_{y}\delta_{y},h_{UAV,\tau}+d_{h}\delta_{h})$,
where $d_{x}$, $d_{y}$, and $d_{h}$ $\in\\{1,-1\\}$. $\delta_{x}$,
$\delta_{y}$, and $\delta_{h}$ are the magnitude of change in the $x$, $y$,
and z axes, respectively. $h_{UAV,\tau}$ is assumed to have a minimum value of
$h_{0}$.

##  III 3D UAV Placement and Power Allocation Formulation

We propose to optimize the UAV placement and PA such that the total users'
data rate is maximized subject to a fairness condition imposed through
satisfying a minimum rate for each user, $R_{min}$. We define the total users'
data rate at $\tau$ as

$R_{\tau}^{\textrm{tot}}=\sum_{i=1}^{\tt{N_{UE}}}W\log_{2}(1+SINR_{i,\tau}).$
(5)

The optimization problem is formulated as follows

$\displaystyle\max_{x_{UAV,\tau},y_{UAV,\tau},h_{UAV,\tau},\alpha_{i,\tau}}R_{\tau}^{\textrm{tot}},$
(6a) $\displaystyle\alpha_{i,\tau}>0,\;\forall\;i\in\\{1,..,\tt{N_{UE}}\\},\;$
(6b)
$\displaystyle{\alpha}_{i,\tau}+{\alpha}_{i+1,\tau}=1,\;\forall\;i\in\\{1,3,..,\tt{N_{UE}}-1\\},\;$
(6c) $\displaystyle R_{i,\tau}\geq
R_{min},\;\forall\;i\in\\{1,..,\tt{N_{UE}}\\},\;$ (6d) $\displaystyle L/2\geq
x_{UAV,\tau}\geq-L/2,\;$ (6e) $\displaystyle L/2\geq y_{UAV,\tau}\geq-L/2,\;$
(6f) $\displaystyle h_{UAV,\tau}\geq h_{0}.\;$ (6g)

###### Remark 1

The problem is feasible if, in addition to the previously stated constraints,
for all $i\in\\{1,3,..,\tt{N_{UE}}-1\\}$, $\alpha_{i,\tau}$ satisfies the
following:

$\alpha_{i,\tau}\geq\frac{2^{{R_{min}}/{W}}-1}{SNR_{i,\tau}},$ (7)

where the received signal to noise ratio at $\tau$ for ${\tt{UE}}_{i}$,
$SNR_{i,\tau}$, is expressed as ${P_{T}\times{g}_{i,\tau}(d_{i,\tau})\times
G}/{\sigma^{2}}$. (7) follows from (2), (3), and (6d).

Solving the non-convex objective function presented in (6) is challenging. By
imposing the condition in (6d), we are making sure that the problem's solution
does not favor the sum rate over fairness, as improving the former may lead to
sacrificing servicing some users.

To solve this problem, however, we propose AI based framework which trains the
UAV to solve (6) efficiently and determine both the optimal PA and 3D UAV
placement accordingly.

##  IV  ProSky: A NEAT-based Framework for UAV Network Resource Management

Our proposed framework is designed to have a UAV environment learned and
modeled using a NN, which as a consequence determines the UAV 3D deployment
and NOMA PA decisions. In some AI techniques, DRL, for example, the NN
topology needs to be defined before training, which is usually hard to select
and tune for practical applications. The NN structure needs to be chosen such
that the trained model learns the environment well but does not overfit it.
During the training process, a NN's weights and biases are to be optimized to
minimize a cost function. The performance of a NN is affected by the choice of
the optimization method. The backpropagation method, for example, used in DRL,
suffers from the local minima problem. To avoid the aforementioned issues, we
design a novel framework that manages the NOMA-mmWave-UAV resources based on
NEAT. Unlike many other AI methods, NEAT evolves and optimizes the parameters
of a number of NNs or solutions simultaneously using GAs. In addition, NEAT
optimizes and complexifies topologies of NNs to allow complex solutions to
evolve. In this section, we provide the NEAT background and present how we
implemented it to solve the problem on hand effectively.

###  IV-A NEAT

In NEAT, a large set of variant-topology NNs, or generations, are tested on a
certain task and evaluated based on a certain reward function. Networks that
perform well are chosen to contribute, through crossover and mutation, in
making a new generation of NNs. Mutation occurs by randomly altering a
connection parameters, or adding a node or connection to a NN to ensure
diversity of solution and maturity of convergence. The evaluation and
evolution processes continue until a certain criterion related to a maximum
number of generations, or average reward is met. For the effectiveness of
NEAT, though, three main concerns are to be taken into account i) disparities
and similarities among NNs need to be well represented to crossover them
meaningfully, ii) NNs need not to disappear prematurely as they could improve
performance as they evolve, iii) an effective way that allows NNs to
complexify but only as performance demands to avoid slowing down learning is
required. To handle the aforementioned issues, NEAT utilizes some fundamental
techniques. One of which is historical marking. Every NN in NEAT is encoded
with two sets. One specifies all nodes, input, output, and hidden, and another
identifies all connections, their connecting node, weights, enablement status,
and an innovation number (IN) assigned uniquely to a connection when created.
In addition, every node is given a historical marking shared among all NNs
with the same node. Hence, NNs can be tracked, allowing topologies crossover.
Another idea of NEAT is speciation. To preserve solutions, NNs are separated
into different species. This allows NNs to compete with their species and
solve the issue of eliminating them prematurely. Complexifying is another key
technique for NEAT. Topologies in NEAT develop incrementally, only as needed,
starting from structures with no hidden layers, allowing for finding minimal
optimal solutions. We will utilize these techniques to solve the problem
stated in (6).

###  IV-B The ProSky Model

Our proposed framework is designed based on NEAT to make the UAV effectively
learn its environment and efficiently take mobility and PA actions such that
fairness and rate objectives are met. The main components of $\tt{ProSky}$ are
described as follows.

Initialization. To train $\tt{ProSky}$ effectively, we initialize a generation
of NNs of size $G_{s}$ structured from input and output nodes randomly
connected. Input and output represent system state and UAV actions
respectively, which need to be carefully designed to determine the most
effective NN model. At any time ${\tau}$ through out the learning process, a
state $s_{\tau}$ describes the relative locations of the UAV to each user,
user's power coefficient, and UAV-$\tt{UE}_{i}$ channel gain. $s_{\tau}$ is
defined as
$s_{\tau}=\big{[}s_{1,\tau}^{T},s_{2,\tau}^{T},...,s_{{\tt{N_{UE}}},\tau}^{T},h_{UAV,\tau}\big{]}^{T}$,
where $s_{i,\tau}=\big{[}\Delta x_{UAV-i,\tau},\Delta
y_{UAV-i,\tau},\alpha_{i,\tau},g_{i,\tau}(d_{i,\tau})\big{]}^{T}$,
$\forall\;i\in\\{1,..,\tt{N_{UE}}\\}$. $\Delta x_{UAV-i,\tau}$ and $\Delta
y_{UAV-i,\tau}$ are the x-axis and y-axis difference between the UAV and
$\tt{UE}_{i}$ locations respectively. The action at ${\tau}$ is defined as
$a_{\tau}=[d_{x}\delta_{x},d_{y}\delta_{y},d_{h}\delta_{h},\delta_{\alpha}^{T}]^{T}$
where $\delta_{\alpha}$ is defined as
$[\delta_{1},\delta_{3},...,\delta_{\tt{N_{UE}}-1}]^{T}$, where $\delta_{i}$
is the change in the PA coefficient of the $\tt{UE}_{i}$.

Evaluation. The learning process of $\tt{ProSky}$ occurs over $E$ episodes
with $T$ time steps each. At every time step a system state is fed to a NN, or
a UAV, which results in a corresponding action. Based on the resulted action,
the UAV is given a reward $r_{\tau}$ defined as follows.

$\begin{split}r_{\tau}=w_{r}\times\frac{R_{\tau}^{\textrm{tot}}}{W}\times\prod_{i=1}^{\tt{N_{UE}}}\mathds{1}_{\big{(}\displaystyle
R_{i,\tau}\geq R_{min}\big{)}}+w_{s}\times\\\
\sum_{i=1}^{\tt{N_{UE}}}\mathds{1}_{\big{(}\displaystyle R_{i,\tau}\geq
R_{min}\big{)}}+w_{u}\times\sum_{i=1}^{\tt{N_{UE}}}\frac{R_{i,\tau}}{W}\mathds{1}_{\big{(}\displaystyle
R_{i,\tau}<R_{min}\big{)}},\end{split}$ (8)

where $\mathds{1}_{(.)}$ is the indicator function. $w_{r}$, $w_{s}$, and
$w_{u}$, which take non negative values, are total rate, and satisfied and
unsatisfied minimum rate weights respectively. The reward is designed, with
some similarity to that proposed in [8] to facilitate comparison between our
NEAT and DRL frameworks, carefully to make the UAV learn efficiently. The
first term in (8) aims to increase the total sum rate only if all users meet
the minimum rate constraint. A reward of $w_{s}$, with a relatively large
value, contributes to $r_{\tau}$ to reinforce satisfying meeting the minimum
rate requirement for every user. In addition, users which do not satisfy
$R_{min}$ result in unsatisfied minimum rate reward which is directly
proportional to the sum of their rates. In other words, the UAV gets rewarded
for any improvements it makes for rates that are below $R_{min}$. The episode
reward for every NN in a generation determines which NNs evolve to the next
generation. The NNs with the highest reward are to be included for the next
generation. For the rest of NNs, the higher the average reward, the more the
probability it gets selected for the evolution process.

Evolution through crossover. Historical marking makes a meaningful crossover
possible. To crossover two NNs, they get aligned based on their INs.
Connections with identical IN, also referred to as matching connections, are
randomly selected to appear in the composed NN. Connections that do not match
are called disjoint if they are within the IN range of the other NN or excess
otherwise. All excess or disjoint connections are included from the NN that
achieved a higher average reward when crossing over.

Evolution through mutation. To diversify solutions, mutation, which either
alters existing connections or contribute a new structure to a network, is
implemented. When a new connection is added between two nodes, it is allocated
a random weight. When a new node is inserted between two existing connected
nodes. While the link between the previous start and new nodes keeps the old
connection parameters, the other link is assigned a weight of $1$.

Solution preservation. This component enables, through speciation, novel
topologies, which may initially perform poorly, to be constructed and
optimized without the worry of being destroyed before they can be fully
investigated. Speciation, basically, divides a generation into several species
based on topological and connectivity similarities. If a compatibility
distance, which is proportional to the weighted sum of the number of excesses,
disjoints, and the variations in weights of matching connection, is less than
a compatibility threshold $\delta_{th}$, a two NNs are said to belong to the
same species [10]. A number of species are created in the first episode and
sequentially ordered. At subsequent episodes, a random NN is selected to
represent each species. Any other generated NN is assigned to the first
species that is compatible with its representative NN. A new species is
created in case there is no compatibility.

Solutions reduction. To avoid having a species takes over the entire
generation by getting too big, an adjusted reward function is introduced [10].
The idea is that the reward of every solution in a species gets normalized by
the number of NNs belongs to the species. Each species gets assigned only
number of NNs proportional to the sum of the adjusted reward of corresponding
species members. That as a consequence may result in eliminating the solutions
with the lowest performance and reduce number of solutions within a species.

The last four components of $\tt{ProSky}$, which are mainly based on NEAT
[10], determine how NNs in $\tt{ProSky}$ are evolved. However, the
initialization and evaluation components are designed by us to get the most
out of NEAT and make it learn the 3D network environment effectively and
manage resources efficiently as we will show in the next section.

##  V ProSky Performance Evaluation

###  V-A Implementation Settings

$\tt{ProSky}$ is trained for $1000$ episodes, or generations, with $300$ time
steps each and $G_{s}$ of $50$ NNs which are assumed to be feed-forward,
fully-connected at the initialization, and activated using ReLU. Weight and
biases are generated from standard normal in a range of $[-30,30]$ with a
mutation rate $0.8$ and $0.7$ respectively. A node, and connection, add or
delete probabilities are all set to $0.2$. $\delta_{th}$ is chosen to be $3$.

The proposed framework is simulated for a $100\times 100$ $m^{2}$ urban area
with $\tt{N_{UE}}=4$. Users are located, relative to the center of
$\mathcal{A}$, at $(4,15)$, $(-44,-49)$, $(-5,21)$ and $(47,49)$. The UAV is
assumed to keep a minimum height of $10$ m, and, at the beginning of each
episode, to be deployed at $(0,0,50)$ and assign $0.5P_{T}$ for each user. The
change in the PA coefficient is set to $\pm 0.01$ for all users. $\delta_{x}$,
$\delta_{y}$, and $\delta_{h}$ are all set to be $1$ m. Channel is modeled
according to [14], where the path loss intercept and exponent are set to
$10^{-6.4}$ and $2$ respectively. Thermal noise power $\sigma^{2}$ is assumed
to be $-84$ dBm. The transmit power $P_{T}$ is set to $20$ dBm. Antenna
configurations $\mathcal{N}_{UAV}\times\mathcal{N}_{UE}$ are chosen to be
$8\times 8$. System bandwidth $W$ and carrier frequency $f_{c}$ are taken $2$
GHz and $28$ GHz respectively.

###  V-B Performance Analysis

In this subsection, we provide the performance of $\tt{ProSky}$ in managing 3D
NOMA-UAV network both in training and testing scenarios. We compare our
finding with the state-of-art technique provided in [3], which throughout the
section will be referred to as SoA. In [3], the authors solved the non-convex
NOMA PA and UAV placement problem, disjointly while restricting the UAV
placement to a 2D plane, set at a height of $50$ in our analysis, using the
conventional optimization framework.

In addition, we compare the $\tt{ProSky}$ performance with the DRL based
algorithm AdaptSky, not just because we are examining the performance of the
two different AI tools these algorithms are built on, but also because
AdaptSky, for the best of our knowledge, is the only AI based framework
proposed to solve the 3D UAV placement and NOMA PA jointly.

For evaluation, we introduce the performance metrics ${R}_{e}^{\textrm{tot}}$
defined as achieved average sum-rate per generation in case of $\tt{ProSky}$
and by averaging the average rate per episode and over the most recent $100$
episodes for AdaptSky. ${R}_{e}^{\textrm{tot}}$ AdaptSky equals zero for all
episodes $ep<100$. The average total rate varies drastically from episode to
another in case of AdaptSky, hence averaging over $100$ episodes was needed to
smooth out the data. Using some performance metrics, including
${R}_{e}^{\textrm{tot}}$, we evaluate $\tt{ProSky}$ training and testing
performance in the following aspects:

Figure 2: Episode training average rewards.

Learning rate. In order to show how efficient and fast $\tt{ProSky}$ can
handle the problem in hand, we, in Fig. 2, show the average reward for
$\tt{ProSky}$ and AdaptSky, as they both learn the UAV environment. The
average rewards is defined similar to ${R}_{e}^{\textrm{tot}}$, i.e., it is
the episode and $100$ episodes reward average for $\tt{ProSky}$and AdaptSky
respectively. The training parameters are set as ${R}_{min}/W=0.5$,
$w_{s}=100$, $w_{r}=1$, and $w_{u}=1$. To demonstrate the consistency of our
proposed method, the figure includes the confidence interval (CI) of one
standard deviation of $10$ different simulation runs. Interestingly,
$\tt{ProSky}$ achieves $96.17$% of the average reward convergence value more
than $5.3$ times faster than that for AdaptSky, even though AdaptSky is built
based on dueling architecture, which advances DRL learning rate [16]. This
speedup rate is $96\%$ higher than the average, and only $2\%$ off of the
highest, achieved end-to-end speedup rate of ActorQ network [17] which
leverages low-precision quantized actors to speed up the learning process.

Furthermore, based on Fig. 2, $\tt{ProSky}$ converges to a slightly better
reward, or total sum data rate, than AdaptSky with less uncertainty.

Figure 3: Average sum rate for different fairness requirements.

Rate and fairness (training). In this scenario, we vary the minimum SE
required ${R}_{min}/W$, as a way to impose fairness among users, and observe
${R}_{e}^{\textrm{tot}}$. We depict the convergence value of
${R}_{e}^{\textrm{tot}}$ over a thousand training episodes in Fig. 3 and show
the CI of $5$ runs. The reward function in this scenario is structured to give
a higher reward whenever a user exceeds ${R}_{min}$, which can be achieved
through setting the rewards parameters as $w_{r}=10$, $w_{s}=100$, and
$w_{u}=1$. We can observe from Fig. 3 that $\tt{ProSky}$ outperforms SoA, very
consistently i.e. with tight CI, and improvement of up to $59.23\%$. In
addition, when the minimum SE is less than or equal $2$ bit/s/Hz,
$\tt{ProSky}$ outperforms AdaptSky with average of $5.85$% which corresponds
to $1.99$ Gbit/s improvement. The uncertainty at a higher SE minimum
requirement is expected as $\tt{ProSky}$ finds the best model by parallelly
trying number of random solutions which may lead to a lower average
performance than AdaptSky, as feasible solutions are getting very limited.

Energy efficiency (testing). In this scenario, we train $\tt{ProSky}$ to
optimize the sum rate objective with a setup similar to that in the learning
rate scenario, but with $R_{min}$=0. The best NN model from training is
imported and tested for every power in the range $-20$ dBm to $80$ dBm with a
step size of $0.1$ dBm, over $300$ time steps through which the UAV interacts
with the observed states. We determine the average SE over all time steps and
calculate the corresponding EE, defined as SE divided by power required for
signal transmission plus some other consumed power assumed to take the value
of $40$ dBm. In Fig. 4, we plot the EE for $\tt{ProSky}$ and the two
baselines, which $\tt{ProSky}$ obviously outperforms. At the green point, the
highest EE point in the curves, $\tt{ProSky}$ shows an improvement of $21.93$%
and $35.71$% over AdaptSky and SoA respectively.

Figure 4:  Energy efficiency performance during testing.

$\tt{ProSky}$ exhibits a fast adapting learning model and offers, while
meeting fairness requirements, huge gain in terms of SE and EE over the model-
based solution proposed in [3], which seem to fall short in handling
complicated networks. Similarly, $\tt{ProSky}$ demonstrates some improvement
over the DRL-based algorithm proposed in [8] in terms of SE and EE, and
significant enchantment in learning rate.

##  VI Conclusion and Discussion

In this paper, we proposed $\tt{ProSky}$, a novel AI-based framework built on
NEAT algorithm which merges GA and deep learning. $\tt{ProSky}$ optimizes the
UAV 3D location while allocating NOMA resources in the mmWave spectrum such
that fairness is guaranteed and the users' sum rate is maximized.
$\tt{ProSky}$ significantly outperforms a model-based scheme, in terms of SE
and EE. Moreover, $\tt{ProSky}$ has shown a huge gain in learning speed and
some gains in SE and EE over a DRL-based baseline. To get the most out of
integrating GA and deep learning, nonetheless, and to draw a more
comprehensive performance analysis, more studies about the applicability of
NEAT and its advances in more generic channel models in multi-UAV networks are
encouraged considering the improvement our findings exhibited in resources
management. We believe NEAT will require more investigations in terms of the
NN evolution and selection process in such cases.

## References

  * [1] E. Yaacoub and M.-S. Alouini, “A key 6G challenge and opportunity—connecting the base of the pyramid: A survey on rural connectivity,” _Proceedings of the IEEE_ , vol. 108, no. 4, pp. 533–582, 2020.
  * [2] N. Adem, A. Benfaid, R. Harib, and A. Alarabi, “How crucial is it for 6G networks to be autonomous?” _arXiv preprint arXiv:2106.06949_ , 2021.
  * [3] X. Liu, J. Wang, N. Zhao, Y. Chen, S. Zhang, Z. Ding, and F. R. Yu, “Placement and power allocation for NOMA-UAV networks,” _IEEE Wireless Communications Letters_ , vol. 8, no. 3, pp. 965–968, 2019.
  * [4] M. Monemi, H. Tabassum, and R. Zahedi, “On the performance of non-orthogonal multiple access (NOMA): Terrestrial vs. aerial networks,” in _2020 IEEE Eighth International Conference on Communications and Networking (ComNet)_ , 2020, pp. 1–8.
  * [5] P. K. Sharma and D. I. Kim, “UAV-enabled downlink wireless system with non-orthogonal multiple access,” in _2017 IEEE Globecom Workshops (GC Wkshps)_. IEEE, 2017, pp. 1–6.
  * [6] H. El Hammouti, M. Benjillali, B. Shihada, and M.-S. Alouini, “Learn-as-you-fly: A distributed algorithm for joint 3D placement and user association in multi-UAVs networks,” _IEEE Transactions on Wireless Communications_ , vol. 18, no. 12, pp. 5831–5844, 2019.
  * [7] C. H. Liu, Z. Chen, J. Tang, J. Xu, and C. Piao, “Energy-efficient UAV control for effective and fair communication coverage: A deep reinforcement learning approach,” _IEEE Journal on Selected Areas in Communications_ , vol. 36, no. 9, pp. 2059–2070, 2018.
  * [8] A. Benfaid, N. Adem, and B. Khalfi, “AdaptSky: a DRL based resource allocation framework in NOMA-UAV networks,” in _2021 IEEE Global Communications Conference (GLOBECOM)_. IEEE, 2021, pp. 01–07.
  * [9] K. O. Stanley, J. Clune, J. Lehman, and R. Miikkulainen, “Designing neural networks through neuroevolution,” _Nature Machine Intelligence_ , vol. 1, no. 1, pp. 24–35, 2019.
  * [10] K. O. Stanley and R. Miikkulainen, “Evolving neural networks through augmenting topologies,” _Evolutionary computation_ , vol. 10, no. 2, pp. 99–127, 2002.
  * [11] H. Kang Kim, R. Becerra, S. Bolufé, C. A. Azurdia-Meza, S. Montejo-Sánchez, and D. Zabala-Blanco, “Neuroevolution-based adaptive antenna array beamforming scheme to improve the V2V communication performance at intersections,” _Sensors_ , vol. 21, no. 9, p. 2956, 2021\.
  * [12] Y. Liu, Z. Qin, Y. Cai, Y. Gao, G. Y. Li, and A. Nallanathan, “UAV communications based on non-orthogonal multiple access,” _IEEE Wireless Communications_ , vol. 26, no. 1, pp. 52–57, 2019.
  * [13] B. Li, Z. Fei, and Y. Zhang, “UAV communications for 5G and beyond: Recent advances and future trends,” _IEEE Internet of Things Journal_ , vol. 6, no. 2, pp. 2241–2263, 2018.
  * [14] M. R. Akdeniz, Y. Liu, M. K. Samimi, S. Sun, S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter wave channel modeling and cellular capacity evaluation,” _IEEE journal on selected areas in communications_ , vol. 32, no. 6, pp. 1164–1179, 2014.
  * [15] M. M. El-Sayed, A. S. Ibrahim, and M. M. Khairy, “Power allocation strategies for non-orthogonal multiple access,” in _2016 International Conference on Selected Topics in Mobile & Wireless Networking (MoWNeT)_. IEEE, 2016, pp. 1–6.
  * [16] Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in _International conference on machine learning_. PMLR, 2016, pp. 1995–2003.
  * [17] S. Krishnan, M. Lam, S. Chitlangia, Z. Wan, G. Barth-maron, A. Faust, and V. J. Reddi, “QuaRL: Quantization for fast and environmentally sustainable reinforcement learning.”


# Charge Density Wave bending observed by Xfel source acting as a tunable
electronic lens for hard x-rays

E. Bellec  CEA Grenoble, IRIG, MEM, NRS, 17 rue des Martyrs, F-38000 Grenoble,
France  D. Ghoneim  Laboratoire de Physique des Solides, Université Paris-
Saclay, CNRS, 91405 Orsay, France  European X-Ray Free-Electron Laser
Facility, D-22869 Schenefeld, Germany  A. Gallo  Laboratoire de Physique des
Solides, Université Paris-Saclay, CNRS, 91405 Orsay, France  V.L.R. Jacques
Laboratoire de Physique des Solides, Université Paris-Saclay, CNRS, 91405
Orsay, France  I. Gonzalez-Vallejo  Max-Born-Institut fur Nichtlineare Optik
und Kurzzeitspektroskopie, 12489 Berlin, Germany  L. Ortega  Laboratoire de
Physique des Solides, Université Paris-Saclay, CNRS, 91405 Orsay, France  M.
Chollet  Linac Coherent Light Source, SLAC National Accelerator Laboratory,
2575 Sand Hill Road, Menlo Park, CA 94025, USA  A. Sinchenko  Laboratoire de
Physique des Solides, Université Paris-Saclay, CNRS, 91405 Orsay, France  D.
Le Bolloc’h  Laboratoire de Physique des Solides, Université Paris-Saclay,
CNRS, 91405 Orsay, France

###### Abstract

Ultrafast X-ray diffraction by the LCLS free-electron laser has been used to
probe Charge Density Wave (CDW) systems under applied external currents. At
sufficiently low currents, CDW wavefronts bend in the direction transverse to
the 2kF wave vector. We show that this shear effect has the ability to focus
or defocus hard X-ray beams, depending of the current direction, making it an
electronic lens of a new kind, tunable at will from the Fraunhofer to the
Fresnel regime. The effect is interpreted using the fractional Fourier
transform showing how the macroscopic curvature of a nanometric modulation can
be beneficially used to modify the propagation of X-ray beams.

###### pacs:

##  I Introduction

A incommensurate Charge Density Wave (CDW) may be highly sensitive to even
small external excitation. It can then appear independent with respect to the
atomic host lattice that yet supports it. For example when NbSe3 is cooled
down, the atomic lattice contracts slightly while the CDW’s period increases
considerably [1, 2]. In the case of rare-earth tritellurides, the CDW can even
flip 90° after an ultra short femto-laser pulse, and return to the equilibrium
position in less than few ps [3, 4]. The CDW can also tilt in the
perpendicular direction after elongating the lattice parameters by less than
1$\%$ by using a tensile machine [5, 6].

A CDW is also very sensitive to external currents. Indeed, when a small
external current is applied, the CDW is submitted to a force due to the
applied field because the CDW is naturally charged. Two deformations are then
observed. The first which concerns the longitudinal direction along the 2kF
wave vector, compresses the CDW near one electrode and expands it at the other
[7, 8]. The second deformation, more recently observed, is located far from
the electrodes. It involves the transverse direction and corresponds to a
curvature of the CDW wavefronts [9, 10]. CDW’s defects are described by
considering only the phase of the modulation
$\eta\cos(2k_{F}x+\Phi(\vec{r}))$[11]. In this specific case, involving a
continuous deformation, contraction, dilatation and shear, a quadratic phase
$\Phi(\vec{r})$ has to be considered, depending of the applied force $f$,
either along the longitudinal direction ($\Phi(x)=f(x^{2}-l^{2}_{x})$) or the
transverse one ($\Phi(y)=f(y^{2}-l^{2}_{y})$) where $2l_{x}$ and $2l_{y}$ are
the sample dimensions in the two perpendicular directions.

The transverse shear under current has been observed in NbSe3 by X-ray micro-
diffraction. Fast mapping techniques were used to get the quadratic CDW’s
phase by phase gradient method. The observed curvature of CDW wavefronts
increases with rising currents and reverses with inverse currents, until a
current threshold is reached, beyond which relaxation is observed. The CDW in
NbSe3 is then able to bend continuously over more than 20$\mu m$, that is a
distance 104 larger than its $\lambda$=14$\AA$ period. This phenomenon
illustrates the extent to which a CDW is able to deform continuously from one
edge to the other edge over macroscopic distances, despite the presence of
numerous defects. This effect can be understood by CDW pinning on sample
surfaces [10].

In this letter, we have undertaken to observe this same phenomenon, but with a
single illumination, from a beam wide enough and fully coherent delivered by
the LCLS XFEL source to see the curvature as a whole. We show in the following
that the CDW bending induced by the small external current modifies the beam
propagation from the Fresnel to the Fraunhofer regime, allowing to
continuously focus hard x-ray beams.

Figure 1: a) Curvature of the CDW wavefronts observed in NbSe3 when the sample
is submitted to a current of I=-1mA. This image has been obtained from a fast
mapping diffraction experiment using a micrometric x-ray beam (from [9]). For
better visibility, the CDW wavelength $\lambda$ has been considerably
increased to match the sample size. b) Experimental configuration s
illustrating the wide angles diffraction setup. Figure 2: c) (0 1.241 0)
satellite reflections associated with the CDW in NbSe3 with increasing
currents. All figures correspond to the integrated intensity over the complete
rocking curve. d) Transverse profiles after vertical integration (bleue
curves) as well as the fit (red curves) based on the model described in the
text.

Femtosecond X-ray diffraction experiment has thus been performed to probe the
incommensurate NbSe3 system submitted to several $dc$ currents. The two
electrical contacts needed for in situ resistivity measurements were located
at more than 500$\mu m$ from each other along the 39$\mu m$ $\times$ 3$\mu m$
$\times$2.25$mm$ single crystal, itself glued on a sapphire substrate. The
connected sample was inserted into a cryostat mounted on the XCS
diffractometer and cooled down to 80K, below the first CDW transition
(Tc1=145K). The satellite reflections associated with the CDW are located at
$\pm\vec{q}_{2kF}$ from each Bragg reflection with the incommensurate wave
vector $q_{2kF}=0.243\pm 0.001(\times\frac{2\pi}{b}$). The threshold current,
measured during the experiment, was equal to Is = 0.8 mA. Each 8 keV x-ray
pulses, of less than 50fs duration, were focused on a spot of about 30$\mu m$
in diameter on the sample, that is slightly smaller than the sample width. The
diffracted intensity was recorded with a 55$\mu$m$\times$55$\mu$m pixel size
detector located 8m further in the horizontal plane (see figure 1b). The Bragg
peaks reflect properties of the host atomic lattice, while the satellite
reflections associated with CDW contain information about both the host
crystal lattice and the CDW modulation. Therefore, rocking curves were made
for the $Q_{020}$ Bragg and the (0 $1+\vec{q}_{2kF}$ 0) satellite reflection
for several currents, up to Is.

Figure 3: a) Slit diffraction from the Fresnel to the Fraunhofer regime
calculated by the FrFT for $\alpha\in[0,\pi/2]$ and 2 different forces of
oposite signs and below, the corresponding real space showing the respective
CDW curvatures. b)-d) Corresponding beam profiles for the 3 forces calculated
at a fixed $\alpha$ at the dotted line position. e) Intensity profile at q=0
versus $\alpha$ and forces $f$. The Fresnel-Frauhofer transition is shifted to
smaller or larger $\alpha$ values or, equivalently, to smaller or larger
distances from the sample by changing the sign of the current applied to the
sample.

The combined effect of the applied electric field and surface pinning induces
curvature of CDW wavefronts. As shown in Fig.2, this shear effect has a
significant incidence on the diffraction pattern. As currents increase, the
beam broadens considerably in the transverse direction and displays
characteristic profiles that can easily be explained by diffraction from
curved CDW wavefronts. The diffraction profile of the 2kF satellite obtained
by the large and fully coherent LCLS xfel beam is then obtained from the
Fourier Transform of a whole curved modulation, whose exact solution is given
in 1D by:

$\begin{split}I(q)\propto&(Erfi[\frac{(1+i)(2af-q)}{2\sqrt{(}2f)}]-Erfi[\frac{(1+i)(2bf-q)}{2\sqrt{(}2f)}])\\\
&\circledast Gauss(q,\sigma)\end{split}$ (1)

considering that the x-ray beam less wide than the sample width, extending
from $a$ to $b$, and a incident Gaussian beam of width $\sigma$. The fit is
shown in Fig.2 , which reproduces the main characteristics of the diffraction
profiles obtained for different currents by changing only the curvature.

The expression (1) of the Fourier Transform (FT) of a modulation including a
quadratic phase term is exact but not explicit. On the other hand the use of
the Fractional Fourier Transform (FrTF) is a more elegant approach to dealing
with space curvature and light propagation. This approach is based on the fact
that since the standard Fourier transform must be applied 4 times successively
to recover the initial object, the FT is nothing else but an angular
transformation of $\alpha=\pi/2$ in the (x, q) plane. A generalized Fourier
transform can thus be developed for any angle of rotation $\alpha$ from 0 to
$\pi/2$, covering the complete propagation of light from the Fresnel to the
Fraunhofer regime. The integral form of the fractional Fourier transform of
any function $f(x)$ can be written in 1D as [12]:

$\mathcal{F}_{\alpha}[g](q)=\int_{-\infty}^{\infty}K(q,x)g(x)du$ (2)

with

$K(q,x)=\sqrt{\frac{1-i\cot\alpha}{2\pi}}\exp[{-i\frac{qx}{\sin\alpha}+\frac{i}{2}(q^{2}+x^{2})\cot\alpha}]$

The expression of the standard Fourier transform is easily recovered for
$\alpha=\pi/2$.

The FrFT is a generalization of the ordinary FT based on a rotational angle in
the usual phase space with applications in optics[13] and x-ray
diffraction[14]. Let’s applied the FrFT to the diffraction of rectangular slit
by considering a quadratic phase along the transverse direction $y$. The 1D
case reads:

$\mathcal{F}_{\alpha}^{f}(q)\propto\int_{-a}^{a}K(q,y)\exp[i2k_{F}y]\exp[if(y^{2}-a^{2})]dx$
(3)

where $f=qE$ is the force induced by the applied current and $\pm a$ the two
sample boundaries (or the beam size). The effect of the finite and imperfect
lattice and the incident beam profile on the diffraction pattern act like a
convolution in the Fraunhofer regime and are neglected here.

The advantage of using FrFT to deal with space curvature then becomes obvious,
since two quadratic terms are present in the integral form. As a consequence,
the curvature of the CDW wavefronts under an applied force is equivalent to
changing the $\alpha$ angle with the force $f$ into another $\alpha^{\prime}$
without force,
$\mathcal{F}^{f}_{\alpha}(q)=\mathcal{F}^{0}_{\alpha\prime}(q^{\prime})$ with:

$\cot\alpha^{\prime}=\cot\alpha+2f$

and to rescale in $q$ with the scale factor:

$q^{\prime}/q=\frac{1}{\sqrt{1+(\cot\alpha+2f)^{2}}}$

We show in Fig.3, the consequence on the diffraction pattern. The slit
diffraction from the Fresnel to the Fraunhofer regime calculated by the FrFT
is shown without force $f$. The two regimes are easily accessible from slits
closed at few tens of micrometers using keV x-ray beams. By changing $f$, the
x-ray propagation is shifted in time, either dilated or contracted depending
of the sign of $f$, i.e. depending of the sign of the CDW curvature. As a
consequence, if the detector is located in between the two regimes, between
Fresnel and Fraunhofer, the diffracted beam will expand and display a
characteristic Fresnel profile for positive currents. On the contrary, for
negative one, the beam profile will approach the Fraunhofer regime depending
of the amplitude of the curvature. It will then be focussed, displaying a
cardinal sinus square profile (see Fig.2).

Two things are worth noting. First, when $\alpha=\pi/2$, i.e. when the
detector is located in the Fraunhofer regime, the sign of the curvature no
longer plays any role in the diffraction profile: positive and negative
currents, of the same intensity, will lead to the same profile, approaching
the Fresnel regime. Second, changing the order of diffraction, measuring (0
$1+\vec{q}_{2kF}$ 0) rather than (0 $1+\vec{q}_{2kF}$ 0) reflection, is
equivalent to inverse the sign of the curvature.

Focusing beams in the hard x-ray regime is not an easy task and has always
been a major limitation in the field of x-rays, especially for large
instruments (synchrotron and XFel sources). To focus high energy x-ray beams
is now achievable either by using reflective (Kirkpatrick–Baez systems,
capillaries, waveguides[15]), refractive (Be lenses[16]) and diffractive
(Fresnel zone plate[17]) optics. This work opens up a new class of focusing
optics, thanks to the ability of a CDW to bend when subjected to a low
current. The resulting nanometric modulation, bent over tens of micrometers,
then acts as a versatile electronic lens, tunable at will and operating in a
wide energy range. Since the direction of curvature and its amplitude can be
controlled by the sign and intensity of the applied current, the diffracted
beam can be focused or defocused, from the Fraunhofer to the Fresnel regime.
However, this study remains a demonstration of principle because the intensity
of the satellite reflection is probably too weak in this compound, but other
systems displaying similar properties could prove more effective.

###### Acknowledgements.

The authors would like to thank G. Abramovici for his help and F. Polack for
his always pertinent optician’s comments.

## References

  * Hodeau  _et al._ [1978] J. Hodeau, M. Marezio, C. Roucau, R. Ayroles, A. Meerschaut, J. Rouxel, and P. Monceau, Charge-density waves in nbse3 at 145k: crystal structures, x-ray and electron diffraction studies, Journal of Physics C: Solid State Physics 11, 4117 (1978).
  * Moudden  _et al._ [1990] A. H. Moudden, J. D. Axe, P. Monceau, and F. Levy, ${\mathit{q}}_{1}$ charge-density wave in ${\mathrm{nbse}}_{3}$, Phys. Rev. Lett. 65, 223 (1990).
  * Kogar  _et al._ [2020] A. Kogar, A. Zong, P. Dolgirev, X. Shen, J. Straquadine, Y. Bie, X. Wang, T. Rohwer, I. Tung, Y. Yang, R. Li, S. Weathersby, S. Park, M. Kozina, E. Sie, H. Wen, P. Jarillo-Herrero, I. Fisher, X. Wang, and N. Gedik, Light-induced charge density wave in late3, Nature Physics 16, 159 (2020).
  * Gonzalez-Vallejo  _et al._ [2022] I. Gonzalez-Vallejo, V. L. R. Jacques, D. Boschetto, G. Rizza, A. Hadj-Azzem, J. Faure, and D. Le Bolloc’h, Time-resolved structural dynamics of the out-of-equilibrium charge density wave phase transition in gdte3, Structural Dynamics 9, 014502 (2022).
  * Straquadine  _et al._ [2022] J. A. W. Straquadine, M. S. Ikeda, and I. R. Fisher, Evidence for realignment of the charge density wave state in ${\mathrm{erte}}_{3}$ and ${\mathrm{tmte}}_{3}$ under uniaxial stress via elastocaloric and elastoresistivity measurements, Phys. Rev. X 12, 021046 (2022).
  * Gallo-Frantz  _et al._ [2023] A. Gallo-Frantz, A. Sinchenko, D. Ghoneim, L. Ortega, P. Godard, P. Renault, P. Grigoriev, A. Hadj-Azzem, P. Monceau, D. Thiaudière, B. Bellec, V. Jacques, and D. Le Bolloc’h, Charge-density-waves tuned by crystal symmetry, Archiv e-prints, arXiv.2203.08117 https://arxiv.org/abs/2306.15712 (2023).
  * DiCarlo  _et al._ [1993] D. DiCarlo, E. Sweetland, M. Sutton, J. D. Brock, and R. E. Thorne, Field-induced charge-density-wave deformations and phase slip in ${\mathrm{nbse}}_{3}$, Phys. Rev. Lett. 70, 845 (1993).
  * Requardt  _et al._ [1998] H. Requardt, F. Y. Nad, P. Monceau, R. Currat, S. Lorenzo, J. E.and Brazovskii, N. Kirova, G. Grübel, and C. Vettier, Direct observation of charge density wave current conversion by spatially resolved synchrotron x-ray studies in ${\mathrm{nbse}}_{3}$, Phys. Rev. Lett. 80, 5631 (1998).
  * Bellec  _et al._ [2020a] E. Bellec, I. Gonzalez-Vallejo, V. L. R. Jacques, A. A. Sinchenko, A. P. Orlov, P. Monceau, S. J. Leake, and D. L. Bolloc’h, Evidence of charge density wave transverse pinning by x-ray micro-diffraction, Phys. Rev. B 101, 125122 (2020a), arXiv:1905.03499 [cond-mat.str-el] .
  * Bellec  _et al._ [2020b] E. Bellec, V. Jacques, J. Caillaux, and D. Le Bolloc´h, The essential role of surface pinning in the dynamics of charge density waves submitted to external dc fields, Eur. Phys. J. B 93, 165 (2020b).
  * Le Bolloc’h  _et al._ [2023] D. Le Bolloc’h, E. Bellec, N. Kirova, and V. Jacques, Tracking defects of electronic crystals by coherent x-ray diffraction, Symmetry 15, 1449 (2023).
  * V. Namias [1980] V. Namias, The fractional fourier transform and its application in quantum mechanics, J. Inst. Math. Its Appl. 25, 241–265 (1980).
  * A.W. Lohmann [1993] A.W. Lohmann, Image rotation, wigner rotation, and the fractional fourier transform, J. Opt. Soc. Am. A 10, 2181 (1993).
  * Le Bolloc’h and Sadoc [2011] D. Le Bolloc’h and J. F. Sadoc, X-ray coherent diffraction interpreted through the fractional fourier transform, The European Physical Journal B 81, 481 (2011).
  * Salditt and Osterhoff [2020] T. Salditt and M. Osterhoff, X-ray focusing and optics, in  _Nanoscale Photonic Imaging_, edited by T. Salditt, A. Egner, and D. R. Luke (Springer International Publishing, Cham, 2020) pp. 71–124.
  * Snigirev  _et al._ [1996] A. Snigirev, V. Kohn, I. Snigireva, and B. Lengeler, A compound refractive lens for focusing high-energy x-rays,  384, 49 (1996).
  * Niemann  _et al._ [1974] B. Niemann, D. Rudolph, and G. Schmahl, Soft x-ray imaging zone plates with large zone numbers for microscopic and spectroscopic applications, Optics Communications 12, 160 (1974).


# Image and AIS Data Fusion Technique
for Maritime Computer Vision Applications

Emre Gülsoylu
University of Hamburg
Paul Koch
Fraunhofer CML
Mert Yıldız
Fraunhofer CML
Manfred Constapel
Fraunhofer CML
André Peter Kelm
University of Hamburg
{emre.guelsoylu<EMAIL_ADDRESS>
{paul.koch, mert.yildiz<EMAIL_ADDRESS>
###### Abstract

Deep learning object detection methods, like YOLOv5, are effective in
identifying maritime vessels but often lack detailed information important for
practical applications. In this paper, we addressed this problem by developing
a technique that fuses Automatic Identification System (AIS) data with vessels
detected in images to create datasets. This fusion enriches ship images with
vessel-related data, such as type, size, speed, and direction. Our approach
associates detected ships to their corresponding AIS messages by estimating
distance and azimuth using a homography-based method suitable for both fixed
and periodically panning cameras. This technique is useful for creating
datasets for waterway traffic management, encounter detection, and
surveillance. We introduce a novel dataset comprising of images taken in
various weather conditions and their corresponding AIS messages. This dataset
offers a stable baseline for refining vessel detection algorithms and
trajectory prediction models. To assess our method’s performance, we manually
annotated a portion of this dataset. The results are showing an overall
association accuracy of 74.76 %, with the association accuracy for fixed
cameras reaching 85.06 %. This demonstrates the potential of our approach in
creating datasets for vessel detection, pose estimation and auto-labelling
pipelines.

##  1 Introduction

Maritime computer vision datasets are limited [29], particularly due to costly
manual annotation and lack of available images. Considering that the state-of-
the-art Deep Learning (DL) object detection approaches like YOLOv5 [13]
perform well for localizing maritime vessels, a technique to create maritime
computer vision datasets can be developed. However, these object detection
models can only provide limited maritime-related classes out-of-the-box.
Applications like maritime autonomous surface ships (MAAS) require
contextualized information such as the type, dimensions, speed, and course of
a vessel [17]. A system that matches AIS data with detected vessels from an
object detection algorithm can help estimate the pose of the vessel, support
in performing collision avoidance [5] and aid in the creation of extensive
maritime computer vision datasets.

In this paper, an easy to use technique that fuses Automatic Identification
System (AIS) data with webcam-based images was developed. The technique adapts
an already existing homography based coordinate transformation to associate
detected ships in image space and available location information in world
coodinates [8]. However, this paper extends the existing technique for
rotating cameras. The proposed technique enables a scalable way to fuse images
from fixed and panning cameras with the extensive amount of data provided by
AIS. This type of system can support the creation of maritime computer vision
datasets with less to no human intervention needed. Moreover, alterations on
the technique have been made to ensure accurate spatial resolution when
transforming from image-to-world coordinates.

A dataset has been created with this method and made publicly available
https://github.com/egulsoylu/image-ais-fusion.

##  2 Background

###  2.1 Automatic Identification System

The Automatic Identification System [15] is a radio-based communication system
for the exchange of ship related parameters between ships and vessel traffic
services (VTS). AIS was initially introduced to improve efficiency and safety,
especially for collision avoidance in maritime navigation [12]. Besides
collision avoidance, it is currently used for various purposes, including
maritime surveillance, [17], waterway management [11], and environment
protection [20]. As defined in the International Convention for the Safety of
Life at Sea (SOLAS) [15], the transmission of AIS messages is mandatory for
certain vessels, including cargo ships, passenger vessels, and in some cases,
fishing vessels. Since there are webcams based on the banks of the Elbe river
that provide publicly available video streams, a huge volume of images can be
collected and fused with AIS data.

Although AIS messages have issues with reliability and manipulation, they
still provide valuable data for maritime navigation. Some of the well-known
issues of AIS are uncertainty associated with technical equipment for Global
Positioning System (GPS) [19], temporal differences [12], and malicious crew
that switches off the transceiver [18]. Moreover, data redundancy [21], and
noise [26] are the problems mentioned by previous studies. The range and the
equipment significantly impact GPS-related issues, which can be minimized by
applying trajectory correction methods [27, 30]. As AIS messages contain
detailed information about the vessel, matching images with AIS data can
benefit maritime informatics in training an object detection model to infer
more information from an image and support marine traffic against the
reliability issues of AIS.

###  2.2 Object Detection for Vessels

Object detection models can effectively localize and classify vessels in
various images. Adaptations of DL vessel detection methods for maritime
applications include Gupta et al.’s integration of Support Vector Machine, bag
of features, and CNNs [14], and Li et al.’s lightweight model modifying YOLOv3
[16]. Chang et al. enhanced YOLOv3 for better ship detection in both visible
and infrared images [9], and Xie et al. developed a more efficient network by
integrating YOLOv4 with several advanced techniques, achieving high
performance with significantly fewer parameters [31].

As can be seen, the vessel detection field is very active, with different
methods being proposed on a regular basis, most of which depends on YOLO
family object detection methods. In terms of vessel localization and maritime-
related classification, these modifications improves the performance and
sometimes efficiency. However, complex tasks require more than classifying a
vessel type and this paper focuses on fusion of bounding boxes with AIS data.
Therefore, a more recent version of a general purpose object detection model,
YOLOv5 was fine-tuned for this paper.

###  2.3 Fusion of Images and AIS Data

The literature shows four directly related works on the fusion of camera-based
images and AIS data. National Land Survey of Finland worked on sensor fusion
for autonomous vessel navigation [3], funded by European Space Agency.
However, this work’s details are unclear due to the lack of published
material.

Lu et al. [17] introduced a framework for vessel identification by fusing
images with AIS data. They proposed a distance estimation method based on the
ship size by monocular vision. To improve the fusion accuracy, a method that
is commonly used by the seaman called Dead Reckoning (DR) was utilised. It is
a method to predict the current position based on a ship’s previously observed
speed, heading, and course. The most significant limitation of this work is
the distance and bearing estimation, which leads to lower accuracy in matching
the bounding boxes of vessels with related AIS messages. As the distance is
predicted by the ship’s overall length (LOA) and the width of the detected
bounding box, the accuracy of distance estimation depends on the ship’s
heading and is highly sensitive to the localisation performance. Since the
dimensions of bounding boxes can be affected by the background, water
reflections, and weather conditions in general, distance estimation based on
bounding boxes is not reliable. Moreover, this method produces incorrect
results when the azimuth angle between the ship’s heading and the camera is
small. In other words, if a vessel is perpendicular to the camera’s image
plane, the width of the bounding box will represent the vessel’s breadth
instead of LOA.

Qu et al. [22] introduced another framework for the fusion of camera-based
vessel detection and AIS data. They utilise a YOLO-based vessel detection
network and arrival time estimation to associate AIS messages with the
detected vessels. The arrival time estimation takes the distance between the
camera and a vessel into account for predicting the time the vessel will be in
the field of view.

Carillo-Perez et al. [8] propose a method for georeferencing ship masks that
uses the homography method to transform coordinates from image coordinates to
world coordinates along with a novel dataset for maritime monitoring that
contains images with taken by a static camera. They investigated instance
segmentation methods focusing on more robust performance such as Mask-RCNN and
DetectoRS as well as faster models such as YOLACT, Centermask-Lite. Their
method of calculating the homography matrix relies on manually detecting
antennas on each vessel in the image and matching them to the associated AIS
message. The creation of pairs of image coordinates and world coordinates in
this way is based on an assumption: The location information from the AIS data
is reliable. However, as discussed in section 2.1, AIS data may not be
reliable and may lead to errors in the estimation of the homography matrix.
Also, this method would only work for fixed cameras, and more steps are
required to apply it to a panning camera.

In this paper, we have generalised the method proposed by Carillo-Perez et al.
[8] on the panning cameras and created a novel dataset using this method which
is publicly available. Moreover, we created image space-world coordinates
pairs for the calculation of homography matrix with artificial structures that
does not move such as concrete levee crowns, pier poles and corners of
buildings. The improvements on georeferencing method using homography is
discussed in Section 3 and the details about the dataset is presented in
Section 3.4.

##  3 Methodology

The proposed procedure has two main steps, (1) vessel detection and (2)
coordinate transformation to match vessel images with related AIS messages.
First, the images are preprocessed to filter irrelevant images out. Amongst
the filtered images, an image is fed into a fine-tuned YOLO model and bounding
boxes are extracted. The image’s timestamp is taken with the camera location,
and AIS messages are filtered based on the time and location information. Then
bounding boxes are matched with AIS messages, and the results are saved. The
overview of the pipeline is shown in Figure 1.

Figure 1: The pipeline for the technique. The technique takes an image, its
timestamp and AIS messages of that day. After generating bounding boxes of the
vessels on the image and filtering the AIS data, the system associates the
predicted locations with the predicted bounding boxes. Red labels represent
the location prediction based on AIS data and the unique identifier number.
Orange labels represent the bounding box centre and associated unique
identifier number.

###  3.1 Object Detection for Vessels

Object detection for vessels is one of the two main prerequisites for the
fusion of images and AIS messages. As mentioned in Section 2.2, various YOLO
models and their modified variants have been successfully used for vessel
detection. Also, Lu et al. [17] demonstrated that a fine-tuned YOLOv5 model
performs well for maritime surveillance. In this paper, similar to Lu et al.’s
work, a YOLOv5XL model was fine-tuned with the manually annotated dataset by
changing the last layer from 80 neurons to only one, which represents the
”Vessel” class. The actual vessel types are redundant as this information can
be extracted from successfully matched AIS messages. Before fine-tuning, the
mean average precision (mAP) of the model was 0.332 at an intersection over
union (IoU) threshold of 0.5 and fine-tuned version performs 0.951 mAP at 0.5
IoU threshold on the test set.

###  3.2 Transformation Between World Coordinates and Image Coordinates

AIS messages contain the spatial location of vessels according to World
Geodetic System 1984 (WGS84) [1]. Transforming the world coordinates
(latitude, longitude) into image coordinates $(x,y)$ enables the fusion of
bounding boxes with AIS messages. In this section, two methods will be
introduced for transformation between world coordinates and image coordinates.
First, azimuth and distance estimation by interpolation will be presented in
the Section 3.2.2. Then the homography approach will be explained in Section
3.2.3 for different camera characteristics.

####  3.2.1 Keypoint Selection

For each camera, we selected an image with optimal visibility where the
scene’s features were distinctly clear, in order to manually select keypoints.
More than ten keypoints, such as fixed buoys, antennas, dock walls, if
applicable, significant landmarks (e.g., Elbphilharmonie), were selected for
each image. Image coordinates of each key point were noted, and their
corresponding world coordinates were detected with the help of Google Maps.
The azimuth angle and the distance were derived from world coordinates for
each key point by using inverse geodetic function with WGS84 ellipsoid [1].
Figure 2 shows the keypoints on the map and the image for the Neumühlen
camera.

Figure 2: World (above) and image (below) coordinates for the keypoints of the
Neumühlen camera represented with red points.

####  3.2.2 Azimuth and Distance Estimation by Interpolation

Azimuth is the horizontal angle between the north and a point of interest in
degrees. Every image aligned with the horizon has a linear relationship
between the azimuth and the pixel coordinates on the $X$ axis. In the dataset,
the images are almost perfectly aligned with the horizon and demonstrate the
linear relationship between the $X$ axis and the azimuth degrees. Since the
camera and the ship locations are roughly known, linear interpolation between
the image’s left and right edge can map each pixel on the $X$ axis with an
azimuth angle. The distance can be estimated with interpolation approach
assuming that the keypoints cover the related parts of the image.

####  3.2.3 Homography

The lack of intrinsic and some extrinsic parameters prevented the estimation
of the camera matrix, which would be helpful for transformation from world
coordinates to image coordinates. Nevertheless, the relation between two
images of the same planar surface can be found with homography. In this paper,
instead of two images of the same planar surface, an image and a map were used
of the same area to estimate the homography matrix. Therefore, the
transformation between the map coordinates, $lon$, $lat$, and the image
coordinates, $x$, $y$ in pixels can be performed as shown in Equation 1, where
$w$ is the scale factor and $h_{i,j}$ denotes the elements of the homography
matrix. The value 1 is appended to the image coordinates to obtain their
homogeneous matrix to avoid dimensional mismatch.

$\begin{bmatrix}w\,lat\\\ w\,lon\\\
w\end{bmatrix}=\begin{bmatrix}h_{11}&h_{12}&h_{13}\\\ h_{21}&h_{22}&h_{23}\\\
h_{31}&h_{32}&h_{33}\end{bmatrix}\begin{bmatrix}x\\\ y\\\ 1\end{bmatrix}$ (1)

##### Transformation in Fixed Cameras

First, the keypoints were detected manually to estimate the homography matrix
for the fixed cameras. Then the corresponding world coordinates were detected
from Google Maps. After having two sets of points in different coordinate
systems, the homography matrix was estimated. Therefore, the world coordinates
provided by AIS messages can be transformed into image coordinates which can
be later matched with related bounding box centres.

##### Transformation in Panning Cameras

Due to irregular image collection intervals, the images collected from panning
cameras require localisation first. Since the images from various angles are
present, a panorama for each panning camera was created by stitch images.
After having a panorama image, the method followed for fixed cameras can be
applied for homography estimation to transform world coordinates into panorama
image coordinates. Unlike fixed cameras, there is one more step to take:
transforming query image coordinates to panorama image coordinates. Each image
was used as a query image and localised in the related panorama image with
template matching, using cross-correlation. The transformation from query to
panorama image coordinates was applied using the highest cross-correlation
value as the offset. Thus the ships were localised in the query image based on
the AIS message in two steps (1) world coordinates to panorama image
coordinates, (2) query image coordinates to panorama image coordinates.

Cross-correlation is sensitive to small changes as it is a pixel-level
comparison [10]. To improve the localisation of query images in panoramas, a
future extraction algorithm, Oriented FAST and rotated BRIEF (ORB) [23], was
tried. However, it did not outperform the template matching approach. The main
reason for this is that the images contain cranes, wind turbines and other
structurally identical and moving objects, which limits the performance of the
algorithm during the keypoint matching process.

###  3.3 Bounding Box-AIS Message Association

AIS data is filtered by a region of interest defined manually for each camera
and 30 seconds period before and after the image’s timestamp. The region of
interest is the whole area that panning cameras cover. Figure 3 illustrates
the area for each webcam. Then filtered messages are compared with the
bounding boxes in the related image. In this dataset, the images are less
frequent than the AIS messages, and there is no one-to-one relation between
the images and the messages. Therefore, the location of vessels is predicted
by interpolation according to the image’s timestamp, assuming that the vessel
speed is constant since the time frame is not big enough to significantly
change the speed. If there are not enough datapoints to interpolate for a
vessel at a time, then the location transmitted in AIS message is used.
Applying interpolation also helps reducing the negative effects of the
unstable video stream problem. After getting the predictions, AIS messages are
assigned to the closest bounding box in the image by using nearest neighbour
search based on k-dimensional tree [6].

Figure 3: Regions of interest for each camera.

###  3.4 Dataset

The visible light images in this dataset were collected from five public
webcam streams between 11.01.2022 and 20.03.2022 during the daylight hours.
These webcams are located on the bank of Elbe river and its tributary
Norderelbe, near the centre of Hamburg. Three of these webcams are fixed,
while the other three are panning, one of which demonstrates both
characteristics. Fixed cameras are those that do not change direction and
always point in one direction. Although the wind changes their direction
slightly from time to time, these changes are so small that they can be
ignored. Panning cameras, on the other hand, change their direction with yaw
rotation and therefore cover a wider area than the fixed cameras.

One of the cameras shows two characteristics: It is primarily a panning
camera, however, after a short transition, it focuses on a building for a
minute and behaves like a fixed camera. Therefore, this camera can be treated
as two cameras with different characteristics. Although the camera angle is
slightly different because of the panning movement during the transition from
panning to fixed characteristic, it is still closer to a fixed camera than a
panning camera. The images produced by this camera were divided into three
classes with appearance-based classification (1) Panning images, which were
treated as query images and compared with a panorama, (2) Fixed images, which
contain the images of Dockland Office, (3) Transition images, which were
captured during the transition.

The histogram of each image is compared with other histograms that belong to
one of these classes. The other histograms are from a subset of the images
that were captured on 11.01.2022. In this subset, 25 panning, 24 fixed, and 17
transition image histograms are present. A class was assigned based on the
smallest Euclidean Distance between the histograms. Before this method,
Structural Similarity Index (SSIM) [28] and edge difference zero-count were
used, but these methods did not provide satisfactory results. Although the
edge detection method seemed logical initially, as the transition images were
out of focus, it could not outperform the histogram comparison. Table 1 shows
the characteristics of the webcams.

Name | Type | Resolution (px) | Location (Lat, Lon) | Direction
---|---|---|---|---
Altona | Fixed | 1280 x 720 | 53.54387, 9.94275 | South West
Blockbräu | Fixed | 1280 x 720 | 53.54553, 9.96957 | South East
Neumühlen | Fixed | 1920 x 1080 | 53.54388, 9.91692 | South East
Altona | Panning | 1280 x 720 | 53.54387, 9.94275 | South East-South
Elbe | Panning | 1280 x 720 | 53.54722, 9.96338 | South East-South West
Hafencity | Panning | 1920 x 1080 | 53.53903, 9.99345 | South-South West
Table 1: Characteristics of the webcams on the banks of the Elbe river.

There are a few drawbacks to collect images from a video stream. First,
network problems affect image collection significantly, as a delay can occur
both on the server and the client sides. Delays affect the timestamps during
image collection, which result in predictions that are significantly in front
of the vessel. Moreover, some images were repeated without a clear repetition
pattern due to an unreliable data stream. Because of the random nature of
image repetition, a preprocessing step was used for filtering them out. The
images were compared with 20 previous frames by using histogram comparison,
and the duplicate images were removed. As the traffic is not always dense, the
images did not always have a ship in them. By using both the fine-tuned YOLO
model and checking the presence of AIS messages for each camera’s RoI, the
images without any ships were detected and removed. These preprocessing steps
resulted in the removal of images with rain drops which covers the whole lens,
dark lighting conditions and heavy fog.

Panning cameras got affected the most by the infrequent data collection. Since
the images were collected in various intervals, the camera angle was different
each time an image was captured, making it harder to handle images from
panning cameras individually. Panoramas were created for each camera, and each
individual image was compared with the corresponding panorama image to
overcome this problem (see Section 3.2.3).

Due to the difficulty of annotating the high amount of collected images, only
a subset from collected images were annotated in YOLO format with
"makesense.ai". In total, 1515 images were annotated of which 1062 images were
used during training, 299 images were during validation, and 154 images during
test while fine-tuning the YOLOv5 model. After bounding box annotation, the
unique identifier numbers were assigned for bounding boxes manually with the
help of AIS data provided. During the annotation process, "marinetraffic.com"
was also used as it provides images and MMSI numbers of vessels. It is worth
mentioning that the colour changes of the ferries due to advertisements made
the annotation process more difficult. 1658 bounding box-AIS message pairs
were created in 381 images.

Besides image_id, bbox, category_id and unique_id the annotations contain
details about the vessel taken from the associated AIS message. A sample
annotation is shown in the Listing LABEL:annotation_snippet.

Listing 1: A sample annotation snippet for the vessel information that was
extracted from the associated AIS message.

⬇

”annotations”: [

{

”image_id”: 0,

”bbox”: [

363.0,

602.0,

199.0,

56.0

],

”vessel_info”: {

”type”: 70,

”latitude”: ”53.542968”,

”longitude”: ”9.935401”,

”heading”: 268.3,

”course”: 271.38,

”length”: 29,

”width”: 7,

”speed”: 8.73,

}

},

…

]

##  4 Results and Discussion

###  4.1 Coordinate Transformation

Preliminary results show that homography outperforms azimuth and distance
estimation by interpolation, as interpolation needs full coverage of the
related parts of the image. However, it is hard to find keypoints that cover
the whole area. Extrapolation can be a solution for this in exchange of
precision. Because of this reason, homography is used as the technique for
transforming coordinates.

Camera Name |  Camera Type |  Maximum
Error (px) |  Minimum Error (px) |  Mean
Error (px) |  Standard Deviation |  Keypoint Count
---|---|---|---|---|---|---
Blockbräu |  Fixed |  161.51 |  4.24 |  39.97 |  31.84 |  27
Altona |  Fixed |  40.26 |  4.12 |  16.58 |  9.46 |  15
Neumühlen |  Fixed |  31.38 |  0.00 |  11.39 |  10.29 |  16
Elbe |  Panning |  451.46 |  43.46 |  146.85 |  123.38 |  11
Altona |  Panning |  267.32 |  7.28 |  90.04 |  88.29 |  16
Hafencity |  Panning |  245.00 |  17.80 |  124.04 |  63.02 |  17
Table 2: Coordinate transformation error from world coordinates to (panorama)
image coordinates.

Table 2 shows the coordinate transformation error introduced by homography. As
the table shows, the fixed cameras produce less error compared to panning
cameras because of the distortion introduced during the panorama creation in
the coordinate transformation step. Also, considering that the panorama image
is significantly bigger than an individual image, they are more likely to have
bigger errors. It is clear that an increase in the number of keypoints affects
the performance positively. It is to be noted that continuous values for the
pixel errors stem from the homography matrix and are purely used for
evaluation purposes.

###  4.2 Image-AIS Matching

The rate of successfully associated bounding box-AIS message pairs was used to
evaluate the performance of image-AIS fusion. The accuracy values were
calculated as shown in equation 2. The Maritime Mobile Service Identity (MMSI)
number assigned to the bounding box was manually verified using the vessel
images at "marinetraffic.com". If the visual inspection can confirm that the
detected vessel is associated with the correct MMSI, the association is
counted as correct. For the privacy reasons, the public dataset does not
contain MMSI but a unique identifier number.

As the panorama images have bigger regions of interest, AIS messages that do
not land on the query image are not filtered out. Since AIS messages are
sequentially assigned with the closest bounding box, sometimes AIS messages
from outside the query image can be associated with the closest bounding box,
which is usually an incorrect match. Vessels that transmit at large intervals
or not at all, for example moored vessels, create a problem as their bounding
box can be detected but cannot be associated with an AIS message correctly.

$Accuracy=\frac{Correct\,Associations}{Total\,Number\,of\,Pairs}*100$ (2)

Camera Name |  Image Count |  Accuracy (%) |  Correct
Pred. |  Total
Pred.
---|---|---|---|---
Altona |  60 |  34.52 |  29 |  84
Altona-F |  61 |  43.00 |  43 |  100
Blockbraeu |  70 |  77.25 |  180 |  233
Elbe |  60 |  24.48 |  24 |  143
Hafencity |  60 |  80.33 |  49 |  61
Neumühlen |  70 |  94.13 |  625 |  664
Total |  381 |  74.79 |  961 |  1285
Table 3: The accuracy of successful associations for cameras. Altona-F is the
fixed behaviour of the Altona camera.

Table 3 shows the accuracy of successful associations for each camera. The
overall performance of the system is 74.79 % accuracy in matching bounding
boxes with AIS messages. For fixed and panning cameras, the system achieves
85.06 % and 39.24 %, respectively.

Fixed cameras outperform panning cameras because of the extra step of
localisation of query images in panorama images, which fixed cameras do not
have. Any bounding box-AIS message pairs that are present on an unsuccessfully
localised query image are counted as unsuccessful matches whether they are
successfully matched or not, as the localisation step comes prior to the
matching step.

Tidal changes slightly affect the accuracy negatively since the keypoints were
selected on sea level and the plane predicted by homography does not update
itself accordingly. Weather conditions, such as fog and rain, can blur or
entirely obstruct vessels in images. Although it is a rare scenario, sometimes
birds can block the view of cameras.

The camera Elbe, which covers the widest area compared to the other cameras,
is the worst performing one with an accuracy of 24.48 %. This poor performance
can be explained in two aspects. Considering the hardware aspect, Elbe is a
panning camera located furthest from the river without a cover for the rain.
It has the lowest number of keypoints, which results in the poor
transformation from AIS message coordinates to image coordinates. Moreover, it
introduces the highest amount of distortion when the panorama is created
because of the wide area it covers, which results in poor performance for
localising a query image in the panorama image. The accuracy of correct
localisation of a query image is 58.33 %. Considering these drawbacks, the
Elbe camera requires more manual work for the selection of new keypoints, and
a better technique of localisation of query images in panorama images.

Neumühlen is a fixed camera that faces towards a pier where service boats and
tug boats moor. More importantly, the direction of the camera is not
perpendicular to the river. This makes the association easier as the error
introduced by the unstable video stream becomes more tolerable because of the
smaller displacement in the image, as mentioned in Section 3.4. When the
vessels move slower or moor, it is easier to correctly associate bounding
boxes with AIS messages as long as they keep transmitting AIS messages which
tug boats mostly do since they only moor for a short period of time. The slow
but consistent traffic can explain the high performance of the Neumühlen
camera.

Comparing the results with Lu et al.’s framework, which obtains 69.35 %
without the short-time trajectory prediction method (DR), and 75.70 % with DR,
Qu et al.’s framework outperform Lu’s framework with 81.42 % overall accuracy.
Our approach, on the other hand, outperforms both works for static cameras
while the overall accuracy is slightly lower than Qu et al.’s framework. These
works do not use a dataset consisting of images taken from web cameras with
different and unknown parameters. Additionally, the datasets do not contain
weather conditions such as fog and rain. Lu’s dataset consists of images of
the English Channel, while Qu’s dataset consists of images of the Yangtze
River, both of which are significantly broader than the Elbe river. Therefore,
the traffic of the Elbe River is expected to be denser which requires more
precision in the predictions and handling for the occluded vessels.

##  5 Conclusion

In this paper, a technique was developed for creating maritime computer vision
datasets by fusing ship bounding boxes with related AIS messages. The
technique extends Carillo’s technique [8] by adapting the homography-based
coordinate transformation for non-static cameras, thus increases the number of
application use cases. The technique consists of a fine-tuned YOLOv5 vessel
detection model and a homography-based coordinate transformation module that
can project world coordinates extracted from AIS messages onto images to match
detected vessels’ bounding boxes with AIS messages.

The results show that the proposed system achieves an accuracy of 85.06 % with
fixed cameras while achieving an overall accuracy of 74.79 %. These results
were achieved with images taken in various weather conditions during the day.
Compared to the similar frameworks, it performs better for fixed cameras while
can work with the panning cameras.

Using the proposed technique, a dataset consisting of images and associated
AIS messages can be created covering the Elbe River in Hamburg, a major hub
and the third busiest port in Europe. Therefore, images offer a rich variety
of ship types and multi-ship-encounter scenarios. Consequently, it stands as a
valuable cornerstone for advancing maritime research through machine learning
and artificial intelligence, serving as an indispensable resource for ongoing
projects such as LEAS [4], AUTOSHIP [7], and AEGIS [2]. Some possible use
cases include inter alia research on elevating the precision of multi-sensor
fusion techniques, thereby significantly enhancing navigational safety in the
maritime domain and the creation of digital twins, opening doors to a
multitude of possibilities for maritime system modeling and simulation,
ultimately fostering a comprehensive understanding of the maritime ecosystem.

##  6 Future Work

The proposed dataset is particularly useful for designing auto-labelling
pipelines to enhance existing object detection models or to solve pose
estimation tasks. Current investigations use the annotated data and the
homography matrix to generate labelled bounding box annotations in 3D world
space, which are fed into keypoint detection networks to learn pose estimation
from image space without the need to generate a homography matrix.

In terms of improving the proposed technique, the localisation of query images
in panoramic images using template matching can be considered as the main
limitation of this work. Improving this aspect of the fusion process of
panning cameras would increase the accuracy significantly as the results show
that the main problem is not the coordinate transformation but the template
matching. This problem can be overcome by implementing feature-based
localisation techniques for most cases. However, ships that occlude the
landscape, or weather conditions such as fog and rain require a completely
different approach.

Currently, keypoint selection for homography estimation is a process that has
to be done manually. Considering that the dataset only contains images of the
Elbe river and the camera locations are roughly known, it is possible to apply
shape alignment methods to align the landmarks with the map with a similar
approach of Shi et al. [24]. Although sports fields have proper shapes since
they are human-made, in the future, trying to register the sea charts with
images of Elbe can be fruitful. Moreover, ground-to-satellite image matching
methods [25] can be an area to explore for automatic keypoint selection.

## References

  * [1] World geodetic system 1984.  https://www.unoosa.org/pdf/icg/2012/template/WGS_84.pdf, 1984.
  * [2] Advanced, efficient and green intermodal systems.  https://aegis.autonomous-ship.org/, 2020.
  * [3] Artificial intelligence / machine learning sensor fusion for autonomous vessel navigation (navisp-el1-020 maritime ai-nav), May 2021.
  * [4] Leas – shore-side decision support for traffic situations with highly automated or autonomous vessels using ai.  https://research.fleetmon.com/projects/leas-decision-recommendations-for-vts-with-autonomous-vessels-using-ai/, 2022\.
  * [5] M Baldauf, K Benedict, S Fischer, F Motz, and J-U Schröder-Hinrichs.  Collision avoidance systems in air and maritime traffic.  Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 225(3):333–343, 2011.
  * [6] Jon Louis Bentley.  Multidimensional binary search trees used for associative searching.  Communications of the ACM, 18(9):509–517, 1975.
  * [7] V Bolbot, G Theotokatos, E Boulougouris, LAL Wennersberg, H Nordahl, Ø J Rødseth, J Faivre, and M Molica Colella.  Paving the way toward autonomous shipping development for european waters–the autoship project.  2020\.
  * [8] Borja Carrillo-Perez, Sarah Barnes, and Maurice Stephan.  Ship segmentation and georeferencing from static oblique view images.  Sensors, 22(7):2713, 2022.
  * [9] Lena Chang, Yi-Ting Chen, Jung-Hua Wang, and Yang-Lang Chang.  Modified yolov3 for ship detection with visible and infrared images.  Electronics, 11(5):739, 2022.
  * [10] Misganu Debella-Gilo and Andreas Kääb.  Sub-pixel precision image matching for measuring surface displacements on mass movements using normalized cross-correlation.  Remote Sensing of Environment, 115(1):130–142, 2011.
  * [11] Lei Du, Floris Goerlandt, and Pentti Kujala.  Review and analysis of methods for assessing maritime waterway risk based on non-accident critical events detected from ais data.  Reliability Engineering & System Safety, 200:106933, 2020.
  * [12] Ties Emmens, Chintan Amrit, Asad Abdi, and Mayukh Ghosh.  The promises and perils of automatic identification system data.  Expert Systems with Applications, 178:114975, 2021.
  * [13] Glenn Jocher et. al.  ultralytics/yolov5: v6.0 - YOLOv5n ’Nano’ models, Roboflow integration, TensorFlow export, OpenCV DNN support, 2021.
  * [14] Vishal Gupta, Monish Gupta, and Parveen Singla.  Ship detection from highly cluttered images using convolutional neural network.  Wireless Personal Communications, 121(1):287–305, 2021.
  * [15] London IMO.  Solas. consolidated edition, 2001.  2001\.
  * [16] Zhelin Li, Lining Zhao, Xu Han, and Mingyang Pan.  Lightweight ship detection methods based on yolov3 and densenet.  Mathematical Problems in Engineering, 2020, 2020.
  * [17] Yongqiang Lu, Hongjie Ma, Edward Smart, Branislav Vuksanovic, John Chiverton, Shanker Radhakrishna Prabhu, Malcolm Glaister, Eric Dunston, and Chris Hancock.  Fusion of camera-based vessel detection and ais for maritime surveillance.  In 2021 26th International Conference on Automation and Computing (ICAC), pages 1–6, 2021.
  * [18] Fabio Mazzarella, Michele Vespe, and Carlos Santamaria.  Sar ship detection and self-reporting data fusion based on traffic knowledge.  IEEE Geoscience and Remote Sensing Letters, 12(8):1685–1689, 2015\.
  * [19] United States Department of Defense.  Global positioning system standard positioning service performance standard.  https://www.gps.gov/technical/ps/2008-SPS-performance-standard.pdf, 2008\.
  * [20] Captain Edward Page.  Maximizing maritime safety and environmental protection with ais: (automatic identification system).  In OCEANS 2017 - Anchorage, pages 1–4, 2017.
  * [21] Giuliana Pallotta, Michele Vespe, and Karna Bryan.  Vessel pattern knowledge discovery from ais data: A framework for anomaly detection and route prediction.  Entropy, 15(6):2218–2245, 2013.
  * [22] Jingxiang Qu, Yu Guo, Yuxu Lu, Fenghua Zhu, Yingchun Huan, and Ryan Wen Liu.  Intelligent maritime surveillance framework driven by fusion of camera-based vessel detection and ais data.  In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 2280–2285, 2022.
  * [23] Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski.  Orb: An efficient alternative to sift or surf.  In 2011 International conference on computer vision, pages 2564–2571. Ieee, 2011.
  * [24] Feng Shi, Paul Marchwica, Juan Camilo Gamboa Higuera, Mike Jamieson, Mehrsan Javan, and Parthipan Siva.  Self-supervised shape alignment for sports field registration.  In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3768–3777, 2022.
  * [25] Yujiao Shi, Xin Yu, Liu Liu, Dylan Campbell, Piotr Koniusz, and Hongdong Li.  Accurate 3-dof camera geo-localization via ground-to-satellite image matching.  arXiv preprint arXiv:2203.14148, 2022.
  * [26] Ming-Cheng Tsou.  Discovering knowledge from ais database for application in vts.  The Journal of Navigation, 63(3):449–469, 2010.
  * [27] Tamara A. Volkova, Yulia E. Balykina, and Alexander Bespalov.  Predicting ship trajectory based on neural networks using ais data.  Journal of Marine Science and Engineering, 9(3), 2021.
  * [28] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE transactions on image processing, 13(4):600–612, 2004.
  * [29] Chris M Ward, Josh Harguess, and Cameron Hilton.  Ship classification from overhead imagery using synthetic data and domain adaptation.  In OCEANS 2018 MTS/IEEE Charleston, pages 1–5. IEEE, 2018.
  * [30] Tong Xiaopeng, Chen Xu, Sang Lingzhi, Mao Zhe, and Wu Qing.  Vessel trajectory prediction in curving channel of inland river.  In 2015 International Conference on Transportation Information and Safety (ICTIS), pages 706–714, 2015.
  * [31] Puchun Xie, Ran Tao, Xin Luo, and Youqun Shi.  Yolov4-mobilenetv2-dw-lcarm: A real-time ship detection network.  In International Conference on Knowledge Management in Organizations, pages 281–293. Springer, 2022.


* Richter (2020) B. Richter.  _From Categories to Homotopy Theory_.  Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2020.  ISBN 9781108479622.  URL https://books.google.com/books?id=pnzUDwAAQBAJ.
  * Studený et al. (2010) Milan Studený, Jiří Vomlel, and Raymond Hemmecke.  A geometric view on learning bayesian network structures.  _International Journal of Approximate Reasoning_ , 51:573–586, 06 2010.  doi:10.1016/j.ijar.2010.01.014.
  * Fong and Spivak (2018) Brendan Fong and David I Spivak.  _Seven Sketches in Compositionality: An Invitation to Applied Category Theory_.  Cambridge University Press, 2018.  URL http://arxiv.org/abs/1803.05316.  cite arxiv:1803.05316Comment: 341+xii pages.
  * Coecke et al. (2016) Bob Coecke, Tobias Fritz, and Robert W. Spekkens.  A mathematical theory of resources.  _Information and Computation_ , 250:59–86, oct 2016.  doi:10.1016/j.ic.2016.02.008.  URL https://doi.org/10.1016%2Fj.ic.2016.02.008.
  * Pearl (1989) Judea Pearl.  _Probabilistic reasoning in intelligent systems - networks of plausible inference_.  Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann, 1989.
  * Schölkopf and Smola (2002) B. Schölkopf and A. J. Smola.  _Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond_.  MIT Press, 2002.
  * Imbens and Rubin (2015) Guido W. Imbens and Donald B. Rubin.  _Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction_.  Cambridge University Press, USA, 2015.  ISBN 0521885884.
  * Janzing et al. (2013) Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, and Bernhard Schölkopf.  Quantifying causal influences.  _The Annals of Statistics_ , 41(5):2324 – 2358, 2013.  doi:10.1214/13-AOS1145.  URL https://doi.org/10.1214/13-AOS1145.
  * Muandet et al. (2021) Krikamol Muandet, Motonobu Kanagawa, Sorawit Saengkyongam, and Sanparith Marukatat.  Counterfactual mean embeddings.  _Journal of Machine Learning Research_ , 22(162):1–71, 2021.  URL http://jmlr.org/papers/v22/20-185.html.
  * McInnes et al. (2018) Leland McInnes, John Healy, and James Melville.  Umap: Uniform manifold approximation and projection for dimension reduction, 2018.  URL http://arxiv.org/abs/1802.03426.  cite arxiv:1802.03426Comment: Reference implementation available at http://github.com/lmcinnes/umap.
  * Heymann et al. (2020) Bejamin Heymann, Michel de Lara, and Jean-Philippe Chancelier.  Causal information with information fields.  In _Neural Inforamtion Processing Systems Workshop on Causal Discovery and Causality-inspired Machine Learning_ , 2020.
  * Witsenhausen (1975) H. S. Witsenhausen.  The intrinsic model for discrete stochastic control: Some open problems.  In A. Bensoussan and J. L. Lions, editors, _Control Theory, Numerical Methods and Computer Systems Modelling_ , pages 322–335, Berlin, Heidelberg, 1975. Springer Berlin Heidelberg.  ISBN 978-3-642-46317-4.
  * Heymann et al. (2021) Benjamin Heymann, Michel de Lara, and Jean-Philippe Chancelier.  Causal inference theory with information dependency models, 2021.  URL https://arxiv.org/abs/2108.03099.
  * Dawid (2010) A. Philip Dawid.  Beware of the dag!  In Isabelle Guyon, Dominik Janzing, and Bernhard Schölkopf, editors, _Causality: Objectives and Assessment (NIPS 2008 Workshop), Whistler, Canada, December 12, 2008_ , volume 6 of _JMLR Proceedings_ , pages 59–86. JMLR.org, 2010.  URL http://proceedings.mlr.press/v6/dawid10a.html.
  * Evans (2018) Robin J. Evans.  Margins of discrete Bayesian networks.  _The Annals of Statistics_ , 46(6A):2623 – 2656, 2018.  doi:10.1214/17-AOS1631.  URL https://doi.org/10.1214/17-AOS1631.
  * Forré and Mooij (2017) Patrick Forré and Joris M. Mooij.  Markov properties for graphical models with cycles and latent variables, 2017.