# Integrating Counter-UAS Systems into the UTM System for Reliable Decision
Making

Abdulhadi Shoufan and Ruba Alkadi  Center of Cyber-Physical Systems, Khalifa
University, Abu Dhabi, UAE (emails: Abdulhadi.shoufan, Ruba.alkadi @ku.ac.ae)

###### Abstract

Despite significant progress, the deployment of UAV technology in commercial
and civil applications is still lagging. This is essentially due to the risks
associated with drone flights and the lack of coordinated technologies that
would mitigate these risks. While Unmanned Aircraft System Traffic Management
systems (UTM) are being developed worldwide to enable safe operation, the
counter-drone technology operates on an all-enemy basis and regards any
sighted drone as a threat. This situation is essentially caused by the lack of
information exchange between stakeholders. Without the exchange of relevant
information, a counter-drone system can misclassify drones and initiate
erroneous interdiction procedures. This paper proposes a system that
integrates counter-drone technology into the UTM system for information
exchange and coordination using a set of clarification protocols towards
accountable response to sighted drones. The system functionality and
performance were evaluated by simulation.

###### Index Terms:

UAV, Counter-UAS, UTM.

##  I Introduction

The drone market is growing rapidly with diverse applications in construction,
agriculture, insurance, oil and gas industry, film making, sky photography,
parcel delivery, journalism, security, law enforcement, and civil defense[20].
Despite this, the operation of UAVs in urban areas is still in the exploratory
stage [41]. That is, we neither get our online orders delivered by UAVs nor
can we ride a taxi drone, although the technical capabilities of today’s UAVs
allow for the deployment in such applications [13]. Indeed, flying a drone is
associated with security, privacy, and safety risks which have limited the
progress and penetration of UAV applications across many sectors in the urban
airspace [5, 22]. Safety is, without doubt, the most urgent requirement when
it comes to drone operation. This is confirmed by the frequent reports on
drone incidents and intrusions worldwide [24, 15, 14].

To support safe drone operations and counter violations many technologies are
available. These technologies can be divided broadly into three categories as
depicted in Fig. 1. The first category encompasses preventive onboard
solutions such as sense&avoid and geofencing [16], as well as response
technologies such as parachuting and self-recovery systems [21]. The second
category addresses illegal UAVs and includes counter-drone systems from
detection to interdiction. Detection and classification use different
modalities such as radar, images, acoustic, and radio-frequency signals [6,
37]. Malicious drones can be deactivated using diverse interdiction
technologies such as jamming, catching, or shooting [26, 32]. Finally, UTM-
driven technologies, including mission authorization systems and remote
identification, have been put forward recently to coordinate UAV activities in
the low-altitude airspace [18, 8].

Figure 1: Three technology categories for UAV and airspace safety

While these technologies are supposed to complement each other towards safe
airspace, the current implementations of category-2 and category-3
technologies lack the necessary level of integration and appear to follow
conflicting goals: While UTM-driven technologies promote the legalization of
drone operations, the counter-drone systems take a skeptical position and
consider any drone in the sky as a threat. This suggests that information
exchange between UTM systems and counter-UAS systems is mandatory to avoid
conflicts. These observations were already reported in the recent survey of
counter-drone technology [30]. Hence, the first contribution of this paper is
to propose a novel architecture for integrating CUAS systems in the UTM system
(Section III). This architecture will allow the CUAS system to obtain
information about the registration status of the sighted drone and whether it
is authorized to perform the observed flight. With this information, the CUAS
system can classify the drone as cooperative and tolerate its existence in the
operation zone of the CUAS system. To access relevant information in the UTM
databases, a CUAS system requires an initial knowledge about the drone. We
propose to obtain this initial knowledge from the drone itself through the
broadcast remote identification. Remote identification (RID) is becoming a
core technology of UTM systems in many countries.

No matter how reliable these technologies are, technical and non-technical
errors can occur:

  1. 1.

The RID module on board the drone can fail.

  2. 2.

The RID receiver of the CUAS system can fail.

  3. 3.

A UTM system operator can fail to update the databases or to do so on time.

  4. 4.

An inquiry to the UTM databases can fail for technical reasons.

  5. 5.

The drone operator can exceed the permitted flight time by mistake or for an
urgent reason.

  6. 6.

The drone operator can deviate from the approved mission trajectory by mistake
or for mandatory reasons.

In none of these cases should a CUAS system immediately classify a drone as
uncooperative and start an interdiction operation. On the other hand, the
outcomes of such cases are indeed ambiguous and can indicate uncooperative or
malicious drone operations that should be countered. These thoughts suggest
that a counter-drone system needs more information to clarify such cases and
classify them correctly towards a responsible decision. Therefore, the second
contribution of this paper is to present a post-detection model for the drone
and a set of protocols that help in clarifying ambiguous cases, see Section
IV.

The developed models and protocols are evaluated by simulation. First, we used
an appropriate Simulink toolkit to test the correctness of the model and
protocols. Then, we implemented the same using Node.js to assess the
performance of the system in a networked mode. The simulation setup and
results are present in Section VI.

##  II Related Work

###  II-A UTM Systems

The concept of unmanned traffic management (UTM) refers to an ecosystem for
controlling the operation of unmanned aerial systems [18, 41]. Data exchange
is at the core of UTM where authorized unmanned service suppliers (USS)
provide cloud-based services to the different stakeholder. Examples of these
services include UAV control [42], efficient and fair unmanned traffic control
[12], flight planning and scheduling [23], geofencing [35], path optimization
and collision avoidance [11], weather and contingency management [31, 27] ,
orchestrating of UAV services [10], and supporting the internet-of-drones
(IoD) [4].

UTM systems are still in the development stage. Some authors have highlighted
several challenges and issues in the design of these systems. Wolter et al.
pointed out multiple obstacles in the current experimental setups which relate
to standardization, information quality, and the transition from human-centric
design to automation [39]. Several authors addressed the security of UTM
systems and presented their vulnerabilities to various cyber and physical
attacks [4]. The lack of a legal framework for UTM system operations was
highlighted in [33]. The authors described the fundamentals of such a legal
framework which should provide the needed certainty for all stakeholders.

###  II-B Counter-Drone Systems

The counter-drone industry has boomed in recent years. A report published by
the ”Center for the Study of the Drone” at Bard College shows that there are
537 counter-unmanned aerial systems (CUAS) on the market [25]. Researchers
showed wide interest in this field, especially regarding the detection and
classification of small UAVs. Different technologies were proposed including
radar, radio-frequency detection, acoustic systems, and computer vision [17,
28, 6, 3]. Parallel to the advances in detection and classification
technologies, some researchers investigated technical solutions for drone
interdiction. Wyder et al. classified these technologies according to their
impact on the target drone into three main categories: signal interception,
propeller restriction, and aerial takedown [40]. Due to its undisruptive
nature, signal interception has received substantial attention for
interdiction in urban areas. Depending on the operation mode of the drone,
Roth et al. identified two methods of signal interception-based interdiction:
drone hacking and GPS spoofing [32]. Propeller restriction refers to capturing
uncooperative drones usually using a net. The net could be launched either
manually by a skilled operator on the ground or autonomously by another flying
drone [7]. Finally, a variety of aerial takedown technologies was presented.
These include hunting by eagles [29] and shooting by machine guns or laser
[32].

###  II-C Coordination between UTM and Counter-Drone Systems

Despite the strong relationship between UTM and counter-drone systems, the
interaction between these systems remained unaddressed in the literature.
Recently, Park et. al [30] surveyed more than 180 publications on the
literature of counter-drone systems. In their concluding remarks, they
highlighted the necessity of integrating a well-defined drone identification
scheme into counter drone systems. Sandor has highlighted that with the spread
of UTM systems, we need to define the problems, the scope, and the operational
environment [34]. The author defined and classified many functions related to
UTM and interestingly suggested surveillance as a UTM service with different
technologies for the detection of cooperative and non-cooperative vehicles.
However, the author did not take into consideration the critical function of a
counter-drone system which is interdiction. Apart from this, we are not aware
of any literature that addressed the coordination of the UTM and counter-drone
systems’ activities.

##  III System Architecture

Figure 2: System Architecture for Coordinated Drone Interdiction

The proposed system architecture relies on and extends the UAS traffic
management system (UTM) released by NASA [18]. The proposed architecture in
Fig. 2 aims to grant CUAS systems access to the Flight Information Management
System (FIMS). The latter should contain at least two databases: a database of
the identities of registered drones (ID-DB) and a database of authorized
missions (AUTH-DB). In the following, we first describe the system agents in
brief. Then, we describe the databases and how the CUAS system interacts
within the UTM system.

The Authority in our architecture refers to the agent that regulates and
oversees civil airspace to assure safe and secure operation by providing air
navigation services and facilitating air connectivity. All drones should be
registered at the authority. Additionally, operators can obtain mission
authorization directly from the authority or from authorized service
suppliers. The authority along with such service suppliers keeps relevant
information about drone registrations and mission authorizations in the ID-DB
and AUTH-DB databases.

The Unmanned Service Supplier (USS) is an entity that provides services to
subscribed UAS operators to help them meet the operational requirements
specified by the authority. Operation planning, strategic and tactical de-
confliction, remote identification, and airspace authorization are examples of
the services provided by a USS. An operator may subscribe to one or more USS’s
to use multiple services.

The UAS operator is a primary agent in the UTM system who is responsible for
operating the drone in different modes for various purposes. UAS operators may
opt to utilize a USS or to provide their own services to meet regulatory
obligations. So, it is the operator’s responsibility to meet the requirements
established for the type of operation and the associated airspace volume or
route.

The Counter Unmanned Aerial System (CUAS) is an active agent in the UTM
architecture according to our proposal. By far, the deployment of anti-drone
technology against civil drones has been served as individual efforts by
different agents interested in protecting their territories against malicious
drones or by law enforcement for protecting public safety. We advocate for
integrating the CUAS technology into the UTM system to enable the exchange of
information between relevant agents and support coordinated efforts for
cooperative vs. uncooperative drone identification.

Finally, the Court is added as an agent to the system to benefit from the
information infrastructure for evidence collection in case of legal conflicts.
Basically, following to any interdiction process, the authority can file a
lawsuit to the court to initiate forensics work. Similarly, a UAS operator
might file a lawsuit in case her or his vehicle was intercepted, fined, or
jammed unlawfully.

###  III-A Identity Database (ID-DB)

Aviation authorities worldwide are mandating the deployment of remote
identification to promote safe and accountable drone operations. The European
Union Aviation Safety Agency (EASA) has published related regulations in March
2019 (Commission delegated regulation (EU) 2019/945) that were amended in
April 2020 ((EU) 2020/1058). These regulations mandate that all unmanned
aircraft should be equipped with a remote identification system [2]. The FAA
in the USA has published a final rule for remote identification in January
2021 [1]. According to this, the remote identification message should include
information about the drone’s identity, location, altitude, and velocity in
addition to the control station location and elevation, a time mark, and
emergency status. A database with Remote ID information should provide three
levels of access:

  1. 1.

Level 1: Information available to the public (the UAS unique identifier).

  2. 2.

Level 2: Information available to designated public safety and airspace
management officials (personally identifiable information).

  3. 3.

Level 3: Information available to the FAA and certain Federal, State, and
local agencies (tracking data).

In our architecture, we propose including this database in the FIMS. Certified
CUAS systems should be given access to this database on level 2 or 3 depending
on the predetermined authority of these systems. We further assume that this
database is available and its content is protected from security attacks.
Specifically, Remote ID entries in the database are assumed to be authentic.
This means that a CUAS can use the ID-DB to verify the authenticity of a
received Remote ID.

###  III-B Authorization Database (AUTH-DB)

CUAS systems are usually used in protected areas such as controlled zones
around airports. But even in such areas, drones can have a reason to fly, for
example, to help firefighters nearby. So, the CUAS needs a possibility to
check whether a detected drone has permission to fly in this area and time.
The FAA has implemented a system referred to as Low Altitude Authorization and
Notification Capability (LAANC) [19]. This system automates the application
and approval process for airspace authorizations. Like Remote ID, the LAANC is
considered a part of the UTM ecosystem. UAS service suppliers support the
authorization process: A drone pilot submits a request through a LAANC USS.
The request is checked against multiple airspace data sources by the FAA. If
approved, the pilot receives the authorization in near real-time.

For our purpose, we propose that all mission authorizations should be logged
in AUTH-DB. An entry in this database should include information about the
drone ID and the mission date, time, and area. Operators of counter-drone
systems should have access to the AUTH-DB to verify the performance
authorization of a detected drone. The AUTH-DB must be protected against
security attacks such as entry manipulation.

###  III-C CUAS Interactions within the UTM System

The proposed architecture in Fig. 2 allows the CUAS to interact with different
agents in the UTM system and to perform different checks using the identity
and the authorization databases towards cooperative vs. uncooperative
identification. For this, we make the following assumptions:

  1. 1.

Any drone in flight should share its ID remotely.

  2. 2.

A CUAS has the necessary technology to receive the Remote ID and check its
authenticity and validity by accessing the ID-DB.

  3. 3.

The ID-DB should be kept up-to-date by the authority or any certified USS.
Nonetheless, legacy/expired IDs are not removed from the database.

  4. 4.

Any drone flying in the operation zone of a CUAS (i.e., the zone in which the
CUAS is permitted to interdict), is required to have a performance
authorization.

  5. 5.

A performance authorization includes at least information about the drone ID
as well as the date, time, and area of flight.

  6. 6.

The CUAS has access to the AUTH-DB to verify whether a detected drone is
authorized to fly in the respective area and time.

  7. 7.

The AUTH-DB should be kept up-to-date by the authority or any certified agent.

In the following section, the CUAS interaction with the UTM system will be
detailed by defining a post-detection model for the drone.

##  IV Post-Detection Model for Drones

Figure 3:  Post-detection model for drones from CUAS perspective. The drone is
modeled as a finite-state machine. A cooperative drone passes multiple checks
transiting from one green state to another. A drone that fails some check
transits to an orange state that can trigger some clarification protocol.

Based on the architecture presented in the previous section, we model a drone
from a CUAS perspective using the finite state machine illustrated in Fig. 3.
The transition from one state to another is triggered either by the result of
some checks that the CUAS performs independently or by the outcome of some
protocol execution between the CUAS and other agents. In the beginning, the
system is set in the Surveillance state. When the CUAS identifies an object as
a drone, it transits to the state Drone Detected. The machine states are
highlighted using three colors:

  1. 1.

The green color indicates the desired state which is reached through a
successful check of some conditions. For example, when the CUAS verifies that
the drone is transmitting a Remote ID, the next state would be ID Received.
Furthermore, the model has two special green states in which the drone is
tolerated despite some failures, as will be described in Section V.

  2. 2.

A drone that reaches a red state is an uncooperative one. It will be subject
to an approved interdiction, which can be started immediately or after a
timeout.

  3. 3.

The orange color describes an ambiguous state of the detected drone. The CUAS
essentially tries to clarify this state by executing some protocol involving
other system agents. As a result, the drone either returns to a green state or
moves to a red state.

We first explain the case of a cooperative drone, which is represented by the
vertical green path. When a drone is detected, the CUAS receives its Remote
ID. The successful reception of the ID is followed by verifying its
authenticity. If the ID is authentic, the CUAS checks if it is available in
the identity database (ID-DB). If this is the case, the ID’s expiry date is
checked. If the ID is valid, the CUAS accesses the authorization database
(AUTH-DB) to verify whether the drone is authorized to perform the observed
mission. If the check is successful, the CUAS verifies if the drone is
complying with the mission plan regarding the area and time. If this is the
case, the FSM returns to the state Drone Detected. The CUAS can repeat these
checks until the drone has completed its mission or disappeared from the
detection range.

We now explain in brief what happens when any of the previous checks fails.
Detailed descriptions of the different cases and protocols will follow in
Section V. First, when the CUAS does not receive an ID from the drone (No ID
Received), the CUAS uses knowledge about the flight time and drone location to
inquire about a potential operator in the authorization database (AUTH-DB). If
no such operator is found, the drone state changes to No ID & No Potential
Operator. Depending on the CUAS policy and the authority, an Immediate
Interdiction process can be started. In contrast, if the CUAS finds a
potential operator in the AUTH-DB, the machine transits to the state No ID But
Potential Operator. Herein, the CUAS reads the ID of the potential operator
and initiates Protocol 1 to verify whether this ID belongs to the actual
operator who is flying the detected drone. Executing this protocol can lead to
one of four results:

  1. 1.

Restoring the ID transmission by the operator.

  2. 2.

Tolerating the ID transmission failure.

  3. 3.

Immediate interdiction of the drone.

  4. 4.

Delayed interdiction of the drone.

The CUAS should verify the authenticity of a received remote identification.
When this verification is not successful, the CUAS classifies the drone as
uncooperative. The next state is Fake ID leading to the state Immediate
Interdiction. In contrast, if the ID is authentic, the CUAS looks for a
related entry in the ID-DB. If no such ID is found, the next state will be ID
not in ID-DB. The CUAS tries to clarify this situation by checking if the
unknown ID has a corresponding entry in the AUTH-DB. If this is not the case,
the CUAS moves to the state Unknown ID and executes Protocol 2 to clarify the
reason for the missing ID and the missing performance authorization. Executing
this protocol can lead to one of four results:

  1. 1.

Restoring technical issues causing database misses.

  2. 2.

Tolerating the missing ID and authorization.

  3. 3.

Immediate interdiction of the drone.

  4. 4.

Delayed interdiction of the drone.

In contrast, if AUTH-DB has a related entry, the CUAS initiates Protocol 3 to
clarify the reason for the missing ID in the ID-DB. Executing this protocol
may lead to restoring the technical issue in the ID-DB or tolerating the
missing ID.

An authentic and known ID can be invalid for different reasons. For example,
the operator may miss renewing the registration (if required) or the
authority/USS may miss updating the ID-DB on time. When the CUAS finds out
that the received ID has expired (Expired ID), it checks if the AUTH-DB has
authorization for this ID in the current time and area. If no entry is found,
the drone moves to the state Expired ID & Unauthorized.  Protocol 4 is
initiated to clarify this state. Three outcomes are possible:

  1. 1.

Restoring the technical issues in both databases.

  2. 2.

Tolerating the expired ID and the missing authorization.

  3. 3.

Delayed interdiction.

In contrast, if the CUAS finds a related entry in the AUTH-DB, it initiates
Protocol 5 with two possible outcomes:

  1. 1.

Restoring the technical issue related to the ID-DB.

  2. 2.

Tolerating the expired ID.

If the received ID is adequate (authentic, known, and valid) but the AUTH-DB
doesn’t have a performance authorization for this drone, then Protocol 6 is
executed. Three outcomes are possible:

  1. 1.

Restoring the technical issue in the AUTH-DB.

  2. 2.

Tolerating the missing performance authorization.

  3. 3.

Delayed interdiction.

When the CUAS finds out that the drone is flying outside the area specified in
the authorization certificate in the AUTH-DB, it executes Protocol 7. This
protocol has one of the following outcomes:

  1. 1.

Bringing the drone back to the authorized area by the operator.

  2. 2.

Tolerating the area violation.

  3. 3.

Delayed interdiction.

Finally, when the CUAS finds out that the operator has exceeded the mission
time specified in the authorization certificate, it executes Protocol 8. This
protocol has one of the following outcomes:

  1. 1.

Terminating the mission by the operator.

  2. 2.

Tolerating time violation

  3. 3.

Delayed interdiction.

The states Tolerate ID Failure and Tolerate AUTH Failure can be reached from
different states as described above. To keep the diagram clear, we did not
draw the related transitions but used the numbers 1 to 6 to refer to them.
Apart from this, these states are equivalent to the states Valid ID and
Authorized Mission, respectively. Nonetheless, if called twice (or another
number of times depending on the authority policy), a protocol will generate
an interdiction approval to avoid deadlocks. This means that the system does
not tolerate a UAS operator who fails to clarify his identity repeatedly.

##  V Clarification Protocols

The clarification protocols aim at resolving unclear cases towards an accurate
classification of the detected drone. In this section, we describe these
protocols by first explaining when a protocol is triggered. Then, we summarize
the possible outcomes of the protocol using a table and describe its flow
using a sequence diagram as far as necessary. For simplicity, we use simple
natural language to label the exchanged messages. In the text, the labels are
in capital letters for clarity.

###  V-A Protocol 1: Clarify Missing ID Transmission

The objective of Protocol 1 is to clarify the situation when the CUAS does not
receive an ID from the drone. In this case, the CUAS accesses the
authorization database (AUTH-DB) to check if there is an operator who is
authorized to fly in the respective time and zone. If such an entry exists in
the database, the corresponding operator is regarded as the potential operator
of the detected drone. The CUAS initiates Protocol 1 to clarify whether this
potential operator the actual one is, i.e., the one operating the detected
drone. First, the CUAS sends a message named NO ID BUT POTENTIAL OPERATOR to
the authority. This message includes the ID of the potential operator. The
authority sends a CHECK/RESTORE ID TRANSMISSION message to the potential
operator or his USS. The authority receives one of four responses or no
response from the operator as outlined in Table I (CASE 1 to CASE 5). CASE 6
occurs when the CUAS does not confirm ID restoration, which the operator
claims to have done in CASE 5. Next, we describe how each of these cases is
treated.

TABLE I: Outcomes of Protocol 1 (Clarify Missing ID Transmission)

|  Operator Response |  Risk Assessment |  Authority Response to Operator |  Authority Response to CUAS
---|---|---|---|---
CASE 1 |  No response | No | NA |  Authorize immediate interdiction
CASE 2 |  I am not flying | No | NA |  Authorize immediate interdiction
CASE 3 |  I am already transmitting my ID | Yes |  Order mission completion or stop |  Order mission tolerance or authorize timed interdiction
CASE 4 |  I am not able to restore ID | Yes |  Order mission completion or stop |  Order mission tolerance or authorize timed interdiction
CASE 5 |  I restored ID transmission | No | NA |  Verify ID restoration
CASE 6 |  Unconfirmed ID restoration | Yes |  Order mission completion or stop |  Order mission tolerance or authorize timed interdiction

In the case of no response (CASE 1) or when the operator confirms that he/she
is not flying (CASE 2), the authority may have no possibility for further
checks. Hence, it sends an INTERDICT IMMEDIATELY message to the CUAS. The
latter performs the interdiction and reports this to the authority, which may
file a lawsuit case, see Fig. 4.

Figure 4: Protocol 1 (Clarify Missing ID Transmission, CASE 1 and CASE 2).
Clarify the situation when the CUAS does not receive an ID from the drone. In
these two cases, the drone operator either does not respond before the timeout
or respond with a ”I AM NOT FLYING” message.

When the operator confirms that he is already transmitting the ID (CASE 3) or
that he is unable to restore his ID transmission for any reason (CASE 4), then
the authority must conduct a risk assessment, see Fig. 5. The risk assessment
should take into consideration the criticality of the mission and the no-fly
zone. If the estimated risk is low, the authority sends a TOLERATE ID FAILURE
message to CUAS and a COMPLETE MISSION message to the operator. In contrast,
if the risk is high, the authority sends a STOP MISSION message to the
operator and an INTERDICT AFTER TIME-OUT message to the CUAS. Receiving this
message, the CUAS can interdict the drone after the time-out specified in this
message.

Figure 5: Protocol 1 (Clarify Missing ID Transmission, CASE 3 and CASE 4). The
drone operator claims that he/she is unable to restore ID, or that he/she is
already broadcasting the ID.

When the operator confirms that he has restored the ID transmission (CASE 5),
then the authority asks the CUAS for confirmation by sending a CONFIRM ID
RESTORATION! message, see Fig. 6. There are two ways to respond to this
message: implicit and explicit. In the explicit method, which is illustrated
in Fig. 6, the CUAS sends dedicated messages (ID RESTORATION CONFIRMED or ID
RESTORATION NOT CONFIRMED) to the authority to confirm or invalidate the
restoration of the ID transmission, respectively. Using the implicit method,
the CUAS returns to the state _”Drone Detected”_ as shown in the post-
detection model of Fig. 3. Now, if the ID transmission was correctly restored,
the CUAS would receive the ID and move to the state _”ID Received”_. This
would be regarded as an implicit confirmation. In contrast, if the ID
restoration was not accomplished, the CUAS would re-send the message NO ID BUT
POTENTIAL OPERATOR to the authority. The latter would consider the repeated
reception of this message as an invalidation of the ID restoration. An
invalidated ID restoration (CASE 6) prompts the authority to perform a fast
risk assessment and to act depending on the assessed risk level as illustrated
in Fig. 6.

Figure 6: Protocol 1 (Clarify Missing ID Transmission, CASE 5 and CASE 6). The
drone operator claims that the ID is restored.

###  V-B Protocol 2: Clarify Unknown ID Issue

The objective of this protocol is to clarify the situation when the CUAS
receives an authentic Remote ID but does not find a related entry, neither in
the ID-DB nor in the AUTH-DB. This can have technical reasons related to the
update or retrieval of these databases. In the worst case, however, receiving
an unknown ID can indicate a security break. First, the CAUS sends an UNKNOWN
ID message to the authority. This message includes the received Remote ID. The
authority performs necessary checks to find out whether such an ID exists or
whether there is a technical issue causing the misses in the databases.
Depending on the result of these checks, three outcomes are possible as
summarized in Table II.

TABLE II: Outcomes of Protocol 2 (Clarify Unknown ID Issue)

|  Description |  Authority Response to Operator |  Authority Response to CUAS
---|---|---|---
CASE 1 |  No technical issues, security break | NA |  Authorize immediate interdiction
CASE 2 |  Correct ID, issue with the ID-BD |  Order mission stop |  Authorize timed interdiction
CASE 3 |  Correct ID, issues with both databases | NA |  Order mission tolerance

In failure-free operation, all authentic IDs are expected to be in the ID
database. If the authority finds no technical issues related to the databases,
it assumes a security break (CASE 1) because the CUAS has already verified the
authenticity of the received ID. For example, the operator can have access to
the authority’s private key. As a response, the authority sends an IMMEDIATE
INTERDICTION AUTHORIZATION message to the CAUS. To keep the drone intact for
forensics analysis, the authority may request the CUAS to use a non-destroying
technology for the interdiction in this case.

Figure 7: Protocol 2 (Clarify Unknown ID Issue)

Alternatively, the authority may find out that the drone is registered but not
authorized to fly (CASE 2). This means that the CUAS conclusion regarding the
performance authorization was correct but there is a technical failure related
to the ID-DB because it did not show the ID. In this case, the authority sends
a STOP MISSION message to the operator and an INTERDICT AFTER TIME-OUT message
to the CUAS.

Finally, the authority may identify a technical issue related to the
management of both databases (CASE 3). In this case, the CUAS receives a
TOLERATE AUTH FAILURE message. This message implies that the ID is authentic,
so the CUAS does not need to recheck this.

###  V-C Protocol 3: Clarify Missing ID in ID-DB

This protocol is used to clarify the situation when the CUAS receives an
authentic Remote ID but does not find a related entry in the ID-DB and, at the
same time, the AUTH-DB has a performance authorization for this ID in the
respective time and area. At first glance, one may attribute this case to an
administrative fault. In particular, we may assume that the authority had
issued a performance authorization without checking the validity of the ID.
While this scenario is possible, it is excluded from our analysis because we
assumed in Section III-C that expired IDs are not removed from the database.

TABLE III: Outcomes of Protocol 3 (Clarify Missing ID in ID-DB)

| Description |  | Authority response to CUAS
---
CASE 1 |  | Unresolved technical
---
issue in ID-DB
Tolerate ID failure
CASE 2 |  | Resolved technical
---
issue in ID-DB
Confirm ID restoration

First, the CAUS sends an ID-BD MISS message to the authority. This message
includes the received Remote ID. The authority performs necessary checks. If
it identifies a technical issue related to ID-DB management, it sends a
TOLERATE ID Failure message to the CUAS and works on resolving the technical
issue. Alternatively, the authority can resolve this issue and request the
CUAS to confirm this restoration. The latter can respond explicitly or
implicitly. Note that the operator is not involved in this protocol. The
outcomes of this protocol are summarized in Table III.

###  V-D Protocol 4: Clarify Unauthorized & Expired ID

Figure 8: Protocol 4 (Clarify Unauthorized & Expired ID)

This protocol deals with a drone that is transmitting an authentic but expired
ID. Also, the drone is not authorized to fly. In this case, the CAUS first
sends the message EXPIRED ID & UNAUTHORIZED MISSION to the authority. This
message includes the received Remote ID. The authority performs necessary
checks to find out if there are any technical issues related to the databases.
If no issue is found, the authority sends a STOP MISSION message to the
operator and a TIMED INTERDICTION AUTHORIZATION. In contrast, if a technical
issue is identified, the authority sends a TOLERATE AUTH FAILURE message to
the CUAS and works on fixing the problem. The message sequence diagram of this
protocol is shown in Fig. 8.

###  V-E Protocol 5: Clarify Authorized But Expired ID

TABLE IV: Outcomes of Protocol 5 (Clarify Authorized But Expired ID)

| Description |  | Authority response
---
to CUAS
CASE 1 |  | Unresolved technical
---
issue in ID-DB
Tolerate expired ID
CASE 2 |  | Resolved technical
---
issue in ID-DB
Confirm valid ID entry

This protocol is executed when the CUAS receives an authentic Remote ID, which
is available in the ID-DB but no more valid. However, the AUTH-DB has a
performance authorization for this ID. This situation should be attributed to
one of two issues: a technical issue in the ID-DB or an administrative mistake
by issuing a performance authorization without checking the validity of the
ID.

First, the CAUS sends the message EXPIRED ID BUT AUTHORIZED MISSION to the
authority. The latter sends a TOLERATE EXPIRED ID message to the CUAS and
works on fixing the problem. Alternatively, the authority updates the ID-DB
immediately and requests the CUAS to confirm. The outcomes of this protocol
are summarized in Table IV.

###  V-F Protocol 6: Clarify Missing AUTH

TABLE V: Outcomes of Protocol 6 (Clarify Missing AUTH)

| Description |  Risk Assessment |  Authority Response to Operator |  Authority Response to CUAS
---|---|---|---|---
CASE 1 |  Technical issue with AUTH-DB | No | NA |  Technical issue resolved
CASE 2 |  No technical issue High risk | Yes |  Order mission stop |  Authorize timed interdiction
CASE 3 |  No technical issue Low risk | Yes |  Order mission completion |  Order mission tolerance

This protocol is executed when the received Remote ID is authentic, included
in the ID-DB, and valid; but there is no related performance authorization in
the AUTH-DB.

To clarify this situation, the CUAS sends an AUTH-DB MISS message to the
authority, see Fig. 9. The latter checks if the missing authorization is due
to a technical problem such as the lack of a timely update (CASE 1), see Table
V. If this is the case, the issue is resolved and the CUAS is informed by
sending the message AUTH-DB MISS RESOLVED. Otherwise and depending on a risk
assessment, the authority requests the operator to stop the mission and the
CUAS to interdict after a time-out (CASE 2). Alternatively, the CUAS can be
requested to tolerate this mission if the risk is low (CASE 3).

Figure 9: Protocol 6 (Clarify Missing AUTH)

###  V-G Protocol 7: Clarify Area Violation

TABLE VI: Outcomes of Protocol 7 (Clarify Area Violation)

|  Operator Response /Description |  Risk Assessment |  Authority Response to Operator |  Authority Response to CUAS
---|---|---|---|---
CASE 1 |  No response | No |  Order mission stop |  Authorize timed interdiction
CASE 2 |  I am already flying in authorized area | Yes |  Order mission completion or stop |  Order mission tolerance or authorize timed interdiction
CASE 3 |  I cannot return to authorized area | Yes |  Order mission completion or stop |  Order mission tolerance or authorize timed interdiction
CASE 4 |  I returned to authorized area | No | NA |  Verify return to authorized area
CASE 5 |  Unconfirmed return to authorized area | No |  Order mission stop |  Authorize timed interdiction

This protocol is executed when the received Remote ID is adequate and the
AUTH-DB has a related performance authorization. However, the operator has
flown the drone beyond the authorized zone as specified in the performance
authorization. The objective of this protocol is to resolve the issues
associated with area violations. In this case, the CUAS sends an AREA
VIOLATION message to the authority. The latter requests the operator to RETURN
TO AUTHORIZED AREA. Table VI summarizes five possible cases for clarifying
this situation.

If the operator does not respond to this message (CASE 1), the authority
requests the operator to stop the mission and authorizes the CUAS to interdict
the drone after a time-out, see Fig. 10.

Figure 10: Protocol 7 (Clarify Area Violation, CASE 1). Resolve the issues
associated with area violations. This scenario happens when the operator does
not respond to the authority message.

When the operator contradicts the request by claiming that he did not exceed
the permitted area (CASE 2) or when he declares that he cannot return to the
authorized zone (CASE3), the authority must perform a fast risk assessment to
take an appropriate decision as detailed in Fig. 11.

Figure 11: Protocol 7 (Clarify Area Violation, CASE 2 and CASE 3).The operator
responds to the authority message.

In CASE 4, the operator confirms that he returned to the authorized area. The
authority asks the CUAS for confirmation. This confirmation can be explicit or
implicit similar to what was explained in  Protocol 1. If this proves untrue,
or when the operator keeps violating the area restriction (CASE 5), the
authority may decide to request the operator to stop the mission and the CUAS
to interdict after a time-out. The sequence diagrams for these cases are
similar to the previous ones, hence not given here to avoid repetition.

###  V-H Protocol 8: Clarify Time Violation

111Protocol 8 is very similar to Protocol 7. The operator response may lead to
five possible outcomes with similar cases. For space reasons, we omitted the
outcomes table and the sequence diagrams.

This protocol is executed when the operator has exceeded the flight time
specified in the performance authorization according to the data in the AUTH-
DB. To resolve the time violation issue, the CUAS sends a TIME VIOLATION
message to the authority. The latter requests the operator to STOP MISSION IF
AUTHORIZED TIME IS EXCEEDED.

If the operator does not respond to this message (CASE 1), the authority
requests the operator to stop the mission and authorizes the CUAS to interdict
the drone after a time-out.When the operator contradicts the request by
claiming that he did not exceed permitted time (CASE 2) or when he declares
that he cannot stop the mission for any reason (CASE 3), the authority must
perform a fast risk assessment to take an appropriate decision.In CASE 4, the
operator confirms that he stopped the mission. The authority asks the CUAS for
confirmation. If the latter invalidates the mission stop (CASE 5), the
authority decides to request the operator to stop the mission and the CUAS to
interdict after a time-out.

##  VI System Simulation and Evaluation

To validate the post-detection model and the clarification protocols, we first
developed a high-level model for the system in Matlab. The purpose of this
model is to verify the functionality of the proposed system and to identify
any errors such as unreachable states, missing transitions, and deadlocks. For
this, we used the Stateflow Toolbox [38] that provides a graphical programming
language to design and test combinatorial and sequential decision logic using
state machines and flowcharts. Besides, this tool offers run-time and edit-
time checks to ensure design consistency and completeness before
implementation. We created three state machines using this toolbox: the post-
detection model that runs on the CUAS side and two additional machines to
model the behavior of authority and the operator. The three models were
synchronized using global events and executed in parallel to analyze the
interaction between the three agents. The toolbox generates sequence diagrams
to visualize the system operation as illustrated in the example of Figure 12
that shows the sequence diagram of case 3 of Protocol 8.

Figure 12: Simulated sequence diagram of Protocol 8 CASE 3

In addition to this functional simulation, we implemented the models and
protocols and conducted performance testing to evaluate the system behavior in
real-time. The goal is to analyze the system performance under different
scenarios and with multiple drones. For this purpose, we used Node.js that
offers an asynchronous event-driven environment for building scalable
applications. In particular, an event loop handles multiple requests
simultaneously and invokes a callback function only when a response is
returned. We used an open-source library called WebSocket to create a full-
duplex connection for message exchange between the server and clients. Since
the authority is the central agent in the UTM system, we defined it as the
system server. Multiple CUAS and UAV operators can be connected as clients
through different ports. The authority and the CUAS operators are granted
access to the AUTH-DB and ID-DB databases implemented in SQL. An overview of
the simulation setup is illustrated in Figure 13. Furthermore, we defined
entity relationships between the drone, operator, and missions, as illustrated
in Figure 14. This entity-relationship diagram shows that an operator is
identified using an 8-byte identifier and can be associated with one or more
registered drones. The drone is associated with one or more authorized
missions, as long as these missions don’t overlap in time.

Figure 13: Overview of the simulation environment. Figure 14: Entity
relationship diagram

We defined simulation scenarios to estimate what we refer to as _clarification
time_ , which is the time elapsed between detecting a drone by the CUAS system
and receiving a related decision from the authority. The simulation scenarios
were defined paying attention to complicated cases that account for worst-case
clarification times. Multiple scenarios were generated simultaneously to
explore the system behavior under peak load conditions. A library called
_async_ was used in both the CUAS and authority scripts to handle multiple
requests in an asynchronous non-blocking mode. We prepared an operator script
to model different behaviors throughout the simulation and mimic the different
responses to authority messages. Furthermore, we used a Node.js module called
_child_process_ to run the operator’s script multiple times concurrently. By
this means, we could vary the number of detected drones from 1 to 250 in steps
of 50 and measure the average clarification time for each protocol. The system
was simulated on a local machine with an Intel Core i9-8950HK CPU running at
2.90 GHz and 32-GB RAM.

Figure 15: Clarification time of executing the proposed protocols for
different numbers of drones.  Figure 16: Clarification time of proposed
protocols for 250 drones.

Figure 15 summarizes the simulation results showing the average clarification
time for different protocols and different numbers of drones. Assume, for
instance, that the CUAS system has detected 50 drones that all submit their
remote identification. However, the CUAS cannot find any of these IDs in the
ID-DB. The CUAS would initiate protocol 3 to clarify these cases. Figure 15
shows that in this case, every drone requires around 3.3 seconds to be
clarified, on average. It is important to highlight that this value refers to
the average clarification time for _every_ drone. So, it does not mean that
the 50 drones would require $3.3\times 50=165$ seconds. This is due to the
system’s capability of concurrent processing. The box-and-whisker plot in
Figure 16 demonstrates this aspect more clearly. Taking Protocol 1 and the
case of 250 drones as an example, we can see that the system can clarify 75%
of the cases in less than seven seconds while the maximum clarification time
is 7.5 seconds only. In the next section we discuss how these results should
be intepreted.

##  VII Discussion

To understand the impact of the clarification process on the system
performance, we should consider the simulation results in the context of a
full counter-drone operation that usually includes three steps:
detection/classification, clarification, and interdiction. Remember that the
system can decide to tolerate the drone as outlined in Figure 3. In such a
case, the clarification time becomes insignificant because there is no need to
counter the drone. In contrast, the clarification protocols can affect the
system performance when the drone is classified as uncooperative and needs to
be interdicted immediately or after a timeout. So, we have four delay
components in total:

  1. 1.

Detection and classification time.

  2. 2.

Clarification time

  3. 3.

Timeout

  4. 4.

Intediction time

We now discuss these time components and try to assign some example values
analytically or based on the literature. Reports on the time needed to detect
and classify a drone are scarce. Recently, Basak et al. [9] proposed a
combined drone detection and classification framework using Yolo Lite. They
reported a mean inference time of 1.16 seconds for the detection and
classification of one drone using a single signal. The timeout should be
sufficient for at least allowing the operator to land his drone safely. Safe
landing takes place at low speeds in the range of a couple of meters per
second, e.g., 4 m/sec as reported in [36]. So, if the drone is at an altitude
of 100m, the landing would take around 25 seconds. The interdiction time
depends on the used technology. Jamming is one of the fastest solutions due to
its non-kinetic nature. Although the jamming signal should be directed to
reduce side effects, the market is rich in off-the-shelf solutions with
omnidirectional interceptors that block the communication immediately after
switching on. So, the interdiction time of such systems would be negligible.
Finally, the simulation results showed that the clarification time is around
2.5 seconds in the case of one drone.

For these example values, we conclude that the contribution of the
clarification time to the total system delay is $2.5/(1.16+2.5)=68\%$, when
immediate interdiction is required, $2.5/(1.16+2.5+25)=9\%$, when the drone
should be interdicted after a timeout or $2.5/(1.16+2.5+\infty)=0\%$, when the
drone should be tolerated.

Although this quantitative analysis gives an overall estimation of the
proposed solution, it has some limitations which make the results look
optimistic or pessimistic. For example, our simulation ran on a single
computer. While a real deployment would use distributed systems and servers
that are typically faster, network delays would affect the clarification time.
Furthermore, we considered a best-case interdiction time of zero which makes
our results pessimistic. On the other hand, we did not include the operator’s
response time during the clarification process. Indeed, real-time measurements
on a working implementation of the system with detailed testing scenarios are
indispensable for understanding the system’s capabilities and challenges.

Apart from the timing aspects, we concentrated on functional aspects of the
system and did not implement security techniques, e.g., for message encryption
and authentication. System security is extremely important for this
application. Intruders could attack the system and inject fake messages that
can lead to drone misclassification with fatal consequences. Finally, in the
proposed system, we did not define the messages’ formats. We believe that such
activities are beyond the scope of this research and should be addressed
collectively by consortiums interested in standardization.

##  VIII Conclusion

Currently, developers of UTM systems and counter-drone systems are working in
parallel. Drone operators are still benefiting from the current legal
situation that prohibits the interdiction of aerial vehicles. Many efforts are
on the way to change this situation and we will soon see regulations for
counter-drone operations. But like always, regulations are for those who
follow them. The system must be prepared to deal with malfunctions, technical
faults, and malicious users. The paper showed that the real-time coordination
of the drone and counter-drone operation is not a trivial task. The simulation
results are promising but real deployments and pilot tests are required to
understand the actual dimensions of the problem and the challenges to
overcome. We hope that the proposed architecture, models, and protocols will
give some directions for system developers to start integrating CUAS and UTM
systems and address open issues.

## References

  * [1] Federal Aviation Administration.  UAS Remote Identification Overview, 2021.
  * [2] European Union Aviation Safety Agency.  Commission Delegated Regulations (EU) 2020/1058, 2020.
  * [3] Cemal Aker and Sinan Kalkan.  Using deep networks for drone detection.  In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6. IEEE, 2017.
  * [4] Azza Allouch, Omar Cheikhrouhou, Anis Koubâa, Khalifa Toumi, Mohamed Khalgui, and Tuan Nguyen Gia.  Utm-chain: blockchain-based secure unmanned traffic management for internet of drones.  Sensors, 21(9):3049, 2021.
  * [5] Riham Altawy and Amr M Youssef.  Security, privacy, and safety aspects of civilian drones: A survey.  ACM Transactions on Cyber-Physical Systems, 1(2):1–25, 2016.
  * [6] Muhammad Zohaib Anwar, Zeeshan Kaleem, and Abbas Jamalipour.  Machine learning inspired sound-based amateur drone detection for public safety applications.  IEEE Transactions on Vehicular Technology, 68(3):2526–2534, 2019\.
  * [7] Michael J Armstrong, George R Hutchins, and Timothy A Wachob.  Interdiction and recovery for small unmanned aircraft systems, September 3 2019.  US Patent 10,401,129.
  * [8] Cristina Barrado, Mario Boyero, Luigi Brucculeri, Giancarlo Ferrara, Andrew Hately, Peter Hullah, David Martin-Marrero, Enric Pastor, Anthony Peter Rushton, and Andreas Volkert.  U-space concept of operations: A key enabler for opening airspace to emerging low-altitude operations.  Aerospace, 7(3):24, 2020.
  * [9] Sanjoy Basak, Sreeraj Rajendran, Sofie Pollin, and Bart Scheers.  Combined rf-based drone detection and classification.  IEEE Transactions on Cognitive Communications and Networking, 2021\.
  * [10] Oussama Bekkouche, Miloud Bagaa, and Tarik Taleb.  Toward a utm-based service orchestration for uavs in mec-nfv environment.  In 2019 IEEE Global Communications Conference (GLOBECOM), pages 1–6. IEEE, 2019.
  * [11] Anjan Chakrabarty, Corey A Ippolito, Joshua Baculi, Kalmanje S Krishnakumar, and Sebastian Hening.  Vehicle to vehicle (v2v) communication for collision avoidance for multi-copters flying in utm–tcl4.  In AIAA Scitech 2019 Forum, page 0690, 2019.
  * [12] Christopher Chin, Karthik Gopalakrishnan, Maxim Egorov, Antony Evans, and Hamsa Balakrishnan.  Efficiency and fairness in unmanned air traffic flow management.  IEEE Transactions on Intelligent Transportation Systems, 2021.
  * [13] Christopher Courtin, Michael J Burton, Alison Yu, Patrick Butler, Parker D Vascik, and R John Hansman.  Feasibility study of short takeoff and landing urban air mobility vehicles using geometric programming.  In 2018 Aviation Technology, Integration, and Operations Conference, page 4151, 2018.
  * [14] Gordon Darroch.  Gatwick airport: Drones ground flights.  BBC, 2018-20-12.
  * [15] DeDrone.  Worldwide drone incidents, 2020.
  * [16] Jan Dentler, Somasundar Kannan, Miguel Angel Olivares Mendez, and Holger Voos.  A real-time model predictive position control with collision avoidance for commercial low-cost quadrotors.  In 2016 IEEE conference on control applications (CCA), pages 519–525. IEEE, 2016.
  * [17] Jedrzej Drozdowicz, Maciej Wielgo, Piotr Samczynski, Krzysztof Kulpa, Jaroslaw Krzonkalla, Maj Mordzonek, Marcin Bryl, and Zbigniew Jakielaszek.  35 ghz fmcw drone detection system.  In 2016 17th International Radar Symposium (IRS), pages 1–4. IEEE, 2016.
  * [18] FAA.  Unmanned aircraft system (uas) traffic management (utm) concept of operation.  Technical report, 2020.
  * [19] FAA.  Uas data exchange (laanc).  https://www.faa.gov/uas/programs_partnerships/data_exchange/, 2020-29-02.
  * [20] Mostafa Hassanalian and Abdessattar Abdelkefi.  Classifications, applications, and design challenges of drones: A review.  Progress in Aerospace Sciences, 91:99–131, 2017.
  * [21] Geon-Hwan Kim, Imtiaz Mahmud, and You-Ze Cho.  Self-recovery scheme using neighbor information for multi-drone ad hoc networks.  In 2017 23rd Asia-Pacific Conference on Communications (APCC), pages 1–5. IEEE, 2017.
  * [22] Chao Lin, Debiao He, Neeraj Kumar, Kim-Kwang Raymond Choo, Alexey Vinel, and Xinyi Huang.  Security and privacy for the internet of drones: Challenges and solutions.  IEEE Communications Magazine, 56(1):64–69, 2018.
  * [23] Huan Liu, Xiamiao Li, Guohua Wu, Mingfeng Fan, Rui Wang, Liang Gao, and Witold Pedrycz.  An iterative two-phase optimization method based on divide and conquer framework for integrated scheduling of multiple uavs.  IEEE Transactions on Intelligent Transportation Systems, 2020.
  * [24] Georgia Lykou, Dimitrios Moustakas, and Dimitris Gritzalis.  Defending airports from uas: A survey on cyber-attacks and counter-drone sensing technologies.  Sensors, 20(12):3537, 2020.
  * [25] Arthur Holland Michel.  Counter-drone systems, 2019.
  * [26] Thomas Multerer, Alexander Ganis, Ulrich Prechtel, Enric Miralles, Askold Meusling, Jan Mietzner, Martin Vossiek, Mirko Loghi, and Volker Ziegler.  Low-cost jamming system against small drones using a 3d mimo radar based tracking.  In 2017 European Radar Conference (EURAD), pages 299–302. IEEE, 2017.
  * [27] Natasha Neogi, Siddhartha Bhattacharyya, Daniel Griessler, Harshitha Kiran, and Marco Carvalho.  Assuring intelligent systems: Contingency management for uas.  IEEE Transactions on Intelligent Transportation Systems, 2021.
  * [28] Phuc Nguyen, Mahesh Ravindranatha, Anh Nguyen, Richard Han, and Tam Vu.  Investigating cost-effective rf-based detection of drones.  In Proceedings of the 2nd Workshop on Micro Aerial Vehicle Networks, Systems, and Applications for Civilian Use, pages 17–22, 2016.
  * [29] James O’Malley.  The no drone zone.  Engineering & Technology, 14(2):34–38, 2019.
  * [30] Seongjoon Park, Hyeong Tae Kim, Sangmin Lee, Hyeontae Joo, and Hwangnam Kim.  Survey on anti-drone systems: Components, designs, and challenges.  IEEE Access, 9:42635–42659, 2021.
  * [31] Colleen Reiche, Adam P Cohen, and Chris Fernando.  An initial assessment of the potential weather barriers of urban air mobility.  IEEE Transactions on Intelligent Transportation Systems, 2021.
  * [32] Julian Rothe, Michael Strohmeier, and Sergio Montenegro.  A concept for catching drones with a net carried by cooperative uavs.  In 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages 126–132. IEEE, 2019.
  * [33] Richard Ryan, Saba Al-Rubaye, Graham Braithwaite, and Dimitrios Panagiotakopoulos.  The legal framework of utm for uas.  In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC), pages 1–5. IEEE, 2020.
  * [34] Zsolt Sándor.  Challenges caused by the unmanned aerial vehicle in the air traffic management.  Periodica polytechnica transportation engineering, 47(2):96–105, 2019.
  * [35] Mia Stevens and Ella Atkins.  Geofence definition and deconfliction for uas traffic management.  IEEE Transactions on Intelligent Transportation Systems, 2020.
  * [36] Indreswari Suroso and Erwhin Irmawan.  Analysis of uav multicopter of air photography in new yogyakarta international airports.  TELKOMNIKA, 17(1):521–528, 2019.
  * [37] Bilal Taha and Abdulhadi Shoufan.  Machine learning-based drone detection and classification: State-of-the-art in research.  IEEE Access, 7:138669–138682, 2019.
  * [38] Inc. The MathWorks.  Stateflow toolbox, 2021.
  * [39] Cynthia Wolter, Lynne Martin, and Kimberly Jobe.  Human-system interaction issues and proposed solutions to promote successful maturation of the utm system.  In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC), pages 1–7. IEEE, 2020.
  * [40] Philippe Martin Wyder, Yan-Song Chen, Adrian J Lasrado, Rafael J Pelles, Robert Kwiatkowski, Edith OA Comas, Richard Kennedy, Arjun Mangla, Zixi Huang, Xiaotian Hu, et al.  Autonomous drone hunter operating by deep learning and all-onboard computations in gps-denied environments.  PloS one, 14(11), 2019.
  * [41] Chenchen Xu, Xiaohan Liao, Junming Tan, Huping Ye, and Haiying Lu.  Recent research progress of unmanned aerial vehicle regulation policies and technologies in urban low altitude.  IEEE Access, 8:74175–74194, 2020.
  * [42] Jiazhen Zhou, Dawei Sun, Inseok Hwang, and Dengfeng Sun.  Control protocol design and analysis for unmanned aircraft system traffic management.  IEEE Transactions on Intelligent Transportation Systems, 2021.


# On Input Formats for Radar Micro-Doppler Signature Processing by
Convolutional Neural Networks

Mikolaj Czerkawski1  Carmine Clemente1  Craig Michie1  Christos Tachtatzis1
<EMAIL_ADDRESS>1Department of Electronic and Electrical
Engineering, University of Strathclyde, Glasgow, UK

###### Abstract

Convolutional neural networks have often been proposed for processing radar
Micro-Doppler signatures, most commonly with the goal of classifying the
signals. The majority of works tend to disregard phase information from the
complex time-frequency representation. Here, the utility of the phase
information, as well as the optimal format of the Doppler-time input for a
convolutional neural network, is analysed. It is found that the performance
achieved by convolutional neural network classifiers is heavily influenced by
the type of input representation, even across formats with equivalent
information. Furthermore, it is demonstrated that the phase component of the
Doppler-time representation contains rich information useful for
classification and that unwrapping the phase in the temporal dimension can
improve the results compared to a magnitude-only solution, improving accuracy
from 0.920 to 0.938 on the tested human activity dataset. Further improvement
of 0.947 is achieved by training a linear classifier on embeddings from
multiple-formats.

###### keywords:

Radar, Automatic Target Recognition, Convolutional Neural Networks,
Classification, Spectrogram

##  1 Introduction

Doppler-time representations are widely used for tasks involving high-level
processing of Micro-Doppler radar signatures. Doppler-time representation is
most commonly obtained via the Short-Time Fourier Transform of the rreceived
radar signal, yielding 2D arrays of complex values. The spectrogram format is
often extracted as the squared magnitude of the resulting complex STFT (Short-
Time Fourier Transform). Despite the wide use of convolutional neural networks
for processing this type of signals Seyfioglu2018 ; Seyfioglu2019 ;
Shrestha2020 ; Zhu2020 ; Addabbo2021 ; Li2022 ; CzerkawskiInterference ;
CzerkawskiCoherence , it is not immediately obvious whether the spectrogram
representation, or any other format, should be preferred. In this work, a set
of experiments is designed and executed to provide further insight into the
decision-making process in this context.

The main contribution of this work is the analysis of the effect of input
representation on the performance of the classifier, including a novel method
based on multi-domain classifier. It has been found that the format of the
input representation has a considerable influence on the performance of a
convolutional neural network classifier, and hence, the findings presented
here can bring a valuable impact on how classifiers of Micro-Doppler
signatures are designed.

A brief overview of the past approaches to the problem is provided in Section
2. This is followed by a description of the proposed experiments and
discussion of the respective results in Section 3. The paper concludes with
Section 5.

##  2 Related Work

The focus of the presented work lies on the classification of targets from
Doppler-time signal representation, a task, where each sample is mapped to a
separate confidence score corresponding to an individual class. While there
are other tasks that involve Doppler-time representations, such as image
synthesis, the task of classification appears to be the most prominent and is
hence the primary topic of this work. In the work of Seyfioglu2018
introducing a deep convolutional autoencoder for radar-based human activity
classification, the spectrograms are cropped to 4 seconds, and then converted
to a single-channel representation of magnitude normalized to range of 0 and
1. A similar practice is followed in a related work on transfer learning
techniques for Micro-Doppler motion classification in Seyfioglu2019 , with
further spatial downsampling employed. Further, in other works such as
Shrestha2020 ; Zhu2020 ; Addabbo2021 ; Li2022 , the authors operate on
spectrogram representation. In a separate line of work, classifiers operating
on real and imaginary 2-channel representations have been proposed in
CzerkawskiInterference ; CzerkawskiCoherence .

To date, implementations of convolutional neural network models for
classification of Doppler-time representations either rely on magnitude
single-channel input, or real-imaginary double-channel. The experiments
presented in this work aim to bridge the resulting gap, by exploring the
utility of a phase-based representation and comparing it to the conventional
formats.

##  3 Methodology

In this section, several approaches of supplying Doppler-time signals to
convolutional neural networks are discussed, as well as potential methods of
quantifying their effectiveness.

###  3.1 Potential Formats

Polar 2-channel Format. A commonly used representation of a Doppler-time power
distribution is its magnitude. In some applications Seyfioglu2018 ;
Seyfioglu2019 ; Shrestha2020 ; Zhu2020 ; Addabbo2021 ; Li2022 , the polar
magnitude component constitutes the entire representation. For completeness,
the phases of the complex value can be maintained as a second channel.
However, phase can be problematic due to its limited range of 2$\pi$ and the
risk of wrapping, which can yield distortions in the signal. On the other
hand, unwrapped phase can result in very large variance of the phase
component, which could be difficult to process for a neural network. A
potential solution could involve normalized unwrapped phase, but that changes
the original phase scaling, which could be an important piece of information.

Rectangular 2-channel Format. The issues with phase ambiguity can be
alleviated by using the quadrature format, where real and imaginary parts of
the complex value are mapped into individual channels. However, that makes the
magnitude information harder to recover for the network (it must approximate
$\ell_{2}$ norm, or learn other features that will deliver the equivalent
information). Furthermore, the small differences in phase may be difficult to
detect and again, a computationally complex inverse tangent operation must be
approximated, or a set of features delivering equivalent information in a
different manner.

Polar-Rectangular 4-channel Format. An alternative solution is to use both
rectangular and polar formats, in a 4-channel representation. For many
applications, this should not greatly increase model size, since only the
first (and potentially last) layers must be changed, doubling the number of
weights compared to a 2-channel counterpart.

###  3.2 Processing Complex Data by Neural Networks

Deep Neural Networks are known to be highly expressive models, meaning that
with enough width and depth of layers, they can approximate a wide set of
functions. Based on this, it could be argued that the rectangular and polar
representations can both be a valid format of the input data, since they
contain equivalent and complete information. Should one format be favourable,
the network is theoretically capable of learning an approximation of a
function needed to transform into that format. For example, it is rather
straightforward to learn the trigonometric function of sine over a finite
range (which will indeed be finite in practical scenarios). However, learning
these functions, or more complex $\ell_{2}$ norm required to calculate
magnitude from rectangular format, may introduce an unnecessary burden on the
network, where more capacity is spent on learning the transform, rather than
the higher abstraction representations that are more useful for
classification. This effect may be particularly apparent for scenarios with
relatively small networks and small datasets, which is often the case for
Doppler-time signal classification problems.

Figure 1: Single-channel formats considered in analysis.

###  3.3 Testing Methods

Two methods of testing the utility of each representation are proposed herein.
First, a classification test with separate representation-specific models is
conducted by training single-domain models on the same training dataset and
with the same hyperparameters, and then comparing the achieved accuracy.
However, the dynamics of each network training can be quite different and
difficult to control. This motivates another type of experiment, where a model
consisting of several domain-specific convolutional encoders is trained on all
domain simultaneously (as later shown in Figure 3). To ensure that no domain
input is ignored, at any step only 2 domain encodings are supplied to the
final classifier head, selected at random, which forces the model to learn
from all domains in order to minimize the classification error. This leads to
a shared multi-domain model that is then analysed in several respects to
provide insight into which input representations should be preferred.

##  4 Results

###  4.1 Training Details

The dataset used for this work contains signatures of human motions (available
at http://researchdata.gla.ac.uk/848/) glasgowdataset , previously used in
works such as LiLSTM ; LiTGRS ; CzerkawskiInterference ; CzerkawskiCoherence .
The dataset contains 1,752 samples of 6 human activities (walking, sitting
Down, standing up, object pick up, drinking, and fall), and has been split
into 50% training, 25% validation, and 25% testing subsets. While each sample
comes from the same FMCW radar that offers ranging capability, all range bins
of the Range-Time representations are summed to yield a one-dimensional
signature, that is then subject to Short-Time Fourier Transform with a 128-bin
Blackman window of 0.2 seconds and 0.19-second overlap. The resulting time-
frequency distribution is resampled to yield a 128 by 128 element matrix of
complex values. Depending on the tested representation, the rectangular or
polar components of the resulting complex value is obtained as the network
input.

The employed convolutional neural network architecture is based on the model
used in the previously published papers of CzerkawskiInterference ;
CzerkawskiCoherence  (which are loosely based on models from Seyfioglu2018 ;
Seyfioglu2019 ), accepting 128 by 128 image input, and it consists of 5
convolutional blocks, each consisting of a convolutional layer (kernel of 3 by
3 with a stride of 2), followed by a batch normalization layer and a LeakyReLU
activation, as illustrated in Figure 2. The output of the last convolutional
block contains 8,192 values, and is linearly mapped to 128-dimensional
intermediate embedding, which, in turn, is linearly mapped to 6 confidence
scores. Depending on the type of the representation, the number of input
channels in_ch of the 128 by 128 image is adjusted, as indicated in Figure 2.

The model is trained for a fixed number of 50 epochs with an Adam optimizer
set to a learning rate of $10^{-4}$. A scheduler ReduceLROnPlateau with a
factor of 0.5 and patience of 4 epochs is used to adjust the learning rate
during the training process.

###  4.2 Classification Test (Single-Domain)

Figure 2: Diagram of the employed convolutional neural network architecture.
The variable in_ch indicates the number of input channels that changes
depending on the input representation format (either 1, 2, or 4 channels, as
described in the text).

A direct way to compare the utility of each representation is to train the
same model (apart from the rather minor difference in the number of input
channels) with fixed hyperparameters and the number of epochs and compare the
performance achieved with each input type. The results for such an experiment
are contained in Table 1, where 10 different models have been trained on
specific representations of interest, including a group of 5 polar-based
representations (magnitude, phase (W), unwrapped phase (U)111For all
experiments with phase unwrapping, this is executed in the temporal dimension,
as shown in Figure 1, polar with raw phase (W), and polar with unwrapped phase
(U)), a group of quadrature-based representations (real, imaginary,
rectangular), and finally, a combined 4-channel representation containing
rectangular and polar (with either wrapped or unwrapped phase). The accuracy
is measured on the test set on both clean samples and samples with additive
Gaussian noise of 0 dB SNR.

Table 1: Classification accuracy with separate
representation-specific models. Mode | Accuracy | Accuracy
---|---|---
| (Clean) | 0 dB Noise
Magnitude | 0.856 | 0.683
Phase (W) | 0.237 | 0.224
Phase (U) | 0.769 | 0.374
Polar 2-channel (W) | 0.349 | 0.269
Polar 2-channel (U) | 0.797 | 0.532
Real | 0.820 | 0.744
Imaginary | 0.833 | 0.756
Rectangular 2-channel | 0.815 | 0.749
Pol-Rect 4-channel (W) | 0.311 | 0.265
Pol-Rect 4-channel (U) | 0.795 | 0.441
|  |

As shown in Table 1, the highest level of performance is achieved using
magnitude input (0.856), and quadrature-based representations (0.815-0.833).
Out of these representations, the magnitude-based model appears to be more
susceptible to noise (0.683 accuracy at 0dB) when compared to the quadrature-
based models (0.744-0.756). Wrapped phase input yields very low performance of
0.237, yet its unwrapped equivalent achieves a far from trivial 0.769
accuracy. The disruptive nature of wrapped phase appears to also affect the
performance when it is merely a part of the input representation, such as
Polar 2-channel or Pol-Rect 4-channel. These observations verify the intuition
that using the unwrapped phase leads to a significant boost of performance.

At this point, an important finding worth noting is that phase information may
contain useful discriminative features for Doppler-time signal classification,
as indicated by the accuracy of 0.769 obtained with Phase (U). In this
experiment, concatenation of magnitude with wrapped or unwrapped phase results
in reduced performance compared to 0.856 accuracy achieved with magnitude-only
input. This could be attributed to the relationship between the labels and the
input being more complex than in the case of pure magnitude. This could be
attributed to the fact that the magnitude and phase representations exhibit
vastly different visual features and hence require disparate kernel filters.
Since the same encoder topology is preserved (with the only change being the
number of input channels), learning kernel weights that are optimal for the
two diverse domains is challenging. One solution to this problem is presented
in the following section, where both magnitude and phase representations are
processed by representation-specific encoders to boost performance.

###  4.3 Classification Test (Multi-Domain)

It is possible that each model reported in Table 1 had a different training
path due to a number of stochastic factors, like random sampling and network
initialization. Testing with repetitions can mitigate that to some degree, but
the space of variation can still remain large. An alternative is proposed to
train a model accepting all 5 individual representations simultaneously with
representation-specific convolutional modules (as illustrated in Figure 3) and
summing the outputs of all modules to yield a combined representation. During
training, the output of each entry module is omitted with uniform probability
(by sampling only 2 out of 5 representations at any point), which aims to
prevent the network from ignoring any individual representation. During
testing, the accuracy achieved using any combination of representation can be
compared as an indication of how important each one is. No scaling of the
embedding vector is necessary, since the classifier head is linear, which
guarantees preserved class confidence ratios in the output. The network is
trained for 125 epochs instead of 50 to accommodate for the fact that each
representation will be present in only 40% of the optimization steps on
average.

Figure 3: Multi-domain classifier training, where only 2 domains are used at
any training step.

Since the model has the flexibility to take any combination of the 5 trained
representations, it is possible to test the level of performance achieved with
many input format types, while reusing the same trained model. The resulting
accuracy levels when utilizing each representation in isolation and
combinations thereof are presented in Table 2.

In this case, it is shown that including unwrapped phase along with magnitude
leads to a gain in accuracy from 0.920 to 0.938 under noiseless conditions.
This type of representation, Polar 2-channel (U), leads to the highest
accuracy of all representation in this experiment. Upon addition of 0dB
Gaussian noise, the accuracy is still as high as 0.916, exceeded marginally
only by the representation of Pol-Rect 4-channel (U), where 0.918 accuracy is
reached. To further confirm the utility of phase information, it can be
observed that the accuracy for the two phase-only representations is as high
as 0.616 for wrapped phase and 0.877 for unwrapped phase. The rectangular
representations seem to perform somewhat worse than polar-based counterparts,
with an accuracy range of 0.893-0.904 under clean conditions. Lastly, using a
combination of polar and rectangular does not seem to be beneficial when
compared to the Polar 2-Channel (U) variant.

Table 2: Classification performance with a combined
Multi-Domain model. Mode | Accuracy | Accuracy
---|---|---
| (Clean) | 0 dB Noise
Magnitude | 0.920 | 0.863
Phase (W) | 0.616 | 0.461
Phase (U) | 0.877 | 0.760
Polar 2-channel (W) | 0.911 | 0.861
Polar 2-channel (U) | 0.938 | 0.916
Real | 0.893 | 0.856
Imaginary | 0.895 | 0.858
Rectangular 2-channel | 0.904 | 0.893
Pol-Rect 4-channel (W) | 0.920 | 0.909
Pol-Rect 4-channel (U) | 0.925 | 0.918
Pol-Rect 5-channel (U&W) | 0.927 | 0.906
|  |

In both classification experiments, the wrapped phase representation appears
to decrease performance when appended to another representation. This
motivates another set of experiments aimed to explore further why and how the
wrapped phase representation influences the classification process.

###  4.4 Further Analysis

First, it is explored whether there is any correlation between the classes
predicted from individual representation. A strong correlation could mean that
the test dataset contains samples that are inherently difficult to classify
correctly based on what has been learned in the training dataset. To explore
this, the incorrect predictions from each representation are compared to
predictions based on the other representation, as shown in Table 3. The first
row contains the total number of incorrect predictions from each
representation. The following table rows contain the fraction of these
incorrect predictions that yield the same class prediction from another
representation. For the problematic representation of wrapped phase, the
consistency with other representations is quite small. For high-performing
representations, like magnitude and imaginary, the overlap can be as high as
0.514, indicating that more than half of the incorrectly predicted samples
based on the magnitude yields the same incorrect class label using imaginary
representation, which could be a sign of inherently misguiding features in the
training dataset that do not generalize well.

Table 3: Comparison of achieved classification performance Count | 47 | 46 | 35 | 168 | 54
---|---|---|---|---|---
| Re | Im | Mag | Ph (W) | Ph (U)
Re | 1.000 | 0.478 | 0.429 | 0.065 | 0.278
Im | 0.468 | 1.000 | 0.514 | 0.054 | 0.204
Mag | 0.319 | 0.391 | 1.000 | 0.030 | 0.185
Ph (W) | 0.234 | 0.196 | 0.143 | 1.000 | 0.111
Ph (U) | 0.319 | 0.239 | 0.286 | 0.036 | 1.000
|  |  |  |  |

An indication of the performance upper bound can be obtained by counting the
number of samples where at least one modality leads to correct prediction. For
the discussed experiment, this has been recorded to be the case for 432
samples out of the total 438 samples in the test dataset, yielding an upper
bound in accuracy of 0.986. It means that potentially, even higher performance
could be achieved by applying further processing to the output from individual
representations (for example, internal embeddings or output confidences).

Further perspective on this can be provided by counting the number of samples
in the test dataset, where only a single representation yields the correct
result. This is shown in Table 4, where it can be seen that some
representations do indeed yield a rather unique set of features that
outperforms other representations, most prominently the unwrapped phase is the
only representation correctly identifying 8 out of the 438 samples. In total,
there is a total of 18 out of 438 samples that are correctly identified by
only one of the representations, equivalent to a potential accuracy margin of
0.041. This means that the accuracy of 0.920 achieved using the magnitude
representation could potentially be increased to 0.956 (+16/438) by taking
advantage of unique predictions from other representations. This requires an
additional processing step to pick-out the right set of predictions based on
the multiple representations.

Table 4: Number of unique samples in the test dataset where
only one representation yields correct output. Representation | Unique Sample Count
---|---
Real | 0 out of 438
Imaginary | 4 out of 438
Magnitude | 2 out of 438
Phase (W) | 4 out of 438
Phase (U) | 8 out of 438
Total | 18 out of 438
|

Whether this level of performance is achievable with the following encoders
(or rather, how easy it is to apply a correction to a collection of mode-
specific scores) is explored by training a ‘meta’ classifier module (on the
original training subset) that either takes the set of 5 embeddings of size
128 (provided by the penultimate layer in the diagram in Figure 2), or the set
of 6 final prediction scores. The extra model contains two hidden linear
layers with 64 channels each, and is retrained for 10 epochs. Four
configurations are tested, with one variable being the presence of LeakyReLU
non-linearity in the hidden representations, and the other being the type of
input (either a set of confidence scores of 5 x 6 or a set of embeddings of 5
x 128). The results in Table 5 indicate that the top accuracy of 0.938
recorded for the polar representation with unwrapped phase can be pushed
further to 0.947 by processing the embeddings with a linear module. For
confidence-based ‘meta’ module, no gain is observed.

Table 5: Performance achieved by employing a ’meta’
prediction module. Input Type | Activation | Test Accuracy
---|---|---
Confidence | Linear | 0.929
Confidence | LeakyReLU | 0.938
Embedding | Linear | 0.947
Embedding | LeakyReLU | 0.941
Base Model : Polar (U) | - | 0.938
|  |
Figure 4: Illustration of saliency map for each input representation and each
class. A threshold of 25% of the maximum value is applied for visualization.

The phase unwrapping in temporal domain appears to bring significant benefits,
while the wrapped phase is generally found to be a challenging representation
to work with. This can be further confirmed by inspecting saliency maps
Simonyan2014  in Figure 4 computed as the average gradient magnitude for each
class on the test dataset. For visualization purposes, a threshold of 25% of
maximum value is applied. It can be observed that the gradients of the wrapped
phase are distributed quite uniformly over the entire image, including high-
velocity regions (top and bottom), where little information about the type of
activity should be present. This effect is reduced in the case of unwrapped
phase in the bottom row of the grid, similarly to the saliency maps obtained
for the magnitude representation. The rectangular representation (Real and
Imaginary) appear to also be quite sensitive to high-velocity regions, but not
as strongly, and not as uniformly, as the wrapped phase, which is consistent
with the achieved intermediate level of performance. In conclusion, it appears
that the wrapped phase constitutes a noisy input representation, increasing
the risk of spurious correlations in the training dataset that do not
generalize well, and simply obscuring a degree of useful information. For this
reason, even though the unwrapping operation may appear trivial, it has a
strong effect on the capability of a convolutional neural network classifier.

##  5 Conclusion

The presented work confirms the strong influence of Doppler-time
representation format on convolutional neural network classifiers. The
magnitude spectrogram representation remains the most forgiving input
representation, significant information can be extracted from the phase
component of the time-frequency distribution, especially when unwrapping in
temporal domain is applied.

Further analysis demonstrates that depending on the representation, different
sets of features of the signatures are learned by a convolutional encoder,
meaning that specific input representations can be more helpful than others
for some of the samples. Further post-processing can be employed to take
advantage of this divergence, as demonstrated herein.

To the best of the authors’ knowledge, this is the first systematic analysis
of Doppler-time representations in the context of convolutional neural network
classifiers. A novel method for assessing the utility of each representation
type by training a Multi-Domain classifier is presented. The key impact of
this work can be summarized by the improved performance from the conventional
magnitude-based solution achieving 0.920, to 0.938 when unwrapped phase is
included in the input representation, or to 0.947 when a post-processing meta
module is employed.

Further work could be done to confirm the same results on different datasets
and different classification tasks that operate on radar Doppler-time
signatures.

## References

## References

  * [1] Seyfioǧlu, M., Özbayoǧlu, A. & Gürbüz, S. Deep convolutional autoencoder for radar-based classification of similar aided and unaided human activities. IEEE Transactions On Aerospace And Electronic Systems. 54, 1709-1723 (2018)
  * [2] Seyfioglu, M., Erol, B., Gurbuz, S. & Amin, M. DNN Transfer Learning from Diversified Micro-Doppler for Motion Classification. IEEE Transactions On Aerospace And Electronic Systems. 55, 2164-2180 (2019)
  * [3] Shrestha, A., Li, H., Le Kernec, J. & Fioranelli, F. Continuous Human Activity Classification from FMCW Radar with Bi-LSTM Networks. IEEE Sensors Journal. 20, 13607-13619 (2020)
  * [4] Zhu, J., Chen, H. & Ye, W. A Hybrid CNN-LSTM Network for the Classification of Human Activities Based on Micro-Doppler Radar. IEEE Access. 8 pp. 24713-24720 (2020)
  * [5] Addabbo, P., Bernardi, M., Biondi, F., Cimitile, M., Clemente, C. & Orlando, D. Temporal convolutional neural networks for radar micro-doppler based gait recognition†. Sensors (Switzerland). 21, 1-15 (2021)
  * [6] Li, X., He, Y., Fioranelli, F. & Jing, X. Semisupervised Human Activity Recognition with Radar Micro-Doppler Signatures. IEEE Transactions On Geoscience And Remote Sensing. 60 (2022)
  * [7] Czerkawski, M., Ilioudis, C., Clemente, C., Michie, C., Andonovic, I. & Tachtatzis, C. A novel micro-doppler coherence loss for deep learning radar applications. European Radar Conference (EuRAD) 2021. (2021,9)
  * [8] Czerkawski, M., Ilioudis, C., Clemente, C., Michie, C., Andonovic, I. & Tachtatzis, C. Interference Motion Removal for Doppler Radar Vital Sign Detection Using Variational Encoder-Decoder Neural Network. 2021 IEEE Radar Conference (RadarConf21). pp. 1-6 (2021)
  * [9] Li, Z., Le Kernec, J., Fioranelli, F., Romain, O., Zhang, L. & Yang, S. An LSTM Approach to Short-range personnel recognition using Radar Signals. 2021 IEEE Radar Conference (RadarConf21). pp. 1-6 (2021)
  * [10] Li, X., He, Y., Fioranelli, F. & Jing, X. Semisupervised Human Activity Recognition With Radar Micro-Doppler Signatures. IEEE Transactions On Geoscience And Remote Sensing. 60 pp. 1-12 (2022)
  * [11] Fioranelli, F., Shah, S., Li, H., Shrestha, A. & Le Kernec, J. Radar signatures of human activities. (University of Glasgow,2019), http://researchdata.gla.ac.uk/id/eprint/848
  * [12] Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. 2nd International Conference On Learning Representations, ICLR 2014 - Workshop Track Proceedings. pp. 1-8 (2014)