Corresponding author: Subash Neupane (e-mail<EMAIL_ADDRESS>
# Security Considerations in AI-Robotics: A Survey of Current Methods,
Challenges, and Opportunities

SUBASH NEUPANE1  SHASWATA MITRA2  IVAN A. FERNANDEZ3  SWAYAMJIT SAHA4  SUDIP
MITTAL5  JINGDAO CHEN6  NISHA PILLAI7  SHAHRAM RAHIMI8  Department of Computer
Science & Engineering, Mississippi State University, Mississippi, (email:
<EMAIL_ADDRESS> Department of Computer Science & Engineering, Mississippi
State University, Mississippi, (email<EMAIL_ADDRESS> Department of
Computer Science & Engineering, Mississippi State University, Mississippi,
(email<EMAIL_ADDRESS> Department of Computer Science & Engineering,
Mississippi State University, Mississippi, (email<EMAIL_ADDRESS>Department of Computer Science & Engineering, Mississippi State University,
Mississippi, (email<EMAIL_ADDRESS> Department of Computer Science &
Engineering, Mississippi State University, Mississippi, (email:
<EMAIL_ADDRESS> Department of Computer Science & Engineering,
Mississippi State University, Mississippi, (email<EMAIL_ADDRESS>Department of Computer Science & Engineering, Mississippi State University,
Mississippi, (email<EMAIL_ADDRESS>
###### Abstract

Robotics and Artificial Intelligence (AI) have been inextricably intertwined
since their inception. Today, AI-Robotics systems have become an integral part
of our daily lives, from robotic vacuum cleaners to semi-autonomous cars.
These systems are built upon three fundamental architectural elements:
perception, navigation and planning, and control. However, while the
integration of AI-Robotics systems has enhanced the quality our lives, it has
also presented a serious problem - these systems are vulnerable to security
attacks. The physical components, algorithms, and data that make up AI-
Robotics systems can be exploited by malicious actors, potentially leading to
dire consequences. Motivated by the need to address the security concerns in
AI-Robotics systems, this paper presents a comprehensive survey and taxonomy
across three dimensions: attack surfaces, ethical and legal concerns, and
Human-Robot Interaction (HRI) security. Our goal is to provide users,
developers and other stakeholders with a holistic understanding of these areas
to enhance the overall AI-Robotics system security. We begin by surveying
potential attack surfaces and provide mitigating defensive strategies. We then
delve into ethical issues, such as dependency and psychological impact, as
well as the legal concerns regarding accountability for these systems.
Besides, emerging trends such as HRI are discussed, considering privacy,
integrity, safety, trustworthiness, and explainability concerns. Finally, we
present our vision for future research directions in this dynamic and
promising field.

###### Index Terms:

AI-Robotics, Cybersecurity, Attack Surfaces, Ethical and Legal Concerns,
Human-Robot Interaction (HRI) Security.

##  I Introduction and Motivation

Robotics is a multidisciplinary field of engineering that deals with the
design, construction, and operation of intelligent machines called robots,
capable of sensing the environment and executing tasks autonomously or through
human guidance. Robotic systems are comprised of sensors and actuators that
can interact with the environment to accomplish a set of tasks [1]. The field
has made remarkable strides in reshaping industries such as transportation,
manufacturing, healthcare, agriculture, entertainment, space exploration,
military, and beyond. On the other hand, Artificial Intelligence (AI) is the
simulation of human intelligence processes by machines, especially computer
systems. In this context, AI-Robotics systems may be referred to as
integration and collaboration between AI and robotic technologies. These
systems combine AI algorithms, which enable machines to perform tasks that
typically require human intelligence, with robotic platforms designed to
interact with and manipulate the physical world. The recent advances in
Machine Learning (ML) and Deep Learning (DL) techniques have paved the way for
the development of robots with cognitive abilities to execute functions such
as perception, navigation, planning, and control. This empowerment enables
robots to perceive, learn, deduce, and make real-time decisions, closely
mimicking human behaviors. With ML/DL algorithms continuing to advance, the
synergy with robotics propels us toward a future where robots play pivotal
roles in societies, offering unprecedented levels of automation, efficiency,
and adaptability.

As AI-Robotics systems become increasingly sophisticated, versatile, and
integrated into our daily lives, it also revolutionize how we work, live, and
explore the world. For example, in the domain of autonomous vehicles and
transportation, self-driving cars (Waymo [2]), drones, and Unmanned Aerial
Vehicles (UAVs) are revolutionizing transportation systems. Similarly, in
industrial automation and manufacturing processes, to improve efficiency,
productivity, and safety, collaborative robots [3] perform tasks such as
assembly, welding, material handling, and quality inspection. In healthcare,
AI-Robotics systems have also demonstrated their potential in revolutionizing
surgery as evidenced by Smart Tissue Autonomous Robot (STAR) [4], and
rehabilitation processes as discussed in [5]. In agriculture, where they
assist in diverse tasks like planting, harvesting, and crop health monitoring,
enabling the realization of precision agriculture, as highlighted in the
research of Shamshiri et al. [6] and Gupta et al. [7, 8]. Alongside day-to-day
civilian activities, defense and military operations are equally utilizing AI-
Robotics systems to improve strategic capabilities while minimizing human
exposure to danger [9]. From autonomous surveillance drones that enhance
situational awareness and warfare missions to robotic vehicles for logistics
transportation and reconnaissance in hazardous environments, AI-Robotics is
revolutionizing how military forces plan, execute, and adapt.

The security of robotic systems has been an active research paradigm for a
long time [10, 11, 12, 13, 14, 15, 16, 17, 18]. Due to their pervasive nature,
AI-Robotics systems are susceptible to diverse security attacks by threat
actors aiming to exploit vulnerabilities in the underlying AI technologies.
These attacks are capable of of compromising the functionality, safety, and
reliability of a system. Adversarial attacks, data breaches, and system
manipulations can lead AI-Robotics systems to cause accidents, property
damage, and even endanger human lives.

Recently, there have been several examples of adversarial attacks on semi-
autonomous vehicles that demonstrate the security risk in AI-Robotics. In
2020, researchers manipulated Tesla cars into speeding up by 50 miles per hour
by subtly altering a speed limit sign on the side of a road, fooling the car’s
Mobileye EyeQ3 camera system [19]. While, in 2015, a Jeep Cherokee got
remotely hacked while being driven, causing Fiat Chrysler to recall 1.4
million vehicles and pay $105 million in fines to the National Highway Traffic
and Safety Administration [20]. The vulnerability occurred due to a lack of
code signing in the head unit of the car or Electronic Control Unit (ECU),
allowing the gateway chip to get reprogrammed and send instructions to vehicle
components, such as the steering wheel and brakes [21]. Furthermore, attacks
on confidentiality, privacy, integrity, and availability can affect trust and
acceptance among users [10, 22, 23, 24]. Therefore, understanding the security
dimension of AI-Robotics systems and familiarizing oneself with the existing
methods to defend against in the event of an attack holds the utmost
importance.

Furthermore, the integration of robots into various aspects of our lives has
given rise to pressing ethical and legal concerns. These concerns extend
beyond security attacks and are intertwined with the growing cognitive
capabilities of AI-Robotics systems. Ethically, the influence of robots on
human behavior and emotions is a significant issue. In scenarios involving
medical, social and assistive robots, the potential for robots to form
emotional bonds with users raises questions about risk of dependence on
machines and their psychological impact on individuals [25]. Also, the overt
reliance on autonomous robotic agents for decision-making processes can have
negative impacts on human cognitive thinking and reasoning capacities,
ultimately leading to a loss of human autonomy [26]. On the legal front,
navigating liability and responsibility within robotic systems presents a
complex challenge. For instance, determining accountability when a “remote
assisted telesurgery” procedure goes awry requires careful consideration.
Moreover, the exchange of personal data and information between robots and
humans raises data privacy concerns, necessitating adherence to legal
frameworks like the 2020 California Consumer Privacy Act (CCPA) [27] and the
2016 General Data Protection Regulation (GDPR)[28].

Besides, growing cognitive capabilities of AI-Robotics systems have allowed
robots to interact with humans raising security concerns over Human-Robot
Interaction (HRI). Ethical and legal dimension of these interaction revolve
around integrity, privacy, safety of these interactions. Addressing these
ethical and legal intricacies is paramount to ensuring responsible, respectful
and secure interactions between robots and humans.

Motivated by these three dimensions i.e. attack surfaces and mitigation
strategies, ethical and legal concerns, and HRI security, this paper aims
provide a holistic review of these critical aspect to improve overall security
posture of AI-Robotics systems.

To the best of our knowledge, this paper is the first to dissect the AI-
enabled attack surfaces in a generic hybrid AI-Robotics architecture 1.
Furthermore, we also investigate the ethical and legal concerns surrounding
the accessibility, fairness, trustworthiness, explainability, and
accountability of robotic applications, along with the security studies of
HRI.

The major contributions of this paper are as follows:

  * •

We present a taxonomy structured around three dimensions of AI-Robotics
systems security: attack surfaces and mitigation strategies, ethical and legal
concerns they raise, and security of HRI. This taxonomy is designed to augment
the overall security of these systems by offering stakeholders comprehensive
insights across these dimensions.

  * •

We provide a comprehensive survey of the attack surfaces within hybrid AI-
robotics systems, encompassing architectural layers such as perception,
navigation and planning, and control. In addition, we present state-of-the-art
defenses aimed at significantly reducing the risk of attacks across these
layers.

  * •

We discuss the challenges inherent within AI models, frameworks, software and
explain how the existing vulnerabilities within these tools can inadvertently
contribute to undermining the integrity of the final robotic systems.

  * •

We present the multifaceted exploration of the ethical and legal dimensions
associated with AI-Robotics systems.

  * •

We address the intricate security issues pertaining to HRI, with the focus on
privacy concerns. Within this context, we recommend strategic defensive
solutions.

  * •

We provide research recommendations organized according to the three
dimensions mentioned earlier, with the goal of enhancing the robustness and
resilience of future AI-Robotic systems.

The remainder of the sections are as follows: Section II provides background
on robotics architecture. Section III summarizes our survey and taxonomy.
Following that, we investigate on attack surfaces on various layer of robotic
architecture in Section IV. Then, we discuss ethical & legal concerns
surrounding the use of AI-Robotic systems in Section V. Section VI discusses
security studies pertaining human robot interaction. Section VII identifies
research challenges and provides future direction in developing secured
robotic systems. Finally, Section VIII concludes our survey.

##  II AI-Robotics Systems Architecture

Figure 1: A generic robot architecture consisting of perception, navigation,
and control stacks. Icons from [29]

Robotic systems can be designed using a variety of architectural paradigms. In
this section, we will explore these paradigms further. Murphy et al. [30]
describe a range of current architectures such as hierarchical, reactive, and
hybrid deliberative-reactive in terms of the relationships between three
primitives (i.e., sense, plan, and act) for organizing intelligence in robots.
Table I attempts to characterize these three fundamental building blocks based
on their inputs and outputs. Hierarchical paradigms in robotics follow a top-
down approach, where the robot first senses the world, then plans its next
action, and finally executes it. A hierarchical paradigm is also known as a
deliberative architecture. Each layer in the hierarchy depends on the layer
below it for information and guidance. Reactive paradigms prioritize immediate
responses directly connected to sensor input, without extensive planning.
Hybrid deliberative-reactive paradigms strike a balance between reactive
responses and higher-level decision-making, utilizing both immediate reactions
and planned actions based on sensory information.

Primitives |  Input |  Output
---|---|---
Sense |  Sensor data from LiDAR, Cameras, GPS etc. |  Sensed information such as Obstacle location, Occupancy grids etc.
Plan |  Map information |  Waypoints, Path, Trajectory
Act |  Waypoints |  Actuator commands

TABLE I: Robot primitives defined in terms of input and output. Adapted from
[30].

Modern AI-Robotics systems commonly adopt a hybrid architecture that combines
both deliberative and reactive modules. Figure 1 illustrates a generic three-
layered hybrid architecture that corresponds to the sense-plan-act paradigm,
with Perception, Navigation & Planning, and Control as its constituent layers.
In the subsequent subsection, we will provide a detailed explanation of each
of these layers.

###  II-A Robotic Perception

The perception stack of the robot architecture involves sensing and
understanding relevant information about the state of the operating
environment by leveraging a number of sensors, for example, GPS,
accelerometer, inertial measurement unit (IMU), gyroscope, magnetometer,
camera, and tactile sensors among others [31]. The combination of multiple
sensors as depicted in Fig 1 make up perception stack. These sensors play a
pivotal role in actively sensing the raw data that represent the physical
state of robotics systems in the environment. Furthermore, this sensory data
aids in planning and calculation of actuator signals in the subsequent stages.
[32].

Modern AI-Robotics systems require high-level sensing abilities, such as
object detection, localization, positional information, and interaction with
humans. An important aspect of achieving successful perception functions in
robotics is gaining a deep understanding of the operating environment. One
highly influential approach for environment understanding, especially in the
context of mobile robots and autonomous vehicles, is the representation of the
environment through 2D mappings known as occupancy grid mapping [33]. However,
attention has now shifted towards 3D representations. Recent technological
advancements, including Octomaps [34], and the widespread use of RGBD sensors
with depth capabilities have significantly enhanced the construction of larger
and more detailed 3D maps for detailed 3D-like environment representations.
Efforts have been focused on semantically labeling these maps by integrating
them with Simultaneous Localization and Mapping (SLAM) frameworks. This
integration enables autonomous navigation and precise self-localization based
on the generated map.

Furthermore, the advances in AI have enabled the gathering, analysis, and
prediction of visual, audio, depth, and tactile information that is required
for perception functions [35]. Object detection and localization are mostly
performed through visual information (image) and depth information using
Convolution Neural Network (CNN) architecture [36, 37, 38], and its variants
Recurrent CNN [39, 40], and convolutional auto-encoder [41]. More recently,
the state-of-the-art Visual Transformer [42] and its variants have also been
used to improve perception performance [43], object detection, and tracking
[44] among others.

Next, we elaborate on how sensory data from the perception stack is utilized
in path planning and navigation.

###  II-B Robotic Planning and Navigation

The input to the navigation stack is the sensory information and map obtained
by the various sensors from the perception stack [II-A] as depicted in Fig 1.
Navigation algorithms are responsible for the generation of a series of
waypoints, which are then passed on to the control stack [II-C]. Classic
robotic navigation and planning algorithms typically involve explicit rule-
based methods and predefined algorithms for path planning and obstacle
avoidance. These algorithms often rely on geometric models and predefined maps
of the environment. They can be categorized into four groups, including _graph
search algorithms_ (for example, A* [45]), _sampling-based algorithms_ such as
Rapidly-exploring Random Tree (RRT [46]), _interpolating curve algorithms_
[47], and _reaction-based algorithms_ such as Dynamic Window Approach (DWA
[48]).

AI/ML algorithms, on the other hand, offer a distinct advantage in object and
environment identification as they can autonomously learn patterns and
features directly from sensor data, eliminating the need for manual feature
engineering or predefined models. This capability has resulted in remarkable
enhancements in the performance of robotic navigation and planning, leading to
its adoption for generating local costmaps as well as for both global and
local planning tasks, thereby paving the way for more adaptive and efficient
robotic systems.

The process of navigating towards a long-term goal is referred to as global
planning. Local and global planners work together to achieve navigation goals.
Using Deep Reinforcement Learning (DRL) technique, Kastner et al. [49], for
example, combine two planners that operate at different scales: a global
planner that optimizes long-term navigation by producing high-level waypoints
and a local planner that navigates from one-waypoint to the next while
avoiding obstacles. On the other hand, local path planning and costmap
generation refers to the process of short-term path planning to avoid
obstacles and is used in mobile robots, where the navigation is solely based
on local landmarks and environmental factors. Costmaps are utilized in
objection tasks in small-scale robotic applications. Work that leverages CNN
and costmaps can be found in [50], while map-based Deep Reinforcement Learning
(DRL) for mobile robot navigation for the obstacle detection task based on a
generated local probabilistic costmap is presented in [51]. Deep learning
models that employ CNNs for local path planning by circumventing objects for
autonomous navigation can be found in the works of [52]. Similarly, Ritcher et
al. [53], proposed a combination of a fully connected feed-forward network and
an autoencoder to model collision probability, and more recently, Regler et
al. [54] presented a DRL-based local planner with a conventional navigation
stack, and Chen et al. [55] presented DRL for path planning, dynamic obstacle
avoidance, and goal approaching. However, an AI-Robotic systems should also
employ navigation algorithms and planning that are responsive to unexpected
changes such as grasping novel objects [56] and moving objects [55] in the
dynamic environment.

In the next section, we delve into the control stack of the architecture that
is responsible for manipulating robotic behaviors.

###  II-C Robotic Control

The input to the control stack is the waypoint information from the navigation
and planning stack II-B, as illustrated in Fig. 1. The control stack is
responsible for generating control signals by leveraging various algorithms to
manipulate the behavior of robots. It involves designing control systems that
enable robots to perform specific tasks such as moving actuators based on the
planned path given by the planning and navigation stack. The goal of this is
to develop algorithms that can effectively and efficiently command the robot’s
actuators based on sensor feedback.

Some of the classic control algorithms include Proportional-Integral-
Derivative (PID) control [57] and Linear-Quadratic-Regulator (LQR) [58]. PID
adjusts control signals based on proportional, integral, and derivative terms.
It provides stable and robust control by continuously comparing desired and
actual states and minimizing the error. LQR on the other hand, is used to
control non-linear dynamical systems by computing the optimal control signal
needed to move the system to a desired state. These algorithms, while
effective for many applications, often rely on explicit mathematical models
and predefined control strategies.

In recent years, there has been a significant shift towards leveraging AI to
enhance robotic control capabilities. In particular, Deep Reinforcement
Learning (DRL), has gained considerable attention in robotic control,
including locomotion, manipulation, and autonomous vehicle control [59]. DRL
algorithms allow robots to learn control policies through interactions with
the environment, receiving rewards or penalties based on their actions.
Kalashnikov et al. [60] demonstrated the use of DRL for controlling a robotic
arm to grasp and manipulate objects with a high success rates in their study.
In another line of work, the authors in [61] leveraged DRL techniques for
robot controllers to learn a range of dynamic manipulation behaviors, such as
stacking large Lego blocks at a range of locations, threading wooden rings
onto a tight-fitting peg, inserting a shoe tree into a shoe, screwing bottle
caps onto bottles, and assembling wheels in a toy airplane.

In the next section, we discuss our survey approach and taxonomy.

##  III Survey Approach & Taxonomy

This survey builds upon the intricate relationship between physical and
digital vulnerabilities within AI-Robotic systems. It adopts a comprehensive
approach by providing a taxonomy that explores the existing attack surfaces
along with defensive strategies that are relevant to AI-Robotics systems. The
primary objective is to understand how various attack vectors impact different
components of robotic architecture. To achieve this, we first introduce the
hybrid AI-Robotics architecture, comprising three layers: perception,
navigation and planning, and control (see Section II). The foundational
understanding of different architectural layers of robotic systems allows us
to identify the attack surfaces and their impact on these components.

We begin the survey by categorizing the AI-Robotic systems into three
dimensions, including attack surfaces, ethical and legal concerns, and human-
robot interaction security, as illustrated in our taxonomy (See Fig. 2). The
first category explores attack surfaces and their impact on the architectural
components of robotic systems. The second category, on the other hand,
discusses the ethical and legal concerns surrounding the use of robotics
systems in various domains such as healthcare, the military, autonomous
vehicles, manufacturing, and agriculture. Finally, we explore the security
landscape of human-robot interaction and recommend defensive strategies to
safeguard privacy concerns.

The attack surfaces as depicted in Fig.2, are divided into three distinct
domains: physical attack, digital attack, and other relevant attacks. Physical
attack encompasses attacks targeting the perception layer (attack in input
sensors, see Section IV-A) within the hybrid robotic architecture. Digital
attack, on the other hand, pertains to attacks directed at the navigation and
planning, as well as the control layer. The interconnectedness between
physical and digital attacks becomes evident, with physical attacks having a
direct impact on digital systems. For example, if input sensors such as
cameras, LiDAR, or RGBD sensors are tampered with, they could produce poisoned
data. When these compromised data (see Section IV-B3) are used by digital
systems like object recognition algorithms, it may lead to misclassifications,
thereby affecting navigation and planning (see Section IV-B1). Furthermore, it
can also cause mistimed actuation commands within the control systems (see
Section IV-B2). On the other hand, the manipulation of physical objects within
the environment can introduce a new set of challenges, potentially leading to
deception, malfunctioning, or erratic behaviors in robotic systems. A
compelling illustration of this is the work by Eykholt et al. [62], who
showcased physical attacks using adversarial stickers to deliberately deceive
an autonomous vehicle’s object recognition system. This illustrates how
vulnerabilities in physical components (both environment and sensors) can
cascade into the digital realm, causing major disruptions and interruptions in
robotic system operation. Furthermore, AI-Robotics systems can also suffer
from inherent challenges of AI/ML models itself such as model training attacks
(see Section IV-B3) and model inference attacks (see Section IV-B3).

Figure 2: An overview of our proposed taxonomy. The central parent node
represents the overarching AI Robotic System, from which three primary
branches emerge: Attack Surface, Ethical and Legal Concerns, and Human-Robot
Interaction. Attack surfaces is further into two families: physical attack and
digital attack. Physical attacks encompass attacks on the perception layer
(input sensors). Digital attacks include attacks on navigation and planning,
attacks on actuators, and attacks on AI models (training and inference
attacks).

The third category within the attack surfaces as illustrated in taxonomic Fig
2 is other relevant attacks. These include backdoors, malware, ransomware, as
well as network and communication-based attacks on robotics systems. We
discuss these attacks in greater detail in Section IV-C.

Following this, our focus extends to the ethical and legal concerns (see
Section V) surrounding use of these systems. We explore the ethical dilemmas
arising from robotics technology in various domains such as healthcare,
autonomous vehicles, assistive robots, military, and manufacturing as
illustrated in Fig 2. The focus is to showcase the importance of ethical
considerations and their influence in the development, deployment, and use of
AI-Robotics systems in the aforementioned domains. For instance, health care
robotics systems such as surgical robots must be developed that align with
medical ethics, ensuring patient safety and privacy. Another important issue
to consider is how to program AI agents in complex ethical scenarios. Examples
of this include the trolley problem and the self-sacrifice dilemma as depicted
in Fig 7 in relation to Autonomous Vehicles (AVs). Aside from these concerns,
the overt reliance on autonomous robotic agents for decision-making processes
can have adverse effects on human cognitive thinking and reasoning abilities,
ultimately leading to a reduction in human autonomy [26]. From legal point of
view, we explore two regulations that ensures privacy protection to personal
data. The first one is California Consumer Privacy Act (CCPA[27] in case of
the United States (US) and General Data Protection Regulations (GDPR)[28] in
case of Europe (EU). Both CCPA and GDPR provide similar privacy protections to
consumers.

Furthermore, we expand our survey to encompass the realm of Human-Robot
Interaction (HRI) security (see Section VI), delving into the intricacies of
physical, cognitive and social interactions between people and robots. We
survey current approaches that are aimed at safeguarding interactions between
humans and AI-Robotic systems.

The survey of attack surfaces and their defensive solutions based on the
taxonomy in Figure 2 is available in Section IV while the survey of ethical
and legal concerns surrounding AI-Robotic systems is in Section V, and human-
robot interaction in Section VI. Next, we begin our survey of attack patterns
and defensive security strategies discussion in the next section.

##  IV Attack Surfaces

In the context of robotic systems, an attack surface is a set of potentially
vulnerable points on the boundary of a system, a system element, or an
environment, through which an adversary attempts to compromise or manipulate
the robot’s functionalities [63]. Attack surfaces on robotic platforms can be
broadly divided into two categories such as physical attack and digital attack
[64] as illustrated in Fig 2.

###  IV-A Physical Attack

Physical attacks involve the act of physically manipulating hardware or
components with the aim of either disrupting its normal functioning or
obtaining unauthorized access. For example, tampering, destruction, removal,
theft, or manipulation of embedded sensors disrupt the perception of robotic
systems. Humanoid and social robots that interact with human counterparts are
susceptible to these attacks, as evidenced by the authors in [65]. On the
other hand, hardware-level trojans presented in [66] can manipulate physical
components such as integrated circuits and cause robotic systems to
malfunction abruptly.

Over the next few paragraphs, we will investigate an array of attack vectors
encompassing techniques like jamming and spoofing in relation to input
sensors. Our analysis will also extend to how these vectors influence the
perception layer (see Figure 1) within hybrid AI-Robotics systems
architecture.

Perception layer is the first component of AI-Robotics architecture (see
Section II-A). It serves as a vital pillar in AI-Robotics systems, utilizing
an array of sensors such as cameras, Light Detection And Ranging (LiDAR),
radar, Global Positioning System (GPS), and Inertial Measurement Unit (IMU)
among others, to comprehend and navigate intricate environments. These sensors
constitute physical devices, and any attempts to compromise their integrity
fall within the realm of physical attacks, as elaborated in Section III of our
taxonomy. They play a pivotal role in enabling AI-Robotics systems to create a
comprehensive world model, facilitating collision avoidance, precise path and
trajectory planning, and accurate location mapping. However, the perception
layer becomes a natural target for adversaries, who aim to tamper with or
compromise the quality of sensor data [67]. Manipulation of sensor data
affects the integrity and reliability of the perception process, thereby
degrading the performance of the navigation and planning layers, and
ultimately influencing the behaviors in the control layer. A summary of the
attack surface, affected sensors, attack summary, and proposed defensive
strategies can be found in Table II.

Figure 3: An illustration of GPS spoofing attack on UAVs, in which an adversary redirects the UAV to a false location by transmitting deceptive GPS signals. Attack Surface |  Medium |  Attack Summary |  Defensive Strategy |  Research
---|---|---|---|---
Jamming |  Camera, GPS, LiDAR, Ultrasonic sensors |  • Use of intense light to jam camera sensors. • Using bright light source (blinding light) to render LiDAR renders the sensor ineffective in perceiving objects. |  • Incorporating a near-infrared-cut filter and photochromic lenses to selectively filter certain types of light. |  [68, 69, 70, 71]
Spoofing |  GPS, LiDAR |  • GPS spoofing attack designed to manipulate navigation system. • Adversary may spoof LiDAR sensors using techniques such as laser projection[72, 69, 73], shape manipulation[74], and object manipulation [75]. |  • Filtering [76], Randomization [69], and Fusion [77]. • Anomaly detection sytems [78]. • Spoofing detection systems for radars-based spoofing attacks [79]. |  [80], [72, 69, 73, 74]
Manipulation |  IMUs |  • Manipulate the gyroscope, accelerometer, and magnetometer sensors. • Disrupt the accurate measurement of linear and angular velocity and cause errors in control system. |  • Inertial sensors safeguarding. • Sensor fusion techniques. |  [81]
Software vulnerabilities |  OpenCV, TensorFlow Image, Open3D |  • Adversary may exploit software vulnerabilities within programs responsible for processing sensor data. |  • Data verification and validation. • Data Sanitization. • Patching and updating for known vulnerabilities. |  [82, 83, 84]
TABLE II: Overview of attacks on input sensors along with defensive
strategies.

In the realm of adversarial attacks, it is possible for adversaries to engage
in various tactics such as jamming or spoofing input sensors, causing
disruptions, impairments, and deceptions that hinder the accurate perception
of the environment by these sensors. In [68], the authors discuss the use of
intense light to jam camera sensors, leading to a significant deterioration in
the quality of captured images and subsequently affecting object recognition.
Another study in [69] focuses on blinding attacks targeted at LiDAR sensors,
where a bright light source with the same wavelength as the LiDAR renders the
sensor ineffective in perceiving objects from the direction of the light
source. Similarly, in [70] researchers simulate jamming attacks on ultrasonic
sensors, while the vulnerability of GPS signals to jamming devices is
investigated in [71]. These attacks could also lead to other types of attacks,
such as a Denial-of-Service (DoS) attack wherein the perception sensors become
unavailable for authorized users as a result of malicious jamming.

Recent studies suggest adversaries can also spoof LiDAR sensors including
three techniques such as laser projection techniques [72, 69, 73], shape
manipulation strategies [74], and object manipulation approaches [75]. An
example of GPS spoofing attack is presented in [80] wherein a GPS spoofing
attack is meticulously designed to manipulate navigation systems. For
instance, in Figure 3, a GPS spoofing attack on a UAV is depicted. In this
scenario, the attacker manipulates the path trajectory and alters the true
position of the UAV by transmitting false or spoofed GPS signals.

In addition, some studies have brought to light the vulnerability of IMUs
employed in robotic systems to targeted attacks. These attacks aim to
manipulate the gyroscope, accelerometer, and magnetometer sensors [85]. Such
attacks aim to disrupt the accurate measurement of linear and angular
velocity, consequently causing the robotic systems to lose control and in some
cases deceive object detection tasks [81].

In addition to the aforementioned attacks, robotic systems are susceptible to
software vulnerabilities within programs responsible for processing sensor
data, such as OpenCV [82], TensorFlow Image [83], and Open3D [84]. Despite
their widespread use, these frameworks have certain limitations and
vulnerabilities, as they often rely on assumptions regarding the structure and
format of sensor data. As a result, if these assumptions are not met, these
frameworks may fail [86]. For example, OpenCV has been discovered to be
vulnerable to a heap-based buffer overflow (CVE-2018-5268) [87], which can be
triggered by parsing a malicious image during the loading process. Similarly,
vulnerabilities can arise from the lack of, or incomplete, or invalid
validation of input parameters (CVE-2021-37650) [88] in the TensorFlow
library, assuming that all records in the examined dataset are strings.

To mitigate blinding attacks on cameras, a defense strategy involves
incorporating a near-infrared-cut filter during daytime operations [89]. Also,
the utilizing photochromic lenses to selectively filter certain types of light
can enhance the overall image quality. In a separate study [71], the authors
propose a method for detecting GPS jamming incidents. Similarly,
countermeasures against spoofing attacks encompass techniques such as
filtering [76], randomization [69], and fusion [77] to enhance the robustness
and reliability of the sensors. Additionally, the advancement of machine
learning techniques has led to the development of spoofing detection methods,
as demonstrated in [78]. This research paper introduces an anomaly detection
approach that employs an auto-encoder architecture specifically designed for
LiDAR sensors to identify sensor spoofing attacks. Furthermore, a proposal in
[79] outlines a strategy to detect and counter radar-based spoofing attacks by
implementing a spatio-temporal challenge-response technique. This method
verifies analog domain physical signals, effectively preventing sensor
spoofing incidents. Besides, verification and sanity checking of input sensor
data should be conducted to ensure input data are free from adversarial
inputs.

Next, we delve into digital attacks, where we explore attack patterns on the
navigation and planning layers and control layers (actuators) of a hybrid AI-
Robotic architecture. Furthermore, we also discuss the inherent challenges of
ML models.

###  IV-B Digital Attack

Digital attack pertains to software-related attacks which aims to exploit the
software vulnerability within the robot’s operating system, algorithms,
firmware, communication channels, or other software component (for example AI-
based agents) to manipulate navigation & planning (see Section IV-B1) and
control layer (see Section IV-B2) of a robotic systems. The literature also
shows AI-specific attacks such as training attacks (see Section IV-B3) and
inference attacks (see Section IV-B3) that impact robotic systems.

In the subsequent subsections, we present a comprehensive analysis of attack
patterns and their associated security strategies within the diverse
components of AI-Robotics systems.

####  IV-B1 Attacks on Navigation and Planning with associated Security
Strategies

Attacks on the navigation and planning layer (see Section II-B) of robotic
system architecture present a complex and concerning challenge in the realm of
AI, automation, and cybersecurity. Contrary to the perception-oriented attacks
through robotic input sensors described in the previous Section IV-A, where
the objective is to tamper with perception sensors. Attacks on navigation and
planning relate to manipulating the AI-Robotics system’s navigation systems by
leveraging the poisoned data generated from compromised sensors. With the
rapid integration of autonomous robots into daily human lives, from self-
driving cars to industrial automation, from military applications to health
care, ensuring the security and integrity of their navigation and planning
systems becomes paramount. Table III provides an overview of various forms of
attacks on the navigation and planning layers of the hybrid AI-Robotics system
architecture, along with defense strategies.

Navigational attacks can manifest in various forms, ranging from simple and
localized disruptions to controlled, sophisticated, widespread manipulations.
One of the most common forms of attack involves sensor spoofing or jamming
[90, 91], where adversaries intentionally deceive the robot’s sensors, such as
GPS, camera, LiDAR, etc, to provide false or distorted information. Doing so
can mislead the robot’s perception of its environment, leading to potential
collisions, misrouting, or compromised decision-making. Furthermore,
navigational systems could be vulnerable to data manipulation attacks [92].
These attacks involve altering the mapping or localization data robots use,
causing incorrect interpretations of the environment. For example, in an urban
setting, this could result in a robot delivering packages to the wrong address
or navigating hazardous routes, endangering pedestrians and other vehicles.

Attack Surface |  Medium |  Attack Summary |  Defense Strategies |  Research
---|---|---|---|---
|  Communication Sensor (GPS, Radio, etc,) |  • Transmit spoofed positional data. • Block communication channel. |  • Need minimal control input for scope limitation. • Need secured recovery algorithm. |  [93, 94, 95, 96, 97, 98, 80, 99, 90]
|  Visual Sensor (Camera, LiDAR, etc,) |  • Falsify environment information for incorrect perception. • Manipulate operational environment. |  • Incorporate adversarial training. • Need to use advanced robust AI algorithm. |  [100, 101, 89, 69, 102, 103, 74, 75, 81, 90]
Navigation |  Environmental Force Sensor (IMU, etc,) |  • Alters operational environment. |  • Reduce single-sensor decision dependency. |  [104, 105, 85, 106, 90]
|  Black-box DRL models and planners (Image Segmentation, 3-D point cloud, etc,) |  • Exploit and optimize the cost function for controlled degradation. |  • Through careful tuning and environment-specific modifications are needed. • Development on non-iterative planning schemes. |  [107, 108, 15, 99, 109, 110, 111, 112]
Planning |  Coordination algorithms (Robust [113, 114] , Adaptive and reactive [115, 116]) |  • Break communication for coordinated systems and prevent recovery. • Exploit risk-reward trade-off algorithms. |  • Utilize GNN algorithms for decentralized information propagation. • Research on risk/uncertainty-aware search algorithms. |  [117], [118, 119, 120, 121, 122, 97, 98, 99, 123]
TABLE III: Research summary on adversarial attacks on robotic navigation and
planning systems with proposed defense strategies.

To prevent sensor spoofing by the adversary and enable secure and optimized
navigation, Bianchin et al. [93] developed an algorithm to detect spoofed
sensor readings and falsified control inputs intended to maximize the
traversing trajectory. The authors consider an undetectable attack over Global
Navigation Satellite System (GNSS) sensor and a Radio Signal Strength
Indicator (RSSI) sensor. The detection algorithm works on the principle of
nominal control input. In another work by Yang et al. [94], the authors
introduced timing-based adversarial strategies against a DRL-based navigation
system by jamming in physical noise patterns on the selected time frames. The
attack imposes adversarial timing based on three principle scenarios: pulsed
zero-out attack (noise through off-the-shelf hardware), Gaussian average on
sensor fusion (noisy sensor fusion system developed by Gaussian filter), and
adversarial noise patterns (fast gradient sign method to generate adversarial
pattern against trained Deep Q-network (DQN)). Through the attack, both value
and policy-based DRL algorithms are easily manipulated by a black-box
adversarial attacking agent. As a preventive measure, the authors suggested
adapting adversarial training to train DQN and Asynchronous Advantage Actor-
Critic (A3C) models by noisy states.

On the other hand, attacks on planning phase are equally disruptive. It
targets algorithms responsible for determining a robot’s actions and decisions
based on sensory information. Adversaries could introduce subtle changes to
the robot’s decision-making process, causing it to make sub-optimal choices or
even hazardous actions [124, 111]. For example, an autonomous drone might be
tricked into flying into a restricted area or performing unauthorized
surveillance. Such actions can be in the form of a targeted signal or sensor
distortion or in the form of data manipulation. In the work of Wang et al.
[125], the authors proposed jamming attacks using Reinforcement Learning (RL)
to influence the Unmanned Aerial Vehicle (UAV) performance and then developed
anti-jamming strategies using double DQN to mitigate the attack in UAV
trajectory planning.

Similarly, Bhandarkar et al. [109] proposed DRL techniques to launch Sybil
attacks and transmit spoofed beacon signals to disrupt the path planning
logic. Hickling et al. [110], on the other hand, proposed to utilize the
explainability of DL methods. In their approach, the planning agent is trained
with a Deep Deterministic Policy Gradient (DDPG) with Prioritized Experience
Replay (PER) DRL scheme, that utilizes Artificial Potential Field (APF) to
improve training time and obstacle avoidance performance. Along the same
lines, Peng et al. [111] proposed an Online Path Planning (OPP) framework to
continuously update the environmental information for the planner in
adversarial settings. For intelligently selecting the best path from the OPP
output, the Bayesian network and fuzzy logic are used to quantify the bias of
each optimization objective.

Additionally, SLAM is the problem of planning and controlling the motion of a
robot to build the most accurate and complete model of the surrounding
environment [14]. Active SLAM problems fall under Partially Observable Markov
Decision Processes (POMDPs), where both action and observation uncertainties
affect navigation and planning. In several cases, adversaries exploit such
uncertainty to disrupt the autonomous navigation and planning system. In the
work by Wang et al. [112], the authors introduced an infrared light-based
attack that can alter environment perception results and introduce SLAM errors
to an autonomous vehicle. For defensive measures, the authors developed a
software-based detection module. Similarly, Li et al. [123] demonstrated an
attack that exploits timing-sensitive multimodal data from different sensors
to cause destabilization of the SLAM perception model in cyber-physical
systems. Researchers have emphasized various recommendations to prevent and
mitigate beyond-line-of-sight and active spatial perception errors from
getting exploited [14] and require active research to mature and achieve real-
world impact.

In the next section, we explore attacks on the control layer of hybrid AI-
Robotics systems architecture with a focus on attacks on actuators and
recommend corresponding security strategies.

Figure 4: An overview of attack surfaces in control systems and their impact
on actuators within the operating environment. Icons from [29].

####  IV-B2 Actuator Attacks with associated Security Strategies

In the robotic systems architecture (see Section II-C), robots act on the
environment by leveraging its joint manipulators (in the case of robot arms or
humanoids), or wheels (in the case of ground mobile robots) or propellers (in
the case of aerial and marine robots). To act, these systems need actuators.
An Actuator is a mechanical device that utilizes various sources of energy,
such as hydraulics, external signals, pneumatics, and electricity, to convert
them into motion that can be manipulated according to specified control
parameters [126]. Electric motors, hydraulic actuators, and pneumatic
actuators are the most common types of actuators used in robotics. As
highlighted in the Section III and depicted in the taxonomy Figure 2, attacks
on actuators are classified as digital attacks.

Potential attack surfaces for robot actuators include actuator hardware and
firmware, communication protocols, and network components such as those used
by tele-operated robots. Fig 4 illustrates the attack scenario wherein the
adversary targets the firmware, manipulates the access control mechanism, or
employs backdoor techniques to influence the behavior of robotic systems
within their operating environment. Table IV summarizes various types of
attacks on actuators along with defensive strategies. To gain access to the
robot control systems, attackers can install hardware-level backdoors or kill
switches to bypass control protection mechanisms [127]. For example, Sasi
[128] developed a backdoor program that allows attackers to remotely control
UAV robotic systems. Another area that is vulnerable to attack is actuator
firmware, largely due to insecure communication (for example, using the public
internet to update a robot’s firmware). Several studies [129, 130, 131]
indicate that attackers can exploit communication vulnerabilities of public
networks to gain full control of robotic systems.

Authentication (for example Sybil attack [132]) and access control are often
overlooked in robotic systems. Researchers indicate that some robotic
application interfaces are designed without the need for a login portal,
allowing anyone to access them remotely [133], while others that are equipped
with access control systems are also known to suffer from vulnerabilities.
These include communication packets containing credentials in clear text or
weakly encrypted, systems with default or hard-coded credentials that can be
found in publicly accessible user manuals, or in some cases, a lack of access
rights configuration allowing adversaries with low privileges to control
robotic systems [134].

Recent scientific research has shown the potential vulnerability of the
control systems and actuators of teleoperated surgical robots to malicious
attacks. In their study, Bonaci et al. [135], demonstrated the ability to
exert unauthorized control over various surgical functions. Furthermore,
various research has revealed targeted attacks that can bypass the robot’s
input sensors, including the surgeon’s input, and directly impact the
actuators [136]. These attacks result in behaviors such as the involuntary
halting of the system or the abrupt movement of robotic arms. Notably, similar
attacks on actuators have been observed in other robotic applications, such as
mobile robotics [137, 138], robotic Cyber-Physical Systems (CPS) [139, 140],
and aircraft systems [141].

Security strategies for robot actuators involve several measures to ensure the
safety and integrity of the robot’s operations. One crucial aspect is the
sanitization of control signals that are transmitted to the actuators, which
helps protect against adversarial inputs. To achieve this, concepts like
dynamic filtering [142] can be applied to filter out anomalous control
signals. Moreover, an access control mechanism can also be implemented to
ensure only verified programs send control signals to the actuator driver.
Another approach is to develop anomaly detection systems to monitor anomalies
during robot manipulation tasks. For example, the authors in [143], presented
a method for multimodal anomaly detection during robot manipulation. In a
separate study, the authors in [144] developed RoboADS, a robot anomaly
detection system that is capable of detecting both sensor and actuator
misbehaviors. The alerts generated by these detection models can then be
analyzed by robot operators to conduct further investigations if necessary.
Furthermore, rule-based actions can be implemented in the actuator code to
prohibit the robot from executing undesired actions.

Moving forward, we examine the inherent challenges within AI models and their
potential impacts on AI-Robotics systems.

Attack Surface |  Medium |  Attack Summary |  Defensive Strategy |  Research
---|---|---|---|---
Hardware |  Backdoors and kill switch |  • Installation of hardware-level backdoors or kill switches to bypass control protection mechanisms. • Backdoor programs to remotely control robotic systems, e.g., UAVs. |  • Secure design and development. • Utilizing formal verification methods to ensure the robotic system’s software and hardware meet specified security requirements. |  [127, 128]
Firmware |  Insecure network communication |  • Attackers can exploit communication vulnerabilities of public networks to install malware and obtain full control of robotic system. |  • Employing secure communication protocols and encryption mechanisms between robotic components. • Update firmware and patch software using secured connection. |  [129, 130, 131]
Access Controls |  Access controls mechanism and authentication systems |  • Lack of proper access control mechanism allowing anyone to access and control remotely. |  • Inclusion of robust access control to ensure only verified programs send control signals to the actuator driver. |  [133, 134]
Others |  Manipulation of surgical functions (surgical robots) |  • Targeted attacks can bypass the robot’s input sensors, including the surgeon’s input, and directly impact functioning of actuators. |  • Sanitization of control signals. • Dynamic filtering [142] can be applied to filter out anomalous control signals. |  [135, 136]
TABLE IV: Overview of attack patterns in actuators and associated defensive
strategies.

####  IV-B3 Attack on AI Models

Modern robots contain multiple software components such as object recognition
[145], voice commands [146], and facial recognition [147], that are usually
built using AI-based pipelines. At a high level, AI pipeline entails data
collection, preprocessing, model selection, training, validation, and
deployment. These AI pipelines can be used to solve previously intractable
problems in robotics using adaptable and data-driven methodologies. However,
this same adaptability can also lead to exploitation by attackers [148]
because of the inherent complexity and the many stages involved in training,
validating, and deploying an AI system. AI components that involve deep
learning (i.e. neural networks) can be especially vulnerable due to their
“black box” nature, which means that even when the security is compromised, it
may be difficult to pinpoint exactly where the problem occurred and how to
address it [149]. The nature of this attack is classified as a digital attack,
as delineated in our taxonomy in Section III and depicted in Fig 2.

In the following subsections, we explore training and inference attacks on AI
models.

##### Training Attacks and associated Security Strategies

Figure 5: An example of training attack on AI pipeline that causes
misclassification of objects and misjudgment of object distance.

Training attacks involve the deliberate alteration, tainting, and modification
of the training dataset. Such an attack intends to manipulate the training
process and compromise the performance and behavior of AI models. There are
several forms of security attacks on the training process or training data on
AI agents of AI-Robotic systems. One form of attack is known as model
poisoning, where the training data for AI models is tainted through label
corruption. Such attacks might lead AI model to function unexpectedly or is
unable to recognize the current input data. Figure 5 presents a high-level
overview of a training attack. The adversary tampers sensory data obtained
from perception sensors such as cameras, IMUs, LiDAR, etc., to create a
poisoned dataset, which is then utilized to train an AI model. As a result,
the trained AI model is compromised and may exhibit misclassifications, such
as inaccurately identifying objects or misjudging distances between objects.
There are several well-known model poisoning cases such as the Microsoft Tay
chatbot incident [150] and the VirusTotal poisoning incident [151]. The Tay
chatbot incident involved an experimental conversational chatbot released by
Microsoft on Twitter, which is a popular social media platform. Tay is
designed to work by learning from real-world conversations and improving its
language capabilities over time. Unfortunately, malicious internet users
exploited this property by coordinating to send messages to it involving
racist, misogynistic, and anti-semitic language. As a result, Tay started to
repeat these abusive and offensive messages and the experiment had to be
suspended. Another example is the VirusTotal poisoning incident, where mutant
samples uploaded to the training platform were able to fool an ML model into
incorrectly classifying inputs. VirusTotal is a popular platform for analyzing
suspected malware files and automatically sharing them with the security
community. However, attackers were able to compromise the integrity of this
platform by uploading fake samples such that the platform would classify even
benign files as malicious. In essence, these model poisoning cases show that
an AI model can be compromised if the source of its training data is not
validated. Robotic systems that utilize publicly-sourced training data can be
vulnerable to these model poisoning attacks. In addition, outsourced training
can potentially introduce the risk of malicious backdoors [152].

One strategy to prevent model poisoning attacks is either preventing a model
from being poisoned or repairing a network after it is poisoned [153]. Another
strategy involves identifying poisoned data as outliers in the input space
[154, 155], employing certain input preprocessing techniques [156], or by
trimming out irregular samples when computing the loss function [157]. In the
event of poisoned model, it may be possible to detect the planted artifacts
using generative adversarial networks [158] and repair them using pruning and
fine-tuning [152]. Pruning works by removing neurons that are non-essential or
dormant for clean inputs, thus reducing the risk that the network will produce
unwanted behavior for bad inputs. On the other hand, fine-tuning works by
taking a network trained with untrusted sources and then further training the
network with trusted inputs. A combination of the two, fine-pruning [152], can
even defeat more sophisticated pruning-aware attacks.

Building upon the exploration of training attacks and associated security
strategies, in the next section, we delve into inference attacks and their
corresponding security strategies.

##### Inference Attacks and associated Security Strategies

A model inference attack is a type of privacy attack wherein an adversary
attempts to reverse-engineer or infer sensitive information about the training
data or individual data points used to train an AI model by querying the model
and analyzing its outputs [159]. Construction of model inference attack
usually involves generating shadow models (a replica of the target model that
mimics its behavior as closely as possible) using shadow training [159]. An
example of a model inference attack is depicted in Figure 6. Inference attacks
on AI-Robotic systems may include model inversion, model evasion, and model
alteration among others.

  * •

Model Inversion attacks also known as attribute inference attack [160] occur
when an adversary strategically queries the inference API to extract
potentially private information embedded within the training data [161].
Research has shown that certain data augmentation techniques can mitigate
overfitting and indirectly reduce the risk of membership inference attacks
[159] but such methods may incur an accuracy penalty [161].

  * •

Model Evasion tactics are used by attackers to prevent the AI model from
computing the correct output. In the image domain, adversarial techniques can
take the form of simple transformation of the input (cropping, shearing,
translation), adding white noise in the background, or constructing
adversarial examples by carefully perturbing the input to achieve desired
output [162]. In the natural language domain, adversarial techniques can take
the form of a Happy String, wherein the benign input is tacked onto malicious
query points, or by using bad characters, words, or sentences that can still
appear semantically-sound from a human perspective [163]. Certain model
evasion strategies can be especially stealthy since they employ optimization
techniques to find the smallest perturbations that can still cause a change in
the model output [164].

  * •

Model Alteration occurs when attackers are able to gain access to the network
weights and alter them without the knowledge of the user. These attacks are
essentially deployment-stage backdoor attacks where the attacker can cause the
network to produce incorrect output for a specific input while producing
correct outputs for most other inputs [165, 166]. Such attacks can also cause
neural networks to become black-box trojans. Since the behavior of a neural
network can significantly change while appearing benign from the outside if
only minimal model parameters are changed [167, 165]. These attacks can take
the form of network structure modification [167], flipping weight bits[165],
or subnet replacement [166].

Figure 6: An overview of a membership inference attack where the attacker
queries the target model utilizing the shadow training set and shadow model to
obtain a prediction vector containing probabilities for each class [159].

Defense strategies to preserve the privacy of users’ data (in case of
membership inference attacks), especially images for computer vision problems,
is to apply a reconstruction and perturbation network. This method preserves
image features while obfuscating the individual’s face [168, 169].
Differential Privacy (DP) is another technique that can mitigate privacy issue
by computing a theoretical bound on the effect that any particular input has
on the output. Employing such strategy may provide guarantees in certain
situations that information will not be leaked [170]. Another technique to
prevent data leakage is learning with selective forgetting [171], where a
class-specific synthetic signal is used to forget arbitrary classes without
using the original data. In a similar vein, defense against model evasion
techniques that utilize adversarial inputs is to formally check the security
properties of a deep neural network with Satisfiability Modulo Theory or
symbolic interval analysis [172]. In the case of evasion attacks, two forms of
defense can be implemented. The first type focuses on improving an AI model’s
robustness by introducing methods that make the model’s weights and layers
more difficult to steal or reverse-engineer [173]. This can be accomplished
via online learning and weight readjustment or by introducing bounded
perturbations [173]. The second form employs a hybrid approach. This involves
integrating AI algorithms to deal with unknown quantities with tried and
tested methods from legacy systems, which are designed to tackle known
robotics problems [174]. This approach provides an extra degree of
verification to ensure comprehensive protection.

###  IV-C Other Relevant Attacks

As highlighted in Section III, AI-Robotics systems can also suffer from other
common cybersecurity attacks. Based on our taxonomic Figure 2, some of these
attacks include backdoors, malware, ransomware, and attacks on communication
networks. In the following paragraphs, we explain these attacks in detail.

A backdoor attack is a stealthy technique of circumventing normal
authentication methods to gain unauthorized access to a system. Malicious
actors can insert a backdoor in robotic systems to remotely control them. In
some cases, threat actors such as malicious manufacturers may leave, backdoor,
on purpose, in robotic systems to track and monitor the activities of the
robot and its operator without the owner’s knowledge [10].

Malware, on the other hand, can target robotic systems such as surgical
robots. Chung et al.[175], for example, exploited ROS vulnerabilities and
developed smart self-learning malware capable of tracking the movements of the
robot’s arms and triggering the attack payload during a surgical procedure.
Similarly, ransomware attacks in the context of industrial robots attempt to
lock them to extort ransom from the manufacturer. In an effort to augment
robot control system security, the authors in [176] presented a customized
industrial robot ransomware, which showcased the possibility of industrial
robots being susceptible to ransomware attacks.

Another potential candidate of attack is the robotic communication network.
Adversaries may target vulnerability of existing communication protocols which
might lead to jamming, de-authentication, traffic analysis, eavesdropping,
false data injection, denial of service, replay, or man-in-the-middle attacks.
Jamming and de-authentication attacks are quite common in small UAVs [177].
Traffic analysis and eavesdropping are passive attacks used on the unencrypted
channels of UAVs to extract the meaningful and sensitive flight and motion
data [13] whereas false data injection and replay attacks involve insertion of
incorrect routing information into the caches of UAVs [13, 12].

Turning to another crucial aspect, we will explore the ethical and legal
concerns that encompass the utilization of AI-Robotics systems in various
domains.

##  V Ethical & Legal Concerns

Apart from the attack surfaces discussed in preceding section, there are also
ethical and legal concerns surrounding robots. In this section, we will
explore these aspects in greater detail.

###  V-A Ethical Concerns

One of the popular topics that have an ethical dimension in robotics domain is
robot ethics or roboethics \- a term first coined by Gianmarco Verrugio [178].
It is a combination of ethics (what’s right and wrong) and robotics (robots
and technology). It focuses on comprehending and overseeing the ethical
outcomes and aftermaths arising from robotics technology, especially with
regard to intelligent and autonomous robots, within our societal context
[179]. Roboethics encompasses a diverse range of application areas where
ethical considerations play a pivotal role in shaping the development,
deployment, and use of robots and autonomous systems.

In following paragraphs, we explain ethical concerns surrounding use of AI-
Robotics systems across various domains including healthcare, autonomous
vehicles, assistive robots, military, and manufacturing as delineated in
Section III and depicted in Fig. 2.

In the realm of healthcare, medical robotic technologies are increasingly
utilized for surgeries [4], rehabilitation [5], and caregiving. Ethical
questions arise concerning patient privacy, cost of implementation, and
informed consent for robotic procedures among others. Consider, for instance,
the substantial expense associated with robotic surgery. This naturally
prompts us to ask[180]: “Given that there is marginal benefit from using
robots, is it ethical to impose financial burden on patients or the medical
system?” Other ethical quandaries include patients’ autonomy and raise
questions such as how to ensure patients fully understand and consent to
robotic procedures? Do they have the right to opt for or against robotic
assistance in surgery? Similarly, there are questions about data privacy and
security. Questions like, What safeguards are in place to protect patient data
collected and transmitted by medical robots? How do we ensure privacy and
prevent data breaches?

Medical roboethics guides the development of these systems to ensure patient
safety, privacy, and equitable access. Furthermore, the design and development
of medical robots must comply with privacy rules, including the Health
Insurance Portability and Accountability Act (HIPAA) [181] and the European
Union’s General Data Protection Regulation (EU GDPR) [182]. These regulations
guide engineers and developers in ensuring the privacy and security of user
data (see Subsection VI-A1 for privacy concerns around human robot
interaction). By adhering to these policies, developers can mitigate the risk
of data breaches, unauthorized access, and misuse of sensitive medical
information.

Figure 7: A graphical depiction of the ethical dilemma for the AVs in an
inevitable crash. Option (A): crash with the cyclist on the right; Option (B):
continue straight and crash with two pedestrians; Option (C): self-sacrifice
by crashing with the truck on the left. Adapted from [183].

Another critical area of robotic application is Autonomous Vehicles (AV).
There are two fundamental ethical concerns surrounding AVs, including the
trolley problem and the self-sacrifice dilemma [183]. A visual illustration of
these dilemma is presented in Fig 7. The first one refers to a self-driving
car deciding whether to continue on its present path, potentially hitting and
harming multiple pedestrians, or to swerve and divert its path, potentially
killing a single individual [184, 185]. Then, in such a scenario, what is the
correct decision? It raises questions about how these decisions should be
programmed by AI agents, who bears the ethical responsibility, or how societal
values and cultural norms should influence the behavior of these vehicles?.
The self-sacrifice dilemma, on the other hand, is a variant of the trolley
problem that addresses the ethical question of whether an autonomous vehicle
should prioritize the safety of its occupants (passengers) over the safety of
pedestrians or other individuals outside the vehicle. In this scenario, the
self-driving car is confronted with an unavoidable collision and must
determine whether to take actions that would minimize harm to its passengers
at the expense of greater harm to pedestrians or vice versa [186]. Then again,
what is the correct decision? Should the AI agents be programmed in a way that
saves more people by sacrificing the driver? How can these decisions be made
transparent? This issue underscores the need to consider integrating societal
values, legal frameworks, and cultural norms in designing AI agents that can
intelligently adapt their behavior in such ethically complex situations.

In a similar vein, assistive robots (for example rehabilitation robots,
robotized wheelchairs, walkers, etc.) designed to assist people with
disabilities, or the elderly are becoming increasingly prevalent. These robots
can improve autonomy and quality of life, but ethical quandaries arise
surrounding questions of dependency, risk, cost, benefits, and the potential
for social isolation induced by growing reliance on machines for companionship
and care. For example, to what extent should individuals using assistive
robots maintain control over the actions and decisions of the robot?, who is
liable in case of accidents or errors made by assistive robots?, and how can
we maintain a balance between the benefits of companionship and the potential
risks of emotional dependence [25] on machines?.

The use of sophisticated and automated robotic weapons by the military in
conflict has profound ethical concerns. Particularly, lethal autonomous
weapons raise concerns about moral accountability for machine actions, the
potential dehumanization of combat. For instance, how can we ensure meaningful
human involvement in critical decisions, especially in life-and-death
situations?, how can we program AI agents that distinguishes combatants and
non-combatants? how can we ensure that targeting decisions made by AI agents
aligns with humanitarian principles and laws? This is a critical issue because
during the fog of war it is difficult enough for humans to effectively
distinguish whether a target is legitimate. So, how can we ensure AI-Robotics
systems perform better ethical decision making than human soldiers?[187].

Additionally, in the industries like manufacturing and agriculture, ethical
considerations surrounding human-robot collaboration, job displacement, and
responsible automation are central to the roboethics discourse.

###  V-B Legal Concerns

Laws in robotics are influenced by fictional writers such as Isaac Asimov. His
three laws of robotics [188], which first appeared in a story called
“Runaround”, include the following:

  1. 1.

A robot may not harm a human being, or, through inaction, allow a human being
to come to harm.

  2. 2.

A robot must obey the orders given to it by human beings, except where such
orders would conflict with the First Law.

  3. 3.

A robot must protect its own existence, as long as such protection does not
conflict with the First or Second Law.

While the laws straightforwardly establish robots as subordinate to humans,
their implementation can be challenging in diverse real-world scenarios.
Nevertheless, robot engineers can view them as guidelines for programming AI
agents within robotic systems. Legal and regulatory concerns stem from the
application of legal and moral theories pertaining to liability and
responsibility [189]. Consider the utilization of surgical robots in hostile
military environments for procedures like “remote robotic-assisted
telesurgery” or “autonomous robotic surgery”. In the event of an unintended
event mid-surgery, where harm is inflicted upon human counterparts due to
malfunctioning or attack on robot actuators, this may result in legal
consequences. This prompts a discussion regarding which party should be held
liable?, such as the system manufacturer, software developer, the autonomous
agents involved (this is difficult because as they are not legal persons), or
the human operators overseeing the operation of these devices [190].

Another legal dimension within the AI-Robotics systems in security of personal
data. To safeguard the personal information various policy framework has been
proposed both in the United States (US) known as 2020 California Consumer
Privacy Act (CCPA) [27]) and 2016 General Data Protection Regulation (GDPR
[28]) in the Europe (EU).

Before CCPA, in the US, the Executive Office of the President, National
Science and Technology Council, Committee on Technology issued a report,
“Preparing for the Future of Artificial Intelligence” (“AI Report”), in
October 2016 [191]. Given the importance of AI in the future, the report
recommended “the Department of Transportation should work with industry and
researchers on ways to increase sharing of data for safety, research, and
other purposes.” Prior to this, the same administration also presented a
discussion draft on “Consumer Privacy Bill of Rights Act”. It proposed seven
principles including transparency; individual control; respect for context;
focused collection and responsible use; security; access and accuracy; and
accountability, however, the act was not enacted [192]. To alleviate the
concerns of use of personal data by large technology companies CCPA was
brought to legislation in 2018. The regulation defines personal data and
information, enforce requirements for company data security, allow data
portability and deletion rights, and impose fines on organizations that
violate them. This law went to effect in 2020, after undergoing several
amendments. The CCPA is based on a consumer protection framework and applies
more narrowly to enterprises that meet specified revenue or number of
California residents’ personal information collection requirements [193].

Similarly, in the EU, GDPR law was signed in April 2016 and implemented in May
2016. The new GDPR mandates explicit informed consent for data use. Through
informed consent, the users are given the right to opt in or out from any
potential processing of their data through the “right of explanation.” GDPR
also empowers users to track or remove their data through the “right to be
forgotten.” To counter the problem of information leakage, the authors in
[194] presented two efficient approaches, such as unlearning and amnesiac
unlearning. Their approach is sucessful in removing personal data. In similar
vein, Villaronga et al. [195] examined the issue of AI memory in the context
of the right to be forgotten and explored the inadequacies of current privacy
laws for tackling the complexities and challenges of AI technology. The paper
analyzed legal implications for AI as well as the conflict between privacy and
transparency under EU privacy law. In a separate study, the authors in [196]
proposed an approach that cleanses the weights in a deep neural network to
forget information selectively.

In the subsequent section, we delve into challenges and risks that arise in
the context of human-robot interaction.

##  VI Human-Robot Interaction (HRI) Security Studies

Human-Robot Interaction (HRI) is an emerging field that examines the physical,
cognitive, and social interactions between people and robots. While HRI is an
important component of robotic autonomy, studies have also identified several
key vulnerabilities that can compromise the integrity, privacy, and safety of
these interactions. For example, unauthorized access to a collaborative robot
(co-bot) in a manufacturing setting could involve potential manipulation of
the robot’s behavior. Similarly, privacy is another major concern, as the
collection and storage of personal and sensitive information during
interactions with social robots used in healthcare settings can result in
privacy breaches or unauthorized use of data.

In accordance with the taxonomy introduced in Section III, we conduct a survey
of the challenges and concerns in the subsequent subsections. We also
recommend countermeasures such as Privacy Enhancing Technologies (PET) that
bolsters the integrity, privacy and confidentiality of data exchanged between
humans and robots.

###  VI-A Challenges and Concerns

####  VI-A1 Privacy Concerns Around Robots

User privacy needs to be considered and protected when designing robotic
assistive systems. Privacy concerns might influence human behavior when robots
monitor them. In their work [197], the authors analyzed how older adults
perceive privacy and their attitudes to engage in privacy-enhancing behavior
under three natural settings – a camera, a stationary robot, and a mobile
robot. Authors found that more participants engaged in more privacy-enhancing
behavior such as self-censoring of speech or covering up the camera when being
watched by a camera compared to embodied robots. Their analysis implies that
participants are less cautious about their privacy around robots due to their
unfamiliarity with them.

Robots endanger people’s privacy by directly monitoring them and increasing
access to their information. A study in [198], examined 1090 German-speaking
older adults verifying the factors that influence their privacy concerns and
preferences. The paper observed a significant correlation between the people
with practical robotic experience and the interest. The participants
interested in human robots showed higher comfort levels around robots than the
users who selected machine robots. Most participants did not express interest
in sharing their health-related information with anonymous health services.
Results indicated that participants prefer not to transmit their location and
position to others unless there is an emergency. Furthermore, their level of
comfort varied depends on the data collection modalities.

####  VI-A2 Social Acceptance To Robot Threats

People exposed to robots that are capable of rejecting human commands could
result in a negative attitude towards robots. In the context of their
experimental investigation [199], the authors investigated social acceptance
of perceived robot autonomy in case of threats to human identity and threats
to jobs. Using the Mechanical Turk crowdsourcing platform, they surveyed 176
US participants showing videos of autonomous robots that could disregard human
commands and non-autonomous robots that could only obey human commands.
Participants were assessed on their perception of robotic threats to jobs,
resources, safety, and human identity. They also evaluated participants’
negative attitudes towards robots and support for the robotic research. The
survey concluded that the participants viewed autonomous robots as more
realistic threats than those that were not autonomous. Generally, humans are
more hesitant to trust especially those they perceive to be deceptive [200,
201]. However, within their study [202] found that robotic failures in their
experiments did not negatively affect user attitude when using a human-like
robot embodiment. Furthermore, as outlined in their research endeavor [203],
researchers confirm that error-prone robots are more believable.

####  VI-A3 Internet-Connected Devices

Storing and sharing personal information of parents and children is a
potential threat in internet-connected robotic toys. Within the scope of
empirically evaluating the security and privacy perspective about parent
attitudes and child privacy, the authors in [204], made recommendations for
toy designers related to the security and privacy of the toys. They focused on
identifying parents’ privacy expectations and concerns, children’s mental
models of their privacy, and the ethical perspective of parental controls.
They conducted semi-structured interviews with thirty parents and children,
following a demonstration and interaction session with two different toys.
Authors suggested that robotic toy designers must be transparent and
communicate with parents that the toy is recording and delete recordings after
a fixed time from the app and the cloud servers. Such studies aid designers in
devising internet-connected games and toys without violating privacy rights.

On the other hand, smart homes, more specifically, Robot-integrated Smart
Homes (RiSH) [205], are on the rise. These smart homes integrate a home
service robot, a home sensor network, a body sensor network, a mobile device,
cloud servers, and remote caregivers. RiSH can be used for research in
assistive technologies for elderly care. The interconnectivity of these
systems through the Internet of Things (IoT), however, exposes them to various
cybersecurity attacks and poses a number of security concerns. In their
research [206], the authors collected responses from semi-structured
interviews with fifteen smart home residents to identify the gap between risks
identified by security researchers and the user’s security concerns and their
mitigation methods. The findings showcased smart home users’ lack of security
concerns, ranging from not feeling personally targeted or trusting potentially
adversarial actors to believing existing mitigation strategies to be
sufficient.

####  VI-A4 Trust

Psychologists define trust as a mental state in humans [207]. In relation to
HRI, there are various definitions of trust; for example, Lee et al. [208],
define it as “the attitude that an agent will help achieve an individual’s
goals in a situation characterized by uncertainty and vulnerability” whereas
Hancock et al. [209], defines it as “the reliance by an agent that actions
prejudicial to their well-being will not be undertaken by influential others”.
The key takeaway from these concepts is whether a robot’s activities and
behaviors align with human interests [210].

In their work [209], the authors identified three categories of factors that
affect trust in HRI: robot-related factors, task and environmental factors,
and human-related factors. The first category pertains to the quality of
operation performed by the robot from the perspective of human operator and
includes subtasks [210] such as transparency and feedback in decision-making
[211], situational awareness [212], autonomy [213], fault tolerance [214], and
engagement [215] to facilitate naturalistic interaction. In contrast, task and
environment-related factors, as identified in various other studies, such as
the nature of the task [216], risk and human safety [217], and the task site
[213] , influence trust between robots and humans. The third category, on the
other hand, is concerned with human-based factors, such as the general
perception about robots [218], expectations of human operators towards
automation [219], past experience with robots [220], and operators’
understanding of robotic systems [221].

###  VI-B Privacy Enhancing Technologies

In the context of AI-Robotics, Privacy Enhancing Technologies (PET) refer to
the use of various technological solutions and strategies to protect
individuals’ privacy when interacting with robotic systems. Their purpose is
to protect user identities by offering anonymity, pseudonymity,
unobservability, and unlinkability of both users and data subjects [222, 223].
A report by the Federal Reserve Bank of San Francisco [224] categorizes PETs
in three categories: altering data, shielding data, and systems +
architecture. A graphical depiction of these categories and subcategories is
presented in Fig. 8. The first category of PETs are tools that alter data
itself. Examples of these include anonymization, pseudonymization,
differential privacy, and synthetic data. The second category focuses on
hiding and shielding the data. This usually involves the use of cryptographic
techniques. Some examples include encryption, homomorphic encryption, and
privacy-enhanced hardware. The final category of PETs are new systems and
processes for data activities. The report suggested that some of these systems
also enable greater transparency and oversight across data activities,
including collection, processing, transfer, use, and storage.

Figure 8: A graphical illustration of Privacy Enhancing Technologies (PET).
Adapted from [224].

Authors in [225] surveyed PETs for solving personalization paradox. Their
approach included much broader PETs as compared to the report by Federal
Reserve Bank of San Francisco. Some of these technologies include anti-tracing
(fingerprinting), data obfuscation (perturbation, private information
retrieval), privacy preservation certification (attribute-based signatures,
sanitizable signatures), self-destructing data systems (vanish, neuralyzer),
statistical disclosure control (anonymizing DBs), and secure communication
(client-service encryption, end-to-end encryption).

PET such as access control mechanisms are popular in achieving privacy and
data protection in natural settings. Furthermore, the right choice of sensors
that adapt to differing privacy expectations is essential in reducing privacy
vulnerabilities [226]. To address this, the authors in [227] proposed an app-
based user interface to enable users to be in control of their data. The
interface is intended to enable transparency and accuracy in data processing
by communicating with the software running on the robot. This transparent
interface displayed the robot’s visual, auditory, position, and communication
information to help users identify the possible privacy violations caused by
social robots. However, the human feedback evaluation revealed that
participants had somewhat mixed attitudes towards this self-learning social
robot. Moreover, the work in [228] recommends balancing privacy with utility
in robot systems. The qualitative and quantitative evaluations of privacy
preferences of end-users in their study have shown concerns about both privacy
and physical harm.

In relation to trust in HRI, to enhance trust in robotic applications, trust
models must be developed for a specific form of human-robot interaction or a
specific type of robotic agent [210]. For example, a trust model for
evacuation robots must be different from a trust model for surgical robots
because the expected outcomes of these domain-specific robotic systems are
distinct.

In the preceding sections, we presented a survey on attack patterns targeting
various components of AI-Robotics systems, discussed ethical and legal
concerns surrounding the use of these systems, and delved into the security
landscape of human-robot interaction. Next, we detail some future research
directions that aim to improve the overall security posture of AI-Robotics.

##  VII Future Research & Discussion

Previous sections discuss the security risks and challenges faced by AI-
Robotics systems. More, specifically, we explored three dimensions of AI-
Robotics systems including, attack surfaces, ethical and legal concerns, and
security of HRI. As these AI-Robotics systems become more pervasive, their
security becomes critical. We believe that the holistic understanding of
abovementioned dimensions will effectively strengthen the overall security
posture of these systems. Security of input sensors and sensory data from
these sensors (e.g., training data, models) becomes essential to ensuring the
integrity of these systems. The data complexity only increases as you consider
that an AI-Robotics system could be comprised of many robots (i.e., a system
of systems). We see future research focusing on ensuring trustworthy AI-
Robotics systems that are capable of withstand various attack scenarios,
adhere to the ethical and legal principles of the domain in which they are
deployed, and ensure the interactions with these systems remain private.

In the following subsections, we outline seven essential domains that
encompass various research areas critical for enhancing the security of AI-
Robotics systems. As depicted in Fig 9 these include research on attack
surfaces mitigation, securing swarm robotics and knowledge transfer, security
of human-robot ecosystems, explainability in ai-robotics, securing
verification, validation, and evaluation of ai-robotics systems, and education
initiatives to raise robotics security awareness.

Figure 9: A visual representation outlining the future research direction
aimed at improving the security of AI-Robotics systems.

###  VII-A Research on Attack Surfaces Mitigation

In Section IV, we explored various attack surfaces on AI-Robotics systems. The
first component that an adversary may target is the embedded sensors that are
responsible to percept the environment. Adversaries can either physically
damage sensors or they can manipulate sensors by jamming or spoofing the
sensor signals. The future robotic systems should be designed with
adversaries’ capability in mind. For example, to secure the sensors from a
physical attack such as vandalism defenders can apply perimeter security with
robust access controls. Similarly, in case of other attack patterns such as
jamming and spoofing the defenders may integrate lightweight spoofing and
jamming detection algorithms [78, 79]. Another important aspect that future
AI-Robotics systems designers and engineers must consider is to have some
mechanism for verification and sanity checking of input sensors data. Such
process would thwart adversaries attempt to supply adversarial input to AI
agents.

To secure the actuators from attacks, the research is leaning towards
development of anomaly detection systems. These systems are capable of
monitoring the anomalies during robot manipulation task [143]. Such methods
can also be extended to filter out anomalies in sensor data [144].
Furthermore, control signal sanitization mechanism must be integrated in
robotics systems of the future to defend against adversarial input to
actuators.

Future security strategies to secure the AI agents used by AI-Robtics systems
during training phase may include certain preprocessing techniques [156] such
as identifying poisoned data as an outlier in the input space [154, 155].
Furthermore, the robotic systems engineers leverage generative models such as
Generative Adversarial Networks to detect the planted artifacts in poisoned
model [158]. In case of protecting privacy during inference attack various
security strategies can be implemented for future robotic application.
Concepts like differential privacy, selective forgetting [171], data
obfuscation [168, 169] can be considered.

###  VII-B Securing Swarm Robotics and Knowledge Transfer

As noted before, robots are becoming more integrated in various aspects of
human life, and securing human-robot interactions is going to be vital moving
forward. Similarly, securing the interactions between robots in multi-robot
systems is another major challenge. Advances in swarm robotics [229] have led
to large-scale swarm applications in which teams of robots work together to
achieve a common goal. Sharing information across agents has led to new
architectures [230] including making use of shared-memory structures called
blackboards. Other approaches look to use decentralized security solutions
like secured federated learning [231] based on cybersecurity knowledge graphs
[232] to protect the rest of the swarm if a specific robot is targeted by an
adversarial attack.

###  VII-C Security of Human-Robot Ecosystems

In human-robot ecosystems where all agents work independently, a major
challenge is preventing conflicts between robotic agents and humans utilizing
shared resources [233]. How are plans prioritized? How is information
exchanged? Can negotiations occur?

In scenarios where AI-Robotics systems assist humans (i.e., HRI applications),
trusting inferences made by these robotic systems is another challenge [234].
Sensors in robots could become faulty or biased over time and produce
inaccurate measurements. How does a robot re-calibrate a faulty sensor in an
autonomous setting? How do you fuse information in a multi-robot system that
may or may not have a faulty sensor (or a faulty robot)? These are important
questions that must be addressed in order to trust these systems in scenarios
with high variability and where human life is at stake.

It is important to note that in human-robot ecosystems, agents can be both
physical and digital. In particular, agents can be a digital representation of
a physical system (i.e., digital twin[235]). Digital twins are being used to
improve human-robot collaboration in manufacturing [236]. Improving the
robotic system through human experience and knowledge is also an important
area of research. An emerging concept is the use of a human digital twin (HDT)
in an elaborate system-of-systems where human decision making is captured by
data-driven approaches [237], [238].

###  VII-D Explainability in AI-Robotics

Explainable AI (XAI) provides a mechanism for securing HRI applications Roque
et al. [239] describe and implement an explainable AI solution that can be
used by humans to calibrate their trust in their systems. While most current
XAI solutions are geared towards domain experts [240], other approaches [241]
target non-experts when explaining robot failures. In order to provide truly
comprehensive XAI solutions in AI-Robotics, end-users with no (or very little)
AI/ML experience must be considered. In fact, the majority of end-users will
not be experts in many robotic applications.

Another emerging research topic is neurorobotics. Since the beginning, AI and
robotics have always drawn inspiration from biology. Today, researchers [242]
are leveraging neuroethology, the combination of neuroscience and ethology, to
study the effect of neural network perturbations in hopes of explaining a
robot’s behavior.

###  VII-E Safe Robot Learning

The increased use of data-driven approaches in safety-critical applications
has raised the question of safety (and safe guarantees) for such learning
algorithms [243]. Brunke et al. [244] define safe learning control as the
problem of safe decision-making under uncertainties using machine learning.
Reinforcement learning (i.e., data-driven) approaches are highly adaptable but
do not provide safety guarantees. Conversely, control theory (i.e., model-
driven) approaches provide guarantees within known operation conditions but
are less adaptable. Current research [245] is investigating the merger of
control theory and reinforcement learning to provide generalizable and safe
solutions within certain boundaries.

Many of these solutions use physics-based environments during model training.
In certain scenarios, these environments will also have to be protected to
avoid corruption and unintended disclosure of information (e.g., performance
bounds, physical constraints) for any derived model.

###  VII-F Securing Verification, Validation, and Evaluation of AI-Robotics
Systems

Ensuring safe, secure, and trustworthy AI is an active area of research [246].
This naturally extends to AI-Robotics systems. How can these systems be
verified and validated in large-dimensional and dynamic environments? How do
you ensure that these systems are robust to situations different from those
encountered during the training/design phase (i.e., new/unseen data)? How do
you capture all the decisions made by an AI-Robotics system for traceability?
The future of AI-Robotics will look at mitigating security risks and data
protection.

###  VII-G Education Initiatives to Raise Robotics Security Awareness

Raising robotics security awareness and training is crucial in our
increasingly interconnected world, where robots and autonomous systems are
becoming more prevalent.

Hands-on hacking exercises on various forms of cyber-attacks on Robotic
systems is an effective tool for security education to raise awareness about
various forms of cyberattacks to Robotic systems stakeholders. For example,
within the scope of their scholarly investigation [247], the authors reviewed
the current landscape of hacking exercises, their efficacy, and challenges
through a set of pedagogical principles. Moreover, in an effort to promote
awareness and bolster the security of AI-Robotics in academic environments,
researchers [23] are advocating for the inclusion of robotics curricula that
equip students with the knowledge and abilities required for developing
resilient and secure robotic applications.

Similarly, researchers also recommend game-based teaching approaches to make
the learning enjoyable, and such studies indicate that it has positive
cognitive and affective effects in training people [248, 249]. In their work
[248], the authors designed security requirements educational game to educate
stakeholders about cyber security awareness and to empirically evaluate the
impact of game-based training. The game included puzzle cards with security-
related terms and concepts that gave a deeper understanding of an attack in a
real scenario. Along the same lines, the researchers in [249] investigated
contrasting security decisions of security experts, managers, and computer
scientists from the three demographics to learn the difference in their
perception of security. Their game primarily focused on player’s decisions in
handling physical security, firewalls, antivirus, network monitoring, and
intrusion detection.

On the other hand, there are companies and professional associations that
provide training and certification programs to educate stakeholders with the
necessary skills and knowledge to secure robotic systems. For example, the
Association for Advancing Automation (A3) [250] offers certification in
robotics systems security to ensure that engineers and integrators have the
appropriate cybersecurity knowledge and skills.

##  VIII Conclusion

Since its onset, robotics has been tightly coupled with AI. The first general-
purpose robot, Shakey, was developed in the late 1960s and the first robotic
AI architecture came with it. Today, AI-Robotics systems are deeply embedded
in our everyday lives (e.g., robotic vacuum cleaners, cars with semi-
autonomous capabilities, delivery robots). A common AI-Robotics system may use
its sensors to collect data about itself and the environment. The physical
system, algorithms, and data are all susceptible to security attacks.

In this paper, we provide an overview of attack surfaces and associated
defensive strategies relevant to AI-Robotics systems. We introduce a taxonomy
of security attacks on these robotics systems and identified two major
categories: physical and digital attacks. In order to highlight the
vulnerabilities of AI-Robotics systems, we decompose fundamental robotic AI
architectures into primitives (i.e., sense, plan, attack) and stacks (i.e.,
perception, planning & navigation, and control) providing security risks at
each layer. Emerging technologies, including human-robot interaction, are
discussed while making note of privacy, ethical, and trustworthiness concerns.
Finally, we provide our perceived future research direction to aid researchers
interested in this exciting field.

## Acknowledgements

The work reported herein was supported by the National Science Foundation
(NSF) (Award #2246920). Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the authors and do not
necessarily reflect the views of the NSF. We would like to acknowledge Charles
Raines for his help in compiling literature related to ROS, OpenCV, and
Tensorflow.

The authors would also like to thank the PATENT Lab (Predictive Analytics and
TEchnology iNTegration Laboratory) at the Department of Computer Science and
Engineering, Mississippi State University.

## References

  * [1] Robot system architectures.  https://web.stanford.edu/class/cs237b/pdfs/lecture/lecture_14.pdf.
  * [2] The world’s most experienced driver.  https://waymo.com/.
  * [3] Fahad Sherwani, Muhammad Mujtaba Asad, and Babul Salam Kader K Ibrahim.  Collaborative robots and industrial revolution 4.0 (ir 4.0).  In 2020 International Conference on Emerging Trends in Smart Technologies (ICETST), pages 1–5. IEEE, 2020.
  * [4] Simon Leonard, Kyle L Wu, Yonjae Kim, Axel Krieger, and Peter CW Kim.  Smart tissue anastomosis robot (star): A vision-guided robotics system for laparoscopic suturing.  IEEE Transactions on Biomedical Engineering, 61(4):1305–1317, 2014\.
  * [5] Qingsong Ai, Zemin Liu, Wei Meng, Quan Liu, and Sheng Q Xie.  Machine learning in robot assisted upper limb rehabilitation: A focused review.  IEEE Transactions on Cognitive and Developmental Systems, 2021.
  * [6] Redmond R Shamshiri, Cornelia Weltzien, Ibrahim A Hameed, Ian J Yule, Tony E Grift, Siva K Balasundram, Lenka Pitonakova, Desa Ahmad, and Girish Chowdhary.  Research and development in agricultural robotics: A perspective of digital farming.  2018\.
  * [7] Maanak Gupta, Mahmoud Abdelsalam, Sajad Khorsandroo, and Sudip Mittal.  Security and privacy in smart farming: Challenges and opportunities.  IEEE access, 8:34564–34584, 2020.
  * [8] Sina Sontowski, Maanak Gupta, Sai Sree Laya Chukkapalli, Mahmoud Abdelsalam, Sudip Mittal, Anupam Joshi, and Ravi Sandhu.  Cyber attacks on smart farming infrastructure.  In 2020 IEEE 6th International Conference on Collaboration and Internet Computing (CIC), pages 135–143. IEEE, 2020.
  * [9] Patrick Lin, George Bekey, and Keith Abney.  Autonomous military robotics: Risk, ethics, and design.  2008\.
  * [10] Jean-Paul A Yaacoub, Hassan N Noura, Ola Salman, and Ali Chehab.  Robotics cyber security: Vulnerabilities, attacks, countermeasures, and recommendations.  International Journal of Information Security, pages 1–44, 2022\.
  * [11] Sunakshi Jaitly, Harshit Malhotra, and Bharat Bhushan.  Security vulnerabilities and countermeasures against jamming attacks in wireless sensor networks: A survey.  In 2017 International Conference on Computer, Communications and Electronics (Comptelix), pages 559–564. IEEE, 2017.
  * [12] Bing Wu, Jianmin Chen, Jie Wu, and Mihaela Cardei.  A survey of attacks and countermeasures in mobile ad hoc networks.  Wireless network security, pages 103–135, 2007.
  * [13] Yassine Mekdad, Ahmet Aris, Leonardo Babun, Abdeslam El Fergougui, Mauro Conti, Riccardo Lazzeretti, and A Selcuk Uluagac.  A survey on security and privacy issues of uavs.  Computer Networks, 224:109626, 2023.
  * [14] Julio A Placed, Jared Strader, Henry Carrillo, Nikolay Atanasov, Vadim Indelman, Luca Carlone, and José A Castellanos.  A survey on active simultaneous localization and mapping: State of the art and new frontiers.  IEEE Transactions on Robotics, 2023.
  * [15] Li Li, Kangye Qu, and Kuo-Yi Lin.  A survey on attack resilient of uav motion planning.  In 2020 IEEE 16th International Conference on Control & Automation (ICCA), pages 558–563. IEEE, 2020.
  * [16] Ayyoub Lamssaggad, N. Benamar, A. Hafid, and M. Msahli.  A survey on the current security landscape of intelligent transportation systems.  IEEE Access, 9:9180–9208, 2021.
  * [17] Sandeep Nair Narayanan, Sudip Mittal, and Anupam Joshi.  Obd_securealert: An anomaly detection system for vehicles.  In 2016 IEEE International Conference on Smart Computing (SMARTCOMP), pages 1–6. IEEE, 2016.
  * [18] Sai Sree Laya Chukkapalli, Sudip Mittal, Maanak Gupta, Mahmoud Abdelsalam, Anupam Joshi, Ravi Sandhu, and Karuna Joshi.  Ontologies and artificial intelligence systems for the cooperative smart farming ecosystem.  Ieee Access, 8:164045–164064, 2020.
  * [19] Hackers can trick a tesla into accelerating by 50 miles per hour.  https://www.technologyreview.com/2020/02/19/868188/hackers-can-trick-a-tesla-into-accelerating-by-50-miles-per-hour, February, 2020.  Accessed: 2023-08-8.
  * [20] Legendary hackers charlie miller and chris valasek talk cybersecurity and autonomous vehicles at tc sessions: Mobility 2022.  https://tcrn.ch/3iN2mRx, 2022.  Accessed: 2022-09-26.
  * [21] Charlie Miller.  Lessons learned from hacking a car.  IEEE Design & Test, 36(6):7–9, 2019.
  * [22] Artificial intelligence and cybersecurity opportunities and challenges.  https://www.nitrd.gov/pubs/AI-CS-Tech-Summary-2020.pdf, March, 2023\.  Accessed: 2023-07-27.
  * [23] Sudip Mittal and Jingdao Chen.  Ai security threats against pervasive robotic systems: A course for next generation cybersecurity workforce.  arXiv preprint arXiv:2302.07953, 2023.
  * [24] Elisa Bertino, Ravi Sandhu, Bhavani Thuraisingham, Indrakshi Ray, Wenjia Li, Maanak Gupta, and Sudip Mittal.  Security and privacy for emerging iot and cps domains.  In Proceedings of the Twelfth ACM Conference on Data and Application Security and Privacy, pages 336–337, 2022.
  * [25] Laurel Riek and Don Howard.  A code of ethics for the human-robot interaction profession.  Proceedings of we robot, 2014.
  * [26] Edoardo Datteri, Guglielmo Tamburrini, et al.  Ethical reflections on health care robotics.  Ethics and robotics, pages 35–48, 2009.
  * [27] California consumer privacy act.  https://oag.ca.gov/privacy/ccpa, 2023.  Accessed: 2023-10-05.
  * [28] General data protection regulation.  https:GeneralDataProtectionRegulation.  Accessed: 07-08-23.
  * [29] [Online] Flaticon.  Accessed:2023-08-30.  https://www.flaticon.com/.
  * [30] Robin R Murphy.  Introduction to AI robotics.  MIT press, 2019.
  * [31] Zhanat Kappassov, Juan-Antonio Corrales, and Véronique Perdereau.  Tactile sensing in dexterous robot hands.  Robotics and Autonomous Systems, 74:195–220, 2015.
  * [32] Pritam Dash, Mehdi Karimibiuki, and Karthik Pattabiraman.  Stealthy attacks against robotic vehicles protected by control-based intrusion detection techniques.  Digital Threats: Research and Practice, 2(1):1–25, 2021.
  * [33] Hans Moravec and Alberto Elfes.  High resolution maps from wide angle sonar.  In Proceedings. 1985 IEEE international conference on robotics and automation, volume 2, pages 116–121. IEEE, 1985.
  * [34] Armin Hornung, KM Wurm, Maren Bennewitz, C Stachniss, and W Burgard.  An efficient probabilistic 3d mapping framework based on octrees armin hornung.  Autonomous Robots Journal. Springer, 2013.
  * [35] Artúr István Károly, Péter Galambos, József Kuti, and Imre J Rudas.  Deep learning in robotics: Survey on model structures and training strategies.  IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(1):266–279, 2020.
  * [36] Max Schwarz, Hannes Schulz, and Sven Behnke.  Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features.  In 2015 IEEE international conference on robotics and automation (ICRA), pages 1329–1335. IEEE, 2015.
  * [37] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  * [38] Clement Farabet, Camille Couprie, Laurent Najman, and Yann LeCun.  Learning hierarchical features for scene labeling.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1915–1929, 2013.
  * [39] Pedro Pinheiro and Ronan Collobert.  Recurrent convolutional neural networks for scene labeling.  In International conference on machine learning, pages 82–90. PMLR, 2014.
  * [40] Charles Moore, Shaswata Mitra, Nisha Pillai, Marc Moore, Sudip Mittal, Cindy Bethel, and Jingdao Chen.  Ura*: Uncertainty-aware path planning using image-based aerial-to-ground traversability estimation for off-road environments.  arXiv preprint arXiv:2309.08814, 2023.
  * [41] Wadim Kehl, Fausto Milletari, Federico Tombari, Slobodan Ilic, and Nassir Navab.  Deep learning of local rgb-d patches for 3d object detection and 6d pose estimation.  In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 205–220. Springer, 2016.
  * [42] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [43] Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Yang, and Jiaqi Ma.  V2x-vit: Vehicle-to-everything cooperative perception with vision transformer.  In European conference on computer vision, pages 107–124. Springer, 2022.
  * [44] Changhong Fu, Weiyu Peng, Sihang Li, Junjie Ye, and Ziang Cao.  Local perception-aware transformer for aerial tracking.  In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12122–12129. IEEE, 2022.
  * [45] Peter E Hart, Nils J Nilsson, and Bertram Raphael.  A formal basis for the heuristic determination of minimum cost paths.  IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968.
  * [46] Steven LaValle.  Rapidly-exploring random trees: A new tool for path planning.  Research Report 9811, 1998.
  * [47] James Reeds and Lawrence Shepp.  Optimal paths for a car that goes both forwards and backwards.  Pacific journal of mathematics, 145(2):367–393, 1990.
  * [48] Dieter Fox, Wolfram Burgard, and Sebastian Thrun.  The dynamic window approach to collision avoidance.  IEEE Robotics & Automation Magazine, 4(1):23–33, 1997.
  * [49] Linh Kästner, Xinlin Zhao, Teham Buiyan, Junhui Li, Zhengcheng Shen, Jens Lambrecht, and Cornelius Marx.  Connecting deep-reinforcement-learning-based obstacle avoidance with conventional global planners using waypoint generators.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1213–1220. IEEE, 2021.
  * [50] Noé Pérez-Higueras, Fernando Caballero, and Luis Merino.  Learning human-aware path planning with fully convolutional networks.  In 2018 IEEE international conference on robotics and automation (ICRA), pages 5897–5902. IEEE, 2018.
  * [51] Guangda Chen, Lifan Pan, Pei Xu, Zhiqiang Wang, Peichen Wu, Jianmin Ji, Xiaoping Chen, et al.  Robot navigation with map-based deep reinforcement learning.  In 2020 IEEE International Conference on Networking, Sensing and Control (ICNSC), pages 1–6. IEEE, 2020.
  * [52] Abhijith R Puthussery, Karthik P Haradi, Berat A Erol, Patrick Benavidez, Paul Rad, and Mo Jamshidi.  A deep vision landmark framework for robot navigation.  In 2017 12th system of systems engineering conference (SoSE), pages 1–6. IEEE, 2017.
  * [53] Charles Richter and Nicholas Roy.  Safe visual navigation via deep learning and novelty detection.  2017\.
  * [54] Peter Regier, Lukas Gesing, and Maren Bennewitz.  Deep reinforcement learning for navigation in cluttered environments.  In Proc. of the Intl. Conf. on Machine Learning and Applications (CMLA), 2020.
  * [55] Pengzhan Chen, Jiean Pei, Weiqing Lu, and Mingzhen Li.  A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance.  Neurocomputing, 497:64–75, 2022.
  * [56] Douglas Morrison, Peter Corke, and Jürgen Leitner.  Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach.  arXiv preprint arXiv:1804.05172, 2018.
  * [57] John A Shaw.  The pid control algorithm.  Process control solutions, 2, 2003.
  * [58] Lal Bahadur Prasad, Barjeev Tyagi, and Hari Om Gupta.  Optimal control of nonlinear inverted pendulum system using pid controller and lqr: performance analysis without and with disturbance input.  International Journal of Automation and Computing, 11:661–670, 2014\.
  * [59] Jens Kober, J Andrew Bagnell, and Jan Peters.  Reinforcement learning in robotics: A survey.  The International Journal of Robotics Research, 32(11):1238–1274, 2013.
  * [60] Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al.  Scalable deep reinforcement learning for vision-based robotic manipulation.  In Conference on Robot Learning, pages 651–673. PMLR, 2018.
  * [61] Sergey Levine, Nolan Wagener, and Pieter Abbeel.  Learning contact-rich manipulation skills with guided policy search.  CoRR, abs/1501.05611, 2015.
  * [62] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song.  Robust physical-world attacks on deep learning visual classification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1625–1634, 2018.
  * [63] Ron Ross, Victoria Pillitteri, Kelley Dempsey, Mark Riddle, and Gary Guissanie.  Protecting controlled unclassified information in nonfederal systems and organizations.  Technical report, National Institute of Standards and Technology, 2019\.
  * [64] Siegfried Hollerer, Clara Fischer, Bernhard Brenner, Maximilian Papa, Sebastian Schlund, Wolfgang Kastner, Joachim Fabini, and Tanja Zseby.  Cobot attack: a security assessment exemplified by a specific collaborative robot.  Procedia Manufacturing, 54:191–196, 2021.
  * [65] Samson O Oruma, Mary Sánchez-Gordón, Ricardo Colomo-Palacios, Vasileios Gkioulos, and Joakim K Hansen.  A systematic review on social robots in public spaces: Threat landscape and attack surface.  Computers, 11(12):181, 2022.
  * [66] Swarup Bhunia, Michael S Hsiao, Mainak Banga, and Seetharam Narasimhan.  Hardware trojan attacks: Threat analysis and countermeasures.  Proceedings of the IEEE, 102(8):1229–1247, 2014.
  * [67] Yao Deng, Tiehua Zhang, Guannan Lou, Xi Zheng, Jiong Jin, and Qing-Long Han.  Deep learning-based autonomous driving systems: A survey of attacks and defenses.  IEEE Transactions on Industrial Informatics, 17(12):7897–7912, 2021\.
  * [68] Chen Yan, Wenyuan Xu, and Jianhao Liu.  Can you trust autonomous vehicles: Contactless attacks against sensors of self-driving vehicle.  Def Con, 24(8):109, 2016.
  * [69] Hocheol Shin, Dohyun Kim, Yujin Kwon, and Yongdae Kim.  Illusion and dazzle: Adversarial optical channel exploits against lidars for automotive applications.  In Cryptographic Hardware and Embedded Systems–CHES 2017: 19th International Conference, Taipei, Taiwan, September 25-28, 2017, Proceedings, pages 445–467. Springer, 2017.
  * [70] Bing Shun Lim, Sye Loong Keoh, and Vrizlynn LL Thing.  Autonomous vehicle ultrasonic sensor vulnerability and impact assessment.  In 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), pages 231–236. IEEE, 2018.
  * [71] Gorkem Kar, Hossen Mustafa, Yan Wang, Yingying Chen, Wenyuan Xu, Marco Gruteser, and Tam Vu.  Detection of on-road vehicles emanating gps interference.  In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, pages 621–632, 2014.
  * [72] Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z Morley Mao.  Adversarial sensor attack on lidar-based perception in autonomous driving.  In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pages 2267–2281, 2019.
  * [73] Zizhi Jin, Ji Xiaoyu, Yushi Cheng, Bo Yang, Chen Yan, and Wenyuan Xu.  Pla-lidar: Physical laser attacks against lidar-based 3d object detection in autonomous vehicle.  In 2023 IEEE Symposium on Security and Privacy (SP), pages 710–727. IEEE Computer Society, 2022.
  * [74] Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li.  Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks.  In 2021 IEEE Symposium on Security and Privacy (SP), pages 176–194. IEEE, 2021.
  * [75] Yi Zhu, Chenglin Miao, Tianhang Zheng, Foad Hajiaghajani, Lu Su, and Chunming Qiao.  Can we use arbitrary objects to attack lidar perception in autonomous driving?  In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 1945–1960, 2021.
  * [76] Denis Foo Kune, John Backes, Shane S Clark, Daniel Kramer, Matthew Reynolds, Kevin Fu, Yongdae Kim, and Wenyuan Xu.  Ghost talk: Mitigating emi signal injection attacks against analog sensors.  In 2013 IEEE Symposium on Security and Privacy, pages 145–159. IEEE, 2013.
  * [77] Wenyuan Xu, Chen Yan, Weibin Jia, Xiaoyu Ji, and Jianhao Liu.  Analyzing and enhancing the security of ultrasonic sensors for autonomous vehicles.  IEEE Internet of Things Journal, 5(6):5015–5029, 2018.
  * [78] Sean Rivera, Sofiane Lagraa, Antonio Ken Iannillo, and R. State.  Auto-encoding robot state against sensor spoofing attacks.  2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pages 252–257, 2019.
  * [79] P. Kapoor, A. Vora, and K. Kang.  Detecting and mitigating spoofing attack against an automotive radar.  2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), pages 1–6, 2018.
  * [80] Kexiong Curtis Zeng, Shinan Liu, Yuanchao Shu, Dong Wang, Haoyu Li, Yanzhi Dou, Gang Wang, and Yaling Yang.  All your $\\{$GPS$\\}$ are belong to us: Towards stealthy manipulation of road navigation systems.  In 27th USENIX security symposium (USENIX security 18), pages 1527–1544, 2018.
  * [81] Xiaoyu Ji, Yushi Cheng, Yuepeng Zhang, Kai Wang, Chen Yan, Wenyuan Xu, and Kevin Fu.  Poltergeist: Acoustic adversarial machine learning against cameras and computer vision.  In 2021 IEEE Symposium on Security and Privacy (SP), pages 160–175. IEEE, 2021.
  * [82] Gary Bradski.  The opencv library.  Dr. Dobb’s Journal: Software Tools for the Professional Programmer, 25(11):120–123, 2000.
  * [83] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al.  Tensorflow: Large-scale machine learning on heterogeneous distributed systems.  arXiv preprint arXiv:1603.04467, 2016.
  * [84] Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun.  Open3d: A modern library for 3d data processing.  arXiv preprint arXiv:1801.09847, 2018.
  * [85] Shoei Nashimoto, Daisuke Suzuki, Takeshi Sugawara, and Kazuo Sakiyama.  Sensor con-fusion: Defeating kalman filter in signal injection attack.  In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, pages 511–524, 2018.
  * [86] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [87] Cve-2018-5268 detail.  https://nvd.nist.gov/vuln/detail/CVE-2018-5268.
  * [88] Cve-2021-37650 detail.  https://nvd.nist.gov/vuln/detail/CVE-2021-37650.
  * [89] Jonathan Petit, Bas Stottelaar, Michael Feiri, and Frank Kargl.  Remote attacks on automated vehicles sensors: Experiments on camera and lidar.  Black Hat Europe, 11(2015):995, 2015.
  * [90] Yuan Xu, Xingshuo Han, Gelei Deng, Jiwei Li, Yang Liu, and Tianwei Zhang.  Sok: Rethinking sensor spoofing attacks against robotic vehicles from a systematic view.  arXiv preprint arXiv:2205.04662, 2022.
  * [91] Ji Hyoung Ryu, Muhammad Irfan, Aamir Reyaz, et al.  A review on sensor network issues and robotics.  Journal of Sensors, 2015, 2015.
  * [92] Akashdeep Bhardwaj, Vinay Avasthi, and Sam Goundar.  Cyber security attacks on robotic platforms.  Network Security, 2019(10):13–19, 2019.
  * [93] Gianluca Bianchin, Yin-Chen Liu, and Fabio Pasqualetti.  Secure navigation of robots in adversarial environments.  IEEE Control Systems Letters, 4(1):1–6, 2019.
  * [94] Chao-Han Huck Yang, Jun Qi, Pin-Yu Chen, Yi Ouyang, I-Te Danny Hung, Chin-Hui Lee, and Xiaoli Ma.  Enhanced adversarial strategically-timed attacks against deep reinforcement learning.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3407–3411. IEEE, 2020.
  * [95] Vishal Dey, Vikramkumar Pudi, Anupam Chattopadhyay, and Yuval Elovici.  Security vulnerabilities of unmanned aerial vehicles and countermeasures: An experimental study.  In 2018 31st international conference on VLSI design and 2018 17th international conference on embedded systems (VLSID), pages 398–403. IEEE, 2018.
  * [96] Daojing He, Hong Liu, Sammy Chan, and Mohsen Guizani.  How to govern the non-cooperative amateur drones?  IEEE Network, 33(3):184–189, 2019.
  * [97] Daojing He, Yinrong Qiao, Shiqing Chen, Xiao Du, Wenjie Chen, Sencun Zhu, and Mohsen Guizani.  A friendly and low-cost technique for capturing non-cooperative civilian unmanned aerial vehicles.  IEEE Network, 33(2):146–151, 2018.
  * [98] Harshad Sathaye, Martin Strohmeier, Vincent Lenders, and Aanjhan Ranganathan.  An experimental study of $\\{$GPS$\\}$ spoofing and takeover attacks on $\\{$UAVs$\\}$.  In 31st USENIX Security Symposium (USENIX Security 22), pages 3503–3520, 2022.
  * [99] Junjie Shen, Jun Yeon Won, Zeyuan Chen, and Qi Alfred Chen.  Drift with devil: Security of $\\{$Multi-Sensor$\\}$ fusion based localization in $\\{$High-Level$\\}$ autonomous driving under $\\{$GPS$\\}$ spoofing.  In 29th USENIX Security Symposium (USENIX Security 20), pages 931–948, 2020.
  * [100] Yiming Li, Congcong Wen, Felix Juefei-Xu, and Chen Feng.  Fooling lidar perception via adversarial trajectory perturbation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7898–7907, 2021.
  * [101] Kanglan Tang, Junjie Shen, and Qi Alfred Chen.  Fooling perception via location: a case of region-of-interest attacks on traffic light detection in autonomous driving.  In NDSS Workshop on Automotive and Autonomous Vehicle Security (AutoSec), 2021.
  * [102] Zizhi Jin, Xiaoyu Ji, Yushi Cheng, Bo Yang, Chen Yan, and Wenyuan Xu.  Pla-lidar: Physical laser attacks against lidar-based 3d object detection in autonomous vehicle.  In 2023 IEEE Symposium on Security and Privacy (SP), pages 1822–1839. IEEE, 2023.
  * [103] Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z Morley Mao.  Towards robust $\\{$LiDAR-based$\\}$ perception in autonomous driving: General black-box adversarial sensor attack and countermeasures.  In 29th USENIX Security Symposium (USENIX Security 20), pages 877–894, 2020.
  * [104] Yunmok Son, Hocheol Shin, Dongkwan Kim, Youngseok Park, Juhwan Noh, Kibum Choi, Jungwoo Choi, and Yongdae Kim.  Rocking drones with intentional sound noise on gyroscopic sensors.  In 24th USENIX Security Symposium (USENIX Security 15), pages 881–896, 2015.
  * [105] Timothy Trippel, Ofir Weisse, Wenyuan Xu, Peter Honeyman, and Kevin Fu.  Walnut: Waging doubt on the integrity of mems accelerometers with acoustic injection attacks.  In 2017 IEEE European symposium on security and privacy (EuroS&P), pages 3–18. IEEE, 2017.
  * [106] Yazhou Tu, Zhiqiang Lin, Insup Lee, and Xiali Hei.  Injected and delivered: Fabricating implicit control over actuation systems by spoofing inertial sensors.  In 27th USENIX Security Symposium (USENIX Security 18), pages 1545–1562, 2018.
  * [107] Sai Vemprala and Ashish Kapoor.  Adversarial attacks on optimization based planners.  In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9943–9949. IEEE, 2021.
  * [108] Davis Rempe, Jonah Philion, Leonidas J Guibas, Sanja Fidler, and Or Litany.  Generating useful accident-prone driving scenarios via a learned traffic prior.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17305–17315, 2022.
  * [109] Adhitya Bantwal Bhandarkar, Sudharman K Jayaweera, and Steven A Lane.  Adversarial sybil attacks against deep rl based drone trajectory planning.  In MILCOM 2022-2022 IEEE Military Communications Conference (MILCOM), pages 1–6. IEEE, 2022.
  * [110] Thomas Hickling, Nabil Aouf, and Phillippa Spencer.  Robust adversarial attacks detection based on explainable deep reinforcement learning for uav guidance and planning.  arXiv preprint arXiv:2206.02670, 2022.
  * [111] Xingguang Peng and Demin Xu.  Intelligent online path planning for uavs in adversarial environments.  International Journal of Advanced Robotic Systems, 9(1):3, 2012\.
  * [112] Wei Wang, Yao Yao, Xin Liu, Xiang Li, Pei Hao, and Ting Zhu.  I can see the light: Attacks on autonomous vehicles using invisible lights.  In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 1930–1944, 2021.
  * [113] Lifeng Zhou, Vasileios Tzoumas, George J Pappas, and Pratap Tokekar.  Resilient active target tracking with multiple robots.  IEEE Robotics and Automation Letters, 4(1):129–136, 2018.
  * [114] David Saldana, Amanda Prorok, Shreyas Sundaram, Mario FM Campos, and Vijay Kumar.  Resilient consensus for time-varying networks of dynamic agents.  In 2017 American control conference (ACC), pages 252–258. IEEE, 2017.
  * [115] Ragesh K Ramachandran, James A Preiss, and Gaurav S Sukhatme.  Resilience by reconfiguration: Exploiting heterogeneity in robot teams.  In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6518–6525. IEEE, 2019.
  * [116] Junnan Song and Shalabh Gupta.  Care: Cooperative autonomy for resilience and efficiency of robot teams for complete coverage of unknown environments under robot failures.  Autonomous Robots, 44:647–671, 2020.
  * [117] Lifeng Zhou and Pratap Tokekar.  Multi-robot coordination and planning in uncertain and adversarial environments.  Current Robotics Reports, 2:147–157, 2021.
  * [118] Hyongju Park and Seth Hutchinson.  Robust rendezvous for multi-robot system with random node failures: an optimization approach.  Autonomous Robots, 42:1807–1818, 2018.
  * [119] Fan Yang and Nilanjan Chakraborty.  Algorithm for optimal chance constrained linear assignment.  In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 801–808. IEEE, 2017.
  * [120] Lifeng Zhou and Pratap Tokekar.  An approximation algorithm for risk-averse submodular optimization.  In International Workshop on the Algorithmic Foundations of Robotics, pages 144–159. Springer, 2018.
  * [121] Ekaterina Tolstaya, Fernando Gama, James Paulos, George Pappas, Vijay Kumar, and Alejandro Ribeiro.  Learning decentralized controllers for robot swarms with graph neural networks.  In Conference on robot learning, pages 671–682. PMLR, 2020.
  * [122] Qingbiao Li, Fernando Gama, Alejandro Ribeiro, and Amanda Prorok.  Graph neural networks for decentralized multi-robot path planning.  In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11785–11792. IEEE, 2020.
  * [123] Ao Li, Jinwen Wang, and Ning Zhang.  Chronos: Timing interference as a new attack vector on autonomous cyber-physical systems.  In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 2426–2428, 2021.
  * [124] Xueyuan Wang and M Cenk Gursoy.  Resilient uav path planning for data collection under adversarial attacks.  In ICC 2022-IEEE International Conference on Communications, pages 625–630. IEEE, 2022.
  * [125] Xueyuan Wang and M Cenk Gursoy.  Resilient path planning for uavs in data collection under adversarial attacks.  IEEE Transactions on Information Forensics and Security, 2023.
  * [126] Damith Herath and David St-Onge.  Foundations of Robotics: A Multidisciplinary Approach with Python and ROS.  Springer, 2022.