# Fingerprinting Robot Movements via Acoustic Side Channel

Ryan Shah  University of StrathclydeUnited Kingdom<EMAIL_ADDRESS>,
Mujeeb Ahmed  University of StrathclydeUnited Kingdom
<EMAIL_ADDRESS>and  Shishir Nagaraja  Newcastle UniversityUnited
Kingdom<EMAIL_ADDRESS>
###### Abstract.

In this paper, we present an acoustic side channel attack which makes use of
smartphone microphones recording a robot in operation to exploit acoustic
properties of the sound to fingerprint a robot’s movements. In this work we
consider the possibility of an insider adversary who is within physical
proximity of a robotic system (such as a technician or robot operator),
equipped with only their smartphone microphone. Through the acoustic side-
channel, we demonstrate that it is indeed possible to fingerprint not only
individual robot movements within 3D space, but also patterns of movements
which could lead to inferring the purpose of the movements (i.e. surgical
procedures which a surgical robot is undertaking) and hence, resulting in
potential privacy violations. Upon evaluation, we find that individual robot
movements can be fingerprinted with around 75% accuracy, decreasing slightly
with more fine-grained movement meta-data such as distance and speed.
Furthermore, workflows could be reconstructed with around 62% accuracy as a
whole, with more complex movements such as pick-and-place or packing
reconstructed with near perfect accuracy. As well as this, in some
environments such as surgical settings, audio may be recorded and transmitted
over VoIP, such as for education/teaching purposes or in remote telemedicine.
The question here is, can the same attack be successful even when VoIP
communication is employed, and how does packet loss impact the captured audio
and the success of the attack? Using the same characteristics of acoustic
sound for plain audio captured by the smartphone, the attack was 90% accurate
in fingerprinting VoIP samples on average across baseline movements, which is
around 15% higher than the baseline without the VoIP codec employed. This is
an interesting result as it opens up new research questions regarding
anonymous communications to protect robotic systems from acoustic side channel
attacks via VoIP communication networks.

robot, security, privacy, acoustic, side channel, attack, passive, deep
learning, neural network, voip

††ccs: Security and privacy Systems security††ccs: Security and privacy Side-
channel analysis and countermeasures

##  1\. Introduction

The prominence of teleoperated robotic systems has seen a recent rise in a
variety of application areas, such as industrial (Quarta et al., 2017) and
surgical environments (Ahn et al., 2015; Tewari et al., 2002), with promises
of higher levels of accuracy and precision. Given that many of these systems
are becoming increasingly connected, they are vulnerable to an expanded threat
landscape in the cyber domain. Attacks from this angle are primarily active
attacks such as tampering with the integrity of messages in-flight or
hijacking the robot controller directly (Bonaci et al., 2015). However, little
attention has been paid to the capabilities of a passive attacker and the
damage potential of stealthier attacks. Specifically, passive attacks such
side channel attacks which exploit information leakages without the need to
change the normal behaviour of the system, can result in huge losses that stem
from the compromise of operational confidentiality. Side channel attacks in
the cyber domain have the potential to compromise the operational
confidentiality of organisations that own such systems (Shah et al., 2022),
yet those targeting robots in the physical domain are still to be explored.

In this paper, we aim to investigate whether an adversary can exploit
information leakages from the acoustic side channel, by capturing audible
emanations from a robotic system during normal operations, to mount an attack
that targets operational confidentiality. In this context, we look at two
possible threats posed by an insider attacker. First, a malicious robot
operator or technician on the ground could use a recording device, such as a
smartphone, near the robot to record entire workflows or individual movements.
By fingerprinting this leaked information, they could sell this on to
competing organisations for a malicious advantage. While it can be argued that
an attacker may not be able to get close enough to the robot to place the
recording device, many robotic systems now employ sensors to aid safety
mechanisms to prevent harm to nearby humans or environmental changes. This can
allow the attacker windows of opportunity to place the recording device near
the robot or be near enough to capture meaningful acoustic emanations. A
second possible threat comes from a telemonitoring perspective. While
telemonitoring is less common in industrial settings, in surgical settings the
use of medical recording devices, such as medical data recorders or
intraoperative video recorders, are used for post-surgical review or teaching
(alongside patient consent) to learn from suboptimal scenarios and improve
performance (Saun et al., 2019; Vodafone, 2019; Al-Jabir et al., 2020). While
privacy laws and medicolegal requirements govern the use of such devices, data
from them is not typically required as evidence in court so long as patient
confidentiality is maintained (Dalen et al., 2019). However, acoustic
emanations captured by such recordings could reveal the operations the robot
is carrying out, and ultimately piece together surgical procedures. In
combination with other metadata, such as patient admission and exit times,
this could compromise patient confidentiality.

In this attack, we recorded the acoustic emanations, through a smartphone
recorder, for individual robot movements, as well as recording entire
workflows corresponding to typical warehousing operations such as picking and
placing objects from one place to another. Using the collected data, we
extract a set of acoustic characteristics which are used as input to an
artificial neural network (ANN). We found that baseline movements (of minimum
speed and distance) can be fingerprinted with at least ~75% accuracy as a
baseline. The speed and distance of movements are not as successfully
fingerprinted in this attack, compared to the radio frequency side channel.
Entire warehousing workflows with ~64% accuracy. Ultimately, it is clear that
a passive insider adversary has the potential not only to reveal what a robot
is doing but take the resulting liabilities of such an attack to an extreme
that impacts even the organisations that employ them. As well as this, in
certain robotics environments, such as in surgical settings, procedures may be
streamed and/or recorded for viewing, education or research (Muensterer et
al., 2014; Kulkarni et al., 2020; Hosseini et al., 2013). Therefore, it is
important to question how VoIP impacts the audio samples for movements and
workflows and, ultimately, the success of the attack. Using the Opus codec – a
common choice for most modern VoIP applications – the attack was 90% accurate
for computing movement fingerprints for baseline speed and distance, which is
nearly 15% more accurate than the baseline without the Opus codec employed,
presenting new research questions regarding side channel attacks via VoIP
communication networks which target robotic systems.

The remainder of this paper is as follows. In Section 2 we provide background
on teleoperated robots and acoustic emanations, to which we then describe the
threat model. We then outline the attack and our findings in Section 3, and
provide an in-depth discussion in Section 5. In Section 6 we discuss related
work and conclude in Section 7.

##  2\. Background

###  2.1. Teleoperated Robots

The use of robotics has seen an increase in installations in a variety of
application areas (Reuters, 2019) and play pivotal roles bringing benefits to
quality of service, efficiency and precision, among others. Among them,
teleoperated robots are most prominent in many industrial (Aschenbrenner et
al., 2015; Avila et al., 2020; Grabowski et al., 2021; Li et al., 2017; Bartoš
et al., 2021) and surgical (Sung and Gill, 2001; Tewari et al., 2002;
Hannaford et al., 2012) environments, and share a common system architecture.
This type of system makes use of a human operator (i.e. a specially-trained
surgeon) who operates the controller (i.e. surgeon’s console or teach pendant)
which translates human movements or inputs into those which the robot can
interpret. These input (console and other sources of information) and output
(actuators) devices are linked together via an electronic control system (ECS)
and typically connected to the organisation’s network in which the robot
operates. An overview of a typical teleoperated robot architecture can be seen
in Figure 1.

Figure 1. Teleoperated Robot Architecture

###  2.2. Acoustic Characteristics

While the robot moves, its electromechanical components will emit (audible)
sound, which when captured could be used to mount an information leakage
attack. The first step to determining an appropriate attack strategy is to
understand the different characteristics of acoustic emanations, and what may
be most useful from an attack perspective.

####  2.2.1. Root Mean Square Energy

Root Mean Square (RMS) energy (Bachu et al., 2008) is a measure of the
amplitude based on all samples in a frame of audio, and can be thought of as
an indicator of loudness of the audio signal (Panagiotakis and Tziritas,
2005). This may be useful in the context of this attack given that
combinations of movements (i.e. simultaneous movements along 2 or more axes)
may emanate a louder sound given the use of more stepper motors, for example.
As well as this, as the robot axes pass the microphone, the sound may be
louder and thus this feature may help provide further information to the
discrimination between movements in different positions.

####  2.2.2. Zero-Crossing Rate

Zero-Crossing Rate (ZCR) is a measure of the number of times a signal crosses
the horizontal time axis and can help identify pitch variations in monophonic
tones (sound emitted from one location) (Panagiotakis and Tziritas, 2005).
Given the robot is stationary in this case, the ZCR may be a useful feature
candidate.

####  2.2.3. Spectral Centroid

The spectral centroid provides information corresponding to frequency bands
contain most of the energy, wherein lower centroid (energy) values is likened
to duller sounds and higher centroid values for brighter sounds (Le et al.,
2011). In a robotic system, smaller movement distances and speeds will
naturally require less energy and appear more dull sounding to the human ear,
whereas faster and longer movements have better tonality, and may ultimately
provide useful for distinguishing between different movements of the same
source.

####  2.2.4. Spectral Bandwidth

Spectral bandwidth is defined as the full width of band of light (wavelength
interval) at half the peak maximum (Klapuri and Davy, 2007; Atahan et al.,
2021). Acoustic signals oscillate about a point, and the bandwidth for each
time interval in a signal is the sum of maximum deviation on both sides of
this point. The point of the centroid of the signal may vary for different
robot movements and thus may be an important feature for fingerprinting.

####  2.2.5. Spectral Rolloff

Spectral rolloff is the fraction of frequency bins under a cutoff point where
the total energy of the spectrum is contained, and can help distinguish
between noisy sounds and more harmonic sounds (below roll-off point) (Kos et
al., 2013). This feature may provide useful to this attack as it can roll off
frequencies that may fall outside of the useful range of frequencies where the
energy of the acoustic energy of movements is contained.

####  2.2.6. Spectral Contrast

Spectral contrast is the measure of energy of frequencies in windows of time
(Jiang et al., 2002) and can help identify strong spectral peaks to reflect
the distribution between harmonic and non-harmonic components of the acoustic
emanations. As a robot moves, the frequency contents may have energy that
changes with time and capturing the spectral contrast can help measure this
energy variation.

####  2.2.7. Chroma Feature

Chroma feature, sometimes referred to as a chromagram, profiles a sound into
12 pitch class profiles (Müller, 2015). In music analysis, the attempt is to
capture the harmonic and melodic characteristics of a song where pitches can
be categorised to one of the scales in the equally-tempered set of the notes
$\\{C,C\\#,D,D\\#,E,F,F\\#,G,G\\#,A,A\\#,B\\}$ (Cho and Bello, 2013; Paulus et
al., 2010). While recorded robot movements are not akin to songs that are
analysed in this fashion, the pitch of sound may correlate with the speed and
distance of movement and may provide useful as a mid-level feature for
fingerprinting movements.

####  2.2.8. Mel-Cepstrum Frequency Coefficients

The Mel scale is a scale of pitches that is felt to be equal in distance from
one another. For example, in audible acoustics listened by a human,
differences in frequency content can be observed if the source of acoustic
emanations are in the same distance and atmosphere (Greenwood, 1997; Martinez
et al., 2012). The short-term power spectrum of acoustic emanations can be
represented by the Mel frequency cepstral (MFC) and a combination of
coefficients (MFCCs) make up the MFC. The MFC equally distributes frequency
bands to approximate human auditory response. If variations in robot movements
can be inferred from audible sound, then looking at the mel frequency
coefficients (the list of amplitudes of the spectrum in the mel scale) will
provide useful information to the attack.

###  2.3. Threat Model

Many previous attacks focus on an active attacker, which can involve the
tampering of messages (Bonaci et al., 2015) or replaying attacks between the
robot or controller (McClean et al., 2013). In this work, the primary attacker
is a passive insider, such as a malicious technician or operator. Being an
insider close to the robot would allow them to record the acoustic emanations
during the robot’s normal operations using a smartphone, which they may have
on them and use covertly (Peng and Choi, 2013). As well as this, it is also
possible than an insider attacker is able to covertly plant a microphone which
could transmit recorded audio to the attacker remotely or be retrieved at a
later time. In either case, if an attacker is able to mount an information
leakage attack to fingerprint robot movement patterns from acoustic
emanations, this could lead to the revelation of daily workflows (i.e. in a
warehouse) and ultimately compromise the operational confidentiality of the
organisation. For example, this information could be given to competitors to
gain an advantage or use it maliciously.

A second possible threat comes from a telemonitoring perspective. While
telemonitoring is less common in industrial settings, in surgical settings the
use of medical recording devices, such as medical data recorders or
intraoperative video recorders, are used for post-surgical review or teaching
(alongside patient consent) to learn from suboptimal scenarios and improve
performance (Saun et al., 2019; Vodafone, 2019; Al-Jabir et al., 2020). While
privacy laws and medicolegal requirements govern the use of such devices, data
from them is not typically required as evidence in court so long as patient
confidentiality is maintained (Dalen et al., 2019). However, acoustic
emanations captured by such recordings could reveal the operations the robot
is carrying out, and ultimately piece together surgical procedures. In
combination with other metadata, such as patient admission and exit times,
this could compromise patient confidentiality.

Ultimately, reviewing the nature of acoustic emanations in robotic systems, as
well as the proposed threat model, the aim is to investigate whether an
advesary will be able to record the acoustic emanations from a robot during
its normal operation, and make use of distinct features present across the
recorded audio to fingerprint robot movements and workflows. Several
hypothetical factors will come into play which could influence the potential
success of this attack. First, the type of operations being carried out by the
robot can vary in terms of speed and distance of movement, and so the attack
should be robust enough to fingerprint between these parameters. Second, the
distance at which the microphone is situated away from the robot, naturally
due to the Doppler effect (Doppler, 1903) wherein sounds soften with distance,
will also have an impact on the success of the attack and should be
investigated. Finally, given that in some cases VoIP technology will be
employed, such as for recording purposes or to livestream medical procedures
with surgical robots, the impact of VoIP on the attack should be evaluated.

###  2.4. Hypotheses and Goals

In this work we aim to investigate whether an advesary will be able to
effectively record the acoustic emanations from a robot during its normal
operation, and make use of distinct features present across the recorded audio
to fingerprint robot movements and workflows. We hypothesise that several
factors will come into play which could influence the potential success of
this attack. First, the type of operations being carried out by the robot can
vary in terms of speed and distance of movement, and so the attack should be
robust enough to fingerprint between these parameters. Second, we also
hypothesise that the distance at which the microphone is situated away from
the robot, naturally due to the Doppler effect (Doppler, 1903) wherein sounds
soften with distance, will also have an impact on the success of the attack
and should be investigated. Ultimately, the following research questions are
proposed:

  1. ($R_{1}$)

Can an attacker fingerprint individual robot movements on each axes, as well
as permutations of them?

  2. ($R_{2}$)

How is movement fingerprinting affected by:

    1. (i)

The speed and distance of movements?

    2. (ii)

The distance the recording device (i.e. smartphone) is away from the robot?

  3. ($R_{3}$)

Can entire robot workflows be reconstructed from acoustic emanations?

  4. ($R_{4}$)

How do VoIP codecs influence the success of the attack?

##  3\. Attack Methodology

In this paper, we investigate an acoustic side-channel attack which exploits
audio emanations from a robot during its operation. Specifically, the aim of
this attack is to fingerprint a robots movements from acoustic characteristics
alone, recorded by smartphone devices in a passive manner. For subsequent
discussion, we aim to answer the following questions:

  1. ($R_{1}$)

Can an attacker fingerprint individual robot movements on each axes, as well
as permutations of them?

  2. ($R_{2}$)

How is movement fingerprinting affected by:

    1. (i)

The speed and distance of movements?

    2. (ii)

The distance the smartphone or microphone is away from the robot?

  3. ($R_{3}$)

Can an attacker recover information about the objects a robot is handling,
such as its weight?

Figure 2. Robot Environment for Acoustic Side Channel

###  3.1. Robot Environment

The context of this study surrounds modern teleoperated surgical robots, whos
typical architecture can be viewed (at a high level) as a pairing between the
robotic system itself and its controller (surgeon’s console). For this work,
we use uFactory’s uARM Swift Pro which runs on an Arduino Mega 2560 with
MicroPython installed. The controller is emulated on a Windows 10 laptop which
uses the uARM Python (3.8.X) SDK to enable controller instructions to be
written in Python which are then translated into Gcode that is understood by
the robot. An overview of the robot environment used in this study is depicted
in Figure 2. For capturing the acoustic emanations which arise when the robot
operates, we position the robot in the center of a table with the
smartphone/microphone placed in several distances away (30cm to 1m) from the
robot as shown in Figure 2.

###  3.2. Experiment Parameters

With the robot setup for evaluating our acoustic side-channel attack for
fingerprinting the robot’s movements, we now outline the parameters of our
study. Specifically, we will discuss the speed and distance of the movement
operation being carried out, the type of smartphone/micrphone, the distance
the smartphone microphone is away from the robot and finally,

Speed and Distance. In addition to capturing the acoustic emanations which
arise during operation along the X, Y and Z axes, and combined movement
operations, it is important to evaluate more fine-grained movements. To this,
we programmed robot movements with varying distances (in millimetres) as well
as varying speeds of movement (mm/s). This is because in realistic cases, a
surgical robot for example would not move in each direction with constant
distance and speed. Therefore, it is vital to understand whether an adversary
can also fingerprint meta-information as well as just the movements
themselves.

Microphone Distance. In terms of recording the acoustic emanations during
robot operation, it is important to evaluate the impact of distance the
microphone is away from the robot. In a real situation, it is highly unlikely
that an adversary would be very close to or in front of the robot, especially
in cases like surgical robots where it could not only be dangerous to stand
too close but being close enough may trigger potential safety features
implemented to prevent injury. For this study, given the size of our uARM
robot ($150mm\times 140mm\times 281mm$) and the volume of sound which is given
off during its operation, we cannot investigate large distances as would be
granted with a large surgical robot, for example. However, given this
limitation, we recorded sounds at distances ranging from 30cm to 1m.

VoIP. The final parameter for this study is to evaluate the impact VoIP has on
the success of the attack. For this study, the codec employed by the majority
of VoIP applications is Opus (Valin et al., 2012, 2016). The first step is to
observe how the codec performs, but also how packet loss will also affect
audio quality and the success of the attack.

Figure 3. Depiction of Common Warehousing Workflows Our dataset contains
common warehousing workflows such as pushing, pulling, packing and moving
objects

###  3.3. Movement Dataset

After determining the appropriate acoustic features to extract from the
captured sounds, the next step was to create the dataset. In this dataset,
there are 2 subsets. Within both subsets, there are samples pertaining to both
individual and permutations of movements with varying speeds and distances of
movement, the microphone distance, and robotic warehousing workflows (Figure
3). These workflows include those such as pick-and-place, push and pull
operations, which were replicated from those found in existing industrial
robot datasets such as the Forward Dynamics Dataset Using KUKA LWR and Baxter
(Polydoros and Nalpantidis, 2016) for pick and place and the Inverse Dynamics
Dataset Using KUKA (Rueckert et al., 2017) for push/pull. For these workflows,
movements were slightly perturbated to account for a small degree of entropy
that may be present in real-world operations (i.e. those that may arise due to
drift in equipment calibration or wear-and-tear). In contrast to the first
subset, the second subset contains the same samples but are passed through the
Opus codec to evaluate the impact of VoIP on recorded audio in this attack.
Specifically, while all samples are passed through the Opus codec, they are
further split by packet loss. Packet loss has been shown to negatively impact
call quality in VoIP communications (Ortega et al., 2018; Laghari et al.,
2020), as they induce impact in the form of dropped calls or parts of speech,
slow rate of speech (latency) or noise/interference. Because of this, these
further subsets are divided by packet loss values of 1%, 5%, 10%, 25% and 50%.
As a whole, the first subset contains 27.2K samples for individual movements
and 658 samples for warehousing workflows, with each using 20% of the total
samples for validation and another 20% for testing. The second contains the
same amount of samples for each of the packet losses evaluated.

Dataset Pre-Processing. The features in the dataset, as listed above, are
computed using the librosa (McFee et al., 2015) Python library. For each
feature, the mean value of each feature across each signal sample is taken and
computed from a Short-Time Fourier Transform (STFT) with a Hann window and FFT
length of 8192. For the MFCCs, 14 coefficients were used. Typically, 8–13 are
used with the zeroth excluded given it only represents the average log-energy
of the input signal (Rao and Vuppala, 2014). However, given this is a new
problem to be explored, this is also kept to later examine its importance for
fingerprinting.

###  3.4. Neural Network

Before an evaluation can take place, an important step is constructing an
appropriate neural network architecture for fingerprinting movements and
ensuring a successful attack. To create the neural network, a sequential model
was used where neurons are grouped in a linear fashion. This was created using
the Keras API (Chollet et al., 2015). The parameters and structure for the
layers in the neural network were evaluated on the dataset using a cross-
validated grid search to find the most optimal number of neurons, layers,
acttion function and dropouts if necessary. The input for the maximum number
of neurons to be tested was calculated using the formula proposed by Demuth et
al. (Demuth et al., 2014) with an alpha branching factor of $2$. Using the
grid search with 3 cross validations, the most optimal neural network
architecture for this feature set consists of 5 layers. First, the input layer
containing 21 neurons for each of the input features. Next, there are 4 hidden
layers. The first is a Dense layer with 290 neurons and uses the ReLU
activation function (Eckle and Schmidt-Hieber, 2019). The next hidden layer is
a Dropout layer which is used to randomly set input units to 0 at a rate of
$0.05$ at each step during training to prevent overfitting. The next layer is
another Dense layer of 350 neurons with ReLU activation, followed by another
Dropout with a rate of $0.05$ to prevent overfitting. Finally, the last layer
is a Dense output layer of 7 neurons, one for each of the movement classes,
and uses the SoftMax activation function (Dunne and Campbell, 1997) to have
the output in the range of $[0,1]$ for use as predicted probabilities. Sparse
categorical cross-entropy is used as labels are integers and not one-hot
encoded, for which categorical cross-entropy would be used (Zhang and Sabuncu,
2018). The optimiser used is Adam with a learning rate of $0.001$. This
learning rate was chosen as others, such as those with higher learning rates,
resulted in lowered accuracy scores. The model was fitted with a batch size of
32 and was run for 1000 epochs.

Choice of Activation and Optimisation Functions. The ReLU activation function
was chosen over other activation functions, as the reduced likelihood of
vanishing gradient allows for a constant gradient resulting in faster
learning. Further, the sparsity of representations are shown to be more
beneficial than dense representations, as seen in other activations such as
sigmoids (Krizhevsky et al., 2012; Li and Yuan, 2017; Agarap, 2018). The
softmax activation function, combined with categorical cross-entropy (Zhang
and Sabuncu, 2018) for the loss function, was chosen due to the simple fact
that this is a multi-class classification problem. Simply, a sample can belong
to one of the 7 classes, with each class corresponding to one of the robot
movements. As well as this, the Adam optimiser was an ideal candidate. It is
an extension to the Stochastic Gradient Descent (SGD) method, based on
adaptive estimation of first- and second-order moments (Kingma and Ba, 2014).
Specifically, it allows for the updating of network weights iteratively based
on the training data, and fits best with the weighted sample sets in
opposition to other tried methods such as standard SGD, RMSProp and SGD +
Nesterov Momentum.

##  4\. Evaluation

After setting up the robot environment and capturing the acoustic emanations
during various stages of operations, the next step is to evaluate the success
of the attack. As per the research questions listed above, the evaluation of
this attack and related results will be set out in that order.

###  4.1. Individual Movement Fingerprints

The first research question ($R_{1}$) aims to investigate whether an attacker
can infer individual movements (on each axis) and permutations of these
movements from the recorded audio. To compare this against other parameters,
this experiment is considered as a baseline where the speed and distance of
movement are the lowest possible values (1mm and 12.5mm/s respectively), and
no VoIP codec used. As seen in Table 1, an average accuracy of around 75% can
be observed across all movements, with the YZ movement having the highest
precision among the movements. In comparison with the RF side channel, there
is a clear drop in accuracy of around 20% but the acoustic side channel
outperforms traffic analysis by around 10%. Interestingly, Y-involved
movements are better recovered than other movements overall, which was not the
case in the RF side channel (albeit a higher accuracy). This may be due to the
Y-axis moving across the microphone range. Looking at the Z-involved
movements, these are among the lowest. This may be due to the Z axis involving
a vertical movement only and not moving nearer the microphone for better
recording.

Movement | Precision | Recall
---|---|---
X | 76% | 81%
Y | 77% | 78%
Z | 61% | 71%
XY | 78% | 80%
XZ | 68% | 65%
YZ | 85% | 78%
XYZ | 72% | 67%
Accuracy | 75%
Table 1. Baseline Classification Results As a whole, the baseline accuracy is
75% which is fairly good inference accuracy for an attacker. Z-based movements
show the lowest precision and recall for fingerprinting, perhaps due to
vertical motion and no horizontal spread across the recording device

###  4.2. Impact of Movement Distance

For the next research question ($R_{2}$), the evaluation will look into how
the distance and speed ($R_{2(i)}$) of robot movements, and the distance of
the recording device ($R_{2(ii)}$), impact the success of fingerprinting
movements from the acoustic side channel. First, as a robot moves, there is
likely to be more sound that can be recovered as the distance of movement
increases. As seen in Table 2, an increase by a single distance unit increases
the model accuracy by 1%, improving Y-involved movement precision by around
10%. furthermore, the Z movement also gains a slight increase in precision.
Unfortunately, this results in lowered accuracy for the other movements. This
increase in distance results in the sound of movement being held for longer
and may either provide useful for distinguishing variance between movements or
even reduce this variance. To explore this, larger distances of movements are
explored. At 5mm, there is a drop in accuracy of around 4%, with X-involved
movements having much higher accuracy. At 10mm, the accuracy of the model
overall decreases significantly to 57%. Y-involved movements in this case are
much poorly fingerprinted, yet X-involved movements have a further increase in
precision. For the Z movement at this stage, there is unfortunately a further
drop in precision but the recall remains relatively similar. At 25mm, the
accuracy starts to improve by 7% with the X movement having similar precision
and recall to 10mm, and most other movements have an increase in both
precision and recall. Finally, at 50mm, the accuracy nears that of the
baseline and 2mm, however X-involved movement accuracy is significantly
improved.

D = Distance (mm), P = Precision, R = Recall
---
| D=2 | D=5 | D=10 | D=25 | D=50 |
P | R | P | R | P | R | P | R | P | R |
X | 69% | 71% | 77% | 87% | 85% | 58% | 83% | 70% | 86% | 84% |
Y | 88% | 77% | 77% | 79% | 80% | 54% | 66% | 43% | 90% | 88% |
Z | 65% | 83% | 64% | 79% | 51% | 81% | 64% | 71% | 83% | 66% |
XY | 68% | 60% | 67% | 63% | 63% | 53% | 57% | 65% | 79% | 81% |
XZ | 62% | 57% | 83% | 47% | 67% | 49% | 60% | 61% | 64% | 79% |
YZ | 94% | 94% | 66% | 83% | 37% | 54% | 55% | 57% | 56% | 59% |
XYZ | 69% | 81% | 76% | 68% | 45% | 52% | 63% | 84% | 64% | 58% |
Accuracy | 76% | 72% | 57% | 64% | 74% |
Table 2. Classification Results With Distance Parameter At a slight increase
in distance, the accuracy remains similar to the baseline, but further
increases in distances lead to a reduction in fingerprinting accuracy.
Notably, unlike the baseline, X-involved movement are better fingerprinted at
distance

###  4.3. Impact of Movement Speed

After looking at movement distance, the next parameter for robot movements is
the speed at which the robot is moving along each of the axes ($R_{2(i)}$). As
seen in Table 3, the speed parameter is less accurately fingerprinted by the
attack compared to the distance parameter by at least 10% on average.
Interestingly, a similar pattern is observed rgarding X-involved movements,
with accuracy increasing with speed, except from the XYZ movement. While there
are slight drops in accuracy, the precision and recall across most movements
remains similar as speed increases. This is interesting, as the initial
hypothesis was that a higher speed would result in higher pitched acoustic
emanations, however the results seem to contradict this. In any case, perhaps
the perceptual characteristics for human audio, while a clear pitch change is
present listening to the robot in the lab, the feature algorithms regarding
pitch (i.e. chroma feature) may not pick up on this for robot sounds.

S = Speed (mm/s), P = Precision, R = Recall |
---|---
| S=25 | S=50 | S=75 | S=100 |
P | R | P | R | P | R | P | R |
X | 57% | 81% | 54% | 74% | 78% | 53% | 72% | 81% |
Y | 79% | 76% | 61% | 42% | 59% | 45% | 72% | 69% |
Z | 50% | 56% | 52% | 58% | 62% | 84% | 77% | 75% |
XY | 73% | 72% | 46% | 40% | 67% | 57% | 57% | 70% |
XZ | 79% | 57% | 75% | 79% | 57% | 60% | 60% | 56% |
YZ | 67% | 59% | 66% | 45% | 53% | 65% | 66% | 69% |
XYZ | 65% | 63% | 51% | 66% | 54% | 57% | 62% | 47% |
Accuracy | 66% | 58% | 60% | 66% |
Table 3. Classification Results With Speed Parameter The speed parameter
performs worse than the distance parameter in the acoustic side channel, a
similar pattern as seen with the radio frequency side channel

###  4.4. Microphone Distance

While observing more fine-grained information leakage is useful to an
attacker, one problem that may impact the success of the attack is the
distance the recording device is away from the robot – in this case, the
smartphone. Naturally, due to the Doppler effect, the intensity of sound (i.e.
loudness) decreases over distances, and one would hypothesise that because of
this the accuracy may be significantly impacted as the distance of recording
increases. In this experiment, two other microphone distances (50cm and 100cm)
are also tested in addition to the baseline recorded at 30cm. While these are
not large recording distances, given the small scale of the robot used for the
evaluation of the attack, these are relatively suitable candidates to be
tested. As seen in Table 4, as the distance the microphone is away the robot
is increased, the accuracy of the attack compared to the baseline decreases by
around 10% at each recording distance step. Notably, this is much more
significant for Z-based movements which were previously described to have
poorer fingerprinting accuracy due to the limited range of motion that does
not cross the recording device (remains stationary and moves vertically). In
this case, a point a future work may be to evaluate the impact on position of
the smartphone around the robot, aside from facing in front. Collectively,
inference from multiple angles may provide better fingerprinting accuracy in
all cases.

MD = Microphone Distance (cm), P = Precision, R = Recall
---
| MD=30 | MD=50 | MD=100
| P | R | P | R | P | R
X | 76% | 81% | 57% | 79% | 75% | 76%
Y | 77% | 78% | 67% | 74% | 67% | 68%
Z | 61% | 71% | 48% | 66% | 45% | 63%
XY | 78% | 80% | 88% | 91% | 64% | 68%
XZ | 68% | 65% | 61% | 52% | 51% | 40%
YZ | 85% | 78% | 83% | 54% | 47% | 35%
XYZ | 72% | 67% | 52% | 39% | 33% | 33%
Accuracy | 75% | 65% | 54%
Table 4. Classification Results With Microphone Distance As the microphone
distance increases away from the robot being recorded, on average the accuracy
decreases around 10% at each step compared to the baseline - more
significantly for Z-based movements

###  4.5. Workflow Recovery

The next step in the evaluation looks at whether entire warehousing workflows
can be reconstructed through the acoustic side channel attack. While a pattern
matching approach can be successful using individual movement fingerprints,
the ability to reconstruct entire workflows may be useful from an auditing
perspective, for example, where offsets in normal movement signals can be
flagged and investigated further. As seen in Table 5, the explored warehousing
workflows can be recovered on average with around 62% accuracy. Notably, the
pick-and-place and packing workflows are recovered with much higher success
than the push and pull workflows. Simply, the former have much more variation
in the pattern of movements and thus the variance helps with fingerprinting.
In the case of push and pull movements, they are highly similar and it can be
hypothesised that only the direction of movement away from the microphone
(i.e. pull is a reverse of push) provides at least some degree of accuracy
between the two.

Workflow | Precision | Recall
---|---|---
Push | 37% | 16%
Pull | 31% | 59%
Pick-and-Place | 100% | 96%
Packing | 97% | 100%
Accuracy | 64%
Table 5. Workflow Reconstruction Results Common warehousing workflows can be
reconstructed in their entirety are better recovered through the acoustic side
channel if they are more complex and varied. Push and pull operations are less
accurate due to the fact they are very similar movements

###  4.6. Impact of VoIP

In certain robotics environments, such as in surgical settings, procedures may
be streamed and/or recorded for viewing, education or research (Muensterer et
al., 2014; Kulkarni et al., 2020; Hosseini et al., 2013). Therefore, it is
important to question how VoIP impacts the audio samples for movements and
workflows and, ultimately, the success of the attack. In many modern VoIP
applications, the Opus codec is the preferred choice (Valin et al., 2012,
2016) given its standardisation and rank of higher quality compared to other
audio formats for a variety of bitrates. To explore this,the open-source
nature of Opus allows for easy implementation to encode and decode the audio
samples and, during decoding, investigate various packet losses. In VoIP
applications, Packet Loss Concealment (PLC) is used as a decoder feature for
receiving data from an unreliable source, which masks the effects of packet
loss in VoIP communications. In realistic settings, packets may arrive late,
be dropped or be corrupted, which may result in not only a lowered audio
quality but in the worst case, dropped parts of the audio or the entire audio
sample entirely. Given that in VoIP applications, a 1% packet loss is
considered an acceptable rate for VoIP to minimise impact on call quality
(James et al., 2004; Amirzade Dana et al., 2020), however in the event of
network failures or availability attacks this may be higher. For completeness,
5 packet losses of 1%, 5%, 10%, 25% and 50% are evaluated. Furthermore, as it
was shown that constant bitrate quality does not perform as well as variable
bitrate quality (Rämö and Toukomaa, 2011), samples are encoded and decoded
with variable bitrate. This experiment used the same model as the previous
experiments, but with a batch size of 256 and 100 epochs of training. As seen
in Table 6, the results for the baseline speed and distance of movement
(12.5mm/s and 1mm respectively) under various packet losses via the Opus codec
can be seen. Interestingly, at low packet loss, the classification accuracy is
around 90% and increases by around 15% compared to the baseline without VoIP
employed. Further, X movements are more accurately fingerprinted across all
packet losses compared to the baseline without VoIP. As the packet loss
reaches more undesirable amounts of 25% and 50%, the accuracy slightly
decreases but the accuracy still remains much higher than the baseline without
VoIP. This may be due to the PLC algorithm switching between CELT or SILK mode
and variable bit rate. Specifically, frames that are deemed important are re-
encoded at a lower bitrate and allows for partial recovery for improtant lost
packets. This may be targetting the movement audio within the sample thus
leading to higher variance among classes.

L = Loss (%), P = Precision, R = Recall |
---|---
| L=1 | L=5 | L=10 | L=25 | L=50 |
P | R | P | R | P | R | P | R | P | R |
X | 99% | 99% | 99% | 100% | 100% | 100% | 100% | 100% | 99% | 100% |
Y | 90% | 94% | 87% | 96% | 90% | 92% | 82% | 97% | 86% | 97% |
Z | 86% | 72% | 88% | 68% | 91% | 73% | 86% | 72% | 90% | 74% |
XY | 88% | 91% | 91% | 88% | 88% | 91% | 94% | 81% | 90% | 81% |
XZ | 89% | 93% | 82% | 83% | 82% | 82% | 80% | 85% | 80% | 85% |
YZ | 86% | 89% | 87% | 88% | 85% | 89% | 91% | 80% | 91% | 85% |
XYZ | 93% | 98% | 94% | 97% | 92% | 96% | 89% | 97% | 90% | 97% |
Accuracy | 90% | 90% | 90% | 88% | 89% |
Table 6. Classification Results (Baseline) With Opus Codec and Packet Loss
Interestingly, the precision and recall remains relatively similar across
packet losses, with a slightly drop in accuracy for undesirable large packet
losses. Notably, there is an increase in accuracy of around 15% compared to
the baseline without the Opus codec employed

##  5\. Discussion

The acoustic side channel attack we propose showcases the potential for
successfully compromising the operational confidentiality of organisations in
which robotic systems under attack are deployed. While many active attacks
have shown to result in potentially devastating consequences, the capabilities
of a passive insider attacker are truly underestimated. In this section, a
discussion on the proposed attack is provided.

###  5.1. Influence of Noise

During the recording of acoustic samples for robot movements, there is likely
some degree of background noise that should be accounted for. Given the
recordings were made in a computer lab, background noise effects may include
the likes of light chatter, keyboard tapping and rolling chairs, among others.
While relatively good accuracy is observed even with the background noise, it
is important to also look into techniques to eliminate such noise to determine
whether this results in better fingerprinting accuracy.

In human audio, sound recordings contain the relative signal of the
oscillations due to density and pressure of air in the ear. In digital audio,
sound waves are encoded in digital form as numerical samples in a continuous
sequence. The recordings taken in this attack are recorded at a sampling rate
of 44.1KHz with 16-bit depth and thus there are $65,536$ possible values the
signal can take in the sequence.

Figure 4. FFT of Acoustic Signal Peaks can be observed at 60Hz corresponding
to electric hum, with other peaks at 150Hz and 200Hz (among others) which may
correlate with robot movement Figure 5. FFT of Acoustic Signal (Filtered) The
amplitude at points correlating with electric hum or those outwidth the human
hearing range are set to 0 (filtered out)

As shown in Figure 4, the amplitude of the frequency content of the acoustic
signal can be observed using the Fast Fourier Transform (FFT). In this attack,
we make use of techniques originally applied to human acoustics, but given
that the robot movements produce sound that is audible to the human ear as
well. Looking at the frequency content, notable amplitude is not found past
1KHz, so this is zoomed in further to 250Hz. There is a notable spike around
60Hz, which is the frequency standard common to alternating current and is an
effect known as electric hum due to electrical noise getting into an acoustic
recording medium. The next largest peaks can be observed at around 150Hz and
200Hz which may correspond with the robot movements. As a first step to noise
reduction/filtering, one technique is amplitude filtering, where the
amplitudes of FFT values to be filtered can be set to 0Hz, to which the
original signal can be recreated using an inverse FFT. In this experiment, the
electric hum, as well as frequencies outwidth the human hearing range of
~20Hz–20KHz are filtered by dropping the amplitude of these ranges. A
depiction of the amplitude drop can be seen in Figure 5. Looking at Table 7,
the accuracy of baseline movement fingerprints can be observed with amplitude
filtering in place. While the accuracy overall decreases by 1% compared to the
baseline without amplitude filtering, the precision for Y and XY movements
increase. This may be due to unfortunate noise events present in these samples
that the filter has rectified. However, there is still a reduction in overall
accuracy, which may mean that electric hum and other peaks may not be the best
indicators of noise to remove when recording a robotics system. In this case,
as a point of future work other noise reduction techniques that have shown to
be successful in other areas, such as stationary or non-stationary spectral
gating (Neumann and Schuller, 1991; Inouye et al., 2014) which reduces noise
in time-domain signals by estimating noise thresholds for the frequency bands
in a signal to gate (mask) noise below the threshold, are worth exploring in
the hope the attack accuracy may increase.

Movement | Precision | Recall
---|---|---
X | 72% | 81%
Y | 75% | 73%
Z | 68% | 72%
XY | 86% | 71%
XZ | 69% | 68%
YZ | 76% | 83%
XYZ | 72% | 70%
Accuracy | 74%
Table 7. Amplitude Filtering Classification Results While the accuracy is
slightly reduced compared to the baseline with no filtering, the precision for
some movements increases further, with better recall seen in most cases

###  5.2. Other VoIP Codecs

Opus is the primary choice for many VoIP applications due to its royalty free
and open source nature, alongside the benefits of higher quality and low-
bandwidth streaming, in comparison with other codecs such as Speex (Valin,
2016) or SILK (Vos et al., 2010) (Opus’ predecessor). While it may be
interesting to evaluate other codecs, Opus is the main choice for the majority
of modern applications, such as Zoom, Teams and Discord (Rajaratnam et al.,
2018; Castro, 2020) and is taking over previously dominating codecs.

###  5.3. Defences

While the attack is successful, and even more so when the attack targets VoIP
communiations, a natural question pertains to countermeasures and defences
against the acoustic side channel attack. In this work, acoustic emanations
result in unintentional information leakages about robot behaviours and can
ultimately lead to the compromise of operational confidentiality.

One defence that could be considered is to make use of vibration- or sound-
reduction mechanisms to hinder the effect of the attack. As seen in Section
4.4, as the microphone distance increases the accuracy of fingerprinting also
decreases. While this is due to the Doppler effect that is naturally at play
with regard to sound intensity (i.e. loudness), a reduction in this from other
means may result in the same outcome of reduced success of fingerprinting.
Techniques in this space include the likes of using vibration isolation pads
(Desai and Patil, [n.d.]) or damping to reduce vibration (Gravagne et al.,
2001; Khan and Li, 2020) for the robot as a whole. In the case of noise
reduction for robot components such as stepper motors, potential defences
include using a clean damper (Ma et al., 2019) or higher resolution stepper
motors.

Another potential defence is to make use of a masking noise, to interfere with
attack inference by distorting the signal related to information leakage in
the acoustic side channel (Backes et al., 2010; Anand and Saxena, 2016; Kim et
al., 2015). Adding a masking signal has shown success, but two challenges need
to be addressed. First, the mask must be similar to the signal requiring
masking to ensure difficult separation. Second, the masking noise should not
cause any degrading effect on usability of the robotic system. For example, if
the masking noise is to cover up other sound such as those used for
emergencies or other operator feedback, then this will be much less than ideal
and potentially lead to catastrophic liabilities.

##  6\. Related Work

While acoustic side channel attacks have not been explored for robotic
systems, enhancing the novelty of this work, there has been previous research
in the area of acoustic side channels. In a similar respect to robotics, the
exploration of information leakage in the acoustic side channel has been
explored for 3D printers (Backes et al., 2010) – some of which making use of
smartphones to carry out the attack (Song et al., 2016; Bilal, 2017) – and
additive manufacturing systems (Chhetri et al., 2017). However, many of these
attacks solely focus on IP theft. The acoustic side channel attack presented
in this work focus solely on the movement of the robot arm and the compromise
of operational confidentiality, which when looking at the bigger picture is
much more valuable to an attacker. Furthermore, the reconstruction of G-code
is an unnecessary extra step as movements which correspond to these can be
inferred from individual movement fingerprinting under the assumption the
robot is operated by an Arduino. Furthermore, while the robot in this work is
operated by an Arduino, the focus is on reconstructing movements from the
acoustic emanations, irrespective of the microcontroller used and thus applies
to robotic systems in general and not those restricted to being operated by an
Arduino.

##  7\. Conclusion

In conclusion, it is clear that even acoustic emanations provide a high level
of accuracy for fingerprinting movements and showcases a highly important
passive side channel attack in the physical domain, which can be carried out
with a fairly cheap smartphone. While more fine-grained movements and entire
workflows in warehousing settings can be inferred, our contributions
demonstrate that the recent usage of VoIP technologies also leave potential
for information leakage through these communication channels, with the result
leaving movement fingeprints to be more accurately reconstructed. This is an
interesting result as it opens up new research questions regarding anonymous
communications to protect robotic systems from acoustic side channel attacks
via VoIP communication networks.

###### Acknowledgements.

The authors are grateful for the support by the Engineering and Physical
Sciences Research Council (11288S170484-102) and the support of the National
Measurement System of the UK Department of Business, Energy & Industrial
Strategy, which funded this work as part of NPL’s Data Science program.

## References

  * (1)
  * Agarap (2018) Abien Fred Agarap. 2018\.  Deep learning using rectified linear units (relu).  _arXiv preprint arXiv:1803.08375_ (2018).
  * Ahn et al. (2015) Ho Seok Ahn, Min Ho Lee, and Bruce A MacDonald. 2015.  Healthcare robot systems for a hospital environment: CareBot and ReceptionBot. In _2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)_. IEEE, 571–576.
  * Al-Jabir et al. (2020) A. Al-Jabir, A. Kerwan, M. Nicola, Z. Alsafi, M. Khan, C. Sohrabi, N. O’Neill, C. Iosifidis, M. Griffin, G. Mathew, and R. Agha. 2020\.  Impact of the Coronavirus (COVID-19) pandemic on surgical practice - Part 1.  _Int J Surg_ 79 (Jul 2020), 168–179.
  * Amirzade Dana et al. (2020) Parvaneh Amirzade Dana, Zahra Esmaeilbeig, and Mohammad-Reza Sadeghi. 2020.  Reliability enhancement and packet loss recovery of any steganographic method in voice over IP.  _Wireless Networks_ 26, 8 (2020), 5817–5823.
  * Anand and Saxena (2016) S Abhishek Anand and Nitesh Saxena. 2016.  A sound for a sound: Mitigating acoustic side channel attacks on password keystrokes with active sounds. In _International Conference on Financial Cryptography and Data Security_. Springer, 346–364.
  * Aschenbrenner et al. (2015) Doris Aschenbrenner, Michael Fritscher, Felix Sittner, Markus Krauß, and Klaus Schilling. 2015\.  Teleoperation of an industrial robot in an active production line.  _IFAC-PapersOnLine_ 48, 10 (2015), 159–164.
  * Atahan et al. (2021) Yunus Atahan, Ahmet Elbir, Abdullah Enes Keskin, Osman Kiraz, Bulent Kirval, and Nizamettin Aydin. 2021.  Music Genre Classification Using Acoustic Features and Autoencoders. In _2021 Innovations in Intelligent Systems and Applications Conference (ASYU)_. IEEE, 1–5.
  * Avila et al. (2020) Jose Luis Ordoñez Avila, Hector Jimenez, Tania Marquez, Carlos Muñoz, Alberto Max Carrazco, Maria Elena Perdomo, David Miselem, and David Nolasco. 2020.  Study Case: Teleoperated Voice Picking Robots prototype as a logistic solution in Honduras. In _2020 5th International Conference on Control and Robotics Engineering (ICCRE)_. IEEE, 19–24.
  * Bachu et al. (2008) RG Bachu, S Kopparthi, B Adapa, and BD Barkana. 2008\.  Separation of voiced and unvoiced using zero crossing rate and energy of the speech signal. In _American Society for Engineering Education (ASEE) zone conference proceedings_. American Society for Engineering Education, 1–7.
  * Backes et al. (2010) Michael Backes, Markus Dürmuth, Sebastian Gerling, Manfred Pinkal, Caroline Sporleder, et al. 2010\.  Acoustic $\\{$Side-Channel$\\}$ Attacks on Printers. In _19th USENIX Security Symposium (USENIX Security 10)_.
  * Bartoš et al. (2021) Michal Bartoš, Vladimír Bulej, Martin Bohušík, Ján Stanček, Vitalii Ivanov, and Peter Macek. 2021\.  An overview of robot applications in automotive industry.  _Transportation Research Procedia_ 55 (2021), 837–844.
  * Bilal (2017) Muhammad Bilal. 2017\.  A review of internet of things architecture, technologies and analysis smartphone-based attacks against 3D printers.  _arXiv preprint arXiv:1708.04560_ (2017).
  * Bonaci et al. (2015) Tamara Bonaci, Jeffrey Herron, Tariq Yusuf, Junjie Yan, Tadayoshi Kohno, and Howard Jay Chizeck. 2015.  To make a robot secure: An experimental analysis of cyber security threats against teleoperated surgical robots.  _arXiv preprint arXiv:1504.04339_ (2015).
  * Castro (2020) Rodolfo Castro. 2020\.  Is your company’s network ready for Microsoft teams.
  * Chhetri et al. (2017) Sujit Rokka Chhetri, Arquimedes Canedo, and Mohammad Abdullah Al Faruque. 2017.  Confidentiality breach through acoustic side-channel in cyber-physical additive manufacturing systems.  _ACM Transactions on Cyber-Physical Systems_ 2, 1 (2017), 1–25.
  * Cho and Bello (2013) Taemin Cho and Juan P Bello. 2013.  On the relative importance of individual components of chord recognition systems.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ 22, 2 (2013), 477–492.
  * Chollet et al. (2015) François Chollet et al. 2015\.  Keras.  https://keras.io.
  * Dalen et al. (2019) ASHM Dalen, J Legemaate, WS Schlack, DA Legemate, and MP Schijven. 2019.  Legal perspectives on black box recording devices in the operating environment.  _Journal of British Surgery_ 106, 11 (2019), 1433–1441.
  * Demuth et al. (2014) Howard B Demuth, Mark H Beale, Orlando De Jess, and Martin T Hagan. 2014.  _Neural network design_.  Martin Hagan.
  * Desai and Patil ([n.d.]) Tanvi D Desai and SR Patil. [n.d.].  Experimental and Numerical Analysis of Vibration Isolation Materials on Vibration Reduction within Plazma Torch.  ([n. d.]).
  * Doppler (1903) Christian Doppler. 1903\.  _Ueber das farbige Licht der Doppelsterne und einiger anderer Gestirne des Himmels: Versuch einer das Bradley’sche Aberrations-Theorem als integrirenden Theil in sich schliessenden allgemeineren Theorie_.  K. Böhm Gesellschaft der Wissenschaften.
  * Dunne and Campbell (1997) Rob A Dunne and Norm A Campbell. 1997.  On the pairing of the softmax activation and cross-entropy penalty functions and the derivation of the softmax activation function. In _Proc. 8th Aust. Conf. on the Neural Networks, Melbourne_ , Vol. 181. Citeseer, 185\.
  * Eckle and Schmidt-Hieber (2019) Konstantin Eckle and Johannes Schmidt-Hieber. 2019.  A comparison of deep networks with ReLU activation function and linear spline-type methods.  _Neural Networks_ 110 (2019), 232–242.
  * Grabowski et al. (2021) Andrzej Grabowski, Jarosław Jankowski, and Mieszko Wodzyński. 2021.  Teleoperated mobile robot with two arms: the influence of a human-machine interface, VR training and operator age.  _International Journal of Human-Computer Studies_ 156 (2021), 102707\.
  * Gravagne et al. (2001) Ian A Gravagne, Christopher D Rahn, and Ian D Walker. 2001\.  Good vibrations: a vibration damping setpoint controller for continuum robots. In _Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164)_ , Vol. 4. IEEE, 3877–3884.
  * Greenwood (1997) Donald D Greenwood. 1997\.  The Mel Scale’s disqualifying bias and a consistency of pitch-difference equisections in 1956 with equal cochlear distances and equal frequency ratios.  _Hearing research_ 103, 1-2 (1997), 199–224.
  * Hannaford et al. (2012) Blake Hannaford, Jacob Rosen, Diana W Friedman, Hawkeye King, Phillip Roan, Lei Cheng, Daniel Glozman, Ji Ma, Sina Nia Kosari, and Lee White. 2012\.  Raven-II: an open platform for surgical robotics research.  _IEEE Transactions on Biomedical Engineering_ 60, 4 (2012), 954–959.
  * Hosseini et al. (2013) Azamossadat Hosseini, Hamid Moghaddasi, Samad Sajadi, and Mozhgan Karimi. 2013.  Telesurgery information management systems in university hospitals of Tehran.  _Archives of Advances in Biosciences_ 4, 4 (2013).
  * Inouye et al. (2014) Joshua M Inouye, Silvia S Blemker, and David I Inouye. 2014\.  Towards undistorted and noise-free speech in an MRI scanner: correlation subtraction followed by spectral noise gating.  _The Journal of the Acoustical Society of America_ 135, 3 (2014), 1019–1022.
  * James et al. (2004) Jim H James, Bing Chen, and Laurie Garrison. 2004.  Implementing VoIP: a voice transmission performance progress report.  _IEEE Communications Magazine_ 42, 7 (2004), 36–41.
  * Jiang et al. (2002) Dan-Ning Jiang, Lie Lu, Hong-Jiang Zhang, Jian-Hua Tao, and Lian-Hong Cai. 2002.  Music type classification by spectral contrast feature. In _Proceedings. IEEE International Conference on Multimedia and Expo_ , Vol. 1. IEEE, 113–116.
  * Khan and Li (2020) Ameer Hamza Khan and Shuai Li. 2020.  Sliding mode control with PID sliding surface for active vibration damping of pneumatically actuated soft robots.  _IEEE Access_ 8 (2020), 88793–88800.
  * Kim et al. (2015) Younghyun Kim, Woo Suk Lee, Vijay Raghunathan, Niraj K Jha, and Anand Raghunathan. 2015.  Vibration-based secure side channel for medical devices. In _2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC)_. IEEE, 1–6.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Klapuri and Davy (2007) Anssi Klapuri and Manuel Davy. 2007.  Signal processing methods for music transcription.  (2007).
  * Kos et al. (2013) Marko Kos, Zdravko Kačič, and Damjan Vlaj. 2013\.  Acoustic classification and segmentation using modified spectral roll-off and variance-based features.  _Digital Signal Processing_ 23, 2 (2013), 659–674.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012\.  Imagenet classification with deep convolutional neural networks. In _Advances in neural information processing systems_. 1097–1105.
  * Kulkarni et al. (2020) Sushmita Kulkarni, Dattaprasad A Torse, and Deepak Kulkarni. 2020.  A Cloud based Medical Transcription using Speech Recognition Technologies.  _International Research Journal of Engineering and Technology (IRJET)_ 7, 5 (2020), 6160–6163.
  * Laghari et al. (2020) Asif Ali Laghari, Rashid Ali Laghari, Asif Ali Wagan, and Aamir Iqbal Umrani. 2020.  Effect of packet loss and reorder on quality of audio streaming.  _EAI Endorsed Transactions on Scalable Information Systems_ 7, 25 (2020).
  * Le et al. (2011) Phu Ngoc Le, Eliathamby Ambikairajah, Julien Epps, Vidhyasaharan Sethu, and Eric HC Choi. 2011\.  Investigation of spectral centroid features for cognitive load classification.  _Speech Communication_ 53, 4 (2011), 540–551.
  * Li et al. (2017) Chunxu Li, Chenguang Yang, Jian Wan, Andy SK Annamalai, and Angelo Cangelosi. 2017.  Teleoperation control of Baxter robot using Kalman filter-based sensor fusion.  _Systems Science & Control Engineering_ 5, 1 (2017), 156–167.
  * Li and Yuan (2017) Yuanzhi Li and Yang Yuan. 2017.  Convergence analysis of two-layer neural networks with relu activation. In _Advances in neural information processing systems_. 597–607.
  * Ma et al. (2019) Xinbo Ma, Pak Kin Wong, and Jing Zhao. 2019.  Practical multi-objective control for automotive semi-active suspension system with nonlinear hydraulic adjustable damper.  _Mechanical Systems and Signal Processing_ 117 (2019), 667–688.
  * Martinez et al. (2012) Jorge Martinez, Hector Perez, Enrique Escamilla, and Masahisa Mabo Suzuki. 2012.  Speaker recognition using Mel frequency Cepstral Coefficients (MFCC) and Vector quantization (VQ) techniques. In _CONIELECOMP 2012, 22nd International Conference on Electrical Communications and Computers_. IEEE, 248–251.
  * McClean et al. (2013) Jarrod McClean, Christopher Stull, Charles Farrar, and David Mascarenas. 2013.  A preliminary cyber-physical security assessment of the robot operating system (ros). In _Unmanned Systems Technology XV_ , Vol. 8741. International Society for Optics and Photonics, 874110.
  * McFee et al. (2015) Brian McFee, Colin Raffel, Dawen Liang, Daniel P Ellis, Matt McVicar, Eric Battenberg, and Oriol Nieto. 2015.  librosa: Audio and music signal analysis in python. In _Proceedings of the 14th python in science conference_ , Vol. 8. Citeseer, 18–25.
  * Muensterer et al. (2014) Oliver J Muensterer, Martin Lacher, Christoph Zoeller, Matthew Bronstein, and Joachim Kübler. 2014.  Google Glass in pediatric surgery: an exploratory study.  _International journal of surgery_ 12, 4 (2014), 281–289.
  * Müller (2015) Meinard Müller. 2015\.  Fundamentals of Music Processing.  (2015).
  * Neumann and Schuller (1991) Ingrid Neumann and Gerd Schuller. 1991.  Spectral and temporal gating mechanisms enhance the clutter rejection in the echolocating bat, Rhinolophus rouxi.  _Journal of comparative physiology A_ 169, 1 (1991), 109–116.
  * Ortega et al. (2018) Martín Ortega Ortega, Gustavo Chafla Altamirano, and Mara Falconí Abad. 2018.  Evaluation of the voice quality and QoS in real calls using different voice over IP codecs. In _2018 IEEE Colombian Conference on Communications and Computing (COLCOM)_. IEEE, 1–6.
  * Panagiotakis and Tziritas (2005) Costas Panagiotakis and Georgios Tziritas. 2005.  A speech/music discriminator based on RMS and zero-crossings.  _IEEE Transactions on multimedia_ 7, 1 (2005), 155–166.
  * Paulus et al. (2010) Jouni Paulus, Meinard Müller, and Anssi Klapuri. 2010\.  State of the Art Report: Audio-Based Music Structure Analysis.. In _Ismir_. Utrecht, 625–636.
  * Peng and Choi (2013) Yinni Peng and Susanne YP Choi. 2013.  Mobile phone use among migrant factory workers in south China: Technologies of power and resistance.  _The China Quarterly_ 215 (2013), 553–571.
  * Polydoros and Nalpantidis (2016) Athanasios S Polydoros and Lazaros Nalpantidis. 2016.  A reservoir computing approach for learning forward dynamics of industrial manipulators. In _2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 612–618.
  * Quarta et al. (2017) Davide Quarta, Marcello Pogliani, Mario Polino, Federico Maggi, Andrea Maria Zanchettin, and Stefano Zanero. 2017.  An experimental security analysis of an industrial robot controller. In _2017 IEEE Symposium on Security and Privacy (SP)_. IEEE, 268–286.
  * Rajaratnam et al. (2018) Krishan Rajaratnam, Kunal Shah, and Jugal Kalita. 2018\.  Isolated and ensemble audio preprocessing methods for detecting adversarial examples against automatic speech recognition.  _arXiv preprint arXiv:1809.04397_ (2018).
  * Rämö and Toukomaa (2011) Anssi Rämö and Henri Toukomaa. 2011.  Voice quality characterization of IETF Opus codec. In _Twelfth Annual Conference of the International Speech Communication Association_.
  * Rao and Vuppala (2014) K Sreenivasa Rao and Anil Kumar Vuppala. 2014.  _Speech processing in mobile environments_.  Springer.
  * Reuters (2019) Reuters. 2019.  U.S. companies put record number of robots to work in 2018\.  https://www.reuters.com/article/us-usa-economy-robots/u-s-companies-put-record-number-of-robots-to-work-in-2018-idUSKCN1QH0K0.
  * Rueckert et al. (2017) Elmar Rueckert, Moritz Nakatenus, Samuele Tosatto, and Jan Peters. 2017.  Learning inverse dynamics models in o (n) time with lstm networks. In _2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)_. IEEE, 811–816.
  * Saun et al. (2019) Tomas J Saun, Kevin J Zuo, and Teodor P Grantcharov. 2019\.  Video technologies for recording open surgery: a systematic review.  _Surgical innovation_ 26, 5 (2019), 599–612.
  * Shah et al. (2022) Ryan Shah, Chuadhry Mujeeb Ahmed, and Shishir Nagaraja. 2022\.  Can You Still See Me?: Reconstructing Robot Operations Over End-to-End Encrypted Channels.  _arXiv preprint arXiv:2205.08426_ (2022).
  * Song et al. (2016) Chen Song, Feng Lin, Zhongjie Ba, Kui Ren, Chi Zhou, and Wenyao Xu. 2016\.  My smartphone knows what you print: Exploring smartphone-based side-channel attacks against 3d printers. In _Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security_. 895–907.
  * Sung and Gill (2001) Gyung Tak Sung and Inderbir S Gill. 2001.  Robotic laparoscopic surgery: a comparison of the da Vinci and Zeus systems.  _Urology_ 58, 6 (2001), 893–898.
  * Tewari et al. (2002) Ashutosh Tewari, James Peabody, Richard Sarle, Guruswami Balakrishnan, Ashok Hemal, Alok Shrivastava, and Mani Menon. 2002\.  Technique of da Vinci robot-assisted anatomic radical prostatectomy.  _Urology_ 60, 4 (2002), 569–572.
  * Valin (2016) Jean-Marc Valin. 2016\.  Speex: A free codec for free speech.  _arXiv preprint arXiv:1602.08668_ (2016).
  * Valin et al. (2016) Jean-Marc Valin, Gregory Maxwell, Timothy B Terriberry, and Koen Vos. 2016.  High-quality, low-delay music coding in the opus codec.  _arXiv preprint arXiv:1602.04845_ (2016).
  * Valin et al. (2012) Jean-Marc Valin, Koen Vos, and Timothy Terriberry. 2012\.  _Definition of the Opus audio codec_.  Technical Report.
  * Vodafone (2019) Vodafone. 2019.  5GDig: The winners – from Skype for surgeons to AR.  https://www.vodafone.com/news/technology/5gdig-winners-2019-supponor-ar-surgeonmate.
  * Vos et al. (2010) Koen Vos, Soeren Jensen, and Karsten Soerensen. 2010.  SILK speech codec.  _IETF draft_ 30 (2010).
  * Zhang and Sabuncu (2018) Zhilu Zhang and Mert Sabuncu. 2018.  Generalized cross entropy loss for training deep neural networks with noisy labels. In _Advances in neural information processing systems_. 8778–8788.